2024-04-27T03:51:06.650 INFO:root:teuthology version: 0.0.1.dev269+g9481b1d 2024-04-27T03:51:06.650 DEBUG:teuthology.run:Teuthology command: teuthology --name teuthology-2024-04-27_03:08:09-upgrade-main-distro-default-smithi --description upgrade/cephfs/mds_upgrade_sequence/{bluestore-bitmap centos_9.stream conf/{client mds mgr mon osd} fail_fs/yes overrides/{ignorelist_health ignorelist_upgrade ignorelist_wrongly_marked_down pg-warn pg_health syntax} roles tasks/{0-from/quincy 1-volume/{0-create 1-ranks/1 2-allow_standby_replay/yes 3-inline/no 4-verify} 2-client/fuse 3-upgrade-mgr-staggered 4-config-upgrade/{fail_fs} 5-upgrade-with-workload 6-verify}} --archive /home/teuthworker/archive/teuthology-2024-04-27_03:08:09-upgrade-main-distro-default-smithi/7675783 --verbose --owner scheduled_teuthology@teuthology -- /home/teuthworker/archive/teuthology-2024-04-27_03:08:09-upgrade-main-distro-default-smithi/7675783/orig.config.yaml 2024-04-27T03:51:06.706 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-04-27T03:51:06.806 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/teuthology-2024-04-27_03:08:09-upgrade-main-distro-default-smithi/7675783 branch: main description: upgrade/cephfs/mds_upgrade_sequence/{bluestore-bitmap centos_9.stream conf/{client mds mgr mon osd} fail_fs/yes overrides/{ignorelist_health ignorelist_upgrade ignorelist_wrongly_marked_down pg-warn pg_health syntax} roles tasks/{0-from/quincy 1-volume/{0-create 1-ranks/1 2-allow_standby_replay/yes 3-inline/no 4-verify} 2-client/fuse 3-upgrade-mgr-staggered 4-config-upgrade/{fail_fs} 5-upgrade-with-workload 6-verify}} email: ceph-qa@ceph.com first_in_suite: false job_id: '7675783' kernel: kdb: 1 sha1: distro last_in_suite: false machine_type: smithi meta: - desc: 'setup ceph/quincy ' name: teuthology-2024-04-27_03:08:09-upgrade-main-distro-default-smithi no_nested_subset: false nuke-on-error: true os_type: centos os_version: 9.stream overrides: admin_socket: branch: main ceph: conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 rados mon op timeout: 900 rados osd op timeout: 900 global: mon pg warn min per osd: 0 mds: debug mds: 20 debug mds balancer: 20 debug ms: 1 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: debug client: 20 debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon down mkfs grace: 300 mon op complaint time: 120 osd: bdev async discard: true bdev enable discard: true bluestore allocator: bitmap bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug ms: 1 debug osd: 20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore osd op complaint time: 180 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - FS_DEGRADED - FS_INLINE_DATA_DEPRECATED - FS_WITH_FAILED_MDS - MDS_ALL_DOWN - MDS_DAMAGE - MDS_DEGRADED - MDS_FAILED - MDS_INSUFFICIENT_STANDBY - MDS_UP_LESS_THAN_MAX - filesystem is online with fewer MDS than max_mds - POOL_APP_NOT_ENABLED - do not have an application enabled - overall HEALTH_ - Replacing daemon - deprecated feature inline_data - OSD_DOWN - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding - PG_AVAILABILITY - PG_DEGRADED - Reduced data availability sha1: 4e213a58e787437ff508e6c890ccd5fd32dc96b5 ceph-deploy: bluestore: true conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} osd: bdev async discard: true bdev enable discard: true bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore fs: xfs install: ceph: flavor: default sha1: 4e213a58e787437ff508e6c890ccd5fd32dc96b5 kclient: syntax: v1 selinux: allowlist: - scontext=system_u:system_r:logrotate_t:s0 - scontext=system_u:system_r:getty_t:s0 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: main sha1: 3067e9f133f954b642fdf21114a79edb58590948 owner: scheduled_teuthology@teuthology priority: 850 repo: https://git.ceph.com/ceph.git roles: - - host.a - client.0 - osd.0 - osd.1 - osd.2 - - host.b - client.1 - osd.3 - osd.4 - osd.5 seed: 1487 sha1: 4e213a58e787437ff508e6c890ccd5fd32dc96b5 sleep_before_teardown: 0 subset: 16/32 suite: upgrade suite_branch: main suite_path: /home/teuthworker/src/git.ceph.com_ceph_3067e9f133f954b642fdf21114a79edb58590948/qa suite_relpath: qa suite_repo: https://git.ceph.com/ceph.git suite_sha1: 3067e9f133f954b642fdf21114a79edb58590948 targets: smithi162.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBDf6sVRRTSiqvkX+fbZWNAcRYHJslXRzTLFccLgVXzFazc5Jfx2Emd5XuO+nwOYakkoHP3JufzEDm151i1Aoi9o= smithi181.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBEtNFoOquI7mNXOJLHLdcnyh7cIgoneEfOhj0xbTqRro+WEYsvFITBxGXKQDyGVPFTSl9rJiTR8wedfsTsRQr0w= tasks: - install: branch: quincy exclude_packages: - ceph-volume - print: '**** done install task...' - cephadm: cephadm_branch: quincy cephadm_git_url: https://github.com/ceph/ceph conf: osd: osd_class_default_list: '*' osd_class_load_list: '*' image: quay.ceph.io/ceph-ci/ceph:quincy roleless: true - print: '**** done end installing quincy cephadm ...' - cephadm.shell: host.a: - ceph config set mgr mgr/cephadm/use_repo_digest true --force - print: '**** done cephadm.shell ceph config set mgr...' - cephadm.shell: host.a: - ceph orch status - ceph orch ps - ceph orch ls - ceph orch host ls - ceph orch device ls - cephadm.shell: host.a: - ceph fs volume create cephfs --placement=4 - ceph fs dump - cephadm.shell: host.a: - ceph fs set cephfs max_mds 1 - cephadm.shell: host.a: - ceph fs set cephfs allow_standby_replay true - cephadm.shell: host.a: - ceph fs set cephfs inline_data false - cephadm.shell: host.a: - ceph fs dump - ceph --format=json fs dump | jq -e ".filesystems | length == 1" - while ! ceph --format=json mds versions | jq -e ". | add == 4"; do sleep 1; done - fs.pre_upgrade_save: null - ceph-fuse: null - print: '**** done client' - parallel: - upgrade-tasks - workload-tasks - cephadm.shell: host.a: - ceph fs dump - fs.post_upgrade_checks: null teuthology: fragments_dropped: [] meta: {} postmerge: [] variables: fail_fs: true teuthology_branch: main teuthology_sha1: 9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17 timestamp: 2024-04-27_03:08:09 tube: smithi upgrade-tasks: sequential: - cephadm.shell: env: - sha1 host.a: - ceph config set mon mon_warn_on_insecure_global_id_reclaim false --force - ceph config set mon mon_warn_on_insecure_global_id_reclaim_allowed false --force - ceph config set global log_to_journald false --force - ceph orch upgrade start --image quay.ceph.io/ceph-ci/ceph:$sha1 --daemon-types mgr - while ceph orch upgrade status | jq '.in_progress' | grep true && ! ceph orch upgrade status | jq '.message' | grep Error ; do ceph orch ps ; ceph versions ; ceph orch upgrade status ; sleep 30 ; done - ceph versions | jq -e '.mgr | length == 1' - ceph versions | jq -e '.mgr | keys' | grep $sha1 - ceph versions | jq -e '.overall | length == 2' - ceph orch upgrade check quay.ceph.io/ceph-ci/ceph:$sha1 | jq -e '.up_to_date | length == 2' - ceph orch ps - cephadm.shell: env: - sha1 host.a: - ceph config set mgr mgr/orchestrator/fail_fs true - cephadm.shell: env: - sha1 host.a: - ceph config set mon mon_warn_on_insecure_global_id_reclaim false --force - ceph config set mon mon_warn_on_insecure_global_id_reclaim_allowed false --force - ceph config set global log_to_journald false --force - ceph orch upgrade start --image quay.ceph.io/ceph-ci/ceph:$sha1 - cephadm.shell: env: - sha1 host.a: - while ceph orch upgrade status | jq '.in_progress' | grep true && ! ceph orch upgrade status | jq '.message' | grep Error ; do ceph orch ps ; ceph versions ; ceph fs dump; ceph orch upgrade status ; ceph health detail ; sleep 30 ; done - ceph orch ps - ceph orch upgrade status - ceph health detail - ceph versions - echo "wait for servicemap items w/ changing names to refresh" - sleep 60 - ceph orch ps - ceph versions - ceph versions | jq -e '.overall | length == 1' - ceph versions | jq -e '.overall | keys' | grep $sha1 user: teuthology verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.2226885 workload-tasks: sequential: - workunit: clients: all: - suites/fsstress.sh 2024-04-27T03:51:06.806 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph_3067e9f133f954b642fdf21114a79edb58590948/qa; will attempt to use it 2024-04-27T03:51:06.807 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph_3067e9f133f954b642fdf21114a79edb58590948/qa/tasks 2024-04-27T03:51:06.807 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-04-27T03:51:06.808 INFO:teuthology.task.internal:Checking packages... 2024-04-27T03:51:06.833 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'default' and ceph hash '4e213a58e787437ff508e6c890ccd5fd32dc96b5' 2024-04-27T03:51:06.833 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-04-27T03:51:06.833 INFO:teuthology.packaging:ref: None 2024-04-27T03:51:06.833 INFO:teuthology.packaging:tag: None 2024-04-27T03:51:06.833 INFO:teuthology.packaging:branch: main 2024-04-27T03:51:06.834 INFO:teuthology.packaging:sha1: 4e213a58e787437ff508e6c890ccd5fd32dc96b5 2024-04-27T03:51:06.834 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=main 2024-04-27T03:51:07.213 INFO:teuthology.task.internal:Found packages for ceph version 19.0.0-1670.g1fb5d106 2024-04-27T03:51:07.214 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-04-27T03:51:07.226 INFO:teuthology.task.internal:no buildpackages task found 2024-04-27T03:51:07.226 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-04-27T03:51:07.239 INFO:teuthology.task.internal:Saving configuration 2024-04-27T03:51:07.260 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-04-27T03:51:07.271 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-04-27T03:51:07.294 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi162.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/teuthology-2024-04-27_03:08:09-upgrade-main-distro-default-smithi/7675783', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-27 03:45:45.617586', 'locked_by': 'scheduled_teuthology@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBDf6sVRRTSiqvkX+fbZWNAcRYHJslXRzTLFccLgVXzFazc5Jfx2Emd5XuO+nwOYakkoHP3JufzEDm151i1Aoi9o='} 2024-04-27T03:51:07.315 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi181.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/teuthology-2024-04-27_03:08:09-upgrade-main-distro-default-smithi/7675783', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-27 03:45:45.619268', 'locked_by': 'scheduled_teuthology@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBEtNFoOquI7mNXOJLHLdcnyh7cIgoneEfOhj0xbTqRro+WEYsvFITBxGXKQDyGVPFTSl9rJiTR8wedfsTsRQr0w='} 2024-04-27T03:51:07.315 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-04-27T03:51:07.326 INFO:teuthology.task.internal:roles: ubuntu@smithi162.front.sepia.ceph.com - ['host.a', 'client.0', 'osd.0', 'osd.1', 'osd.2'] 2024-04-27T03:51:07.326 INFO:teuthology.task.internal:roles: ubuntu@smithi181.front.sepia.ceph.com - ['host.b', 'client.1', 'osd.3', 'osd.4', 'osd.5'] 2024-04-27T03:51:07.326 INFO:teuthology.run_tasks:Running task console_log... 2024-04-27T03:51:07.400 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7fe7fec350d0>, signals=[15]) 2024-04-27T03:51:07.400 INFO:teuthology.run_tasks:Running task internal.connect... 2024-04-27T03:51:07.411 INFO:teuthology.task.internal:Opening connections... 2024-04-27T03:51:07.412 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi162.front.sepia.ceph.com 2024-04-27T03:51:07.414 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi162.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-27T03:51:07.488 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi181.front.sepia.ceph.com 2024-04-27T03:51:07.489 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi181.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-27T03:51:07.562 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-04-27T03:51:07.574 DEBUG:teuthology.orchestra.run.smithi162:> uname -m 2024-04-27T03:51:07.591 INFO:teuthology.orchestra.run.smithi162.stdout:x86_64 2024-04-27T03:51:07.592 DEBUG:teuthology.orchestra.run.smithi162:> cat /etc/os-release 2024-04-27T03:51:07.648 INFO:teuthology.orchestra.run.smithi162.stdout:NAME="CentOS Stream" 2024-04-27T03:51:07.648 INFO:teuthology.orchestra.run.smithi162.stdout:VERSION="9" 2024-04-27T03:51:07.648 INFO:teuthology.orchestra.run.smithi162.stdout:ID="centos" 2024-04-27T03:51:07.648 INFO:teuthology.orchestra.run.smithi162.stdout:ID_LIKE="rhel fedora" 2024-04-27T03:51:07.648 INFO:teuthology.orchestra.run.smithi162.stdout:VERSION_ID="9" 2024-04-27T03:51:07.648 INFO:teuthology.orchestra.run.smithi162.stdout:PLATFORM_ID="platform:el9" 2024-04-27T03:51:07.648 INFO:teuthology.orchestra.run.smithi162.stdout:PRETTY_NAME="CentOS Stream 9" 2024-04-27T03:51:07.648 INFO:teuthology.orchestra.run.smithi162.stdout:ANSI_COLOR="0;31" 2024-04-27T03:51:07.648 INFO:teuthology.orchestra.run.smithi162.stdout:LOGO="fedora-logo-icon" 2024-04-27T03:51:07.648 INFO:teuthology.orchestra.run.smithi162.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-04-27T03:51:07.648 INFO:teuthology.orchestra.run.smithi162.stdout:HOME_URL="https://centos.org/" 2024-04-27T03:51:07.648 INFO:teuthology.orchestra.run.smithi162.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-04-27T03:51:07.649 INFO:teuthology.orchestra.run.smithi162.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-04-27T03:51:07.649 INFO:teuthology.orchestra.run.smithi162.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-04-27T03:51:07.649 INFO:teuthology.lock.ops:Updating smithi162.front.sepia.ceph.com on lock server 2024-04-27T03:51:07.674 DEBUG:teuthology.orchestra.run.smithi181:> uname -m 2024-04-27T03:51:07.692 INFO:teuthology.orchestra.run.smithi181.stdout:x86_64 2024-04-27T03:51:07.692 DEBUG:teuthology.orchestra.run.smithi181:> cat /etc/os-release 2024-04-27T03:51:07.749 INFO:teuthology.orchestra.run.smithi181.stdout:NAME="CentOS Stream" 2024-04-27T03:51:07.749 INFO:teuthology.orchestra.run.smithi181.stdout:VERSION="9" 2024-04-27T03:51:07.750 INFO:teuthology.orchestra.run.smithi181.stdout:ID="centos" 2024-04-27T03:51:07.750 INFO:teuthology.orchestra.run.smithi181.stdout:ID_LIKE="rhel fedora" 2024-04-27T03:51:07.750 INFO:teuthology.orchestra.run.smithi181.stdout:VERSION_ID="9" 2024-04-27T03:51:07.750 INFO:teuthology.orchestra.run.smithi181.stdout:PLATFORM_ID="platform:el9" 2024-04-27T03:51:07.750 INFO:teuthology.orchestra.run.smithi181.stdout:PRETTY_NAME="CentOS Stream 9" 2024-04-27T03:51:07.750 INFO:teuthology.orchestra.run.smithi181.stdout:ANSI_COLOR="0;31" 2024-04-27T03:51:07.750 INFO:teuthology.orchestra.run.smithi181.stdout:LOGO="fedora-logo-icon" 2024-04-27T03:51:07.750 INFO:teuthology.orchestra.run.smithi181.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-04-27T03:51:07.750 INFO:teuthology.orchestra.run.smithi181.stdout:HOME_URL="https://centos.org/" 2024-04-27T03:51:07.750 INFO:teuthology.orchestra.run.smithi181.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-04-27T03:51:07.751 INFO:teuthology.orchestra.run.smithi181.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-04-27T03:51:07.751 INFO:teuthology.orchestra.run.smithi181.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-04-27T03:51:07.751 INFO:teuthology.lock.ops:Updating smithi181.front.sepia.ceph.com on lock server 2024-04-27T03:51:07.773 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-04-27T03:51:07.788 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-04-27T03:51:07.800 INFO:teuthology.task.internal:Checking for old test directory... 2024-04-27T03:51:07.800 DEBUG:teuthology.orchestra.run.smithi162:> test '!' -e /home/ubuntu/cephtest 2024-04-27T03:51:07.803 DEBUG:teuthology.orchestra.run.smithi181:> test '!' -e /home/ubuntu/cephtest 2024-04-27T03:51:07.819 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-04-27T03:51:07.831 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-04-27T03:51:07.831 DEBUG:teuthology.orchestra.run.smithi162:> test -z $(ls -A /var/lib/ceph) 2024-04-27T03:51:07.860 DEBUG:teuthology.orchestra.run.smithi181:> test -z $(ls -A /var/lib/ceph) 2024-04-27T03:51:07.954 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-04-27T03:51:08.013 INFO:teuthology.run_tasks:Running task kernel... 2024-04-27T03:51:08.038 INFO:teuthology.task.kernel:normalize config orig: {'kdb': 1, 'sha1': 'distro'} 2024-04-27T03:51:08.038 INFO:teuthology.task.kernel:config {'host.a': {'kdb': 1, 'sha1': 'distro'}, 'host.b': {'kdb': 1, 'sha1': 'distro'}}, timeout 300 2024-04-27T03:51:08.038 DEBUG:teuthology.orchestra.run.smithi162:> test -f /run/.containerenv -o -f /.dockerenv 2024-04-27T03:51:08.039 DEBUG:teuthology.orchestra.run.smithi181:> test -f /run/.containerenv -o -f /.dockerenv 2024-04-27T03:51:08.055 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-27T03:51:08.055 DEBUG:teuthology.orchestra.run.smithi162:> uname -r 2024-04-27T03:51:08.056 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-27T03:51:08.057 DEBUG:teuthology.orchestra.run.smithi181:> uname -r 2024-04-27T03:51:08.114 INFO:teuthology.orchestra.run.smithi162.stdout:5.14.0-437.el9.x86_64 2024-04-27T03:51:08.114 INFO:teuthology.task.kernel:Running kernel on smithi162: 5.14.0-437.el9.x86_64 2024-04-27T03:51:08.114 DEBUG:teuthology.orchestra.run.smithi162:> sudo yum install -y kernel 2024-04-27T03:51:08.116 INFO:teuthology.orchestra.run.smithi181.stdout:5.14.0-437.el9.x86_64 2024-04-27T03:51:08.116 INFO:teuthology.task.kernel:Running kernel on smithi181: 5.14.0-437.el9.x86_64 2024-04-27T03:51:08.116 DEBUG:teuthology.orchestra.run.smithi181:> sudo yum install -y kernel 2024-04-27T03:51:10.761 INFO:teuthology.orchestra.run.smithi181.stdout:CentOS Stream 9 - BaseOS 8.6 MB/s | 8.0 MB 00:00 2024-04-27T03:51:11.497 INFO:teuthology.orchestra.run.smithi162.stdout:CentOS Stream 9 - BaseOS 4.8 MB/s | 8.0 MB 00:01 2024-04-27T03:51:15.553 INFO:teuthology.orchestra.run.smithi181.stdout:CentOS Stream 9 - AppStream 5.1 MB/s | 19 MB 00:03 2024-04-27T03:51:19.745 INFO:teuthology.orchestra.run.smithi181.stdout:CentOS Stream 9 - CRB 17 MB/s | 6.2 MB 00:00 2024-04-27T03:51:21.679 INFO:teuthology.orchestra.run.smithi181.stdout:CentOS Stream 9 - Extras packages 29 kB/s | 16 kB 00:00 2024-04-27T03:51:23.597 INFO:teuthology.orchestra.run.smithi181.stdout:Extra Packages for Enterprise Linux 12 MB/s | 21 MB 00:01 2024-04-27T03:51:28.555 INFO:teuthology.orchestra.run.smithi181.stdout:lab-extras 29 kB/s | 1.7 kB 00:00 2024-04-27T03:51:30.444 INFO:teuthology.orchestra.run.smithi181.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-04-27T03:51:30.444 INFO:teuthology.orchestra.run.smithi181.stdout:Package kernel-5.14.0-437.el9.x86_64 is already installed. 2024-04-27T03:51:30.536 INFO:teuthology.orchestra.run.smithi181.stdout:Dependencies resolved. 2024-04-27T03:51:30.536 INFO:teuthology.orchestra.run.smithi181.stdout:Nothing to do. 2024-04-27T03:51:30.537 INFO:teuthology.orchestra.run.smithi181.stdout:Complete! 2024-04-27T03:51:30.636 DEBUG:teuthology.orchestra.run.smithi181:> echo no | sudo yum reinstall kernel || true 2024-04-27T03:51:31.134 INFO:teuthology.orchestra.run.smithi181.stdout:Last metadata expiration check: 0:00:03 ago on Sat 27 Apr 2024 03:51:28 AM UTC. 2024-04-27T03:51:31.253 INFO:teuthology.orchestra.run.smithi181.stdout:Dependencies resolved. 2024-04-27T03:51:31.270 INFO:teuthology.orchestra.run.smithi181.stdout:================================================================================ 2024-04-27T03:51:31.270 INFO:teuthology.orchestra.run.smithi181.stdout: Package Architecture Version Repository Size 2024-04-27T03:51:31.270 INFO:teuthology.orchestra.run.smithi181.stdout:================================================================================ 2024-04-27T03:51:31.270 INFO:teuthology.orchestra.run.smithi181.stdout:Reinstalling: 2024-04-27T03:51:31.270 INFO:teuthology.orchestra.run.smithi181.stdout: kernel x86_64 5.14.0-437.el9 baseos 255 k 2024-04-27T03:51:31.270 INFO:teuthology.orchestra.run.smithi181.stdout: 2024-04-27T03:51:31.271 INFO:teuthology.orchestra.run.smithi181.stdout:Transaction Summary 2024-04-27T03:51:31.271 INFO:teuthology.orchestra.run.smithi181.stdout:================================================================================ 2024-04-27T03:51:31.271 INFO:teuthology.orchestra.run.smithi181.stdout: 2024-04-27T03:51:31.271 INFO:teuthology.orchestra.run.smithi181.stdout:Total download size: 255 k 2024-04-27T03:51:31.271 INFO:teuthology.orchestra.run.smithi181.stdout:Installed size: 0 2024-04-27T03:51:31.271 INFO:teuthology.orchestra.run.smithi181.stderr:Operation aborted. 2024-04-27T03:51:31.323 INFO:teuthology.orchestra.run.smithi181.stdout:Is this ok [y/N]: 2024-04-27T03:51:31.323 DEBUG:teuthology.orchestra.run.smithi181:> sudo yum reinstall -y kernel || true 2024-04-27T03:51:31.817 INFO:teuthology.orchestra.run.smithi181.stdout:Last metadata expiration check: 0:00:03 ago on Sat 27 Apr 2024 03:51:28 AM UTC. 2024-04-27T03:51:31.933 INFO:teuthology.orchestra.run.smithi181.stdout:Dependencies resolved. 2024-04-27T03:51:31.934 INFO:teuthology.orchestra.run.smithi181.stdout:================================================================================ 2024-04-27T03:51:31.934 INFO:teuthology.orchestra.run.smithi181.stdout: Package Architecture Version Repository Size 2024-04-27T03:51:31.934 INFO:teuthology.orchestra.run.smithi181.stdout:================================================================================ 2024-04-27T03:51:31.934 INFO:teuthology.orchestra.run.smithi181.stdout:Reinstalling: 2024-04-27T03:51:31.934 INFO:teuthology.orchestra.run.smithi181.stdout: kernel x86_64 5.14.0-437.el9 baseos 255 k 2024-04-27T03:51:31.934 INFO:teuthology.orchestra.run.smithi181.stdout: 2024-04-27T03:51:31.934 INFO:teuthology.orchestra.run.smithi181.stdout:Transaction Summary 2024-04-27T03:51:31.934 INFO:teuthology.orchestra.run.smithi181.stdout:================================================================================ 2024-04-27T03:51:31.935 INFO:teuthology.orchestra.run.smithi181.stdout: 2024-04-27T03:51:31.935 INFO:teuthology.orchestra.run.smithi181.stdout:Total download size: 255 k 2024-04-27T03:51:31.935 INFO:teuthology.orchestra.run.smithi181.stdout:Installed size: 0 2024-04-27T03:51:31.935 INFO:teuthology.orchestra.run.smithi181.stdout:Downloading Packages: 2024-04-27T03:51:32.273 INFO:teuthology.orchestra.run.smithi181.stdout:kernel-5.14.0-437.el9.x86_64.rpm 1.3 MB/s | 255 kB 00:00 2024-04-27T03:51:32.274 INFO:teuthology.orchestra.run.smithi181.stdout:-------------------------------------------------------------------------------- 2024-04-27T03:51:32.274 INFO:teuthology.orchestra.run.smithi181.stdout:Total 752 kB/s | 255 kB 00:00 2024-04-27T03:51:32.292 INFO:teuthology.orchestra.run.smithi181.stdout:Running transaction check 2024-04-27T03:51:32.477 INFO:teuthology.orchestra.run.smithi181.stdout:Transaction check succeeded. 2024-04-27T03:51:32.477 INFO:teuthology.orchestra.run.smithi181.stdout:Running transaction test 2024-04-27T03:51:32.482 INFO:teuthology.orchestra.run.smithi181.stdout:Transaction test succeeded. 2024-04-27T03:51:32.482 INFO:teuthology.orchestra.run.smithi181.stdout:Running transaction 2024-04-27T03:51:32.629 INFO:teuthology.orchestra.run.smithi181.stdout: Preparing : 1/1 2024-04-27T03:51:32.699 INFO:teuthology.orchestra.run.smithi181.stdout: Reinstalling : kernel-5.14.0-437.el9.x86_64 1/2 2024-04-27T03:51:32.699 INFO:teuthology.orchestra.run.smithi181.stdout: Cleanup : kernel-5.14.0-437.el9.x86_64 2/2 2024-04-27T03:51:32.952 INFO:teuthology.orchestra.run.smithi162.stdout:CentOS Stream 9 - AppStream 956 kB/s | 19 MB 00:20 2024-04-27T03:51:33.395 INFO:teuthology.orchestra.run.smithi181.stdout: Running scriptlet: kernel-5.14.0-437.el9.x86_64 2/2 2024-04-27T03:51:33.395 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : kernel-5.14.0-437.el9.x86_64 1/2 2024-04-27T03:51:34.077 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : kernel-5.14.0-437.el9.x86_64 2/2 2024-04-27T03:51:34.078 INFO:teuthology.orchestra.run.smithi181.stdout: 2024-04-27T03:51:34.078 INFO:teuthology.orchestra.run.smithi181.stdout:Reinstalled: 2024-04-27T03:51:34.078 INFO:teuthology.orchestra.run.smithi181.stdout: kernel-5.14.0-437.el9.x86_64 2024-04-27T03:51:34.078 INFO:teuthology.orchestra.run.smithi181.stdout: 2024-04-27T03:51:34.078 INFO:teuthology.orchestra.run.smithi181.stdout:Complete! 2024-04-27T03:51:34.282 DEBUG:teuthology.orchestra.run.smithi181:> rpm -q kernel | sort -rV | head -n 1 2024-04-27T03:51:34.329 INFO:teuthology.orchestra.run.smithi181.stdout:kernel-5.14.0-437.el9.x86_64 2024-04-27T03:51:34.329 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-437.el9.x86_64 2024-04-27T03:51:34.330 INFO:teuthology.task.kernel:Newest distro kernel installed and running 2024-04-27T03:51:34.330 INFO:teuthology.task.kernel:Enabling kdb on host.b... 2024-04-27T03:51:34.330 DEBUG:teuthology.orchestra.run.smithi181:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-04-27T03:51:34.361 INFO:teuthology.orchestra.run.smithi181.stdout:ttyS1 2024-04-27T03:51:34.382 DEBUG:teuthology.parallel:result is None 2024-04-27T03:51:38.093 INFO:teuthology.orchestra.run.smithi162.stdout:CentOS Stream 9 - CRB 4.1 MB/s | 6.2 MB 00:01 2024-04-27T03:51:40.302 INFO:teuthology.orchestra.run.smithi162.stdout:CentOS Stream 9 - Extras packages 20 kB/s | 16 kB 00:00 2024-04-27T03:51:42.112 INFO:teuthology.orchestra.run.smithi162.stdout:Extra Packages for Enterprise Linux 12 MB/s | 21 MB 00:01 2024-04-27T03:51:47.178 INFO:teuthology.orchestra.run.smithi162.stdout:lab-extras 13 kB/s | 1.7 kB 00:00 2024-04-27T03:51:49.067 INFO:teuthology.orchestra.run.smithi162.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-04-27T03:51:49.067 INFO:teuthology.orchestra.run.smithi162.stdout:Package kernel-5.14.0-437.el9.x86_64 is already installed. 2024-04-27T03:51:49.169 INFO:teuthology.orchestra.run.smithi162.stdout:Dependencies resolved. 2024-04-27T03:51:49.170 INFO:teuthology.orchestra.run.smithi162.stdout:Nothing to do. 2024-04-27T03:51:49.171 INFO:teuthology.orchestra.run.smithi162.stdout:Complete! 2024-04-27T03:51:49.303 DEBUG:teuthology.orchestra.run.smithi162:> echo no | sudo yum reinstall kernel || true 2024-04-27T03:51:49.803 INFO:teuthology.orchestra.run.smithi162.stdout:Last metadata expiration check: 0:00:02 ago on Sat 27 Apr 2024 03:51:47 AM UTC. 2024-04-27T03:51:49.931 INFO:teuthology.orchestra.run.smithi162.stdout:Dependencies resolved. 2024-04-27T03:51:49.936 INFO:teuthology.orchestra.run.smithi162.stdout:================================================================================ 2024-04-27T03:51:49.936 INFO:teuthology.orchestra.run.smithi162.stdout: Package Architecture Version Repository Size 2024-04-27T03:51:49.936 INFO:teuthology.orchestra.run.smithi162.stdout:================================================================================ 2024-04-27T03:51:49.936 INFO:teuthology.orchestra.run.smithi162.stdout:Reinstalling: 2024-04-27T03:51:49.936 INFO:teuthology.orchestra.run.smithi162.stdout: kernel x86_64 5.14.0-437.el9 baseos 255 k 2024-04-27T03:51:49.937 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-04-27T03:51:49.937 INFO:teuthology.orchestra.run.smithi162.stdout:Transaction Summary 2024-04-27T03:51:49.937 INFO:teuthology.orchestra.run.smithi162.stdout:================================================================================ 2024-04-27T03:51:49.937 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-04-27T03:51:49.937 INFO:teuthology.orchestra.run.smithi162.stdout:Total download size: 255 k 2024-04-27T03:51:49.937 INFO:teuthology.orchestra.run.smithi162.stdout:Installed size: 0 2024-04-27T03:51:49.937 INFO:teuthology.orchestra.run.smithi162.stderr:Operation aborted. 2024-04-27T03:51:50.002 INFO:teuthology.orchestra.run.smithi162.stdout:Is this ok [y/N]: 2024-04-27T03:51:50.002 DEBUG:teuthology.orchestra.run.smithi162:> sudo yum reinstall -y kernel || true 2024-04-27T03:51:50.491 INFO:teuthology.orchestra.run.smithi162.stdout:Last metadata expiration check: 0:00:03 ago on Sat 27 Apr 2024 03:51:47 AM UTC. 2024-04-27T03:51:50.607 INFO:teuthology.orchestra.run.smithi162.stdout:Dependencies resolved. 2024-04-27T03:51:50.608 INFO:teuthology.orchestra.run.smithi162.stdout:================================================================================ 2024-04-27T03:51:50.608 INFO:teuthology.orchestra.run.smithi162.stdout: Package Architecture Version Repository Size 2024-04-27T03:51:50.608 INFO:teuthology.orchestra.run.smithi162.stdout:================================================================================ 2024-04-27T03:51:50.608 INFO:teuthology.orchestra.run.smithi162.stdout:Reinstalling: 2024-04-27T03:51:50.608 INFO:teuthology.orchestra.run.smithi162.stdout: kernel x86_64 5.14.0-437.el9 baseos 255 k 2024-04-27T03:51:50.608 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-04-27T03:51:50.608 INFO:teuthology.orchestra.run.smithi162.stdout:Transaction Summary 2024-04-27T03:51:50.608 INFO:teuthology.orchestra.run.smithi162.stdout:================================================================================ 2024-04-27T03:51:50.608 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-04-27T03:51:50.608 INFO:teuthology.orchestra.run.smithi162.stdout:Total download size: 255 k 2024-04-27T03:51:50.608 INFO:teuthology.orchestra.run.smithi162.stdout:Installed size: 0 2024-04-27T03:51:50.609 INFO:teuthology.orchestra.run.smithi162.stdout:Downloading Packages: 2024-04-27T03:51:50.879 INFO:teuthology.orchestra.run.smithi162.stdout:kernel-5.14.0-437.el9.x86_64.rpm 1.5 MB/s | 255 kB 00:00 2024-04-27T03:51:50.880 INFO:teuthology.orchestra.run.smithi162.stdout:-------------------------------------------------------------------------------- 2024-04-27T03:51:50.880 INFO:teuthology.orchestra.run.smithi162.stdout:Total 939 kB/s | 255 kB 00:00 2024-04-27T03:51:50.898 INFO:teuthology.orchestra.run.smithi162.stdout:Running transaction check 2024-04-27T03:51:51.085 INFO:teuthology.orchestra.run.smithi162.stdout:Transaction check succeeded. 2024-04-27T03:51:51.085 INFO:teuthology.orchestra.run.smithi162.stdout:Running transaction test 2024-04-27T03:51:51.090 INFO:teuthology.orchestra.run.smithi162.stdout:Transaction test succeeded. 2024-04-27T03:51:51.090 INFO:teuthology.orchestra.run.smithi162.stdout:Running transaction 2024-04-27T03:51:51.222 INFO:teuthology.orchestra.run.smithi162.stdout: Preparing : 1/1 2024-04-27T03:51:51.289 INFO:teuthology.orchestra.run.smithi162.stdout: Reinstalling : kernel-5.14.0-437.el9.x86_64 1/2 2024-04-27T03:51:51.289 INFO:teuthology.orchestra.run.smithi162.stdout: Cleanup : kernel-5.14.0-437.el9.x86_64 2/2 2024-04-27T03:51:51.964 INFO:teuthology.orchestra.run.smithi162.stdout: Running scriptlet: kernel-5.14.0-437.el9.x86_64 2/2 2024-04-27T03:51:51.964 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : kernel-5.14.0-437.el9.x86_64 1/2 2024-04-27T03:51:52.617 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : kernel-5.14.0-437.el9.x86_64 2/2 2024-04-27T03:51:52.617 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-04-27T03:51:52.617 INFO:teuthology.orchestra.run.smithi162.stdout:Reinstalled: 2024-04-27T03:51:52.617 INFO:teuthology.orchestra.run.smithi162.stdout: kernel-5.14.0-437.el9.x86_64 2024-04-27T03:51:52.617 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-04-27T03:51:52.617 INFO:teuthology.orchestra.run.smithi162.stdout:Complete! 2024-04-27T03:51:52.825 DEBUG:teuthology.orchestra.run.smithi162:> rpm -q kernel | sort -rV | head -n 1 2024-04-27T03:51:52.872 INFO:teuthology.orchestra.run.smithi162.stdout:kernel-5.14.0-437.el9.x86_64 2024-04-27T03:51:52.872 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-437.el9.x86_64 2024-04-27T03:51:52.872 INFO:teuthology.task.kernel:Newest distro kernel installed and running 2024-04-27T03:51:52.873 INFO:teuthology.task.kernel:Enabling kdb on host.a... 2024-04-27T03:51:52.873 DEBUG:teuthology.orchestra.run.smithi162:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-04-27T03:51:52.898 INFO:teuthology.orchestra.run.smithi162.stdout:ttyS1 2024-04-27T03:51:52.924 DEBUG:teuthology.parallel:result is None 2024-04-27T03:51:52.924 INFO:teuthology.run_tasks:Running task internal.base... 2024-04-27T03:51:52.934 INFO:teuthology.task.internal:Creating test directory... 2024-04-27T03:51:52.934 DEBUG:teuthology.orchestra.run.smithi162:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-04-27T03:51:52.967 DEBUG:teuthology.orchestra.run.smithi181:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-04-27T03:51:52.984 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-04-27T03:51:52.995 INFO:teuthology.run_tasks:Running task internal.archive... 2024-04-27T03:51:53.008 INFO:teuthology.task.internal:Creating archive directory... 2024-04-27T03:51:53.008 DEBUG:teuthology.orchestra.run.smithi162:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-04-27T03:51:53.021 DEBUG:teuthology.orchestra.run.smithi181:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-04-27T03:51:53.306 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-04-27T03:51:53.317 INFO:teuthology.task.internal:Enabling coredump saving... 2024-04-27T03:51:53.317 DEBUG:teuthology.orchestra.run.smithi162:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-04-27T03:51:53.320 DEBUG:teuthology.orchestra.run.smithi181:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-04-27T03:51:53.348 INFO:teuthology.orchestra.run.smithi162.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-27T03:51:53.357 INFO:teuthology.orchestra.run.smithi181.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-27T03:51:53.357 INFO:teuthology.orchestra.run.smithi162.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-27T03:51:53.367 INFO:teuthology.orchestra.run.smithi181.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-27T03:51:53.369 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-04-27T03:51:53.377 INFO:teuthology.task.internal:Configuring sudo... 2024-04-27T03:51:53.377 DEBUG:teuthology.orchestra.run.smithi162:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-04-27T03:51:53.401 DEBUG:teuthology.orchestra.run.smithi181:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-04-27T03:51:53.435 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-04-27T03:51:53.445 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-04-27T03:51:53.445 DEBUG:teuthology.orchestra.run.smithi162:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-04-27T03:51:53.465 DEBUG:teuthology.orchestra.run.smithi181:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-04-27T03:51:53.492 DEBUG:teuthology.orchestra.run.smithi162:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-27T03:51:53.560 DEBUG:teuthology.orchestra.run.smithi162:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-27T03:51:53.607 DEBUG:teuthology.orchestra.run.smithi162:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-27T03:51:53.688 DEBUG:teuthology.orchestra.run.smithi162:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-27T03:51:53.726 DEBUG:teuthology.orchestra.run.smithi162:> set -ex 2024-04-27T03:51:53.726 DEBUG:teuthology.orchestra.run.smithi162:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-04-27T03:51:53.790 DEBUG:teuthology.orchestra.run.smithi181:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-27T03:51:53.835 DEBUG:teuthology.orchestra.run.smithi181:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-27T03:51:53.883 DEBUG:teuthology.orchestra.run.smithi181:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-27T03:51:53.968 DEBUG:teuthology.orchestra.run.smithi181:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-27T03:51:54.007 DEBUG:teuthology.orchestra.run.smithi181:> set -ex 2024-04-27T03:51:54.007 DEBUG:teuthology.orchestra.run.smithi181:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-04-27T03:51:54.074 DEBUG:teuthology.orchestra.run.smithi162:> sudo service rsyslog restart 2024-04-27T03:51:54.077 DEBUG:teuthology.orchestra.run.smithi181:> sudo service rsyslog restart 2024-04-27T03:51:54.105 INFO:teuthology.orchestra.run.smithi162.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-27T03:51:54.158 INFO:teuthology.orchestra.run.smithi181.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-27T03:51:54.464 INFO:teuthology.run_tasks:Running task internal.timer... 2024-04-27T03:51:54.472 INFO:teuthology.task.internal:Starting timer... 2024-04-27T03:51:54.472 INFO:teuthology.run_tasks:Running task pcp... 2024-04-27T03:51:54.501 INFO:teuthology.run_tasks:Running task selinux... 2024-04-27T03:51:54.510 DEBUG:teuthology.task:Applying overrides for task selinux: {'allowlist': ['scontext=system_u:system_r:logrotate_t:s0', 'scontext=system_u:system_r:getty_t:s0']} 2024-04-27T03:51:54.510 DEBUG:teuthology.orchestra.run.smithi162:> sudo service auditd rotate 2024-04-27T03:51:54.553 INFO:teuthology.orchestra.run.smithi162.stdout:Rotating logs: 2024-04-27T03:51:54.555 DEBUG:teuthology.orchestra.run.smithi181:> sudo service auditd rotate 2024-04-27T03:51:54.610 INFO:teuthology.orchestra.run.smithi181.stdout:Rotating logs: 2024-04-27T03:51:54.611 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-04-27T03:51:54.612 DEBUG:teuthology.orchestra.run.smithi162:> /usr/sbin/getenforce 2024-04-27T03:51:54.630 INFO:teuthology.orchestra.run.smithi162.stdout:Permissive 2024-04-27T03:51:54.631 DEBUG:teuthology.orchestra.run.smithi181:> /usr/sbin/getenforce 2024-04-27T03:51:54.660 INFO:teuthology.orchestra.run.smithi181.stdout:Permissive 2024-04-27T03:51:54.660 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi162.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi181.front.sepia.ceph.com': 'permissive'} 2024-04-27T03:51:54.661 DEBUG:teuthology.orchestra.run.smithi162:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-27T03:51:54.693 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-27T03:51:54.694 DEBUG:teuthology.orchestra.run.smithi181:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-27T03:51:54.724 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-27T03:51:54.724 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-04-27T03:51:54.724 DEBUG:teuthology.orchestra.run.smithi162:> sudo /usr/sbin/setenforce permissive 2024-04-27T03:51:54.766 DEBUG:teuthology.orchestra.run.smithi181:> sudo /usr/sbin/setenforce permissive 2024-04-27T03:51:54.791 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-04-27T03:51:54.800 DEBUG:teuthology.repo_utils:Setting repo remote to https://git.ceph.com/ceph-cm-ansible.git 2024-04-27T03:51:54.806 INFO:teuthology.repo_utils:Fetching git.ceph.com_ceph-cm-ansible_main from origin 2024-04-27T03:51:54.951 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-04-27T03:51:54.968 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-04-27T03:51:54.969 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi162.front.sepia.ceph.com,smithi181.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-04-27T03:55:24.795 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi162.front.sepia.ceph.com'), Remote(name='ubuntu@smithi181.front.sepia.ceph.com')] 2024-04-27T03:55:24.796 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi162.front.sepia.ceph.com' 2024-04-27T03:55:24.797 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi162.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-27T03:55:24.873 DEBUG:teuthology.orchestra.run.smithi162:> true 2024-04-27T03:55:24.957 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi162.front.sepia.ceph.com' 2024-04-27T03:55:24.957 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi181.front.sepia.ceph.com' 2024-04-27T03:55:24.957 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi181.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-27T03:55:25.034 DEBUG:teuthology.orchestra.run.smithi181:> true 2024-04-27T03:55:25.114 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi181.front.sepia.ceph.com' 2024-04-27T03:55:25.114 INFO:teuthology.run_tasks:Running task clock... 2024-04-27T03:55:25.126 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-04-27T03:55:25.126 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-04-27T03:55:25.127 DEBUG:teuthology.orchestra.run.smithi162:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-27T03:55:25.129 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-04-27T03:55:25.129 DEBUG:teuthology.orchestra.run.smithi181:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-27T03:55:25.154 INFO:teuthology.orchestra.run.smithi162.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-04-27T03:55:25.166 INFO:teuthology.orchestra.run.smithi162.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-04-27T03:55:25.181 INFO:teuthology.orchestra.run.smithi181.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-04-27T03:55:25.188 INFO:teuthology.orchestra.run.smithi162.stderr:sudo: ntpd: command not found 2024-04-27T03:55:25.194 INFO:teuthology.orchestra.run.smithi181.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-04-27T03:55:25.198 INFO:teuthology.orchestra.run.smithi162.stdout:506 Cannot talk to daemon 2024-04-27T03:55:25.211 INFO:teuthology.orchestra.run.smithi162.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-04-27T03:55:25.217 INFO:teuthology.orchestra.run.smithi181.stderr:sudo: ntpd: command not found 2024-04-27T03:55:25.224 INFO:teuthology.orchestra.run.smithi162.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-04-27T03:55:25.227 INFO:teuthology.orchestra.run.smithi181.stdout:506 Cannot talk to daemon 2024-04-27T03:55:25.241 INFO:teuthology.orchestra.run.smithi181.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-04-27T03:55:25.253 INFO:teuthology.orchestra.run.smithi181.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-04-27T03:55:25.279 INFO:teuthology.orchestra.run.smithi162.stderr:bash: line 1: ntpq: command not found 2024-04-27T03:55:25.283 INFO:teuthology.orchestra.run.smithi162.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-27T03:55:25.283 INFO:teuthology.orchestra.run.smithi162.stdout:=============================================================================== 2024-04-27T03:55:25.283 INFO:teuthology.orchestra.run.smithi162.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-27T03:55:25.283 INFO:teuthology.orchestra.run.smithi162.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-27T03:55:25.283 INFO:teuthology.orchestra.run.smithi162.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-27T03:55:25.283 INFO:teuthology.orchestra.run.smithi162.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-27T03:55:25.312 INFO:teuthology.orchestra.run.smithi181.stderr:bash: line 1: ntpq: command not found 2024-04-27T03:55:25.315 INFO:teuthology.orchestra.run.smithi181.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-27T03:55:25.315 INFO:teuthology.orchestra.run.smithi181.stdout:=============================================================================== 2024-04-27T03:55:25.315 INFO:teuthology.orchestra.run.smithi181.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-27T03:55:25.316 INFO:teuthology.orchestra.run.smithi181.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-27T03:55:25.316 INFO:teuthology.orchestra.run.smithi181.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-27T03:55:25.316 INFO:teuthology.orchestra.run.smithi181.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-27T03:55:25.316 INFO:teuthology.run_tasks:Running task install... 2024-04-27T03:55:25.324 DEBUG:teuthology.task.install:project ceph 2024-04-27T03:55:25.324 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': '4e213a58e787437ff508e6c890ccd5fd32dc96b5'}} 2024-04-27T03:55:25.324 DEBUG:teuthology.task.install:config {'branch': 'quincy', 'exclude_packages': ['ceph-volume'], 'flavor': 'default', 'sha1': '4e213a58e787437ff508e6c890ccd5fd32dc96b5'} 2024-04-27T03:55:25.324 INFO:teuthology.task.install:Using flavor: default 2024-04-27T03:55:25.330 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-04-27T03:55:25.330 INFO:teuthology.task.install:extra packages: [] 2024-04-27T03:55:25.330 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': 'quincy', 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': ['ceph-volume'], 'extra_packages': [], 'extra_system_packages': [], 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '4e213a58e787437ff508e6c890ccd5fd32dc96b5', 'tag': None, 'wait_for_package': False} 2024-04-27T03:55:25.331 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-04-27T03:55:25.331 INFO:teuthology.packaging:ref: None 2024-04-27T03:55:25.331 INFO:teuthology.packaging:tag: None 2024-04-27T03:55:25.331 INFO:teuthology.packaging:branch: quincy 2024-04-27T03:55:25.331 INFO:teuthology.packaging:sha1: 4e213a58e787437ff508e6c890ccd5fd32dc96b5 2024-04-27T03:55:25.331 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=quincy 2024-04-27T03:55:25.333 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': 'quincy', 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': ['ceph-volume'], 'extra_packages': [], 'extra_system_packages': [], 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '4e213a58e787437ff508e6c890ccd5fd32dc96b5', 'tag': None, 'wait_for_package': False} 2024-04-27T03:55:25.333 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-04-27T03:55:25.333 INFO:teuthology.packaging:ref: None 2024-04-27T03:55:25.333 INFO:teuthology.packaging:tag: None 2024-04-27T03:55:25.333 INFO:teuthology.packaging:branch: quincy 2024-04-27T03:55:25.334 INFO:teuthology.packaging:sha1: 4e213a58e787437ff508e6c890ccd5fd32dc96b5 2024-04-27T03:55:25.334 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=quincy 2024-04-27T03:55:25.590 INFO:teuthology.task.install.rpm:Pulling from https://3.chacra.ceph.com/r/ceph/quincy/2be3efa97a7972f8e925bd899b474329979b74d6/centos/9/flavors/default/ 2024-04-27T03:55:25.590 INFO:teuthology.task.install.rpm:Package version is 17.2.7-907.g2be3efa9 2024-04-27T03:55:25.726 INFO:teuthology.task.install.rpm:Pulling from https://3.chacra.ceph.com/r/ceph/quincy/2be3efa97a7972f8e925bd899b474329979b74d6/centos/9/flavors/default/ 2024-04-27T03:55:25.726 INFO:teuthology.task.install.rpm:Package version is 17.2.7-907.g2be3efa9 2024-04-27T03:55:25.787 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://3.chacra.ceph.com/r/ceph/quincy/2be3efa97a7972f8e925bd899b474329979b74d6/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://3.chacra.ceph.com/r/ceph/quincy/2be3efa97a7972f8e925bd899b474329979b74d6/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://3.chacra.ceph.com/r/ceph/quincy/2be3efa97a7972f8e925bd899b474329979b74d6/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-04-27T03:55:25.788 DEBUG:teuthology.orchestra.run.smithi181:> set -ex 2024-04-27T03:55:25.788 DEBUG:teuthology.orchestra.run.smithi181:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-04-27T03:55:25.814 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd on remote rpm x86_64 2024-04-27T03:55:25.814 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-04-27T03:55:25.814 INFO:teuthology.packaging:ref: None 2024-04-27T03:55:25.814 INFO:teuthology.packaging:tag: None 2024-04-27T03:55:25.814 INFO:teuthology.packaging:branch: quincy 2024-04-27T03:55:25.814 INFO:teuthology.packaging:sha1: 4e213a58e787437ff508e6c890ccd5fd32dc96b5 2024-04-27T03:55:25.814 DEBUG:teuthology.orchestra.run.smithi181:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;ref/quincy/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-04-27T03:55:25.879 DEBUG:teuthology.orchestra.run.smithi181:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-04-27T03:55:25.892 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://3.chacra.ceph.com/r/ceph/quincy/2be3efa97a7972f8e925bd899b474329979b74d6/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://3.chacra.ceph.com/r/ceph/quincy/2be3efa97a7972f8e925bd899b474329979b74d6/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://3.chacra.ceph.com/r/ceph/quincy/2be3efa97a7972f8e925bd899b474329979b74d6/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-04-27T03:55:25.892 DEBUG:teuthology.orchestra.run.smithi162:> set -ex 2024-04-27T03:55:25.892 DEBUG:teuthology.orchestra.run.smithi162:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-04-27T03:55:25.917 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd on remote rpm x86_64 2024-04-27T03:55:25.917 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-04-27T03:55:25.917 INFO:teuthology.packaging:ref: None 2024-04-27T03:55:25.917 INFO:teuthology.packaging:tag: None 2024-04-27T03:55:25.917 INFO:teuthology.packaging:branch: quincy 2024-04-27T03:55:25.917 INFO:teuthology.packaging:sha1: 4e213a58e787437ff508e6c890ccd5fd32dc96b5 2024-04-27T03:55:25.917 DEBUG:teuthology.orchestra.run.smithi162:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;ref/quincy/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-04-27T03:55:25.953 DEBUG:teuthology.orchestra.run.smithi181:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-04-27T03:55:25.982 DEBUG:teuthology.orchestra.run.smithi162:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-04-27T03:55:26.016 INFO:teuthology.orchestra.run.smithi181.stdout:check_obsoletes = 1 2024-04-27T03:55:26.018 DEBUG:teuthology.orchestra.run.smithi181:> sudo yum clean all 2024-04-27T03:55:26.055 DEBUG:teuthology.orchestra.run.smithi162:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-04-27T03:55:26.118 INFO:teuthology.orchestra.run.smithi162.stdout:check_obsoletes = 1 2024-04-27T03:55:26.120 DEBUG:teuthology.orchestra.run.smithi162:> sudo yum clean all 2024-04-27T03:55:26.243 INFO:teuthology.orchestra.run.smithi181.stdout:48 files removed 2024-04-27T03:55:26.270 DEBUG:teuthology.orchestra.run.smithi181:> sudo yum -y install ceph-radosgw 2024-04-27T03:55:26.347 INFO:teuthology.orchestra.run.smithi162.stdout:48 files removed 2024-04-27T03:55:26.374 DEBUG:teuthology.orchestra.run.smithi162:> sudo yum -y install ceph-radosgw 2024-04-27T03:55:26.977 INFO:teuthology.orchestra.run.smithi181.stdout:ceph packages for x86_64 172 kB/s | 77 kB 00:00 2024-04-27T03:55:27.104 INFO:teuthology.orchestra.run.smithi162.stdout:ceph packages for x86_64 164 kB/s | 77 kB 00:00 2024-04-27T03:55:27.387 INFO:teuthology.orchestra.run.smithi181.stdout:ceph noarch packages 29 kB/s | 11 kB 00:00 2024-04-27T03:55:27.531 INFO:teuthology.orchestra.run.smithi162.stdout:ceph noarch packages 28 kB/s | 11 kB 00:00 2024-04-27T03:55:27.753 INFO:teuthology.orchestra.run.smithi181.stdout:ceph source packages 5.4 kB/s | 1.8 kB 00:00 2024-04-27T03:55:27.907 INFO:teuthology.orchestra.run.smithi162.stdout:ceph source packages 5.3 kB/s | 1.8 kB 00:00 2024-04-27T03:55:28.423 INFO:teuthology.orchestra.run.smithi162.stdout:CentOS Stream 9 - BaseOS 17 MB/s | 8.0 MB 00:00 2024-04-27T03:55:28.929 INFO:teuthology.orchestra.run.smithi181.stdout:CentOS Stream 9 - BaseOS 7.0 MB/s | 8.0 MB 00:01 2024-04-27T03:55:32.506 INFO:teuthology.orchestra.run.smithi162.stdout:CentOS Stream 9 - AppStream 6.3 MB/s | 19 MB 00:02 2024-04-27T03:55:45.072 INFO:teuthology.orchestra.run.smithi162.stdout:CentOS Stream 9 - CRB 706 kB/s | 6.2 MB 00:08 2024-04-27T03:55:46.639 INFO:teuthology.orchestra.run.smithi162.stdout:CentOS Stream 9 - Extras packages 40 kB/s | 16 kB 00:00 2024-04-27T03:55:48.267 INFO:teuthology.orchestra.run.smithi162.stdout:Extra Packages for Enterprise Linux 14 MB/s | 21 MB 00:01 2024-04-27T03:55:52.619 INFO:teuthology.orchestra.run.smithi181.stdout:CentOS Stream 9 - AppStream 860 kB/s | 19 MB 00:22 2024-04-27T03:55:53.423 INFO:teuthology.orchestra.run.smithi162.stdout:lab-extras 34 kB/s | 1.7 kB 00:00 2024-04-27T03:55:55.560 INFO:teuthology.orchestra.run.smithi162.stdout:Dependencies resolved. 2024-04-27T03:55:55.562 INFO:teuthology.orchestra.run.smithi162.stdout:================================================================================ 2024-04-27T03:55:55.562 INFO:teuthology.orchestra.run.smithi162.stdout: Package Arch Version Repository Size 2024-04-27T03:55:55.562 INFO:teuthology.orchestra.run.smithi162.stdout:================================================================================ 2024-04-27T03:55:55.562 INFO:teuthology.orchestra.run.smithi162.stdout:Installing: 2024-04-27T03:55:55.562 INFO:teuthology.orchestra.run.smithi162.stdout: ceph-radosgw x86_64 2:17.2.7-907.g2be3efa9.el9 ceph 14 M 2024-04-27T03:55:55.562 INFO:teuthology.orchestra.run.smithi162.stdout:Upgrading: 2024-04-27T03:55:55.562 INFO:teuthology.orchestra.run.smithi162.stdout: librados2 x86_64 2:17.2.7-907.g2be3efa9.el9 ceph 3.1 M 2024-04-27T03:55:55.562 INFO:teuthology.orchestra.run.smithi162.stdout: librbd1 x86_64 2:17.2.7-907.g2be3efa9.el9 ceph 2.9 M 2024-04-27T03:55:55.562 INFO:teuthology.orchestra.run.smithi162.stdout:Installing dependencies: 2024-04-27T03:55:55.562 INFO:teuthology.orchestra.run.smithi162.stdout: ceph-base x86_64 2:17.2.7-907.g2be3efa9.el9 ceph 5.0 M 2024-04-27T03:55:55.563 INFO:teuthology.orchestra.run.smithi162.stdout: ceph-common x86_64 2:17.2.7-907.g2be3efa9.el9 ceph 22 M 2024-04-27T03:55:55.563 INFO:teuthology.orchestra.run.smithi162.stdout: ceph-selinux x86_64 2:17.2.7-907.g2be3efa9.el9 ceph 25 k 2024-04-27T03:55:55.563 INFO:teuthology.orchestra.run.smithi162.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-04-27T03:55:55.563 INFO:teuthology.orchestra.run.smithi162.stdout: libcephfs2 x86_64 2:17.2.7-907.g2be3efa9.el9 ceph 637 k 2024-04-27T03:55:55.563 INFO:teuthology.orchestra.run.smithi162.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-04-27T03:55:55.563 INFO:teuthology.orchestra.run.smithi162.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-04-27T03:55:55.563 INFO:teuthology.orchestra.run.smithi162.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-04-27T03:55:55.563 INFO:teuthology.orchestra.run.smithi162.stdout: libradosstriper1 x86_64 2:17.2.7-907.g2be3efa9.el9 ceph 457 k 2024-04-27T03:55:55.563 INFO:teuthology.orchestra.run.smithi162.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-04-27T03:55:55.563 INFO:teuthology.orchestra.run.smithi162.stdout: librgw2 x86_64 2:17.2.7-907.g2be3efa9.el9 ceph 4.9 M 2024-04-27T03:55:55.563 INFO:teuthology.orchestra.run.smithi162.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-04-27T03:55:55.563 INFO:teuthology.orchestra.run.smithi162.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-04-27T03:55:55.563 INFO:teuthology.orchestra.run.smithi162.stdout: python3-ceph-argparse x86_64 2:17.2.7-907.g2be3efa9.el9 ceph 45 k 2024-04-27T03:55:55.563 INFO:teuthology.orchestra.run.smithi162.stdout: python3-ceph-common x86_64 2:17.2.7-907.g2be3efa9.el9 ceph 118 k 2024-04-27T03:55:55.564 INFO:teuthology.orchestra.run.smithi162.stdout: python3-cephfs x86_64 2:17.2.7-907.g2be3efa9.el9 ceph 156 k 2024-04-27T03:55:55.564 INFO:teuthology.orchestra.run.smithi162.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-04-27T03:55:55.564 INFO:teuthology.orchestra.run.smithi162.stdout: python3-rados x86_64 2:17.2.7-907.g2be3efa9.el9 ceph 320 k 2024-04-27T03:55:55.564 INFO:teuthology.orchestra.run.smithi162.stdout: python3-rbd x86_64 2:17.2.7-907.g2be3efa9.el9 ceph 293 k 2024-04-27T03:55:55.564 INFO:teuthology.orchestra.run.smithi162.stdout: python3-rgw x86_64 2:17.2.7-907.g2be3efa9.el9 ceph 99 k 2024-04-27T03:55:55.564 INFO:teuthology.orchestra.run.smithi162.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-04-27T03:55:55.564 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-04-27T03:55:55.564 INFO:teuthology.orchestra.run.smithi162.stdout:Transaction Summary 2024-04-27T03:55:55.564 INFO:teuthology.orchestra.run.smithi162.stdout:================================================================================ 2024-04-27T03:55:55.564 INFO:teuthology.orchestra.run.smithi162.stdout:Install 22 Packages 2024-04-27T03:55:55.564 INFO:teuthology.orchestra.run.smithi162.stdout:Upgrade 2 Packages 2024-04-27T03:55:55.564 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-04-27T03:55:55.565 INFO:teuthology.orchestra.run.smithi162.stdout:Total download size: 58 M 2024-04-27T03:55:55.565 INFO:teuthology.orchestra.run.smithi162.stdout:Downloading Packages: 2024-04-27T03:55:56.803 INFO:teuthology.orchestra.run.smithi181.stdout:CentOS Stream 9 - CRB 11 MB/s | 6.2 MB 00:00 2024-04-27T03:55:57.482 INFO:teuthology.orchestra.run.smithi162.stdout:(1/24): ceph-radosgw-17.2.7-907.g2be3efa9.el9.x 8.3 MB/s | 14 MB 00:01 2024-04-27T03:55:57.699 INFO:teuthology.orchestra.run.smithi162.stdout:(2/24): ceph-common-17.2.7-907.g2be3efa9.el9.x8 12 MB/s | 22 MB 00:01 2024-04-27T03:55:57.724 INFO:teuthology.orchestra.run.smithi162.stdout:(3/24): ceph-selinux-17.2.7-907.g2be3efa9.el9.x 104 kB/s | 25 kB 00:00 2024-04-27T03:55:57.766 INFO:teuthology.orchestra.run.smithi162.stdout:(4/24): libcephfs2-17.2.7-907.g2be3efa9.el9.x86 9.3 MB/s | 637 kB 00:00 2024-04-27T03:55:57.800 INFO:teuthology.orchestra.run.smithi162.stdout:(5/24): libradosstriper1-17.2.7-907.g2be3efa9.e 5.9 MB/s | 457 kB 00:00 2024-04-27T03:55:57.899 INFO:teuthology.orchestra.run.smithi162.stdout:(6/24): python3-ceph-argparse-17.2.7-907.g2be3e 458 kB/s | 45 kB 00:00 2024-04-27T03:55:57.959 INFO:teuthology.orchestra.run.smithi162.stdout:(7/24): python3-ceph-common-17.2.7-907.g2be3efa 1.9 MB/s | 118 kB 00:00 2024-04-27T03:55:58.042 INFO:teuthology.orchestra.run.smithi162.stdout:(8/24): librgw2-17.2.7-907.g2be3efa9.el9.x86_64 18 MB/s | 4.9 MB 00:00 2024-04-27T03:55:58.067 INFO:teuthology.orchestra.run.smithi162.stdout:(9/24): python3-cephfs-17.2.7-907.g2be3efa9.el9 1.4 MB/s | 156 kB 00:00 2024-04-27T03:55:58.093 INFO:teuthology.orchestra.run.smithi162.stdout:(10/24): python3-rados-17.2.7-907.g2be3efa9.el9 6.2 MB/s | 320 kB 00:00 2024-04-27T03:55:58.119 INFO:teuthology.orchestra.run.smithi162.stdout:(11/24): python3-rbd-17.2.7-907.g2be3efa9.el9.x 5.6 MB/s | 293 kB 00:00 2024-04-27T03:55:58.144 INFO:teuthology.orchestra.run.smithi162.stdout:(12/24): python3-rgw-17.2.7-907.g2be3efa9.el9.x 1.9 MB/s | 99 kB 00:00 2024-04-27T03:55:58.302 INFO:teuthology.orchestra.run.smithi162.stdout:(13/24): librabbitmq-0.11.0-7.el9.x86_64.rpm 286 kB/s | 45 kB 00:00 2024-04-27T03:55:58.352 INFO:teuthology.orchestra.run.smithi162.stdout:(14/24): libpmemobj-1.12.1-1.el9.x86_64.rpm 686 kB/s | 160 kB 00:00 2024-04-27T03:55:58.461 INFO:teuthology.orchestra.run.smithi162.stdout:(15/24): ceph-base-17.2.7-907.g2be3efa9.el9.x86 1.9 MB/s | 5.0 MB 00:02 2024-04-27T03:55:58.570 INFO:teuthology.orchestra.run.smithi162.stdout:(16/24): lttng-ust-2.12.0-6.el9.x86_64.rpm 1.3 MB/s | 292 kB 00:00 2024-04-27T03:55:58.595 INFO:teuthology.orchestra.run.smithi162.stdout:(17/24): python3-prettytable-0.7.2-27.el9.noarc 316 kB/s | 42 kB 00:00 2024-04-27T03:55:58.620 INFO:teuthology.orchestra.run.smithi162.stdout:(18/24): librdkafka-1.6.1-102.el9.x86_64.rpm 2.0 MB/s | 662 kB 00:00 2024-04-27T03:55:58.996 INFO:teuthology.orchestra.run.smithi162.stdout:(19/24): liboath-2.6.7-2.el9.x86_64.rpm 121 kB/s | 49 kB 00:00 2024-04-27T03:55:59.096 INFO:teuthology.orchestra.run.smithi162.stdout:(20/24): libunwind-1.6.2-1.el9.x86_64.rpm 142 kB/s | 67 kB 00:00 2024-04-27T03:55:59.204 INFO:teuthology.orchestra.run.smithi181.stdout:CentOS Stream 9 - Extras packages 13 kB/s | 16 kB 00:01 2024-04-27T03:55:59.529 INFO:teuthology.orchestra.run.smithi162.stdout:(21/24): gperftools-libs-2.9.1-2.el9.x86_64.rpm 322 kB/s | 309 kB 00:00 2024-04-27T03:55:59.929 INFO:teuthology.orchestra.run.smithi162.stdout:(22/24): librbd1-17.2.7-907.g2be3efa9.el9.x86_6 7.3 MB/s | 2.9 MB 00:00 2024-04-27T03:56:00.396 INFO:teuthology.orchestra.run.smithi162.stdout:(23/24): librados2-17.2.7-907.g2be3efa9.el9.x86 2.4 MB/s | 3.1 MB 00:01 2024-04-27T03:56:00.475 INFO:teuthology.orchestra.run.smithi181.stdout:Extra Packages for Enterprise Linux 18 MB/s | 21 MB 00:01 2024-04-27T03:56:01.804 INFO:teuthology.orchestra.run.smithi162.stdout:(24/24): thrift-0.15.0-2.el9.x86_64.rpm 577 kB/s | 1.6 MB 00:02 2024-04-27T03:56:01.807 INFO:teuthology.orchestra.run.smithi162.stdout:-------------------------------------------------------------------------------- 2024-04-27T03:56:01.809 INFO:teuthology.orchestra.run.smithi162.stdout:Total 9.2 MB/s | 58 MB 00:06 2024-04-27T03:56:01.868 INFO:teuthology.orchestra.run.smithi162.stdout:Running transaction check 2024-04-27T03:56:01.900 INFO:teuthology.orchestra.run.smithi162.stdout:Transaction check succeeded. 2024-04-27T03:56:01.900 INFO:teuthology.orchestra.run.smithi162.stdout:Running transaction test 2024-04-27T03:56:02.336 INFO:teuthology.orchestra.run.smithi162.stdout:Transaction test succeeded. 2024-04-27T03:56:02.336 INFO:teuthology.orchestra.run.smithi162.stdout:Running transaction 2024-04-27T03:56:03.619 INFO:teuthology.orchestra.run.smithi162.stdout: Preparing : 1/1 2024-04-27T03:56:03.958 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 1/26 2024-04-27T03:56:04.098 INFO:teuthology.orchestra.run.smithi162.stdout: Upgrading : librados2-2:17.2.7-907.g2be3efa9.el9.x86_64 2/26 2024-04-27T03:56:04.191 INFO:teuthology.orchestra.run.smithi162.stdout: Running scriptlet: librados2-2:17.2.7-907.g2be3efa9.el9.x86_64 2/26 2024-04-27T03:56:04.269 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : python3-rados-2:17.2.7-907.g2be3efa9.el9.x86_64 3/26 2024-04-27T03:56:04.300 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : libcephfs2-2:17.2.7-907.g2be3efa9.el9.x86_64 4/26 2024-04-27T03:56:04.387 INFO:teuthology.orchestra.run.smithi162.stdout: Running scriptlet: libcephfs2-2:17.2.7-907.g2be3efa9.el9.x86_64 4/26 2024-04-27T03:56:04.433 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : thrift-0.15.0-2.el9.x86_64 5/26 2024-04-27T03:56:04.462 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 6/26 2024-04-27T03:56:04.896 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 7/26 2024-04-27T03:56:04.927 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : librgw2-2:17.2.7-907.g2be3efa9.el9.x86_64 8/26 2024-04-27T03:56:04.981 INFO:teuthology.orchestra.run.smithi162.stdout: Running scriptlet: librgw2-2:17.2.7-907.g2be3efa9.el9.x86_64 8/26 2024-04-27T03:56:05.010 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : liboath-2.6.7-2.el9.x86_64 9/26 2024-04-27T03:56:05.053 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : python3-ceph-argparse-2:17.2.7-907.g2be3efa9.el9.x 10/26 2024-04-27T03:56:05.090 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : python3-cephfs-2:17.2.7-907.g2be3efa9.el9.x86_64 11/26 2024-04-27T03:56:05.152 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : python3-rgw-2:17.2.7-907.g2be3efa9.el9.x86_64 12/26 2024-04-27T03:56:05.180 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : libradosstriper1-2:17.2.7-907.g2be3efa9.el9.x86_64 13/26 2024-04-27T03:56:05.245 INFO:teuthology.orchestra.run.smithi162.stdout: Running scriptlet: libradosstriper1-2:17.2.7-907.g2be3efa9.el9.x86_64 13/26 2024-04-27T03:56:05.283 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 14/26 2024-04-27T03:56:05.321 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 15/26 2024-04-27T03:56:05.354 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 16/26 2024-04-27T03:56:05.572 INFO:teuthology.orchestra.run.smithi181.stdout:lab-extras 15 kB/s | 1.7 kB 00:00 2024-04-27T03:56:05.648 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 17/26 2024-04-27T03:56:05.702 INFO:teuthology.orchestra.run.smithi162.stdout: Upgrading : librbd1-2:17.2.7-907.g2be3efa9.el9.x86_64 18/26 2024-04-27T03:56:05.773 INFO:teuthology.orchestra.run.smithi162.stdout: Running scriptlet: librbd1-2:17.2.7-907.g2be3efa9.el9.x86_64 18/26 2024-04-27T03:56:05.846 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : python3-rbd-2:17.2.7-907.g2be3efa9.el9.x86_64 19/26 2024-04-27T03:56:05.877 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : python3-ceph-common-2:17.2.7-907.g2be3efa9.el9.x86 20/26 2024-04-27T03:56:07.766 INFO:teuthology.orchestra.run.smithi181.stdout:Dependencies resolved. 2024-04-27T03:56:07.768 INFO:teuthology.orchestra.run.smithi181.stdout:================================================================================ 2024-04-27T03:56:07.768 INFO:teuthology.orchestra.run.smithi181.stdout: Package Arch Version Repository Size 2024-04-27T03:56:07.768 INFO:teuthology.orchestra.run.smithi181.stdout:================================================================================ 2024-04-27T03:56:07.768 INFO:teuthology.orchestra.run.smithi181.stdout:Installing: 2024-04-27T03:56:07.768 INFO:teuthology.orchestra.run.smithi181.stdout: ceph-radosgw x86_64 2:17.2.7-907.g2be3efa9.el9 ceph 14 M 2024-04-27T03:56:07.768 INFO:teuthology.orchestra.run.smithi181.stdout:Upgrading: 2024-04-27T03:56:07.768 INFO:teuthology.orchestra.run.smithi181.stdout: librados2 x86_64 2:17.2.7-907.g2be3efa9.el9 ceph 3.1 M 2024-04-27T03:56:07.769 INFO:teuthology.orchestra.run.smithi181.stdout: librbd1 x86_64 2:17.2.7-907.g2be3efa9.el9 ceph 2.9 M 2024-04-27T03:56:07.769 INFO:teuthology.orchestra.run.smithi181.stdout:Installing dependencies: 2024-04-27T03:56:07.769 INFO:teuthology.orchestra.run.smithi181.stdout: ceph-base x86_64 2:17.2.7-907.g2be3efa9.el9 ceph 5.0 M 2024-04-27T03:56:07.769 INFO:teuthology.orchestra.run.smithi181.stdout: ceph-common x86_64 2:17.2.7-907.g2be3efa9.el9 ceph 22 M 2024-04-27T03:56:07.769 INFO:teuthology.orchestra.run.smithi181.stdout: ceph-selinux x86_64 2:17.2.7-907.g2be3efa9.el9 ceph 25 k 2024-04-27T03:56:07.769 INFO:teuthology.orchestra.run.smithi181.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-04-27T03:56:07.769 INFO:teuthology.orchestra.run.smithi181.stdout: libcephfs2 x86_64 2:17.2.7-907.g2be3efa9.el9 ceph 637 k 2024-04-27T03:56:07.769 INFO:teuthology.orchestra.run.smithi181.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-04-27T03:56:07.769 INFO:teuthology.orchestra.run.smithi181.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-04-27T03:56:07.769 INFO:teuthology.orchestra.run.smithi181.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-04-27T03:56:07.769 INFO:teuthology.orchestra.run.smithi181.stdout: libradosstriper1 x86_64 2:17.2.7-907.g2be3efa9.el9 ceph 457 k 2024-04-27T03:56:07.769 INFO:teuthology.orchestra.run.smithi181.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-04-27T03:56:07.769 INFO:teuthology.orchestra.run.smithi181.stdout: librgw2 x86_64 2:17.2.7-907.g2be3efa9.el9 ceph 4.9 M 2024-04-27T03:56:07.769 INFO:teuthology.orchestra.run.smithi181.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-04-27T03:56:07.769 INFO:teuthology.orchestra.run.smithi181.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-04-27T03:56:07.770 INFO:teuthology.orchestra.run.smithi181.stdout: python3-ceph-argparse x86_64 2:17.2.7-907.g2be3efa9.el9 ceph 45 k 2024-04-27T03:56:07.770 INFO:teuthology.orchestra.run.smithi181.stdout: python3-ceph-common x86_64 2:17.2.7-907.g2be3efa9.el9 ceph 118 k 2024-04-27T03:56:07.770 INFO:teuthology.orchestra.run.smithi181.stdout: python3-cephfs x86_64 2:17.2.7-907.g2be3efa9.el9 ceph 156 k 2024-04-27T03:56:07.770 INFO:teuthology.orchestra.run.smithi181.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-04-27T03:56:07.770 INFO:teuthology.orchestra.run.smithi181.stdout: python3-rados x86_64 2:17.2.7-907.g2be3efa9.el9 ceph 320 k 2024-04-27T03:56:07.770 INFO:teuthology.orchestra.run.smithi181.stdout: python3-rbd x86_64 2:17.2.7-907.g2be3efa9.el9 ceph 293 k 2024-04-27T03:56:07.770 INFO:teuthology.orchestra.run.smithi181.stdout: python3-rgw x86_64 2:17.2.7-907.g2be3efa9.el9 ceph 99 k 2024-04-27T03:56:07.770 INFO:teuthology.orchestra.run.smithi181.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-04-27T03:56:07.770 INFO:teuthology.orchestra.run.smithi181.stdout: 2024-04-27T03:56:07.770 INFO:teuthology.orchestra.run.smithi181.stdout:Transaction Summary 2024-04-27T03:56:07.770 INFO:teuthology.orchestra.run.smithi181.stdout:================================================================================ 2024-04-27T03:56:07.770 INFO:teuthology.orchestra.run.smithi181.stdout:Install 22 Packages 2024-04-27T03:56:07.770 INFO:teuthology.orchestra.run.smithi181.stdout:Upgrade 2 Packages 2024-04-27T03:56:07.770 INFO:teuthology.orchestra.run.smithi181.stdout: 2024-04-27T03:56:07.771 INFO:teuthology.orchestra.run.smithi181.stdout:Total download size: 58 M 2024-04-27T03:56:07.771 INFO:teuthology.orchestra.run.smithi181.stdout:Downloading Packages: 2024-04-27T03:56:08.254 INFO:teuthology.orchestra.run.smithi162.stdout: Running scriptlet: ceph-common-2:17.2.7-907.g2be3efa9.el9.x86_64 21/26 2024-04-27T03:56:08.351 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : ceph-common-2:17.2.7-907.g2be3efa9.el9.x86_64 21/26 2024-04-27T03:56:08.856 INFO:teuthology.orchestra.run.smithi162.stdout: Running scriptlet: ceph-common-2:17.2.7-907.g2be3efa9.el9.x86_64 21/26 2024-04-27T03:56:08.912 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : ceph-base-2:17.2.7-907.g2be3efa9.el9.x86_64 22/26 2024-04-27T03:56:09.002 INFO:teuthology.orchestra.run.smithi162.stdout: Running scriptlet: ceph-base-2:17.2.7-907.g2be3efa9.el9.x86_64 22/26 2024-04-27T03:56:09.002 INFO:teuthology.orchestra.run.smithi162.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-04-27T03:56:09.002 INFO:teuthology.orchestra.run.smithi162.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-04-27T03:56:09.002 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-04-27T03:56:09.030 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : ceph-selinux-2:17.2.7-907.g2be3efa9.el9.x86_64 23/26 2024-04-27T03:56:09.689 INFO:teuthology.orchestra.run.smithi181.stdout:(1/24): ceph-base-17.2.7-907.g2be3efa9.el9.x86_ 3.3 MB/s | 5.0 MB 00:01 2024-04-27T03:56:09.806 INFO:teuthology.orchestra.run.smithi181.stdout:(2/24): ceph-selinux-17.2.7-907.g2be3efa9.el9.x 216 kB/s | 25 kB 00:00 2024-04-27T03:56:10.023 INFO:teuthology.orchestra.run.smithi181.stdout:(3/24): ceph-radosgw-17.2.7-907.g2be3efa9.el9.x 7.7 MB/s | 14 MB 00:01 2024-04-27T03:56:10.056 INFO:teuthology.orchestra.run.smithi181.stdout:(4/24): libcephfs2-17.2.7-907.g2be3efa9.el9.x86 2.5 MB/s | 637 kB 00:00 2024-04-27T03:56:10.107 INFO:teuthology.orchestra.run.smithi181.stdout:(5/24): libradosstriper1-17.2.7-907.g2be3efa9.e 5.4 MB/s | 457 kB 00:00 2024-04-27T03:56:10.199 INFO:teuthology.orchestra.run.smithi181.stdout:(6/24): python3-ceph-argparse-17.2.7-907.g2be3e 493 kB/s | 45 kB 00:00 2024-04-27T03:56:10.282 INFO:teuthology.orchestra.run.smithi181.stdout:(7/24): python3-ceph-common-17.2.7-907.g2be3efa 1.4 MB/s | 118 kB 00:00 2024-04-27T03:56:10.524 INFO:teuthology.orchestra.run.smithi181.stdout:(8/24): ceph-common-17.2.7-907.g2be3efa9.el9.x8 9.6 MB/s | 22 MB 00:02 2024-04-27T03:56:10.549 INFO:teuthology.orchestra.run.smithi181.stdout:(9/24): python3-cephfs-17.2.7-907.g2be3efa9.el9 585 kB/s | 156 kB 00:00 2024-04-27T03:56:10.591 INFO:teuthology.orchestra.run.smithi181.stdout:(10/24): python3-rados-17.2.7-907.g2be3efa9.el9 4.7 MB/s | 320 kB 00:00 2024-04-27T03:56:10.617 INFO:teuthology.orchestra.run.smithi181.stdout:(11/24): python3-rbd-17.2.7-907.g2be3efa9.el9.x 4.3 MB/s | 293 kB 00:00 2024-04-27T03:56:10.642 INFO:teuthology.orchestra.run.smithi181.stdout:(12/24): python3-rgw-17.2.7-907.g2be3efa9.el9.x 1.9 MB/s | 99 kB 00:00 2024-04-27T03:56:10.725 INFO:teuthology.orchestra.run.smithi181.stdout:(13/24): librgw2-17.2.7-907.g2be3efa9.el9.x86_6 7.3 MB/s | 4.9 MB 00:00 2024-04-27T03:56:10.809 INFO:teuthology.orchestra.run.smithi181.stdout:(14/24): librabbitmq-0.11.0-7.el9.x86_64.rpm 271 kB/s | 45 kB 00:00 2024-04-27T03:56:10.834 INFO:teuthology.orchestra.run.smithi181.stdout:(15/24): libpmemobj-1.12.1-1.el9.x86_64.rpm 737 kB/s | 160 kB 00:00 2024-04-27T03:56:10.927 INFO:teuthology.orchestra.run.smithi181.stdout:(16/24): python3-prettytable-0.7.2-27.el9.noarc 454 kB/s | 42 kB 00:00 2024-04-27T03:56:10.977 INFO:teuthology.orchestra.run.smithi181.stdout:(17/24): lttng-ust-2.12.0-6.el9.x86_64.rpm 1.7 MB/s | 292 kB 00:00 2024-04-27T03:56:11.036 INFO:teuthology.orchestra.run.smithi181.stdout:(18/24): librdkafka-1.6.1-102.el9.x86_64.rpm 2.1 MB/s | 662 kB 00:00 2024-04-27T03:56:11.545 INFO:teuthology.orchestra.run.smithi181.stdout:(19/24): liboath-2.6.7-2.el9.x86_64.rpm 86 kB/s | 49 kB 00:00 2024-04-27T03:56:11.620 INFO:teuthology.orchestra.run.smithi181.stdout:(20/24): libunwind-1.6.2-1.el9.x86_64.rpm 126 kB/s | 67 kB 00:00 2024-04-27T03:56:11.803 INFO:teuthology.orchestra.run.smithi181.stdout:(21/24): gperftools-libs-2.9.1-2.el9.x86_64.rpm 353 kB/s | 309 kB 00:00 2024-04-27T03:56:12.004 INFO:teuthology.orchestra.run.smithi181.stdout:(22/24): librados2-17.2.7-907.g2be3efa9.el9.x86 8.0 MB/s | 3.1 MB 00:00 2024-04-27T03:56:12.104 INFO:teuthology.orchestra.run.smithi181.stdout:(23/24): librbd1-17.2.7-907.g2be3efa9.el9.x86_6 9.8 MB/s | 2.9 MB 00:00 2024-04-27T03:56:13.512 INFO:teuthology.orchestra.run.smithi181.stdout:(24/24): thrift-0.15.0-2.el9.x86_64.rpm 823 kB/s | 1.6 MB 00:01 2024-04-27T03:56:13.517 INFO:teuthology.orchestra.run.smithi181.stdout:-------------------------------------------------------------------------------- 2024-04-27T03:56:13.518 INFO:teuthology.orchestra.run.smithi181.stdout:Total 10 MB/s | 58 MB 00:05 2024-04-27T03:56:13.577 INFO:teuthology.orchestra.run.smithi181.stdout:Running transaction check 2024-04-27T03:56:13.611 INFO:teuthology.orchestra.run.smithi181.stdout:Transaction check succeeded. 2024-04-27T03:56:13.611 INFO:teuthology.orchestra.run.smithi181.stdout:Running transaction test 2024-04-27T03:56:14.047 INFO:teuthology.orchestra.run.smithi181.stdout:Transaction test succeeded. 2024-04-27T03:56:14.048 INFO:teuthology.orchestra.run.smithi181.stdout:Running transaction 2024-04-27T03:56:15.305 INFO:teuthology.orchestra.run.smithi181.stdout: Preparing : 1/1 2024-04-27T03:56:15.694 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 1/26 2024-04-27T03:56:15.756 INFO:teuthology.orchestra.run.smithi181.stdout: Upgrading : librados2-2:17.2.7-907.g2be3efa9.el9.x86_64 2/26 2024-04-27T03:56:15.847 INFO:teuthology.orchestra.run.smithi181.stdout: Running scriptlet: librados2-2:17.2.7-907.g2be3efa9.el9.x86_64 2/26 2024-04-27T03:56:15.926 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-rados-2:17.2.7-907.g2be3efa9.el9.x86_64 3/26 2024-04-27T03:56:15.949 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : libcephfs2-2:17.2.7-907.g2be3efa9.el9.x86_64 4/26 2024-04-27T03:56:16.027 INFO:teuthology.orchestra.run.smithi181.stdout: Running scriptlet: libcephfs2-2:17.2.7-907.g2be3efa9.el9.x86_64 4/26 2024-04-27T03:56:16.074 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : thrift-0.15.0-2.el9.x86_64 5/26 2024-04-27T03:56:16.103 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 6/26 2024-04-27T03:56:16.538 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 7/26 2024-04-27T03:56:16.562 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : librgw2-2:17.2.7-907.g2be3efa9.el9.x86_64 8/26 2024-04-27T03:56:16.624 INFO:teuthology.orchestra.run.smithi181.stdout: Running scriptlet: librgw2-2:17.2.7-907.g2be3efa9.el9.x86_64 8/26 2024-04-27T03:56:16.653 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : liboath-2.6.7-2.el9.x86_64 9/26 2024-04-27T03:56:16.696 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-ceph-argparse-2:17.2.7-907.g2be3efa9.el9.x 10/26 2024-04-27T03:56:16.750 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-cephfs-2:17.2.7-907.g2be3efa9.el9.x86_64 11/26 2024-04-27T03:56:16.812 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-rgw-2:17.2.7-907.g2be3efa9.el9.x86_64 12/26 2024-04-27T03:56:16.839 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : libradosstriper1-2:17.2.7-907.g2be3efa9.el9.x86_64 13/26 2024-04-27T03:56:16.904 INFO:teuthology.orchestra.run.smithi181.stdout: Running scriptlet: libradosstriper1-2:17.2.7-907.g2be3efa9.el9.x86_64 13/26 2024-04-27T03:56:16.943 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 14/26 2024-04-27T03:56:16.973 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 15/26 2024-04-27T03:56:17.005 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 16/26 2024-04-27T03:56:17.299 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 17/26 2024-04-27T03:56:17.335 INFO:teuthology.orchestra.run.smithi181.stdout: Upgrading : librbd1-2:17.2.7-907.g2be3efa9.el9.x86_64 18/26 2024-04-27T03:56:17.425 INFO:teuthology.orchestra.run.smithi181.stdout: Running scriptlet: librbd1-2:17.2.7-907.g2be3efa9.el9.x86_64 18/26 2024-04-27T03:56:17.497 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-rbd-2:17.2.7-907.g2be3efa9.el9.x86_64 19/26 2024-04-27T03:56:17.521 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-ceph-common-2:17.2.7-907.g2be3efa9.el9.x86 20/26 2024-04-27T03:56:18.589 INFO:teuthology.orchestra.run.smithi162.stdout: Running scriptlet: ceph-selinux-2:17.2.7-907.g2be3efa9.el9.x86_64 23/26 2024-04-27T03:56:18.589 INFO:teuthology.orchestra.run.smithi162.stdout:skipping the directory /sys 2024-04-27T03:56:18.589 INFO:teuthology.orchestra.run.smithi162.stdout:skipping the directory /proc 2024-04-27T03:56:18.589 INFO:teuthology.orchestra.run.smithi162.stdout:skipping the directory /mnt 2024-04-27T03:56:18.590 INFO:teuthology.orchestra.run.smithi162.stdout:skipping the directory /var/tmp 2024-04-27T03:56:18.590 INFO:teuthology.orchestra.run.smithi162.stdout:skipping the directory /home 2024-04-27T03:56:18.590 INFO:teuthology.orchestra.run.smithi162.stdout:skipping the directory /root 2024-04-27T03:56:18.590 INFO:teuthology.orchestra.run.smithi162.stdout:skipping the directory /tmp 2024-04-27T03:56:18.590 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-04-27T03:56:19.887 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : ceph-radosgw-2:17.2.7-907.g2be3efa9.el9.x86_64 24/26 2024-04-27T03:56:19.926 INFO:teuthology.orchestra.run.smithi181.stdout: Running scriptlet: ceph-common-2:17.2.7-907.g2be3efa9.el9.x86_64 21/26 2024-04-27T03:56:19.964 INFO:teuthology.orchestra.run.smithi162.stdout: Running scriptlet: ceph-radosgw-2:17.2.7-907.g2be3efa9.el9.x86_64 24/26 2024-04-27T03:56:19.964 INFO:teuthology.orchestra.run.smithi162.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-27T03:56:19.965 INFO:teuthology.orchestra.run.smithi162.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-04-27T03:56:19.965 INFO:teuthology.orchestra.run.smithi162.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-27T03:56:19.965 INFO:teuthology.orchestra.run.smithi162.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-27T03:56:19.965 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-04-27T03:56:19.966 INFO:teuthology.orchestra.run.smithi162.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 25/26 2024-04-27T03:56:20.025 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : ceph-common-2:17.2.7-907.g2be3efa9.el9.x86_64 21/26 2024-04-27T03:56:20.050 INFO:teuthology.orchestra.run.smithi162.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 25/26 2024-04-27T03:56:20.050 INFO:teuthology.orchestra.run.smithi162.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 26/26 2024-04-27T03:56:20.532 INFO:teuthology.orchestra.run.smithi181.stdout: Running scriptlet: ceph-common-2:17.2.7-907.g2be3efa9.el9.x86_64 21/26 2024-04-27T03:56:20.570 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : ceph-base-2:17.2.7-907.g2be3efa9.el9.x86_64 22/26 2024-04-27T03:56:20.686 INFO:teuthology.orchestra.run.smithi181.stdout: Running scriptlet: ceph-base-2:17.2.7-907.g2be3efa9.el9.x86_64 22/26 2024-04-27T03:56:20.686 INFO:teuthology.orchestra.run.smithi181.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-04-27T03:56:20.686 INFO:teuthology.orchestra.run.smithi181.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-04-27T03:56:20.686 INFO:teuthology.orchestra.run.smithi181.stdout: 2024-04-27T03:56:20.713 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : ceph-selinux-2:17.2.7-907.g2be3efa9.el9.x86_64 23/26 2024-04-27T03:56:21.282 INFO:teuthology.orchestra.run.smithi162.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 26/26 2024-04-27T03:56:21.282 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : ceph-base-2:17.2.7-907.g2be3efa9.el9.x86_64 1/26 2024-04-27T03:56:21.282 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : ceph-common-2:17.2.7-907.g2be3efa9.el9.x86_64 2/26 2024-04-27T03:56:21.282 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : ceph-radosgw-2:17.2.7-907.g2be3efa9.el9.x86_64 3/26 2024-04-27T03:56:21.282 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : ceph-selinux-2:17.2.7-907.g2be3efa9.el9.x86_64 4/26 2024-04-27T03:56:21.282 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : libcephfs2-2:17.2.7-907.g2be3efa9.el9.x86_64 5/26 2024-04-27T03:56:21.282 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : libradosstriper1-2:17.2.7-907.g2be3efa9.el9.x86_64 6/26 2024-04-27T03:56:21.283 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : librgw2-2:17.2.7-907.g2be3efa9.el9.x86_64 7/26 2024-04-27T03:56:21.283 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : python3-ceph-argparse-2:17.2.7-907.g2be3efa9.el9.x 8/26 2024-04-27T03:56:21.283 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : python3-ceph-common-2:17.2.7-907.g2be3efa9.el9.x86 9/26 2024-04-27T03:56:21.283 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : python3-cephfs-2:17.2.7-907.g2be3efa9.el9.x86_64 10/26 2024-04-27T03:56:21.283 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : python3-rados-2:17.2.7-907.g2be3efa9.el9.x86_64 11/26 2024-04-27T03:56:21.283 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : python3-rbd-2:17.2.7-907.g2be3efa9.el9.x86_64 12/26 2024-04-27T03:56:21.283 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : python3-rgw-2:17.2.7-907.g2be3efa9.el9.x86_64 13/26 2024-04-27T03:56:21.283 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 14/26 2024-04-27T03:56:21.283 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 15/26 2024-04-27T03:56:21.283 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 16/26 2024-04-27T03:56:21.283 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 17/26 2024-04-27T03:56:21.283 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 18/26 2024-04-27T03:56:21.283 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 19/26 2024-04-27T03:56:21.284 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 20/26 2024-04-27T03:56:21.284 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 21/26 2024-04-27T03:56:21.284 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 22/26 2024-04-27T03:56:21.284 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : librados2-2:17.2.7-907.g2be3efa9.el9.x86_64 23/26 2024-04-27T03:56:21.284 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 24/26 2024-04-27T03:56:21.284 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : librbd1-2:17.2.7-907.g2be3efa9.el9.x86_64 25/26 2024-04-27T03:56:21.984 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 26/26 2024-04-27T03:56:21.984 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-04-27T03:56:21.984 INFO:teuthology.orchestra.run.smithi162.stdout:Upgraded: 2024-04-27T03:56:21.985 INFO:teuthology.orchestra.run.smithi162.stdout: librados2-2:17.2.7-907.g2be3efa9.el9.x86_64 2024-04-27T03:56:21.985 INFO:teuthology.orchestra.run.smithi162.stdout: librbd1-2:17.2.7-907.g2be3efa9.el9.x86_64 2024-04-27T03:56:21.985 INFO:teuthology.orchestra.run.smithi162.stdout:Installed: 2024-04-27T03:56:21.985 INFO:teuthology.orchestra.run.smithi162.stdout: ceph-base-2:17.2.7-907.g2be3efa9.el9.x86_64 2024-04-27T03:56:21.985 INFO:teuthology.orchestra.run.smithi162.stdout: ceph-common-2:17.2.7-907.g2be3efa9.el9.x86_64 2024-04-27T03:56:21.985 INFO:teuthology.orchestra.run.smithi162.stdout: ceph-radosgw-2:17.2.7-907.g2be3efa9.el9.x86_64 2024-04-27T03:56:21.985 INFO:teuthology.orchestra.run.smithi162.stdout: ceph-selinux-2:17.2.7-907.g2be3efa9.el9.x86_64 2024-04-27T03:56:21.986 INFO:teuthology.orchestra.run.smithi162.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-04-27T03:56:21.986 INFO:teuthology.orchestra.run.smithi162.stdout: libcephfs2-2:17.2.7-907.g2be3efa9.el9.x86_64 2024-04-27T03:56:21.986 INFO:teuthology.orchestra.run.smithi162.stdout: liboath-2.6.7-2.el9.x86_64 2024-04-27T03:56:21.986 INFO:teuthology.orchestra.run.smithi162.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-04-27T03:56:21.986 INFO:teuthology.orchestra.run.smithi162.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-04-27T03:56:21.986 INFO:teuthology.orchestra.run.smithi162.stdout: libradosstriper1-2:17.2.7-907.g2be3efa9.el9.x86_64 2024-04-27T03:56:21.987 INFO:teuthology.orchestra.run.smithi162.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-04-27T03:56:21.987 INFO:teuthology.orchestra.run.smithi162.stdout: librgw2-2:17.2.7-907.g2be3efa9.el9.x86_64 2024-04-27T03:56:21.987 INFO:teuthology.orchestra.run.smithi162.stdout: libunwind-1.6.2-1.el9.x86_64 2024-04-27T03:56:21.987 INFO:teuthology.orchestra.run.smithi162.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-04-27T03:56:21.987 INFO:teuthology.orchestra.run.smithi162.stdout: python3-ceph-argparse-2:17.2.7-907.g2be3efa9.el9.x86_64 2024-04-27T03:56:21.987 INFO:teuthology.orchestra.run.smithi162.stdout: python3-ceph-common-2:17.2.7-907.g2be3efa9.el9.x86_64 2024-04-27T03:56:21.988 INFO:teuthology.orchestra.run.smithi162.stdout: python3-cephfs-2:17.2.7-907.g2be3efa9.el9.x86_64 2024-04-27T03:56:21.988 INFO:teuthology.orchestra.run.smithi162.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-04-27T03:56:21.988 INFO:teuthology.orchestra.run.smithi162.stdout: python3-rados-2:17.2.7-907.g2be3efa9.el9.x86_64 2024-04-27T03:56:21.988 INFO:teuthology.orchestra.run.smithi162.stdout: python3-rbd-2:17.2.7-907.g2be3efa9.el9.x86_64 2024-04-27T03:56:21.988 INFO:teuthology.orchestra.run.smithi162.stdout: python3-rgw-2:17.2.7-907.g2be3efa9.el9.x86_64 2024-04-27T03:56:21.988 INFO:teuthology.orchestra.run.smithi162.stdout: thrift-0.15.0-2.el9.x86_64 2024-04-27T03:56:21.989 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-04-27T03:56:21.989 INFO:teuthology.orchestra.run.smithi162.stdout:Complete! 2024-04-27T03:56:22.334 DEBUG:teuthology.orchestra.run.smithi162:> sudo yum -y install ceph-test 2024-04-27T03:56:22.867 INFO:teuthology.orchestra.run.smithi162.stdout:Last metadata expiration check: 0:00:29 ago on Sat 27 Apr 2024 03:55:53 AM UTC. 2024-04-27T03:56:22.988 INFO:teuthology.orchestra.run.smithi162.stdout:Dependencies resolved. 2024-04-27T03:56:22.989 INFO:teuthology.orchestra.run.smithi162.stdout:================================================================================ 2024-04-27T03:56:22.989 INFO:teuthology.orchestra.run.smithi162.stdout: Package Arch Version Repository Size 2024-04-27T03:56:22.989 INFO:teuthology.orchestra.run.smithi162.stdout:================================================================================ 2024-04-27T03:56:22.990 INFO:teuthology.orchestra.run.smithi162.stdout:Installing: 2024-04-27T03:56:22.990 INFO:teuthology.orchestra.run.smithi162.stdout: ceph-test x86_64 2:17.2.7-907.g2be3efa9.el9 ceph 48 M 2024-04-27T03:56:22.990 INFO:teuthology.orchestra.run.smithi162.stdout:Installing dependencies: 2024-04-27T03:56:22.990 INFO:teuthology.orchestra.run.smithi162.stdout: libcephsqlite x86_64 2:17.2.7-907.g2be3efa9.el9 ceph 162 k 2024-04-27T03:56:22.990 INFO:teuthology.orchestra.run.smithi162.stdout: socat x86_64 1.7.4.1-5.el9 appstream 305 k 2024-04-27T03:56:22.990 INFO:teuthology.orchestra.run.smithi162.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-04-27T03:56:22.991 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-04-27T03:56:22.991 INFO:teuthology.orchestra.run.smithi162.stdout:Transaction Summary 2024-04-27T03:56:22.991 INFO:teuthology.orchestra.run.smithi162.stdout:================================================================================ 2024-04-27T03:56:22.991 INFO:teuthology.orchestra.run.smithi162.stdout:Install 4 Packages 2024-04-27T03:56:22.991 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-04-27T03:56:22.992 INFO:teuthology.orchestra.run.smithi162.stdout:Total download size: 48 M 2024-04-27T03:56:22.993 INFO:teuthology.orchestra.run.smithi162.stdout:Installed size: 192 M 2024-04-27T03:56:22.993 INFO:teuthology.orchestra.run.smithi162.stdout:Downloading Packages: 2024-04-27T03:56:23.573 INFO:teuthology.orchestra.run.smithi162.stdout:(1/4): socat-1.7.4.1-5.el9.x86_64.rpm 987 kB/s | 305 kB 00:00 2024-04-27T03:56:23.648 INFO:teuthology.orchestra.run.smithi162.stdout:(2/4): libcephsqlite-17.2.7-907.g2be3efa9.el9.x 423 kB/s | 162 kB 00:00 2024-04-27T03:56:23.679 INFO:teuthology.orchestra.run.smithi162.stdout:(3/4): xmlstarlet-1.6.1-20.el9.x86_64.rpm 604 kB/s | 64 kB 00:00 2024-04-27T03:56:26.866 INFO:teuthology.orchestra.run.smithi162.stdout:(4/4): ceph-test-17.2.7-907.g2be3efa9.el9.x86_6 13 MB/s | 48 MB 00:03 2024-04-27T03:56:26.868 INFO:teuthology.orchestra.run.smithi162.stdout:-------------------------------------------------------------------------------- 2024-04-27T03:56:26.869 INFO:teuthology.orchestra.run.smithi162.stdout:Total 12 MB/s | 48 MB 00:03 2024-04-27T03:56:26.896 INFO:teuthology.orchestra.run.smithi162.stdout:Running transaction check 2024-04-27T03:56:26.915 INFO:teuthology.orchestra.run.smithi162.stdout:Transaction check succeeded. 2024-04-27T03:56:26.915 INFO:teuthology.orchestra.run.smithi162.stdout:Running transaction test 2024-04-27T03:56:27.278 INFO:teuthology.orchestra.run.smithi162.stdout:Transaction test succeeded. 2024-04-27T03:56:27.278 INFO:teuthology.orchestra.run.smithi162.stdout:Running transaction 2024-04-27T03:56:27.797 INFO:teuthology.orchestra.run.smithi162.stdout: Preparing : 1/1 2024-04-27T03:56:27.875 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 1/4 2024-04-27T03:56:27.915 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 2/4 2024-04-27T03:56:27.943 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : libcephsqlite-2:17.2.7-907.g2be3efa9.el9.x86_64 3/4 2024-04-27T03:56:29.887 INFO:teuthology.orchestra.run.smithi181.stdout: Running scriptlet: ceph-selinux-2:17.2.7-907.g2be3efa9.el9.x86_64 23/26 2024-04-27T03:56:29.887 INFO:teuthology.orchestra.run.smithi181.stdout:skipping the directory /sys 2024-04-27T03:56:29.887 INFO:teuthology.orchestra.run.smithi181.stdout:skipping the directory /proc 2024-04-27T03:56:29.887 INFO:teuthology.orchestra.run.smithi181.stdout:skipping the directory /mnt 2024-04-27T03:56:29.887 INFO:teuthology.orchestra.run.smithi181.stdout:skipping the directory /var/tmp 2024-04-27T03:56:29.887 INFO:teuthology.orchestra.run.smithi181.stdout:skipping the directory /home 2024-04-27T03:56:29.887 INFO:teuthology.orchestra.run.smithi181.stdout:skipping the directory /root 2024-04-27T03:56:29.888 INFO:teuthology.orchestra.run.smithi181.stdout:skipping the directory /tmp 2024-04-27T03:56:29.888 INFO:teuthology.orchestra.run.smithi181.stdout: 2024-04-27T03:56:31.136 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : ceph-radosgw-2:17.2.7-907.g2be3efa9.el9.x86_64 24/26 2024-04-27T03:56:31.205 INFO:teuthology.orchestra.run.smithi181.stdout: Running scriptlet: ceph-radosgw-2:17.2.7-907.g2be3efa9.el9.x86_64 24/26 2024-04-27T03:56:31.205 INFO:teuthology.orchestra.run.smithi181.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-27T03:56:31.205 INFO:teuthology.orchestra.run.smithi181.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-04-27T03:56:31.206 INFO:teuthology.orchestra.run.smithi181.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-27T03:56:31.206 INFO:teuthology.orchestra.run.smithi181.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-27T03:56:31.206 INFO:teuthology.orchestra.run.smithi181.stdout: 2024-04-27T03:56:31.207 INFO:teuthology.orchestra.run.smithi181.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 25/26 2024-04-27T03:56:31.288 INFO:teuthology.orchestra.run.smithi181.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 25/26 2024-04-27T03:56:31.289 INFO:teuthology.orchestra.run.smithi181.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 26/26 2024-04-27T03:56:32.316 INFO:teuthology.orchestra.run.smithi162.stdout: Running scriptlet: libcephsqlite-2:17.2.7-907.g2be3efa9.el9.x86_64 3/4 2024-04-27T03:56:32.455 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : ceph-test-2:17.2.7-907.g2be3efa9.el9.x86_64 4/4 2024-04-27T03:56:32.670 INFO:teuthology.orchestra.run.smithi181.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 26/26 2024-04-27T03:56:32.670 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : ceph-base-2:17.2.7-907.g2be3efa9.el9.x86_64 1/26 2024-04-27T03:56:32.670 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : ceph-common-2:17.2.7-907.g2be3efa9.el9.x86_64 2/26 2024-04-27T03:56:32.670 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : ceph-radosgw-2:17.2.7-907.g2be3efa9.el9.x86_64 3/26 2024-04-27T03:56:32.670 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : ceph-selinux-2:17.2.7-907.g2be3efa9.el9.x86_64 4/26 2024-04-27T03:56:32.670 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : libcephfs2-2:17.2.7-907.g2be3efa9.el9.x86_64 5/26 2024-04-27T03:56:32.670 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : libradosstriper1-2:17.2.7-907.g2be3efa9.el9.x86_64 6/26 2024-04-27T03:56:32.670 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : librgw2-2:17.2.7-907.g2be3efa9.el9.x86_64 7/26 2024-04-27T03:56:32.671 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-ceph-argparse-2:17.2.7-907.g2be3efa9.el9.x 8/26 2024-04-27T03:56:32.671 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-ceph-common-2:17.2.7-907.g2be3efa9.el9.x86 9/26 2024-04-27T03:56:32.671 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-cephfs-2:17.2.7-907.g2be3efa9.el9.x86_64 10/26 2024-04-27T03:56:32.671 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-rados-2:17.2.7-907.g2be3efa9.el9.x86_64 11/26 2024-04-27T03:56:32.671 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-rbd-2:17.2.7-907.g2be3efa9.el9.x86_64 12/26 2024-04-27T03:56:32.671 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-rgw-2:17.2.7-907.g2be3efa9.el9.x86_64 13/26 2024-04-27T03:56:32.671 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 14/26 2024-04-27T03:56:32.671 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 15/26 2024-04-27T03:56:32.671 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 16/26 2024-04-27T03:56:32.671 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 17/26 2024-04-27T03:56:32.671 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 18/26 2024-04-27T03:56:32.671 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 19/26 2024-04-27T03:56:32.671 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 20/26 2024-04-27T03:56:32.671 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 21/26 2024-04-27T03:56:32.672 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 22/26 2024-04-27T03:56:32.672 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : librados2-2:17.2.7-907.g2be3efa9.el9.x86_64 23/26 2024-04-27T03:56:32.672 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 24/26 2024-04-27T03:56:32.672 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : librbd1-2:17.2.7-907.g2be3efa9.el9.x86_64 25/26 2024-04-27T03:56:33.356 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 26/26 2024-04-27T03:56:33.356 INFO:teuthology.orchestra.run.smithi181.stdout: 2024-04-27T03:56:33.356 INFO:teuthology.orchestra.run.smithi181.stdout:Upgraded: 2024-04-27T03:56:33.356 INFO:teuthology.orchestra.run.smithi181.stdout: librados2-2:17.2.7-907.g2be3efa9.el9.x86_64 2024-04-27T03:56:33.356 INFO:teuthology.orchestra.run.smithi181.stdout: librbd1-2:17.2.7-907.g2be3efa9.el9.x86_64 2024-04-27T03:56:33.356 INFO:teuthology.orchestra.run.smithi181.stdout:Installed: 2024-04-27T03:56:33.356 INFO:teuthology.orchestra.run.smithi181.stdout: ceph-base-2:17.2.7-907.g2be3efa9.el9.x86_64 2024-04-27T03:56:33.356 INFO:teuthology.orchestra.run.smithi181.stdout: ceph-common-2:17.2.7-907.g2be3efa9.el9.x86_64 2024-04-27T03:56:33.356 INFO:teuthology.orchestra.run.smithi181.stdout: ceph-radosgw-2:17.2.7-907.g2be3efa9.el9.x86_64 2024-04-27T03:56:33.356 INFO:teuthology.orchestra.run.smithi181.stdout: ceph-selinux-2:17.2.7-907.g2be3efa9.el9.x86_64 2024-04-27T03:56:33.357 INFO:teuthology.orchestra.run.smithi181.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-04-27T03:56:33.357 INFO:teuthology.orchestra.run.smithi181.stdout: libcephfs2-2:17.2.7-907.g2be3efa9.el9.x86_64 2024-04-27T03:56:33.357 INFO:teuthology.orchestra.run.smithi181.stdout: liboath-2.6.7-2.el9.x86_64 2024-04-27T03:56:33.357 INFO:teuthology.orchestra.run.smithi181.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-04-27T03:56:33.357 INFO:teuthology.orchestra.run.smithi181.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-04-27T03:56:33.357 INFO:teuthology.orchestra.run.smithi181.stdout: libradosstriper1-2:17.2.7-907.g2be3efa9.el9.x86_64 2024-04-27T03:56:33.357 INFO:teuthology.orchestra.run.smithi181.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-04-27T03:56:33.357 INFO:teuthology.orchestra.run.smithi181.stdout: librgw2-2:17.2.7-907.g2be3efa9.el9.x86_64 2024-04-27T03:56:33.357 INFO:teuthology.orchestra.run.smithi181.stdout: libunwind-1.6.2-1.el9.x86_64 2024-04-27T03:56:33.357 INFO:teuthology.orchestra.run.smithi181.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-04-27T03:56:33.357 INFO:teuthology.orchestra.run.smithi181.stdout: python3-ceph-argparse-2:17.2.7-907.g2be3efa9.el9.x86_64 2024-04-27T03:56:33.357 INFO:teuthology.orchestra.run.smithi181.stdout: python3-ceph-common-2:17.2.7-907.g2be3efa9.el9.x86_64 2024-04-27T03:56:33.357 INFO:teuthology.orchestra.run.smithi181.stdout: python3-cephfs-2:17.2.7-907.g2be3efa9.el9.x86_64 2024-04-27T03:56:33.357 INFO:teuthology.orchestra.run.smithi181.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-04-27T03:56:33.357 INFO:teuthology.orchestra.run.smithi181.stdout: python3-rados-2:17.2.7-907.g2be3efa9.el9.x86_64 2024-04-27T03:56:33.358 INFO:teuthology.orchestra.run.smithi181.stdout: python3-rbd-2:17.2.7-907.g2be3efa9.el9.x86_64 2024-04-27T03:56:33.358 INFO:teuthology.orchestra.run.smithi181.stdout: python3-rgw-2:17.2.7-907.g2be3efa9.el9.x86_64 2024-04-27T03:56:33.358 INFO:teuthology.orchestra.run.smithi181.stdout: thrift-0.15.0-2.el9.x86_64 2024-04-27T03:56:33.358 INFO:teuthology.orchestra.run.smithi181.stdout: 2024-04-27T03:56:33.358 INFO:teuthology.orchestra.run.smithi181.stdout:Complete! 2024-04-27T03:56:33.641 DEBUG:teuthology.orchestra.run.smithi181:> sudo yum -y install ceph-test 2024-04-27T03:56:34.182 INFO:teuthology.orchestra.run.smithi181.stdout:Last metadata expiration check: 0:00:29 ago on Sat 27 Apr 2024 03:56:05 AM UTC. 2024-04-27T03:56:34.301 INFO:teuthology.orchestra.run.smithi181.stdout:Dependencies resolved. 2024-04-27T03:56:34.302 INFO:teuthology.orchestra.run.smithi181.stdout:================================================================================ 2024-04-27T03:56:34.302 INFO:teuthology.orchestra.run.smithi181.stdout: Package Arch Version Repository Size 2024-04-27T03:56:34.302 INFO:teuthology.orchestra.run.smithi181.stdout:================================================================================ 2024-04-27T03:56:34.302 INFO:teuthology.orchestra.run.smithi181.stdout:Installing: 2024-04-27T03:56:34.302 INFO:teuthology.orchestra.run.smithi181.stdout: ceph-test x86_64 2:17.2.7-907.g2be3efa9.el9 ceph 48 M 2024-04-27T03:56:34.302 INFO:teuthology.orchestra.run.smithi181.stdout:Installing dependencies: 2024-04-27T03:56:34.302 INFO:teuthology.orchestra.run.smithi181.stdout: libcephsqlite x86_64 2:17.2.7-907.g2be3efa9.el9 ceph 162 k 2024-04-27T03:56:34.302 INFO:teuthology.orchestra.run.smithi181.stdout: socat x86_64 1.7.4.1-5.el9 appstream 305 k 2024-04-27T03:56:34.302 INFO:teuthology.orchestra.run.smithi181.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-04-27T03:56:34.302 INFO:teuthology.orchestra.run.smithi181.stdout: 2024-04-27T03:56:34.302 INFO:teuthology.orchestra.run.smithi181.stdout:Transaction Summary 2024-04-27T03:56:34.302 INFO:teuthology.orchestra.run.smithi181.stdout:================================================================================ 2024-04-27T03:56:34.303 INFO:teuthology.orchestra.run.smithi181.stdout:Install 4 Packages 2024-04-27T03:56:34.303 INFO:teuthology.orchestra.run.smithi181.stdout: 2024-04-27T03:56:34.303 INFO:teuthology.orchestra.run.smithi181.stdout:Total download size: 48 M 2024-04-27T03:56:34.303 INFO:teuthology.orchestra.run.smithi181.stdout:Installed size: 192 M 2024-04-27T03:56:34.303 INFO:teuthology.orchestra.run.smithi181.stdout:Downloading Packages: 2024-04-27T03:56:34.606 INFO:teuthology.orchestra.run.smithi162.stdout: Running scriptlet: ceph-test-2:17.2.7-907.g2be3efa9.el9.x86_64 4/4 2024-04-27T03:56:34.606 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : ceph-test-2:17.2.7-907.g2be3efa9.el9.x86_64 1/4 2024-04-27T03:56:34.606 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : libcephsqlite-2:17.2.7-907.g2be3efa9.el9.x86_64 2/4 2024-04-27T03:56:34.606 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 3/4 2024-04-27T03:56:34.776 INFO:teuthology.orchestra.run.smithi181.stdout:(1/4): socat-1.7.4.1-5.el9.x86_64.rpm 1.6 MB/s | 305 kB 00:00 2024-04-27T03:56:34.842 INFO:teuthology.orchestra.run.smithi181.stdout:(2/4): xmlstarlet-1.6.1-20.el9.x86_64.rpm 990 kB/s | 64 kB 00:00 2024-04-27T03:56:34.869 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 4/4 2024-04-27T03:56:34.869 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-04-27T03:56:34.869 INFO:teuthology.orchestra.run.smithi162.stdout:Installed: 2024-04-27T03:56:34.869 INFO:teuthology.orchestra.run.smithi162.stdout: ceph-test-2:17.2.7-907.g2be3efa9.el9.x86_64 2024-04-27T03:56:34.870 INFO:teuthology.orchestra.run.smithi162.stdout: libcephsqlite-2:17.2.7-907.g2be3efa9.el9.x86_64 2024-04-27T03:56:34.870 INFO:teuthology.orchestra.run.smithi162.stdout: socat-1.7.4.1-5.el9.x86_64 2024-04-27T03:56:34.870 INFO:teuthology.orchestra.run.smithi162.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-04-27T03:56:34.870 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-04-27T03:56:34.870 INFO:teuthology.orchestra.run.smithi162.stdout:Complete! 2024-04-27T03:56:35.144 DEBUG:teuthology.orchestra.run.smithi162:> sudo yum -y install ceph 2024-04-27T03:56:35.279 INFO:teuthology.orchestra.run.smithi181.stdout:(3/4): libcephsqlite-17.2.7-907.g2be3efa9.el9.x 234 kB/s | 162 kB 00:00 2024-04-27T03:56:35.649 INFO:teuthology.orchestra.run.smithi162.stdout:Last metadata expiration check: 0:00:42 ago on Sat 27 Apr 2024 03:55:53 AM UTC. 2024-04-27T03:56:35.775 INFO:teuthology.orchestra.run.smithi162.stdout:Dependencies resolved. 2024-04-27T03:56:35.778 INFO:teuthology.orchestra.run.smithi162.stdout:================================================================================ 2024-04-27T03:56:35.778 INFO:teuthology.orchestra.run.smithi162.stdout: Package Arch Version Repository Size 2024-04-27T03:56:35.778 INFO:teuthology.orchestra.run.smithi162.stdout:================================================================================ 2024-04-27T03:56:35.778 INFO:teuthology.orchestra.run.smithi162.stdout:Installing: 2024-04-27T03:56:35.778 INFO:teuthology.orchestra.run.smithi162.stdout: ceph x86_64 2:17.2.7-907.g2be3efa9.el9 ceph 6.5 k 2024-04-27T03:56:35.778 INFO:teuthology.orchestra.run.smithi162.stdout:Installing dependencies: 2024-04-27T03:56:35.778 INFO:teuthology.orchestra.run.smithi162.stdout: ceph-mds x86_64 2:17.2.7-907.g2be3efa9.el9 ceph 2.1 M 2024-04-27T03:56:35.778 INFO:teuthology.orchestra.run.smithi162.stdout: ceph-mgr x86_64 2:17.2.7-907.g2be3efa9.el9 ceph 1.4 M 2024-04-27T03:56:35.778 INFO:teuthology.orchestra.run.smithi162.stdout: ceph-mgr-modules-core noarch 2:17.2.7-907.g2be3efa9.el9 ceph-noarch 240 k 2024-04-27T03:56:35.778 INFO:teuthology.orchestra.run.smithi162.stdout: ceph-mon x86_64 2:17.2.7-907.g2be3efa9.el9 ceph 3.9 M 2024-04-27T03:56:35.779 INFO:teuthology.orchestra.run.smithi162.stdout: ceph-osd x86_64 2:17.2.7-907.g2be3efa9.el9 ceph 15 M 2024-04-27T03:56:35.779 INFO:teuthology.orchestra.run.smithi162.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-04-27T03:56:35.779 INFO:teuthology.orchestra.run.smithi162.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-04-27T03:56:35.779 INFO:teuthology.orchestra.run.smithi162.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-04-27T03:56:35.779 INFO:teuthology.orchestra.run.smithi162.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-04-27T03:56:35.779 INFO:teuthology.orchestra.run.smithi162.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-04-27T03:56:35.779 INFO:teuthology.orchestra.run.smithi162.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-04-27T03:56:35.779 INFO:teuthology.orchestra.run.smithi162.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-04-27T03:56:35.779 INFO:teuthology.orchestra.run.smithi162.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-04-27T03:56:35.779 INFO:teuthology.orchestra.run.smithi162.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-04-27T03:56:35.779 INFO:teuthology.orchestra.run.smithi162.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-04-27T03:56:35.779 INFO:teuthology.orchestra.run.smithi162.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-04-27T03:56:35.779 INFO:teuthology.orchestra.run.smithi162.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-04-27T03:56:35.780 INFO:teuthology.orchestra.run.smithi162.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-04-27T03:56:35.780 INFO:teuthology.orchestra.run.smithi162.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-04-27T03:56:35.780 INFO:teuthology.orchestra.run.smithi162.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-04-27T03:56:35.780 INFO:teuthology.orchestra.run.smithi162.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-04-27T03:56:35.780 INFO:teuthology.orchestra.run.smithi162.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-04-27T03:56:35.780 INFO:teuthology.orchestra.run.smithi162.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-04-27T03:56:35.780 INFO:teuthology.orchestra.run.smithi162.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-04-27T03:56:35.780 INFO:teuthology.orchestra.run.smithi162.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-04-27T03:56:35.780 INFO:teuthology.orchestra.run.smithi162.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-04-27T03:56:35.780 INFO:teuthology.orchestra.run.smithi162.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-04-27T03:56:35.780 INFO:teuthology.orchestra.run.smithi162.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-04-27T03:56:35.780 INFO:teuthology.orchestra.run.smithi162.stdout: python3-urllib3 noarch 1.26.5-5.el9 baseos 215 k 2024-04-27T03:56:35.780 INFO:teuthology.orchestra.run.smithi162.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-04-27T03:56:35.780 INFO:teuthology.orchestra.run.smithi162.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-04-27T03:56:35.780 INFO:teuthology.orchestra.run.smithi162.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-04-27T03:56:35.781 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-04-27T03:56:35.781 INFO:teuthology.orchestra.run.smithi162.stdout:Transaction Summary 2024-04-27T03:56:35.781 INFO:teuthology.orchestra.run.smithi162.stdout:================================================================================ 2024-04-27T03:56:35.781 INFO:teuthology.orchestra.run.smithi162.stdout:Install 33 Packages 2024-04-27T03:56:35.781 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-04-27T03:56:35.781 INFO:teuthology.orchestra.run.smithi162.stdout:Total download size: 27 M 2024-04-27T03:56:35.782 INFO:teuthology.orchestra.run.smithi162.stdout:Installed size: 96 M 2024-04-27T03:56:35.782 INFO:teuthology.orchestra.run.smithi162.stdout:Downloading Packages: 2024-04-27T03:56:36.406 INFO:teuthology.orchestra.run.smithi162.stdout:(1/33): ceph-17.2.7-907.g2be3efa9.el9.x86_64.rp 40 kB/s | 6.5 kB 00:00 2024-04-27T03:56:36.607 INFO:teuthology.orchestra.run.smithi162.stdout:(2/33): ceph-mgr-17.2.7-907.g2be3efa9.el9.x86_6 3.9 MB/s | 1.4 MB 00:00 2024-04-27T03:56:37.057 INFO:teuthology.orchestra.run.smithi162.stdout:(3/33): ceph-mon-17.2.7-907.g2be3efa9.el9.x86_6 6.0 MB/s | 3.9 MB 00:00 2024-04-27T03:56:37.099 INFO:teuthology.orchestra.run.smithi162.stdout:(4/33): ceph-mds-17.2.7-907.g2be3efa9.el9.x86_6 2.4 MB/s | 2.1 MB 00:00 2024-04-27T03:56:37.124 INFO:teuthology.orchestra.run.smithi162.stdout:(5/33): ceph-mgr-modules-core-17.2.7-907.g2be3e 3.5 MB/s | 240 kB 00:00 2024-04-27T03:56:37.308 INFO:teuthology.orchestra.run.smithi162.stdout:(6/33): python3-cffi-1.14.5-5.el9.x86_64.rpm 1.2 MB/s | 253 kB 00:00 2024-04-27T03:56:37.366 INFO:teuthology.orchestra.run.smithi162.stdout:(7/33): python3-ply-3.11-14.el9.noarch.rpm 1.8 MB/s | 106 kB 00:00 2024-04-27T03:56:37.567 INFO:teuthology.orchestra.run.smithi162.stdout:(8/33): ceph-osd-17.2.7-907.g2be3efa9.el9.x86_6 16 MB/s | 15 MB 00:00 2024-04-27T03:56:37.592 INFO:teuthology.orchestra.run.smithi162.stdout:(9/33): python3-pycparser-2.20-6.el9.noarch.rpm 600 kB/s | 135 kB 00:00 2024-04-27T03:56:37.634 INFO:teuthology.orchestra.run.smithi162.stdout:(10/33): python3-cryptography-36.0.1-4.el9.x86_ 2.4 MB/s | 1.2 MB 00:00 2024-04-27T03:56:37.659 INFO:teuthology.orchestra.run.smithi162.stdout:(11/33): python3-urllib3-1.26.5-5.el9.noarch.rp 3.1 MB/s | 215 kB 00:00 2024-04-27T03:56:37.776 INFO:teuthology.orchestra.run.smithi162.stdout:(12/33): python3-markupsafe-1.1.1-12.el9.x86_64 298 kB/s | 35 kB 00:00 2024-04-27T03:56:37.801 INFO:teuthology.orchestra.run.smithi162.stdout:(13/33): python3-requests-2.25.1-8.el9.noarch.r 533 kB/s | 125 kB 00:00 2024-04-27T03:56:37.826 INFO:teuthology.orchestra.run.smithi162.stdout:(14/33): python3-mako-1.1.4-6.el9.noarch.rpm 895 kB/s | 172 kB 00:00 2024-04-27T03:56:37.860 INFO:teuthology.orchestra.run.smithi162.stdout:(15/33): python3-pytz-2021.1-5.el9.noarch.rpm 608 kB/s | 51 kB 00:00 2024-04-27T03:56:37.910 INFO:teuthology.orchestra.run.smithi162.stdout:(16/33): python3-toml-0.10.2-6.el9.noarch.rpm 383 kB/s | 42 kB 00:00 2024-04-27T03:56:37.994 INFO:teuthology.orchestra.run.smithi162.stdout:(17/33): python3-bcrypt-3.2.2-1.el9.x86_64.rpm 260 kB/s | 43 kB 00:00 2024-04-27T03:56:38.061 INFO:teuthology.orchestra.run.smithi162.stdout:(18/33): python3-jaraco-8.2.1-3.el9.noarch.rpm 161 kB/s | 11 kB 00:00 2024-04-27T03:56:38.086 INFO:teuthology.orchestra.run.smithi162.stdout:(19/33): python3-cheroot-8.6.0-4.el9.noarch.rpm 761 kB/s | 172 kB 00:00 2024-04-27T03:56:38.120 INFO:teuthology.orchestra.run.smithi162.stdout:(20/33): python3-jaraco-classes-3.2.1-5.el9.noa 301 kB/s | 18 kB 00:00 2024-04-27T03:56:38.145 INFO:teuthology.orchestra.run.smithi162.stdout:(21/33): python3-jaraco-collections-3.0.0-8.el9 394 kB/s | 23 kB 00:00 2024-04-27T03:56:38.179 INFO:teuthology.orchestra.run.smithi162.stdout:(22/33): python3-cherrypy-18.6.1-2.el9.noarch.r 1.3 MB/s | 358 kB 00:00 2024-04-27T03:56:38.196 INFO:teuthology.orchestra.run.smithi162.stdout:(23/33): python3-jaraco-functools-3.5.0-2.el9.n 258 kB/s | 19 kB 00:00 2024-04-27T03:56:38.221 INFO:teuthology.orchestra.run.smithi162.stdout:(24/33): python3-jaraco-text-3.2.0-6.el9.noarch 261 kB/s | 20 kB 00:00 2024-04-27T03:56:38.246 INFO:teuthology.orchestra.run.smithi162.stdout:(25/33): python3-logutils-0.3.5-21.el9.noarch.r 692 kB/s | 46 kB 00:00 2024-04-27T03:56:38.305 INFO:teuthology.orchestra.run.smithi162.stdout:(26/33): python3-portend-3.1.0-2.el9.noarch.rpm 281 kB/s | 16 kB 00:00 2024-04-27T03:56:38.330 INFO:teuthology.orchestra.run.smithi162.stdout:(27/33): python3-more-itertools-8.12.0-2.el9.no 725 kB/s | 79 kB 00:00 2024-04-27T03:56:38.355 INFO:teuthology.orchestra.run.smithi162.stdout:(28/33): python3-pecan-1.4.2-3.el9.noarch.rpm 2.0 MB/s | 272 kB 00:00 2024-04-27T03:56:38.380 INFO:teuthology.orchestra.run.smithi162.stdout:(29/33): python3-pyOpenSSL-21.0.0-1.el9.noarch. 1.2 MB/s | 90 kB 00:00 2024-04-27T03:56:38.414 INFO:teuthology.orchestra.run.smithi162.stdout:(30/33): python3-tempora-5.0.0-2.el9.noarch.rpm 426 kB/s | 36 kB 00:00 2024-04-27T03:56:38.439 INFO:teuthology.orchestra.run.smithi162.stdout:(31/33): python3-webob-1.8.7-6.el9.noarch.rpm 2.7 MB/s | 230 kB 00:00 2024-04-27T03:56:38.473 INFO:teuthology.orchestra.run.smithi162.stdout:(32/33): python3-zc-lockfile-2.0-10.el9.noarch. 343 kB/s | 20 kB 00:00 2024-04-27T03:56:38.506 INFO:teuthology.orchestra.run.smithi162.stdout:(33/33): python3-werkzeug-2.0.3-3.el9.1.noarch. 3.3 MB/s | 427 kB 00:00 2024-04-27T03:56:38.508 INFO:teuthology.orchestra.run.smithi162.stdout:-------------------------------------------------------------------------------- 2024-04-27T03:56:38.508 INFO:teuthology.orchestra.run.smithi162.stdout:Total 10 MB/s | 27 MB 00:02 2024-04-27T03:56:38.627 INFO:teuthology.orchestra.run.smithi162.stdout:Running transaction check 2024-04-27T03:56:38.664 INFO:teuthology.orchestra.run.smithi162.stdout:Transaction check succeeded. 2024-04-27T03:56:38.664 INFO:teuthology.orchestra.run.smithi162.stdout:Running transaction test 2024-04-27T03:56:38.934 INFO:teuthology.orchestra.run.smithi162.stdout:Transaction test succeeded. 2024-04-27T03:56:38.934 INFO:teuthology.orchestra.run.smithi162.stdout:Running transaction 2024-04-27T03:56:40.122 INFO:teuthology.orchestra.run.smithi162.stdout: Preparing : 1/1 2024-04-27T03:56:40.184 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 1/33 2024-04-27T03:56:40.216 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 2/33 2024-04-27T03:56:40.257 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 3/33 2024-04-27T03:56:40.350 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 4/33 2024-04-27T03:56:40.445 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 5/33 2024-04-27T03:56:40.484 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 6/33 2024-04-27T03:56:40.528 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 7/33 2024-04-27T03:56:40.558 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 8/33 2024-04-27T03:56:40.593 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 9/33 2024-04-27T03:56:40.626 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 10/33 2024-04-27T03:56:40.666 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 11/33 2024-04-27T03:56:40.701 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 12/33 2024-04-27T03:56:40.736 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 13/33 2024-04-27T03:56:40.766 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 14/33 2024-04-27T03:56:40.839 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 15/33 2024-04-27T03:56:41.003 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 16/33 2024-04-27T03:56:41.082 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 17/33 2024-04-27T03:56:41.135 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 18/33 2024-04-27T03:56:41.173 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 19/33 2024-04-27T03:56:41.231 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : python3-ply-3.11-14.el9.noarch 20/33 2024-04-27T03:56:41.295 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 21/33 2024-04-27T03:56:41.481 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 22/33 2024-04-27T03:56:41.535 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 23/33 2024-04-27T03:56:41.606 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 24/33 2024-04-27T03:56:41.694 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 25/33 2024-04-27T03:56:41.734 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 26/33 2024-04-27T03:56:41.851 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 27/33 2024-04-27T03:56:42.006 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : ceph-mgr-modules-core-2:17.2.7-907.g2be3efa9.el9.n 28/33 2024-04-27T03:56:42.036 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : ceph-mgr-2:17.2.7-907.g2be3efa9.el9.x86_64 29/33 2024-04-27T03:56:42.062 INFO:teuthology.orchestra.run.smithi162.stdout: Running scriptlet: ceph-mgr-2:17.2.7-907.g2be3efa9.el9.x86_64 29/33 2024-04-27T03:56:42.062 INFO:teuthology.orchestra.run.smithi162.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-27T03:56:42.062 INFO:teuthology.orchestra.run.smithi162.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-04-27T03:56:42.062 INFO:teuthology.orchestra.run.smithi162.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-27T03:56:42.062 INFO:teuthology.orchestra.run.smithi162.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-27T03:56:42.062 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-04-27T03:56:43.387 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : ceph-osd-2:17.2.7-907.g2be3efa9.el9.x86_64 30/33 2024-04-27T03:56:43.413 INFO:teuthology.orchestra.run.smithi162.stdout: Running scriptlet: ceph-osd-2:17.2.7-907.g2be3efa9.el9.x86_64 30/33 2024-04-27T03:56:43.413 INFO:teuthology.orchestra.run.smithi162.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-27T03:56:43.413 INFO:teuthology.orchestra.run.smithi162.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-04-27T03:56:43.414 INFO:teuthology.orchestra.run.smithi162.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-27T03:56:43.414 INFO:teuthology.orchestra.run.smithi162.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-27T03:56:43.414 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-04-27T03:56:43.780 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : ceph-mon-2:17.2.7-907.g2be3efa9.el9.x86_64 31/33 2024-04-27T03:56:43.805 INFO:teuthology.orchestra.run.smithi162.stdout: Running scriptlet: ceph-mon-2:17.2.7-907.g2be3efa9.el9.x86_64 31/33 2024-04-27T03:56:43.805 INFO:teuthology.orchestra.run.smithi162.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-27T03:56:43.805 INFO:teuthology.orchestra.run.smithi162.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-04-27T03:56:43.805 INFO:teuthology.orchestra.run.smithi162.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-27T03:56:43.806 INFO:teuthology.orchestra.run.smithi162.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-27T03:56:43.806 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-04-27T03:56:44.005 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : ceph-mds-2:17.2.7-907.g2be3efa9.el9.x86_64 32/33 2024-04-27T03:56:44.028 INFO:teuthology.orchestra.run.smithi162.stdout: Running scriptlet: ceph-mds-2:17.2.7-907.g2be3efa9.el9.x86_64 32/33 2024-04-27T03:56:44.028 INFO:teuthology.orchestra.run.smithi162.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-27T03:56:44.028 INFO:teuthology.orchestra.run.smithi162.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-04-27T03:56:44.028 INFO:teuthology.orchestra.run.smithi162.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-27T03:56:44.028 INFO:teuthology.orchestra.run.smithi162.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-27T03:56:44.028 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-04-27T03:56:44.091 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : ceph-2:17.2.7-907.g2be3efa9.el9.x86_64 33/33 2024-04-27T03:56:45.684 INFO:teuthology.orchestra.run.smithi162.stdout: Running scriptlet: ceph-2:17.2.7-907.g2be3efa9.el9.x86_64 33/33 2024-04-27T03:56:45.684 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : ceph-2:17.2.7-907.g2be3efa9.el9.x86_64 1/33 2024-04-27T03:56:45.684 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : ceph-mds-2:17.2.7-907.g2be3efa9.el9.x86_64 2/33 2024-04-27T03:56:45.684 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : ceph-mgr-2:17.2.7-907.g2be3efa9.el9.x86_64 3/33 2024-04-27T03:56:45.684 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : ceph-mon-2:17.2.7-907.g2be3efa9.el9.x86_64 4/33 2024-04-27T03:56:45.684 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : ceph-osd-2:17.2.7-907.g2be3efa9.el9.x86_64 5/33 2024-04-27T03:56:45.684 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : ceph-mgr-modules-core-2:17.2.7-907.g2be3efa9.el9.n 6/33 2024-04-27T03:56:45.684 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 7/33 2024-04-27T03:56:45.684 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 8/33 2024-04-27T03:56:45.684 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : python3-ply-3.11-14.el9.noarch 9/33 2024-04-27T03:56:45.684 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 10/33 2024-04-27T03:56:45.684 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 11/33 2024-04-27T03:56:45.685 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 12/33 2024-04-27T03:56:45.685 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 13/33 2024-04-27T03:56:45.685 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 14/33 2024-04-27T03:56:45.685 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 15/33 2024-04-27T03:56:45.685 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 16/33 2024-04-27T03:56:45.685 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 17/33 2024-04-27T03:56:45.685 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 18/33 2024-04-27T03:56:45.685 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 19/33 2024-04-27T03:56:45.685 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 20/33 2024-04-27T03:56:45.685 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 21/33 2024-04-27T03:56:45.685 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 22/33 2024-04-27T03:56:45.686 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 23/33 2024-04-27T03:56:45.686 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 24/33 2024-04-27T03:56:45.686 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 25/33 2024-04-27T03:56:45.687 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 26/33 2024-04-27T03:56:45.687 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 27/33 2024-04-27T03:56:45.687 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 28/33 2024-04-27T03:56:45.687 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 29/33 2024-04-27T03:56:45.687 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 30/33 2024-04-27T03:56:45.687 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 31/33 2024-04-27T03:56:45.687 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 32/33 2024-04-27T03:56:46.626 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 33/33 2024-04-27T03:56:46.626 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-04-27T03:56:46.626 INFO:teuthology.orchestra.run.smithi162.stdout:Installed: 2024-04-27T03:56:46.626 INFO:teuthology.orchestra.run.smithi162.stdout: ceph-2:17.2.7-907.g2be3efa9.el9.x86_64 2024-04-27T03:56:46.626 INFO:teuthology.orchestra.run.smithi162.stdout: ceph-mds-2:17.2.7-907.g2be3efa9.el9.x86_64 2024-04-27T03:56:46.626 INFO:teuthology.orchestra.run.smithi162.stdout: ceph-mgr-2:17.2.7-907.g2be3efa9.el9.x86_64 2024-04-27T03:56:46.626 INFO:teuthology.orchestra.run.smithi162.stdout: ceph-mgr-modules-core-2:17.2.7-907.g2be3efa9.el9.noarch 2024-04-27T03:56:46.626 INFO:teuthology.orchestra.run.smithi162.stdout: ceph-mon-2:17.2.7-907.g2be3efa9.el9.x86_64 2024-04-27T03:56:46.626 INFO:teuthology.orchestra.run.smithi162.stdout: ceph-osd-2:17.2.7-907.g2be3efa9.el9.x86_64 2024-04-27T03:56:46.627 INFO:teuthology.orchestra.run.smithi162.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-04-27T03:56:46.627 INFO:teuthology.orchestra.run.smithi162.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-04-27T03:56:46.627 INFO:teuthology.orchestra.run.smithi162.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-04-27T03:56:46.627 INFO:teuthology.orchestra.run.smithi162.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-04-27T03:56:46.627 INFO:teuthology.orchestra.run.smithi162.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-04-27T03:56:46.627 INFO:teuthology.orchestra.run.smithi162.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-04-27T03:56:46.627 INFO:teuthology.orchestra.run.smithi162.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-04-27T03:56:46.627 INFO:teuthology.orchestra.run.smithi162.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-04-27T03:56:46.627 INFO:teuthology.orchestra.run.smithi162.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-04-27T03:56:46.627 INFO:teuthology.orchestra.run.smithi162.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-04-27T03:56:46.627 INFO:teuthology.orchestra.run.smithi162.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-04-27T03:56:46.627 INFO:teuthology.orchestra.run.smithi162.stdout: python3-mako-1.1.4-6.el9.noarch 2024-04-27T03:56:46.628 INFO:teuthology.orchestra.run.smithi162.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-04-27T03:56:46.628 INFO:teuthology.orchestra.run.smithi162.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-04-27T03:56:46.628 INFO:teuthology.orchestra.run.smithi162.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-04-27T03:56:46.628 INFO:teuthology.orchestra.run.smithi162.stdout: python3-ply-3.11-14.el9.noarch 2024-04-27T03:56:46.628 INFO:teuthology.orchestra.run.smithi162.stdout: python3-portend-3.1.0-2.el9.noarch 2024-04-27T03:56:46.628 INFO:teuthology.orchestra.run.smithi162.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-04-27T03:56:46.628 INFO:teuthology.orchestra.run.smithi162.stdout: python3-pycparser-2.20-6.el9.noarch 2024-04-27T03:56:46.628 INFO:teuthology.orchestra.run.smithi162.stdout: python3-pytz-2021.1-5.el9.noarch 2024-04-27T03:56:46.628 INFO:teuthology.orchestra.run.smithi162.stdout: python3-requests-2.25.1-8.el9.noarch 2024-04-27T03:56:46.629 INFO:teuthology.orchestra.run.smithi162.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-04-27T03:56:46.629 INFO:teuthology.orchestra.run.smithi162.stdout: python3-toml-0.10.2-6.el9.noarch 2024-04-27T03:56:46.629 INFO:teuthology.orchestra.run.smithi162.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-04-27T03:56:46.629 INFO:teuthology.orchestra.run.smithi162.stdout: python3-webob-1.8.7-6.el9.noarch 2024-04-27T03:56:46.629 INFO:teuthology.orchestra.run.smithi162.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-04-27T03:56:46.629 INFO:teuthology.orchestra.run.smithi162.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-04-27T03:56:46.629 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-04-27T03:56:46.629 INFO:teuthology.orchestra.run.smithi162.stdout:Complete! 2024-04-27T03:56:46.974 DEBUG:teuthology.orchestra.run.smithi162:> sudo yum -y install ceph-base 2024-04-27T03:56:47.496 INFO:teuthology.orchestra.run.smithi162.stdout:Last metadata expiration check: 0:00:54 ago on Sat 27 Apr 2024 03:55:53 AM UTC. 2024-04-27T03:56:47.574 INFO:teuthology.orchestra.run.smithi162.stdout:Package ceph-base-2:17.2.7-907.g2be3efa9.el9.x86_64 is already installed. 2024-04-27T03:56:47.616 INFO:teuthology.orchestra.run.smithi162.stdout:Dependencies resolved. 2024-04-27T03:56:47.617 INFO:teuthology.orchestra.run.smithi162.stdout:Nothing to do. 2024-04-27T03:56:47.617 INFO:teuthology.orchestra.run.smithi162.stdout:Complete! 2024-04-27T03:56:47.683 DEBUG:teuthology.orchestra.run.smithi162:> sudo yum -y install cephadm 2024-04-27T03:56:48.188 INFO:teuthology.orchestra.run.smithi162.stdout:Last metadata expiration check: 0:00:55 ago on Sat 27 Apr 2024 03:55:53 AM UTC. 2024-04-27T03:56:48.309 INFO:teuthology.orchestra.run.smithi162.stdout:Dependencies resolved. 2024-04-27T03:56:48.324 INFO:teuthology.orchestra.run.smithi162.stdout:================================================================================ 2024-04-27T03:56:48.325 INFO:teuthology.orchestra.run.smithi162.stdout: Package Arch Version Repository Size 2024-04-27T03:56:48.325 INFO:teuthology.orchestra.run.smithi162.stdout:================================================================================ 2024-04-27T03:56:48.325 INFO:teuthology.orchestra.run.smithi162.stdout:Installing: 2024-04-27T03:56:48.325 INFO:teuthology.orchestra.run.smithi162.stdout: cephadm noarch 2:17.2.7-907.g2be3efa9.el9 ceph-noarch 82 k 2024-04-27T03:56:48.325 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-04-27T03:56:48.325 INFO:teuthology.orchestra.run.smithi162.stdout:Transaction Summary 2024-04-27T03:56:48.325 INFO:teuthology.orchestra.run.smithi162.stdout:================================================================================ 2024-04-27T03:56:48.325 INFO:teuthology.orchestra.run.smithi162.stdout:Install 1 Package 2024-04-27T03:56:48.325 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-04-27T03:56:48.325 INFO:teuthology.orchestra.run.smithi162.stdout:Total download size: 82 k 2024-04-27T03:56:48.325 INFO:teuthology.orchestra.run.smithi162.stdout:Installed size: 372 k 2024-04-27T03:56:48.325 INFO:teuthology.orchestra.run.smithi162.stdout:Downloading Packages: 2024-04-27T03:56:48.629 INFO:teuthology.orchestra.run.smithi162.stdout:cephadm-17.2.7-907.g2be3efa9.el9.noarch.rpm 258 kB/s | 82 kB 00:00 2024-04-27T03:56:48.629 INFO:teuthology.orchestra.run.smithi162.stdout:-------------------------------------------------------------------------------- 2024-04-27T03:56:48.630 INFO:teuthology.orchestra.run.smithi162.stdout:Total 256 kB/s | 82 kB 00:00 2024-04-27T03:56:48.630 INFO:teuthology.orchestra.run.smithi162.stdout:Running transaction check 2024-04-27T03:56:48.635 INFO:teuthology.orchestra.run.smithi162.stdout:Transaction check succeeded. 2024-04-27T03:56:48.635 INFO:teuthology.orchestra.run.smithi162.stdout:Running transaction test 2024-04-27T03:56:48.637 INFO:teuthology.orchestra.run.smithi162.stdout:Transaction test succeeded. 2024-04-27T03:56:48.637 INFO:teuthology.orchestra.run.smithi162.stdout:Running transaction 2024-04-27T03:56:48.714 INFO:teuthology.orchestra.run.smithi162.stdout: Preparing : 1/1 2024-04-27T03:56:49.134 INFO:teuthology.orchestra.run.smithi162.stdout: Running scriptlet: cephadm-2:17.2.7-907.g2be3efa9.el9.noarch 1/1 2024-04-27T03:56:49.222 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : cephadm-2:17.2.7-907.g2be3efa9.el9.noarch 1/1 2024-04-27T03:56:49.552 INFO:teuthology.orchestra.run.smithi162.stdout: Running scriptlet: cephadm-2:17.2.7-907.g2be3efa9.el9.noarch 1/1 2024-04-27T03:56:49.797 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : cephadm-2:17.2.7-907.g2be3efa9.el9.noarch 1/1 2024-04-27T03:56:49.797 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-04-27T03:56:49.797 INFO:teuthology.orchestra.run.smithi162.stdout:Installed: 2024-04-27T03:56:49.797 INFO:teuthology.orchestra.run.smithi162.stdout: cephadm-2:17.2.7-907.g2be3efa9.el9.noarch 2024-04-27T03:56:49.797 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-04-27T03:56:49.797 INFO:teuthology.orchestra.run.smithi162.stdout:Complete! 2024-04-27T03:56:50.015 DEBUG:teuthology.orchestra.run.smithi162:> sudo yum -y install ceph-immutable-object-cache 2024-04-27T03:56:50.528 INFO:teuthology.orchestra.run.smithi162.stdout:Last metadata expiration check: 0:00:57 ago on Sat 27 Apr 2024 03:55:53 AM UTC. 2024-04-27T03:56:50.652 INFO:teuthology.orchestra.run.smithi162.stdout:Dependencies resolved. 2024-04-27T03:56:50.653 INFO:teuthology.orchestra.run.smithi162.stdout:================================================================================ 2024-04-27T03:56:50.653 INFO:teuthology.orchestra.run.smithi162.stdout: Package Arch Version Repo Size 2024-04-27T03:56:50.653 INFO:teuthology.orchestra.run.smithi162.stdout:================================================================================ 2024-04-27T03:56:50.653 INFO:teuthology.orchestra.run.smithi162.stdout:Installing: 2024-04-27T03:56:50.653 INFO:teuthology.orchestra.run.smithi162.stdout: ceph-immutable-object-cache x86_64 2:17.2.7-907.g2be3efa9.el9 ceph 137 k 2024-04-27T03:56:50.653 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-04-27T03:56:50.653 INFO:teuthology.orchestra.run.smithi162.stdout:Transaction Summary 2024-04-27T03:56:50.653 INFO:teuthology.orchestra.run.smithi162.stdout:================================================================================ 2024-04-27T03:56:50.653 INFO:teuthology.orchestra.run.smithi162.stdout:Install 1 Package 2024-04-27T03:56:50.653 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-04-27T03:56:50.653 INFO:teuthology.orchestra.run.smithi162.stdout:Total download size: 137 k 2024-04-27T03:56:50.653 INFO:teuthology.orchestra.run.smithi162.stdout:Installed size: 379 k 2024-04-27T03:56:50.653 INFO:teuthology.orchestra.run.smithi162.stdout:Downloading Packages: 2024-04-27T03:56:50.866 INFO:teuthology.orchestra.run.smithi181.stdout:(4/4): ceph-test-17.2.7-907.g2be3efa9.el9.x86_6 2.9 MB/s | 48 MB 00:16 2024-04-27T03:56:50.868 INFO:teuthology.orchestra.run.smithi181.stdout:-------------------------------------------------------------------------------- 2024-04-27T03:56:50.868 INFO:teuthology.orchestra.run.smithi181.stdout:Total 2.9 MB/s | 48 MB 00:16 2024-04-27T03:56:50.895 INFO:teuthology.orchestra.run.smithi181.stdout:Running transaction check 2024-04-27T03:56:50.914 INFO:teuthology.orchestra.run.smithi181.stdout:Transaction check succeeded. 2024-04-27T03:56:50.914 INFO:teuthology.orchestra.run.smithi181.stdout:Running transaction test 2024-04-27T03:56:51.000 INFO:teuthology.orchestra.run.smithi162.stdout:ceph-immutable-object-cache-17.2.7-907.g2be3efa 397 kB/s | 137 kB 00:00 2024-04-27T03:56:51.001 INFO:teuthology.orchestra.run.smithi162.stdout:-------------------------------------------------------------------------------- 2024-04-27T03:56:51.001 INFO:teuthology.orchestra.run.smithi162.stdout:Total 395 kB/s | 137 kB 00:00 2024-04-27T03:56:51.002 INFO:teuthology.orchestra.run.smithi162.stdout:Running transaction check 2024-04-27T03:56:51.009 INFO:teuthology.orchestra.run.smithi162.stdout:Transaction check succeeded. 2024-04-27T03:56:51.009 INFO:teuthology.orchestra.run.smithi162.stdout:Running transaction test 2024-04-27T03:56:51.069 INFO:teuthology.orchestra.run.smithi162.stdout:Transaction test succeeded. 2024-04-27T03:56:51.069 INFO:teuthology.orchestra.run.smithi162.stdout:Running transaction 2024-04-27T03:56:51.214 INFO:teuthology.orchestra.run.smithi162.stdout: Preparing : 1/1 2024-04-27T03:56:51.277 INFO:teuthology.orchestra.run.smithi181.stdout:Transaction test succeeded. 2024-04-27T03:56:51.277 INFO:teuthology.orchestra.run.smithi181.stdout:Running transaction 2024-04-27T03:56:51.301 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : ceph-immutable-object-cache-2:17.2.7-907.g2be3efa9.e 1/1 2024-04-27T03:56:51.326 INFO:teuthology.orchestra.run.smithi162.stdout: Running scriptlet: ceph-immutable-object-cache-2:17.2.7-907.g2be3efa9.e 1/1 2024-04-27T03:56:51.326 INFO:teuthology.orchestra.run.smithi162.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-27T03:56:51.326 INFO:teuthology.orchestra.run.smithi162.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-04-27T03:56:51.326 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-04-27T03:56:51.925 INFO:teuthology.orchestra.run.smithi181.stdout: Preparing : 1/1 2024-04-27T03:56:52.007 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 1/4 2024-04-27T03:56:52.048 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 2/4 2024-04-27T03:56:52.076 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : libcephsqlite-2:17.2.7-907.g2be3efa9.el9.x86_64 3/4 2024-04-27T03:56:52.165 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : ceph-immutable-object-cache-2:17.2.7-907.g2be3efa9.e 1/1 2024-04-27T03:56:52.165 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-04-27T03:56:52.165 INFO:teuthology.orchestra.run.smithi162.stdout:Installed: 2024-04-27T03:56:52.165 INFO:teuthology.orchestra.run.smithi162.stdout: ceph-immutable-object-cache-2:17.2.7-907.g2be3efa9.el9.x86_64 2024-04-27T03:56:52.165 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-04-27T03:56:52.165 INFO:teuthology.orchestra.run.smithi162.stdout:Complete! 2024-04-27T03:56:52.364 DEBUG:teuthology.orchestra.run.smithi162:> sudo yum -y install ceph-mgr 2024-04-27T03:56:52.880 INFO:teuthology.orchestra.run.smithi162.stdout:Last metadata expiration check: 0:00:59 ago on Sat 27 Apr 2024 03:55:53 AM UTC. 2024-04-27T03:56:52.962 INFO:teuthology.orchestra.run.smithi162.stdout:Package ceph-mgr-2:17.2.7-907.g2be3efa9.el9.x86_64 is already installed. 2024-04-27T03:56:53.006 INFO:teuthology.orchestra.run.smithi162.stdout:Dependencies resolved. 2024-04-27T03:56:53.007 INFO:teuthology.orchestra.run.smithi162.stdout:Nothing to do. 2024-04-27T03:56:53.007 INFO:teuthology.orchestra.run.smithi162.stdout:Complete! 2024-04-27T03:56:53.064 DEBUG:teuthology.orchestra.run.smithi162:> sudo yum -y install ceph-mgr-dashboard 2024-04-27T03:56:53.562 INFO:teuthology.orchestra.run.smithi162.stdout:Last metadata expiration check: 0:01:00 ago on Sat 27 Apr 2024 03:55:53 AM UTC. 2024-04-27T03:56:53.683 INFO:teuthology.orchestra.run.smithi162.stdout:Dependencies resolved. 2024-04-27T03:56:53.684 INFO:teuthology.orchestra.run.smithi162.stdout:================================================================================ 2024-04-27T03:56:53.684 INFO:teuthology.orchestra.run.smithi162.stdout: Package Arch Version Repository Size 2024-04-27T03:56:53.684 INFO:teuthology.orchestra.run.smithi162.stdout:================================================================================ 2024-04-27T03:56:53.684 INFO:teuthology.orchestra.run.smithi162.stdout:Installing: 2024-04-27T03:56:53.685 INFO:teuthology.orchestra.run.smithi162.stdout: ceph-mgr-dashboard noarch 2:17.2.7-907.g2be3efa9.el9 ceph-noarch 3.4 M 2024-04-27T03:56:53.685 INFO:teuthology.orchestra.run.smithi162.stdout:Installing dependencies: 2024-04-27T03:56:53.685 INFO:teuthology.orchestra.run.smithi162.stdout: ceph-grafana-dashboards noarch 2:17.2.7-907.g2be3efa9.el9 ceph-noarch 24 k 2024-04-27T03:56:53.685 INFO:teuthology.orchestra.run.smithi162.stdout: ceph-prometheus-alerts noarch 2:17.2.7-907.g2be3efa9.el9 ceph-noarch 15 k 2024-04-27T03:56:53.685 INFO:teuthology.orchestra.run.smithi162.stdout: python3-jwt noarch 2.4.0-1.el9 epel 41 k 2024-04-27T03:56:53.685 INFO:teuthology.orchestra.run.smithi162.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-04-27T03:56:53.685 INFO:teuthology.orchestra.run.smithi162.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-04-27T03:56:53.685 INFO:teuthology.orchestra.run.smithi162.stdout:Installing weak dependencies: 2024-04-27T03:56:53.685 INFO:teuthology.orchestra.run.smithi162.stdout: python3-jwt+crypto noarch 2.4.0-1.el9 epel 9.0 k 2024-04-27T03:56:53.685 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-04-27T03:56:53.685 INFO:teuthology.orchestra.run.smithi162.stdout:Transaction Summary 2024-04-27T03:56:53.685 INFO:teuthology.orchestra.run.smithi162.stdout:================================================================================ 2024-04-27T03:56:53.685 INFO:teuthology.orchestra.run.smithi162.stdout:Install 7 Packages 2024-04-27T03:56:53.685 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-04-27T03:56:53.686 INFO:teuthology.orchestra.run.smithi162.stdout:Total download size: 3.7 M 2024-04-27T03:56:53.686 INFO:teuthology.orchestra.run.smithi162.stdout:Installed size: 80 M 2024-04-27T03:56:53.686 INFO:teuthology.orchestra.run.smithi162.stdout:Downloading Packages: 2024-04-27T03:56:54.070 INFO:teuthology.orchestra.run.smithi162.stdout:(1/7): ceph-grafana-dashboards-17.2.7-907.g2be3 130 kB/s | 24 kB 00:00 2024-04-27T03:56:54.170 INFO:teuthology.orchestra.run.smithi162.stdout:(2/7): ceph-prometheus-alerts-17.2.7-907.g2be3e 51 kB/s | 15 kB 00:00 2024-04-27T03:56:54.187 INFO:teuthology.orchestra.run.smithi162.stdout:(3/7): python3-jwt+crypto-2.4.0-1.el9.noarch.rp 77 kB/s | 9.0 kB 00:00 2024-04-27T03:56:54.279 INFO:teuthology.orchestra.run.smithi162.stdout:(4/7): python3-jwt-2.4.0-1.el9.noarch.rpm 377 kB/s | 41 kB 00:00 2024-04-27T03:56:54.312 INFO:teuthology.orchestra.run.smithi162.stdout:(5/7): python3-repoze-lru-0.7-16.el9.noarch.rpm 246 kB/s | 31 kB 00:00 2024-04-27T03:56:54.413 INFO:teuthology.orchestra.run.smithi162.stdout:(6/7): python3-routes-2.5.1-5.el9.noarch.rpm 1.4 MB/s | 188 kB 00:00 2024-04-27T03:56:54.471 INFO:teuthology.orchestra.run.smithi162.stdout:(7/7): ceph-mgr-dashboard-17.2.7-907.g2be3efa9. 5.7 MB/s | 3.4 MB 00:00 2024-04-27T03:56:54.473 INFO:teuthology.orchestra.run.smithi162.stdout:-------------------------------------------------------------------------------- 2024-04-27T03:56:54.474 INFO:teuthology.orchestra.run.smithi162.stdout:Total 4.6 MB/s | 3.7 MB 00:00 2024-04-27T03:56:54.474 INFO:teuthology.orchestra.run.smithi162.stdout:Running transaction check 2024-04-27T03:56:54.482 INFO:teuthology.orchestra.run.smithi162.stdout:Transaction check succeeded. 2024-04-27T03:56:54.482 INFO:teuthology.orchestra.run.smithi162.stdout:Running transaction test 2024-04-27T03:56:54.543 INFO:teuthology.orchestra.run.smithi162.stdout:Transaction test succeeded. 2024-04-27T03:56:54.544 INFO:teuthology.orchestra.run.smithi162.stdout:Running transaction 2024-04-27T03:56:54.847 INFO:teuthology.orchestra.run.smithi162.stdout: Preparing : 1/1 2024-04-27T03:56:54.906 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : python3-jwt-2.4.0-1.el9.noarch 1/7 2024-04-27T03:56:54.940 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : python3-jwt+crypto-2.4.0-1.el9.noarch 2/7 2024-04-27T03:56:54.987 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 3/7 2024-04-27T03:56:55.027 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 4/7 2024-04-27T03:56:55.055 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : ceph-prometheus-alerts-2:17.2.7-907.g2be3efa9.el9.no 5/7 2024-04-27T03:56:56.059 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : ceph-grafana-dashboards-2:17.2.7-907.g2be3efa9.el9.n 6/7 2024-04-27T03:56:56.092 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : ceph-mgr-dashboard-2:17.2.7-907.g2be3efa9.el9.noarch 7/7 2024-04-27T03:56:56.434 INFO:teuthology.orchestra.run.smithi181.stdout: Running scriptlet: libcephsqlite-2:17.2.7-907.g2be3efa9.el9.x86_64 3/4 2024-04-27T03:56:56.597 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : ceph-test-2:17.2.7-907.g2be3efa9.el9.x86_64 4/4 2024-04-27T03:56:57.159 INFO:teuthology.orchestra.run.smithi162.stdout: Running scriptlet: ceph-mgr-dashboard-2:17.2.7-907.g2be3efa9.el9.noarch 7/7 2024-04-27T03:56:57.159 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : ceph-grafana-dashboards-2:17.2.7-907.g2be3efa9.el9.n 1/7 2024-04-27T03:56:57.160 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : ceph-mgr-dashboard-2:17.2.7-907.g2be3efa9.el9.noarch 2/7 2024-04-27T03:56:57.160 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : ceph-prometheus-alerts-2:17.2.7-907.g2be3efa9.el9.no 3/7 2024-04-27T03:56:57.160 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : python3-jwt+crypto-2.4.0-1.el9.noarch 4/7 2024-04-27T03:56:57.160 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : python3-jwt-2.4.0-1.el9.noarch 5/7 2024-04-27T03:56:57.160 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 6/7 2024-04-27T03:56:57.427 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 7/7 2024-04-27T03:56:57.427 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-04-27T03:56:57.427 INFO:teuthology.orchestra.run.smithi162.stdout:Installed: 2024-04-27T03:56:57.427 INFO:teuthology.orchestra.run.smithi162.stdout: ceph-grafana-dashboards-2:17.2.7-907.g2be3efa9.el9.noarch 2024-04-27T03:56:57.427 INFO:teuthology.orchestra.run.smithi162.stdout: ceph-mgr-dashboard-2:17.2.7-907.g2be3efa9.el9.noarch 2024-04-27T03:56:57.427 INFO:teuthology.orchestra.run.smithi162.stdout: ceph-prometheus-alerts-2:17.2.7-907.g2be3efa9.el9.noarch 2024-04-27T03:56:57.428 INFO:teuthology.orchestra.run.smithi162.stdout: python3-jwt-2.4.0-1.el9.noarch 2024-04-27T03:56:57.428 INFO:teuthology.orchestra.run.smithi162.stdout: python3-jwt+crypto-2.4.0-1.el9.noarch 2024-04-27T03:56:57.428 INFO:teuthology.orchestra.run.smithi162.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-04-27T03:56:57.428 INFO:teuthology.orchestra.run.smithi162.stdout: python3-routes-2.5.1-5.el9.noarch 2024-04-27T03:56:57.428 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-04-27T03:56:57.428 INFO:teuthology.orchestra.run.smithi162.stdout:Complete! 2024-04-27T03:56:57.623 DEBUG:teuthology.orchestra.run.smithi162:> sudo yum -y install ceph-mgr-diskprediction-local 2024-04-27T03:56:58.135 INFO:teuthology.orchestra.run.smithi162.stdout:Last metadata expiration check: 0:01:05 ago on Sat 27 Apr 2024 03:55:53 AM UTC. 2024-04-27T03:56:58.258 INFO:teuthology.orchestra.run.smithi162.stdout:Dependencies resolved. 2024-04-27T03:56:58.259 INFO:teuthology.orchestra.run.smithi162.stdout:====================================================================================== 2024-04-27T03:56:58.259 INFO:teuthology.orchestra.run.smithi162.stdout: Package Arch Version Repository Size 2024-04-27T03:56:58.259 INFO:teuthology.orchestra.run.smithi162.stdout:====================================================================================== 2024-04-27T03:56:58.259 INFO:teuthology.orchestra.run.smithi162.stdout:Installing: 2024-04-27T03:56:58.259 INFO:teuthology.orchestra.run.smithi162.stdout: ceph-mgr-diskprediction-local noarch 2:17.2.7-907.g2be3efa9.el9 ceph-noarch 7.4 M 2024-04-27T03:56:58.259 INFO:teuthology.orchestra.run.smithi162.stdout:Installing dependencies: 2024-04-27T03:56:58.259 INFO:teuthology.orchestra.run.smithi162.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-04-27T03:56:58.259 INFO:teuthology.orchestra.run.smithi162.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-04-27T03:56:58.260 INFO:teuthology.orchestra.run.smithi162.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-04-27T03:56:58.260 INFO:teuthology.orchestra.run.smithi162.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-04-27T03:56:58.260 INFO:teuthology.orchestra.run.smithi162.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-04-27T03:56:58.260 INFO:teuthology.orchestra.run.smithi162.stdout: openblas x86_64 0.3.21-2.el9 appstream 35 k 2024-04-27T03:56:58.260 INFO:teuthology.orchestra.run.smithi162.stdout: openblas-openmp x86_64 0.3.21-2.el9 appstream 4.7 M 2024-04-27T03:56:58.260 INFO:teuthology.orchestra.run.smithi162.stdout: python3-devel x86_64 3.9.18-3.el9 appstream 244 k 2024-04-27T03:56:58.260 INFO:teuthology.orchestra.run.smithi162.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-04-27T03:56:58.260 INFO:teuthology.orchestra.run.smithi162.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-04-27T03:56:58.260 INFO:teuthology.orchestra.run.smithi162.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-04-27T03:56:58.260 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-04-27T03:56:58.260 INFO:teuthology.orchestra.run.smithi162.stdout:Transaction Summary 2024-04-27T03:56:58.260 INFO:teuthology.orchestra.run.smithi162.stdout:====================================================================================== 2024-04-27T03:56:58.260 INFO:teuthology.orchestra.run.smithi162.stdout:Install 12 Packages 2024-04-27T03:56:58.260 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-04-27T03:56:58.261 INFO:teuthology.orchestra.run.smithi162.stdout:Total download size: 38 M 2024-04-27T03:56:58.261 INFO:teuthology.orchestra.run.smithi162.stdout:Installed size: 209 M 2024-04-27T03:56:58.261 INFO:teuthology.orchestra.run.smithi162.stdout:Downloading Packages: 2024-04-27T03:56:58.687 INFO:teuthology.orchestra.run.smithi162.stdout:(1/12): libquadmath-11.4.1-3.el9.x86_64.rpm 803 kB/s | 195 kB 00:00 2024-04-27T03:56:58.738 INFO:teuthology.orchestra.run.smithi181.stdout: Running scriptlet: ceph-test-2:17.2.7-907.g2be3efa9.el9.x86_64 4/4 2024-04-27T03:56:58.738 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : ceph-test-2:17.2.7-907.g2be3efa9.el9.x86_64 1/4 2024-04-27T03:56:58.738 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : libcephsqlite-2:17.2.7-907.g2be3efa9.el9.x86_64 2/4 2024-04-27T03:56:58.738 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 3/4 2024-04-27T03:56:58.779 INFO:teuthology.orchestra.run.smithi162.stdout:(2/12): libgfortran-11.4.1-3.el9.x86_64.rpm 2.3 MB/s | 803 kB 00:00 2024-04-27T03:56:58.804 INFO:teuthology.orchestra.run.smithi162.stdout:(3/12): flexiblas-3.0.4-8.el9.x86_64.rpm 280 kB/s | 33 kB 00:00 2024-04-27T03:56:58.904 INFO:teuthology.orchestra.run.smithi162.stdout:(4/12): flexiblas-openblas-openmp-3.0.4-8.el9.x 182 kB/s | 18 kB 00:00 2024-04-27T03:56:58.988 INFO:teuthology.orchestra.run.smithi162.stdout:(5/12): openblas-0.3.21-2.el9.x86_64.rpm 423 kB/s | 35 kB 00:00 2024-04-27T03:56:58.994 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 4/4 2024-04-27T03:56:58.994 INFO:teuthology.orchestra.run.smithi181.stdout: 2024-04-27T03:56:58.994 INFO:teuthology.orchestra.run.smithi181.stdout:Installed: 2024-04-27T03:56:58.994 INFO:teuthology.orchestra.run.smithi181.stdout: ceph-test-2:17.2.7-907.g2be3efa9.el9.x86_64 2024-04-27T03:56:58.994 INFO:teuthology.orchestra.run.smithi181.stdout: libcephsqlite-2:17.2.7-907.g2be3efa9.el9.x86_64 2024-04-27T03:56:58.994 INFO:teuthology.orchestra.run.smithi181.stdout: socat-1.7.4.1-5.el9.x86_64 2024-04-27T03:56:58.994 INFO:teuthology.orchestra.run.smithi181.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-04-27T03:56:58.994 INFO:teuthology.orchestra.run.smithi181.stdout: 2024-04-27T03:56:58.994 INFO:teuthology.orchestra.run.smithi181.stdout:Complete! 2024-04-27T03:56:59.072 INFO:teuthology.orchestra.run.smithi162.stdout:(6/12): ceph-mgr-diskprediction-local-17.2.7-90 12 MB/s | 7.4 MB 00:00 2024-04-27T03:56:59.263 DEBUG:teuthology.orchestra.run.smithi181:> sudo yum -y install ceph 2024-04-27T03:56:59.272 INFO:teuthology.orchestra.run.smithi162.stdout:(7/12): python3-devel-3.9.18-3.el9.x86_64.rpm 1.2 MB/s | 244 kB 00:00 2024-04-27T03:56:59.647 INFO:teuthology.orchestra.run.smithi162.stdout:(8/12): flexiblas-netlib-3.0.4-8.el9.x86_64.rpm 3.4 MB/s | 3.0 MB 00:00 2024-04-27T03:56:59.730 INFO:teuthology.orchestra.run.smithi162.stdout:(9/12): python3-numpy-f2py-1.20.1-5.el9.x86_64. 2.9 MB/s | 250 kB 00:00 2024-04-27T03:56:59.769 INFO:teuthology.orchestra.run.smithi181.stdout:Last metadata expiration check: 0:00:54 ago on Sat 27 Apr 2024 03:56:05 AM UTC. 2024-04-27T03:56:59.897 INFO:teuthology.orchestra.run.smithi181.stdout:Dependencies resolved. 2024-04-27T03:56:59.899 INFO:teuthology.orchestra.run.smithi181.stdout:================================================================================ 2024-04-27T03:56:59.899 INFO:teuthology.orchestra.run.smithi181.stdout: Package Arch Version Repository Size 2024-04-27T03:56:59.899 INFO:teuthology.orchestra.run.smithi181.stdout:================================================================================ 2024-04-27T03:56:59.899 INFO:teuthology.orchestra.run.smithi181.stdout:Installing: 2024-04-27T03:56:59.899 INFO:teuthology.orchestra.run.smithi181.stdout: ceph x86_64 2:17.2.7-907.g2be3efa9.el9 ceph 6.5 k 2024-04-27T03:56:59.899 INFO:teuthology.orchestra.run.smithi181.stdout:Installing dependencies: 2024-04-27T03:56:59.899 INFO:teuthology.orchestra.run.smithi181.stdout: ceph-mds x86_64 2:17.2.7-907.g2be3efa9.el9 ceph 2.1 M 2024-04-27T03:56:59.900 INFO:teuthology.orchestra.run.smithi181.stdout: ceph-mgr x86_64 2:17.2.7-907.g2be3efa9.el9 ceph 1.4 M 2024-04-27T03:56:59.900 INFO:teuthology.orchestra.run.smithi181.stdout: ceph-mgr-modules-core noarch 2:17.2.7-907.g2be3efa9.el9 ceph-noarch 240 k 2024-04-27T03:56:59.900 INFO:teuthology.orchestra.run.smithi181.stdout: ceph-mon x86_64 2:17.2.7-907.g2be3efa9.el9 ceph 3.9 M 2024-04-27T03:56:59.900 INFO:teuthology.orchestra.run.smithi181.stdout: ceph-osd x86_64 2:17.2.7-907.g2be3efa9.el9 ceph 15 M 2024-04-27T03:56:59.900 INFO:teuthology.orchestra.run.smithi181.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-04-27T03:56:59.900 INFO:teuthology.orchestra.run.smithi181.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-04-27T03:56:59.900 INFO:teuthology.orchestra.run.smithi181.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-04-27T03:56:59.900 INFO:teuthology.orchestra.run.smithi181.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-04-27T03:56:59.900 INFO:teuthology.orchestra.run.smithi181.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-04-27T03:56:59.900 INFO:teuthology.orchestra.run.smithi181.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-04-27T03:56:59.900 INFO:teuthology.orchestra.run.smithi181.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-04-27T03:56:59.900 INFO:teuthology.orchestra.run.smithi181.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-04-27T03:56:59.900 INFO:teuthology.orchestra.run.smithi181.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-04-27T03:56:59.900 INFO:teuthology.orchestra.run.smithi181.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-04-27T03:56:59.901 INFO:teuthology.orchestra.run.smithi181.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-04-27T03:56:59.901 INFO:teuthology.orchestra.run.smithi181.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-04-27T03:56:59.901 INFO:teuthology.orchestra.run.smithi181.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-04-27T03:56:59.901 INFO:teuthology.orchestra.run.smithi181.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-04-27T03:56:59.901 INFO:teuthology.orchestra.run.smithi181.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-04-27T03:56:59.901 INFO:teuthology.orchestra.run.smithi181.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-04-27T03:56:59.901 INFO:teuthology.orchestra.run.smithi181.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-04-27T03:56:59.901 INFO:teuthology.orchestra.run.smithi181.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-04-27T03:56:59.901 INFO:teuthology.orchestra.run.smithi181.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-04-27T03:56:59.901 INFO:teuthology.orchestra.run.smithi181.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-04-27T03:56:59.901 INFO:teuthology.orchestra.run.smithi181.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-04-27T03:56:59.901 INFO:teuthology.orchestra.run.smithi181.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-04-27T03:56:59.901 INFO:teuthology.orchestra.run.smithi181.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-04-27T03:56:59.901 INFO:teuthology.orchestra.run.smithi181.stdout: python3-urllib3 noarch 1.26.5-5.el9 baseos 215 k 2024-04-27T03:56:59.901 INFO:teuthology.orchestra.run.smithi181.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-04-27T03:56:59.901 INFO:teuthology.orchestra.run.smithi181.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-04-27T03:56:59.902 INFO:teuthology.orchestra.run.smithi181.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-04-27T03:56:59.902 INFO:teuthology.orchestra.run.smithi181.stdout: 2024-04-27T03:56:59.902 INFO:teuthology.orchestra.run.smithi181.stdout:Transaction Summary 2024-04-27T03:56:59.902 INFO:teuthology.orchestra.run.smithi181.stdout:================================================================================ 2024-04-27T03:56:59.902 INFO:teuthology.orchestra.run.smithi181.stdout:Install 33 Packages 2024-04-27T03:56:59.902 INFO:teuthology.orchestra.run.smithi181.stdout: 2024-04-27T03:56:59.902 INFO:teuthology.orchestra.run.smithi181.stdout:Total download size: 27 M 2024-04-27T03:56:59.903 INFO:teuthology.orchestra.run.smithi181.stdout:Installed size: 96 M 2024-04-27T03:56:59.903 INFO:teuthology.orchestra.run.smithi181.stdout:Downloading Packages: 2024-04-27T03:57:00.131 INFO:teuthology.orchestra.run.smithi162.stdout:(10/12): openblas-openmp-0.3.21-2.el9.x86_64.rp 4.1 MB/s | 4.7 MB 00:01 2024-04-27T03:57:00.440 INFO:teuthology.orchestra.run.smithi181.stdout:(1/33): ceph-17.2.7-907.g2be3efa9.el9.x86_64.rp 27 kB/s | 6.5 kB 00:00 2024-04-27T03:57:00.557 INFO:teuthology.orchestra.run.smithi181.stdout:(2/33): ceph-mgr-17.2.7-907.g2be3efa9.el9.x86_6 3.9 MB/s | 1.4 MB 00:00 2024-04-27T03:57:00.633 INFO:teuthology.orchestra.run.smithi181.stdout:(3/33): ceph-mds-17.2.7-907.g2be3efa9.el9.x86_6 4.8 MB/s | 2.1 MB 00:00 2024-04-27T03:57:00.647 INFO:teuthology.orchestra.run.smithi162.stdout:(11/12): python3-numpy-1.20.1-5.el9.x86_64.rpm 3.7 MB/s | 5.1 MB 00:01 2024-04-27T03:57:00.716 INFO:teuthology.orchestra.run.smithi181.stdout:(4/33): ceph-mgr-modules-core-17.2.7-907.g2be3e 2.8 MB/s | 240 kB 00:00 2024-04-27T03:57:00.875 INFO:teuthology.orchestra.run.smithi181.stdout:(5/33): python3-cffi-1.14.5-5.el9.x86_64.rpm 1.6 MB/s | 253 kB 00:00 2024-04-27T03:57:00.975 INFO:teuthology.orchestra.run.smithi181.stdout:(6/33): python3-cryptography-36.0.1-4.el9.x86_6 12 MB/s | 1.2 MB 00:00 2024-04-27T03:57:01.017 INFO:teuthology.orchestra.run.smithi181.stdout:(7/33): python3-ply-3.11-14.el9.noarch.rpm 2.5 MB/s | 106 kB 00:00 2024-04-27T03:57:01.067 INFO:teuthology.orchestra.run.smithi181.stdout:(8/33): python3-pycparser-2.20-6.el9.noarch.rpm 2.6 MB/s | 135 kB 00:00 2024-04-27T03:57:01.118 INFO:teuthology.orchestra.run.smithi181.stdout:(9/33): python3-requests-2.25.1-8.el9.noarch.rp 2.4 MB/s | 125 kB 00:00 2024-04-27T03:57:01.168 INFO:teuthology.orchestra.run.smithi181.stdout:(10/33): python3-urllib3-1.26.5-5.el9.noarch.rp 4.2 MB/s | 215 kB 00:00 2024-04-27T03:57:01.327 INFO:teuthology.orchestra.run.smithi181.stdout:(11/33): ceph-osd-17.2.7-907.g2be3efa9.el9.x86_ 20 MB/s | 15 MB 00:00 2024-04-27T03:57:01.410 INFO:teuthology.orchestra.run.smithi181.stdout:(12/33): python3-markupsafe-1.1.1-12.el9.x86_64 419 kB/s | 35 kB 00:00 2024-04-27T03:57:01.477 INFO:teuthology.orchestra.run.smithi181.stdout:(13/33): python3-mako-1.1.4-6.el9.noarch.rpm 558 kB/s | 172 kB 00:00 2024-04-27T03:57:01.494 INFO:teuthology.orchestra.run.smithi181.stdout:(14/33): python3-pytz-2021.1-5.el9.noarch.rpm 609 kB/s | 51 kB 00:00 2024-04-27T03:57:01.528 INFO:teuthology.orchestra.run.smithi181.stdout:(15/33): python3-toml-0.10.2-6.el9.noarch.rpm 829 kB/s | 42 kB 00:00 2024-04-27T03:57:01.661 INFO:teuthology.orchestra.run.smithi181.stdout:(16/33): python3-bcrypt-3.2.2-1.el9.x86_64.rpm 260 kB/s | 43 kB 00:00 2024-04-27T03:57:01.711 INFO:teuthology.orchestra.run.smithi181.stdout:(17/33): python3-cheroot-8.6.0-4.el9.noarch.rpm 1.1 MB/s | 172 kB 00:00 2024-04-27T03:57:01.795 INFO:teuthology.orchestra.run.smithi181.stdout:(18/33): python3-cherrypy-18.6.1-2.el9.noarch.r 2.6 MB/s | 358 kB 00:00 2024-04-27T03:57:01.812 INFO:teuthology.orchestra.run.smithi181.stdout:(19/33): python3-jaraco-8.2.1-3.el9.noarch.rpm 106 kB/s | 11 kB 00:00 2024-04-27T03:57:01.837 INFO:teuthology.orchestra.run.smithi181.stdout:(20/33): python3-jaraco-classes-3.2.1-5.el9.noa 421 kB/s | 18 kB 00:00 2024-04-27T03:57:01.862 INFO:teuthology.orchestra.run.smithi181.stdout:(21/33): python3-jaraco-collections-3.0.0-8.el9 461 kB/s | 23 kB 00:00 2024-04-27T03:57:01.888 INFO:teuthology.orchestra.run.smithi181.stdout:(22/33): python3-jaraco-functools-3.5.0-2.el9.n 386 kB/s | 19 kB 00:00 2024-04-27T03:57:01.913 INFO:teuthology.orchestra.run.smithi181.stdout:(23/33): python3-jaraco-text-3.2.0-6.el9.noarch 392 kB/s | 20 kB 00:00 2024-04-27T03:57:01.938 INFO:teuthology.orchestra.run.smithi181.stdout:(24/33): python3-logutils-0.3.5-21.el9.noarch.r 925 kB/s | 46 kB 00:00 2024-04-27T03:57:01.955 INFO:teuthology.orchestra.run.smithi181.stdout:(25/33): python3-more-itertools-8.12.0-2.el9.no 1.8 MB/s | 79 kB 00:00 2024-04-27T03:57:01.997 INFO:teuthology.orchestra.run.smithi181.stdout:(26/33): python3-pecan-1.4.2-3.el9.noarch.rpm 4.6 MB/s | 272 kB 00:00 2024-04-27T03:57:02.022 INFO:teuthology.orchestra.run.smithi181.stdout:(27/33): python3-portend-3.1.0-2.el9.noarch.rpm 245 kB/s | 16 kB 00:00 2024-04-27T03:57:02.073 INFO:teuthology.orchestra.run.smithi181.stdout:(28/33): python3-pyOpenSSL-21.0.0-1.el9.noarch. 1.2 MB/s | 90 kB 00:00 2024-04-27T03:57:02.098 INFO:teuthology.orchestra.run.smithi181.stdout:(29/33): python3-tempora-5.0.0-2.el9.noarch.rpm 475 kB/s | 36 kB 00:00 2024-04-27T03:57:02.123 INFO:teuthology.orchestra.run.smithi181.stdout:(30/33): python3-webob-1.8.7-6.el9.noarch.rpm 4.5 MB/s | 230 kB 00:00 2024-04-27T03:57:02.157 INFO:teuthology.orchestra.run.smithi181.stdout:(31/33): python3-werkzeug-2.0.3-3.el9.1.noarch. 7.1 MB/s | 427 kB 00:00 2024-04-27T03:57:02.182 INFO:teuthology.orchestra.run.smithi181.stdout:(32/33): python3-zc-lockfile-2.0-10.el9.noarch. 344 kB/s | 20 kB 00:00 2024-04-27T03:57:02.256 INFO:teuthology.orchestra.run.smithi162.stdout:(12/12): python3-scipy-1.6.2-8.el9.x86_64.rpm 6.4 MB/s | 16 MB 00:02 2024-04-27T03:57:02.257 INFO:teuthology.orchestra.run.smithi162.stdout:-------------------------------------------------------------------------------- 2024-04-27T03:57:02.257 INFO:teuthology.orchestra.run.smithi162.stdout:Total 9.5 MB/s | 38 MB 00:03 2024-04-27T03:57:02.282 INFO:teuthology.orchestra.run.smithi181.stdout:(33/33): ceph-mon-17.2.7-907.g2be3efa9.el9.x86_ 2.1 MB/s | 3.9 MB 00:01 2024-04-27T03:57:02.286 INFO:teuthology.orchestra.run.smithi181.stdout:-------------------------------------------------------------------------------- 2024-04-27T03:57:02.286 INFO:teuthology.orchestra.run.smithi181.stdout:Total 11 MB/s | 27 MB 00:02 2024-04-27T03:57:02.394 INFO:teuthology.orchestra.run.smithi181.stdout:Running transaction check 2024-04-27T03:57:02.430 INFO:teuthology.orchestra.run.smithi181.stdout:Transaction check succeeded. 2024-04-27T03:57:02.430 INFO:teuthology.orchestra.run.smithi181.stdout:Running transaction test 2024-04-27T03:57:02.606 INFO:teuthology.orchestra.run.smithi162.stdout:Running transaction check 2024-04-27T03:57:02.633 INFO:teuthology.orchestra.run.smithi162.stdout:Transaction check succeeded. 2024-04-27T03:57:02.633 INFO:teuthology.orchestra.run.smithi162.stdout:Running transaction test 2024-04-27T03:57:02.693 INFO:teuthology.orchestra.run.smithi181.stdout:Transaction test succeeded. 2024-04-27T03:57:02.694 INFO:teuthology.orchestra.run.smithi181.stdout:Running transaction 2024-04-27T03:57:03.048 INFO:teuthology.orchestra.run.smithi162.stdout:Transaction test succeeded. 2024-04-27T03:57:03.048 INFO:teuthology.orchestra.run.smithi162.stdout:Running transaction 2024-04-27T03:57:03.810 INFO:teuthology.orchestra.run.smithi162.stdout: Preparing : 1/1 2024-04-27T03:57:03.820 INFO:teuthology.orchestra.run.smithi181.stdout: Preparing : 1/1 2024-04-27T03:57:03.877 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 1/12 2024-04-27T03:57:03.893 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 1/33 2024-04-27T03:57:03.925 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 2/33 2024-04-27T03:57:03.928 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 2/12 2024-04-27T03:57:03.958 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 3/33 2024-04-27T03:57:04.029 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 3/12 2024-04-27T03:57:04.048 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 4/33 2024-04-27T03:57:04.063 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : python3-devel-3.9.18-3.el9.x86_64 4/12 2024-04-27T03:57:04.128 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 5/33 2024-04-27T03:57:04.169 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 6/33 2024-04-27T03:57:04.212 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 7/33 2024-04-27T03:57:04.242 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : openblas-0.3.21-2.el9.x86_64 5/12 2024-04-27T03:57:04.242 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 8/33 2024-04-27T03:57:04.277 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 9/33 2024-04-27T03:57:04.279 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 6/12 2024-04-27T03:57:04.310 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 10/33 2024-04-27T03:57:04.351 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 11/33 2024-04-27T03:57:04.356 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 7/12 2024-04-27T03:57:04.385 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 12/33 2024-04-27T03:57:04.420 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 13/33 2024-04-27T03:57:04.458 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 14/33 2024-04-27T03:57:04.530 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 15/33 2024-04-27T03:57:04.668 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 16/33 2024-04-27T03:57:04.749 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 17/33 2024-04-27T03:57:04.802 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 18/33 2024-04-27T03:57:04.841 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 19/33 2024-04-27T03:57:04.890 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-ply-3.11-14.el9.noarch 20/33 2024-04-27T03:57:04.953 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 21/33 2024-04-27T03:57:05.012 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 8/12 2024-04-27T03:57:05.135 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 9/12 2024-04-27T03:57:05.137 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 22/33 2024-04-27T03:57:05.188 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 23/33 2024-04-27T03:57:05.256 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 24/33 2024-04-27T03:57:05.345 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 25/33 2024-04-27T03:57:05.386 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 26/33 2024-04-27T03:57:05.502 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 27/33 2024-04-27T03:57:05.665 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : ceph-mgr-modules-core-2:17.2.7-907.g2be3efa9.el9.n 28/33 2024-04-27T03:57:05.697 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : ceph-mgr-2:17.2.7-907.g2be3efa9.el9.x86_64 29/33 2024-04-27T03:57:05.721 INFO:teuthology.orchestra.run.smithi181.stdout: Running scriptlet: ceph-mgr-2:17.2.7-907.g2be3efa9.el9.x86_64 29/33 2024-04-27T03:57:05.721 INFO:teuthology.orchestra.run.smithi181.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-27T03:57:05.721 INFO:teuthology.orchestra.run.smithi181.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-04-27T03:57:05.721 INFO:teuthology.orchestra.run.smithi181.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-27T03:57:05.721 INFO:teuthology.orchestra.run.smithi181.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-27T03:57:05.721 INFO:teuthology.orchestra.run.smithi181.stdout: 2024-04-27T03:57:06.369 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 10/12 2024-04-27T03:57:07.031 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : ceph-osd-2:17.2.7-907.g2be3efa9.el9.x86_64 30/33 2024-04-27T03:57:07.057 INFO:teuthology.orchestra.run.smithi181.stdout: Running scriptlet: ceph-osd-2:17.2.7-907.g2be3efa9.el9.x86_64 30/33 2024-04-27T03:57:07.057 INFO:teuthology.orchestra.run.smithi181.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-27T03:57:07.057 INFO:teuthology.orchestra.run.smithi181.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-04-27T03:57:07.057 INFO:teuthology.orchestra.run.smithi181.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-27T03:57:07.058 INFO:teuthology.orchestra.run.smithi181.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-27T03:57:07.058 INFO:teuthology.orchestra.run.smithi181.stdout: 2024-04-27T03:57:07.413 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 11/12 2024-04-27T03:57:07.423 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : ceph-mon-2:17.2.7-907.g2be3efa9.el9.x86_64 31/33 2024-04-27T03:57:07.436 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : ceph-mgr-diskprediction-local-2:17.2.7-907.g2be3ef 12/12 2024-04-27T03:57:07.447 INFO:teuthology.orchestra.run.smithi181.stdout: Running scriptlet: ceph-mon-2:17.2.7-907.g2be3efa9.el9.x86_64 31/33 2024-04-27T03:57:07.447 INFO:teuthology.orchestra.run.smithi181.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-27T03:57:07.447 INFO:teuthology.orchestra.run.smithi181.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-04-27T03:57:07.447 INFO:teuthology.orchestra.run.smithi181.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-27T03:57:07.447 INFO:teuthology.orchestra.run.smithi181.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-27T03:57:07.447 INFO:teuthology.orchestra.run.smithi181.stdout: 2024-04-27T03:57:07.649 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : ceph-mds-2:17.2.7-907.g2be3efa9.el9.x86_64 32/33 2024-04-27T03:57:07.672 INFO:teuthology.orchestra.run.smithi181.stdout: Running scriptlet: ceph-mds-2:17.2.7-907.g2be3efa9.el9.x86_64 32/33 2024-04-27T03:57:07.672 INFO:teuthology.orchestra.run.smithi181.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-27T03:57:07.672 INFO:teuthology.orchestra.run.smithi181.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-04-27T03:57:07.672 INFO:teuthology.orchestra.run.smithi181.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-27T03:57:07.672 INFO:teuthology.orchestra.run.smithi181.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-27T03:57:07.673 INFO:teuthology.orchestra.run.smithi181.stdout: 2024-04-27T03:57:07.727 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : ceph-2:17.2.7-907.g2be3efa9.el9.x86_64 33/33 2024-04-27T03:57:09.342 INFO:teuthology.orchestra.run.smithi181.stdout: Running scriptlet: ceph-2:17.2.7-907.g2be3efa9.el9.x86_64 33/33 2024-04-27T03:57:09.342 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : ceph-2:17.2.7-907.g2be3efa9.el9.x86_64 1/33 2024-04-27T03:57:09.342 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : ceph-mds-2:17.2.7-907.g2be3efa9.el9.x86_64 2/33 2024-04-27T03:57:09.342 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : ceph-mgr-2:17.2.7-907.g2be3efa9.el9.x86_64 3/33 2024-04-27T03:57:09.342 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : ceph-mon-2:17.2.7-907.g2be3efa9.el9.x86_64 4/33 2024-04-27T03:57:09.342 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : ceph-osd-2:17.2.7-907.g2be3efa9.el9.x86_64 5/33 2024-04-27T03:57:09.342 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : ceph-mgr-modules-core-2:17.2.7-907.g2be3efa9.el9.n 6/33 2024-04-27T03:57:09.342 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 7/33 2024-04-27T03:57:09.342 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 8/33 2024-04-27T03:57:09.342 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-ply-3.11-14.el9.noarch 9/33 2024-04-27T03:57:09.343 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 10/33 2024-04-27T03:57:09.343 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 11/33 2024-04-27T03:57:09.343 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 12/33 2024-04-27T03:57:09.343 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 13/33 2024-04-27T03:57:09.343 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 14/33 2024-04-27T03:57:09.343 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 15/33 2024-04-27T03:57:09.343 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 16/33 2024-04-27T03:57:09.343 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 17/33 2024-04-27T03:57:09.343 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 18/33 2024-04-27T03:57:09.343 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 19/33 2024-04-27T03:57:09.343 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 20/33 2024-04-27T03:57:09.343 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 21/33 2024-04-27T03:57:09.343 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 22/33 2024-04-27T03:57:09.343 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 23/33 2024-04-27T03:57:09.344 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 24/33 2024-04-27T03:57:09.344 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 25/33 2024-04-27T03:57:09.345 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 26/33 2024-04-27T03:57:09.345 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 27/33 2024-04-27T03:57:09.345 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 28/33 2024-04-27T03:57:09.345 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 29/33 2024-04-27T03:57:09.345 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 30/33 2024-04-27T03:57:09.345 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 31/33 2024-04-27T03:57:09.345 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 32/33 2024-04-27T03:57:10.075 INFO:teuthology.orchestra.run.smithi162.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:17.2.7-907.g2be3ef 12/12 2024-04-27T03:57:10.075 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : ceph-mgr-diskprediction-local-2:17.2.7-907.g2be3ef 1/12 2024-04-27T03:57:10.075 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 2/12 2024-04-27T03:57:10.075 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 3/12 2024-04-27T03:57:10.075 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 4/12 2024-04-27T03:57:10.076 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 5/12 2024-04-27T03:57:10.076 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 6/12 2024-04-27T03:57:10.076 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 7/12 2024-04-27T03:57:10.076 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 8/12 2024-04-27T03:57:10.076 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : python3-devel-3.9.18-3.el9.x86_64 9/12 2024-04-27T03:57:10.076 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 10/12 2024-04-27T03:57:10.076 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 11/12 2024-04-27T03:57:10.216 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 33/33 2024-04-27T03:57:10.216 INFO:teuthology.orchestra.run.smithi181.stdout: 2024-04-27T03:57:10.216 INFO:teuthology.orchestra.run.smithi181.stdout:Installed: 2024-04-27T03:57:10.217 INFO:teuthology.orchestra.run.smithi181.stdout: ceph-2:17.2.7-907.g2be3efa9.el9.x86_64 2024-04-27T03:57:10.217 INFO:teuthology.orchestra.run.smithi181.stdout: ceph-mds-2:17.2.7-907.g2be3efa9.el9.x86_64 2024-04-27T03:57:10.217 INFO:teuthology.orchestra.run.smithi181.stdout: ceph-mgr-2:17.2.7-907.g2be3efa9.el9.x86_64 2024-04-27T03:57:10.217 INFO:teuthology.orchestra.run.smithi181.stdout: ceph-mgr-modules-core-2:17.2.7-907.g2be3efa9.el9.noarch 2024-04-27T03:57:10.217 INFO:teuthology.orchestra.run.smithi181.stdout: ceph-mon-2:17.2.7-907.g2be3efa9.el9.x86_64 2024-04-27T03:57:10.217 INFO:teuthology.orchestra.run.smithi181.stdout: ceph-osd-2:17.2.7-907.g2be3efa9.el9.x86_64 2024-04-27T03:57:10.217 INFO:teuthology.orchestra.run.smithi181.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-04-27T03:57:10.217 INFO:teuthology.orchestra.run.smithi181.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-04-27T03:57:10.217 INFO:teuthology.orchestra.run.smithi181.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-04-27T03:57:10.217 INFO:teuthology.orchestra.run.smithi181.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-04-27T03:57:10.217 INFO:teuthology.orchestra.run.smithi181.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-04-27T03:57:10.217 INFO:teuthology.orchestra.run.smithi181.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-04-27T03:57:10.217 INFO:teuthology.orchestra.run.smithi181.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-04-27T03:57:10.217 INFO:teuthology.orchestra.run.smithi181.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-04-27T03:57:10.217 INFO:teuthology.orchestra.run.smithi181.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-04-27T03:57:10.218 INFO:teuthology.orchestra.run.smithi181.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-04-27T03:57:10.218 INFO:teuthology.orchestra.run.smithi181.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-04-27T03:57:10.218 INFO:teuthology.orchestra.run.smithi181.stdout: python3-mako-1.1.4-6.el9.noarch 2024-04-27T03:57:10.218 INFO:teuthology.orchestra.run.smithi181.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-04-27T03:57:10.218 INFO:teuthology.orchestra.run.smithi181.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-04-27T03:57:10.218 INFO:teuthology.orchestra.run.smithi181.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-04-27T03:57:10.218 INFO:teuthology.orchestra.run.smithi181.stdout: python3-ply-3.11-14.el9.noarch 2024-04-27T03:57:10.218 INFO:teuthology.orchestra.run.smithi181.stdout: python3-portend-3.1.0-2.el9.noarch 2024-04-27T03:57:10.218 INFO:teuthology.orchestra.run.smithi181.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-04-27T03:57:10.218 INFO:teuthology.orchestra.run.smithi181.stdout: python3-pycparser-2.20-6.el9.noarch 2024-04-27T03:57:10.218 INFO:teuthology.orchestra.run.smithi181.stdout: python3-pytz-2021.1-5.el9.noarch 2024-04-27T03:57:10.218 INFO:teuthology.orchestra.run.smithi181.stdout: python3-requests-2.25.1-8.el9.noarch 2024-04-27T03:57:10.218 INFO:teuthology.orchestra.run.smithi181.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-04-27T03:57:10.219 INFO:teuthology.orchestra.run.smithi181.stdout: python3-toml-0.10.2-6.el9.noarch 2024-04-27T03:57:10.219 INFO:teuthology.orchestra.run.smithi181.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-04-27T03:57:10.219 INFO:teuthology.orchestra.run.smithi181.stdout: python3-webob-1.8.7-6.el9.noarch 2024-04-27T03:57:10.219 INFO:teuthology.orchestra.run.smithi181.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-04-27T03:57:10.219 INFO:teuthology.orchestra.run.smithi181.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-04-27T03:57:10.219 INFO:teuthology.orchestra.run.smithi181.stdout: 2024-04-27T03:57:10.219 INFO:teuthology.orchestra.run.smithi181.stdout:Complete! 2024-04-27T03:57:10.452 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 12/12 2024-04-27T03:57:10.452 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-04-27T03:57:10.452 INFO:teuthology.orchestra.run.smithi162.stdout:Installed: 2024-04-27T03:57:10.453 INFO:teuthology.orchestra.run.smithi162.stdout: ceph-mgr-diskprediction-local-2:17.2.7-907.g2be3efa9.el9.noarch 2024-04-27T03:57:10.453 INFO:teuthology.orchestra.run.smithi162.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-04-27T03:57:10.453 INFO:teuthology.orchestra.run.smithi162.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-04-27T03:57:10.453 INFO:teuthology.orchestra.run.smithi162.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-04-27T03:57:10.453 INFO:teuthology.orchestra.run.smithi162.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-04-27T03:57:10.453 INFO:teuthology.orchestra.run.smithi162.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-04-27T03:57:10.453 INFO:teuthology.orchestra.run.smithi162.stdout: openblas-0.3.21-2.el9.x86_64 2024-04-27T03:57:10.453 INFO:teuthology.orchestra.run.smithi162.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-04-27T03:57:10.453 INFO:teuthology.orchestra.run.smithi162.stdout: python3-devel-3.9.18-3.el9.x86_64 2024-04-27T03:57:10.453 INFO:teuthology.orchestra.run.smithi162.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-04-27T03:57:10.453 INFO:teuthology.orchestra.run.smithi162.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-04-27T03:57:10.453 INFO:teuthology.orchestra.run.smithi162.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-04-27T03:57:10.454 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-04-27T03:57:10.454 INFO:teuthology.orchestra.run.smithi162.stdout:Complete! 2024-04-27T03:57:10.535 DEBUG:teuthology.orchestra.run.smithi181:> sudo yum -y install ceph-base 2024-04-27T03:57:10.780 DEBUG:teuthology.orchestra.run.smithi162:> sudo yum -y install ceph-mgr-rook 2024-04-27T03:57:11.043 INFO:teuthology.orchestra.run.smithi181.stdout:Last metadata expiration check: 0:01:06 ago on Sat 27 Apr 2024 03:56:05 AM UTC. 2024-04-27T03:57:11.122 INFO:teuthology.orchestra.run.smithi181.stdout:Package ceph-base-2:17.2.7-907.g2be3efa9.el9.x86_64 is already installed. 2024-04-27T03:57:11.163 INFO:teuthology.orchestra.run.smithi181.stdout:Dependencies resolved. 2024-04-27T03:57:11.164 INFO:teuthology.orchestra.run.smithi181.stdout:Nothing to do. 2024-04-27T03:57:11.164 INFO:teuthology.orchestra.run.smithi181.stdout:Complete! 2024-04-27T03:57:11.236 DEBUG:teuthology.orchestra.run.smithi181:> sudo yum -y install cephadm 2024-04-27T03:57:11.301 INFO:teuthology.orchestra.run.smithi162.stdout:Last metadata expiration check: 0:01:18 ago on Sat 27 Apr 2024 03:55:53 AM UTC. 2024-04-27T03:57:11.423 INFO:teuthology.orchestra.run.smithi162.stdout:Dependencies resolved. 2024-04-27T03:57:11.425 INFO:teuthology.orchestra.run.smithi162.stdout:================================================================================ 2024-04-27T03:57:11.425 INFO:teuthology.orchestra.run.smithi162.stdout: Package Arch Version Repository Size 2024-04-27T03:57:11.425 INFO:teuthology.orchestra.run.smithi162.stdout:================================================================================ 2024-04-27T03:57:11.425 INFO:teuthology.orchestra.run.smithi162.stdout:Installing: 2024-04-27T03:57:11.425 INFO:teuthology.orchestra.run.smithi162.stdout: ceph-mgr-rook noarch 2:17.2.7-907.g2be3efa9.el9 ceph-noarch 47 k 2024-04-27T03:57:11.425 INFO:teuthology.orchestra.run.smithi162.stdout:Installing dependencies: 2024-04-27T03:57:11.425 INFO:teuthology.orchestra.run.smithi162.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-04-27T03:57:11.425 INFO:teuthology.orchestra.run.smithi162.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-04-27T03:57:11.425 INFO:teuthology.orchestra.run.smithi162.stdout: python3-google-auth noarch 1:2.29.0-1.el9 epel 206 k 2024-04-27T03:57:11.425 INFO:teuthology.orchestra.run.smithi162.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-04-27T03:57:11.425 INFO:teuthology.orchestra.run.smithi162.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-04-27T03:57:11.425 INFO:teuthology.orchestra.run.smithi162.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-04-27T03:57:11.425 INFO:teuthology.orchestra.run.smithi162.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-04-27T03:57:11.426 INFO:teuthology.orchestra.run.smithi162.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-04-27T03:57:11.426 INFO:teuthology.orchestra.run.smithi162.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-04-27T03:57:11.426 INFO:teuthology.orchestra.run.smithi162.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-04-27T03:57:11.426 INFO:teuthology.orchestra.run.smithi162.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-04-27T03:57:11.426 INFO:teuthology.orchestra.run.smithi162.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-04-27T03:57:11.426 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-04-27T03:57:11.426 INFO:teuthology.orchestra.run.smithi162.stdout:Transaction Summary 2024-04-27T03:57:11.426 INFO:teuthology.orchestra.run.smithi162.stdout:================================================================================ 2024-04-27T03:57:11.426 INFO:teuthology.orchestra.run.smithi162.stdout:Install 13 Packages 2024-04-27T03:57:11.426 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-04-27T03:57:11.427 INFO:teuthology.orchestra.run.smithi162.stdout:Total download size: 2.2 M 2024-04-27T03:57:11.427 INFO:teuthology.orchestra.run.smithi162.stdout:Installed size: 26 M 2024-04-27T03:57:11.427 INFO:teuthology.orchestra.run.smithi162.stdout:Downloading Packages: 2024-04-27T03:57:11.741 INFO:teuthology.orchestra.run.smithi181.stdout:Last metadata expiration check: 0:01:06 ago on Sat 27 Apr 2024 03:56:05 AM UTC. 2024-04-27T03:57:11.860 INFO:teuthology.orchestra.run.smithi181.stdout:Dependencies resolved. 2024-04-27T03:57:11.860 INFO:teuthology.orchestra.run.smithi181.stdout:================================================================================ 2024-04-27T03:57:11.861 INFO:teuthology.orchestra.run.smithi181.stdout: Package Arch Version Repository Size 2024-04-27T03:57:11.861 INFO:teuthology.orchestra.run.smithi181.stdout:================================================================================ 2024-04-27T03:57:11.861 INFO:teuthology.orchestra.run.smithi181.stdout:Installing: 2024-04-27T03:57:11.861 INFO:teuthology.orchestra.run.smithi181.stdout: cephadm noarch 2:17.2.7-907.g2be3efa9.el9 ceph-noarch 82 k 2024-04-27T03:57:11.861 INFO:teuthology.orchestra.run.smithi181.stdout: 2024-04-27T03:57:11.861 INFO:teuthology.orchestra.run.smithi181.stdout:Transaction Summary 2024-04-27T03:57:11.861 INFO:teuthology.orchestra.run.smithi181.stdout:================================================================================ 2024-04-27T03:57:11.861 INFO:teuthology.orchestra.run.smithi181.stdout:Install 1 Package 2024-04-27T03:57:11.861 INFO:teuthology.orchestra.run.smithi181.stdout: 2024-04-27T03:57:11.861 INFO:teuthology.orchestra.run.smithi181.stdout:Total download size: 82 k 2024-04-27T03:57:11.861 INFO:teuthology.orchestra.run.smithi181.stdout:Installed size: 372 k 2024-04-27T03:57:11.861 INFO:teuthology.orchestra.run.smithi181.stdout:Downloading Packages: 2024-04-27T03:57:12.019 INFO:teuthology.orchestra.run.smithi162.stdout:(1/13): python3-jsonpatch-1.21-16.el9.noarch.rp 266 kB/s | 26 kB 00:00 2024-04-27T03:57:12.044 INFO:teuthology.orchestra.run.smithi162.stdout:(2/13): python3-jsonpointer-2.0-4.el9.noarch.rp 158 kB/s | 19 kB 00:00 2024-04-27T03:57:12.128 INFO:teuthology.orchestra.run.smithi162.stdout:(3/13): python3-oauthlib-3.1.1-5.el9.noarch.rpm 2.0 MB/s | 222 kB 00:00 2024-04-27T03:57:12.153 INFO:teuthology.orchestra.run.smithi162.stdout:(4/13): python3-pyasn1-0.4.8-6.el9.noarch.rpm 1.4 MB/s | 159 kB 00:00 2024-04-27T03:57:12.172 INFO:teuthology.orchestra.run.smithi181.stdout:cephadm-17.2.7-907.g2be3efa9.el9.noarch.rpm 264 kB/s | 82 kB 00:00 2024-04-27T03:57:12.173 INFO:teuthology.orchestra.run.smithi181.stdout:-------------------------------------------------------------------------------- 2024-04-27T03:57:12.173 INFO:teuthology.orchestra.run.smithi181.stdout:Total 263 kB/s | 82 kB 00:00 2024-04-27T03:57:12.173 INFO:teuthology.orchestra.run.smithi181.stdout:Running transaction check 2024-04-27T03:57:12.178 INFO:teuthology.orchestra.run.smithi181.stdout:Transaction check succeeded. 2024-04-27T03:57:12.178 INFO:teuthology.orchestra.run.smithi181.stdout:Running transaction test 2024-04-27T03:57:12.180 INFO:teuthology.orchestra.run.smithi181.stdout:Transaction test succeeded. 2024-04-27T03:57:12.180 INFO:teuthology.orchestra.run.smithi181.stdout:Running transaction 2024-04-27T03:57:12.195 INFO:teuthology.orchestra.run.smithi162.stdout:(5/13): python3-pyasn1-modules-0.4.8-6.el9.noar 4.1 MB/s | 279 kB 00:00 2024-04-27T03:57:12.220 INFO:teuthology.orchestra.run.smithi162.stdout:(6/13): ceph-mgr-rook-17.2.7-907.g2be3efa9.el9. 160 kB/s | 47 kB 00:00 2024-04-27T03:57:12.245 INFO:teuthology.orchestra.run.smithi162.stdout:(7/13): python3-requests-oauthlib-1.3.0-12.el9. 583 kB/s | 54 kB 00:00 2024-04-27T03:57:12.252 INFO:teuthology.orchestra.run.smithi181.stdout: Preparing : 1/1 2024-04-27T03:57:12.496 INFO:teuthology.orchestra.run.smithi162.stdout:(8/13): python3-certifi-2021.10.8-2.el9.noarch. 54 kB/s | 15 kB 00:00 2024-04-27T03:57:12.521 INFO:teuthology.orchestra.run.smithi162.stdout:(9/13): python3-cachetools-4.2.4-1.el9.noarch.r 99 kB/s | 32 kB 00:00 2024-04-27T03:57:12.720 INFO:teuthology.orchestra.run.smithi181.stdout: Running scriptlet: cephadm-2:17.2.7-907.g2be3efa9.el9.noarch 1/1 2024-04-27T03:57:12.755 INFO:teuthology.orchestra.run.smithi162.stdout:(10/13): python3-google-auth-2.29.0-1.el9.noarc 404 kB/s | 206 kB 00:00 2024-04-27T03:57:12.816 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : cephadm-2:17.2.7-907.g2be3efa9.el9.noarch 1/1 2024-04-27T03:57:12.871 INFO:teuthology.orchestra.run.smithi162.stdout:(11/13): python3-rsa-4.9-2.el9.noarch.rpm 169 kB/s | 59 kB 00:00 2024-04-27T03:57:13.097 INFO:teuthology.orchestra.run.smithi162.stdout:(12/13): python3-websocket-client-1.2.3-2.el9.n 262 kB/s | 90 kB 00:00 2024-04-27T03:57:13.111 INFO:teuthology.orchestra.run.smithi181.stdout: Running scriptlet: cephadm-2:17.2.7-907.g2be3efa9.el9.noarch 1/1 2024-04-27T03:57:13.247 INFO:teuthology.orchestra.run.smithi162.stdout:(13/13): python3-kubernetes-26.1.0-2.el9.noarch 1.4 MB/s | 1.0 MB 00:00 2024-04-27T03:57:13.248 INFO:teuthology.orchestra.run.smithi162.stdout:-------------------------------------------------------------------------------- 2024-04-27T03:57:13.248 INFO:teuthology.orchestra.run.smithi162.stdout:Total 1.2 MB/s | 2.2 MB 00:01 2024-04-27T03:57:13.315 INFO:teuthology.orchestra.run.smithi162.stdout:Running transaction check 2024-04-27T03:57:13.331 INFO:teuthology.orchestra.run.smithi162.stdout:Transaction check succeeded. 2024-04-27T03:57:13.331 INFO:teuthology.orchestra.run.smithi162.stdout:Running transaction test 2024-04-27T03:57:13.347 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : cephadm-2:17.2.7-907.g2be3efa9.el9.noarch 1/1 2024-04-27T03:57:13.347 INFO:teuthology.orchestra.run.smithi181.stdout: 2024-04-27T03:57:13.347 INFO:teuthology.orchestra.run.smithi181.stdout:Installed: 2024-04-27T03:57:13.347 INFO:teuthology.orchestra.run.smithi181.stdout: cephadm-2:17.2.7-907.g2be3efa9.el9.noarch 2024-04-27T03:57:13.347 INFO:teuthology.orchestra.run.smithi181.stdout: 2024-04-27T03:57:13.347 INFO:teuthology.orchestra.run.smithi181.stdout:Complete! 2024-04-27T03:57:13.415 INFO:teuthology.orchestra.run.smithi162.stdout:Transaction test succeeded. 2024-04-27T03:57:13.416 INFO:teuthology.orchestra.run.smithi162.stdout:Running transaction 2024-04-27T03:57:13.551 DEBUG:teuthology.orchestra.run.smithi181:> sudo yum -y install ceph-immutable-object-cache 2024-04-27T03:57:14.059 INFO:teuthology.orchestra.run.smithi162.stdout: Preparing : 1/1 2024-04-27T03:57:14.068 INFO:teuthology.orchestra.run.smithi181.stdout:Last metadata expiration check: 0:01:09 ago on Sat 27 Apr 2024 03:56:05 AM UTC. 2024-04-27T03:57:14.189 INFO:teuthology.orchestra.run.smithi181.stdout:Dependencies resolved. 2024-04-27T03:57:14.189 INFO:teuthology.orchestra.run.smithi181.stdout:================================================================================ 2024-04-27T03:57:14.189 INFO:teuthology.orchestra.run.smithi181.stdout: Package Arch Version Repo Size 2024-04-27T03:57:14.190 INFO:teuthology.orchestra.run.smithi181.stdout:================================================================================ 2024-04-27T03:57:14.190 INFO:teuthology.orchestra.run.smithi181.stdout:Installing: 2024-04-27T03:57:14.190 INFO:teuthology.orchestra.run.smithi181.stdout: ceph-immutable-object-cache x86_64 2:17.2.7-907.g2be3efa9.el9 ceph 137 k 2024-04-27T03:57:14.190 INFO:teuthology.orchestra.run.smithi181.stdout: 2024-04-27T03:57:14.190 INFO:teuthology.orchestra.run.smithi181.stdout:Transaction Summary 2024-04-27T03:57:14.190 INFO:teuthology.orchestra.run.smithi181.stdout:================================================================================ 2024-04-27T03:57:14.190 INFO:teuthology.orchestra.run.smithi181.stdout:Install 1 Package 2024-04-27T03:57:14.190 INFO:teuthology.orchestra.run.smithi181.stdout: 2024-04-27T03:57:14.191 INFO:teuthology.orchestra.run.smithi181.stdout:Total download size: 137 k 2024-04-27T03:57:14.191 INFO:teuthology.orchestra.run.smithi181.stdout:Installed size: 379 k 2024-04-27T03:57:14.191 INFO:teuthology.orchestra.run.smithi181.stdout:Downloading Packages: 2024-04-27T03:57:14.229 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 1/13 2024-04-27T03:57:14.282 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 2/13 2024-04-27T03:57:14.337 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : python3-rsa-4.9-2.el9.noarch 3/13 2024-04-27T03:57:14.374 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 4/13 2024-04-27T03:57:14.416 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 5/13 2024-04-27T03:57:14.495 INFO:teuthology.orchestra.run.smithi181.stdout:ceph-immutable-object-cache-17.2.7-907.g2be3efa 453 kB/s | 137 kB 00:00 2024-04-27T03:57:14.495 INFO:teuthology.orchestra.run.smithi181.stdout:-------------------------------------------------------------------------------- 2024-04-27T03:57:14.496 INFO:teuthology.orchestra.run.smithi181.stdout:Total 450 kB/s | 137 kB 00:00 2024-04-27T03:57:14.496 INFO:teuthology.orchestra.run.smithi181.stdout:Running transaction check 2024-04-27T03:57:14.503 INFO:teuthology.orchestra.run.smithi181.stdout:Transaction check succeeded. 2024-04-27T03:57:14.503 INFO:teuthology.orchestra.run.smithi181.stdout:Running transaction test 2024-04-27T03:57:14.509 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 6/13 2024-04-27T03:57:14.565 INFO:teuthology.orchestra.run.smithi181.stdout:Transaction test succeeded. 2024-04-27T03:57:14.565 INFO:teuthology.orchestra.run.smithi181.stdout:Running transaction 2024-04-27T03:57:14.619 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : python3-google-auth-1:2.29.0-1.el9.noarch 7/13 2024-04-27T03:57:14.667 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 8/13 2024-04-27T03:57:14.718 INFO:teuthology.orchestra.run.smithi181.stdout: Preparing : 1/1 2024-04-27T03:57:14.804 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : ceph-immutable-object-cache-2:17.2.7-907.g2be3efa9.e 1/1 2024-04-27T03:57:14.830 INFO:teuthology.orchestra.run.smithi181.stdout: Running scriptlet: ceph-immutable-object-cache-2:17.2.7-907.g2be3efa9.e 1/1 2024-04-27T03:57:14.830 INFO:teuthology.orchestra.run.smithi181.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-27T03:57:14.830 INFO:teuthology.orchestra.run.smithi181.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-04-27T03:57:14.830 INFO:teuthology.orchestra.run.smithi181.stdout: 2024-04-27T03:57:15.390 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 9/13 2024-04-27T03:57:15.439 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 10/13 2024-04-27T03:57:15.473 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 11/13 2024-04-27T03:57:15.519 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 12/13 2024-04-27T03:57:15.543 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : ceph-mgr-rook-2:17.2.7-907.g2be3efa9.el9.noarch 13/13 2024-04-27T03:57:15.659 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : ceph-immutable-object-cache-2:17.2.7-907.g2be3efa9.e 1/1 2024-04-27T03:57:15.659 INFO:teuthology.orchestra.run.smithi181.stdout: 2024-04-27T03:57:15.659 INFO:teuthology.orchestra.run.smithi181.stdout:Installed: 2024-04-27T03:57:15.659 INFO:teuthology.orchestra.run.smithi181.stdout: ceph-immutable-object-cache-2:17.2.7-907.g2be3efa9.el9.x86_64 2024-04-27T03:57:15.659 INFO:teuthology.orchestra.run.smithi181.stdout: 2024-04-27T03:57:15.659 INFO:teuthology.orchestra.run.smithi181.stdout:Complete! 2024-04-27T03:57:15.864 DEBUG:teuthology.orchestra.run.smithi181:> sudo yum -y install ceph-mgr 2024-04-27T03:57:16.226 INFO:teuthology.orchestra.run.smithi162.stdout: Running scriptlet: ceph-mgr-rook-2:17.2.7-907.g2be3efa9.el9.noarch 13/13 2024-04-27T03:57:16.226 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : ceph-mgr-rook-2:17.2.7-907.g2be3efa9.el9.noarch 1/13 2024-04-27T03:57:16.227 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 2/13 2024-04-27T03:57:16.227 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 3/13 2024-04-27T03:57:16.227 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 4/13 2024-04-27T03:57:16.227 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 5/13 2024-04-27T03:57:16.227 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 6/13 2024-04-27T03:57:16.227 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 7/13 2024-04-27T03:57:16.227 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 8/13 2024-04-27T03:57:16.227 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 9/13 2024-04-27T03:57:16.227 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : python3-google-auth-1:2.29.0-1.el9.noarch 10/13 2024-04-27T03:57:16.227 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 11/13 2024-04-27T03:57:16.227 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 12/13 2024-04-27T03:57:16.377 INFO:teuthology.orchestra.run.smithi181.stdout:Last metadata expiration check: 0:01:11 ago on Sat 27 Apr 2024 03:56:05 AM UTC. 2024-04-27T03:57:16.457 INFO:teuthology.orchestra.run.smithi181.stdout:Package ceph-mgr-2:17.2.7-907.g2be3efa9.el9.x86_64 is already installed. 2024-04-27T03:57:16.499 INFO:teuthology.orchestra.run.smithi181.stdout:Dependencies resolved. 2024-04-27T03:57:16.500 INFO:teuthology.orchestra.run.smithi181.stdout:Nothing to do. 2024-04-27T03:57:16.500 INFO:teuthology.orchestra.run.smithi181.stdout:Complete! 2024-04-27T03:57:16.523 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 13/13 2024-04-27T03:57:16.524 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-04-27T03:57:16.524 INFO:teuthology.orchestra.run.smithi162.stdout:Installed: 2024-04-27T03:57:16.524 INFO:teuthology.orchestra.run.smithi162.stdout: ceph-mgr-rook-2:17.2.7-907.g2be3efa9.el9.noarch 2024-04-27T03:57:16.524 INFO:teuthology.orchestra.run.smithi162.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-04-27T03:57:16.524 INFO:teuthology.orchestra.run.smithi162.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-04-27T03:57:16.524 INFO:teuthology.orchestra.run.smithi162.stdout: python3-google-auth-1:2.29.0-1.el9.noarch 2024-04-27T03:57:16.524 INFO:teuthology.orchestra.run.smithi162.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-04-27T03:57:16.524 INFO:teuthology.orchestra.run.smithi162.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-04-27T03:57:16.524 INFO:teuthology.orchestra.run.smithi162.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-04-27T03:57:16.524 INFO:teuthology.orchestra.run.smithi162.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-04-27T03:57:16.524 INFO:teuthology.orchestra.run.smithi162.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-04-27T03:57:16.524 INFO:teuthology.orchestra.run.smithi162.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-04-27T03:57:16.524 INFO:teuthology.orchestra.run.smithi162.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-04-27T03:57:16.524 INFO:teuthology.orchestra.run.smithi162.stdout: python3-rsa-4.9-2.el9.noarch 2024-04-27T03:57:16.525 INFO:teuthology.orchestra.run.smithi162.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-04-27T03:57:16.525 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-04-27T03:57:16.525 INFO:teuthology.orchestra.run.smithi162.stdout:Complete! 2024-04-27T03:57:16.570 DEBUG:teuthology.orchestra.run.smithi181:> sudo yum -y install ceph-mgr-dashboard 2024-04-27T03:57:16.751 DEBUG:teuthology.orchestra.run.smithi162:> sudo yum -y install ceph-mgr-cephadm 2024-04-27T03:57:17.075 INFO:teuthology.orchestra.run.smithi181.stdout:Last metadata expiration check: 0:01:12 ago on Sat 27 Apr 2024 03:56:05 AM UTC. 2024-04-27T03:57:17.196 INFO:teuthology.orchestra.run.smithi181.stdout:Dependencies resolved. 2024-04-27T03:57:17.197 INFO:teuthology.orchestra.run.smithi181.stdout:================================================================================ 2024-04-27T03:57:17.197 INFO:teuthology.orchestra.run.smithi181.stdout: Package Arch Version Repository Size 2024-04-27T03:57:17.197 INFO:teuthology.orchestra.run.smithi181.stdout:================================================================================ 2024-04-27T03:57:17.197 INFO:teuthology.orchestra.run.smithi181.stdout:Installing: 2024-04-27T03:57:17.198 INFO:teuthology.orchestra.run.smithi181.stdout: ceph-mgr-dashboard noarch 2:17.2.7-907.g2be3efa9.el9 ceph-noarch 3.4 M 2024-04-27T03:57:17.198 INFO:teuthology.orchestra.run.smithi181.stdout:Installing dependencies: 2024-04-27T03:57:17.198 INFO:teuthology.orchestra.run.smithi181.stdout: ceph-grafana-dashboards noarch 2:17.2.7-907.g2be3efa9.el9 ceph-noarch 24 k 2024-04-27T03:57:17.198 INFO:teuthology.orchestra.run.smithi181.stdout: ceph-prometheus-alerts noarch 2:17.2.7-907.g2be3efa9.el9 ceph-noarch 15 k 2024-04-27T03:57:17.198 INFO:teuthology.orchestra.run.smithi181.stdout: python3-jwt noarch 2.4.0-1.el9 epel 41 k 2024-04-27T03:57:17.199 INFO:teuthology.orchestra.run.smithi181.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-04-27T03:57:17.199 INFO:teuthology.orchestra.run.smithi181.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-04-27T03:57:17.199 INFO:teuthology.orchestra.run.smithi181.stdout:Installing weak dependencies: 2024-04-27T03:57:17.199 INFO:teuthology.orchestra.run.smithi181.stdout: python3-jwt+crypto noarch 2.4.0-1.el9 epel 9.0 k 2024-04-27T03:57:17.199 INFO:teuthology.orchestra.run.smithi181.stdout: 2024-04-27T03:57:17.199 INFO:teuthology.orchestra.run.smithi181.stdout:Transaction Summary 2024-04-27T03:57:17.200 INFO:teuthology.orchestra.run.smithi181.stdout:================================================================================ 2024-04-27T03:57:17.200 INFO:teuthology.orchestra.run.smithi181.stdout:Install 7 Packages 2024-04-27T03:57:17.200 INFO:teuthology.orchestra.run.smithi181.stdout: 2024-04-27T03:57:17.201 INFO:teuthology.orchestra.run.smithi181.stdout:Total download size: 3.7 M 2024-04-27T03:57:17.201 INFO:teuthology.orchestra.run.smithi181.stdout:Installed size: 80 M 2024-04-27T03:57:17.201 INFO:teuthology.orchestra.run.smithi181.stdout:Downloading Packages: 2024-04-27T03:57:17.259 INFO:teuthology.orchestra.run.smithi162.stdout:Last metadata expiration check: 0:01:24 ago on Sat 27 Apr 2024 03:55:53 AM UTC. 2024-04-27T03:57:17.379 INFO:teuthology.orchestra.run.smithi162.stdout:Dependencies resolved. 2024-04-27T03:57:17.380 INFO:teuthology.orchestra.run.smithi162.stdout:================================================================================ 2024-04-27T03:57:17.380 INFO:teuthology.orchestra.run.smithi162.stdout: Package Arch Version Repository Size 2024-04-27T03:57:17.380 INFO:teuthology.orchestra.run.smithi162.stdout:================================================================================ 2024-04-27T03:57:17.380 INFO:teuthology.orchestra.run.smithi162.stdout:Installing: 2024-04-27T03:57:17.380 INFO:teuthology.orchestra.run.smithi162.stdout: ceph-mgr-cephadm noarch 2:17.2.7-907.g2be3efa9.el9 ceph-noarch 123 k 2024-04-27T03:57:17.380 INFO:teuthology.orchestra.run.smithi162.stdout:Installing dependencies: 2024-04-27T03:57:17.381 INFO:teuthology.orchestra.run.smithi162.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-04-27T03:57:17.381 INFO:teuthology.orchestra.run.smithi162.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-04-27T03:57:17.381 INFO:teuthology.orchestra.run.smithi162.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-04-27T03:57:17.381 INFO:teuthology.orchestra.run.smithi162.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-04-27T03:57:17.381 INFO:teuthology.orchestra.run.smithi162.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-04-27T03:57:17.381 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-04-27T03:57:17.381 INFO:teuthology.orchestra.run.smithi162.stdout:Transaction Summary 2024-04-27T03:57:17.381 INFO:teuthology.orchestra.run.smithi162.stdout:================================================================================ 2024-04-27T03:57:17.381 INFO:teuthology.orchestra.run.smithi162.stdout:Install 6 Packages 2024-04-27T03:57:17.381 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-04-27T03:57:17.382 INFO:teuthology.orchestra.run.smithi162.stdout:Total download size: 7.0 M 2024-04-27T03:57:17.382 INFO:teuthology.orchestra.run.smithi162.stdout:Installed size: 33 M 2024-04-27T03:57:17.382 INFO:teuthology.orchestra.run.smithi162.stdout:Downloading Packages: 2024-04-27T03:57:17.548 INFO:teuthology.orchestra.run.smithi181.stdout:(1/7): ceph-prometheus-alerts-17.2.7-907.g2be3e 72 kB/s | 15 kB 00:00 2024-04-27T03:57:17.641 INFO:teuthology.orchestra.run.smithi181.stdout:(2/7): ceph-grafana-dashboards-17.2.7-907.g2be3 83 kB/s | 24 kB 00:00 2024-04-27T03:57:17.957 INFO:teuthology.orchestra.run.smithi181.stdout:(3/7): python3-jwt+crypto-2.4.0-1.el9.noarch.rp 22 kB/s | 9.0 kB 00:00 2024-04-27T03:57:18.015 INFO:teuthology.orchestra.run.smithi162.stdout:(1/6): python3-jinja2-2.11.3-5.el9.noarch.rpm 1.6 MB/s | 248 kB 00:00 2024-04-27T03:57:18.058 INFO:teuthology.orchestra.run.smithi181.stdout:(4/7): ceph-mgr-dashboard-17.2.7-907.g2be3efa9. 4.7 MB/s | 3.4 MB 00:00 2024-04-27T03:57:18.090 INFO:teuthology.orchestra.run.smithi162.stdout:(2/6): ceph-mgr-cephadm-17.2.7-907.g2be3efa9.el 536 kB/s | 123 kB 00:00 2024-04-27T03:57:18.149 INFO:teuthology.orchestra.run.smithi181.stdout:(5/7): python3-repoze-lru-0.7-16.el9.noarch.rpm 160 kB/s | 31 kB 00:00 2024-04-27T03:57:18.225 INFO:teuthology.orchestra.run.smithi181.stdout:(6/7): python3-jwt-2.4.0-1.el9.noarch.rpm 73 kB/s | 41 kB 00:00 2024-04-27T03:57:18.265 INFO:teuthology.orchestra.run.smithi162.stdout:(3/6): python3-babel-2.9.1-2.el9.noarch.rpm 15 MB/s | 6.0 MB 00:00 2024-04-27T03:57:18.557 INFO:teuthology.orchestra.run.smithi162.stdout:(4/6): python3-natsort-7.1.1-5.el9.noarch.rpm 124 kB/s | 58 kB 00:00 2024-04-27T03:57:18.599 INFO:teuthology.orchestra.run.smithi162.stdout:(5/6): python3-typing-extensions-4.4.0-2.el9.no 153 kB/s | 51 kB 00:00 2024-04-27T03:57:18.674 INFO:teuthology.orchestra.run.smithi162.stdout:(6/6): python3-asyncssh-2.13.2-5.el9.noarch.rpm 831 kB/s | 548 kB 00:00 2024-04-27T03:57:18.675 INFO:teuthology.orchestra.run.smithi162.stdout:-------------------------------------------------------------------------------- 2024-04-27T03:57:18.675 INFO:teuthology.orchestra.run.smithi162.stdout:Total 5.4 MB/s | 7.0 MB 00:01 2024-04-27T03:57:18.745 INFO:teuthology.orchestra.run.smithi162.stdout:Running transaction check 2024-04-27T03:57:18.757 INFO:teuthology.orchestra.run.smithi162.stdout:Transaction check succeeded. 2024-04-27T03:57:18.757 INFO:teuthology.orchestra.run.smithi162.stdout:Running transaction test 2024-04-27T03:57:18.758 INFO:teuthology.orchestra.run.smithi181.stdout:(7/7): python3-routes-2.5.1-5.el9.noarch.rpm 269 kB/s | 188 kB 00:00 2024-04-27T03:57:18.763 INFO:teuthology.orchestra.run.smithi181.stdout:-------------------------------------------------------------------------------- 2024-04-27T03:57:18.763 INFO:teuthology.orchestra.run.smithi181.stdout:Total 2.3 MB/s | 3.7 MB 00:01 2024-04-27T03:57:18.763 INFO:teuthology.orchestra.run.smithi181.stdout:Running transaction check 2024-04-27T03:57:18.772 INFO:teuthology.orchestra.run.smithi181.stdout:Transaction check succeeded. 2024-04-27T03:57:18.772 INFO:teuthology.orchestra.run.smithi181.stdout:Running transaction test 2024-04-27T03:57:18.834 INFO:teuthology.orchestra.run.smithi181.stdout:Transaction test succeeded. 2024-04-27T03:57:18.836 INFO:teuthology.orchestra.run.smithi181.stdout:Running transaction 2024-04-27T03:57:18.854 INFO:teuthology.orchestra.run.smithi162.stdout:Transaction test succeeded. 2024-04-27T03:57:18.855 INFO:teuthology.orchestra.run.smithi162.stdout:Running transaction 2024-04-27T03:57:19.136 INFO:teuthology.orchestra.run.smithi181.stdout: Preparing : 1/1 2024-04-27T03:57:19.154 INFO:teuthology.orchestra.run.smithi162.stdout: Preparing : 1/1 2024-04-27T03:57:19.209 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-jwt-2.4.0-1.el9.noarch 1/7 2024-04-27T03:57:19.244 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-jwt+crypto-2.4.0-1.el9.noarch 2/7 2024-04-27T03:57:19.290 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 3/7 2024-04-27T03:57:19.322 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 1/6 2024-04-27T03:57:19.322 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 4/7 2024-04-27T03:57:19.350 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : ceph-prometheus-alerts-2:17.2.7-907.g2be3efa9.el9.no 5/7 2024-04-27T03:57:19.384 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 2/6 2024-04-27T03:57:19.905 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 3/6 2024-04-27T03:57:20.003 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 4/6 2024-04-27T03:57:20.068 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 5/6 2024-04-27T03:57:20.096 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : ceph-mgr-cephadm-2:17.2.7-907.g2be3efa9.el9.noarch 6/6 2024-04-27T03:57:20.358 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : ceph-grafana-dashboards-2:17.2.7-907.g2be3efa9.el9.n 6/7 2024-04-27T03:57:20.388 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : ceph-mgr-dashboard-2:17.2.7-907.g2be3efa9.el9.noarch 7/7 2024-04-27T03:57:20.872 INFO:teuthology.orchestra.run.smithi162.stdout: Running scriptlet: ceph-mgr-cephadm-2:17.2.7-907.g2be3efa9.el9.noarch 6/6 2024-04-27T03:57:20.872 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : ceph-mgr-cephadm-2:17.2.7-907.g2be3efa9.el9.noarch 1/6 2024-04-27T03:57:20.872 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 2/6 2024-04-27T03:57:20.872 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 3/6 2024-04-27T03:57:20.872 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 4/6 2024-04-27T03:57:20.873 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 5/6 2024-04-27T03:57:21.121 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 6/6 2024-04-27T03:57:21.121 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-04-27T03:57:21.121 INFO:teuthology.orchestra.run.smithi162.stdout:Installed: 2024-04-27T03:57:21.121 INFO:teuthology.orchestra.run.smithi162.stdout: ceph-mgr-cephadm-2:17.2.7-907.g2be3efa9.el9.noarch 2024-04-27T03:57:21.121 INFO:teuthology.orchestra.run.smithi162.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-04-27T03:57:21.122 INFO:teuthology.orchestra.run.smithi162.stdout: python3-babel-2.9.1-2.el9.noarch 2024-04-27T03:57:21.122 INFO:teuthology.orchestra.run.smithi162.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-04-27T03:57:21.122 INFO:teuthology.orchestra.run.smithi162.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-04-27T03:57:21.122 INFO:teuthology.orchestra.run.smithi162.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-04-27T03:57:21.122 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-04-27T03:57:21.122 INFO:teuthology.orchestra.run.smithi162.stdout:Complete! 2024-04-27T03:57:21.346 DEBUG:teuthology.orchestra.run.smithi162:> sudo yum -y install ceph-fuse 2024-04-27T03:57:21.378 INFO:teuthology.orchestra.run.smithi181.stdout: Running scriptlet: ceph-mgr-dashboard-2:17.2.7-907.g2be3efa9.el9.noarch 7/7 2024-04-27T03:57:21.378 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : ceph-grafana-dashboards-2:17.2.7-907.g2be3efa9.el9.n 1/7 2024-04-27T03:57:21.379 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : ceph-mgr-dashboard-2:17.2.7-907.g2be3efa9.el9.noarch 2/7 2024-04-27T03:57:21.379 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : ceph-prometheus-alerts-2:17.2.7-907.g2be3efa9.el9.no 3/7 2024-04-27T03:57:21.379 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-jwt+crypto-2.4.0-1.el9.noarch 4/7 2024-04-27T03:57:21.379 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-jwt-2.4.0-1.el9.noarch 5/7 2024-04-27T03:57:21.379 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 6/7 2024-04-27T03:57:21.617 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 7/7 2024-04-27T03:57:21.617 INFO:teuthology.orchestra.run.smithi181.stdout: 2024-04-27T03:57:21.617 INFO:teuthology.orchestra.run.smithi181.stdout:Installed: 2024-04-27T03:57:21.618 INFO:teuthology.orchestra.run.smithi181.stdout: ceph-grafana-dashboards-2:17.2.7-907.g2be3efa9.el9.noarch 2024-04-27T03:57:21.618 INFO:teuthology.orchestra.run.smithi181.stdout: ceph-mgr-dashboard-2:17.2.7-907.g2be3efa9.el9.noarch 2024-04-27T03:57:21.618 INFO:teuthology.orchestra.run.smithi181.stdout: ceph-prometheus-alerts-2:17.2.7-907.g2be3efa9.el9.noarch 2024-04-27T03:57:21.618 INFO:teuthology.orchestra.run.smithi181.stdout: python3-jwt-2.4.0-1.el9.noarch 2024-04-27T03:57:21.618 INFO:teuthology.orchestra.run.smithi181.stdout: python3-jwt+crypto-2.4.0-1.el9.noarch 2024-04-27T03:57:21.618 INFO:teuthology.orchestra.run.smithi181.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-04-27T03:57:21.618 INFO:teuthology.orchestra.run.smithi181.stdout: python3-routes-2.5.1-5.el9.noarch 2024-04-27T03:57:21.618 INFO:teuthology.orchestra.run.smithi181.stdout: 2024-04-27T03:57:21.618 INFO:teuthology.orchestra.run.smithi181.stdout:Complete! 2024-04-27T03:57:21.827 DEBUG:teuthology.orchestra.run.smithi181:> sudo yum -y install ceph-mgr-diskprediction-local 2024-04-27T03:57:21.853 INFO:teuthology.orchestra.run.smithi162.stdout:Last metadata expiration check: 0:01:28 ago on Sat 27 Apr 2024 03:55:53 AM UTC. 2024-04-27T03:57:21.971 INFO:teuthology.orchestra.run.smithi162.stdout:Dependencies resolved. 2024-04-27T03:57:21.972 INFO:teuthology.orchestra.run.smithi162.stdout:================================================================================ 2024-04-27T03:57:21.972 INFO:teuthology.orchestra.run.smithi162.stdout: Package Arch Version Repository Size 2024-04-27T03:57:21.972 INFO:teuthology.orchestra.run.smithi162.stdout:================================================================================ 2024-04-27T03:57:21.972 INFO:teuthology.orchestra.run.smithi162.stdout:Installing: 2024-04-27T03:57:21.972 INFO:teuthology.orchestra.run.smithi162.stdout: ceph-fuse x86_64 2:17.2.7-907.g2be3efa9.el9 ceph 812 k 2024-04-27T03:57:21.972 INFO:teuthology.orchestra.run.smithi162.stdout:Installing dependencies: 2024-04-27T03:57:21.972 INFO:teuthology.orchestra.run.smithi162.stdout: fuse x86_64 2.9.9-15.el9 baseos 80 k 2024-04-27T03:57:21.973 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-04-27T03:57:21.973 INFO:teuthology.orchestra.run.smithi162.stdout:Transaction Summary 2024-04-27T03:57:21.973 INFO:teuthology.orchestra.run.smithi162.stdout:================================================================================ 2024-04-27T03:57:21.973 INFO:teuthology.orchestra.run.smithi162.stdout:Install 2 Packages 2024-04-27T03:57:21.973 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-04-27T03:57:21.973 INFO:teuthology.orchestra.run.smithi162.stdout:Total download size: 893 k 2024-04-27T03:57:21.973 INFO:teuthology.orchestra.run.smithi162.stdout:Installed size: 2.6 M 2024-04-27T03:57:21.974 INFO:teuthology.orchestra.run.smithi162.stdout:Downloading Packages: 2024-04-27T03:57:22.342 INFO:teuthology.orchestra.run.smithi162.stdout:(1/2): fuse-2.9.9-15.el9.x86_64.rpm 404 kB/s | 80 kB 00:00 2024-04-27T03:57:22.344 INFO:teuthology.orchestra.run.smithi181.stdout:Last metadata expiration check: 0:01:17 ago on Sat 27 Apr 2024 03:56:05 AM UTC. 2024-04-27T03:57:22.466 INFO:teuthology.orchestra.run.smithi181.stdout:Dependencies resolved. 2024-04-27T03:57:22.467 INFO:teuthology.orchestra.run.smithi162.stdout:(2/2): ceph-fuse-17.2.7-907.g2be3efa9.el9.x86_6 2.4 MB/s | 812 kB 00:00 2024-04-27T03:57:22.468 INFO:teuthology.orchestra.run.smithi181.stdout:====================================================================================== 2024-04-27T03:57:22.468 INFO:teuthology.orchestra.run.smithi181.stdout: Package Arch Version Repository Size 2024-04-27T03:57:22.468 INFO:teuthology.orchestra.run.smithi181.stdout:====================================================================================== 2024-04-27T03:57:22.468 INFO:teuthology.orchestra.run.smithi181.stdout:Installing: 2024-04-27T03:57:22.468 INFO:teuthology.orchestra.run.smithi181.stdout: ceph-mgr-diskprediction-local noarch 2:17.2.7-907.g2be3efa9.el9 ceph-noarch 7.4 M 2024-04-27T03:57:22.468 INFO:teuthology.orchestra.run.smithi181.stdout:Installing dependencies: 2024-04-27T03:57:22.468 INFO:teuthology.orchestra.run.smithi181.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-04-27T03:57:22.468 INFO:teuthology.orchestra.run.smithi181.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-04-27T03:57:22.468 INFO:teuthology.orchestra.run.smithi181.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-04-27T03:57:22.468 INFO:teuthology.orchestra.run.smithi181.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-04-27T03:57:22.469 INFO:teuthology.orchestra.run.smithi181.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-04-27T03:57:22.469 INFO:teuthology.orchestra.run.smithi181.stdout: openblas x86_64 0.3.21-2.el9 appstream 35 k 2024-04-27T03:57:22.469 INFO:teuthology.orchestra.run.smithi181.stdout: openblas-openmp x86_64 0.3.21-2.el9 appstream 4.7 M 2024-04-27T03:57:22.469 INFO:teuthology.orchestra.run.smithi181.stdout: python3-devel x86_64 3.9.18-3.el9 appstream 244 k 2024-04-27T03:57:22.469 INFO:teuthology.orchestra.run.smithi181.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-04-27T03:57:22.469 INFO:teuthology.orchestra.run.smithi181.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-04-27T03:57:22.469 INFO:teuthology.orchestra.run.smithi181.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-04-27T03:57:22.469 INFO:teuthology.orchestra.run.smithi181.stdout: 2024-04-27T03:57:22.469 INFO:teuthology.orchestra.run.smithi181.stdout:Transaction Summary 2024-04-27T03:57:22.469 INFO:teuthology.orchestra.run.smithi181.stdout:====================================================================================== 2024-04-27T03:57:22.469 INFO:teuthology.orchestra.run.smithi181.stdout:Install 12 Packages 2024-04-27T03:57:22.469 INFO:teuthology.orchestra.run.smithi181.stdout: 2024-04-27T03:57:22.470 INFO:teuthology.orchestra.run.smithi162.stdout:-------------------------------------------------------------------------------- 2024-04-27T03:57:22.470 INFO:teuthology.orchestra.run.smithi162.stdout:Total 1.8 MB/s | 893 kB 00:00 2024-04-27T03:57:22.470 INFO:teuthology.orchestra.run.smithi181.stdout:Total download size: 38 M 2024-04-27T03:57:22.470 INFO:teuthology.orchestra.run.smithi181.stdout:Installed size: 209 M 2024-04-27T03:57:22.471 INFO:teuthology.orchestra.run.smithi181.stdout:Downloading Packages: 2024-04-27T03:57:22.480 INFO:teuthology.orchestra.run.smithi162.stdout:Running transaction check 2024-04-27T03:57:22.503 INFO:teuthology.orchestra.run.smithi162.stdout:Transaction check succeeded. 2024-04-27T03:57:22.503 INFO:teuthology.orchestra.run.smithi162.stdout:Running transaction test 2024-04-27T03:57:22.579 INFO:teuthology.orchestra.run.smithi162.stdout:Transaction test succeeded. 2024-04-27T03:57:22.580 INFO:teuthology.orchestra.run.smithi162.stdout:Running transaction 2024-04-27T03:57:22.775 INFO:teuthology.orchestra.run.smithi162.stdout: Preparing : 1/1 2024-04-27T03:57:22.905 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : fuse-2.9.9-15.el9.x86_64 1/2 2024-04-27T03:57:22.982 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : ceph-fuse-2:17.2.7-907.g2be3efa9.el9.x86_64 2/2 2024-04-27T03:57:23.241 INFO:teuthology.orchestra.run.smithi181.stdout:(1/12): libquadmath-11.4.1-3.el9.x86_64.rpm 996 kB/s | 195 kB 00:00 2024-04-27T03:57:23.366 INFO:teuthology.orchestra.run.smithi181.stdout:(2/12): libgfortran-11.4.1-3.el9.x86_64.rpm 2.4 MB/s | 803 kB 00:00 2024-04-27T03:57:23.616 INFO:teuthology.orchestra.run.smithi181.stdout:(3/12): ceph-mgr-diskprediction-local-17.2.7-90 13 MB/s | 7.4 MB 00:00 2024-04-27T03:57:23.666 INFO:teuthology.orchestra.run.smithi162.stdout: Running scriptlet: ceph-fuse-2:17.2.7-907.g2be3efa9.el9.x86_64 2/2 2024-04-27T03:57:23.666 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : ceph-fuse-2:17.2.7-907.g2be3efa9.el9.x86_64 1/2 2024-04-27T03:57:23.896 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 2/2 2024-04-27T03:57:23.896 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-04-27T03:57:23.896 INFO:teuthology.orchestra.run.smithi162.stdout:Installed: 2024-04-27T03:57:23.896 INFO:teuthology.orchestra.run.smithi162.stdout: ceph-fuse-2:17.2.7-907.g2be3efa9.el9.x86_64 fuse-2.9.9-15.el9.x86_64 2024-04-27T03:57:23.896 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-04-27T03:57:23.896 INFO:teuthology.orchestra.run.smithi162.stdout:Complete! 2024-04-27T03:57:23.916 INFO:teuthology.orchestra.run.smithi181.stdout:(4/12): flexiblas-3.0.4-8.el9.x86_64.rpm 48 kB/s | 33 kB 00:00 2024-04-27T03:57:24.163 DEBUG:teuthology.orchestra.run.smithi162:> sudo yum -y install librados-devel 2024-04-27T03:57:24.250 INFO:teuthology.orchestra.run.smithi181.stdout:(5/12): flexiblas-openblas-openmp-3.0.4-8.el9.x 29 kB/s | 18 kB 00:00 2024-04-27T03:57:24.617 INFO:teuthology.orchestra.run.smithi181.stdout:(6/12): openblas-0.3.21-2.el9.x86_64.rpm 52 kB/s | 35 kB 00:00 2024-04-27T03:57:24.673 INFO:teuthology.orchestra.run.smithi162.stdout:Last metadata expiration check: 0:01:31 ago on Sat 27 Apr 2024 03:55:53 AM UTC. 2024-04-27T03:57:24.796 INFO:teuthology.orchestra.run.smithi162.stdout:Dependencies resolved. 2024-04-27T03:57:24.797 INFO:teuthology.orchestra.run.smithi162.stdout:================================================================================ 2024-04-27T03:57:24.797 INFO:teuthology.orchestra.run.smithi162.stdout: Package Arch Version Repo Size 2024-04-27T03:57:24.797 INFO:teuthology.orchestra.run.smithi162.stdout:================================================================================ 2024-04-27T03:57:24.797 INFO:teuthology.orchestra.run.smithi162.stdout:Installing: 2024-04-27T03:57:24.797 INFO:teuthology.orchestra.run.smithi162.stdout: librados-devel x86_64 2:17.2.7-907.g2be3efa9.el9 ceph 126 k 2024-04-27T03:57:24.797 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-04-27T03:57:24.797 INFO:teuthology.orchestra.run.smithi162.stdout:Transaction Summary 2024-04-27T03:57:24.797 INFO:teuthology.orchestra.run.smithi162.stdout:================================================================================ 2024-04-27T03:57:24.797 INFO:teuthology.orchestra.run.smithi162.stdout:Install 1 Package 2024-04-27T03:57:24.797 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-04-27T03:57:24.797 INFO:teuthology.orchestra.run.smithi162.stdout:Total download size: 126 k 2024-04-27T03:57:24.798 INFO:teuthology.orchestra.run.smithi162.stdout:Installed size: 448 k 2024-04-27T03:57:24.798 INFO:teuthology.orchestra.run.smithi162.stdout:Downloading Packages: 2024-04-27T03:57:25.146 INFO:teuthology.orchestra.run.smithi162.stdout:librados-devel-17.2.7-907.g2be3efa9.el9.x86_64. 362 kB/s | 126 kB 00:00 2024-04-27T03:57:25.147 INFO:teuthology.orchestra.run.smithi162.stdout:-------------------------------------------------------------------------------- 2024-04-27T03:57:25.147 INFO:teuthology.orchestra.run.smithi162.stdout:Total 361 kB/s | 126 kB 00:00 2024-04-27T03:57:25.147 INFO:teuthology.orchestra.run.smithi162.stdout:Running transaction check 2024-04-27T03:57:25.153 INFO:teuthology.orchestra.run.smithi162.stdout:Transaction check succeeded. 2024-04-27T03:57:25.153 INFO:teuthology.orchestra.run.smithi162.stdout:Running transaction test 2024-04-27T03:57:25.214 INFO:teuthology.orchestra.run.smithi162.stdout:Transaction test succeeded. 2024-04-27T03:57:25.214 INFO:teuthology.orchestra.run.smithi162.stdout:Running transaction 2024-04-27T03:57:25.367 INFO:teuthology.orchestra.run.smithi162.stdout: Preparing : 1/1 2024-04-27T03:57:25.468 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : librados-devel-2:17.2.7-907.g2be3efa9.el9.x86_64 1/1 2024-04-27T03:57:25.659 INFO:teuthology.orchestra.run.smithi181.stdout:(7/12): python3-devel-3.9.18-3.el9.x86_64.rpm 234 kB/s | 244 kB 00:01 2024-04-27T03:57:25.844 INFO:teuthology.orchestra.run.smithi162.stdout: Running scriptlet: librados-devel-2:17.2.7-907.g2be3efa9.el9.x86_64 1/1 2024-04-27T03:57:26.104 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : librados-devel-2:17.2.7-907.g2be3efa9.el9.x86_64 1/1 2024-04-27T03:57:26.105 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-04-27T03:57:26.105 INFO:teuthology.orchestra.run.smithi162.stdout:Installed: 2024-04-27T03:57:26.105 INFO:teuthology.orchestra.run.smithi162.stdout: librados-devel-2:17.2.7-907.g2be3efa9.el9.x86_64 2024-04-27T03:57:26.105 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-04-27T03:57:26.105 INFO:teuthology.orchestra.run.smithi162.stdout:Complete! 2024-04-27T03:57:26.330 DEBUG:teuthology.orchestra.run.smithi162:> sudo yum -y install libcephfs2 2024-04-27T03:57:26.850 INFO:teuthology.orchestra.run.smithi162.stdout:Last metadata expiration check: 0:01:33 ago on Sat 27 Apr 2024 03:55:53 AM UTC. 2024-04-27T03:57:26.931 INFO:teuthology.orchestra.run.smithi162.stdout:Package libcephfs2-2:17.2.7-907.g2be3efa9.el9.x86_64 is already installed. 2024-04-27T03:57:26.974 INFO:teuthology.orchestra.run.smithi162.stdout:Dependencies resolved. 2024-04-27T03:57:26.975 INFO:teuthology.orchestra.run.smithi162.stdout:Nothing to do. 2024-04-27T03:57:26.975 INFO:teuthology.orchestra.run.smithi162.stdout:Complete! 2024-04-27T03:57:27.050 DEBUG:teuthology.orchestra.run.smithi162:> sudo yum -y install libcephfs-devel 2024-04-27T03:57:27.562 INFO:teuthology.orchestra.run.smithi162.stdout:Last metadata expiration check: 0:01:34 ago on Sat 27 Apr 2024 03:55:53 AM UTC. 2024-04-27T03:57:27.683 INFO:teuthology.orchestra.run.smithi162.stdout:Dependencies resolved. 2024-04-27T03:57:27.683 INFO:teuthology.orchestra.run.smithi162.stdout:================================================================================ 2024-04-27T03:57:27.684 INFO:teuthology.orchestra.run.smithi162.stdout: Package Arch Version Repo Size 2024-04-27T03:57:27.684 INFO:teuthology.orchestra.run.smithi162.stdout:================================================================================ 2024-04-27T03:57:27.684 INFO:teuthology.orchestra.run.smithi162.stdout:Installing: 2024-04-27T03:57:27.684 INFO:teuthology.orchestra.run.smithi162.stdout: libcephfs-devel x86_64 2:17.2.7-907.g2be3efa9.el9 ceph 25 k 2024-04-27T03:57:27.684 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-04-27T03:57:27.684 INFO:teuthology.orchestra.run.smithi162.stdout:Transaction Summary 2024-04-27T03:57:27.684 INFO:teuthology.orchestra.run.smithi162.stdout:================================================================================ 2024-04-27T03:57:27.684 INFO:teuthology.orchestra.run.smithi162.stdout:Install 1 Package 2024-04-27T03:57:27.684 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-04-27T03:57:27.684 INFO:teuthology.orchestra.run.smithi162.stdout:Total download size: 25 k 2024-04-27T03:57:27.684 INFO:teuthology.orchestra.run.smithi162.stdout:Installed size: 110 k 2024-04-27T03:57:27.684 INFO:teuthology.orchestra.run.smithi162.stdout:Downloading Packages: 2024-04-27T03:57:27.918 INFO:teuthology.orchestra.run.smithi162.stdout:libcephfs-devel-17.2.7-907.g2be3efa9.el9.x86_64 107 kB/s | 25 kB 00:00 2024-04-27T03:57:27.919 INFO:teuthology.orchestra.run.smithi162.stdout:-------------------------------------------------------------------------------- 2024-04-27T03:57:27.919 INFO:teuthology.orchestra.run.smithi162.stdout:Total 106 kB/s | 25 kB 00:00 2024-04-27T03:57:27.919 INFO:teuthology.orchestra.run.smithi162.stdout:Running transaction check 2024-04-27T03:57:27.923 INFO:teuthology.orchestra.run.smithi162.stdout:Transaction check succeeded. 2024-04-27T03:57:27.923 INFO:teuthology.orchestra.run.smithi162.stdout:Running transaction test 2024-04-27T03:57:27.924 INFO:teuthology.orchestra.run.smithi162.stdout:Transaction test succeeded. 2024-04-27T03:57:27.925 INFO:teuthology.orchestra.run.smithi162.stdout:Running transaction 2024-04-27T03:57:28.012 INFO:teuthology.orchestra.run.smithi162.stdout: Preparing : 1/1 2024-04-27T03:57:28.114 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : libcephfs-devel-2:17.2.7-907.g2be3efa9.el9.x86_64 1/1 2024-04-27T03:57:28.159 INFO:teuthology.orchestra.run.smithi181.stdout:(8/12): flexiblas-netlib-3.0.4-8.el9.x86_64.rpm 639 kB/s | 3.0 MB 00:04 2024-04-27T03:57:28.433 INFO:teuthology.orchestra.run.smithi162.stdout: Running scriptlet: libcephfs-devel-2:17.2.7-907.g2be3efa9.el9.x86_64 1/1 2024-04-27T03:57:28.644 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : libcephfs-devel-2:17.2.7-907.g2be3efa9.el9.x86_64 1/1 2024-04-27T03:57:28.644 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-04-27T03:57:28.644 INFO:teuthology.orchestra.run.smithi162.stdout:Installed: 2024-04-27T03:57:28.644 INFO:teuthology.orchestra.run.smithi162.stdout: libcephfs-devel-2:17.2.7-907.g2be3efa9.el9.x86_64 2024-04-27T03:57:28.644 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-04-27T03:57:28.644 INFO:teuthology.orchestra.run.smithi162.stdout:Complete! 2024-04-27T03:57:28.819 DEBUG:teuthology.orchestra.run.smithi162:> sudo yum -y install librados2 2024-04-27T03:57:29.084 INFO:teuthology.orchestra.run.smithi181.stdout:(9/12): python3-numpy-f2py-1.20.1-5.el9.x86_64. 271 kB/s | 250 kB 00:00 2024-04-27T03:57:29.332 INFO:teuthology.orchestra.run.smithi162.stdout:Last metadata expiration check: 0:01:36 ago on Sat 27 Apr 2024 03:55:53 AM UTC. 2024-04-27T03:57:29.412 INFO:teuthology.orchestra.run.smithi162.stdout:Package librados2-2:17.2.7-907.g2be3efa9.el9.x86_64 is already installed. 2024-04-27T03:57:29.452 INFO:teuthology.orchestra.run.smithi162.stdout:Dependencies resolved. 2024-04-27T03:57:29.453 INFO:teuthology.orchestra.run.smithi162.stdout:Nothing to do. 2024-04-27T03:57:29.453 INFO:teuthology.orchestra.run.smithi162.stdout:Complete! 2024-04-27T03:57:29.525 DEBUG:teuthology.orchestra.run.smithi162:> sudo yum -y install librbd1 2024-04-27T03:57:30.032 INFO:teuthology.orchestra.run.smithi162.stdout:Last metadata expiration check: 0:01:37 ago on Sat 27 Apr 2024 03:55:53 AM UTC. 2024-04-27T03:57:30.110 INFO:teuthology.orchestra.run.smithi162.stdout:Package librbd1-2:17.2.7-907.g2be3efa9.el9.x86_64 is already installed. 2024-04-27T03:57:30.150 INFO:teuthology.orchestra.run.smithi162.stdout:Dependencies resolved. 2024-04-27T03:57:30.151 INFO:teuthology.orchestra.run.smithi162.stdout:Nothing to do. 2024-04-27T03:57:30.151 INFO:teuthology.orchestra.run.smithi162.stdout:Complete! 2024-04-27T03:57:30.216 DEBUG:teuthology.orchestra.run.smithi162:> sudo yum -y install python3-rados 2024-04-27T03:57:30.724 INFO:teuthology.orchestra.run.smithi162.stdout:Last metadata expiration check: 0:01:37 ago on Sat 27 Apr 2024 03:55:53 AM UTC. 2024-04-27T03:57:30.803 INFO:teuthology.orchestra.run.smithi162.stdout:Package python3-rados-2:17.2.7-907.g2be3efa9.el9.x86_64 is already installed. 2024-04-27T03:57:30.845 INFO:teuthology.orchestra.run.smithi162.stdout:Dependencies resolved. 2024-04-27T03:57:30.846 INFO:teuthology.orchestra.run.smithi162.stdout:Nothing to do. 2024-04-27T03:57:30.846 INFO:teuthology.orchestra.run.smithi162.stdout:Complete! 2024-04-27T03:57:30.914 DEBUG:teuthology.orchestra.run.smithi162:> sudo yum -y install python3-rgw 2024-04-27T03:57:31.234 INFO:teuthology.orchestra.run.smithi181.stdout:(10/12): openblas-openmp-0.3.21-2.el9.x86_64.rp 695 kB/s | 4.7 MB 00:06 2024-04-27T03:57:31.421 INFO:teuthology.orchestra.run.smithi162.stdout:Last metadata expiration check: 0:01:38 ago on Sat 27 Apr 2024 03:55:53 AM UTC. 2024-04-27T03:57:31.499 INFO:teuthology.orchestra.run.smithi162.stdout:Package python3-rgw-2:17.2.7-907.g2be3efa9.el9.x86_64 is already installed. 2024-04-27T03:57:31.540 INFO:teuthology.orchestra.run.smithi162.stdout:Dependencies resolved. 2024-04-27T03:57:31.541 INFO:teuthology.orchestra.run.smithi162.stdout:Nothing to do. 2024-04-27T03:57:31.542 INFO:teuthology.orchestra.run.smithi162.stdout:Complete! 2024-04-27T03:57:31.604 DEBUG:teuthology.orchestra.run.smithi162:> sudo yum -y install python3-cephfs 2024-04-27T03:57:32.110 INFO:teuthology.orchestra.run.smithi162.stdout:Last metadata expiration check: 0:01:39 ago on Sat 27 Apr 2024 03:55:53 AM UTC. 2024-04-27T03:57:32.188 INFO:teuthology.orchestra.run.smithi162.stdout:Package python3-cephfs-2:17.2.7-907.g2be3efa9.el9.x86_64 is already installed. 2024-04-27T03:57:32.228 INFO:teuthology.orchestra.run.smithi162.stdout:Dependencies resolved. 2024-04-27T03:57:32.230 INFO:teuthology.orchestra.run.smithi162.stdout:Nothing to do. 2024-04-27T03:57:32.230 INFO:teuthology.orchestra.run.smithi162.stdout:Complete! 2024-04-27T03:57:32.287 DEBUG:teuthology.orchestra.run.smithi162:> sudo yum -y install python3-rbd 2024-04-27T03:57:32.667 INFO:teuthology.orchestra.run.smithi181.stdout:(11/12): python3-numpy-1.20.1-5.el9.x86_64.rpm 745 kB/s | 5.1 MB 00:07 2024-04-27T03:57:32.791 INFO:teuthology.orchestra.run.smithi162.stdout:Last metadata expiration check: 0:01:39 ago on Sat 27 Apr 2024 03:55:53 AM UTC. 2024-04-27T03:57:32.868 INFO:teuthology.orchestra.run.smithi162.stdout:Package python3-rbd-2:17.2.7-907.g2be3efa9.el9.x86_64 is already installed. 2024-04-27T03:57:32.910 INFO:teuthology.orchestra.run.smithi162.stdout:Dependencies resolved. 2024-04-27T03:57:32.910 INFO:teuthology.orchestra.run.smithi162.stdout:Nothing to do. 2024-04-27T03:57:32.911 INFO:teuthology.orchestra.run.smithi162.stdout:Complete! 2024-04-27T03:57:32.970 DEBUG:teuthology.orchestra.run.smithi162:> sudo yum -y install rbd-fuse 2024-04-27T03:57:33.478 INFO:teuthology.orchestra.run.smithi162.stdout:Last metadata expiration check: 0:01:40 ago on Sat 27 Apr 2024 03:55:53 AM UTC. 2024-04-27T03:57:33.596 INFO:teuthology.orchestra.run.smithi162.stdout:Dependencies resolved. 2024-04-27T03:57:33.597 INFO:teuthology.orchestra.run.smithi162.stdout:================================================================================ 2024-04-27T03:57:33.597 INFO:teuthology.orchestra.run.smithi162.stdout: Package Architecture Version Repository Size 2024-04-27T03:57:33.597 INFO:teuthology.orchestra.run.smithi162.stdout:================================================================================ 2024-04-27T03:57:33.597 INFO:teuthology.orchestra.run.smithi162.stdout:Installing: 2024-04-27T03:57:33.597 INFO:teuthology.orchestra.run.smithi162.stdout: rbd-fuse x86_64 2:17.2.7-907.g2be3efa9.el9 ceph 82 k 2024-04-27T03:57:33.597 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-04-27T03:57:33.597 INFO:teuthology.orchestra.run.smithi162.stdout:Transaction Summary 2024-04-27T03:57:33.597 INFO:teuthology.orchestra.run.smithi162.stdout:================================================================================ 2024-04-27T03:57:33.597 INFO:teuthology.orchestra.run.smithi162.stdout:Install 1 Package 2024-04-27T03:57:33.597 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-04-27T03:57:33.598 INFO:teuthology.orchestra.run.smithi162.stdout:Total download size: 82 k 2024-04-27T03:57:33.598 INFO:teuthology.orchestra.run.smithi162.stdout:Installed size: 222 k 2024-04-27T03:57:33.598 INFO:teuthology.orchestra.run.smithi162.stdout:Downloading Packages: 2024-04-27T03:57:33.812 INFO:teuthology.orchestra.run.smithi162.stdout:rbd-fuse-17.2.7-907.g2be3efa9.el9.x86_64.rpm 381 kB/s | 82 kB 00:00 2024-04-27T03:57:33.813 INFO:teuthology.orchestra.run.smithi162.stdout:-------------------------------------------------------------------------------- 2024-04-27T03:57:33.813 INFO:teuthology.orchestra.run.smithi162.stdout:Total 379 kB/s | 82 kB 00:00 2024-04-27T03:57:33.813 INFO:teuthology.orchestra.run.smithi162.stdout:Running transaction check 2024-04-27T03:57:33.820 INFO:teuthology.orchestra.run.smithi162.stdout:Transaction check succeeded. 2024-04-27T03:57:33.820 INFO:teuthology.orchestra.run.smithi162.stdout:Running transaction test 2024-04-27T03:57:33.883 INFO:teuthology.orchestra.run.smithi162.stdout:Transaction test succeeded. 2024-04-27T03:57:33.883 INFO:teuthology.orchestra.run.smithi162.stdout:Running transaction 2024-04-27T03:57:34.034 INFO:teuthology.orchestra.run.smithi162.stdout: Preparing : 1/1 2024-04-27T03:57:34.124 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : rbd-fuse-2:17.2.7-907.g2be3efa9.el9.x86_64 1/1 2024-04-27T03:57:34.450 INFO:teuthology.orchestra.run.smithi162.stdout: Running scriptlet: rbd-fuse-2:17.2.7-907.g2be3efa9.el9.x86_64 1/1 2024-04-27T03:57:34.685 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : rbd-fuse-2:17.2.7-907.g2be3efa9.el9.x86_64 1/1 2024-04-27T03:57:34.686 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-04-27T03:57:34.686 INFO:teuthology.orchestra.run.smithi162.stdout:Installed: 2024-04-27T03:57:34.686 INFO:teuthology.orchestra.run.smithi162.stdout: rbd-fuse-2:17.2.7-907.g2be3efa9.el9.x86_64 2024-04-27T03:57:34.686 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-04-27T03:57:34.686 INFO:teuthology.orchestra.run.smithi162.stdout:Complete! 2024-04-27T03:57:34.916 DEBUG:teuthology.orchestra.run.smithi162:> sudo yum -y install rbd-mirror 2024-04-27T03:57:35.441 INFO:teuthology.orchestra.run.smithi162.stdout:Last metadata expiration check: 0:01:42 ago on Sat 27 Apr 2024 03:55:53 AM UTC. 2024-04-27T03:57:35.563 INFO:teuthology.orchestra.run.smithi162.stdout:Dependencies resolved. 2024-04-27T03:57:35.564 INFO:teuthology.orchestra.run.smithi162.stdout:================================================================================ 2024-04-27T03:57:35.564 INFO:teuthology.orchestra.run.smithi162.stdout: Package Arch Version Repo Size 2024-04-27T03:57:35.564 INFO:teuthology.orchestra.run.smithi162.stdout:================================================================================ 2024-04-27T03:57:35.565 INFO:teuthology.orchestra.run.smithi162.stdout:Installing: 2024-04-27T03:57:35.565 INFO:teuthology.orchestra.run.smithi162.stdout: rbd-mirror x86_64 2:17.2.7-907.g2be3efa9.el9 ceph 2.9 M 2024-04-27T03:57:35.565 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-04-27T03:57:35.565 INFO:teuthology.orchestra.run.smithi162.stdout:Transaction Summary 2024-04-27T03:57:35.565 INFO:teuthology.orchestra.run.smithi162.stdout:================================================================================ 2024-04-27T03:57:35.565 INFO:teuthology.orchestra.run.smithi162.stdout:Install 1 Package 2024-04-27T03:57:35.565 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-04-27T03:57:35.565 INFO:teuthology.orchestra.run.smithi162.stdout:Total download size: 2.9 M 2024-04-27T03:57:35.565 INFO:teuthology.orchestra.run.smithi162.stdout:Installed size: 12 M 2024-04-27T03:57:35.565 INFO:teuthology.orchestra.run.smithi162.stdout:Downloading Packages: 2024-04-27T03:57:36.882 INFO:teuthology.orchestra.run.smithi162.stdout:rbd-mirror-17.2.7-907.g2be3efa9.el9.x86_64.rpm 2.2 MB/s | 2.9 MB 00:01 2024-04-27T03:57:36.883 INFO:teuthology.orchestra.run.smithi162.stdout:-------------------------------------------------------------------------------- 2024-04-27T03:57:36.883 INFO:teuthology.orchestra.run.smithi162.stdout:Total 2.2 MB/s | 2.9 MB 00:01 2024-04-27T03:57:36.883 INFO:teuthology.orchestra.run.smithi162.stdout:Running transaction check 2024-04-27T03:57:36.892 INFO:teuthology.orchestra.run.smithi162.stdout:Transaction check succeeded. 2024-04-27T03:57:36.893 INFO:teuthology.orchestra.run.smithi162.stdout:Running transaction test 2024-04-27T03:57:36.971 INFO:teuthology.orchestra.run.smithi162.stdout:Transaction test succeeded. 2024-04-27T03:57:36.972 INFO:teuthology.orchestra.run.smithi162.stdout:Running transaction 2024-04-27T03:57:37.393 INFO:teuthology.orchestra.run.smithi162.stdout: Preparing : 1/1 2024-04-27T03:57:37.458 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : rbd-mirror-2:17.2.7-907.g2be3efa9.el9.x86_64 1/1 2024-04-27T03:57:37.482 INFO:teuthology.orchestra.run.smithi162.stdout: Running scriptlet: rbd-mirror-2:17.2.7-907.g2be3efa9.el9.x86_64 1/1 2024-04-27T03:57:37.482 INFO:teuthology.orchestra.run.smithi162.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-27T03:57:37.482 INFO:teuthology.orchestra.run.smithi162.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-04-27T03:57:37.482 INFO:teuthology.orchestra.run.smithi162.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-27T03:57:37.483 INFO:teuthology.orchestra.run.smithi162.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-27T03:57:37.483 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-04-27T03:57:38.575 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : rbd-mirror-2:17.2.7-907.g2be3efa9.el9.x86_64 1/1 2024-04-27T03:57:38.575 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-04-27T03:57:38.575 INFO:teuthology.orchestra.run.smithi162.stdout:Installed: 2024-04-27T03:57:38.575 INFO:teuthology.orchestra.run.smithi162.stdout: rbd-mirror-2:17.2.7-907.g2be3efa9.el9.x86_64 2024-04-27T03:57:38.575 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-04-27T03:57:38.576 INFO:teuthology.orchestra.run.smithi162.stdout:Complete! 2024-04-27T03:57:38.813 DEBUG:teuthology.orchestra.run.smithi162:> sudo yum -y install rbd-nbd 2024-04-27T03:57:39.342 INFO:teuthology.orchestra.run.smithi162.stdout:Last metadata expiration check: 0:01:46 ago on Sat 27 Apr 2024 03:55:53 AM UTC. 2024-04-27T03:57:39.468 INFO:teuthology.orchestra.run.smithi162.stdout:Dependencies resolved. 2024-04-27T03:57:39.469 INFO:teuthology.orchestra.run.smithi162.stdout:================================================================================ 2024-04-27T03:57:39.469 INFO:teuthology.orchestra.run.smithi162.stdout: Package Architecture Version Repository Size 2024-04-27T03:57:39.469 INFO:teuthology.orchestra.run.smithi162.stdout:================================================================================ 2024-04-27T03:57:39.469 INFO:teuthology.orchestra.run.smithi162.stdout:Installing: 2024-04-27T03:57:39.470 INFO:teuthology.orchestra.run.smithi162.stdout: rbd-nbd x86_64 2:17.2.7-907.g2be3efa9.el9 ceph 165 k 2024-04-27T03:57:39.470 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-04-27T03:57:39.470 INFO:teuthology.orchestra.run.smithi162.stdout:Transaction Summary 2024-04-27T03:57:39.470 INFO:teuthology.orchestra.run.smithi162.stdout:================================================================================ 2024-04-27T03:57:39.470 INFO:teuthology.orchestra.run.smithi162.stdout:Install 1 Package 2024-04-27T03:57:39.470 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-04-27T03:57:39.470 INFO:teuthology.orchestra.run.smithi162.stdout:Total download size: 165 k 2024-04-27T03:57:39.470 INFO:teuthology.orchestra.run.smithi162.stdout:Installed size: 477 k 2024-04-27T03:57:39.470 INFO:teuthology.orchestra.run.smithi162.stdout:Downloading Packages: 2024-04-27T03:57:39.695 INFO:teuthology.orchestra.run.smithi162.stdout:rbd-nbd-17.2.7-907.g2be3efa9.el9.x86_64.rpm 735 kB/s | 165 kB 00:00 2024-04-27T03:57:39.695 INFO:teuthology.orchestra.run.smithi162.stdout:-------------------------------------------------------------------------------- 2024-04-27T03:57:39.696 INFO:teuthology.orchestra.run.smithi162.stdout:Total 730 kB/s | 165 kB 00:00 2024-04-27T03:57:39.696 INFO:teuthology.orchestra.run.smithi162.stdout:Running transaction check 2024-04-27T03:57:39.703 INFO:teuthology.orchestra.run.smithi162.stdout:Transaction check succeeded. 2024-04-27T03:57:39.703 INFO:teuthology.orchestra.run.smithi162.stdout:Running transaction test 2024-04-27T03:57:39.766 INFO:teuthology.orchestra.run.smithi162.stdout:Transaction test succeeded. 2024-04-27T03:57:39.767 INFO:teuthology.orchestra.run.smithi162.stdout:Running transaction 2024-04-27T03:57:39.916 INFO:teuthology.orchestra.run.smithi162.stdout: Preparing : 1/1 2024-04-27T03:57:40.024 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : rbd-nbd-2:17.2.7-907.g2be3efa9.el9.x86_64 1/1 2024-04-27T03:57:40.421 INFO:teuthology.orchestra.run.smithi162.stdout: Running scriptlet: rbd-nbd-2:17.2.7-907.g2be3efa9.el9.x86_64 1/1 2024-04-27T03:57:40.685 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : rbd-nbd-2:17.2.7-907.g2be3efa9.el9.x86_64 1/1 2024-04-27T03:57:40.685 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-04-27T03:57:40.685 INFO:teuthology.orchestra.run.smithi162.stdout:Installed: 2024-04-27T03:57:40.685 INFO:teuthology.orchestra.run.smithi162.stdout: rbd-nbd-2:17.2.7-907.g2be3efa9.el9.x86_64 2024-04-27T03:57:40.685 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-04-27T03:57:40.685 INFO:teuthology.orchestra.run.smithi162.stdout:Complete! 2024-04-27T03:57:40.915 DEBUG:teuthology.parallel:result is None 2024-04-27T03:57:50.523 INFO:teuthology.orchestra.run.smithi181.stdout:(12/12): python3-scipy-1.6.2-8.el9.x86_64.rpm 776 kB/s | 16 MB 00:21 2024-04-27T03:57:50.525 INFO:teuthology.orchestra.run.smithi181.stdout:-------------------------------------------------------------------------------- 2024-04-27T03:57:50.525 INFO:teuthology.orchestra.run.smithi181.stdout:Total 1.4 MB/s | 38 MB 00:28 2024-04-27T03:57:50.882 INFO:teuthology.orchestra.run.smithi181.stdout:Running transaction check 2024-04-27T03:57:50.908 INFO:teuthology.orchestra.run.smithi181.stdout:Transaction check succeeded. 2024-04-27T03:57:50.909 INFO:teuthology.orchestra.run.smithi181.stdout:Running transaction test 2024-04-27T03:57:51.316 INFO:teuthology.orchestra.run.smithi181.stdout:Transaction test succeeded. 2024-04-27T03:57:51.317 INFO:teuthology.orchestra.run.smithi181.stdout:Running transaction 2024-04-27T03:57:52.071 INFO:teuthology.orchestra.run.smithi181.stdout: Preparing : 1/1 2024-04-27T03:57:52.128 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 1/12 2024-04-27T03:57:52.180 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 2/12 2024-04-27T03:57:52.279 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 3/12 2024-04-27T03:57:52.331 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-devel-3.9.18-3.el9.x86_64 4/12 2024-04-27T03:57:52.508 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : openblas-0.3.21-2.el9.x86_64 5/12 2024-04-27T03:57:52.537 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 6/12 2024-04-27T03:57:52.614 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 7/12 2024-04-27T03:57:53.256 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 8/12 2024-04-27T03:57:53.368 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 9/12 2024-04-27T03:57:54.600 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 10/12 2024-04-27T03:57:55.634 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 11/12 2024-04-27T03:57:55.662 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : ceph-mgr-diskprediction-local-2:17.2.7-907.g2be3ef 12/12 2024-04-27T03:57:58.143 INFO:teuthology.orchestra.run.smithi181.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:17.2.7-907.g2be3ef 12/12 2024-04-27T03:57:58.143 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : ceph-mgr-diskprediction-local-2:17.2.7-907.g2be3ef 1/12 2024-04-27T03:57:58.143 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 2/12 2024-04-27T03:57:58.143 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 3/12 2024-04-27T03:57:58.143 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 4/12 2024-04-27T03:57:58.143 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 5/12 2024-04-27T03:57:58.143 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 6/12 2024-04-27T03:57:58.143 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 7/12 2024-04-27T03:57:58.143 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 8/12 2024-04-27T03:57:58.143 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-devel-3.9.18-3.el9.x86_64 9/12 2024-04-27T03:57:58.143 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 10/12 2024-04-27T03:57:58.143 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 11/12 2024-04-27T03:57:58.515 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 12/12 2024-04-27T03:57:58.515 INFO:teuthology.orchestra.run.smithi181.stdout: 2024-04-27T03:57:58.515 INFO:teuthology.orchestra.run.smithi181.stdout:Installed: 2024-04-27T03:57:58.515 INFO:teuthology.orchestra.run.smithi181.stdout: ceph-mgr-diskprediction-local-2:17.2.7-907.g2be3efa9.el9.noarch 2024-04-27T03:57:58.515 INFO:teuthology.orchestra.run.smithi181.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-04-27T03:57:58.515 INFO:teuthology.orchestra.run.smithi181.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-04-27T03:57:58.515 INFO:teuthology.orchestra.run.smithi181.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-04-27T03:57:58.515 INFO:teuthology.orchestra.run.smithi181.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-04-27T03:57:58.515 INFO:teuthology.orchestra.run.smithi181.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-04-27T03:57:58.515 INFO:teuthology.orchestra.run.smithi181.stdout: openblas-0.3.21-2.el9.x86_64 2024-04-27T03:57:58.516 INFO:teuthology.orchestra.run.smithi181.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-04-27T03:57:58.516 INFO:teuthology.orchestra.run.smithi181.stdout: python3-devel-3.9.18-3.el9.x86_64 2024-04-27T03:57:58.516 INFO:teuthology.orchestra.run.smithi181.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-04-27T03:57:58.516 INFO:teuthology.orchestra.run.smithi181.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-04-27T03:57:58.516 INFO:teuthology.orchestra.run.smithi181.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-04-27T03:57:58.516 INFO:teuthology.orchestra.run.smithi181.stdout: 2024-04-27T03:57:58.516 INFO:teuthology.orchestra.run.smithi181.stdout:Complete! 2024-04-27T03:57:58.924 DEBUG:teuthology.orchestra.run.smithi181:> sudo yum -y install ceph-mgr-rook 2024-04-27T03:57:59.443 INFO:teuthology.orchestra.run.smithi181.stdout:Last metadata expiration check: 0:01:54 ago on Sat 27 Apr 2024 03:56:05 AM UTC. 2024-04-27T03:57:59.563 INFO:teuthology.orchestra.run.smithi181.stdout:Dependencies resolved. 2024-04-27T03:57:59.564 INFO:teuthology.orchestra.run.smithi181.stdout:================================================================================ 2024-04-27T03:57:59.565 INFO:teuthology.orchestra.run.smithi181.stdout: Package Arch Version Repository Size 2024-04-27T03:57:59.565 INFO:teuthology.orchestra.run.smithi181.stdout:================================================================================ 2024-04-27T03:57:59.565 INFO:teuthology.orchestra.run.smithi181.stdout:Installing: 2024-04-27T03:57:59.565 INFO:teuthology.orchestra.run.smithi181.stdout: ceph-mgr-rook noarch 2:17.2.7-907.g2be3efa9.el9 ceph-noarch 47 k 2024-04-27T03:57:59.565 INFO:teuthology.orchestra.run.smithi181.stdout:Installing dependencies: 2024-04-27T03:57:59.565 INFO:teuthology.orchestra.run.smithi181.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-04-27T03:57:59.565 INFO:teuthology.orchestra.run.smithi181.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-04-27T03:57:59.565 INFO:teuthology.orchestra.run.smithi181.stdout: python3-google-auth noarch 1:2.29.0-1.el9 epel 206 k 2024-04-27T03:57:59.565 INFO:teuthology.orchestra.run.smithi181.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-04-27T03:57:59.565 INFO:teuthology.orchestra.run.smithi181.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-04-27T03:57:59.565 INFO:teuthology.orchestra.run.smithi181.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-04-27T03:57:59.565 INFO:teuthology.orchestra.run.smithi181.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-04-27T03:57:59.565 INFO:teuthology.orchestra.run.smithi181.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-04-27T03:57:59.566 INFO:teuthology.orchestra.run.smithi181.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-04-27T03:57:59.566 INFO:teuthology.orchestra.run.smithi181.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-04-27T03:57:59.566 INFO:teuthology.orchestra.run.smithi181.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-04-27T03:57:59.566 INFO:teuthology.orchestra.run.smithi181.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-04-27T03:57:59.566 INFO:teuthology.orchestra.run.smithi181.stdout: 2024-04-27T03:57:59.566 INFO:teuthology.orchestra.run.smithi181.stdout:Transaction Summary 2024-04-27T03:57:59.566 INFO:teuthology.orchestra.run.smithi181.stdout:================================================================================ 2024-04-27T03:57:59.566 INFO:teuthology.orchestra.run.smithi181.stdout:Install 13 Packages 2024-04-27T03:57:59.566 INFO:teuthology.orchestra.run.smithi181.stdout: 2024-04-27T03:57:59.567 INFO:teuthology.orchestra.run.smithi181.stdout:Total download size: 2.2 M 2024-04-27T03:57:59.567 INFO:teuthology.orchestra.run.smithi181.stdout:Installed size: 26 M 2024-04-27T03:57:59.567 INFO:teuthology.orchestra.run.smithi181.stdout:Downloading Packages: 2024-04-27T03:58:00.188 INFO:teuthology.orchestra.run.smithi181.stdout:(1/13): python3-jsonpointer-2.0-4.el9.noarch.rp 100 kB/s | 19 kB 00:00 2024-04-27T03:58:00.213 INFO:teuthology.orchestra.run.smithi181.stdout:(2/13): ceph-mgr-rook-17.2.7-907.g2be3efa9.el9. 218 kB/s | 47 kB 00:00 2024-04-27T03:58:00.238 INFO:teuthology.orchestra.run.smithi181.stdout:(3/13): python3-jsonpatch-1.21-16.el9.noarch.rp 105 kB/s | 26 kB 00:00 2024-04-27T03:58:00.330 INFO:teuthology.orchestra.run.smithi181.stdout:(4/13): python3-oauthlib-3.1.1-5.el9.noarch.rpm 1.5 MB/s | 222 kB 00:00 2024-04-27T03:58:00.363 INFO:teuthology.orchestra.run.smithi181.stdout:(5/13): python3-pyasn1-0.4.8-6.el9.noarch.rpm 1.0 MB/s | 159 kB 00:00 2024-04-27T03:58:00.405 INFO:teuthology.orchestra.run.smithi181.stdout:(6/13): python3-requests-oauthlib-1.3.0-12.el9. 721 kB/s | 54 kB 00:00 2024-04-27T03:58:00.506 INFO:teuthology.orchestra.run.smithi181.stdout:(7/13): python3-pyasn1-modules-0.4.8-6.el9.noar 1.0 MB/s | 279 kB 00:00 2024-04-27T03:58:00.639 INFO:teuthology.orchestra.run.smithi181.stdout:(8/13): python3-certifi-2021.10.8-2.el9.noarch. 64 kB/s | 15 kB 00:00 2024-04-27T03:58:00.664 INFO:teuthology.orchestra.run.smithi181.stdout:(9/13): python3-cachetools-4.2.4-1.el9.noarch.r 107 kB/s | 32 kB 00:00 2024-04-27T03:58:00.781 INFO:teuthology.orchestra.run.smithi181.stdout:(10/13): python3-google-auth-2.29.0-1.el9.noarc 749 kB/s | 206 kB 00:00 2024-04-27T03:58:00.840 INFO:teuthology.orchestra.run.smithi181.stdout:(11/13): python3-rsa-4.9-2.el9.noarch.rpm 337 kB/s | 59 kB 00:00 2024-04-27T03:58:00.932 INFO:teuthology.orchestra.run.smithi181.stdout:(12/13): python3-kubernetes-26.1.0-2.el9.noarch 3.5 MB/s | 1.0 MB 00:00 2024-04-27T03:58:00.957 INFO:teuthology.orchestra.run.smithi181.stdout:(13/13): python3-websocket-client-1.2.3-2.el9.n 512 kB/s | 90 kB 00:00 2024-04-27T03:58:00.962 INFO:teuthology.orchestra.run.smithi181.stdout:-------------------------------------------------------------------------------- 2024-04-27T03:58:00.962 INFO:teuthology.orchestra.run.smithi181.stdout:Total 1.6 MB/s | 2.2 MB 00:01 2024-04-27T03:58:01.025 INFO:teuthology.orchestra.run.smithi181.stdout:Running transaction check 2024-04-27T03:58:01.040 INFO:teuthology.orchestra.run.smithi181.stdout:Transaction check succeeded. 2024-04-27T03:58:01.040 INFO:teuthology.orchestra.run.smithi181.stdout:Running transaction test 2024-04-27T03:58:01.124 INFO:teuthology.orchestra.run.smithi181.stdout:Transaction test succeeded. 2024-04-27T03:58:01.126 INFO:teuthology.orchestra.run.smithi181.stdout:Running transaction 2024-04-27T03:58:01.705 INFO:teuthology.orchestra.run.smithi181.stdout: Preparing : 1/1 2024-04-27T03:58:01.862 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 1/13 2024-04-27T03:58:01.917 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 2/13 2024-04-27T03:58:01.981 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-rsa-4.9-2.el9.noarch 3/13 2024-04-27T03:58:02.018 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 4/13 2024-04-27T03:58:02.059 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 5/13 2024-04-27T03:58:02.154 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 6/13 2024-04-27T03:58:02.283 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-google-auth-1:2.29.0-1.el9.noarch 7/13 2024-04-27T03:58:02.336 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 8/13 2024-04-27T03:58:03.048 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 9/13 2024-04-27T03:58:03.108 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 10/13 2024-04-27T03:58:03.142 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 11/13 2024-04-27T03:58:03.196 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 12/13 2024-04-27T03:58:03.220 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : ceph-mgr-rook-2:17.2.7-907.g2be3efa9.el9.noarch 13/13 2024-04-27T03:58:03.955 INFO:teuthology.orchestra.run.smithi181.stdout: Running scriptlet: ceph-mgr-rook-2:17.2.7-907.g2be3efa9.el9.noarch 13/13 2024-04-27T03:58:03.955 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : ceph-mgr-rook-2:17.2.7-907.g2be3efa9.el9.noarch 1/13 2024-04-27T03:58:03.956 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 2/13 2024-04-27T03:58:03.956 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 3/13 2024-04-27T03:58:03.956 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 4/13 2024-04-27T03:58:03.956 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 5/13 2024-04-27T03:58:03.956 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 6/13 2024-04-27T03:58:03.956 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 7/13 2024-04-27T03:58:03.956 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 8/13 2024-04-27T03:58:03.956 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 9/13 2024-04-27T03:58:03.956 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-google-auth-1:2.29.0-1.el9.noarch 10/13 2024-04-27T03:58:03.956 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 11/13 2024-04-27T03:58:03.956 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 12/13 2024-04-27T03:58:04.253 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 13/13 2024-04-27T03:58:04.253 INFO:teuthology.orchestra.run.smithi181.stdout: 2024-04-27T03:58:04.253 INFO:teuthology.orchestra.run.smithi181.stdout:Installed: 2024-04-27T03:58:04.253 INFO:teuthology.orchestra.run.smithi181.stdout: ceph-mgr-rook-2:17.2.7-907.g2be3efa9.el9.noarch 2024-04-27T03:58:04.253 INFO:teuthology.orchestra.run.smithi181.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-04-27T03:58:04.253 INFO:teuthology.orchestra.run.smithi181.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-04-27T03:58:04.253 INFO:teuthology.orchestra.run.smithi181.stdout: python3-google-auth-1:2.29.0-1.el9.noarch 2024-04-27T03:58:04.253 INFO:teuthology.orchestra.run.smithi181.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-04-27T03:58:04.253 INFO:teuthology.orchestra.run.smithi181.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-04-27T03:58:04.253 INFO:teuthology.orchestra.run.smithi181.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-04-27T03:58:04.253 INFO:teuthology.orchestra.run.smithi181.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-04-27T03:58:04.253 INFO:teuthology.orchestra.run.smithi181.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-04-27T03:58:04.253 INFO:teuthology.orchestra.run.smithi181.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-04-27T03:58:04.254 INFO:teuthology.orchestra.run.smithi181.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-04-27T03:58:04.254 INFO:teuthology.orchestra.run.smithi181.stdout: python3-rsa-4.9-2.el9.noarch 2024-04-27T03:58:04.254 INFO:teuthology.orchestra.run.smithi181.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-04-27T03:58:04.254 INFO:teuthology.orchestra.run.smithi181.stdout: 2024-04-27T03:58:04.254 INFO:teuthology.orchestra.run.smithi181.stdout:Complete! 2024-04-27T03:58:04.479 DEBUG:teuthology.orchestra.run.smithi181:> sudo yum -y install ceph-mgr-cephadm 2024-04-27T03:58:04.993 INFO:teuthology.orchestra.run.smithi181.stdout:Last metadata expiration check: 0:01:59 ago on Sat 27 Apr 2024 03:56:05 AM UTC. 2024-04-27T03:58:05.112 INFO:teuthology.orchestra.run.smithi181.stdout:Dependencies resolved. 2024-04-27T03:58:05.113 INFO:teuthology.orchestra.run.smithi181.stdout:================================================================================ 2024-04-27T03:58:05.113 INFO:teuthology.orchestra.run.smithi181.stdout: Package Arch Version Repository Size 2024-04-27T03:58:05.113 INFO:teuthology.orchestra.run.smithi181.stdout:================================================================================ 2024-04-27T03:58:05.113 INFO:teuthology.orchestra.run.smithi181.stdout:Installing: 2024-04-27T03:58:05.113 INFO:teuthology.orchestra.run.smithi181.stdout: ceph-mgr-cephadm noarch 2:17.2.7-907.g2be3efa9.el9 ceph-noarch 123 k 2024-04-27T03:58:05.113 INFO:teuthology.orchestra.run.smithi181.stdout:Installing dependencies: 2024-04-27T03:58:05.113 INFO:teuthology.orchestra.run.smithi181.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-04-27T03:58:05.113 INFO:teuthology.orchestra.run.smithi181.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-04-27T03:58:05.113 INFO:teuthology.orchestra.run.smithi181.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-04-27T03:58:05.113 INFO:teuthology.orchestra.run.smithi181.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-04-27T03:58:05.113 INFO:teuthology.orchestra.run.smithi181.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-04-27T03:58:05.113 INFO:teuthology.orchestra.run.smithi181.stdout: 2024-04-27T03:58:05.114 INFO:teuthology.orchestra.run.smithi181.stdout:Transaction Summary 2024-04-27T03:58:05.114 INFO:teuthology.orchestra.run.smithi181.stdout:================================================================================ 2024-04-27T03:58:05.114 INFO:teuthology.orchestra.run.smithi181.stdout:Install 6 Packages 2024-04-27T03:58:05.114 INFO:teuthology.orchestra.run.smithi181.stdout: 2024-04-27T03:58:05.114 INFO:teuthology.orchestra.run.smithi181.stdout:Total download size: 7.0 M 2024-04-27T03:58:05.114 INFO:teuthology.orchestra.run.smithi181.stdout:Installed size: 33 M 2024-04-27T03:58:05.115 INFO:teuthology.orchestra.run.smithi181.stdout:Downloading Packages: 2024-04-27T03:58:05.675 INFO:teuthology.orchestra.run.smithi181.stdout:(1/6): ceph-mgr-cephadm-17.2.7-907.g2be3efa9.el 381 kB/s | 123 kB 00:00 2024-04-27T03:58:06.959 INFO:teuthology.orchestra.run.smithi181.stdout:(2/6): python3-asyncssh-2.13.2-5.el9.noarch.rpm 427 kB/s | 548 kB 00:01 2024-04-27T03:58:07.067 INFO:teuthology.orchestra.run.smithi181.stdout:(3/6): python3-natsort-7.1.1-5.el9.noarch.rpm 533 kB/s | 58 kB 00:00 2024-04-27T03:58:07.176 INFO:teuthology.orchestra.run.smithi181.stdout:(4/6): python3-typing-extensions-4.4.0-2.el9.no 470 kB/s | 51 kB 00:00 2024-04-27T03:58:45.980 INFO:teuthology.orchestra.run.smithi181.stdout:(5/6): python3-jinja2-2.11.3-5.el9.noarch.rpm 6.1 kB/s | 248 kB 00:40 2024-04-27T03:58:46.254 INFO:teuthology.orchestra.run.smithi181.stdout:(6/6): python3-babel-2.9.1-2.el9.noarch.rpm 149 kB/s | 6.0 MB 00:40 2024-04-27T03:58:46.255 INFO:teuthology.orchestra.run.smithi181.stdout:-------------------------------------------------------------------------------- 2024-04-27T03:58:46.255 INFO:teuthology.orchestra.run.smithi181.stdout:Total 173 kB/s | 7.0 MB 00:41 2024-04-27T03:58:46.325 INFO:teuthology.orchestra.run.smithi181.stdout:Running transaction check 2024-04-27T03:58:46.337 INFO:teuthology.orchestra.run.smithi181.stdout:Transaction check succeeded. 2024-04-27T03:58:46.337 INFO:teuthology.orchestra.run.smithi181.stdout:Running transaction test 2024-04-27T03:58:46.432 INFO:teuthology.orchestra.run.smithi181.stdout:Transaction test succeeded. 2024-04-27T03:58:46.433 INFO:teuthology.orchestra.run.smithi181.stdout:Running transaction 2024-04-27T03:58:46.733 INFO:teuthology.orchestra.run.smithi181.stdout: Preparing : 1/1 2024-04-27T03:58:46.904 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 1/6 2024-04-27T03:58:46.964 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 2/6 2024-04-27T03:58:47.497 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 3/6 2024-04-27T03:58:47.585 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 4/6 2024-04-27T03:58:47.639 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 5/6 2024-04-27T03:58:47.668 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : ceph-mgr-cephadm-2:17.2.7-907.g2be3efa9.el9.noarch 6/6 2024-04-27T03:58:48.451 INFO:teuthology.orchestra.run.smithi181.stdout: Running scriptlet: ceph-mgr-cephadm-2:17.2.7-907.g2be3efa9.el9.noarch 6/6 2024-04-27T03:58:48.451 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : ceph-mgr-cephadm-2:17.2.7-907.g2be3efa9.el9.noarch 1/6 2024-04-27T03:58:48.451 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 2/6 2024-04-27T03:58:48.451 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 3/6 2024-04-27T03:58:48.451 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 4/6 2024-04-27T03:58:48.451 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 5/6 2024-04-27T03:58:48.703 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 6/6 2024-04-27T03:58:48.704 INFO:teuthology.orchestra.run.smithi181.stdout: 2024-04-27T03:58:48.704 INFO:teuthology.orchestra.run.smithi181.stdout:Installed: 2024-04-27T03:58:48.704 INFO:teuthology.orchestra.run.smithi181.stdout: ceph-mgr-cephadm-2:17.2.7-907.g2be3efa9.el9.noarch 2024-04-27T03:58:48.704 INFO:teuthology.orchestra.run.smithi181.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-04-27T03:58:48.704 INFO:teuthology.orchestra.run.smithi181.stdout: python3-babel-2.9.1-2.el9.noarch 2024-04-27T03:58:48.704 INFO:teuthology.orchestra.run.smithi181.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-04-27T03:58:48.704 INFO:teuthology.orchestra.run.smithi181.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-04-27T03:58:48.704 INFO:teuthology.orchestra.run.smithi181.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-04-27T03:58:48.704 INFO:teuthology.orchestra.run.smithi181.stdout: 2024-04-27T03:58:48.704 INFO:teuthology.orchestra.run.smithi181.stdout:Complete! 2024-04-27T03:58:48.945 DEBUG:teuthology.orchestra.run.smithi181:> sudo yum -y install ceph-fuse 2024-04-27T03:58:49.479 INFO:teuthology.orchestra.run.smithi181.stdout:Last metadata expiration check: 0:02:44 ago on Sat 27 Apr 2024 03:56:05 AM UTC. 2024-04-27T03:58:49.600 INFO:teuthology.orchestra.run.smithi181.stdout:Dependencies resolved. 2024-04-27T03:58:49.601 INFO:teuthology.orchestra.run.smithi181.stdout:================================================================================ 2024-04-27T03:58:49.601 INFO:teuthology.orchestra.run.smithi181.stdout: Package Arch Version Repository Size 2024-04-27T03:58:49.601 INFO:teuthology.orchestra.run.smithi181.stdout:================================================================================ 2024-04-27T03:58:49.601 INFO:teuthology.orchestra.run.smithi181.stdout:Installing: 2024-04-27T03:58:49.601 INFO:teuthology.orchestra.run.smithi181.stdout: ceph-fuse x86_64 2:17.2.7-907.g2be3efa9.el9 ceph 812 k 2024-04-27T03:58:49.601 INFO:teuthology.orchestra.run.smithi181.stdout:Installing dependencies: 2024-04-27T03:58:49.602 INFO:teuthology.orchestra.run.smithi181.stdout: fuse x86_64 2.9.9-15.el9 baseos 80 k 2024-04-27T03:58:49.602 INFO:teuthology.orchestra.run.smithi181.stdout: 2024-04-27T03:58:49.602 INFO:teuthology.orchestra.run.smithi181.stdout:Transaction Summary 2024-04-27T03:58:49.602 INFO:teuthology.orchestra.run.smithi181.stdout:================================================================================ 2024-04-27T03:58:49.602 INFO:teuthology.orchestra.run.smithi181.stdout:Install 2 Packages 2024-04-27T03:58:49.602 INFO:teuthology.orchestra.run.smithi181.stdout: 2024-04-27T03:58:49.602 INFO:teuthology.orchestra.run.smithi181.stdout:Total download size: 893 k 2024-04-27T03:58:49.603 INFO:teuthology.orchestra.run.smithi181.stdout:Installed size: 2.6 M 2024-04-27T03:58:49.603 INFO:teuthology.orchestra.run.smithi181.stdout:Downloading Packages: 2024-04-27T03:58:49.848 INFO:teuthology.orchestra.run.smithi181.stdout:(1/2): fuse-2.9.9-15.el9.x86_64.rpm 634 kB/s | 80 kB 00:00 2024-04-27T03:58:50.048 INFO:teuthology.orchestra.run.smithi181.stdout:(2/2): ceph-fuse-17.2.7-907.g2be3efa9.el9.x86_6 2.4 MB/s | 812 kB 00:00 2024-04-27T03:58:50.049 INFO:teuthology.orchestra.run.smithi181.stdout:-------------------------------------------------------------------------------- 2024-04-27T03:58:50.049 INFO:teuthology.orchestra.run.smithi181.stdout:Total 1.9 MB/s | 893 kB 00:00 2024-04-27T03:58:50.062 INFO:teuthology.orchestra.run.smithi181.stdout:Running transaction check 2024-04-27T03:58:50.082 INFO:teuthology.orchestra.run.smithi181.stdout:Transaction check succeeded. 2024-04-27T03:58:50.082 INFO:teuthology.orchestra.run.smithi181.stdout:Running transaction test 2024-04-27T03:58:50.158 INFO:teuthology.orchestra.run.smithi181.stdout:Transaction test succeeded. 2024-04-27T03:58:50.158 INFO:teuthology.orchestra.run.smithi181.stdout:Running transaction 2024-04-27T03:58:50.337 INFO:teuthology.orchestra.run.smithi181.stdout: Preparing : 1/1 2024-04-27T03:58:50.468 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : fuse-2.9.9-15.el9.x86_64 1/2 2024-04-27T03:58:50.537 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : ceph-fuse-2:17.2.7-907.g2be3efa9.el9.x86_64 2/2 2024-04-27T03:58:51.198 INFO:teuthology.orchestra.run.smithi181.stdout: Running scriptlet: ceph-fuse-2:17.2.7-907.g2be3efa9.el9.x86_64 2/2 2024-04-27T03:58:51.198 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : ceph-fuse-2:17.2.7-907.g2be3efa9.el9.x86_64 1/2 2024-04-27T03:58:51.446 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 2/2 2024-04-27T03:58:51.446 INFO:teuthology.orchestra.run.smithi181.stdout: 2024-04-27T03:58:51.446 INFO:teuthology.orchestra.run.smithi181.stdout:Installed: 2024-04-27T03:58:51.446 INFO:teuthology.orchestra.run.smithi181.stdout: ceph-fuse-2:17.2.7-907.g2be3efa9.el9.x86_64 fuse-2.9.9-15.el9.x86_64 2024-04-27T03:58:51.446 INFO:teuthology.orchestra.run.smithi181.stdout: 2024-04-27T03:58:51.446 INFO:teuthology.orchestra.run.smithi181.stdout:Complete! 2024-04-27T03:58:51.699 DEBUG:teuthology.orchestra.run.smithi181:> sudo yum -y install librados-devel 2024-04-27T03:58:52.221 INFO:teuthology.orchestra.run.smithi181.stdout:Last metadata expiration check: 0:02:47 ago on Sat 27 Apr 2024 03:56:05 AM UTC. 2024-04-27T03:58:52.345 INFO:teuthology.orchestra.run.smithi181.stdout:Dependencies resolved. 2024-04-27T03:58:52.346 INFO:teuthology.orchestra.run.smithi181.stdout:================================================================================ 2024-04-27T03:58:52.346 INFO:teuthology.orchestra.run.smithi181.stdout: Package Arch Version Repo Size 2024-04-27T03:58:52.346 INFO:teuthology.orchestra.run.smithi181.stdout:================================================================================ 2024-04-27T03:58:52.346 INFO:teuthology.orchestra.run.smithi181.stdout:Installing: 2024-04-27T03:58:52.346 INFO:teuthology.orchestra.run.smithi181.stdout: librados-devel x86_64 2:17.2.7-907.g2be3efa9.el9 ceph 126 k 2024-04-27T03:58:52.346 INFO:teuthology.orchestra.run.smithi181.stdout: 2024-04-27T03:58:52.346 INFO:teuthology.orchestra.run.smithi181.stdout:Transaction Summary 2024-04-27T03:58:52.346 INFO:teuthology.orchestra.run.smithi181.stdout:================================================================================ 2024-04-27T03:58:52.346 INFO:teuthology.orchestra.run.smithi181.stdout:Install 1 Package 2024-04-27T03:58:52.346 INFO:teuthology.orchestra.run.smithi181.stdout: 2024-04-27T03:58:52.347 INFO:teuthology.orchestra.run.smithi181.stdout:Total download size: 126 k 2024-04-27T03:58:52.347 INFO:teuthology.orchestra.run.smithi181.stdout:Installed size: 448 k 2024-04-27T03:58:52.347 INFO:teuthology.orchestra.run.smithi181.stdout:Downloading Packages: 2024-04-27T03:58:52.580 INFO:teuthology.orchestra.run.smithi181.stdout:librados-devel-17.2.7-907.g2be3efa9.el9.x86_64. 543 kB/s | 126 kB 00:00 2024-04-27T03:58:52.580 INFO:teuthology.orchestra.run.smithi181.stdout:-------------------------------------------------------------------------------- 2024-04-27T03:58:52.580 INFO:teuthology.orchestra.run.smithi181.stdout:Total 539 kB/s | 126 kB 00:00 2024-04-27T03:58:52.581 INFO:teuthology.orchestra.run.smithi181.stdout:Running transaction check 2024-04-27T03:58:52.586 INFO:teuthology.orchestra.run.smithi181.stdout:Transaction check succeeded. 2024-04-27T03:58:52.586 INFO:teuthology.orchestra.run.smithi181.stdout:Running transaction test 2024-04-27T03:58:52.648 INFO:teuthology.orchestra.run.smithi181.stdout:Transaction test succeeded. 2024-04-27T03:58:52.648 INFO:teuthology.orchestra.run.smithi181.stdout:Running transaction 2024-04-27T03:58:52.796 INFO:teuthology.orchestra.run.smithi181.stdout: Preparing : 1/1 2024-04-27T03:58:52.900 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : librados-devel-2:17.2.7-907.g2be3efa9.el9.x86_64 1/1 2024-04-27T03:58:53.312 INFO:teuthology.orchestra.run.smithi181.stdout: Running scriptlet: librados-devel-2:17.2.7-907.g2be3efa9.el9.x86_64 1/1 2024-04-27T03:58:53.527 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : librados-devel-2:17.2.7-907.g2be3efa9.el9.x86_64 1/1 2024-04-27T03:58:53.527 INFO:teuthology.orchestra.run.smithi181.stdout: 2024-04-27T03:58:53.527 INFO:teuthology.orchestra.run.smithi181.stdout:Installed: 2024-04-27T03:58:53.527 INFO:teuthology.orchestra.run.smithi181.stdout: librados-devel-2:17.2.7-907.g2be3efa9.el9.x86_64 2024-04-27T03:58:53.527 INFO:teuthology.orchestra.run.smithi181.stdout: 2024-04-27T03:58:53.527 INFO:teuthology.orchestra.run.smithi181.stdout:Complete! 2024-04-27T03:58:53.764 DEBUG:teuthology.orchestra.run.smithi181:> sudo yum -y install libcephfs2 2024-04-27T03:58:54.283 INFO:teuthology.orchestra.run.smithi181.stdout:Last metadata expiration check: 0:02:49 ago on Sat 27 Apr 2024 03:56:05 AM UTC. 2024-04-27T03:58:54.365 INFO:teuthology.orchestra.run.smithi181.stdout:Package libcephfs2-2:17.2.7-907.g2be3efa9.el9.x86_64 is already installed. 2024-04-27T03:58:54.407 INFO:teuthology.orchestra.run.smithi181.stdout:Dependencies resolved. 2024-04-27T03:58:54.408 INFO:teuthology.orchestra.run.smithi181.stdout:Nothing to do. 2024-04-27T03:58:54.409 INFO:teuthology.orchestra.run.smithi181.stdout:Complete! 2024-04-27T03:58:54.494 DEBUG:teuthology.orchestra.run.smithi181:> sudo yum -y install libcephfs-devel 2024-04-27T03:58:55.027 INFO:teuthology.orchestra.run.smithi181.stdout:Last metadata expiration check: 0:02:50 ago on Sat 27 Apr 2024 03:56:05 AM UTC. 2024-04-27T03:58:55.147 INFO:teuthology.orchestra.run.smithi181.stdout:Dependencies resolved. 2024-04-27T03:58:55.147 INFO:teuthology.orchestra.run.smithi181.stdout:================================================================================ 2024-04-27T03:58:55.147 INFO:teuthology.orchestra.run.smithi181.stdout: Package Arch Version Repo Size 2024-04-27T03:58:55.148 INFO:teuthology.orchestra.run.smithi181.stdout:================================================================================ 2024-04-27T03:58:55.148 INFO:teuthology.orchestra.run.smithi181.stdout:Installing: 2024-04-27T03:58:55.148 INFO:teuthology.orchestra.run.smithi181.stdout: libcephfs-devel x86_64 2:17.2.7-907.g2be3efa9.el9 ceph 25 k 2024-04-27T03:58:55.148 INFO:teuthology.orchestra.run.smithi181.stdout: 2024-04-27T03:58:55.148 INFO:teuthology.orchestra.run.smithi181.stdout:Transaction Summary 2024-04-27T03:58:55.148 INFO:teuthology.orchestra.run.smithi181.stdout:================================================================================ 2024-04-27T03:58:55.148 INFO:teuthology.orchestra.run.smithi181.stdout:Install 1 Package 2024-04-27T03:58:55.148 INFO:teuthology.orchestra.run.smithi181.stdout: 2024-04-27T03:58:55.149 INFO:teuthology.orchestra.run.smithi181.stdout:Total download size: 25 k 2024-04-27T03:58:55.149 INFO:teuthology.orchestra.run.smithi181.stdout:Installed size: 110 k 2024-04-27T03:58:55.149 INFO:teuthology.orchestra.run.smithi181.stdout:Downloading Packages: 2024-04-27T03:58:55.325 INFO:teuthology.orchestra.run.smithi181.stdout:libcephfs-devel-17.2.7-907.g2be3efa9.el9.x86_64 141 kB/s | 25 kB 00:00 2024-04-27T03:58:55.326 INFO:teuthology.orchestra.run.smithi181.stdout:-------------------------------------------------------------------------------- 2024-04-27T03:58:55.326 INFO:teuthology.orchestra.run.smithi181.stdout:Total 140 kB/s | 25 kB 00:00 2024-04-27T03:58:55.326 INFO:teuthology.orchestra.run.smithi181.stdout:Running transaction check 2024-04-27T03:58:55.329 INFO:teuthology.orchestra.run.smithi181.stdout:Transaction check succeeded. 2024-04-27T03:58:55.329 INFO:teuthology.orchestra.run.smithi181.stdout:Running transaction test 2024-04-27T03:58:55.331 INFO:teuthology.orchestra.run.smithi181.stdout:Transaction test succeeded. 2024-04-27T03:58:55.331 INFO:teuthology.orchestra.run.smithi181.stdout:Running transaction 2024-04-27T03:58:55.417 INFO:teuthology.orchestra.run.smithi181.stdout: Preparing : 1/1 2024-04-27T03:58:55.536 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : libcephfs-devel-2:17.2.7-907.g2be3efa9.el9.x86_64 1/1 2024-04-27T03:58:55.865 INFO:teuthology.orchestra.run.smithi181.stdout: Running scriptlet: libcephfs-devel-2:17.2.7-907.g2be3efa9.el9.x86_64 1/1 2024-04-27T03:58:56.069 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : libcephfs-devel-2:17.2.7-907.g2be3efa9.el9.x86_64 1/1 2024-04-27T03:58:56.069 INFO:teuthology.orchestra.run.smithi181.stdout: 2024-04-27T03:58:56.069 INFO:teuthology.orchestra.run.smithi181.stdout:Installed: 2024-04-27T03:58:56.069 INFO:teuthology.orchestra.run.smithi181.stdout: libcephfs-devel-2:17.2.7-907.g2be3efa9.el9.x86_64 2024-04-27T03:58:56.069 INFO:teuthology.orchestra.run.smithi181.stdout: 2024-04-27T03:58:56.069 INFO:teuthology.orchestra.run.smithi181.stdout:Complete! 2024-04-27T03:58:56.235 DEBUG:teuthology.orchestra.run.smithi181:> sudo yum -y install librados2 2024-04-27T03:58:56.750 INFO:teuthology.orchestra.run.smithi181.stdout:Last metadata expiration check: 0:02:51 ago on Sat 27 Apr 2024 03:56:05 AM UTC. 2024-04-27T03:58:56.831 INFO:teuthology.orchestra.run.smithi181.stdout:Package librados2-2:17.2.7-907.g2be3efa9.el9.x86_64 is already installed. 2024-04-27T03:58:56.871 INFO:teuthology.orchestra.run.smithi181.stdout:Dependencies resolved. 2024-04-27T03:58:56.872 INFO:teuthology.orchestra.run.smithi181.stdout:Nothing to do. 2024-04-27T03:58:56.872 INFO:teuthology.orchestra.run.smithi181.stdout:Complete! 2024-04-27T03:58:56.938 DEBUG:teuthology.orchestra.run.smithi181:> sudo yum -y install librbd1 2024-04-27T03:58:57.455 INFO:teuthology.orchestra.run.smithi181.stdout:Last metadata expiration check: 0:02:52 ago on Sat 27 Apr 2024 03:56:05 AM UTC. 2024-04-27T03:58:57.535 INFO:teuthology.orchestra.run.smithi181.stdout:Package librbd1-2:17.2.7-907.g2be3efa9.el9.x86_64 is already installed. 2024-04-27T03:58:57.575 INFO:teuthology.orchestra.run.smithi181.stdout:Dependencies resolved. 2024-04-27T03:58:57.576 INFO:teuthology.orchestra.run.smithi181.stdout:Nothing to do. 2024-04-27T03:58:57.576 INFO:teuthology.orchestra.run.smithi181.stdout:Complete! 2024-04-27T03:58:57.637 DEBUG:teuthology.orchestra.run.smithi181:> sudo yum -y install python3-rados 2024-04-27T03:58:58.145 INFO:teuthology.orchestra.run.smithi181.stdout:Last metadata expiration check: 0:02:53 ago on Sat 27 Apr 2024 03:56:05 AM UTC. 2024-04-27T03:58:58.224 INFO:teuthology.orchestra.run.smithi181.stdout:Package python3-rados-2:17.2.7-907.g2be3efa9.el9.x86_64 is already installed. 2024-04-27T03:58:58.265 INFO:teuthology.orchestra.run.smithi181.stdout:Dependencies resolved. 2024-04-27T03:58:58.266 INFO:teuthology.orchestra.run.smithi181.stdout:Nothing to do. 2024-04-27T03:58:58.266 INFO:teuthology.orchestra.run.smithi181.stdout:Complete! 2024-04-27T03:58:58.330 DEBUG:teuthology.orchestra.run.smithi181:> sudo yum -y install python3-rgw 2024-04-27T03:58:58.834 INFO:teuthology.orchestra.run.smithi181.stdout:Last metadata expiration check: 0:02:53 ago on Sat 27 Apr 2024 03:56:05 AM UTC. 2024-04-27T03:58:58.929 INFO:teuthology.orchestra.run.smithi181.stdout:Package python3-rgw-2:17.2.7-907.g2be3efa9.el9.x86_64 is already installed. 2024-04-27T03:58:58.970 INFO:teuthology.orchestra.run.smithi181.stdout:Dependencies resolved. 2024-04-27T03:58:58.971 INFO:teuthology.orchestra.run.smithi181.stdout:Nothing to do. 2024-04-27T03:58:58.971 INFO:teuthology.orchestra.run.smithi181.stdout:Complete! 2024-04-27T03:58:59.039 DEBUG:teuthology.orchestra.run.smithi181:> sudo yum -y install python3-cephfs 2024-04-27T03:58:59.562 INFO:teuthology.orchestra.run.smithi181.stdout:Last metadata expiration check: 0:02:54 ago on Sat 27 Apr 2024 03:56:05 AM UTC. 2024-04-27T03:58:59.641 INFO:teuthology.orchestra.run.smithi181.stdout:Package python3-cephfs-2:17.2.7-907.g2be3efa9.el9.x86_64 is already installed. 2024-04-27T03:58:59.683 INFO:teuthology.orchestra.run.smithi181.stdout:Dependencies resolved. 2024-04-27T03:58:59.683 INFO:teuthology.orchestra.run.smithi181.stdout:Nothing to do. 2024-04-27T03:58:59.683 INFO:teuthology.orchestra.run.smithi181.stdout:Complete! 2024-04-27T03:58:59.747 DEBUG:teuthology.orchestra.run.smithi181:> sudo yum -y install python3-rbd 2024-04-27T03:59:00.257 INFO:teuthology.orchestra.run.smithi181.stdout:Last metadata expiration check: 0:02:55 ago on Sat 27 Apr 2024 03:56:05 AM UTC. 2024-04-27T03:59:00.335 INFO:teuthology.orchestra.run.smithi181.stdout:Package python3-rbd-2:17.2.7-907.g2be3efa9.el9.x86_64 is already installed. 2024-04-27T03:59:00.376 INFO:teuthology.orchestra.run.smithi181.stdout:Dependencies resolved. 2024-04-27T03:59:00.377 INFO:teuthology.orchestra.run.smithi181.stdout:Nothing to do. 2024-04-27T03:59:00.377 INFO:teuthology.orchestra.run.smithi181.stdout:Complete! 2024-04-27T03:59:00.434 DEBUG:teuthology.orchestra.run.smithi181:> sudo yum -y install rbd-fuse 2024-04-27T03:59:00.950 INFO:teuthology.orchestra.run.smithi181.stdout:Last metadata expiration check: 0:02:55 ago on Sat 27 Apr 2024 03:56:05 AM UTC. 2024-04-27T03:59:01.071 INFO:teuthology.orchestra.run.smithi181.stdout:Dependencies resolved. 2024-04-27T03:59:01.072 INFO:teuthology.orchestra.run.smithi181.stdout:================================================================================ 2024-04-27T03:59:01.072 INFO:teuthology.orchestra.run.smithi181.stdout: Package Architecture Version Repository Size 2024-04-27T03:59:01.072 INFO:teuthology.orchestra.run.smithi181.stdout:================================================================================ 2024-04-27T03:59:01.072 INFO:teuthology.orchestra.run.smithi181.stdout:Installing: 2024-04-27T03:59:01.072 INFO:teuthology.orchestra.run.smithi181.stdout: rbd-fuse x86_64 2:17.2.7-907.g2be3efa9.el9 ceph 82 k 2024-04-27T03:59:01.072 INFO:teuthology.orchestra.run.smithi181.stdout: 2024-04-27T03:59:01.072 INFO:teuthology.orchestra.run.smithi181.stdout:Transaction Summary 2024-04-27T03:59:01.072 INFO:teuthology.orchestra.run.smithi181.stdout:================================================================================ 2024-04-27T03:59:01.072 INFO:teuthology.orchestra.run.smithi181.stdout:Install 1 Package 2024-04-27T03:59:01.072 INFO:teuthology.orchestra.run.smithi181.stdout: 2024-04-27T03:59:01.072 INFO:teuthology.orchestra.run.smithi181.stdout:Total download size: 82 k 2024-04-27T03:59:01.072 INFO:teuthology.orchestra.run.smithi181.stdout:Installed size: 222 k 2024-04-27T03:59:01.073 INFO:teuthology.orchestra.run.smithi181.stdout:Downloading Packages: 2024-04-27T03:59:01.278 INFO:teuthology.orchestra.run.smithi181.stdout:rbd-fuse-17.2.7-907.g2be3efa9.el9.x86_64.rpm 398 kB/s | 82 kB 00:00 2024-04-27T03:59:01.279 INFO:teuthology.orchestra.run.smithi181.stdout:-------------------------------------------------------------------------------- 2024-04-27T03:59:01.279 INFO:teuthology.orchestra.run.smithi181.stdout:Total 395 kB/s | 82 kB 00:00 2024-04-27T03:59:01.279 INFO:teuthology.orchestra.run.smithi181.stdout:Running transaction check 2024-04-27T03:59:01.285 INFO:teuthology.orchestra.run.smithi181.stdout:Transaction check succeeded. 2024-04-27T03:59:01.285 INFO:teuthology.orchestra.run.smithi181.stdout:Running transaction test 2024-04-27T03:59:01.349 INFO:teuthology.orchestra.run.smithi181.stdout:Transaction test succeeded. 2024-04-27T03:59:01.349 INFO:teuthology.orchestra.run.smithi181.stdout:Running transaction 2024-04-27T03:59:01.498 INFO:teuthology.orchestra.run.smithi181.stdout: Preparing : 1/1 2024-04-27T03:59:01.606 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : rbd-fuse-2:17.2.7-907.g2be3efa9.el9.x86_64 1/1 2024-04-27T03:59:01.942 INFO:teuthology.orchestra.run.smithi181.stdout: Running scriptlet: rbd-fuse-2:17.2.7-907.g2be3efa9.el9.x86_64 1/1 2024-04-27T03:59:02.159 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : rbd-fuse-2:17.2.7-907.g2be3efa9.el9.x86_64 1/1 2024-04-27T03:59:02.159 INFO:teuthology.orchestra.run.smithi181.stdout: 2024-04-27T03:59:02.159 INFO:teuthology.orchestra.run.smithi181.stdout:Installed: 2024-04-27T03:59:02.159 INFO:teuthology.orchestra.run.smithi181.stdout: rbd-fuse-2:17.2.7-907.g2be3efa9.el9.x86_64 2024-04-27T03:59:02.159 INFO:teuthology.orchestra.run.smithi181.stdout: 2024-04-27T03:59:02.159 INFO:teuthology.orchestra.run.smithi181.stdout:Complete! 2024-04-27T03:59:02.382 DEBUG:teuthology.orchestra.run.smithi181:> sudo yum -y install rbd-mirror 2024-04-27T03:59:02.901 INFO:teuthology.orchestra.run.smithi181.stdout:Last metadata expiration check: 0:02:57 ago on Sat 27 Apr 2024 03:56:05 AM UTC. 2024-04-27T03:59:03.023 INFO:teuthology.orchestra.run.smithi181.stdout:Dependencies resolved. 2024-04-27T03:59:03.024 INFO:teuthology.orchestra.run.smithi181.stdout:================================================================================ 2024-04-27T03:59:03.024 INFO:teuthology.orchestra.run.smithi181.stdout: Package Arch Version Repo Size 2024-04-27T03:59:03.024 INFO:teuthology.orchestra.run.smithi181.stdout:================================================================================ 2024-04-27T03:59:03.024 INFO:teuthology.orchestra.run.smithi181.stdout:Installing: 2024-04-27T03:59:03.024 INFO:teuthology.orchestra.run.smithi181.stdout: rbd-mirror x86_64 2:17.2.7-907.g2be3efa9.el9 ceph 2.9 M 2024-04-27T03:59:03.024 INFO:teuthology.orchestra.run.smithi181.stdout: 2024-04-27T03:59:03.024 INFO:teuthology.orchestra.run.smithi181.stdout:Transaction Summary 2024-04-27T03:59:03.024 INFO:teuthology.orchestra.run.smithi181.stdout:================================================================================ 2024-04-27T03:59:03.025 INFO:teuthology.orchestra.run.smithi181.stdout:Install 1 Package 2024-04-27T03:59:03.025 INFO:teuthology.orchestra.run.smithi181.stdout: 2024-04-27T03:59:03.025 INFO:teuthology.orchestra.run.smithi181.stdout:Total download size: 2.9 M 2024-04-27T03:59:03.025 INFO:teuthology.orchestra.run.smithi181.stdout:Installed size: 12 M 2024-04-27T03:59:03.025 INFO:teuthology.orchestra.run.smithi181.stdout:Downloading Packages: 2024-04-27T03:59:03.407 INFO:teuthology.orchestra.run.smithi181.stdout:rbd-mirror-17.2.7-907.g2be3efa9.el9.x86_64.rpm 7.7 MB/s | 2.9 MB 00:00 2024-04-27T03:59:03.408 INFO:teuthology.orchestra.run.smithi181.stdout:-------------------------------------------------------------------------------- 2024-04-27T03:59:03.408 INFO:teuthology.orchestra.run.smithi181.stdout:Total 7.7 MB/s | 2.9 MB 00:00 2024-04-27T03:59:03.408 INFO:teuthology.orchestra.run.smithi181.stdout:Running transaction check 2024-04-27T03:59:03.417 INFO:teuthology.orchestra.run.smithi181.stdout:Transaction check succeeded. 2024-04-27T03:59:03.417 INFO:teuthology.orchestra.run.smithi181.stdout:Running transaction test 2024-04-27T03:59:03.496 INFO:teuthology.orchestra.run.smithi181.stdout:Transaction test succeeded. 2024-04-27T03:59:03.496 INFO:teuthology.orchestra.run.smithi181.stdout:Running transaction 2024-04-27T03:59:03.914 INFO:teuthology.orchestra.run.smithi181.stdout: Preparing : 1/1 2024-04-27T03:59:03.983 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : rbd-mirror-2:17.2.7-907.g2be3efa9.el9.x86_64 1/1 2024-04-27T03:59:04.007 INFO:teuthology.orchestra.run.smithi181.stdout: Running scriptlet: rbd-mirror-2:17.2.7-907.g2be3efa9.el9.x86_64 1/1 2024-04-27T03:59:04.008 INFO:teuthology.orchestra.run.smithi181.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-27T03:59:04.008 INFO:teuthology.orchestra.run.smithi181.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-04-27T03:59:04.008 INFO:teuthology.orchestra.run.smithi181.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-27T03:59:04.008 INFO:teuthology.orchestra.run.smithi181.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-27T03:59:04.008 INFO:teuthology.orchestra.run.smithi181.stdout: 2024-04-27T03:59:05.070 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : rbd-mirror-2:17.2.7-907.g2be3efa9.el9.x86_64 1/1 2024-04-27T03:59:05.070 INFO:teuthology.orchestra.run.smithi181.stdout: 2024-04-27T03:59:05.070 INFO:teuthology.orchestra.run.smithi181.stdout:Installed: 2024-04-27T03:59:05.070 INFO:teuthology.orchestra.run.smithi181.stdout: rbd-mirror-2:17.2.7-907.g2be3efa9.el9.x86_64 2024-04-27T03:59:05.070 INFO:teuthology.orchestra.run.smithi181.stdout: 2024-04-27T03:59:05.070 INFO:teuthology.orchestra.run.smithi181.stdout:Complete! 2024-04-27T03:59:05.335 DEBUG:teuthology.orchestra.run.smithi181:> sudo yum -y install rbd-nbd 2024-04-27T03:59:05.857 INFO:teuthology.orchestra.run.smithi181.stdout:Last metadata expiration check: 0:03:00 ago on Sat 27 Apr 2024 03:56:05 AM UTC. 2024-04-27T03:59:05.979 INFO:teuthology.orchestra.run.smithi181.stdout:Dependencies resolved. 2024-04-27T03:59:05.979 INFO:teuthology.orchestra.run.smithi181.stdout:================================================================================ 2024-04-27T03:59:05.980 INFO:teuthology.orchestra.run.smithi181.stdout: Package Architecture Version Repository Size 2024-04-27T03:59:05.980 INFO:teuthology.orchestra.run.smithi181.stdout:================================================================================ 2024-04-27T03:59:05.980 INFO:teuthology.orchestra.run.smithi181.stdout:Installing: 2024-04-27T03:59:05.980 INFO:teuthology.orchestra.run.smithi181.stdout: rbd-nbd x86_64 2:17.2.7-907.g2be3efa9.el9 ceph 165 k 2024-04-27T03:59:05.980 INFO:teuthology.orchestra.run.smithi181.stdout: 2024-04-27T03:59:05.980 INFO:teuthology.orchestra.run.smithi181.stdout:Transaction Summary 2024-04-27T03:59:05.980 INFO:teuthology.orchestra.run.smithi181.stdout:================================================================================ 2024-04-27T03:59:05.980 INFO:teuthology.orchestra.run.smithi181.stdout:Install 1 Package 2024-04-27T03:59:05.980 INFO:teuthology.orchestra.run.smithi181.stdout: 2024-04-27T03:59:05.980 INFO:teuthology.orchestra.run.smithi181.stdout:Total download size: 165 k 2024-04-27T03:59:05.980 INFO:teuthology.orchestra.run.smithi181.stdout:Installed size: 477 k 2024-04-27T03:59:05.980 INFO:teuthology.orchestra.run.smithi181.stdout:Downloading Packages: 2024-04-27T03:59:06.211 INFO:teuthology.orchestra.run.smithi181.stdout:rbd-nbd-17.2.7-907.g2be3efa9.el9.x86_64.rpm 717 kB/s | 165 kB 00:00 2024-04-27T03:59:06.212 INFO:teuthology.orchestra.run.smithi181.stdout:-------------------------------------------------------------------------------- 2024-04-27T03:59:06.212 INFO:teuthology.orchestra.run.smithi181.stdout:Total 712 kB/s | 165 kB 00:00 2024-04-27T03:59:06.212 INFO:teuthology.orchestra.run.smithi181.stdout:Running transaction check 2024-04-27T03:59:06.219 INFO:teuthology.orchestra.run.smithi181.stdout:Transaction check succeeded. 2024-04-27T03:59:06.220 INFO:teuthology.orchestra.run.smithi181.stdout:Running transaction test 2024-04-27T03:59:06.281 INFO:teuthology.orchestra.run.smithi181.stdout:Transaction test succeeded. 2024-04-27T03:59:06.282 INFO:teuthology.orchestra.run.smithi181.stdout:Running transaction 2024-04-27T03:59:06.432 INFO:teuthology.orchestra.run.smithi181.stdout: Preparing : 1/1 2024-04-27T03:59:06.524 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : rbd-nbd-2:17.2.7-907.g2be3efa9.el9.x86_64 1/1 2024-04-27T03:59:06.933 INFO:teuthology.orchestra.run.smithi181.stdout: Running scriptlet: rbd-nbd-2:17.2.7-907.g2be3efa9.el9.x86_64 1/1 2024-04-27T03:59:07.183 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : rbd-nbd-2:17.2.7-907.g2be3efa9.el9.x86_64 1/1 2024-04-27T03:59:07.183 INFO:teuthology.orchestra.run.smithi181.stdout: 2024-04-27T03:59:07.183 INFO:teuthology.orchestra.run.smithi181.stdout:Installed: 2024-04-27T03:59:07.183 INFO:teuthology.orchestra.run.smithi181.stdout: rbd-nbd-2:17.2.7-907.g2be3efa9.el9.x86_64 2024-04-27T03:59:07.183 INFO:teuthology.orchestra.run.smithi181.stdout: 2024-04-27T03:59:07.183 INFO:teuthology.orchestra.run.smithi181.stdout:Complete! 2024-04-27T03:59:07.406 DEBUG:teuthology.parallel:result is None 2024-04-27T03:59:07.406 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-04-27T03:59:07.406 INFO:teuthology.packaging:ref: None 2024-04-27T03:59:07.407 INFO:teuthology.packaging:tag: None 2024-04-27T03:59:07.407 INFO:teuthology.packaging:branch: quincy 2024-04-27T03:59:07.407 INFO:teuthology.packaging:sha1: 4e213a58e787437ff508e6c890ccd5fd32dc96b5 2024-04-27T03:59:07.407 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=quincy 2024-04-27T03:59:07.628 DEBUG:teuthology.orchestra.run.smithi162:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-04-27T03:59:07.653 INFO:teuthology.orchestra.run.smithi162.stdout:17.2.7-907.g2be3efa9.el9 2024-04-27T03:59:07.654 INFO:teuthology.packaging:The installed version of ceph is 17.2.7-907.g2be3efa9.el9 2024-04-27T03:59:07.654 INFO:teuthology.task.install:The correct ceph version 17.2.7-907.g2be3efa9 is installed. 2024-04-27T03:59:07.656 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-04-27T03:59:07.656 INFO:teuthology.packaging:ref: None 2024-04-27T03:59:07.656 INFO:teuthology.packaging:tag: None 2024-04-27T03:59:07.656 INFO:teuthology.packaging:branch: quincy 2024-04-27T03:59:07.656 INFO:teuthology.packaging:sha1: 4e213a58e787437ff508e6c890ccd5fd32dc96b5 2024-04-27T03:59:07.656 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=quincy 2024-04-27T03:59:07.890 DEBUG:teuthology.orchestra.run.smithi181:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-04-27T03:59:07.911 INFO:teuthology.orchestra.run.smithi181.stdout:17.2.7-907.g2be3efa9.el9 2024-04-27T03:59:07.911 INFO:teuthology.packaging:The installed version of ceph is 17.2.7-907.g2be3efa9.el9 2024-04-27T03:59:07.911 INFO:teuthology.task.install:The correct ceph version 17.2.7-907.g2be3efa9 is installed. 2024-04-27T03:59:07.913 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-04-27T03:59:07.913 DEBUG:teuthology.orchestra.run.smithi162:> set -ex 2024-04-27T03:59:07.913 DEBUG:teuthology.orchestra.run.smithi162:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-04-27T03:59:07.942 DEBUG:teuthology.orchestra.run.smithi181:> set -ex 2024-04-27T03:59:07.943 DEBUG:teuthology.orchestra.run.smithi181:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-04-27T03:59:07.974 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-04-27T03:59:07.975 DEBUG:teuthology.orchestra.run.smithi162:> set -ex 2024-04-27T03:59:07.975 DEBUG:teuthology.orchestra.run.smithi162:> sudo dd of=/usr/bin/daemon-helper 2024-04-27T03:59:08.009 DEBUG:teuthology.orchestra.run.smithi162:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-04-27T03:59:08.076 DEBUG:teuthology.orchestra.run.smithi181:> set -ex 2024-04-27T03:59:08.076 DEBUG:teuthology.orchestra.run.smithi181:> sudo dd of=/usr/bin/daemon-helper 2024-04-27T03:59:08.100 DEBUG:teuthology.orchestra.run.smithi181:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-04-27T03:59:08.164 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-04-27T03:59:08.164 DEBUG:teuthology.orchestra.run.smithi162:> set -ex 2024-04-27T03:59:08.164 DEBUG:teuthology.orchestra.run.smithi162:> sudo dd of=/usr/bin/adjust-ulimits 2024-04-27T03:59:08.190 DEBUG:teuthology.orchestra.run.smithi162:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-04-27T03:59:08.256 DEBUG:teuthology.orchestra.run.smithi181:> set -ex 2024-04-27T03:59:08.256 DEBUG:teuthology.orchestra.run.smithi181:> sudo dd of=/usr/bin/adjust-ulimits 2024-04-27T03:59:08.279 DEBUG:teuthology.orchestra.run.smithi181:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-04-27T03:59:08.341 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-04-27T03:59:08.342 DEBUG:teuthology.orchestra.run.smithi162:> set -ex 2024-04-27T03:59:08.342 DEBUG:teuthology.orchestra.run.smithi162:> sudo dd of=/usr/bin/stdin-killer 2024-04-27T03:59:08.365 DEBUG:teuthology.orchestra.run.smithi162:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-04-27T03:59:08.430 DEBUG:teuthology.orchestra.run.smithi181:> set -ex 2024-04-27T03:59:08.430 DEBUG:teuthology.orchestra.run.smithi181:> sudo dd of=/usr/bin/stdin-killer 2024-04-27T03:59:08.453 DEBUG:teuthology.orchestra.run.smithi181:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-04-27T03:59:08.516 INFO:teuthology.run_tasks:Running task print... 2024-04-27T03:59:08.541 INFO:teuthology.task.print:**** done install task... 2024-04-27T03:59:08.542 INFO:teuthology.run_tasks:Running task cephadm... 2024-04-27T03:59:08.648 INFO:tasks.cephadm:Config: {'cephadm_branch': 'quincy', 'cephadm_git_url': 'https://github.com/ceph/ceph', 'conf': {'osd': {'osd_class_default_list': '*', 'osd_class_load_list': '*', 'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}, 'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'mon pg warn min per osd': 0}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug client': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon down mkfs grace': 300, 'mon op complaint time': 120}}, 'image': 'quay.ceph.io/ceph-ci/ceph:quincy', 'roleless': True, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'do not have an application enabled', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'OSD_DOWN', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability'], 'sha1': '4e213a58e787437ff508e6c890ccd5fd32dc96b5'} 2024-04-27T03:59:08.648 INFO:tasks.cephadm:Cluster image is quay.ceph.io/ceph-ci/ceph:quincy 2024-04-27T03:59:08.649 INFO:tasks.cephadm:Cluster fsid is 7f9750d6-044a-11ef-bc93-c7b262605968 2024-04-27T03:59:08.649 INFO:tasks.cephadm:Choosing monitor IPs and ports... 2024-04-27T03:59:08.649 INFO:tasks.cephadm:No mon roles; fabricating mons 2024-04-27T03:59:08.649 INFO:tasks.cephadm:Monitor IPs: {'mon.smithi162': '172.21.15.162', 'mon.smithi181': '172.21.15.181'} 2024-04-27T03:59:08.649 INFO:tasks.cephadm:Normalizing hostnames... 2024-04-27T03:59:08.649 DEBUG:teuthology.orchestra.run.smithi162:> sudo hostname $(hostname -s) 2024-04-27T03:59:08.673 DEBUG:teuthology.orchestra.run.smithi181:> sudo hostname $(hostname -s) 2024-04-27T03:59:08.698 INFO:tasks.cephadm:Downloading cephadm (repo https://github.com/ceph/ceph ref quincy)... 2024-04-27T03:59:08.698 DEBUG:teuthology.orchestra.run.smithi162:> curl --silent https://raw.githubusercontent.com/ceph/ceph/quincy/src/cephadm/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-04-27T03:59:08.878 INFO:teuthology.orchestra.run.smithi162.stdout:-rw-r--r--. 1 ubuntu ubuntu 375153 Apr 27 03:59 /home/ubuntu/cephtest/cephadm 2024-04-27T03:59:08.878 DEBUG:teuthology.orchestra.run.smithi181:> curl --silent https://raw.githubusercontent.com/ceph/ceph/quincy/src/cephadm/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-04-27T03:59:09.045 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r--. 1 ubuntu ubuntu 375153 Apr 27 03:59 /home/ubuntu/cephtest/cephadm 2024-04-27T03:59:09.045 DEBUG:teuthology.orchestra.run.smithi162:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-04-27T03:59:09.060 DEBUG:teuthology.orchestra.run.smithi181:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-04-27T03:59:09.086 INFO:tasks.cephadm:Pulling image quay.ceph.io/ceph-ci/ceph:quincy on all hosts... 2024-04-27T03:59:09.086 DEBUG:teuthology.orchestra.run.smithi162:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy pull 2024-04-27T03:59:09.103 DEBUG:teuthology.orchestra.run.smithi181:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy pull 2024-04-27T03:59:10.672 INFO:teuthology.orchestra.run.smithi162.stderr:Pulling container image quay.ceph.io/ceph-ci/ceph:quincy... 2024-04-27T03:59:10.765 INFO:teuthology.orchestra.run.smithi181.stderr:Pulling container image quay.ceph.io/ceph-ci/ceph:quincy... 2024-04-27T03:59:55.389 INFO:teuthology.orchestra.run.smithi181.stdout:{ 2024-04-27T03:59:55.389 INFO:teuthology.orchestra.run.smithi181.stdout: "ceph_version": "ceph version 17.2.7-907-g2be3efa9 (2be3efa97a7972f8e925bd899b474329979b74d6) quincy (stable)", 2024-04-27T03:59:55.389 INFO:teuthology.orchestra.run.smithi181.stdout: "image_id": "1c583463371507dd553af71bf60c9ae8747aff79bb666e4e29ef4e17d0bc3a13", 2024-04-27T03:59:55.389 INFO:teuthology.orchestra.run.smithi181.stdout: "repo_digests": [ 2024-04-27T03:59:55.389 INFO:teuthology.orchestra.run.smithi181.stdout: "quay.ceph.io/ceph-ci/ceph@sha256:d31f278ece6dbd17d1fa5e598f509017f54b76fafe1c4416391fcc51b9718ee9" 2024-04-27T03:59:55.390 INFO:teuthology.orchestra.run.smithi181.stdout: ] 2024-04-27T03:59:55.390 INFO:teuthology.orchestra.run.smithi181.stdout:} 2024-04-27T03:59:57.050 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-04-27T03:59:57.050 INFO:teuthology.orchestra.run.smithi162.stdout: "ceph_version": "ceph version 17.2.7-907-g2be3efa9 (2be3efa97a7972f8e925bd899b474329979b74d6) quincy (stable)", 2024-04-27T03:59:57.050 INFO:teuthology.orchestra.run.smithi162.stdout: "image_id": "1c583463371507dd553af71bf60c9ae8747aff79bb666e4e29ef4e17d0bc3a13", 2024-04-27T03:59:57.050 INFO:teuthology.orchestra.run.smithi162.stdout: "repo_digests": [ 2024-04-27T03:59:57.050 INFO:teuthology.orchestra.run.smithi162.stdout: "quay.ceph.io/ceph-ci/ceph@sha256:d31f278ece6dbd17d1fa5e598f509017f54b76fafe1c4416391fcc51b9718ee9" 2024-04-27T03:59:57.051 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-04-27T03:59:57.051 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-04-27T03:59:57.069 DEBUG:teuthology.orchestra.run.smithi162:> sudo mkdir -p /etc/ceph 2024-04-27T03:59:57.127 DEBUG:teuthology.orchestra.run.smithi181:> sudo mkdir -p /etc/ceph 2024-04-27T03:59:57.159 DEBUG:teuthology.orchestra.run.smithi162:> sudo chmod 777 /etc/ceph 2024-04-27T03:59:57.183 DEBUG:teuthology.orchestra.run.smithi181:> sudo chmod 777 /etc/ceph 2024-04-27T03:59:57.226 INFO:tasks.cephadm:Writing seed config... 2024-04-27T03:59:57.227 INFO:tasks.cephadm: override: [osd] osd_class_default_list = * 2024-04-27T03:59:57.227 INFO:tasks.cephadm: override: [osd] osd_class_load_list = * 2024-04-27T03:59:57.228 INFO:tasks.cephadm: override: [osd] bdev async discard = True 2024-04-27T03:59:57.228 INFO:tasks.cephadm: override: [osd] bdev enable discard = True 2024-04-27T03:59:57.228 INFO:tasks.cephadm: override: [osd] bluestore allocator = bitmap 2024-04-27T03:59:57.228 INFO:tasks.cephadm: override: [osd] bluestore block size = 96636764160 2024-04-27T03:59:57.228 INFO:tasks.cephadm: override: [osd] bluestore fsck on mount = True 2024-04-27T03:59:57.228 INFO:tasks.cephadm: override: [osd] debug bluefs = 1/20 2024-04-27T03:59:57.228 INFO:tasks.cephadm: override: [osd] debug bluestore = 1/20 2024-04-27T03:59:57.228 INFO:tasks.cephadm: override: [osd] debug ms = 1 2024-04-27T03:59:57.228 INFO:tasks.cephadm: override: [osd] debug osd = 20 2024-04-27T03:59:57.228 INFO:tasks.cephadm: override: [osd] debug rocksdb = 4/10 2024-04-27T03:59:57.228 INFO:tasks.cephadm: override: [osd] mon osd backfillfull_ratio = 0.85 2024-04-27T03:59:57.228 INFO:tasks.cephadm: override: [osd] mon osd full ratio = 0.9 2024-04-27T03:59:57.228 INFO:tasks.cephadm: override: [osd] mon osd nearfull ratio = 0.8 2024-04-27T03:59:57.228 INFO:tasks.cephadm: override: [osd] osd failsafe full ratio = 0.95 2024-04-27T03:59:57.229 INFO:tasks.cephadm: override: [osd] osd objectstore = bluestore 2024-04-27T03:59:57.229 INFO:tasks.cephadm: override: [osd] osd op complaint time = 180 2024-04-27T03:59:57.229 INFO:tasks.cephadm: override: [client] client mount timeout = 600 2024-04-27T03:59:57.229 INFO:tasks.cephadm: override: [client] debug client = 20 2024-04-27T03:59:57.229 INFO:tasks.cephadm: override: [client] debug ms = 1 2024-04-27T03:59:57.229 INFO:tasks.cephadm: override: [client] rados mon op timeout = 900 2024-04-27T03:59:57.229 INFO:tasks.cephadm: override: [client] rados osd op timeout = 900 2024-04-27T03:59:57.229 INFO:tasks.cephadm: override: [global] mon pg warn min per osd = 0 2024-04-27T03:59:57.229 INFO:tasks.cephadm: override: [mds] debug mds = 20 2024-04-27T03:59:57.229 INFO:tasks.cephadm: override: [mds] debug mds balancer = 20 2024-04-27T03:59:57.229 INFO:tasks.cephadm: override: [mds] debug ms = 1 2024-04-27T03:59:57.229 INFO:tasks.cephadm: override: [mds] mds debug frag = True 2024-04-27T03:59:57.229 INFO:tasks.cephadm: override: [mds] mds debug scatterstat = True 2024-04-27T03:59:57.229 INFO:tasks.cephadm: override: [mds] mds op complaint time = 180 2024-04-27T03:59:57.230 INFO:tasks.cephadm: override: [mds] mds verify scatter = True 2024-04-27T03:59:57.230 INFO:tasks.cephadm: override: [mds] osd op complaint time = 180 2024-04-27T03:59:57.230 INFO:tasks.cephadm: override: [mds] rados mon op timeout = 900 2024-04-27T03:59:57.230 INFO:tasks.cephadm: override: [mds] rados osd op timeout = 900 2024-04-27T03:59:57.230 INFO:tasks.cephadm: override: [mgr] debug client = 20 2024-04-27T03:59:57.230 INFO:tasks.cephadm: override: [mgr] debug mgr = 20 2024-04-27T03:59:57.230 INFO:tasks.cephadm: override: [mgr] debug ms = 1 2024-04-27T03:59:57.230 INFO:tasks.cephadm: override: [mon] debug mon = 20 2024-04-27T03:59:57.230 INFO:tasks.cephadm: override: [mon] debug ms = 1 2024-04-27T03:59:57.230 INFO:tasks.cephadm: override: [mon] debug paxos = 20 2024-04-27T03:59:57.230 INFO:tasks.cephadm: override: [mon] mon down mkfs grace = 300 2024-04-27T03:59:57.230 INFO:tasks.cephadm: override: [mon] mon op complaint time = 120 2024-04-27T03:59:57.231 DEBUG:teuthology.orchestra.run.smithi162:> set -ex 2024-04-27T03:59:57.231 DEBUG:teuthology.orchestra.run.smithi162:> dd of=/home/ubuntu/cephtest/seed.ceph.conf 2024-04-27T03:59:57.246 DEBUG:tasks.cephadm:Final config: [global] # make logging friendly to teuthology log_to_file = true log_to_stderr = false log to journald = false mon cluster log to file = true mon cluster log file level = debug mon clock drift allowed = 1.000 # replicate across OSDs, not hosts osd crush chooseleaf type = 0 #osd pool default size = 2 osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd # enable some debugging auth debug = true ms die on old message = true ms die on bug = true debug asserts on shutdown = true # adjust warnings mon max pg per osd = 10000# >= luminous mon pg warn max object skew = 0 mon osd allow primary affinity = true mon osd allow pg remap = true mon warn on legacy crush tunables = false mon warn on crush straw calc version zero = false mon warn on no sortbitwise = false mon warn on osd down out interval zero = false mon warn on too few osds = false mon_warn_on_pool_pg_num_not_power_of_two = false # disable pg_autoscaler by default for new pools osd_pool_default_pg_autoscale_mode = off # tests delete pools mon allow pool delete = true fsid = 7f9750d6-044a-11ef-bc93-c7b262605968 mon pg warn min per osd = 0 [osd] osd scrub load threshold = 5.0 osd scrub max interval = 600 osd mclock profile = high_recovery_ops osd recover clone overlap = true osd recovery max chunk = 1048576 osd deep scrub update digest min age = 30 osd map max advance = 10 osd memory target autotune = true # debugging osd debug shutdown = true osd debug op order = true osd debug verify stray on activate = true osd debug pg log writeout = true osd debug verify cached snaps = true osd debug verify missing on start = true osd debug misdirected ops = true osd op queue = debug_random osd op queue cut off = debug_random osd shutdown pgref assert = true bdev debug aio = true osd sloppy crc = true osd_class_default_list = * osd_class_load_list = * bdev async discard = True bdev enable discard = True bluestore allocator = bitmap bluestore block size = 96636764160 bluestore fsck on mount = True debug bluefs = 1/20 debug bluestore = 1/20 debug ms = 1 debug osd = 20 debug rocksdb = 4/10 mon osd backfillfull_ratio = 0.85 mon osd full ratio = 0.9 mon osd nearfull ratio = 0.8 osd failsafe full ratio = 0.95 osd objectstore = bluestore osd op complaint time = 180 [mgr] mon reweight min pgs per osd = 4 mon reweight min bytes per osd = 10 mgr/telemetry/nag = false debug client = 20 debug mgr = 20 debug ms = 1 [mon] mon data avail warn = 5 mon mgr mkfs grace = 240 mon reweight min pgs per osd = 4 mon osd reporter subtree level = osd mon osd prime pg temp = true mon reweight min bytes per osd = 10 # rotate auth tickets quickly to exercise renewal paths auth mon ticket ttl = 660# 11m auth service ticket ttl = 240# 4m # don't complain about global id reclaim mon_warn_on_insecure_global_id_reclaim = false mon_warn_on_insecure_global_id_reclaim_allowed = false debug mon = 20 debug ms = 1 debug paxos = 20 mon down mkfs grace = 300 mon op complaint time = 120 [client.rgw] rgw cache enabled = true rgw enable ops log = true rgw enable usage log = true [client] client mount timeout = 600 debug client = 20 debug ms = 1 rados mon op timeout = 900 rados osd op timeout = 900 [mds] debug mds = 20 debug mds balancer = 20 debug ms = 1 mds debug frag = True mds debug scatterstat = True mds op complaint time = 180 mds verify scatter = True osd op complaint time = 180 rados mon op timeout = 900 rados osd op timeout = 900 2024-04-27T03:59:57.246 DEBUG:teuthology.orchestra.run.smithi162:mon.smithi162> sudo journalctl -f -n 0 -u ceph-7f9750d6-044a-11ef-bc93-c7b262605968@mon.smithi162.service 2024-04-27T03:59:57.289 INFO:tasks.cephadm:Bootstrapping... 2024-04-27T03:59:57.289 DEBUG:teuthology.orchestra.run.smithi162:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy -v bootstrap --fsid 7f9750d6-044a-11ef-bc93-c7b262605968 --config /home/ubuntu/cephtest/seed.ceph.conf --output-config /etc/ceph/ceph.conf --output-keyring /etc/ceph/ceph.client.admin.keyring --output-pub-ssh-key /home/ubuntu/cephtest/ceph.pub --mon-ip 172.21.15.162 --skip-admin-label && sudo chmod +r /etc/ceph/ceph.client.admin.keyring 2024-04-27T03:59:57.487 INFO:teuthology.orchestra.run.smithi162.stdout:-------------------------------------------------------------------------------- 2024-04-27T03:59:57.487 INFO:teuthology.orchestra.run.smithi162.stdout:cephadm ['--image', 'quay.ceph.io/ceph-ci/ceph:quincy', '-v', 'bootstrap', '--fsid', '7f9750d6-044a-11ef-bc93-c7b262605968', '--config', '/home/ubuntu/cephtest/seed.ceph.conf', '--output-config', '/etc/ceph/ceph.conf', '--output-keyring', '/etc/ceph/ceph.client.admin.keyring', '--output-pub-ssh-key', '/home/ubuntu/cephtest/ceph.pub', '--mon-ip', '172.21.15.162', '--skip-admin-label'] 2024-04-27T03:59:57.537 INFO:teuthology.orchestra.run.smithi162.stdout:/bin/podman: stdout 4.9.4-dev 2024-04-27T03:59:57.538 INFO:teuthology.orchestra.run.smithi162.stderr:Specifying an fsid for your cluster offers no advantages and may increase the likelihood of fsid conflicts. 2024-04-27T03:59:57.538 INFO:teuthology.orchestra.run.smithi162.stdout:Verifying podman|docker is present... 2024-04-27T03:59:57.569 INFO:teuthology.orchestra.run.smithi162.stdout:/bin/podman: stdout 4.9.4-dev 2024-04-27T03:59:57.569 INFO:teuthology.orchestra.run.smithi162.stdout:Verifying lvm2 is present... 2024-04-27T03:59:57.569 INFO:teuthology.orchestra.run.smithi162.stdout:Verifying time synchronization is in place... 2024-04-27T03:59:57.577 INFO:teuthology.orchestra.run.smithi162.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-04-27T03:59:57.577 INFO:teuthology.orchestra.run.smithi162.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-04-27T03:59:57.584 INFO:teuthology.orchestra.run.smithi162.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-04-27T03:59:57.584 INFO:teuthology.orchestra.run.smithi162.stdout:systemctl: stdout inactive 2024-04-27T03:59:57.590 INFO:teuthology.orchestra.run.smithi162.stdout:systemctl: stdout enabled 2024-04-27T03:59:57.597 INFO:teuthology.orchestra.run.smithi162.stdout:systemctl: stdout active 2024-04-27T03:59:57.597 INFO:teuthology.orchestra.run.smithi162.stdout:Unit chronyd.service is enabled and running 2024-04-27T03:59:57.597 INFO:teuthology.orchestra.run.smithi162.stdout:Repeating the final host check... 2024-04-27T03:59:57.620 INFO:teuthology.orchestra.run.smithi162.stdout:/bin/podman: stdout 4.9.4-dev 2024-04-27T03:59:57.621 INFO:teuthology.orchestra.run.smithi162.stdout:podman (/bin/podman) version 4.9.4 is present 2024-04-27T03:59:57.621 INFO:teuthology.orchestra.run.smithi162.stdout:systemctl is present 2024-04-27T03:59:57.621 INFO:teuthology.orchestra.run.smithi162.stdout:lvcreate is present 2024-04-27T03:59:57.627 INFO:teuthology.orchestra.run.smithi162.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-04-27T03:59:57.627 INFO:teuthology.orchestra.run.smithi162.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-04-27T03:59:57.633 INFO:teuthology.orchestra.run.smithi162.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-04-27T03:59:57.634 INFO:teuthology.orchestra.run.smithi162.stdout:systemctl: stdout inactive 2024-04-27T03:59:57.640 INFO:teuthology.orchestra.run.smithi162.stdout:systemctl: stdout enabled 2024-04-27T03:59:57.647 INFO:teuthology.orchestra.run.smithi162.stdout:systemctl: stdout active 2024-04-27T03:59:57.647 INFO:teuthology.orchestra.run.smithi162.stdout:Unit chronyd.service is enabled and running 2024-04-27T03:59:57.647 INFO:teuthology.orchestra.run.smithi162.stdout:Host looks OK 2024-04-27T03:59:57.647 INFO:teuthology.orchestra.run.smithi162.stdout:Cluster fsid: 7f9750d6-044a-11ef-bc93-c7b262605968 2024-04-27T03:59:57.647 INFO:teuthology.orchestra.run.smithi162.stdout:Acquiring lock 140283684631888 on /run/cephadm/7f9750d6-044a-11ef-bc93-c7b262605968.lock 2024-04-27T03:59:57.647 INFO:teuthology.orchestra.run.smithi162.stdout:Lock 140283684631888 acquired on /run/cephadm/7f9750d6-044a-11ef-bc93-c7b262605968.lock 2024-04-27T03:59:57.647 INFO:teuthology.orchestra.run.smithi162.stdout:Verifying IP 172.21.15.162 port 3300 ... 2024-04-27T03:59:57.648 INFO:teuthology.orchestra.run.smithi162.stdout:Verifying IP 172.21.15.162 port 6789 ... 2024-04-27T03:59:57.648 INFO:teuthology.orchestra.run.smithi162.stdout:Base mon IP(s) is [172.21.15.162:3300, 172.21.15.162:6789], mon addrv is [v2:172.21.15.162:3300,v1:172.21.15.162:6789] 2024-04-27T03:59:57.652 INFO:teuthology.orchestra.run.smithi162.stdout:/sbin/ip: stdout default via 172.21.15.254 dev enp3s0f1 proto dhcp src 172.21.15.162 metric 100 2024-04-27T03:59:57.652 INFO:teuthology.orchestra.run.smithi162.stdout:/sbin/ip: stdout 172.21.0.0/20 dev enp3s0f1 proto kernel scope link src 172.21.15.162 metric 100 2024-04-27T03:59:57.655 INFO:teuthology.orchestra.run.smithi162.stdout:/sbin/ip: stdout ::1 dev lo proto kernel metric 256 pref medium 2024-04-27T03:59:57.655 INFO:teuthology.orchestra.run.smithi162.stdout:/sbin/ip: stdout fe80::/64 dev enp3s0f1 proto kernel metric 1024 pref medium 2024-04-27T03:59:57.655 INFO:teuthology.orchestra.run.smithi162.stdout:/sbin/ip: stdout default via fe80::327c:5e00:6487:71e0 dev enp3s0f1 proto ra metric 100 pref medium 2024-04-27T03:59:57.657 INFO:teuthology.orchestra.run.smithi162.stdout:/sbin/ip: stdout 1: lo: mtu 65536 state UNKNOWN qlen 1000 2024-04-27T03:59:57.657 INFO:teuthology.orchestra.run.smithi162.stdout:/sbin/ip: stdout inet6 ::1/128 scope host 2024-04-27T03:59:57.657 INFO:teuthology.orchestra.run.smithi162.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-04-27T03:59:57.657 INFO:teuthology.orchestra.run.smithi162.stdout:/sbin/ip: stdout 5: enp3s0f1: mtu 1500 state UP qlen 1000 2024-04-27T03:59:57.657 INFO:teuthology.orchestra.run.smithi162.stdout:/sbin/ip: stdout inet6 fe80::b637:b830:5199:1015/64 scope link noprefixroute 2024-04-27T03:59:57.658 INFO:teuthology.orchestra.run.smithi162.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-04-27T03:59:57.658 INFO:teuthology.orchestra.run.smithi162.stdout:Mon IP `172.21.15.162` is in CIDR network `172.21.0.0/20` 2024-04-27T03:59:57.658 INFO:teuthology.orchestra.run.smithi162.stdout:Mon IP `172.21.15.162` is in CIDR network `172.21.0.0/20` 2024-04-27T03:59:57.658 INFO:teuthology.orchestra.run.smithi162.stdout:Inferred mon public CIDR from local network configuration ['172.21.0.0/20', '172.21.0.0/20'] 2024-04-27T03:59:57.659 INFO:teuthology.orchestra.run.smithi162.stdout:Internal network (--cluster-network) has not been provided, OSD replication will default to the public_network 2024-04-27T03:59:57.659 INFO:teuthology.orchestra.run.smithi162.stdout:Pulling container image quay.ceph.io/ceph-ci/ceph:quincy... 2024-04-27T03:59:58.142 INFO:teuthology.orchestra.run.smithi162.stdout:/bin/podman: stdout 1c583463371507dd553af71bf60c9ae8747aff79bb666e4e29ef4e17d0bc3a13 2024-04-27T03:59:58.142 INFO:teuthology.orchestra.run.smithi162.stdout:/bin/podman: stderr Trying to pull quay.ceph.io/ceph-ci/ceph:quincy... 2024-04-27T03:59:58.142 INFO:teuthology.orchestra.run.smithi162.stdout:/bin/podman: stderr Getting image source signatures 2024-04-27T03:59:58.143 INFO:teuthology.orchestra.run.smithi162.stdout:/bin/podman: stderr Copying blob sha256:4bba5d13506b552b86ba49891516dd450cd5841e76166fa879451331012cf28e 2024-04-27T03:59:58.143 INFO:teuthology.orchestra.run.smithi162.stdout:/bin/podman: stderr Copying blob sha256:3d665f4c032644be9cae748af04bee69f2bfe96cdc3616dabe0ab17899633ab5 2024-04-27T03:59:58.143 INFO:teuthology.orchestra.run.smithi162.stdout:/bin/podman: stderr Copying config sha256:1c583463371507dd553af71bf60c9ae8747aff79bb666e4e29ef4e17d0bc3a13 2024-04-27T03:59:58.143 INFO:teuthology.orchestra.run.smithi162.stdout:/bin/podman: stderr Writing manifest to image destination 2024-04-27T03:59:59.048 INFO:teuthology.orchestra.run.smithi162.stdout:ceph: stdout ceph version 17.2.7-907-g2be3efa9 (2be3efa97a7972f8e925bd899b474329979b74d6) quincy (stable) 2024-04-27T03:59:59.049 INFO:teuthology.orchestra.run.smithi162.stdout:Ceph version: ceph version 17.2.7-907-g2be3efa9 (2be3efa97a7972f8e925bd899b474329979b74d6) quincy (stable) 2024-04-27T03:59:59.049 INFO:teuthology.orchestra.run.smithi162.stdout:Extracting ceph user uid/gid from container image... 2024-04-27T04:00:00.071 INFO:teuthology.orchestra.run.smithi162.stdout:stat: stdout 167 167 2024-04-27T04:00:00.072 INFO:teuthology.orchestra.run.smithi162.stdout:Creating initial keys... 2024-04-27T04:00:01.304 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph-authtool: stdout AQBAeCxmnfgWFxAA22ZGKfF96bSZ9tyGZutoSw== 2024-04-27T04:00:02.241 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph-authtool: stdout AQBBeCxmVqaPJhAAqn4cjFxnL/eHTbmocleglA== 2024-04-27T04:00:03.297 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph-authtool: stdout AQBCeCxmPq+hIRAAxwG8hDikhm/quIZlOhv5FQ== 2024-04-27T04:00:03.298 INFO:teuthology.orchestra.run.smithi162.stdout:Creating initial monmap... 2024-04-27T04:00:04.153 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: monmap file /tmp/monmap 2024-04-27T04:00:04.154 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/monmaptool: stdout setting min_mon_release = octopus 2024-04-27T04:00:04.154 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: set fsid to 7f9750d6-044a-11ef-bc93-c7b262605968 2024-04-27T04:00:04.154 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-04-27T04:00:04.154 INFO:teuthology.orchestra.run.smithi162.stdout:monmaptool for smithi162 [v2:172.21.15.162:3300,v1:172.21.15.162:6789] on /usr/bin/monmaptool: monmap file /tmp/monmap 2024-04-27T04:00:04.154 INFO:teuthology.orchestra.run.smithi162.stdout:setting min_mon_release = octopus 2024-04-27T04:00:04.154 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/monmaptool: set fsid to 7f9750d6-044a-11ef-bc93-c7b262605968 2024-04-27T04:00:04.154 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-04-27T04:00:04.154 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-04-27T04:00:04.154 INFO:teuthology.orchestra.run.smithi162.stdout:Creating mon... 2024-04-27T04:00:05.001 INFO:teuthology.orchestra.run.smithi162.stdout:create mon.smithi162 on 2024-04-27T04:00:05.212 INFO:teuthology.orchestra.run.smithi162.stdout:systemctl: stderr Removed "/etc/systemd/system/multi-user.target.wants/ceph.target". 2024-04-27T04:00:05.351 INFO:teuthology.orchestra.run.smithi162.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /etc/systemd/system/ceph.target. 2024-04-27T04:00:05.519 INFO:teuthology.orchestra.run.smithi162.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph-7f9750d6-044a-11ef-bc93-c7b262605968.target → /etc/systemd/system/ceph-7f9750d6-044a-11ef-bc93-c7b262605968.target. 2024-04-27T04:00:05.520 INFO:teuthology.orchestra.run.smithi162.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph.target.wants/ceph-7f9750d6-044a-11ef-bc93-c7b262605968.target → /etc/systemd/system/ceph-7f9750d6-044a-11ef-bc93-c7b262605968.target. 2024-04-27T04:00:05.687 INFO:teuthology.orchestra.run.smithi162.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-7f9750d6-044a-11ef-bc93-c7b262605968@mon.smithi162 2024-04-27T04:00:05.687 INFO:teuthology.orchestra.run.smithi162.stdout:systemctl: stderr Failed to reset failed state of unit ceph-7f9750d6-044a-11ef-bc93-c7b262605968@mon.smithi162.service: Unit ceph-7f9750d6-044a-11ef-bc93-c7b262605968@mon.smithi162.service not loaded. 2024-04-27T04:00:05.843 INFO:teuthology.orchestra.run.smithi162.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-7f9750d6-044a-11ef-bc93-c7b262605968.target.wants/ceph-7f9750d6-044a-11ef-bc93-c7b262605968@mon.smithi162.service → /etc/systemd/system/ceph-7f9750d6-044a-11ef-bc93-c7b262605968@.service. 2024-04-27T04:00:06.418 INFO:teuthology.orchestra.run.smithi162.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-04-27T04:00:06.418 INFO:teuthology.orchestra.run.smithi162.stdout:systemctl: stdout disabled 2024-04-27T04:00:06.426 INFO:teuthology.orchestra.run.smithi162.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-04-27T04:00:06.426 INFO:teuthology.orchestra.run.smithi162.stdout:systemctl: stdout inactive 2024-04-27T04:00:06.426 INFO:teuthology.orchestra.run.smithi162.stdout:firewalld.service is not enabled 2024-04-27T04:00:06.426 INFO:teuthology.orchestra.run.smithi162.stdout:Not possible to enable service . firewalld.service is not available 2024-04-27T04:00:06.427 INFO:teuthology.orchestra.run.smithi162.stdout:Waiting for mon to start... 2024-04-27T04:00:06.427 INFO:teuthology.orchestra.run.smithi162.stdout:Waiting for mon... 2024-04-27T04:00:07.511 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stdout cluster: 2024-04-27T04:00:07.512 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stdout id: 7f9750d6-044a-11ef-bc93-c7b262605968 2024-04-27T04:00:07.512 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stdout health: HEALTH_OK 2024-04-27T04:00:07.512 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stdout 2024-04-27T04:00:07.512 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stdout services: 2024-04-27T04:00:07.512 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stdout mon: 1 daemons, quorum smithi162 (age 0.610577s) 2024-04-27T04:00:07.512 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stdout mgr: no daemons active 2024-04-27T04:00:07.512 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stdout osd: 0 osds: 0 up, 0 in 2024-04-27T04:00:07.512 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stdout 2024-04-27T04:00:07.512 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stdout data: 2024-04-27T04:00:07.512 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stdout pools: 0 pools, 0 pgs 2024-04-27T04:00:07.512 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stdout objects: 0 objects, 0 B 2024-04-27T04:00:07.512 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stdout usage: 0 B used, 0 B / 0 B avail 2024-04-27T04:00:07.512 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stdout pgs: 2024-04-27T04:00:07.513 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stdout 2024-04-27T04:00:07.513 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:06.872+0000 7f1b3dfc8700 1 Processor -- start 2024-04-27T04:00:07.513 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:06.872+0000 7f1b3dfc8700 1 -- start start 2024-04-27T04:00:07.513 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:06.872+0000 7f1b3dfc8700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1b38076bf0 0x7f1b38105490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:00:07.513 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:06.872+0000 7f1b3dfc8700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b381059d0 con 0x7f1b38076bf0 2024-04-27T04:00:07.513 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:06.873+0000 7f1b377fe700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1b38076bf0 0x7f1b38105490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:00:07.513 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:06.873+0000 7f1b377fe700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1b38076bf0 0x7f1b38105490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:47608/0 (socket says 172.21.15.162:47608) 2024-04-27T04:00:07.513 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:06.873+0000 7f1b377fe700 1 -- 172.21.15.162:0/4196029472 learned_addr learned my addr 172.21.15.162:0/4196029472 (peer_addr_for_me v2:172.21.15.162:0/0) 2024-04-27T04:00:07.513 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:06.873+0000 7f1b377fe700 1 -- 172.21.15.162:0/4196029472 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b38105b10 con 0x7f1b38076bf0 2024-04-27T04:00:07.513 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:06.873+0000 7f1b377fe700 1 --2- 172.21.15.162:0/4196029472 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1b38076bf0 0x7f1b38105490 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f1b2800fa00 tx=0x7f1b28009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=99963e33c745001e server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:00:07.513 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:06.873+0000 7f1b367fc700 1 -- 172.21.15.162:0/4196029472 <== mon.0 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f1b28009cb0 con 0x7f1b38076bf0 2024-04-27T04:00:07.513 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:06.873+0000 7f1b367fc700 1 -- 172.21.15.162:0/4196029472 <== mon.0 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1b28009e10 con 0x7f1b38076bf0 2024-04-27T04:00:07.513 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:06.873+0000 7f1b367fc700 1 -- 172.21.15.162:0/4196029472 <== mon.0 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f1b280175d0 con 0x7f1b38076bf0 2024-04-27T04:00:07.513 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:06.874+0000 7f1b3dfc8700 1 -- 172.21.15.162:0/4196029472 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1b38076bf0 msgr2=0x7f1b38105490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:00:07.513 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:06.874+0000 7f1b3dfc8700 1 --2- 172.21.15.162:0/4196029472 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1b38076bf0 0x7f1b38105490 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f1b2800fa00 tx=0x7f1b28009250 comp rx=0 tx=0).stop 2024-04-27T04:00:07.514 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:06.874+0000 7f1b3dfc8700 1 -- 172.21.15.162:0/4196029472 shutdown_connections 2024-04-27T04:00:07.514 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:06.874+0000 7f1b3dfc8700 1 --2- 172.21.15.162:0/4196029472 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1b38076bf0 0x7f1b38105490 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:00:07.514 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:06.874+0000 7f1b3dfc8700 1 -- 172.21.15.162:0/4196029472 >> 172.21.15.162:0/4196029472 conn(0x7f1b380fcf00 msgr2=0x7f1b380ff310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:00:07.514 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:06.874+0000 7f1b3dfc8700 1 -- 172.21.15.162:0/4196029472 shutdown_connections 2024-04-27T04:00:07.514 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:06.874+0000 7f1b3dfc8700 1 -- 172.21.15.162:0/4196029472 wait complete. 2024-04-27T04:00:07.514 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:06.875+0000 7f1b3dfc8700 1 Processor -- start 2024-04-27T04:00:07.514 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:06.875+0000 7f1b3dfc8700 1 -- start start 2024-04-27T04:00:07.514 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:06.875+0000 7f1b3dfc8700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1b38076bf0 0x7f1b3810ec60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:00:07.514 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:06.875+0000 7f1b3dfc8700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b3810f1a0 con 0x7f1b38076bf0 2024-04-27T04:00:07.514 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:06.875+0000 7f1b377fe700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1b38076bf0 0x7f1b3810ec60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:00:07.514 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:06.876+0000 7f1b377fe700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1b38076bf0 0x7f1b3810ec60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:47618/0 (socket says 172.21.15.162:47618) 2024-04-27T04:00:07.514 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:06.876+0000 7f1b377fe700 1 -- 172.21.15.162:0/87467019 learned_addr learned my addr 172.21.15.162:0/87467019 (peer_addr_for_me v2:172.21.15.162:0/0) 2024-04-27T04:00:07.514 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:06.876+0000 7f1b377fe700 1 -- 172.21.15.162:0/87467019 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b2800f610 con 0x7f1b38076bf0 2024-04-27T04:00:07.514 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:06.876+0000 7f1b377fe700 1 --2- 172.21.15.162:0/87467019 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1b38076bf0 0x7f1b3810ec60 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f1b28000c00 tx=0x7f1b28017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:00:07.515 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:06.876+0000 7f1b1ffff700 1 -- 172.21.15.162:0/87467019 <== mon.0 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f1b28021780 con 0x7f1b38076bf0 2024-04-27T04:00:07.515 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:06.876+0000 7f1b1ffff700 1 -- 172.21.15.162:0/87467019 <== mon.0 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1b280218e0 con 0x7f1b38076bf0 2024-04-27T04:00:07.515 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:06.876+0000 7f1b3dfc8700 1 -- 172.21.15.162:0/87467019 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1b3810f3a0 con 0x7f1b38076bf0 2024-04-27T04:00:07.515 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:06.876+0000 7f1b1ffff700 1 -- 172.21.15.162:0/87467019 <== mon.0 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f1b28021bb0 con 0x7f1b38076bf0 2024-04-27T04:00:07.515 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:06.876+0000 7f1b3dfc8700 1 -- 172.21.15.162:0/87467019 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1b3810f7d0 con 0x7f1b38076bf0 2024-04-27T04:00:07.515 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:06.877+0000 7f1b1ffff700 1 -- 172.21.15.162:0/87467019 <== mon.0 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 1) v1 ==== 529+0+0 (secure 0 0 0) 0x7f1b2801e050 con 0x7f1b38076bf0 2024-04-27T04:00:07.515 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:06.877+0000 7f1b1ffff700 1 -- 172.21.15.162:0/87467019 <== mon.0 v2:172.21.15.162:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 721+0+0 (secure 0 0 0) 0x7f1b280163b0 con 0x7f1b38076bf0 2024-04-27T04:00:07.515 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:06.877+0000 7f1b1dffb700 1 -- 172.21.15.162:0/87467019 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1b180052b0 con 0x7f1b38076bf0 2024-04-27T04:00:07.515 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:06.880+0000 7f1b1ffff700 1 -- 172.21.15.162:0/87467019 <== mon.0 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71547 (secure 0 0 0) 0x7f1b2801c070 con 0x7f1b38076bf0 2024-04-27T04:00:07.515 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:06.952+0000 7f1b1dffb700 1 -- 172.21.15.162:0/87467019 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "status"} v 0) v1 -- 0x7f1b18005830 con 0x7f1b38076bf0 2024-04-27T04:00:07.515 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:06.953+0000 7f1b1ffff700 1 -- 172.21.15.162:0/87467019 <== mon.0 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "status"}]=0 v0) v1 ==== 54+0+325 (secure 0 0 0) 0x7f1b280294f0 con 0x7f1b38076bf0 2024-04-27T04:00:07.515 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:06.954+0000 7f1b1dffb700 1 -- 172.21.15.162:0/87467019 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1b38076bf0 msgr2=0x7f1b3810ec60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:00:07.515 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:06.954+0000 7f1b1dffb700 1 --2- 172.21.15.162:0/87467019 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1b38076bf0 0x7f1b3810ec60 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f1b28000c00 tx=0x7f1b28017000 comp rx=0 tx=0).stop 2024-04-27T04:00:07.515 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:06.954+0000 7f1b1dffb700 1 -- 172.21.15.162:0/87467019 shutdown_connections 2024-04-27T04:00:07.515 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:06.954+0000 7f1b1dffb700 1 --2- 172.21.15.162:0/87467019 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1b38076bf0 0x7f1b3810ec60 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:00:07.516 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:06.954+0000 7f1b1dffb700 1 -- 172.21.15.162:0/87467019 >> 172.21.15.162:0/87467019 conn(0x7f1b380fcf00 msgr2=0x7f1b38105ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:00:07.516 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:06.954+0000 7f1b1dffb700 1 -- 172.21.15.162:0/87467019 shutdown_connections 2024-04-27T04:00:07.516 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:06.954+0000 7f1b1dffb700 1 -- 172.21.15.162:0/87467019 wait complete. 2024-04-27T04:00:07.516 INFO:teuthology.orchestra.run.smithi162.stdout:mon is available 2024-04-27T04:00:07.516 INFO:teuthology.orchestra.run.smithi162.stdout:Assimilating anything we can from ceph.conf... 2024-04-27T04:00:08.598 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stdout 2024-04-27T04:00:08.598 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stdout [global] 2024-04-27T04:00:08.598 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stdout fsid = 7f9750d6-044a-11ef-bc93-c7b262605968 2024-04-27T04:00:08.598 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stdout mon_host = [v2:172.21.15.162:3300,v1:172.21.15.162:6789] 2024-04-27T04:00:08.598 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-04-27T04:00:08.598 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-04-27T04:00:08.598 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-04-27T04:00:08.598 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-04-27T04:00:08.598 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stdout 2024-04-27T04:00:08.598 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stdout [mgr] 2024-04-27T04:00:08.598 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-04-27T04:00:08.599 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stdout 2024-04-27T04:00:08.599 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stdout [osd] 2024-04-27T04:00:08.599 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-04-27T04:00:08.599 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-04-27T04:00:08.599 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-04-27T04:00:08.599 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-04-27T04:00:08.599 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-04-27T04:00:08.599 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:07.913+0000 7f339ab73700 1 Processor -- start 2024-04-27T04:00:08.599 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:07.913+0000 7f339ab73700 1 -- start start 2024-04-27T04:00:08.599 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:07.913+0000 7f339ab73700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3394102850 0x7f3394102c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:00:08.599 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:07.913+0000 7f339ab73700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33941031a0 con 0x7f3394102850 2024-04-27T04:00:08.599 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:07.914+0000 7f339890f700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3394102850 0x7f3394102c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:00:08.599 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:07.914+0000 7f339890f700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3394102850 0x7f3394102c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:47630/0 (socket says 172.21.15.162:47630) 2024-04-27T04:00:08.599 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:07.914+0000 7f339890f700 1 -- 172.21.15.162:0/1924207297 learned_addr learned my addr 172.21.15.162:0/1924207297 (peer_addr_for_me v2:172.21.15.162:0/0) 2024-04-27T04:00:08.600 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:07.914+0000 7f339890f700 1 -- 172.21.15.162:0/1924207297 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f33941032e0 con 0x7f3394102850 2024-04-27T04:00:08.600 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:07.914+0000 7f339890f700 1 --2- 172.21.15.162:0/1924207297 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3394102850 0x7f3394102c60 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f338c009e50 tx=0x7f338c009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=877502f3085789bb server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:00:08.600 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:07.914+0000 7f33937fe700 1 -- 172.21.15.162:0/1924207297 <== mon.0 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f338c0092e0 con 0x7f3394102850 2024-04-27T04:00:08.600 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:07.914+0000 7f33937fe700 1 -- 172.21.15.162:0/1924207297 <== mon.0 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f338c00ba20 con 0x7f3394102850 2024-04-27T04:00:08.600 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:07.914+0000 7f33937fe700 1 -- 172.21.15.162:0/1924207297 <== mon.0 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f338c00bd30 con 0x7f3394102850 2024-04-27T04:00:08.600 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:07.915+0000 7f339ab73700 1 -- 172.21.15.162:0/1924207297 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3394102850 msgr2=0x7f3394102c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:00:08.600 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:07.915+0000 7f339ab73700 1 --2- 172.21.15.162:0/1924207297 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3394102850 0x7f3394102c60 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f338c009e50 tx=0x7f338c009840 comp rx=0 tx=0).stop 2024-04-27T04:00:08.600 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:07.915+0000 7f339ab73700 1 -- 172.21.15.162:0/1924207297 shutdown_connections 2024-04-27T04:00:08.600 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:07.915+0000 7f339ab73700 1 --2- 172.21.15.162:0/1924207297 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3394102850 0x7f3394102c60 unknown :-1 s=CLOSED pgs=3 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:00:08.600 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:07.915+0000 7f339ab73700 1 -- 172.21.15.162:0/1924207297 >> 172.21.15.162:0/1924207297 conn(0x7f33940fdff0 msgr2=0x7f3394100400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:00:08.600 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:07.915+0000 7f339ab73700 1 -- 172.21.15.162:0/1924207297 shutdown_connections 2024-04-27T04:00:08.600 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:07.915+0000 7f339ab73700 1 -- 172.21.15.162:0/1924207297 wait complete. 2024-04-27T04:00:08.600 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:07.915+0000 7f339ab73700 1 Processor -- start 2024-04-27T04:00:08.600 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:07.915+0000 7f339ab73700 1 -- start start 2024-04-27T04:00:08.600 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:07.916+0000 7f339ab73700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3394102850 0x7f3394077b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:00:08.601 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:07.916+0000 7f339ab73700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3394078050 con 0x7f3394102850 2024-04-27T04:00:08.601 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:07.916+0000 7f339890f700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3394102850 0x7f3394077b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:00:08.601 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:07.916+0000 7f339890f700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3394102850 0x7f3394077b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:47642/0 (socket says 172.21.15.162:47642) 2024-04-27T04:00:08.601 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:07.916+0000 7f339890f700 1 -- 172.21.15.162:0/2284204855 learned_addr learned my addr 172.21.15.162:0/2284204855 (peer_addr_for_me v2:172.21.15.162:0/0) 2024-04-27T04:00:08.601 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:07.916+0000 7f339890f700 1 -- 172.21.15.162:0/2284204855 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f338c009ae0 con 0x7f3394102850 2024-04-27T04:00:08.601 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:07.916+0000 7f339890f700 1 --2- 172.21.15.162:0/2284204855 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3394102850 0x7f3394077b10 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7f3394103590 tx=0x7f338c017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:00:08.601 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:07.916+0000 7f33917fa700 1 -- 172.21.15.162:0/2284204855 <== mon.0 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f338c012b90 con 0x7f3394102850 2024-04-27T04:00:08.601 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:07.916+0000 7f33917fa700 1 -- 172.21.15.162:0/2284204855 <== mon.0 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f338c012d10 con 0x7f3394102850 2024-04-27T04:00:08.601 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:07.916+0000 7f339ab73700 1 -- 172.21.15.162:0/2284204855 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3394076120 con 0x7f3394102850 2024-04-27T04:00:08.601 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:07.916+0000 7f33917fa700 1 -- 172.21.15.162:0/2284204855 <== mon.0 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f338c01e400 con 0x7f3394102850 2024-04-27T04:00:08.601 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:07.916+0000 7f339ab73700 1 -- 172.21.15.162:0/2284204855 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3394076610 con 0x7f3394102850 2024-04-27T04:00:08.601 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:07.917+0000 7f33917fa700 1 -- 172.21.15.162:0/2284204855 <== mon.0 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 1) v1 ==== 529+0+0 (secure 0 0 0) 0x7f338c019030 con 0x7f3394102850 2024-04-27T04:00:08.601 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:07.917+0000 7f33917fa700 1 -- 172.21.15.162:0/2284204855 <== mon.0 v2:172.21.15.162:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 721+0+0 (secure 0 0 0) 0x7f338c026400 con 0x7f3394102850 2024-04-27T04:00:08.602 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:07.917+0000 7f339ab73700 1 -- 172.21.15.162:0/2284204855 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f339404bed0 con 0x7f3394102850 2024-04-27T04:00:08.602 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:07.919+0000 7f33917fa700 1 -- 172.21.15.162:0/2284204855 <== mon.0 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71547 (secure 0 0 0) 0x7f338c017070 con 0x7f3394102850 2024-04-27T04:00:08.602 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:07.985+0000 7f339ab73700 1 -- 172.21.15.162:0/2284204855 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) v1 -- 0x7f3394103b70 con 0x7f3394102850 2024-04-27T04:00:08.602 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:07.989+0000 7f33917fa700 1 -- 172.21.15.162:0/2284204855 <== mon.0 v2:172.21.15.162:3300/0 7 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7f338c038400 con 0x7f3394102850 2024-04-27T04:00:08.602 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:07.989+0000 7f33917fa700 1 -- 172.21.15.162:0/2284204855 <== mon.0 v2:172.21.15.162:3300/0 8 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v2) v1 ==== 70+0+431 (secure 0 0 0) 0x7f338c026610 con 0x7f3394102850 2024-04-27T04:00:08.602 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:07.990+0000 7f339ab73700 1 -- 172.21.15.162:0/2284204855 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3394102850 msgr2=0x7f3394077b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:00:08.602 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:07.990+0000 7f339ab73700 1 --2- 172.21.15.162:0/2284204855 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3394102850 0x7f3394077b10 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7f3394103590 tx=0x7f338c017000 comp rx=0 tx=0).stop 2024-04-27T04:00:08.602 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:07.990+0000 7f339ab73700 1 -- 172.21.15.162:0/2284204855 shutdown_connections 2024-04-27T04:00:08.602 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:07.990+0000 7f339ab73700 1 --2- 172.21.15.162:0/2284204855 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3394102850 0x7f3394077b10 unknown :-1 s=CLOSED pgs=4 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:00:08.602 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:07.990+0000 7f339ab73700 1 -- 172.21.15.162:0/2284204855 >> 172.21.15.162:0/2284204855 conn(0x7f33940fdff0 msgr2=0x7f33941052d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:00:08.602 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:07.990+0000 7f339ab73700 1 -- 172.21.15.162:0/2284204855 shutdown_connections 2024-04-27T04:00:08.602 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:07.990+0000 7f339ab73700 1 -- 172.21.15.162:0/2284204855 wait complete. 2024-04-27T04:00:08.602 INFO:teuthology.orchestra.run.smithi162.stdout:Generating new minimal ceph.conf... 2024-04-27T04:00:09.622 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:09.010+0000 7fb863dbb700 1 Processor -- start 2024-04-27T04:00:09.622 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:09.010+0000 7fb863dbb700 1 -- start start 2024-04-27T04:00:09.622 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:09.010+0000 7fb863dbb700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb85c076bf0 0x7fb85c105460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:00:09.622 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:09.010+0000 7fb863dbb700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb85c1059a0 con 0x7fb85c076bf0 2024-04-27T04:00:09.622 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:09.010+0000 7fb861b57700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb85c076bf0 0x7fb85c105460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:00:09.622 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:09.010+0000 7fb861b57700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb85c076bf0 0x7fb85c105460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:47650/0 (socket says 172.21.15.162:47650) 2024-04-27T04:00:09.622 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:09.010+0000 7fb861b57700 1 -- 172.21.15.162:0/4066793801 learned_addr learned my addr 172.21.15.162:0/4066793801 (peer_addr_for_me v2:172.21.15.162:0/0) 2024-04-27T04:00:09.622 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:09.010+0000 7fb861b57700 1 -- 172.21.15.162:0/4066793801 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb85c105ae0 con 0x7fb85c076bf0 2024-04-27T04:00:09.622 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:09.011+0000 7fb861b57700 1 --2- 172.21.15.162:0/4066793801 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb85c076bf0 0x7fb85c105460 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7fb8580098e0 tx=0x7fb858009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bb2021da889f65cb server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:00:09.622 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:09.011+0000 7fb860b55700 1 -- 172.21.15.162:0/4066793801 <== mon.0 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fb85800b860 con 0x7fb85c076bf0 2024-04-27T04:00:09.623 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:09.011+0000 7fb860b55700 1 -- 172.21.15.162:0/4066793801 <== mon.0 v2:172.21.15.162:3300/0 2 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7fb85800be50 con 0x7fb85c076bf0 2024-04-27T04:00:09.623 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:09.011+0000 7fb860b55700 1 -- 172.21.15.162:0/4066793801 <== mon.0 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fb858010380 con 0x7fb85c076bf0 2024-04-27T04:00:09.623 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:09.011+0000 7fb863dbb700 1 -- 172.21.15.162:0/4066793801 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb85c076bf0 msgr2=0x7fb85c105460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:00:09.623 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:09.011+0000 7fb863dbb700 1 --2- 172.21.15.162:0/4066793801 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb85c076bf0 0x7fb85c105460 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7fb8580098e0 tx=0x7fb858009250 comp rx=0 tx=0).stop 2024-04-27T04:00:09.623 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:09.011+0000 7fb863dbb700 1 -- 172.21.15.162:0/4066793801 shutdown_connections 2024-04-27T04:00:09.623 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:09.011+0000 7fb863dbb700 1 --2- 172.21.15.162:0/4066793801 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb85c076bf0 0x7fb85c105460 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:00:09.623 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:09.011+0000 7fb863dbb700 1 -- 172.21.15.162:0/4066793801 >> 172.21.15.162:0/4066793801 conn(0x7fb85c0fcef0 msgr2=0x7fb85c0ff300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:00:09.623 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:09.011+0000 7fb863dbb700 1 -- 172.21.15.162:0/4066793801 shutdown_connections 2024-04-27T04:00:09.623 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:09.011+0000 7fb863dbb700 1 -- 172.21.15.162:0/4066793801 wait complete. 2024-04-27T04:00:09.623 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:09.012+0000 7fb863dbb700 1 Processor -- start 2024-04-27T04:00:09.623 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:09.012+0000 7fb863dbb700 1 -- start start 2024-04-27T04:00:09.623 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:09.012+0000 7fb863dbb700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb85c076bf0 0x7fb85c111600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:00:09.623 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:09.012+0000 7fb863dbb700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb85c111b40 con 0x7fb85c076bf0 2024-04-27T04:00:09.623 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:09.012+0000 7fb861b57700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb85c076bf0 0x7fb85c111600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:00:09.623 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:09.012+0000 7fb861b57700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb85c076bf0 0x7fb85c111600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:47652/0 (socket says 172.21.15.162:47652) 2024-04-27T04:00:09.624 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:09.012+0000 7fb861b57700 1 -- 172.21.15.162:0/82791669 learned_addr learned my addr 172.21.15.162:0/82791669 (peer_addr_for_me v2:172.21.15.162:0/0) 2024-04-27T04:00:09.624 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:09.012+0000 7fb861b57700 1 -- 172.21.15.162:0/82791669 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb8580094f0 con 0x7fb85c076bf0 2024-04-27T04:00:09.624 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:09.013+0000 7fb861b57700 1 --2- 172.21.15.162:0/82791669 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb85c076bf0 0x7fb85c111600 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fb858003050 tx=0x7fb858009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:00:09.624 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:09.013+0000 7fb8527fc700 1 -- 172.21.15.162:0/82791669 <== mon.0 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fb858010760 con 0x7fb85c076bf0 2024-04-27T04:00:09.624 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:09.013+0000 7fb8527fc700 1 -- 172.21.15.162:0/82791669 <== mon.0 v2:172.21.15.162:3300/0 2 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7fb858010d50 con 0x7fb85c076bf0 2024-04-27T04:00:09.624 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:09.013+0000 7fb863dbb700 1 -- 172.21.15.162:0/82791669 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb85c111d40 con 0x7fb85c076bf0 2024-04-27T04:00:09.624 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:09.013+0000 7fb8527fc700 1 -- 172.21.15.162:0/82791669 <== mon.0 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fb85801ae20 con 0x7fb85c076bf0 2024-04-27T04:00:09.624 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:09.013+0000 7fb863dbb700 1 -- 172.21.15.162:0/82791669 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb85c1128b0 con 0x7fb85c076bf0 2024-04-27T04:00:09.624 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:09.013+0000 7fb8527fc700 1 -- 172.21.15.162:0/82791669 <== mon.0 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 1) v1 ==== 529+0+0 (secure 0 0 0) 0x7fb85802e440 con 0x7fb85c076bf0 2024-04-27T04:00:09.624 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:09.013+0000 7fb8527fc700 1 -- 172.21.15.162:0/82791669 <== mon.0 v2:172.21.15.162:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 721+0+0 (secure 0 0 0) 0x7fb858024740 con 0x7fb85c076bf0 2024-04-27T04:00:09.624 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:09.014+0000 7fb863dbb700 1 -- 172.21.15.162:0/82791669 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb85c05f5c0 con 0x7fb85c076bf0 2024-04-27T04:00:09.624 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:09.016+0000 7fb8527fc700 1 -- 172.21.15.162:0/82791669 <== mon.0 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71547 (secure 0 0 0) 0x7fb858029070 con 0x7fb85c076bf0 2024-04-27T04:00:09.624 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:09.082+0000 7fb863dbb700 1 -- 172.21.15.162:0/82791669 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) v1 -- 0x7fb85c106860 con 0x7fb85c076bf0 2024-04-27T04:00:09.624 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:09.082+0000 7fb8527fc700 1 -- 172.21.15.162:0/82791669 <== mon.0 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v2) v1 ==== 76+0+177 (secure 0 0 0) 0x7fb858006d70 con 0x7fb85c076bf0 2024-04-27T04:00:09.624 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:09.083+0000 7fb863dbb700 1 -- 172.21.15.162:0/82791669 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb85c076bf0 msgr2=0x7fb85c111600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:00:09.625 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:09.083+0000 7fb863dbb700 1 --2- 172.21.15.162:0/82791669 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb85c076bf0 0x7fb85c111600 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fb858003050 tx=0x7fb858009250 comp rx=0 tx=0).stop 2024-04-27T04:00:09.625 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:09.083+0000 7fb863dbb700 1 -- 172.21.15.162:0/82791669 shutdown_connections 2024-04-27T04:00:09.625 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:09.083+0000 7fb863dbb700 1 --2- 172.21.15.162:0/82791669 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb85c076bf0 0x7fb85c111600 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:00:09.625 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:09.083+0000 7fb863dbb700 1 -- 172.21.15.162:0/82791669 >> 172.21.15.162:0/82791669 conn(0x7fb85c0fcef0 msgr2=0x7fb85c0fd9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:00:09.625 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:09.083+0000 7fb863dbb700 1 -- 172.21.15.162:0/82791669 shutdown_connections 2024-04-27T04:00:09.625 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:09.083+0000 7fb863dbb700 1 -- 172.21.15.162:0/82791669 wait complete. 2024-04-27T04:00:09.625 INFO:teuthology.orchestra.run.smithi162.stdout:Restarting the monitor... 2024-04-27T04:00:09.870 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:00:09 smithi162 systemd[1]: Stopping Ceph mon.smithi162 for 7f9750d6-044a-11ef-bc93-c7b262605968... 2024-04-27T04:00:10.139 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:00:09 smithi162 ceph-7f9750d6-044a-11ef-bc93-c7b262605968-mon-smithi162[26452]: 2024-04-27T04:00:09.868+0000 7fc27262e700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.smithi162 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-04-27T04:00:10.139 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:00:09 smithi162 ceph-7f9750d6-044a-11ef-bc93-c7b262605968-mon-smithi162[26452]: 2024-04-27T04:00:09.868+0000 7fc27262e700 -1 mon.smithi162@0(leader) e1 *** Got Signal Terminated *** 2024-04-27T04:00:10.140 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:00:10 smithi162 podman[26713]: 2024-04-27 04:00:10.032696298 +0000 UTC m=+0.257851866 container died 8ba6ac56fe9afe4313a5f516d29aeac5274523635d67e5b649303550c7ac32dc (image=quay.ceph.io/ceph-ci/ceph:quincy, name=ceph-7f9750d6-044a-11ef-bc93-c7b262605968-mon-smithi162, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, ceph=True, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.build-date=20240424, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, RELEASE=quincy-2be3efa, org.label-schema.license=GPLv2) 2024-04-27T04:00:10.639 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:00:10 smithi162 podman[26713]: 2024-04-27 04:00:10.402107235 +0000 UTC m=+0.627262846 container cleanup 8ba6ac56fe9afe4313a5f516d29aeac5274523635d67e5b649303550c7ac32dc (image=quay.ceph.io/ceph-ci/ceph:quincy, name=ceph-7f9750d6-044a-11ef-bc93-c7b262605968-mon-smithi162, org.label-schema.vendor=CentOS, ceph=True, io.buildah.version=1.33.5, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, RELEASE=quincy-2be3efa, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240424, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-27T04:00:10.639 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:00:10 smithi162 bash[26713]: ceph-7f9750d6-044a-11ef-bc93-c7b262605968-mon-smithi162 2024-04-27T04:00:10.979 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:00:10 smithi162 podman[26727]: 2024-04-27 04:00:10.652859958 +0000 UTC m=+0.613099094 container remove 8ba6ac56fe9afe4313a5f516d29aeac5274523635d67e5b649303550c7ac32dc (image=quay.ceph.io/ceph-ci/ceph:quincy, name=ceph-7f9750d6-044a-11ef-bc93-c7b262605968-mon-smithi162, GIT_CLEAN=True, org.label-schema.build-date=20240424, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , RELEASE=quincy-2be3efa, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2) 2024-04-27T04:00:10.979 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:00:10 smithi162 systemd[1]: ceph-7f9750d6-044a-11ef-bc93-c7b262605968@mon.smithi162.service: Deactivated successfully. 2024-04-27T04:00:10.979 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:00:10 smithi162 systemd[1]: Stopped Ceph mon.smithi162 for 7f9750d6-044a-11ef-bc93-c7b262605968. 2024-04-27T04:00:10.979 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:00:10 smithi162 systemd[1]: Starting Ceph mon.smithi162 for 7f9750d6-044a-11ef-bc93-c7b262605968... 2024-04-27T04:00:11.265 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:00:10 smithi162 podman[26822]: 2024-04-27 04:00:10.978827357 +0000 UTC m=+0.103246075 container create 0f1e3682b9abe73f7036e1738bc1dc3ccb64fdc5377b198e5b3f4cd759ea5a3c (image=quay.ceph.io/ceph-ci/ceph:quincy, name=ceph-7f9750d6-044a-11ef-bc93-c7b262605968-mon-smithi162, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, maintainer=Guillaume Abrioux , RELEASE=quincy-2be3efa, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, ceph=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240424, org.label-schema.schema-version=1.0, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, CEPH_POINT_RELEASE=) 2024-04-27T04:00:11.265 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:00:10 smithi162 podman[26822]: 2024-04-27 04:00:10.893217408 +0000 UTC m=+0.017636095 image pull 1c583463371507dd553af71bf60c9ae8747aff79bb666e4e29ef4e17d0bc3a13 quay.ceph.io/ceph-ci/ceph:quincy 2024-04-27T04:00:11.265 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:00:11 smithi162 podman[26822]: 2024-04-27 04:00:11.189376748 +0000 UTC m=+0.313795420 container init 0f1e3682b9abe73f7036e1738bc1dc3ccb64fdc5377b198e5b3f4cd759ea5a3c (image=quay.ceph.io/ceph-ci/ceph:quincy, name=ceph-7f9750d6-044a-11ef-bc93-c7b262605968-mon-smithi162, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, ceph=True, RELEASE=quincy-2be3efa, org.label-schema.build-date=20240424, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_CLEAN=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-27T04:00:11.265 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:00:11 smithi162 podman[26822]: 2024-04-27 04:00:11.192222103 +0000 UTC m=+0.316640776 container start 0f1e3682b9abe73f7036e1738bc1dc3ccb64fdc5377b198e5b3f4cd759ea5a3c (image=quay.ceph.io/ceph-ci/ceph:quincy, name=ceph-7f9750d6-044a-11ef-bc93-c7b262605968-mon-smithi162, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.build-date=20240424, RELEASE=quincy-2be3efa, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, ceph=True, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=) 2024-04-27T04:00:11.265 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:00:11 smithi162 ceph-mon[26841]: set uid:gid to 167:167 (ceph:ceph) 2024-04-27T04:00:11.265 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:00:11 smithi162 ceph-mon[26841]: ceph version 17.2.7-907-g2be3efa9 (2be3efa97a7972f8e925bd899b474329979b74d6) quincy (stable), process ceph-mon, pid 2 2024-04-27T04:00:11.266 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:00:11 smithi162 ceph-mon[26841]: pidfile_write: ignore empty --pid-file 2024-04-27T04:00:11.266 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:00:11 smithi162 ceph-mon[26841]: load: jerasure load: lrc 2024-04-27T04:00:11.266 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:00:11 smithi162 ceph-mon[26841]: rocksdb: RocksDB version: 6.15.5 2024-04-27T04:00:11.266 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:00:11 smithi162 ceph-mon[26841]: rocksdb: Git sha rocksdb_build_git_sha:@0@ 2024-04-27T04:00:11.266 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:00:11 smithi162 ceph-mon[26841]: rocksdb: Compile date Apr 26 2024 2024-04-27T04:00:11.266 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:00:11 smithi162 ceph-mon[26841]: rocksdb: DB SUMMARY 2024-04-27T04:00:11.266 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:00:11 smithi162 ceph-mon[26841]: rocksdb: DB Session ID: NWA5R4JT27PBG8OGKAC7 2024-04-27T04:00:11.266 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:00:11 smithi162 ceph-mon[26841]: rocksdb: CURRENT file: CURRENT 2024-04-27T04:00:11.266 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:00:11 smithi162 ceph-mon[26841]: rocksdb: IDENTITY file: IDENTITY 2024-04-27T04:00:11.266 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:00:11 smithi162 ceph-mon[26841]: rocksdb: MANIFEST file: MANIFEST-000009 size: 131 Bytes 2024-04-27T04:00:11.266 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:00:11 smithi162 ceph-mon[26841]: rocksdb: SST files in /var/lib/ceph/mon/ceph-smithi162/store.db dir, Total Num: 1, files: 000008.sst 2024-04-27T04:00:11.266 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:00:11 smithi162 ceph-mon[26841]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-smithi162/store.db: 000010.log size: 98176 ; 2024-04-27T04:00:11.266 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:00:11 smithi162 ceph-mon[26841]: rocksdb: Options.error_if_exists: 0 2024-04-27T04:00:11.267 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:00:11 smithi162 ceph-mon[26841]: rocksdb: Options.create_if_missing: 0 2024-04-27T04:00:11.267 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:00:11 smithi162 ceph-mon[26841]: rocksdb: Options.paranoid_checks: 1 2024-04-27T04:00:11.267 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:00:11 smithi162 ceph-mon[26841]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-04-27T04:00:11.267 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:00:11 smithi162 ceph-mon[26841]: rocksdb: Options.env: 0x564742161b00 2024-04-27T04:00:11.267 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:00:11 smithi162 ceph-mon[26841]: rocksdb: Options.fs: Posix File System 2024-04-27T04:00:11.267 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:00:11 smithi162 ceph-mon[26841]: rocksdb: Options.info_log: 0x56474427b620 2024-04-27T04:00:11.267 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:00:11 smithi162 ceph-mon[26841]: rocksdb: Options.max_file_opening_threads: 16 2024-04-27T04:00:11.267 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:00:11 smithi162 ceph-mon[26841]: rocksdb: Options.statistics: (nil) 2024-04-27T04:00:11.267 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:00:11 smithi162 ceph-mon[26841]: rocksdb: Options.use_fsync: 0 2024-04-27T04:00:11.267 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:00:11 smithi162 ceph-mon[26841]: rocksdb: Options.max_log_file_size: 0 2024-04-27T04:00:11.267 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:00:11 smithi162 ceph-mon[26841]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-04-27T04:00:11.267 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:00:11 smithi162 ceph-mon[26841]: rocksdb: Options.log_file_time_to_roll: 0 2024-04-27T04:00:11.267 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:00:11 smithi162 ceph-mon[26841]: rocksdb: Options.keep_log_file_num: 1000 2024-04-27T04:00:11.268 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:00:11 smithi162 ceph-mon[26841]: rocksdb: Options.recycle_log_file_num: 0 2024-04-27T04:00:11.268 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:00:11 smithi162 ceph-mon[26841]: rocksdb: Options.allow_fallocate: 1 2024-04-27T04:00:11.268 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:00:11 smithi162 ceph-mon[26841]: rocksdb: Options.allow_mmap_reads: 0 2024-04-27T04:00:11.268 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:00:11 smithi162 ceph-mon[26841]: rocksdb: Options.allow_mmap_writes: 0 2024-04-27T04:00:11.268 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:00:11 smithi162 ceph-mon[26841]: rocksdb: Options.use_direct_reads: 0 2024-04-27T04:00:11.268 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:00:11 smithi162 ceph-mon[26841]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-04-27T04:00:11.268 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:00:11 smithi162 ceph-mon[26841]: rocksdb: Options.create_missing_column_families: 0 2024-04-27T04:00:11.268 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:00:11 smithi162 ceph-mon[26841]: rocksdb: Options.db_log_dir: 2024-04-27T04:00:11.268 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:00:11 smithi162 ceph-mon[26841]: rocksdb: Options.wal_dir: /var/lib/ceph/mon/ceph-smithi162/store.db 2024-04-27T04:00:11.268 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:00:11 smithi162 ceph-mon[26841]: rocksdb: Options.table_cache_numshardbits: 6 2024-04-27T04:00:11.268 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:00:11 smithi162 ceph-mon[26841]: rocksdb: Options.WAL_ttl_seconds: 0 2024-04-27T04:00:11.268 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:00:11 smithi162 ceph-mon[26841]: rocksdb: Options.WAL_size_limit_MB: 0 2024-04-27T04:00:11.268 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:00:11 smithi162 ceph-mon[26841]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-04-27T04:00:11.268 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:00:11 smithi162 ceph-mon[26841]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-04-27T04:00:11.268 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:00:11 smithi162 ceph-mon[26841]: rocksdb: Options.is_fd_close_on_exec: 1 2024-04-27T04:00:11.269 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:00:11 smithi162 ceph-mon[26841]: rocksdb: Options.advise_random_on_open: 1 2024-04-27T04:00:11.269 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:00:11 smithi162 ceph-mon[26841]: rocksdb: Options.db_write_buffer_size: 0 2024-04-27T04:00:11.269 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:00:11 smithi162 ceph-mon[26841]: rocksdb: Options.write_buffer_manager: 0x564744f6e9c0 2024-04-27T04:00:11.269 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:00:11 smithi162 ceph-mon[26841]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-04-27T04:00:11.269 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:00:11 smithi162 ceph-mon[26841]: rocksdb: Options.new_table_reader_for_compaction_inputs: 0 2024-04-27T04:00:11.269 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:00:11 smithi162 ceph-mon[26841]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-04-27T04:00:11.269 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:00:11 smithi162 ceph-mon[26841]: rocksdb: Options.use_adaptive_mutex: 0 2024-04-27T04:00:11.269 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:00:11 smithi162 ceph-mon[26841]: rocksdb: Options.rate_limiter: (nil) 2024-04-27T04:00:11.269 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:00:11 smithi162 ceph-mon[26841]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-04-27T04:00:11.269 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:00:11 smithi162 ceph-mon[26841]: rocksdb: Options.wal_recovery_mode: 2 2024-04-27T04:00:11.269 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:00:11 smithi162 ceph-mon[26841]: rocksdb: Options.enable_thread_tracking: 0 2024-04-27T04:00:11.269 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:00:11 smithi162 ceph-mon[26841]: rocksdb: Options.enable_pipelined_write: 0 2024-04-27T04:00:11.271 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:00:11 smithi162 ceph-mon[26841]: rocksdb: Options.unordered_write: 0 2024-04-27T04:00:11.271 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:00:11 smithi162 ceph-mon[26841]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-04-27T04:00:11.271 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:00:11 smithi162 ceph-mon[26841]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-04-27T04:00:11.271 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:00:11 smithi162 ceph-mon[26841]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-04-27T04:00:11.271 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:00:11 smithi162 ceph-mon[26841]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-04-27T04:00:11.271 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:00:11 smithi162 ceph-mon[26841]: rocksdb: Options.row_cache: None 2024-04-27T04:00:11.271 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:00:11 smithi162 ceph-mon[26841]: rocksdb: Options.wal_filter: None 2024-04-27T04:00:11.271 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:00:11 smithi162 ceph-mon[26841]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-04-27T04:00:11.271 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:00:11 smithi162 ceph-mon[26841]: rocksdb: Options.allow_ingest_behind: 0 2024-04-27T04:00:11.271 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:00:11 smithi162 ceph-mon[26841]: rocksdb: Options.preserve_deletes: 0 2024-04-27T04:00:11.272 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:00:11 smithi162 ceph-mon[26841]: rocksdb: Options.two_write_queues: 0 2024-04-27T04:00:11.272 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:00:11 smithi162 ceph-mon[26841]: rocksdb: Options.manual_wal_flush: 0 2024-04-27T04:00:11.272 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:00:11 smithi162 ceph-mon[26841]: rocksdb: Options.atomic_flush: 0 2024-04-27T04:00:11.272 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:00:11 smithi162 ceph-mon[26841]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-04-27T04:00:11.272 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:00:11 smithi162 ceph-mon[26841]: rocksdb: Options.persist_stats_to_disk: 0 2024-04-27T04:00:11.272 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:00:11 smithi162 ceph-mon[26841]: rocksdb: Options.write_dbid_to_manifest: 0 2024-04-27T04:00:11.272 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:00:11 smithi162 ceph-mon[26841]: rocksdb: Options.log_readahead_size: 0 2024-04-27T04:00:11.272 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:00:11 smithi162 ceph-mon[26841]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-04-27T04:00:11.272 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:00:11 smithi162 ceph-mon[26841]: rocksdb: Options.best_efforts_recovery: 0 2024-04-27T04:00:11.272 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:00:11 smithi162 ceph-mon[26841]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-04-27T04:00:11.272 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:00:11 smithi162 ceph-mon[26841]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-04-27T04:00:11.272 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:00:11 smithi162 ceph-mon[26841]: rocksdb: Options.allow_data_in_errors: 0 2024-04-27T04:00:11.272 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:00:11 smithi162 ceph-mon[26841]: rocksdb: Options.db_host_id: __hostname__ 2024-04-27T04:00:11.273 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:00:11 smithi162 ceph-mon[26841]: rocksdb: Options.max_background_jobs: 2 2024-04-27T04:00:11.273 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:00:11 smithi162 ceph-mon[26841]: rocksdb: Options.max_background_compactions: -1 2024-04-27T04:00:11.273 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:00:11 smithi162 ceph-mon[26841]: rocksdb: Options.max_subcompactions: 1 2024-04-27T04:00:11.273 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:00:11 smithi162 ceph-mon[26841]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-04-27T04:00:11.273 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:00:11 smithi162 ceph-mon[26841]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-04-27T04:00:11.273 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:00:11 smithi162 ceph-mon[26841]: rocksdb: Options.delayed_write_rate : 16777216 2024-04-27T04:00:11.273 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:00:11 smithi162 ceph-mon[26841]: rocksdb: Options.max_total_wal_size: 0 2024-04-27T04:00:11.273 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:00:11 smithi162 ceph-mon[26841]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-04-27T04:00:11.273 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:00:11 smithi162 ceph-mon[26841]: rocksdb: Options.stats_dump_period_sec: 600 2024-04-27T04:00:11.273 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:00:11 smithi162 ceph-mon[26841]: rocksdb: Options.stats_persist_period_sec: 600 2024-04-27T04:00:11.273 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:00:11 smithi162 ceph-mon[26841]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-04-27T04:00:11.273 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:00:11 smithi162 ceph-mon[26841]: rocksdb: Options.max_open_files: -1 2024-04-27T04:00:11.273 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:00:11 smithi162 ceph-mon[26841]: rocksdb: Options.bytes_per_sync: 0 2024-04-27T04:00:11.273 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:00:11 smithi162 ceph-mon[26841]: rocksdb: Options.wal_bytes_per_sync: 0 2024-04-27T04:00:11.273 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:00:11 smithi162 ceph-mon[26841]: rocksdb: Options.strict_bytes_per_sync: 0 2024-04-27T04:00:11.274 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:00:11 smithi162 ceph-mon[26841]: rocksdb: Options.compaction_readahead_size: 0 2024-04-27T04:00:11.274 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:00:11 smithi162 ceph-mon[26841]: rocksdb: Options.max_background_flushes: -1 2024-04-27T04:00:11.274 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:00:11 smithi162 ceph-mon[26841]: rocksdb: Compression algorithms supported: 2024-04-27T04:00:11.274 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:00:11 smithi162 ceph-mon[26841]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-04-27T04:00:11.274 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:00:11 smithi162 ceph-mon[26841]: rocksdb: kZSTD supported: 0 2024-04-27T04:00:11.274 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:00:11 smithi162 ceph-mon[26841]: rocksdb: kXpressCompression supported: 0 2024-04-27T04:00:11.274 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:00:11 smithi162 ceph-mon[26841]: rocksdb: kLZ4HCCompression supported: 1 2024-04-27T04:00:11.274 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:00:11 smithi162 ceph-mon[26841]: rocksdb: kZlibCompression supported: 1 2024-04-27T04:00:11.274 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:00:11 smithi162 ceph-mon[26841]: rocksdb: kSnappyCompression supported: 1 2024-04-27T04:00:11.274 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:00:11 smithi162 ceph-mon[26841]: rocksdb: kLZ4Compression supported: 1 2024-04-27T04:00:11.274 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:00:11 smithi162 ceph-mon[26841]: rocksdb: kBZip2Compression supported: 0 2024-04-27T04:00:11.274 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:00:11 smithi162 ceph-mon[26841]: rocksdb: Fast CRC32 supported: Supported on x86 2024-04-27T04:00:11.275 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:00:11 smithi162 ceph-mon[26841]: rocksdb: [db/version_set.cc:4724] Recovering from manifest file: /var/lib/ceph/mon/ceph-smithi162/store.db/MANIFEST-000009 2024-04-27T04:00:11.275 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:00:11 smithi162 ceph-mon[26841]: rocksdb: [db/column_family.cc:595] --------------- Options for column family [default]: 2024-04-27T04:00:11.275 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:00:11 smithi162 ceph-mon[26841]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-04-27T04:00:11.275 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:00:11 smithi162 ceph-mon[26841]: rocksdb: Options.merge_operator: 2024-04-27T04:00:11.275 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:00:11 smithi162 ceph-mon[26841]: rocksdb: Options.compaction_filter: None 2024-04-27T04:00:11.275 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:00:11 smithi162 ceph-mon[26841]: rocksdb: Options.compaction_filter_factory: None 2024-04-27T04:00:11.275 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:00:11 smithi162 ceph-mon[26841]: rocksdb: Options.sst_partitioner_factory: None 2024-04-27T04:00:11.275 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:00:11 smithi162 ceph-mon[26841]: rocksdb: Options.memtable_factory: SkipListFactory 2024-04-27T04:00:11.275 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:00:11 smithi162 ceph-mon[26841]: rocksdb: Options.table_factory: BlockBasedTable 2024-04-27T04:00:11.275 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:00:11 smithi162 ceph-mon[26841]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x56474424ef58) 2024-04-27T04:00:11.275 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout: cache_index_and_filter_blocks: 1 2024-04-27T04:00:11.275 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-04-27T04:00:11.275 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-04-27T04:00:11.275 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout: pin_top_level_index_and_filter: 1 2024-04-27T04:00:11.276 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout: index_type: 0 2024-04-27T04:00:11.276 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout: data_block_index_type: 0 2024-04-27T04:00:11.276 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout: index_shortening: 1 2024-04-27T04:00:11.276 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout: data_block_hash_table_util_ratio: 0.750000 2024-04-27T04:00:11.276 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout: hash_index_allow_collision: 1 2024-04-27T04:00:11.276 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout: checksum: 1 2024-04-27T04:00:11.276 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout: no_block_cache: 0 2024-04-27T04:00:11.276 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout: block_cache: 0x564744281090 2024-04-27T04:00:11.276 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout: block_cache_name: BinnedLRUCache 2024-04-27T04:00:11.276 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout: block_cache_options: 2024-04-27T04:00:11.276 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout: capacity : 536870912 2024-04-27T04:00:11.276 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout: num_shard_bits : 4 2024-04-27T04:00:11.276 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout: strict_capacity_limit : 0 2024-04-27T04:00:11.277 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout: high_pri_pool_ratio: 0.000 2024-04-27T04:00:11.277 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout: block_cache_compressed: (nil) 2024-04-27T04:00:11.277 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout: persistent_cache: (nil) 2024-04-27T04:00:11.277 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout: block_size: 4096 2024-04-27T04:00:11.277 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout: block_size_deviation: 10 2024-04-27T04:00:11.277 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout: block_restart_interval: 16 2024-04-27T04:00:11.277 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout: index_block_restart_interval: 1 2024-04-27T04:00:11.277 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout: metadata_block_size: 4096 2024-04-27T04:00:11.277 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout: partition_filters: 0 2024-04-27T04:00:11.277 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout: use_delta_encoding: 1 2024-04-27T04:00:11.277 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout: filter_policy: rocksdb.BuiltinBloomFilter 2024-04-27T04:00:11.277 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout: whole_key_filtering: 1 2024-04-27T04:00:11.277 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout: verify_compression: 0 2024-04-27T04:00:11.277 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout: read_amp_bytes_per_bit: 0 2024-04-27T04:00:11.277 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout: format_version: 4 2024-04-27T04:00:11.278 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout: enable_index_compression: 1 2024-04-27T04:00:11.278 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout: block_align: 0 2024-04-27T04:00:11.278 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:00:11 smithi162 ceph-mon[26841]: rocksdb: Options.write_buffer_size: 33554432 2024-04-27T04:00:11.278 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:00:11 smithi162 ceph-mon[26841]: rocksdb: Options.max_write_buffer_number: 2 2024-04-27T04:00:11.278 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:00:11 smithi162 ceph-mon[26841]: rocksdb: Options.compression: NoCompression 2024-04-27T04:00:11.278 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:00:11 smithi162 ceph-mon[26841]: rocksdb: Options.bottommost_compression: Disabled 2024-04-27T04:00:11.278 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:00:11 smithi162 ceph-mon[26841]: rocksdb: Options.prefix_extractor: nullptr 2024-04-27T04:00:11.278 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:00:11 smithi162 ceph-mon[26841]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-04-27T04:00:11.278 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:00:11 smithi162 ceph-mon[26841]: rocksdb: Options.num_levels: 7 2024-04-27T04:00:11.278 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:00:11 smithi162 ceph-mon[26841]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-04-27T04:00:11.278 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:00:11 smithi162 ceph-mon[26841]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-04-27T04:00:11.278 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:00:11 smithi162 ceph-mon[26841]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-04-27T04:00:11.278 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:00:11 smithi162 ceph-mon[26841]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-04-27T04:00:11.279 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:00:11 smithi162 ceph-mon[26841]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-04-27T04:00:11.279 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:00:11 smithi162 ceph-mon[26841]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-04-27T04:00:11.279 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:00:11 smithi162 ceph-mon[26841]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-04-27T04:00:11.279 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:00:11 smithi162 ceph-mon[26841]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-04-27T04:00:11.279 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:00:11 smithi162 ceph-mon[26841]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-04-27T04:00:11.279 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:00:11 smithi162 ceph-mon[26841]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-04-27T04:00:11.279 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:00:11 smithi162 ceph-mon[26841]: rocksdb: Options.compression_opts.window_bits: -14 2024-04-27T04:00:11.279 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:00:11 smithi162 ceph-mon[26841]: rocksdb: Options.compression_opts.level: 32767 2024-04-27T04:00:11.279 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:00:11 smithi162 ceph-mon[26841]: rocksdb: Options.compression_opts.strategy: 0 2024-04-27T04:00:11.279 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:00:11 smithi162 ceph-mon[26841]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-04-27T04:00:11.279 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:00:11 smithi162 ceph-mon[26841]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-04-27T04:00:11.279 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:00:11 smithi162 ceph-mon[26841]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-04-27T04:00:11.279 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:00:11 smithi162 ceph-mon[26841]: rocksdb: Options.compression_opts.enabled: false 2024-04-27T04:00:11.279 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:00:11 smithi162 ceph-mon[26841]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-04-27T04:00:11.280 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:00:11 smithi162 ceph-mon[26841]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-04-27T04:00:11.280 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:00:11 smithi162 ceph-mon[26841]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-04-27T04:00:11.280 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:00:11 smithi162 ceph-mon[26841]: rocksdb: Options.target_file_size_base: 67108864 2024-04-27T04:00:11.280 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:00:11 smithi162 ceph-mon[26841]: rocksdb: Options.target_file_size_multiplier: 1 2024-04-27T04:00:11.280 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:00:11 smithi162 ceph-mon[26841]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-04-27T04:00:11.280 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:00:11 smithi162 ceph-mon[26841]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-04-27T04:00:11.280 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:00:11 smithi162 ceph-mon[26841]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-04-27T04:00:11.280 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:00:11 smithi162 ceph-mon[26841]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-04-27T04:00:11.280 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:00:11 smithi162 ceph-mon[26841]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-04-27T04:00:11.280 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:00:11 smithi162 ceph-mon[26841]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-04-27T04:00:11.280 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:00:11 smithi162 ceph-mon[26841]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-04-27T04:00:11.280 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:00:11 smithi162 ceph-mon[26841]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-04-27T04:00:11.280 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:00:11 smithi162 ceph-mon[26841]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-04-27T04:00:11.281 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:00:11 smithi162 ceph-mon[26841]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-04-27T04:00:11.281 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:00:11 smithi162 ceph-mon[26841]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-04-27T04:00:11.281 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:00:11 smithi162 ceph-mon[26841]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-04-27T04:00:11.281 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:00:11 smithi162 ceph-mon[26841]: rocksdb: Options.arena_block_size: 4194304 2024-04-27T04:00:11.281 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:00:11 smithi162 ceph-mon[26841]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-04-27T04:00:11.281 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:00:11 smithi162 ceph-mon[26841]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-04-27T04:00:11.281 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:00:11 smithi162 ceph-mon[26841]: rocksdb: Options.rate_limit_delay_max_milliseconds: 100 2024-04-27T04:00:11.281 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:00:11 smithi162 ceph-mon[26841]: rocksdb: Options.disable_auto_compactions: 0 2024-04-27T04:00:11.281 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:00:11 smithi162 ceph-mon[26841]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-04-27T04:00:11.281 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:00:11 smithi162 ceph-mon[26841]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-04-27T04:00:11.281 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:00:11 smithi162 ceph-mon[26841]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-04-27T04:00:11.281 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:00:11 smithi162 ceph-mon[26841]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-04-27T04:00:11.281 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:00:11 smithi162 ceph-mon[26841]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-04-27T04:00:11.281 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:00:11 smithi162 ceph-mon[26841]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-04-27T04:00:11.282 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:00:11 smithi162 ceph-mon[26841]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-04-27T04:00:11.282 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:00:11 smithi162 ceph-mon[26841]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-04-27T04:00:11.282 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:00:11 smithi162 ceph-mon[26841]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-04-27T04:00:11.282 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:00:11 smithi162 ceph-mon[26841]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-04-27T04:00:11.282 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:00:11 smithi162 ceph-mon[26841]: rocksdb: Options.table_properties_collectors: 2024-04-27T04:00:11.282 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:00:11 smithi162 ceph-mon[26841]: rocksdb: Options.inplace_update_support: 0 2024-04-27T04:00:11.282 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:00:11 smithi162 ceph-mon[26841]: rocksdb: Options.inplace_update_num_locks: 10000 2024-04-27T04:00:11.282 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:00:11 smithi162 ceph-mon[26841]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-04-27T04:00:11.282 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:00:11 smithi162 ceph-mon[26841]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-04-27T04:00:11.282 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:00:11 smithi162 ceph-mon[26841]: rocksdb: Options.memtable_huge_page_size: 0 2024-04-27T04:00:11.282 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:00:11 smithi162 ceph-mon[26841]: rocksdb: Options.bloom_locality: 0 2024-04-27T04:00:11.282 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:00:11 smithi162 ceph-mon[26841]: rocksdb: Options.max_successive_merges: 0 2024-04-27T04:00:11.282 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:00:11 smithi162 ceph-mon[26841]: rocksdb: Options.optimize_filters_for_hits: 0 2024-04-27T04:00:11.282 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:00:11 smithi162 ceph-mon[26841]: rocksdb: Options.paranoid_file_checks: 0 2024-04-27T04:00:11.283 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:00:11 smithi162 ceph-mon[26841]: rocksdb: Options.force_consistency_checks: 1 2024-04-27T04:00:11.283 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:00:11 smithi162 ceph-mon[26841]: rocksdb: Options.report_bg_io_stats: 0 2024-04-27T04:00:11.283 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:00:11 smithi162 ceph-mon[26841]: rocksdb: Options.ttl: 2592000 2024-04-27T04:00:11.283 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:00:11 smithi162 ceph-mon[26841]: rocksdb: Options.periodic_compaction_seconds: 0 2024-04-27T04:00:11.283 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:00:11 smithi162 ceph-mon[26841]: rocksdb: Options.enable_blob_files: false 2024-04-27T04:00:11.283 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:00:11 smithi162 ceph-mon[26841]: rocksdb: Options.min_blob_size: 0 2024-04-27T04:00:11.283 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:00:11 smithi162 ceph-mon[26841]: rocksdb: Options.blob_file_size: 268435456 2024-04-27T04:00:11.283 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:00:11 smithi162 ceph-mon[26841]: rocksdb: Options.blob_compression_type: NoCompression 2024-04-27T04:00:11.283 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:00:11 smithi162 ceph-mon[26841]: rocksdb: Options.enable_blob_garbage_collection: false 2024-04-27T04:00:11.283 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:00:11 smithi162 ceph-mon[26841]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-04-27T04:00:11.283 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:00:11 smithi162 ceph-mon[26841]: rocksdb: [db/version_set.cc:4764] Recovered from manifest file:/var/lib/ceph/mon/ceph-smithi162/store.db/MANIFEST-000009 succeeded,manifest_file_number is 9, next_file_number is 11, last_sequence is 5, log_number is 5,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 0 2024-04-27T04:00:11.283 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:00:11 smithi162 ceph-mon[26841]: rocksdb: [db/version_set.cc:4779] Column family [default] (ID 0), log number is 5 2024-04-27T04:00:11.283 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:00:11 smithi162 ceph-mon[26841]: rocksdb: [db/version_set.cc:4082] Creating manifest 13 2024-04-27T04:00:11.283 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:00:11 smithi162 ceph-mon[26841]: rocksdb: EVENT_LOG_v1 {"time_micros": 1714190411217963, "job": 1, "event": "recovery_started", "wal_files": [10]} 2024-04-27T04:00:11.284 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:00:11 smithi162 ceph-mon[26841]: rocksdb: [db/db_impl/db_impl_open.cc:845] Recovering log #10 mode 2 2024-04-27T04:00:11.284 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:00:11 smithi162 ceph-mon[26841]: rocksdb: [table/block_based/filter_policy.cc:991] Using legacy Bloom filter with high (20) bits/key. Dramatic filter space and/or accuracy improvement is available with format_version>=5. 2024-04-27T04:00:11.284 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:00:11 smithi162 ceph-mon[26841]: rocksdb: EVENT_LOG_v1 {"time_micros": 1714190411218885, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 14, "file_size": 93382, "file_checksum": "", "file_checksum_func_name": "Unknown", "table_properties": {"data_size": 91494, "index_size": 248, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 709, "raw_key_size": 13576, "raw_average_key_size": 49, "raw_value_size": 84058, "raw_average_value_size": 307, "num_data_blocks": 11, "num_entries": 273, "num_deletions": 3, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "rocksdb.BuiltinBloomFilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; ", "creation_time": 1714190411, "oldest_key_time": 0, "file_creation_time": 0, "db_id": "e34ebe7d-df06-467d-81b8-43c81d228e86", "db_session_id": "NWA5R4JT27PBG8OGKAC7"}} 2024-04-27T04:00:11.284 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:00:11 smithi162 ceph-mon[26841]: rocksdb: [db/version_set.cc:4082] Creating manifest 15 2024-04-27T04:00:11.284 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:00:11 smithi162 ceph-mon[26841]: rocksdb: EVENT_LOG_v1 {"time_micros": 1714190411219216, "job": 1, "event": "recovery_finished"} 2024-04-27T04:00:11.284 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:00:11 smithi162 ceph-mon[26841]: rocksdb: [file/delete_scheduler.cc:69] Deleted file /var/lib/ceph/mon/ceph-smithi162/store.db/000010.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-04-27T04:00:11.284 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:00:11 smithi162 ceph-mon[26841]: rocksdb: [db/db_impl/db_impl_open.cc:1700] SstFileManager instance 0x564744215180 2024-04-27T04:00:11.284 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:00:11 smithi162 ceph-mon[26841]: rocksdb: DB pointer 0x5647442ea000 2024-04-27T04:00:11.284 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:00:11 smithi162 ceph-mon[26841]: rocksdb: [db/db_impl/db_impl.cc:901] ------- DUMPING STATS ------- 2024-04-27T04:00:11.284 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:00:11 smithi162 ceph-mon[26841]: rocksdb: [db/db_impl/db_impl.cc:903] 2024-04-27T04:00:11.284 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout: ** DB Stats ** 2024-04-27T04:00:11.284 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-04-27T04:00:11.284 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-04-27T04:00:11.285 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-04-27T04:00:11.285 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-04-27T04:00:11.285 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-04-27T04:00:11.285 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 MB, 0.00 MB/s 2024-04-27T04:00:11.285 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-04-27T04:00:11.285 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout: 2024-04-27T04:00:11.285 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout: ** Compaction Stats [default] ** 2024-04-27T04:00:11.285 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop 2024-04-27T04:00:11.285 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout: ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-04-27T04:00:11.285 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout: L0 2/0 92.99 KB 0.5 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 133.1 0.00 0.00 1 0.001 0 0 2024-04-27T04:00:11.285 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout: Sum 2/0 92.99 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 133.1 0.00 0.00 1 0.001 0 0 2024-04-27T04:00:11.285 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 133.1 0.00 0.00 1 0.001 0 0 2024-04-27T04:00:11.285 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout: 2024-04-27T04:00:11.285 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout: ** Compaction Stats [default] ** 2024-04-27T04:00:11.286 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop 2024-04-27T04:00:11.286 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-04-27T04:00:11.286 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 133.1 0.00 0.00 1 0.001 0 0 2024-04-27T04:00:11.286 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-04-27T04:00:11.286 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-04-27T04:00:11.286 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-04-27T04:00:11.286 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-04-27T04:00:11.286 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-04-27T04:00:11.286 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout: AddFile(Keys): cumulative 0, interval 0 2024-04-27T04:00:11.286 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout: Cumulative compaction: 0.00 GB write, 29.01 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-04-27T04:00:11.286 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout: Interval compaction: 0.00 GB write, 29.01 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-04-27T04:00:11.287 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-04-27T04:00:11.287 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout: 2024-04-27T04:00:11.287 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout: ** File Read Latency Histogram By Level [default] ** 2024-04-27T04:00:11.287 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout: 2024-04-27T04:00:11.287 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout: ** Compaction Stats [default] ** 2024-04-27T04:00:11.287 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop 2024-04-27T04:00:11.287 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout: ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-04-27T04:00:11.287 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout: L0 2/0 92.99 KB 0.5 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 133.1 0.00 0.00 1 0.001 0 0 2024-04-27T04:00:11.287 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout: Sum 2/0 92.99 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 133.1 0.00 0.00 1 0.001 0 0 2024-04-27T04:00:11.287 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 2024-04-27T04:00:11.287 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout: 2024-04-27T04:00:11.287 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout: ** Compaction Stats [default] ** 2024-04-27T04:00:11.287 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop 2024-04-27T04:00:11.288 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-04-27T04:00:11.288 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 133.1 0.00 0.00 1 0.001 0 0 2024-04-27T04:00:11.288 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-04-27T04:00:11.288 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-04-27T04:00:11.288 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-04-27T04:00:11.288 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-04-27T04:00:11.288 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-04-27T04:00:11.288 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout: AddFile(Keys): cumulative 0, interval 0 2024-04-27T04:00:11.288 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout: Cumulative compaction: 0.00 GB write, 28.72 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-04-27T04:00:11.288 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout: Interval compaction: 0.00 GB write, 0.00 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-04-27T04:00:11.288 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-04-27T04:00:11.288 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout: 2024-04-27T04:00:11.288 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout: ** File Read Latency Histogram By Level [default] ** 2024-04-27T04:00:11.288 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:00:11 smithi162 ceph-mon[26841]: starting mon.smithi162 rank 0 at public addrs [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] at bind addrs [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon_data /var/lib/ceph/mon/ceph-smithi162 fsid 7f9750d6-044a-11ef-bc93-c7b262605968 2024-04-27T04:00:11.289 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:00:11 smithi162 ceph-mon[26841]: mon.smithi162@-1(???) e1 preinit fsid 7f9750d6-044a-11ef-bc93-c7b262605968 2024-04-27T04:00:11.289 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:00:11 smithi162 ceph-mon[26841]: mon.smithi162@-1(???).mds e1 new map 2024-04-27T04:00:11.289 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:00:11 smithi162 ceph-mon[26841]: mon.smithi162@-1(???).mds e1 print_map 2024-04-27T04:00:11.289 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout: e1 2024-04-27T04:00:11.289 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-04-27T04:00:11.289 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-27T04:00:11.289 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout: legacy client fscid: -1 2024-04-27T04:00:11.289 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout: 2024-04-27T04:00:11.289 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout: No filesystems configured 2024-04-27T04:00:11.289 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:00:11 smithi162 ceph-mon[26841]: mon.smithi162@-1(???).osd e1 crush map has features 3314932999778484224, adjusting msgr requires 2024-04-27T04:00:11.289 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:00:11 smithi162 ceph-mon[26841]: mon.smithi162@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-04-27T04:00:11.289 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:00:11 smithi162 ceph-mon[26841]: mon.smithi162@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-04-27T04:00:11.289 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:00:11 smithi162 ceph-mon[26841]: mon.smithi162@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-04-27T04:00:11.289 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:00:11 smithi162 ceph-mon[26841]: mon.smithi162@-1(???).paxosservice(auth 1..2) refresh upgraded, format 0 -> 3 2024-04-27T04:00:11.290 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:00:11 smithi162 bash[26822]: 0f1e3682b9abe73f7036e1738bc1dc3ccb64fdc5377b198e5b3f4cd759ea5a3c 2024-04-27T04:00:11.290 INFO:teuthology.orchestra.run.smithi162.stdout:Setting public_network to 172.21.0.0/20 in mon config section 2024-04-27T04:00:11.586 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:00:11 smithi162 systemd[1]: Started Ceph mon.smithi162 for 7f9750d6-044a-11ef-bc93-c7b262605968. 2024-04-27T04:00:11.586 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:00:11 smithi162 ceph-mon[26841]: mon.smithi162 is new leader, mons smithi162 in quorum (ranks 0) 2024-04-27T04:00:11.586 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:00:11 smithi162 ceph-mon[26841]: monmap e1: 1 mons at {smithi162=[v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-04-27T04:00:11.586 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:00:11 smithi162 ceph-mon[26841]: fsmap 2024-04-27T04:00:11.586 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:00:11 smithi162 ceph-mon[26841]: osdmap e1: 0 total, 0 up, 0 in 2024-04-27T04:00:11.587 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:00:11 smithi162 ceph-mon[26841]: mgrmap e1: no daemons active 2024-04-27T04:00:12.371 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:11.683+0000 7f1ffd3a8700 1 Processor -- start 2024-04-27T04:00:12.371 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:11.683+0000 7f1ffd3a8700 1 -- start start 2024-04-27T04:00:12.371 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:11.683+0000 7f1ffd3a8700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1ff8076bf0 0x7f1ff8105460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:00:12.371 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:11.683+0000 7f1ffd3a8700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ff81059a0 con 0x7f1ff8076bf0 2024-04-27T04:00:12.372 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:11.683+0000 7f1ff6ffd700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1ff8076bf0 0x7f1ff8105460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:00:12.372 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:11.683+0000 7f1ff6ffd700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1ff8076bf0 0x7f1ff8105460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:47656/0 (socket says 172.21.15.162:47656) 2024-04-27T04:00:12.372 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:11.683+0000 7f1ff6ffd700 1 -- 172.21.15.162:0/1616558285 learned_addr learned my addr 172.21.15.162:0/1616558285 (peer_addr_for_me v2:172.21.15.162:0/0) 2024-04-27T04:00:12.372 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:11.683+0000 7f1ff6ffd700 1 -- 172.21.15.162:0/1616558285 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ff8105ae0 con 0x7f1ff8076bf0 2024-04-27T04:00:12.372 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:11.684+0000 7f1ff6ffd700 1 --2- 172.21.15.162:0/1616558285 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1ff8076bf0 0x7f1ff8105460 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f1fe80098e0 tx=0x7f1fe8009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=25f482e117a7a245 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:00:12.372 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:11.684+0000 7f1ff5ffb700 1 -- 172.21.15.162:0/1616558285 <== mon.0 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f1fe800b860 con 0x7f1ff8076bf0 2024-04-27T04:00:12.372 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:11.684+0000 7f1ff5ffb700 1 -- 172.21.15.162:0/1616558285 <== mon.0 v2:172.21.15.162:3300/0 2 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7f1fe800be50 con 0x7f1ff8076bf0 2024-04-27T04:00:12.372 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:11.684+0000 7f1ff5ffb700 1 -- 172.21.15.162:0/1616558285 <== mon.0 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f1fe8010380 con 0x7f1ff8076bf0 2024-04-27T04:00:12.372 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:11.684+0000 7f1ffd3a8700 1 -- 172.21.15.162:0/1616558285 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1ff8076bf0 msgr2=0x7f1ff8105460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:00:12.372 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:11.684+0000 7f1ffd3a8700 1 --2- 172.21.15.162:0/1616558285 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1ff8076bf0 0x7f1ff8105460 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f1fe80098e0 tx=0x7f1fe8009250 comp rx=0 tx=0).stop 2024-04-27T04:00:12.372 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:11.684+0000 7f1ffd3a8700 1 -- 172.21.15.162:0/1616558285 shutdown_connections 2024-04-27T04:00:12.372 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:11.684+0000 7f1ffd3a8700 1 --2- 172.21.15.162:0/1616558285 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1ff8076bf0 0x7f1ff8105460 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:00:12.372 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:11.684+0000 7f1ffd3a8700 1 -- 172.21.15.162:0/1616558285 >> 172.21.15.162:0/1616558285 conn(0x7f1ff80fcef0 msgr2=0x7f1ff80ff300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:00:12.373 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:11.684+0000 7f1ffd3a8700 1 -- 172.21.15.162:0/1616558285 shutdown_connections 2024-04-27T04:00:12.373 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:11.685+0000 7f1ffd3a8700 1 -- 172.21.15.162:0/1616558285 wait complete. 2024-04-27T04:00:12.373 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:11.685+0000 7f1ffd3a8700 1 Processor -- start 2024-04-27T04:00:12.373 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:11.685+0000 7f1ffd3a8700 1 -- start start 2024-04-27T04:00:12.373 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:11.685+0000 7f1ffd3a8700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1ff8076bf0 0x7f1ff8102db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:00:12.373 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:11.685+0000 7f1ffd3a8700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ff81032f0 con 0x7f1ff8076bf0 2024-04-27T04:00:12.373 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:11.685+0000 7f1ff6ffd700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1ff8076bf0 0x7f1ff8102db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:00:12.373 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:11.685+0000 7f1ff6ffd700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1ff8076bf0 0x7f1ff8102db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:47668/0 (socket says 172.21.15.162:47668) 2024-04-27T04:00:12.373 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:11.685+0000 7f1ff6ffd700 1 -- 172.21.15.162:0/2638016437 learned_addr learned my addr 172.21.15.162:0/2638016437 (peer_addr_for_me v2:172.21.15.162:0/0) 2024-04-27T04:00:12.373 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:11.686+0000 7f1ff6ffd700 1 -- 172.21.15.162:0/2638016437 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1fe80094f0 con 0x7f1ff8076bf0 2024-04-27T04:00:12.373 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:11.686+0000 7f1ff6ffd700 1 --2- 172.21.15.162:0/2638016437 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1ff8076bf0 0x7f1ff8102db0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f1fe8003050 tx=0x7f1fe8009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:00:12.373 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:11.686+0000 7f1fdf7fe700 1 -- 172.21.15.162:0/2638016437 <== mon.0 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f1fe8010760 con 0x7f1ff8076bf0 2024-04-27T04:00:12.373 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:11.686+0000 7f1fdf7fe700 1 -- 172.21.15.162:0/2638016437 <== mon.0 v2:172.21.15.162:3300/0 2 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7f1fe8010d50 con 0x7f1ff8076bf0 2024-04-27T04:00:12.373 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:11.686+0000 7f1ffd3a8700 1 -- 172.21.15.162:0/2638016437 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1ff8101360 con 0x7f1ff8076bf0 2024-04-27T04:00:12.374 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:11.686+0000 7f1fdf7fe700 1 -- 172.21.15.162:0/2638016437 <== mon.0 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f1fe801ae20 con 0x7f1ff8076bf0 2024-04-27T04:00:12.374 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:11.686+0000 7f1ffd3a8700 1 -- 172.21.15.162:0/2638016437 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1ff8101850 con 0x7f1ff8076bf0 2024-04-27T04:00:12.374 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:11.686+0000 7f1fdf7fe700 1 -- 172.21.15.162:0/2638016437 <== mon.0 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 1) v1 ==== 529+0+0 (secure 0 0 0) 0x7f1fe802e440 con 0x7f1ff8076bf0 2024-04-27T04:00:12.374 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:11.686+0000 7f1fdf7fe700 1 -- 172.21.15.162:0/2638016437 <== mon.0 v2:172.21.15.162:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 721+0+0 (secure 0 0 0) 0x7f1fe8024740 con 0x7f1ff8076bf0 2024-04-27T04:00:12.374 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:11.687+0000 7f1ffd3a8700 1 -- 172.21.15.162:0/2638016437 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1ff804bed0 con 0x7f1ff8076bf0 2024-04-27T04:00:12.374 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:11.689+0000 7f1fdf7fe700 1 -- 172.21.15.162:0/2638016437 <== mon.0 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71547 (secure 0 0 0) 0x7f1fe8029070 con 0x7f1ff8076bf0 2024-04-27T04:00:12.374 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:11.755+0000 7f1ffd3a8700 1 -- 172.21.15.162:0/2638016437 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command([{prefix=config set, name=public_network}] v 0) v1 -- 0x7f1ff8101a90 con 0x7f1ff8076bf0 2024-04-27T04:00:12.374 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:11.760+0000 7f1fdf7fe700 1 -- 172.21.15.162:0/2638016437 <== mon.0 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{prefix=config set, name=public_network}]=0 v3)=0 v3) v1 ==== 124+0+0 (secure 0 0 0) 0x7f1fe8010ec0 con 0x7f1ff8076bf0 2024-04-27T04:00:12.374 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:11.760+0000 7f1ffd3a8700 1 -- 172.21.15.162:0/2638016437 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1ff8076bf0 msgr2=0x7f1ff8102db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:00:12.374 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:11.760+0000 7f1ffd3a8700 1 --2- 172.21.15.162:0/2638016437 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1ff8076bf0 0x7f1ff8102db0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f1fe8003050 tx=0x7f1fe8009250 comp rx=0 tx=0).stop 2024-04-27T04:00:12.374 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:11.761+0000 7f1ffd3a8700 1 -- 172.21.15.162:0/2638016437 shutdown_connections 2024-04-27T04:00:12.374 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:11.761+0000 7f1ffd3a8700 1 --2- 172.21.15.162:0/2638016437 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1ff8076bf0 0x7f1ff8102db0 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:00:12.374 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:11.761+0000 7f1ffd3a8700 1 -- 172.21.15.162:0/2638016437 >> 172.21.15.162:0/2638016437 conn(0x7f1ff80fcef0 msgr2=0x7f1ff80fd9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:00:12.375 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:11.761+0000 7f1ffd3a8700 1 -- 172.21.15.162:0/2638016437 shutdown_connections 2024-04-27T04:00:12.375 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:11.761+0000 7f1ffd3a8700 1 -- 172.21.15.162:0/2638016437 wait complete. 2024-04-27T04:00:12.375 INFO:teuthology.orchestra.run.smithi162.stdout:Wrote config to /etc/ceph/ceph.conf 2024-04-27T04:00:12.375 INFO:teuthology.orchestra.run.smithi162.stdout:Wrote keyring to /etc/ceph/ceph.client.admin.keyring 2024-04-27T04:00:12.375 INFO:teuthology.orchestra.run.smithi162.stdout:Creating mgr... 2024-04-27T04:00:12.375 INFO:teuthology.orchestra.run.smithi162.stdout:Verifying port 9283 ... 2024-04-27T04:00:12.540 INFO:teuthology.orchestra.run.smithi162.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-7f9750d6-044a-11ef-bc93-c7b262605968@mgr.smithi162.oennfn 2024-04-27T04:00:12.540 INFO:teuthology.orchestra.run.smithi162.stdout:systemctl: stderr Failed to reset failed state of unit ceph-7f9750d6-044a-11ef-bc93-c7b262605968@mgr.smithi162.oennfn.service: Unit ceph-7f9750d6-044a-11ef-bc93-c7b262605968@mgr.smithi162.oennfn.service not loaded. 2024-04-27T04:00:12.686 INFO:teuthology.orchestra.run.smithi162.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-7f9750d6-044a-11ef-bc93-c7b262605968.target.wants/ceph-7f9750d6-044a-11ef-bc93-c7b262605968@mgr.smithi162.oennfn.service → /etc/systemd/system/ceph-7f9750d6-044a-11ef-bc93-c7b262605968@.service. 2024-04-27T04:00:13.027 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:00:12 smithi162 ceph-mon[26841]: from='client.? 172.21.15.162:0/2638016437' entity='client.admin' 2024-04-27T04:00:13.205 INFO:teuthology.orchestra.run.smithi162.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-04-27T04:00:13.206 INFO:teuthology.orchestra.run.smithi162.stdout:systemctl: stdout disabled 2024-04-27T04:00:13.213 INFO:teuthology.orchestra.run.smithi162.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-04-27T04:00:13.213 INFO:teuthology.orchestra.run.smithi162.stdout:systemctl: stdout inactive 2024-04-27T04:00:13.213 INFO:teuthology.orchestra.run.smithi162.stdout:firewalld.service is not enabled 2024-04-27T04:00:13.214 INFO:teuthology.orchestra.run.smithi162.stdout:Not possible to enable service . firewalld.service is not available 2024-04-27T04:00:13.222 INFO:teuthology.orchestra.run.smithi162.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-04-27T04:00:13.222 INFO:teuthology.orchestra.run.smithi162.stdout:systemctl: stdout disabled 2024-04-27T04:00:13.230 INFO:teuthology.orchestra.run.smithi162.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-04-27T04:00:13.230 INFO:teuthology.orchestra.run.smithi162.stdout:systemctl: stdout inactive 2024-04-27T04:00:13.230 INFO:teuthology.orchestra.run.smithi162.stdout:firewalld.service is not enabled 2024-04-27T04:00:13.230 INFO:teuthology.orchestra.run.smithi162.stdout:Not possible to open ports <[9283]>. firewalld.service is not available 2024-04-27T04:00:13.230 INFO:teuthology.orchestra.run.smithi162.stdout:Waiting for mgr to start... 2024-04-27T04:00:13.230 INFO:teuthology.orchestra.run.smithi162.stdout:Waiting for mgr... 2024-04-27T04:00:13.961 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:00:13 smithi162 ceph-mon[26841]: from='client.? 172.21.15.162:0/1951840678' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-04-27T04:00:14.295 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stdout 2024-04-27T04:00:14.295 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stdout { 2024-04-27T04:00:14.295 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stdout "fsid": "7f9750d6-044a-11ef-bc93-c7b262605968", 2024-04-27T04:00:14.295 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stdout "health": { 2024-04-27T04:00:14.296 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-04-27T04:00:14.296 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-04-27T04:00:14.296 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-04-27T04:00:14.296 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stdout }, 2024-04-27T04:00:14.296 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-04-27T04:00:14.296 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-04-27T04:00:14.296 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stdout 0 2024-04-27T04:00:14.296 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stdout ], 2024-04-27T04:00:14.296 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-04-27T04:00:14.296 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stdout "smithi162" 2024-04-27T04:00:14.296 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stdout ], 2024-04-27T04:00:14.296 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stdout "quorum_age": 2, 2024-04-27T04:00:14.296 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stdout "monmap": { 2024-04-27T04:00:14.296 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-27T04:00:14.297 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "quincy", 2024-04-27T04:00:14.297 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-04-27T04:00:14.297 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stdout }, 2024-04-27T04:00:14.297 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-04-27T04:00:14.297 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-27T04:00:14.297 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-04-27T04:00:14.297 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-04-27T04:00:14.297 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-04-27T04:00:14.297 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-04-27T04:00:14.305 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-04-27T04:00:14.305 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-04-27T04:00:14.305 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stdout }, 2024-04-27T04:00:14.305 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-04-27T04:00:14.305 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-04-27T04:00:14.305 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-04-27T04:00:14.305 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-04-27T04:00:14.305 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-04-27T04:00:14.305 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-04-27T04:00:14.305 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-04-27T04:00:14.305 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-04-27T04:00:14.305 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-04-27T04:00:14.306 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stdout }, 2024-04-27T04:00:14.306 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-04-27T04:00:14.306 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-27T04:00:14.306 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-04-27T04:00:14.306 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-04-27T04:00:14.306 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stdout }, 2024-04-27T04:00:14.306 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-04-27T04:00:14.306 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stdout "available": false, 2024-04-27T04:00:14.306 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-04-27T04:00:14.306 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stdout "modules": [ 2024-04-27T04:00:14.306 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stdout "iostat", 2024-04-27T04:00:14.306 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stdout "nfs", 2024-04-27T04:00:14.306 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stdout "restful" 2024-04-27T04:00:14.307 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stdout ], 2024-04-27T04:00:14.307 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-27T04:00:14.307 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stdout }, 2024-04-27T04:00:14.307 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-04-27T04:00:14.307 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-27T04:00:14.307 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stdout "modified": "2024-04-27T04:00:06.344053+0000", 2024-04-27T04:00:14.307 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-27T04:00:14.307 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stdout }, 2024-04-27T04:00:14.307 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-04-27T04:00:14.307 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stdout } 2024-04-27T04:00:14.307 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:13.684+0000 7f45bf2d0700 1 Processor -- start 2024-04-27T04:00:14.307 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:13.684+0000 7f45bf2d0700 1 -- start start 2024-04-27T04:00:14.307 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:13.685+0000 7f45bf2d0700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f45b8102850 0x7f45b8102c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:00:14.307 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:13.685+0000 7f45bf2d0700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45b81031a0 con 0x7f45b8102850 2024-04-27T04:00:14.308 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:13.685+0000 7f45bd06c700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f45b8102850 0x7f45b8102c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:00:14.308 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:13.685+0000 7f45bd06c700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f45b8102850 0x7f45b8102c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:38248/0 (socket says 172.21.15.162:38248) 2024-04-27T04:00:14.308 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:13.685+0000 7f45bd06c700 1 -- 172.21.15.162:0/205998232 learned_addr learned my addr 172.21.15.162:0/205998232 (peer_addr_for_me v2:172.21.15.162:0/0) 2024-04-27T04:00:14.308 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:13.685+0000 7f45bd06c700 1 -- 172.21.15.162:0/205998232 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f45b81032e0 con 0x7f45b8102850 2024-04-27T04:00:14.308 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:13.685+0000 7f45bd06c700 1 --2- 172.21.15.162:0/205998232 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f45b8102850 0x7f45b8102c60 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f45b40098e0 tx=0x7f45b4009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e057496b83d0f3d4 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:00:14.308 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:13.685+0000 7f45affff700 1 -- 172.21.15.162:0/205998232 <== mon.0 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f45b400b860 con 0x7f45b8102850 2024-04-27T04:00:14.308 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:13.686+0000 7f45affff700 1 -- 172.21.15.162:0/205998232 <== mon.0 v2:172.21.15.162:3300/0 2 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7f45b400be50 con 0x7f45b8102850 2024-04-27T04:00:14.308 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:13.686+0000 7f45affff700 1 -- 172.21.15.162:0/205998232 <== mon.0 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f45b4010380 con 0x7f45b8102850 2024-04-27T04:00:14.308 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:13.686+0000 7f45bf2d0700 1 -- 172.21.15.162:0/205998232 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f45b8102850 msgr2=0x7f45b8102c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:00:14.308 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:13.686+0000 7f45bf2d0700 1 --2- 172.21.15.162:0/205998232 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f45b8102850 0x7f45b8102c60 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f45b40098e0 tx=0x7f45b4009250 comp rx=0 tx=0).stop 2024-04-27T04:00:14.308 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:13.686+0000 7f45bf2d0700 1 -- 172.21.15.162:0/205998232 shutdown_connections 2024-04-27T04:00:14.308 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:13.686+0000 7f45bf2d0700 1 --2- 172.21.15.162:0/205998232 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f45b8102850 0x7f45b8102c60 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:00:14.309 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:13.686+0000 7f45bf2d0700 1 -- 172.21.15.162:0/205998232 >> 172.21.15.162:0/205998232 conn(0x7f45b80fdff0 msgr2=0x7f45b8100400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:00:14.309 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:13.686+0000 7f45bf2d0700 1 -- 172.21.15.162:0/205998232 shutdown_connections 2024-04-27T04:00:14.309 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:13.686+0000 7f45bf2d0700 1 -- 172.21.15.162:0/205998232 wait complete. 2024-04-27T04:00:14.309 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:13.687+0000 7f45bf2d0700 1 Processor -- start 2024-04-27T04:00:14.309 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:13.687+0000 7f45bf2d0700 1 -- start start 2024-04-27T04:00:14.309 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:13.687+0000 7f45bf2d0700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f45b8102850 0x7f45b8077b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:00:14.309 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:13.687+0000 7f45bf2d0700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45b8078050 con 0x7f45b8102850 2024-04-27T04:00:14.309 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:13.687+0000 7f45bd06c700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f45b8102850 0x7f45b8077b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:00:14.309 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:13.687+0000 7f45bd06c700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f45b8102850 0x7f45b8077b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:38260/0 (socket says 172.21.15.162:38260) 2024-04-27T04:00:14.309 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:13.687+0000 7f45bd06c700 1 -- 172.21.15.162:0/1951840678 learned_addr learned my addr 172.21.15.162:0/1951840678 (peer_addr_for_me v2:172.21.15.162:0/0) 2024-04-27T04:00:14.309 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:13.687+0000 7f45bd06c700 1 -- 172.21.15.162:0/1951840678 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f45b40094f0 con 0x7f45b8102850 2024-04-27T04:00:14.309 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:13.688+0000 7f45bd06c700 1 --2- 172.21.15.162:0/1951840678 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f45b8102850 0x7f45b8077b10 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f45b4000c00 tx=0x7f45b4009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:00:14.309 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:13.688+0000 7f45adffb700 1 -- 172.21.15.162:0/1951840678 <== mon.0 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f45b40107c0 con 0x7f45b8102850 2024-04-27T04:00:14.309 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:13.688+0000 7f45adffb700 1 -- 172.21.15.162:0/1951840678 <== mon.0 v2:172.21.15.162:3300/0 2 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7f45b4010db0 con 0x7f45b8102850 2024-04-27T04:00:14.310 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:13.688+0000 7f45bf2d0700 1 -- 172.21.15.162:0/1951840678 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f45b80760c0 con 0x7f45b8102850 2024-04-27T04:00:14.310 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:13.688+0000 7f45adffb700 1 -- 172.21.15.162:0/1951840678 <== mon.0 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f45b401ad00 con 0x7f45b8102850 2024-04-27T04:00:14.310 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:13.688+0000 7f45bf2d0700 1 -- 172.21.15.162:0/1951840678 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f45b80765b0 con 0x7f45b8102850 2024-04-27T04:00:14.310 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:13.689+0000 7f45adffb700 1 -- 172.21.15.162:0/1951840678 <== mon.0 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 1) v1 ==== 529+0+0 (secure 0 0 0) 0x7f45b4024440 con 0x7f45b8102850 2024-04-27T04:00:14.310 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:13.689+0000 7f45adffb700 1 -- 172.21.15.162:0/1951840678 <== mon.0 v2:172.21.15.162:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 721+0+0 (secure 0 0 0) 0x7f45b4019710 con 0x7f45b8102850 2024-04-27T04:00:14.310 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:13.689+0000 7f45bf2d0700 1 -- 172.21.15.162:0/1951840678 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f45b804bed0 con 0x7f45b8102850 2024-04-27T04:00:14.310 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:13.692+0000 7f45adffb700 1 -- 172.21.15.162:0/1951840678 <== mon.0 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71547 (secure 0 0 0) 0x7f45b4042790 con 0x7f45b8102850 2024-04-27T04:00:14.310 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:13.774+0000 7f45bf2d0700 1 -- 172.21.15.162:0/1951840678 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) v1 -- 0x7f45b8104670 con 0x7f45b8102850 2024-04-27T04:00:14.310 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:13.774+0000 7f45adffb700 1 -- 172.21.15.162:0/1951840678 <== mon.0 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) v1 ==== 79+0+1248 (secure 0 0 0) 0x7f45b4031020 con 0x7f45b8102850 2024-04-27T04:00:14.310 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:13.775+0000 7f45bf2d0700 1 -- 172.21.15.162:0/1951840678 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f45b8102850 msgr2=0x7f45b8077b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:00:14.310 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:13.775+0000 7f45bf2d0700 1 --2- 172.21.15.162:0/1951840678 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f45b8102850 0x7f45b8077b10 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f45b4000c00 tx=0x7f45b4009250 comp rx=0 tx=0).stop 2024-04-27T04:00:14.310 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:13.775+0000 7f45bf2d0700 1 -- 172.21.15.162:0/1951840678 shutdown_connections 2024-04-27T04:00:14.310 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:13.775+0000 7f45bf2d0700 1 --2- 172.21.15.162:0/1951840678 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f45b8102850 0x7f45b8077b10 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:00:14.311 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:13.775+0000 7f45bf2d0700 1 -- 172.21.15.162:0/1951840678 >> 172.21.15.162:0/1951840678 conn(0x7f45b80fdff0 msgr2=0x7f45b80fea30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:00:14.311 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:13.775+0000 7f45bf2d0700 1 -- 172.21.15.162:0/1951840678 shutdown_connections 2024-04-27T04:00:14.311 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:13.775+0000 7f45bf2d0700 1 -- 172.21.15.162:0/1951840678 wait complete. 2024-04-27T04:00:14.311 INFO:teuthology.orchestra.run.smithi162.stdout:mgr not available, waiting (1/15)... 2024-04-27T04:00:17.139 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:00:16 smithi162 ceph-mon[26841]: from='client.? 172.21.15.162:0/2808456388' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-04-27T04:00:17.427 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stdout 2024-04-27T04:00:17.428 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stdout { 2024-04-27T04:00:17.428 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stdout "fsid": "7f9750d6-044a-11ef-bc93-c7b262605968", 2024-04-27T04:00:17.428 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stdout "health": { 2024-04-27T04:00:17.428 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-04-27T04:00:17.428 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-04-27T04:00:17.428 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-04-27T04:00:17.428 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stdout }, 2024-04-27T04:00:17.428 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-04-27T04:00:17.428 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-04-27T04:00:17.428 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stdout 0 2024-04-27T04:00:17.428 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stdout ], 2024-04-27T04:00:17.428 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-04-27T04:00:17.429 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stdout "smithi162" 2024-04-27T04:00:17.429 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stdout ], 2024-04-27T04:00:17.429 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stdout "quorum_age": 5, 2024-04-27T04:00:17.429 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stdout "monmap": { 2024-04-27T04:00:17.429 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-27T04:00:17.429 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "quincy", 2024-04-27T04:00:17.429 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-04-27T04:00:17.429 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stdout }, 2024-04-27T04:00:17.429 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-04-27T04:00:17.429 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-27T04:00:17.429 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-04-27T04:00:17.429 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-04-27T04:00:17.429 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-04-27T04:00:17.437 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-04-27T04:00:17.438 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-04-27T04:00:17.438 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-04-27T04:00:17.438 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stdout }, 2024-04-27T04:00:17.438 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-04-27T04:00:17.438 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-04-27T04:00:17.438 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-04-27T04:00:17.438 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-04-27T04:00:17.438 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-04-27T04:00:17.438 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-04-27T04:00:17.438 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-04-27T04:00:17.438 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-04-27T04:00:17.438 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-04-27T04:00:17.439 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stdout }, 2024-04-27T04:00:17.439 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-04-27T04:00:17.439 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-27T04:00:17.439 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-04-27T04:00:17.439 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-04-27T04:00:17.439 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stdout }, 2024-04-27T04:00:17.439 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-04-27T04:00:17.439 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stdout "available": false, 2024-04-27T04:00:17.439 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-04-27T04:00:17.439 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stdout "modules": [ 2024-04-27T04:00:17.439 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stdout "iostat", 2024-04-27T04:00:17.439 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stdout "nfs", 2024-04-27T04:00:17.439 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stdout "restful" 2024-04-27T04:00:17.439 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stdout ], 2024-04-27T04:00:17.439 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-27T04:00:17.440 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stdout }, 2024-04-27T04:00:17.440 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-04-27T04:00:17.440 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-27T04:00:17.440 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stdout "modified": "2024-04-27T04:00:06.344053+0000", 2024-04-27T04:00:17.440 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-27T04:00:17.440 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stdout }, 2024-04-27T04:00:17.440 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-04-27T04:00:17.440 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stdout } 2024-04-27T04:00:17.440 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:16.705+0000 7f9868471700 1 Processor -- start 2024-04-27T04:00:17.440 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:16.706+0000 7f9868471700 1 -- start start 2024-04-27T04:00:17.440 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:16.706+0000 7f9868471700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9860100cd0 0x7f98601010e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:00:17.440 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:16.706+0000 7f9868471700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9860101620 con 0x7f9860100cd0 2024-04-27T04:00:17.440 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:16.706+0000 7f986620d700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9860100cd0 0x7f98601010e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:00:17.440 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:16.706+0000 7f986620d700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9860100cd0 0x7f98601010e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:38270/0 (socket says 172.21.15.162:38270) 2024-04-27T04:00:17.440 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:16.706+0000 7f986620d700 1 -- 172.21.15.162:0/1512630821 learned_addr learned my addr 172.21.15.162:0/1512630821 (peer_addr_for_me v2:172.21.15.162:0/0) 2024-04-27T04:00:17.441 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:16.706+0000 7f986620d700 1 -- 172.21.15.162:0/1512630821 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9860101760 con 0x7f9860100cd0 2024-04-27T04:00:17.441 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:16.707+0000 7f986620d700 1 --2- 172.21.15.162:0/1512630821 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9860100cd0 0x7f98601010e0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f985c0098e0 tx=0x7f985c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c7d9e28dc3ed42e3 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:00:17.441 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:16.707+0000 7f986520b700 1 -- 172.21.15.162:0/1512630821 <== mon.0 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f985c00b860 con 0x7f9860100cd0 2024-04-27T04:00:17.441 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:16.707+0000 7f986520b700 1 -- 172.21.15.162:0/1512630821 <== mon.0 v2:172.21.15.162:3300/0 2 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7f985c00be50 con 0x7f9860100cd0 2024-04-27T04:00:17.441 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:16.707+0000 7f986520b700 1 -- 172.21.15.162:0/1512630821 <== mon.0 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f985c010380 con 0x7f9860100cd0 2024-04-27T04:00:17.441 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:16.707+0000 7f9868471700 1 -- 172.21.15.162:0/1512630821 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9860100cd0 msgr2=0x7f98601010e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:00:17.441 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:16.707+0000 7f9868471700 1 --2- 172.21.15.162:0/1512630821 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9860100cd0 0x7f98601010e0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f985c0098e0 tx=0x7f985c009250 comp rx=0 tx=0).stop 2024-04-27T04:00:17.441 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:16.708+0000 7f9868471700 1 -- 172.21.15.162:0/1512630821 shutdown_connections 2024-04-27T04:00:17.441 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:16.708+0000 7f9868471700 1 --2- 172.21.15.162:0/1512630821 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9860100cd0 0x7f98601010e0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:00:17.441 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:16.708+0000 7f9868471700 1 -- 172.21.15.162:0/1512630821 >> 172.21.15.162:0/1512630821 conn(0x7f98600fc6f0 msgr2=0x7f98600feb00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:00:17.441 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:16.708+0000 7f9868471700 1 -- 172.21.15.162:0/1512630821 shutdown_connections 2024-04-27T04:00:17.441 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:16.708+0000 7f9868471700 1 -- 172.21.15.162:0/1512630821 wait complete. 2024-04-27T04:00:17.441 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:16.708+0000 7f9868471700 1 Processor -- start 2024-04-27T04:00:17.441 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:16.708+0000 7f9868471700 1 -- start start 2024-04-27T04:00:17.441 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:16.708+0000 7f9868471700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9860100cd0 0x7f9860103700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:00:17.442 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:16.708+0000 7f9868471700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9860103c40 con 0x7f9860100cd0 2024-04-27T04:00:17.442 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:16.708+0000 7f986620d700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9860100cd0 0x7f9860103700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:00:17.442 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:16.708+0000 7f986620d700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9860100cd0 0x7f9860103700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:38286/0 (socket says 172.21.15.162:38286) 2024-04-27T04:00:17.442 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:16.708+0000 7f986620d700 1 -- 172.21.15.162:0/2808456388 learned_addr learned my addr 172.21.15.162:0/2808456388 (peer_addr_for_me v2:172.21.15.162:0/0) 2024-04-27T04:00:17.442 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:16.709+0000 7f986620d700 1 -- 172.21.15.162:0/2808456388 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f985c0094f0 con 0x7f9860100cd0 2024-04-27T04:00:17.442 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:16.709+0000 7f986620d700 1 --2- 172.21.15.162:0/2808456388 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9860100cd0 0x7f9860103700 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f985c000c00 tx=0x7f985c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:00:17.442 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:16.709+0000 7f9856ffd700 1 -- 172.21.15.162:0/2808456388 <== mon.0 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f985c0107c0 con 0x7f9860100cd0 2024-04-27T04:00:17.442 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:16.709+0000 7f9856ffd700 1 -- 172.21.15.162:0/2808456388 <== mon.0 v2:172.21.15.162:3300/0 2 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7f985c010db0 con 0x7f9860100cd0 2024-04-27T04:00:17.442 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:16.709+0000 7f9868471700 1 -- 172.21.15.162:0/2808456388 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9860101cb0 con 0x7f9860100cd0 2024-04-27T04:00:17.442 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:16.709+0000 7f9868471700 1 -- 172.21.15.162:0/2808456388 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f98601021a0 con 0x7f9860100cd0 2024-04-27T04:00:17.442 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:16.709+0000 7f9856ffd700 1 -- 172.21.15.162:0/2808456388 <== mon.0 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f985c01ad00 con 0x7f9860100cd0 2024-04-27T04:00:17.442 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:16.709+0000 7f9856ffd700 1 -- 172.21.15.162:0/2808456388 <== mon.0 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 1) v1 ==== 529+0+0 (secure 0 0 0) 0x7f985c024460 con 0x7f9860100cd0 2024-04-27T04:00:17.442 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:16.709+0000 7f9856ffd700 1 -- 172.21.15.162:0/2808456388 <== mon.0 v2:172.21.15.162:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 721+0+0 (secure 0 0 0) 0x7f985c019710 con 0x7f9860100cd0 2024-04-27T04:00:17.443 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:16.710+0000 7f9868471700 1 -- 172.21.15.162:0/2808456388 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f98440052e0 con 0x7f9860100cd0 2024-04-27T04:00:17.443 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:16.712+0000 7f9856ffd700 1 -- 172.21.15.162:0/2808456388 <== mon.0 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71547 (secure 0 0 0) 0x7f985c01a430 con 0x7f9860100cd0 2024-04-27T04:00:17.443 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:16.784+0000 7f9868471700 1 -- 172.21.15.162:0/2808456388 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) v1 -- 0x7f9844005860 con 0x7f9860100cd0 2024-04-27T04:00:17.443 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:16.784+0000 7f9856ffd700 1 -- 172.21.15.162:0/2808456388 <== mon.0 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) v1 ==== 79+0+1248 (secure 0 0 0) 0x7f985c01a5d0 con 0x7f9860100cd0 2024-04-27T04:00:17.443 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:16.785+0000 7f9868471700 1 -- 172.21.15.162:0/2808456388 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9860100cd0 msgr2=0x7f9860103700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:00:17.443 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:16.785+0000 7f9868471700 1 --2- 172.21.15.162:0/2808456388 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9860100cd0 0x7f9860103700 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f985c000c00 tx=0x7f985c009250 comp rx=0 tx=0).stop 2024-04-27T04:00:17.443 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:16.785+0000 7f9868471700 1 -- 172.21.15.162:0/2808456388 shutdown_connections 2024-04-27T04:00:17.443 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:16.785+0000 7f9868471700 1 --2- 172.21.15.162:0/2808456388 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9860100cd0 0x7f9860103700 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:00:17.443 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:16.785+0000 7f9868471700 1 -- 172.21.15.162:0/2808456388 >> 172.21.15.162:0/2808456388 conn(0x7f98600fc6f0 msgr2=0x7f98600fd1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:00:17.443 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:16.786+0000 7f9868471700 1 -- 172.21.15.162:0/2808456388 shutdown_connections 2024-04-27T04:00:17.443 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:16.786+0000 7f9868471700 1 -- 172.21.15.162:0/2808456388 wait complete. 2024-04-27T04:00:17.443 INFO:teuthology.orchestra.run.smithi162.stdout:mgr not available, waiting (2/15)... 2024-04-27T04:00:20.139 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:00:19 smithi162 ceph-mon[26841]: Activating manager daemon smithi162.oennfn 2024-04-27T04:00:20.139 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:00:19 smithi162 ceph-mon[26841]: mgrmap e2: smithi162.oennfn(active, starting, since 0.00290608s) 2024-04-27T04:00:20.139 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:00:19 smithi162 ceph-mon[26841]: from='mgr.14100 172.21.15.162:0/1338173847' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-04-27T04:00:20.139 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:00:19 smithi162 ceph-mon[26841]: from='mgr.14100 172.21.15.162:0/1338173847' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-04-27T04:00:20.139 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:00:19 smithi162 ceph-mon[26841]: from='mgr.14100 172.21.15.162:0/1338173847' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-04-27T04:00:20.139 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:00:19 smithi162 ceph-mon[26841]: from='mgr.14100 172.21.15.162:0/1338173847' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "mon metadata", "id": "smithi162"}]: dispatch 2024-04-27T04:00:20.140 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:00:19 smithi162 ceph-mon[26841]: from='mgr.14100 172.21.15.162:0/1338173847' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "mgr metadata", "who": "smithi162.oennfn", "id": "smithi162.oennfn"}]: dispatch 2024-04-27T04:00:20.140 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:00:19 smithi162 ceph-mon[26841]: Manager daemon smithi162.oennfn is now available 2024-04-27T04:00:20.140 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:00:19 smithi162 ceph-mon[26841]: from='mgr.14100 172.21.15.162:0/1338173847' entity='mgr.smithi162.oennfn' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi162.oennfn/mirror_snapshot_schedule"}]: dispatch 2024-04-27T04:00:20.140 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:00:19 smithi162 ceph-mon[26841]: from='mgr.14100 172.21.15.162:0/1338173847' entity='mgr.smithi162.oennfn' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi162.oennfn/trash_purge_schedule"}]: dispatch 2024-04-27T04:00:20.140 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:00:19 smithi162 ceph-mon[26841]: from='mgr.14100 172.21.15.162:0/1338173847' entity='mgr.smithi162.oennfn' 2024-04-27T04:00:20.140 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:00:19 smithi162 ceph-mon[26841]: from='mgr.14100 172.21.15.162:0/1338173847' entity='mgr.smithi162.oennfn' 2024-04-27T04:00:20.140 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:00:19 smithi162 ceph-mon[26841]: from='mgr.14100 172.21.15.162:0/1338173847' entity='mgr.smithi162.oennfn' 2024-04-27T04:00:20.501 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stdout 2024-04-27T04:00:20.501 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stdout { 2024-04-27T04:00:20.501 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stdout "fsid": "7f9750d6-044a-11ef-bc93-c7b262605968", 2024-04-27T04:00:20.501 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stdout "health": { 2024-04-27T04:00:20.501 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-04-27T04:00:20.501 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-04-27T04:00:20.501 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-04-27T04:00:20.501 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stdout }, 2024-04-27T04:00:20.501 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-04-27T04:00:20.502 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-04-27T04:00:20.502 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stdout 0 2024-04-27T04:00:20.502 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stdout ], 2024-04-27T04:00:20.502 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-04-27T04:00:20.502 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stdout "smithi162" 2024-04-27T04:00:20.502 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stdout ], 2024-04-27T04:00:20.502 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stdout "quorum_age": 8, 2024-04-27T04:00:20.502 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stdout "monmap": { 2024-04-27T04:00:20.502 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-27T04:00:20.502 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "quincy", 2024-04-27T04:00:20.502 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-04-27T04:00:20.502 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stdout }, 2024-04-27T04:00:20.502 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-04-27T04:00:20.502 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-27T04:00:20.503 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-04-27T04:00:20.503 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-04-27T04:00:20.503 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-04-27T04:00:20.511 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-04-27T04:00:20.511 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-04-27T04:00:20.511 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-04-27T04:00:20.511 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stdout }, 2024-04-27T04:00:20.511 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-04-27T04:00:20.511 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-04-27T04:00:20.511 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-04-27T04:00:20.511 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-04-27T04:00:20.511 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-04-27T04:00:20.511 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-04-27T04:00:20.511 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-04-27T04:00:20.511 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-04-27T04:00:20.511 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-04-27T04:00:20.512 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stdout }, 2024-04-27T04:00:20.512 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-04-27T04:00:20.512 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-27T04:00:20.512 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-04-27T04:00:20.512 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-04-27T04:00:20.512 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stdout }, 2024-04-27T04:00:20.512 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-04-27T04:00:20.512 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stdout "available": false, 2024-04-27T04:00:20.512 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-04-27T04:00:20.512 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stdout "modules": [ 2024-04-27T04:00:20.512 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stdout "iostat", 2024-04-27T04:00:20.512 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stdout "nfs", 2024-04-27T04:00:20.512 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stdout "restful" 2024-04-27T04:00:20.512 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stdout ], 2024-04-27T04:00:20.513 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-27T04:00:20.513 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stdout }, 2024-04-27T04:00:20.513 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-04-27T04:00:20.513 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-27T04:00:20.513 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stdout "modified": "2024-04-27T04:00:06.344053+0000", 2024-04-27T04:00:20.513 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-27T04:00:20.513 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stdout }, 2024-04-27T04:00:20.513 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-04-27T04:00:20.513 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stdout } 2024-04-27T04:00:20.513 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:19.817+0000 7f203d475700 1 Processor -- start 2024-04-27T04:00:20.513 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:19.817+0000 7f203d475700 1 -- start start 2024-04-27T04:00:20.513 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:19.817+0000 7f203d475700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2038076b70 0x7f20381054e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:00:20.513 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:19.817+0000 7f203d475700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2038105a20 con 0x7f2038076b70 2024-04-27T04:00:20.514 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:19.817+0000 7f2036ffd700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2038076b70 0x7f20381054e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:00:20.514 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:19.817+0000 7f2036ffd700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2038076b70 0x7f20381054e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:38374/0 (socket says 172.21.15.162:38374) 2024-04-27T04:00:20.514 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:19.817+0000 7f2036ffd700 1 -- 172.21.15.162:0/4120178072 learned_addr learned my addr 172.21.15.162:0/4120178072 (peer_addr_for_me v2:172.21.15.162:0/0) 2024-04-27T04:00:20.514 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:19.818+0000 7f2036ffd700 1 -- 172.21.15.162:0/4120178072 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2038105b60 con 0x7f2038076b70 2024-04-27T04:00:20.514 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:19.818+0000 7f2036ffd700 1 --2- 172.21.15.162:0/4120178072 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2038076b70 0x7f20381054e0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f20280098e0 tx=0x7f2028009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a3413796904ec9a7 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:00:20.514 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:19.818+0000 7f2035ffb700 1 -- 172.21.15.162:0/4120178072 <== mon.0 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f202800b860 con 0x7f2038076b70 2024-04-27T04:00:20.514 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:19.818+0000 7f2035ffb700 1 -- 172.21.15.162:0/4120178072 <== mon.0 v2:172.21.15.162:3300/0 2 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7f202800be50 con 0x7f2038076b70 2024-04-27T04:00:20.514 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:19.818+0000 7f2035ffb700 1 -- 172.21.15.162:0/4120178072 <== mon.0 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f2028010380 con 0x7f2038076b70 2024-04-27T04:00:20.514 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:19.818+0000 7f203d475700 1 -- 172.21.15.162:0/4120178072 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2038076b70 msgr2=0x7f20381054e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:00:20.514 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:19.818+0000 7f203d475700 1 --2- 172.21.15.162:0/4120178072 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2038076b70 0x7f20381054e0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f20280098e0 tx=0x7f2028009250 comp rx=0 tx=0).stop 2024-04-27T04:00:20.514 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:19.818+0000 7f203d475700 1 -- 172.21.15.162:0/4120178072 shutdown_connections 2024-04-27T04:00:20.514 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:19.818+0000 7f203d475700 1 --2- 172.21.15.162:0/4120178072 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2038076b70 0x7f20381054e0 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:00:20.514 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:19.818+0000 7f203d475700 1 -- 172.21.15.162:0/4120178072 >> 172.21.15.162:0/4120178072 conn(0x7f20380fcef0 msgr2=0x7f20380ff300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:00:20.514 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:19.819+0000 7f203d475700 1 -- 172.21.15.162:0/4120178072 shutdown_connections 2024-04-27T04:00:20.515 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:19.819+0000 7f203d475700 1 -- 172.21.15.162:0/4120178072 wait complete. 2024-04-27T04:00:20.515 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:19.819+0000 7f203d475700 1 Processor -- start 2024-04-27T04:00:20.515 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:19.819+0000 7f203d475700 1 -- start start 2024-04-27T04:00:20.515 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:19.819+0000 7f203d475700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2038076b70 0x7f2038102e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:00:20.515 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:19.819+0000 7f203d475700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2038103370 con 0x7f2038076b70 2024-04-27T04:00:20.515 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:19.819+0000 7f2036ffd700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2038076b70 0x7f2038102e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:00:20.515 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:19.819+0000 7f2036ffd700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2038076b70 0x7f2038102e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:38376/0 (socket says 172.21.15.162:38376) 2024-04-27T04:00:20.515 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:19.819+0000 7f2036ffd700 1 -- 172.21.15.162:0/1180317648 learned_addr learned my addr 172.21.15.162:0/1180317648 (peer_addr_for_me v2:172.21.15.162:0/0) 2024-04-27T04:00:20.515 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:19.820+0000 7f2036ffd700 1 -- 172.21.15.162:0/1180317648 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20280094f0 con 0x7f2038076b70 2024-04-27T04:00:20.515 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:19.820+0000 7f2036ffd700 1 --2- 172.21.15.162:0/1180317648 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2038076b70 0x7f2038102e30 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f2028006b00 tx=0x7f2028009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:00:20.515 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:19.820+0000 7f201f7fe700 1 -- 172.21.15.162:0/1180317648 <== mon.0 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f20280107c0 con 0x7f2038076b70 2024-04-27T04:00:20.515 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:19.820+0000 7f201f7fe700 1 -- 172.21.15.162:0/1180317648 <== mon.0 v2:172.21.15.162:3300/0 2 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7f2028010db0 con 0x7f2038076b70 2024-04-27T04:00:20.515 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:19.820+0000 7f203d475700 1 -- 172.21.15.162:0/1180317648 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f20381013e0 con 0x7f2038076b70 2024-04-27T04:00:20.516 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:19.820+0000 7f201f7fe700 1 -- 172.21.15.162:0/1180317648 <== mon.0 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f202801acf0 con 0x7f2038076b70 2024-04-27T04:00:20.516 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:19.820+0000 7f203d475700 1 -- 172.21.15.162:0/1180317648 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f20381018d0 con 0x7f2038076b70 2024-04-27T04:00:20.516 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:19.821+0000 7f201f7fe700 1 -- 172.21.15.162:0/1180317648 <== mon.0 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 2) v1 ==== 43132+0+0 (secure 0 0 0) 0x7f20280193c0 con 0x7f2038076b70 2024-04-27T04:00:20.516 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:19.821+0000 7f201f7fe700 1 -- 172.21.15.162:0/1180317648 <== mon.0 v2:172.21.15.162:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 721+0+0 (secure 0 0 0) 0x7f202804b040 con 0x7f2038076b70 2024-04-27T04:00:20.516 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:19.821+0000 7f203d475700 1 -- 172.21.15.162:0/1180317648 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f203804bed0 con 0x7f2038076b70 2024-04-27T04:00:20.516 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:19.823+0000 7f201f7fe700 1 -- 172.21.15.162:0/1180317648 <== mon.0 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71547 (secure 0 0 0) 0x7f2028019670 con 0x7f2038076b70 2024-04-27T04:00:20.516 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:19.892+0000 7f203d475700 1 -- 172.21.15.162:0/1180317648 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) v1 -- 0x7f2038101ef0 con 0x7f2038076b70 2024-04-27T04:00:20.516 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:19.892+0000 7f201f7fe700 1 -- 172.21.15.162:0/1180317648 <== mon.0 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) v1 ==== 79+0+1248 (secure 0 0 0) 0x7f2028022340 con 0x7f2038076b70 2024-04-27T04:00:20.516 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:19.893+0000 7f203d475700 1 -- 172.21.15.162:0/1180317648 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2038076b70 msgr2=0x7f2038102e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:00:20.516 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:19.893+0000 7f203d475700 1 --2- 172.21.15.162:0/1180317648 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2038076b70 0x7f2038102e30 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f2028006b00 tx=0x7f2028009250 comp rx=0 tx=0).stop 2024-04-27T04:00:20.516 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:19.893+0000 7f203d475700 1 -- 172.21.15.162:0/1180317648 shutdown_connections 2024-04-27T04:00:20.516 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:19.893+0000 7f203d475700 1 --2- 172.21.15.162:0/1180317648 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2038076b70 0x7f2038102e30 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:00:20.516 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:19.893+0000 7f203d475700 1 -- 172.21.15.162:0/1180317648 >> 172.21.15.162:0/1180317648 conn(0x7f20380fcef0 msgr2=0x7f20380fd9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:00:20.516 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:19.893+0000 7f203d475700 1 -- 172.21.15.162:0/1180317648 shutdown_connections 2024-04-27T04:00:20.517 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:19.893+0000 7f203d475700 1 -- 172.21.15.162:0/1180317648 wait complete. 2024-04-27T04:00:20.517 INFO:teuthology.orchestra.run.smithi162.stdout:mgr not available, waiting (3/15)... 2024-04-27T04:00:20.889 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:00:20 smithi162 ceph-mon[26841]: from='client.? 172.21.15.162:0/1180317648' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-04-27T04:00:20.889 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:00:20 smithi162 ceph-mon[26841]: mgrmap e3: smithi162.oennfn(active, since 1.00657s) 2024-04-27T04:00:22.889 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:00:22 smithi162 ceph-mon[26841]: mgrmap e4: smithi162.oennfn(active, since 2s) 2024-04-27T04:00:26.384 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:00:25 smithi162 ceph-mon[26841]: from='client.? 172.21.15.162:0/1857730161' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-04-27T04:00:32.307 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stdout 2024-04-27T04:00:32.307 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stdout { 2024-04-27T04:00:32.307 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stdout "fsid": "7f9750d6-044a-11ef-bc93-c7b262605968", 2024-04-27T04:00:32.308 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stdout "health": { 2024-04-27T04:00:32.308 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-04-27T04:00:32.308 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-04-27T04:00:32.308 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-04-27T04:00:32.308 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stdout }, 2024-04-27T04:00:32.308 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-04-27T04:00:32.308 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-04-27T04:00:32.308 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stdout 0 2024-04-27T04:00:32.308 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stdout ], 2024-04-27T04:00:32.308 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-04-27T04:00:32.308 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stdout "smithi162" 2024-04-27T04:00:32.308 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stdout ], 2024-04-27T04:00:32.308 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stdout "quorum_age": 14, 2024-04-27T04:00:32.308 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stdout "monmap": { 2024-04-27T04:00:32.309 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-27T04:00:32.309 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "quincy", 2024-04-27T04:00:32.309 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-04-27T04:00:32.309 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stdout }, 2024-04-27T04:00:32.309 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-04-27T04:00:32.309 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-27T04:00:32.309 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-04-27T04:00:32.309 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-04-27T04:00:32.309 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-04-27T04:00:32.309 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-04-27T04:00:32.309 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-04-27T04:00:32.310 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-04-27T04:00:32.310 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stdout }, 2024-04-27T04:00:32.310 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-04-27T04:00:32.310 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-04-27T04:00:32.310 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-04-27T04:00:32.310 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-04-27T04:00:32.310 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-04-27T04:00:32.310 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-04-27T04:00:32.310 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-04-27T04:00:32.310 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-04-27T04:00:32.310 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-04-27T04:00:32.310 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stdout }, 2024-04-27T04:00:32.310 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-04-27T04:00:32.310 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-27T04:00:32.310 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-04-27T04:00:32.311 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-04-27T04:00:32.311 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stdout }, 2024-04-27T04:00:32.311 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-04-27T04:00:32.311 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stdout "available": true, 2024-04-27T04:00:32.311 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-04-27T04:00:32.311 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stdout "modules": [ 2024-04-27T04:00:32.311 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stdout "iostat", 2024-04-27T04:00:32.311 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stdout "nfs", 2024-04-27T04:00:32.311 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stdout "restful" 2024-04-27T04:00:32.311 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stdout ], 2024-04-27T04:00:32.311 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-27T04:00:32.311 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stdout }, 2024-04-27T04:00:32.311 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-04-27T04:00:32.311 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-27T04:00:32.311 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stdout "modified": "2024-04-27T04:00:06.344053+0000", 2024-04-27T04:00:32.312 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-27T04:00:32.312 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stdout }, 2024-04-27T04:00:32.312 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-04-27T04:00:32.312 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stdout } 2024-04-27T04:00:32.312 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:25.618+0000 7ff81014d700 1 Processor -- start 2024-04-27T04:00:32.312 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:25.619+0000 7ff81014d700 1 -- start start 2024-04-27T04:00:32.312 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:25.619+0000 7ff81014d700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff808076b70 0x7ff8081054c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:00:32.312 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:25.619+0000 7ff81014d700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff808105a00 con 0x7ff808076b70 2024-04-27T04:00:32.312 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:25.619+0000 7ff80dee9700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff808076b70 0x7ff8081054c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:00:32.312 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:25.619+0000 7ff80dee9700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff808076b70 0x7ff8081054c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:36732/0 (socket says 172.21.15.162:36732) 2024-04-27T04:00:32.312 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:25.619+0000 7ff80dee9700 1 -- 172.21.15.162:0/1986165829 learned_addr learned my addr 172.21.15.162:0/1986165829 (peer_addr_for_me v2:172.21.15.162:0/0) 2024-04-27T04:00:32.312 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:25.619+0000 7ff80dee9700 1 -- 172.21.15.162:0/1986165829 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff808105b40 con 0x7ff808076b70 2024-04-27T04:00:32.312 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:25.620+0000 7ff80dee9700 1 --2- 172.21.15.162:0/1986165829 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff808076b70 0x7ff8081054c0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7ff8040098e0 tx=0x7ff804009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a47128877075c389 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:00:32.313 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:25.620+0000 7ff80cee7700 1 -- 172.21.15.162:0/1986165829 <== mon.0 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7ff80400b860 con 0x7ff808076b70 2024-04-27T04:00:32.313 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:25.620+0000 7ff80cee7700 1 -- 172.21.15.162:0/1986165829 <== mon.0 v2:172.21.15.162:3300/0 2 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7ff80400be50 con 0x7ff808076b70 2024-04-27T04:00:32.313 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:25.620+0000 7ff80cee7700 1 -- 172.21.15.162:0/1986165829 <== mon.0 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7ff804010380 con 0x7ff808076b70 2024-04-27T04:00:32.313 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:25.620+0000 7ff81014d700 1 -- 172.21.15.162:0/1986165829 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff808076b70 msgr2=0x7ff8081054c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:00:32.313 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:25.620+0000 7ff81014d700 1 --2- 172.21.15.162:0/1986165829 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff808076b70 0x7ff8081054c0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7ff8040098e0 tx=0x7ff804009250 comp rx=0 tx=0).stop 2024-04-27T04:00:32.313 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:25.620+0000 7ff81014d700 1 -- 172.21.15.162:0/1986165829 shutdown_connections 2024-04-27T04:00:32.313 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:25.620+0000 7ff81014d700 1 --2- 172.21.15.162:0/1986165829 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff808076b70 0x7ff8081054c0 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:00:32.313 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:25.620+0000 7ff81014d700 1 -- 172.21.15.162:0/1986165829 >> 172.21.15.162:0/1986165829 conn(0x7ff8080fced0 msgr2=0x7ff8080ff2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:00:32.313 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:25.620+0000 7ff81014d700 1 -- 172.21.15.162:0/1986165829 shutdown_connections 2024-04-27T04:00:32.313 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:25.621+0000 7ff81014d700 1 -- 172.21.15.162:0/1986165829 wait complete. 2024-04-27T04:00:32.313 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:25.621+0000 7ff81014d700 1 Processor -- start 2024-04-27T04:00:32.313 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:25.621+0000 7ff81014d700 1 -- start start 2024-04-27T04:00:32.313 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:25.621+0000 7ff81014d700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff808076b70 0x7ff8081115d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:00:32.313 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:25.621+0000 7ff81014d700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff808111b10 con 0x7ff808076b70 2024-04-27T04:00:32.313 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:25.621+0000 7ff80dee9700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff808076b70 0x7ff8081115d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:00:32.314 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:25.621+0000 7ff80dee9700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff808076b70 0x7ff8081115d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:36734/0 (socket says 172.21.15.162:36734) 2024-04-27T04:00:32.314 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:25.621+0000 7ff80dee9700 1 -- 172.21.15.162:0/1857730161 learned_addr learned my addr 172.21.15.162:0/1857730161 (peer_addr_for_me v2:172.21.15.162:0/0) 2024-04-27T04:00:32.314 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:25.622+0000 7ff80dee9700 1 -- 172.21.15.162:0/1857730161 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff8040094f0 con 0x7ff808076b70 2024-04-27T04:00:32.314 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:25.622+0000 7ff80dee9700 1 --2- 172.21.15.162:0/1857730161 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff808076b70 0x7ff8081115d0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7ff804003050 tx=0x7ff804009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:00:32.314 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:25.622+0000 7ff7fe7fc700 1 -- 172.21.15.162:0/1857730161 <== mon.0 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7ff804010760 con 0x7ff808076b70 2024-04-27T04:00:32.314 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:25.622+0000 7ff7fe7fc700 1 -- 172.21.15.162:0/1857730161 <== mon.0 v2:172.21.15.162:3300/0 2 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7ff804010d50 con 0x7ff808076b70 2024-04-27T04:00:32.314 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:25.622+0000 7ff81014d700 1 -- 172.21.15.162:0/1857730161 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff808111d10 con 0x7ff808076b70 2024-04-27T04:00:32.314 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:25.622+0000 7ff7fe7fc700 1 -- 172.21.15.162:0/1857730161 <== mon.0 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7ff80401ae20 con 0x7ff808076b70 2024-04-27T04:00:32.314 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:25.622+0000 7ff81014d700 1 -- 172.21.15.162:0/1857730161 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff808112880 con 0x7ff808076b70 2024-04-27T04:00:32.314 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:25.623+0000 7ff7fe7fc700 1 -- 172.21.15.162:0/1857730161 <== mon.0 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 4) v1 ==== 43365+0+0 (secure 0 0 0) 0x7ff80402e440 con 0x7ff808076b70 2024-04-27T04:00:32.314 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:25.623+0000 7ff81014d700 1 -- 172.21.15.162:0/1857730161 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff80805f5c0 con 0x7ff808076b70 2024-04-27T04:00:32.314 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:25.623+0000 7ff7fe7fc700 1 --2- 172.21.15.162:0/1857730161 >> [v2:172.21.15.162:6800/1480340393,v1:172.21.15.162:6801/1480340393] conn(0x7ff7f40367e0 0x7ff7f4038c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:00:32.315 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:25.623+0000 7ff7fe7fc700 1 -- 172.21.15.162:0/1857730161 <== mon.0 v2:172.21.15.162:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 721+0+0 (secure 0 0 0) 0x7ff80404a4f0 con 0x7ff808076b70 2024-04-27T04:00:32.315 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:25.623+0000 7ff80d6e8700 1 --2- 172.21.15.162:0/1857730161 >> [v2:172.21.15.162:6800/1480340393,v1:172.21.15.162:6801/1480340393] conn(0x7ff7f40367e0 0x7ff7f4038c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:00:32.315 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:25.623+0000 7ff80d6e8700 1 --2- 172.21.15.162:0/1857730161 >> [v2:172.21.15.162:6800/1480340393,v1:172.21.15.162:6801/1480340393] conn(0x7ff7f40367e0 0x7ff7f4038c90 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7ff7f8009990 tx=0x7ff7f8006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:00:32.315 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:25.628+0000 7ff7fe7fc700 1 -- 172.21.15.162:0/1857730161 <== mon.0 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7ff804018760 con 0x7ff808076b70 2024-04-27T04:00:32.315 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:25.916+0000 7ff81014d700 1 -- 172.21.15.162:0/1857730161 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) v1 -- 0x7ff80804bed0 con 0x7ff808076b70 2024-04-27T04:00:32.315 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:25.917+0000 7ff7fe7fc700 1 -- 172.21.15.162:0/1857730161 <== mon.0 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) v1 ==== 79+0+1248 (secure 0 0 0) 0x7ff80402d3a0 con 0x7ff808076b70 2024-04-27T04:00:32.315 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:25.920+0000 7ff81014d700 1 -- 172.21.15.162:0/1857730161 >> [v2:172.21.15.162:6800/1480340393,v1:172.21.15.162:6801/1480340393] conn(0x7ff7f40367e0 msgr2=0x7ff7f4038c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:00:32.315 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:25.920+0000 7ff81014d700 1 --2- 172.21.15.162:0/1857730161 >> [v2:172.21.15.162:6800/1480340393,v1:172.21.15.162:6801/1480340393] conn(0x7ff7f40367e0 0x7ff7f4038c90 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7ff7f8009990 tx=0x7ff7f8006e40 comp rx=0 tx=0).stop 2024-04-27T04:00:32.315 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:25.920+0000 7ff81014d700 1 -- 172.21.15.162:0/1857730161 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff808076b70 msgr2=0x7ff8081115d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:00:32.315 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:25.920+0000 7ff81014d700 1 --2- 172.21.15.162:0/1857730161 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff808076b70 0x7ff8081115d0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7ff804003050 tx=0x7ff804009250 comp rx=0 tx=0).stop 2024-04-27T04:00:32.315 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:25.920+0000 7ff81014d700 1 -- 172.21.15.162:0/1857730161 shutdown_connections 2024-04-27T04:00:32.315 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:25.920+0000 7ff81014d700 1 --2- 172.21.15.162:0/1857730161 >> [v2:172.21.15.162:6800/1480340393,v1:172.21.15.162:6801/1480340393] conn(0x7ff7f40367e0 0x7ff7f4038c90 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:00:32.315 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:25.920+0000 7ff81014d700 1 --2- 172.21.15.162:0/1857730161 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff808076b70 0x7ff8081115d0 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:00:32.315 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:25.920+0000 7ff81014d700 1 -- 172.21.15.162:0/1857730161 >> 172.21.15.162:0/1857730161 conn(0x7ff8080fced0 msgr2=0x7ff8080fd9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:00:32.316 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:25.920+0000 7ff81014d700 1 -- 172.21.15.162:0/1857730161 shutdown_connections 2024-04-27T04:00:32.316 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:25.920+0000 7ff81014d700 1 -- 172.21.15.162:0/1857730161 wait complete. 2024-04-27T04:00:32.316 INFO:teuthology.orchestra.run.smithi162.stdout:mgr is available 2024-04-27T04:00:33.389 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:00:32 smithi162 ceph-mon[26841]: from='client.? 172.21.15.162:0/4138929118' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-04-27T04:00:33.465 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stdout 2024-04-27T04:00:33.466 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stdout [global] 2024-04-27T04:00:33.466 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stdout fsid = 7f9750d6-044a-11ef-bc93-c7b262605968 2024-04-27T04:00:33.466 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-04-27T04:00:33.466 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-04-27T04:00:33.466 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-04-27T04:00:33.466 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-04-27T04:00:33.466 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stdout 2024-04-27T04:00:33.466 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stdout [mgr] 2024-04-27T04:00:33.466 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-04-27T04:00:33.466 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stdout 2024-04-27T04:00:33.466 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stdout [osd] 2024-04-27T04:00:33.467 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-04-27T04:00:33.467 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-04-27T04:00:33.467 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-04-27T04:00:33.467 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-04-27T04:00:33.467 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-04-27T04:00:33.467 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:32.739+0000 7f0c90106700 1 Processor -- start 2024-04-27T04:00:33.467 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:32.739+0000 7f0c90106700 1 -- start start 2024-04-27T04:00:33.467 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:32.739+0000 7f0c90106700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0c88076c10 0x7f0c88105440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:00:33.467 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:32.739+0000 7f0c90106700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c88105980 con 0x7f0c88076c10 2024-04-27T04:00:33.467 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:32.740+0000 7f0c8dea2700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0c88076c10 0x7f0c88105440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:00:33.467 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:32.740+0000 7f0c8dea2700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0c88076c10 0x7f0c88105440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:36748/0 (socket says 172.21.15.162:36748) 2024-04-27T04:00:33.467 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:32.740+0000 7f0c8dea2700 1 -- 172.21.15.162:0/1278233159 learned_addr learned my addr 172.21.15.162:0/1278233159 (peer_addr_for_me v2:172.21.15.162:0/0) 2024-04-27T04:00:33.467 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:32.740+0000 7f0c8dea2700 1 -- 172.21.15.162:0/1278233159 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c88105ac0 con 0x7f0c88076c10 2024-04-27T04:00:33.467 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:32.740+0000 7f0c8dea2700 1 --2- 172.21.15.162:0/1278233159 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0c88076c10 0x7f0c88105440 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f0c80009e50 tx=0x7f0c80009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9ff0fa928be41719 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:00:33.468 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:32.740+0000 7f0c8cea0700 1 -- 172.21.15.162:0/1278233159 <== mon.0 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f0c800092e0 con 0x7f0c88076c10 2024-04-27T04:00:33.468 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:32.740+0000 7f0c8cea0700 1 -- 172.21.15.162:0/1278233159 <== mon.0 v2:172.21.15.162:3300/0 2 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7f0c80012430 con 0x7f0c88076c10 2024-04-27T04:00:33.468 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:32.740+0000 7f0c8cea0700 1 -- 172.21.15.162:0/1278233159 <== mon.0 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f0c800075c0 con 0x7f0c88076c10 2024-04-27T04:00:33.468 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:32.741+0000 7f0c90106700 1 -- 172.21.15.162:0/1278233159 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0c88076c10 msgr2=0x7f0c88105440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:00:33.468 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:32.741+0000 7f0c90106700 1 --2- 172.21.15.162:0/1278233159 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0c88076c10 0x7f0c88105440 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f0c80009e50 tx=0x7f0c80009840 comp rx=0 tx=0).stop 2024-04-27T04:00:33.468 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:32.741+0000 7f0c90106700 1 -- 172.21.15.162:0/1278233159 shutdown_connections 2024-04-27T04:00:33.468 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:32.741+0000 7f0c90106700 1 --2- 172.21.15.162:0/1278233159 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0c88076c10 0x7f0c88105440 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:00:33.468 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:32.741+0000 7f0c90106700 1 -- 172.21.15.162:0/1278233159 >> 172.21.15.162:0/1278233159 conn(0x7f0c880fcef0 msgr2=0x7f0c880ff300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:00:33.468 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:32.741+0000 7f0c90106700 1 -- 172.21.15.162:0/1278233159 shutdown_connections 2024-04-27T04:00:33.468 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:32.741+0000 7f0c90106700 1 -- 172.21.15.162:0/1278233159 wait complete. 2024-04-27T04:00:33.468 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:32.741+0000 7f0c90106700 1 Processor -- start 2024-04-27T04:00:33.468 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:32.741+0000 7f0c90106700 1 -- start start 2024-04-27T04:00:33.468 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:32.742+0000 7f0c90106700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0c88076c10 0x7f0c88102d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:00:33.468 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:32.742+0000 7f0c90106700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c881032d0 con 0x7f0c88076c10 2024-04-27T04:00:33.469 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:32.742+0000 7f0c8dea2700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0c88076c10 0x7f0c88102d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:00:33.469 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:32.742+0000 7f0c8dea2700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0c88076c10 0x7f0c88102d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:36750/0 (socket says 172.21.15.162:36750) 2024-04-27T04:00:33.469 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:32.742+0000 7f0c8dea2700 1 -- 172.21.15.162:0/4138929118 learned_addr learned my addr 172.21.15.162:0/4138929118 (peer_addr_for_me v2:172.21.15.162:0/0) 2024-04-27T04:00:33.469 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:32.742+0000 7f0c8dea2700 1 -- 172.21.15.162:0/4138929118 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c80009ae0 con 0x7f0c88076c10 2024-04-27T04:00:33.469 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:32.742+0000 7f0c8dea2700 1 --2- 172.21.15.162:0/4138929118 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0c88076c10 0x7f0c88102d90 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f0c80009fd0 tx=0x7f0c80009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:00:33.469 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:32.742+0000 7f0c867fc700 1 -- 172.21.15.162:0/4138929118 <== mon.0 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f0c800092e0 con 0x7f0c88076c10 2024-04-27T04:00:33.469 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:32.742+0000 7f0c90106700 1 -- 172.21.15.162:0/4138929118 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0c881013a0 con 0x7f0c88076c10 2024-04-27T04:00:33.469 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:32.742+0000 7f0c867fc700 1 -- 172.21.15.162:0/4138929118 <== mon.0 v2:172.21.15.162:3300/0 2 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7f0c80007d10 con 0x7f0c88076c10 2024-04-27T04:00:33.469 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:32.742+0000 7f0c867fc700 1 -- 172.21.15.162:0/4138929118 <== mon.0 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f0c8001c930 con 0x7f0c88076c10 2024-04-27T04:00:33.469 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:32.742+0000 7f0c90106700 1 -- 172.21.15.162:0/4138929118 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0c88101890 con 0x7f0c88076c10 2024-04-27T04:00:33.469 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:32.743+0000 7f0c867fc700 1 -- 172.21.15.162:0/4138929118 <== mon.0 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 4) v1 ==== 43365+0+0 (secure 0 0 0) 0x7f0c80019070 con 0x7f0c88076c10 2024-04-27T04:00:33.469 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:32.743+0000 7f0c867fc700 1 --2- 172.21.15.162:0/4138929118 >> [v2:172.21.15.162:6800/1480340393,v1:172.21.15.162:6801/1480340393] conn(0x7f0c70036800 0x7f0c70038cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:00:33.469 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:32.743+0000 7f0c867fc700 1 -- 172.21.15.162:0/4138929118 <== mon.0 v2:172.21.15.162:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 721+0+0 (secure 0 0 0) 0x7f0c80049d50 con 0x7f0c88076c10 2024-04-27T04:00:33.470 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:32.743+0000 7f0c8d6a1700 1 --2- 172.21.15.162:0/4138929118 >> [v2:172.21.15.162:6800/1480340393,v1:172.21.15.162:6801/1480340393] conn(0x7f0c70036800 0x7f0c70038cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:00:33.470 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:32.743+0000 7f0c90106700 1 -- 172.21.15.162:0/4138929118 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0c8804bed0 con 0x7f0c88076c10 2024-04-27T04:00:33.470 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:32.744+0000 7f0c8d6a1700 1 --2- 172.21.15.162:0/4138929118 >> [v2:172.21.15.162:6800/1480340393,v1:172.21.15.162:6801/1480340393] conn(0x7f0c70036800 0x7f0c70038cb0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f0c7c009990 tx=0x7f0c7c006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:00:33.470 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:32.748+0000 7f0c867fc700 1 -- 172.21.15.162:0/4138929118 <== mon.0 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f0c80017080 con 0x7f0c88076c10 2024-04-27T04:00:33.470 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:32.935+0000 7f0c90106700 1 -- 172.21.15.162:0/4138929118 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) v1 -- 0x7f0c88101c90 con 0x7f0c88076c10 2024-04-27T04:00:33.470 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:32.936+0000 7f0c867fc700 1 -- 172.21.15.162:0/4138929118 <== mon.0 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v3) v1 ==== 70+0+373 (secure 0 0 0) 0x7f0c80020020 con 0x7f0c88076c10 2024-04-27T04:00:33.470 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:32.939+0000 7f0c90106700 1 -- 172.21.15.162:0/4138929118 >> [v2:172.21.15.162:6800/1480340393,v1:172.21.15.162:6801/1480340393] conn(0x7f0c70036800 msgr2=0x7f0c70038cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:00:33.470 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:32.939+0000 7f0c90106700 1 --2- 172.21.15.162:0/4138929118 >> [v2:172.21.15.162:6800/1480340393,v1:172.21.15.162:6801/1480340393] conn(0x7f0c70036800 0x7f0c70038cb0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f0c7c009990 tx=0x7f0c7c006e40 comp rx=0 tx=0).stop 2024-04-27T04:00:33.470 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:32.939+0000 7f0c90106700 1 -- 172.21.15.162:0/4138929118 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0c88076c10 msgr2=0x7f0c88102d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:00:33.470 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:32.939+0000 7f0c90106700 1 --2- 172.21.15.162:0/4138929118 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0c88076c10 0x7f0c88102d90 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f0c80009fd0 tx=0x7f0c80009840 comp rx=0 tx=0).stop 2024-04-27T04:00:33.470 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:32.939+0000 7f0c90106700 1 -- 172.21.15.162:0/4138929118 shutdown_connections 2024-04-27T04:00:33.470 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:32.939+0000 7f0c90106700 1 --2- 172.21.15.162:0/4138929118 >> [v2:172.21.15.162:6800/1480340393,v1:172.21.15.162:6801/1480340393] conn(0x7f0c70036800 0x7f0c70038cb0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:00:33.470 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:32.939+0000 7f0c90106700 1 --2- 172.21.15.162:0/4138929118 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0c88076c10 0x7f0c88102d90 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:00:33.471 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:32.939+0000 7f0c90106700 1 -- 172.21.15.162:0/4138929118 >> 172.21.15.162:0/4138929118 conn(0x7f0c880fcef0 msgr2=0x7f0c880fda40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:00:33.471 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:32.939+0000 7f0c90106700 1 -- 172.21.15.162:0/4138929118 shutdown_connections 2024-04-27T04:00:33.471 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:32.939+0000 7f0c90106700 1 -- 172.21.15.162:0/4138929118 wait complete. 2024-04-27T04:00:33.471 INFO:teuthology.orchestra.run.smithi162.stdout:Enabling cephadm module... 2024-04-27T04:00:34.389 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:00:34 smithi162 ceph-mon[26841]: from='client.? 172.21.15.162:0/3924316327' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "cephadm"}]: dispatch 2024-04-27T04:00:34.704 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:33.846+0000 7fd8b5955700 1 Processor -- start 2024-04-27T04:00:34.704 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:33.846+0000 7fd8b5955700 1 -- start start 2024-04-27T04:00:34.704 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:33.846+0000 7fd8b5955700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd8b0076c10 0x7fd8b0105440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:00:34.704 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:33.846+0000 7fd8b5955700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8b0105980 con 0x7fd8b0076c10 2024-04-27T04:00:34.704 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:33.846+0000 7fd8aeffd700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd8b0076c10 0x7fd8b0105440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:00:34.704 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:33.846+0000 7fd8aeffd700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd8b0076c10 0x7fd8b0105440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:54158/0 (socket says 172.21.15.162:54158) 2024-04-27T04:00:34.704 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:33.846+0000 7fd8aeffd700 1 -- 172.21.15.162:0/3162196525 learned_addr learned my addr 172.21.15.162:0/3162196525 (peer_addr_for_me v2:172.21.15.162:0/0) 2024-04-27T04:00:34.704 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:33.846+0000 7fd8aeffd700 1 -- 172.21.15.162:0/3162196525 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8b0105ac0 con 0x7fd8b0076c10 2024-04-27T04:00:34.704 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:33.846+0000 7fd8aeffd700 1 --2- 172.21.15.162:0/3162196525 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd8b0076c10 0x7fd8b0105440 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fd8a00098e0 tx=0x7fd8a0009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8f8aace7f49311d8 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:00:34.705 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:33.847+0000 7fd8adffb700 1 -- 172.21.15.162:0/3162196525 <== mon.0 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fd8a000b860 con 0x7fd8b0076c10 2024-04-27T04:00:34.705 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:33.847+0000 7fd8adffb700 1 -- 172.21.15.162:0/3162196525 <== mon.0 v2:172.21.15.162:3300/0 2 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7fd8a000be50 con 0x7fd8b0076c10 2024-04-27T04:00:34.705 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:33.847+0000 7fd8adffb700 1 -- 172.21.15.162:0/3162196525 <== mon.0 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fd8a0010380 con 0x7fd8b0076c10 2024-04-27T04:00:34.705 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:33.847+0000 7fd8b5955700 1 -- 172.21.15.162:0/3162196525 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd8b0076c10 msgr2=0x7fd8b0105440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:00:34.705 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:33.847+0000 7fd8b5955700 1 --2- 172.21.15.162:0/3162196525 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd8b0076c10 0x7fd8b0105440 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fd8a00098e0 tx=0x7fd8a0009250 comp rx=0 tx=0).stop 2024-04-27T04:00:34.705 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:33.847+0000 7fd8b5955700 1 -- 172.21.15.162:0/3162196525 shutdown_connections 2024-04-27T04:00:34.705 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:33.847+0000 7fd8b5955700 1 --2- 172.21.15.162:0/3162196525 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd8b0076c10 0x7fd8b0105440 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:00:34.705 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:33.847+0000 7fd8b5955700 1 -- 172.21.15.162:0/3162196525 >> 172.21.15.162:0/3162196525 conn(0x7fd8b00fcef0 msgr2=0x7fd8b00ff300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:00:34.705 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:33.847+0000 7fd8b5955700 1 -- 172.21.15.162:0/3162196525 shutdown_connections 2024-04-27T04:00:34.705 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:33.847+0000 7fd8b5955700 1 -- 172.21.15.162:0/3162196525 wait complete. 2024-04-27T04:00:34.705 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:33.848+0000 7fd8b5955700 1 Processor -- start 2024-04-27T04:00:34.705 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:33.848+0000 7fd8b5955700 1 -- start start 2024-04-27T04:00:34.705 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:33.848+0000 7fd8b5955700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd8b0102d50 0x7fd8b0101340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:00:34.705 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:33.848+0000 7fd8b5955700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8b0103160 con 0x7fd8b0102d50 2024-04-27T04:00:34.706 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:33.848+0000 7fd8aeffd700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd8b0102d50 0x7fd8b0101340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:00:34.706 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:33.848+0000 7fd8aeffd700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd8b0102d50 0x7fd8b0101340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:54164/0 (socket says 172.21.15.162:54164) 2024-04-27T04:00:34.706 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:33.848+0000 7fd8aeffd700 1 -- 172.21.15.162:0/3924316327 learned_addr learned my addr 172.21.15.162:0/3924316327 (peer_addr_for_me v2:172.21.15.162:0/0) 2024-04-27T04:00:34.706 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:33.848+0000 7fd8aeffd700 1 -- 172.21.15.162:0/3924316327 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8a00094f0 con 0x7fd8b0102d50 2024-04-27T04:00:34.706 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:33.848+0000 7fd8aeffd700 1 --2- 172.21.15.162:0/3924316327 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd8b0102d50 0x7fd8b0101340 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fd8a000b440 tx=0x7fd8a0009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:00:34.706 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:33.849+0000 7fd8977fe700 1 -- 172.21.15.162:0/3924316327 <== mon.0 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fd8a0010600 con 0x7fd8b0102d50 2024-04-27T04:00:34.706 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:33.849+0000 7fd8977fe700 1 -- 172.21.15.162:0/3924316327 <== mon.0 v2:172.21.15.162:3300/0 2 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7fd8a001f070 con 0x7fd8b0102d50 2024-04-27T04:00:34.706 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:33.849+0000 7fd8b5955700 1 -- 172.21.15.162:0/3924316327 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd8b0101880 con 0x7fd8b0102d50 2024-04-27T04:00:34.706 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:33.849+0000 7fd8977fe700 1 -- 172.21.15.162:0/3924316327 <== mon.0 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fd8a0026b90 con 0x7fd8b0102d50 2024-04-27T04:00:34.706 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:33.849+0000 7fd8b5955700 1 -- 172.21.15.162:0/3924316327 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd8b0101d10 con 0x7fd8b0102d50 2024-04-27T04:00:34.706 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:33.850+0000 7fd8977fe700 1 -- 172.21.15.162:0/3924316327 <== mon.0 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 4) v1 ==== 43365+0+0 (secure 0 0 0) 0x7fd8a0026cf0 con 0x7fd8b0102d50 2024-04-27T04:00:34.706 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:33.850+0000 7fd8977fe700 1 --2- 172.21.15.162:0/3924316327 >> [v2:172.21.15.162:6800/1480340393,v1:172.21.15.162:6801/1480340393] conn(0x7fd898036800 0x7fd898038cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:00:34.707 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:33.850+0000 7fd8977fe700 1 -- 172.21.15.162:0/3924316327 <== mon.0 v2:172.21.15.162:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 721+0+0 (secure 0 0 0) 0x7fd8a0021070 con 0x7fd8b0102d50 2024-04-27T04:00:34.707 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:33.850+0000 7fd8b5955700 1 -- 172.21.15.162:0/3924316327 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd8b004bed0 con 0x7fd8b0102d50 2024-04-27T04:00:34.707 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:33.850+0000 7fd8ae7fc700 1 --2- 172.21.15.162:0/3924316327 >> [v2:172.21.15.162:6800/1480340393,v1:172.21.15.162:6801/1480340393] conn(0x7fd898036800 0x7fd898038cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:00:34.707 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:33.850+0000 7fd8ae7fc700 1 --2- 172.21.15.162:0/3924316327 >> [v2:172.21.15.162:6800/1480340393,v1:172.21.15.162:6801/1480340393] conn(0x7fd898036800 0x7fd898038cb0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fd8a4009990 tx=0x7fd8a4006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:00:34.707 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:33.854+0000 7fd8977fe700 1 -- 172.21.15.162:0/3924316327 <== mon.0 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7fd8a001b440 con 0x7fd8b0102d50 2024-04-27T04:00:34.707 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:34.089+0000 7fd8b5955700 1 -- 172.21.15.162:0/3924316327 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "cephadm"} v 0) v1 -- 0x7fd8b0117770 con 0x7fd8b0102d50 2024-04-27T04:00:34.707 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:34.142+0000 7fd8977fe700 1 -- 172.21.15.162:0/3924316327 <== mon.0 v2:172.21.15.162:3300/0 7 ==== mgrmap(e 5) v1 ==== 43376+0+0 (secure 0 0 0) 0x7fd8a0010c60 con 0x7fd8b0102d50 2024-04-27T04:00:34.707 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:34.143+0000 7fd8977fe700 1 -- 172.21.15.162:0/3924316327 <== mon.0 v2:172.21.15.162:3300/0 8 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "cephadm"}]=0 v5) v1 ==== 86+0+0 (secure 0 0 0) 0x7fd8a0024380 con 0x7fd8b0102d50 2024-04-27T04:00:34.707 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:34.145+0000 7fd8b5955700 1 -- 172.21.15.162:0/3924316327 >> [v2:172.21.15.162:6800/1480340393,v1:172.21.15.162:6801/1480340393] conn(0x7fd898036800 msgr2=0x7fd898038cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:00:34.707 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:34.145+0000 7fd8b5955700 1 --2- 172.21.15.162:0/3924316327 >> [v2:172.21.15.162:6800/1480340393,v1:172.21.15.162:6801/1480340393] conn(0x7fd898036800 0x7fd898038cb0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fd8a4009990 tx=0x7fd8a4006e40 comp rx=0 tx=0).stop 2024-04-27T04:00:34.707 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:34.145+0000 7fd8b5955700 1 -- 172.21.15.162:0/3924316327 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd8b0102d50 msgr2=0x7fd8b0101340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:00:34.707 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:34.145+0000 7fd8b5955700 1 --2- 172.21.15.162:0/3924316327 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd8b0102d50 0x7fd8b0101340 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fd8a000b440 tx=0x7fd8a0009250 comp rx=0 tx=0).stop 2024-04-27T04:00:34.707 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:34.146+0000 7fd8b5955700 1 -- 172.21.15.162:0/3924316327 shutdown_connections 2024-04-27T04:00:34.708 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:34.146+0000 7fd8b5955700 1 --2- 172.21.15.162:0/3924316327 >> [v2:172.21.15.162:6800/1480340393,v1:172.21.15.162:6801/1480340393] conn(0x7fd898036800 0x7fd898038cb0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:00:34.708 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:34.146+0000 7fd8b5955700 1 --2- 172.21.15.162:0/3924316327 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd8b0102d50 0x7fd8b0101340 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:00:34.708 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:34.146+0000 7fd8b5955700 1 -- 172.21.15.162:0/3924316327 >> 172.21.15.162:0/3924316327 conn(0x7fd8b00fcef0 msgr2=0x7fd8b00fdb10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:00:34.708 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:34.146+0000 7fd8b5955700 1 -- 172.21.15.162:0/3924316327 shutdown_connections 2024-04-27T04:00:34.708 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:34.146+0000 7fd8b5955700 1 -- 172.21.15.162:0/3924316327 wait complete. 2024-04-27T04:00:35.389 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:00:35 smithi162 ceph-mon[26841]: from='client.? 172.21.15.162:0/3924316327' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "cephadm"}]': finished 2024-04-27T04:00:35.389 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:00:35 smithi162 ceph-mon[26841]: mgrmap e5: smithi162.oennfn(active, since 14s) 2024-04-27T04:00:36.054 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stdout { 2024-04-27T04:00:36.054 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stdout "epoch": 5, 2024-04-27T04:00:36.054 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stdout "available": true, 2024-04-27T04:00:36.054 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stdout "active_name": "smithi162.oennfn", 2024-04-27T04:00:36.054 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-04-27T04:00:36.054 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stdout } 2024-04-27T04:00:36.054 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:35.189+0000 7fe6c7efa700 1 Processor -- start 2024-04-27T04:00:36.054 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:35.189+0000 7fe6c7efa700 1 -- start start 2024-04-27T04:00:36.054 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:35.189+0000 7fe6c7efa700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe6c0078c50 0x7fe6c0077250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:00:36.054 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:35.189+0000 7fe6c7efa700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6c0079060 con 0x7fe6c0078c50 2024-04-27T04:00:36.055 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:35.189+0000 7fe6c5c96700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe6c0078c50 0x7fe6c0077250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:00:36.055 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:35.189+0000 7fe6c5c96700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe6c0078c50 0x7fe6c0077250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:54196/0 (socket says 172.21.15.162:54196) 2024-04-27T04:00:36.055 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:35.189+0000 7fe6c5c96700 1 -- 172.21.15.162:0/3142179808 learned_addr learned my addr 172.21.15.162:0/3142179808 (peer_addr_for_me v2:172.21.15.162:0/0) 2024-04-27T04:00:36.055 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:35.190+0000 7fe6c5c96700 1 -- 172.21.15.162:0/3142179808 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe6c0077790 con 0x7fe6c0078c50 2024-04-27T04:00:36.055 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:35.190+0000 7fe6c5c96700 1 --2- 172.21.15.162:0/3142179808 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe6c0078c50 0x7fe6c0077250 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fe6bc0098e0 tx=0x7fe6bc009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4a4790f517de989c server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:00:36.055 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:35.190+0000 7fe6c4c94700 1 -- 172.21.15.162:0/3142179808 <== mon.0 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fe6bc00b860 con 0x7fe6c0078c50 2024-04-27T04:00:36.055 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:35.190+0000 7fe6c4c94700 1 -- 172.21.15.162:0/3142179808 <== mon.0 v2:172.21.15.162:3300/0 2 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7fe6bc00be50 con 0x7fe6c0078c50 2024-04-27T04:00:36.055 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:35.190+0000 7fe6c4c94700 1 -- 172.21.15.162:0/3142179808 <== mon.0 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fe6bc010380 con 0x7fe6c0078c50 2024-04-27T04:00:36.055 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:35.191+0000 7fe6c7efa700 1 -- 172.21.15.162:0/3142179808 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe6c0078c50 msgr2=0x7fe6c0077250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:00:36.055 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:35.191+0000 7fe6c7efa700 1 --2- 172.21.15.162:0/3142179808 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe6c0078c50 0x7fe6c0077250 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fe6bc0098e0 tx=0x7fe6bc009250 comp rx=0 tx=0).stop 2024-04-27T04:00:36.055 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:35.191+0000 7fe6c7efa700 1 -- 172.21.15.162:0/3142179808 shutdown_connections 2024-04-27T04:00:36.055 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:35.191+0000 7fe6c7efa700 1 --2- 172.21.15.162:0/3142179808 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe6c0078c50 0x7fe6c0077250 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:00:36.055 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:35.191+0000 7fe6c7efa700 1 -- 172.21.15.162:0/3142179808 >> 172.21.15.162:0/3142179808 conn(0x7fe6c00fe030 msgr2=0x7fe6c0100440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:00:36.055 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:35.191+0000 7fe6c7efa700 1 -- 172.21.15.162:0/3142179808 shutdown_connections 2024-04-27T04:00:36.056 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:35.191+0000 7fe6c7efa700 1 -- 172.21.15.162:0/3142179808 wait complete. 2024-04-27T04:00:36.056 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:35.191+0000 7fe6c7efa700 1 Processor -- start 2024-04-27T04:00:36.056 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:35.191+0000 7fe6c7efa700 1 -- start start 2024-04-27T04:00:36.056 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:35.192+0000 7fe6c7efa700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe6c0078c50 0x7fe6c0113800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:00:36.056 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:35.192+0000 7fe6c7efa700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6c0113d40 con 0x7fe6c0078c50 2024-04-27T04:00:36.056 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:35.192+0000 7fe6c5c96700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe6c0078c50 0x7fe6c0113800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:00:36.056 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:35.192+0000 7fe6c5c96700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe6c0078c50 0x7fe6c0113800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:54200/0 (socket says 172.21.15.162:54200) 2024-04-27T04:00:36.056 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:35.192+0000 7fe6c5c96700 1 -- 172.21.15.162:0/3522073199 learned_addr learned my addr 172.21.15.162:0/3522073199 (peer_addr_for_me v2:172.21.15.162:0/0) 2024-04-27T04:00:36.056 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:35.192+0000 7fe6c5c96700 1 -- 172.21.15.162:0/3522073199 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe6bc0094f0 con 0x7fe6c0078c50 2024-04-27T04:00:36.056 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:35.192+0000 7fe6c5c96700 1 --2- 172.21.15.162:0/3522073199 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe6c0078c50 0x7fe6c0113800 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fe6bc006fd0 tx=0x7fe6bc009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:00:36.056 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:35.192+0000 7fe6b67fc700 1 -- 172.21.15.162:0/3522073199 <== mon.0 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fe6bc0107c0 con 0x7fe6c0078c50 2024-04-27T04:00:36.056 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:35.192+0000 7fe6c7efa700 1 -- 172.21.15.162:0/3522073199 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe6c0113f40 con 0x7fe6c0078c50 2024-04-27T04:00:36.056 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:35.192+0000 7fe6b67fc700 1 -- 172.21.15.162:0/3522073199 <== mon.0 v2:172.21.15.162:3300/0 2 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7fe6bc010db0 con 0x7fe6c0078c50 2024-04-27T04:00:36.056 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:35.192+0000 7fe6b67fc700 1 -- 172.21.15.162:0/3522073199 <== mon.0 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fe6bc01ada0 con 0x7fe6c0078c50 2024-04-27T04:00:36.057 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:35.192+0000 7fe6c7efa700 1 -- 172.21.15.162:0/3522073199 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe6c0114ab0 con 0x7fe6c0078c50 2024-04-27T04:00:36.057 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:35.193+0000 7fe6b67fc700 1 -- 172.21.15.162:0/3522073199 <== mon.0 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 5) v1 ==== 43376+0+0 (secure 0 0 0) 0x7fe6bc019450 con 0x7fe6c0078c50 2024-04-27T04:00:36.057 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:35.193+0000 7fe6c7efa700 1 -- 172.21.15.162:0/3522073199 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe6c005f5c0 con 0x7fe6c0078c50 2024-04-27T04:00:36.057 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:35.193+0000 7fe6b67fc700 1 --2- 172.21.15.162:0/3522073199 >> [v2:172.21.15.162:6800/1480340393,v1:172.21.15.162:6801/1480340393] conn(0x7fe6ac036850 0x7fe6ac038d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:00:36.057 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:35.193+0000 7fe6b67fc700 1 -- 172.21.15.162:0/3522073199 <== mon.0 v2:172.21.15.162:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 721+0+0 (secure 0 0 0) 0x7fe6bc049d10 con 0x7fe6c0078c50 2024-04-27T04:00:36.057 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:35.193+0000 7fe6c5495700 1 -- 172.21.15.162:0/3522073199 >> [v2:172.21.15.162:6800/1480340393,v1:172.21.15.162:6801/1480340393] conn(0x7fe6ac036850 msgr2=0x7fe6ac038d00 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.162:6800/1480340393 2024-04-27T04:00:36.057 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:35.193+0000 7fe6c5495700 1 --2- 172.21.15.162:0/3522073199 >> [v2:172.21.15.162:6800/1480340393,v1:172.21.15.162:6801/1480340393] conn(0x7fe6ac036850 0x7fe6ac038d00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-27T04:00:36.057 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:35.198+0000 7fe6b67fc700 1 -- 172.21.15.162:0/3522073199 <== mon.0 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7fe6bc04f020 con 0x7fe6c0078c50 2024-04-27T04:00:36.057 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:35.394+0000 7fe6c5495700 1 -- 172.21.15.162:0/3522073199 >> [v2:172.21.15.162:6800/1480340393,v1:172.21.15.162:6801/1480340393] conn(0x7fe6ac036850 msgr2=0x7fe6ac038d00 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.162:6800/1480340393 2024-04-27T04:00:36.057 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:35.394+0000 7fe6c5495700 1 --2- 172.21.15.162:0/3522073199 >> [v2:172.21.15.162:6800/1480340393,v1:172.21.15.162:6801/1480340393] conn(0x7fe6ac036850 0x7fe6ac038d00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-27T04:00:36.057 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:35.477+0000 7fe6c7efa700 1 -- 172.21.15.162:0/3522073199 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) v1 -- 0x7fe6c0114df0 con 0x7fe6c0078c50 2024-04-27T04:00:36.057 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:35.478+0000 7fe6b67fc700 1 -- 172.21.15.162:0/3522073199 <== mon.0 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v5) v1 ==== 56+0+103 (secure 0 0 0) 0x7fe6bc020020 con 0x7fe6c0078c50 2024-04-27T04:00:36.057 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:35.480+0000 7fe6c7efa700 1 -- 172.21.15.162:0/3522073199 >> [v2:172.21.15.162:6800/1480340393,v1:172.21.15.162:6801/1480340393] conn(0x7fe6ac036850 msgr2=0x7fe6ac038d00 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-27T04:00:36.058 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:35.480+0000 7fe6c7efa700 1 --2- 172.21.15.162:0/3522073199 >> [v2:172.21.15.162:6800/1480340393,v1:172.21.15.162:6801/1480340393] conn(0x7fe6ac036850 0x7fe6ac038d00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:00:36.058 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:35.480+0000 7fe6c7efa700 1 -- 172.21.15.162:0/3522073199 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe6c0078c50 msgr2=0x7fe6c0113800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:00:36.058 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:35.480+0000 7fe6c7efa700 1 --2- 172.21.15.162:0/3522073199 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe6c0078c50 0x7fe6c0113800 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fe6bc006fd0 tx=0x7fe6bc009250 comp rx=0 tx=0).stop 2024-04-27T04:00:36.058 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:35.480+0000 7fe6c7efa700 1 -- 172.21.15.162:0/3522073199 shutdown_connections 2024-04-27T04:00:36.058 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:35.480+0000 7fe6c7efa700 1 --2- 172.21.15.162:0/3522073199 >> [v2:172.21.15.162:6800/1480340393,v1:172.21.15.162:6801/1480340393] conn(0x7fe6ac036850 0x7fe6ac038d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:00:36.058 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:35.480+0000 7fe6c7efa700 1 --2- 172.21.15.162:0/3522073199 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe6c0078c50 0x7fe6c0113800 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:00:36.058 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:35.480+0000 7fe6c7efa700 1 -- 172.21.15.162:0/3522073199 >> 172.21.15.162:0/3522073199 conn(0x7fe6c00fe030 msgr2=0x7fe6c00feae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:00:36.058 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:35.480+0000 7fe6c7efa700 1 -- 172.21.15.162:0/3522073199 shutdown_connections 2024-04-27T04:00:36.058 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:35.480+0000 7fe6c7efa700 1 -- 172.21.15.162:0/3522073199 wait complete. 2024-04-27T04:00:36.058 INFO:teuthology.orchestra.run.smithi162.stdout:Waiting for the mgr to restart... 2024-04-27T04:00:36.058 INFO:teuthology.orchestra.run.smithi162.stdout:Waiting for mgr epoch 5... 2024-04-27T04:00:36.314 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:00:36 smithi162 ceph-mon[26841]: from='client.? 172.21.15.162:0/3522073199' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-04-27T04:00:40.889 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:00:40 smithi162 ceph-mon[26841]: Active manager daemon smithi162.oennfn restarted 2024-04-27T04:00:40.889 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:00:40 smithi162 ceph-mon[26841]: Activating manager daemon smithi162.oennfn 2024-04-27T04:00:40.889 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:00:40 smithi162 ceph-mon[26841]: osdmap e2: 0 total, 0 up, 0 in 2024-04-27T04:00:40.889 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:00:40 smithi162 ceph-mon[26841]: mgrmap e6: smithi162.oennfn(active, starting, since 0.00324363s) 2024-04-27T04:00:40.889 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:00:40 smithi162 ceph-mon[26841]: from='mgr.14120 172.21.15.162:0/1771817378' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "mon metadata", "id": "smithi162"}]: dispatch 2024-04-27T04:00:40.889 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:00:40 smithi162 ceph-mon[26841]: from='mgr.14120 172.21.15.162:0/1771817378' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "mgr metadata", "who": "smithi162.oennfn", "id": "smithi162.oennfn"}]: dispatch 2024-04-27T04:00:40.890 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:00:40 smithi162 ceph-mon[26841]: from='mgr.14120 172.21.15.162:0/1771817378' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-04-27T04:00:40.890 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:00:40 smithi162 ceph-mon[26841]: from='mgr.14120 172.21.15.162:0/1771817378' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-04-27T04:00:40.890 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:00:40 smithi162 ceph-mon[26841]: from='mgr.14120 172.21.15.162:0/1771817378' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-04-27T04:00:40.890 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:00:40 smithi162 ceph-mon[26841]: Manager daemon smithi162.oennfn is now available 2024-04-27T04:00:40.890 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:00:40 smithi162 ceph-mon[26841]: from='mgr.14120 172.21.15.162:0/1771817378' entity='mgr.smithi162.oennfn' 2024-04-27T04:00:40.890 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:00:40 smithi162 ceph-mon[26841]: from='mgr.14120 172.21.15.162:0/1771817378' entity='mgr.smithi162.oennfn' 2024-04-27T04:00:40.890 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:00:40 smithi162 ceph-mon[26841]: from='mgr.14120 172.21.15.162:0/1771817378' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T04:00:40.890 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:00:40 smithi162 ceph-mon[26841]: from='mgr.14120 172.21.15.162:0/1771817378' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T04:00:40.890 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:00:40 smithi162 ceph-mon[26841]: from='mgr.14120 172.21.15.162:0/1771817378' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T04:00:40.890 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:00:40 smithi162 ceph-mon[26841]: from='mgr.14120 172.21.15.162:0/1771817378' entity='mgr.smithi162.oennfn' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi162.oennfn/mirror_snapshot_schedule"}]: dispatch 2024-04-27T04:00:40.890 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:00:40 smithi162 ceph-mon[26841]: from='mgr.14120 172.21.15.162:0/1771817378' entity='mgr.smithi162.oennfn' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi162.oennfn/trash_purge_schedule"}]: dispatch 2024-04-27T04:00:41.985 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:00:41 smithi162 ceph-mon[26841]: Found migration_current of "None". Setting to last migration. 2024-04-27T04:00:41.985 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:00:41 smithi162 ceph-mon[26841]: mgrmap e7: smithi162.oennfn(active, since 1.00512s) 2024-04-27T04:00:42.237 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stdout { 2024-04-27T04:00:42.237 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 7, 2024-04-27T04:00:42.237 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stdout "initialized": true 2024-04-27T04:00:42.237 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stdout } 2024-04-27T04:00:42.237 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:36.413+0000 7f1c6cd01700 1 Processor -- start 2024-04-27T04:00:42.237 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:36.413+0000 7f1c6cd01700 1 -- start start 2024-04-27T04:00:42.238 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:36.413+0000 7f1c6cd01700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1c68077b70 0x7f1c68076120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:00:42.238 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:36.413+0000 7f1c6cd01700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c68077f80 con 0x7f1c68077b70 2024-04-27T04:00:42.238 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:36.414+0000 7f1c6659c700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1c68077b70 0x7f1c68076120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:00:42.238 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:36.414+0000 7f1c6659c700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1c68077b70 0x7f1c68076120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:54212/0 (socket says 172.21.15.162:54212) 2024-04-27T04:00:42.238 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:36.414+0000 7f1c6659c700 1 -- 172.21.15.162:0/2522605854 learned_addr learned my addr 172.21.15.162:0/2522605854 (peer_addr_for_me v2:172.21.15.162:0/0) 2024-04-27T04:00:42.238 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:36.414+0000 7f1c6659c700 1 -- 172.21.15.162:0/2522605854 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c68076660 con 0x7f1c68077b70 2024-04-27T04:00:42.238 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:36.414+0000 7f1c6659c700 1 --2- 172.21.15.162:0/2522605854 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1c68077b70 0x7f1c68076120 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f1c580098e0 tx=0x7f1c58009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bd3b8a662b0b446d server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:00:42.238 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:36.414+0000 7f1c6559a700 1 -- 172.21.15.162:0/2522605854 <== mon.0 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f1c5800b860 con 0x7f1c68077b70 2024-04-27T04:00:42.238 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:36.414+0000 7f1c6559a700 1 -- 172.21.15.162:0/2522605854 <== mon.0 v2:172.21.15.162:3300/0 2 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7f1c5800be50 con 0x7f1c68077b70 2024-04-27T04:00:42.238 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:36.414+0000 7f1c6559a700 1 -- 172.21.15.162:0/2522605854 <== mon.0 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f1c58010380 con 0x7f1c68077b70 2024-04-27T04:00:42.238 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:36.415+0000 7f1c6cd01700 1 -- 172.21.15.162:0/2522605854 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1c68077b70 msgr2=0x7f1c68076120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:00:42.238 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:36.415+0000 7f1c6cd01700 1 --2- 172.21.15.162:0/2522605854 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1c68077b70 0x7f1c68076120 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f1c580098e0 tx=0x7f1c58009250 comp rx=0 tx=0).stop 2024-04-27T04:00:42.238 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:36.415+0000 7f1c6cd01700 1 -- 172.21.15.162:0/2522605854 shutdown_connections 2024-04-27T04:00:42.239 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:36.415+0000 7f1c6cd01700 1 --2- 172.21.15.162:0/2522605854 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1c68077b70 0x7f1c68076120 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:00:42.239 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:36.415+0000 7f1c6cd01700 1 -- 172.21.15.162:0/2522605854 >> 172.21.15.162:0/2522605854 conn(0x7f1c680fdfb0 msgr2=0x7f1c68100420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:00:42.239 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:36.415+0000 7f1c6cd01700 1 -- 172.21.15.162:0/2522605854 shutdown_connections 2024-04-27T04:00:42.239 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:36.415+0000 7f1c6cd01700 1 -- 172.21.15.162:0/2522605854 wait complete. 2024-04-27T04:00:42.239 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:36.416+0000 7f1c6cd01700 1 Processor -- start 2024-04-27T04:00:42.239 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:36.416+0000 7f1c6cd01700 1 -- start start 2024-04-27T04:00:42.239 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:36.416+0000 7f1c6cd01700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1c68077b70 0x7f1c68113830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:00:42.239 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:36.416+0000 7f1c6cd01700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c68113d70 con 0x7f1c68077b70 2024-04-27T04:00:42.239 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:36.416+0000 7f1c6659c700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1c68077b70 0x7f1c68113830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:00:42.239 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:36.416+0000 7f1c6659c700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1c68077b70 0x7f1c68113830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:54214/0 (socket says 172.21.15.162:54214) 2024-04-27T04:00:42.239 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:36.416+0000 7f1c6659c700 1 -- 172.21.15.162:0/1752198451 learned_addr learned my addr 172.21.15.162:0/1752198451 (peer_addr_for_me v2:172.21.15.162:0/0) 2024-04-27T04:00:42.239 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:36.417+0000 7f1c6659c700 1 -- 172.21.15.162:0/1752198451 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c580094f0 con 0x7f1c68077b70 2024-04-27T04:00:42.239 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:36.417+0000 7f1c6659c700 1 --2- 172.21.15.162:0/1752198451 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1c68077b70 0x7f1c68113830 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f1c58006b00 tx=0x7f1c58009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:00:42.239 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:36.417+0000 7f1c56ffd700 1 -- 172.21.15.162:0/1752198451 <== mon.0 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f1c580107c0 con 0x7f1c68077b70 2024-04-27T04:00:42.240 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:36.417+0000 7f1c56ffd700 1 -- 172.21.15.162:0/1752198451 <== mon.0 v2:172.21.15.162:3300/0 2 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7f1c58010db0 con 0x7f1c68077b70 2024-04-27T04:00:42.240 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:36.417+0000 7f1c6cd01700 1 -- 172.21.15.162:0/1752198451 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1c68113f70 con 0x7f1c68077b70 2024-04-27T04:00:42.240 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:36.417+0000 7f1c56ffd700 1 -- 172.21.15.162:0/1752198451 <== mon.0 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f1c5801acf0 con 0x7f1c68077b70 2024-04-27T04:00:42.240 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:36.417+0000 7f1c6cd01700 1 -- 172.21.15.162:0/1752198451 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1c68114ae0 con 0x7f1c68077b70 2024-04-27T04:00:42.240 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:36.418+0000 7f1c56ffd700 1 -- 172.21.15.162:0/1752198451 <== mon.0 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 5) v1 ==== 43376+0+0 (secure 0 0 0) 0x7f1c580193c0 con 0x7f1c68077b70 2024-04-27T04:00:42.240 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:36.418+0000 7f1c56ffd700 1 --2- 172.21.15.162:0/1752198451 >> [v2:172.21.15.162:6800/1480340393,v1:172.21.15.162:6801/1480340393] conn(0x7f1c50036850 0x7f1c50038d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:00:42.240 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:36.418+0000 7f1c56ffd700 1 -- 172.21.15.162:0/1752198451 --> [v2:172.21.15.162:6800/1480340393,v1:172.21.15.162:6801/1480340393] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7f1c50039410 con 0x7f1c50036850 2024-04-27T04:00:42.240 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:36.418+0000 7f1c56ffd700 1 -- 172.21.15.162:0/1752198451 <== mon.0 v2:172.21.15.162:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 721+0+0 (secure 0 0 0) 0x7f1c5804b500 con 0x7f1c68077b70 2024-04-27T04:00:42.240 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:36.418+0000 7f1c65d9b700 1 -- 172.21.15.162:0/1752198451 >> [v2:172.21.15.162:6800/1480340393,v1:172.21.15.162:6801/1480340393] conn(0x7f1c50036850 msgr2=0x7f1c50038d00 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.162:6800/1480340393 2024-04-27T04:00:42.240 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:36.418+0000 7f1c65d9b700 1 --2- 172.21.15.162:0/1752198451 >> [v2:172.21.15.162:6800/1480340393,v1:172.21.15.162:6801/1480340393] conn(0x7f1c50036850 0x7f1c50038d00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-27T04:00:42.240 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:36.619+0000 7f1c65d9b700 1 -- 172.21.15.162:0/1752198451 >> [v2:172.21.15.162:6800/1480340393,v1:172.21.15.162:6801/1480340393] conn(0x7f1c50036850 msgr2=0x7f1c50038d00 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.162:6800/1480340393 2024-04-27T04:00:42.240 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:36.619+0000 7f1c65d9b700 1 --2- 172.21.15.162:0/1752198451 >> [v2:172.21.15.162:6800/1480340393,v1:172.21.15.162:6801/1480340393] conn(0x7f1c50036850 0x7f1c50038d00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-27T04:00:42.240 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:37.020+0000 7f1c65d9b700 1 -- 172.21.15.162:0/1752198451 >> [v2:172.21.15.162:6800/1480340393,v1:172.21.15.162:6801/1480340393] conn(0x7f1c50036850 msgr2=0x7f1c50038d00 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.162:6800/1480340393 2024-04-27T04:00:42.240 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:37.020+0000 7f1c65d9b700 1 --2- 172.21.15.162:0/1752198451 >> [v2:172.21.15.162:6800/1480340393,v1:172.21.15.162:6801/1480340393] conn(0x7f1c50036850 0x7f1c50038d00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-27T04:00:42.241 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:37.821+0000 7f1c65d9b700 1 -- 172.21.15.162:0/1752198451 >> [v2:172.21.15.162:6800/1480340393,v1:172.21.15.162:6801/1480340393] conn(0x7f1c50036850 msgr2=0x7f1c50038d00 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.162:6800/1480340393 2024-04-27T04:00:42.241 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:37.821+0000 7f1c65d9b700 1 --2- 172.21.15.162:0/1752198451 >> [v2:172.21.15.162:6800/1480340393,v1:172.21.15.162:6801/1480340393] conn(0x7f1c50036850 0x7f1c50038d00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-27T04:00:42.241 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:39.422+0000 7f1c65d9b700 1 -- 172.21.15.162:0/1752198451 >> [v2:172.21.15.162:6800/1480340393,v1:172.21.15.162:6801/1480340393] conn(0x7f1c50036850 msgr2=0x7f1c50038d00 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.162:6800/1480340393 2024-04-27T04:00:42.241 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:39.422+0000 7f1c65d9b700 1 --2- 172.21.15.162:0/1752198451 >> [v2:172.21.15.162:6800/1480340393,v1:172.21.15.162:6801/1480340393] conn(0x7f1c50036850 0x7f1c50038d00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2024-04-27T04:00:42.241 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:40.642+0000 7f1c56ffd700 1 -- 172.21.15.162:0/1752198451 <== mon.0 v2:172.21.15.162:3300/0 6 ==== mgrmap(e 6) v1 ==== 43143+0+0 (secure 0 0 0) 0x7f1c5802c3a0 con 0x7f1c68077b70 2024-04-27T04:00:42.241 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:40.642+0000 7f1c56ffd700 1 -- 172.21.15.162:0/1752198451 >> [v2:172.21.15.162:6800/1480340393,v1:172.21.15.162:6801/1480340393] conn(0x7f1c50036850 msgr2=0x7f1c50038d00 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-27T04:00:42.241 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:40.642+0000 7f1c56ffd700 1 --2- 172.21.15.162:0/1752198451 >> [v2:172.21.15.162:6800/1480340393,v1:172.21.15.162:6801/1480340393] conn(0x7f1c50036850 0x7f1c50038d00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:00:42.241 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:41.643+0000 7f1c56ffd700 1 -- 172.21.15.162:0/1752198451 <== mon.0 v2:172.21.15.162:3300/0 7 ==== mgrmap(e 7) v1 ==== 43270+0+0 (secure 0 0 0) 0x7f1c58023da0 con 0x7f1c68077b70 2024-04-27T04:00:42.241 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:41.643+0000 7f1c56ffd700 1 --2- 172.21.15.162:0/1752198451 >> [v2:172.21.15.162:6800/8067198,v1:172.21.15.162:6801/8067198] conn(0x7f1c50039ff0 0x7f1c5003c3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:00:42.241 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:41.643+0000 7f1c56ffd700 1 -- 172.21.15.162:0/1752198451 --> [v2:172.21.15.162:6800/8067198,v1:172.21.15.162:6801/8067198] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7f1c50039410 con 0x7f1c50039ff0 2024-04-27T04:00:42.241 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:41.644+0000 7f1c65d9b700 1 --2- 172.21.15.162:0/1752198451 >> [v2:172.21.15.162:6800/8067198,v1:172.21.15.162:6801/8067198] conn(0x7f1c50039ff0 0x7f1c5003c3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:00:42.241 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:41.644+0000 7f1c65d9b700 1 --2- 172.21.15.162:0/1752198451 >> [v2:172.21.15.162:6800/8067198,v1:172.21.15.162:6801/8067198] conn(0x7f1c50039ff0 0x7f1c5003c3d0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f1c5c0039d0 tx=0x7f1c5c009350 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:00:42.241 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:41.645+0000 7f1c56ffd700 1 -- 172.21.15.162:0/1752198451 <== mgr.14120 v2:172.21.15.162:6800/8067198 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+6549 (secure 0 0 0) 0x7f1c50039410 con 0x7f1c50039ff0 2024-04-27T04:00:42.241 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:41.651+0000 7f1c6cd01700 1 -- 172.21.15.162:0/1752198451 --> [v2:172.21.15.162:6800/8067198,v1:172.21.15.162:6801/8067198] -- command(tid 1: {"prefix": "mgr_status"}) v1 -- 0x7f1c68114400 con 0x7f1c50039ff0 2024-04-27T04:00:42.241 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:41.651+0000 7f1c56ffd700 1 -- 172.21.15.162:0/1752198451 <== mgr.14120 v2:172.21.15.162:6800/8067198 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+51 (secure 0 0 0) 0x7f1c68114400 con 0x7f1c50039ff0 2024-04-27T04:00:42.242 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:41.651+0000 7f1c6cd01700 1 -- 172.21.15.162:0/1752198451 >> [v2:172.21.15.162:6800/8067198,v1:172.21.15.162:6801/8067198] conn(0x7f1c50039ff0 msgr2=0x7f1c5003c3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:00:42.242 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:41.651+0000 7f1c6cd01700 1 --2- 172.21.15.162:0/1752198451 >> [v2:172.21.15.162:6800/8067198,v1:172.21.15.162:6801/8067198] conn(0x7f1c50039ff0 0x7f1c5003c3d0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f1c5c0039d0 tx=0x7f1c5c009350 comp rx=0 tx=0).stop 2024-04-27T04:00:42.242 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:41.651+0000 7f1c6cd01700 1 -- 172.21.15.162:0/1752198451 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1c68077b70 msgr2=0x7f1c68113830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:00:42.242 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:41.651+0000 7f1c6cd01700 1 --2- 172.21.15.162:0/1752198451 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1c68077b70 0x7f1c68113830 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f1c58006b00 tx=0x7f1c58009250 comp rx=0 tx=0).stop 2024-04-27T04:00:42.242 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:41.651+0000 7f1c6cd01700 1 -- 172.21.15.162:0/1752198451 shutdown_connections 2024-04-27T04:00:42.242 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:41.651+0000 7f1c6cd01700 1 --2- 172.21.15.162:0/1752198451 >> [v2:172.21.15.162:6800/8067198,v1:172.21.15.162:6801/8067198] conn(0x7f1c50039ff0 0x7f1c5003c3d0 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:00:42.242 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:41.651+0000 7f1c6cd01700 1 --2- 172.21.15.162:0/1752198451 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1c68077b70 0x7f1c68113830 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:00:42.242 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:41.651+0000 7f1c6cd01700 1 --2- 172.21.15.162:0/1752198451 >> [v2:172.21.15.162:6800/1480340393,v1:172.21.15.162:6801/1480340393] conn(0x7f1c50036850 0x7f1c50038d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:00:42.242 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:41.651+0000 7f1c6cd01700 1 -- 172.21.15.162:0/1752198451 >> 172.21.15.162:0/1752198451 conn(0x7f1c680fdfb0 msgr2=0x7f1c680feaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:00:42.242 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:41.651+0000 7f1c6cd01700 1 -- 172.21.15.162:0/1752198451 shutdown_connections 2024-04-27T04:00:42.242 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:41.651+0000 7f1c6cd01700 1 -- 172.21.15.162:0/1752198451 wait complete. 2024-04-27T04:00:42.242 INFO:teuthology.orchestra.run.smithi162.stdout:mgr epoch 5 is available 2024-04-27T04:00:42.243 INFO:teuthology.orchestra.run.smithi162.stdout:Setting orchestrator backend to cephadm... 2024-04-27T04:00:43.124 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:00:42 smithi162 ceph-mon[26841]: from='client.14124 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-04-27T04:00:43.125 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:00:42 smithi162 ceph-mon[26841]: from='client.14124 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-04-27T04:00:43.125 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:00:42 smithi162 ceph-mon[26841]: [27/Apr/2024:04:00:41] ENGINE Bus STARTING 2024-04-27T04:00:43.125 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:00:42 smithi162 ceph-mon[26841]: [27/Apr/2024:04:00:41] ENGINE Serving on https://172.21.15.162:7150 2024-04-27T04:00:43.125 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:00:42 smithi162 ceph-mon[26841]: [27/Apr/2024:04:00:41] ENGINE Bus STARTED 2024-04-27T04:00:43.125 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:00:42 smithi162 ceph-mon[26841]: from='mgr.14120 172.21.15.162:0/1771817378' entity='mgr.smithi162.oennfn' 2024-04-27T04:00:43.125 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:00:42 smithi162 ceph-mon[26841]: from='mgr.14120 172.21.15.162:0/1771817378' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T04:00:43.125 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:00:42 smithi162 ceph-mon[26841]: from='mgr.14120 172.21.15.162:0/1771817378' entity='mgr.smithi162.oennfn' 2024-04-27T04:00:43.125 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:00:42 smithi162 ceph-mon[26841]: from='mgr.14120 172.21.15.162:0/1771817378' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T04:00:43.392 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:42.582+0000 7f81079fc700 1 Processor -- start 2024-04-27T04:00:43.392 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:42.582+0000 7f81079fc700 1 -- start start 2024-04-27T04:00:43.392 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:42.582+0000 7f81079fc700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8100103ed0 0x7f81001042e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:00:43.392 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:42.582+0000 7f81079fc700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8100070d90 con 0x7f8100103ed0 2024-04-27T04:00:43.393 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:42.583+0000 7f8105798700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8100103ed0 0x7f81001042e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:00:43.393 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:42.583+0000 7f8105798700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8100103ed0 0x7f81001042e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:54262/0 (socket says 172.21.15.162:54262) 2024-04-27T04:00:43.393 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:42.583+0000 7f8105798700 1 -- 172.21.15.162:0/1421958507 learned_addr learned my addr 172.21.15.162:0/1421958507 (peer_addr_for_me v2:172.21.15.162:0/0) 2024-04-27T04:00:43.393 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:42.583+0000 7f8105798700 1 -- 172.21.15.162:0/1421958507 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8100104820 con 0x7f8100103ed0 2024-04-27T04:00:43.393 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:42.583+0000 7f8105798700 1 --2- 172.21.15.162:0/1421958507 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8100103ed0 0x7f81001042e0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f80fc0098e0 tx=0x7f80fc009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=600559c87e7ef68c server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:00:43.393 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:42.583+0000 7f80f7fff700 1 -- 172.21.15.162:0/1421958507 <== mon.0 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f80fc00b860 con 0x7f8100103ed0 2024-04-27T04:00:43.393 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:42.583+0000 7f80f7fff700 1 -- 172.21.15.162:0/1421958507 <== mon.0 v2:172.21.15.162:3300/0 2 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7f80fc00be50 con 0x7f8100103ed0 2024-04-27T04:00:43.393 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:42.583+0000 7f80f7fff700 1 -- 172.21.15.162:0/1421958507 <== mon.0 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f80fc010380 con 0x7f8100103ed0 2024-04-27T04:00:43.393 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:42.584+0000 7f81079fc700 1 -- 172.21.15.162:0/1421958507 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8100103ed0 msgr2=0x7f81001042e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:00:43.393 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:42.584+0000 7f81079fc700 1 --2- 172.21.15.162:0/1421958507 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8100103ed0 0x7f81001042e0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f80fc0098e0 tx=0x7f80fc009250 comp rx=0 tx=0).stop 2024-04-27T04:00:43.393 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:42.584+0000 7f81079fc700 1 -- 172.21.15.162:0/1421958507 shutdown_connections 2024-04-27T04:00:43.393 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:42.584+0000 7f81079fc700 1 --2- 172.21.15.162:0/1421958507 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8100103ed0 0x7f81001042e0 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:00:43.393 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:42.584+0000 7f81079fc700 1 -- 172.21.15.162:0/1421958507 >> 172.21.15.162:0/1421958507 conn(0x7f81000fc6f0 msgr2=0x7f81000feb00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:00:43.394 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:42.584+0000 7f81079fc700 1 -- 172.21.15.162:0/1421958507 shutdown_connections 2024-04-27T04:00:43.394 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:42.584+0000 7f81079fc700 1 -- 172.21.15.162:0/1421958507 wait complete. 2024-04-27T04:00:43.394 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:42.584+0000 7f81079fc700 1 Processor -- start 2024-04-27T04:00:43.394 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:42.584+0000 7f81079fc700 1 -- start start 2024-04-27T04:00:43.394 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:42.584+0000 7f81079fc700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8100103ed0 0x7f81001115c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:00:43.394 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:42.584+0000 7f81079fc700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8100111b00 con 0x7f8100103ed0 2024-04-27T04:00:43.394 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:42.585+0000 7f8105798700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8100103ed0 0x7f81001115c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:00:43.394 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:42.585+0000 7f8105798700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8100103ed0 0x7f81001115c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:54270/0 (socket says 172.21.15.162:54270) 2024-04-27T04:00:43.394 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:42.585+0000 7f8105798700 1 -- 172.21.15.162:0/2735142963 learned_addr learned my addr 172.21.15.162:0/2735142963 (peer_addr_for_me v2:172.21.15.162:0/0) 2024-04-27T04:00:43.394 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:42.585+0000 7f8105798700 1 -- 172.21.15.162:0/2735142963 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f80fc0094f0 con 0x7f8100103ed0 2024-04-27T04:00:43.394 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:42.585+0000 7f8105798700 1 --2- 172.21.15.162:0/2735142963 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8100103ed0 0x7f81001115c0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f80fc00b440 tx=0x7f80fc009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:00:43.394 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:42.585+0000 7f80f5ffb700 1 -- 172.21.15.162:0/2735142963 <== mon.0 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f80fc010760 con 0x7f8100103ed0 2024-04-27T04:00:43.394 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:42.585+0000 7f80f5ffb700 1 -- 172.21.15.162:0/2735142963 <== mon.0 v2:172.21.15.162:3300/0 2 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7f80fc010d50 con 0x7f8100103ed0 2024-04-27T04:00:43.394 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:42.585+0000 7f80f5ffb700 1 -- 172.21.15.162:0/2735142963 <== mon.0 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f80fc01ae20 con 0x7f8100103ed0 2024-04-27T04:00:43.395 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:42.585+0000 7f81079fc700 1 -- 172.21.15.162:0/2735142963 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8100111d00 con 0x7f8100103ed0 2024-04-27T04:00:43.395 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:42.585+0000 7f81079fc700 1 -- 172.21.15.162:0/2735142963 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8100112870 con 0x7f8100103ed0 2024-04-27T04:00:43.395 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:42.586+0000 7f80f5ffb700 1 -- 172.21.15.162:0/2735142963 <== mon.0 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 7) v1 ==== 43270+0+0 (secure 0 0 0) 0x7f80fc0108c0 con 0x7f8100103ed0 2024-04-27T04:00:43.395 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:42.586+0000 7f80f5ffb700 1 --2- 172.21.15.162:0/2735142963 >> [v2:172.21.15.162:6800/8067198,v1:172.21.15.162:6801/8067198] conn(0x7f80ec036770 0x7f80ec038c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:00:43.395 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:42.586+0000 7f81079fc700 1 -- 172.21.15.162:0/2735142963 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f810005f5c0 con 0x7f8100103ed0 2024-04-27T04:00:43.395 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:42.586+0000 7f8104f97700 1 --2- 172.21.15.162:0/2735142963 >> [v2:172.21.15.162:6800/8067198,v1:172.21.15.162:6801/8067198] conn(0x7f80ec036770 0x7f80ec038c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:00:43.395 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:42.586+0000 7f80f5ffb700 1 -- 172.21.15.162:0/2735142963 <== mon.0 v2:172.21.15.162:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 936+0+0 (secure 0 0 0) 0x7f80fc04a430 con 0x7f8100103ed0 2024-04-27T04:00:43.395 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:42.587+0000 7f8104f97700 1 --2- 172.21.15.162:0/2735142963 >> [v2:172.21.15.162:6800/8067198,v1:172.21.15.162:6801/8067198] conn(0x7f80ec036770 0x7f80ec038c20 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f80f0009990 tx=0x7f80f0006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:00:43.395 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:42.591+0000 7f80f5ffb700 1 -- 172.21.15.162:0/2735142963 <== mon.0 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f80fc02e440 con 0x7f8100103ed0 2024-04-27T04:00:43.395 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:42.789+0000 7f81079fc700 1 -- 172.21.15.162:0/2735142963 --> [v2:172.21.15.162:6800/8067198,v1:172.21.15.162:6801/8067198] -- mgr_command(tid 0: {"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}) v1 -- 0x7f81000fd8b0 con 0x7f80ec036770 2024-04-27T04:00:43.395 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:42.796+0000 7f80f5ffb700 1 -- 172.21.15.162:0/2735142963 <== mgr.14120 v2:172.21.15.162:6800/8067198 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f81000fd8b0 con 0x7f80ec036770 2024-04-27T04:00:43.395 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:42.799+0000 7f81079fc700 1 -- 172.21.15.162:0/2735142963 >> [v2:172.21.15.162:6800/8067198,v1:172.21.15.162:6801/8067198] conn(0x7f80ec036770 msgr2=0x7f80ec038c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:00:43.395 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:42.799+0000 7f81079fc700 1 --2- 172.21.15.162:0/2735142963 >> [v2:172.21.15.162:6800/8067198,v1:172.21.15.162:6801/8067198] conn(0x7f80ec036770 0x7f80ec038c20 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f80f0009990 tx=0x7f80f0006e40 comp rx=0 tx=0).stop 2024-04-27T04:00:43.396 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:42.799+0000 7f81079fc700 1 -- 172.21.15.162:0/2735142963 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8100103ed0 msgr2=0x7f81001115c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:00:43.396 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:42.799+0000 7f81079fc700 1 --2- 172.21.15.162:0/2735142963 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8100103ed0 0x7f81001115c0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f80fc00b440 tx=0x7f80fc009250 comp rx=0 tx=0).stop 2024-04-27T04:00:43.396 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:42.799+0000 7f81079fc700 1 -- 172.21.15.162:0/2735142963 shutdown_connections 2024-04-27T04:00:43.396 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:42.799+0000 7f81079fc700 1 --2- 172.21.15.162:0/2735142963 >> [v2:172.21.15.162:6800/8067198,v1:172.21.15.162:6801/8067198] conn(0x7f80ec036770 0x7f80ec038c20 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:00:43.396 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:42.799+0000 7f81079fc700 1 --2- 172.21.15.162:0/2735142963 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8100103ed0 0x7f81001115c0 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:00:43.396 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:42.799+0000 7f81079fc700 1 -- 172.21.15.162:0/2735142963 >> 172.21.15.162:0/2735142963 conn(0x7f81000fc6f0 msgr2=0x7f81000fd1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:00:43.396 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:42.799+0000 7f81079fc700 1 -- 172.21.15.162:0/2735142963 shutdown_connections 2024-04-27T04:00:43.396 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:42.799+0000 7f81079fc700 1 -- 172.21.15.162:0/2735142963 wait complete. 2024-04-27T04:00:44.005 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:00:43 smithi162 ceph-mon[26841]: from='client.14132 -' entity='client.admin' cmd=[{"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}]: dispatch 2024-04-27T04:00:44.005 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:00:43 smithi162 ceph-mon[26841]: mgrmap e8: smithi162.oennfn(active, since 2s) 2024-04-27T04:00:44.556 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stdout value unchanged 2024-04-27T04:00:44.556 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:43.807+0000 7fc24f106700 1 Processor -- start 2024-04-27T04:00:44.556 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:43.807+0000 7fc24f106700 1 -- start start 2024-04-27T04:00:44.556 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:43.807+0000 7fc24f106700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc248102810 0x7fc248102c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:00:44.557 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:43.807+0000 7fc24f106700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc248103160 con 0x7fc248102810 2024-04-27T04:00:44.557 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:43.807+0000 7fc24cea2700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc248102810 0x7fc248102c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:00:44.557 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:43.807+0000 7fc24cea2700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc248102810 0x7fc248102c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:47692/0 (socket says 172.21.15.162:47692) 2024-04-27T04:00:44.557 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:43.807+0000 7fc24cea2700 1 -- 172.21.15.162:0/2779712985 learned_addr learned my addr 172.21.15.162:0/2779712985 (peer_addr_for_me v2:172.21.15.162:0/0) 2024-04-27T04:00:44.557 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:43.807+0000 7fc24cea2700 1 -- 172.21.15.162:0/2779712985 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc2481032a0 con 0x7fc248102810 2024-04-27T04:00:44.557 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:43.807+0000 7fc24cea2700 1 --2- 172.21.15.162:0/2779712985 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc248102810 0x7fc248102c20 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fc23c0098e0 tx=0x7fc23c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3062d76610cc2a6a server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:00:44.557 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:43.808+0000 7fc2477fe700 1 -- 172.21.15.162:0/2779712985 <== mon.0 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fc23c00b860 con 0x7fc248102810 2024-04-27T04:00:44.557 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:43.808+0000 7fc2477fe700 1 -- 172.21.15.162:0/2779712985 <== mon.0 v2:172.21.15.162:3300/0 2 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7fc23c00be50 con 0x7fc248102810 2024-04-27T04:00:44.557 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:43.808+0000 7fc2477fe700 1 -- 172.21.15.162:0/2779712985 <== mon.0 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fc23c010380 con 0x7fc248102810 2024-04-27T04:00:44.557 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:43.808+0000 7fc24f106700 1 -- 172.21.15.162:0/2779712985 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc248102810 msgr2=0x7fc248102c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:00:44.557 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:43.808+0000 7fc24f106700 1 --2- 172.21.15.162:0/2779712985 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc248102810 0x7fc248102c20 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fc23c0098e0 tx=0x7fc23c009250 comp rx=0 tx=0).stop 2024-04-27T04:00:44.557 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:43.808+0000 7fc24f106700 1 -- 172.21.15.162:0/2779712985 shutdown_connections 2024-04-27T04:00:44.557 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:43.808+0000 7fc24f106700 1 --2- 172.21.15.162:0/2779712985 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc248102810 0x7fc248102c20 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:00:44.557 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:43.808+0000 7fc24f106700 1 -- 172.21.15.162:0/2779712985 >> 172.21.15.162:0/2779712985 conn(0x7fc2480fdfd0 msgr2=0x7fc2481003e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:00:44.557 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:43.808+0000 7fc24f106700 1 -- 172.21.15.162:0/2779712985 shutdown_connections 2024-04-27T04:00:44.558 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:43.808+0000 7fc24f106700 1 -- 172.21.15.162:0/2779712985 wait complete. 2024-04-27T04:00:44.558 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:43.809+0000 7fc24f106700 1 Processor -- start 2024-04-27T04:00:44.558 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:43.809+0000 7fc24f106700 1 -- start start 2024-04-27T04:00:44.558 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:43.809+0000 7fc24f106700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc248102810 0x7fc248078c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:00:44.558 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:43.809+0000 7fc24f106700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc248079160 con 0x7fc248102810 2024-04-27T04:00:44.558 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:43.809+0000 7fc24cea2700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc248102810 0x7fc248078c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:00:44.558 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:43.809+0000 7fc24cea2700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc248102810 0x7fc248078c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:47702/0 (socket says 172.21.15.162:47702) 2024-04-27T04:00:44.558 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:43.809+0000 7fc24cea2700 1 -- 172.21.15.162:0/706487812 learned_addr learned my addr 172.21.15.162:0/706487812 (peer_addr_for_me v2:172.21.15.162:0/0) 2024-04-27T04:00:44.558 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:43.809+0000 7fc24cea2700 1 -- 172.21.15.162:0/706487812 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc23c0094f0 con 0x7fc248102810 2024-04-27T04:00:44.558 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:43.810+0000 7fc24cea2700 1 --2- 172.21.15.162:0/706487812 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc248102810 0x7fc248078c20 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fc23c00ba90 tx=0x7fc23c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:00:44.558 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:43.810+0000 7fc2457fa700 1 -- 172.21.15.162:0/706487812 <== mon.0 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fc23c0107c0 con 0x7fc248102810 2024-04-27T04:00:44.558 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:43.810+0000 7fc2457fa700 1 -- 172.21.15.162:0/706487812 <== mon.0 v2:172.21.15.162:3300/0 2 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7fc23c010db0 con 0x7fc248102810 2024-04-27T04:00:44.558 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:43.810+0000 7fc24f106700 1 -- 172.21.15.162:0/706487812 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc2480771d0 con 0x7fc248102810 2024-04-27T04:00:44.558 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:43.810+0000 7fc2457fa700 1 -- 172.21.15.162:0/706487812 <== mon.0 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fc23c01add0 con 0x7fc248102810 2024-04-27T04:00:44.559 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:43.810+0000 7fc24f106700 1 -- 172.21.15.162:0/706487812 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc2480776c0 con 0x7fc248102810 2024-04-27T04:00:44.559 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:43.811+0000 7fc2457fa700 1 -- 172.21.15.162:0/706487812 <== mon.0 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 8) v1 ==== 43376+0+0 (secure 0 0 0) 0x7fc23c02e440 con 0x7fc248102810 2024-04-27T04:00:44.559 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:43.811+0000 7fc2457fa700 1 --2- 172.21.15.162:0/706487812 >> [v2:172.21.15.162:6800/8067198,v1:172.21.15.162:6801/8067198] conn(0x7fc238036850 0x7fc238038d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:00:44.559 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:43.811+0000 7fc24f106700 1 -- 172.21.15.162:0/706487812 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc24804bed0 con 0x7fc248102810 2024-04-27T04:00:44.559 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:43.811+0000 7fc2457fa700 1 -- 172.21.15.162:0/706487812 <== mon.0 v2:172.21.15.162:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 936+0+0 (secure 0 0 0) 0x7fc23c04a710 con 0x7fc248102810 2024-04-27T04:00:44.559 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:43.811+0000 7fc247fff700 1 --2- 172.21.15.162:0/706487812 >> [v2:172.21.15.162:6800/8067198,v1:172.21.15.162:6801/8067198] conn(0x7fc238036850 0x7fc238038d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:00:44.559 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:43.811+0000 7fc247fff700 1 --2- 172.21.15.162:0/706487812 >> [v2:172.21.15.162:6800/8067198,v1:172.21.15.162:6801/8067198] conn(0x7fc238036850 0x7fc238038d00 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fc234009990 tx=0x7fc234006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:00:44.559 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:43.815+0000 7fc2457fa700 1 -- 172.21.15.162:0/706487812 <== mon.0 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7fc23c023d20 con 0x7fc248102810 2024-04-27T04:00:44.559 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:44.003+0000 7fc24f106700 1 -- 172.21.15.162:0/706487812 --> [v2:172.21.15.162:6800/8067198,v1:172.21.15.162:6801/8067198] -- mgr_command(tid 0: {"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}) v1 -- 0x7fc248077dd0 con 0x7fc238036850 2024-04-27T04:00:44.559 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:44.004+0000 7fc2457fa700 1 -- 172.21.15.162:0/706487812 <== mgr.14120 v2:172.21.15.162:6800/8067198 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+16 (secure 0 0 0) 0x7fc248077dd0 con 0x7fc238036850 2024-04-27T04:00:44.559 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:44.006+0000 7fc24f106700 1 -- 172.21.15.162:0/706487812 >> [v2:172.21.15.162:6800/8067198,v1:172.21.15.162:6801/8067198] conn(0x7fc238036850 msgr2=0x7fc238038d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:00:44.559 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:44.006+0000 7fc24f106700 1 --2- 172.21.15.162:0/706487812 >> [v2:172.21.15.162:6800/8067198,v1:172.21.15.162:6801/8067198] conn(0x7fc238036850 0x7fc238038d00 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fc234009990 tx=0x7fc234006e40 comp rx=0 tx=0).stop 2024-04-27T04:00:44.559 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:44.006+0000 7fc24f106700 1 -- 172.21.15.162:0/706487812 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc248102810 msgr2=0x7fc248078c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:00:44.559 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:44.006+0000 7fc24f106700 1 --2- 172.21.15.162:0/706487812 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc248102810 0x7fc248078c20 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fc23c00ba90 tx=0x7fc23c009250 comp rx=0 tx=0).stop 2024-04-27T04:00:44.560 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:44.006+0000 7fc24f106700 1 -- 172.21.15.162:0/706487812 shutdown_connections 2024-04-27T04:00:44.560 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:44.006+0000 7fc24f106700 1 --2- 172.21.15.162:0/706487812 >> [v2:172.21.15.162:6800/8067198,v1:172.21.15.162:6801/8067198] conn(0x7fc238036850 0x7fc238038d00 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:00:44.560 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:44.006+0000 7fc24f106700 1 --2- 172.21.15.162:0/706487812 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc248102810 0x7fc248078c20 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:00:44.560 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:44.006+0000 7fc24f106700 1 -- 172.21.15.162:0/706487812 >> 172.21.15.162:0/706487812 conn(0x7fc2480fdfd0 msgr2=0x7fc2480feaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:00:44.560 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:44.006+0000 7fc24f106700 1 -- 172.21.15.162:0/706487812 shutdown_connections 2024-04-27T04:00:44.560 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:44.006+0000 7fc24f106700 1 -- 172.21.15.162:0/706487812 wait complete. 2024-04-27T04:00:44.560 INFO:teuthology.orchestra.run.smithi162.stdout:Generating ssh key... 2024-04-27T04:00:44.943 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:00:44 smithi162 ceph-mon[26841]: from='client.14134 -' entity='client.admin' cmd=[{"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}]: dispatch 2024-04-27T04:00:45.813 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:44.955+0000 7fec8adb8700 1 Processor -- start 2024-04-27T04:00:45.814 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:44.955+0000 7fec8adb8700 1 -- start start 2024-04-27T04:00:45.814 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:44.955+0000 7fec8adb8700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fec841016f0 0x7fec84101b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:00:45.814 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:44.955+0000 7fec8adb8700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec84102040 con 0x7fec841016f0 2024-04-27T04:00:45.814 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:44.956+0000 7fec88b54700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fec841016f0 0x7fec84101b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:00:45.814 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:44.956+0000 7fec88b54700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fec841016f0 0x7fec84101b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:47710/0 (socket says 172.21.15.162:47710) 2024-04-27T04:00:45.814 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:44.956+0000 7fec88b54700 1 -- 172.21.15.162:0/2730740017 learned_addr learned my addr 172.21.15.162:0/2730740017 (peer_addr_for_me v2:172.21.15.162:0/0) 2024-04-27T04:00:45.814 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:44.956+0000 7fec88b54700 1 -- 172.21.15.162:0/2730740017 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec84102180 con 0x7fec841016f0 2024-04-27T04:00:45.814 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:44.956+0000 7fec88b54700 1 --2- 172.21.15.162:0/2730740017 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fec841016f0 0x7fec84101b00 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fec7c009e50 tx=0x7fec7c009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bc39b80071da3fe8 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:00:45.814 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:44.956+0000 7fec837fe700 1 -- 172.21.15.162:0/2730740017 <== mon.0 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fec7c0092e0 con 0x7fec841016f0 2024-04-27T04:00:45.814 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:44.956+0000 7fec837fe700 1 -- 172.21.15.162:0/2730740017 <== mon.0 v2:172.21.15.162:3300/0 2 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7fec7c012430 con 0x7fec841016f0 2024-04-27T04:00:45.815 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:44.956+0000 7fec837fe700 1 -- 172.21.15.162:0/2730740017 <== mon.0 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fec7c0075c0 con 0x7fec841016f0 2024-04-27T04:00:45.815 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:44.957+0000 7fec8adb8700 1 -- 172.21.15.162:0/2730740017 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fec841016f0 msgr2=0x7fec84101b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:00:45.815 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:44.957+0000 7fec8adb8700 1 --2- 172.21.15.162:0/2730740017 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fec841016f0 0x7fec84101b00 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fec7c009e50 tx=0x7fec7c009840 comp rx=0 tx=0).stop 2024-04-27T04:00:45.815 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:44.957+0000 7fec8adb8700 1 -- 172.21.15.162:0/2730740017 shutdown_connections 2024-04-27T04:00:45.815 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:44.957+0000 7fec8adb8700 1 --2- 172.21.15.162:0/2730740017 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fec841016f0 0x7fec84101b00 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:00:45.815 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:44.957+0000 7fec8adb8700 1 -- 172.21.15.162:0/2730740017 >> 172.21.15.162:0/2730740017 conn(0x7fec840fced0 msgr2=0x7fec840ff2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:00:45.815 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:44.957+0000 7fec8adb8700 1 -- 172.21.15.162:0/2730740017 shutdown_connections 2024-04-27T04:00:45.815 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:44.957+0000 7fec8adb8700 1 -- 172.21.15.162:0/2730740017 wait complete. 2024-04-27T04:00:45.815 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:44.957+0000 7fec8adb8700 1 Processor -- start 2024-04-27T04:00:45.815 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:44.957+0000 7fec8adb8700 1 -- start start 2024-04-27T04:00:45.815 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:44.958+0000 7fec8adb8700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fec841016f0 0x7fec84076b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:00:45.815 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:44.958+0000 7fec8adb8700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec84114280 con 0x7fec841016f0 2024-04-27T04:00:45.815 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:44.958+0000 7fec88b54700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fec841016f0 0x7fec84076b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:00:45.815 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:44.958+0000 7fec88b54700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fec841016f0 0x7fec84076b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:47726/0 (socket says 172.21.15.162:47726) 2024-04-27T04:00:45.816 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:44.958+0000 7fec88b54700 1 -- 172.21.15.162:0/3956789267 learned_addr learned my addr 172.21.15.162:0/3956789267 (peer_addr_for_me v2:172.21.15.162:0/0) 2024-04-27T04:00:45.816 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:44.958+0000 7fec88b54700 1 -- 172.21.15.162:0/3956789267 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec7c009ae0 con 0x7fec841016f0 2024-04-27T04:00:45.816 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:44.958+0000 7fec88b54700 1 --2- 172.21.15.162:0/3956789267 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fec841016f0 0x7fec84076b60 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fec7c009fd0 tx=0x7fec7c009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:00:45.816 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:44.958+0000 7fec817fa700 1 -- 172.21.15.162:0/3956789267 <== mon.0 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fec7c0092e0 con 0x7fec841016f0 2024-04-27T04:00:45.816 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:44.958+0000 7fec817fa700 1 -- 172.21.15.162:0/3956789267 <== mon.0 v2:172.21.15.162:3300/0 2 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7fec7c007cc0 con 0x7fec841016f0 2024-04-27T04:00:45.816 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:44.958+0000 7fec8adb8700 1 -- 172.21.15.162:0/3956789267 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fec84114420 con 0x7fec841016f0 2024-04-27T04:00:45.816 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:44.958+0000 7fec817fa700 1 -- 172.21.15.162:0/3956789267 <== mon.0 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fec7c01c9a0 con 0x7fec841016f0 2024-04-27T04:00:45.816 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:44.958+0000 7fec8adb8700 1 -- 172.21.15.162:0/3956789267 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fec84114910 con 0x7fec841016f0 2024-04-27T04:00:45.816 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:44.959+0000 7fec817fa700 1 -- 172.21.15.162:0/3956789267 <== mon.0 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 8) v1 ==== 43376+0+0 (secure 0 0 0) 0x7fec7c019070 con 0x7fec841016f0 2024-04-27T04:00:45.816 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:44.959+0000 7fec8adb8700 1 -- 172.21.15.162:0/3956789267 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fec8404bed0 con 0x7fec841016f0 2024-04-27T04:00:45.816 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:44.959+0000 7fec817fa700 1 --2- 172.21.15.162:0/3956789267 >> [v2:172.21.15.162:6800/8067198,v1:172.21.15.162:6801/8067198] conn(0x7fec70036850 0x7fec70038d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:00:45.816 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:44.959+0000 7fec817fa700 1 -- 172.21.15.162:0/3956789267 <== mon.0 v2:172.21.15.162:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 936+0+0 (secure 0 0 0) 0x7fec7c049d90 con 0x7fec841016f0 2024-04-27T04:00:45.816 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:44.959+0000 7fec83fff700 1 --2- 172.21.15.162:0/3956789267 >> [v2:172.21.15.162:6800/8067198,v1:172.21.15.162:6801/8067198] conn(0x7fec70036850 0x7fec70038d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:00:45.816 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:44.960+0000 7fec83fff700 1 --2- 172.21.15.162:0/3956789267 >> [v2:172.21.15.162:6800/8067198,v1:172.21.15.162:6801/8067198] conn(0x7fec70036850 0x7fec70038d00 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fec74009990 tx=0x7fec74006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:00:45.817 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:44.964+0000 7fec817fa700 1 -- 172.21.15.162:0/3956789267 <== mon.0 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7fec7c019910 con 0x7fec841016f0 2024-04-27T04:00:45.817 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:45.147+0000 7fec8adb8700 1 -- 172.21.15.162:0/3956789267 --> [v2:172.21.15.162:6800/8067198,v1:172.21.15.162:6801/8067198] -- mgr_command(tid 0: {"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}) v1 -- 0x7fec840fe0c0 con 0x7fec70036850 2024-04-27T04:00:45.817 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:45.293+0000 7fec817fa700 1 -- 172.21.15.162:0/3956789267 <== mgr.14120 v2:172.21.15.162:6800/8067198 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7fec840fe0c0 con 0x7fec70036850 2024-04-27T04:00:45.817 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:45.296+0000 7fec8adb8700 1 -- 172.21.15.162:0/3956789267 >> [v2:172.21.15.162:6800/8067198,v1:172.21.15.162:6801/8067198] conn(0x7fec70036850 msgr2=0x7fec70038d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:00:45.817 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:45.296+0000 7fec8adb8700 1 --2- 172.21.15.162:0/3956789267 >> [v2:172.21.15.162:6800/8067198,v1:172.21.15.162:6801/8067198] conn(0x7fec70036850 0x7fec70038d00 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fec74009990 tx=0x7fec74006e40 comp rx=0 tx=0).stop 2024-04-27T04:00:45.817 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:45.296+0000 7fec8adb8700 1 -- 172.21.15.162:0/3956789267 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fec841016f0 msgr2=0x7fec84076b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:00:45.817 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:45.296+0000 7fec8adb8700 1 --2- 172.21.15.162:0/3956789267 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fec841016f0 0x7fec84076b60 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fec7c009fd0 tx=0x7fec7c009840 comp rx=0 tx=0).stop 2024-04-27T04:00:45.817 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:45.296+0000 7fec8adb8700 1 -- 172.21.15.162:0/3956789267 shutdown_connections 2024-04-27T04:00:45.817 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:45.296+0000 7fec8adb8700 1 --2- 172.21.15.162:0/3956789267 >> [v2:172.21.15.162:6800/8067198,v1:172.21.15.162:6801/8067198] conn(0x7fec70036850 0x7fec70038d00 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:00:45.817 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:45.296+0000 7fec8adb8700 1 --2- 172.21.15.162:0/3956789267 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fec841016f0 0x7fec84076b60 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:00:45.817 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:45.296+0000 7fec8adb8700 1 -- 172.21.15.162:0/3956789267 >> 172.21.15.162:0/3956789267 conn(0x7fec840fced0 msgr2=0x7fec840fd9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:00:45.817 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:45.296+0000 7fec8adb8700 1 -- 172.21.15.162:0/3956789267 shutdown_connections 2024-04-27T04:00:45.817 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:45.296+0000 7fec8adb8700 1 -- 172.21.15.162:0/3956789267 wait complete. 2024-04-27T04:00:46.139 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:00:45 smithi162 ceph-mon[26841]: from='client.14136 -' entity='client.admin' cmd=[{"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}]: dispatch 2024-04-27T04:00:46.139 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:00:45 smithi162 ceph-mon[26841]: Generating ssh key... 2024-04-27T04:00:46.139 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:00:45 smithi162 ceph-mon[26841]: from='mgr.14120 172.21.15.162:0/1771817378' entity='mgr.smithi162.oennfn' 2024-04-27T04:00:46.139 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:00:45 smithi162 ceph-mon[26841]: from='mgr.14120 172.21.15.162:0/1771817378' entity='mgr.smithi162.oennfn' 2024-04-27T04:00:47.004 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stdout ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCsyNs/PPSvGfQvecvlcm0yIAIOC8AiCY43p0mnQkNKpgbEY8ztgp8kaJkd7OAHl1xOiuers1f+vwjTFg5H5BrK5lxKyQVNJKI05Uwm0Gbjig8QrRmDCLRU/6PxKqYFF68a8JClGlZFFtBSo7/IgHqj/KOzREI44MfnCDF8zODIiyeGIr9r2hydOb81UKmNvMbL2NdEb1aiUxjcqgDX1tec6uMZ0ztzyP0lE0l/TFBbNGW0WXYpD179+UMhhBVUUJuLeLdGqmEpEFb+NaPjv2Vj7eE1U7AWNguDCGfqdpWvhrCR0iEgLOXZSDDpsSsZKMWa7NlhUmy98OdbwwZ4O1do/iJYe0nTEeFxCdajSp1Ptk6FfxT7Lo15SBUY3aLHAUgsn90YMC3YTKY1/VetUX5vA8x9cP6i6YplNOwzCWxWH3193OyElJg5/eVRM5WcpC6JYruBz65A8w0HwTFldvLXHaH5SyP5LHkzYfOmGkaaeUVJlApyNccXQt3yp5vGiJk= ceph-7f9750d6-044a-11ef-bc93-c7b262605968 2024-04-27T04:00:47.004 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:46.265+0000 7fdf0eaa1700 1 Processor -- start 2024-04-27T04:00:47.005 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:46.265+0000 7fdf0eaa1700 1 -- start start 2024-04-27T04:00:47.005 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:46.265+0000 7fdf0eaa1700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fdf08100c70 0x7fdf08101080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:00:47.005 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:46.265+0000 7fdf0eaa1700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf081015c0 con 0x7fdf08100c70 2024-04-27T04:00:47.005 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:46.265+0000 7fdf0c83d700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fdf08100c70 0x7fdf08101080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:00:47.005 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:46.265+0000 7fdf0c83d700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fdf08100c70 0x7fdf08101080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:47736/0 (socket says 172.21.15.162:47736) 2024-04-27T04:00:47.005 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:46.266+0000 7fdf0c83d700 1 -- 172.21.15.162:0/3909473878 learned_addr learned my addr 172.21.15.162:0/3909473878 (peer_addr_for_me v2:172.21.15.162:0/0) 2024-04-27T04:00:47.005 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:46.266+0000 7fdf0c83d700 1 -- 172.21.15.162:0/3909473878 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf08101700 con 0x7fdf08100c70 2024-04-27T04:00:47.005 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:46.266+0000 7fdf0c83d700 1 --2- 172.21.15.162:0/3909473878 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fdf08100c70 0x7fdf08101080 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fdf040098e0 tx=0x7fdf04009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e9ed26a079550dfb server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:00:47.005 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:46.266+0000 7fdeff7fe700 1 -- 172.21.15.162:0/3909473878 <== mon.0 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fdf0400b860 con 0x7fdf08100c70 2024-04-27T04:00:47.005 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:46.266+0000 7fdeff7fe700 1 -- 172.21.15.162:0/3909473878 <== mon.0 v2:172.21.15.162:3300/0 2 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7fdf0400be50 con 0x7fdf08100c70 2024-04-27T04:00:47.005 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:46.266+0000 7fdeff7fe700 1 -- 172.21.15.162:0/3909473878 <== mon.0 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fdf04010380 con 0x7fdf08100c70 2024-04-27T04:00:47.006 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:46.267+0000 7fdf0eaa1700 1 -- 172.21.15.162:0/3909473878 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fdf08100c70 msgr2=0x7fdf08101080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:00:47.006 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:46.267+0000 7fdf0eaa1700 1 --2- 172.21.15.162:0/3909473878 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fdf08100c70 0x7fdf08101080 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fdf040098e0 tx=0x7fdf04009250 comp rx=0 tx=0).stop 2024-04-27T04:00:47.006 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:46.267+0000 7fdf0eaa1700 1 -- 172.21.15.162:0/3909473878 shutdown_connections 2024-04-27T04:00:47.006 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:46.267+0000 7fdf0eaa1700 1 --2- 172.21.15.162:0/3909473878 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fdf08100c70 0x7fdf08101080 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:00:47.006 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:46.267+0000 7fdf0eaa1700 1 -- 172.21.15.162:0/3909473878 >> 172.21.15.162:0/3909473878 conn(0x7fdf080fc6f0 msgr2=0x7fdf080feb00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:00:47.006 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:46.267+0000 7fdf0eaa1700 1 -- 172.21.15.162:0/3909473878 shutdown_connections 2024-04-27T04:00:47.006 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:46.267+0000 7fdf0eaa1700 1 -- 172.21.15.162:0/3909473878 wait complete. 2024-04-27T04:00:47.006 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:46.267+0000 7fdf0eaa1700 1 Processor -- start 2024-04-27T04:00:47.006 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:46.267+0000 7fdf0eaa1700 1 -- start start 2024-04-27T04:00:47.006 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:46.268+0000 7fdf0eaa1700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fdf08100c70 0x7fdf081036a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:00:47.007 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:46.268+0000 7fdf0eaa1700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf08103be0 con 0x7fdf08100c70 2024-04-27T04:00:47.007 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:46.268+0000 7fdf0c83d700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fdf08100c70 0x7fdf081036a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:00:47.007 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:46.268+0000 7fdf0c83d700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fdf08100c70 0x7fdf081036a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:47742/0 (socket says 172.21.15.162:47742) 2024-04-27T04:00:47.007 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:46.268+0000 7fdf0c83d700 1 -- 172.21.15.162:0/235301649 learned_addr learned my addr 172.21.15.162:0/235301649 (peer_addr_for_me v2:172.21.15.162:0/0) 2024-04-27T04:00:47.007 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:46.268+0000 7fdf0c83d700 1 -- 172.21.15.162:0/235301649 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf040094f0 con 0x7fdf08100c70 2024-04-27T04:00:47.007 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:46.268+0000 7fdf0c83d700 1 --2- 172.21.15.162:0/235301649 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fdf08100c70 0x7fdf081036a0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fdf0400b440 tx=0x7fdf04009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:00:47.007 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:46.268+0000 7fdefd7fa700 1 -- 172.21.15.162:0/235301649 <== mon.0 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fdf04010760 con 0x7fdf08100c70 2024-04-27T04:00:47.007 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:46.268+0000 7fdf0eaa1700 1 -- 172.21.15.162:0/235301649 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdf08101c50 con 0x7fdf08100c70 2024-04-27T04:00:47.007 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:46.268+0000 7fdefd7fa700 1 -- 172.21.15.162:0/235301649 <== mon.0 v2:172.21.15.162:3300/0 2 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7fdf04010d50 con 0x7fdf08100c70 2024-04-27T04:00:47.007 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:46.268+0000 7fdefd7fa700 1 -- 172.21.15.162:0/235301649 <== mon.0 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fdf0401ae20 con 0x7fdf08100c70 2024-04-27T04:00:47.007 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:46.268+0000 7fdf0eaa1700 1 -- 172.21.15.162:0/235301649 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdf08102140 con 0x7fdf08100c70 2024-04-27T04:00:47.007 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:46.269+0000 7fdefd7fa700 1 -- 172.21.15.162:0/235301649 <== mon.0 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 8) v1 ==== 43376+0+0 (secure 0 0 0) 0x7fdf0402e440 con 0x7fdf08100c70 2024-04-27T04:00:47.007 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:46.269+0000 7fdf0eaa1700 1 -- 172.21.15.162:0/235301649 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdf0804bed0 con 0x7fdf08100c70 2024-04-27T04:00:47.007 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:46.269+0000 7fdefd7fa700 1 --2- 172.21.15.162:0/235301649 >> [v2:172.21.15.162:6800/8067198,v1:172.21.15.162:6801/8067198] conn(0x7fdef0036830 0x7fdef0038ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:00:47.008 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:46.269+0000 7fdefd7fa700 1 -- 172.21.15.162:0/235301649 <== mon.0 v2:172.21.15.162:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 936+0+0 (secure 0 0 0) 0x7fdf0404a5b0 con 0x7fdf08100c70 2024-04-27T04:00:47.008 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:46.270+0000 7fdefffff700 1 --2- 172.21.15.162:0/235301649 >> [v2:172.21.15.162:6800/8067198,v1:172.21.15.162:6801/8067198] conn(0x7fdef0036830 0x7fdef0038ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:00:47.008 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:46.270+0000 7fdefffff700 1 --2- 172.21.15.162:0/235301649 >> [v2:172.21.15.162:6800/8067198,v1:172.21.15.162:6801/8067198] conn(0x7fdef0036830 0x7fdef0038ce0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fdef8009990 tx=0x7fdef8006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:00:47.008 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:46.274+0000 7fdefd7fa700 1 -- 172.21.15.162:0/235301649 <== mon.0 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7fdf04018940 con 0x7fdf08100c70 2024-04-27T04:00:47.008 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:46.460+0000 7fdf0eaa1700 1 -- 172.21.15.162:0/235301649 --> [v2:172.21.15.162:6800/8067198,v1:172.21.15.162:6801/8067198] -- mgr_command(tid 0: {"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}) v1 -- 0x7fdf08102900 con 0x7fdef0036830 2024-04-27T04:00:47.008 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:46.460+0000 7fdefd7fa700 1 -- 172.21.15.162:0/235301649 <== mgr.14120 v2:172.21.15.162:6800/8067198 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+595 (secure 0 0 0) 0x7fdf08102900 con 0x7fdef0036830 2024-04-27T04:00:47.008 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:46.462+0000 7fdf0eaa1700 1 -- 172.21.15.162:0/235301649 >> [v2:172.21.15.162:6800/8067198,v1:172.21.15.162:6801/8067198] conn(0x7fdef0036830 msgr2=0x7fdef0038ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:00:47.008 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:46.462+0000 7fdf0eaa1700 1 --2- 172.21.15.162:0/235301649 >> [v2:172.21.15.162:6800/8067198,v1:172.21.15.162:6801/8067198] conn(0x7fdef0036830 0x7fdef0038ce0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fdef8009990 tx=0x7fdef8006e40 comp rx=0 tx=0).stop 2024-04-27T04:00:47.008 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:46.462+0000 7fdf0eaa1700 1 -- 172.21.15.162:0/235301649 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fdf08100c70 msgr2=0x7fdf081036a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:00:47.008 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:46.462+0000 7fdf0eaa1700 1 --2- 172.21.15.162:0/235301649 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fdf08100c70 0x7fdf081036a0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fdf0400b440 tx=0x7fdf04009250 comp rx=0 tx=0).stop 2024-04-27T04:00:47.008 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:46.463+0000 7fdf0eaa1700 1 -- 172.21.15.162:0/235301649 shutdown_connections 2024-04-27T04:00:47.008 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:46.463+0000 7fdf0eaa1700 1 --2- 172.21.15.162:0/235301649 >> [v2:172.21.15.162:6800/8067198,v1:172.21.15.162:6801/8067198] conn(0x7fdef0036830 0x7fdef0038ce0 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:00:47.008 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:46.463+0000 7fdf0eaa1700 1 --2- 172.21.15.162:0/235301649 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fdf08100c70 0x7fdf081036a0 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:00:47.009 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:46.463+0000 7fdf0eaa1700 1 -- 172.21.15.162:0/235301649 >> 172.21.15.162:0/235301649 conn(0x7fdf080fc6f0 msgr2=0x7fdf080fd1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:00:47.009 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:46.463+0000 7fdf0eaa1700 1 -- 172.21.15.162:0/235301649 shutdown_connections 2024-04-27T04:00:47.009 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:46.463+0000 7fdf0eaa1700 1 -- 172.21.15.162:0/235301649 wait complete. 2024-04-27T04:00:47.009 INFO:teuthology.orchestra.run.smithi162.stdout:Wrote public SSH key to /home/ubuntu/cephtest/ceph.pub 2024-04-27T04:00:47.009 INFO:teuthology.orchestra.run.smithi162.stdout:Adding key to root@localhost authorized_keys... 2024-04-27T04:00:47.009 INFO:teuthology.orchestra.run.smithi162.stdout:Adding host smithi162... 2024-04-27T04:00:47.389 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:00:47 smithi162 ceph-mon[26841]: from='client.14138 -' entity='client.admin' cmd=[{"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}]: dispatch 2024-04-27T04:00:48.465 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:00:48 smithi162 ceph-mon[26841]: from='client.14140 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi162", "addr": "172.21.15.162", "target": ["mon-mgr", ""]}]: dispatch 2024-04-27T04:00:48.465 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:00:48 smithi162 ceph-mon[26841]: Deploying cephadm binary to smithi162 2024-04-27T04:00:50.003 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stdout Added host 'smithi162' with addr '172.21.15.162' 2024-04-27T04:00:50.003 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:47.414+0000 7f5ec485c700 1 Processor -- start 2024-04-27T04:00:50.003 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:47.414+0000 7f5ec485c700 1 -- start start 2024-04-27T04:00:50.003 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:47.414+0000 7f5ec485c700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5ec0100c70 0x7f5ec0101080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:00:50.003 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:47.414+0000 7f5ec485c700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ec01015c0 con 0x7f5ec0100c70 2024-04-27T04:00:50.003 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:47.415+0000 7f5ebe59c700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5ec0100c70 0x7f5ec0101080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:00:50.003 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:47.415+0000 7f5ebe59c700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5ec0100c70 0x7f5ec0101080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:47752/0 (socket says 172.21.15.162:47752) 2024-04-27T04:00:50.003 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:47.415+0000 7f5ebe59c700 1 -- 172.21.15.162:0/2420746369 learned_addr learned my addr 172.21.15.162:0/2420746369 (peer_addr_for_me v2:172.21.15.162:0/0) 2024-04-27T04:00:50.003 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:47.415+0000 7f5ebe59c700 1 -- 172.21.15.162:0/2420746369 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ec0101700 con 0x7f5ec0100c70 2024-04-27T04:00:50.004 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:47.415+0000 7f5ebe59c700 1 --2- 172.21.15.162:0/2420746369 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5ec0100c70 0x7f5ec0101080 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f5eb00098e0 tx=0x7f5eb0009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d45174668e1f8367 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:00:50.004 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:47.415+0000 7f5ebd59a700 1 -- 172.21.15.162:0/2420746369 <== mon.0 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f5eb000b860 con 0x7f5ec0100c70 2024-04-27T04:00:50.004 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:47.415+0000 7f5ebd59a700 1 -- 172.21.15.162:0/2420746369 <== mon.0 v2:172.21.15.162:3300/0 2 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7f5eb000be50 con 0x7f5ec0100c70 2024-04-27T04:00:50.004 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:47.415+0000 7f5ebd59a700 1 -- 172.21.15.162:0/2420746369 <== mon.0 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f5eb0010380 con 0x7f5ec0100c70 2024-04-27T04:00:50.004 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:47.416+0000 7f5ec485c700 1 -- 172.21.15.162:0/2420746369 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5ec0100c70 msgr2=0x7f5ec0101080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:00:50.004 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:47.416+0000 7f5ec485c700 1 --2- 172.21.15.162:0/2420746369 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5ec0100c70 0x7f5ec0101080 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f5eb00098e0 tx=0x7f5eb0009250 comp rx=0 tx=0).stop 2024-04-27T04:00:50.004 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:47.416+0000 7f5ec485c700 1 -- 172.21.15.162:0/2420746369 shutdown_connections 2024-04-27T04:00:50.004 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:47.416+0000 7f5ec485c700 1 --2- 172.21.15.162:0/2420746369 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5ec0100c70 0x7f5ec0101080 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:00:50.004 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:47.416+0000 7f5ec485c700 1 -- 172.21.15.162:0/2420746369 >> 172.21.15.162:0/2420746369 conn(0x7f5ec00fc6f0 msgr2=0x7f5ec00feb00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:00:50.004 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:47.416+0000 7f5ec485c700 1 -- 172.21.15.162:0/2420746369 shutdown_connections 2024-04-27T04:00:50.004 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:47.416+0000 7f5ec485c700 1 -- 172.21.15.162:0/2420746369 wait complete. 2024-04-27T04:00:50.004 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:47.416+0000 7f5ec485c700 1 Processor -- start 2024-04-27T04:00:50.004 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:47.416+0000 7f5ec485c700 1 -- start start 2024-04-27T04:00:50.004 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:47.417+0000 7f5ec485c700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5ec0100c70 0x7f5ec0111570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:00:50.004 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:47.417+0000 7f5ec485c700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ec0111ab0 con 0x7f5ec0100c70 2024-04-27T04:00:50.004 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:47.417+0000 7f5ebe59c700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5ec0100c70 0x7f5ec0111570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:00:50.005 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:47.417+0000 7f5ebe59c700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5ec0100c70 0x7f5ec0111570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:47766/0 (socket says 172.21.15.162:47766) 2024-04-27T04:00:50.005 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:47.417+0000 7f5ebe59c700 1 -- 172.21.15.162:0/1790756184 learned_addr learned my addr 172.21.15.162:0/1790756184 (peer_addr_for_me v2:172.21.15.162:0/0) 2024-04-27T04:00:50.005 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:47.417+0000 7f5ebe59c700 1 -- 172.21.15.162:0/1790756184 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5eb00094f0 con 0x7f5ec0100c70 2024-04-27T04:00:50.005 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:47.417+0000 7f5ebe59c700 1 --2- 172.21.15.162:0/1790756184 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5ec0100c70 0x7f5ec0111570 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f5eb000b440 tx=0x7f5eb0009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:00:50.005 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:47.417+0000 7f5eaeffd700 1 -- 172.21.15.162:0/1790756184 <== mon.0 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f5eb0010760 con 0x7f5ec0100c70 2024-04-27T04:00:50.005 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:47.417+0000 7f5eaeffd700 1 -- 172.21.15.162:0/1790756184 <== mon.0 v2:172.21.15.162:3300/0 2 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7f5eb0010d50 con 0x7f5ec0100c70 2024-04-27T04:00:50.005 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:47.417+0000 7f5eaeffd700 1 -- 172.21.15.162:0/1790756184 <== mon.0 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f5eb001ae20 con 0x7f5ec0100c70 2024-04-27T04:00:50.005 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:47.417+0000 7f5ec485c700 1 -- 172.21.15.162:0/1790756184 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5ec0111cb0 con 0x7f5ec0100c70 2024-04-27T04:00:50.005 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:47.417+0000 7f5ec485c700 1 -- 172.21.15.162:0/1790756184 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5ec0112820 con 0x7f5ec0100c70 2024-04-27T04:00:50.005 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:47.418+0000 7f5eaeffd700 1 -- 172.21.15.162:0/1790756184 <== mon.0 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 8) v1 ==== 43376+0+0 (secure 0 0 0) 0x7f5eb00108c0 con 0x7f5ec0100c70 2024-04-27T04:00:50.005 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:47.418+0000 7f5ec485c700 1 -- 172.21.15.162:0/1790756184 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5ec005f5c0 con 0x7f5ec0100c70 2024-04-27T04:00:50.005 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:47.418+0000 7f5eaeffd700 1 --2- 172.21.15.162:0/1790756184 >> [v2:172.21.15.162:6800/8067198,v1:172.21.15.162:6801/8067198] conn(0x7f5ea8036830 0x7f5ea8038ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:00:50.005 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:47.418+0000 7f5eaeffd700 1 -- 172.21.15.162:0/1790756184 <== mon.0 v2:172.21.15.162:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 936+0+0 (secure 0 0 0) 0x7f5eb004a590 con 0x7f5ec0100c70 2024-04-27T04:00:50.005 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:47.418+0000 7f5ebdd9b700 1 --2- 172.21.15.162:0/1790756184 >> [v2:172.21.15.162:6800/8067198,v1:172.21.15.162:6801/8067198] conn(0x7f5ea8036830 0x7f5ea8038ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:00:50.005 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:47.419+0000 7f5ebdd9b700 1 --2- 172.21.15.162:0/1790756184 >> [v2:172.21.15.162:6800/8067198,v1:172.21.15.162:6801/8067198] conn(0x7f5ea8036830 0x7f5ea8038ce0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f5eb4009990 tx=0x7f5eb4006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:00:50.006 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:47.423+0000 7f5eaeffd700 1 -- 172.21.15.162:0/1790756184 <== mon.0 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f5eb0018530 con 0x7f5ec0100c70 2024-04-27T04:00:50.006 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:47.613+0000 7f5ec485c700 1 -- 172.21.15.162:0/1790756184 --> [v2:172.21.15.162:6800/8067198,v1:172.21.15.162:6801/8067198] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi162", "addr": "172.21.15.162", "target": ["mon-mgr", ""]}) v1 -- 0x7f5ec00fd8e0 con 0x7f5ea8036830 2024-04-27T04:00:50.006 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:49.442+0000 7f5eaeffd700 1 -- 172.21.15.162:0/1790756184 <== mgr.14120 v2:172.21.15.162:6800/8067198 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+49 (secure 0 0 0) 0x7f5ec00fd8e0 con 0x7f5ea8036830 2024-04-27T04:00:50.006 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:49.445+0000 7f5ec485c700 1 -- 172.21.15.162:0/1790756184 >> [v2:172.21.15.162:6800/8067198,v1:172.21.15.162:6801/8067198] conn(0x7f5ea8036830 msgr2=0x7f5ea8038ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:00:50.006 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:49.445+0000 7f5ec485c700 1 --2- 172.21.15.162:0/1790756184 >> [v2:172.21.15.162:6800/8067198,v1:172.21.15.162:6801/8067198] conn(0x7f5ea8036830 0x7f5ea8038ce0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f5eb4009990 tx=0x7f5eb4006e40 comp rx=0 tx=0).stop 2024-04-27T04:00:50.006 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:49.445+0000 7f5ec485c700 1 -- 172.21.15.162:0/1790756184 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5ec0100c70 msgr2=0x7f5ec0111570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:00:50.006 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:49.445+0000 7f5ec485c700 1 --2- 172.21.15.162:0/1790756184 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5ec0100c70 0x7f5ec0111570 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f5eb000b440 tx=0x7f5eb0009250 comp rx=0 tx=0).stop 2024-04-27T04:00:50.006 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:49.445+0000 7f5ec485c700 1 -- 172.21.15.162:0/1790756184 shutdown_connections 2024-04-27T04:00:50.006 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:49.445+0000 7f5ec485c700 1 --2- 172.21.15.162:0/1790756184 >> [v2:172.21.15.162:6800/8067198,v1:172.21.15.162:6801/8067198] conn(0x7f5ea8036830 0x7f5ea8038ce0 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:00:50.006 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:49.445+0000 7f5ec485c700 1 --2- 172.21.15.162:0/1790756184 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5ec0100c70 0x7f5ec0111570 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:00:50.006 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:49.445+0000 7f5ec485c700 1 -- 172.21.15.162:0/1790756184 >> 172.21.15.162:0/1790756184 conn(0x7f5ec00fc6f0 msgr2=0x7f5ec00fd1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:00:50.006 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:49.445+0000 7f5ec485c700 1 -- 172.21.15.162:0/1790756184 shutdown_connections 2024-04-27T04:00:50.006 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:49.445+0000 7f5ec485c700 1 -- 172.21.15.162:0/1790756184 wait complete. 2024-04-27T04:00:50.006 INFO:teuthology.orchestra.run.smithi162.stdout:Deploying mon service with default placement... 2024-04-27T04:00:50.628 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:00:50 smithi162 ceph-mon[26841]: from='mgr.14120 172.21.15.162:0/1771817378' entity='mgr.smithi162.oennfn' 2024-04-27T04:00:50.628 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:00:50 smithi162 ceph-mon[26841]: Added host smithi162 2024-04-27T04:00:50.628 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:00:50 smithi162 ceph-mon[26841]: from='mgr.14120 172.21.15.162:0/1771817378' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T04:00:52.003 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:00:51 smithi162 ceph-mon[26841]: from='client.14142 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "target": ["mon-mgr", ""]}]: dispatch 2024-04-27T04:00:52.003 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:00:51 smithi162 ceph-mon[26841]: Saving service mon spec with placement count:5 2024-04-27T04:00:52.003 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:00:51 smithi162 ceph-mon[26841]: from='mgr.14120 172.21.15.162:0/1771817378' entity='mgr.smithi162.oennfn' 2024-04-27T04:00:52.260 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stdout Scheduled mon update... 2024-04-27T04:00:52.260 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:50.473+0000 7f5bdbaeb700 1 Processor -- start 2024-04-27T04:00:52.260 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:50.473+0000 7f5bdbaeb700 1 -- start start 2024-04-27T04:00:52.260 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:50.473+0000 7f5bdbaeb700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5bd4076b90 0x7f5bd41054e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:00:52.260 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:50.473+0000 7f5bdbaeb700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5bd4105a20 con 0x7f5bd4076b90 2024-04-27T04:00:52.260 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:50.473+0000 7f5bd9887700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5bd4076b90 0x7f5bd41054e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:00:52.260 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:50.473+0000 7f5bd9887700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5bd4076b90 0x7f5bd41054e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:47768/0 (socket says 172.21.15.162:47768) 2024-04-27T04:00:52.260 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:50.473+0000 7f5bd9887700 1 -- 172.21.15.162:0/3630736456 learned_addr learned my addr 172.21.15.162:0/3630736456 (peer_addr_for_me v2:172.21.15.162:0/0) 2024-04-27T04:00:52.261 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:50.474+0000 7f5bd9887700 1 -- 172.21.15.162:0/3630736456 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5bd4105b60 con 0x7f5bd4076b90 2024-04-27T04:00:52.261 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:50.474+0000 7f5bd9887700 1 --2- 172.21.15.162:0/3630736456 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5bd4076b90 0x7f5bd41054e0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f5bd00098e0 tx=0x7f5bd0009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a9e192b9094bcfec server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:00:52.261 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:50.474+0000 7f5bd8885700 1 -- 172.21.15.162:0/3630736456 <== mon.0 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f5bd000b860 con 0x7f5bd4076b90 2024-04-27T04:00:52.261 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:50.474+0000 7f5bd8885700 1 -- 172.21.15.162:0/3630736456 <== mon.0 v2:172.21.15.162:3300/0 2 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7f5bd000be50 con 0x7f5bd4076b90 2024-04-27T04:00:52.261 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:50.474+0000 7f5bd8885700 1 -- 172.21.15.162:0/3630736456 <== mon.0 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f5bd0010380 con 0x7f5bd4076b90 2024-04-27T04:00:52.261 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:50.474+0000 7f5bdbaeb700 1 -- 172.21.15.162:0/3630736456 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5bd4076b90 msgr2=0x7f5bd41054e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:00:52.261 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:50.474+0000 7f5bdbaeb700 1 --2- 172.21.15.162:0/3630736456 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5bd4076b90 0x7f5bd41054e0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f5bd00098e0 tx=0x7f5bd0009250 comp rx=0 tx=0).stop 2024-04-27T04:00:52.261 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:50.474+0000 7f5bdbaeb700 1 -- 172.21.15.162:0/3630736456 shutdown_connections 2024-04-27T04:00:52.261 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:50.474+0000 7f5bdbaeb700 1 --2- 172.21.15.162:0/3630736456 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5bd4076b90 0x7f5bd41054e0 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:00:52.261 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:50.474+0000 7f5bdbaeb700 1 -- 172.21.15.162:0/3630736456 >> 172.21.15.162:0/3630736456 conn(0x7f5bd40fcef0 msgr2=0x7f5bd40ff300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:00:52.261 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:50.475+0000 7f5bdbaeb700 1 -- 172.21.15.162:0/3630736456 shutdown_connections 2024-04-27T04:00:52.261 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:50.475+0000 7f5bdbaeb700 1 -- 172.21.15.162:0/3630736456 wait complete. 2024-04-27T04:00:52.261 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:50.475+0000 7f5bdbaeb700 1 Processor -- start 2024-04-27T04:00:52.261 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:50.475+0000 7f5bdbaeb700 1 -- start start 2024-04-27T04:00:52.261 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:50.475+0000 7f5bdbaeb700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5bd4076b90 0x7f5bd4111610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:00:52.262 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:50.475+0000 7f5bdbaeb700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5bd4111b50 con 0x7f5bd4076b90 2024-04-27T04:00:52.262 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:50.475+0000 7f5bd9887700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5bd4076b90 0x7f5bd4111610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:00:52.262 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:50.475+0000 7f5bd9887700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5bd4076b90 0x7f5bd4111610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:47772/0 (socket says 172.21.15.162:47772) 2024-04-27T04:00:52.262 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:50.475+0000 7f5bd9887700 1 -- 172.21.15.162:0/2799676451 learned_addr learned my addr 172.21.15.162:0/2799676451 (peer_addr_for_me v2:172.21.15.162:0/0) 2024-04-27T04:00:52.262 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:50.476+0000 7f5bd9887700 1 -- 172.21.15.162:0/2799676451 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5bd00094f0 con 0x7f5bd4076b90 2024-04-27T04:00:52.262 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:50.476+0000 7f5bd9887700 1 --2- 172.21.15.162:0/2799676451 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5bd4076b90 0x7f5bd4111610 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f5bd0006b00 tx=0x7f5bd0009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:00:52.262 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:50.476+0000 7f5bca7fc700 1 -- 172.21.15.162:0/2799676451 <== mon.0 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f5bd00107c0 con 0x7f5bd4076b90 2024-04-27T04:00:52.262 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:50.476+0000 7f5bdbaeb700 1 -- 172.21.15.162:0/2799676451 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5bd4111d50 con 0x7f5bd4076b90 2024-04-27T04:00:52.262 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:50.476+0000 7f5bca7fc700 1 -- 172.21.15.162:0/2799676451 <== mon.0 v2:172.21.15.162:3300/0 2 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7f5bd0010db0 con 0x7f5bd4076b90 2024-04-27T04:00:52.262 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:50.476+0000 7f5bca7fc700 1 -- 172.21.15.162:0/2799676451 <== mon.0 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f5bd001acf0 con 0x7f5bd4076b90 2024-04-27T04:00:52.262 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:50.476+0000 7f5bdbaeb700 1 -- 172.21.15.162:0/2799676451 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5bd41128c0 con 0x7f5bd4076b90 2024-04-27T04:00:52.262 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:50.477+0000 7f5bca7fc700 1 -- 172.21.15.162:0/2799676451 <== mon.0 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 8) v1 ==== 43376+0+0 (secure 0 0 0) 0x7f5bd00193c0 con 0x7f5bd4076b90 2024-04-27T04:00:52.262 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:50.477+0000 7f5bca7fc700 1 --2- 172.21.15.162:0/2799676451 >> [v2:172.21.15.162:6800/8067198,v1:172.21.15.162:6801/8067198] conn(0x7f5bc0036850 0x7f5bc0038d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:00:52.262 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:50.477+0000 7f5bdbaeb700 1 -- 172.21.15.162:0/2799676451 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5bd405f5c0 con 0x7f5bd4076b90 2024-04-27T04:00:52.263 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:50.477+0000 7f5bca7fc700 1 -- 172.21.15.162:0/2799676451 <== mon.0 v2:172.21.15.162:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 936+0+0 (secure 0 0 0) 0x7f5bd004a790 con 0x7f5bd4076b90 2024-04-27T04:00:52.263 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:50.477+0000 7f5bd9086700 1 --2- 172.21.15.162:0/2799676451 >> [v2:172.21.15.162:6800/8067198,v1:172.21.15.162:6801/8067198] conn(0x7f5bc0036850 0x7f5bc0038d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:00:52.263 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:50.477+0000 7f5bd9086700 1 --2- 172.21.15.162:0/2799676451 >> [v2:172.21.15.162:6800/8067198,v1:172.21.15.162:6801/8067198] conn(0x7f5bc0036850 0x7f5bc0038d00 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f5bc4009990 tx=0x7f5bc4006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:00:52.263 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:50.481+0000 7f5bca7fc700 1 -- 172.21.15.162:0/2799676451 <== mon.0 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f5bd0022700 con 0x7f5bd4076b90 2024-04-27T04:00:52.263 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:50.707+0000 7f5bdbaeb700 1 -- 172.21.15.162:0/2799676451 --> [v2:172.21.15.162:6800/8067198,v1:172.21.15.162:6801/8067198] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "target": ["mon-mgr", ""]}) v1 -- 0x7f5bd40fe120 con 0x7f5bc0036850 2024-04-27T04:00:52.263 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:50.715+0000 7f5bca7fc700 1 -- 172.21.15.162:0/2799676451 <== mgr.14120 v2:172.21.15.162:6800/8067198 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+24 (secure 0 0 0) 0x7f5bd40fe120 con 0x7f5bc0036850 2024-04-27T04:00:52.263 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:50.717+0000 7f5bdbaeb700 1 -- 172.21.15.162:0/2799676451 >> [v2:172.21.15.162:6800/8067198,v1:172.21.15.162:6801/8067198] conn(0x7f5bc0036850 msgr2=0x7f5bc0038d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:00:52.263 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:50.717+0000 7f5bdbaeb700 1 --2- 172.21.15.162:0/2799676451 >> [v2:172.21.15.162:6800/8067198,v1:172.21.15.162:6801/8067198] conn(0x7f5bc0036850 0x7f5bc0038d00 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f5bc4009990 tx=0x7f5bc4006e40 comp rx=0 tx=0).stop 2024-04-27T04:00:52.263 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:50.717+0000 7f5bdbaeb700 1 -- 172.21.15.162:0/2799676451 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5bd4076b90 msgr2=0x7f5bd4111610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:00:52.263 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:50.717+0000 7f5bdbaeb700 1 --2- 172.21.15.162:0/2799676451 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5bd4076b90 0x7f5bd4111610 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f5bd0006b00 tx=0x7f5bd0009250 comp rx=0 tx=0).stop 2024-04-27T04:00:52.263 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:50.717+0000 7f5bdbaeb700 1 -- 172.21.15.162:0/2799676451 shutdown_connections 2024-04-27T04:00:52.263 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:50.717+0000 7f5bdbaeb700 1 --2- 172.21.15.162:0/2799676451 >> [v2:172.21.15.162:6800/8067198,v1:172.21.15.162:6801/8067198] conn(0x7f5bc0036850 0x7f5bc0038d00 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:00:52.263 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:50.717+0000 7f5bdbaeb700 1 --2- 172.21.15.162:0/2799676451 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5bd4076b90 0x7f5bd4111610 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:00:52.263 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:50.717+0000 7f5bdbaeb700 1 -- 172.21.15.162:0/2799676451 >> 172.21.15.162:0/2799676451 conn(0x7f5bd40fcef0 msgr2=0x7f5bd40fda10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:00:52.263 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:50.717+0000 7f5bdbaeb700 1 -- 172.21.15.162:0/2799676451 shutdown_connections 2024-04-27T04:00:52.264 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:50.717+0000 7f5bdbaeb700 1 -- 172.21.15.162:0/2799676451 wait complete. 2024-04-27T04:00:52.264 INFO:teuthology.orchestra.run.smithi162.stdout:Deploying mgr service with default placement... 2024-04-27T04:00:53.138 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:00:52 smithi162 ceph-mon[26841]: from='mgr.14120 172.21.15.162:0/1771817378' entity='mgr.smithi162.oennfn' 2024-04-27T04:00:53.139 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:00:52 smithi162 ceph-mon[26841]: from='mgr.14120 172.21.15.162:0/1771817378' entity='mgr.smithi162.oennfn' 2024-04-27T04:00:54.062 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stdout Scheduled mgr update... 2024-04-27T04:00:54.062 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:52.727+0000 7f9752fa7700 1 Processor -- start 2024-04-27T04:00:54.062 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:52.727+0000 7f9752fa7700 1 -- start start 2024-04-27T04:00:54.063 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:52.727+0000 7f9752fa7700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f974c102810 0x7f974c102c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:00:54.063 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:52.727+0000 7f9752fa7700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f974c103160 con 0x7f974c102810 2024-04-27T04:00:54.063 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:52.727+0000 7f9750d43700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f974c102810 0x7f974c102c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:00:54.063 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:52.727+0000 7f9750d43700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f974c102810 0x7f974c102c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:47782/0 (socket says 172.21.15.162:47782) 2024-04-27T04:00:54.063 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:52.727+0000 7f9750d43700 1 -- 172.21.15.162:0/814882558 learned_addr learned my addr 172.21.15.162:0/814882558 (peer_addr_for_me v2:172.21.15.162:0/0) 2024-04-27T04:00:54.063 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:52.727+0000 7f9750d43700 1 -- 172.21.15.162:0/814882558 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f974c1032a0 con 0x7f974c102810 2024-04-27T04:00:54.063 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:52.728+0000 7f9750d43700 1 --2- 172.21.15.162:0/814882558 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f974c102810 0x7f974c102c20 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f97480098e0 tx=0x7f9748009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=58a78d3a5db523f2 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:00:54.063 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:52.728+0000 7f97437fe700 1 -- 172.21.15.162:0/814882558 <== mon.0 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f974800b860 con 0x7f974c102810 2024-04-27T04:00:54.063 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:52.728+0000 7f97437fe700 1 -- 172.21.15.162:0/814882558 <== mon.0 v2:172.21.15.162:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f974800be90 con 0x7f974c102810 2024-04-27T04:00:54.063 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:52.728+0000 7f97437fe700 1 -- 172.21.15.162:0/814882558 <== mon.0 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f97480103c0 con 0x7f974c102810 2024-04-27T04:00:54.063 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:52.728+0000 7f9752fa7700 1 -- 172.21.15.162:0/814882558 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f974c102810 msgr2=0x7f974c102c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:00:54.063 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:52.728+0000 7f9752fa7700 1 --2- 172.21.15.162:0/814882558 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f974c102810 0x7f974c102c20 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f97480098e0 tx=0x7f9748009250 comp rx=0 tx=0).stop 2024-04-27T04:00:54.063 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:52.728+0000 7f9752fa7700 1 -- 172.21.15.162:0/814882558 shutdown_connections 2024-04-27T04:00:54.063 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:52.728+0000 7f9752fa7700 1 --2- 172.21.15.162:0/814882558 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f974c102810 0x7f974c102c20 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:00:54.063 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:52.728+0000 7f9752fa7700 1 -- 172.21.15.162:0/814882558 >> 172.21.15.162:0/814882558 conn(0x7f974c0fdfd0 msgr2=0x7f974c1003e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:00:54.064 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:52.728+0000 7f9752fa7700 1 -- 172.21.15.162:0/814882558 shutdown_connections 2024-04-27T04:00:54.064 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:52.728+0000 7f9752fa7700 1 -- 172.21.15.162:0/814882558 wait complete. 2024-04-27T04:00:54.064 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:52.729+0000 7f9752fa7700 1 Processor -- start 2024-04-27T04:00:54.064 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:52.729+0000 7f9752fa7700 1 -- start start 2024-04-27T04:00:54.064 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:52.729+0000 7f9752fa7700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f974c102810 0x7f974c1116c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:00:54.064 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:52.729+0000 7f9752fa7700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f974c111c00 con 0x7f974c102810 2024-04-27T04:00:54.064 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:52.729+0000 7f9750d43700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f974c102810 0x7f974c1116c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:00:54.064 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:52.729+0000 7f9750d43700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f974c102810 0x7f974c1116c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:47798/0 (socket says 172.21.15.162:47798) 2024-04-27T04:00:54.064 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:52.729+0000 7f9750d43700 1 -- 172.21.15.162:0/535825513 learned_addr learned my addr 172.21.15.162:0/535825513 (peer_addr_for_me v2:172.21.15.162:0/0) 2024-04-27T04:00:54.064 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:52.730+0000 7f9750d43700 1 -- 172.21.15.162:0/535825513 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97480094f0 con 0x7f974c102810 2024-04-27T04:00:54.064 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:52.730+0000 7f9750d43700 1 --2- 172.21.15.162:0/535825513 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f974c102810 0x7f974c1116c0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f9748011c10 tx=0x7f9748009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:00:54.064 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:52.730+0000 7f97417fa700 1 -- 172.21.15.162:0/535825513 <== mon.0 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f9748010890 con 0x7f974c102810 2024-04-27T04:00:54.064 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:52.730+0000 7f97417fa700 1 -- 172.21.15.162:0/535825513 <== mon.0 v2:172.21.15.162:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f9748010ec0 con 0x7f974c102810 2024-04-27T04:00:54.064 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:52.730+0000 7f9752fa7700 1 -- 172.21.15.162:0/535825513 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f974c111e00 con 0x7f974c102810 2024-04-27T04:00:54.064 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:52.730+0000 7f97417fa700 1 -- 172.21.15.162:0/535825513 <== mon.0 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f97480243f0 con 0x7f974c102810 2024-04-27T04:00:54.065 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:52.730+0000 7f9752fa7700 1 -- 172.21.15.162:0/535825513 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f974c112970 con 0x7f974c102810 2024-04-27T04:00:54.065 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:52.731+0000 7f9752fa7700 1 -- 172.21.15.162:0/535825513 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f974c05f5c0 con 0x7f974c102810 2024-04-27T04:00:54.065 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:52.731+0000 7f97417fa700 1 -- 172.21.15.162:0/535825513 <== mon.0 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 8) v1 ==== 43376+0+0 (secure 0 0 0) 0x7f974802d3f0 con 0x7f974c102810 2024-04-27T04:00:54.065 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:52.732+0000 7f97417fa700 1 --2- 172.21.15.162:0/535825513 >> [v2:172.21.15.162:6800/8067198,v1:172.21.15.162:6801/8067198] conn(0x7f97340368f0 0x7f9734038da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:00:54.065 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:52.732+0000 7f97417fa700 1 -- 172.21.15.162:0/535825513 <== mon.0 v2:172.21.15.162:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 936+0+0 (secure 0 0 0) 0x7f974804a2c0 con 0x7f974c102810 2024-04-27T04:00:54.065 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:52.732+0000 7f9743fff700 1 --2- 172.21.15.162:0/535825513 >> [v2:172.21.15.162:6800/8067198,v1:172.21.15.162:6801/8067198] conn(0x7f97340368f0 0x7f9734038da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:00:54.065 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:52.732+0000 7f9743fff700 1 --2- 172.21.15.162:0/535825513 >> [v2:172.21.15.162:6800/8067198,v1:172.21.15.162:6801/8067198] conn(0x7f97340368f0 0x7f9734038da0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f973c009990 tx=0x7f973c006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:00:54.065 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:52.736+0000 7f97417fa700 1 -- 172.21.15.162:0/535825513 <== mon.0 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f974802b8e0 con 0x7f974c102810 2024-04-27T04:00:54.065 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:52.958+0000 7f9752fa7700 1 -- 172.21.15.162:0/535825513 --> [v2:172.21.15.162:6800/8067198,v1:172.21.15.162:6801/8067198] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "target": ["mon-mgr", ""]}) v1 -- 0x7f974c0ff230 con 0x7f97340368f0 2024-04-27T04:00:54.065 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:52.965+0000 7f97417fa700 1 -- 172.21.15.162:0/535825513 <== mgr.14120 v2:172.21.15.162:6800/8067198 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+24 (secure 0 0 0) 0x7f974c0ff230 con 0x7f97340368f0 2024-04-27T04:00:54.065 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:52.967+0000 7f9752fa7700 1 -- 172.21.15.162:0/535825513 >> [v2:172.21.15.162:6800/8067198,v1:172.21.15.162:6801/8067198] conn(0x7f97340368f0 msgr2=0x7f9734038da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:00:54.065 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:52.967+0000 7f9752fa7700 1 --2- 172.21.15.162:0/535825513 >> [v2:172.21.15.162:6800/8067198,v1:172.21.15.162:6801/8067198] conn(0x7f97340368f0 0x7f9734038da0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f973c009990 tx=0x7f973c006e40 comp rx=0 tx=0).stop 2024-04-27T04:00:54.065 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:52.967+0000 7f9752fa7700 1 -- 172.21.15.162:0/535825513 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f974c102810 msgr2=0x7f974c1116c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:00:54.065 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:52.967+0000 7f9752fa7700 1 --2- 172.21.15.162:0/535825513 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f974c102810 0x7f974c1116c0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f9748011c10 tx=0x7f9748009250 comp rx=0 tx=0).stop 2024-04-27T04:00:54.065 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:52.967+0000 7f9752fa7700 1 -- 172.21.15.162:0/535825513 shutdown_connections 2024-04-27T04:00:54.065 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:52.967+0000 7f9752fa7700 1 --2- 172.21.15.162:0/535825513 >> [v2:172.21.15.162:6800/8067198,v1:172.21.15.162:6801/8067198] conn(0x7f97340368f0 0x7f9734038da0 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:00:54.066 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:52.967+0000 7f9752fa7700 1 --2- 172.21.15.162:0/535825513 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f974c102810 0x7f974c1116c0 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:00:54.066 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:52.967+0000 7f9752fa7700 1 -- 172.21.15.162:0/535825513 >> 172.21.15.162:0/535825513 conn(0x7f974c0fdfd0 msgr2=0x7f974c0feb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:00:54.066 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:52.967+0000 7f9752fa7700 1 -- 172.21.15.162:0/535825513 shutdown_connections 2024-04-27T04:00:54.066 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:52.967+0000 7f9752fa7700 1 -- 172.21.15.162:0/535825513 wait complete. 2024-04-27T04:00:54.066 INFO:teuthology.orchestra.run.smithi162.stdout:Deploying crash service with default placement... 2024-04-27T04:00:54.138 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:00:53 smithi162 ceph-mon[26841]: from='client.14144 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "target": ["mon-mgr", ""]}]: dispatch 2024-04-27T04:00:54.139 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:00:53 smithi162 ceph-mon[26841]: Saving service mgr spec with placement count:2 2024-04-27T04:00:54.139 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:00:53 smithi162 ceph-mon[26841]: from='mgr.14120 172.21.15.162:0/1771817378' entity='mgr.smithi162.oennfn' 2024-04-27T04:00:54.139 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:00:53 smithi162 ceph-mon[26841]: from='mgr.14120 172.21.15.162:0/1771817378' entity='mgr.smithi162.oennfn' 2024-04-27T04:00:55.294 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stdout Scheduled crash update... 2024-04-27T04:00:55.295 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:54.453+0000 7f51b71d5700 1 Processor -- start 2024-04-27T04:00:55.295 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:54.453+0000 7f51b71d5700 1 -- start start 2024-04-27T04:00:55.295 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:54.454+0000 7f51b71d5700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f51b0076b70 0x7f51b0105500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:00:55.295 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:54.454+0000 7f51b71d5700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51b0105a40 con 0x7f51b0076b70 2024-04-27T04:00:55.295 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:54.454+0000 7f51b4f71700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f51b0076b70 0x7f51b0105500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:00:55.295 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:54.454+0000 7f51b4f71700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f51b0076b70 0x7f51b0105500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:37902/0 (socket says 172.21.15.162:37902) 2024-04-27T04:00:55.295 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:54.454+0000 7f51b4f71700 1 -- 172.21.15.162:0/1971922088 learned_addr learned my addr 172.21.15.162:0/1971922088 (peer_addr_for_me v2:172.21.15.162:0/0) 2024-04-27T04:00:55.295 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:54.454+0000 7f51b4f71700 1 -- 172.21.15.162:0/1971922088 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51b0105b80 con 0x7f51b0076b70 2024-04-27T04:00:55.295 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:54.454+0000 7f51b4f71700 1 --2- 172.21.15.162:0/1971922088 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f51b0076b70 0x7f51b0105500 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f51a40098e0 tx=0x7f51a4009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1a7d8a1d65f761da server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:00:55.295 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:54.454+0000 7f51af7fe700 1 -- 172.21.15.162:0/1971922088 <== mon.0 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f51a400b860 con 0x7f51b0076b70 2024-04-27T04:00:55.295 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:54.454+0000 7f51af7fe700 1 -- 172.21.15.162:0/1971922088 <== mon.0 v2:172.21.15.162:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f51a400be90 con 0x7f51b0076b70 2024-04-27T04:00:55.296 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:54.454+0000 7f51af7fe700 1 -- 172.21.15.162:0/1971922088 <== mon.0 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f51a40103c0 con 0x7f51b0076b70 2024-04-27T04:00:55.296 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:54.455+0000 7f51b71d5700 1 -- 172.21.15.162:0/1971922088 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f51b0076b70 msgr2=0x7f51b0105500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:00:55.296 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:54.455+0000 7f51b71d5700 1 --2- 172.21.15.162:0/1971922088 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f51b0076b70 0x7f51b0105500 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f51a40098e0 tx=0x7f51a4009250 comp rx=0 tx=0).stop 2024-04-27T04:00:55.296 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:54.455+0000 7f51b71d5700 1 -- 172.21.15.162:0/1971922088 shutdown_connections 2024-04-27T04:00:55.296 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:54.455+0000 7f51b71d5700 1 --2- 172.21.15.162:0/1971922088 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f51b0076b70 0x7f51b0105500 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:00:55.296 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:54.455+0000 7f51b71d5700 1 -- 172.21.15.162:0/1971922088 >> 172.21.15.162:0/1971922088 conn(0x7f51b00fcef0 msgr2=0x7f51b00ff300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:00:55.296 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:54.455+0000 7f51b71d5700 1 -- 172.21.15.162:0/1971922088 shutdown_connections 2024-04-27T04:00:55.296 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:54.455+0000 7f51b71d5700 1 -- 172.21.15.162:0/1971922088 wait complete. 2024-04-27T04:00:55.296 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:54.456+0000 7f51b71d5700 1 Processor -- start 2024-04-27T04:00:55.296 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:54.456+0000 7f51b71d5700 1 -- start start 2024-04-27T04:00:55.296 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:54.456+0000 7f51b71d5700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f51b0076b70 0x7f51b0111650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:00:55.296 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:54.456+0000 7f51b71d5700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51b0111b90 con 0x7f51b0076b70 2024-04-27T04:00:55.296 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:54.456+0000 7f51b4f71700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f51b0076b70 0x7f51b0111650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:00:55.296 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:54.456+0000 7f51b4f71700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f51b0076b70 0x7f51b0111650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:37904/0 (socket says 172.21.15.162:37904) 2024-04-27T04:00:55.297 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:54.456+0000 7f51b4f71700 1 -- 172.21.15.162:0/2867071326 learned_addr learned my addr 172.21.15.162:0/2867071326 (peer_addr_for_me v2:172.21.15.162:0/0) 2024-04-27T04:00:55.297 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:54.456+0000 7f51b4f71700 1 -- 172.21.15.162:0/2867071326 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51a40094f0 con 0x7f51b0076b70 2024-04-27T04:00:55.297 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:54.456+0000 7f51b4f71700 1 --2- 172.21.15.162:0/2867071326 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f51b0076b70 0x7f51b0111650 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f51a4006d40 tx=0x7f51a4009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:00:55.297 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:54.457+0000 7f51ad7fa700 1 -- 172.21.15.162:0/2867071326 <== mon.0 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f51a4010840 con 0x7f51b0076b70 2024-04-27T04:00:55.297 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:54.457+0000 7f51b71d5700 1 -- 172.21.15.162:0/2867071326 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f51b0111d90 con 0x7f51b0076b70 2024-04-27T04:00:55.297 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:54.457+0000 7f51ad7fa700 1 -- 172.21.15.162:0/2867071326 <== mon.0 v2:172.21.15.162:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f51a4010e70 con 0x7f51b0076b70 2024-04-27T04:00:55.297 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:54.457+0000 7f51ad7fa700 1 -- 172.21.15.162:0/2867071326 <== mon.0 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f51a4024ea0 con 0x7f51b0076b70 2024-04-27T04:00:55.297 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:54.457+0000 7f51b71d5700 1 -- 172.21.15.162:0/2867071326 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f51b0112900 con 0x7f51b0076b70 2024-04-27T04:00:55.297 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:54.457+0000 7f51ad7fa700 1 -- 172.21.15.162:0/2867071326 <== mon.0 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 8) v1 ==== 43376+0+0 (secure 0 0 0) 0x7f51a401a510 con 0x7f51b0076b70 2024-04-27T04:00:55.297 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:54.457+0000 7f51b71d5700 1 -- 172.21.15.162:0/2867071326 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f51b005f5c0 con 0x7f51b0076b70 2024-04-27T04:00:55.297 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:54.458+0000 7f51ad7fa700 1 --2- 172.21.15.162:0/2867071326 >> [v2:172.21.15.162:6800/8067198,v1:172.21.15.162:6801/8067198] conn(0x7f51a0036800 0x7f51a0038cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:00:55.297 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:54.458+0000 7f51ad7fa700 1 -- 172.21.15.162:0/2867071326 <== mon.0 v2:172.21.15.162:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 936+0+0 (secure 0 0 0) 0x7f51a404a870 con 0x7f51b0076b70 2024-04-27T04:00:55.297 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:54.458+0000 7f51affff700 1 --2- 172.21.15.162:0/2867071326 >> [v2:172.21.15.162:6800/8067198,v1:172.21.15.162:6801/8067198] conn(0x7f51a0036800 0x7f51a0038cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:00:55.297 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:54.458+0000 7f51affff700 1 --2- 172.21.15.162:0/2867071326 >> [v2:172.21.15.162:6800/8067198,v1:172.21.15.162:6801/8067198] conn(0x7f51a0036800 0x7f51a0038cb0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f519c009990 tx=0x7f519c006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:00:55.297 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:54.462+0000 7f51ad7fa700 1 -- 172.21.15.162:0/2867071326 <== mon.0 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f51a4018940 con 0x7f51b0076b70 2024-04-27T04:00:55.298 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:54.660+0000 7f51b71d5700 1 -- 172.21.15.162:0/2867071326 --> [v2:172.21.15.162:6800/8067198,v1:172.21.15.162:6801/8067198] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "crash", "target": ["mon-mgr", ""]}) v1 -- 0x7f51b00fe0b0 con 0x7f51a0036800 2024-04-27T04:00:55.298 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:54.664+0000 7f51ad7fa700 1 -- 172.21.15.162:0/2867071326 <== mgr.14120 v2:172.21.15.162:6800/8067198 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+26 (secure 0 0 0) 0x7f51b00fe0b0 con 0x7f51a0036800 2024-04-27T04:00:55.298 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:54.666+0000 7f51b71d5700 1 -- 172.21.15.162:0/2867071326 >> [v2:172.21.15.162:6800/8067198,v1:172.21.15.162:6801/8067198] conn(0x7f51a0036800 msgr2=0x7f51a0038cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:00:55.298 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:54.666+0000 7f51b71d5700 1 --2- 172.21.15.162:0/2867071326 >> [v2:172.21.15.162:6800/8067198,v1:172.21.15.162:6801/8067198] conn(0x7f51a0036800 0x7f51a0038cb0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f519c009990 tx=0x7f519c006e40 comp rx=0 tx=0).stop 2024-04-27T04:00:55.298 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:54.666+0000 7f51b71d5700 1 -- 172.21.15.162:0/2867071326 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f51b0076b70 msgr2=0x7f51b0111650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:00:55.298 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:54.666+0000 7f51b71d5700 1 --2- 172.21.15.162:0/2867071326 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f51b0076b70 0x7f51b0111650 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f51a4006d40 tx=0x7f51a4009250 comp rx=0 tx=0).stop 2024-04-27T04:00:55.298 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:54.666+0000 7f51b71d5700 1 -- 172.21.15.162:0/2867071326 shutdown_connections 2024-04-27T04:00:55.298 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:54.666+0000 7f51b71d5700 1 --2- 172.21.15.162:0/2867071326 >> [v2:172.21.15.162:6800/8067198,v1:172.21.15.162:6801/8067198] conn(0x7f51a0036800 0x7f51a0038cb0 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:00:55.298 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:54.666+0000 7f51b71d5700 1 --2- 172.21.15.162:0/2867071326 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f51b0076b70 0x7f51b0111650 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:00:55.298 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:54.666+0000 7f51b71d5700 1 -- 172.21.15.162:0/2867071326 >> 172.21.15.162:0/2867071326 conn(0x7f51b00fcef0 msgr2=0x7f51b00fd9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:00:55.298 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:54.666+0000 7f51b71d5700 1 -- 172.21.15.162:0/2867071326 shutdown_connections 2024-04-27T04:00:55.298 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:54.666+0000 7f51b71d5700 1 -- 172.21.15.162:0/2867071326 wait complete. 2024-04-27T04:00:55.298 INFO:teuthology.orchestra.run.smithi162.stdout:Deploying prometheus service with default placement... 2024-04-27T04:00:55.889 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:00:55 smithi162 ceph-mon[26841]: from='client.14146 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "crash", "target": ["mon-mgr", ""]}]: dispatch 2024-04-27T04:00:55.889 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:00:55 smithi162 ceph-mon[26841]: Saving service crash spec with placement * 2024-04-27T04:00:55.889 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:00:55 smithi162 ceph-mon[26841]: from='mgr.14120 172.21.15.162:0/1771817378' entity='mgr.smithi162.oennfn' 2024-04-27T04:00:55.889 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:00:55 smithi162 ceph-mon[26841]: from='mgr.14120 172.21.15.162:0/1771817378' entity='mgr.smithi162.oennfn' 2024-04-27T04:00:56.614 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stdout Scheduled prometheus update... 2024-04-27T04:00:56.615 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:55.762+0000 7fe7a12d4700 1 Processor -- start 2024-04-27T04:00:56.615 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:55.762+0000 7fe7a12d4700 1 -- start start 2024-04-27T04:00:56.615 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:55.762+0000 7fe7a12d4700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe79c076c50 0x7fe79c105420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:00:56.615 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:55.762+0000 7fe7a12d4700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe79c105960 con 0x7fe79c076c50 2024-04-27T04:00:56.615 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:55.762+0000 7fe79affd700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe79c076c50 0x7fe79c105420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:00:56.615 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:55.762+0000 7fe79affd700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe79c076c50 0x7fe79c105420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:37918/0 (socket says 172.21.15.162:37918) 2024-04-27T04:00:56.615 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:55.762+0000 7fe79affd700 1 -- 172.21.15.162:0/2471178961 learned_addr learned my addr 172.21.15.162:0/2471178961 (peer_addr_for_me v2:172.21.15.162:0/0) 2024-04-27T04:00:56.615 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:55.763+0000 7fe79affd700 1 -- 172.21.15.162:0/2471178961 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe79c105aa0 con 0x7fe79c076c50 2024-04-27T04:00:56.615 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:55.763+0000 7fe79affd700 1 --2- 172.21.15.162:0/2471178961 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe79c076c50 0x7fe79c105420 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fe794009e50 tx=0x7fe794009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6b85f2f7200e56ef server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:00:56.615 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:55.763+0000 7fe799ffb700 1 -- 172.21.15.162:0/2471178961 <== mon.0 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fe7940092e0 con 0x7fe79c076c50 2024-04-27T04:00:56.615 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:55.763+0000 7fe799ffb700 1 -- 172.21.15.162:0/2471178961 <== mon.0 v2:172.21.15.162:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fe794012430 con 0x7fe79c076c50 2024-04-27T04:00:56.615 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:55.763+0000 7fe799ffb700 1 -- 172.21.15.162:0/2471178961 <== mon.0 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fe7940075a0 con 0x7fe79c076c50 2024-04-27T04:00:56.615 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:55.763+0000 7fe7a12d4700 1 -- 172.21.15.162:0/2471178961 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe79c076c50 msgr2=0x7fe79c105420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:00:56.616 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:55.763+0000 7fe7a12d4700 1 --2- 172.21.15.162:0/2471178961 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe79c076c50 0x7fe79c105420 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fe794009e50 tx=0x7fe794009840 comp rx=0 tx=0).stop 2024-04-27T04:00:56.616 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:55.763+0000 7fe7a12d4700 1 -- 172.21.15.162:0/2471178961 shutdown_connections 2024-04-27T04:00:56.616 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:55.763+0000 7fe7a12d4700 1 --2- 172.21.15.162:0/2471178961 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe79c076c50 0x7fe79c105420 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:00:56.616 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:55.763+0000 7fe7a12d4700 1 -- 172.21.15.162:0/2471178961 >> 172.21.15.162:0/2471178961 conn(0x7fe79c0fcef0 msgr2=0x7fe79c0ff300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:00:56.616 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:55.763+0000 7fe7a12d4700 1 -- 172.21.15.162:0/2471178961 shutdown_connections 2024-04-27T04:00:56.616 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:55.763+0000 7fe7a12d4700 1 -- 172.21.15.162:0/2471178961 wait complete. 2024-04-27T04:00:56.616 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:55.764+0000 7fe7a12d4700 1 Processor -- start 2024-04-27T04:00:56.616 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:55.764+0000 7fe7a12d4700 1 -- start start 2024-04-27T04:00:56.616 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:55.764+0000 7fe7a12d4700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe79c076c50 0x7fe79c111680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:00:56.616 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:55.764+0000 7fe7a12d4700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe79c111bc0 con 0x7fe79c076c50 2024-04-27T04:00:56.616 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:55.764+0000 7fe79affd700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe79c076c50 0x7fe79c111680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:00:56.617 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:55.764+0000 7fe79affd700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe79c076c50 0x7fe79c111680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:37920/0 (socket says 172.21.15.162:37920) 2024-04-27T04:00:56.617 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:55.764+0000 7fe79affd700 1 -- 172.21.15.162:0/2791872204 learned_addr learned my addr 172.21.15.162:0/2791872204 (peer_addr_for_me v2:172.21.15.162:0/0) 2024-04-27T04:00:56.617 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:55.765+0000 7fe79affd700 1 -- 172.21.15.162:0/2791872204 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe794009ae0 con 0x7fe79c076c50 2024-04-27T04:00:56.617 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:55.765+0000 7fe79affd700 1 --2- 172.21.15.162:0/2791872204 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe79c076c50 0x7fe79c111680 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fe794012f20 tx=0x7fe794009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:00:56.617 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:55.765+0000 7fe7837fe700 1 -- 172.21.15.162:0/2791872204 <== mon.0 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fe7940092e0 con 0x7fe79c076c50 2024-04-27T04:00:56.617 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:55.765+0000 7fe7837fe700 1 -- 172.21.15.162:0/2791872204 <== mon.0 v2:172.21.15.162:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fe794007d30 con 0x7fe79c076c50 2024-04-27T04:00:56.617 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:55.765+0000 7fe7a12d4700 1 -- 172.21.15.162:0/2791872204 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe79c111dc0 con 0x7fe79c076c50 2024-04-27T04:00:56.617 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:55.765+0000 7fe7837fe700 1 -- 172.21.15.162:0/2791872204 <== mon.0 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fe794025a00 con 0x7fe79c076c50 2024-04-27T04:00:56.617 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:55.765+0000 7fe7a12d4700 1 -- 172.21.15.162:0/2791872204 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe79c112930 con 0x7fe79c076c50 2024-04-27T04:00:56.617 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:55.766+0000 7fe7837fe700 1 -- 172.21.15.162:0/2791872204 <== mon.0 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 8) v1 ==== 43376+0+0 (secure 0 0 0) 0x7fe794019050 con 0x7fe79c076c50 2024-04-27T04:00:56.617 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:55.766+0000 7fe7837fe700 1 --2- 172.21.15.162:0/2791872204 >> [v2:172.21.15.162:6800/8067198,v1:172.21.15.162:6801/8067198] conn(0x7fe7840368a0 0x7fe784038d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:00:56.617 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:55.766+0000 7fe7837fe700 1 -- 172.21.15.162:0/2791872204 <== mon.0 v2:172.21.15.162:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 936+0+0 (secure 0 0 0) 0x7fe79404aff0 con 0x7fe79c076c50 2024-04-27T04:00:56.618 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:55.766+0000 7fe79a7fc700 1 --2- 172.21.15.162:0/2791872204 >> [v2:172.21.15.162:6800/8067198,v1:172.21.15.162:6801/8067198] conn(0x7fe7840368a0 0x7fe784038d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:00:56.618 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:55.766+0000 7fe7a12d4700 1 -- 172.21.15.162:0/2791872204 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe79c05f5c0 con 0x7fe79c076c50 2024-04-27T04:00:56.618 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:55.766+0000 7fe79a7fc700 1 --2- 172.21.15.162:0/2791872204 >> [v2:172.21.15.162:6800/8067198,v1:172.21.15.162:6801/8067198] conn(0x7fe7840368a0 0x7fe784038d50 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fe790009990 tx=0x7fe790006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:00:56.618 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:55.771+0000 7fe7837fe700 1 -- 172.21.15.162:0/2791872204 <== mon.0 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7fe794020020 con 0x7fe79c076c50 2024-04-27T04:00:56.618 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:55.982+0000 7fe7a12d4700 1 -- 172.21.15.162:0/2791872204 --> [v2:172.21.15.162:6800/8067198,v1:172.21.15.162:6801/8067198] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "prometheus", "target": ["mon-mgr", ""]}) v1 -- 0x7fe79c0fe130 con 0x7fe7840368a0 2024-04-27T04:00:56.618 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:55.986+0000 7fe7837fe700 1 -- 172.21.15.162:0/2791872204 <== mgr.14120 v2:172.21.15.162:6800/8067198 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+31 (secure 0 0 0) 0x7fe79c0fe130 con 0x7fe7840368a0 2024-04-27T04:00:56.618 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:55.988+0000 7fe7a12d4700 1 -- 172.21.15.162:0/2791872204 >> [v2:172.21.15.162:6800/8067198,v1:172.21.15.162:6801/8067198] conn(0x7fe7840368a0 msgr2=0x7fe784038d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:00:56.618 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:55.988+0000 7fe7a12d4700 1 --2- 172.21.15.162:0/2791872204 >> [v2:172.21.15.162:6800/8067198,v1:172.21.15.162:6801/8067198] conn(0x7fe7840368a0 0x7fe784038d50 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fe790009990 tx=0x7fe790006e40 comp rx=0 tx=0).stop 2024-04-27T04:00:56.618 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:55.988+0000 7fe7a12d4700 1 -- 172.21.15.162:0/2791872204 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe79c076c50 msgr2=0x7fe79c111680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:00:56.618 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:55.988+0000 7fe7a12d4700 1 --2- 172.21.15.162:0/2791872204 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe79c076c50 0x7fe79c111680 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fe794012f20 tx=0x7fe794009840 comp rx=0 tx=0).stop 2024-04-27T04:00:56.618 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:55.988+0000 7fe7a12d4700 1 -- 172.21.15.162:0/2791872204 shutdown_connections 2024-04-27T04:00:56.618 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:55.988+0000 7fe7a12d4700 1 --2- 172.21.15.162:0/2791872204 >> [v2:172.21.15.162:6800/8067198,v1:172.21.15.162:6801/8067198] conn(0x7fe7840368a0 0x7fe784038d50 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:00:56.618 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:55.988+0000 7fe7a12d4700 1 --2- 172.21.15.162:0/2791872204 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe79c076c50 0x7fe79c111680 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:00:56.619 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:55.988+0000 7fe7a12d4700 1 -- 172.21.15.162:0/2791872204 >> 172.21.15.162:0/2791872204 conn(0x7fe79c0fcef0 msgr2=0x7fe79c0fda20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:00:56.619 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:55.989+0000 7fe7a12d4700 1 -- 172.21.15.162:0/2791872204 shutdown_connections 2024-04-27T04:00:56.619 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:55.989+0000 7fe7a12d4700 1 -- 172.21.15.162:0/2791872204 wait complete. 2024-04-27T04:00:56.619 INFO:teuthology.orchestra.run.smithi162.stdout:Deploying grafana service with default placement... 2024-04-27T04:00:57.209 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:00:56 smithi162 ceph-mon[26841]: from='client.14148 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "prometheus", "target": ["mon-mgr", ""]}]: dispatch 2024-04-27T04:00:57.209 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:00:56 smithi162 ceph-mon[26841]: Saving service prometheus spec with placement count:1 2024-04-27T04:00:57.209 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:00:56 smithi162 ceph-mon[26841]: from='mgr.14120 172.21.15.162:0/1771817378' entity='mgr.smithi162.oennfn' 2024-04-27T04:00:57.942 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stdout Scheduled grafana update... 2024-04-27T04:00:57.949 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:57.030+0000 7efe3aff4700 1 Processor -- start 2024-04-27T04:00:57.949 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:57.030+0000 7efe3aff4700 1 -- start start 2024-04-27T04:00:57.949 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:57.030+0000 7efe3aff4700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7efe34102890 0x7efe34102ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:00:57.949 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:57.030+0000 7efe3aff4700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe341031e0 con 0x7efe34102890 2024-04-27T04:00:57.949 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:57.031+0000 7efe38d90700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7efe34102890 0x7efe34102ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:00:57.949 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:57.031+0000 7efe38d90700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7efe34102890 0x7efe34102ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:37932/0 (socket says 172.21.15.162:37932) 2024-04-27T04:00:57.949 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:57.031+0000 7efe38d90700 1 -- 172.21.15.162:0/412729488 learned_addr learned my addr 172.21.15.162:0/412729488 (peer_addr_for_me v2:172.21.15.162:0/0) 2024-04-27T04:00:57.949 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:57.031+0000 7efe38d90700 1 -- 172.21.15.162:0/412729488 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe34103320 con 0x7efe34102890 2024-04-27T04:00:57.949 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:57.031+0000 7efe38d90700 1 --2- 172.21.15.162:0/412729488 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7efe34102890 0x7efe34102ca0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7efe280098e0 tx=0x7efe28009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dda748efacb1593a server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:00:57.950 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:57.031+0000 7efe337fe700 1 -- 172.21.15.162:0/412729488 <== mon.0 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7efe2800b860 con 0x7efe34102890 2024-04-27T04:00:57.950 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:57.031+0000 7efe337fe700 1 -- 172.21.15.162:0/412729488 <== mon.0 v2:172.21.15.162:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7efe2800be90 con 0x7efe34102890 2024-04-27T04:00:57.950 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:57.031+0000 7efe337fe700 1 -- 172.21.15.162:0/412729488 <== mon.0 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7efe280103c0 con 0x7efe34102890 2024-04-27T04:00:57.950 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:57.032+0000 7efe3aff4700 1 -- 172.21.15.162:0/412729488 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7efe34102890 msgr2=0x7efe34102ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:00:57.950 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:57.032+0000 7efe3aff4700 1 --2- 172.21.15.162:0/412729488 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7efe34102890 0x7efe34102ca0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7efe280098e0 tx=0x7efe28009250 comp rx=0 tx=0).stop 2024-04-27T04:00:57.950 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:57.032+0000 7efe3aff4700 1 -- 172.21.15.162:0/412729488 shutdown_connections 2024-04-27T04:00:57.950 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:57.032+0000 7efe3aff4700 1 --2- 172.21.15.162:0/412729488 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7efe34102890 0x7efe34102ca0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:00:57.950 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:57.032+0000 7efe3aff4700 1 -- 172.21.15.162:0/412729488 >> 172.21.15.162:0/412729488 conn(0x7efe340fe070 msgr2=0x7efe34100480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:00:57.950 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:57.032+0000 7efe3aff4700 1 -- 172.21.15.162:0/412729488 shutdown_connections 2024-04-27T04:00:57.950 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:57.032+0000 7efe3aff4700 1 -- 172.21.15.162:0/412729488 wait complete. 2024-04-27T04:00:57.950 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:57.033+0000 7efe3aff4700 1 Processor -- start 2024-04-27T04:00:57.950 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:57.033+0000 7efe3aff4700 1 -- start start 2024-04-27T04:00:57.951 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:57.033+0000 7efe3aff4700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7efe34102890 0x7efe341115d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:00:57.951 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:57.033+0000 7efe3aff4700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe34111b10 con 0x7efe34102890 2024-04-27T04:00:57.951 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:57.033+0000 7efe38d90700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7efe34102890 0x7efe341115d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:00:57.951 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:57.033+0000 7efe38d90700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7efe34102890 0x7efe341115d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:37946/0 (socket says 172.21.15.162:37946) 2024-04-27T04:00:57.951 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:57.033+0000 7efe38d90700 1 -- 172.21.15.162:0/2584663815 learned_addr learned my addr 172.21.15.162:0/2584663815 (peer_addr_for_me v2:172.21.15.162:0/0) 2024-04-27T04:00:57.951 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:57.033+0000 7efe38d90700 1 -- 172.21.15.162:0/2584663815 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe280094f0 con 0x7efe34102890 2024-04-27T04:00:57.951 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:57.034+0000 7efe38d90700 1 --2- 172.21.15.162:0/2584663815 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7efe34102890 0x7efe341115d0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7efe280090b0 tx=0x7efe28009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:00:57.951 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:57.034+0000 7efe317fa700 1 -- 172.21.15.162:0/2584663815 <== mon.0 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7efe28010840 con 0x7efe34102890 2024-04-27T04:00:57.951 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:57.034+0000 7efe317fa700 1 -- 172.21.15.162:0/2584663815 <== mon.0 v2:172.21.15.162:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7efe28010e70 con 0x7efe34102890 2024-04-27T04:00:57.951 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:57.034+0000 7efe3aff4700 1 -- 172.21.15.162:0/2584663815 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efe34111d10 con 0x7efe34102890 2024-04-27T04:00:57.951 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:57.034+0000 7efe317fa700 1 -- 172.21.15.162:0/2584663815 <== mon.0 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7efe28024e90 con 0x7efe34102890 2024-04-27T04:00:57.951 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:57.034+0000 7efe3aff4700 1 -- 172.21.15.162:0/2584663815 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efe34112880 con 0x7efe34102890 2024-04-27T04:00:57.951 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:57.035+0000 7efe3aff4700 1 -- 172.21.15.162:0/2584663815 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efe3405f5c0 con 0x7efe34102890 2024-04-27T04:00:57.952 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:57.035+0000 7efe317fa700 1 -- 172.21.15.162:0/2584663815 <== mon.0 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 8) v1 ==== 43376+0+0 (secure 0 0 0) 0x7efe28024430 con 0x7efe34102890 2024-04-27T04:00:57.952 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:57.035+0000 7efe317fa700 1 --2- 172.21.15.162:0/2584663815 >> [v2:172.21.15.162:6800/8067198,v1:172.21.15.162:6801/8067198] conn(0x7efe24036800 0x7efe24038cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:00:57.952 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:57.035+0000 7efe317fa700 1 -- 172.21.15.162:0/2584663815 <== mon.0 v2:172.21.15.162:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 936+0+0 (secure 0 0 0) 0x7efe2804a760 con 0x7efe34102890 2024-04-27T04:00:57.952 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:57.035+0000 7efe33fff700 1 --2- 172.21.15.162:0/2584663815 >> [v2:172.21.15.162:6800/8067198,v1:172.21.15.162:6801/8067198] conn(0x7efe24036800 0x7efe24038cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:00:57.952 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:57.036+0000 7efe33fff700 1 --2- 172.21.15.162:0/2584663815 >> [v2:172.21.15.162:6800/8067198,v1:172.21.15.162:6801/8067198] conn(0x7efe24036800 0x7efe24038cb0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7efe20009990 tx=0x7efe20006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:00:57.952 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:57.040+0000 7efe317fa700 1 -- 172.21.15.162:0/2584663815 <== mon.0 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7efe2802c520 con 0x7efe34102890 2024-04-27T04:00:57.952 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:57.248+0000 7efe3aff4700 1 -- 172.21.15.162:0/2584663815 --> [v2:172.21.15.162:6800/8067198,v1:172.21.15.162:6801/8067198] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "grafana", "target": ["mon-mgr", ""]}) v1 -- 0x7efe340ff210 con 0x7efe24036800 2024-04-27T04:00:57.952 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:57.253+0000 7efe317fa700 1 -- 172.21.15.162:0/2584663815 <== mgr.14120 v2:172.21.15.162:6800/8067198 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+28 (secure 0 0 0) 0x7efe340ff210 con 0x7efe24036800 2024-04-27T04:00:57.952 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:57.255+0000 7efe3aff4700 1 -- 172.21.15.162:0/2584663815 >> [v2:172.21.15.162:6800/8067198,v1:172.21.15.162:6801/8067198] conn(0x7efe24036800 msgr2=0x7efe24038cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:00:57.953 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:57.255+0000 7efe3aff4700 1 --2- 172.21.15.162:0/2584663815 >> [v2:172.21.15.162:6800/8067198,v1:172.21.15.162:6801/8067198] conn(0x7efe24036800 0x7efe24038cb0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7efe20009990 tx=0x7efe20006e40 comp rx=0 tx=0).stop 2024-04-27T04:00:57.953 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:57.255+0000 7efe3aff4700 1 -- 172.21.15.162:0/2584663815 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7efe34102890 msgr2=0x7efe341115d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:00:57.953 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:57.255+0000 7efe3aff4700 1 --2- 172.21.15.162:0/2584663815 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7efe34102890 0x7efe341115d0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7efe280090b0 tx=0x7efe28009250 comp rx=0 tx=0).stop 2024-04-27T04:00:57.953 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:57.255+0000 7efe3aff4700 1 -- 172.21.15.162:0/2584663815 shutdown_connections 2024-04-27T04:00:57.953 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:57.255+0000 7efe3aff4700 1 --2- 172.21.15.162:0/2584663815 >> [v2:172.21.15.162:6800/8067198,v1:172.21.15.162:6801/8067198] conn(0x7efe24036800 0x7efe24038cb0 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:00:57.953 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:57.255+0000 7efe3aff4700 1 --2- 172.21.15.162:0/2584663815 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7efe34102890 0x7efe341115d0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:00:57.953 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:57.255+0000 7efe3aff4700 1 -- 172.21.15.162:0/2584663815 >> 172.21.15.162:0/2584663815 conn(0x7efe340fe070 msgr2=0x7efe340feb00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:00:57.953 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:57.255+0000 7efe3aff4700 1 -- 172.21.15.162:0/2584663815 shutdown_connections 2024-04-27T04:00:57.953 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:57.255+0000 7efe3aff4700 1 -- 172.21.15.162:0/2584663815 wait complete. 2024-04-27T04:00:57.953 INFO:teuthology.orchestra.run.smithi162.stdout:Deploying node-exporter service with default placement... 2024-04-27T04:00:58.358 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:00:58 smithi162 ceph-mon[26841]: from='client.14150 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "grafana", "target": ["mon-mgr", ""]}]: dispatch 2024-04-27T04:00:58.358 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:00:58 smithi162 ceph-mon[26841]: Saving service grafana spec with placement count:1 2024-04-27T04:00:58.358 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:00:58 smithi162 ceph-mon[26841]: from='mgr.14120 172.21.15.162:0/1771817378' entity='mgr.smithi162.oennfn' 2024-04-27T04:00:59.515 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stdout Scheduled node-exporter update... 2024-04-27T04:00:59.515 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:58.386+0000 7f301cc86700 1 Processor -- start 2024-04-27T04:00:59.515 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:58.386+0000 7f301cc86700 1 -- start start 2024-04-27T04:00:59.515 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:58.386+0000 7f301cc86700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3018102810 0x7f3018102c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:00:59.515 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:58.386+0000 7f301cc86700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3018103160 con 0x7f3018102810 2024-04-27T04:00:59.515 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:58.386+0000 7f301659c700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3018102810 0x7f3018102c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:00:59.515 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:58.386+0000 7f301659c700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3018102810 0x7f3018102c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:37958/0 (socket says 172.21.15.162:37958) 2024-04-27T04:00:59.515 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:58.386+0000 7f301659c700 1 -- 172.21.15.162:0/2289599822 learned_addr learned my addr 172.21.15.162:0/2289599822 (peer_addr_for_me v2:172.21.15.162:0/0) 2024-04-27T04:00:59.515 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:58.387+0000 7f301659c700 1 -- 172.21.15.162:0/2289599822 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f30181032a0 con 0x7f3018102810 2024-04-27T04:00:59.516 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:58.387+0000 7f301659c700 1 --2- 172.21.15.162:0/2289599822 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3018102810 0x7f3018102c20 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f30080098e0 tx=0x7f3008009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=816aa7f37cc8772e server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:00:59.516 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:58.387+0000 7f3015d9b700 1 -- 172.21.15.162:0/2289599822 <== mon.0 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f300800b860 con 0x7f3018102810 2024-04-27T04:00:59.516 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:58.387+0000 7f3015d9b700 1 -- 172.21.15.162:0/2289599822 <== mon.0 v2:172.21.15.162:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f300800be90 con 0x7f3018102810 2024-04-27T04:00:59.516 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:58.387+0000 7f3015d9b700 1 -- 172.21.15.162:0/2289599822 <== mon.0 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f30080103c0 con 0x7f3018102810 2024-04-27T04:00:59.516 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:58.388+0000 7f301cc86700 1 -- 172.21.15.162:0/2289599822 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3018102810 msgr2=0x7f3018102c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:00:59.516 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:58.388+0000 7f301cc86700 1 --2- 172.21.15.162:0/2289599822 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3018102810 0x7f3018102c20 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f30080098e0 tx=0x7f3008009250 comp rx=0 tx=0).stop 2024-04-27T04:00:59.516 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:58.388+0000 7f301cc86700 1 -- 172.21.15.162:0/2289599822 shutdown_connections 2024-04-27T04:00:59.516 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:58.388+0000 7f301cc86700 1 --2- 172.21.15.162:0/2289599822 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3018102810 0x7f3018102c20 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:00:59.516 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:58.388+0000 7f301cc86700 1 -- 172.21.15.162:0/2289599822 >> 172.21.15.162:0/2289599822 conn(0x7f30180fdfd0 msgr2=0x7f30181003e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:00:59.516 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:58.388+0000 7f301cc86700 1 -- 172.21.15.162:0/2289599822 shutdown_connections 2024-04-27T04:00:59.516 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:58.388+0000 7f301cc86700 1 -- 172.21.15.162:0/2289599822 wait complete. 2024-04-27T04:00:59.516 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:58.388+0000 7f301cc86700 1 Processor -- start 2024-04-27T04:00:59.516 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:58.388+0000 7f301cc86700 1 -- start start 2024-04-27T04:00:59.516 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:58.389+0000 7f301cc86700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3018102810 0x7f3018078c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:00:59.517 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:58.389+0000 7f301cc86700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3018079160 con 0x7f3018102810 2024-04-27T04:00:59.517 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:58.389+0000 7f301659c700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3018102810 0x7f3018078c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:00:59.517 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:58.389+0000 7f301659c700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3018102810 0x7f3018078c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:37972/0 (socket says 172.21.15.162:37972) 2024-04-27T04:00:59.517 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:58.389+0000 7f301659c700 1 -- 172.21.15.162:0/3530350389 learned_addr learned my addr 172.21.15.162:0/3530350389 (peer_addr_for_me v2:172.21.15.162:0/0) 2024-04-27T04:00:59.517 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:58.389+0000 7f301659c700 1 -- 172.21.15.162:0/3530350389 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f30080094f0 con 0x7f3018102810 2024-04-27T04:00:59.517 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:58.389+0000 7f301659c700 1 --2- 172.21.15.162:0/3530350389 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3018102810 0x7f3018078c20 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f3008011c10 tx=0x7f3008009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:00:59.517 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:58.389+0000 7f300f7fe700 1 -- 172.21.15.162:0/3530350389 <== mon.0 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f3008010890 con 0x7f3018102810 2024-04-27T04:00:59.517 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:58.389+0000 7f300f7fe700 1 -- 172.21.15.162:0/3530350389 <== mon.0 v2:172.21.15.162:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f3008010ec0 con 0x7f3018102810 2024-04-27T04:00:59.517 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:58.389+0000 7f300f7fe700 1 -- 172.21.15.162:0/3530350389 <== mon.0 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f30080243f0 con 0x7f3018102810 2024-04-27T04:00:59.517 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:58.389+0000 7f301cc86700 1 -- 172.21.15.162:0/3530350389 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f30180771d0 con 0x7f3018102810 2024-04-27T04:00:59.517 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:58.389+0000 7f301cc86700 1 -- 172.21.15.162:0/3530350389 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f30180776c0 con 0x7f3018102810 2024-04-27T04:00:59.517 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:58.390+0000 7f300f7fe700 1 -- 172.21.15.162:0/3530350389 <== mon.0 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 8) v1 ==== 43376+0+0 (secure 0 0 0) 0x7f300802d3f0 con 0x7f3018102810 2024-04-27T04:00:59.517 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:58.390+0000 7f300f7fe700 1 --2- 172.21.15.162:0/3530350389 >> [v2:172.21.15.162:6800/8067198,v1:172.21.15.162:6801/8067198] conn(0x7f3004036850 0x7f3004038d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:00:59.518 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:58.390+0000 7f301cc86700 1 -- 172.21.15.162:0/3530350389 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f301804bed0 con 0x7f3018102810 2024-04-27T04:00:59.518 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:58.390+0000 7f300f7fe700 1 -- 172.21.15.162:0/3530350389 <== mon.0 v2:172.21.15.162:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 936+0+0 (secure 0 0 0) 0x7f300804a240 con 0x7f3018102810 2024-04-27T04:00:59.518 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:58.391+0000 7f300dd9b700 1 --2- 172.21.15.162:0/3530350389 >> [v2:172.21.15.162:6800/8067198,v1:172.21.15.162:6801/8067198] conn(0x7f3004036850 0x7f3004038d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:00:59.518 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:58.391+0000 7f300dd9b700 1 --2- 172.21.15.162:0/3530350389 >> [v2:172.21.15.162:6800/8067198,v1:172.21.15.162:6801/8067198] conn(0x7f3004036850 0x7f3004038d00 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f3000009990 tx=0x7f3000006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:00:59.518 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:58.395+0000 7f300f7fe700 1 -- 172.21.15.162:0/3530350389 <== mon.0 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f300802b8e0 con 0x7f3018102810 2024-04-27T04:00:59.518 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:58.590+0000 7f301cc86700 1 -- 172.21.15.162:0/3530350389 --> [v2:172.21.15.162:6800/8067198,v1:172.21.15.162:6801/8067198] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "node-exporter", "target": ["mon-mgr", ""]}) v1 -- 0x7f3018077e80 con 0x7f3004036850 2024-04-27T04:00:59.518 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:58.594+0000 7f300f7fe700 1 -- 172.21.15.162:0/3530350389 <== mgr.14120 v2:172.21.15.162:6800/8067198 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+34 (secure 0 0 0) 0x7f3018077e80 con 0x7f3004036850 2024-04-27T04:00:59.518 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:58.597+0000 7f301cc86700 1 -- 172.21.15.162:0/3530350389 >> [v2:172.21.15.162:6800/8067198,v1:172.21.15.162:6801/8067198] conn(0x7f3004036850 msgr2=0x7f3004038d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:00:59.518 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:58.597+0000 7f301cc86700 1 --2- 172.21.15.162:0/3530350389 >> [v2:172.21.15.162:6800/8067198,v1:172.21.15.162:6801/8067198] conn(0x7f3004036850 0x7f3004038d00 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f3000009990 tx=0x7f3000006e40 comp rx=0 tx=0).stop 2024-04-27T04:00:59.518 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:58.597+0000 7f301cc86700 1 -- 172.21.15.162:0/3530350389 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3018102810 msgr2=0x7f3018078c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:00:59.518 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:58.597+0000 7f301cc86700 1 --2- 172.21.15.162:0/3530350389 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3018102810 0x7f3018078c20 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f3008011c10 tx=0x7f3008009250 comp rx=0 tx=0).stop 2024-04-27T04:00:59.519 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:58.597+0000 7f301cc86700 1 -- 172.21.15.162:0/3530350389 shutdown_connections 2024-04-27T04:00:59.519 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:58.597+0000 7f301cc86700 1 --2- 172.21.15.162:0/3530350389 >> [v2:172.21.15.162:6800/8067198,v1:172.21.15.162:6801/8067198] conn(0x7f3004036850 0x7f3004038d00 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:00:59.519 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:58.597+0000 7f301cc86700 1 --2- 172.21.15.162:0/3530350389 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3018102810 0x7f3018078c20 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:00:59.519 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:58.597+0000 7f301cc86700 1 -- 172.21.15.162:0/3530350389 >> 172.21.15.162:0/3530350389 conn(0x7f30180fdfd0 msgr2=0x7f30180feb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:00:59.519 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:58.597+0000 7f301cc86700 1 -- 172.21.15.162:0/3530350389 shutdown_connections 2024-04-27T04:00:59.519 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:58.597+0000 7f301cc86700 1 -- 172.21.15.162:0/3530350389 wait complete. 2024-04-27T04:00:59.519 INFO:teuthology.orchestra.run.smithi162.stdout:Deploying alertmanager service with default placement... 2024-04-27T04:00:59.876 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:00:59 smithi162 ceph-mon[26841]: from='client.14152 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "node-exporter", "target": ["mon-mgr", ""]}]: dispatch 2024-04-27T04:00:59.876 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:00:59 smithi162 ceph-mon[26841]: Saving service node-exporter spec with placement * 2024-04-27T04:00:59.876 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:00:59 smithi162 ceph-mon[26841]: from='mgr.14120 172.21.15.162:0/1771817378' entity='mgr.smithi162.oennfn' 2024-04-27T04:01:00.686 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stdout Scheduled alertmanager update... 2024-04-27T04:01:00.686 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:59.905+0000 7faa458a6700 1 Processor -- start 2024-04-27T04:01:00.686 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:59.905+0000 7faa458a6700 1 -- start start 2024-04-27T04:01:00.687 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:59.905+0000 7faa458a6700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7faa40102c90 0x7faa401030a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:01:00.687 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:59.905+0000 7faa458a6700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa40103670 con 0x7faa40102c90 2024-04-27T04:01:00.687 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:59.905+0000 7faa3effd700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7faa40102c90 0x7faa401030a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:01:00.687 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:59.905+0000 7faa3effd700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7faa40102c90 0x7faa401030a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:37988/0 (socket says 172.21.15.162:37988) 2024-04-27T04:01:00.687 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:59.905+0000 7faa3effd700 1 -- 172.21.15.162:0/3835326977 learned_addr learned my addr 172.21.15.162:0/3835326977 (peer_addr_for_me v2:172.21.15.162:0/0) 2024-04-27T04:01:00.687 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:59.905+0000 7faa3effd700 1 -- 172.21.15.162:0/3835326977 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa401037b0 con 0x7faa40102c90 2024-04-27T04:01:00.687 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:59.905+0000 7faa3effd700 1 --2- 172.21.15.162:0/3835326977 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7faa40102c90 0x7faa401030a0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7faa300098e0 tx=0x7faa30009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7aacef70f485cfbe server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:01:00.687 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:59.906+0000 7faa3dffb700 1 -- 172.21.15.162:0/3835326977 <== mon.0 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7faa3000b860 con 0x7faa40102c90 2024-04-27T04:01:00.687 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:59.906+0000 7faa3dffb700 1 -- 172.21.15.162:0/3835326977 <== mon.0 v2:172.21.15.162:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7faa3000be90 con 0x7faa40102c90 2024-04-27T04:01:00.687 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:59.906+0000 7faa3dffb700 1 -- 172.21.15.162:0/3835326977 <== mon.0 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7faa300103c0 con 0x7faa40102c90 2024-04-27T04:01:00.687 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:59.906+0000 7faa458a6700 1 -- 172.21.15.162:0/3835326977 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7faa40102c90 msgr2=0x7faa401030a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:01:00.687 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:59.906+0000 7faa458a6700 1 --2- 172.21.15.162:0/3835326977 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7faa40102c90 0x7faa401030a0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7faa300098e0 tx=0x7faa30009250 comp rx=0 tx=0).stop 2024-04-27T04:01:00.687 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:59.906+0000 7faa458a6700 1 -- 172.21.15.162:0/3835326977 shutdown_connections 2024-04-27T04:01:00.687 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:59.906+0000 7faa458a6700 1 --2- 172.21.15.162:0/3835326977 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7faa40102c90 0x7faa401030a0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:01:00.687 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:59.906+0000 7faa458a6700 1 -- 172.21.15.162:0/3835326977 >> 172.21.15.162:0/3835326977 conn(0x7faa400fc6f0 msgr2=0x7faa400feb00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:01:00.687 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:59.906+0000 7faa458a6700 1 -- 172.21.15.162:0/3835326977 shutdown_connections 2024-04-27T04:01:00.688 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:59.906+0000 7faa458a6700 1 -- 172.21.15.162:0/3835326977 wait complete. 2024-04-27T04:01:00.688 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:59.907+0000 7faa458a6700 1 Processor -- start 2024-04-27T04:01:00.688 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:59.907+0000 7faa458a6700 1 -- start start 2024-04-27T04:01:00.688 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:59.907+0000 7faa458a6700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7faa40102c90 0x7faa40113810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:01:00.688 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:59.907+0000 7faa458a6700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa40113d50 con 0x7faa40102c90 2024-04-27T04:01:00.688 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:59.907+0000 7faa3effd700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7faa40102c90 0x7faa40113810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:01:00.688 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:59.907+0000 7faa3effd700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7faa40102c90 0x7faa40113810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:38002/0 (socket says 172.21.15.162:38002) 2024-04-27T04:01:00.688 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:59.907+0000 7faa3effd700 1 -- 172.21.15.162:0/3962768681 learned_addr learned my addr 172.21.15.162:0/3962768681 (peer_addr_for_me v2:172.21.15.162:0/0) 2024-04-27T04:01:00.688 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:59.908+0000 7faa3effd700 1 -- 172.21.15.162:0/3962768681 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa300094f0 con 0x7faa40102c90 2024-04-27T04:01:00.688 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:59.908+0000 7faa3effd700 1 --2- 172.21.15.162:0/3962768681 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7faa40102c90 0x7faa40113810 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7faa30011680 tx=0x7faa30009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:01:00.688 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:59.908+0000 7faa277fe700 1 -- 172.21.15.162:0/3962768681 <== mon.0 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7faa30010890 con 0x7faa40102c90 2024-04-27T04:01:00.688 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:59.908+0000 7faa277fe700 1 -- 172.21.15.162:0/3962768681 <== mon.0 v2:172.21.15.162:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7faa30010ec0 con 0x7faa40102c90 2024-04-27T04:01:00.688 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:59.908+0000 7faa458a6700 1 -- 172.21.15.162:0/3962768681 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faa40113f50 con 0x7faa40102c90 2024-04-27T04:01:00.688 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:59.908+0000 7faa277fe700 1 -- 172.21.15.162:0/3962768681 <== mon.0 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7faa300243f0 con 0x7faa40102c90 2024-04-27T04:01:00.688 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:59.908+0000 7faa458a6700 1 -- 172.21.15.162:0/3962768681 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faa40114ac0 con 0x7faa40102c90 2024-04-27T04:01:00.688 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:59.909+0000 7faa277fe700 1 -- 172.21.15.162:0/3962768681 <== mon.0 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 8) v1 ==== 43376+0+0 (secure 0 0 0) 0x7faa3002d3f0 con 0x7faa40102c90 2024-04-27T04:01:00.689 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:59.909+0000 7faa458a6700 1 -- 172.21.15.162:0/3962768681 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faa4005f5c0 con 0x7faa40102c90 2024-04-27T04:01:00.689 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:59.909+0000 7faa277fe700 1 --2- 172.21.15.162:0/3962768681 >> [v2:172.21.15.162:6800/8067198,v1:172.21.15.162:6801/8067198] conn(0x7faa28036850 0x7faa28038d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:01:00.689 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:59.909+0000 7faa277fe700 1 -- 172.21.15.162:0/3962768681 <== mon.0 v2:172.21.15.162:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 936+0+0 (secure 0 0 0) 0x7faa3004a230 con 0x7faa40102c90 2024-04-27T04:01:00.689 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:59.909+0000 7faa3e7fc700 1 --2- 172.21.15.162:0/3962768681 >> [v2:172.21.15.162:6800/8067198,v1:172.21.15.162:6801/8067198] conn(0x7faa28036850 0x7faa28038d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:01:00.689 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:59.909+0000 7faa3e7fc700 1 --2- 172.21.15.162:0/3962768681 >> [v2:172.21.15.162:6800/8067198,v1:172.21.15.162:6801/8067198] conn(0x7faa28036850 0x7faa28038d00 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7faa34009990 tx=0x7faa34006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:01:00.689 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:00:59.914+0000 7faa277fe700 1 -- 172.21.15.162:0/3962768681 <== mon.0 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7faa3002b8e0 con 0x7faa40102c90 2024-04-27T04:01:00.689 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:01:00.111+0000 7faa458a6700 1 -- 172.21.15.162:0/3962768681 --> [v2:172.21.15.162:6800/8067198,v1:172.21.15.162:6801/8067198] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "alertmanager", "target": ["mon-mgr", ""]}) v1 -- 0x7faa400fd8b0 con 0x7faa28036850 2024-04-27T04:01:00.689 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:01:00.116+0000 7faa277fe700 1 -- 172.21.15.162:0/3962768681 <== mgr.14120 v2:172.21.15.162:6800/8067198 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+33 (secure 0 0 0) 0x7faa400fd8b0 con 0x7faa28036850 2024-04-27T04:01:00.689 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:01:00.118+0000 7faa458a6700 1 -- 172.21.15.162:0/3962768681 >> [v2:172.21.15.162:6800/8067198,v1:172.21.15.162:6801/8067198] conn(0x7faa28036850 msgr2=0x7faa28038d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:01:00.689 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:01:00.118+0000 7faa458a6700 1 --2- 172.21.15.162:0/3962768681 >> [v2:172.21.15.162:6800/8067198,v1:172.21.15.162:6801/8067198] conn(0x7faa28036850 0x7faa28038d00 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7faa34009990 tx=0x7faa34006e40 comp rx=0 tx=0).stop 2024-04-27T04:01:00.689 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:01:00.118+0000 7faa458a6700 1 -- 172.21.15.162:0/3962768681 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7faa40102c90 msgr2=0x7faa40113810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:01:00.689 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:01:00.118+0000 7faa458a6700 1 --2- 172.21.15.162:0/3962768681 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7faa40102c90 0x7faa40113810 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7faa30011680 tx=0x7faa30009250 comp rx=0 tx=0).stop 2024-04-27T04:01:00.689 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:01:00.118+0000 7faa458a6700 1 -- 172.21.15.162:0/3962768681 shutdown_connections 2024-04-27T04:01:00.689 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:01:00.118+0000 7faa458a6700 1 --2- 172.21.15.162:0/3962768681 >> [v2:172.21.15.162:6800/8067198,v1:172.21.15.162:6801/8067198] conn(0x7faa28036850 0x7faa28038d00 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:01:00.689 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:01:00.118+0000 7faa458a6700 1 --2- 172.21.15.162:0/3962768681 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7faa40102c90 0x7faa40113810 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:01:00.690 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:01:00.118+0000 7faa458a6700 1 -- 172.21.15.162:0/3962768681 >> 172.21.15.162:0/3962768681 conn(0x7faa400fc6f0 msgr2=0x7faa400fd1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:01:00.690 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:01:00.118+0000 7faa458a6700 1 -- 172.21.15.162:0/3962768681 shutdown_connections 2024-04-27T04:01:00.690 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:01:00.118+0000 7faa458a6700 1 -- 172.21.15.162:0/3962768681 wait complete. 2024-04-27T04:01:01.389 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:01:01 smithi162 ceph-mon[26841]: from='client.14154 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "alertmanager", "target": ["mon-mgr", ""]}]: dispatch 2024-04-27T04:01:01.389 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:01:01 smithi162 ceph-mon[26841]: Saving service alertmanager spec with placement count:1 2024-04-27T04:01:01.389 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:01:01 smithi162 ceph-mon[26841]: from='mgr.14120 172.21.15.162:0/1771817378' entity='mgr.smithi162.oennfn' 2024-04-27T04:01:01.910 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:01:01.098+0000 7efebd4cd700 1 Processor -- start 2024-04-27T04:01:01.910 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:01:01.098+0000 7efebd4cd700 1 -- start start 2024-04-27T04:01:01.910 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:01:01.098+0000 7efebd4cd700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7efeb8102880 0x7efeb8102c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:01:01.911 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:01:01.098+0000 7efebd4cd700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efeb81031d0 con 0x7efeb8102880 2024-04-27T04:01:01.911 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:01:01.099+0000 7efeb6ffd700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7efeb8102880 0x7efeb8102c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:01:01.911 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:01:01.099+0000 7efeb6ffd700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7efeb8102880 0x7efeb8102c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:38014/0 (socket says 172.21.15.162:38014) 2024-04-27T04:01:01.911 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:01:01.099+0000 7efeb6ffd700 1 -- 172.21.15.162:0/938915591 learned_addr learned my addr 172.21.15.162:0/938915591 (peer_addr_for_me v2:172.21.15.162:0/0) 2024-04-27T04:01:01.911 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:01:01.099+0000 7efeb6ffd700 1 -- 172.21.15.162:0/938915591 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efeb8103310 con 0x7efeb8102880 2024-04-27T04:01:01.911 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:01:01.099+0000 7efeb6ffd700 1 --2- 172.21.15.162:0/938915591 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7efeb8102880 0x7efeb8102c90 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7efeb0009e50 tx=0x7efeb0009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ac1590e047613e74 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:01:01.911 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:01:01.099+0000 7efeb5ffb700 1 -- 172.21.15.162:0/938915591 <== mon.0 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7efeb00092e0 con 0x7efeb8102880 2024-04-27T04:01:01.911 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:01:01.099+0000 7efeb5ffb700 1 -- 172.21.15.162:0/938915591 <== mon.0 v2:172.21.15.162:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7efeb0012430 con 0x7efeb8102880 2024-04-27T04:01:01.911 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:01:01.099+0000 7efeb5ffb700 1 -- 172.21.15.162:0/938915591 <== mon.0 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7efeb00075a0 con 0x7efeb8102880 2024-04-27T04:01:01.911 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:01:01.100+0000 7efebd4cd700 1 -- 172.21.15.162:0/938915591 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7efeb8102880 msgr2=0x7efeb8102c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:01:01.911 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:01:01.100+0000 7efebd4cd700 1 --2- 172.21.15.162:0/938915591 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7efeb8102880 0x7efeb8102c90 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7efeb0009e50 tx=0x7efeb0009840 comp rx=0 tx=0).stop 2024-04-27T04:01:01.911 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:01:01.100+0000 7efebd4cd700 1 -- 172.21.15.162:0/938915591 shutdown_connections 2024-04-27T04:01:01.911 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:01:01.100+0000 7efebd4cd700 1 --2- 172.21.15.162:0/938915591 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7efeb8102880 0x7efeb8102c90 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:01:01.911 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:01:01.100+0000 7efebd4cd700 1 -- 172.21.15.162:0/938915591 >> 172.21.15.162:0/938915591 conn(0x7efeb80fe060 msgr2=0x7efeb8100470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:01:01.912 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:01:01.100+0000 7efebd4cd700 1 -- 172.21.15.162:0/938915591 shutdown_connections 2024-04-27T04:01:01.912 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:01:01.100+0000 7efebd4cd700 1 -- 172.21.15.162:0/938915591 wait complete. 2024-04-27T04:01:01.912 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:01:01.100+0000 7efebd4cd700 1 Processor -- start 2024-04-27T04:01:01.912 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:01:01.101+0000 7efebd4cd700 1 -- start start 2024-04-27T04:01:01.912 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:01:01.101+0000 7efebd4cd700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7efeb8102880 0x7efeb8111620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:01:01.912 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:01:01.101+0000 7efebd4cd700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efeb8111b60 con 0x7efeb8102880 2024-04-27T04:01:01.912 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:01:01.101+0000 7efeb6ffd700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7efeb8102880 0x7efeb8111620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:01:01.912 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:01:01.101+0000 7efeb6ffd700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7efeb8102880 0x7efeb8111620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:38018/0 (socket says 172.21.15.162:38018) 2024-04-27T04:01:01.912 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:01:01.101+0000 7efeb6ffd700 1 -- 172.21.15.162:0/156810406 learned_addr learned my addr 172.21.15.162:0/156810406 (peer_addr_for_me v2:172.21.15.162:0/0) 2024-04-27T04:01:01.912 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:01:01.101+0000 7efeb6ffd700 1 -- 172.21.15.162:0/156810406 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efeb0009ae0 con 0x7efeb8102880 2024-04-27T04:01:01.912 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:01:01.101+0000 7efeb6ffd700 1 --2- 172.21.15.162:0/156810406 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7efeb8102880 0x7efeb8111620 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7efeb0009fd0 tx=0x7efeb0009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:01:01.912 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:01:01.101+0000 7efeaf7fe700 1 -- 172.21.15.162:0/156810406 <== mon.0 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7efeb00092e0 con 0x7efeb8102880 2024-04-27T04:01:01.912 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:01:01.101+0000 7efeaf7fe700 1 -- 172.21.15.162:0/156810406 <== mon.0 v2:172.21.15.162:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7efeb0007ce0 con 0x7efeb8102880 2024-04-27T04:01:01.912 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:01:01.101+0000 7efebd4cd700 1 -- 172.21.15.162:0/156810406 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efeb8111d60 con 0x7efeb8102880 2024-04-27T04:01:01.912 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:01:01.101+0000 7efeaf7fe700 1 -- 172.21.15.162:0/156810406 <== mon.0 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7efeb0025990 con 0x7efeb8102880 2024-04-27T04:01:01.913 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:01:01.101+0000 7efebd4cd700 1 -- 172.21.15.162:0/156810406 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efeb81128d0 con 0x7efeb8102880 2024-04-27T04:01:01.913 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:01:01.102+0000 7efeaf7fe700 1 -- 172.21.15.162:0/156810406 <== mon.0 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 8) v1 ==== 43376+0+0 (secure 0 0 0) 0x7efeb0019050 con 0x7efeb8102880 2024-04-27T04:01:01.913 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:01:01.102+0000 7efebd4cd700 1 -- 172.21.15.162:0/156810406 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efeb805f5c0 con 0x7efeb8102880 2024-04-27T04:01:01.913 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:01:01.102+0000 7efeaf7fe700 1 --2- 172.21.15.162:0/156810406 >> [v2:172.21.15.162:6800/8067198,v1:172.21.15.162:6801/8067198] conn(0x7efea40368a0 0x7efea4038d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:01:01.913 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:01:01.102+0000 7efeaf7fe700 1 -- 172.21.15.162:0/156810406 <== mon.0 v2:172.21.15.162:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 936+0+0 (secure 0 0 0) 0x7efeb004a2c0 con 0x7efeb8102880 2024-04-27T04:01:01.913 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:01:01.103+0000 7efeb67fc700 1 --2- 172.21.15.162:0/156810406 >> [v2:172.21.15.162:6800/8067198,v1:172.21.15.162:6801/8067198] conn(0x7efea40368a0 0x7efea4038d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:01:01.913 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:01:01.103+0000 7efeb67fc700 1 --2- 172.21.15.162:0/156810406 >> [v2:172.21.15.162:6800/8067198,v1:172.21.15.162:6801/8067198] conn(0x7efea40368a0 0x7efea4038d50 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7efea0009990 tx=0x7efea0006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:01:01.913 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:01:01.107+0000 7efeaf7fe700 1 -- 172.21.15.162:0/156810406 <== mon.0 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7efeb002d350 con 0x7efeb8102880 2024-04-27T04:01:01.913 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:01:01.292+0000 7efebd4cd700 1 -- 172.21.15.162:0/156810406 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/container_init}] v 0) v1 -- 0x7efeb804bed0 con 0x7efeb8102880 2024-04-27T04:01:01.913 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:01:01.298+0000 7efeaf7fe700 1 -- 172.21.15.162:0/156810406 <== mon.0 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/container_init}]=0 v7)=0 v7) v1 ==== 142+0+0 (secure 0 0 0) 0x7efeb0017020 con 0x7efeb8102880 2024-04-27T04:01:01.913 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:01:01.300+0000 7efebd4cd700 1 -- 172.21.15.162:0/156810406 >> [v2:172.21.15.162:6800/8067198,v1:172.21.15.162:6801/8067198] conn(0x7efea40368a0 msgr2=0x7efea4038d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:01:01.913 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:01:01.300+0000 7efebd4cd700 1 --2- 172.21.15.162:0/156810406 >> [v2:172.21.15.162:6800/8067198,v1:172.21.15.162:6801/8067198] conn(0x7efea40368a0 0x7efea4038d50 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7efea0009990 tx=0x7efea0006e40 comp rx=0 tx=0).stop 2024-04-27T04:01:01.913 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:01:01.300+0000 7efebd4cd700 1 -- 172.21.15.162:0/156810406 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7efeb8102880 msgr2=0x7efeb8111620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:01:01.914 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:01:01.300+0000 7efebd4cd700 1 --2- 172.21.15.162:0/156810406 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7efeb8102880 0x7efeb8111620 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7efeb0009fd0 tx=0x7efeb0009840 comp rx=0 tx=0).stop 2024-04-27T04:01:01.914 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:01:01.300+0000 7efebd4cd700 1 -- 172.21.15.162:0/156810406 shutdown_connections 2024-04-27T04:01:01.914 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:01:01.300+0000 7efebd4cd700 1 --2- 172.21.15.162:0/156810406 >> [v2:172.21.15.162:6800/8067198,v1:172.21.15.162:6801/8067198] conn(0x7efea40368a0 0x7efea4038d50 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:01:01.914 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:01:01.300+0000 7efebd4cd700 1 --2- 172.21.15.162:0/156810406 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7efeb8102880 0x7efeb8111620 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:01:01.914 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:01:01.300+0000 7efebd4cd700 1 -- 172.21.15.162:0/156810406 >> 172.21.15.162:0/156810406 conn(0x7efeb80fe060 msgr2=0x7efeb80febb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:01:01.914 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:01:01.301+0000 7efebd4cd700 1 -- 172.21.15.162:0/156810406 shutdown_connections 2024-04-27T04:01:01.914 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:01:01.301+0000 7efebd4cd700 1 -- 172.21.15.162:0/156810406 wait complete. 2024-04-27T04:01:02.389 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:01:02 smithi162 ceph-mon[26841]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-27T04:01:02.389 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:01:02 smithi162 ceph-mon[26841]: from='client.? 172.21.15.162:0/156810406' entity='client.admin' 2024-04-27T04:01:03.017 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:01:02.266+0000 7fcb6be5d700 1 Processor -- start 2024-04-27T04:01:03.017 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:01:02.266+0000 7fcb6be5d700 1 -- start start 2024-04-27T04:01:03.017 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:01:02.267+0000 7fcb6be5d700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fcb641028c0 0x7fcb64102cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:01:03.017 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:01:02.267+0000 7fcb6be5d700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb64103210 con 0x7fcb641028c0 2024-04-27T04:01:03.017 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:01:02.267+0000 7fcb69bf9700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fcb641028c0 0x7fcb64102cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:01:03.017 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:01:02.267+0000 7fcb69bf9700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fcb641028c0 0x7fcb64102cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:38034/0 (socket says 172.21.15.162:38034) 2024-04-27T04:01:03.017 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:01:02.267+0000 7fcb69bf9700 1 -- 172.21.15.162:0/2162633321 learned_addr learned my addr 172.21.15.162:0/2162633321 (peer_addr_for_me v2:172.21.15.162:0/0) 2024-04-27T04:01:03.017 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:01:02.267+0000 7fcb69bf9700 1 -- 172.21.15.162:0/2162633321 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb64103350 con 0x7fcb641028c0 2024-04-27T04:01:03.017 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:01:02.267+0000 7fcb69bf9700 1 --2- 172.21.15.162:0/2162633321 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fcb641028c0 0x7fcb64102cd0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fcb60009e50 tx=0x7fcb60009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ca76afd0c249e3ef server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:01:03.017 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:01:02.267+0000 7fcb68bf7700 1 -- 172.21.15.162:0/2162633321 <== mon.0 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fcb600092e0 con 0x7fcb641028c0 2024-04-27T04:01:03.018 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:01:02.267+0000 7fcb68bf7700 1 -- 172.21.15.162:0/2162633321 <== mon.0 v2:172.21.15.162:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fcb60012430 con 0x7fcb641028c0 2024-04-27T04:01:03.018 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:01:02.267+0000 7fcb68bf7700 1 -- 172.21.15.162:0/2162633321 <== mon.0 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fcb600075a0 con 0x7fcb641028c0 2024-04-27T04:01:03.018 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:01:02.268+0000 7fcb6be5d700 1 -- 172.21.15.162:0/2162633321 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fcb641028c0 msgr2=0x7fcb64102cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:01:03.018 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:01:02.268+0000 7fcb6be5d700 1 --2- 172.21.15.162:0/2162633321 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fcb641028c0 0x7fcb64102cd0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fcb60009e50 tx=0x7fcb60009840 comp rx=0 tx=0).stop 2024-04-27T04:01:03.018 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:01:02.268+0000 7fcb6be5d700 1 -- 172.21.15.162:0/2162633321 shutdown_connections 2024-04-27T04:01:03.018 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:01:02.268+0000 7fcb6be5d700 1 --2- 172.21.15.162:0/2162633321 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fcb641028c0 0x7fcb64102cd0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:01:03.018 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:01:02.268+0000 7fcb6be5d700 1 -- 172.21.15.162:0/2162633321 >> 172.21.15.162:0/2162633321 conn(0x7fcb640fe060 msgr2=0x7fcb64100470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:01:03.018 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:01:02.268+0000 7fcb6be5d700 1 -- 172.21.15.162:0/2162633321 shutdown_connections 2024-04-27T04:01:03.018 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:01:02.268+0000 7fcb6be5d700 1 -- 172.21.15.162:0/2162633321 wait complete. 2024-04-27T04:01:03.018 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:01:02.269+0000 7fcb6be5d700 1 Processor -- start 2024-04-27T04:01:03.018 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:01:02.269+0000 7fcb6be5d700 1 -- start start 2024-04-27T04:01:03.018 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:01:02.269+0000 7fcb6be5d700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fcb641028c0 0x7fcb64077b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:01:03.018 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:01:02.269+0000 7fcb6be5d700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb64078050 con 0x7fcb641028c0 2024-04-27T04:01:03.018 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:01:02.269+0000 7fcb69bf9700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fcb641028c0 0x7fcb64077b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:01:03.019 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:01:02.269+0000 7fcb69bf9700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fcb641028c0 0x7fcb64077b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:38038/0 (socket says 172.21.15.162:38038) 2024-04-27T04:01:03.019 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:01:02.269+0000 7fcb69bf9700 1 -- 172.21.15.162:0/1367952659 learned_addr learned my addr 172.21.15.162:0/1367952659 (peer_addr_for_me v2:172.21.15.162:0/0) 2024-04-27T04:01:03.019 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:01:02.269+0000 7fcb69bf9700 1 -- 172.21.15.162:0/1367952659 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb60009ae0 con 0x7fcb641028c0 2024-04-27T04:01:03.019 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:01:02.269+0000 7fcb69bf9700 1 --2- 172.21.15.162:0/1367952659 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fcb641028c0 0x7fcb64077b10 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fcb60012de0 tx=0x7fcb60009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:01:03.019 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:01:02.269+0000 7fcb5a7fc700 1 -- 172.21.15.162:0/1367952659 <== mon.0 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fcb600092e0 con 0x7fcb641028c0 2024-04-27T04:01:03.019 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:01:02.269+0000 7fcb5a7fc700 1 -- 172.21.15.162:0/1367952659 <== mon.0 v2:172.21.15.162:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fcb60007d30 con 0x7fcb641028c0 2024-04-27T04:01:03.019 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:01:02.269+0000 7fcb6be5d700 1 -- 172.21.15.162:0/1367952659 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcb640760c0 con 0x7fcb641028c0 2024-04-27T04:01:03.019 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:01:02.270+0000 7fcb5a7fc700 1 -- 172.21.15.162:0/1367952659 <== mon.0 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fcb60025880 con 0x7fcb641028c0 2024-04-27T04:01:03.019 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:01:02.270+0000 7fcb6be5d700 1 -- 172.21.15.162:0/1367952659 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcb640765b0 con 0x7fcb641028c0 2024-04-27T04:01:03.019 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:01:02.270+0000 7fcb5a7fc700 1 -- 172.21.15.162:0/1367952659 <== mon.0 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 8) v1 ==== 43376+0+0 (secure 0 0 0) 0x7fcb60019050 con 0x7fcb641028c0 2024-04-27T04:01:03.019 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:01:02.270+0000 7fcb5a7fc700 1 --2- 172.21.15.162:0/1367952659 >> [v2:172.21.15.162:6800/8067198,v1:172.21.15.162:6801/8067198] conn(0x7fcb500368a0 0x7fcb50038d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:01:03.019 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:01:02.270+0000 7fcb5a7fc700 1 -- 172.21.15.162:0/1367952659 <== mon.0 v2:172.21.15.162:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 936+0+0 (secure 0 0 0) 0x7fcb6004a0d0 con 0x7fcb641028c0 2024-04-27T04:01:03.019 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:01:02.271+0000 7fcb693f8700 1 --2- 172.21.15.162:0/1367952659 >> [v2:172.21.15.162:6800/8067198,v1:172.21.15.162:6801/8067198] conn(0x7fcb500368a0 0x7fcb50038d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:01:03.019 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:01:02.271+0000 7fcb6be5d700 1 -- 172.21.15.162:0/1367952659 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcb6404bed0 con 0x7fcb641028c0 2024-04-27T04:01:03.020 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:01:02.271+0000 7fcb693f8700 1 --2- 172.21.15.162:0/1367952659 >> [v2:172.21.15.162:6800/8067198,v1:172.21.15.162:6801/8067198] conn(0x7fcb500368a0 0x7fcb50038d50 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fcb54009990 tx=0x7fcb54006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:01:03.020 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:01:02.275+0000 7fcb5a7fc700 1 -- 172.21.15.162:0/1367952659 <== mon.0 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7fcb60020020 con 0x7fcb641028c0 2024-04-27T04:01:03.020 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:01:02.462+0000 7fcb6be5d700 1 -- 172.21.15.162:0/1367952659 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command([{prefix=config set, name=mgr/dashboard/ssl_server_port}] v 0) v1 -- 0x7fcb64104610 con 0x7fcb641028c0 2024-04-27T04:01:03.020 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:01:02.468+0000 7fcb5a7fc700 1 -- 172.21.15.162:0/1367952659 <== mon.0 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/dashboard/ssl_server_port}]=0 v8)=0 v8) v1 ==== 130+0+0 (secure 0 0 0) 0x7fcb60017070 con 0x7fcb641028c0 2024-04-27T04:01:03.020 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:01:02.470+0000 7fcb6be5d700 1 -- 172.21.15.162:0/1367952659 >> [v2:172.21.15.162:6800/8067198,v1:172.21.15.162:6801/8067198] conn(0x7fcb500368a0 msgr2=0x7fcb50038d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:01:03.020 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:01:02.470+0000 7fcb6be5d700 1 --2- 172.21.15.162:0/1367952659 >> [v2:172.21.15.162:6800/8067198,v1:172.21.15.162:6801/8067198] conn(0x7fcb500368a0 0x7fcb50038d50 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fcb54009990 tx=0x7fcb54006e40 comp rx=0 tx=0).stop 2024-04-27T04:01:03.020 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:01:02.471+0000 7fcb6be5d700 1 -- 172.21.15.162:0/1367952659 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fcb641028c0 msgr2=0x7fcb64077b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:01:03.020 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:01:02.471+0000 7fcb6be5d700 1 --2- 172.21.15.162:0/1367952659 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fcb641028c0 0x7fcb64077b10 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fcb60012de0 tx=0x7fcb60009840 comp rx=0 tx=0).stop 2024-04-27T04:01:03.020 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:01:02.471+0000 7fcb6be5d700 1 -- 172.21.15.162:0/1367952659 shutdown_connections 2024-04-27T04:01:03.020 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:01:02.471+0000 7fcb6be5d700 1 --2- 172.21.15.162:0/1367952659 >> [v2:172.21.15.162:6800/8067198,v1:172.21.15.162:6801/8067198] conn(0x7fcb500368a0 0x7fcb50038d50 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:01:03.020 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:01:02.471+0000 7fcb6be5d700 1 --2- 172.21.15.162:0/1367952659 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fcb641028c0 0x7fcb64077b10 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:01:03.020 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:01:02.471+0000 7fcb6be5d700 1 -- 172.21.15.162:0/1367952659 >> 172.21.15.162:0/1367952659 conn(0x7fcb640fe060 msgr2=0x7fcb640fec00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:01:03.020 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:01:02.471+0000 7fcb6be5d700 1 -- 172.21.15.162:0/1367952659 shutdown_connections 2024-04-27T04:01:03.020 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:01:02.471+0000 7fcb6be5d700 1 -- 172.21.15.162:0/1367952659 wait complete. 2024-04-27T04:01:03.020 INFO:teuthology.orchestra.run.smithi162.stdout:Enabling the dashboard module... 2024-04-27T04:01:03.674 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:01:03 smithi162 ceph-mon[26841]: from='client.? 172.21.15.162:0/1367952659' entity='client.admin' 2024-04-27T04:01:03.674 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:01:03 smithi162 ceph-mon[26841]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-27T04:01:04.812 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:01:04 smithi162 ceph-mon[26841]: from='client.? 172.21.15.162:0/1513512022' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "dashboard"}]: dispatch 2024-04-27T04:01:05.140 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:01:03.424+0000 7f0492685700 1 Processor -- start 2024-04-27T04:01:05.141 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:01:03.425+0000 7f0492685700 1 -- start start 2024-04-27T04:01:05.141 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:01:03.425+0000 7f0492685700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f048c102880 0x7f048c102c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:01:05.141 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:01:03.425+0000 7f0492685700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f048c1031d0 con 0x7f048c102880 2024-04-27T04:01:05.141 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:01:03.425+0000 7f048bfff700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f048c102880 0x7f048c102c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:01:05.141 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:01:03.425+0000 7f048bfff700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f048c102880 0x7f048c102c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:38048/0 (socket says 172.21.15.162:38048) 2024-04-27T04:01:05.141 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:01:03.425+0000 7f048bfff700 1 -- 172.21.15.162:0/1871207871 learned_addr learned my addr 172.21.15.162:0/1871207871 (peer_addr_for_me v2:172.21.15.162:0/0) 2024-04-27T04:01:05.141 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:01:03.425+0000 7f048bfff700 1 -- 172.21.15.162:0/1871207871 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f048c103310 con 0x7f048c102880 2024-04-27T04:01:05.141 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:01:03.425+0000 7f048bfff700 1 --2- 172.21.15.162:0/1871207871 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f048c102880 0x7f048c102c90 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f047c0098e0 tx=0x7f047c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=31155a729d75723e server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:01:05.141 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:01:03.426+0000 7f048affd700 1 -- 172.21.15.162:0/1871207871 <== mon.0 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f047c00b860 con 0x7f048c102880 2024-04-27T04:01:05.141 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:01:03.426+0000 7f048affd700 1 -- 172.21.15.162:0/1871207871 <== mon.0 v2:172.21.15.162:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f047c00be90 con 0x7f048c102880 2024-04-27T04:01:05.141 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:01:03.426+0000 7f048affd700 1 -- 172.21.15.162:0/1871207871 <== mon.0 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f047c0103c0 con 0x7f048c102880 2024-04-27T04:01:05.141 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:01:03.426+0000 7f0492685700 1 -- 172.21.15.162:0/1871207871 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f048c102880 msgr2=0x7f048c102c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:01:05.142 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:01:03.426+0000 7f0492685700 1 --2- 172.21.15.162:0/1871207871 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f048c102880 0x7f048c102c90 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f047c0098e0 tx=0x7f047c009250 comp rx=0 tx=0).stop 2024-04-27T04:01:05.142 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:01:03.426+0000 7f0492685700 1 -- 172.21.15.162:0/1871207871 shutdown_connections 2024-04-27T04:01:05.142 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:01:03.426+0000 7f0492685700 1 --2- 172.21.15.162:0/1871207871 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f048c102880 0x7f048c102c90 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:01:05.142 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:01:03.426+0000 7f0492685700 1 -- 172.21.15.162:0/1871207871 >> 172.21.15.162:0/1871207871 conn(0x7f048c0fe060 msgr2=0x7f048c100470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:01:05.142 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:01:03.426+0000 7f0492685700 1 -- 172.21.15.162:0/1871207871 shutdown_connections 2024-04-27T04:01:05.142 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:01:03.426+0000 7f0492685700 1 -- 172.21.15.162:0/1871207871 wait complete. 2024-04-27T04:01:05.142 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:01:03.427+0000 7f0492685700 1 Processor -- start 2024-04-27T04:01:05.142 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:01:03.427+0000 7f0492685700 1 -- start start 2024-04-27T04:01:05.142 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:01:03.427+0000 7f0492685700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f048c102880 0x7f048c078c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:01:05.142 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:01:03.427+0000 7f0492685700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f048c079160 con 0x7f048c102880 2024-04-27T04:01:05.142 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:01:03.427+0000 7f048bfff700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f048c102880 0x7f048c078c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:01:05.142 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:01:03.427+0000 7f048bfff700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f048c102880 0x7f048c078c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:38056/0 (socket says 172.21.15.162:38056) 2024-04-27T04:01:05.142 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:01:03.427+0000 7f048bfff700 1 -- 172.21.15.162:0/1513512022 learned_addr learned my addr 172.21.15.162:0/1513512022 (peer_addr_for_me v2:172.21.15.162:0/0) 2024-04-27T04:01:05.143 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:01:03.427+0000 7f048bfff700 1 -- 172.21.15.162:0/1513512022 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f047c0094f0 con 0x7f048c102880 2024-04-27T04:01:05.143 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:01:03.428+0000 7f048bfff700 1 --2- 172.21.15.162:0/1513512022 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f048c102880 0x7f048c078c20 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f047c0090b0 tx=0x7f047c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:01:05.143 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:01:03.428+0000 7f0488ff9700 1 -- 172.21.15.162:0/1513512022 <== mon.0 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f047c010840 con 0x7f048c102880 2024-04-27T04:01:05.143 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:01:03.428+0000 7f0488ff9700 1 -- 172.21.15.162:0/1513512022 <== mon.0 v2:172.21.15.162:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f047c010e70 con 0x7f048c102880 2024-04-27T04:01:05.143 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:01:03.428+0000 7f0492685700 1 -- 172.21.15.162:0/1513512022 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f048c0771d0 con 0x7f048c102880 2024-04-27T04:01:05.143 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:01:03.428+0000 7f0488ff9700 1 -- 172.21.15.162:0/1513512022 <== mon.0 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f047c024e40 con 0x7f048c102880 2024-04-27T04:01:05.143 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:01:03.428+0000 7f0492685700 1 -- 172.21.15.162:0/1513512022 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f048c0776c0 con 0x7f048c102880 2024-04-27T04:01:05.143 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:01:03.429+0000 7f0488ff9700 1 -- 172.21.15.162:0/1513512022 <== mon.0 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 8) v1 ==== 43376+0+0 (secure 0 0 0) 0x7f047c01a4c0 con 0x7f048c102880 2024-04-27T04:01:05.143 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:01:03.429+0000 7f0492685700 1 -- 172.21.15.162:0/1513512022 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f048c04bed0 con 0x7f048c102880 2024-04-27T04:01:05.143 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:01:03.429+0000 7f0488ff9700 1 --2- 172.21.15.162:0/1513512022 >> [v2:172.21.15.162:6800/8067198,v1:172.21.15.162:6801/8067198] conn(0x7f0478036850 0x7f0478038d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:01:05.143 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:01:03.429+0000 7f0488ff9700 1 -- 172.21.15.162:0/1513512022 <== mon.0 v2:172.21.15.162:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 936+0+0 (secure 0 0 0) 0x7f047c04a7b0 con 0x7f048c102880 2024-04-27T04:01:05.143 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:01:03.429+0000 7f048b7fe700 1 --2- 172.21.15.162:0/1513512022 >> [v2:172.21.15.162:6800/8067198,v1:172.21.15.162:6801/8067198] conn(0x7f0478036850 0x7f0478038d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:01:05.143 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:01:03.430+0000 7f048b7fe700 1 --2- 172.21.15.162:0/1513512022 >> [v2:172.21.15.162:6800/8067198,v1:172.21.15.162:6801/8067198] conn(0x7f0478036850 0x7f0478038d00 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f0474009990 tx=0x7f0474006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:01:05.144 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:01:03.434+0000 7f0488ff9700 1 -- 172.21.15.162:0/1513512022 <== mon.0 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f047c018940 con 0x7f048c102880 2024-04-27T04:01:05.144 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:01:03.674+0000 7f0492685700 1 -- 172.21.15.162:0/1513512022 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "dashboard"} v 0) v1 -- 0x7f048c104620 con 0x7f048c102880 2024-04-27T04:01:05.144 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:01:04.477+0000 7f0488ff9700 1 -- 172.21.15.162:0/1513512022 <== mon.0 v2:172.21.15.162:3300/0 7 ==== mgrmap(e 9) v1 ==== 43389+0+0 (secure 0 0 0) 0x7f047c024430 con 0x7f048c102880 2024-04-27T04:01:05.144 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:01:04.479+0000 7f0488ff9700 1 -- 172.21.15.162:0/1513512022 <== mon.0 v2:172.21.15.162:3300/0 8 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "dashboard"}]=0 v9) v1 ==== 88+0+0 (secure 0 0 0) 0x7f047c02d580 con 0x7f048c102880 2024-04-27T04:01:05.144 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:01:04.481+0000 7f0492685700 1 -- 172.21.15.162:0/1513512022 >> [v2:172.21.15.162:6800/8067198,v1:172.21.15.162:6801/8067198] conn(0x7f0478036850 msgr2=0x7f0478038d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:01:05.144 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:01:04.481+0000 7f0492685700 1 --2- 172.21.15.162:0/1513512022 >> [v2:172.21.15.162:6800/8067198,v1:172.21.15.162:6801/8067198] conn(0x7f0478036850 0x7f0478038d00 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f0474009990 tx=0x7f0474006e40 comp rx=0 tx=0).stop 2024-04-27T04:01:05.144 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:01:04.481+0000 7f0492685700 1 -- 172.21.15.162:0/1513512022 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f048c102880 msgr2=0x7f048c078c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:01:05.144 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:01:04.481+0000 7f0492685700 1 --2- 172.21.15.162:0/1513512022 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f048c102880 0x7f048c078c20 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f047c0090b0 tx=0x7f047c009250 comp rx=0 tx=0).stop 2024-04-27T04:01:05.144 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:01:04.481+0000 7f0492685700 1 -- 172.21.15.162:0/1513512022 shutdown_connections 2024-04-27T04:01:05.144 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:01:04.481+0000 7f0492685700 1 --2- 172.21.15.162:0/1513512022 >> [v2:172.21.15.162:6800/8067198,v1:172.21.15.162:6801/8067198] conn(0x7f0478036850 0x7f0478038d00 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:01:05.144 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:01:04.481+0000 7f0492685700 1 --2- 172.21.15.162:0/1513512022 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f048c102880 0x7f048c078c20 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:01:05.144 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:01:04.481+0000 7f0492685700 1 -- 172.21.15.162:0/1513512022 >> 172.21.15.162:0/1513512022 conn(0x7f048c0fe060 msgr2=0x7f048c0feb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:01:05.144 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:01:04.482+0000 7f0492685700 1 -- 172.21.15.162:0/1513512022 shutdown_connections 2024-04-27T04:01:05.144 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:01:04.482+0000 7f0492685700 1 -- 172.21.15.162:0/1513512022 wait complete. 2024-04-27T04:01:05.732 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:01:05 smithi162 ceph-mon[26841]: from='client.? 172.21.15.162:0/1513512022' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "dashboard"}]': finished 2024-04-27T04:01:05.732 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:01:05 smithi162 ceph-mon[26841]: mgrmap e9: smithi162.oennfn(active, since 23s) 2024-04-27T04:01:06.390 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stdout { 2024-04-27T04:01:06.390 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stdout "epoch": 9, 2024-04-27T04:01:06.390 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stdout "available": true, 2024-04-27T04:01:06.390 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stdout "active_name": "smithi162.oennfn", 2024-04-27T04:01:06.390 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-04-27T04:01:06.390 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stdout } 2024-04-27T04:01:06.391 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:01:05.526+0000 7febfa433700 1 Processor -- start 2024-04-27T04:01:06.391 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:01:05.526+0000 7febfa433700 1 -- start start 2024-04-27T04:01:06.391 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:01:05.526+0000 7febfa433700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7febf41028d0 0x7febf4102ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:01:06.391 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:01:05.526+0000 7febfa433700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7febf4103220 con 0x7febf41028d0 2024-04-27T04:01:06.391 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:01:05.526+0000 7febf3fff700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7febf41028d0 0x7febf4102ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:01:06.391 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:01:05.526+0000 7febf3fff700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7febf41028d0 0x7febf4102ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:57270/0 (socket says 172.21.15.162:57270) 2024-04-27T04:01:06.391 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:01:05.526+0000 7febf3fff700 1 -- 172.21.15.162:0/3302015176 learned_addr learned my addr 172.21.15.162:0/3302015176 (peer_addr_for_me v2:172.21.15.162:0/0) 2024-04-27T04:01:06.391 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:01:05.527+0000 7febf3fff700 1 -- 172.21.15.162:0/3302015176 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7febf4103360 con 0x7febf41028d0 2024-04-27T04:01:06.391 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:01:05.527+0000 7febf3fff700 1 --2- 172.21.15.162:0/3302015176 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7febf41028d0 0x7febf4102ce0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7febe40098e0 tx=0x7febe4009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d0a04f5056f08542 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:01:06.391 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:01:05.527+0000 7febf2ffd700 1 -- 172.21.15.162:0/3302015176 <== mon.0 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7febe400b860 con 0x7febf41028d0 2024-04-27T04:01:06.391 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:01:05.527+0000 7febf2ffd700 1 -- 172.21.15.162:0/3302015176 <== mon.0 v2:172.21.15.162:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7febe400be90 con 0x7febf41028d0 2024-04-27T04:01:06.391 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:01:05.527+0000 7febf2ffd700 1 -- 172.21.15.162:0/3302015176 <== mon.0 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7febe40103c0 con 0x7febf41028d0 2024-04-27T04:01:06.391 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:01:05.527+0000 7febfa433700 1 -- 172.21.15.162:0/3302015176 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7febf41028d0 msgr2=0x7febf4102ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:01:06.391 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:01:05.528+0000 7febfa433700 1 --2- 172.21.15.162:0/3302015176 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7febf41028d0 0x7febf4102ce0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7febe40098e0 tx=0x7febe4009250 comp rx=0 tx=0).stop 2024-04-27T04:01:06.392 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:01:05.528+0000 7febfa433700 1 -- 172.21.15.162:0/3302015176 shutdown_connections 2024-04-27T04:01:06.392 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:01:05.528+0000 7febfa433700 1 --2- 172.21.15.162:0/3302015176 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7febf41028d0 0x7febf4102ce0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:01:06.392 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:01:05.528+0000 7febfa433700 1 -- 172.21.15.162:0/3302015176 >> 172.21.15.162:0/3302015176 conn(0x7febf40fe090 msgr2=0x7febf41004a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:01:06.392 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:01:05.528+0000 7febfa433700 1 -- 172.21.15.162:0/3302015176 shutdown_connections 2024-04-27T04:01:06.392 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:01:05.528+0000 7febfa433700 1 -- 172.21.15.162:0/3302015176 wait complete. 2024-04-27T04:01:06.392 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:01:05.528+0000 7febfa433700 1 Processor -- start 2024-04-27T04:01:06.392 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:01:05.528+0000 7febfa433700 1 -- start start 2024-04-27T04:01:06.392 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:01:05.529+0000 7febfa433700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7febf41028d0 0x7febf4078c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:01:06.392 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:01:05.529+0000 7febfa433700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7febf4079160 con 0x7febf41028d0 2024-04-27T04:01:06.392 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:01:05.529+0000 7febf3fff700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7febf41028d0 0x7febf4078c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:01:06.392 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:01:05.529+0000 7febf3fff700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7febf41028d0 0x7febf4078c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:57286/0 (socket says 172.21.15.162:57286) 2024-04-27T04:01:06.392 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:01:05.529+0000 7febf3fff700 1 -- 172.21.15.162:0/951676078 learned_addr learned my addr 172.21.15.162:0/951676078 (peer_addr_for_me v2:172.21.15.162:0/0) 2024-04-27T04:01:06.392 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:01:05.529+0000 7febf3fff700 1 -- 172.21.15.162:0/951676078 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7febe40094f0 con 0x7febf41028d0 2024-04-27T04:01:06.392 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:01:05.529+0000 7febf3fff700 1 --2- 172.21.15.162:0/951676078 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7febf41028d0 0x7febf4078c20 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7febe4011680 tx=0x7febe4009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:01:06.393 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:01:05.529+0000 7febf0ff9700 1 -- 172.21.15.162:0/951676078 <== mon.0 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7febe4010890 con 0x7febf41028d0 2024-04-27T04:01:06.393 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:01:05.529+0000 7febf0ff9700 1 -- 172.21.15.162:0/951676078 <== mon.0 v2:172.21.15.162:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7febe4010ec0 con 0x7febf41028d0 2024-04-27T04:01:06.393 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:01:05.529+0000 7febfa433700 1 -- 172.21.15.162:0/951676078 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7febf40771d0 con 0x7febf41028d0 2024-04-27T04:01:06.393 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:01:05.529+0000 7febf0ff9700 1 -- 172.21.15.162:0/951676078 <== mon.0 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7febe40243f0 con 0x7febf41028d0 2024-04-27T04:01:06.393 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:01:05.530+0000 7febfa433700 1 -- 172.21.15.162:0/951676078 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7febf40776c0 con 0x7febf41028d0 2024-04-27T04:01:06.393 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:01:05.530+0000 7febf0ff9700 1 -- 172.21.15.162:0/951676078 <== mon.0 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 9) v1 ==== 43389+0+0 (secure 0 0 0) 0x7febe402d3f0 con 0x7febf41028d0 2024-04-27T04:01:06.393 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:01:05.530+0000 7febf0ff9700 1 --2- 172.21.15.162:0/951676078 >> [v2:172.21.15.162:6800/8067198,v1:172.21.15.162:6801/8067198] conn(0x7febdc0368a0 0x7febdc038d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:01:06.393 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:01:05.531+0000 7febf0ff9700 1 -- 172.21.15.162:0/951676078 <== mon.0 v2:172.21.15.162:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 936+0+0 (secure 0 0 0) 0x7febe404a2d0 con 0x7febf41028d0 2024-04-27T04:01:06.393 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:01:05.531+0000 7febf37fe700 1 -- 172.21.15.162:0/951676078 >> [v2:172.21.15.162:6800/8067198,v1:172.21.15.162:6801/8067198] conn(0x7febdc0368a0 msgr2=0x7febdc038d50 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.162:6800/8067198 2024-04-27T04:01:06.393 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:01:05.531+0000 7febf37fe700 1 --2- 172.21.15.162:0/951676078 >> [v2:172.21.15.162:6800/8067198,v1:172.21.15.162:6801/8067198] conn(0x7febdc0368a0 0x7febdc038d50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-27T04:01:06.393 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:01:05.531+0000 7febfa433700 1 -- 172.21.15.162:0/951676078 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7febf404bed0 con 0x7febf41028d0 2024-04-27T04:01:06.393 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:01:05.535+0000 7febf0ff9700 1 -- 172.21.15.162:0/951676078 <== mon.0 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7febe402b8e0 con 0x7febf41028d0 2024-04-27T04:01:06.393 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:01:05.732+0000 7febf37fe700 1 -- 172.21.15.162:0/951676078 >> [v2:172.21.15.162:6800/8067198,v1:172.21.15.162:6801/8067198] conn(0x7febdc0368a0 msgr2=0x7febdc038d50 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.162:6800/8067198 2024-04-27T04:01:06.394 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:01:05.732+0000 7febf37fe700 1 --2- 172.21.15.162:0/951676078 >> [v2:172.21.15.162:6800/8067198,v1:172.21.15.162:6801/8067198] conn(0x7febdc0368a0 0x7febdc038d50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-27T04:01:06.394 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:01:05.848+0000 7febfa433700 1 -- 172.21.15.162:0/951676078 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) v1 -- 0x7febf4077b40 con 0x7febf41028d0 2024-04-27T04:01:06.394 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:01:05.848+0000 7febf0ff9700 1 -- 172.21.15.162:0/951676078 <== mon.0 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v9) v1 ==== 56+0+103 (secure 0 0 0) 0x7febe402c350 con 0x7febf41028d0 2024-04-27T04:01:06.394 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:01:05.850+0000 7febfa433700 1 -- 172.21.15.162:0/951676078 >> [v2:172.21.15.162:6800/8067198,v1:172.21.15.162:6801/8067198] conn(0x7febdc0368a0 msgr2=0x7febdc038d50 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-27T04:01:06.394 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:01:05.850+0000 7febfa433700 1 --2- 172.21.15.162:0/951676078 >> [v2:172.21.15.162:6800/8067198,v1:172.21.15.162:6801/8067198] conn(0x7febdc0368a0 0x7febdc038d50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:01:06.394 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:01:05.850+0000 7febfa433700 1 -- 172.21.15.162:0/951676078 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7febf41028d0 msgr2=0x7febf4078c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:01:06.394 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:01:05.850+0000 7febfa433700 1 --2- 172.21.15.162:0/951676078 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7febf41028d0 0x7febf4078c20 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7febe4011680 tx=0x7febe4009250 comp rx=0 tx=0).stop 2024-04-27T04:01:06.394 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:01:05.850+0000 7febfa433700 1 -- 172.21.15.162:0/951676078 shutdown_connections 2024-04-27T04:01:06.394 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:01:05.850+0000 7febfa433700 1 --2- 172.21.15.162:0/951676078 >> [v2:172.21.15.162:6800/8067198,v1:172.21.15.162:6801/8067198] conn(0x7febdc0368a0 0x7febdc038d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:01:06.394 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:01:05.850+0000 7febfa433700 1 --2- 172.21.15.162:0/951676078 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7febf41028d0 0x7febf4078c20 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:01:06.394 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:01:05.850+0000 7febfa433700 1 -- 172.21.15.162:0/951676078 >> 172.21.15.162:0/951676078 conn(0x7febf40fe090 msgr2=0x7febf40feb00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:01:06.394 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:01:05.851+0000 7febfa433700 1 -- 172.21.15.162:0/951676078 shutdown_connections 2024-04-27T04:01:06.394 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:01:05.851+0000 7febfa433700 1 -- 172.21.15.162:0/951676078 wait complete. 2024-04-27T04:01:06.394 INFO:teuthology.orchestra.run.smithi162.stdout:Waiting for the mgr to restart... 2024-04-27T04:01:06.395 INFO:teuthology.orchestra.run.smithi162.stdout:Waiting for mgr epoch 9... 2024-04-27T04:01:06.808 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:01:06 smithi162 ceph-mon[26841]: from='client.? 172.21.15.162:0/951676078' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-04-27T04:01:11.083 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:01:11 smithi162 ceph-mon[26841]: Active manager daemon smithi162.oennfn restarted 2024-04-27T04:01:11.084 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:01:11 smithi162 ceph-mon[26841]: Activating manager daemon smithi162.oennfn 2024-04-27T04:01:11.084 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:01:11 smithi162 ceph-mon[26841]: osdmap e3: 0 total, 0 up, 0 in 2024-04-27T04:01:11.084 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:01:11 smithi162 ceph-mon[26841]: mgrmap e10: smithi162.oennfn(active, starting, since 0.00328642s) 2024-04-27T04:01:11.084 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:01:11 smithi162 ceph-mon[26841]: from='mgr.14162 172.21.15.162:0/1568051568' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "mon metadata", "id": "smithi162"}]: dispatch 2024-04-27T04:01:11.084 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:01:11 smithi162 ceph-mon[26841]: from='mgr.14162 172.21.15.162:0/1568051568' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "mgr metadata", "who": "smithi162.oennfn", "id": "smithi162.oennfn"}]: dispatch 2024-04-27T04:01:11.084 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:01:11 smithi162 ceph-mon[26841]: from='mgr.14162 172.21.15.162:0/1568051568' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-04-27T04:01:11.084 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:01:11 smithi162 ceph-mon[26841]: from='mgr.14162 172.21.15.162:0/1568051568' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-04-27T04:01:11.084 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:01:11 smithi162 ceph-mon[26841]: from='mgr.14162 172.21.15.162:0/1568051568' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-04-27T04:01:11.084 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:01:11 smithi162 ceph-mon[26841]: Manager daemon smithi162.oennfn is now available 2024-04-27T04:01:11.084 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:01:11 smithi162 ceph-mon[26841]: from='mgr.14162 172.21.15.162:0/1568051568' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T04:01:11.084 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:01:11 smithi162 ceph-mon[26841]: from='mgr.14162 172.21.15.162:0/1568051568' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T04:01:11.084 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:01:11 smithi162 ceph-mon[26841]: from='mgr.14162 172.21.15.162:0/1568051568' entity='mgr.smithi162.oennfn' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi162.oennfn/mirror_snapshot_schedule"}]: dispatch 2024-04-27T04:01:11.084 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:01:11 smithi162 ceph-mon[26841]: from='mgr.14162 172.21.15.162:0/1568051568' entity='mgr.smithi162.oennfn' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi162.oennfn/trash_purge_schedule"}]: dispatch 2024-04-27T04:01:12.576 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stdout { 2024-04-27T04:01:12.576 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 11, 2024-04-27T04:01:12.577 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stdout "initialized": true 2024-04-27T04:01:12.577 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stdout } 2024-04-27T04:01:12.577 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:01:06.811+0000 7f83269f1700 1 Processor -- start 2024-04-27T04:01:12.577 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:01:06.811+0000 7f83269f1700 1 -- start start 2024-04-27T04:01:12.577 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:01:06.811+0000 7f83269f1700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f83200712d0 0x7f8320105470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:01:12.577 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:01:06.811+0000 7f83269f1700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83201059b0 con 0x7f83200712d0 2024-04-27T04:01:12.577 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:01:06.811+0000 7f831ffff700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f83200712d0 0x7f8320105470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:01:12.577 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:01:06.811+0000 7f831ffff700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f83200712d0 0x7f8320105470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:57290/0 (socket says 172.21.15.162:57290) 2024-04-27T04:01:12.577 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:01:06.811+0000 7f831ffff700 1 -- 172.21.15.162:0/1949854059 learned_addr learned my addr 172.21.15.162:0/1949854059 (peer_addr_for_me v2:172.21.15.162:0/0) 2024-04-27T04:01:12.577 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:01:06.812+0000 7f831ffff700 1 -- 172.21.15.162:0/1949854059 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8320105af0 con 0x7f83200712d0 2024-04-27T04:01:12.577 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:01:06.812+0000 7f831ffff700 1 --2- 172.21.15.162:0/1949854059 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f83200712d0 0x7f8320105470 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f83140098e0 tx=0x7f8314009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=89e974565405fbdd server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:01:12.577 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:01:06.812+0000 7f831effd700 1 -- 172.21.15.162:0/1949854059 <== mon.0 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f831400b860 con 0x7f83200712d0 2024-04-27T04:01:12.577 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:01:06.812+0000 7f831effd700 1 -- 172.21.15.162:0/1949854059 <== mon.0 v2:172.21.15.162:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f831400be90 con 0x7f83200712d0 2024-04-27T04:01:12.578 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:01:06.812+0000 7f831effd700 1 -- 172.21.15.162:0/1949854059 <== mon.0 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f83140103c0 con 0x7f83200712d0 2024-04-27T04:01:12.578 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:01:06.813+0000 7f83269f1700 1 -- 172.21.15.162:0/1949854059 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f83200712d0 msgr2=0x7f8320105470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:01:12.578 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:01:06.813+0000 7f83269f1700 1 --2- 172.21.15.162:0/1949854059 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f83200712d0 0x7f8320105470 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f83140098e0 tx=0x7f8314009250 comp rx=0 tx=0).stop 2024-04-27T04:01:12.578 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:01:06.813+0000 7f83269f1700 1 -- 172.21.15.162:0/1949854059 shutdown_connections 2024-04-27T04:01:12.578 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:01:06.813+0000 7f83269f1700 1 --2- 172.21.15.162:0/1949854059 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f83200712d0 0x7f8320105470 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:01:12.578 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:01:06.813+0000 7f83269f1700 1 -- 172.21.15.162:0/1949854059 >> 172.21.15.162:0/1949854059 conn(0x7f83200fcea0 msgr2=0x7f83200ff2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:01:12.578 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:01:06.813+0000 7f83269f1700 1 -- 172.21.15.162:0/1949854059 shutdown_connections 2024-04-27T04:01:12.578 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:01:06.813+0000 7f83269f1700 1 -- 172.21.15.162:0/1949854059 wait complete. 2024-04-27T04:01:12.578 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:01:06.813+0000 7f83269f1700 1 Processor -- start 2024-04-27T04:01:12.578 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:01:06.814+0000 7f83269f1700 1 -- start start 2024-04-27T04:01:12.578 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:01:06.814+0000 7f83269f1700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f83200712d0 0x7f8320102dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:01:12.578 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:01:06.814+0000 7f83269f1700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8320103300 con 0x7f83200712d0 2024-04-27T04:01:12.578 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:01:06.814+0000 7f831ffff700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f83200712d0 0x7f8320102dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:01:12.579 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:01:06.814+0000 7f831ffff700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f83200712d0 0x7f8320102dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:57292/0 (socket says 172.21.15.162:57292) 2024-04-27T04:01:12.579 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:01:06.814+0000 7f831ffff700 1 -- 172.21.15.162:0/4140209127 learned_addr learned my addr 172.21.15.162:0/4140209127 (peer_addr_for_me v2:172.21.15.162:0/0) 2024-04-27T04:01:12.579 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:01:06.814+0000 7f831ffff700 1 -- 172.21.15.162:0/4140209127 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f83140094f0 con 0x7f83200712d0 2024-04-27T04:01:12.579 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:01:06.814+0000 7f831ffff700 1 --2- 172.21.15.162:0/4140209127 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f83200712d0 0x7f8320102dc0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f83140105b0 tx=0x7f8314009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:01:12.579 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:01:06.815+0000 7f831cff9700 1 -- 172.21.15.162:0/4140209127 <== mon.0 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f8314010840 con 0x7f83200712d0 2024-04-27T04:01:12.579 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:01:06.815+0000 7f831cff9700 1 -- 172.21.15.162:0/4140209127 <== mon.0 v2:172.21.15.162:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f8314010e70 con 0x7f83200712d0 2024-04-27T04:01:12.579 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:01:06.815+0000 7f83269f1700 1 -- 172.21.15.162:0/4140209127 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8320101370 con 0x7f83200712d0 2024-04-27T04:01:12.579 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:01:06.815+0000 7f831cff9700 1 -- 172.21.15.162:0/4140209127 <== mon.0 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f83140243f0 con 0x7f83200712d0 2024-04-27T04:01:12.579 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:01:06.815+0000 7f83269f1700 1 -- 172.21.15.162:0/4140209127 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8320101860 con 0x7f83200712d0 2024-04-27T04:01:12.579 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:01:06.815+0000 7f831cff9700 1 -- 172.21.15.162:0/4140209127 <== mon.0 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 9) v1 ==== 43389+0+0 (secure 0 0 0) 0x7f8314024550 con 0x7f83200712d0 2024-04-27T04:01:12.579 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:01:06.816+0000 7f831cff9700 1 --2- 172.21.15.162:0/4140209127 >> [v2:172.21.15.162:6800/8067198,v1:172.21.15.162:6801/8067198] conn(0x7f83080368a0 0x7f8308038d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:01:12.579 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:01:06.816+0000 7f831cff9700 1 -- 172.21.15.162:0/4140209127 --> [v2:172.21.15.162:6800/8067198,v1:172.21.15.162:6801/8067198] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7f8308039460 con 0x7f83080368a0 2024-04-27T04:01:12.579 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:01:06.816+0000 7f831cff9700 1 -- 172.21.15.162:0/4140209127 <== mon.0 v2:172.21.15.162:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 936+0+0 (secure 0 0 0) 0x7f831404b6f0 con 0x7f83200712d0 2024-04-27T04:01:12.579 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:01:06.816+0000 7f831f7fe700 1 -- 172.21.15.162:0/4140209127 >> [v2:172.21.15.162:6800/8067198,v1:172.21.15.162:6801/8067198] conn(0x7f83080368a0 msgr2=0x7f8308038d50 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.162:6800/8067198 2024-04-27T04:01:12.580 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:01:06.816+0000 7f831f7fe700 1 --2- 172.21.15.162:0/4140209127 >> [v2:172.21.15.162:6800/8067198,v1:172.21.15.162:6801/8067198] conn(0x7f83080368a0 0x7f8308038d50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-27T04:01:12.580 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:01:07.017+0000 7f831f7fe700 1 -- 172.21.15.162:0/4140209127 >> [v2:172.21.15.162:6800/8067198,v1:172.21.15.162:6801/8067198] conn(0x7f83080368a0 msgr2=0x7f8308038d50 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.162:6800/8067198 2024-04-27T04:01:12.580 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:01:07.017+0000 7f831f7fe700 1 --2- 172.21.15.162:0/4140209127 >> [v2:172.21.15.162:6800/8067198,v1:172.21.15.162:6801/8067198] conn(0x7f83080368a0 0x7f8308038d50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-27T04:01:12.580 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:01:07.418+0000 7f831f7fe700 1 -- 172.21.15.162:0/4140209127 >> [v2:172.21.15.162:6800/8067198,v1:172.21.15.162:6801/8067198] conn(0x7f83080368a0 msgr2=0x7f8308038d50 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.162:6800/8067198 2024-04-27T04:01:12.580 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:01:07.418+0000 7f831f7fe700 1 --2- 172.21.15.162:0/4140209127 >> [v2:172.21.15.162:6800/8067198,v1:172.21.15.162:6801/8067198] conn(0x7f83080368a0 0x7f8308038d50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-27T04:01:12.580 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:01:08.219+0000 7f831f7fe700 1 -- 172.21.15.162:0/4140209127 >> [v2:172.21.15.162:6800/8067198,v1:172.21.15.162:6801/8067198] conn(0x7f83080368a0 msgr2=0x7f8308038d50 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.162:6800/8067198 2024-04-27T04:01:12.580 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:01:08.219+0000 7f831f7fe700 1 --2- 172.21.15.162:0/4140209127 >> [v2:172.21.15.162:6800/8067198,v1:172.21.15.162:6801/8067198] conn(0x7f83080368a0 0x7f8308038d50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-27T04:01:12.580 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:01:09.820+0000 7f831f7fe700 1 -- 172.21.15.162:0/4140209127 >> [v2:172.21.15.162:6800/8067198,v1:172.21.15.162:6801/8067198] conn(0x7f83080368a0 msgr2=0x7f8308038d50 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.162:6800/8067198 2024-04-27T04:01:12.580 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:01:09.820+0000 7f831f7fe700 1 --2- 172.21.15.162:0/4140209127 >> [v2:172.21.15.162:6800/8067198,v1:172.21.15.162:6801/8067198] conn(0x7f83080368a0 0x7f8308038d50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2024-04-27T04:01:12.580 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:01:10.998+0000 7f831cff9700 1 -- 172.21.15.162:0/4140209127 <== mon.0 v2:172.21.15.162:3300/0 6 ==== mgrmap(e 10) v1 ==== 43156+0+0 (secure 0 0 0) 0x7f831401a7c0 con 0x7f83200712d0 2024-04-27T04:01:12.580 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:01:10.998+0000 7f831cff9700 1 -- 172.21.15.162:0/4140209127 >> [v2:172.21.15.162:6800/8067198,v1:172.21.15.162:6801/8067198] conn(0x7f83080368a0 msgr2=0x7f8308038d50 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-27T04:01:12.580 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:01:10.998+0000 7f831cff9700 1 --2- 172.21.15.162:0/4140209127 >> [v2:172.21.15.162:6800/8067198,v1:172.21.15.162:6801/8067198] conn(0x7f83080368a0 0x7f8308038d50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:01:12.580 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:01:12.000+0000 7f831cff9700 1 -- 172.21.15.162:0/4140209127 <== mon.0 v2:172.21.15.162:3300/0 7 ==== mgrmap(e 11) v1 ==== 43283+0+0 (secure 0 0 0) 0x7f831400e8c0 con 0x7f83200712d0 2024-04-27T04:01:12.580 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:01:12.000+0000 7f831cff9700 1 --2- 172.21.15.162:0/4140209127 >> [v2:172.21.15.162:6800/1482851081,v1:172.21.15.162:6801/1482851081] conn(0x7f830803a090 0x7f830803c470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:01:12.581 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:01:12.000+0000 7f831cff9700 1 -- 172.21.15.162:0/4140209127 --> [v2:172.21.15.162:6800/1482851081,v1:172.21.15.162:6801/1482851081] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7f8308039460 con 0x7f830803a090 2024-04-27T04:01:12.581 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:01:12.000+0000 7f831f7fe700 1 --2- 172.21.15.162:0/4140209127 >> [v2:172.21.15.162:6800/1482851081,v1:172.21.15.162:6801/1482851081] conn(0x7f830803a090 0x7f830803c470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:01:12.581 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:01:12.001+0000 7f831f7fe700 1 --2- 172.21.15.162:0/4140209127 >> [v2:172.21.15.162:6800/1482851081,v1:172.21.15.162:6801/1482851081] conn(0x7f830803a090 0x7f830803c470 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f83100039d0 tx=0x7f83100092f0 comp rx=0 tx=0).ready entity=mgr.14162 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:01:12.581 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:01:12.001+0000 7f831cff9700 1 -- 172.21.15.162:0/4140209127 <== mgr.14162 v2:172.21.15.162:6800/1482851081 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+6549 (secure 0 0 0) 0x7f8308039460 con 0x7f830803a090 2024-04-27T04:01:12.581 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:01:12.007+0000 7f83269f1700 1 -- 172.21.15.162:0/4140209127 --> [v2:172.21.15.162:6800/1482851081,v1:172.21.15.162:6801/1482851081] -- command(tid 1: {"prefix": "mgr_status"}) v1 -- 0x7f832004bed0 con 0x7f830803a090 2024-04-27T04:01:12.581 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:01:12.007+0000 7f831cff9700 1 -- 172.21.15.162:0/4140209127 <== mgr.14162 v2:172.21.15.162:6800/1482851081 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+52 (secure 0 0 0) 0x7f832004bed0 con 0x7f830803a090 2024-04-27T04:01:12.581 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:01:12.008+0000 7f83269f1700 1 -- 172.21.15.162:0/4140209127 >> [v2:172.21.15.162:6800/1482851081,v1:172.21.15.162:6801/1482851081] conn(0x7f830803a090 msgr2=0x7f830803c470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:01:12.581 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:01:12.008+0000 7f83269f1700 1 --2- 172.21.15.162:0/4140209127 >> [v2:172.21.15.162:6800/1482851081,v1:172.21.15.162:6801/1482851081] conn(0x7f830803a090 0x7f830803c470 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f83100039d0 tx=0x7f83100092f0 comp rx=0 tx=0).stop 2024-04-27T04:01:12.581 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:01:12.008+0000 7f83269f1700 1 -- 172.21.15.162:0/4140209127 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f83200712d0 msgr2=0x7f8320102dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:01:12.581 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:01:12.008+0000 7f83269f1700 1 --2- 172.21.15.162:0/4140209127 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f83200712d0 0x7f8320102dc0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f83140105b0 tx=0x7f8314009250 comp rx=0 tx=0).stop 2024-04-27T04:01:12.581 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:01:12.008+0000 7f83269f1700 1 -- 172.21.15.162:0/4140209127 shutdown_connections 2024-04-27T04:01:12.581 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:01:12.008+0000 7f83269f1700 1 --2- 172.21.15.162:0/4140209127 >> [v2:172.21.15.162:6800/1482851081,v1:172.21.15.162:6801/1482851081] conn(0x7f830803a090 0x7f830803c470 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:01:12.581 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:01:12.008+0000 7f83269f1700 1 --2- 172.21.15.162:0/4140209127 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f83200712d0 0x7f8320102dc0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:01:12.581 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:01:12.008+0000 7f83269f1700 1 --2- 172.21.15.162:0/4140209127 >> [v2:172.21.15.162:6800/8067198,v1:172.21.15.162:6801/8067198] conn(0x7f83080368a0 0x7f8308038d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:01:12.582 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:01:12.008+0000 7f83269f1700 1 -- 172.21.15.162:0/4140209127 >> 172.21.15.162:0/4140209127 conn(0x7f83200fcea0 msgr2=0x7f83200fda10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:01:12.582 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:01:12.008+0000 7f83269f1700 1 -- 172.21.15.162:0/4140209127 shutdown_connections 2024-04-27T04:01:12.582 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:01:12.008+0000 7f83269f1700 1 -- 172.21.15.162:0/4140209127 wait complete. 2024-04-27T04:01:12.582 INFO:teuthology.orchestra.run.smithi162.stdout:mgr epoch 9 is available 2024-04-27T04:01:12.582 INFO:teuthology.orchestra.run.smithi162.stdout:Generating a dashboard self-signed certificate... 2024-04-27T04:01:12.889 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:01:12 smithi162 ceph-mon[26841]: [27/Apr/2024:04:01:11] ENGINE Bus STARTING 2024-04-27T04:01:12.889 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:01:12 smithi162 ceph-mon[26841]: [27/Apr/2024:04:01:11] ENGINE Serving on https://172.21.15.162:7150 2024-04-27T04:01:12.889 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:01:12 smithi162 ceph-mon[26841]: [27/Apr/2024:04:01:11] ENGINE Bus STARTED 2024-04-27T04:01:12.889 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:01:12 smithi162 ceph-mon[26841]: from='mgr.14162 172.21.15.162:0/1568051568' entity='mgr.smithi162.oennfn' 2024-04-27T04:01:12.889 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:01:12 smithi162 ceph-mon[26841]: from='client.14166 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-04-27T04:01:12.889 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:01:12 smithi162 ceph-mon[26841]: mgrmap e11: smithi162.oennfn(active, since 1.00598s) 2024-04-27T04:01:12.889 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:01:12 smithi162 ceph-mon[26841]: from='client.14166 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-04-27T04:01:12.889 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:01:12 smithi162 ceph-mon[26841]: from='mgr.14162 172.21.15.162:0/1568051568' entity='mgr.smithi162.oennfn' 2024-04-27T04:01:13.971 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stdout Self-signed certificate created 2024-04-27T04:01:13.972 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:01:12.988+0000 7ff229fd3700 1 Processor -- start 2024-04-27T04:01:13.972 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:01:12.988+0000 7ff229fd3700 1 -- start start 2024-04-27T04:01:13.972 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:01:12.988+0000 7ff229fd3700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff224076bf0 0x7ff224105440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:01:13.972 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:01:12.988+0000 7ff229fd3700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff224105980 con 0x7ff224076bf0 2024-04-27T04:01:13.972 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:01:12.989+0000 7ff2237fe700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff224076bf0 0x7ff224105440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:01:13.972 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:01:12.989+0000 7ff2237fe700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff224076bf0 0x7ff224105440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:57348/0 (socket says 172.21.15.162:57348) 2024-04-27T04:01:13.972 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:01:12.989+0000 7ff2237fe700 1 -- 172.21.15.162:0/753721771 learned_addr learned my addr 172.21.15.162:0/753721771 (peer_addr_for_me v2:172.21.15.162:0/0) 2024-04-27T04:01:13.972 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:01:12.989+0000 7ff2237fe700 1 -- 172.21.15.162:0/753721771 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff224105ac0 con 0x7ff224076bf0 2024-04-27T04:01:13.972 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:01:12.989+0000 7ff2237fe700 1 --2- 172.21.15.162:0/753721771 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff224076bf0 0x7ff224105440 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7ff2140098e0 tx=0x7ff214009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=57bf9c68ea6e51f5 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:01:13.972 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:01:12.989+0000 7ff2227fc700 1 -- 172.21.15.162:0/753721771 <== mon.0 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7ff21400b860 con 0x7ff224076bf0 2024-04-27T04:01:13.972 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:01:12.989+0000 7ff2227fc700 1 -- 172.21.15.162:0/753721771 <== mon.0 v2:172.21.15.162:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7ff21400be90 con 0x7ff224076bf0 2024-04-27T04:01:13.972 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:01:12.989+0000 7ff2227fc700 1 -- 172.21.15.162:0/753721771 <== mon.0 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7ff2140103c0 con 0x7ff224076bf0 2024-04-27T04:01:13.972 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:01:12.990+0000 7ff229fd3700 1 -- 172.21.15.162:0/753721771 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff224076bf0 msgr2=0x7ff224105440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:01:13.973 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:01:12.990+0000 7ff229fd3700 1 --2- 172.21.15.162:0/753721771 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff224076bf0 0x7ff224105440 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7ff2140098e0 tx=0x7ff214009250 comp rx=0 tx=0).stop 2024-04-27T04:01:13.973 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:01:12.990+0000 7ff229fd3700 1 -- 172.21.15.162:0/753721771 shutdown_connections 2024-04-27T04:01:13.973 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:01:12.990+0000 7ff229fd3700 1 --2- 172.21.15.162:0/753721771 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff224076bf0 0x7ff224105440 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:01:13.973 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:01:12.990+0000 7ff229fd3700 1 -- 172.21.15.162:0/753721771 >> 172.21.15.162:0/753721771 conn(0x7ff2240fce90 msgr2=0x7ff2240ff2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:01:13.973 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:01:12.990+0000 7ff229fd3700 1 -- 172.21.15.162:0/753721771 shutdown_connections 2024-04-27T04:01:13.973 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:01:12.990+0000 7ff229fd3700 1 -- 172.21.15.162:0/753721771 wait complete. 2024-04-27T04:01:13.973 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:01:12.990+0000 7ff229fd3700 1 Processor -- start 2024-04-27T04:01:13.973 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:01:12.991+0000 7ff229fd3700 1 -- start start 2024-04-27T04:01:13.973 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:01:12.991+0000 7ff229fd3700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff224076bf0 0x7ff224111620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:01:13.973 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:01:12.991+0000 7ff229fd3700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff224111b60 con 0x7ff224076bf0 2024-04-27T04:01:13.973 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:01:12.991+0000 7ff2237fe700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff224076bf0 0x7ff224111620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:01:13.973 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:01:12.991+0000 7ff2237fe700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff224076bf0 0x7ff224111620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:57356/0 (socket says 172.21.15.162:57356) 2024-04-27T04:01:13.973 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:01:12.991+0000 7ff2237fe700 1 -- 172.21.15.162:0/93182637 learned_addr learned my addr 172.21.15.162:0/93182637 (peer_addr_for_me v2:172.21.15.162:0/0) 2024-04-27T04:01:13.973 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:01:12.991+0000 7ff2237fe700 1 -- 172.21.15.162:0/93182637 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff2140094f0 con 0x7ff224076bf0 2024-04-27T04:01:13.974 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:01:12.991+0000 7ff2237fe700 1 --2- 172.21.15.162:0/93182637 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff224076bf0 0x7ff224111620 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7ff214003080 tx=0x7ff214009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:01:13.974 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:01:12.991+0000 7ff20bfff700 1 -- 172.21.15.162:0/93182637 <== mon.0 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7ff214011920 con 0x7ff224076bf0 2024-04-27T04:01:13.974 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:01:12.991+0000 7ff20bfff700 1 -- 172.21.15.162:0/93182637 <== mon.0 v2:172.21.15.162:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7ff214010d10 con 0x7ff224076bf0 2024-04-27T04:01:13.974 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:01:12.992+0000 7ff20bfff700 1 -- 172.21.15.162:0/93182637 <== mon.0 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7ff21401ae50 con 0x7ff224076bf0 2024-04-27T04:01:13.974 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:01:12.992+0000 7ff229fd3700 1 -- 172.21.15.162:0/93182637 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff224111d60 con 0x7ff224076bf0 2024-04-27T04:01:13.974 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:01:12.992+0000 7ff229fd3700 1 -- 172.21.15.162:0/93182637 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff2241128d0 con 0x7ff224076bf0 2024-04-27T04:01:13.974 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:01:12.992+0000 7ff20bfff700 1 -- 172.21.15.162:0/93182637 <== mon.0 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 11) v1 ==== 43283+0+0 (secure 0 0 0) 0x7ff214024480 con 0x7ff224076bf0 2024-04-27T04:01:13.974 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:01:12.992+0000 7ff20bfff700 1 --2- 172.21.15.162:0/93182637 >> [v2:172.21.15.162:6800/1482851081,v1:172.21.15.162:6801/1482851081] conn(0x7ff20c036770 0x7ff20c038c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:01:13.974 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:01:12.992+0000 7ff20bfff700 1 -- 172.21.15.162:0/93182637 <== mon.0 v2:172.21.15.162:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1151+0+0 (secure 0 0 0) 0x7ff21404a580 con 0x7ff224076bf0 2024-04-27T04:01:13.974 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:01:12.993+0000 7ff229fd3700 1 -- 172.21.15.162:0/93182637 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff22405f5c0 con 0x7ff224076bf0 2024-04-27T04:01:13.974 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:01:12.993+0000 7ff222ffd700 1 --2- 172.21.15.162:0/93182637 >> [v2:172.21.15.162:6800/1482851081,v1:172.21.15.162:6801/1482851081] conn(0x7ff20c036770 0x7ff20c038c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:01:13.974 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:01:12.993+0000 7ff222ffd700 1 --2- 172.21.15.162:0/93182637 >> [v2:172.21.15.162:6800/1482851081,v1:172.21.15.162:6801/1482851081] conn(0x7ff20c036770 0x7ff20c038c20 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7ff218009990 tx=0x7ff218006e40 comp rx=0 tx=0).ready entity=mgr.14162 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:01:13.974 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:01:12.998+0000 7ff20bfff700 1 -- 172.21.15.162:0/93182637 <== mon.0 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7ff214018710 con 0x7ff224076bf0 2024-04-27T04:01:13.974 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:01:13.200+0000 7ff229fd3700 1 -- 172.21.15.162:0/93182637 --> [v2:172.21.15.162:6800/1482851081,v1:172.21.15.162:6801/1482851081] -- mgr_command(tid 0: {"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}) v1 -- 0x7ff2240fe0f0 con 0x7ff20c036770 2024-04-27T04:01:13.975 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:01:13.388+0000 7ff20bfff700 1 -- 172.21.15.162:0/93182637 <== mgr.14162 v2:172.21.15.162:6800/1482851081 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+32 (secure 0 0 0) 0x7ff2240fe0f0 con 0x7ff20c036770 2024-04-27T04:01:13.975 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:01:13.391+0000 7ff229fd3700 1 -- 172.21.15.162:0/93182637 >> [v2:172.21.15.162:6800/1482851081,v1:172.21.15.162:6801/1482851081] conn(0x7ff20c036770 msgr2=0x7ff20c038c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:01:13.975 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:01:13.391+0000 7ff229fd3700 1 --2- 172.21.15.162:0/93182637 >> [v2:172.21.15.162:6800/1482851081,v1:172.21.15.162:6801/1482851081] conn(0x7ff20c036770 0x7ff20c038c20 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7ff218009990 tx=0x7ff218006e40 comp rx=0 tx=0).stop 2024-04-27T04:01:13.975 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:01:13.391+0000 7ff229fd3700 1 -- 172.21.15.162:0/93182637 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff224076bf0 msgr2=0x7ff224111620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:01:13.975 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:01:13.391+0000 7ff229fd3700 1 --2- 172.21.15.162:0/93182637 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff224076bf0 0x7ff224111620 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7ff214003080 tx=0x7ff214009250 comp rx=0 tx=0).stop 2024-04-27T04:01:13.975 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:01:13.391+0000 7ff229fd3700 1 -- 172.21.15.162:0/93182637 shutdown_connections 2024-04-27T04:01:13.975 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:01:13.391+0000 7ff229fd3700 1 --2- 172.21.15.162:0/93182637 >> [v2:172.21.15.162:6800/1482851081,v1:172.21.15.162:6801/1482851081] conn(0x7ff20c036770 0x7ff20c038c20 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:01:13.975 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:01:13.391+0000 7ff229fd3700 1 --2- 172.21.15.162:0/93182637 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff224076bf0 0x7ff224111620 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:01:13.975 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:01:13.391+0000 7ff229fd3700 1 -- 172.21.15.162:0/93182637 >> 172.21.15.162:0/93182637 conn(0x7ff2240fce90 msgr2=0x7ff2240fd9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:01:13.975 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:01:13.391+0000 7ff229fd3700 1 -- 172.21.15.162:0/93182637 shutdown_connections 2024-04-27T04:01:13.975 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:01:13.391+0000 7ff229fd3700 1 -- 172.21.15.162:0/93182637 wait complete. 2024-04-27T04:01:13.975 INFO:teuthology.orchestra.run.smithi162.stdout:Creating initial admin user... 2024-04-27T04:01:14.606 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:01:14 smithi162 ceph-mon[26841]: from='client.14174 -' entity='client.admin' cmd=[{"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}]: dispatch 2024-04-27T04:01:14.606 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:01:14 smithi162 ceph-mon[26841]: from='mgr.14162 172.21.15.162:0/1568051568' entity='mgr.smithi162.oennfn' 2024-04-27T04:01:14.606 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:01:14 smithi162 ceph-mon[26841]: from='mgr.14162 172.21.15.162:0/1568051568' entity='mgr.smithi162.oennfn' 2024-04-27T04:01:14.606 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:01:14 smithi162 ceph-mon[26841]: from='mgr.14162 172.21.15.162:0/1568051568' entity='mgr.smithi162.oennfn' 2024-04-27T04:01:14.606 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:01:14 smithi162 ceph-mon[26841]: mgrmap e12: smithi162.oennfn(active, since 2s) 2024-04-27T04:01:15.889 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:01:15 smithi162 ceph-mon[26841]: from='client.14176 -' entity='client.admin' cmd=[{"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}]: dispatch 2024-04-27T04:01:15.889 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:01:15 smithi162 ceph-mon[26841]: from='mgr.14162 172.21.15.162:0/1568051568' entity='mgr.smithi162.oennfn' 2024-04-27T04:01:15.993 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stdout {"username": "admin", "password": "$2b$12$JxxPDZrJLiAnr0BdKc3eNeHYPA2myew/w2TAenkYn5ZA/R5koX82e", "roles": ["administrator"], "name": null, "email": null, "lastUpdate": 1714190474, "enabled": true, "pwdExpirationDate": null, "pwdUpdateRequired": true} 2024-04-27T04:01:15.993 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:01:14.377+0000 7fe4db1e0700 1 Processor -- start 2024-04-27T04:01:15.993 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:01:14.377+0000 7fe4db1e0700 1 -- start start 2024-04-27T04:01:15.993 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:01:14.377+0000 7fe4db1e0700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe4d4076c00 0x7fe4d4105480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:01:15.993 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:01:14.377+0000 7fe4db1e0700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4d41059c0 con 0x7fe4d4076c00 2024-04-27T04:01:15.993 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:01:14.378+0000 7fe4d8f7c700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe4d4076c00 0x7fe4d4105480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:01:15.993 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:01:14.378+0000 7fe4d8f7c700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe4d4076c00 0x7fe4d4105480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:42146/0 (socket says 172.21.15.162:42146) 2024-04-27T04:01:15.993 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:01:14.378+0000 7fe4d8f7c700 1 -- 172.21.15.162:0/907081284 learned_addr learned my addr 172.21.15.162:0/907081284 (peer_addr_for_me v2:172.21.15.162:0/0) 2024-04-27T04:01:15.993 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:01:14.378+0000 7fe4d8f7c700 1 -- 172.21.15.162:0/907081284 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe4d4105b00 con 0x7fe4d4076c00 2024-04-27T04:01:15.993 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:01:14.378+0000 7fe4d8f7c700 1 --2- 172.21.15.162:0/907081284 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe4d4076c00 0x7fe4d4105480 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fe4c80098e0 tx=0x7fe4c8009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=73d83c902380f0b2 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:01:15.994 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:01:14.378+0000 7fe4d37fe700 1 -- 172.21.15.162:0/907081284 <== mon.0 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fe4c800b860 con 0x7fe4d4076c00 2024-04-27T04:01:15.994 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:01:14.378+0000 7fe4d37fe700 1 -- 172.21.15.162:0/907081284 <== mon.0 v2:172.21.15.162:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fe4c800be90 con 0x7fe4d4076c00 2024-04-27T04:01:15.994 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:01:14.378+0000 7fe4d37fe700 1 -- 172.21.15.162:0/907081284 <== mon.0 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fe4c80103c0 con 0x7fe4d4076c00 2024-04-27T04:01:15.994 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:01:14.379+0000 7fe4db1e0700 1 -- 172.21.15.162:0/907081284 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe4d4076c00 msgr2=0x7fe4d4105480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:01:15.994 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:01:14.379+0000 7fe4db1e0700 1 --2- 172.21.15.162:0/907081284 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe4d4076c00 0x7fe4d4105480 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fe4c80098e0 tx=0x7fe4c8009250 comp rx=0 tx=0).stop 2024-04-27T04:01:15.994 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:01:14.379+0000 7fe4db1e0700 1 -- 172.21.15.162:0/907081284 shutdown_connections 2024-04-27T04:01:15.994 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:01:14.379+0000 7fe4db1e0700 1 --2- 172.21.15.162:0/907081284 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe4d4076c00 0x7fe4d4105480 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:01:15.994 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:01:14.379+0000 7fe4db1e0700 1 -- 172.21.15.162:0/907081284 >> 172.21.15.162:0/907081284 conn(0x7fe4d40fcf50 msgr2=0x7fe4d40ff360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:01:15.994 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:01:14.379+0000 7fe4db1e0700 1 -- 172.21.15.162:0/907081284 shutdown_connections 2024-04-27T04:01:15.994 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:01:14.379+0000 7fe4db1e0700 1 -- 172.21.15.162:0/907081284 wait complete. 2024-04-27T04:01:15.994 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:01:14.379+0000 7fe4db1e0700 1 Processor -- start 2024-04-27T04:01:15.994 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:01:14.380+0000 7fe4db1e0700 1 -- start start 2024-04-27T04:01:15.994 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:01:14.380+0000 7fe4db1e0700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe4d4076c00 0x7fe4d4111670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:01:15.994 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:01:14.380+0000 7fe4db1e0700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4d4111bb0 con 0x7fe4d4076c00 2024-04-27T04:01:15.995 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:01:14.380+0000 7fe4d8f7c700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe4d4076c00 0x7fe4d4111670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:01:15.995 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:01:14.380+0000 7fe4d8f7c700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe4d4076c00 0x7fe4d4111670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:42152/0 (socket says 172.21.15.162:42152) 2024-04-27T04:01:15.995 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:01:14.380+0000 7fe4d8f7c700 1 -- 172.21.15.162:0/982024781 learned_addr learned my addr 172.21.15.162:0/982024781 (peer_addr_for_me v2:172.21.15.162:0/0) 2024-04-27T04:01:15.995 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:01:14.380+0000 7fe4d8f7c700 1 -- 172.21.15.162:0/982024781 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe4c80094f0 con 0x7fe4d4076c00 2024-04-27T04:01:15.995 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:01:14.380+0000 7fe4d8f7c700 1 --2- 172.21.15.162:0/982024781 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe4d4076c00 0x7fe4d4111670 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fe4c8003080 tx=0x7fe4c8009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:01:15.995 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:01:14.381+0000 7fe4d17fa700 1 -- 172.21.15.162:0/982024781 <== mon.0 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fe4c8011920 con 0x7fe4d4076c00 2024-04-27T04:01:15.995 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:01:14.381+0000 7fe4d17fa700 1 -- 172.21.15.162:0/982024781 <== mon.0 v2:172.21.15.162:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fe4c8010d10 con 0x7fe4d4076c00 2024-04-27T04:01:15.995 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:01:14.381+0000 7fe4d17fa700 1 -- 172.21.15.162:0/982024781 <== mon.0 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fe4c801ae50 con 0x7fe4d4076c00 2024-04-27T04:01:15.995 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:01:14.381+0000 7fe4db1e0700 1 -- 172.21.15.162:0/982024781 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe4d4111db0 con 0x7fe4d4076c00 2024-04-27T04:01:15.995 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:01:14.381+0000 7fe4db1e0700 1 -- 172.21.15.162:0/982024781 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe4d4112920 con 0x7fe4d4076c00 2024-04-27T04:01:15.995 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:01:14.382+0000 7fe4d17fa700 1 -- 172.21.15.162:0/982024781 <== mon.0 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 12) v1 ==== 43389+0+0 (secure 0 0 0) 0x7fe4c802e440 con 0x7fe4d4076c00 2024-04-27T04:01:15.995 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:01:14.382+0000 7fe4db1e0700 1 -- 172.21.15.162:0/982024781 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe4d405f5c0 con 0x7fe4d4076c00 2024-04-27T04:01:15.995 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:01:14.382+0000 7fe4d17fa700 1 --2- 172.21.15.162:0/982024781 >> [v2:172.21.15.162:6800/1482851081,v1:172.21.15.162:6801/1482851081] conn(0x7fe4c4036850 0x7fe4c4038d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:01:15.996 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:01:14.382+0000 7fe4d17fa700 1 -- 172.21.15.162:0/982024781 <== mon.0 v2:172.21.15.162:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1151+0+0 (secure 0 0 0) 0x7fe4c804ab10 con 0x7fe4d4076c00 2024-04-27T04:01:15.996 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:01:14.382+0000 7fe4d3fff700 1 --2- 172.21.15.162:0/982024781 >> [v2:172.21.15.162:6800/1482851081,v1:172.21.15.162:6801/1482851081] conn(0x7fe4c4036850 0x7fe4c4038d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:01:15.996 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:01:14.382+0000 7fe4d3fff700 1 --2- 172.21.15.162:0/982024781 >> [v2:172.21.15.162:6800/1482851081,v1:172.21.15.162:6801/1482851081] conn(0x7fe4c4036850 0x7fe4c4038d00 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fe4c0009990 tx=0x7fe4c0006e40 comp rx=0 tx=0).ready entity=mgr.14162 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:01:15.996 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:01:14.387+0000 7fe4d17fa700 1 -- 172.21.15.162:0/982024781 <== mon.0 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7fe4c8018940 con 0x7fe4d4076c00 2024-04-27T04:01:15.996 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:01:14.595+0000 7fe4db1e0700 1 -- 172.21.15.162:0/982024781 --> [v2:172.21.15.162:6800/1482851081,v1:172.21.15.162:6801/1482851081] -- mgr_command(tid 0: {"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}) v1 -- 0x7fe4d405f3b0 con 0x7fe4c4036850 2024-04-27T04:01:15.996 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:01:14.819+0000 7fe4d17fa700 1 -- 172.21.15.162:0/982024781 <== mgr.14162 v2:172.21.15.162:6800/1482851081 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+252 (secure 0 0 0) 0x7fe4d405f3b0 con 0x7fe4c4036850 2024-04-27T04:01:15.996 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:01:14.821+0000 7fe4db1e0700 1 -- 172.21.15.162:0/982024781 >> [v2:172.21.15.162:6800/1482851081,v1:172.21.15.162:6801/1482851081] conn(0x7fe4c4036850 msgr2=0x7fe4c4038d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:01:15.996 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:01:14.821+0000 7fe4db1e0700 1 --2- 172.21.15.162:0/982024781 >> [v2:172.21.15.162:6800/1482851081,v1:172.21.15.162:6801/1482851081] conn(0x7fe4c4036850 0x7fe4c4038d00 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fe4c0009990 tx=0x7fe4c0006e40 comp rx=0 tx=0).stop 2024-04-27T04:01:15.996 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:01:14.821+0000 7fe4db1e0700 1 -- 172.21.15.162:0/982024781 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe4d4076c00 msgr2=0x7fe4d4111670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:01:15.996 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:01:14.821+0000 7fe4db1e0700 1 --2- 172.21.15.162:0/982024781 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe4d4076c00 0x7fe4d4111670 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fe4c8003080 tx=0x7fe4c8009250 comp rx=0 tx=0).stop 2024-04-27T04:01:15.996 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:01:14.821+0000 7fe4db1e0700 1 -- 172.21.15.162:0/982024781 shutdown_connections 2024-04-27T04:01:15.996 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:01:14.821+0000 7fe4db1e0700 1 --2- 172.21.15.162:0/982024781 >> [v2:172.21.15.162:6800/1482851081,v1:172.21.15.162:6801/1482851081] conn(0x7fe4c4036850 0x7fe4c4038d00 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:01:15.996 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:01:14.821+0000 7fe4db1e0700 1 --2- 172.21.15.162:0/982024781 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe4d4076c00 0x7fe4d4111670 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:01:15.996 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:01:14.821+0000 7fe4db1e0700 1 -- 172.21.15.162:0/982024781 >> 172.21.15.162:0/982024781 conn(0x7fe4d40fcf50 msgr2=0x7fe4d40fda70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:01:15.996 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:01:14.821+0000 7fe4db1e0700 1 -- 172.21.15.162:0/982024781 shutdown_connections 2024-04-27T04:01:15.997 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:01:14.821+0000 7fe4db1e0700 1 -- 172.21.15.162:0/982024781 wait complete. 2024-04-27T04:01:15.997 INFO:teuthology.orchestra.run.smithi162.stdout:Fetching dashboard port number... 2024-04-27T04:01:16.889 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:01:16 smithi162 ceph-mon[26841]: from='client.? 172.21.15.162:0/2543161238' entity='client.admin' cmd=[{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]: dispatch 2024-04-27T04:01:17.225 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stdout 8443 2024-04-27T04:01:17.225 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:01:16.436+0000 7f081ffaf700 1 Processor -- start 2024-04-27T04:01:17.225 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:01:16.436+0000 7f081ffaf700 1 -- start start 2024-04-27T04:01:17.225 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:01:16.436+0000 7f081ffaf700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0818101cc0 0x7f08181020d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:01:17.225 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:01:16.436+0000 7f081ffaf700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0818102610 con 0x7f0818101cc0 2024-04-27T04:01:17.225 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:01:16.436+0000 7f081dd4b700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0818101cc0 0x7f08181020d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:01:17.225 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:01:16.436+0000 7f081dd4b700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0818101cc0 0x7f08181020d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:42154/0 (socket says 172.21.15.162:42154) 2024-04-27T04:01:17.225 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:01:16.436+0000 7f081dd4b700 1 -- 172.21.15.162:0/3168388112 learned_addr learned my addr 172.21.15.162:0/3168388112 (peer_addr_for_me v2:172.21.15.162:0/0) 2024-04-27T04:01:17.225 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:01:16.436+0000 7f081dd4b700 1 -- 172.21.15.162:0/3168388112 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0818102750 con 0x7f0818101cc0 2024-04-27T04:01:17.225 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:01:16.437+0000 7f081dd4b700 1 --2- 172.21.15.162:0/3168388112 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0818101cc0 0x7f08181020d0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f081400bb80 tx=0x7f081400b4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=12a05ce5f1e4e13e server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:01:17.225 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:01:16.437+0000 7f081cd49700 1 -- 172.21.15.162:0/3168388112 <== mon.0 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f081400dd00 con 0x7f0818101cc0 2024-04-27T04:01:17.225 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:01:16.437+0000 7f081cd49700 1 -- 172.21.15.162:0/3168388112 <== mon.0 v2:172.21.15.162:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f081400de60 con 0x7f0818101cc0 2024-04-27T04:01:17.226 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:01:16.437+0000 7f081cd49700 1 -- 172.21.15.162:0/3168388112 <== mon.0 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f0814018920 con 0x7f0818101cc0 2024-04-27T04:01:17.226 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:01:16.437+0000 7f081ffaf700 1 -- 172.21.15.162:0/3168388112 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0818101cc0 msgr2=0x7f08181020d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:01:17.226 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:01:16.437+0000 7f081ffaf700 1 --2- 172.21.15.162:0/3168388112 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0818101cc0 0x7f08181020d0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f081400bb80 tx=0x7f081400b4f0 comp rx=0 tx=0).stop 2024-04-27T04:01:17.226 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:01:16.437+0000 7f081ffaf700 1 -- 172.21.15.162:0/3168388112 shutdown_connections 2024-04-27T04:01:17.226 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:01:16.437+0000 7f081ffaf700 1 --2- 172.21.15.162:0/3168388112 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0818101cc0 0x7f08181020d0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:01:17.226 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:01:16.437+0000 7f081ffaf700 1 -- 172.21.15.162:0/3168388112 >> 172.21.15.162:0/3168388112 conn(0x7f08180fc6f0 msgr2=0x7f08180feb00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:01:17.226 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:01:16.437+0000 7f081ffaf700 1 -- 172.21.15.162:0/3168388112 shutdown_connections 2024-04-27T04:01:17.226 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:01:16.437+0000 7f081ffaf700 1 -- 172.21.15.162:0/3168388112 wait complete. 2024-04-27T04:01:17.226 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:01:16.438+0000 7f081ffaf700 1 Processor -- start 2024-04-27T04:01:17.226 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:01:16.438+0000 7f081ffaf700 1 -- start start 2024-04-27T04:01:17.226 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:01:16.438+0000 7f081ffaf700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0818101cc0 0x7f081810f390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:01:17.226 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:01:16.438+0000 7f081ffaf700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f081810f8d0 con 0x7f0818101cc0 2024-04-27T04:01:17.226 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:01:16.438+0000 7f081dd4b700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0818101cc0 0x7f081810f390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:01:17.226 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:01:16.438+0000 7f081dd4b700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0818101cc0 0x7f081810f390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:42160/0 (socket says 172.21.15.162:42160) 2024-04-27T04:01:17.227 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:01:16.438+0000 7f081dd4b700 1 -- 172.21.15.162:0/2543161238 learned_addr learned my addr 172.21.15.162:0/2543161238 (peer_addr_for_me v2:172.21.15.162:0/0) 2024-04-27T04:01:17.227 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:01:16.438+0000 7f081dd4b700 1 -- 172.21.15.162:0/2543161238 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f081400b790 con 0x7f0818101cc0 2024-04-27T04:01:17.227 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:01:16.439+0000 7f081dd4b700 1 --2- 172.21.15.162:0/2543161238 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0818101cc0 0x7f081810f390 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f081400b760 tx=0x7f081400b4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:01:17.227 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:01:16.439+0000 7f080e7fc700 1 -- 172.21.15.162:0/2543161238 <== mon.0 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f0814010bd0 con 0x7f0818101cc0 2024-04-27T04:01:17.227 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:01:16.439+0000 7f080e7fc700 1 -- 172.21.15.162:0/2543161238 <== mon.0 v2:172.21.15.162:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f081401d050 con 0x7f0818101cc0 2024-04-27T04:01:17.227 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:01:16.439+0000 7f081ffaf700 1 -- 172.21.15.162:0/2543161238 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f081810fad0 con 0x7f0818101cc0 2024-04-27T04:01:17.227 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:01:16.439+0000 7f080e7fc700 1 -- 172.21.15.162:0/2543161238 <== mon.0 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f0814022670 con 0x7f0818101cc0 2024-04-27T04:01:17.227 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:01:16.439+0000 7f081ffaf700 1 -- 172.21.15.162:0/2543161238 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0818110640 con 0x7f0818101cc0 2024-04-27T04:01:17.227 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:01:16.439+0000 7f080e7fc700 1 -- 172.21.15.162:0/2543161238 <== mon.0 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 12) v1 ==== 43389+0+0 (secure 0 0 0) 0x7f0814022870 con 0x7f0818101cc0 2024-04-27T04:01:17.227 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:01:16.440+0000 7f081ffaf700 1 -- 172.21.15.162:0/2543161238 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f081805f5c0 con 0x7f0818101cc0 2024-04-27T04:01:17.227 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:01:16.440+0000 7f080e7fc700 1 --2- 172.21.15.162:0/2543161238 >> [v2:172.21.15.162:6800/1482851081,v1:172.21.15.162:6801/1482851081] conn(0x7f08040368a0 0x7f0804038d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:01:17.227 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:01:16.440+0000 7f080e7fc700 1 -- 172.21.15.162:0/2543161238 <== mon.0 v2:172.21.15.162:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1151+0+0 (secure 0 0 0) 0x7f0814014070 con 0x7f0818101cc0 2024-04-27T04:01:17.227 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:01:16.440+0000 7f081d54a700 1 --2- 172.21.15.162:0/2543161238 >> [v2:172.21.15.162:6800/1482851081,v1:172.21.15.162:6801/1482851081] conn(0x7f08040368a0 0x7f0804038d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:01:17.228 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:01:16.440+0000 7f081d54a700 1 --2- 172.21.15.162:0/2543161238 >> [v2:172.21.15.162:6800/1482851081,v1:172.21.15.162:6801/1482851081] conn(0x7f08040368a0 0x7f0804038d50 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f0808009990 tx=0x7f0808006e40 comp rx=0 tx=0).ready entity=mgr.14162 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:01:17.228 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:01:16.444+0000 7f080e7fc700 1 -- 172.21.15.162:0/2543161238 <== mon.0 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f081400e360 con 0x7f0818101cc0 2024-04-27T04:01:17.228 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:01:16.632+0000 7f081ffaf700 1 -- 172.21.15.162:0/2543161238 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"} v 0) v1 -- 0x7f081804bed0 con 0x7f0818101cc0 2024-04-27T04:01:17.228 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:01:16.633+0000 7f080e7fc700 1 -- 172.21.15.162:0/2543161238 <== mon.0 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]=0 v8) v1 ==== 112+0+5 (secure 0 0 0) 0x7f0814021b20 con 0x7f0818101cc0 2024-04-27T04:01:17.228 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:01:16.635+0000 7f081ffaf700 1 -- 172.21.15.162:0/2543161238 >> [v2:172.21.15.162:6800/1482851081,v1:172.21.15.162:6801/1482851081] conn(0x7f08040368a0 msgr2=0x7f0804038d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:01:17.228 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:01:16.635+0000 7f081ffaf700 1 --2- 172.21.15.162:0/2543161238 >> [v2:172.21.15.162:6800/1482851081,v1:172.21.15.162:6801/1482851081] conn(0x7f08040368a0 0x7f0804038d50 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f0808009990 tx=0x7f0808006e40 comp rx=0 tx=0).stop 2024-04-27T04:01:17.228 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:01:16.635+0000 7f081ffaf700 1 -- 172.21.15.162:0/2543161238 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0818101cc0 msgr2=0x7f081810f390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:01:17.228 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:01:16.635+0000 7f081ffaf700 1 --2- 172.21.15.162:0/2543161238 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0818101cc0 0x7f081810f390 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f081400b760 tx=0x7f081400b4f0 comp rx=0 tx=0).stop 2024-04-27T04:01:17.228 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:01:16.635+0000 7f081ffaf700 1 -- 172.21.15.162:0/2543161238 shutdown_connections 2024-04-27T04:01:17.228 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:01:16.635+0000 7f081ffaf700 1 --2- 172.21.15.162:0/2543161238 >> [v2:172.21.15.162:6800/1482851081,v1:172.21.15.162:6801/1482851081] conn(0x7f08040368a0 0x7f0804038d50 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:01:17.228 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:01:16.635+0000 7f081ffaf700 1 --2- 172.21.15.162:0/2543161238 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0818101cc0 0x7f081810f390 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:01:17.228 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:01:16.635+0000 7f081ffaf700 1 -- 172.21.15.162:0/2543161238 >> 172.21.15.162:0/2543161238 conn(0x7f08180fc6f0 msgr2=0x7f0818105cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:01:17.228 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:01:16.635+0000 7f081ffaf700 1 -- 172.21.15.162:0/2543161238 shutdown_connections 2024-04-27T04:01:17.228 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:01:16.635+0000 7f081ffaf700 1 -- 172.21.15.162:0/2543161238 wait complete. 2024-04-27T04:01:17.231 INFO:teuthology.orchestra.run.smithi162.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-04-27T04:01:17.231 INFO:teuthology.orchestra.run.smithi162.stdout:systemctl: stdout disabled 2024-04-27T04:01:17.239 INFO:teuthology.orchestra.run.smithi162.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-04-27T04:01:17.239 INFO:teuthology.orchestra.run.smithi162.stdout:systemctl: stdout inactive 2024-04-27T04:01:17.239 INFO:teuthology.orchestra.run.smithi162.stdout:firewalld.service is not enabled 2024-04-27T04:01:17.239 INFO:teuthology.orchestra.run.smithi162.stdout:Not possible to open ports <[8443]>. firewalld.service is not available 2024-04-27T04:01:17.241 INFO:teuthology.orchestra.run.smithi162.stdout:Ceph Dashboard is now available at: 2024-04-27T04:01:17.241 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-04-27T04:01:17.241 INFO:teuthology.orchestra.run.smithi162.stdout: URL: https://smithi162:8443/ 2024-04-27T04:01:17.241 INFO:teuthology.orchestra.run.smithi162.stdout: User: admin 2024-04-27T04:01:17.241 INFO:teuthology.orchestra.run.smithi162.stdout: Password: 42lopd43xn 2024-04-27T04:01:17.241 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-04-27T04:01:17.242 INFO:teuthology.orchestra.run.smithi162.stdout:Saving cluster configuration to /var/lib/ceph/7f9750d6-044a-11ef-bc93-c7b262605968/config directory 2024-04-27T04:01:17.242 INFO:teuthology.orchestra.run.smithi162.stdout:Enabling autotune for osd_memory_target 2024-04-27T04:01:18.430 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:01:17.673+0000 7fb51033d700 1 Processor -- start 2024-04-27T04:01:18.430 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:01:17.673+0000 7fb51033d700 1 -- start start 2024-04-27T04:01:18.430 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:01:17.673+0000 7fb51033d700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb508102880 0x7fb508102c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:01:18.430 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:01:17.673+0000 7fb51033d700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5081031d0 con 0x7fb508102880 2024-04-27T04:01:18.430 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:01:17.673+0000 7fb50e0d9700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb508102880 0x7fb508102c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:01:18.430 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:01:17.673+0000 7fb50e0d9700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb508102880 0x7fb508102c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:42162/0 (socket says 172.21.15.162:42162) 2024-04-27T04:01:18.430 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:01:17.673+0000 7fb50e0d9700 1 -- 172.21.15.162:0/2315407981 learned_addr learned my addr 172.21.15.162:0/2315407981 (peer_addr_for_me v2:172.21.15.162:0/0) 2024-04-27T04:01:18.430 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:01:17.674+0000 7fb50e0d9700 1 -- 172.21.15.162:0/2315407981 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb508103310 con 0x7fb508102880 2024-04-27T04:01:18.431 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:01:17.674+0000 7fb50e0d9700 1 --2- 172.21.15.162:0/2315407981 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb508102880 0x7fb508102c90 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fb5040098e0 tx=0x7fb504009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5531d1049e5d16c9 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:01:18.431 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:01:17.674+0000 7fb50d0d7700 1 -- 172.21.15.162:0/2315407981 <== mon.0 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fb50400b860 con 0x7fb508102880 2024-04-27T04:01:18.431 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:01:17.674+0000 7fb50d0d7700 1 -- 172.21.15.162:0/2315407981 <== mon.0 v2:172.21.15.162:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fb50400be90 con 0x7fb508102880 2024-04-27T04:01:18.431 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:01:17.674+0000 7fb50d0d7700 1 -- 172.21.15.162:0/2315407981 <== mon.0 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fb5040103c0 con 0x7fb508102880 2024-04-27T04:01:18.431 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:01:17.674+0000 7fb51033d700 1 -- 172.21.15.162:0/2315407981 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb508102880 msgr2=0x7fb508102c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:01:18.431 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:01:17.674+0000 7fb51033d700 1 --2- 172.21.15.162:0/2315407981 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb508102880 0x7fb508102c90 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fb5040098e0 tx=0x7fb504009250 comp rx=0 tx=0).stop 2024-04-27T04:01:18.431 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:01:17.675+0000 7fb51033d700 1 -- 172.21.15.162:0/2315407981 shutdown_connections 2024-04-27T04:01:18.431 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:01:17.675+0000 7fb51033d700 1 --2- 172.21.15.162:0/2315407981 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb508102880 0x7fb508102c90 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:01:18.431 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:01:17.675+0000 7fb51033d700 1 -- 172.21.15.162:0/2315407981 >> 172.21.15.162:0/2315407981 conn(0x7fb5080fe060 msgr2=0x7fb508100470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:01:18.431 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:01:17.675+0000 7fb51033d700 1 -- 172.21.15.162:0/2315407981 shutdown_connections 2024-04-27T04:01:18.431 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:01:17.675+0000 7fb51033d700 1 -- 172.21.15.162:0/2315407981 wait complete. 2024-04-27T04:01:18.431 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:01:17.675+0000 7fb51033d700 1 Processor -- start 2024-04-27T04:01:18.432 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:01:17.675+0000 7fb51033d700 1 -- start start 2024-04-27T04:01:18.432 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:01:17.675+0000 7fb51033d700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb508102880 0x7fb508111620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:01:18.432 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:01:17.675+0000 7fb51033d700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb508111b60 con 0x7fb508102880 2024-04-27T04:01:18.432 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:01:17.675+0000 7fb50e0d9700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb508102880 0x7fb508111620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:01:18.432 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:01:17.675+0000 7fb50e0d9700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb508102880 0x7fb508111620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:42170/0 (socket says 172.21.15.162:42170) 2024-04-27T04:01:18.432 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:01:17.675+0000 7fb50e0d9700 1 -- 172.21.15.162:0/4118834696 learned_addr learned my addr 172.21.15.162:0/4118834696 (peer_addr_for_me v2:172.21.15.162:0/0) 2024-04-27T04:01:18.432 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:01:17.676+0000 7fb50e0d9700 1 -- 172.21.15.162:0/4118834696 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb5040094f0 con 0x7fb508102880 2024-04-27T04:01:18.432 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:01:17.676+0000 7fb50e0d9700 1 --2- 172.21.15.162:0/4118834696 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb508102880 0x7fb508111620 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fb5040090b0 tx=0x7fb504009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:01:18.432 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:01:17.676+0000 7fb4feffd700 1 -- 172.21.15.162:0/4118834696 <== mon.0 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fb504010840 con 0x7fb508102880 2024-04-27T04:01:18.432 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:01:17.676+0000 7fb4feffd700 1 -- 172.21.15.162:0/4118834696 <== mon.0 v2:172.21.15.162:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fb504010e70 con 0x7fb508102880 2024-04-27T04:01:18.432 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:01:17.676+0000 7fb51033d700 1 -- 172.21.15.162:0/4118834696 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb508111d60 con 0x7fb508102880 2024-04-27T04:01:18.432 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:01:17.676+0000 7fb4feffd700 1 -- 172.21.15.162:0/4118834696 <== mon.0 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fb504024e40 con 0x7fb508102880 2024-04-27T04:01:18.432 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:01:17.676+0000 7fb51033d700 1 -- 172.21.15.162:0/4118834696 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb5081128d0 con 0x7fb508102880 2024-04-27T04:01:18.432 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:01:17.677+0000 7fb4feffd700 1 -- 172.21.15.162:0/4118834696 <== mon.0 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 12) v1 ==== 43389+0+0 (secure 0 0 0) 0x7fb50401a4e0 con 0x7fb508102880 2024-04-27T04:01:18.433 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:01:17.677+0000 7fb4feffd700 1 --2- 172.21.15.162:0/4118834696 >> [v2:172.21.15.162:6800/1482851081,v1:172.21.15.162:6801/1482851081] conn(0x7fb4f40368a0 0x7fb4f4038d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:01:18.433 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:01:17.677+0000 7fb4feffd700 1 -- 172.21.15.162:0/4118834696 <== mon.0 v2:172.21.15.162:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1151+0+0 (secure 0 0 0) 0x7fb50404a8a0 con 0x7fb508102880 2024-04-27T04:01:18.433 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:01:17.677+0000 7fb50d8d8700 1 --2- 172.21.15.162:0/4118834696 >> [v2:172.21.15.162:6800/1482851081,v1:172.21.15.162:6801/1482851081] conn(0x7fb4f40368a0 0x7fb4f4038d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:01:18.433 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:01:17.677+0000 7fb51033d700 1 -- 172.21.15.162:0/4118834696 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb50805f5c0 con 0x7fb508102880 2024-04-27T04:01:18.433 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:01:17.677+0000 7fb50d8d8700 1 --2- 172.21.15.162:0/4118834696 >> [v2:172.21.15.162:6800/1482851081,v1:172.21.15.162:6801/1482851081] conn(0x7fb4f40368a0 0x7fb4f4038d50 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fb4f8009990 tx=0x7fb4f8006e40 comp rx=0 tx=0).ready entity=mgr.14162 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:01:18.433 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:01:17.682+0000 7fb4feffd700 1 -- 172.21.15.162:0/4118834696 <== mon.0 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7fb504018940 con 0x7fb508102880 2024-04-27T04:01:18.433 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:01:17.821+0000 7fb4feffd700 1 -- 172.21.15.162:0/4118834696 <== mon.0 v2:172.21.15.162:3300/0 7 ==== mgrmap(e 13) v1 ==== 43433+0+0 (secure 0 0 0) 0x7fb504024430 con 0x7fb508102880 2024-04-27T04:01:18.433 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:01:17.869+0000 7fb51033d700 1 -- 172.21.15.162:0/4118834696 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command([{prefix=config set, name=osd_memory_target_autotune}] v 0) v1 -- 0x7fb50804bed0 con 0x7fb508102880 2024-04-27T04:01:18.433 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:01:17.869+0000 7fb4feffd700 1 -- 172.21.15.162:0/4118834696 <== mon.0 v2:172.21.15.162:3300/0 8 ==== mon_command_ack([{prefix=config set, name=osd_memory_target_autotune}]=0 v8)=0 v8) v1 ==== 127+0+0 (secure 0 0 0) 0x7fb5040480c0 con 0x7fb508102880 2024-04-27T04:01:18.433 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:01:17.871+0000 7fb51033d700 1 -- 172.21.15.162:0/4118834696 >> [v2:172.21.15.162:6800/1482851081,v1:172.21.15.162:6801/1482851081] conn(0x7fb4f40368a0 msgr2=0x7fb4f4038d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:01:18.433 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:01:17.871+0000 7fb51033d700 1 --2- 172.21.15.162:0/4118834696 >> [v2:172.21.15.162:6800/1482851081,v1:172.21.15.162:6801/1482851081] conn(0x7fb4f40368a0 0x7fb4f4038d50 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fb4f8009990 tx=0x7fb4f8006e40 comp rx=0 tx=0).stop 2024-04-27T04:01:18.433 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:01:17.872+0000 7fb51033d700 1 -- 172.21.15.162:0/4118834696 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb508102880 msgr2=0x7fb508111620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:01:18.433 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:01:17.872+0000 7fb51033d700 1 --2- 172.21.15.162:0/4118834696 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb508102880 0x7fb508111620 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fb5040090b0 tx=0x7fb504009250 comp rx=0 tx=0).stop 2024-04-27T04:01:18.434 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:01:17.872+0000 7fb51033d700 1 -- 172.21.15.162:0/4118834696 shutdown_connections 2024-04-27T04:01:18.434 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:01:17.872+0000 7fb51033d700 1 --2- 172.21.15.162:0/4118834696 >> [v2:172.21.15.162:6800/1482851081,v1:172.21.15.162:6801/1482851081] conn(0x7fb4f40368a0 0x7fb4f4038d50 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:01:18.434 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:01:17.872+0000 7fb51033d700 1 --2- 172.21.15.162:0/4118834696 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb508102880 0x7fb508111620 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:01:18.434 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:01:17.872+0000 7fb51033d700 1 -- 172.21.15.162:0/4118834696 >> 172.21.15.162:0/4118834696 conn(0x7fb5080fe060 msgr2=0x7fb5080feb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:01:18.434 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:01:17.872+0000 7fb51033d700 1 -- 172.21.15.162:0/4118834696 shutdown_connections 2024-04-27T04:01:18.434 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:01:17.872+0000 7fb51033d700 1 -- 172.21.15.162:0/4118834696 wait complete. 2024-04-27T04:01:19.136 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:01:18 smithi162 ceph-mon[26841]: mgrmap e13: smithi162.oennfn(active, since 6s) 2024-04-27T04:01:19.611 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:01:18.831+0000 7f4713493700 1 Processor -- start 2024-04-27T04:01:19.611 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:01:18.832+0000 7f4713493700 1 -- start start 2024-04-27T04:01:19.611 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:01:18.832+0000 7f4713493700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f470c100cb0 0x7f470c1010c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:01:19.611 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:01:18.832+0000 7f4713493700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f470c101600 con 0x7f470c100cb0 2024-04-27T04:01:19.611 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:01:18.832+0000 7f471122f700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f470c100cb0 0x7f470c1010c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:01:19.611 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:01:18.832+0000 7f471122f700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f470c100cb0 0x7f470c1010c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:42182/0 (socket says 172.21.15.162:42182) 2024-04-27T04:01:19.611 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:01:18.832+0000 7f471122f700 1 -- 172.21.15.162:0/2298955916 learned_addr learned my addr 172.21.15.162:0/2298955916 (peer_addr_for_me v2:172.21.15.162:0/0) 2024-04-27T04:01:19.611 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:01:18.832+0000 7f471122f700 1 -- 172.21.15.162:0/2298955916 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f470c101740 con 0x7f470c100cb0 2024-04-27T04:01:19.611 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:01:18.832+0000 7f471122f700 1 --2- 172.21.15.162:0/2298955916 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f470c100cb0 0x7f470c1010c0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f47080098e0 tx=0x7f4708009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7990c7d83a1f5306 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:01:19.612 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:01:18.833+0000 7f4703fff700 1 -- 172.21.15.162:0/2298955916 <== mon.0 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f470800b860 con 0x7f470c100cb0 2024-04-27T04:01:19.612 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:01:18.833+0000 7f4703fff700 1 -- 172.21.15.162:0/2298955916 <== mon.0 v2:172.21.15.162:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f470800be90 con 0x7f470c100cb0 2024-04-27T04:01:19.612 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:01:18.833+0000 7f4703fff700 1 -- 172.21.15.162:0/2298955916 <== mon.0 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f47080103c0 con 0x7f470c100cb0 2024-04-27T04:01:19.612 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:01:18.833+0000 7f4713493700 1 -- 172.21.15.162:0/2298955916 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f470c100cb0 msgr2=0x7f470c1010c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:01:19.612 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:01:18.833+0000 7f4713493700 1 --2- 172.21.15.162:0/2298955916 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f470c100cb0 0x7f470c1010c0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f47080098e0 tx=0x7f4708009250 comp rx=0 tx=0).stop 2024-04-27T04:01:19.612 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:01:18.833+0000 7f4713493700 1 -- 172.21.15.162:0/2298955916 shutdown_connections 2024-04-27T04:01:19.612 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:01:18.833+0000 7f4713493700 1 --2- 172.21.15.162:0/2298955916 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f470c100cb0 0x7f470c1010c0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:01:19.612 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:01:18.833+0000 7f4713493700 1 -- 172.21.15.162:0/2298955916 >> 172.21.15.162:0/2298955916 conn(0x7f470c0fc6f0 msgr2=0x7f470c0feb00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:01:19.612 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:01:18.833+0000 7f4713493700 1 -- 172.21.15.162:0/2298955916 shutdown_connections 2024-04-27T04:01:19.612 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:01:18.833+0000 7f4713493700 1 -- 172.21.15.162:0/2298955916 wait complete. 2024-04-27T04:01:19.612 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:01:18.834+0000 7f4713493700 1 Processor -- start 2024-04-27T04:01:19.612 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:01:18.834+0000 7f4713493700 1 -- start start 2024-04-27T04:01:19.612 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:01:18.834+0000 7f4713493700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f470c100cb0 0x7f470c1115e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:01:19.612 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:01:18.834+0000 7f4713493700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f470c111b20 con 0x7f470c100cb0 2024-04-27T04:01:19.613 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:01:18.834+0000 7f471122f700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f470c100cb0 0x7f470c1115e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:01:19.613 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:01:18.834+0000 7f471122f700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f470c100cb0 0x7f470c1115e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:42184/0 (socket says 172.21.15.162:42184) 2024-04-27T04:01:19.613 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:01:18.834+0000 7f471122f700 1 -- 172.21.15.162:0/1189907555 learned_addr learned my addr 172.21.15.162:0/1189907555 (peer_addr_for_me v2:172.21.15.162:0/0) 2024-04-27T04:01:19.613 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:01:18.834+0000 7f471122f700 1 -- 172.21.15.162:0/1189907555 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47080094f0 con 0x7f470c100cb0 2024-04-27T04:01:19.613 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:01:18.834+0000 7f471122f700 1 --2- 172.21.15.162:0/1189907555 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f470c100cb0 0x7f470c1115e0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f47080090b0 tx=0x7f4708009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:01:19.613 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:01:18.835+0000 7f4701ffb700 1 -- 172.21.15.162:0/1189907555 <== mon.0 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f4708010840 con 0x7f470c100cb0 2024-04-27T04:01:19.613 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:01:18.835+0000 7f4701ffb700 1 -- 172.21.15.162:0/1189907555 <== mon.0 v2:172.21.15.162:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f4708010e70 con 0x7f470c100cb0 2024-04-27T04:01:19.613 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:01:18.835+0000 7f4713493700 1 -- 172.21.15.162:0/1189907555 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f470c111d20 con 0x7f470c100cb0 2024-04-27T04:01:19.613 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:01:18.835+0000 7f4701ffb700 1 -- 172.21.15.162:0/1189907555 <== mon.0 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f4708024e40 con 0x7f470c100cb0 2024-04-27T04:01:19.613 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:01:18.835+0000 7f4713493700 1 -- 172.21.15.162:0/1189907555 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f470c112890 con 0x7f470c100cb0 2024-04-27T04:01:19.613 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:01:18.836+0000 7f4701ffb700 1 -- 172.21.15.162:0/1189907555 <== mon.0 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 13) v1 ==== 43433+0+0 (secure 0 0 0) 0x7f470801a530 con 0x7f470c100cb0 2024-04-27T04:01:19.613 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:01:18.836+0000 7f4713493700 1 -- 172.21.15.162:0/1189907555 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f470c05f5c0 con 0x7f470c100cb0 2024-04-27T04:01:19.613 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:01:18.836+0000 7f4701ffb700 1 --2- 172.21.15.162:0/1189907555 >> [v2:172.21.15.162:6800/1482851081,v1:172.21.15.162:6801/1482851081] conn(0x7f46f80368f0 0x7f46f8038da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:01:19.613 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:01:18.836+0000 7f4701ffb700 1 -- 172.21.15.162:0/1189907555 <== mon.0 v2:172.21.15.162:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1151+0+0 (secure 0 0 0) 0x7f470804aa00 con 0x7f470c100cb0 2024-04-27T04:01:19.613 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:01:18.836+0000 7f4710a2e700 1 --2- 172.21.15.162:0/1189907555 >> [v2:172.21.15.162:6800/1482851081,v1:172.21.15.162:6801/1482851081] conn(0x7f46f80368f0 0x7f46f8038da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:01:19.614 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:01:18.836+0000 7f4710a2e700 1 --2- 172.21.15.162:0/1189907555 >> [v2:172.21.15.162:6800/1482851081,v1:172.21.15.162:6801/1482851081] conn(0x7f46f80368f0 0x7f46f8038da0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f46fc009990 tx=0x7f46fc006e40 comp rx=0 tx=0).ready entity=mgr.14162 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:01:19.614 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:01:18.841+0000 7f4701ffb700 1 -- 172.21.15.162:0/1189907555 <== mon.0 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f4708018940 con 0x7f470c100cb0 2024-04-27T04:01:19.614 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:01:19.135+0000 7f4713493700 1 -- 172.21.15.162:0/1189907555 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command([{prefix=config-key set, key=mgr/dashboard/cluster/status}] v 0) v1 -- 0x7f470c04bed0 con 0x7f470c100cb0 2024-04-27T04:01:19.614 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:01:19.140+0000 7f4701ffb700 1 -- 172.21.15.162:0/1189907555 <== mon.0 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{prefix=config-key set, key=mgr/dashboard/cluster/status}]=0 set mgr/dashboard/cluster/status v33)=0 set mgr/dashboard/cluster/status v33) v1 ==== 153+0+0 (secure 0 0 0) 0x7f4708018350 con 0x7f470c100cb0 2024-04-27T04:01:19.614 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:01:19.142+0000 7f4713493700 1 -- 172.21.15.162:0/1189907555 >> [v2:172.21.15.162:6800/1482851081,v1:172.21.15.162:6801/1482851081] conn(0x7f46f80368f0 msgr2=0x7f46f8038da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:01:19.614 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:01:19.142+0000 7f4713493700 1 --2- 172.21.15.162:0/1189907555 >> [v2:172.21.15.162:6800/1482851081,v1:172.21.15.162:6801/1482851081] conn(0x7f46f80368f0 0x7f46f8038da0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f46fc009990 tx=0x7f46fc006e40 comp rx=0 tx=0).stop 2024-04-27T04:01:19.614 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:01:19.142+0000 7f4713493700 1 -- 172.21.15.162:0/1189907555 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f470c100cb0 msgr2=0x7f470c1115e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:01:19.614 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:01:19.142+0000 7f4713493700 1 --2- 172.21.15.162:0/1189907555 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f470c100cb0 0x7f470c1115e0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f47080090b0 tx=0x7f4708009250 comp rx=0 tx=0).stop 2024-04-27T04:01:19.614 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:01:19.142+0000 7f4713493700 1 -- 172.21.15.162:0/1189907555 shutdown_connections 2024-04-27T04:01:19.614 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:01:19.143+0000 7f4713493700 1 --2- 172.21.15.162:0/1189907555 >> [v2:172.21.15.162:6800/1482851081,v1:172.21.15.162:6801/1482851081] conn(0x7f46f80368f0 0x7f46f8038da0 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:01:19.614 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:01:19.143+0000 7f4713493700 1 --2- 172.21.15.162:0/1189907555 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f470c100cb0 0x7f470c1115e0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:01:19.614 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:01:19.143+0000 7f4713493700 1 -- 172.21.15.162:0/1189907555 >> 172.21.15.162:0/1189907555 conn(0x7f470c0fc6f0 msgr2=0x7f470c0fd180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:01:19.614 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:01:19.143+0000 7f4713493700 1 -- 172.21.15.162:0/1189907555 shutdown_connections 2024-04-27T04:01:19.614 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr 2024-04-27T04:01:19.143+0000 7f4713493700 1 -- 172.21.15.162:0/1189907555 wait complete. 2024-04-27T04:01:19.614 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/bin/ceph: stderr set mgr/dashboard/cluster/status 2024-04-27T04:01:19.615 INFO:teuthology.orchestra.run.smithi162.stdout:You can access the Ceph CLI as following in case of multi-cluster or non-default config: 2024-04-27T04:01:19.615 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-04-27T04:01:19.615 INFO:teuthology.orchestra.run.smithi162.stdout: sudo /home/ubuntu/cephtest/cephadm shell --fsid 7f9750d6-044a-11ef-bc93-c7b262605968 -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring 2024-04-27T04:01:19.615 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-04-27T04:01:19.615 INFO:teuthology.orchestra.run.smithi162.stdout:Or, if you are only running a single cluster on this host: 2024-04-27T04:01:19.615 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-04-27T04:01:19.615 INFO:teuthology.orchestra.run.smithi162.stdout: sudo /home/ubuntu/cephtest/cephadm shell 2024-04-27T04:01:19.615 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-04-27T04:01:19.615 INFO:teuthology.orchestra.run.smithi162.stdout:Please consider enabling telemetry to help improve Ceph: 2024-04-27T04:01:19.615 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-04-27T04:01:19.615 INFO:teuthology.orchestra.run.smithi162.stdout: ceph telemetry on 2024-04-27T04:01:19.615 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-04-27T04:01:19.615 INFO:teuthology.orchestra.run.smithi162.stdout:For more information see: 2024-04-27T04:01:19.615 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-04-27T04:01:19.616 INFO:teuthology.orchestra.run.smithi162.stdout: https://docs.ceph.com/docs/master/mgr/telemetry/ 2024-04-27T04:01:19.616 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-04-27T04:01:19.616 INFO:teuthology.orchestra.run.smithi162.stdout:Bootstrap complete. 2024-04-27T04:01:19.642 INFO:tasks.cephadm:Fetching config... 2024-04-27T04:01:19.642 DEBUG:teuthology.orchestra.run.smithi162:> set -ex 2024-04-27T04:01:19.643 DEBUG:teuthology.orchestra.run.smithi162:> dd if=/etc/ceph/ceph.conf of=/dev/stdout 2024-04-27T04:01:19.699 INFO:tasks.cephadm:Fetching client.admin keyring... 2024-04-27T04:01:19.699 DEBUG:teuthology.orchestra.run.smithi162:> set -ex 2024-04-27T04:01:19.699 DEBUG:teuthology.orchestra.run.smithi162:> dd if=/etc/ceph/ceph.client.admin.keyring of=/dev/stdout 2024-04-27T04:01:19.754 INFO:tasks.cephadm:Fetching mon keyring... 2024-04-27T04:01:19.755 DEBUG:teuthology.orchestra.run.smithi162:> set -ex 2024-04-27T04:01:19.755 DEBUG:teuthology.orchestra.run.smithi162:> sudo dd if=/var/lib/ceph/7f9750d6-044a-11ef-bc93-c7b262605968/mon.smithi162/keyring of=/dev/stdout 2024-04-27T04:01:19.819 INFO:tasks.cephadm:Fetching pub ssh key... 2024-04-27T04:01:19.819 DEBUG:teuthology.orchestra.run.smithi162:> set -ex 2024-04-27T04:01:19.819 DEBUG:teuthology.orchestra.run.smithi162:> dd if=/home/ubuntu/cephtest/ceph.pub of=/dev/stdout 2024-04-27T04:01:19.873 INFO:tasks.cephadm:Installing pub ssh key for root users... 2024-04-27T04:01:19.874 DEBUG:teuthology.orchestra.run.smithi162:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCsyNs/PPSvGfQvecvlcm0yIAIOC8AiCY43p0mnQkNKpgbEY8ztgp8kaJkd7OAHl1xOiuers1f+vwjTFg5H5BrK5lxKyQVNJKI05Uwm0Gbjig8QrRmDCLRU/6PxKqYFF68a8JClGlZFFtBSo7/IgHqj/KOzREI44MfnCDF8zODIiyeGIr9r2hydOb81UKmNvMbL2NdEb1aiUxjcqgDX1tec6uMZ0ztzyP0lE0l/TFBbNGW0WXYpD179+UMhhBVUUJuLeLdGqmEpEFb+NaPjv2Vj7eE1U7AWNguDCGfqdpWvhrCR0iEgLOXZSDDpsSsZKMWa7NlhUmy98OdbwwZ4O1do/iJYe0nTEeFxCdajSp1Ptk6FfxT7Lo15SBUY3aLHAUgsn90YMC3YTKY1/VetUX5vA8x9cP6i6YplNOwzCWxWH3193OyElJg5/eVRM5WcpC6JYruBz65A8w0HwTFldvLXHaH5SyP5LHkzYfOmGkaaeUVJlApyNccXQt3yp5vGiJk= ceph-7f9750d6-044a-11ef-bc93-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-04-27T04:01:19.946 INFO:teuthology.orchestra.run.smithi162.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCsyNs/PPSvGfQvecvlcm0yIAIOC8AiCY43p0mnQkNKpgbEY8ztgp8kaJkd7OAHl1xOiuers1f+vwjTFg5H5BrK5lxKyQVNJKI05Uwm0Gbjig8QrRmDCLRU/6PxKqYFF68a8JClGlZFFtBSo7/IgHqj/KOzREI44MfnCDF8zODIiyeGIr9r2hydOb81UKmNvMbL2NdEb1aiUxjcqgDX1tec6uMZ0ztzyP0lE0l/TFBbNGW0WXYpD179+UMhhBVUUJuLeLdGqmEpEFb+NaPjv2Vj7eE1U7AWNguDCGfqdpWvhrCR0iEgLOXZSDDpsSsZKMWa7NlhUmy98OdbwwZ4O1do/iJYe0nTEeFxCdajSp1Ptk6FfxT7Lo15SBUY3aLHAUgsn90YMC3YTKY1/VetUX5vA8x9cP6i6YplNOwzCWxWH3193OyElJg5/eVRM5WcpC6JYruBz65A8w0HwTFldvLXHaH5SyP5LHkzYfOmGkaaeUVJlApyNccXQt3yp5vGiJk= ceph-7f9750d6-044a-11ef-bc93-c7b262605968 2024-04-27T04:01:19.957 DEBUG:teuthology.orchestra.run.smithi181:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCsyNs/PPSvGfQvecvlcm0yIAIOC8AiCY43p0mnQkNKpgbEY8ztgp8kaJkd7OAHl1xOiuers1f+vwjTFg5H5BrK5lxKyQVNJKI05Uwm0Gbjig8QrRmDCLRU/6PxKqYFF68a8JClGlZFFtBSo7/IgHqj/KOzREI44MfnCDF8zODIiyeGIr9r2hydOb81UKmNvMbL2NdEb1aiUxjcqgDX1tec6uMZ0ztzyP0lE0l/TFBbNGW0WXYpD179+UMhhBVUUJuLeLdGqmEpEFb+NaPjv2Vj7eE1U7AWNguDCGfqdpWvhrCR0iEgLOXZSDDpsSsZKMWa7NlhUmy98OdbwwZ4O1do/iJYe0nTEeFxCdajSp1Ptk6FfxT7Lo15SBUY3aLHAUgsn90YMC3YTKY1/VetUX5vA8x9cP6i6YplNOwzCWxWH3193OyElJg5/eVRM5WcpC6JYruBz65A8w0HwTFldvLXHaH5SyP5LHkzYfOmGkaaeUVJlApyNccXQt3yp5vGiJk= ceph-7f9750d6-044a-11ef-bc93-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-04-27T04:01:19.996 INFO:teuthology.orchestra.run.smithi181.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCsyNs/PPSvGfQvecvlcm0yIAIOC8AiCY43p0mnQkNKpgbEY8ztgp8kaJkd7OAHl1xOiuers1f+vwjTFg5H5BrK5lxKyQVNJKI05Uwm0Gbjig8QrRmDCLRU/6PxKqYFF68a8JClGlZFFtBSo7/IgHqj/KOzREI44MfnCDF8zODIiyeGIr9r2hydOb81UKmNvMbL2NdEb1aiUxjcqgDX1tec6uMZ0ztzyP0lE0l/TFBbNGW0WXYpD179+UMhhBVUUJuLeLdGqmEpEFb+NaPjv2Vj7eE1U7AWNguDCGfqdpWvhrCR0iEgLOXZSDDpsSsZKMWa7NlhUmy98OdbwwZ4O1do/iJYe0nTEeFxCdajSp1Ptk6FfxT7Lo15SBUY3aLHAUgsn90YMC3YTKY1/VetUX5vA8x9cP6i6YplNOwzCWxWH3193OyElJg5/eVRM5WcpC6JYruBz65A8w0HwTFldvLXHaH5SyP5LHkzYfOmGkaaeUVJlApyNccXQt3yp5vGiJk= ceph-7f9750d6-044a-11ef-bc93-c7b262605968 2024-04-27T04:01:20.007 DEBUG:teuthology.orchestra.run.smithi162:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 7f9750d6-044a-11ef-bc93-c7b262605968 -- ceph config set mgr mgr/cephadm/allow_ptrace true 2024-04-27T04:01:20.261 INFO:teuthology.orchestra.run.smithi162.stderr:Inferring config /var/lib/ceph/7f9750d6-044a-11ef-bc93-c7b262605968/mon.smithi162/config 2024-04-27T04:01:20.383 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:01:20 smithi162 ceph-mon[26841]: from='client.? 172.21.15.162:0/1189907555' entity='client.admin' 2024-04-27T04:01:22.803 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:01:22.799+0000 7ff622309700 1 -- 172.21.15.162:0/3066481683 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff61c0fca40 msgr2=0x7ff61c0fce50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:01:22.803 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:01:22.799+0000 7ff622309700 1 --2- 172.21.15.162:0/3066481683 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff61c0fca40 0x7ff61c0fce50 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7ff60c0048c0 tx=0x7ff60c009e30 comp rx=0 tx=0).stop 2024-04-27T04:01:22.803 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:01:22.799+0000 7ff622309700 1 -- 172.21.15.162:0/3066481683 shutdown_connections 2024-04-27T04:01:22.803 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:01:22.799+0000 7ff622309700 1 --2- 172.21.15.162:0/3066481683 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff61c0fca40 0x7ff61c0fce50 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:01:22.803 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:01:22.799+0000 7ff622309700 1 -- 172.21.15.162:0/3066481683 >> 172.21.15.162:0/3066481683 conn(0x7ff61c0f81a0 msgr2=0x7ff61c0fa5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:01:22.803 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:01:22.799+0000 7ff622309700 1 -- 172.21.15.162:0/3066481683 shutdown_connections 2024-04-27T04:01:22.803 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:01:22.800+0000 7ff622309700 1 -- 172.21.15.162:0/3066481683 wait complete. 2024-04-27T04:01:22.803 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:01:22.800+0000 7ff622309700 1 Processor -- start 2024-04-27T04:01:22.803 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:01:22.800+0000 7ff622309700 1 -- start start 2024-04-27T04:01:22.803 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:01:22.801+0000 7ff622309700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff61c0fca40 0x7ff61c071d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:01:22.804 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:01:22.801+0000 7ff622309700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff61c072240 con 0x7ff61c0fca40 2024-04-27T04:01:22.804 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:01:22.801+0000 7ff61bfff700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff61c0fca40 0x7ff61c071d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:01:22.804 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:01:22.801+0000 7ff61bfff700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff61c0fca40 0x7ff61c071d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:42208/0 (socket says 172.21.15.162:42208) 2024-04-27T04:01:22.804 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:01:22.801+0000 7ff61bfff700 1 -- 172.21.15.162:0/3911784874 learned_addr learned my addr 172.21.15.162:0/3911784874 (peer_addr_for_me v2:172.21.15.162:0/0) 2024-04-27T04:01:22.804 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:01:22.801+0000 7ff61bfff700 1 -- 172.21.15.162:0/3911784874 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff60c005040 con 0x7ff61c0fca40 2024-04-27T04:01:22.804 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:01:22.801+0000 7ff61bfff700 1 --2- 172.21.15.162:0/3911784874 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff61c0fca40 0x7ff61c071d00 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7ff60c0096b0 tx=0x7ff60c009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:01:22.804 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:01:22.801+0000 7ff618ff9700 1 -- 172.21.15.162:0/3911784874 <== mon.0 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7ff60c00bed0 con 0x7ff61c0fca40 2024-04-27T04:01:22.804 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:01:22.801+0000 7ff618ff9700 1 -- 172.21.15.162:0/3911784874 <== mon.0 v2:172.21.15.162:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7ff60c00b5c0 con 0x7ff61c0fca40 2024-04-27T04:01:22.804 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:01:22.801+0000 7ff622309700 1 -- 172.21.15.162:0/3911784874 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff61c070310 con 0x7ff61c0fca40 2024-04-27T04:01:22.804 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:01:22.802+0000 7ff618ff9700 1 -- 172.21.15.162:0/3911784874 <== mon.0 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7ff60c01c5b0 con 0x7ff61c0fca40 2024-04-27T04:01:22.804 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:01:22.802+0000 7ff622309700 1 -- 172.21.15.162:0/3911784874 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff61c070800 con 0x7ff61c0fca40 2024-04-27T04:01:22.805 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:01:22.802+0000 7ff618ff9700 1 -- 172.21.15.162:0/3911784874 <== mon.0 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 13) v1 ==== 43433+0+0 (secure 0 0 0) 0x7ff60c01c820 con 0x7ff61c0fca40 2024-04-27T04:01:22.805 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:01:22.803+0000 7ff618ff9700 1 --2- 172.21.15.162:0/3911784874 >> [v2:172.21.15.162:6800/1482851081,v1:172.21.15.162:6801/1482851081] conn(0x7ff604036880 0x7ff604038d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:01:22.806 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:01:22.803+0000 7ff618ff9700 1 -- 172.21.15.162:0/3911784874 <== mon.0 v2:172.21.15.162:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1151+0+0 (secure 0 0 0) 0x7ff60c049a00 con 0x7ff61c0fca40 2024-04-27T04:01:22.806 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:01:22.803+0000 7ff61b7fe700 1 --2- 172.21.15.162:0/3911784874 >> [v2:172.21.15.162:6800/1482851081,v1:172.21.15.162:6801/1482851081] conn(0x7ff604036880 0x7ff604038d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:01:22.806 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:01:22.803+0000 7ff622309700 1 -- 172.21.15.162:0/3911784874 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff61c05f5c0 con 0x7ff61c0fca40 2024-04-27T04:01:22.806 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:01:22.803+0000 7ff61b7fe700 1 --2- 172.21.15.162:0/3911784874 >> [v2:172.21.15.162:6800/1482851081,v1:172.21.15.162:6801/1482851081] conn(0x7ff604036880 0x7ff604038d30 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7ff610009990 tx=0x7ff610006e40 comp rx=0 tx=0).ready entity=mgr.14162 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:01:22.813 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:01:22.811+0000 7ff618ff9700 1 -- 172.21.15.162:0/3911784874 <== mon.0 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7ff60c023300 con 0x7ff61c0fca40 2024-04-27T04:01:23.018 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:01:23.017+0000 7ff622309700 1 -- 172.21.15.162:0/3911784874 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/allow_ptrace}] v 0) v1 -- 0x7ff61c102830 con 0x7ff61c0fca40 2024-04-27T04:01:23.021 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:01:23.020+0000 7ff618ff9700 1 -- 172.21.15.162:0/3911784874 <== mon.0 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/allow_ptrace}]=0 v9)=0 v9) v1 ==== 125+0+0 (secure 0 0 0) 0x7ff60c0330c0 con 0x7ff61c0fca40 2024-04-27T04:01:23.024 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:01:23.023+0000 7ff622309700 1 -- 172.21.15.162:0/3911784874 >> [v2:172.21.15.162:6800/1482851081,v1:172.21.15.162:6801/1482851081] conn(0x7ff604036880 msgr2=0x7ff604038d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:01:23.024 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:01:23.023+0000 7ff622309700 1 --2- 172.21.15.162:0/3911784874 >> [v2:172.21.15.162:6800/1482851081,v1:172.21.15.162:6801/1482851081] conn(0x7ff604036880 0x7ff604038d30 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7ff610009990 tx=0x7ff610006e40 comp rx=0 tx=0).stop 2024-04-27T04:01:23.025 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:01:23.023+0000 7ff622309700 1 -- 172.21.15.162:0/3911784874 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff61c0fca40 msgr2=0x7ff61c071d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:01:23.025 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:01:23.023+0000 7ff622309700 1 --2- 172.21.15.162:0/3911784874 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff61c0fca40 0x7ff61c071d00 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7ff60c0096b0 tx=0x7ff60c009e30 comp rx=0 tx=0).stop 2024-04-27T04:01:23.025 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:01:23.023+0000 7ff622309700 1 -- 172.21.15.162:0/3911784874 shutdown_connections 2024-04-27T04:01:23.025 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:01:23.023+0000 7ff622309700 1 --2- 172.21.15.162:0/3911784874 >> [v2:172.21.15.162:6800/1482851081,v1:172.21.15.162:6801/1482851081] conn(0x7ff604036880 0x7ff604038d30 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:01:23.025 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:01:23.023+0000 7ff622309700 1 --2- 172.21.15.162:0/3911784874 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff61c0fca40 0x7ff61c071d00 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:01:23.025 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:01:23.023+0000 7ff622309700 1 -- 172.21.15.162:0/3911784874 >> 172.21.15.162:0/3911784874 conn(0x7ff61c0f81a0 msgr2=0x7ff61c103b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:01:23.025 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:01:23.023+0000 7ff622309700 1 -- 172.21.15.162:0/3911784874 shutdown_connections 2024-04-27T04:01:23.025 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:01:23.023+0000 7ff622309700 1 -- 172.21.15.162:0/3911784874 wait complete. 2024-04-27T04:01:23.810 INFO:tasks.cephadm:Distributing conf and client.admin keyring to all hosts + 0755 2024-04-27T04:01:23.810 DEBUG:teuthology.orchestra.run.smithi162:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 7f9750d6-044a-11ef-bc93-c7b262605968 -- ceph orch client-keyring set client.admin '*' --mode 0755 2024-04-27T04:01:24.139 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:01:24 smithi162 ceph-mon[26841]: from='client.? 172.21.15.162:0/3911784874' entity='client.admin' 2024-04-27T04:01:24.246 INFO:teuthology.orchestra.run.smithi162.stderr:Inferring config /var/lib/ceph/7f9750d6-044a-11ef-bc93-c7b262605968/mon.smithi162/config 2024-04-27T04:01:25.261 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:01:25 smithi162 ceph-mon[26841]: from='mgr.14162 172.21.15.162:0/1568051568' entity='mgr.smithi162.oennfn' 2024-04-27T04:01:25.261 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:01:25 smithi162 ceph-mon[26841]: from='mgr.14162 172.21.15.162:0/1568051568' entity='mgr.smithi162.oennfn' 2024-04-27T04:01:25.261 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:01:25 smithi162 ceph-mon[26841]: from='mgr.14162 172.21.15.162:0/1568051568' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "config rm", "who": "osd/host:smithi162", "name": "osd_memory_target"}]: dispatch 2024-04-27T04:01:25.261 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:01:25 smithi162 ceph-mon[26841]: from='mgr.14162 172.21.15.162:0/1568051568' entity='mgr.smithi162.oennfn' 2024-04-27T04:01:25.261 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:01:25 smithi162 ceph-mon[26841]: from='mgr.14162 172.21.15.162:0/1568051568' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi162", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-04-27T04:01:25.261 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:01:25 smithi162 ceph-mon[26841]: from='mgr.14162 172.21.15.162:0/1568051568' entity='mgr.smithi162.oennfn' cmd='[{"prefix": "auth get-or-create", "entity": "client.crash.smithi162", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]': finished 2024-04-27T04:01:25.262 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:01:25 smithi162 ceph-mon[26841]: from='mgr.14162 172.21.15.162:0/1568051568' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T04:01:25.262 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:01:25 smithi162 ceph-mon[26841]: Deploying daemon crash.smithi162 on smithi162 2024-04-27T04:01:25.803 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:01:25.800+0000 7fc4a18b9700 1 -- 172.21.15.162:0/1325903733 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc49c0fb930 msgr2=0x7fc49c0fbd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:01:25.804 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:01:25.800+0000 7fc4a18b9700 1 --2- 172.21.15.162:0/1325903733 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc49c0fb930 0x7fc49c0fbd40 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fc48c0048c0 tx=0x7fc48c009e30 comp rx=0 tx=0).stop 2024-04-27T04:01:25.804 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:01:25.800+0000 7fc4a18b9700 1 -- 172.21.15.162:0/1325903733 shutdown_connections 2024-04-27T04:01:25.804 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:01:25.800+0000 7fc4a18b9700 1 --2- 172.21.15.162:0/1325903733 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc49c0fb930 0x7fc49c0fbd40 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:01:25.804 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:01:25.800+0000 7fc4a18b9700 1 -- 172.21.15.162:0/1325903733 >> 172.21.15.162:0/1325903733 conn(0x7fc49c0f7070 msgr2=0x7fc49c0f94c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:01:25.804 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:01:25.800+0000 7fc4a18b9700 1 -- 172.21.15.162:0/1325903733 shutdown_connections 2024-04-27T04:01:25.804 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:01:25.800+0000 7fc4a18b9700 1 -- 172.21.15.162:0/1325903733 wait complete. 2024-04-27T04:01:25.804 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:01:25.801+0000 7fc4a18b9700 1 Processor -- start 2024-04-27T04:01:25.804 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:01:25.801+0000 7fc4a18b9700 1 -- start start 2024-04-27T04:01:25.805 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:01:25.801+0000 7fc4a18b9700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc49c0fb930 0x7fc49c10d070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:01:25.805 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:01:25.801+0000 7fc4a18b9700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc49c10d5b0 con 0x7fc49c0fb930 2024-04-27T04:01:25.805 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:01:25.801+0000 7fc49affd700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc49c0fb930 0x7fc49c10d070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:01:25.805 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:01:25.801+0000 7fc49affd700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc49c0fb930 0x7fc49c10d070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:50298/0 (socket says 172.21.15.162:50298) 2024-04-27T04:01:25.805 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:01:25.801+0000 7fc49affd700 1 -- 172.21.15.162:0/2242945360 learned_addr learned my addr 172.21.15.162:0/2242945360 (peer_addr_for_me v2:172.21.15.162:0/0) 2024-04-27T04:01:25.805 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:01:25.802+0000 7fc49affd700 1 -- 172.21.15.162:0/2242945360 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc48c005040 con 0x7fc49c0fb930 2024-04-27T04:01:25.805 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:01:25.802+0000 7fc49affd700 1 --2- 172.21.15.162:0/2242945360 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc49c0fb930 0x7fc49c10d070 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fc48c0096b0 tx=0x7fc48c009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:01:25.805 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:01:25.802+0000 7fc4837fe700 1 -- 172.21.15.162:0/2242945360 <== mon.0 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fc48c00bed0 con 0x7fc49c0fb930 2024-04-27T04:01:25.805 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:01:25.802+0000 7fc4837fe700 1 -- 172.21.15.162:0/2242945360 <== mon.0 v2:172.21.15.162:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fc48c00b5c0 con 0x7fc49c0fb930 2024-04-27T04:01:25.805 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:01:25.802+0000 7fc4a18b9700 1 -- 172.21.15.162:0/2242945360 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc49c10d7b0 con 0x7fc49c0fb930 2024-04-27T04:01:25.805 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:01:25.802+0000 7fc4837fe700 1 -- 172.21.15.162:0/2242945360 <== mon.0 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fc48c01c5b0 con 0x7fc49c0fb930 2024-04-27T04:01:25.805 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:01:25.802+0000 7fc4a18b9700 1 -- 172.21.15.162:0/2242945360 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc49c10e320 con 0x7fc49c0fb930 2024-04-27T04:01:25.806 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:01:25.803+0000 7fc4837fe700 1 -- 172.21.15.162:0/2242945360 <== mon.0 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 13) v1 ==== 43433+0+0 (secure 0 0 0) 0x7fc48c013a80 con 0x7fc49c0fb930 2024-04-27T04:01:25.806 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:01:25.803+0000 7fc4a18b9700 1 -- 172.21.15.162:0/2242945360 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc49c101e40 con 0x7fc49c0fb930 2024-04-27T04:01:25.806 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:01:25.803+0000 7fc4837fe700 1 --2- 172.21.15.162:0/2242945360 >> [v2:172.21.15.162:6800/1482851081,v1:172.21.15.162:6801/1482851081] conn(0x7fc4840368d0 0x7fc484038d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:01:25.806 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:01:25.803+0000 7fc4837fe700 1 -- 172.21.15.162:0/2242945360 <== mon.0 v2:172.21.15.162:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1151+0+0 (secure 0 0 0) 0x7fc48c049a00 con 0x7fc49c0fb930 2024-04-27T04:01:25.807 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:01:25.803+0000 7fc49a7fc700 1 --2- 172.21.15.162:0/2242945360 >> [v2:172.21.15.162:6800/1482851081,v1:172.21.15.162:6801/1482851081] conn(0x7fc4840368d0 0x7fc484038d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:01:25.807 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:01:25.804+0000 7fc49a7fc700 1 --2- 172.21.15.162:0/2242945360 >> [v2:172.21.15.162:6800/1482851081,v1:172.21.15.162:6801/1482851081] conn(0x7fc4840368d0 0x7fc484038d80 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fc490009990 tx=0x7fc490006e40 comp rx=0 tx=0).ready entity=mgr.14162 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:01:25.809 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:01:25.808+0000 7fc4837fe700 1 -- 172.21.15.162:0/2242945360 <== mon.0 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7fc48c011960 con 0x7fc49c0fb930 2024-04-27T04:01:26.015 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:01:26.014+0000 7fc4a18b9700 1 -- 172.21.15.162:0/2242945360 --> [v2:172.21.15.162:6800/1482851081,v1:172.21.15.162:6801/1482851081] -- mgr_command(tid 0: {"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}) v1 -- 0x7fc49c05f660 con 0x7fc4840368d0 2024-04-27T04:01:26.018 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:01:26.017+0000 7fc4837fe700 1 -- 172.21.15.162:0/2242945360 <== mgr.14162 v2:172.21.15.162:6800/1482851081 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7fc49c05f660 con 0x7fc4840368d0 2024-04-27T04:01:26.020 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:01:26.019+0000 7fc4a18b9700 1 -- 172.21.15.162:0/2242945360 >> [v2:172.21.15.162:6800/1482851081,v1:172.21.15.162:6801/1482851081] conn(0x7fc4840368d0 msgr2=0x7fc484038d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:01:26.020 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:01:26.019+0000 7fc4a18b9700 1 --2- 172.21.15.162:0/2242945360 >> [v2:172.21.15.162:6800/1482851081,v1:172.21.15.162:6801/1482851081] conn(0x7fc4840368d0 0x7fc484038d80 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fc490009990 tx=0x7fc490006e40 comp rx=0 tx=0).stop 2024-04-27T04:01:26.020 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:01:26.019+0000 7fc4a18b9700 1 -- 172.21.15.162:0/2242945360 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc49c0fb930 msgr2=0x7fc49c10d070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:01:26.020 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:01:26.019+0000 7fc4a18b9700 1 --2- 172.21.15.162:0/2242945360 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc49c0fb930 0x7fc49c10d070 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fc48c0096b0 tx=0x7fc48c009e30 comp rx=0 tx=0).stop 2024-04-27T04:01:26.020 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:01:26.019+0000 7fc4a18b9700 1 -- 172.21.15.162:0/2242945360 shutdown_connections 2024-04-27T04:01:26.021 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:01:26.019+0000 7fc4a18b9700 1 --2- 172.21.15.162:0/2242945360 >> [v2:172.21.15.162:6800/1482851081,v1:172.21.15.162:6801/1482851081] conn(0x7fc4840368d0 0x7fc484038d80 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:01:26.021 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:01:26.019+0000 7fc4a18b9700 1 --2- 172.21.15.162:0/2242945360 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc49c0fb930 0x7fc49c10d070 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:01:26.021 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:01:26.019+0000 7fc4a18b9700 1 -- 172.21.15.162:0/2242945360 >> 172.21.15.162:0/2242945360 conn(0x7fc49c0f7070 msgr2=0x7fc49c065c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:01:26.021 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:01:26.019+0000 7fc4a18b9700 1 -- 172.21.15.162:0/2242945360 shutdown_connections 2024-04-27T04:01:26.021 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:01:26.019+0000 7fc4a18b9700 1 -- 172.21.15.162:0/2242945360 wait complete. 2024-04-27T04:01:27.139 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:01:27 smithi162 ceph-mon[26841]: from='client.14186 -' entity='client.admin' cmd=[{"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}]: dispatch 2024-04-27T04:01:27.139 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:01:27 smithi162 ceph-mon[26841]: from='mgr.14162 172.21.15.162:0/1568051568' entity='mgr.smithi162.oennfn' 2024-04-27T04:01:27.214 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi181 2024-04-27T04:01:27.214 DEBUG:teuthology.orchestra.run.smithi181:> set -ex 2024-04-27T04:01:27.214 DEBUG:teuthology.orchestra.run.smithi181:> dd of=/etc/ceph/ceph.conf 2024-04-27T04:01:27.232 DEBUG:teuthology.orchestra.run.smithi181:> set -ex 2024-04-27T04:01:27.232 DEBUG:teuthology.orchestra.run.smithi181:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-04-27T04:01:27.290 INFO:tasks.cephadm:Adding host smithi181 to orchestrator... 2024-04-27T04:01:27.290 DEBUG:teuthology.orchestra.run.smithi162:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 7f9750d6-044a-11ef-bc93-c7b262605968 -- ceph orch host add smithi181 2024-04-27T04:01:27.791 INFO:teuthology.orchestra.run.smithi162.stderr:Inferring config /var/lib/ceph/7f9750d6-044a-11ef-bc93-c7b262605968/mon.smithi162/config 2024-04-27T04:01:29.139 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:01:28 smithi162 ceph-mon[26841]: from='mgr.14162 172.21.15.162:0/1568051568' entity='mgr.smithi162.oennfn' 2024-04-27T04:01:29.139 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:01:28 smithi162 ceph-mon[26841]: from='mgr.14162 172.21.15.162:0/1568051568' entity='mgr.smithi162.oennfn' 2024-04-27T04:01:29.139 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:01:28 smithi162 ceph-mon[26841]: from='mgr.14162 172.21.15.162:0/1568051568' entity='mgr.smithi162.oennfn' 2024-04-27T04:01:29.139 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:01:28 smithi162 ceph-mon[26841]: from='mgr.14162 172.21.15.162:0/1568051568' entity='mgr.smithi162.oennfn' 2024-04-27T04:01:29.139 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:01:28 smithi162 ceph-mon[26841]: Deploying daemon node-exporter.smithi162 on smithi162 2024-04-27T04:01:29.429 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:01:29.426+0000 7f14056d8700 1 -- 172.21.15.162:0/4181674471 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f14000fca60 msgr2=0x7f14000fce70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:01:29.429 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:01:29.426+0000 7f14056d8700 1 --2- 172.21.15.162:0/4181674471 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f14000fca60 0x7f14000fce70 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f13f0009c60 tx=0x7f13f0004d10 comp rx=0 tx=0).stop 2024-04-27T04:01:29.429 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:01:29.426+0000 7f14056d8700 1 -- 172.21.15.162:0/4181674471 shutdown_connections 2024-04-27T04:01:29.429 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:01:29.426+0000 7f14056d8700 1 --2- 172.21.15.162:0/4181674471 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f14000fca60 0x7f14000fce70 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:01:29.429 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:01:29.426+0000 7f14056d8700 1 -- 172.21.15.162:0/4181674471 >> 172.21.15.162:0/4181674471 conn(0x7f14000f8200 msgr2=0x7f14000fa610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:01:29.429 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:01:29.426+0000 7f14056d8700 1 -- 172.21.15.162:0/4181674471 shutdown_connections 2024-04-27T04:01:29.429 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:01:29.426+0000 7f14056d8700 1 -- 172.21.15.162:0/4181674471 wait complete. 2024-04-27T04:01:29.429 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:01:29.427+0000 7f14056d8700 1 Processor -- start 2024-04-27T04:01:29.429 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:01:29.427+0000 7f14056d8700 1 -- start start 2024-04-27T04:01:29.429 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:01:29.427+0000 7f14056d8700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f14000fca60 0x7f1400071d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:01:29.429 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:01:29.427+0000 7f14056d8700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1400072240 con 0x7f14000fca60 2024-04-27T04:01:29.430 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:01:29.427+0000 7f13feffd700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f14000fca60 0x7f1400071d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:01:29.430 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:01:29.427+0000 7f13feffd700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f14000fca60 0x7f1400071d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:50332/0 (socket says 172.21.15.162:50332) 2024-04-27T04:01:29.430 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:01:29.427+0000 7f13feffd700 1 -- 172.21.15.162:0/1520169668 learned_addr learned my addr 172.21.15.162:0/1520169668 (peer_addr_for_me v2:172.21.15.162:0/0) 2024-04-27T04:01:29.430 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:01:29.427+0000 7f13feffd700 1 -- 172.21.15.162:0/1520169668 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f13f0009940 con 0x7f14000fca60 2024-04-27T04:01:29.430 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:01:29.427+0000 7f13feffd700 1 --2- 172.21.15.162:0/1520169668 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f14000fca60 0x7f1400071d00 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f13f0004e60 tx=0x7f13f0004d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:01:29.430 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:01:29.428+0000 7f13e77fe700 1 -- 172.21.15.162:0/1520169668 <== mon.0 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f13f0012410 con 0x7f14000fca60 2024-04-27T04:01:29.430 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:01:29.428+0000 7f13e77fe700 1 -- 172.21.15.162:0/1520169668 <== mon.0 v2:172.21.15.162:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f13f0012a40 con 0x7f14000fca60 2024-04-27T04:01:29.430 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:01:29.428+0000 7f14056d8700 1 -- 172.21.15.162:0/1520169668 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1400070310 con 0x7f14000fca60 2024-04-27T04:01:29.430 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:01:29.428+0000 7f13e77fe700 1 -- 172.21.15.162:0/1520169668 <== mon.0 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f13f001ba40 con 0x7f14000fca60 2024-04-27T04:01:29.430 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:01:29.428+0000 7f14056d8700 1 -- 172.21.15.162:0/1520169668 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1400070800 con 0x7f14000fca60 2024-04-27T04:01:29.431 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:01:29.429+0000 7f13e77fe700 1 -- 172.21.15.162:0/1520169668 <== mon.0 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 13) v1 ==== 43433+0+0 (secure 0 0 0) 0x7f13f0019040 con 0x7f14000fca60 2024-04-27T04:01:29.431 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:01:29.429+0000 7f14056d8700 1 -- 172.21.15.162:0/1520169668 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f140005f5c0 con 0x7f14000fca60 2024-04-27T04:01:29.431 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:01:29.429+0000 7f13e77fe700 1 --2- 172.21.15.162:0/1520169668 >> [v2:172.21.15.162:6800/1482851081,v1:172.21.15.162:6801/1482851081] conn(0x7f13e80368d0 0x7f13e8038d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:01:29.431 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:01:29.429+0000 7f13e77fe700 1 -- 172.21.15.162:0/1520169668 <== mon.0 v2:172.21.15.162:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1151+0+0 (secure 0 0 0) 0x7f13f0049c50 con 0x7f14000fca60 2024-04-27T04:01:29.431 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:01:29.429+0000 7f13fe7fc700 1 --2- 172.21.15.162:0/1520169668 >> [v2:172.21.15.162:6800/1482851081,v1:172.21.15.162:6801/1482851081] conn(0x7f13e80368d0 0x7f13e8038d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:01:29.432 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:01:29.429+0000 7f13fe7fc700 1 --2- 172.21.15.162:0/1520169668 >> [v2:172.21.15.162:6800/1482851081,v1:172.21.15.162:6801/1482851081] conn(0x7f13e80368d0 0x7f13e8038d80 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f13f4009990 tx=0x7f13f4006e40 comp rx=0 tx=0).ready entity=mgr.14162 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:01:29.434 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:01:29.434+0000 7f13e77fe700 1 -- 172.21.15.162:0/1520169668 <== mon.0 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f13f002b8d0 con 0x7f14000fca60 2024-04-27T04:01:29.670 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:01:29.668+0000 7f14056d8700 1 -- 172.21.15.162:0/1520169668 --> [v2:172.21.15.162:6800/1482851081,v1:172.21.15.162:6801/1482851081] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi181", "target": ["mon-mgr", ""]}) v1 -- 0x7f14001064b0 con 0x7f13e80368d0 2024-04-27T04:01:30.888 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:01:30 smithi162 ceph-mon[26841]: from='client.14188 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi181", "target": ["mon-mgr", ""]}]: dispatch 2024-04-27T04:01:30.888 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:01:30 smithi162 ceph-mon[26841]: Deploying cephadm binary to smithi181 2024-04-27T04:01:31.511 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:01:31.509+0000 7f13e77fe700 1 -- 172.21.15.162:0/1520169668 <== mgr.14162 v2:172.21.15.162:6800/1482851081 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+49 (secure 0 0 0) 0x7f14001064b0 con 0x7f13e80368d0 2024-04-27T04:01:31.511 INFO:teuthology.orchestra.run.smithi162.stdout:Added host 'smithi181' with addr '172.21.15.181' 2024-04-27T04:01:31.513 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:01:31.512+0000 7f14056d8700 1 -- 172.21.15.162:0/1520169668 >> [v2:172.21.15.162:6800/1482851081,v1:172.21.15.162:6801/1482851081] conn(0x7f13e80368d0 msgr2=0x7f13e8038d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:01:31.513 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:01:31.512+0000 7f14056d8700 1 --2- 172.21.15.162:0/1520169668 >> [v2:172.21.15.162:6800/1482851081,v1:172.21.15.162:6801/1482851081] conn(0x7f13e80368d0 0x7f13e8038d80 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f13f4009990 tx=0x7f13f4006e40 comp rx=0 tx=0).stop 2024-04-27T04:01:31.513 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:01:31.512+0000 7f14056d8700 1 -- 172.21.15.162:0/1520169668 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f14000fca60 msgr2=0x7f1400071d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:01:31.513 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:01:31.512+0000 7f14056d8700 1 --2- 172.21.15.162:0/1520169668 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f14000fca60 0x7f1400071d00 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f13f0004e60 tx=0x7f13f0004d10 comp rx=0 tx=0).stop 2024-04-27T04:01:31.514 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:01:31.512+0000 7f14056d8700 1 -- 172.21.15.162:0/1520169668 shutdown_connections 2024-04-27T04:01:31.514 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:01:31.512+0000 7f14056d8700 1 --2- 172.21.15.162:0/1520169668 >> [v2:172.21.15.162:6800/1482851081,v1:172.21.15.162:6801/1482851081] conn(0x7f13e80368d0 0x7f13e8038d80 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:01:31.514 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:01:31.512+0000 7f14056d8700 1 --2- 172.21.15.162:0/1520169668 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f14000fca60 0x7f1400071d00 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:01:31.514 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:01:31.512+0000 7f14056d8700 1 -- 172.21.15.162:0/1520169668 >> 172.21.15.162:0/1520169668 conn(0x7f14000f8200 msgr2=0x7f1400103ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:01:31.514 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:01:31.512+0000 7f14056d8700 1 -- 172.21.15.162:0/1520169668 shutdown_connections 2024-04-27T04:01:31.514 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:01:31.512+0000 7f14056d8700 1 -- 172.21.15.162:0/1520169668 wait complete. 2024-04-27T04:01:32.125 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:01:32 smithi162 ceph-mon[26841]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-27T04:01:32.126 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:01:32 smithi162 ceph-mon[26841]: from='mgr.14162 172.21.15.162:0/1568051568' entity='mgr.smithi162.oennfn' 2024-04-27T04:01:32.126 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:01:32 smithi162 ceph-mon[26841]: from='mgr.14162 172.21.15.162:0/1568051568' entity='mgr.smithi162.oennfn' 2024-04-27T04:01:32.126 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:01:32 smithi162 ceph-mon[26841]: Added host smithi181 2024-04-27T04:01:32.642 DEBUG:teuthology.orchestra.run.smithi162:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 7f9750d6-044a-11ef-bc93-c7b262605968 -- ceph orch host ls --format=json 2024-04-27T04:01:32.914 INFO:teuthology.orchestra.run.smithi162.stderr:Inferring config /var/lib/ceph/7f9750d6-044a-11ef-bc93-c7b262605968/mon.smithi162/config 2024-04-27T04:01:33.639 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:01:33 smithi162 ceph-mon[26841]: from='mgr.14162 172.21.15.162:0/1568051568' entity='mgr.smithi162.oennfn' 2024-04-27T04:01:33.639 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:01:33 smithi162 ceph-mon[26841]: from='mgr.14162 172.21.15.162:0/1568051568' entity='mgr.smithi162.oennfn' 2024-04-27T04:01:33.639 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:01:33 smithi162 ceph-mon[26841]: from='mgr.14162 172.21.15.162:0/1568051568' entity='mgr.smithi162.oennfn' 2024-04-27T04:01:33.639 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:01:33 smithi162 ceph-mon[26841]: from='mgr.14162 172.21.15.162:0/1568051568' entity='mgr.smithi162.oennfn' 2024-04-27T04:01:33.639 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:01:33 smithi162 ceph-mon[26841]: Deploying daemon alertmanager.smithi162 on smithi162 2024-04-27T04:01:34.601 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:01:34 smithi162 ceph-mon[26841]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-27T04:01:35.205 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:01:35.123+0000 7f7d605ac700 1 -- 172.21.15.162:0/2970779385 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7d580fb940 msgr2=0x7f7d580fbd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:01:35.205 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:01:35.123+0000 7f7d605ac700 1 --2- 172.21.15.162:0/2970779385 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7d580fb940 0x7f7d580fbd50 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f7d540048c0 tx=0x7f7d54009e30 comp rx=0 tx=0).stop 2024-04-27T04:01:35.205 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:01:35.123+0000 7f7d605ac700 1 -- 172.21.15.162:0/2970779385 shutdown_connections 2024-04-27T04:01:35.205 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:01:35.123+0000 7f7d605ac700 1 --2- 172.21.15.162:0/2970779385 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7d580fb940 0x7f7d580fbd50 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:01:35.206 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:01:35.123+0000 7f7d605ac700 1 -- 172.21.15.162:0/2970779385 >> 172.21.15.162:0/2970779385 conn(0x7f7d580f70a0 msgr2=0x7f7d580f94d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:01:35.206 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:01:35.123+0000 7f7d605ac700 1 -- 172.21.15.162:0/2970779385 shutdown_connections 2024-04-27T04:01:35.206 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:01:35.123+0000 7f7d605ac700 1 -- 172.21.15.162:0/2970779385 wait complete. 2024-04-27T04:01:35.206 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:01:35.123+0000 7f7d605ac700 1 Processor -- start 2024-04-27T04:01:35.206 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:01:35.123+0000 7f7d605ac700 1 -- start start 2024-04-27T04:01:35.206 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:01:35.123+0000 7f7d605ac700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7d580fb940 0x7f7d58065c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:01:35.206 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:01:35.123+0000 7f7d605ac700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d5810fd50 con 0x7f7d580fb940 2024-04-27T04:01:35.206 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:01:35.124+0000 7f7d5e348700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7d580fb940 0x7f7d58065c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:01:35.206 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:01:35.124+0000 7f7d5e348700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7d580fb940 0x7f7d58065c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:33234/0 (socket says 172.21.15.162:33234) 2024-04-27T04:01:35.206 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:01:35.124+0000 7f7d5e348700 1 -- 172.21.15.162:0/2852814108 learned_addr learned my addr 172.21.15.162:0/2852814108 (peer_addr_for_me v2:172.21.15.162:0/0) 2024-04-27T04:01:35.206 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:01:35.124+0000 7f7d5e348700 1 -- 172.21.15.162:0/2852814108 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d54005040 con 0x7f7d580fb940 2024-04-27T04:01:35.206 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:01:35.124+0000 7f7d5e348700 1 --2- 172.21.15.162:0/2852814108 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7d580fb940 0x7f7d58065c10 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f7d540096b0 tx=0x7f7d54009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:01:35.206 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:01:35.124+0000 7f7d4effd700 1 -- 172.21.15.162:0/2852814108 <== mon.0 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f7d5400bed0 con 0x7f7d580fb940 2024-04-27T04:01:35.207 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:01:35.124+0000 7f7d4effd700 1 -- 172.21.15.162:0/2852814108 <== mon.0 v2:172.21.15.162:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f7d5400b5c0 con 0x7f7d580fb940 2024-04-27T04:01:35.207 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:01:35.124+0000 7f7d605ac700 1 -- 172.21.15.162:0/2852814108 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7d5810fef0 con 0x7f7d580fb940 2024-04-27T04:01:35.207 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:01:35.124+0000 7f7d4effd700 1 -- 172.21.15.162:0/2852814108 <== mon.0 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f7d5401c5b0 con 0x7f7d580fb940 2024-04-27T04:01:35.207 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:01:35.124+0000 7f7d605ac700 1 -- 172.21.15.162:0/2852814108 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7d58110380 con 0x7f7d580fb940 2024-04-27T04:01:35.207 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:01:35.125+0000 7f7d4effd700 1 -- 172.21.15.162:0/2852814108 <== mon.0 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 13) v1 ==== 43433+0+0 (secure 0 0 0) 0x7f7d5401c710 con 0x7f7d580fb940 2024-04-27T04:01:35.207 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:01:35.125+0000 7f7d4effd700 1 --2- 172.21.15.162:0/2852814108 >> [v2:172.21.15.162:6800/1482851081,v1:172.21.15.162:6801/1482851081] conn(0x7f7d44036920 0x7f7d44038dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:01:35.207 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:01:35.125+0000 7f7d4effd700 1 -- 172.21.15.162:0/2852814108 <== mon.0 v2:172.21.15.162:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1151+0+0 (secure 0 0 0) 0x7f7d540499f0 con 0x7f7d580fb940 2024-04-27T04:01:35.207 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:01:35.125+0000 7f7d605ac700 1 -- 172.21.15.162:0/2852814108 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7d58105900 con 0x7f7d580fb940 2024-04-27T04:01:35.207 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:01:35.125+0000 7f7d5db47700 1 --2- 172.21.15.162:0/2852814108 >> [v2:172.21.15.162:6800/1482851081,v1:172.21.15.162:6801/1482851081] conn(0x7f7d44036920 0x7f7d44038dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:01:35.207 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:01:35.126+0000 7f7d5db47700 1 --2- 172.21.15.162:0/2852814108 >> [v2:172.21.15.162:6800/1482851081,v1:172.21.15.162:6801/1482851081] conn(0x7f7d44036920 0x7f7d44038dd0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f7d48009990 tx=0x7f7d48006e40 comp rx=0 tx=0).ready entity=mgr.14162 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:01:35.207 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:01:35.130+0000 7f7d4effd700 1 -- 172.21.15.162:0/2852814108 <== mon.0 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f7d54025400 con 0x7f7d580fb940 2024-04-27T04:01:35.371 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:01:35.371+0000 7f7d605ac700 1 -- 172.21.15.162:0/2852814108 --> [v2:172.21.15.162:6800/1482851081,v1:172.21.15.162:6801/1482851081] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f7d58101830 con 0x7f7d44036920 2024-04-27T04:01:35.372 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:01:35.372+0000 7f7d4effd700 1 -- 172.21.15.162:0/2852814108 <== mgr.14162 v2:172.21.15.162:6800/1482851081 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+161 (secure 0 0 0) 0x7f7d48009590 con 0x7f7d44036920 2024-04-27T04:01:35.372 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-04-27T04:01:35.372 INFO:teuthology.orchestra.run.smithi162.stdout:[{"addr": "172.21.15.162", "hostname": "smithi162", "labels": [], "status": ""}, {"addr": "172.21.15.181", "hostname": "smithi181", "labels": [], "status": ""}] 2024-04-27T04:01:35.374 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:01:35.374+0000 7f7d605ac700 1 -- 172.21.15.162:0/2852814108 >> [v2:172.21.15.162:6800/1482851081,v1:172.21.15.162:6801/1482851081] conn(0x7f7d44036920 msgr2=0x7f7d44038dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:01:35.375 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:01:35.374+0000 7f7d605ac700 1 --2- 172.21.15.162:0/2852814108 >> [v2:172.21.15.162:6800/1482851081,v1:172.21.15.162:6801/1482851081] conn(0x7f7d44036920 0x7f7d44038dd0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f7d48009990 tx=0x7f7d48006e40 comp rx=0 tx=0).stop 2024-04-27T04:01:35.375 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:01:35.374+0000 7f7d605ac700 1 -- 172.21.15.162:0/2852814108 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7d580fb940 msgr2=0x7f7d58065c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:01:35.375 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:01:35.374+0000 7f7d605ac700 1 --2- 172.21.15.162:0/2852814108 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7d580fb940 0x7f7d58065c10 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f7d540096b0 tx=0x7f7d54009e30 comp rx=0 tx=0).stop 2024-04-27T04:01:35.375 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:01:35.374+0000 7f7d605ac700 1 -- 172.21.15.162:0/2852814108 shutdown_connections 2024-04-27T04:01:35.375 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:01:35.374+0000 7f7d605ac700 1 --2- 172.21.15.162:0/2852814108 >> [v2:172.21.15.162:6800/1482851081,v1:172.21.15.162:6801/1482851081] conn(0x7f7d44036920 0x7f7d44038dd0 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:01:35.375 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:01:35.374+0000 7f7d605ac700 1 --2- 172.21.15.162:0/2852814108 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7d580fb940 0x7f7d58065c10 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:01:35.375 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:01:35.374+0000 7f7d605ac700 1 -- 172.21.15.162:0/2852814108 >> 172.21.15.162:0/2852814108 conn(0x7f7d580f70a0 msgr2=0x7f7d58102a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:01:35.375 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:01:35.374+0000 7f7d605ac700 1 -- 172.21.15.162:0/2852814108 shutdown_connections 2024-04-27T04:01:35.375 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:01:35.375+0000 7f7d605ac700 1 -- 172.21.15.162:0/2852814108 wait complete. 2024-04-27T04:01:36.370 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:01:36 smithi162 ceph-mon[26841]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-27T04:01:36.371 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:01:36 smithi162 ceph-mon[26841]: from='client.14190 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-27T04:01:36.371 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:01:36 smithi162 ceph-mon[26841]: from='mgr.14162 172.21.15.162:0/1568051568' entity='mgr.smithi162.oennfn' 2024-04-27T04:01:36.857 INFO:tasks.cephadm:Setting crush tunables to default 2024-04-27T04:01:36.857 DEBUG:teuthology.orchestra.run.smithi162:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 7f9750d6-044a-11ef-bc93-c7b262605968 -- ceph osd crush tunables default 2024-04-27T04:01:37.901 INFO:teuthology.orchestra.run.smithi162.stderr:Inferring config /var/lib/ceph/7f9750d6-044a-11ef-bc93-c7b262605968/mon.smithi162/config 2024-04-27T04:01:38.628 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:01:38 smithi162 ceph-mon[26841]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-27T04:01:39.777 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:01:39.776+0000 7f1af5c66700 1 -- 172.21.15.162:0/1903313610 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1af00fca60 msgr2=0x7f1af00fce70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:01:39.777 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:01:39.776+0000 7f1af5c66700 1 --2- 172.21.15.162:0/1903313610 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1af00fca60 0x7f1af00fce70 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f1ae00048c0 tx=0x7f1ae0009e30 comp rx=0 tx=0).stop 2024-04-27T04:01:39.777 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:01:39.776+0000 7f1af5c66700 1 -- 172.21.15.162:0/1903313610 shutdown_connections 2024-04-27T04:01:39.777 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:01:39.776+0000 7f1af5c66700 1 --2- 172.21.15.162:0/1903313610 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1af00fca60 0x7f1af00fce70 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:01:39.777 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:01:39.776+0000 7f1af5c66700 1 -- 172.21.15.162:0/1903313610 >> 172.21.15.162:0/1903313610 conn(0x7f1af00f81c0 msgr2=0x7f1af00fa5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:01:39.777 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:01:39.776+0000 7f1af5c66700 1 -- 172.21.15.162:0/1903313610 shutdown_connections 2024-04-27T04:01:39.778 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:01:39.776+0000 7f1af5c66700 1 -- 172.21.15.162:0/1903313610 wait complete. 2024-04-27T04:01:39.778 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:01:39.776+0000 7f1af5c66700 1 Processor -- start 2024-04-27T04:01:39.778 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:01:39.776+0000 7f1af5c66700 1 -- start start 2024-04-27T04:01:39.778 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:01:39.776+0000 7f1af5c66700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1af00fca60 0x7f1af0071d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:01:39.778 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:01:39.776+0000 7f1af5c66700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1af0072240 con 0x7f1af00fca60 2024-04-27T04:01:39.778 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:01:39.777+0000 7f1aef7fe700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1af00fca60 0x7f1af0071d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:01:39.778 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:01:39.777+0000 7f1aef7fe700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1af00fca60 0x7f1af0071d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:33254/0 (socket says 172.21.15.162:33254) 2024-04-27T04:01:39.778 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:01:39.777+0000 7f1aef7fe700 1 -- 172.21.15.162:0/548522848 learned_addr learned my addr 172.21.15.162:0/548522848 (peer_addr_for_me v2:172.21.15.162:0/0) 2024-04-27T04:01:39.778 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:01:39.777+0000 7f1aef7fe700 1 -- 172.21.15.162:0/548522848 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ae0005040 con 0x7f1af00fca60 2024-04-27T04:01:39.778 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:01:39.777+0000 7f1aef7fe700 1 --2- 172.21.15.162:0/548522848 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1af00fca60 0x7f1af0071d00 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f1ae0009680 tx=0x7f1ae0009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:01:39.778 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:01:39.777+0000 7f1ad7fff700 1 -- 172.21.15.162:0/548522848 <== mon.0 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f1ae000b5c0 con 0x7f1af00fca60 2024-04-27T04:01:39.779 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:01:39.777+0000 7f1ad7fff700 1 -- 172.21.15.162:0/548522848 <== mon.0 v2:172.21.15.162:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f1ae000be70 con 0x7f1af00fca60 2024-04-27T04:01:39.779 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:01:39.777+0000 7f1ad7fff700 1 -- 172.21.15.162:0/548522848 <== mon.0 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f1ae00135a0 con 0x7f1af00fca60 2024-04-27T04:01:39.779 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:01:39.777+0000 7f1af5c66700 1 -- 172.21.15.162:0/548522848 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1af0070310 con 0x7f1af00fca60 2024-04-27T04:01:39.780 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:01:39.777+0000 7f1af5c66700 1 -- 172.21.15.162:0/548522848 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1af0070800 con 0x7f1af00fca60 2024-04-27T04:01:39.781 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:01:39.778+0000 7f1ad7fff700 1 -- 172.21.15.162:0/548522848 <== mon.0 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 13) v1 ==== 43433+0+0 (secure 0 0 0) 0x7f1ae0013860 con 0x7f1af00fca60 2024-04-27T04:01:39.781 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:01:39.778+0000 7f1af5c66700 1 -- 172.21.15.162:0/548522848 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1af005f5c0 con 0x7f1af00fca60 2024-04-27T04:01:39.781 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:01:39.778+0000 7f1ad7fff700 1 --2- 172.21.15.162:0/548522848 >> [v2:172.21.15.162:6800/1482851081,v1:172.21.15.162:6801/1482851081] conn(0x7f1ad8036880 0x7f1ad8038d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:01:39.781 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:01:39.778+0000 7f1ad7fff700 1 -- 172.21.15.162:0/548522848 <== mon.0 v2:172.21.15.162:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1151+0+0 (secure 0 0 0) 0x7f1ae00493c0 con 0x7f1af00fca60 2024-04-27T04:01:39.781 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:01:39.778+0000 7f1aeeffd700 1 --2- 172.21.15.162:0/548522848 >> [v2:172.21.15.162:6800/1482851081,v1:172.21.15.162:6801/1482851081] conn(0x7f1ad8036880 0x7f1ad8038d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:01:39.782 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:01:39.779+0000 7f1aeeffd700 1 --2- 172.21.15.162:0/548522848 >> [v2:172.21.15.162:6800/1482851081,v1:172.21.15.162:6801/1482851081] conn(0x7f1ad8036880 0x7f1ad8038d30 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f1ae4009990 tx=0x7f1ae4006e40 comp rx=0 tx=0).ready entity=mgr.14162 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:01:39.783 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:01:39.783+0000 7f1ad7fff700 1 -- 172.21.15.162:0/548522848 <== mon.0 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f1ae0011960 con 0x7f1af00fca60 2024-04-27T04:01:39.978 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:01:39.978+0000 7f1af5c66700 1 -- 172.21.15.162:0/548522848 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) v1 -- 0x7f1af0106440 con 0x7f1af00fca60 2024-04-27T04:01:40.346 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:01:40.345+0000 7f1ad7fff700 1 -- 172.21.15.162:0/548522848 <== mon.0 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v4) v1 ==== 124+0+0 (secure 0 0 0) 0x7f1ae0011370 con 0x7f1af00fca60 2024-04-27T04:01:40.349 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:01:40.348+0000 7f1af5c66700 1 -- 172.21.15.162:0/548522848 >> [v2:172.21.15.162:6800/1482851081,v1:172.21.15.162:6801/1482851081] conn(0x7f1ad8036880 msgr2=0x7f1ad8038d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:01:40.349 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:01:40.348+0000 7f1af5c66700 1 --2- 172.21.15.162:0/548522848 >> [v2:172.21.15.162:6800/1482851081,v1:172.21.15.162:6801/1482851081] conn(0x7f1ad8036880 0x7f1ad8038d30 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f1ae4009990 tx=0x7f1ae4006e40 comp rx=0 tx=0).stop 2024-04-27T04:01:40.349 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:01:40.348+0000 7f1af5c66700 1 -- 172.21.15.162:0/548522848 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1af00fca60 msgr2=0x7f1af0071d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:01:40.349 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:01:40.348+0000 7f1af5c66700 1 --2- 172.21.15.162:0/548522848 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1af00fca60 0x7f1af0071d00 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f1ae0009680 tx=0x7f1ae0009e30 comp rx=0 tx=0).stop 2024-04-27T04:01:40.349 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:01:40.348+0000 7f1af5c66700 1 -- 172.21.15.162:0/548522848 shutdown_connections 2024-04-27T04:01:40.349 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:01:40.348+0000 7f1af5c66700 1 --2- 172.21.15.162:0/548522848 >> [v2:172.21.15.162:6800/1482851081,v1:172.21.15.162:6801/1482851081] conn(0x7f1ad8036880 0x7f1ad8038d30 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:01:40.350 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:01:40.348+0000 7f1af5c66700 1 --2- 172.21.15.162:0/548522848 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1af00fca60 0x7f1af0071d00 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:01:40.350 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:01:40.348+0000 7f1af5c66700 1 -- 172.21.15.162:0/548522848 >> 172.21.15.162:0/548522848 conn(0x7f1af00f81c0 msgr2=0x7f1af0103ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:01:40.350 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:01:40.349+0000 7f1af5c66700 1 -- 172.21.15.162:0/548522848 shutdown_connections 2024-04-27T04:01:40.350 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:01:40.349+0000 7f1af5c66700 1 -- 172.21.15.162:0/548522848 wait complete. 2024-04-27T04:01:40.351 INFO:teuthology.orchestra.run.smithi162.stderr:adjusted tunables profile to default 2024-04-27T04:01:40.633 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:01:40 smithi162 ceph-mon[26841]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-27T04:01:40.633 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:01:40 smithi162 ceph-mon[26841]: from='client.? 172.21.15.162:0/548522848' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-04-27T04:01:41.526 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:01:41 smithi162 ceph-mon[26841]: from='client.? 172.21.15.162:0/548522848' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-04-27T04:01:41.527 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:01:41 smithi162 ceph-mon[26841]: osdmap e4: 0 total, 0 up, 0 in 2024-04-27T04:01:41.792 INFO:tasks.cephadm:Adding mon.smithi162 on smithi162 2024-04-27T04:01:41.792 INFO:tasks.cephadm:Adding mon.smithi181 on smithi181 2024-04-27T04:01:41.792 DEBUG:teuthology.orchestra.run.smithi181:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 7f9750d6-044a-11ef-bc93-c7b262605968 -- ceph orch apply mon '2;smithi162:172.21.15.162=smithi162;smithi181:172.21.15.181=smithi181' 2024-04-27T04:01:42.132 INFO:teuthology.orchestra.run.smithi181.stderr:Inferring config /etc/ceph/ceph.conf 2024-04-27T04:01:42.181 INFO:teuthology.orchestra.run.smithi181.stderr:Inferring config /etc/ceph/ceph.conf 2024-04-27T04:01:42.638 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:01:42 smithi162 ceph-mon[26841]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-27T04:01:43.681 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:01:43.679+0000 7f13f96e7700 1 -- 172.21.15.181:0/1113727094 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f13f40fe0a0 msgr2=0x7f13f40fe4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:01:43.681 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:01:43.679+0000 7f13f96e7700 1 --2- 172.21.15.181:0/1113727094 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f13f40fe0a0 0x7f13f40fe4b0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f13e40048c0 tx=0x7f13e4009e30 comp rx=0 tx=0).stop 2024-04-27T04:01:43.681 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:01:43.679+0000 7f13f96e7700 1 -- 172.21.15.181:0/1113727094 shutdown_connections 2024-04-27T04:01:43.682 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:01:43.679+0000 7f13f96e7700 1 --2- 172.21.15.181:0/1113727094 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f13f40fe0a0 0x7f13f40fe4b0 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:01:43.682 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:01:43.679+0000 7f13f96e7700 1 -- 172.21.15.181:0/1113727094 >> 172.21.15.181:0/1113727094 conn(0x7f13f40f6840 msgr2=0x7f13f40f8c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:01:43.682 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:01:43.679+0000 7f13f96e7700 1 -- 172.21.15.181:0/1113727094 shutdown_connections 2024-04-27T04:01:43.682 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:01:43.679+0000 7f13f96e7700 1 -- 172.21.15.181:0/1113727094 wait complete. 2024-04-27T04:01:43.682 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:01:43.680+0000 7f13f96e7700 1 Processor -- start 2024-04-27T04:01:43.682 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:01:43.680+0000 7f13f96e7700 1 -- start start 2024-04-27T04:01:43.682 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:01:43.680+0000 7f13f96e7700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f13f40fe0a0 0x7f13f410d090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:01:43.682 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:01:43.680+0000 7f13f96e7700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13f410d5d0 con 0x7f13f40fe0a0 2024-04-27T04:01:43.682 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:01:43.680+0000 7f13f2ffd700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f13f40fe0a0 0x7f13f410d090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:01:43.682 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:01:43.680+0000 7f13f2ffd700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f13f40fe0a0 0x7f13f410d090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.181:32926/0 (socket says 172.21.15.181:32926) 2024-04-27T04:01:43.682 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:01:43.680+0000 7f13f2ffd700 1 -- 172.21.15.181:0/784106904 learned_addr learned my addr 172.21.15.181:0/784106904 (peer_addr_for_me v2:172.21.15.181:0/0) 2024-04-27T04:01:43.682 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:01:43.681+0000 7f13f2ffd700 1 -- 172.21.15.181:0/784106904 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f13e4005040 con 0x7f13f40fe0a0 2024-04-27T04:01:43.683 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:01:43.681+0000 7f13f2ffd700 1 --2- 172.21.15.181:0/784106904 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f13f40fe0a0 0x7f13f410d090 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f13e40096b0 tx=0x7f13e4009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:01:43.683 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:01:43.681+0000 7f13db7fe700 1 -- 172.21.15.181:0/784106904 <== mon.0 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f13e400bed0 con 0x7f13f40fe0a0 2024-04-27T04:01:43.683 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:01:43.681+0000 7f13db7fe700 1 -- 172.21.15.181:0/784106904 <== mon.0 v2:172.21.15.162:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f13e400b5c0 con 0x7f13f40fe0a0 2024-04-27T04:01:43.683 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:01:43.681+0000 7f13db7fe700 1 -- 172.21.15.181:0/784106904 <== mon.0 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f13e401c5b0 con 0x7f13f40fe0a0 2024-04-27T04:01:43.683 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:01:43.681+0000 7f13f96e7700 1 -- 172.21.15.181:0/784106904 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f13f410d7d0 con 0x7f13f40fe0a0 2024-04-27T04:01:43.683 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:01:43.681+0000 7f13f96e7700 1 -- 172.21.15.181:0/784106904 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f13f410e340 con 0x7f13f40fe0a0 2024-04-27T04:01:43.684 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:01:43.682+0000 7f13db7fe700 1 -- 172.21.15.181:0/784106904 <== mon.0 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 13) v1 ==== 43433+0+0 (secure 0 0 0) 0x7f13e401c820 con 0x7f13f40fe0a0 2024-04-27T04:01:43.684 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:01:43.682+0000 7f13db7fe700 1 --2- 172.21.15.181:0/784106904 >> [v2:172.21.15.162:6800/1482851081,v1:172.21.15.162:6801/1482851081] conn(0x7f13dc036880 0x7f13dc038d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:01:43.684 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:01:43.682+0000 7f13db7fe700 1 -- 172.21.15.181:0/784106904 <== mon.0 v2:172.21.15.162:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1151+0+0 (secure 0 0 0) 0x7f13e4049b40 con 0x7f13f40fe0a0 2024-04-27T04:01:43.684 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:01:43.682+0000 7f13f96e7700 1 -- 172.21.15.181:0/784106904 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f13f4102f70 con 0x7f13f40fe0a0 2024-04-27T04:01:43.684 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:01:43.682+0000 7f13f27fc700 1 --2- 172.21.15.181:0/784106904 >> [v2:172.21.15.162:6800/1482851081,v1:172.21.15.162:6801/1482851081] conn(0x7f13dc036880 0x7f13dc038d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:01:43.684 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:01:43.683+0000 7f13f27fc700 1 --2- 172.21.15.181:0/784106904 >> [v2:172.21.15.162:6800/1482851081,v1:172.21.15.162:6801/1482851081] conn(0x7f13dc036880 0x7f13dc038d30 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f13e8009990 tx=0x7f13e8006e40 comp rx=0 tx=0).ready entity=mgr.14162 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:01:43.687 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:01:43.687+0000 7f13db7fe700 1 -- 172.21.15.181:0/784106904 <== mon.0 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f13e4023300 con 0x7f13f40fe0a0 2024-04-27T04:01:43.881 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:01:43.881+0000 7f13f96e7700 1 -- 172.21.15.181:0/784106904 --> [v2:172.21.15.162:6800/1482851081,v1:172.21.15.162:6801/1482851081] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "placement": "2;smithi162:172.21.15.162=smithi162;smithi181:172.21.15.181=smithi181", "target": ["mon-mgr", ""]}) v1 -- 0x7f13f410e580 con 0x7f13dc036880 2024-04-27T04:01:43.886 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:01:43.887+0000 7f13db7fe700 1 -- 172.21.15.181:0/784106904 <== mgr.14162 v2:172.21.15.162:6800/1482851081 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+24 (secure 0 0 0) 0x7f13f410e580 con 0x7f13dc036880 2024-04-27T04:01:43.887 INFO:teuthology.orchestra.run.smithi181.stdout:Scheduled mon update... 2024-04-27T04:01:43.888 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:01:43 smithi162 ceph-mon[26841]: from='mgr.14162 172.21.15.162:0/1568051568' entity='mgr.smithi162.oennfn' 2024-04-27T04:01:43.888 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:01:43 smithi162 ceph-mon[26841]: from='mgr.14162 172.21.15.162:0/1568051568' entity='mgr.smithi162.oennfn' 2024-04-27T04:01:43.888 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:01:43 smithi162 ceph-mon[26841]: from='mgr.14162 172.21.15.162:0/1568051568' entity='mgr.smithi162.oennfn' 2024-04-27T04:01:43.888 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:01:43 smithi162 ceph-mon[26841]: from='mgr.14162 172.21.15.162:0/1568051568' entity='mgr.smithi162.oennfn' 2024-04-27T04:01:43.889 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:01:43 smithi162 ceph-mon[26841]: from='mgr.14162 172.21.15.162:0/1568051568' entity='mgr.smithi162.oennfn' 2024-04-27T04:01:43.889 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:01:43 smithi162 ceph-mon[26841]: from='mgr.14162 172.21.15.162:0/1568051568' entity='mgr.smithi162.oennfn' 2024-04-27T04:01:43.889 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:01:43 smithi162 ceph-mon[26841]: from='mgr.14162 172.21.15.162:0/1568051568' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "dashboard set-grafana-api-ssl-verify", "value": "false"}]: dispatch 2024-04-27T04:01:43.889 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:01:43 smithi162 ceph-mon[26841]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-grafana-api-ssl-verify", "value": "false"}]: dispatch 2024-04-27T04:01:43.889 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:01:43 smithi162 ceph-mon[26841]: from='mgr.14162 172.21.15.162:0/1568051568' entity='mgr.smithi162.oennfn' 2024-04-27T04:01:43.889 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:01:43 smithi162 ceph-mon[26841]: Deploying daemon grafana.smithi162 on smithi162 2024-04-27T04:01:43.889 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:01:43.889+0000 7f13f96e7700 1 -- 172.21.15.181:0/784106904 >> [v2:172.21.15.162:6800/1482851081,v1:172.21.15.162:6801/1482851081] conn(0x7f13dc036880 msgr2=0x7f13dc038d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:01:43.890 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:01:43.889+0000 7f13f96e7700 1 --2- 172.21.15.181:0/784106904 >> [v2:172.21.15.162:6800/1482851081,v1:172.21.15.162:6801/1482851081] conn(0x7f13dc036880 0x7f13dc038d30 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f13e8009990 tx=0x7f13e8006e40 comp rx=0 tx=0).stop 2024-04-27T04:01:43.890 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:01:43.889+0000 7f13f96e7700 1 -- 172.21.15.181:0/784106904 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f13f40fe0a0 msgr2=0x7f13f410d090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:01:43.890 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:01:43.889+0000 7f13f96e7700 1 --2- 172.21.15.181:0/784106904 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f13f40fe0a0 0x7f13f410d090 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f13e40096b0 tx=0x7f13e4009e30 comp rx=0 tx=0).stop 2024-04-27T04:01:43.890 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:01:43.889+0000 7f13f96e7700 1 -- 172.21.15.181:0/784106904 shutdown_connections 2024-04-27T04:01:43.890 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:01:43.889+0000 7f13f96e7700 1 --2- 172.21.15.181:0/784106904 >> [v2:172.21.15.162:6800/1482851081,v1:172.21.15.162:6801/1482851081] conn(0x7f13dc036880 0x7f13dc038d30 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:01:43.890 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:01:43.889+0000 7f13f96e7700 1 --2- 172.21.15.181:0/784106904 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f13f40fe0a0 0x7f13f410d090 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:01:43.890 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:01:43.889+0000 7f13f96e7700 1 -- 172.21.15.181:0/784106904 >> 172.21.15.181:0/784106904 conn(0x7f13f40f6840 msgr2=0x7f13f4103ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:01:43.890 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:01:43.889+0000 7f13f96e7700 1 -- 172.21.15.181:0/784106904 shutdown_connections 2024-04-27T04:01:43.890 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:01:43.889+0000 7f13f96e7700 1 -- 172.21.15.181:0/784106904 wait complete. 2024-04-27T04:01:44.466 DEBUG:teuthology.orchestra.run.smithi181:mon.smithi181> sudo journalctl -f -n 0 -u ceph-7f9750d6-044a-11ef-bc93-c7b262605968@mon.smithi181.service 2024-04-27T04:01:44.469 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-04-27T04:01:44.469 DEBUG:teuthology.orchestra.run.smithi181:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 7f9750d6-044a-11ef-bc93-c7b262605968 -- ceph mon dump -f json 2024-04-27T04:01:44.638 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:01:44 smithi162 ceph-mon[26841]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-27T04:01:44.638 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:01:44 smithi162 ceph-mon[26841]: from='mgr.14162 172.21.15.162:0/1568051568' entity='mgr.smithi162.oennfn' 2024-04-27T04:01:44.740 INFO:teuthology.orchestra.run.smithi181.stderr:Inferring config /etc/ceph/ceph.conf 2024-04-27T04:01:44.810 INFO:teuthology.orchestra.run.smithi181.stderr:Inferring config /etc/ceph/ceph.conf 2024-04-27T04:01:45.888 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:01:45 smithi162 ceph-mon[26841]: from='client.14194 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "2;smithi162:172.21.15.162=smithi162;smithi181:172.21.15.181=smithi181", "target": ["mon-mgr", ""]}]: dispatch 2024-04-27T04:01:45.888 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:01:45 smithi162 ceph-mon[26841]: Saving service mon spec with placement smithi162:172.21.15.162=smithi162;smithi181:172.21.15.181=smithi181;count:2 2024-04-27T04:01:46.249 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:01:46.248+0000 7f49f7640700 1 -- 172.21.15.181:0/1575452357 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f49f00fca30 msgr2=0x7f49f00fce40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:01:46.249 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:01:46.248+0000 7f49f7640700 1 --2- 172.21.15.181:0/1575452357 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f49f00fca30 0x7f49f00fce40 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f49e8009c60 tx=0x7f49e8004d10 comp rx=0 tx=0).stop 2024-04-27T04:01:46.250 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:01:46.248+0000 7f49f7640700 1 -- 172.21.15.181:0/1575452357 shutdown_connections 2024-04-27T04:01:46.250 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:01:46.248+0000 7f49f7640700 1 --2- 172.21.15.181:0/1575452357 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f49f00fca30 0x7f49f00fce40 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:01:46.250 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:01:46.248+0000 7f49f7640700 1 -- 172.21.15.181:0/1575452357 >> 172.21.15.181:0/1575452357 conn(0x7f49f00f8170 msgr2=0x7f49f00fa5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:01:46.250 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:01:46.248+0000 7f49f7640700 1 -- 172.21.15.181:0/1575452357 shutdown_connections 2024-04-27T04:01:46.250 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:01:46.248+0000 7f49f7640700 1 -- 172.21.15.181:0/1575452357 wait complete. 2024-04-27T04:01:46.250 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:01:46.248+0000 7f49f7640700 1 Processor -- start 2024-04-27T04:01:46.250 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:01:46.248+0000 7f49f7640700 1 -- start start 2024-04-27T04:01:46.250 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:01:46.248+0000 7f49f7640700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f49f00fca30 0x7f49f010d0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:01:46.250 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:01:46.248+0000 7f49f7640700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49f010d5f0 con 0x7f49f00fca30 2024-04-27T04:01:46.250 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:01:46.249+0000 7f49f53dc700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f49f00fca30 0x7f49f010d0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:01:46.250 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:01:46.249+0000 7f49f53dc700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f49f00fca30 0x7f49f010d0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.181:32954/0 (socket says 172.21.15.181:32954) 2024-04-27T04:01:46.250 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:01:46.249+0000 7f49f53dc700 1 -- 172.21.15.181:0/1030426393 learned_addr learned my addr 172.21.15.181:0/1030426393 (peer_addr_for_me v2:172.21.15.181:0/0) 2024-04-27T04:01:46.251 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:01:46.249+0000 7f49f53dc700 1 -- 172.21.15.181:0/1030426393 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49e8009940 con 0x7f49f00fca30 2024-04-27T04:01:46.251 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:01:46.249+0000 7f49f53dc700 1 --2- 172.21.15.181:0/1030426393 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f49f00fca30 0x7f49f010d0b0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f49e8009500 tx=0x7f49e8004d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:01:46.251 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:01:46.249+0000 7f49e5ffb700 1 -- 172.21.15.181:0/1030426393 <== mon.0 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f49e8007c80 con 0x7f49f00fca30 2024-04-27T04:01:46.251 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:01:46.249+0000 7f49e5ffb700 1 -- 172.21.15.181:0/1030426393 <== mon.0 v2:172.21.15.162:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f49e801d910 con 0x7f49f00fca30 2024-04-27T04:01:46.251 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:01:46.249+0000 7f49f7640700 1 -- 172.21.15.181:0/1030426393 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f49f010d7f0 con 0x7f49f00fca30 2024-04-27T04:01:46.251 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:01:46.249+0000 7f49e5ffb700 1 -- 172.21.15.181:0/1030426393 <== mon.0 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f49e80129d0 con 0x7f49f00fca30 2024-04-27T04:01:46.251 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:01:46.249+0000 7f49f7640700 1 -- 172.21.15.181:0/1030426393 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f49f010e360 con 0x7f49f00fca30 2024-04-27T04:01:46.252 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:01:46.250+0000 7f49e5ffb700 1 -- 172.21.15.181:0/1030426393 <== mon.0 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 13) v1 ==== 43433+0+0 (secure 0 0 0) 0x7f49e8019040 con 0x7f49f00fca30 2024-04-27T04:01:46.252 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:01:46.250+0000 7f49f7640700 1 -- 172.21.15.181:0/1030426393 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f49f0102f40 con 0x7f49f00fca30 2024-04-27T04:01:46.252 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:01:46.251+0000 7f49e5ffb700 1 --2- 172.21.15.181:0/1030426393 >> [v2:172.21.15.162:6800/1482851081,v1:172.21.15.162:6801/1482851081] conn(0x7f49dc0368d0 0x7f49dc038d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:01:46.252 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:01:46.251+0000 7f49e5ffb700 1 -- 172.21.15.181:0/1030426393 <== mon.0 v2:172.21.15.162:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1151+0+0 (secure 0 0 0) 0x7f49e804a310 con 0x7f49f00fca30 2024-04-27T04:01:46.252 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:01:46.251+0000 7f49f4bdb700 1 --2- 172.21.15.181:0/1030426393 >> [v2:172.21.15.162:6800/1482851081,v1:172.21.15.162:6801/1482851081] conn(0x7f49dc0368d0 0x7f49dc038d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:01:46.252 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:01:46.251+0000 7f49f4bdb700 1 --2- 172.21.15.181:0/1030426393 >> [v2:172.21.15.162:6800/1482851081,v1:172.21.15.162:6801/1482851081] conn(0x7f49dc0368d0 0x7f49dc038d80 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f49e0009990 tx=0x7f49e0006e40 comp rx=0 tx=0).ready entity=mgr.14162 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:01:46.255 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:01:46.255+0000 7f49e5ffb700 1 -- 172.21.15.181:0/1030426393 <== mon.0 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f49e802eb40 con 0x7f49f00fca30 2024-04-27T04:01:46.518 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:01:46.518+0000 7f49f7640700 1 -- 172.21.15.181:0/1030426393 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f49f010ed70 con 0x7f49f00fca30 2024-04-27T04:01:46.519 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:01:46.518+0000 7f49e5ffb700 1 -- 172.21.15.181:0/1030426393 <== mon.0 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+743 (secure 0 0 0) 0x7f49e8017020 con 0x7f49f00fca30 2024-04-27T04:01:46.519 INFO:teuthology.orchestra.run.smithi181.stdout: 2024-04-27T04:01:46.519 INFO:teuthology.orchestra.run.smithi181.stdout:{"epoch":1,"fsid":"7f9750d6-044a-11ef-bc93-c7b262605968","modified":"2024-04-27T04:00:03.626312Z","created":"2024-04-27T04:00:03.626312Z","min_mon_release":17,"min_mon_release_name":"quincy","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy"],"optional":[]},"mons":[{"rank":0,"name":"smithi162","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:3300","nonce":0},{"type":"v1","addr":"172.21.15.162:6789","nonce":0}]},"addr":"172.21.15.162:6789/0","public_addr":"172.21.15.162:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-04-27T04:01:46.521 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:01:46.520+0000 7f49f7640700 1 -- 172.21.15.181:0/1030426393 >> [v2:172.21.15.162:6800/1482851081,v1:172.21.15.162:6801/1482851081] conn(0x7f49dc0368d0 msgr2=0x7f49dc038d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:01:46.521 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:01:46.520+0000 7f49f7640700 1 --2- 172.21.15.181:0/1030426393 >> [v2:172.21.15.162:6800/1482851081,v1:172.21.15.162:6801/1482851081] conn(0x7f49dc0368d0 0x7f49dc038d80 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f49e0009990 tx=0x7f49e0006e40 comp rx=0 tx=0).stop 2024-04-27T04:01:46.521 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:01:46.520+0000 7f49f7640700 1 -- 172.21.15.181:0/1030426393 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f49f00fca30 msgr2=0x7f49f010d0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:01:46.521 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:01:46.520+0000 7f49f7640700 1 --2- 172.21.15.181:0/1030426393 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f49f00fca30 0x7f49f010d0b0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f49e8009500 tx=0x7f49e8004d10 comp rx=0 tx=0).stop 2024-04-27T04:01:46.521 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:01:46.520+0000 7f49f7640700 1 -- 172.21.15.181:0/1030426393 shutdown_connections 2024-04-27T04:01:46.521 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:01:46.520+0000 7f49f7640700 1 --2- 172.21.15.181:0/1030426393 >> [v2:172.21.15.162:6800/1482851081,v1:172.21.15.162:6801/1482851081] conn(0x7f49dc0368d0 0x7f49dc038d80 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:01:46.521 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:01:46.520+0000 7f49f7640700 1 --2- 172.21.15.181:0/1030426393 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f49f00fca30 0x7f49f010d0b0 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:01:46.521 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:01:46.520+0000 7f49f7640700 1 -- 172.21.15.181:0/1030426393 >> 172.21.15.181:0/1030426393 conn(0x7f49f00f8170 msgr2=0x7f49f0103b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:01:46.521 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:01:46.520+0000 7f49f7640700 1 -- 172.21.15.181:0/1030426393 shutdown_connections 2024-04-27T04:01:46.521 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:01:46.520+0000 7f49f7640700 1 -- 172.21.15.181:0/1030426393 wait complete. 2024-04-27T04:01:46.522 INFO:teuthology.orchestra.run.smithi181.stderr:dumped monmap epoch 1 2024-04-27T04:01:46.888 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:01:46 smithi162 ceph-mon[26841]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-27T04:01:46.888 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:01:46 smithi162 ceph-mon[26841]: from='mgr.14162 172.21.15.162:0/1568051568' entity='mgr.smithi162.oennfn' 2024-04-27T04:01:47.888 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:01:47 smithi162 ceph-mon[26841]: from='client.? 172.21.15.181:0/1030426393' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-27T04:01:48.029 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-04-27T04:01:48.030 DEBUG:teuthology.orchestra.run.smithi181:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 7f9750d6-044a-11ef-bc93-c7b262605968 -- ceph mon dump -f json 2024-04-27T04:01:48.275 INFO:teuthology.orchestra.run.smithi181.stderr:Inferring config /etc/ceph/ceph.conf 2024-04-27T04:01:48.322 INFO:teuthology.orchestra.run.smithi181.stderr:Inferring config /etc/ceph/ceph.conf 2024-04-27T04:01:48.890 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:01:48 smithi162 ceph-mon[26841]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-27T04:01:49.829 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:01:49.828+0000 7fb06ac11700 1 -- 172.21.15.181:0/2722075193 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb0640fca30 msgr2=0x7fb0640fce40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:01:49.829 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:01:49.828+0000 7fb06ac11700 1 --2- 172.21.15.181:0/2722075193 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb0640fca30 0x7fb0640fce40 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fb0580048c0 tx=0x7fb058009e30 comp rx=0 tx=0).stop 2024-04-27T04:01:49.829 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:01:49.828+0000 7fb06ac11700 1 -- 172.21.15.181:0/2722075193 shutdown_connections 2024-04-27T04:01:49.829 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:01:49.828+0000 7fb06ac11700 1 --2- 172.21.15.181:0/2722075193 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb0640fca30 0x7fb0640fce40 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:01:49.829 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:01:49.828+0000 7fb06ac11700 1 -- 172.21.15.181:0/2722075193 >> 172.21.15.181:0/2722075193 conn(0x7fb0640f8170 msgr2=0x7fb0640fa5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:01:49.830 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:01:49.828+0000 7fb06ac11700 1 -- 172.21.15.181:0/2722075193 shutdown_connections 2024-04-27T04:01:49.830 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:01:49.829+0000 7fb06ac11700 1 -- 172.21.15.181:0/2722075193 wait complete. 2024-04-27T04:01:49.830 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:01:49.829+0000 7fb06ac11700 1 Processor -- start 2024-04-27T04:01:49.830 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:01:49.829+0000 7fb06ac11700 1 -- start start 2024-04-27T04:01:49.830 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:01:49.829+0000 7fb06ac11700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb0640fca30 0x7fb064071d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:01:49.830 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:01:49.829+0000 7fb06ac11700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb064072240 con 0x7fb0640fca30 2024-04-27T04:01:49.831 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:01:49.830+0000 7fb0689ad700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb0640fca30 0x7fb064071d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:01:49.831 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:01:49.830+0000 7fb0689ad700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb0640fca30 0x7fb064071d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.181:59122/0 (socket says 172.21.15.181:59122) 2024-04-27T04:01:49.831 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:01:49.830+0000 7fb0689ad700 1 -- 172.21.15.181:0/4201841669 learned_addr learned my addr 172.21.15.181:0/4201841669 (peer_addr_for_me v2:172.21.15.181:0/0) 2024-04-27T04:01:49.832 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:01:49.830+0000 7fb0689ad700 1 -- 172.21.15.181:0/4201841669 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb058005040 con 0x7fb0640fca30 2024-04-27T04:01:49.832 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:01:49.830+0000 7fb0689ad700 1 --2- 172.21.15.181:0/4201841669 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb0640fca30 0x7fb064071d00 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fb0580096b0 tx=0x7fb058009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:01:49.832 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:01:49.830+0000 7fb0617fa700 1 -- 172.21.15.181:0/4201841669 <== mon.0 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fb05800bed0 con 0x7fb0640fca30 2024-04-27T04:01:49.832 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:01:49.830+0000 7fb06ac11700 1 -- 172.21.15.181:0/4201841669 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb064070310 con 0x7fb0640fca30 2024-04-27T04:01:49.832 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:01:49.830+0000 7fb0617fa700 1 -- 172.21.15.181:0/4201841669 <== mon.0 v2:172.21.15.162:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fb05800b5c0 con 0x7fb0640fca30 2024-04-27T04:01:49.832 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:01:49.830+0000 7fb0617fa700 1 -- 172.21.15.181:0/4201841669 <== mon.0 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fb05801c5b0 con 0x7fb0640fca30 2024-04-27T04:01:49.832 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:01:49.830+0000 7fb06ac11700 1 -- 172.21.15.181:0/4201841669 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb064070800 con 0x7fb0640fca30 2024-04-27T04:01:49.832 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:01:49.831+0000 7fb0617fa700 1 -- 172.21.15.181:0/4201841669 <== mon.0 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 13) v1 ==== 43433+0+0 (secure 0 0 0) 0x7fb05801c820 con 0x7fb0640fca30 2024-04-27T04:01:49.832 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:01:49.831+0000 7fb06ac11700 1 -- 172.21.15.181:0/4201841669 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb06405f5c0 con 0x7fb0640fca30 2024-04-27T04:01:49.832 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:01:49.831+0000 7fb0617fa700 1 --2- 172.21.15.181:0/4201841669 >> [v2:172.21.15.162:6800/1482851081,v1:172.21.15.162:6801/1482851081] conn(0x7fb054036880 0x7fb054038d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:01:49.832 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:01:49.831+0000 7fb0617fa700 1 -- 172.21.15.181:0/4201841669 <== mon.0 v2:172.21.15.162:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1151+0+0 (secure 0 0 0) 0x7fb058049c90 con 0x7fb0640fca30 2024-04-27T04:01:49.832 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:01:49.832+0000 7fb063fff700 1 --2- 172.21.15.181:0/4201841669 >> [v2:172.21.15.162:6800/1482851081,v1:172.21.15.162:6801/1482851081] conn(0x7fb054036880 0x7fb054038d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:01:49.833 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:01:49.832+0000 7fb063fff700 1 --2- 172.21.15.181:0/4201841669 >> [v2:172.21.15.162:6800/1482851081,v1:172.21.15.162:6801/1482851081] conn(0x7fb054036880 0x7fb054038d30 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fb050009990 tx=0x7fb050006e40 comp rx=0 tx=0).ready entity=mgr.14162 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:01:49.836 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:01:49.836+0000 7fb0617fa700 1 -- 172.21.15.181:0/4201841669 <== mon.0 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7fb058023300 con 0x7fb0640fca30 2024-04-27T04:01:50.096 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:01:50.095+0000 7fb06ac11700 1 -- 172.21.15.181:0/4201841669 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7fb064106410 con 0x7fb0640fca30 2024-04-27T04:01:50.096 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:01:50.096+0000 7fb0617fa700 1 -- 172.21.15.181:0/4201841669 <== mon.0 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+743 (secure 0 0 0) 0x7fb058011b10 con 0x7fb0640fca30 2024-04-27T04:01:50.096 INFO:teuthology.orchestra.run.smithi181.stdout: 2024-04-27T04:01:50.097 INFO:teuthology.orchestra.run.smithi181.stdout:{"epoch":1,"fsid":"7f9750d6-044a-11ef-bc93-c7b262605968","modified":"2024-04-27T04:00:03.626312Z","created":"2024-04-27T04:00:03.626312Z","min_mon_release":17,"min_mon_release_name":"quincy","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy"],"optional":[]},"mons":[{"rank":0,"name":"smithi162","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:3300","nonce":0},{"type":"v1","addr":"172.21.15.162:6789","nonce":0}]},"addr":"172.21.15.162:6789/0","public_addr":"172.21.15.162:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-04-27T04:01:50.098 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:01:50.098+0000 7fb06ac11700 1 -- 172.21.15.181:0/4201841669 >> [v2:172.21.15.162:6800/1482851081,v1:172.21.15.162:6801/1482851081] conn(0x7fb054036880 msgr2=0x7fb054038d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:01:50.098 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:01:50.098+0000 7fb06ac11700 1 --2- 172.21.15.181:0/4201841669 >> [v2:172.21.15.162:6800/1482851081,v1:172.21.15.162:6801/1482851081] conn(0x7fb054036880 0x7fb054038d30 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fb050009990 tx=0x7fb050006e40 comp rx=0 tx=0).stop 2024-04-27T04:01:50.099 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:01:50.098+0000 7fb06ac11700 1 -- 172.21.15.181:0/4201841669 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb0640fca30 msgr2=0x7fb064071d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:01:50.099 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:01:50.098+0000 7fb06ac11700 1 --2- 172.21.15.181:0/4201841669 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb0640fca30 0x7fb064071d00 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fb0580096b0 tx=0x7fb058009e30 comp rx=0 tx=0).stop 2024-04-27T04:01:50.099 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:01:50.098+0000 7fb06ac11700 1 -- 172.21.15.181:0/4201841669 shutdown_connections 2024-04-27T04:01:50.099 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:01:50.098+0000 7fb06ac11700 1 --2- 172.21.15.181:0/4201841669 >> [v2:172.21.15.162:6800/1482851081,v1:172.21.15.162:6801/1482851081] conn(0x7fb054036880 0x7fb054038d30 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:01:50.099 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:01:50.098+0000 7fb06ac11700 1 --2- 172.21.15.181:0/4201841669 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb0640fca30 0x7fb064071d00 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:01:50.099 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:01:50.098+0000 7fb06ac11700 1 -- 172.21.15.181:0/4201841669 >> 172.21.15.181:0/4201841669 conn(0x7fb0640f8170 msgr2=0x7fb064103b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:01:50.099 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:01:50.098+0000 7fb06ac11700 1 -- 172.21.15.181:0/4201841669 shutdown_connections 2024-04-27T04:01:50.099 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:01:50.098+0000 7fb06ac11700 1 -- 172.21.15.181:0/4201841669 wait complete. 2024-04-27T04:01:50.100 INFO:teuthology.orchestra.run.smithi181.stderr:dumped monmap epoch 1 2024-04-27T04:01:50.888 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:01:50 smithi162 ceph-mon[26841]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-27T04:01:50.889 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:01:50 smithi162 ceph-mon[26841]: from='client.? 172.21.15.181:0/4201841669' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-27T04:01:51.581 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-04-27T04:01:51.582 DEBUG:teuthology.orchestra.run.smithi181:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 7f9750d6-044a-11ef-bc93-c7b262605968 -- ceph mon dump -f json 2024-04-27T04:01:51.837 INFO:teuthology.orchestra.run.smithi181.stderr:Inferring config /etc/ceph/ceph.conf 2024-04-27T04:01:51.884 INFO:teuthology.orchestra.run.smithi181.stderr:Inferring config /etc/ceph/ceph.conf 2024-04-27T04:01:52.638 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:01:52 smithi162 ceph-mon[26841]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-27T04:01:53.328 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:01:53.326+0000 7f0edb934700 1 -- 172.21.15.181:0/2876287367 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0ed40fca20 msgr2=0x7f0ed40fce30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:01:53.328 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:01:53.326+0000 7f0edb934700 1 --2- 172.21.15.181:0/2876287367 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0ed40fca20 0x7f0ed40fce30 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f0ec40048c0 tx=0x7f0ec4009e30 comp rx=0 tx=0).stop 2024-04-27T04:01:53.328 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:01:53.326+0000 7f0edb934700 1 -- 172.21.15.181:0/2876287367 shutdown_connections 2024-04-27T04:01:53.328 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:01:53.326+0000 7f0edb934700 1 --2- 172.21.15.181:0/2876287367 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0ed40fca20 0x7f0ed40fce30 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:01:53.328 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:01:53.326+0000 7f0edb934700 1 -- 172.21.15.181:0/2876287367 >> 172.21.15.181:0/2876287367 conn(0x7f0ed40f81a0 msgr2=0x7f0ed40fa5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:01:53.328 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:01:53.326+0000 7f0edb934700 1 -- 172.21.15.181:0/2876287367 shutdown_connections 2024-04-27T04:01:53.328 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:01:53.326+0000 7f0edb934700 1 -- 172.21.15.181:0/2876287367 wait complete. 2024-04-27T04:01:53.328 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:01:53.327+0000 7f0edb934700 1 Processor -- start 2024-04-27T04:01:53.329 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:01:53.327+0000 7f0edb934700 1 -- start start 2024-04-27T04:01:53.329 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:01:53.327+0000 7f0edb934700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0ed40fca20 0x7f0ed410d0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:01:53.329 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:01:53.327+0000 7f0edb934700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ed410d5e0 con 0x7f0ed40fca20 2024-04-27T04:01:53.329 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:01:53.327+0000 7f0ed96d0700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0ed40fca20 0x7f0ed410d0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:01:53.329 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:01:53.327+0000 7f0ed96d0700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0ed40fca20 0x7f0ed410d0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.181:59140/0 (socket says 172.21.15.181:59140) 2024-04-27T04:01:53.329 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:01:53.327+0000 7f0ed96d0700 1 -- 172.21.15.181:0/860545698 learned_addr learned my addr 172.21.15.181:0/860545698 (peer_addr_for_me v2:172.21.15.181:0/0) 2024-04-27T04:01:53.329 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:01:53.327+0000 7f0ed96d0700 1 -- 172.21.15.181:0/860545698 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ec4005040 con 0x7f0ed40fca20 2024-04-27T04:01:53.329 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:01:53.328+0000 7f0ed96d0700 1 --2- 172.21.15.181:0/860545698 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0ed40fca20 0x7f0ed410d0a0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f0ec40096b0 tx=0x7f0ec4009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:01:53.329 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:01:53.328+0000 7f0ec9ffb700 1 -- 172.21.15.181:0/860545698 <== mon.0 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f0ec400bed0 con 0x7f0ed40fca20 2024-04-27T04:01:53.329 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:01:53.328+0000 7f0ec9ffb700 1 -- 172.21.15.181:0/860545698 <== mon.0 v2:172.21.15.162:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f0ec400b5c0 con 0x7f0ed40fca20 2024-04-27T04:01:53.329 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:01:53.328+0000 7f0edb934700 1 -- 172.21.15.181:0/860545698 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0ed410d7e0 con 0x7f0ed40fca20 2024-04-27T04:01:53.329 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:01:53.328+0000 7f0ec9ffb700 1 -- 172.21.15.181:0/860545698 <== mon.0 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f0ec401c5b0 con 0x7f0ed40fca20 2024-04-27T04:01:53.329 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:01:53.328+0000 7f0edb934700 1 -- 172.21.15.181:0/860545698 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0ed410e350 con 0x7f0ed40fca20 2024-04-27T04:01:53.331 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:01:53.329+0000 7f0ec9ffb700 1 -- 172.21.15.181:0/860545698 <== mon.0 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 13) v1 ==== 43433+0+0 (secure 0 0 0) 0x7f0ec401c820 con 0x7f0ed40fca20 2024-04-27T04:01:53.331 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:01:53.329+0000 7f0edb934700 1 -- 172.21.15.181:0/860545698 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0ed4102f30 con 0x7f0ed40fca20 2024-04-27T04:01:53.331 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:01:53.329+0000 7f0ec9ffb700 1 --2- 172.21.15.181:0/860545698 >> [v2:172.21.15.162:6800/1482851081,v1:172.21.15.162:6801/1482851081] conn(0x7f0ec0036880 0x7f0ec0038d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:01:53.331 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:01:53.329+0000 7f0ec9ffb700 1 -- 172.21.15.181:0/860545698 <== mon.0 v2:172.21.15.162:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1151+0+0 (secure 0 0 0) 0x7f0ec4049b50 con 0x7f0ed40fca20 2024-04-27T04:01:53.331 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:01:53.329+0000 7f0ed8ecf700 1 --2- 172.21.15.181:0/860545698 >> [v2:172.21.15.162:6800/1482851081,v1:172.21.15.162:6801/1482851081] conn(0x7f0ec0036880 0x7f0ec0038d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:01:53.331 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:01:53.329+0000 7f0ed8ecf700 1 --2- 172.21.15.181:0/860545698 >> [v2:172.21.15.162:6800/1482851081,v1:172.21.15.162:6801/1482851081] conn(0x7f0ec0036880 0x7f0ec0038d30 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f0ed0009990 tx=0x7f0ed0006e40 comp rx=0 tx=0).ready entity=mgr.14162 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:01:53.334 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:01:53.334+0000 7f0ec9ffb700 1 -- 172.21.15.181:0/860545698 <== mon.0 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f0ec4023300 con 0x7f0ed40fca20 2024-04-27T04:01:53.595 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:01:53.595+0000 7f0edb934700 1 -- 172.21.15.181:0/860545698 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f0ed410ed60 con 0x7f0ed40fca20 2024-04-27T04:01:53.595 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:01:53.595+0000 7f0ec9ffb700 1 -- 172.21.15.181:0/860545698 <== mon.0 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+743 (secure 0 0 0) 0x7f0ec4011b60 con 0x7f0ed40fca20 2024-04-27T04:01:53.596 INFO:teuthology.orchestra.run.smithi181.stdout: 2024-04-27T04:01:53.596 INFO:teuthology.orchestra.run.smithi181.stdout:{"epoch":1,"fsid":"7f9750d6-044a-11ef-bc93-c7b262605968","modified":"2024-04-27T04:00:03.626312Z","created":"2024-04-27T04:00:03.626312Z","min_mon_release":17,"min_mon_release_name":"quincy","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy"],"optional":[]},"mons":[{"rank":0,"name":"smithi162","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:3300","nonce":0},{"type":"v1","addr":"172.21.15.162:6789","nonce":0}]},"addr":"172.21.15.162:6789/0","public_addr":"172.21.15.162:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-04-27T04:01:53.597 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:01:53.597+0000 7f0edb934700 1 -- 172.21.15.181:0/860545698 >> [v2:172.21.15.162:6800/1482851081,v1:172.21.15.162:6801/1482851081] conn(0x7f0ec0036880 msgr2=0x7f0ec0038d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:01:53.597 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:01:53.597+0000 7f0edb934700 1 --2- 172.21.15.181:0/860545698 >> [v2:172.21.15.162:6800/1482851081,v1:172.21.15.162:6801/1482851081] conn(0x7f0ec0036880 0x7f0ec0038d30 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f0ed0009990 tx=0x7f0ed0006e40 comp rx=0 tx=0).stop 2024-04-27T04:01:53.597 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:01:53.597+0000 7f0edb934700 1 -- 172.21.15.181:0/860545698 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0ed40fca20 msgr2=0x7f0ed410d0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:01:53.598 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:01:53.597+0000 7f0edb934700 1 --2- 172.21.15.181:0/860545698 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0ed40fca20 0x7f0ed410d0a0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f0ec40096b0 tx=0x7f0ec4009e30 comp rx=0 tx=0).stop 2024-04-27T04:01:53.598 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:01:53.597+0000 7f0edb934700 1 -- 172.21.15.181:0/860545698 shutdown_connections 2024-04-27T04:01:53.598 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:01:53.597+0000 7f0edb934700 1 --2- 172.21.15.181:0/860545698 >> [v2:172.21.15.162:6800/1482851081,v1:172.21.15.162:6801/1482851081] conn(0x7f0ec0036880 0x7f0ec0038d30 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:01:53.598 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:01:53.597+0000 7f0edb934700 1 --2- 172.21.15.181:0/860545698 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0ed40fca20 0x7f0ed410d0a0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:01:53.598 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:01:53.597+0000 7f0edb934700 1 -- 172.21.15.181:0/860545698 >> 172.21.15.181:0/860545698 conn(0x7f0ed40f81a0 msgr2=0x7f0ed4103b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:01:53.598 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:01:53.597+0000 7f0edb934700 1 -- 172.21.15.181:0/860545698 shutdown_connections 2024-04-27T04:01:53.598 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:01:53.597+0000 7f0edb934700 1 -- 172.21.15.181:0/860545698 wait complete. 2024-04-27T04:01:53.599 INFO:teuthology.orchestra.run.smithi181.stderr:dumped monmap epoch 1 2024-04-27T04:01:54.888 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:01:54 smithi162 ceph-mon[26841]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-27T04:01:54.889 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:01:54 smithi162 ceph-mon[26841]: from='client.? 172.21.15.181:0/860545698' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-27T04:01:55.208 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-04-27T04:01:55.209 DEBUG:teuthology.orchestra.run.smithi181:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 7f9750d6-044a-11ef-bc93-c7b262605968 -- ceph mon dump -f json 2024-04-27T04:01:55.464 INFO:teuthology.orchestra.run.smithi181.stderr:Inferring config /etc/ceph/ceph.conf 2024-04-27T04:01:55.512 INFO:teuthology.orchestra.run.smithi181.stderr:Inferring config /etc/ceph/ceph.conf 2024-04-27T04:01:56.791 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:01:56 smithi162 ceph-mon[26841]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-27T04:01:56.831 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:01:56.830+0000 7f9361e63700 1 -- 172.21.15.181:0/3859485367 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f935c0fb940 msgr2=0x7f935c0fbd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:01:56.832 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:01:56.830+0000 7f9361e63700 1 --2- 172.21.15.181:0/3859485367 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f935c0fb940 0x7f935c0fbd50 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f934c0048c0 tx=0x7f934c009e30 comp rx=0 tx=0).stop 2024-04-27T04:01:56.832 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:01:56.831+0000 7f9361e63700 1 -- 172.21.15.181:0/3859485367 shutdown_connections 2024-04-27T04:01:56.832 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:01:56.831+0000 7f9361e63700 1 --2- 172.21.15.181:0/3859485367 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f935c0fb940 0x7f935c0fbd50 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:01:56.832 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:01:56.831+0000 7f9361e63700 1 -- 172.21.15.181:0/3859485367 >> 172.21.15.181:0/3859485367 conn(0x7f935c0f70a0 msgr2=0x7f935c0f94d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:01:56.832 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:01:56.831+0000 7f9361e63700 1 -- 172.21.15.181:0/3859485367 shutdown_connections 2024-04-27T04:01:56.832 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:01:56.831+0000 7f9361e63700 1 -- 172.21.15.181:0/3859485367 wait complete. 2024-04-27T04:01:56.832 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:01:56.831+0000 7f9361e63700 1 Processor -- start 2024-04-27T04:01:56.832 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:01:56.831+0000 7f9361e63700 1 -- start start 2024-04-27T04:01:56.832 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:01:56.831+0000 7f9361e63700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f935c0fb940 0x7f935c066820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:01:56.832 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:01:56.831+0000 7f9361e63700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f935c10fdb0 con 0x7f935c0fb940 2024-04-27T04:01:56.832 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:01:56.832+0000 7f935b7fe700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f935c0fb940 0x7f935c066820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:01:56.832 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:01:56.832+0000 7f935b7fe700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f935c0fb940 0x7f935c066820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.181:59148/0 (socket says 172.21.15.181:59148) 2024-04-27T04:01:56.832 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:01:56.832+0000 7f935b7fe700 1 -- 172.21.15.181:0/1577984498 learned_addr learned my addr 172.21.15.181:0/1577984498 (peer_addr_for_me v2:172.21.15.181:0/0) 2024-04-27T04:01:56.833 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:01:56.832+0000 7f935b7fe700 1 -- 172.21.15.181:0/1577984498 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f934c005040 con 0x7f935c0fb940 2024-04-27T04:01:56.834 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:01:56.832+0000 7f935b7fe700 1 --2- 172.21.15.181:0/1577984498 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f935c0fb940 0x7f935c066820 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f934c0096b0 tx=0x7f934c009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:01:56.834 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:01:56.832+0000 7f9343fff700 1 -- 172.21.15.181:0/1577984498 <== mon.0 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f934c00bed0 con 0x7f935c0fb940 2024-04-27T04:01:56.834 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:01:56.832+0000 7f9343fff700 1 -- 172.21.15.181:0/1577984498 <== mon.0 v2:172.21.15.162:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f934c00b5c0 con 0x7f935c0fb940 2024-04-27T04:01:56.834 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:01:56.832+0000 7f9361e63700 1 -- 172.21.15.181:0/1577984498 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f935c10fef0 con 0x7f935c0fb940 2024-04-27T04:01:56.834 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:01:56.832+0000 7f9343fff700 1 -- 172.21.15.181:0/1577984498 <== mon.0 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f934c01c5b0 con 0x7f935c0fb940 2024-04-27T04:01:56.834 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:01:56.832+0000 7f9361e63700 1 -- 172.21.15.181:0/1577984498 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f935c1103e0 con 0x7f935c0fb940 2024-04-27T04:01:56.834 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:01:56.833+0000 7f9343fff700 1 -- 172.21.15.181:0/1577984498 <== mon.0 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 13) v1 ==== 43433+0+0 (secure 0 0 0) 0x7f934c01c820 con 0x7f935c0fb940 2024-04-27T04:01:56.834 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:01:56.833+0000 7f9361e63700 1 -- 172.21.15.181:0/1577984498 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f935c105900 con 0x7f935c0fb940 2024-04-27T04:01:56.835 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:01:56.833+0000 7f9343fff700 1 --2- 172.21.15.181:0/1577984498 >> [v2:172.21.15.162:6800/1482851081,v1:172.21.15.162:6801/1482851081] conn(0x7f9344036880 0x7f9344038d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:01:56.835 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:01:56.833+0000 7f9343fff700 1 -- 172.21.15.181:0/1577984498 <== mon.0 v2:172.21.15.162:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1151+0+0 (secure 0 0 0) 0x7f934c049a30 con 0x7f935c0fb940 2024-04-27T04:01:56.835 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:01:56.833+0000 7f935affd700 1 --2- 172.21.15.181:0/1577984498 >> [v2:172.21.15.162:6800/1482851081,v1:172.21.15.162:6801/1482851081] conn(0x7f9344036880 0x7f9344038d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:01:56.835 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:01:56.834+0000 7f935affd700 1 --2- 172.21.15.181:0/1577984498 >> [v2:172.21.15.162:6800/1482851081,v1:172.21.15.162:6801/1482851081] conn(0x7f9344036880 0x7f9344038d30 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f9350009990 tx=0x7f9350006e40 comp rx=0 tx=0).ready entity=mgr.14162 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:01:56.838 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:01:56.838+0000 7f9343fff700 1 -- 172.21.15.181:0/1577984498 <== mon.0 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f934c023300 con 0x7f935c0fb940 2024-04-27T04:01:57.101 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:01:57.101+0000 7f9361e63700 1 -- 172.21.15.181:0/1577984498 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f935c05f5c0 con 0x7f935c0fb940 2024-04-27T04:01:57.101 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:01:57.101+0000 7f9343fff700 1 -- 172.21.15.181:0/1577984498 <== mon.0 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+743 (secure 0 0 0) 0x7f934c011b60 con 0x7f935c0fb940 2024-04-27T04:01:57.102 INFO:teuthology.orchestra.run.smithi181.stdout: 2024-04-27T04:01:57.102 INFO:teuthology.orchestra.run.smithi181.stdout:{"epoch":1,"fsid":"7f9750d6-044a-11ef-bc93-c7b262605968","modified":"2024-04-27T04:00:03.626312Z","created":"2024-04-27T04:00:03.626312Z","min_mon_release":17,"min_mon_release_name":"quincy","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy"],"optional":[]},"mons":[{"rank":0,"name":"smithi162","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:3300","nonce":0},{"type":"v1","addr":"172.21.15.162:6789","nonce":0}]},"addr":"172.21.15.162:6789/0","public_addr":"172.21.15.162:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-04-27T04:01:57.104 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:01:57.103+0000 7f9361e63700 1 -- 172.21.15.181:0/1577984498 >> [v2:172.21.15.162:6800/1482851081,v1:172.21.15.162:6801/1482851081] conn(0x7f9344036880 msgr2=0x7f9344038d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:01:57.104 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:01:57.103+0000 7f9361e63700 1 --2- 172.21.15.181:0/1577984498 >> [v2:172.21.15.162:6800/1482851081,v1:172.21.15.162:6801/1482851081] conn(0x7f9344036880 0x7f9344038d30 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f9350009990 tx=0x7f9350006e40 comp rx=0 tx=0).stop 2024-04-27T04:01:57.104 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:01:57.103+0000 7f9361e63700 1 -- 172.21.15.181:0/1577984498 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f935c0fb940 msgr2=0x7f935c066820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:01:57.104 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:01:57.103+0000 7f9361e63700 1 --2- 172.21.15.181:0/1577984498 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f935c0fb940 0x7f935c066820 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f934c0096b0 tx=0x7f934c009e30 comp rx=0 tx=0).stop 2024-04-27T04:01:57.104 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:01:57.103+0000 7f9361e63700 1 -- 172.21.15.181:0/1577984498 shutdown_connections 2024-04-27T04:01:57.104 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:01:57.103+0000 7f9361e63700 1 --2- 172.21.15.181:0/1577984498 >> [v2:172.21.15.162:6800/1482851081,v1:172.21.15.162:6801/1482851081] conn(0x7f9344036880 0x7f9344038d30 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:01:57.104 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:01:57.103+0000 7f9361e63700 1 --2- 172.21.15.181:0/1577984498 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f935c0fb940 0x7f935c066820 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:01:57.104 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:01:57.103+0000 7f9361e63700 1 -- 172.21.15.181:0/1577984498 >> 172.21.15.181:0/1577984498 conn(0x7f935c0f70a0 msgr2=0x7f935c102a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:01:57.104 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:01:57.104+0000 7f9361e63700 1 -- 172.21.15.181:0/1577984498 shutdown_connections 2024-04-27T04:01:57.104 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:01:57.104+0000 7f9361e63700 1 -- 172.21.15.181:0/1577984498 wait complete. 2024-04-27T04:01:57.105 INFO:teuthology.orchestra.run.smithi181.stderr:dumped monmap epoch 1 2024-04-27T04:01:57.764 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:01:57 smithi162 ceph-mon[26841]: from='client.? 172.21.15.181:0/1577984498' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-27T04:01:58.587 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-04-27T04:01:58.587 DEBUG:teuthology.orchestra.run.smithi181:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 7f9750d6-044a-11ef-bc93-c7b262605968 -- ceph mon dump -f json 2024-04-27T04:01:58.638 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:01:58 smithi162 ceph-mon[26841]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-27T04:01:58.833 INFO:teuthology.orchestra.run.smithi181.stderr:Inferring config /etc/ceph/ceph.conf 2024-04-27T04:01:58.880 INFO:teuthology.orchestra.run.smithi181.stderr:Inferring config /etc/ceph/ceph.conf 2024-04-27T04:02:00.267 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:02:00.264+0000 7fef03288700 1 -- 172.21.15.181:0/2911728063 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7feefc0fb940 msgr2=0x7feefc0fbd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:02:00.267 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:02:00.264+0000 7fef03288700 1 --2- 172.21.15.181:0/2911728063 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7feefc0fb940 0x7feefc0fbd50 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7feef40048c0 tx=0x7feef4009e30 comp rx=0 tx=0).stop 2024-04-27T04:02:00.267 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:02:00.264+0000 7fef03288700 1 -- 172.21.15.181:0/2911728063 shutdown_connections 2024-04-27T04:02:00.267 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:02:00.264+0000 7fef03288700 1 --2- 172.21.15.181:0/2911728063 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7feefc0fb940 0x7feefc0fbd50 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:02:00.267 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:02:00.264+0000 7fef03288700 1 -- 172.21.15.181:0/2911728063 >> 172.21.15.181:0/2911728063 conn(0x7feefc0f70a0 msgr2=0x7feefc0f94d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:02:00.267 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:02:00.265+0000 7fef03288700 1 -- 172.21.15.181:0/2911728063 shutdown_connections 2024-04-27T04:02:00.267 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:02:00.265+0000 7fef03288700 1 -- 172.21.15.181:0/2911728063 wait complete. 2024-04-27T04:02:00.267 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:02:00.265+0000 7fef03288700 1 Processor -- start 2024-04-27T04:02:00.267 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:02:00.265+0000 7fef03288700 1 -- start start 2024-04-27T04:02:00.267 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:02:00.265+0000 7fef03288700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7feefc0fb940 0x7feefc10bf90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:02:00.268 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:02:00.265+0000 7fef03288700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feefc10c4d0 con 0x7feefc0fb940 2024-04-27T04:02:00.268 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:02:00.266+0000 7fef01024700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7feefc0fb940 0x7feefc10bf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:02:00.268 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:02:00.266+0000 7fef01024700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7feefc0fb940 0x7feefc10bf90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.181:43984/0 (socket says 172.21.15.181:43984) 2024-04-27T04:02:00.268 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:02:00.266+0000 7fef01024700 1 -- 172.21.15.181:0/2405549221 learned_addr learned my addr 172.21.15.181:0/2405549221 (peer_addr_for_me v2:172.21.15.181:0/0) 2024-04-27T04:02:00.268 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:02:00.266+0000 7fef01024700 1 -- 172.21.15.181:0/2405549221 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feef4005040 con 0x7feefc0fb940 2024-04-27T04:02:00.268 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:02:00.266+0000 7fef01024700 1 --2- 172.21.15.181:0/2405549221 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7feefc0fb940 0x7feefc10bf90 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7feef4009680 tx=0x7feef4009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:02:00.269 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:02:00.266+0000 7feef1ffb700 1 -- 172.21.15.181:0/2405549221 <== mon.0 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7feef400b5c0 con 0x7feefc0fb940 2024-04-27T04:02:00.269 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:02:00.266+0000 7feef1ffb700 1 -- 172.21.15.181:0/2405549221 <== mon.0 v2:172.21.15.162:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7feef400be70 con 0x7feefc0fb940 2024-04-27T04:02:00.269 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:02:00.266+0000 7fef03288700 1 -- 172.21.15.181:0/2405549221 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feefc10c6d0 con 0x7feefc0fb940 2024-04-27T04:02:00.269 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:02:00.266+0000 7feef1ffb700 1 -- 172.21.15.181:0/2405549221 <== mon.0 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7feef40135a0 con 0x7feefc0fb940 2024-04-27T04:02:00.269 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:02:00.266+0000 7fef03288700 1 -- 172.21.15.181:0/2405549221 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feefc10d240 con 0x7feefc0fb940 2024-04-27T04:02:00.269 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:02:00.267+0000 7feef1ffb700 1 -- 172.21.15.181:0/2405549221 <== mon.0 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 13) v1 ==== 43433+0+0 (secure 0 0 0) 0x7feef401c430 con 0x7feefc0fb940 2024-04-27T04:02:00.270 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:02:00.268+0000 7feef1ffb700 1 --2- 172.21.15.181:0/2405549221 >> [v2:172.21.15.162:6800/1482851081,v1:172.21.15.162:6801/1482851081] conn(0x7feee8036880 0x7feee8038d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:02:00.270 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:02:00.267+0000 7fef03288700 1 -- 172.21.15.181:0/2405549221 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feefc101e50 con 0x7feefc0fb940 2024-04-27T04:02:00.270 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:02:00.268+0000 7feef1ffb700 1 -- 172.21.15.181:0/2405549221 <== mon.0 v2:172.21.15.162:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1151+0+0 (secure 0 0 0) 0x7feef4049460 con 0x7feefc0fb940 2024-04-27T04:02:00.270 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:02:00.268+0000 7fef00823700 1 --2- 172.21.15.181:0/2405549221 >> [v2:172.21.15.162:6800/1482851081,v1:172.21.15.162:6801/1482851081] conn(0x7feee8036880 0x7feee8038d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:02:00.270 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:02:00.268+0000 7fef00823700 1 --2- 172.21.15.181:0/2405549221 >> [v2:172.21.15.162:6800/1482851081,v1:172.21.15.162:6801/1482851081] conn(0x7feee8036880 0x7feee8038d30 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7feeec009990 tx=0x7feeec006e40 comp rx=0 tx=0).ready entity=mgr.14162 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:02:00.273 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:02:00.272+0000 7feef1ffb700 1 -- 172.21.15.181:0/2405549221 <== mon.0 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7feef401c7d0 con 0x7feefc0fb940 2024-04-27T04:02:00.379 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:02:00 smithi162 ceph-mon[26841]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-27T04:02:00.380 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:02:00 smithi162 ceph-mon[26841]: from='mgr.14162 172.21.15.162:0/1568051568' entity='mgr.smithi162.oennfn' 2024-04-27T04:02:00.380 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:02:00 smithi162 ceph-mon[26841]: from='mgr.14162 172.21.15.162:0/1568051568' entity='mgr.smithi162.oennfn' 2024-04-27T04:02:00.380 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:02:00 smithi162 ceph-mon[26841]: from='mgr.14162 172.21.15.162:0/1568051568' entity='mgr.smithi162.oennfn' 2024-04-27T04:02:00.380 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:02:00 smithi162 ceph-mon[26841]: from='mgr.14162 172.21.15.162:0/1568051568' entity='mgr.smithi162.oennfn' 2024-04-27T04:02:00.380 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:02:00 smithi162 ceph-mon[26841]: from='mgr.14162 172.21.15.162:0/1568051568' entity='mgr.smithi162.oennfn' 2024-04-27T04:02:00.380 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:02:00 smithi162 ceph-mon[26841]: from='mgr.14162 172.21.15.162:0/1568051568' entity='mgr.smithi162.oennfn' 2024-04-27T04:02:00.380 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:02:00 smithi162 ceph-mon[26841]: Deploying daemon prometheus.smithi162 on smithi162 2024-04-27T04:02:00.532 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:02:00.531+0000 7fef03288700 1 -- 172.21.15.181:0/2405549221 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7feefc10dc50 con 0x7feefc0fb940 2024-04-27T04:02:00.533 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:02:00.532+0000 7feef1ffb700 1 -- 172.21.15.181:0/2405549221 <== mon.0 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+743 (secure 0 0 0) 0x7feef4021020 con 0x7feefc0fb940 2024-04-27T04:02:00.533 INFO:teuthology.orchestra.run.smithi181.stdout: 2024-04-27T04:02:00.533 INFO:teuthology.orchestra.run.smithi181.stdout:{"epoch":1,"fsid":"7f9750d6-044a-11ef-bc93-c7b262605968","modified":"2024-04-27T04:00:03.626312Z","created":"2024-04-27T04:00:03.626312Z","min_mon_release":17,"min_mon_release_name":"quincy","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy"],"optional":[]},"mons":[{"rank":0,"name":"smithi162","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:3300","nonce":0},{"type":"v1","addr":"172.21.15.162:6789","nonce":0}]},"addr":"172.21.15.162:6789/0","public_addr":"172.21.15.162:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-04-27T04:02:00.535 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:02:00.534+0000 7fef03288700 1 -- 172.21.15.181:0/2405549221 >> [v2:172.21.15.162:6800/1482851081,v1:172.21.15.162:6801/1482851081] conn(0x7feee8036880 msgr2=0x7feee8038d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:02:00.535 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:02:00.534+0000 7fef03288700 1 --2- 172.21.15.181:0/2405549221 >> [v2:172.21.15.162:6800/1482851081,v1:172.21.15.162:6801/1482851081] conn(0x7feee8036880 0x7feee8038d30 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7feeec009990 tx=0x7feeec006e40 comp rx=0 tx=0).stop 2024-04-27T04:02:00.535 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:02:00.534+0000 7fef03288700 1 -- 172.21.15.181:0/2405549221 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7feefc0fb940 msgr2=0x7feefc10bf90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:02:00.535 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:02:00.534+0000 7fef03288700 1 --2- 172.21.15.181:0/2405549221 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7feefc0fb940 0x7feefc10bf90 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7feef4009680 tx=0x7feef4009e30 comp rx=0 tx=0).stop 2024-04-27T04:02:00.536 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:02:00.534+0000 7fef03288700 1 -- 172.21.15.181:0/2405549221 shutdown_connections 2024-04-27T04:02:00.536 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:02:00.534+0000 7fef03288700 1 --2- 172.21.15.181:0/2405549221 >> [v2:172.21.15.162:6800/1482851081,v1:172.21.15.162:6801/1482851081] conn(0x7feee8036880 0x7feee8038d30 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:02:00.536 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:02:00.534+0000 7fef03288700 1 --2- 172.21.15.181:0/2405549221 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7feefc0fb940 0x7feefc10bf90 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:02:00.536 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:02:00.534+0000 7fef03288700 1 -- 172.21.15.181:0/2405549221 >> 172.21.15.181:0/2405549221 conn(0x7feefc0f70a0 msgr2=0x7feefc102a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:02:00.536 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:02:00.534+0000 7fef03288700 1 -- 172.21.15.181:0/2405549221 shutdown_connections 2024-04-27T04:02:00.536 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:02:00.534+0000 7fef03288700 1 -- 172.21.15.181:0/2405549221 wait complete. 2024-04-27T04:02:00.536 INFO:teuthology.orchestra.run.smithi181.stderr:dumped monmap epoch 1 2024-04-27T04:02:01.638 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:02:01 smithi162 ceph-mon[26841]: from='client.? 172.21.15.181:0/2405549221' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-27T04:02:01.638 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:02:01 smithi162 ceph-mon[26841]: from='mgr.14162 172.21.15.162:0/1568051568' entity='mgr.smithi162.oennfn' 2024-04-27T04:02:02.124 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-04-27T04:02:02.124 DEBUG:teuthology.orchestra.run.smithi181:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 7f9750d6-044a-11ef-bc93-c7b262605968 -- ceph mon dump -f json 2024-04-27T04:02:02.394 INFO:teuthology.orchestra.run.smithi181.stderr:Inferring config /etc/ceph/ceph.conf 2024-04-27T04:02:02.443 INFO:teuthology.orchestra.run.smithi181.stderr:Inferring config /etc/ceph/ceph.conf 2024-04-27T04:02:02.638 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:02:02 smithi162 ceph-mon[26841]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-27T04:02:03.851 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:02:03.846+0000 7f6fff45b700 1 -- 172.21.15.181:0/3686789954 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6ff80fca30 msgr2=0x7f6ff80fce40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:02:03.851 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:02:03.846+0000 7f6fff45b700 1 --2- 172.21.15.181:0/3686789954 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6ff80fca30 0x7f6ff80fce40 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f6ff00048c0 tx=0x7f6ff0009e30 comp rx=0 tx=0).stop 2024-04-27T04:02:03.851 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:02:03.847+0000 7f6fff45b700 1 -- 172.21.15.181:0/3686789954 shutdown_connections 2024-04-27T04:02:03.851 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:02:03.847+0000 7f6fff45b700 1 --2- 172.21.15.181:0/3686789954 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6ff80fca30 0x7f6ff80fce40 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:02:03.852 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:02:03.847+0000 7f6fff45b700 1 -- 172.21.15.181:0/3686789954 >> 172.21.15.181:0/3686789954 conn(0x7f6ff80f8170 msgr2=0x7f6ff80fa5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:02:03.852 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:02:03.847+0000 7f6fff45b700 1 -- 172.21.15.181:0/3686789954 shutdown_connections 2024-04-27T04:02:03.852 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:02:03.847+0000 7f6fff45b700 1 -- 172.21.15.181:0/3686789954 wait complete. 2024-04-27T04:02:03.852 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:02:03.847+0000 7f6fff45b700 1 Processor -- start 2024-04-27T04:02:03.852 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:02:03.847+0000 7f6fff45b700 1 -- start start 2024-04-27T04:02:03.852 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:02:03.847+0000 7f6fff45b700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6ff80fca30 0x7f6ff810d090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:02:03.852 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:02:03.847+0000 7f6fff45b700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ff810d5d0 con 0x7f6ff80fca30 2024-04-27T04:02:03.852 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:02:03.848+0000 7f6ffd1f7700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6ff80fca30 0x7f6ff810d090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:02:03.852 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:02:03.848+0000 7f6ffd1f7700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6ff80fca30 0x7f6ff810d090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.181:44002/0 (socket says 172.21.15.181:44002) 2024-04-27T04:02:03.852 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:02:03.848+0000 7f6ffd1f7700 1 -- 172.21.15.181:0/1601857855 learned_addr learned my addr 172.21.15.181:0/1601857855 (peer_addr_for_me v2:172.21.15.181:0/0) 2024-04-27T04:02:03.852 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:02:03.848+0000 7f6ffd1f7700 1 -- 172.21.15.181:0/1601857855 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ff0005040 con 0x7f6ff80fca30 2024-04-27T04:02:03.852 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:02:03.848+0000 7f6ffd1f7700 1 --2- 172.21.15.181:0/1601857855 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6ff80fca30 0x7f6ff810d090 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f6ff00096b0 tx=0x7f6ff0009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:02:03.852 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:02:03.848+0000 7f6ff5ffb700 1 -- 172.21.15.181:0/1601857855 <== mon.0 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f6ff000bed0 con 0x7f6ff80fca30 2024-04-27T04:02:03.853 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:02:03.848+0000 7f6ff5ffb700 1 -- 172.21.15.181:0/1601857855 <== mon.0 v2:172.21.15.162:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f6ff000b5c0 con 0x7f6ff80fca30 2024-04-27T04:02:03.853 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:02:03.848+0000 7f6fff45b700 1 -- 172.21.15.181:0/1601857855 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6ff810d7d0 con 0x7f6ff80fca30 2024-04-27T04:02:03.853 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:02:03.848+0000 7f6ff5ffb700 1 -- 172.21.15.181:0/1601857855 <== mon.0 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f6ff001c5b0 con 0x7f6ff80fca30 2024-04-27T04:02:03.853 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:02:03.848+0000 7f6fff45b700 1 -- 172.21.15.181:0/1601857855 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6ff810e340 con 0x7f6ff80fca30 2024-04-27T04:02:03.853 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:02:03.849+0000 7f6ff5ffb700 1 -- 172.21.15.181:0/1601857855 <== mon.0 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 13) v1 ==== 43433+0+0 (secure 0 0 0) 0x7f6ff001c820 con 0x7f6ff80fca30 2024-04-27T04:02:03.853 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:02:03.849+0000 7f6fff45b700 1 -- 172.21.15.181:0/1601857855 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6ff8102f40 con 0x7f6ff80fca30 2024-04-27T04:02:03.853 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:02:03.849+0000 7f6ff5ffb700 1 --2- 172.21.15.181:0/1601857855 >> [v2:172.21.15.162:6800/1482851081,v1:172.21.15.162:6801/1482851081] conn(0x7f6fe8036880 0x7f6fe8038d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:02:03.853 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:02:03.849+0000 7f6ff5ffb700 1 -- 172.21.15.181:0/1601857855 <== mon.0 v2:172.21.15.162:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1151+0+0 (secure 0 0 0) 0x7f6ff0049af0 con 0x7f6ff80fca30 2024-04-27T04:02:03.853 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:02:03.850+0000 7f6ffc9f6700 1 --2- 172.21.15.181:0/1601857855 >> [v2:172.21.15.162:6800/1482851081,v1:172.21.15.162:6801/1482851081] conn(0x7f6fe8036880 0x7f6fe8038d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:02:03.853 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:02:03.850+0000 7f6ffc9f6700 1 --2- 172.21.15.181:0/1601857855 >> [v2:172.21.15.162:6800/1482851081,v1:172.21.15.162:6801/1482851081] conn(0x7f6fe8036880 0x7f6fe8038d30 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f6fe4009990 tx=0x7f6fe4006e40 comp rx=0 tx=0).ready entity=mgr.14162 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:02:03.855 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:02:03.854+0000 7f6ff5ffb700 1 -- 172.21.15.181:0/1601857855 <== mon.0 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f6ff0025400 con 0x7f6ff80fca30 2024-04-27T04:02:04.114 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:02:04.113+0000 7f6fff45b700 1 -- 172.21.15.181:0/1601857855 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f6ff810ed50 con 0x7f6ff80fca30 2024-04-27T04:02:04.115 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:02:04.114+0000 7f6ff5ffb700 1 -- 172.21.15.181:0/1601857855 <== mon.0 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+743 (secure 0 0 0) 0x7f6ff0011370 con 0x7f6ff80fca30 2024-04-27T04:02:04.115 INFO:teuthology.orchestra.run.smithi181.stdout: 2024-04-27T04:02:04.115 INFO:teuthology.orchestra.run.smithi181.stdout:{"epoch":1,"fsid":"7f9750d6-044a-11ef-bc93-c7b262605968","modified":"2024-04-27T04:00:03.626312Z","created":"2024-04-27T04:00:03.626312Z","min_mon_release":17,"min_mon_release_name":"quincy","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy"],"optional":[]},"mons":[{"rank":0,"name":"smithi162","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:3300","nonce":0},{"type":"v1","addr":"172.21.15.162:6789","nonce":0}]},"addr":"172.21.15.162:6789/0","public_addr":"172.21.15.162:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-04-27T04:02:04.117 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:02:04.115+0000 7f6fff45b700 1 -- 172.21.15.181:0/1601857855 >> [v2:172.21.15.162:6800/1482851081,v1:172.21.15.162:6801/1482851081] conn(0x7f6fe8036880 msgr2=0x7f6fe8038d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:02:04.117 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:02:04.115+0000 7f6fff45b700 1 --2- 172.21.15.181:0/1601857855 >> [v2:172.21.15.162:6800/1482851081,v1:172.21.15.162:6801/1482851081] conn(0x7f6fe8036880 0x7f6fe8038d30 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f6fe4009990 tx=0x7f6fe4006e40 comp rx=0 tx=0).stop 2024-04-27T04:02:04.117 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:02:04.116+0000 7f6fff45b700 1 -- 172.21.15.181:0/1601857855 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6ff80fca30 msgr2=0x7f6ff810d090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:02:04.117 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:02:04.116+0000 7f6fff45b700 1 --2- 172.21.15.181:0/1601857855 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6ff80fca30 0x7f6ff810d090 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f6ff00096b0 tx=0x7f6ff0009e30 comp rx=0 tx=0).stop 2024-04-27T04:02:04.117 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:02:04.116+0000 7f6fff45b700 1 -- 172.21.15.181:0/1601857855 shutdown_connections 2024-04-27T04:02:04.117 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:02:04.116+0000 7f6fff45b700 1 --2- 172.21.15.181:0/1601857855 >> [v2:172.21.15.162:6800/1482851081,v1:172.21.15.162:6801/1482851081] conn(0x7f6fe8036880 0x7f6fe8038d30 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:02:04.117 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:02:04.116+0000 7f6fff45b700 1 --2- 172.21.15.181:0/1601857855 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6ff80fca30 0x7f6ff810d090 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:02:04.117 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:02:04.116+0000 7f6fff45b700 1 -- 172.21.15.181:0/1601857855 >> 172.21.15.181:0/1601857855 conn(0x7f6ff80f8170 msgr2=0x7f6ff8103b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:02:04.117 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:02:04.116+0000 7f6fff45b700 1 -- 172.21.15.181:0/1601857855 shutdown_connections 2024-04-27T04:02:04.117 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:02:04.116+0000 7f6fff45b700 1 -- 172.21.15.181:0/1601857855 wait complete. 2024-04-27T04:02:04.118 INFO:teuthology.orchestra.run.smithi181.stderr:dumped monmap epoch 1 2024-04-27T04:02:04.638 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:02:04 smithi162 ceph-mon[26841]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-27T04:02:04.639 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:02:04 smithi162 ceph-mon[26841]: from='client.? 172.21.15.181:0/1601857855' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-27T04:02:05.712 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-04-27T04:02:05.713 DEBUG:teuthology.orchestra.run.smithi181:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 7f9750d6-044a-11ef-bc93-c7b262605968 -- ceph mon dump -f json 2024-04-27T04:02:05.963 INFO:teuthology.orchestra.run.smithi181.stderr:Inferring config /etc/ceph/ceph.conf 2024-04-27T04:02:06.012 INFO:teuthology.orchestra.run.smithi181.stderr:Inferring config /etc/ceph/ceph.conf 2024-04-27T04:02:06.638 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:02:06 smithi162 ceph-mon[26841]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-27T04:02:07.443 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:02:07.441+0000 7fbd42a7c700 1 -- 172.21.15.181:0/2254180130 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fbd3c0fe080 msgr2=0x7fbd3c0fe490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:02:07.443 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:02:07.441+0000 7fbd42a7c700 1 --2- 172.21.15.181:0/2254180130 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fbd3c0fe080 0x7fbd3c0fe490 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fbd300048c0 tx=0x7fbd30009e30 comp rx=0 tx=0).stop 2024-04-27T04:02:07.443 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:02:07.441+0000 7fbd42a7c700 1 -- 172.21.15.181:0/2254180130 shutdown_connections 2024-04-27T04:02:07.443 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:02:07.441+0000 7fbd42a7c700 1 --2- 172.21.15.181:0/2254180130 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fbd3c0fe080 0x7fbd3c0fe490 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:02:07.443 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:02:07.441+0000 7fbd42a7c700 1 -- 172.21.15.181:0/2254180130 >> 172.21.15.181:0/2254180130 conn(0x7fbd3c0f6820 msgr2=0x7fbd3c0f8c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:02:07.443 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:02:07.441+0000 7fbd42a7c700 1 -- 172.21.15.181:0/2254180130 shutdown_connections 2024-04-27T04:02:07.443 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:02:07.441+0000 7fbd42a7c700 1 -- 172.21.15.181:0/2254180130 wait complete. 2024-04-27T04:02:07.443 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:02:07.442+0000 7fbd42a7c700 1 Processor -- start 2024-04-27T04:02:07.443 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:02:07.442+0000 7fbd42a7c700 1 -- start start 2024-04-27T04:02:07.443 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:02:07.442+0000 7fbd42a7c700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fbd3c0fe080 0x7fbd3c10f2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:02:07.444 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:02:07.442+0000 7fbd42a7c700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd3c10f7f0 con 0x7fbd3c0fe080 2024-04-27T04:02:07.445 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:02:07.442+0000 7fbd40818700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fbd3c0fe080 0x7fbd3c10f2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:02:07.446 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:02:07.442+0000 7fbd40818700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fbd3c0fe080 0x7fbd3c10f2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.181:44014/0 (socket says 172.21.15.181:44014) 2024-04-27T04:02:07.446 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:02:07.442+0000 7fbd40818700 1 -- 172.21.15.181:0/3990537158 learned_addr learned my addr 172.21.15.181:0/3990537158 (peer_addr_for_me v2:172.21.15.181:0/0) 2024-04-27T04:02:07.446 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:02:07.443+0000 7fbd40818700 1 -- 172.21.15.181:0/3990537158 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd30005040 con 0x7fbd3c0fe080 2024-04-27T04:02:07.446 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:02:07.443+0000 7fbd40818700 1 --2- 172.21.15.181:0/3990537158 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fbd3c0fe080 0x7fbd3c10f2b0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fbd30009680 tx=0x7fbd30009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:02:07.446 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:02:07.443+0000 7fbd397fa700 1 -- 172.21.15.181:0/3990537158 <== mon.0 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fbd3000b5c0 con 0x7fbd3c0fe080 2024-04-27T04:02:07.446 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:02:07.443+0000 7fbd397fa700 1 -- 172.21.15.181:0/3990537158 <== mon.0 v2:172.21.15.162:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fbd3000be70 con 0x7fbd3c0fe080 2024-04-27T04:02:07.446 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:02:07.443+0000 7fbd42a7c700 1 -- 172.21.15.181:0/3990537158 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbd3c10f9f0 con 0x7fbd3c0fe080 2024-04-27T04:02:07.446 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:02:07.443+0000 7fbd397fa700 1 -- 172.21.15.181:0/3990537158 <== mon.0 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fbd300135a0 con 0x7fbd3c0fe080 2024-04-27T04:02:07.446 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:02:07.443+0000 7fbd42a7c700 1 -- 172.21.15.181:0/3990537158 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbd3c110560 con 0x7fbd3c0fe080 2024-04-27T04:02:07.446 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:02:07.444+0000 7fbd397fa700 1 -- 172.21.15.181:0/3990537158 <== mon.0 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 13) v1 ==== 43433+0+0 (secure 0 0 0) 0x7fbd3001c430 con 0x7fbd3c0fe080 2024-04-27T04:02:07.446 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:02:07.444+0000 7fbd42a7c700 1 -- 172.21.15.181:0/3990537158 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbd3c102f50 con 0x7fbd3c0fe080 2024-04-27T04:02:07.446 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:02:07.444+0000 7fbd397fa700 1 --2- 172.21.15.181:0/3990537158 >> [v2:172.21.15.162:6800/1482851081,v1:172.21.15.162:6801/1482851081] conn(0x7fbd2c036880 0x7fbd2c038d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:02:07.446 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:02:07.444+0000 7fbd397fa700 1 -- 172.21.15.181:0/3990537158 <== mon.0 v2:172.21.15.162:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1151+0+0 (secure 0 0 0) 0x7fbd30049460 con 0x7fbd3c0fe080 2024-04-27T04:02:07.447 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:02:07.444+0000 7fbd3bfff700 1 --2- 172.21.15.181:0/3990537158 >> [v2:172.21.15.162:6800/1482851081,v1:172.21.15.162:6801/1482851081] conn(0x7fbd2c036880 0x7fbd2c038d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:02:07.447 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:02:07.445+0000 7fbd3bfff700 1 --2- 172.21.15.181:0/3990537158 >> [v2:172.21.15.162:6800/1482851081,v1:172.21.15.162:6801/1482851081] conn(0x7fbd2c036880 0x7fbd2c038d30 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fbd28009990 tx=0x7fbd28006e40 comp rx=0 tx=0).ready entity=mgr.14162 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:02:07.450 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:02:07.449+0000 7fbd397fa700 1 -- 172.21.15.181:0/3990537158 <== mon.0 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7fbd3001c7d0 con 0x7fbd3c0fe080 2024-04-27T04:02:07.714 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:02:07.713+0000 7fbd42a7c700 1 -- 172.21.15.181:0/3990537158 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7fbd3c1107a0 con 0x7fbd3c0fe080 2024-04-27T04:02:07.715 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:02:07.714+0000 7fbd397fa700 1 -- 172.21.15.181:0/3990537158 <== mon.0 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+743 (secure 0 0 0) 0x7fbd30021070 con 0x7fbd3c0fe080 2024-04-27T04:02:07.715 INFO:teuthology.orchestra.run.smithi181.stdout: 2024-04-27T04:02:07.716 INFO:teuthology.orchestra.run.smithi181.stdout:{"epoch":1,"fsid":"7f9750d6-044a-11ef-bc93-c7b262605968","modified":"2024-04-27T04:00:03.626312Z","created":"2024-04-27T04:00:03.626312Z","min_mon_release":17,"min_mon_release_name":"quincy","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy"],"optional":[]},"mons":[{"rank":0,"name":"smithi162","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:3300","nonce":0},{"type":"v1","addr":"172.21.15.162:6789","nonce":0}]},"addr":"172.21.15.162:6789/0","public_addr":"172.21.15.162:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-04-27T04:02:07.717 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:02:07.716+0000 7fbd42a7c700 1 -- 172.21.15.181:0/3990537158 >> [v2:172.21.15.162:6800/1482851081,v1:172.21.15.162:6801/1482851081] conn(0x7fbd2c036880 msgr2=0x7fbd2c038d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:02:07.717 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:02:07.716+0000 7fbd42a7c700 1 --2- 172.21.15.181:0/3990537158 >> [v2:172.21.15.162:6800/1482851081,v1:172.21.15.162:6801/1482851081] conn(0x7fbd2c036880 0x7fbd2c038d30 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fbd28009990 tx=0x7fbd28006e40 comp rx=0 tx=0).stop 2024-04-27T04:02:07.717 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:02:07.716+0000 7fbd42a7c700 1 -- 172.21.15.181:0/3990537158 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fbd3c0fe080 msgr2=0x7fbd3c10f2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:02:07.718 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:02:07.716+0000 7fbd42a7c700 1 --2- 172.21.15.181:0/3990537158 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fbd3c0fe080 0x7fbd3c10f2b0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fbd30009680 tx=0x7fbd30009e30 comp rx=0 tx=0).stop 2024-04-27T04:02:07.718 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:02:07.716+0000 7fbd42a7c700 1 -- 172.21.15.181:0/3990537158 shutdown_connections 2024-04-27T04:02:07.718 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:02:07.716+0000 7fbd42a7c700 1 --2- 172.21.15.181:0/3990537158 >> [v2:172.21.15.162:6800/1482851081,v1:172.21.15.162:6801/1482851081] conn(0x7fbd2c036880 0x7fbd2c038d30 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:02:07.718 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:02:07.716+0000 7fbd42a7c700 1 --2- 172.21.15.181:0/3990537158 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fbd3c0fe080 0x7fbd3c10f2b0 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:02:07.718 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:02:07.716+0000 7fbd42a7c700 1 -- 172.21.15.181:0/3990537158 >> 172.21.15.181:0/3990537158 conn(0x7fbd3c0f6820 msgr2=0x7fbd3c0f8c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:02:07.718 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:02:07.716+0000 7fbd42a7c700 1 -- 172.21.15.181:0/3990537158 shutdown_connections 2024-04-27T04:02:07.718 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:02:07.716+0000 7fbd42a7c700 1 -- 172.21.15.181:0/3990537158 wait complete. 2024-04-27T04:02:07.718 INFO:teuthology.orchestra.run.smithi181.stderr:dumped monmap epoch 1 2024-04-27T04:02:08.638 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:02:08 smithi162 ceph-mon[26841]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-27T04:02:08.639 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:02:08 smithi162 ceph-mon[26841]: from='client.? 172.21.15.181:0/3990537158' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-27T04:02:09.171 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-04-27T04:02:09.171 DEBUG:teuthology.orchestra.run.smithi181:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 7f9750d6-044a-11ef-bc93-c7b262605968 -- ceph mon dump -f json 2024-04-27T04:02:09.419 INFO:teuthology.orchestra.run.smithi181.stderr:Inferring config /etc/ceph/ceph.conf 2024-04-27T04:02:09.467 INFO:teuthology.orchestra.run.smithi181.stderr:Inferring config /etc/ceph/ceph.conf 2024-04-27T04:02:10.594 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:02:10 smithi162 ceph-mon[26841]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-27T04:02:10.594 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:02:10 smithi162 ceph-mon[26841]: from='mgr.14162 172.21.15.162:0/1568051568' entity='mgr.smithi162.oennfn' 2024-04-27T04:02:10.594 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:02:10 smithi162 ceph-mon[26841]: from='mgr.14162 172.21.15.162:0/1568051568' entity='mgr.smithi162.oennfn' 2024-04-27T04:02:10.594 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:02:10 smithi162 ceph-mon[26841]: from='mgr.14162 172.21.15.162:0/1568051568' entity='mgr.smithi162.oennfn' 2024-04-27T04:02:10.594 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:02:10 smithi162 ceph-mon[26841]: from='mgr.14162 172.21.15.162:0/1568051568' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "mgr module enable", "module": "prometheus"}]: dispatch 2024-04-27T04:02:10.997 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:02:10.995+0000 7fd8d9177700 1 -- 172.21.15.181:0/19443604 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd8d40fb930 msgr2=0x7fd8d40fbd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:02:10.997 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:02:10.995+0000 7fd8d9177700 1 --2- 172.21.15.181:0/19443604 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd8d40fb930 0x7fd8d40fbd40 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fd8c80048c0 tx=0x7fd8c8009e30 comp rx=0 tx=0).stop 2024-04-27T04:02:10.997 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:02:10.995+0000 7fd8d9177700 1 -- 172.21.15.181:0/19443604 shutdown_connections 2024-04-27T04:02:10.998 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:02:10.995+0000 7fd8d9177700 1 --2- 172.21.15.181:0/19443604 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd8d40fb930 0x7fd8d40fbd40 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:02:10.998 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:02:10.995+0000 7fd8d9177700 1 -- 172.21.15.181:0/19443604 >> 172.21.15.181:0/19443604 conn(0x7fd8d40f7070 msgr2=0x7fd8d40f94c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:02:10.998 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:02:10.995+0000 7fd8d9177700 1 -- 172.21.15.181:0/19443604 shutdown_connections 2024-04-27T04:02:10.998 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:02:10.995+0000 7fd8d9177700 1 -- 172.21.15.181:0/19443604 wait complete. 2024-04-27T04:02:10.998 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:02:10.996+0000 7fd8d9177700 1 Processor -- start 2024-04-27T04:02:10.998 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:02:10.996+0000 7fd8d9177700 1 -- start start 2024-04-27T04:02:10.998 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:02:10.996+0000 7fd8d9177700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd8d40fb930 0x7fd8d410bfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:02:10.998 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:02:10.996+0000 7fd8d9177700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8d410c4f0 con 0x7fd8d40fb930 2024-04-27T04:02:10.998 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:02:10.996+0000 7fd8d2d9d700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd8d40fb930 0x7fd8d410bfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:02:10.998 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:02:10.996+0000 7fd8d2d9d700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd8d40fb930 0x7fd8d410bfb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.181:47718/0 (socket says 172.21.15.181:47718) 2024-04-27T04:02:10.998 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:02:10.996+0000 7fd8d2d9d700 1 -- 172.21.15.181:0/1953023242 learned_addr learned my addr 172.21.15.181:0/1953023242 (peer_addr_for_me v2:172.21.15.181:0/0) 2024-04-27T04:02:10.998 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:02:10.997+0000 7fd8d2d9d700 1 -- 172.21.15.181:0/1953023242 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8c8005040 con 0x7fd8d40fb930 2024-04-27T04:02:11.000 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:02:10.997+0000 7fd8d2d9d700 1 --2- 172.21.15.181:0/1953023242 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd8d40fb930 0x7fd8d410bfb0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fd8c80096b0 tx=0x7fd8c8009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:02:11.000 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:02:10.997+0000 7fd8bb7fe700 1 -- 172.21.15.181:0/1953023242 <== mon.0 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fd8c800bed0 con 0x7fd8d40fb930 2024-04-27T04:02:11.000 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:02:10.997+0000 7fd8d9177700 1 -- 172.21.15.181:0/1953023242 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd8d410c6f0 con 0x7fd8d40fb930 2024-04-27T04:02:11.000 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:02:10.997+0000 7fd8bb7fe700 1 -- 172.21.15.181:0/1953023242 <== mon.0 v2:172.21.15.162:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fd8c800b5c0 con 0x7fd8d40fb930 2024-04-27T04:02:11.000 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:02:10.997+0000 7fd8bb7fe700 1 -- 172.21.15.181:0/1953023242 <== mon.0 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fd8c801c5b0 con 0x7fd8d40fb930 2024-04-27T04:02:11.000 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:02:10.997+0000 7fd8d9177700 1 -- 172.21.15.181:0/1953023242 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd8d410d260 con 0x7fd8d40fb930 2024-04-27T04:02:11.000 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:02:10.998+0000 7fd8bb7fe700 1 -- 172.21.15.181:0/1953023242 <== mon.0 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 14) v1 ==== 43447+0+0 (secure 0 0 0) 0x7fd8c801c820 con 0x7fd8d40fb930 2024-04-27T04:02:11.000 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:02:10.998+0000 7fd8bb7fe700 1 --2- 172.21.15.181:0/1953023242 >> [v2:172.21.15.162:6800/1482851081,v1:172.21.15.162:6801/1482851081] conn(0x7fd8bc0368d0 0x7fd8bc038d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:02:11.000 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:02:10.998+0000 7fd8bb7fe700 1 -- 172.21.15.181:0/1953023242 <== mon.0 v2:172.21.15.162:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1151+0+0 (secure 0 0 0) 0x7fd8c8049b20 con 0x7fd8d40fb930 2024-04-27T04:02:11.000 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:02:10.998+0000 7fd8d9177700 1 -- 172.21.15.181:0/1953023242 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd8d4101e40 con 0x7fd8d40fb930 2024-04-27T04:02:11.000 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:02:10.998+0000 7fd8d259c700 1 -- 172.21.15.181:0/1953023242 >> [v2:172.21.15.162:6800/1482851081,v1:172.21.15.162:6801/1482851081] conn(0x7fd8bc0368d0 msgr2=0x7fd8bc038d80 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.162:6800/1482851081 2024-04-27T04:02:11.000 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:02:10.998+0000 7fd8d259c700 1 --2- 172.21.15.181:0/1953023242 >> [v2:172.21.15.162:6800/1482851081,v1:172.21.15.162:6801/1482851081] conn(0x7fd8bc0368d0 0x7fd8bc038d80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-27T04:02:11.003 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:02:11.003+0000 7fd8bb7fe700 1 -- 172.21.15.181:0/1953023242 <== mon.0 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7fd8c8023300 con 0x7fd8d40fb930 2024-04-27T04:02:11.200 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:02:11.199+0000 7fd8d259c700 1 -- 172.21.15.181:0/1953023242 >> [v2:172.21.15.162:6800/1482851081,v1:172.21.15.162:6801/1482851081] conn(0x7fd8bc0368d0 msgr2=0x7fd8bc038d80 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.162:6800/1482851081 2024-04-27T04:02:11.200 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:02:11.199+0000 7fd8d259c700 1 --2- 172.21.15.181:0/1953023242 >> [v2:172.21.15.162:6800/1482851081,v1:172.21.15.162:6801/1482851081] conn(0x7fd8bc0368d0 0x7fd8bc038d80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-27T04:02:11.266 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:02:11.265+0000 7fd8d9177700 1 -- 172.21.15.181:0/1953023242 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7fd8d41017f0 con 0x7fd8d40fb930 2024-04-27T04:02:11.266 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:02:11.265+0000 7fd8bb7fe700 1 -- 172.21.15.181:0/1953023242 <== mon.0 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+743 (secure 0 0 0) 0x7fd8c8034090 con 0x7fd8d40fb930 2024-04-27T04:02:11.267 INFO:teuthology.orchestra.run.smithi181.stdout: 2024-04-27T04:02:11.267 INFO:teuthology.orchestra.run.smithi181.stdout:{"epoch":1,"fsid":"7f9750d6-044a-11ef-bc93-c7b262605968","modified":"2024-04-27T04:00:03.626312Z","created":"2024-04-27T04:00:03.626312Z","min_mon_release":17,"min_mon_release_name":"quincy","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy"],"optional":[]},"mons":[{"rank":0,"name":"smithi162","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:3300","nonce":0},{"type":"v1","addr":"172.21.15.162:6789","nonce":0}]},"addr":"172.21.15.162:6789/0","public_addr":"172.21.15.162:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-04-27T04:02:11.268 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:02:11.267+0000 7fd8d9177700 1 -- 172.21.15.181:0/1953023242 >> [v2:172.21.15.162:6800/1482851081,v1:172.21.15.162:6801/1482851081] conn(0x7fd8bc0368d0 msgr2=0x7fd8bc038d80 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-27T04:02:11.269 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:02:11.267+0000 7fd8d9177700 1 --2- 172.21.15.181:0/1953023242 >> [v2:172.21.15.162:6800/1482851081,v1:172.21.15.162:6801/1482851081] conn(0x7fd8bc0368d0 0x7fd8bc038d80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:02:11.269 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:02:11.267+0000 7fd8d9177700 1 -- 172.21.15.181:0/1953023242 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd8d40fb930 msgr2=0x7fd8d410bfb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:02:11.269 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:02:11.267+0000 7fd8d9177700 1 --2- 172.21.15.181:0/1953023242 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd8d40fb930 0x7fd8d410bfb0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fd8c80096b0 tx=0x7fd8c8009e30 comp rx=0 tx=0).stop 2024-04-27T04:02:11.269 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:02:11.267+0000 7fd8d9177700 1 -- 172.21.15.181:0/1953023242 shutdown_connections 2024-04-27T04:02:11.269 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:02:11.267+0000 7fd8d9177700 1 --2- 172.21.15.181:0/1953023242 >> [v2:172.21.15.162:6800/1482851081,v1:172.21.15.162:6801/1482851081] conn(0x7fd8bc0368d0 0x7fd8bc038d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:02:11.269 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:02:11.267+0000 7fd8d9177700 1 --2- 172.21.15.181:0/1953023242 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd8d40fb930 0x7fd8d410bfb0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:02:11.269 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:02:11.267+0000 7fd8d9177700 1 -- 172.21.15.181:0/1953023242 >> 172.21.15.181:0/1953023242 conn(0x7fd8d40f7070 msgr2=0x7fd8d4102a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:02:11.269 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:02:11.268+0000 7fd8d9177700 1 -- 172.21.15.181:0/1953023242 shutdown_connections 2024-04-27T04:02:11.269 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:02:11.268+0000 7fd8d9177700 1 -- 172.21.15.181:0/1953023242 wait complete. 2024-04-27T04:02:11.270 INFO:teuthology.orchestra.run.smithi181.stderr:dumped monmap epoch 1 2024-04-27T04:02:11.638 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:02:11 smithi162 ceph-mon[26841]: from='mgr.14162 172.21.15.162:0/1568051568' entity='mgr.smithi162.oennfn' cmd='[{"prefix": "mgr module enable", "module": "prometheus"}]': finished 2024-04-27T04:02:11.638 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:02:11 smithi162 ceph-mon[26841]: mgrmap e14: smithi162.oennfn(active, since 59s) 2024-04-27T04:02:12.528 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:02:12 smithi162 ceph-mon[26841]: from='client.? 172.21.15.181:0/1953023242' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-27T04:02:12.879 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-04-27T04:02:12.879 DEBUG:teuthology.orchestra.run.smithi181:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 7f9750d6-044a-11ef-bc93-c7b262605968 -- ceph mon dump -f json 2024-04-27T04:02:13.123 INFO:teuthology.orchestra.run.smithi181.stderr:Inferring config /etc/ceph/ceph.conf 2024-04-27T04:02:13.171 INFO:teuthology.orchestra.run.smithi181.stderr:Inferring config /etc/ceph/ceph.conf 2024-04-27T04:02:14.565 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:02:14.563+0000 7f74ff26f700 1 -- 172.21.15.181:0/1098740364 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f74f80fca40 msgr2=0x7f74f80fce50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:02:14.565 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:02:14.563+0000 7f74ff26f700 1 --2- 172.21.15.181:0/1098740364 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f74f80fca40 0x7f74f80fce50 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f74f40048c0 tx=0x7f74f4009e30 comp rx=0 tx=0).stop 2024-04-27T04:02:14.566 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:02:14.563+0000 7f74ff26f700 1 -- 172.21.15.181:0/1098740364 shutdown_connections 2024-04-27T04:02:14.566 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:02:14.563+0000 7f74ff26f700 1 --2- 172.21.15.181:0/1098740364 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f74f80fca40 0x7f74f80fce50 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:02:14.566 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:02:14.563+0000 7f74ff26f700 1 -- 172.21.15.181:0/1098740364 >> 172.21.15.181:0/1098740364 conn(0x7f74f80f81a0 msgr2=0x7f74f80fa5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:02:14.566 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:02:14.563+0000 7f74ff26f700 1 -- 172.21.15.181:0/1098740364 shutdown_connections 2024-04-27T04:02:14.566 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:02:14.563+0000 7f74ff26f700 1 -- 172.21.15.181:0/1098740364 wait complete. 2024-04-27T04:02:14.566 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:02:14.563+0000 7f74ff26f700 1 Processor -- start 2024-04-27T04:02:14.566 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:02:14.563+0000 7f74ff26f700 1 -- start start 2024-04-27T04:02:14.566 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:02:14.563+0000 7f74ff26f700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f74f80fca40 0x7f74f8071d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:02:14.566 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:02:14.563+0000 7f74ff26f700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74f8072240 con 0x7f74f80fca40 2024-04-27T04:02:14.566 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:02:14.564+0000 7f74fd00b700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f74f80fca40 0x7f74f8071d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:02:14.566 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:02:14.564+0000 7f74fd00b700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f74f80fca40 0x7f74f8071d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.181:47732/0 (socket says 172.21.15.181:47732) 2024-04-27T04:02:14.566 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:02:14.564+0000 7f74fd00b700 1 -- 172.21.15.181:0/890836901 learned_addr learned my addr 172.21.15.181:0/890836901 (peer_addr_for_me v2:172.21.15.181:0/0) 2024-04-27T04:02:14.566 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:02:14.564+0000 7f74fd00b700 1 -- 172.21.15.181:0/890836901 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f74f4005040 con 0x7f74f80fca40 2024-04-27T04:02:14.567 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:02:14.564+0000 7f74fd00b700 1 --2- 172.21.15.181:0/890836901 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f74f80fca40 0x7f74f8071d00 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f74f40096b0 tx=0x7f74f4009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:02:14.567 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:02:14.564+0000 7f74edffb700 1 -- 172.21.15.181:0/890836901 <== mon.0 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f74f400bed0 con 0x7f74f80fca40 2024-04-27T04:02:14.567 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:02:14.564+0000 7f74edffb700 1 -- 172.21.15.181:0/890836901 <== mon.0 v2:172.21.15.162:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f74f400b5c0 con 0x7f74f80fca40 2024-04-27T04:02:14.567 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:02:14.564+0000 7f74ff26f700 1 -- 172.21.15.181:0/890836901 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f74f8070310 con 0x7f74f80fca40 2024-04-27T04:02:14.567 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:02:14.564+0000 7f74edffb700 1 -- 172.21.15.181:0/890836901 <== mon.0 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f74f401c5b0 con 0x7f74f80fca40 2024-04-27T04:02:14.567 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:02:14.565+0000 7f74ff26f700 1 -- 172.21.15.181:0/890836901 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f74f8070800 con 0x7f74f80fca40 2024-04-27T04:02:14.568 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:02:14.565+0000 7f74ff26f700 1 -- 172.21.15.181:0/890836901 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f74f805f5c0 con 0x7f74f80fca40 2024-04-27T04:02:14.568 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:02:14.565+0000 7f74edffb700 1 -- 172.21.15.181:0/890836901 <== mon.0 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 14) v1 ==== 43447+0+0 (secure 0 0 0) 0x7f74f4013420 con 0x7f74f80fca40 2024-04-27T04:02:14.568 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:02:14.566+0000 7f74edffb700 1 --2- 172.21.15.181:0/890836901 >> [v2:172.21.15.162:6800/1482851081,v1:172.21.15.162:6801/1482851081] conn(0x7f74e40368d0 0x7f74e4038d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:02:14.568 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:02:14.566+0000 7f74edffb700 1 -- 172.21.15.181:0/890836901 <== mon.0 v2:172.21.15.162:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1151+0+0 (secure 0 0 0) 0x7f74f4049ab0 con 0x7f74f80fca40 2024-04-27T04:02:14.568 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:02:14.566+0000 7f74fc80a700 1 -- 172.21.15.181:0/890836901 >> [v2:172.21.15.162:6800/1482851081,v1:172.21.15.162:6801/1482851081] conn(0x7f74e40368d0 msgr2=0x7f74e4038d80 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.162:6800/1482851081 2024-04-27T04:02:14.568 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:02:14.566+0000 7f74fc80a700 1 --2- 172.21.15.181:0/890836901 >> [v2:172.21.15.162:6800/1482851081,v1:172.21.15.162:6801/1482851081] conn(0x7f74e40368d0 0x7f74e4038d80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-27T04:02:14.571 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:02:14.570+0000 7f74edffb700 1 -- 172.21.15.181:0/890836901 <== mon.0 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f74f4025400 con 0x7f74f80fca40 2024-04-27T04:02:14.768 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:02:14.767+0000 7f74fc80a700 1 -- 172.21.15.181:0/890836901 >> [v2:172.21.15.162:6800/1482851081,v1:172.21.15.162:6801/1482851081] conn(0x7f74e40368d0 msgr2=0x7f74e4038d80 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.162:6800/1482851081 2024-04-27T04:02:14.768 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:02:14.767+0000 7f74fc80a700 1 --2- 172.21.15.181:0/890836901 >> [v2:172.21.15.162:6800/1482851081,v1:172.21.15.162:6801/1482851081] conn(0x7f74e40368d0 0x7f74e4038d80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-27T04:02:14.831 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:02:14.830+0000 7f74ff26f700 1 -- 172.21.15.181:0/890836901 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f74f8106420 con 0x7f74f80fca40 2024-04-27T04:02:14.832 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:02:14.831+0000 7f74edffb700 1 -- 172.21.15.181:0/890836901 <== mon.0 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+743 (secure 0 0 0) 0x7f74f4004d30 con 0x7f74f80fca40 2024-04-27T04:02:14.832 INFO:teuthology.orchestra.run.smithi181.stdout: 2024-04-27T04:02:14.832 INFO:teuthology.orchestra.run.smithi181.stdout:{"epoch":1,"fsid":"7f9750d6-044a-11ef-bc93-c7b262605968","modified":"2024-04-27T04:00:03.626312Z","created":"2024-04-27T04:00:03.626312Z","min_mon_release":17,"min_mon_release_name":"quincy","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy"],"optional":[]},"mons":[{"rank":0,"name":"smithi162","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:3300","nonce":0},{"type":"v1","addr":"172.21.15.162:6789","nonce":0}]},"addr":"172.21.15.162:6789/0","public_addr":"172.21.15.162:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-04-27T04:02:14.834 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:02:14.833+0000 7f74ff26f700 1 -- 172.21.15.181:0/890836901 >> [v2:172.21.15.162:6800/1482851081,v1:172.21.15.162:6801/1482851081] conn(0x7f74e40368d0 msgr2=0x7f74e4038d80 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-27T04:02:14.835 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:02:14.833+0000 7f74ff26f700 1 --2- 172.21.15.181:0/890836901 >> [v2:172.21.15.162:6800/1482851081,v1:172.21.15.162:6801/1482851081] conn(0x7f74e40368d0 0x7f74e4038d80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:02:14.835 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:02:14.833+0000 7f74ff26f700 1 -- 172.21.15.181:0/890836901 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f74f80fca40 msgr2=0x7f74f8071d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:02:14.835 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:02:14.833+0000 7f74ff26f700 1 --2- 172.21.15.181:0/890836901 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f74f80fca40 0x7f74f8071d00 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f74f40096b0 tx=0x7f74f4009e30 comp rx=0 tx=0).stop 2024-04-27T04:02:14.835 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:02:14.833+0000 7f74ff26f700 1 -- 172.21.15.181:0/890836901 shutdown_connections 2024-04-27T04:02:14.835 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:02:14.833+0000 7f74ff26f700 1 --2- 172.21.15.181:0/890836901 >> [v2:172.21.15.162:6800/1482851081,v1:172.21.15.162:6801/1482851081] conn(0x7f74e40368d0 0x7f74e4038d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:02:14.835 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:02:14.833+0000 7f74ff26f700 1 --2- 172.21.15.181:0/890836901 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f74f80fca40 0x7f74f8071d00 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:02:14.835 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:02:14.833+0000 7f74ff26f700 1 -- 172.21.15.181:0/890836901 >> 172.21.15.181:0/890836901 conn(0x7f74f80f81a0 msgr2=0x7f74f8103b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:02:14.835 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:02:14.834+0000 7f74ff26f700 1 -- 172.21.15.181:0/890836901 shutdown_connections 2024-04-27T04:02:14.835 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:02:14.834+0000 7f74ff26f700 1 -- 172.21.15.181:0/890836901 wait complete. 2024-04-27T04:02:14.836 INFO:teuthology.orchestra.run.smithi181.stderr:dumped monmap epoch 1 2024-04-27T04:02:15.027 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:02:14 smithi162 ceph-mon[26841]: from='client.? 172.21.15.181:0/890836901' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-27T04:02:16.496 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-04-27T04:02:16.496 DEBUG:teuthology.orchestra.run.smithi181:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 7f9750d6-044a-11ef-bc93-c7b262605968 -- ceph mon dump -f json 2024-04-27T04:02:16.741 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:02:16 smithi162 ceph-mon[26841]: Active manager daemon smithi162.oennfn restarted 2024-04-27T04:02:16.741 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:02:16 smithi162 ceph-mon[26841]: Activating manager daemon smithi162.oennfn 2024-04-27T04:02:16.741 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:02:16 smithi162 ceph-mon[26841]: osdmap e5: 0 total, 0 up, 0 in 2024-04-27T04:02:16.741 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:02:16 smithi162 ceph-mon[26841]: mgrmap e15: smithi162.oennfn(active, starting, since 0.00315499s) 2024-04-27T04:02:16.741 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:02:16 smithi162 ceph-mon[26841]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "mon metadata", "id": "smithi162"}]: dispatch 2024-04-27T04:02:16.741 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:02:16 smithi162 ceph-mon[26841]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "mgr metadata", "who": "smithi162.oennfn", "id": "smithi162.oennfn"}]: dispatch 2024-04-27T04:02:16.741 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:02:16 smithi162 ceph-mon[26841]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-04-27T04:02:16.741 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:02:16 smithi162 ceph-mon[26841]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-04-27T04:02:16.741 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:02:16 smithi162 ceph-mon[26841]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-04-27T04:02:16.741 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:02:16 smithi162 ceph-mon[26841]: Manager daemon smithi162.oennfn is now available 2024-04-27T04:02:16.741 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:02:16 smithi162 ceph-mon[26841]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T04:02:16.741 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:02:16 smithi162 ceph-mon[26841]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T04:02:16.742 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:02:16 smithi162 ceph-mon[26841]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' 2024-04-27T04:02:16.742 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:02:16 smithi162 ceph-mon[26841]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-27T04:02:16.745 INFO:teuthology.orchestra.run.smithi181.stderr:Inferring config /etc/ceph/ceph.conf 2024-04-27T04:02:16.794 INFO:teuthology.orchestra.run.smithi181.stderr:Inferring config /etc/ceph/ceph.conf 2024-04-27T04:02:18.138 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:02:17 smithi162 ceph-mon[26841]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi162.oennfn/mirror_snapshot_schedule"}]: dispatch 2024-04-27T04:02:18.138 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:02:17 smithi162 ceph-mon[26841]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi162.oennfn/trash_purge_schedule"}]: dispatch 2024-04-27T04:02:18.139 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:02:17 smithi162 ceph-mon[26841]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' 2024-04-27T04:02:18.139 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:02:17 smithi162 ceph-mon[26841]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' 2024-04-27T04:02:18.139 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:02:17 smithi162 ceph-mon[26841]: mgrmap e16: smithi162.oennfn(active, since 1.00509s) 2024-04-27T04:02:18.344 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:02:18.342+0000 7f0f72422700 1 -- 172.21.15.181:0/2499375356 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0f6c0fb940 msgr2=0x7f0f6c0fbd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:02:18.345 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:02:18.342+0000 7f0f72422700 1 --2- 172.21.15.181:0/2499375356 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0f6c0fb940 0x7f0f6c0fbd50 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f0f540048c0 tx=0x7f0f54009e30 comp rx=0 tx=0).stop 2024-04-27T04:02:18.345 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:02:18.342+0000 7f0f72422700 1 -- 172.21.15.181:0/2499375356 shutdown_connections 2024-04-27T04:02:18.345 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:02:18.342+0000 7f0f72422700 1 --2- 172.21.15.181:0/2499375356 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0f6c0fb940 0x7f0f6c0fbd50 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:02:18.345 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:02:18.342+0000 7f0f72422700 1 -- 172.21.15.181:0/2499375356 >> 172.21.15.181:0/2499375356 conn(0x7f0f6c0f70a0 msgr2=0x7f0f6c0f94d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:02:18.345 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:02:18.342+0000 7f0f72422700 1 -- 172.21.15.181:0/2499375356 shutdown_connections 2024-04-27T04:02:18.345 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:02:18.342+0000 7f0f72422700 1 -- 172.21.15.181:0/2499375356 wait complete. 2024-04-27T04:02:18.345 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:02:18.343+0000 7f0f72422700 1 Processor -- start 2024-04-27T04:02:18.345 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:02:18.343+0000 7f0f72422700 1 -- start start 2024-04-27T04:02:18.345 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:02:18.343+0000 7f0f72422700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0f6c0fb940 0x7f0f6c065bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:02:18.345 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:02:18.343+0000 7f0f72422700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f6c10fdb0 con 0x7f0f6c0fb940 2024-04-27T04:02:18.345 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:02:18.343+0000 7f0f6bfff700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0f6c0fb940 0x7f0f6c065bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:02:18.345 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:02:18.343+0000 7f0f6bfff700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0f6c0fb940 0x7f0f6c065bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.181:56760/0 (socket says 172.21.15.181:56760) 2024-04-27T04:02:18.345 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:02:18.343+0000 7f0f6bfff700 1 -- 172.21.15.181:0/458349557 learned_addr learned my addr 172.21.15.181:0/458349557 (peer_addr_for_me v2:172.21.15.181:0/0) 2024-04-27T04:02:18.346 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:02:18.344+0000 7f0f6bfff700 1 -- 172.21.15.181:0/458349557 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0f54005040 con 0x7f0f6c0fb940 2024-04-27T04:02:18.346 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:02:18.344+0000 7f0f6bfff700 1 --2- 172.21.15.181:0/458349557 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0f6c0fb940 0x7f0f6c065bb0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f0f540096b0 tx=0x7f0f54009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:02:18.347 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:02:18.344+0000 7f0f68ff9700 1 -- 172.21.15.181:0/458349557 <== mon.0 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f0f5400bed0 con 0x7f0f6c0fb940 2024-04-27T04:02:18.347 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:02:18.344+0000 7f0f68ff9700 1 -- 172.21.15.181:0/458349557 <== mon.0 v2:172.21.15.162:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f0f5400b5c0 con 0x7f0f6c0fb940 2024-04-27T04:02:18.347 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:02:18.344+0000 7f0f68ff9700 1 -- 172.21.15.181:0/458349557 <== mon.0 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f0f5401c5b0 con 0x7f0f6c0fb940 2024-04-27T04:02:18.347 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:02:18.344+0000 7f0f72422700 1 -- 172.21.15.181:0/458349557 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0f6c10fef0 con 0x7f0f6c0fb940 2024-04-27T04:02:18.347 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:02:18.344+0000 7f0f72422700 1 -- 172.21.15.181:0/458349557 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0f6c1103e0 con 0x7f0f6c0fb940 2024-04-27T04:02:18.347 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:02:18.345+0000 7f0f72422700 1 -- 172.21.15.181:0/458349557 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0f6c105900 con 0x7f0f6c0fb940 2024-04-27T04:02:18.347 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:02:18.345+0000 7f0f68ff9700 1 -- 172.21.15.181:0/458349557 <== mon.0 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 16) v1 ==== 43297+0+0 (secure 0 0 0) 0x7f0f54013a80 con 0x7f0f6c0fb940 2024-04-27T04:02:18.348 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:02:18.345+0000 7f0f68ff9700 1 --2- 172.21.15.181:0/458349557 >> [v2:172.21.15.162:6800/4079782599,v1:172.21.15.162:6801/4079782599] conn(0x7f0f58036710 0x7f0f58038bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:02:18.348 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:02:18.345+0000 7f0f68ff9700 1 -- 172.21.15.181:0/458349557 <== mon.0 v2:172.21.15.162:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1366+0+0 (secure 0 0 0) 0x7f0f540498d0 con 0x7f0f6c0fb940 2024-04-27T04:02:18.348 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:02:18.346+0000 7f0f6b7fe700 1 --2- 172.21.15.181:0/458349557 >> [v2:172.21.15.162:6800/4079782599,v1:172.21.15.162:6801/4079782599] conn(0x7f0f58036710 0x7f0f58038bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:02:18.348 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:02:18.346+0000 7f0f6b7fe700 1 --2- 172.21.15.181:0/458349557 >> [v2:172.21.15.162:6800/4079782599,v1:172.21.15.162:6801/4079782599] conn(0x7f0f58036710 0x7f0f58038bc0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f0f5c009990 tx=0x7f0f5c006e40 comp rx=0 tx=0).ready entity=mgr.14210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:02:18.351 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:02:18.351+0000 7f0f68ff9700 1 -- 172.21.15.181:0/458349557 <== mon.0 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f0f54011940 con 0x7f0f6c0fb940 2024-04-27T04:02:18.625 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:02:18.624+0000 7f0f72422700 1 -- 172.21.15.181:0/458349557 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f0f6c05f5c0 con 0x7f0f6c0fb940 2024-04-27T04:02:18.626 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:02:18.624+0000 7f0f68ff9700 1 -- 172.21.15.181:0/458349557 <== mon.0 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+743 (secure 0 0 0) 0x7f0f54011370 con 0x7f0f6c0fb940 2024-04-27T04:02:18.626 INFO:teuthology.orchestra.run.smithi181.stdout: 2024-04-27T04:02:18.626 INFO:teuthology.orchestra.run.smithi181.stdout:{"epoch":1,"fsid":"7f9750d6-044a-11ef-bc93-c7b262605968","modified":"2024-04-27T04:00:03.626312Z","created":"2024-04-27T04:00:03.626312Z","min_mon_release":17,"min_mon_release_name":"quincy","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy"],"optional":[]},"mons":[{"rank":0,"name":"smithi162","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:3300","nonce":0},{"type":"v1","addr":"172.21.15.162:6789","nonce":0}]},"addr":"172.21.15.162:6789/0","public_addr":"172.21.15.162:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-04-27T04:02:18.628 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:02:18.627+0000 7f0f72422700 1 -- 172.21.15.181:0/458349557 >> [v2:172.21.15.162:6800/4079782599,v1:172.21.15.162:6801/4079782599] conn(0x7f0f58036710 msgr2=0x7f0f58038bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:02:18.628 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:02:18.627+0000 7f0f72422700 1 --2- 172.21.15.181:0/458349557 >> [v2:172.21.15.162:6800/4079782599,v1:172.21.15.162:6801/4079782599] conn(0x7f0f58036710 0x7f0f58038bc0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f0f5c009990 tx=0x7f0f5c006e40 comp rx=0 tx=0).stop 2024-04-27T04:02:18.628 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:02:18.627+0000 7f0f72422700 1 -- 172.21.15.181:0/458349557 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0f6c0fb940 msgr2=0x7f0f6c065bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:02:18.628 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:02:18.627+0000 7f0f72422700 1 --2- 172.21.15.181:0/458349557 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0f6c0fb940 0x7f0f6c065bb0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f0f540096b0 tx=0x7f0f54009e30 comp rx=0 tx=0).stop 2024-04-27T04:02:18.628 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:02:18.627+0000 7f0f72422700 1 -- 172.21.15.181:0/458349557 shutdown_connections 2024-04-27T04:02:18.628 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:02:18.627+0000 7f0f72422700 1 --2- 172.21.15.181:0/458349557 >> [v2:172.21.15.162:6800/4079782599,v1:172.21.15.162:6801/4079782599] conn(0x7f0f58036710 0x7f0f58038bc0 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:02:18.628 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:02:18.627+0000 7f0f72422700 1 --2- 172.21.15.181:0/458349557 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0f6c0fb940 0x7f0f6c065bb0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:02:18.628 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:02:18.627+0000 7f0f72422700 1 -- 172.21.15.181:0/458349557 >> 172.21.15.181:0/458349557 conn(0x7f0f6c0f70a0 msgr2=0x7f0f6c102a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:02:18.628 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:02:18.627+0000 7f0f72422700 1 -- 172.21.15.181:0/458349557 shutdown_connections 2024-04-27T04:02:18.628 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:02:18.627+0000 7f0f72422700 1 -- 172.21.15.181:0/458349557 wait complete. 2024-04-27T04:02:18.629 INFO:teuthology.orchestra.run.smithi181.stderr:dumped monmap epoch 1 2024-04-27T04:02:19.294 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:02:18 smithi162 ceph-mon[26841]: [27/Apr/2024:04:02:17] ENGINE Bus STARTING 2024-04-27T04:02:19.294 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:02:18 smithi162 ceph-mon[26841]: [27/Apr/2024:04:02:17] ENGINE Serving on https://172.21.15.162:7150 2024-04-27T04:02:19.294 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:02:18 smithi162 ceph-mon[26841]: [27/Apr/2024:04:02:17] ENGINE Bus STARTED 2024-04-27T04:02:19.294 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:02:18 smithi162 ceph-mon[26841]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' 2024-04-27T04:02:19.294 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:02:18 smithi162 ceph-mon[26841]: from='client.? 172.21.15.181:0/458349557' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-27T04:02:19.295 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:02:18 smithi162 ceph-mon[26841]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' 2024-04-27T04:02:20.138 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:02:19 smithi162 ceph-mon[26841]: mgrmap e17: smithi162.oennfn(active, since 2s) 2024-04-27T04:02:20.254 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-04-27T04:02:20.255 DEBUG:teuthology.orchestra.run.smithi181:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 7f9750d6-044a-11ef-bc93-c7b262605968 -- ceph mon dump -f json 2024-04-27T04:02:20.653 INFO:teuthology.orchestra.run.smithi181.stderr:Inferring config /etc/ceph/ceph.conf 2024-04-27T04:02:20.713 INFO:teuthology.orchestra.run.smithi181.stderr:Inferring config /etc/ceph/ceph.conf 2024-04-27T04:02:22.138 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:02:21 smithi162 ceph-mon[26841]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' 2024-04-27T04:02:22.138 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:02:21 smithi162 ceph-mon[26841]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' 2024-04-27T04:02:22.138 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:02:21 smithi162 ceph-mon[26841]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' 2024-04-27T04:02:22.139 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:02:21 smithi162 ceph-mon[26841]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' 2024-04-27T04:02:22.139 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:02:21 smithi162 ceph-mon[26841]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "config rm", "who": "osd/host:smithi162", "name": "osd_memory_target"}]: dispatch 2024-04-27T04:02:22.950 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:02:22.949+0000 7fc93cc8c700 1 -- 172.21.15.181:0/2802453301 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc9381019d0 msgr2=0x7fc938101da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:02:22.950 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:02:22.949+0000 7fc93cc8c700 1 --2- 172.21.15.181:0/2802453301 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc9381019d0 0x7fc938101da0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fc92c0048c0 tx=0x7fc92c009e30 comp rx=0 tx=0).stop 2024-04-27T04:02:22.950 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:02:22.949+0000 7fc93cc8c700 1 -- 172.21.15.181:0/2802453301 shutdown_connections 2024-04-27T04:02:22.950 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:02:22.949+0000 7fc93cc8c700 1 --2- 172.21.15.181:0/2802453301 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc9381019d0 0x7fc938101da0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:02:22.951 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:02:22.949+0000 7fc93cc8c700 1 -- 172.21.15.181:0/2802453301 >> 172.21.15.181:0/2802453301 conn(0x7fc938071800 msgr2=0x7fc938071c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:02:22.951 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:02:22.949+0000 7fc93cc8c700 1 -- 172.21.15.181:0/2802453301 shutdown_connections 2024-04-27T04:02:22.951 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:02:22.949+0000 7fc93cc8c700 1 -- 172.21.15.181:0/2802453301 wait complete. 2024-04-27T04:02:22.951 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:02:22.950+0000 7fc93cc8c700 1 Processor -- start 2024-04-27T04:02:22.951 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:02:22.950+0000 7fc93cc8c700 1 -- start start 2024-04-27T04:02:22.951 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:02:22.950+0000 7fc93cc8c700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc9381019d0 0x7fc93810e0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:02:22.951 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:02:22.950+0000 7fc93cc8c700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc93810e610 con 0x7fc9381019d0 2024-04-27T04:02:22.951 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:02:22.950+0000 7fc93659c700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc9381019d0 0x7fc93810e0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:02:22.951 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:02:22.950+0000 7fc93659c700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc9381019d0 0x7fc93810e0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.181:56778/0 (socket says 172.21.15.181:56778) 2024-04-27T04:02:22.951 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:02:22.950+0000 7fc93659c700 1 -- 172.21.15.181:0/341183125 learned_addr learned my addr 172.21.15.181:0/341183125 (peer_addr_for_me v2:172.21.15.181:0/0) 2024-04-27T04:02:22.951 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:02:22.951+0000 7fc93659c700 1 -- 172.21.15.181:0/341183125 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc92c005040 con 0x7fc9381019d0 2024-04-27T04:02:22.952 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:02:22.951+0000 7fc93659c700 1 --2- 172.21.15.181:0/341183125 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc9381019d0 0x7fc93810e0d0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fc92c00b820 tx=0x7fc92c009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:02:22.953 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:02:22.952+0000 7fc926ffd700 1 -- 172.21.15.181:0/341183125 <== mon.0 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fc92c0093c0 con 0x7fc9381019d0 2024-04-27T04:02:22.953 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:02:22.952+0000 7fc93cc8c700 1 -- 172.21.15.181:0/341183125 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc93810e810 con 0x7fc9381019d0 2024-04-27T04:02:22.953 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:02:22.952+0000 7fc93cc8c700 1 -- 172.21.15.181:0/341183125 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc93810ed00 con 0x7fc9381019d0 2024-04-27T04:02:22.953 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:02:22.952+0000 7fc926ffd700 1 -- 172.21.15.181:0/341183125 <== mon.0 v2:172.21.15.162:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fc92c013420 con 0x7fc9381019d0 2024-04-27T04:02:22.953 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:02:22.952+0000 7fc926ffd700 1 -- 172.21.15.181:0/341183125 <== mon.0 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fc92c0255c0 con 0x7fc9381019d0 2024-04-27T04:02:22.953 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:02:22.953+0000 7fc926ffd700 1 -- 172.21.15.181:0/341183125 <== mon.0 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 17) v1 ==== 43491+0+0 (secure 0 0 0) 0x7fc92c01c3f0 con 0x7fc9381019d0 2024-04-27T04:02:22.953 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:02:22.953+0000 7fc93cc8c700 1 -- 172.21.15.181:0/341183125 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc93810ef40 con 0x7fc9381019d0 2024-04-27T04:02:22.953 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:02:22.953+0000 7fc926ffd700 1 --2- 172.21.15.181:0/341183125 >> [v2:172.21.15.162:6800/4079782599,v1:172.21.15.162:6801/4079782599] conn(0x7fc920036970 0x7fc920038e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:02:22.954 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:02:22.953+0000 7fc926ffd700 1 -- 172.21.15.181:0/341183125 <== mon.0 v2:172.21.15.162:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1366+0+0 (secure 0 0 0) 0x7fc92c049b50 con 0x7fc9381019d0 2024-04-27T04:02:22.954 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:02:22.953+0000 7fc935d9b700 1 --2- 172.21.15.181:0/341183125 >> [v2:172.21.15.162:6800/4079782599,v1:172.21.15.162:6801/4079782599] conn(0x7fc920036970 0x7fc920038e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:02:22.954 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:02:22.953+0000 7fc935d9b700 1 --2- 172.21.15.181:0/341183125 >> [v2:172.21.15.162:6800/4079782599,v1:172.21.15.162:6801/4079782599] conn(0x7fc920036970 0x7fc920038e20 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fc928009990 tx=0x7fc928006e40 comp rx=0 tx=0).ready entity=mgr.14210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:02:22.957 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:02:22.958+0000 7fc926ffd700 1 -- 172.21.15.181:0/341183125 <== mon.0 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7fc93810ef40 con 0x7fc9381019d0 2024-04-27T04:02:23.237 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:02:23.237+0000 7fc93cc8c700 1 -- 172.21.15.181:0/341183125 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7fc93810ef40 con 0x7fc9381019d0 2024-04-27T04:02:23.238 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:02:23.238+0000 7fc926ffd700 1 -- 172.21.15.181:0/341183125 <== mon.0 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+743 (secure 0 0 0) 0x7fc93810ef40 con 0x7fc9381019d0 2024-04-27T04:02:23.238 INFO:teuthology.orchestra.run.smithi181.stdout: 2024-04-27T04:02:23.238 INFO:teuthology.orchestra.run.smithi181.stdout:{"epoch":1,"fsid":"7f9750d6-044a-11ef-bc93-c7b262605968","modified":"2024-04-27T04:00:03.626312Z","created":"2024-04-27T04:00:03.626312Z","min_mon_release":17,"min_mon_release_name":"quincy","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy"],"optional":[]},"mons":[{"rank":0,"name":"smithi162","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:3300","nonce":0},{"type":"v1","addr":"172.21.15.162:6789","nonce":0}]},"addr":"172.21.15.162:6789/0","public_addr":"172.21.15.162:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-04-27T04:02:23.240 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:02:23.240+0000 7fc93cc8c700 1 -- 172.21.15.181:0/341183125 >> [v2:172.21.15.162:6800/4079782599,v1:172.21.15.162:6801/4079782599] conn(0x7fc920036970 msgr2=0x7fc920038e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:02:23.240 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:02:23.240+0000 7fc93cc8c700 1 --2- 172.21.15.181:0/341183125 >> [v2:172.21.15.162:6800/4079782599,v1:172.21.15.162:6801/4079782599] conn(0x7fc920036970 0x7fc920038e20 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fc928009990 tx=0x7fc928006e40 comp rx=0 tx=0).stop 2024-04-27T04:02:23.240 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:02:23.240+0000 7fc93cc8c700 1 -- 172.21.15.181:0/341183125 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc9381019d0 msgr2=0x7fc93810e0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:02:23.241 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:02:23.240+0000 7fc93cc8c700 1 --2- 172.21.15.181:0/341183125 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc9381019d0 0x7fc93810e0d0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fc92c00b820 tx=0x7fc92c009e30 comp rx=0 tx=0).stop 2024-04-27T04:02:23.241 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:02:23.240+0000 7fc93cc8c700 1 -- 172.21.15.181:0/341183125 shutdown_connections 2024-04-27T04:02:23.241 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:02:23.240+0000 7fc93cc8c700 1 --2- 172.21.15.181:0/341183125 >> [v2:172.21.15.162:6800/4079782599,v1:172.21.15.162:6801/4079782599] conn(0x7fc920036970 0x7fc920038e20 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:02:23.241 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:02:23.240+0000 7fc93cc8c700 1 --2- 172.21.15.181:0/341183125 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc9381019d0 0x7fc93810e0d0 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:02:23.241 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:02:23.240+0000 7fc93cc8c700 1 -- 172.21.15.181:0/341183125 >> 172.21.15.181:0/341183125 conn(0x7fc938071800 msgr2=0x7fc938103510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:02:23.241 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:02:23.240+0000 7fc93cc8c700 1 -- 172.21.15.181:0/341183125 shutdown_connections 2024-04-27T04:02:23.241 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:02:23.240+0000 7fc93cc8c700 1 -- 172.21.15.181:0/341183125 wait complete. 2024-04-27T04:02:23.241 INFO:teuthology.orchestra.run.smithi181.stderr:dumped monmap epoch 1 2024-04-27T04:02:23.389 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:02:22 smithi162 ceph-mon[26841]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' 2024-04-27T04:02:23.389 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:02:22 smithi162 ceph-mon[26841]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' 2024-04-27T04:02:23.389 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:02:22 smithi162 ceph-mon[26841]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' 2024-04-27T04:02:23.389 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:02:22 smithi162 ceph-mon[26841]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' 2024-04-27T04:02:23.389 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:02:22 smithi162 ceph-mon[26841]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "config rm", "who": "osd/host:smithi181", "name": "osd_memory_target"}]: dispatch 2024-04-27T04:02:23.389 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:02:22 smithi162 ceph-mon[26841]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T04:02:23.389 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:02:22 smithi162 ceph-mon[26841]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T04:02:23.389 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:02:22 smithi162 ceph-mon[26841]: Updating smithi162:/etc/ceph/ceph.conf 2024-04-27T04:02:23.390 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:02:22 smithi162 ceph-mon[26841]: Updating smithi181:/etc/ceph/ceph.conf 2024-04-27T04:02:23.390 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:02:22 smithi162 ceph-mon[26841]: Updating smithi162:/var/lib/ceph/7f9750d6-044a-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-27T04:02:23.390 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:02:22 smithi162 ceph-mon[26841]: Updating smithi181:/var/lib/ceph/7f9750d6-044a-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-27T04:02:23.390 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:02:22 smithi162 ceph-mon[26841]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' 2024-04-27T04:02:23.390 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:02:22 smithi162 ceph-mon[26841]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' 2024-04-27T04:02:23.390 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:02:22 smithi162 ceph-mon[26841]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' 2024-04-27T04:02:23.390 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:02:22 smithi162 ceph-mon[26841]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' 2024-04-27T04:02:23.390 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:02:22 smithi162 ceph-mon[26841]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' 2024-04-27T04:02:23.390 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:02:22 smithi162 ceph-mon[26841]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi181", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-04-27T04:02:23.390 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:02:22 smithi162 ceph-mon[26841]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' cmd='[{"prefix": "auth get-or-create", "entity": "client.crash.smithi181", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]': finished 2024-04-27T04:02:23.390 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:02:22 smithi162 ceph-mon[26841]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T04:02:24.388 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:02:23 smithi162 ceph-mon[26841]: Updating smithi162:/etc/ceph/ceph.client.admin.keyring 2024-04-27T04:02:24.389 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:02:23 smithi162 ceph-mon[26841]: Updating smithi181:/etc/ceph/ceph.client.admin.keyring 2024-04-27T04:02:24.389 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:02:23 smithi162 ceph-mon[26841]: Updating smithi162:/var/lib/ceph/7f9750d6-044a-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-27T04:02:24.389 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:02:23 smithi162 ceph-mon[26841]: Updating smithi181:/var/lib/ceph/7f9750d6-044a-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-27T04:02:24.389 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:02:23 smithi162 ceph-mon[26841]: Deploying daemon crash.smithi181 on smithi181 2024-04-27T04:02:24.389 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:02:23 smithi162 ceph-mon[26841]: from='client.? 172.21.15.181:0/341183125' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-27T04:02:24.728 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-04-27T04:02:24.728 DEBUG:teuthology.orchestra.run.smithi181:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 7f9750d6-044a-11ef-bc93-c7b262605968 -- ceph mon dump -f json 2024-04-27T04:02:24.978 INFO:teuthology.orchestra.run.smithi181.stderr:Inferring config /var/lib/ceph/7f9750d6-044a-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-27T04:02:27.105 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:02:27.104+0000 7f85a6edb700 1 -- 172.21.15.181:0/1824468876 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f85a00fb280 msgr2=0x7f85a00fb650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:02:27.105 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:02:27.104+0000 7f85a6edb700 1 --2- 172.21.15.181:0/1824468876 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f85a00fb280 0x7f85a00fb650 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f85940048c0 tx=0x7f8594009e30 comp rx=0 tx=0).stop 2024-04-27T04:02:27.106 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:02:27.104+0000 7f85a6edb700 1 -- 172.21.15.181:0/1824468876 shutdown_connections 2024-04-27T04:02:27.106 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:02:27.104+0000 7f85a6edb700 1 --2- 172.21.15.181:0/1824468876 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f85a00fb280 0x7f85a00fb650 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:02:27.106 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:02:27.104+0000 7f85a6edb700 1 -- 172.21.15.181:0/1824468876 >> 172.21.15.181:0/1824468876 conn(0x7f85a0073090 msgr2=0x7f85a0073490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:02:27.106 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:02:27.104+0000 7f85a6edb700 1 -- 172.21.15.181:0/1824468876 shutdown_connections 2024-04-27T04:02:27.106 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:02:27.104+0000 7f85a6edb700 1 -- 172.21.15.181:0/1824468876 wait complete. 2024-04-27T04:02:27.106 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:02:27.104+0000 7f85a6edb700 1 Processor -- start 2024-04-27T04:02:27.106 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:02:27.104+0000 7f85a6edb700 1 -- start start 2024-04-27T04:02:27.106 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:02:27.105+0000 7f85a6edb700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f85a00fb280 0x7f85a0119740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:02:27.106 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:02:27.105+0000 7f85a6edb700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85a0119c80 con 0x7f85a00fb280 2024-04-27T04:02:27.106 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:02:27.105+0000 7f85a4c77700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f85a00fb280 0x7f85a0119740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:02:27.106 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:02:27.105+0000 7f85a4c77700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f85a00fb280 0x7f85a0119740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.181:56822/0 (socket says 172.21.15.181:56822) 2024-04-27T04:02:27.106 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:02:27.105+0000 7f85a4c77700 1 -- 172.21.15.181:0/1182641356 learned_addr learned my addr 172.21.15.181:0/1182641356 (peer_addr_for_me v2:172.21.15.181:0/0) 2024-04-27T04:02:27.106 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:02:27.105+0000 7f85a4c77700 1 -- 172.21.15.181:0/1182641356 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8594005040 con 0x7f85a00fb280 2024-04-27T04:02:27.107 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:02:27.105+0000 7f85a4c77700 1 --2- 172.21.15.181:0/1182641356 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f85a00fb280 0x7f85a0119740 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f859400b820 tx=0x7f8594009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:02:27.107 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:02:27.106+0000 7f859d7fa700 1 -- 172.21.15.181:0/1182641356 <== mon.0 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f85940093c0 con 0x7f85a00fb280 2024-04-27T04:02:27.107 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:02:27.106+0000 7f85a6edb700 1 -- 172.21.15.181:0/1182641356 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f85a0119e80 con 0x7f85a00fb280 2024-04-27T04:02:27.107 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:02:27.106+0000 7f859d7fa700 1 -- 172.21.15.181:0/1182641356 <== mon.0 v2:172.21.15.162:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f8594013420 con 0x7f85a00fb280 2024-04-27T04:02:27.107 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:02:27.106+0000 7f85a6edb700 1 -- 172.21.15.181:0/1182641356 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f85a011a2f0 con 0x7f85a00fb280 2024-04-27T04:02:27.107 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:02:27.106+0000 7f859d7fa700 1 -- 172.21.15.181:0/1182641356 <== mon.0 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f8594025560 con 0x7f85a00fb280 2024-04-27T04:02:27.108 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:02:27.107+0000 7f859d7fa700 1 -- 172.21.15.181:0/1182641356 <== mon.0 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 17) v1 ==== 43491+0+0 (secure 0 0 0) 0x7f8594013a90 con 0x7f85a00fb280 2024-04-27T04:02:27.108 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:02:27.106+0000 7f85a6edb700 1 -- 172.21.15.181:0/1182641356 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f85a00fc040 con 0x7f85a00fb280 2024-04-27T04:02:27.108 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:02:27.107+0000 7f859d7fa700 1 --2- 172.21.15.181:0/1182641356 >> [v2:172.21.15.162:6800/4079782599,v1:172.21.15.162:6801/4079782599] conn(0x7f8590036970 0x7f8590038e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:02:27.108 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:02:27.107+0000 7f859d7fa700 1 -- 172.21.15.181:0/1182641356 <== mon.0 v2:172.21.15.162:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1366+0+0 (secure 0 0 0) 0x7f8594049a30 con 0x7f85a00fb280 2024-04-27T04:02:27.108 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:02:27.107+0000 7f859ffff700 1 --2- 172.21.15.181:0/1182641356 >> [v2:172.21.15.162:6800/4079782599,v1:172.21.15.162:6801/4079782599] conn(0x7f8590036970 0x7f8590038e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:02:27.108 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:02:27.107+0000 7f859ffff700 1 --2- 172.21.15.181:0/1182641356 >> [v2:172.21.15.162:6800/4079782599,v1:172.21.15.162:6801/4079782599] conn(0x7f8590036970 0x7f8590038e20 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f858c009990 tx=0x7f858c006e40 comp rx=0 tx=0).ready entity=mgr.14210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:02:27.111 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:02:27.112+0000 7f859d7fa700 1 -- 172.21.15.181:0/1182641356 <== mon.0 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f8594023300 con 0x7f85a00fb280 2024-04-27T04:02:27.409 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:02:27.408+0000 7f85a6edb700 1 -- 172.21.15.181:0/1182641356 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f85a010d420 con 0x7f85a00fb280 2024-04-27T04:02:27.409 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:02:27.408+0000 7f859d7fa700 1 -- 172.21.15.181:0/1182641356 <== mon.0 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+743 (secure 0 0 0) 0x7f8594011b60 con 0x7f85a00fb280 2024-04-27T04:02:27.409 INFO:teuthology.orchestra.run.smithi181.stdout: 2024-04-27T04:02:27.409 INFO:teuthology.orchestra.run.smithi181.stdout:{"epoch":1,"fsid":"7f9750d6-044a-11ef-bc93-c7b262605968","modified":"2024-04-27T04:00:03.626312Z","created":"2024-04-27T04:00:03.626312Z","min_mon_release":17,"min_mon_release_name":"quincy","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy"],"optional":[]},"mons":[{"rank":0,"name":"smithi162","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:3300","nonce":0},{"type":"v1","addr":"172.21.15.162:6789","nonce":0}]},"addr":"172.21.15.162:6789/0","public_addr":"172.21.15.162:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-04-27T04:02:27.411 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:02:27.411+0000 7f85a6edb700 1 -- 172.21.15.181:0/1182641356 >> [v2:172.21.15.162:6800/4079782599,v1:172.21.15.162:6801/4079782599] conn(0x7f8590036970 msgr2=0x7f8590038e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:02:27.411 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:02:27.411+0000 7f85a6edb700 1 --2- 172.21.15.181:0/1182641356 >> [v2:172.21.15.162:6800/4079782599,v1:172.21.15.162:6801/4079782599] conn(0x7f8590036970 0x7f8590038e20 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f858c009990 tx=0x7f858c006e40 comp rx=0 tx=0).stop 2024-04-27T04:02:27.411 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:02:27.411+0000 7f85a6edb700 1 -- 172.21.15.181:0/1182641356 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f85a00fb280 msgr2=0x7f85a0119740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:02:27.411 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:02:27.411+0000 7f85a6edb700 1 --2- 172.21.15.181:0/1182641356 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f85a00fb280 0x7f85a0119740 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f859400b820 tx=0x7f8594009e30 comp rx=0 tx=0).stop 2024-04-27T04:02:27.411 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:02:27.411+0000 7f85a6edb700 1 -- 172.21.15.181:0/1182641356 shutdown_connections 2024-04-27T04:02:27.411 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:02:27.411+0000 7f85a6edb700 1 --2- 172.21.15.181:0/1182641356 >> [v2:172.21.15.162:6800/4079782599,v1:172.21.15.162:6801/4079782599] conn(0x7f8590036970 0x7f8590038e20 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:02:27.411 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:02:27.411+0000 7f85a6edb700 1 --2- 172.21.15.181:0/1182641356 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f85a00fb280 0x7f85a0119740 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:02:27.411 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:02:27.411+0000 7f85a6edb700 1 -- 172.21.15.181:0/1182641356 >> 172.21.15.181:0/1182641356 conn(0x7f85a0073090 msgr2=0x7f85a00f9f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:02:27.411 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:02:27.411+0000 7f85a6edb700 1 -- 172.21.15.181:0/1182641356 shutdown_connections 2024-04-27T04:02:27.411 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:02:27.411+0000 7f85a6edb700 1 -- 172.21.15.181:0/1182641356 wait complete. 2024-04-27T04:02:27.412 INFO:teuthology.orchestra.run.smithi181.stderr:dumped monmap epoch 1 2024-04-27T04:02:27.993 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:02:27 smithi162 ceph-mon[26841]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' 2024-04-27T04:02:27.993 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:02:27 smithi162 ceph-mon[26841]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' 2024-04-27T04:02:27.993 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:02:27 smithi162 ceph-mon[26841]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' 2024-04-27T04:02:27.993 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:02:27 smithi162 ceph-mon[26841]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' 2024-04-27T04:02:27.993 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:02:27 smithi162 ceph-mon[26841]: Deploying daemon node-exporter.smithi181 on smithi181 2024-04-27T04:02:27.993 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:02:27 smithi162 ceph-mon[26841]: from='client.? 172.21.15.181:0/1182641356' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-27T04:02:29.001 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-04-27T04:02:29.002 DEBUG:teuthology.orchestra.run.smithi181:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 7f9750d6-044a-11ef-bc93-c7b262605968 -- ceph mon dump -f json 2024-04-27T04:02:29.617 INFO:teuthology.orchestra.run.smithi181.stderr:Inferring config /var/lib/ceph/7f9750d6-044a-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-27T04:02:31.618 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:02:31.617+0000 7f114c9db700 1 -- 172.21.15.181:0/376751458 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1148102a90 msgr2=0x7f1148102e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:02:31.619 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:02:31.617+0000 7f114c9db700 1 --2- 172.21.15.181:0/376751458 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1148102a90 0x7f1148102e60 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f113c0048c0 tx=0x7f113c009e30 comp rx=0 tx=0).stop 2024-04-27T04:02:31.619 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:02:31.617+0000 7f114c9db700 1 -- 172.21.15.181:0/376751458 shutdown_connections 2024-04-27T04:02:31.619 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:02:31.617+0000 7f114c9db700 1 --2- 172.21.15.181:0/376751458 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1148102a90 0x7f1148102e60 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:02:31.619 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:02:31.617+0000 7f114c9db700 1 -- 172.21.15.181:0/376751458 >> 172.21.15.181:0/376751458 conn(0x7f11480f85b0 msgr2=0x7f11480fa9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:02:31.619 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:02:31.617+0000 7f114c9db700 1 -- 172.21.15.181:0/376751458 shutdown_connections 2024-04-27T04:02:31.619 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:02:31.618+0000 7f114c9db700 1 -- 172.21.15.181:0/376751458 wait complete. 2024-04-27T04:02:31.619 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:02:31.618+0000 7f114c9db700 1 Processor -- start 2024-04-27T04:02:31.619 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:02:31.618+0000 7f114c9db700 1 -- start start 2024-04-27T04:02:31.620 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:02:31.618+0000 7f114c9db700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1148102a90 0x7f1148070280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:02:31.620 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:02:31.618+0000 7f114c9db700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1148072210 con 0x7f1148102a90 2024-04-27T04:02:31.620 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:02:31.619+0000 7f114659c700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1148102a90 0x7f1148070280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:02:31.620 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:02:31.619+0000 7f114659c700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1148102a90 0x7f1148070280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.181:54562/0 (socket says 172.21.15.181:54562) 2024-04-27T04:02:31.620 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:02:31.619+0000 7f114659c700 1 -- 172.21.15.181:0/544355557 learned_addr learned my addr 172.21.15.181:0/544355557 (peer_addr_for_me v2:172.21.15.181:0/0) 2024-04-27T04:02:31.620 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:02:31.619+0000 7f114659c700 1 -- 172.21.15.181:0/544355557 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f113c005040 con 0x7f1148102a90 2024-04-27T04:02:31.620 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:02:31.619+0000 7f114659c700 1 --2- 172.21.15.181:0/544355557 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1148102a90 0x7f1148070280 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f113c00b820 tx=0x7f113c009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:02:31.621 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:02:31.619+0000 7f1136ffd700 1 -- 172.21.15.181:0/544355557 <== mon.0 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f113c0093c0 con 0x7f1148102a90 2024-04-27T04:02:31.621 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:02:31.619+0000 7f1136ffd700 1 -- 172.21.15.181:0/544355557 <== mon.0 v2:172.21.15.162:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f113c013420 con 0x7f1148102a90 2024-04-27T04:02:31.622 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:02:31.619+0000 7f114c9db700 1 -- 172.21.15.181:0/544355557 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1148070820 con 0x7f1148102a90 2024-04-27T04:02:31.622 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:02:31.619+0000 7f1136ffd700 1 -- 172.21.15.181:0/544355557 <== mon.0 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f113c01c5b0 con 0x7f1148102a90 2024-04-27T04:02:31.622 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:02:31.619+0000 7f114c9db700 1 -- 172.21.15.181:0/544355557 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1148070d10 con 0x7f1148102a90 2024-04-27T04:02:31.622 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:02:31.620+0000 7f1136ffd700 1 -- 172.21.15.181:0/544355557 <== mon.0 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 17) v1 ==== 43491+0+0 (secure 0 0 0) 0x7f113c01c7f0 con 0x7f1148102a90 2024-04-27T04:02:31.622 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:02:31.620+0000 7f1136ffd700 1 --2- 172.21.15.181:0/544355557 >> [v2:172.21.15.162:6800/4079782599,v1:172.21.15.162:6801/4079782599] conn(0x7f11300369c0 0x7f1130038e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:02:31.622 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:02:31.620+0000 7f114c9db700 1 -- 172.21.15.181:0/544355557 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1148101d00 con 0x7f1148102a90 2024-04-27T04:02:31.622 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:02:31.620+0000 7f1136ffd700 1 -- 172.21.15.181:0/544355557 <== mon.0 v2:172.21.15.162:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1366+0+0 (secure 0 0 0) 0x7f113c049b50 con 0x7f1148102a90 2024-04-27T04:02:31.622 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:02:31.620+0000 7f1145d9b700 1 --2- 172.21.15.181:0/544355557 >> [v2:172.21.15.162:6800/4079782599,v1:172.21.15.162:6801/4079782599] conn(0x7f11300369c0 0x7f1130038e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:02:31.622 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:02:31.621+0000 7f1145d9b700 1 --2- 172.21.15.181:0/544355557 >> [v2:172.21.15.162:6800/4079782599,v1:172.21.15.162:6801/4079782599] conn(0x7f11300369c0 0x7f1130038e70 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f1138009990 tx=0x7f1138006e40 comp rx=0 tx=0).ready entity=mgr.14210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:02:31.625 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:02:31.625+0000 7f1136ffd700 1 -- 172.21.15.181:0/544355557 <== mon.0 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f113c023300 con 0x7f1148102a90 2024-04-27T04:02:31.895 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:02:31.895+0000 7f114c9db700 1 -- 172.21.15.181:0/544355557 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f11480fe190 con 0x7f1148102a90 2024-04-27T04:02:31.895 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:02:31.896+0000 7f1136ffd700 1 -- 172.21.15.181:0/544355557 <== mon.0 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+743 (secure 0 0 0) 0x7f113c011b60 con 0x7f1148102a90 2024-04-27T04:02:31.896 INFO:teuthology.orchestra.run.smithi181.stdout: 2024-04-27T04:02:31.896 INFO:teuthology.orchestra.run.smithi181.stdout:{"epoch":1,"fsid":"7f9750d6-044a-11ef-bc93-c7b262605968","modified":"2024-04-27T04:00:03.626312Z","created":"2024-04-27T04:00:03.626312Z","min_mon_release":17,"min_mon_release_name":"quincy","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy"],"optional":[]},"mons":[{"rank":0,"name":"smithi162","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:3300","nonce":0},{"type":"v1","addr":"172.21.15.162:6789","nonce":0}]},"addr":"172.21.15.162:6789/0","public_addr":"172.21.15.162:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-04-27T04:02:31.898 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:02:31.898+0000 7f114c9db700 1 -- 172.21.15.181:0/544355557 >> [v2:172.21.15.162:6800/4079782599,v1:172.21.15.162:6801/4079782599] conn(0x7f11300369c0 msgr2=0x7f1130038e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:02:31.898 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:02:31.898+0000 7f114c9db700 1 --2- 172.21.15.181:0/544355557 >> [v2:172.21.15.162:6800/4079782599,v1:172.21.15.162:6801/4079782599] conn(0x7f11300369c0 0x7f1130038e70 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f1138009990 tx=0x7f1138006e40 comp rx=0 tx=0).stop 2024-04-27T04:02:31.898 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:02:31.898+0000 7f114c9db700 1 -- 172.21.15.181:0/544355557 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1148102a90 msgr2=0x7f1148070280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:02:31.898 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:02:31.898+0000 7f114c9db700 1 --2- 172.21.15.181:0/544355557 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1148102a90 0x7f1148070280 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f113c00b820 tx=0x7f113c009e30 comp rx=0 tx=0).stop 2024-04-27T04:02:31.898 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:02:31.898+0000 7f114c9db700 1 -- 172.21.15.181:0/544355557 shutdown_connections 2024-04-27T04:02:31.898 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:02:31.898+0000 7f114c9db700 1 --2- 172.21.15.181:0/544355557 >> [v2:172.21.15.162:6800/4079782599,v1:172.21.15.162:6801/4079782599] conn(0x7f11300369c0 0x7f1130038e70 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:02:31.899 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:02:31.898+0000 7f114c9db700 1 --2- 172.21.15.181:0/544355557 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1148102a90 0x7f1148070280 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:02:31.899 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:02:31.898+0000 7f114c9db700 1 -- 172.21.15.181:0/544355557 >> 172.21.15.181:0/544355557 conn(0x7f11480f85b0 msgr2=0x7f11480f9c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:02:31.899 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:02:31.898+0000 7f114c9db700 1 -- 172.21.15.181:0/544355557 shutdown_connections 2024-04-27T04:02:31.899 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:02:31.898+0000 7f114c9db700 1 -- 172.21.15.181:0/544355557 wait complete. 2024-04-27T04:02:31.899 INFO:teuthology.orchestra.run.smithi181.stderr:dumped monmap epoch 1 2024-04-27T04:02:33.138 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:02:32 smithi162 ceph-mon[26841]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' 2024-04-27T04:02:33.139 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:02:32 smithi162 ceph-mon[26841]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-27T04:02:33.139 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:02:32 smithi162 ceph-mon[26841]: from='client.? 172.21.15.181:0/544355557' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-27T04:02:33.139 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:02:32 smithi162 ceph-mon[26841]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' 2024-04-27T04:02:33.139 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:02:32 smithi162 ceph-mon[26841]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' 2024-04-27T04:02:33.139 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:02:32 smithi162 ceph-mon[26841]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' 2024-04-27T04:02:33.139 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:02:32 smithi162 ceph-mon[26841]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' 2024-04-27T04:02:33.139 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:02:32 smithi162 ceph-mon[26841]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi181.vtwuhz", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-27T04:02:33.139 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:02:32 smithi162 ceph-mon[26841]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.smithi181.vtwuhz", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-04-27T04:02:33.139 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:02:32 smithi162 ceph-mon[26841]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-27T04:02:33.139 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:02:32 smithi162 ceph-mon[26841]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T04:02:33.139 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:02:32 smithi162 ceph-mon[26841]: Deploying daemon mgr.smithi181.vtwuhz on smithi181 2024-04-27T04:02:33.599 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-04-27T04:02:33.599 DEBUG:teuthology.orchestra.run.smithi181:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 7f9750d6-044a-11ef-bc93-c7b262605968 -- ceph mon dump -f json 2024-04-27T04:02:33.990 INFO:teuthology.orchestra.run.smithi181.stderr:Inferring config /var/lib/ceph/7f9750d6-044a-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-27T04:02:35.888 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:02:35.883+0000 7f8cfe4b5700 1 -- 172.21.15.181:0/1693037147 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8cf80fb890 msgr2=0x7f8cf80fbc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:02:35.888 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:02:35.883+0000 7f8cfe4b5700 1 --2- 172.21.15.181:0/1693037147 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8cf80fb890 0x7f8cf80fbc60 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f8ce80048c0 tx=0x7f8ce8009e30 comp rx=0 tx=0).stop 2024-04-27T04:02:35.888 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:02:35.883+0000 7f8cfe4b5700 1 -- 172.21.15.181:0/1693037147 shutdown_connections 2024-04-27T04:02:35.888 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:02:35.883+0000 7f8cfe4b5700 1 --2- 172.21.15.181:0/1693037147 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8cf80fb890 0x7f8cf80fbc60 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:02:35.888 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:02:35.883+0000 7f8cfe4b5700 1 -- 172.21.15.181:0/1693037147 >> 172.21.15.181:0/1693037147 conn(0x7f8cf80f7520 msgr2=0x7f8cf80f9930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:02:35.888 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:02:35.883+0000 7f8cfe4b5700 1 -- 172.21.15.181:0/1693037147 shutdown_connections 2024-04-27T04:02:35.888 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:02:35.883+0000 7f8cfe4b5700 1 -- 172.21.15.181:0/1693037147 wait complete. 2024-04-27T04:02:35.888 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:02:35.883+0000 7f8cfe4b5700 1 Processor -- start 2024-04-27T04:02:35.888 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:02:35.884+0000 7f8cfe4b5700 1 -- start start 2024-04-27T04:02:35.888 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:02:35.884+0000 7f8cfe4b5700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8cf80fb890 0x7f8cf8115320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:02:35.888 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:02:35.884+0000 7f8cfe4b5700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8cf8115860 con 0x7f8cf80fb890 2024-04-27T04:02:35.889 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:02:35.884+0000 7f8cf7fff700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8cf80fb890 0x7f8cf8115320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:02:35.889 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:02:35.884+0000 7f8cf7fff700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8cf80fb890 0x7f8cf8115320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.181:54590/0 (socket says 172.21.15.181:54590) 2024-04-27T04:02:35.889 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:02:35.884+0000 7f8cf7fff700 1 -- 172.21.15.181:0/3512072700 learned_addr learned my addr 172.21.15.181:0/3512072700 (peer_addr_for_me v2:172.21.15.181:0/0) 2024-04-27T04:02:35.889 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:02:35.885+0000 7f8cf7fff700 1 -- 172.21.15.181:0/3512072700 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ce8005040 con 0x7f8cf80fb890 2024-04-27T04:02:35.889 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:02:35.885+0000 7f8cf7fff700 1 --2- 172.21.15.181:0/3512072700 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8cf80fb890 0x7f8cf8115320 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f8ce800b7f0 tx=0x7f8ce8009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:02:35.889 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:02:35.885+0000 7f8cf4ff9700 1 -- 172.21.15.181:0/3512072700 <== mon.0 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f8ce8009370 con 0x7f8cf80fb890 2024-04-27T04:02:35.889 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:02:35.885+0000 7f8cf4ff9700 1 -- 172.21.15.181:0/3512072700 <== mon.0 v2:172.21.15.162:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f8ce8013420 con 0x7f8cf80fb890 2024-04-27T04:02:35.889 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:02:35.885+0000 7f8cfe4b5700 1 -- 172.21.15.181:0/3512072700 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8cf8115a60 con 0x7f8cf80fb890 2024-04-27T04:02:35.889 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:02:35.885+0000 7f8cf4ff9700 1 -- 172.21.15.181:0/3512072700 <== mon.0 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f8ce801c590 con 0x7f8cf80fb890 2024-04-27T04:02:35.889 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:02:35.885+0000 7f8cfe4b5700 1 -- 172.21.15.181:0/3512072700 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8cf8115f50 con 0x7f8cf80fb890 2024-04-27T04:02:35.889 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:02:35.886+0000 7f8cf4ff9700 1 -- 172.21.15.181:0/3512072700 <== mon.0 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 17) v1 ==== 43491+0+0 (secure 0 0 0) 0x7f8ce8026440 con 0x7f8cf80fb890 2024-04-27T04:02:35.889 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:02:35.887+0000 7f8cf4ff9700 1 --2- 172.21.15.181:0/3512072700 >> [v2:172.21.15.162:6800/4079782599,v1:172.21.15.162:6801/4079782599] conn(0x7f8ce00369c0 0x7f8ce0038e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:02:35.890 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:02:35.887+0000 7f8cf4ff9700 1 -- 172.21.15.181:0/3512072700 <== mon.0 v2:172.21.15.162:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1366+0+0 (secure 0 0 0) 0x7f8ce80494a0 con 0x7f8cf80fb890 2024-04-27T04:02:35.890 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:02:35.887+0000 7f8cfe4b5700 1 -- 172.21.15.181:0/3512072700 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8cf80664c0 con 0x7f8cf80fb890 2024-04-27T04:02:35.890 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:02:35.887+0000 7f8cf77fe700 1 --2- 172.21.15.181:0/3512072700 >> [v2:172.21.15.162:6800/4079782599,v1:172.21.15.162:6801/4079782599] conn(0x7f8ce00369c0 0x7f8ce0038e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:02:35.890 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:02:35.887+0000 7f8cf77fe700 1 --2- 172.21.15.181:0/3512072700 >> [v2:172.21.15.162:6800/4079782599,v1:172.21.15.162:6801/4079782599] conn(0x7f8ce00369c0 0x7f8ce0038e70 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f8cec009990 tx=0x7f8cec006e40 comp rx=0 tx=0).ready entity=mgr.14210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:02:35.891 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:02:35.892+0000 7f8cf4ff9700 1 -- 172.21.15.181:0/3512072700 <== mon.0 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f8ce8021070 con 0x7f8cf80fb890 2024-04-27T04:02:36.199 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:02:36.199+0000 7f8cfe4b5700 1 -- 172.21.15.181:0/3512072700 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f8cf8109000 con 0x7f8cf80fb890 2024-04-27T04:02:36.200 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:02:36.200+0000 7f8cf4ff9700 1 -- 172.21.15.181:0/3512072700 <== mon.0 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+743 (secure 0 0 0) 0x7f8ce80123c0 con 0x7f8cf80fb890 2024-04-27T04:02:36.200 INFO:teuthology.orchestra.run.smithi181.stdout: 2024-04-27T04:02:36.200 INFO:teuthology.orchestra.run.smithi181.stdout:{"epoch":1,"fsid":"7f9750d6-044a-11ef-bc93-c7b262605968","modified":"2024-04-27T04:00:03.626312Z","created":"2024-04-27T04:00:03.626312Z","min_mon_release":17,"min_mon_release_name":"quincy","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy"],"optional":[]},"mons":[{"rank":0,"name":"smithi162","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:3300","nonce":0},{"type":"v1","addr":"172.21.15.162:6789","nonce":0}]},"addr":"172.21.15.162:6789/0","public_addr":"172.21.15.162:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-04-27T04:02:36.202 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:02:36.202+0000 7f8cfe4b5700 1 -- 172.21.15.181:0/3512072700 >> [v2:172.21.15.162:6800/4079782599,v1:172.21.15.162:6801/4079782599] conn(0x7f8ce00369c0 msgr2=0x7f8ce0038e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:02:36.202 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:02:36.202+0000 7f8cfe4b5700 1 --2- 172.21.15.181:0/3512072700 >> [v2:172.21.15.162:6800/4079782599,v1:172.21.15.162:6801/4079782599] conn(0x7f8ce00369c0 0x7f8ce0038e70 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f8cec009990 tx=0x7f8cec006e40 comp rx=0 tx=0).stop 2024-04-27T04:02:36.202 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:02:36.202+0000 7f8cfe4b5700 1 -- 172.21.15.181:0/3512072700 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8cf80fb890 msgr2=0x7f8cf8115320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:02:36.202 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:02:36.202+0000 7f8cfe4b5700 1 --2- 172.21.15.181:0/3512072700 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8cf80fb890 0x7f8cf8115320 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f8ce800b7f0 tx=0x7f8ce8009e30 comp rx=0 tx=0).stop 2024-04-27T04:02:36.202 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:02:36.202+0000 7f8cfe4b5700 1 -- 172.21.15.181:0/3512072700 shutdown_connections 2024-04-27T04:02:36.202 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:02:36.202+0000 7f8cfe4b5700 1 --2- 172.21.15.181:0/3512072700 >> [v2:172.21.15.162:6800/4079782599,v1:172.21.15.162:6801/4079782599] conn(0x7f8ce00369c0 0x7f8ce0038e70 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:02:36.203 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:02:36.202+0000 7f8cfe4b5700 1 --2- 172.21.15.181:0/3512072700 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8cf80fb890 0x7f8cf8115320 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:02:36.203 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:02:36.202+0000 7f8cfe4b5700 1 -- 172.21.15.181:0/3512072700 >> 172.21.15.181:0/3512072700 conn(0x7f8cf80f7520 msgr2=0x7f8cf80f7c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:02:36.203 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:02:36.202+0000 7f8cfe4b5700 1 -- 172.21.15.181:0/3512072700 shutdown_connections 2024-04-27T04:02:36.203 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:02:36.202+0000 7f8cfe4b5700 1 -- 172.21.15.181:0/3512072700 wait complete. 2024-04-27T04:02:36.203 INFO:teuthology.orchestra.run.smithi181.stderr:dumped monmap epoch 1 2024-04-27T04:02:37.138 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:02:36 smithi162 ceph-mon[26841]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' 2024-04-27T04:02:37.139 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:02:36 smithi162 ceph-mon[26841]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' 2024-04-27T04:02:37.139 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:02:36 smithi162 ceph-mon[26841]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' 2024-04-27T04:02:37.139 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:02:36 smithi162 ceph-mon[26841]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' 2024-04-27T04:02:37.139 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:02:36 smithi162 ceph-mon[26841]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-27T04:02:37.139 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:02:36 smithi162 ceph-mon[26841]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T04:02:37.139 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:02:36 smithi162 ceph-mon[26841]: Deploying daemon mon.smithi181 on smithi181 2024-04-27T04:02:37.139 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:02:36 smithi162 ceph-mon[26841]: from='client.? 172.21.15.181:0/3512072700' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-27T04:02:37.139 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:02:36 smithi162 ceph-mon[26841]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' 2024-04-27T04:02:38.138 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:02:37 smithi162 ceph-mon[26841]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-27T04:02:38.178 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-04-27T04:02:38.178 DEBUG:teuthology.orchestra.run.smithi181:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 7f9750d6-044a-11ef-bc93-c7b262605968 -- ceph mon dump -f json 2024-04-27T04:02:38.573 INFO:teuthology.orchestra.run.smithi181.stderr:Inferring config /var/lib/ceph/7f9750d6-044a-11ef-bc93-c7b262605968/mon.smithi181/config 2024-04-27T04:02:40.138 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:02:39 smithi162 ceph-mon[26841]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-27T04:02:40.215 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:02:40.213+0000 7fc86780b700 1 -- 172.21.15.181:0/249606278 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc860101ab0 msgr2=0x7fc860101e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:02:40.215 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:02:40.213+0000 7fc86780b700 1 --2- 172.21.15.181:0/249606278 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc860101ab0 0x7fc860101e80 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fc85c0048c0 tx=0x7fc85c009e30 comp rx=0 tx=0).stop 2024-04-27T04:02:40.215 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:02:40.213+0000 7fc86780b700 1 -- 172.21.15.181:0/249606278 shutdown_connections 2024-04-27T04:02:40.215 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:02:40.213+0000 7fc86780b700 1 --2- 172.21.15.181:0/249606278 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc860101ab0 0x7fc860101e80 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:02:40.215 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:02:40.213+0000 7fc86780b700 1 -- 172.21.15.181:0/249606278 >> 172.21.15.181:0/249606278 conn(0x7fc8600f75d0 msgr2=0x7fc8600f99e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:02:40.215 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:02:40.213+0000 7fc86780b700 1 -- 172.21.15.181:0/249606278 shutdown_connections 2024-04-27T04:02:40.215 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:02:40.213+0000 7fc86780b700 1 -- 172.21.15.181:0/249606278 wait complete. 2024-04-27T04:02:40.216 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:02:40.214+0000 7fc86780b700 1 Processor -- start 2024-04-27T04:02:40.216 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:02:40.214+0000 7fc86780b700 1 -- start start 2024-04-27T04:02:40.216 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:02:40.214+0000 7fc86780b700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc860101ab0 0x7fc860112130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:02:40.216 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:02:40.214+0000 7fc86780b700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc860112670 con 0x7fc860101ab0 2024-04-27T04:02:40.216 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:02:40.214+0000 7fc8655a7700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc860101ab0 0x7fc860112130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:02:40.216 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:02:40.214+0000 7fc8655a7700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc860101ab0 0x7fc860112130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.181:48374/0 (socket says 172.21.15.181:48374) 2024-04-27T04:02:40.216 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:02:40.214+0000 7fc8655a7700 1 -- 172.21.15.181:0/1286090148 learned_addr learned my addr 172.21.15.181:0/1286090148 (peer_addr_for_me v2:172.21.15.181:0/0) 2024-04-27T04:02:40.216 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:02:40.215+0000 7fc8655a7700 1 -- 172.21.15.181:0/1286090148 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc85c005040 con 0x7fc860101ab0 2024-04-27T04:02:40.216 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:02:40.215+0000 7fc8655a7700 1 --2- 172.21.15.181:0/1286090148 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc860101ab0 0x7fc860112130 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fc85c00b820 tx=0x7fc85c009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:02:40.216 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:02:40.215+0000 7fc855ffb700 1 -- 172.21.15.181:0/1286090148 <== mon.0 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fc85c0093c0 con 0x7fc860101ab0 2024-04-27T04:02:40.216 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:02:40.215+0000 7fc86780b700 1 -- 172.21.15.181:0/1286090148 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc860112870 con 0x7fc860101ab0 2024-04-27T04:02:40.216 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:02:40.215+0000 7fc855ffb700 1 -- 172.21.15.181:0/1286090148 <== mon.0 v2:172.21.15.162:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fc85c013420 con 0x7fc860101ab0 2024-04-27T04:02:40.216 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:02:40.215+0000 7fc855ffb700 1 -- 172.21.15.181:0/1286090148 <== mon.0 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fc85c01c5b0 con 0x7fc860101ab0 2024-04-27T04:02:40.216 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:02:40.215+0000 7fc86780b700 1 -- 172.21.15.181:0/1286090148 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc860112d60 con 0x7fc860101ab0 2024-04-27T04:02:40.217 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:02:40.216+0000 7fc86780b700 1 -- 172.21.15.181:0/1286090148 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc8600fd1b0 con 0x7fc860101ab0 2024-04-27T04:02:40.218 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:02:40.216+0000 7fc855ffb700 1 -- 172.21.15.181:0/1286090148 <== mon.0 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 17) v1 ==== 43491+0+0 (secure 0 0 0) 0x7fc85c013a90 con 0x7fc860101ab0 2024-04-27T04:02:40.218 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:02:40.217+0000 7fc855ffb700 1 --2- 172.21.15.181:0/1286090148 >> [v2:172.21.15.162:6800/4079782599,v1:172.21.15.162:6801/4079782599] conn(0x7fc84c0369c0 0x7fc84c038e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:02:40.218 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:02:40.217+0000 7fc855ffb700 1 -- 172.21.15.181:0/1286090148 <== mon.0 v2:172.21.15.162:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1366+0+0 (secure 0 0 0) 0x7fc85c049b10 con 0x7fc860101ab0 2024-04-27T04:02:40.218 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:02:40.217+0000 7fc864da6700 1 --2- 172.21.15.181:0/1286090148 >> [v2:172.21.15.162:6800/4079782599,v1:172.21.15.162:6801/4079782599] conn(0x7fc84c0369c0 0x7fc84c038e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:02:40.218 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:02:40.217+0000 7fc864da6700 1 --2- 172.21.15.181:0/1286090148 >> [v2:172.21.15.162:6800/4079782599,v1:172.21.15.162:6801/4079782599] conn(0x7fc84c0369c0 0x7fc84c038e70 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fc850009990 tx=0x7fc850006e40 comp rx=0 tx=0).ready entity=mgr.14210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:02:40.221 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:02:40.222+0000 7fc855ffb700 1 -- 172.21.15.181:0/1286090148 <== mon.0 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7fc85c0098a0 con 0x7fc860101ab0 2024-04-27T04:02:40.498 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:02:40.498+0000 7fc86780b700 1 -- 172.21.15.181:0/1286090148 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7fc86005f5c0 con 0x7fc860101ab0 2024-04-27T04:02:40.499 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:02:40.499+0000 7fc855ffb700 1 -- 172.21.15.181:0/1286090148 <== mon.0 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+743 (secure 0 0 0) 0x7fc85c011370 con 0x7fc860101ab0 2024-04-27T04:02:40.499 INFO:teuthology.orchestra.run.smithi181.stdout: 2024-04-27T04:02:40.499 INFO:teuthology.orchestra.run.smithi181.stdout:{"epoch":1,"fsid":"7f9750d6-044a-11ef-bc93-c7b262605968","modified":"2024-04-27T04:00:03.626312Z","created":"2024-04-27T04:00:03.626312Z","min_mon_release":17,"min_mon_release_name":"quincy","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy"],"optional":[]},"mons":[{"rank":0,"name":"smithi162","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:3300","nonce":0},{"type":"v1","addr":"172.21.15.162:6789","nonce":0}]},"addr":"172.21.15.162:6789/0","public_addr":"172.21.15.162:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-04-27T04:02:40.501 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:02:40.501+0000 7fc86780b700 1 -- 172.21.15.181:0/1286090148 >> [v2:172.21.15.162:6800/4079782599,v1:172.21.15.162:6801/4079782599] conn(0x7fc84c0369c0 msgr2=0x7fc84c038e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:02:40.501 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:02:40.501+0000 7fc86780b700 1 --2- 172.21.15.181:0/1286090148 >> [v2:172.21.15.162:6800/4079782599,v1:172.21.15.162:6801/4079782599] conn(0x7fc84c0369c0 0x7fc84c038e70 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fc850009990 tx=0x7fc850006e40 comp rx=0 tx=0).stop 2024-04-27T04:02:40.501 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:02:40.501+0000 7fc86780b700 1 -- 172.21.15.181:0/1286090148 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc860101ab0 msgr2=0x7fc860112130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:02:40.501 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:02:40.501+0000 7fc86780b700 1 --2- 172.21.15.181:0/1286090148 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc860101ab0 0x7fc860112130 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fc85c00b820 tx=0x7fc85c009e30 comp rx=0 tx=0).stop 2024-04-27T04:02:40.501 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:02:40.501+0000 7fc86780b700 1 -- 172.21.15.181:0/1286090148 shutdown_connections 2024-04-27T04:02:40.501 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:02:40.501+0000 7fc86780b700 1 --2- 172.21.15.181:0/1286090148 >> [v2:172.21.15.162:6800/4079782599,v1:172.21.15.162:6801/4079782599] conn(0x7fc84c0369c0 0x7fc84c038e70 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:02:40.501 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:02:40.501+0000 7fc86780b700 1 --2- 172.21.15.181:0/1286090148 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc860101ab0 0x7fc860112130 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:02:40.501 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:02:40.501+0000 7fc86780b700 1 -- 172.21.15.181:0/1286090148 >> 172.21.15.181:0/1286090148 conn(0x7fc8600f75d0 msgr2=0x7fc8600f8cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:02:40.501 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:02:40.501+0000 7fc86780b700 1 -- 172.21.15.181:0/1286090148 shutdown_connections 2024-04-27T04:02:40.502 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:02:40.501+0000 7fc86780b700 1 -- 172.21.15.181:0/1286090148 wait complete. 2024-04-27T04:02:40.502 INFO:teuthology.orchestra.run.smithi181.stderr:dumped monmap epoch 1 2024-04-27T04:02:42.374 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-04-27T04:02:42.378 DEBUG:teuthology.orchestra.run.smithi181:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 7f9750d6-044a-11ef-bc93-c7b262605968 -- ceph mon dump -f json 2024-04-27T04:02:42.625 INFO:teuthology.orchestra.run.smithi181.stderr:Inferring config /var/lib/ceph/7f9750d6-044a-11ef-bc93-c7b262605968/mon.smithi181/config 2024-04-27T04:02:46.139 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:02:45 smithi162 ceph-mon[26841]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "mon metadata", "id": "smithi162"}]: dispatch 2024-04-27T04:02:46.139 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:02:45 smithi162 ceph-mon[26841]: mon.smithi162 calling monitor election 2024-04-27T04:02:46.139 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:02:45 smithi162 ceph-mon[26841]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "mon metadata", "id": "smithi181"}]: dispatch 2024-04-27T04:02:46.139 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:02:45 smithi162 ceph-mon[26841]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-27T04:02:46.139 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:02:45 smithi162 ceph-mon[26841]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "mon metadata", "id": "smithi181"}]: dispatch 2024-04-27T04:02:46.139 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:02:45 smithi162 ceph-mon[26841]: from='mgr.? 172.21.15.181:0/637090075' entity='mgr.smithi181.vtwuhz' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi181.vtwuhz/crt"}]: dispatch 2024-04-27T04:02:46.139 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:02:45 smithi162 ceph-mon[26841]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "mon metadata", "id": "smithi181"}]: dispatch 2024-04-27T04:02:46.139 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:02:45 smithi162 ceph-mon[26841]: mon.smithi181 calling monitor election 2024-04-27T04:02:46.139 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:02:45 smithi162 ceph-mon[26841]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-27T04:02:46.139 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:02:45 smithi162 ceph-mon[26841]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "mon metadata", "id": "smithi181"}]: dispatch 2024-04-27T04:02:46.140 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:02:45 smithi162 ceph-mon[26841]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "mon metadata", "id": "smithi181"}]: dispatch 2024-04-27T04:02:46.140 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:02:45 smithi162 ceph-mon[26841]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-27T04:02:46.140 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:02:45 smithi162 ceph-mon[26841]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "mon metadata", "id": "smithi181"}]: dispatch 2024-04-27T04:02:46.140 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:02:45 smithi162 ceph-mon[26841]: mon.smithi162 is new leader, mons smithi162,smithi181 in quorum (ranks 0,1) 2024-04-27T04:02:46.140 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:02:45 smithi162 ceph-mon[26841]: monmap e2: 2 mons at {smithi162=[v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0],smithi181=[v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-04-27T04:02:46.140 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:02:45 smithi162 ceph-mon[26841]: fsmap 2024-04-27T04:02:46.140 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:02:45 smithi162 ceph-mon[26841]: osdmap e5: 0 total, 0 up, 0 in 2024-04-27T04:02:46.140 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:02:45 smithi162 ceph-mon[26841]: mgrmap e17: smithi162.oennfn(active, since 28s) 2024-04-27T04:02:46.140 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:02:45 smithi162 ceph-mon[26841]: overall HEALTH_OK 2024-04-27T04:02:46.140 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:02:45 smithi162 ceph-mon[26841]: Standby manager daemon smithi181.vtwuhz started 2024-04-27T04:02:46.140 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:02:45 smithi162 ceph-mon[26841]: from='mgr.? 172.21.15.181:0/637090075' entity='mgr.smithi181.vtwuhz' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-27T04:02:46.140 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:02:45 smithi162 ceph-mon[26841]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' 2024-04-27T04:02:46.140 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:02:45 smithi162 ceph-mon[26841]: from='mgr.? 172.21.15.181:0/637090075' entity='mgr.smithi181.vtwuhz' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi181.vtwuhz/key"}]: dispatch 2024-04-27T04:02:46.140 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:02:45 smithi162 ceph-mon[26841]: from='mgr.? 172.21.15.181:0/637090075' entity='mgr.smithi181.vtwuhz' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-27T04:02:46.141 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:02:45 smithi162 ceph-mon[26841]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' 2024-04-27T04:02:46.141 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:02:45 smithi162 ceph-mon[26841]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' 2024-04-27T04:02:46.141 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:02:45 smithi162 ceph-mon[26841]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' 2024-04-27T04:02:46.173 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:02:45 smithi181 ceph-mon[33774]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' 2024-04-27T04:02:47.064 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:02:46 smithi181 ceph-mon[33774]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' 2024-04-27T04:02:47.064 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:02:46 smithi181 ceph-mon[33774]: mgrmap e18: smithi162.oennfn(active, since 29s), standbys: smithi181.vtwuhz 2024-04-27T04:02:47.064 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:02:46 smithi181 ceph-mon[33774]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "mgr metadata", "who": "smithi181.vtwuhz", "id": "smithi181.vtwuhz"}]: dispatch 2024-04-27T04:02:47.065 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:02:46 smithi181 ceph-mon[33774]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T04:02:47.065 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:02:46 smithi181 ceph-mon[33774]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "mon metadata", "id": "smithi181"}]: dispatch 2024-04-27T04:02:47.065 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:02:46 smithi181 ceph-mon[33774]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-27T04:02:47.082 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:02:47.081+0000 7efce5d81700 1 -- 172.21.15.181:0/499112547 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7efccc003580 msgr2=0x7efccc005a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:02:47.082 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:02:47.081+0000 7efce5d81700 1 --2- 172.21.15.181:0/499112547 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7efccc003580 0x7efccc005a00 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7efce00fb9b0 tx=0x7efcd000b040 comp rx=0 tx=0).stop 2024-04-27T04:02:47.082 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:02:47.081+0000 7efce5d81700 1 -- 172.21.15.181:0/499112547 shutdown_connections 2024-04-27T04:02:47.082 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:02:47.081+0000 7efce5d81700 1 --2- 172.21.15.181:0/499112547 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7efccc003580 0x7efccc005a00 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:02:47.082 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:02:47.081+0000 7efce5d81700 1 -- 172.21.15.181:0/499112547 >> 172.21.15.181:0/499112547 conn(0x7efce0071970 msgr2=0x7efce0071d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:02:47.082 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:02:47.081+0000 7efce5d81700 1 -- 172.21.15.181:0/499112547 shutdown_connections 2024-04-27T04:02:47.082 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:02:47.081+0000 7efce5d81700 1 -- 172.21.15.181:0/499112547 wait complete. 2024-04-27T04:02:47.084 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:02:47.081+0000 7efce5d81700 1 Processor -- start 2024-04-27T04:02:47.084 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:02:47.082+0000 7efce5d81700 1 -- start start 2024-04-27T04:02:47.084 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:02:47.082+0000 7efce5d81700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7efccc003580 0x7efce00fd690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:02:47.084 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:02:47.082+0000 7efce5d81700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7efce00fdbd0 0x7efce00fe040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:02:47.084 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:02:47.082+0000 7efce5d81700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efce0119540 con 0x7efce00fdbd0 2024-04-27T04:02:47.084 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:02:47.082+0000 7efce5d81700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efce0119680 con 0x7efccc003580 2024-04-27T04:02:47.084 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:02:47.082+0000 7efcdf7fe700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7efccc003580 0x7efce00fd690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:02:47.084 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:02:47.082+0000 7efcdf7fe700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7efccc003580 0x7efce00fd690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:34852/0 (socket says 172.21.15.181:34852) 2024-04-27T04:02:47.084 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:02:47.082+0000 7efcdf7fe700 1 -- 172.21.15.181:0/2334466697 learned_addr learned my addr 172.21.15.181:0/2334466697 (peer_addr_for_me v2:172.21.15.181:0/0) 2024-04-27T04:02:47.084 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:02:47.082+0000 7efcdf7fe700 1 -- 172.21.15.181:0/2334466697 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7efccc003580 msgr2=0x7efce00fd690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 12 2024-04-27T04:02:47.084 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:02:47.082+0000 7efcd6ffd700 1 --2- 172.21.15.181:0/2334466697 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7efce00fdbd0 0x7efce00fe040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:02:47.085 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:02:47.082+0000 7efcdf7fe700 1 -- 172.21.15.181:0/2334466697 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7efccc003580 msgr2=0x7efce00fd690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-04-27T04:02:47.085 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:02:47.082+0000 7efcdf7fe700 1 --2- 172.21.15.181:0/2334466697 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7efccc003580 0x7efce00fd690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2024-04-27T04:02:47.085 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:02:47.082+0000 7efcdf7fe700 1 --2- 172.21.15.181:0/2334466697 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7efccc003580 0x7efce00fd690 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-27T04:02:47.085 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:02:47.082+0000 7efcd6ffd700 1 -- 172.21.15.181:0/2334466697 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7efccc003580 msgr2=0x7efce00fd690 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-27T04:02:47.085 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:02:47.082+0000 7efcd6ffd700 1 --2- 172.21.15.181:0/2334466697 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7efccc003580 0x7efce00fd690 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:02:47.085 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:02:47.082+0000 7efcd6ffd700 1 -- 172.21.15.181:0/2334466697 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efcd0005040 con 0x7efce00fdbd0 2024-04-27T04:02:47.085 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:02:47.082+0000 7efcd6ffd700 1 --2- 172.21.15.181:0/2334466697 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7efce00fdbd0 0x7efce00fe040 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7efcc800b7c0 tx=0x7efcc80091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:02:47.085 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:02:47.083+0000 7efcdcff9700 1 -- 172.21.15.181:0/2334466697 <== mon.0 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7efcc8011810 con 0x7efce00fdbd0 2024-04-27T04:02:47.085 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:02:47.083+0000 7efce5d81700 1 -- 172.21.15.181:0/2334466697 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efce0119880 con 0x7efce00fdbd0 2024-04-27T04:02:47.085 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:02:47.083+0000 7efcdcff9700 1 -- 172.21.15.181:0/2334466697 <== mon.0 v2:172.21.15.162:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7efcc8011e40 con 0x7efce00fdbd0 2024-04-27T04:02:47.085 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:02:47.083+0000 7efcdcff9700 1 -- 172.21.15.181:0/2334466697 <== mon.0 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7efcc800f2e0 con 0x7efce00fdbd0 2024-04-27T04:02:47.085 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:02:47.083+0000 7efce5d81700 1 -- 172.21.15.181:0/2334466697 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efce0119d70 con 0x7efce00fdbd0 2024-04-27T04:02:47.086 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:02:47.084+0000 7efce5d81700 1 -- 172.21.15.181:0/2334466697 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efce00fd1c0 con 0x7efce00fdbd0 2024-04-27T04:02:47.087 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:02:47.084+0000 7efcdcff9700 1 -- 172.21.15.181:0/2334466697 <== mon.0 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 18) v1 ==== 86575+0+0 (secure 0 0 0) 0x7efcc8010380 con 0x7efce00fdbd0 2024-04-27T04:02:47.087 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:02:47.084+0000 7efcdcff9700 1 --2- 172.21.15.181:0/2334466697 >> [v2:172.21.15.162:6800/4079782599,v1:172.21.15.162:6801/4079782599] conn(0x7efcc0069320 0x7efcc006b7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:02:47.087 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:02:47.085+0000 7efcdcff9700 1 -- 172.21.15.181:0/2334466697 <== mon.0 v2:172.21.15.162:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1366+0+0 (secure 0 0 0) 0x7efcc8086390 con 0x7efce00fdbd0 2024-04-27T04:02:47.087 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:02:47.085+0000 7efcdf7fe700 1 --2- 172.21.15.181:0/2334466697 >> [v2:172.21.15.162:6800/4079782599,v1:172.21.15.162:6801/4079782599] conn(0x7efcc0069320 0x7efcc006b7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:02:47.087 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:02:47.085+0000 7efcdf7fe700 1 --2- 172.21.15.181:0/2334466697 >> [v2:172.21.15.162:6800/4079782599,v1:172.21.15.162:6801/4079782599] conn(0x7efcc0069320 0x7efcc006b7d0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7efcd0007f50 tx=0x7efcd000b040 comp rx=0 tx=0).ready entity=mgr.14210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:02:47.090 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:02:47.090+0000 7efcdcff9700 1 -- 172.21.15.181:0/2334466697 <== mon.0 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7efcc80536c0 con 0x7efce00fdbd0 2024-04-27T04:02:47.139 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:02:46 smithi162 ceph-mon[26841]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' 2024-04-27T04:02:47.139 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:02:46 smithi162 ceph-mon[26841]: mgrmap e18: smithi162.oennfn(active, since 29s), standbys: smithi181.vtwuhz 2024-04-27T04:02:47.139 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:02:46 smithi162 ceph-mon[26841]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "mgr metadata", "who": "smithi181.vtwuhz", "id": "smithi181.vtwuhz"}]: dispatch 2024-04-27T04:02:47.139 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:02:46 smithi162 ceph-mon[26841]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T04:02:47.139 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:02:46 smithi162 ceph-mon[26841]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "mon metadata", "id": "smithi181"}]: dispatch 2024-04-27T04:02:47.140 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:02:46 smithi162 ceph-mon[26841]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-27T04:02:47.355 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:02:47.354+0000 7efce5d81700 1 -- 172.21.15.181:0/2334466697 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7efce005f5c0 con 0x7efce00fdbd0 2024-04-27T04:02:47.356 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:02:47.355+0000 7efcdcff9700 1 -- 172.21.15.181:0/2334466697 <== mon.0 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 2 v2) v1 ==== 95+0+1021 (secure 0 0 0) 0x7efce005f5c0 con 0x7efce00fdbd0 2024-04-27T04:02:47.356 INFO:teuthology.orchestra.run.smithi181.stdout: 2024-04-27T04:02:47.356 INFO:teuthology.orchestra.run.smithi181.stdout:{"epoch":2,"fsid":"7f9750d6-044a-11ef-bc93-c7b262605968","modified":"2024-04-27T04:02:40.633767Z","created":"2024-04-27T04:00:03.626312Z","min_mon_release":17,"min_mon_release_name":"quincy","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy"],"optional":[]},"mons":[{"rank":0,"name":"smithi162","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:3300","nonce":0},{"type":"v1","addr":"172.21.15.162:6789","nonce":0}]},"addr":"172.21.15.162:6789/0","public_addr":"172.21.15.162:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"smithi181","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:3300","nonce":0},{"type":"v1","addr":"172.21.15.181:6789","nonce":0}]},"addr":"172.21.15.181:6789/0","public_addr":"172.21.15.181:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-04-27T04:02:47.358 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:02:47.357+0000 7efce5d81700 1 -- 172.21.15.181:0/2334466697 >> [v2:172.21.15.162:6800/4079782599,v1:172.21.15.162:6801/4079782599] conn(0x7efcc0069320 msgr2=0x7efcc006b7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:02:47.359 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:02:47.357+0000 7efce5d81700 1 --2- 172.21.15.181:0/2334466697 >> [v2:172.21.15.162:6800/4079782599,v1:172.21.15.162:6801/4079782599] conn(0x7efcc0069320 0x7efcc006b7d0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7efcd0007f50 tx=0x7efcd000b040 comp rx=0 tx=0).stop 2024-04-27T04:02:47.359 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:02:47.357+0000 7efce5d81700 1 -- 172.21.15.181:0/2334466697 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7efce00fdbd0 msgr2=0x7efce00fe040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:02:47.359 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:02:47.357+0000 7efce5d81700 1 --2- 172.21.15.181:0/2334466697 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7efce00fdbd0 0x7efce00fe040 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7efcc800b7c0 tx=0x7efcc80091a0 comp rx=0 tx=0).stop 2024-04-27T04:02:47.359 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:02:47.357+0000 7efce5d81700 1 -- 172.21.15.181:0/2334466697 shutdown_connections 2024-04-27T04:02:47.359 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:02:47.357+0000 7efce5d81700 1 --2- 172.21.15.181:0/2334466697 >> [v2:172.21.15.162:6800/4079782599,v1:172.21.15.162:6801/4079782599] conn(0x7efcc0069320 0x7efcc006b7d0 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:02:47.359 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:02:47.357+0000 7efce5d81700 1 --2- 172.21.15.181:0/2334466697 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7efccc003580 0x7efce00fd690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:02:47.359 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:02:47.357+0000 7efce5d81700 1 --2- 172.21.15.181:0/2334466697 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7efce00fdbd0 0x7efce00fe040 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:02:47.359 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:02:47.357+0000 7efce5d81700 1 -- 172.21.15.181:0/2334466697 >> 172.21.15.181:0/2334466697 conn(0x7efce0071970 msgr2=0x7efce0065a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:02:47.359 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:02:47.357+0000 7efce5d81700 1 -- 172.21.15.181:0/2334466697 shutdown_connections 2024-04-27T04:02:47.359 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:02:47.357+0000 7efce5d81700 1 -- 172.21.15.181:0/2334466697 wait complete. 2024-04-27T04:02:47.360 INFO:teuthology.orchestra.run.smithi181.stderr:dumped monmap epoch 2 2024-04-27T04:02:47.977 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:02:47 smithi181 ceph-mon[33774]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-27T04:02:47.977 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:02:47 smithi181 ceph-mon[33774]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' 2024-04-27T04:02:47.977 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:02:47 smithi181 ceph-mon[33774]: from='client.? 172.21.15.181:0/2334466697' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-27T04:02:48.005 INFO:tasks.cephadm:Generating final ceph.conf file... 2024-04-27T04:02:48.005 DEBUG:teuthology.orchestra.run.smithi162:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 7f9750d6-044a-11ef-bc93-c7b262605968 -- ceph config generate-minimal-conf 2024-04-27T04:02:48.032 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:02:47 smithi162 ceph-mon[26841]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-27T04:02:48.032 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:02:47 smithi162 ceph-mon[26841]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' 2024-04-27T04:02:48.032 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:02:47 smithi162 ceph-mon[26841]: from='client.? 172.21.15.181:0/2334466697' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-27T04:02:48.286 INFO:teuthology.orchestra.run.smithi162.stderr:Inferring config /var/lib/ceph/7f9750d6-044a-11ef-bc93-c7b262605968/mon.smithi162/config 2024-04-27T04:02:49.423 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:02:49 smithi181 ceph-mon[33774]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' 2024-04-27T04:02:49.424 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:02:49 smithi181 ceph-mon[33774]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' 2024-04-27T04:02:49.424 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:02:49 smithi181 ceph-mon[33774]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T04:02:49.424 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:02:49 smithi181 ceph-mon[33774]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T04:02:49.424 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:02:49 smithi181 ceph-mon[33774]: Updating smithi162:/etc/ceph/ceph.conf 2024-04-27T04:02:49.424 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:02:49 smithi181 ceph-mon[33774]: Updating smithi181:/etc/ceph/ceph.conf 2024-04-27T04:02:49.424 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:02:49 smithi181 ceph-mon[33774]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' 2024-04-27T04:02:49.424 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:02:49 smithi181 ceph-mon[33774]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' 2024-04-27T04:02:49.424 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:02:49 smithi181 ceph-mon[33774]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' 2024-04-27T04:02:49.424 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:02:49 smithi181 ceph-mon[33774]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' 2024-04-27T04:02:49.424 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:02:49 smithi181 ceph-mon[33774]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' 2024-04-27T04:02:49.424 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:02:49 smithi181 ceph-mon[33774]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-27T04:02:49.425 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:02:49 smithi181 ceph-mon[33774]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-27T04:02:49.425 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:02:49 smithi181 ceph-mon[33774]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T04:02:49.469 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:02:49 smithi162 ceph-mon[26841]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' 2024-04-27T04:02:49.469 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:02:49 smithi162 ceph-mon[26841]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' 2024-04-27T04:02:49.469 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:02:49 smithi162 ceph-mon[26841]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T04:02:49.469 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:02:49 smithi162 ceph-mon[26841]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T04:02:49.470 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:02:49 smithi162 ceph-mon[26841]: Updating smithi162:/etc/ceph/ceph.conf 2024-04-27T04:02:49.470 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:02:49 smithi162 ceph-mon[26841]: Updating smithi181:/etc/ceph/ceph.conf 2024-04-27T04:02:49.470 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:02:49 smithi162 ceph-mon[26841]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' 2024-04-27T04:02:49.470 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:02:49 smithi162 ceph-mon[26841]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' 2024-04-27T04:02:49.470 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:02:49 smithi162 ceph-mon[26841]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' 2024-04-27T04:02:49.470 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:02:49 smithi162 ceph-mon[26841]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' 2024-04-27T04:02:49.470 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:02:49 smithi162 ceph-mon[26841]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' 2024-04-27T04:02:49.470 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:02:49 smithi162 ceph-mon[26841]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-27T04:02:49.471 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:02:49 smithi162 ceph-mon[26841]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-27T04:02:49.471 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:02:49 smithi162 ceph-mon[26841]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T04:02:50.319 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:02:50 smithi162 ceph-mon[26841]: Updating smithi162:/var/lib/ceph/7f9750d6-044a-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-27T04:02:50.319 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:02:50 smithi162 ceph-mon[26841]: Updating smithi181:/var/lib/ceph/7f9750d6-044a-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-27T04:02:50.319 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:02:50 smithi162 ceph-mon[26841]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-27T04:02:50.319 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:02:50 smithi162 ceph-mon[26841]: Reconfiguring mon.smithi162 (unknown last config time)... 2024-04-27T04:02:50.319 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:02:50 smithi162 ceph-mon[26841]: Reconfiguring daemon mon.smithi162 on smithi162 2024-04-27T04:02:50.556 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:02:50.555+0000 7f67fa231700 1 -- 172.21.15.162:0/1454963323 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f67f4102a50 msgr2=0x7f67f4102e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:02:50.556 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:02:50.555+0000 7f67fa231700 1 --2- 172.21.15.162:0/1454963323 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f67f4102a50 0x7f67f4102e20 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f67e80048c0 tx=0x7f67e8009e30 comp rx=0 tx=0).stop 2024-04-27T04:02:50.556 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:02:50.555+0000 7f67fa231700 1 -- 172.21.15.162:0/1454963323 shutdown_connections 2024-04-27T04:02:50.556 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:02:50.555+0000 7f67fa231700 1 --2- 172.21.15.162:0/1454963323 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f67f4102a50 0x7f67f4102e20 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:02:50.556 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:02:50.555+0000 7f67fa231700 1 -- 172.21.15.162:0/1454963323 >> 172.21.15.162:0/1454963323 conn(0x7f67f4073090 msgr2=0x7f67f4073490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:02:50.556 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:02:50.555+0000 7f67fa231700 1 -- 172.21.15.162:0/1454963323 shutdown_connections 2024-04-27T04:02:50.557 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:02:50.555+0000 7f67fa231700 1 -- 172.21.15.162:0/1454963323 wait complete. 2024-04-27T04:02:50.557 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:02:50.555+0000 7f67fa231700 1 Processor -- start 2024-04-27T04:02:50.557 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:02:50.555+0000 7f67fa231700 1 -- start start 2024-04-27T04:02:50.558 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:02:50.556+0000 7f67fa231700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f67f4102a50 0x7f67f40fc210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:02:50.558 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:02:50.556+0000 7f67fa231700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f67f40fc750 0x7f67f40fcbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:02:50.558 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:02:50.556+0000 7f67fa231700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67f4119540 con 0x7f67f4102a50 2024-04-27T04:02:50.558 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:02:50.556+0000 7f67fa231700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67f4119680 con 0x7f67f40fc750 2024-04-27T04:02:50.558 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:02:50.556+0000 7f67f37fe700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f67f4102a50 0x7f67f40fc210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:02:50.558 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:02:50.556+0000 7f67f37fe700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f67f4102a50 0x7f67f40fc210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:43158/0 (socket says 172.21.15.162:43158) 2024-04-27T04:02:50.559 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:02:50.556+0000 7f67f37fe700 1 -- 172.21.15.162:0/4083049265 learned_addr learned my addr 172.21.15.162:0/4083049265 (peer_addr_for_me v2:172.21.15.162:0/0) 2024-04-27T04:02:50.559 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:02:50.556+0000 7f67f37fe700 1 -- 172.21.15.162:0/4083049265 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f67f40fc750 msgr2=0x7f67f40fcbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:02:50.559 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:02:50.556+0000 7f67f37fe700 1 --2- 172.21.15.162:0/4083049265 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f67f40fc750 0x7f67f40fcbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:02:50.559 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:02:50.556+0000 7f67f37fe700 1 -- 172.21.15.162:0/4083049265 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67e8005040 con 0x7f67f4102a50 2024-04-27T04:02:50.559 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:02:50.556+0000 7f67f37fe700 1 --2- 172.21.15.162:0/4083049265 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f67f4102a50 0x7f67f40fc210 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f67e800b5c0 tx=0x7f67e8009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:02:50.559 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:02:50.556+0000 7f67dbfff700 1 -- 172.21.15.162:0/4083049265 <== mon.0 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f67e800e040 con 0x7f67f4102a50 2024-04-27T04:02:50.559 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:02:50.556+0000 7f67dbfff700 1 -- 172.21.15.162:0/4083049265 <== mon.0 v2:172.21.15.162:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f67e8009950 con 0x7f67f4102a50 2024-04-27T04:02:50.559 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:02:50.556+0000 7f67fa231700 1 -- 172.21.15.162:0/4083049265 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f67f4119820 con 0x7f67f4102a50 2024-04-27T04:02:50.559 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:02:50.556+0000 7f67dbfff700 1 -- 172.21.15.162:0/4083049265 <== mon.0 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f67e801c3f0 con 0x7f67f4102a50 2024-04-27T04:02:50.559 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:02:50.557+0000 7f67fa231700 1 -- 172.21.15.162:0/4083049265 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f67f4119de0 con 0x7f67f4102a50 2024-04-27T04:02:50.559 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:02:50.558+0000 7f67fa231700 1 -- 172.21.15.162:0/4083049265 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f67f4101d70 con 0x7f67f4102a50 2024-04-27T04:02:50.560 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:02:50.558+0000 7f67dbfff700 1 -- 172.21.15.162:0/4083049265 <== mon.0 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 18) v1 ==== 86575+0+0 (secure 0 0 0) 0x7f67e80137d0 con 0x7f67f4102a50 2024-04-27T04:02:50.560 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:02:50.558+0000 7f67dbfff700 1 --2- 172.21.15.162:0/4083049265 >> [v2:172.21.15.162:6800/4079782599,v1:172.21.15.162:6801/4079782599] conn(0x7f67dc069360 0x7f67dc06b810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:02:50.560 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:02:50.558+0000 7f67dbfff700 1 -- 172.21.15.162:0/4083049265 <== mon.0 v2:172.21.15.162:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1366+0+0 (secure 0 0 0) 0x7f67e8087460 con 0x7f67f4102a50 2024-04-27T04:02:50.560 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:02:50.558+0000 7f67f2ffd700 1 --2- 172.21.15.162:0/4083049265 >> [v2:172.21.15.162:6800/4079782599,v1:172.21.15.162:6801/4079782599] conn(0x7f67dc069360 0x7f67dc06b810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:02:50.560 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:02:50.559+0000 7f67f2ffd700 1 --2- 172.21.15.162:0/4083049265 >> [v2:172.21.15.162:6800/4079782599,v1:172.21.15.162:6801/4079782599] conn(0x7f67dc069360 0x7f67dc06b810 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f67e4006fd0 tx=0x7f67e4006dd0 comp rx=0 tx=0).ready entity=mgr.14210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:02:50.563 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:02:50.563+0000 7f67dbfff700 1 -- 172.21.15.162:0/4083049265 <== mon.0 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f67e8054870 con 0x7f67f4102a50 2024-04-27T04:02:50.673 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:02:50 smithi181 ceph-mon[33774]: Updating smithi162:/var/lib/ceph/7f9750d6-044a-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-27T04:02:50.673 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:02:50 smithi181 ceph-mon[33774]: Updating smithi181:/var/lib/ceph/7f9750d6-044a-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-27T04:02:50.674 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:02:50 smithi181 ceph-mon[33774]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-27T04:02:50.674 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:02:50 smithi181 ceph-mon[33774]: Reconfiguring mon.smithi162 (unknown last config time)... 2024-04-27T04:02:50.674 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:02:50 smithi181 ceph-mon[33774]: Reconfiguring daemon mon.smithi162 on smithi162 2024-04-27T04:02:50.754 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:02:50.754+0000 7f67fa231700 1 -- 172.21.15.162:0/4083049265 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) v1 -- 0x7f67f40fe350 con 0x7f67f4102a50 2024-04-27T04:02:50.755 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:02:50.754+0000 7f67dbfff700 1 -- 172.21.15.162:0/4083049265 <== mon.0 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v10) v1 ==== 76+0+227 (secure 0 0 0) 0x7f67e8086c10 con 0x7f67f4102a50 2024-04-27T04:02:50.755 INFO:teuthology.orchestra.run.smithi162.stdout:# minimal ceph.conf for 7f9750d6-044a-11ef-bc93-c7b262605968 2024-04-27T04:02:50.755 INFO:teuthology.orchestra.run.smithi162.stdout:[global] 2024-04-27T04:02:50.755 INFO:teuthology.orchestra.run.smithi162.stdout: fsid = 7f9750d6-044a-11ef-bc93-c7b262605968 2024-04-27T04:02:50.756 INFO:teuthology.orchestra.run.smithi162.stdout: mon_host = [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] 2024-04-27T04:02:50.758 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:02:50.757+0000 7f67fa231700 1 -- 172.21.15.162:0/4083049265 >> [v2:172.21.15.162:6800/4079782599,v1:172.21.15.162:6801/4079782599] conn(0x7f67dc069360 msgr2=0x7f67dc06b810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:02:50.758 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:02:50.757+0000 7f67fa231700 1 --2- 172.21.15.162:0/4083049265 >> [v2:172.21.15.162:6800/4079782599,v1:172.21.15.162:6801/4079782599] conn(0x7f67dc069360 0x7f67dc06b810 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f67e4006fd0 tx=0x7f67e4006dd0 comp rx=0 tx=0).stop 2024-04-27T04:02:50.758 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:02:50.757+0000 7f67fa231700 1 -- 172.21.15.162:0/4083049265 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f67f4102a50 msgr2=0x7f67f40fc210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:02:50.758 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:02:50.757+0000 7f67fa231700 1 --2- 172.21.15.162:0/4083049265 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f67f4102a50 0x7f67f40fc210 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f67e800b5c0 tx=0x7f67e8009e30 comp rx=0 tx=0).stop 2024-04-27T04:02:50.758 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:02:50.757+0000 7f67fa231700 1 -- 172.21.15.162:0/4083049265 shutdown_connections 2024-04-27T04:02:50.758 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:02:50.757+0000 7f67fa231700 1 --2- 172.21.15.162:0/4083049265 >> [v2:172.21.15.162:6800/4079782599,v1:172.21.15.162:6801/4079782599] conn(0x7f67dc069360 0x7f67dc06b810 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:02:50.758 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:02:50.757+0000 7f67fa231700 1 --2- 172.21.15.162:0/4083049265 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f67f4102a50 0x7f67f40fc210 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:02:50.758 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:02:50.757+0000 7f67fa231700 1 --2- 172.21.15.162:0/4083049265 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f67f40fc750 0x7f67f40fcbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:02:50.758 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:02:50.757+0000 7f67fa231700 1 -- 172.21.15.162:0/4083049265 >> 172.21.15.162:0/4083049265 conn(0x7f67f4073090 msgr2=0x7f67f40f9f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:02:50.759 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:02:50.757+0000 7f67fa231700 1 -- 172.21.15.162:0/4083049265 shutdown_connections 2024-04-27T04:02:50.759 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:02:50.757+0000 7f67fa231700 1 -- 172.21.15.162:0/4083049265 wait complete. 2024-04-27T04:02:51.635 INFO:tasks.cephadm:Distributing (final) config and client.admin keyring... 2024-04-27T04:02:51.635 DEBUG:teuthology.orchestra.run.smithi162:> set -ex 2024-04-27T04:02:51.635 DEBUG:teuthology.orchestra.run.smithi162:> sudo dd of=/etc/ceph/ceph.conf 2024-04-27T04:02:51.701 DEBUG:teuthology.orchestra.run.smithi162:> set -ex 2024-04-27T04:02:51.702 DEBUG:teuthology.orchestra.run.smithi162:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-04-27T04:02:51.766 DEBUG:teuthology.orchestra.run.smithi181:> set -ex 2024-04-27T04:02:51.766 DEBUG:teuthology.orchestra.run.smithi181:> sudo dd of=/etc/ceph/ceph.conf 2024-04-27T04:02:51.798 DEBUG:teuthology.orchestra.run.smithi181:> set -ex 2024-04-27T04:02:51.798 DEBUG:teuthology.orchestra.run.smithi181:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-04-27T04:02:51.866 INFO:tasks.cephadm:Deploying OSDs... 2024-04-27T04:02:51.867 DEBUG:teuthology.orchestra.run.smithi162:> set -ex 2024-04-27T04:02:51.867 DEBUG:teuthology.orchestra.run.smithi162:> dd if=/scratch_devs of=/dev/stdout 2024-04-27T04:02:51.881 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-04-27T04:02:51.881 DEBUG:teuthology.orchestra.run.smithi162:> stat /dev/vg_nvme/lv_1 2024-04-27T04:02:51.888 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:02:51 smithi162 ceph-mon[26841]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-27T04:02:51.888 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:02:51 smithi162 ceph-mon[26841]: from='client.? 172.21.15.162:0/4083049265' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T04:02:51.888 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:02:51 smithi162 ceph-mon[26841]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' 2024-04-27T04:02:51.888 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:02:51 smithi162 ceph-mon[26841]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' 2024-04-27T04:02:51.888 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:02:51 smithi162 ceph-mon[26841]: Reconfiguring mgr.smithi162.oennfn (unknown last config time)... 2024-04-27T04:02:51.888 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:02:51 smithi162 ceph-mon[26841]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi162.oennfn", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-27T04:02:51.889 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:02:51 smithi162 ceph-mon[26841]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-27T04:02:51.889 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:02:51 smithi162 ceph-mon[26841]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T04:02:51.889 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:02:51 smithi162 ceph-mon[26841]: Reconfiguring daemon mgr.smithi162.oennfn on smithi162 2024-04-27T04:02:51.902 INFO:teuthology.orchestra.run.smithi162.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-04-27T04:02:51.902 INFO:teuthology.orchestra.run.smithi162.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-27T04:02:51.902 INFO:teuthology.orchestra.run.smithi162.stdout:Device: 5h/5d Inode: 664 Links: 1 2024-04-27T04:02:51.902 INFO:teuthology.orchestra.run.smithi162.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-27T04:02:51.902 INFO:teuthology.orchestra.run.smithi162.stdout:Context: system_u:object_r:device_t:s0 2024-04-27T04:02:51.902 INFO:teuthology.orchestra.run.smithi162.stdout:Access: 2024-04-27 04:02:08.621822964 +0000 2024-04-27T04:02:51.902 INFO:teuthology.orchestra.run.smithi162.stdout:Modify: 2024-04-27 03:55:04.576762429 +0000 2024-04-27T04:02:51.902 INFO:teuthology.orchestra.run.smithi162.stdout:Change: 2024-04-27 03:55:04.576762429 +0000 2024-04-27T04:02:51.903 INFO:teuthology.orchestra.run.smithi162.stdout: Birth: 2024-04-27 03:55:04.576762429 +0000 2024-04-27T04:02:51.903 DEBUG:teuthology.orchestra.run.smithi162:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-04-27T04:02:51.923 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:02:51 smithi181 ceph-mon[33774]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-27T04:02:51.924 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:02:51 smithi181 ceph-mon[33774]: from='client.? 172.21.15.162:0/4083049265' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T04:02:51.924 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:02:51 smithi181 ceph-mon[33774]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' 2024-04-27T04:02:51.924 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:02:51 smithi181 ceph-mon[33774]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' 2024-04-27T04:02:51.924 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:02:51 smithi181 ceph-mon[33774]: Reconfiguring mgr.smithi162.oennfn (unknown last config time)... 2024-04-27T04:02:51.924 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:02:51 smithi181 ceph-mon[33774]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi162.oennfn", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-27T04:02:51.924 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:02:51 smithi181 ceph-mon[33774]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-27T04:02:51.924 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:02:51 smithi181 ceph-mon[33774]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T04:02:51.924 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:02:51 smithi181 ceph-mon[33774]: Reconfiguring daemon mgr.smithi162.oennfn on smithi162 2024-04-27T04:02:51.968 INFO:teuthology.orchestra.run.smithi162.stderr:1+0 records in 2024-04-27T04:02:51.968 INFO:teuthology.orchestra.run.smithi162.stderr:1+0 records out 2024-04-27T04:02:51.968 INFO:teuthology.orchestra.run.smithi162.stderr:512 bytes copied, 0.000256333 s, 2.0 MB/s 2024-04-27T04:02:51.970 DEBUG:teuthology.orchestra.run.smithi162:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-04-27T04:02:52.027 DEBUG:teuthology.orchestra.run.smithi162:> stat /dev/vg_nvme/lv_2 2024-04-27T04:02:52.081 INFO:teuthology.orchestra.run.smithi162.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-04-27T04:02:52.082 INFO:teuthology.orchestra.run.smithi162.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-27T04:02:52.082 INFO:teuthology.orchestra.run.smithi162.stdout:Device: 5h/5d Inode: 686 Links: 1 2024-04-27T04:02:52.082 INFO:teuthology.orchestra.run.smithi162.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-27T04:02:52.082 INFO:teuthology.orchestra.run.smithi162.stdout:Context: system_u:object_r:device_t:s0 2024-04-27T04:02:52.082 INFO:teuthology.orchestra.run.smithi162.stdout:Access: 2024-04-27 04:02:08.621822964 +0000 2024-04-27T04:02:52.082 INFO:teuthology.orchestra.run.smithi162.stdout:Modify: 2024-04-27 03:55:04.921754597 +0000 2024-04-27T04:02:52.082 INFO:teuthology.orchestra.run.smithi162.stdout:Change: 2024-04-27 03:55:04.921754597 +0000 2024-04-27T04:02:52.082 INFO:teuthology.orchestra.run.smithi162.stdout: Birth: 2024-04-27 03:55:04.921754597 +0000 2024-04-27T04:02:52.082 DEBUG:teuthology.orchestra.run.smithi162:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-04-27T04:02:52.145 INFO:teuthology.orchestra.run.smithi162.stderr:1+0 records in 2024-04-27T04:02:52.145 INFO:teuthology.orchestra.run.smithi162.stderr:1+0 records out 2024-04-27T04:02:52.145 INFO:teuthology.orchestra.run.smithi162.stderr:512 bytes copied, 0.000209144 s, 2.4 MB/s 2024-04-27T04:02:52.146 DEBUG:teuthology.orchestra.run.smithi162:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-04-27T04:02:52.201 DEBUG:teuthology.orchestra.run.smithi162:> stat /dev/vg_nvme/lv_3 2024-04-27T04:02:52.256 INFO:teuthology.orchestra.run.smithi162.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-04-27T04:02:52.257 INFO:teuthology.orchestra.run.smithi162.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-27T04:02:52.257 INFO:teuthology.orchestra.run.smithi162.stdout:Device: 5h/5d Inode: 703 Links: 1 2024-04-27T04:02:52.257 INFO:teuthology.orchestra.run.smithi162.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-27T04:02:52.257 INFO:teuthology.orchestra.run.smithi162.stdout:Context: system_u:object_r:device_t:s0 2024-04-27T04:02:52.257 INFO:teuthology.orchestra.run.smithi162.stdout:Access: 2024-04-27 04:02:08.622822943 +0000 2024-04-27T04:02:52.257 INFO:teuthology.orchestra.run.smithi162.stdout:Modify: 2024-04-27 03:55:05.269746697 +0000 2024-04-27T04:02:52.257 INFO:teuthology.orchestra.run.smithi162.stdout:Change: 2024-04-27 03:55:05.269746697 +0000 2024-04-27T04:02:52.257 INFO:teuthology.orchestra.run.smithi162.stdout: Birth: 2024-04-27 03:55:05.269746697 +0000 2024-04-27T04:02:52.257 DEBUG:teuthology.orchestra.run.smithi162:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-04-27T04:02:52.319 INFO:teuthology.orchestra.run.smithi162.stderr:1+0 records in 2024-04-27T04:02:52.319 INFO:teuthology.orchestra.run.smithi162.stderr:1+0 records out 2024-04-27T04:02:52.319 INFO:teuthology.orchestra.run.smithi162.stderr:512 bytes copied, 0.000236325 s, 2.2 MB/s 2024-04-27T04:02:52.320 DEBUG:teuthology.orchestra.run.smithi162:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-04-27T04:02:52.375 DEBUG:teuthology.orchestra.run.smithi162:> stat /dev/vg_nvme/lv_4 2024-04-27T04:02:52.429 INFO:teuthology.orchestra.run.smithi162.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-04-27T04:02:52.430 INFO:teuthology.orchestra.run.smithi162.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-27T04:02:52.430 INFO:teuthology.orchestra.run.smithi162.stdout:Device: 5h/5d Inode: 722 Links: 1 2024-04-27T04:02:52.430 INFO:teuthology.orchestra.run.smithi162.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-27T04:02:52.430 INFO:teuthology.orchestra.run.smithi162.stdout:Context: system_u:object_r:device_t:s0 2024-04-27T04:02:52.430 INFO:teuthology.orchestra.run.smithi162.stdout:Access: 2024-04-27 04:02:08.622822943 +0000 2024-04-27T04:02:52.430 INFO:teuthology.orchestra.run.smithi162.stdout:Modify: 2024-04-27 03:55:05.610738957 +0000 2024-04-27T04:02:52.430 INFO:teuthology.orchestra.run.smithi162.stdout:Change: 2024-04-27 03:55:05.610738957 +0000 2024-04-27T04:02:52.430 INFO:teuthology.orchestra.run.smithi162.stdout: Birth: 2024-04-27 03:55:05.610738957 +0000 2024-04-27T04:02:52.430 DEBUG:teuthology.orchestra.run.smithi162:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-04-27T04:02:52.491 INFO:teuthology.orchestra.run.smithi162.stderr:1+0 records in 2024-04-27T04:02:52.492 INFO:teuthology.orchestra.run.smithi162.stderr:1+0 records out 2024-04-27T04:02:52.492 INFO:teuthology.orchestra.run.smithi162.stderr:512 bytes copied, 0.00018743 s, 2.7 MB/s 2024-04-27T04:02:52.493 DEBUG:teuthology.orchestra.run.smithi162:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-04-27T04:02:52.549 DEBUG:teuthology.orchestra.run.smithi181:> set -ex 2024-04-27T04:02:52.549 DEBUG:teuthology.orchestra.run.smithi181:> dd if=/scratch_devs of=/dev/stdout 2024-04-27T04:02:52.568 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-04-27T04:02:52.568 DEBUG:teuthology.orchestra.run.smithi181:> stat /dev/vg_nvme/lv_1 2024-04-27T04:02:52.625 INFO:teuthology.orchestra.run.smithi181.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-04-27T04:02:52.625 INFO:teuthology.orchestra.run.smithi181.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-27T04:02:52.626 INFO:teuthology.orchestra.run.smithi181.stdout:Device: 5h/5d Inode: 666 Links: 1 2024-04-27T04:02:52.626 INFO:teuthology.orchestra.run.smithi181.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-27T04:02:52.626 INFO:teuthology.orchestra.run.smithi181.stdout:Context: system_u:object_r:device_t:s0 2024-04-27T04:02:52.626 INFO:teuthology.orchestra.run.smithi181.stdout:Access: 2024-04-27 04:02:38.852165495 +0000 2024-04-27T04:02:52.626 INFO:teuthology.orchestra.run.smithi181.stdout:Modify: 2024-04-27 03:55:04.546703344 +0000 2024-04-27T04:02:52.626 INFO:teuthology.orchestra.run.smithi181.stdout:Change: 2024-04-27 03:55:04.546703344 +0000 2024-04-27T04:02:52.626 INFO:teuthology.orchestra.run.smithi181.stdout: Birth: 2024-04-27 03:55:04.546703344 +0000 2024-04-27T04:02:52.626 DEBUG:teuthology.orchestra.run.smithi181:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-04-27T04:02:52.691 INFO:teuthology.orchestra.run.smithi181.stderr:1+0 records in 2024-04-27T04:02:52.691 INFO:teuthology.orchestra.run.smithi181.stderr:1+0 records out 2024-04-27T04:02:52.692 INFO:teuthology.orchestra.run.smithi181.stderr:512 bytes copied, 0.000234278 s, 2.2 MB/s 2024-04-27T04:02:52.693 DEBUG:teuthology.orchestra.run.smithi181:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-04-27T04:02:52.750 DEBUG:teuthology.orchestra.run.smithi181:> stat /dev/vg_nvme/lv_2 2024-04-27T04:02:52.807 INFO:teuthology.orchestra.run.smithi181.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-04-27T04:02:52.807 INFO:teuthology.orchestra.run.smithi181.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-27T04:02:52.807 INFO:teuthology.orchestra.run.smithi181.stdout:Device: 5h/5d Inode: 684 Links: 1 2024-04-27T04:02:52.807 INFO:teuthology.orchestra.run.smithi181.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-27T04:02:52.807 INFO:teuthology.orchestra.run.smithi181.stdout:Context: system_u:object_r:device_t:s0 2024-04-27T04:02:52.807 INFO:teuthology.orchestra.run.smithi181.stdout:Access: 2024-04-27 04:02:38.852165495 +0000 2024-04-27T04:02:52.808 INFO:teuthology.orchestra.run.smithi181.stdout:Modify: 2024-04-27 03:55:04.873696124 +0000 2024-04-27T04:02:52.808 INFO:teuthology.orchestra.run.smithi181.stdout:Change: 2024-04-27 03:55:04.873696124 +0000 2024-04-27T04:02:52.808 INFO:teuthology.orchestra.run.smithi181.stdout: Birth: 2024-04-27 03:55:04.873696124 +0000 2024-04-27T04:02:52.808 DEBUG:teuthology.orchestra.run.smithi181:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-04-27T04:02:52.873 INFO:teuthology.orchestra.run.smithi181.stderr:1+0 records in 2024-04-27T04:02:52.873 INFO:teuthology.orchestra.run.smithi181.stderr:1+0 records out 2024-04-27T04:02:52.874 INFO:teuthology.orchestra.run.smithi181.stderr:512 bytes copied, 0.000247917 s, 2.1 MB/s 2024-04-27T04:02:52.875 DEBUG:teuthology.orchestra.run.smithi181:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-04-27T04:02:52.931 DEBUG:teuthology.orchestra.run.smithi181:> stat /dev/vg_nvme/lv_3 2024-04-27T04:02:52.987 INFO:teuthology.orchestra.run.smithi181.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-04-27T04:02:52.987 INFO:teuthology.orchestra.run.smithi181.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-27T04:02:52.987 INFO:teuthology.orchestra.run.smithi181.stdout:Device: 5h/5d Inode: 704 Links: 1 2024-04-27T04:02:52.988 INFO:teuthology.orchestra.run.smithi181.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-27T04:02:52.988 INFO:teuthology.orchestra.run.smithi181.stdout:Context: system_u:object_r:device_t:s0 2024-04-27T04:02:52.988 INFO:teuthology.orchestra.run.smithi181.stdout:Access: 2024-04-27 04:02:38.852165495 +0000 2024-04-27T04:02:52.988 INFO:teuthology.orchestra.run.smithi181.stdout:Modify: 2024-04-27 03:55:05.200688905 +0000 2024-04-27T04:02:52.988 INFO:teuthology.orchestra.run.smithi181.stdout:Change: 2024-04-27 03:55:05.200688905 +0000 2024-04-27T04:02:52.988 INFO:teuthology.orchestra.run.smithi181.stdout: Birth: 2024-04-27 03:55:05.200688905 +0000 2024-04-27T04:02:52.988 DEBUG:teuthology.orchestra.run.smithi181:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-04-27T04:02:53.051 INFO:teuthology.orchestra.run.smithi181.stderr:1+0 records in 2024-04-27T04:02:53.051 INFO:teuthology.orchestra.run.smithi181.stderr:1+0 records out 2024-04-27T04:02:53.051 INFO:teuthology.orchestra.run.smithi181.stderr:512 bytes copied, 0.000212356 s, 2.4 MB/s 2024-04-27T04:02:53.052 DEBUG:teuthology.orchestra.run.smithi181:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-04-27T04:02:53.108 DEBUG:teuthology.orchestra.run.smithi181:> stat /dev/vg_nvme/lv_4 2024-04-27T04:02:53.163 INFO:teuthology.orchestra.run.smithi181.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-04-27T04:02:53.164 INFO:teuthology.orchestra.run.smithi181.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-27T04:02:53.164 INFO:teuthology.orchestra.run.smithi181.stdout:Device: 5h/5d Inode: 721 Links: 1 2024-04-27T04:02:53.164 INFO:teuthology.orchestra.run.smithi181.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-27T04:02:53.164 INFO:teuthology.orchestra.run.smithi181.stdout:Context: system_u:object_r:device_t:s0 2024-04-27T04:02:53.164 INFO:teuthology.orchestra.run.smithi181.stdout:Access: 2024-04-27 04:02:38.852165495 +0000 2024-04-27T04:02:53.164 INFO:teuthology.orchestra.run.smithi181.stdout:Modify: 2024-04-27 03:55:05.551681156 +0000 2024-04-27T04:02:53.164 INFO:teuthology.orchestra.run.smithi181.stdout:Change: 2024-04-27 03:55:05.551681156 +0000 2024-04-27T04:02:53.164 INFO:teuthology.orchestra.run.smithi181.stdout: Birth: 2024-04-27 03:55:05.551681156 +0000 2024-04-27T04:02:53.164 DEBUG:teuthology.orchestra.run.smithi181:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-04-27T04:02:53.226 INFO:teuthology.orchestra.run.smithi181.stderr:1+0 records in 2024-04-27T04:02:53.226 INFO:teuthology.orchestra.run.smithi181.stderr:1+0 records out 2024-04-27T04:02:53.226 INFO:teuthology.orchestra.run.smithi181.stderr:512 bytes copied, 0.000194354 s, 2.6 MB/s 2024-04-27T04:02:53.227 DEBUG:teuthology.orchestra.run.smithi181:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-04-27T04:02:53.283 INFO:tasks.cephadm:Deploying osd.0 on smithi162 with /dev/vg_nvme/lv_4... 2024-04-27T04:02:53.283 DEBUG:teuthology.orchestra.run.smithi162:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 7f9750d6-044a-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-04-27T04:02:54.097 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:02:53 smithi162 ceph-mon[26841]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-27T04:02:54.097 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:02:53 smithi162 ceph-mon[26841]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' 2024-04-27T04:02:54.097 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:02:53 smithi162 ceph-mon[26841]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' 2024-04-27T04:02:54.097 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:02:53 smithi162 ceph-mon[26841]: Reconfiguring crash.smithi162 (monmap changed)... 2024-04-27T04:02:54.097 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:02:53 smithi162 ceph-mon[26841]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi162", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-04-27T04:02:54.097 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:02:53 smithi162 ceph-mon[26841]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T04:02:54.097 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:02:53 smithi162 ceph-mon[26841]: Reconfiguring daemon crash.smithi162 on smithi162 2024-04-27T04:02:54.111 INFO:teuthology.orchestra.run.smithi162.stderr:Inferring config /var/lib/ceph/7f9750d6-044a-11ef-bc93-c7b262605968/mon.smithi162/config 2024-04-27T04:02:54.174 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:02:53 smithi181 ceph-mon[33774]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-27T04:02:54.174 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:02:53 smithi181 ceph-mon[33774]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' 2024-04-27T04:02:54.174 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:02:53 smithi181 ceph-mon[33774]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' 2024-04-27T04:02:54.174 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:02:53 smithi181 ceph-mon[33774]: Reconfiguring crash.smithi162 (monmap changed)... 2024-04-27T04:02:54.174 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:02:53 smithi181 ceph-mon[33774]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi162", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-04-27T04:02:54.174 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:02:53 smithi181 ceph-mon[33774]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T04:02:54.174 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:02:53 smithi181 ceph-mon[33774]: Reconfiguring daemon crash.smithi162 on smithi162 2024-04-27T04:02:55.423 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:02:55 smithi181 ceph-mon[33774]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' 2024-04-27T04:02:55.423 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:02:55 smithi181 ceph-mon[33774]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' 2024-04-27T04:02:55.424 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:02:55 smithi181 ceph-mon[33774]: Reconfiguring alertmanager.smithi162 (dependencies changed)... 2024-04-27T04:02:55.424 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:02:55 smithi181 ceph-mon[33774]: Reconfiguring daemon alertmanager.smithi162 on smithi162 2024-04-27T04:02:55.638 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:02:55 smithi162 ceph-mon[26841]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' 2024-04-27T04:02:55.638 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:02:55 smithi162 ceph-mon[26841]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' 2024-04-27T04:02:55.638 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:02:55 smithi162 ceph-mon[26841]: Reconfiguring alertmanager.smithi162 (dependencies changed)... 2024-04-27T04:02:55.638 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:02:55 smithi162 ceph-mon[26841]: Reconfiguring daemon alertmanager.smithi162 on smithi162 2024-04-27T04:02:56.423 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:02:56 smithi181 ceph-mon[33774]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-27T04:02:56.475 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:02:56 smithi162 ceph-mon[26841]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-27T04:02:58.310 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:02:58 smithi162 ceph-mon[26841]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-27T04:02:58.311 INFO:teuthology.orchestra.run.smithi162.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-04-27T04:02:58.311 INFO:teuthology.orchestra.run.smithi162.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-04-27T04:02:58.311 INFO:teuthology.orchestra.run.smithi162.stderr: stderr: 10+0 records in 2024-04-27T04:02:58.311 INFO:teuthology.orchestra.run.smithi162.stderr:10+0 records out 2024-04-27T04:02:58.311 INFO:teuthology.orchestra.run.smithi162.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.013854 s, 757 MB/s 2024-04-27T04:02:58.311 INFO:teuthology.orchestra.run.smithi162.stderr:--> Zapping successful for: 2024-04-27T04:02:58.423 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:02:58 smithi181 ceph-mon[33774]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-27T04:02:58.991 DEBUG:teuthology.orchestra.run.smithi162:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 7f9750d6-044a-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi162:vg_nvme/lv_4 2024-04-27T04:02:59.428 INFO:teuthology.orchestra.run.smithi162.stderr:Inferring config /var/lib/ceph/7f9750d6-044a-11ef-bc93-c7b262605968/mon.smithi162/config 2024-04-27T04:03:00.351 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:03:00 smithi162 ceph-mon[26841]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-27T04:03:00.423 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:03:00 smithi181 ceph-mon[33774]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-27T04:03:00.825 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:03:00.823+0000 7f40bb725700 1 -- 172.21.15.162:0/3074043467 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f40b4066b50 msgr2=0x7f40b40ff9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:03:00.825 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:03:00.823+0000 7f40bb725700 1 --2- 172.21.15.162:0/3074043467 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f40b4066b50 0x7f40b40ff9b0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f40b0004940 tx=0x7f40b0009e30 comp rx=0 tx=0).stop 2024-04-27T04:03:00.825 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:03:00.823+0000 7f40bb725700 1 -- 172.21.15.162:0/3074043467 shutdown_connections 2024-04-27T04:03:00.826 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:03:00.823+0000 7f40bb725700 1 --2- 172.21.15.162:0/3074043467 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f40b40ffef0 0x7f40b4102310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:03:00.826 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:03:00.823+0000 7f40bb725700 1 --2- 172.21.15.162:0/3074043467 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f40b4066b50 0x7f40b40ff9b0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:03:00.826 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:03:00.823+0000 7f40bb725700 1 -- 172.21.15.162:0/3074043467 >> 172.21.15.162:0/3074043467 conn(0x7f40b40f7420 msgr2=0x7f40b40f9850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:03:00.826 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:03:00.823+0000 7f40bb725700 1 -- 172.21.15.162:0/3074043467 shutdown_connections 2024-04-27T04:03:00.826 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:03:00.823+0000 7f40bb725700 1 -- 172.21.15.162:0/3074043467 wait complete. 2024-04-27T04:03:00.826 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:03:00.824+0000 7f40bb725700 1 Processor -- start 2024-04-27T04:03:00.826 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:03:00.824+0000 7f40bb725700 1 -- start start 2024-04-27T04:03:00.826 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:03:00.824+0000 7f40bb725700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f40b4066b50 0x7f40b410da30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:03:00.826 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:03:00.824+0000 7f40bb725700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f40b40ffef0 0x7f40b410df70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:03:00.826 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:03:00.824+0000 7f40bb725700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40b410e570 con 0x7f40b40ffef0 2024-04-27T04:03:00.826 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:03:00.824+0000 7f40bb725700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40b410edc0 con 0x7f40b4066b50 2024-04-27T04:03:00.826 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:03:00.824+0000 7f40b8cc0700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f40b40ffef0 0x7f40b410df70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:03:00.826 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:03:00.824+0000 7f40b8cc0700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f40b40ffef0 0x7f40b410df70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:60446/0 (socket says 172.21.15.162:60446) 2024-04-27T04:03:00.827 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:03:00.824+0000 7f40b8cc0700 1 -- 172.21.15.162:0/503341913 learned_addr learned my addr 172.21.15.162:0/503341913 (peer_addr_for_me v2:172.21.15.162:0/0) 2024-04-27T04:03:00.827 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:03:00.824+0000 7f40b94c1700 1 --2- 172.21.15.162:0/503341913 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f40b4066b50 0x7f40b410da30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:03:00.827 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:03:00.824+0000 7f40b8cc0700 1 -- 172.21.15.162:0/503341913 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f40b4066b50 msgr2=0x7f40b410da30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:03:00.827 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:03:00.824+0000 7f40b8cc0700 1 --2- 172.21.15.162:0/503341913 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f40b4066b50 0x7f40b410da30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:03:00.827 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:03:00.824+0000 7f40b8cc0700 1 -- 172.21.15.162:0/503341913 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f40b0005040 con 0x7f40b40ffef0 2024-04-27T04:03:00.827 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:03:00.825+0000 7f40b8cc0700 1 --2- 172.21.15.162:0/503341913 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f40b40ffef0 0x7f40b410df70 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f40a400c830 tx=0x7f40a4004470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:03:00.828 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:03:00.825+0000 7f40a9ffb700 1 -- 172.21.15.162:0/503341913 <== mon.0 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f40a400a760 con 0x7f40b40ffef0 2024-04-27T04:03:00.828 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:03:00.825+0000 7f40bb725700 1 -- 172.21.15.162:0/503341913 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f40b410f0a0 con 0x7f40b40ffef0 2024-04-27T04:03:00.829 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:03:00.825+0000 7f40a9ffb700 1 -- 172.21.15.162:0/503341913 <== mon.0 v2:172.21.15.162:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f40a400ad90 con 0x7f40b40ffef0 2024-04-27T04:03:00.829 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:03:00.825+0000 7f40a9ffb700 1 -- 172.21.15.162:0/503341913 <== mon.0 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f40a40113a0 con 0x7f40b40ffef0 2024-04-27T04:03:00.829 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:03:00.825+0000 7f40bb725700 1 -- 172.21.15.162:0/503341913 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f40b410f5e0 con 0x7f40b40ffef0 2024-04-27T04:03:00.829 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:03:00.826+0000 7f40bb725700 1 -- 172.21.15.162:0/503341913 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f40b405f5c0 con 0x7f40b40ffef0 2024-04-27T04:03:00.829 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:03:00.827+0000 7f40a9ffb700 1 -- 172.21.15.162:0/503341913 <== mon.0 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 18) v1 ==== 86575+0+0 (secure 0 0 0) 0x7f40a4012400 con 0x7f40b40ffef0 2024-04-27T04:03:00.829 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:03:00.827+0000 7f40a9ffb700 1 --2- 172.21.15.162:0/503341913 >> [v2:172.21.15.162:6800/4079782599,v1:172.21.15.162:6801/4079782599] conn(0x7f40a0069210 0x7f40a006b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:03:00.829 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:03:00.827+0000 7f40a9ffb700 1 -- 172.21.15.162:0/503341913 <== mon.0 v2:172.21.15.162:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1366+0+0 (secure 0 0 0) 0x7f40a4086120 con 0x7f40b40ffef0 2024-04-27T04:03:00.829 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:03:00.827+0000 7f40b94c1700 1 --2- 172.21.15.162:0/503341913 >> [v2:172.21.15.162:6800/4079782599,v1:172.21.15.162:6801/4079782599] conn(0x7f40a0069210 0x7f40a006b6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:03:00.829 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:03:00.828+0000 7f40b94c1700 1 --2- 172.21.15.162:0/503341913 >> [v2:172.21.15.162:6800/4079782599,v1:172.21.15.162:6801/4079782599] conn(0x7f40a0069210 0x7f40a006b6c0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f40b000b3b0 tx=0x7f40b0009e30 comp rx=0 tx=0).ready entity=mgr.14210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:03:00.834 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:03:00.834+0000 7f40a9ffb700 1 -- 172.21.15.162:0/503341913 <== mon.0 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f40a4056fc0 con 0x7f40b40ffef0 2024-04-27T04:03:01.047 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:03:01.046+0000 7f40bb725700 1 -- 172.21.15.162:0/503341913 --> [v2:172.21.15.162:6800/4079782599,v1:172.21.15.162:6801/4079782599] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi162:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) v1 -- 0x7f40b410f9d0 con 0x7f40a0069210 2024-04-27T04:03:01.274 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:03:01 smithi162 ceph-mon[26841]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-27T04:03:01.274 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:03:01 smithi162 ceph-mon[26841]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-27T04:03:01.274 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:03:01 smithi162 ceph-mon[26841]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T04:03:01.423 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:03:01 smithi181 ceph-mon[33774]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-27T04:03:01.424 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:03:01 smithi181 ceph-mon[33774]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-27T04:03:01.424 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:03:01 smithi181 ceph-mon[33774]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T04:03:02.313 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:03:02 smithi162 ceph-mon[26841]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-27T04:03:02.313 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:03:02 smithi162 ceph-mon[26841]: from='client.14246 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi162:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-27T04:03:02.314 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:03:02 smithi162 ceph-mon[26841]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' 2024-04-27T04:03:02.314 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:03:02 smithi162 ceph-mon[26841]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' 2024-04-27T04:03:02.314 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:03:02 smithi162 ceph-mon[26841]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-27T04:03:02.423 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:03:02 smithi181 ceph-mon[33774]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-27T04:03:02.423 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:03:02 smithi181 ceph-mon[33774]: from='client.14246 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi162:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-27T04:03:02.423 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:03:02 smithi181 ceph-mon[33774]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' 2024-04-27T04:03:02.423 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:03:02 smithi181 ceph-mon[33774]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' 2024-04-27T04:03:02.424 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:03:02 smithi181 ceph-mon[33774]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-27T04:03:03.388 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:03:03 smithi162 ceph-mon[26841]: Reconfiguring grafana.smithi162 (dependencies changed)... 2024-04-27T04:03:03.388 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:03:03 smithi162 ceph-mon[26841]: Reconfiguring daemon grafana.smithi162 on smithi162 2024-04-27T04:03:03.673 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:03:03 smithi181 ceph-mon[33774]: Reconfiguring grafana.smithi162 (dependencies changed)... 2024-04-27T04:03:03.674 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:03:03 smithi181 ceph-mon[33774]: Reconfiguring daemon grafana.smithi162 on smithi162 2024-04-27T04:03:04.377 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:03:04 smithi162 ceph-mon[26841]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-27T04:03:04.673 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:03:04 smithi181 ceph-mon[33774]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-27T04:03:06.378 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:03:06 smithi162 ceph-mon[26841]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-27T04:03:06.673 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:03:06 smithi181 ceph-mon[33774]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-27T04:03:07.638 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:03:07 smithi162 ceph-mon[26841]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' 2024-04-27T04:03:07.639 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:03:07 smithi162 ceph-mon[26841]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' 2024-04-27T04:03:07.639 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:03:07 smithi162 ceph-mon[26841]: Reconfiguring prometheus.smithi162 (dependencies changed)... 2024-04-27T04:03:07.639 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:03:07 smithi162 ceph-mon[26841]: Reconfiguring daemon prometheus.smithi162 on smithi162 2024-04-27T04:03:07.673 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:03:07 smithi181 ceph-mon[33774]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' 2024-04-27T04:03:07.673 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:03:07 smithi181 ceph-mon[33774]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' 2024-04-27T04:03:07.673 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:03:07 smithi181 ceph-mon[33774]: Reconfiguring prometheus.smithi162 (dependencies changed)... 2024-04-27T04:03:07.673 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:03:07 smithi181 ceph-mon[33774]: Reconfiguring daemon prometheus.smithi162 on smithi162 2024-04-27T04:03:08.638 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:03:08 smithi162 ceph-mon[26841]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-27T04:03:08.639 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:03:08 smithi162 ceph-mon[26841]: from='client.? 172.21.15.162:0/3330175758' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0ae69d7f-0ab4-4672-89b9-3563a10a1bf2"}]: dispatch 2024-04-27T04:03:08.639 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:03:08 smithi162 ceph-mon[26841]: from='client.? 172.21.15.162:0/3330175758' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "0ae69d7f-0ab4-4672-89b9-3563a10a1bf2"}]': finished 2024-04-27T04:03:08.639 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:03:08 smithi162 ceph-mon[26841]: osdmap e6: 1 total, 0 up, 1 in 2024-04-27T04:03:08.639 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:03:08 smithi162 ceph-mon[26841]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-27T04:03:08.639 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:03:08 smithi162 ceph-mon[26841]: from='client.? 172.21.15.162:0/832176331' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-27T04:03:08.673 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:03:08 smithi181 ceph-mon[33774]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-27T04:03:08.673 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:03:08 smithi181 ceph-mon[33774]: from='client.? 172.21.15.162:0/3330175758' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0ae69d7f-0ab4-4672-89b9-3563a10a1bf2"}]: dispatch 2024-04-27T04:03:08.674 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:03:08 smithi181 ceph-mon[33774]: from='client.? 172.21.15.162:0/3330175758' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "0ae69d7f-0ab4-4672-89b9-3563a10a1bf2"}]': finished 2024-04-27T04:03:08.674 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:03:08 smithi181 ceph-mon[33774]: osdmap e6: 1 total, 0 up, 1 in 2024-04-27T04:03:08.674 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:03:08 smithi181 ceph-mon[33774]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-27T04:03:08.674 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:03:08 smithi181 ceph-mon[33774]: from='client.? 172.21.15.162:0/832176331' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-27T04:03:10.638 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:03:10 smithi162 ceph-mon[26841]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-27T04:03:10.673 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:03:10 smithi181 ceph-mon[33774]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-27T04:03:12.388 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:03:12 smithi162 ceph-mon[26841]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-27T04:03:12.423 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:03:12 smithi181 ceph-mon[33774]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-27T04:03:14.388 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:03:14 smithi162 ceph-mon[26841]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-27T04:03:14.673 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:03:14 smithi181 ceph-mon[33774]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-27T04:03:16.558 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:03:16 smithi162 ceph-mon[26841]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-27T04:03:16.673 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:03:16 smithi181 ceph-mon[33774]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-27T04:03:17.479 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:03:17 smithi181 ceph-mon[33774]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-27T04:03:17.479 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:03:17 smithi181 ceph-mon[33774]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' 2024-04-27T04:03:17.479 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:03:17 smithi181 ceph-mon[33774]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' 2024-04-27T04:03:17.479 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:03:17 smithi181 ceph-mon[33774]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi181", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-04-27T04:03:17.479 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:03:17 smithi181 ceph-mon[33774]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T04:03:17.493 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:03:17 smithi162 ceph-mon[26841]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-27T04:03:17.493 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:03:17 smithi162 ceph-mon[26841]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' 2024-04-27T04:03:17.493 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:03:17 smithi162 ceph-mon[26841]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' 2024-04-27T04:03:17.493 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:03:17 smithi162 ceph-mon[26841]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi181", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-04-27T04:03:17.493 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:03:17 smithi162 ceph-mon[26841]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T04:03:18.388 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:03:18 smithi162 ceph-mon[26841]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-27T04:03:18.388 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:03:18 smithi162 ceph-mon[26841]: Reconfiguring crash.smithi181 (monmap changed)... 2024-04-27T04:03:18.388 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:03:18 smithi162 ceph-mon[26841]: Reconfiguring daemon crash.smithi181 on smithi181 2024-04-27T04:03:18.388 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:03:18 smithi162 ceph-mon[26841]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-04-27T04:03:18.388 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:03:18 smithi162 ceph-mon[26841]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T04:03:18.389 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:03:18 smithi162 ceph-mon[26841]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' 2024-04-27T04:03:18.389 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:03:18 smithi162 ceph-mon[26841]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' 2024-04-27T04:03:18.389 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:03:18 smithi162 ceph-mon[26841]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi181.vtwuhz", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-27T04:03:18.389 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:03:18 smithi162 ceph-mon[26841]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-27T04:03:18.389 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:03:18 smithi162 ceph-mon[26841]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T04:03:18.423 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:03:18 smithi181 ceph-mon[33774]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-27T04:03:18.424 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:03:18 smithi181 ceph-mon[33774]: Reconfiguring crash.smithi181 (monmap changed)... 2024-04-27T04:03:18.424 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:03:18 smithi181 ceph-mon[33774]: Reconfiguring daemon crash.smithi181 on smithi181 2024-04-27T04:03:18.424 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:03:18 smithi181 ceph-mon[33774]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-04-27T04:03:18.424 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:03:18 smithi181 ceph-mon[33774]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T04:03:18.424 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:03:18 smithi181 ceph-mon[33774]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' 2024-04-27T04:03:18.424 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:03:18 smithi181 ceph-mon[33774]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' 2024-04-27T04:03:18.424 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:03:18 smithi181 ceph-mon[33774]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi181.vtwuhz", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-27T04:03:18.424 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:03:18 smithi181 ceph-mon[33774]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-27T04:03:18.424 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:03:18 smithi181 ceph-mon[33774]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T04:03:19.563 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:03:19 smithi162 ceph-mon[26841]: Deploying daemon osd.0 on smithi162 2024-04-27T04:03:19.563 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:03:19 smithi162 ceph-mon[26841]: Reconfiguring mgr.smithi181.vtwuhz (monmap changed)... 2024-04-27T04:03:19.563 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:03:19 smithi162 ceph-mon[26841]: Reconfiguring daemon mgr.smithi181.vtwuhz on smithi181 2024-04-27T04:03:19.673 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:03:19 smithi181 ceph-mon[33774]: Deploying daemon osd.0 on smithi162 2024-04-27T04:03:19.673 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:03:19 smithi181 ceph-mon[33774]: Reconfiguring mgr.smithi181.vtwuhz (monmap changed)... 2024-04-27T04:03:19.673 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:03:19 smithi181 ceph-mon[33774]: Reconfiguring daemon mgr.smithi181.vtwuhz on smithi181 2024-04-27T04:03:20.423 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:03:20 smithi181 ceph-mon[33774]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-27T04:03:20.424 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:03:20 smithi181 ceph-mon[33774]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' 2024-04-27T04:03:20.424 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:03:20 smithi181 ceph-mon[33774]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' 2024-04-27T04:03:20.424 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:03:20 smithi181 ceph-mon[33774]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-27T04:03:20.424 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:03:20 smithi181 ceph-mon[33774]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-27T04:03:20.424 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:03:20 smithi181 ceph-mon[33774]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T04:03:20.447 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:03:20 smithi162 ceph-mon[26841]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-27T04:03:20.447 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:03:20 smithi162 ceph-mon[26841]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' 2024-04-27T04:03:20.447 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:03:20 smithi162 ceph-mon[26841]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' 2024-04-27T04:03:20.447 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:03:20 smithi162 ceph-mon[26841]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-27T04:03:20.447 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:03:20 smithi162 ceph-mon[26841]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-27T04:03:20.447 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:03:20 smithi162 ceph-mon[26841]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T04:03:21.343 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:03:21 smithi162 ceph-mon[26841]: Reconfiguring mon.smithi181 (monmap changed)... 2024-04-27T04:03:21.343 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:03:21 smithi162 ceph-mon[26841]: Reconfiguring daemon mon.smithi181 on smithi181 2024-04-27T04:03:21.343 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:03:21 smithi162 ceph-mon[26841]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' 2024-04-27T04:03:21.344 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:03:21 smithi162 ceph-mon[26841]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' 2024-04-27T04:03:21.344 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:03:21 smithi162 ceph-mon[26841]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-04-27T04:03:21.344 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:03:21 smithi162 ceph-mon[26841]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "dashboard set-alertmanager-api-host", "value": "http://smithi162:9093"}]: dispatch 2024-04-27T04:03:21.344 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:03:21 smithi162 ceph-mon[26841]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' 2024-04-27T04:03:21.344 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:03:21 smithi162 ceph-mon[26841]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-04-27T04:03:21.344 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:03:21 smithi162 ceph-mon[26841]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "dashboard set-grafana-api-url", "value": "https://smithi162:3000"}]: dispatch 2024-04-27T04:03:21.344 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:03:21 smithi162 ceph-mon[26841]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' 2024-04-27T04:03:21.344 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:03:21 smithi162 ceph-mon[26841]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-04-27T04:03:21.344 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:03:21 smithi162 ceph-mon[26841]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "dashboard set-prometheus-api-host", "value": "http://smithi162:9095"}]: dispatch 2024-04-27T04:03:21.344 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:03:21 smithi162 ceph-mon[26841]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' 2024-04-27T04:03:21.344 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:03:21 smithi162 ceph-mon[26841]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T04:03:21.674 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:03:21 smithi181 ceph-mon[33774]: Reconfiguring mon.smithi181 (monmap changed)... 2024-04-27T04:03:21.674 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:03:21 smithi181 ceph-mon[33774]: Reconfiguring daemon mon.smithi181 on smithi181 2024-04-27T04:03:21.674 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:03:21 smithi181 ceph-mon[33774]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' 2024-04-27T04:03:21.674 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:03:21 smithi181 ceph-mon[33774]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' 2024-04-27T04:03:21.674 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:03:21 smithi181 ceph-mon[33774]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-04-27T04:03:21.674 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:03:21 smithi181 ceph-mon[33774]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "dashboard set-alertmanager-api-host", "value": "http://smithi162:9093"}]: dispatch 2024-04-27T04:03:21.674 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:03:21 smithi181 ceph-mon[33774]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' 2024-04-27T04:03:21.674 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:03:21 smithi181 ceph-mon[33774]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-04-27T04:03:21.674 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:03:21 smithi181 ceph-mon[33774]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "dashboard set-grafana-api-url", "value": "https://smithi162:3000"}]: dispatch 2024-04-27T04:03:21.674 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:03:21 smithi181 ceph-mon[33774]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' 2024-04-27T04:03:21.674 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:03:21 smithi181 ceph-mon[33774]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-04-27T04:03:21.675 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:03:21 smithi181 ceph-mon[33774]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "dashboard set-prometheus-api-host", "value": "http://smithi162:9095"}]: dispatch 2024-04-27T04:03:21.675 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:03:21 smithi181 ceph-mon[33774]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' 2024-04-27T04:03:21.675 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:03:21 smithi181 ceph-mon[33774]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T04:03:22.583 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:03:22 smithi162 ceph-mon[26841]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-27T04:03:22.583 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:03:22 smithi162 ceph-mon[26841]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-04-27T04:03:22.583 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:03:22 smithi162 ceph-mon[26841]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-alertmanager-api-host", "value": "http://smithi162:9093"}]: dispatch 2024-04-27T04:03:22.584 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:03:22 smithi162 ceph-mon[26841]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-04-27T04:03:22.584 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:03:22 smithi162 ceph-mon[26841]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-grafana-api-url", "value": "https://smithi162:3000"}]: dispatch 2024-04-27T04:03:22.584 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:03:22 smithi162 ceph-mon[26841]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-04-27T04:03:22.584 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:03:22 smithi162 ceph-mon[26841]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-prometheus-api-host", "value": "http://smithi162:9095"}]: dispatch 2024-04-27T04:03:22.673 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:03:22 smithi181 ceph-mon[33774]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-27T04:03:22.674 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:03:22 smithi181 ceph-mon[33774]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-04-27T04:03:22.674 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:03:22 smithi181 ceph-mon[33774]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-alertmanager-api-host", "value": "http://smithi162:9093"}]: dispatch 2024-04-27T04:03:22.674 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:03:22 smithi181 ceph-mon[33774]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-04-27T04:03:22.674 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:03:22 smithi181 ceph-mon[33774]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-grafana-api-url", "value": "https://smithi162:3000"}]: dispatch 2024-04-27T04:03:22.674 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:03:22 smithi181 ceph-mon[33774]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-04-27T04:03:22.674 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:03:22 smithi181 ceph-mon[33774]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-prometheus-api-host", "value": "http://smithi162:9095"}]: dispatch 2024-04-27T04:03:23.787 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:03:23 smithi162 ceph-mon[26841]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' 2024-04-27T04:03:23.787 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:03:23 smithi162 ceph-mon[26841]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' 2024-04-27T04:03:23.787 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:03:23 smithi162 ceph-mon[26841]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-27T04:03:23.923 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:03:23 smithi181 ceph-mon[33774]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' 2024-04-27T04:03:23.923 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:03:23 smithi181 ceph-mon[33774]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' 2024-04-27T04:03:23.924 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:03:23 smithi181 ceph-mon[33774]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-27T04:03:24.888 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:03:24 smithi162 ceph-mon[26841]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' 2024-04-27T04:03:24.888 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:03:24 smithi162 ceph-mon[26841]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' 2024-04-27T04:03:24.923 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:03:24 smithi181 ceph-mon[33774]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' 2024-04-27T04:03:24.923 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:03:24 smithi181 ceph-mon[33774]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' 2024-04-27T04:03:25.861 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:03:25 smithi162 ceph-mon[26841]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-27T04:03:25.923 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:03:25 smithi181 ceph-mon[33774]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-27T04:03:27.531 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:03:27.530+0000 7f40a9ffb700 1 -- 172.21.15.162:0/503341913 <== mgr.14210 v2:172.21.15.162:6800/4079782599 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f40b410f9d0 con 0x7f40a0069210 2024-04-27T04:03:27.531 INFO:teuthology.orchestra.run.smithi162.stdout:Created osd(s) 0 on host 'smithi162' 2024-04-27T04:03:27.532 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:03:27.533+0000 7f40bb725700 1 -- 172.21.15.162:0/503341913 >> [v2:172.21.15.162:6800/4079782599,v1:172.21.15.162:6801/4079782599] conn(0x7f40a0069210 msgr2=0x7f40a006b6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:03:27.533 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:03:27.533+0000 7f40bb725700 1 --2- 172.21.15.162:0/503341913 >> [v2:172.21.15.162:6800/4079782599,v1:172.21.15.162:6801/4079782599] conn(0x7f40a0069210 0x7f40a006b6c0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f40b000b3b0 tx=0x7f40b0009e30 comp rx=0 tx=0).stop 2024-04-27T04:03:27.533 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:03:27.533+0000 7f40bb725700 1 -- 172.21.15.162:0/503341913 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f40b40ffef0 msgr2=0x7f40b410df70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:03:27.533 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:03:27.533+0000 7f40bb725700 1 --2- 172.21.15.162:0/503341913 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f40b40ffef0 0x7f40b410df70 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f40a400c830 tx=0x7f40a4004470 comp rx=0 tx=0).stop 2024-04-27T04:03:27.533 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:03:27.533+0000 7f40bb725700 1 -- 172.21.15.162:0/503341913 shutdown_connections 2024-04-27T04:03:27.533 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:03:27.533+0000 7f40bb725700 1 --2- 172.21.15.162:0/503341913 >> [v2:172.21.15.162:6800/4079782599,v1:172.21.15.162:6801/4079782599] conn(0x7f40a0069210 0x7f40a006b6c0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:03:27.533 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:03:27.533+0000 7f40bb725700 1 --2- 172.21.15.162:0/503341913 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f40b4066b50 0x7f40b410da30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:03:27.533 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:03:27.533+0000 7f40bb725700 1 --2- 172.21.15.162:0/503341913 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f40b40ffef0 0x7f40b410df70 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:03:27.533 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:03:27.533+0000 7f40bb725700 1 -- 172.21.15.162:0/503341913 >> 172.21.15.162:0/503341913 conn(0x7f40b40f7420 msgr2=0x7f40b40f7840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:03:27.533 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:03:27.533+0000 7f40bb725700 1 -- 172.21.15.162:0/503341913 shutdown_connections 2024-04-27T04:03:27.533 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:03:27.533+0000 7f40bb725700 1 -- 172.21.15.162:0/503341913 wait complete. 2024-04-27T04:03:28.059 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:03:27 smithi162 ceph-mon[26841]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-27T04:03:28.059 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:03:27 smithi162 ceph-mon[26841]: from='osd.0 [v2:172.21.15.162:6802/3784722138,v1:172.21.15.162:6803/3784722138]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-04-27T04:03:28.059 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:03:27 smithi162 ceph-mon[26841]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' 2024-04-27T04:03:28.059 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:03:27 smithi162 ceph-mon[26841]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' 2024-04-27T04:03:28.086 DEBUG:teuthology.orchestra.run.smithi162:osd.0> sudo journalctl -f -n 0 -u ceph-7f9750d6-044a-11ef-bc93-c7b262605968@osd.0.service 2024-04-27T04:03:28.089 INFO:tasks.cephadm:Deploying osd.1 on smithi162 with /dev/vg_nvme/lv_3... 2024-04-27T04:03:28.089 DEBUG:teuthology.orchestra.run.smithi162:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 7f9750d6-044a-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-04-27T04:03:28.173 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:03:27 smithi181 ceph-mon[33774]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-27T04:03:28.173 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:03:27 smithi181 ceph-mon[33774]: from='osd.0 [v2:172.21.15.162:6802/3784722138,v1:172.21.15.162:6803/3784722138]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-04-27T04:03:28.174 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:03:27 smithi181 ceph-mon[33774]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' 2024-04-27T04:03:28.174 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:03:27 smithi181 ceph-mon[33774]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' 2024-04-27T04:03:28.728 INFO:teuthology.orchestra.run.smithi162.stderr:Inferring config /var/lib/ceph/7f9750d6-044a-11ef-bc93-c7b262605968/mon.smithi162/config 2024-04-27T04:03:28.997 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:03:28 smithi162 ceph-mon[26841]: from='osd.0 [v2:172.21.15.162:6802/3784722138,v1:172.21.15.162:6803/3784722138]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-04-27T04:03:28.997 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:03:28 smithi162 ceph-mon[26841]: osdmap e7: 1 total, 0 up, 1 in 2024-04-27T04:03:28.997 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:03:28 smithi162 ceph-mon[26841]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-27T04:03:28.997 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:03:28 smithi162 ceph-mon[26841]: from='osd.0 [v2:172.21.15.162:6802/3784722138,v1:172.21.15.162:6803/3784722138]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi162", "root=default"]}]: dispatch 2024-04-27T04:03:28.998 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:03:28 smithi162 ceph-mon[26841]: from='osd.0 [v2:172.21.15.162:6802/3784722138,v1:172.21.15.162:6803/3784722138]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi162", "root=default"]}]': finished 2024-04-27T04:03:28.998 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:03:28 smithi162 ceph-mon[26841]: osdmap e8: 1 total, 0 up, 1 in 2024-04-27T04:03:28.998 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:03:28 smithi162 ceph-mon[26841]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-27T04:03:28.998 INFO:journalctl@ceph.osd.0.smithi162.stdout:Apr 27 04:03:28 smithi162 ceph-7f9750d6-044a-11ef-bc93-c7b262605968-osd-0[45256]: 2024-04-27T04:03:28.735+0000 7f1a85db2700 -1 osd.0 0 waiting for initial osdmap 2024-04-27T04:03:28.998 INFO:journalctl@ceph.osd.0.smithi162.stdout:Apr 27 04:03:28 smithi162 ceph-7f9750d6-044a-11ef-bc93-c7b262605968-osd-0[45256]: 2024-04-27T04:03:28.741+0000 7f1a7fbab700 -1 osd.0 8 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-27T04:03:29.173 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:03:28 smithi181 ceph-mon[33774]: from='osd.0 [v2:172.21.15.162:6802/3784722138,v1:172.21.15.162:6803/3784722138]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-04-27T04:03:29.173 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:03:28 smithi181 ceph-mon[33774]: osdmap e7: 1 total, 0 up, 1 in 2024-04-27T04:03:29.174 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:03:28 smithi181 ceph-mon[33774]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-27T04:03:29.174 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:03:28 smithi181 ceph-mon[33774]: from='osd.0 [v2:172.21.15.162:6802/3784722138,v1:172.21.15.162:6803/3784722138]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi162", "root=default"]}]: dispatch 2024-04-27T04:03:29.174 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:03:28 smithi181 ceph-mon[33774]: from='osd.0 [v2:172.21.15.162:6802/3784722138,v1:172.21.15.162:6803/3784722138]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi162", "root=default"]}]': finished 2024-04-27T04:03:29.174 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:03:28 smithi181 ceph-mon[33774]: osdmap e8: 1 total, 0 up, 1 in 2024-04-27T04:03:29.174 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:03:28 smithi181 ceph-mon[33774]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-27T04:03:30.003 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:03:29 smithi162 ceph-mon[26841]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-27T04:03:30.004 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:03:29 smithi162 ceph-mon[26841]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-27T04:03:30.004 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:03:29 smithi162 ceph-mon[26841]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' 2024-04-27T04:03:30.004 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:03:29 smithi162 ceph-mon[26841]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' 2024-04-27T04:03:30.004 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:03:29 smithi162 ceph-mon[26841]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-27T04:03:30.173 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:03:29 smithi181 ceph-mon[33774]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-27T04:03:30.173 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:03:29 smithi181 ceph-mon[33774]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-27T04:03:30.173 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:03:29 smithi181 ceph-mon[33774]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' 2024-04-27T04:03:30.174 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:03:29 smithi181 ceph-mon[33774]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' 2024-04-27T04:03:30.174 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:03:29 smithi181 ceph-mon[33774]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-27T04:03:30.707 INFO:teuthology.orchestra.run.smithi162.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-04-27T04:03:30.707 INFO:teuthology.orchestra.run.smithi162.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-04-27T04:03:30.707 INFO:teuthology.orchestra.run.smithi162.stderr: stderr: 10+0 records in 2024-04-27T04:03:30.707 INFO:teuthology.orchestra.run.smithi162.stderr:10+0 records out 2024-04-27T04:03:30.708 INFO:teuthology.orchestra.run.smithi162.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.013569 s, 773 MB/s 2024-04-27T04:03:30.708 INFO:teuthology.orchestra.run.smithi162.stderr:--> Zapping successful for: 2024-04-27T04:03:30.877 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:03:30 smithi162 ceph-mon[26841]: purged_snaps scrub starts 2024-04-27T04:03:30.878 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:03:30 smithi162 ceph-mon[26841]: purged_snaps scrub ok 2024-04-27T04:03:30.878 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:03:30 smithi162 ceph-mon[26841]: osd.0 [v2:172.21.15.162:6802/3784722138,v1:172.21.15.162:6803/3784722138] boot 2024-04-27T04:03:30.878 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:03:30 smithi162 ceph-mon[26841]: osdmap e9: 1 total, 1 up, 1 in 2024-04-27T04:03:30.878 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:03:30 smithi162 ceph-mon[26841]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-27T04:03:31.075 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:03:30 smithi181 ceph-mon[33774]: purged_snaps scrub starts 2024-04-27T04:03:31.075 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:03:30 smithi181 ceph-mon[33774]: purged_snaps scrub ok 2024-04-27T04:03:31.075 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:03:30 smithi181 ceph-mon[33774]: osd.0 [v2:172.21.15.162:6802/3784722138,v1:172.21.15.162:6803/3784722138] boot 2024-04-27T04:03:31.075 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:03:30 smithi181 ceph-mon[33774]: osdmap e9: 1 total, 1 up, 1 in 2024-04-27T04:03:31.075 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:03:30 smithi181 ceph-mon[33774]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-27T04:03:31.844 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:03:31 smithi162 ceph-mon[26841]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-27T04:03:31.844 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:03:31 smithi162 ceph-mon[26841]: osdmap e10: 1 total, 1 up, 1 in 2024-04-27T04:03:31.844 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:03:31 smithi162 ceph-mon[26841]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-27T04:03:32.087 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:03:31 smithi181 ceph-mon[33774]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-27T04:03:32.087 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:03:31 smithi181 ceph-mon[33774]: osdmap e10: 1 total, 1 up, 1 in 2024-04-27T04:03:32.087 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:03:31 smithi181 ceph-mon[33774]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-27T04:03:32.136 DEBUG:teuthology.orchestra.run.smithi162:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 7f9750d6-044a-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi162:vg_nvme/lv_3 2024-04-27T04:03:32.704 INFO:teuthology.orchestra.run.smithi162.stderr:Inferring config /var/lib/ceph/7f9750d6-044a-11ef-bc93-c7b262605968/mon.smithi162/config 2024-04-27T04:03:33.930 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:03:33 smithi162 ceph-mon[26841]: pgmap v36: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-04-27T04:03:34.173 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:03:33 smithi181 ceph-mon[33774]: pgmap v36: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-04-27T04:03:34.925 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:03:34.924+0000 7f524596c700 1 -- 172.21.15.162:0/144984902 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5240066af0 msgr2=0x7f52400ffa90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:03:34.925 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:03:34.924+0000 7f524596c700 1 --2- 172.21.15.162:0/144984902 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5240066af0 0x7f52400ffa90 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f5230004940 tx=0x7f5230009e30 comp rx=0 tx=0).stop 2024-04-27T04:03:34.925 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:03:34.924+0000 7f524596c700 1 -- 172.21.15.162:0/144984902 shutdown_connections 2024-04-27T04:03:34.925 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:03:34.924+0000 7f524596c700 1 --2- 172.21.15.162:0/144984902 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f52400fffd0 0x7f52401023b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:03:34.925 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:03:34.924+0000 7f524596c700 1 --2- 172.21.15.162:0/144984902 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5240066af0 0x7f52400ffa90 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:03:34.926 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:03:34.924+0000 7f524596c700 1 -- 172.21.15.162:0/144984902 >> 172.21.15.162:0/144984902 conn(0x7f52400f74c0 msgr2=0x7f52400f98d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:03:34.926 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:03:34.924+0000 7f524596c700 1 -- 172.21.15.162:0/144984902 shutdown_connections 2024-04-27T04:03:34.926 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:03:34.924+0000 7f524596c700 1 -- 172.21.15.162:0/144984902 wait complete. 2024-04-27T04:03:34.926 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:03:34.924+0000 7f524596c700 1 Processor -- start 2024-04-27T04:03:34.926 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:03:34.924+0000 7f524596c700 1 -- start start 2024-04-27T04:03:34.926 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:03:34.925+0000 7f524596c700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5240066af0 0x7f524010dac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:03:34.926 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:03:34.925+0000 7f524596c700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f52400fffd0 0x7f524010e000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:03:34.926 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:03:34.925+0000 7f524596c700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f524010e690 con 0x7f52400fffd0 2024-04-27T04:03:34.926 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:03:34.925+0000 7f524596c700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f524010ee30 con 0x7f5240066af0 2024-04-27T04:03:34.926 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:03:34.925+0000 7f523e7fc700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f52400fffd0 0x7f524010e000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:03:34.926 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:03:34.925+0000 7f523e7fc700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f52400fffd0 0x7f524010e000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:51752/0 (socket says 172.21.15.162:51752) 2024-04-27T04:03:34.926 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:03:34.925+0000 7f523e7fc700 1 -- 172.21.15.162:0/615560953 learned_addr learned my addr 172.21.15.162:0/615560953 (peer_addr_for_me v2:172.21.15.162:0/0) 2024-04-27T04:03:34.926 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:03:34.925+0000 7f523effd700 1 --2- 172.21.15.162:0/615560953 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5240066af0 0x7f524010dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:03:34.926 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:03:34.925+0000 7f523e7fc700 1 -- 172.21.15.162:0/615560953 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5240066af0 msgr2=0x7f524010dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:03:34.927 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:03:34.925+0000 7f523e7fc700 1 --2- 172.21.15.162:0/615560953 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5240066af0 0x7f524010dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:03:34.927 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:03:34.925+0000 7f523e7fc700 1 -- 172.21.15.162:0/615560953 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5230005040 con 0x7f52400fffd0 2024-04-27T04:03:34.927 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:03:34.925+0000 7f523e7fc700 1 --2- 172.21.15.162:0/615560953 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f52400fffd0 0x7f524010e000 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f523400b790 tx=0x7f5234009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:03:34.927 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:03:34.925+0000 7f52277fe700 1 -- 172.21.15.162:0/615560953 <== mon.0 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f523400f770 con 0x7f52400fffd0 2024-04-27T04:03:34.927 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:03:34.925+0000 7f52277fe700 1 -- 172.21.15.162:0/615560953 <== mon.0 v2:172.21.15.162:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f523400fda0 con 0x7f52400fffd0 2024-04-27T04:03:34.927 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:03:34.926+0000 7f524596c700 1 -- 172.21.15.162:0/615560953 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f524010f110 con 0x7f52400fffd0 2024-04-27T04:03:34.927 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:03:34.926+0000 7f52277fe700 1 -- 172.21.15.162:0/615560953 <== mon.0 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f523400d310 con 0x7f52400fffd0 2024-04-27T04:03:34.927 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:03:34.926+0000 7f524596c700 1 -- 172.21.15.162:0/615560953 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f524010f6d0 con 0x7f52400fffd0 2024-04-27T04:03:34.928 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:03:34.927+0000 7f524596c700 1 -- 172.21.15.162:0/615560953 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f524005f5c0 con 0x7f52400fffd0 2024-04-27T04:03:34.928 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:03:34.927+0000 7f52277fe700 1 -- 172.21.15.162:0/615560953 <== mon.0 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 18) v1 ==== 86575+0+0 (secure 0 0 0) 0x7f5234010420 con 0x7f52400fffd0 2024-04-27T04:03:34.928 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:03:34.928+0000 7f52277fe700 1 --2- 172.21.15.162:0/615560953 >> [v2:172.21.15.162:6800/4079782599,v1:172.21.15.162:6801/4079782599] conn(0x7f5228069210 0x7f522806b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:03:34.928 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:03:34.928+0000 7f52277fe700 1 -- 172.21.15.162:0/615560953 <== mon.0 v2:172.21.15.162:3300/0 5 ==== osd_map(10..10 src has 1..10) v4 ==== 1978+0+0 (secure 0 0 0) 0x7f52340861f0 con 0x7f52400fffd0 2024-04-27T04:03:34.928 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:03:34.928+0000 7f523effd700 1 --2- 172.21.15.162:0/615560953 >> [v2:172.21.15.162:6800/4079782599,v1:172.21.15.162:6801/4079782599] conn(0x7f5228069210 0x7f522806b6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:03:34.928 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:03:34.928+0000 7f523effd700 1 --2- 172.21.15.162:0/615560953 >> [v2:172.21.15.162:6800/4079782599,v1:172.21.15.162:6801/4079782599] conn(0x7f5228069210 0x7f522806b6c0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f523000b970 tx=0x7f5230009e30 comp rx=0 tx=0).ready entity=mgr.14210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:03:34.932 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:03:34.932+0000 7f52277fe700 1 -- 172.21.15.162:0/615560953 <== mon.0 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f5234056ed0 con 0x7f52400fffd0 2024-04-27T04:03:35.146 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:03:35.145+0000 7f524596c700 1 -- 172.21.15.162:0/615560953 --> [v2:172.21.15.162:6800/4079782599,v1:172.21.15.162:6801/4079782599] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi162:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) v1 -- 0x7f524010fa00 con 0x7f5228069210 2024-04-27T04:03:35.292 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:03:34 smithi162 ceph-mon[26841]: Detected new or changed devices on smithi162 2024-04-27T04:03:35.292 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:03:34 smithi162 ceph-mon[26841]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' 2024-04-27T04:03:35.292 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:03:34 smithi162 ceph-mon[26841]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' 2024-04-27T04:03:35.292 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:03:34 smithi162 ceph-mon[26841]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-27T04:03:35.293 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:03:34 smithi162 ceph-mon[26841]: Adjusting osd_memory_target on smithi162 to 12864M 2024-04-27T04:03:35.293 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:03:34 smithi162 ceph-mon[26841]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' 2024-04-27T04:03:35.293 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:03:34 smithi162 ceph-mon[26841]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T04:03:35.293 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:03:34 smithi162 ceph-mon[26841]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T04:03:35.293 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:03:34 smithi162 ceph-mon[26841]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' 2024-04-27T04:03:35.293 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:03:34 smithi162 ceph-mon[26841]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T04:03:35.330 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:03:34 smithi181 ceph-mon[33774]: Detected new or changed devices on smithi162 2024-04-27T04:03:35.330 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:03:34 smithi181 ceph-mon[33774]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' 2024-04-27T04:03:35.330 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:03:34 smithi181 ceph-mon[33774]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' 2024-04-27T04:03:35.330 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:03:34 smithi181 ceph-mon[33774]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-27T04:03:35.330 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:03:34 smithi181 ceph-mon[33774]: Adjusting osd_memory_target on smithi162 to 12864M 2024-04-27T04:03:35.330 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:03:34 smithi181 ceph-mon[33774]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' 2024-04-27T04:03:35.330 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:03:34 smithi181 ceph-mon[33774]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T04:03:35.330 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:03:34 smithi181 ceph-mon[33774]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T04:03:35.330 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:03:34 smithi181 ceph-mon[33774]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' 2024-04-27T04:03:35.331 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:03:34 smithi181 ceph-mon[33774]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T04:03:36.004 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:03:35 smithi162 ceph-mon[26841]: pgmap v37: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-04-27T04:03:36.004 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:03:35 smithi162 ceph-mon[26841]: from='client.14264 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi162:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-27T04:03:36.005 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:03:35 smithi162 ceph-mon[26841]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-27T04:03:36.005 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:03:35 smithi162 ceph-mon[26841]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-27T04:03:36.005 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:03:35 smithi162 ceph-mon[26841]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T04:03:36.005 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:03:35 smithi162 ceph-mon[26841]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' 2024-04-27T04:03:36.005 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:03:35 smithi162 ceph-mon[26841]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' 2024-04-27T04:03:36.423 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:03:35 smithi181 ceph-mon[33774]: pgmap v37: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-04-27T04:03:36.423 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:03:35 smithi181 ceph-mon[33774]: from='client.14264 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi162:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-27T04:03:36.423 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:03:35 smithi181 ceph-mon[33774]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-27T04:03:36.424 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:03:35 smithi181 ceph-mon[33774]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-27T04:03:36.424 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:03:35 smithi181 ceph-mon[33774]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T04:03:36.424 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:03:35 smithi181 ceph-mon[33774]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' 2024-04-27T04:03:36.424 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:03:35 smithi181 ceph-mon[33774]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' 2024-04-27T04:03:38.258 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:03:37 smithi162 ceph-mon[26841]: pgmap v38: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-04-27T04:03:38.259 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:03:37 smithi162 ceph-mon[26841]: from='client.? 172.21.15.162:0/539591541' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "95bdc9f0-c438-4a88-942d-dffb0c61c308"}]: dispatch 2024-04-27T04:03:38.259 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:03:37 smithi162 ceph-mon[26841]: from='client.? 172.21.15.162:0/539591541' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "95bdc9f0-c438-4a88-942d-dffb0c61c308"}]': finished 2024-04-27T04:03:38.259 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:03:37 smithi162 ceph-mon[26841]: osdmap e11: 2 total, 1 up, 2 in 2024-04-27T04:03:38.259 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:03:37 smithi162 ceph-mon[26841]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-27T04:03:38.423 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:03:37 smithi181 ceph-mon[33774]: pgmap v38: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-04-27T04:03:38.424 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:03:37 smithi181 ceph-mon[33774]: from='client.? 172.21.15.162:0/539591541' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "95bdc9f0-c438-4a88-942d-dffb0c61c308"}]: dispatch 2024-04-27T04:03:38.424 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:03:37 smithi181 ceph-mon[33774]: from='client.? 172.21.15.162:0/539591541' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "95bdc9f0-c438-4a88-942d-dffb0c61c308"}]': finished 2024-04-27T04:03:38.424 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:03:37 smithi181 ceph-mon[33774]: osdmap e11: 2 total, 1 up, 2 in 2024-04-27T04:03:38.424 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:03:37 smithi181 ceph-mon[33774]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-27T04:03:39.079 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:03:38 smithi162 ceph-mon[26841]: from='client.? 172.21.15.162:0/1563605446' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-27T04:03:39.423 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:03:38 smithi181 ceph-mon[33774]: from='client.? 172.21.15.162:0/1563605446' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-27T04:03:40.138 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:03:40 smithi162 ceph-mon[26841]: pgmap v40: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-04-27T04:03:40.423 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:03:40 smithi181 ceph-mon[33774]: pgmap v40: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-04-27T04:03:41.388 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:03:41 smithi162 ceph-mon[26841]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' 2024-04-27T04:03:41.388 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:03:41 smithi162 ceph-mon[26841]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' 2024-04-27T04:03:41.388 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:03:41 smithi162 ceph-mon[26841]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T04:03:41.388 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:03:41 smithi162 ceph-mon[26841]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T04:03:41.388 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:03:41 smithi162 ceph-mon[26841]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' 2024-04-27T04:03:41.423 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:03:41 smithi181 ceph-mon[33774]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' 2024-04-27T04:03:41.423 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:03:41 smithi181 ceph-mon[33774]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' 2024-04-27T04:03:41.423 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:03:41 smithi181 ceph-mon[33774]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T04:03:41.423 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:03:41 smithi181 ceph-mon[33774]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T04:03:41.424 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:03:41 smithi181 ceph-mon[33774]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' 2024-04-27T04:03:42.388 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:03:42 smithi162 ceph-mon[26841]: pgmap v41: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-04-27T04:03:42.423 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:03:42 smithi181 ceph-mon[33774]: pgmap v41: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-04-27T04:03:44.388 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:03:44 smithi162 ceph-mon[26841]: pgmap v42: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-04-27T04:03:44.423 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:03:44 smithi181 ceph-mon[33774]: pgmap v42: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-04-27T04:03:45.383 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:03:45 smithi162 ceph-mon[26841]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-04-27T04:03:45.384 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:03:45 smithi162 ceph-mon[26841]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T04:03:45.423 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:03:45 smithi181 ceph-mon[33774]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-04-27T04:03:45.423 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:03:45 smithi181 ceph-mon[33774]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T04:03:46.388 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:03:46 smithi162 ceph-mon[26841]: pgmap v43: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-04-27T04:03:46.388 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:03:46 smithi162 ceph-mon[26841]: Deploying daemon osd.1 on smithi162 2024-04-27T04:03:46.423 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:03:46 smithi181 ceph-mon[33774]: pgmap v43: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-04-27T04:03:46.423 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:03:46 smithi181 ceph-mon[33774]: Deploying daemon osd.1 on smithi162 2024-04-27T04:03:47.388 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:03:47 smithi162 ceph-mon[26841]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-27T04:03:47.423 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:03:47 smithi181 ceph-mon[33774]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-27T04:03:48.183 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:03:48 smithi162 ceph-mon[26841]: pgmap v44: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-04-27T04:03:48.423 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:03:48 smithi181 ceph-mon[33774]: pgmap v44: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-04-27T04:03:50.388 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:03:50 smithi162 ceph-mon[26841]: pgmap v45: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-04-27T04:03:50.388 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:03:50 smithi162 ceph-mon[26841]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' 2024-04-27T04:03:50.388 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:03:50 smithi162 ceph-mon[26841]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' 2024-04-27T04:03:50.388 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:03:50 smithi162 ceph-mon[26841]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T04:03:50.423 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:03:50 smithi181 ceph-mon[33774]: pgmap v45: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-04-27T04:03:50.423 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:03:50 smithi181 ceph-mon[33774]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' 2024-04-27T04:03:50.423 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:03:50 smithi181 ceph-mon[33774]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' 2024-04-27T04:03:50.423 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:03:50 smithi181 ceph-mon[33774]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T04:03:52.299 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:03:52 smithi162 ceph-mon[26841]: pgmap v46: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-04-27T04:03:52.299 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:03:52 smithi162 ceph-mon[26841]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' 2024-04-27T04:03:52.299 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:03:52 smithi162 ceph-mon[26841]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' 2024-04-27T04:03:52.423 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:03:52 smithi181 ceph-mon[33774]: pgmap v46: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-04-27T04:03:52.424 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:03:52 smithi181 ceph-mon[33774]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' 2024-04-27T04:03:52.424 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:03:52 smithi181 ceph-mon[33774]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' 2024-04-27T04:03:52.963 INFO:teuthology.orchestra.run.smithi162.stdout:Created osd(s) 1 on host 'smithi162' 2024-04-27T04:03:52.963 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:03:52.961+0000 7f52277fe700 1 -- 172.21.15.162:0/615560953 <== mgr.14210 v2:172.21.15.162:6800/4079782599 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f524010fa00 con 0x7f5228069210 2024-04-27T04:03:52.964 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:03:52.964+0000 7f524596c700 1 -- 172.21.15.162:0/615560953 >> [v2:172.21.15.162:6800/4079782599,v1:172.21.15.162:6801/4079782599] conn(0x7f5228069210 msgr2=0x7f522806b6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:03:52.964 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:03:52.964+0000 7f524596c700 1 --2- 172.21.15.162:0/615560953 >> [v2:172.21.15.162:6800/4079782599,v1:172.21.15.162:6801/4079782599] conn(0x7f5228069210 0x7f522806b6c0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f523000b970 tx=0x7f5230009e30 comp rx=0 tx=0).stop 2024-04-27T04:03:52.964 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:03:52.964+0000 7f524596c700 1 -- 172.21.15.162:0/615560953 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f52400fffd0 msgr2=0x7f524010e000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:03:52.965 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:03:52.964+0000 7f524596c700 1 --2- 172.21.15.162:0/615560953 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f52400fffd0 0x7f524010e000 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f523400b790 tx=0x7f5234009250 comp rx=0 tx=0).stop 2024-04-27T04:03:52.965 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:03:52.964+0000 7f524596c700 1 -- 172.21.15.162:0/615560953 shutdown_connections 2024-04-27T04:03:52.965 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:03:52.964+0000 7f524596c700 1 --2- 172.21.15.162:0/615560953 >> [v2:172.21.15.162:6800/4079782599,v1:172.21.15.162:6801/4079782599] conn(0x7f5228069210 0x7f522806b6c0 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:03:52.965 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:03:52.964+0000 7f524596c700 1 --2- 172.21.15.162:0/615560953 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5240066af0 0x7f524010dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:03:52.965 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:03:52.964+0000 7f524596c700 1 --2- 172.21.15.162:0/615560953 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f52400fffd0 0x7f524010e000 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:03:52.965 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:03:52.964+0000 7f524596c700 1 -- 172.21.15.162:0/615560953 >> 172.21.15.162:0/615560953 conn(0x7f52400f74c0 msgr2=0x7f52400f7890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:03:52.965 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:03:52.964+0000 7f524596c700 1 -- 172.21.15.162:0/615560953 shutdown_connections 2024-04-27T04:03:52.965 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:03:52.964+0000 7f524596c700 1 -- 172.21.15.162:0/615560953 wait complete. 2024-04-27T04:03:53.755 DEBUG:teuthology.orchestra.run.smithi162:osd.1> sudo journalctl -f -n 0 -u ceph-7f9750d6-044a-11ef-bc93-c7b262605968@osd.1.service 2024-04-27T04:03:53.758 INFO:tasks.cephadm:Deploying osd.2 on smithi162 with /dev/vg_nvme/lv_2... 2024-04-27T04:03:53.758 DEBUG:teuthology.orchestra.run.smithi162:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 7f9750d6-044a-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-04-27T04:03:54.173 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:03:53 smithi181 ceph-mon[33774]: pgmap v47: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-04-27T04:03:54.173 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:03:53 smithi181 ceph-mon[33774]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' 2024-04-27T04:03:54.173 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:03:53 smithi181 ceph-mon[33774]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' 2024-04-27T04:03:54.173 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:03:53 smithi181 ceph-mon[33774]: from='osd.1 [v2:172.21.15.162:6810/3642637124,v1:172.21.15.162:6811/3642637124]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-04-27T04:03:54.201 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:03:53 smithi162 ceph-mon[26841]: pgmap v47: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-04-27T04:03:54.201 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:03:53 smithi162 ceph-mon[26841]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' 2024-04-27T04:03:54.201 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:03:53 smithi162 ceph-mon[26841]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' 2024-04-27T04:03:54.201 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:03:53 smithi162 ceph-mon[26841]: from='osd.1 [v2:172.21.15.162:6810/3642637124,v1:172.21.15.162:6811/3642637124]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-04-27T04:03:54.495 INFO:teuthology.orchestra.run.smithi162.stderr:Inferring config /var/lib/ceph/7f9750d6-044a-11ef-bc93-c7b262605968/mon.smithi162/config 2024-04-27T04:03:55.423 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:03:55 smithi181 ceph-mon[33774]: from='osd.1 [v2:172.21.15.162:6810/3642637124,v1:172.21.15.162:6811/3642637124]' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-04-27T04:03:55.423 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:03:55 smithi181 ceph-mon[33774]: osdmap e12: 2 total, 1 up, 2 in 2024-04-27T04:03:55.423 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:03:55 smithi181 ceph-mon[33774]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-27T04:03:55.424 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:03:55 smithi181 ceph-mon[33774]: from='osd.1 [v2:172.21.15.162:6810/3642637124,v1:172.21.15.162:6811/3642637124]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi162", "root=default"]}]: dispatch 2024-04-27T04:03:55.450 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:03:55 smithi162 ceph-mon[26841]: from='osd.1 [v2:172.21.15.162:6810/3642637124,v1:172.21.15.162:6811/3642637124]' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-04-27T04:03:55.450 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:03:55 smithi162 ceph-mon[26841]: osdmap e12: 2 total, 1 up, 2 in 2024-04-27T04:03:55.450 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:03:55 smithi162 ceph-mon[26841]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-27T04:03:55.450 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:03:55 smithi162 ceph-mon[26841]: from='osd.1 [v2:172.21.15.162:6810/3642637124,v1:172.21.15.162:6811/3642637124]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi162", "root=default"]}]: dispatch 2024-04-27T04:03:55.450 INFO:journalctl@ceph.osd.1.smithi162.stdout:Apr 27 04:03:55 smithi162 ceph-7f9750d6-044a-11ef-bc93-c7b262605968-osd-1[52837]: 2024-04-27T04:03:55.148+0000 7fbcb904f700 -1 osd.1 0 waiting for initial osdmap 2024-04-27T04:03:55.450 INFO:journalctl@ceph.osd.1.smithi162.stdout:Apr 27 04:03:55 smithi162 ceph-7f9750d6-044a-11ef-bc93-c7b262605968-osd-1[52837]: 2024-04-27T04:03:55.152+0000 7fbcb3649700 -1 osd.1 13 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-27T04:03:56.368 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:03:56 smithi162 ceph-mon[26841]: pgmap v49: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-04-27T04:03:56.368 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:03:56 smithi162 ceph-mon[26841]: from='osd.1 [v2:172.21.15.162:6810/3642637124,v1:172.21.15.162:6811/3642637124]' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi162", "root=default"]}]': finished 2024-04-27T04:03:56.368 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:03:56 smithi162 ceph-mon[26841]: osdmap e13: 2 total, 1 up, 2 in 2024-04-27T04:03:56.368 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:03:56 smithi162 ceph-mon[26841]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-27T04:03:56.368 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:03:56 smithi162 ceph-mon[26841]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-27T04:03:56.423 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:03:56 smithi181 ceph-mon[33774]: pgmap v49: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-04-27T04:03:56.423 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:03:56 smithi181 ceph-mon[33774]: from='osd.1 [v2:172.21.15.162:6810/3642637124,v1:172.21.15.162:6811/3642637124]' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi162", "root=default"]}]': finished 2024-04-27T04:03:56.423 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:03:56 smithi181 ceph-mon[33774]: osdmap e13: 2 total, 1 up, 2 in 2024-04-27T04:03:56.424 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:03:56 smithi181 ceph-mon[33774]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-27T04:03:56.424 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:03:56 smithi181 ceph-mon[33774]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-27T04:03:56.943 INFO:teuthology.orchestra.run.smithi162.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-04-27T04:03:56.944 INFO:teuthology.orchestra.run.smithi162.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-04-27T04:03:56.944 INFO:teuthology.orchestra.run.smithi162.stderr: stderr: 10+0 records in 2024-04-27T04:03:56.944 INFO:teuthology.orchestra.run.smithi162.stderr:10+0 records out 2024-04-27T04:03:56.944 INFO:teuthology.orchestra.run.smithi162.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0131801 s, 796 MB/s 2024-04-27T04:03:56.944 INFO:teuthology.orchestra.run.smithi162.stderr:--> Zapping successful for: 2024-04-27T04:03:57.423 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:03:57 smithi181 ceph-mon[33774]: purged_snaps scrub starts 2024-04-27T04:03:57.424 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:03:57 smithi181 ceph-mon[33774]: purged_snaps scrub ok 2024-04-27T04:03:57.424 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:03:57 smithi181 ceph-mon[33774]: osd.1 [v2:172.21.15.162:6810/3642637124,v1:172.21.15.162:6811/3642637124] boot 2024-04-27T04:03:57.424 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:03:57 smithi181 ceph-mon[33774]: osdmap e14: 2 total, 2 up, 2 in 2024-04-27T04:03:57.424 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:03:57 smithi181 ceph-mon[33774]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-27T04:03:57.522 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:03:57 smithi162 ceph-mon[26841]: purged_snaps scrub starts 2024-04-27T04:03:57.522 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:03:57 smithi162 ceph-mon[26841]: purged_snaps scrub ok 2024-04-27T04:03:57.522 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:03:57 smithi162 ceph-mon[26841]: osd.1 [v2:172.21.15.162:6810/3642637124,v1:172.21.15.162:6811/3642637124] boot 2024-04-27T04:03:57.522 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:03:57 smithi162 ceph-mon[26841]: osdmap e14: 2 total, 2 up, 2 in 2024-04-27T04:03:57.522 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:03:57 smithi162 ceph-mon[26841]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-27T04:03:57.546 DEBUG:teuthology.orchestra.run.smithi162:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 7f9750d6-044a-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi162:vg_nvme/lv_2 2024-04-27T04:03:57.918 INFO:teuthology.orchestra.run.smithi162.stderr:Inferring config /var/lib/ceph/7f9750d6-044a-11ef-bc93-c7b262605968/mon.smithi162/config 2024-04-27T04:03:58.328 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:03:58 smithi162 ceph-mon[26841]: pgmap v52: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-04-27T04:03:58.328 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:03:58 smithi162 ceph-mon[26841]: osdmap e15: 2 total, 2 up, 2 in 2024-04-27T04:03:58.423 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:03:58 smithi181 ceph-mon[33774]: pgmap v52: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-04-27T04:03:58.423 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:03:58 smithi181 ceph-mon[33774]: osdmap e15: 2 total, 2 up, 2 in 2024-04-27T04:04:00.105 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:04:00.103+0000 7ff329973700 1 -- 172.21.15.162:0/2158099675 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff3240fe420 msgr2=0x7ff3240fe830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:04:00.105 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:04:00.103+0000 7ff329973700 1 --2- 172.21.15.162:0/2158099675 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff3240fe420 0x7ff3240fe830 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7ff314004940 tx=0x7ff314009e30 comp rx=0 tx=0).stop 2024-04-27T04:04:00.105 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:04:00.103+0000 7ff329973700 1 -- 172.21.15.162:0/2158099675 shutdown_connections 2024-04-27T04:04:00.105 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:04:00.103+0000 7ff329973700 1 --2- 172.21.15.162:0/2158099675 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff3240fed70 0x7ff3240faeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:04:00.105 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:04:00.103+0000 7ff329973700 1 --2- 172.21.15.162:0/2158099675 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff3240fe420 0x7ff3240fe830 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:04:00.105 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:04:00.103+0000 7ff329973700 1 -- 172.21.15.162:0/2158099675 >> 172.21.15.162:0/2158099675 conn(0x7ff3240f6be0 msgr2=0x7ff3240f8ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:04:00.105 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:04:00.104+0000 7ff329973700 1 -- 172.21.15.162:0/2158099675 shutdown_connections 2024-04-27T04:04:00.105 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:04:00.104+0000 7ff329973700 1 -- 172.21.15.162:0/2158099675 wait complete. 2024-04-27T04:04:00.105 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:04:00.104+0000 7ff329973700 1 Processor -- start 2024-04-27T04:04:00.105 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:04:00.104+0000 7ff329973700 1 -- start start 2024-04-27T04:04:00.106 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:04:00.104+0000 7ff329973700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff3240fe420 0x7ff3240fda30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:04:00.106 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:04:00.104+0000 7ff329973700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff3240fed70 0x7ff3240fc0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:04:00.106 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:04:00.104+0000 7ff329973700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3240fe030 con 0x7ff3240fed70 2024-04-27T04:04:00.106 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:04:00.104+0000 7ff329973700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3240fc620 con 0x7ff3240fe420 2024-04-27T04:04:00.106 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:04:00.104+0000 7ff3227fc700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff3240fed70 0x7ff3240fc0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:04:00.106 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:04:00.104+0000 7ff3227fc700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff3240fed70 0x7ff3240fc0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:51294/0 (socket says 172.21.15.162:51294) 2024-04-27T04:04:00.106 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:04:00.104+0000 7ff3227fc700 1 -- 172.21.15.162:0/2127201807 learned_addr learned my addr 172.21.15.162:0/2127201807 (peer_addr_for_me v2:172.21.15.162:0/0) 2024-04-27T04:04:00.106 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:04:00.105+0000 7ff322ffd700 1 --2- 172.21.15.162:0/2127201807 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff3240fe420 0x7ff3240fda30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:04:00.106 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:04:00.105+0000 7ff3227fc700 1 -- 172.21.15.162:0/2127201807 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff3240fe420 msgr2=0x7ff3240fda30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:04:00.106 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:04:00.105+0000 7ff3227fc700 1 --2- 172.21.15.162:0/2127201807 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff3240fe420 0x7ff3240fda30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:04:00.106 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:04:00.105+0000 7ff3227fc700 1 -- 172.21.15.162:0/2127201807 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff314005040 con 0x7ff3240fed70 2024-04-27T04:04:00.106 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:04:00.105+0000 7ff3227fc700 1 --2- 172.21.15.162:0/2127201807 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff3240fed70 0x7ff3240fc0e0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7ff31800c830 tx=0x7ff318004470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:04:00.106 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:04:00.105+0000 7ff30b7fe700 1 -- 172.21.15.162:0/2127201807 <== mon.0 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff31800a760 con 0x7ff3240fed70 2024-04-27T04:04:00.107 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:04:00.105+0000 7ff329973700 1 -- 172.21.15.162:0/2127201807 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff3240fc8d0 con 0x7ff3240fed70 2024-04-27T04:04:00.107 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:04:00.105+0000 7ff30b7fe700 1 -- 172.21.15.162:0/2127201807 <== mon.0 v2:172.21.15.162:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7ff31800ad90 con 0x7ff3240fed70 2024-04-27T04:04:00.107 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:04:00.105+0000 7ff30b7fe700 1 -- 172.21.15.162:0/2127201807 <== mon.0 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff3180113a0 con 0x7ff3240fed70 2024-04-27T04:04:00.107 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:04:00.105+0000 7ff329973700 1 -- 172.21.15.162:0/2127201807 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff3240fce90 con 0x7ff3240fed70 2024-04-27T04:04:00.108 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:04:00.106+0000 7ff329973700 1 -- 172.21.15.162:0/2127201807 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff32404c710 con 0x7ff3240fed70 2024-04-27T04:04:00.108 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:04:00.107+0000 7ff30b7fe700 1 -- 172.21.15.162:0/2127201807 <== mon.0 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 18) v1 ==== 86575+0+0 (secure 0 0 0) 0x7ff318012400 con 0x7ff3240fed70 2024-04-27T04:04:00.108 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:04:00.107+0000 7ff30b7fe700 1 --2- 172.21.15.162:0/2127201807 >> [v2:172.21.15.162:6800/4079782599,v1:172.21.15.162:6801/4079782599] conn(0x7ff30c069210 0x7ff30c06b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:04:00.108 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:04:00.107+0000 7ff30b7fe700 1 -- 172.21.15.162:0/2127201807 <== mon.0 v2:172.21.15.162:3300/0 5 ==== osd_map(15..15 src has 1..15) v4 ==== 2410+0+0 (secure 0 0 0) 0x7ff318086530 con 0x7ff3240fed70 2024-04-27T04:04:00.108 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:04:00.107+0000 7ff322ffd700 1 --2- 172.21.15.162:0/2127201807 >> [v2:172.21.15.162:6800/4079782599,v1:172.21.15.162:6801/4079782599] conn(0x7ff30c069210 0x7ff30c06b6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:04:00.108 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:04:00.107+0000 7ff322ffd700 1 --2- 172.21.15.162:0/2127201807 >> [v2:172.21.15.162:6800/4079782599,v1:172.21.15.162:6801/4079782599] conn(0x7ff30c069210 0x7ff30c06b6c0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7ff31400b3b0 tx=0x7ff314009e30 comp rx=0 tx=0).ready entity=mgr.14210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:04:00.111 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:04:00.111+0000 7ff30b7fe700 1 -- 172.21.15.162:0/2127201807 <== mon.0 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7ff318056fc0 con 0x7ff3240fed70 2024-04-27T04:04:00.316 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:04:00.316+0000 7ff329973700 1 -- 172.21.15.162:0/2127201807 --> [v2:172.21.15.162:6800/4079782599,v1:172.21.15.162:6801/4079782599] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi162:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) v1 -- 0x7ff3241024f0 con 0x7ff30c069210 2024-04-27T04:04:00.423 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:04:00 smithi181 ceph-mon[33774]: pgmap v54: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-04-27T04:04:00.620 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:04:00 smithi162 ceph-mon[26841]: pgmap v54: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-04-27T04:04:01.592 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:04:01 smithi181 ceph-mon[33774]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' 2024-04-27T04:04:01.592 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:04:01 smithi181 ceph-mon[33774]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' 2024-04-27T04:04:01.592 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:04:01 smithi181 ceph-mon[33774]: from='client.14282 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi162:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-27T04:04:01.592 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:04:01 smithi181 ceph-mon[33774]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-27T04:04:01.592 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:04:01 smithi181 ceph-mon[33774]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-27T04:04:01.592 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:04:01 smithi181 ceph-mon[33774]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T04:04:01.637 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:04:01 smithi162 ceph-mon[26841]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' 2024-04-27T04:04:01.637 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:04:01 smithi162 ceph-mon[26841]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' 2024-04-27T04:04:01.637 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:04:01 smithi162 ceph-mon[26841]: from='client.14282 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi162:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-27T04:04:01.637 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:04:01 smithi162 ceph-mon[26841]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-27T04:04:01.637 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:04:01 smithi162 ceph-mon[26841]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-27T04:04:01.637 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:04:01 smithi162 ceph-mon[26841]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T04:04:02.485 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:04:02 smithi162 ceph-mon[26841]: pgmap v55: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-04-27T04:04:02.485 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:04:02 smithi162 ceph-mon[26841]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-27T04:04:02.520 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:04:02 smithi181 ceph-mon[33774]: pgmap v55: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-04-27T04:04:02.520 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:04:02 smithi181 ceph-mon[33774]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-27T04:04:04.388 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:04:04 smithi162 ceph-mon[26841]: pgmap v56: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-04-27T04:04:04.388 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:04:04 smithi162 ceph-mon[26841]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' 2024-04-27T04:04:04.388 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:04:04 smithi162 ceph-mon[26841]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' 2024-04-27T04:04:04.388 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:04:04 smithi162 ceph-mon[26841]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-27T04:04:04.389 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:04:04 smithi162 ceph-mon[26841]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-27T04:04:04.389 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:04:04 smithi162 ceph-mon[26841]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' 2024-04-27T04:04:04.389 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:04:04 smithi162 ceph-mon[26841]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T04:04:04.389 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:04:04 smithi162 ceph-mon[26841]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T04:04:04.389 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:04:04 smithi162 ceph-mon[26841]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' 2024-04-27T04:04:04.389 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:04:04 smithi162 ceph-mon[26841]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T04:04:04.673 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:04:04 smithi181 ceph-mon[33774]: pgmap v56: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-04-27T04:04:04.673 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:04:04 smithi181 ceph-mon[33774]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' 2024-04-27T04:04:04.674 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:04:04 smithi181 ceph-mon[33774]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' 2024-04-27T04:04:04.674 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:04:04 smithi181 ceph-mon[33774]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-27T04:04:04.674 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:04:04 smithi181 ceph-mon[33774]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-27T04:04:04.674 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:04:04 smithi181 ceph-mon[33774]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' 2024-04-27T04:04:04.674 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:04:04 smithi181 ceph-mon[33774]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T04:04:04.674 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:04:04 smithi181 ceph-mon[33774]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T04:04:04.674 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:04:04 smithi181 ceph-mon[33774]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' 2024-04-27T04:04:04.674 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:04:04 smithi181 ceph-mon[33774]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T04:04:05.499 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:04:05 smithi162 ceph-mon[26841]: Detected new or changed devices on smithi162 2024-04-27T04:04:05.499 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:04:05 smithi162 ceph-mon[26841]: Adjusting osd_memory_target on smithi162 to 6432M 2024-04-27T04:04:05.499 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:04:05 smithi162 ceph-mon[26841]: from='client.? 172.21.15.162:0/1015317359' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a773b386-e75b-432a-9d1b-2206301b96c3"}]: dispatch 2024-04-27T04:04:05.499 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:04:05 smithi162 ceph-mon[26841]: from='client.? 172.21.15.162:0/1015317359' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "a773b386-e75b-432a-9d1b-2206301b96c3"}]': finished 2024-04-27T04:04:05.499 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:04:05 smithi162 ceph-mon[26841]: osdmap e16: 3 total, 2 up, 3 in 2024-04-27T04:04:05.499 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:04:05 smithi162 ceph-mon[26841]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-27T04:04:05.674 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:04:05 smithi181 ceph-mon[33774]: Detected new or changed devices on smithi162 2024-04-27T04:04:05.674 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:04:05 smithi181 ceph-mon[33774]: Adjusting osd_memory_target on smithi162 to 6432M 2024-04-27T04:04:05.674 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:04:05 smithi181 ceph-mon[33774]: from='client.? 172.21.15.162:0/1015317359' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a773b386-e75b-432a-9d1b-2206301b96c3"}]: dispatch 2024-04-27T04:04:05.674 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:04:05 smithi181 ceph-mon[33774]: from='client.? 172.21.15.162:0/1015317359' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "a773b386-e75b-432a-9d1b-2206301b96c3"}]': finished 2024-04-27T04:04:05.674 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:04:05 smithi181 ceph-mon[33774]: osdmap e16: 3 total, 2 up, 3 in 2024-04-27T04:04:05.674 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:04:05 smithi181 ceph-mon[33774]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-27T04:04:06.388 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:04:06 smithi162 ceph-mon[26841]: pgmap v57: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-04-27T04:04:06.388 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:04:06 smithi162 ceph-mon[26841]: from='client.? 172.21.15.162:0/2887757223' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-27T04:04:06.673 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:04:06 smithi181 ceph-mon[33774]: pgmap v57: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-04-27T04:04:06.673 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:04:06 smithi181 ceph-mon[33774]: from='client.? 172.21.15.162:0/2887757223' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-27T04:04:08.638 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:04:08 smithi162 ceph-mon[26841]: pgmap v59: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-04-27T04:04:08.638 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:04:08 smithi162 ceph-mon[26841]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' 2024-04-27T04:04:08.638 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:04:08 smithi162 ceph-mon[26841]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' 2024-04-27T04:04:08.638 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:04:08 smithi162 ceph-mon[26841]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T04:04:08.638 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:04:08 smithi162 ceph-mon[26841]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T04:04:08.638 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:04:08 smithi162 ceph-mon[26841]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' 2024-04-27T04:04:08.673 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:04:08 smithi181 ceph-mon[33774]: pgmap v59: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-04-27T04:04:08.673 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:04:08 smithi181 ceph-mon[33774]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' 2024-04-27T04:04:08.674 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:04:08 smithi181 ceph-mon[33774]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' 2024-04-27T04:04:08.674 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:04:08 smithi181 ceph-mon[33774]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T04:04:08.674 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:04:08 smithi181 ceph-mon[33774]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T04:04:08.674 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:04:08 smithi181 ceph-mon[33774]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' 2024-04-27T04:04:10.562 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:04:10 smithi162 ceph-mon[26841]: pgmap v60: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-04-27T04:04:10.673 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:04:10 smithi181 ceph-mon[33774]: pgmap v60: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-04-27T04:04:12.388 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:04:12 smithi162 ceph-mon[26841]: pgmap v61: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-04-27T04:04:12.388 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:04:12 smithi162 ceph-mon[26841]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-04-27T04:04:12.388 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:04:12 smithi162 ceph-mon[26841]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T04:04:12.423 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:04:12 smithi181 ceph-mon[33774]: pgmap v61: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-04-27T04:04:12.423 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:04:12 smithi181 ceph-mon[33774]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-04-27T04:04:12.423 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:04:12 smithi181 ceph-mon[33774]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T04:04:13.500 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:04:13 smithi162 ceph-mon[26841]: Deploying daemon osd.2 on smithi162 2024-04-27T04:04:13.673 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:04:13 smithi181 ceph-mon[33774]: Deploying daemon osd.2 on smithi162 2024-04-27T04:04:14.464 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:04:14 smithi162 ceph-mon[26841]: pgmap v62: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-04-27T04:04:14.673 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:04:14 smithi181 ceph-mon[33774]: pgmap v62: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-04-27T04:04:16.388 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:04:16 smithi162 ceph-mon[26841]: pgmap v63: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-04-27T04:04:16.673 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:04:16 smithi181 ceph-mon[33774]: pgmap v63: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-04-27T04:04:17.618 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:04:17 smithi162 ceph-mon[26841]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-27T04:04:17.673 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:04:17 smithi181 ceph-mon[33774]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-27T04:04:18.592 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:04:18 smithi162 ceph-mon[26841]: pgmap v64: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-04-27T04:04:18.592 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:04:18 smithi162 ceph-mon[26841]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' 2024-04-27T04:04:18.592 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:04:18 smithi162 ceph-mon[26841]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' 2024-04-27T04:04:18.592 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:04:18 smithi162 ceph-mon[26841]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T04:04:18.673 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:04:18 smithi181 ceph-mon[33774]: pgmap v64: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-04-27T04:04:18.673 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:04:18 smithi181 ceph-mon[33774]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' 2024-04-27T04:04:18.673 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:04:18 smithi181 ceph-mon[33774]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' 2024-04-27T04:04:18.673 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:04:18 smithi181 ceph-mon[33774]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T04:04:20.638 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:04:20 smithi162 ceph-mon[26841]: pgmap v65: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-04-27T04:04:20.673 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:04:20 smithi181 ceph-mon[33774]: pgmap v65: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-04-27T04:04:20.737 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:04:20.736+0000 7ff30b7fe700 1 -- 172.21.15.162:0/2127201807 <== mgr.14210 v2:172.21.15.162:6800/4079782599 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7ff3241024f0 con 0x7ff30c069210 2024-04-27T04:04:20.738 INFO:teuthology.orchestra.run.smithi162.stdout:Created osd(s) 2 on host 'smithi162' 2024-04-27T04:04:20.738 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:04:20.739+0000 7ff329973700 1 -- 172.21.15.162:0/2127201807 >> [v2:172.21.15.162:6800/4079782599,v1:172.21.15.162:6801/4079782599] conn(0x7ff30c069210 msgr2=0x7ff30c06b6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:04:20.738 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:04:20.739+0000 7ff329973700 1 --2- 172.21.15.162:0/2127201807 >> [v2:172.21.15.162:6800/4079782599,v1:172.21.15.162:6801/4079782599] conn(0x7ff30c069210 0x7ff30c06b6c0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7ff31400b3b0 tx=0x7ff314009e30 comp rx=0 tx=0).stop 2024-04-27T04:04:20.739 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:04:20.739+0000 7ff329973700 1 -- 172.21.15.162:0/2127201807 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff3240fed70 msgr2=0x7ff3240fc0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:04:20.739 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:04:20.739+0000 7ff329973700 1 --2- 172.21.15.162:0/2127201807 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff3240fed70 0x7ff3240fc0e0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7ff31800c830 tx=0x7ff318004470 comp rx=0 tx=0).stop 2024-04-27T04:04:20.739 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:04:20.739+0000 7ff329973700 1 -- 172.21.15.162:0/2127201807 shutdown_connections 2024-04-27T04:04:20.739 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:04:20.739+0000 7ff329973700 1 --2- 172.21.15.162:0/2127201807 >> [v2:172.21.15.162:6800/4079782599,v1:172.21.15.162:6801/4079782599] conn(0x7ff30c069210 0x7ff30c06b6c0 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:04:20.739 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:04:20.739+0000 7ff329973700 1 --2- 172.21.15.162:0/2127201807 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff3240fe420 0x7ff3240fda30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:04:20.739 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:04:20.739+0000 7ff329973700 1 --2- 172.21.15.162:0/2127201807 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff3240fed70 0x7ff3240fc0e0 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:04:20.739 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:04:20.739+0000 7ff329973700 1 -- 172.21.15.162:0/2127201807 >> 172.21.15.162:0/2127201807 conn(0x7ff3240f6be0 msgr2=0x7ff3240f8290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:04:20.739 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:04:20.739+0000 7ff329973700 1 -- 172.21.15.162:0/2127201807 shutdown_connections 2024-04-27T04:04:20.739 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:04:20.739+0000 7ff329973700 1 -- 172.21.15.162:0/2127201807 wait complete. 2024-04-27T04:04:21.577 DEBUG:teuthology.orchestra.run.smithi162:osd.2> sudo journalctl -f -n 0 -u ceph-7f9750d6-044a-11ef-bc93-c7b262605968@osd.2.service 2024-04-27T04:04:21.579 INFO:tasks.cephadm:Deploying osd.3 on smithi181 with /dev/vg_nvme/lv_4... 2024-04-27T04:04:21.579 DEBUG:teuthology.orchestra.run.smithi181:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 7f9750d6-044a-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-04-27T04:04:21.840 INFO:teuthology.orchestra.run.smithi181.stderr:Inferring config /var/lib/ceph/7f9750d6-044a-11ef-bc93-c7b262605968/mon.smithi181/config 2024-04-27T04:04:21.888 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:04:21 smithi162 ceph-mon[26841]: pgmap v66: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-04-27T04:04:21.888 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:04:21 smithi162 ceph-mon[26841]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' 2024-04-27T04:04:21.888 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:04:21 smithi162 ceph-mon[26841]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' 2024-04-27T04:04:21.888 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:04:21 smithi162 ceph-mon[26841]: from='osd.2 [v2:172.21.15.162:6818/3830961923,v1:172.21.15.162:6819/3830961923]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-04-27T04:04:21.923 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:04:21 smithi181 ceph-mon[33774]: pgmap v66: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-04-27T04:04:21.923 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:04:21 smithi181 ceph-mon[33774]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' 2024-04-27T04:04:21.923 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:04:21 smithi181 ceph-mon[33774]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' 2024-04-27T04:04:21.923 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:04:21 smithi181 ceph-mon[33774]: from='osd.2 [v2:172.21.15.162:6818/3830961923,v1:172.21.15.162:6819/3830961923]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-04-27T04:04:22.888 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:04:22 smithi162 ceph-mon[26841]: from='osd.2 [v2:172.21.15.162:6818/3830961923,v1:172.21.15.162:6819/3830961923]' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-04-27T04:04:22.888 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:04:22 smithi162 ceph-mon[26841]: osdmap e17: 3 total, 2 up, 3 in 2024-04-27T04:04:22.889 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:04:22 smithi162 ceph-mon[26841]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-27T04:04:22.889 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:04:22 smithi162 ceph-mon[26841]: from='osd.2 [v2:172.21.15.162:6818/3830961923,v1:172.21.15.162:6819/3830961923]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi162", "root=default"]}]: dispatch 2024-04-27T04:04:22.889 INFO:journalctl@ceph.osd.2.smithi162.stdout:Apr 27 04:04:22 smithi162 ceph-7f9750d6-044a-11ef-bc93-c7b262605968-osd-2[60531]: 2024-04-27T04:04:22.705+0000 7f4a5b329700 -1 osd.2 0 waiting for initial osdmap 2024-04-27T04:04:22.889 INFO:journalctl@ceph.osd.2.smithi162.stdout:Apr 27 04:04:22 smithi162 ceph-7f9750d6-044a-11ef-bc93-c7b262605968-osd-2[60531]: 2024-04-27T04:04:22.711+0000 7f4a57927700 -1 osd.2 18 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-27T04:04:22.955 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:04:22 smithi181 ceph-mon[33774]: from='osd.2 [v2:172.21.15.162:6818/3830961923,v1:172.21.15.162:6819/3830961923]' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-04-27T04:04:22.956 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:04:22 smithi181 ceph-mon[33774]: osdmap e17: 3 total, 2 up, 3 in 2024-04-27T04:04:22.956 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:04:22 smithi181 ceph-mon[33774]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-27T04:04:22.956 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:04:22 smithi181 ceph-mon[33774]: from='osd.2 [v2:172.21.15.162:6818/3830961923,v1:172.21.15.162:6819/3830961923]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi162", "root=default"]}]: dispatch 2024-04-27T04:04:23.624 INFO:teuthology.orchestra.run.smithi181.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-04-27T04:04:23.625 INFO:teuthology.orchestra.run.smithi181.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-04-27T04:04:23.625 INFO:teuthology.orchestra.run.smithi181.stderr: stderr: 10+0 records in 2024-04-27T04:04:23.625 INFO:teuthology.orchestra.run.smithi181.stderr:10+0 records out 2024-04-27T04:04:23.625 INFO:teuthology.orchestra.run.smithi181.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0130144 s, 806 MB/s 2024-04-27T04:04:23.625 INFO:teuthology.orchestra.run.smithi181.stderr:--> Zapping successful for: 2024-04-27T04:04:23.923 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:04:23 smithi181 ceph-mon[33774]: pgmap v68: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-04-27T04:04:23.924 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:04:23 smithi181 ceph-mon[33774]: from='osd.2 [v2:172.21.15.162:6818/3830961923,v1:172.21.15.162:6819/3830961923]' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi162", "root=default"]}]': finished 2024-04-27T04:04:23.924 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:04:23 smithi181 ceph-mon[33774]: osdmap e18: 3 total, 2 up, 3 in 2024-04-27T04:04:23.924 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:04:23 smithi181 ceph-mon[33774]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-27T04:04:23.924 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:04:23 smithi181 ceph-mon[33774]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-27T04:04:24.138 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:04:23 smithi162 ceph-mon[26841]: pgmap v68: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-04-27T04:04:24.138 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:04:23 smithi162 ceph-mon[26841]: from='osd.2 [v2:172.21.15.162:6818/3830961923,v1:172.21.15.162:6819/3830961923]' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi162", "root=default"]}]': finished 2024-04-27T04:04:24.138 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:04:23 smithi162 ceph-mon[26841]: osdmap e18: 3 total, 2 up, 3 in 2024-04-27T04:04:24.138 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:04:23 smithi162 ceph-mon[26841]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-27T04:04:24.138 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:04:23 smithi162 ceph-mon[26841]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-27T04:04:24.179 DEBUG:teuthology.orchestra.run.smithi181:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 7f9750d6-044a-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi181:vg_nvme/lv_4 2024-04-27T04:04:24.431 INFO:teuthology.orchestra.run.smithi181.stderr:Inferring config /var/lib/ceph/7f9750d6-044a-11ef-bc93-c7b262605968/mon.smithi181/config 2024-04-27T04:04:24.888 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:04:24 smithi162 ceph-mon[26841]: purged_snaps scrub starts 2024-04-27T04:04:24.888 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:04:24 smithi162 ceph-mon[26841]: purged_snaps scrub ok 2024-04-27T04:04:24.888 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:04:24 smithi162 ceph-mon[26841]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-27T04:04:24.888 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:04:24 smithi162 ceph-mon[26841]: osd.2 [v2:172.21.15.162:6818/3830961923,v1:172.21.15.162:6819/3830961923] boot 2024-04-27T04:04:24.888 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:04:24 smithi162 ceph-mon[26841]: osdmap e19: 3 total, 3 up, 3 in 2024-04-27T04:04:24.888 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:04:24 smithi162 ceph-mon[26841]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-27T04:04:24.889 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:04:24 smithi162 ceph-mon[26841]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' 2024-04-27T04:04:24.889 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:04:24 smithi162 ceph-mon[26841]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' 2024-04-27T04:04:25.170 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:04:24 smithi181 ceph-mon[33774]: purged_snaps scrub starts 2024-04-27T04:04:25.170 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:04:24 smithi181 ceph-mon[33774]: purged_snaps scrub ok 2024-04-27T04:04:25.170 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:04:24 smithi181 ceph-mon[33774]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-27T04:04:25.170 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:04:24 smithi181 ceph-mon[33774]: osd.2 [v2:172.21.15.162:6818/3830961923,v1:172.21.15.162:6819/3830961923] boot 2024-04-27T04:04:25.170 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:04:24 smithi181 ceph-mon[33774]: osdmap e19: 3 total, 3 up, 3 in 2024-04-27T04:04:25.170 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:04:24 smithi181 ceph-mon[33774]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-27T04:04:25.170 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:04:24 smithi181 ceph-mon[33774]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' 2024-04-27T04:04:25.170 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:04:24 smithi181 ceph-mon[33774]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' 2024-04-27T04:04:25.756 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:04:25 smithi181 ceph-mon[33774]: pgmap v71: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-04-27T04:04:25.756 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:04:25 smithi181 ceph-mon[33774]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32}]: dispatch 2024-04-27T04:04:25.756 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:04:25 smithi181 ceph-mon[33774]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32}]': finished 2024-04-27T04:04:25.756 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:04:25 smithi181 ceph-mon[33774]: osdmap e20: 3 total, 3 up, 3 in 2024-04-27T04:04:25.756 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:04:25 smithi181 ceph-mon[33774]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-04-27T04:04:25.888 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:04:25 smithi162 ceph-mon[26841]: pgmap v71: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-04-27T04:04:25.888 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:04:25 smithi162 ceph-mon[26841]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32}]: dispatch 2024-04-27T04:04:25.888 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:04:25 smithi162 ceph-mon[26841]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32}]': finished 2024-04-27T04:04:25.889 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:04:25 smithi162 ceph-mon[26841]: osdmap e20: 3 total, 3 up, 3 in 2024-04-27T04:04:25.889 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:04:25 smithi162 ceph-mon[26841]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-04-27T04:04:25.956 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:04:25.955+0000 7f5496b9c700 1 -- 172.21.15.181:0/2260773784 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f54900fbcf0 msgr2=0x7f54900fc100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:04:25.956 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:04:25.955+0000 7f5496b9c700 1 --2- 172.21.15.181:0/2260773784 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f54900fbcf0 0x7f54900fc100 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f5484004940 tx=0x7f5484009e30 comp rx=0 tx=0).stop 2024-04-27T04:04:25.957 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:04:25.956+0000 7f5496b9c700 1 -- 172.21.15.181:0/2260773784 shutdown_connections 2024-04-27T04:04:25.957 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:04:25.956+0000 7f5496b9c700 1 --2- 172.21.15.181:0/2260773784 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f54900fcfa0 0x7f54900fd410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:04:25.957 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:04:25.956+0000 7f5496b9c700 1 --2- 172.21.15.181:0/2260773784 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f54900fbcf0 0x7f54900fc100 unknown :-1 s=CLOSED pgs=3 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:04:25.957 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:04:25.956+0000 7f5496b9c700 1 -- 172.21.15.181:0/2260773784 >> 172.21.15.181:0/2260773784 conn(0x7f54900f7450 msgr2=0x7f54900f9880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:04:25.957 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:04:25.956+0000 7f5496b9c700 1 -- 172.21.15.181:0/2260773784 shutdown_connections 2024-04-27T04:04:25.957 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:04:25.956+0000 7f5496b9c700 1 -- 172.21.15.181:0/2260773784 wait complete. 2024-04-27T04:04:25.957 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:04:25.956+0000 7f5496b9c700 1 Processor -- start 2024-04-27T04:04:25.957 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:04:25.956+0000 7f5496b9c700 1 -- start start 2024-04-27T04:04:25.959 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:04:25.956+0000 7f5496b9c700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f54900fbcf0 0x7f549010d890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:04:25.959 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:04:25.957+0000 7f5496b9c700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f54900fcfa0 0x7f549010ddd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:04:25.959 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:04:25.957+0000 7f5496b9c700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f549010e3d0 con 0x7f54900fcfa0 2024-04-27T04:04:25.959 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:04:25.957+0000 7f5496b9c700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f549010e540 con 0x7f54900fbcf0 2024-04-27T04:04:25.959 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:04:25.957+0000 7f5494938700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f54900fbcf0 0x7f549010d890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:04:25.959 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:04:25.957+0000 7f5494938700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f54900fbcf0 0x7f549010d890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:47344/0 (socket says 172.21.15.181:47344) 2024-04-27T04:04:25.959 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:04:25.957+0000 7f5494938700 1 -- 172.21.15.181:0/290607419 learned_addr learned my addr 172.21.15.181:0/290607419 (peer_addr_for_me v2:172.21.15.181:0/0) 2024-04-27T04:04:25.959 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:04:25.957+0000 7f548ffff700 1 --2- 172.21.15.181:0/290607419 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f54900fcfa0 0x7f549010ddd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:04:25.959 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:04:25.957+0000 7f5494938700 1 -- 172.21.15.181:0/290607419 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f54900fcfa0 msgr2=0x7f549010ddd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:04:25.959 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:04:25.957+0000 7f5494938700 1 --2- 172.21.15.181:0/290607419 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f54900fcfa0 0x7f549010ddd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:04:25.959 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:04:25.957+0000 7f5494938700 1 -- 172.21.15.181:0/290607419 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5484005040 con 0x7f54900fbcf0 2024-04-27T04:04:25.960 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:04:25.957+0000 7f5494938700 1 --2- 172.21.15.181:0/290607419 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f54900fbcf0 0x7f549010d890 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7f548400b3b0 tx=0x7f5484009e30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:04:25.960 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:04:25.957+0000 7f548d7fa700 1 -- 172.21.15.181:0/290607419 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f548400e040 con 0x7f54900fbcf0 2024-04-27T04:04:25.960 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:04:25.957+0000 7f5496b9c700 1 -- 172.21.15.181:0/290607419 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5490112f40 con 0x7f54900fbcf0 2024-04-27T04:04:25.960 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:04:25.957+0000 7f548d7fa700 1 -- 172.21.15.181:0/290607419 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f5484004290 con 0x7f54900fbcf0 2024-04-27T04:04:25.960 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:04:25.957+0000 7f548d7fa700 1 -- 172.21.15.181:0/290607419 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f548401c3f0 con 0x7f54900fbcf0 2024-04-27T04:04:25.960 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:04:25.957+0000 7f5496b9c700 1 -- 172.21.15.181:0/290607419 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5490113470 con 0x7f54900fbcf0 2024-04-27T04:04:25.960 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:04:25.958+0000 7f5496b9c700 1 -- 172.21.15.181:0/290607419 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5490101b30 con 0x7f54900fbcf0 2024-04-27T04:04:25.960 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:04:25.959+0000 7f548d7fa700 1 -- 172.21.15.181:0/290607419 <== mon.1 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 18) v1 ==== 86575+0+0 (secure 0 0 0) 0x7f548400bbf0 con 0x7f54900fbcf0 2024-04-27T04:04:25.961 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:04:25.959+0000 7f548d7fa700 1 --2- 172.21.15.181:0/290607419 >> [v2:172.21.15.162:6800/4079782599,v1:172.21.15.162:6801/4079782599] conn(0x7f5480069210 0x7f548006b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:04:25.961 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:04:25.959+0000 7f548d7fa700 1 -- 172.21.15.181:0/290607419 <== mon.1 v2:172.21.15.181:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 3217+0+0 (secure 0 0 0) 0x7f5484087b10 con 0x7f54900fbcf0 2024-04-27T04:04:25.961 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:04:25.959+0000 7f548ffff700 1 --2- 172.21.15.181:0/290607419 >> [v2:172.21.15.162:6800/4079782599,v1:172.21.15.162:6801/4079782599] conn(0x7f5480069210 0x7f548006b6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:04:25.961 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:04:25.959+0000 7f548ffff700 1 --2- 172.21.15.181:0/290607419 >> [v2:172.21.15.162:6800/4079782599,v1:172.21.15.162:6801/4079782599] conn(0x7f5480069210 0x7f548006b6c0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f547c009fd0 tx=0x7f547c00b040 comp rx=0 tx=0).ready entity=mgr.14210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:04:25.964 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:04:25.964+0000 7f548d7fa700 1 -- 172.21.15.181:0/290607419 <== mon.1 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f5484058270 con 0x7f54900fbcf0 2024-04-27T04:04:26.157 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:04:26.157+0000 7f5496b9c700 1 -- 172.21.15.181:0/290607419 --> [v2:172.21.15.162:6800/4079782599,v1:172.21.15.162:6801/4079782599] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi181:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) v1 -- 0x7f549002b400 con 0x7f5480069210 2024-04-27T04:04:26.888 INFO:journalctl@ceph.osd.0.smithi162.stdout:Apr 27 04:04:26 smithi162 sudo[64664]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-04-27T04:04:26.889 INFO:journalctl@ceph.osd.0.smithi162.stdout:Apr 27 04:04:26 smithi162 sudo[64664]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-27T04:04:26.889 INFO:journalctl@ceph.osd.0.smithi162.stdout:Apr 27 04:04:26 smithi162 sudo[64664]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-27T04:04:26.889 INFO:journalctl@ceph.osd.0.smithi162.stdout:Apr 27 04:04:26 smithi162 sudo[64664]: pam_unix(sudo:session): session closed for user root 2024-04-27T04:04:26.985 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:04:26.985+0000 7f548d7fa700 1 -- 172.21.15.181:0/290607419 <== mon.1 v2:172.21.15.181:3300/0 7 ==== mgrmap(e 19) v1 ==== 86631+0+0 (secure 0 0 0) 0x7f5484054bb0 con 0x7f54900fbcf0 2024-04-27T04:04:27.173 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:04:26 smithi181 ceph-mon[33774]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-04-27T04:04:27.173 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:04:26 smithi181 ceph-mon[33774]: osdmap e21: 3 total, 3 up, 3 in 2024-04-27T04:04:27.173 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:04:26 smithi181 ceph-mon[33774]: from='client.24125 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi181:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-27T04:04:27.174 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:04:26 smithi181 ceph-mon[33774]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-27T04:04:27.174 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:04:26 smithi181 ceph-mon[33774]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-27T04:04:27.174 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:04:26 smithi181 ceph-mon[33774]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T04:04:27.350 INFO:journalctl@ceph.osd.0.smithi162.stdout:Apr 27 04:04:27 smithi162 sudo[64696]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-04-27T04:04:27.351 INFO:journalctl@ceph.osd.0.smithi162.stdout:Apr 27 04:04:27 smithi162 sudo[64696]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-27T04:04:27.351 INFO:journalctl@ceph.osd.0.smithi162.stdout:Apr 27 04:04:27 smithi162 sudo[64696]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-27T04:04:27.351 INFO:journalctl@ceph.osd.0.smithi162.stdout:Apr 27 04:04:27 smithi162 sudo[64696]: pam_unix(sudo:session): session closed for user root 2024-04-27T04:04:27.351 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:04:26 smithi162 ceph-mon[26841]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-04-27T04:04:27.351 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:04:26 smithi162 ceph-mon[26841]: osdmap e21: 3 total, 3 up, 3 in 2024-04-27T04:04:27.351 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:04:26 smithi162 ceph-mon[26841]: from='client.24125 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi181:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-27T04:04:27.351 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:04:26 smithi162 ceph-mon[26841]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-27T04:04:27.351 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:04:26 smithi162 ceph-mon[26841]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-27T04:04:27.351 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:04:26 smithi162 ceph-mon[26841]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T04:04:27.638 INFO:journalctl@ceph.osd.1.smithi162.stdout:Apr 27 04:04:27 smithi162 sudo[64770]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-04-27T04:04:27.638 INFO:journalctl@ceph.osd.1.smithi162.stdout:Apr 27 04:04:27 smithi162 sudo[64770]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-27T04:04:27.638 INFO:journalctl@ceph.osd.1.smithi162.stdout:Apr 27 04:04:27 smithi162 sudo[64770]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-27T04:04:27.638 INFO:journalctl@ceph.osd.1.smithi162.stdout:Apr 27 04:04:27 smithi162 sudo[64770]: pam_unix(sudo:session): session closed for user root 2024-04-27T04:04:27.890 INFO:journalctl@ceph.osd.1.smithi162.stdout:Apr 27 04:04:27 smithi162 sudo[64859]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-04-27T04:04:27.890 INFO:journalctl@ceph.osd.1.smithi162.stdout:Apr 27 04:04:27 smithi162 sudo[64859]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-27T04:04:27.890 INFO:journalctl@ceph.osd.1.smithi162.stdout:Apr 27 04:04:27 smithi162 sudo[64859]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-27T04:04:27.890 INFO:journalctl@ceph.osd.1.smithi162.stdout:Apr 27 04:04:27 smithi162 sudo[64859]: pam_unix(sudo:session): session closed for user root 2024-04-27T04:04:28.149 INFO:journalctl@ceph.osd.2.smithi162.stdout:Apr 27 04:04:27 smithi162 sudo[64877]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-04-27T04:04:28.149 INFO:journalctl@ceph.osd.2.smithi162.stdout:Apr 27 04:04:27 smithi162 sudo[64877]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-27T04:04:28.149 INFO:journalctl@ceph.osd.2.smithi162.stdout:Apr 27 04:04:27 smithi162 sudo[64877]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-27T04:04:28.149 INFO:journalctl@ceph.osd.2.smithi162.stdout:Apr 27 04:04:27 smithi162 sudo[64877]: pam_unix(sudo:session): session closed for user root 2024-04-27T04:04:28.150 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:04:27 smithi162 ceph-mon[26841]: pgmap v74: 1 pgs: 1 unknown; 0 B data, 872 MiB used, 267 GiB / 268 GiB avail 2024-04-27T04:04:28.150 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:04:27 smithi162 ceph-mon[26841]: osdmap e22: 3 total, 3 up, 3 in 2024-04-27T04:04:28.150 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:04:27 smithi162 ceph-mon[26841]: mgrmap e19: smithi162.oennfn(active, since 2m), standbys: smithi181.vtwuhz 2024-04-27T04:04:28.150 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:04:27 smithi162 ceph-mon[26841]: Detected new or changed devices on smithi162 2024-04-27T04:04:28.150 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:04:27 smithi162 ceph-mon[26841]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' 2024-04-27T04:04:28.150 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:04:27 smithi162 ceph-mon[26841]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' 2024-04-27T04:04:28.150 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:04:27 smithi162 ceph-mon[26841]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-27T04:04:28.150 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:04:27 smithi162 ceph-mon[26841]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-27T04:04:28.150 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:04:27 smithi162 ceph-mon[26841]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-27T04:04:28.150 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:04:27 smithi162 ceph-mon[26841]: Adjusting osd_memory_target on smithi162 to 4288M 2024-04-27T04:04:28.151 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:04:27 smithi162 ceph-mon[26841]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' 2024-04-27T04:04:28.151 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:04:27 smithi162 ceph-mon[26841]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T04:04:28.151 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:04:27 smithi162 ceph-mon[26841]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T04:04:28.151 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:04:27 smithi162 ceph-mon[26841]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' 2024-04-27T04:04:28.151 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:04:27 smithi162 ceph-mon[26841]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' 2024-04-27T04:04:28.151 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:04:27 smithi162 ceph-mon[26841]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T04:04:28.174 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:04:27 smithi181 ceph-mon[33774]: pgmap v74: 1 pgs: 1 unknown; 0 B data, 872 MiB used, 267 GiB / 268 GiB avail 2024-04-27T04:04:28.174 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:04:27 smithi181 ceph-mon[33774]: osdmap e22: 3 total, 3 up, 3 in 2024-04-27T04:04:28.174 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:04:27 smithi181 ceph-mon[33774]: mgrmap e19: smithi162.oennfn(active, since 2m), standbys: smithi181.vtwuhz 2024-04-27T04:04:28.174 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:04:27 smithi181 ceph-mon[33774]: Detected new or changed devices on smithi162 2024-04-27T04:04:28.174 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:04:27 smithi181 ceph-mon[33774]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' 2024-04-27T04:04:28.174 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:04:27 smithi181 ceph-mon[33774]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' 2024-04-27T04:04:28.174 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:04:27 smithi181 ceph-mon[33774]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-27T04:04:28.174 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:04:27 smithi181 ceph-mon[33774]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-27T04:04:28.174 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:04:27 smithi181 ceph-mon[33774]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-27T04:04:28.174 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:04:27 smithi181 ceph-mon[33774]: Adjusting osd_memory_target on smithi162 to 4288M 2024-04-27T04:04:28.175 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:04:27 smithi181 ceph-mon[33774]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' 2024-04-27T04:04:28.175 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:04:27 smithi181 ceph-mon[33774]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T04:04:28.175 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:04:27 smithi181 ceph-mon[33774]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T04:04:28.175 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:04:27 smithi181 ceph-mon[33774]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' 2024-04-27T04:04:28.175 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:04:27 smithi181 ceph-mon[33774]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' 2024-04-27T04:04:28.175 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:04:27 smithi181 ceph-mon[33774]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T04:04:28.410 INFO:journalctl@ceph.osd.2.smithi162.stdout:Apr 27 04:04:28 smithi162 sudo[64884]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-04-27T04:04:28.410 INFO:journalctl@ceph.osd.2.smithi162.stdout:Apr 27 04:04:28 smithi162 sudo[64884]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-27T04:04:28.410 INFO:journalctl@ceph.osd.2.smithi162.stdout:Apr 27 04:04:28 smithi162 sudo[64884]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-27T04:04:28.410 INFO:journalctl@ceph.osd.2.smithi162.stdout:Apr 27 04:04:28 smithi162 sudo[64884]: pam_unix(sudo:session): session closed for user root 2024-04-27T04:04:28.668 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:04:28 smithi162 sudo[64970]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-04-27T04:04:28.668 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:04:28 smithi162 sudo[64970]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-27T04:04:28.668 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:04:28 smithi162 sudo[64970]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-27T04:04:28.668 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:04:28 smithi162 sudo[64970]: pam_unix(sudo:session): session closed for user root 2024-04-27T04:04:29.138 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:04:28 smithi162 sudo[64993]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-04-27T04:04:29.138 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:04:28 smithi162 sudo[64993]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-27T04:04:29.138 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:04:28 smithi162 sudo[64993]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-27T04:04:29.138 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:04:28 smithi162 sudo[64993]: pam_unix(sudo:session): session closed for user root 2024-04-27T04:04:29.163 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:04:28 smithi181 sudo[37262]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-04-27T04:04:29.164 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:04:28 smithi181 sudo[37262]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-27T04:04:29.164 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:04:28 smithi181 sudo[37262]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-27T04:04:29.164 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:04:28 smithi181 sudo[37262]: pam_unix(sudo:session): session closed for user root 2024-04-27T04:04:29.422 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:04:29 smithi162 ceph-mon[26841]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-27T04:04:29.422 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:04:29 smithi162 ceph-mon[26841]: from='client.? 172.21.15.181:0/1744495518' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "756091bc-1594-479d-9390-cb60e6b65ebe"}]: dispatch 2024-04-27T04:04:29.422 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:04:29 smithi162 ceph-mon[26841]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-27T04:04:29.422 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:04:29 smithi162 ceph-mon[26841]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "mon metadata", "id": "smithi162"}]: dispatch 2024-04-27T04:04:29.422 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:04:29 smithi162 ceph-mon[26841]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "756091bc-1594-479d-9390-cb60e6b65ebe"}]: dispatch 2024-04-27T04:04:29.423 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:04:29 smithi162 ceph-mon[26841]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "mon metadata", "id": "smithi181"}]: dispatch 2024-04-27T04:04:29.423 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:04:29 smithi162 ceph-mon[26841]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "756091bc-1594-479d-9390-cb60e6b65ebe"}]': finished 2024-04-27T04:04:29.423 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:04:29 smithi162 ceph-mon[26841]: osdmap e23: 4 total, 3 up, 4 in 2024-04-27T04:04:29.423 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:04:29 smithi162 ceph-mon[26841]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-27T04:04:29.423 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:04:29 smithi162 ceph-mon[26841]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "mon metadata", "id": "smithi162"}]: dispatch 2024-04-27T04:04:29.423 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:04:29 smithi162 ceph-mon[26841]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "mon metadata", "id": "smithi181"}]: dispatch 2024-04-27T04:04:29.423 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:04:29 smithi181 sudo[37265]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-04-27T04:04:29.423 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:04:29 smithi181 sudo[37265]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-27T04:04:29.423 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:04:29 smithi181 sudo[37265]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-27T04:04:29.424 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:04:29 smithi181 sudo[37265]: pam_unix(sudo:session): session closed for user root 2024-04-27T04:04:29.424 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:04:29 smithi181 ceph-mon[33774]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-27T04:04:29.424 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:04:29 smithi181 ceph-mon[33774]: from='client.? 172.21.15.181:0/1744495518' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "756091bc-1594-479d-9390-cb60e6b65ebe"}]: dispatch 2024-04-27T04:04:29.424 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:04:29 smithi181 ceph-mon[33774]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-27T04:04:29.424 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:04:29 smithi181 ceph-mon[33774]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "mon metadata", "id": "smithi162"}]: dispatch 2024-04-27T04:04:29.424 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:04:29 smithi181 ceph-mon[33774]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "756091bc-1594-479d-9390-cb60e6b65ebe"}]: dispatch 2024-04-27T04:04:29.424 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:04:29 smithi181 ceph-mon[33774]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "mon metadata", "id": "smithi181"}]: dispatch 2024-04-27T04:04:29.424 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:04:29 smithi181 ceph-mon[33774]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "756091bc-1594-479d-9390-cb60e6b65ebe"}]': finished 2024-04-27T04:04:29.424 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:04:29 smithi181 ceph-mon[33774]: osdmap e23: 4 total, 3 up, 4 in 2024-04-27T04:04:29.424 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:04:29 smithi181 ceph-mon[33774]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-27T04:04:29.424 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:04:29 smithi181 ceph-mon[33774]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "mon metadata", "id": "smithi162"}]: dispatch 2024-04-27T04:04:29.424 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:04:29 smithi181 ceph-mon[33774]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "mon metadata", "id": "smithi181"}]: dispatch 2024-04-27T04:04:30.388 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:04:30 smithi162 ceph-mon[26841]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-27T04:04:30.388 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:04:30 smithi162 ceph-mon[26841]: pgmap v77: 1 pgs: 1 active+clean; 577 KiB data, 872 MiB used, 267 GiB / 268 GiB avail 2024-04-27T04:04:30.388 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:04:30 smithi162 ceph-mon[26841]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-27T04:04:30.388 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:04:30 smithi162 ceph-mon[26841]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' 2024-04-27T04:04:30.388 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:04:30 smithi162 ceph-mon[26841]: from='client.? 172.21.15.181:0/3886205871' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-27T04:04:30.673 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:04:30 smithi181 ceph-mon[33774]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-27T04:04:30.673 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:04:30 smithi181 ceph-mon[33774]: pgmap v77: 1 pgs: 1 active+clean; 577 KiB data, 872 MiB used, 267 GiB / 268 GiB avail 2024-04-27T04:04:30.673 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:04:30 smithi181 ceph-mon[33774]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-27T04:04:30.674 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:04:30 smithi181 ceph-mon[33774]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' 2024-04-27T04:04:30.674 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:04:30 smithi181 ceph-mon[33774]: from='client.? 172.21.15.181:0/3886205871' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-27T04:04:32.476 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:04:32 smithi181 ceph-mon[33774]: pgmap v78: 1 pgs: 1 active+clean; 577 KiB data, 873 MiB used, 267 GiB / 268 GiB avail 2024-04-27T04:04:32.476 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:04:32 smithi181 ceph-mon[33774]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' 2024-04-27T04:04:32.476 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:04:32 smithi181 ceph-mon[33774]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' 2024-04-27T04:04:32.476 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:04:32 smithi181 ceph-mon[33774]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T04:04:32.476 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:04:32 smithi181 ceph-mon[33774]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T04:04:32.476 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:04:32 smithi181 ceph-mon[33774]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' 2024-04-27T04:04:32.477 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:04:32 smithi181 ceph-mon[33774]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-27T04:04:32.638 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:04:32 smithi162 ceph-mon[26841]: pgmap v78: 1 pgs: 1 active+clean; 577 KiB data, 873 MiB used, 267 GiB / 268 GiB avail 2024-04-27T04:04:32.638 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:04:32 smithi162 ceph-mon[26841]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' 2024-04-27T04:04:32.638 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:04:32 smithi162 ceph-mon[26841]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' 2024-04-27T04:04:32.638 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:04:32 smithi162 ceph-mon[26841]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T04:04:32.638 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:04:32 smithi162 ceph-mon[26841]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T04:04:32.638 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:04:32 smithi162 ceph-mon[26841]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' 2024-04-27T04:04:32.638 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:04:32 smithi162 ceph-mon[26841]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-27T04:04:34.284 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:04:34 smithi181 ceph-mon[33774]: pgmap v79: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail; 4.5 KiB/s rd, 196 KiB/s wr, 13 op/s 2024-04-27T04:04:34.638 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:04:34 smithi162 ceph-mon[26841]: pgmap v79: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail; 4.5 KiB/s rd, 196 KiB/s wr, 13 op/s 2024-04-27T04:04:36.423 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:04:36 smithi181 ceph-mon[33774]: pgmap v80: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail; 3.0 KiB/s rd, 130 KiB/s wr, 9 op/s 2024-04-27T04:04:36.423 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:04:36 smithi181 ceph-mon[33774]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-04-27T04:04:36.423 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:04:36 smithi181 ceph-mon[33774]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T04:04:36.638 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:04:36 smithi162 ceph-mon[26841]: pgmap v80: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail; 3.0 KiB/s rd, 130 KiB/s wr, 9 op/s 2024-04-27T04:04:36.638 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:04:36 smithi162 ceph-mon[26841]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-04-27T04:04:36.638 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:04:36 smithi162 ceph-mon[26841]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T04:04:37.388 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:04:37 smithi162 ceph-mon[26841]: Deploying daemon osd.3 on smithi181 2024-04-27T04:04:37.523 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:04:37 smithi181 ceph-mon[33774]: Deploying daemon osd.3 on smithi181 2024-04-27T04:04:38.523 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:04:38 smithi181 ceph-mon[33774]: pgmap v81: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail; 2.2 KiB/s rd, 98 KiB/s wr, 6 op/s 2024-04-27T04:04:38.638 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:04:38 smithi162 ceph-mon[26841]: pgmap v81: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail; 2.2 KiB/s rd, 98 KiB/s wr, 6 op/s 2024-04-27T04:04:40.507 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:04:40 smithi181 ceph-mon[33774]: pgmap v82: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail; 1.8 KiB/s rd, 78 KiB/s wr, 5 op/s 2024-04-27T04:04:40.638 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:04:40 smithi162 ceph-mon[26841]: pgmap v82: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail; 1.8 KiB/s rd, 78 KiB/s wr, 5 op/s 2024-04-27T04:04:42.371 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:04:42 smithi181 ceph-mon[33774]: pgmap v83: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail; 1.5 KiB/s rd, 65 KiB/s wr, 4 op/s 2024-04-27T04:04:42.371 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:04:42 smithi181 ceph-mon[33774]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' 2024-04-27T04:04:42.371 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:04:42 smithi181 ceph-mon[33774]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' 2024-04-27T04:04:42.372 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:04:42 smithi181 ceph-mon[33774]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T04:04:42.388 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:04:42 smithi162 ceph-mon[26841]: pgmap v83: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail; 1.5 KiB/s rd, 65 KiB/s wr, 4 op/s 2024-04-27T04:04:42.388 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:04:42 smithi162 ceph-mon[26841]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' 2024-04-27T04:04:42.388 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:04:42 smithi162 ceph-mon[26841]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' 2024-04-27T04:04:42.388 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:04:42 smithi162 ceph-mon[26841]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T04:04:43.669 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:04:43.668+0000 7f548d7fa700 1 -- 172.21.15.181:0/290607419 <== mgr.14210 v2:172.21.15.162:6800/4079782599 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f549002b400 con 0x7f5480069210 2024-04-27T04:04:43.669 INFO:teuthology.orchestra.run.smithi181.stdout:Created osd(s) 3 on host 'smithi181' 2024-04-27T04:04:43.672 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:04:43.671+0000 7f5496b9c700 1 -- 172.21.15.181:0/290607419 >> [v2:172.21.15.162:6800/4079782599,v1:172.21.15.162:6801/4079782599] conn(0x7f5480069210 msgr2=0x7f548006b6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:04:43.672 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:04:43.671+0000 7f5496b9c700 1 --2- 172.21.15.181:0/290607419 >> [v2:172.21.15.162:6800/4079782599,v1:172.21.15.162:6801/4079782599] conn(0x7f5480069210 0x7f548006b6c0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f547c009fd0 tx=0x7f547c00b040 comp rx=0 tx=0).stop 2024-04-27T04:04:43.672 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:04:43.671+0000 7f5496b9c700 1 -- 172.21.15.181:0/290607419 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f54900fbcf0 msgr2=0x7f549010d890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:04:43.672 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:04:43.671+0000 7f5496b9c700 1 --2- 172.21.15.181:0/290607419 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f54900fbcf0 0x7f549010d890 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7f548400b3b0 tx=0x7f5484009e30 comp rx=0 tx=0).stop 2024-04-27T04:04:43.672 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:04:43.671+0000 7f5496b9c700 1 -- 172.21.15.181:0/290607419 shutdown_connections 2024-04-27T04:04:43.672 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:04:43.671+0000 7f5496b9c700 1 --2- 172.21.15.181:0/290607419 >> [v2:172.21.15.162:6800/4079782599,v1:172.21.15.162:6801/4079782599] conn(0x7f5480069210 0x7f548006b6c0 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:04:43.672 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:04:43.671+0000 7f5496b9c700 1 --2- 172.21.15.181:0/290607419 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f54900fbcf0 0x7f549010d890 unknown :-1 s=CLOSED pgs=4 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:04:43.672 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:04:43.671+0000 7f5496b9c700 1 --2- 172.21.15.181:0/290607419 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f54900fcfa0 0x7f549010ddd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:04:43.672 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:04:43.671+0000 7f5496b9c700 1 -- 172.21.15.181:0/290607419 >> 172.21.15.181:0/290607419 conn(0x7f54900f7450 msgr2=0x7f54900f8a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:04:43.672 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:04:43.671+0000 7f5496b9c700 1 -- 172.21.15.181:0/290607419 shutdown_connections 2024-04-27T04:04:43.672 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:04:43.671+0000 7f5496b9c700 1 -- 172.21.15.181:0/290607419 wait complete. 2024-04-27T04:04:44.388 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:04:44 smithi162 ceph-mon[26841]: pgmap v84: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail; 1.5 KiB/s rd, 65 KiB/s wr, 4 op/s 2024-04-27T04:04:44.388 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:04:44 smithi162 ceph-mon[26841]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' 2024-04-27T04:04:44.388 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:04:44 smithi162 ceph-mon[26841]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' 2024-04-27T04:04:44.404 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:04:44 smithi181 ceph-mon[33774]: pgmap v84: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail; 1.5 KiB/s rd, 65 KiB/s wr, 4 op/s 2024-04-27T04:04:44.404 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:04:44 smithi181 ceph-mon[33774]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' 2024-04-27T04:04:44.404 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:04:44 smithi181 ceph-mon[33774]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' 2024-04-27T04:04:45.242 DEBUG:teuthology.orchestra.run.smithi181:osd.3> sudo journalctl -f -n 0 -u ceph-7f9750d6-044a-11ef-bc93-c7b262605968@osd.3.service 2024-04-27T04:04:45.244 INFO:tasks.cephadm:Deploying osd.4 on smithi181 with /dev/vg_nvme/lv_3... 2024-04-27T04:04:45.244 DEBUG:teuthology.orchestra.run.smithi181:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 7f9750d6-044a-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-04-27T04:04:45.388 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:04:45 smithi162 ceph-mon[26841]: from='osd.3 [v2:172.21.15.181:6800/3695382003,v1:172.21.15.181:6801/3695382003]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-04-27T04:04:45.388 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:04:45 smithi162 ceph-mon[26841]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-04-27T04:04:45.423 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:04:45 smithi181 ceph-mon[33774]: from='osd.3 [v2:172.21.15.181:6800/3695382003,v1:172.21.15.181:6801/3695382003]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-04-27T04:04:45.423 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:04:45 smithi181 ceph-mon[33774]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-04-27T04:04:45.539 INFO:teuthology.orchestra.run.smithi181.stderr:Inferring config /var/lib/ceph/7f9750d6-044a-11ef-bc93-c7b262605968/mon.smithi181/config 2024-04-27T04:04:46.302 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:04:46 smithi181 ceph-mon[33774]: pgmap v85: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail 2024-04-27T04:04:46.302 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:04:46 smithi181 ceph-mon[33774]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-04-27T04:04:46.302 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:04:46 smithi181 ceph-mon[33774]: osdmap e24: 4 total, 3 up, 4 in 2024-04-27T04:04:46.302 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:04:46 smithi181 ceph-mon[33774]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-27T04:04:46.302 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:04:46 smithi181 ceph-mon[33774]: from='osd.3 [v2:172.21.15.181:6800/3695382003,v1:172.21.15.181:6801/3695382003]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi181", "root=default"]}]: dispatch 2024-04-27T04:04:46.302 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:04:46 smithi181 ceph-mon[33774]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi181", "root=default"]}]: dispatch 2024-04-27T04:04:46.302 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:04:46 smithi181 ceph-mon[33774]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' 2024-04-27T04:04:46.302 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:04:46 smithi181 ceph-mon[33774]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' 2024-04-27T04:04:46.303 INFO:journalctl@ceph.osd.3.smithi181.stdout:Apr 27 04:04:46 smithi181 ceph-7f9750d6-044a-11ef-bc93-c7b262605968-osd-3[39355]: 2024-04-27T04:04:46.125+0000 7f9b465ee700 -1 osd.3 0 waiting for initial osdmap 2024-04-27T04:04:46.303 INFO:journalctl@ceph.osd.3.smithi181.stdout:Apr 27 04:04:46 smithi181 ceph-7f9750d6-044a-11ef-bc93-c7b262605968-osd-3[39355]: 2024-04-27T04:04:46.130+0000 7f9b413e9700 -1 osd.3 25 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-27T04:04:46.388 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:04:46 smithi162 ceph-mon[26841]: pgmap v85: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail 2024-04-27T04:04:46.388 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:04:46 smithi162 ceph-mon[26841]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-04-27T04:04:46.388 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:04:46 smithi162 ceph-mon[26841]: osdmap e24: 4 total, 3 up, 4 in 2024-04-27T04:04:46.388 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:04:46 smithi162 ceph-mon[26841]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-27T04:04:46.388 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:04:46 smithi162 ceph-mon[26841]: from='osd.3 [v2:172.21.15.181:6800/3695382003,v1:172.21.15.181:6801/3695382003]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi181", "root=default"]}]: dispatch 2024-04-27T04:04:46.388 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:04:46 smithi162 ceph-mon[26841]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi181", "root=default"]}]: dispatch 2024-04-27T04:04:46.388 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:04:46 smithi162 ceph-mon[26841]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' 2024-04-27T04:04:46.389 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:04:46 smithi162 ceph-mon[26841]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' 2024-04-27T04:04:47.210 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:04:47 smithi181 ceph-mon[33774]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi181", "root=default"]}]': finished 2024-04-27T04:04:47.211 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:04:47 smithi181 ceph-mon[33774]: osdmap e25: 4 total, 3 up, 4 in 2024-04-27T04:04:47.211 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:04:47 smithi181 ceph-mon[33774]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-27T04:04:47.211 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:04:47 smithi181 ceph-mon[33774]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-27T04:04:47.211 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:04:47 smithi181 ceph-mon[33774]: osd.3 [v2:172.21.15.181:6800/3695382003,v1:172.21.15.181:6801/3695382003] boot 2024-04-27T04:04:47.211 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:04:47 smithi181 ceph-mon[33774]: osdmap e26: 4 total, 4 up, 4 in 2024-04-27T04:04:47.211 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:04:47 smithi181 ceph-mon[33774]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-27T04:04:47.211 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:04:47 smithi181 ceph-mon[33774]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-27T04:04:47.388 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:04:47 smithi162 ceph-mon[26841]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi181", "root=default"]}]': finished 2024-04-27T04:04:47.388 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:04:47 smithi162 ceph-mon[26841]: osdmap e25: 4 total, 3 up, 4 in 2024-04-27T04:04:47.388 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:04:47 smithi162 ceph-mon[26841]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-27T04:04:47.388 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:04:47 smithi162 ceph-mon[26841]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-27T04:04:47.388 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:04:47 smithi162 ceph-mon[26841]: osd.3 [v2:172.21.15.181:6800/3695382003,v1:172.21.15.181:6801/3695382003] boot 2024-04-27T04:04:47.388 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:04:47 smithi162 ceph-mon[26841]: osdmap e26: 4 total, 4 up, 4 in 2024-04-27T04:04:47.388 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:04:47 smithi162 ceph-mon[26841]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-27T04:04:47.389 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:04:47 smithi162 ceph-mon[26841]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-27T04:04:48.022 INFO:teuthology.orchestra.run.smithi181.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-04-27T04:04:48.022 INFO:teuthology.orchestra.run.smithi181.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-04-27T04:04:48.022 INFO:teuthology.orchestra.run.smithi181.stderr: stderr: 10+0 records in 2024-04-27T04:04:48.022 INFO:teuthology.orchestra.run.smithi181.stderr:10+0 records out 2024-04-27T04:04:48.022 INFO:teuthology.orchestra.run.smithi181.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0122415 s, 857 MB/s 2024-04-27T04:04:48.022 INFO:teuthology.orchestra.run.smithi181.stderr:--> Zapping successful for: 2024-04-27T04:04:48.612 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:04:48 smithi181 ceph-mon[33774]: purged_snaps scrub starts 2024-04-27T04:04:48.612 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:04:48 smithi181 ceph-mon[33774]: purged_snaps scrub ok 2024-04-27T04:04:48.612 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:04:48 smithi181 ceph-mon[33774]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail 2024-04-27T04:04:48.613 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:04:48 smithi181 ceph-mon[33774]: osdmap e27: 4 total, 4 up, 4 in 2024-04-27T04:04:48.636 DEBUG:teuthology.orchestra.run.smithi181:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 7f9750d6-044a-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi181:vg_nvme/lv_3 2024-04-27T04:04:48.638 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:04:48 smithi162 ceph-mon[26841]: purged_snaps scrub starts 2024-04-27T04:04:48.638 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:04:48 smithi162 ceph-mon[26841]: purged_snaps scrub ok 2024-04-27T04:04:48.638 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:04:48 smithi162 ceph-mon[26841]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail 2024-04-27T04:04:48.638 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:04:48 smithi162 ceph-mon[26841]: osdmap e27: 4 total, 4 up, 4 in 2024-04-27T04:04:49.113 INFO:teuthology.orchestra.run.smithi181.stderr:Inferring config /var/lib/ceph/7f9750d6-044a-11ef-bc93-c7b262605968/mon.smithi181/config 2024-04-27T04:04:49.511 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:04:49 smithi181 ceph-mon[33774]: osdmap e28: 4 total, 4 up, 4 in 2024-04-27T04:04:49.638 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:04:49 smithi162 ceph-mon[26841]: osdmap e28: 4 total, 4 up, 4 in 2024-04-27T04:04:50.423 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:04:50 smithi181 ceph-mon[33774]: pgmap v92: 1 pgs: 1 remapped+peering; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-04-27T04:04:50.423 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:04:50 smithi181 ceph-mon[33774]: osdmap e29: 4 total, 4 up, 4 in 2024-04-27T04:04:50.638 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:04:50 smithi162 ceph-mon[26841]: pgmap v92: 1 pgs: 1 remapped+peering; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-04-27T04:04:50.638 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:04:50 smithi162 ceph-mon[26841]: osdmap e29: 4 total, 4 up, 4 in 2024-04-27T04:04:51.172 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:04:51.170+0000 7f1b1cdd4700 1 -- 172.21.15.181:0/3556886499 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1b180fdfd0 msgr2=0x7f1b180fe440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:04:51.172 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:04:51.170+0000 7f1b1cdd4700 1 --2- 172.21.15.181:0/3556886499 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1b180fdfd0 0x7f1b180fe440 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f1b0c004940 tx=0x7f1b0c009e30 comp rx=0 tx=0).stop 2024-04-27T04:04:51.172 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:04:51.171+0000 7f1b1cdd4700 1 -- 172.21.15.181:0/3556886499 shutdown_connections 2024-04-27T04:04:51.172 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:04:51.171+0000 7f1b1cdd4700 1 --2- 172.21.15.181:0/3556886499 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1b180fdfd0 0x7f1b180fe440 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:04:51.172 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:04:51.171+0000 7f1b1cdd4700 1 --2- 172.21.15.181:0/3556886499 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1b180fce60 0x7f1b180fd270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:04:51.172 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:04:51.171+0000 7f1b1cdd4700 1 -- 172.21.15.181:0/3556886499 >> 172.21.15.181:0/3556886499 conn(0x7f1b180f8540 msgr2=0x7f1b180fa950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:04:51.172 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:04:51.171+0000 7f1b1cdd4700 1 -- 172.21.15.181:0/3556886499 shutdown_connections 2024-04-27T04:04:51.172 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:04:51.171+0000 7f1b1cdd4700 1 -- 172.21.15.181:0/3556886499 wait complete. 2024-04-27T04:04:51.172 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:04:51.171+0000 7f1b1cdd4700 1 Processor -- start 2024-04-27T04:04:51.172 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:04:51.171+0000 7f1b1cdd4700 1 -- start start 2024-04-27T04:04:51.173 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:04:51.171+0000 7f1b1cdd4700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1b180fce60 0x7f1b18071c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:04:51.173 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:04:51.171+0000 7f1b1cdd4700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1b180fdfd0 0x7f1b180702b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:04:51.173 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:04:51.171+0000 7f1b1cdd4700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b180707f0 con 0x7f1b180fce60 2024-04-27T04:04:51.173 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:04:51.171+0000 7f1b1cdd4700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b18070960 con 0x7f1b180fdfd0 2024-04-27T04:04:51.173 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:04:51.171+0000 7f1b15d9b700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1b180fdfd0 0x7f1b180702b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:04:51.173 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:04:51.171+0000 7f1b15d9b700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1b180fdfd0 0x7f1b180702b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:42902/0 (socket says 172.21.15.181:42902) 2024-04-27T04:04:51.173 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:04:51.171+0000 7f1b15d9b700 1 -- 172.21.15.181:0/2143990930 learned_addr learned my addr 172.21.15.181:0/2143990930 (peer_addr_for_me v2:172.21.15.181:0/0) 2024-04-27T04:04:51.173 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:04:51.172+0000 7f1b15d9b700 1 -- 172.21.15.181:0/2143990930 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1b180fce60 msgr2=0x7f1b18071c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:04:51.173 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:04:51.172+0000 7f1b15d9b700 1 --2- 172.21.15.181:0/2143990930 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1b180fce60 0x7f1b18071c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:04:51.173 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:04:51.172+0000 7f1b15d9b700 1 -- 172.21.15.181:0/2143990930 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b0c005040 con 0x7f1b180fdfd0 2024-04-27T04:04:51.173 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:04:51.172+0000 7f1b15d9b700 1 --2- 172.21.15.181:0/2143990930 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1b180fdfd0 0x7f1b180702b0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f1b0c00b3b0 tx=0x7f1b0c009e30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:04:51.173 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:04:51.172+0000 7f1b06ffd700 1 -- 172.21.15.181:0/2143990930 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1b0c00e040 con 0x7f1b180fdfd0 2024-04-27T04:04:51.173 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:04:51.172+0000 7f1b06ffd700 1 -- 172.21.15.181:0/2143990930 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f1b0c00bcc0 con 0x7f1b180fdfd0 2024-04-27T04:04:51.173 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:04:51.172+0000 7f1b06ffd700 1 -- 172.21.15.181:0/2143990930 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1b0c01c630 con 0x7f1b180fdfd0 2024-04-27T04:04:51.173 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:04:51.172+0000 7f1b1cdd4700 1 -- 172.21.15.181:0/2143990930 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1b18070be0 con 0x7f1b180fdfd0 2024-04-27T04:04:51.174 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:04:51.172+0000 7f1b1cdd4700 1 -- 172.21.15.181:0/2143990930 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1b18115a40 con 0x7f1b180fdfd0 2024-04-27T04:04:51.174 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:04:51.173+0000 7f1b1cdd4700 1 -- 172.21.15.181:0/2143990930 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1b1804c710 con 0x7f1b180fdfd0 2024-04-27T04:04:51.175 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:04:51.173+0000 7f1b06ffd700 1 -- 172.21.15.181:0/2143990930 <== mon.1 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 19) v1 ==== 86631+0+0 (secure 0 0 0) 0x7f1b0c013980 con 0x7f1b180fdfd0 2024-04-27T04:04:51.175 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:04:51.174+0000 7f1b06ffd700 1 --2- 172.21.15.181:0/2143990930 >> [v2:172.21.15.162:6800/4079782599,v1:172.21.15.162:6801/4079782599] conn(0x7f1b000692a0 0x7f1b0006b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:04:51.175 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:04:51.174+0000 7f1b06ffd700 1 -- 172.21.15.181:0/2143990930 <== mon.1 v2:172.21.15.181:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 3770+0+0 (secure 0 0 0) 0x7f1b0c087870 con 0x7f1b180fdfd0 2024-04-27T04:04:51.175 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:04:51.174+0000 7f1b1659c700 1 --2- 172.21.15.181:0/2143990930 >> [v2:172.21.15.162:6800/4079782599,v1:172.21.15.162:6801/4079782599] conn(0x7f1b000692a0 0x7f1b0006b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:04:51.175 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:04:51.174+0000 7f1b1659c700 1 --2- 172.21.15.181:0/2143990930 >> [v2:172.21.15.162:6800/4079782599,v1:172.21.15.162:6801/4079782599] conn(0x7f1b000692a0 0x7f1b0006b750 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f1b08006fd0 tx=0x7f1b08006db0 comp rx=0 tx=0).ready entity=mgr.14210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:04:51.177 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:04:51.178+0000 7f1b06ffd700 1 -- 172.21.15.181:0/2143990930 <== mon.1 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f1b0c057cd0 con 0x7f1b180fdfd0 2024-04-27T04:04:51.372 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:04:51.373+0000 7f1b1cdd4700 1 -- 172.21.15.181:0/2143990930 --> [v2:172.21.15.162:6800/4079782599,v1:172.21.15.162:6801/4079782599] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi181:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) v1 -- 0x7f1b18102270 con 0x7f1b000692a0 2024-04-27T04:04:51.638 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:04:51 smithi162 ceph-mon[26841]: osdmap e30: 4 total, 4 up, 4 in 2024-04-27T04:04:51.638 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:04:51 smithi162 ceph-mon[26841]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' 2024-04-27T04:04:51.638 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:04:51 smithi162 ceph-mon[26841]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' 2024-04-27T04:04:51.639 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:04:51 smithi162 ceph-mon[26841]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-27T04:04:51.639 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:04:51 smithi162 ceph-mon[26841]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' 2024-04-27T04:04:51.639 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:04:51 smithi162 ceph-mon[26841]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T04:04:51.639 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:04:51 smithi162 ceph-mon[26841]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T04:04:51.639 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:04:51 smithi162 ceph-mon[26841]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' 2024-04-27T04:04:51.639 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:04:51 smithi162 ceph-mon[26841]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T04:04:51.639 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:04:51 smithi162 ceph-mon[26841]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T04:04:51.639 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:04:51 smithi162 ceph-mon[26841]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T04:04:51.639 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:04:51 smithi162 ceph-mon[26841]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' 2024-04-27T04:04:51.673 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:04:51 smithi181 ceph-mon[33774]: osdmap e30: 4 total, 4 up, 4 in 2024-04-27T04:04:51.673 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:04:51 smithi181 ceph-mon[33774]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' 2024-04-27T04:04:51.673 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:04:51 smithi181 ceph-mon[33774]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' 2024-04-27T04:04:51.673 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:04:51 smithi181 ceph-mon[33774]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-27T04:04:51.673 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:04:51 smithi181 ceph-mon[33774]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' 2024-04-27T04:04:51.673 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:04:51 smithi181 ceph-mon[33774]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T04:04:51.674 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:04:51 smithi181 ceph-mon[33774]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T04:04:51.674 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:04:51 smithi181 ceph-mon[33774]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' 2024-04-27T04:04:51.674 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:04:51 smithi181 ceph-mon[33774]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T04:04:51.674 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:04:51 smithi181 ceph-mon[33774]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T04:04:51.674 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:04:51 smithi181 ceph-mon[33774]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T04:04:51.674 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:04:51 smithi181 ceph-mon[33774]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' 2024-04-27T04:04:52.423 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:04:52 smithi181 ceph-mon[33774]: Detected new or changed devices on smithi181 2024-04-27T04:04:52.423 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:04:52 smithi181 ceph-mon[33774]: Adjusting osd_memory_target on smithi181 to 15936M 2024-04-27T04:04:52.423 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:04:52 smithi181 ceph-mon[33774]: pgmap v95: 1 pgs: 1 remapped+peering; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-04-27T04:04:52.424 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:04:52 smithi181 ceph-mon[33774]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-27T04:04:52.424 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:04:52 smithi181 ceph-mon[33774]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-27T04:04:52.424 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:04:52 smithi181 ceph-mon[33774]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T04:04:52.638 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:04:52 smithi162 ceph-mon[26841]: Detected new or changed devices on smithi181 2024-04-27T04:04:52.638 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:04:52 smithi162 ceph-mon[26841]: Adjusting osd_memory_target on smithi181 to 15936M 2024-04-27T04:04:52.638 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:04:52 smithi162 ceph-mon[26841]: pgmap v95: 1 pgs: 1 remapped+peering; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-04-27T04:04:52.638 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:04:52 smithi162 ceph-mon[26841]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-27T04:04:52.638 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:04:52 smithi162 ceph-mon[26841]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-27T04:04:52.638 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:04:52 smithi162 ceph-mon[26841]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T04:04:53.423 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:04:53 smithi181 ceph-mon[33774]: from='client.24149 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi181:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-27T04:04:53.638 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:04:53 smithi162 ceph-mon[26841]: from='client.24149 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi181:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-27T04:04:54.382 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:04:54 smithi181 ceph-mon[33774]: pgmap v96: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail; 106 KiB/s, 0 objects/s recovering 2024-04-27T04:04:54.382 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:04:54 smithi181 ceph-mon[33774]: from='client.? 172.21.15.181:0/3157960538' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "bd70494d-061c-4468-b837-26150c2d4208"}]: dispatch 2024-04-27T04:04:54.382 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:04:54 smithi181 ceph-mon[33774]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "bd70494d-061c-4468-b837-26150c2d4208"}]: dispatch 2024-04-27T04:04:54.383 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:04:54 smithi181 ceph-mon[33774]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "bd70494d-061c-4468-b837-26150c2d4208"}]': finished 2024-04-27T04:04:54.383 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:04:54 smithi181 ceph-mon[33774]: osdmap e31: 5 total, 4 up, 5 in 2024-04-27T04:04:54.383 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:04:54 smithi181 ceph-mon[33774]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-27T04:04:54.638 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:04:54 smithi162 ceph-mon[26841]: pgmap v96: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail; 106 KiB/s, 0 objects/s recovering 2024-04-27T04:04:54.638 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:04:54 smithi162 ceph-mon[26841]: from='client.? 172.21.15.181:0/3157960538' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "bd70494d-061c-4468-b837-26150c2d4208"}]: dispatch 2024-04-27T04:04:54.638 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:04:54 smithi162 ceph-mon[26841]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "bd70494d-061c-4468-b837-26150c2d4208"}]: dispatch 2024-04-27T04:04:54.639 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:04:54 smithi162 ceph-mon[26841]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "bd70494d-061c-4468-b837-26150c2d4208"}]': finished 2024-04-27T04:04:54.639 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:04:54 smithi162 ceph-mon[26841]: osdmap e31: 5 total, 4 up, 5 in 2024-04-27T04:04:54.639 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:04:54 smithi162 ceph-mon[26841]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-27T04:04:55.638 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:04:55 smithi162 ceph-mon[26841]: from='client.? 172.21.15.181:0/1388423546' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-27T04:04:55.673 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:04:55 smithi181 ceph-mon[33774]: from='client.? 172.21.15.181:0/1388423546' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-27T04:04:56.638 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:04:56 smithi162 ceph-mon[26841]: pgmap v98: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-04-27T04:04:56.673 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:04:56 smithi181 ceph-mon[33774]: pgmap v98: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-04-27T04:04:58.411 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:04:58 smithi162 ceph-mon[26841]: pgmap v99: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail; 78 KiB/s, 0 objects/s recovering 2024-04-27T04:04:58.673 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:04:58 smithi181 ceph-mon[33774]: pgmap v99: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail; 78 KiB/s, 0 objects/s recovering 2024-04-27T04:05:00.583 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:05:00 smithi181 ceph-mon[33774]: pgmap v100: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail; 69 KiB/s, 0 objects/s recovering 2024-04-27T04:05:00.638 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:05:00 smithi162 ceph-mon[26841]: pgmap v100: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail; 69 KiB/s, 0 objects/s recovering 2024-04-27T04:05:01.571 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:05:01 smithi181 ceph-mon[33774]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-04-27T04:05:01.571 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:05:01 smithi181 ceph-mon[33774]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T04:05:01.638 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:05:01 smithi162 ceph-mon[26841]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-04-27T04:05:01.638 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:05:01 smithi162 ceph-mon[26841]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T04:05:02.423 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:05:02 smithi181 ceph-mon[33774]: pgmap v101: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-04-27T04:05:02.423 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:05:02 smithi181 ceph-mon[33774]: Deploying daemon osd.4 on smithi181 2024-04-27T04:05:02.423 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:05:02 smithi181 ceph-mon[33774]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-27T04:05:02.638 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:05:02 smithi162 ceph-mon[26841]: pgmap v101: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-04-27T04:05:02.638 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:05:02 smithi162 ceph-mon[26841]: Deploying daemon osd.4 on smithi181 2024-04-27T04:05:02.638 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:05:02 smithi162 ceph-mon[26841]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-27T04:05:04.334 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:05:04 smithi181 ceph-mon[33774]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-04-27T04:05:04.638 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:05:04 smithi162 ceph-mon[26841]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-04-27T04:05:06.468 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:05:06 smithi181 ceph-mon[33774]: pgmap v103: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-04-27T04:05:06.469 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:05:06 smithi181 ceph-mon[33774]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' 2024-04-27T04:05:06.469 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:05:06 smithi181 ceph-mon[33774]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T04:05:06.469 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:05:06 smithi181 ceph-mon[33774]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' 2024-04-27T04:05:06.638 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:05:06 smithi162 ceph-mon[26841]: pgmap v103: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-04-27T04:05:06.638 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:05:06 smithi162 ceph-mon[26841]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' 2024-04-27T04:05:06.638 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:05:06 smithi162 ceph-mon[26841]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T04:05:06.638 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:05:06 smithi162 ceph-mon[26841]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' 2024-04-27T04:05:08.423 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:05:08 smithi181 ceph-mon[33774]: pgmap v104: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-04-27T04:05:08.638 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:05:08 smithi162 ceph-mon[26841]: pgmap v104: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-04-27T04:05:09.134 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:05:09.134+0000 7f1b06ffd700 1 -- 172.21.15.181:0/2143990930 <== mgr.14210 v2:172.21.15.162:6800/4079782599 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f1b18102270 con 0x7f1b000692a0 2024-04-27T04:05:09.134 INFO:teuthology.orchestra.run.smithi181.stdout:Created osd(s) 4 on host 'smithi181' 2024-04-27T04:05:09.136 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:05:09.136+0000 7f1b1cdd4700 1 -- 172.21.15.181:0/2143990930 >> [v2:172.21.15.162:6800/4079782599,v1:172.21.15.162:6801/4079782599] conn(0x7f1b000692a0 msgr2=0x7f1b0006b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:05:09.136 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:05:09.136+0000 7f1b1cdd4700 1 --2- 172.21.15.181:0/2143990930 >> [v2:172.21.15.162:6800/4079782599,v1:172.21.15.162:6801/4079782599] conn(0x7f1b000692a0 0x7f1b0006b750 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f1b08006fd0 tx=0x7f1b08006db0 comp rx=0 tx=0).stop 2024-04-27T04:05:09.136 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:05:09.136+0000 7f1b1cdd4700 1 -- 172.21.15.181:0/2143990930 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1b180fdfd0 msgr2=0x7f1b180702b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:05:09.136 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:05:09.136+0000 7f1b1cdd4700 1 --2- 172.21.15.181:0/2143990930 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1b180fdfd0 0x7f1b180702b0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f1b0c00b3b0 tx=0x7f1b0c009e30 comp rx=0 tx=0).stop 2024-04-27T04:05:09.136 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:05:09.136+0000 7f1b1cdd4700 1 -- 172.21.15.181:0/2143990930 shutdown_connections 2024-04-27T04:05:09.136 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:05:09.136+0000 7f1b1cdd4700 1 --2- 172.21.15.181:0/2143990930 >> [v2:172.21.15.162:6800/4079782599,v1:172.21.15.162:6801/4079782599] conn(0x7f1b000692a0 0x7f1b0006b750 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:05:09.136 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:05:09.136+0000 7f1b1cdd4700 1 --2- 172.21.15.181:0/2143990930 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1b180fce60 0x7f1b18071c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:05:09.136 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:05:09.136+0000 7f1b1cdd4700 1 --2- 172.21.15.181:0/2143990930 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1b180fdfd0 0x7f1b180702b0 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:05:09.136 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:05:09.136+0000 7f1b1cdd4700 1 -- 172.21.15.181:0/2143990930 >> 172.21.15.181:0/2143990930 conn(0x7f1b180f8540 msgr2=0x7f1b180f9b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:05:09.136 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:05:09.137+0000 7f1b1cdd4700 1 -- 172.21.15.181:0/2143990930 shutdown_connections 2024-04-27T04:05:09.137 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:05:09.137+0000 7f1b1cdd4700 1 -- 172.21.15.181:0/2143990930 wait complete. 2024-04-27T04:05:10.052 DEBUG:teuthology.orchestra.run.smithi181:osd.4> sudo journalctl -f -n 0 -u ceph-7f9750d6-044a-11ef-bc93-c7b262605968@osd.4.service 2024-04-27T04:05:10.054 INFO:tasks.cephadm:Deploying osd.5 on smithi181 with /dev/vg_nvme/lv_2... 2024-04-27T04:05:10.055 DEBUG:teuthology.orchestra.run.smithi181:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 7f9750d6-044a-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-04-27T04:05:10.352 INFO:teuthology.orchestra.run.smithi181.stderr:Inferring config /var/lib/ceph/7f9750d6-044a-11ef-bc93-c7b262605968/mon.smithi181/config 2024-04-27T04:05:10.388 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:05:10 smithi162 ceph-mon[26841]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-04-27T04:05:10.388 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:05:10 smithi162 ceph-mon[26841]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' 2024-04-27T04:05:10.388 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:05:10 smithi162 ceph-mon[26841]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' 2024-04-27T04:05:10.388 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:05:10 smithi162 ceph-mon[26841]: from='osd.4 [v2:172.21.15.181:6808/2791715456,v1:172.21.15.181:6809/2791715456]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-04-27T04:05:10.388 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:05:10 smithi162 ceph-mon[26841]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-04-27T04:05:10.423 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:05:10 smithi181 ceph-mon[33774]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-04-27T04:05:10.423 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:05:10 smithi181 ceph-mon[33774]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' 2024-04-27T04:05:10.423 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:05:10 smithi181 ceph-mon[33774]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' 2024-04-27T04:05:10.424 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:05:10 smithi181 ceph-mon[33774]: from='osd.4 [v2:172.21.15.181:6808/2791715456,v1:172.21.15.181:6809/2791715456]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-04-27T04:05:10.424 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:05:10 smithi181 ceph-mon[33774]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-04-27T04:05:11.325 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:05:11 smithi181 ceph-mon[33774]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' 2024-04-27T04:05:11.325 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:05:11 smithi181 ceph-mon[33774]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' 2024-04-27T04:05:11.325 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:05:11 smithi181 ceph-mon[33774]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-04-27T04:05:11.325 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:05:11 smithi181 ceph-mon[33774]: osdmap e32: 5 total, 4 up, 5 in 2024-04-27T04:05:11.325 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:05:11 smithi181 ceph-mon[33774]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-27T04:05:11.325 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:05:11 smithi181 ceph-mon[33774]: from='osd.4 [v2:172.21.15.181:6808/2791715456,v1:172.21.15.181:6809/2791715456]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi181", "root=default"]}]: dispatch 2024-04-27T04:05:11.325 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:05:11 smithi181 ceph-mon[33774]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi181", "root=default"]}]: dispatch 2024-04-27T04:05:11.388 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:05:11 smithi162 ceph-mon[26841]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' 2024-04-27T04:05:11.388 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:05:11 smithi162 ceph-mon[26841]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' 2024-04-27T04:05:11.388 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:05:11 smithi162 ceph-mon[26841]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-04-27T04:05:11.388 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:05:11 smithi162 ceph-mon[26841]: osdmap e32: 5 total, 4 up, 5 in 2024-04-27T04:05:11.388 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:05:11 smithi162 ceph-mon[26841]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-27T04:05:11.388 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:05:11 smithi162 ceph-mon[26841]: from='osd.4 [v2:172.21.15.181:6808/2791715456,v1:172.21.15.181:6809/2791715456]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi181", "root=default"]}]: dispatch 2024-04-27T04:05:11.388 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:05:11 smithi162 ceph-mon[26841]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi181", "root=default"]}]: dispatch 2024-04-27T04:05:11.611 INFO:journalctl@ceph.osd.4.smithi181.stdout:Apr 27 04:05:11 smithi181 ceph-7f9750d6-044a-11ef-bc93-c7b262605968-osd-4[46239]: 2024-04-27T04:05:11.340+0000 7f88182fe700 -1 osd.4 0 waiting for initial osdmap 2024-04-27T04:05:11.611 INFO:journalctl@ceph.osd.4.smithi181.stdout:Apr 27 04:05:11 smithi181 ceph-7f9750d6-044a-11ef-bc93-c7b262605968-osd-4[46239]: 2024-04-27T04:05:11.349+0000 7f88140fb700 -1 osd.4 33 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-27T04:05:12.537 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:05:12 smithi181 ceph-mon[33774]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-04-27T04:05:12.538 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:05:12 smithi181 ceph-mon[33774]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi181", "root=default"]}]': finished 2024-04-27T04:05:12.538 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:05:12 smithi181 ceph-mon[33774]: osdmap e33: 5 total, 4 up, 5 in 2024-04-27T04:05:12.538 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:05:12 smithi181 ceph-mon[33774]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-27T04:05:12.538 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:05:12 smithi181 ceph-mon[33774]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-27T04:05:12.638 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:05:12 smithi162 ceph-mon[26841]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-04-27T04:05:12.638 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:05:12 smithi162 ceph-mon[26841]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi181", "root=default"]}]': finished 2024-04-27T04:05:12.638 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:05:12 smithi162 ceph-mon[26841]: osdmap e33: 5 total, 4 up, 5 in 2024-04-27T04:05:12.638 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:05:12 smithi162 ceph-mon[26841]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-27T04:05:12.638 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:05:12 smithi162 ceph-mon[26841]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-27T04:05:13.423 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:05:13 smithi181 ceph-mon[33774]: purged_snaps scrub starts 2024-04-27T04:05:13.423 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:05:13 smithi181 ceph-mon[33774]: purged_snaps scrub ok 2024-04-27T04:05:13.424 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:05:13 smithi181 ceph-mon[33774]: osd.4 [v2:172.21.15.181:6808/2791715456,v1:172.21.15.181:6809/2791715456] boot 2024-04-27T04:05:13.424 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:05:13 smithi181 ceph-mon[33774]: osdmap e34: 5 total, 5 up, 5 in 2024-04-27T04:05:13.424 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:05:13 smithi181 ceph-mon[33774]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-27T04:05:13.424 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:05:13 smithi181 ceph-mon[33774]: pgmap v110: 1 pgs: 1 peering; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-04-27T04:05:13.576 INFO:teuthology.orchestra.run.smithi181.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-04-27T04:05:13.576 INFO:teuthology.orchestra.run.smithi181.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-04-27T04:05:13.576 INFO:teuthology.orchestra.run.smithi181.stderr: stderr: 10+0 records in 2024-04-27T04:05:13.576 INFO:teuthology.orchestra.run.smithi181.stderr:10+0 records out 2024-04-27T04:05:13.576 INFO:teuthology.orchestra.run.smithi181.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.012381 s, 847 MB/s 2024-04-27T04:05:13.576 INFO:teuthology.orchestra.run.smithi181.stderr:--> Zapping successful for: 2024-04-27T04:05:13.638 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:05:13 smithi162 ceph-mon[26841]: purged_snaps scrub starts 2024-04-27T04:05:13.638 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:05:13 smithi162 ceph-mon[26841]: purged_snaps scrub ok 2024-04-27T04:05:13.638 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:05:13 smithi162 ceph-mon[26841]: osd.4 [v2:172.21.15.181:6808/2791715456,v1:172.21.15.181:6809/2791715456] boot 2024-04-27T04:05:13.638 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:05:13 smithi162 ceph-mon[26841]: osdmap e34: 5 total, 5 up, 5 in 2024-04-27T04:05:13.638 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:05:13 smithi162 ceph-mon[26841]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-27T04:05:13.638 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:05:13 smithi162 ceph-mon[26841]: pgmap v110: 1 pgs: 1 peering; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-04-27T04:05:14.511 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:05:14 smithi181 ceph-mon[33774]: osdmap e35: 5 total, 5 up, 5 in 2024-04-27T04:05:14.512 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:05:14 smithi181 ceph-mon[33774]: Detected new or changed devices on smithi181 2024-04-27T04:05:14.512 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:05:14 smithi181 ceph-mon[33774]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' 2024-04-27T04:05:14.512 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:05:14 smithi181 ceph-mon[33774]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' 2024-04-27T04:05:14.512 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:05:14 smithi181 ceph-mon[33774]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-27T04:05:14.512 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:05:14 smithi181 ceph-mon[33774]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-27T04:05:14.513 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:05:14 smithi181 ceph-mon[33774]: Adjusting osd_memory_target on smithi181 to 7968M 2024-04-27T04:05:14.513 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:05:14 smithi181 ceph-mon[33774]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' 2024-04-27T04:05:14.513 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:05:14 smithi181 ceph-mon[33774]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T04:05:14.513 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:05:14 smithi181 ceph-mon[33774]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T04:05:14.513 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:05:14 smithi181 ceph-mon[33774]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' 2024-04-27T04:05:14.513 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:05:14 smithi181 ceph-mon[33774]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T04:05:14.513 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:05:14 smithi181 ceph-mon[33774]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T04:05:14.514 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:05:14 smithi181 ceph-mon[33774]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T04:05:14.514 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:05:14 smithi181 ceph-mon[33774]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' 2024-04-27T04:05:14.535 DEBUG:teuthology.orchestra.run.smithi181:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 7f9750d6-044a-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi181:vg_nvme/lv_2 2024-04-27T04:05:14.638 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:05:14 smithi162 ceph-mon[26841]: osdmap e35: 5 total, 5 up, 5 in 2024-04-27T04:05:14.638 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:05:14 smithi162 ceph-mon[26841]: Detected new or changed devices on smithi181 2024-04-27T04:05:14.638 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:05:14 smithi162 ceph-mon[26841]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' 2024-04-27T04:05:14.638 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:05:14 smithi162 ceph-mon[26841]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' 2024-04-27T04:05:14.638 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:05:14 smithi162 ceph-mon[26841]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-27T04:05:14.638 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:05:14 smithi162 ceph-mon[26841]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-27T04:05:14.638 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:05:14 smithi162 ceph-mon[26841]: Adjusting osd_memory_target on smithi181 to 7968M 2024-04-27T04:05:14.638 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:05:14 smithi162 ceph-mon[26841]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' 2024-04-27T04:05:14.639 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:05:14 smithi162 ceph-mon[26841]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T04:05:14.639 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:05:14 smithi162 ceph-mon[26841]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T04:05:14.639 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:05:14 smithi162 ceph-mon[26841]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' 2024-04-27T04:05:14.639 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:05:14 smithi162 ceph-mon[26841]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T04:05:14.639 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:05:14 smithi162 ceph-mon[26841]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T04:05:14.639 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:05:14 smithi162 ceph-mon[26841]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T04:05:14.639 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:05:14 smithi162 ceph-mon[26841]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' 2024-04-27T04:05:14.786 INFO:teuthology.orchestra.run.smithi181.stderr:Inferring config /var/lib/ceph/7f9750d6-044a-11ef-bc93-c7b262605968/mon.smithi181/config 2024-04-27T04:05:15.634 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:05:15 smithi181 ceph-mon[33774]: pgmap v112: 1 pgs: 1 peering; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-04-27T04:05:15.637 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:05:15 smithi162 ceph-mon[26841]: pgmap v112: 1 pgs: 1 peering; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-04-27T04:05:16.151 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:05:16.149+0000 7f54cdaa6700 1 -- 172.21.15.181:0/3289204996 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f54c80fce60 msgr2=0x7f54c80fd270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:05:16.151 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:05:16.149+0000 7f54cdaa6700 1 --2- 172.21.15.181:0/3289204996 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f54c80fce60 0x7f54c80fd270 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f54b8004940 tx=0x7f54b8009e30 comp rx=0 tx=0).stop 2024-04-27T04:05:16.151 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:05:16.149+0000 7f54cdaa6700 1 -- 172.21.15.181:0/3289204996 shutdown_connections 2024-04-27T04:05:16.151 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:05:16.149+0000 7f54cdaa6700 1 --2- 172.21.15.181:0/3289204996 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f54c80fdfd0 0x7f54c80fe440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:05:16.151 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:05:16.149+0000 7f54cdaa6700 1 --2- 172.21.15.181:0/3289204996 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f54c80fce60 0x7f54c80fd270 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:05:16.152 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:05:16.149+0000 7f54cdaa6700 1 -- 172.21.15.181:0/3289204996 >> 172.21.15.181:0/3289204996 conn(0x7f54c80f8540 msgr2=0x7f54c80fa950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:05:16.152 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:05:16.149+0000 7f54cdaa6700 1 -- 172.21.15.181:0/3289204996 shutdown_connections 2024-04-27T04:05:16.152 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:05:16.149+0000 7f54cdaa6700 1 -- 172.21.15.181:0/3289204996 wait complete. 2024-04-27T04:05:16.152 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:05:16.150+0000 7f54cdaa6700 1 Processor -- start 2024-04-27T04:05:16.152 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:05:16.150+0000 7f54cdaa6700 1 -- start start 2024-04-27T04:05:16.152 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:05:16.150+0000 7f54cdaa6700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f54c80fce60 0x7f54c810d930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:05:16.152 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:05:16.150+0000 7f54cdaa6700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f54c80fdfd0 0x7f54c810de70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:05:16.152 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:05:16.150+0000 7f54cdaa6700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54c810e500 con 0x7f54c80fdfd0 2024-04-27T04:05:16.152 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:05:16.150+0000 7f54cdaa6700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54c8112ea0 con 0x7f54c80fce60 2024-04-27T04:05:16.152 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:05:16.150+0000 7f54c77fe700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f54c80fce60 0x7f54c810d930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:05:16.152 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:05:16.150+0000 7f54c77fe700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f54c80fce60 0x7f54c810d930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:35392/0 (socket says 172.21.15.181:35392) 2024-04-27T04:05:16.152 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:05:16.150+0000 7f54c77fe700 1 -- 172.21.15.181:0/3175818494 learned_addr learned my addr 172.21.15.181:0/3175818494 (peer_addr_for_me v2:172.21.15.181:0/0) 2024-04-27T04:05:16.152 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:05:16.150+0000 7f54beffd700 1 --2- 172.21.15.181:0/3175818494 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f54c80fdfd0 0x7f54c810de70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:05:16.152 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:05:16.150+0000 7f54c77fe700 1 -- 172.21.15.181:0/3175818494 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f54c80fdfd0 msgr2=0x7f54c810de70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:05:16.152 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:05:16.150+0000 7f54c77fe700 1 --2- 172.21.15.181:0/3175818494 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f54c80fdfd0 0x7f54c810de70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:05:16.153 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:05:16.150+0000 7f54c77fe700 1 -- 172.21.15.181:0/3175818494 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f54b8005040 con 0x7f54c80fce60 2024-04-27T04:05:16.153 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:05:16.150+0000 7f54beffd700 1 --2- 172.21.15.181:0/3175818494 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f54c80fdfd0 0x7f54c810de70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-27T04:05:16.153 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:05:16.151+0000 7f54c77fe700 1 --2- 172.21.15.181:0/3175818494 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f54c80fce60 0x7f54c810d930 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f54b800b3b0 tx=0x7f54b8009e30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:05:16.153 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:05:16.151+0000 7f54c4ff9700 1 -- 172.21.15.181:0/3175818494 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f54b800e040 con 0x7f54c80fce60 2024-04-27T04:05:16.153 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:05:16.151+0000 7f54cdaa6700 1 -- 172.21.15.181:0/3175818494 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f54c81130a0 con 0x7f54c80fce60 2024-04-27T04:05:16.153 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:05:16.151+0000 7f54c4ff9700 1 -- 172.21.15.181:0/3175818494 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f54b800bcc0 con 0x7f54c80fce60 2024-04-27T04:05:16.153 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:05:16.151+0000 7f54c4ff9700 1 -- 172.21.15.181:0/3175818494 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f54b801c630 con 0x7f54c80fce60 2024-04-27T04:05:16.154 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:05:16.151+0000 7f54cdaa6700 1 -- 172.21.15.181:0/3175818494 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f54c8113610 con 0x7f54c80fce60 2024-04-27T04:05:16.154 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:05:16.152+0000 7f54cdaa6700 1 -- 172.21.15.181:0/3175818494 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f54c804c710 con 0x7f54c80fce60 2024-04-27T04:05:16.154 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:05:16.152+0000 7f54c4ff9700 1 -- 172.21.15.181:0/3175818494 <== mon.1 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 19) v1 ==== 86631+0+0 (secure 0 0 0) 0x7f54b8026440 con 0x7f54c80fce60 2024-04-27T04:05:16.154 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:05:16.152+0000 7f54c4ff9700 1 --2- 172.21.15.181:0/3175818494 >> [v2:172.21.15.162:6800/4079782599,v1:172.21.15.162:6801/4079782599] conn(0x7f54b40692a0 0x7f54b406b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:05:16.154 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:05:16.153+0000 7f54c4ff9700 1 -- 172.21.15.181:0/3175818494 <== mon.1 v2:172.21.15.181:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 4202+0+0 (secure 0 0 0) 0x7f54b8087af0 con 0x7f54c80fce60 2024-04-27T04:05:16.154 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:05:16.153+0000 7f54beffd700 1 --2- 172.21.15.181:0/3175818494 >> [v2:172.21.15.162:6800/4079782599,v1:172.21.15.162:6801/4079782599] conn(0x7f54b40692a0 0x7f54b406b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:05:16.154 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:05:16.153+0000 7f54beffd700 1 --2- 172.21.15.181:0/3175818494 >> [v2:172.21.15.162:6800/4079782599,v1:172.21.15.162:6801/4079782599] conn(0x7f54b40692a0 0x7f54b406b750 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f54b0006fd0 tx=0x7f54b0008040 comp rx=0 tx=0).ready entity=mgr.14210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:05:16.157 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:05:16.157+0000 7f54c4ff9700 1 -- 172.21.15.181:0/3175818494 <== mon.1 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f54b8057d20 con 0x7f54c80fce60 2024-04-27T04:05:16.351 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:05:16.350+0000 7f54cdaa6700 1 -- 172.21.15.181:0/3175818494 --> [v2:172.21.15.162:6800/4079782599,v1:172.21.15.162:6801/4079782599] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi181:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) v1 -- 0x7f54c8102270 con 0x7f54b40692a0 2024-04-27T04:05:16.673 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:05:16 smithi181 ceph-mon[33774]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-27T04:05:16.673 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:05:16 smithi181 ceph-mon[33774]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-27T04:05:16.673 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:05:16 smithi181 ceph-mon[33774]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T04:05:16.888 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:05:16 smithi162 ceph-mon[26841]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-27T04:05:16.888 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:05:16 smithi162 ceph-mon[26841]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-27T04:05:16.888 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:05:16 smithi162 ceph-mon[26841]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T04:05:17.673 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:05:17 smithi181 ceph-mon[33774]: from='client.24167 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi181:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-27T04:05:17.673 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:05:17 smithi181 ceph-mon[33774]: pgmap v113: 1 pgs: 1 peering; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-04-27T04:05:17.673 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:05:17 smithi181 ceph-mon[33774]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-27T04:05:17.679 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:05:17 smithi162 ceph-mon[26841]: from='client.24167 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi181:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-27T04:05:17.679 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:05:17 smithi162 ceph-mon[26841]: pgmap v113: 1 pgs: 1 peering; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-04-27T04:05:17.679 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:05:17 smithi162 ceph-mon[26841]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-27T04:05:18.923 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:05:18 smithi181 ceph-mon[33774]: from='client.? 172.21.15.181:0/1446013698' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "15455ef6-2853-4a84-aaa9-1797e78302a9"}]: dispatch 2024-04-27T04:05:18.923 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:05:18 smithi181 ceph-mon[33774]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "15455ef6-2853-4a84-aaa9-1797e78302a9"}]: dispatch 2024-04-27T04:05:18.924 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:05:18 smithi181 ceph-mon[33774]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "15455ef6-2853-4a84-aaa9-1797e78302a9"}]': finished 2024-04-27T04:05:18.924 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:05:18 smithi181 ceph-mon[33774]: osdmap e36: 6 total, 5 up, 6 in 2024-04-27T04:05:18.924 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:05:18 smithi181 ceph-mon[33774]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-27T04:05:19.138 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:05:18 smithi162 ceph-mon[26841]: from='client.? 172.21.15.181:0/1446013698' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "15455ef6-2853-4a84-aaa9-1797e78302a9"}]: dispatch 2024-04-27T04:05:19.138 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:05:18 smithi162 ceph-mon[26841]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "15455ef6-2853-4a84-aaa9-1797e78302a9"}]: dispatch 2024-04-27T04:05:19.138 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:05:18 smithi162 ceph-mon[26841]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "15455ef6-2853-4a84-aaa9-1797e78302a9"}]': finished 2024-04-27T04:05:19.138 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:05:18 smithi162 ceph-mon[26841]: osdmap e36: 6 total, 5 up, 6 in 2024-04-27T04:05:19.138 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:05:18 smithi162 ceph-mon[26841]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-27T04:05:20.137 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:05:19 smithi162 ceph-mon[26841]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-04-27T04:05:20.138 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:05:19 smithi162 ceph-mon[26841]: from='client.? 172.21.15.181:0/1652693378' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-27T04:05:20.173 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:05:19 smithi181 ceph-mon[33774]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-04-27T04:05:20.173 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:05:19 smithi181 ceph-mon[33774]: from='client.? 172.21.15.181:0/1652693378' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-27T04:05:22.138 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:05:21 smithi162 ceph-mon[26841]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-04-27T04:05:22.173 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:05:21 smithi181 ceph-mon[33774]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-04-27T04:05:23.787 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:05:23 smithi181 ceph-mon[33774]: pgmap v117: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-04-27T04:05:24.138 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:05:23 smithi162 ceph-mon[26841]: pgmap v117: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-04-27T04:05:26.138 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:05:25 smithi162 ceph-mon[26841]: pgmap v118: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-04-27T04:05:26.173 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:05:25 smithi181 ceph-mon[33774]: pgmap v118: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-04-27T04:05:27.138 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:05:26 smithi162 ceph-mon[26841]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-04-27T04:05:27.138 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:05:26 smithi162 ceph-mon[26841]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T04:05:27.138 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:05:26 smithi162 ceph-mon[26841]: Deploying daemon osd.5 on smithi181 2024-04-27T04:05:27.173 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:05:26 smithi181 ceph-mon[33774]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-04-27T04:05:27.173 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:05:26 smithi181 ceph-mon[33774]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T04:05:27.173 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:05:26 smithi181 ceph-mon[33774]: Deploying daemon osd.5 on smithi181 2024-04-27T04:05:27.851 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:05:27 smithi181 ceph-mon[33774]: pgmap v119: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-04-27T04:05:28.040 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:05:27 smithi162 ceph-mon[26841]: pgmap v119: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-04-27T04:05:30.042 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:05:29 smithi181 ceph-mon[33774]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-04-27T04:05:30.137 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:05:29 smithi162 ceph-mon[26841]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-04-27T04:05:32.173 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:05:31 smithi181 ceph-mon[33774]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-04-27T04:05:32.173 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:05:31 smithi181 ceph-mon[33774]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' 2024-04-27T04:05:32.174 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:05:31 smithi181 ceph-mon[33774]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' 2024-04-27T04:05:32.174 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:05:31 smithi181 ceph-mon[33774]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T04:05:32.174 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:05:31 smithi181 ceph-mon[33774]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-27T04:05:32.388 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:05:31 smithi162 ceph-mon[26841]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-04-27T04:05:32.388 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:05:31 smithi162 ceph-mon[26841]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' 2024-04-27T04:05:32.388 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:05:31 smithi162 ceph-mon[26841]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' 2024-04-27T04:05:32.388 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:05:31 smithi162 ceph-mon[26841]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T04:05:32.388 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:05:31 smithi162 ceph-mon[26841]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-27T04:05:34.173 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:05:33 smithi181 ceph-mon[33774]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-04-27T04:05:34.387 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:05:33 smithi162 ceph-mon[26841]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-04-27T04:05:35.024 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:05:35.024+0000 7f54c4ff9700 1 -- 172.21.15.181:0/3175818494 <== mgr.14210 v2:172.21.15.162:6800/4079782599 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f54c8102270 con 0x7f54b40692a0 2024-04-27T04:05:35.025 INFO:teuthology.orchestra.run.smithi181.stdout:Created osd(s) 5 on host 'smithi181' 2024-04-27T04:05:35.026 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:05:35.026+0000 7f54cdaa6700 1 -- 172.21.15.181:0/3175818494 >> [v2:172.21.15.162:6800/4079782599,v1:172.21.15.162:6801/4079782599] conn(0x7f54b40692a0 msgr2=0x7f54b406b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:05:35.026 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:05:35.026+0000 7f54cdaa6700 1 --2- 172.21.15.181:0/3175818494 >> [v2:172.21.15.162:6800/4079782599,v1:172.21.15.162:6801/4079782599] conn(0x7f54b40692a0 0x7f54b406b750 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f54b0006fd0 tx=0x7f54b0008040 comp rx=0 tx=0).stop 2024-04-27T04:05:35.026 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:05:35.026+0000 7f54cdaa6700 1 -- 172.21.15.181:0/3175818494 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f54c80fce60 msgr2=0x7f54c810d930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:05:35.026 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:05:35.026+0000 7f54cdaa6700 1 --2- 172.21.15.181:0/3175818494 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f54c80fce60 0x7f54c810d930 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f54b800b3b0 tx=0x7f54b8009e30 comp rx=0 tx=0).stop 2024-04-27T04:05:35.027 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:05:35.026+0000 7f54cdaa6700 1 -- 172.21.15.181:0/3175818494 shutdown_connections 2024-04-27T04:05:35.027 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:05:35.026+0000 7f54cdaa6700 1 --2- 172.21.15.181:0/3175818494 >> [v2:172.21.15.162:6800/4079782599,v1:172.21.15.162:6801/4079782599] conn(0x7f54b40692a0 0x7f54b406b750 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:05:35.027 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:05:35.026+0000 7f54cdaa6700 1 --2- 172.21.15.181:0/3175818494 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f54c80fce60 0x7f54c810d930 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:05:35.027 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:05:35.026+0000 7f54cdaa6700 1 --2- 172.21.15.181:0/3175818494 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f54c80fdfd0 0x7f54c810de70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:05:35.027 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:05:35.026+0000 7f54cdaa6700 1 -- 172.21.15.181:0/3175818494 >> 172.21.15.181:0/3175818494 conn(0x7f54c80f8540 msgr2=0x7f54c80f9b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:05:35.027 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:05:35.026+0000 7f54cdaa6700 1 -- 172.21.15.181:0/3175818494 shutdown_connections 2024-04-27T04:05:35.027 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:05:35.026+0000 7f54cdaa6700 1 -- 172.21.15.181:0/3175818494 wait complete. 2024-04-27T04:05:35.638 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:05:35 smithi162 ceph-mon[26841]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' 2024-04-27T04:05:35.638 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:05:35 smithi162 ceph-mon[26841]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' 2024-04-27T04:05:35.638 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:05:35 smithi162 ceph-mon[26841]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T04:05:35.638 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:05:35 smithi162 ceph-mon[26841]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T04:05:35.638 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:05:35 smithi162 ceph-mon[26841]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' 2024-04-27T04:05:35.638 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:05:35 smithi162 ceph-mon[26841]: from='osd.5 [v2:172.21.15.181:6816/3193687515,v1:172.21.15.181:6817/3193687515]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-04-27T04:05:35.638 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:05:35 smithi162 ceph-mon[26841]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-04-27T04:05:35.638 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:05:35 smithi162 ceph-mon[26841]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' 2024-04-27T04:05:35.638 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:05:35 smithi162 ceph-mon[26841]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' 2024-04-27T04:05:35.638 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:05:35 smithi162 ceph-mon[26841]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T04:05:35.673 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:05:35 smithi181 ceph-mon[33774]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' 2024-04-27T04:05:35.673 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:05:35 smithi181 ceph-mon[33774]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' 2024-04-27T04:05:35.673 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:05:35 smithi181 ceph-mon[33774]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T04:05:35.673 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:05:35 smithi181 ceph-mon[33774]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T04:05:35.673 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:05:35 smithi181 ceph-mon[33774]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' 2024-04-27T04:05:35.673 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:05:35 smithi181 ceph-mon[33774]: from='osd.5 [v2:172.21.15.181:6816/3193687515,v1:172.21.15.181:6817/3193687515]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-04-27T04:05:35.673 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:05:35 smithi181 ceph-mon[33774]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-04-27T04:05:35.673 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:05:35 smithi181 ceph-mon[33774]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' 2024-04-27T04:05:35.673 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:05:35 smithi181 ceph-mon[33774]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' 2024-04-27T04:05:35.673 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:05:35 smithi181 ceph-mon[33774]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T04:05:35.746 DEBUG:teuthology.orchestra.run.smithi181:osd.5> sudo journalctl -f -n 0 -u ceph-7f9750d6-044a-11ef-bc93-c7b262605968@osd.5.service 2024-04-27T04:05:35.748 INFO:tasks.cephadm:Waiting for 6 OSDs to come up... 2024-04-27T04:05:35.748 DEBUG:teuthology.orchestra.run.smithi162:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 7f9750d6-044a-11ef-bc93-c7b262605968 -- ceph osd stat -f json 2024-04-27T04:05:36.000 INFO:teuthology.orchestra.run.smithi162.stderr:Inferring config /var/lib/ceph/7f9750d6-044a-11ef-bc93-c7b262605968/mon.smithi162/config 2024-04-27T04:05:36.388 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:05:36 smithi162 ceph-mon[26841]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-04-27T04:05:36.388 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:05:36 smithi162 ceph-mon[26841]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-04-27T04:05:36.388 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:05:36 smithi162 ceph-mon[26841]: osdmap e37: 6 total, 5 up, 6 in 2024-04-27T04:05:36.388 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:05:36 smithi162 ceph-mon[26841]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-27T04:05:36.388 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:05:36 smithi162 ceph-mon[26841]: from='osd.5 [v2:172.21.15.181:6816/3193687515,v1:172.21.15.181:6817/3193687515]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi181", "root=default"]}]: dispatch 2024-04-27T04:05:36.388 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:05:36 smithi162 ceph-mon[26841]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi181", "root=default"]}]: dispatch 2024-04-27T04:05:36.592 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:05:36 smithi181 ceph-mon[33774]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-04-27T04:05:36.592 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:05:36 smithi181 ceph-mon[33774]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-04-27T04:05:36.592 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:05:36 smithi181 ceph-mon[33774]: osdmap e37: 6 total, 5 up, 6 in 2024-04-27T04:05:36.592 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:05:36 smithi181 ceph-mon[33774]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-27T04:05:36.593 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:05:36 smithi181 ceph-mon[33774]: from='osd.5 [v2:172.21.15.181:6816/3193687515,v1:172.21.15.181:6817/3193687515]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi181", "root=default"]}]: dispatch 2024-04-27T04:05:36.593 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:05:36 smithi181 ceph-mon[33774]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi181", "root=default"]}]: dispatch 2024-04-27T04:05:36.875 INFO:journalctl@ceph.osd.5.smithi181.stdout:Apr 27 04:05:36 smithi181 ceph-7f9750d6-044a-11ef-bc93-c7b262605968-osd-5[52697]: 2024-04-27T04:05:36.774+0000 7f8e3da7a700 -1 osd.5 0 waiting for initial osdmap 2024-04-27T04:05:36.875 INFO:journalctl@ceph.osd.5.smithi181.stdout:Apr 27 04:05:36 smithi181 ceph-7f9750d6-044a-11ef-bc93-c7b262605968-osd-5[52697]: 2024-04-27T04:05:36.782+0000 7f8e36871700 -1 osd.5 38 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-27T04:05:37.693 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:05:37.693+0000 7fc62d1b9700 1 -- 172.21.15.162:0/2431733574 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc6280fcff0 msgr2=0x7fc6280fd460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:05:37.693 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:05:37.693+0000 7fc62d1b9700 1 --2- 172.21.15.162:0/2431733574 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc6280fcff0 0x7fc6280fd460 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fc61c004940 tx=0x7fc61c009e30 comp rx=0 tx=0).stop 2024-04-27T04:05:37.693 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:05:37.693+0000 7fc62d1b9700 1 -- 172.21.15.162:0/2431733574 shutdown_connections 2024-04-27T04:05:37.693 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:05:37.693+0000 7fc62d1b9700 1 --2- 172.21.15.162:0/2431733574 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc6280fcff0 0x7fc6280fd460 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:05:37.693 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:05:37.693+0000 7fc62d1b9700 1 --2- 172.21.15.162:0/2431733574 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc6280fbd40 0x7fc6280fc150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:05:37.693 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:05:37.693+0000 7fc62d1b9700 1 -- 172.21.15.162:0/2431733574 >> 172.21.15.162:0/2431733574 conn(0x7fc6280f74c0 msgr2=0x7fc6280f98d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:05:37.693 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:05:37.693+0000 7fc62d1b9700 1 -- 172.21.15.162:0/2431733574 shutdown_connections 2024-04-27T04:05:37.693 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:05:37.693+0000 7fc62d1b9700 1 -- 172.21.15.162:0/2431733574 wait complete. 2024-04-27T04:05:37.694 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:05:37.694+0000 7fc62d1b9700 1 Processor -- start 2024-04-27T04:05:37.695 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:05:37.694+0000 7fc62d1b9700 1 -- start start 2024-04-27T04:05:37.695 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:05:37.694+0000 7fc62d1b9700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc6280fbd40 0x7fc628112ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:05:37.695 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:05:37.694+0000 7fc62d1b9700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc6280fcff0 0x7fc628115000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:05:37.695 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:05:37.694+0000 7fc62d1b9700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc628066ab0 con 0x7fc6280fbd40 2024-04-27T04:05:37.695 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:05:37.694+0000 7fc62d1b9700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc628066bf0 con 0x7fc6280fcff0 2024-04-27T04:05:37.695 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:05:37.694+0000 7fc626d9d700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc6280fbd40 0x7fc628112ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:05:37.695 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:05:37.694+0000 7fc626d9d700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc6280fbd40 0x7fc628112ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:40006/0 (socket says 172.21.15.162:40006) 2024-04-27T04:05:37.695 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:05:37.694+0000 7fc626d9d700 1 -- 172.21.15.162:0/827881619 learned_addr learned my addr 172.21.15.162:0/827881619 (peer_addr_for_me v2:172.21.15.162:0/0) 2024-04-27T04:05:37.695 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:05:37.695+0000 7fc62659c700 1 --2- 172.21.15.162:0/827881619 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc6280fcff0 0x7fc628115000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:05:37.695 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:05:37.695+0000 7fc626d9d700 1 -- 172.21.15.162:0/827881619 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc6280fcff0 msgr2=0x7fc628115000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:05:37.695 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:05:37.695+0000 7fc626d9d700 1 --2- 172.21.15.162:0/827881619 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc6280fcff0 0x7fc628115000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:05:37.695 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:05:37.695+0000 7fc626d9d700 1 -- 172.21.15.162:0/827881619 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc61c005040 con 0x7fc6280fbd40 2024-04-27T04:05:37.695 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:05:37.695+0000 7fc626d9d700 1 --2- 172.21.15.162:0/827881619 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc6280fbd40 0x7fc628112ab0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fc61800eb40 tx=0x7fc61800b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:05:37.695 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:05:37.695+0000 7fc60f7fe700 1 -- 172.21.15.162:0/827881619 <== mon.0 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc618007cc0 con 0x7fc6280fbd40 2024-04-27T04:05:37.696 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:05:37.695+0000 7fc60f7fe700 1 -- 172.21.15.162:0/827881619 <== mon.0 v2:172.21.15.162:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fc61800f040 con 0x7fc6280fbd40 2024-04-27T04:05:37.696 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:05:37.695+0000 7fc62d1b9700 1 -- 172.21.15.162:0/827881619 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc628115540 con 0x7fc6280fbd40 2024-04-27T04:05:37.697 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:05:37.695+0000 7fc60f7fe700 1 -- 172.21.15.162:0/827881619 <== mon.0 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc61800ca10 con 0x7fc6280fbd40 2024-04-27T04:05:37.697 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:05:37.695+0000 7fc62d1b9700 1 -- 172.21.15.162:0/827881619 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc628115890 con 0x7fc6280fbd40 2024-04-27T04:05:37.697 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:05:37.696+0000 7fc62d1b9700 1 -- 172.21.15.162:0/827881619 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc62805f5c0 con 0x7fc6280fbd40 2024-04-27T04:05:37.697 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:05:37.697+0000 7fc60f7fe700 1 -- 172.21.15.162:0/827881619 <== mon.0 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 19) v1 ==== 86631+0+0 (secure 0 0 0) 0x7fc61800cc80 con 0x7fc6280fbd40 2024-04-27T04:05:37.697 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:05:37.697+0000 7fc60f7fe700 1 --2- 172.21.15.162:0/827881619 >> [v2:172.21.15.162:6800/4079782599,v1:172.21.15.162:6801/4079782599] conn(0x7fc6100692a0 0x7fc61006b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:05:37.697 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:05:37.697+0000 7fc60f7fe700 1 -- 172.21.15.162:0/827881619 <== mon.0 v2:172.21.15.162:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 4354+0+0 (secure 0 0 0) 0x7fc618086930 con 0x7fc6280fbd40 2024-04-27T04:05:37.697 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:05:37.697+0000 7fc62659c700 1 --2- 172.21.15.162:0/827881619 >> [v2:172.21.15.162:6800/4079782599,v1:172.21.15.162:6801/4079782599] conn(0x7fc6100692a0 0x7fc61006b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:05:37.697 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:05:37.697+0000 7fc62659c700 1 --2- 172.21.15.162:0/827881619 >> [v2:172.21.15.162:6800/4079782599,v1:172.21.15.162:6801/4079782599] conn(0x7fc6100692a0 0x7fc61006b750 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fc61c009b30 tx=0x7fc61c009e30 comp rx=0 tx=0).ready entity=mgr.14210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:05:37.700 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:05:37.701+0000 7fc60f7fe700 1 -- 172.21.15.162:0/827881619 <== mon.0 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7fc61804e250 con 0x7fc6280fbd40 2024-04-27T04:05:37.894 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:05:37.895+0000 7fc62d1b9700 1 -- 172.21.15.162:0/827881619 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) v1 -- 0x7fc62804bed0 con 0x7fc6280fbd40 2024-04-27T04:05:37.894 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:05:37.895+0000 7fc60f7fe700 1 -- 172.21.15.162:0/827881619 <== mon.0 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v39) v1 ==== 74+0+130 (secure 0 0 0) 0x7fc6180530e0 con 0x7fc6280fbd40 2024-04-27T04:05:37.895 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-04-27T04:05:37.897 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:05:37.897+0000 7fc62d1b9700 1 -- 172.21.15.162:0/827881619 >> [v2:172.21.15.162:6800/4079782599,v1:172.21.15.162:6801/4079782599] conn(0x7fc6100692a0 msgr2=0x7fc61006b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:05:37.897 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:05:37.897+0000 7fc62d1b9700 1 --2- 172.21.15.162:0/827881619 >> [v2:172.21.15.162:6800/4079782599,v1:172.21.15.162:6801/4079782599] conn(0x7fc6100692a0 0x7fc61006b750 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fc61c009b30 tx=0x7fc61c009e30 comp rx=0 tx=0).stop 2024-04-27T04:05:37.897 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:05:37.897+0000 7fc62d1b9700 1 -- 172.21.15.162:0/827881619 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc6280fbd40 msgr2=0x7fc628112ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:05:37.897 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:05:37.897+0000 7fc62d1b9700 1 --2- 172.21.15.162:0/827881619 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc6280fbd40 0x7fc628112ab0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fc61800eb40 tx=0x7fc61800b040 comp rx=0 tx=0).stop 2024-04-27T04:05:37.897 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:05:37.897+0000 7fc62d1b9700 1 -- 172.21.15.162:0/827881619 shutdown_connections 2024-04-27T04:05:37.897 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:05:37.897+0000 7fc62d1b9700 1 --2- 172.21.15.162:0/827881619 >> [v2:172.21.15.162:6800/4079782599,v1:172.21.15.162:6801/4079782599] conn(0x7fc6100692a0 0x7fc61006b750 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:05:37.897 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:05:37.897+0000 7fc62d1b9700 1 --2- 172.21.15.162:0/827881619 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc6280fbd40 0x7fc628112ab0 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:05:37.897 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:05:37.897+0000 7fc62d1b9700 1 --2- 172.21.15.162:0/827881619 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc6280fcff0 0x7fc628115000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:05:37.897 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:05:37.897+0000 7fc62d1b9700 1 -- 172.21.15.162:0/827881619 >> 172.21.15.162:0/827881619 conn(0x7fc6280f74c0 msgr2=0x7fc6280f8af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:05:37.897 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:05:37.897+0000 7fc62d1b9700 1 -- 172.21.15.162:0/827881619 shutdown_connections 2024-04-27T04:05:37.897 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:05:37.898+0000 7fc62d1b9700 1 -- 172.21.15.162:0/827881619 wait complete. 2024-04-27T04:05:38.138 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:05:37 smithi162 ceph-mon[26841]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-04-27T04:05:38.138 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:05:37 smithi162 ceph-mon[26841]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi181", "root=default"]}]': finished 2024-04-27T04:05:38.138 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:05:37 smithi162 ceph-mon[26841]: osdmap e38: 6 total, 5 up, 6 in 2024-04-27T04:05:38.138 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:05:37 smithi162 ceph-mon[26841]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-27T04:05:38.138 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:05:37 smithi162 ceph-mon[26841]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-27T04:05:38.173 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:05:37 smithi181 ceph-mon[33774]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-04-27T04:05:38.173 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:05:37 smithi181 ceph-mon[33774]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi181", "root=default"]}]': finished 2024-04-27T04:05:38.173 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:05:37 smithi181 ceph-mon[33774]: osdmap e38: 6 total, 5 up, 6 in 2024-04-27T04:05:38.173 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:05:37 smithi181 ceph-mon[33774]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-27T04:05:38.173 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:05:37 smithi181 ceph-mon[33774]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-27T04:05:38.536 INFO:teuthology.orchestra.run.smithi162.stdout:{"epoch":39,"num_osds":6,"num_up_osds":6,"osd_up_since":1714190737,"num_in_osds":6,"osd_in_since":1714190718,"num_remapped_pgs":0} 2024-04-27T04:05:38.537 DEBUG:teuthology.orchestra.run.smithi162:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell --fsid 7f9750d6-044a-11ef-bc93-c7b262605968 -- ceph osd dump --format=json 2024-04-27T04:05:38.793 INFO:teuthology.orchestra.run.smithi162.stderr:Inferring config /var/lib/ceph/7f9750d6-044a-11ef-bc93-c7b262605968/mon.smithi162/config 2024-04-27T04:05:39.099 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:05:38 smithi162 ceph-mon[26841]: purged_snaps scrub starts 2024-04-27T04:05:39.099 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:05:38 smithi162 ceph-mon[26841]: purged_snaps scrub ok 2024-04-27T04:05:39.099 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:05:38 smithi162 ceph-mon[26841]: osd.5 [v2:172.21.15.181:6816/3193687515,v1:172.21.15.181:6817/3193687515] boot 2024-04-27T04:05:39.099 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:05:38 smithi162 ceph-mon[26841]: osdmap e39: 6 total, 6 up, 6 in 2024-04-27T04:05:39.099 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:05:38 smithi162 ceph-mon[26841]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-27T04:05:39.099 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:05:38 smithi162 ceph-mon[26841]: from='client.? 172.21.15.162:0/827881619' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-04-27T04:05:39.099 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:05:38 smithi162 ceph-mon[26841]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' 2024-04-27T04:05:39.099 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:05:38 smithi162 ceph-mon[26841]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' 2024-04-27T04:05:39.099 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:05:38 smithi162 ceph-mon[26841]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-27T04:05:39.099 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:05:38 smithi162 ceph-mon[26841]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-27T04:05:39.099 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:05:38 smithi162 ceph-mon[26841]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-27T04:05:39.100 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:05:38 smithi162 ceph-mon[26841]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' 2024-04-27T04:05:39.100 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:05:38 smithi162 ceph-mon[26841]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T04:05:39.100 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:05:38 smithi162 ceph-mon[26841]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T04:05:39.100 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:05:38 smithi162 ceph-mon[26841]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' 2024-04-27T04:05:39.173 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:05:38 smithi181 ceph-mon[33774]: purged_snaps scrub starts 2024-04-27T04:05:39.173 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:05:38 smithi181 ceph-mon[33774]: purged_snaps scrub ok 2024-04-27T04:05:39.173 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:05:38 smithi181 ceph-mon[33774]: osd.5 [v2:172.21.15.181:6816/3193687515,v1:172.21.15.181:6817/3193687515] boot 2024-04-27T04:05:39.173 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:05:38 smithi181 ceph-mon[33774]: osdmap e39: 6 total, 6 up, 6 in 2024-04-27T04:05:39.173 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:05:38 smithi181 ceph-mon[33774]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-27T04:05:39.174 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:05:38 smithi181 ceph-mon[33774]: from='client.? 172.21.15.162:0/827881619' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-04-27T04:05:39.174 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:05:38 smithi181 ceph-mon[33774]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' 2024-04-27T04:05:39.174 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:05:38 smithi181 ceph-mon[33774]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' 2024-04-27T04:05:39.174 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:05:38 smithi181 ceph-mon[33774]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-27T04:05:39.174 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:05:38 smithi181 ceph-mon[33774]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-27T04:05:39.174 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:05:38 smithi181 ceph-mon[33774]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-27T04:05:39.174 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:05:38 smithi181 ceph-mon[33774]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' 2024-04-27T04:05:39.174 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:05:38 smithi181 ceph-mon[33774]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T04:05:39.174 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:05:38 smithi181 ceph-mon[33774]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T04:05:39.174 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:05:38 smithi181 ceph-mon[33774]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' 2024-04-27T04:05:39.887 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:05:39 smithi162 ceph-mon[26841]: Detected new or changed devices on smithi181 2024-04-27T04:05:39.888 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:05:39 smithi162 ceph-mon[26841]: Adjusting osd_memory_target on smithi181 to 5312M 2024-04-27T04:05:39.888 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:05:39 smithi162 ceph-mon[26841]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-27T04:05:39.888 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:05:39 smithi162 ceph-mon[26841]: osdmap e40: 6 total, 6 up, 6 in 2024-04-27T04:05:40.173 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:05:39 smithi181 ceph-mon[33774]: Detected new or changed devices on smithi181 2024-04-27T04:05:40.173 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:05:39 smithi181 ceph-mon[33774]: Adjusting osd_memory_target on smithi181 to 5312M 2024-04-27T04:05:40.173 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:05:39 smithi181 ceph-mon[33774]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-27T04:05:40.174 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:05:39 smithi181 ceph-mon[33774]: osdmap e40: 6 total, 6 up, 6 in 2024-04-27T04:05:40.356 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:05:40.346+0000 7f5ed8060700 1 -- 172.21.15.162:0/3267315991 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5ed0102de0 msgr2=0x7f5ed01031b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:05:40.356 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:05:40.346+0000 7f5ed8060700 1 --2- 172.21.15.162:0/3267315991 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5ed0102de0 0x7f5ed01031b0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f5ecc004940 tx=0x7f5ecc009e30 comp rx=0 tx=0).stop 2024-04-27T04:05:40.356 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:05:40.346+0000 7f5ed8060700 1 -- 172.21.15.162:0/3267315991 shutdown_connections 2024-04-27T04:05:40.356 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:05:40.346+0000 7f5ed8060700 1 --2- 172.21.15.162:0/3267315991 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5ed0103780 0x7f5ed00fcdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:05:40.356 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:05:40.346+0000 7f5ed8060700 1 --2- 172.21.15.162:0/3267315991 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5ed0102de0 0x7f5ed01031b0 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:05:40.356 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:05:40.346+0000 7f5ed8060700 1 -- 172.21.15.162:0/3267315991 >> 172.21.15.162:0/3267315991 conn(0x7f5ed00f8990 msgr2=0x7f5ed00fada0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:05:40.357 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:05:40.346+0000 7f5ed8060700 1 -- 172.21.15.162:0/3267315991 shutdown_connections 2024-04-27T04:05:40.357 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:05:40.346+0000 7f5ed8060700 1 -- 172.21.15.162:0/3267315991 wait complete. 2024-04-27T04:05:40.357 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:05:40.346+0000 7f5ed8060700 1 Processor -- start 2024-04-27T04:05:40.357 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:05:40.346+0000 7f5ed8060700 1 -- start start 2024-04-27T04:05:40.357 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:05:40.347+0000 7f5ed8060700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5ed0102de0 0x7f5ed0070260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:05:40.357 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:05:40.347+0000 7f5ed8060700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5ed0103780 0x7f5ed00707a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:05:40.357 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:05:40.347+0000 7f5ed8060700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ed0072160 con 0x7f5ed0103780 2024-04-27T04:05:40.357 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:05:40.347+0000 7f5ed8060700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ed0070ce0 con 0x7f5ed0102de0 2024-04-27T04:05:40.357 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:05:40.347+0000 7f5ed55fb700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5ed0103780 0x7f5ed00707a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:05:40.357 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:05:40.347+0000 7f5ed55fb700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5ed0103780 0x7f5ed00707a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:40024/0 (socket says 172.21.15.162:40024) 2024-04-27T04:05:40.357 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:05:40.347+0000 7f5ed55fb700 1 -- 172.21.15.162:0/2201762195 learned_addr learned my addr 172.21.15.162:0/2201762195 (peer_addr_for_me v2:172.21.15.162:0/0) 2024-04-27T04:05:40.357 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:05:40.347+0000 7f5ed5dfc700 1 --2- 172.21.15.162:0/2201762195 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5ed0102de0 0x7f5ed0070260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:05:40.357 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:05:40.347+0000 7f5ed55fb700 1 -- 172.21.15.162:0/2201762195 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5ed0102de0 msgr2=0x7f5ed0070260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:05:40.358 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:05:40.347+0000 7f5ed55fb700 1 --2- 172.21.15.162:0/2201762195 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5ed0102de0 0x7f5ed0070260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:05:40.358 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:05:40.347+0000 7f5ed55fb700 1 -- 172.21.15.162:0/2201762195 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ecc005040 con 0x7f5ed0103780 2024-04-27T04:05:40.358 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:05:40.347+0000 7f5ed55fb700 1 --2- 172.21.15.162:0/2201762195 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5ed0103780 0x7f5ed00707a0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f5ec000eb40 tx=0x7f5ec000b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:05:40.358 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:05:40.347+0000 7f5ec67fc700 1 -- 172.21.15.162:0/2201762195 <== mon.0 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5ec0007cc0 con 0x7f5ed0103780 2024-04-27T04:05:40.358 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:05:40.347+0000 7f5ec67fc700 1 -- 172.21.15.162:0/2201762195 <== mon.0 v2:172.21.15.162:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f5ec000f040 con 0x7f5ed0103780 2024-04-27T04:05:40.358 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:05:40.347+0000 7f5ed8060700 1 -- 172.21.15.162:0/2201762195 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5ed0070f60 con 0x7f5ed0103780 2024-04-27T04:05:40.358 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:05:40.347+0000 7f5ec67fc700 1 -- 172.21.15.162:0/2201762195 <== mon.0 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5ec00095e0 con 0x7f5ed0103780 2024-04-27T04:05:40.358 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:05:40.347+0000 7f5ed8060700 1 -- 172.21.15.162:0/2201762195 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5ed0119ff0 con 0x7f5ed0103780 2024-04-27T04:05:40.358 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:05:40.348+0000 7f5ed8060700 1 -- 172.21.15.162:0/2201762195 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5ed01000f0 con 0x7f5ed0103780 2024-04-27T04:05:40.358 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:05:40.349+0000 7f5ec67fc700 1 -- 172.21.15.162:0/2201762195 <== mon.0 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 19) v1 ==== 86631+0+0 (secure 0 0 0) 0x7f5ec0009740 con 0x7f5ed0103780 2024-04-27T04:05:40.358 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:05:40.349+0000 7f5ec67fc700 1 --2- 172.21.15.162:0/2201762195 >> [v2:172.21.15.162:6800/4079782599,v1:172.21.15.162:6801/4079782599] conn(0x7f5ebc0692a0 0x7f5ebc06b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:05:40.358 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:05:40.349+0000 7f5ec67fc700 1 -- 172.21.15.162:0/2201762195 <== mon.0 v2:172.21.15.162:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4634+0+0 (secure 0 0 0) 0x7f5ec0086950 con 0x7f5ed0103780 2024-04-27T04:05:40.358 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:05:40.349+0000 7f5ed5dfc700 1 --2- 172.21.15.162:0/2201762195 >> [v2:172.21.15.162:6800/4079782599,v1:172.21.15.162:6801/4079782599] conn(0x7f5ebc0692a0 0x7f5ebc06b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:05:40.358 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:05:40.349+0000 7f5ed5dfc700 1 --2- 172.21.15.162:0/2201762195 >> [v2:172.21.15.162:6800/4079782599,v1:172.21.15.162:6801/4079782599] conn(0x7f5ebc0692a0 0x7f5ebc06b750 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f5ecc007f50 tx=0x7f5ecc009e30 comp rx=0 tx=0).ready entity=mgr.14210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:05:40.359 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:05:40.353+0000 7f5ec67fc700 1 -- 172.21.15.162:0/2201762195 <== mon.0 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f5ec004e250 con 0x7f5ed0103780 2024-04-27T04:05:40.537 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:05:40.538+0000 7f5ed8060700 1 -- 172.21.15.162:0/2201762195 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f5ed0109ee0 con 0x7f5ed0103780 2024-04-27T04:05:40.538 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:05:40.539+0000 7f5ec67fc700 1 -- 172.21.15.162:0/2201762195 <== mon.0 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v40) v1 ==== 74+0+11141 (secure 0 0 0) 0x7f5ec0052ff0 con 0x7f5ed0103780 2024-04-27T04:05:40.538 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-04-27T04:05:40.538 INFO:teuthology.orchestra.run.smithi162.stdout:{"epoch":40,"fsid":"7f9750d6-044a-11ef-bc93-c7b262605968","created":"2024-04-27T04:00:06.343616+0000","modified":"2024-04-27T04:05:38.774913+0000","last_up_change":"2024-04-27T04:05:37.769065+0000","last_in_change":"2024-04-27T04:05:18.597695+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":14,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":6,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-27T04:04:24.971779+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}}],"osds":[{"osd":0,"uuid":"0ae69d7f-0ab4-4672-89b9-3563a10a1bf2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6802","nonce":3784722138},{"type":"v1","addr":"172.21.15.162:6803","nonce":3784722138}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6804","nonce":3784722138},{"type":"v1","addr":"172.21.15.162:6805","nonce":3784722138}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6808","nonce":3784722138},{"type":"v1","addr":"172.21.15.162:6809","nonce":3784722138}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6806","nonce":3784722138},{"type":"v1","addr":"172.21.15.162:6807","nonce":3784722138}]},"public_addr":"172.21.15.162:6803/3784722138","cluster_addr":"172.21.15.162:6805/3784722138","heartbeat_back_addr":"172.21.15.162:6809/3784722138","heartbeat_front_addr":"172.21.15.162:6807/3784722138","state":["exists","up"]},{"osd":1,"uuid":"95bdc9f0-c438-4a88-942d-dffb0c61c308","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6810","nonce":3642637124},{"type":"v1","addr":"172.21.15.162:6811","nonce":3642637124}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6812","nonce":3642637124},{"type":"v1","addr":"172.21.15.162:6813","nonce":3642637124}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6816","nonce":3642637124},{"type":"v1","addr":"172.21.15.162:6817","nonce":3642637124}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6814","nonce":3642637124},{"type":"v1","addr":"172.21.15.162:6815","nonce":3642637124}]},"public_addr":"172.21.15.162:6811/3642637124","cluster_addr":"172.21.15.162:6813/3642637124","heartbeat_back_addr":"172.21.15.162:6817/3642637124","heartbeat_front_addr":"172.21.15.162:6815/3642637124","state":["exists","up"]},{"osd":2,"uuid":"a773b386-e75b-432a-9d1b-2206301b96c3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":19,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6818","nonce":3830961923},{"type":"v1","addr":"172.21.15.162:6819","nonce":3830961923}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6820","nonce":3830961923},{"type":"v1","addr":"172.21.15.162:6821","nonce":3830961923}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6824","nonce":3830961923},{"type":"v1","addr":"172.21.15.162:6825","nonce":3830961923}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6822","nonce":3830961923},{"type":"v1","addr":"172.21.15.162:6823","nonce":3830961923}]},"public_addr":"172.21.15.162:6819/3830961923","cluster_addr":"172.21.15.162:6821/3830961923","heartbeat_back_addr":"172.21.15.162:6825/3830961923","heartbeat_front_addr":"172.21.15.162:6823/3830961923","state":["exists","up"]},{"osd":3,"uuid":"756091bc-1594-479d-9390-cb60e6b65ebe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":3695382003},{"type":"v1","addr":"172.21.15.181:6801","nonce":3695382003}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":3695382003},{"type":"v1","addr":"172.21.15.181:6803","nonce":3695382003}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6806","nonce":3695382003},{"type":"v1","addr":"172.21.15.181:6807","nonce":3695382003}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":3695382003},{"type":"v1","addr":"172.21.15.181:6805","nonce":3695382003}]},"public_addr":"172.21.15.181:6801/3695382003","cluster_addr":"172.21.15.181:6803/3695382003","heartbeat_back_addr":"172.21.15.181:6807/3695382003","heartbeat_front_addr":"172.21.15.181:6805/3695382003","state":["exists","up"]},{"osd":4,"uuid":"bd70494d-061c-4468-b837-26150c2d4208","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6808","nonce":2791715456},{"type":"v1","addr":"172.21.15.181:6809","nonce":2791715456}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":2791715456},{"type":"v1","addr":"172.21.15.181:6811","nonce":2791715456}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":2791715456},{"type":"v1","addr":"172.21.15.181:6815","nonce":2791715456}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6812","nonce":2791715456},{"type":"v1","addr":"172.21.15.181:6813","nonce":2791715456}]},"public_addr":"172.21.15.181:6809/2791715456","cluster_addr":"172.21.15.181:6811/2791715456","heartbeat_back_addr":"172.21.15.181:6815/2791715456","heartbeat_front_addr":"172.21.15.181:6813/2791715456","state":["exists","up"]},{"osd":5,"uuid":"15455ef6-2853-4a84-aaa9-1797e78302a9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":3193687515},{"type":"v1","addr":"172.21.15.181:6817","nonce":3193687515}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6818","nonce":3193687515},{"type":"v1","addr":"172.21.15.181:6819","nonce":3193687515}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6822","nonce":3193687515},{"type":"v1","addr":"172.21.15.181:6823","nonce":3193687515}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":3193687515},{"type":"v1","addr":"172.21.15.181:6821","nonce":3193687515}]},"public_addr":"172.21.15.181:6817/3193687515","cluster_addr":"172.21.15.181:6819/3193687515","heartbeat_back_addr":"172.21.15.181:6823/3193687515","heartbeat_front_addr":"172.21.15.181:6821/3193687515","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-04-27T04:03:28.522704+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-04-27T04:03:54.648083+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-04-27T04:04:22.107316+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-04-27T04:04:46.020733+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-04-27T04:05:10.634082+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-04-27T04:05:35.756217+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.162:0/795779642":"2024-04-28T04:02:16.661212+0000","172.21.15.162:0/2336042321":"2024-04-28T04:02:16.661212+0000","172.21.15.162:0/3590827229":"2024-04-28T04:01:10.996874+0000","172.21.15.162:6801/1482851081":"2024-04-28T04:02:16.661212+0000","172.21.15.162:6801/8067198":"2024-04-28T04:01:10.996874+0000","172.21.15.162:0/2623011446":"2024-04-28T04:00:40.642043+0000","172.21.15.162:6800/1482851081":"2024-04-28T04:02:16.661212+0000","172.21.15.162:6801/1480340393":"2024-04-28T04:00:40.642043+0000","172.21.15.162:0/3073351439":"2024-04-28T04:01:10.996874+0000","172.21.15.162:0/2129785584":"2024-04-28T04:01:10.996874+0000","172.21.15.162:0/4025365522":"2024-04-28T04:00:40.642043+0000","172.21.15.162:6800/1480340393":"2024-04-28T04:00:40.642043+0000","172.21.15.162:0/1703886136":"2024-04-28T04:02:16.661212+0000","172.21.15.162:6800/8067198":"2024-04-28T04:01:10.996874+0000","172.21.15.162:0/2579239891":"2024-04-28T04:00:40.642043+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-27T04:05:40.541 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:05:40.541+0000 7f5ed8060700 1 -- 172.21.15.162:0/2201762195 >> [v2:172.21.15.162:6800/4079782599,v1:172.21.15.162:6801/4079782599] conn(0x7f5ebc0692a0 msgr2=0x7f5ebc06b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:05:40.541 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:05:40.541+0000 7f5ed8060700 1 --2- 172.21.15.162:0/2201762195 >> [v2:172.21.15.162:6800/4079782599,v1:172.21.15.162:6801/4079782599] conn(0x7f5ebc0692a0 0x7f5ebc06b750 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f5ecc007f50 tx=0x7f5ecc009e30 comp rx=0 tx=0).stop 2024-04-27T04:05:40.541 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:05:40.541+0000 7f5ed8060700 1 -- 172.21.15.162:0/2201762195 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5ed0103780 msgr2=0x7f5ed00707a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:05:40.541 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:05:40.541+0000 7f5ed8060700 1 --2- 172.21.15.162:0/2201762195 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5ed0103780 0x7f5ed00707a0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f5ec000eb40 tx=0x7f5ec000b040 comp rx=0 tx=0).stop 2024-04-27T04:05:40.541 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:05:40.541+0000 7f5ed8060700 1 -- 172.21.15.162:0/2201762195 shutdown_connections 2024-04-27T04:05:40.541 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:05:40.541+0000 7f5ed8060700 1 --2- 172.21.15.162:0/2201762195 >> [v2:172.21.15.162:6800/4079782599,v1:172.21.15.162:6801/4079782599] conn(0x7f5ebc0692a0 0x7f5ebc06b750 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:05:40.541 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:05:40.541+0000 7f5ed8060700 1 --2- 172.21.15.162:0/2201762195 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5ed0102de0 0x7f5ed0070260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:05:40.541 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:05:40.541+0000 7f5ed8060700 1 --2- 172.21.15.162:0/2201762195 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5ed0103780 0x7f5ed00707a0 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:05:40.541 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:05:40.541+0000 7f5ed8060700 1 -- 172.21.15.162:0/2201762195 >> 172.21.15.162:0/2201762195 conn(0x7f5ed00f8990 msgr2=0x7f5ed00fa6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:05:40.541 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:05:40.541+0000 7f5ed8060700 1 -- 172.21.15.162:0/2201762195 shutdown_connections 2024-04-27T04:05:40.541 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:05:40.542+0000 7f5ed8060700 1 -- 172.21.15.162:0/2201762195 wait complete. 2024-04-27T04:05:41.101 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:05:40 smithi162 ceph-mon[26841]: from='client.? 172.21.15.162:0/2201762195' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-27T04:05:41.127 INFO:tasks.cephadm.ceph_manager.ceph:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-27T04:04:24.971779+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}] 2024-04-27T04:05:41.127 DEBUG:teuthology.orchestra.run.smithi162:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell --fsid 7f9750d6-044a-11ef-bc93-c7b262605968 -- ceph osd pool get .mgr pg_num 2024-04-27T04:05:41.173 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:05:40 smithi181 ceph-mon[33774]: from='client.? 172.21.15.162:0/2201762195' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-27T04:05:41.379 INFO:teuthology.orchestra.run.smithi162.stderr:Inferring config /var/lib/ceph/7f9750d6-044a-11ef-bc93-c7b262605968/mon.smithi162/config 2024-04-27T04:05:42.138 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:05:41 smithi162 ceph-mon[26841]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-27T04:05:42.173 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:05:41 smithi181 ceph-mon[33774]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-27T04:05:43.023 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:05:43.021+0000 7f3d6816c700 1 -- 172.21.15.162:0/3836904503 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3d60103780 msgr2=0x7f3d600fcdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:05:43.023 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:05:43.021+0000 7f3d6816c700 1 --2- 172.21.15.162:0/3836904503 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3d60103780 0x7f3d600fcdf0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f3d5c004940 tx=0x7f3d5c009e30 comp rx=0 tx=0).stop 2024-04-27T04:05:43.023 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:05:43.021+0000 7f3d6816c700 1 -- 172.21.15.162:0/3836904503 shutdown_connections 2024-04-27T04:05:43.023 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:05:43.021+0000 7f3d6816c700 1 --2- 172.21.15.162:0/3836904503 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3d60103780 0x7f3d600fcdf0 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:05:43.023 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:05:43.021+0000 7f3d6816c700 1 --2- 172.21.15.162:0/3836904503 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3d60102de0 0x7f3d601031b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:05:43.024 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:05:43.021+0000 7f3d6816c700 1 -- 172.21.15.162:0/3836904503 >> 172.21.15.162:0/3836904503 conn(0x7f3d600f8990 msgr2=0x7f3d600fada0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:05:43.024 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:05:43.022+0000 7f3d6816c700 1 -- 172.21.15.162:0/3836904503 shutdown_connections 2024-04-27T04:05:43.024 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:05:43.022+0000 7f3d6816c700 1 -- 172.21.15.162:0/3836904503 wait complete. 2024-04-27T04:05:43.024 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:05:43.022+0000 7f3d6816c700 1 Processor -- start 2024-04-27T04:05:43.024 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:05:43.022+0000 7f3d6816c700 1 -- start start 2024-04-27T04:05:43.024 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:05:43.022+0000 7f3d6816c700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3d60102de0 0x7f3d60113990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:05:43.024 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:05:43.022+0000 7f3d6816c700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3d60103780 0x7f3d60113ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:05:43.024 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:05:43.022+0000 7f3d6816c700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d60114520 con 0x7f3d60102de0 2024-04-27T04:05:43.024 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:05:43.022+0000 7f3d6816c700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d6010db20 con 0x7f3d60103780 2024-04-27T04:05:43.024 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:05:43.022+0000 7f3d65f08700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3d60102de0 0x7f3d60113990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:05:43.024 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:05:43.022+0000 7f3d65f08700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3d60102de0 0x7f3d60113990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:40040/0 (socket says 172.21.15.162:40040) 2024-04-27T04:05:43.024 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:05:43.022+0000 7f3d65f08700 1 -- 172.21.15.162:0/4196339530 learned_addr learned my addr 172.21.15.162:0/4196339530 (peer_addr_for_me v2:172.21.15.162:0/0) 2024-04-27T04:05:43.024 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:05:43.022+0000 7f3d65707700 1 --2- 172.21.15.162:0/4196339530 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3d60103780 0x7f3d60113ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:05:43.024 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:05:43.023+0000 7f3d65f08700 1 -- 172.21.15.162:0/4196339530 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3d60103780 msgr2=0x7f3d60113ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:05:43.025 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:05:43.023+0000 7f3d65f08700 1 --2- 172.21.15.162:0/4196339530 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3d60103780 0x7f3d60113ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:05:43.025 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:05:43.023+0000 7f3d65f08700 1 -- 172.21.15.162:0/4196339530 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3d5c005040 con 0x7f3d60102de0 2024-04-27T04:05:43.025 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:05:43.023+0000 7f3d65707700 1 --2- 172.21.15.162:0/4196339530 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3d60103780 0x7f3d60113ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-27T04:05:43.025 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:05:43.023+0000 7f3d65f08700 1 --2- 172.21.15.162:0/4196339530 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3d60102de0 0x7f3d60113990 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f3d50002a50 tx=0x7f3d500046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:05:43.025 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:05:43.023+0000 7f3d567fc700 1 -- 172.21.15.162:0/4196339530 <== mon.0 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3d50007cc0 con 0x7f3d60102de0 2024-04-27T04:05:43.025 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:05:43.023+0000 7f3d567fc700 1 -- 172.21.15.162:0/4196339530 <== mon.0 v2:172.21.15.162:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f3d5000f040 con 0x7f3d60102de0 2024-04-27T04:05:43.025 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:05:43.023+0000 7f3d567fc700 1 -- 172.21.15.162:0/4196339530 <== mon.0 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3d50009640 con 0x7f3d60102de0 2024-04-27T04:05:43.025 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:05:43.023+0000 7f3d6816c700 1 -- 172.21.15.162:0/4196339530 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3d6010de00 con 0x7f3d60102de0 2024-04-27T04:05:43.025 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:05:43.023+0000 7f3d6816c700 1 -- 172.21.15.162:0/4196339530 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3d6010e3c0 con 0x7f3d60102de0 2024-04-27T04:05:43.026 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:05:43.024+0000 7f3d6816c700 1 -- 172.21.15.162:0/4196339530 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3d600fe530 con 0x7f3d60102de0 2024-04-27T04:05:43.026 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:05:43.024+0000 7f3d567fc700 1 -- 172.21.15.162:0/4196339530 <== mon.0 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 19) v1 ==== 86631+0+0 (secure 0 0 0) 0x7f3d500097a0 con 0x7f3d60102de0 2024-04-27T04:05:43.026 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:05:43.025+0000 7f3d567fc700 1 --2- 172.21.15.162:0/4196339530 >> [v2:172.21.15.162:6800/4079782599,v1:172.21.15.162:6801/4079782599] conn(0x7f3d4c0692a0 0x7f3d4c06b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:05:43.026 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:05:43.025+0000 7f3d65707700 1 --2- 172.21.15.162:0/4196339530 >> [v2:172.21.15.162:6800/4079782599,v1:172.21.15.162:6801/4079782599] conn(0x7f3d4c0692a0 0x7f3d4c06b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:05:43.026 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:05:43.025+0000 7f3d567fc700 1 -- 172.21.15.162:0/4196339530 <== mon.0 v2:172.21.15.162:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4634+0+0 (secure 0 0 0) 0x7f3d50086b80 con 0x7f3d60102de0 2024-04-27T04:05:43.026 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:05:43.025+0000 7f3d65707700 1 --2- 172.21.15.162:0/4196339530 >> [v2:172.21.15.162:6800/4079782599,v1:172.21.15.162:6801/4079782599] conn(0x7f3d4c0692a0 0x7f3d4c06b750 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f3d6010ece0 tx=0x7f3d5c009e30 comp rx=0 tx=0).ready entity=mgr.14210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:05:43.028 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:05:43.029+0000 7f3d567fc700 1 -- 172.21.15.162:0/4196339530 <== mon.0 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f3d50056c80 con 0x7f3d60102de0 2024-04-27T04:05:43.213 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:05:43.213+0000 7f3d6816c700 1 -- 172.21.15.162:0/4196339530 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f3d60109ec0 con 0x7f3d60102de0 2024-04-27T04:05:43.214 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:05:43.214+0000 7f3d567fc700 1 -- 172.21.15.162:0/4196339530 <== mon.0 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v40) v1 ==== 93+0+10 (secure 0 0 0) 0x7f3d50056810 con 0x7f3d60102de0 2024-04-27T04:05:43.214 INFO:teuthology.orchestra.run.smithi162.stdout:pg_num: 1 2024-04-27T04:05:43.215 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:05:43.216+0000 7f3d6816c700 1 -- 172.21.15.162:0/4196339530 >> [v2:172.21.15.162:6800/4079782599,v1:172.21.15.162:6801/4079782599] conn(0x7f3d4c0692a0 msgr2=0x7f3d4c06b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:05:43.216 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:05:43.216+0000 7f3d6816c700 1 --2- 172.21.15.162:0/4196339530 >> [v2:172.21.15.162:6800/4079782599,v1:172.21.15.162:6801/4079782599] conn(0x7f3d4c0692a0 0x7f3d4c06b750 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f3d6010ece0 tx=0x7f3d5c009e30 comp rx=0 tx=0).stop 2024-04-27T04:05:43.216 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:05:43.216+0000 7f3d6816c700 1 -- 172.21.15.162:0/4196339530 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3d60102de0 msgr2=0x7f3d60113990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:05:43.216 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:05:43.216+0000 7f3d6816c700 1 --2- 172.21.15.162:0/4196339530 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3d60102de0 0x7f3d60113990 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f3d50002a50 tx=0x7f3d500046b0 comp rx=0 tx=0).stop 2024-04-27T04:05:43.216 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:05:43.216+0000 7f3d6816c700 1 -- 172.21.15.162:0/4196339530 shutdown_connections 2024-04-27T04:05:43.216 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:05:43.216+0000 7f3d6816c700 1 --2- 172.21.15.162:0/4196339530 >> [v2:172.21.15.162:6800/4079782599,v1:172.21.15.162:6801/4079782599] conn(0x7f3d4c0692a0 0x7f3d4c06b750 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:05:43.216 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:05:43.216+0000 7f3d6816c700 1 --2- 172.21.15.162:0/4196339530 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3d60102de0 0x7f3d60113990 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:05:43.216 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:05:43.216+0000 7f3d6816c700 1 --2- 172.21.15.162:0/4196339530 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3d60103780 0x7f3d60113ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:05:43.216 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:05:43.216+0000 7f3d6816c700 1 -- 172.21.15.162:0/4196339530 >> 172.21.15.162:0/4196339530 conn(0x7f3d600f8990 msgr2=0x7f3d600f9970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:05:43.216 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:05:43.216+0000 7f3d6816c700 1 -- 172.21.15.162:0/4196339530 shutdown_connections 2024-04-27T04:05:43.217 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:05:43.216+0000 7f3d6816c700 1 -- 172.21.15.162:0/4196339530 wait complete. 2024-04-27T04:05:43.817 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:05:43 smithi162 ceph-mon[26841]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-27T04:05:43.817 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:05:43 smithi162 ceph-mon[26841]: from='client.? 172.21.15.162:0/4196339530' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-27T04:05:43.841 INFO:tasks.cephadm:Setting up client nodes... 2024-04-27T04:05:43.841 DEBUG:teuthology.orchestra.run.smithi162:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 7f9750d6-044a-11ef-bc93-c7b262605968 -- ceph auth get-or-create client.0 mon 'allow *' osd 'allow *' mds 'allow *' mgr 'allow *' 2024-04-27T04:05:44.091 INFO:teuthology.orchestra.run.smithi162.stderr:Inferring config /var/lib/ceph/7f9750d6-044a-11ef-bc93-c7b262605968/mon.smithi162/config 2024-04-27T04:05:44.173 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:05:43 smithi181 ceph-mon[33774]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-27T04:05:44.173 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:05:43 smithi181 ceph-mon[33774]: from='client.? 172.21.15.162:0/4196339530' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-27T04:05:45.678 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:05:45.677+0000 7f5090bca700 1 -- 172.21.15.162:0/4178714903 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f508c0fbce0 msgr2=0x7f508c0fc0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:05:45.678 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:05:45.677+0000 7f5090bca700 1 --2- 172.21.15.162:0/4178714903 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f508c0fbce0 0x7f508c0fc0f0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f507c004940 tx=0x7f507c009e30 comp rx=0 tx=0).stop 2024-04-27T04:05:45.678 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:05:45.677+0000 7f5090bca700 1 -- 172.21.15.162:0/4178714903 shutdown_connections 2024-04-27T04:05:45.678 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:05:45.677+0000 7f5090bca700 1 --2- 172.21.15.162:0/4178714903 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f508c0fcf90 0x7f508c0fd400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:05:45.678 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:05:45.677+0000 7f5090bca700 1 --2- 172.21.15.162:0/4178714903 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f508c0fbce0 0x7f508c0fc0f0 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:05:45.678 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:05:45.677+0000 7f5090bca700 1 -- 172.21.15.162:0/4178714903 >> 172.21.15.162:0/4178714903 conn(0x7f508c0f7440 msgr2=0x7f508c0f9870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:05:45.678 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:05:45.677+0000 7f5090bca700 1 -- 172.21.15.162:0/4178714903 shutdown_connections 2024-04-27T04:05:45.678 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:05:45.677+0000 7f5090bca700 1 -- 172.21.15.162:0/4178714903 wait complete. 2024-04-27T04:05:45.679 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:05:45.677+0000 7f5090bca700 1 Processor -- start 2024-04-27T04:05:45.679 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:05:45.677+0000 7f5090bca700 1 -- start start 2024-04-27T04:05:45.679 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:05:45.677+0000 7f5090bca700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f508c0fbce0 0x7f508c066cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:05:45.679 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:05:45.677+0000 7f5090bca700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f508c0fcf90 0x7f508c112720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:05:45.679 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:05:45.677+0000 7f5090bca700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f508c112c60 con 0x7f508c0fbce0 2024-04-27T04:05:45.679 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:05:45.677+0000 7f5090bca700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f508c112da0 con 0x7f508c0fcf90 2024-04-27T04:05:45.679 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:05:45.678+0000 7f508a59c700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f508c0fbce0 0x7f508c066cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:05:45.679 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:05:45.678+0000 7f508a59c700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f508c0fbce0 0x7f508c066cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:41146/0 (socket says 172.21.15.162:41146) 2024-04-27T04:05:45.679 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:05:45.678+0000 7f508a59c700 1 -- 172.21.15.162:0/1187165190 learned_addr learned my addr 172.21.15.162:0/1187165190 (peer_addr_for_me v2:172.21.15.162:0/0) 2024-04-27T04:05:45.679 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:05:45.678+0000 7f5089d9b700 1 --2- 172.21.15.162:0/1187165190 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f508c0fcf90 0x7f508c112720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:05:45.679 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:05:45.678+0000 7f508a59c700 1 -- 172.21.15.162:0/1187165190 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f508c0fcf90 msgr2=0x7f508c112720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:05:45.679 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:05:45.678+0000 7f508a59c700 1 --2- 172.21.15.162:0/1187165190 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f508c0fcf90 0x7f508c112720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:05:45.679 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:05:45.678+0000 7f508a59c700 1 -- 172.21.15.162:0/1187165190 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f507c005040 con 0x7f508c0fbce0 2024-04-27T04:05:45.679 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:05:45.678+0000 7f508a59c700 1 --2- 172.21.15.162:0/1187165190 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f508c0fbce0 0x7f508c066cf0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f507c009150 tx=0x7f507c009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:05:45.680 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:05:45.678+0000 7f507affd700 1 -- 172.21.15.162:0/1187165190 <== mon.0 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f507c00e040 con 0x7f508c0fbce0 2024-04-27T04:05:45.680 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:05:45.678+0000 7f507affd700 1 -- 172.21.15.162:0/1187165190 <== mon.0 v2:172.21.15.162:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f507c004290 con 0x7f508c0fbce0 2024-04-27T04:05:45.680 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:05:45.678+0000 7f5090bca700 1 -- 172.21.15.162:0/1187165190 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f508c112fa0 con 0x7f508c0fbce0 2024-04-27T04:05:45.680 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:05:45.678+0000 7f507affd700 1 -- 172.21.15.162:0/1187165190 <== mon.0 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f507c01c630 con 0x7f508c0fbce0 2024-04-27T04:05:45.680 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:05:45.678+0000 7f5090bca700 1 -- 172.21.15.162:0/1187165190 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f508c1134d0 con 0x7f508c0fbce0 2024-04-27T04:05:45.681 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:05:45.679+0000 7f5090bca700 1 -- 172.21.15.162:0/1187165190 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f508c04c710 con 0x7f508c0fbce0 2024-04-27T04:05:45.681 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:05:45.679+0000 7f507affd700 1 -- 172.21.15.162:0/1187165190 <== mon.0 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 19) v1 ==== 86631+0+0 (secure 0 0 0) 0x7f507c00bbf0 con 0x7f508c0fbce0 2024-04-27T04:05:45.681 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:05:45.680+0000 7f507affd700 1 --2- 172.21.15.162:0/1187165190 >> [v2:172.21.15.162:6800/4079782599,v1:172.21.15.162:6801/4079782599] conn(0x7f50740692a0 0x7f507406b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:05:45.681 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:05:45.680+0000 7f5089d9b700 1 --2- 172.21.15.162:0/1187165190 >> [v2:172.21.15.162:6800/4079782599,v1:172.21.15.162:6801/4079782599] conn(0x7f50740692a0 0x7f507406b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:05:45.681 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:05:45.680+0000 7f507affd700 1 -- 172.21.15.162:0/1187165190 <== mon.0 v2:172.21.15.162:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4634+0+0 (secure 0 0 0) 0x7f507c087a10 con 0x7f508c0fbce0 2024-04-27T04:05:45.681 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:05:45.680+0000 7f5089d9b700 1 --2- 172.21.15.162:0/1187165190 >> [v2:172.21.15.162:6800/4079782599,v1:172.21.15.162:6801/4079782599] conn(0x7f50740692a0 0x7f507406b750 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f5080007fd0 tx=0x7f5080009040 comp rx=0 tx=0).ready entity=mgr.14210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:05:45.683 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:05:45.684+0000 7f507affd700 1 -- 172.21.15.162:0/1187165190 <== mon.0 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f507c057bb0 con 0x7f508c0fbce0 2024-04-27T04:05:45.948 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:05:45.949+0000 7f5090bca700 1 -- 172.21.15.162:0/1187165190 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} v 0) v1 -- 0x7f508c1047d0 con 0x7f508c0fbce0 2024-04-27T04:05:45.952 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:05:45.953+0000 7f507affd700 1 -- 172.21.15.162:0/1187165190 <== mon.0 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]=0 v14) v1 ==== 170+0+59 (secure 0 0 0) 0x7f507c057740 con 0x7f508c0fbce0 2024-04-27T04:05:45.952 INFO:teuthology.orchestra.run.smithi162.stdout:[client.0] 2024-04-27T04:05:45.952 INFO:teuthology.orchestra.run.smithi162.stdout: key = AQCZeSxmvlejOBAA3vhZ4C/Ok4UhbTz0Ifit2Q== 2024-04-27T04:05:45.954 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:05:45.955+0000 7f5090bca700 1 -- 172.21.15.162:0/1187165190 >> [v2:172.21.15.162:6800/4079782599,v1:172.21.15.162:6801/4079782599] conn(0x7f50740692a0 msgr2=0x7f507406b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:05:45.954 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:05:45.955+0000 7f5090bca700 1 --2- 172.21.15.162:0/1187165190 >> [v2:172.21.15.162:6800/4079782599,v1:172.21.15.162:6801/4079782599] conn(0x7f50740692a0 0x7f507406b750 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f5080007fd0 tx=0x7f5080009040 comp rx=0 tx=0).stop 2024-04-27T04:05:45.954 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:05:45.955+0000 7f5090bca700 1 -- 172.21.15.162:0/1187165190 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f508c0fbce0 msgr2=0x7f508c066cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:05:45.955 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:05:45.955+0000 7f5090bca700 1 --2- 172.21.15.162:0/1187165190 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f508c0fbce0 0x7f508c066cf0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f507c009150 tx=0x7f507c009e30 comp rx=0 tx=0).stop 2024-04-27T04:05:45.955 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:05:45.955+0000 7f5090bca700 1 -- 172.21.15.162:0/1187165190 shutdown_connections 2024-04-27T04:05:45.955 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:05:45.955+0000 7f5090bca700 1 --2- 172.21.15.162:0/1187165190 >> [v2:172.21.15.162:6800/4079782599,v1:172.21.15.162:6801/4079782599] conn(0x7f50740692a0 0x7f507406b750 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:05:45.955 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:05:45.955+0000 7f5090bca700 1 --2- 172.21.15.162:0/1187165190 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f508c0fbce0 0x7f508c066cf0 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:05:45.955 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:05:45.955+0000 7f5090bca700 1 --2- 172.21.15.162:0/1187165190 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f508c0fcf90 0x7f508c112720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:05:45.955 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:05:45.955+0000 7f5090bca700 1 -- 172.21.15.162:0/1187165190 >> 172.21.15.162:0/1187165190 conn(0x7f508c0f7440 msgr2=0x7f508c0f8a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:05:45.955 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:05:45.955+0000 7f5090bca700 1 -- 172.21.15.162:0/1187165190 shutdown_connections 2024-04-27T04:05:45.955 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:05:45.955+0000 7f5090bca700 1 -- 172.21.15.162:0/1187165190 wait complete. 2024-04-27T04:05:46.138 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:05:45 smithi162 ceph-mon[26841]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-27T04:05:46.173 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:05:45 smithi181 ceph-mon[33774]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-27T04:05:46.650 DEBUG:teuthology.orchestra.run.smithi162:> set -ex 2024-04-27T04:05:46.650 DEBUG:teuthology.orchestra.run.smithi162:> sudo dd of=/etc/ceph/ceph.client.0.keyring 2024-04-27T04:05:46.650 DEBUG:teuthology.orchestra.run.smithi162:> sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-04-27T04:05:46.691 DEBUG:teuthology.orchestra.run.smithi181:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 7f9750d6-044a-11ef-bc93-c7b262605968 -- ceph auth get-or-create client.1 mon 'allow *' osd 'allow *' mds 'allow *' mgr 'allow *' 2024-04-27T04:05:46.888 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:05:46 smithi162 ceph-mon[26841]: from='client.? 172.21.15.162:0/1187165190' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-04-27T04:05:46.888 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:05:46 smithi162 ceph-mon[26841]: from='client.? 172.21.15.162:0/1187165190' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-04-27T04:05:46.888 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:05:46 smithi162 ceph-mon[26841]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-27T04:05:46.953 INFO:teuthology.orchestra.run.smithi181.stderr:Inferring config /var/lib/ceph/7f9750d6-044a-11ef-bc93-c7b262605968/mon.smithi181/config 2024-04-27T04:05:47.066 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:05:46 smithi181 ceph-mon[33774]: from='client.? 172.21.15.162:0/1187165190' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-04-27T04:05:47.066 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:05:46 smithi181 ceph-mon[33774]: from='client.? 172.21.15.162:0/1187165190' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-04-27T04:05:47.067 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:05:46 smithi181 ceph-mon[33774]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-27T04:05:48.223 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:05:47 smithi181 ceph-mon[33774]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-27T04:05:48.388 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:05:47 smithi162 ceph-mon[26841]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-27T04:05:48.460 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:05:48.461+0000 7fb9fd708700 1 -- 172.21.15.181:0/2258291617 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb9f80fe4f0 msgr2=0x7fb9f80fe900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:05:48.460 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:05:48.461+0000 7fb9fd708700 1 --2- 172.21.15.181:0/2258291617 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb9f80fe4f0 0x7fb9f80fe900 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7fb9e8009ce0 tx=0x7fb9e8004d10 comp rx=0 tx=0).stop 2024-04-27T04:05:48.460 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:05:48.461+0000 7fb9fd708700 1 -- 172.21.15.181:0/2258291617 shutdown_connections 2024-04-27T04:05:48.460 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:05:48.461+0000 7fb9fd708700 1 --2- 172.21.15.181:0/2258291617 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb9f80fee40 0x7fb9f80faf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:05:48.461 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:05:48.461+0000 7fb9fd708700 1 --2- 172.21.15.181:0/2258291617 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb9f80fe4f0 0x7fb9f80fe900 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:05:48.461 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:05:48.461+0000 7fb9fd708700 1 -- 172.21.15.181:0/2258291617 >> 172.21.15.181:0/2258291617 conn(0x7fb9f80f6bf0 msgr2=0x7fb9f80f9000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:05:48.461 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:05:48.461+0000 7fb9fd708700 1 -- 172.21.15.181:0/2258291617 shutdown_connections 2024-04-27T04:05:48.461 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:05:48.461+0000 7fb9fd708700 1 -- 172.21.15.181:0/2258291617 wait complete. 2024-04-27T04:05:48.463 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:05:48.462+0000 7fb9fd708700 1 Processor -- start 2024-04-27T04:05:48.463 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:05:48.462+0000 7fb9fd708700 1 -- start start 2024-04-27T04:05:48.463 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:05:48.462+0000 7fb9fd708700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb9f80fe4f0 0x7fb9f810da50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:05:48.463 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:05:48.462+0000 7fb9fd708700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb9f80fee40 0x7fb9f810df90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:05:48.463 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:05:48.462+0000 7fb9fd708700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9f810e620 con 0x7fb9f80fe4f0 2024-04-27T04:05:48.463 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:05:48.462+0000 7fb9fd708700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9f8112fc0 con 0x7fb9f80fee40 2024-04-27T04:05:48.463 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:05:48.462+0000 7fb9f67fc700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb9f80fee40 0x7fb9f810df90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:05:48.463 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:05:48.462+0000 7fb9f67fc700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb9f80fee40 0x7fb9f810df90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:42914/0 (socket says 172.21.15.181:42914) 2024-04-27T04:05:48.463 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:05:48.462+0000 7fb9f67fc700 1 -- 172.21.15.181:0/1206819317 learned_addr learned my addr 172.21.15.181:0/1206819317 (peer_addr_for_me v2:172.21.15.181:0/0) 2024-04-27T04:05:48.464 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:05:48.462+0000 7fb9f6ffd700 1 --2- 172.21.15.181:0/1206819317 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb9f80fe4f0 0x7fb9f810da50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:05:48.464 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:05:48.462+0000 7fb9f67fc700 1 -- 172.21.15.181:0/1206819317 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb9f80fe4f0 msgr2=0x7fb9f810da50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:05:48.464 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:05:48.462+0000 7fb9f67fc700 1 --2- 172.21.15.181:0/1206819317 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb9f80fe4f0 0x7fb9f810da50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:05:48.464 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:05:48.462+0000 7fb9f67fc700 1 -- 172.21.15.181:0/1206819317 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb9e8009940 con 0x7fb9f80fee40 2024-04-27T04:05:48.464 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:05:48.462+0000 7fb9f6ffd700 1 --2- 172.21.15.181:0/1206819317 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb9f80fe4f0 0x7fb9f810da50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-27T04:05:48.464 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:05:48.462+0000 7fb9f67fc700 1 --2- 172.21.15.181:0/1206819317 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb9f80fee40 0x7fb9f810df90 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7fb9e000b790 tx=0x7fb9e0009250 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:05:48.464 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:05:48.463+0000 7fb9ef7fe700 1 -- 172.21.15.181:0/1206819317 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb9e000f770 con 0x7fb9f80fee40 2024-04-27T04:05:48.464 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:05:48.463+0000 7fb9fd708700 1 -- 172.21.15.181:0/1206819317 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb9f8113220 con 0x7fb9f80fee40 2024-04-27T04:05:48.464 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:05:48.463+0000 7fb9ef7fe700 1 -- 172.21.15.181:0/1206819317 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fb9e000fda0 con 0x7fb9f80fee40 2024-04-27T04:05:48.464 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:05:48.463+0000 7fb9ef7fe700 1 -- 172.21.15.181:0/1206819317 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb9e000d310 con 0x7fb9f80fee40 2024-04-27T04:05:48.464 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:05:48.463+0000 7fb9fd708700 1 -- 172.21.15.181:0/1206819317 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb9f8113790 con 0x7fb9f80fee40 2024-04-27T04:05:48.464 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:05:48.464+0000 7fb9fd708700 1 -- 172.21.15.181:0/1206819317 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb9f804c710 con 0x7fb9f80fee40 2024-04-27T04:05:48.464 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:05:48.464+0000 7fb9ef7fe700 1 -- 172.21.15.181:0/1206819317 <== mon.1 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 19) v1 ==== 86631+0+0 (secure 0 0 0) 0x7fb9e0010420 con 0x7fb9f80fee40 2024-04-27T04:05:48.465 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:05:48.464+0000 7fb9ef7fe700 1 --2- 172.21.15.181:0/1206819317 >> [v2:172.21.15.162:6800/4079782599,v1:172.21.15.162:6801/4079782599] conn(0x7fb9e40692a0 0x7fb9e406b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:05:48.465 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:05:48.464+0000 7fb9ef7fe700 1 -- 172.21.15.181:0/1206819317 <== mon.1 v2:172.21.15.181:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4634+0+0 (secure 0 0 0) 0x7fb9e0086e80 con 0x7fb9f80fee40 2024-04-27T04:05:48.465 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:05:48.465+0000 7fb9f6ffd700 1 --2- 172.21.15.181:0/1206819317 >> [v2:172.21.15.162:6800/4079782599,v1:172.21.15.162:6801/4079782599] conn(0x7fb9e40692a0 0x7fb9e406b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:05:48.465 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:05:48.465+0000 7fb9f6ffd700 1 --2- 172.21.15.181:0/1206819317 >> [v2:172.21.15.162:6800/4079782599,v1:172.21.15.162:6801/4079782599] conn(0x7fb9e40692a0 0x7fb9e406b750 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fb9e80095a0 tx=0x7fb9e8004d10 comp rx=0 tx=0).ready entity=mgr.14210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:05:48.468 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:05:48.469+0000 7fb9ef7fe700 1 -- 172.21.15.181:0/1206819317 <== mon.1 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7fb9e0056f80 con 0x7fb9f80fee40 2024-04-27T04:05:48.726 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:05:48.726+0000 7fb9fd708700 1 -- 172.21.15.181:0/1206819317 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} v 0) v1 -- 0x7fb9f8113bf0 con 0x7fb9f80fee40 2024-04-27T04:05:48.730 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:05:48.731+0000 7fb9ef7fe700 1 -- 172.21.15.181:0/1206819317 <== mon.1 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]=0 v15) v1 ==== 170+0+59 (secure 0 0 0) 0x7fb9e0056b10 con 0x7fb9f80fee40 2024-04-27T04:05:48.730 INFO:teuthology.orchestra.run.smithi181.stdout:[client.1] 2024-04-27T04:05:48.730 INFO:teuthology.orchestra.run.smithi181.stdout: key = AQCceSxm/ttlKxAAMA1ED2ZVGVL1J3dUnIdwZQ== 2024-04-27T04:05:48.734 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:05:48.733+0000 7fb9fd708700 1 -- 172.21.15.181:0/1206819317 >> [v2:172.21.15.162:6800/4079782599,v1:172.21.15.162:6801/4079782599] conn(0x7fb9e40692a0 msgr2=0x7fb9e406b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:05:48.734 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:05:48.733+0000 7fb9fd708700 1 --2- 172.21.15.181:0/1206819317 >> [v2:172.21.15.162:6800/4079782599,v1:172.21.15.162:6801/4079782599] conn(0x7fb9e40692a0 0x7fb9e406b750 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fb9e80095a0 tx=0x7fb9e8004d10 comp rx=0 tx=0).stop 2024-04-27T04:05:48.734 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:05:48.733+0000 7fb9fd708700 1 -- 172.21.15.181:0/1206819317 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb9f80fee40 msgr2=0x7fb9f810df90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:05:48.734 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:05:48.733+0000 7fb9fd708700 1 --2- 172.21.15.181:0/1206819317 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb9f80fee40 0x7fb9f810df90 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7fb9e000b790 tx=0x7fb9e0009250 comp rx=0 tx=0).stop 2024-04-27T04:05:48.735 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:05:48.733+0000 7fb9fd708700 1 -- 172.21.15.181:0/1206819317 shutdown_connections 2024-04-27T04:05:48.735 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:05:48.733+0000 7fb9fd708700 1 --2- 172.21.15.181:0/1206819317 >> [v2:172.21.15.162:6800/4079782599,v1:172.21.15.162:6801/4079782599] conn(0x7fb9e40692a0 0x7fb9e406b750 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:05:48.735 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:05:48.733+0000 7fb9fd708700 1 --2- 172.21.15.181:0/1206819317 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb9f80fe4f0 0x7fb9f810da50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:05:48.735 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:05:48.733+0000 7fb9fd708700 1 --2- 172.21.15.181:0/1206819317 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb9f80fee40 0x7fb9f810df90 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:05:48.735 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:05:48.733+0000 7fb9fd708700 1 -- 172.21.15.181:0/1206819317 >> 172.21.15.181:0/1206819317 conn(0x7fb9f80f6bf0 msgr2=0x7fb9f80f8190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:05:48.735 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:05:48.733+0000 7fb9fd708700 1 -- 172.21.15.181:0/1206819317 shutdown_connections 2024-04-27T04:05:48.735 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:05:48.733+0000 7fb9fd708700 1 -- 172.21.15.181:0/1206819317 wait complete. 2024-04-27T04:05:49.130 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:05:48 smithi181 ceph-mon[33774]: from='client.? 172.21.15.181:0/1206819317' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-04-27T04:05:49.130 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:05:48 smithi181 ceph-mon[33774]: from='client.? ' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-04-27T04:05:49.131 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:05:48 smithi181 ceph-mon[33774]: from='client.? ' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-04-27T04:05:49.318 DEBUG:teuthology.orchestra.run.smithi181:> set -ex 2024-04-27T04:05:49.318 DEBUG:teuthology.orchestra.run.smithi181:> sudo dd of=/etc/ceph/ceph.client.1.keyring 2024-04-27T04:05:49.318 DEBUG:teuthology.orchestra.run.smithi181:> sudo chmod 0644 /etc/ceph/ceph.client.1.keyring 2024-04-27T04:05:49.355 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-04-27T04:05:49.355 INFO:tasks.cephadm.ceph_manager.ceph:waiting for mgr available 2024-04-27T04:05:49.355 DEBUG:teuthology.orchestra.run.smithi162:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell --fsid 7f9750d6-044a-11ef-bc93-c7b262605968 -- ceph mgr dump --format=json 2024-04-27T04:05:49.380 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:05:48 smithi162 ceph-mon[26841]: from='client.? 172.21.15.181:0/1206819317' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-04-27T04:05:49.380 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:05:48 smithi162 ceph-mon[26841]: from='client.? ' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-04-27T04:05:49.380 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:05:48 smithi162 ceph-mon[26841]: from='client.? ' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-04-27T04:05:49.617 INFO:teuthology.orchestra.run.smithi162.stderr:Inferring config /var/lib/ceph/7f9750d6-044a-11ef-bc93-c7b262605968/mon.smithi162/config 2024-04-27T04:05:50.036 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:05:49 smithi162 ceph-mon[26841]: pgmap v134: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-27T04:05:50.423 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:05:49 smithi181 ceph-mon[33774]: pgmap v134: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-27T04:05:51.302 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:05:51.301+0000 7fc3f3b33700 1 -- 172.21.15.162:0/20746398 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc3ec0fc590 msgr2=0x7fc3ec100bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:05:51.302 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:05:51.301+0000 7fc3f3b33700 1 --2- 172.21.15.162:0/20746398 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc3ec0fc590 0x7fc3ec100bb0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fc3dc004940 tx=0x7fc3dc009e30 comp rx=0 tx=0).stop 2024-04-27T04:05:51.302 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:05:51.301+0000 7fc3f3b33700 1 -- 172.21.15.162:0/20746398 shutdown_connections 2024-04-27T04:05:51.302 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:05:51.301+0000 7fc3f3b33700 1 --2- 172.21.15.162:0/20746398 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc3ec0fc590 0x7fc3ec100bb0 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:05:51.302 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:05:51.301+0000 7fc3f3b33700 1 --2- 172.21.15.162:0/20746398 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc3ec0fbbf0 0x7fc3ec0fbfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:05:51.302 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:05:51.301+0000 7fc3f3b33700 1 -- 172.21.15.162:0/20746398 >> 172.21.15.162:0/20746398 conn(0x7fc3ec0f7850 msgr2=0x7fc3ec0f9c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:05:51.302 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:05:51.301+0000 7fc3f3b33700 1 -- 172.21.15.162:0/20746398 shutdown_connections 2024-04-27T04:05:51.302 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:05:51.301+0000 7fc3f3b33700 1 -- 172.21.15.162:0/20746398 wait complete. 2024-04-27T04:05:51.302 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:05:51.302+0000 7fc3f3b33700 1 Processor -- start 2024-04-27T04:05:51.302 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:05:51.302+0000 7fc3f3b33700 1 -- start start 2024-04-27T04:05:51.302 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:05:51.302+0000 7fc3f3b33700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc3ec0fbbf0 0x7fc3ec11a040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:05:51.303 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:05:51.302+0000 7fc3f3b33700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc3ec0fc590 0x7fc3ec11a580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:05:51.303 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:05:51.302+0000 7fc3f3b33700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3ec11abd0 con 0x7fc3ec0fc590 2024-04-27T04:05:51.303 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:05:51.302+0000 7fc3f3b33700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3ec1141b0 con 0x7fc3ec0fbbf0 2024-04-27T04:05:51.303 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:05:51.302+0000 7fc3f10ce700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc3ec0fc590 0x7fc3ec11a580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:05:51.303 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:05:51.302+0000 7fc3f10ce700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc3ec0fc590 0x7fc3ec11a580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:41166/0 (socket says 172.21.15.162:41166) 2024-04-27T04:05:51.303 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:05:51.302+0000 7fc3f10ce700 1 -- 172.21.15.162:0/4176272653 learned_addr learned my addr 172.21.15.162:0/4176272653 (peer_addr_for_me v2:172.21.15.162:0/0) 2024-04-27T04:05:51.303 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:05:51.302+0000 7fc3f18cf700 1 --2- 172.21.15.162:0/4176272653 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc3ec0fbbf0 0x7fc3ec11a040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:05:51.303 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:05:51.302+0000 7fc3f10ce700 1 -- 172.21.15.162:0/4176272653 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc3ec0fbbf0 msgr2=0x7fc3ec11a040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:05:51.303 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:05:51.302+0000 7fc3f10ce700 1 --2- 172.21.15.162:0/4176272653 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc3ec0fbbf0 0x7fc3ec11a040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:05:51.303 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:05:51.302+0000 7fc3f10ce700 1 -- 172.21.15.162:0/4176272653 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc3dc005040 con 0x7fc3ec0fc590 2024-04-27T04:05:51.303 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:05:51.302+0000 7fc3f10ce700 1 --2- 172.21.15.162:0/4176272653 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc3ec0fc590 0x7fc3ec11a580 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fc3dc00b3b0 tx=0x7fc3dc009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:05:51.303 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:05:51.302+0000 7fc3e27fc700 1 -- 172.21.15.162:0/4176272653 <== mon.0 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc3dc00e040 con 0x7fc3ec0fc590 2024-04-27T04:05:51.303 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:05:51.302+0000 7fc3f3b33700 1 -- 172.21.15.162:0/4176272653 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc3ec114430 con 0x7fc3ec0fc590 2024-04-27T04:05:51.303 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:05:51.302+0000 7fc3e27fc700 1 -- 172.21.15.162:0/4176272653 <== mon.0 v2:172.21.15.162:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fc3dc00bbc0 con 0x7fc3ec0fc590 2024-04-27T04:05:51.304 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:05:51.302+0000 7fc3e27fc700 1 -- 172.21.15.162:0/4176272653 <== mon.0 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc3dc01c3f0 con 0x7fc3ec0fc590 2024-04-27T04:05:51.304 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:05:51.303+0000 7fc3f3b33700 1 -- 172.21.15.162:0/4176272653 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc3ec114970 con 0x7fc3ec0fc590 2024-04-27T04:05:51.304 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:05:51.303+0000 7fc3f3b33700 1 -- 172.21.15.162:0/4176272653 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc3ec066b10 con 0x7fc3ec0fc590 2024-04-27T04:05:51.305 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:05:51.304+0000 7fc3e27fc700 1 -- 172.21.15.162:0/4176272653 <== mon.0 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 19) v1 ==== 86631+0+0 (secure 0 0 0) 0x7fc3dc00bd30 con 0x7fc3ec0fc590 2024-04-27T04:05:51.305 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:05:51.304+0000 7fc3e27fc700 1 --2- 172.21.15.162:0/4176272653 >> [v2:172.21.15.162:6800/4079782599,v1:172.21.15.162:6801/4079782599] conn(0x7fc3d80692a0 0x7fc3d806b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:05:51.305 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:05:51.304+0000 7fc3e27fc700 1 -- 172.21.15.162:0/4176272653 <== mon.0 v2:172.21.15.162:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4634+0+0 (secure 0 0 0) 0x7fc3dc088100 con 0x7fc3ec0fc590 2024-04-27T04:05:51.305 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:05:51.304+0000 7fc3f18cf700 1 --2- 172.21.15.162:0/4176272653 >> [v2:172.21.15.162:6800/4079782599,v1:172.21.15.162:6801/4079782599] conn(0x7fc3d80692a0 0x7fc3d806b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:05:51.305 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:05:51.305+0000 7fc3f18cf700 1 --2- 172.21.15.162:0/4176272653 >> [v2:172.21.15.162:6800/4079782599,v1:172.21.15.162:6801/4079782599] conn(0x7fc3d80692a0 0x7fc3d806b750 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fc3e8007fd0 tx=0x7fc3e8009040 comp rx=0 tx=0).ready entity=mgr.14210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:05:51.307 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:05:51.309+0000 7fc3e27fc700 1 -- 172.21.15.162:0/4176272653 <== mon.0 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7fc3dc058230 con 0x7fc3ec0fc590 2024-04-27T04:05:51.544 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:05:51.545+0000 7fc3f3b33700 1 -- 172.21.15.162:0/4176272653 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) v1 -- 0x7fc3ec04c710 con 0x7fc3ec0fc590 2024-04-27T04:05:51.547 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:05:51.548+0000 7fc3e27fc700 1 -- 172.21.15.162:0/4176272653 <== mon.0 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v19) v1 ==== 74+0+167188 (secure 0 0 0) 0x7fc3dc057dc0 con 0x7fc3ec0fc590 2024-04-27T04:05:51.549 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-04-27T04:05:51.551 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:05:51.550+0000 7fc3f3b33700 1 -- 172.21.15.162:0/4176272653 >> [v2:172.21.15.162:6800/4079782599,v1:172.21.15.162:6801/4079782599] conn(0x7fc3d80692a0 msgr2=0x7fc3d806b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:05:51.551 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:05:51.550+0000 7fc3f3b33700 1 --2- 172.21.15.162:0/4176272653 >> [v2:172.21.15.162:6800/4079782599,v1:172.21.15.162:6801/4079782599] conn(0x7fc3d80692a0 0x7fc3d806b750 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fc3e8007fd0 tx=0x7fc3e8009040 comp rx=0 tx=0).stop 2024-04-27T04:05:51.551 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:05:51.550+0000 7fc3f3b33700 1 -- 172.21.15.162:0/4176272653 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc3ec0fc590 msgr2=0x7fc3ec11a580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:05:51.551 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:05:51.550+0000 7fc3f3b33700 1 --2- 172.21.15.162:0/4176272653 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc3ec0fc590 0x7fc3ec11a580 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fc3dc00b3b0 tx=0x7fc3dc009e30 comp rx=0 tx=0).stop 2024-04-27T04:05:51.551 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:05:51.550+0000 7fc3f3b33700 1 -- 172.21.15.162:0/4176272653 shutdown_connections 2024-04-27T04:05:51.551 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:05:51.550+0000 7fc3f3b33700 1 --2- 172.21.15.162:0/4176272653 >> [v2:172.21.15.162:6800/4079782599,v1:172.21.15.162:6801/4079782599] conn(0x7fc3d80692a0 0x7fc3d806b750 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:05:51.551 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:05:51.550+0000 7fc3f3b33700 1 --2- 172.21.15.162:0/4176272653 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc3ec0fbbf0 0x7fc3ec11a040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:05:51.551 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:05:51.550+0000 7fc3f3b33700 1 --2- 172.21.15.162:0/4176272653 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc3ec0fc590 0x7fc3ec11a580 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:05:51.551 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:05:51.550+0000 7fc3f3b33700 1 -- 172.21.15.162:0/4176272653 >> 172.21.15.162:0/4176272653 conn(0x7fc3ec0f7850 msgr2=0x7fc3ec0f8e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:05:51.551 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:05:51.550+0000 7fc3f3b33700 1 -- 172.21.15.162:0/4176272653 shutdown_connections 2024-04-27T04:05:51.551 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:05:51.551+0000 7fc3f3b33700 1 -- 172.21.15.162:0/4176272653 wait complete. 2024-04-27T04:05:52.138 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:05:51 smithi162 ceph-mon[26841]: pgmap v135: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-27T04:05:52.138 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:05:51 smithi162 ceph-mon[26841]: from='client.? 172.21.15.162:0/4176272653' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-04-27T04:05:52.274 INFO:teuthology.orchestra.run.smithi162.stdout:{"epoch":19,"flags":0,"active_gid":14210,"active_name":"smithi162.oennfn","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6800","nonce":4079782599},{"type":"v1","addr":"172.21.15.162:6801","nonce":4079782599}]},"active_addr":"172.21.15.162:6801/4079782599","active_change":"2024-04-27T04:02:16.661329+0000","active_mgr_features":4540138320759226367,"available":true,"standbys":[{"gid":14230,"name":"smithi181.vtwuhz","mgr_features":4540138320759226367,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2400","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"7","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.1.5","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"drive_group_interval":{"name":"drive_group_interval","type":"float","level":"advanced","flags":0,"default_value":"300.0","min":"","max":"","enum_allowed":[],"desc":"interval in seconds between re-application of applied drive_groups","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["cephadm","dashboard","iostat","nfs","prometheus","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2400","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"7","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.1.5","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"drive_group_interval":{"name":"drive_group_interval","type":"float","level":"advanced","flags":0,"default_value":"300.0","min":"","max":"","enum_allowed":[],"desc":"interval in seconds between re-application of applied drive_groups","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{"dashboard":"https://172.21.15.162:8443/","prometheus":"http://172.21.15.162:9283/"},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":5,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.162:0","nonce":1349712416}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.162:0","nonce":1971036271}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.162:0","nonce":3944767668}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.162:0","nonce":2783280109}]}]} 2024-04-27T04:05:52.278 INFO:tasks.cephadm.ceph_manager.ceph:mgr available! 2024-04-27T04:05:52.278 INFO:tasks.cephadm.ceph_manager.ceph:waiting for all up 2024-04-27T04:05:52.278 DEBUG:teuthology.orchestra.run.smithi162:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell --fsid 7f9750d6-044a-11ef-bc93-c7b262605968 -- ceph osd dump --format=json 2024-04-27T04:05:52.423 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:05:51 smithi181 ceph-mon[33774]: pgmap v135: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-27T04:05:52.423 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:05:51 smithi181 ceph-mon[33774]: from='client.? 172.21.15.162:0/4176272653' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-04-27T04:05:52.573 INFO:teuthology.orchestra.run.smithi162.stderr:Inferring config /var/lib/ceph/7f9750d6-044a-11ef-bc93-c7b262605968/mon.smithi162/config 2024-04-27T04:05:54.200 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:05:54.199+0000 7f1650b45700 1 -- 172.21.15.162:0/2606259408 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f164c102de0 msgr2=0x7f164c1031b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:05:54.200 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:05:54.199+0000 7f1650b45700 1 --2- 172.21.15.162:0/2606259408 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f164c102de0 0x7f164c1031b0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f163c004940 tx=0x7f163c009e30 comp rx=0 tx=0).stop 2024-04-27T04:05:54.200 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:05:54.199+0000 7f1650b45700 1 -- 172.21.15.162:0/2606259408 shutdown_connections 2024-04-27T04:05:54.200 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:05:54.199+0000 7f1650b45700 1 --2- 172.21.15.162:0/2606259408 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f164c103780 0x7f164c0fcdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:05:54.200 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:05:54.199+0000 7f1650b45700 1 --2- 172.21.15.162:0/2606259408 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f164c102de0 0x7f164c1031b0 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:05:54.200 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:05:54.199+0000 7f1650b45700 1 -- 172.21.15.162:0/2606259408 >> 172.21.15.162:0/2606259408 conn(0x7f164c0f8950 msgr2=0x7f164c0fad60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:05:54.200 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:05:54.199+0000 7f1650b45700 1 -- 172.21.15.162:0/2606259408 shutdown_connections 2024-04-27T04:05:54.200 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:05:54.199+0000 7f1650b45700 1 -- 172.21.15.162:0/2606259408 wait complete. 2024-04-27T04:05:54.200 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:05:54.200+0000 7f1650b45700 1 Processor -- start 2024-04-27T04:05:54.200 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:05:54.200+0000 7f1650b45700 1 -- start start 2024-04-27T04:05:54.200 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:05:54.200+0000 7f1650b45700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f164c102de0 0x7f164c070260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:05:54.201 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:05:54.200+0000 7f1650b45700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f164c103780 0x7f164c0707a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:05:54.201 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:05:54.200+0000 7f1650b45700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f164c070d30 con 0x7f164c102de0 2024-04-27T04:05:54.201 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:05:54.200+0000 7f1650b45700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f164c070e70 con 0x7f164c103780 2024-04-27T04:05:54.201 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:05:54.200+0000 7f164a59c700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f164c102de0 0x7f164c070260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:05:54.201 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:05:54.200+0000 7f164a59c700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f164c102de0 0x7f164c070260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:34310/0 (socket says 172.21.15.162:34310) 2024-04-27T04:05:54.201 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:05:54.200+0000 7f164a59c700 1 -- 172.21.15.162:0/2565096025 learned_addr learned my addr 172.21.15.162:0/2565096025 (peer_addr_for_me v2:172.21.15.162:0/0) 2024-04-27T04:05:54.201 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:05:54.200+0000 7f1641d9b700 1 --2- 172.21.15.162:0/2565096025 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f164c103780 0x7f164c0707a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:05:54.201 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:05:54.200+0000 7f164a59c700 1 -- 172.21.15.162:0/2565096025 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f164c103780 msgr2=0x7f164c0707a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:05:54.201 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:05:54.200+0000 7f164a59c700 1 --2- 172.21.15.162:0/2565096025 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f164c103780 0x7f164c0707a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:05:54.201 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:05:54.200+0000 7f164a59c700 1 -- 172.21.15.162:0/2565096025 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f163c005040 con 0x7f164c102de0 2024-04-27T04:05:54.201 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:05:54.200+0000 7f164a59c700 1 --2- 172.21.15.162:0/2565096025 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f164c102de0 0x7f164c070260 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f163c00b8b0 tx=0x7f163c009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:05:54.201 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:05:54.201+0000 7f16437fe700 1 -- 172.21.15.162:0/2565096025 <== mon.0 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f163c00e040 con 0x7f164c102de0 2024-04-27T04:05:54.201 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:05:54.201+0000 7f1650b45700 1 -- 172.21.15.162:0/2565096025 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f164c0710c0 con 0x7f164c102de0 2024-04-27T04:05:54.201 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:05:54.201+0000 7f16437fe700 1 -- 172.21.15.162:0/2565096025 <== mon.0 v2:172.21.15.162:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f163c004290 con 0x7f164c102de0 2024-04-27T04:05:54.201 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:05:54.201+0000 7f16437fe700 1 -- 172.21.15.162:0/2565096025 <== mon.0 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f163c01c3f0 con 0x7f164c102de0 2024-04-27T04:05:54.202 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:05:54.201+0000 7f1650b45700 1 -- 172.21.15.162:0/2565096025 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f164c11a2a0 con 0x7f164c102de0 2024-04-27T04:05:54.202 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:05:54.202+0000 7f1650b45700 1 -- 172.21.15.162:0/2565096025 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f164c1000f0 con 0x7f164c102de0 2024-04-27T04:05:54.202 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:05:54.202+0000 7f16437fe700 1 -- 172.21.15.162:0/2565096025 <== mon.0 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 19) v1 ==== 86631+0+0 (secure 0 0 0) 0x7f163c00baf0 con 0x7f164c102de0 2024-04-27T04:05:54.202 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:05:54.202+0000 7f16437fe700 1 --2- 172.21.15.162:0/2565096025 >> [v2:172.21.15.162:6800/4079782599,v1:172.21.15.162:6801/4079782599] conn(0x7f16380692a0 0x7f163806b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:05:54.203 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:05:54.202+0000 7f16437fe700 1 -- 172.21.15.162:0/2565096025 <== mon.0 v2:172.21.15.162:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4634+0+0 (secure 0 0 0) 0x7f163c088190 con 0x7f164c102de0 2024-04-27T04:05:54.203 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:05:54.202+0000 7f1641d9b700 1 --2- 172.21.15.162:0/2565096025 >> [v2:172.21.15.162:6800/4079782599,v1:172.21.15.162:6801/4079782599] conn(0x7f16380692a0 0x7f163806b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:05:54.203 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:05:54.203+0000 7f1641d9b700 1 --2- 172.21.15.162:0/2565096025 >> [v2:172.21.15.162:6800/4079782599,v1:172.21.15.162:6801/4079782599] conn(0x7f16380692a0 0x7f163806b750 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f1634007fd0 tx=0x7f1634009040 comp rx=0 tx=0).ready entity=mgr.14210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:05:54.206 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:05:54.207+0000 7f16437fe700 1 -- 172.21.15.162:0/2565096025 <== mon.0 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f163c054800 con 0x7f164c102de0 2024-04-27T04:05:54.387 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:05:53 smithi162 ceph-mon[26841]: pgmap v136: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-27T04:05:54.394 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:05:54.395+0000 7f1650b45700 1 -- 172.21.15.162:0/2565096025 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f164c0fe530 con 0x7f164c102de0 2024-04-27T04:05:54.395 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:05:54.396+0000 7f16437fe700 1 -- 172.21.15.162:0/2565096025 <== mon.0 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v40) v1 ==== 74+0+11141 (secure 0 0 0) 0x7f163c057e20 con 0x7f164c102de0 2024-04-27T04:05:54.396 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-04-27T04:05:54.396 INFO:teuthology.orchestra.run.smithi162.stdout:{"epoch":40,"fsid":"7f9750d6-044a-11ef-bc93-c7b262605968","created":"2024-04-27T04:00:06.343616+0000","modified":"2024-04-27T04:05:38.774913+0000","last_up_change":"2024-04-27T04:05:37.769065+0000","last_in_change":"2024-04-27T04:05:18.597695+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":14,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":6,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-27T04:04:24.971779+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}}],"osds":[{"osd":0,"uuid":"0ae69d7f-0ab4-4672-89b9-3563a10a1bf2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6802","nonce":3784722138},{"type":"v1","addr":"172.21.15.162:6803","nonce":3784722138}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6804","nonce":3784722138},{"type":"v1","addr":"172.21.15.162:6805","nonce":3784722138}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6808","nonce":3784722138},{"type":"v1","addr":"172.21.15.162:6809","nonce":3784722138}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6806","nonce":3784722138},{"type":"v1","addr":"172.21.15.162:6807","nonce":3784722138}]},"public_addr":"172.21.15.162:6803/3784722138","cluster_addr":"172.21.15.162:6805/3784722138","heartbeat_back_addr":"172.21.15.162:6809/3784722138","heartbeat_front_addr":"172.21.15.162:6807/3784722138","state":["exists","up"]},{"osd":1,"uuid":"95bdc9f0-c438-4a88-942d-dffb0c61c308","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6810","nonce":3642637124},{"type":"v1","addr":"172.21.15.162:6811","nonce":3642637124}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6812","nonce":3642637124},{"type":"v1","addr":"172.21.15.162:6813","nonce":3642637124}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6816","nonce":3642637124},{"type":"v1","addr":"172.21.15.162:6817","nonce":3642637124}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6814","nonce":3642637124},{"type":"v1","addr":"172.21.15.162:6815","nonce":3642637124}]},"public_addr":"172.21.15.162:6811/3642637124","cluster_addr":"172.21.15.162:6813/3642637124","heartbeat_back_addr":"172.21.15.162:6817/3642637124","heartbeat_front_addr":"172.21.15.162:6815/3642637124","state":["exists","up"]},{"osd":2,"uuid":"a773b386-e75b-432a-9d1b-2206301b96c3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":19,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6818","nonce":3830961923},{"type":"v1","addr":"172.21.15.162:6819","nonce":3830961923}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6820","nonce":3830961923},{"type":"v1","addr":"172.21.15.162:6821","nonce":3830961923}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6824","nonce":3830961923},{"type":"v1","addr":"172.21.15.162:6825","nonce":3830961923}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6822","nonce":3830961923},{"type":"v1","addr":"172.21.15.162:6823","nonce":3830961923}]},"public_addr":"172.21.15.162:6819/3830961923","cluster_addr":"172.21.15.162:6821/3830961923","heartbeat_back_addr":"172.21.15.162:6825/3830961923","heartbeat_front_addr":"172.21.15.162:6823/3830961923","state":["exists","up"]},{"osd":3,"uuid":"756091bc-1594-479d-9390-cb60e6b65ebe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":3695382003},{"type":"v1","addr":"172.21.15.181:6801","nonce":3695382003}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":3695382003},{"type":"v1","addr":"172.21.15.181:6803","nonce":3695382003}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6806","nonce":3695382003},{"type":"v1","addr":"172.21.15.181:6807","nonce":3695382003}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":3695382003},{"type":"v1","addr":"172.21.15.181:6805","nonce":3695382003}]},"public_addr":"172.21.15.181:6801/3695382003","cluster_addr":"172.21.15.181:6803/3695382003","heartbeat_back_addr":"172.21.15.181:6807/3695382003","heartbeat_front_addr":"172.21.15.181:6805/3695382003","state":["exists","up"]},{"osd":4,"uuid":"bd70494d-061c-4468-b837-26150c2d4208","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6808","nonce":2791715456},{"type":"v1","addr":"172.21.15.181:6809","nonce":2791715456}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":2791715456},{"type":"v1","addr":"172.21.15.181:6811","nonce":2791715456}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":2791715456},{"type":"v1","addr":"172.21.15.181:6815","nonce":2791715456}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6812","nonce":2791715456},{"type":"v1","addr":"172.21.15.181:6813","nonce":2791715456}]},"public_addr":"172.21.15.181:6809/2791715456","cluster_addr":"172.21.15.181:6811/2791715456","heartbeat_back_addr":"172.21.15.181:6815/2791715456","heartbeat_front_addr":"172.21.15.181:6813/2791715456","state":["exists","up"]},{"osd":5,"uuid":"15455ef6-2853-4a84-aaa9-1797e78302a9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":3193687515},{"type":"v1","addr":"172.21.15.181:6817","nonce":3193687515}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6818","nonce":3193687515},{"type":"v1","addr":"172.21.15.181:6819","nonce":3193687515}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6822","nonce":3193687515},{"type":"v1","addr":"172.21.15.181:6823","nonce":3193687515}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":3193687515},{"type":"v1","addr":"172.21.15.181:6821","nonce":3193687515}]},"public_addr":"172.21.15.181:6817/3193687515","cluster_addr":"172.21.15.181:6819/3193687515","heartbeat_back_addr":"172.21.15.181:6823/3193687515","heartbeat_front_addr":"172.21.15.181:6821/3193687515","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-04-27T04:03:28.522704+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-04-27T04:03:54.648083+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-04-27T04:04:22.107316+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-04-27T04:04:46.020733+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-04-27T04:05:10.634082+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-04-27T04:05:35.756217+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.162:0/795779642":"2024-04-28T04:02:16.661212+0000","172.21.15.162:0/2336042321":"2024-04-28T04:02:16.661212+0000","172.21.15.162:0/3590827229":"2024-04-28T04:01:10.996874+0000","172.21.15.162:6801/1482851081":"2024-04-28T04:02:16.661212+0000","172.21.15.162:6801/8067198":"2024-04-28T04:01:10.996874+0000","172.21.15.162:0/2623011446":"2024-04-28T04:00:40.642043+0000","172.21.15.162:6800/1482851081":"2024-04-28T04:02:16.661212+0000","172.21.15.162:6801/1480340393":"2024-04-28T04:00:40.642043+0000","172.21.15.162:0/3073351439":"2024-04-28T04:01:10.996874+0000","172.21.15.162:0/2129785584":"2024-04-28T04:01:10.996874+0000","172.21.15.162:0/4025365522":"2024-04-28T04:00:40.642043+0000","172.21.15.162:6800/1480340393":"2024-04-28T04:00:40.642043+0000","172.21.15.162:0/1703886136":"2024-04-28T04:02:16.661212+0000","172.21.15.162:6800/8067198":"2024-04-28T04:01:10.996874+0000","172.21.15.162:0/2579239891":"2024-04-28T04:00:40.642043+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-27T04:05:54.398 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:05:54.398+0000 7f1650b45700 1 -- 172.21.15.162:0/2565096025 >> [v2:172.21.15.162:6800/4079782599,v1:172.21.15.162:6801/4079782599] conn(0x7f16380692a0 msgr2=0x7f163806b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:05:54.398 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:05:54.398+0000 7f1650b45700 1 --2- 172.21.15.162:0/2565096025 >> [v2:172.21.15.162:6800/4079782599,v1:172.21.15.162:6801/4079782599] conn(0x7f16380692a0 0x7f163806b750 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f1634007fd0 tx=0x7f1634009040 comp rx=0 tx=0).stop 2024-04-27T04:05:54.398 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:05:54.398+0000 7f1650b45700 1 -- 172.21.15.162:0/2565096025 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f164c102de0 msgr2=0x7f164c070260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:05:54.398 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:05:54.398+0000 7f1650b45700 1 --2- 172.21.15.162:0/2565096025 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f164c102de0 0x7f164c070260 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f163c00b8b0 tx=0x7f163c009e30 comp rx=0 tx=0).stop 2024-04-27T04:05:54.398 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:05:54.398+0000 7f1650b45700 1 -- 172.21.15.162:0/2565096025 shutdown_connections 2024-04-27T04:05:54.398 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:05:54.398+0000 7f1650b45700 1 --2- 172.21.15.162:0/2565096025 >> [v2:172.21.15.162:6800/4079782599,v1:172.21.15.162:6801/4079782599] conn(0x7f16380692a0 0x7f163806b750 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:05:54.399 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:05:54.398+0000 7f1650b45700 1 --2- 172.21.15.162:0/2565096025 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f164c102de0 0x7f164c070260 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:05:54.399 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:05:54.398+0000 7f1650b45700 1 --2- 172.21.15.162:0/2565096025 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f164c103780 0x7f164c0707a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:05:54.399 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:05:54.398+0000 7f1650b45700 1 -- 172.21.15.162:0/2565096025 >> 172.21.15.162:0/2565096025 conn(0x7f164c0f8950 msgr2=0x7f164c1073d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:05:54.399 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:05:54.399+0000 7f1650b45700 1 -- 172.21.15.162:0/2565096025 shutdown_connections 2024-04-27T04:05:54.399 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:05:54.399+0000 7f1650b45700 1 -- 172.21.15.162:0/2565096025 wait complete. 2024-04-27T04:05:54.423 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:05:53 smithi181 ceph-mon[33774]: pgmap v136: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-27T04:05:55.021 INFO:tasks.cephadm.ceph_manager.ceph:all up! 2024-04-27T04:05:55.021 DEBUG:teuthology.orchestra.run.smithi162:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell --fsid 7f9750d6-044a-11ef-bc93-c7b262605968 -- ceph osd dump --format=json 2024-04-27T04:05:55.137 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:05:54 smithi162 ceph-mon[26841]: from='client.? 172.21.15.162:0/2565096025' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-27T04:05:55.271 INFO:teuthology.orchestra.run.smithi162.stderr:Inferring config /var/lib/ceph/7f9750d6-044a-11ef-bc93-c7b262605968/mon.smithi162/config 2024-04-27T04:05:55.423 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:05:54 smithi181 ceph-mon[33774]: from='client.? 172.21.15.162:0/2565096025' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-27T04:05:56.387 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:05:55 smithi162 ceph-mon[26841]: pgmap v137: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-27T04:05:56.423 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:05:55 smithi181 ceph-mon[33774]: pgmap v137: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-27T04:05:56.911 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:05:56.909+0000 7f48ef6ce700 1 -- 172.21.15.162:0/703684477 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f48e80fbd40 msgr2=0x7f48e80fc1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:05:56.911 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:05:56.909+0000 7f48ef6ce700 1 --2- 172.21.15.162:0/703684477 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f48e80fbd40 0x7f48e80fc1b0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f48d8004940 tx=0x7f48d8009e30 comp rx=0 tx=0).stop 2024-04-27T04:05:56.911 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:05:56.909+0000 7f48ef6ce700 1 -- 172.21.15.162:0/703684477 shutdown_connections 2024-04-27T04:05:56.911 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:05:56.909+0000 7f48ef6ce700 1 --2- 172.21.15.162:0/703684477 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f48e80fbd40 0x7f48e80fc1b0 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:05:56.912 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:05:56.909+0000 7f48ef6ce700 1 --2- 172.21.15.162:0/703684477 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f48e8101d90 0x7f48e8102160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:05:56.912 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:05:56.909+0000 7f48ef6ce700 1 -- 172.21.15.162:0/703684477 >> 172.21.15.162:0/703684477 conn(0x7f48e8071d00 msgr2=0x7f48e8072100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:05:56.912 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:05:56.909+0000 7f48ef6ce700 1 -- 172.21.15.162:0/703684477 shutdown_connections 2024-04-27T04:05:56.912 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:05:56.909+0000 7f48ef6ce700 1 -- 172.21.15.162:0/703684477 wait complete. 2024-04-27T04:05:56.912 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:05:56.909+0000 7f48ef6ce700 1 Processor -- start 2024-04-27T04:05:56.912 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:05:56.909+0000 7f48ef6ce700 1 -- start start 2024-04-27T04:05:56.912 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:05:56.910+0000 7f48ef6ce700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f48e80fbd40 0x7f48e8112a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:05:56.912 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:05:56.910+0000 7f48ef6ce700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f48e8101d90 0x7f48e8112fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:05:56.912 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:05:56.910+0000 7f48ef6ce700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48e8113620 con 0x7f48e80fbd40 2024-04-27T04:05:56.912 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:05:56.910+0000 7f48ef6ce700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48e810ca10 con 0x7f48e8101d90 2024-04-27T04:05:56.912 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:05:56.910+0000 7f48ed46a700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f48e80fbd40 0x7f48e8112a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:05:56.913 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:05:56.910+0000 7f48ed46a700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f48e80fbd40 0x7f48e8112a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:34332/0 (socket says 172.21.15.162:34332) 2024-04-27T04:05:56.913 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:05:56.910+0000 7f48ed46a700 1 -- 172.21.15.162:0/2553791506 learned_addr learned my addr 172.21.15.162:0/2553791506 (peer_addr_for_me v2:172.21.15.162:0/0) 2024-04-27T04:05:56.913 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:05:56.910+0000 7f48ecc69700 1 --2- 172.21.15.162:0/2553791506 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f48e8101d90 0x7f48e8112fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:05:56.913 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:05:56.910+0000 7f48ed46a700 1 -- 172.21.15.162:0/2553791506 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f48e8101d90 msgr2=0x7f48e8112fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:05:56.913 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:05:56.910+0000 7f48ed46a700 1 --2- 172.21.15.162:0/2553791506 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f48e8101d90 0x7f48e8112fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:05:56.913 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:05:56.910+0000 7f48ed46a700 1 -- 172.21.15.162:0/2553791506 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48d8005040 con 0x7f48e80fbd40 2024-04-27T04:05:56.913 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:05:56.910+0000 7f48ed46a700 1 --2- 172.21.15.162:0/2553791506 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f48e80fbd40 0x7f48e8112a90 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f48e0002a50 tx=0x7f48e00046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:05:56.913 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:05:56.910+0000 7f48ddffb700 1 -- 172.21.15.162:0/2553791506 <== mon.0 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f48e0007cc0 con 0x7f48e80fbd40 2024-04-27T04:05:56.913 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:05:56.910+0000 7f48ef6ce700 1 -- 172.21.15.162:0/2553791506 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f48e810ccf0 con 0x7f48e80fbd40 2024-04-27T04:05:56.913 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:05:56.910+0000 7f48ddffb700 1 -- 172.21.15.162:0/2553791506 <== mon.0 v2:172.21.15.162:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f48e000f040 con 0x7f48e80fbd40 2024-04-27T04:05:56.913 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:05:56.910+0000 7f48ddffb700 1 -- 172.21.15.162:0/2553791506 <== mon.0 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f48e000da30 con 0x7f48e80fbd40 2024-04-27T04:05:56.913 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:05:56.911+0000 7f48ef6ce700 1 -- 172.21.15.162:0/2553791506 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f48e810d280 con 0x7f48e80fbd40 2024-04-27T04:05:56.913 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:05:56.911+0000 7f48ef6ce700 1 -- 172.21.15.162:0/2553791506 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f48e80fd430 con 0x7f48e80fbd40 2024-04-27T04:05:56.913 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:05:56.912+0000 7f48ddffb700 1 -- 172.21.15.162:0/2553791506 <== mon.0 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 19) v1 ==== 86631+0+0 (secure 0 0 0) 0x7f48e000db90 con 0x7f48e80fbd40 2024-04-27T04:05:56.914 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:05:56.912+0000 7f48ddffb700 1 --2- 172.21.15.162:0/2553791506 >> [v2:172.21.15.162:6800/4079782599,v1:172.21.15.162:6801/4079782599] conn(0x7f48d40692a0 0x7f48d406b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:05:56.914 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:05:56.912+0000 7f48ddffb700 1 -- 172.21.15.162:0/2553791506 <== mon.0 v2:172.21.15.162:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4634+0+0 (secure 0 0 0) 0x7f48e0086c10 con 0x7f48e80fbd40 2024-04-27T04:05:56.914 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:05:56.912+0000 7f48ecc69700 1 --2- 172.21.15.162:0/2553791506 >> [v2:172.21.15.162:6800/4079782599,v1:172.21.15.162:6801/4079782599] conn(0x7f48d40692a0 0x7f48d406b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:05:56.914 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:05:56.913+0000 7f48ecc69700 1 --2- 172.21.15.162:0/2553791506 >> [v2:172.21.15.162:6800/4079782599,v1:172.21.15.162:6801/4079782599] conn(0x7f48d40692a0 0x7f48d406b750 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f48e810dcf0 tx=0x7f48d8009e30 comp rx=0 tx=0).ready entity=mgr.14210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:05:56.916 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:05:56.917+0000 7f48ddffb700 1 -- 172.21.15.162:0/2553791506 <== mon.0 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f48e0056c90 con 0x7f48e80fbd40 2024-04-27T04:05:57.107 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:05:57.108+0000 7f48ef6ce700 1 -- 172.21.15.162:0/2553791506 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f48e8108df0 con 0x7f48e80fbd40 2024-04-27T04:05:57.108 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:05:57.109+0000 7f48ddffb700 1 -- 172.21.15.162:0/2553791506 <== mon.0 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v40) v1 ==== 74+0+11141 (secure 0 0 0) 0x7f48e0056820 con 0x7f48e80fbd40 2024-04-27T04:05:57.108 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-04-27T04:05:57.108 INFO:teuthology.orchestra.run.smithi162.stdout:{"epoch":40,"fsid":"7f9750d6-044a-11ef-bc93-c7b262605968","created":"2024-04-27T04:00:06.343616+0000","modified":"2024-04-27T04:05:38.774913+0000","last_up_change":"2024-04-27T04:05:37.769065+0000","last_in_change":"2024-04-27T04:05:18.597695+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":14,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":6,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-27T04:04:24.971779+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}}],"osds":[{"osd":0,"uuid":"0ae69d7f-0ab4-4672-89b9-3563a10a1bf2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6802","nonce":3784722138},{"type":"v1","addr":"172.21.15.162:6803","nonce":3784722138}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6804","nonce":3784722138},{"type":"v1","addr":"172.21.15.162:6805","nonce":3784722138}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6808","nonce":3784722138},{"type":"v1","addr":"172.21.15.162:6809","nonce":3784722138}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6806","nonce":3784722138},{"type":"v1","addr":"172.21.15.162:6807","nonce":3784722138}]},"public_addr":"172.21.15.162:6803/3784722138","cluster_addr":"172.21.15.162:6805/3784722138","heartbeat_back_addr":"172.21.15.162:6809/3784722138","heartbeat_front_addr":"172.21.15.162:6807/3784722138","state":["exists","up"]},{"osd":1,"uuid":"95bdc9f0-c438-4a88-942d-dffb0c61c308","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6810","nonce":3642637124},{"type":"v1","addr":"172.21.15.162:6811","nonce":3642637124}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6812","nonce":3642637124},{"type":"v1","addr":"172.21.15.162:6813","nonce":3642637124}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6816","nonce":3642637124},{"type":"v1","addr":"172.21.15.162:6817","nonce":3642637124}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6814","nonce":3642637124},{"type":"v1","addr":"172.21.15.162:6815","nonce":3642637124}]},"public_addr":"172.21.15.162:6811/3642637124","cluster_addr":"172.21.15.162:6813/3642637124","heartbeat_back_addr":"172.21.15.162:6817/3642637124","heartbeat_front_addr":"172.21.15.162:6815/3642637124","state":["exists","up"]},{"osd":2,"uuid":"a773b386-e75b-432a-9d1b-2206301b96c3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":19,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6818","nonce":3830961923},{"type":"v1","addr":"172.21.15.162:6819","nonce":3830961923}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6820","nonce":3830961923},{"type":"v1","addr":"172.21.15.162:6821","nonce":3830961923}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6824","nonce":3830961923},{"type":"v1","addr":"172.21.15.162:6825","nonce":3830961923}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6822","nonce":3830961923},{"type":"v1","addr":"172.21.15.162:6823","nonce":3830961923}]},"public_addr":"172.21.15.162:6819/3830961923","cluster_addr":"172.21.15.162:6821/3830961923","heartbeat_back_addr":"172.21.15.162:6825/3830961923","heartbeat_front_addr":"172.21.15.162:6823/3830961923","state":["exists","up"]},{"osd":3,"uuid":"756091bc-1594-479d-9390-cb60e6b65ebe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":3695382003},{"type":"v1","addr":"172.21.15.181:6801","nonce":3695382003}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":3695382003},{"type":"v1","addr":"172.21.15.181:6803","nonce":3695382003}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6806","nonce":3695382003},{"type":"v1","addr":"172.21.15.181:6807","nonce":3695382003}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":3695382003},{"type":"v1","addr":"172.21.15.181:6805","nonce":3695382003}]},"public_addr":"172.21.15.181:6801/3695382003","cluster_addr":"172.21.15.181:6803/3695382003","heartbeat_back_addr":"172.21.15.181:6807/3695382003","heartbeat_front_addr":"172.21.15.181:6805/3695382003","state":["exists","up"]},{"osd":4,"uuid":"bd70494d-061c-4468-b837-26150c2d4208","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6808","nonce":2791715456},{"type":"v1","addr":"172.21.15.181:6809","nonce":2791715456}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":2791715456},{"type":"v1","addr":"172.21.15.181:6811","nonce":2791715456}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":2791715456},{"type":"v1","addr":"172.21.15.181:6815","nonce":2791715456}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6812","nonce":2791715456},{"type":"v1","addr":"172.21.15.181:6813","nonce":2791715456}]},"public_addr":"172.21.15.181:6809/2791715456","cluster_addr":"172.21.15.181:6811/2791715456","heartbeat_back_addr":"172.21.15.181:6815/2791715456","heartbeat_front_addr":"172.21.15.181:6813/2791715456","state":["exists","up"]},{"osd":5,"uuid":"15455ef6-2853-4a84-aaa9-1797e78302a9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":3193687515},{"type":"v1","addr":"172.21.15.181:6817","nonce":3193687515}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6818","nonce":3193687515},{"type":"v1","addr":"172.21.15.181:6819","nonce":3193687515}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6822","nonce":3193687515},{"type":"v1","addr":"172.21.15.181:6823","nonce":3193687515}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":3193687515},{"type":"v1","addr":"172.21.15.181:6821","nonce":3193687515}]},"public_addr":"172.21.15.181:6817/3193687515","cluster_addr":"172.21.15.181:6819/3193687515","heartbeat_back_addr":"172.21.15.181:6823/3193687515","heartbeat_front_addr":"172.21.15.181:6821/3193687515","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-04-27T04:03:28.522704+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-04-27T04:03:54.648083+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-04-27T04:04:22.107316+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-04-27T04:04:46.020733+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-04-27T04:05:10.634082+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-04-27T04:05:35.756217+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.162:0/795779642":"2024-04-28T04:02:16.661212+0000","172.21.15.162:0/2336042321":"2024-04-28T04:02:16.661212+0000","172.21.15.162:0/3590827229":"2024-04-28T04:01:10.996874+0000","172.21.15.162:6801/1482851081":"2024-04-28T04:02:16.661212+0000","172.21.15.162:6801/8067198":"2024-04-28T04:01:10.996874+0000","172.21.15.162:0/2623011446":"2024-04-28T04:00:40.642043+0000","172.21.15.162:6800/1482851081":"2024-04-28T04:02:16.661212+0000","172.21.15.162:6801/1480340393":"2024-04-28T04:00:40.642043+0000","172.21.15.162:0/3073351439":"2024-04-28T04:01:10.996874+0000","172.21.15.162:0/2129785584":"2024-04-28T04:01:10.996874+0000","172.21.15.162:0/4025365522":"2024-04-28T04:00:40.642043+0000","172.21.15.162:6800/1480340393":"2024-04-28T04:00:40.642043+0000","172.21.15.162:0/1703886136":"2024-04-28T04:02:16.661212+0000","172.21.15.162:6800/8067198":"2024-04-28T04:01:10.996874+0000","172.21.15.162:0/2579239891":"2024-04-28T04:00:40.642043+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-27T04:05:57.110 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:05:57.111+0000 7f48ef6ce700 1 -- 172.21.15.162:0/2553791506 >> [v2:172.21.15.162:6800/4079782599,v1:172.21.15.162:6801/4079782599] conn(0x7f48d40692a0 msgr2=0x7f48d406b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:05:57.110 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:05:57.111+0000 7f48ef6ce700 1 --2- 172.21.15.162:0/2553791506 >> [v2:172.21.15.162:6800/4079782599,v1:172.21.15.162:6801/4079782599] conn(0x7f48d40692a0 0x7f48d406b750 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f48e810dcf0 tx=0x7f48d8009e30 comp rx=0 tx=0).stop 2024-04-27T04:05:57.111 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:05:57.111+0000 7f48ef6ce700 1 -- 172.21.15.162:0/2553791506 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f48e80fbd40 msgr2=0x7f48e8112a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:05:57.111 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:05:57.111+0000 7f48ef6ce700 1 --2- 172.21.15.162:0/2553791506 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f48e80fbd40 0x7f48e8112a90 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f48e0002a50 tx=0x7f48e00046b0 comp rx=0 tx=0).stop 2024-04-27T04:05:57.111 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:05:57.111+0000 7f48ef6ce700 1 -- 172.21.15.162:0/2553791506 shutdown_connections 2024-04-27T04:05:57.111 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:05:57.111+0000 7f48ef6ce700 1 --2- 172.21.15.162:0/2553791506 >> [v2:172.21.15.162:6800/4079782599,v1:172.21.15.162:6801/4079782599] conn(0x7f48d40692a0 0x7f48d406b750 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:05:57.111 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:05:57.111+0000 7f48ef6ce700 1 --2- 172.21.15.162:0/2553791506 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f48e80fbd40 0x7f48e8112a90 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:05:57.111 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:05:57.111+0000 7f48ef6ce700 1 --2- 172.21.15.162:0/2553791506 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f48e8101d90 0x7f48e8112fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:05:57.111 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:05:57.111+0000 7f48ef6ce700 1 -- 172.21.15.162:0/2553791506 >> 172.21.15.162:0/2553791506 conn(0x7f48e8071d00 msgr2=0x7f48e8104a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:05:57.111 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:05:57.111+0000 7f48ef6ce700 1 -- 172.21.15.162:0/2553791506 shutdown_connections 2024-04-27T04:05:57.111 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:05:57.111+0000 7f48ef6ce700 1 -- 172.21.15.162:0/2553791506 wait complete. 2024-04-27T04:05:57.899 DEBUG:teuthology.orchestra.run.smithi162:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell --fsid 7f9750d6-044a-11ef-bc93-c7b262605968 -- ceph tell osd.0 flush_pg_stats 2024-04-27T04:05:57.900 DEBUG:teuthology.orchestra.run.smithi162:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell --fsid 7f9750d6-044a-11ef-bc93-c7b262605968 -- ceph tell osd.1 flush_pg_stats 2024-04-27T04:05:57.901 DEBUG:teuthology.orchestra.run.smithi162:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell --fsid 7f9750d6-044a-11ef-bc93-c7b262605968 -- ceph tell osd.2 flush_pg_stats 2024-04-27T04:05:57.901 DEBUG:teuthology.orchestra.run.smithi162:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell --fsid 7f9750d6-044a-11ef-bc93-c7b262605968 -- ceph tell osd.3 flush_pg_stats 2024-04-27T04:05:57.901 DEBUG:teuthology.orchestra.run.smithi162:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell --fsid 7f9750d6-044a-11ef-bc93-c7b262605968 -- ceph tell osd.4 flush_pg_stats 2024-04-27T04:05:57.902 DEBUG:teuthology.orchestra.run.smithi162:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell --fsid 7f9750d6-044a-11ef-bc93-c7b262605968 -- ceph tell osd.5 flush_pg_stats 2024-04-27T04:05:58.138 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:05:57 smithi162 ceph-mon[26841]: pgmap v138: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-27T04:05:58.139 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:05:57 smithi162 ceph-mon[26841]: from='client.? 172.21.15.162:0/2553791506' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-27T04:05:58.248 INFO:teuthology.orchestra.run.smithi162.stderr:Inferring config /var/lib/ceph/7f9750d6-044a-11ef-bc93-c7b262605968/mon.smithi162/config 2024-04-27T04:05:58.248 INFO:teuthology.orchestra.run.smithi162.stderr:Inferring config /var/lib/ceph/7f9750d6-044a-11ef-bc93-c7b262605968/mon.smithi162/config 2024-04-27T04:05:58.313 INFO:teuthology.orchestra.run.smithi162.stderr:Inferring config /var/lib/ceph/7f9750d6-044a-11ef-bc93-c7b262605968/mon.smithi162/config 2024-04-27T04:05:58.423 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:05:57 smithi181 ceph-mon[33774]: pgmap v138: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-27T04:05:58.423 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:05:57 smithi181 ceph-mon[33774]: from='client.? 172.21.15.162:0/2553791506' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-27T04:06:00.137 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:05:59 smithi162 ceph-mon[26841]: pgmap v139: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-27T04:06:00.423 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:05:59 smithi181 ceph-mon[33774]: pgmap v139: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-27T04:06:00.720 INFO:teuthology.orchestra.run.smithi162.stderr:Inferring config /var/lib/ceph/7f9750d6-044a-11ef-bc93-c7b262605968/mon.smithi162/config 2024-04-27T04:06:00.721 INFO:teuthology.orchestra.run.smithi162.stderr:Inferring config /var/lib/ceph/7f9750d6-044a-11ef-bc93-c7b262605968/mon.smithi162/config 2024-04-27T04:06:00.721 INFO:teuthology.orchestra.run.smithi162.stderr:Inferring config /var/lib/ceph/7f9750d6-044a-11ef-bc93-c7b262605968/mon.smithi162/config 2024-04-27T04:06:01.698 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:01.695+0000 7f36d88a2700 1 -- 172.21.15.162:0/2336580606 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f36d4103790 msgr2=0x7f36d40fce00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:06:01.699 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:01.695+0000 7f36d88a2700 1 --2- 172.21.15.162:0/2336580606 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f36d4103790 0x7f36d40fce00 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f36c4004940 tx=0x7f36c4009e30 comp rx=0 tx=0).stop 2024-04-27T04:06:01.699 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:01.696+0000 7f36d88a2700 1 -- 172.21.15.162:0/2336580606 shutdown_connections 2024-04-27T04:06:01.699 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:01.696+0000 7f36d88a2700 1 --2- 172.21.15.162:0/2336580606 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f36d4103790 0x7f36d40fce00 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:06:01.699 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:01.696+0000 7f36d88a2700 1 --2- 172.21.15.162:0/2336580606 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f36d4102df0 0x7f36d41031c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:06:01.699 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:01.696+0000 7f36d88a2700 1 -- 172.21.15.162:0/2336580606 >> 172.21.15.162:0/2336580606 conn(0x7f36d40f8980 msgr2=0x7f36d40fad90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:06:01.699 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:01.696+0000 7f36d88a2700 1 -- 172.21.15.162:0/2336580606 shutdown_connections 2024-04-27T04:06:01.699 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:01.696+0000 7f36d88a2700 1 -- 172.21.15.162:0/2336580606 wait complete. 2024-04-27T04:06:01.699 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:01.696+0000 7f36d88a2700 1 Processor -- start 2024-04-27T04:06:01.699 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:01.696+0000 7f36d88a2700 1 -- start start 2024-04-27T04:06:01.699 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:01.696+0000 7f36d88a2700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f36d4102df0 0x7f36d4113c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:06:01.700 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:01.696+0000 7f36d88a2700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f36d4103790 0x7f36d4114160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:06:01.700 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:01.696+0000 7f36d88a2700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36d41147b0 con 0x7f36d4103790 2024-04-27T04:06:01.700 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:01.696+0000 7f36d88a2700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36d410dba0 con 0x7f36d4102df0 2024-04-27T04:06:01.700 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:01.697+0000 7f36d1d9b700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f36d4103790 0x7f36d4114160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:06:01.700 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:01.697+0000 7f36d1d9b700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f36d4103790 0x7f36d4114160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:34344/0 (socket says 172.21.15.162:34344) 2024-04-27T04:06:01.700 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:01.697+0000 7f36d1d9b700 1 -- 172.21.15.162:0/4275062068 learned_addr learned my addr 172.21.15.162:0/4275062068 (peer_addr_for_me v2:172.21.15.162:0/0) 2024-04-27T04:06:01.700 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:01.697+0000 7f36d259c700 1 --2- 172.21.15.162:0/4275062068 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f36d4102df0 0x7f36d4113c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:06:01.700 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:01.697+0000 7f36d1d9b700 1 -- 172.21.15.162:0/4275062068 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f36d4102df0 msgr2=0x7f36d4113c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:06:01.700 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:01.697+0000 7f36d1d9b700 1 --2- 172.21.15.162:0/4275062068 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f36d4102df0 0x7f36d4113c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:06:01.701 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:01.697+0000 7f36d1d9b700 1 -- 172.21.15.162:0/4275062068 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f36c4005040 con 0x7f36d4103790 2024-04-27T04:06:01.701 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:01.697+0000 7f36d1d9b700 1 --2- 172.21.15.162:0/4275062068 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f36d4103790 0x7f36d4114160 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f36c4009100 tx=0x7f36c4009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:06:01.701 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:01.697+0000 7f36c2ffd700 1 -- 172.21.15.162:0/4275062068 <== mon.0 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f36c400e040 con 0x7f36d4103790 2024-04-27T04:06:01.701 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:01.697+0000 7f36d88a2700 1 -- 172.21.15.162:0/4275062068 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f36d410de20 con 0x7f36d4103790 2024-04-27T04:06:01.701 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:01.697+0000 7f36c2ffd700 1 -- 172.21.15.162:0/4275062068 <== mon.0 v2:172.21.15.162:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f36c4009af0 con 0x7f36d4103790 2024-04-27T04:06:01.701 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:01.697+0000 7f36c2ffd700 1 -- 172.21.15.162:0/4275062068 <== mon.0 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f36c401c3f0 con 0x7f36d4103790 2024-04-27T04:06:01.701 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:01.697+0000 7f36d88a2700 1 -- 172.21.15.162:0/4275062068 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f36d410e3e0 con 0x7f36d4103790 2024-04-27T04:06:01.702 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:01.698+0000 7f36d88a2700 1 -- 172.21.15.162:0/4275062068 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f36d410e980 con 0x7f36d4103790 2024-04-27T04:06:01.704 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:01.699+0000 7f36c2ffd700 1 -- 172.21.15.162:0/4275062068 <== mon.0 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 19) v1 ==== 86631+0+0 (secure 0 0 0) 0x7f36c400baf0 con 0x7f36d4103790 2024-04-27T04:06:01.704 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:01.700+0000 7f36c2ffd700 1 --2- 172.21.15.162:0/4275062068 >> [v2:172.21.15.162:6800/4079782599,v1:172.21.15.162:6801/4079782599] conn(0x7f36bc0692a0 0x7f36bc06b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:06:01.704 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:01.700+0000 7f36c2ffd700 1 -- 172.21.15.162:0/4275062068 <== mon.0 v2:172.21.15.162:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4634+0+0 (secure 0 0 0) 0x7f36c40880e0 con 0x7f36d4103790 2024-04-27T04:06:01.704 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:01.700+0000 7f36d259c700 1 --2- 172.21.15.162:0/4275062068 >> [v2:172.21.15.162:6800/4079782599,v1:172.21.15.162:6801/4079782599] conn(0x7f36bc0692a0 0x7f36bc06b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:06:01.704 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:01.700+0000 7f36c2ffd700 1 --2- 172.21.15.162:0/4275062068 >> [v2:172.21.15.181:6808/2791715456,v1:172.21.15.181:6809/2791715456] conn(0x7f36bc06ef70 0x7f36bc071380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:06:01.704 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:01.700+0000 7f36c2ffd700 1 -- 172.21.15.162:0/4275062068 --> [v2:172.21.15.181:6808/2791715456,v1:172.21.15.181:6809/2791715456] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f36bc071a30 con 0x7f36bc06ef70 2024-04-27T04:06:01.704 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:01.700+0000 7f36c2ffd700 1 -- 172.21.15.162:0/4275062068 <== mon.0 v2:172.21.15.162:3300/0 6 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7f36c408a6d0 con 0x7f36d4103790 2024-04-27T04:06:01.705 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:01.700+0000 7f36d259c700 1 --2- 172.21.15.162:0/4275062068 >> [v2:172.21.15.162:6800/4079782599,v1:172.21.15.162:6801/4079782599] conn(0x7f36bc0692a0 0x7f36bc06b750 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f36c8004ed0 tx=0x7f36c800b040 comp rx=0 tx=0).ready entity=mgr.14210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:06:01.705 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:01.700+0000 7f36d2d9d700 1 --2- 172.21.15.162:0/4275062068 >> [v2:172.21.15.181:6808/2791715456,v1:172.21.15.181:6809/2791715456] conn(0x7f36bc06ef70 0x7f36bc071380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:06:01.705 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:01.701+0000 7f36d2d9d700 1 --2- 172.21.15.162:0/4275062068 >> [v2:172.21.15.181:6808/2791715456,v1:172.21.15.181:6809/2791715456] conn(0x7f36bc06ef70 0x7f36bc071380 crc :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:06:01.705 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:01.701+0000 7f36c2ffd700 1 -- 172.21.15.162:0/4275062068 <== osd.4 v2:172.21.15.181:6808/2791715456 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+24147 (crc 0 0 0) 0x7f36bc071a30 con 0x7f36bc06ef70 2024-04-27T04:06:01.717 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:01.717+0000 7f36d88a2700 1 -- 172.21.15.162:0/4275062068 --> [v2:172.21.15.181:6808/2791715456,v1:172.21.15.181:6809/2791715456] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f36d405e660 con 0x7f36bc06ef70 2024-04-27T04:06:01.717 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:01.717+0000 7f36c2ffd700 1 -- 172.21.15.162:0/4275062068 <== osd.4 v2:172.21.15.181:6808/2791715456 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f36d405e660 con 0x7f36bc06ef70 2024-04-27T04:06:01.719 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:01.717+0000 7f36d88a2700 1 -- 172.21.15.162:0/4275062068 >> [v2:172.21.15.181:6808/2791715456,v1:172.21.15.181:6809/2791715456] conn(0x7f36bc06ef70 msgr2=0x7f36bc071380 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:06:01.719 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:01.717+0000 7f36d88a2700 1 --2- 172.21.15.162:0/4275062068 >> [v2:172.21.15.181:6808/2791715456,v1:172.21.15.181:6809/2791715456] conn(0x7f36bc06ef70 0x7f36bc071380 crc :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:06:01.719 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:01.717+0000 7f36d88a2700 1 -- 172.21.15.162:0/4275062068 >> [v2:172.21.15.162:6800/4079782599,v1:172.21.15.162:6801/4079782599] conn(0x7f36bc0692a0 msgr2=0x7f36bc06b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:06:01.719 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:01.717+0000 7f36d88a2700 1 --2- 172.21.15.162:0/4275062068 >> [v2:172.21.15.162:6800/4079782599,v1:172.21.15.162:6801/4079782599] conn(0x7f36bc0692a0 0x7f36bc06b750 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f36c8004ed0 tx=0x7f36c800b040 comp rx=0 tx=0).stop 2024-04-27T04:06:01.719 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:01.717+0000 7f36d88a2700 1 -- 172.21.15.162:0/4275062068 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f36d4103790 msgr2=0x7f36d4114160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:06:01.719 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:01.717+0000 7f36d88a2700 1 --2- 172.21.15.162:0/4275062068 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f36d4103790 0x7f36d4114160 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f36c4009100 tx=0x7f36c4009e30 comp rx=0 tx=0).stop 2024-04-27T04:06:01.720 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:01.717+0000 7f36d88a2700 1 -- 172.21.15.162:0/4275062068 shutdown_connections 2024-04-27T04:06:01.720 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:01.718+0000 7f36d88a2700 1 --2- 172.21.15.162:0/4275062068 >> [v2:172.21.15.162:6800/4079782599,v1:172.21.15.162:6801/4079782599] conn(0x7f36bc0692a0 0x7f36bc06b750 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:06:01.720 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:01.718+0000 7f36d88a2700 1 --2- 172.21.15.162:0/4275062068 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f36d4102df0 0x7f36d4113c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:06:01.720 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:01.718+0000 7f36d88a2700 1 --2- 172.21.15.162:0/4275062068 >> [v2:172.21.15.181:6808/2791715456,v1:172.21.15.181:6809/2791715456] conn(0x7f36bc06ef70 0x7f36bc071380 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:06:01.720 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:01.718+0000 7f36d88a2700 1 --2- 172.21.15.162:0/4275062068 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f36d4103790 0x7f36d4114160 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:06:01.720 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:01.718+0000 7f36d88a2700 1 -- 172.21.15.162:0/4275062068 >> 172.21.15.162:0/4275062068 conn(0x7f36d40f8980 msgr2=0x7f36d4102200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:06:01.720 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:01.718+0000 7f36d88a2700 1 -- 172.21.15.162:0/4275062068 shutdown_connections 2024-04-27T04:06:01.720 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:01.718+0000 7f36d88a2700 1 -- 172.21.15.162:0/4275062068 wait complete. 2024-04-27T04:06:02.239 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:06:01 smithi181 ceph-mon[33774]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-27T04:06:02.239 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:06:01 smithi181 ceph-mon[33774]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-27T04:06:02.293 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:06:01 smithi162 ceph-mon[26841]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-27T04:06:02.293 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:06:01 smithi162 ceph-mon[26841]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-27T04:06:03.321 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:03.321+0000 7f7105627700 1 -- 172.21.15.162:0/1782502796 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f71000fb4e0 msgr2=0x7f71000fb950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:06:03.321 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:03.321+0000 7f7105627700 1 --2- 172.21.15.162:0/1782502796 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f71000fb4e0 0x7f71000fb950 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f70e8004a50 tx=0x7f70e800af40 comp rx=0 tx=0).stop 2024-04-27T04:06:03.321 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:03.321+0000 7f7105627700 1 -- 172.21.15.162:0/1782502796 shutdown_connections 2024-04-27T04:06:03.321 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:03.321+0000 7f7105627700 1 --2- 172.21.15.162:0/1782502796 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f71000fb4e0 0x7f71000fb950 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:06:03.322 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:03.321+0000 7f7105627700 1 --2- 172.21.15.162:0/1782502796 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f71001014f0 0x7f71001018c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:06:03.322 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:03.321+0000 7f7105627700 1 -- 172.21.15.162:0/1782502796 >> 172.21.15.162:0/1782502796 conn(0x7f7100073350 msgr2=0x7f7100073750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:06:03.322 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:03.321+0000 7f7105627700 1 -- 172.21.15.162:0/1782502796 shutdown_connections 2024-04-27T04:06:03.322 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:03.321+0000 7f7105627700 1 -- 172.21.15.162:0/1782502796 wait complete. 2024-04-27T04:06:03.322 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:03.321+0000 7f7105627700 1 Processor -- start 2024-04-27T04:06:03.323 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:03.321+0000 7f7105627700 1 -- start start 2024-04-27T04:06:03.323 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:03.322+0000 7f7105627700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f71000fb4e0 0x7f7100113930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:06:03.323 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:03.322+0000 7f7105627700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f71001014f0 0x7f7100113e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:06:03.323 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:03.322+0000 7f7105627700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71001144c0 con 0x7f71001014f0 2024-04-27T04:06:03.324 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:03.322+0000 7f7105627700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f710010dac0 con 0x7f71000fb4e0 2024-04-27T04:06:03.324 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:03.322+0000 7f70feffd700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f71000fb4e0 0x7f7100113930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:06:03.324 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:03.322+0000 7f70feffd700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f71000fb4e0 0x7f7100113930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.162:37304/0 (socket says 172.21.15.162:37304) 2024-04-27T04:06:03.324 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:03.322+0000 7f70feffd700 1 -- 172.21.15.162:0/4094506300 learned_addr learned my addr 172.21.15.162:0/4094506300 (peer_addr_for_me v2:172.21.15.162:0/0) 2024-04-27T04:06:03.324 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:03.322+0000 7f70f67fc700 1 --2- 172.21.15.162:0/4094506300 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f71001014f0 0x7f7100113e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:06:03.324 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:03.322+0000 7f70feffd700 1 -- 172.21.15.162:0/4094506300 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f71001014f0 msgr2=0x7f7100113e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:06:03.324 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:03.322+0000 7f70feffd700 1 --2- 172.21.15.162:0/4094506300 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f71001014f0 0x7f7100113e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:06:03.324 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:03.322+0000 7f70feffd700 1 -- 172.21.15.162:0/4094506300 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f70e8005040 con 0x7f71000fb4e0 2024-04-27T04:06:03.324 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:03.322+0000 7f70f67fc700 1 --2- 172.21.15.162:0/4094506300 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f71001014f0 0x7f7100113e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-27T04:06:03.324 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:03.322+0000 7f70feffd700 1 --2- 172.21.15.162:0/4094506300 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f71000fb4e0 0x7f7100113930 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f70f000eb40 tx=0x7f70f000b040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:06:03.324 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:03.322+0000 7f70f7fff700 1 -- 172.21.15.162:0/4094506300 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f70f0007cc0 con 0x7f71000fb4e0 2024-04-27T04:06:03.324 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:03.323+0000 7f70f7fff700 1 -- 172.21.15.162:0/4094506300 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f70f000f040 con 0x7f71000fb4e0 2024-04-27T04:06:03.324 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:03.323+0000 7f70f7fff700 1 -- 172.21.15.162:0/4094506300 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f70f000ca10 con 0x7f71000fb4e0 2024-04-27T04:06:03.324 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:03.323+0000 7f7105627700 1 -- 172.21.15.162:0/4094506300 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f710010dd40 con 0x7f71000fb4e0 2024-04-27T04:06:03.324 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:03.323+0000 7f7105627700 1 -- 172.21.15.162:0/4094506300 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f710010e2d0 con 0x7f71000fb4e0 2024-04-27T04:06:03.326 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:03.323+0000 7f7105627700 1 -- 172.21.15.162:0/4094506300 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f710010e7c0 con 0x7f71000fb4e0 2024-04-27T04:06:03.326 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:03.324+0000 7f70f7fff700 1 -- 172.21.15.162:0/4094506300 <== mon.1 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 19) v1 ==== 86631+0+0 (secure 0 0 0) 0x7f70f000cc80 con 0x7f71000fb4e0 2024-04-27T04:06:03.326 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:03.324+0000 7f70f7fff700 1 --2- 172.21.15.162:0/4094506300 >> [v2:172.21.15.162:6800/4079782599,v1:172.21.15.162:6801/4079782599] conn(0x7f70ec0692a0 0x7f70ec06b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:06:03.326 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:03.324+0000 7f70f7fff700 1 -- 172.21.15.162:0/4094506300 <== mon.1 v2:172.21.15.181:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4634+0+0 (secure 0 0 0) 0x7f70f0086af0 con 0x7f71000fb4e0 2024-04-27T04:06:03.326 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:03.324+0000 7f70f67fc700 1 --2- 172.21.15.162:0/4094506300 >> [v2:172.21.15.162:6800/4079782599,v1:172.21.15.162:6801/4079782599] conn(0x7f70ec0692a0 0x7f70ec06b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:06:03.326 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:03.324+0000 7f70f7fff700 1 --2- 172.21.15.162:0/4094506300 >> [v2:172.21.15.162:6810/3642637124,v1:172.21.15.162:6811/3642637124] conn(0x7f70ec06ef70 0x7f70ec071380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:06:03.326 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:03.325+0000 7f70f7fff700 1 -- 172.21.15.162:0/4094506300 --> [v2:172.21.15.162:6810/3642637124,v1:172.21.15.162:6811/3642637124] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f70ec071a30 con 0x7f70ec06ef70 2024-04-27T04:06:03.326 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:03.325+0000 7f70f7fff700 1 -- 172.21.15.162:0/4094506300 <== mon.1 v2:172.21.15.181:3300/0 6 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7f70f0088ea0 con 0x7f71000fb4e0 2024-04-27T04:06:03.326 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:03.325+0000 7f70ff7fe700 1 --2- 172.21.15.162:0/4094506300 >> [v2:172.21.15.162:6810/3642637124,v1:172.21.15.162:6811/3642637124] conn(0x7f70ec06ef70 0x7f70ec071380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:06:03.326 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:03.325+0000 7f70f67fc700 1 --2- 172.21.15.162:0/4094506300 >> [v2:172.21.15.162:6800/4079782599,v1:172.21.15.162:6801/4079782599] conn(0x7f70ec0692a0 0x7f70ec06b750 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f70e800ac40 tx=0x7f70e800af40 comp rx=0 tx=0).ready entity=mgr.14210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:06:03.327 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:03.325+0000 7f70ff7fe700 1 --2- 172.21.15.162:0/4094506300 >> [v2:172.21.15.162:6810/3642637124,v1:172.21.15.162:6811/3642637124] conn(0x7f70ec06ef70 0x7f70ec071380 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:06:03.327 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:03.326+0000 7f70f7fff700 1 -- 172.21.15.162:0/4094506300 <== osd.1 v2:172.21.15.162:6810/3642637124 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+24147 (crc 0 0 0) 0x7f70ec071a30 con 0x7f70ec06ef70 2024-04-27T04:06:03.341 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:03.340+0000 7f7105627700 1 -- 172.21.15.162:0/4094506300 --> [v2:172.21.15.162:6810/3642637124,v1:172.21.15.162:6811/3642637124] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f710005e660 con 0x7f70ec06ef70 2024-04-27T04:06:03.341 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:03.341+0000 7f70f7fff700 1 -- 172.21.15.162:0/4094506300 <== osd.1 v2:172.21.15.162:6810/3642637124 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f710005e660 con 0x7f70ec06ef70 2024-04-27T04:06:03.342 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:03.341+0000 7f7105627700 1 -- 172.21.15.162:0/4094506300 >> [v2:172.21.15.162:6810/3642637124,v1:172.21.15.162:6811/3642637124] conn(0x7f70ec06ef70 msgr2=0x7f70ec071380 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:06:03.342 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:03.341+0000 7f7105627700 1 --2- 172.21.15.162:0/4094506300 >> [v2:172.21.15.162:6810/3642637124,v1:172.21.15.162:6811/3642637124] conn(0x7f70ec06ef70 0x7f70ec071380 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:06:03.342 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:03.341+0000 7f7105627700 1 -- 172.21.15.162:0/4094506300 >> [v2:172.21.15.162:6800/4079782599,v1:172.21.15.162:6801/4079782599] conn(0x7f70ec0692a0 msgr2=0x7f70ec06b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:06:03.342 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:03.341+0000 7f7105627700 1 --2- 172.21.15.162:0/4094506300 >> [v2:172.21.15.162:6800/4079782599,v1:172.21.15.162:6801/4079782599] conn(0x7f70ec0692a0 0x7f70ec06b750 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f70e800ac40 tx=0x7f70e800af40 comp rx=0 tx=0).stop 2024-04-27T04:06:03.342 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:03.341+0000 7f7105627700 1 -- 172.21.15.162:0/4094506300 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f71000fb4e0 msgr2=0x7f7100113930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:06:03.342 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:03.341+0000 7f7105627700 1 --2- 172.21.15.162:0/4094506300 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f71000fb4e0 0x7f7100113930 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f70f000eb40 tx=0x7f70f000b040 comp rx=0 tx=0).stop 2024-04-27T04:06:03.342 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:03.341+0000 7f7105627700 1 -- 172.21.15.162:0/4094506300 shutdown_connections 2024-04-27T04:06:03.342 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:03.341+0000 7f7105627700 1 --2- 172.21.15.162:0/4094506300 >> [v2:172.21.15.162:6810/3642637124,v1:172.21.15.162:6811/3642637124] conn(0x7f70ec06ef70 0x7f70ec071380 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:06:03.342 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:03.341+0000 7f7105627700 1 --2- 172.21.15.162:0/4094506300 >> [v2:172.21.15.162:6800/4079782599,v1:172.21.15.162:6801/4079782599] conn(0x7f70ec0692a0 0x7f70ec06b750 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:06:03.342 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:03.341+0000 7f7105627700 1 --2- 172.21.15.162:0/4094506300 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f71000fb4e0 0x7f7100113930 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:06:03.342 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:03.341+0000 7f7105627700 1 --2- 172.21.15.162:0/4094506300 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f71001014f0 0x7f7100113e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:06:03.342 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:03.341+0000 7f7105627700 1 -- 172.21.15.162:0/4094506300 >> 172.21.15.162:0/4094506300 conn(0x7f7100073350 msgr2=0x7f71000fa540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:06:03.342 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:03.341+0000 7f7105627700 1 -- 172.21.15.162:0/4094506300 shutdown_connections 2024-04-27T04:06:03.343 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:03.341+0000 7f7105627700 1 -- 172.21.15.162:0/4094506300 wait complete. 2024-04-27T04:06:04.121 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:06:03 smithi162 ceph-mon[26841]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-27T04:06:04.423 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:06:03 smithi181 ceph-mon[33774]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-27T04:06:04.737 INFO:teuthology.orchestra.run.smithi162.stdout:60129542173 2024-04-27T04:06:04.738 DEBUG:teuthology.orchestra.run.smithi162:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell --fsid 7f9750d6-044a-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.1 2024-04-27T04:06:06.137 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:06:05 smithi162 ceph-mon[26841]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-27T04:06:06.423 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:06:05 smithi181 ceph-mon[33774]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-27T04:06:06.620 INFO:teuthology.orchestra.run.smithi162.stdout:146028888076 2024-04-27T04:06:06.621 DEBUG:teuthology.orchestra.run.smithi162:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell --fsid 7f9750d6-044a-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.4 2024-04-27T04:06:07.759 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:07.756+0000 7f1f37fd4700 1 -- 172.21.15.162:0/2465169991 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1f30102de0 msgr2=0x7f1f301031b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:06:07.759 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:07.756+0000 7f1f37fd4700 1 --2- 172.21.15.162:0/2465169991 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1f30102de0 0x7f1f301031b0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f1f2c004940 tx=0x7f1f2c009e30 comp rx=0 tx=0).stop 2024-04-27T04:06:07.759 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:07.756+0000 7f1f37fd4700 1 -- 172.21.15.162:0/2465169991 shutdown_connections 2024-04-27T04:06:07.759 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:07.756+0000 7f1f37fd4700 1 --2- 172.21.15.162:0/2465169991 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1f30103780 0x7f1f300fcdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:06:07.759 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:07.756+0000 7f1f37fd4700 1 --2- 172.21.15.162:0/2465169991 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1f30102de0 0x7f1f301031b0 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:06:07.759 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:07.756+0000 7f1f37fd4700 1 -- 172.21.15.162:0/2465169991 >> 172.21.15.162:0/2465169991 conn(0x7f1f300f8990 msgr2=0x7f1f300fada0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:06:07.759 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:07.756+0000 7f1f37fd4700 1 -- 172.21.15.162:0/2465169991 shutdown_connections 2024-04-27T04:06:07.759 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:07.756+0000 7f1f37fd4700 1 -- 172.21.15.162:0/2465169991 wait complete. 2024-04-27T04:06:07.759 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:07.756+0000 7f1f37fd4700 1 Processor -- start 2024-04-27T04:06:07.760 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:07.757+0000 7f1f37fd4700 1 -- start start 2024-04-27T04:06:07.760 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:07.757+0000 7f1f37fd4700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1f30102de0 0x7f1f30071ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:06:07.760 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:07.757+0000 7f1f37fd4700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1f30103780 0x7f1f30070260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:06:07.760 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:07.757+0000 7f1f37fd4700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f300707a0 con 0x7f1f30103780 2024-04-27T04:06:07.760 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:07.757+0000 7f1f37fd4700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f30070910 con 0x7f1f30102de0 2024-04-27T04:06:07.760 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:07.757+0000 7f1f3556f700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1f30103780 0x7f1f30070260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:06:07.760 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:07.757+0000 7f1f3556f700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1f30103780 0x7f1f30070260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:51754/0 (socket says 172.21.15.162:51754) 2024-04-27T04:06:07.760 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:07.757+0000 7f1f3556f700 1 -- 172.21.15.162:0/3777485896 learned_addr learned my addr 172.21.15.162:0/3777485896 (peer_addr_for_me v2:172.21.15.162:0/0) 2024-04-27T04:06:07.760 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:07.757+0000 7f1f35d70700 1 --2- 172.21.15.162:0/3777485896 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1f30102de0 0x7f1f30071ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:06:07.760 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:07.757+0000 7f1f3556f700 1 -- 172.21.15.162:0/3777485896 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1f30102de0 msgr2=0x7f1f30071ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:06:07.760 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:07.757+0000 7f1f3556f700 1 --2- 172.21.15.162:0/3777485896 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1f30102de0 0x7f1f30071ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:06:07.760 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:07.757+0000 7f1f3556f700 1 -- 172.21.15.162:0/3777485896 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f2c005040 con 0x7f1f30103780 2024-04-27T04:06:07.760 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:07.758+0000 7f1f3556f700 1 --2- 172.21.15.162:0/3777485896 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1f30103780 0x7f1f30070260 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f1f20002a50 tx=0x7f1f200046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:06:07.760 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:07.758+0000 7f1f267fc700 1 -- 172.21.15.162:0/3777485896 <== mon.0 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1f20007cc0 con 0x7f1f30103780 2024-04-27T04:06:07.761 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:07.758+0000 7f1f267fc700 1 -- 172.21.15.162:0/3777485896 <== mon.0 v2:172.21.15.162:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f1f2000f040 con 0x7f1f30103780 2024-04-27T04:06:07.761 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:07.758+0000 7f1f37fd4700 1 -- 172.21.15.162:0/3777485896 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1f30070ba0 con 0x7f1f30103780 2024-04-27T04:06:07.761 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:07.758+0000 7f1f267fc700 1 -- 172.21.15.162:0/3777485896 <== mon.0 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1f2000db30 con 0x7f1f30103780 2024-04-27T04:06:07.761 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:07.758+0000 7f1f37fd4700 1 -- 172.21.15.162:0/3777485896 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1f300710e0 con 0x7f1f30103780 2024-04-27T04:06:07.761 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:07.759+0000 7f1f37fd4700 1 -- 172.21.15.162:0/3777485896 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f1f3005e660 con 0x7f1f30103780 2024-04-27T04:06:07.762 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:07.759+0000 7f1f267fc700 1 -- 172.21.15.162:0/3777485896 <== mon.0 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 19) v1 ==== 86631+0+0 (secure 0 0 0) 0x7f1f2000dd30 con 0x7f1f30103780 2024-04-27T04:06:07.762 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:07.760+0000 7f1f267fc700 1 --2- 172.21.15.162:0/3777485896 >> [v2:172.21.15.162:6800/4079782599,v1:172.21.15.162:6801/4079782599] conn(0x7f1f1c0692a0 0x7f1f1c06b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:06:07.762 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:07.760+0000 7f1f267fc700 1 -- 172.21.15.162:0/3777485896 <== mon.0 v2:172.21.15.162:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4634+0+0 (secure 0 0 0) 0x7f1f20086aa0 con 0x7f1f30103780 2024-04-27T04:06:07.762 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:07.760+0000 7f1f35d70700 1 --2- 172.21.15.162:0/3777485896 >> [v2:172.21.15.162:6800/4079782599,v1:172.21.15.162:6801/4079782599] conn(0x7f1f1c0692a0 0x7f1f1c06b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:06:07.762 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:07.760+0000 7f1f267fc700 1 --2- 172.21.15.162:0/3777485896 >> [v2:172.21.15.181:6816/3193687515,v1:172.21.15.181:6817/3193687515] conn(0x7f1f1c06ef70 0x7f1f1c071380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:06:07.762 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:07.760+0000 7f1f267fc700 1 -- 172.21.15.162:0/3777485896 --> [v2:172.21.15.181:6816/3193687515,v1:172.21.15.181:6817/3193687515] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f1f1c071a30 con 0x7f1f1c06ef70 2024-04-27T04:06:07.762 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:07.760+0000 7f1f267fc700 1 -- 172.21.15.162:0/3777485896 <== mon.0 v2:172.21.15.162:3300/0 6 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1f2008b220 con 0x7f1f30103780 2024-04-27T04:06:07.762 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:07.760+0000 7f1f35d70700 1 --2- 172.21.15.162:0/3777485896 >> [v2:172.21.15.162:6800/4079782599,v1:172.21.15.162:6801/4079782599] conn(0x7f1f1c0692a0 0x7f1f1c06b750 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f1f2c004f90 tx=0x7f1f2c009e30 comp rx=0 tx=0).ready entity=mgr.14210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:06:07.762 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:07.760+0000 7f1f36571700 1 --2- 172.21.15.162:0/3777485896 >> [v2:172.21.15.181:6816/3193687515,v1:172.21.15.181:6817/3193687515] conn(0x7f1f1c06ef70 0x7f1f1c071380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:06:07.763 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:07.761+0000 7f1f36571700 1 --2- 172.21.15.162:0/3777485896 >> [v2:172.21.15.181:6816/3193687515,v1:172.21.15.181:6817/3193687515] conn(0x7f1f1c06ef70 0x7f1f1c071380 crc :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:06:07.763 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:07.761+0000 7f1f267fc700 1 -- 172.21.15.162:0/3777485896 <== osd.5 v2:172.21.15.181:6816/3193687515 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+24147 (crc 0 0 0) 0x7f1f1c071a30 con 0x7f1f1c06ef70 2024-04-27T04:06:07.779 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:07.778+0000 7f1f37fd4700 1 -- 172.21.15.162:0/3777485896 --> [v2:172.21.15.181:6816/3193687515,v1:172.21.15.181:6817/3193687515] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f1f3005ea10 con 0x7f1f1c06ef70 2024-04-27T04:06:07.780 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:07.779+0000 7f1f267fc700 1 -- 172.21.15.162:0/3777485896 <== osd.5 v2:172.21.15.181:6816/3193687515 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f1f3005ea10 con 0x7f1f1c06ef70 2024-04-27T04:06:07.782 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:07.779+0000 7f1f37fd4700 1 -- 172.21.15.162:0/3777485896 >> [v2:172.21.15.181:6816/3193687515,v1:172.21.15.181:6817/3193687515] conn(0x7f1f1c06ef70 msgr2=0x7f1f1c071380 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:06:07.782 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:07.779+0000 7f1f37fd4700 1 --2- 172.21.15.162:0/3777485896 >> [v2:172.21.15.181:6816/3193687515,v1:172.21.15.181:6817/3193687515] conn(0x7f1f1c06ef70 0x7f1f1c071380 crc :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:06:07.782 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:07.779+0000 7f1f37fd4700 1 -- 172.21.15.162:0/3777485896 >> [v2:172.21.15.162:6800/4079782599,v1:172.21.15.162:6801/4079782599] conn(0x7f1f1c0692a0 msgr2=0x7f1f1c06b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:06:07.782 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:07.779+0000 7f1f37fd4700 1 --2- 172.21.15.162:0/3777485896 >> [v2:172.21.15.162:6800/4079782599,v1:172.21.15.162:6801/4079782599] conn(0x7f1f1c0692a0 0x7f1f1c06b750 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f1f2c004f90 tx=0x7f1f2c009e30 comp rx=0 tx=0).stop 2024-04-27T04:06:07.782 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:07.779+0000 7f1f37fd4700 1 -- 172.21.15.162:0/3777485896 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1f30103780 msgr2=0x7f1f30070260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:06:07.782 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:07.779+0000 7f1f37fd4700 1 --2- 172.21.15.162:0/3777485896 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1f30103780 0x7f1f30070260 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f1f20002a50 tx=0x7f1f200046b0 comp rx=0 tx=0).stop 2024-04-27T04:06:07.782 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:07.780+0000 7f1f37fd4700 1 -- 172.21.15.162:0/3777485896 shutdown_connections 2024-04-27T04:06:07.782 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:07.780+0000 7f1f37fd4700 1 --2- 172.21.15.162:0/3777485896 >> [v2:172.21.15.181:6816/3193687515,v1:172.21.15.181:6817/3193687515] conn(0x7f1f1c06ef70 0x7f1f1c071380 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:06:07.782 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:07.780+0000 7f1f37fd4700 1 --2- 172.21.15.162:0/3777485896 >> [v2:172.21.15.162:6800/4079782599,v1:172.21.15.162:6801/4079782599] conn(0x7f1f1c0692a0 0x7f1f1c06b750 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:06:07.782 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:07.780+0000 7f1f37fd4700 1 --2- 172.21.15.162:0/3777485896 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1f30102de0 0x7f1f30071ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:06:07.782 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:07.780+0000 7f1f37fd4700 1 --2- 172.21.15.162:0/3777485896 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1f30103780 0x7f1f30070260 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:06:07.783 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:07.780+0000 7f1f37fd4700 1 -- 172.21.15.162:0/3777485896 >> 172.21.15.162:0/3777485896 conn(0x7f1f300f8990 msgr2=0x7f1f300fa6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:06:07.783 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:07.780+0000 7f1f37fd4700 1 -- 172.21.15.162:0/3777485896 shutdown_connections 2024-04-27T04:06:07.783 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:07.780+0000 7f1f37fd4700 1 -- 172.21.15.162:0/3777485896 wait complete. 2024-04-27T04:06:07.992 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:07.990+0000 7fb2a1d75700 1 -- 172.21.15.162:0/194452597 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb29c102e50 msgr2=0x7fb29c103220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:06:07.992 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:07.990+0000 7fb2a1d75700 1 --2- 172.21.15.162:0/194452597 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb29c102e50 0x7fb29c103220 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fb28c004940 tx=0x7fb28c009e30 comp rx=0 tx=0).stop 2024-04-27T04:06:07.992 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:07.990+0000 7fb2a1d75700 1 -- 172.21.15.162:0/194452597 shutdown_connections 2024-04-27T04:06:07.992 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:07.990+0000 7fb2a1d75700 1 --2- 172.21.15.162:0/194452597 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb29c065640 0x7fb29c065ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:06:07.992 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:07.990+0000 7fb2a1d75700 1 --2- 172.21.15.162:0/194452597 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb29c102e50 0x7fb29c103220 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:06:07.993 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:07.990+0000 7fb2a1d75700 1 -- 172.21.15.162:0/194452597 >> 172.21.15.162:0/194452597 conn(0x7fb29c071d00 msgr2=0x7fb29c072100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:06:07.993 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:07.990+0000 7fb2a1d75700 1 -- 172.21.15.162:0/194452597 shutdown_connections 2024-04-27T04:06:07.993 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:07.990+0000 7fb2a1d75700 1 -- 172.21.15.162:0/194452597 wait complete. 2024-04-27T04:06:07.993 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:07.991+0000 7fb2a1d75700 1 Processor -- start 2024-04-27T04:06:07.993 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:07.991+0000 7fb2a1d75700 1 -- start start 2024-04-27T04:06:07.993 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:07.991+0000 7fb2a1d75700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb29c065640 0x7fb29c113ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:06:07.993 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:07.991+0000 7fb2a1d75700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb29c102e50 0x7fb29c114020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:06:07.993 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:07.991+0000 7fb2a1d75700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb29c114670 con 0x7fb29c102e50 2024-04-27T04:06:07.993 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:07.991+0000 7fb2a1d75700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb29c10da60 con 0x7fb29c065640 2024-04-27T04:06:07.993 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:07.991+0000 7fb29affd700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb29c102e50 0x7fb29c114020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:06:07.993 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:07.991+0000 7fb29affd700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb29c102e50 0x7fb29c114020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:51772/0 (socket says 172.21.15.162:51772) 2024-04-27T04:06:07.993 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:07.991+0000 7fb29affd700 1 -- 172.21.15.162:0/1975262567 learned_addr learned my addr 172.21.15.162:0/1975262567 (peer_addr_for_me v2:172.21.15.162:0/0) 2024-04-27T04:06:07.993 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:07.992+0000 7fb29b7fe700 1 --2- 172.21.15.162:0/1975262567 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb29c065640 0x7fb29c113ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:06:07.994 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:07.992+0000 7fb29affd700 1 -- 172.21.15.162:0/1975262567 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb29c065640 msgr2=0x7fb29c113ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:06:07.994 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:07.992+0000 7fb29affd700 1 --2- 172.21.15.162:0/1975262567 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb29c065640 0x7fb29c113ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:06:07.994 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:07.992+0000 7fb29affd700 1 -- 172.21.15.162:0/1975262567 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb28c005040 con 0x7fb29c102e50 2024-04-27T04:06:07.994 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:07.992+0000 7fb29affd700 1 --2- 172.21.15.162:0/1975262567 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb29c102e50 0x7fb29c114020 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fb29000eb40 tx=0x7fb29000b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:06:07.994 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:07.992+0000 7fb283fff700 1 -- 172.21.15.162:0/1975262567 <== mon.0 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb290007cc0 con 0x7fb29c102e50 2024-04-27T04:06:07.994 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:07.992+0000 7fb2a1d75700 1 -- 172.21.15.162:0/1975262567 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb29c10dce0 con 0x7fb29c102e50 2024-04-27T04:06:07.994 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:07.992+0000 7fb283fff700 1 -- 172.21.15.162:0/1975262567 <== mon.0 v2:172.21.15.162:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fb29000f040 con 0x7fb29c102e50 2024-04-27T04:06:07.994 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:07.992+0000 7fb283fff700 1 -- 172.21.15.162:0/1975262567 <== mon.0 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb29000ca10 con 0x7fb29c102e50 2024-04-27T04:06:07.994 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:07.992+0000 7fb2a1d75700 1 -- 172.21.15.162:0/1975262567 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb29c10e220 con 0x7fb29c102e50 2024-04-27T04:06:07.995 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:07.993+0000 7fb2a1d75700 1 -- 172.21.15.162:0/1975262567 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fb29c10e800 con 0x7fb29c102e50 2024-04-27T04:06:07.996 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:07.993+0000 7fb283fff700 1 -- 172.21.15.162:0/1975262567 <== mon.0 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 19) v1 ==== 86631+0+0 (secure 0 0 0) 0x7fb29000cc80 con 0x7fb29c102e50 2024-04-27T04:06:07.996 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:07.994+0000 7fb283fff700 1 --2- 172.21.15.162:0/1975262567 >> [v2:172.21.15.162:6800/4079782599,v1:172.21.15.162:6801/4079782599] conn(0x7fb2840692a0 0x7fb28406b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:06:07.996 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:07.994+0000 7fb29b7fe700 1 --2- 172.21.15.162:0/1975262567 >> [v2:172.21.15.162:6800/4079782599,v1:172.21.15.162:6801/4079782599] conn(0x7fb2840692a0 0x7fb28406b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:06:07.996 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:07.994+0000 7fb283fff700 1 -- 172.21.15.162:0/1975262567 <== mon.0 v2:172.21.15.162:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4634+0+0 (secure 0 0 0) 0x7fb2900875b0 con 0x7fb29c102e50 2024-04-27T04:06:07.996 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:07.994+0000 7fb283fff700 1 --2- 172.21.15.162:0/1975262567 >> [v2:172.21.15.162:6818/3830961923,v1:172.21.15.162:6819/3830961923] conn(0x7fb28406ef70 0x7fb284071380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:06:07.996 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:07.994+0000 7fb283fff700 1 -- 172.21.15.162:0/1975262567 --> [v2:172.21.15.162:6818/3830961923,v1:172.21.15.162:6819/3830961923] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fb284071a30 con 0x7fb28406ef70 2024-04-27T04:06:07.996 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:07.994+0000 7fb283fff700 1 -- 172.21.15.162:0/1975262567 <== mon.0 v2:172.21.15.162:3300/0 6 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb290087990 con 0x7fb29c102e50 2024-04-27T04:06:07.996 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:07.994+0000 7fb29bfff700 1 --2- 172.21.15.162:0/1975262567 >> [v2:172.21.15.162:6818/3830961923,v1:172.21.15.162:6819/3830961923] conn(0x7fb28406ef70 0x7fb284071380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:06:07.996 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:07.994+0000 7fb29b7fe700 1 --2- 172.21.15.162:0/1975262567 >> [v2:172.21.15.162:6800/4079782599,v1:172.21.15.162:6801/4079782599] conn(0x7fb2840692a0 0x7fb28406b750 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fb28c009b30 tx=0x7fb28c009e30 comp rx=0 tx=0).ready entity=mgr.14210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:06:07.996 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:07.994+0000 7fb29bfff700 1 --2- 172.21.15.162:0/1975262567 >> [v2:172.21.15.162:6818/3830961923,v1:172.21.15.162:6819/3830961923] conn(0x7fb28406ef70 0x7fb284071380 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:06:07.996 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:07.995+0000 7fb283fff700 1 -- 172.21.15.162:0/1975262567 <== osd.2 v2:172.21.15.162:6818/3830961923 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+24147 (crc 0 0 0) 0x7fb284071a30 con 0x7fb28406ef70 2024-04-27T04:06:08.010 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:08.010+0000 7fb2a1d75700 1 -- 172.21.15.162:0/1975262567 --> [v2:172.21.15.162:6818/3830961923,v1:172.21.15.162:6819/3830961923] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fb29c05e660 con 0x7fb28406ef70 2024-04-27T04:06:08.010 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:08.010+0000 7fb283fff700 1 -- 172.21.15.162:0/1975262567 <== osd.2 v2:172.21.15.162:6818/3830961923 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7fb29c05e660 con 0x7fb28406ef70 2024-04-27T04:06:08.011 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:08.010+0000 7fb2a1d75700 1 -- 172.21.15.162:0/1975262567 >> [v2:172.21.15.162:6818/3830961923,v1:172.21.15.162:6819/3830961923] conn(0x7fb28406ef70 msgr2=0x7fb284071380 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:06:08.011 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:08.010+0000 7fb2a1d75700 1 --2- 172.21.15.162:0/1975262567 >> [v2:172.21.15.162:6818/3830961923,v1:172.21.15.162:6819/3830961923] conn(0x7fb28406ef70 0x7fb284071380 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:06:08.011 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:08.010+0000 7fb2a1d75700 1 -- 172.21.15.162:0/1975262567 >> [v2:172.21.15.162:6800/4079782599,v1:172.21.15.162:6801/4079782599] conn(0x7fb2840692a0 msgr2=0x7fb28406b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:06:08.011 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:08.010+0000 7fb2a1d75700 1 --2- 172.21.15.162:0/1975262567 >> [v2:172.21.15.162:6800/4079782599,v1:172.21.15.162:6801/4079782599] conn(0x7fb2840692a0 0x7fb28406b750 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fb28c009b30 tx=0x7fb28c009e30 comp rx=0 tx=0).stop 2024-04-27T04:06:08.011 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:08.010+0000 7fb2a1d75700 1 -- 172.21.15.162:0/1975262567 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb29c102e50 msgr2=0x7fb29c114020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:06:08.011 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:08.010+0000 7fb2a1d75700 1 --2- 172.21.15.162:0/1975262567 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb29c102e50 0x7fb29c114020 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fb29000eb40 tx=0x7fb29000b040 comp rx=0 tx=0).stop 2024-04-27T04:06:08.011 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:08.010+0000 7fb2a1d75700 1 -- 172.21.15.162:0/1975262567 shutdown_connections 2024-04-27T04:06:08.012 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:08.010+0000 7fb2a1d75700 1 --2- 172.21.15.162:0/1975262567 >> [v2:172.21.15.162:6800/4079782599,v1:172.21.15.162:6801/4079782599] conn(0x7fb2840692a0 0x7fb28406b750 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:06:08.012 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:08.010+0000 7fb2a1d75700 1 --2- 172.21.15.162:0/1975262567 >> [v2:172.21.15.162:6818/3830961923,v1:172.21.15.162:6819/3830961923] conn(0x7fb28406ef70 0x7fb284071380 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:06:08.012 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:08.010+0000 7fb2a1d75700 1 --2- 172.21.15.162:0/1975262567 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb29c065640 0x7fb29c113ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:06:08.012 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:08.010+0000 7fb2a1d75700 1 --2- 172.21.15.162:0/1975262567 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb29c102e50 0x7fb29c114020 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:06:08.012 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:08.010+0000 7fb2a1d75700 1 -- 172.21.15.162:0/1975262567 >> 172.21.15.162:0/1975262567 conn(0x7fb29c071d00 msgr2=0x7fb29c0faf60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:06:08.012 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:08.011+0000 7fb2a1d75700 1 -- 172.21.15.162:0/1975262567 shutdown_connections 2024-04-27T04:06:08.012 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:08.011+0000 7fb2a1d75700 1 -- 172.21.15.162:0/1975262567 wait complete. 2024-04-27T04:06:08.137 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:06:07 smithi162 ceph-mon[26841]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-27T04:06:08.423 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:06:07 smithi181 ceph-mon[33774]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-27T04:06:08.628 INFO:teuthology.orchestra.run.smithi162.stdout:167503724551 2024-04-27T04:06:08.629 DEBUG:teuthology.orchestra.run.smithi162:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell --fsid 7f9750d6-044a-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.5 2024-04-27T04:06:09.318 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:09.318+0000 7f8c3580d700 1 -- 172.21.15.162:0/4084495943 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8c30101d90 msgr2=0x7f8c30102160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:06:09.319 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:09.318+0000 7f8c3580d700 1 --2- 172.21.15.162:0/4084495943 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8c30101d90 0x7f8c30102160 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f8c20009ce0 tx=0x7f8c20004d10 comp rx=0 tx=0).stop 2024-04-27T04:06:09.319 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:09.318+0000 7f8c3580d700 1 -- 172.21.15.162:0/4084495943 shutdown_connections 2024-04-27T04:06:09.319 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:09.318+0000 7f8c3580d700 1 --2- 172.21.15.162:0/4084495943 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8c300fbd40 0x7f8c300fc1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:06:09.319 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:09.318+0000 7f8c3580d700 1 --2- 172.21.15.162:0/4084495943 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8c30101d90 0x7f8c30102160 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:06:09.319 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:09.318+0000 7f8c3580d700 1 -- 172.21.15.162:0/4084495943 >> 172.21.15.162:0/4084495943 conn(0x7f8c30071d00 msgr2=0x7f8c30072100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:06:09.319 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:09.318+0000 7f8c3580d700 1 -- 172.21.15.162:0/4084495943 shutdown_connections 2024-04-27T04:06:09.319 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:09.318+0000 7f8c3580d700 1 -- 172.21.15.162:0/4084495943 wait complete. 2024-04-27T04:06:09.319 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:09.318+0000 7f8c3580d700 1 Processor -- start 2024-04-27T04:06:09.319 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:09.319+0000 7f8c3580d700 1 -- start start 2024-04-27T04:06:09.322 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:09.319+0000 7f8c3580d700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8c300fbd40 0x7f8c3010ecb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:06:09.322 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:09.319+0000 7f8c3580d700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8c30101d90 0x7f8c3010f1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:06:09.322 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:09.319+0000 7f8c3580d700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c3010f8d0 con 0x7f8c300fbd40 2024-04-27T04:06:09.322 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:09.319+0000 7f8c3580d700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c3010fa10 con 0x7f8c30101d90 2024-04-27T04:06:09.322 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:09.319+0000 7f8c2effd700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8c300fbd40 0x7f8c3010ecb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:06:09.322 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:09.319+0000 7f8c2effd700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8c300fbd40 0x7f8c3010ecb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:51782/0 (socket says 172.21.15.162:51782) 2024-04-27T04:06:09.322 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:09.319+0000 7f8c2effd700 1 -- 172.21.15.162:0/816293225 learned_addr learned my addr 172.21.15.162:0/816293225 (peer_addr_for_me v2:172.21.15.162:0/0) 2024-04-27T04:06:09.322 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:09.319+0000 7f8c2effd700 1 -- 172.21.15.162:0/816293225 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8c30101d90 msgr2=0x7f8c3010f1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:06:09.323 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:09.319+0000 7f8c2e7fc700 1 --2- 172.21.15.162:0/816293225 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8c30101d90 0x7f8c3010f1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:06:09.323 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:09.319+0000 7f8c2effd700 1 --2- 172.21.15.162:0/816293225 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8c30101d90 0x7f8c3010f1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:06:09.323 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:09.319+0000 7f8c2effd700 1 -- 172.21.15.162:0/816293225 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c20009940 con 0x7f8c300fbd40 2024-04-27T04:06:09.323 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:09.319+0000 7f8c2e7fc700 1 --2- 172.21.15.162:0/816293225 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8c30101d90 0x7f8c3010f1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-27T04:06:09.323 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:09.319+0000 7f8c2effd700 1 --2- 172.21.15.162:0/816293225 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8c300fbd40 0x7f8c3010ecb0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f8c2000be90 tx=0x7f8c20004d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:06:09.323 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:09.319+0000 7f8c277fe700 1 -- 172.21.15.162:0/816293225 <== mon.0 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8c2000e040 con 0x7f8c300fbd40 2024-04-27T04:06:09.323 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:09.320+0000 7f8c277fe700 1 -- 172.21.15.162:0/816293225 <== mon.0 v2:172.21.15.162:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f8c20008ea0 con 0x7f8c300fbd40 2024-04-27T04:06:09.323 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:09.320+0000 7f8c3580d700 1 -- 172.21.15.162:0/816293225 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8c30117d20 con 0x7f8c300fbd40 2024-04-27T04:06:09.323 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:09.320+0000 7f8c277fe700 1 -- 172.21.15.162:0/816293225 <== mon.0 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8c2001c940 con 0x7f8c300fbd40 2024-04-27T04:06:09.323 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:09.320+0000 7f8c3580d700 1 -- 172.21.15.162:0/816293225 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8c301182b0 con 0x7f8c300fbd40 2024-04-27T04:06:09.323 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:09.321+0000 7f8c3580d700 1 -- 172.21.15.162:0/816293225 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f8c300fd430 con 0x7f8c300fbd40 2024-04-27T04:06:09.323 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:09.321+0000 7f8c277fe700 1 -- 172.21.15.162:0/816293225 <== mon.0 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 19) v1 ==== 86631+0+0 (secure 0 0 0) 0x7f8c2001cbb0 con 0x7f8c300fbd40 2024-04-27T04:06:09.323 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:09.321+0000 7f8c277fe700 1 --2- 172.21.15.162:0/816293225 >> [v2:172.21.15.162:6800/4079782599,v1:172.21.15.162:6801/4079782599] conn(0x7f8c1c0692a0 0x7f8c1c06b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:06:09.323 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:09.321+0000 7f8c277fe700 1 -- 172.21.15.162:0/816293225 <== mon.0 v2:172.21.15.162:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4634+0+0 (secure 0 0 0) 0x7f8c20088aa0 con 0x7f8c300fbd40 2024-04-27T04:06:09.323 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:09.321+0000 7f8c2e7fc700 1 --2- 172.21.15.162:0/816293225 >> [v2:172.21.15.162:6800/4079782599,v1:172.21.15.162:6801/4079782599] conn(0x7f8c1c0692a0 0x7f8c1c06b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:06:09.324 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:09.322+0000 7f8c277fe700 1 --2- 172.21.15.162:0/816293225 >> [v2:172.21.15.162:6802/3784722138,v1:172.21.15.162:6803/3784722138] conn(0x7f8c1c06ef70 0x7f8c1c071380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:06:09.324 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:09.322+0000 7f8c277fe700 1 -- 172.21.15.162:0/816293225 --> [v2:172.21.15.162:6802/3784722138,v1:172.21.15.162:6803/3784722138] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f8c1c071a30 con 0x7f8c1c06ef70 2024-04-27T04:06:09.324 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:09.322+0000 7f8c277fe700 1 -- 172.21.15.162:0/816293225 <== mon.0 v2:172.21.15.162:3300/0 6 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8c2008d220 con 0x7f8c300fbd40 2024-04-27T04:06:09.324 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:09.322+0000 7f8c2f7fe700 1 --2- 172.21.15.162:0/816293225 >> [v2:172.21.15.162:6802/3784722138,v1:172.21.15.162:6803/3784722138] conn(0x7f8c1c06ef70 0x7f8c1c071380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:06:09.324 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:09.322+0000 7f8c2e7fc700 1 --2- 172.21.15.162:0/816293225 >> [v2:172.21.15.162:6800/4079782599,v1:172.21.15.162:6801/4079782599] conn(0x7f8c1c0692a0 0x7f8c1c06b750 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f8c18006fd0 tx=0x7f8c18008040 comp rx=0 tx=0).ready entity=mgr.14210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:06:09.324 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:09.322+0000 7f8c2f7fe700 1 --2- 172.21.15.162:0/816293225 >> [v2:172.21.15.162:6802/3784722138,v1:172.21.15.162:6803/3784722138] conn(0x7f8c1c06ef70 0x7f8c1c071380 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:06:09.324 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:09.323+0000 7f8c277fe700 1 -- 172.21.15.162:0/816293225 <== osd.0 v2:172.21.15.162:6802/3784722138 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+24147 (crc 0 0 0) 0x7f8c1c071a30 con 0x7f8c1c06ef70 2024-04-27T04:06:09.337 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:09.337+0000 7f8c3580d700 1 -- 172.21.15.162:0/816293225 --> [v2:172.21.15.162:6802/3784722138,v1:172.21.15.162:6803/3784722138] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f8c300faa10 con 0x7f8c1c06ef70 2024-04-27T04:06:09.337 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:09.337+0000 7f8c277fe700 1 -- 172.21.15.162:0/816293225 <== osd.0 v2:172.21.15.162:6802/3784722138 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f8c300faa10 con 0x7f8c1c06ef70 2024-04-27T04:06:09.338 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:09.338+0000 7f8c3580d700 1 -- 172.21.15.162:0/816293225 >> [v2:172.21.15.162:6802/3784722138,v1:172.21.15.162:6803/3784722138] conn(0x7f8c1c06ef70 msgr2=0x7f8c1c071380 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:06:09.338 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:09.338+0000 7f8c3580d700 1 --2- 172.21.15.162:0/816293225 >> [v2:172.21.15.162:6802/3784722138,v1:172.21.15.162:6803/3784722138] conn(0x7f8c1c06ef70 0x7f8c1c071380 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:06:09.338 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:09.338+0000 7f8c3580d700 1 -- 172.21.15.162:0/816293225 >> [v2:172.21.15.162:6800/4079782599,v1:172.21.15.162:6801/4079782599] conn(0x7f8c1c0692a0 msgr2=0x7f8c1c06b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:06:09.339 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:09.338+0000 7f8c3580d700 1 --2- 172.21.15.162:0/816293225 >> [v2:172.21.15.162:6800/4079782599,v1:172.21.15.162:6801/4079782599] conn(0x7f8c1c0692a0 0x7f8c1c06b750 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f8c18006fd0 tx=0x7f8c18008040 comp rx=0 tx=0).stop 2024-04-27T04:06:09.339 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:09.338+0000 7f8c3580d700 1 -- 172.21.15.162:0/816293225 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8c300fbd40 msgr2=0x7f8c3010ecb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:06:09.339 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:09.338+0000 7f8c3580d700 1 --2- 172.21.15.162:0/816293225 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8c300fbd40 0x7f8c3010ecb0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f8c2000be90 tx=0x7f8c20004d10 comp rx=0 tx=0).stop 2024-04-27T04:06:09.339 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:09.338+0000 7f8c3580d700 1 -- 172.21.15.162:0/816293225 shutdown_connections 2024-04-27T04:06:09.339 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:09.338+0000 7f8c3580d700 1 --2- 172.21.15.162:0/816293225 >> [v2:172.21.15.162:6800/4079782599,v1:172.21.15.162:6801/4079782599] conn(0x7f8c1c0692a0 0x7f8c1c06b750 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:06:09.339 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:09.338+0000 7f8c3580d700 1 --2- 172.21.15.162:0/816293225 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8c300fbd40 0x7f8c3010ecb0 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:06:09.339 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:09.338+0000 7f8c3580d700 1 --2- 172.21.15.162:0/816293225 >> [v2:172.21.15.162:6802/3784722138,v1:172.21.15.162:6803/3784722138] conn(0x7f8c1c06ef70 0x7f8c1c071380 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:06:09.339 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:09.338+0000 7f8c3580d700 1 --2- 172.21.15.162:0/816293225 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8c30101d90 0x7f8c3010f1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:06:09.339 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:09.338+0000 7f8c3580d700 1 -- 172.21.15.162:0/816293225 >> 172.21.15.162:0/816293225 conn(0x7f8c30071d00 msgr2=0x7f8c300fa0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:06:09.339 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:09.338+0000 7f8c3580d700 1 -- 172.21.15.162:0/816293225 shutdown_connections 2024-04-27T04:06:09.339 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:09.338+0000 7f8c3580d700 1 -- 172.21.15.162:0/816293225 wait complete. 2024-04-27T04:06:10.115 INFO:teuthology.orchestra.run.smithi162.stdout:38654705699 2024-04-27T04:06:10.115 DEBUG:teuthology.orchestra.run.smithi162:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell --fsid 7f9750d6-044a-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.0 2024-04-27T04:06:10.387 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:06:09 smithi162 ceph-mon[26841]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-27T04:06:10.390 INFO:teuthology.orchestra.run.smithi162.stderr:Inferring config /var/lib/ceph/7f9750d6-044a-11ef-bc93-c7b262605968/mon.smithi162/config 2024-04-27T04:06:10.423 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:06:09 smithi181 ceph-mon[33774]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-27T04:06:10.475 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:09.551+0000 7f0887b85700 1 -- 172.21.15.162:0/3662022597 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f08800fddf0 msgr2=0x7f08800fe1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:06:10.475 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:09.551+0000 7f0887b85700 1 --2- 172.21.15.162:0/3662022597 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f08800fddf0 0x7f08800fe1c0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f087c004940 tx=0x7f087c009e30 comp rx=0 tx=0).stop 2024-04-27T04:06:10.475 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:09.551+0000 7f0887b85700 1 -- 172.21.15.162:0/3662022597 shutdown_connections 2024-04-27T04:06:10.475 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:09.551+0000 7f0887b85700 1 --2- 172.21.15.162:0/3662022597 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f08800fe790 0x7f0880106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:06:10.476 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:09.551+0000 7f0887b85700 1 --2- 172.21.15.162:0/3662022597 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f08800fddf0 0x7f08800fe1c0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:06:10.476 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:09.551+0000 7f0887b85700 1 -- 172.21.15.162:0/3662022597 >> 172.21.15.162:0/3662022597 conn(0x7f08800f7870 msgr2=0x7f08800f9c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:06:10.476 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:09.552+0000 7f0887b85700 1 -- 172.21.15.162:0/3662022597 shutdown_connections 2024-04-27T04:06:10.476 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:09.552+0000 7f0887b85700 1 -- 172.21.15.162:0/3662022597 wait complete. 2024-04-27T04:06:10.476 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:09.552+0000 7f0887b85700 1 Processor -- start 2024-04-27T04:06:10.476 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:09.552+0000 7f0887b85700 1 -- start start 2024-04-27T04:06:10.476 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:09.553+0000 7f0887b85700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f08800fddf0 0x7f0880113980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:06:10.476 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:09.553+0000 7f0887b85700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f08800fe790 0x7f0880113ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:06:10.476 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:09.553+0000 7f0887b85700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0880114510 con 0x7f08800fddf0 2024-04-27T04:06:10.476 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:09.553+0000 7f0887b85700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f088010db10 con 0x7f08800fe790 2024-04-27T04:06:10.476 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:09.553+0000 7f0885921700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f08800fddf0 0x7f0880113980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:06:10.477 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:09.553+0000 7f0885921700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f08800fddf0 0x7f0880113980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:51812/0 (socket says 172.21.15.162:51812) 2024-04-27T04:06:10.477 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:09.553+0000 7f0885921700 1 -- 172.21.15.162:0/3909175494 learned_addr learned my addr 172.21.15.162:0/3909175494 (peer_addr_for_me v2:172.21.15.162:0/0) 2024-04-27T04:06:10.477 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:09.553+0000 7f0885921700 1 -- 172.21.15.162:0/3909175494 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f08800fe790 msgr2=0x7f0880113ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:06:10.477 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:09.553+0000 7f0885921700 1 --2- 172.21.15.162:0/3909175494 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f08800fe790 0x7f0880113ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:06:10.477 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:09.553+0000 7f0885921700 1 -- 172.21.15.162:0/3909175494 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f087c005040 con 0x7f08800fddf0 2024-04-27T04:06:10.477 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:09.553+0000 7f0885921700 1 --2- 172.21.15.162:0/3909175494 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f08800fddf0 0x7f0880113980 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f087c009380 tx=0x7f087c009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:06:10.477 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:09.553+0000 7f08767fc700 1 -- 172.21.15.162:0/3909175494 <== mon.0 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f087c00e040 con 0x7f08800fddf0 2024-04-27T04:06:10.477 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:09.553+0000 7f08767fc700 1 -- 172.21.15.162:0/3909175494 <== mon.0 v2:172.21.15.162:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f087c009af0 con 0x7f08800fddf0 2024-04-27T04:06:10.477 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:09.553+0000 7f0887b85700 1 -- 172.21.15.162:0/3909175494 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f088010dd90 con 0x7f08800fddf0 2024-04-27T04:06:10.477 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:09.553+0000 7f08767fc700 1 -- 172.21.15.162:0/3909175494 <== mon.0 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f087c01c630 con 0x7f08800fddf0 2024-04-27T04:06:10.477 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:09.553+0000 7f0887b85700 1 -- 172.21.15.162:0/3909175494 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f088010e2d0 con 0x7f08800fddf0 2024-04-27T04:06:10.478 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:09.554+0000 7f0887b85700 1 -- 172.21.15.162:0/3909175494 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f088010e840 con 0x7f08800fddf0 2024-04-27T04:06:10.478 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:09.555+0000 7f08767fc700 1 -- 172.21.15.162:0/3909175494 <== mon.0 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 19) v1 ==== 86631+0+0 (secure 0 0 0) 0x7f087c00bbc0 con 0x7f08800fddf0 2024-04-27T04:06:10.478 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:09.555+0000 7f08767fc700 1 --2- 172.21.15.162:0/3909175494 >> [v2:172.21.15.162:6800/4079782599,v1:172.21.15.162:6801/4079782599] conn(0x7f086c0692a0 0x7f086c06b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:06:10.478 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:09.555+0000 7f08767fc700 1 -- 172.21.15.162:0/3909175494 <== mon.0 v2:172.21.15.162:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4634+0+0 (secure 0 0 0) 0x7f087c088b10 con 0x7f08800fddf0 2024-04-27T04:06:10.478 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:09.555+0000 7f0885120700 1 --2- 172.21.15.162:0/3909175494 >> [v2:172.21.15.162:6800/4079782599,v1:172.21.15.162:6801/4079782599] conn(0x7f086c0692a0 0x7f086c06b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:06:10.478 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:09.555+0000 7f08767fc700 1 --2- 172.21.15.162:0/3909175494 >> [v2:172.21.15.181:6800/3695382003,v1:172.21.15.181:6801/3695382003] conn(0x7f086c06ef70 0x7f086c071380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:06:10.478 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:09.555+0000 7f08767fc700 1 -- 172.21.15.162:0/3909175494 --> [v2:172.21.15.181:6800/3695382003,v1:172.21.15.181:6801/3695382003] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f086c071a30 con 0x7f086c06ef70 2024-04-27T04:06:10.478 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:09.555+0000 7f08767fc700 1 -- 172.21.15.162:0/3909175494 <== mon.0 v2:172.21.15.162:3300/0 6 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7f087c088f10 con 0x7f08800fddf0 2024-04-27T04:06:10.478 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:09.556+0000 7f0885120700 1 --2- 172.21.15.162:0/3909175494 >> [v2:172.21.15.162:6800/4079782599,v1:172.21.15.162:6801/4079782599] conn(0x7f086c0692a0 0x7f086c06b750 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f088010ecd0 tx=0x7f0870007d40 comp rx=0 tx=0).ready entity=mgr.14210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:06:10.478 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:09.556+0000 7f0886122700 1 --2- 172.21.15.162:0/3909175494 >> [v2:172.21.15.181:6800/3695382003,v1:172.21.15.181:6801/3695382003] conn(0x7f086c06ef70 0x7f086c071380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:06:10.478 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:09.556+0000 7f0886122700 1 --2- 172.21.15.162:0/3909175494 >> [v2:172.21.15.181:6800/3695382003,v1:172.21.15.181:6801/3695382003] conn(0x7f086c06ef70 0x7f086c071380 crc :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:06:10.478 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:09.557+0000 7f08767fc700 1 -- 172.21.15.162:0/3909175494 <== osd.3 v2:172.21.15.181:6800/3695382003 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+24147 (crc 0 0 0) 0x7f086c071a30 con 0x7f086c06ef70 2024-04-27T04:06:10.478 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:09.571+0000 7f0887b85700 1 -- 172.21.15.162:0/3909175494 --> [v2:172.21.15.181:6800/3695382003,v1:172.21.15.181:6801/3695382003] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f08800f9690 con 0x7f086c06ef70 2024-04-27T04:06:10.479 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:09.572+0000 7f08767fc700 1 -- 172.21.15.162:0/3909175494 <== osd.3 v2:172.21.15.181:6800/3695382003 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f08800f9690 con 0x7f086c06ef70 2024-04-27T04:06:10.479 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:09.572+0000 7f0887b85700 1 -- 172.21.15.162:0/3909175494 >> [v2:172.21.15.181:6800/3695382003,v1:172.21.15.181:6801/3695382003] conn(0x7f086c06ef70 msgr2=0x7f086c071380 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:06:10.479 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:09.572+0000 7f0887b85700 1 --2- 172.21.15.162:0/3909175494 >> [v2:172.21.15.181:6800/3695382003,v1:172.21.15.181:6801/3695382003] conn(0x7f086c06ef70 0x7f086c071380 crc :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:06:10.479 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:09.572+0000 7f0887b85700 1 -- 172.21.15.162:0/3909175494 >> [v2:172.21.15.162:6800/4079782599,v1:172.21.15.162:6801/4079782599] conn(0x7f086c0692a0 msgr2=0x7f086c06b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:06:10.479 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:09.572+0000 7f0887b85700 1 --2- 172.21.15.162:0/3909175494 >> [v2:172.21.15.162:6800/4079782599,v1:172.21.15.162:6801/4079782599] conn(0x7f086c0692a0 0x7f086c06b750 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f088010ecd0 tx=0x7f0870007d40 comp rx=0 tx=0).stop 2024-04-27T04:06:10.479 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:09.572+0000 7f0887b85700 1 -- 172.21.15.162:0/3909175494 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f08800fddf0 msgr2=0x7f0880113980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:06:10.480 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:09.572+0000 7f0887b85700 1 --2- 172.21.15.162:0/3909175494 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f08800fddf0 0x7f0880113980 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f087c009380 tx=0x7f087c009e30 comp rx=0 tx=0).stop 2024-04-27T04:06:10.480 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:09.573+0000 7f0887b85700 1 -- 172.21.15.162:0/3909175494 shutdown_connections 2024-04-27T04:06:10.480 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:09.573+0000 7f0887b85700 1 --2- 172.21.15.162:0/3909175494 >> [v2:172.21.15.162:6800/4079782599,v1:172.21.15.162:6801/4079782599] conn(0x7f086c0692a0 0x7f086c06b750 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:06:10.480 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:09.573+0000 7f0887b85700 1 --2- 172.21.15.162:0/3909175494 >> [v2:172.21.15.181:6800/3695382003,v1:172.21.15.181:6801/3695382003] conn(0x7f086c06ef70 0x7f086c071380 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:06:10.480 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:09.573+0000 7f0887b85700 1 --2- 172.21.15.162:0/3909175494 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f08800fddf0 0x7f0880113980 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:06:10.480 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:09.573+0000 7f0887b85700 1 --2- 172.21.15.162:0/3909175494 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f08800fe790 0x7f0880113ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:06:10.480 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:09.573+0000 7f0887b85700 1 -- 172.21.15.162:0/3909175494 >> 172.21.15.162:0/3909175494 conn(0x7f08800f7870 msgr2=0x7f08800f8d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:06:10.480 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:09.573+0000 7f0887b85700 1 -- 172.21.15.162:0/3909175494 shutdown_connections 2024-04-27T04:06:10.480 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:09.573+0000 7f0887b85700 1 -- 172.21.15.162:0/3909175494 wait complete. 2024-04-27T04:06:11.027 INFO:teuthology.orchestra.run.smithi162.stdout:111669149715 2024-04-27T04:06:11.027 DEBUG:teuthology.orchestra.run.smithi162:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell --fsid 7f9750d6-044a-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.3 2024-04-27T04:06:11.411 INFO:teuthology.orchestra.run.smithi162.stderr:Inferring config /var/lib/ceph/7f9750d6-044a-11ef-bc93-c7b262605968/mon.smithi162/config 2024-04-27T04:06:11.412 INFO:teuthology.orchestra.run.smithi162.stdout:81604378648 2024-04-27T04:06:11.412 DEBUG:teuthology.orchestra.run.smithi162:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell --fsid 7f9750d6-044a-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.2 2024-04-27T04:06:11.416 INFO:teuthology.orchestra.run.smithi162.stderr:Inferring config /var/lib/ceph/7f9750d6-044a-11ef-bc93-c7b262605968/mon.smithi162/config 2024-04-27T04:06:12.345 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:06:11 smithi162 ceph-mon[26841]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-27T04:06:12.423 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:06:11 smithi181 ceph-mon[33774]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-27T04:06:14.137 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:06:13 smithi162 ceph-mon[26841]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-27T04:06:14.423 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:06:13 smithi181 ceph-mon[33774]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-27T04:06:14.932 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:14.932+0000 7f0f0e2db700 1 -- 172.21.15.162:0/426062533 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0f08102e50 msgr2=0x7f0f08103220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:06:14.933 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:14.932+0000 7f0f0e2db700 1 --2- 172.21.15.162:0/426062533 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0f08102e50 0x7f0f08103220 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f0ef8004940 tx=0x7f0ef8009e30 comp rx=0 tx=0).stop 2024-04-27T04:06:14.933 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:14.932+0000 7f0f0e2db700 1 -- 172.21.15.162:0/426062533 shutdown_connections 2024-04-27T04:06:14.933 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:14.932+0000 7f0f0e2db700 1 --2- 172.21.15.162:0/426062533 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0f08065640 0x7f0f08065ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:06:14.933 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:14.932+0000 7f0f0e2db700 1 --2- 172.21.15.162:0/426062533 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0f08102e50 0x7f0f08103220 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:06:14.933 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:14.932+0000 7f0f0e2db700 1 -- 172.21.15.162:0/426062533 >> 172.21.15.162:0/426062533 conn(0x7f0f08071d00 msgr2=0x7f0f08072100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:06:14.933 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:14.932+0000 7f0f0e2db700 1 -- 172.21.15.162:0/426062533 shutdown_connections 2024-04-27T04:06:14.933 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:14.932+0000 7f0f0e2db700 1 -- 172.21.15.162:0/426062533 wait complete. 2024-04-27T04:06:14.933 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:14.933+0000 7f0f0e2db700 1 Processor -- start 2024-04-27T04:06:14.935 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:14.933+0000 7f0f0e2db700 1 -- start start 2024-04-27T04:06:14.935 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:14.933+0000 7f0f0e2db700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0f08065640 0x7f0f08113b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:06:14.935 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:14.933+0000 7f0f0e2db700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0f08102e50 0x7f0f081140c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:06:14.935 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:14.933+0000 7f0f0e2db700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f081147a0 con 0x7f0f08102e50 2024-04-27T04:06:14.935 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:14.933+0000 7f0f0e2db700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f0810db00 con 0x7f0f08065640 2024-04-27T04:06:14.935 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:14.933+0000 7f0f077fe700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0f08102e50 0x7f0f081140c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:06:14.935 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:14.933+0000 7f0f077fe700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0f08102e50 0x7f0f081140c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:47048/0 (socket says 172.21.15.162:47048) 2024-04-27T04:06:14.936 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:14.933+0000 7f0f077fe700 1 -- 172.21.15.162:0/3698501744 learned_addr learned my addr 172.21.15.162:0/3698501744 (peer_addr_for_me v2:172.21.15.162:0/0) 2024-04-27T04:06:14.936 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:14.933+0000 7f0f07fff700 1 --2- 172.21.15.162:0/3698501744 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0f08065640 0x7f0f08113b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:06:14.936 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:14.933+0000 7f0f077fe700 1 -- 172.21.15.162:0/3698501744 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0f08065640 msgr2=0x7f0f08113b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:06:14.936 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:14.933+0000 7f0f077fe700 1 --2- 172.21.15.162:0/3698501744 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0f08065640 0x7f0f08113b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:06:14.936 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:14.933+0000 7f0f077fe700 1 -- 172.21.15.162:0/3698501744 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ef8005040 con 0x7f0f08102e50 2024-04-27T04:06:14.936 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:14.934+0000 7f0f077fe700 1 --2- 172.21.15.162:0/3698501744 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0f08102e50 0x7f0f081140c0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f0efc00b790 tx=0x7f0efc009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:06:14.936 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:14.934+0000 7f0f04ff9700 1 -- 172.21.15.162:0/3698501744 <== mon.0 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0efc00f6f0 con 0x7f0f08102e50 2024-04-27T04:06:14.936 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:14.934+0000 7f0f04ff9700 1 -- 172.21.15.162:0/3698501744 <== mon.0 v2:172.21.15.162:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f0efc00fd20 con 0x7f0f08102e50 2024-04-27T04:06:14.936 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:14.934+0000 7f0f0e2db700 1 -- 172.21.15.162:0/3698501744 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0f0810dde0 con 0x7f0f08102e50 2024-04-27T04:06:14.936 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:14.934+0000 7f0f04ff9700 1 -- 172.21.15.162:0/3698501744 <== mon.0 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0efc0103a0 con 0x7f0f08102e50 2024-04-27T04:06:14.936 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:14.934+0000 7f0f0e2db700 1 -- 172.21.15.162:0/3698501744 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0f0810e3a0 con 0x7f0f08102e50 2024-04-27T04:06:14.936 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:14.935+0000 7f0f0e2db700 1 -- 172.21.15.162:0/3698501744 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0f080fe540 con 0x7f0f08102e50 2024-04-27T04:06:14.936 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:14.935+0000 7f0f04ff9700 1 -- 172.21.15.162:0/3698501744 <== mon.0 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 19) v1 ==== 86631+0+0 (secure 0 0 0) 0x7f0efc011400 con 0x7f0f08102e50 2024-04-27T04:06:14.937 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:14.935+0000 7f0f04ff9700 1 --2- 172.21.15.162:0/3698501744 >> [v2:172.21.15.162:6800/4079782599,v1:172.21.15.162:6801/4079782599] conn(0x7f0ef00692a0 0x7f0ef006b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:06:14.937 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:14.936+0000 7f0f04ff9700 1 -- 172.21.15.162:0/3698501744 <== mon.0 v2:172.21.15.162:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4634+0+0 (secure 0 0 0) 0x7f0efc086c40 con 0x7f0f08102e50 2024-04-27T04:06:14.937 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:14.936+0000 7f0f07fff700 1 --2- 172.21.15.162:0/3698501744 >> [v2:172.21.15.162:6800/4079782599,v1:172.21.15.162:6801/4079782599] conn(0x7f0ef00692a0 0x7f0ef006b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:06:14.937 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:14.936+0000 7f0f07fff700 1 --2- 172.21.15.162:0/3698501744 >> [v2:172.21.15.162:6800/4079782599,v1:172.21.15.162:6801/4079782599] conn(0x7f0ef00692a0 0x7f0ef006b750 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f0ef8004f90 tx=0x7f0ef8009e30 comp rx=0 tx=0).ready entity=mgr.14210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:06:14.940 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:14.940+0000 7f0f04ff9700 1 -- 172.21.15.162:0/3698501744 <== mon.0 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f0efc056d40 con 0x7f0f08102e50 2024-04-27T04:06:15.133 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:15.132+0000 7f0f0e2db700 1 -- 172.21.15.162:0/3698501744 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f0f0805f5c0 con 0x7f0f08102e50 2024-04-27T04:06:15.133 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:15.133+0000 7f0f04ff9700 1 -- 172.21.15.162:0/3698501744 <== mon.0 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f0efc0568d0 con 0x7f0f08102e50 2024-04-27T04:06:15.133 INFO:teuthology.orchestra.run.smithi162.stdout:60129542175 2024-04-27T04:06:15.136 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:15.135+0000 7f0f0e2db700 1 -- 172.21.15.162:0/3698501744 >> [v2:172.21.15.162:6800/4079782599,v1:172.21.15.162:6801/4079782599] conn(0x7f0ef00692a0 msgr2=0x7f0ef006b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:06:15.136 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:15.135+0000 7f0f0e2db700 1 --2- 172.21.15.162:0/3698501744 >> [v2:172.21.15.162:6800/4079782599,v1:172.21.15.162:6801/4079782599] conn(0x7f0ef00692a0 0x7f0ef006b750 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f0ef8004f90 tx=0x7f0ef8009e30 comp rx=0 tx=0).stop 2024-04-27T04:06:15.136 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:15.135+0000 7f0f0e2db700 1 -- 172.21.15.162:0/3698501744 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0f08102e50 msgr2=0x7f0f081140c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:06:15.136 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:15.135+0000 7f0f0e2db700 1 --2- 172.21.15.162:0/3698501744 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0f08102e50 0x7f0f081140c0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f0efc00b790 tx=0x7f0efc009250 comp rx=0 tx=0).stop 2024-04-27T04:06:15.136 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:15.135+0000 7f0f0e2db700 1 -- 172.21.15.162:0/3698501744 shutdown_connections 2024-04-27T04:06:15.136 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:15.135+0000 7f0f0e2db700 1 --2- 172.21.15.162:0/3698501744 >> [v2:172.21.15.162:6800/4079782599,v1:172.21.15.162:6801/4079782599] conn(0x7f0ef00692a0 0x7f0ef006b750 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:06:15.136 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:15.135+0000 7f0f0e2db700 1 --2- 172.21.15.162:0/3698501744 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0f08065640 0x7f0f08113b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:06:15.137 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:15.135+0000 7f0f0e2db700 1 --2- 172.21.15.162:0/3698501744 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0f08102e50 0x7f0f081140c0 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:06:15.137 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:15.135+0000 7f0f0e2db700 1 -- 172.21.15.162:0/3698501744 >> 172.21.15.162:0/3698501744 conn(0x7f0f08071d00 msgr2=0x7f0f080fa0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:06:15.137 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:15.136+0000 7f0f0e2db700 1 -- 172.21.15.162:0/3698501744 shutdown_connections 2024-04-27T04:06:15.137 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:15.136+0000 7f0f0e2db700 1 -- 172.21.15.162:0/3698501744 wait complete. 2024-04-27T04:06:15.149 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:15.149+0000 7f959a8d3700 1 -- 172.21.15.162:0/369792294 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9594102de0 msgr2=0x7f95941031b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:06:15.150 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:15.149+0000 7f959a8d3700 1 --2- 172.21.15.162:0/369792294 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9594102de0 0x7f95941031b0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f9584004940 tx=0x7f9584009e30 comp rx=0 tx=0).stop 2024-04-27T04:06:15.150 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:15.149+0000 7f959a8d3700 1 -- 172.21.15.162:0/369792294 shutdown_connections 2024-04-27T04:06:15.150 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:15.149+0000 7f959a8d3700 1 --2- 172.21.15.162:0/369792294 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9594103780 0x7f95940fcdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:06:15.150 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:15.149+0000 7f959a8d3700 1 --2- 172.21.15.162:0/369792294 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9594102de0 0x7f95941031b0 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:06:15.150 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:15.149+0000 7f959a8d3700 1 -- 172.21.15.162:0/369792294 >> 172.21.15.162:0/369792294 conn(0x7f95940f8990 msgr2=0x7f95940fada0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:06:15.150 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:15.149+0000 7f959a8d3700 1 -- 172.21.15.162:0/369792294 shutdown_connections 2024-04-27T04:06:15.150 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:15.149+0000 7f959a8d3700 1 -- 172.21.15.162:0/369792294 wait complete. 2024-04-27T04:06:15.152 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:15.150+0000 7f959a8d3700 1 Processor -- start 2024-04-27T04:06:15.152 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:15.150+0000 7f959a8d3700 1 -- start start 2024-04-27T04:06:15.152 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:15.150+0000 7f959a8d3700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9594102de0 0x7f9594115bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:06:15.152 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:15.150+0000 7f959a8d3700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9594103780 0x7f95941160f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:06:15.152 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:15.150+0000 7f959a8d3700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9594116740 con 0x7f9594103780 2024-04-27T04:06:15.152 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:15.150+0000 7f959a8d3700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f959410fd20 con 0x7f9594102de0 2024-04-27T04:06:15.152 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:15.150+0000 7f95937fe700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9594103780 0x7f95941160f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:06:15.152 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:15.150+0000 7f95937fe700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9594103780 0x7f95941160f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:47074/0 (socket says 172.21.15.162:47074) 2024-04-27T04:06:15.152 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:15.150+0000 7f95937fe700 1 -- 172.21.15.162:0/3645364511 learned_addr learned my addr 172.21.15.162:0/3645364511 (peer_addr_for_me v2:172.21.15.162:0/0) 2024-04-27T04:06:15.153 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:15.150+0000 7f9593fff700 1 --2- 172.21.15.162:0/3645364511 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9594102de0 0x7f9594115bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:06:15.153 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:15.151+0000 7f95937fe700 1 -- 172.21.15.162:0/3645364511 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9594102de0 msgr2=0x7f9594115bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:06:15.153 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:15.151+0000 7f95937fe700 1 --2- 172.21.15.162:0/3645364511 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9594102de0 0x7f9594115bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:06:15.153 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:15.151+0000 7f95937fe700 1 -- 172.21.15.162:0/3645364511 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9584005040 con 0x7f9594103780 2024-04-27T04:06:15.153 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:15.151+0000 7f9593fff700 1 --2- 172.21.15.162:0/3645364511 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9594102de0 0x7f9594115bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-27T04:06:15.153 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:15.151+0000 7f95937fe700 1 --2- 172.21.15.162:0/3645364511 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9594103780 0x7f95941160f0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f9588002a80 tx=0x7f9588004630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:06:15.153 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:15.151+0000 7f9590ff9700 1 -- 172.21.15.162:0/3645364511 <== mon.0 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9588007cc0 con 0x7f9594103780 2024-04-27T04:06:15.153 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:15.151+0000 7f9590ff9700 1 -- 172.21.15.162:0/3645364511 <== mon.0 v2:172.21.15.162:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f958800f040 con 0x7f9594103780 2024-04-27T04:06:15.153 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:15.151+0000 7f959a8d3700 1 -- 172.21.15.162:0/3645364511 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9594110000 con 0x7f9594103780 2024-04-27T04:06:15.153 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:15.151+0000 7f9590ff9700 1 -- 172.21.15.162:0/3645364511 <== mon.0 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f958800daf0 con 0x7f9594103780 2024-04-27T04:06:15.154 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:15.151+0000 7f959a8d3700 1 -- 172.21.15.162:0/3645364511 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f95941105c0 con 0x7f9594103780 2024-04-27T04:06:15.155 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:15.152+0000 7f959a8d3700 1 -- 172.21.15.162:0/3645364511 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9594106a30 con 0x7f9594103780 2024-04-27T04:06:15.156 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:15.152+0000 7f9590ff9700 1 -- 172.21.15.162:0/3645364511 <== mon.0 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 19) v1 ==== 86631+0+0 (secure 0 0 0) 0x7f958800dc90 con 0x7f9594103780 2024-04-27T04:06:15.156 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:15.153+0000 7f9590ff9700 1 --2- 172.21.15.162:0/3645364511 >> [v2:172.21.15.162:6800/4079782599,v1:172.21.15.162:6801/4079782599] conn(0x7f957c0692a0 0x7f957c06b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:06:15.156 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:15.153+0000 7f9590ff9700 1 -- 172.21.15.162:0/3645364511 <== mon.0 v2:172.21.15.162:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4634+0+0 (secure 0 0 0) 0x7f9588086560 con 0x7f9594103780 2024-04-27T04:06:15.156 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:15.153+0000 7f9593fff700 1 --2- 172.21.15.162:0/3645364511 >> [v2:172.21.15.162:6800/4079782599,v1:172.21.15.162:6801/4079782599] conn(0x7f957c0692a0 0x7f957c06b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:06:15.156 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:15.153+0000 7f9593fff700 1 --2- 172.21.15.162:0/3645364511 >> [v2:172.21.15.162:6800/4079782599,v1:172.21.15.162:6801/4079782599] conn(0x7f957c0692a0 0x7f957c06b750 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f9584004560 tx=0x7f9584009e30 comp rx=0 tx=0).ready entity=mgr.14210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:06:15.157 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:15.157+0000 7f9590ff9700 1 -- 172.21.15.162:0/3645364511 <== mon.0 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f958804d250 con 0x7f9594103780 2024-04-27T04:06:15.357 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:15.356+0000 7f959a8d3700 1 -- 172.21.15.162:0/3645364511 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) v1 -- 0x7f95940fe530 con 0x7f9594103780 2024-04-27T04:06:15.357 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:15.357+0000 7f9590ff9700 1 -- 172.21.15.162:0/3645364511 <== mon.0 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f95880565e0 con 0x7f9594103780 2024-04-27T04:06:15.357 INFO:teuthology.orchestra.run.smithi162.stdout:146028888079 2024-04-27T04:06:15.359 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:15.359+0000 7f959a8d3700 1 -- 172.21.15.162:0/3645364511 >> [v2:172.21.15.162:6800/4079782599,v1:172.21.15.162:6801/4079782599] conn(0x7f957c0692a0 msgr2=0x7f957c06b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:06:15.359 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:15.359+0000 7f959a8d3700 1 --2- 172.21.15.162:0/3645364511 >> [v2:172.21.15.162:6800/4079782599,v1:172.21.15.162:6801/4079782599] conn(0x7f957c0692a0 0x7f957c06b750 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f9584004560 tx=0x7f9584009e30 comp rx=0 tx=0).stop 2024-04-27T04:06:15.359 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:15.359+0000 7f959a8d3700 1 -- 172.21.15.162:0/3645364511 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9594103780 msgr2=0x7f95941160f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:06:15.359 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:15.359+0000 7f959a8d3700 1 --2- 172.21.15.162:0/3645364511 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9594103780 0x7f95941160f0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f9588002a80 tx=0x7f9588004630 comp rx=0 tx=0).stop 2024-04-27T04:06:15.360 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:15.359+0000 7f959a8d3700 1 -- 172.21.15.162:0/3645364511 shutdown_connections 2024-04-27T04:06:15.360 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:15.359+0000 7f959a8d3700 1 --2- 172.21.15.162:0/3645364511 >> [v2:172.21.15.162:6800/4079782599,v1:172.21.15.162:6801/4079782599] conn(0x7f957c0692a0 0x7f957c06b750 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:06:15.360 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:15.359+0000 7f959a8d3700 1 --2- 172.21.15.162:0/3645364511 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9594102de0 0x7f9594115bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:06:15.360 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:15.359+0000 7f959a8d3700 1 --2- 172.21.15.162:0/3645364511 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9594103780 0x7f95941160f0 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:06:15.360 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:15.359+0000 7f959a8d3700 1 -- 172.21.15.162:0/3645364511 >> 172.21.15.162:0/3645364511 conn(0x7f95940f8990 msgr2=0x7f9594107380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:06:15.360 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:15.359+0000 7f959a8d3700 1 -- 172.21.15.162:0/3645364511 shutdown_connections 2024-04-27T04:06:15.360 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:15.359+0000 7f959a8d3700 1 -- 172.21.15.162:0/3645364511 wait complete. 2024-04-27T04:06:15.863 INFO:tasks.cephadm.ceph_manager.ceph:need seq 60129542173 got 60129542175 for osd.1 2024-04-27T04:06:15.864 DEBUG:teuthology.parallel:result is None 2024-04-27T04:06:15.882 INFO:teuthology.orchestra.run.smithi162.stderr:Inferring config /var/lib/ceph/7f9750d6-044a-11ef-bc93-c7b262605968/mon.smithi162/config 2024-04-27T04:06:16.137 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:06:15 smithi162 ceph-mon[26841]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-27T04:06:16.138 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:06:15 smithi162 ceph-mon[26841]: from='client.? 172.21.15.162:0/3698501744' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-04-27T04:06:16.138 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:06:15 smithi162 ceph-mon[26841]: from='client.? 172.21.15.162:0/3645364511' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-04-27T04:06:16.423 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:06:15 smithi181 ceph-mon[33774]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-27T04:06:16.423 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:06:15 smithi181 ceph-mon[33774]: from='client.? 172.21.15.162:0/3698501744' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-04-27T04:06:16.423 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:06:15 smithi181 ceph-mon[33774]: from='client.? 172.21.15.162:0/3645364511' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-04-27T04:06:16.589 INFO:teuthology.orchestra.run.smithi162.stderr:Inferring config /var/lib/ceph/7f9750d6-044a-11ef-bc93-c7b262605968/mon.smithi162/config 2024-04-27T04:06:17.261 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:06:17 smithi162 ceph-mon[26841]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-27T04:06:17.423 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:06:17 smithi181 ceph-mon[33774]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-27T04:06:18.195 INFO:tasks.cephadm.ceph_manager.ceph:need seq 146028888076 got 146028888079 for osd.4 2024-04-27T04:06:18.195 DEBUG:teuthology.parallel:result is None 2024-04-27T04:06:18.288 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:06:18 smithi162 ceph-mon[26841]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-27T04:06:18.423 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:06:18 smithi181 ceph-mon[33774]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-27T04:06:18.578 INFO:teuthology.orchestra.run.smithi162.stderr:Inferring config /var/lib/ceph/7f9750d6-044a-11ef-bc93-c7b262605968/mon.smithi162/config 2024-04-27T04:06:18.777 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:18.776+0000 7fc7771e7700 1 -- 172.21.15.162:0/1071874899 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc770102680 msgr2=0x7fc7700fbcf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:06:18.777 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:18.776+0000 7fc7771e7700 1 --2- 172.21.15.162:0/1071874899 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc770102680 0x7fc7700fbcf0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fc75c004940 tx=0x7fc75c009e30 comp rx=0 tx=0).stop 2024-04-27T04:06:18.777 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:18.776+0000 7fc7771e7700 1 -- 172.21.15.162:0/1071874899 shutdown_connections 2024-04-27T04:06:18.777 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:18.776+0000 7fc7771e7700 1 --2- 172.21.15.162:0/1071874899 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc770102680 0x7fc7700fbcf0 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:06:18.777 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:18.776+0000 7fc7771e7700 1 --2- 172.21.15.162:0/1071874899 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc770101ce0 0x7fc7701020b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:06:18.777 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:18.776+0000 7fc7771e7700 1 -- 172.21.15.162:0/1071874899 >> 172.21.15.162:0/1071874899 conn(0x7fc7700f7830 msgr2=0x7fc7700f9c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:06:18.777 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:18.776+0000 7fc7771e7700 1 -- 172.21.15.162:0/1071874899 shutdown_connections 2024-04-27T04:06:18.777 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:18.776+0000 7fc7771e7700 1 -- 172.21.15.162:0/1071874899 wait complete. 2024-04-27T04:06:18.777 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:18.777+0000 7fc7771e7700 1 Processor -- start 2024-04-27T04:06:18.777 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:18.777+0000 7fc7771e7700 1 -- start start 2024-04-27T04:06:18.779 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:18.777+0000 7fc7771e7700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc770101ce0 0x7fc770102b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:06:18.779 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:18.777+0000 7fc7771e7700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc770102680 0x7fc770103080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:06:18.779 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:18.777+0000 7fc7771e7700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc770104b20 con 0x7fc770101ce0 2024-04-27T04:06:18.779 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:18.777+0000 7fc7771e7700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7701035c0 con 0x7fc770102680 2024-04-27T04:06:18.779 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:18.777+0000 7fc774f83700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc770101ce0 0x7fc770102b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:06:18.779 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:18.777+0000 7fc774f83700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc770101ce0 0x7fc770102b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:47094/0 (socket says 172.21.15.162:47094) 2024-04-27T04:06:18.779 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:18.777+0000 7fc774f83700 1 -- 172.21.15.162:0/2458541794 learned_addr learned my addr 172.21.15.162:0/2458541794 (peer_addr_for_me v2:172.21.15.162:0/0) 2024-04-27T04:06:18.779 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:18.777+0000 7fc774f83700 1 -- 172.21.15.162:0/2458541794 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc770102680 msgr2=0x7fc770103080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:06:18.779 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:18.777+0000 7fc774f83700 1 --2- 172.21.15.162:0/2458541794 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc770102680 0x7fc770103080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:06:18.779 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:18.777+0000 7fc774f83700 1 -- 172.21.15.162:0/2458541794 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc75c005040 con 0x7fc770101ce0 2024-04-27T04:06:18.780 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:18.778+0000 7fc774f83700 1 --2- 172.21.15.162:0/2458541794 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc770101ce0 0x7fc770102b40 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fc76400b790 tx=0x7fc764009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:06:18.780 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:18.778+0000 7fc76d7fa700 1 -- 172.21.15.162:0/2458541794 <== mon.0 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc76400f770 con 0x7fc770101ce0 2024-04-27T04:06:18.780 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:18.778+0000 7fc7771e7700 1 -- 172.21.15.162:0/2458541794 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc7701038a0 con 0x7fc770101ce0 2024-04-27T04:06:18.780 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:18.778+0000 7fc76d7fa700 1 -- 172.21.15.162:0/2458541794 <== mon.0 v2:172.21.15.162:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fc76400fda0 con 0x7fc770101ce0 2024-04-27T04:06:18.780 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:18.778+0000 7fc76d7fa700 1 -- 172.21.15.162:0/2458541794 <== mon.0 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc76400d310 con 0x7fc770101ce0 2024-04-27T04:06:18.780 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:18.778+0000 7fc7771e7700 1 -- 172.21.15.162:0/2458541794 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc77011a1f0 con 0x7fc770101ce0 2024-04-27T04:06:18.780 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:18.779+0000 7fc7771e7700 1 -- 172.21.15.162:0/2458541794 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc7700fd430 con 0x7fc770101ce0 2024-04-27T04:06:18.781 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:18.779+0000 7fc76d7fa700 1 -- 172.21.15.162:0/2458541794 <== mon.0 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 19) v1 ==== 86631+0+0 (secure 0 0 0) 0x7fc764010420 con 0x7fc770101ce0 2024-04-27T04:06:18.781 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:18.780+0000 7fc76d7fa700 1 --2- 172.21.15.162:0/2458541794 >> [v2:172.21.15.162:6800/4079782599,v1:172.21.15.162:6801/4079782599] conn(0x7fc7600692a0 0x7fc76006b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:06:18.781 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:18.780+0000 7fc76d7fa700 1 -- 172.21.15.162:0/2458541794 <== mon.0 v2:172.21.15.162:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4634+0+0 (secure 0 0 0) 0x7fc764086e80 con 0x7fc770101ce0 2024-04-27T04:06:18.781 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:18.780+0000 7fc76ffff700 1 --2- 172.21.15.162:0/2458541794 >> [v2:172.21.15.162:6800/4079782599,v1:172.21.15.162:6801/4079782599] conn(0x7fc7600692a0 0x7fc76006b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:06:18.781 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:18.780+0000 7fc76ffff700 1 --2- 172.21.15.162:0/2458541794 >> [v2:172.21.15.162:6800/4079782599,v1:172.21.15.162:6801/4079782599] conn(0x7fc7600692a0 0x7fc76006b750 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fc75c004f90 tx=0x7fc75c009e30 comp rx=0 tx=0).ready entity=mgr.14210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:06:18.784 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:18.784+0000 7fc76d7fa700 1 -- 172.21.15.162:0/2458541794 <== mon.0 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7fc7640534f0 con 0x7fc770101ce0 2024-04-27T04:06:18.975 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:18.974+0000 7fc7771e7700 1 -- 172.21.15.162:0/2458541794 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) v1 -- 0x7fc77011a3e0 con 0x7fc770101ce0 2024-04-27T04:06:18.975 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:18.975+0000 7fc76d7fa700 1 -- 172.21.15.162:0/2458541794 <== mon.0 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fc764056b10 con 0x7fc770101ce0 2024-04-27T04:06:18.975 INFO:teuthology.orchestra.run.smithi162.stdout:167503724553 2024-04-27T04:06:18.977 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:18.977+0000 7fc7771e7700 1 -- 172.21.15.162:0/2458541794 >> [v2:172.21.15.162:6800/4079782599,v1:172.21.15.162:6801/4079782599] conn(0x7fc7600692a0 msgr2=0x7fc76006b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:06:18.977 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:18.977+0000 7fc7771e7700 1 --2- 172.21.15.162:0/2458541794 >> [v2:172.21.15.162:6800/4079782599,v1:172.21.15.162:6801/4079782599] conn(0x7fc7600692a0 0x7fc76006b750 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fc75c004f90 tx=0x7fc75c009e30 comp rx=0 tx=0).stop 2024-04-27T04:06:18.977 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:18.977+0000 7fc7771e7700 1 -- 172.21.15.162:0/2458541794 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc770101ce0 msgr2=0x7fc770102b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:06:18.977 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:18.977+0000 7fc7771e7700 1 --2- 172.21.15.162:0/2458541794 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc770101ce0 0x7fc770102b40 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fc76400b790 tx=0x7fc764009250 comp rx=0 tx=0).stop 2024-04-27T04:06:18.977 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:18.977+0000 7fc7771e7700 1 -- 172.21.15.162:0/2458541794 shutdown_connections 2024-04-27T04:06:18.978 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:18.977+0000 7fc7771e7700 1 --2- 172.21.15.162:0/2458541794 >> [v2:172.21.15.162:6800/4079782599,v1:172.21.15.162:6801/4079782599] conn(0x7fc7600692a0 0x7fc76006b750 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:06:18.978 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:18.977+0000 7fc7771e7700 1 --2- 172.21.15.162:0/2458541794 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc770101ce0 0x7fc770102b40 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:06:18.978 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:18.977+0000 7fc7771e7700 1 --2- 172.21.15.162:0/2458541794 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc770102680 0x7fc770103080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:06:18.978 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:18.977+0000 7fc7771e7700 1 -- 172.21.15.162:0/2458541794 >> 172.21.15.162:0/2458541794 conn(0x7fc7700f7830 msgr2=0x7fc770066280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:06:18.978 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:18.977+0000 7fc7771e7700 1 -- 172.21.15.162:0/2458541794 shutdown_connections 2024-04-27T04:06:18.978 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:18.977+0000 7fc7771e7700 1 -- 172.21.15.162:0/2458541794 wait complete. 2024-04-27T04:06:19.304 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:06:19 smithi162 ceph-mon[26841]: from='client.? 172.21.15.162:0/2458541794' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-04-27T04:06:19.423 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:06:19 smithi181 ceph-mon[33774]: from='client.? 172.21.15.162:0/2458541794' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-04-27T04:06:20.165 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:20.166+0000 7fc0f7195700 1 -- 172.21.15.162:0/1551095760 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc0f00fbd40 msgr2=0x7fc0f00fc1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:06:20.165 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:20.166+0000 7fc0f7195700 1 --2- 172.21.15.162:0/1551095760 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc0f00fbd40 0x7fc0f00fc1b0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fc0e0004940 tx=0x7fc0e0009e30 comp rx=0 tx=0).stop 2024-04-27T04:06:20.166 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:20.166+0000 7fc0f7195700 1 -- 172.21.15.162:0/1551095760 shutdown_connections 2024-04-27T04:06:20.166 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:20.166+0000 7fc0f7195700 1 --2- 172.21.15.162:0/1551095760 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc0f00fbd40 0x7fc0f00fc1b0 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:06:20.166 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:20.166+0000 7fc0f7195700 1 --2- 172.21.15.162:0/1551095760 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc0f0101d90 0x7fc0f0102160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:06:20.166 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:20.166+0000 7fc0f7195700 1 -- 172.21.15.162:0/1551095760 >> 172.21.15.162:0/1551095760 conn(0x7fc0f0071d00 msgr2=0x7fc0f0072100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:06:20.166 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:20.166+0000 7fc0f7195700 1 -- 172.21.15.162:0/1551095760 shutdown_connections 2024-04-27T04:06:20.166 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:06:20 smithi162 ceph-mon[26841]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-27T04:06:20.168 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:20.166+0000 7fc0f7195700 1 -- 172.21.15.162:0/1551095760 wait complete. 2024-04-27T04:06:20.168 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:20.166+0000 7fc0f7195700 1 Processor -- start 2024-04-27T04:06:20.168 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:20.167+0000 7fc0f7195700 1 -- start start 2024-04-27T04:06:20.168 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:20.167+0000 7fc0f7195700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc0f00fbd40 0x7fc0f010ecb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:06:20.168 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:20.167+0000 7fc0f7195700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc0f0101d90 0x7fc0f010f1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:06:20.168 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:20.167+0000 7fc0f7195700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0f010f730 con 0x7fc0f00fbd40 2024-04-27T04:06:20.168 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:20.167+0000 7fc0f7195700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0f010f870 con 0x7fc0f0101d90 2024-04-27T04:06:20.168 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:20.167+0000 7fc0f4f31700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc0f00fbd40 0x7fc0f010ecb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:06:20.168 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:20.167+0000 7fc0f4f31700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc0f00fbd40 0x7fc0f010ecb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:47116/0 (socket says 172.21.15.162:47116) 2024-04-27T04:06:20.168 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:20.167+0000 7fc0f4f31700 1 -- 172.21.15.162:0/4105291720 learned_addr learned my addr 172.21.15.162:0/4105291720 (peer_addr_for_me v2:172.21.15.162:0/0) 2024-04-27T04:06:20.168 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:20.167+0000 7fc0e7fff700 1 --2- 172.21.15.162:0/4105291720 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc0f0101d90 0x7fc0f010f1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:06:20.168 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:20.167+0000 7fc0f4f31700 1 -- 172.21.15.162:0/4105291720 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc0f0101d90 msgr2=0x7fc0f010f1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:06:20.168 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:20.167+0000 7fc0f4f31700 1 --2- 172.21.15.162:0/4105291720 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc0f0101d90 0x7fc0f010f1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:06:20.169 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:20.167+0000 7fc0f4f31700 1 -- 172.21.15.162:0/4105291720 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc0e0005040 con 0x7fc0f00fbd40 2024-04-27T04:06:20.169 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:20.167+0000 7fc0f4f31700 1 --2- 172.21.15.162:0/4105291720 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc0f00fbd40 0x7fc0f010ecb0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fc0ec00c830 tx=0x7fc0ec004470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:06:20.169 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:20.168+0000 7fc0e57fa700 1 -- 172.21.15.162:0/4105291720 <== mon.0 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc0ec00a760 con 0x7fc0f00fbd40 2024-04-27T04:06:20.169 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:20.168+0000 7fc0f7195700 1 -- 172.21.15.162:0/4105291720 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc0f0117b60 con 0x7fc0f00fbd40 2024-04-27T04:06:20.169 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:20.168+0000 7fc0e57fa700 1 -- 172.21.15.162:0/4105291720 <== mon.0 v2:172.21.15.162:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fc0ec00ad90 con 0x7fc0f00fbd40 2024-04-27T04:06:20.169 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:20.168+0000 7fc0e57fa700 1 -- 172.21.15.162:0/4105291720 <== mon.0 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc0ec0113a0 con 0x7fc0f00fbd40 2024-04-27T04:06:20.169 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:20.168+0000 7fc0f7195700 1 -- 172.21.15.162:0/4105291720 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc0f0118120 con 0x7fc0f00fbd40 2024-04-27T04:06:20.169 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:20.169+0000 7fc0f7195700 1 -- 172.21.15.162:0/4105291720 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc0f00fd430 con 0x7fc0f00fbd40 2024-04-27T04:06:20.170 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:20.169+0000 7fc0e57fa700 1 -- 172.21.15.162:0/4105291720 <== mon.0 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 19) v1 ==== 86631+0+0 (secure 0 0 0) 0x7fc0ec012400 con 0x7fc0f00fbd40 2024-04-27T04:06:20.170 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:20.170+0000 7fc0e57fa700 1 --2- 172.21.15.162:0/4105291720 >> [v2:172.21.15.162:6800/4079782599,v1:172.21.15.162:6801/4079782599] conn(0x7fc0d80692a0 0x7fc0d806b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:06:20.170 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:20.170+0000 7fc0e57fa700 1 -- 172.21.15.162:0/4105291720 <== mon.0 v2:172.21.15.162:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4634+0+0 (secure 0 0 0) 0x7fc0ec086df0 con 0x7fc0f00fbd40 2024-04-27T04:06:20.170 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:20.170+0000 7fc0e7fff700 1 --2- 172.21.15.162:0/4105291720 >> [v2:172.21.15.162:6800/4079782599,v1:172.21.15.162:6801/4079782599] conn(0x7fc0d80692a0 0x7fc0d806b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:06:20.170 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:20.170+0000 7fc0e7fff700 1 --2- 172.21.15.162:0/4105291720 >> [v2:172.21.15.162:6800/4079782599,v1:172.21.15.162:6801/4079782599] conn(0x7fc0d80692a0 0x7fc0d806b750 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fc0e000b3b0 tx=0x7fc0e0009e30 comp rx=0 tx=0).ready entity=mgr.14210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:06:20.173 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:20.174+0000 7fc0e57fa700 1 -- 172.21.15.162:0/4105291720 <== mon.0 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7fc0ec053500 con 0x7fc0f00fbd40 2024-04-27T04:06:20.366 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:20.366+0000 7fc0f7195700 1 -- 172.21.15.162:0/4105291720 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7fc0f005f5c0 con 0x7fc0f00fbd40 2024-04-27T04:06:20.366 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:20.367+0000 7fc0e57fa700 1 -- 172.21.15.162:0/4105291720 <== mon.0 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fc0ec056b20 con 0x7fc0f00fbd40 2024-04-27T04:06:20.366 INFO:teuthology.orchestra.run.smithi162.stdout:111669149717 2024-04-27T04:06:20.368 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:20.369+0000 7fc0f7195700 1 -- 172.21.15.162:0/4105291720 >> [v2:172.21.15.162:6800/4079782599,v1:172.21.15.162:6801/4079782599] conn(0x7fc0d80692a0 msgr2=0x7fc0d806b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:06:20.368 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:20.369+0000 7fc0f7195700 1 --2- 172.21.15.162:0/4105291720 >> [v2:172.21.15.162:6800/4079782599,v1:172.21.15.162:6801/4079782599] conn(0x7fc0d80692a0 0x7fc0d806b750 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fc0e000b3b0 tx=0x7fc0e0009e30 comp rx=0 tx=0).stop 2024-04-27T04:06:20.368 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:20.369+0000 7fc0f7195700 1 -- 172.21.15.162:0/4105291720 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc0f00fbd40 msgr2=0x7fc0f010ecb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:06:20.368 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:20.369+0000 7fc0f7195700 1 --2- 172.21.15.162:0/4105291720 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc0f00fbd40 0x7fc0f010ecb0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fc0ec00c830 tx=0x7fc0ec004470 comp rx=0 tx=0).stop 2024-04-27T04:06:20.368 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:20.369+0000 7fc0f7195700 1 -- 172.21.15.162:0/4105291720 shutdown_connections 2024-04-27T04:06:20.369 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:20.369+0000 7fc0f7195700 1 --2- 172.21.15.162:0/4105291720 >> [v2:172.21.15.162:6800/4079782599,v1:172.21.15.162:6801/4079782599] conn(0x7fc0d80692a0 0x7fc0d806b750 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:06:20.369 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:20.369+0000 7fc0f7195700 1 --2- 172.21.15.162:0/4105291720 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc0f00fbd40 0x7fc0f010ecb0 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:06:20.369 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:20.369+0000 7fc0f7195700 1 --2- 172.21.15.162:0/4105291720 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc0f0101d90 0x7fc0f010f1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:06:20.369 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:20.369+0000 7fc0f7195700 1 -- 172.21.15.162:0/4105291720 >> 172.21.15.162:0/4105291720 conn(0x7fc0f0071d00 msgr2=0x7fc0f00fa0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:06:20.369 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:20.369+0000 7fc0f7195700 1 -- 172.21.15.162:0/4105291720 shutdown_connections 2024-04-27T04:06:20.369 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:20.369+0000 7fc0f7195700 1 -- 172.21.15.162:0/4105291720 wait complete. 2024-04-27T04:06:20.423 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:06:20 smithi181 ceph-mon[33774]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-27T04:06:21.387 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:06:21 smithi162 ceph-mon[26841]: from='client.? 172.21.15.162:0/4105291720' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-04-27T04:06:21.423 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:06:21 smithi181 ceph-mon[33774]: from='client.? 172.21.15.162:0/4105291720' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-04-27T04:06:21.475 INFO:tasks.cephadm.ceph_manager.ceph:need seq 111669149715 got 111669149717 for osd.3 2024-04-27T04:06:21.475 DEBUG:teuthology.parallel:result is None 2024-04-27T04:06:21.846 INFO:tasks.cephadm.ceph_manager.ceph:need seq 167503724551 got 167503724553 for osd.5 2024-04-27T04:06:21.846 DEBUG:teuthology.parallel:result is None 2024-04-27T04:06:22.137 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:06:22 smithi162 ceph-mon[26841]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-27T04:06:22.423 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:06:22 smithi181 ceph-mon[33774]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-27T04:06:23.087 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:23.084+0000 7f5da1d5d700 1 -- 172.21.15.162:0/4034827541 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5d9c1026b0 msgr2=0x7f5d9c0fbd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:06:23.087 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:23.084+0000 7f5da1d5d700 1 --2- 172.21.15.162:0/4034827541 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5d9c1026b0 0x7f5d9c0fbd20 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f5d90004940 tx=0x7f5d90009e30 comp rx=0 tx=0).stop 2024-04-27T04:06:23.087 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:23.084+0000 7f5da1d5d700 1 -- 172.21.15.162:0/4034827541 shutdown_connections 2024-04-27T04:06:23.087 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:23.084+0000 7f5da1d5d700 1 --2- 172.21.15.162:0/4034827541 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5d9c1026b0 0x7f5d9c0fbd20 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:06:23.088 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:23.084+0000 7f5da1d5d700 1 --2- 172.21.15.162:0/4034827541 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5d9c101d10 0x7f5d9c1020e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:06:23.088 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:23.084+0000 7f5da1d5d700 1 -- 172.21.15.162:0/4034827541 >> 172.21.15.162:0/4034827541 conn(0x7f5d9c0f7860 msgr2=0x7f5d9c0f9c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:06:23.088 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:23.084+0000 7f5da1d5d700 1 -- 172.21.15.162:0/4034827541 shutdown_connections 2024-04-27T04:06:23.088 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:23.084+0000 7f5da1d5d700 1 -- 172.21.15.162:0/4034827541 wait complete. 2024-04-27T04:06:23.088 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:23.084+0000 7f5da1d5d700 1 Processor -- start 2024-04-27T04:06:23.089 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:23.084+0000 7f5da1d5d700 1 -- start start 2024-04-27T04:06:23.089 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:23.084+0000 7f5da1d5d700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5d9c101d10 0x7f5d9c0fd990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:06:23.089 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:23.084+0000 7f5da1d5d700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5d9c1026b0 0x7f5d9c0fded0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:06:23.089 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:23.084+0000 7f5da1d5d700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d9c0ff8e0 con 0x7f5d9c101d10 2024-04-27T04:06:23.089 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:23.084+0000 7f5da1d5d700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d9c0fe410 con 0x7f5d9c1026b0 2024-04-27T04:06:23.089 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:23.085+0000 7f5d9b7fe700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5d9c101d10 0x7f5d9c0fd990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:06:23.089 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:23.085+0000 7f5d9b7fe700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5d9c101d10 0x7f5d9c0fd990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:47138/0 (socket says 172.21.15.162:47138) 2024-04-27T04:06:23.089 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:23.085+0000 7f5d9b7fe700 1 -- 172.21.15.162:0/1684956069 learned_addr learned my addr 172.21.15.162:0/1684956069 (peer_addr_for_me v2:172.21.15.162:0/0) 2024-04-27T04:06:23.090 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:23.085+0000 7f5d9affd700 1 --2- 172.21.15.162:0/1684956069 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5d9c1026b0 0x7f5d9c0fded0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:06:23.090 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:23.085+0000 7f5d9b7fe700 1 -- 172.21.15.162:0/1684956069 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5d9c1026b0 msgr2=0x7f5d9c0fded0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:06:23.090 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:23.085+0000 7f5d9b7fe700 1 --2- 172.21.15.162:0/1684956069 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5d9c1026b0 0x7f5d9c0fded0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:06:23.090 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:23.085+0000 7f5d9b7fe700 1 -- 172.21.15.162:0/1684956069 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d90005040 con 0x7f5d9c101d10 2024-04-27T04:06:23.090 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:23.085+0000 7f5d9b7fe700 1 --2- 172.21.15.162:0/1684956069 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5d9c101d10 0x7f5d9c0fd990 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f5d8c002a50 tx=0x7f5d8c0046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:06:23.090 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:23.085+0000 7f5d83fff700 1 -- 172.21.15.162:0/1684956069 <== mon.0 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5d8c007cc0 con 0x7f5d9c101d10 2024-04-27T04:06:23.090 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:23.085+0000 7f5d83fff700 1 -- 172.21.15.162:0/1684956069 <== mon.0 v2:172.21.15.162:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f5d8c00f040 con 0x7f5d9c101d10 2024-04-27T04:06:23.091 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:23.085+0000 7f5da1d5d700 1 -- 172.21.15.162:0/1684956069 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5d9c0fe6f0 con 0x7f5d9c101d10 2024-04-27T04:06:23.091 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:23.085+0000 7f5d83fff700 1 -- 172.21.15.162:0/1684956069 <== mon.0 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5d8c00da30 con 0x7f5d9c101d10 2024-04-27T04:06:23.091 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:23.085+0000 7f5da1d5d700 1 -- 172.21.15.162:0/1684956069 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5d9c11a120 con 0x7f5d9c101d10 2024-04-27T04:06:23.091 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:23.086+0000 7f5da1d5d700 1 -- 172.21.15.162:0/1684956069 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5d9c0fd460 con 0x7f5d9c101d10 2024-04-27T04:06:23.091 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:23.086+0000 7f5d83fff700 1 -- 172.21.15.162:0/1684956069 <== mon.0 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 19) v1 ==== 86631+0+0 (secure 0 0 0) 0x7f5d8c00db90 con 0x7f5d9c101d10 2024-04-27T04:06:23.091 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:23.087+0000 7f5d83fff700 1 --2- 172.21.15.162:0/1684956069 >> [v2:172.21.15.162:6800/4079782599,v1:172.21.15.162:6801/4079782599] conn(0x7f5d840692a0 0x7f5d8406b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:06:23.092 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:23.087+0000 7f5d83fff700 1 -- 172.21.15.162:0/1684956069 <== mon.0 v2:172.21.15.162:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4634+0+0 (secure 0 0 0) 0x7f5d8c086b70 con 0x7f5d9c101d10 2024-04-27T04:06:23.092 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:23.087+0000 7f5d9affd700 1 --2- 172.21.15.162:0/1684956069 >> [v2:172.21.15.162:6800/4079782599,v1:172.21.15.162:6801/4079782599] conn(0x7f5d840692a0 0x7f5d8406b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:06:23.092 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:23.087+0000 7f5d9affd700 1 --2- 172.21.15.162:0/1684956069 >> [v2:172.21.15.162:6800/4079782599,v1:172.21.15.162:6801/4079782599] conn(0x7f5d840692a0 0x7f5d8406b750 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f5d9c0ff0c0 tx=0x7f5d90009e30 comp rx=0 tx=0).ready entity=mgr.14210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:06:23.093 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:23.091+0000 7f5d83fff700 1 -- 172.21.15.162:0/1684956069 <== mon.0 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f5d8c053250 con 0x7f5d9c101d10 2024-04-27T04:06:23.282 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:23.283+0000 7f5da1d5d700 1 -- 172.21.15.162:0/1684956069 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f5d9c04c710 con 0x7f5d9c101d10 2024-04-27T04:06:23.283 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:23.284+0000 7f5d83fff700 1 -- 172.21.15.162:0/1684956069 <== mon.0 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f5d8c056870 con 0x7f5d9c101d10 2024-04-27T04:06:23.284 INFO:teuthology.orchestra.run.smithi162.stdout:38654705702 2024-04-27T04:06:23.286 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:23.287+0000 7f5da1d5d700 1 -- 172.21.15.162:0/1684956069 >> [v2:172.21.15.162:6800/4079782599,v1:172.21.15.162:6801/4079782599] conn(0x7f5d840692a0 msgr2=0x7f5d8406b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:06:23.286 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:23.287+0000 7f5da1d5d700 1 --2- 172.21.15.162:0/1684956069 >> [v2:172.21.15.162:6800/4079782599,v1:172.21.15.162:6801/4079782599] conn(0x7f5d840692a0 0x7f5d8406b750 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f5d9c0ff0c0 tx=0x7f5d90009e30 comp rx=0 tx=0).stop 2024-04-27T04:06:23.287 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:23.287+0000 7f5da1d5d700 1 -- 172.21.15.162:0/1684956069 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5d9c101d10 msgr2=0x7f5d9c0fd990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:06:23.287 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:23.287+0000 7f5da1d5d700 1 --2- 172.21.15.162:0/1684956069 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5d9c101d10 0x7f5d9c0fd990 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f5d8c002a50 tx=0x7f5d8c0046b0 comp rx=0 tx=0).stop 2024-04-27T04:06:23.287 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:23.287+0000 7f5da1d5d700 1 -- 172.21.15.162:0/1684956069 shutdown_connections 2024-04-27T04:06:23.287 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:23.287+0000 7f5da1d5d700 1 --2- 172.21.15.162:0/1684956069 >> [v2:172.21.15.162:6800/4079782599,v1:172.21.15.162:6801/4079782599] conn(0x7f5d840692a0 0x7f5d8406b750 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:06:23.287 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:23.287+0000 7f5da1d5d700 1 --2- 172.21.15.162:0/1684956069 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5d9c101d10 0x7f5d9c0fd990 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:06:23.287 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:23.287+0000 7f5da1d5d700 1 --2- 172.21.15.162:0/1684956069 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5d9c1026b0 0x7f5d9c0fded0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:06:23.287 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:23.287+0000 7f5da1d5d700 1 -- 172.21.15.162:0/1684956069 >> 172.21.15.162:0/1684956069 conn(0x7f5d9c0f7860 msgr2=0x7f5d9c066270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:06:23.287 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:23.287+0000 7f5da1d5d700 1 -- 172.21.15.162:0/1684956069 shutdown_connections 2024-04-27T04:06:23.287 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:23.287+0000 7f5da1d5d700 1 -- 172.21.15.162:0/1684956069 wait complete. 2024-04-27T04:06:23.932 INFO:tasks.cephadm.ceph_manager.ceph:need seq 38654705699 got 38654705702 for osd.0 2024-04-27T04:06:23.933 DEBUG:teuthology.parallel:result is None 2024-04-27T04:06:24.137 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:06:24 smithi162 ceph-mon[26841]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-27T04:06:24.138 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:06:24 smithi162 ceph-mon[26841]: from='client.? 172.21.15.162:0/1684956069' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-04-27T04:06:24.142 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:24.143+0000 7fc1bfd55700 1 -- 172.21.15.162:0/327395463 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc1b8102680 msgr2=0x7fc1b80fbcf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:06:24.142 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:24.143+0000 7fc1bfd55700 1 --2- 172.21.15.162:0/327395463 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc1b8102680 0x7fc1b80fbcf0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fc1a8004940 tx=0x7fc1a8009e30 comp rx=0 tx=0).stop 2024-04-27T04:06:24.142 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:24.143+0000 7fc1bfd55700 1 -- 172.21.15.162:0/327395463 shutdown_connections 2024-04-27T04:06:24.142 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:24.143+0000 7fc1bfd55700 1 --2- 172.21.15.162:0/327395463 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc1b8102680 0x7fc1b80fbcf0 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:06:24.142 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:24.143+0000 7fc1bfd55700 1 --2- 172.21.15.162:0/327395463 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc1b8101ce0 0x7fc1b81020b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:06:24.142 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:24.143+0000 7fc1bfd55700 1 -- 172.21.15.162:0/327395463 >> 172.21.15.162:0/327395463 conn(0x7fc1b80f7850 msgr2=0x7fc1b80f9c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:06:24.143 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:24.143+0000 7fc1bfd55700 1 -- 172.21.15.162:0/327395463 shutdown_connections 2024-04-27T04:06:24.143 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:24.143+0000 7fc1bfd55700 1 -- 172.21.15.162:0/327395463 wait complete. 2024-04-27T04:06:24.144 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:24.144+0000 7fc1bfd55700 1 Processor -- start 2024-04-27T04:06:24.144 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:24.144+0000 7fc1bfd55700 1 -- start start 2024-04-27T04:06:24.145 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:24.144+0000 7fc1bfd55700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc1b8101ce0 0x7fc1b810ec90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:06:24.145 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:24.144+0000 7fc1bfd55700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc1b8102680 0x7fc1b810f1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:06:24.145 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:24.144+0000 7fc1bfd55700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1b810f710 con 0x7fc1b8102680 2024-04-27T04:06:24.145 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:24.144+0000 7fc1bfd55700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1b810f850 con 0x7fc1b8101ce0 2024-04-27T04:06:24.145 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:24.144+0000 7fc1bd2f0700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc1b8102680 0x7fc1b810f1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:06:24.145 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:24.144+0000 7fc1bd2f0700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc1b8102680 0x7fc1b810f1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:56818/0 (socket says 172.21.15.162:56818) 2024-04-27T04:06:24.145 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:24.144+0000 7fc1bd2f0700 1 -- 172.21.15.162:0/3650618871 learned_addr learned my addr 172.21.15.162:0/3650618871 (peer_addr_for_me v2:172.21.15.162:0/0) 2024-04-27T04:06:24.145 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:24.144+0000 7fc1bdaf1700 1 --2- 172.21.15.162:0/3650618871 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc1b8101ce0 0x7fc1b810ec90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:06:24.145 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:24.144+0000 7fc1bd2f0700 1 -- 172.21.15.162:0/3650618871 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc1b8101ce0 msgr2=0x7fc1b810ec90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:06:24.145 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:24.144+0000 7fc1bd2f0700 1 --2- 172.21.15.162:0/3650618871 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc1b8101ce0 0x7fc1b810ec90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:06:24.145 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:24.144+0000 7fc1bd2f0700 1 -- 172.21.15.162:0/3650618871 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc1a8005040 con 0x7fc1b8102680 2024-04-27T04:06:24.145 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:24.144+0000 7fc1bd2f0700 1 --2- 172.21.15.162:0/3650618871 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc1b8102680 0x7fc1b810f1d0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fc1a8009100 tx=0x7fc1a8009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:06:24.145 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:24.144+0000 7fc1ae7fc700 1 -- 172.21.15.162:0/3650618871 <== mon.0 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc1a800e040 con 0x7fc1b8102680 2024-04-27T04:06:24.145 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:24.145+0000 7fc1bfd55700 1 -- 172.21.15.162:0/3650618871 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc1b8117b40 con 0x7fc1b8102680 2024-04-27T04:06:24.146 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:24.145+0000 7fc1ae7fc700 1 -- 172.21.15.162:0/3650618871 <== mon.0 v2:172.21.15.162:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fc1a8009af0 con 0x7fc1b8102680 2024-04-27T04:06:24.146 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:24.145+0000 7fc1ae7fc700 1 -- 172.21.15.162:0/3650618871 <== mon.0 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc1a801c3f0 con 0x7fc1b8102680 2024-04-27T04:06:24.146 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:24.145+0000 7fc1bfd55700 1 -- 172.21.15.162:0/3650618871 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc1b81180d0 con 0x7fc1b8102680 2024-04-27T04:06:24.146 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:24.145+0000 7fc1bfd55700 1 -- 172.21.15.162:0/3650618871 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc1b80fd430 con 0x7fc1b8102680 2024-04-27T04:06:24.146 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:24.146+0000 7fc1ae7fc700 1 -- 172.21.15.162:0/3650618871 <== mon.0 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 19) v1 ==== 86631+0+0 (secure 0 0 0) 0x7fc1a800baf0 con 0x7fc1b8102680 2024-04-27T04:06:24.147 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:24.146+0000 7fc1ae7fc700 1 --2- 172.21.15.162:0/3650618871 >> [v2:172.21.15.162:6800/4079782599,v1:172.21.15.162:6801/4079782599] conn(0x7fc1a40692a0 0x7fc1a406b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:06:24.147 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:24.146+0000 7fc1ae7fc700 1 -- 172.21.15.162:0/3650618871 <== mon.0 v2:172.21.15.162:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4634+0+0 (secure 0 0 0) 0x7fc1a80880e0 con 0x7fc1b8102680 2024-04-27T04:06:24.147 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:24.146+0000 7fc1bdaf1700 1 --2- 172.21.15.162:0/3650618871 >> [v2:172.21.15.162:6800/4079782599,v1:172.21.15.162:6801/4079782599] conn(0x7fc1a40692a0 0x7fc1a406b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:06:24.147 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:24.147+0000 7fc1bdaf1700 1 --2- 172.21.15.162:0/3650618871 >> [v2:172.21.15.162:6800/4079782599,v1:172.21.15.162:6801/4079782599] conn(0x7fc1a40692a0 0x7fc1a406b750 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fc1b0007970 tx=0x7fc1b0009da0 comp rx=0 tx=0).ready entity=mgr.14210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:06:24.150 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:24.151+0000 7fc1ae7fc700 1 -- 172.21.15.162:0/3650618871 <== mon.0 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7fc1a8054750 con 0x7fc1b8102680 2024-04-27T04:06:24.338 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:24.338+0000 7fc1bfd55700 1 -- 172.21.15.162:0/3650618871 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7fc1b8108df0 con 0x7fc1b8102680 2024-04-27T04:06:24.338 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:24.338+0000 7fc1ae7fc700 1 -- 172.21.15.162:0/3650618871 <== mon.0 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fc1a8057d70 con 0x7fc1b8102680 2024-04-27T04:06:24.338 INFO:teuthology.orchestra.run.smithi162.stdout:81604378651 2024-04-27T04:06:24.340 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:24.340+0000 7fc1bfd55700 1 -- 172.21.15.162:0/3650618871 >> [v2:172.21.15.162:6800/4079782599,v1:172.21.15.162:6801/4079782599] conn(0x7fc1a40692a0 msgr2=0x7fc1a406b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:06:24.340 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:24.340+0000 7fc1bfd55700 1 --2- 172.21.15.162:0/3650618871 >> [v2:172.21.15.162:6800/4079782599,v1:172.21.15.162:6801/4079782599] conn(0x7fc1a40692a0 0x7fc1a406b750 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fc1b0007970 tx=0x7fc1b0009da0 comp rx=0 tx=0).stop 2024-04-27T04:06:24.340 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:24.340+0000 7fc1bfd55700 1 -- 172.21.15.162:0/3650618871 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc1b8102680 msgr2=0x7fc1b810f1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:06:24.340 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:24.340+0000 7fc1bfd55700 1 --2- 172.21.15.162:0/3650618871 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc1b8102680 0x7fc1b810f1d0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fc1a8009100 tx=0x7fc1a8009e30 comp rx=0 tx=0).stop 2024-04-27T04:06:24.340 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:24.340+0000 7fc1bfd55700 1 -- 172.21.15.162:0/3650618871 shutdown_connections 2024-04-27T04:06:24.340 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:24.340+0000 7fc1bfd55700 1 --2- 172.21.15.162:0/3650618871 >> [v2:172.21.15.162:6800/4079782599,v1:172.21.15.162:6801/4079782599] conn(0x7fc1a40692a0 0x7fc1a406b750 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:06:24.340 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:24.340+0000 7fc1bfd55700 1 --2- 172.21.15.162:0/3650618871 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc1b8101ce0 0x7fc1b810ec90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:06:24.340 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:24.340+0000 7fc1bfd55700 1 --2- 172.21.15.162:0/3650618871 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc1b8102680 0x7fc1b810f1d0 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:06:24.340 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:24.340+0000 7fc1bfd55700 1 -- 172.21.15.162:0/3650618871 >> 172.21.15.162:0/3650618871 conn(0x7fc1b80f7850 msgr2=0x7fc1b80f8880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:06:24.340 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:24.341+0000 7fc1bfd55700 1 -- 172.21.15.162:0/3650618871 shutdown_connections 2024-04-27T04:06:24.340 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:24.341+0000 7fc1bfd55700 1 -- 172.21.15.162:0/3650618871 wait complete. 2024-04-27T04:06:24.423 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:06:24 smithi181 ceph-mon[33774]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-27T04:06:24.423 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:06:24 smithi181 ceph-mon[33774]: from='client.? 172.21.15.162:0/1684956069' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-04-27T04:06:25.021 INFO:tasks.cephadm.ceph_manager.ceph:need seq 81604378648 got 81604378651 for osd.2 2024-04-27T04:06:25.021 DEBUG:teuthology.parallel:result is None 2024-04-27T04:06:25.021 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2024-04-27T04:06:25.022 DEBUG:teuthology.orchestra.run.smithi162:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell --fsid 7f9750d6-044a-11ef-bc93-c7b262605968 -- ceph pg dump --format=json 2024-04-27T04:06:25.275 INFO:teuthology.orchestra.run.smithi162.stderr:Inferring config /var/lib/ceph/7f9750d6-044a-11ef-bc93-c7b262605968/mon.smithi162/config 2024-04-27T04:06:25.387 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:06:25 smithi162 ceph-mon[26841]: from='client.? 172.21.15.162:0/3650618871' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-04-27T04:06:25.423 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:06:25 smithi181 ceph-mon[33774]: from='client.? 172.21.15.162:0/3650618871' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-04-27T04:06:26.388 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:06:26 smithi162 ceph-mon[26841]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-27T04:06:26.423 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:06:26 smithi181 ceph-mon[33774]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-27T04:06:26.997 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:26.996+0000 7fe3e4faf700 1 -- 172.21.15.162:0/1304498665 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe3e00fc6b0 msgr2=0x7fe3e00fcb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:06:26.997 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:26.996+0000 7fe3e4faf700 1 --2- 172.21.15.162:0/1304498665 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe3e00fc6b0 0x7fe3e00fcb20 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fe3d0004940 tx=0x7fe3d0009e30 comp rx=0 tx=0).stop 2024-04-27T04:06:26.997 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:26.996+0000 7fe3e4faf700 1 -- 172.21.15.162:0/1304498665 shutdown_connections 2024-04-27T04:06:26.997 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:26.996+0000 7fe3e4faf700 1 --2- 172.21.15.162:0/1304498665 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe3e00fc6b0 0x7fe3e00fcb20 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:06:26.997 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:26.996+0000 7fe3e4faf700 1 --2- 172.21.15.162:0/1304498665 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe3e0101d90 0x7fe3e0102160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:06:26.997 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:26.996+0000 7fe3e4faf700 1 -- 172.21.15.162:0/1304498665 >> 172.21.15.162:0/1304498665 conn(0x7fe3e0071460 msgr2=0x7fe3e0071860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:06:26.997 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:26.997+0000 7fe3e4faf700 1 -- 172.21.15.162:0/1304498665 shutdown_connections 2024-04-27T04:06:26.998 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:26.997+0000 7fe3e4faf700 1 -- 172.21.15.162:0/1304498665 wait complete. 2024-04-27T04:06:26.998 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:26.997+0000 7fe3e4faf700 1 Processor -- start 2024-04-27T04:06:26.998 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:26.997+0000 7fe3e4faf700 1 -- start start 2024-04-27T04:06:26.998 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:26.997+0000 7fe3e4faf700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe3e00fc6b0 0x7fe3e0113980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:06:26.998 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:26.997+0000 7fe3e4faf700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe3e0101d90 0x7fe3e0113ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:06:26.998 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:26.997+0000 7fe3e4faf700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3e0114510 con 0x7fe3e00fc6b0 2024-04-27T04:06:26.998 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:26.997+0000 7fe3e4faf700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3e010db10 con 0x7fe3e0101d90 2024-04-27T04:06:26.998 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:26.997+0000 7fe3de59c700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe3e00fc6b0 0x7fe3e0113980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:06:26.998 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:26.997+0000 7fe3de59c700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe3e00fc6b0 0x7fe3e0113980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:56840/0 (socket says 172.21.15.162:56840) 2024-04-27T04:06:26.998 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:26.997+0000 7fe3de59c700 1 -- 172.21.15.162:0/1200669005 learned_addr learned my addr 172.21.15.162:0/1200669005 (peer_addr_for_me v2:172.21.15.162:0/0) 2024-04-27T04:06:26.998 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:26.997+0000 7fe3ddd9b700 1 --2- 172.21.15.162:0/1200669005 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe3e0101d90 0x7fe3e0113ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:06:26.998 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:26.998+0000 7fe3de59c700 1 -- 172.21.15.162:0/1200669005 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe3e0101d90 msgr2=0x7fe3e0113ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:06:26.998 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:26.998+0000 7fe3de59c700 1 --2- 172.21.15.162:0/1200669005 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe3e0101d90 0x7fe3e0113ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:06:26.999 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:26.998+0000 7fe3de59c700 1 -- 172.21.15.162:0/1200669005 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3d0005040 con 0x7fe3e00fc6b0 2024-04-27T04:06:26.999 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:26.998+0000 7fe3de59c700 1 --2- 172.21.15.162:0/1200669005 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe3e00fc6b0 0x7fe3e0113980 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fe3d4002a50 tx=0x7fe3d40046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:06:26.999 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:26.998+0000 7fe3ceffd700 1 -- 172.21.15.162:0/1200669005 <== mon.0 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe3d4007cc0 con 0x7fe3e00fc6b0 2024-04-27T04:06:26.999 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:26.998+0000 7fe3ceffd700 1 -- 172.21.15.162:0/1200669005 <== mon.0 v2:172.21.15.162:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fe3d400f040 con 0x7fe3e00fc6b0 2024-04-27T04:06:26.999 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:26.998+0000 7fe3e4faf700 1 -- 172.21.15.162:0/1200669005 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe3e010ddf0 con 0x7fe3e00fc6b0 2024-04-27T04:06:26.999 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:26.998+0000 7fe3ceffd700 1 -- 172.21.15.162:0/1200669005 <== mon.0 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe3d4009640 con 0x7fe3e00fc6b0 2024-04-27T04:06:26.999 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:26.998+0000 7fe3e4faf700 1 -- 172.21.15.162:0/1200669005 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe3e010e330 con 0x7fe3e00fc6b0 2024-04-27T04:06:27.000 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:26.999+0000 7fe3e4faf700 1 -- 172.21.15.162:0/1200669005 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe3e0066010 con 0x7fe3e00fc6b0 2024-04-27T04:06:27.000 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:26.999+0000 7fe3ceffd700 1 -- 172.21.15.162:0/1200669005 <== mon.0 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 19) v1 ==== 86631+0+0 (secure 0 0 0) 0x7fe3d40097c0 con 0x7fe3e00fc6b0 2024-04-27T04:06:27.000 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:27.000+0000 7fe3ceffd700 1 --2- 172.21.15.162:0/1200669005 >> [v2:172.21.15.162:6800/4079782599,v1:172.21.15.162:6801/4079782599] conn(0x7fe3c80692a0 0x7fe3c806b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:06:27.000 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:27.000+0000 7fe3ceffd700 1 -- 172.21.15.162:0/1200669005 <== mon.0 v2:172.21.15.162:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4634+0+0 (secure 0 0 0) 0x7fe3d4086b70 con 0x7fe3e00fc6b0 2024-04-27T04:06:27.000 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:27.000+0000 7fe3ddd9b700 1 --2- 172.21.15.162:0/1200669005 >> [v2:172.21.15.162:6800/4079782599,v1:172.21.15.162:6801/4079782599] conn(0x7fe3c80692a0 0x7fe3c806b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:06:27.000 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:27.000+0000 7fe3ddd9b700 1 --2- 172.21.15.162:0/1200669005 >> [v2:172.21.15.162:6800/4079782599,v1:172.21.15.162:6801/4079782599] conn(0x7fe3c80692a0 0x7fe3c806b750 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fe3d000b3b0 tx=0x7fe3d0009e30 comp rx=0 tx=0).ready entity=mgr.14210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:06:27.003 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:27.004+0000 7fe3ceffd700 1 -- 172.21.15.162:0/1200669005 <== mon.0 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7fe3d4056c70 con 0x7fe3e00fc6b0 2024-04-27T04:06:27.186 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:27.186+0000 7fe3e4faf700 1 -- 172.21.15.162:0/1200669005 --> [v2:172.21.15.162:6800/4079782599,v1:172.21.15.162:6801/4079782599] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fe3e0108e70 con 0x7fe3c80692a0 2024-04-27T04:06:27.187 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:27.188+0000 7fe3ceffd700 1 -- 172.21.15.162:0/1200669005 <== mgr.14210 v2:172.21.15.162:6800/4079782599 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+28350 (secure 0 0 0) 0x7fe3e0108e70 con 0x7fe3c80692a0 2024-04-27T04:06:27.187 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-04-27T04:06:27.189 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:27.190+0000 7fe3e4faf700 1 -- 172.21.15.162:0/1200669005 >> [v2:172.21.15.162:6800/4079782599,v1:172.21.15.162:6801/4079782599] conn(0x7fe3c80692a0 msgr2=0x7fe3c806b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:06:27.190 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:27.190+0000 7fe3e4faf700 1 --2- 172.21.15.162:0/1200669005 >> [v2:172.21.15.162:6800/4079782599,v1:172.21.15.162:6801/4079782599] conn(0x7fe3c80692a0 0x7fe3c806b750 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fe3d000b3b0 tx=0x7fe3d0009e30 comp rx=0 tx=0).stop 2024-04-27T04:06:27.190 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:27.190+0000 7fe3e4faf700 1 -- 172.21.15.162:0/1200669005 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe3e00fc6b0 msgr2=0x7fe3e0113980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:06:27.190 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:27.190+0000 7fe3e4faf700 1 --2- 172.21.15.162:0/1200669005 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe3e00fc6b0 0x7fe3e0113980 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fe3d4002a50 tx=0x7fe3d40046b0 comp rx=0 tx=0).stop 2024-04-27T04:06:27.190 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:27.190+0000 7fe3e4faf700 1 -- 172.21.15.162:0/1200669005 shutdown_connections 2024-04-27T04:06:27.190 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:27.190+0000 7fe3e4faf700 1 --2- 172.21.15.162:0/1200669005 >> [v2:172.21.15.162:6800/4079782599,v1:172.21.15.162:6801/4079782599] conn(0x7fe3c80692a0 0x7fe3c806b750 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:06:27.190 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:27.190+0000 7fe3e4faf700 1 --2- 172.21.15.162:0/1200669005 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe3e00fc6b0 0x7fe3e0113980 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:06:27.190 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:27.190+0000 7fe3e4faf700 1 --2- 172.21.15.162:0/1200669005 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe3e0101d90 0x7fe3e0113ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:06:27.190 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:27.190+0000 7fe3e4faf700 1 -- 172.21.15.162:0/1200669005 >> 172.21.15.162:0/1200669005 conn(0x7fe3e0071460 msgr2=0x7fe3e0105740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:06:27.190 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:27.190+0000 7fe3e4faf700 1 -- 172.21.15.162:0/1200669005 shutdown_connections 2024-04-27T04:06:27.190 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:27.190+0000 7fe3e4faf700 1 -- 172.21.15.162:0/1200669005 wait complete. 2024-04-27T04:06:27.191 INFO:teuthology.orchestra.run.smithi162.stderr:dumped all 2024-04-27T04:06:27.813 INFO:teuthology.orchestra.run.smithi162.stdout:{"pg_ready":true,"pg_map":{"version":153,"stamp":"2024-04-27T04:06:26.706423+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":3,"acting":3,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":3,"num_osds":6,"num_per_pool_osds":6,"num_per_pool_omap_osds":3,"kb":562470912,"kb_used":1788036,"kb_used_data":3872,"kb_used_omap":9,"kb_used_meta":1784054,"kb_avail":560682876,"statfs":{"total":575970213888,"available":574139265024,"internally_reserved":0,"allocated":3964928,"data_stored":2605601,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":10224,"internal_metadata":1826871312},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001986"},"pg_stats":[{"pgid":"1.0","version":"23'117","reported_seq":190,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-27T04:05:37.772252+0000","last_change":"2024-04-27T04:05:12.340987+0000","last_active":"2024-04-27T04:05:37.772252+0000","last_peered":"2024-04-27T04:05:37.772252+0000","last_clean":"2024-04-27T04:05:37.772252+0000","last_became_active":"2024-04-27T04:05:12.340632+0000","last_became_peered":"2024-04-27T04:05:12.340632+0000","last_unstale":"2024-04-27T04:05:37.772252+0000","last_undegraded":"2024-04-27T04:05:37.772252+0000","last_fullsized":"2024-04-27T04:05:37.772252+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T04:04:24.972184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T04:04:24.972184+0000","last_clean_scrub_stamp":"2024-04-27T04:04:24.972184+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T14:10:09.705502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":2375680,"data_stored":2361472,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":3,"acting":3,"num_store_stats":4}],"osd_stats":[{"osd":5,"up_from":39,"seq":167503724555,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":93745152,"kb_used":297712,"kb_used_data":356,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":93447440,"statfs":{"total":95995035648,"available":95690178560,"internally_reserved":0,"allocated":364544,"data_stored":139193,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,4],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51200000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46300000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55800000000000005}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61699999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54300000000000004}]}]},{"osd":4,"up_from":34,"seq":146028888081,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":93745152,"kb_used":297720,"kb_used_data":356,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":93447432,"statfs":{"total":95995035648,"available":95690170368,"internally_reserved":0,"allocated":364544,"data_stored":139193,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Apr 27 04:06:15 2024","interfaces":[{"interface":"back","average":{"1min":0.53500000000000003,"5min":0.53500000000000003,"15min":0.53500000000000003},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.66900000000000004,"5min":0.66900000000000004,"15min":0.66900000000000004},"last":0.60599999999999998},{"interface":"front","average":{"1min":0.51200000000000001,"5min":0.51200000000000001,"15min":0.51200000000000001},"min":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.59399999999999997,"5min":0.59399999999999997,"15min":0.59399999999999997},"last":0.622}]},{"osd":1,"last update":"Sat Apr 27 04:06:15 2024","interfaces":[{"interface":"back","average":{"1min":0.50900000000000001,"5min":0.50900000000000001,"15min":0.50900000000000001},"min":{"1min":0.34999999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.67200000000000004,"5min":0.67200000000000004,"15min":0.67200000000000004},"last":0.46100000000000002},{"interface":"front","average":{"1min":0.52300000000000002,"5min":0.52300000000000002,"15min":0.52300000000000002},"min":{"1min":0.35699999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.71099999999999997,"5min":0.71099999999999997,"15min":0.71099999999999997},"last":0.53800000000000003}]},{"osd":2,"last update":"Sat Apr 27 04:06:15 2024","interfaces":[{"interface":"back","average":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.52200000000000002},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.68200000000000005,"5min":0.68200000000000005,"15min":0.68200000000000005},"last":0.59599999999999997},{"interface":"front","average":{"1min":0.52400000000000002,"5min":0.52400000000000002,"15min":0.52400000000000002},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.67300000000000004,"5min":0.67300000000000004,"15min":0.67300000000000004},"last":0.45200000000000001}]},{"osd":3,"last update":"Sat Apr 27 04:06:15 2024","interfaces":[{"interface":"back","average":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"min":{"1min":0.22500000000000001,"5min":0.22500000000000001,"15min":0.22500000000000001},"max":{"1min":0.59299999999999997,"5min":0.59299999999999997,"15min":0.59299999999999997},"last":0.30399999999999999},{"interface":"front","average":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"min":{"1min":0.249,"5min":0.249,"15min":0.249},"max":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"last":0.34200000000000003}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.33300000000000002}]}]},{"osd":3,"up_from":26,"seq":111669149718,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":298304,"kb_used_data":936,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":93446848,"statfs":{"total":95995035648,"available":95689572352,"internally_reserved":0,"allocated":958464,"data_stored":729561,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Apr 27 04:05:46 2024","interfaces":[{"interface":"back","average":{"1min":0.753,"5min":0.753,"15min":0.753},"min":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":5.6779999999999999,"5min":5.6779999999999999,"15min":5.6779999999999999},"last":0.60699999999999998},{"interface":"front","average":{"1min":0.48499999999999999,"5min":0.48499999999999999,"15min":0.48499999999999999},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.69099999999999995,"5min":0.69099999999999995,"15min":0.69099999999999995},"last":0.63200000000000001}]},{"osd":1,"last update":"Sat Apr 27 04:05:46 2024","interfaces":[{"interface":"back","average":{"1min":0.48999999999999999,"5min":0.48999999999999999,"15min":0.48999999999999999},"min":{"1min":0.33500000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.72499999999999998,"5min":0.72499999999999998,"15min":0.72499999999999998},"last":0.61799999999999999},{"interface":"front","average":{"1min":0.49399999999999999,"5min":0.49399999999999999,"15min":0.49399999999999999},"min":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.71399999999999997,"5min":0.71399999999999997,"15min":0.71399999999999997},"last":0.56699999999999995}]},{"osd":2,"last update":"Sat Apr 27 04:05:46 2024","interfaces":[{"interface":"back","average":{"1min":0.48499999999999999,"5min":0.48499999999999999,"15min":0.48499999999999999},"min":{"1min":0.33000000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.73299999999999998,"5min":0.73299999999999998,"15min":0.73299999999999998},"last":0.58299999999999996},{"interface":"front","average":{"1min":0.48699999999999999,"5min":0.48699999999999999,"15min":0.48699999999999999},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.68000000000000005,"5min":0.68000000000000005,"15min":0.68000000000000005},"last":0.64700000000000002}]},{"osd":4,"last update":"Sat Apr 27 04:06:12 2024","interfaces":[{"interface":"back","average":{"1min":0.37,"5min":0.37,"15min":0.37},"min":{"1min":0.29199999999999998,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.47199999999999998,"5min":0.47199999999999998,"15min":0.47199999999999998},"last":0.39500000000000002},{"interface":"front","average":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"min":{"1min":0.28199999999999997,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.52600000000000002,"5min":0.52600000000000002,"15min":0.52600000000000002},"last":0.40300000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.443}]}]},{"osd":2,"up_from":19,"seq":81604378652,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":93745152,"kb_used":297716,"kb_used_data":352,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":93447436,"statfs":{"total":95995035648,"available":95690174464,"internally_reserved":0,"allocated":360448,"data_stored":138532,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Apr 27 04:05:24 2024","interfaces":[{"interface":"back","average":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"min":{"1min":0.24199999999999999,"5min":0.24199999999999999,"15min":0.24199999999999999},"max":{"1min":0.505,"5min":0.505,"15min":0.505},"last":0.309},{"interface":"front","average":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"min":{"1min":0.27000000000000002,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.50800000000000001,"5min":0.50800000000000001,"15min":0.50800000000000001},"last":0.29699999999999999}]},{"osd":1,"last update":"Sat Apr 27 04:05:24 2024","interfaces":[{"interface":"back","average":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"min":{"1min":0.28100000000000003,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":0.54300000000000004,"5min":0.54300000000000004,"15min":0.54300000000000004},"last":0.34499999999999997},{"interface":"front","average":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"min":{"1min":0.26300000000000001,"5min":0.26300000000000001,"15min":0.26300000000000001},"max":{"1min":0.60299999999999998,"5min":0.60299999999999998,"15min":0.60299999999999998},"last":0.27300000000000002}]},{"osd":3,"last update":"Sat Apr 27 04:05:48 2024","interfaces":[{"interface":"back","average":{"1min":0.53800000000000003,"5min":0.53800000000000003,"15min":0.53800000000000003},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.72799999999999998,"5min":0.72799999999999998,"15min":0.72799999999999998},"last":0.52500000000000002},{"interface":"front","average":{"1min":0.53200000000000003,"5min":0.53200000000000003,"15min":0.53200000000000003},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.68500000000000005,"5min":0.68500000000000005,"15min":0.68500000000000005},"last":0.50600000000000001}]},{"osd":4,"last update":"Sat Apr 27 04:06:16 2024","interfaces":[{"interface":"back","average":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.52200000000000002},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.89500000000000002,"5min":0.89500000000000002,"15min":0.89500000000000002},"last":0.51500000000000001},{"interface":"front","average":{"1min":0.502,"5min":0.502,"15min":0.502},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.69499999999999995,"5min":0.69499999999999995,"15min":0.69499999999999995},"last":0.48699999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53700000000000003}]}]},{"osd":0,"up_from":9,"seq":38654705703,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":298292,"kb_used_data":936,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":93446860,"statfs":{"total":95995035648,"available":95689584640,"internally_reserved":0,"allocated":958464,"data_stored":729561,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[1,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sat Apr 27 04:06:03 2024","interfaces":[{"interface":"back","average":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"min":{"1min":0.23400000000000001,"5min":0.186,"15min":0.186},"max":{"1min":0.52000000000000002,"5min":0.57899999999999996,"15min":0.57899999999999996},"last":0.438},{"interface":"front","average":{"1min":0.35799999999999998,"5min":0.34599999999999997,"15min":0.34399999999999997},"min":{"1min":0.26300000000000001,"5min":0.191,"15min":0.191},"max":{"1min":0.54300000000000004,"5min":0.54300000000000004,"15min":0.54300000000000004},"last":0.36899999999999999}]},{"osd":2,"last update":"Sat Apr 27 04:05:25 2024","interfaces":[{"interface":"back","average":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"min":{"1min":0.29799999999999999,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.54100000000000004,"5min":0.54100000000000004,"15min":0.54100000000000004},"last":0.40400000000000003},{"interface":"front","average":{"1min":0.44,"5min":0.44,"15min":0.44},"min":{"1min":0.313,"5min":0.313,"15min":0.313},"max":{"1min":0.56499999999999995,"5min":0.56499999999999995,"15min":0.56499999999999995},"last":0.42299999999999999}]},{"osd":3,"last update":"Sat Apr 27 04:05:47 2024","interfaces":[{"interface":"back","average":{"1min":0.53800000000000003,"5min":0.53800000000000003,"15min":0.53800000000000003},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.72799999999999998,"5min":0.72799999999999998,"15min":0.72799999999999998},"last":0.58699999999999997},{"interface":"front","average":{"1min":0.55900000000000005,"5min":0.55900000000000005,"15min":0.55900000000000005},"min":{"1min":0.42799999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":0.71299999999999997,"5min":0.71299999999999997,"15min":0.71299999999999997},"last":0.56499999999999995}]},{"osd":4,"last update":"Sat Apr 27 04:06:13 2024","interfaces":[{"interface":"back","average":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.71199999999999997,"5min":0.71199999999999997,"15min":0.71199999999999997},"last":0.60099999999999998},{"interface":"front","average":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.73499999999999999,"5min":0.73499999999999999,"15min":0.73499999999999999},"last":0.53300000000000003}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51500000000000001}]}]},{"osd":1,"up_from":14,"seq":60129542177,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":298292,"kb_used_data":936,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":93446860,"statfs":{"total":95995035648,"available":95689584640,"internally_reserved":0,"allocated":958464,"data_stored":729561,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Apr 27 04:06:00 2024","interfaces":[{"interface":"back","average":{"1min":0.35299999999999998,"5min":0.33300000000000002,"15min":0.33000000000000002},"min":{"1min":0.245,"5min":0.245,"15min":0.245},"max":{"1min":0.49199999999999999,"5min":0.53300000000000003,"15min":0.53300000000000003},"last":0.307},{"interface":"front","average":{"1min":0.36599999999999999,"5min":0.33900000000000002,"15min":0.33500000000000002},"min":{"1min":0.248,"5min":0.22600000000000001,"15min":0.22600000000000001},"max":{"1min":0.52000000000000002,"5min":0.54900000000000004,"15min":0.54900000000000004},"last":0.39800000000000002}]},{"osd":2,"last update":"Sat Apr 27 04:05:25 2024","interfaces":[{"interface":"back","average":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"min":{"1min":0.24099999999999999,"5min":0.24099999999999999,"15min":0.24099999999999999},"max":{"1min":0.54300000000000004,"5min":0.54300000000000004,"15min":0.54300000000000004},"last":0.35099999999999998},{"interface":"front","average":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"min":{"1min":0.26300000000000001,"5min":0.26300000000000001,"15min":0.26300000000000001},"max":{"1min":0.55100000000000005,"5min":0.55100000000000005,"15min":0.55100000000000005},"last":0.36399999999999999}]},{"osd":3,"last update":"Sat Apr 27 04:05:46 2024","interfaces":[{"interface":"back","average":{"1min":0.57899999999999996,"5min":0.57899999999999996,"15min":0.57899999999999996},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.747,"5min":0.747,"15min":0.747},"last":0.60099999999999998},{"interface":"front","average":{"1min":0.58199999999999996,"5min":0.58199999999999996,"15min":0.58199999999999996},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.71399999999999997,"5min":0.71399999999999997,"15min":0.71399999999999997},"last":0.56599999999999995}]},{"osd":4,"last update":"Sat Apr 27 04:06:16 2024","interfaces":[{"interface":"back","average":{"1min":0.56299999999999994,"5min":0.56299999999999994,"15min":0.56299999999999994},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.753,"5min":0.753,"15min":0.753},"last":0.61699999999999999},{"interface":"front","average":{"1min":0.57799999999999996,"5min":0.57799999999999996,"15min":0.57799999999999996},"min":{"1min":0.441,"5min":0.441,"15min":0.441},"max":{"1min":0.78400000000000003,"5min":0.78400000000000003,"15min":0.78400000000000003},"last":0.57699999999999996}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51200000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-27T04:06:27.815 DEBUG:teuthology.orchestra.run.smithi162:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell --fsid 7f9750d6-044a-11ef-bc93-c7b262605968 -- ceph pg dump --format=json 2024-04-27T04:06:28.070 INFO:teuthology.orchestra.run.smithi162.stderr:Inferring config /var/lib/ceph/7f9750d6-044a-11ef-bc93-c7b262605968/mon.smithi162/config 2024-04-27T04:06:28.300 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:06:28 smithi162 ceph-mon[26841]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-27T04:06:28.301 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:06:28 smithi162 ceph-mon[26841]: from='client.14416 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-27T04:06:28.423 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:06:28 smithi181 ceph-mon[33774]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-27T04:06:28.423 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:06:28 smithi181 ceph-mon[33774]: from='client.14416 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-27T04:06:29.769 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:29.765+0000 7fcdabbe0700 1 -- 172.21.15.162:0/3842021841 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fcda4065640 msgr2=0x7fcda4065ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:06:29.769 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:29.765+0000 7fcdabbe0700 1 --2- 172.21.15.162:0/3842021841 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fcda4065640 0x7fcda4065ab0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fcda0004940 tx=0x7fcda0009e30 comp rx=0 tx=0).stop 2024-04-27T04:06:29.769 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:29.765+0000 7fcdabbe0700 1 -- 172.21.15.162:0/3842021841 shutdown_connections 2024-04-27T04:06:29.770 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:29.765+0000 7fcdabbe0700 1 --2- 172.21.15.162:0/3842021841 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fcda4065640 0x7fcda4065ab0 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:06:29.770 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:29.765+0000 7fcdabbe0700 1 --2- 172.21.15.162:0/3842021841 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcda4102e50 0x7fcda4103220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:06:29.770 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:29.765+0000 7fcdabbe0700 1 -- 172.21.15.162:0/3842021841 >> 172.21.15.162:0/3842021841 conn(0x7fcda4071d00 msgr2=0x7fcda4072100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:06:29.770 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:29.765+0000 7fcdabbe0700 1 -- 172.21.15.162:0/3842021841 shutdown_connections 2024-04-27T04:06:29.770 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:29.765+0000 7fcdabbe0700 1 -- 172.21.15.162:0/3842021841 wait complete. 2024-04-27T04:06:29.770 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:29.766+0000 7fcdabbe0700 1 Processor -- start 2024-04-27T04:06:29.770 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:29.766+0000 7fcdabbe0700 1 -- start start 2024-04-27T04:06:29.770 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:29.766+0000 7fcdabbe0700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcda4065640 0x7fcda40fbdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:06:29.770 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:29.766+0000 7fcdabbe0700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fcda4102e50 0x7fcda40fc2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:06:29.770 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:29.766+0000 7fcdabbe0700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcda40fc880 con 0x7fcda4102e50 2024-04-27T04:06:29.770 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:29.766+0000 7fcdabbe0700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcda40fc9c0 con 0x7fcda4065640 2024-04-27T04:06:29.770 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:29.766+0000 7fcda917b700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fcda4102e50 0x7fcda40fc2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:06:29.771 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:29.766+0000 7fcda917b700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fcda4102e50 0x7fcda40fc2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:56848/0 (socket says 172.21.15.162:56848) 2024-04-27T04:06:29.771 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:29.766+0000 7fcda917b700 1 -- 172.21.15.162:0/4256088948 learned_addr learned my addr 172.21.15.162:0/4256088948 (peer_addr_for_me v2:172.21.15.162:0/0) 2024-04-27T04:06:29.771 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:29.766+0000 7fcda997c700 1 --2- 172.21.15.162:0/4256088948 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcda4065640 0x7fcda40fbdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:06:29.771 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:29.766+0000 7fcda917b700 1 -- 172.21.15.162:0/4256088948 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcda4065640 msgr2=0x7fcda40fbdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:06:29.771 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:29.766+0000 7fcda917b700 1 --2- 172.21.15.162:0/4256088948 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcda4065640 0x7fcda40fbdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:06:29.771 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:29.766+0000 7fcda917b700 1 -- 172.21.15.162:0/4256088948 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcda0005040 con 0x7fcda4102e50 2024-04-27T04:06:29.771 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:29.767+0000 7fcda917b700 1 --2- 172.21.15.162:0/4256088948 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fcda4102e50 0x7fcda40fc2f0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fcda000b3b0 tx=0x7fcda0009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:06:29.771 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:29.767+0000 7fcd9a7fc700 1 -- 172.21.15.162:0/4256088948 <== mon.0 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fcda000e040 con 0x7fcda4102e50 2024-04-27T04:06:29.771 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:29.767+0000 7fcdabbe0700 1 -- 172.21.15.162:0/4256088948 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcda40fcc10 con 0x7fcda4102e50 2024-04-27T04:06:29.771 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:29.767+0000 7fcd9a7fc700 1 -- 172.21.15.162:0/4256088948 <== mon.0 v2:172.21.15.162:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fcda0013420 con 0x7fcda4102e50 2024-04-27T04:06:29.771 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:29.767+0000 7fcd9a7fc700 1 -- 172.21.15.162:0/4256088948 <== mon.0 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fcda001c630 con 0x7fcda4102e50 2024-04-27T04:06:29.772 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:29.767+0000 7fcdabbe0700 1 -- 172.21.15.162:0/4256088948 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcda411a2a0 con 0x7fcda4102e50 2024-04-27T04:06:29.772 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:29.768+0000 7fcdabbe0700 1 -- 172.21.15.162:0/4256088948 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcda4100100 con 0x7fcda4102e50 2024-04-27T04:06:29.772 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:29.768+0000 7fcd9a7fc700 1 -- 172.21.15.162:0/4256088948 <== mon.0 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 19) v1 ==== 86631+0+0 (secure 0 0 0) 0x7fcda0013590 con 0x7fcda4102e50 2024-04-27T04:06:29.772 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:29.768+0000 7fcd9a7fc700 1 --2- 172.21.15.162:0/4256088948 >> [v2:172.21.15.162:6800/4079782599,v1:172.21.15.162:6801/4079782599] conn(0x7fcd900692a0 0x7fcd9006b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:06:29.772 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:29.769+0000 7fcd9a7fc700 1 -- 172.21.15.162:0/4256088948 <== mon.0 v2:172.21.15.162:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4634+0+0 (secure 0 0 0) 0x7fcda0088020 con 0x7fcda4102e50 2024-04-27T04:06:29.772 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:29.769+0000 7fcda997c700 1 --2- 172.21.15.162:0/4256088948 >> [v2:172.21.15.162:6800/4079782599,v1:172.21.15.162:6801/4079782599] conn(0x7fcd900692a0 0x7fcd9006b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:06:29.772 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:29.769+0000 7fcda997c700 1 --2- 172.21.15.162:0/4256088948 >> [v2:172.21.15.162:6800/4079782599,v1:172.21.15.162:6801/4079782599] conn(0x7fcd900692a0 0x7fcd9006b750 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fcd940041d0 tx=0x7fcd940043c0 comp rx=0 tx=0).ready entity=mgr.14210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:06:29.772 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:29.773+0000 7fcd9a7fc700 1 -- 172.21.15.162:0/4256088948 <== mon.0 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7fcda0054690 con 0x7fcda4102e50 2024-04-27T04:06:29.961 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:29.962+0000 7fcdabbe0700 1 -- 172.21.15.162:0/4256088948 --> [v2:172.21.15.162:6800/4079782599,v1:172.21.15.162:6801/4079782599] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fcda4109f00 con 0x7fcd900692a0 2024-04-27T04:06:29.962 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:29.963+0000 7fcd9a7fc700 1 -- 172.21.15.162:0/4256088948 <== mgr.14210 v2:172.21.15.162:6800/4079782599 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+28350 (secure 0 0 0) 0x7fcda4109f00 con 0x7fcd900692a0 2024-04-27T04:06:29.963 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-04-27T04:06:29.965 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:29.966+0000 7fcdabbe0700 1 -- 172.21.15.162:0/4256088948 >> [v2:172.21.15.162:6800/4079782599,v1:172.21.15.162:6801/4079782599] conn(0x7fcd900692a0 msgr2=0x7fcd9006b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:06:29.965 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:29.966+0000 7fcdabbe0700 1 --2- 172.21.15.162:0/4256088948 >> [v2:172.21.15.162:6800/4079782599,v1:172.21.15.162:6801/4079782599] conn(0x7fcd900692a0 0x7fcd9006b750 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fcd940041d0 tx=0x7fcd940043c0 comp rx=0 tx=0).stop 2024-04-27T04:06:29.965 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:29.966+0000 7fcdabbe0700 1 -- 172.21.15.162:0/4256088948 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fcda4102e50 msgr2=0x7fcda40fc2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:06:29.965 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:29.966+0000 7fcdabbe0700 1 --2- 172.21.15.162:0/4256088948 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fcda4102e50 0x7fcda40fc2f0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fcda000b3b0 tx=0x7fcda0009e30 comp rx=0 tx=0).stop 2024-04-27T04:06:29.965 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:29.966+0000 7fcdabbe0700 1 -- 172.21.15.162:0/4256088948 shutdown_connections 2024-04-27T04:06:29.965 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:29.966+0000 7fcdabbe0700 1 --2- 172.21.15.162:0/4256088948 >> [v2:172.21.15.162:6800/4079782599,v1:172.21.15.162:6801/4079782599] conn(0x7fcd900692a0 0x7fcd9006b750 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:06:29.965 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:29.966+0000 7fcdabbe0700 1 --2- 172.21.15.162:0/4256088948 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcda4065640 0x7fcda40fbdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:06:29.966 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:29.966+0000 7fcdabbe0700 1 --2- 172.21.15.162:0/4256088948 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fcda4102e50 0x7fcda40fc2f0 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:06:29.966 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:29.966+0000 7fcdabbe0700 1 -- 172.21.15.162:0/4256088948 >> 172.21.15.162:0/4256088948 conn(0x7fcda4071d00 msgr2=0x7fcda40fb000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:06:29.966 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:29.966+0000 7fcdabbe0700 1 -- 172.21.15.162:0/4256088948 shutdown_connections 2024-04-27T04:06:29.966 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:29.966+0000 7fcdabbe0700 1 -- 172.21.15.162:0/4256088948 wait complete. 2024-04-27T04:06:29.966 INFO:teuthology.orchestra.run.smithi162.stderr:dumped all 2024-04-27T04:06:30.138 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:06:30 smithi162 ceph-mon[26841]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-27T04:06:30.423 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:06:30 smithi181 ceph-mon[33774]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-27T04:06:30.547 INFO:teuthology.orchestra.run.smithi162.stdout:{"pg_ready":true,"pg_map":{"version":154,"stamp":"2024-04-27T04:06:28.706772+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":3,"acting":3,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":3,"num_osds":6,"num_per_pool_osds":6,"num_per_pool_omap_osds":3,"kb":562470912,"kb_used":1788036,"kb_used_data":3872,"kb_used_omap":9,"kb_used_meta":1784054,"kb_avail":560682876,"statfs":{"total":575970213888,"available":574139265024,"internally_reserved":0,"allocated":3964928,"data_stored":2605601,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":10224,"internal_metadata":1826871312},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002029"},"pg_stats":[{"pgid":"1.0","version":"23'117","reported_seq":190,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-27T04:05:37.772252+0000","last_change":"2024-04-27T04:05:12.340987+0000","last_active":"2024-04-27T04:05:37.772252+0000","last_peered":"2024-04-27T04:05:37.772252+0000","last_clean":"2024-04-27T04:05:37.772252+0000","last_became_active":"2024-04-27T04:05:12.340632+0000","last_became_peered":"2024-04-27T04:05:12.340632+0000","last_unstale":"2024-04-27T04:05:37.772252+0000","last_undegraded":"2024-04-27T04:05:37.772252+0000","last_fullsized":"2024-04-27T04:05:37.772252+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T04:04:24.972184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T04:04:24.972184+0000","last_clean_scrub_stamp":"2024-04-27T04:04:24.972184+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T14:10:09.705502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":2375680,"data_stored":2361472,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":3,"acting":3,"num_store_stats":4}],"osd_stats":[{"osd":5,"up_from":39,"seq":167503724556,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":93745152,"kb_used":297712,"kb_used_data":356,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":93447440,"statfs":{"total":95995035648,"available":95690178560,"internally_reserved":0,"allocated":364544,"data_stored":139193,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,4],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52200000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.41199999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.377}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.43099999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.39400000000000002}]}]},{"osd":4,"up_from":34,"seq":146028888082,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":93745152,"kb_used":297720,"kb_used_data":356,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":93447432,"statfs":{"total":95995035648,"available":95690170368,"internally_reserved":0,"allocated":364544,"data_stored":139193,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Apr 27 04:06:15 2024","interfaces":[{"interface":"back","average":{"1min":0.53500000000000003,"5min":0.53500000000000003,"15min":0.53500000000000003},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.66900000000000004,"5min":0.66900000000000004,"15min":0.66900000000000004},"last":0.48199999999999998},{"interface":"front","average":{"1min":0.51200000000000001,"5min":0.51200000000000001,"15min":0.51200000000000001},"min":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.59399999999999997,"5min":0.59399999999999997,"15min":0.59399999999999997},"last":0.46800000000000003}]},{"osd":1,"last update":"Sat Apr 27 04:06:15 2024","interfaces":[{"interface":"back","average":{"1min":0.50900000000000001,"5min":0.50900000000000001,"15min":0.50900000000000001},"min":{"1min":0.34999999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.67200000000000004,"5min":0.67200000000000004,"15min":0.67200000000000004},"last":0.45700000000000002},{"interface":"front","average":{"1min":0.52300000000000002,"5min":0.52300000000000002,"15min":0.52300000000000002},"min":{"1min":0.35699999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.71099999999999997,"5min":0.71099999999999997,"15min":0.71099999999999997},"last":0.50600000000000001}]},{"osd":2,"last update":"Sat Apr 27 04:06:15 2024","interfaces":[{"interface":"back","average":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.52200000000000002},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.68200000000000005,"5min":0.68200000000000005,"15min":0.68200000000000005},"last":0.42399999999999999},{"interface":"front","average":{"1min":0.52400000000000002,"5min":0.52400000000000002,"15min":0.52400000000000002},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.67300000000000004,"5min":0.67300000000000004,"15min":0.67300000000000004},"last":0.443}]},{"osd":3,"last update":"Sat Apr 27 04:06:15 2024","interfaces":[{"interface":"back","average":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"min":{"1min":0.22500000000000001,"5min":0.22500000000000001,"15min":0.22500000000000001},"max":{"1min":0.59299999999999997,"5min":0.59299999999999997,"15min":0.59299999999999997},"last":0.34000000000000002},{"interface":"front","average":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"min":{"1min":0.249,"5min":0.249,"15min":0.249},"max":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"last":0.38600000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40699999999999997}]}]},{"osd":3,"up_from":26,"seq":111669149719,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":298304,"kb_used_data":936,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":93446848,"statfs":{"total":95995035648,"available":95689572352,"internally_reserved":0,"allocated":958464,"data_stored":729561,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Apr 27 04:05:46 2024","interfaces":[{"interface":"back","average":{"1min":0.753,"5min":0.753,"15min":0.753},"min":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":5.6779999999999999,"5min":5.6779999999999999,"15min":5.6779999999999999},"last":0.60699999999999998},{"interface":"front","average":{"1min":0.48499999999999999,"5min":0.48499999999999999,"15min":0.48499999999999999},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.69099999999999995,"5min":0.69099999999999995,"15min":0.69099999999999995},"last":0.63200000000000001}]},{"osd":1,"last update":"Sat Apr 27 04:05:46 2024","interfaces":[{"interface":"back","average":{"1min":0.48999999999999999,"5min":0.48999999999999999,"15min":0.48999999999999999},"min":{"1min":0.33500000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.72499999999999998,"5min":0.72499999999999998,"15min":0.72499999999999998},"last":0.61799999999999999},{"interface":"front","average":{"1min":0.49399999999999999,"5min":0.49399999999999999,"15min":0.49399999999999999},"min":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.71399999999999997,"5min":0.71399999999999997,"15min":0.71399999999999997},"last":0.56699999999999995}]},{"osd":2,"last update":"Sat Apr 27 04:05:46 2024","interfaces":[{"interface":"back","average":{"1min":0.48499999999999999,"5min":0.48499999999999999,"15min":0.48499999999999999},"min":{"1min":0.33000000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.73299999999999998,"5min":0.73299999999999998,"15min":0.73299999999999998},"last":0.58299999999999996},{"interface":"front","average":{"1min":0.48699999999999999,"5min":0.48699999999999999,"15min":0.48699999999999999},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.68000000000000005,"5min":0.68000000000000005,"15min":0.68000000000000005},"last":0.64700000000000002}]},{"osd":4,"last update":"Sat Apr 27 04:06:12 2024","interfaces":[{"interface":"back","average":{"1min":0.37,"5min":0.37,"15min":0.37},"min":{"1min":0.29199999999999998,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.47199999999999998,"5min":0.47199999999999998,"15min":0.47199999999999998},"last":0.39500000000000002},{"interface":"front","average":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"min":{"1min":0.28199999999999997,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.52600000000000002,"5min":0.52600000000000002,"15min":0.52600000000000002},"last":0.40300000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.443}]}]},{"osd":2,"up_from":19,"seq":81604378652,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":93745152,"kb_used":297716,"kb_used_data":352,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":93447436,"statfs":{"total":95995035648,"available":95690174464,"internally_reserved":0,"allocated":360448,"data_stored":138532,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Apr 27 04:05:24 2024","interfaces":[{"interface":"back","average":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"min":{"1min":0.24199999999999999,"5min":0.24199999999999999,"15min":0.24199999999999999},"max":{"1min":0.505,"5min":0.505,"15min":0.505},"last":0.309},{"interface":"front","average":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"min":{"1min":0.27000000000000002,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.50800000000000001,"5min":0.50800000000000001,"15min":0.50800000000000001},"last":0.29699999999999999}]},{"osd":1,"last update":"Sat Apr 27 04:05:24 2024","interfaces":[{"interface":"back","average":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"min":{"1min":0.28100000000000003,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":0.54300000000000004,"5min":0.54300000000000004,"15min":0.54300000000000004},"last":0.34499999999999997},{"interface":"front","average":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"min":{"1min":0.26300000000000001,"5min":0.26300000000000001,"15min":0.26300000000000001},"max":{"1min":0.60299999999999998,"5min":0.60299999999999998,"15min":0.60299999999999998},"last":0.27300000000000002}]},{"osd":3,"last update":"Sat Apr 27 04:05:48 2024","interfaces":[{"interface":"back","average":{"1min":0.53800000000000003,"5min":0.53800000000000003,"15min":0.53800000000000003},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.72799999999999998,"5min":0.72799999999999998,"15min":0.72799999999999998},"last":0.52500000000000002},{"interface":"front","average":{"1min":0.53200000000000003,"5min":0.53200000000000003,"15min":0.53200000000000003},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.68500000000000005,"5min":0.68500000000000005,"15min":0.68500000000000005},"last":0.50600000000000001}]},{"osd":4,"last update":"Sat Apr 27 04:06:16 2024","interfaces":[{"interface":"back","average":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.52200000000000002},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.89500000000000002,"5min":0.89500000000000002,"15min":0.89500000000000002},"last":0.51500000000000001},{"interface":"front","average":{"1min":0.502,"5min":0.502,"15min":0.502},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.69499999999999995,"5min":0.69499999999999995,"15min":0.69499999999999995},"last":0.48699999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53700000000000003}]}]},{"osd":0,"up_from":9,"seq":38654705703,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":298292,"kb_used_data":936,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":93446860,"statfs":{"total":95995035648,"available":95689584640,"internally_reserved":0,"allocated":958464,"data_stored":729561,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[1,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sat Apr 27 04:06:03 2024","interfaces":[{"interface":"back","average":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"min":{"1min":0.23400000000000001,"5min":0.186,"15min":0.186},"max":{"1min":0.52000000000000002,"5min":0.57899999999999996,"15min":0.57899999999999996},"last":0.438},{"interface":"front","average":{"1min":0.35799999999999998,"5min":0.34599999999999997,"15min":0.34399999999999997},"min":{"1min":0.26300000000000001,"5min":0.191,"15min":0.191},"max":{"1min":0.54300000000000004,"5min":0.54300000000000004,"15min":0.54300000000000004},"last":0.36899999999999999}]},{"osd":2,"last update":"Sat Apr 27 04:05:25 2024","interfaces":[{"interface":"back","average":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"min":{"1min":0.29799999999999999,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.54100000000000004,"5min":0.54100000000000004,"15min":0.54100000000000004},"last":0.40400000000000003},{"interface":"front","average":{"1min":0.44,"5min":0.44,"15min":0.44},"min":{"1min":0.313,"5min":0.313,"15min":0.313},"max":{"1min":0.56499999999999995,"5min":0.56499999999999995,"15min":0.56499999999999995},"last":0.42299999999999999}]},{"osd":3,"last update":"Sat Apr 27 04:05:47 2024","interfaces":[{"interface":"back","average":{"1min":0.53800000000000003,"5min":0.53800000000000003,"15min":0.53800000000000003},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.72799999999999998,"5min":0.72799999999999998,"15min":0.72799999999999998},"last":0.58699999999999997},{"interface":"front","average":{"1min":0.55900000000000005,"5min":0.55900000000000005,"15min":0.55900000000000005},"min":{"1min":0.42799999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":0.71299999999999997,"5min":0.71299999999999997,"15min":0.71299999999999997},"last":0.56499999999999995}]},{"osd":4,"last update":"Sat Apr 27 04:06:13 2024","interfaces":[{"interface":"back","average":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.71199999999999997,"5min":0.71199999999999997,"15min":0.71199999999999997},"last":0.60099999999999998},{"interface":"front","average":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.73499999999999999,"5min":0.73499999999999999,"15min":0.73499999999999999},"last":0.53300000000000003}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51500000000000001}]}]},{"osd":1,"up_from":14,"seq":60129542178,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":298292,"kb_used_data":936,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":93446860,"statfs":{"total":95995035648,"available":95689584640,"internally_reserved":0,"allocated":958464,"data_stored":729561,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Apr 27 04:06:00 2024","interfaces":[{"interface":"back","average":{"1min":0.35299999999999998,"5min":0.33300000000000002,"15min":0.33000000000000002},"min":{"1min":0.245,"5min":0.245,"15min":0.245},"max":{"1min":0.49199999999999999,"5min":0.53300000000000003,"15min":0.53300000000000003},"last":0.36599999999999999},{"interface":"front","average":{"1min":0.36599999999999999,"5min":0.33900000000000002,"15min":0.33500000000000002},"min":{"1min":0.248,"5min":0.22600000000000001,"15min":0.22600000000000001},"max":{"1min":0.52000000000000002,"5min":0.54900000000000004,"15min":0.54900000000000004},"last":0.42599999999999999}]},{"osd":2,"last update":"Sat Apr 27 04:05:25 2024","interfaces":[{"interface":"back","average":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"min":{"1min":0.24099999999999999,"5min":0.24099999999999999,"15min":0.24099999999999999},"max":{"1min":0.54300000000000004,"5min":0.54300000000000004,"15min":0.54300000000000004},"last":0.40899999999999997},{"interface":"front","average":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"min":{"1min":0.26300000000000001,"5min":0.26300000000000001,"15min":0.26300000000000001},"max":{"1min":0.55100000000000005,"5min":0.55100000000000005,"15min":0.55100000000000005},"last":0.39600000000000002}]},{"osd":3,"last update":"Sat Apr 27 04:05:46 2024","interfaces":[{"interface":"back","average":{"1min":0.57899999999999996,"5min":0.57899999999999996,"15min":0.57899999999999996},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.747,"5min":0.747,"15min":0.747},"last":0.72199999999999998},{"interface":"front","average":{"1min":0.58199999999999996,"5min":0.58199999999999996,"15min":0.58199999999999996},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.71399999999999997,"5min":0.71399999999999997,"15min":0.71399999999999997},"last":0.72999999999999998}]},{"osd":4,"last update":"Sat Apr 27 04:06:16 2024","interfaces":[{"interface":"back","average":{"1min":0.56299999999999994,"5min":0.56299999999999994,"15min":0.56299999999999994},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.753,"5min":0.753,"15min":0.753},"last":0.52700000000000002},{"interface":"front","average":{"1min":0.57799999999999996,"5min":0.57799999999999996,"15min":0.57799999999999996},"min":{"1min":0.441,"5min":0.441,"15min":0.441},"max":{"1min":0.78400000000000003,"5min":0.78400000000000003,"15min":0.78400000000000003},"last":0.67800000000000005}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63500000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-27T04:06:30.548 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2024-04-27T04:06:30.549 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-04-27T04:06:30.549 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy 2024-04-27T04:06:30.549 DEBUG:teuthology.orchestra.run.smithi162:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell --fsid 7f9750d6-044a-11ef-bc93-c7b262605968 -- ceph health --format=json 2024-04-27T04:06:30.805 INFO:teuthology.orchestra.run.smithi162.stderr:Inferring config /var/lib/ceph/7f9750d6-044a-11ef-bc93-c7b262605968/mon.smithi162/config 2024-04-27T04:06:31.235 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:06:31 smithi162 ceph-mon[26841]: from='client.14420 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-27T04:06:31.423 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:06:31 smithi181 ceph-mon[33774]: from='client.14420 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-27T04:06:32.075 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:06:32 smithi162 ceph-mon[26841]: pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-27T04:06:32.075 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:06:32 smithi162 ceph-mon[26841]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-27T04:06:32.324 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:06:32 smithi181 ceph-mon[33774]: pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-27T04:06:32.324 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:06:32 smithi181 ceph-mon[33774]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-27T04:06:32.333 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:32.314+0000 7fb9acd4b700 1 -- 172.21.15.162:0/2262848828 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb9a80fc590 msgr2=0x7fb9a8100bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:06:32.333 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:32.314+0000 7fb9acd4b700 1 --2- 172.21.15.162:0/2262848828 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb9a80fc590 0x7fb9a8100bb0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fb99c004940 tx=0x7fb99c009e30 comp rx=0 tx=0).stop 2024-04-27T04:06:32.334 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:32.314+0000 7fb9acd4b700 1 -- 172.21.15.162:0/2262848828 shutdown_connections 2024-04-27T04:06:32.334 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:32.314+0000 7fb9acd4b700 1 --2- 172.21.15.162:0/2262848828 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb9a80fc590 0x7fb9a8100bb0 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:06:32.334 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:32.314+0000 7fb9acd4b700 1 --2- 172.21.15.162:0/2262848828 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb9a80fbbf0 0x7fb9a80fbfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:06:32.334 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:32.314+0000 7fb9acd4b700 1 -- 172.21.15.162:0/2262848828 >> 172.21.15.162:0/2262848828 conn(0x7fb9a80f7870 msgr2=0x7fb9a80f9c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:06:32.334 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:32.314+0000 7fb9acd4b700 1 -- 172.21.15.162:0/2262848828 shutdown_connections 2024-04-27T04:06:32.334 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:32.314+0000 7fb9acd4b700 1 -- 172.21.15.162:0/2262848828 wait complete. 2024-04-27T04:06:32.334 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:32.314+0000 7fb9acd4b700 1 Processor -- start 2024-04-27T04:06:32.334 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:32.314+0000 7fb9acd4b700 1 -- start start 2024-04-27T04:06:32.334 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:32.314+0000 7fb9acd4b700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb9a80fbbf0 0x7fb9a8119f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:06:32.334 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:32.314+0000 7fb9acd4b700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb9a80fc590 0x7fb9a811a4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:06:32.334 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:32.314+0000 7fb9acd4b700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9a811ab10 con 0x7fb9a80fc590 2024-04-27T04:06:32.334 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:32.314+0000 7fb9acd4b700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9a81140f0 con 0x7fb9a80fbbf0 2024-04-27T04:06:32.334 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:32.315+0000 7fb9a5d9b700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb9a80fc590 0x7fb9a811a4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:06:32.335 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:32.315+0000 7fb9a5d9b700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb9a80fc590 0x7fb9a811a4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:56854/0 (socket says 172.21.15.162:56854) 2024-04-27T04:06:32.335 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:32.315+0000 7fb9a5d9b700 1 -- 172.21.15.162:0/533397756 learned_addr learned my addr 172.21.15.162:0/533397756 (peer_addr_for_me v2:172.21.15.162:0/0) 2024-04-27T04:06:32.335 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:32.315+0000 7fb9a659c700 1 --2- 172.21.15.162:0/533397756 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb9a80fbbf0 0x7fb9a8119f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:06:32.335 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:32.315+0000 7fb9a5d9b700 1 -- 172.21.15.162:0/533397756 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb9a80fbbf0 msgr2=0x7fb9a8119f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:06:32.335 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:32.315+0000 7fb9a5d9b700 1 --2- 172.21.15.162:0/533397756 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb9a80fbbf0 0x7fb9a8119f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:06:32.335 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:32.315+0000 7fb9a5d9b700 1 -- 172.21.15.162:0/533397756 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb99c005040 con 0x7fb9a80fc590 2024-04-27T04:06:32.335 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:32.315+0000 7fb9a5d9b700 1 --2- 172.21.15.162:0/533397756 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb9a80fc590 0x7fb9a811a4c0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fb99c009180 tx=0x7fb99c009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:06:32.335 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:32.315+0000 7fb996ffd700 1 -- 172.21.15.162:0/533397756 <== mon.0 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb99c00e040 con 0x7fb9a80fc590 2024-04-27T04:06:32.335 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:32.315+0000 7fb9acd4b700 1 -- 172.21.15.162:0/533397756 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb9a8114370 con 0x7fb9a80fc590 2024-04-27T04:06:32.335 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:32.315+0000 7fb996ffd700 1 -- 172.21.15.162:0/533397756 <== mon.0 v2:172.21.15.162:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fb99c013420 con 0x7fb9a80fc590 2024-04-27T04:06:32.335 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:32.315+0000 7fb996ffd700 1 -- 172.21.15.162:0/533397756 <== mon.0 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb99c01c630 con 0x7fb9a80fc590 2024-04-27T04:06:32.335 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:32.315+0000 7fb9acd4b700 1 -- 172.21.15.162:0/533397756 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb9a81148b0 con 0x7fb9a80fc590 2024-04-27T04:06:32.335 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:32.316+0000 7fb9acd4b700 1 -- 172.21.15.162:0/533397756 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb9a8066b30 con 0x7fb9a80fc590 2024-04-27T04:06:32.335 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:32.317+0000 7fb996ffd700 1 -- 172.21.15.162:0/533397756 <== mon.0 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 19) v1 ==== 86631+0+0 (secure 0 0 0) 0x7fb99c013590 con 0x7fb9a80fc590 2024-04-27T04:06:32.336 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:32.317+0000 7fb996ffd700 1 --2- 172.21.15.162:0/533397756 >> [v2:172.21.15.162:6800/4079782599,v1:172.21.15.162:6801/4079782599] conn(0x7fb9900692a0 0x7fb99006b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:06:32.336 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:32.317+0000 7fb996ffd700 1 -- 172.21.15.162:0/533397756 <== mon.0 v2:172.21.15.162:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4634+0+0 (secure 0 0 0) 0x7fb99c088050 con 0x7fb9a80fc590 2024-04-27T04:06:32.336 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:32.317+0000 7fb9a659c700 1 --2- 172.21.15.162:0/533397756 >> [v2:172.21.15.162:6800/4079782599,v1:172.21.15.162:6801/4079782599] conn(0x7fb9900692a0 0x7fb99006b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:06:32.336 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:32.317+0000 7fb9a659c700 1 --2- 172.21.15.162:0/533397756 >> [v2:172.21.15.162:6800/4079782599,v1:172.21.15.162:6801/4079782599] conn(0x7fb9900692a0 0x7fb99006b750 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fb998009fd0 tx=0x7fb99800b040 comp rx=0 tx=0).ready entity=mgr.14210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:06:32.336 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:32.321+0000 7fb996ffd700 1 -- 172.21.15.162:0/533397756 <== mon.0 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7fb99c058150 con 0x7fb9a80fc590 2024-04-27T04:06:32.550 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:32.551+0000 7fb9acd4b700 1 -- 172.21.15.162:0/533397756 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7fb9a804c710 con 0x7fb9a80fc590 2024-04-27T04:06:32.551 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:32.552+0000 7fb996ffd700 1 -- 172.21.15.162:0/533397756 <== mon.0 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+46 (secure 0 0 0) 0x7fb99c057ce0 con 0x7fb9a80fc590 2024-04-27T04:06:32.551 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-04-27T04:06:32.551 INFO:teuthology.orchestra.run.smithi162.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-04-27T04:06:32.553 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:32.554+0000 7fb9acd4b700 1 -- 172.21.15.162:0/533397756 >> [v2:172.21.15.162:6800/4079782599,v1:172.21.15.162:6801/4079782599] conn(0x7fb9900692a0 msgr2=0x7fb99006b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:06:32.553 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:32.554+0000 7fb9acd4b700 1 --2- 172.21.15.162:0/533397756 >> [v2:172.21.15.162:6800/4079782599,v1:172.21.15.162:6801/4079782599] conn(0x7fb9900692a0 0x7fb99006b750 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fb998009fd0 tx=0x7fb99800b040 comp rx=0 tx=0).stop 2024-04-27T04:06:32.553 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:32.554+0000 7fb9acd4b700 1 -- 172.21.15.162:0/533397756 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb9a80fc590 msgr2=0x7fb9a811a4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:06:32.553 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:32.554+0000 7fb9acd4b700 1 --2- 172.21.15.162:0/533397756 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb9a80fc590 0x7fb9a811a4c0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fb99c009180 tx=0x7fb99c009e30 comp rx=0 tx=0).stop 2024-04-27T04:06:32.553 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:32.554+0000 7fb9acd4b700 1 -- 172.21.15.162:0/533397756 shutdown_connections 2024-04-27T04:06:32.553 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:32.554+0000 7fb9acd4b700 1 --2- 172.21.15.162:0/533397756 >> [v2:172.21.15.162:6800/4079782599,v1:172.21.15.162:6801/4079782599] conn(0x7fb9900692a0 0x7fb99006b750 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:06:32.553 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:32.554+0000 7fb9acd4b700 1 --2- 172.21.15.162:0/533397756 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb9a80fbbf0 0x7fb9a8119f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:06:32.553 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:32.554+0000 7fb9acd4b700 1 --2- 172.21.15.162:0/533397756 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb9a80fc590 0x7fb9a811a4c0 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:06:32.554 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:32.554+0000 7fb9acd4b700 1 -- 172.21.15.162:0/533397756 >> 172.21.15.162:0/533397756 conn(0x7fb9a80f7870 msgr2=0x7fb9a810cb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:06:32.554 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:32.554+0000 7fb9acd4b700 1 -- 172.21.15.162:0/533397756 shutdown_connections 2024-04-27T04:06:32.554 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:32.554+0000 7fb9acd4b700 1 -- 172.21.15.162:0/533397756 wait complete. 2024-04-27T04:06:33.237 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:06:33 smithi162 ceph-mon[26841]: from='client.? 172.21.15.162:0/533397756' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-04-27T04:06:33.265 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy done 2024-04-27T04:06:33.265 INFO:tasks.cephadm:Setup complete, yielding 2024-04-27T04:06:33.266 INFO:teuthology.run_tasks:Running task print... 2024-04-27T04:06:33.275 INFO:teuthology.task.print:**** done end installing quincy cephadm ... 2024-04-27T04:06:33.275 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-04-27T04:06:33.284 INFO:tasks.cephadm:Running commands on role host.a host ubuntu@smithi162.front.sepia.ceph.com 2024-04-27T04:06:33.284 DEBUG:teuthology.orchestra.run.smithi162:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 7f9750d6-044a-11ef-bc93-c7b262605968 -- bash -c 'ceph config set mgr mgr/cephadm/use_repo_digest true --force' 2024-04-27T04:06:33.423 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:06:33 smithi181 ceph-mon[33774]: from='client.? 172.21.15.162:0/533397756' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-04-27T04:06:33.542 INFO:teuthology.orchestra.run.smithi162.stderr:Inferring config /var/lib/ceph/7f9750d6-044a-11ef-bc93-c7b262605968/mon.smithi162/config 2024-04-27T04:06:34.319 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:06:34 smithi162 ceph-mon[26841]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-27T04:06:34.422 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:06:34 smithi181 ceph-mon[33774]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-27T04:06:35.160 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:35.159+0000 7fd4d2184700 1 -- 172.21.15.162:0/1289838517 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd4cc0fcf60 msgr2=0x7fd4cc0fd3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:06:35.160 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:35.159+0000 7fd4d2184700 1 --2- 172.21.15.162:0/1289838517 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd4cc0fcf60 0x7fd4cc0fd3d0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fd4b4004940 tx=0x7fd4b4009e30 comp rx=0 tx=0).stop 2024-04-27T04:06:35.160 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:35.159+0000 7fd4d2184700 1 -- 172.21.15.162:0/1289838517 shutdown_connections 2024-04-27T04:06:35.160 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:35.159+0000 7fd4d2184700 1 --2- 172.21.15.162:0/1289838517 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd4cc0fcf60 0x7fd4cc0fd3d0 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:06:35.160 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:35.159+0000 7fd4d2184700 1 --2- 172.21.15.162:0/1289838517 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd4cc0fbcb0 0x7fd4cc0fc0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:06:35.160 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:35.159+0000 7fd4d2184700 1 -- 172.21.15.162:0/1289838517 >> 172.21.15.162:0/1289838517 conn(0x7fd4cc0f7430 msgr2=0x7fd4cc0f9840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:06:35.160 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:35.159+0000 7fd4d2184700 1 -- 172.21.15.162:0/1289838517 shutdown_connections 2024-04-27T04:06:35.161 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:35.159+0000 7fd4d2184700 1 -- 172.21.15.162:0/1289838517 wait complete. 2024-04-27T04:06:35.161 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:35.160+0000 7fd4d2184700 1 Processor -- start 2024-04-27T04:06:35.161 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:35.160+0000 7fd4d2184700 1 -- start start 2024-04-27T04:06:35.161 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:35.160+0000 7fd4d2184700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd4cc0fbcb0 0x7fd4cc10c930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:06:35.161 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:35.160+0000 7fd4d2184700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd4cc0fcf60 0x7fd4cc10ce70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:06:35.161 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:35.160+0000 7fd4d2184700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4cc10d470 con 0x7fd4cc0fcf60 2024-04-27T04:06:35.161 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:35.160+0000 7fd4d2184700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4cc10dcc0 con 0x7fd4cc0fbcb0 2024-04-27T04:06:35.161 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:35.160+0000 7fd4c3fff700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd4cc0fcf60 0x7fd4cc10ce70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:06:35.161 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:35.160+0000 7fd4c3fff700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd4cc0fcf60 0x7fd4cc10ce70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:42002/0 (socket says 172.21.15.162:42002) 2024-04-27T04:06:35.161 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:35.160+0000 7fd4c3fff700 1 -- 172.21.15.162:0/4044289413 learned_addr learned my addr 172.21.15.162:0/4044289413 (peer_addr_for_me v2:172.21.15.162:0/0) 2024-04-27T04:06:35.161 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:35.160+0000 7fd4cb7fe700 1 --2- 172.21.15.162:0/4044289413 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd4cc0fbcb0 0x7fd4cc10c930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:06:35.161 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:35.160+0000 7fd4c3fff700 1 -- 172.21.15.162:0/4044289413 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd4cc0fbcb0 msgr2=0x7fd4cc10c930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:06:35.161 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:35.160+0000 7fd4c3fff700 1 --2- 172.21.15.162:0/4044289413 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd4cc0fbcb0 0x7fd4cc10c930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:06:35.161 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:35.160+0000 7fd4c3fff700 1 -- 172.21.15.162:0/4044289413 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd4b4005040 con 0x7fd4cc0fcf60 2024-04-27T04:06:35.161 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:35.160+0000 7fd4cb7fe700 1 --2- 172.21.15.162:0/4044289413 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd4cc0fbcb0 0x7fd4cc10c930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-27T04:06:35.162 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:35.161+0000 7fd4c3fff700 1 --2- 172.21.15.162:0/4044289413 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd4cc0fcf60 0x7fd4cc10ce70 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fd4b400b3b0 tx=0x7fd4b4009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:06:35.162 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:35.161+0000 7fd4c8ff9700 1 -- 172.21.15.162:0/4044289413 <== mon.0 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd4b400e040 con 0x7fd4cc0fcf60 2024-04-27T04:06:35.162 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:35.161+0000 7fd4c8ff9700 1 -- 172.21.15.162:0/4044289413 <== mon.0 v2:172.21.15.162:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fd4b400bcc0 con 0x7fd4cc0fcf60 2024-04-27T04:06:35.162 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:35.161+0000 7fd4c8ff9700 1 -- 172.21.15.162:0/4044289413 <== mon.0 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd4b401c630 con 0x7fd4cc0fcf60 2024-04-27T04:06:35.162 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:35.161+0000 7fd4d2184700 1 -- 172.21.15.162:0/4044289413 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd4cc10df40 con 0x7fd4cc0fcf60 2024-04-27T04:06:35.162 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:35.161+0000 7fd4d2184700 1 -- 172.21.15.162:0/4044289413 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd4cc10e480 con 0x7fd4cc0fcf60 2024-04-27T04:06:35.163 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:35.162+0000 7fd4d2184700 1 -- 172.21.15.162:0/4044289413 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd4cc05f5c0 con 0x7fd4cc0fcf60 2024-04-27T04:06:35.163 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:35.162+0000 7fd4c8ff9700 1 -- 172.21.15.162:0/4044289413 <== mon.0 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 19) v1 ==== 86631+0+0 (secure 0 0 0) 0x7fd4b4013980 con 0x7fd4cc0fcf60 2024-04-27T04:06:35.163 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:35.162+0000 7fd4c8ff9700 1 --2- 172.21.15.162:0/4044289413 >> [v2:172.21.15.162:6800/4079782599,v1:172.21.15.162:6801/4079782599] conn(0x7fd4b80692a0 0x7fd4b806b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:06:35.163 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:35.163+0000 7fd4c8ff9700 1 -- 172.21.15.162:0/4044289413 <== mon.0 v2:172.21.15.162:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4634+0+0 (secure 0 0 0) 0x7fd4b4087bd0 con 0x7fd4cc0fcf60 2024-04-27T04:06:35.163 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:35.163+0000 7fd4cb7fe700 1 --2- 172.21.15.162:0/4044289413 >> [v2:172.21.15.162:6800/4079782599,v1:172.21.15.162:6801/4079782599] conn(0x7fd4b80692a0 0x7fd4b806b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:06:35.163 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:35.163+0000 7fd4cb7fe700 1 --2- 172.21.15.162:0/4044289413 >> [v2:172.21.15.162:6800/4079782599,v1:172.21.15.162:6801/4079782599] conn(0x7fd4b80692a0 0x7fd4b806b750 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fd4bc007fd0 tx=0x7fd4bc009040 comp rx=0 tx=0).ready entity=mgr.14210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:06:35.166 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:35.167+0000 7fd4c8ff9700 1 -- 172.21.15.162:0/4044289413 <== mon.0 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7fd4b4057cd0 con 0x7fd4cc0fcf60 2024-04-27T04:06:35.352 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:35.353+0000 7fd4d2184700 1 -- 172.21.15.162:0/4044289413 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/use_repo_digest}] v 0) v1 -- 0x7fd4cc04bed0 con 0x7fd4cc0fcf60 2024-04-27T04:06:35.356 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:35.357+0000 7fd4c8ff9700 1 -- 172.21.15.162:0/4044289413 <== mon.0 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/use_repo_digest}]=0 v20)=0 v20) v1 ==== 143+0+0 (secure 0 0 0) 0x7fd4b4057860 con 0x7fd4cc0fcf60 2024-04-27T04:06:35.359 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:35.360+0000 7fd4d2184700 1 -- 172.21.15.162:0/4044289413 >> [v2:172.21.15.162:6800/4079782599,v1:172.21.15.162:6801/4079782599] conn(0x7fd4b80692a0 msgr2=0x7fd4b806b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:06:35.359 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:35.360+0000 7fd4d2184700 1 --2- 172.21.15.162:0/4044289413 >> [v2:172.21.15.162:6800/4079782599,v1:172.21.15.162:6801/4079782599] conn(0x7fd4b80692a0 0x7fd4b806b750 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fd4bc007fd0 tx=0x7fd4bc009040 comp rx=0 tx=0).stop 2024-04-27T04:06:35.359 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:35.360+0000 7fd4d2184700 1 -- 172.21.15.162:0/4044289413 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd4cc0fcf60 msgr2=0x7fd4cc10ce70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:06:35.359 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:35.360+0000 7fd4d2184700 1 --2- 172.21.15.162:0/4044289413 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd4cc0fcf60 0x7fd4cc10ce70 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fd4b400b3b0 tx=0x7fd4b4009e30 comp rx=0 tx=0).stop 2024-04-27T04:06:35.359 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:35.360+0000 7fd4d2184700 1 -- 172.21.15.162:0/4044289413 shutdown_connections 2024-04-27T04:06:35.359 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:35.360+0000 7fd4d2184700 1 --2- 172.21.15.162:0/4044289413 >> [v2:172.21.15.162:6800/4079782599,v1:172.21.15.162:6801/4079782599] conn(0x7fd4b80692a0 0x7fd4b806b750 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:06:35.359 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:35.360+0000 7fd4d2184700 1 --2- 172.21.15.162:0/4044289413 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd4cc0fbcb0 0x7fd4cc10c930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:06:35.359 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:35.360+0000 7fd4d2184700 1 --2- 172.21.15.162:0/4044289413 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd4cc0fcf60 0x7fd4cc10ce70 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:06:35.359 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:35.360+0000 7fd4d2184700 1 -- 172.21.15.162:0/4044289413 >> 172.21.15.162:0/4044289413 conn(0x7fd4cc0f7430 msgr2=0x7fd4cc0f8b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:06:35.359 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:35.360+0000 7fd4d2184700 1 -- 172.21.15.162:0/4044289413 shutdown_connections 2024-04-27T04:06:35.360 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:35.360+0000 7fd4d2184700 1 -- 172.21.15.162:0/4044289413 wait complete. 2024-04-27T04:06:36.025 INFO:teuthology.run_tasks:Running task print... 2024-04-27T04:06:36.036 INFO:teuthology.task.print:**** done cephadm.shell ceph config set mgr... 2024-04-27T04:06:36.036 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-04-27T04:06:36.086 INFO:tasks.cephadm:Running commands on role host.a host ubuntu@smithi162.front.sepia.ceph.com 2024-04-27T04:06:36.087 DEBUG:teuthology.orchestra.run.smithi162:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 7f9750d6-044a-11ef-bc93-c7b262605968 -- bash -c 'ceph orch status' 2024-04-27T04:06:36.340 INFO:teuthology.orchestra.run.smithi162.stderr:Inferring config /var/lib/ceph/7f9750d6-044a-11ef-bc93-c7b262605968/mon.smithi162/config 2024-04-27T04:06:36.387 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:06:36 smithi162 ceph-mon[26841]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-27T04:06:36.388 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:06:36 smithi162 ceph-mon[26841]: from='client.? 172.21.15.162:0/4044289413' entity='client.admin' 2024-04-27T04:06:36.388 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:06:36 smithi162 ceph-mon[26841]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T04:06:36.388 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:06:36 smithi162 ceph-mon[26841]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T04:06:36.388 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:06:36 smithi162 ceph-mon[26841]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T04:06:36.388 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:06:36 smithi162 ceph-mon[26841]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' 2024-04-27T04:06:36.423 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:06:36 smithi181 ceph-mon[33774]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-27T04:06:36.423 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:06:36 smithi181 ceph-mon[33774]: from='client.? 172.21.15.162:0/4044289413' entity='client.admin' 2024-04-27T04:06:36.423 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:06:36 smithi181 ceph-mon[33774]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T04:06:36.423 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:06:36 smithi181 ceph-mon[33774]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T04:06:36.423 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:06:36 smithi181 ceph-mon[33774]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T04:06:36.423 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:06:36 smithi181 ceph-mon[33774]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' 2024-04-27T04:06:38.005 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:38.004+0000 7fb02c3f9700 1 -- 172.21.15.162:0/1345527438 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb0240fcf70 msgr2=0x7fb0240fd3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:06:38.005 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:38.004+0000 7fb02c3f9700 1 --2- 172.21.15.162:0/1345527438 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb0240fcf70 0x7fb0240fd3e0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fb014004940 tx=0x7fb014009e30 comp rx=0 tx=0).stop 2024-04-27T04:06:38.005 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:38.004+0000 7fb02c3f9700 1 -- 172.21.15.162:0/1345527438 shutdown_connections 2024-04-27T04:06:38.005 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:38.004+0000 7fb02c3f9700 1 --2- 172.21.15.162:0/1345527438 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb0240fcf70 0x7fb0240fd3e0 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:06:38.005 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:38.004+0000 7fb02c3f9700 1 --2- 172.21.15.162:0/1345527438 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb0240fbcc0 0x7fb0240fc0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:06:38.005 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:38.004+0000 7fb02c3f9700 1 -- 172.21.15.162:0/1345527438 >> 172.21.15.162:0/1345527438 conn(0x7fb0240f7440 msgr2=0x7fb0240f9850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:06:38.005 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:38.004+0000 7fb02c3f9700 1 -- 172.21.15.162:0/1345527438 shutdown_connections 2024-04-27T04:06:38.005 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:38.004+0000 7fb02c3f9700 1 -- 172.21.15.162:0/1345527438 wait complete. 2024-04-27T04:06:38.006 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:38.005+0000 7fb02c3f9700 1 Processor -- start 2024-04-27T04:06:38.006 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:38.005+0000 7fb02c3f9700 1 -- start start 2024-04-27T04:06:38.006 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:38.005+0000 7fb02c3f9700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb0240fbcc0 0x7fb02410d980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:06:38.006 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:38.005+0000 7fb02c3f9700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb0240fcf70 0x7fb02410dec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:06:38.006 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:38.005+0000 7fb02c3f9700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb02410e4c0 con 0x7fb0240fcf70 2024-04-27T04:06:38.006 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:38.005+0000 7fb02c3f9700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb02410e630 con 0x7fb0240fbcc0 2024-04-27T04:06:38.006 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:38.005+0000 7fb029994700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb0240fcf70 0x7fb02410dec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:06:38.006 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:38.005+0000 7fb029994700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb0240fcf70 0x7fb02410dec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:42012/0 (socket says 172.21.15.162:42012) 2024-04-27T04:06:38.006 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:38.005+0000 7fb029994700 1 -- 172.21.15.162:0/3127433299 learned_addr learned my addr 172.21.15.162:0/3127433299 (peer_addr_for_me v2:172.21.15.162:0/0) 2024-04-27T04:06:38.006 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:38.005+0000 7fb029994700 1 -- 172.21.15.162:0/3127433299 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb0240fbcc0 msgr2=0x7fb02410d980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:06:38.006 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:38.005+0000 7fb029994700 1 --2- 172.21.15.162:0/3127433299 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb0240fbcc0 0x7fb02410d980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:06:38.006 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:38.005+0000 7fb029994700 1 -- 172.21.15.162:0/3127433299 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb014005040 con 0x7fb0240fcf70 2024-04-27T04:06:38.006 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:38.006+0000 7fb029994700 1 --2- 172.21.15.162:0/3127433299 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb0240fcf70 0x7fb02410dec0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fb01400b3b0 tx=0x7fb014009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:06:38.007 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:38.006+0000 7fb01affd700 1 -- 172.21.15.162:0/3127433299 <== mon.0 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb01400e040 con 0x7fb0240fcf70 2024-04-27T04:06:38.007 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:38.006+0000 7fb02c3f9700 1 -- 172.21.15.162:0/3127433299 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb024112fe0 con 0x7fb0240fcf70 2024-04-27T04:06:38.007 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:38.006+0000 7fb01affd700 1 -- 172.21.15.162:0/3127433299 <== mon.0 v2:172.21.15.162:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fb01400bcc0 con 0x7fb0240fcf70 2024-04-27T04:06:38.007 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:38.006+0000 7fb01affd700 1 -- 172.21.15.162:0/3127433299 <== mon.0 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb01401c630 con 0x7fb0240fcf70 2024-04-27T04:06:38.007 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:38.006+0000 7fb02c3f9700 1 -- 172.21.15.162:0/3127433299 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb024113570 con 0x7fb0240fcf70 2024-04-27T04:06:38.008 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:38.007+0000 7fb02c3f9700 1 -- 172.21.15.162:0/3127433299 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb024101b00 con 0x7fb0240fcf70 2024-04-27T04:06:38.008 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:38.007+0000 7fb01affd700 1 -- 172.21.15.162:0/3127433299 <== mon.0 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 19) v1 ==== 86631+0+0 (secure 0 0 0) 0x7fb014013980 con 0x7fb0240fcf70 2024-04-27T04:06:38.008 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:38.008+0000 7fb01affd700 1 --2- 172.21.15.162:0/3127433299 >> [v2:172.21.15.162:6800/4079782599,v1:172.21.15.162:6801/4079782599] conn(0x7fb0100692a0 0x7fb01006b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:06:38.008 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:38.008+0000 7fb01affd700 1 -- 172.21.15.162:0/3127433299 <== mon.0 v2:172.21.15.162:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4634+0+0 (secure 0 0 0) 0x7fb014087c40 con 0x7fb0240fcf70 2024-04-27T04:06:38.008 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:38.008+0000 7fb02a195700 1 --2- 172.21.15.162:0/3127433299 >> [v2:172.21.15.162:6800/4079782599,v1:172.21.15.162:6801/4079782599] conn(0x7fb0100692a0 0x7fb01006b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:06:38.008 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:38.008+0000 7fb02a195700 1 --2- 172.21.15.162:0/3127433299 >> [v2:172.21.15.162:6800/4079782599,v1:172.21.15.162:6801/4079782599] conn(0x7fb0100692a0 0x7fb01006b750 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fb020004d00 tx=0x7fb020009ce0 comp rx=0 tx=0).ready entity=mgr.14210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:06:38.011 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:38.012+0000 7fb01affd700 1 -- 172.21.15.162:0/3127433299 <== mon.0 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7fb014057cc0 con 0x7fb0240fcf70 2024-04-27T04:06:38.137 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:06:38 smithi162 ceph-mon[26841]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-27T04:06:38.217 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:38.217+0000 7fb02c3f9700 1 -- 172.21.15.162:0/3127433299 --> [v2:172.21.15.162:6800/4079782599,v1:172.21.15.162:6801/4079782599] -- mgr_command(tid 0: {"prefix": "orch status", "target": ["mon-mgr", ""]}) v1 -- 0x7fb02402b450 con 0x7fb0100692a0 2024-04-27T04:06:38.218 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:38.219+0000 7fb01affd700 1 -- 172.21.15.162:0/3127433299 <== mgr.14210 v2:172.21.15.162:6800/4079782599 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+43 (secure 0 0 0) 0x7fb02402b450 con 0x7fb0100692a0 2024-04-27T04:06:38.218 INFO:teuthology.orchestra.run.smithi162.stdout:Backend: cephadm 2024-04-27T04:06:38.218 INFO:teuthology.orchestra.run.smithi162.stdout:Available: Yes 2024-04-27T04:06:38.218 INFO:teuthology.orchestra.run.smithi162.stdout:Paused: No 2024-04-27T04:06:38.220 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:38.221+0000 7fb02c3f9700 1 -- 172.21.15.162:0/3127433299 >> [v2:172.21.15.162:6800/4079782599,v1:172.21.15.162:6801/4079782599] conn(0x7fb0100692a0 msgr2=0x7fb01006b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:06:38.220 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:38.221+0000 7fb02c3f9700 1 --2- 172.21.15.162:0/3127433299 >> [v2:172.21.15.162:6800/4079782599,v1:172.21.15.162:6801/4079782599] conn(0x7fb0100692a0 0x7fb01006b750 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fb020004d00 tx=0x7fb020009ce0 comp rx=0 tx=0).stop 2024-04-27T04:06:38.220 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:38.221+0000 7fb02c3f9700 1 -- 172.21.15.162:0/3127433299 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb0240fcf70 msgr2=0x7fb02410dec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:06:38.220 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:38.221+0000 7fb02c3f9700 1 --2- 172.21.15.162:0/3127433299 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb0240fcf70 0x7fb02410dec0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fb01400b3b0 tx=0x7fb014009e30 comp rx=0 tx=0).stop 2024-04-27T04:06:38.220 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:38.221+0000 7fb02c3f9700 1 -- 172.21.15.162:0/3127433299 shutdown_connections 2024-04-27T04:06:38.220 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:38.221+0000 7fb02c3f9700 1 --2- 172.21.15.162:0/3127433299 >> [v2:172.21.15.162:6800/4079782599,v1:172.21.15.162:6801/4079782599] conn(0x7fb0100692a0 0x7fb01006b750 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:06:38.220 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:38.221+0000 7fb02c3f9700 1 --2- 172.21.15.162:0/3127433299 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb0240fbcc0 0x7fb02410d980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:06:38.220 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:38.221+0000 7fb02c3f9700 1 --2- 172.21.15.162:0/3127433299 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb0240fcf70 0x7fb02410dec0 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:06:38.220 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:38.221+0000 7fb02c3f9700 1 -- 172.21.15.162:0/3127433299 >> 172.21.15.162:0/3127433299 conn(0x7fb0240f7440 msgr2=0x7fb0240667e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:06:38.221 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:38.221+0000 7fb02c3f9700 1 -- 172.21.15.162:0/3127433299 shutdown_connections 2024-04-27T04:06:38.221 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:38.221+0000 7fb02c3f9700 1 -- 172.21.15.162:0/3127433299 wait complete. 2024-04-27T04:06:38.423 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:06:38 smithi181 ceph-mon[33774]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-27T04:06:38.975 DEBUG:teuthology.orchestra.run.smithi162:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 7f9750d6-044a-11ef-bc93-c7b262605968 -- bash -c 'ceph orch ps' 2024-04-27T04:06:39.231 INFO:teuthology.orchestra.run.smithi162.stderr:Inferring config /var/lib/ceph/7f9750d6-044a-11ef-bc93-c7b262605968/mon.smithi162/config 2024-04-27T04:06:39.334 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:06:39 smithi162 ceph-mon[26841]: from='client.14432 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-27T04:06:39.422 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:06:39 smithi181 ceph-mon[33774]: from='client.14432 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-27T04:06:40.387 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:06:40 smithi162 ceph-mon[26841]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-27T04:06:40.422 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:06:40 smithi181 ceph-mon[33774]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-27T04:06:40.936 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:40.935+0000 7fd379968700 1 -- 172.21.15.162:0/3300380426 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd3740fcf70 msgr2=0x7fd3740fd3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:06:40.936 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:40.935+0000 7fd379968700 1 --2- 172.21.15.162:0/3300380426 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd3740fcf70 0x7fd3740fd3e0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fd364004940 tx=0x7fd364009e30 comp rx=0 tx=0).stop 2024-04-27T04:06:40.936 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:40.935+0000 7fd379968700 1 -- 172.21.15.162:0/3300380426 shutdown_connections 2024-04-27T04:06:40.936 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:40.935+0000 7fd379968700 1 --2- 172.21.15.162:0/3300380426 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd3740fcf70 0x7fd3740fd3e0 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:06:40.936 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:40.935+0000 7fd379968700 1 --2- 172.21.15.162:0/3300380426 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd3740fbcc0 0x7fd3740fc0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:06:40.936 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:40.935+0000 7fd379968700 1 -- 172.21.15.162:0/3300380426 >> 172.21.15.162:0/3300380426 conn(0x7fd3740f7440 msgr2=0x7fd3740f9850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:06:40.936 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:40.935+0000 7fd379968700 1 -- 172.21.15.162:0/3300380426 shutdown_connections 2024-04-27T04:06:40.936 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:40.935+0000 7fd379968700 1 -- 172.21.15.162:0/3300380426 wait complete. 2024-04-27T04:06:40.936 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:40.936+0000 7fd379968700 1 Processor -- start 2024-04-27T04:06:40.937 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:40.936+0000 7fd379968700 1 -- start start 2024-04-27T04:06:40.937 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:40.936+0000 7fd379968700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd3740fbcc0 0x7fd374066d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:06:40.937 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:40.936+0000 7fd379968700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd3740fcf70 0x7fd374112740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:06:40.937 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:40.936+0000 7fd379968700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd374112c80 con 0x7fd3740fcf70 2024-04-27T04:06:40.937 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:40.936+0000 7fd379968700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd374112dc0 con 0x7fd3740fbcc0 2024-04-27T04:06:40.937 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:40.936+0000 7fd3727fc700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd3740fcf70 0x7fd374112740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:06:40.937 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:40.936+0000 7fd3727fc700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd3740fcf70 0x7fd374112740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:42036/0 (socket says 172.21.15.162:42036) 2024-04-27T04:06:40.937 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:40.936+0000 7fd3727fc700 1 -- 172.21.15.162:0/2520415487 learned_addr learned my addr 172.21.15.162:0/2520415487 (peer_addr_for_me v2:172.21.15.162:0/0) 2024-04-27T04:06:40.937 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:40.936+0000 7fd372ffd700 1 --2- 172.21.15.162:0/2520415487 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd3740fbcc0 0x7fd374066d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:06:40.937 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:40.936+0000 7fd3727fc700 1 -- 172.21.15.162:0/2520415487 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd3740fbcc0 msgr2=0x7fd374066d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:06:40.937 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:40.936+0000 7fd3727fc700 1 --2- 172.21.15.162:0/2520415487 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd3740fbcc0 0x7fd374066d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:06:40.937 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:40.936+0000 7fd3727fc700 1 -- 172.21.15.162:0/2520415487 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd364005040 con 0x7fd3740fcf70 2024-04-27T04:06:40.937 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:40.937+0000 7fd3727fc700 1 --2- 172.21.15.162:0/2520415487 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd3740fcf70 0x7fd374112740 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fd364009fd0 tx=0x7fd364009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:06:40.937 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:40.937+0000 7fd35b7fe700 1 -- 172.21.15.162:0/2520415487 <== mon.0 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd36400e040 con 0x7fd3740fcf70 2024-04-27T04:06:40.938 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:40.937+0000 7fd379968700 1 -- 172.21.15.162:0/2520415487 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd374113040 con 0x7fd3740fcf70 2024-04-27T04:06:40.938 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:40.937+0000 7fd35b7fe700 1 -- 172.21.15.162:0/2520415487 <== mon.0 v2:172.21.15.162:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fd36400bcc0 con 0x7fd3740fcf70 2024-04-27T04:06:40.938 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:40.937+0000 7fd35b7fe700 1 -- 172.21.15.162:0/2520415487 <== mon.0 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd36401c630 con 0x7fd3740fcf70 2024-04-27T04:06:40.938 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:40.937+0000 7fd379968700 1 -- 172.21.15.162:0/2520415487 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd374113600 con 0x7fd3740fcf70 2024-04-27T04:06:40.939 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:40.938+0000 7fd379968700 1 -- 172.21.15.162:0/2520415487 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd37404c710 con 0x7fd3740fcf70 2024-04-27T04:06:40.939 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:40.938+0000 7fd35b7fe700 1 -- 172.21.15.162:0/2520415487 <== mon.0 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 19) v1 ==== 86631+0+0 (secure 0 0 0) 0x7fd364013420 con 0x7fd3740fcf70 2024-04-27T04:06:40.939 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:40.939+0000 7fd35b7fe700 1 --2- 172.21.15.162:0/2520415487 >> [v2:172.21.15.162:6800/4079782599,v1:172.21.15.162:6801/4079782599] conn(0x7fd35c0692a0 0x7fd35c06b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:06:40.939 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:40.939+0000 7fd35b7fe700 1 -- 172.21.15.162:0/2520415487 <== mon.0 v2:172.21.15.162:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4634+0+0 (secure 0 0 0) 0x7fd364088100 con 0x7fd3740fcf70 2024-04-27T04:06:40.939 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:40.939+0000 7fd372ffd700 1 --2- 172.21.15.162:0/2520415487 >> [v2:172.21.15.162:6800/4079782599,v1:172.21.15.162:6801/4079782599] conn(0x7fd35c0692a0 0x7fd35c06b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:06:40.939 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:40.939+0000 7fd372ffd700 1 --2- 172.21.15.162:0/2520415487 >> [v2:172.21.15.162:6800/4079782599,v1:172.21.15.162:6801/4079782599] conn(0x7fd35c0692a0 0x7fd35c06b750 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fd368004f40 tx=0x7fd36800b040 comp rx=0 tx=0).ready entity=mgr.14210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:06:40.942 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:40.943+0000 7fd35b7fe700 1 -- 172.21.15.162:0/2520415487 <== mon.0 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7fd364058200 con 0x7fd3740fcf70 2024-04-27T04:06:41.141 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:41.141+0000 7fd379968700 1 -- 172.21.15.162:0/2520415487 --> [v2:172.21.15.162:6800/4079782599,v1:172.21.15.162:6801/4079782599] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7fd374101300 con 0x7fd35c0692a0 2024-04-27T04:06:41.149 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:41.150+0000 7fd35b7fe700 1 -- 172.21.15.162:0/2520415487 <== mgr.14210 v2:172.21.15.162:6800/4079782599 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+2682 (secure 0 0 0) 0x7fd374101300 con 0x7fd35c0692a0 2024-04-27T04:06:41.149 INFO:teuthology.orchestra.run.smithi162.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-27T04:06:41.149 INFO:teuthology.orchestra.run.smithi162.stdout:alertmanager.smithi162 smithi162 *:9093,9094 running (3m) 2m ago 4m 25.9M - 0.25.0 c8568f914cd2 61972851fcce 2024-04-27T04:06:41.149 INFO:teuthology.orchestra.run.smithi162.stdout:crash.smithi162 smithi162 running (5m) 2m ago 5m 6660k - 17.2.7-907-g2be3efa9 1c5834633715 228c98df4289 2024-04-27T04:06:41.149 INFO:teuthology.orchestra.run.smithi162.stdout:crash.smithi181 smithi181 running (4m) 66s ago 4m 6665k - 17.2.7-907-g2be3efa9 1c5834633715 c8e553e3df2b 2024-04-27T04:06:41.149 INFO:teuthology.orchestra.run.smithi162.stdout:grafana.smithi162 smithi162 *:3000 running (3m) 2m ago 4m 78.4M - 9.4.7 954c08fa6188 7d3364d5cddc 2024-04-27T04:06:41.149 INFO:teuthology.orchestra.run.smithi162.stdout:mgr.smithi162.oennfn smithi162 *:9283 running (6m) 2m ago 6m 472M - 17.2.7-907-g2be3efa9 1c5834633715 e768e27ad56e 2024-04-27T04:06:41.149 INFO:teuthology.orchestra.run.smithi162.stdout:mgr.smithi181.vtwuhz smithi181 *:8443,9283 running (4m) 66s ago 4m 432M - 17.2.7-907-g2be3efa9 1c5834633715 618e7918caea 2024-04-27T04:06:41.150 INFO:teuthology.orchestra.run.smithi162.stdout:mon.smithi162 smithi162 running (6m) 2m ago 6m 45.8M 2048M 17.2.7-907-g2be3efa9 1c5834633715 0f1e3682b9ab 2024-04-27T04:06:41.150 INFO:teuthology.orchestra.run.smithi162.stdout:mon.smithi181 smithi181 running (4m) 66s ago 4m 38.7M 2048M 17.2.7-907-g2be3efa9 1c5834633715 96e48aa3912f 2024-04-27T04:06:41.150 INFO:teuthology.orchestra.run.smithi162.stdout:node-exporter.smithi162 smithi162 *:9100 running (5m) 2m ago 5m 15.3M - 1.5.0 0da6a335fe13 e4535d340cea 2024-04-27T04:06:41.150 INFO:teuthology.orchestra.run.smithi162.stdout:node-exporter.smithi181 smithi181 *:9100 running (4m) 66s ago 4m 15.1M - 1.5.0 0da6a335fe13 3cc278142657 2024-04-27T04:06:41.150 INFO:teuthology.orchestra.run.smithi162.stdout:osd.0 smithi162 running (3m) 2m ago 3m 41.1M 4288M 17.2.7-907-g2be3efa9 1c5834633715 4a9df3aa938d 2024-04-27T04:06:41.150 INFO:teuthology.orchestra.run.smithi162.stdout:osd.1 smithi162 running (2m) 2m ago 2m 43.0M 4288M 17.2.7-907-g2be3efa9 1c5834633715 aefb464c46e8 2024-04-27T04:06:41.150 INFO:teuthology.orchestra.run.smithi162.stdout:osd.2 smithi162 running (2m) 2m ago 2m 39.3M 4288M 17.2.7-907-g2be3efa9 1c5834633715 70f16d86d38e 2024-04-27T04:06:41.150 INFO:teuthology.orchestra.run.smithi162.stdout:osd.3 smithi181 running (2m) 66s ago 2m 39.3M 5312M 17.2.7-907-g2be3efa9 1c5834633715 030d829be54e 2024-04-27T04:06:41.150 INFO:teuthology.orchestra.run.smithi162.stdout:osd.4 smithi181 running (95s) 66s ago 95s 36.0M 5312M 17.2.7-907-g2be3efa9 1c5834633715 512afbee7cf7 2024-04-27T04:06:41.150 INFO:teuthology.orchestra.run.smithi162.stdout:osd.5 smithi181 running (70s) 66s ago 70s 10.0M 5312M 17.2.7-907-g2be3efa9 1c5834633715 f47a1daac1ea 2024-04-27T04:06:41.150 INFO:teuthology.orchestra.run.smithi162.stdout:prometheus.smithi162 smithi162 *:9095 running (3m) 2m ago 4m 38.0M - 2.43.0 a07b618ecd1d 63755cc04fec 2024-04-27T04:06:41.151 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:41.152+0000 7fd379968700 1 -- 172.21.15.162:0/2520415487 >> [v2:172.21.15.162:6800/4079782599,v1:172.21.15.162:6801/4079782599] conn(0x7fd35c0692a0 msgr2=0x7fd35c06b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:06:41.151 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:41.152+0000 7fd379968700 1 --2- 172.21.15.162:0/2520415487 >> [v2:172.21.15.162:6800/4079782599,v1:172.21.15.162:6801/4079782599] conn(0x7fd35c0692a0 0x7fd35c06b750 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fd368004f40 tx=0x7fd36800b040 comp rx=0 tx=0).stop 2024-04-27T04:06:41.151 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:41.152+0000 7fd379968700 1 -- 172.21.15.162:0/2520415487 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd3740fcf70 msgr2=0x7fd374112740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:06:41.151 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:41.152+0000 7fd379968700 1 --2- 172.21.15.162:0/2520415487 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd3740fcf70 0x7fd374112740 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fd364009fd0 tx=0x7fd364009e30 comp rx=0 tx=0).stop 2024-04-27T04:06:41.152 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:41.152+0000 7fd379968700 1 -- 172.21.15.162:0/2520415487 shutdown_connections 2024-04-27T04:06:41.152 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:41.152+0000 7fd379968700 1 --2- 172.21.15.162:0/2520415487 >> [v2:172.21.15.162:6800/4079782599,v1:172.21.15.162:6801/4079782599] conn(0x7fd35c0692a0 0x7fd35c06b750 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:06:41.152 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:41.152+0000 7fd379968700 1 --2- 172.21.15.162:0/2520415487 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd3740fbcc0 0x7fd374066d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:06:41.152 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:41.152+0000 7fd379968700 1 --2- 172.21.15.162:0/2520415487 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd3740fcf70 0x7fd374112740 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:06:41.152 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:41.152+0000 7fd379968700 1 -- 172.21.15.162:0/2520415487 >> 172.21.15.162:0/2520415487 conn(0x7fd3740f7440 msgr2=0x7fd3740f8ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:06:41.152 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:41.152+0000 7fd379968700 1 -- 172.21.15.162:0/2520415487 shutdown_connections 2024-04-27T04:06:41.152 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:41.152+0000 7fd379968700 1 -- 172.21.15.162:0/2520415487 wait complete. 2024-04-27T04:06:41.748 DEBUG:teuthology.orchestra.run.smithi162:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 7f9750d6-044a-11ef-bc93-c7b262605968 -- bash -c 'ceph orch ls' 2024-04-27T04:06:42.001 INFO:teuthology.orchestra.run.smithi162.stderr:Inferring config /var/lib/ceph/7f9750d6-044a-11ef-bc93-c7b262605968/mon.smithi162/config 2024-04-27T04:06:42.368 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:06:42 smithi162 ceph-mon[26841]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-27T04:06:42.368 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:06:42 smithi162 ceph-mon[26841]: from='client.14436 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-27T04:06:42.423 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:06:42 smithi181 ceph-mon[33774]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-27T04:06:42.423 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:06:42 smithi181 ceph-mon[33774]: from='client.14436 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-27T04:06:43.599 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:43.599+0000 7f918504b700 1 -- 172.21.15.162:0/4282241675 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f91800fe420 msgr2=0x7f91800fe830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:06:43.600 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:43.599+0000 7f918504b700 1 --2- 172.21.15.162:0/4282241675 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f91800fe420 0x7f91800fe830 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f9174004940 tx=0x7f9174009e30 comp rx=0 tx=0).stop 2024-04-27T04:06:43.600 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:43.599+0000 7f918504b700 1 -- 172.21.15.162:0/4282241675 shutdown_connections 2024-04-27T04:06:43.600 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:43.599+0000 7f918504b700 1 --2- 172.21.15.162:0/4282241675 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f91800fed70 0x7f91800faeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:06:43.600 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:43.599+0000 7f918504b700 1 --2- 172.21.15.162:0/4282241675 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f91800fe420 0x7f91800fe830 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:06:43.600 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:43.599+0000 7f918504b700 1 -- 172.21.15.162:0/4282241675 >> 172.21.15.162:0/4282241675 conn(0x7f91800f6be0 msgr2=0x7f91800f8ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:06:43.600 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:43.599+0000 7f918504b700 1 -- 172.21.15.162:0/4282241675 shutdown_connections 2024-04-27T04:06:43.600 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:43.599+0000 7f918504b700 1 -- 172.21.15.162:0/4282241675 wait complete. 2024-04-27T04:06:43.600 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:43.600+0000 7f918504b700 1 Processor -- start 2024-04-27T04:06:43.600 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:43.600+0000 7f918504b700 1 -- start start 2024-04-27T04:06:43.600 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:43.600+0000 7f918504b700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f91800fe420 0x7f918010d8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:06:43.600 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:43.600+0000 7f918504b700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f91800fed70 0x7f918010de30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:06:43.600 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:43.600+0000 7f918504b700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f918010e430 con 0x7f91800fed70 2024-04-27T04:06:43.600 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:43.600+0000 7f918504b700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f918010e5a0 con 0x7f91800fe420 2024-04-27T04:06:43.601 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:43.600+0000 7f917e59c700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f91800fed70 0x7f918010de30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:06:43.601 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:43.600+0000 7f917e59c700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f91800fed70 0x7f918010de30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:39894/0 (socket says 172.21.15.162:39894) 2024-04-27T04:06:43.601 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:43.600+0000 7f917e59c700 1 -- 172.21.15.162:0/475225103 learned_addr learned my addr 172.21.15.162:0/475225103 (peer_addr_for_me v2:172.21.15.162:0/0) 2024-04-27T04:06:43.601 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:43.600+0000 7f917ed9d700 1 --2- 172.21.15.162:0/475225103 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f91800fe420 0x7f918010d8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:06:43.601 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:43.600+0000 7f917e59c700 1 -- 172.21.15.162:0/475225103 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f91800fe420 msgr2=0x7f918010d8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:06:43.601 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:43.600+0000 7f917e59c700 1 --2- 172.21.15.162:0/475225103 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f91800fe420 0x7f918010d8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:06:43.601 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:43.600+0000 7f917e59c700 1 -- 172.21.15.162:0/475225103 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9174005040 con 0x7f91800fed70 2024-04-27T04:06:43.601 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:43.600+0000 7f917e59c700 1 --2- 172.21.15.162:0/475225103 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f91800fed70 0x7f918010de30 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f9170002a50 tx=0x7f91700046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:06:43.601 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:43.600+0000 7f91677fe700 1 -- 172.21.15.162:0/475225103 <== mon.0 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9170007cc0 con 0x7f91800fed70 2024-04-27T04:06:43.601 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:43.600+0000 7f91677fe700 1 -- 172.21.15.162:0/475225103 <== mon.0 v2:172.21.15.162:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f917000f040 con 0x7f91800fed70 2024-04-27T04:06:43.601 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:43.600+0000 7f918504b700 1 -- 172.21.15.162:0/475225103 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9180113000 con 0x7f91800fed70 2024-04-27T04:06:43.601 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:43.600+0000 7f91677fe700 1 -- 172.21.15.162:0/475225103 <== mon.0 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f917000da30 con 0x7f91800fed70 2024-04-27T04:06:43.601 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:43.601+0000 7f918504b700 1 -- 172.21.15.162:0/475225103 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9180113590 con 0x7f91800fed70 2024-04-27T04:06:43.602 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:43.601+0000 7f918504b700 1 -- 172.21.15.162:0/475225103 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f918004c710 con 0x7f91800fed70 2024-04-27T04:06:43.602 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:43.602+0000 7f91677fe700 1 -- 172.21.15.162:0/475225103 <== mon.0 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 19) v1 ==== 86631+0+0 (secure 0 0 0) 0x7f917000db90 con 0x7f91800fed70 2024-04-27T04:06:43.602 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:43.602+0000 7f91677fe700 1 --2- 172.21.15.162:0/475225103 >> [v2:172.21.15.162:6800/4079782599,v1:172.21.15.162:6801/4079782599] conn(0x7f91680692a0 0x7f916806b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:06:43.602 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:43.602+0000 7f91677fe700 1 -- 172.21.15.162:0/475225103 <== mon.0 v2:172.21.15.162:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4634+0+0 (secure 0 0 0) 0x7f9170086c10 con 0x7f91800fed70 2024-04-27T04:06:43.603 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:43.602+0000 7f917ed9d700 1 --2- 172.21.15.162:0/475225103 >> [v2:172.21.15.162:6800/4079782599,v1:172.21.15.162:6801/4079782599] conn(0x7f91680692a0 0x7f916806b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:06:43.603 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:43.603+0000 7f917ed9d700 1 --2- 172.21.15.162:0/475225103 >> [v2:172.21.15.162:6800/4079782599,v1:172.21.15.162:6801/4079782599] conn(0x7f91680692a0 0x7f916806b750 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f917400b970 tx=0x7f9174009e30 comp rx=0 tx=0).ready entity=mgr.14210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:06:43.605 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:43.607+0000 7f91677fe700 1 -- 172.21.15.162:0/475225103 <== mon.0 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f9170056c90 con 0x7f91800fed70 2024-04-27T04:06:43.800 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:43.801+0000 7f918504b700 1 -- 172.21.15.162:0/475225103 --> [v2:172.21.15.162:6800/4079782599,v1:172.21.15.162:6801/4079782599] -- mgr_command(tid 0: {"prefix": "orch ls", "target": ["mon-mgr", ""]}) v1 -- 0x7f91801024f0 con 0x7f91680692a0 2024-04-27T04:06:43.805 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:43.806+0000 7f91677fe700 1 -- 172.21.15.162:0/475225103 <== mgr.14210 v2:172.21.15.162:6800/4079782599 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+1179 (secure 0 0 0) 0x7f91801024f0 con 0x7f91680692a0 2024-04-27T04:06:43.805 INFO:teuthology.orchestra.run.smithi162.stdout:NAME PORTS RUNNING REFRESHED AGE PLACEMENT 2024-04-27T04:06:43.805 INFO:teuthology.orchestra.run.smithi162.stdout:alertmanager ?:9093,9094 1/1 2m ago 5m count:1 2024-04-27T04:06:43.805 INFO:teuthology.orchestra.run.smithi162.stdout:crash 2/2 2m ago 5m * 2024-04-27T04:06:43.806 INFO:teuthology.orchestra.run.smithi162.stdout:grafana ?:3000 1/1 2m ago 5m count:1 2024-04-27T04:06:43.806 INFO:teuthology.orchestra.run.smithi162.stdout:mgr 2/2 2m ago 5m count:2 2024-04-27T04:06:43.806 INFO:teuthology.orchestra.run.smithi162.stdout:mon 2/2 2m ago 4m smithi162:172.21.15.162=smithi162;smithi181:172.21.15.181=smithi181;count:2 2024-04-27T04:06:43.806 INFO:teuthology.orchestra.run.smithi162.stdout:node-exporter ?:9100 2/2 2m ago 5m * 2024-04-27T04:06:43.806 INFO:teuthology.orchestra.run.smithi162.stdout:osd 6 2m ago - 2024-04-27T04:06:43.806 INFO:teuthology.orchestra.run.smithi162.stdout:prometheus ?:9095 1/1 2m ago 5m count:1 2024-04-27T04:06:43.807 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:43.808+0000 7f918504b700 1 -- 172.21.15.162:0/475225103 >> [v2:172.21.15.162:6800/4079782599,v1:172.21.15.162:6801/4079782599] conn(0x7f91680692a0 msgr2=0x7f916806b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:06:43.807 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:43.808+0000 7f918504b700 1 --2- 172.21.15.162:0/475225103 >> [v2:172.21.15.162:6800/4079782599,v1:172.21.15.162:6801/4079782599] conn(0x7f91680692a0 0x7f916806b750 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f917400b970 tx=0x7f9174009e30 comp rx=0 tx=0).stop 2024-04-27T04:06:43.808 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:43.808+0000 7f918504b700 1 -- 172.21.15.162:0/475225103 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f91800fed70 msgr2=0x7f918010de30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:06:43.808 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:43.808+0000 7f918504b700 1 --2- 172.21.15.162:0/475225103 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f91800fed70 0x7f918010de30 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f9170002a50 tx=0x7f91700046b0 comp rx=0 tx=0).stop 2024-04-27T04:06:43.808 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:43.808+0000 7f918504b700 1 -- 172.21.15.162:0/475225103 shutdown_connections 2024-04-27T04:06:43.808 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:43.808+0000 7f918504b700 1 --2- 172.21.15.162:0/475225103 >> [v2:172.21.15.162:6800/4079782599,v1:172.21.15.162:6801/4079782599] conn(0x7f91680692a0 0x7f916806b750 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:06:43.808 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:43.808+0000 7f918504b700 1 --2- 172.21.15.162:0/475225103 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f91800fe420 0x7f918010d8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:06:43.808 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:43.808+0000 7f918504b700 1 --2- 172.21.15.162:0/475225103 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f91800fed70 0x7f918010de30 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:06:43.808 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:43.808+0000 7f918504b700 1 -- 172.21.15.162:0/475225103 >> 172.21.15.162:0/475225103 conn(0x7f91800f6be0 msgr2=0x7f91800f8290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:06:43.808 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:43.808+0000 7f918504b700 1 -- 172.21.15.162:0/475225103 shutdown_connections 2024-04-27T04:06:43.808 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:43.808+0000 7f918504b700 1 -- 172.21.15.162:0/475225103 wait complete. 2024-04-27T04:06:44.387 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:06:44 smithi162 ceph-mon[26841]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-27T04:06:44.423 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:06:44 smithi181 ceph-mon[33774]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-27T04:06:44.470 DEBUG:teuthology.orchestra.run.smithi162:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 7f9750d6-044a-11ef-bc93-c7b262605968 -- bash -c 'ceph orch host ls' 2024-04-27T04:06:44.720 INFO:teuthology.orchestra.run.smithi162.stderr:Inferring config /var/lib/ceph/7f9750d6-044a-11ef-bc93-c7b262605968/mon.smithi162/config 2024-04-27T04:06:45.138 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:06:45 smithi162 ceph-mon[26841]: from='client.14440 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-27T04:06:45.423 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:06:45 smithi181 ceph-mon[33774]: from='client.14440 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-27T04:06:46.188 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:46.186+0000 7f25602d3700 1 -- 172.21.15.162:0/1003246364 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f25580fcc50 msgr2=0x7f25580fd0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:06:46.188 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:46.186+0000 7f25602d3700 1 --2- 172.21.15.162:0/1003246364 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f25580fcc50 0x7f25580fd0c0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f2548004940 tx=0x7f2548009e30 comp rx=0 tx=0).stop 2024-04-27T04:06:46.188 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:46.186+0000 7f25602d3700 1 -- 172.21.15.162:0/1003246364 shutdown_connections 2024-04-27T04:06:46.188 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:46.186+0000 7f25602d3700 1 --2- 172.21.15.162:0/1003246364 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f25580fcc50 0x7f25580fd0c0 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:06:46.188 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:46.186+0000 7f25602d3700 1 --2- 172.21.15.162:0/1003246364 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f25580fc300 0x7f25580fc710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:06:46.188 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:46.186+0000 7f25602d3700 1 -- 172.21.15.162:0/1003246364 >> 172.21.15.162:0/1003246364 conn(0x7f25580f6bd0 msgr2=0x7f25580f9000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:06:46.188 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:46.186+0000 7f25602d3700 1 -- 172.21.15.162:0/1003246364 shutdown_connections 2024-04-27T04:06:46.189 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:46.186+0000 7f25602d3700 1 -- 172.21.15.162:0/1003246364 wait complete. 2024-04-27T04:06:46.189 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:46.187+0000 7f25602d3700 1 Processor -- start 2024-04-27T04:06:46.189 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:46.187+0000 7f25602d3700 1 -- start start 2024-04-27T04:06:46.189 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:46.187+0000 7f25602d3700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f25580fc300 0x7f255810d960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:06:46.189 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:46.187+0000 7f25602d3700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f25580fcc50 0x7f255810dea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:06:46.189 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:46.187+0000 7f25602d3700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f255810e530 con 0x7f25580fc300 2024-04-27T04:06:46.189 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:46.187+0000 7f25602d3700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2558112e80 con 0x7f25580fcc50 2024-04-27T04:06:46.189 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:46.187+0000 7f255e06f700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f25580fc300 0x7f255810d960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:06:46.189 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:46.187+0000 7f255e06f700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f25580fc300 0x7f255810d960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:39916/0 (socket says 172.21.15.162:39916) 2024-04-27T04:06:46.189 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:46.187+0000 7f255e06f700 1 -- 172.21.15.162:0/2814852170 learned_addr learned my addr 172.21.15.162:0/2814852170 (peer_addr_for_me v2:172.21.15.162:0/0) 2024-04-27T04:06:46.189 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:46.187+0000 7f255d86e700 1 --2- 172.21.15.162:0/2814852170 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f25580fcc50 0x7f255810dea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:06:46.189 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:46.187+0000 7f255e06f700 1 -- 172.21.15.162:0/2814852170 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f25580fcc50 msgr2=0x7f255810dea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:06:46.190 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:46.187+0000 7f255e06f700 1 --2- 172.21.15.162:0/2814852170 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f25580fcc50 0x7f255810dea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:06:46.190 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:46.187+0000 7f255e06f700 1 -- 172.21.15.162:0/2814852170 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2548005040 con 0x7f25580fc300 2024-04-27T04:06:46.190 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:46.187+0000 7f255e06f700 1 --2- 172.21.15.162:0/2814852170 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f25580fc300 0x7f255810d960 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f255000bbb0 tx=0x7f2550009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:06:46.190 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:46.187+0000 7f254effd700 1 -- 172.21.15.162:0/2814852170 <== mon.0 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f25500092e0 con 0x7f25580fc300 2024-04-27T04:06:46.190 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:46.187+0000 7f254effd700 1 -- 172.21.15.162:0/2814852170 <== mon.0 v2:172.21.15.162:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f255000cbd0 con 0x7f25580fc300 2024-04-27T04:06:46.190 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:46.187+0000 7f25602d3700 1 -- 172.21.15.162:0/2814852170 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f25581130e0 con 0x7f25580fc300 2024-04-27T04:06:46.190 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:46.188+0000 7f254effd700 1 -- 172.21.15.162:0/2814852170 <== mon.0 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2550007400 con 0x7f25580fc300 2024-04-27T04:06:46.190 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:46.188+0000 7f25602d3700 1 -- 172.21.15.162:0/2814852170 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2558113650 con 0x7f25580fc300 2024-04-27T04:06:46.191 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:46.189+0000 7f25602d3700 1 -- 172.21.15.162:0/2814852170 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2558101c50 con 0x7f25580fc300 2024-04-27T04:06:46.191 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:46.189+0000 7f254effd700 1 -- 172.21.15.162:0/2814852170 <== mon.0 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 19) v1 ==== 86631+0+0 (secure 0 0 0) 0x7f2550012070 con 0x7f25580fc300 2024-04-27T04:06:46.191 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:46.189+0000 7f254effd700 1 --2- 172.21.15.162:0/2814852170 >> [v2:172.21.15.162:6800/4079782599,v1:172.21.15.162:6801/4079782599] conn(0x7f25440692a0 0x7f254406b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:06:46.191 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:46.190+0000 7f255d86e700 1 --2- 172.21.15.162:0/2814852170 >> [v2:172.21.15.162:6800/4079782599,v1:172.21.15.162:6801/4079782599] conn(0x7f25440692a0 0x7f254406b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:06:46.191 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:46.190+0000 7f254effd700 1 -- 172.21.15.162:0/2814852170 <== mon.0 v2:172.21.15.162:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4634+0+0 (secure 0 0 0) 0x7f2550086a10 con 0x7f25580fc300 2024-04-27T04:06:46.191 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:46.190+0000 7f255d86e700 1 --2- 172.21.15.162:0/2814852170 >> [v2:172.21.15.162:6800/4079782599,v1:172.21.15.162:6801/4079782599] conn(0x7f25440692a0 0x7f254406b750 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f2548000c00 tx=0x7f2548009e30 comp rx=0 tx=0).ready entity=mgr.14210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:06:46.194 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:46.194+0000 7f254effd700 1 -- 172.21.15.162:0/2814852170 <== mon.0 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f2550056b10 con 0x7f25580fc300 2024-04-27T04:06:46.387 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:06:46 smithi162 ceph-mon[26841]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-27T04:06:46.388 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:46.387+0000 7f25602d3700 1 -- 172.21.15.162:0/2814852170 --> [v2:172.21.15.162:6800/4079782599,v1:172.21.15.162:6801/4079782599] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""]}) v1 -- 0x7f255802b340 con 0x7f25440692a0 2024-04-27T04:06:46.389 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:46.389+0000 7f254effd700 1 -- 172.21.15.162:0/2814852170 <== mgr.14210 v2:172.21.15.162:6800/4079782599 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+148 (secure 0 0 0) 0x7f255802b340 con 0x7f25440692a0 2024-04-27T04:06:46.389 INFO:teuthology.orchestra.run.smithi162.stdout:HOST ADDR LABELS STATUS 2024-04-27T04:06:46.389 INFO:teuthology.orchestra.run.smithi162.stdout:smithi162 172.21.15.162 2024-04-27T04:06:46.389 INFO:teuthology.orchestra.run.smithi162.stdout:smithi181 172.21.15.181 2024-04-27T04:06:46.389 INFO:teuthology.orchestra.run.smithi162.stdout:2 hosts in cluster 2024-04-27T04:06:46.391 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:46.391+0000 7f25602d3700 1 -- 172.21.15.162:0/2814852170 >> [v2:172.21.15.162:6800/4079782599,v1:172.21.15.162:6801/4079782599] conn(0x7f25440692a0 msgr2=0x7f254406b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:06:46.391 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:46.391+0000 7f25602d3700 1 --2- 172.21.15.162:0/2814852170 >> [v2:172.21.15.162:6800/4079782599,v1:172.21.15.162:6801/4079782599] conn(0x7f25440692a0 0x7f254406b750 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f2548000c00 tx=0x7f2548009e30 comp rx=0 tx=0).stop 2024-04-27T04:06:46.391 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:46.391+0000 7f25602d3700 1 -- 172.21.15.162:0/2814852170 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f25580fc300 msgr2=0x7f255810d960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:06:46.391 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:46.391+0000 7f25602d3700 1 --2- 172.21.15.162:0/2814852170 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f25580fc300 0x7f255810d960 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f255000bbb0 tx=0x7f2550009840 comp rx=0 tx=0).stop 2024-04-27T04:06:46.391 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:46.391+0000 7f25602d3700 1 -- 172.21.15.162:0/2814852170 shutdown_connections 2024-04-27T04:06:46.391 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:46.391+0000 7f25602d3700 1 --2- 172.21.15.162:0/2814852170 >> [v2:172.21.15.162:6800/4079782599,v1:172.21.15.162:6801/4079782599] conn(0x7f25440692a0 0x7f254406b750 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:06:46.391 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:46.391+0000 7f25602d3700 1 --2- 172.21.15.162:0/2814852170 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f25580fc300 0x7f255810d960 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:06:46.391 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:46.391+0000 7f25602d3700 1 --2- 172.21.15.162:0/2814852170 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f25580fcc50 0x7f255810dea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:06:46.391 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:46.391+0000 7f25602d3700 1 -- 172.21.15.162:0/2814852170 >> 172.21.15.162:0/2814852170 conn(0x7f25580f6bd0 msgr2=0x7f25580f8280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:06:46.392 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:46.391+0000 7f25602d3700 1 -- 172.21.15.162:0/2814852170 shutdown_connections 2024-04-27T04:06:46.392 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:46.391+0000 7f25602d3700 1 -- 172.21.15.162:0/2814852170 wait complete. 2024-04-27T04:06:46.422 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:06:46 smithi181 ceph-mon[33774]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-27T04:06:47.059 DEBUG:teuthology.orchestra.run.smithi162:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 7f9750d6-044a-11ef-bc93-c7b262605968 -- bash -c 'ceph orch device ls' 2024-04-27T04:06:47.137 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:06:47 smithi162 ceph-mon[26841]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-27T04:06:47.314 INFO:teuthology.orchestra.run.smithi162.stderr:Inferring config /var/lib/ceph/7f9750d6-044a-11ef-bc93-c7b262605968/mon.smithi162/config 2024-04-27T04:06:47.422 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:06:47 smithi181 ceph-mon[33774]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-27T04:06:48.388 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:06:48 smithi162 ceph-mon[26841]: from='client.14444 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-27T04:06:48.388 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:06:48 smithi162 ceph-mon[26841]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-27T04:06:48.423 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:06:48 smithi181 ceph-mon[33774]: from='client.14444 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-27T04:06:48.423 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:06:48 smithi181 ceph-mon[33774]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-27T04:06:48.894 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:48.891+0000 7fa77bb04700 1 -- 172.21.15.162:0/3931564032 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa7740fce80 msgr2=0x7fa7740fd290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:06:48.894 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:48.891+0000 7fa77bb04700 1 --2- 172.21.15.162:0/3931564032 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa7740fce80 0x7fa7740fd290 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fa770004940 tx=0x7fa770009e30 comp rx=0 tx=0).stop 2024-04-27T04:06:48.894 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:48.891+0000 7fa77bb04700 1 -- 172.21.15.162:0/3931564032 shutdown_connections 2024-04-27T04:06:48.894 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:48.891+0000 7fa77bb04700 1 --2- 172.21.15.162:0/3931564032 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa7740fe080 0x7fa7740fe4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:06:48.894 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:48.891+0000 7fa77bb04700 1 --2- 172.21.15.162:0/3931564032 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa7740fce80 0x7fa7740fd290 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:06:48.894 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:48.891+0000 7fa77bb04700 1 -- 172.21.15.162:0/3931564032 >> 172.21.15.162:0/3931564032 conn(0x7fa7740f8590 msgr2=0x7fa7740fa9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:06:48.895 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:48.891+0000 7fa77bb04700 1 -- 172.21.15.162:0/3931564032 shutdown_connections 2024-04-27T04:06:48.895 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:48.891+0000 7fa77bb04700 1 -- 172.21.15.162:0/3931564032 wait complete. 2024-04-27T04:06:48.895 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:48.892+0000 7fa77bb04700 1 Processor -- start 2024-04-27T04:06:48.895 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:48.892+0000 7fa77bb04700 1 -- start start 2024-04-27T04:06:48.895 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:48.892+0000 7fa77bb04700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa7740fce80 0x7fa77410da40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:06:48.895 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:48.892+0000 7fa77bb04700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa7740fe080 0x7fa77410df80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:06:48.895 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:48.892+0000 7fa77bb04700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa77410e580 con 0x7fa7740fe080 2024-04-27T04:06:48.895 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:48.892+0000 7fa77bb04700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa77410edd0 con 0x7fa7740fce80 2024-04-27T04:06:48.895 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:48.892+0000 7fa77909f700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa7740fe080 0x7fa77410df80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:06:48.895 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:48.892+0000 7fa77909f700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa7740fe080 0x7fa77410df80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:39940/0 (socket says 172.21.15.162:39940) 2024-04-27T04:06:48.895 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:48.892+0000 7fa77909f700 1 -- 172.21.15.162:0/4000650292 learned_addr learned my addr 172.21.15.162:0/4000650292 (peer_addr_for_me v2:172.21.15.162:0/0) 2024-04-27T04:06:48.895 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:48.892+0000 7fa7798a0700 1 --2- 172.21.15.162:0/4000650292 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa7740fce80 0x7fa77410da40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:06:48.896 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:48.892+0000 7fa77909f700 1 -- 172.21.15.162:0/4000650292 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa7740fce80 msgr2=0x7fa77410da40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:06:48.896 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:48.892+0000 7fa77909f700 1 --2- 172.21.15.162:0/4000650292 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa7740fce80 0x7fa77410da40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:06:48.896 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:48.892+0000 7fa77909f700 1 -- 172.21.15.162:0/4000650292 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa770005040 con 0x7fa7740fe080 2024-04-27T04:06:48.896 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:48.892+0000 7fa7798a0700 1 --2- 172.21.15.162:0/4000650292 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa7740fce80 0x7fa77410da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-27T04:06:48.896 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:48.893+0000 7fa77909f700 1 --2- 172.21.15.162:0/4000650292 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa7740fe080 0x7fa77410df80 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fa76400c8c0 tx=0x7fa7640044c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:06:48.896 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:48.893+0000 7fa76a7fc700 1 -- 172.21.15.162:0/4000650292 <== mon.0 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa76400a980 con 0x7fa7740fe080 2024-04-27T04:06:48.896 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:48.893+0000 7fa76a7fc700 1 -- 172.21.15.162:0/4000650292 <== mon.0 v2:172.21.15.162:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fa76400ce60 con 0x7fa7740fe080 2024-04-27T04:06:48.896 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:48.893+0000 7fa77bb04700 1 -- 172.21.15.162:0/4000650292 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa77410f0b0 con 0x7fa7740fe080 2024-04-27T04:06:48.896 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:48.893+0000 7fa76a7fc700 1 -- 172.21.15.162:0/4000650292 <== mon.0 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa7640115e0 con 0x7fa7740fe080 2024-04-27T04:06:48.896 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:48.893+0000 7fa77bb04700 1 -- 172.21.15.162:0/4000650292 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa77410f5f0 con 0x7fa7740fe080 2024-04-27T04:06:48.896 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:48.894+0000 7fa77bb04700 1 -- 172.21.15.162:0/4000650292 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa77405f5c0 con 0x7fa7740fe080 2024-04-27T04:06:48.897 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:48.894+0000 7fa76a7fc700 1 -- 172.21.15.162:0/4000650292 <== mon.0 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 19) v1 ==== 86631+0+0 (secure 0 0 0) 0x7fa764012400 con 0x7fa7740fe080 2024-04-27T04:06:48.897 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:48.894+0000 7fa76a7fc700 1 --2- 172.21.15.162:0/4000650292 >> [v2:172.21.15.162:6800/4079782599,v1:172.21.15.162:6801/4079782599] conn(0x7fa7600692a0 0x7fa76006b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:06:48.897 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:48.895+0000 7fa76a7fc700 1 -- 172.21.15.162:0/4000650292 <== mon.0 v2:172.21.15.162:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4634+0+0 (secure 0 0 0) 0x7fa764086e60 con 0x7fa7740fe080 2024-04-27T04:06:48.898 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:48.895+0000 7fa7798a0700 1 --2- 172.21.15.162:0/4000650292 >> [v2:172.21.15.162:6800/4079782599,v1:172.21.15.162:6801/4079782599] conn(0x7fa7600692a0 0x7fa76006b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:06:48.898 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:48.895+0000 7fa7798a0700 1 --2- 172.21.15.162:0/4000650292 >> [v2:172.21.15.162:6800/4079782599,v1:172.21.15.162:6801/4079782599] conn(0x7fa7600692a0 0x7fa76006b750 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fa770004890 tx=0x7fa770009e30 comp rx=0 tx=0).ready entity=mgr.14210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:06:48.899 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:48.899+0000 7fa76a7fc700 1 -- 172.21.15.162:0/4000650292 <== mon.0 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7fa764057000 con 0x7fa7740fe080 2024-04-27T04:06:49.097 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:49.097+0000 7fa77bb04700 1 -- 172.21.15.162:0/4000650292 --> [v2:172.21.15.162:6800/4079782599,v1:172.21.15.162:6801/4079782599] -- mgr_command(tid 0: {"prefix": "orch device ls", "target": ["mon-mgr", ""]}) v1 -- 0x7fa77410f960 con 0x7fa7600692a0 2024-04-27T04:06:49.098 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:49.098+0000 7fa76a7fc700 1 -- 172.21.15.162:0/4000650292 <== mgr.14210 v2:172.21.15.162:6800/4079782599 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+399 (secure 0 0 0) 0x7fa77410f960 con 0x7fa7600692a0 2024-04-27T04:06:49.098 INFO:teuthology.orchestra.run.smithi162.stdout:HOST PATH TYPE DEVICE ID SIZE AVAILABLE REFRESHED REJECT REASONS 2024-04-27T04:06:49.099 INFO:teuthology.orchestra.run.smithi162.stdout:smithi162 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_CVFT623300JU400BGN 372G No 2m ago Has a FileSystem, LVM detected 2024-04-27T04:06:49.099 INFO:teuthology.orchestra.run.smithi162.stdout:smithi181 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_CVFT6246000E400BGN 372G No 70s ago Has a FileSystem, LVM detected 2024-04-27T04:06:49.101 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:49.100+0000 7fa77bb04700 1 -- 172.21.15.162:0/4000650292 >> [v2:172.21.15.162:6800/4079782599,v1:172.21.15.162:6801/4079782599] conn(0x7fa7600692a0 msgr2=0x7fa76006b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:06:49.101 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:49.100+0000 7fa77bb04700 1 --2- 172.21.15.162:0/4000650292 >> [v2:172.21.15.162:6800/4079782599,v1:172.21.15.162:6801/4079782599] conn(0x7fa7600692a0 0x7fa76006b750 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fa770004890 tx=0x7fa770009e30 comp rx=0 tx=0).stop 2024-04-27T04:06:49.101 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:49.100+0000 7fa77bb04700 1 -- 172.21.15.162:0/4000650292 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa7740fe080 msgr2=0x7fa77410df80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:06:49.101 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:49.100+0000 7fa77bb04700 1 --2- 172.21.15.162:0/4000650292 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa7740fe080 0x7fa77410df80 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fa76400c8c0 tx=0x7fa7640044c0 comp rx=0 tx=0).stop 2024-04-27T04:06:49.101 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:49.101+0000 7fa77bb04700 1 -- 172.21.15.162:0/4000650292 shutdown_connections 2024-04-27T04:06:49.102 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:49.101+0000 7fa77bb04700 1 --2- 172.21.15.162:0/4000650292 >> [v2:172.21.15.162:6800/4079782599,v1:172.21.15.162:6801/4079782599] conn(0x7fa7600692a0 0x7fa76006b750 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:06:49.102 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:49.101+0000 7fa77bb04700 1 --2- 172.21.15.162:0/4000650292 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa7740fce80 0x7fa77410da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:06:49.102 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:49.101+0000 7fa77bb04700 1 --2- 172.21.15.162:0/4000650292 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa7740fe080 0x7fa77410df80 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:06:49.102 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:49.101+0000 7fa77bb04700 1 -- 172.21.15.162:0/4000650292 >> 172.21.15.162:0/4000650292 conn(0x7fa7740f8590 msgr2=0x7fa7740f9d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:06:49.102 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:49.101+0000 7fa77bb04700 1 -- 172.21.15.162:0/4000650292 shutdown_connections 2024-04-27T04:06:49.102 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:49.101+0000 7fa77bb04700 1 -- 172.21.15.162:0/4000650292 wait complete. 2024-04-27T04:06:49.697 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-04-27T04:06:49.707 INFO:tasks.cephadm:Running commands on role host.a host ubuntu@smithi162.front.sepia.ceph.com 2024-04-27T04:06:49.708 DEBUG:teuthology.orchestra.run.smithi162:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 7f9750d6-044a-11ef-bc93-c7b262605968 -- bash -c 'ceph fs volume create cephfs --placement=4' 2024-04-27T04:06:49.960 INFO:teuthology.orchestra.run.smithi162.stderr:Inferring config /var/lib/ceph/7f9750d6-044a-11ef-bc93-c7b262605968/mon.smithi162/config 2024-04-27T04:06:50.333 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:06:50 smithi162 ceph-mon[26841]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-27T04:06:50.333 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:06:50 smithi162 ceph-mon[26841]: from='client.14448 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-27T04:06:50.423 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:06:50 smithi181 ceph-mon[33774]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-27T04:06:50.423 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:06:50 smithi181 ceph-mon[33774]: from='client.14448 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-27T04:06:51.551 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:51.550+0000 7f040542b700 1 -- 172.21.15.162:0/1582809987 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f04000fbcc0 msgr2=0x7f04000fc0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:06:51.551 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:51.550+0000 7f040542b700 1 --2- 172.21.15.162:0/1582809987 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f04000fbcc0 0x7f04000fc0d0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f03f4004940 tx=0x7f03f4009e30 comp rx=0 tx=0).stop 2024-04-27T04:06:51.551 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:51.550+0000 7f040542b700 1 -- 172.21.15.162:0/1582809987 shutdown_connections 2024-04-27T04:06:51.551 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:51.550+0000 7f040542b700 1 --2- 172.21.15.162:0/1582809987 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f04000fcf70 0x7f04000fd3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:06:51.551 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:51.550+0000 7f040542b700 1 --2- 172.21.15.162:0/1582809987 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f04000fbcc0 0x7f04000fc0d0 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:06:51.551 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:51.550+0000 7f040542b700 1 -- 172.21.15.162:0/1582809987 >> 172.21.15.162:0/1582809987 conn(0x7f04000f7420 msgr2=0x7f04000f9850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:06:51.551 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:51.550+0000 7f040542b700 1 -- 172.21.15.162:0/1582809987 shutdown_connections 2024-04-27T04:06:51.551 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:51.550+0000 7f040542b700 1 -- 172.21.15.162:0/1582809987 wait complete. 2024-04-27T04:06:51.551 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:51.551+0000 7f040542b700 1 Processor -- start 2024-04-27T04:06:51.551 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:51.551+0000 7f040542b700 1 -- start start 2024-04-27T04:06:51.551 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:51.551+0000 7f040542b700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f04000fbcc0 0x7f0400066d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:06:51.552 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:51.551+0000 7f040542b700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f04000fcf70 0x7f0400112760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:06:51.552 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:51.551+0000 7f040542b700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0400112ca0 con 0x7f04000fbcc0 2024-04-27T04:06:51.552 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:51.551+0000 7f040542b700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0400112e10 con 0x7f04000fcf70 2024-04-27T04:06:51.553 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:51.551+0000 7f03feffd700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f04000fbcc0 0x7f0400066d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:06:51.553 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:51.551+0000 7f03feffd700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f04000fbcc0 0x7f0400066d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:39962/0 (socket says 172.21.15.162:39962) 2024-04-27T04:06:51.553 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:51.551+0000 7f03feffd700 1 -- 172.21.15.162:0/3579896952 learned_addr learned my addr 172.21.15.162:0/3579896952 (peer_addr_for_me v2:172.21.15.162:0/0) 2024-04-27T04:06:51.553 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:51.551+0000 7f03fe7fc700 1 --2- 172.21.15.162:0/3579896952 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f04000fcf70 0x7f0400112760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:06:51.553 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:51.551+0000 7f03feffd700 1 -- 172.21.15.162:0/3579896952 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f04000fcf70 msgr2=0x7f0400112760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:06:51.553 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:51.551+0000 7f03feffd700 1 --2- 172.21.15.162:0/3579896952 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f04000fcf70 0x7f0400112760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:06:51.553 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:51.551+0000 7f03feffd700 1 -- 172.21.15.162:0/3579896952 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f03f4005040 con 0x7f04000fbcc0 2024-04-27T04:06:51.554 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:51.551+0000 7f03fe7fc700 1 --2- 172.21.15.162:0/3579896952 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f04000fcf70 0x7f0400112760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-27T04:06:51.554 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:51.551+0000 7f03feffd700 1 --2- 172.21.15.162:0/3579896952 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f04000fbcc0 0x7f0400066d30 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f03f4009150 tx=0x7f03f4009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:06:51.554 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:51.552+0000 7f03e77fe700 1 -- 172.21.15.162:0/3579896952 <== mon.0 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f03f400e040 con 0x7f04000fbcc0 2024-04-27T04:06:51.554 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:51.552+0000 7f03e77fe700 1 -- 172.21.15.162:0/3579896952 <== mon.0 v2:172.21.15.162:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f03f400bcc0 con 0x7f04000fbcc0 2024-04-27T04:06:51.554 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:51.552+0000 7f040542b700 1 -- 172.21.15.162:0/3579896952 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0400113090 con 0x7f04000fbcc0 2024-04-27T04:06:51.554 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:51.552+0000 7f03e77fe700 1 -- 172.21.15.162:0/3579896952 <== mon.0 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f03f401c630 con 0x7f04000fbcc0 2024-04-27T04:06:51.554 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:51.552+0000 7f040542b700 1 -- 172.21.15.162:0/3579896952 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0400113650 con 0x7f04000fbcc0 2024-04-27T04:06:51.554 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:51.553+0000 7f040542b700 1 -- 172.21.15.162:0/3579896952 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f040004c710 con 0x7f04000fbcc0 2024-04-27T04:06:51.554 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:51.553+0000 7f03e77fe700 1 -- 172.21.15.162:0/3579896952 <== mon.0 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 19) v1 ==== 86631+0+0 (secure 0 0 0) 0x7f03f4013420 con 0x7f04000fbcc0 2024-04-27T04:06:51.555 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:51.554+0000 7f03e77fe700 1 --2- 172.21.15.162:0/3579896952 >> [v2:172.21.15.162:6800/4079782599,v1:172.21.15.162:6801/4079782599] conn(0x7f03e80692a0 0x7f03e806b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:06:51.555 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:51.554+0000 7f03fe7fc700 1 --2- 172.21.15.162:0/3579896952 >> [v2:172.21.15.162:6800/4079782599,v1:172.21.15.162:6801/4079782599] conn(0x7f03e80692a0 0x7f03e806b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:06:51.555 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:51.554+0000 7f03e77fe700 1 -- 172.21.15.162:0/3579896952 <== mon.0 v2:172.21.15.162:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4634+0+0 (secure 0 0 0) 0x7f03f4087c60 con 0x7f04000fbcc0 2024-04-27T04:06:51.555 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:51.554+0000 7f03fe7fc700 1 --2- 172.21.15.162:0/3579896952 >> [v2:172.21.15.162:6800/4079782599,v1:172.21.15.162:6801/4079782599] conn(0x7f03e80692a0 0x7f03e806b750 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f03f0006fd0 tx=0x7f03f0008040 comp rx=0 tx=0).ready entity=mgr.14210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:06:51.558 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:51.558+0000 7f03e77fe700 1 -- 172.21.15.162:0/3579896952 <== mon.0 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f03f4057d60 con 0x7f04000fbcc0 2024-04-27T04:06:51.772 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:51.772+0000 7f040542b700 1 -- 172.21.15.162:0/3579896952 --> [v2:172.21.15.162:6800/4079782599,v1:172.21.15.162:6801/4079782599] -- mgr_command(tid 0: {"prefix": "fs volume create", "name": "cephfs", "placement": "4", "target": ["mon-mgr", ""]}) v1 -- 0x7f0400101250 con 0x7f03e80692a0 2024-04-27T04:06:52.387 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:06:52 smithi162 ceph-mon[26841]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-27T04:06:52.387 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:06:52 smithi162 ceph-mon[26841]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "osd pool create", "pool": "cephfs.cephfs.meta"}]: dispatch 2024-04-27T04:06:52.423 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:06:52 smithi181 ceph-mon[33774]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-27T04:06:52.423 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:06:52 smithi181 ceph-mon[33774]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "osd pool create", "pool": "cephfs.cephfs.meta"}]: dispatch 2024-04-27T04:06:53.114 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:53.114+0000 7f03e77fe700 1 -- 172.21.15.162:0/3579896952 <== mgr.14210 v2:172.21.15.162:6800/4079782599 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f0400101250 con 0x7f03e80692a0 2024-04-27T04:06:53.118 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:53.118+0000 7f040542b700 1 -- 172.21.15.162:0/3579896952 >> [v2:172.21.15.162:6800/4079782599,v1:172.21.15.162:6801/4079782599] conn(0x7f03e80692a0 msgr2=0x7f03e806b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:06:53.118 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:53.118+0000 7f040542b700 1 --2- 172.21.15.162:0/3579896952 >> [v2:172.21.15.162:6800/4079782599,v1:172.21.15.162:6801/4079782599] conn(0x7f03e80692a0 0x7f03e806b750 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f03f0006fd0 tx=0x7f03f0008040 comp rx=0 tx=0).stop 2024-04-27T04:06:53.119 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:53.118+0000 7f040542b700 1 -- 172.21.15.162:0/3579896952 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f04000fbcc0 msgr2=0x7f0400066d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:06:53.119 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:53.118+0000 7f040542b700 1 --2- 172.21.15.162:0/3579896952 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f04000fbcc0 0x7f0400066d30 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f03f4009150 tx=0x7f03f4009e30 comp rx=0 tx=0).stop 2024-04-27T04:06:53.119 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:53.118+0000 7f040542b700 1 -- 172.21.15.162:0/3579896952 shutdown_connections 2024-04-27T04:06:53.119 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:53.118+0000 7f040542b700 1 --2- 172.21.15.162:0/3579896952 >> [v2:172.21.15.162:6800/4079782599,v1:172.21.15.162:6801/4079782599] conn(0x7f03e80692a0 0x7f03e806b750 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:06:53.119 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:53.118+0000 7f040542b700 1 --2- 172.21.15.162:0/3579896952 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f04000fbcc0 0x7f0400066d30 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:06:53.119 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:53.118+0000 7f040542b700 1 --2- 172.21.15.162:0/3579896952 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f04000fcf70 0x7f0400112760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:06:53.119 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:53.118+0000 7f040542b700 1 -- 172.21.15.162:0/3579896952 >> 172.21.15.162:0/3579896952 conn(0x7f04000f7420 msgr2=0x7f04000f8ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:06:53.119 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:53.118+0000 7f040542b700 1 -- 172.21.15.162:0/3579896952 shutdown_connections 2024-04-27T04:06:53.119 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:53.118+0000 7f040542b700 1 -- 172.21.15.162:0/3579896952 wait complete. 2024-04-27T04:06:53.387 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:06:53 smithi162 ceph-mon[26841]: from='client.14452 -' entity='client.admin' cmd=[{"prefix": "fs volume create", "name": "cephfs", "placement": "4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-27T04:06:53.387 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:06:53 smithi162 ceph-mon[26841]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' cmd='[{"prefix": "osd pool create", "pool": "cephfs.cephfs.meta"}]': finished 2024-04-27T04:06:53.388 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:06:53 smithi162 ceph-mon[26841]: osdmap e41: 6 total, 6 up, 6 in 2024-04-27T04:06:53.388 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:06:53 smithi162 ceph-mon[26841]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' cmd=[{"bulk": true, "prefix": "osd pool create", "pool": "cephfs.cephfs.data"}]: dispatch 2024-04-27T04:06:53.388 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:06:53 smithi162 ceph-7f9750d6-044a-11ef-bc93-c7b262605968-mon-smithi162[26837]: 2024-04-27T04:06:53.094+0000 7f82b91ab700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-27T04:06:53.423 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:06:53 smithi181 ceph-mon[33774]: from='client.14452 -' entity='client.admin' cmd=[{"prefix": "fs volume create", "name": "cephfs", "placement": "4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-27T04:06:53.423 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:06:53 smithi181 ceph-mon[33774]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' cmd='[{"prefix": "osd pool create", "pool": "cephfs.cephfs.meta"}]': finished 2024-04-27T04:06:53.423 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:06:53 smithi181 ceph-mon[33774]: osdmap e41: 6 total, 6 up, 6 in 2024-04-27T04:06:53.423 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:06:53 smithi181 ceph-mon[33774]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' cmd=[{"bulk": true, "prefix": "osd pool create", "pool": "cephfs.cephfs.data"}]: dispatch 2024-04-27T04:06:53.820 DEBUG:teuthology.orchestra.run.smithi162:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 7f9750d6-044a-11ef-bc93-c7b262605968 -- bash -c 'ceph fs dump' 2024-04-27T04:06:54.356 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:06:54 smithi162 ceph-mon[26841]: pgmap v167: 33 pgs: 19 creating+peering, 13 unknown, 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-27T04:06:54.356 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:06:54 smithi162 ceph-mon[26841]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-27T04:06:54.356 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:06:54 smithi162 ceph-mon[26841]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' cmd='[{"bulk": true, "prefix": "osd pool create", "pool": "cephfs.cephfs.data"}]': finished 2024-04-27T04:06:54.356 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:06:54 smithi162 ceph-mon[26841]: osdmap e42: 6 total, 6 up, 6 in 2024-04-27T04:06:54.356 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:06:54 smithi162 ceph-mon[26841]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs.cephfs.meta", "data": "cephfs.cephfs.data"}]: dispatch 2024-04-27T04:06:54.356 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:06:54 smithi162 ceph-mon[26841]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-27T04:06:54.356 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:06:54 smithi162 ceph-mon[26841]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-04-27T04:06:54.356 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:06:54 smithi162 ceph-mon[26841]: osdmap e43: 6 total, 6 up, 6 in 2024-04-27T04:06:54.356 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:06:54 smithi162 ceph-mon[26841]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs.cephfs.meta", "data": "cephfs.cephfs.data"}]': finished 2024-04-27T04:06:54.356 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:06:54 smithi162 ceph-mon[26841]: fsmap cephfs:0 2024-04-27T04:06:54.357 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:06:54 smithi162 ceph-mon[26841]: Saving service mds.cephfs spec with placement count:4 2024-04-27T04:06:54.357 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:06:54 smithi162 ceph-mon[26841]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' 2024-04-27T04:06:54.357 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:06:54 smithi162 ceph-mon[26841]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T04:06:54.357 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:06:54 smithi162 ceph-mon[26841]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T04:06:54.357 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:06:54 smithi162 ceph-mon[26841]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T04:06:54.357 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:06:54 smithi162 ceph-mon[26841]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' 2024-04-27T04:06:54.357 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:06:54 smithi162 ceph-mon[26841]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi162.gzlyjj", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-27T04:06:54.357 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:06:54 smithi162 ceph-mon[26841]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' cmd='[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi162.gzlyjj", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-27T04:06:54.357 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:06:54 smithi162 ceph-mon[26841]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T04:06:54.423 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:06:54 smithi181 ceph-mon[33774]: pgmap v167: 33 pgs: 19 creating+peering, 13 unknown, 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-27T04:06:54.423 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:06:54 smithi181 ceph-mon[33774]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-27T04:06:54.423 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:06:54 smithi181 ceph-mon[33774]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' cmd='[{"bulk": true, "prefix": "osd pool create", "pool": "cephfs.cephfs.data"}]': finished 2024-04-27T04:06:54.423 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:06:54 smithi181 ceph-mon[33774]: osdmap e42: 6 total, 6 up, 6 in 2024-04-27T04:06:54.423 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:06:54 smithi181 ceph-mon[33774]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs.cephfs.meta", "data": "cephfs.cephfs.data"}]: dispatch 2024-04-27T04:06:54.423 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:06:54 smithi181 ceph-mon[33774]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-27T04:06:54.423 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:06:54 smithi181 ceph-mon[33774]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-04-27T04:06:54.424 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:06:54 smithi181 ceph-mon[33774]: osdmap e43: 6 total, 6 up, 6 in 2024-04-27T04:06:54.424 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:06:54 smithi181 ceph-mon[33774]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs.cephfs.meta", "data": "cephfs.cephfs.data"}]': finished 2024-04-27T04:06:54.424 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:06:54 smithi181 ceph-mon[33774]: fsmap cephfs:0 2024-04-27T04:06:54.424 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:06:54 smithi181 ceph-mon[33774]: Saving service mds.cephfs spec with placement count:4 2024-04-27T04:06:54.424 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:06:54 smithi181 ceph-mon[33774]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' 2024-04-27T04:06:54.424 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:06:54 smithi181 ceph-mon[33774]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T04:06:54.424 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:06:54 smithi181 ceph-mon[33774]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T04:06:54.424 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:06:54 smithi181 ceph-mon[33774]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T04:06:54.424 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:06:54 smithi181 ceph-mon[33774]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' 2024-04-27T04:06:54.424 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:06:54 smithi181 ceph-mon[33774]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi162.gzlyjj", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-27T04:06:54.424 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:06:54 smithi181 ceph-mon[33774]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' cmd='[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi162.gzlyjj", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-27T04:06:54.424 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:06:54 smithi181 ceph-mon[33774]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T04:06:54.608 INFO:teuthology.orchestra.run.smithi162.stderr:Inferring config /var/lib/ceph/7f9750d6-044a-11ef-bc93-c7b262605968/mon.smithi162/config 2024-04-27T04:06:55.385 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:06:55 smithi162 ceph-mon[26841]: Deploying daemon mds.cephfs.smithi162.gzlyjj on smithi162 2024-04-27T04:06:55.385 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:06:55 smithi162 ceph-mon[26841]: osdmap e44: 6 total, 6 up, 6 in 2024-04-27T04:06:55.423 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:06:55 smithi181 ceph-mon[33774]: Deploying daemon mds.cephfs.smithi162.gzlyjj on smithi162 2024-04-27T04:06:55.423 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:06:55 smithi181 ceph-mon[33774]: osdmap e44: 6 total, 6 up, 6 in 2024-04-27T04:06:56.287 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:06:56 smithi162 ceph-mon[26841]: pgmap v171: 65 pgs: 24 creating+peering, 40 unknown, 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-27T04:06:56.287 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:06:56 smithi162 ceph-mon[26841]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-04-27T04:06:56.287 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:06:56 smithi162 ceph-mon[26841]: osdmap e45: 6 total, 6 up, 6 in 2024-04-27T04:06:56.423 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:06:56 smithi181 ceph-mon[33774]: pgmap v171: 65 pgs: 24 creating+peering, 40 unknown, 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-27T04:06:56.423 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:06:56 smithi181 ceph-mon[33774]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-04-27T04:06:56.423 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:06:56 smithi181 ceph-mon[33774]: osdmap e45: 6 total, 6 up, 6 in 2024-04-27T04:06:56.662 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:56.660+0000 7f981659f700 1 -- 172.21.15.162:0/2858412802 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9810066240 msgr2=0x7f98100ff860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:06:56.662 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:56.660+0000 7f981659f700 1 --2- 172.21.15.162:0/2858412802 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9810066240 0x7f98100ff860 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f9808009ce0 tx=0x7f9808004d10 comp rx=0 tx=0).stop 2024-04-27T04:06:56.662 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:56.660+0000 7f981659f700 1 -- 172.21.15.162:0/2858412802 shutdown_connections 2024-04-27T04:06:56.662 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:56.660+0000 7f981659f700 1 --2- 172.21.15.162:0/2858412802 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f98100ffda0 0x7f9810102180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:06:56.662 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:56.660+0000 7f981659f700 1 --2- 172.21.15.162:0/2858412802 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9810066240 0x7f98100ff860 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:06:56.662 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:56.660+0000 7f981659f700 1 -- 172.21.15.162:0/2858412802 >> 172.21.15.162:0/2858412802 conn(0x7f98100f7250 msgr2=0x7f98100f9680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:06:56.662 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:56.660+0000 7f981659f700 1 -- 172.21.15.162:0/2858412802 shutdown_connections 2024-04-27T04:06:56.662 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:56.660+0000 7f981659f700 1 -- 172.21.15.162:0/2858412802 wait complete. 2024-04-27T04:06:56.662 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:56.661+0000 7f981659f700 1 Processor -- start 2024-04-27T04:06:56.662 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:56.661+0000 7f981659f700 1 -- start start 2024-04-27T04:06:56.663 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:56.661+0000 7f981659f700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9810066240 0x7f98100fd290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:06:56.663 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:56.661+0000 7f981659f700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f98100ffda0 0x7f98100fb760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:06:56.663 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:56.661+0000 7f981659f700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98100fbd10 con 0x7f9810066240 2024-04-27T04:06:56.663 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:56.661+0000 7f981659f700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98100fbe80 con 0x7f98100ffda0 2024-04-27T04:06:56.663 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:56.661+0000 7f981559d700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9810066240 0x7f98100fd290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:06:56.663 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:56.661+0000 7f981559d700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9810066240 0x7f98100fd290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:46506/0 (socket says 172.21.15.162:46506) 2024-04-27T04:06:56.663 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:56.661+0000 7f981559d700 1 -- 172.21.15.162:0/1521523153 learned_addr learned my addr 172.21.15.162:0/1521523153 (peer_addr_for_me v2:172.21.15.162:0/0) 2024-04-27T04:06:56.663 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:56.661+0000 7f9814d9c700 1 --2- 172.21.15.162:0/1521523153 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f98100ffda0 0x7f98100fb760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:06:56.663 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:56.661+0000 7f981559d700 1 -- 172.21.15.162:0/1521523153 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f98100ffda0 msgr2=0x7f98100fb760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:06:56.663 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:56.661+0000 7f981559d700 1 --2- 172.21.15.162:0/1521523153 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f98100ffda0 0x7f98100fb760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:06:56.663 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:56.661+0000 7f981559d700 1 -- 172.21.15.162:0/1521523153 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9808009940 con 0x7f9810066240 2024-04-27T04:06:56.663 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:56.662+0000 7f981559d700 1 --2- 172.21.15.162:0/1521523153 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9810066240 0x7f98100fd290 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f9808009380 tx=0x7f9808004d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:06:56.663 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:56.662+0000 7f980dffb700 1 -- 172.21.15.162:0/1521523153 <== mon.0 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f980800e040 con 0x7f9810066240 2024-04-27T04:06:56.663 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:56.662+0000 7f980dffb700 1 -- 172.21.15.162:0/1521523153 <== mon.0 v2:172.21.15.162:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f98080128e0 con 0x7f9810066240 2024-04-27T04:06:56.664 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:56.662+0000 7f980dffb700 1 -- 172.21.15.162:0/1521523153 <== mon.0 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f980801c910 con 0x7f9810066240 2024-04-27T04:06:56.664 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:56.662+0000 7f981659f700 1 -- 172.21.15.162:0/1521523153 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f98100fc100 con 0x7f9810066240 2024-04-27T04:06:56.664 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:56.662+0000 7f981659f700 1 -- 172.21.15.162:0/1521523153 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9810115a40 con 0x7f9810066240 2024-04-27T04:06:56.665 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:56.663+0000 7f981659f700 1 -- 172.21.15.162:0/1521523153 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f981005f5c0 con 0x7f9810066240 2024-04-27T04:06:56.665 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:56.663+0000 7f980dffb700 1 -- 172.21.15.162:0/1521523153 <== mon.0 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 19) v1 ==== 86631+0+0 (secure 0 0 0) 0x7f9808012a50 con 0x7f9810066240 2024-04-27T04:06:56.665 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:56.664+0000 7f980dffb700 1 --2- 172.21.15.162:0/1521523153 >> [v2:172.21.15.162:6800/4079782599,v1:172.21.15.162:6801/4079782599] conn(0x7f97f80692a0 0x7f97f806b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:06:56.665 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:56.664+0000 7f980dffb700 1 -- 172.21.15.162:0/1521523153 <== mon.0 v2:172.21.15.162:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5464+0+0 (secure 0 0 0) 0x7f9808088e20 con 0x7f9810066240 2024-04-27T04:06:56.665 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:56.664+0000 7f9814d9c700 1 --2- 172.21.15.162:0/1521523153 >> [v2:172.21.15.162:6800/4079782599,v1:172.21.15.162:6801/4079782599] conn(0x7f97f80692a0 0x7f97f806b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:06:56.665 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:56.664+0000 7f9814d9c700 1 --2- 172.21.15.162:0/1521523153 >> [v2:172.21.15.162:6800/4079782599,v1:172.21.15.162:6801/4079782599] conn(0x7f97f80692a0 0x7f97f806b750 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f9804004e00 tx=0x7f980400b040 comp rx=0 tx=0).ready entity=mgr.14210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:06:56.668 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:56.668+0000 7f980dffb700 1 -- 172.21.15.162:0/1521523153 <== mon.0 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f98080550d0 con 0x7f9810066240 2024-04-27T04:06:56.914 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:56.914+0000 7f981659f700 1 -- 172.21.15.162:0/1521523153 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f9810115d30 con 0x7f9810066240 2024-04-27T04:06:56.914 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:56.914+0000 7f980dffb700 1 -- 172.21.15.162:0/1521523153 <== mon.0 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 2 v2) v1 ==== 75+0+1076 (secure 0 0 0) 0x7f98080586f0 con 0x7f9810066240 2024-04-27T04:06:56.914 INFO:teuthology.orchestra.run.smithi162.stdout:e2 2024-04-27T04:06:56.915 INFO:teuthology.orchestra.run.smithi162.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-04-27T04:06:56.915 INFO:teuthology.orchestra.run.smithi162.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-27T04:06:56.915 INFO:teuthology.orchestra.run.smithi162.stdout:legacy client fscid: 1 2024-04-27T04:06:56.915 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-04-27T04:06:56.915 INFO:teuthology.orchestra.run.smithi162.stdout:Filesystem 'cephfs' (1) 2024-04-27T04:06:56.915 INFO:teuthology.orchestra.run.smithi162.stdout:fs_name cephfs 2024-04-27T04:06:56.915 INFO:teuthology.orchestra.run.smithi162.stdout:epoch 2 2024-04-27T04:06:56.915 INFO:teuthology.orchestra.run.smithi162.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2024-04-27T04:06:56.915 INFO:teuthology.orchestra.run.smithi162.stdout:created 2024-04-27T04:06:53.095749+0000 2024-04-27T04:06:56.915 INFO:teuthology.orchestra.run.smithi162.stdout:modified 2024-04-27T04:06:53.095830+0000 2024-04-27T04:06:56.915 INFO:teuthology.orchestra.run.smithi162.stdout:tableserver 0 2024-04-27T04:06:56.915 INFO:teuthology.orchestra.run.smithi162.stdout:root 0 2024-04-27T04:06:56.915 INFO:teuthology.orchestra.run.smithi162.stdout:session_timeout 60 2024-04-27T04:06:56.915 INFO:teuthology.orchestra.run.smithi162.stdout:session_autoclose 300 2024-04-27T04:06:56.915 INFO:teuthology.orchestra.run.smithi162.stdout:max_file_size 1099511627776 2024-04-27T04:06:56.916 INFO:teuthology.orchestra.run.smithi162.stdout:required_client_features {} 2024-04-27T04:06:56.916 INFO:teuthology.orchestra.run.smithi162.stdout:last_failure 0 2024-04-27T04:06:56.916 INFO:teuthology.orchestra.run.smithi162.stdout:last_failure_osd_epoch 0 2024-04-27T04:06:56.916 INFO:teuthology.orchestra.run.smithi162.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-27T04:06:56.916 INFO:teuthology.orchestra.run.smithi162.stdout:max_mds 1 2024-04-27T04:06:56.916 INFO:teuthology.orchestra.run.smithi162.stdout:in 2024-04-27T04:06:56.916 INFO:teuthology.orchestra.run.smithi162.stdout:up {} 2024-04-27T04:06:56.916 INFO:teuthology.orchestra.run.smithi162.stdout:failed 2024-04-27T04:06:56.916 INFO:teuthology.orchestra.run.smithi162.stdout:damaged 2024-04-27T04:06:56.916 INFO:teuthology.orchestra.run.smithi162.stdout:stopped 2024-04-27T04:06:56.916 INFO:teuthology.orchestra.run.smithi162.stdout:data_pools [3] 2024-04-27T04:06:56.916 INFO:teuthology.orchestra.run.smithi162.stdout:metadata_pool 2 2024-04-27T04:06:56.916 INFO:teuthology.orchestra.run.smithi162.stdout:inline_data disabled 2024-04-27T04:06:56.916 INFO:teuthology.orchestra.run.smithi162.stdout:balancer 2024-04-27T04:06:56.916 INFO:teuthology.orchestra.run.smithi162.stdout:standby_count_wanted 0 2024-04-27T04:06:56.916 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-04-27T04:06:56.917 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-04-27T04:06:56.917 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:56.916+0000 7f981659f700 1 -- 172.21.15.162:0/1521523153 >> [v2:172.21.15.162:6800/4079782599,v1:172.21.15.162:6801/4079782599] conn(0x7f97f80692a0 msgr2=0x7f97f806b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:06:56.917 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:56.916+0000 7f981659f700 1 --2- 172.21.15.162:0/1521523153 >> [v2:172.21.15.162:6800/4079782599,v1:172.21.15.162:6801/4079782599] conn(0x7f97f80692a0 0x7f97f806b750 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f9804004e00 tx=0x7f980400b040 comp rx=0 tx=0).stop 2024-04-27T04:06:56.917 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:56.916+0000 7f981659f700 1 -- 172.21.15.162:0/1521523153 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9810066240 msgr2=0x7f98100fd290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:06:56.917 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:56.916+0000 7f981659f700 1 --2- 172.21.15.162:0/1521523153 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9810066240 0x7f98100fd290 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f9808009380 tx=0x7f9808004d10 comp rx=0 tx=0).stop 2024-04-27T04:06:56.918 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:56.916+0000 7f981659f700 1 -- 172.21.15.162:0/1521523153 shutdown_connections 2024-04-27T04:06:56.918 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:56.916+0000 7f981659f700 1 --2- 172.21.15.162:0/1521523153 >> [v2:172.21.15.162:6800/4079782599,v1:172.21.15.162:6801/4079782599] conn(0x7f97f80692a0 0x7f97f806b750 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:06:56.918 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:56.916+0000 7f981659f700 1 --2- 172.21.15.162:0/1521523153 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9810066240 0x7f98100fd290 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:06:56.918 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:56.916+0000 7f981659f700 1 --2- 172.21.15.162:0/1521523153 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f98100ffda0 0x7f98100fb760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:06:56.918 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:56.916+0000 7f981659f700 1 -- 172.21.15.162:0/1521523153 >> 172.21.15.162:0/1521523153 conn(0x7f98100f7250 msgr2=0x7f98100f7670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:06:56.918 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:56.916+0000 7f981659f700 1 -- 172.21.15.162:0/1521523153 shutdown_connections 2024-04-27T04:06:56.918 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:56.916+0000 7f981659f700 1 -- 172.21.15.162:0/1521523153 wait complete. 2024-04-27T04:06:56.918 INFO:teuthology.orchestra.run.smithi162.stderr:dumped fsmap epoch 2 2024-04-27T04:06:57.137 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:06:57 smithi162 ceph-mon[26841]: from='client.? 172.21.15.162:0/1521523153' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-27T04:06:57.423 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:06:57 smithi181 ceph-mon[33774]: from='client.? 172.21.15.162:0/1521523153' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-27T04:06:57.927 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-04-27T04:06:57.937 INFO:tasks.cephadm:Running commands on role host.a host ubuntu@smithi162.front.sepia.ceph.com 2024-04-27T04:06:57.937 DEBUG:teuthology.orchestra.run.smithi162:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 7f9750d6-044a-11ef-bc93-c7b262605968 -- bash -c 'ceph fs set cephfs max_mds 1' 2024-04-27T04:06:58.173 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:06:58 smithi181 ceph-mon[33774]: pgmap v173: 65 pgs: 24 creating+peering, 22 unknown, 19 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-27T04:06:58.173 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:06:58 smithi181 ceph-mon[33774]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' 2024-04-27T04:06:58.173 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:06:58 smithi181 ceph-mon[33774]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' 2024-04-27T04:06:58.173 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:06:58 smithi181 ceph-mon[33774]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' 2024-04-27T04:06:58.173 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:06:58 smithi181 ceph-mon[33774]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi181.dzomgh", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-27T04:06:58.173 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:06:58 smithi181 ceph-mon[33774]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' cmd='[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi181.dzomgh", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-27T04:06:58.174 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:06:58 smithi181 ceph-mon[33774]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T04:06:58.195 INFO:teuthology.orchestra.run.smithi162.stderr:Inferring config /var/lib/ceph/7f9750d6-044a-11ef-bc93-c7b262605968/mon.smithi162/config 2024-04-27T04:06:58.318 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:06:58 smithi162 ceph-mon[26841]: pgmap v173: 65 pgs: 24 creating+peering, 22 unknown, 19 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-27T04:06:58.319 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:06:58 smithi162 ceph-mon[26841]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' 2024-04-27T04:06:58.319 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:06:58 smithi162 ceph-mon[26841]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' 2024-04-27T04:06:58.319 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:06:58 smithi162 ceph-mon[26841]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' 2024-04-27T04:06:58.319 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:06:58 smithi162 ceph-mon[26841]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi181.dzomgh", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-27T04:06:58.319 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:06:58 smithi162 ceph-mon[26841]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' cmd='[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi181.dzomgh", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-27T04:06:58.319 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:06:58 smithi162 ceph-mon[26841]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T04:06:59.318 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:06:59 smithi162 ceph-mon[26841]: Deploying daemon mds.cephfs.smithi181.dzomgh on smithi181 2024-04-27T04:06:59.318 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:06:59 smithi162 ceph-mon[26841]: mds.? [v2:172.21.15.162:6826/998468496,v1:172.21.15.162:6827/998468496] up:boot 2024-04-27T04:06:59.318 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:06:59 smithi162 ceph-mon[26841]: daemon mds.cephfs.smithi162.gzlyjj assigned to filesystem cephfs as rank 0 (now has 1 ranks) 2024-04-27T04:06:59.318 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:06:59 smithi162 ceph-mon[26841]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-04-27T04:06:59.318 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:06:59 smithi162 ceph-mon[26841]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-04-27T04:06:59.318 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:06:59 smithi162 ceph-mon[26841]: Cluster is now healthy 2024-04-27T04:06:59.318 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:06:59 smithi162 ceph-mon[26841]: fsmap cephfs:0 1 up:standby 2024-04-27T04:06:59.318 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:06:59 smithi162 ceph-mon[26841]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi162.gzlyjj"}]: dispatch 2024-04-27T04:06:59.318 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:06:59 smithi162 ceph-mon[26841]: fsmap cephfs:1 {0=cephfs.smithi162.gzlyjj=up:creating} 2024-04-27T04:06:59.318 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:06:59 smithi162 ceph-mon[26841]: daemon mds.cephfs.smithi162.gzlyjj is now active in filesystem cephfs as rank 0 2024-04-27T04:06:59.423 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:06:59 smithi181 ceph-mon[33774]: Deploying daemon mds.cephfs.smithi181.dzomgh on smithi181 2024-04-27T04:06:59.423 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:06:59 smithi181 ceph-mon[33774]: mds.? [v2:172.21.15.162:6826/998468496,v1:172.21.15.162:6827/998468496] up:boot 2024-04-27T04:06:59.423 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:06:59 smithi181 ceph-mon[33774]: daemon mds.cephfs.smithi162.gzlyjj assigned to filesystem cephfs as rank 0 (now has 1 ranks) 2024-04-27T04:06:59.423 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:06:59 smithi181 ceph-mon[33774]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-04-27T04:06:59.423 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:06:59 smithi181 ceph-mon[33774]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-04-27T04:06:59.423 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:06:59 smithi181 ceph-mon[33774]: Cluster is now healthy 2024-04-27T04:06:59.423 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:06:59 smithi181 ceph-mon[33774]: fsmap cephfs:0 1 up:standby 2024-04-27T04:06:59.423 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:06:59 smithi181 ceph-mon[33774]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi162.gzlyjj"}]: dispatch 2024-04-27T04:06:59.423 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:06:59 smithi181 ceph-mon[33774]: fsmap cephfs:1 {0=cephfs.smithi162.gzlyjj=up:creating} 2024-04-27T04:06:59.424 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:06:59 smithi181 ceph-mon[33774]: daemon mds.cephfs.smithi162.gzlyjj is now active in filesystem cephfs as rank 0 2024-04-27T04:06:59.820 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:59.818+0000 7f7577e41700 1 -- 172.21.15.162:0/4207254551 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f75700fbcb0 msgr2=0x7f75700fc0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:06:59.820 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:59.818+0000 7f7577e41700 1 --2- 172.21.15.162:0/4207254551 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f75700fbcb0 0x7f75700fc0c0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f756c004940 tx=0x7f756c009e30 comp rx=0 tx=0).stop 2024-04-27T04:06:59.820 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:59.818+0000 7f7577e41700 1 -- 172.21.15.162:0/4207254551 shutdown_connections 2024-04-27T04:06:59.821 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:59.818+0000 7f7577e41700 1 --2- 172.21.15.162:0/4207254551 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f75700fcf60 0x7f75700fd3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:06:59.821 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:59.818+0000 7f7577e41700 1 --2- 172.21.15.162:0/4207254551 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f75700fbcb0 0x7f75700fc0c0 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:06:59.821 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:59.818+0000 7f7577e41700 1 -- 172.21.15.162:0/4207254551 >> 172.21.15.162:0/4207254551 conn(0x7f75700f7430 msgr2=0x7f75700f9840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:06:59.821 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:59.818+0000 7f7577e41700 1 -- 172.21.15.162:0/4207254551 shutdown_connections 2024-04-27T04:06:59.821 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:59.818+0000 7f7577e41700 1 -- 172.21.15.162:0/4207254551 wait complete. 2024-04-27T04:06:59.821 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:59.819+0000 7f7577e41700 1 Processor -- start 2024-04-27T04:06:59.821 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:59.819+0000 7f7577e41700 1 -- start start 2024-04-27T04:06:59.821 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:59.819+0000 7f7577e41700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f75700fbcb0 0x7f75701106a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:06:59.821 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:59.819+0000 7f7577e41700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f75700fcf60 0x7f7570112bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:06:59.821 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:59.819+0000 7f7577e41700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7570066330 con 0x7f75700fbcb0 2024-04-27T04:06:59.821 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:59.819+0000 7f7577e41700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7570066470 con 0x7f75700fcf60 2024-04-27T04:06:59.821 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:59.819+0000 7f7575bdd700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f75700fbcb0 0x7f75701106a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:06:59.821 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:59.819+0000 7f7575bdd700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f75700fbcb0 0x7f75701106a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:46532/0 (socket says 172.21.15.162:46532) 2024-04-27T04:06:59.821 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:59.819+0000 7f7575bdd700 1 -- 172.21.15.162:0/2787560245 learned_addr learned my addr 172.21.15.162:0/2787560245 (peer_addr_for_me v2:172.21.15.162:0/0) 2024-04-27T04:06:59.822 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:59.819+0000 7f75753dc700 1 --2- 172.21.15.162:0/2787560245 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f75700fcf60 0x7f7570112bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:06:59.822 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:59.819+0000 7f7575bdd700 1 -- 172.21.15.162:0/2787560245 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f75700fcf60 msgr2=0x7f7570112bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:06:59.822 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:59.819+0000 7f7575bdd700 1 --2- 172.21.15.162:0/2787560245 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f75700fcf60 0x7f7570112bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:06:59.822 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:59.819+0000 7f7575bdd700 1 -- 172.21.15.162:0/2787560245 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f756c005040 con 0x7f75700fbcb0 2024-04-27T04:06:59.822 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:59.820+0000 7f7575bdd700 1 --2- 172.21.15.162:0/2787560245 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f75700fbcb0 0x7f75701106a0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f756c013870 tx=0x7f756c009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:06:59.822 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:59.820+0000 7f75667fc700 1 -- 172.21.15.162:0/2787560245 <== mon.0 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f756c021070 con 0x7f75700fbcb0 2024-04-27T04:06:59.822 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:59.820+0000 7f7577e41700 1 -- 172.21.15.162:0/2787560245 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7570113130 con 0x7f75700fbcb0 2024-04-27T04:06:59.822 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:59.820+0000 7f75667fc700 1 -- 172.21.15.162:0/2787560245 <== mon.0 v2:172.21.15.162:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f756c009430 con 0x7f75700fbcb0 2024-04-27T04:06:59.822 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:59.820+0000 7f75667fc700 1 -- 172.21.15.162:0/2787560245 <== mon.0 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f756c00e070 con 0x7f75700fbcb0 2024-04-27T04:06:59.822 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:59.820+0000 7f7577e41700 1 -- 172.21.15.162:0/2787560245 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7570113670 con 0x7f75700fbcb0 2024-04-27T04:06:59.823 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:59.821+0000 7f7577e41700 1 -- 172.21.15.162:0/2787560245 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f757005f5c0 con 0x7f75700fbcb0 2024-04-27T04:06:59.823 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:59.821+0000 7f75667fc700 1 -- 172.21.15.162:0/2787560245 <== mon.0 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 19) v1 ==== 86631+0+0 (secure 0 0 0) 0x7f756c01c750 con 0x7f75700fbcb0 2024-04-27T04:06:59.823 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:59.822+0000 7f75667fc700 1 --2- 172.21.15.162:0/2787560245 >> [v2:172.21.15.162:6800/4079782599,v1:172.21.15.162:6801/4079782599] conn(0x7f755c069230 0x7f755c06b6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:06:59.823 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:59.822+0000 7f75667fc700 1 -- 172.21.15.162:0/2787560245 <== mon.0 v2:172.21.15.162:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5464+0+0 (secure 0 0 0) 0x7f756c088460 con 0x7f75700fbcb0 2024-04-27T04:06:59.823 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:59.822+0000 7f75753dc700 1 --2- 172.21.15.162:0/2787560245 >> [v2:172.21.15.162:6800/4079782599,v1:172.21.15.162:6801/4079782599] conn(0x7f755c069230 0x7f755c06b6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:06:59.823 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:59.822+0000 7f75753dc700 1 --2- 172.21.15.162:0/2787560245 >> [v2:172.21.15.162:6800/4079782599,v1:172.21.15.162:6801/4079782599] conn(0x7f755c069230 0x7f755c06b6e0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f7560009b80 tx=0x7f756000b040 comp rx=0 tx=0).ready entity=mgr.14210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:06:59.826 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:06:59.826+0000 7f75667fc700 1 -- 172.21.15.162:0/2787560245 <== mon.0 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f756c054790 con 0x7f75700fbcb0 2024-04-27T04:07:00.069 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:07:00.069+0000 7f7577e41700 1 -- 172.21.15.162:0/2787560245 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "1"} v 0) v1 -- 0x7f75701138e0 con 0x7f75700fbcb0 2024-04-27T04:07:00.137 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:07:00.137+0000 7f75667fc700 1 -- 172.21.15.162:0/2787560245 <== mon.0 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "1"}]=0 v6) v1 ==== 105+0+0 (secure 0 0 0) 0x7f756c057db0 con 0x7f75700fbcb0 2024-04-27T04:07:00.140 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:07:00.139+0000 7f7577e41700 1 -- 172.21.15.162:0/2787560245 >> [v2:172.21.15.162:6800/4079782599,v1:172.21.15.162:6801/4079782599] conn(0x7f755c069230 msgr2=0x7f755c06b6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:07:00.140 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:07:00.140+0000 7f7577e41700 1 --2- 172.21.15.162:0/2787560245 >> [v2:172.21.15.162:6800/4079782599,v1:172.21.15.162:6801/4079782599] conn(0x7f755c069230 0x7f755c06b6e0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f7560009b80 tx=0x7f756000b040 comp rx=0 tx=0).stop 2024-04-27T04:07:00.140 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:07:00.140+0000 7f7577e41700 1 -- 172.21.15.162:0/2787560245 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f75700fbcb0 msgr2=0x7f75701106a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:07:00.140 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:07:00.140+0000 7f7577e41700 1 --2- 172.21.15.162:0/2787560245 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f75700fbcb0 0x7f75701106a0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f756c013870 tx=0x7f756c009e30 comp rx=0 tx=0).stop 2024-04-27T04:07:00.140 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:07:00.140+0000 7f7577e41700 1 -- 172.21.15.162:0/2787560245 shutdown_connections 2024-04-27T04:07:00.140 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:07:00.140+0000 7f7577e41700 1 --2- 172.21.15.162:0/2787560245 >> [v2:172.21.15.162:6800/4079782599,v1:172.21.15.162:6801/4079782599] conn(0x7f755c069230 0x7f755c06b6e0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:07:00.140 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:07:00.140+0000 7f7577e41700 1 --2- 172.21.15.162:0/2787560245 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f75700fbcb0 0x7f75701106a0 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:07:00.140 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:07:00.140+0000 7f7577e41700 1 --2- 172.21.15.162:0/2787560245 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f75700fcf60 0x7f7570112bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:07:00.140 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:07:00.140+0000 7f7577e41700 1 -- 172.21.15.162:0/2787560245 >> 172.21.15.162:0/2787560245 conn(0x7f75700f7430 msgr2=0x7f75700f8a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:07:00.140 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:07:00.140+0000 7f7577e41700 1 -- 172.21.15.162:0/2787560245 shutdown_connections 2024-04-27T04:07:00.141 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:07:00.140+0000 7f7577e41700 1 -- 172.21.15.162:0/2787560245 wait complete. 2024-04-27T04:07:00.387 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:07:00 smithi162 ceph-mon[26841]: pgmap v174: 65 pgs: 65 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-27T04:07:00.388 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:07:00 smithi162 ceph-mon[26841]: mds.? [v2:172.21.15.162:6826/998468496,v1:172.21.15.162:6827/998468496] up:active 2024-04-27T04:07:00.388 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:07:00 smithi162 ceph-mon[26841]: fsmap cephfs:1 {0=cephfs.smithi162.gzlyjj=up:active} 2024-04-27T04:07:00.388 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:07:00 smithi162 ceph-mon[26841]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' 2024-04-27T04:07:00.388 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:07:00 smithi162 ceph-mon[26841]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' 2024-04-27T04:07:00.388 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:07:00 smithi162 ceph-mon[26841]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' 2024-04-27T04:07:00.388 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:07:00 smithi162 ceph-mon[26841]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi162.xcnapk", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-27T04:07:00.388 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:07:00 smithi162 ceph-mon[26841]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' cmd='[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi162.xcnapk", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-27T04:07:00.388 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:07:00 smithi162 ceph-mon[26841]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T04:07:00.388 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:07:00 smithi162 ceph-mon[26841]: from='client.? 172.21.15.162:0/2787560245' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "1"}]: dispatch 2024-04-27T04:07:00.423 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:07:00 smithi181 ceph-mon[33774]: pgmap v174: 65 pgs: 65 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-27T04:07:00.423 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:07:00 smithi181 ceph-mon[33774]: mds.? [v2:172.21.15.162:6826/998468496,v1:172.21.15.162:6827/998468496] up:active 2024-04-27T04:07:00.423 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:07:00 smithi181 ceph-mon[33774]: fsmap cephfs:1 {0=cephfs.smithi162.gzlyjj=up:active} 2024-04-27T04:07:00.423 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:07:00 smithi181 ceph-mon[33774]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' 2024-04-27T04:07:00.423 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:07:00 smithi181 ceph-mon[33774]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' 2024-04-27T04:07:00.423 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:07:00 smithi181 ceph-mon[33774]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' 2024-04-27T04:07:00.423 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:07:00 smithi181 ceph-mon[33774]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi162.xcnapk", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-27T04:07:00.423 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:07:00 smithi181 ceph-mon[33774]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' cmd='[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi162.xcnapk", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-27T04:07:00.423 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:07:00 smithi181 ceph-mon[33774]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T04:07:00.423 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:07:00 smithi181 ceph-mon[33774]: from='client.? 172.21.15.162:0/2787560245' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "1"}]: dispatch 2024-04-27T04:07:00.718 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-04-27T04:07:00.727 INFO:tasks.cephadm:Running commands on role host.a host ubuntu@smithi162.front.sepia.ceph.com 2024-04-27T04:07:00.728 DEBUG:teuthology.orchestra.run.smithi162:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 7f9750d6-044a-11ef-bc93-c7b262605968 -- bash -c 'ceph fs set cephfs allow_standby_replay true' 2024-04-27T04:07:01.387 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:07:01 smithi162 ceph-mon[26841]: Deploying daemon mds.cephfs.smithi162.xcnapk on smithi162 2024-04-27T04:07:01.388 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:07:01 smithi162 ceph-mon[26841]: mds.? [v2:172.21.15.181:6824/658214993,v1:172.21.15.181:6825/658214993] up:boot 2024-04-27T04:07:01.388 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:07:01 smithi162 ceph-mon[26841]: from='client.? 172.21.15.162:0/2787560245' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "1"}]': finished 2024-04-27T04:07:01.388 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:07:01 smithi162 ceph-mon[26841]: fsmap cephfs:1 {0=cephfs.smithi162.gzlyjj=up:active} 1 up:standby 2024-04-27T04:07:01.388 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:07:01 smithi162 ceph-mon[26841]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi181.dzomgh"}]: dispatch 2024-04-27T04:07:01.388 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:07:01 smithi162 ceph-mon[26841]: fsmap cephfs:1 {0=cephfs.smithi162.gzlyjj=up:active} 1 up:standby 2024-04-27T04:07:01.423 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:07:01 smithi181 ceph-mon[33774]: Deploying daemon mds.cephfs.smithi162.xcnapk on smithi162 2024-04-27T04:07:01.423 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:07:01 smithi181 ceph-mon[33774]: mds.? [v2:172.21.15.181:6824/658214993,v1:172.21.15.181:6825/658214993] up:boot 2024-04-27T04:07:01.423 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:07:01 smithi181 ceph-mon[33774]: from='client.? 172.21.15.162:0/2787560245' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "1"}]': finished 2024-04-27T04:07:01.423 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:07:01 smithi181 ceph-mon[33774]: fsmap cephfs:1 {0=cephfs.smithi162.gzlyjj=up:active} 1 up:standby 2024-04-27T04:07:01.423 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:07:01 smithi181 ceph-mon[33774]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi181.dzomgh"}]: dispatch 2024-04-27T04:07:01.423 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:07:01 smithi181 ceph-mon[33774]: fsmap cephfs:1 {0=cephfs.smithi162.gzlyjj=up:active} 1 up:standby 2024-04-27T04:07:01.830 INFO:teuthology.orchestra.run.smithi162.stderr:Inferring config /var/lib/ceph/7f9750d6-044a-11ef-bc93-c7b262605968/mon.smithi162/config 2024-04-27T04:07:02.423 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:07:02 smithi181 ceph-mon[33774]: pgmap v175: 65 pgs: 65 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 0 B/s wr, 1 op/s 2024-04-27T04:07:02.423 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:07:02 smithi181 ceph-mon[33774]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-27T04:07:02.638 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:07:02 smithi162 ceph-mon[26841]: pgmap v175: 65 pgs: 65 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 0 B/s wr, 1 op/s 2024-04-27T04:07:02.638 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:07:02 smithi162 ceph-mon[26841]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-27T04:07:03.510 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:07:03.504+0000 7fa2beb80700 1 -- 172.21.15.162:0/1838004973 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa2b80fff80 msgr2=0x7fa2b8102360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:07:03.511 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:07:03.504+0000 7fa2beb80700 1 --2- 172.21.15.162:0/1838004973 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa2b80fff80 0x7fa2b8102360 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fa2a4004940 tx=0x7fa2a4009e30 comp rx=0 tx=0).stop 2024-04-27T04:07:03.511 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:07:03.504+0000 7fa2beb80700 1 -- 172.21.15.162:0/1838004973 shutdown_connections 2024-04-27T04:07:03.511 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:07:03.504+0000 7fa2beb80700 1 --2- 172.21.15.162:0/1838004973 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa2b80fff80 0x7fa2b8102360 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:07:03.511 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:07:03.504+0000 7fa2beb80700 1 --2- 172.21.15.162:0/1838004973 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa2b8066220 0x7fa2b80ffa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:07:03.511 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:07:03.504+0000 7fa2beb80700 1 -- 172.21.15.162:0/1838004973 >> 172.21.15.162:0/1838004973 conn(0x7fa2b80f7430 msgr2=0x7fa2b80f9840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:07:03.511 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:07:03.504+0000 7fa2beb80700 1 -- 172.21.15.162:0/1838004973 shutdown_connections 2024-04-27T04:07:03.511 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:07:03.504+0000 7fa2beb80700 1 -- 172.21.15.162:0/1838004973 wait complete. 2024-04-27T04:07:03.511 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:07:03.505+0000 7fa2beb80700 1 Processor -- start 2024-04-27T04:07:03.511 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:07:03.505+0000 7fa2beb80700 1 -- start start 2024-04-27T04:07:03.511 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:07:03.505+0000 7fa2beb80700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa2b8066220 0x7fa2b810d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:07:03.511 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:07:03.505+0000 7fa2beb80700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa2b80fff80 0x7fa2b810ded0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:07:03.511 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:07:03.505+0000 7fa2beb80700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2b810e4d0 con 0x7fa2b8066220 2024-04-27T04:07:03.511 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:07:03.505+0000 7fa2beb80700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2b810ed20 con 0x7fa2b80fff80 2024-04-27T04:07:03.512 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:07:03.505+0000 7fa2bc91c700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa2b8066220 0x7fa2b810d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:07:03.512 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:07:03.505+0000 7fa2bc91c700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa2b8066220 0x7fa2b810d990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:38698/0 (socket says 172.21.15.162:38698) 2024-04-27T04:07:03.512 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:07:03.505+0000 7fa2bc91c700 1 -- 172.21.15.162:0/614408847 learned_addr learned my addr 172.21.15.162:0/614408847 (peer_addr_for_me v2:172.21.15.162:0/0) 2024-04-27T04:07:03.512 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:07:03.505+0000 7fa2b7fff700 1 --2- 172.21.15.162:0/614408847 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa2b80fff80 0x7fa2b810ded0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:07:03.512 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:07:03.506+0000 7fa2bc91c700 1 -- 172.21.15.162:0/614408847 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa2b80fff80 msgr2=0x7fa2b810ded0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:07:03.512 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:07:03.506+0000 7fa2bc91c700 1 --2- 172.21.15.162:0/614408847 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa2b80fff80 0x7fa2b810ded0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:07:03.512 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:07:03.506+0000 7fa2bc91c700 1 -- 172.21.15.162:0/614408847 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2a4005040 con 0x7fa2b8066220 2024-04-27T04:07:03.512 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:07:03.506+0000 7fa2bc91c700 1 --2- 172.21.15.162:0/614408847 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa2b8066220 0x7fa2b810d990 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fa2ac002a80 tx=0x7fa2ac0046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:07:03.512 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:07:03.506+0000 7fa2b57fa700 1 -- 172.21.15.162:0/614408847 <== mon.0 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa2ac007cc0 con 0x7fa2b8066220 2024-04-27T04:07:03.512 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:07:03.506+0000 7fa2beb80700 1 -- 172.21.15.162:0/614408847 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa2b810f000 con 0x7fa2b8066220 2024-04-27T04:07:03.512 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:07:03.506+0000 7fa2b57fa700 1 -- 172.21.15.162:0/614408847 <== mon.0 v2:172.21.15.162:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fa2ac00f040 con 0x7fa2b8066220 2024-04-27T04:07:03.512 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:07:03.506+0000 7fa2b57fa700 1 -- 172.21.15.162:0/614408847 <== mon.0 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa2ac00db30 con 0x7fa2b8066220 2024-04-27T04:07:03.512 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:07:03.506+0000 7fa2beb80700 1 -- 172.21.15.162:0/614408847 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa2b810f540 con 0x7fa2b8066220 2024-04-27T04:07:03.512 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:07:03.507+0000 7fa2beb80700 1 -- 172.21.15.162:0/614408847 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa2b805f5c0 con 0x7fa2b8066220 2024-04-27T04:07:03.513 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:07:03.507+0000 7fa2b57fa700 1 -- 172.21.15.162:0/614408847 <== mon.0 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 19) v1 ==== 86631+0+0 (secure 0 0 0) 0x7fa2ac00dc90 con 0x7fa2b8066220 2024-04-27T04:07:03.513 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:07:03.508+0000 7fa2b57fa700 1 --2- 172.21.15.162:0/614408847 >> [v2:172.21.15.162:6800/4079782599,v1:172.21.15.162:6801/4079782599] conn(0x7fa2a80692a0 0x7fa2a806b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:07:03.513 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:07:03.508+0000 7fa2b57fa700 1 -- 172.21.15.162:0/614408847 <== mon.0 v2:172.21.15.162:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5464+0+0 (secure 0 0 0) 0x7fa2ac0869c0 con 0x7fa2b8066220 2024-04-27T04:07:03.513 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:07:03.508+0000 7fa2b7fff700 1 --2- 172.21.15.162:0/614408847 >> [v2:172.21.15.162:6800/4079782599,v1:172.21.15.162:6801/4079782599] conn(0x7fa2a80692a0 0x7fa2a806b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:07:03.513 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:07:03.508+0000 7fa2b7fff700 1 --2- 172.21.15.162:0/614408847 >> [v2:172.21.15.162:6800/4079782599,v1:172.21.15.162:6801/4079782599] conn(0x7fa2a80692a0 0x7fa2a806b750 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fa2a400b970 tx=0x7fa2a4009e30 comp rx=0 tx=0).ready entity=mgr.14210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:07:03.513 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:07:03.512+0000 7fa2b57fa700 1 -- 172.21.15.162:0/614408847 <== mon.0 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7fa2ac04d250 con 0x7fa2b8066220 2024-04-27T04:07:03.761 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:07:03.761+0000 7fa2beb80700 1 -- 172.21.15.162:0/614408847 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"} v 0) v1 -- 0x7fa2b804bed0 con 0x7fa2b8066220 2024-04-27T04:07:04.146 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:07:04.145+0000 7fa2b57fa700 1 -- 172.21.15.162:0/614408847 <== mon.0 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]=0 v8) v1 ==== 121+0+0 (secure 0 0 0) 0x7fa2ac0566a0 con 0x7fa2b8066220 2024-04-27T04:07:04.148 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:07:04.148+0000 7fa2beb80700 1 -- 172.21.15.162:0/614408847 >> [v2:172.21.15.162:6800/4079782599,v1:172.21.15.162:6801/4079782599] conn(0x7fa2a80692a0 msgr2=0x7fa2a806b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:07:04.148 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:07:04.148+0000 7fa2beb80700 1 --2- 172.21.15.162:0/614408847 >> [v2:172.21.15.162:6800/4079782599,v1:172.21.15.162:6801/4079782599] conn(0x7fa2a80692a0 0x7fa2a806b750 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fa2a400b970 tx=0x7fa2a4009e30 comp rx=0 tx=0).stop 2024-04-27T04:07:04.148 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:07:04.148+0000 7fa2beb80700 1 -- 172.21.15.162:0/614408847 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa2b8066220 msgr2=0x7fa2b810d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:07:04.149 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:07:04.148+0000 7fa2beb80700 1 --2- 172.21.15.162:0/614408847 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa2b8066220 0x7fa2b810d990 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fa2ac002a80 tx=0x7fa2ac0046b0 comp rx=0 tx=0).stop 2024-04-27T04:07:04.149 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:07:04.148+0000 7fa2beb80700 1 -- 172.21.15.162:0/614408847 shutdown_connections 2024-04-27T04:07:04.149 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:07:04.148+0000 7fa2beb80700 1 --2- 172.21.15.162:0/614408847 >> [v2:172.21.15.162:6800/4079782599,v1:172.21.15.162:6801/4079782599] conn(0x7fa2a80692a0 0x7fa2a806b750 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:07:04.149 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:07:04.148+0000 7fa2beb80700 1 --2- 172.21.15.162:0/614408847 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa2b8066220 0x7fa2b810d990 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:07:04.149 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:07:04.148+0000 7fa2beb80700 1 --2- 172.21.15.162:0/614408847 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa2b80fff80 0x7fa2b810ded0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:07:04.149 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:07:04.148+0000 7fa2beb80700 1 -- 172.21.15.162:0/614408847 >> 172.21.15.162:0/614408847 conn(0x7fa2b80f7430 msgr2=0x7fa2b80f7800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:07:04.149 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:07:04.148+0000 7fa2beb80700 1 -- 172.21.15.162:0/614408847 shutdown_connections 2024-04-27T04:07:04.149 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:07:04.148+0000 7fa2beb80700 1 -- 172.21.15.162:0/614408847 wait complete. 2024-04-27T04:07:04.387 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:07:04 smithi162 ceph-mon[26841]: pgmap v176: 65 pgs: 65 active+clean; 579 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.5 KiB/s wr, 5 op/s 2024-04-27T04:07:04.388 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:07:04 smithi162 ceph-mon[26841]: from='client.? 172.21.15.162:0/614408847' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-04-27T04:07:04.388 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:07:04 smithi162 ceph-mon[26841]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' 2024-04-27T04:07:04.388 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:07:04 smithi162 ceph-mon[26841]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' 2024-04-27T04:07:04.388 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:07:04 smithi162 ceph-mon[26841]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' 2024-04-27T04:07:04.388 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:07:04 smithi162 ceph-mon[26841]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi181.bkkcdm", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-27T04:07:04.388 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:07:04 smithi162 ceph-mon[26841]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' cmd='[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi181.bkkcdm", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-27T04:07:04.388 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:07:04 smithi162 ceph-mon[26841]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T04:07:04.423 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:07:04 smithi181 ceph-mon[33774]: pgmap v176: 65 pgs: 65 active+clean; 579 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.5 KiB/s wr, 5 op/s 2024-04-27T04:07:04.423 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:07:04 smithi181 ceph-mon[33774]: from='client.? 172.21.15.162:0/614408847' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-04-27T04:07:04.423 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:07:04 smithi181 ceph-mon[33774]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' 2024-04-27T04:07:04.423 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:07:04 smithi181 ceph-mon[33774]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' 2024-04-27T04:07:04.423 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:07:04 smithi181 ceph-mon[33774]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' 2024-04-27T04:07:04.423 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:07:04 smithi181 ceph-mon[33774]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi181.bkkcdm", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-27T04:07:04.423 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:07:04 smithi181 ceph-mon[33774]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' cmd='[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi181.bkkcdm", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-27T04:07:04.423 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:07:04 smithi181 ceph-mon[33774]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T04:07:04.801 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-04-27T04:07:04.811 INFO:tasks.cephadm:Running commands on role host.a host ubuntu@smithi162.front.sepia.ceph.com 2024-04-27T04:07:04.811 DEBUG:teuthology.orchestra.run.smithi162:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 7f9750d6-044a-11ef-bc93-c7b262605968 -- bash -c 'ceph fs set cephfs inline_data false' 2024-04-27T04:07:05.066 INFO:teuthology.orchestra.run.smithi162.stderr:Inferring config /var/lib/ceph/7f9750d6-044a-11ef-bc93-c7b262605968/mon.smithi162/config 2024-04-27T04:07:05.412 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:07:05 smithi162 ceph-mon[26841]: Deploying daemon mds.cephfs.smithi181.bkkcdm on smithi181 2024-04-27T04:07:05.412 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:07:05 smithi162 ceph-mon[26841]: from='client.? 172.21.15.162:0/614408847' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-04-27T04:07:05.412 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:07:05 smithi162 ceph-mon[26841]: mds.? [v2:172.21.15.162:6828/1296160676,v1:172.21.15.162:6829/1296160676] up:boot 2024-04-27T04:07:05.412 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:07:05 smithi162 ceph-mon[26841]: fsmap cephfs:1 {0=cephfs.smithi162.gzlyjj=up:active} 2 up:standby 2024-04-27T04:07:05.412 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:07:05 smithi162 ceph-mon[26841]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi162.xcnapk"}]: dispatch 2024-04-27T04:07:05.412 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:07:05 smithi162 ceph-mon[26841]: fsmap cephfs:1 {0=cephfs.smithi162.gzlyjj=up:active} 1 up:standby-replay 1 up:standby 2024-04-27T04:07:05.422 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:07:05 smithi181 ceph-mon[33774]: Deploying daemon mds.cephfs.smithi181.bkkcdm on smithi181 2024-04-27T04:07:05.423 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:07:05 smithi181 ceph-mon[33774]: from='client.? 172.21.15.162:0/614408847' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-04-27T04:07:05.423 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:07:05 smithi181 ceph-mon[33774]: mds.? [v2:172.21.15.162:6828/1296160676,v1:172.21.15.162:6829/1296160676] up:boot 2024-04-27T04:07:05.423 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:07:05 smithi181 ceph-mon[33774]: fsmap cephfs:1 {0=cephfs.smithi162.gzlyjj=up:active} 2 up:standby 2024-04-27T04:07:05.423 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:07:05 smithi181 ceph-mon[33774]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi162.xcnapk"}]: dispatch 2024-04-27T04:07:05.423 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:07:05 smithi181 ceph-mon[33774]: fsmap cephfs:1 {0=cephfs.smithi162.gzlyjj=up:active} 1 up:standby-replay 1 up:standby 2024-04-27T04:07:06.173 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:07:06 smithi181 ceph-mon[33774]: pgmap v177: 65 pgs: 65 active+clean; 579 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.3 KiB/s wr, 4 op/s 2024-04-27T04:07:06.450 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:07:06 smithi162 ceph-mon[26841]: pgmap v177: 65 pgs: 65 active+clean; 579 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.3 KiB/s wr, 4 op/s 2024-04-27T04:07:06.649 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:07:06.648+0000 7f21950ae700 1 -- 172.21.15.162:0/3066165910 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f21900fbcc0 msgr2=0x7f21900fc0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:07:06.649 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:07:06.648+0000 7f21950ae700 1 --2- 172.21.15.162:0/3066165910 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f21900fbcc0 0x7f21900fc0d0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f2180004940 tx=0x7f2180009e30 comp rx=0 tx=0).stop 2024-04-27T04:07:06.649 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:07:06.648+0000 7f21950ae700 1 -- 172.21.15.162:0/3066165910 shutdown_connections 2024-04-27T04:07:06.649 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:07:06.648+0000 7f21950ae700 1 --2- 172.21.15.162:0/3066165910 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f21900fcf70 0x7f21900fd3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:07:06.649 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:07:06.648+0000 7f21950ae700 1 --2- 172.21.15.162:0/3066165910 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f21900fbcc0 0x7f21900fc0d0 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:07:06.649 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:07:06.648+0000 7f21950ae700 1 -- 172.21.15.162:0/3066165910 >> 172.21.15.162:0/3066165910 conn(0x7f21900f7440 msgr2=0x7f21900f9850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:07:06.649 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:07:06.648+0000 7f21950ae700 1 -- 172.21.15.162:0/3066165910 shutdown_connections 2024-04-27T04:07:06.649 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:07:06.648+0000 7f21950ae700 1 -- 172.21.15.162:0/3066165910 wait complete. 2024-04-27T04:07:06.651 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:07:06.649+0000 7f21950ae700 1 Processor -- start 2024-04-27T04:07:06.651 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:07:06.649+0000 7f21950ae700 1 -- start start 2024-04-27T04:07:06.651 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:07:06.649+0000 7f21950ae700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f21900fbcc0 0x7f219010d8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:07:06.651 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:07:06.649+0000 7f21950ae700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f21900fcf70 0x7f219010de30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:07:06.651 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:07:06.649+0000 7f21950ae700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f219010e430 con 0x7f21900fbcc0 2024-04-27T04:07:06.651 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:07:06.649+0000 7f21950ae700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f219010e5a0 con 0x7f21900fcf70 2024-04-27T04:07:06.651 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:07:06.649+0000 7f218ed9d700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f21900fbcc0 0x7f219010d8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:07:06.651 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:07:06.650+0000 7f218e59c700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f21900fcf70 0x7f219010de30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:07:06.651 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:07:06.650+0000 7f218ed9d700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f21900fbcc0 0x7f219010d8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:38724/0 (socket says 172.21.15.162:38724) 2024-04-27T04:07:06.651 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:07:06.650+0000 7f218ed9d700 1 -- 172.21.15.162:0/37191657 learned_addr learned my addr 172.21.15.162:0/37191657 (peer_addr_for_me v2:172.21.15.162:0/0) 2024-04-27T04:07:06.651 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:07:06.650+0000 7f218e59c700 1 -- 172.21.15.162:0/37191657 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f21900fbcc0 msgr2=0x7f219010d8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:07:06.652 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:07:06.650+0000 7f218e59c700 1 --2- 172.21.15.162:0/37191657 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f21900fbcc0 0x7f219010d8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:07:06.652 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:07:06.650+0000 7f218e59c700 1 -- 172.21.15.162:0/37191657 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2180005040 con 0x7f21900fcf70 2024-04-27T04:07:06.652 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:07:06.650+0000 7f218ed9d700 1 --2- 172.21.15.162:0/37191657 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f21900fbcc0 0x7f219010d8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-27T04:07:06.652 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:07:06.650+0000 7f218e59c700 1 --2- 172.21.15.162:0/37191657 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f21900fcf70 0x7f219010de30 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f2184009fd0 tx=0x7f218400b040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:07:06.652 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:07:06.650+0000 7f21777fe700 1 -- 172.21.15.162:0/37191657 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2184004e90 con 0x7f21900fcf70 2024-04-27T04:07:06.652 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:07:06.650+0000 7f21950ae700 1 -- 172.21.15.162:0/37191657 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2190112fa0 con 0x7f21900fcf70 2024-04-27T04:07:06.652 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:07:06.650+0000 7f21777fe700 1 -- 172.21.15.162:0/37191657 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f218400f040 con 0x7f21900fcf70 2024-04-27T04:07:06.652 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:07:06.650+0000 7f21777fe700 1 -- 172.21.15.162:0/37191657 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f218400ca50 con 0x7f21900fcf70 2024-04-27T04:07:06.652 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:07:06.650+0000 7f21950ae700 1 -- 172.21.15.162:0/37191657 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f21901134b0 con 0x7f21900fcf70 2024-04-27T04:07:06.653 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:07:06.651+0000 7f21950ae700 1 -- 172.21.15.162:0/37191657 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2190101b00 con 0x7f21900fcf70 2024-04-27T04:07:06.653 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:07:06.652+0000 7f21777fe700 1 -- 172.21.15.162:0/37191657 <== mon.1 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 19) v1 ==== 86631+0+0 (secure 0 0 0) 0x7f2184007cc0 con 0x7f21900fcf70 2024-04-27T04:07:06.653 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:07:06.652+0000 7f21777fe700 1 --2- 172.21.15.162:0/37191657 >> [v2:172.21.15.162:6800/4079782599,v1:172.21.15.162:6801/4079782599] conn(0x7f21780692a0 0x7f217806b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:07:06.653 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:07:06.652+0000 7f218ed9d700 1 --2- 172.21.15.162:0/37191657 >> [v2:172.21.15.162:6800/4079782599,v1:172.21.15.162:6801/4079782599] conn(0x7f21780692a0 0x7f217806b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:07:06.653 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:07:06.652+0000 7f21777fe700 1 -- 172.21.15.162:0/37191657 <== mon.1 v2:172.21.15.181:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5464+0+0 (secure 0 0 0) 0x7f2184086c80 con 0x7f21900fcf70 2024-04-27T04:07:06.653 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:07:06.653+0000 7f218ed9d700 1 --2- 172.21.15.162:0/37191657 >> [v2:172.21.15.162:6800/4079782599,v1:172.21.15.162:6801/4079782599] conn(0x7f21780692a0 0x7f217806b750 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f218000b0d0 tx=0x7f2180009e30 comp rx=0 tx=0).ready entity=mgr.14210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:07:06.656 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:07:06.657+0000 7f21777fe700 1 -- 172.21.15.162:0/37191657 <== mon.1 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f218404e250 con 0x7f21900fcf70 2024-04-27T04:07:06.916 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:07:06.916+0000 7f21950ae700 1 -- 172.21.15.162:0/37191657 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "inline_data", "val": "false"} v 0) v1 -- 0x7f219004c710 con 0x7f21900fcf70 2024-04-27T04:07:07.158 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:07:07.159+0000 7f21777fe700 1 -- 172.21.15.162:0/37191657 <== mon.1 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "inline_data", "val": "false"}]=0 inline data disabled v10) v1 ==== 133+0+0 (secure 0 0 0) 0x7f21840569d0 con 0x7f21900fcf70 2024-04-27T04:07:07.162 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:07:07.162+0000 7f21950ae700 1 -- 172.21.15.162:0/37191657 >> [v2:172.21.15.162:6800/4079782599,v1:172.21.15.162:6801/4079782599] conn(0x7f21780692a0 msgr2=0x7f217806b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:07:07.162 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:07:07.162+0000 7f21950ae700 1 --2- 172.21.15.162:0/37191657 >> [v2:172.21.15.162:6800/4079782599,v1:172.21.15.162:6801/4079782599] conn(0x7f21780692a0 0x7f217806b750 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f218000b0d0 tx=0x7f2180009e30 comp rx=0 tx=0).stop 2024-04-27T04:07:07.162 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:07:07.162+0000 7f21950ae700 1 -- 172.21.15.162:0/37191657 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f21900fcf70 msgr2=0x7f219010de30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:07:07.162 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:07:07.162+0000 7f21950ae700 1 --2- 172.21.15.162:0/37191657 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f21900fcf70 0x7f219010de30 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f2184009fd0 tx=0x7f218400b040 comp rx=0 tx=0).stop 2024-04-27T04:07:07.162 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:07:07.162+0000 7f21950ae700 1 -- 172.21.15.162:0/37191657 shutdown_connections 2024-04-27T04:07:07.162 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:07:07.162+0000 7f21950ae700 1 --2- 172.21.15.162:0/37191657 >> [v2:172.21.15.162:6800/4079782599,v1:172.21.15.162:6801/4079782599] conn(0x7f21780692a0 0x7f217806b750 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:07:07.162 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:07:07.162+0000 7f21950ae700 1 --2- 172.21.15.162:0/37191657 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f21900fbcc0 0x7f219010d8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:07:07.162 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:07:07.162+0000 7f21950ae700 1 --2- 172.21.15.162:0/37191657 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f21900fcf70 0x7f219010de30 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:07:07.162 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:07:07.162+0000 7f21950ae700 1 -- 172.21.15.162:0/37191657 >> 172.21.15.162:0/37191657 conn(0x7f21900f7440 msgr2=0x7f2190066860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:07:07.162 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:07:07.162+0000 7f21950ae700 1 -- 172.21.15.162:0/37191657 shutdown_connections 2024-04-27T04:07:07.162 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:07:07.162+0000 7f21950ae700 1 -- 172.21.15.162:0/37191657 wait complete. 2024-04-27T04:07:07.163 INFO:teuthology.orchestra.run.smithi162.stderr:inline data disabled 2024-04-27T04:07:07.871 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:07:07 smithi181 ceph-mon[33774]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' 2024-04-27T04:07:07.871 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:07:07 smithi181 ceph-mon[33774]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' 2024-04-27T04:07:07.871 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:07:07 smithi181 ceph-mon[33774]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' 2024-04-27T04:07:07.871 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:07:07 smithi181 ceph-mon[33774]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' 2024-04-27T04:07:07.871 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:07:07 smithi181 ceph-mon[33774]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' 2024-04-27T04:07:07.872 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:07:07 smithi181 ceph-mon[33774]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T04:07:07.872 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:07:07 smithi181 ceph-mon[33774]: pgmap v178: 65 pgs: 65 active+clean; 579 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 176 B/s rd, 1.1 KiB/s wr, 3 op/s 2024-04-27T04:07:07.872 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:07:07 smithi181 ceph-mon[33774]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' 2024-04-27T04:07:07.872 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:07:07 smithi181 ceph-mon[33774]: from='client.? 172.21.15.162:0/37191657' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "inline_data", "val": "false"}]: dispatch 2024-04-27T04:07:07.872 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:07:07 smithi181 ceph-mon[33774]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "inline_data", "val": "false"}]: dispatch 2024-04-27T04:07:07.872 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:07:07 smithi181 ceph-mon[33774]: mds.? [v2:172.21.15.181:6826/3756475453,v1:172.21.15.181:6827/3756475453] up:boot 2024-04-27T04:07:07.872 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:07:07 smithi181 ceph-mon[33774]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "inline_data", "val": "false"}]': finished 2024-04-27T04:07:07.872 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:07:07 smithi181 ceph-mon[33774]: fsmap cephfs:1 {0=cephfs.smithi162.gzlyjj=up:active} 1 up:standby-replay 2 up:standby 2024-04-27T04:07:07.872 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:07:07 smithi181 ceph-mon[33774]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi181.bkkcdm"}]: dispatch 2024-04-27T04:07:07.887 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:07:07 smithi162 ceph-mon[26841]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' 2024-04-27T04:07:07.887 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:07:07 smithi162 ceph-mon[26841]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' 2024-04-27T04:07:07.887 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:07:07 smithi162 ceph-mon[26841]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' 2024-04-27T04:07:07.888 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:07:07 smithi162 ceph-mon[26841]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' 2024-04-27T04:07:07.888 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:07:07 smithi162 ceph-mon[26841]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' 2024-04-27T04:07:07.888 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:07:07 smithi162 ceph-mon[26841]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T04:07:07.888 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:07:07 smithi162 ceph-mon[26841]: pgmap v178: 65 pgs: 65 active+clean; 579 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 176 B/s rd, 1.1 KiB/s wr, 3 op/s 2024-04-27T04:07:07.888 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:07:07 smithi162 ceph-mon[26841]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' 2024-04-27T04:07:07.888 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:07:07 smithi162 ceph-mon[26841]: from='client.? 172.21.15.162:0/37191657' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "inline_data", "val": "false"}]: dispatch 2024-04-27T04:07:07.888 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:07:07 smithi162 ceph-mon[26841]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "inline_data", "val": "false"}]: dispatch 2024-04-27T04:07:07.888 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:07:07 smithi162 ceph-mon[26841]: mds.? [v2:172.21.15.181:6826/3756475453,v1:172.21.15.181:6827/3756475453] up:boot 2024-04-27T04:07:07.888 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:07:07 smithi162 ceph-mon[26841]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "inline_data", "val": "false"}]': finished 2024-04-27T04:07:07.888 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:07:07 smithi162 ceph-mon[26841]: fsmap cephfs:1 {0=cephfs.smithi162.gzlyjj=up:active} 1 up:standby-replay 2 up:standby 2024-04-27T04:07:07.888 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:07:07 smithi162 ceph-mon[26841]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi181.bkkcdm"}]: dispatch 2024-04-27T04:07:08.311 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-04-27T04:07:08.322 INFO:tasks.cephadm:Running commands on role host.a host ubuntu@smithi162.front.sepia.ceph.com 2024-04-27T04:07:08.322 DEBUG:teuthology.orchestra.run.smithi162:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 7f9750d6-044a-11ef-bc93-c7b262605968 -- bash -c 'ceph fs dump' 2024-04-27T04:07:09.226 INFO:teuthology.orchestra.run.smithi162.stderr:Inferring config /var/lib/ceph/7f9750d6-044a-11ef-bc93-c7b262605968/mon.smithi162/config 2024-04-27T04:07:09.887 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:07:09 smithi162 ceph-mon[26841]: mds.? [v2:172.21.15.181:6824/658214993,v1:172.21.15.181:6825/658214993] up:standby-replay 2024-04-27T04:07:09.888 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:07:09 smithi162 ceph-mon[26841]: mds.? [v2:172.21.15.162:6828/1296160676,v1:172.21.15.162:6829/1296160676] up:standby 2024-04-27T04:07:09.888 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:07:09 smithi162 ceph-mon[26841]: Dropping low affinity active daemon mds.cephfs.smithi162.gzlyjj in favor of higher affinity standby. 2024-04-27T04:07:09.888 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:07:09 smithi162 ceph-mon[26841]: Replacing daemon mds.cephfs.smithi162.gzlyjj as rank 0 with standby daemon mds.cephfs.smithi181.dzomgh 2024-04-27T04:07:09.888 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:07:09 smithi162 ceph-mon[26841]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-04-27T04:07:09.888 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:07:09 smithi162 ceph-mon[26841]: fsmap cephfs:1 {0=cephfs.smithi162.gzlyjj=up:active} 1 up:standby-replay 2 up:standby 2024-04-27T04:07:09.888 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:07:09 smithi162 ceph-mon[26841]: osdmap e46: 6 total, 6 up, 6 in 2024-04-27T04:07:09.888 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:07:09 smithi162 ceph-mon[26841]: fsmap cephfs:1/1 {0=cephfs.smithi181.dzomgh=up:replay} 2 up:standby 2024-04-27T04:07:09.888 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:07:09 smithi162 ceph-mon[26841]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' 2024-04-27T04:07:09.888 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:07:09 smithi162 ceph-mon[26841]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' 2024-04-27T04:07:09.888 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:07:09 smithi162 ceph-mon[26841]: pgmap v180: 65 pgs: 65 active+clean; 579 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.1 KiB/s rd, 1.3 KiB/s wr, 5 op/s 2024-04-27T04:07:09.923 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:07:09 smithi181 ceph-mon[33774]: mds.? [v2:172.21.15.181:6824/658214993,v1:172.21.15.181:6825/658214993] up:standby-replay 2024-04-27T04:07:09.923 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:07:09 smithi181 ceph-mon[33774]: mds.? [v2:172.21.15.162:6828/1296160676,v1:172.21.15.162:6829/1296160676] up:standby 2024-04-27T04:07:09.923 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:07:09 smithi181 ceph-mon[33774]: Dropping low affinity active daemon mds.cephfs.smithi162.gzlyjj in favor of higher affinity standby. 2024-04-27T04:07:09.923 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:07:09 smithi181 ceph-mon[33774]: Replacing daemon mds.cephfs.smithi162.gzlyjj as rank 0 with standby daemon mds.cephfs.smithi181.dzomgh 2024-04-27T04:07:09.923 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:07:09 smithi181 ceph-mon[33774]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-04-27T04:07:09.923 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:07:09 smithi181 ceph-mon[33774]: fsmap cephfs:1 {0=cephfs.smithi162.gzlyjj=up:active} 1 up:standby-replay 2 up:standby 2024-04-27T04:07:09.923 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:07:09 smithi181 ceph-mon[33774]: osdmap e46: 6 total, 6 up, 6 in 2024-04-27T04:07:09.923 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:07:09 smithi181 ceph-mon[33774]: fsmap cephfs:1/1 {0=cephfs.smithi181.dzomgh=up:replay} 2 up:standby 2024-04-27T04:07:09.924 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:07:09 smithi181 ceph-mon[33774]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' 2024-04-27T04:07:09.924 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:07:09 smithi181 ceph-mon[33774]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' 2024-04-27T04:07:09.924 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:07:09 smithi181 ceph-mon[33774]: pgmap v180: 65 pgs: 65 active+clean; 579 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.1 KiB/s rd, 1.3 KiB/s wr, 5 op/s 2024-04-27T04:07:10.690 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:07:10 smithi162 ceph-mon[26841]: mds.? [v2:172.21.15.162:6826/2629103894,v1:172.21.15.162:6827/2629103894] up:boot 2024-04-27T04:07:10.690 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:07:10 smithi162 ceph-mon[26841]: mds.? [v2:172.21.15.181:6824/658214993,v1:172.21.15.181:6825/658214993] up:reconnect 2024-04-27T04:07:10.690 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:07:10 smithi162 ceph-mon[26841]: fsmap cephfs:1/1 {0=cephfs.smithi181.dzomgh=up:reconnect} 3 up:standby 2024-04-27T04:07:10.691 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:07:10 smithi162 ceph-mon[26841]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi162.gzlyjj"}]: dispatch 2024-04-27T04:07:10.922 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:07:10 smithi181 ceph-mon[33774]: mds.? [v2:172.21.15.162:6826/2629103894,v1:172.21.15.162:6827/2629103894] up:boot 2024-04-27T04:07:10.923 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:07:10 smithi181 ceph-mon[33774]: mds.? [v2:172.21.15.181:6824/658214993,v1:172.21.15.181:6825/658214993] up:reconnect 2024-04-27T04:07:10.923 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:07:10 smithi181 ceph-mon[33774]: fsmap cephfs:1/1 {0=cephfs.smithi181.dzomgh=up:reconnect} 3 up:standby 2024-04-27T04:07:10.923 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:07:10 smithi181 ceph-mon[33774]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi162.gzlyjj"}]: dispatch 2024-04-27T04:07:11.419 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:07:11.418+0000 7f02939d5700 1 -- 172.21.15.162:0/324326517 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f028c0fce60 msgr2=0x7f028c0fd270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:07:11.419 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:07:11.418+0000 7f02939d5700 1 --2- 172.21.15.162:0/324326517 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f028c0fce60 0x7f028c0fd270 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f0288004940 tx=0x7f0288009e30 comp rx=0 tx=0).stop 2024-04-27T04:07:11.419 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:07:11.419+0000 7f02939d5700 1 -- 172.21.15.162:0/324326517 shutdown_connections 2024-04-27T04:07:11.420 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:07:11.419+0000 7f02939d5700 1 --2- 172.21.15.162:0/324326517 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f028c0fdfd0 0x7f028c0fe440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:07:11.420 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:07:11.419+0000 7f02939d5700 1 --2- 172.21.15.162:0/324326517 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f028c0fce60 0x7f028c0fd270 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:07:11.420 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:07:11.419+0000 7f02939d5700 1 -- 172.21.15.162:0/324326517 >> 172.21.15.162:0/324326517 conn(0x7f028c0f8520 msgr2=0x7f028c0fa950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:07:11.420 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:07:11.419+0000 7f02939d5700 1 -- 172.21.15.162:0/324326517 shutdown_connections 2024-04-27T04:07:11.420 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:07:11.419+0000 7f02939d5700 1 -- 172.21.15.162:0/324326517 wait complete. 2024-04-27T04:07:11.420 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:07:11.419+0000 7f02939d5700 1 Processor -- start 2024-04-27T04:07:11.420 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:07:11.419+0000 7f02939d5700 1 -- start start 2024-04-27T04:07:11.420 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:07:11.419+0000 7f02939d5700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f028c0fce60 0x7f028c10d960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:07:11.420 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:07:11.419+0000 7f02939d5700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f028c0fdfd0 0x7f028c10dea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:07:11.420 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:07:11.419+0000 7f02939d5700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f028c10e530 con 0x7f028c0fce60 2024-04-27T04:07:11.420 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:07:11.419+0000 7f02939d5700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f028c112ed0 con 0x7f028c0fdfd0 2024-04-27T04:07:11.420 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:07:11.419+0000 7f0291771700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f028c0fce60 0x7f028c10d960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:07:11.420 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:07:11.419+0000 7f0291771700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f028c0fce60 0x7f028c10d960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:38748/0 (socket says 172.21.15.162:38748) 2024-04-27T04:07:11.420 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:07:11.419+0000 7f0291771700 1 -- 172.21.15.162:0/4000691778 learned_addr learned my addr 172.21.15.162:0/4000691778 (peer_addr_for_me v2:172.21.15.162:0/0) 2024-04-27T04:07:11.421 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:07:11.420+0000 7f0290f70700 1 --2- 172.21.15.162:0/4000691778 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f028c0fdfd0 0x7f028c10dea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:07:11.421 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:07:11.420+0000 7f0291771700 1 -- 172.21.15.162:0/4000691778 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f028c0fdfd0 msgr2=0x7f028c10dea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:07:11.421 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:07:11.420+0000 7f0291771700 1 --2- 172.21.15.162:0/4000691778 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f028c0fdfd0 0x7f028c10dea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:07:11.421 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:07:11.420+0000 7f0291771700 1 -- 172.21.15.162:0/4000691778 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0288005040 con 0x7f028c0fce60 2024-04-27T04:07:11.421 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:07:11.420+0000 7f0291771700 1 --2- 172.21.15.162:0/4000691778 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f028c0fce60 0x7f028c10d960 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f028800b3b0 tx=0x7f0288009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:07:11.421 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:07:11.420+0000 7f0281ffb700 1 -- 172.21.15.162:0/4000691778 <== mon.0 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f028800e040 con 0x7f028c0fce60 2024-04-27T04:07:11.421 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:07:11.420+0000 7f0281ffb700 1 -- 172.21.15.162:0/4000691778 <== mon.0 v2:172.21.15.162:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f028800bcc0 con 0x7f028c0fce60 2024-04-27T04:07:11.421 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:07:11.420+0000 7f02939d5700 1 -- 172.21.15.162:0/4000691778 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f028c1130d0 con 0x7f028c0fce60 2024-04-27T04:07:11.421 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:07:11.420+0000 7f0281ffb700 1 -- 172.21.15.162:0/4000691778 <== mon.0 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f028801c630 con 0x7f028c0fce60 2024-04-27T04:07:11.421 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:07:11.420+0000 7f02939d5700 1 -- 172.21.15.162:0/4000691778 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f028c113640 con 0x7f028c0fce60 2024-04-27T04:07:11.422 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:07:11.421+0000 7f02939d5700 1 -- 172.21.15.162:0/4000691778 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f028c04c710 con 0x7f028c0fce60 2024-04-27T04:07:11.422 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:07:11.422+0000 7f0281ffb700 1 -- 172.21.15.162:0/4000691778 <== mon.0 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 19) v1 ==== 86631+0+0 (secure 0 0 0) 0x7f0288013420 con 0x7f028c0fce60 2024-04-27T04:07:11.422 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:07:11.422+0000 7f0281ffb700 1 --2- 172.21.15.162:0/4000691778 >> [v2:172.21.15.162:6800/4079782599,v1:172.21.15.162:6801/4079782599] conn(0x7f02780692a0 0x7f027806b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:07:11.422 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:07:11.422+0000 7f0281ffb700 1 -- 172.21.15.162:0/4000691778 <== mon.0 v2:172.21.15.162:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5550+0+0 (secure 0 0 0) 0x7f0288088030 con 0x7f028c0fce60 2024-04-27T04:07:11.422 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:07:11.422+0000 7f0290f70700 1 --2- 172.21.15.162:0/4000691778 >> [v2:172.21.15.162:6800/4079782599,v1:172.21.15.162:6801/4079782599] conn(0x7f02780692a0 0x7f027806b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:07:11.422 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:07:11.422+0000 7f0290f70700 1 --2- 172.21.15.162:0/4000691778 >> [v2:172.21.15.162:6800/4079782599,v1:172.21.15.162:6801/4079782599] conn(0x7f02780692a0 0x7f027806b750 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f027c006fd0 tx=0x7f027c008040 comp rx=0 tx=0).ready entity=mgr.14210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:07:11.425 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:07:11.426+0000 7f0281ffb700 1 -- 172.21.15.162:0/4000691778 <== mon.0 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f0288057d20 con 0x7f028c0fce60 2024-04-27T04:07:11.671 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:07:11.672+0000 7f02939d5700 1 -- 172.21.15.162:0/4000691778 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f028c113a80 con 0x7f028c0fce60 2024-04-27T04:07:11.671 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:07:11.672+0000 7f0281ffb700 1 -- 172.21.15.162:0/4000691778 <== mon.0 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 15 v15) v1 ==== 76+0+1841 (secure 0 0 0) 0x7f02880578b0 con 0x7f028c0fce60 2024-04-27T04:07:11.672 INFO:teuthology.orchestra.run.smithi162.stdout:e15 2024-04-27T04:07:11.672 INFO:teuthology.orchestra.run.smithi162.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-04-27T04:07:11.672 INFO:teuthology.orchestra.run.smithi162.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-27T04:07:11.672 INFO:teuthology.orchestra.run.smithi162.stdout:legacy client fscid: 1 2024-04-27T04:07:11.672 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-04-27T04:07:11.672 INFO:teuthology.orchestra.run.smithi162.stdout:Filesystem 'cephfs' (1) 2024-04-27T04:07:11.672 INFO:teuthology.orchestra.run.smithi162.stdout:fs_name cephfs 2024-04-27T04:07:11.672 INFO:teuthology.orchestra.run.smithi162.stdout:epoch 15 2024-04-27T04:07:11.672 INFO:teuthology.orchestra.run.smithi162.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-04-27T04:07:11.672 INFO:teuthology.orchestra.run.smithi162.stdout:created 2024-04-27T04:06:53.095749+0000 2024-04-27T04:07:11.672 INFO:teuthology.orchestra.run.smithi162.stdout:modified 2024-04-27T04:07:11.285033+0000 2024-04-27T04:07:11.673 INFO:teuthology.orchestra.run.smithi162.stdout:tableserver 0 2024-04-27T04:07:11.673 INFO:teuthology.orchestra.run.smithi162.stdout:root 0 2024-04-27T04:07:11.673 INFO:teuthology.orchestra.run.smithi162.stdout:session_timeout 60 2024-04-27T04:07:11.673 INFO:teuthology.orchestra.run.smithi162.stdout:session_autoclose 300 2024-04-27T04:07:11.673 INFO:teuthology.orchestra.run.smithi162.stdout:max_file_size 1099511627776 2024-04-27T04:07:11.673 INFO:teuthology.orchestra.run.smithi162.stdout:required_client_features {} 2024-04-27T04:07:11.673 INFO:teuthology.orchestra.run.smithi162.stdout:last_failure 0 2024-04-27T04:07:11.673 INFO:teuthology.orchestra.run.smithi162.stdout:last_failure_osd_epoch 46 2024-04-27T04:07:11.673 INFO:teuthology.orchestra.run.smithi162.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-27T04:07:11.673 INFO:teuthology.orchestra.run.smithi162.stdout:max_mds 1 2024-04-27T04:07:11.673 INFO:teuthology.orchestra.run.smithi162.stdout:in 0 2024-04-27T04:07:11.673 INFO:teuthology.orchestra.run.smithi162.stdout:up {0=24273} 2024-04-27T04:07:11.673 INFO:teuthology.orchestra.run.smithi162.stdout:failed 2024-04-27T04:07:11.673 INFO:teuthology.orchestra.run.smithi162.stdout:damaged 2024-04-27T04:07:11.674 INFO:teuthology.orchestra.run.smithi162.stdout:stopped 2024-04-27T04:07:11.674 INFO:teuthology.orchestra.run.smithi162.stdout:data_pools [3] 2024-04-27T04:07:11.674 INFO:teuthology.orchestra.run.smithi162.stdout:metadata_pool 2 2024-04-27T04:07:11.674 INFO:teuthology.orchestra.run.smithi162.stdout:inline_data disabled 2024-04-27T04:07:11.674 INFO:teuthology.orchestra.run.smithi162.stdout:balancer 2024-04-27T04:07:11.674 INFO:teuthology.orchestra.run.smithi162.stdout:standby_count_wanted 1 2024-04-27T04:07:11.674 INFO:teuthology.orchestra.run.smithi162.stdout:[mds.cephfs.smithi181.dzomgh{0:24273} state up:active seq 6 join_fscid=1 addr [v2:172.21.15.181:6824/658214993,v1:172.21.15.181:6825/658214993] compat {c=[1],r=[1],i=[7ff]}] 2024-04-27T04:07:11.674 INFO:teuthology.orchestra.run.smithi162.stdout:[mds.cephfs.smithi162.xcnapk{0:14476} state up:standby-replay seq 2 join_fscid=1 addr [v2:172.21.15.162:6828/1296160676,v1:172.21.15.162:6829/1296160676] compat {c=[1],r=[1],i=[7ff]}] 2024-04-27T04:07:11.674 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-04-27T04:07:11.675 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-04-27T04:07:11.675 INFO:teuthology.orchestra.run.smithi162.stdout:Standby daemons: 2024-04-27T04:07:11.675 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-04-27T04:07:11.675 INFO:teuthology.orchestra.run.smithi162.stdout:[mds.cephfs.smithi162.gzlyjj{-1:14488} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.162:6826/2629103894,v1:172.21.15.162:6827/2629103894] compat {c=[1],r=[1],i=[7ff]}] 2024-04-27T04:07:11.675 INFO:teuthology.orchestra.run.smithi162.stdout:[mds.cephfs.smithi181.bkkcdm{-1:24287} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.181:6826/3756475453,v1:172.21.15.181:6827/3756475453] compat {c=[1],r=[1],i=[7ff]}] 2024-04-27T04:07:11.676 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:07:11.675+0000 7f02939d5700 1 -- 172.21.15.162:0/4000691778 >> [v2:172.21.15.162:6800/4079782599,v1:172.21.15.162:6801/4079782599] conn(0x7f02780692a0 msgr2=0x7f027806b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:07:11.676 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:07:11.675+0000 7f02939d5700 1 --2- 172.21.15.162:0/4000691778 >> [v2:172.21.15.162:6800/4079782599,v1:172.21.15.162:6801/4079782599] conn(0x7f02780692a0 0x7f027806b750 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f027c006fd0 tx=0x7f027c008040 comp rx=0 tx=0).stop 2024-04-27T04:07:11.676 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:07:11.675+0000 7f02939d5700 1 -- 172.21.15.162:0/4000691778 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f028c0fce60 msgr2=0x7f028c10d960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:07:11.676 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:07:11.675+0000 7f02939d5700 1 --2- 172.21.15.162:0/4000691778 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f028c0fce60 0x7f028c10d960 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f028800b3b0 tx=0x7f0288009e30 comp rx=0 tx=0).stop 2024-04-27T04:07:11.676 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:07:11.675+0000 7f02939d5700 1 -- 172.21.15.162:0/4000691778 shutdown_connections 2024-04-27T04:07:11.677 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:07:11.675+0000 7f02939d5700 1 --2- 172.21.15.162:0/4000691778 >> [v2:172.21.15.162:6800/4079782599,v1:172.21.15.162:6801/4079782599] conn(0x7f02780692a0 0x7f027806b750 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:07:11.677 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:07:11.675+0000 7f02939d5700 1 --2- 172.21.15.162:0/4000691778 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f028c0fce60 0x7f028c10d960 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:07:11.677 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:07:11.675+0000 7f02939d5700 1 --2- 172.21.15.162:0/4000691778 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f028c0fdfd0 0x7f028c10dea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:07:11.677 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:07:11.675+0000 7f02939d5700 1 -- 172.21.15.162:0/4000691778 >> 172.21.15.162:0/4000691778 conn(0x7f028c0f8520 msgr2=0x7f028c0f9be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:07:11.677 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:07:11.675+0000 7f02939d5700 1 -- 172.21.15.162:0/4000691778 shutdown_connections 2024-04-27T04:07:11.678 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:07:11.675+0000 7f02939d5700 1 -- 172.21.15.162:0/4000691778 wait complete. 2024-04-27T04:07:11.678 INFO:teuthology.orchestra.run.smithi162.stderr:dumped fsmap epoch 15 2024-04-27T04:07:11.887 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:07:11 smithi162 ceph-mon[26841]: mds.? [v2:172.21.15.181:6824/658214993,v1:172.21.15.181:6825/658214993] up:rejoin 2024-04-27T04:07:11.888 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:07:11 smithi162 ceph-mon[26841]: mds.? [v2:172.21.15.181:6826/3756475453,v1:172.21.15.181:6827/3756475453] up:standby 2024-04-27T04:07:11.888 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:07:11 smithi162 ceph-mon[26841]: fsmap cephfs:1/1 {0=cephfs.smithi181.dzomgh=up:rejoin} 3 up:standby 2024-04-27T04:07:11.888 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:07:11 smithi162 ceph-mon[26841]: daemon mds.cephfs.smithi181.dzomgh is now active in filesystem cephfs as rank 0 2024-04-27T04:07:11.888 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:07:11 smithi162 ceph-mon[26841]: pgmap v181: 65 pgs: 65 active+clean; 579 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.1 KiB/s rd, 1.3 KiB/s wr, 4 op/s 2024-04-27T04:07:11.888 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:07:11 smithi162 ceph-mon[26841]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-04-27T04:07:11.888 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:07:11 smithi162 ceph-mon[26841]: Cluster is now healthy 2024-04-27T04:07:11.889 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:07:11 smithi162 ceph-mon[26841]: mds.? [v2:172.21.15.181:6824/658214993,v1:172.21.15.181:6825/658214993] up:active 2024-04-27T04:07:11.889 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:07:11 smithi162 ceph-mon[26841]: fsmap cephfs:1 {0=cephfs.smithi181.dzomgh=up:active} 1 up:standby-replay 2 up:standby 2024-04-27T04:07:11.923 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:07:11 smithi181 ceph-mon[33774]: mds.? [v2:172.21.15.181:6824/658214993,v1:172.21.15.181:6825/658214993] up:rejoin 2024-04-27T04:07:11.923 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:07:11 smithi181 ceph-mon[33774]: mds.? [v2:172.21.15.181:6826/3756475453,v1:172.21.15.181:6827/3756475453] up:standby 2024-04-27T04:07:11.923 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:07:11 smithi181 ceph-mon[33774]: fsmap cephfs:1/1 {0=cephfs.smithi181.dzomgh=up:rejoin} 3 up:standby 2024-04-27T04:07:11.923 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:07:11 smithi181 ceph-mon[33774]: daemon mds.cephfs.smithi181.dzomgh is now active in filesystem cephfs as rank 0 2024-04-27T04:07:11.923 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:07:11 smithi181 ceph-mon[33774]: pgmap v181: 65 pgs: 65 active+clean; 579 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.1 KiB/s rd, 1.3 KiB/s wr, 4 op/s 2024-04-27T04:07:11.923 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:07:11 smithi181 ceph-mon[33774]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-04-27T04:07:11.923 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:07:11 smithi181 ceph-mon[33774]: Cluster is now healthy 2024-04-27T04:07:11.923 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:07:11 smithi181 ceph-mon[33774]: mds.? [v2:172.21.15.181:6824/658214993,v1:172.21.15.181:6825/658214993] up:active 2024-04-27T04:07:11.924 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:07:11 smithi181 ceph-mon[33774]: fsmap cephfs:1 {0=cephfs.smithi181.dzomgh=up:active} 1 up:standby-replay 2 up:standby 2024-04-27T04:07:12.692 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:07:12 smithi162 ceph-mon[26841]: from='client.? 172.21.15.162:0/4000691778' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-27T04:07:12.693 DEBUG:teuthology.orchestra.run.smithi162:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 7f9750d6-044a-11ef-bc93-c7b262605968 -- bash -c 'ceph --format=json fs dump | jq -e ".filesystems | length == 1"' 2024-04-27T04:07:12.923 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:07:12 smithi181 ceph-mon[33774]: from='client.? 172.21.15.162:0/4000691778' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-27T04:07:13.428 INFO:teuthology.orchestra.run.smithi162.stderr:Inferring config /var/lib/ceph/7f9750d6-044a-11ef-bc93-c7b262605968/mon.smithi162/config 2024-04-27T04:07:13.842 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:07:13 smithi162 ceph-mon[26841]: pgmap v182: 65 pgs: 65 active+clean; 579 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 4.0 KiB/s rd, 0 B/s wr, 4 op/s 2024-04-27T04:07:13.922 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:07:13 smithi181 ceph-mon[33774]: pgmap v182: 65 pgs: 65 active+clean; 579 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 4.0 KiB/s rd, 0 B/s wr, 4 op/s 2024-04-27T04:07:14.967 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:07:14.968+0000 7fd807d85700 1 -- 172.21.15.162:0/216891527 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd800066ab0 msgr2=0x7fd8000ffad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:07:14.967 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:07:14.968+0000 7fd807d85700 1 --2- 172.21.15.162:0/216891527 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd800066ab0 0x7fd8000ffad0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fd7fc004940 tx=0x7fd7fc009e30 comp rx=0 tx=0).stop 2024-04-27T04:07:14.967 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:07:14.968+0000 7fd807d85700 1 -- 172.21.15.162:0/216891527 shutdown_connections 2024-04-27T04:07:14.967 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:07:14.968+0000 7fd807d85700 1 --2- 172.21.15.162:0/216891527 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd800100010 0x7fd8001023f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:07:14.968 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:07:14.968+0000 7fd807d85700 1 --2- 172.21.15.162:0/216891527 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd800066ab0 0x7fd8000ffad0 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:07:14.968 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:07:14.968+0000 7fd807d85700 1 -- 172.21.15.162:0/216891527 >> 172.21.15.162:0/216891527 conn(0x7fd8000f74c0 msgr2=0x7fd8000f98d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:07:14.968 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:07:14.968+0000 7fd807d85700 1 -- 172.21.15.162:0/216891527 shutdown_connections 2024-04-27T04:07:14.968 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:07:14.968+0000 7fd807d85700 1 -- 172.21.15.162:0/216891527 wait complete. 2024-04-27T04:07:14.970 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:07:14.969+0000 7fd807d85700 1 Processor -- start 2024-04-27T04:07:14.970 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:07:14.969+0000 7fd807d85700 1 -- start start 2024-04-27T04:07:14.970 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:07:14.969+0000 7fd807d85700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd800066ab0 0x7fd80010da50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:07:14.970 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:07:14.969+0000 7fd807d85700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd800100010 0x7fd80010df90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:07:14.970 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:07:14.969+0000 7fd807d85700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd80010e590 con 0x7fd800100010 2024-04-27T04:07:14.970 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:07:14.969+0000 7fd807d85700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd80010ede0 con 0x7fd800066ab0 2024-04-27T04:07:14.970 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:07:14.969+0000 7fd805320700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd800100010 0x7fd80010df90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:07:14.970 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:07:14.969+0000 7fd805320700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd800100010 0x7fd80010df90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:38134/0 (socket says 172.21.15.162:38134) 2024-04-27T04:07:14.971 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:07:14.969+0000 7fd805320700 1 -- 172.21.15.162:0/3450095299 learned_addr learned my addr 172.21.15.162:0/3450095299 (peer_addr_for_me v2:172.21.15.162:0/0) 2024-04-27T04:07:14.971 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:07:14.969+0000 7fd805b21700 1 --2- 172.21.15.162:0/3450095299 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd800066ab0 0x7fd80010da50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:07:14.971 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:07:14.969+0000 7fd805320700 1 -- 172.21.15.162:0/3450095299 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd800066ab0 msgr2=0x7fd80010da50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:07:14.971 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:07:14.969+0000 7fd805320700 1 --2- 172.21.15.162:0/3450095299 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd800066ab0 0x7fd80010da50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:07:14.971 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:07:14.969+0000 7fd805320700 1 -- 172.21.15.162:0/3450095299 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd7fc005040 con 0x7fd800100010 2024-04-27T04:07:14.971 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:07:14.969+0000 7fd805b21700 1 --2- 172.21.15.162:0/3450095299 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd800066ab0 0x7fd80010da50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-27T04:07:14.971 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:07:14.969+0000 7fd805320700 1 --2- 172.21.15.162:0/3450095299 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd800100010 0x7fd80010df90 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fd7f000c830 tx=0x7fd7f0004470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:07:14.971 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:07:14.970+0000 7fd7f67fc700 1 -- 172.21.15.162:0/3450095299 <== mon.0 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd7f000a760 con 0x7fd800100010 2024-04-27T04:07:14.971 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:07:14.970+0000 7fd7f67fc700 1 -- 172.21.15.162:0/3450095299 <== mon.0 v2:172.21.15.162:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fd7f000ad90 con 0x7fd800100010 2024-04-27T04:07:14.971 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:07:14.970+0000 7fd807d85700 1 -- 172.21.15.162:0/3450095299 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd80010f0c0 con 0x7fd800100010 2024-04-27T04:07:14.971 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:07:14.970+0000 7fd7f67fc700 1 -- 172.21.15.162:0/3450095299 <== mon.0 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd7f00113a0 con 0x7fd800100010 2024-04-27T04:07:14.971 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:07:14.970+0000 7fd807d85700 1 -- 172.21.15.162:0/3450095299 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd80010f600 con 0x7fd800100010 2024-04-27T04:07:14.971 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:07:14.971+0000 7fd807d85700 1 -- 172.21.15.162:0/3450095299 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd80005f5c0 con 0x7fd800100010 2024-04-27T04:07:14.971 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:07:14.971+0000 7fd7f67fc700 1 -- 172.21.15.162:0/3450095299 <== mon.0 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 19) v1 ==== 86631+0+0 (secure 0 0 0) 0x7fd7f0012400 con 0x7fd800100010 2024-04-27T04:07:14.972 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:07:14.971+0000 7fd7f67fc700 1 --2- 172.21.15.162:0/3450095299 >> [v2:172.21.15.162:6800/4079782599,v1:172.21.15.162:6801/4079782599] conn(0x7fd7ec0692a0 0x7fd7ec06b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:07:14.972 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:07:14.972+0000 7fd7f67fc700 1 -- 172.21.15.162:0/3450095299 <== mon.0 v2:172.21.15.162:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5550+0+0 (secure 0 0 0) 0x7fd7f0087290 con 0x7fd800100010 2024-04-27T04:07:14.972 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:07:14.972+0000 7fd805b21700 1 --2- 172.21.15.162:0/3450095299 >> [v2:172.21.15.162:6800/4079782599,v1:172.21.15.162:6801/4079782599] conn(0x7fd7ec0692a0 0x7fd7ec06b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:07:14.972 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:07:14.972+0000 7fd805b21700 1 --2- 172.21.15.162:0/3450095299 >> [v2:172.21.15.162:6800/4079782599,v1:172.21.15.162:6801/4079782599] conn(0x7fd7ec0692a0 0x7fd7ec06b750 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fd7fc00b3b0 tx=0x7fd7fc009e30 comp rx=0 tx=0).ready entity=mgr.14210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:07:14.974 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:07:14.976+0000 7fd7f67fc700 1 -- 172.21.15.162:0/3450095299 <== mon.0 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7fd7f0057000 con 0x7fd800100010 2024-04-27T04:07:15.233 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:07:15.234+0000 7fd807d85700 1 -- 172.21.15.162:0/3450095299 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd8001058c0 con 0x7fd800100010 2024-04-27T04:07:15.234 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:07:15.235+0000 7fd7f67fc700 1 -- 172.21.15.162:0/3450095299 <== mon.0 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 15 v15) v1 ==== 94+0+4705 (secure 0 0 0) 0x7fd7f0056b90 con 0x7fd800100010 2024-04-27T04:07:15.237 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:07:15.238+0000 7fd807d85700 1 -- 172.21.15.162:0/3450095299 >> [v2:172.21.15.162:6800/4079782599,v1:172.21.15.162:6801/4079782599] conn(0x7fd7ec0692a0 msgr2=0x7fd7ec06b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:07:15.237 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:07:15.238+0000 7fd807d85700 1 --2- 172.21.15.162:0/3450095299 >> [v2:172.21.15.162:6800/4079782599,v1:172.21.15.162:6801/4079782599] conn(0x7fd7ec0692a0 0x7fd7ec06b750 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fd7fc00b3b0 tx=0x7fd7fc009e30 comp rx=0 tx=0).stop 2024-04-27T04:07:15.237 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:07:15.238+0000 7fd807d85700 1 -- 172.21.15.162:0/3450095299 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd800100010 msgr2=0x7fd80010df90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:07:15.237 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:07:15.238+0000 7fd807d85700 1 --2- 172.21.15.162:0/3450095299 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd800100010 0x7fd80010df90 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fd7f000c830 tx=0x7fd7f0004470 comp rx=0 tx=0).stop 2024-04-27T04:07:15.238 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:07:15.238+0000 7fd807d85700 1 -- 172.21.15.162:0/3450095299 shutdown_connections 2024-04-27T04:07:15.238 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:07:15.238+0000 7fd807d85700 1 --2- 172.21.15.162:0/3450095299 >> [v2:172.21.15.162:6800/4079782599,v1:172.21.15.162:6801/4079782599] conn(0x7fd7ec0692a0 0x7fd7ec06b750 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:07:15.238 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:07:15.238+0000 7fd807d85700 1 --2- 172.21.15.162:0/3450095299 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd800066ab0 0x7fd80010da50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:07:15.238 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:07:15.238+0000 7fd807d85700 1 --2- 172.21.15.162:0/3450095299 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd800100010 0x7fd80010df90 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:07:15.238 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:07:15.238+0000 7fd807d85700 1 -- 172.21.15.162:0/3450095299 >> 172.21.15.162:0/3450095299 conn(0x7fd8000f74c0 msgr2=0x7fd8000f7890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:07:15.238 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:07:15.238+0000 7fd807d85700 1 -- 172.21.15.162:0/3450095299 shutdown_connections 2024-04-27T04:07:15.238 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:07:15.238+0000 7fd807d85700 1 -- 172.21.15.162:0/3450095299 wait complete. 2024-04-27T04:07:15.238 INFO:teuthology.orchestra.run.smithi162.stderr:dumped fsmap epoch 15 2024-04-27T04:07:15.250 INFO:teuthology.orchestra.run.smithi162.stdout:true 2024-04-27T04:07:15.829 DEBUG:teuthology.orchestra.run.smithi162:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 7f9750d6-044a-11ef-bc93-c7b262605968 -- bash -c 'while ! ceph --format=json mds versions | jq -e ". | add == 4"; do sleep 1; done' 2024-04-27T04:07:16.138 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:07:15 smithi162 ceph-mon[26841]: pgmap v183: 65 pgs: 65 active+clean; 579 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 4.0 KiB/s rd, 0 B/s wr, 4 op/s 2024-04-27T04:07:16.138 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:07:15 smithi162 ceph-mon[26841]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' 2024-04-27T04:07:16.138 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:07:15 smithi162 ceph-mon[26841]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' 2024-04-27T04:07:16.138 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:07:15 smithi162 ceph-mon[26841]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T04:07:16.138 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:07:15 smithi162 ceph-mon[26841]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T04:07:16.138 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:07:15 smithi162 ceph-mon[26841]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' 2024-04-27T04:07:16.138 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:07:15 smithi162 ceph-mon[26841]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T04:07:16.138 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:07:15 smithi162 ceph-mon[26841]: from='client.? 172.21.15.162:0/3450095299' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-27T04:07:16.168 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:07:15 smithi181 ceph-mon[33774]: pgmap v183: 65 pgs: 65 active+clean; 579 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 4.0 KiB/s rd, 0 B/s wr, 4 op/s 2024-04-27T04:07:16.168 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:07:15 smithi181 ceph-mon[33774]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' 2024-04-27T04:07:16.168 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:07:15 smithi181 ceph-mon[33774]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' 2024-04-27T04:07:16.168 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:07:15 smithi181 ceph-mon[33774]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T04:07:16.168 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:07:15 smithi181 ceph-mon[33774]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T04:07:16.168 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:07:15 smithi181 ceph-mon[33774]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' 2024-04-27T04:07:16.168 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:07:15 smithi181 ceph-mon[33774]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T04:07:16.168 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:07:15 smithi181 ceph-mon[33774]: from='client.? 172.21.15.162:0/3450095299' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-27T04:07:16.756 INFO:teuthology.orchestra.run.smithi162.stderr:Inferring config /var/lib/ceph/7f9750d6-044a-11ef-bc93-c7b262605968/mon.smithi162/config 2024-04-27T04:07:17.137 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:07:16 smithi162 ceph-mon[26841]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-27T04:07:17.172 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:07:16 smithi181 ceph-mon[33774]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-27T04:07:18.173 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:07:17 smithi181 ceph-mon[33774]: pgmap v184: 65 pgs: 65 active+clean; 579 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 6.4 KiB/s rd, 0 B/s wr, 6 op/s 2024-04-27T04:07:18.173 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:07:17 smithi181 ceph-mon[33774]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' 2024-04-27T04:07:18.173 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:07:17 smithi181 ceph-mon[33774]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' 2024-04-27T04:07:18.329 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:07:17 smithi162 ceph-mon[26841]: pgmap v184: 65 pgs: 65 active+clean; 579 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 6.4 KiB/s rd, 0 B/s wr, 6 op/s 2024-04-27T04:07:18.329 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:07:17 smithi162 ceph-mon[26841]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' 2024-04-27T04:07:18.329 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:07:17 smithi162 ceph-mon[26841]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' 2024-04-27T04:07:18.829 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:07:18.828+0000 7fb504251700 1 -- 172.21.15.162:0/146676897 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb4fc0fbcb0 msgr2=0x7fb4fc0fc0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:07:18.829 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:07:18.828+0000 7fb504251700 1 --2- 172.21.15.162:0/146676897 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb4fc0fbcb0 0x7fb4fc0fc0c0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fb4f4009ce0 tx=0x7fb4f4004d10 comp rx=0 tx=0).stop 2024-04-27T04:07:18.829 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:07:18.828+0000 7fb504251700 1 -- 172.21.15.162:0/146676897 shutdown_connections 2024-04-27T04:07:18.829 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:07:18.828+0000 7fb504251700 1 --2- 172.21.15.162:0/146676897 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb4fc0fcf60 0x7fb4fc0fd3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:07:18.829 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:07:18.828+0000 7fb504251700 1 --2- 172.21.15.162:0/146676897 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb4fc0fbcb0 0x7fb4fc0fc0c0 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:07:18.829 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:07:18.828+0000 7fb504251700 1 -- 172.21.15.162:0/146676897 >> 172.21.15.162:0/146676897 conn(0x7fb4fc0f7430 msgr2=0x7fb4fc0f9840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:07:18.829 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:07:18.829+0000 7fb504251700 1 -- 172.21.15.162:0/146676897 shutdown_connections 2024-04-27T04:07:18.829 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:07:18.829+0000 7fb504251700 1 -- 172.21.15.162:0/146676897 wait complete. 2024-04-27T04:07:18.830 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:07:18.829+0000 7fb504251700 1 Processor -- start 2024-04-27T04:07:18.830 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:07:18.829+0000 7fb504251700 1 -- start start 2024-04-27T04:07:18.830 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:07:18.829+0000 7fb504251700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb4fc0fbcb0 0x7fb4fc10ca90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:07:18.830 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:07:18.829+0000 7fb504251700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb4fc0fcf60 0x7fb4fc10cfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:07:18.830 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:07:18.829+0000 7fb504251700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4fc10d660 con 0x7fb4fc0fbcb0 2024-04-27T04:07:18.830 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:07:18.829+0000 7fb504251700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4fc10de50 con 0x7fb4fc0fcf60 2024-04-27T04:07:18.830 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:07:18.829+0000 7fb501fed700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb4fc0fbcb0 0x7fb4fc10ca90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:07:18.830 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:07:18.829+0000 7fb501fed700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb4fc0fbcb0 0x7fb4fc10ca90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:38150/0 (socket says 172.21.15.162:38150) 2024-04-27T04:07:18.830 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:07:18.829+0000 7fb501fed700 1 -- 172.21.15.162:0/2562771595 learned_addr learned my addr 172.21.15.162:0/2562771595 (peer_addr_for_me v2:172.21.15.162:0/0) 2024-04-27T04:07:18.830 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:07:18.830+0000 7fb5017ec700 1 --2- 172.21.15.162:0/2562771595 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb4fc0fcf60 0x7fb4fc10cfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:07:18.830 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:07:18.830+0000 7fb501fed700 1 -- 172.21.15.162:0/2562771595 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb4fc0fcf60 msgr2=0x7fb4fc10cfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:07:18.830 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:07:18.830+0000 7fb501fed700 1 --2- 172.21.15.162:0/2562771595 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb4fc0fcf60 0x7fb4fc10cfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:07:18.831 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:07:18.830+0000 7fb501fed700 1 -- 172.21.15.162:0/2562771595 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb4f4009940 con 0x7fb4fc0fbcb0 2024-04-27T04:07:18.831 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:07:18.830+0000 7fb501fed700 1 --2- 172.21.15.162:0/2562771595 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb4fc0fbcb0 0x7fb4fc10ca90 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fb4f4004740 tx=0x7fb4f4004d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:07:18.831 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:07:18.830+0000 7fb4f27fc700 1 -- 172.21.15.162:0/2562771595 <== mon.0 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb4f400e040 con 0x7fb4fc0fbcb0 2024-04-27T04:07:18.831 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:07:18.830+0000 7fb4f27fc700 1 -- 172.21.15.162:0/2562771595 <== mon.0 v2:172.21.15.162:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fb4f40128e0 con 0x7fb4fc0fbcb0 2024-04-27T04:07:18.831 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:07:18.830+0000 7fb504251700 1 -- 172.21.15.162:0/2562771595 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb4fc10e0d0 con 0x7fb4fc0fbcb0 2024-04-27T04:07:18.831 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:07:18.830+0000 7fb4f27fc700 1 -- 172.21.15.162:0/2562771595 <== mon.0 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb4f401ca80 con 0x7fb4fc0fbcb0 2024-04-27T04:07:18.831 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:07:18.830+0000 7fb504251700 1 -- 172.21.15.162:0/2562771595 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb4fc10e690 con 0x7fb4fc0fbcb0 2024-04-27T04:07:18.832 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:07:18.831+0000 7fb504251700 1 -- 172.21.15.162:0/2562771595 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb4fc05f5c0 con 0x7fb4fc0fbcb0 2024-04-27T04:07:18.832 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:07:18.831+0000 7fb4f27fc700 1 -- 172.21.15.162:0/2562771595 <== mon.0 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 19) v1 ==== 86631+0+0 (secure 0 0 0) 0x7fb4f4012a50 con 0x7fb4fc0fbcb0 2024-04-27T04:07:18.832 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:07:18.832+0000 7fb4f27fc700 1 --2- 172.21.15.162:0/2562771595 >> [v2:172.21.15.162:6800/4079782599,v1:172.21.15.162:6801/4079782599] conn(0x7fb4e80692a0 0x7fb4e806b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:07:18.832 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:07:18.832+0000 7fb4f27fc700 1 -- 172.21.15.162:0/2562771595 <== mon.0 v2:172.21.15.162:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5550+0+0 (secure 0 0 0) 0x7fb4f4088ef0 con 0x7fb4fc0fbcb0 2024-04-27T04:07:18.832 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:07:18.832+0000 7fb5017ec700 1 --2- 172.21.15.162:0/2562771595 >> [v2:172.21.15.162:6800/4079782599,v1:172.21.15.162:6801/4079782599] conn(0x7fb4e80692a0 0x7fb4e806b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:07:18.832 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:07:18.832+0000 7fb5017ec700 1 --2- 172.21.15.162:0/2562771595 >> [v2:172.21.15.162:6800/4079782599,v1:172.21.15.162:6801/4079782599] conn(0x7fb4e80692a0 0x7fb4e806b750 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fb4ec006fd0 tx=0x7fb4ec008040 comp rx=0 tx=0).ready entity=mgr.14210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:07:18.835 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:07:18.836+0000 7fb4f27fc700 1 -- 172.21.15.162:0/2562771595 <== mon.0 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7fb4f4058be0 con 0x7fb4fc0fbcb0 2024-04-27T04:07:19.107 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:07:19.107+0000 7fb504251700 1 -- 172.21.15.162:0/2562771595 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "mds versions", "format": "json"} v 0) v1 -- 0x7fb4fc04bed0 con 0x7fb4fc0fbcb0 2024-04-27T04:07:19.107 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:07:19.108+0000 7fb4f27fc700 1 -- 172.21.15.162:0/2562771595 <== mon.0 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "mds versions", "format": "json"}]=0 v15) v1 ==== 78+0+99 (secure 0 0 0) 0x7fb4f4058770 con 0x7fb4fc0fbcb0 2024-04-27T04:07:19.109 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:07:19.110+0000 7fb504251700 1 -- 172.21.15.162:0/2562771595 >> [v2:172.21.15.162:6800/4079782599,v1:172.21.15.162:6801/4079782599] conn(0x7fb4e80692a0 msgr2=0x7fb4e806b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:07:19.110 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:07:19.110+0000 7fb504251700 1 --2- 172.21.15.162:0/2562771595 >> [v2:172.21.15.162:6800/4079782599,v1:172.21.15.162:6801/4079782599] conn(0x7fb4e80692a0 0x7fb4e806b750 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fb4ec006fd0 tx=0x7fb4ec008040 comp rx=0 tx=0).stop 2024-04-27T04:07:19.110 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:07:19.110+0000 7fb504251700 1 -- 172.21.15.162:0/2562771595 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb4fc0fbcb0 msgr2=0x7fb4fc10ca90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:07:19.110 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:07:19.110+0000 7fb504251700 1 --2- 172.21.15.162:0/2562771595 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb4fc0fbcb0 0x7fb4fc10ca90 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fb4f4004740 tx=0x7fb4f4004d10 comp rx=0 tx=0).stop 2024-04-27T04:07:19.110 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:07:19.110+0000 7fb504251700 1 -- 172.21.15.162:0/2562771595 shutdown_connections 2024-04-27T04:07:19.110 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:07:19.110+0000 7fb504251700 1 --2- 172.21.15.162:0/2562771595 >> [v2:172.21.15.162:6800/4079782599,v1:172.21.15.162:6801/4079782599] conn(0x7fb4e80692a0 0x7fb4e806b750 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:07:19.110 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:07:19.110+0000 7fb504251700 1 --2- 172.21.15.162:0/2562771595 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb4fc0fbcb0 0x7fb4fc10ca90 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:07:19.110 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:07:19.110+0000 7fb504251700 1 --2- 172.21.15.162:0/2562771595 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb4fc0fcf60 0x7fb4fc10cfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:07:19.110 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:07:19.110+0000 7fb504251700 1 -- 172.21.15.162:0/2562771595 >> 172.21.15.162:0/2562771595 conn(0x7fb4fc0f7430 msgr2=0x7fb4fc0f8a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:07:19.110 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:07:19.110+0000 7fb504251700 1 -- 172.21.15.162:0/2562771595 shutdown_connections 2024-04-27T04:07:19.110 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:07:19.110+0000 7fb504251700 1 -- 172.21.15.162:0/2562771595 wait complete. 2024-04-27T04:07:19.119 INFO:teuthology.orchestra.run.smithi162.stdout:true 2024-04-27T04:07:19.759 INFO:teuthology.run_tasks:Running task fs.pre_upgrade_save... 2024-04-27T04:07:19.777 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-27T04:07:19.888 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:07:19.887+0000 7f484a99d640 1 -- 172.21.15.162:0/3336670592 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4844066110 msgr2=0x7f48440fee30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:07:19.889 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:07:19.887+0000 7f484a99d640 1 --2- 172.21.15.162:0/3336670592 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4844066110 0x7f48440fee30 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f48340048d0 tx=0x7f4834009da0 comp rx=0 tx=0).stop 2024-04-27T04:07:19.889 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:07:19.887+0000 7f484a99d640 1 -- 172.21.15.162:0/3336670592 shutdown_connections 2024-04-27T04:07:19.889 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:07:19.887+0000 7f484a99d640 1 --2- 172.21.15.162:0/3336670592 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f48440ff370 0x7f4844101750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:07:19.889 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:07:19.887+0000 7f484a99d640 1 --2- 172.21.15.162:0/3336670592 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4844066110 0x7f48440fee30 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:07:19.889 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:07:19.887+0000 7f484a99d640 1 -- 172.21.15.162:0/3336670592 >> 172.21.15.162:0/3336670592 conn(0x7f48440f6840 msgr2=0x7f48440f8c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:07:19.889 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:07:19.887+0000 7f484a99d640 1 -- 172.21.15.162:0/3336670592 shutdown_connections 2024-04-27T04:07:19.889 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:07:19.887+0000 7f484a99d640 1 -- 172.21.15.162:0/3336670592 wait complete. 2024-04-27T04:07:19.889 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:07:19.888+0000 7f484a99d640 1 Processor -- start 2024-04-27T04:07:19.889 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:07:19.888+0000 7f484a99d640 1 -- start start 2024-04-27T04:07:19.889 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:07:19.888+0000 7f484a99d640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4844066110 0x7f484410ef30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:07:19.889 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:07:19.888+0000 7f484a99d640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f48440ff370 0x7f484410f470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:07:19.889 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:07:19.888+0000 7f484a99d640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f484410fac0 con 0x7f48440ff370 2024-04-27T04:07:19.889 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:07:19.888+0000 7f484a99d640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48441144c0 con 0x7f4844066110 2024-04-27T04:07:19.890 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:07:19.888+0000 7f48437fe640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f48440ff370 0x7f484410f470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:07:19.890 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:07:19.888+0000 7f48437fe640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f48440ff370 0x7f484410f470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:38162/0 (socket says 172.21.15.162:38162) 2024-04-27T04:07:19.890 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:07:19.888+0000 7f48437fe640 1 -- 172.21.15.162:0/2990956017 learned_addr learned my addr 172.21.15.162:0/2990956017 (peer_addr_for_me v2:172.21.15.162:0/0) 2024-04-27T04:07:19.890 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:07:19.888+0000 7f4843fff640 1 --2- 172.21.15.162:0/2990956017 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4844066110 0x7f484410ef30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:07:19.890 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:07:19.888+0000 7f48437fe640 1 -- 172.21.15.162:0/2990956017 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4844066110 msgr2=0x7f484410ef30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:07:19.890 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:07:19.888+0000 7f48437fe640 1 --2- 172.21.15.162:0/2990956017 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4844066110 0x7f484410ef30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:07:19.890 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:07:19.888+0000 7f48437fe640 1 -- 172.21.15.162:0/2990956017 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4834005040 con 0x7f48440ff370 2024-04-27T04:07:19.890 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:07:19.889+0000 7f48437fe640 1 --2- 172.21.15.162:0/2990956017 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f48440ff370 0x7f484410f470 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f4838002870 tx=0x7f4838004600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:07:19.890 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:07:19.889+0000 7f4840ff9640 1 -- 172.21.15.162:0/2990956017 <== mon.0 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4838007c30 con 0x7f48440ff370 2024-04-27T04:07:19.890 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:07:19.889+0000 7f4840ff9640 1 -- 172.21.15.162:0/2990956017 <== mon.0 v2:172.21.15.162:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f483800f040 con 0x7f48440ff370 2024-04-27T04:07:19.890 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:07:19.889+0000 7f484a99d640 1 -- 172.21.15.162:0/2990956017 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4844114660 con 0x7f48440ff370 2024-04-27T04:07:19.890 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:07:19.889+0000 7f4840ff9640 1 -- 172.21.15.162:0/2990956017 <== mon.0 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f48380095f0 con 0x7f48440ff370 2024-04-27T04:07:19.891 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:07:19.889+0000 7f484a99d640 1 -- 172.21.15.162:0/2990956017 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4844114b50 con 0x7f48440ff370 2024-04-27T04:07:19.891 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:07:19.890+0000 7f484a99d640 1 -- 172.21.15.162:0/2990956017 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f484404bcc0 con 0x7f48440ff370 2024-04-27T04:07:19.892 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:07:19.890+0000 7f4840ff9640 1 -- 172.21.15.162:0/2990956017 <== mon.0 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 19) v1 ==== 86631+0+0 (secure 0 0 0) 0x7f483800d6b0 con 0x7f48440ff370 2024-04-27T04:07:19.892 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:07:19.890+0000 7f4840ff9640 1 --2- 172.21.15.162:0/2990956017 >> [v2:172.21.15.162:6800/4079782599,v1:172.21.15.162:6801/4079782599] conn(0x7f482c069210 0x7f482c06b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:07:19.892 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:07:19.891+0000 7f4840ff9640 1 -- 172.21.15.162:0/2990956017 <== mon.0 v2:172.21.15.162:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5550+0+0 (secure 0 0 0) 0x7f48380869e0 con 0x7f48440ff370 2024-04-27T04:07:19.892 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:07:19.891+0000 7f4843fff640 1 --2- 172.21.15.162:0/2990956017 >> [v2:172.21.15.162:6800/4079782599,v1:172.21.15.162:6801/4079782599] conn(0x7f482c069210 0x7f482c06b6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:07:19.892 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:07:19.891+0000 7f4843fff640 1 --2- 172.21.15.162:0/2990956017 >> [v2:172.21.15.162:6800/4079782599,v1:172.21.15.162:6801/4079782599] conn(0x7f482c069210 0x7f482c06b6c0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f4834000c00 tx=0x7f4834009da0 comp rx=0 tx=0).ready entity=mgr.14210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:07:19.893 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:07:19.895+0000 7f4840ff9640 1 -- 172.21.15.162:0/2990956017 <== mon.0 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f4838052cf0 con 0x7f48440ff370 2024-04-27T04:07:20.063 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:07:20.064+0000 7f484a99d640 1 -- 172.21.15.162:0/2990956017 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f4844104d20 con 0x7f48440ff370 2024-04-27T04:07:20.064 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:07:20.065+0000 7f4840ff9640 1 -- 172.21.15.162:0/2990956017 <== mon.0 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v46) v1 ==== 74+0+14456 (secure 0 0 0) 0x7f4838056310 con 0x7f48440ff370 2024-04-27T04:07:20.064 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-04-27T04:07:20.064 INFO:teuthology.orchestra.run.smithi162.stdout:{"epoch":46,"fsid":"7f9750d6-044a-11ef-bc93-c7b262605968","created":"2024-04-27T04:00:06.343616+0000","modified":"2024-04-27T04:07:08.543594+0000","last_up_change":"2024-04-27T04:05:37.769065+0000","last_in_change":"2024-04-27T04:05:18.597695+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":14,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":6,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-27T04:04:24.971779+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"cephfs.cephfs.meta","create_time":"2024-04-27T04:06:52.008169+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"44","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":3,"pool_name":"cephfs.cephfs.data","create_time":"2024-04-27T04:06:52.322198+0000","flags":131073,"flags_names":"hashpspool,bulk","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"45","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"0ae69d7f-0ab4-4672-89b9-3563a10a1bf2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6802","nonce":3784722138},{"type":"v1","addr":"172.21.15.162:6803","nonce":3784722138}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6804","nonce":3784722138},{"type":"v1","addr":"172.21.15.162:6805","nonce":3784722138}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6808","nonce":3784722138},{"type":"v1","addr":"172.21.15.162:6809","nonce":3784722138}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6806","nonce":3784722138},{"type":"v1","addr":"172.21.15.162:6807","nonce":3784722138}]},"public_addr":"172.21.15.162:6803/3784722138","cluster_addr":"172.21.15.162:6805/3784722138","heartbeat_back_addr":"172.21.15.162:6809/3784722138","heartbeat_front_addr":"172.21.15.162:6807/3784722138","state":["exists","up"]},{"osd":1,"uuid":"95bdc9f0-c438-4a88-942d-dffb0c61c308","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6810","nonce":3642637124},{"type":"v1","addr":"172.21.15.162:6811","nonce":3642637124}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6812","nonce":3642637124},{"type":"v1","addr":"172.21.15.162:6813","nonce":3642637124}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6816","nonce":3642637124},{"type":"v1","addr":"172.21.15.162:6817","nonce":3642637124}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6814","nonce":3642637124},{"type":"v1","addr":"172.21.15.162:6815","nonce":3642637124}]},"public_addr":"172.21.15.162:6811/3642637124","cluster_addr":"172.21.15.162:6813/3642637124","heartbeat_back_addr":"172.21.15.162:6817/3642637124","heartbeat_front_addr":"172.21.15.162:6815/3642637124","state":["exists","up"]},{"osd":2,"uuid":"a773b386-e75b-432a-9d1b-2206301b96c3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":19,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6818","nonce":3830961923},{"type":"v1","addr":"172.21.15.162:6819","nonce":3830961923}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6820","nonce":3830961923},{"type":"v1","addr":"172.21.15.162:6821","nonce":3830961923}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6824","nonce":3830961923},{"type":"v1","addr":"172.21.15.162:6825","nonce":3830961923}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6822","nonce":3830961923},{"type":"v1","addr":"172.21.15.162:6823","nonce":3830961923}]},"public_addr":"172.21.15.162:6819/3830961923","cluster_addr":"172.21.15.162:6821/3830961923","heartbeat_back_addr":"172.21.15.162:6825/3830961923","heartbeat_front_addr":"172.21.15.162:6823/3830961923","state":["exists","up"]},{"osd":3,"uuid":"756091bc-1594-479d-9390-cb60e6b65ebe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":3695382003},{"type":"v1","addr":"172.21.15.181:6801","nonce":3695382003}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":3695382003},{"type":"v1","addr":"172.21.15.181:6803","nonce":3695382003}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6806","nonce":3695382003},{"type":"v1","addr":"172.21.15.181:6807","nonce":3695382003}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":3695382003},{"type":"v1","addr":"172.21.15.181:6805","nonce":3695382003}]},"public_addr":"172.21.15.181:6801/3695382003","cluster_addr":"172.21.15.181:6803/3695382003","heartbeat_back_addr":"172.21.15.181:6807/3695382003","heartbeat_front_addr":"172.21.15.181:6805/3695382003","state":["exists","up"]},{"osd":4,"uuid":"bd70494d-061c-4468-b837-26150c2d4208","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6808","nonce":2791715456},{"type":"v1","addr":"172.21.15.181:6809","nonce":2791715456}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":2791715456},{"type":"v1","addr":"172.21.15.181:6811","nonce":2791715456}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":2791715456},{"type":"v1","addr":"172.21.15.181:6815","nonce":2791715456}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6812","nonce":2791715456},{"type":"v1","addr":"172.21.15.181:6813","nonce":2791715456}]},"public_addr":"172.21.15.181:6809/2791715456","cluster_addr":"172.21.15.181:6811/2791715456","heartbeat_back_addr":"172.21.15.181:6815/2791715456","heartbeat_front_addr":"172.21.15.181:6813/2791715456","state":["exists","up"]},{"osd":5,"uuid":"15455ef6-2853-4a84-aaa9-1797e78302a9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":3193687515},{"type":"v1","addr":"172.21.15.181:6817","nonce":3193687515}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6818","nonce":3193687515},{"type":"v1","addr":"172.21.15.181:6819","nonce":3193687515}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6822","nonce":3193687515},{"type":"v1","addr":"172.21.15.181:6823","nonce":3193687515}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":3193687515},{"type":"v1","addr":"172.21.15.181:6821","nonce":3193687515}]},"public_addr":"172.21.15.181:6817/3193687515","cluster_addr":"172.21.15.181:6819/3193687515","heartbeat_back_addr":"172.21.15.181:6823/3193687515","heartbeat_front_addr":"172.21.15.181:6821/3193687515","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-04-27T04:03:28.522704+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-04-27T04:03:54.648083+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-04-27T04:04:22.107316+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-04-27T04:04:46.020733+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-04-27T04:05:10.634082+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-04-27T04:05:35.756217+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.162:0/2579239891":"2024-04-28T04:00:40.642043+0000","172.21.15.162:0/4025365522":"2024-04-28T04:00:40.642043+0000","172.21.15.162:0/1703886136":"2024-04-28T04:02:16.661212+0000","172.21.15.162:0/2129785584":"2024-04-28T04:01:10.996874+0000","172.21.15.162:6801/1480340393":"2024-04-28T04:00:40.642043+0000","172.21.15.162:6800/1482851081":"2024-04-28T04:02:16.661212+0000","172.21.15.162:6800/8067198":"2024-04-28T04:01:10.996874+0000","172.21.15.162:0/2623011446":"2024-04-28T04:00:40.642043+0000","172.21.15.162:6801/8067198":"2024-04-28T04:01:10.996874+0000","172.21.15.162:6801/1482851081":"2024-04-28T04:02:16.661212+0000","172.21.15.162:0/3073351439":"2024-04-28T04:01:10.996874+0000","172.21.15.162:0/3590827229":"2024-04-28T04:01:10.996874+0000","172.21.15.162:6827/998468496":"2024-04-28T04:07:08.543563+0000","172.21.15.162:0/2336042321":"2024-04-28T04:02:16.661212+0000","172.21.15.162:0/795779642":"2024-04-28T04:02:16.661212+0000","172.21.15.162:6800/1480340393":"2024-04-28T04:00:40.642043+0000","172.21.15.162:6826/998468496":"2024-04-28T04:07:08.543563+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-27T04:07:20.066 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:07:20.067+0000 7f484a99d640 1 -- 172.21.15.162:0/2990956017 >> [v2:172.21.15.162:6800/4079782599,v1:172.21.15.162:6801/4079782599] conn(0x7f482c069210 msgr2=0x7f482c06b6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:07:20.066 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:07:20.067+0000 7f484a99d640 1 --2- 172.21.15.162:0/2990956017 >> [v2:172.21.15.162:6800/4079782599,v1:172.21.15.162:6801/4079782599] conn(0x7f482c069210 0x7f482c06b6c0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f4834000c00 tx=0x7f4834009da0 comp rx=0 tx=0).stop 2024-04-27T04:07:20.067 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:07:20.067+0000 7f484a99d640 1 -- 172.21.15.162:0/2990956017 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f48440ff370 msgr2=0x7f484410f470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:07:20.067 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:07:20.067+0000 7f484a99d640 1 --2- 172.21.15.162:0/2990956017 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f48440ff370 0x7f484410f470 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f4838002870 tx=0x7f4838004600 comp rx=0 tx=0).stop 2024-04-27T04:07:20.067 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:07:20.067+0000 7f484a99d640 1 -- 172.21.15.162:0/2990956017 shutdown_connections 2024-04-27T04:07:20.067 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:07:20.067+0000 7f484a99d640 1 --2- 172.21.15.162:0/2990956017 >> [v2:172.21.15.162:6800/4079782599,v1:172.21.15.162:6801/4079782599] conn(0x7f482c069210 0x7f482c06b6c0 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:07:20.067 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:07:20.067+0000 7f484a99d640 1 --2- 172.21.15.162:0/2990956017 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f48440ff370 0x7f484410f470 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:07:20.067 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:07:20.067+0000 7f484a99d640 1 --2- 172.21.15.162:0/2990956017 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4844066110 0x7f484410ef30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:07:20.067 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:07:20.067+0000 7f484a99d640 1 -- 172.21.15.162:0/2990956017 >> 172.21.15.162:0/2990956017 conn(0x7f48440f6840 msgr2=0x7f48440fae70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:07:20.067 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:07:20.067+0000 7f484a99d640 1 -- 172.21.15.162:0/2990956017 shutdown_connections 2024-04-27T04:07:20.067 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:07:20.067+0000 7f484a99d640 1 -- 172.21.15.162:0/2990956017 wait complete. 2024-04-27T04:07:20.078 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-27T04:04:24.971779+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}, {'pool': 2, 'pool_name': 'cephfs.cephfs.meta', 'create_time': '2024-04-27T04:06:52.008169+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '44', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 3, 'pool_name': 'cephfs.cephfs.data', 'create_time': '2024-04-27T04:06:52.322198+0000', 'flags': 131073, 'flags_names': 'hashpspool,bulk', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '45', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-27T04:07:20.078 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-27T04:07:20.172 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:07:19 smithi181 ceph-mon[33774]: pgmap v185: 65 pgs: 65 active+clean; 579 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 5.9 KiB/s rd, 0 B/s wr, 6 op/s 2024-04-27T04:07:20.173 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:07:19 smithi181 ceph-mon[33774]: from='client.? 172.21.15.162:0/2562771595' entity='client.admin' cmd=[{"prefix": "mds versions", "format": "json"}]: dispatch 2024-04-27T04:07:20.185 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:07:20.185+0000 7f4aa819d640 1 -- 172.21.15.162:0/75107986 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4aa00fd3b0 msgr2=0x7f4aa00fd820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:07:20.185 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:07:20.185+0000 7f4aa819d640 1 --2- 172.21.15.162:0/75107986 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4aa00fd3b0 0x7f4aa00fd820 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f4a900048d0 tx=0x7f4a90009da0 comp rx=0 tx=0).stop 2024-04-27T04:07:20.186 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:07:20.185+0000 7f4aa819d640 1 -- 172.21.15.162:0/75107986 shutdown_connections 2024-04-27T04:07:20.186 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:07:20.185+0000 7f4aa819d640 1 --2- 172.21.15.162:0/75107986 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4aa00fd3b0 0x7f4aa00fd820 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:07:20.186 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:07:20.185+0000 7f4aa819d640 1 --2- 172.21.15.162:0/75107986 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4aa00fc180 0x7f4aa00fc590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:07:20.186 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:07:20.185+0000 7f4aa819d640 1 -- 172.21.15.162:0/75107986 >> 172.21.15.162:0/75107986 conn(0x7f4aa00f78c0 msgr2=0x7f4aa00f9cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:07:20.186 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:07:20.185+0000 7f4aa819d640 1 -- 172.21.15.162:0/75107986 shutdown_connections 2024-04-27T04:07:20.186 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:07:20.185+0000 7f4aa819d640 1 -- 172.21.15.162:0/75107986 wait complete. 2024-04-27T04:07:20.186 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:07:20.185+0000 7f4aa819d640 1 Processor -- start 2024-04-27T04:07:20.186 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:07:20.185+0000 7f4aa819d640 1 -- start start 2024-04-27T04:07:20.186 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:07:20.185+0000 7f4aa819d640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4aa00fc180 0x7f4aa010cca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:07:20.186 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:07:20.185+0000 7f4aa819d640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4aa00fd3b0 0x7f4aa010d1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:07:20.186 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:07:20.185+0000 7f4aa819d640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4aa010d860 con 0x7f4aa00fc180 2024-04-27T04:07:20.186 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:07:20.185+0000 7f4aa819d640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4aa0112220 con 0x7f4aa00fd3b0 2024-04-27T04:07:20.186 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:07:20.186+0000 7f4aa5f39640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4aa00fc180 0x7f4aa010cca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:07:20.187 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:07:20.186+0000 7f4aa5f39640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4aa00fc180 0x7f4aa010cca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:38182/0 (socket says 172.21.15.162:38182) 2024-04-27T04:07:20.187 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:07:20.186+0000 7f4aa5f39640 1 -- 172.21.15.162:0/869509307 learned_addr learned my addr 172.21.15.162:0/869509307 (peer_addr_for_me v2:172.21.15.162:0/0) 2024-04-27T04:07:20.187 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:07:20.186+0000 7f4aa5738640 1 --2- 172.21.15.162:0/869509307 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4aa00fd3b0 0x7f4aa010d1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:07:20.187 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:07:20.186+0000 7f4aa5f39640 1 -- 172.21.15.162:0/869509307 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4aa00fd3b0 msgr2=0x7f4aa010d1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:07:20.187 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:07:20.186+0000 7f4aa5f39640 1 --2- 172.21.15.162:0/869509307 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4aa00fd3b0 0x7f4aa010d1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:07:20.187 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:07:20.186+0000 7f4aa5f39640 1 -- 172.21.15.162:0/869509307 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a90005040 con 0x7f4aa00fc180 2024-04-27T04:07:20.187 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:07:20.186+0000 7f4aa5f39640 1 --2- 172.21.15.162:0/869509307 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4aa00fc180 0x7f4aa010cca0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f4a9c00eba0 tx=0x7f4a9c00b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:07:20.187 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:07:20.186+0000 7f4a967fc640 1 -- 172.21.15.162:0/869509307 <== mon.0 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4a9c007c30 con 0x7f4aa00fc180 2024-04-27T04:07:20.187 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:07:20.186+0000 7f4a967fc640 1 -- 172.21.15.162:0/869509307 <== mon.0 v2:172.21.15.162:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f4a9c00f040 con 0x7f4aa00fc180 2024-04-27T04:07:20.187 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:07:20.186+0000 7f4aa819d640 1 -- 172.21.15.162:0/869509307 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4aa01123c0 con 0x7f4aa00fc180 2024-04-27T04:07:20.187 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:07:20.186+0000 7f4a967fc640 1 -- 172.21.15.162:0/869509307 <== mon.0 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4a9c009680 con 0x7f4aa00fc180 2024-04-27T04:07:20.187 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:07:20.186+0000 7f4aa819d640 1 -- 172.21.15.162:0/869509307 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4aa01128b0 con 0x7f4aa00fc180 2024-04-27T04:07:20.188 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:07:20.187+0000 7f4aa819d640 1 -- 172.21.15.162:0/869509307 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4aa004bcc0 con 0x7f4aa00fc180 2024-04-27T04:07:20.189 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:07:20.188+0000 7f4a967fc640 1 -- 172.21.15.162:0/869509307 <== mon.0 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 19) v1 ==== 86631+0+0 (secure 0 0 0) 0x7f4a9c00c710 con 0x7f4aa00fc180 2024-04-27T04:07:20.189 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:07:20.188+0000 7f4a967fc640 1 --2- 172.21.15.162:0/869509307 >> [v2:172.21.15.162:6800/4079782599,v1:172.21.15.162:6801/4079782599] conn(0x7f4a8c069210 0x7f4a8c06b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:07:20.189 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:07:20.188+0000 7f4a967fc640 1 -- 172.21.15.162:0/869509307 <== mon.0 v2:172.21.15.162:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5550+0+0 (secure 0 0 0) 0x7f4a9c0873c0 con 0x7f4aa00fc180 2024-04-27T04:07:20.189 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:07:20.189+0000 7f4aa5738640 1 --2- 172.21.15.162:0/869509307 >> [v2:172.21.15.162:6800/4079782599,v1:172.21.15.162:6801/4079782599] conn(0x7f4a8c069210 0x7f4a8c06b6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:07:20.189 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:07:20.189+0000 7f4aa5738640 1 --2- 172.21.15.162:0/869509307 >> [v2:172.21.15.162:6800/4079782599,v1:172.21.15.162:6801/4079782599] conn(0x7f4a8c069210 0x7f4a8c06b6c0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f4a900317e0 tx=0x7f4a90009da0 comp rx=0 tx=0).ready entity=mgr.14210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:07:20.192 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:07:20.193+0000 7f4a967fc640 1 -- 172.21.15.162:0/869509307 <== mon.0 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f4a9c0536a0 con 0x7f4aa00fc180 2024-04-27T04:07:20.358 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:07:20.359+0000 7f4aa819d640 1 -- 172.21.15.162:0/869509307 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f4aa0112b50 con 0x7f4aa00fc180 2024-04-27T04:07:20.358 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:07:20.359+0000 7f4a967fc640 1 -- 172.21.15.162:0/869509307 <== mon.0 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v46) v1 ==== 93+0+10 (secure 0 0 0) 0x7f4a9c056cc0 con 0x7f4aa00fc180 2024-04-27T04:07:20.358 INFO:teuthology.orchestra.run.smithi162.stdout:pg_num: 1 2024-04-27T04:07:20.361 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:07:20.361+0000 7f4aa819d640 1 -- 172.21.15.162:0/869509307 >> [v2:172.21.15.162:6800/4079782599,v1:172.21.15.162:6801/4079782599] conn(0x7f4a8c069210 msgr2=0x7f4a8c06b6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:07:20.361 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:07:20.361+0000 7f4aa819d640 1 --2- 172.21.15.162:0/869509307 >> [v2:172.21.15.162:6800/4079782599,v1:172.21.15.162:6801/4079782599] conn(0x7f4a8c069210 0x7f4a8c06b6c0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f4a900317e0 tx=0x7f4a90009da0 comp rx=0 tx=0).stop 2024-04-27T04:07:20.361 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:07:20.361+0000 7f4aa819d640 1 -- 172.21.15.162:0/869509307 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4aa00fc180 msgr2=0x7f4aa010cca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:07:20.361 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:07:20.361+0000 7f4aa819d640 1 --2- 172.21.15.162:0/869509307 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4aa00fc180 0x7f4aa010cca0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f4a9c00eba0 tx=0x7f4a9c00b040 comp rx=0 tx=0).stop 2024-04-27T04:07:20.361 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:07:20.361+0000 7f4aa819d640 1 -- 172.21.15.162:0/869509307 shutdown_connections 2024-04-27T04:07:20.361 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:07:20.361+0000 7f4aa819d640 1 --2- 172.21.15.162:0/869509307 >> [v2:172.21.15.162:6800/4079782599,v1:172.21.15.162:6801/4079782599] conn(0x7f4a8c069210 0x7f4a8c06b6c0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:07:20.361 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:07:20.361+0000 7f4aa819d640 1 --2- 172.21.15.162:0/869509307 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4aa00fd3b0 0x7f4aa010d1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:07:20.361 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:07:20.361+0000 7f4aa819d640 1 --2- 172.21.15.162:0/869509307 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4aa00fc180 0x7f4aa010cca0 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:07:20.361 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:07:20.361+0000 7f4aa819d640 1 -- 172.21.15.162:0/869509307 >> 172.21.15.162:0/869509307 conn(0x7f4aa00f78c0 msgr2=0x7f4aa00f8f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:07:20.361 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:07:20.361+0000 7f4aa819d640 1 -- 172.21.15.162:0/869509307 shutdown_connections 2024-04-27T04:07:20.361 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:07:20.361+0000 7f4aa819d640 1 -- 172.21.15.162:0/869509307 wait complete. 2024-04-27T04:07:20.370 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:07:19 smithi162 ceph-mon[26841]: pgmap v185: 65 pgs: 65 active+clean; 579 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 5.9 KiB/s rd, 0 B/s wr, 6 op/s 2024-04-27T04:07:20.370 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:07:19 smithi162 ceph-mon[26841]: from='client.? 172.21.15.162:0/2562771595' entity='client.admin' cmd=[{"prefix": "mds versions", "format": "json"}]: dispatch 2024-04-27T04:07:20.371 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs.cephfs.meta pg_num 2024-04-27T04:07:20.475 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:07:20.474+0000 7f261091d640 1 -- 172.21.15.162:0/154318906 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f260c0fc330 msgr2=0x7f260c0fc7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:07:20.475 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:07:20.474+0000 7f261091d640 1 --2- 172.21.15.162:0/154318906 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f260c0fc330 0x7f260c0fc7a0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f26000048d0 tx=0x7f2600009da0 comp rx=0 tx=0).stop 2024-04-27T04:07:20.475 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:07:20.474+0000 7f261091d640 1 -- 172.21.15.162:0/154318906 shutdown_connections 2024-04-27T04:07:20.475 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:07:20.474+0000 7f261091d640 1 --2- 172.21.15.162:0/154318906 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f260c0fc330 0x7f260c0fc7a0 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:07:20.476 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:07:20.474+0000 7f261091d640 1 --2- 172.21.15.162:0/154318906 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f260c0fb060 0x7f260c0fb470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:07:20.476 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:07:20.474+0000 7f261091d640 1 -- 172.21.15.162:0/154318906 >> 172.21.15.162:0/154318906 conn(0x7f260c0f6820 msgr2=0x7f260c0f8c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:07:20.476 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:07:20.474+0000 7f261091d640 1 -- 172.21.15.162:0/154318906 shutdown_connections 2024-04-27T04:07:20.476 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:07:20.474+0000 7f261091d640 1 -- 172.21.15.162:0/154318906 wait complete. 2024-04-27T04:07:20.476 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:07:20.474+0000 7f261091d640 1 Processor -- start 2024-04-27T04:07:20.476 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:07:20.475+0000 7f261091d640 1 -- start start 2024-04-27T04:07:20.476 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:07:20.475+0000 7f261091d640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f260c0fb060 0x7f260c10bb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:07:20.476 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:07:20.475+0000 7f261091d640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f260c0fc330 0x7f260c10c0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:07:20.476 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:07:20.475+0000 7f261091d640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f260c10c720 con 0x7f260c0fb060 2024-04-27T04:07:20.476 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:07:20.475+0000 7f261091d640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f260c111120 con 0x7f260c0fc330 2024-04-27T04:07:20.476 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:07:20.475+0000 7f260a59c640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f260c0fb060 0x7f260c10bb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:07:20.476 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:07:20.475+0000 7f260a59c640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f260c0fb060 0x7f260c10bb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:38204/0 (socket says 172.21.15.162:38204) 2024-04-27T04:07:20.476 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:07:20.475+0000 7f260a59c640 1 -- 172.21.15.162:0/1656454859 learned_addr learned my addr 172.21.15.162:0/1656454859 (peer_addr_for_me v2:172.21.15.162:0/0) 2024-04-27T04:07:20.476 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:07:20.475+0000 7f2609d9b640 1 --2- 172.21.15.162:0/1656454859 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f260c0fc330 0x7f260c10c0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:07:20.477 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:07:20.475+0000 7f260a59c640 1 -- 172.21.15.162:0/1656454859 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f260c0fc330 msgr2=0x7f260c10c0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:07:20.477 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:07:20.475+0000 7f260a59c640 1 --2- 172.21.15.162:0/1656454859 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f260c0fc330 0x7f260c10c0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:07:20.477 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:07:20.475+0000 7f260a59c640 1 -- 172.21.15.162:0/1656454859 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2600005040 con 0x7f260c0fb060 2024-04-27T04:07:20.477 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:07:20.475+0000 7f2609d9b640 1 --2- 172.21.15.162:0/1656454859 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f260c0fc330 0x7f260c10c0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-27T04:07:20.477 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:07:20.475+0000 7f260a59c640 1 --2- 172.21.15.162:0/1656454859 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f260c0fb060 0x7f260c10bb90 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f25fc00c810 tx=0x7f25fc0043c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:07:20.477 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:07:20.475+0000 7f25faffd640 1 -- 172.21.15.162:0/1656454859 <== mon.0 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f25fc00a820 con 0x7f260c0fb060 2024-04-27T04:07:20.477 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:07:20.475+0000 7f261091d640 1 -- 172.21.15.162:0/1656454859 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f260c1112c0 con 0x7f260c0fb060 2024-04-27T04:07:20.477 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:07:20.475+0000 7f25faffd640 1 -- 172.21.15.162:0/1656454859 <== mon.0 v2:172.21.15.162:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f25fc00ae50 con 0x7f260c0fb060 2024-04-27T04:07:20.477 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:07:20.475+0000 7f25faffd640 1 -- 172.21.15.162:0/1656454859 <== mon.0 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f25fc0115c0 con 0x7f260c0fb060 2024-04-27T04:07:20.477 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:07:20.475+0000 7f261091d640 1 -- 172.21.15.162:0/1656454859 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f260c1117b0 con 0x7f260c0fb060 2024-04-27T04:07:20.477 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:07:20.476+0000 7f261091d640 1 -- 172.21.15.162:0/1656454859 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f260c04bcc0 con 0x7f260c0fb060 2024-04-27T04:07:20.478 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:07:20.477+0000 7f25faffd640 1 -- 172.21.15.162:0/1656454859 <== mon.0 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 19) v1 ==== 86631+0+0 (secure 0 0 0) 0x7f25fc00a980 con 0x7f260c0fb060 2024-04-27T04:07:20.478 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:07:20.478+0000 7f25faffd640 1 --2- 172.21.15.162:0/1656454859 >> [v2:172.21.15.162:6800/4079782599,v1:172.21.15.162:6801/4079782599] conn(0x7f25f4069260 0x7f25f406b710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:07:20.478 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:07:20.478+0000 7f25faffd640 1 -- 172.21.15.162:0/1656454859 <== mon.0 v2:172.21.15.162:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5550+0+0 (secure 0 0 0) 0x7f25fc087810 con 0x7f260c0fb060 2024-04-27T04:07:20.478 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:07:20.478+0000 7f2609d9b640 1 --2- 172.21.15.162:0/1656454859 >> [v2:172.21.15.162:6800/4079782599,v1:172.21.15.162:6801/4079782599] conn(0x7f25f4069260 0x7f25f406b710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:07:20.478 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:07:20.478+0000 7f2609d9b640 1 --2- 172.21.15.162:0/1656454859 >> [v2:172.21.15.162:6800/4079782599,v1:172.21.15.162:6801/4079782599] conn(0x7f25f4069260 0x7f25f406b710 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f2600003080 tx=0x7f2600009da0 comp rx=0 tx=0).ready entity=mgr.14210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:07:20.481 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:07:20.482+0000 7f25faffd640 1 -- 172.21.15.162:0/1656454859 <== mon.0 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f25fc0539d0 con 0x7f260c0fb060 2024-04-27T04:07:20.656 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:07:20.657+0000 7f261091d640 1 -- 172.21.15.162:0/1656454859 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs.cephfs.meta", "var": "pg_num"} v 0) v1 -- 0x7f260c111a90 con 0x7f260c0fb060 2024-04-27T04:07:20.656 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:07:20.657+0000 7f25faffd640 1 -- 172.21.15.162:0/1656454859 <== mon.0 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs.cephfs.meta", "var": "pg_num"}]=0 v46) v1 ==== 107+0+11 (secure 0 0 0) 0x7f25fc016020 con 0x7f260c0fb060 2024-04-27T04:07:20.656 INFO:teuthology.orchestra.run.smithi162.stdout:pg_num: 32 2024-04-27T04:07:20.658 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:07:20.659+0000 7f261091d640 1 -- 172.21.15.162:0/1656454859 >> [v2:172.21.15.162:6800/4079782599,v1:172.21.15.162:6801/4079782599] conn(0x7f25f4069260 msgr2=0x7f25f406b710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:07:20.658 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:07:20.659+0000 7f261091d640 1 --2- 172.21.15.162:0/1656454859 >> [v2:172.21.15.162:6800/4079782599,v1:172.21.15.162:6801/4079782599] conn(0x7f25f4069260 0x7f25f406b710 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f2600003080 tx=0x7f2600009da0 comp rx=0 tx=0).stop 2024-04-27T04:07:20.658 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:07:20.659+0000 7f261091d640 1 -- 172.21.15.162:0/1656454859 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f260c0fb060 msgr2=0x7f260c10bb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:07:20.659 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:07:20.659+0000 7f261091d640 1 --2- 172.21.15.162:0/1656454859 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f260c0fb060 0x7f260c10bb90 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f25fc00c810 tx=0x7f25fc0043c0 comp rx=0 tx=0).stop 2024-04-27T04:07:20.659 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:07:20.659+0000 7f261091d640 1 -- 172.21.15.162:0/1656454859 shutdown_connections 2024-04-27T04:07:20.659 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:07:20.659+0000 7f261091d640 1 --2- 172.21.15.162:0/1656454859 >> [v2:172.21.15.162:6800/4079782599,v1:172.21.15.162:6801/4079782599] conn(0x7f25f4069260 0x7f25f406b710 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:07:20.659 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:07:20.659+0000 7f261091d640 1 --2- 172.21.15.162:0/1656454859 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f260c0fc330 0x7f260c10c0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:07:20.659 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:07:20.659+0000 7f261091d640 1 --2- 172.21.15.162:0/1656454859 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f260c0fb060 0x7f260c10bb90 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:07:20.659 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:07:20.659+0000 7f261091d640 1 -- 172.21.15.162:0/1656454859 >> 172.21.15.162:0/1656454859 conn(0x7f260c0f6820 msgr2=0x7f260c0f7f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:07:20.659 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:07:20.659+0000 7f261091d640 1 -- 172.21.15.162:0/1656454859 shutdown_connections 2024-04-27T04:07:20.659 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:07:20.659+0000 7f261091d640 1 -- 172.21.15.162:0/1656454859 wait complete. 2024-04-27T04:07:20.670 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs.cephfs.data pg_num 2024-04-27T04:07:20.770 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:07:20.769+0000 7f646c0ae640 1 -- 172.21.15.162:0/1451820245 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6464065010 msgr2=0x7f64640fed70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:07:20.770 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:07:20.769+0000 7f646c0ae640 1 --2- 172.21.15.162:0/1451820245 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6464065010 0x7f64640fed70 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f645c009c50 tx=0x7f645c004d10 comp rx=0 tx=0).stop 2024-04-27T04:07:20.770 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:07:20.769+0000 7f646c0ae640 1 -- 172.21.15.162:0/1451820245 shutdown_connections 2024-04-27T04:07:20.770 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:07:20.769+0000 7f646c0ae640 1 --2- 172.21.15.162:0/1451820245 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f64640ff2b0 0x7f6464101690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:07:20.771 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:07:20.769+0000 7f646c0ae640 1 --2- 172.21.15.162:0/1451820245 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6464065010 0x7f64640fed70 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:07:20.771 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:07:20.769+0000 7f646c0ae640 1 -- 172.21.15.162:0/1451820245 >> 172.21.15.162:0/1451820245 conn(0x7f64640f6820 msgr2=0x7f64640f8c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:07:20.771 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:07:20.769+0000 7f646c0ae640 1 -- 172.21.15.162:0/1451820245 shutdown_connections 2024-04-27T04:07:20.771 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:07:20.769+0000 7f646c0ae640 1 -- 172.21.15.162:0/1451820245 wait complete. 2024-04-27T04:07:20.771 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:07:20.770+0000 7f646c0ae640 1 Processor -- start 2024-04-27T04:07:20.771 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:07:20.770+0000 7f646c0ae640 1 -- start start 2024-04-27T04:07:20.771 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:07:20.770+0000 7f646c0ae640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6464065010 0x7f646410cd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:07:20.771 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:07:20.770+0000 7f646c0ae640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f64640ff2b0 0x7f646410d250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:07:20.771 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:07:20.770+0000 7f646c0ae640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f646410d8a0 con 0x7f64640ff2b0 2024-04-27T04:07:20.771 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:07:20.770+0000 7f646c0ae640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6464112260 con 0x7f6464065010 2024-04-27T04:07:20.771 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:07:20.770+0000 7f6469649640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f64640ff2b0 0x7f646410d250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:07:20.771 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:07:20.770+0000 7f6469649640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f64640ff2b0 0x7f646410d250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:38222/0 (socket says 172.21.15.162:38222) 2024-04-27T04:07:20.771 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:07:20.770+0000 7f6469649640 1 -- 172.21.15.162:0/3036590342 learned_addr learned my addr 172.21.15.162:0/3036590342 (peer_addr_for_me v2:172.21.15.162:0/0) 2024-04-27T04:07:20.771 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:07:20.770+0000 7f6469e4a640 1 --2- 172.21.15.162:0/3036590342 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6464065010 0x7f646410cd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:07:20.771 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:07:20.770+0000 7f6469649640 1 -- 172.21.15.162:0/3036590342 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6464065010 msgr2=0x7f646410cd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:07:20.772 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:07:20.770+0000 7f6469649640 1 --2- 172.21.15.162:0/3036590342 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6464065010 0x7f646410cd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:07:20.772 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:07:20.770+0000 7f6469649640 1 -- 172.21.15.162:0/3036590342 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f645c0098b0 con 0x7f64640ff2b0 2024-04-27T04:07:20.772 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:07:20.771+0000 7f6469649640 1 --2- 172.21.15.162:0/3036590342 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f64640ff2b0 0x7f646410d250 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f645400c790 tx=0x7f64540043c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:07:20.772 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:07:20.771+0000 7f645a7fc640 1 -- 172.21.15.162:0/3036590342 <== mon.0 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f645400a820 con 0x7f64640ff2b0 2024-04-27T04:07:20.772 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:07:20.771+0000 7f645a7fc640 1 -- 172.21.15.162:0/3036590342 <== mon.0 v2:172.21.15.162:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f645400ae50 con 0x7f64640ff2b0 2024-04-27T04:07:20.772 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:07:20.771+0000 7f645a7fc640 1 -- 172.21.15.162:0/3036590342 <== mon.0 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f64540115e0 con 0x7f64640ff2b0 2024-04-27T04:07:20.772 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:07:20.771+0000 7f646c0ae640 1 -- 172.21.15.162:0/3036590342 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6464112400 con 0x7f64640ff2b0 2024-04-27T04:07:20.772 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:07:20.771+0000 7f646c0ae640 1 -- 172.21.15.162:0/3036590342 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6464112950 con 0x7f64640ff2b0 2024-04-27T04:07:20.773 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:07:20.772+0000 7f646c0ae640 1 -- 172.21.15.162:0/3036590342 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f646404bcc0 con 0x7f64640ff2b0 2024-04-27T04:07:20.773 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:07:20.772+0000 7f645a7fc640 1 -- 172.21.15.162:0/3036590342 <== mon.0 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 19) v1 ==== 86631+0+0 (secure 0 0 0) 0x7f6454012400 con 0x7f64640ff2b0 2024-04-27T04:07:20.773 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:07:20.773+0000 7f645a7fc640 1 --2- 172.21.15.162:0/3036590342 >> [v2:172.21.15.162:6800/4079782599,v1:172.21.15.162:6801/4079782599] conn(0x7f6450069210 0x7f645006b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:07:20.773 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:07:20.773+0000 7f6469e4a640 1 --2- 172.21.15.162:0/3036590342 >> [v2:172.21.15.162:6800/4079782599,v1:172.21.15.162:6801/4079782599] conn(0x7f6450069210 0x7f645006b6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:07:20.773 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:07:20.773+0000 7f645a7fc640 1 -- 172.21.15.162:0/3036590342 <== mon.0 v2:172.21.15.162:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5550+0+0 (secure 0 0 0) 0x7f64540872e0 con 0x7f64640ff2b0 2024-04-27T04:07:20.773 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:07:20.773+0000 7f6469e4a640 1 --2- 172.21.15.162:0/3036590342 >> [v2:172.21.15.162:6800/4079782599,v1:172.21.15.162:6801/4079782599] conn(0x7f6450069210 0x7f645006b6c0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f646410e1d0 tx=0x7f645c02e040 comp rx=0 tx=0).ready entity=mgr.14210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:07:20.776 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:07:20.777+0000 7f645a7fc640 1 -- 172.21.15.162:0/3036590342 <== mon.0 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f6454053720 con 0x7f64640ff2b0 2024-04-27T04:07:20.961 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:07:20.961+0000 7f646c0ae640 1 -- 172.21.15.162:0/3036590342 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs.cephfs.data", "var": "pg_num"} v 0) v1 -- 0x7f6464104cd0 con 0x7f64640ff2b0 2024-04-27T04:07:20.961 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:07:20.962+0000 7f645a7fc640 1 -- 172.21.15.162:0/3036590342 <== mon.0 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs.cephfs.data", "var": "pg_num"}]=0 v46) v1 ==== 107+0+11 (secure 0 0 0) 0x7f6454056d40 con 0x7f64640ff2b0 2024-04-27T04:07:20.961 INFO:teuthology.orchestra.run.smithi162.stdout:pg_num: 32 2024-04-27T04:07:20.963 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:07:20.964+0000 7f646c0ae640 1 -- 172.21.15.162:0/3036590342 >> [v2:172.21.15.162:6800/4079782599,v1:172.21.15.162:6801/4079782599] conn(0x7f6450069210 msgr2=0x7f645006b6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:07:20.963 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:07:20.964+0000 7f646c0ae640 1 --2- 172.21.15.162:0/3036590342 >> [v2:172.21.15.162:6800/4079782599,v1:172.21.15.162:6801/4079782599] conn(0x7f6450069210 0x7f645006b6c0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f646410e1d0 tx=0x7f645c02e040 comp rx=0 tx=0).stop 2024-04-27T04:07:20.964 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:07:20.964+0000 7f646c0ae640 1 -- 172.21.15.162:0/3036590342 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f64640ff2b0 msgr2=0x7f646410d250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:07:20.964 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:07:20.964+0000 7f646c0ae640 1 --2- 172.21.15.162:0/3036590342 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f64640ff2b0 0x7f646410d250 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f645400c790 tx=0x7f64540043c0 comp rx=0 tx=0).stop 2024-04-27T04:07:20.964 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:07:20.964+0000 7f646c0ae640 1 -- 172.21.15.162:0/3036590342 shutdown_connections 2024-04-27T04:07:20.964 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:07:20.964+0000 7f646c0ae640 1 --2- 172.21.15.162:0/3036590342 >> [v2:172.21.15.162:6800/4079782599,v1:172.21.15.162:6801/4079782599] conn(0x7f6450069210 0x7f645006b6c0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:07:20.964 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:07:20.964+0000 7f646c0ae640 1 --2- 172.21.15.162:0/3036590342 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f64640ff2b0 0x7f646410d250 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:07:20.964 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:07:20.964+0000 7f646c0ae640 1 --2- 172.21.15.162:0/3036590342 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6464065010 0x7f646410cd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:07:20.964 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:07:20.964+0000 7f646c0ae640 1 -- 172.21.15.162:0/3036590342 >> 172.21.15.162:0/3036590342 conn(0x7f64640f6820 msgr2=0x7f64640f6c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:07:20.964 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:07:20.964+0000 7f646c0ae640 1 -- 172.21.15.162:0/3036590342 shutdown_connections 2024-04-27T04:07:20.964 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:07:20.964+0000 7f646c0ae640 1 -- 172.21.15.162:0/3036590342 wait complete. 2024-04-27T04:07:20.973 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:07:20 smithi162 ceph-mon[26841]: from='client.? 172.21.15.162:0/2990956017' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-27T04:07:20.973 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:07:20 smithi162 ceph-mon[26841]: from='client.? 172.21.15.162:0/869509307' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-27T04:07:20.973 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:07:20 smithi162 ceph-mon[26841]: from='client.? 172.21.15.162:0/1656454859' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs.cephfs.meta", "var": "pg_num"}]: dispatch 2024-04-27T04:07:20.974 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-27T04:07:21.071 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:07:21.069+0000 7f37136ae640 1 -- 172.21.15.162:0/3532041264 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f370c0fc180 msgr2=0x7f370c0fc590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:07:21.071 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:07:21.069+0000 7f37136ae640 1 --2- 172.21.15.162:0/3532041264 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f370c0fc180 0x7f370c0fc590 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f3704009c50 tx=0x7f3704004d10 comp rx=0 tx=0).stop 2024-04-27T04:07:21.071 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:07:21.069+0000 7f37136ae640 1 -- 172.21.15.162:0/3532041264 shutdown_connections 2024-04-27T04:07:21.071 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:07:21.069+0000 7f37136ae640 1 --2- 172.21.15.162:0/3532041264 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f370c0fd450 0x7f370c0fd8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:07:21.071 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:07:21.069+0000 7f37136ae640 1 --2- 172.21.15.162:0/3532041264 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f370c0fc180 0x7f370c0fc590 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:07:21.071 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:07:21.070+0000 7f37136ae640 1 -- 172.21.15.162:0/3532041264 >> 172.21.15.162:0/3532041264 conn(0x7f370c0f7960 msgr2=0x7f370c0f9d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:07:21.071 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:07:21.070+0000 7f37136ae640 1 -- 172.21.15.162:0/3532041264 shutdown_connections 2024-04-27T04:07:21.071 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:07:21.070+0000 7f37136ae640 1 -- 172.21.15.162:0/3532041264 wait complete. 2024-04-27T04:07:21.072 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:07:21.070+0000 7f37136ae640 1 Processor -- start 2024-04-27T04:07:21.072 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:07:21.070+0000 7f37136ae640 1 -- start start 2024-04-27T04:07:21.072 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:07:21.070+0000 7f37136ae640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f370c0fc180 0x7f370c10cc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:07:21.072 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:07:21.070+0000 7f37136ae640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f370c0fd450 0x7f370c10d1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:07:21.072 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:07:21.070+0000 7f37136ae640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f370c10d800 con 0x7f370c0fd450 2024-04-27T04:07:21.072 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:07:21.070+0000 7f37136ae640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f370c112190 con 0x7f370c0fc180 2024-04-27T04:07:21.072 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:07:21.070+0000 7f3710c49640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f370c0fd450 0x7f370c10d1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:07:21.072 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:07:21.070+0000 7f3710c49640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f370c0fd450 0x7f370c10d1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:38234/0 (socket says 172.21.15.162:38234) 2024-04-27T04:07:21.072 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:07:21.070+0000 7f3710c49640 1 -- 172.21.15.162:0/2150995130 learned_addr learned my addr 172.21.15.162:0/2150995130 (peer_addr_for_me v2:172.21.15.162:0/0) 2024-04-27T04:07:21.072 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:07:21.071+0000 7f371144a640 1 --2- 172.21.15.162:0/2150995130 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f370c0fc180 0x7f370c10cc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:07:21.072 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:07:21.071+0000 7f3710c49640 1 -- 172.21.15.162:0/2150995130 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f370c0fc180 msgr2=0x7f370c10cc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:07:21.072 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:07:21.071+0000 7f3710c49640 1 --2- 172.21.15.162:0/2150995130 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f370c0fc180 0x7f370c10cc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:07:21.073 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:07:21.071+0000 7f3710c49640 1 -- 172.21.15.162:0/2150995130 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f37040098b0 con 0x7f370c0fd450 2024-04-27T04:07:21.073 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:07:21.071+0000 7f3710c49640 1 --2- 172.21.15.162:0/2150995130 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f370c0fd450 0x7f370c10d1b0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f3700002a80 tx=0x7f3700004600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:07:21.073 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:07:21.071+0000 7f3709ffb640 1 -- 172.21.15.162:0/2150995130 <== mon.0 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3700007c30 con 0x7f370c0fd450 2024-04-27T04:07:21.073 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:07:21.071+0000 7f37136ae640 1 -- 172.21.15.162:0/2150995130 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f370c112330 con 0x7f370c0fd450 2024-04-27T04:07:21.073 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:07:21.071+0000 7f3709ffb640 1 -- 172.21.15.162:0/2150995130 <== mon.0 v2:172.21.15.162:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f370000f040 con 0x7f370c0fd450 2024-04-27T04:07:21.073 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:07:21.071+0000 7f3709ffb640 1 -- 172.21.15.162:0/2150995130 <== mon.0 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f37000096b0 con 0x7f370c0fd450 2024-04-27T04:07:21.073 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:07:21.071+0000 7f37136ae640 1 -- 172.21.15.162:0/2150995130 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f370c112820 con 0x7f370c0fd450 2024-04-27T04:07:21.074 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:07:21.072+0000 7f37136ae640 1 -- 172.21.15.162:0/2150995130 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f370c04bcc0 con 0x7f370c0fd450 2024-04-27T04:07:21.074 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:07:21.072+0000 7f3709ffb640 1 -- 172.21.15.162:0/2150995130 <== mon.0 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 19) v1 ==== 86631+0+0 (secure 0 0 0) 0x7f370000d6b0 con 0x7f370c0fd450 2024-04-27T04:07:21.074 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:07:21.073+0000 7f3709ffb640 1 --2- 172.21.15.162:0/2150995130 >> [v2:172.21.15.162:6800/4079782599,v1:172.21.15.162:6801/4079782599] conn(0x7f36f4069210 0x7f36f406b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:07:21.074 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:07:21.073+0000 7f3709ffb640 1 -- 172.21.15.162:0/2150995130 <== mon.0 v2:172.21.15.162:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5550+0+0 (secure 0 0 0) 0x7f3700086c00 con 0x7f370c0fd450 2024-04-27T04:07:21.075 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:07:21.073+0000 7f371144a640 1 --2- 172.21.15.162:0/2150995130 >> [v2:172.21.15.162:6800/4079782599,v1:172.21.15.162:6801/4079782599] conn(0x7f36f4069210 0x7f36f406b6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:07:21.075 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:07:21.073+0000 7f371144a640 1 --2- 172.21.15.162:0/2150995130 >> [v2:172.21.15.162:6800/4079782599,v1:172.21.15.162:6801/4079782599] conn(0x7f36f4069210 0x7f36f406b6c0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f3704000f80 tx=0x7f370402e040 comp rx=0 tx=0).ready entity=mgr.14210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:07:21.076 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:07:21.077+0000 7f3709ffb640 1 -- 172.21.15.162:0/2150995130 <== mon.0 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f370004d250 con 0x7f370c0fd450 2024-04-27T04:07:21.173 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:07:20 smithi181 ceph-mon[33774]: from='client.? 172.21.15.162:0/2990956017' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-27T04:07:21.173 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:07:20 smithi181 ceph-mon[33774]: from='client.? 172.21.15.162:0/869509307' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-27T04:07:21.173 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:07:20 smithi181 ceph-mon[33774]: from='client.? 172.21.15.162:0/1656454859' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs.cephfs.meta", "var": "pg_num"}]: dispatch 2024-04-27T04:07:21.284 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:07:21.284+0000 7f37136ae640 1 -- 172.21.15.162:0/2150995130 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f370c112a70 con 0x7f370c0fd450 2024-04-27T04:07:21.284 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:07:21.285+0000 7f3709ffb640 1 -- 172.21.15.162:0/2150995130 <== mon.0 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 15 v15) v1 ==== 94+0+4705 (secure 0 0 0) 0x7f370008c020 con 0x7f370c0fd450 2024-04-27T04:07:21.284 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-04-27T04:07:21.284 INFO:teuthology.orchestra.run.smithi162.stdout:{"epoch":15,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14488,"name":"cephfs.smithi162.gzlyjj","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6827/2629103894","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6826","nonce":2629103894},{"type":"v1","addr":"172.21.15.162:6827","nonce":2629103894}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":13},{"gid":24287,"name":"cephfs.smithi181.bkkcdm","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.181:6827/3756475453","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6826","nonce":3756475453},{"type":"v1","addr":"172.21.15.181:6827","nonce":3756475453}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":14}],"filesystems":[{"mdsmap":{"epoch":15,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-27T04:06:53.095749+0000","modified":"2024-04-27T04:07:11.285033+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":46,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":24273},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14476":{"gid":14476,"name":"cephfs.smithi162.xcnapk","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":2,"addr":"172.21.15.162:6829/1296160676","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6828","nonce":1296160676},{"type":"v1","addr":"172.21.15.162:6829","nonce":1296160676}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24273":{"gid":24273,"name":"cephfs.smithi181.dzomgh","rank":0,"incarnation":12,"state":"up:active","state_seq":6,"addr":"172.21.15.181:6825/658214993","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":658214993},{"type":"v1","addr":"172.21.15.181:6825","nonce":658214993}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":1},"id":1}]} 2024-04-27T04:07:21.285 INFO:teuthology.orchestra.run.smithi162.stderr:dumped fsmap epoch 15 2024-04-27T04:07:21.287 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:07:21.287+0000 7f37136ae640 1 -- 172.21.15.162:0/2150995130 >> [v2:172.21.15.162:6800/4079782599,v1:172.21.15.162:6801/4079782599] conn(0x7f36f4069210 msgr2=0x7f36f406b6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:07:21.287 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:07:21.287+0000 7f37136ae640 1 --2- 172.21.15.162:0/2150995130 >> [v2:172.21.15.162:6800/4079782599,v1:172.21.15.162:6801/4079782599] conn(0x7f36f4069210 0x7f36f406b6c0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f3704000f80 tx=0x7f370402e040 comp rx=0 tx=0).stop 2024-04-27T04:07:21.287 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:07:21.287+0000 7f37136ae640 1 -- 172.21.15.162:0/2150995130 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f370c0fd450 msgr2=0x7f370c10d1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:07:21.287 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:07:21.287+0000 7f37136ae640 1 --2- 172.21.15.162:0/2150995130 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f370c0fd450 0x7f370c10d1b0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f3700002a80 tx=0x7f3700004600 comp rx=0 tx=0).stop 2024-04-27T04:07:21.287 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:07:21.287+0000 7f37136ae640 1 -- 172.21.15.162:0/2150995130 shutdown_connections 2024-04-27T04:07:21.287 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:07:21.287+0000 7f37136ae640 1 --2- 172.21.15.162:0/2150995130 >> [v2:172.21.15.162:6800/4079782599,v1:172.21.15.162:6801/4079782599] conn(0x7f36f4069210 0x7f36f406b6c0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:07:21.287 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:07:21.287+0000 7f37136ae640 1 --2- 172.21.15.162:0/2150995130 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f370c0fd450 0x7f370c10d1b0 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:07:21.287 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:07:21.287+0000 7f37136ae640 1 --2- 172.21.15.162:0/2150995130 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f370c0fc180 0x7f370c10cc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:07:21.287 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:07:21.287+0000 7f37136ae640 1 -- 172.21.15.162:0/2150995130 >> 172.21.15.162:0/2150995130 conn(0x7f370c0f7960 msgr2=0x7f370c0f8ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:07:21.287 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:07:21.288+0000 7f37136ae640 1 -- 172.21.15.162:0/2150995130 shutdown_connections 2024-04-27T04:07:21.287 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:07:21.288+0000 7f37136ae640 1 -- 172.21.15.162:0/2150995130 wait complete. 2024-04-27T04:07:21.298 DEBUG:tasks.fs:fs fscid=1,name=cephfs state = {'epoch': 15, 'max_mds': 1, 'flags': 50} 2024-04-27T04:07:21.298 INFO:teuthology.run_tasks:Running task ceph-fuse... 2024-04-27T04:07:21.330 INFO:tasks.ceph_fuse:Running ceph_fuse task... 2024-04-27T04:07:21.331 INFO:tasks.ceph_fuse:config is {'client.0': {}, 'client.1': {}} 2024-04-27T04:07:21.331 INFO:tasks.ceph_fuse:client.0 config is {} 2024-04-27T04:07:21.331 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-04-27T04:07:21.331 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-27T04:07:21.331 INFO:tasks.ceph_fuse:client.1 config is {} 2024-04-27T04:07:21.331 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-04-27T04:07:21.331 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-04-27T04:07:21.332 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-27T04:07:21.332 DEBUG:teuthology.orchestra.run.smithi181:> ip netns list 2024-04-27T04:07:21.351 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-27T04:07:21.352 DEBUG:teuthology.orchestra.run.smithi181:> sudo ip link delete ceph-brx 2024-04-27T04:07:21.425 INFO:teuthology.orchestra.run.smithi181.stderr:Cannot find device "ceph-brx" 2024-04-27T04:07:21.426 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-27T04:07:21.427 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-27T04:07:21.427 DEBUG:teuthology.orchestra.run.smithi162:> ip netns list 2024-04-27T04:07:21.442 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-27T04:07:21.442 DEBUG:teuthology.orchestra.run.smithi162:> sudo ip link delete ceph-brx 2024-04-27T04:07:21.507 INFO:teuthology.orchestra.run.smithi162.stderr:Cannot find device "ceph-brx" 2024-04-27T04:07:21.508 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-27T04:07:21.508 INFO:tasks.ceph_fuse:Mounting ceph-fuse clients... 2024-04-27T04:07:21.509 DEBUG:teuthology.orchestra.run.smithi162:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell --fsid 7f9750d6-044a-11ef-bc93-c7b262605968 -- ceph fs ls 2024-04-27T04:07:21.825 INFO:teuthology.orchestra.run.smithi162.stderr:Inferring config /var/lib/ceph/7f9750d6-044a-11ef-bc93-c7b262605968/mon.smithi162/config 2024-04-27T04:07:22.173 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:07:21 smithi181 ceph-mon[33774]: pgmap v186: 65 pgs: 65 active+clean; 579 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 5.0 KiB/s rd, 0 B/s wr, 5 op/s 2024-04-27T04:07:22.173 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:07:21 smithi181 ceph-mon[33774]: from='client.? 172.21.15.162:0/3036590342' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs.cephfs.data", "var": "pg_num"}]: dispatch 2024-04-27T04:07:22.173 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:07:21 smithi181 ceph-mon[33774]: from='client.? 172.21.15.162:0/2150995130' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-27T04:07:22.173 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:07:21 smithi181 ceph-mon[33774]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' 2024-04-27T04:07:22.173 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:07:21 smithi181 ceph-mon[33774]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' 2024-04-27T04:07:22.173 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:07:21 smithi181 ceph-mon[33774]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T04:07:22.173 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:07:21 smithi181 ceph-mon[33774]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T04:07:22.173 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:07:21 smithi181 ceph-mon[33774]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' 2024-04-27T04:07:22.203 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:07:21 smithi162 ceph-mon[26841]: pgmap v186: 65 pgs: 65 active+clean; 579 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 5.0 KiB/s rd, 0 B/s wr, 5 op/s 2024-04-27T04:07:22.203 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:07:21 smithi162 ceph-mon[26841]: from='client.? 172.21.15.162:0/3036590342' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs.cephfs.data", "var": "pg_num"}]: dispatch 2024-04-27T04:07:22.203 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:07:21 smithi162 ceph-mon[26841]: from='client.? 172.21.15.162:0/2150995130' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-27T04:07:22.204 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:07:21 smithi162 ceph-mon[26841]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' 2024-04-27T04:07:22.204 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:07:21 smithi162 ceph-mon[26841]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' 2024-04-27T04:07:22.204 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:07:21 smithi162 ceph-mon[26841]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T04:07:22.204 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:07:21 smithi162 ceph-mon[26841]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T04:07:22.204 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:07:21 smithi162 ceph-mon[26841]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' 2024-04-27T04:07:23.425 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:07:23.425+0000 7f7b72f49700 1 -- 172.21.15.162:0/2263708688 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7b6c1026a0 msgr2=0x7f7b6c0fbd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:07:23.425 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:07:23.425+0000 7f7b72f49700 1 --2- 172.21.15.162:0/2263708688 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7b6c1026a0 0x7f7b6c0fbd10 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f7b58004940 tx=0x7f7b58009e30 comp rx=0 tx=0).stop 2024-04-27T04:07:23.425 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:07:23.425+0000 7f7b72f49700 1 -- 172.21.15.162:0/2263708688 shutdown_connections 2024-04-27T04:07:23.425 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:07:23.425+0000 7f7b72f49700 1 --2- 172.21.15.162:0/2263708688 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7b6c1026a0 0x7f7b6c0fbd10 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:07:23.425 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:07:23.425+0000 7f7b72f49700 1 --2- 172.21.15.162:0/2263708688 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7b6c101d00 0x7f7b6c1020d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:07:23.425 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:07:23.425+0000 7f7b72f49700 1 -- 172.21.15.162:0/2263708688 >> 172.21.15.162:0/2263708688 conn(0x7f7b6c0f78d0 msgr2=0x7f7b6c0f9ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:07:23.425 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:07:23.425+0000 7f7b72f49700 1 -- 172.21.15.162:0/2263708688 shutdown_connections 2024-04-27T04:07:23.425 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:07:23.425+0000 7f7b72f49700 1 -- 172.21.15.162:0/2263708688 wait complete. 2024-04-27T04:07:23.425 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:07:23.426+0000 7f7b72f49700 1 Processor -- start 2024-04-27T04:07:23.426 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:07:23.426+0000 7f7b72f49700 1 -- start start 2024-04-27T04:07:23.429 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:07:23.426+0000 7f7b72f49700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7b6c101d00 0x7f7b6c113b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:07:23.429 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:07:23.426+0000 7f7b72f49700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7b6c1026a0 0x7f7b6c1140b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:07:23.429 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:07:23.426+0000 7f7b72f49700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b6c114700 con 0x7f7b6c101d00 2024-04-27T04:07:23.429 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:07:23.426+0000 7f7b72f49700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b6c10daf0 con 0x7f7b6c1026a0 2024-04-27T04:07:23.429 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:07:23.426+0000 7f7b70ce5700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7b6c101d00 0x7f7b6c113b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:07:23.429 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:07:23.426+0000 7f7b70ce5700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7b6c101d00 0x7f7b6c113b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:38242/0 (socket says 172.21.15.162:38242) 2024-04-27T04:07:23.429 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:07:23.426+0000 7f7b70ce5700 1 -- 172.21.15.162:0/2663475443 learned_addr learned my addr 172.21.15.162:0/2663475443 (peer_addr_for_me v2:172.21.15.162:0/0) 2024-04-27T04:07:23.429 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:07:23.427+0000 7f7b6bfff700 1 --2- 172.21.15.162:0/2663475443 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7b6c1026a0 0x7f7b6c1140b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:07:23.429 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:07:23.427+0000 7f7b70ce5700 1 -- 172.21.15.162:0/2663475443 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7b6c1026a0 msgr2=0x7f7b6c1140b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:07:23.429 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:07:23.427+0000 7f7b70ce5700 1 --2- 172.21.15.162:0/2663475443 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7b6c1026a0 0x7f7b6c1140b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:07:23.430 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:07:23.427+0000 7f7b70ce5700 1 -- 172.21.15.162:0/2663475443 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b58005040 con 0x7f7b6c101d00 2024-04-27T04:07:23.430 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:07:23.427+0000 7f7b70ce5700 1 --2- 172.21.15.162:0/2663475443 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7b6c101d00 0x7f7b6c113b70 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f7b6000c830 tx=0x7f7b60004470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:07:23.430 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:07:23.427+0000 7f7b697fa700 1 -- 172.21.15.162:0/2663475443 <== mon.0 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7b600106f0 con 0x7f7b6c101d00 2024-04-27T04:07:23.430 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:07:23.427+0000 7f7b697fa700 1 -- 172.21.15.162:0/2663475443 <== mon.0 v2:172.21.15.162:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f7b60010d20 con 0x7f7b6c101d00 2024-04-27T04:07:23.430 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:07:23.427+0000 7f7b697fa700 1 -- 172.21.15.162:0/2663475443 <== mon.0 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7b600113a0 con 0x7f7b6c101d00 2024-04-27T04:07:23.430 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:07:23.427+0000 7f7b72f49700 1 -- 172.21.15.162:0/2663475443 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7b6c10ddd0 con 0x7f7b6c101d00 2024-04-27T04:07:23.430 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:07:23.427+0000 7f7b72f49700 1 -- 172.21.15.162:0/2663475443 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7b6c10e360 con 0x7f7b6c101d00 2024-04-27T04:07:23.430 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:07:23.428+0000 7f7b72f49700 1 -- 172.21.15.162:0/2663475443 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7b6c0fd450 con 0x7f7b6c101d00 2024-04-27T04:07:23.430 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:07:23.428+0000 7f7b697fa700 1 -- 172.21.15.162:0/2663475443 <== mon.0 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 19) v1 ==== 86631+0+0 (secure 0 0 0) 0x7f7b60012400 con 0x7f7b6c101d00 2024-04-27T04:07:23.430 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:07:23.429+0000 7f7b697fa700 1 --2- 172.21.15.162:0/2663475443 >> [v2:172.21.15.162:6800/4079782599,v1:172.21.15.162:6801/4079782599] conn(0x7f7b5c0692a0 0x7f7b5c06b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:07:23.430 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:07:23.429+0000 7f7b697fa700 1 -- 172.21.15.162:0/2663475443 <== mon.0 v2:172.21.15.162:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5550+0+0 (secure 0 0 0) 0x7f7b600870c0 con 0x7f7b6c101d00 2024-04-27T04:07:23.430 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:07:23.429+0000 7f7b6bfff700 1 --2- 172.21.15.162:0/2663475443 >> [v2:172.21.15.162:6800/4079782599,v1:172.21.15.162:6801/4079782599] conn(0x7f7b5c0692a0 0x7f7b5c06b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:07:23.430 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:07:23.429+0000 7f7b6bfff700 1 --2- 172.21.15.162:0/2663475443 >> [v2:172.21.15.162:6800/4079782599,v1:172.21.15.162:6801/4079782599] conn(0x7f7b5c0692a0 0x7f7b5c06b750 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f7b58004890 tx=0x7f7b58009e30 comp rx=0 tx=0).ready entity=mgr.14210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:07:23.432 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:07:23.433+0000 7f7b697fa700 1 -- 172.21.15.162:0/2663475443 <== mon.0 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f7b60056ed0 con 0x7f7b6c101d00 2024-04-27T04:07:23.662 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:07:23.663+0000 7f7b72f49700 1 -- 172.21.15.162:0/2663475443 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f7b6c108e30 con 0x7f7b6c101d00 2024-04-27T04:07:23.662 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:07:23.663+0000 7f7b697fa700 1 -- 172.21.15.162:0/2663475443 <== mon.0 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v15) v1 ==== 53+0+83 (secure 0 0 0) 0x7f7b60056a60 con 0x7f7b6c101d00 2024-04-27T04:07:23.663 INFO:teuthology.orchestra.run.smithi162.stdout:name: cephfs, metadata pool: cephfs.cephfs.meta, data pools: [cephfs.cephfs.data ] 2024-04-27T04:07:23.664 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:07:23.665+0000 7f7b72f49700 1 -- 172.21.15.162:0/2663475443 >> [v2:172.21.15.162:6800/4079782599,v1:172.21.15.162:6801/4079782599] conn(0x7f7b5c0692a0 msgr2=0x7f7b5c06b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:07:23.664 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:07:23.665+0000 7f7b72f49700 1 --2- 172.21.15.162:0/2663475443 >> [v2:172.21.15.162:6800/4079782599,v1:172.21.15.162:6801/4079782599] conn(0x7f7b5c0692a0 0x7f7b5c06b750 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f7b58004890 tx=0x7f7b58009e30 comp rx=0 tx=0).stop 2024-04-27T04:07:23.665 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:07:23.665+0000 7f7b72f49700 1 -- 172.21.15.162:0/2663475443 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7b6c101d00 msgr2=0x7f7b6c113b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:07:23.665 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:07:23.665+0000 7f7b72f49700 1 --2- 172.21.15.162:0/2663475443 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7b6c101d00 0x7f7b6c113b70 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f7b6000c830 tx=0x7f7b60004470 comp rx=0 tx=0).stop 2024-04-27T04:07:23.665 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:07:23.666+0000 7f7b72f49700 1 -- 172.21.15.162:0/2663475443 shutdown_connections 2024-04-27T04:07:23.665 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:07:23.666+0000 7f7b72f49700 1 --2- 172.21.15.162:0/2663475443 >> [v2:172.21.15.162:6800/4079782599,v1:172.21.15.162:6801/4079782599] conn(0x7f7b5c0692a0 0x7f7b5c06b750 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:07:23.665 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:07:23.666+0000 7f7b72f49700 1 --2- 172.21.15.162:0/2663475443 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7b6c101d00 0x7f7b6c113b70 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:07:23.665 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:07:23.666+0000 7f7b72f49700 1 --2- 172.21.15.162:0/2663475443 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7b6c1026a0 0x7f7b6c1140b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:07:23.665 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:07:23.666+0000 7f7b72f49700 1 -- 172.21.15.162:0/2663475443 >> 172.21.15.162:0/2663475443 conn(0x7f7b6c0f78d0 msgr2=0x7f7b6c065e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:07:23.665 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:07:23.666+0000 7f7b72f49700 1 -- 172.21.15.162:0/2663475443 shutdown_connections 2024-04-27T04:07:23.665 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:07:23.666+0000 7f7b72f49700 1 -- 172.21.15.162:0/2663475443 wait complete. 2024-04-27T04:07:24.172 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:07:23 smithi181 ceph-mon[33774]: pgmap v187: 65 pgs: 65 active+clean; 579 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 5.4 KiB/s rd, 0 B/s wr, 6 op/s 2024-04-27T04:07:24.173 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:07:23 smithi181 ceph-mon[33774]: from='client.? 172.21.15.162:0/2663475443' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-04-27T04:07:24.273 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:07:23 smithi162 ceph-mon[26841]: pgmap v187: 65 pgs: 65 active+clean; 579 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 5.4 KiB/s rd, 0 B/s wr, 6 op/s 2024-04-27T04:07:24.273 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:07:23 smithi162 ceph-mon[26841]: from='client.? 172.21.15.162:0/2663475443' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-04-27T04:07:24.301 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-04-27T04:07:24.301 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-27T04:07:24.301 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi162.front.sepia.ceph.com 2024-04-27T04:07:24.301 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-27T04:07:24.301 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-27T04:07:24.301 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-04-27T04:07:24.301 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-04-27T04:07:24.301 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-27T04:07:24.301 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-27T04:07:24.301 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-27T04:07:24.302 DEBUG:teuthology.orchestra.run.smithi162:> ip addr 2024-04-27T04:07:24.319 INFO:teuthology.orchestra.run.smithi162.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-27T04:07:24.319 INFO:teuthology.orchestra.run.smithi162.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-27T04:07:24.319 INFO:teuthology.orchestra.run.smithi162.stdout: inet 127.0.0.1/8 scope host lo 2024-04-27T04:07:24.319 INFO:teuthology.orchestra.run.smithi162.stdout: valid_lft forever preferred_lft forever 2024-04-27T04:07:24.319 INFO:teuthology.orchestra.run.smithi162.stdout: inet6 ::1/128 scope host 2024-04-27T04:07:24.319 INFO:teuthology.orchestra.run.smithi162.stdout: valid_lft forever preferred_lft forever 2024-04-27T04:07:24.319 INFO:teuthology.orchestra.run.smithi162.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-27T04:07:24.319 INFO:teuthology.orchestra.run.smithi162.stdout: link/ether 0c:c4:7a:d9:37:f8 brd ff:ff:ff:ff:ff:ff 2024-04-27T04:07:24.319 INFO:teuthology.orchestra.run.smithi162.stdout: altname enp7s0f0 2024-04-27T04:07:24.319 INFO:teuthology.orchestra.run.smithi162.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-27T04:07:24.319 INFO:teuthology.orchestra.run.smithi162.stdout: link/ether 0c:c4:7a:d9:37:f9 brd ff:ff:ff:ff:ff:ff 2024-04-27T04:07:24.319 INFO:teuthology.orchestra.run.smithi162.stdout: altname enp7s0f1 2024-04-27T04:07:24.320 INFO:teuthology.orchestra.run.smithi162.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-27T04:07:24.320 INFO:teuthology.orchestra.run.smithi162.stdout: link/ether 0c:c4:7a:88:7f:40 brd ff:ff:ff:ff:ff:ff 2024-04-27T04:07:24.320 INFO:teuthology.orchestra.run.smithi162.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-27T04:07:24.320 INFO:teuthology.orchestra.run.smithi162.stdout: link/ether 0c:c4:7a:88:7f:41 brd ff:ff:ff:ff:ff:ff 2024-04-27T04:07:24.320 INFO:teuthology.orchestra.run.smithi162.stdout: inet 172.21.15.162/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-04-27T04:07:24.320 INFO:teuthology.orchestra.run.smithi162.stdout: valid_lft 42190sec preferred_lft 42190sec 2024-04-27T04:07:24.320 INFO:teuthology.orchestra.run.smithi162.stdout: inet6 fe80::b637:b830:5199:1015/64 scope link noprefixroute 2024-04-27T04:07:24.320 INFO:teuthology.orchestra.run.smithi162.stdout: valid_lft forever preferred_lft forever 2024-04-27T04:07:24.320 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-04-27T04:07:24.321 DEBUG:teuthology.orchestra.run.smithi162:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-27T04:07:24.321 DEBUG:teuthology.orchestra.run.smithi162:> set -e 2024-04-27T04:07:24.321 DEBUG:teuthology.orchestra.run.smithi162:> sudo ip link add name ceph-brx type bridge 2024-04-27T04:07:24.321 DEBUG:teuthology.orchestra.run.smithi162:> sudo ip addr flush dev ceph-brx 2024-04-27T04:07:24.321 DEBUG:teuthology.orchestra.run.smithi162:> sudo ip link set ceph-brx up 2024-04-27T04:07:24.321 DEBUG:teuthology.orchestra.run.smithi162:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-04-27T04:07:24.321 DEBUG:teuthology.orchestra.run.smithi162:> ') 2024-04-27T04:07:24.409 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:07:24 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-27T04:07:24.496 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:07:24 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-27T04:07:24.503 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-27T04:07:24.503 DEBUG:teuthology.orchestra.run.smithi162:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-27T04:07:24.527 INFO:teuthology.orchestra.run.smithi162.stdout:1 2024-04-27T04:07:24.529 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-27T04:07:24.529 DEBUG:teuthology.orchestra.run.smithi162:> route 2024-04-27T04:07:24.596 INFO:teuthology.orchestra.run.smithi162.stdout:Kernel IP routing table 2024-04-27T04:07:24.596 INFO:teuthology.orchestra.run.smithi162.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-27T04:07:24.597 INFO:teuthology.orchestra.run.smithi162.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-04-27T04:07:24.597 INFO:teuthology.orchestra.run.smithi162.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-04-27T04:07:24.597 INFO:teuthology.orchestra.run.smithi162.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-27T04:07:24.597 DEBUG:teuthology.orchestra.run.smithi162:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-27T04:07:24.597 DEBUG:teuthology.orchestra.run.smithi162:> set -e 2024-04-27T04:07:24.598 DEBUG:teuthology.orchestra.run.smithi162:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-04-27T04:07:24.598 DEBUG:teuthology.orchestra.run.smithi162:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-04-27T04:07:24.598 DEBUG:teuthology.orchestra.run.smithi162:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-04-27T04:07:24.598 DEBUG:teuthology.orchestra.run.smithi162:> ') 2024-04-27T04:07:24.684 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:07:24 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-27T04:07:24.863 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:07:24 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-27T04:07:24.869 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-27T04:07:24.869 DEBUG:teuthology.orchestra.run.smithi162:> ip netns list 2024-04-27T04:07:24.884 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-27T04:07:24.884 DEBUG:teuthology.orchestra.run.smithi162:> ip netns list-id 2024-04-27T04:07:24.939 DEBUG:teuthology.orchestra.run.smithi162:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-27T04:07:24.939 DEBUG:teuthology.orchestra.run.smithi162:> set -e 2024-04-27T04:07:24.939 DEBUG:teuthology.orchestra.run.smithi162:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-27T04:07:24.939 DEBUG:teuthology.orchestra.run.smithi162:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.0 0 2024-04-27T04:07:24.939 DEBUG:teuthology.orchestra.run.smithi162:> ') 2024-04-27T04:07:25.026 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:07:25 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-27T04:07:25.048 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:07:25 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-27T04:07:25.054 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.0' with 192.168.0.1/16 2024-04-27T04:07:25.054 DEBUG:teuthology.orchestra.run.smithi162:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-27T04:07:25.054 DEBUG:teuthology.orchestra.run.smithi162:> set -e 2024-04-27T04:07:25.054 DEBUG:teuthology.orchestra.run.smithi162:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.0 type veth peer name brx.0 2024-04-27T04:07:25.054 DEBUG:teuthology.orchestra.run.smithi162:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-04-27T04:07:25.055 DEBUG:teuthology.orchestra.run.smithi162:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set veth0 up 2024-04-27T04:07:25.055 DEBUG:teuthology.orchestra.run.smithi162:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set lo up 2024-04-27T04:07:25.055 DEBUG:teuthology.orchestra.run.smithi162:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip route add default via 192.168.255.254 2024-04-27T04:07:25.055 DEBUG:teuthology.orchestra.run.smithi162:> ') 2024-04-27T04:07:25.102 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:07:25 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-27T04:07:25.169 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:07:25 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-27T04:07:25.175 DEBUG:teuthology.orchestra.run.smithi162:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-27T04:07:25.175 DEBUG:teuthology.orchestra.run.smithi162:> set -e 2024-04-27T04:07:25.175 DEBUG:teuthology.orchestra.run.smithi162:> sudo ip link set brx.0 up 2024-04-27T04:07:25.175 DEBUG:teuthology.orchestra.run.smithi162:> sudo ip link set dev brx.0 master ceph-brx 2024-04-27T04:07:25.175 DEBUG:teuthology.orchestra.run.smithi162:> ') 2024-04-27T04:07:25.222 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:07:25 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-27T04:07:25.243 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:07:25 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-27T04:07:25.249 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2024-04-27T04:07:25.249 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-27T04:07:25.249 DEBUG:teuthology.orchestra.run.smithi162:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-27T04:07:25.263 INFO:teuthology.orchestra.run.smithi162.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-27T04:07:25.263 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-27T04:07:25.263 DEBUG:teuthology.orchestra.run.smithi162:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-27T04:07:25.318 DEBUG:teuthology.orchestra.run.smithi162:> sudo modprobe fuse 2024-04-27T04:07:25.385 DEBUG:teuthology.orchestra.run.smithi162:> cat /proc/self/mounts | awk '{print $2}' 2024-04-27T04:07:25.440 INFO:teuthology.orchestra.run.smithi162.stdout:/proc 2024-04-27T04:07:25.441 INFO:teuthology.orchestra.run.smithi162.stdout:/sys 2024-04-27T04:07:25.441 INFO:teuthology.orchestra.run.smithi162.stdout:/dev 2024-04-27T04:07:25.441 INFO:teuthology.orchestra.run.smithi162.stdout:/sys/kernel/security 2024-04-27T04:07:25.441 INFO:teuthology.orchestra.run.smithi162.stdout:/dev/shm 2024-04-27T04:07:25.441 INFO:teuthology.orchestra.run.smithi162.stdout:/dev/pts 2024-04-27T04:07:25.441 INFO:teuthology.orchestra.run.smithi162.stdout:/run 2024-04-27T04:07:25.441 INFO:teuthology.orchestra.run.smithi162.stdout:/sys/fs/cgroup 2024-04-27T04:07:25.441 INFO:teuthology.orchestra.run.smithi162.stdout:/sys/fs/pstore 2024-04-27T04:07:25.441 INFO:teuthology.orchestra.run.smithi162.stdout:/sys/fs/bpf 2024-04-27T04:07:25.441 INFO:teuthology.orchestra.run.smithi162.stdout:/ 2024-04-27T04:07:25.441 INFO:teuthology.orchestra.run.smithi162.stdout:/sys/fs/selinux 2024-04-27T04:07:25.442 INFO:teuthology.orchestra.run.smithi162.stdout:/proc/sys/fs/binfmt_misc 2024-04-27T04:07:25.442 INFO:teuthology.orchestra.run.smithi162.stdout:/dev/hugepages 2024-04-27T04:07:25.442 INFO:teuthology.orchestra.run.smithi162.stdout:/dev/mqueue 2024-04-27T04:07:25.442 INFO:teuthology.orchestra.run.smithi162.stdout:/sys/kernel/debug 2024-04-27T04:07:25.442 INFO:teuthology.orchestra.run.smithi162.stdout:/sys/kernel/tracing 2024-04-27T04:07:25.442 INFO:teuthology.orchestra.run.smithi162.stdout:/run/credentials/systemd-sysusers.service 2024-04-27T04:07:25.442 INFO:teuthology.orchestra.run.smithi162.stdout:/sys/fs/fuse/connections 2024-04-27T04:07:25.442 INFO:teuthology.orchestra.run.smithi162.stdout:/sys/kernel/config 2024-04-27T04:07:25.442 INFO:teuthology.orchestra.run.smithi162.stdout:/run/credentials/systemd-sysctl.service 2024-04-27T04:07:25.442 INFO:teuthology.orchestra.run.smithi162.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-04-27T04:07:25.442 INFO:teuthology.orchestra.run.smithi162.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-04-27T04:07:25.442 INFO:teuthology.orchestra.run.smithi162.stdout:/var/lib/nfs/rpc_pipefs 2024-04-27T04:07:25.442 INFO:teuthology.orchestra.run.smithi162.stdout:/run/user/1000 2024-04-27T04:07:25.442 INFO:teuthology.orchestra.run.smithi162.stdout:/var/lib/ceph 2024-04-27T04:07:25.443 INFO:teuthology.orchestra.run.smithi162.stdout:/var/lib/containers/storage/overlay 2024-04-27T04:07:25.443 INFO:teuthology.orchestra.run.smithi162.stdout:/var/lib/containers/storage/overlay/0674fd5f25a0eff15b38f3d6b16289eca30c4e43c203c0534e3660b649e3b543/merged 2024-04-27T04:07:25.443 INFO:teuthology.orchestra.run.smithi162.stdout:/var/lib/containers/storage/overlay/dccecda601d100b7076a92e9efdf9374779fb1dc90ae27904f42af41dbd7f631/merged 2024-04-27T04:07:25.443 INFO:teuthology.orchestra.run.smithi162.stdout:/run/user/0 2024-04-27T04:07:25.443 INFO:teuthology.orchestra.run.smithi162.stdout:/proc/sys/fs/binfmt_misc 2024-04-27T04:07:25.443 INFO:teuthology.orchestra.run.smithi162.stdout:/var/lib/containers/storage/overlay/33d6eee449a9397965b23ff3f9f2d49b07ddba89cdf8f8c85f3c518b43c951c4/merged 2024-04-27T04:07:25.443 INFO:teuthology.orchestra.run.smithi162.stdout:/var/lib/containers/storage/overlay/88387cc764788714303b50128c6e4f6d2c5ac748d3f680bb42a3fe21dce28c73/merged 2024-04-27T04:07:25.443 INFO:teuthology.orchestra.run.smithi162.stdout:/var/lib/containers/storage/overlay/5c91f3cdb2d0b0add7c4805eb544566366020a1560b9341d67e0173e1c11ff24/merged 2024-04-27T04:07:25.443 INFO:teuthology.orchestra.run.smithi162.stdout:/var/lib/containers/storage/overlay/1e5e948a443f46ad8ce7411f5ac990fbcb227bc66b77007b90845ba3abfb8804/merged 2024-04-27T04:07:25.443 INFO:teuthology.orchestra.run.smithi162.stdout:/var/lib/containers/storage/overlay/2002b84357c57fa3b3a11448199a020dd6ac9c3537289122571f5ce79a2f9091/merged 2024-04-27T04:07:25.443 INFO:teuthology.orchestra.run.smithi162.stdout:/var/lib/containers/storage/overlay/1605786ea27e934785c7a4cdfbd7bc34f9f455609393e91fd9aeb473d05cab4d/merged 2024-04-27T04:07:25.443 INFO:teuthology.orchestra.run.smithi162.stdout:/var/lib/containers/storage/overlay/fec4a0ddd6cf6d5085414098b4793ddaafb3a8d030992b8aa4fc6b83d118d57a/merged 2024-04-27T04:07:25.443 INFO:teuthology.orchestra.run.smithi162.stdout:/var/lib/containers/storage/overlay/73a8d5e4ef4a7d3417ebe8f23c54c2e417dc3ffad34e44a289bc41fdca4b1a19/merged 2024-04-27T04:07:25.443 INFO:teuthology.orchestra.run.smithi162.stdout:/var/lib/containers/storage/overlay/599be7996aff574ce32a6ff6b19314a9b4cc7705f90fd84e0bb4e3059b214199/merged 2024-04-27T04:07:25.443 INFO:teuthology.orchestra.run.smithi162.stdout:/var/lib/containers/storage/overlay/f92cfe6843111445ce8186925e08ede0eec68a19bc3c00dd50511e5037f88cc9/merged 2024-04-27T04:07:25.444 INFO:teuthology.orchestra.run.smithi162.stdout:/run/netns 2024-04-27T04:07:25.444 INFO:teuthology.orchestra.run.smithi162.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-27T04:07:25.444 INFO:teuthology.orchestra.run.smithi162.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-27T04:07:25.444 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-27T04:07:25.444 DEBUG:teuthology.orchestra.run.smithi162:> ls /sys/fs/fuse/connections 2024-04-27T04:07:25.496 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-04-27T04:07:25.497 DEBUG:teuthology.orchestra.run.smithi162:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2024-04-27T04:07:25.539 DEBUG:teuthology.orchestra.run.smithi162:> sudo modprobe fuse 2024-04-27T04:07:25.562 DEBUG:teuthology.orchestra.run.smithi162:> cat /proc/self/mounts | awk '{print $2}' 2024-04-27T04:07:25.618 INFO:teuthology.orchestra.run.smithi162.stdout:/proc 2024-04-27T04:07:25.618 INFO:teuthology.orchestra.run.smithi162.stdout:/sys 2024-04-27T04:07:25.618 INFO:teuthology.orchestra.run.smithi162.stdout:/dev 2024-04-27T04:07:25.618 INFO:teuthology.orchestra.run.smithi162.stdout:/sys/kernel/security 2024-04-27T04:07:25.618 INFO:teuthology.orchestra.run.smithi162.stdout:/dev/shm 2024-04-27T04:07:25.618 INFO:teuthology.orchestra.run.smithi162.stdout:/dev/pts 2024-04-27T04:07:25.618 INFO:teuthology.orchestra.run.smithi162.stdout:/run 2024-04-27T04:07:25.618 INFO:teuthology.orchestra.run.smithi162.stdout:/sys/fs/cgroup 2024-04-27T04:07:25.618 INFO:teuthology.orchestra.run.smithi162.stdout:/sys/fs/pstore 2024-04-27T04:07:25.619 INFO:teuthology.orchestra.run.smithi162.stdout:/sys/fs/bpf 2024-04-27T04:07:25.619 INFO:teuthology.orchestra.run.smithi162.stdout:/ 2024-04-27T04:07:25.619 INFO:teuthology.orchestra.run.smithi162.stdout:/sys/fs/selinux 2024-04-27T04:07:25.619 INFO:teuthology.orchestra.run.smithi162.stdout:/proc/sys/fs/binfmt_misc 2024-04-27T04:07:25.619 INFO:teuthology.orchestra.run.smithi162.stdout:/dev/hugepages 2024-04-27T04:07:25.619 INFO:teuthology.orchestra.run.smithi162.stdout:/dev/mqueue 2024-04-27T04:07:25.619 INFO:teuthology.orchestra.run.smithi162.stdout:/sys/kernel/debug 2024-04-27T04:07:25.619 INFO:teuthology.orchestra.run.smithi162.stdout:/sys/kernel/tracing 2024-04-27T04:07:25.619 INFO:teuthology.orchestra.run.smithi162.stdout:/run/credentials/systemd-sysusers.service 2024-04-27T04:07:25.619 INFO:teuthology.orchestra.run.smithi162.stdout:/sys/fs/fuse/connections 2024-04-27T04:07:25.619 INFO:teuthology.orchestra.run.smithi162.stdout:/sys/kernel/config 2024-04-27T04:07:25.619 INFO:teuthology.orchestra.run.smithi162.stdout:/run/credentials/systemd-sysctl.service 2024-04-27T04:07:25.619 INFO:teuthology.orchestra.run.smithi162.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-04-27T04:07:25.619 INFO:teuthology.orchestra.run.smithi162.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-04-27T04:07:25.619 INFO:teuthology.orchestra.run.smithi162.stdout:/var/lib/nfs/rpc_pipefs 2024-04-27T04:07:25.620 INFO:teuthology.orchestra.run.smithi162.stdout:/run/user/1000 2024-04-27T04:07:25.620 INFO:teuthology.orchestra.run.smithi162.stdout:/var/lib/ceph 2024-04-27T04:07:25.620 INFO:teuthology.orchestra.run.smithi162.stdout:/var/lib/containers/storage/overlay 2024-04-27T04:07:25.620 INFO:teuthology.orchestra.run.smithi162.stdout:/var/lib/containers/storage/overlay/0674fd5f25a0eff15b38f3d6b16289eca30c4e43c203c0534e3660b649e3b543/merged 2024-04-27T04:07:25.620 INFO:teuthology.orchestra.run.smithi162.stdout:/var/lib/containers/storage/overlay/dccecda601d100b7076a92e9efdf9374779fb1dc90ae27904f42af41dbd7f631/merged 2024-04-27T04:07:25.620 INFO:teuthology.orchestra.run.smithi162.stdout:/run/user/0 2024-04-27T04:07:25.620 INFO:teuthology.orchestra.run.smithi162.stdout:/proc/sys/fs/binfmt_misc 2024-04-27T04:07:25.620 INFO:teuthology.orchestra.run.smithi162.stdout:/var/lib/containers/storage/overlay/33d6eee449a9397965b23ff3f9f2d49b07ddba89cdf8f8c85f3c518b43c951c4/merged 2024-04-27T04:07:25.620 INFO:teuthology.orchestra.run.smithi162.stdout:/var/lib/containers/storage/overlay/88387cc764788714303b50128c6e4f6d2c5ac748d3f680bb42a3fe21dce28c73/merged 2024-04-27T04:07:25.620 INFO:teuthology.orchestra.run.smithi162.stdout:/var/lib/containers/storage/overlay/5c91f3cdb2d0b0add7c4805eb544566366020a1560b9341d67e0173e1c11ff24/merged 2024-04-27T04:07:25.620 INFO:teuthology.orchestra.run.smithi162.stdout:/var/lib/containers/storage/overlay/1e5e948a443f46ad8ce7411f5ac990fbcb227bc66b77007b90845ba3abfb8804/merged 2024-04-27T04:07:25.620 INFO:teuthology.orchestra.run.smithi162.stdout:/var/lib/containers/storage/overlay/2002b84357c57fa3b3a11448199a020dd6ac9c3537289122571f5ce79a2f9091/merged 2024-04-27T04:07:25.620 INFO:teuthology.orchestra.run.smithi162.stdout:/var/lib/containers/storage/overlay/1605786ea27e934785c7a4cdfbd7bc34f9f455609393e91fd9aeb473d05cab4d/merged 2024-04-27T04:07:25.620 INFO:teuthology.orchestra.run.smithi162.stdout:/var/lib/containers/storage/overlay/fec4a0ddd6cf6d5085414098b4793ddaafb3a8d030992b8aa4fc6b83d118d57a/merged 2024-04-27T04:07:25.621 INFO:teuthology.orchestra.run.smithi162.stdout:/var/lib/containers/storage/overlay/73a8d5e4ef4a7d3417ebe8f23c54c2e417dc3ffad34e44a289bc41fdca4b1a19/merged 2024-04-27T04:07:25.621 INFO:teuthology.orchestra.run.smithi162.stdout:/var/lib/containers/storage/overlay/599be7996aff574ce32a6ff6b19314a9b4cc7705f90fd84e0bb4e3059b214199/merged 2024-04-27T04:07:25.621 INFO:teuthology.orchestra.run.smithi162.stdout:/var/lib/containers/storage/overlay/f92cfe6843111445ce8186925e08ede0eec68a19bc3c00dd50511e5037f88cc9/merged 2024-04-27T04:07:25.621 INFO:teuthology.orchestra.run.smithi162.stdout:/run/netns 2024-04-27T04:07:25.621 INFO:teuthology.orchestra.run.smithi162.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-27T04:07:25.621 INFO:teuthology.orchestra.run.smithi162.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-27T04:07:25.621 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-27T04:07:25.621 DEBUG:teuthology.orchestra.run.smithi162:> ls /sys/fs/fuse/connections 2024-04-27T04:07:25.638 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi162.stderr:2024-04-27T04:07:25.639+0000 7f72050c3300 -1 init, newargv = 0x561bbe8ac770 newargc=15 2024-04-27T04:07:25.639 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi162.stderr:ceph-fuse[74353]: starting ceph client 2024-04-27T04:07:25.643 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi162.stderr:ceph-fuse[74353]: starting fuse 2024-04-27T04:07:25.653 INFO:teuthology.orchestra.run.smithi162.stdout:70 2024-04-27T04:07:25.653 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [70] 2024-04-27T04:07:25.654 DEBUG:teuthology.orchestra.run.smithi162:> sudo stdin-killer -- python3 -c ' 2024-04-27T04:07:25.654 DEBUG:teuthology.orchestra.run.smithi162:> import glob 2024-04-27T04:07:25.654 DEBUG:teuthology.orchestra.run.smithi162:> import re 2024-04-27T04:07:25.654 DEBUG:teuthology.orchestra.run.smithi162:> import os 2024-04-27T04:07:25.654 DEBUG:teuthology.orchestra.run.smithi162:> import subprocess 2024-04-27T04:07:25.654 DEBUG:teuthology.orchestra.run.smithi162:> 2024-04-27T04:07:25.654 DEBUG:teuthology.orchestra.run.smithi162:> def _find_admin_socket(client_name): 2024-04-27T04:07:25.654 DEBUG:teuthology.orchestra.run.smithi162:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-04-27T04:07:25.654 DEBUG:teuthology.orchestra.run.smithi162:> files = glob.glob(asok_path) 2024-04-27T04:07:25.654 DEBUG:teuthology.orchestra.run.smithi162:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-04-27T04:07:25.654 DEBUG:teuthology.orchestra.run.smithi162:> 2024-04-27T04:07:25.654 DEBUG:teuthology.orchestra.run.smithi162:> # Given a non-glob path, it better be there 2024-04-27T04:07:25.654 DEBUG:teuthology.orchestra.run.smithi162:> if "*" not in asok_path: 2024-04-27T04:07:25.654 DEBUG:teuthology.orchestra.run.smithi162:> assert(len(files) == 1) 2024-04-27T04:07:25.655 DEBUG:teuthology.orchestra.run.smithi162:> return files[0] 2024-04-27T04:07:25.655 DEBUG:teuthology.orchestra.run.smithi162:> 2024-04-27T04:07:25.655 DEBUG:teuthology.orchestra.run.smithi162:> for f in files: 2024-04-27T04:07:25.655 DEBUG:teuthology.orchestra.run.smithi162:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-04-27T04:07:25.655 DEBUG:teuthology.orchestra.run.smithi162:> if os.path.exists("/proc/{0}".format(pid)): 2024-04-27T04:07:25.655 DEBUG:teuthology.orchestra.run.smithi162:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-04-27T04:07:25.655 DEBUG:teuthology.orchestra.run.smithi162:> contents = proc_f.read() 2024-04-27T04:07:25.655 DEBUG:teuthology.orchestra.run.smithi162:> if mountpoint in contents: 2024-04-27T04:07:25.655 DEBUG:teuthology.orchestra.run.smithi162:> return f 2024-04-27T04:07:25.655 DEBUG:teuthology.orchestra.run.smithi162:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-04-27T04:07:25.655 DEBUG:teuthology.orchestra.run.smithi162:> 2024-04-27T04:07:25.655 DEBUG:teuthology.orchestra.run.smithi162:> print(_find_admin_socket("client.0")) 2024-04-27T04:07:25.655 DEBUG:teuthology.orchestra.run.smithi162:> ' 2024-04-27T04:07:25.772 INFO:teuthology.orchestra.run.smithi162.stdout:/var/run/ceph/ceph-client.0.74353.asok 2024-04-27T04:07:25.777 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:07:25 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-27T04:07:25.782 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.74353.asok 2024-04-27T04:07:25.783 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-27T04:07:25.783 DEBUG:teuthology.orchestra.run.smithi162:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.74353.asok status 2024-04-27T04:07:25.912 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-04-27T04:07:25.913 INFO:teuthology.orchestra.run.smithi162.stdout: "metadata": { 2024-04-27T04:07:25.913 INFO:teuthology.orchestra.run.smithi162.stdout: "ceph_sha1": "2be3efa97a7972f8e925bd899b474329979b74d6", 2024-04-27T04:07:25.913 INFO:teuthology.orchestra.run.smithi162.stdout: "ceph_version": "ceph version 17.2.7-907-g2be3efa9 (2be3efa97a7972f8e925bd899b474329979b74d6) quincy (stable)", 2024-04-27T04:07:25.913 INFO:teuthology.orchestra.run.smithi162.stdout: "entity_id": "0", 2024-04-27T04:07:25.913 INFO:teuthology.orchestra.run.smithi162.stdout: "hostname": "smithi162", 2024-04-27T04:07:25.913 INFO:teuthology.orchestra.run.smithi162.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2024-04-27T04:07:25.913 INFO:teuthology.orchestra.run.smithi162.stdout: "pid": "74353", 2024-04-27T04:07:25.914 INFO:teuthology.orchestra.run.smithi162.stdout: "root": "/" 2024-04-27T04:07:25.914 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-04-27T04:07:25.914 INFO:teuthology.orchestra.run.smithi162.stdout: "dentry_count": 0, 2024-04-27T04:07:25.914 INFO:teuthology.orchestra.run.smithi162.stdout: "dentry_pinned_count": 0, 2024-04-27T04:07:25.914 INFO:teuthology.orchestra.run.smithi162.stdout: "id": 14528, 2024-04-27T04:07:25.914 INFO:teuthology.orchestra.run.smithi162.stdout: "inst": { 2024-04-27T04:07:25.914 INFO:teuthology.orchestra.run.smithi162.stdout: "name": { 2024-04-27T04:07:25.914 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "client", 2024-04-27T04:07:25.914 INFO:teuthology.orchestra.run.smithi162.stdout: "num": 14528 2024-04-27T04:07:25.914 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-04-27T04:07:25.914 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": { 2024-04-27T04:07:25.915 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v1", 2024-04-27T04:07:25.915 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "192.168.0.1:0", 2024-04-27T04:07:25.915 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 91994784 2024-04-27T04:07:25.915 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-04-27T04:07:25.915 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-04-27T04:07:25.915 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": { 2024-04-27T04:07:25.915 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v1", 2024-04-27T04:07:25.915 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "192.168.0.1:0", 2024-04-27T04:07:25.915 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 91994784 2024-04-27T04:07:25.915 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-04-27T04:07:25.915 INFO:teuthology.orchestra.run.smithi162.stdout: "inst_str": "client.14528 192.168.0.1:0/91994784", 2024-04-27T04:07:25.915 INFO:teuthology.orchestra.run.smithi162.stdout: "addr_str": "192.168.0.1:0/91994784", 2024-04-27T04:07:25.915 INFO:teuthology.orchestra.run.smithi162.stdout: "inode_count": 1, 2024-04-27T04:07:25.915 INFO:teuthology.orchestra.run.smithi162.stdout: "mds_epoch": 15, 2024-04-27T04:07:25.916 INFO:teuthology.orchestra.run.smithi162.stdout: "osd_epoch": 46, 2024-04-27T04:07:25.916 INFO:teuthology.orchestra.run.smithi162.stdout: "osd_epoch_barrier": 0, 2024-04-27T04:07:25.916 INFO:teuthology.orchestra.run.smithi162.stdout: "blocklisted": false, 2024-04-27T04:07:25.916 INFO:teuthology.orchestra.run.smithi162.stdout: "fs_name": "cephfs" 2024-04-27T04:07:25.916 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-04-27T04:07:25.921 DEBUG:teuthology.orchestra.run.smithi162:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell --fsid 7f9750d6-044a-11ef-bc93-c7b262605968 -- ceph fs ls 2024-04-27T04:07:26.137 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:07:25 smithi162 ceph-mon[26841]: pgmap v188: 65 pgs: 65 active+clean; 579 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 3.0 KiB/s rd, 0 B/s wr, 3 op/s 2024-04-27T04:07:26.175 INFO:teuthology.orchestra.run.smithi162.stderr:Inferring config /var/lib/ceph/7f9750d6-044a-11ef-bc93-c7b262605968/mon.smithi162/config 2024-04-27T04:07:26.422 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:07:25 smithi181 ceph-mon[33774]: pgmap v188: 65 pgs: 65 active+clean; 579 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 3.0 KiB/s rd, 0 B/s wr, 3 op/s 2024-04-27T04:07:27.902 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:07:27.901+0000 7f4f07c95700 1 -- 172.21.15.162:0/1472151888 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4f00101d00 msgr2=0x7f4f001020d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:07:27.902 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:07:27.901+0000 7f4f07c95700 1 --2- 172.21.15.162:0/1472151888 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4f00101d00 0x7f4f001020d0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f4efc004940 tx=0x7f4efc009e30 comp rx=0 tx=0).stop 2024-04-27T04:07:27.902 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:07:27.901+0000 7f4f07c95700 1 -- 172.21.15.162:0/1472151888 shutdown_connections 2024-04-27T04:07:27.902 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:07:27.901+0000 7f4f07c95700 1 --2- 172.21.15.162:0/1472151888 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4f001026a0 0x7f4f000fbd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:07:27.902 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:07:27.901+0000 7f4f07c95700 1 --2- 172.21.15.162:0/1472151888 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4f00101d00 0x7f4f001020d0 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:07:27.902 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:07:27.901+0000 7f4f07c95700 1 -- 172.21.15.162:0/1472151888 >> 172.21.15.162:0/1472151888 conn(0x7f4f000f7870 msgr2=0x7f4f000f9c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:07:27.902 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:07:27.901+0000 7f4f07c95700 1 -- 172.21.15.162:0/1472151888 shutdown_connections 2024-04-27T04:07:27.902 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:07:27.901+0000 7f4f07c95700 1 -- 172.21.15.162:0/1472151888 wait complete. 2024-04-27T04:07:27.902 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:07:27.901+0000 7f4f07c95700 1 Processor -- start 2024-04-27T04:07:27.902 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:07:27.901+0000 7f4f07c95700 1 -- start start 2024-04-27T04:07:27.902 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:07:27.902+0000 7f4f07c95700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4f00101d00 0x7f4f00113980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:07:27.902 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:07:27.902+0000 7f4f07c95700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4f001026a0 0x7f4f00113ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:07:27.903 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:07:27.902+0000 7f4f07c95700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f00114510 con 0x7f4f00101d00 2024-04-27T04:07:27.903 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:07:27.902+0000 7f4f07c95700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f0010db10 con 0x7f4f001026a0 2024-04-27T04:07:27.903 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:07:27.902+0000 7f4f05a31700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4f00101d00 0x7f4f00113980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:07:27.903 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:07:27.902+0000 7f4f05a31700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4f00101d00 0x7f4f00113980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:38548/0 (socket says 172.21.15.162:38548) 2024-04-27T04:07:27.903 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:07:27.902+0000 7f4f05a31700 1 -- 172.21.15.162:0/2878933601 learned_addr learned my addr 172.21.15.162:0/2878933601 (peer_addr_for_me v2:172.21.15.162:0/0) 2024-04-27T04:07:27.903 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:07:27.902+0000 7f4f05230700 1 --2- 172.21.15.162:0/2878933601 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4f001026a0 0x7f4f00113ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:07:27.903 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:07:27.902+0000 7f4f05a31700 1 -- 172.21.15.162:0/2878933601 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4f001026a0 msgr2=0x7f4f00113ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:07:27.903 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:07:27.902+0000 7f4f05a31700 1 --2- 172.21.15.162:0/2878933601 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4f001026a0 0x7f4f00113ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:07:27.903 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:07:27.902+0000 7f4f05a31700 1 -- 172.21.15.162:0/2878933601 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4efc005040 con 0x7f4f00101d00 2024-04-27T04:07:27.903 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:07:27.902+0000 7f4f05a31700 1 --2- 172.21.15.162:0/2878933601 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4f00101d00 0x7f4f00113980 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f4efc00b3b0 tx=0x7f4efc009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:07:27.903 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:07:27.902+0000 7f4ef67fc700 1 -- 172.21.15.162:0/2878933601 <== mon.0 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4efc00e040 con 0x7f4f00101d00 2024-04-27T04:07:27.903 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:07:27.902+0000 7f4f07c95700 1 -- 172.21.15.162:0/2878933601 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4f0010dd90 con 0x7f4f00101d00 2024-04-27T04:07:27.903 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:07:27.902+0000 7f4ef67fc700 1 -- 172.21.15.162:0/2878933601 <== mon.0 v2:172.21.15.162:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f4efc00bbc0 con 0x7f4f00101d00 2024-04-27T04:07:27.903 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:07:27.903+0000 7f4ef67fc700 1 -- 172.21.15.162:0/2878933601 <== mon.0 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4efc01c3f0 con 0x7f4f00101d00 2024-04-27T04:07:27.904 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:07:27.903+0000 7f4f07c95700 1 -- 172.21.15.162:0/2878933601 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4f0010e350 con 0x7f4f00101d00 2024-04-27T04:07:27.904 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:07:27.903+0000 7f4f07c95700 1 -- 172.21.15.162:0/2878933601 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4f000fd450 con 0x7f4f00101d00 2024-04-27T04:07:27.904 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:07:27.904+0000 7f4ef67fc700 1 -- 172.21.15.162:0/2878933601 <== mon.0 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 19) v1 ==== 86631+0+0 (secure 0 0 0) 0x7f4efc00bd30 con 0x7f4f00101d00 2024-04-27T04:07:27.904 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:07:27.904+0000 7f4ef67fc700 1 --2- 172.21.15.162:0/2878933601 >> [v2:172.21.15.162:6800/4079782599,v1:172.21.15.162:6801/4079782599] conn(0x7f4eec0692a0 0x7f4eec06b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:07:27.905 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:07:27.904+0000 7f4ef67fc700 1 -- 172.21.15.162:0/2878933601 <== mon.0 v2:172.21.15.162:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5550+0+0 (secure 0 0 0) 0x7f4efc088480 con 0x7f4f00101d00 2024-04-27T04:07:27.905 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:07:27.904+0000 7f4f05230700 1 --2- 172.21.15.162:0/2878933601 >> [v2:172.21.15.162:6800/4079782599,v1:172.21.15.162:6801/4079782599] conn(0x7f4eec0692a0 0x7f4eec06b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:07:27.905 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:07:27.905+0000 7f4f05230700 1 --2- 172.21.15.162:0/2878933601 >> [v2:172.21.15.162:6800/4079782599,v1:172.21.15.162:6801/4079782599] conn(0x7f4eec0692a0 0x7f4eec06b750 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f4f0010ecd0 tx=0x7f4ef0009ee0 comp rx=0 tx=0).ready entity=mgr.14210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:07:27.907 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:07:27.909+0000 7f4ef67fc700 1 -- 172.21.15.162:0/2878933601 <== mon.0 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f4efc058170 con 0x7f4f00101d00 2024-04-27T04:07:28.137 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:07:27 smithi162 ceph-mon[26841]: pgmap v189: 65 pgs: 65 active+clean; 579 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 3.0 KiB/s rd, 0 B/s wr, 3 op/s 2024-04-27T04:07:28.143 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:07:28.144+0000 7f4f07c95700 1 -- 172.21.15.162:0/2878933601 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f4f00108e50 con 0x7f4f00101d00 2024-04-27T04:07:28.143 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:07:28.144+0000 7f4ef67fc700 1 -- 172.21.15.162:0/2878933601 <== mon.0 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v15) v1 ==== 53+0+83 (secure 0 0 0) 0x7f4efc086e60 con 0x7f4f00101d00 2024-04-27T04:07:28.144 INFO:teuthology.orchestra.run.smithi162.stdout:name: cephfs, metadata pool: cephfs.cephfs.meta, data pools: [cephfs.cephfs.data ] 2024-04-27T04:07:28.146 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:07:28.146+0000 7f4f07c95700 1 -- 172.21.15.162:0/2878933601 >> [v2:172.21.15.162:6800/4079782599,v1:172.21.15.162:6801/4079782599] conn(0x7f4eec0692a0 msgr2=0x7f4eec06b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:07:28.146 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:07:28.146+0000 7f4f07c95700 1 --2- 172.21.15.162:0/2878933601 >> [v2:172.21.15.162:6800/4079782599,v1:172.21.15.162:6801/4079782599] conn(0x7f4eec0692a0 0x7f4eec06b750 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f4f0010ecd0 tx=0x7f4ef0009ee0 comp rx=0 tx=0).stop 2024-04-27T04:07:28.146 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:07:28.146+0000 7f4f07c95700 1 -- 172.21.15.162:0/2878933601 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4f00101d00 msgr2=0x7f4f00113980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:07:28.146 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:07:28.146+0000 7f4f07c95700 1 --2- 172.21.15.162:0/2878933601 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4f00101d00 0x7f4f00113980 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f4efc00b3b0 tx=0x7f4efc009e30 comp rx=0 tx=0).stop 2024-04-27T04:07:28.146 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:07:28.147+0000 7f4f07c95700 1 -- 172.21.15.162:0/2878933601 shutdown_connections 2024-04-27T04:07:28.146 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:07:28.147+0000 7f4f07c95700 1 --2- 172.21.15.162:0/2878933601 >> [v2:172.21.15.162:6800/4079782599,v1:172.21.15.162:6801/4079782599] conn(0x7f4eec0692a0 0x7f4eec06b750 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:07:28.146 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:07:28.147+0000 7f4f07c95700 1 --2- 172.21.15.162:0/2878933601 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4f00101d00 0x7f4f00113980 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:07:28.146 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:07:28.147+0000 7f4f07c95700 1 --2- 172.21.15.162:0/2878933601 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4f001026a0 0x7f4f00113ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:07:28.146 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:07:28.147+0000 7f4f07c95700 1 -- 172.21.15.162:0/2878933601 >> 172.21.15.162:0/2878933601 conn(0x7f4f000f7870 msgr2=0x7f4f00066280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:07:28.146 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:07:28.147+0000 7f4f07c95700 1 -- 172.21.15.162:0/2878933601 shutdown_connections 2024-04-27T04:07:28.146 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:07:28.147+0000 7f4f07c95700 1 -- 172.21.15.162:0/2878933601 wait complete. 2024-04-27T04:07:28.423 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:07:27 smithi181 ceph-mon[33774]: pgmap v189: 65 pgs: 65 active+clean; 579 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 3.0 KiB/s rd, 0 B/s wr, 3 op/s 2024-04-27T04:07:28.807 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-04-27T04:07:28.807 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-27T04:07:28.807 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi181.front.sepia.ceph.com 2024-04-27T04:07:28.807 INFO:tasks.cephfs.mount:self.client.name = client.1 2024-04-27T04:07:28.808 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-04-27T04:07:28.808 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-04-27T04:07:28.808 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-04-27T04:07:28.808 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-27T04:07:28.808 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.1' 2024-04-27T04:07:28.808 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-27T04:07:28.808 DEBUG:teuthology.orchestra.run.smithi181:> ip addr 2024-04-27T04:07:28.827 INFO:teuthology.orchestra.run.smithi181.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-27T04:07:28.827 INFO:teuthology.orchestra.run.smithi181.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-27T04:07:28.827 INFO:teuthology.orchestra.run.smithi181.stdout: inet 127.0.0.1/8 scope host lo 2024-04-27T04:07:28.827 INFO:teuthology.orchestra.run.smithi181.stdout: valid_lft forever preferred_lft forever 2024-04-27T04:07:28.827 INFO:teuthology.orchestra.run.smithi181.stdout: inet6 ::1/128 scope host 2024-04-27T04:07:28.827 INFO:teuthology.orchestra.run.smithi181.stdout: valid_lft forever preferred_lft forever 2024-04-27T04:07:28.828 INFO:teuthology.orchestra.run.smithi181.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-27T04:07:28.828 INFO:teuthology.orchestra.run.smithi181.stdout: link/ether 0c:c4:7a:d9:39:10 brd ff:ff:ff:ff:ff:ff 2024-04-27T04:07:28.828 INFO:teuthology.orchestra.run.smithi181.stdout: altname enp7s0f0 2024-04-27T04:07:28.828 INFO:teuthology.orchestra.run.smithi181.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-27T04:07:28.828 INFO:teuthology.orchestra.run.smithi181.stdout: link/ether 0c:c4:7a:d9:39:11 brd ff:ff:ff:ff:ff:ff 2024-04-27T04:07:28.828 INFO:teuthology.orchestra.run.smithi181.stdout: altname enp7s0f1 2024-04-27T04:07:28.828 INFO:teuthology.orchestra.run.smithi181.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-27T04:07:28.828 INFO:teuthology.orchestra.run.smithi181.stdout: link/ether 0c:c4:7a:88:7b:9c brd ff:ff:ff:ff:ff:ff 2024-04-27T04:07:28.828 INFO:teuthology.orchestra.run.smithi181.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-27T04:07:28.828 INFO:teuthology.orchestra.run.smithi181.stdout: link/ether 0c:c4:7a:88:7b:9d brd ff:ff:ff:ff:ff:ff 2024-04-27T04:07:28.828 INFO:teuthology.orchestra.run.smithi181.stdout: inet 172.21.15.181/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-04-27T04:07:28.828 INFO:teuthology.orchestra.run.smithi181.stdout: valid_lft 42160sec preferred_lft 42160sec 2024-04-27T04:07:28.828 INFO:teuthology.orchestra.run.smithi181.stdout: inet6 fe80::a55a:32a:5ff:faa2/64 scope link noprefixroute 2024-04-27T04:07:28.829 INFO:teuthology.orchestra.run.smithi181.stdout: valid_lft forever preferred_lft forever 2024-04-27T04:07:28.829 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-04-27T04:07:28.829 DEBUG:teuthology.orchestra.run.smithi181:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-27T04:07:28.829 DEBUG:teuthology.orchestra.run.smithi181:> set -e 2024-04-27T04:07:28.829 DEBUG:teuthology.orchestra.run.smithi181:> sudo ip link add name ceph-brx type bridge 2024-04-27T04:07:28.829 DEBUG:teuthology.orchestra.run.smithi181:> sudo ip addr flush dev ceph-brx 2024-04-27T04:07:28.829 DEBUG:teuthology.orchestra.run.smithi181:> sudo ip link set ceph-brx up 2024-04-27T04:07:28.829 DEBUG:teuthology.orchestra.run.smithi181:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-04-27T04:07:28.829 DEBUG:teuthology.orchestra.run.smithi181:> ') 2024-04-27T04:07:28.923 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:07:28 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-27T04:07:28.987 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:07:28 smithi181 ceph-mon[33774]: from='client.? 172.21.15.162:0/2878933601' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-04-27T04:07:29.011 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:07:29 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-27T04:07:29.018 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-27T04:07:29.018 DEBUG:teuthology.orchestra.run.smithi181:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-27T04:07:29.042 INFO:teuthology.orchestra.run.smithi181.stdout:1 2024-04-27T04:07:29.043 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-27T04:07:29.043 DEBUG:teuthology.orchestra.run.smithi181:> route 2024-04-27T04:07:29.132 INFO:teuthology.orchestra.run.smithi181.stdout:Kernel IP routing table 2024-04-27T04:07:29.133 INFO:teuthology.orchestra.run.smithi181.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-27T04:07:29.133 INFO:teuthology.orchestra.run.smithi181.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-04-27T04:07:29.133 INFO:teuthology.orchestra.run.smithi181.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-04-27T04:07:29.133 INFO:teuthology.orchestra.run.smithi181.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-27T04:07:29.133 DEBUG:teuthology.orchestra.run.smithi181:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-27T04:07:29.133 DEBUG:teuthology.orchestra.run.smithi181:> set -e 2024-04-27T04:07:29.133 DEBUG:teuthology.orchestra.run.smithi181:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-04-27T04:07:29.134 DEBUG:teuthology.orchestra.run.smithi181:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-04-27T04:07:29.134 DEBUG:teuthology.orchestra.run.smithi181:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-04-27T04:07:29.134 DEBUG:teuthology.orchestra.run.smithi181:> ') 2024-04-27T04:07:29.137 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:07:28 smithi162 ceph-mon[26841]: from='client.? 172.21.15.162:0/2878933601' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-04-27T04:07:29.181 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:07:29 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-27T04:07:29.360 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:07:29 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-27T04:07:29.367 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-27T04:07:29.367 DEBUG:teuthology.orchestra.run.smithi181:> ip netns list 2024-04-27T04:07:29.382 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-27T04:07:29.382 DEBUG:teuthology.orchestra.run.smithi181:> ip netns list-id 2024-04-27T04:07:29.436 DEBUG:teuthology.orchestra.run.smithi181:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-27T04:07:29.436 DEBUG:teuthology.orchestra.run.smithi181:> set -e 2024-04-27T04:07:29.436 DEBUG:teuthology.orchestra.run.smithi181:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-27T04:07:29.437 DEBUG:teuthology.orchestra.run.smithi181:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.1 0 2024-04-27T04:07:29.437 DEBUG:teuthology.orchestra.run.smithi181:> ') 2024-04-27T04:07:29.524 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:07:29 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-27T04:07:29.547 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:07:29 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-27T04:07:29.556 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.1' with 192.168.0.1/16 2024-04-27T04:07:29.556 DEBUG:teuthology.orchestra.run.smithi181:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-27T04:07:29.556 DEBUG:teuthology.orchestra.run.smithi181:> set -e 2024-04-27T04:07:29.556 DEBUG:teuthology.orchestra.run.smithi181:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.1 type veth peer name brx.0 2024-04-27T04:07:29.556 DEBUG:teuthology.orchestra.run.smithi181:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-04-27T04:07:29.556 DEBUG:teuthology.orchestra.run.smithi181:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip link set veth0 up 2024-04-27T04:07:29.557 DEBUG:teuthology.orchestra.run.smithi181:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip link set lo up 2024-04-27T04:07:29.557 DEBUG:teuthology.orchestra.run.smithi181:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip route add default via 192.168.255.254 2024-04-27T04:07:29.557 DEBUG:teuthology.orchestra.run.smithi181:> ') 2024-04-27T04:07:29.605 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:07:29 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-27T04:07:29.676 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:07:29 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-27T04:07:29.682 DEBUG:teuthology.orchestra.run.smithi181:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-27T04:07:29.682 DEBUG:teuthology.orchestra.run.smithi181:> set -e 2024-04-27T04:07:29.682 DEBUG:teuthology.orchestra.run.smithi181:> sudo ip link set brx.0 up 2024-04-27T04:07:29.682 DEBUG:teuthology.orchestra.run.smithi181:> sudo ip link set dev brx.0 master ceph-brx 2024-04-27T04:07:29.682 DEBUG:teuthology.orchestra.run.smithi181:> ') 2024-04-27T04:07:29.730 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:07:29 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-27T04:07:29.751 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:07:29 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-27T04:07:29.758 INFO:tasks.cephfs.fuse_mount:Client client.1 config is {} 2024-04-27T04:07:29.758 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-27T04:07:29.758 DEBUG:teuthology.orchestra.run.smithi181:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2024-04-27T04:07:29.772 INFO:teuthology.orchestra.run.smithi181.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2024-04-27T04:07:29.772 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-27T04:07:29.773 DEBUG:teuthology.orchestra.run.smithi181:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2024-04-27T04:07:29.827 DEBUG:teuthology.orchestra.run.smithi181:> sudo modprobe fuse 2024-04-27T04:07:29.892 DEBUG:teuthology.orchestra.run.smithi181:> cat /proc/self/mounts | awk '{print $2}' 2024-04-27T04:07:29.948 INFO:teuthology.orchestra.run.smithi181.stdout:/proc 2024-04-27T04:07:29.948 INFO:teuthology.orchestra.run.smithi181.stdout:/sys 2024-04-27T04:07:29.949 INFO:teuthology.orchestra.run.smithi181.stdout:/dev 2024-04-27T04:07:29.949 INFO:teuthology.orchestra.run.smithi181.stdout:/sys/kernel/security 2024-04-27T04:07:29.949 INFO:teuthology.orchestra.run.smithi181.stdout:/dev/shm 2024-04-27T04:07:29.949 INFO:teuthology.orchestra.run.smithi181.stdout:/dev/pts 2024-04-27T04:07:29.949 INFO:teuthology.orchestra.run.smithi181.stdout:/run 2024-04-27T04:07:29.949 INFO:teuthology.orchestra.run.smithi181.stdout:/sys/fs/cgroup 2024-04-27T04:07:29.949 INFO:teuthology.orchestra.run.smithi181.stdout:/sys/fs/pstore 2024-04-27T04:07:29.949 INFO:teuthology.orchestra.run.smithi181.stdout:/sys/fs/bpf 2024-04-27T04:07:29.949 INFO:teuthology.orchestra.run.smithi181.stdout:/ 2024-04-27T04:07:29.949 INFO:teuthology.orchestra.run.smithi181.stdout:/sys/fs/selinux 2024-04-27T04:07:29.949 INFO:teuthology.orchestra.run.smithi181.stdout:/proc/sys/fs/binfmt_misc 2024-04-27T04:07:29.949 INFO:teuthology.orchestra.run.smithi181.stdout:/dev/mqueue 2024-04-27T04:07:29.949 INFO:teuthology.orchestra.run.smithi181.stdout:/dev/hugepages 2024-04-27T04:07:29.950 INFO:teuthology.orchestra.run.smithi181.stdout:/sys/kernel/debug 2024-04-27T04:07:29.950 INFO:teuthology.orchestra.run.smithi181.stdout:/sys/kernel/tracing 2024-04-27T04:07:29.950 INFO:teuthology.orchestra.run.smithi181.stdout:/run/credentials/systemd-sysusers.service 2024-04-27T04:07:29.950 INFO:teuthology.orchestra.run.smithi181.stdout:/sys/kernel/config 2024-04-27T04:07:29.950 INFO:teuthology.orchestra.run.smithi181.stdout:/sys/fs/fuse/connections 2024-04-27T04:07:29.950 INFO:teuthology.orchestra.run.smithi181.stdout:/run/credentials/systemd-sysctl.service 2024-04-27T04:07:29.950 INFO:teuthology.orchestra.run.smithi181.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-04-27T04:07:29.950 INFO:teuthology.orchestra.run.smithi181.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-04-27T04:07:29.950 INFO:teuthology.orchestra.run.smithi181.stdout:/var/lib/nfs/rpc_pipefs 2024-04-27T04:07:29.950 INFO:teuthology.orchestra.run.smithi181.stdout:/run/user/1000 2024-04-27T04:07:29.950 INFO:teuthology.orchestra.run.smithi181.stdout:/var/lib/ceph 2024-04-27T04:07:29.950 INFO:teuthology.orchestra.run.smithi181.stdout:/run/user/0 2024-04-27T04:07:29.950 INFO:teuthology.orchestra.run.smithi181.stdout:/proc/sys/fs/binfmt_misc 2024-04-27T04:07:29.951 INFO:teuthology.orchestra.run.smithi181.stdout:/var/lib/containers/storage/overlay 2024-04-27T04:07:29.951 INFO:teuthology.orchestra.run.smithi181.stdout:/var/lib/containers/storage/overlay/c5b9be1e28bc62d98205f43f93effaefade6a6cb987958a80f5cfdaffb8e99ae/merged 2024-04-27T04:07:29.951 INFO:teuthology.orchestra.run.smithi181.stdout:/var/lib/containers/storage/overlay/615c10559882532d2fd3ef820d8158e358e4981fcd9098970459280433396e45/merged 2024-04-27T04:07:29.951 INFO:teuthology.orchestra.run.smithi181.stdout:/var/lib/containers/storage/overlay/7c67f2e7bc478b2f06518815565daf57472fe7762c4ae970c43b9dab5d465d38/merged 2024-04-27T04:07:29.951 INFO:teuthology.orchestra.run.smithi181.stdout:/var/lib/containers/storage/overlay/d17f4892b6e5395a8f3116d193a0f2e48f3942357e5690d1f69c68c81b05fea7/merged 2024-04-27T04:07:29.951 INFO:teuthology.orchestra.run.smithi181.stdout:/var/lib/containers/storage/overlay/b8b65e96f0c53034c7f92db2162824d5d700f63a1ee3e50cf1a889b4318769d3/merged 2024-04-27T04:07:29.951 INFO:teuthology.orchestra.run.smithi181.stdout:/var/lib/containers/storage/overlay/68f0b8d5b72ad3e2b914afb2188c4f1c46899129d37668580e463961218bc29b/merged 2024-04-27T04:07:29.951 INFO:teuthology.orchestra.run.smithi181.stdout:/var/lib/containers/storage/overlay/bea1ee327404651f730f9e0fa5e86f54efca17a7f7cb4e7373c0075aafb4ae86/merged 2024-04-27T04:07:29.951 INFO:teuthology.orchestra.run.smithi181.stdout:/var/lib/containers/storage/overlay/1a99610d4309d24ce132fb348824b887693779f73becccb9115b0a0274fde2d4/merged 2024-04-27T04:07:29.951 INFO:teuthology.orchestra.run.smithi181.stdout:/var/lib/containers/storage/overlay/0b01e16c40929efd2a9e139b9041a5a42223e62be3d31993919330c46a884250/merged 2024-04-27T04:07:29.951 INFO:teuthology.orchestra.run.smithi181.stdout:/run/netns 2024-04-27T04:07:29.951 INFO:teuthology.orchestra.run.smithi181.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-27T04:07:29.951 INFO:teuthology.orchestra.run.smithi181.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-27T04:07:29.952 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-27T04:07:29.952 DEBUG:teuthology.orchestra.run.smithi181:> ls /sys/fs/fuse/connections 2024-04-27T04:07:30.004 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-04-27T04:07:30.004 DEBUG:teuthology.orchestra.run.smithi181:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.1 --id 1) 2024-04-27T04:07:30.046 DEBUG:teuthology.orchestra.run.smithi181:> sudo modprobe fuse 2024-04-27T04:07:30.065 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:07:29 smithi181 ceph-mon[33774]: pgmap v190: 65 pgs: 65 active+clean; 579 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.3 KiB/s rd, 170 B/s wr, 2 op/s 2024-04-27T04:07:30.070 DEBUG:teuthology.orchestra.run.smithi181:> cat /proc/self/mounts | awk '{print $2}' 2024-04-27T04:07:30.125 INFO:teuthology.orchestra.run.smithi181.stdout:/proc 2024-04-27T04:07:30.126 INFO:teuthology.orchestra.run.smithi181.stdout:/sys 2024-04-27T04:07:30.126 INFO:teuthology.orchestra.run.smithi181.stdout:/dev 2024-04-27T04:07:30.126 INFO:teuthology.orchestra.run.smithi181.stdout:/sys/kernel/security 2024-04-27T04:07:30.126 INFO:teuthology.orchestra.run.smithi181.stdout:/dev/shm 2024-04-27T04:07:30.126 INFO:teuthology.orchestra.run.smithi181.stdout:/dev/pts 2024-04-27T04:07:30.126 INFO:teuthology.orchestra.run.smithi181.stdout:/run 2024-04-27T04:07:30.126 INFO:teuthology.orchestra.run.smithi181.stdout:/sys/fs/cgroup 2024-04-27T04:07:30.126 INFO:teuthology.orchestra.run.smithi181.stdout:/sys/fs/pstore 2024-04-27T04:07:30.126 INFO:teuthology.orchestra.run.smithi181.stdout:/sys/fs/bpf 2024-04-27T04:07:30.126 INFO:teuthology.orchestra.run.smithi181.stdout:/ 2024-04-27T04:07:30.126 INFO:teuthology.orchestra.run.smithi181.stdout:/sys/fs/selinux 2024-04-27T04:07:30.126 INFO:teuthology.orchestra.run.smithi181.stdout:/proc/sys/fs/binfmt_misc 2024-04-27T04:07:30.126 INFO:teuthology.orchestra.run.smithi181.stdout:/dev/mqueue 2024-04-27T04:07:30.126 INFO:teuthology.orchestra.run.smithi181.stdout:/dev/hugepages 2024-04-27T04:07:30.127 INFO:teuthology.orchestra.run.smithi181.stdout:/sys/kernel/debug 2024-04-27T04:07:30.127 INFO:teuthology.orchestra.run.smithi181.stdout:/sys/kernel/tracing 2024-04-27T04:07:30.127 INFO:teuthology.orchestra.run.smithi181.stdout:/run/credentials/systemd-sysusers.service 2024-04-27T04:07:30.127 INFO:teuthology.orchestra.run.smithi181.stdout:/sys/kernel/config 2024-04-27T04:07:30.127 INFO:teuthology.orchestra.run.smithi181.stdout:/sys/fs/fuse/connections 2024-04-27T04:07:30.127 INFO:teuthology.orchestra.run.smithi181.stdout:/run/credentials/systemd-sysctl.service 2024-04-27T04:07:30.127 INFO:teuthology.orchestra.run.smithi181.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-04-27T04:07:30.127 INFO:teuthology.orchestra.run.smithi181.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-04-27T04:07:30.127 INFO:teuthology.orchestra.run.smithi181.stdout:/var/lib/nfs/rpc_pipefs 2024-04-27T04:07:30.127 INFO:teuthology.orchestra.run.smithi181.stdout:/run/user/1000 2024-04-27T04:07:30.128 INFO:teuthology.orchestra.run.smithi181.stdout:/var/lib/ceph 2024-04-27T04:07:30.128 INFO:teuthology.orchestra.run.smithi181.stdout:/run/user/0 2024-04-27T04:07:30.128 INFO:teuthology.orchestra.run.smithi181.stdout:/proc/sys/fs/binfmt_misc 2024-04-27T04:07:30.128 INFO:teuthology.orchestra.run.smithi181.stdout:/var/lib/containers/storage/overlay 2024-04-27T04:07:30.128 INFO:teuthology.orchestra.run.smithi181.stdout:/var/lib/containers/storage/overlay/c5b9be1e28bc62d98205f43f93effaefade6a6cb987958a80f5cfdaffb8e99ae/merged 2024-04-27T04:07:30.128 INFO:teuthology.orchestra.run.smithi181.stdout:/var/lib/containers/storage/overlay/615c10559882532d2fd3ef820d8158e358e4981fcd9098970459280433396e45/merged 2024-04-27T04:07:30.128 INFO:teuthology.orchestra.run.smithi181.stdout:/var/lib/containers/storage/overlay/7c67f2e7bc478b2f06518815565daf57472fe7762c4ae970c43b9dab5d465d38/merged 2024-04-27T04:07:30.128 INFO:teuthology.orchestra.run.smithi181.stdout:/var/lib/containers/storage/overlay/d17f4892b6e5395a8f3116d193a0f2e48f3942357e5690d1f69c68c81b05fea7/merged 2024-04-27T04:07:30.128 INFO:teuthology.orchestra.run.smithi181.stdout:/var/lib/containers/storage/overlay/b8b65e96f0c53034c7f92db2162824d5d700f63a1ee3e50cf1a889b4318769d3/merged 2024-04-27T04:07:30.128 INFO:teuthology.orchestra.run.smithi181.stdout:/var/lib/containers/storage/overlay/68f0b8d5b72ad3e2b914afb2188c4f1c46899129d37668580e463961218bc29b/merged 2024-04-27T04:07:30.128 INFO:teuthology.orchestra.run.smithi181.stdout:/var/lib/containers/storage/overlay/bea1ee327404651f730f9e0fa5e86f54efca17a7f7cb4e7373c0075aafb4ae86/merged 2024-04-27T04:07:30.128 INFO:teuthology.orchestra.run.smithi181.stdout:/var/lib/containers/storage/overlay/1a99610d4309d24ce132fb348824b887693779f73becccb9115b0a0274fde2d4/merged 2024-04-27T04:07:30.128 INFO:teuthology.orchestra.run.smithi181.stdout:/var/lib/containers/storage/overlay/0b01e16c40929efd2a9e139b9041a5a42223e62be3d31993919330c46a884250/merged 2024-04-27T04:07:30.128 INFO:teuthology.orchestra.run.smithi181.stdout:/run/netns 2024-04-27T04:07:30.128 INFO:teuthology.orchestra.run.smithi181.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-27T04:07:30.129 INFO:teuthology.orchestra.run.smithi181.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-27T04:07:30.129 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-27T04:07:30.129 DEBUG:teuthology.orchestra.run.smithi181:> ls /sys/fs/fuse/connections 2024-04-27T04:07:30.136 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi181.stderr:2024-04-27T04:07:30.137+0000 7fe6924a2300 -1 init, newargv = 0x55ef99d197e0 newargc=15 2024-04-27T04:07:30.136 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi181.stderr:ceph-fuse[58940]: starting ceph client 2024-04-27T04:07:30.141 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi181.stderr:ceph-fuse[58940]: starting fuse 2024-04-27T04:07:30.152 INFO:teuthology.orchestra.run.smithi181.stdout:112 2024-04-27T04:07:30.152 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [112] 2024-04-27T04:07:30.153 DEBUG:teuthology.orchestra.run.smithi181:> sudo stdin-killer -- python3 -c ' 2024-04-27T04:07:30.153 DEBUG:teuthology.orchestra.run.smithi181:> import glob 2024-04-27T04:07:30.153 DEBUG:teuthology.orchestra.run.smithi181:> import re 2024-04-27T04:07:30.153 DEBUG:teuthology.orchestra.run.smithi181:> import os 2024-04-27T04:07:30.153 DEBUG:teuthology.orchestra.run.smithi181:> import subprocess 2024-04-27T04:07:30.153 DEBUG:teuthology.orchestra.run.smithi181:> 2024-04-27T04:07:30.153 DEBUG:teuthology.orchestra.run.smithi181:> def _find_admin_socket(client_name): 2024-04-27T04:07:30.153 DEBUG:teuthology.orchestra.run.smithi181:> asok_path = "/var/run/ceph/ceph-client.1.*.asok" 2024-04-27T04:07:30.153 DEBUG:teuthology.orchestra.run.smithi181:> files = glob.glob(asok_path) 2024-04-27T04:07:30.153 DEBUG:teuthology.orchestra.run.smithi181:> mountpoint = "/home/ubuntu/cephtest/mnt.1" 2024-04-27T04:07:30.153 DEBUG:teuthology.orchestra.run.smithi181:> 2024-04-27T04:07:30.153 DEBUG:teuthology.orchestra.run.smithi181:> # Given a non-glob path, it better be there 2024-04-27T04:07:30.154 DEBUG:teuthology.orchestra.run.smithi181:> if "*" not in asok_path: 2024-04-27T04:07:30.154 DEBUG:teuthology.orchestra.run.smithi181:> assert(len(files) == 1) 2024-04-27T04:07:30.154 DEBUG:teuthology.orchestra.run.smithi181:> return files[0] 2024-04-27T04:07:30.154 DEBUG:teuthology.orchestra.run.smithi181:> 2024-04-27T04:07:30.154 DEBUG:teuthology.orchestra.run.smithi181:> for f in files: 2024-04-27T04:07:30.154 DEBUG:teuthology.orchestra.run.smithi181:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-04-27T04:07:30.154 DEBUG:teuthology.orchestra.run.smithi181:> if os.path.exists("/proc/{0}".format(pid)): 2024-04-27T04:07:30.154 DEBUG:teuthology.orchestra.run.smithi181:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-04-27T04:07:30.154 DEBUG:teuthology.orchestra.run.smithi181:> contents = proc_f.read() 2024-04-27T04:07:30.154 DEBUG:teuthology.orchestra.run.smithi181:> if mountpoint in contents: 2024-04-27T04:07:30.154 DEBUG:teuthology.orchestra.run.smithi181:> return f 2024-04-27T04:07:30.154 DEBUG:teuthology.orchestra.run.smithi181:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-04-27T04:07:30.154 DEBUG:teuthology.orchestra.run.smithi181:> 2024-04-27T04:07:30.154 DEBUG:teuthology.orchestra.run.smithi181:> print(_find_admin_socket("client.1")) 2024-04-27T04:07:30.154 DEBUG:teuthology.orchestra.run.smithi181:> ' 2024-04-27T04:07:30.269 INFO:teuthology.orchestra.run.smithi181.stdout:/var/run/ceph/ceph-client.1.58940.asok 2024-04-27T04:07:30.272 INFO:teuthology.orchestra.run.smithi181.stderr:2024-04-27T04:07:30 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-27T04:07:30.279 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.1.58940.asok 2024-04-27T04:07:30.279 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-27T04:07:30.280 DEBUG:teuthology.orchestra.run.smithi181:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.1.58940.asok status 2024-04-27T04:07:30.387 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:07:29 smithi162 ceph-mon[26841]: pgmap v190: 65 pgs: 65 active+clean; 579 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.3 KiB/s rd, 170 B/s wr, 2 op/s 2024-04-27T04:07:30.408 INFO:teuthology.orchestra.run.smithi181.stdout:{ 2024-04-27T04:07:30.408 INFO:teuthology.orchestra.run.smithi181.stdout: "metadata": { 2024-04-27T04:07:30.408 INFO:teuthology.orchestra.run.smithi181.stdout: "ceph_sha1": "2be3efa97a7972f8e925bd899b474329979b74d6", 2024-04-27T04:07:30.408 INFO:teuthology.orchestra.run.smithi181.stdout: "ceph_version": "ceph version 17.2.7-907-g2be3efa9 (2be3efa97a7972f8e925bd899b474329979b74d6) quincy (stable)", 2024-04-27T04:07:30.408 INFO:teuthology.orchestra.run.smithi181.stdout: "entity_id": "1", 2024-04-27T04:07:30.408 INFO:teuthology.orchestra.run.smithi181.stdout: "hostname": "smithi181", 2024-04-27T04:07:30.409 INFO:teuthology.orchestra.run.smithi181.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.1", 2024-04-27T04:07:30.409 INFO:teuthology.orchestra.run.smithi181.stdout: "pid": "58940", 2024-04-27T04:07:30.409 INFO:teuthology.orchestra.run.smithi181.stdout: "root": "/" 2024-04-27T04:07:30.409 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-04-27T04:07:30.409 INFO:teuthology.orchestra.run.smithi181.stdout: "dentry_count": 0, 2024-04-27T04:07:30.409 INFO:teuthology.orchestra.run.smithi181.stdout: "dentry_pinned_count": 0, 2024-04-27T04:07:30.409 INFO:teuthology.orchestra.run.smithi181.stdout: "id": 24323, 2024-04-27T04:07:30.409 INFO:teuthology.orchestra.run.smithi181.stdout: "inst": { 2024-04-27T04:07:30.409 INFO:teuthology.orchestra.run.smithi181.stdout: "name": { 2024-04-27T04:07:30.409 INFO:teuthology.orchestra.run.smithi181.stdout: "type": "client", 2024-04-27T04:07:30.409 INFO:teuthology.orchestra.run.smithi181.stdout: "num": 24323 2024-04-27T04:07:30.409 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-04-27T04:07:30.409 INFO:teuthology.orchestra.run.smithi181.stdout: "addr": { 2024-04-27T04:07:30.409 INFO:teuthology.orchestra.run.smithi181.stdout: "type": "v1", 2024-04-27T04:07:30.410 INFO:teuthology.orchestra.run.smithi181.stdout: "addr": "192.168.0.1:0", 2024-04-27T04:07:30.410 INFO:teuthology.orchestra.run.smithi181.stdout: "nonce": 2156375244 2024-04-27T04:07:30.410 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-04-27T04:07:30.410 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-04-27T04:07:30.410 INFO:teuthology.orchestra.run.smithi181.stdout: "addr": { 2024-04-27T04:07:30.410 INFO:teuthology.orchestra.run.smithi181.stdout: "type": "v1", 2024-04-27T04:07:30.410 INFO:teuthology.orchestra.run.smithi181.stdout: "addr": "192.168.0.1:0", 2024-04-27T04:07:30.410 INFO:teuthology.orchestra.run.smithi181.stdout: "nonce": 2156375244 2024-04-27T04:07:30.410 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-04-27T04:07:30.410 INFO:teuthology.orchestra.run.smithi181.stdout: "inst_str": "client.24323 192.168.0.1:0/2156375244", 2024-04-27T04:07:30.410 INFO:teuthology.orchestra.run.smithi181.stdout: "addr_str": "192.168.0.1:0/2156375244", 2024-04-27T04:07:30.410 INFO:teuthology.orchestra.run.smithi181.stdout: "inode_count": 1, 2024-04-27T04:07:30.410 INFO:teuthology.orchestra.run.smithi181.stdout: "mds_epoch": 15, 2024-04-27T04:07:30.410 INFO:teuthology.orchestra.run.smithi181.stdout: "osd_epoch": 46, 2024-04-27T04:07:30.410 INFO:teuthology.orchestra.run.smithi181.stdout: "osd_epoch_barrier": 0, 2024-04-27T04:07:30.411 INFO:teuthology.orchestra.run.smithi181.stdout: "blocklisted": false, 2024-04-27T04:07:30.411 INFO:teuthology.orchestra.run.smithi181.stdout: "fs_name": "cephfs" 2024-04-27T04:07:30.411 INFO:teuthology.orchestra.run.smithi181.stdout:} 2024-04-27T04:07:30.417 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-27T04:07:30.417 DEBUG:teuthology.orchestra.run.smithi162:> stat --file-system '--printf=%T 2024-04-27T04:07:30.417 DEBUG:teuthology.orchestra.run.smithi162:> ' -- /home/ubuntu/cephtest/mnt.0 2024-04-27T04:07:30.437 INFO:teuthology.orchestra.run.smithi162.stdout:fuseblk 2024-04-27T04:07:30.437 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2024-04-27T04:07:30.438 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-27T04:07:30.438 DEBUG:teuthology.orchestra.run.smithi162:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-27T04:07:30.510 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-27T04:07:30.510 DEBUG:teuthology.orchestra.run.smithi181:> stat --file-system '--printf=%T 2024-04-27T04:07:30.511 DEBUG:teuthology.orchestra.run.smithi181:> ' -- /home/ubuntu/cephtest/mnt.1 2024-04-27T04:07:30.526 INFO:teuthology.orchestra.run.smithi181.stdout:fuseblk 2024-04-27T04:07:30.526 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.1 2024-04-27T04:07:30.526 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-27T04:07:30.526 DEBUG:teuthology.orchestra.run.smithi181:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2024-04-27T04:07:30.592 INFO:teuthology.run_tasks:Running task print... 2024-04-27T04:07:30.605 INFO:teuthology.task.print:**** done client 2024-04-27T04:07:30.606 INFO:teuthology.run_tasks:Running task parallel... 2024-04-27T04:07:30.621 INFO:teuthology.task.parallel:starting parallel... 2024-04-27T04:07:30.621 INFO:teuthology.task.parallel:In parallel, running task sequential... 2024-04-27T04:07:30.622 INFO:teuthology.task.sequential:In sequential, running task cephadm.shell... 2024-04-27T04:07:30.622 INFO:tasks.cephadm:Running commands on role host.a host ubuntu@smithi162.front.sepia.ceph.com 2024-04-27T04:07:30.622 DEBUG:teuthology.orchestra.run.smithi162:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 7f9750d6-044a-11ef-bc93-c7b262605968 -e sha1=4e213a58e787437ff508e6c890ccd5fd32dc96b5 -- bash -c 'ceph config set mon mon_warn_on_insecure_global_id_reclaim false --force' 2024-04-27T04:07:30.624 INFO:teuthology.task.parallel:In parallel, running task sequential... 2024-04-27T04:07:30.624 INFO:teuthology.task.sequential:In sequential, running task workunit... 2024-04-27T04:07:30.627 INFO:tasks.workunit:Pulling workunits from ref 3067e9f133f954b642fdf21114a79edb58590948 2024-04-27T04:07:30.628 INFO:tasks.workunit:Making a separate scratch dir for every client... 2024-04-27T04:07:30.628 INFO:tasks.workunit:timeout=3h 2024-04-27T04:07:30.628 INFO:tasks.workunit:cleanup=True 2024-04-27T04:07:30.628 DEBUG:teuthology.orchestra.run.smithi162:> stat -- /home/ubuntu/cephtest/mnt.0 2024-04-27T04:07:30.648 INFO:teuthology.orchestra.run.smithi162.stdout: File: /home/ubuntu/cephtest/mnt.0 2024-04-27T04:07:30.648 INFO:teuthology.orchestra.run.smithi162.stdout: Size: 0 Blocks: 1 IO Block: 4194304 directory 2024-04-27T04:07:30.648 INFO:teuthology.orchestra.run.smithi162.stdout:Device: 46h/70d Inode: 1 Links: 2 2024-04-27T04:07:30.648 INFO:teuthology.orchestra.run.smithi162.stdout:Access: (1777/drwxrwxrwt) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-27T04:07:30.648 INFO:teuthology.orchestra.run.smithi162.stdout:Context: system_u:object_r:fusefs_t:s0 2024-04-27T04:07:30.649 INFO:teuthology.orchestra.run.smithi162.stdout:Access: 1970-01-01 00:00:00.000000000 +0000 2024-04-27T04:07:30.649 INFO:teuthology.orchestra.run.smithi162.stdout:Modify: 2024-04-27 04:06:58.123584411 +0000 2024-04-27T04:07:30.649 INFO:teuthology.orchestra.run.smithi162.stdout:Change: 2024-04-27 04:07:30.508689372 +0000 2024-04-27T04:07:30.649 INFO:teuthology.orchestra.run.smithi162.stdout: Birth: - 2024-04-27T04:07:30.649 INFO:tasks.workunit:Did not need to create dir /home/ubuntu/cephtest/mnt.0 2024-04-27T04:07:30.649 DEBUG:teuthology.orchestra.run.smithi162:> cd -- /home/ubuntu/cephtest/mnt.0 && sudo install -d -m 0755 --owner=ubuntu -- client.0 2024-04-27T04:07:30.719 DEBUG:teuthology.orchestra.run.smithi181:> stat -- /home/ubuntu/cephtest/mnt.1 2024-04-27T04:07:30.734 INFO:teuthology.orchestra.run.smithi181.stdout: File: /home/ubuntu/cephtest/mnt.1 2024-04-27T04:07:30.735 INFO:teuthology.orchestra.run.smithi181.stdout: Size: 0 Blocks: 1 IO Block: 4194304 directory 2024-04-27T04:07:30.735 INFO:teuthology.orchestra.run.smithi181.stdout:Device: 70h/112d Inode: 1 Links: 3 2024-04-27T04:07:30.735 INFO:teuthology.orchestra.run.smithi181.stdout:Access: (1777/drwxrwxrwt) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-27T04:07:30.735 INFO:teuthology.orchestra.run.smithi181.stdout:Context: system_u:object_r:fusefs_t:s0 2024-04-27T04:07:30.735 INFO:teuthology.orchestra.run.smithi181.stdout:Access: 1970-01-01 00:00:00.000000000 +0000 2024-04-27T04:07:30.735 INFO:teuthology.orchestra.run.smithi181.stdout:Modify: 2024-04-27 04:07:30.715451322 +0000 2024-04-27T04:07:30.735 INFO:teuthology.orchestra.run.smithi181.stdout:Change: 2024-04-27 04:07:30.715451322 +0000 2024-04-27T04:07:30.735 INFO:teuthology.orchestra.run.smithi181.stdout: Birth: - 2024-04-27T04:07:30.735 INFO:tasks.workunit:Did not need to create dir /home/ubuntu/cephtest/mnt.1 2024-04-27T04:07:30.735 DEBUG:teuthology.orchestra.run.smithi181:> cd -- /home/ubuntu/cephtest/mnt.1 && sudo install -d -m 0755 --owner=ubuntu -- client.1 2024-04-27T04:07:30.803 DEBUG:teuthology.orchestra.run.smithi162:> rm -rf /home/ubuntu/cephtest/clone.client.0 && git clone https://git.ceph.com/ceph.git /home/ubuntu/cephtest/clone.client.0 && cd /home/ubuntu/cephtest/clone.client.0 && git checkout 3067e9f133f954b642fdf21114a79edb58590948 2024-04-27T04:07:30.804 DEBUG:teuthology.orchestra.run.smithi181:> rm -rf /home/ubuntu/cephtest/clone.client.1 && git clone https://git.ceph.com/ceph.git /home/ubuntu/cephtest/clone.client.1 && cd /home/ubuntu/cephtest/clone.client.1 && git checkout 3067e9f133f954b642fdf21114a79edb58590948 2024-04-27T04:07:30.819 INFO:tasks.workunit.client.0.smithi162.stderr:Cloning into '/home/ubuntu/cephtest/clone.client.0'... 2024-04-27T04:07:30.859 INFO:tasks.workunit.client.1.smithi181.stderr:Cloning into '/home/ubuntu/cephtest/clone.client.1'... 2024-04-27T04:07:30.898 INFO:teuthology.orchestra.run.smithi162.stderr:Inferring config /var/lib/ceph/7f9750d6-044a-11ef-bc93-c7b262605968/mon.smithi162/config 2024-04-27T04:07:32.214 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:07:31 smithi162 ceph-mon[26841]: pgmap v191: 65 pgs: 65 active+clean; 579 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 938 B/s rd, 170 B/s wr, 1 op/s 2024-04-27T04:07:32.214 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:07:31 smithi162 ceph-mon[26841]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-27T04:07:32.422 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:07:31 smithi181 ceph-mon[33774]: pgmap v191: 65 pgs: 65 active+clean; 579 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 938 B/s rd, 170 B/s wr, 1 op/s 2024-04-27T04:07:32.423 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:07:31 smithi181 ceph-mon[33774]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-27T04:07:32.582 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:07:32.582+0000 7f5da565f700 1 -- 172.21.15.162:0/4218755864 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5da00fdfe0 msgr2=0x7f5da00fe450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:07:32.582 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:07:32.582+0000 7f5da565f700 1 --2- 172.21.15.162:0/4218755864 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5da00fdfe0 0x7f5da00fe450 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f5d94004940 tx=0x7f5d94009e30 comp rx=0 tx=0).stop 2024-04-27T04:07:32.583 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:07:32.582+0000 7f5da565f700 1 -- 172.21.15.162:0/4218755864 shutdown_connections 2024-04-27T04:07:32.583 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:07:32.582+0000 7f5da565f700 1 --2- 172.21.15.162:0/4218755864 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5da00fdfe0 0x7f5da00fe450 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:07:32.583 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:07:32.582+0000 7f5da565f700 1 --2- 172.21.15.162:0/4218755864 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5da00fce70 0x7f5da00fd280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:07:32.583 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:07:32.582+0000 7f5da565f700 1 -- 172.21.15.162:0/4218755864 >> 172.21.15.162:0/4218755864 conn(0x7f5da00f8510 msgr2=0x7f5da00fa960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:07:32.583 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:07:32.582+0000 7f5da565f700 1 -- 172.21.15.162:0/4218755864 shutdown_connections 2024-04-27T04:07:32.583 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:07:32.582+0000 7f5da565f700 1 -- 172.21.15.162:0/4218755864 wait complete. 2024-04-27T04:07:32.583 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:07:32.582+0000 7f5da565f700 1 Processor -- start 2024-04-27T04:07:32.583 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:07:32.582+0000 7f5da565f700 1 -- start start 2024-04-27T04:07:32.583 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:07:32.582+0000 7f5da565f700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5da00fce70 0x7f5da0071de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:07:32.583 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:07:32.582+0000 7f5da565f700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5da00fdfe0 0x7f5da00702b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:07:32.583 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:07:32.582+0000 7f5da565f700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5da00708f0 con 0x7f5da00fdfe0 2024-04-27T04:07:32.583 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:07:32.582+0000 7f5da565f700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5da0070a60 con 0x7f5da00fce70 2024-04-27T04:07:32.583 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:07:32.583+0000 7f5d9e7fc700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5da00fdfe0 0x7f5da00702b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:07:32.583 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:07:32.583+0000 7f5d9e7fc700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5da00fdfe0 0x7f5da00702b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:38578/0 (socket says 172.21.15.162:38578) 2024-04-27T04:07:32.584 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:07:32.583+0000 7f5d9e7fc700 1 -- 172.21.15.162:0/1220301659 learned_addr learned my addr 172.21.15.162:0/1220301659 (peer_addr_for_me v2:172.21.15.162:0/0) 2024-04-27T04:07:32.584 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:07:32.583+0000 7f5d9effd700 1 --2- 172.21.15.162:0/1220301659 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5da00fce70 0x7f5da0071de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:07:32.584 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:07:32.583+0000 7f5d9e7fc700 1 -- 172.21.15.162:0/1220301659 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5da00fce70 msgr2=0x7f5da0071de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:07:32.584 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:07:32.583+0000 7f5d9e7fc700 1 --2- 172.21.15.162:0/1220301659 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5da00fce70 0x7f5da0071de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:07:32.584 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:07:32.583+0000 7f5d9e7fc700 1 -- 172.21.15.162:0/1220301659 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d94005040 con 0x7f5da00fdfe0 2024-04-27T04:07:32.584 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:07:32.583+0000 7f5d9e7fc700 1 --2- 172.21.15.162:0/1220301659 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5da00fdfe0 0x7f5da00702b0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f5d9400b3b0 tx=0x7f5d94009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:07:32.584 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:07:32.583+0000 7f5d877fe700 1 -- 172.21.15.162:0/1220301659 <== mon.0 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5d9400e040 con 0x7f5da00fdfe0 2024-04-27T04:07:32.584 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:07:32.583+0000 7f5d877fe700 1 -- 172.21.15.162:0/1220301659 <== mon.0 v2:172.21.15.162:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f5d9400bcc0 con 0x7f5da00fdfe0 2024-04-27T04:07:32.584 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:07:32.583+0000 7f5da565f700 1 -- 172.21.15.162:0/1220301659 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5da0070ce0 con 0x7f5da00fdfe0 2024-04-27T04:07:32.584 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:07:32.583+0000 7f5d877fe700 1 -- 172.21.15.162:0/1220301659 <== mon.0 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5d9401c630 con 0x7f5da00fdfe0 2024-04-27T04:07:32.584 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:07:32.583+0000 7f5da565f700 1 -- 172.21.15.162:0/1220301659 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5da0115a20 con 0x7f5da00fdfe0 2024-04-27T04:07:32.585 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:07:32.584+0000 7f5da565f700 1 -- 172.21.15.162:0/1220301659 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5da005f5c0 con 0x7f5da00fdfe0 2024-04-27T04:07:32.585 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:07:32.585+0000 7f5d877fe700 1 -- 172.21.15.162:0/1220301659 <== mon.0 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 19) v1 ==== 86631+0+0 (secure 0 0 0) 0x7f5d94013980 con 0x7f5da00fdfe0 2024-04-27T04:07:32.585 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:07:32.585+0000 7f5d877fe700 1 --2- 172.21.15.162:0/1220301659 >> [v2:172.21.15.162:6800/4079782599,v1:172.21.15.162:6801/4079782599] conn(0x7f5d880692a0 0x7f5d8806b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:07:32.585 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:07:32.585+0000 7f5d877fe700 1 -- 172.21.15.162:0/1220301659 <== mon.0 v2:172.21.15.162:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5550+0+0 (secure 0 0 0) 0x7f5d94088010 con 0x7f5da00fdfe0 2024-04-27T04:07:32.586 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:07:32.585+0000 7f5d9effd700 1 --2- 172.21.15.162:0/1220301659 >> [v2:172.21.15.162:6800/4079782599,v1:172.21.15.162:6801/4079782599] conn(0x7f5d880692a0 0x7f5d8806b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:07:32.586 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:07:32.586+0000 7f5d9effd700 1 --2- 172.21.15.162:0/1220301659 >> [v2:172.21.15.162:6800/4079782599,v1:172.21.15.162:6801/4079782599] conn(0x7f5d880692a0 0x7f5d8806b750 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f5d90006fd0 tx=0x7f5d90008040 comp rx=0 tx=0).ready entity=mgr.14210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:07:32.588 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:07:32.589+0000 7f5d877fe700 1 -- 172.21.15.162:0/1220301659 <== mon.0 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f5d94054270 con 0x7f5da00fdfe0 2024-04-27T04:07:32.779 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:07:32.780+0000 7f5da565f700 1 -- 172.21.15.162:0/1220301659 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command([{prefix=config set, name=mon_warn_on_insecure_global_id_reclaim}] v 0) v1 -- 0x7f5da0115c40 con 0x7f5da00fdfe0 2024-04-27T04:07:32.780 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:07:32.781+0000 7f5d877fe700 1 -- 172.21.15.162:0/1220301659 <== mon.0 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_warn_on_insecure_global_id_reclaim}]=0 v21)=0 v21) v1 ==== 155+0+0 (secure 0 0 0) 0x7f5d94057890 con 0x7f5da00fdfe0 2024-04-27T04:07:32.782 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:07:32.783+0000 7f5da565f700 1 -- 172.21.15.162:0/1220301659 >> [v2:172.21.15.162:6800/4079782599,v1:172.21.15.162:6801/4079782599] conn(0x7f5d880692a0 msgr2=0x7f5d8806b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:07:32.782 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:07:32.783+0000 7f5da565f700 1 --2- 172.21.15.162:0/1220301659 >> [v2:172.21.15.162:6800/4079782599,v1:172.21.15.162:6801/4079782599] conn(0x7f5d880692a0 0x7f5d8806b750 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f5d90006fd0 tx=0x7f5d90008040 comp rx=0 tx=0).stop 2024-04-27T04:07:32.782 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:07:32.783+0000 7f5da565f700 1 -- 172.21.15.162:0/1220301659 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5da00fdfe0 msgr2=0x7f5da00702b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:07:32.782 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:07:32.783+0000 7f5da565f700 1 --2- 172.21.15.162:0/1220301659 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5da00fdfe0 0x7f5da00702b0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f5d9400b3b0 tx=0x7f5d94009e30 comp rx=0 tx=0).stop 2024-04-27T04:07:32.782 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:07:32.783+0000 7f5da565f700 1 -- 172.21.15.162:0/1220301659 shutdown_connections 2024-04-27T04:07:32.782 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:07:32.783+0000 7f5da565f700 1 --2- 172.21.15.162:0/1220301659 >> [v2:172.21.15.162:6800/4079782599,v1:172.21.15.162:6801/4079782599] conn(0x7f5d880692a0 0x7f5d8806b750 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:07:32.782 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:07:32.783+0000 7f5da565f700 1 --2- 172.21.15.162:0/1220301659 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5da00fce70 0x7f5da0071de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:07:32.782 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:07:32.783+0000 7f5da565f700 1 --2- 172.21.15.162:0/1220301659 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5da00fdfe0 0x7f5da00702b0 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:07:32.782 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:07:32.783+0000 7f5da565f700 1 -- 172.21.15.162:0/1220301659 >> 172.21.15.162:0/1220301659 conn(0x7f5da00f8510 msgr2=0x7f5da00f9b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:07:32.783 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:07:32.783+0000 7f5da565f700 1 -- 172.21.15.162:0/1220301659 shutdown_connections 2024-04-27T04:07:32.783 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:07:32.783+0000 7f5da565f700 1 -- 172.21.15.162:0/1220301659 wait complete. 2024-04-27T04:07:33.449 DEBUG:teuthology.orchestra.run.smithi162:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 7f9750d6-044a-11ef-bc93-c7b262605968 -e sha1=4e213a58e787437ff508e6c890ccd5fd32dc96b5 -- bash -c 'ceph config set mon mon_warn_on_insecure_global_id_reclaim_allowed false --force' 2024-04-27T04:07:33.747 INFO:teuthology.orchestra.run.smithi162.stderr:Inferring config /var/lib/ceph/7f9750d6-044a-11ef-bc93-c7b262605968/mon.smithi162/config 2024-04-27T04:07:34.137 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:07:33 smithi162 ceph-mon[26841]: pgmap v192: 65 pgs: 65 active+clean; 585 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.8 KiB/s rd, 682 B/s wr, 3 op/s 2024-04-27T04:07:34.423 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:07:33 smithi181 ceph-mon[33774]: pgmap v192: 65 pgs: 65 active+clean; 585 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.8 KiB/s rd, 682 B/s wr, 3 op/s 2024-04-27T04:07:35.416 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:07:35.416+0000 7f244b855700 1 -- 172.21.15.162:0/3549972104 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f24440fed30 msgr2=0x7f24440fae70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:07:35.416 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:07:35.416+0000 7f244b855700 1 --2- 172.21.15.162:0/3549972104 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f24440fed30 0x7f24440fae70 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f2434004940 tx=0x7f2434009e30 comp rx=0 tx=0).stop 2024-04-27T04:07:35.416 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:07:35.416+0000 7f244b855700 1 -- 172.21.15.162:0/3549972104 shutdown_connections 2024-04-27T04:07:35.416 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:07:35.416+0000 7f244b855700 1 --2- 172.21.15.162:0/3549972104 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f24440fed30 0x7f24440fae70 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:07:35.417 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:07:35.416+0000 7f244b855700 1 --2- 172.21.15.162:0/3549972104 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f24440fe3e0 0x7f24440fe7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:07:35.417 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:07:35.416+0000 7f244b855700 1 -- 172.21.15.162:0/3549972104 >> 172.21.15.162:0/3549972104 conn(0x7f24440f6b80 msgr2=0x7f24440f8fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:07:35.417 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:07:35.416+0000 7f244b855700 1 -- 172.21.15.162:0/3549972104 shutdown_connections 2024-04-27T04:07:35.417 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:07:35.416+0000 7f244b855700 1 -- 172.21.15.162:0/3549972104 wait complete. 2024-04-27T04:07:35.417 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:07:35.416+0000 7f244b855700 1 Processor -- start 2024-04-27T04:07:35.417 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:07:35.416+0000 7f244b855700 1 -- start start 2024-04-27T04:07:35.419 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:07:35.417+0000 7f244b855700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f24440fe3e0 0x7f244410da30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:07:35.419 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:07:35.417+0000 7f244b855700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f24440fed30 0x7f244410df70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:07:35.419 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:07:35.417+0000 7f244b855700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f244410e570 con 0x7f24440fed30 2024-04-27T04:07:35.419 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:07:35.417+0000 7f244b855700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f244410ed70 con 0x7f24440fe3e0 2024-04-27T04:07:35.419 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:07:35.417+0000 7f2448df0700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f24440fed30 0x7f244410df70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:07:35.419 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:07:35.417+0000 7f2448df0700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f24440fed30 0x7f244410df70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:54424/0 (socket says 172.21.15.162:54424) 2024-04-27T04:07:35.419 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:07:35.417+0000 7f2448df0700 1 -- 172.21.15.162:0/1965139450 learned_addr learned my addr 172.21.15.162:0/1965139450 (peer_addr_for_me v2:172.21.15.162:0/0) 2024-04-27T04:07:35.419 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:07:35.417+0000 7f2448df0700 1 -- 172.21.15.162:0/1965139450 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f24440fe3e0 msgr2=0x7f244410da30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:07:35.419 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:07:35.417+0000 7f24495f1700 1 --2- 172.21.15.162:0/1965139450 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f24440fe3e0 0x7f244410da30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:07:35.419 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:07:35.417+0000 7f2448df0700 1 --2- 172.21.15.162:0/1965139450 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f24440fe3e0 0x7f244410da30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:07:35.420 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:07:35.417+0000 7f2448df0700 1 -- 172.21.15.162:0/1965139450 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2434005040 con 0x7f24440fed30 2024-04-27T04:07:35.420 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:07:35.417+0000 7f2448df0700 1 --2- 172.21.15.162:0/1965139450 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f24440fed30 0x7f244410df70 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f2434009150 tx=0x7f2434009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:07:35.420 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:07:35.417+0000 7f2439ffb700 1 -- 172.21.15.162:0/1965139450 <== mon.0 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f243400e040 con 0x7f24440fed30 2024-04-27T04:07:35.420 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:07:35.417+0000 7f244b855700 1 -- 172.21.15.162:0/1965139450 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f244410eff0 con 0x7f24440fed30 2024-04-27T04:07:35.420 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:07:35.417+0000 7f2439ffb700 1 -- 172.21.15.162:0/1965139450 <== mon.0 v2:172.21.15.162:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f243400bcc0 con 0x7f24440fed30 2024-04-27T04:07:35.420 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:07:35.417+0000 7f2439ffb700 1 -- 172.21.15.162:0/1965139450 <== mon.0 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f243401c630 con 0x7f24440fed30 2024-04-27T04:07:35.420 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:07:35.418+0000 7f244b855700 1 -- 172.21.15.162:0/1965139450 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f244410f530 con 0x7f24440fed30 2024-04-27T04:07:35.420 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:07:35.419+0000 7f244b855700 1 -- 172.21.15.162:0/1965139450 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f244405f5c0 con 0x7f24440fed30 2024-04-27T04:07:35.420 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:07:35.419+0000 7f2439ffb700 1 -- 172.21.15.162:0/1965139450 <== mon.0 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 19) v1 ==== 86631+0+0 (secure 0 0 0) 0x7f2434026440 con 0x7f24440fed30 2024-04-27T04:07:35.421 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:07:35.419+0000 7f2439ffb700 1 --2- 172.21.15.162:0/1965139450 >> [v2:172.21.15.162:6800/4079782599,v1:172.21.15.162:6801/4079782599] conn(0x7f24300692a0 0x7f243006b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:07:35.421 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:07:35.419+0000 7f2439ffb700 1 -- 172.21.15.162:0/1965139450 <== mon.0 v2:172.21.15.162:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5550+0+0 (secure 0 0 0) 0x7f2434087ff0 con 0x7f24440fed30 2024-04-27T04:07:35.422 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:07:35.419+0000 7f24495f1700 1 --2- 172.21.15.162:0/1965139450 >> [v2:172.21.15.162:6800/4079782599,v1:172.21.15.162:6801/4079782599] conn(0x7f24300692a0 0x7f243006b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:07:35.422 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:07:35.420+0000 7f24495f1700 1 --2- 172.21.15.162:0/1965139450 >> [v2:172.21.15.162:6800/4079782599,v1:172.21.15.162:6801/4079782599] conn(0x7f24300692a0 0x7f243006b750 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f2440004c40 tx=0x7f2440009040 comp rx=0 tx=0).ready entity=mgr.14210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:07:35.423 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:07:35.424+0000 7f2439ffb700 1 -- 172.21.15.162:0/1965139450 <== mon.0 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f2434057d60 con 0x7f24440fed30 2024-04-27T04:07:35.611 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:07:35.610+0000 7f244b855700 1 -- 172.21.15.162:0/1965139450 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command([{prefix=config set, name=mon_warn_on_insecure_global_id_reclaim_allowed}] v 0) v1 -- 0x7f244404bed0 con 0x7f24440fed30 2024-04-27T04:07:35.611 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:07:35.611+0000 7f2439ffb700 1 -- 172.21.15.162:0/1965139450 <== mon.0 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_warn_on_insecure_global_id_reclaim_allowed}]=0 v21)=0 v21) v1 ==== 163+0+0 (secure 0 0 0) 0x7f24340578f0 con 0x7f24440fed30 2024-04-27T04:07:35.614 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:07:35.613+0000 7f244b855700 1 -- 172.21.15.162:0/1965139450 >> [v2:172.21.15.162:6800/4079782599,v1:172.21.15.162:6801/4079782599] conn(0x7f24300692a0 msgr2=0x7f243006b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:07:35.614 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:07:35.613+0000 7f244b855700 1 --2- 172.21.15.162:0/1965139450 >> [v2:172.21.15.162:6800/4079782599,v1:172.21.15.162:6801/4079782599] conn(0x7f24300692a0 0x7f243006b750 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f2440004c40 tx=0x7f2440009040 comp rx=0 tx=0).stop 2024-04-27T04:07:35.614 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:07:35.613+0000 7f244b855700 1 -- 172.21.15.162:0/1965139450 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f24440fed30 msgr2=0x7f244410df70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:07:35.614 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:07:35.613+0000 7f244b855700 1 --2- 172.21.15.162:0/1965139450 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f24440fed30 0x7f244410df70 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f2434009150 tx=0x7f2434009e30 comp rx=0 tx=0).stop 2024-04-27T04:07:35.614 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:07:35.613+0000 7f244b855700 1 -- 172.21.15.162:0/1965139450 shutdown_connections 2024-04-27T04:07:35.614 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:07:35.613+0000 7f244b855700 1 --2- 172.21.15.162:0/1965139450 >> [v2:172.21.15.162:6800/4079782599,v1:172.21.15.162:6801/4079782599] conn(0x7f24300692a0 0x7f243006b750 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:07:35.614 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:07:35.613+0000 7f244b855700 1 --2- 172.21.15.162:0/1965139450 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f24440fe3e0 0x7f244410da30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:07:35.615 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:07:35.613+0000 7f244b855700 1 --2- 172.21.15.162:0/1965139450 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f24440fed30 0x7f244410df70 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:07:35.615 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:07:35.613+0000 7f244b855700 1 -- 172.21.15.162:0/1965139450 >> 172.21.15.162:0/1965139450 conn(0x7f24440f6b80 msgr2=0x7f24440f8280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:07:35.615 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:07:35.613+0000 7f244b855700 1 -- 172.21.15.162:0/1965139450 shutdown_connections 2024-04-27T04:07:35.615 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:07:35.613+0000 7f244b855700 1 -- 172.21.15.162:0/1965139450 wait complete. 2024-04-27T04:07:36.190 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:07:35 smithi162 ceph-mon[26841]: pgmap v193: 65 pgs: 65 active+clean; 585 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.4 KiB/s rd, 682 B/s wr, 2 op/s 2024-04-27T04:07:36.215 DEBUG:teuthology.orchestra.run.smithi162:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 7f9750d6-044a-11ef-bc93-c7b262605968 -e sha1=4e213a58e787437ff508e6c890ccd5fd32dc96b5 -- bash -c 'ceph config set global log_to_journald false --force' 2024-04-27T04:07:36.422 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:07:35 smithi181 ceph-mon[33774]: pgmap v193: 65 pgs: 65 active+clean; 585 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.4 KiB/s rd, 682 B/s wr, 2 op/s 2024-04-27T04:07:36.467 INFO:teuthology.orchestra.run.smithi162.stderr:Inferring config /var/lib/ceph/7f9750d6-044a-11ef-bc93-c7b262605968/mon.smithi162/config 2024-04-27T04:07:38.073 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:07:38.072+0000 7f76be302700 1 -- 172.21.15.162:0/235344762 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f76b80fdfd0 msgr2=0x7f76b80fe440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:07:38.073 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:07:38.072+0000 7f76be302700 1 --2- 172.21.15.162:0/235344762 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f76b80fdfd0 0x7f76b80fe440 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f76ac004940 tx=0x7f76ac009e30 comp rx=0 tx=0).stop 2024-04-27T04:07:38.074 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:07:38.072+0000 7f76be302700 1 -- 172.21.15.162:0/235344762 shutdown_connections 2024-04-27T04:07:38.074 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:07:38.072+0000 7f76be302700 1 --2- 172.21.15.162:0/235344762 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f76b80fdfd0 0x7f76b80fe440 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:07:38.074 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:07:38.072+0000 7f76be302700 1 --2- 172.21.15.162:0/235344762 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f76b80fce60 0x7f76b80fd270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:07:38.074 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:07:38.072+0000 7f76be302700 1 -- 172.21.15.162:0/235344762 >> 172.21.15.162:0/235344762 conn(0x7f76b80f8540 msgr2=0x7f76b80fa950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:07:38.074 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:07:38.072+0000 7f76be302700 1 -- 172.21.15.162:0/235344762 shutdown_connections 2024-04-27T04:07:38.074 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:07:38.072+0000 7f76be302700 1 -- 172.21.15.162:0/235344762 wait complete. 2024-04-27T04:07:38.078 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:07:38.072+0000 7f76be302700 1 Processor -- start 2024-04-27T04:07:38.078 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:07:38.072+0000 7f76be302700 1 -- start start 2024-04-27T04:07:38.078 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:07:38.073+0000 7f76be302700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f76b80fce60 0x7f76b8071c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:07:38.078 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:07:38.073+0000 7f76be302700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f76b80fdfd0 0x7f76b80702b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:07:38.078 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:07:38.073+0000 7f76be302700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76b80707f0 con 0x7f76b80fdfd0 2024-04-27T04:07:38.078 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:07:38.073+0000 7f76be302700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76b8070960 con 0x7f76b80fce60 2024-04-27T04:07:38.078 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:07:38.073+0000 7f76b77fe700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f76b80fdfd0 0x7f76b80702b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:07:38.078 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:07:38.073+0000 7f76b77fe700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f76b80fdfd0 0x7f76b80702b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:54436/0 (socket says 172.21.15.162:54436) 2024-04-27T04:07:38.078 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:07:38.073+0000 7f76b77fe700 1 -- 172.21.15.162:0/496977416 learned_addr learned my addr 172.21.15.162:0/496977416 (peer_addr_for_me v2:172.21.15.162:0/0) 2024-04-27T04:07:38.078 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:07:38.073+0000 7f76b7fff700 1 --2- 172.21.15.162:0/496977416 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f76b80fce60 0x7f76b8071c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:07:38.079 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:07:38.073+0000 7f76b77fe700 1 -- 172.21.15.162:0/496977416 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f76b80fce60 msgr2=0x7f76b8071c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:07:38.079 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:07:38.073+0000 7f76b77fe700 1 --2- 172.21.15.162:0/496977416 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f76b80fce60 0x7f76b8071c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:07:38.079 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:07:38.073+0000 7f76b77fe700 1 -- 172.21.15.162:0/496977416 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f76ac005040 con 0x7f76b80fdfd0 2024-04-27T04:07:38.079 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:07:38.073+0000 7f76b77fe700 1 --2- 172.21.15.162:0/496977416 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f76b80fdfd0 0x7f76b80702b0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f76ac00b3b0 tx=0x7f76ac009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:07:38.079 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:07:38.073+0000 7f76b4ff9700 1 -- 172.21.15.162:0/496977416 <== mon.0 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f76ac00e040 con 0x7f76b80fdfd0 2024-04-27T04:07:38.079 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:07:38.073+0000 7f76b4ff9700 1 -- 172.21.15.162:0/496977416 <== mon.0 v2:172.21.15.162:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f76ac00bbf0 con 0x7f76b80fdfd0 2024-04-27T04:07:38.079 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:07:38.073+0000 7f76b4ff9700 1 -- 172.21.15.162:0/496977416 <== mon.0 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f76ac01c630 con 0x7f76b80fdfd0 2024-04-27T04:07:38.079 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:07:38.073+0000 7f76be302700 1 -- 172.21.15.162:0/496977416 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f76b8070be0 con 0x7f76b80fdfd0 2024-04-27T04:07:38.079 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:07:38.073+0000 7f76be302700 1 -- 172.21.15.162:0/496977416 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f76b8115a10 con 0x7f76b80fdfd0 2024-04-27T04:07:38.080 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:07:38.075+0000 7f76be302700 1 -- 172.21.15.162:0/496977416 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f76b804c710 con 0x7f76b80fdfd0 2024-04-27T04:07:38.080 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:07:38.075+0000 7f76b4ff9700 1 -- 172.21.15.162:0/496977416 <== mon.0 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 19) v1 ==== 86631+0+0 (secure 0 0 0) 0x7f76ac00bd60 con 0x7f76b80fdfd0 2024-04-27T04:07:38.080 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:07:38.075+0000 7f76b4ff9700 1 --2- 172.21.15.162:0/496977416 >> [v2:172.21.15.162:6800/4079782599,v1:172.21.15.162:6801/4079782599] conn(0x7f76a00692a0 0x7f76a006b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:07:38.080 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:07:38.075+0000 7f76b4ff9700 1 -- 172.21.15.162:0/496977416 <== mon.0 v2:172.21.15.162:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5550+0+0 (secure 0 0 0) 0x7f76ac087dd0 con 0x7f76b80fdfd0 2024-04-27T04:07:38.080 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:07:38.075+0000 7f76b7fff700 1 --2- 172.21.15.162:0/496977416 >> [v2:172.21.15.162:6800/4079782599,v1:172.21.15.162:6801/4079782599] conn(0x7f76a00692a0 0x7f76a006b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:07:38.081 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:07:38.076+0000 7f76b7fff700 1 --2- 172.21.15.162:0/496977416 >> [v2:172.21.15.162:6800/4079782599,v1:172.21.15.162:6801/4079782599] conn(0x7f76a00692a0 0x7f76a006b750 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f76a8009bf0 tx=0x7f76a80091a0 comp rx=0 tx=0).ready entity=mgr.14210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:07:38.081 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:07:38.080+0000 7f76b4ff9700 1 -- 172.21.15.162:0/496977416 <== mon.0 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f76ac057be0 con 0x7f76b80fdfd0 2024-04-27T04:07:38.266 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:07:38.266+0000 7f76be302700 1 -- 172.21.15.162:0/496977416 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command([{prefix=config set, name=log_to_journald}] v 0) v1 -- 0x7f76b8115c80 con 0x7f76b80fdfd0 2024-04-27T04:07:38.267 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:07:37 smithi162 ceph-mon[26841]: pgmap v194: 65 pgs: 65 active+clean; 585 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.4 KiB/s rd, 682 B/s wr, 2 op/s 2024-04-27T04:07:38.267 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:07:38.266+0000 7f76b4ff9700 1 -- 172.21.15.162:0/496977416 <== mon.0 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{prefix=config set, name=log_to_journald}]=0 v21)=0 v21) v1 ==== 135+0+0 (secure 0 0 0) 0x7f76ac057770 con 0x7f76b80fdfd0 2024-04-27T04:07:38.269 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:07:38.269+0000 7f76be302700 1 -- 172.21.15.162:0/496977416 >> [v2:172.21.15.162:6800/4079782599,v1:172.21.15.162:6801/4079782599] conn(0x7f76a00692a0 msgr2=0x7f76a006b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:07:38.270 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:07:38.269+0000 7f76be302700 1 --2- 172.21.15.162:0/496977416 >> [v2:172.21.15.162:6800/4079782599,v1:172.21.15.162:6801/4079782599] conn(0x7f76a00692a0 0x7f76a006b750 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f76a8009bf0 tx=0x7f76a80091a0 comp rx=0 tx=0).stop 2024-04-27T04:07:38.270 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:07:38.269+0000 7f76be302700 1 -- 172.21.15.162:0/496977416 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f76b80fdfd0 msgr2=0x7f76b80702b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:07:38.270 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:07:38.269+0000 7f76be302700 1 --2- 172.21.15.162:0/496977416 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f76b80fdfd0 0x7f76b80702b0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f76ac00b3b0 tx=0x7f76ac009e30 comp rx=0 tx=0).stop 2024-04-27T04:07:38.270 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:07:38.269+0000 7f76be302700 1 -- 172.21.15.162:0/496977416 shutdown_connections 2024-04-27T04:07:38.270 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:07:38.269+0000 7f76be302700 1 --2- 172.21.15.162:0/496977416 >> [v2:172.21.15.162:6800/4079782599,v1:172.21.15.162:6801/4079782599] conn(0x7f76a00692a0 0x7f76a006b750 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:07:38.270 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:07:38.269+0000 7f76be302700 1 --2- 172.21.15.162:0/496977416 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f76b80fce60 0x7f76b8071c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:07:38.270 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:07:38.269+0000 7f76be302700 1 --2- 172.21.15.162:0/496977416 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f76b80fdfd0 0x7f76b80702b0 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:07:38.270 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:07:38.269+0000 7f76be302700 1 -- 172.21.15.162:0/496977416 >> 172.21.15.162:0/496977416 conn(0x7f76b80f8540 msgr2=0x7f76b80f9b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:07:38.270 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:07:38.269+0000 7f76be302700 1 -- 172.21.15.162:0/496977416 shutdown_connections 2024-04-27T04:07:38.270 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:07:38.269+0000 7f76be302700 1 -- 172.21.15.162:0/496977416 wait complete. 2024-04-27T04:07:38.422 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:07:37 smithi181 ceph-mon[33774]: pgmap v194: 65 pgs: 65 active+clean; 585 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.4 KiB/s rd, 682 B/s wr, 2 op/s 2024-04-27T04:07:38.985 DEBUG:teuthology.orchestra.run.smithi162:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 7f9750d6-044a-11ef-bc93-c7b262605968 -e sha1=4e213a58e787437ff508e6c890ccd5fd32dc96b5 -- bash -c 'ceph orch upgrade start --image quay.ceph.io/ceph-ci/ceph:$sha1 --daemon-types mgr' 2024-04-27T04:07:39.238 INFO:teuthology.orchestra.run.smithi162.stderr:Inferring config /var/lib/ceph/7f9750d6-044a-11ef-bc93-c7b262605968/mon.smithi162/config 2024-04-27T04:07:40.137 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:07:39 smithi162 ceph-mon[26841]: pgmap v195: 65 pgs: 65 active+clean; 585 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.8 KiB/s rd, 767 B/s wr, 3 op/s 2024-04-27T04:07:40.242 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:07:39 smithi181 ceph-mon[33774]: pgmap v195: 65 pgs: 65 active+clean; 585 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.8 KiB/s rd, 767 B/s wr, 3 op/s 2024-04-27T04:07:40.830 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:07:40.827+0000 7fb6ed3b8700 1 -- 172.21.15.162:0/3912656260 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb6e80fcfa0 msgr2=0x7fb6e80fd410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:07:40.830 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:07:40.827+0000 7fb6ed3b8700 1 --2- 172.21.15.162:0/3912656260 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb6e80fcfa0 0x7fb6e80fd410 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fb6d0004940 tx=0x7fb6d0009e30 comp rx=0 tx=0).stop 2024-04-27T04:07:40.830 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:07:40.827+0000 7fb6ed3b8700 1 -- 172.21.15.162:0/3912656260 shutdown_connections 2024-04-27T04:07:40.830 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:07:40.827+0000 7fb6ed3b8700 1 --2- 172.21.15.162:0/3912656260 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb6e80fcfa0 0x7fb6e80fd410 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:07:40.830 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:07:40.827+0000 7fb6ed3b8700 1 --2- 172.21.15.162:0/3912656260 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb6e80fbe30 0x7fb6e80fc240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:07:40.830 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:07:40.827+0000 7fb6ed3b8700 1 -- 172.21.15.162:0/3912656260 >> 172.21.15.162:0/3912656260 conn(0x7fb6e80f7440 msgr2=0x7fb6e80f9870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:07:40.830 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:07:40.827+0000 7fb6ed3b8700 1 -- 172.21.15.162:0/3912656260 shutdown_connections 2024-04-27T04:07:40.830 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:07:40.827+0000 7fb6ed3b8700 1 -- 172.21.15.162:0/3912656260 wait complete. 2024-04-27T04:07:40.830 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:07:40.827+0000 7fb6ed3b8700 1 Processor -- start 2024-04-27T04:07:40.830 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:07:40.827+0000 7fb6ed3b8700 1 -- start start 2024-04-27T04:07:40.831 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:07:40.828+0000 7fb6ed3b8700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb6e80fbe30 0x7fb6e8110730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:07:40.831 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:07:40.828+0000 7fb6ed3b8700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb6e80fcfa0 0x7fb6e8112c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:07:40.831 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:07:40.828+0000 7fb6ed3b8700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6e8066bc0 con 0x7fb6e80fcfa0 2024-04-27T04:07:40.831 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:07:40.828+0000 7fb6ed3b8700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6e8066d00 con 0x7fb6e80fbe30 2024-04-27T04:07:40.831 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:07:40.828+0000 7fb6de7fc700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb6e80fcfa0 0x7fb6e8112c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:07:40.831 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:07:40.828+0000 7fb6de7fc700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb6e80fcfa0 0x7fb6e8112c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:54446/0 (socket says 172.21.15.162:54446) 2024-04-27T04:07:40.831 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:07:40.828+0000 7fb6de7fc700 1 -- 172.21.15.162:0/3525381881 learned_addr learned my addr 172.21.15.162:0/3525381881 (peer_addr_for_me v2:172.21.15.162:0/0) 2024-04-27T04:07:40.831 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:07:40.828+0000 7fb6e6ffd700 1 --2- 172.21.15.162:0/3525381881 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb6e80fbe30 0x7fb6e8110730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:07:40.831 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:07:40.828+0000 7fb6de7fc700 1 -- 172.21.15.162:0/3525381881 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb6e80fbe30 msgr2=0x7fb6e8110730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:07:40.831 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:07:40.828+0000 7fb6de7fc700 1 --2- 172.21.15.162:0/3525381881 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb6e80fbe30 0x7fb6e8110730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:07:40.831 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:07:40.828+0000 7fb6de7fc700 1 -- 172.21.15.162:0/3525381881 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb6d0005040 con 0x7fb6e80fcfa0 2024-04-27T04:07:40.831 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:07:40.828+0000 7fb6e6ffd700 1 --2- 172.21.15.162:0/3525381881 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb6e80fbe30 0x7fb6e8110730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-27T04:07:40.831 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:07:40.828+0000 7fb6de7fc700 1 --2- 172.21.15.162:0/3525381881 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb6e80fcfa0 0x7fb6e8112c80 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fb6d000b3b0 tx=0x7fb6d0009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:07:40.831 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:07:40.828+0000 7fb6dffff700 1 -- 172.21.15.162:0/3525381881 <== mon.0 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb6d000e040 con 0x7fb6e80fcfa0 2024-04-27T04:07:40.832 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:07:40.828+0000 7fb6ed3b8700 1 -- 172.21.15.162:0/3525381881 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb6e81131c0 con 0x7fb6e80fcfa0 2024-04-27T04:07:40.832 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:07:40.828+0000 7fb6dffff700 1 -- 172.21.15.162:0/3525381881 <== mon.0 v2:172.21.15.162:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fb6d000bcc0 con 0x7fb6e80fcfa0 2024-04-27T04:07:40.832 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:07:40.828+0000 7fb6dffff700 1 -- 172.21.15.162:0/3525381881 <== mon.0 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb6d001c630 con 0x7fb6e80fcfa0 2024-04-27T04:07:40.832 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:07:40.828+0000 7fb6ed3b8700 1 -- 172.21.15.162:0/3525381881 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb6e8113780 con 0x7fb6e80fcfa0 2024-04-27T04:07:40.832 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:07:40.829+0000 7fb6ed3b8700 1 -- 172.21.15.162:0/3525381881 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb6e805f5c0 con 0x7fb6e80fcfa0 2024-04-27T04:07:40.832 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:07:40.830+0000 7fb6dffff700 1 -- 172.21.15.162:0/3525381881 <== mon.0 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 19) v1 ==== 86631+0+0 (secure 0 0 0) 0x7fb6d0013980 con 0x7fb6e80fcfa0 2024-04-27T04:07:40.833 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:07:40.830+0000 7fb6dffff700 1 --2- 172.21.15.162:0/3525381881 >> [v2:172.21.15.162:6800/4079782599,v1:172.21.15.162:6801/4079782599] conn(0x7fb6d40692a0 0x7fb6d406b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:07:40.833 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:07:40.830+0000 7fb6e6ffd700 1 --2- 172.21.15.162:0/3525381881 >> [v2:172.21.15.162:6800/4079782599,v1:172.21.15.162:6801/4079782599] conn(0x7fb6d40692a0 0x7fb6d406b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:07:40.833 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:07:40.830+0000 7fb6dffff700 1 -- 172.21.15.162:0/3525381881 <== mon.0 v2:172.21.15.162:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5550+0+0 (secure 0 0 0) 0x7fb6d0088010 con 0x7fb6e80fcfa0 2024-04-27T04:07:40.833 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:07:40.831+0000 7fb6e6ffd700 1 --2- 172.21.15.162:0/3525381881 >> [v2:172.21.15.162:6800/4079782599,v1:172.21.15.162:6801/4079782599] conn(0x7fb6d40692a0 0x7fb6d406b750 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fb6d80041d0 tx=0x7fb6d8009040 comp rx=0 tx=0).ready entity=mgr.14210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:07:40.834 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:07:40.835+0000 7fb6dffff700 1 -- 172.21.15.162:0/3525381881 <== mon.0 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7fb6d0054270 con 0x7fb6e80fcfa0 2024-04-27T04:07:41.041 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:07:41.041+0000 7fb6ed3b8700 1 -- 172.21.15.162:0/3525381881 --> [v2:172.21.15.162:6800/4079782599,v1:172.21.15.162:6801/4079782599] -- mgr_command(tid 0: {"prefix": "orch upgrade start", "image": "quay.ceph.io/ceph-ci/ceph:4e213a58e787437ff508e6c890ccd5fd32dc96b5", "daemon_types": "mgr", "target": ["mon-mgr", ""]}) v1 -- 0x7fb6e81012c0 con 0x7fb6d40692a0 2024-04-27T04:07:42.387 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:07:41 smithi162 ceph-mon[26841]: pgmap v196: 65 pgs: 65 active+clean; 585 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.3 KiB/s rd, 597 B/s wr, 2 op/s 2024-04-27T04:07:42.387 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:07:41 smithi162 ceph-mon[26841]: from='client.14550 -' entity='client.admin' cmd=[{"prefix": "orch upgrade start", "image": "quay.ceph.io/ceph-ci/ceph:4e213a58e787437ff508e6c890ccd5fd32dc96b5", "daemon_types": "mgr", "target": ["mon-mgr", ""]}]: dispatch 2024-04-27T04:07:42.422 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:07:41 smithi181 ceph-mon[33774]: pgmap v196: 65 pgs: 65 active+clean; 585 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.3 KiB/s rd, 597 B/s wr, 2 op/s 2024-04-27T04:07:42.423 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:07:41 smithi181 ceph-mon[33774]: from='client.14550 -' entity='client.admin' cmd=[{"prefix": "orch upgrade start", "image": "quay.ceph.io/ceph-ci/ceph:4e213a58e787437ff508e6c890ccd5fd32dc96b5", "daemon_types": "mgr", "target": ["mon-mgr", ""]}]: dispatch 2024-04-27T04:07:44.387 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:07:43 smithi162 ceph-mon[26841]: pgmap v197: 65 pgs: 65 active+clean; 588 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 1023 B/s wr, 3 op/s 2024-04-27T04:07:44.426 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:07:43 smithi181 ceph-mon[33774]: pgmap v197: 65 pgs: 65 active+clean; 588 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 1023 B/s wr, 3 op/s 2024-04-27T04:07:46.387 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:07:45 smithi162 ceph-mon[26841]: pgmap v198: 65 pgs: 65 active+clean; 588 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 2 op/s 2024-04-27T04:07:46.422 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:07:45 smithi181 ceph-mon[33774]: pgmap v198: 65 pgs: 65 active+clean; 588 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 2 op/s 2024-04-27T04:07:47.387 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:07:46 smithi162 ceph-mon[26841]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-27T04:07:47.422 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:07:46 smithi181 ceph-mon[33774]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-27T04:07:48.387 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:07:47 smithi162 ceph-mon[26841]: pgmap v199: 65 pgs: 65 active+clean; 588 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 2 op/s 2024-04-27T04:07:48.422 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:07:47 smithi181 ceph-mon[33774]: pgmap v199: 65 pgs: 65 active+clean; 588 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 2 op/s 2024-04-27T04:07:50.387 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:07:49 smithi162 ceph-mon[26841]: pgmap v200: 65 pgs: 65 active+clean; 588 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.6 KiB/s rd, 511 B/s wr, 2 op/s 2024-04-27T04:07:50.422 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:07:49 smithi181 ceph-mon[33774]: pgmap v200: 65 pgs: 65 active+clean; 588 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.6 KiB/s rd, 511 B/s wr, 2 op/s 2024-04-27T04:07:52.387 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:07:51 smithi162 ceph-mon[26841]: pgmap v201: 65 pgs: 65 active+clean; 588 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 426 B/s wr, 1 op/s 2024-04-27T04:07:52.423 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:07:51 smithi181 ceph-mon[33774]: pgmap v201: 65 pgs: 65 active+clean; 588 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 426 B/s wr, 1 op/s 2024-04-27T04:07:54.387 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:07:53 smithi162 ceph-mon[26841]: pgmap v202: 65 pgs: 65 active+clean; 588 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.6 KiB/s rd, 426 B/s wr, 2 op/s 2024-04-27T04:07:54.422 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:07:53 smithi181 ceph-mon[33774]: pgmap v202: 65 pgs: 65 active+clean; 588 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.6 KiB/s rd, 426 B/s wr, 2 op/s 2024-04-27T04:07:56.387 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:07:55 smithi162 ceph-mon[26841]: pgmap v203: 65 pgs: 65 active+clean; 588 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:07:56.422 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:07:55 smithi181 ceph-mon[33774]: pgmap v203: 65 pgs: 65 active+clean; 588 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:07:58.387 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:07:57 smithi162 ceph-mon[26841]: pgmap v204: 65 pgs: 65 active+clean; 588 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:07:58.422 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:07:57 smithi181 ceph-mon[33774]: pgmap v204: 65 pgs: 65 active+clean; 588 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:08:00.387 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:07:59 smithi162 ceph-mon[26841]: pgmap v205: 65 pgs: 65 active+clean; 588 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:08:00.422 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:07:59 smithi181 ceph-mon[33774]: pgmap v205: 65 pgs: 65 active+clean; 588 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:08:02.248 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:08:01 smithi181 ceph-mon[33774]: pgmap v206: 65 pgs: 65 active+clean; 588 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:08:02.248 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:08:01 smithi181 ceph-mon[33774]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-27T04:08:02.387 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:08:01 smithi162 ceph-mon[26841]: pgmap v206: 65 pgs: 65 active+clean; 588 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:08:02.387 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:08:01 smithi162 ceph-mon[26841]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-27T04:08:04.387 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:08:03 smithi162 ceph-mon[26841]: pgmap v207: 65 pgs: 65 active+clean; 588 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:08:04.422 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:08:03 smithi181 ceph-mon[33774]: pgmap v207: 65 pgs: 65 active+clean; 588 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:08:06.387 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:08:05 smithi162 ceph-mon[26841]: pgmap v208: 65 pgs: 65 active+clean; 588 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:08:06.422 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:08:05 smithi181 ceph-mon[33774]: pgmap v208: 65 pgs: 65 active+clean; 588 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:08:08.387 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:08:07 smithi162 ceph-mon[26841]: pgmap v209: 65 pgs: 65 active+clean; 588 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:08:08.422 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:08:07 smithi181 ceph-mon[33774]: pgmap v209: 65 pgs: 65 active+clean; 588 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:08:10.137 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:08:09 smithi162 ceph-mon[26841]: pgmap v210: 65 pgs: 65 active+clean; 588 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:08:10.422 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:08:09 smithi181 ceph-mon[33774]: pgmap v210: 65 pgs: 65 active+clean; 588 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:08:12.638 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:08:12 smithi162 ceph-mon[26841]: pgmap v211: 65 pgs: 65 active+clean; 588 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:08:12.672 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:08:12 smithi181 ceph-mon[33774]: pgmap v211: 65 pgs: 65 active+clean; 588 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:08:14.637 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:08:14 smithi162 ceph-mon[26841]: pgmap v212: 65 pgs: 65 active+clean; 588 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:08:14.672 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:08:14 smithi181 ceph-mon[33774]: pgmap v212: 65 pgs: 65 active+clean; 588 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:08:16.637 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:08:16 smithi162 ceph-mon[26841]: pgmap v213: 65 pgs: 65 active+clean; 588 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:08:16.672 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:08:16 smithi181 ceph-mon[33774]: pgmap v213: 65 pgs: 65 active+clean; 588 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:08:17.637 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:08:17 smithi162 ceph-mon[26841]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-27T04:08:17.672 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:08:17 smithi181 ceph-mon[33774]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-27T04:08:18.637 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:08:18 smithi162 ceph-mon[26841]: pgmap v214: 65 pgs: 65 active+clean; 588 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:08:18.672 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:08:18 smithi181 ceph-mon[33774]: pgmap v214: 65 pgs: 65 active+clean; 588 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:08:20.637 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:08:20 smithi162 ceph-mon[26841]: pgmap v215: 65 pgs: 65 active+clean; 588 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:08:20.672 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:08:20 smithi181 ceph-mon[33774]: pgmap v215: 65 pgs: 65 active+clean; 588 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:08:22.637 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:08:22 smithi162 ceph-mon[26841]: pgmap v216: 65 pgs: 65 active+clean; 588 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:08:22.637 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:08:22 smithi162 ceph-mon[26841]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T04:08:22.672 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:08:22 smithi181 ceph-mon[33774]: pgmap v216: 65 pgs: 65 active+clean; 588 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:08:22.673 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:08:22 smithi181 ceph-mon[33774]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T04:08:23.637 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:08:23 smithi162 ceph-mon[26841]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T04:08:23.637 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:08:23 smithi162 ceph-mon[26841]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T04:08:23.637 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:08:23 smithi162 ceph-mon[26841]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' 2024-04-27T04:08:23.672 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:08:23 smithi181 ceph-mon[33774]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T04:08:23.672 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:08:23 smithi181 ceph-mon[33774]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T04:08:23.673 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:08:23 smithi181 ceph-mon[33774]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' 2024-04-27T04:08:24.637 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:08:24 smithi162 ceph-mon[26841]: pgmap v217: 65 pgs: 65 active+clean; 588 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:08:24.672 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:08:24 smithi181 ceph-mon[33774]: pgmap v217: 65 pgs: 65 active+clean; 588 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:08:26.637 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:08:26 smithi162 ceph-mon[26841]: pgmap v218: 65 pgs: 65 active+clean; 588 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:08:26.672 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:08:26 smithi181 ceph-mon[33774]: pgmap v218: 65 pgs: 65 active+clean; 588 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:08:28.637 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:08:28 smithi162 ceph-mon[26841]: pgmap v219: 65 pgs: 65 active+clean; 588 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:08:28.672 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:08:28 smithi181 ceph-mon[33774]: pgmap v219: 65 pgs: 65 active+clean; 588 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:08:30.599 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:08:30 smithi162 ceph-mon[26841]: pgmap v220: 65 pgs: 65 active+clean; 588 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:08:30.672 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:08:30 smithi181 ceph-mon[33774]: pgmap v220: 65 pgs: 65 active+clean; 588 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:08:32.637 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:08:32 smithi162 ceph-mon[26841]: pgmap v221: 65 pgs: 65 active+clean; 588 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:08:32.637 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:08:32 smithi162 ceph-mon[26841]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-27T04:08:32.672 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:08:32 smithi181 ceph-mon[33774]: pgmap v221: 65 pgs: 65 active+clean; 588 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:08:32.672 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:08:32 smithi181 ceph-mon[33774]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-27T04:08:34.637 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:08:34 smithi162 ceph-mon[26841]: pgmap v222: 65 pgs: 65 active+clean; 588 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:08:34.672 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:08:34 smithi181 ceph-mon[33774]: pgmap v222: 65 pgs: 65 active+clean; 588 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:08:36.514 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:08:36 smithi162 ceph-mon[26841]: pgmap v223: 65 pgs: 65 active+clean; 588 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:08:36.672 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:08:36 smithi181 ceph-mon[33774]: pgmap v223: 65 pgs: 65 active+clean; 588 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:08:37.605 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:08:37.605+0000 7fb6dffff700 1 -- 172.21.15.162:0/3525381881 <== mgr.14210 v2:172.21.15.162:6800/4079782599 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+89 (secure 0 0 0) 0x7fb6e81012c0 con 0x7fb6d40692a0 2024-04-27T04:08:37.605 INFO:teuthology.orchestra.run.smithi162.stdout:Initiating upgrade to quay.ceph.io/ceph-ci/ceph:4e213a58e787437ff508e6c890ccd5fd32dc96b5 2024-04-27T04:08:37.607 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:08:37.607+0000 7fb6ed3b8700 1 -- 172.21.15.162:0/3525381881 >> [v2:172.21.15.162:6800/4079782599,v1:172.21.15.162:6801/4079782599] conn(0x7fb6d40692a0 msgr2=0x7fb6d406b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:08:37.608 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:08:37.607+0000 7fb6ed3b8700 1 --2- 172.21.15.162:0/3525381881 >> [v2:172.21.15.162:6800/4079782599,v1:172.21.15.162:6801/4079782599] conn(0x7fb6d40692a0 0x7fb6d406b750 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fb6d80041d0 tx=0x7fb6d8009040 comp rx=0 tx=0).stop 2024-04-27T04:08:37.608 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:08:37.607+0000 7fb6ed3b8700 1 -- 172.21.15.162:0/3525381881 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb6e80fcfa0 msgr2=0x7fb6e8112c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:08:37.608 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:08:37.607+0000 7fb6ed3b8700 1 --2- 172.21.15.162:0/3525381881 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb6e80fcfa0 0x7fb6e8112c80 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fb6d000b3b0 tx=0x7fb6d0009e30 comp rx=0 tx=0).stop 2024-04-27T04:08:37.608 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:08:37.607+0000 7fb6ed3b8700 1 -- 172.21.15.162:0/3525381881 shutdown_connections 2024-04-27T04:08:37.608 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:08:37.607+0000 7fb6ed3b8700 1 --2- 172.21.15.162:0/3525381881 >> [v2:172.21.15.162:6800/4079782599,v1:172.21.15.162:6801/4079782599] conn(0x7fb6d40692a0 0x7fb6d406b750 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:08:37.608 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:08:37.607+0000 7fb6ed3b8700 1 --2- 172.21.15.162:0/3525381881 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb6e80fbe30 0x7fb6e8110730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:08:37.608 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:08:37.607+0000 7fb6ed3b8700 1 --2- 172.21.15.162:0/3525381881 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb6e80fcfa0 0x7fb6e8112c80 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:08:37.608 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:08:37.607+0000 7fb6ed3b8700 1 -- 172.21.15.162:0/3525381881 >> 172.21.15.162:0/3525381881 conn(0x7fb6e80f7440 msgr2=0x7fb6e80f8b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:08:37.608 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:08:37.608+0000 7fb6ed3b8700 1 -- 172.21.15.162:0/3525381881 shutdown_connections 2024-04-27T04:08:37.608 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:08:37.608+0000 7fb6ed3b8700 1 -- 172.21.15.162:0/3525381881 wait complete. 2024-04-27T04:08:38.265 DEBUG:teuthology.orchestra.run.smithi162:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 7f9750d6-044a-11ef-bc93-c7b262605968 -e sha1=4e213a58e787437ff508e6c890ccd5fd32dc96b5 -- bash -c 'while ceph orch upgrade status | jq '"'"'.in_progress'"'"' | grep true && ! ceph orch upgrade status | jq '"'"'.message'"'"' | grep Error ; do ceph orch ps ; ceph versions ; ceph orch upgrade status ; sleep 30 ; done' 2024-04-27T04:08:38.575 INFO:teuthology.orchestra.run.smithi162.stderr:Inferring config /var/lib/ceph/7f9750d6-044a-11ef-bc93-c7b262605968/mon.smithi162/config 2024-04-27T04:08:38.637 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:08:38 smithi162 ceph-mon[26841]: pgmap v224: 65 pgs: 65 active+clean; 588 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:08:38.638 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:08:38 smithi162 ceph-mon[26841]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' 2024-04-27T04:08:38.638 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:08:38 smithi162 ceph-mon[26841]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T04:08:38.638 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:08:38 smithi162 ceph-mon[26841]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T04:08:38.638 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:08:38 smithi162 ceph-mon[26841]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T04:08:38.638 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:08:38 smithi162 ceph-mon[26841]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' 2024-04-27T04:08:38.672 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:08:38 smithi181 ceph-mon[33774]: pgmap v224: 65 pgs: 65 active+clean; 588 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:08:38.673 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:08:38 smithi181 ceph-mon[33774]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' 2024-04-27T04:08:38.673 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:08:38 smithi181 ceph-mon[33774]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T04:08:38.673 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:08:38 smithi181 ceph-mon[33774]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T04:08:38.673 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:08:38 smithi181 ceph-mon[33774]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T04:08:38.673 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:08:38 smithi181 ceph-mon[33774]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' 2024-04-27T04:08:39.596 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:08:39 smithi162 ceph-mon[26841]: Upgrade: Started with target quay.ceph.io/ceph-ci/ceph:4e213a58e787437ff508e6c890ccd5fd32dc96b5 2024-04-27T04:08:39.596 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:08:39 smithi162 ceph-mon[26841]: Upgrade: First pull of quay.ceph.io/ceph-ci/ceph:4e213a58e787437ff508e6c890ccd5fd32dc96b5 2024-04-27T04:08:39.672 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:08:39 smithi181 ceph-mon[33774]: Upgrade: Started with target quay.ceph.io/ceph-ci/ceph:4e213a58e787437ff508e6c890ccd5fd32dc96b5 2024-04-27T04:08:39.672 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:08:39 smithi181 ceph-mon[33774]: Upgrade: First pull of quay.ceph.io/ceph-ci/ceph:4e213a58e787437ff508e6c890ccd5fd32dc96b5 2024-04-27T04:08:40.510 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:08:40 smithi162 ceph-mon[26841]: pgmap v225: 65 pgs: 65 active+clean; 588 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:08:40.672 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:08:40 smithi181 ceph-mon[33774]: pgmap v225: 65 pgs: 65 active+clean; 588 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:08:40.957 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:08:40.728+0000 7fe730b5a700 1 -- 172.21.15.162:0/1768798782 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe72c0fbcb0 msgr2=0x7fe72c0fc0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:08:40.957 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:08:40.728+0000 7fe730b5a700 1 --2- 172.21.15.162:0/1768798782 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe72c0fbcb0 0x7fe72c0fc0c0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fe71c004940 tx=0x7fe71c009e30 comp rx=0 tx=0).stop 2024-04-27T04:08:40.957 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:08:40.728+0000 7fe730b5a700 1 -- 172.21.15.162:0/1768798782 shutdown_connections 2024-04-27T04:08:40.957 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:08:40.728+0000 7fe730b5a700 1 --2- 172.21.15.162:0/1768798782 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe72c0fcf60 0x7fe72c0fd3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:08:40.957 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:08:40.728+0000 7fe730b5a700 1 --2- 172.21.15.162:0/1768798782 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe72c0fbcb0 0x7fe72c0fc0c0 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:08:40.957 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:08:40.728+0000 7fe730b5a700 1 -- 172.21.15.162:0/1768798782 >> 172.21.15.162:0/1768798782 conn(0x7fe72c0f7430 msgr2=0x7fe72c0f9840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:08:40.957 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:08:40.728+0000 7fe730b5a700 1 -- 172.21.15.162:0/1768798782 shutdown_connections 2024-04-27T04:08:40.957 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:08:40.728+0000 7fe730b5a700 1 -- 172.21.15.162:0/1768798782 wait complete. 2024-04-27T04:08:40.957 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:08:40.729+0000 7fe730b5a700 1 Processor -- start 2024-04-27T04:08:40.957 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:08:40.729+0000 7fe730b5a700 1 -- start start 2024-04-27T04:08:40.957 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:08:40.729+0000 7fe730b5a700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe72c0fbcb0 0x7fe72c114020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:08:40.958 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:08:40.729+0000 7fe730b5a700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe72c0fcf60 0x7fe72c114560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:08:40.958 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:08:40.730+0000 7fe730b5a700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe72c114b60 con 0x7fe72c0fcf60 2024-04-27T04:08:40.958 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:08:40.730+0000 7fe730b5a700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe72c1153b0 con 0x7fe72c0fbcb0 2024-04-27T04:08:40.958 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:08:40.730+0000 7fe729d9b700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe72c0fcf60 0x7fe72c114560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:08:40.958 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:08:40.730+0000 7fe729d9b700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe72c0fcf60 0x7fe72c114560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:54016/0 (socket says 172.21.15.162:54016) 2024-04-27T04:08:40.958 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:08:40.730+0000 7fe729d9b700 1 -- 172.21.15.162:0/605708689 learned_addr learned my addr 172.21.15.162:0/605708689 (peer_addr_for_me v2:172.21.15.162:0/0) 2024-04-27T04:08:40.958 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:08:40.730+0000 7fe72a59c700 1 --2- 172.21.15.162:0/605708689 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe72c0fbcb0 0x7fe72c114020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:08:40.958 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:08:40.730+0000 7fe729d9b700 1 -- 172.21.15.162:0/605708689 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe72c0fbcb0 msgr2=0x7fe72c114020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:08:40.958 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:08:40.730+0000 7fe729d9b700 1 --2- 172.21.15.162:0/605708689 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe72c0fbcb0 0x7fe72c114020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:08:40.958 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:08:40.730+0000 7fe729d9b700 1 -- 172.21.15.162:0/605708689 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe71c005040 con 0x7fe72c0fcf60 2024-04-27T04:08:40.958 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:08:40.730+0000 7fe729d9b700 1 --2- 172.21.15.162:0/605708689 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe72c0fcf60 0x7fe72c114560 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fe72000c8c0 tx=0x7fe7200044c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:08:40.958 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:08:40.731+0000 7fe71affd700 1 -- 172.21.15.162:0/605708689 <== mon.0 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe72000a980 con 0x7fe72c0fcf60 2024-04-27T04:08:40.958 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:08:40.731+0000 7fe71affd700 1 -- 172.21.15.162:0/605708689 <== mon.0 v2:172.21.15.162:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fe72000ce60 con 0x7fe72c0fcf60 2024-04-27T04:08:40.959 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:08:40.731+0000 7fe71affd700 1 -- 172.21.15.162:0/605708689 <== mon.0 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe7200115e0 con 0x7fe72c0fcf60 2024-04-27T04:08:40.959 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:08:40.731+0000 7fe730b5a700 1 -- 172.21.15.162:0/605708689 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe72c115690 con 0x7fe72c0fcf60 2024-04-27T04:08:40.959 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:08:40.731+0000 7fe730b5a700 1 -- 172.21.15.162:0/605708689 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe72c115b80 con 0x7fe72c0fcf60 2024-04-27T04:08:40.959 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:08:40.732+0000 7fe730b5a700 1 -- 172.21.15.162:0/605708689 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe72c1018f0 con 0x7fe72c0fcf60 2024-04-27T04:08:40.959 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:08:40.733+0000 7fe71affd700 1 -- 172.21.15.162:0/605708689 <== mon.0 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 19) v1 ==== 86631+0+0 (secure 0 0 0) 0x7fe720012400 con 0x7fe72c0fcf60 2024-04-27T04:08:40.959 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:08:40.734+0000 7fe71affd700 1 --2- 172.21.15.162:0/605708689 >> [v2:172.21.15.162:6800/4079782599,v1:172.21.15.162:6801/4079782599] conn(0x7fe714069300 0x7fe71406b7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:08:40.959 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:08:40.734+0000 7fe71affd700 1 -- 172.21.15.162:0/605708689 <== mon.0 v2:172.21.15.162:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5550+0+0 (secure 0 0 0) 0x7fe720087390 con 0x7fe72c0fcf60 2024-04-27T04:08:40.959 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:08:40.734+0000 7fe72a59c700 1 --2- 172.21.15.162:0/605708689 >> [v2:172.21.15.162:6800/4079782599,v1:172.21.15.162:6801/4079782599] conn(0x7fe714069300 0x7fe71406b7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:08:40.959 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:08:40.735+0000 7fe72a59c700 1 --2- 172.21.15.162:0/605708689 >> [v2:172.21.15.162:6800/4079782599,v1:172.21.15.162:6801/4079782599] conn(0x7fe714069300 0x7fe71406b7b0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fe71c00b970 tx=0x7fe71c009e30 comp rx=0 tx=0).ready entity=mgr.14210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:08:40.959 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:08:40.742+0000 7fe71affd700 1 -- 172.21.15.162:0/605708689 <== mon.0 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7fe720057060 con 0x7fe72c0fcf60 2024-04-27T04:08:41.037 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:08:41.037+0000 7fe730b5a700 1 -- 172.21.15.162:0/605708689 --> [v2:172.21.15.162:6800/4079782599,v1:172.21.15.162:6801/4079782599] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fe72c115eb0 con 0x7fe714069300 2024-04-27T04:08:41.038 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:08:41.039+0000 7fe71affd700 1 -- 172.21.15.162:0/605708689 <== mgr.14210 v2:172.21.15.162:6800/4079782599 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+351 (secure 0 0 0) 0x7fe72c115eb0 con 0x7fe714069300 2024-04-27T04:08:41.042 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:08:41.042+0000 7fe730b5a700 1 -- 172.21.15.162:0/605708689 >> [v2:172.21.15.162:6800/4079782599,v1:172.21.15.162:6801/4079782599] conn(0x7fe714069300 msgr2=0x7fe71406b7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:08:41.042 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:08:41.042+0000 7fe730b5a700 1 --2- 172.21.15.162:0/605708689 >> [v2:172.21.15.162:6800/4079782599,v1:172.21.15.162:6801/4079782599] conn(0x7fe714069300 0x7fe71406b7b0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fe71c00b970 tx=0x7fe71c009e30 comp rx=0 tx=0).stop 2024-04-27T04:08:41.042 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:08:41.042+0000 7fe730b5a700 1 -- 172.21.15.162:0/605708689 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe72c0fcf60 msgr2=0x7fe72c114560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:08:41.042 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:08:41.042+0000 7fe730b5a700 1 --2- 172.21.15.162:0/605708689 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe72c0fcf60 0x7fe72c114560 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fe72000c8c0 tx=0x7fe7200044c0 comp rx=0 tx=0).stop 2024-04-27T04:08:41.042 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:08:41.042+0000 7fe730b5a700 1 -- 172.21.15.162:0/605708689 shutdown_connections 2024-04-27T04:08:41.042 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:08:41.042+0000 7fe730b5a700 1 --2- 172.21.15.162:0/605708689 >> [v2:172.21.15.162:6800/4079782599,v1:172.21.15.162:6801/4079782599] conn(0x7fe714069300 0x7fe71406b7b0 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:08:41.042 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:08:41.042+0000 7fe730b5a700 1 --2- 172.21.15.162:0/605708689 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe72c0fbcb0 0x7fe72c114020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:08:41.042 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:08:41.042+0000 7fe730b5a700 1 --2- 172.21.15.162:0/605708689 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe72c0fcf60 0x7fe72c114560 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:08:41.043 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:08:41.042+0000 7fe730b5a700 1 -- 172.21.15.162:0/605708689 >> 172.21.15.162:0/605708689 conn(0x7fe72c0f7430 msgr2=0x7fe72c065b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:08:41.043 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:08:41.042+0000 7fe730b5a700 1 -- 172.21.15.162:0/605708689 shutdown_connections 2024-04-27T04:08:41.043 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:08:41.042+0000 7fe730b5a700 1 -- 172.21.15.162:0/605708689 wait complete. 2024-04-27T04:08:41.057 INFO:teuthology.orchestra.run.smithi162.stdout:true 2024-04-27T04:08:41.154 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:08:41.154+0000 7f92b159a700 1 -- 172.21.15.162:0/1758560131 <== mon.0 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f92a4013b70 con 0x7f92b4066e90 2024-04-27T04:08:41.156 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:08:41.155+0000 7f92b89e3700 1 -- 172.21.15.162:0/1758560131 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f92b4066e90 msgr2=0x7f92b40ffa40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:08:41.156 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:08:41.155+0000 7f92b89e3700 1 --2- 172.21.15.162:0/1758560131 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f92b4066e90 0x7f92b40ffa40 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f92a4004940 tx=0x7f92a4009e30 comp rx=0 tx=0).stop 2024-04-27T04:08:41.156 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:08:41.155+0000 7f92b89e3700 1 -- 172.21.15.162:0/1758560131 shutdown_connections 2024-04-27T04:08:41.156 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:08:41.155+0000 7f92b89e3700 1 --2- 172.21.15.162:0/1758560131 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f92b40fff80 0x7f92b4102360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:08:41.156 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:08:41.155+0000 7f92b89e3700 1 --2- 172.21.15.162:0/1758560131 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f92b4066e90 0x7f92b40ffa40 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:08:41.156 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:08:41.155+0000 7f92b89e3700 1 -- 172.21.15.162:0/1758560131 >> 172.21.15.162:0/1758560131 conn(0x7f92b40f7450 msgr2=0x7f92b40f9860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:08:41.157 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:08:41.156+0000 7f92b89e3700 1 -- 172.21.15.162:0/1758560131 shutdown_connections 2024-04-27T04:08:41.157 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:08:41.156+0000 7f92b89e3700 1 -- 172.21.15.162:0/1758560131 wait complete. 2024-04-27T04:08:41.157 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:08:41.157+0000 7f92b89e3700 1 Processor -- start 2024-04-27T04:08:41.157 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:08:41.157+0000 7f92b89e3700 1 -- start start 2024-04-27T04:08:41.158 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:08:41.157+0000 7f92b89e3700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f92b4066e90 0x7f92b410d8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:08:41.158 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:08:41.157+0000 7f92b89e3700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f92b40fff80 0x7f92b410de00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:08:41.158 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:08:41.157+0000 7f92b89e3700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92b410e400 con 0x7f92b4066e90 2024-04-27T04:08:41.158 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:08:41.157+0000 7f92b89e3700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92b410e570 con 0x7f92b40fff80 2024-04-27T04:08:41.158 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:08:41.157+0000 7f92b259c700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f92b4066e90 0x7f92b410d8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:08:41.158 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:08:41.157+0000 7f92b259c700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f92b4066e90 0x7f92b410d8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:54026/0 (socket says 172.21.15.162:54026) 2024-04-27T04:08:41.158 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:08:41.157+0000 7f92b259c700 1 -- 172.21.15.162:0/4020914447 learned_addr learned my addr 172.21.15.162:0/4020914447 (peer_addr_for_me v2:172.21.15.162:0/0) 2024-04-27T04:08:41.158 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:08:41.157+0000 7f92b1d9b700 1 --2- 172.21.15.162:0/4020914447 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f92b40fff80 0x7f92b410de00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:08:41.158 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:08:41.158+0000 7f92b259c700 1 -- 172.21.15.162:0/4020914447 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f92b40fff80 msgr2=0x7f92b410de00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:08:41.158 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:08:41.158+0000 7f92b259c700 1 --2- 172.21.15.162:0/4020914447 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f92b40fff80 0x7f92b410de00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:08:41.158 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:08:41.158+0000 7f92b259c700 1 -- 172.21.15.162:0/4020914447 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f92a4005040 con 0x7f92b4066e90 2024-04-27T04:08:41.158 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:08:41.158+0000 7f92b1d9b700 1 --2- 172.21.15.162:0/4020914447 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f92b40fff80 0x7f92b410de00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-27T04:08:41.158 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:08:41.158+0000 7f92b259c700 1 --2- 172.21.15.162:0/4020914447 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f92b4066e90 0x7f92b410d8c0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f92a4014f90 tx=0x7f92a4009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:08:41.162 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:08:41.158+0000 7f92a2ffd700 1 -- 172.21.15.162:0/4020914447 <== mon.0 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f92a400e040 con 0x7f92b4066e90 2024-04-27T04:08:41.162 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:08:41.158+0000 7f92a2ffd700 1 -- 172.21.15.162:0/4020914447 <== mon.0 v2:172.21.15.162:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f92a4009370 con 0x7f92b4066e90 2024-04-27T04:08:41.162 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:08:41.158+0000 7f92a2ffd700 1 -- 172.21.15.162:0/4020914447 <== mon.0 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f92a4013b70 con 0x7f92b4066e90 2024-04-27T04:08:41.162 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:08:41.159+0000 7f92b89e3700 1 -- 172.21.15.162:0/4020914447 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f92b40fb820 con 0x7f92b4066e90 2024-04-27T04:08:41.162 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:08:41.159+0000 7f92b89e3700 1 -- 172.21.15.162:0/4020914447 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f92b40fbd00 con 0x7f92b4066e90 2024-04-27T04:08:41.162 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:08:41.160+0000 7f92b89e3700 1 -- 172.21.15.162:0/4020914447 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f92940052e0 con 0x7f92b4066e90 2024-04-27T04:08:41.169 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:08:41.161+0000 7f92a2ffd700 1 -- 172.21.15.162:0/4020914447 <== mon.0 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 19) v1 ==== 86631+0+0 (secure 0 0 0) 0x7f92a401cc00 con 0x7f92b4066e90 2024-04-27T04:08:41.170 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:08:41.161+0000 7f92a2ffd700 1 --2- 172.21.15.162:0/4020914447 >> [v2:172.21.15.162:6800/4079782599,v1:172.21.15.162:6801/4079782599] conn(0x7f929c0692a0 0x7f929c06b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:08:41.170 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:08:41.166+0000 7f92b1d9b700 1 --2- 172.21.15.162:0/4020914447 >> [v2:172.21.15.162:6800/4079782599,v1:172.21.15.162:6801/4079782599] conn(0x7f929c0692a0 0x7f929c06b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:08:41.170 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:08:41.170+0000 7f92a2ffd700 1 -- 172.21.15.162:0/4020914447 <== mon.0 v2:172.21.15.162:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5550+0+0 (secure 0 0 0) 0x7f92a4088730 con 0x7f92b4066e90 2024-04-27T04:08:41.171 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:08:41.170+0000 7f92b1d9b700 1 --2- 172.21.15.162:0/4020914447 >> [v2:172.21.15.162:6800/4079782599,v1:172.21.15.162:6801/4079782599] conn(0x7f929c0692a0 0x7f929c06b750 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f92a8009b80 tx=0x7f92a800b040 comp rx=0 tx=0).ready entity=mgr.14210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:08:41.171 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:08:41.170+0000 7f92a2ffd700 1 -- 172.21.15.162:0/4020914447 <== mon.0 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f92a4058540 con 0x7f92b4066e90 2024-04-27T04:08:41.386 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:08:41.386+0000 7f92b89e3700 1 -- 172.21.15.162:0/4020914447 --> [v2:172.21.15.162:6800/4079782599,v1:172.21.15.162:6801/4079782599] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f9294000bf0 con 0x7f929c0692a0 2024-04-27T04:08:41.387 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:08:41.388+0000 7f92a2ffd700 1 -- 172.21.15.162:0/4020914447 <== mgr.14210 v2:172.21.15.162:6800/4079782599 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+351 (secure 0 0 0) 0x7f9294000bf0 con 0x7f929c0692a0 2024-04-27T04:08:41.390 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:08:41.390+0000 7f92b89e3700 1 -- 172.21.15.162:0/4020914447 >> [v2:172.21.15.162:6800/4079782599,v1:172.21.15.162:6801/4079782599] conn(0x7f929c0692a0 msgr2=0x7f929c06b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:08:41.390 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:08:41.390+0000 7f92b89e3700 1 --2- 172.21.15.162:0/4020914447 >> [v2:172.21.15.162:6800/4079782599,v1:172.21.15.162:6801/4079782599] conn(0x7f929c0692a0 0x7f929c06b750 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f92a8009b80 tx=0x7f92a800b040 comp rx=0 tx=0).stop 2024-04-27T04:08:41.390 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:08:41.390+0000 7f92b89e3700 1 -- 172.21.15.162:0/4020914447 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f92b4066e90 msgr2=0x7f92b410d8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:08:41.390 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:08:41.390+0000 7f92b89e3700 1 --2- 172.21.15.162:0/4020914447 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f92b4066e90 0x7f92b410d8c0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f92a4014f90 tx=0x7f92a4009e30 comp rx=0 tx=0).stop 2024-04-27T04:08:41.390 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:08:41.390+0000 7f92b89e3700 1 -- 172.21.15.162:0/4020914447 shutdown_connections 2024-04-27T04:08:41.391 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:08:41.390+0000 7f92b89e3700 1 --2- 172.21.15.162:0/4020914447 >> [v2:172.21.15.162:6800/4079782599,v1:172.21.15.162:6801/4079782599] conn(0x7f929c0692a0 0x7f929c06b750 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:08:41.391 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:08:41.390+0000 7f92b89e3700 1 --2- 172.21.15.162:0/4020914447 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f92b4066e90 0x7f92b410d8c0 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:08:41.391 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:08:41.390+0000 7f92b89e3700 1 --2- 172.21.15.162:0/4020914447 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f92b40fff80 0x7f92b410de00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:08:41.391 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:08:41.390+0000 7f92b89e3700 1 -- 172.21.15.162:0/4020914447 >> 172.21.15.162:0/4020914447 conn(0x7f92b40f7450 msgr2=0x7f92b40f7820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:08:41.391 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:08:41.391+0000 7f92b89e3700 1 -- 172.21.15.162:0/4020914447 shutdown_connections 2024-04-27T04:08:41.391 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:08:41.391+0000 7f92b89e3700 1 -- 172.21.15.162:0/4020914447 wait complete. 2024-04-27T04:08:41.502 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:08:41.502+0000 7ff57f5d5700 1 -- 172.21.15.162:0/1995607769 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff5781002e0 msgr2=0x7ff5781026c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:08:41.502 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:08:41.502+0000 7ff57f5d5700 1 --2- 172.21.15.162:0/1995607769 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff5781002e0 0x7ff5781026c0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7ff568004940 tx=0x7ff568009e30 comp rx=0 tx=0).stop 2024-04-27T04:08:41.502 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:08:41.502+0000 7ff57f5d5700 1 -- 172.21.15.162:0/1995607769 shutdown_connections 2024-04-27T04:08:41.503 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:08:41.502+0000 7ff57f5d5700 1 --2- 172.21.15.162:0/1995607769 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff5781002e0 0x7ff5781026c0 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:08:41.503 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:08:41.502+0000 7ff57f5d5700 1 --2- 172.21.15.162:0/1995607769 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff5780fd9c0 0x7ff5780ffda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:08:41.503 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:08:41.502+0000 7ff57f5d5700 1 -- 172.21.15.162:0/1995607769 >> 172.21.15.162:0/1995607769 conn(0x7ff5780f7440 msgr2=0x7ff5780f9870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:08:41.504 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:08:41.503+0000 7ff57f5d5700 1 -- 172.21.15.162:0/1995607769 shutdown_connections 2024-04-27T04:08:41.504 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:08:41.503+0000 7ff57f5d5700 1 -- 172.21.15.162:0/1995607769 wait complete. 2024-04-27T04:08:41.504 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:08:41.503+0000 7ff57f5d5700 1 Processor -- start 2024-04-27T04:08:41.504 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:08:41.504+0000 7ff57f5d5700 1 -- start start 2024-04-27T04:08:41.504 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:08:41.504+0000 7ff57f5d5700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff5780fd9c0 0x7ff578116530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:08:41.504 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:08:41.504+0000 7ff57f5d5700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff5781002e0 0x7ff578116a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:08:41.504 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:08:41.504+0000 7ff57f5d5700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff578117070 con 0x7ff5780fd9c0 2024-04-27T04:08:41.505 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:08:41.504+0000 7ff57f5d5700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff578117370 con 0x7ff5781002e0 2024-04-27T04:08:41.505 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:08:41.504+0000 7ff57cb70700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff5781002e0 0x7ff578116a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:08:41.505 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:08:41.504+0000 7ff57cb70700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff5781002e0 0x7ff578116a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.162:34100/0 (socket says 172.21.15.162:34100) 2024-04-27T04:08:41.505 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:08:41.504+0000 7ff57cb70700 1 -- 172.21.15.162:0/2010209211 learned_addr learned my addr 172.21.15.162:0/2010209211 (peer_addr_for_me v2:172.21.15.162:0/0) 2024-04-27T04:08:41.505 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:08:41.505+0000 7ff57cb70700 1 -- 172.21.15.162:0/2010209211 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff5780fd9c0 msgr2=0x7ff578116530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:08:41.505 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:08:41.505+0000 7ff57cb70700 1 --2- 172.21.15.162:0/2010209211 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff5780fd9c0 0x7ff578116530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:08:41.506 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:08:41.505+0000 7ff57cb70700 1 -- 172.21.15.162:0/2010209211 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff568005040 con 0x7ff5781002e0 2024-04-27T04:08:41.506 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:08:41.505+0000 7ff57cb70700 1 --2- 172.21.15.162:0/2010209211 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff5781002e0 0x7ff578116a70 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7ff568009040 tx=0x7ff568009e30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:08:41.508 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:08:41.509+0000 7ff56dffb700 1 -- 172.21.15.162:0/2010209211 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff568021070 con 0x7ff5781002e0 2024-04-27T04:08:41.508 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:08:41.509+0000 7ff56dffb700 1 -- 172.21.15.162:0/2010209211 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7ff56801c8d0 con 0x7ff5781002e0 2024-04-27T04:08:41.509 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:08:41.510+0000 7ff56dffb700 1 -- 172.21.15.162:0/2010209211 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff56800e070 con 0x7ff5781002e0 2024-04-27T04:08:41.509 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:08:41.510+0000 7ff57f5d5700 1 -- 172.21.15.162:0/2010209211 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff5781175f0 con 0x7ff5781002e0 2024-04-27T04:08:41.510 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:08:41.510+0000 7ff57f5d5700 1 -- 172.21.15.162:0/2010209211 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff578117a40 con 0x7ff5781002e0 2024-04-27T04:08:41.512 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:08:41.512+0000 7ff57f5d5700 1 -- 172.21.15.162:0/2010209211 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff57805f5c0 con 0x7ff5781002e0 2024-04-27T04:08:41.512 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:08:41.512+0000 7ff56dffb700 1 -- 172.21.15.162:0/2010209211 <== mon.1 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 19) v1 ==== 86631+0+0 (secure 0 0 0) 0x7ff56801c400 con 0x7ff5781002e0 2024-04-27T04:08:41.512 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:08:41.512+0000 7ff56dffb700 1 --2- 172.21.15.162:0/2010209211 >> [v2:172.21.15.162:6800/4079782599,v1:172.21.15.162:6801/4079782599] conn(0x7ff564069220 0x7ff56406b6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:08:41.513 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:08:41.513+0000 7ff56dffb700 1 -- 172.21.15.162:0/2010209211 <== mon.1 v2:172.21.15.181:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5550+0+0 (secure 0 0 0) 0x7ff5680881b0 con 0x7ff5781002e0 2024-04-27T04:08:41.513 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:08:41.513+0000 7ff57d371700 1 --2- 172.21.15.162:0/2010209211 >> [v2:172.21.15.162:6800/4079782599,v1:172.21.15.162:6801/4079782599] conn(0x7ff564069220 0x7ff56406b6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:08:41.513 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:08:41.514+0000 7ff57d371700 1 --2- 172.21.15.162:0/2010209211 >> [v2:172.21.15.162:6800/4079782599,v1:172.21.15.162:6801/4079782599] conn(0x7ff564069220 0x7ff56406b6d0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7ff578066940 tx=0x7ff57400b040 comp rx=0 tx=0).ready entity=mgr.14210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:08:41.517 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:08:41.518+0000 7ff56dffb700 1 -- 172.21.15.162:0/2010209211 <== mon.1 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7ff568058020 con 0x7ff5781002e0 2024-04-27T04:08:41.863 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:08:41.863+0000 7ff57f5d5700 1 -- 172.21.15.162:0/2010209211 --> [v2:172.21.15.162:6800/4079782599,v1:172.21.15.162:6801/4079782599] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7ff578117d10 con 0x7ff564069220 2024-04-27T04:08:41.881 INFO:teuthology.orchestra.run.smithi162.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-27T04:08:41.881 INFO:teuthology.orchestra.run.smithi162.stdout:alertmanager.smithi162 smithi162 *:9093,9094 running (5m) 80s ago 6m 24.6M - 0.25.0 c8568f914cd2 61972851fcce 2024-04-27T04:08:41.881 INFO:teuthology.orchestra.run.smithi162.stdout:crash.smithi162 smithi162 running (7m) 80s ago 7m 6660k - 17.2.7-907-g2be3efa9 1c5834633715 228c98df4289 2024-04-27T04:08:41.881 INFO:teuthology.orchestra.run.smithi162.stdout:crash.smithi181 smithi181 running (6m) 84s ago 6m 6665k - 17.2.7-907-g2be3efa9 1c5834633715 c8e553e3df2b 2024-04-27T04:08:41.881 INFO:teuthology.orchestra.run.smithi162.stdout:grafana.smithi162 smithi162 *:3000 running (5m) 80s ago 6m 84.2M - 9.4.7 954c08fa6188 7d3364d5cddc 2024-04-27T04:08:41.882 INFO:teuthology.orchestra.run.smithi162.stdout:mds.cephfs.smithi162.gzlyjj smithi162 running (105s) 80s ago 104s 10.4M - 17.2.7-907-g2be3efa9 1c5834633715 27e5b030dd32 2024-04-27T04:08:41.882 INFO:teuthology.orchestra.run.smithi162.stdout:mds.cephfs.smithi162.xcnapk smithi162 running (98s) 80s ago 97s 13.2M - 17.2.7-907-g2be3efa9 1c5834633715 38f9030df029 2024-04-27T04:08:41.882 INFO:teuthology.orchestra.run.smithi162.stdout:mds.cephfs.smithi181.bkkcdm smithi181 running (95s) 84s ago 95s 12.1M - 17.2.7-907-g2be3efa9 1c5834633715 af55afca1263 2024-04-27T04:08:41.882 INFO:teuthology.orchestra.run.smithi162.stdout:mds.cephfs.smithi181.dzomgh smithi181 running (102s) 84s ago 102s 15.9M - 17.2.7-907-g2be3efa9 1c5834633715 6733296597f3 2024-04-27T04:08:41.882 INFO:teuthology.orchestra.run.smithi162.stdout:mgr.smithi162.oennfn smithi162 *:9283 running (8m) 80s ago 8m 479M - 17.2.7-907-g2be3efa9 1c5834633715 e768e27ad56e 2024-04-27T04:08:41.882 INFO:teuthology.orchestra.run.smithi162.stdout:mgr.smithi181.vtwuhz smithi181 *:8443,9283 running (6m) 84s ago 6m 433M - 17.2.7-907-g2be3efa9 1c5834633715 618e7918caea 2024-04-27T04:08:41.882 INFO:teuthology.orchestra.run.smithi162.stdout:mon.smithi162 smithi162 running (8m) 80s ago 8m 53.9M 2048M 17.2.7-907-g2be3efa9 1c5834633715 0f1e3682b9ab 2024-04-27T04:08:41.882 INFO:teuthology.orchestra.run.smithi162.stdout:mon.smithi181 smithi181 running (6m) 84s ago 6m 44.5M 2048M 17.2.7-907-g2be3efa9 1c5834633715 96e48aa3912f 2024-04-27T04:08:41.882 INFO:teuthology.orchestra.run.smithi162.stdout:node-exporter.smithi162 smithi162 *:9100 running (7m) 80s ago 7m 15.7M - 1.5.0 0da6a335fe13 e4535d340cea 2024-04-27T04:08:41.882 INFO:teuthology.orchestra.run.smithi162.stdout:node-exporter.smithi181 smithi181 *:9100 running (6m) 84s ago 6m 15.2M - 1.5.0 0da6a335fe13 3cc278142657 2024-04-27T04:08:41.882 INFO:teuthology.orchestra.run.smithi162.stdout:osd.0 smithi162 running (5m) 80s ago 5m 48.2M 4288M 17.2.7-907-g2be3efa9 1c5834633715 4a9df3aa938d 2024-04-27T04:08:41.883 INFO:teuthology.orchestra.run.smithi162.stdout:osd.1 smithi162 running (4m) 80s ago 4m 48.4M 4288M 17.2.7-907-g2be3efa9 1c5834633715 aefb464c46e8 2024-04-27T04:08:41.883 INFO:teuthology.orchestra.run.smithi162.stdout:osd.2 smithi162 running (4m) 80s ago 4m 44.5M 4288M 17.2.7-907-g2be3efa9 1c5834633715 70f16d86d38e 2024-04-27T04:08:41.883 INFO:teuthology.orchestra.run.smithi162.stdout:osd.3 smithi181 running (4m) 84s ago 4m 44.5M 5312M 17.2.7-907-g2be3efa9 1c5834633715 030d829be54e 2024-04-27T04:08:41.883 INFO:teuthology.orchestra.run.smithi162.stdout:osd.4 smithi181 running (3m) 84s ago 3m 41.6M 5312M 17.2.7-907-g2be3efa9 1c5834633715 512afbee7cf7 2024-04-27T04:08:41.883 INFO:teuthology.orchestra.run.smithi162.stdout:osd.5 smithi181 running (3m) 84s ago 3m 40.9M 5312M 17.2.7-907-g2be3efa9 1c5834633715 f47a1daac1ea 2024-04-27T04:08:41.883 INFO:teuthology.orchestra.run.smithi162.stdout:prometheus.smithi162 smithi162 *:9095 running (5m) 80s ago 6m 54.6M - 2.43.0 a07b618ecd1d 63755cc04fec 2024-04-27T04:08:41.884 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:08:41.881+0000 7ff56dffb700 1 -- 172.21.15.162:0/2010209211 <== mgr.14210 v2:172.21.15.162:6800/4079782599 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3410 (secure 0 0 0) 0x7ff578117d10 con 0x7ff564069220 2024-04-27T04:08:41.884 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:08:41.883+0000 7ff57f5d5700 1 -- 172.21.15.162:0/2010209211 >> [v2:172.21.15.162:6800/4079782599,v1:172.21.15.162:6801/4079782599] conn(0x7ff564069220 msgr2=0x7ff56406b6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:08:41.884 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:08:41.883+0000 7ff57f5d5700 1 --2- 172.21.15.162:0/2010209211 >> [v2:172.21.15.162:6800/4079782599,v1:172.21.15.162:6801/4079782599] conn(0x7ff564069220 0x7ff56406b6d0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7ff578066940 tx=0x7ff57400b040 comp rx=0 tx=0).stop 2024-04-27T04:08:41.884 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:08:41.883+0000 7ff57f5d5700 1 -- 172.21.15.162:0/2010209211 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff5781002e0 msgr2=0x7ff578116a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:08:41.884 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:08:41.883+0000 7ff57f5d5700 1 --2- 172.21.15.162:0/2010209211 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff5781002e0 0x7ff578116a70 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7ff568009040 tx=0x7ff568009e30 comp rx=0 tx=0).stop 2024-04-27T04:08:41.884 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:08:41.883+0000 7ff57f5d5700 1 -- 172.21.15.162:0/2010209211 shutdown_connections 2024-04-27T04:08:41.884 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:08:41.883+0000 7ff57f5d5700 1 --2- 172.21.15.162:0/2010209211 >> [v2:172.21.15.162:6800/4079782599,v1:172.21.15.162:6801/4079782599] conn(0x7ff564069220 0x7ff56406b6d0 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:08:41.884 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:08:41.883+0000 7ff57f5d5700 1 --2- 172.21.15.162:0/2010209211 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff5780fd9c0 0x7ff578116530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:08:41.885 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:08:41.883+0000 7ff57f5d5700 1 --2- 172.21.15.162:0/2010209211 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff5781002e0 0x7ff578116a70 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:08:41.885 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:08:41.884+0000 7ff57f5d5700 1 -- 172.21.15.162:0/2010209211 >> 172.21.15.162:0/2010209211 conn(0x7ff5780f7440 msgr2=0x7ff5780f7860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:08:41.885 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:08:41.884+0000 7ff57f5d5700 1 -- 172.21.15.162:0/2010209211 shutdown_connections 2024-04-27T04:08:41.885 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:08:41.884+0000 7ff57f5d5700 1 -- 172.21.15.162:0/2010209211 wait complete. 2024-04-27T04:08:42.042 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:08:42.043+0000 7f3f5e719700 1 -- 172.21.15.162:0/1813739095 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3f580662c0 msgr2=0x7f3f580ff9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:08:42.042 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:08:42.043+0000 7f3f5e719700 1 --2- 172.21.15.162:0/1813739095 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3f580662c0 0x7f3f580ff9a0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f3f48004940 tx=0x7f3f48009e30 comp rx=0 tx=0).stop 2024-04-27T04:08:42.042 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:08:42.043+0000 7f3f5e719700 1 -- 172.21.15.162:0/1813739095 shutdown_connections 2024-04-27T04:08:42.043 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:08:42.043+0000 7f3f5e719700 1 --2- 172.21.15.162:0/1813739095 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3f580ffee0 0x7f3f58102300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:08:42.043 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:08:42.043+0000 7f3f5e719700 1 --2- 172.21.15.162:0/1813739095 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3f580662c0 0x7f3f580ff9a0 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:08:42.043 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:08:42.043+0000 7f3f5e719700 1 -- 172.21.15.162:0/1813739095 >> 172.21.15.162:0/1813739095 conn(0x7f3f580f7430 msgr2=0x7f3f580f9840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:08:42.043 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:08:42.044+0000 7f3f5e719700 1 -- 172.21.15.162:0/1813739095 shutdown_connections 2024-04-27T04:08:42.043 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:08:42.044+0000 7f3f5e719700 1 -- 172.21.15.162:0/1813739095 wait complete. 2024-04-27T04:08:42.043 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:08:42.044+0000 7f3f5e719700 1 Processor -- start 2024-04-27T04:08:42.044 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:08:42.045+0000 7f3f5e719700 1 -- start start 2024-04-27T04:08:42.044 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:08:42.045+0000 7f3f5e719700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3f580662c0 0x7f3f5810fbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:08:42.044 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:08:42.045+0000 7f3f5e719700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3f580ffee0 0x7f3f581100f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:08:42.044 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:08:42.045+0000 7f3f5e719700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f581106f0 con 0x7f3f580ffee0 2024-04-27T04:08:42.044 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:08:42.045+0000 7f3f5e719700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f58110f40 con 0x7f3f580662c0 2024-04-27T04:08:42.045 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:08:42.045+0000 7f3f57fff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3f580662c0 0x7f3f5810fbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:08:42.045 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:08:42.045+0000 7f3f57fff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3f580662c0 0x7f3f5810fbb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.162:34124/0 (socket says 172.21.15.162:34124) 2024-04-27T04:08:42.045 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:08:42.045+0000 7f3f57fff700 1 -- 172.21.15.162:0/4270028597 learned_addr learned my addr 172.21.15.162:0/4270028597 (peer_addr_for_me v2:172.21.15.162:0/0) 2024-04-27T04:08:42.045 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:08:42.045+0000 7f3f577fe700 1 --2- 172.21.15.162:0/4270028597 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3f580ffee0 0x7f3f581100f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:08:42.045 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:08:42.046+0000 7f3f57fff700 1 -- 172.21.15.162:0/4270028597 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3f580ffee0 msgr2=0x7f3f581100f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:08:42.045 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:08:42.046+0000 7f3f57fff700 1 --2- 172.21.15.162:0/4270028597 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3f580ffee0 0x7f3f581100f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:08:42.046 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:08:42.046+0000 7f3f57fff700 1 -- 172.21.15.162:0/4270028597 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f48005040 con 0x7f3f580662c0 2024-04-27T04:08:42.046 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:08:42.046+0000 7f3f577fe700 1 --2- 172.21.15.162:0/4270028597 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3f580ffee0 0x7f3f581100f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-27T04:08:42.046 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:08:42.046+0000 7f3f57fff700 1 --2- 172.21.15.162:0/4270028597 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3f580662c0 0x7f3f5810fbb0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f3f48004090 tx=0x7f3f48009e30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:08:42.046 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:08:42.046+0000 7f3f54ff9700 1 -- 172.21.15.162:0/4270028597 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3f48021070 con 0x7f3f580662c0 2024-04-27T04:08:42.046 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:08:42.046+0000 7f3f5e719700 1 -- 172.21.15.162:0/4270028597 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3f581111c0 con 0x7f3f580662c0 2024-04-27T04:08:42.046 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:08:42.046+0000 7f3f54ff9700 1 -- 172.21.15.162:0/4270028597 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f3f4801c400 con 0x7f3f580662c0 2024-04-27T04:08:42.046 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:08:42.046+0000 7f3f5e719700 1 -- 172.21.15.162:0/4270028597 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3f58111700 con 0x7f3f580662c0 2024-04-27T04:08:42.047 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:08:42.047+0000 7f3f54ff9700 1 -- 172.21.15.162:0/4270028597 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3f4800e030 con 0x7f3f580662c0 2024-04-27T04:08:42.047 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:08:42.048+0000 7f3f54ff9700 1 -- 172.21.15.162:0/4270028597 <== mon.1 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 19) v1 ==== 86631+0+0 (secure 0 0 0) 0x7f3f4800b760 con 0x7f3f580662c0 2024-04-27T04:08:42.048 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:08:42.049+0000 7f3f54ff9700 1 --2- 172.21.15.162:0/4270028597 >> [v2:172.21.15.162:6800/4079782599,v1:172.21.15.162:6801/4079782599] conn(0x7f3f400690a0 0x7f3f4006b550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:08:42.048 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:08:42.049+0000 7f3f54ff9700 1 -- 172.21.15.162:0/4270028597 <== mon.1 v2:172.21.15.181:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5550+0+0 (secure 0 0 0) 0x7f3f480883c0 con 0x7f3f580662c0 2024-04-27T04:08:42.048 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:08:42.049+0000 7f3f577fe700 1 --2- 172.21.15.162:0/4270028597 >> [v2:172.21.15.162:6800/4079782599,v1:172.21.15.162:6801/4079782599] conn(0x7f3f400690a0 0x7f3f4006b550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:08:42.048 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:08:42.049+0000 7f3f5e719700 1 -- 172.21.15.162:0/4270028597 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3f5805f5c0 con 0x7f3f580662c0 2024-04-27T04:08:42.049 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:08:42.050+0000 7f3f577fe700 1 --2- 172.21.15.162:0/4270028597 >> [v2:172.21.15.162:6800/4079782599,v1:172.21.15.162:6801/4079782599] conn(0x7f3f400690a0 0x7f3f4006b550 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f3f4c009b80 tx=0x7f3f4c00b040 comp rx=0 tx=0).ready entity=mgr.14210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:08:42.053 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:08:42.054+0000 7f3f54ff9700 1 -- 172.21.15.162:0/4270028597 <== mon.1 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f3f48058150 con 0x7f3f580662c0 2024-04-27T04:08:42.341 INFO:tasks.workunit.client.1.smithi181.stderr:Updating files: 99% (12424/12492) Updating files: 100% (12492/12492) Updating files: 100% (12492/12492), done. 2024-04-27T04:08:42.433 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:08:42.433+0000 7f3f5e719700 1 -- 172.21.15.162:0/4270028597 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f3f581059a0 con 0x7f3f580662c0 2024-04-27T04:08:42.434 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:08:42.433+0000 7f3f54ff9700 1 -- 172.21.15.162:0/4270028597 <== mon.1 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+638 (secure 0 0 0) 0x7f3f48057ce0 con 0x7f3f580662c0 2024-04-27T04:08:42.434 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-04-27T04:08:42.434 INFO:teuthology.orchestra.run.smithi162.stdout: "mon": { 2024-04-27T04:08:42.434 INFO:teuthology.orchestra.run.smithi162.stdout: "ceph version 17.2.7-907-g2be3efa9 (2be3efa97a7972f8e925bd899b474329979b74d6) quincy (stable)": 2 2024-04-27T04:08:42.434 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-04-27T04:08:42.434 INFO:teuthology.orchestra.run.smithi162.stdout: "mgr": { 2024-04-27T04:08:42.434 INFO:teuthology.orchestra.run.smithi162.stdout: "ceph version 17.2.7-907-g2be3efa9 (2be3efa97a7972f8e925bd899b474329979b74d6) quincy (stable)": 2 2024-04-27T04:08:42.434 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-04-27T04:08:42.434 INFO:teuthology.orchestra.run.smithi162.stdout: "osd": { 2024-04-27T04:08:42.435 INFO:teuthology.orchestra.run.smithi162.stdout: "ceph version 17.2.7-907-g2be3efa9 (2be3efa97a7972f8e925bd899b474329979b74d6) quincy (stable)": 6 2024-04-27T04:08:42.435 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-04-27T04:08:42.435 INFO:teuthology.orchestra.run.smithi162.stdout: "mds": { 2024-04-27T04:08:42.435 INFO:teuthology.orchestra.run.smithi162.stdout: "ceph version 17.2.7-907-g2be3efa9 (2be3efa97a7972f8e925bd899b474329979b74d6) quincy (stable)": 4 2024-04-27T04:08:42.435 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-04-27T04:08:42.435 INFO:teuthology.orchestra.run.smithi162.stdout: "overall": { 2024-04-27T04:08:42.435 INFO:teuthology.orchestra.run.smithi162.stdout: "ceph version 17.2.7-907-g2be3efa9 (2be3efa97a7972f8e925bd899b474329979b74d6) quincy (stable)": 14 2024-04-27T04:08:42.435 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-04-27T04:08:42.435 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-04-27T04:08:42.436 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:08:42.436+0000 7f3f5e719700 1 -- 172.21.15.162:0/4270028597 >> [v2:172.21.15.162:6800/4079782599,v1:172.21.15.162:6801/4079782599] conn(0x7f3f400690a0 msgr2=0x7f3f4006b550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:08:42.436 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:08:42.436+0000 7f3f5e719700 1 --2- 172.21.15.162:0/4270028597 >> [v2:172.21.15.162:6800/4079782599,v1:172.21.15.162:6801/4079782599] conn(0x7f3f400690a0 0x7f3f4006b550 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f3f4c009b80 tx=0x7f3f4c00b040 comp rx=0 tx=0).stop 2024-04-27T04:08:42.436 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:08:42.436+0000 7f3f5e719700 1 -- 172.21.15.162:0/4270028597 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3f580662c0 msgr2=0x7f3f5810fbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:08:42.436 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:08:42.436+0000 7f3f5e719700 1 --2- 172.21.15.162:0/4270028597 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3f580662c0 0x7f3f5810fbb0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f3f48004090 tx=0x7f3f48009e30 comp rx=0 tx=0).stop 2024-04-27T04:08:42.436 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:08:42.436+0000 7f3f5e719700 1 -- 172.21.15.162:0/4270028597 shutdown_connections 2024-04-27T04:08:42.436 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:08:42.436+0000 7f3f5e719700 1 --2- 172.21.15.162:0/4270028597 >> [v2:172.21.15.162:6800/4079782599,v1:172.21.15.162:6801/4079782599] conn(0x7f3f400690a0 0x7f3f4006b550 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:08:42.437 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:08:42.436+0000 7f3f5e719700 1 --2- 172.21.15.162:0/4270028597 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3f580662c0 0x7f3f5810fbb0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:08:42.437 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:08:42.436+0000 7f3f5e719700 1 --2- 172.21.15.162:0/4270028597 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3f580ffee0 0x7f3f581100f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:08:42.437 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:08:42.436+0000 7f3f5e719700 1 -- 172.21.15.162:0/4270028597 >> 172.21.15.162:0/4270028597 conn(0x7f3f580f7430 msgr2=0x7f3f580fb9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:08:42.437 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:08:42.437+0000 7f3f5e719700 1 -- 172.21.15.162:0/4270028597 shutdown_connections 2024-04-27T04:08:42.437 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:08:42.437+0000 7f3f5e719700 1 -- 172.21.15.162:0/4270028597 wait complete. 2024-04-27T04:08:42.544 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:08:42.545+0000 7f5c78bf2700 1 -- 172.21.15.162:0/2675633369 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5c740fc290 msgr2=0x7f5c740fc6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:08:42.545 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:08:42.545+0000 7f5c78bf2700 1 --2- 172.21.15.162:0/2675633369 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5c740fc290 0x7f5c740fc6a0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f5c64004940 tx=0x7f5c64009e30 comp rx=0 tx=0).stop 2024-04-27T04:08:42.545 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:08:42.545+0000 7f5c78bf2700 1 -- 172.21.15.162:0/2675633369 shutdown_connections 2024-04-27T04:08:42.545 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:08:42.545+0000 7f5c78bf2700 1 --2- 172.21.15.162:0/2675633369 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5c740fcbe0 0x7f5c740fd050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:08:42.545 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:08:42.545+0000 7f5c78bf2700 1 --2- 172.21.15.162:0/2675633369 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5c740fc290 0x7f5c740fc6a0 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:08:42.545 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:08:42.545+0000 7f5c78bf2700 1 -- 172.21.15.162:0/2675633369 >> 172.21.15.162:0/2675633369 conn(0x7f5c740f6b80 msgr2=0x7f5c740f8fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:08:42.545 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:08:42.546+0000 7f5c78bf2700 1 -- 172.21.15.162:0/2675633369 shutdown_connections 2024-04-27T04:08:42.545 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:08:42.546+0000 7f5c78bf2700 1 -- 172.21.15.162:0/2675633369 wait complete. 2024-04-27T04:08:42.546 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:08:42.546+0000 7f5c78bf2700 1 Processor -- start 2024-04-27T04:08:42.546 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:08:42.546+0000 7f5c78bf2700 1 -- start start 2024-04-27T04:08:42.546 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:08:42.547+0000 7f5c78bf2700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5c740fc290 0x7f5c741106c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:08:42.546 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:08:42.547+0000 7f5c78bf2700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5c740fcbe0 0x7f5c74112c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:08:42.546 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:08:42.547+0000 7f5c78bf2700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c740fbc80 con 0x7f5c740fc290 2024-04-27T04:08:42.546 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:08:42.547+0000 7f5c78bf2700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c740fbdc0 con 0x7f5c740fcbe0 2024-04-27T04:08:42.546 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:08:42.547+0000 7f5c7259c700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5c740fc290 0x7f5c741106c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:08:42.547 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:08:42.547+0000 7f5c7259c700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5c740fc290 0x7f5c741106c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:54062/0 (socket says 172.21.15.162:54062) 2024-04-27T04:08:42.547 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:08:42.547+0000 7f5c7259c700 1 -- 172.21.15.162:0/3274799605 learned_addr learned my addr 172.21.15.162:0/3274799605 (peer_addr_for_me v2:172.21.15.162:0/0) 2024-04-27T04:08:42.547 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:08:42.547+0000 7f5c71d9b700 1 --2- 172.21.15.162:0/3274799605 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5c740fcbe0 0x7f5c74112c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:08:42.547 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:08:42.547+0000 7f5c7259c700 1 -- 172.21.15.162:0/3274799605 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5c740fcbe0 msgr2=0x7f5c74112c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:08:42.547 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:08:42.547+0000 7f5c7259c700 1 --2- 172.21.15.162:0/3274799605 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5c740fcbe0 0x7f5c74112c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:08:42.547 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:08:42.547+0000 7f5c7259c700 1 -- 172.21.15.162:0/3274799605 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c64005040 con 0x7f5c740fc290 2024-04-27T04:08:42.547 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:08:42.547+0000 7f5c7259c700 1 --2- 172.21.15.162:0/3274799605 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5c740fc290 0x7f5c741106c0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f5c64004090 tx=0x7f5c64009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:08:42.548 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:08:42.548+0000 7f5c62ffd700 1 -- 172.21.15.162:0/3274799605 <== mon.0 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5c64021070 con 0x7f5c740fc290 2024-04-27T04:08:42.548 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:08:42.548+0000 7f5c78bf2700 1 -- 172.21.15.162:0/3274799605 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5c74113150 con 0x7f5c740fc290 2024-04-27T04:08:42.548 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:08:42.548+0000 7f5c62ffd700 1 -- 172.21.15.162:0/3274799605 <== mon.0 v2:172.21.15.162:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f5c6401c400 con 0x7f5c740fc290 2024-04-27T04:08:42.548 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:08:42.548+0000 7f5c62ffd700 1 -- 172.21.15.162:0/3274799605 <== mon.0 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5c6400e070 con 0x7f5c740fc290 2024-04-27T04:08:42.548 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:08:42.548+0000 7f5c78bf2700 1 -- 172.21.15.162:0/3274799605 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5c7410c770 con 0x7f5c740fc290 2024-04-27T04:08:42.548 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:08:42.549+0000 7f5c60ff9700 1 -- 172.21.15.162:0/3274799605 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5c580052b0 con 0x7f5c740fc290 2024-04-27T04:08:42.549 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:08:42.551+0000 7f5c62ffd700 1 -- 172.21.15.162:0/3274799605 <== mon.0 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 19) v1 ==== 86631+0+0 (secure 0 0 0) 0x7f5c6401ca90 con 0x7f5c740fc290 2024-04-27T04:08:42.550 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:08:42.551+0000 7f5c62ffd700 1 --2- 172.21.15.162:0/3274799605 >> [v2:172.21.15.162:6800/4079782599,v1:172.21.15.162:6801/4079782599] conn(0x7f5c5c0692b0 0x7f5c5c06b760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:08:42.550 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:08:42.551+0000 7f5c62ffd700 1 -- 172.21.15.162:0/3274799605 <== mon.0 v2:172.21.15.162:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5550+0+0 (secure 0 0 0) 0x7f5c640888d0 con 0x7f5c740fc290 2024-04-27T04:08:42.550 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:08:42.551+0000 7f5c71d9b700 1 --2- 172.21.15.162:0/3274799605 >> [v2:172.21.15.162:6800/4079782599,v1:172.21.15.162:6801/4079782599] conn(0x7f5c5c0692b0 0x7f5c5c06b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:08:42.551 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:08:42.552+0000 7f5c71d9b700 1 --2- 172.21.15.162:0/3274799605 >> [v2:172.21.15.162:6800/4079782599,v1:172.21.15.162:6801/4079782599] conn(0x7f5c5c0692b0 0x7f5c5c06b760 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f5c68009b80 tx=0x7f5c6800b040 comp rx=0 tx=0).ready entity=mgr.14210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:08:42.553 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:08:42.554+0000 7f5c62ffd700 1 -- 172.21.15.162:0/3274799605 <== mon.0 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f5c64054c50 con 0x7f5c740fc290 2024-04-27T04:08:42.637 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:08:42 smithi162 ceph-mon[26841]: pgmap v226: 65 pgs: 65 active+clean; 588 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:08:42.637 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:08:42 smithi162 ceph-mon[26841]: from='client.14554 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-27T04:08:42.637 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:08:42 smithi162 ceph-mon[26841]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' 2024-04-27T04:08:42.637 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:08:42 smithi162 ceph-mon[26841]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T04:08:42.638 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:08:42 smithi162 ceph-mon[26841]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' 2024-04-27T04:08:42.657 INFO:tasks.workunit.client.1.smithi181.stderr:Note: switching to '3067e9f133f954b642fdf21114a79edb58590948'. 2024-04-27T04:08:42.658 INFO:tasks.workunit.client.1.smithi181.stderr: 2024-04-27T04:08:42.658 INFO:tasks.workunit.client.1.smithi181.stderr:You are in 'detached HEAD' state. You can look around, make experimental 2024-04-27T04:08:42.658 INFO:tasks.workunit.client.1.smithi181.stderr:changes and commit them, and you can discard any commits you make in this 2024-04-27T04:08:42.658 INFO:tasks.workunit.client.1.smithi181.stderr:state without impacting any branches by switching back to a branch. 2024-04-27T04:08:42.658 INFO:tasks.workunit.client.1.smithi181.stderr: 2024-04-27T04:08:42.658 INFO:tasks.workunit.client.1.smithi181.stderr:If you want to create a new branch to retain commits you create, you may 2024-04-27T04:08:42.658 INFO:tasks.workunit.client.1.smithi181.stderr:do so (now or later) by using -c with the switch command. Example: 2024-04-27T04:08:42.658 INFO:tasks.workunit.client.1.smithi181.stderr: 2024-04-27T04:08:42.658 INFO:tasks.workunit.client.1.smithi181.stderr: git switch -c 2024-04-27T04:08:42.658 INFO:tasks.workunit.client.1.smithi181.stderr: 2024-04-27T04:08:42.658 INFO:tasks.workunit.client.1.smithi181.stderr:Or undo this operation with: 2024-04-27T04:08:42.658 INFO:tasks.workunit.client.1.smithi181.stderr: 2024-04-27T04:08:42.659 INFO:tasks.workunit.client.1.smithi181.stderr: git switch - 2024-04-27T04:08:42.659 INFO:tasks.workunit.client.1.smithi181.stderr: 2024-04-27T04:08:42.659 INFO:tasks.workunit.client.1.smithi181.stderr:Turn off this advice by setting config variable advice.detachedHead to false 2024-04-27T04:08:42.659 INFO:tasks.workunit.client.1.smithi181.stderr: 2024-04-27T04:08:42.659 INFO:tasks.workunit.client.1.smithi181.stderr:HEAD is now at 3067e9f133f Merge pull request #57105 from zdover23/wip-doc-2024-04-26-mgr-dashboard 2024-04-27T04:08:42.660 DEBUG:teuthology.orchestra.run.smithi181:> cd -- /home/ubuntu/cephtest/clone.client.1/qa/workunits && if test -e Makefile ; then make ; fi && find -executable -type f -printf '%P\0' >/home/ubuntu/cephtest/workunits.list.client.1 2024-04-27T04:08:42.672 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:08:42 smithi181 ceph-mon[33774]: pgmap v226: 65 pgs: 65 active+clean; 588 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:08:42.672 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:08:42 smithi181 ceph-mon[33774]: from='client.14554 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-27T04:08:42.672 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:08:42 smithi181 ceph-mon[33774]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' 2024-04-27T04:08:42.672 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:08:42 smithi181 ceph-mon[33774]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T04:08:42.673 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:08:42 smithi181 ceph-mon[33774]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' 2024-04-27T04:08:42.697 INFO:tasks.workunit.client.1.smithi181.stdout:for d in direct_io fs ; do ( cd $d ; make all ) ; done 2024-04-27T04:08:42.699 INFO:tasks.workunit.client.1.smithi181.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.1/qa/workunits/direct_io' 2024-04-27T04:08:42.699 INFO:tasks.workunit.client.1.smithi181.stdout:cc -Wall -Wextra -D_GNU_SOURCE direct_io_test.c -o direct_io_test 2024-04-27T04:08:42.965 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:08:42.966+0000 7f5c60ff9700 1 -- 172.21.15.162:0/3274799605 --> [v2:172.21.15.162:6800/4079782599,v1:172.21.15.162:6801/4079782599] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f5c58000bc0 con 0x7f5c5c0692b0 2024-04-27T04:08:42.967 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:08:42.968+0000 7f5c62ffd700 1 -- 172.21.15.162:0/3274799605 <== mgr.14210 v2:172.21.15.162:6800/4079782599 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+439 (secure 0 0 0) 0x7f5c58000bc0 con 0x7f5c5c0692b0 2024-04-27T04:08:42.967 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-04-27T04:08:42.967 INFO:teuthology.orchestra.run.smithi162.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:90a3a9fa9ceb8977f6dad9ccd36fe46018d07e2118199820523a6b35301d9c16", 2024-04-27T04:08:42.967 INFO:teuthology.orchestra.run.smithi162.stdout: "in_progress": true, 2024-04-27T04:08:42.967 INFO:teuthology.orchestra.run.smithi162.stdout: "which": "Upgrading daemons of type(s) mgr", 2024-04-27T04:08:42.967 INFO:teuthology.orchestra.run.smithi162.stdout: "services_complete": [], 2024-04-27T04:08:42.967 INFO:teuthology.orchestra.run.smithi162.stdout: "progress": "0/2 daemons upgraded", 2024-04-27T04:08:42.967 INFO:teuthology.orchestra.run.smithi162.stdout: "message": "Pulling quay.ceph.io/ceph-ci/ceph@sha256:90a3a9fa9ceb8977f6dad9ccd36fe46018d07e2118199820523a6b35301d9c16 image on host smithi181", 2024-04-27T04:08:42.967 INFO:teuthology.orchestra.run.smithi162.stdout: "is_paused": false 2024-04-27T04:08:42.968 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-04-27T04:08:42.969 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:08:42.971+0000 7f5c60ff9700 1 -- 172.21.15.162:0/3274799605 >> [v2:172.21.15.162:6800/4079782599,v1:172.21.15.162:6801/4079782599] conn(0x7f5c5c0692b0 msgr2=0x7f5c5c06b760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:08:42.969 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:08:42.971+0000 7f5c60ff9700 1 --2- 172.21.15.162:0/3274799605 >> [v2:172.21.15.162:6800/4079782599,v1:172.21.15.162:6801/4079782599] conn(0x7f5c5c0692b0 0x7f5c5c06b760 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f5c68009b80 tx=0x7f5c6800b040 comp rx=0 tx=0).stop 2024-04-27T04:08:42.970 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:08:42.971+0000 7f5c60ff9700 1 -- 172.21.15.162:0/3274799605 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5c740fc290 msgr2=0x7f5c741106c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:08:42.970 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:08:42.971+0000 7f5c60ff9700 1 --2- 172.21.15.162:0/3274799605 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5c740fc290 0x7f5c741106c0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f5c64004090 tx=0x7f5c64009e30 comp rx=0 tx=0).stop 2024-04-27T04:08:42.970 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:08:42.971+0000 7f5c60ff9700 1 -- 172.21.15.162:0/3274799605 shutdown_connections 2024-04-27T04:08:42.970 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:08:42.971+0000 7f5c60ff9700 1 --2- 172.21.15.162:0/3274799605 >> [v2:172.21.15.162:6800/4079782599,v1:172.21.15.162:6801/4079782599] conn(0x7f5c5c0692b0 0x7f5c5c06b760 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:08:42.970 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:08:42.971+0000 7f5c60ff9700 1 --2- 172.21.15.162:0/3274799605 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5c740fc290 0x7f5c741106c0 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:08:42.970 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:08:42.971+0000 7f5c60ff9700 1 --2- 172.21.15.162:0/3274799605 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5c740fcbe0 0x7f5c74112c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:08:42.971 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:08:42.971+0000 7f5c60ff9700 1 -- 172.21.15.162:0/3274799605 >> 172.21.15.162:0/3274799605 conn(0x7f5c740f6b80 msgr2=0x7f5c740f8240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:08:42.971 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:08:42.971+0000 7f5c60ff9700 1 -- 172.21.15.162:0/3274799605 shutdown_connections 2024-04-27T04:08:42.971 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:08:42.972+0000 7f5c60ff9700 1 -- 172.21.15.162:0/3274799605 wait complete. 2024-04-27T04:08:43.279 INFO:tasks.workunit.client.1.smithi181.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_sync_io.c -o test_sync_io 2024-04-27T04:08:43.333 INFO:tasks.workunit.client.1.smithi181.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_short_dio_read.c -o test_short_dio_read 2024-04-27T04:08:43.373 INFO:tasks.workunit.client.1.smithi181.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.1/qa/workunits/direct_io' 2024-04-27T04:08:43.375 INFO:tasks.workunit.client.1.smithi181.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.1/qa/workunits/fs' 2024-04-27T04:08:43.375 INFO:tasks.workunit.client.1.smithi181.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_o_trunc.c -o test_o_trunc 2024-04-27T04:08:43.418 INFO:tasks.workunit.client.1.smithi181.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.1/qa/workunits/fs' 2024-04-27T04:08:43.422 DEBUG:teuthology.orchestra.run.smithi181:> set -ex 2024-04-27T04:08:43.422 DEBUG:teuthology.orchestra.run.smithi181:> dd if=/home/ubuntu/cephtest/workunits.list.client.1 of=/dev/stdout 2024-04-27T04:08:43.477 INFO:tasks.workunit:Running workunits matching suites/fsstress.sh on client.1... 2024-04-27T04:08:43.478 INFO:tasks.workunit:Running workunit suites/fsstress.sh... 2024-04-27T04:08:43.478 DEBUG:teuthology.orchestra.run.smithi181:workunit test suites/fsstress.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.1/client.1/tmp && cd -- /home/ubuntu/cephtest/mnt.1/client.1/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=3067e9f133f954b642fdf21114a79edb58590948 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="1" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.1 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.1 CEPH_MNT=/home/ubuntu/cephtest/mnt.1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.1/qa/workunits/suites/fsstress.sh 2024-04-27T04:08:43.542 INFO:tasks.workunit.client.1.smithi181.stderr:+ mkdir -p fsstress 2024-04-27T04:08:43.544 INFO:tasks.workunit.client.1.smithi181.stderr:+ pushd fsstress 2024-04-27T04:08:43.545 INFO:tasks.workunit.client.1.smithi181.stdout:~/cephtest/mnt.1/client.1/tmp/fsstress ~/cephtest/mnt.1/client.1/tmp 2024-04-27T04:08:43.546 INFO:tasks.workunit.client.1.smithi181.stderr:+ wget -q -O ltp-full.tgz http://download.ceph.com/qa/ltp-full-20091231.tgz 2024-04-27T04:08:43.637 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:08:43 smithi162 ceph-mon[26841]: from='client.14558 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-27T04:08:43.637 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:08:43 smithi162 ceph-mon[26841]: Upgrade: Target is version 19.0.0-3253-g4e213a58 (unknown) 2024-04-27T04:08:43.637 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:08:43 smithi162 ceph-mon[26841]: Upgrade: Target container is quay.ceph.io/ceph-ci/ceph@sha256:90a3a9fa9ceb8977f6dad9ccd36fe46018d07e2118199820523a6b35301d9c16, digests ['quay.ceph.io/ceph-ci/ceph@sha256:90a3a9fa9ceb8977f6dad9ccd36fe46018d07e2118199820523a6b35301d9c16'] 2024-04-27T04:08:43.637 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:08:43 smithi162 ceph-mon[26841]: Upgrade: Need to upgrade myself (mgr.smithi162.oennfn) 2024-04-27T04:08:43.637 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:08:43 smithi162 ceph-mon[26841]: from='client.24337 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-27T04:08:43.637 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:08:43 smithi162 ceph-mon[26841]: Upgrade: Pulling quay.ceph.io/ceph-ci/ceph@sha256:90a3a9fa9ceb8977f6dad9ccd36fe46018d07e2118199820523a6b35301d9c16 on smithi181 2024-04-27T04:08:43.638 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:08:43 smithi162 ceph-mon[26841]: from='client.? 172.21.15.162:0/4270028597' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-27T04:08:43.672 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:08:43 smithi181 ceph-mon[33774]: from='client.14558 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-27T04:08:43.672 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:08:43 smithi181 ceph-mon[33774]: Upgrade: Target is version 19.0.0-3253-g4e213a58 (unknown) 2024-04-27T04:08:43.673 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:08:43 smithi181 ceph-mon[33774]: Upgrade: Target container is quay.ceph.io/ceph-ci/ceph@sha256:90a3a9fa9ceb8977f6dad9ccd36fe46018d07e2118199820523a6b35301d9c16, digests ['quay.ceph.io/ceph-ci/ceph@sha256:90a3a9fa9ceb8977f6dad9ccd36fe46018d07e2118199820523a6b35301d9c16'] 2024-04-27T04:08:43.673 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:08:43 smithi181 ceph-mon[33774]: Upgrade: Need to upgrade myself (mgr.smithi162.oennfn) 2024-04-27T04:08:43.673 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:08:43 smithi181 ceph-mon[33774]: from='client.24337 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-27T04:08:43.673 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:08:43 smithi181 ceph-mon[33774]: Upgrade: Pulling quay.ceph.io/ceph-ci/ceph@sha256:90a3a9fa9ceb8977f6dad9ccd36fe46018d07e2118199820523a6b35301d9c16 on smithi181 2024-04-27T04:08:43.673 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:08:43 smithi181 ceph-mon[33774]: from='client.? 172.21.15.162:0/4270028597' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-27T04:08:44.097 INFO:tasks.workunit.client.1.smithi181.stderr:+ tar xzf ltp-full.tgz 2024-04-27T04:08:44.638 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:08:44 smithi162 ceph-mon[26841]: pgmap v227: 65 pgs: 65 active+clean; 588 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:08:44.638 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:08:44 smithi162 ceph-mon[26841]: from='client.14568 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-27T04:08:44.672 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:08:44 smithi181 ceph-mon[33774]: pgmap v227: 65 pgs: 65 active+clean; 588 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:08:44.672 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:08:44 smithi181 ceph-mon[33774]: from='client.14568 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-27T04:08:46.637 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:08:46 smithi162 ceph-mon[26841]: pgmap v228: 65 pgs: 65 active+clean; 731 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 14 KiB/s wr, 4 op/s 2024-04-27T04:08:46.672 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:08:46 smithi181 ceph-mon[33774]: pgmap v228: 65 pgs: 65 active+clean; 731 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 14 KiB/s wr, 4 op/s 2024-04-27T04:08:47.600 INFO:tasks.workunit.client.0.smithi162.stderr:Updating files: 93% (11654/12492) Updating files: 94% (11743/12492) Updating files: 95% (11868/12492) Updating files: 96% (11993/12492) Updating files: 97% (12118/12492) Updating files: 98% (12243/12492) Updating files: 99% (12368/12492) Updating files: 100% (12492/12492) Updating files: 100% (12492/12492), done. 2024-04-27T04:08:47.637 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:08:47 smithi162 ceph-mon[26841]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-27T04:08:47.672 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:08:47 smithi181 ceph-mon[33774]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-27T04:08:48.192 INFO:tasks.workunit.client.0.smithi162.stderr:Note: switching to '3067e9f133f954b642fdf21114a79edb58590948'. 2024-04-27T04:08:48.193 INFO:tasks.workunit.client.0.smithi162.stderr: 2024-04-27T04:08:48.193 INFO:tasks.workunit.client.0.smithi162.stderr:You are in 'detached HEAD' state. You can look around, make experimental 2024-04-27T04:08:48.193 INFO:tasks.workunit.client.0.smithi162.stderr:changes and commit them, and you can discard any commits you make in this 2024-04-27T04:08:48.193 INFO:tasks.workunit.client.0.smithi162.stderr:state without impacting any branches by switching back to a branch. 2024-04-27T04:08:48.193 INFO:tasks.workunit.client.0.smithi162.stderr: 2024-04-27T04:08:48.193 INFO:tasks.workunit.client.0.smithi162.stderr:If you want to create a new branch to retain commits you create, you may 2024-04-27T04:08:48.193 INFO:tasks.workunit.client.0.smithi162.stderr:do so (now or later) by using -c with the switch command. Example: 2024-04-27T04:08:48.193 INFO:tasks.workunit.client.0.smithi162.stderr: 2024-04-27T04:08:48.193 INFO:tasks.workunit.client.0.smithi162.stderr: git switch -c 2024-04-27T04:08:48.193 INFO:tasks.workunit.client.0.smithi162.stderr: 2024-04-27T04:08:48.193 INFO:tasks.workunit.client.0.smithi162.stderr:Or undo this operation with: 2024-04-27T04:08:48.193 INFO:tasks.workunit.client.0.smithi162.stderr: 2024-04-27T04:08:48.193 INFO:tasks.workunit.client.0.smithi162.stderr: git switch - 2024-04-27T04:08:48.194 INFO:tasks.workunit.client.0.smithi162.stderr: 2024-04-27T04:08:48.194 INFO:tasks.workunit.client.0.smithi162.stderr:Turn off this advice by setting config variable advice.detachedHead to false 2024-04-27T04:08:48.194 INFO:tasks.workunit.client.0.smithi162.stderr: 2024-04-27T04:08:48.194 INFO:tasks.workunit.client.0.smithi162.stderr:HEAD is now at 3067e9f133f Merge pull request #57105 from zdover23/wip-doc-2024-04-26-mgr-dashboard 2024-04-27T04:08:48.195 DEBUG:teuthology.orchestra.run.smithi162:> cd -- /home/ubuntu/cephtest/clone.client.0/qa/workunits && if test -e Makefile ; then make ; fi && find -executable -type f -printf '%P\0' >/home/ubuntu/cephtest/workunits.list.client.0 2024-04-27T04:08:48.453 INFO:tasks.workunit.client.0.smithi162.stdout:for d in direct_io fs ; do ( cd $d ; make all ) ; done 2024-04-27T04:08:48.455 INFO:tasks.workunit.client.0.smithi162.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-04-27T04:08:48.455 INFO:tasks.workunit.client.0.smithi162.stdout:cc -Wall -Wextra -D_GNU_SOURCE direct_io_test.c -o direct_io_test 2024-04-27T04:08:48.637 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:08:48 smithi162 ceph-mon[26841]: pgmap v229: 65 pgs: 65 active+clean; 1.1 MiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 56 KiB/s wr, 13 op/s 2024-04-27T04:08:48.672 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:08:48 smithi181 ceph-mon[33774]: pgmap v229: 65 pgs: 65 active+clean; 1.1 MiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 56 KiB/s wr, 13 op/s 2024-04-27T04:08:50.637 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:08:50 smithi162 ceph-mon[26841]: pgmap v230: 65 pgs: 65 active+clean; 16 MiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 48 KiB/s rd, 1.3 MiB/s wr, 75 op/s 2024-04-27T04:08:50.672 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:08:50 smithi181 ceph-mon[33774]: pgmap v230: 65 pgs: 65 active+clean; 16 MiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 48 KiB/s rd, 1.3 MiB/s wr, 75 op/s 2024-04-27T04:08:52.637 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:08:52 smithi162 ceph-mon[26841]: pgmap v231: 65 pgs: 65 active+clean; 17 MiB data, 1.8 GiB used, 535 GiB / 536 GiB avail; 48 KiB/s rd, 1.4 MiB/s wr, 90 op/s 2024-04-27T04:08:52.672 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:08:52 smithi181 ceph-mon[33774]: pgmap v231: 65 pgs: 65 active+clean; 17 MiB data, 1.8 GiB used, 535 GiB / 536 GiB avail; 48 KiB/s rd, 1.4 MiB/s wr, 90 op/s 2024-04-27T04:08:54.610 INFO:tasks.workunit.client.0.smithi162.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_sync_io.c -o test_sync_io 2024-04-27T04:08:54.637 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:08:54 smithi162 ceph-mon[26841]: pgmap v232: 65 pgs: 65 active+clean; 22 MiB data, 1.8 GiB used, 535 GiB / 536 GiB avail; 352 KiB/s rd, 1.9 MiB/s wr, 175 op/s 2024-04-27T04:08:54.672 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:08:54 smithi181 ceph-mon[33774]: pgmap v232: 65 pgs: 65 active+clean; 22 MiB data, 1.8 GiB used, 535 GiB / 536 GiB avail; 352 KiB/s rd, 1.9 MiB/s wr, 175 op/s 2024-04-27T04:08:54.811 INFO:tasks.workunit.client.0.smithi162.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_short_dio_read.c -o test_short_dio_read 2024-04-27T04:08:54.894 INFO:tasks.workunit.client.0.smithi162.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-04-27T04:08:54.895 INFO:tasks.workunit.client.0.smithi162.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-04-27T04:08:54.896 INFO:tasks.workunit.client.0.smithi162.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_o_trunc.c -o test_o_trunc 2024-04-27T04:08:55.124 INFO:tasks.workunit.client.0.smithi162.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-04-27T04:08:55.127 DEBUG:teuthology.orchestra.run.smithi162:> set -ex 2024-04-27T04:08:55.127 DEBUG:teuthology.orchestra.run.smithi162:> dd if=/home/ubuntu/cephtest/workunits.list.client.0 of=/dev/stdout 2024-04-27T04:08:55.143 INFO:tasks.workunit:Running workunits matching suites/fsstress.sh on client.0... 2024-04-27T04:08:55.144 INFO:tasks.workunit:Running workunit suites/fsstress.sh... 2024-04-27T04:08:55.144 DEBUG:teuthology.orchestra.run.smithi162:workunit test suites/fsstress.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=3067e9f133f954b642fdf21114a79edb58590948 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/fsstress.sh 2024-04-27T04:08:55.208 INFO:tasks.workunit.client.0.smithi162.stderr:+ mkdir -p fsstress 2024-04-27T04:08:55.212 INFO:tasks.workunit.client.0.smithi162.stderr:+ pushd fsstress 2024-04-27T04:08:55.213 INFO:tasks.workunit.client.0.smithi162.stdout:~/cephtest/mnt.0/client.0/tmp/fsstress ~/cephtest/mnt.0/client.0/tmp 2024-04-27T04:08:55.213 INFO:tasks.workunit.client.0.smithi162.stderr:+ wget -q -O ltp-full.tgz http://download.ceph.com/qa/ltp-full-20091231.tgz 2024-04-27T04:08:56.185 INFO:tasks.workunit.client.0.smithi162.stderr:+ tar xzf ltp-full.tgz 2024-04-27T04:08:56.637 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:08:56 smithi162 ceph-mon[26841]: pgmap v233: 65 pgs: 65 active+clean; 25 MiB data, 1.8 GiB used, 535 GiB / 536 GiB avail; 351 KiB/s rd, 2.2 MiB/s wr, 220 op/s 2024-04-27T04:08:56.672 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:08:56 smithi181 ceph-mon[33774]: pgmap v233: 65 pgs: 65 active+clean; 25 MiB data, 1.8 GiB used, 535 GiB / 536 GiB avail; 351 KiB/s rd, 2.2 MiB/s wr, 220 op/s 2024-04-27T04:08:58.637 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:08:58 smithi162 ceph-mon[26841]: pgmap v234: 65 pgs: 65 active+clean; 27 MiB data, 1.8 GiB used, 535 GiB / 536 GiB avail; 351 KiB/s rd, 2.4 MiB/s wr, 238 op/s 2024-04-27T04:08:58.672 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:08:58 smithi181 ceph-mon[33774]: pgmap v234: 65 pgs: 65 active+clean; 27 MiB data, 1.8 GiB used, 535 GiB / 536 GiB avail; 351 KiB/s rd, 2.4 MiB/s wr, 238 op/s 2024-04-27T04:09:00.637 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:09:00 smithi162 ceph-mon[26841]: pgmap v235: 65 pgs: 65 active+clean; 49 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail; 1.0 MiB/s rd, 4.2 MiB/s wr, 321 op/s 2024-04-27T04:09:00.672 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:09:00 smithi181 ceph-mon[33774]: pgmap v235: 65 pgs: 65 active+clean; 49 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail; 1.0 MiB/s rd, 4.2 MiB/s wr, 321 op/s 2024-04-27T04:09:02.637 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:09:02 smithi162 ceph-mon[26841]: pgmap v236: 65 pgs: 65 active+clean; 51 MiB data, 1.9 GiB used, 534 GiB / 536 GiB avail; 978 KiB/s rd, 3.0 MiB/s wr, 277 op/s 2024-04-27T04:09:02.637 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:09:02 smithi162 ceph-mon[26841]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-27T04:09:02.672 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:09:02 smithi181 ceph-mon[33774]: pgmap v236: 65 pgs: 65 active+clean; 51 MiB data, 1.9 GiB used, 534 GiB / 536 GiB avail; 978 KiB/s rd, 3.0 MiB/s wr, 277 op/s 2024-04-27T04:09:02.672 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:09:02 smithi181 ceph-mon[33774]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-27T04:09:04.637 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:09:04 smithi162 ceph-mon[26841]: pgmap v237: 65 pgs: 65 active+clean; 67 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail; 1.4 MiB/s rd, 4.5 MiB/s wr, 379 op/s 2024-04-27T04:09:04.672 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:09:04 smithi181 ceph-mon[33774]: pgmap v237: 65 pgs: 65 active+clean; 67 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail; 1.4 MiB/s rd, 4.5 MiB/s wr, 379 op/s 2024-04-27T04:09:06.637 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:09:06 smithi162 ceph-mon[26841]: pgmap v238: 65 pgs: 65 active+clean; 73 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail; 1.5 MiB/s rd, 4.5 MiB/s wr, 362 op/s 2024-04-27T04:09:06.672 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:09:06 smithi181 ceph-mon[33774]: pgmap v238: 65 pgs: 65 active+clean; 73 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail; 1.5 MiB/s rd, 4.5 MiB/s wr, 362 op/s 2024-04-27T04:09:08.637 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:09:08 smithi162 ceph-mon[26841]: pgmap v239: 65 pgs: 65 active+clean; 74 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail; 1.5 MiB/s rd, 4.3 MiB/s wr, 345 op/s 2024-04-27T04:09:08.672 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:09:08 smithi181 ceph-mon[33774]: pgmap v239: 65 pgs: 65 active+clean; 74 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail; 1.5 MiB/s rd, 4.3 MiB/s wr, 345 op/s 2024-04-27T04:09:10.637 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:09:10 smithi162 ceph-mon[26841]: pgmap v240: 65 pgs: 65 active+clean; 82 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail; 2.0 MiB/s rd, 4.8 MiB/s wr, 428 op/s 2024-04-27T04:09:10.672 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:09:10 smithi181 ceph-mon[33774]: pgmap v240: 65 pgs: 65 active+clean; 82 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail; 2.0 MiB/s rd, 4.8 MiB/s wr, 428 op/s 2024-04-27T04:09:12.637 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:09:12 smithi162 ceph-mon[26841]: pgmap v241: 65 pgs: 65 active+clean; 82 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail; 1.3 MiB/s rd, 2.9 MiB/s wr, 356 op/s 2024-04-27T04:09:12.672 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:09:12 smithi181 ceph-mon[33774]: pgmap v241: 65 pgs: 65 active+clean; 82 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail; 1.3 MiB/s rd, 2.9 MiB/s wr, 356 op/s 2024-04-27T04:09:13.106 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:09:13.105+0000 7fb6037fe700 1 -- 172.21.15.162:0/2126443095 <== mon.0 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb5f8013b70 con 0x7fb6040fbcb0 2024-04-27T04:09:13.106 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:09:13.105+0000 7fb60b090700 1 -- 172.21.15.162:0/2126443095 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb6040fbcb0 msgr2=0x7fb6040fc0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:09:13.106 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:09:13.105+0000 7fb60b090700 1 --2- 172.21.15.162:0/2126443095 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb6040fbcb0 0x7fb6040fc0c0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fb5f8004940 tx=0x7fb5f8009e30 comp rx=0 tx=0).stop 2024-04-27T04:09:13.106 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:09:13.105+0000 7fb60b090700 1 -- 172.21.15.162:0/2126443095 shutdown_connections 2024-04-27T04:09:13.106 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:09:13.105+0000 7fb60b090700 1 --2- 172.21.15.162:0/2126443095 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb6040fcf60 0x7fb6040fd3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:09:13.106 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:09:13.105+0000 7fb60b090700 1 --2- 172.21.15.162:0/2126443095 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb6040fbcb0 0x7fb6040fc0c0 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:09:13.106 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:09:13.105+0000 7fb60b090700 1 -- 172.21.15.162:0/2126443095 >> 172.21.15.162:0/2126443095 conn(0x7fb6040f7430 msgr2=0x7fb6040f9840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:09:13.106 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:09:13.106+0000 7fb60b090700 1 -- 172.21.15.162:0/2126443095 shutdown_connections 2024-04-27T04:09:13.106 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:09:13.106+0000 7fb60b090700 1 -- 172.21.15.162:0/2126443095 wait complete. 2024-04-27T04:09:13.106 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:09:13.106+0000 7fb60b090700 1 Processor -- start 2024-04-27T04:09:13.107 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:09:13.106+0000 7fb60b090700 1 -- start start 2024-04-27T04:09:13.108 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:09:13.107+0000 7fb60b090700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb6040fbcb0 0x7fb60410b720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:09:13.108 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:09:13.107+0000 7fb60b090700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb6040fcf60 0x7fb60410bc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:09:13.108 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:09:13.107+0000 7fb60b090700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb60410c2b0 con 0x7fb6040fbcb0 2024-04-27T04:09:13.108 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:09:13.107+0000 7fb60b090700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb604111d60 con 0x7fb6040fcf60 2024-04-27T04:09:13.108 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:09:13.107+0000 7fb608e2c700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb6040fbcb0 0x7fb60410b720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:09:13.108 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:09:13.107+0000 7fb608e2c700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb6040fbcb0 0x7fb60410b720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:44598/0 (socket says 172.21.15.162:44598) 2024-04-27T04:09:13.108 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:09:13.107+0000 7fb608e2c700 1 -- 172.21.15.162:0/796341542 learned_addr learned my addr 172.21.15.162:0/796341542 (peer_addr_for_me v2:172.21.15.162:0/0) 2024-04-27T04:09:13.108 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:09:13.107+0000 7fb603fff700 1 --2- 172.21.15.162:0/796341542 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb6040fcf60 0x7fb60410bc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:09:13.108 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:09:13.107+0000 7fb608e2c700 1 -- 172.21.15.162:0/796341542 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb6040fcf60 msgr2=0x7fb60410bc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:09:13.109 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:09:13.107+0000 7fb608e2c700 1 --2- 172.21.15.162:0/796341542 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb6040fcf60 0x7fb60410bc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:09:13.109 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:09:13.107+0000 7fb608e2c700 1 -- 172.21.15.162:0/796341542 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb5f8005040 con 0x7fb6040fbcb0 2024-04-27T04:09:13.109 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:09:13.107+0000 7fb603fff700 1 --2- 172.21.15.162:0/796341542 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb6040fcf60 0x7fb60410bc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-27T04:09:13.109 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:09:13.107+0000 7fb608e2c700 1 --2- 172.21.15.162:0/796341542 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb6040fbcb0 0x7fb60410b720 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7fb5f800b0d0 tx=0x7fb5f8009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:09:13.109 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:09:13.108+0000 7fb6017fa700 1 -- 172.21.15.162:0/796341542 <== mon.0 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb5f800e030 con 0x7fb6040fbcb0 2024-04-27T04:09:13.109 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:09:13.108+0000 7fb6017fa700 1 -- 172.21.15.162:0/796341542 <== mon.0 v2:172.21.15.162:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fb5f801c400 con 0x7fb6040fbcb0 2024-04-27T04:09:13.109 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:09:13.108+0000 7fb60b090700 1 -- 172.21.15.162:0/796341542 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb604111fe0 con 0x7fb6040fbcb0 2024-04-27T04:09:13.109 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:09:13.108+0000 7fb6017fa700 1 -- 172.21.15.162:0/796341542 <== mon.0 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb5f800b630 con 0x7fb6040fbcb0 2024-04-27T04:09:13.109 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:09:13.108+0000 7fb60b090700 1 -- 172.21.15.162:0/796341542 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb6041125a0 con 0x7fb6040fbcb0 2024-04-27T04:09:13.110 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:09:13.109+0000 7fb60b090700 1 -- 172.21.15.162:0/796341542 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb60405f5c0 con 0x7fb6040fbcb0 2024-04-27T04:09:13.110 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:09:13.109+0000 7fb6017fa700 1 -- 172.21.15.162:0/796341542 <== mon.0 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 19) v1 ==== 86631+0+0 (secure 0 0 0) 0x7fb5f801c570 con 0x7fb6040fbcb0 2024-04-27T04:09:13.110 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:09:13.110+0000 7fb6017fa700 1 --2- 172.21.15.162:0/796341542 >> [v2:172.21.15.162:6800/4079782599,v1:172.21.15.162:6801/4079782599] conn(0x7fb5f40692a0 0x7fb5f406b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:09:13.110 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:09:13.110+0000 7fb6017fa700 1 -- 172.21.15.162:0/796341542 <== mon.0 v2:172.21.15.162:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5550+0+0 (secure 0 0 0) 0x7fb5f8088ab0 con 0x7fb6040fbcb0 2024-04-27T04:09:13.110 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:09:13.110+0000 7fb603fff700 1 --2- 172.21.15.162:0/796341542 >> [v2:172.21.15.162:6800/4079782599,v1:172.21.15.162:6801/4079782599] conn(0x7fb5f40692a0 0x7fb5f406b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:09:13.110 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:09:13.110+0000 7fb603fff700 1 --2- 172.21.15.162:0/796341542 >> [v2:172.21.15.162:6800/4079782599,v1:172.21.15.162:6801/4079782599] conn(0x7fb5f40692a0 0x7fb5f406b750 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fb5f0009b80 tx=0x7fb5f000b040 comp rx=0 tx=0).ready entity=mgr.14210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:09:13.114 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:09:13.114+0000 7fb6017fa700 1 -- 172.21.15.162:0/796341542 <== mon.0 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7fb5f8054e30 con 0x7fb6040fbcb0 2024-04-27T04:09:13.366 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:09:13.367+0000 7fb60b090700 1 -- 172.21.15.162:0/796341542 --> [v2:172.21.15.162:6800/4079782599,v1:172.21.15.162:6801/4079782599] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fb60410cbc0 con 0x7fb5f40692a0 2024-04-27T04:09:13.368 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:09:13.368+0000 7fb6017fa700 1 -- 172.21.15.162:0/796341542 <== mgr.14210 v2:172.21.15.162:6800/4079782599 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+439 (secure 0 0 0) 0x7fb60410cbc0 con 0x7fb5f40692a0 2024-04-27T04:09:13.370 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:09:13.370+0000 7fb60b090700 1 -- 172.21.15.162:0/796341542 >> [v2:172.21.15.162:6800/4079782599,v1:172.21.15.162:6801/4079782599] conn(0x7fb5f40692a0 msgr2=0x7fb5f406b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:09:13.370 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:09:13.370+0000 7fb60b090700 1 --2- 172.21.15.162:0/796341542 >> [v2:172.21.15.162:6800/4079782599,v1:172.21.15.162:6801/4079782599] conn(0x7fb5f40692a0 0x7fb5f406b750 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fb5f0009b80 tx=0x7fb5f000b040 comp rx=0 tx=0).stop 2024-04-27T04:09:13.370 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:09:13.370+0000 7fb60b090700 1 -- 172.21.15.162:0/796341542 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb6040fbcb0 msgr2=0x7fb60410b720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:09:13.370 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:09:13.370+0000 7fb60b090700 1 --2- 172.21.15.162:0/796341542 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb6040fbcb0 0x7fb60410b720 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7fb5f800b0d0 tx=0x7fb5f8009e30 comp rx=0 tx=0).stop 2024-04-27T04:09:13.370 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:09:13.370+0000 7fb60b090700 1 -- 172.21.15.162:0/796341542 shutdown_connections 2024-04-27T04:09:13.370 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:09:13.370+0000 7fb60b090700 1 --2- 172.21.15.162:0/796341542 >> [v2:172.21.15.162:6800/4079782599,v1:172.21.15.162:6801/4079782599] conn(0x7fb5f40692a0 0x7fb5f406b750 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:09:13.371 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:09:13.370+0000 7fb60b090700 1 --2- 172.21.15.162:0/796341542 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb6040fbcb0 0x7fb60410b720 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:09:13.371 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:09:13.370+0000 7fb60b090700 1 --2- 172.21.15.162:0/796341542 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb6040fcf60 0x7fb60410bc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:09:13.371 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:09:13.370+0000 7fb60b090700 1 -- 172.21.15.162:0/796341542 >> 172.21.15.162:0/796341542 conn(0x7fb6040f7430 msgr2=0x7fb604065b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:09:13.371 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:09:13.371+0000 7fb60b090700 1 -- 172.21.15.162:0/796341542 shutdown_connections 2024-04-27T04:09:13.371 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:09:13.371+0000 7fb60b090700 1 -- 172.21.15.162:0/796341542 wait complete. 2024-04-27T04:09:13.380 INFO:teuthology.orchestra.run.smithi162.stdout:true 2024-04-27T04:09:13.485 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:09:13.485+0000 7f248ec2d700 1 -- 172.21.15.162:0/1731331316 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f24880fbcb0 msgr2=0x7f24880fc0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:09:13.486 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:09:13.485+0000 7f248ec2d700 1 --2- 172.21.15.162:0/1731331316 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f24880fbcb0 0x7f24880fc0c0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f247c004940 tx=0x7f247c009e30 comp rx=0 tx=0).stop 2024-04-27T04:09:13.486 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:09:13.485+0000 7f248ec2d700 1 -- 172.21.15.162:0/1731331316 shutdown_connections 2024-04-27T04:09:13.486 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:09:13.485+0000 7f248ec2d700 1 --2- 172.21.15.162:0/1731331316 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f24880fcf60 0x7f24880fd3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:09:13.486 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:09:13.485+0000 7f248ec2d700 1 --2- 172.21.15.162:0/1731331316 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f24880fbcb0 0x7f24880fc0c0 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:09:13.486 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:09:13.485+0000 7f248ec2d700 1 -- 172.21.15.162:0/1731331316 >> 172.21.15.162:0/1731331316 conn(0x7f24880f7430 msgr2=0x7f24880f9840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:09:13.486 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:09:13.486+0000 7f248ec2d700 1 -- 172.21.15.162:0/1731331316 shutdown_connections 2024-04-27T04:09:13.486 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:09:13.486+0000 7f248ec2d700 1 -- 172.21.15.162:0/1731331316 wait complete. 2024-04-27T04:09:13.488 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:09:13.486+0000 7f248ec2d700 1 Processor -- start 2024-04-27T04:09:13.488 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:09:13.486+0000 7f248ec2d700 1 -- start start 2024-04-27T04:09:13.488 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:09:13.487+0000 7f248ec2d700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f24880fbcb0 0x7f248810d7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:09:13.488 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:09:13.487+0000 7f248ec2d700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f24880fcf60 0x7f248810dd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:09:13.488 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:09:13.487+0000 7f248ec2d700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f248810e300 con 0x7f24880fbcb0 2024-04-27T04:09:13.488 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:09:13.487+0000 7f248ec2d700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f248810eb50 con 0x7f24880fcf60 2024-04-27T04:09:13.488 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:09:13.487+0000 7f248c9c9700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f24880fbcb0 0x7f248810d7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:09:13.488 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:09:13.487+0000 7f248c9c9700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f24880fbcb0 0x7f248810d7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:57712/0 (socket says 172.21.15.162:57712) 2024-04-27T04:09:13.488 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:09:13.487+0000 7f248c9c9700 1 -- 172.21.15.162:0/588531590 learned_addr learned my addr 172.21.15.162:0/588531590 (peer_addr_for_me v2:172.21.15.162:0/0) 2024-04-27T04:09:13.488 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:09:13.487+0000 7f2487fff700 1 --2- 172.21.15.162:0/588531590 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f24880fcf60 0x7f248810dd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:09:13.488 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:09:13.487+0000 7f248c9c9700 1 -- 172.21.15.162:0/588531590 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f24880fcf60 msgr2=0x7f248810dd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:09:13.489 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:09:13.487+0000 7f248c9c9700 1 --2- 172.21.15.162:0/588531590 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f24880fcf60 0x7f248810dd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:09:13.489 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:09:13.487+0000 7f248c9c9700 1 -- 172.21.15.162:0/588531590 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f247c005040 con 0x7f24880fbcb0 2024-04-27T04:09:13.489 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:09:13.487+0000 7f2487fff700 1 --2- 172.21.15.162:0/588531590 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f24880fcf60 0x7f248810dd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-27T04:09:13.489 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:09:13.488+0000 7f248c9c9700 1 --2- 172.21.15.162:0/588531590 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f24880fbcb0 0x7f248810d7c0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f247c004890 tx=0x7f247c009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:09:13.489 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:09:13.488+0000 7f24857fa700 1 -- 172.21.15.162:0/588531590 <== mon.0 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f247c021070 con 0x7f24880fbcb0 2024-04-27T04:09:13.489 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:09:13.488+0000 7f24857fa700 1 -- 172.21.15.162:0/588531590 <== mon.0 v2:172.21.15.162:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f247c013dd0 con 0x7f24880fbcb0 2024-04-27T04:09:13.489 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:09:13.488+0000 7f248ec2d700 1 -- 172.21.15.162:0/588531590 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f248810edd0 con 0x7f24880fbcb0 2024-04-27T04:09:13.489 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:09:13.488+0000 7f24857fa700 1 -- 172.21.15.162:0/588531590 <== mon.0 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f247c00e070 con 0x7f24880fbcb0 2024-04-27T04:09:13.489 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:09:13.488+0000 7f248ec2d700 1 -- 172.21.15.162:0/588531590 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f248810f2b0 con 0x7f24880fbcb0 2024-04-27T04:09:13.490 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:09:13.489+0000 7f248ec2d700 1 -- 172.21.15.162:0/588531590 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2488101af0 con 0x7f24880fbcb0 2024-04-27T04:09:13.490 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:09:13.490+0000 7f24857fa700 1 -- 172.21.15.162:0/588531590 <== mon.0 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 19) v1 ==== 86631+0+0 (secure 0 0 0) 0x7f247c01cad0 con 0x7f24880fbcb0 2024-04-27T04:09:13.490 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:09:13.490+0000 7f24857fa700 1 --2- 172.21.15.162:0/588531590 >> [v2:172.21.15.162:6800/4079782599,v1:172.21.15.162:6801/4079782599] conn(0x7f24780692b0 0x7f247806b760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:09:13.490 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:09:13.490+0000 7f24857fa700 1 -- 172.21.15.162:0/588531590 <== mon.0 v2:172.21.15.162:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5550+0+0 (secure 0 0 0) 0x7f247c088520 con 0x7f24880fbcb0 2024-04-27T04:09:13.491 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:09:13.491+0000 7f2487fff700 1 --2- 172.21.15.162:0/588531590 >> [v2:172.21.15.162:6800/4079782599,v1:172.21.15.162:6801/4079782599] conn(0x7f24780692b0 0x7f247806b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:09:13.491 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:09:13.492+0000 7f2487fff700 1 --2- 172.21.15.162:0/588531590 >> [v2:172.21.15.162:6800/4079782599,v1:172.21.15.162:6801/4079782599] conn(0x7f24780692b0 0x7f247806b760 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f2474009b80 tx=0x7f247400b040 comp rx=0 tx=0).ready entity=mgr.14210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:09:13.496 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:09:13.496+0000 7f24857fa700 1 -- 172.21.15.162:0/588531590 <== mon.0 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f247c058330 con 0x7f24880fbcb0 2024-04-27T04:09:13.748 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:09:13.748+0000 7f248ec2d700 1 -- 172.21.15.162:0/588531590 --> [v2:172.21.15.162:6800/4079782599,v1:172.21.15.162:6801/4079782599] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f2488104810 con 0x7f24780692b0 2024-04-27T04:09:13.748 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:09:13.749+0000 7f24857fa700 1 -- 172.21.15.162:0/588531590 <== mgr.14210 v2:172.21.15.162:6800/4079782599 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+439 (secure 0 0 0) 0x7f2488104810 con 0x7f24780692b0 2024-04-27T04:09:13.752 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:09:13.752+0000 7f248ec2d700 1 -- 172.21.15.162:0/588531590 >> [v2:172.21.15.162:6800/4079782599,v1:172.21.15.162:6801/4079782599] conn(0x7f24780692b0 msgr2=0x7f247806b760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:09:13.752 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:09:13.752+0000 7f248ec2d700 1 --2- 172.21.15.162:0/588531590 >> [v2:172.21.15.162:6800/4079782599,v1:172.21.15.162:6801/4079782599] conn(0x7f24780692b0 0x7f247806b760 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f2474009b80 tx=0x7f247400b040 comp rx=0 tx=0).stop 2024-04-27T04:09:13.752 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:09:13.752+0000 7f248ec2d700 1 -- 172.21.15.162:0/588531590 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f24880fbcb0 msgr2=0x7f248810d7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:09:13.752 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:09:13.752+0000 7f248ec2d700 1 --2- 172.21.15.162:0/588531590 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f24880fbcb0 0x7f248810d7c0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f247c004890 tx=0x7f247c009e30 comp rx=0 tx=0).stop 2024-04-27T04:09:13.752 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:09:13.752+0000 7f248ec2d700 1 -- 172.21.15.162:0/588531590 shutdown_connections 2024-04-27T04:09:13.752 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:09:13.752+0000 7f248ec2d700 1 --2- 172.21.15.162:0/588531590 >> [v2:172.21.15.162:6800/4079782599,v1:172.21.15.162:6801/4079782599] conn(0x7f24780692b0 0x7f247806b760 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:09:13.752 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:09:13.752+0000 7f248ec2d700 1 --2- 172.21.15.162:0/588531590 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f24880fbcb0 0x7f248810d7c0 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:09:13.752 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:09:13.752+0000 7f248ec2d700 1 --2- 172.21.15.162:0/588531590 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f24880fcf60 0x7f248810dd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:09:13.752 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:09:13.752+0000 7f248ec2d700 1 -- 172.21.15.162:0/588531590 >> 172.21.15.162:0/588531590 conn(0x7f24880f7430 msgr2=0x7f24880f86f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:09:13.752 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:09:13.752+0000 7f248ec2d700 1 -- 172.21.15.162:0/588531590 shutdown_connections 2024-04-27T04:09:13.752 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:09:13.752+0000 7f248ec2d700 1 -- 172.21.15.162:0/588531590 wait complete. 2024-04-27T04:09:13.866 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:09:13.865+0000 7f629e413700 1 -- 172.21.15.162:0/3578735095 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f62980fce60 msgr2=0x7f62980fd270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:09:13.866 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:09:13.865+0000 7f629e413700 1 --2- 172.21.15.162:0/3578735095 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f62980fce60 0x7f62980fd270 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f6288004940 tx=0x7f6288009e30 comp rx=0 tx=0).stop 2024-04-27T04:09:13.866 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:09:13.865+0000 7f629e413700 1 -- 172.21.15.162:0/3578735095 shutdown_connections 2024-04-27T04:09:13.867 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:09:13.865+0000 7f629e413700 1 --2- 172.21.15.162:0/3578735095 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f62980fdfd0 0x7f62980fe440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:09:13.867 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:09:13.865+0000 7f629e413700 1 --2- 172.21.15.162:0/3578735095 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f62980fce60 0x7f62980fd270 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:09:13.867 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:09:13.865+0000 7f629e413700 1 -- 172.21.15.162:0/3578735095 >> 172.21.15.162:0/3578735095 conn(0x7f62980f8540 msgr2=0x7f62980fa950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:09:13.867 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:09:13.865+0000 7f629e413700 1 -- 172.21.15.162:0/3578735095 shutdown_connections 2024-04-27T04:09:13.867 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:09:13.865+0000 7f629e413700 1 -- 172.21.15.162:0/3578735095 wait complete. 2024-04-27T04:09:13.867 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:09:13.865+0000 7f629e413700 1 Processor -- start 2024-04-27T04:09:13.867 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:09:13.866+0000 7f629e413700 1 -- start start 2024-04-27T04:09:13.867 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:09:13.866+0000 7f629e413700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f62980fce60 0x7f629810d8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:09:13.867 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:09:13.866+0000 7f629e413700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f62980fdfd0 0x7f629810ddf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:09:13.867 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:09:13.866+0000 7f629e413700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f629810e3f0 con 0x7f62980fce60 2024-04-27T04:09:13.867 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:09:13.866+0000 7f629e413700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f629810e560 con 0x7f62980fdfd0 2024-04-27T04:09:13.867 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:09:13.866+0000 7f6297fff700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f62980fce60 0x7f629810d8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:09:13.867 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:09:13.866+0000 7f6297fff700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f62980fce60 0x7f629810d8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:57726/0 (socket says 172.21.15.162:57726) 2024-04-27T04:09:13.867 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:09:13.866+0000 7f6297fff700 1 -- 172.21.15.162:0/2569441217 learned_addr learned my addr 172.21.15.162:0/2569441217 (peer_addr_for_me v2:172.21.15.162:0/0) 2024-04-27T04:09:13.868 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:09:13.866+0000 7f62977fe700 1 --2- 172.21.15.162:0/2569441217 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f62980fdfd0 0x7f629810ddf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:09:13.868 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:09:13.866+0000 7f6297fff700 1 -- 172.21.15.162:0/2569441217 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f62980fdfd0 msgr2=0x7f629810ddf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:09:13.868 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:09:13.866+0000 7f6297fff700 1 --2- 172.21.15.162:0/2569441217 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f62980fdfd0 0x7f629810ddf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:09:13.868 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:09:13.866+0000 7f6297fff700 1 -- 172.21.15.162:0/2569441217 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6288005040 con 0x7f62980fce60 2024-04-27T04:09:13.868 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:09:13.866+0000 7f6297fff700 1 --2- 172.21.15.162:0/2569441217 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f62980fce60 0x7f629810d8b0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f6288009150 tx=0x7f6288009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:09:13.868 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:09:13.866+0000 7f6294ff9700 1 -- 172.21.15.162:0/2569441217 <== mon.0 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f628800e040 con 0x7f62980fce60 2024-04-27T04:09:13.868 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:09:13.866+0000 7f6294ff9700 1 -- 172.21.15.162:0/2569441217 <== mon.0 v2:172.21.15.162:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f6288004290 con 0x7f62980fce60 2024-04-27T04:09:13.868 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:09:13.867+0000 7f629e413700 1 -- 172.21.15.162:0/2569441217 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6298112f60 con 0x7f62980fce60 2024-04-27T04:09:13.868 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:09:13.867+0000 7f6294ff9700 1 -- 172.21.15.162:0/2569441217 <== mon.0 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f628801c630 con 0x7f62980fce60 2024-04-27T04:09:13.868 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:09:13.867+0000 7f629e413700 1 -- 172.21.15.162:0/2569441217 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f62980703d0 con 0x7f62980fce60 2024-04-27T04:09:13.869 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:09:13.868+0000 7f629e413700 1 -- 172.21.15.162:0/2569441217 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f629804c710 con 0x7f62980fce60 2024-04-27T04:09:13.869 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:09:13.868+0000 7f6294ff9700 1 -- 172.21.15.162:0/2569441217 <== mon.0 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 19) v1 ==== 86631+0+0 (secure 0 0 0) 0x7f628800bbf0 con 0x7f62980fce60 2024-04-27T04:09:13.869 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:09:13.868+0000 7f6294ff9700 1 --2- 172.21.15.162:0/2569441217 >> [v2:172.21.15.162:6800/4079782599,v1:172.21.15.162:6801/4079782599] conn(0x7f628006d6b0 0x7f628006fb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:09:13.869 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:09:13.868+0000 7f6294ff9700 1 -- 172.21.15.162:0/2569441217 <== mon.0 v2:172.21.15.162:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5550+0+0 (secure 0 0 0) 0x7f6288087eb0 con 0x7f62980fce60 2024-04-27T04:09:13.870 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:09:13.869+0000 7f62977fe700 1 --2- 172.21.15.162:0/2569441217 >> [v2:172.21.15.162:6800/4079782599,v1:172.21.15.162:6801/4079782599] conn(0x7f628006d6b0 0x7f628006fb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:09:13.870 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:09:13.869+0000 7f62977fe700 1 --2- 172.21.15.162:0/2569441217 >> [v2:172.21.15.162:6800/4079782599,v1:172.21.15.162:6801/4079782599] conn(0x7f628006d6b0 0x7f628006fb60 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f628c009fd0 tx=0x7f628c00b040 comp rx=0 tx=0).ready entity=mgr.14210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:09:13.872 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:09:13.873+0000 7f6294ff9700 1 -- 172.21.15.162:0/2569441217 <== mon.0 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f6288057c20 con 0x7f62980fce60 2024-04-27T04:09:14.079 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:09:14.079+0000 7f629e413700 1 -- 172.21.15.162:0/2569441217 --> [v2:172.21.15.162:6800/4079782599,v1:172.21.15.162:6801/4079782599] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f62981022b0 con 0x7f628006d6b0 2024-04-27T04:09:14.090 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:09:14.090+0000 7f6294ff9700 1 -- 172.21.15.162:0/2569441217 <== mgr.14210 v2:172.21.15.162:6800/4079782599 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3344 (secure 0 0 0) 0x7f62981022b0 con 0x7f628006d6b0 2024-04-27T04:09:14.091 INFO:teuthology.orchestra.run.smithi162.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-27T04:09:14.091 INFO:teuthology.orchestra.run.smithi162.stdout:alertmanager.smithi162 smithi162 *:9093,9094 running (6m) 112s ago 7m 24.6M - 0.25.0 c8568f914cd2 61972851fcce 2024-04-27T04:09:14.091 INFO:teuthology.orchestra.run.smithi162.stdout:crash.smithi162 smithi162 running (7m) 112s ago 7m 6660k - 17.2.7-907-g2be3efa9 1c5834633715 228c98df4289 2024-04-27T04:09:14.091 INFO:teuthology.orchestra.run.smithi162.stdout:crash.smithi181 smithi181 running (6m) 117s ago 6m 6665k - 17.2.7-907-g2be3efa9 1c5834633715 c8e553e3df2b 2024-04-27T04:09:14.091 INFO:teuthology.orchestra.run.smithi162.stdout:grafana.smithi162 smithi162 *:3000 running (6m) 112s ago 7m 84.2M - 9.4.7 954c08fa6188 7d3364d5cddc 2024-04-27T04:09:14.092 INFO:teuthology.orchestra.run.smithi162.stdout:mds.cephfs.smithi162.gzlyjj smithi162 running (2m) 112s ago 2m 10.4M - 17.2.7-907-g2be3efa9 1c5834633715 27e5b030dd32 2024-04-27T04:09:14.092 INFO:teuthology.orchestra.run.smithi162.stdout:mds.cephfs.smithi162.xcnapk smithi162 running (2m) 112s ago 2m 13.2M - 17.2.7-907-g2be3efa9 1c5834633715 38f9030df029 2024-04-27T04:09:14.092 INFO:teuthology.orchestra.run.smithi162.stdout:mds.cephfs.smithi181.bkkcdm smithi181 running (2m) 117s ago 2m 12.1M - 17.2.7-907-g2be3efa9 1c5834633715 af55afca1263 2024-04-27T04:09:14.092 INFO:teuthology.orchestra.run.smithi162.stdout:mds.cephfs.smithi181.dzomgh smithi181 running (2m) 117s ago 2m 15.9M - 17.2.7-907-g2be3efa9 1c5834633715 6733296597f3 2024-04-27T04:09:14.092 INFO:teuthology.orchestra.run.smithi162.stdout:mgr.smithi162.oennfn smithi162 *:9283 running (9m) 112s ago 9m 479M - 17.2.7-907-g2be3efa9 1c5834633715 e768e27ad56e 2024-04-27T04:09:14.092 INFO:teuthology.orchestra.run.smithi162.stdout:mgr.smithi181.vtwuhz smithi181 *:8443,9283 running (6m) 117s ago 6m 433M - 17.2.7-907-g2be3efa9 1c5834633715 618e7918caea 2024-04-27T04:09:14.092 INFO:teuthology.orchestra.run.smithi162.stdout:mon.smithi162 smithi162 running (9m) 112s ago 9m 53.9M 2048M 17.2.7-907-g2be3efa9 1c5834633715 0f1e3682b9ab 2024-04-27T04:09:14.092 INFO:teuthology.orchestra.run.smithi162.stdout:mon.smithi181 smithi181 running (6m) 117s ago 6m 44.5M 2048M 17.2.7-907-g2be3efa9 1c5834633715 96e48aa3912f 2024-04-27T04:09:14.092 INFO:teuthology.orchestra.run.smithi162.stdout:node-exporter.smithi162 smithi162 *:9100 running (7m) 112s ago 7m 15.7M - 1.5.0 0da6a335fe13 e4535d340cea 2024-04-27T04:09:14.092 INFO:teuthology.orchestra.run.smithi162.stdout:node-exporter.smithi181 smithi181 *:9100 running (6m) 117s ago 6m 15.2M - 1.5.0 0da6a335fe13 3cc278142657 2024-04-27T04:09:14.092 INFO:teuthology.orchestra.run.smithi162.stdout:osd.0 smithi162 running (5m) 112s ago 5m 48.2M 4288M 17.2.7-907-g2be3efa9 1c5834633715 4a9df3aa938d 2024-04-27T04:09:14.093 INFO:teuthology.orchestra.run.smithi162.stdout:osd.1 smithi162 running (5m) 112s ago 5m 48.4M 4288M 17.2.7-907-g2be3efa9 1c5834633715 aefb464c46e8 2024-04-27T04:09:14.093 INFO:teuthology.orchestra.run.smithi162.stdout:osd.2 smithi162 running (4m) 112s ago 4m 44.5M 4288M 17.2.7-907-g2be3efa9 1c5834633715 70f16d86d38e 2024-04-27T04:09:14.093 INFO:teuthology.orchestra.run.smithi162.stdout:osd.3 smithi181 running (4m) 117s ago 4m 44.5M 5312M 17.2.7-907-g2be3efa9 1c5834633715 030d829be54e 2024-04-27T04:09:14.093 INFO:teuthology.orchestra.run.smithi162.stdout:osd.4 smithi181 running (4m) 117s ago 4m 41.6M 5312M 17.2.7-907-g2be3efa9 1c5834633715 512afbee7cf7 2024-04-27T04:09:14.093 INFO:teuthology.orchestra.run.smithi162.stdout:osd.5 smithi181 running (3m) 117s ago 3m 40.9M 5312M 17.2.7-907-g2be3efa9 1c5834633715 f47a1daac1ea 2024-04-27T04:09:14.093 INFO:teuthology.orchestra.run.smithi162.stdout:prometheus.smithi162 smithi162 *:9095 running (5m) 112s ago 7m 54.6M - 2.43.0 a07b618ecd1d 63755cc04fec 2024-04-27T04:09:14.094 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:09:14.093+0000 7f629e413700 1 -- 172.21.15.162:0/2569441217 >> [v2:172.21.15.162:6800/4079782599,v1:172.21.15.162:6801/4079782599] conn(0x7f628006d6b0 msgr2=0x7f628006fb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:09:14.094 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:09:14.093+0000 7f629e413700 1 --2- 172.21.15.162:0/2569441217 >> [v2:172.21.15.162:6800/4079782599,v1:172.21.15.162:6801/4079782599] conn(0x7f628006d6b0 0x7f628006fb60 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f628c009fd0 tx=0x7f628c00b040 comp rx=0 tx=0).stop 2024-04-27T04:09:14.094 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:09:14.093+0000 7f629e413700 1 -- 172.21.15.162:0/2569441217 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f62980fce60 msgr2=0x7f629810d8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:09:14.094 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:09:14.093+0000 7f629e413700 1 --2- 172.21.15.162:0/2569441217 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f62980fce60 0x7f629810d8b0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f6288009150 tx=0x7f6288009e30 comp rx=0 tx=0).stop 2024-04-27T04:09:14.094 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:09:14.093+0000 7f629e413700 1 -- 172.21.15.162:0/2569441217 shutdown_connections 2024-04-27T04:09:14.094 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:09:14.093+0000 7f629e413700 1 --2- 172.21.15.162:0/2569441217 >> [v2:172.21.15.162:6800/4079782599,v1:172.21.15.162:6801/4079782599] conn(0x7f628006d6b0 0x7f628006fb60 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:09:14.094 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:09:14.093+0000 7f629e413700 1 --2- 172.21.15.162:0/2569441217 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f62980fce60 0x7f629810d8b0 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:09:14.095 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:09:14.093+0000 7f629e413700 1 --2- 172.21.15.162:0/2569441217 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f62980fdfd0 0x7f629810ddf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:09:14.095 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:09:14.093+0000 7f629e413700 1 -- 172.21.15.162:0/2569441217 >> 172.21.15.162:0/2569441217 conn(0x7f62980f8540 msgr2=0x7f62980f9ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:09:14.095 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:09:14.093+0000 7f629e413700 1 -- 172.21.15.162:0/2569441217 shutdown_connections 2024-04-27T04:09:14.095 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:09:14.093+0000 7f629e413700 1 -- 172.21.15.162:0/2569441217 wait complete. 2024-04-27T04:09:14.193 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:09:14.192+0000 7feceb60d700 1 -- 172.21.15.162:0/958081230 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fece40fbcb0 msgr2=0x7fece40fc0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:09:14.193 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:09:14.192+0000 7feceb60d700 1 --2- 172.21.15.162:0/958081230 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fece40fbcb0 0x7fece40fc0c0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7fece0004940 tx=0x7fece0009e30 comp rx=0 tx=0).stop 2024-04-27T04:09:14.194 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:09:14.192+0000 7feceb60d700 1 -- 172.21.15.162:0/958081230 shutdown_connections 2024-04-27T04:09:14.194 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:09:14.192+0000 7feceb60d700 1 --2- 172.21.15.162:0/958081230 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fece40fcf60 0x7fece40fd3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:09:14.194 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:09:14.192+0000 7feceb60d700 1 --2- 172.21.15.162:0/958081230 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fece40fbcb0 0x7fece40fc0c0 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:09:14.194 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:09:14.192+0000 7feceb60d700 1 -- 172.21.15.162:0/958081230 >> 172.21.15.162:0/958081230 conn(0x7fece40f7430 msgr2=0x7fece40f9840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:09:14.194 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:09:14.192+0000 7feceb60d700 1 -- 172.21.15.162:0/958081230 shutdown_connections 2024-04-27T04:09:14.194 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:09:14.192+0000 7feceb60d700 1 -- 172.21.15.162:0/958081230 wait complete. 2024-04-27T04:09:14.194 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:09:14.192+0000 7feceb60d700 1 Processor -- start 2024-04-27T04:09:14.194 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:09:14.192+0000 7feceb60d700 1 -- start start 2024-04-27T04:09:14.194 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:09:14.192+0000 7feceb60d700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fece40fbcb0 0x7fece410c920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:09:14.194 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:09:14.193+0000 7feceb60d700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fece40fcf60 0x7fece410ce60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:09:14.194 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:09:14.193+0000 7feceb60d700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fece410d4f0 con 0x7fece40fcf60 2024-04-27T04:09:14.194 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:09:14.193+0000 7feceb60d700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fece410dce0 con 0x7fece40fbcb0 2024-04-27T04:09:14.194 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:09:14.193+0000 7fece8ba8700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fece40fcf60 0x7fece410ce60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:09:14.195 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:09:14.193+0000 7fece8ba8700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fece40fcf60 0x7fece410ce60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:57756/0 (socket says 172.21.15.162:57756) 2024-04-27T04:09:14.195 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:09:14.193+0000 7fece8ba8700 1 -- 172.21.15.162:0/987234570 learned_addr learned my addr 172.21.15.162:0/987234570 (peer_addr_for_me v2:172.21.15.162:0/0) 2024-04-27T04:09:14.195 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:09:14.193+0000 7fece93a9700 1 --2- 172.21.15.162:0/987234570 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fece40fbcb0 0x7fece410c920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:09:14.195 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:09:14.193+0000 7fece8ba8700 1 -- 172.21.15.162:0/987234570 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fece40fbcb0 msgr2=0x7fece410c920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:09:14.195 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:09:14.193+0000 7fece8ba8700 1 --2- 172.21.15.162:0/987234570 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fece40fbcb0 0x7fece410c920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:09:14.195 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:09:14.193+0000 7fece8ba8700 1 -- 172.21.15.162:0/987234570 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fece0005040 con 0x7fece40fcf60 2024-04-27T04:09:14.195 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:09:14.193+0000 7fece8ba8700 1 --2- 172.21.15.162:0/987234570 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fece40fcf60 0x7fece410ce60 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7fecd400b790 tx=0x7fecd4009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:09:14.195 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:09:14.193+0000 7fecd9ffb700 1 -- 172.21.15.162:0/987234570 <== mon.0 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fecd400f6f0 con 0x7fece40fcf60 2024-04-27T04:09:14.195 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:09:14.193+0000 7fecd9ffb700 1 -- 172.21.15.162:0/987234570 <== mon.0 v2:172.21.15.162:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fecd400fd20 con 0x7fece40fcf60 2024-04-27T04:09:14.195 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:09:14.193+0000 7feceb60d700 1 -- 172.21.15.162:0/987234570 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fece410dfc0 con 0x7fece40fcf60 2024-04-27T04:09:14.195 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:09:14.193+0000 7fecd9ffb700 1 -- 172.21.15.162:0/987234570 <== mon.0 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fecd40103a0 con 0x7fece40fcf60 2024-04-27T04:09:14.196 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:09:14.193+0000 7feceb60d700 1 -- 172.21.15.162:0/987234570 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fece410e580 con 0x7fece40fcf60 2024-04-27T04:09:14.196 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:09:14.194+0000 7feceb60d700 1 -- 172.21.15.162:0/987234570 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fece405f5c0 con 0x7fece40fcf60 2024-04-27T04:09:14.196 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:09:14.195+0000 7fecd9ffb700 1 -- 172.21.15.162:0/987234570 <== mon.0 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 19) v1 ==== 86631+0+0 (secure 0 0 0) 0x7fecd4011400 con 0x7fece40fcf60 2024-04-27T04:09:14.197 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:09:14.195+0000 7fecd9ffb700 1 --2- 172.21.15.162:0/987234570 >> [v2:172.21.15.162:6800/4079782599,v1:172.21.15.162:6801/4079782599] conn(0x7fecd00692b0 0x7fecd006b760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:09:14.197 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:09:14.195+0000 7fecd9ffb700 1 -- 172.21.15.162:0/987234570 <== mon.0 v2:172.21.15.162:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5550+0+0 (secure 0 0 0) 0x7fecd4086fd0 con 0x7fece40fcf60 2024-04-27T04:09:14.197 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:09:14.195+0000 7fece93a9700 1 --2- 172.21.15.162:0/987234570 >> [v2:172.21.15.162:6800/4079782599,v1:172.21.15.162:6801/4079782599] conn(0x7fecd00692b0 0x7fecd006b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:09:14.197 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:09:14.196+0000 7fece93a9700 1 --2- 172.21.15.162:0/987234570 >> [v2:172.21.15.162:6800/4079782599,v1:172.21.15.162:6801/4079782599] conn(0x7fecd00692b0 0x7fecd006b760 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fece0009fd0 tx=0x7fece0009e30 comp rx=0 tx=0).ready entity=mgr.14210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:09:14.199 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:09:14.200+0000 7fecd9ffb700 1 -- 172.21.15.162:0/987234570 <== mon.0 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7fecd4056d40 con 0x7fece40fcf60 2024-04-27T04:09:14.535 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:09:14.535+0000 7feceb60d700 1 -- 172.21.15.162:0/987234570 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7fece404bed0 con 0x7fece40fcf60 2024-04-27T04:09:14.535 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:09:14.536+0000 7fecd9ffb700 1 -- 172.21.15.162:0/987234570 <== mon.0 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+638 (secure 0 0 0) 0x7fecd40568d0 con 0x7fece40fcf60 2024-04-27T04:09:14.535 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-04-27T04:09:14.536 INFO:teuthology.orchestra.run.smithi162.stdout: "mon": { 2024-04-27T04:09:14.536 INFO:teuthology.orchestra.run.smithi162.stdout: "ceph version 17.2.7-907-g2be3efa9 (2be3efa97a7972f8e925bd899b474329979b74d6) quincy (stable)": 2 2024-04-27T04:09:14.536 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-04-27T04:09:14.536 INFO:teuthology.orchestra.run.smithi162.stdout: "mgr": { 2024-04-27T04:09:14.536 INFO:teuthology.orchestra.run.smithi162.stdout: "ceph version 17.2.7-907-g2be3efa9 (2be3efa97a7972f8e925bd899b474329979b74d6) quincy (stable)": 2 2024-04-27T04:09:14.536 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-04-27T04:09:14.536 INFO:teuthology.orchestra.run.smithi162.stdout: "osd": { 2024-04-27T04:09:14.536 INFO:teuthology.orchestra.run.smithi162.stdout: "ceph version 17.2.7-907-g2be3efa9 (2be3efa97a7972f8e925bd899b474329979b74d6) quincy (stable)": 6 2024-04-27T04:09:14.536 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-04-27T04:09:14.536 INFO:teuthology.orchestra.run.smithi162.stdout: "mds": { 2024-04-27T04:09:14.536 INFO:teuthology.orchestra.run.smithi162.stdout: "ceph version 17.2.7-907-g2be3efa9 (2be3efa97a7972f8e925bd899b474329979b74d6) quincy (stable)": 4 2024-04-27T04:09:14.536 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-04-27T04:09:14.536 INFO:teuthology.orchestra.run.smithi162.stdout: "overall": { 2024-04-27T04:09:14.536 INFO:teuthology.orchestra.run.smithi162.stdout: "ceph version 17.2.7-907-g2be3efa9 (2be3efa97a7972f8e925bd899b474329979b74d6) quincy (stable)": 14 2024-04-27T04:09:14.537 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-04-27T04:09:14.537 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-04-27T04:09:14.538 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:09:14.538+0000 7feceb60d700 1 -- 172.21.15.162:0/987234570 >> [v2:172.21.15.162:6800/4079782599,v1:172.21.15.162:6801/4079782599] conn(0x7fecd00692b0 msgr2=0x7fecd006b760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:09:14.538 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:09:14.538+0000 7feceb60d700 1 --2- 172.21.15.162:0/987234570 >> [v2:172.21.15.162:6800/4079782599,v1:172.21.15.162:6801/4079782599] conn(0x7fecd00692b0 0x7fecd006b760 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fece0009fd0 tx=0x7fece0009e30 comp rx=0 tx=0).stop 2024-04-27T04:09:14.538 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:09:14.538+0000 7feceb60d700 1 -- 172.21.15.162:0/987234570 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fece40fcf60 msgr2=0x7fece410ce60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:09:14.538 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:09:14.538+0000 7feceb60d700 1 --2- 172.21.15.162:0/987234570 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fece40fcf60 0x7fece410ce60 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7fecd400b790 tx=0x7fecd4009250 comp rx=0 tx=0).stop 2024-04-27T04:09:14.539 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:09:14.538+0000 7feceb60d700 1 -- 172.21.15.162:0/987234570 shutdown_connections 2024-04-27T04:09:14.539 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:09:14.538+0000 7feceb60d700 1 --2- 172.21.15.162:0/987234570 >> [v2:172.21.15.162:6800/4079782599,v1:172.21.15.162:6801/4079782599] conn(0x7fecd00692b0 0x7fecd006b760 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:09:14.539 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:09:14.538+0000 7feceb60d700 1 --2- 172.21.15.162:0/987234570 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fece40fbcb0 0x7fece410c920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:09:14.539 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:09:14.538+0000 7feceb60d700 1 --2- 172.21.15.162:0/987234570 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fece40fcf60 0x7fece410ce60 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:09:14.539 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:09:14.538+0000 7feceb60d700 1 -- 172.21.15.162:0/987234570 >> 172.21.15.162:0/987234570 conn(0x7fece40f7430 msgr2=0x7fece40f8a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:09:14.539 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:09:14.538+0000 7feceb60d700 1 -- 172.21.15.162:0/987234570 shutdown_connections 2024-04-27T04:09:14.539 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:09:14.538+0000 7feceb60d700 1 -- 172.21.15.162:0/987234570 wait complete. 2024-04-27T04:09:14.637 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:09:14 smithi162 ceph-mon[26841]: pgmap v242: 65 pgs: 65 active+clean; 97 MiB data, 2.1 GiB used, 534 GiB / 536 GiB avail; 2.0 MiB/s rd, 4.1 MiB/s wr, 450 op/s 2024-04-27T04:09:14.637 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:09:14 smithi162 ceph-mon[26841]: from='client.14572 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-27T04:09:14.643 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:09:14.641+0000 7f132a695700 1 -- 172.21.15.162:0/2051843167 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f13240fe420 msgr2=0x7f13240fe830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:09:14.643 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:09:14.641+0000 7f132a695700 1 --2- 172.21.15.162:0/2051843167 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f13240fe420 0x7f13240fe830 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f1318004940 tx=0x7f1318009e30 comp rx=0 tx=0).stop 2024-04-27T04:09:14.643 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:09:14.641+0000 7f132a695700 1 -- 172.21.15.162:0/2051843167 shutdown_connections 2024-04-27T04:09:14.643 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:09:14.641+0000 7f132a695700 1 --2- 172.21.15.162:0/2051843167 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f13240fed70 0x7f13240faeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:09:14.643 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:09:14.641+0000 7f132a695700 1 --2- 172.21.15.162:0/2051843167 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f13240fe420 0x7f13240fe830 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:09:14.643 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:09:14.641+0000 7f132a695700 1 -- 172.21.15.162:0/2051843167 >> 172.21.15.162:0/2051843167 conn(0x7f13240f6be0 msgr2=0x7f13240f8ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:09:14.643 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:09:14.642+0000 7f132a695700 1 -- 172.21.15.162:0/2051843167 shutdown_connections 2024-04-27T04:09:14.643 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:09:14.642+0000 7f132a695700 1 -- 172.21.15.162:0/2051843167 wait complete. 2024-04-27T04:09:14.644 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:09:14.642+0000 7f132a695700 1 Processor -- start 2024-04-27T04:09:14.644 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:09:14.642+0000 7f132a695700 1 -- start start 2024-04-27T04:09:14.644 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:09:14.642+0000 7f132a695700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f13240fe420 0x7f13240fda30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:09:14.644 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:09:14.642+0000 7f132a695700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f13240fed70 0x7f13240fc0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:09:14.644 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:09:14.642+0000 7f132a695700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13240fe030 con 0x7f13240fed70 2024-04-27T04:09:14.644 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:09:14.642+0000 7f132a695700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13240fc620 con 0x7f13240fe420 2024-04-27T04:09:14.644 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:09:14.642+0000 7f13237fe700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f13240fed70 0x7f13240fc0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:09:14.644 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:09:14.642+0000 7f13237fe700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f13240fed70 0x7f13240fc0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:57784/0 (socket says 172.21.15.162:57784) 2024-04-27T04:09:14.644 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:09:14.643+0000 7f13237fe700 1 -- 172.21.15.162:0/3685592744 learned_addr learned my addr 172.21.15.162:0/3685592744 (peer_addr_for_me v2:172.21.15.162:0/0) 2024-04-27T04:09:14.644 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:09:14.643+0000 7f1323fff700 1 --2- 172.21.15.162:0/3685592744 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f13240fe420 0x7f13240fda30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:09:14.644 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:09:14.643+0000 7f13237fe700 1 -- 172.21.15.162:0/3685592744 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f13240fe420 msgr2=0x7f13240fda30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:09:14.644 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:09:14.643+0000 7f13237fe700 1 --2- 172.21.15.162:0/3685592744 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f13240fe420 0x7f13240fda30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:09:14.644 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:09:14.643+0000 7f13237fe700 1 -- 172.21.15.162:0/3685592744 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1318005040 con 0x7f13240fed70 2024-04-27T04:09:14.644 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:09:14.643+0000 7f13237fe700 1 --2- 172.21.15.162:0/3685592744 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f13240fed70 0x7f13240fc0e0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f1314002a80 tx=0x7f13140046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:09:14.644 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:09:14.643+0000 7f1320ff9700 1 -- 172.21.15.162:0/3685592744 <== mon.0 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1314007cc0 con 0x7f13240fed70 2024-04-27T04:09:14.645 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:09:14.643+0000 7f132a695700 1 -- 172.21.15.162:0/3685592744 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f13240fc8d0 con 0x7f13240fed70 2024-04-27T04:09:14.645 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:09:14.643+0000 7f1320ff9700 1 -- 172.21.15.162:0/3685592744 <== mon.0 v2:172.21.15.162:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f131400f040 con 0x7f13240fed70 2024-04-27T04:09:14.645 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:09:14.643+0000 7f1320ff9700 1 -- 172.21.15.162:0/3685592744 <== mon.0 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f131400dac0 con 0x7f13240fed70 2024-04-27T04:09:14.645 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:09:14.643+0000 7f132a695700 1 -- 172.21.15.162:0/3685592744 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f13240fce90 con 0x7f13240fed70 2024-04-27T04:09:14.646 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:09:14.644+0000 7f132a695700 1 -- 172.21.15.162:0/3685592744 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f132404c710 con 0x7f13240fed70 2024-04-27T04:09:14.646 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:09:14.645+0000 7f1320ff9700 1 -- 172.21.15.162:0/3685592744 <== mon.0 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 19) v1 ==== 86631+0+0 (secure 0 0 0) 0x7f131400dc20 con 0x7f13240fed70 2024-04-27T04:09:14.646 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:09:14.645+0000 7f1320ff9700 1 --2- 172.21.15.162:0/3685592744 >> [v2:172.21.15.162:6800/4079782599,v1:172.21.15.162:6801/4079782599] conn(0x7f130c0692a0 0x7f130c06b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:09:14.646 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:09:14.645+0000 7f1320ff9700 1 -- 172.21.15.162:0/3685592744 <== mon.0 v2:172.21.15.162:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5550+0+0 (secure 0 0 0) 0x7f1314086da0 con 0x7f13240fed70 2024-04-27T04:09:14.646 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:09:14.645+0000 7f1323fff700 1 --2- 172.21.15.162:0/3685592744 >> [v2:172.21.15.162:6800/4079782599,v1:172.21.15.162:6801/4079782599] conn(0x7f130c0692a0 0x7f130c06b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:09:14.646 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:09:14.645+0000 7f1323fff700 1 --2- 172.21.15.162:0/3685592744 >> [v2:172.21.15.162:6800/4079782599,v1:172.21.15.162:6801/4079782599] conn(0x7f130c0692a0 0x7f130c06b750 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f131800b970 tx=0x7f1318009e30 comp rx=0 tx=0).ready entity=mgr.14210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:09:14.649 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:09:14.650+0000 7f1320ff9700 1 -- 172.21.15.162:0/3685592744 <== mon.0 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f1314056b10 con 0x7f13240fed70 2024-04-27T04:09:14.672 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:09:14 smithi181 ceph-mon[33774]: pgmap v242: 65 pgs: 65 active+clean; 97 MiB data, 2.1 GiB used, 534 GiB / 536 GiB avail; 2.0 MiB/s rd, 4.1 MiB/s wr, 450 op/s 2024-04-27T04:09:14.672 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:09:14 smithi181 ceph-mon[33774]: from='client.14572 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-27T04:09:14.895 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:09:14.895+0000 7f132a695700 1 -- 172.21.15.162:0/3685592744 --> [v2:172.21.15.162:6800/4079782599,v1:172.21.15.162:6801/4079782599] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f1324102410 con 0x7f130c0692a0 2024-04-27T04:09:14.896 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:09:14.896+0000 7f1320ff9700 1 -- 172.21.15.162:0/3685592744 <== mgr.14210 v2:172.21.15.162:6800/4079782599 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+439 (secure 0 0 0) 0x7f1324102410 con 0x7f130c0692a0 2024-04-27T04:09:14.896 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-04-27T04:09:14.896 INFO:teuthology.orchestra.run.smithi162.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:90a3a9fa9ceb8977f6dad9ccd36fe46018d07e2118199820523a6b35301d9c16", 2024-04-27T04:09:14.896 INFO:teuthology.orchestra.run.smithi162.stdout: "in_progress": true, 2024-04-27T04:09:14.896 INFO:teuthology.orchestra.run.smithi162.stdout: "which": "Upgrading daemons of type(s) mgr", 2024-04-27T04:09:14.897 INFO:teuthology.orchestra.run.smithi162.stdout: "services_complete": [], 2024-04-27T04:09:14.897 INFO:teuthology.orchestra.run.smithi162.stdout: "progress": "0/2 daemons upgraded", 2024-04-27T04:09:14.897 INFO:teuthology.orchestra.run.smithi162.stdout: "message": "Pulling quay.ceph.io/ceph-ci/ceph@sha256:90a3a9fa9ceb8977f6dad9ccd36fe46018d07e2118199820523a6b35301d9c16 image on host smithi181", 2024-04-27T04:09:14.897 INFO:teuthology.orchestra.run.smithi162.stdout: "is_paused": false 2024-04-27T04:09:14.897 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-04-27T04:09:14.898 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:09:14.898+0000 7f132a695700 1 -- 172.21.15.162:0/3685592744 >> [v2:172.21.15.162:6800/4079782599,v1:172.21.15.162:6801/4079782599] conn(0x7f130c0692a0 msgr2=0x7f130c06b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:09:14.898 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:09:14.898+0000 7f132a695700 1 --2- 172.21.15.162:0/3685592744 >> [v2:172.21.15.162:6800/4079782599,v1:172.21.15.162:6801/4079782599] conn(0x7f130c0692a0 0x7f130c06b750 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f131800b970 tx=0x7f1318009e30 comp rx=0 tx=0).stop 2024-04-27T04:09:14.899 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:09:14.899+0000 7f132a695700 1 -- 172.21.15.162:0/3685592744 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f13240fed70 msgr2=0x7f13240fc0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:09:14.899 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:09:14.899+0000 7f132a695700 1 --2- 172.21.15.162:0/3685592744 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f13240fed70 0x7f13240fc0e0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f1314002a80 tx=0x7f13140046b0 comp rx=0 tx=0).stop 2024-04-27T04:09:14.899 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:09:14.899+0000 7f132a695700 1 -- 172.21.15.162:0/3685592744 shutdown_connections 2024-04-27T04:09:14.899 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:09:14.899+0000 7f132a695700 1 --2- 172.21.15.162:0/3685592744 >> [v2:172.21.15.162:6800/4079782599,v1:172.21.15.162:6801/4079782599] conn(0x7f130c0692a0 0x7f130c06b750 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:09:14.899 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:09:14.899+0000 7f132a695700 1 --2- 172.21.15.162:0/3685592744 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f13240fe420 0x7f13240fda30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:09:14.899 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:09:14.899+0000 7f132a695700 1 --2- 172.21.15.162:0/3685592744 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f13240fed70 0x7f13240fc0e0 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:09:14.899 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:09:14.899+0000 7f132a695700 1 -- 172.21.15.162:0/3685592744 >> 172.21.15.162:0/3685592744 conn(0x7f13240f6be0 msgr2=0x7f13240f81f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:09:14.899 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:09:14.899+0000 7f132a695700 1 -- 172.21.15.162:0/3685592744 shutdown_connections 2024-04-27T04:09:14.899 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:09:14.899+0000 7f132a695700 1 -- 172.21.15.162:0/3685592744 wait complete. 2024-04-27T04:09:15.637 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:09:15 smithi162 ceph-mon[26841]: from='client.14576 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-27T04:09:15.637 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:09:15 smithi162 ceph-mon[26841]: from='client.14580 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-27T04:09:15.637 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:09:15 smithi162 ceph-mon[26841]: from='client.? 172.21.15.162:0/987234570' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-27T04:09:15.672 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:09:15 smithi181 ceph-mon[33774]: from='client.14576 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-27T04:09:15.672 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:09:15 smithi181 ceph-mon[33774]: from='client.14580 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-27T04:09:15.672 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:09:15 smithi181 ceph-mon[33774]: from='client.? 172.21.15.162:0/987234570' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-27T04:09:16.637 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:09:16 smithi162 ceph-mon[26841]: pgmap v243: 65 pgs: 65 active+clean; 101 MiB data, 2.1 GiB used, 534 GiB / 536 GiB avail; 1.5 MiB/s rd, 3.0 MiB/s wr, 388 op/s 2024-04-27T04:09:16.637 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:09:16 smithi162 ceph-mon[26841]: from='client.14588 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-27T04:09:16.672 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:09:16 smithi181 ceph-mon[33774]: pgmap v243: 65 pgs: 65 active+clean; 101 MiB data, 2.1 GiB used, 534 GiB / 536 GiB avail; 1.5 MiB/s rd, 3.0 MiB/s wr, 388 op/s 2024-04-27T04:09:16.672 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:09:16 smithi181 ceph-mon[33774]: from='client.14588 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-27T04:09:17.672 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:09:17 smithi181 ceph-mon[33774]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-27T04:09:17.678 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:09:17 smithi162 ceph-mon[26841]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-27T04:09:18.672 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:09:18 smithi181 ceph-mon[33774]: pgmap v244: 65 pgs: 65 active+clean; 107 MiB data, 2.1 GiB used, 534 GiB / 536 GiB avail; 1.5 MiB/s rd, 3.0 MiB/s wr, 343 op/s 2024-04-27T04:09:18.887 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:09:18 smithi162 ceph-mon[26841]: pgmap v244: 65 pgs: 65 active+clean; 107 MiB data, 2.1 GiB used, 534 GiB / 536 GiB avail; 1.5 MiB/s rd, 3.0 MiB/s wr, 343 op/s 2024-04-27T04:09:20.672 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:09:20 smithi181 ceph-mon[33774]: pgmap v245: 65 pgs: 65 active+clean; 118 MiB data, 2.1 GiB used, 534 GiB / 536 GiB avail; 1.9 MiB/s rd, 3.8 MiB/s wr, 413 op/s 2024-04-27T04:09:20.888 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:09:20 smithi162 ceph-mon[26841]: pgmap v245: 65 pgs: 65 active+clean; 118 MiB data, 2.1 GiB used, 534 GiB / 536 GiB avail; 1.9 MiB/s rd, 3.8 MiB/s wr, 413 op/s 2024-04-27T04:09:22.672 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:09:22 smithi181 ceph-mon[33774]: pgmap v246: 65 pgs: 65 active+clean; 120 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail; 1.3 MiB/s rd, 3.4 MiB/s wr, 328 op/s 2024-04-27T04:09:22.887 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:09:22 smithi162 ceph-mon[26841]: pgmap v246: 65 pgs: 65 active+clean; 120 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail; 1.3 MiB/s rd, 3.4 MiB/s wr, 328 op/s 2024-04-27T04:09:23.672 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:09:23 smithi181 ceph-mon[33774]: pgmap v247: 65 pgs: 65 active+clean; 130 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail; 1.7 MiB/s rd, 4.2 MiB/s wr, 426 op/s 2024-04-27T04:09:23.887 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:09:23 smithi162 ceph-mon[26841]: pgmap v247: 65 pgs: 65 active+clean; 130 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail; 1.7 MiB/s rd, 4.2 MiB/s wr, 426 op/s 2024-04-27T04:09:25.387 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:09:25 smithi162 ceph-mon[26841]: Upgrade: Updating mgr.smithi181.vtwuhz 2024-04-27T04:09:25.387 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:09:25 smithi162 ceph-mon[26841]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' 2024-04-27T04:09:25.387 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:09:25 smithi162 ceph-mon[26841]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi181.vtwuhz", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-27T04:09:25.387 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:09:25 smithi162 ceph-mon[26841]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-27T04:09:25.387 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:09:25 smithi162 ceph-mon[26841]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T04:09:25.388 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:09:25 smithi162 ceph-mon[26841]: Deploying daemon mgr.smithi181.vtwuhz on smithi181 2024-04-27T04:09:25.422 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:09:25 smithi181 ceph-mon[33774]: Upgrade: Updating mgr.smithi181.vtwuhz 2024-04-27T04:09:25.422 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:09:25 smithi181 ceph-mon[33774]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' 2024-04-27T04:09:25.423 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:09:25 smithi181 ceph-mon[33774]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi181.vtwuhz", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-27T04:09:25.423 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:09:25 smithi181 ceph-mon[33774]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-27T04:09:25.423 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:09:25 smithi181 ceph-mon[33774]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T04:09:25.423 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:09:25 smithi181 ceph-mon[33774]: Deploying daemon mgr.smithi181.vtwuhz on smithi181 2024-04-27T04:09:26.314 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:09:26 smithi181 ceph-mon[33774]: pgmap v248: 65 pgs: 65 active+clean; 133 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail; 1.0 MiB/s rd, 3.2 MiB/s wr, 388 op/s 2024-04-27T04:09:26.387 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:09:26 smithi162 ceph-mon[26841]: pgmap v248: 65 pgs: 65 active+clean; 133 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail; 1.0 MiB/s rd, 3.2 MiB/s wr, 388 op/s 2024-04-27T04:09:28.377 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:09:28 smithi181 ceph-mon[33774]: pgmap v249: 65 pgs: 65 active+clean; 137 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 1.3 MiB/s rd, 3.2 MiB/s wr, 364 op/s 2024-04-27T04:09:28.387 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:09:28 smithi162 ceph-mon[26841]: pgmap v249: 65 pgs: 65 active+clean; 137 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 1.3 MiB/s rd, 3.2 MiB/s wr, 364 op/s 2024-04-27T04:09:30.387 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:09:30 smithi162 ceph-mon[26841]: pgmap v250: 65 pgs: 65 active+clean; 149 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 1.5 MiB/s rd, 3.7 MiB/s wr, 446 op/s 2024-04-27T04:09:30.422 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:09:30 smithi181 ceph-mon[33774]: pgmap v250: 65 pgs: 65 active+clean; 149 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 1.5 MiB/s rd, 3.7 MiB/s wr, 446 op/s 2024-04-27T04:09:32.199 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:09:32 smithi181 ceph-mon[33774]: pgmap v251: 65 pgs: 65 active+clean; 150 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 1.2 MiB/s rd, 2.9 MiB/s wr, 367 op/s 2024-04-27T04:09:32.200 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:09:32 smithi181 ceph-mon[33774]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-27T04:09:32.387 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:09:32 smithi162 ceph-mon[26841]: pgmap v251: 65 pgs: 65 active+clean; 150 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 1.2 MiB/s rd, 2.9 MiB/s wr, 367 op/s 2024-04-27T04:09:32.387 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:09:32 smithi162 ceph-mon[26841]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-27T04:09:33.637 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:09:33 smithi162 ceph-mon[26841]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' 2024-04-27T04:09:33.637 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:09:33 smithi162 ceph-mon[26841]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' 2024-04-27T04:09:33.637 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:09:33 smithi162 ceph-mon[26841]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T04:09:33.672 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:09:33 smithi181 ceph-mon[33774]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' 2024-04-27T04:09:33.672 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:09:33 smithi181 ceph-mon[33774]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' 2024-04-27T04:09:33.672 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:09:33 smithi181 ceph-mon[33774]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T04:09:34.422 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:09:34 smithi181 ceph-mon[33774]: pgmap v252: 65 pgs: 65 active+clean; 160 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 1.7 MiB/s rd, 3.6 MiB/s wr, 442 op/s 2024-04-27T04:09:34.637 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:09:34 smithi162 ceph-mon[26841]: pgmap v252: 65 pgs: 65 active+clean; 160 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 1.7 MiB/s rd, 3.6 MiB/s wr, 442 op/s 2024-04-27T04:09:36.637 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:09:36 smithi162 ceph-mon[26841]: pgmap v253: 65 pgs: 65 active+clean; 171 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 1.7 MiB/s rd, 3.6 MiB/s wr, 380 op/s 2024-04-27T04:09:36.672 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:09:36 smithi181 ceph-mon[33774]: pgmap v253: 65 pgs: 65 active+clean; 171 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 1.7 MiB/s rd, 3.6 MiB/s wr, 380 op/s 2024-04-27T04:09:36.761 INFO:tasks.workunit.client.1.smithi181.stderr:+ pushd ltp-full-20091231/testcases/kernel/fs/fsstress 2024-04-27T04:09:36.769 INFO:tasks.workunit.client.1.smithi181.stdout:~/cephtest/mnt.1/client.1/tmp/fsstress/ltp-full-20091231/testcases/kernel/fs/fsstress ~/cephtest/mnt.1/client.1/tmp/fsstress ~/cephtest/mnt.1/client.1/tmp 2024-04-27T04:09:36.769 INFO:tasks.workunit.client.1.smithi181.stderr:+ make 2024-04-27T04:09:37.609 INFO:tasks.workunit.client.1.smithi181.stdout:cc -DNO_XFS -I/home/ubuntu/cephtest/mnt.1/client.1/tmp/fsstress/ltp-full-20091231/testcases/kernel/fs/fsstress -D_LARGEFILE64_SOURCE -D_GNU_SOURCE -I../../../../include -I../../../../include -L../../../../lib fsstress.c -o fsstress 2024-04-27T04:09:38.528 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:09:38 smithi181 ceph-mon[33774]: pgmap v254: 65 pgs: 65 active+clean; 175 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 1.7 MiB/s rd, 3.7 MiB/s wr, 332 op/s 2024-04-27T04:09:38.637 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:09:38 smithi162 ceph-mon[26841]: pgmap v254: 65 pgs: 65 active+clean; 175 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 1.7 MiB/s rd, 3.7 MiB/s wr, 332 op/s 2024-04-27T04:09:39.101 INFO:tasks.workunit.client.1.smithi181.stderr:++ readlink -f fsstress 2024-04-27T04:09:39.104 INFO:tasks.workunit.client.1.smithi181.stderr:+ BIN=/home/ubuntu/cephtest/mnt.1/client.1/tmp/fsstress/ltp-full-20091231/testcases/kernel/fs/fsstress/fsstress 2024-04-27T04:09:39.104 INFO:tasks.workunit.client.1.smithi181.stderr:+ popd 2024-04-27T04:09:39.106 INFO:tasks.workunit.client.1.smithi181.stdout:~/cephtest/mnt.1/client.1/tmp/fsstress ~/cephtest/mnt.1/client.1/tmp 2024-04-27T04:09:39.106 INFO:tasks.workunit.client.1.smithi181.stderr:+ popd 2024-04-27T04:09:39.107 INFO:tasks.workunit.client.1.smithi181.stdout:~/cephtest/mnt.1/client.1/tmp 2024-04-27T04:09:39.108 INFO:tasks.workunit.client.1.smithi181.stderr:++ mktemp -d -p . 2024-04-27T04:09:39.309 INFO:tasks.workunit.client.1.smithi181.stderr:+ T=./tmp.ppze6CcZx3 2024-04-27T04:09:39.309 INFO:tasks.workunit.client.1.smithi181.stderr:+ /home/ubuntu/cephtest/mnt.1/client.1/tmp/fsstress/ltp-full-20091231/testcases/kernel/fs/fsstress/fsstress -d ./tmp.ppze6CcZx3 -l 1 -n 1000 -p 10 -v 2024-04-27T04:09:39.314 INFO:tasks.workunit.client.1.smithi181.stdout:seed = 1713942061 2024-04-27T04:09:39.320 INFO:tasks.workunit.client.1.smithi181.stdout:1/0: dread - no filename 2024-04-27T04:09:39.320 INFO:tasks.workunit.client.1.smithi181.stdout:1/1: dread - no filename 2024-04-27T04:09:39.322 INFO:tasks.workunit.client.1.smithi181.stdout:2/0: dread - no filename 2024-04-27T04:09:39.322 INFO:tasks.workunit.client.1.smithi181.stdout:0/0: getdents . 0 2024-04-27T04:09:39.323 INFO:tasks.workunit.client.1.smithi181.stdout:0/1: rename - no filename 2024-04-27T04:09:39.323 INFO:tasks.workunit.client.1.smithi181.stdout:0/2: write - no filename 2024-04-27T04:09:39.323 INFO:tasks.workunit.client.1.smithi181.stdout:0/3: dread - no filename 2024-04-27T04:09:39.327 INFO:tasks.workunit.client.1.smithi181.stdout:1/2: creat f0 x:0 0 0 2024-04-27T04:09:39.327 INFO:tasks.workunit.client.1.smithi181.stdout:1/3: read - f0 zero size 2024-04-27T04:09:39.327 INFO:tasks.workunit.client.1.smithi181.stdout:1/4: rmdir - no directory 2024-04-27T04:09:39.328 INFO:tasks.workunit.client.1.smithi181.stdout:1/5: write f0 [121172,38995] 0 2024-04-27T04:09:39.329 INFO:tasks.workunit.client.1.smithi181.stdout:1/6: chown f0 1 1 2024-04-27T04:09:39.329 INFO:tasks.workunit.client.1.smithi181.stdout:4/0: write - no filename 2024-04-27T04:09:39.329 INFO:tasks.workunit.client.1.smithi181.stdout:4/1: write - no filename 2024-04-27T04:09:39.329 INFO:tasks.workunit.client.1.smithi181.stdout:4/2: chown . 1 1 2024-04-27T04:09:39.329 INFO:tasks.workunit.client.1.smithi181.stdout:4/3: fdatasync - no filename 2024-04-27T04:09:39.329 INFO:tasks.workunit.client.1.smithi181.stdout:4/4: dwrite - no filename 2024-04-27T04:09:39.330 INFO:tasks.workunit.client.1.smithi181.stdout:4/5: dread - no filename 2024-04-27T04:09:39.330 INFO:tasks.workunit.client.1.smithi181.stdout:4/6: write - no filename 2024-04-27T04:09:39.330 INFO:tasks.workunit.client.1.smithi181.stdout:4/7: rename - no filename 2024-04-27T04:09:39.330 INFO:tasks.workunit.client.1.smithi181.stdout:2/1: getdents . 0 2024-04-27T04:09:39.331 INFO:tasks.workunit.client.1.smithi181.stdout:0/4: mknod c0 0 2024-04-27T04:09:39.331 INFO:tasks.workunit.client.1.smithi181.stdout:0/5: write - no filename 2024-04-27T04:09:39.333 INFO:tasks.workunit.client.1.smithi181.stdout:5/0: write - no filename 2024-04-27T04:09:39.333 INFO:tasks.workunit.client.1.smithi181.stdout:5/1: stat - no entries 2024-04-27T04:09:39.333 INFO:tasks.workunit.client.1.smithi181.stdout:5/2: read - no filename 2024-04-27T04:09:39.333 INFO:tasks.workunit.client.1.smithi181.stdout:5/3: dwrite - no filename 2024-04-27T04:09:39.333 INFO:tasks.workunit.client.1.smithi181.stdout:5/4: write - no filename 2024-04-27T04:09:39.333 INFO:tasks.workunit.client.1.smithi181.stdout:1/7: mkdir d1 0 2024-04-27T04:09:39.336 INFO:tasks.workunit.client.1.smithi181.stdout:3/0: mkdir d0 0 2024-04-27T04:09:39.336 INFO:tasks.workunit.client.1.smithi181.stdout:3/1: dwrite - no filename 2024-04-27T04:09:39.336 INFO:tasks.workunit.client.1.smithi181.stdout:3/2: readlink - no filename 2024-04-27T04:09:39.338 INFO:tasks.workunit.client.1.smithi181.stdout:6/0: rename - no filename 2024-04-27T04:09:39.340 INFO:tasks.workunit.client.1.smithi181.stdout:4/8: mkdir d0 0 2024-04-27T04:09:39.341 INFO:tasks.workunit.client.1.smithi181.stdout:2/2: mknod c0 0 2024-04-27T04:09:39.341 INFO:tasks.workunit.client.1.smithi181.stdout:2/3: chown c0 381 1 2024-04-27T04:09:39.341 INFO:tasks.workunit.client.1.smithi181.stdout:2/4: write - no filename 2024-04-27T04:09:39.344 INFO:tasks.workunit.client.1.smithi181.stdout:7/0: chown . 22322 1 2024-04-27T04:09:39.344 INFO:tasks.workunit.client.1.smithi181.stdout:7/1: dwrite - no filename 2024-04-27T04:09:39.344 INFO:tasks.workunit.client.1.smithi181.stdout:7/2: truncate - no filename 2024-04-27T04:09:39.344 INFO:tasks.workunit.client.1.smithi181.stdout:5/5: symlink l0 0 2024-04-27T04:09:39.346 INFO:tasks.workunit.client.1.smithi181.stdout:1/8: mkdir d1/d2 0 2024-04-27T04:09:39.346 INFO:tasks.workunit.client.1.smithi181.stdout:1/9: chown d1 93 1 2024-04-27T04:09:39.347 INFO:tasks.workunit.client.1.smithi181.stdout:1/10: stat d1/d2 0 2024-04-27T04:09:39.348 INFO:tasks.workunit.client.1.smithi181.stdout:1/11: dread f0 [0,4194304] 0 2024-04-27T04:09:39.349 INFO:tasks.workunit.client.1.smithi181.stdout:1/12: readlink - no filename 2024-04-27T04:09:39.349 INFO:tasks.workunit.client.1.smithi181.stdout:3/3: creat d0/f1 x:0 0 0 2024-04-27T04:09:39.349 INFO:tasks.workunit.client.1.smithi181.stdout:1/13: dread f0 [0,4194304] 0 2024-04-27T04:09:39.350 INFO:tasks.workunit.client.1.smithi181.stdout:2/5: creat f1 x:0 0 0 2024-04-27T04:09:39.350 INFO:tasks.workunit.client.1.smithi181.stdout:1/14: chown d1/d2 30574363 1 2024-04-27T04:09:39.350 INFO:tasks.workunit.client.1.smithi181.stdout:2/6: truncate f1 496151 0 2024-04-27T04:09:39.356 INFO:tasks.workunit.client.1.smithi181.stdout:8/0: fdatasync - no filename 2024-04-27T04:09:39.356 INFO:tasks.workunit.client.1.smithi181.stdout:8/1: readlink - no filename 2024-04-27T04:09:39.356 INFO:tasks.workunit.client.1.smithi181.stdout:8/2: rmdir - no directory 2024-04-27T04:09:39.356 INFO:tasks.workunit.client.1.smithi181.stdout:8/3: chown . 55 1 2024-04-27T04:09:39.356 INFO:tasks.workunit.client.1.smithi181.stdout:8/4: write - no filename 2024-04-27T04:09:39.356 INFO:tasks.workunit.client.1.smithi181.stdout:8/5: chown . 4881281 1 2024-04-27T04:09:39.356 INFO:tasks.workunit.client.1.smithi181.stdout:8/6: dread - no filename 2024-04-27T04:09:39.356 INFO:tasks.workunit.client.1.smithi181.stdout:8/7: chown . 1327 1 2024-04-27T04:09:39.356 INFO:tasks.workunit.client.1.smithi181.stdout:8/8: chown . 29 1 2024-04-27T04:09:39.356 INFO:tasks.workunit.client.1.smithi181.stdout:8/9: rename - no filename 2024-04-27T04:09:39.356 INFO:tasks.workunit.client.1.smithi181.stdout:8/10: link - no file 2024-04-27T04:09:39.356 INFO:tasks.workunit.client.1.smithi181.stdout:8/11: dread - no filename 2024-04-27T04:09:39.356 INFO:tasks.workunit.client.1.smithi181.stdout:8/12: dwrite - no filename 2024-04-27T04:09:39.356 INFO:tasks.workunit.client.1.smithi181.stdout:8/13: write - no filename 2024-04-27T04:09:39.357 INFO:tasks.workunit.client.1.smithi181.stdout:6/1: getdents . 0 2024-04-27T04:09:39.357 INFO:tasks.workunit.client.1.smithi181.stdout:6/2: rename - no filename 2024-04-27T04:09:39.357 INFO:tasks.workunit.client.1.smithi181.stdout:6/3: write - no filename 2024-04-27T04:09:39.357 INFO:tasks.workunit.client.1.smithi181.stdout:6/4: truncate - no filename 2024-04-27T04:09:39.361 INFO:tasks.workunit.client.1.smithi181.stdout:4/9: mkdir d0/d1 0 2024-04-27T04:09:39.361 INFO:tasks.workunit.client.1.smithi181.stdout:4/10: dread - no filename 2024-04-27T04:09:39.361 INFO:tasks.workunit.client.1.smithi181.stdout:4/11: dwrite - no filename 2024-04-27T04:09:39.362 INFO:tasks.workunit.client.1.smithi181.stdout:4/12: dread - no filename 2024-04-27T04:09:39.362 INFO:tasks.workunit.client.1.smithi181.stdout:3/4: mknod d0/c2 0 2024-04-27T04:09:39.362 INFO:tasks.workunit.client.1.smithi181.stdout:2/7: creat f2 x:0 0 0 2024-04-27T04:09:39.362 INFO:tasks.workunit.client.1.smithi181.stdout:3/5: truncate d0/f1 805400 0 2024-04-27T04:09:39.364 INFO:tasks.workunit.client.1.smithi181.stdout:9/0: dread - no filename 2024-04-27T04:09:39.364 INFO:tasks.workunit.client.1.smithi181.stdout:9/1: chown . 921 1 2024-04-27T04:09:39.364 INFO:tasks.workunit.client.1.smithi181.stdout:9/2: truncate - no filename 2024-04-27T04:09:39.365 INFO:tasks.workunit.client.1.smithi181.stdout:9/3: dread - no filename 2024-04-27T04:09:39.365 INFO:tasks.workunit.client.1.smithi181.stdout:9/4: dwrite - no filename 2024-04-27T04:09:39.365 INFO:tasks.workunit.client.1.smithi181.stdout:7/3: creat f0 x:0 0 0 2024-04-27T04:09:39.365 INFO:tasks.workunit.client.1.smithi181.stdout:7/4: chown f0 154 1 2024-04-27T04:09:39.367 INFO:tasks.workunit.client.1.smithi181.stdout:1/15: mknod d1/d2/c3 0 2024-04-27T04:09:39.370 INFO:tasks.workunit.client.1.smithi181.stdout:3/6: creat d0/f3 x:0 0 0 2024-04-27T04:09:39.371 INFO:tasks.workunit.client.1.smithi181.stdout:8/14: mknod c0 0 2024-04-27T04:09:39.376 INFO:tasks.workunit.client.1.smithi181.stdout:7/5: unlink f0 0 2024-04-27T04:09:39.377 INFO:tasks.workunit.client.1.smithi181.stdout:7/6: chown . 155 1 2024-04-27T04:09:39.377 INFO:tasks.workunit.client.1.smithi181.stdout:5/6: link l0 l1 0 2024-04-27T04:09:39.377 INFO:tasks.workunit.client.1.smithi181.stdout:5/7: fsync - no filename 2024-04-27T04:09:39.377 INFO:tasks.workunit.client.1.smithi181.stdout:5/8: write - no filename 2024-04-27T04:09:39.377 INFO:tasks.workunit.client.1.smithi181.stdout:5/9: dwrite - no filename 2024-04-27T04:09:39.380 INFO:tasks.workunit.client.1.smithi181.stdout:6/5: creat f0 x:0 0 0 2024-04-27T04:09:39.380 INFO:tasks.workunit.client.1.smithi181.stdout:6/6: read - f0 zero size 2024-04-27T04:09:39.381 INFO:tasks.workunit.client.1.smithi181.stdout:4/13: creat d0/d1/f2 x:0 0 0 2024-04-27T04:09:39.382 INFO:tasks.workunit.client.1.smithi181.stdout:4/14: dread - d0/d1/f2 zero size 2024-04-27T04:09:39.387 INFO:tasks.workunit.client.1.smithi181.stdout:2/8: link f2 f3 0 2024-04-27T04:09:39.387 INFO:tasks.workunit.client.1.smithi181.stdout:2/9: readlink - no filename 2024-04-27T04:09:39.388 INFO:tasks.workunit.client.1.smithi181.stdout:3/7: creat d0/f4 x:0 0 0 2024-04-27T04:09:39.389 INFO:tasks.workunit.client.1.smithi181.stdout:3/8: readlink - no filename 2024-04-27T04:09:39.390 INFO:tasks.workunit.client.1.smithi181.stdout:8/15: mkdir d1 0 2024-04-27T04:09:39.390 INFO:tasks.workunit.client.1.smithi181.stdout:8/16: read - no filename 2024-04-27T04:09:39.391 INFO:tasks.workunit.client.1.smithi181.stdout:1/16: dwrite f0 [0,4194304] 0 2024-04-27T04:09:39.393 INFO:tasks.workunit.client.1.smithi181.stdout:9/5: mkdir d0 0 2024-04-27T04:09:39.393 INFO:tasks.workunit.client.1.smithi181.stdout:9/6: dread - no filename 2024-04-27T04:09:39.396 INFO:tasks.workunit.client.1.smithi181.stdout:7/7: mkdir d1 0 2024-04-27T04:09:39.396 INFO:tasks.workunit.client.1.smithi181.stdout:7/8: dread - no filename 2024-04-27T04:09:39.396 INFO:tasks.workunit.client.1.smithi181.stdout:7/9: write - no filename 2024-04-27T04:09:39.396 INFO:tasks.workunit.client.1.smithi181.stdout:7/10: fsync - no filename 2024-04-27T04:09:39.396 INFO:tasks.workunit.client.1.smithi181.stdout:7/11: readlink - no filename 2024-04-27T04:09:39.397 INFO:tasks.workunit.client.1.smithi181.stdout:1/17: dread f0 [0,4194304] 0 2024-04-27T04:09:39.401 INFO:tasks.workunit.client.1.smithi181.stdout:4/15: mknod d0/d1/c3 0 2024-04-27T04:09:39.403 INFO:tasks.workunit.client.1.smithi181.stdout:2/10: mkdir d4 0 2024-04-27T04:09:39.405 INFO:tasks.workunit.client.1.smithi181.stdout:3/9: creat d0/f5 x:0 0 0 2024-04-27T04:09:39.407 INFO:tasks.workunit.client.1.smithi181.stdout:8/17: unlink c0 0 2024-04-27T04:09:39.407 INFO:tasks.workunit.client.1.smithi181.stdout:8/18: write - no filename 2024-04-27T04:09:39.408 INFO:tasks.workunit.client.1.smithi181.stdout:8/19: fsync - no filename 2024-04-27T04:09:39.408 INFO:tasks.workunit.client.1.smithi181.stdout:8/20: write - no filename 2024-04-27T04:09:39.408 INFO:tasks.workunit.client.1.smithi181.stdout:8/21: dread - no filename 2024-04-27T04:09:39.411 INFO:tasks.workunit.client.1.smithi181.stdout:1/18: creat d1/d2/f4 x:0 0 0 2024-04-27T04:09:39.412 INFO:tasks.workunit.client.1.smithi181.stdout:5/10: chown l1 9715152 1 2024-04-27T04:09:39.416 INFO:tasks.workunit.client.1.smithi181.stdout:3/10: creat d0/f6 x:0 0 0 2024-04-27T04:09:39.418 INFO:tasks.workunit.client.1.smithi181.stdout:9/7: symlink d0/l1 0 2024-04-27T04:09:39.419 INFO:tasks.workunit.client.1.smithi181.stdout:9/8: rename d0 to d0/d2 22 2024-04-27T04:09:39.419 INFO:tasks.workunit.client.1.smithi181.stdout:9/9: dwrite - no filename 2024-04-27T04:09:39.419 INFO:tasks.workunit.client.1.smithi181.stdout:9/10: dread - no filename 2024-04-27T04:09:39.419 INFO:tasks.workunit.client.1.smithi181.stdout:9/11: truncate - no filename 2024-04-27T04:09:39.420 INFO:tasks.workunit.client.1.smithi181.stdout:7/12: mkdir d1/d2 0 2024-04-27T04:09:39.420 INFO:tasks.workunit.client.1.smithi181.stdout:7/13: write - no filename 2024-04-27T04:09:39.420 INFO:tasks.workunit.client.1.smithi181.stdout:7/14: write - no filename 2024-04-27T04:09:39.420 INFO:tasks.workunit.client.1.smithi181.stdout:7/15: dwrite - no filename 2024-04-27T04:09:39.421 INFO:tasks.workunit.client.1.smithi181.stdout:9/12: readlink d0/l1 0 2024-04-27T04:09:39.421 INFO:tasks.workunit.client.1.smithi181.stdout:9/13: dwrite - no filename 2024-04-27T04:09:39.421 INFO:tasks.workunit.client.1.smithi181.stdout:9/14: truncate - no filename 2024-04-27T04:09:39.421 INFO:tasks.workunit.client.1.smithi181.stdout:7/16: chown d1/d2 131061 1 2024-04-27T04:09:39.422 INFO:tasks.workunit.client.1.smithi181.stdout:1/19: mkdir d1/d2/d5 0 2024-04-27T04:09:39.423 INFO:tasks.workunit.client.1.smithi181.stdout:9/15: chown d0/l1 76233338 1 2024-04-27T04:09:39.423 INFO:tasks.workunit.client.1.smithi181.stdout:9/16: dread - no filename 2024-04-27T04:09:39.423 INFO:tasks.workunit.client.1.smithi181.stdout:2/11: dwrite f2 [0,4194304] 0 2024-04-27T04:09:39.423 INFO:tasks.workunit.client.1.smithi181.stdout:9/17: chown d0 0 1 2024-04-27T04:09:39.423 INFO:tasks.workunit.client.1.smithi181.stdout:9/18: dwrite - no filename 2024-04-27T04:09:39.424 INFO:tasks.workunit.client.1.smithi181.stdout:9/19: stat d0/l1 0 2024-04-27T04:09:39.424 INFO:tasks.workunit.client.1.smithi181.stdout:2/12: write f3 [1914315,31664] 0 2024-04-27T04:09:39.424 INFO:tasks.workunit.client.1.smithi181.stdout:9/20: readlink d0/l1 0 2024-04-27T04:09:39.431 INFO:tasks.workunit.client.1.smithi181.stdout:2/13: dwrite f3 [0,4194304] 0 2024-04-27T04:09:39.679 INFO:tasks.workunit.client.1.smithi181.stdout:8/22: mkdir d1/d2 0 2024-04-27T04:09:39.680 INFO:tasks.workunit.client.1.smithi181.stdout:3/11: mknod d0/c7 0 2024-04-27T04:09:39.680 INFO:tasks.workunit.client.1.smithi181.stdout:3/12: truncate d0/f1 1118952 0 2024-04-27T04:09:39.692 INFO:tasks.workunit.client.1.smithi181.stdout:5/11: rename l0 to l2 0 2024-04-27T04:09:39.694 INFO:tasks.workunit.client.1.smithi181.stdout:3/13: creat d0/f8 x:0 0 0 2024-04-27T04:09:39.695 INFO:tasks.workunit.client.1.smithi181.stdout:3/14: fdatasync d0/f1 0 2024-04-27T04:09:39.696 INFO:tasks.workunit.client.1.smithi181.stdout:3/15: write d0/f6 [202244,89434] 0 2024-04-27T04:09:39.696 INFO:tasks.workunit.client.1.smithi181.stdout:7/17: symlink d1/d2/l3 0 2024-04-27T04:09:39.697 INFO:tasks.workunit.client.1.smithi181.stdout:3/16: write d0/f6 [160796,26376] 0 2024-04-27T04:09:39.697 INFO:tasks.workunit.client.1.smithi181.stdout:7/18: readlink d1/d2/l3 0 2024-04-27T04:09:39.698 INFO:tasks.workunit.client.1.smithi181.stdout:7/19: write - no filename 2024-04-27T04:09:39.698 INFO:tasks.workunit.client.1.smithi181.stdout:1/20: mkdir d1/d2/d5/d6 0 2024-04-27T04:09:39.701 INFO:tasks.workunit.client.1.smithi181.stdout:2/14: symlink d4/l5 0 2024-04-27T04:09:39.701 INFO:tasks.workunit.client.1.smithi181.stdout:2/15: readlink d4/l5 0 2024-04-27T04:09:39.702 INFO:tasks.workunit.client.1.smithi181.stdout:5/12: mknod c3 0 2024-04-27T04:09:39.702 INFO:tasks.workunit.client.1.smithi181.stdout:5/13: dread - no filename 2024-04-27T04:09:39.705 INFO:tasks.workunit.client.1.smithi181.stdout:2/16: dread f3 [0,4194304] 0 2024-04-27T04:09:39.709 INFO:tasks.workunit.client.1.smithi181.stdout:3/17: symlink d0/l9 0 2024-04-27T04:09:39.710 INFO:tasks.workunit.client.1.smithi181.stdout:3/18: write d0/f3 [998142,87081] 0 2024-04-27T04:09:39.712 INFO:tasks.workunit.client.1.smithi181.stdout:7/20: creat d1/f4 x:0 0 0 2024-04-27T04:09:39.713 INFO:tasks.workunit.client.1.smithi181.stdout:7/21: chown d1/f4 53088538 1 2024-04-27T04:09:39.713 INFO:tasks.workunit.client.1.smithi181.stdout:7/22: write d1/f4 [607323,98551] 0 2024-04-27T04:09:39.714 INFO:tasks.workunit.client.1.smithi181.stdout:1/21: creat d1/f7 x:0 0 0 2024-04-27T04:09:39.714 INFO:tasks.workunit.client.1.smithi181.stdout:9/21: link d0/l1 d0/l3 0 2024-04-27T04:09:39.714 INFO:tasks.workunit.client.1.smithi181.stdout:1/22: chown d1/d2 3572131 1 2024-04-27T04:09:39.719 INFO:tasks.workunit.client.1.smithi181.stdout:5/14: mkdir d4 0 2024-04-27T04:09:39.720 INFO:tasks.workunit.client.1.smithi181.stdout:8/23: rmdir d1/d2 0 2024-04-27T04:09:39.721 INFO:tasks.workunit.client.1.smithi181.stdout:8/24: fdatasync - no filename 2024-04-27T04:09:39.721 INFO:tasks.workunit.client.1.smithi181.stdout:8/25: unlink - no file 2024-04-27T04:09:39.722 INFO:tasks.workunit.client.1.smithi181.stdout:7/23: dwrite d1/f4 [0,4194304] 0 2024-04-27T04:09:39.723 INFO:tasks.workunit.client.1.smithi181.stdout:2/17: creat d4/f6 x:0 0 0 2024-04-27T04:09:39.724 INFO:tasks.workunit.client.1.smithi181.stdout:2/18: chown c0 1 1 2024-04-27T04:09:39.725 INFO:tasks.workunit.client.1.smithi181.stdout:3/19: mknod d0/ca 0 2024-04-27T04:09:39.725 INFO:tasks.workunit.client.1.smithi181.stdout:3/20: readlink d0/l9 0 2024-04-27T04:09:39.725 INFO:tasks.workunit.client.1.smithi181.stdout:3/21: fdatasync d0/f3 0 2024-04-27T04:09:39.729 INFO:tasks.workunit.client.1.smithi181.stdout:9/22: symlink d0/l4 0 2024-04-27T04:09:39.734 INFO:tasks.workunit.client.1.smithi181.stdout:8/26: creat d1/f3 x:0 0 0 2024-04-27T04:09:39.736 INFO:tasks.workunit.client.1.smithi181.stdout:2/19: unlink f1 0 2024-04-27T04:09:39.742 INFO:tasks.workunit.client.1.smithi181.stdout:9/23: rename d0/l4 to d0/l5 0 2024-04-27T04:09:39.742 INFO:tasks.workunit.client.1.smithi181.stdout:9/24: dwrite - no filename 2024-04-27T04:09:39.742 INFO:tasks.workunit.client.1.smithi181.stdout:9/25: dwrite - no filename 2024-04-27T04:09:39.742 INFO:tasks.workunit.client.1.smithi181.stdout:9/26: fdatasync - no filename 2024-04-27T04:09:39.742 INFO:tasks.workunit.client.1.smithi181.stdout:9/27: truncate - no filename 2024-04-27T04:09:39.743 INFO:tasks.workunit.client.1.smithi181.stdout:9/28: dwrite - no filename 2024-04-27T04:09:39.744 INFO:tasks.workunit.client.1.smithi181.stdout:5/15: symlink d4/l5 0 2024-04-27T04:09:39.747 INFO:tasks.workunit.client.1.smithi181.stdout:2/20: creat d4/f7 x:0 0 0 2024-04-27T04:09:39.749 INFO:tasks.workunit.client.1.smithi181.stdout:9/29: creat d0/f6 x:0 0 0 2024-04-27T04:09:39.749 INFO:tasks.workunit.client.1.smithi181.stdout:5/16: mknod d4/c6 0 2024-04-27T04:09:39.753 INFO:tasks.workunit.client.1.smithi181.stdout:5/17: rmdir d4 39 2024-04-27T04:09:39.753 INFO:tasks.workunit.client.1.smithi181.stdout:5/18: truncate - no filename 2024-04-27T04:09:39.755 INFO:tasks.workunit.client.1.smithi181.stdout:2/21: dread f3 [0,4194304] 0 2024-04-27T04:09:39.755 INFO:tasks.workunit.client.1.smithi181.stdout:2/22: dread - d4/f7 zero size 2024-04-27T04:09:40.637 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:09:40 smithi162 ceph-mon[26841]: pgmap v255: 65 pgs: 65 active+clean; 187 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 1.7 MiB/s rd, 4.4 MiB/s wr, 409 op/s 2024-04-27T04:09:40.637 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:09:40 smithi162 ceph-mon[26841]: Standby manager daemon smithi181.vtwuhz restarted 2024-04-27T04:09:40.637 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:09:40 smithi162 ceph-mon[26841]: Standby manager daemon smithi181.vtwuhz started 2024-04-27T04:09:40.637 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:09:40 smithi162 ceph-mon[26841]: from='mgr.? 172.21.15.181:0/3181919980' entity='mgr.smithi181.vtwuhz' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi181.vtwuhz/crt"}]: dispatch 2024-04-27T04:09:40.637 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:09:40 smithi162 ceph-mon[26841]: from='mgr.? 172.21.15.181:0/3181919980' entity='mgr.smithi181.vtwuhz' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-27T04:09:40.637 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:09:40 smithi162 ceph-mon[26841]: from='mgr.? 172.21.15.181:0/3181919980' entity='mgr.smithi181.vtwuhz' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi181.vtwuhz/key"}]: dispatch 2024-04-27T04:09:40.637 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:09:40 smithi162 ceph-mon[26841]: from='mgr.? 172.21.15.181:0/3181919980' entity='mgr.smithi181.vtwuhz' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-27T04:09:40.659 INFO:tasks.workunit.client.1.smithi181.stdout:0/6: getdents . 0 2024-04-27T04:09:40.661 INFO:tasks.workunit.client.1.smithi181.stdout:0/7: symlink l1 0 2024-04-27T04:09:40.672 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:09:40 smithi181 ceph-mon[33774]: pgmap v255: 65 pgs: 65 active+clean; 187 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 1.7 MiB/s rd, 4.4 MiB/s wr, 409 op/s 2024-04-27T04:09:40.672 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:09:40 smithi181 ceph-mon[33774]: Standby manager daemon smithi181.vtwuhz restarted 2024-04-27T04:09:40.673 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:09:40 smithi181 ceph-mon[33774]: Standby manager daemon smithi181.vtwuhz started 2024-04-27T04:09:40.673 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:09:40 smithi181 ceph-mon[33774]: from='mgr.? 172.21.15.181:0/3181919980' entity='mgr.smithi181.vtwuhz' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi181.vtwuhz/crt"}]: dispatch 2024-04-27T04:09:40.673 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:09:40 smithi181 ceph-mon[33774]: from='mgr.? 172.21.15.181:0/3181919980' entity='mgr.smithi181.vtwuhz' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-27T04:09:40.673 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:09:40 smithi181 ceph-mon[33774]: from='mgr.? 172.21.15.181:0/3181919980' entity='mgr.smithi181.vtwuhz' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi181.vtwuhz/key"}]: dispatch 2024-04-27T04:09:40.673 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:09:40 smithi181 ceph-mon[33774]: from='mgr.? 172.21.15.181:0/3181919980' entity='mgr.smithi181.vtwuhz' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-27T04:09:40.697 INFO:tasks.workunit.client.1.smithi181.stdout:1/23: write f0 [4978007,113643] 0 2024-04-27T04:09:40.698 INFO:tasks.workunit.client.1.smithi181.stdout:1/24: chown d1/d2 445078 1 2024-04-27T04:09:40.698 INFO:tasks.workunit.client.1.smithi181.stdout:1/25: chown f0 41840322 1 2024-04-27T04:09:40.706 INFO:tasks.workunit.client.1.smithi181.stdout:1/26: dwrite d1/d2/f4 [0,4194304] 0 2024-04-27T04:09:40.709 INFO:tasks.workunit.client.1.smithi181.stdout:1/27: creat d1/d2/d5/d6/f8 x:0 0 0 2024-04-27T04:09:40.716 INFO:tasks.workunit.client.1.smithi181.stdout:2/23: dwrite f2 [4194304,4194304] 0 2024-04-27T04:09:40.720 INFO:tasks.workunit.client.1.smithi181.stdout:2/24: creat d4/f8 x:0 0 0 2024-04-27T04:09:40.721 INFO:tasks.workunit.client.1.smithi181.stdout:2/25: dread - d4/f6 zero size 2024-04-27T04:09:40.725 INFO:tasks.workunit.client.1.smithi181.stdout:2/26: mkdir d4/d9 0 2024-04-27T04:09:40.725 INFO:tasks.workunit.client.1.smithi181.stdout:2/27: write f3 [4564082,1885] 0 2024-04-27T04:09:40.726 INFO:tasks.workunit.client.1.smithi181.stdout:2/28: dread - d4/f6 zero size 2024-04-27T04:09:40.728 INFO:tasks.workunit.client.1.smithi181.stdout:2/29: symlink d4/la 0 2024-04-27T04:09:40.729 INFO:tasks.workunit.client.1.smithi181.stdout:2/30: chown d4 1092481896 1 2024-04-27T04:09:40.729 INFO:tasks.workunit.client.1.smithi181.stdout:2/31: chown f3 10 1 2024-04-27T04:09:40.730 INFO:tasks.workunit.client.1.smithi181.stdout:2/32: truncate d4/f8 438505 0 2024-04-27T04:09:40.734 INFO:tasks.workunit.client.1.smithi181.stdout:2/33: dread f2 [0,4194304] 0 2024-04-27T04:09:40.736 INFO:tasks.workunit.client.1.smithi181.stdout:5/19: mknod d4/c7 0 2024-04-27T04:09:40.737 INFO:tasks.workunit.client.1.smithi181.stdout:5/20: chown c3 42012 1 2024-04-27T04:09:40.737 INFO:tasks.workunit.client.1.smithi181.stdout:2/34: fdatasync f2 0 2024-04-27T04:09:40.738 INFO:tasks.workunit.client.1.smithi181.stdout:3/22: rmdir d0 39 2024-04-27T04:09:40.740 INFO:tasks.workunit.client.1.smithi181.stdout:5/21: mknod d4/c8 0 2024-04-27T04:09:40.740 INFO:tasks.workunit.client.1.smithi181.stdout:5/22: dread - no filename 2024-04-27T04:09:40.740 INFO:tasks.workunit.client.1.smithi181.stdout:5/23: chown d4/c8 8 1 2024-04-27T04:09:40.741 INFO:tasks.workunit.client.1.smithi181.stdout:5/24: dwrite - no filename 2024-04-27T04:09:40.741 INFO:tasks.workunit.client.1.smithi181.stdout:5/25: chown d4/c6 27 1 2024-04-27T04:09:40.741 INFO:tasks.workunit.client.1.smithi181.stdout:2/35: creat d4/fb x:0 0 0 2024-04-27T04:09:40.742 INFO:tasks.workunit.client.1.smithi181.stdout:5/26: chown d4/l5 213 1 2024-04-27T04:09:40.742 INFO:tasks.workunit.client.1.smithi181.stdout:5/27: write - no filename 2024-04-27T04:09:40.742 INFO:tasks.workunit.client.1.smithi181.stdout:5/28: write - no filename 2024-04-27T04:09:40.742 INFO:tasks.workunit.client.1.smithi181.stdout:5/29: write - no filename 2024-04-27T04:09:40.743 INFO:tasks.workunit.client.1.smithi181.stdout:3/23: write d0/f5 [160098,50806] 0 2024-04-27T04:09:40.744 INFO:tasks.workunit.client.1.smithi181.stdout:8/27: fsync d1/f3 0 2024-04-27T04:09:40.745 INFO:tasks.workunit.client.1.smithi181.stdout:5/30: mknod d4/c9 0 2024-04-27T04:09:40.745 INFO:tasks.workunit.client.1.smithi181.stdout:5/31: dread - no filename 2024-04-27T04:09:40.747 INFO:tasks.workunit.client.1.smithi181.stdout:2/36: mknod d4/d9/cc 0 2024-04-27T04:09:40.750 INFO:tasks.workunit.client.1.smithi181.stdout:5/32: unlink d4/c9 0 2024-04-27T04:09:40.754 INFO:tasks.workunit.client.1.smithi181.stdout:8/28: link d1/f3 d1/f4 0 2024-04-27T04:09:40.755 INFO:tasks.workunit.client.1.smithi181.stdout:8/29: chown d1/f3 15443 1 2024-04-27T04:09:40.755 INFO:tasks.workunit.client.1.smithi181.stdout:9/30: getdents d0 0 2024-04-27T04:09:40.755 INFO:tasks.workunit.client.1.smithi181.stdout:8/30: rename d1 to d1/d5 22 2024-04-27T04:09:40.756 INFO:tasks.workunit.client.1.smithi181.stdout:5/33: unlink d4/c6 0 2024-04-27T04:09:40.756 INFO:tasks.workunit.client.1.smithi181.stdout:5/34: dread - no filename 2024-04-27T04:09:40.756 INFO:tasks.workunit.client.1.smithi181.stdout:5/35: stat d4/c7 0 2024-04-27T04:09:40.759 INFO:tasks.workunit.client.1.smithi181.stdout:2/37: link d4/fb d4/fd 0 2024-04-27T04:09:40.759 INFO:tasks.workunit.client.1.smithi181.stdout:2/38: stat f2 0 2024-04-27T04:09:40.761 INFO:tasks.workunit.client.1.smithi181.stdout:9/31: creat d0/f7 x:0 0 0 2024-04-27T04:09:40.763 INFO:tasks.workunit.client.1.smithi181.stdout:8/31: write d1/f3 [302232,25196] 0 2024-04-27T04:09:40.765 INFO:tasks.workunit.client.1.smithi181.stdout:3/24: getdents d0 0 2024-04-27T04:09:40.768 INFO:tasks.workunit.client.1.smithi181.stdout:2/39: write d4/fd [690576,3788] 0 2024-04-27T04:09:40.771 INFO:tasks.workunit.client.1.smithi181.stdout:8/32: mkdir d1/d6 0 2024-04-27T04:09:40.778 INFO:tasks.workunit.client.1.smithi181.stdout:8/33: dwrite d1/f3 [0,4194304] 0 2024-04-27T04:09:40.784 INFO:tasks.workunit.client.1.smithi181.stdout:3/25: truncate d0/f1 838707 0 2024-04-27T04:09:40.865 INFO:tasks.workunit.client.1.smithi181.stdout:8/34: mknod d1/c7 0 2024-04-27T04:09:40.866 INFO:tasks.workunit.client.1.smithi181.stdout:8/35: write d1/f3 [3016950,126688] 0 2024-04-27T04:09:40.875 INFO:tasks.workunit.client.1.smithi181.stdout:3/26: getdents d0 0 2024-04-27T04:09:40.879 INFO:tasks.workunit.client.1.smithi181.stdout:3/27: mknod d0/cb 0 2024-04-27T04:09:40.880 INFO:tasks.workunit.client.1.smithi181.stdout:3/28: fdatasync d0/f5 0 2024-04-27T04:09:40.888 INFO:tasks.workunit.client.1.smithi181.stdout:3/29: dwrite d0/f5 [0,4194304] 0 2024-04-27T04:09:40.897 INFO:tasks.workunit.client.1.smithi181.stdout:3/30: dwrite d0/f3 [0,4194304] 0 2024-04-27T04:09:41.140 INFO:tasks.workunit.client.1.smithi181.stdout:0/8: getdents . 0 2024-04-27T04:09:41.164 INFO:tasks.workunit.client.1.smithi181.stdout:1/28: truncate f0 3677663 0 2024-04-27T04:09:41.164 INFO:tasks.workunit.client.1.smithi181.stdout:1/29: chown d1/d2/d5/d6 21647273 1 2024-04-27T04:09:41.165 INFO:tasks.workunit.client.1.smithi181.stdout:1/30: fdatasync d1/d2/f4 0 2024-04-27T04:09:41.166 INFO:tasks.workunit.client.1.smithi181.stdout:2/40: fsync d4/f8 0 2024-04-27T04:09:41.166 INFO:tasks.workunit.client.1.smithi181.stdout:2/41: readlink d4/l5 0 2024-04-27T04:09:41.184 INFO:tasks.workunit.client.1.smithi181.stdout:8/36: fsync d1/f4 0 2024-04-27T04:09:41.185 INFO:tasks.workunit.client.1.smithi181.stdout:5/36: getdents d4 0 2024-04-27T04:09:41.339 INFO:tasks.workunit.client.1.smithi181.stdout:9/32: getdents d0 0 2024-04-27T04:09:41.364 INFO:tasks.workunit.client.1.smithi181.stdout:3/31: getdents d0 0 2024-04-27T04:09:41.524 INFO:tasks.workunit.client.1.smithi181.stdout:0/9: mkdir d2 0 2024-04-27T04:09:41.524 INFO:tasks.workunit.client.1.smithi181.stdout:0/10: write - no filename 2024-04-27T04:09:41.529 INFO:tasks.workunit.client.1.smithi181.stdout:2/42: mknod d4/ce 0 2024-04-27T04:09:41.530 INFO:tasks.workunit.client.1.smithi181.stdout:2/43: read - d4/f6 zero size 2024-04-27T04:09:41.531 INFO:tasks.workunit.client.1.smithi181.stdout:8/37: symlink d1/l8 0 2024-04-27T04:09:41.532 INFO:tasks.workunit.client.1.smithi181.stdout:5/37: rmdir d4 39 2024-04-27T04:09:41.534 INFO:tasks.workunit.client.1.smithi181.stdout:9/33: creat d0/f8 x:0 0 0 2024-04-27T04:09:41.537 INFO:tasks.workunit.client.1.smithi181.stdout:3/32: rmdir d0 39 2024-04-27T04:09:41.541 INFO:tasks.workunit.client.1.smithi181.stdout:9/34: dwrite d0/f7 [0,4194304] 0 2024-04-27T04:09:41.543 INFO:tasks.workunit.client.1.smithi181.stdout:2/44: unlink d4/fd 0 2024-04-27T04:09:41.548 INFO:tasks.workunit.client.1.smithi181.stdout:5/38: chown l2 25666 1 2024-04-27T04:09:41.549 INFO:tasks.workunit.client.1.smithi181.stdout:5/39: dwrite - no filename 2024-04-27T04:09:41.558 INFO:tasks.workunit.client.1.smithi181.stdout:2/45: mkdir d4/d9/df 0 2024-04-27T04:09:41.559 INFO:tasks.workunit.client.1.smithi181.stdout:8/38: mknod d1/d6/c9 0 2024-04-27T04:09:41.561 INFO:tasks.workunit.client.1.smithi181.stdout:8/39: truncate d1/f3 4760686 0 2024-04-27T04:09:41.562 INFO:tasks.workunit.client.1.smithi181.stdout:3/33: symlink d0/lc 0 2024-04-27T04:09:41.563 INFO:tasks.workunit.client.1.smithi181.stdout:0/11: rmdir d2 0 2024-04-27T04:09:41.565 INFO:tasks.workunit.client.1.smithi181.stdout:9/35: link d0/f6 d0/f9 0 2024-04-27T04:09:41.567 INFO:tasks.workunit.client.1.smithi181.stdout:5/40: creat d4/fa x:0 0 0 2024-04-27T04:09:41.569 INFO:tasks.workunit.client.1.smithi181.stdout:8/40: mkdir d1/da 0 2024-04-27T04:09:41.570 INFO:tasks.workunit.client.1.smithi181.stdout:3/34: symlink d0/ld 0 2024-04-27T04:09:41.573 INFO:tasks.workunit.client.1.smithi181.stdout:0/12: creat f3 x:0 0 0 2024-04-27T04:09:41.573 INFO:tasks.workunit.client.1.smithi181.stdout:0/13: fdatasync f3 0 2024-04-27T04:09:41.574 INFO:tasks.workunit.client.1.smithi181.stdout:0/14: chown c0 436 1 2024-04-27T04:09:41.574 INFO:tasks.workunit.client.1.smithi181.stdout:9/36: creat d0/fa x:0 0 0 2024-04-27T04:09:41.637 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:09:41 smithi162 ceph-mon[26841]: mgrmap e20: smithi162.oennfn(active, since 7m), standbys: smithi181.vtwuhz 2024-04-27T04:09:41.672 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:09:41 smithi181 ceph-mon[33774]: mgrmap e20: smithi162.oennfn(active, since 7m), standbys: smithi181.vtwuhz 2024-04-27T04:09:41.713 INFO:tasks.workunit.client.1.smithi181.stdout:5/41: rename l2 to d4/lb 0 2024-04-27T04:09:41.714 INFO:tasks.workunit.client.1.smithi181.stdout:5/42: read - d4/fa zero size 2024-04-27T04:09:41.717 INFO:tasks.workunit.client.1.smithi181.stdout:0/15: dwrite f3 [0,4194304] 0 2024-04-27T04:09:41.717 INFO:tasks.workunit.client.1.smithi181.stdout:3/35: mkdir d0/de 0 2024-04-27T04:09:41.722 INFO:tasks.workunit.client.1.smithi181.stdout:5/43: symlink d4/lc 0 2024-04-27T04:09:41.725 INFO:tasks.workunit.client.1.smithi181.stdout:5/44: write d4/fa [1041388,103551] 0 2024-04-27T04:09:41.726 INFO:tasks.workunit.client.1.smithi181.stdout:0/16: symlink l4 0 2024-04-27T04:09:41.729 INFO:tasks.workunit.client.1.smithi181.stdout:9/37: dwrite d0/f9 [0,4194304] 0 2024-04-27T04:09:41.730 INFO:tasks.workunit.client.1.smithi181.stdout:9/38: write d0/f8 [138563,46485] 0 2024-04-27T04:09:41.731 INFO:tasks.workunit.client.1.smithi181.stdout:9/39: write d0/fa [576478,5321] 0 2024-04-27T04:09:41.735 INFO:tasks.workunit.client.1.smithi181.stdout:3/36: creat d0/de/ff x:0 0 0 2024-04-27T04:09:41.736 INFO:tasks.workunit.client.1.smithi181.stdout:0/17: symlink l5 0 2024-04-27T04:09:41.736 INFO:tasks.workunit.client.1.smithi181.stdout:0/18: readlink l5 0 2024-04-27T04:09:41.737 INFO:tasks.workunit.client.1.smithi181.stdout:0/19: write f3 [1666104,7506] 0 2024-04-27T04:09:41.741 INFO:tasks.workunit.client.1.smithi181.stdout:5/45: unlink l1 0 2024-04-27T04:09:41.742 INFO:tasks.workunit.client.1.smithi181.stdout:5/46: truncate d4/fa 2132284 0 2024-04-27T04:09:41.746 INFO:tasks.workunit.client.1.smithi181.stdout:0/20: creat f6 x:0 0 0 2024-04-27T04:09:41.746 INFO:tasks.workunit.client.1.smithi181.stdout:0/21: stat l1 0 2024-04-27T04:09:41.750 INFO:tasks.workunit.client.1.smithi181.stdout:5/47: unlink c3 0 2024-04-27T04:09:41.752 INFO:tasks.workunit.client.1.smithi181.stdout:0/22: creat f7 x:0 0 0 2024-04-27T04:09:41.755 INFO:tasks.workunit.client.1.smithi181.stdout:1/31: fdatasync f0 0 2024-04-27T04:09:41.757 INFO:tasks.workunit.client.1.smithi181.stdout:0/23: dread f3 [0,4194304] 0 2024-04-27T04:09:41.758 INFO:tasks.workunit.client.1.smithi181.stdout:5/48: dread d4/fa [0,4194304] 0 2024-04-27T04:09:41.760 INFO:tasks.workunit.client.1.smithi181.stdout:1/32: symlink d1/d2/d5/d6/l9 0 2024-04-27T04:09:41.761 INFO:tasks.workunit.client.1.smithi181.stdout:1/33: chown d1/d2/c3 310577141 1 2024-04-27T04:09:41.761 INFO:tasks.workunit.client.1.smithi181.stdout:0/24: creat f8 x:0 0 0 2024-04-27T04:09:41.762 INFO:tasks.workunit.client.1.smithi181.stdout:1/34: write d1/f7 [115429,5437] 0 2024-04-27T04:09:41.762 INFO:tasks.workunit.client.1.smithi181.stdout:0/25: write f6 [343638,115360] 0 2024-04-27T04:09:41.765 INFO:tasks.workunit.client.1.smithi181.stdout:5/49: unlink d4/c8 0 2024-04-27T04:09:41.766 INFO:tasks.workunit.client.1.smithi181.stdout:1/35: dread f0 [0,4194304] 0 2024-04-27T04:09:41.767 INFO:tasks.workunit.client.1.smithi181.stdout:0/26: unlink l4 0 2024-04-27T04:09:41.769 INFO:tasks.workunit.client.1.smithi181.stdout:5/50: write d4/fa [2326084,35044] 0 2024-04-27T04:09:41.770 INFO:tasks.workunit.client.1.smithi181.stdout:1/36: dread f0 [0,4194304] 0 2024-04-27T04:09:41.771 INFO:tasks.workunit.client.1.smithi181.stdout:0/27: creat f9 x:0 0 0 2024-04-27T04:09:41.771 INFO:tasks.workunit.client.1.smithi181.stdout:1/37: read d1/f7 [116364,42872] 0 2024-04-27T04:09:41.772 INFO:tasks.workunit.client.1.smithi181.stdout:5/51: symlink d4/ld 0 2024-04-27T04:09:41.772 INFO:tasks.workunit.client.1.smithi181.stdout:1/38: read - d1/d2/d5/d6/f8 zero size 2024-04-27T04:09:41.775 INFO:tasks.workunit.client.1.smithi181.stdout:0/28: creat fa x:0 0 0 2024-04-27T04:09:41.776 INFO:tasks.workunit.client.1.smithi181.stdout:5/52: mkdir d4/de 0 2024-04-27T04:09:41.777 INFO:tasks.workunit.client.1.smithi181.stdout:1/39: creat d1/d2/d5/d6/fa x:0 0 0 2024-04-27T04:09:41.782 INFO:tasks.workunit.client.1.smithi181.stdout:0/29: rename f3 to fb 0 2024-04-27T04:09:41.782 INFO:tasks.workunit.client.1.smithi181.stdout:0/30: rmdir - no directory 2024-04-27T04:09:41.783 INFO:tasks.workunit.client.1.smithi181.stdout:0/31: write fa [452551,7291] 0 2024-04-27T04:09:41.784 INFO:tasks.workunit.client.1.smithi181.stdout:0/32: dread - f7 zero size 2024-04-27T04:09:41.784 INFO:tasks.workunit.client.1.smithi181.stdout:5/53: mkdir d4/de/df 0 2024-04-27T04:09:41.784 INFO:tasks.workunit.client.1.smithi181.stdout:1/40: dwrite d1/d2/f4 [0,4194304] 0 2024-04-27T04:09:41.788 INFO:tasks.workunit.client.1.smithi181.stdout:0/33: mkdir dc 0 2024-04-27T04:09:41.791 INFO:tasks.workunit.client.1.smithi181.stdout:1/41: mkdir d1/d2/d5/d6/db 0 2024-04-27T04:09:41.795 INFO:tasks.workunit.client.1.smithi181.stdout:1/42: readlink d1/d2/d5/d6/l9 0 2024-04-27T04:09:41.795 INFO:tasks.workunit.client.1.smithi181.stdout:5/54: dwrite d4/fa [0,4194304] 0 2024-04-27T04:09:41.798 INFO:tasks.workunit.client.1.smithi181.stdout:5/55: mkdir d4/d10 0 2024-04-27T04:09:41.800 INFO:tasks.workunit.client.1.smithi181.stdout:0/34: dwrite fa [0,4194304] 0 2024-04-27T04:09:41.801 INFO:tasks.workunit.client.1.smithi181.stdout:1/43: unlink d1/d2/c3 0 2024-04-27T04:09:41.801 INFO:tasks.workunit.client.1.smithi181.stdout:0/35: chown f7 1961488 1 2024-04-27T04:09:41.806 INFO:tasks.workunit.client.1.smithi181.stdout:5/56: dread d4/fa [0,4194304] 0 2024-04-27T04:09:41.810 INFO:tasks.workunit.client.1.smithi181.stdout:0/36: rename f9 to dc/fd 0 2024-04-27T04:09:41.814 INFO:tasks.workunit.client.1.smithi181.stdout:1/44: dwrite d1/d2/d5/d6/fa [0,4194304] 0 2024-04-27T04:09:41.817 INFO:tasks.workunit.client.1.smithi181.stdout:0/37: mknod dc/ce 0 2024-04-27T04:09:41.817 INFO:tasks.workunit.client.1.smithi181.stdout:0/38: chown c0 49754510 1 2024-04-27T04:09:41.818 INFO:tasks.workunit.client.1.smithi181.stdout:1/45: symlink d1/lc 0 2024-04-27T04:09:41.819 INFO:tasks.workunit.client.1.smithi181.stdout:1/46: write d1/d2/f4 [2853084,54763] 0 2024-04-27T04:09:41.820 INFO:tasks.workunit.client.1.smithi181.stdout:5/57: mknod d4/de/df/c11 0 2024-04-27T04:09:41.828 INFO:tasks.workunit.client.1.smithi181.stdout:5/58: symlink d4/d10/l12 0 2024-04-27T04:09:41.833 INFO:tasks.workunit.client.1.smithi181.stdout:1/47: dwrite d1/f7 [0,4194304] 0 2024-04-27T04:09:41.834 INFO:tasks.workunit.client.1.smithi181.stdout:5/59: rename d4/fa to d4/f13 0 2024-04-27T04:09:41.835 INFO:tasks.workunit.client.1.smithi181.stdout:1/48: truncate d1/d2/d5/d6/f8 923165 0 2024-04-27T04:09:41.835 INFO:tasks.workunit.client.1.smithi181.stdout:1/49: chown d1 31737753 1 2024-04-27T04:09:41.838 INFO:tasks.workunit.client.1.smithi181.stdout:5/60: creat d4/de/f14 x:0 0 0 2024-04-27T04:09:41.840 INFO:tasks.workunit.client.1.smithi181.stdout:1/50: unlink d1/lc 0 2024-04-27T04:09:41.843 INFO:tasks.workunit.client.1.smithi181.stdout:1/51: write d1/d2/d5/d6/fa [1787153,27973] 0 2024-04-27T04:09:41.846 INFO:tasks.workunit.client.1.smithi181.stdout:1/52: chown d1/d2/d5/d6 2018368 1 2024-04-27T04:09:41.847 INFO:tasks.workunit.client.1.smithi181.stdout:5/61: dwrite d4/de/f14 [0,4194304] 0 2024-04-27T04:09:41.848 INFO:tasks.workunit.client.1.smithi181.stdout:5/62: read d4/f13 [453029,60482] 0 2024-04-27T04:09:41.849 INFO:tasks.workunit.client.1.smithi181.stdout:2/46: getdents d4/d9 0 2024-04-27T04:09:41.852 INFO:tasks.workunit.client.1.smithi181.stdout:5/63: mknod d4/c15 0 2024-04-27T04:09:41.854 INFO:tasks.workunit.client.1.smithi181.stdout:1/53: rename d1/d2/d5/d6/f8 to d1/d2/d5/fd 0 2024-04-27T04:09:41.856 INFO:tasks.workunit.client.1.smithi181.stdout:2/47: mknod d4/d9/c10 0 2024-04-27T04:09:41.858 INFO:tasks.workunit.client.1.smithi181.stdout:1/54: dread d1/f7 [0,4194304] 0 2024-04-27T04:09:41.860 INFO:tasks.workunit.client.1.smithi181.stdout:5/64: rename d4/l5 to d4/de/l16 0 2024-04-27T04:09:41.866 INFO:tasks.workunit.client.1.smithi181.stdout:1/55: mkdir d1/d2/d5/d6/de 0 2024-04-27T04:09:41.874 INFO:tasks.workunit.client.1.smithi181.stdout:1/56: creat d1/d2/d5/d6/de/ff x:0 0 0 2024-04-27T04:09:41.875 INFO:tasks.workunit.client.1.smithi181.stdout:2/48: link d4/fb d4/d9/f11 0 2024-04-27T04:09:41.877 INFO:tasks.workunit.client.1.smithi181.stdout:2/49: chown f3 101073625 1 2024-04-27T04:09:41.877 INFO:tasks.workunit.client.1.smithi181.stdout:1/57: fdatasync d1/f7 0 2024-04-27T04:09:41.882 INFO:tasks.workunit.client.1.smithi181.stdout:0/39: dwrite fb [4194304,4194304] 0 2024-04-27T04:09:41.883 INFO:tasks.workunit.client.1.smithi181.stdout:0/40: chown f6 14141 1 2024-04-27T04:09:41.885 INFO:tasks.workunit.client.1.smithi181.stdout:1/58: dread f0 [0,4194304] 0 2024-04-27T04:09:41.889 INFO:tasks.workunit.client.1.smithi181.stdout:0/41: creat dc/ff x:0 0 0 2024-04-27T04:09:41.889 INFO:tasks.workunit.client.1.smithi181.stdout:0/42: dread - f7 zero size 2024-04-27T04:09:41.889 INFO:tasks.workunit.client.1.smithi181.stdout:0/43: chown f8 45692265 1 2024-04-27T04:09:41.890 INFO:tasks.workunit.client.1.smithi181.stdout:2/50: dread d4/fb [0,4194304] 0 2024-04-27T04:09:41.890 INFO:tasks.workunit.client.1.smithi181.stdout:0/44: readlink l1 0 2024-04-27T04:09:41.891 INFO:tasks.workunit.client.1.smithi181.stdout:0/45: read fa [1503830,127131] 0 2024-04-27T04:09:41.892 INFO:tasks.workunit.client.1.smithi181.stdout:1/59: symlink d1/d2/l10 0 2024-04-27T04:09:41.894 INFO:tasks.workunit.client.1.smithi181.stdout:8/41: truncate d1/f3 2853237 0 2024-04-27T04:09:41.898 INFO:tasks.workunit.client.1.smithi181.stdout:0/46: dwrite fb [4194304,4194304] 0 2024-04-27T04:09:41.899 INFO:tasks.workunit.client.1.smithi181.stdout:2/51: rename d4/d9/cc to d4/d9/c12 0 2024-04-27T04:09:41.904 INFO:tasks.workunit.client.1.smithi181.stdout:8/42: mknod d1/cb 0 2024-04-27T04:09:41.905 INFO:tasks.workunit.client.1.smithi181.stdout:3/37: rmdir d0/de 39 2024-04-27T04:09:41.905 INFO:tasks.workunit.client.1.smithi181.stdout:3/38: chown d0/lc 987233 1 2024-04-27T04:09:41.910 INFO:tasks.workunit.client.1.smithi181.stdout:1/60: symlink d1/d2/d5/d6/db/l11 0 2024-04-27T04:09:41.912 INFO:tasks.workunit.client.1.smithi181.stdout:3/39: dwrite d0/f4 [0,4194304] 0 2024-04-27T04:09:41.913 INFO:tasks.workunit.client.1.smithi181.stdout:3/40: readlink d0/ld 0 2024-04-27T04:09:41.914 INFO:tasks.workunit.client.1.smithi181.stdout:9/40: truncate d0/f6 2066279 0 2024-04-27T04:09:41.916 INFO:tasks.workunit.client.1.smithi181.stdout:2/52: symlink d4/d9/df/l13 0 2024-04-27T04:09:41.921 INFO:tasks.workunit.client.1.smithi181.stdout:1/61: symlink d1/d2/d5/d6/db/l12 0 2024-04-27T04:09:41.926 INFO:tasks.workunit.client.1.smithi181.stdout:2/53: mkdir d4/d9/df/d14 0 2024-04-27T04:09:41.929 INFO:tasks.workunit.client.1.smithi181.stdout:1/62: creat d1/d2/f13 x:0 0 0 2024-04-27T04:09:41.933 INFO:tasks.workunit.client.1.smithi181.stdout:3/41: rename d0/f8 to d0/de/f10 0 2024-04-27T04:09:41.938 INFO:tasks.workunit.client.1.smithi181.stdout:1/63: dwrite d1/d2/d5/d6/de/ff [0,4194304] 0 2024-04-27T04:09:41.938 INFO:tasks.workunit.client.1.smithi181.stdout:1/64: chown d1/d2/d5 341 1 2024-04-27T04:09:41.943 INFO:tasks.workunit.client.1.smithi181.stdout:1/65: dread d1/d2/f4 [0,4194304] 0 2024-04-27T04:09:41.950 INFO:tasks.workunit.client.1.smithi181.stdout:2/54: dread d4/d9/f11 [0,4194304] 0 2024-04-27T04:09:41.950 INFO:tasks.workunit.client.1.smithi181.stdout:2/55: chown d4/f6 16932 1 2024-04-27T04:09:41.951 INFO:tasks.workunit.client.1.smithi181.stdout:1/66: symlink d1/d2/d5/d6/de/l14 0 2024-04-27T04:09:41.954 INFO:tasks.workunit.client.1.smithi181.stdout:1/67: dread f0 [0,4194304] 0 2024-04-27T04:09:41.955 INFO:tasks.workunit.client.1.smithi181.stdout:1/68: stat d1/d2/f13 0 2024-04-27T04:09:41.956 INFO:tasks.workunit.client.1.smithi181.stdout:3/42: rename d0/ld to d0/l11 0 2024-04-27T04:09:41.959 INFO:tasks.workunit.client.1.smithi181.stdout:2/56: creat d4/d9/df/d14/f15 x:0 0 0 2024-04-27T04:09:41.960 INFO:tasks.workunit.client.1.smithi181.stdout:1/69: symlink d1/d2/d5/d6/db/l15 0 2024-04-27T04:09:41.964 INFO:tasks.workunit.client.1.smithi181.stdout:3/43: dwrite d0/f6 [0,4194304] 0 2024-04-27T04:09:41.964 INFO:tasks.workunit.client.1.smithi181.stdout:3/44: chown d0/c2 276 1 2024-04-27T04:09:41.967 INFO:tasks.workunit.client.1.smithi181.stdout:1/70: unlink d1/d2/d5/d6/fa 0 2024-04-27T04:09:41.972 INFO:tasks.workunit.client.1.smithi181.stdout:1/71: creat d1/f16 x:0 0 0 2024-04-27T04:09:41.972 INFO:tasks.workunit.client.1.smithi181.stdout:3/45: dread d0/f1 [0,4194304] 0 2024-04-27T04:09:41.974 INFO:tasks.workunit.client.1.smithi181.stdout:3/46: rmdir d0 39 2024-04-27T04:09:41.981 INFO:tasks.workunit.client.1.smithi181.stdout:1/72: link f0 d1/d2/d5/d6/de/f17 0 2024-04-27T04:09:41.981 INFO:tasks.workunit.client.1.smithi181.stdout:3/47: dread d0/f4 [0,4194304] 0 2024-04-27T04:09:41.982 INFO:tasks.workunit.client.1.smithi181.stdout:1/73: chown d1/d2/d5/d6/de 55 1 2024-04-27T04:09:41.983 INFO:tasks.workunit.client.1.smithi181.stdout:5/65: fsync d4/de/f14 0 2024-04-27T04:09:41.985 INFO:tasks.workunit.client.1.smithi181.stdout:1/74: dread f0 [0,4194304] 0 2024-04-27T04:09:41.986 INFO:tasks.workunit.client.1.smithi181.stdout:1/75: chown d1/d2/f4 238962 1 2024-04-27T04:09:41.987 INFO:tasks.workunit.client.1.smithi181.stdout:3/48: creat d0/de/f12 x:0 0 0 2024-04-27T04:09:41.987 INFO:tasks.workunit.client.1.smithi181.stdout:1/76: stat d1/d2/d5/d6/db 0 2024-04-27T04:09:41.993 INFO:tasks.workunit.client.1.smithi181.stdout:1/77: fdatasync d1/d2/d5/fd 0 2024-04-27T04:09:41.994 INFO:tasks.workunit.client.1.smithi181.stdout:1/78: truncate d1/d2/f13 191877 0 2024-04-27T04:09:41.995 INFO:tasks.workunit.client.1.smithi181.stdout:1/79: chown d1/d2/d5/d6/db/l15 24748533 1 2024-04-27T04:09:42.008 INFO:tasks.workunit.client.1.smithi181.stdout:0/47: rmdir dc 39 2024-04-27T04:09:42.009 INFO:tasks.workunit.client.1.smithi181.stdout:0/48: chown l1 1 1 2024-04-27T04:09:42.010 INFO:tasks.workunit.client.1.smithi181.stdout:0/49: dread - dc/fd zero size 2024-04-27T04:09:42.012 INFO:tasks.workunit.client.1.smithi181.stdout:8/43: unlink d1/f4 0 2024-04-27T04:09:42.014 INFO:tasks.workunit.client.1.smithi181.stdout:0/50: creat dc/f10 x:0 0 0 2024-04-27T04:09:42.018 INFO:tasks.workunit.client.1.smithi181.stdout:0/51: creat dc/f11 x:0 0 0 2024-04-27T04:09:42.020 INFO:tasks.workunit.client.1.smithi181.stdout:0/52: creat dc/f12 x:0 0 0 2024-04-27T04:09:42.022 INFO:tasks.workunit.client.1.smithi181.stdout:8/44: rmdir d1/da 0 2024-04-27T04:09:42.025 INFO:tasks.workunit.client.1.smithi181.stdout:0/53: mknod dc/c13 0 2024-04-27T04:09:42.025 INFO:tasks.workunit.client.1.smithi181.stdout:0/54: stat c0 0 2024-04-27T04:09:42.026 INFO:tasks.workunit.client.1.smithi181.stdout:8/45: mknod d1/d6/cc 0 2024-04-27T04:09:42.027 INFO:tasks.workunit.client.1.smithi181.stdout:8/46: chown d1/l8 928169 1 2024-04-27T04:09:42.028 INFO:tasks.workunit.client.1.smithi181.stdout:0/55: mknod dc/c14 0 2024-04-27T04:09:42.083 INFO:tasks.workunit.client.1.smithi181.stdout:9/41: truncate d0/f7 3098321 0 2024-04-27T04:09:42.088 INFO:tasks.workunit.client.1.smithi181.stdout:9/42: dwrite d0/fa [0,4194304] 0 2024-04-27T04:09:42.089 INFO:tasks.workunit.client.1.smithi181.stdout:9/43: read d0/fa [1311388,120413] 0 2024-04-27T04:09:42.116 INFO:tasks.workunit.client.1.smithi181.stdout:3/49: readlink d0/l11 0 2024-04-27T04:09:42.116 INFO:tasks.workunit.client.1.smithi181.stdout:9/44: mknod d0/cb 0 2024-04-27T04:09:42.117 INFO:tasks.workunit.client.1.smithi181.stdout:3/50: dread - d0/de/f10 zero size 2024-04-27T04:09:42.126 INFO:tasks.workunit.client.1.smithi181.stdout:1/80: rmdir d1/d2 39 2024-04-27T04:09:42.134 INFO:tasks.workunit.client.1.smithi181.stdout:5/66: truncate d4/de/f14 94954 0 2024-04-27T04:09:42.265 INFO:tasks.workunit.client.1.smithi181.stdout:9/45: creat d0/fc x:0 0 0 2024-04-27T04:09:42.265 INFO:tasks.workunit.client.1.smithi181.stdout:3/51: mknod d0/de/c13 0 2024-04-27T04:09:42.272 INFO:tasks.workunit.client.1.smithi181.stdout:9/46: creat d0/fd x:0 0 0 2024-04-27T04:09:42.272 INFO:tasks.workunit.client.1.smithi181.stdout:3/52: read d0/f4 [3901955,27383] 0 2024-04-27T04:09:42.272 INFO:tasks.workunit.client.1.smithi181.stdout:9/47: write d0/f8 [1194325,56718] 0 2024-04-27T04:09:42.273 INFO:tasks.workunit.client.1.smithi181.stdout:1/81: symlink d1/l18 0 2024-04-27T04:09:42.279 INFO:tasks.workunit.client.1.smithi181.stdout:9/48: rename d0/fa to d0/fe 0 2024-04-27T04:09:42.282 INFO:tasks.workunit.client.1.smithi181.stdout:9/49: creat d0/ff x:0 0 0 2024-04-27T04:09:42.283 INFO:tasks.workunit.client.1.smithi181.stdout:9/50: chown d0 55391375 1 2024-04-27T04:09:42.284 INFO:tasks.workunit.client.1.smithi181.stdout:1/82: creat d1/d2/d5/d6/db/f19 x:0 0 0 2024-04-27T04:09:42.285 INFO:tasks.workunit.client.1.smithi181.stdout:3/53: link d0/f6 d0/f14 0 2024-04-27T04:09:42.287 INFO:tasks.workunit.client.1.smithi181.stdout:9/51: write d0/f9 [2034032,24626] 0 2024-04-27T04:09:42.287 INFO:tasks.workunit.client.1.smithi181.stdout:9/52: stat d0/f9 0 2024-04-27T04:09:42.287 INFO:tasks.workunit.client.1.smithi181.stdout:1/83: truncate f0 3524638 0 2024-04-27T04:09:42.288 INFO:tasks.workunit.client.1.smithi181.stdout:9/53: truncate d0/f8 1307485 0 2024-04-27T04:09:42.288 INFO:tasks.workunit.client.1.smithi181.stdout:9/54: chown d0 27 1 2024-04-27T04:09:42.290 INFO:tasks.workunit.client.1.smithi181.stdout:1/84: creat d1/d2/d5/d6/f1a x:0 0 0 2024-04-27T04:09:42.293 INFO:tasks.workunit.client.1.smithi181.stdout:9/55: creat d0/f10 x:0 0 0 2024-04-27T04:09:42.293 INFO:tasks.workunit.client.1.smithi181.stdout:1/85: mkdir d1/d2/d1b 0 2024-04-27T04:09:42.296 INFO:tasks.workunit.client.1.smithi181.stdout:9/56: creat d0/f11 x:0 0 0 2024-04-27T04:09:42.296 INFO:tasks.workunit.client.1.smithi181.stdout:3/54: link d0/f4 d0/f15 0 2024-04-27T04:09:42.298 INFO:tasks.workunit.client.1.smithi181.stdout:9/57: creat d0/f12 x:0 0 0 2024-04-27T04:09:42.303 INFO:tasks.workunit.client.1.smithi181.stdout:3/55: rename d0/de/f12 to d0/f16 0 2024-04-27T04:09:42.307 INFO:tasks.workunit.client.1.smithi181.stdout:3/56: dread d0/f14 [0,4194304] 0 2024-04-27T04:09:42.422 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:09:42 smithi181 ceph-mon[33774]: pgmap v256: 65 pgs: 65 active+clean; 195 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 1.5 MiB/s rd, 4.4 MiB/s wr, 322 op/s 2024-04-27T04:09:42.613 INFO:tasks.workunit.client.1.smithi181.stdout:9/58: rename d0/fe to d0/f13 0 2024-04-27T04:09:42.618 INFO:tasks.workunit.client.1.smithi181.stdout:3/57: fsync d0/f14 0 2024-04-27T04:09:42.621 INFO:tasks.workunit.client.1.smithi181.stdout:3/58: creat d0/de/f17 x:0 0 0 2024-04-27T04:09:42.637 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:09:42 smithi162 ceph-mon[26841]: pgmap v256: 65 pgs: 65 active+clean; 195 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 1.5 MiB/s rd, 4.4 MiB/s wr, 322 op/s 2024-04-27T04:09:42.641 INFO:tasks.workunit.client.1.smithi181.stdout:3/59: dwrite d0/f14 [0,4194304] 0 2024-04-27T04:09:42.645 INFO:tasks.workunit.client.1.smithi181.stdout:3/60: symlink d0/l18 0 2024-04-27T04:09:42.649 INFO:tasks.workunit.client.1.smithi181.stdout:3/61: creat d0/f19 x:0 0 0 2024-04-27T04:09:42.650 INFO:tasks.workunit.client.1.smithi181.stdout:3/62: chown d0/l9 8169964 1 2024-04-27T04:09:42.693 INFO:tasks.workunit.client.1.smithi181.stdout:5/67: dwrite d4/de/f14 [0,4194304] 0 2024-04-27T04:09:42.694 INFO:tasks.workunit.client.1.smithi181.stdout:5/68: chown d4/de/f14 50 1 2024-04-27T04:09:42.695 INFO:tasks.workunit.client.1.smithi181.stdout:5/69: write d4/de/f14 [2155597,94371] 0 2024-04-27T04:09:42.699 INFO:tasks.workunit.client.1.smithi181.stdout:5/70: readlink d4/de/l16 0 2024-04-27T04:09:42.704 INFO:tasks.workunit.client.1.smithi181.stdout:5/71: rename d4/de/l16 to d4/d10/l17 0 2024-04-27T04:09:42.704 INFO:tasks.workunit.client.1.smithi181.stdout:5/72: chown d4/ld 8 1 2024-04-27T04:09:42.705 INFO:tasks.workunit.client.1.smithi181.stdout:5/73: read d4/de/f14 [3327398,10228] 0 2024-04-27T04:09:42.712 INFO:tasks.workunit.client.1.smithi181.stdout:5/74: dwrite d4/de/f14 [0,4194304] 0 2024-04-27T04:09:42.715 INFO:tasks.workunit.client.1.smithi181.stdout:5/75: mkdir d4/d18 0 2024-04-27T04:09:42.720 INFO:tasks.workunit.client.1.smithi181.stdout:5/76: mkdir d4/d19 0 2024-04-27T04:09:42.723 INFO:tasks.workunit.client.1.smithi181.stdout:5/77: unlink d4/de/df/c11 0 2024-04-27T04:09:42.725 INFO:tasks.workunit.client.1.smithi181.stdout:5/78: fdatasync d4/f13 0 2024-04-27T04:09:42.729 INFO:tasks.workunit.client.1.smithi181.stdout:5/79: dread d4/f13 [0,4194304] 0 2024-04-27T04:09:42.731 INFO:tasks.workunit.client.1.smithi181.stdout:5/80: mknod d4/c1a 0 2024-04-27T04:09:42.736 INFO:tasks.workunit.client.1.smithi181.stdout:5/81: dread d4/de/f14 [0,4194304] 0 2024-04-27T04:09:42.738 INFO:tasks.workunit.client.1.smithi181.stdout:8/47: dwrite d1/f3 [0,4194304] 0 2024-04-27T04:09:42.740 INFO:tasks.workunit.client.1.smithi181.stdout:5/82: creat d4/d19/f1b x:0 0 0 2024-04-27T04:09:42.742 INFO:tasks.workunit.client.1.smithi181.stdout:8/48: rename d1/d6/cc to d1/cd 0 2024-04-27T04:09:42.743 INFO:tasks.workunit.client.1.smithi181.stdout:3/63: rmdir d0/de 39 2024-04-27T04:09:42.745 INFO:tasks.workunit.client.1.smithi181.stdout:5/83: write d4/de/f14 [2547635,120875] 0 2024-04-27T04:09:42.746 INFO:tasks.workunit.client.1.smithi181.stdout:8/49: dread d1/f3 [0,4194304] 0 2024-04-27T04:09:42.747 INFO:tasks.workunit.client.1.smithi181.stdout:8/50: read d1/f3 [3117816,120229] 0 2024-04-27T04:09:42.748 INFO:tasks.workunit.client.1.smithi181.stdout:3/64: rename d0 to d0/de/d1a 22 2024-04-27T04:09:42.748 INFO:tasks.workunit.client.1.smithi181.stdout:3/65: chown d0 603266555 1 2024-04-27T04:09:42.749 INFO:tasks.workunit.client.1.smithi181.stdout:3/66: write d0/f6 [1832907,105119] 0 2024-04-27T04:09:42.755 INFO:tasks.workunit.client.1.smithi181.stdout:8/51: mkdir d1/de 0 2024-04-27T04:09:42.760 INFO:tasks.workunit.client.1.smithi181.stdout:8/52: rename d1/f3 to d1/ff 0 2024-04-27T04:09:42.769 INFO:tasks.workunit.client.1.smithi181.stdout:8/53: dwrite d1/ff [4194304,4194304] 0 2024-04-27T04:09:42.770 INFO:tasks.workunit.client.1.smithi181.stdout:8/54: write d1/ff [2656813,37261] 0 2024-04-27T04:09:42.789 INFO:tasks.workunit.client.1.smithi181.stdout:8/55: write d1/ff [8647914,69038] 0 2024-04-27T04:09:42.792 INFO:tasks.workunit.client.1.smithi181.stdout:8/56: mknod d1/c10 0 2024-04-27T04:09:42.793 INFO:tasks.workunit.client.1.smithi181.stdout:8/57: fdatasync d1/ff 0 2024-04-27T04:09:42.795 INFO:tasks.workunit.client.1.smithi181.stdout:8/58: write d1/ff [4533734,70071] 0 2024-04-27T04:09:42.796 INFO:tasks.workunit.client.1.smithi181.stdout:5/84: dwrite d4/f13 [0,4194304] 0 2024-04-27T04:09:42.797 INFO:tasks.workunit.client.1.smithi181.stdout:8/59: read d1/ff [6757338,99162] 0 2024-04-27T04:09:42.801 INFO:tasks.workunit.client.1.smithi181.stdout:9/59: write d0/f7 [3026521,82442] 0 2024-04-27T04:09:42.803 INFO:tasks.workunit.client.1.smithi181.stdout:9/60: truncate d0/f10 220042 0 2024-04-27T04:09:42.804 INFO:tasks.workunit.client.1.smithi181.stdout:5/85: mknod d4/d18/c1c 0 2024-04-27T04:09:42.805 INFO:tasks.workunit.client.1.smithi181.stdout:8/60: dwrite d1/ff [4194304,4194304] 0 2024-04-27T04:09:42.807 INFO:tasks.workunit.client.1.smithi181.stdout:5/86: symlink d4/l1d 0 2024-04-27T04:09:42.809 INFO:tasks.workunit.client.1.smithi181.stdout:5/87: write d4/f13 [2705842,14122] 0 2024-04-27T04:09:42.810 INFO:tasks.workunit.client.1.smithi181.stdout:5/88: readlink d4/l1d 0 2024-04-27T04:09:42.811 INFO:tasks.workunit.client.1.smithi181.stdout:8/61: dread d1/ff [0,4194304] 0 2024-04-27T04:09:42.812 INFO:tasks.workunit.client.1.smithi181.stdout:5/89: creat d4/de/f1e x:0 0 0 2024-04-27T04:09:42.813 INFO:tasks.workunit.client.1.smithi181.stdout:8/62: write d1/ff [8692288,123014] 0 2024-04-27T04:09:42.816 INFO:tasks.workunit.client.1.smithi181.stdout:5/90: rename d4/d18/c1c to d4/de/c1f 0 2024-04-27T04:09:42.816 INFO:tasks.workunit.client.1.smithi181.stdout:5/91: chown d4/lc 10 1 2024-04-27T04:09:42.818 INFO:tasks.workunit.client.1.smithi181.stdout:5/92: truncate d4/de/f1e 509290 0 2024-04-27T04:09:42.818 INFO:tasks.workunit.client.1.smithi181.stdout:5/93: chown d4/d10 14303227 1 2024-04-27T04:09:42.819 INFO:tasks.workunit.client.1.smithi181.stdout:8/63: getdents d1/d6 0 2024-04-27T04:09:42.820 INFO:tasks.workunit.client.1.smithi181.stdout:8/64: mkdir d1/d11 0 2024-04-27T04:09:42.820 INFO:tasks.workunit.client.1.smithi181.stdout:8/65: stat d1/c10 0 2024-04-27T04:09:42.821 INFO:tasks.workunit.client.1.smithi181.stdout:5/94: rename d4/ld to d4/de/df/l20 0 2024-04-27T04:09:42.822 INFO:tasks.workunit.client.1.smithi181.stdout:8/66: mknod d1/c12 0 2024-04-27T04:09:42.823 INFO:tasks.workunit.client.1.smithi181.stdout:5/95: mknod d4/de/df/c21 0 2024-04-27T04:09:42.828 INFO:tasks.workunit.client.1.smithi181.stdout:5/96: getdents d4/d19 0 2024-04-27T04:09:42.829 INFO:tasks.workunit.client.1.smithi181.stdout:8/67: dwrite d1/ff [4194304,4194304] 0 2024-04-27T04:09:42.829 INFO:tasks.workunit.client.1.smithi181.stdout:5/97: chown d4/de/df/c21 2450012 1 2024-04-27T04:09:42.833 INFO:tasks.workunit.client.1.smithi181.stdout:8/68: symlink d1/d6/l13 0 2024-04-27T04:09:42.837 INFO:tasks.workunit.client.1.smithi181.stdout:8/69: symlink d1/d6/l14 0 2024-04-27T04:09:42.843 INFO:tasks.workunit.client.1.smithi181.stdout:5/98: getdents d4/d10 0 2024-04-27T04:09:42.843 INFO:tasks.workunit.client.1.smithi181.stdout:5/99: chown d4/d19/f1b 3750 1 2024-04-27T04:09:42.843 INFO:tasks.workunit.client.1.smithi181.stdout:5/100: read d4/de/f14 [3006681,103044] 0 2024-04-27T04:09:42.853 INFO:tasks.workunit.client.1.smithi181.stdout:5/101: getdents d4/de/df 0 2024-04-27T04:09:42.856 INFO:tasks.workunit.client.1.smithi181.stdout:5/102: mknod d4/de/c22 0 2024-04-27T04:09:42.858 INFO:tasks.workunit.client.1.smithi181.stdout:5/103: creat d4/d18/f23 x:0 0 0 2024-04-27T04:09:42.860 INFO:tasks.workunit.client.1.smithi181.stdout:5/104: creat d4/d18/f24 x:0 0 0 2024-04-27T04:09:42.867 INFO:tasks.workunit.client.1.smithi181.stdout:5/105: dwrite d4/d18/f23 [0,4194304] 0 2024-04-27T04:09:42.877 INFO:tasks.workunit.client.1.smithi181.stdout:5/106: dwrite d4/de/f1e [0,4194304] 0 2024-04-27T04:09:42.878 INFO:tasks.workunit.client.1.smithi181.stdout:5/107: mknod d4/d19/c25 0 2024-04-27T04:09:42.883 INFO:tasks.workunit.client.1.smithi181.stdout:5/108: link d4/de/df/l20 d4/d18/l26 0 2024-04-27T04:09:44.579 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:09:44 smithi181 ceph-mon[33774]: pgmap v257: 65 pgs: 65 active+clean; 243 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 2.0 MiB/s rd, 8.5 MiB/s wr, 374 op/s 2024-04-27T04:09:44.637 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:09:44 smithi162 ceph-mon[26841]: pgmap v257: 65 pgs: 65 active+clean; 243 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 2.0 MiB/s rd, 8.5 MiB/s wr, 374 op/s 2024-04-27T04:09:45.035 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:09:45.034+0000 7fc34f006700 1 -- 172.21.15.162:0/1476194322 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc348066ed0 msgr2=0x7fc3480ff9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:09:45.035 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:09:45.034+0000 7fc34f006700 1 --2- 172.21.15.162:0/1476194322 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc348066ed0 0x7fc3480ff9a0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7fc33c004940 tx=0x7fc33c009e30 comp rx=0 tx=0).stop 2024-04-27T04:09:45.035 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:09:45.035+0000 7fc34f006700 1 -- 172.21.15.162:0/1476194322 shutdown_connections 2024-04-27T04:09:45.035 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:09:45.035+0000 7fc34f006700 1 --2- 172.21.15.162:0/1476194322 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc3480ffee0 0x7fc348102300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:09:45.035 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:09:45.035+0000 7fc34f006700 1 --2- 172.21.15.162:0/1476194322 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc348066ed0 0x7fc3480ff9a0 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:09:45.036 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:09:45.035+0000 7fc34f006700 1 -- 172.21.15.162:0/1476194322 >> 172.21.15.162:0/1476194322 conn(0x7fc3480f7430 msgr2=0x7fc3480f9840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:09:45.036 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:09:45.035+0000 7fc34f006700 1 -- 172.21.15.162:0/1476194322 shutdown_connections 2024-04-27T04:09:45.036 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:09:45.035+0000 7fc34f006700 1 -- 172.21.15.162:0/1476194322 wait complete. 2024-04-27T04:09:45.036 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:09:45.035+0000 7fc34f006700 1 Processor -- start 2024-04-27T04:09:45.036 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:09:45.035+0000 7fc34f006700 1 -- start start 2024-04-27T04:09:45.036 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:09:45.036+0000 7fc34f006700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc348066ed0 0x7fc3480fd1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:09:45.036 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:09:45.036+0000 7fc34f006700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc3480ffee0 0x7fc3480fb8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:09:45.036 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:09:45.036+0000 7fc34f006700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3480fd7a0 con 0x7fc3480ffee0 2024-04-27T04:09:45.036 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:09:45.036+0000 7fc34f006700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3480fbde0 con 0x7fc348066ed0 2024-04-27T04:09:45.036 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:09:45.036+0000 7fc347fff700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc3480ffee0 0x7fc3480fb8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:09:45.036 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:09:45.036+0000 7fc347fff700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc3480ffee0 0x7fc3480fb8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:60136/0 (socket says 172.21.15.162:60136) 2024-04-27T04:09:45.036 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:09:45.036+0000 7fc347fff700 1 -- 172.21.15.162:0/17290197 learned_addr learned my addr 172.21.15.162:0/17290197 (peer_addr_for_me v2:172.21.15.162:0/0) 2024-04-27T04:09:45.036 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:09:45.036+0000 7fc34cda2700 1 --2- 172.21.15.162:0/17290197 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc348066ed0 0x7fc3480fd1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:09:45.037 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:09:45.036+0000 7fc34cda2700 1 -- 172.21.15.162:0/17290197 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc3480ffee0 msgr2=0x7fc3480fb8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:09:45.037 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:09:45.036+0000 7fc34cda2700 1 --2- 172.21.15.162:0/17290197 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc3480ffee0 0x7fc3480fb8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:09:45.037 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:09:45.036+0000 7fc34cda2700 1 -- 172.21.15.162:0/17290197 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc33c005040 con 0x7fc348066ed0 2024-04-27T04:09:45.037 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:09:45.036+0000 7fc347fff700 1 --2- 172.21.15.162:0/17290197 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc3480ffee0 0x7fc3480fb8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-27T04:09:45.037 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:09:45.036+0000 7fc34cda2700 1 --2- 172.21.15.162:0/17290197 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc348066ed0 0x7fc3480fd1a0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fc33c000c00 tx=0x7fc33c009e30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:09:45.037 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:09:45.037+0000 7fc3457fa700 1 -- 172.21.15.162:0/17290197 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc33c00e030 con 0x7fc348066ed0 2024-04-27T04:09:45.037 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:09:45.037+0000 7fc34f006700 1 -- 172.21.15.162:0/17290197 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc3480fc060 con 0x7fc348066ed0 2024-04-27T04:09:45.038 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:09:45.037+0000 7fc3457fa700 1 -- 172.21.15.162:0/17290197 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fc33c013de0 con 0x7fc348066ed0 2024-04-27T04:09:45.038 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:09:45.037+0000 7fc3457fa700 1 -- 172.21.15.162:0/17290197 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc33c01cdc0 con 0x7fc348066ed0 2024-04-27T04:09:45.038 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:09:45.037+0000 7fc34f006700 1 -- 172.21.15.162:0/17290197 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc3480fc5a0 con 0x7fc348066ed0 2024-04-27T04:09:45.038 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:09:45.038+0000 7fc34f006700 1 -- 172.21.15.162:0/17290197 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc34804c710 con 0x7fc348066ed0 2024-04-27T04:09:45.038 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:09:45.038+0000 7fc3457fa700 1 -- 172.21.15.162:0/17290197 <== mon.1 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 20) v1 ==== 89550+0+0 (secure 0 0 0) 0x7fc33c01c650 con 0x7fc348066ed0 2024-04-27T04:09:45.038 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:09:45.039+0000 7fc3457fa700 1 --2- 172.21.15.162:0/17290197 >> [v2:172.21.15.162:6800/4079782599,v1:172.21.15.162:6801/4079782599] conn(0x7fc33806bb70 0x7fc33806e020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:09:45.038 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:09:45.039+0000 7fc3457fa700 1 -- 172.21.15.162:0/17290197 <== mon.1 v2:172.21.15.181:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5550+0+0 (secure 0 0 0) 0x7fc33c08bfe0 con 0x7fc348066ed0 2024-04-27T04:09:45.038 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:09:45.039+0000 7fc347fff700 1 --2- 172.21.15.162:0/17290197 >> [v2:172.21.15.162:6800/4079782599,v1:172.21.15.162:6801/4079782599] conn(0x7fc33806bb70 0x7fc33806e020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:09:45.038 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:09:45.039+0000 7fc347fff700 1 --2- 172.21.15.162:0/17290197 >> [v2:172.21.15.162:6800/4079782599,v1:172.21.15.162:6801/4079782599] conn(0x7fc33806bb70 0x7fc33806e020 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fc3340041d0 tx=0x7fc33400b040 comp rx=0 tx=0).ready entity=mgr.14210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:09:45.046 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:09:45.048+0000 7fc3457fa700 1 -- 172.21.15.162:0/17290197 <== mon.1 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7fc33c05bc70 con 0x7fc348066ed0 2024-04-27T04:09:45.258 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:09:45.259+0000 7fc34f006700 1 -- 172.21.15.162:0/17290197 --> [v2:172.21.15.162:6800/4079782599,v1:172.21.15.162:6801/4079782599] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fc3480f8f20 con 0x7fc33806bb70 2024-04-27T04:09:45.259 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:09:45.261+0000 7fc3457fa700 1 -- 172.21.15.162:0/17290197 <== mgr.14210 v2:172.21.15.162:6800/4079782599 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+321 (secure 0 0 0) 0x7fc3480f8f20 con 0x7fc33806bb70 2024-04-27T04:09:45.262 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:09:45.263+0000 7fc34f006700 1 -- 172.21.15.162:0/17290197 >> [v2:172.21.15.162:6800/4079782599,v1:172.21.15.162:6801/4079782599] conn(0x7fc33806bb70 msgr2=0x7fc33806e020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:09:45.262 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:09:45.263+0000 7fc34f006700 1 --2- 172.21.15.162:0/17290197 >> [v2:172.21.15.162:6800/4079782599,v1:172.21.15.162:6801/4079782599] conn(0x7fc33806bb70 0x7fc33806e020 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fc3340041d0 tx=0x7fc33400b040 comp rx=0 tx=0).stop 2024-04-27T04:09:45.262 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:09:45.263+0000 7fc34f006700 1 -- 172.21.15.162:0/17290197 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc348066ed0 msgr2=0x7fc3480fd1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:09:45.262 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:09:45.263+0000 7fc34f006700 1 --2- 172.21.15.162:0/17290197 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc348066ed0 0x7fc3480fd1a0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fc33c000c00 tx=0x7fc33c009e30 comp rx=0 tx=0).stop 2024-04-27T04:09:45.262 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:09:45.263+0000 7fc34f006700 1 -- 172.21.15.162:0/17290197 shutdown_connections 2024-04-27T04:09:45.262 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:09:45.263+0000 7fc34f006700 1 --2- 172.21.15.162:0/17290197 >> [v2:172.21.15.162:6800/4079782599,v1:172.21.15.162:6801/4079782599] conn(0x7fc33806bb70 0x7fc33806e020 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:09:45.262 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:09:45.263+0000 7fc34f006700 1 --2- 172.21.15.162:0/17290197 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc348066ed0 0x7fc3480fd1a0 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:09:45.262 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:09:45.263+0000 7fc34f006700 1 --2- 172.21.15.162:0/17290197 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc3480ffee0 0x7fc3480fb8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:09:45.262 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:09:45.263+0000 7fc34f006700 1 -- 172.21.15.162:0/17290197 >> 172.21.15.162:0/17290197 conn(0x7fc3480f7430 msgr2=0x7fc3480f7800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:09:45.263 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:09:45.263+0000 7fc34f006700 1 -- 172.21.15.162:0/17290197 shutdown_connections 2024-04-27T04:09:45.263 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:09:45.263+0000 7fc34f006700 1 -- 172.21.15.162:0/17290197 wait complete. 2024-04-27T04:09:45.272 INFO:teuthology.orchestra.run.smithi162.stdout:true 2024-04-27T04:09:45.374 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:09:45.375+0000 7fd3f92b7700 1 -- 172.21.15.162:0/3034700750 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd3f40fefd0 msgr2=0x7fd3f40ff3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:09:45.374 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:09:45.375+0000 7fd3f92b7700 1 --2- 172.21.15.162:0/3034700750 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd3f40fefd0 0x7fd3f40ff3e0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fd3e4004940 tx=0x7fd3e4009e30 comp rx=0 tx=0).stop 2024-04-27T04:09:45.374 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:09:45.375+0000 7fd3f92b7700 1 -- 172.21.15.162:0/3034700750 shutdown_connections 2024-04-27T04:09:45.374 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:09:45.375+0000 7fd3f92b7700 1 --2- 172.21.15.162:0/3034700750 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd3f4100280 0x7fd3f41006f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:09:45.374 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:09:45.375+0000 7fd3f92b7700 1 --2- 172.21.15.162:0/3034700750 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd3f40fefd0 0x7fd3f40ff3e0 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:09:45.374 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:09:45.375+0000 7fd3f92b7700 1 -- 172.21.15.162:0/3034700750 >> 172.21.15.162:0/3034700750 conn(0x7fd3f40fa8f0 msgr2=0x7fd3f40fccd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:09:45.374 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:09:45.375+0000 7fd3f92b7700 1 -- 172.21.15.162:0/3034700750 shutdown_connections 2024-04-27T04:09:45.374 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:09:45.375+0000 7fd3f92b7700 1 -- 172.21.15.162:0/3034700750 wait complete. 2024-04-27T04:09:45.376 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:09:45.376+0000 7fd3f92b7700 1 Processor -- start 2024-04-27T04:09:45.376 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:09:45.376+0000 7fd3f92b7700 1 -- start start 2024-04-27T04:09:45.376 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:09:45.376+0000 7fd3f92b7700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd3f40fefd0 0x7fd3f410fba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:09:45.376 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:09:45.376+0000 7fd3f92b7700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd3f4100280 0x7fd3f41100e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:09:45.376 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:09:45.376+0000 7fd3f92b7700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3f41106e0 con 0x7fd3f40fefd0 2024-04-27T04:09:45.376 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:09:45.376+0000 7fd3f92b7700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3f4110f30 con 0x7fd3f4100280 2024-04-27T04:09:45.377 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:09:45.376+0000 7fd3f2ffd700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd3f40fefd0 0x7fd3f410fba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:09:45.377 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:09:45.376+0000 7fd3f2ffd700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd3f40fefd0 0x7fd3f410fba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:60150/0 (socket says 172.21.15.162:60150) 2024-04-27T04:09:45.377 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:09:45.376+0000 7fd3f27fc700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd3f4100280 0x7fd3f41100e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:09:45.377 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:09:45.376+0000 7fd3f2ffd700 1 -- 172.21.15.162:0/3559669843 learned_addr learned my addr 172.21.15.162:0/3559669843 (peer_addr_for_me v2:172.21.15.162:0/0) 2024-04-27T04:09:45.377 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:09:45.377+0000 7fd3f2ffd700 1 -- 172.21.15.162:0/3559669843 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd3f4100280 msgr2=0x7fd3f41100e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:09:45.377 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:09:45.377+0000 7fd3f2ffd700 1 --2- 172.21.15.162:0/3559669843 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd3f4100280 0x7fd3f41100e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:09:45.377 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:09:45.377+0000 7fd3f2ffd700 1 -- 172.21.15.162:0/3559669843 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd3e4005040 con 0x7fd3f40fefd0 2024-04-27T04:09:45.377 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:09:45.377+0000 7fd3f27fc700 1 --2- 172.21.15.162:0/3559669843 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd3f4100280 0x7fd3f41100e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-27T04:09:45.377 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:09:45.377+0000 7fd3f2ffd700 1 --2- 172.21.15.162:0/3559669843 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd3f40fefd0 0x7fd3f410fba0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7fd3e400b3b0 tx=0x7fd3e4009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:09:45.377 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:09:45.377+0000 7fd3db7fe700 1 -- 172.21.15.162:0/3559669843 <== mon.0 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd3e400e040 con 0x7fd3f40fefd0 2024-04-27T04:09:45.377 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:09:45.377+0000 7fd3db7fe700 1 -- 172.21.15.162:0/3559669843 <== mon.0 v2:172.21.15.162:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fd3e400bbf0 con 0x7fd3f40fefd0 2024-04-27T04:09:45.377 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:09:45.377+0000 7fd3f92b7700 1 -- 172.21.15.162:0/3559669843 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd3f41111b0 con 0x7fd3f40fefd0 2024-04-27T04:09:45.377 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:09:45.377+0000 7fd3db7fe700 1 -- 172.21.15.162:0/3559669843 <== mon.0 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd3e401c630 con 0x7fd3f40fefd0 2024-04-27T04:09:45.377 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:09:45.377+0000 7fd3f92b7700 1 -- 172.21.15.162:0/3559669843 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd3f41116f0 con 0x7fd3f40fefd0 2024-04-27T04:09:45.378 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:09:45.378+0000 7fd3f92b7700 1 -- 172.21.15.162:0/3559669843 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd3f405f5c0 con 0x7fd3f40fefd0 2024-04-27T04:09:45.378 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:09:45.379+0000 7fd3db7fe700 1 -- 172.21.15.162:0/3559669843 <== mon.0 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 20) v1 ==== 89550+0+0 (secure 0 0 0) 0x7fd3e400bd60 con 0x7fd3f40fefd0 2024-04-27T04:09:45.378 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:09:45.379+0000 7fd3db7fe700 1 --2- 172.21.15.162:0/3559669843 >> [v2:172.21.15.162:6800/4079782599,v1:172.21.15.162:6801/4079782599] conn(0x7fd3dc06bbd0 0x7fd3dc06e080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:09:45.378 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:09:45.380+0000 7fd3db7fe700 1 -- 172.21.15.162:0/3559669843 <== mon.0 v2:172.21.15.162:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5550+0+0 (secure 0 0 0) 0x7fd3e408b240 con 0x7fd3f40fefd0 2024-04-27T04:09:45.379 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:09:45.380+0000 7fd3f27fc700 1 --2- 172.21.15.162:0/3559669843 >> [v2:172.21.15.162:6800/4079782599,v1:172.21.15.162:6801/4079782599] conn(0x7fd3dc06bbd0 0x7fd3dc06e080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:09:45.379 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:09:45.380+0000 7fd3f27fc700 1 --2- 172.21.15.162:0/3559669843 >> [v2:172.21.15.162:6800/4079782599,v1:172.21.15.162:6801/4079782599] conn(0x7fd3dc06bbd0 0x7fd3dc06e080 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fd3e8009fd0 tx=0x7fd3e800b040 comp rx=0 tx=0).ready entity=mgr.14210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:09:45.383 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:09:45.384+0000 7fd3db7fe700 1 -- 172.21.15.162:0/3559669843 <== mon.0 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7fd3e405b050 con 0x7fd3f40fefd0 2024-04-27T04:09:45.634 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:09:45.636+0000 7fd3f92b7700 1 -- 172.21.15.162:0/3559669843 --> [v2:172.21.15.162:6800/4079782599,v1:172.21.15.162:6801/4079782599] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fd3f4111ac0 con 0x7fd3dc06bbd0 2024-04-27T04:09:45.635 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:09:45.637+0000 7fd3db7fe700 1 -- 172.21.15.162:0/3559669843 <== mgr.14210 v2:172.21.15.162:6800/4079782599 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+321 (secure 0 0 0) 0x7fd3f4111ac0 con 0x7fd3dc06bbd0 2024-04-27T04:09:45.638 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:09:45.639+0000 7fd3f92b7700 1 -- 172.21.15.162:0/3559669843 >> [v2:172.21.15.162:6800/4079782599,v1:172.21.15.162:6801/4079782599] conn(0x7fd3dc06bbd0 msgr2=0x7fd3dc06e080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:09:45.638 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:09:45.639+0000 7fd3f92b7700 1 --2- 172.21.15.162:0/3559669843 >> [v2:172.21.15.162:6800/4079782599,v1:172.21.15.162:6801/4079782599] conn(0x7fd3dc06bbd0 0x7fd3dc06e080 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fd3e8009fd0 tx=0x7fd3e800b040 comp rx=0 tx=0).stop 2024-04-27T04:09:45.638 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:09:45.639+0000 7fd3f92b7700 1 -- 172.21.15.162:0/3559669843 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd3f40fefd0 msgr2=0x7fd3f410fba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:09:45.638 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:09:45.639+0000 7fd3f92b7700 1 --2- 172.21.15.162:0/3559669843 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd3f40fefd0 0x7fd3f410fba0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7fd3e400b3b0 tx=0x7fd3e4009e30 comp rx=0 tx=0).stop 2024-04-27T04:09:45.639 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:09:45.639+0000 7fd3f92b7700 1 -- 172.21.15.162:0/3559669843 shutdown_connections 2024-04-27T04:09:45.639 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:09:45.639+0000 7fd3f92b7700 1 --2- 172.21.15.162:0/3559669843 >> [v2:172.21.15.162:6800/4079782599,v1:172.21.15.162:6801/4079782599] conn(0x7fd3dc06bbd0 0x7fd3dc06e080 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:09:45.639 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:09:45.639+0000 7fd3f92b7700 1 --2- 172.21.15.162:0/3559669843 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd3f40fefd0 0x7fd3f410fba0 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:09:45.639 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:09:45.639+0000 7fd3f92b7700 1 --2- 172.21.15.162:0/3559669843 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd3f4100280 0x7fd3f41100e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:09:45.639 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:09:45.640+0000 7fd3f92b7700 1 -- 172.21.15.162:0/3559669843 >> 172.21.15.162:0/3559669843 conn(0x7fd3f40fa8f0 msgr2=0x7fd3f40fcef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:09:45.639 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:09:45.640+0000 7fd3f92b7700 1 -- 172.21.15.162:0/3559669843 shutdown_connections 2024-04-27T04:09:45.639 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:09:45.640+0000 7fd3f92b7700 1 -- 172.21.15.162:0/3559669843 wait complete. 2024-04-27T04:09:45.754 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:09:45.753+0000 7fd42651b700 1 -- 172.21.15.162:0/2825530753 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd4200fe4a0 msgr2=0x7fd4200fe8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:09:45.754 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:09:45.753+0000 7fd42651b700 1 --2- 172.21.15.162:0/2825530753 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd4200fe4a0 0x7fd4200fe8b0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7fd410004940 tx=0x7fd410009e30 comp rx=0 tx=0).stop 2024-04-27T04:09:45.754 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:09:45.753+0000 7fd42651b700 1 -- 172.21.15.162:0/2825530753 shutdown_connections 2024-04-27T04:09:45.754 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:09:45.753+0000 7fd42651b700 1 --2- 172.21.15.162:0/2825530753 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd4200fedf0 0x7fd4200faf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:09:45.754 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:09:45.753+0000 7fd42651b700 1 --2- 172.21.15.162:0/2825530753 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd4200fe4a0 0x7fd4200fe8b0 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:09:45.754 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:09:45.753+0000 7fd42651b700 1 -- 172.21.15.162:0/2825530753 >> 172.21.15.162:0/2825530753 conn(0x7fd4200f6b80 msgr2=0x7fd4200f8fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:09:45.754 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:09:45.753+0000 7fd42651b700 1 -- 172.21.15.162:0/2825530753 shutdown_connections 2024-04-27T04:09:45.754 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:09:45.753+0000 7fd42651b700 1 -- 172.21.15.162:0/2825530753 wait complete. 2024-04-27T04:09:45.754 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:09:45.754+0000 7fd42651b700 1 Processor -- start 2024-04-27T04:09:45.755 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:09:45.754+0000 7fd42651b700 1 -- start start 2024-04-27T04:09:45.755 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:09:45.754+0000 7fd42651b700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd4200fe4a0 0x7fd42010d980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:09:45.755 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:09:45.754+0000 7fd42651b700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd4200fedf0 0x7fd42010dec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:09:45.755 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:09:45.754+0000 7fd42651b700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd42010e4c0 con 0x7fd4200fedf0 2024-04-27T04:09:45.755 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:09:45.754+0000 7fd42651b700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd42010ed10 con 0x7fd4200fe4a0 2024-04-27T04:09:45.755 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:09:45.755+0000 7fd41f7fe700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd4200fedf0 0x7fd42010dec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:09:45.755 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:09:45.755+0000 7fd41ffff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd4200fe4a0 0x7fd42010d980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:09:45.755 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:09:45.755+0000 7fd41f7fe700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd4200fedf0 0x7fd42010dec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:60158/0 (socket says 172.21.15.162:60158) 2024-04-27T04:09:45.755 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:09:45.755+0000 7fd41f7fe700 1 -- 172.21.15.162:0/3036682200 learned_addr learned my addr 172.21.15.162:0/3036682200 (peer_addr_for_me v2:172.21.15.162:0/0) 2024-04-27T04:09:45.755 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:09:45.755+0000 7fd41f7fe700 1 -- 172.21.15.162:0/3036682200 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd4200fe4a0 msgr2=0x7fd42010d980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:09:45.755 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:09:45.755+0000 7fd41f7fe700 1 --2- 172.21.15.162:0/3036682200 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd4200fe4a0 0x7fd42010d980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:09:45.755 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:09:45.755+0000 7fd41f7fe700 1 -- 172.21.15.162:0/3036682200 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd410005040 con 0x7fd4200fedf0 2024-04-27T04:09:45.755 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:09:45.755+0000 7fd41f7fe700 1 --2- 172.21.15.162:0/3036682200 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd4200fedf0 0x7fd42010dec0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7fd41400eb40 tx=0x7fd41400b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:09:45.755 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:09:45.755+0000 7fd41cff9700 1 -- 172.21.15.162:0/3036682200 <== mon.0 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd414007cc0 con 0x7fd4200fedf0 2024-04-27T04:09:45.756 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:09:45.755+0000 7fd42651b700 1 -- 172.21.15.162:0/3036682200 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd42010ef90 con 0x7fd4200fedf0 2024-04-27T04:09:45.756 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:09:45.755+0000 7fd41cff9700 1 -- 172.21.15.162:0/3036682200 <== mon.0 v2:172.21.15.162:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fd41400f040 con 0x7fd4200fedf0 2024-04-27T04:09:45.756 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:09:45.755+0000 7fd41cff9700 1 -- 172.21.15.162:0/3036682200 <== mon.0 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd41400ca10 con 0x7fd4200fedf0 2024-04-27T04:09:45.756 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:09:45.755+0000 7fd42651b700 1 -- 172.21.15.162:0/3036682200 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd42010f4d0 con 0x7fd4200fedf0 2024-04-27T04:09:45.757 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:09:45.756+0000 7fd42651b700 1 -- 172.21.15.162:0/3036682200 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd42005f5c0 con 0x7fd4200fedf0 2024-04-27T04:09:45.757 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:09:45.757+0000 7fd41cff9700 1 -- 172.21.15.162:0/3036682200 <== mon.0 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 20) v1 ==== 89550+0+0 (secure 0 0 0) 0x7fd41400cc80 con 0x7fd4200fedf0 2024-04-27T04:09:45.757 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:09:45.757+0000 7fd41cff9700 1 --2- 172.21.15.162:0/3036682200 >> [v2:172.21.15.162:6800/4079782599,v1:172.21.15.162:6801/4079782599] conn(0x7fd40806bbd0 0x7fd40806e080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:09:45.757 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:09:45.758+0000 7fd41cff9700 1 -- 172.21.15.162:0/3036682200 <== mon.0 v2:172.21.15.162:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5550+0+0 (secure 0 0 0) 0x7fd41408a310 con 0x7fd4200fedf0 2024-04-27T04:09:45.757 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:09:45.758+0000 7fd41ffff700 1 --2- 172.21.15.162:0/3036682200 >> [v2:172.21.15.162:6800/4079782599,v1:172.21.15.162:6801/4079782599] conn(0x7fd40806bbd0 0x7fd40806e080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:09:45.757 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:09:45.758+0000 7fd41ffff700 1 --2- 172.21.15.162:0/3036682200 >> [v2:172.21.15.162:6800/4079782599,v1:172.21.15.162:6801/4079782599] conn(0x7fd40806bbd0 0x7fd40806e080 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fd410009b30 tx=0x7fd410009e30 comp rx=0 tx=0).ready entity=mgr.14210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:09:45.761 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:09:45.763+0000 7fd41cff9700 1 -- 172.21.15.162:0/3036682200 <== mon.0 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7fd41405a0b0 con 0x7fd4200fedf0 2024-04-27T04:09:46.023 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:09:46.024+0000 7fd42651b700 1 -- 172.21.15.162:0/3036682200 --> [v2:172.21.15.162:6800/4079782599,v1:172.21.15.162:6801/4079782599] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7fd42010f7b0 con 0x7fd40806bbd0 2024-04-27T04:09:46.035 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:09:46.036+0000 7fd41cff9700 1 -- 172.21.15.162:0/3036682200 <== mgr.14210 v2:172.21.15.162:6800/4079782599 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3344 (secure 0 0 0) 0x7fd42010f7b0 con 0x7fd40806bbd0 2024-04-27T04:09:46.035 INFO:teuthology.orchestra.run.smithi162.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-27T04:09:46.035 INFO:teuthology.orchestra.run.smithi162.stdout:alertmanager.smithi162 smithi162 *:9093,9094 running (6m) 2m ago 8m 24.6M - 0.25.0 c8568f914cd2 61972851fcce 2024-04-27T04:09:46.035 INFO:teuthology.orchestra.run.smithi162.stdout:crash.smithi162 smithi162 running (8m) 2m ago 8m 6660k - 17.2.7-907-g2be3efa9 1c5834633715 228c98df4289 2024-04-27T04:09:46.035 INFO:teuthology.orchestra.run.smithi162.stdout:crash.smithi181 smithi181 running (7m) 2m ago 7m 6665k - 17.2.7-907-g2be3efa9 1c5834633715 c8e553e3df2b 2024-04-27T04:09:46.035 INFO:teuthology.orchestra.run.smithi162.stdout:grafana.smithi162 smithi162 *:3000 running (6m) 2m ago 7m 84.2M - 9.4.7 954c08fa6188 7d3364d5cddc 2024-04-27T04:09:46.035 INFO:teuthology.orchestra.run.smithi162.stdout:mds.cephfs.smithi162.gzlyjj smithi162 running (2m) 2m ago 2m 10.4M - 17.2.7-907-g2be3efa9 1c5834633715 27e5b030dd32 2024-04-27T04:09:46.035 INFO:teuthology.orchestra.run.smithi162.stdout:mds.cephfs.smithi162.xcnapk smithi162 running (2m) 2m ago 2m 13.2M - 17.2.7-907-g2be3efa9 1c5834633715 38f9030df029 2024-04-27T04:09:46.035 INFO:teuthology.orchestra.run.smithi162.stdout:mds.cephfs.smithi181.bkkcdm smithi181 running (2m) 2m ago 2m 12.1M - 17.2.7-907-g2be3efa9 1c5834633715 af55afca1263 2024-04-27T04:09:46.036 INFO:teuthology.orchestra.run.smithi162.stdout:mds.cephfs.smithi181.dzomgh smithi181 running (2m) 2m ago 2m 15.9M - 17.2.7-907-g2be3efa9 1c5834633715 6733296597f3 2024-04-27T04:09:46.036 INFO:teuthology.orchestra.run.smithi162.stdout:mgr.smithi162.oennfn smithi162 *:9283 running (9m) 2m ago 9m 479M - 17.2.7-907-g2be3efa9 1c5834633715 e768e27ad56e 2024-04-27T04:09:46.036 INFO:teuthology.orchestra.run.smithi162.stdout:mgr.smithi181.vtwuhz smithi181 *:8443,9283 starting - - - - 2024-04-27T04:09:46.036 INFO:teuthology.orchestra.run.smithi162.stdout:mon.smithi162 smithi162 running (9m) 2m ago 9m 53.9M 2048M 17.2.7-907-g2be3efa9 1c5834633715 0f1e3682b9ab 2024-04-27T04:09:46.036 INFO:teuthology.orchestra.run.smithi162.stdout:mon.smithi181 smithi181 running (7m) 2m ago 7m 44.5M 2048M 17.2.7-907-g2be3efa9 1c5834633715 96e48aa3912f 2024-04-27T04:09:46.036 INFO:teuthology.orchestra.run.smithi162.stdout:node-exporter.smithi162 smithi162 *:9100 running (8m) 2m ago 8m 15.7M - 1.5.0 0da6a335fe13 e4535d340cea 2024-04-27T04:09:46.036 INFO:teuthology.orchestra.run.smithi162.stdout:node-exporter.smithi181 smithi181 *:9100 running (7m) 2m ago 7m 15.2M - 1.5.0 0da6a335fe13 3cc278142657 2024-04-27T04:09:46.036 INFO:teuthology.orchestra.run.smithi162.stdout:osd.0 smithi162 running (6m) 2m ago 6m 48.2M 4288M 17.2.7-907-g2be3efa9 1c5834633715 4a9df3aa938d 2024-04-27T04:09:46.036 INFO:teuthology.orchestra.run.smithi162.stdout:osd.1 smithi162 running (5m) 2m ago 5m 48.4M 4288M 17.2.7-907-g2be3efa9 1c5834633715 aefb464c46e8 2024-04-27T04:09:46.036 INFO:teuthology.orchestra.run.smithi162.stdout:osd.2 smithi162 running (5m) 2m ago 5m 44.5M 4288M 17.2.7-907-g2be3efa9 1c5834633715 70f16d86d38e 2024-04-27T04:09:46.037 INFO:teuthology.orchestra.run.smithi162.stdout:osd.3 smithi181 running (5m) 2m ago 5m 44.5M 5312M 17.2.7-907-g2be3efa9 1c5834633715 030d829be54e 2024-04-27T04:09:46.037 INFO:teuthology.orchestra.run.smithi162.stdout:osd.4 smithi181 running (4m) 2m ago 4m 41.6M 5312M 17.2.7-907-g2be3efa9 1c5834633715 512afbee7cf7 2024-04-27T04:09:46.037 INFO:teuthology.orchestra.run.smithi162.stdout:osd.5 smithi181 running (4m) 2m ago 4m 40.9M 5312M 17.2.7-907-g2be3efa9 1c5834633715 f47a1daac1ea 2024-04-27T04:09:46.037 INFO:teuthology.orchestra.run.smithi162.stdout:prometheus.smithi162 smithi162 *:9095 running (6m) 2m ago 7m 54.6M - 2.43.0 a07b618ecd1d 63755cc04fec 2024-04-27T04:09:46.037 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:09:46.038+0000 7fd42651b700 1 -- 172.21.15.162:0/3036682200 >> [v2:172.21.15.162:6800/4079782599,v1:172.21.15.162:6801/4079782599] conn(0x7fd40806bbd0 msgr2=0x7fd40806e080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:09:46.038 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:09:46.038+0000 7fd42651b700 1 --2- 172.21.15.162:0/3036682200 >> [v2:172.21.15.162:6800/4079782599,v1:172.21.15.162:6801/4079782599] conn(0x7fd40806bbd0 0x7fd40806e080 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fd410009b30 tx=0x7fd410009e30 comp rx=0 tx=0).stop 2024-04-27T04:09:46.038 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:09:46.038+0000 7fd42651b700 1 -- 172.21.15.162:0/3036682200 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd4200fedf0 msgr2=0x7fd42010dec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:09:46.038 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:09:46.038+0000 7fd42651b700 1 --2- 172.21.15.162:0/3036682200 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd4200fedf0 0x7fd42010dec0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7fd41400eb40 tx=0x7fd41400b040 comp rx=0 tx=0).stop 2024-04-27T04:09:46.038 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:09:46.039+0000 7fd42651b700 1 -- 172.21.15.162:0/3036682200 shutdown_connections 2024-04-27T04:09:46.038 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:09:46.039+0000 7fd42651b700 1 --2- 172.21.15.162:0/3036682200 >> [v2:172.21.15.162:6800/4079782599,v1:172.21.15.162:6801/4079782599] conn(0x7fd40806bbd0 0x7fd40806e080 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:09:46.038 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:09:46.039+0000 7fd42651b700 1 --2- 172.21.15.162:0/3036682200 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd4200fe4a0 0x7fd42010d980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:09:46.038 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:09:46.039+0000 7fd42651b700 1 --2- 172.21.15.162:0/3036682200 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd4200fedf0 0x7fd42010dec0 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:09:46.038 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:09:46.039+0000 7fd42651b700 1 -- 172.21.15.162:0/3036682200 >> 172.21.15.162:0/3036682200 conn(0x7fd4200f6b80 msgr2=0x7fd4200f81c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:09:46.038 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:09:46.039+0000 7fd42651b700 1 -- 172.21.15.162:0/3036682200 shutdown_connections 2024-04-27T04:09:46.038 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:09:46.039+0000 7fd42651b700 1 -- 172.21.15.162:0/3036682200 wait complete. 2024-04-27T04:09:46.168 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:09:46.169+0000 7fc4e3492700 1 -- 172.21.15.162:0/3899964850 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc4dc0ffee0 msgr2=0x7fc4dc102300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:09:46.168 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:09:46.169+0000 7fc4e3492700 1 --2- 172.21.15.162:0/3899964850 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc4dc0ffee0 0x7fc4dc102300 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7fc4cc004940 tx=0x7fc4cc009e30 comp rx=0 tx=0).stop 2024-04-27T04:09:46.168 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:09:46.169+0000 7fc4e3492700 1 -- 172.21.15.162:0/3899964850 shutdown_connections 2024-04-27T04:09:46.168 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:09:46.169+0000 7fc4e3492700 1 --2- 172.21.15.162:0/3899964850 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc4dc0ffee0 0x7fc4dc102300 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:09:46.168 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:09:46.169+0000 7fc4e3492700 1 --2- 172.21.15.162:0/3899964850 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc4dc0662c0 0x7fc4dc0ff9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:09:46.168 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:09:46.169+0000 7fc4e3492700 1 -- 172.21.15.162:0/3899964850 >> 172.21.15.162:0/3899964850 conn(0x7fc4dc0f7430 msgr2=0x7fc4dc0f9840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:09:46.168 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:09:46.169+0000 7fc4e3492700 1 -- 172.21.15.162:0/3899964850 shutdown_connections 2024-04-27T04:09:46.169 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:09:46.169+0000 7fc4e3492700 1 -- 172.21.15.162:0/3899964850 wait complete. 2024-04-27T04:09:46.170 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:09:46.170+0000 7fc4e3492700 1 Processor -- start 2024-04-27T04:09:46.170 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:09:46.170+0000 7fc4e3492700 1 -- start start 2024-04-27T04:09:46.171 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:09:46.170+0000 7fc4e3492700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc4dc0662c0 0x7fc4dc0fd330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:09:46.171 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:09:46.170+0000 7fc4e3492700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc4dc0ffee0 0x7fc4dc0fb8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:09:46.171 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:09:46.170+0000 7fc4e3492700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4dc0fbde0 con 0x7fc4dc0ffee0 2024-04-27T04:09:46.171 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:09:46.170+0000 7fc4e3492700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4dc0fbf20 con 0x7fc4dc0662c0 2024-04-27T04:09:46.171 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:09:46.170+0000 7fc4e0a2d700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc4dc0ffee0 0x7fc4dc0fb8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:09:46.171 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:09:46.170+0000 7fc4e0a2d700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc4dc0ffee0 0x7fc4dc0fb8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:60188/0 (socket says 172.21.15.162:60188) 2024-04-27T04:09:46.171 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:09:46.170+0000 7fc4e0a2d700 1 -- 172.21.15.162:0/4270726186 learned_addr learned my addr 172.21.15.162:0/4270726186 (peer_addr_for_me v2:172.21.15.162:0/0) 2024-04-27T04:09:46.171 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:09:46.170+0000 7fc4e122e700 1 --2- 172.21.15.162:0/4270726186 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc4dc0662c0 0x7fc4dc0fd330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:09:46.171 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:09:46.170+0000 7fc4e0a2d700 1 -- 172.21.15.162:0/4270726186 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc4dc0662c0 msgr2=0x7fc4dc0fd330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:09:46.171 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:09:46.170+0000 7fc4e0a2d700 1 --2- 172.21.15.162:0/4270726186 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc4dc0662c0 0x7fc4dc0fd330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:09:46.172 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:09:46.170+0000 7fc4e0a2d700 1 -- 172.21.15.162:0/4270726186 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc4cc005040 con 0x7fc4dc0ffee0 2024-04-27T04:09:46.172 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:09:46.171+0000 7fc4e0a2d700 1 --2- 172.21.15.162:0/4270726186 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc4dc0ffee0 0x7fc4dc0fb8a0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7fc4cc013870 tx=0x7fc4cc009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:09:46.172 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:09:46.171+0000 7fc4d1ffb700 1 -- 172.21.15.162:0/4270726186 <== mon.0 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc4cc021070 con 0x7fc4dc0ffee0 2024-04-27T04:09:46.172 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:09:46.171+0000 7fc4d1ffb700 1 -- 172.21.15.162:0/4270726186 <== mon.0 v2:172.21.15.162:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fc4cc009430 con 0x7fc4dc0ffee0 2024-04-27T04:09:46.172 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:09:46.171+0000 7fc4e3492700 1 -- 172.21.15.162:0/4270726186 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc4dc0fc1a0 con 0x7fc4dc0ffee0 2024-04-27T04:09:46.172 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:09:46.171+0000 7fc4d1ffb700 1 -- 172.21.15.162:0/4270726186 <== mon.0 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc4cc00e070 con 0x7fc4dc0ffee0 2024-04-27T04:09:46.172 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:09:46.171+0000 7fc4e3492700 1 -- 172.21.15.162:0/4270726186 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc4dc0fc6e0 con 0x7fc4dc0ffee0 2024-04-27T04:09:46.173 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:09:46.172+0000 7fc4e3492700 1 -- 172.21.15.162:0/4270726186 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc4dc05f5c0 con 0x7fc4dc0ffee0 2024-04-27T04:09:46.173 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:09:46.173+0000 7fc4d1ffb700 1 -- 172.21.15.162:0/4270726186 <== mon.0 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 20) v1 ==== 89550+0+0 (secure 0 0 0) 0x7fc4cc01c750 con 0x7fc4dc0ffee0 2024-04-27T04:09:46.173 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:09:46.173+0000 7fc4d1ffb700 1 --2- 172.21.15.162:0/4270726186 >> [v2:172.21.15.162:6800/4079782599,v1:172.21.15.162:6801/4079782599] conn(0x7fc4c806bb60 0x7fc4c806e010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:09:46.173 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:09:46.173+0000 7fc4d1ffb700 1 -- 172.21.15.162:0/4270726186 <== mon.0 v2:172.21.15.162:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5550+0+0 (secure 0 0 0) 0x7fc4cc08b890 con 0x7fc4dc0ffee0 2024-04-27T04:09:46.173 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:09:46.173+0000 7fc4e122e700 1 --2- 172.21.15.162:0/4270726186 >> [v2:172.21.15.162:6800/4079782599,v1:172.21.15.162:6801/4079782599] conn(0x7fc4c806bb60 0x7fc4c806e010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:09:46.173 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:09:46.174+0000 7fc4e122e700 1 --2- 172.21.15.162:0/4270726186 >> [v2:172.21.15.162:6800/4079782599,v1:172.21.15.162:6801/4079782599] conn(0x7fc4c806bb60 0x7fc4c806e010 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fc4dc0fd0c0 tx=0x7fc4d800b040 comp rx=0 tx=0).ready entity=mgr.14210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:09:46.177 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:09:46.178+0000 7fc4d1ffb700 1 -- 172.21.15.162:0/4270726186 <== mon.0 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7fc4cc057b70 con 0x7fc4dc0ffee0 2024-04-27T04:09:46.277 INFO:tasks.workunit.client.1.smithi181.stdout:6/7: sync 2024-04-27T04:09:46.332 INFO:tasks.workunit.client.1.smithi181.stdout:2/57: sync 2024-04-27T04:09:46.332 INFO:tasks.workunit.client.1.smithi181.stdout:4/16: sync 2024-04-27T04:09:46.332 INFO:tasks.workunit.client.1.smithi181.stdout:7/24: sync 2024-04-27T04:09:46.333 INFO:tasks.workunit.client.1.smithi181.stdout:2/58: readlink d4/l5 0 2024-04-27T04:09:46.333 INFO:tasks.workunit.client.1.smithi181.stdout:4/17: creat d0/d1/f4 x:0 0 0 2024-04-27T04:09:46.334 INFO:tasks.workunit.client.1.smithi181.stdout:4/18: read - d0/d1/f4 zero size 2024-04-27T04:09:46.339 INFO:tasks.workunit.client.1.smithi181.stdout:4/19: mkdir d0/d5 0 2024-04-27T04:09:46.340 INFO:tasks.workunit.client.1.smithi181.stdout:7/25: link d1/d2/l3 d1/l5 0 2024-04-27T04:09:46.341 INFO:tasks.workunit.client.1.smithi181.stdout:4/20: truncate d0/d1/f4 58999 0 2024-04-27T04:09:46.341 INFO:tasks.workunit.client.1.smithi181.stdout:4/21: stat d0/d1/f2 0 2024-04-27T04:09:46.342 INFO:tasks.workunit.client.1.smithi181.stdout:7/26: readlink d1/l5 0 2024-04-27T04:09:46.342 INFO:tasks.workunit.client.1.smithi181.stdout:4/22: dread - d0/d1/f2 zero size 2024-04-27T04:09:46.343 INFO:tasks.workunit.client.1.smithi181.stdout:4/23: chown d0 4899 1 2024-04-27T04:09:46.387 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:09:46 smithi162 ceph-mon[26841]: pgmap v258: 65 pgs: 65 active+clean; 277 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 1.6 MiB/s rd, 12 MiB/s wr, 313 op/s 2024-04-27T04:09:46.387 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:09:46 smithi162 ceph-mon[26841]: from='client.24371 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-27T04:09:46.514 INFO:tasks.workunit.client.1.smithi181.stdout:4/24: getdents d0 0 2024-04-27T04:09:46.516 INFO:tasks.workunit.client.1.smithi181.stdout:4/25: dread d0/d1/f4 [0,4194304] 0 2024-04-27T04:09:46.518 INFO:tasks.workunit.client.1.smithi181.stdout:2/59: dwrite d4/fb [0,4194304] 0 2024-04-27T04:09:46.519 INFO:tasks.workunit.client.1.smithi181.stdout:4/26: write d0/d1/f4 [32034,103993] 0 2024-04-27T04:09:46.520 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:09:46.521+0000 7fc4e3492700 1 -- 172.21.15.162:0/4270726186 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7fc4dc115be0 con 0x7fc4dc0ffee0 2024-04-27T04:09:46.520 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:09:46.522+0000 7fc4d1ffb700 1 -- 172.21.15.162:0/4270726186 <== mon.0 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+846 (secure 0 0 0) 0x7fc4cc05b190 con 0x7fc4dc0ffee0 2024-04-27T04:09:46.520 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-04-27T04:09:46.521 INFO:teuthology.orchestra.run.smithi162.stdout: "mon": { 2024-04-27T04:09:46.521 INFO:teuthology.orchestra.run.smithi162.stdout: "ceph version 17.2.7-907-g2be3efa9 (2be3efa97a7972f8e925bd899b474329979b74d6) quincy (stable)": 2 2024-04-27T04:09:46.521 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-04-27T04:09:46.521 INFO:teuthology.orchestra.run.smithi162.stdout: "mgr": { 2024-04-27T04:09:46.521 INFO:teuthology.orchestra.run.smithi162.stdout: "ceph version 17.2.7-907-g2be3efa9 (2be3efa97a7972f8e925bd899b474329979b74d6) quincy (stable)": 1, 2024-04-27T04:09:46.521 INFO:teuthology.orchestra.run.smithi162.stdout: "ceph version 19.0.0-3253-g4e213a58 (4e213a58e787437ff508e6c890ccd5fd32dc96b5) squid (dev)": 1 2024-04-27T04:09:46.521 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-04-27T04:09:46.521 INFO:teuthology.orchestra.run.smithi162.stdout: "osd": { 2024-04-27T04:09:46.521 INFO:teuthology.orchestra.run.smithi162.stdout: "ceph version 17.2.7-907-g2be3efa9 (2be3efa97a7972f8e925bd899b474329979b74d6) quincy (stable)": 6 2024-04-27T04:09:46.521 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-04-27T04:09:46.521 INFO:teuthology.orchestra.run.smithi162.stdout: "mds": { 2024-04-27T04:09:46.521 INFO:teuthology.orchestra.run.smithi162.stdout: "ceph version 17.2.7-907-g2be3efa9 (2be3efa97a7972f8e925bd899b474329979b74d6) quincy (stable)": 4 2024-04-27T04:09:46.521 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-04-27T04:09:46.521 INFO:teuthology.orchestra.run.smithi162.stdout: "overall": { 2024-04-27T04:09:46.522 INFO:teuthology.orchestra.run.smithi162.stdout: "ceph version 17.2.7-907-g2be3efa9 (2be3efa97a7972f8e925bd899b474329979b74d6) quincy (stable)": 13, 2024-04-27T04:09:46.522 INFO:teuthology.orchestra.run.smithi162.stdout: "ceph version 19.0.0-3253-g4e213a58 (4e213a58e787437ff508e6c890ccd5fd32dc96b5) squid (dev)": 1 2024-04-27T04:09:46.522 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-04-27T04:09:46.522 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-04-27T04:09:46.522 INFO:tasks.workunit.client.1.smithi181.stdout:4/27: chown d0/d1/f2 3001 1 2024-04-27T04:09:46.523 INFO:tasks.workunit.client.1.smithi181.stdout:2/60: mkdir d4/d9/d16 0 2024-04-27T04:09:46.524 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:09:46.525+0000 7fc4e3492700 1 -- 172.21.15.162:0/4270726186 >> [v2:172.21.15.162:6800/4079782599,v1:172.21.15.162:6801/4079782599] conn(0x7fc4c806bb60 msgr2=0x7fc4c806e010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:09:46.524 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:09:46.525+0000 7fc4e3492700 1 --2- 172.21.15.162:0/4270726186 >> [v2:172.21.15.162:6800/4079782599,v1:172.21.15.162:6801/4079782599] conn(0x7fc4c806bb60 0x7fc4c806e010 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fc4dc0fd0c0 tx=0x7fc4d800b040 comp rx=0 tx=0).stop 2024-04-27T04:09:46.524 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:09:46.525+0000 7fc4e3492700 1 -- 172.21.15.162:0/4270726186 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc4dc0ffee0 msgr2=0x7fc4dc0fb8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:09:46.524 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:09:46.525+0000 7fc4e3492700 1 --2- 172.21.15.162:0/4270726186 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc4dc0ffee0 0x7fc4dc0fb8a0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7fc4cc013870 tx=0x7fc4cc009e30 comp rx=0 tx=0).stop 2024-04-27T04:09:46.524 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:09:46.525+0000 7fc4e3492700 1 -- 172.21.15.162:0/4270726186 shutdown_connections 2024-04-27T04:09:46.524 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:09:46.525+0000 7fc4e3492700 1 --2- 172.21.15.162:0/4270726186 >> [v2:172.21.15.162:6800/4079782599,v1:172.21.15.162:6801/4079782599] conn(0x7fc4c806bb60 0x7fc4c806e010 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:09:46.524 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:09:46.525+0000 7fc4e3492700 1 --2- 172.21.15.162:0/4270726186 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc4dc0662c0 0x7fc4dc0fd330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:09:46.524 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:09:46.525+0000 7fc4e3492700 1 --2- 172.21.15.162:0/4270726186 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc4dc0ffee0 0x7fc4dc0fb8a0 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:09:46.524 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:09:46.525+0000 7fc4e3492700 1 -- 172.21.15.162:0/4270726186 >> 172.21.15.162:0/4270726186 conn(0x7fc4dc0f7430 msgr2=0x7fc4dc0f7800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:09:46.524 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:09:46.525+0000 7fc4e3492700 1 -- 172.21.15.162:0/4270726186 shutdown_connections 2024-04-27T04:09:46.525 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:09:46.525+0000 7fc4e3492700 1 -- 172.21.15.162:0/4270726186 wait complete. 2024-04-27T04:09:46.526 INFO:tasks.workunit.client.1.smithi181.stdout:2/61: mknod d4/d9/df/c17 0 2024-04-27T04:09:46.527 INFO:tasks.workunit.client.1.smithi181.stdout:2/62: write d4/f8 [1307871,62584] 0 2024-04-27T04:09:46.527 INFO:tasks.workunit.client.1.smithi181.stdout:4/28: creat d0/d5/f6 x:0 0 0 2024-04-27T04:09:46.528 INFO:tasks.workunit.client.1.smithi181.stdout:4/29: dread - d0/d5/f6 zero size 2024-04-27T04:09:46.529 INFO:tasks.workunit.client.1.smithi181.stdout:2/63: write d4/d9/df/d14/f15 [882701,68741] 0 2024-04-27T04:09:46.530 INFO:tasks.workunit.client.1.smithi181.stdout:2/64: chown d4/la 46284 1 2024-04-27T04:09:46.531 INFO:tasks.workunit.client.1.smithi181.stdout:3/67: sync 2024-04-27T04:09:46.532 INFO:tasks.workunit.client.1.smithi181.stdout:1/86: sync 2024-04-27T04:09:46.532 INFO:tasks.workunit.client.1.smithi181.stdout:8/70: sync 2024-04-27T04:09:46.532 INFO:tasks.workunit.client.1.smithi181.stdout:5/109: sync 2024-04-27T04:09:46.532 INFO:tasks.workunit.client.1.smithi181.stdout:0/56: sync 2024-04-27T04:09:46.532 INFO:tasks.workunit.client.1.smithi181.stdout:9/61: sync 2024-04-27T04:09:46.532 INFO:tasks.workunit.client.1.smithi181.stdout:4/30: mkdir d0/d5/d7 0 2024-04-27T04:09:46.534 INFO:tasks.workunit.client.1.smithi181.stdout:5/110: fdatasync d4/de/f1e 0 2024-04-27T04:09:46.534 INFO:tasks.workunit.client.1.smithi181.stdout:4/31: stat d0/d1/c3 0 2024-04-27T04:09:46.535 INFO:tasks.workunit.client.1.smithi181.stdout:5/111: stat d4/c7 0 2024-04-27T04:09:46.536 INFO:tasks.workunit.client.1.smithi181.stdout:9/62: write d0/f7 [2194883,74113] 0 2024-04-27T04:09:46.538 INFO:tasks.workunit.client.1.smithi181.stdout:8/71: creat d1/d6/f15 x:0 0 0 2024-04-27T04:09:46.539 INFO:tasks.workunit.client.1.smithi181.stdout:4/32: write d0/d1/f4 [1018344,63346] 0 2024-04-27T04:09:46.540 INFO:tasks.workunit.client.1.smithi181.stdout:0/57: creat dc/f15 x:0 0 0 2024-04-27T04:09:46.545 INFO:tasks.workunit.client.1.smithi181.stdout:2/65: dwrite d4/f8 [0,4194304] 0 2024-04-27T04:09:46.546 INFO:tasks.workunit.client.1.smithi181.stdout:3/68: write d0/de/f10 [68123,73127] 0 2024-04-27T04:09:46.548 INFO:tasks.workunit.client.1.smithi181.stdout:1/87: dwrite d1/d2/f13 [0,4194304] 0 2024-04-27T04:09:46.552 INFO:tasks.workunit.client.1.smithi181.stdout:5/112: unlink d4/lc 0 2024-04-27T04:09:46.554 INFO:tasks.workunit.client.1.smithi181.stdout:9/63: creat d0/f14 x:0 0 0 2024-04-27T04:09:46.560 INFO:tasks.workunit.client.1.smithi181.stdout:0/58: creat dc/f16 x:0 0 0 2024-04-27T04:09:46.565 INFO:tasks.workunit.client.1.smithi181.stdout:2/66: unlink d4/ce 0 2024-04-27T04:09:46.568 INFO:tasks.workunit.client.1.smithi181.stdout:3/69: dread d0/f4 [0,4194304] 0 2024-04-27T04:09:46.571 INFO:tasks.workunit.client.1.smithi181.stdout:9/64: chown d0/l3 7267 1 2024-04-27T04:09:46.571 INFO:tasks.workunit.client.1.smithi181.stdout:9/65: fdatasync d0/fd 0 2024-04-27T04:09:46.577 INFO:tasks.workunit.client.1.smithi181.stdout:0/59: symlink dc/l17 0 2024-04-27T04:09:46.579 INFO:tasks.workunit.client.1.smithi181.stdout:9/66: dwrite d0/f11 [0,4194304] 0 2024-04-27T04:09:46.579 INFO:tasks.workunit.client.1.smithi181.stdout:0/60: truncate dc/f11 435447 0 2024-04-27T04:09:46.580 INFO:tasks.workunit.client.1.smithi181.stdout:0/61: chown dc/f10 111712 1 2024-04-27T04:09:46.581 INFO:tasks.workunit.client.1.smithi181.stdout:2/67: truncate f3 5796251 0 2024-04-27T04:09:46.581 INFO:tasks.workunit.client.1.smithi181.stdout:0/62: write f8 [207104,58958] 0 2024-04-27T04:09:46.587 INFO:tasks.workunit.client.1.smithi181.stdout:8/72: rmdir d1/d11 0 2024-04-27T04:09:46.588 INFO:tasks.workunit.client.1.smithi181.stdout:8/73: truncate d1/d6/f15 1046001 0 2024-04-27T04:09:46.591 INFO:tasks.workunit.client.1.smithi181.stdout:9/67: write d0/f13 [1620049,129567] 0 2024-04-27T04:09:46.593 INFO:tasks.workunit.client.1.smithi181.stdout:9/68: write d0/f8 [449607,9066] 0 2024-04-27T04:09:46.595 INFO:tasks.workunit.client.1.smithi181.stdout:1/88: rmdir d1/d2/d1b 0 2024-04-27T04:09:46.596 INFO:tasks.workunit.client.1.smithi181.stdout:6/8: sync 2024-04-27T04:09:46.596 INFO:tasks.workunit.client.1.smithi181.stdout:7/27: sync 2024-04-27T04:09:46.596 INFO:tasks.workunit.client.1.smithi181.stdout:8/74: dwrite d1/ff [0,4194304] 0 2024-04-27T04:09:46.604 INFO:tasks.workunit.client.1.smithi181.stdout:3/70: mknod d0/c1b 0 2024-04-27T04:09:46.605 INFO:tasks.workunit.client.1.smithi181.stdout:1/89: truncate d1/d2/d5/d6/f1a 308502 0 2024-04-27T04:09:46.606 INFO:tasks.workunit.client.1.smithi181.stdout:3/71: truncate d0/de/f17 471262 0 2024-04-27T04:09:46.608 INFO:tasks.workunit.client.1.smithi181.stdout:8/75: creat d1/d6/f16 x:0 0 0 2024-04-27T04:09:46.609 INFO:tasks.workunit.client.1.smithi181.stdout:6/9: mknod c1 0 2024-04-27T04:09:46.613 INFO:tasks.workunit.client.1.smithi181.stdout:1/90: rename d1/d2/d5/d6/de to d1/d2/d5/d1c 0 2024-04-27T04:09:46.614 INFO:tasks.workunit.client.1.smithi181.stdout:3/72: dwrite d0/f6 [0,4194304] 0 2024-04-27T04:09:46.615 INFO:tasks.workunit.client.1.smithi181.stdout:3/73: fdatasync d0/f19 0 2024-04-27T04:09:46.615 INFO:tasks.workunit.client.1.smithi181.stdout:3/74: chown d0/de/f10 378084 1 2024-04-27T04:09:46.618 INFO:tasks.workunit.client.1.smithi181.stdout:6/10: mknod c2 0 2024-04-27T04:09:46.619 INFO:tasks.workunit.client.1.smithi181.stdout:1/91: chown d1/d2/d5/d1c/f17 23683892 1 2024-04-27T04:09:46.621 INFO:tasks.workunit.client.1.smithi181.stdout:6/11: dread - f0 zero size 2024-04-27T04:09:46.623 INFO:tasks.workunit.client.1.smithi181.stdout:1/92: mknod d1/d2/c1d 0 2024-04-27T04:09:46.623 INFO:tasks.workunit.client.1.smithi181.stdout:8/76: fdatasync d1/ff 0 2024-04-27T04:09:46.625 INFO:tasks.workunit.client.1.smithi181.stdout:6/12: creat f3 x:0 0 0 2024-04-27T04:09:46.627 INFO:tasks.workunit.client.1.smithi181.stdout:3/75: creat d0/f1c x:0 0 0 2024-04-27T04:09:46.627 INFO:tasks.workunit.client.1.smithi181.stdout:9/69: rmdir d0 39 2024-04-27T04:09:46.629 INFO:tasks.workunit.client.1.smithi181.stdout:4/33: write d0/d1/f2 [953949,101737] 0 2024-04-27T04:09:46.630 INFO:tasks.workunit.client.1.smithi181.stdout:1/93: unlink d1/d2/d5/d1c/l14 0 2024-04-27T04:09:46.632 INFO:tasks.workunit.client.1.smithi181.stdout:8/77: dwrite d1/d6/f16 [0,4194304] 0 2024-04-27T04:09:46.633 INFO:tasks.workunit.client.1.smithi181.stdout:1/94: fsync d1/d2/d5/d1c/ff 0 2024-04-27T04:09:46.637 INFO:tasks.workunit.client.1.smithi181.stdout:3/76: creat d0/de/f1d x:0 0 0 2024-04-27T04:09:46.639 INFO:tasks.workunit.client.1.smithi181.stdout:5/113: write d4/de/f14 [3953524,76059] 0 2024-04-27T04:09:46.642 INFO:tasks.workunit.client.1.smithi181.stdout:4/34: symlink d0/l8 0 2024-04-27T04:09:46.647 INFO:tasks.workunit.client.1.smithi181.stdout:8/78: unlink d1/d6/l13 0 2024-04-27T04:09:46.651 INFO:tasks.workunit.client.1.smithi181.stdout:1/95: mkdir d1/d2/d5/d1c/d1e 0 2024-04-27T04:09:46.653 INFO:tasks.workunit.client.1.smithi181.stdout:9/70: creat d0/f15 x:0 0 0 2024-04-27T04:09:46.653 INFO:tasks.workunit.client.1.smithi181.stdout:9/71: dread - d0/f12 zero size 2024-04-27T04:09:46.657 INFO:tasks.workunit.client.1.smithi181.stdout:9/72: dread d0/f11 [0,4194304] 0 2024-04-27T04:09:46.658 INFO:tasks.workunit.client.1.smithi181.stdout:5/114: mknod d4/d10/c27 0 2024-04-27T04:09:46.659 INFO:tasks.workunit.client.1.smithi181.stdout:9/73: write d0/f14 [2969,126014] 0 2024-04-27T04:09:46.659 INFO:tasks.workunit.client.1.smithi181.stdout:5/115: fdatasync d4/d19/f1b 0 2024-04-27T04:09:46.660 INFO:tasks.workunit.client.1.smithi181.stdout:9/74: write d0/f8 [1882734,120665] 0 2024-04-27T04:09:46.662 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:09:46.661+0000 7f26e6988700 1 -- 172.21.15.162:0/2842501521 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f26e00fcf60 msgr2=0x7f26e00fd3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:09:46.663 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:09:46.661+0000 7f26e6988700 1 --2- 172.21.15.162:0/2842501521 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f26e00fcf60 0x7f26e00fd3d0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f26d4004940 tx=0x7f26d4009e30 comp rx=0 tx=0).stop 2024-04-27T04:09:46.663 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:09:46.661+0000 7f26e6988700 1 -- 172.21.15.162:0/2842501521 shutdown_connections 2024-04-27T04:09:46.663 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:09:46.661+0000 7f26e6988700 1 --2- 172.21.15.162:0/2842501521 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f26e00fcf60 0x7f26e00fd3d0 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:09:46.663 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:09:46.661+0000 7f26e6988700 1 --2- 172.21.15.162:0/2842501521 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f26e00fbcb0 0x7f26e00fc0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:09:46.663 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:09:46.661+0000 7f26e6988700 1 -- 172.21.15.162:0/2842501521 >> 172.21.15.162:0/2842501521 conn(0x7f26e00f7430 msgr2=0x7f26e00f9840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:09:46.663 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:09:46.661+0000 7f26e6988700 1 -- 172.21.15.162:0/2842501521 shutdown_connections 2024-04-27T04:09:46.663 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:09:46.662+0000 7f26e6988700 1 -- 172.21.15.162:0/2842501521 wait complete. 2024-04-27T04:09:46.663 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:09:46.662+0000 7f26e6988700 1 Processor -- start 2024-04-27T04:09:46.663 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:09:46.662+0000 7f26e6988700 1 -- start start 2024-04-27T04:09:46.663 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:09:46.663+0000 7f26e6988700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f26e00fbcb0 0x7f26e010d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:09:46.663 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:09:46.663+0000 7f26e6988700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f26e00fcf60 0x7f26e010ded0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:09:46.663 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:09:46.663+0000 7f26e6988700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26e010e4d0 con 0x7f26e00fcf60 2024-04-27T04:09:46.663 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:09:46.663+0000 7f26e6988700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26e010ed20 con 0x7f26e00fbcb0 2024-04-27T04:09:46.664 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:09:46.663+0000 7f26df7fe700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f26e00fcf60 0x7f26e010ded0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:09:46.664 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:09:46.663+0000 7f26df7fe700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f26e00fcf60 0x7f26e010ded0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:60210/0 (socket says 172.21.15.162:60210) 2024-04-27T04:09:46.664 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:09:46.663+0000 7f26df7fe700 1 -- 172.21.15.162:0/3632577156 learned_addr learned my addr 172.21.15.162:0/3632577156 (peer_addr_for_me v2:172.21.15.162:0/0) 2024-04-27T04:09:46.664 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:09:46.663+0000 7f26dffff700 1 --2- 172.21.15.162:0/3632577156 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f26e00fbcb0 0x7f26e010d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:09:46.664 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:09:46.663+0000 7f26df7fe700 1 -- 172.21.15.162:0/3632577156 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f26e00fbcb0 msgr2=0x7f26e010d990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:09:46.664 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:09:46.663+0000 7f26df7fe700 1 --2- 172.21.15.162:0/3632577156 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f26e00fbcb0 0x7f26e010d990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:09:46.664 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:09:46.663+0000 7f26df7fe700 1 -- 172.21.15.162:0/3632577156 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f26d4005040 con 0x7f26e00fcf60 2024-04-27T04:09:46.664 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:09:46.663+0000 7f26df7fe700 1 --2- 172.21.15.162:0/3632577156 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f26e00fcf60 0x7f26e010ded0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f26d400b3b0 tx=0x7f26d4009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:09:46.664 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:09:46.664+0000 7f26dcff9700 1 -- 172.21.15.162:0/3632577156 <== mon.0 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f26d400e040 con 0x7f26e00fcf60 2024-04-27T04:09:46.664 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:09:46.664+0000 7f26e6988700 1 -- 172.21.15.162:0/3632577156 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f26e010efa0 con 0x7f26e00fcf60 2024-04-27T04:09:46.665 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:09:46.664+0000 7f26dcff9700 1 -- 172.21.15.162:0/3632577156 <== mon.0 v2:172.21.15.162:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f26d400bbf0 con 0x7f26e00fcf60 2024-04-27T04:09:46.665 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:09:46.664+0000 7f26dcff9700 1 -- 172.21.15.162:0/3632577156 <== mon.0 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f26d401c630 con 0x7f26e00fcf60 2024-04-27T04:09:46.665 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:09:46.664+0000 7f26e6988700 1 -- 172.21.15.162:0/3632577156 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f26e010f4e0 con 0x7f26e00fcf60 2024-04-27T04:09:46.665 INFO:tasks.workunit.client.1.smithi181.stdout:1/96: creat d1/d2/d5/d1c/f1f x:0 0 0 2024-04-27T04:09:46.666 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:09:46.665+0000 7f26e6988700 1 -- 172.21.15.162:0/3632577156 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f26e0101af0 con 0x7f26e00fcf60 2024-04-27T04:09:46.666 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:09:46.665+0000 7f26dcff9700 1 -- 172.21.15.162:0/3632577156 <== mon.0 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 20) v1 ==== 89550+0+0 (secure 0 0 0) 0x7f26d400bd60 con 0x7f26e00fcf60 2024-04-27T04:09:46.666 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:09:46.666+0000 7f26dcff9700 1 --2- 172.21.15.162:0/3632577156 >> [v2:172.21.15.162:6800/4079782599,v1:172.21.15.162:6801/4079782599] conn(0x7f26c806bbd0 0x7f26c806e080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:09:46.666 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:09:46.666+0000 7f26dcff9700 1 -- 172.21.15.162:0/3632577156 <== mon.0 v2:172.21.15.162:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5550+0+0 (secure 0 0 0) 0x7f26d408b240 con 0x7f26e00fcf60 2024-04-27T04:09:46.667 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:09:46.666+0000 7f26dffff700 1 --2- 172.21.15.162:0/3632577156 >> [v2:172.21.15.162:6800/4079782599,v1:172.21.15.162:6801/4079782599] conn(0x7f26c806bbd0 0x7f26c806e080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:09:46.667 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:09:46.666+0000 7f26dffff700 1 --2- 172.21.15.162:0/3632577156 >> [v2:172.21.15.162:6800/4079782599,v1:172.21.15.162:6801/4079782599] conn(0x7f26c806bbd0 0x7f26c806e080 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f26d0009fd0 tx=0x7f26d000b040 comp rx=0 tx=0).ready entity=mgr.14210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:09:46.668 INFO:tasks.workunit.client.1.smithi181.stdout:7/28: dwrite d1/f4 [0,4194304] 0 2024-04-27T04:09:46.669 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:09:46.670+0000 7f26dcff9700 1 -- 172.21.15.162:0/3632577156 <== mon.0 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f26d405b050 con 0x7f26e00fcf60 2024-04-27T04:09:46.671 INFO:tasks.workunit.client.1.smithi181.stdout:4/35: creat d0/d5/d7/f9 x:0 0 0 2024-04-27T04:09:46.672 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:09:46 smithi181 ceph-mon[33774]: pgmap v258: 65 pgs: 65 active+clean; 277 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 1.6 MiB/s rd, 12 MiB/s wr, 313 op/s 2024-04-27T04:09:46.672 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:09:46 smithi181 ceph-mon[33774]: from='client.24371 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-27T04:09:46.672 INFO:tasks.workunit.client.1.smithi181.stdout:7/29: dread d1/f4 [0,4194304] 0 2024-04-27T04:09:46.673 INFO:tasks.workunit.client.1.smithi181.stdout:5/116: symlink d4/d10/l28 0 2024-04-27T04:09:46.674 INFO:tasks.workunit.client.1.smithi181.stdout:9/75: mkdir d0/d16 0 2024-04-27T04:09:46.675 INFO:tasks.workunit.client.1.smithi181.stdout:8/79: creat d1/de/f17 x:0 0 0 2024-04-27T04:09:46.770 INFO:tasks.workunit.client.1.smithi181.stdout:7/30: readlink d1/d2/l3 0 2024-04-27T04:09:46.773 INFO:tasks.workunit.client.1.smithi181.stdout:9/76: symlink d0/l17 0 2024-04-27T04:09:46.775 INFO:tasks.workunit.client.1.smithi181.stdout:1/97: mkdir d1/d2/d5/d1c/d1e/d20 0 2024-04-27T04:09:46.775 INFO:tasks.workunit.client.1.smithi181.stdout:6/13: dwrite f0 [0,4194304] 0 2024-04-27T04:09:46.776 INFO:tasks.workunit.client.1.smithi181.stdout:6/14: stat c1 0 2024-04-27T04:09:46.776 INFO:tasks.workunit.client.1.smithi181.stdout:2/68: write f2 [5161909,1778] 0 2024-04-27T04:09:46.777 INFO:tasks.workunit.client.1.smithi181.stdout:2/69: readlink d4/la 0 2024-04-27T04:09:46.778 INFO:tasks.workunit.client.1.smithi181.stdout:2/70: stat d4/d9/c10 0 2024-04-27T04:09:46.779 INFO:tasks.workunit.client.1.smithi181.stdout:2/71: write d4/fb [2937934,4201] 0 2024-04-27T04:09:46.784 INFO:tasks.workunit.client.1.smithi181.stdout:5/117: truncate d4/de/f1e 2238839 0 2024-04-27T04:09:46.785 INFO:tasks.workunit.client.1.smithi181.stdout:5/118: truncate d4/f13 4881442 0 2024-04-27T04:09:46.785 INFO:tasks.workunit.client.1.smithi181.stdout:7/31: symlink d1/d2/l6 0 2024-04-27T04:09:46.786 INFO:tasks.workunit.client.1.smithi181.stdout:3/77: write d0/f15 [2627919,37854] 0 2024-04-27T04:09:46.789 INFO:tasks.workunit.client.1.smithi181.stdout:5/119: dread d4/f13 [0,4194304] 0 2024-04-27T04:09:46.793 INFO:tasks.workunit.client.1.smithi181.stdout:1/98: symlink d1/d2/l21 0 2024-04-27T04:09:46.794 INFO:tasks.workunit.client.1.smithi181.stdout:1/99: truncate d1/f16 1015463 0 2024-04-27T04:09:46.794 INFO:tasks.workunit.client.1.smithi181.stdout:6/15: symlink l4 0 2024-04-27T04:09:46.795 INFO:tasks.workunit.client.1.smithi181.stdout:6/16: chown l4 942787636 1 2024-04-27T04:09:46.802 INFO:tasks.workunit.client.1.smithi181.stdout:7/32: dread d1/f4 [0,4194304] 0 2024-04-27T04:09:46.803 INFO:tasks.workunit.client.1.smithi181.stdout:7/33: stat d1/d2/l3 0 2024-04-27T04:09:46.803 INFO:tasks.workunit.client.1.smithi181.stdout:7/34: stat d1/d2/l3 0 2024-04-27T04:09:46.803 INFO:tasks.workunit.client.1.smithi181.stdout:5/120: rename d4/d19 to d4/d10/d29 0 2024-04-27T04:09:46.804 INFO:tasks.workunit.client.1.smithi181.stdout:9/77: symlink d0/d16/l18 0 2024-04-27T04:09:46.805 INFO:tasks.workunit.client.1.smithi181.stdout:9/78: readlink d0/l5 0 2024-04-27T04:09:46.807 INFO:tasks.workunit.client.1.smithi181.stdout:7/35: dread d1/f4 [0,4194304] 0 2024-04-27T04:09:46.809 INFO:tasks.workunit.client.1.smithi181.stdout:6/17: creat f5 x:0 0 0 2024-04-27T04:09:46.809 INFO:tasks.workunit.client.1.smithi181.stdout:6/18: fsync f0 0 2024-04-27T04:09:46.810 INFO:tasks.workunit.client.1.smithi181.stdout:1/100: dread d1/d2/f4 [0,4194304] 0 2024-04-27T04:09:46.811 INFO:tasks.workunit.client.1.smithi181.stdout:1/101: write d1/d2/d5/d1c/ff [4275229,42805] 0 2024-04-27T04:09:46.812 INFO:tasks.workunit.client.1.smithi181.stdout:1/102: read d1/f16 [111082,49963] 0 2024-04-27T04:09:46.813 INFO:tasks.workunit.client.1.smithi181.stdout:1/103: read f0 [2812647,64418] 0 2024-04-27T04:09:46.814 INFO:tasks.workunit.client.1.smithi181.stdout:1/104: readlink d1/d2/d5/d6/l9 0 2024-04-27T04:09:46.814 INFO:tasks.workunit.client.1.smithi181.stdout:5/121: mkdir d4/d18/d2a 0 2024-04-27T04:09:46.815 INFO:tasks.workunit.client.1.smithi181.stdout:5/122: readlink d4/d10/l17 0 2024-04-27T04:09:46.816 INFO:tasks.workunit.client.1.smithi181.stdout:9/79: rename d0/f12 to d0/d16/f19 0 2024-04-27T04:09:46.817 INFO:tasks.workunit.client.1.smithi181.stdout:8/80: truncate d1/d6/f15 102073 0 2024-04-27T04:09:46.823 INFO:tasks.workunit.client.1.smithi181.stdout:9/80: dwrite d0/ff [0,4194304] 0 2024-04-27T04:09:46.824 INFO:tasks.workunit.client.1.smithi181.stdout:6/19: rename f5 to f6 0 2024-04-27T04:09:46.827 INFO:tasks.workunit.client.1.smithi181.stdout:3/78: write d0/f5 [222869,122133] 0 2024-04-27T04:09:46.831 INFO:tasks.workunit.client.1.smithi181.stdout:5/123: write d4/f13 [5771535,99931] 0 2024-04-27T04:09:46.833 INFO:tasks.workunit.client.1.smithi181.stdout:2/72: dwrite d4/f7 [0,4194304] 0 2024-04-27T04:09:46.833 INFO:tasks.workunit.client.1.smithi181.stdout:5/124: dread - d4/d18/f24 zero size 2024-04-27T04:09:46.834 INFO:tasks.workunit.client.1.smithi181.stdout:2/73: readlink d4/la 0 2024-04-27T04:09:46.835 INFO:tasks.workunit.client.1.smithi181.stdout:8/81: creat d1/de/f18 x:0 0 0 2024-04-27T04:09:46.835 INFO:tasks.workunit.client.1.smithi181.stdout:8/82: rename d1/de to d1/de/d19 22 2024-04-27T04:09:46.840 INFO:tasks.workunit.client.1.smithi181.stdout:9/81: mknod d0/c1a 0 2024-04-27T04:09:46.841 INFO:tasks.workunit.client.1.smithi181.stdout:6/20: mkdir d7 0 2024-04-27T04:09:46.841 INFO:tasks.workunit.client.1.smithi181.stdout:6/21: chown d7 260289151 1 2024-04-27T04:09:46.848 INFO:tasks.workunit.client.1.smithi181.stdout:5/125: chown d4/de/f1e 3358967 1 2024-04-27T04:09:46.849 INFO:tasks.workunit.client.1.smithi181.stdout:7/36: dwrite d1/f4 [0,4194304] 0 2024-04-27T04:09:46.850 INFO:tasks.workunit.client.1.smithi181.stdout:5/126: truncate d4/d18/f23 4450250 0 2024-04-27T04:09:46.851 INFO:tasks.workunit.client.1.smithi181.stdout:7/37: write d1/f4 [4022423,118942] 0 2024-04-27T04:09:46.851 INFO:tasks.workunit.client.1.smithi181.stdout:2/74: mknod d4/d9/df/c18 0 2024-04-27T04:09:46.852 INFO:tasks.workunit.client.1.smithi181.stdout:8/83: mkdir d1/d6/d1a 0 2024-04-27T04:09:46.853 INFO:tasks.workunit.client.1.smithi181.stdout:2/75: dread f2 [4194304,4194304] 0 2024-04-27T04:09:46.854 INFO:tasks.workunit.client.1.smithi181.stdout:9/82: symlink d0/d16/l1b 0 2024-04-27T04:09:46.855 INFO:tasks.workunit.client.1.smithi181.stdout:9/83: readlink d0/d16/l1b 0 2024-04-27T04:09:46.855 INFO:tasks.workunit.client.1.smithi181.stdout:9/84: readlink d0/l17 0 2024-04-27T04:09:46.856 INFO:tasks.workunit.client.1.smithi181.stdout:9/85: write d0/fd [68410,43470] 0 2024-04-27T04:09:46.863 INFO:tasks.workunit.client.1.smithi181.stdout:7/38: mknod d1/d2/c7 0 2024-04-27T04:09:46.864 INFO:tasks.workunit.client.1.smithi181.stdout:8/84: mknod d1/d6/c1b 0 2024-04-27T04:09:46.865 INFO:tasks.workunit.client.1.smithi181.stdout:6/22: getdents d7 0 2024-04-27T04:09:46.866 INFO:tasks.workunit.client.1.smithi181.stdout:3/79: creat d0/f1e x:0 0 0 2024-04-27T04:09:46.867 INFO:tasks.workunit.client.1.smithi181.stdout:3/80: write d0/de/ff [221721,22790] 0 2024-04-27T04:09:46.869 INFO:tasks.workunit.client.1.smithi181.stdout:3/81: chown d0/c1b 525 1 2024-04-27T04:09:46.870 INFO:tasks.workunit.client.1.smithi181.stdout:3/82: readlink d0/l9 0 2024-04-27T04:09:46.871 INFO:tasks.workunit.client.1.smithi181.stdout:3/83: truncate d0/de/ff 705339 0 2024-04-27T04:09:46.872 INFO:tasks.workunit.client.1.smithi181.stdout:7/39: creat d1/f8 x:0 0 0 2024-04-27T04:09:46.873 INFO:tasks.workunit.client.1.smithi181.stdout:6/23: dwrite f3 [0,4194304] 0 2024-04-27T04:09:46.873 INFO:tasks.workunit.client.1.smithi181.stdout:8/85: mkdir d1/d1c 0 2024-04-27T04:09:46.874 INFO:tasks.workunit.client.1.smithi181.stdout:7/40: dread - d1/f8 zero size 2024-04-27T04:09:46.874 INFO:tasks.workunit.client.1.smithi181.stdout:6/24: stat f0 0 2024-04-27T04:09:46.874 INFO:tasks.workunit.client.1.smithi181.stdout:7/41: dread - d1/f8 zero size 2024-04-27T04:09:46.881 INFO:tasks.workunit.client.1.smithi181.stdout:2/76: link d4/d9/df/c18 d4/d9/c19 0 2024-04-27T04:09:46.884 INFO:tasks.workunit.client.1.smithi181.stdout:7/42: rename d1/d2/l3 to d1/d2/l9 0 2024-04-27T04:09:46.884 INFO:tasks.workunit.client.1.smithi181.stdout:7/43: read - d1/f8 zero size 2024-04-27T04:09:46.886 INFO:tasks.workunit.client.1.smithi181.stdout:5/127: link d4/de/c1f d4/c2b 0 2024-04-27T04:09:46.887 INFO:tasks.workunit.client.1.smithi181.stdout:5/128: chown d4/de/f14 8340347 1 2024-04-27T04:09:46.894 INFO:tasks.workunit.client.1.smithi181.stdout:2/77: creat d4/d9/f1a x:0 0 0 2024-04-27T04:09:46.894 INFO:tasks.workunit.client.1.smithi181.stdout:2/78: read d4/f7 [1043708,104522] 0 2024-04-27T04:09:46.896 INFO:tasks.workunit.client.1.smithi181.stdout:9/86: getdents d0/d16 0 2024-04-27T04:09:46.896 INFO:tasks.workunit.client.1.smithi181.stdout:9/87: stat d0/d16/f19 0 2024-04-27T04:09:46.898 INFO:tasks.workunit.client.1.smithi181.stdout:5/129: mkdir d4/de/d2c 0 2024-04-27T04:09:46.900 INFO:tasks.workunit.client.1.smithi181.stdout:5/130: write d4/d18/f24 [776184,118747] 0 2024-04-27T04:09:46.900 INFO:tasks.workunit.client.1.smithi181.stdout:8/86: rmdir d1/d1c 0 2024-04-27T04:09:46.904 INFO:tasks.workunit.client.1.smithi181.stdout:2/79: symlink d4/l1b 0 2024-04-27T04:09:46.904 INFO:tasks.workunit.client.1.smithi181.stdout:2/80: fdatasync d4/d9/f1a 0 2024-04-27T04:09:46.905 INFO:tasks.workunit.client.1.smithi181.stdout:7/44: rename d1/l5 to d1/la 0 2024-04-27T04:09:46.907 INFO:tasks.workunit.client.1.smithi181.stdout:7/45: write d1/f4 [1122610,126305] 0 2024-04-27T04:09:46.907 INFO:tasks.workunit.client.1.smithi181.stdout:7/46: stat d1/d2 0 2024-04-27T04:09:46.908 INFO:tasks.workunit.client.1.smithi181.stdout:9/88: rename d0/f7 to d0/d16/f1c 0 2024-04-27T04:09:46.908 INFO:tasks.workunit.client.1.smithi181.stdout:7/47: stat d1/d2 0 2024-04-27T04:09:46.908 INFO:tasks.workunit.client.1.smithi181.stdout:5/131: rename d4/de to d4/de/d2c/d2d 22 2024-04-27T04:09:46.908 INFO:tasks.workunit.client.1.smithi181.stdout:3/84: truncate d0/f15 761639 0 2024-04-27T04:09:46.909 INFO:tasks.workunit.client.1.smithi181.stdout:7/48: write d1/f8 [576835,57127] 0 2024-04-27T04:09:46.909 INFO:tasks.workunit.client.1.smithi181.stdout:3/85: fsync d0/f14 0 2024-04-27T04:09:46.913 INFO:tasks.workunit.client.1.smithi181.stdout:6/25: truncate f3 3056111 0 2024-04-27T04:09:46.913 INFO:tasks.workunit.client.1.smithi181.stdout:8/87: creat d1/d6/f1d x:0 0 0 2024-04-27T04:09:46.916 INFO:tasks.workunit.client.1.smithi181.stdout:8/88: write d1/de/f17 [314487,123125] 0 2024-04-27T04:09:46.920 INFO:tasks.workunit.client.1.smithi181.stdout:9/89: dwrite d0/fd [0,4194304] 0 2024-04-27T04:09:46.922 INFO:tasks.workunit.client.1.smithi181.stdout:3/86: dwrite d0/f19 [0,4194304] 0 2024-04-27T04:09:46.927 INFO:tasks.workunit.client.1.smithi181.stdout:2/81: rename d4/d9/c12 to d4/d9/df/c1c 0 2024-04-27T04:09:46.928 INFO:tasks.workunit.client.1.smithi181.stdout:5/132: mkdir d4/de/d2c/d2e 0 2024-04-27T04:09:46.929 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:09:46.930+0000 7f26e6988700 1 -- 172.21.15.162:0/3632577156 --> [v2:172.21.15.162:6800/4079782599,v1:172.21.15.162:6801/4079782599] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f26e010f750 con 0x7f26c806bbd0 2024-04-27T04:09:46.929 INFO:tasks.workunit.client.1.smithi181.stdout:2/82: write d4/d9/f1a [871617,19553] 0 2024-04-27T04:09:46.930 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:09:46.931+0000 7f26dcff9700 1 -- 172.21.15.162:0/3632577156 <== mgr.14210 v2:172.21.15.162:6800/4079782599 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+321 (secure 0 0 0) 0x7f26e010f750 con 0x7f26c806bbd0 2024-04-27T04:09:46.930 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-04-27T04:09:46.930 INFO:teuthology.orchestra.run.smithi162.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:90a3a9fa9ceb8977f6dad9ccd36fe46018d07e2118199820523a6b35301d9c16", 2024-04-27T04:09:46.930 INFO:teuthology.orchestra.run.smithi162.stdout: "in_progress": true, 2024-04-27T04:09:46.930 INFO:teuthology.orchestra.run.smithi162.stdout: "which": "Upgrading daemons of type(s) mgr", 2024-04-27T04:09:46.930 INFO:teuthology.orchestra.run.smithi162.stdout: "services_complete": [], 2024-04-27T04:09:46.930 INFO:teuthology.orchestra.run.smithi162.stdout: "progress": "", 2024-04-27T04:09:46.931 INFO:teuthology.orchestra.run.smithi162.stdout: "message": "Currently upgrading mgr daemons", 2024-04-27T04:09:46.931 INFO:teuthology.orchestra.run.smithi162.stdout: "is_paused": false 2024-04-27T04:09:46.931 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-04-27T04:09:46.931 INFO:tasks.workunit.client.1.smithi181.stdout:2/83: chown d4/d9/df/d14/f15 13076114 1 2024-04-27T04:09:46.932 INFO:tasks.workunit.client.1.smithi181.stdout:3/87: creat d0/de/f1f x:0 0 0 2024-04-27T04:09:46.933 INFO:tasks.workunit.client.1.smithi181.stdout:5/133: dread d4/f13 [4194304,4194304] 0 2024-04-27T04:09:46.935 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:09:46.935+0000 7f26e6988700 1 -- 172.21.15.162:0/3632577156 >> [v2:172.21.15.162:6800/4079782599,v1:172.21.15.162:6801/4079782599] conn(0x7f26c806bbd0 msgr2=0x7f26c806e080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:09:46.935 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:09:46.935+0000 7f26e6988700 1 --2- 172.21.15.162:0/3632577156 >> [v2:172.21.15.162:6800/4079782599,v1:172.21.15.162:6801/4079782599] conn(0x7f26c806bbd0 0x7f26c806e080 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f26d0009fd0 tx=0x7f26d000b040 comp rx=0 tx=0).stop 2024-04-27T04:09:46.935 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:09:46.935+0000 7f26e6988700 1 -- 172.21.15.162:0/3632577156 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f26e00fcf60 msgr2=0x7f26e010ded0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:09:46.935 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:09:46.935+0000 7f26e6988700 1 --2- 172.21.15.162:0/3632577156 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f26e00fcf60 0x7f26e010ded0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f26d400b3b0 tx=0x7f26d4009e30 comp rx=0 tx=0).stop 2024-04-27T04:09:46.935 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:09:46.935+0000 7f26e6988700 1 -- 172.21.15.162:0/3632577156 shutdown_connections 2024-04-27T04:09:46.935 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:09:46.936+0000 7f26e6988700 1 --2- 172.21.15.162:0/3632577156 >> [v2:172.21.15.162:6800/4079782599,v1:172.21.15.162:6801/4079782599] conn(0x7f26c806bbd0 0x7f26c806e080 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:09:46.935 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:09:46.936+0000 7f26e6988700 1 --2- 172.21.15.162:0/3632577156 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f26e00fbcb0 0x7f26e010d990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:09:46.935 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:09:46.936+0000 7f26e6988700 1 --2- 172.21.15.162:0/3632577156 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f26e00fcf60 0x7f26e010ded0 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:09:46.935 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:09:46.936+0000 7f26e6988700 1 -- 172.21.15.162:0/3632577156 >> 172.21.15.162:0/3632577156 conn(0x7f26e00f7430 msgr2=0x7f26e00f8a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:09:46.935 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:09:46.936+0000 7f26e6988700 1 -- 172.21.15.162:0/3632577156 shutdown_connections 2024-04-27T04:09:46.935 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:09:46.936+0000 7f26e6988700 1 -- 172.21.15.162:0/3632577156 wait complete. 2024-04-27T04:09:46.936 INFO:tasks.workunit.client.1.smithi181.stdout:5/134: read d4/de/f1e [1984394,57342] 0 2024-04-27T04:09:46.936 INFO:tasks.workunit.client.1.smithi181.stdout:7/49: rename d1/d2/l9 to d1/d2/lb 0 2024-04-27T04:09:46.941 INFO:tasks.workunit.client.1.smithi181.stdout:6/26: write f3 [3308081,60598] 0 2024-04-27T04:09:46.954 INFO:tasks.workunit.client.1.smithi181.stdout:6/27: creat d7/f8 x:0 0 0 2024-04-27T04:09:46.957 INFO:tasks.workunit.client.1.smithi181.stdout:6/28: fdatasync f6 0 2024-04-27T04:09:46.957 INFO:tasks.workunit.client.1.smithi181.stdout:6/29: chown c2 3 1 2024-04-27T04:09:46.958 INFO:tasks.workunit.client.1.smithi181.stdout:6/30: truncate f3 3637062 0 2024-04-27T04:09:46.959 INFO:tasks.workunit.client.1.smithi181.stdout:2/84: getdents d4/d9/df 0 2024-04-27T04:09:46.959 INFO:tasks.workunit.client.1.smithi181.stdout:6/31: chown c1 1019 1 2024-04-27T04:09:46.960 INFO:tasks.workunit.client.1.smithi181.stdout:6/32: fdatasync f3 0 2024-04-27T04:09:46.961 INFO:tasks.workunit.client.1.smithi181.stdout:6/33: write f6 [898560,68684] 0 2024-04-27T04:09:46.965 INFO:tasks.workunit.client.1.smithi181.stdout:8/89: dwrite d1/d6/f15 [0,4194304] 0 2024-04-27T04:09:46.968 INFO:tasks.workunit.client.1.smithi181.stdout:3/88: write d0/f15 [1497145,29776] 0 2024-04-27T04:09:46.970 INFO:tasks.workunit.client.1.smithi181.stdout:3/89: readlink d0/l9 0 2024-04-27T04:09:46.972 INFO:tasks.workunit.client.1.smithi181.stdout:5/135: dwrite d4/de/f1e [0,4194304] 0 2024-04-27T04:09:46.973 INFO:tasks.workunit.client.1.smithi181.stdout:8/90: chown d1/cd 0 1 2024-04-27T04:09:46.974 INFO:tasks.workunit.client.1.smithi181.stdout:2/85: rename d4/d9/df/c18 to d4/d9/df/d14/c1d 0 2024-04-27T04:09:46.974 INFO:tasks.workunit.client.1.smithi181.stdout:3/90: dread d0/de/f17 [0,4194304] 0 2024-04-27T04:09:46.976 INFO:tasks.workunit.client.1.smithi181.stdout:8/91: write d1/d6/f1d [844595,47830] 0 2024-04-27T04:09:46.976 INFO:tasks.workunit.client.1.smithi181.stdout:3/91: chown d0/f5 18961 1 2024-04-27T04:09:46.977 INFO:tasks.workunit.client.1.smithi181.stdout:8/92: fdatasync d1/de/f17 0 2024-04-27T04:09:46.977 INFO:tasks.workunit.client.1.smithi181.stdout:0/63: sync 2024-04-27T04:09:46.979 INFO:tasks.workunit.client.1.smithi181.stdout:0/64: chown l1 42140 1 2024-04-27T04:09:46.980 INFO:tasks.workunit.client.1.smithi181.stdout:2/86: dread d4/fb [0,4194304] 0 2024-04-27T04:09:46.983 INFO:tasks.workunit.client.1.smithi181.stdout:5/136: rename d4/de/df to d4/de/d2f 0 2024-04-27T04:09:46.984 INFO:tasks.workunit.client.1.smithi181.stdout:5/137: write d4/de/f14 [3268292,90877] 0 2024-04-27T04:09:46.988 INFO:tasks.workunit.client.1.smithi181.stdout:6/34: truncate f6 637152 0 2024-04-27T04:09:46.988 INFO:tasks.workunit.client.1.smithi181.stdout:6/35: chown c1 448777868 1 2024-04-27T04:09:46.989 INFO:tasks.workunit.client.1.smithi181.stdout:0/65: creat dc/f18 x:0 0 0 2024-04-27T04:09:46.992 INFO:tasks.workunit.client.1.smithi181.stdout:8/93: rename d1/d6/l14 to d1/d6/d1a/l1e 0 2024-04-27T04:09:46.993 INFO:tasks.workunit.client.1.smithi181.stdout:8/94: write d1/d6/f16 [2095967,125240] 0 2024-04-27T04:09:46.995 INFO:tasks.workunit.client.1.smithi181.stdout:5/138: mknod d4/d10/d29/c30 0 2024-04-27T04:09:46.997 INFO:tasks.workunit.client.1.smithi181.stdout:0/66: symlink dc/l19 0 2024-04-27T04:09:46.999 INFO:tasks.workunit.client.1.smithi181.stdout:8/95: creat d1/f1f x:0 0 0 2024-04-27T04:09:47.002 INFO:tasks.workunit.client.1.smithi181.stdout:0/67: mknod dc/c1a 0 2024-04-27T04:09:47.003 INFO:tasks.workunit.client.1.smithi181.stdout:0/68: chown dc/f10 0 1 2024-04-27T04:09:47.005 INFO:tasks.workunit.client.1.smithi181.stdout:0/69: write dc/f12 [759996,76974] 0 2024-04-27T04:09:47.006 INFO:tasks.workunit.client.1.smithi181.stdout:8/96: dwrite d1/ff [4194304,4194304] 0 2024-04-27T04:09:47.006 INFO:tasks.workunit.client.1.smithi181.stdout:0/70: write f8 [1298651,126885] 0 2024-04-27T04:09:47.009 INFO:tasks.workunit.client.1.smithi181.stdout:8/97: mkdir d1/d20 0 2024-04-27T04:09:47.011 INFO:tasks.workunit.client.1.smithi181.stdout:0/71: creat dc/f1b x:0 0 0 2024-04-27T04:09:47.015 INFO:tasks.workunit.client.1.smithi181.stdout:0/72: creat dc/f1c x:0 0 0 2024-04-27T04:09:47.018 INFO:tasks.workunit.client.1.smithi181.stdout:8/98: dwrite d1/d6/f1d [0,4194304] 0 2024-04-27T04:09:47.023 INFO:tasks.workunit.client.1.smithi181.stdout:8/99: chown d1/cd 173095294 1 2024-04-27T04:09:47.024 INFO:tasks.workunit.client.1.smithi181.stdout:3/92: dwrite d0/f1 [0,4194304] 0 2024-04-27T04:09:47.031 INFO:tasks.workunit.client.1.smithi181.stdout:0/73: dwrite dc/f1b [0,4194304] 0 2024-04-27T04:09:47.033 INFO:tasks.workunit.client.1.smithi181.stdout:0/74: fsync dc/f10 0 2024-04-27T04:09:47.035 INFO:tasks.workunit.client.1.smithi181.stdout:0/75: dread - dc/ff zero size 2024-04-27T04:09:47.037 INFO:tasks.workunit.client.1.smithi181.stdout:3/93: dread d0/f14 [0,4194304] 0 2024-04-27T04:09:47.038 INFO:tasks.workunit.client.1.smithi181.stdout:8/100: dwrite d1/f1f [0,4194304] 0 2024-04-27T04:09:47.038 INFO:tasks.workunit.client.1.smithi181.stdout:0/76: creat dc/f1d x:0 0 0 2024-04-27T04:09:47.039 INFO:tasks.workunit.client.1.smithi181.stdout:3/94: read d0/f5 [2796569,45803] 0 2024-04-27T04:09:47.040 INFO:tasks.workunit.client.1.smithi181.stdout:8/101: fsync d1/d6/f15 0 2024-04-27T04:09:47.040 INFO:tasks.workunit.client.1.smithi181.stdout:3/95: read d0/f1 [427304,59072] 0 2024-04-27T04:09:47.041 INFO:tasks.workunit.client.1.smithi181.stdout:3/96: read - d0/f1c zero size 2024-04-27T04:09:47.043 INFO:tasks.workunit.client.1.smithi181.stdout:0/77: mkdir dc/d1e 0 2024-04-27T04:09:47.046 INFO:tasks.workunit.client.1.smithi181.stdout:8/102: dread d1/d6/f1d [0,4194304] 0 2024-04-27T04:09:47.051 INFO:tasks.workunit.client.1.smithi181.stdout:0/78: mkdir dc/d1f 0 2024-04-27T04:09:47.053 INFO:tasks.workunit.client.1.smithi181.stdout:0/79: write dc/f1b [3133132,55229] 0 2024-04-27T04:09:47.056 INFO:tasks.workunit.client.1.smithi181.stdout:0/80: link dc/f15 dc/d1e/f20 0 2024-04-27T04:09:47.065 INFO:tasks.workunit.client.1.smithi181.stdout:0/81: dwrite dc/fd [0,4194304] 0 2024-04-27T04:09:47.066 INFO:tasks.workunit.client.1.smithi181.stdout:0/82: dread - dc/f1c zero size 2024-04-27T04:09:47.069 INFO:tasks.workunit.client.1.smithi181.stdout:0/83: mknod dc/d1e/c21 0 2024-04-27T04:09:47.069 INFO:tasks.workunit.client.1.smithi181.stdout:0/84: chown dc/f10 27 1 2024-04-27T04:09:47.071 INFO:tasks.workunit.client.1.smithi181.stdout:0/85: write dc/d1e/f20 [839298,33907] 0 2024-04-27T04:09:47.073 INFO:tasks.workunit.client.1.smithi181.stdout:0/86: write dc/f11 [535480,26982] 0 2024-04-27T04:09:47.074 INFO:tasks.workunit.client.1.smithi181.stdout:0/87: creat dc/f22 x:0 0 0 2024-04-27T04:09:47.076 INFO:tasks.workunit.client.1.smithi181.stdout:0/88: creat dc/d1f/f23 x:0 0 0 2024-04-27T04:09:47.096 INFO:tasks.workunit.client.1.smithi181.stdout:3/97: dread d0/de/f10 [0,4194304] 0 2024-04-27T04:09:47.096 INFO:tasks.workunit.client.1.smithi181.stdout:6/36: fsync f6 0 2024-04-27T04:09:47.097 INFO:tasks.workunit.client.1.smithi181.stdout:8/103: readlink d1/d6/d1a/l1e 0 2024-04-27T04:09:47.099 INFO:tasks.workunit.client.1.smithi181.stdout:0/89: dread dc/f15 [0,4194304] 0 2024-04-27T04:09:47.099 INFO:tasks.workunit.client.1.smithi181.stdout:8/104: write d1/ff [5571930,1600] 0 2024-04-27T04:09:47.100 INFO:tasks.workunit.client.1.smithi181.stdout:8/105: truncate d1/de/f18 852197 0 2024-04-27T04:09:47.100 INFO:tasks.workunit.client.1.smithi181.stdout:3/98: mkdir d0/de/d20 0 2024-04-27T04:09:47.101 INFO:tasks.workunit.client.1.smithi181.stdout:2/87: write d4/f7 [3054923,90808] 0 2024-04-27T04:09:47.105 INFO:tasks.workunit.client.1.smithi181.stdout:0/90: rename dc/f12 to dc/d1e/f24 0 2024-04-27T04:09:47.106 INFO:tasks.workunit.client.1.smithi181.stdout:0/91: dread - dc/f1c zero size 2024-04-27T04:09:47.109 INFO:tasks.workunit.client.1.smithi181.stdout:8/106: symlink d1/d6/d1a/l21 0 2024-04-27T04:09:47.111 INFO:tasks.workunit.client.1.smithi181.stdout:5/139: write d4/f13 [3118968,22827] 0 2024-04-27T04:09:47.113 INFO:tasks.workunit.client.1.smithi181.stdout:2/88: mknod d4/d9/d16/c1e 0 2024-04-27T04:09:47.115 INFO:tasks.workunit.client.1.smithi181.stdout:5/140: dread d4/de/f14 [0,4194304] 0 2024-04-27T04:09:47.116 INFO:tasks.workunit.client.1.smithi181.stdout:5/141: chown d4/de/d2f/c21 569 1 2024-04-27T04:09:47.117 INFO:tasks.workunit.client.1.smithi181.stdout:3/99: creat d0/de/d20/f21 x:0 0 0 2024-04-27T04:09:47.117 INFO:tasks.workunit.client.1.smithi181.stdout:5/142: chown d4/c7 117496 1 2024-04-27T04:09:47.120 INFO:tasks.workunit.client.1.smithi181.stdout:0/92: link dc/f16 dc/d1e/f25 0 2024-04-27T04:09:47.121 INFO:tasks.workunit.client.1.smithi181.stdout:2/89: rmdir d4/d9/d16 39 2024-04-27T04:09:47.125 INFO:tasks.workunit.client.1.smithi181.stdout:2/90: dread d4/fb [0,4194304] 0 2024-04-27T04:09:47.125 INFO:tasks.workunit.client.1.smithi181.stdout:5/143: creat d4/de/d2f/f31 x:0 0 0 2024-04-27T04:09:47.133 INFO:tasks.workunit.client.1.smithi181.stdout:4/36: sync 2024-04-27T04:09:47.133 INFO:tasks.workunit.client.1.smithi181.stdout:5/144: dwrite d4/d10/d29/f1b [0,4194304] 0 2024-04-27T04:09:47.135 INFO:tasks.workunit.client.1.smithi181.stdout:5/145: write d4/de/d2f/f31 [140736,111727] 0 2024-04-27T04:09:47.138 INFO:tasks.workunit.client.1.smithi181.stdout:0/93: mkdir dc/d1e/d26 0 2024-04-27T04:09:47.142 INFO:tasks.workunit.client.1.smithi181.stdout:2/91: creat d4/f1f x:0 0 0 2024-04-27T04:09:47.142 INFO:tasks.workunit.client.1.smithi181.stdout:4/37: dwrite d0/d5/f6 [0,4194304] 0 2024-04-27T04:09:47.144 INFO:tasks.workunit.client.1.smithi181.stdout:4/38: write d0/d5/d7/f9 [306964,36001] 0 2024-04-27T04:09:47.144 INFO:tasks.workunit.client.1.smithi181.stdout:5/146: symlink d4/l32 0 2024-04-27T04:09:47.145 INFO:tasks.workunit.client.1.smithi181.stdout:4/39: fsync d0/d1/f2 0 2024-04-27T04:09:47.149 INFO:tasks.workunit.client.1.smithi181.stdout:0/94: symlink dc/l27 0 2024-04-27T04:09:47.150 INFO:tasks.workunit.client.1.smithi181.stdout:0/95: dread - dc/d1f/f23 zero size 2024-04-27T04:09:47.156 INFO:tasks.workunit.client.1.smithi181.stdout:5/147: mknod d4/de/c33 0 2024-04-27T04:09:47.161 INFO:tasks.workunit.client.1.smithi181.stdout:5/148: write d4/de/f1e [411206,89294] 0 2024-04-27T04:09:47.162 INFO:tasks.workunit.client.1.smithi181.stdout:2/92: mknod d4/d9/d16/c20 0 2024-04-27T04:09:47.162 INFO:tasks.workunit.client.1.smithi181.stdout:3/100: getdents d0 0 2024-04-27T04:09:47.171 INFO:tasks.workunit.client.1.smithi181.stdout:2/93: dread f2 [0,4194304] 0 2024-04-27T04:09:47.174 INFO:tasks.workunit.client.1.smithi181.stdout:2/94: dread d4/f7 [0,4194304] 0 2024-04-27T04:09:47.178 INFO:tasks.workunit.client.1.smithi181.stdout:6/37: truncate f6 325504 0 2024-04-27T04:09:47.179 INFO:tasks.workunit.client.1.smithi181.stdout:2/95: dread d4/d9/f1a [0,4194304] 0 2024-04-27T04:09:47.179 INFO:tasks.workunit.client.1.smithi181.stdout:5/149: link d4/c1a d4/c34 0 2024-04-27T04:09:47.180 INFO:tasks.workunit.client.1.smithi181.stdout:3/101: link d0/lc d0/de/d20/l22 0 2024-04-27T04:09:47.184 INFO:tasks.workunit.client.1.smithi181.stdout:2/96: unlink f3 0 2024-04-27T04:09:47.186 INFO:tasks.workunit.client.1.smithi181.stdout:8/107: rmdir d1 39 2024-04-27T04:09:47.189 INFO:tasks.workunit.client.1.smithi181.stdout:3/102: rename d0/f1c to d0/f23 0 2024-04-27T04:09:47.192 INFO:tasks.workunit.client.1.smithi181.stdout:2/97: mknod d4/c21 0 2024-04-27T04:09:47.192 INFO:tasks.workunit.client.1.smithi181.stdout:1/105: sync 2024-04-27T04:09:47.193 INFO:tasks.workunit.client.1.smithi181.stdout:6/38: link d7/f8 d7/f9 0 2024-04-27T04:09:47.195 INFO:tasks.workunit.client.1.smithi181.stdout:8/108: write d1/de/f17 [1188786,81883] 0 2024-04-27T04:09:47.199 INFO:tasks.workunit.client.1.smithi181.stdout:2/98: unlink d4/f1f 0 2024-04-27T04:09:47.204 INFO:tasks.workunit.client.1.smithi181.stdout:5/150: link d4/l1d d4/d18/d2a/l35 0 2024-04-27T04:09:47.205 INFO:tasks.workunit.client.1.smithi181.stdout:0/96: getdents dc/d1e 0 2024-04-27T04:09:47.206 INFO:tasks.workunit.client.1.smithi181.stdout:8/109: symlink d1/l22 0 2024-04-27T04:09:47.207 INFO:tasks.workunit.client.1.smithi181.stdout:0/97: write dc/d1f/f23 [314079,81280] 0 2024-04-27T04:09:47.207 INFO:tasks.workunit.client.1.smithi181.stdout:0/98: read - dc/f22 zero size 2024-04-27T04:09:47.207 INFO:tasks.workunit.client.1.smithi181.stdout:8/110: write d1/f1f [5232279,117775] 0 2024-04-27T04:09:47.208 INFO:tasks.workunit.client.1.smithi181.stdout:8/111: chown d1/f1f 477955832 1 2024-04-27T04:09:47.209 INFO:tasks.workunit.client.1.smithi181.stdout:8/112: write d1/de/f17 [398883,18322] 0 2024-04-27T04:09:47.214 INFO:tasks.workunit.client.1.smithi181.stdout:2/99: dread d4/d9/f1a [0,4194304] 0 2024-04-27T04:09:47.221 INFO:tasks.workunit.client.1.smithi181.stdout:4/40: truncate d0/d1/f4 875057 0 2024-04-27T04:09:47.225 INFO:tasks.workunit.client.1.smithi181.stdout:8/113: symlink d1/l23 0 2024-04-27T04:09:47.229 INFO:tasks.workunit.client.1.smithi181.stdout:2/100: symlink d4/d9/df/l22 0 2024-04-27T04:09:47.230 INFO:tasks.workunit.client.1.smithi181.stdout:2/101: chown d4/d9/df/d14/f15 34100061 1 2024-04-27T04:09:47.231 INFO:tasks.workunit.client.1.smithi181.stdout:2/102: write d4/f8 [1113193,127386] 0 2024-04-27T04:09:47.233 INFO:tasks.workunit.client.1.smithi181.stdout:5/151: rename d4/lb to d4/de/d2f/l36 0 2024-04-27T04:09:47.233 INFO:tasks.workunit.client.1.smithi181.stdout:5/152: stat d4/c7 0 2024-04-27T04:09:47.235 INFO:tasks.workunit.client.1.smithi181.stdout:2/103: dread d4/f8 [0,4194304] 0 2024-04-27T04:09:47.238 INFO:tasks.workunit.client.1.smithi181.stdout:5/153: dread d4/d18/f24 [0,4194304] 0 2024-04-27T04:09:47.241 INFO:tasks.workunit.client.1.smithi181.stdout:9/90: sync 2024-04-27T04:09:47.241 INFO:tasks.workunit.client.1.smithi181.stdout:7/50: sync 2024-04-27T04:09:47.241 INFO:tasks.workunit.client.1.smithi181.stdout:0/99: rename dc/d1e/f20 to dc/d1e/d26/f28 0 2024-04-27T04:09:47.243 INFO:tasks.workunit.client.1.smithi181.stdout:9/91: write d0/f9 [2244140,10978] 0 2024-04-27T04:09:47.243 INFO:tasks.workunit.client.1.smithi181.stdout:7/51: truncate d1/f8 652331 0 2024-04-27T04:09:47.243 INFO:tasks.workunit.client.1.smithi181.stdout:0/100: write dc/f11 [1504789,41555] 0 2024-04-27T04:09:47.245 INFO:tasks.workunit.client.1.smithi181.stdout:0/101: truncate dc/f1d 183425 0 2024-04-27T04:09:47.246 INFO:tasks.workunit.client.1.smithi181.stdout:0/102: dread - dc/f10 zero size 2024-04-27T04:09:47.247 INFO:tasks.workunit.client.1.smithi181.stdout:2/104: fdatasync d4/f6 0 2024-04-27T04:09:47.248 INFO:tasks.workunit.client.1.smithi181.stdout:0/103: fsync dc/f1d 0 2024-04-27T04:09:47.250 INFO:tasks.workunit.client.1.smithi181.stdout:1/106: dwrite d1/d2/f4 [0,4194304] 0 2024-04-27T04:09:47.252 INFO:tasks.workunit.client.1.smithi181.stdout:3/103: write d0/f3 [1001318,18977] 0 2024-04-27T04:09:47.261 INFO:tasks.workunit.client.1.smithi181.stdout:4/41: rename d0/d1 to d0/d5/da 0 2024-04-27T04:09:47.266 INFO:tasks.workunit.client.1.smithi181.stdout:9/92: symlink d0/d16/l1d 0 2024-04-27T04:09:47.271 INFO:tasks.workunit.client.1.smithi181.stdout:2/105: creat d4/d9/d16/f23 x:0 0 0 2024-04-27T04:09:47.273 INFO:tasks.workunit.client.1.smithi181.stdout:6/39: dwrite f6 [0,4194304] 0 2024-04-27T04:09:47.273 INFO:tasks.workunit.client.1.smithi181.stdout:0/104: creat dc/d1f/f29 x:0 0 0 2024-04-27T04:09:47.274 INFO:tasks.workunit.client.1.smithi181.stdout:0/105: dread - dc/d1f/f29 zero size 2024-04-27T04:09:47.276 INFO:tasks.workunit.client.1.smithi181.stdout:0/106: truncate dc/d1f/f29 1038458 0 2024-04-27T04:09:47.278 INFO:tasks.workunit.client.1.smithi181.stdout:6/40: dread f0 [0,4194304] 0 2024-04-27T04:09:47.278 INFO:tasks.workunit.client.1.smithi181.stdout:3/104: dread - d0/de/d20/f21 zero size 2024-04-27T04:09:47.280 INFO:tasks.workunit.client.1.smithi181.stdout:8/114: rename d1/ff to d1/d20/f24 0 2024-04-27T04:09:47.284 INFO:tasks.workunit.client.1.smithi181.stdout:9/93: creat d0/f1e x:0 0 0 2024-04-27T04:09:47.291 INFO:tasks.workunit.client.1.smithi181.stdout:0/107: mknod dc/d1f/c2a 0 2024-04-27T04:09:47.291 INFO:tasks.workunit.client.1.smithi181.stdout:0/108: readlink l1 0 2024-04-27T04:09:47.293 INFO:tasks.workunit.client.1.smithi181.stdout:6/41: creat d7/fa x:0 0 0 2024-04-27T04:09:47.295 INFO:tasks.workunit.client.1.smithi181.stdout:6/42: chown d7/fa 58 1 2024-04-27T04:09:47.295 INFO:tasks.workunit.client.1.smithi181.stdout:2/106: dwrite d4/d9/f11 [0,4194304] 0 2024-04-27T04:09:47.296 INFO:tasks.workunit.client.1.smithi181.stdout:3/105: mknod d0/de/c24 0 2024-04-27T04:09:47.297 INFO:tasks.workunit.client.1.smithi181.stdout:2/107: chown d4/d9 6 1 2024-04-27T04:09:47.299 INFO:tasks.workunit.client.1.smithi181.stdout:2/108: read d4/d9/f1a [344066,97132] 0 2024-04-27T04:09:47.302 INFO:tasks.workunit.client.1.smithi181.stdout:0/109: dwrite dc/d1f/f23 [0,4194304] 0 2024-04-27T04:09:47.303 INFO:tasks.workunit.client.1.smithi181.stdout:2/109: dread - d4/d9/d16/f23 zero size 2024-04-27T04:09:47.305 INFO:tasks.workunit.client.1.smithi181.stdout:1/107: rename d1/f7 to d1/d2/d5/d6/f22 0 2024-04-27T04:09:47.306 INFO:tasks.workunit.client.1.smithi181.stdout:4/42: rename d0/d5/d7 to d0/d5/d7/db 22 2024-04-27T04:09:47.309 INFO:tasks.workunit.client.1.smithi181.stdout:3/106: dwrite d0/de/ff [0,4194304] 0 2024-04-27T04:09:47.315 INFO:tasks.workunit.client.1.smithi181.stdout:9/94: creat d0/f1f x:0 0 0 2024-04-27T04:09:47.317 INFO:tasks.workunit.client.1.smithi181.stdout:5/154: getdents d4/d18 0 2024-04-27T04:09:47.319 INFO:tasks.workunit.client.1.smithi181.stdout:4/43: dwrite d0/d5/da/f2 [0,4194304] 0 2024-04-27T04:09:47.326 INFO:tasks.workunit.client.1.smithi181.stdout:6/43: creat d7/fb x:0 0 0 2024-04-27T04:09:47.330 INFO:tasks.workunit.client.1.smithi181.stdout:4/44: dwrite d0/d5/f6 [0,4194304] 0 2024-04-27T04:09:47.334 INFO:tasks.workunit.client.1.smithi181.stdout:2/110: creat d4/d9/df/d14/f24 x:0 0 0 2024-04-27T04:09:47.334 INFO:tasks.workunit.client.1.smithi181.stdout:4/45: dread d0/d5/d7/f9 [0,4194304] 0 2024-04-27T04:09:47.339 INFO:tasks.workunit.client.1.smithi181.stdout:1/108: unlink d1/d2/d5/d1c/f17 0 2024-04-27T04:09:47.341 INFO:tasks.workunit.client.1.smithi181.stdout:3/107: mknod d0/de/d20/c25 0 2024-04-27T04:09:47.342 INFO:tasks.workunit.client.1.smithi181.stdout:7/52: link d1/la d1/lc 0 2024-04-27T04:09:47.345 INFO:tasks.workunit.client.1.smithi181.stdout:9/95: rename d0/l17 to d0/l20 0 2024-04-27T04:09:47.349 INFO:tasks.workunit.client.1.smithi181.stdout:5/155: unlink d4/c15 0 2024-04-27T04:09:47.354 INFO:tasks.workunit.client.1.smithi181.stdout:6/44: mkdir d7/dc 0 2024-04-27T04:09:47.354 INFO:tasks.workunit.client.1.smithi181.stdout:6/45: dread - d7/f8 zero size 2024-04-27T04:09:47.361 INFO:tasks.workunit.client.1.smithi181.stdout:4/46: creat d0/d5/da/fc x:0 0 0 2024-04-27T04:09:47.363 INFO:tasks.workunit.client.1.smithi181.stdout:1/109: mknod d1/d2/c23 0 2024-04-27T04:09:47.363 INFO:tasks.workunit.client.1.smithi181.stdout:1/110: fdatasync d1/d2/d5/d1c/f1f 0 2024-04-27T04:09:47.365 INFO:tasks.workunit.client.1.smithi181.stdout:1/111: truncate d1/d2/d5/d1c/f1f 786953 0 2024-04-27T04:09:47.366 INFO:tasks.workunit.client.1.smithi181.stdout:7/53: symlink d1/ld 0 2024-04-27T04:09:47.368 INFO:tasks.workunit.client.1.smithi181.stdout:5/156: mknod d4/d18/c37 0 2024-04-27T04:09:47.370 INFO:tasks.workunit.client.1.smithi181.stdout:8/115: truncate d1/f1f 521139 0 2024-04-27T04:09:47.376 INFO:tasks.workunit.client.1.smithi181.stdout:0/110: link dc/c1a dc/d1e/c2b 0 2024-04-27T04:09:47.383 INFO:tasks.workunit.client.1.smithi181.stdout:1/112: symlink d1/d2/d5/d6/db/l24 0 2024-04-27T04:09:47.385 INFO:tasks.workunit.client.1.smithi181.stdout:7/54: mkdir d1/de 0 2024-04-27T04:09:47.388 INFO:tasks.workunit.client.1.smithi181.stdout:7/55: write d1/f4 [3919569,42065] 0 2024-04-27T04:09:47.388 INFO:tasks.workunit.client.1.smithi181.stdout:4/47: dwrite d0/d5/d7/f9 [0,4194304] 0 2024-04-27T04:09:47.388 INFO:tasks.workunit.client.1.smithi181.stdout:5/157: unlink d4/d18/f24 0 2024-04-27T04:09:47.392 INFO:tasks.workunit.client.1.smithi181.stdout:8/116: mkdir d1/d6/d1a/d25 0 2024-04-27T04:09:47.393 INFO:tasks.workunit.client.1.smithi181.stdout:5/158: write d4/de/d2f/f31 [402255,27908] 0 2024-04-27T04:09:47.395 INFO:tasks.workunit.client.1.smithi181.stdout:6/46: link f3 d7/fd 0 2024-04-27T04:09:47.398 INFO:tasks.workunit.client.1.smithi181.stdout:1/113: dwrite d1/d2/d5/d1c/f1f [0,4194304] 0 2024-04-27T04:09:47.400 INFO:tasks.workunit.client.1.smithi181.stdout:5/159: write d4/d18/f23 [2203291,100368] 0 2024-04-27T04:09:47.400 INFO:tasks.workunit.client.1.smithi181.stdout:0/111: creat dc/f2c x:0 0 0 2024-04-27T04:09:47.403 INFO:tasks.workunit.client.1.smithi181.stdout:2/111: dwrite d4/f8 [0,4194304] 0 2024-04-27T04:09:47.406 INFO:tasks.workunit.client.1.smithi181.stdout:2/112: read - d4/d9/df/d14/f24 zero size 2024-04-27T04:09:47.406 INFO:tasks.workunit.client.1.smithi181.stdout:7/56: dwrite d1/f4 [0,4194304] 0 2024-04-27T04:09:47.407 INFO:tasks.workunit.client.1.smithi181.stdout:2/113: readlink d4/d9/df/l13 0 2024-04-27T04:09:47.417 INFO:tasks.workunit.client.1.smithi181.stdout:8/117: creat d1/d6/f26 x:0 0 0 2024-04-27T04:09:47.420 INFO:tasks.workunit.client.1.smithi181.stdout:6/47: creat d7/fe x:0 0 0 2024-04-27T04:09:47.420 INFO:tasks.workunit.client.1.smithi181.stdout:6/48: chown l4 0 1 2024-04-27T04:09:47.424 INFO:tasks.workunit.client.1.smithi181.stdout:0/112: rmdir dc/d1f 39 2024-04-27T04:09:47.426 INFO:tasks.workunit.client.1.smithi181.stdout:3/108: getdents d0 0 2024-04-27T04:09:47.560 INFO:tasks.workunit.client.1.smithi181.stdout:5/160: symlink d4/de/d2f/l38 0 2024-04-27T04:09:47.562 INFO:tasks.workunit.client.1.smithi181.stdout:6/49: dwrite d7/fe [0,4194304] 0 2024-04-27T04:09:47.562 INFO:tasks.workunit.client.1.smithi181.stdout:7/57: mkdir d1/df 0 2024-04-27T04:09:47.564 INFO:tasks.workunit.client.1.smithi181.stdout:8/118: creat d1/de/f27 x:0 0 0 2024-04-27T04:09:47.566 INFO:tasks.workunit.client.1.smithi181.stdout:8/119: write d1/de/f17 [1307066,120159] 0 2024-04-27T04:09:47.567 INFO:tasks.workunit.client.1.smithi181.stdout:3/109: fsync d0/f5 0 2024-04-27T04:09:47.568 INFO:tasks.workunit.client.1.smithi181.stdout:8/120: write d1/d6/f16 [4720780,88817] 0 2024-04-27T04:09:47.569 INFO:tasks.workunit.client.1.smithi181.stdout:8/121: write d1/de/f27 [723941,37232] 0 2024-04-27T04:09:47.570 INFO:tasks.workunit.client.1.smithi181.stdout:7/58: read d1/f8 [315388,26441] 0 2024-04-27T04:09:47.571 INFO:tasks.workunit.client.1.smithi181.stdout:8/122: dread - d1/d6/f26 zero size 2024-04-27T04:09:47.573 INFO:tasks.workunit.client.1.smithi181.stdout:7/59: write d1/f8 [124814,108490] 0 2024-04-27T04:09:47.575 INFO:tasks.workunit.client.1.smithi181.stdout:0/113: symlink dc/l2d 0 2024-04-27T04:09:47.577 INFO:tasks.workunit.client.1.smithi181.stdout:0/114: chown dc/l2d 255 1 2024-04-27T04:09:47.579 INFO:tasks.workunit.client.1.smithi181.stdout:8/123: dread d1/d20/f24 [4194304,4194304] 0 2024-04-27T04:09:47.580 INFO:tasks.workunit.client.1.smithi181.stdout:5/161: creat d4/d18/d2a/f39 x:0 0 0 2024-04-27T04:09:47.581 INFO:tasks.workunit.client.1.smithi181.stdout:8/124: fdatasync d1/d6/f15 0 2024-04-27T04:09:47.582 INFO:tasks.workunit.client.1.smithi181.stdout:0/115: creat dc/d1e/f2e x:0 0 0 2024-04-27T04:09:47.583 INFO:tasks.workunit.client.1.smithi181.stdout:2/114: getdents d4/d9/df/d14 0 2024-04-27T04:09:47.588 INFO:tasks.workunit.client.1.smithi181.stdout:5/162: creat d4/d18/f3a x:0 0 0 2024-04-27T04:09:47.589 INFO:tasks.workunit.client.1.smithi181.stdout:7/60: dread d1/f8 [0,4194304] 0 2024-04-27T04:09:47.590 INFO:tasks.workunit.client.1.smithi181.stdout:8/125: mknod d1/c28 0 2024-04-27T04:09:47.593 INFO:tasks.workunit.client.1.smithi181.stdout:2/115: dwrite d4/fb [0,4194304] 0 2024-04-27T04:09:47.593 INFO:tasks.workunit.client.1.smithi181.stdout:0/116: fdatasync dc/d1f/f29 0 2024-04-27T04:09:47.595 INFO:tasks.workunit.client.1.smithi181.stdout:2/116: read - d4/d9/df/d14/f24 zero size 2024-04-27T04:09:47.596 INFO:tasks.workunit.client.1.smithi181.stdout:5/163: dread d4/d18/f23 [0,4194304] 0 2024-04-27T04:09:47.597 INFO:tasks.workunit.client.1.smithi181.stdout:6/50: getdents d7 0 2024-04-27T04:09:47.599 INFO:tasks.workunit.client.1.smithi181.stdout:1/114: write f0 [508252,110164] 0 2024-04-27T04:09:47.599 INFO:tasks.workunit.client.1.smithi181.stdout:7/61: fsync d1/f8 0 2024-04-27T04:09:47.602 INFO:tasks.workunit.client.1.smithi181.stdout:7/62: write d1/f8 [866259,30660] 0 2024-04-27T04:09:47.603 INFO:tasks.workunit.client.1.smithi181.stdout:6/51: write d7/fe [4645488,60898] 0 2024-04-27T04:09:47.603 INFO:tasks.workunit.client.1.smithi181.stdout:1/115: fdatasync f0 0 2024-04-27T04:09:47.605 INFO:tasks.workunit.client.1.smithi181.stdout:8/126: rename d1/d6/f16 to d1/d6/f29 0 2024-04-27T04:09:47.606 INFO:tasks.workunit.client.1.smithi181.stdout:2/117: dwrite d4/f8 [0,4194304] 0 2024-04-27T04:09:47.608 INFO:tasks.workunit.client.1.smithi181.stdout:2/118: write d4/d9/df/d14/f24 [726537,12549] 0 2024-04-27T04:09:47.610 INFO:tasks.workunit.client.1.smithi181.stdout:0/117: mknod dc/d1e/c2f 0 2024-04-27T04:09:47.624 INFO:tasks.workunit.client.1.smithi181.stdout:8/127: mkdir d1/de/d2a 0 2024-04-27T04:09:47.628 INFO:tasks.workunit.client.1.smithi181.stdout:5/164: rename d4/de/d2f/l36 to d4/l3b 0 2024-04-27T04:09:47.630 INFO:tasks.workunit.client.1.smithi181.stdout:7/63: mkdir d1/df/d10 0 2024-04-27T04:09:47.634 INFO:tasks.workunit.client.1.smithi181.stdout:2/119: fdatasync f2 0 2024-04-27T04:09:47.637 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:09:47 smithi162 ceph-mon[26841]: from='client.14600 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-27T04:09:47.637 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:09:47 smithi162 ceph-mon[26841]: from='client.14604 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-27T04:09:47.637 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:09:47 smithi162 ceph-mon[26841]: from='client.? 172.21.15.162:0/4270726186' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-27T04:09:47.637 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:09:47 smithi162 ceph-mon[26841]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-27T04:09:47.639 INFO:tasks.workunit.client.1.smithi181.stdout:6/52: link f3 d7/dc/ff 0 2024-04-27T04:09:47.640 INFO:tasks.workunit.client.1.smithi181.stdout:6/53: write d7/fe [5355868,85767] 0 2024-04-27T04:09:47.641 INFO:tasks.workunit.client.1.smithi181.stdout:5/165: dread d4/de/f14 [0,4194304] 0 2024-04-27T04:09:47.645 INFO:tasks.workunit.client.1.smithi181.stdout:2/120: stat d4/d9/c19 0 2024-04-27T04:09:47.647 INFO:tasks.workunit.client.1.smithi181.stdout:7/64: rename d1/d2/lb to d1/l11 0 2024-04-27T04:09:47.649 INFO:tasks.workunit.client.1.smithi181.stdout:7/65: dread d1/f8 [0,4194304] 0 2024-04-27T04:09:47.651 INFO:tasks.workunit.client.1.smithi181.stdout:6/54: mknod d7/c10 0 2024-04-27T04:09:47.652 INFO:tasks.workunit.client.1.smithi181.stdout:1/116: getdents d1/d2/d5/d1c/d1e 0 2024-04-27T04:09:47.653 INFO:tasks.workunit.client.1.smithi181.stdout:1/117: rename d1/d2/d5/d1c to d1/d2/d5/d1c/d1e/d20/d25 22 2024-04-27T04:09:47.654 INFO:tasks.workunit.client.1.smithi181.stdout:5/166: creat d4/d10/d29/f3c x:0 0 0 2024-04-27T04:09:47.655 INFO:tasks.workunit.client.1.smithi181.stdout:8/128: link d1/d6/f1d d1/d6/d1a/f2b 0 2024-04-27T04:09:47.656 INFO:tasks.workunit.client.1.smithi181.stdout:5/167: fdatasync d4/d18/f3a 0 2024-04-27T04:09:47.656 INFO:tasks.workunit.client.1.smithi181.stdout:8/129: dread - d1/d6/f26 zero size 2024-04-27T04:09:47.658 INFO:tasks.workunit.client.1.smithi181.stdout:1/118: dread d1/d2/d5/d1c/f1f [0,4194304] 0 2024-04-27T04:09:47.658 INFO:tasks.workunit.client.1.smithi181.stdout:2/121: mknod d4/d9/d16/c25 0 2024-04-27T04:09:47.660 INFO:tasks.workunit.client.1.smithi181.stdout:7/66: unlink d1/d2/c7 0 2024-04-27T04:09:47.661 INFO:tasks.workunit.client.1.smithi181.stdout:7/67: stat d1/ld 0 2024-04-27T04:09:47.661 INFO:tasks.workunit.client.1.smithi181.stdout:1/119: dread d1/d2/f13 [0,4194304] 0 2024-04-27T04:09:47.663 INFO:tasks.workunit.client.1.smithi181.stdout:6/55: truncate d7/f9 934460 0 2024-04-27T04:09:47.665 INFO:tasks.workunit.client.1.smithi181.stdout:5/168: readlink d4/de/d2f/l20 0 2024-04-27T04:09:47.667 INFO:tasks.workunit.client.1.smithi181.stdout:8/130: symlink d1/d6/l2c 0 2024-04-27T04:09:47.672 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:09:47 smithi181 ceph-mon[33774]: from='client.14600 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-27T04:09:47.672 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:09:47 smithi181 ceph-mon[33774]: from='client.14604 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-27T04:09:47.672 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:09:47 smithi181 ceph-mon[33774]: from='client.? 172.21.15.162:0/4270726186' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-27T04:09:47.672 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:09:47 smithi181 ceph-mon[33774]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-27T04:09:47.673 INFO:tasks.workunit.client.1.smithi181.stdout:7/68: dread d1/f8 [0,4194304] 0 2024-04-27T04:09:47.769 INFO:tasks.workunit.client.1.smithi181.stdout:8/131: symlink d1/d6/d1a/l2d 0 2024-04-27T04:09:47.769 INFO:tasks.workunit.client.1.smithi181.stdout:8/132: read d1/d20/f24 [5039039,39934] 0 2024-04-27T04:09:47.773 INFO:tasks.workunit.client.1.smithi181.stdout:1/120: creat d1/d2/d5/d1c/d1e/d20/f26 x:0 0 0 2024-04-27T04:09:47.774 INFO:tasks.workunit.client.1.smithi181.stdout:1/121: chown d1/d2/d5/d1c/d1e/d20 4032250 1 2024-04-27T04:09:47.776 INFO:tasks.workunit.client.1.smithi181.stdout:7/69: creat d1/d2/f12 x:0 0 0 2024-04-27T04:09:47.782 INFO:tasks.workunit.client.1.smithi181.stdout:8/133: getdents d1/de/d2a 0 2024-04-27T04:09:47.790 INFO:tasks.workunit.client.1.smithi181.stdout:0/118: write f7 [1027548,114155] 0 2024-04-27T04:09:47.791 INFO:tasks.workunit.client.1.smithi181.stdout:5/169: rmdir d4/de/d2f 39 2024-04-27T04:09:47.793 INFO:tasks.workunit.client.1.smithi181.stdout:0/119: creat dc/d1e/f30 x:0 0 0 2024-04-27T04:09:47.793 INFO:tasks.workunit.client.1.smithi181.stdout:5/170: truncate d4/d10/d29/f3c 19515 0 2024-04-27T04:09:47.795 INFO:tasks.workunit.client.1.smithi181.stdout:5/171: chown d4/de/d2f/l20 56 1 2024-04-27T04:09:47.797 INFO:tasks.workunit.client.1.smithi181.stdout:5/172: truncate d4/d10/d29/f3c 546453 0 2024-04-27T04:09:47.797 INFO:tasks.workunit.client.1.smithi181.stdout:0/120: rename dc/l19 to dc/d1e/d26/l31 0 2024-04-27T04:09:47.798 INFO:tasks.workunit.client.1.smithi181.stdout:5/173: truncate d4/d18/d2a/f39 929834 0 2024-04-27T04:09:47.802 INFO:tasks.workunit.client.1.smithi181.stdout:5/174: mkdir d4/de/d2c/d2e/d3d 0 2024-04-27T04:09:47.805 INFO:tasks.workunit.client.1.smithi181.stdout:0/121: dwrite dc/f15 [0,4194304] 0 2024-04-27T04:09:47.807 INFO:tasks.workunit.client.1.smithi181.stdout:7/70: getdents d1 0 2024-04-27T04:09:47.813 INFO:tasks.workunit.client.1.smithi181.stdout:5/175: getdents d4/de/d2c/d2e/d3d 0 2024-04-27T04:09:47.818 INFO:tasks.workunit.client.1.smithi181.stdout:0/122: dwrite dc/d1e/f24 [0,4194304] 0 2024-04-27T04:09:47.822 INFO:tasks.workunit.client.1.smithi181.stdout:0/123: creat dc/d1f/f32 x:0 0 0 2024-04-27T04:09:47.831 INFO:tasks.workunit.client.1.smithi181.stdout:2/122: dwrite d4/f7 [0,4194304] 0 2024-04-27T04:09:47.836 INFO:tasks.workunit.client.1.smithi181.stdout:6/56: write d7/dc/ff [1875369,54126] 0 2024-04-27T04:09:47.839 INFO:tasks.workunit.client.1.smithi181.stdout:8/134: write d1/f1f [1002703,9409] 0 2024-04-27T04:09:47.840 INFO:tasks.workunit.client.1.smithi181.stdout:6/57: creat d7/f11 x:0 0 0 2024-04-27T04:09:47.841 INFO:tasks.workunit.client.1.smithi181.stdout:2/123: dwrite d4/d9/df/d14/f15 [0,4194304] 0 2024-04-27T04:09:47.842 INFO:tasks.workunit.client.1.smithi181.stdout:2/124: stat d4/d9/df/c1c 0 2024-04-27T04:09:47.843 INFO:tasks.workunit.client.1.smithi181.stdout:2/125: read d4/d9/df/d14/f15 [1064742,114634] 0 2024-04-27T04:09:47.844 INFO:tasks.workunit.client.1.smithi181.stdout:8/135: chown d1/d6/f1d 1677 1 2024-04-27T04:09:47.846 INFO:tasks.workunit.client.1.smithi181.stdout:6/58: rename d7/f11 to d7/dc/f12 0 2024-04-27T04:09:47.848 INFO:tasks.workunit.client.1.smithi181.stdout:0/124: rmdir dc/d1e 39 2024-04-27T04:09:47.856 INFO:tasks.workunit.client.1.smithi181.stdout:8/136: symlink d1/de/l2e 0 2024-04-27T04:09:47.859 INFO:tasks.workunit.client.1.smithi181.stdout:0/125: dread - dc/d1e/f2e zero size 2024-04-27T04:09:47.860 INFO:tasks.workunit.client.1.smithi181.stdout:8/137: mkdir d1/d6/d2f 0 2024-04-27T04:09:47.865 INFO:tasks.workunit.client.1.smithi181.stdout:8/138: dread d1/d6/f29 [4194304,4194304] 0 2024-04-27T04:09:47.865 INFO:tasks.workunit.client.1.smithi181.stdout:5/176: dwrite d4/de/f14 [0,4194304] 0 2024-04-27T04:09:47.866 INFO:tasks.workunit.client.1.smithi181.stdout:8/139: chown d1/c28 425 1 2024-04-27T04:09:47.870 INFO:tasks.workunit.client.1.smithi181.stdout:5/177: creat d4/de/d2c/f3e x:0 0 0 2024-04-27T04:09:47.871 INFO:tasks.workunit.client.1.smithi181.stdout:5/178: write d4/d10/d29/f1b [1915080,61621] 0 2024-04-27T04:09:47.873 INFO:tasks.workunit.client.1.smithi181.stdout:8/140: mkdir d1/d6/d2f/d30 0 2024-04-27T04:09:47.877 INFO:tasks.workunit.client.1.smithi181.stdout:8/141: mknod d1/c31 0 2024-04-27T04:09:47.877 INFO:tasks.workunit.client.1.smithi181.stdout:2/126: write d4/d9/f1a [247337,51647] 0 2024-04-27T04:09:47.878 INFO:tasks.workunit.client.1.smithi181.stdout:5/179: creat d4/de/d2c/d2e/d3d/f3f x:0 0 0 2024-04-27T04:09:47.880 INFO:tasks.workunit.client.1.smithi181.stdout:8/142: symlink d1/d6/l32 0 2024-04-27T04:09:47.885 INFO:tasks.workunit.client.1.smithi181.stdout:8/143: creat d1/f33 x:0 0 0 2024-04-27T04:09:47.886 INFO:tasks.workunit.client.1.smithi181.stdout:5/180: dread d4/d18/f23 [0,4194304] 0 2024-04-27T04:09:47.887 INFO:tasks.workunit.client.1.smithi181.stdout:8/144: truncate d1/de/f27 913923 0 2024-04-27T04:09:47.888 INFO:tasks.workunit.client.1.smithi181.stdout:5/181: unlink d4/d18/l26 0 2024-04-27T04:09:47.890 INFO:tasks.workunit.client.1.smithi181.stdout:5/182: dread - d4/d18/f3a zero size 2024-04-27T04:09:47.891 INFO:tasks.workunit.client.1.smithi181.stdout:0/126: dwrite fa [4194304,4194304] 0 2024-04-27T04:09:47.893 INFO:tasks.workunit.client.1.smithi181.stdout:0/127: dread - dc/d1e/f30 zero size 2024-04-27T04:09:47.896 INFO:tasks.workunit.client.1.smithi181.stdout:8/145: dread d1/d6/f15 [0,4194304] 0 2024-04-27T04:09:47.900 INFO:tasks.workunit.client.1.smithi181.stdout:2/127: write f2 [4429186,87873] 0 2024-04-27T04:09:47.902 INFO:tasks.workunit.client.1.smithi181.stdout:5/183: dwrite d4/de/f1e [0,4194304] 0 2024-04-27T04:09:47.902 INFO:tasks.workunit.client.1.smithi181.stdout:0/128: creat dc/d1f/f33 x:0 0 0 2024-04-27T04:09:47.908 INFO:tasks.workunit.client.1.smithi181.stdout:2/128: rename f2 to d4/d9/d16/f26 0 2024-04-27T04:09:47.908 INFO:tasks.workunit.client.1.smithi181.stdout:5/184: dread d4/f13 [0,4194304] 0 2024-04-27T04:09:47.910 INFO:tasks.workunit.client.1.smithi181.stdout:5/185: write d4/de/f14 [1359531,39257] 0 2024-04-27T04:09:47.912 INFO:tasks.workunit.client.1.smithi181.stdout:8/146: dwrite d1/d20/f24 [0,4194304] 0 2024-04-27T04:09:47.914 INFO:tasks.workunit.client.1.smithi181.stdout:2/129: getdents d4/d9/df/d14 0 2024-04-27T04:09:47.916 INFO:tasks.workunit.client.1.smithi181.stdout:8/147: creat d1/f34 x:0 0 0 2024-04-27T04:09:47.918 INFO:tasks.workunit.client.1.smithi181.stdout:8/148: creat d1/d6/d2f/f35 x:0 0 0 2024-04-27T04:09:47.920 INFO:tasks.workunit.client.1.smithi181.stdout:8/149: symlink d1/d20/l36 0 2024-04-27T04:09:47.920 INFO:tasks.workunit.client.1.smithi181.stdout:8/150: dread - d1/d6/d2f/f35 zero size 2024-04-27T04:09:47.921 INFO:tasks.workunit.client.1.smithi181.stdout:8/151: readlink d1/d6/l2c 0 2024-04-27T04:09:47.924 INFO:tasks.workunit.client.1.smithi181.stdout:8/152: creat d1/de/d2a/f37 x:0 0 0 2024-04-27T04:09:47.932 INFO:tasks.workunit.client.1.smithi181.stdout:5/186: truncate d4/d10/d29/f3c 281529 0 2024-04-27T04:09:47.943 INFO:tasks.workunit.client.1.smithi181.stdout:8/153: dwrite d1/d6/f1d [0,4194304] 0 2024-04-27T04:09:47.948 INFO:tasks.workunit.client.1.smithi181.stdout:8/154: rename d1/d6/f15 to d1/d6/d2f/d30/f38 0 2024-04-27T04:09:47.950 INFO:tasks.workunit.client.1.smithi181.stdout:8/155: unlink d1/d6/d1a/l2d 0 2024-04-27T04:09:47.952 INFO:tasks.workunit.client.1.smithi181.stdout:5/187: dwrite d4/d10/d29/f3c [0,4194304] 0 2024-04-27T04:09:47.959 INFO:tasks.workunit.client.1.smithi181.stdout:8/156: dwrite d1/de/f17 [0,4194304] 0 2024-04-27T04:09:47.967 INFO:tasks.workunit.client.1.smithi181.stdout:8/157: link d1/de/l2e d1/d6/d1a/l39 0 2024-04-27T04:09:47.968 INFO:tasks.workunit.client.1.smithi181.stdout:8/158: read d1/d6/f29 [3680180,56015] 0 2024-04-27T04:09:47.969 INFO:tasks.workunit.client.1.smithi181.stdout:5/188: truncate d4/d10/d29/f1b 2020508 0 2024-04-27T04:09:47.969 INFO:tasks.workunit.client.1.smithi181.stdout:5/189: fdatasync d4/de/d2c/f3e 0 2024-04-27T04:09:47.970 INFO:tasks.workunit.client.1.smithi181.stdout:8/159: creat d1/d6/d2f/f3a x:0 0 0 2024-04-27T04:09:47.971 INFO:tasks.workunit.client.1.smithi181.stdout:5/190: symlink d4/de/d2f/l40 0 2024-04-27T04:09:47.973 INFO:tasks.workunit.client.1.smithi181.stdout:8/160: dread d1/de/f18 [0,4194304] 0 2024-04-27T04:09:47.974 INFO:tasks.workunit.client.1.smithi181.stdout:8/161: chown d1/de/f27 29 1 2024-04-27T04:09:47.974 INFO:tasks.workunit.client.1.smithi181.stdout:5/191: creat d4/d18/f41 x:0 0 0 2024-04-27T04:09:47.977 INFO:tasks.workunit.client.1.smithi181.stdout:5/192: mknod d4/d10/d29/c42 0 2024-04-27T04:09:47.980 INFO:tasks.workunit.client.1.smithi181.stdout:8/162: dwrite d1/d6/f26 [0,4194304] 0 2024-04-27T04:09:47.984 INFO:tasks.workunit.client.1.smithi181.stdout:8/163: symlink d1/d6/d1a/l3b 0 2024-04-27T04:09:47.987 INFO:tasks.workunit.client.1.smithi181.stdout:8/164: write d1/d6/d2f/d30/f38 [1168369,52432] 0 2024-04-27T04:09:47.992 INFO:tasks.workunit.client.1.smithi181.stdout:8/165: dread d1/d6/f1d [0,4194304] 0 2024-04-27T04:09:47.992 INFO:tasks.workunit.client.1.smithi181.stdout:8/166: stat d1/cb 0 2024-04-27T04:09:47.996 INFO:tasks.workunit.client.1.smithi181.stdout:8/167: link d1/f33 d1/f3c 0 2024-04-27T04:09:47.999 INFO:tasks.workunit.client.1.smithi181.stdout:8/168: rename d1/c7 to d1/d6/d2f/c3d 0 2024-04-27T04:09:48.002 INFO:tasks.workunit.client.1.smithi181.stdout:8/169: rename d1/l8 to d1/d6/d1a/d25/l3e 0 2024-04-27T04:09:48.003 INFO:tasks.workunit.client.1.smithi181.stdout:8/170: fsync d1/de/f17 0 2024-04-27T04:09:48.003 INFO:tasks.workunit.client.1.smithi181.stdout:8/171: stat d1/f1f 0 2024-04-27T04:09:48.004 INFO:tasks.workunit.client.1.smithi181.stdout:8/172: truncate d1/d6/d2f/f35 792852 0 2024-04-27T04:09:48.007 INFO:tasks.workunit.client.1.smithi181.stdout:8/173: rename d1/d6/c1b to d1/de/c3f 0 2024-04-27T04:09:48.014 INFO:tasks.workunit.client.1.smithi181.stdout:8/174: dwrite d1/f1f [0,4194304] 0 2024-04-27T04:09:48.014 INFO:tasks.workunit.client.1.smithi181.stdout:8/175: stat d1/f3c 0 2024-04-27T04:09:48.024 INFO:tasks.workunit.client.1.smithi181.stdout:8/176: rename d1/de to d1/d6/d2f/d40 0 2024-04-27T04:09:48.025 INFO:tasks.workunit.client.1.smithi181.stdout:8/177: mknod d1/d6/d1a/c41 0 2024-04-27T04:09:48.029 INFO:tasks.workunit.client.1.smithi181.stdout:8/178: rename d1/d6/f1d to d1/d6/d2f/d40/d2a/f42 0 2024-04-27T04:09:48.033 INFO:tasks.workunit.client.1.smithi181.stdout:8/179: rmdir d1/d6/d2f/d40 39 2024-04-27T04:09:48.034 INFO:tasks.workunit.client.1.smithi181.stdout:8/180: dread - d1/f3c zero size 2024-04-27T04:09:48.038 INFO:tasks.workunit.client.1.smithi181.stdout:8/181: rename d1/f1f to d1/d6/d2f/d30/f43 0 2024-04-27T04:09:48.041 INFO:tasks.workunit.client.1.smithi181.stdout:8/182: fdatasync d1/d6/f29 0 2024-04-27T04:09:48.135 INFO:tasks.workunit.client.1.smithi181.stdout:9/96: sync 2024-04-27T04:09:48.137 INFO:tasks.workunit.client.1.smithi181.stdout:9/97: read d0/f6 [1499781,86244] 0 2024-04-27T04:09:48.187 INFO:tasks.workunit.client.1.smithi181.stdout:9/98: truncate d0/ff 3675900 0 2024-04-27T04:09:48.194 INFO:tasks.workunit.client.1.smithi181.stdout:9/99: dwrite d0/f8 [0,4194304] 0 2024-04-27T04:09:48.196 INFO:tasks.workunit.client.1.smithi181.stdout:9/100: chown d0/l20 1721879 1 2024-04-27T04:09:48.198 INFO:tasks.workunit.client.1.smithi181.stdout:9/101: symlink d0/l21 0 2024-04-27T04:09:48.199 INFO:tasks.workunit.client.1.smithi181.stdout:9/102: dread - d0/fc zero size 2024-04-27T04:09:48.201 INFO:tasks.workunit.client.1.smithi181.stdout:9/103: creat d0/d16/f22 x:0 0 0 2024-04-27T04:09:48.202 INFO:tasks.workunit.client.1.smithi181.stdout:9/104: creat d0/f23 x:0 0 0 2024-04-27T04:09:48.203 INFO:tasks.workunit.client.1.smithi181.stdout:9/105: truncate d0/d16/f22 666313 0 2024-04-27T04:09:48.206 INFO:tasks.workunit.client.1.smithi181.stdout:9/106: mknod d0/c24 0 2024-04-27T04:09:48.211 INFO:tasks.workunit.client.1.smithi181.stdout:9/107: rename d0/d16/f19 to d0/d16/f25 0 2024-04-27T04:09:48.215 INFO:tasks.workunit.client.1.smithi181.stdout:9/108: link d0/f1f d0/f26 0 2024-04-27T04:09:48.231 INFO:tasks.workunit.client.1.smithi181.stdout:9/109: fsync d0/f23 0 2024-04-27T04:09:48.236 INFO:tasks.workunit.client.1.smithi181.stdout:9/110: link d0/d16/f1c d0/f27 0 2024-04-27T04:09:48.237 INFO:tasks.workunit.client.1.smithi181.stdout:9/111: chown d0/l3 3771073 1 2024-04-27T04:09:48.244 INFO:tasks.workunit.client.1.smithi181.stdout:9/112: dread d0/f9 [0,4194304] 0 2024-04-27T04:09:48.279 INFO:tasks.workunit.client.1.smithi181.stdout:9/113: dwrite d0/f26 [0,4194304] 0 2024-04-27T04:09:48.279 INFO:tasks.workunit.client.1.smithi181.stdout:9/114: dread - d0/d16/f25 zero size 2024-04-27T04:09:48.283 INFO:tasks.workunit.client.1.smithi181.stdout:3/110: sync 2024-04-27T04:09:48.283 INFO:tasks.workunit.client.1.smithi181.stdout:4/48: sync 2024-04-27T04:09:48.284 INFO:tasks.workunit.client.1.smithi181.stdout:4/49: chown d0/d5/f6 226791 1 2024-04-27T04:09:48.285 INFO:tasks.workunit.client.1.smithi181.stdout:9/115: rmdir d0 39 2024-04-27T04:09:48.288 INFO:tasks.workunit.client.1.smithi181.stdout:3/111: mknod d0/c26 0 2024-04-27T04:09:48.290 INFO:tasks.workunit.client.1.smithi181.stdout:9/116: stat d0/l5 0 2024-04-27T04:09:48.294 INFO:tasks.workunit.client.1.smithi181.stdout:9/117: mknod d0/d16/c28 0 2024-04-27T04:09:48.300 INFO:tasks.workunit.client.1.smithi181.stdout:9/118: creat d0/d16/f29 x:0 0 0 2024-04-27T04:09:48.300 INFO:tasks.workunit.client.1.smithi181.stdout:9/119: fsync d0/f26 0 2024-04-27T04:09:48.311 INFO:tasks.workunit.client.1.smithi181.stdout:9/120: getdents d0 0 2024-04-27T04:09:48.314 INFO:tasks.workunit.client.1.smithi181.stdout:9/121: unlink d0/fd 0 2024-04-27T04:09:48.318 INFO:tasks.workunit.client.1.smithi181.stdout:9/122: rename d0/ff to d0/f2a 0 2024-04-27T04:09:48.321 INFO:tasks.workunit.client.1.smithi181.stdout:9/123: write d0/f6 [835192,30850] 0 2024-04-27T04:09:48.323 INFO:tasks.workunit.client.1.smithi181.stdout:9/124: truncate d0/d16/f25 655270 0 2024-04-27T04:09:48.326 INFO:tasks.workunit.client.1.smithi181.stdout:9/125: creat d0/f2b x:0 0 0 2024-04-27T04:09:48.328 INFO:tasks.workunit.client.1.smithi181.stdout:9/126: creat d0/d16/f2c x:0 0 0 2024-04-27T04:09:48.329 INFO:tasks.workunit.client.1.smithi181.stdout:9/127: chown d0/fc 376913 1 2024-04-27T04:09:48.332 INFO:tasks.workunit.client.1.smithi181.stdout:9/128: mknod d0/d16/c2d 0 2024-04-27T04:09:48.339 INFO:tasks.workunit.client.1.smithi181.stdout:9/129: rename d0/f6 to d0/f2e 0 2024-04-27T04:09:48.344 INFO:tasks.workunit.client.1.smithi181.stdout:4/50: truncate d0/d5/f6 1140685 0 2024-04-27T04:09:48.345 INFO:tasks.workunit.client.1.smithi181.stdout:3/112: dread d0/f14 [0,4194304] 0 2024-04-27T04:09:48.346 INFO:tasks.workunit.client.1.smithi181.stdout:9/130: symlink d0/d16/l2f 0 2024-04-27T04:09:48.349 INFO:tasks.workunit.client.1.smithi181.stdout:4/51: dread d0/d5/d7/f9 [0,4194304] 0 2024-04-27T04:09:48.350 INFO:tasks.workunit.client.1.smithi181.stdout:9/131: dread d0/f13 [0,4194304] 0 2024-04-27T04:09:48.352 INFO:tasks.workunit.client.1.smithi181.stdout:9/132: write d0/f13 [1379527,100405] 0 2024-04-27T04:09:48.360 INFO:tasks.workunit.client.1.smithi181.stdout:3/113: dwrite d0/f23 [0,4194304] 0 2024-04-27T04:09:48.372 INFO:tasks.workunit.client.1.smithi181.stdout:3/114: chown d0/de/d20/f21 0 1 2024-04-27T04:09:48.375 INFO:tasks.workunit.client.1.smithi181.stdout:3/115: mknod d0/de/c27 0 2024-04-27T04:09:48.380 INFO:tasks.workunit.client.1.smithi181.stdout:3/116: mkdir d0/d28 0 2024-04-27T04:09:48.385 INFO:tasks.workunit.client.1.smithi181.stdout:3/117: symlink d0/d28/l29 0 2024-04-27T04:09:48.395 INFO:tasks.workunit.client.1.smithi181.stdout:9/133: rename d0/d16 to d0/d30 0 2024-04-27T04:09:48.396 INFO:tasks.workunit.client.1.smithi181.stdout:9/134: truncate d0/f8 5194822 0 2024-04-27T04:09:48.398 INFO:tasks.workunit.client.1.smithi181.stdout:9/135: fdatasync d0/f9 0 2024-04-27T04:09:48.401 INFO:tasks.workunit.client.1.smithi181.stdout:9/136: mknod d0/c31 0 2024-04-27T04:09:48.410 INFO:tasks.workunit.client.1.smithi181.stdout:9/137: link d0/c31 d0/d30/c32 0 2024-04-27T04:09:48.411 INFO:tasks.workunit.client.1.smithi181.stdout:9/138: write d0/d30/f2c [46108,105703] 0 2024-04-27T04:09:48.415 INFO:tasks.workunit.client.1.smithi181.stdout:4/52: dwrite d0/d5/da/f4 [0,4194304] 0 2024-04-27T04:09:48.421 INFO:tasks.workunit.client.1.smithi181.stdout:4/53: mknod d0/d5/d7/cd 0 2024-04-27T04:09:48.425 INFO:tasks.workunit.client.1.smithi181.stdout:4/54: mkdir d0/d5/da/de 0 2024-04-27T04:09:48.427 INFO:tasks.workunit.client.1.smithi181.stdout:9/139: link d0/d30/c28 d0/d30/c33 0 2024-04-27T04:09:48.429 INFO:tasks.workunit.client.1.smithi181.stdout:4/55: creat d0/d5/d7/ff x:0 0 0 2024-04-27T04:09:48.434 INFO:tasks.workunit.client.1.smithi181.stdout:3/118: dwrite d0/f5 [0,4194304] 0 2024-04-27T04:09:48.435 INFO:tasks.workunit.client.1.smithi181.stdout:4/56: rename d0/d5/d7/f9 to d0/d5/d7/f10 0 2024-04-27T04:09:48.436 INFO:tasks.workunit.client.1.smithi181.stdout:9/140: dread d0/f2a [0,4194304] 0 2024-04-27T04:09:48.437 INFO:tasks.workunit.client.1.smithi181.stdout:9/141: readlink d0/d30/l18 0 2024-04-27T04:09:48.440 INFO:tasks.workunit.client.1.smithi181.stdout:3/119: mknod d0/d28/c2a 0 2024-04-27T04:09:48.440 INFO:tasks.workunit.client.1.smithi181.stdout:4/57: dread d0/d5/da/f2 [0,4194304] 0 2024-04-27T04:09:48.445 INFO:tasks.workunit.client.1.smithi181.stdout:9/142: unlink d0/c1a 0 2024-04-27T04:09:48.448 INFO:tasks.workunit.client.1.smithi181.stdout:3/120: write d0/de/d20/f21 [781477,76447] 0 2024-04-27T04:09:48.453 INFO:tasks.workunit.client.1.smithi181.stdout:9/143: mkdir d0/d30/d34 0 2024-04-27T04:09:48.455 INFO:tasks.workunit.client.1.smithi181.stdout:3/121: chown d0/f1e 1844 1 2024-04-27T04:09:48.459 INFO:tasks.workunit.client.1.smithi181.stdout:3/122: rename d0/d28/c2a to d0/de/d20/c2b 0 2024-04-27T04:09:48.462 INFO:tasks.workunit.client.1.smithi181.stdout:9/144: dwrite d0/f13 [0,4194304] 0 2024-04-27T04:09:48.463 INFO:tasks.workunit.client.1.smithi181.stdout:1/122: sync 2024-04-27T04:09:48.465 INFO:tasks.workunit.client.1.smithi181.stdout:9/145: read d0/f27 [2834762,31915] 0 2024-04-27T04:09:48.468 INFO:tasks.workunit.client.1.smithi181.stdout:3/123: dread d0/f23 [0,4194304] 0 2024-04-27T04:09:48.476 INFO:tasks.workunit.client.1.smithi181.stdout:3/124: readlink d0/l11 0 2024-04-27T04:09:48.481 INFO:tasks.workunit.client.1.smithi181.stdout:4/58: dread d0/d5/f6 [0,4194304] 0 2024-04-27T04:09:48.481 INFO:tasks.workunit.client.1.smithi181.stdout:3/125: fdatasync d0/f14 0 2024-04-27T04:09:48.484 INFO:tasks.workunit.client.1.smithi181.stdout:9/146: creat d0/d30/d34/f35 x:0 0 0 2024-04-27T04:09:48.488 INFO:tasks.workunit.client.1.smithi181.stdout:3/126: read d0/f15 [138228,58162] 0 2024-04-27T04:09:48.489 INFO:tasks.workunit.client.1.smithi181.stdout:6/59: sync 2024-04-27T04:09:48.489 INFO:tasks.workunit.client.1.smithi181.stdout:7/71: sync 2024-04-27T04:09:48.490 INFO:tasks.workunit.client.1.smithi181.stdout:4/59: dwrite d0/d5/da/f4 [4194304,4194304] 0 2024-04-27T04:09:48.491 INFO:tasks.workunit.client.1.smithi181.stdout:9/147: mknod d0/c36 0 2024-04-27T04:09:48.497 INFO:tasks.workunit.client.1.smithi181.stdout:3/127: chown d0/f19 1 1 2024-04-27T04:09:48.501 INFO:tasks.workunit.client.1.smithi181.stdout:4/60: symlink d0/d5/d7/l11 0 2024-04-27T04:09:48.502 INFO:tasks.workunit.client.1.smithi181.stdout:4/61: write d0/d5/da/f4 [6393089,3301] 0 2024-04-27T04:09:48.508 INFO:tasks.workunit.client.1.smithi181.stdout:3/128: mkdir d0/de/d2c 0 2024-04-27T04:09:48.516 INFO:tasks.workunit.client.1.smithi181.stdout:1/123: dwrite d1/d2/d5/fd [0,4194304] 0 2024-04-27T04:09:48.521 INFO:tasks.workunit.client.1.smithi181.stdout:4/62: dread d0/d5/da/f2 [0,4194304] 0 2024-04-27T04:09:48.522 INFO:tasks.workunit.client.1.smithi181.stdout:3/129: symlink d0/de/l2d 0 2024-04-27T04:09:48.528 INFO:tasks.workunit.client.1.smithi181.stdout:1/124: mknod d1/d2/d5/d1c/c27 0 2024-04-27T04:09:48.531 INFO:tasks.workunit.client.1.smithi181.stdout:4/63: fdatasync d0/d5/f6 0 2024-04-27T04:09:48.533 INFO:tasks.workunit.client.1.smithi181.stdout:4/64: write d0/d5/d7/ff [414143,16117] 0 2024-04-27T04:09:48.534 INFO:tasks.workunit.client.1.smithi181.stdout:3/130: rename d0/de/d20/c25 to d0/d28/c2e 0 2024-04-27T04:09:48.539 INFO:tasks.workunit.client.1.smithi181.stdout:9/148: dwrite d0/d30/f1c [0,4194304] 0 2024-04-27T04:09:48.540 INFO:tasks.workunit.client.1.smithi181.stdout:9/149: fsync d0/f8 0 2024-04-27T04:09:48.543 INFO:tasks.workunit.client.1.smithi181.stdout:9/150: dread d0/f14 [0,4194304] 0 2024-04-27T04:09:48.543 INFO:tasks.workunit.client.1.smithi181.stdout:4/65: symlink d0/d5/da/l12 0 2024-04-27T04:09:48.544 INFO:tasks.workunit.client.1.smithi181.stdout:4/66: write d0/d5/da/fc [59954,91520] 0 2024-04-27T04:09:48.546 INFO:tasks.workunit.client.1.smithi181.stdout:4/67: readlink d0/d5/d7/l11 0 2024-04-27T04:09:48.548 INFO:tasks.workunit.client.1.smithi181.stdout:1/125: dread d1/d2/d5/d1c/ff [0,4194304] 0 2024-04-27T04:09:48.549 INFO:tasks.workunit.client.1.smithi181.stdout:7/72: truncate d1/f4 799942 0 2024-04-27T04:09:48.567 INFO:tasks.workunit.client.1.smithi181.stdout:4/68: dwrite d0/d5/f6 [0,4194304] 0 2024-04-27T04:09:48.578 INFO:tasks.workunit.client.1.smithi181.stdout:3/131: rmdir d0/de/d2c 0 2024-04-27T04:09:48.583 INFO:tasks.workunit.client.1.smithi181.stdout:3/132: dread d0/de/ff [0,4194304] 0 2024-04-27T04:09:48.584 INFO:tasks.workunit.client.1.smithi181.stdout:7/73: mkdir d1/df/d10/d13 0 2024-04-27T04:09:48.584 INFO:tasks.workunit.client.1.smithi181.stdout:7/74: fsync d1/d2/f12 0 2024-04-27T04:09:48.585 INFO:tasks.workunit.client.1.smithi181.stdout:4/69: fdatasync d0/d5/da/f2 0 2024-04-27T04:09:48.587 INFO:tasks.workunit.client.1.smithi181.stdout:7/75: truncate d1/d2/f12 273541 0 2024-04-27T04:09:48.587 INFO:tasks.workunit.client.1.smithi181.stdout:1/126: write d1/f16 [1015388,47270] 0 2024-04-27T04:09:48.592 INFO:tasks.workunit.client.1.smithi181.stdout:9/151: getdents d0 0 2024-04-27T04:09:48.599 INFO:tasks.workunit.client.1.smithi181.stdout:9/152: rmdir d0/d30 39 2024-04-27T04:09:48.601 INFO:tasks.workunit.client.1.smithi181.stdout:4/70: symlink d0/d5/da/de/l13 0 2024-04-27T04:09:48.605 INFO:tasks.workunit.client.1.smithi181.stdout:7/76: rename d1/l11 to d1/l14 0 2024-04-27T04:09:48.607 INFO:tasks.workunit.client.1.smithi181.stdout:7/77: dread d1/f8 [0,4194304] 0 2024-04-27T04:09:48.609 INFO:tasks.workunit.client.1.smithi181.stdout:4/71: rmdir d0 39 2024-04-27T04:09:48.615 INFO:tasks.workunit.client.1.smithi181.stdout:3/133: truncate d0/de/d20/f21 293346 0 2024-04-27T04:09:48.619 INFO:tasks.workunit.client.1.smithi181.stdout:9/153: creat d0/f37 x:0 0 0 2024-04-27T04:09:48.619 INFO:tasks.workunit.client.1.smithi181.stdout:1/127: dwrite d1/d2/d5/d6/db/f19 [0,4194304] 0 2024-04-27T04:09:48.622 INFO:tasks.workunit.client.1.smithi181.stdout:9/154: chown d0/l5 934 1 2024-04-27T04:09:48.624 INFO:tasks.workunit.client.1.smithi181.stdout:3/134: rmdir d0/d28 39 2024-04-27T04:09:48.627 INFO:tasks.workunit.client.1.smithi181.stdout:4/72: dwrite d0/d5/da/f4 [0,4194304] 0 2024-04-27T04:09:48.629 INFO:tasks.workunit.client.1.smithi181.stdout:1/128: symlink d1/d2/d5/d1c/l28 0 2024-04-27T04:09:48.637 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:09:48 smithi162 ceph-mon[26841]: pgmap v259: 65 pgs: 65 active+clean; 291 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 1.3 MiB/s rd, 13 MiB/s wr, 272 op/s 2024-04-27T04:09:48.637 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:09:48 smithi162 ceph-mon[26841]: from='client.14612 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-27T04:09:48.637 INFO:tasks.workunit.client.1.smithi181.stdout:3/135: read d0/f15 [399809,6317] 0 2024-04-27T04:09:48.637 INFO:tasks.workunit.client.1.smithi181.stdout:1/129: symlink d1/d2/d5/d6/l29 0 2024-04-27T04:09:48.639 INFO:tasks.workunit.client.1.smithi181.stdout:1/130: fsync d1/d2/f4 0 2024-04-27T04:09:48.640 INFO:tasks.workunit.client.1.smithi181.stdout:4/73: dread d0/d5/d7/f10 [0,4194304] 0 2024-04-27T04:09:48.643 INFO:tasks.workunit.client.1.smithi181.stdout:1/131: symlink d1/d2/d5/d1c/d1e/l2a 0 2024-04-27T04:09:48.644 INFO:tasks.workunit.client.1.smithi181.stdout:3/136: creat d0/f2f x:0 0 0 2024-04-27T04:09:48.646 INFO:tasks.workunit.client.1.smithi181.stdout:2/130: sync 2024-04-27T04:09:48.646 INFO:tasks.workunit.client.1.smithi181.stdout:5/193: sync 2024-04-27T04:09:48.646 INFO:tasks.workunit.client.1.smithi181.stdout:8/183: sync 2024-04-27T04:09:48.646 INFO:tasks.workunit.client.1.smithi181.stdout:0/129: sync 2024-04-27T04:09:48.648 INFO:tasks.workunit.client.1.smithi181.stdout:4/74: rename d0/d5/da/fc to d0/d5/f14 0 2024-04-27T04:09:48.649 INFO:tasks.workunit.client.1.smithi181.stdout:7/78: write d1/f4 [1569974,68420] 0 2024-04-27T04:09:48.651 INFO:tasks.workunit.client.1.smithi181.stdout:4/75: write d0/d5/d7/ff [1060330,53187] 0 2024-04-27T04:09:48.652 INFO:tasks.workunit.client.1.smithi181.stdout:5/194: mknod d4/d10/d29/c43 0 2024-04-27T04:09:48.654 INFO:tasks.workunit.client.1.smithi181.stdout:2/131: symlink d4/d9/d16/l27 0 2024-04-27T04:09:48.655 INFO:tasks.workunit.client.1.smithi181.stdout:9/155: write d0/f11 [2973493,53082] 0 2024-04-27T04:09:48.658 INFO:tasks.workunit.client.1.smithi181.stdout:0/130: dread dc/fd [0,4194304] 0 2024-04-27T04:09:48.660 INFO:tasks.workunit.client.1.smithi181.stdout:0/131: write dc/f1d [121288,52357] 0 2024-04-27T04:09:48.662 INFO:tasks.workunit.client.1.smithi181.stdout:4/76: dwrite d0/d5/d7/ff [0,4194304] 0 2024-04-27T04:09:48.665 INFO:tasks.workunit.client.1.smithi181.stdout:4/77: stat d0/d5/da/f4 0 2024-04-27T04:09:48.665 INFO:tasks.workunit.client.1.smithi181.stdout:5/195: fsync d4/f13 0 2024-04-27T04:09:48.665 INFO:tasks.workunit.client.1.smithi181.stdout:2/132: dread d4/d9/df/d14/f24 [0,4194304] 0 2024-04-27T04:09:48.672 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:09:48 smithi181 ceph-mon[33774]: pgmap v259: 65 pgs: 65 active+clean; 291 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 1.3 MiB/s rd, 13 MiB/s wr, 272 op/s 2024-04-27T04:09:48.672 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:09:48 smithi181 ceph-mon[33774]: from='client.14612 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-27T04:09:48.673 INFO:tasks.workunit.client.1.smithi181.stdout:3/137: rename d0/f6 to d0/de/d20/f30 0 2024-04-27T04:09:48.677 INFO:tasks.workunit.client.1.smithi181.stdout:3/138: dread d0/f14 [0,4194304] 0 2024-04-27T04:09:48.682 INFO:tasks.workunit.client.1.smithi181.stdout:5/196: symlink d4/de/d2c/d2e/d3d/l44 0 2024-04-27T04:09:48.684 INFO:tasks.workunit.client.1.smithi181.stdout:1/132: write d1/d2/d5/d6/f22 [2510636,129117] 0 2024-04-27T04:09:48.686 INFO:tasks.workunit.client.1.smithi181.stdout:1/133: chown f0 7 1 2024-04-27T04:09:48.688 INFO:tasks.workunit.client.1.smithi181.stdout:5/197: dwrite d4/f13 [0,4194304] 0 2024-04-27T04:09:48.690 INFO:tasks.workunit.client.1.smithi181.stdout:7/79: rename d1/df/d10 to d1/de/d15 0 2024-04-27T04:09:48.692 INFO:tasks.workunit.client.1.smithi181.stdout:3/139: write d0/de/f10 [81988,14383] 0 2024-04-27T04:09:48.693 INFO:tasks.workunit.client.1.smithi181.stdout:3/140: truncate d0/f14 4338019 0 2024-04-27T04:09:48.696 INFO:tasks.workunit.client.1.smithi181.stdout:1/134: fsync d1/d2/d5/d1c/f1f 0 2024-04-27T04:09:48.700 INFO:tasks.workunit.client.1.smithi181.stdout:4/78: rename d0/d5/da to d0/d15 0 2024-04-27T04:09:48.703 INFO:tasks.workunit.client.1.smithi181.stdout:0/132: dwrite dc/d1e/f25 [0,4194304] 0 2024-04-27T04:09:48.703 INFO:tasks.workunit.client.1.smithi181.stdout:7/80: creat d1/de/f16 x:0 0 0 2024-04-27T04:09:48.713 INFO:tasks.workunit.client.1.smithi181.stdout:5/198: symlink d4/de/l45 0 2024-04-27T04:09:48.714 INFO:tasks.workunit.client.1.smithi181.stdout:5/199: write d4/de/d2c/f3e [37535,28002] 0 2024-04-27T04:09:48.715 INFO:tasks.workunit.client.1.smithi181.stdout:4/79: mknod d0/c16 0 2024-04-27T04:09:48.716 INFO:tasks.workunit.client.1.smithi181.stdout:2/133: getdents d4/d9/df 0 2024-04-27T04:09:48.716 INFO:tasks.workunit.client.1.smithi181.stdout:4/80: readlink d0/d15/de/l13 0 2024-04-27T04:09:48.717 INFO:tasks.workunit.client.1.smithi181.stdout:4/81: read d0/d15/f2 [2982753,119590] 0 2024-04-27T04:09:48.718 INFO:tasks.workunit.client.1.smithi181.stdout:0/133: unlink dc/f1c 0 2024-04-27T04:09:48.720 INFO:tasks.workunit.client.1.smithi181.stdout:5/200: dread d4/d18/f23 [0,4194304] 0 2024-04-27T04:09:48.720 INFO:tasks.workunit.client.1.smithi181.stdout:7/81: mknod d1/de/d15/c17 0 2024-04-27T04:09:48.722 INFO:tasks.workunit.client.1.smithi181.stdout:9/156: link d0/d30/c32 d0/d30/c38 0 2024-04-27T04:09:48.723 INFO:tasks.workunit.client.1.smithi181.stdout:9/157: chown d0/c24 13601 1 2024-04-27T04:09:48.724 INFO:tasks.workunit.client.1.smithi181.stdout:9/158: write d0/d30/f29 [794799,45966] 0 2024-04-27T04:09:48.725 INFO:tasks.workunit.client.1.smithi181.stdout:4/82: dwrite d0/d5/f6 [0,4194304] 0 2024-04-27T04:09:48.726 INFO:tasks.workunit.client.1.smithi181.stdout:9/159: readlink d0/d30/l18 0 2024-04-27T04:09:48.727 INFO:tasks.workunit.client.1.smithi181.stdout:2/134: rmdir d4 39 2024-04-27T04:09:48.729 INFO:tasks.workunit.client.1.smithi181.stdout:4/83: write d0/d5/f6 [2757583,82181] 0 2024-04-27T04:09:48.733 INFO:tasks.workunit.client.1.smithi181.stdout:0/134: mknod dc/d1e/c34 0 2024-04-27T04:09:48.734 INFO:tasks.workunit.client.1.smithi181.stdout:1/135: write d1/d2/f13 [1065760,19496] 0 2024-04-27T04:09:48.740 INFO:tasks.workunit.client.1.smithi181.stdout:5/201: mkdir d4/de/d2c/d46 0 2024-04-27T04:09:48.746 INFO:tasks.workunit.client.1.smithi181.stdout:5/202: write d4/de/d2c/f3e [64824,88963] 0 2024-04-27T04:09:48.747 INFO:tasks.workunit.client.1.smithi181.stdout:0/135: dwrite dc/d1f/f23 [0,4194304] 0 2024-04-27T04:09:48.747 INFO:tasks.workunit.client.1.smithi181.stdout:1/136: dwrite d1/d2/d5/d6/f1a [0,4194304] 0 2024-04-27T04:09:48.751 INFO:tasks.workunit.client.1.smithi181.stdout:4/84: unlink d0/d15/f4 0 2024-04-27T04:09:48.757 INFO:tasks.workunit.client.1.smithi181.stdout:3/141: truncate d0/f1 54178 0 2024-04-27T04:09:48.761 INFO:tasks.workunit.client.1.smithi181.stdout:9/160: truncate d0/f8 3886856 0 2024-04-27T04:09:48.762 INFO:tasks.workunit.client.1.smithi181.stdout:2/135: creat d4/f28 x:0 0 0 2024-04-27T04:09:48.764 INFO:tasks.workunit.client.1.smithi181.stdout:1/137: mkdir d1/d2/d5/d1c/d2b 0 2024-04-27T04:09:48.764 INFO:tasks.workunit.client.1.smithi181.stdout:7/82: dwrite d1/f8 [0,4194304] 0 2024-04-27T04:09:48.766 INFO:tasks.workunit.client.1.smithi181.stdout:2/136: write d4/fb [1712790,106683] 0 2024-04-27T04:09:48.767 INFO:tasks.workunit.client.1.smithi181.stdout:7/83: write d1/f8 [3558966,46740] 0 2024-04-27T04:09:48.768 INFO:tasks.workunit.client.1.smithi181.stdout:4/85: mknod d0/d15/c17 0 2024-04-27T04:09:48.771 INFO:tasks.workunit.client.1.smithi181.stdout:9/161: unlink d0/l5 0 2024-04-27T04:09:48.773 INFO:tasks.workunit.client.1.smithi181.stdout:9/162: read d0/f1f [3683210,23395] 0 2024-04-27T04:09:48.776 INFO:tasks.workunit.client.1.smithi181.stdout:1/138: dwrite d1/d2/d5/d1c/d1e/d20/f26 [0,4194304] 0 2024-04-27T04:09:48.778 INFO:tasks.workunit.client.1.smithi181.stdout:2/137: creat d4/d9/d16/f29 x:0 0 0 2024-04-27T04:09:48.780 INFO:tasks.workunit.client.1.smithi181.stdout:6/60: sync 2024-04-27T04:09:48.781 INFO:tasks.workunit.client.1.smithi181.stdout:7/84: creat d1/df/f18 x:0 0 0 2024-04-27T04:09:48.785 INFO:tasks.workunit.client.1.smithi181.stdout:7/85: chown d1/de/d15/d13 192 1 2024-04-27T04:09:48.786 INFO:tasks.workunit.client.1.smithi181.stdout:4/86: symlink d0/d15/l18 0 2024-04-27T04:09:48.788 INFO:tasks.workunit.client.1.smithi181.stdout:7/86: fsync d1/f8 0 2024-04-27T04:09:48.792 INFO:tasks.workunit.client.1.smithi181.stdout:7/87: write d1/df/f18 [327328,92360] 0 2024-04-27T04:09:48.792 INFO:tasks.workunit.client.1.smithi181.stdout:1/139: dwrite d1/f16 [0,4194304] 0 2024-04-27T04:09:48.794 INFO:tasks.workunit.client.1.smithi181.stdout:9/163: creat d0/d30/d34/f39 x:0 0 0 2024-04-27T04:09:48.796 INFO:tasks.workunit.client.1.smithi181.stdout:7/88: write d1/de/f16 [153881,89711] 0 2024-04-27T04:09:48.797 INFO:tasks.workunit.client.1.smithi181.stdout:3/142: dread d0/f1 [0,4194304] 0 2024-04-27T04:09:48.798 INFO:tasks.workunit.client.1.smithi181.stdout:2/138: creat d4/d9/d16/f2a x:0 0 0 2024-04-27T04:09:48.800 INFO:tasks.workunit.client.1.smithi181.stdout:2/139: fsync d4/f7 0 2024-04-27T04:09:48.801 INFO:tasks.workunit.client.1.smithi181.stdout:6/61: mknod d7/dc/c13 0 2024-04-27T04:09:48.804 INFO:tasks.workunit.client.1.smithi181.stdout:4/87: mkdir d0/d15/de/d19 0 2024-04-27T04:09:48.807 INFO:tasks.workunit.client.1.smithi181.stdout:0/136: getdents dc/d1e 0 2024-04-27T04:09:48.807 INFO:tasks.workunit.client.1.smithi181.stdout:4/88: chown d0/c16 1278 1 2024-04-27T04:09:48.810 INFO:tasks.workunit.client.1.smithi181.stdout:1/140: mkdir d1/d2/d5/d2c 0 2024-04-27T04:09:48.811 INFO:tasks.workunit.client.1.smithi181.stdout:9/164: unlink d0/f11 0 2024-04-27T04:09:48.816 INFO:tasks.workunit.client.1.smithi181.stdout:3/143: dwrite d0/de/f1f [0,4194304] 0 2024-04-27T04:09:48.819 INFO:tasks.workunit.client.1.smithi181.stdout:7/89: rename d1/de/f16 to d1/d2/f19 0 2024-04-27T04:09:48.820 INFO:tasks.workunit.client.1.smithi181.stdout:2/140: dwrite d4/d9/f1a [0,4194304] 0 2024-04-27T04:09:48.820 INFO:tasks.workunit.client.1.smithi181.stdout:6/62: dwrite d7/fe [4194304,4194304] 0 2024-04-27T04:09:48.834 INFO:tasks.workunit.client.1.smithi181.stdout:6/63: read f3 [2636549,26629] 0 2024-04-27T04:09:48.834 INFO:tasks.workunit.client.1.smithi181.stdout:1/141: unlink d1/d2/d5/fd 0 2024-04-27T04:09:48.834 INFO:tasks.workunit.client.1.smithi181.stdout:1/142: chown d1/d2/l10 0 1 2024-04-27T04:09:48.837 INFO:tasks.workunit.client.1.smithi181.stdout:4/89: truncate d0/d15/f2 1515320 0 2024-04-27T04:09:48.839 INFO:tasks.workunit.client.1.smithi181.stdout:3/144: mknod d0/de/d20/c31 0 2024-04-27T04:09:48.842 INFO:tasks.workunit.client.1.smithi181.stdout:1/143: dwrite d1/d2/d5/d6/db/f19 [0,4194304] 0 2024-04-27T04:09:48.845 INFO:tasks.workunit.client.1.smithi181.stdout:7/90: write d1/d2/f19 [819162,119477] 0 2024-04-27T04:09:48.848 INFO:tasks.workunit.client.1.smithi181.stdout:2/141: write d4/d9/d16/f26 [5038439,51322] 0 2024-04-27T04:09:48.851 INFO:tasks.workunit.client.1.smithi181.stdout:1/144: dwrite d1/d2/f4 [4194304,4194304] 0 2024-04-27T04:09:48.857 INFO:tasks.workunit.client.1.smithi181.stdout:6/64: mknod d7/dc/c14 0 2024-04-27T04:09:48.861 INFO:tasks.workunit.client.1.smithi181.stdout:2/142: dwrite d4/f7 [4194304,4194304] 0 2024-04-27T04:09:48.862 INFO:tasks.workunit.client.1.smithi181.stdout:9/165: rename d0/c31 to d0/d30/d34/c3a 0 2024-04-27T04:09:48.863 INFO:tasks.workunit.client.1.smithi181.stdout:2/143: readlink d4/d9/df/l22 0 2024-04-27T04:09:48.865 INFO:tasks.workunit.client.1.smithi181.stdout:0/137: creat dc/f35 x:0 0 0 2024-04-27T04:09:48.867 INFO:tasks.workunit.client.1.smithi181.stdout:1/145: mkdir d1/d2/d5/d6/db/d2d 0 2024-04-27T04:09:48.868 INFO:tasks.workunit.client.1.smithi181.stdout:4/90: getdents d0/d15/de/d19 0 2024-04-27T04:09:48.871 INFO:tasks.workunit.client.1.smithi181.stdout:9/166: symlink d0/d30/d34/l3b 0 2024-04-27T04:09:48.873 INFO:tasks.workunit.client.1.smithi181.stdout:2/144: creat d4/f2b x:0 0 0 2024-04-27T04:09:48.874 INFO:tasks.workunit.client.1.smithi181.stdout:9/167: write d0/f1e [668671,78958] 0 2024-04-27T04:09:48.879 INFO:tasks.workunit.client.1.smithi181.stdout:1/146: dwrite d1/d2/d5/d1c/d1e/d20/f26 [0,4194304] 0 2024-04-27T04:09:48.884 INFO:tasks.workunit.client.1.smithi181.stdout:1/147: chown d1/d2/d5/d6/db/l11 229984835 1 2024-04-27T04:09:48.884 INFO:tasks.workunit.client.1.smithi181.stdout:9/168: dread d0/f27 [0,4194304] 0 2024-04-27T04:09:48.884 INFO:tasks.workunit.client.1.smithi181.stdout:2/145: dread d4/d9/f1a [0,4194304] 0 2024-04-27T04:09:48.884 INFO:tasks.workunit.client.1.smithi181.stdout:1/148: write f0 [3163391,112377] 0 2024-04-27T04:09:48.888 INFO:tasks.workunit.client.1.smithi181.stdout:1/149: write d1/d2/d5/d6/db/f19 [575764,4177] 0 2024-04-27T04:09:48.892 INFO:tasks.workunit.client.1.smithi181.stdout:2/146: dwrite d4/f7 [0,4194304] 0 2024-04-27T04:09:48.897 INFO:tasks.workunit.client.1.smithi181.stdout:7/91: getdents d1/de/d15 0 2024-04-27T04:09:48.899 INFO:tasks.workunit.client.1.smithi181.stdout:1/150: fdatasync d1/d2/d5/d6/db/f19 0 2024-04-27T04:09:48.910 INFO:tasks.workunit.client.1.smithi181.stdout:9/169: dwrite d0/d30/f25 [0,4194304] 0 2024-04-27T04:09:48.915 INFO:tasks.workunit.client.1.smithi181.stdout:2/147: creat d4/f2c x:0 0 0 2024-04-27T04:09:48.916 INFO:tasks.workunit.client.1.smithi181.stdout:3/145: getdents d0/d28 0 2024-04-27T04:09:48.919 INFO:tasks.workunit.client.1.smithi181.stdout:6/65: getdents d7 0 2024-04-27T04:09:48.920 INFO:tasks.workunit.client.1.smithi181.stdout:7/92: mkdir d1/df/d1a 0 2024-04-27T04:09:48.921 INFO:tasks.workunit.client.1.smithi181.stdout:9/170: dwrite d0/d30/f25 [0,4194304] 0 2024-04-27T04:09:48.922 INFO:tasks.workunit.client.1.smithi181.stdout:9/171: chown d0/c24 7171833 1 2024-04-27T04:09:48.923 INFO:tasks.workunit.client.1.smithi181.stdout:1/151: rmdir d1/d2/d5/d1c/d1e/d20 39 2024-04-27T04:09:48.926 INFO:tasks.workunit.client.1.smithi181.stdout:0/138: getdents dc/d1f 0 2024-04-27T04:09:48.927 INFO:tasks.workunit.client.1.smithi181.stdout:4/91: truncate d0/d5/d7/ff 579656 0 2024-04-27T04:09:48.930 INFO:tasks.workunit.client.1.smithi181.stdout:4/92: dread d0/d5/f14 [0,4194304] 0 2024-04-27T04:09:48.937 INFO:tasks.workunit.client.1.smithi181.stdout:6/66: chown d7/f9 8084378 1 2024-04-27T04:09:48.938 INFO:tasks.workunit.client.1.smithi181.stdout:7/93: rmdir d1 39 2024-04-27T04:09:48.939 INFO:tasks.workunit.client.1.smithi181.stdout:2/148: rmdir d4 39 2024-04-27T04:09:48.940 INFO:tasks.workunit.client.1.smithi181.stdout:9/172: truncate d0/f2a 3988271 0 2024-04-27T04:09:48.940 INFO:tasks.workunit.client.1.smithi181.stdout:9/173: readlink d0/d30/l1b 0 2024-04-27T04:09:48.941 INFO:tasks.workunit.client.1.smithi181.stdout:1/152: symlink d1/d2/d5/d6/l2e 0 2024-04-27T04:09:48.943 INFO:tasks.workunit.client.1.smithi181.stdout:8/184: sync 2024-04-27T04:09:48.944 INFO:tasks.workunit.client.1.smithi181.stdout:0/139: readlink dc/d1e/d26/l31 0 2024-04-27T04:09:48.945 INFO:tasks.workunit.client.1.smithi181.stdout:8/185: write d1/d6/d2f/d30/f38 [3922323,86078] 0 2024-04-27T04:09:48.950 INFO:tasks.workunit.client.1.smithi181.stdout:6/67: write d7/dc/f12 [963944,82680] 0 2024-04-27T04:09:48.952 INFO:tasks.workunit.client.1.smithi181.stdout:6/68: fdatasync d7/dc/f12 0 2024-04-27T04:09:48.955 INFO:tasks.workunit.client.1.smithi181.stdout:1/153: rmdir d1/d2/d5/d1c 39 2024-04-27T04:09:48.956 INFO:tasks.workunit.client.1.smithi181.stdout:2/149: dread d4/d9/df/d14/f15 [0,4194304] 0 2024-04-27T04:09:48.957 INFO:tasks.workunit.client.1.smithi181.stdout:2/150: dread - d4/f28 zero size 2024-04-27T04:09:48.957 INFO:tasks.workunit.client.1.smithi181.stdout:0/140: chown dc/d1e/c2b 15066 1 2024-04-27T04:09:48.958 INFO:tasks.workunit.client.1.smithi181.stdout:8/186: creat d1/d6/f44 x:0 0 0 2024-04-27T04:09:48.962 INFO:tasks.workunit.client.1.smithi181.stdout:3/146: creat d0/f32 x:0 0 0 2024-04-27T04:09:48.963 INFO:tasks.workunit.client.1.smithi181.stdout:3/147: fdatasync d0/f2f 0 2024-04-27T04:09:48.966 INFO:tasks.workunit.client.1.smithi181.stdout:9/174: symlink d0/l3c 0 2024-04-27T04:09:48.967 INFO:tasks.workunit.client.1.smithi181.stdout:2/151: dwrite d4/f2c [0,4194304] 0 2024-04-27T04:09:48.968 INFO:tasks.workunit.client.1.smithi181.stdout:0/141: rmdir dc/d1e 39 2024-04-27T04:09:48.968 INFO:tasks.workunit.client.1.smithi181.stdout:9/175: truncate d0/f2e 2891603 0 2024-04-27T04:09:48.970 INFO:tasks.workunit.client.1.smithi181.stdout:8/187: dwrite d1/d6/f26 [0,4194304] 0 2024-04-27T04:09:48.972 INFO:tasks.workunit.client.1.smithi181.stdout:9/176: dread - d0/f15 zero size 2024-04-27T04:09:48.972 INFO:tasks.workunit.client.1.smithi181.stdout:0/142: chown dc/f15 1 1 2024-04-27T04:09:48.974 INFO:tasks.workunit.client.1.smithi181.stdout:7/94: mkdir d1/de/d15/d13/d1b 0 2024-04-27T04:09:48.976 INFO:tasks.workunit.client.1.smithi181.stdout:7/95: write d1/f8 [2360346,21782] 0 2024-04-27T04:09:48.976 INFO:tasks.workunit.client.1.smithi181.stdout:9/177: dread d0/d30/f22 [0,4194304] 0 2024-04-27T04:09:48.976 INFO:tasks.workunit.client.1.smithi181.stdout:3/148: symlink d0/d28/l33 0 2024-04-27T04:09:48.977 INFO:tasks.workunit.client.1.smithi181.stdout:1/154: rmdir d1/d2/d5/d1c 39 2024-04-27T04:09:48.980 INFO:tasks.workunit.client.1.smithi181.stdout:8/188: fdatasync d1/f3c 0 2024-04-27T04:09:48.980 INFO:tasks.workunit.client.1.smithi181.stdout:5/203: sync 2024-04-27T04:09:48.985 INFO:tasks.workunit.client.1.smithi181.stdout:0/143: unlink f7 0 2024-04-27T04:09:48.985 INFO:tasks.workunit.client.1.smithi181.stdout:0/144: dread - dc/f22 zero size 2024-04-27T04:09:48.989 INFO:tasks.workunit.client.1.smithi181.stdout:7/96: mkdir d1/d2/d1c 0 2024-04-27T04:09:48.990 INFO:tasks.workunit.client.1.smithi181.stdout:3/149: creat d0/de/f34 x:0 0 0 2024-04-27T04:09:48.998 INFO:tasks.workunit.client.1.smithi181.stdout:7/97: dwrite d1/d2/f12 [0,4194304] 0 2024-04-27T04:09:49.002 INFO:tasks.workunit.client.1.smithi181.stdout:6/69: dread d7/dc/ff [0,4194304] 0 2024-04-27T04:09:49.003 INFO:tasks.workunit.client.1.smithi181.stdout:6/70: chown d7/f8 32865 1 2024-04-27T04:09:49.008 INFO:tasks.workunit.client.1.smithi181.stdout:2/152: dwrite d4/d9/df/d14/f24 [0,4194304] 0 2024-04-27T04:09:49.009 INFO:tasks.workunit.client.1.smithi181.stdout:6/71: dread d7/f9 [0,4194304] 0 2024-04-27T04:09:49.010 INFO:tasks.workunit.client.1.smithi181.stdout:6/72: write f6 [683453,114715] 0 2024-04-27T04:09:49.013 INFO:tasks.workunit.client.1.smithi181.stdout:3/150: symlink d0/de/l35 0 2024-04-27T04:09:49.019 INFO:tasks.workunit.client.1.smithi181.stdout:8/189: truncate d1/f33 490938 0 2024-04-27T04:09:49.023 INFO:tasks.workunit.client.1.smithi181.stdout:0/145: creat dc/d1e/f36 x:0 0 0 2024-04-27T04:09:49.024 INFO:tasks.workunit.client.1.smithi181.stdout:7/98: chown d1/lc 6 1 2024-04-27T04:09:49.028 INFO:tasks.workunit.client.1.smithi181.stdout:9/178: dwrite d0/fc [0,4194304] 0 2024-04-27T04:09:49.030 INFO:tasks.workunit.client.1.smithi181.stdout:4/93: dwrite d0/d5/d7/ff [0,4194304] 0 2024-04-27T04:09:49.030 INFO:tasks.workunit.client.1.smithi181.stdout:7/99: write d1/d2/f12 [2095756,77073] 0 2024-04-27T04:09:49.034 INFO:tasks.workunit.client.1.smithi181.stdout:1/155: creat d1/f2f x:0 0 0 2024-04-27T04:09:49.036 INFO:tasks.workunit.client.1.smithi181.stdout:5/204: creat d4/de/f47 x:0 0 0 2024-04-27T04:09:49.037 INFO:tasks.workunit.client.1.smithi181.stdout:5/205: chown d4/de/d2c/d2e 0 1 2024-04-27T04:09:49.038 INFO:tasks.workunit.client.1.smithi181.stdout:0/146: dwrite dc/d1f/f32 [0,4194304] 0 2024-04-27T04:09:49.039 INFO:tasks.workunit.client.1.smithi181.stdout:0/147: chown dc/f22 675114 1 2024-04-27T04:09:49.050 INFO:tasks.workunit.client.1.smithi181.stdout:9/179: mkdir d0/d30/d34/d3d 0 2024-04-27T04:09:49.050 INFO:tasks.workunit.client.1.smithi181.stdout:7/100: mknod d1/c1d 0 2024-04-27T04:09:49.051 INFO:tasks.workunit.client.1.smithi181.stdout:6/73: dwrite d7/f9 [0,4194304] 0 2024-04-27T04:09:49.052 INFO:tasks.workunit.client.1.smithi181.stdout:9/180: write d0/f13 [4699880,67748] 0 2024-04-27T04:09:49.053 INFO:tasks.workunit.client.1.smithi181.stdout:9/181: stat d0/l1 0 2024-04-27T04:09:49.059 INFO:tasks.workunit.client.1.smithi181.stdout:5/206: creat d4/d18/f48 x:0 0 0 2024-04-27T04:09:49.059 INFO:tasks.workunit.client.1.smithi181.stdout:0/148: creat dc/d1f/f37 x:0 0 0 2024-04-27T04:09:49.060 INFO:tasks.workunit.client.1.smithi181.stdout:2/153: creat d4/d9/f2d x:0 0 0 2024-04-27T04:09:49.063 INFO:tasks.workunit.client.1.smithi181.stdout:1/156: rmdir d1 39 2024-04-27T04:09:49.071 INFO:tasks.workunit.client.1.smithi181.stdout:7/101: dread d1/f4 [0,4194304] 0 2024-04-27T04:09:49.073 INFO:tasks.workunit.client.1.smithi181.stdout:9/182: rename d0/f14 to d0/d30/d34/d3d/f3e 0 2024-04-27T04:09:49.074 INFO:tasks.workunit.client.1.smithi181.stdout:0/149: symlink dc/d1e/l38 0 2024-04-27T04:09:49.076 INFO:tasks.workunit.client.1.smithi181.stdout:5/207: chown d4/de/d2f/l20 15051 1 2024-04-27T04:09:49.077 INFO:tasks.workunit.client.1.smithi181.stdout:5/208: read - d4/de/d2c/d2e/d3d/f3f zero size 2024-04-27T04:09:49.079 INFO:tasks.workunit.client.1.smithi181.stdout:1/157: fdatasync d1/d2/f4 0 2024-04-27T04:09:49.079 INFO:tasks.workunit.client.1.smithi181.stdout:5/209: truncate d4/de/f47 613004 0 2024-04-27T04:09:49.084 INFO:tasks.workunit.client.1.smithi181.stdout:8/190: getdents d1/d20 0 2024-04-27T04:09:49.090 INFO:tasks.workunit.client.1.smithi181.stdout:2/154: write d4/f6 [356619,113080] 0 2024-04-27T04:09:49.091 INFO:tasks.workunit.client.1.smithi181.stdout:2/155: chown d4/f6 3009 1 2024-04-27T04:09:49.093 INFO:tasks.workunit.client.1.smithi181.stdout:4/94: link d0/d15/f2 d0/d15/f1a 0 2024-04-27T04:09:49.093 INFO:tasks.workunit.client.1.smithi181.stdout:6/74: link d7/c10 d7/dc/c15 0 2024-04-27T04:09:49.095 INFO:tasks.workunit.client.1.smithi181.stdout:7/102: creat d1/d2/d1c/f1e x:0 0 0 2024-04-27T04:09:49.098 INFO:tasks.workunit.client.1.smithi181.stdout:2/156: read d4/d9/f1a [831616,16824] 0 2024-04-27T04:09:49.102 INFO:tasks.workunit.client.1.smithi181.stdout:4/95: fsync d0/d5/d7/f10 0 2024-04-27T04:09:49.104 INFO:tasks.workunit.client.1.smithi181.stdout:4/96: chown d0/d15/l12 1 1 2024-04-27T04:09:49.109 INFO:tasks.workunit.client.1.smithi181.stdout:7/103: mkdir d1/de/d15/d1f 0 2024-04-27T04:09:49.110 INFO:tasks.workunit.client.1.smithi181.stdout:9/183: link d0/l20 d0/d30/l3f 0 2024-04-27T04:09:49.111 INFO:tasks.workunit.client.1.smithi181.stdout:9/184: chown d0/d30/f29 928 1 2024-04-27T04:09:49.111 INFO:tasks.workunit.client.1.smithi181.stdout:0/150: creat dc/f39 x:0 0 0 2024-04-27T04:09:49.115 INFO:tasks.workunit.client.1.smithi181.stdout:5/210: truncate d4/d18/d2a/f39 146639 0 2024-04-27T04:09:49.116 INFO:tasks.workunit.client.1.smithi181.stdout:2/157: mknod d4/d9/df/d14/c2e 0 2024-04-27T04:09:49.119 INFO:tasks.workunit.client.1.smithi181.stdout:5/211: chown d4/de/f14 179749 1 2024-04-27T04:09:49.119 INFO:tasks.workunit.client.1.smithi181.stdout:2/158: fdatasync d4/f28 0 2024-04-27T04:09:49.120 INFO:tasks.workunit.client.1.smithi181.stdout:6/75: write d7/fd [1298273,114421] 0 2024-04-27T04:09:49.124 INFO:tasks.workunit.client.1.smithi181.stdout:4/97: rmdir d0 39 2024-04-27T04:09:49.125 INFO:tasks.workunit.client.1.smithi181.stdout:2/159: write d4/d9/f11 [491600,88559] 0 2024-04-27T04:09:49.125 INFO:tasks.workunit.client.1.smithi181.stdout:7/104: symlink d1/df/l20 0 2024-04-27T04:09:49.126 INFO:tasks.workunit.client.1.smithi181.stdout:0/151: dwrite dc/d1f/f33 [0,4194304] 0 2024-04-27T04:09:49.128 INFO:tasks.workunit.client.1.smithi181.stdout:1/158: rename d1/d2/d5/d1c/c27 to d1/d2/d5/d1c/d1e/c30 0 2024-04-27T04:09:49.134 INFO:tasks.workunit.client.1.smithi181.stdout:0/152: dwrite dc/f15 [0,4194304] 0 2024-04-27T04:09:49.136 INFO:tasks.workunit.client.1.smithi181.stdout:0/153: chown dc/f11 2 1 2024-04-27T04:09:49.136 INFO:tasks.workunit.client.1.smithi181.stdout:8/191: link d1/d6/d2f/d40/f27 d1/d6/d2f/f45 0 2024-04-27T04:09:49.144 INFO:tasks.workunit.client.1.smithi181.stdout:0/154: dwrite dc/d1f/f29 [0,4194304] 0 2024-04-27T04:09:49.144 INFO:tasks.workunit.client.1.smithi181.stdout:6/76: symlink d7/dc/l16 0 2024-04-27T04:09:49.148 INFO:tasks.workunit.client.1.smithi181.stdout:6/77: chown d7/dc/c14 934815 1 2024-04-27T04:09:49.152 INFO:tasks.workunit.client.1.smithi181.stdout:5/212: rename d4/d10/c27 to d4/de/d2c/d2e/c49 0 2024-04-27T04:09:49.155 INFO:tasks.workunit.client.1.smithi181.stdout:8/192: unlink d1/c12 0 2024-04-27T04:09:49.156 INFO:tasks.workunit.client.1.smithi181.stdout:4/98: mknod d0/d15/c1b 0 2024-04-27T04:09:49.158 INFO:tasks.workunit.client.1.smithi181.stdout:5/213: dread d4/de/f47 [0,4194304] 0 2024-04-27T04:09:49.159 INFO:tasks.workunit.client.1.smithi181.stdout:6/78: dwrite d7/f8 [4194304,4194304] 0 2024-04-27T04:09:49.160 INFO:tasks.workunit.client.1.smithi181.stdout:9/185: rename d0/d30 to d0/d40 0 2024-04-27T04:09:49.162 INFO:tasks.workunit.client.1.smithi181.stdout:0/155: dread dc/d1e/f25 [0,4194304] 0 2024-04-27T04:09:49.165 INFO:tasks.workunit.client.1.smithi181.stdout:2/160: mkdir d4/d9/d2f 0 2024-04-27T04:09:49.165 INFO:tasks.workunit.client.1.smithi181.stdout:9/186: dread d0/f9 [0,4194304] 0 2024-04-27T04:09:49.167 INFO:tasks.workunit.client.1.smithi181.stdout:9/187: chown d0/f37 30 1 2024-04-27T04:09:49.167 INFO:tasks.workunit.client.1.smithi181.stdout:2/161: chown d4/d9/df/l13 5193 1 2024-04-27T04:09:49.167 INFO:tasks.workunit.client.1.smithi181.stdout:5/214: dread d4/d18/f23 [0,4194304] 0 2024-04-27T04:09:49.168 INFO:tasks.workunit.client.1.smithi181.stdout:2/162: stat d4/f2c 0 2024-04-27T04:09:49.168 INFO:tasks.workunit.client.1.smithi181.stdout:2/163: chown d4/f7 15729 1 2024-04-27T04:09:49.170 INFO:tasks.workunit.client.1.smithi181.stdout:2/164: dread - d4/d9/d16/f23 zero size 2024-04-27T04:09:49.170 INFO:tasks.workunit.client.1.smithi181.stdout:7/105: rename d1/f4 to d1/de/f21 0 2024-04-27T04:09:49.171 INFO:tasks.workunit.client.1.smithi181.stdout:2/165: stat d4/l1b 0 2024-04-27T04:09:49.171 INFO:tasks.workunit.client.1.smithi181.stdout:7/106: stat d1 0 2024-04-27T04:09:49.174 INFO:tasks.workunit.client.1.smithi181.stdout:9/188: dwrite d0/f2e [0,4194304] 0 2024-04-27T04:09:49.176 INFO:tasks.workunit.client.1.smithi181.stdout:8/193: creat d1/f46 x:0 0 0 2024-04-27T04:09:49.177 INFO:tasks.workunit.client.1.smithi181.stdout:8/194: fdatasync d1/f46 0 2024-04-27T04:09:49.181 INFO:tasks.workunit.client.1.smithi181.stdout:4/99: dread d0/d15/f1a [0,4194304] 0 2024-04-27T04:09:49.188 INFO:tasks.workunit.client.1.smithi181.stdout:4/100: dwrite d0/d5/d7/ff [0,4194304] 0 2024-04-27T04:09:49.194 INFO:tasks.workunit.client.1.smithi181.stdout:9/189: symlink d0/d40/d34/l41 0 2024-04-27T04:09:49.198 INFO:tasks.workunit.client.1.smithi181.stdout:7/107: dwrite d1/de/f21 [0,4194304] 0 2024-04-27T04:09:49.202 INFO:tasks.workunit.client.1.smithi181.stdout:1/159: truncate d1/d2/d5/d6/f1a 2129740 0 2024-04-27T04:09:49.206 INFO:tasks.workunit.client.1.smithi181.stdout:4/101: truncate d0/d15/f2 1797758 0 2024-04-27T04:09:49.207 INFO:tasks.workunit.client.1.smithi181.stdout:9/190: mknod d0/d40/d34/d3d/c42 0 2024-04-27T04:09:49.207 INFO:tasks.workunit.client.1.smithi181.stdout:7/108: dwrite d1/d2/f12 [0,4194304] 0 2024-04-27T04:09:49.208 INFO:tasks.workunit.client.1.smithi181.stdout:7/109: chown d1/d2 1932 1 2024-04-27T04:09:49.209 INFO:tasks.workunit.client.1.smithi181.stdout:7/110: chown d1/df/d1a 2937 1 2024-04-27T04:09:49.209 INFO:tasks.workunit.client.1.smithi181.stdout:9/191: write d0/fc [1347255,97004] 0 2024-04-27T04:09:49.210 INFO:tasks.workunit.client.1.smithi181.stdout:7/111: read - d1/d2/d1c/f1e zero size 2024-04-27T04:09:49.212 INFO:tasks.workunit.client.1.smithi181.stdout:8/195: unlink d1/d6/d1a/l1e 0 2024-04-27T04:09:49.213 INFO:tasks.workunit.client.1.smithi181.stdout:8/196: chown d1/cd 4 1 2024-04-27T04:09:49.218 INFO:tasks.workunit.client.1.smithi181.stdout:0/156: creat dc/f3a x:0 0 0 2024-04-27T04:09:49.218 INFO:tasks.workunit.client.1.smithi181.stdout:1/160: mknod d1/d2/d5/d6/c31 0 2024-04-27T04:09:49.220 INFO:tasks.workunit.client.1.smithi181.stdout:7/112: dwrite d1/de/f21 [0,4194304] 0 2024-04-27T04:09:49.222 INFO:tasks.workunit.client.1.smithi181.stdout:8/197: dread d1/d6/f29 [0,4194304] 0 2024-04-27T04:09:49.223 INFO:tasks.workunit.client.1.smithi181.stdout:4/102: fsync d0/d5/f14 0 2024-04-27T04:09:49.224 INFO:tasks.workunit.client.1.smithi181.stdout:8/198: write d1/d6/d2f/f3a [791467,106869] 0 2024-04-27T04:09:49.227 INFO:tasks.workunit.client.1.smithi181.stdout:1/161: dread d1/d2/d5/d1c/ff [0,4194304] 0 2024-04-27T04:09:49.236 INFO:tasks.workunit.client.1.smithi181.stdout:9/192: dwrite d0/f2e [4194304,4194304] 0 2024-04-27T04:09:49.238 INFO:tasks.workunit.client.1.smithi181.stdout:0/157: mkdir dc/d1f/d3b 0 2024-04-27T04:09:49.239 INFO:tasks.workunit.client.1.smithi181.stdout:7/113: fsync d1/de/f21 0 2024-04-27T04:09:49.241 INFO:tasks.workunit.client.1.smithi181.stdout:5/215: link d4/d18/d2a/l35 d4/de/d2c/d46/l4a 0 2024-04-27T04:09:49.243 INFO:tasks.workunit.client.1.smithi181.stdout:7/114: read d1/f8 [1300753,74382] 0 2024-04-27T04:09:49.247 INFO:tasks.workunit.client.1.smithi181.stdout:4/103: creat d0/d5/f1c x:0 0 0 2024-04-27T04:09:49.247 INFO:tasks.workunit.client.1.smithi181.stdout:9/193: dwrite d0/f15 [0,4194304] 0 2024-04-27T04:09:49.256 INFO:tasks.workunit.client.1.smithi181.stdout:4/104: dwrite d0/d5/f6 [0,4194304] 0 2024-04-27T04:09:49.258 INFO:tasks.workunit.client.1.smithi181.stdout:4/105: write d0/d5/d7/ff [2259599,43386] 0 2024-04-27T04:09:49.264 INFO:tasks.workunit.client.1.smithi181.stdout:7/115: rmdir d1/d2/d1c 39 2024-04-27T04:09:49.265 INFO:tasks.workunit.client.1.smithi181.stdout:7/116: write d1/de/f21 [2931291,61526] 0 2024-04-27T04:09:49.273 INFO:tasks.workunit.client.1.smithi181.stdout:7/117: dwrite d1/df/f18 [0,4194304] 0 2024-04-27T04:09:49.275 INFO:tasks.workunit.client.1.smithi181.stdout:1/162: symlink d1/d2/d5/d6/db/d2d/l32 0 2024-04-27T04:09:49.280 INFO:tasks.workunit.client.1.smithi181.stdout:0/158: creat dc/d1f/d3b/f3c x:0 0 0 2024-04-27T04:09:49.283 INFO:tasks.workunit.client.1.smithi181.stdout:9/194: dwrite d0/f15 [4194304,4194304] 0 2024-04-27T04:09:49.285 INFO:tasks.workunit.client.1.smithi181.stdout:4/106: symlink d0/d15/l1d 0 2024-04-27T04:09:49.289 INFO:tasks.workunit.client.1.smithi181.stdout:5/216: fsync d4/d18/d2a/f39 0 2024-04-27T04:09:49.292 INFO:tasks.workunit.client.1.smithi181.stdout:9/195: dwrite d0/d40/f25 [0,4194304] 0 2024-04-27T04:09:49.294 INFO:tasks.workunit.client.1.smithi181.stdout:1/163: unlink d1/d2/d5/d6/db/l24 0 2024-04-27T04:09:49.296 INFO:tasks.workunit.client.1.smithi181.stdout:0/159: rename dc/d1f/f32 to dc/d1f/f3d 0 2024-04-27T04:09:49.296 INFO:tasks.workunit.client.1.smithi181.stdout:1/164: chown d1/d2/c23 3746 1 2024-04-27T04:09:49.298 INFO:tasks.workunit.client.1.smithi181.stdout:5/217: dwrite d4/d18/f48 [0,4194304] 0 2024-04-27T04:09:49.300 INFO:tasks.workunit.client.1.smithi181.stdout:0/160: write dc/d1f/f29 [5233006,103459] 0 2024-04-27T04:09:49.302 INFO:tasks.workunit.client.1.smithi181.stdout:5/218: truncate d4/de/d2c/d2e/d3d/f3f 142128 0 2024-04-27T04:09:49.309 INFO:tasks.workunit.client.1.smithi181.stdout:0/161: dwrite dc/ff [0,4194304] 0 2024-04-27T04:09:49.311 INFO:tasks.workunit.client.1.smithi181.stdout:9/196: mkdir d0/d40/d34/d43 0 2024-04-27T04:09:49.311 INFO:tasks.workunit.client.1.smithi181.stdout:9/197: fsync d0/fc 0 2024-04-27T04:09:49.317 INFO:tasks.workunit.client.1.smithi181.stdout:4/107: rmdir d0/d15 39 2024-04-27T04:09:49.318 INFO:tasks.workunit.client.1.smithi181.stdout:4/108: readlink d0/d5/d7/l11 0 2024-04-27T04:09:49.318 INFO:tasks.workunit.client.1.smithi181.stdout:4/109: fsync d0/d5/f14 0 2024-04-27T04:09:49.326 INFO:tasks.workunit.client.1.smithi181.stdout:0/162: mknod dc/d1e/c3e 0 2024-04-27T04:09:49.328 INFO:tasks.workunit.client.1.smithi181.stdout:3/151: sync 2024-04-27T04:09:49.334 INFO:tasks.workunit.client.1.smithi181.stdout:0/163: dwrite dc/f39 [0,4194304] 0 2024-04-27T04:09:49.336 INFO:tasks.workunit.client.1.smithi181.stdout:0/164: truncate dc/f2c 923439 0 2024-04-27T04:09:49.344 INFO:tasks.workunit.client.1.smithi181.stdout:7/118: truncate d1/de/f21 689572 0 2024-04-27T04:09:49.346 INFO:tasks.workunit.client.1.smithi181.stdout:0/165: dread dc/d1e/d26/f28 [0,4194304] 0 2024-04-27T04:09:49.347 INFO:tasks.workunit.client.1.smithi181.stdout:3/152: mkdir d0/de/d36 0 2024-04-27T04:09:49.348 INFO:tasks.workunit.client.1.smithi181.stdout:3/153: dread - d0/f32 zero size 2024-04-27T04:09:49.352 INFO:tasks.workunit.client.1.smithi181.stdout:9/198: mkdir d0/d44 0 2024-04-27T04:09:49.354 INFO:tasks.workunit.client.1.smithi181.stdout:1/165: creat d1/f33 x:0 0 0 2024-04-27T04:09:49.358 INFO:tasks.workunit.client.1.smithi181.stdout:4/110: rename d0/d15/l18 to d0/d15/l1e 0 2024-04-27T04:09:49.367 INFO:tasks.workunit.client.1.smithi181.stdout:9/199: creat d0/d40/d34/d3d/f45 x:0 0 0 2024-04-27T04:09:49.370 INFO:tasks.workunit.client.1.smithi181.stdout:7/119: rename d1/de to d1/d22 0 2024-04-27T04:09:49.371 INFO:tasks.workunit.client.1.smithi181.stdout:9/200: dread d0/f2e [4194304,4194304] 0 2024-04-27T04:09:49.374 INFO:tasks.workunit.client.1.smithi181.stdout:4/111: truncate d0/d5/d7/f10 1690979 0 2024-04-27T04:09:49.375 INFO:tasks.workunit.client.1.smithi181.stdout:4/112: write d0/d5/f1c [1001500,60578] 0 2024-04-27T04:09:49.376 INFO:tasks.workunit.client.1.smithi181.stdout:5/219: link d4/de/d2c/d46/l4a d4/de/d2c/l4b 0 2024-04-27T04:09:49.382 INFO:tasks.workunit.client.1.smithi181.stdout:7/120: symlink d1/d22/l23 0 2024-04-27T04:09:49.389 INFO:tasks.workunit.client.1.smithi181.stdout:7/121: dwrite d1/d2/f19 [0,4194304] 0 2024-04-27T04:09:49.389 INFO:tasks.workunit.client.1.smithi181.stdout:9/201: rmdir d0/d40/d34/d3d 39 2024-04-27T04:09:49.390 INFO:tasks.workunit.client.1.smithi181.stdout:7/122: chown d1/d22/l23 90773 1 2024-04-27T04:09:49.392 INFO:tasks.workunit.client.1.smithi181.stdout:5/220: rmdir d4/d18 39 2024-04-27T04:09:49.399 INFO:tasks.workunit.client.1.smithi181.stdout:3/154: creat d0/f37 x:0 0 0 2024-04-27T04:09:49.409 INFO:tasks.workunit.client.1.smithi181.stdout:9/202: unlink d0/d40/d34/f35 0 2024-04-27T04:09:49.411 INFO:tasks.workunit.client.1.smithi181.stdout:9/203: read d0/f10 [88262,41438] 0 2024-04-27T04:09:49.417 INFO:tasks.workunit.client.1.smithi181.stdout:7/123: dread d1/f8 [0,4194304] 0 2024-04-27T04:09:49.418 INFO:tasks.workunit.client.1.smithi181.stdout:9/204: dwrite d0/f1f [0,4194304] 0 2024-04-27T04:09:49.421 INFO:tasks.workunit.client.1.smithi181.stdout:0/166: getdents dc/d1f 0 2024-04-27T04:09:49.424 INFO:tasks.workunit.client.1.smithi181.stdout:7/124: dread d1/f8 [0,4194304] 0 2024-04-27T04:09:49.427 INFO:tasks.workunit.client.1.smithi181.stdout:4/113: write d0/d15/f1a [1921478,82857] 0 2024-04-27T04:09:49.430 INFO:tasks.workunit.client.1.smithi181.stdout:4/114: write d0/d15/f2 [133508,68323] 0 2024-04-27T04:09:49.431 INFO:tasks.workunit.client.1.smithi181.stdout:4/115: truncate d0/d5/f1c 1405843 0 2024-04-27T04:09:49.433 INFO:tasks.workunit.client.1.smithi181.stdout:9/205: read d0/f27 [1036164,108119] 0 2024-04-27T04:09:49.433 INFO:tasks.workunit.client.1.smithi181.stdout:9/206: fdatasync d0/f2b 0 2024-04-27T04:09:49.435 INFO:tasks.workunit.client.1.smithi181.stdout:4/116: fsync d0/d15/f2 0 2024-04-27T04:09:49.436 INFO:tasks.workunit.client.1.smithi181.stdout:1/166: getdents d1/d2/d5/d1c/d1e/d20 0 2024-04-27T04:09:49.438 INFO:tasks.workunit.client.1.smithi181.stdout:0/167: creat dc/d1f/f3f x:0 0 0 2024-04-27T04:09:49.439 INFO:tasks.workunit.client.1.smithi181.stdout:0/168: write dc/ff [850409,118940] 0 2024-04-27T04:09:49.440 INFO:tasks.workunit.client.1.smithi181.stdout:3/155: write d0/f1 [160842,64952] 0 2024-04-27T04:09:49.440 INFO:tasks.workunit.client.1.smithi181.stdout:0/169: chown f8 127 1 2024-04-27T04:09:49.445 INFO:tasks.workunit.client.1.smithi181.stdout:9/207: rename d0/f27 to d0/d44/f46 0 2024-04-27T04:09:49.445 INFO:tasks.workunit.client.1.smithi181.stdout:9/208: stat d0/l3 0 2024-04-27T04:09:49.449 INFO:tasks.workunit.client.1.smithi181.stdout:3/156: dread d0/de/f17 [0,4194304] 0 2024-04-27T04:09:49.450 INFO:tasks.workunit.client.1.smithi181.stdout:0/170: mknod dc/d1f/c40 0 2024-04-27T04:09:49.450 INFO:tasks.workunit.client.1.smithi181.stdout:3/157: chown d0/f1e 0 1 2024-04-27T04:09:49.451 INFO:tasks.workunit.client.1.smithi181.stdout:0/171: write dc/d1f/f37 [544515,4074] 0 2024-04-27T04:09:49.451 INFO:tasks.workunit.client.1.smithi181.stdout:5/221: link d4/de/d2f/l40 d4/de/l4c 0 2024-04-27T04:09:49.458 INFO:tasks.workunit.client.1.smithi181.stdout:9/209: fsync d0/f9 0 2024-04-27T04:09:49.459 INFO:tasks.workunit.client.1.smithi181.stdout:9/210: stat d0/f13 0 2024-04-27T04:09:49.463 INFO:tasks.workunit.client.1.smithi181.stdout:0/172: mkdir dc/d1e/d26/d41 0 2024-04-27T04:09:49.465 INFO:tasks.workunit.client.1.smithi181.stdout:4/117: truncate d0/d15/f1a 284293 0 2024-04-27T04:09:49.466 INFO:tasks.workunit.client.1.smithi181.stdout:3/158: dwrite d0/f23 [0,4194304] 0 2024-04-27T04:09:49.466 INFO:tasks.workunit.client.1.smithi181.stdout:4/118: chown d0/d5/f14 30428564 1 2024-04-27T04:09:49.467 INFO:tasks.workunit.client.1.smithi181.stdout:3/159: fdatasync d0/f19 0 2024-04-27T04:09:49.468 INFO:tasks.workunit.client.1.smithi181.stdout:3/160: stat d0/f15 0 2024-04-27T04:09:49.475 INFO:tasks.workunit.client.1.smithi181.stdout:5/222: mkdir d4/de/d2c/d46/d4d 0 2024-04-27T04:09:49.483 INFO:tasks.workunit.client.1.smithi181.stdout:9/211: mknod d0/d40/d34/c47 0 2024-04-27T04:09:49.492 INFO:tasks.workunit.client.1.smithi181.stdout:0/173: write fb [4254097,87708] 0 2024-04-27T04:09:49.494 INFO:tasks.workunit.client.1.smithi181.stdout:3/161: unlink d0/l9 0 2024-04-27T04:09:49.496 INFO:tasks.workunit.client.1.smithi181.stdout:9/212: creat d0/d44/f48 x:0 0 0 2024-04-27T04:09:49.499 INFO:tasks.workunit.client.1.smithi181.stdout:5/223: symlink d4/d18/l4e 0 2024-04-27T04:09:49.500 INFO:tasks.workunit.client.1.smithi181.stdout:3/162: dwrite d0/f5 [0,4194304] 0 2024-04-27T04:09:49.505 INFO:tasks.workunit.client.1.smithi181.stdout:3/163: dread d0/de/d20/f30 [0,4194304] 0 2024-04-27T04:09:49.508 INFO:tasks.workunit.client.1.smithi181.stdout:9/213: creat d0/d44/f49 x:0 0 0 2024-04-27T04:09:49.509 INFO:tasks.workunit.client.1.smithi181.stdout:4/119: truncate d0/d5/d7/ff 3332272 0 2024-04-27T04:09:49.510 INFO:tasks.workunit.client.1.smithi181.stdout:0/174: dwrite dc/f16 [0,4194304] 0 2024-04-27T04:09:49.511 INFO:tasks.workunit.client.1.smithi181.stdout:5/224: mknod d4/d10/c4f 0 2024-04-27T04:09:49.511 INFO:tasks.workunit.client.1.smithi181.stdout:4/120: fsync d0/d5/f14 0 2024-04-27T04:09:49.512 INFO:tasks.workunit.client.1.smithi181.stdout:5/225: fdatasync d4/d18/f3a 0 2024-04-27T04:09:49.515 INFO:tasks.workunit.client.1.smithi181.stdout:9/214: creat d0/d44/f4a x:0 0 0 2024-04-27T04:09:49.521 INFO:tasks.workunit.client.1.smithi181.stdout:5/226: creat d4/d18/d2a/f50 x:0 0 0 2024-04-27T04:09:49.522 INFO:tasks.workunit.client.1.smithi181.stdout:0/175: dwrite dc/d1f/f3d [0,4194304] 0 2024-04-27T04:09:49.527 INFO:tasks.workunit.client.1.smithi181.stdout:9/215: rename d0/d40/d34 to d0/d44/d4b 0 2024-04-27T04:09:49.528 INFO:tasks.workunit.client.1.smithi181.stdout:9/216: readlink d0/d44/d4b/l3b 0 2024-04-27T04:09:49.529 INFO:tasks.workunit.client.1.smithi181.stdout:9/217: dread - d0/d44/f48 zero size 2024-04-27T04:09:49.529 INFO:tasks.workunit.client.1.smithi181.stdout:4/121: fsync d0/d15/f2 0 2024-04-27T04:09:49.532 INFO:tasks.workunit.client.1.smithi181.stdout:5/227: mknod d4/d18/c51 0 2024-04-27T04:09:49.534 INFO:tasks.workunit.client.1.smithi181.stdout:3/164: link d0/d28/l33 d0/de/d36/l38 0 2024-04-27T04:09:49.538 INFO:tasks.workunit.client.1.smithi181.stdout:9/218: dwrite d0/d44/f48 [0,4194304] 0 2024-04-27T04:09:49.543 INFO:tasks.workunit.client.1.smithi181.stdout:0/176: dread dc/d1e/d26/f28 [0,4194304] 0 2024-04-27T04:09:49.546 INFO:tasks.workunit.client.1.smithi181.stdout:5/228: creat d4/de/d2c/d46/d4d/f52 x:0 0 0 2024-04-27T04:09:49.547 INFO:tasks.workunit.client.1.smithi181.stdout:3/165: dwrite d0/f19 [0,4194304] 0 2024-04-27T04:09:49.549 INFO:tasks.workunit.client.1.smithi181.stdout:3/166: write d0/de/f1d [1006825,90241] 0 2024-04-27T04:09:49.562 INFO:tasks.workunit.client.1.smithi181.stdout:5/229: mknod d4/c53 0 2024-04-27T04:09:49.566 INFO:tasks.workunit.client.1.smithi181.stdout:3/167: symlink d0/de/d36/l39 0 2024-04-27T04:09:49.568 INFO:tasks.workunit.client.1.smithi181.stdout:0/177: creat dc/f42 x:0 0 0 2024-04-27T04:09:49.568 INFO:tasks.workunit.client.1.smithi181.stdout:0/178: chown dc/d1f/c2a 226586 1 2024-04-27T04:09:49.569 INFO:tasks.workunit.client.1.smithi181.stdout:0/179: chown dc/f18 571067 1 2024-04-27T04:09:49.570 INFO:tasks.workunit.client.1.smithi181.stdout:5/230: mkdir d4/d18/d54 0 2024-04-27T04:09:49.573 INFO:tasks.workunit.client.1.smithi181.stdout:9/219: getdents d0 0 2024-04-27T04:09:49.573 INFO:tasks.workunit.client.1.smithi181.stdout:9/220: fsync d0/f1f 0 2024-04-27T04:09:49.576 INFO:tasks.workunit.client.1.smithi181.stdout:0/180: symlink dc/d1e/d26/l43 0 2024-04-27T04:09:49.579 INFO:tasks.workunit.client.1.smithi181.stdout:5/231: rename d4/de/d2c/d2e/d3d to d4/de/d2c/d2e/d55 0 2024-04-27T04:09:49.579 INFO:tasks.workunit.client.1.smithi181.stdout:4/122: dwrite d0/d15/f1a [0,4194304] 0 2024-04-27T04:09:49.582 INFO:tasks.workunit.client.1.smithi181.stdout:4/123: write d0/d5/f6 [3666111,71424] 0 2024-04-27T04:09:49.585 INFO:tasks.workunit.client.1.smithi181.stdout:3/168: mknod d0/c3a 0 2024-04-27T04:09:49.588 INFO:tasks.workunit.client.1.smithi181.stdout:0/181: creat dc/d1f/d3b/f44 x:0 0 0 2024-04-27T04:09:49.594 INFO:tasks.workunit.client.1.smithi181.stdout:5/232: creat d4/de/d2c/d2e/f56 x:0 0 0 2024-04-27T04:09:49.596 INFO:tasks.workunit.client.1.smithi181.stdout:5/233: dread d4/d18/d2a/f39 [0,4194304] 0 2024-04-27T04:09:49.598 INFO:tasks.workunit.client.1.smithi181.stdout:9/221: symlink d0/d44/d4b/d43/l4c 0 2024-04-27T04:09:49.598 INFO:tasks.workunit.client.1.smithi181.stdout:9/222: dread - d0/d44/f4a zero size 2024-04-27T04:09:49.602 INFO:tasks.workunit.client.1.smithi181.stdout:0/182: mknod dc/d1e/c45 0 2024-04-27T04:09:49.608 INFO:tasks.workunit.client.1.smithi181.stdout:5/234: creat d4/d10/f57 x:0 0 0 2024-04-27T04:09:49.610 INFO:tasks.workunit.client.1.smithi181.stdout:9/223: mkdir d0/d4d 0 2024-04-27T04:09:49.618 INFO:tasks.workunit.client.1.smithi181.stdout:6/79: sync 2024-04-27T04:09:49.619 INFO:tasks.workunit.client.1.smithi181.stdout:6/80: read - d7/fb zero size 2024-04-27T04:09:49.619 INFO:tasks.workunit.client.1.smithi181.stdout:6/81: chown d7/dc/c13 162 1 2024-04-27T04:09:49.624 INFO:tasks.workunit.client.1.smithi181.stdout:0/183: unlink fb 0 2024-04-27T04:09:49.631 INFO:tasks.workunit.client.1.smithi181.stdout:0/184: dwrite dc/d1f/d3b/f44 [0,4194304] 0 2024-04-27T04:09:49.632 INFO:tasks.workunit.client.1.smithi181.stdout:0/185: fdatasync dc/f10 0 2024-04-27T04:09:49.633 INFO:tasks.workunit.client.1.smithi181.stdout:6/82: rename l4 to d7/dc/l17 0 2024-04-27T04:09:49.634 INFO:tasks.workunit.client.1.smithi181.stdout:6/83: truncate d7/dc/f12 1561533 0 2024-04-27T04:09:49.636 INFO:tasks.workunit.client.1.smithi181.stdout:5/235: creat d4/d18/d54/f58 x:0 0 0 2024-04-27T04:09:49.637 INFO:tasks.workunit.client.1.smithi181.stdout:5/236: write d4/d18/f41 [68532,30238] 0 2024-04-27T04:09:49.639 INFO:tasks.workunit.client.1.smithi181.stdout:5/237: write d4/d10/f57 [241878,5047] 0 2024-04-27T04:09:49.643 INFO:tasks.workunit.client.1.smithi181.stdout:5/238: dread d4/de/d2c/d2e/d55/f3f [0,4194304] 0 2024-04-27T04:09:49.645 INFO:tasks.workunit.client.1.smithi181.stdout:5/239: write d4/de/d2c/d46/d4d/f52 [870277,107377] 0 2024-04-27T04:09:49.651 INFO:tasks.workunit.client.1.smithi181.stdout:4/124: dwrite d0/d5/d7/f10 [0,4194304] 0 2024-04-27T04:09:49.651 INFO:tasks.workunit.client.1.smithi181.stdout:6/84: rename d7/c10 to d7/c18 0 2024-04-27T04:09:49.652 INFO:tasks.workunit.client.1.smithi181.stdout:6/85: fsync d7/f9 0 2024-04-27T04:09:49.655 INFO:tasks.workunit.client.1.smithi181.stdout:0/186: creat dc/f46 x:0 0 0 2024-04-27T04:09:49.655 INFO:tasks.workunit.client.1.smithi181.stdout:3/169: write d0/f4 [861860,56844] 0 2024-04-27T04:09:49.667 INFO:tasks.workunit.client.1.smithi181.stdout:6/86: creat d7/f19 x:0 0 0 2024-04-27T04:09:49.668 INFO:tasks.workunit.client.1.smithi181.stdout:6/87: chown c2 1 1 2024-04-27T04:09:49.674 INFO:tasks.workunit.client.1.smithi181.stdout:6/88: dwrite f6 [0,4194304] 0 2024-04-27T04:09:49.674 INFO:tasks.workunit.client.1.smithi181.stdout:5/240: fsync d4/d10/d29/f1b 0 2024-04-27T04:09:49.681 INFO:tasks.workunit.client.1.smithi181.stdout:6/89: dwrite d7/f19 [0,4194304] 0 2024-04-27T04:09:49.682 INFO:tasks.workunit.client.1.smithi181.stdout:6/90: chown f6 390543 1 2024-04-27T04:09:49.684 INFO:tasks.workunit.client.1.smithi181.stdout:9/224: dread d0/d40/f1c [0,4194304] 0 2024-04-27T04:09:49.691 INFO:tasks.workunit.client.1.smithi181.stdout:5/241: rename d4/de/d2f/f31 to d4/d10/f59 0 2024-04-27T04:09:49.694 INFO:tasks.workunit.client.1.smithi181.stdout:0/187: write dc/d1e/d26/f28 [1867786,86462] 0 2024-04-27T04:09:49.695 INFO:tasks.workunit.client.1.smithi181.stdout:0/188: read dc/f39 [3472736,5199] 0 2024-04-27T04:09:49.699 INFO:tasks.workunit.client.1.smithi181.stdout:9/225: chown d0/f1f 793 1 2024-04-27T04:09:49.704 INFO:tasks.workunit.client.1.smithi181.stdout:4/125: getdents d0 0 2024-04-27T04:09:49.705 INFO:tasks.workunit.client.1.smithi181.stdout:4/126: fsync d0/d5/f6 0 2024-04-27T04:09:49.708 INFO:tasks.workunit.client.1.smithi181.stdout:0/189: symlink dc/d1e/l47 0 2024-04-27T04:09:49.711 INFO:tasks.workunit.client.1.smithi181.stdout:6/91: rename d7/dc/c15 to d7/dc/c1a 0 2024-04-27T04:09:49.713 INFO:tasks.workunit.client.1.smithi181.stdout:0/190: dread dc/d1e/d26/f28 [0,4194304] 0 2024-04-27T04:09:49.716 INFO:tasks.workunit.client.1.smithi181.stdout:3/170: getdents d0 0 2024-04-27T04:09:49.718 INFO:tasks.workunit.client.1.smithi181.stdout:0/191: dread f8 [0,4194304] 0 2024-04-27T04:09:49.719 INFO:tasks.workunit.client.1.smithi181.stdout:5/242: symlink d4/de/l5a 0 2024-04-27T04:09:49.719 INFO:tasks.workunit.client.1.smithi181.stdout:0/192: dread - dc/f46 zero size 2024-04-27T04:09:49.721 INFO:tasks.workunit.client.1.smithi181.stdout:5/243: dread d4/d18/f41 [0,4194304] 0 2024-04-27T04:09:49.727 INFO:tasks.workunit.client.1.smithi181.stdout:6/92: mkdir d7/d1b 0 2024-04-27T04:09:49.728 INFO:tasks.workunit.client.1.smithi181.stdout:5/244: dwrite d4/d18/f48 [4194304,4194304] 0 2024-04-27T04:09:49.730 INFO:tasks.workunit.client.1.smithi181.stdout:9/226: symlink d0/d4d/l4e 0 2024-04-27T04:09:49.732 INFO:tasks.workunit.client.1.smithi181.stdout:3/171: mknod d0/de/d36/c3b 0 2024-04-27T04:09:49.736 INFO:tasks.workunit.client.1.smithi181.stdout:0/193: rename dc/d1f/f33 to dc/d1f/f48 0 2024-04-27T04:09:49.740 INFO:tasks.workunit.client.1.smithi181.stdout:6/93: mkdir d7/dc/d1c 0 2024-04-27T04:09:49.746 INFO:tasks.workunit.client.1.smithi181.stdout:3/172: dread d0/f15 [0,4194304] 0 2024-04-27T04:09:49.754 INFO:tasks.workunit.client.1.smithi181.stdout:6/94: dread f0 [0,4194304] 0 2024-04-27T04:09:49.757 INFO:tasks.workunit.client.1.smithi181.stdout:6/95: dread d7/dc/ff [0,4194304] 0 2024-04-27T04:09:49.758 INFO:tasks.workunit.client.1.smithi181.stdout:9/227: mkdir d0/d44/d4f 0 2024-04-27T04:09:49.758 INFO:tasks.workunit.client.1.smithi181.stdout:6/96: write d7/dc/f12 [511374,74384] 0 2024-04-27T04:09:49.759 INFO:tasks.workunit.client.1.smithi181.stdout:9/228: readlink d0/d44/d4b/d43/l4c 0 2024-04-27T04:09:49.762 INFO:tasks.workunit.client.1.smithi181.stdout:6/97: dread d7/fd [0,4194304] 0 2024-04-27T04:09:49.768 INFO:tasks.workunit.client.1.smithi181.stdout:6/98: dwrite f6 [0,4194304] 0 2024-04-27T04:09:49.861 INFO:tasks.workunit.client.1.smithi181.stdout:0/194: rename dc/c13 to dc/c49 0 2024-04-27T04:09:49.863 INFO:tasks.workunit.client.1.smithi181.stdout:5/245: link d4/de/l5a d4/de/l5b 0 2024-04-27T04:09:49.867 INFO:tasks.workunit.client.1.smithi181.stdout:0/195: dwrite dc/d1f/f3d [0,4194304] 0 2024-04-27T04:09:49.880 INFO:tasks.workunit.client.1.smithi181.stdout:2/166: sync 2024-04-27T04:09:49.881 INFO:tasks.workunit.client.1.smithi181.stdout:2/167: stat d4/d9/f2d 0 2024-04-27T04:09:49.883 INFO:tasks.workunit.client.1.smithi181.stdout:2/168: truncate d4/d9/d16/f23 926055 0 2024-04-27T04:09:49.884 INFO:tasks.workunit.client.1.smithi181.stdout:2/169: fsync d4/f28 0 2024-04-27T04:09:49.892 INFO:tasks.workunit.client.1.smithi181.stdout:2/170: dwrite d4/d9/d16/f26 [4194304,4194304] 0 2024-04-27T04:09:49.902 INFO:tasks.workunit.client.1.smithi181.stdout:9/229: mkdir d0/d44/d4b/d50 0 2024-04-27T04:09:49.907 INFO:tasks.workunit.client.1.smithi181.stdout:5/246: rename d4/d10/d29/f1b to d4/de/d2c/f5c 0 2024-04-27T04:09:49.910 INFO:tasks.workunit.client.1.smithi181.stdout:0/196: mkdir dc/d1f/d3b/d4a 0 2024-04-27T04:09:49.916 INFO:tasks.workunit.client.1.smithi181.stdout:6/99: mknod d7/d1b/c1d 0 2024-04-27T04:09:49.919 INFO:tasks.workunit.client.1.smithi181.stdout:9/230: creat d0/d44/f51 x:0 0 0 2024-04-27T04:09:49.921 INFO:tasks.workunit.client.1.smithi181.stdout:5/247: creat d4/de/d2f/f5d x:0 0 0 2024-04-27T04:09:49.923 INFO:tasks.workunit.client.1.smithi181.stdout:3/173: truncate d0/de/d20/f30 624227 0 2024-04-27T04:09:49.925 INFO:tasks.workunit.client.1.smithi181.stdout:6/100: mkdir d7/dc/d1e 0 2024-04-27T04:09:49.926 INFO:tasks.workunit.client.1.smithi181.stdout:6/101: truncate d7/fa 335355 0 2024-04-27T04:09:49.931 INFO:tasks.workunit.client.1.smithi181.stdout:0/197: symlink dc/l4b 0 2024-04-27T04:09:49.935 INFO:tasks.workunit.client.1.smithi181.stdout:0/198: dread dc/f16 [0,4194304] 0 2024-04-27T04:09:49.941 INFO:tasks.workunit.client.1.smithi181.stdout:9/231: mknod d0/d44/d4f/c52 0 2024-04-27T04:09:49.949 INFO:tasks.workunit.client.1.smithi181.stdout:2/171: rmdir d4/d9/df/d14 39 2024-04-27T04:09:49.951 INFO:tasks.workunit.client.1.smithi181.stdout:6/102: creat d7/dc/d1c/f1f x:0 0 0 2024-04-27T04:09:49.956 INFO:tasks.workunit.client.1.smithi181.stdout:5/248: link d4/de/d2f/l38 d4/d18/d54/l5e 0 2024-04-27T04:09:49.957 INFO:tasks.workunit.client.1.smithi181.stdout:5/249: chown d4/d18/f3a 12971 1 2024-04-27T04:09:49.960 INFO:tasks.workunit.client.1.smithi181.stdout:0/199: creat dc/d1e/d26/d41/f4c x:0 0 0 2024-04-27T04:09:49.964 INFO:tasks.workunit.client.1.smithi181.stdout:6/103: creat d7/dc/d1c/f20 x:0 0 0 2024-04-27T04:09:49.967 INFO:tasks.workunit.client.1.smithi181.stdout:9/232: fsync d0/d44/d4b/d3d/f3e 0 2024-04-27T04:09:49.968 INFO:tasks.workunit.client.1.smithi181.stdout:9/233: write d0/d44/f48 [4060849,55616] 0 2024-04-27T04:09:49.976 INFO:tasks.workunit.client.1.smithi181.stdout:6/104: mknod d7/c21 0 2024-04-27T04:09:49.982 INFO:tasks.workunit.client.1.smithi181.stdout:9/234: mknod d0/d4d/c53 0 2024-04-27T04:09:49.988 INFO:tasks.workunit.client.1.smithi181.stdout:3/174: link d0/de/f1d d0/f3c 0 2024-04-27T04:09:49.991 INFO:tasks.workunit.client.1.smithi181.stdout:8/199: sync 2024-04-27T04:09:49.991 INFO:tasks.workunit.client.1.smithi181.stdout:1/167: sync 2024-04-27T04:09:49.991 INFO:tasks.workunit.client.1.smithi181.stdout:4/127: sync 2024-04-27T04:09:49.992 INFO:tasks.workunit.client.1.smithi181.stdout:7/125: sync 2024-04-27T04:09:49.992 INFO:tasks.workunit.client.1.smithi181.stdout:4/128: chown d0/d5/f1c 11743045 1 2024-04-27T04:09:49.993 INFO:tasks.workunit.client.1.smithi181.stdout:6/105: symlink d7/dc/d1c/l22 0 2024-04-27T04:09:49.994 INFO:tasks.workunit.client.1.smithi181.stdout:6/106: write f6 [1436997,19902] 0 2024-04-27T04:09:49.996 INFO:tasks.workunit.client.1.smithi181.stdout:9/235: dread - d0/f37 zero size 2024-04-27T04:09:49.997 INFO:tasks.workunit.client.1.smithi181.stdout:8/200: dread d1/d6/d2f/d40/f27 [0,4194304] 0 2024-04-27T04:09:49.998 INFO:tasks.workunit.client.1.smithi181.stdout:8/201: truncate d1/d6/f44 32489 0 2024-04-27T04:09:49.999 INFO:tasks.workunit.client.1.smithi181.stdout:6/107: dread d7/f8 [4194304,4194304] 0 2024-04-27T04:09:50.010 INFO:tasks.workunit.client.1.smithi181.stdout:7/126: dwrite d1/d2/d1c/f1e [0,4194304] 0 2024-04-27T04:09:50.016 INFO:tasks.workunit.client.1.smithi181.stdout:9/236: stat d0/f2e 0 2024-04-27T04:09:50.021 INFO:tasks.workunit.client.1.smithi181.stdout:6/108: creat d7/dc/f23 x:0 0 0 2024-04-27T04:09:50.025 INFO:tasks.workunit.client.1.smithi181.stdout:0/200: getdents dc/d1e/d26/d41 0 2024-04-27T04:09:50.030 INFO:tasks.workunit.client.1.smithi181.stdout:7/127: rmdir d1 39 2024-04-27T04:09:50.031 INFO:tasks.workunit.client.1.smithi181.stdout:2/172: dwrite d4/d9/f1a [0,4194304] 0 2024-04-27T04:09:50.039 INFO:tasks.workunit.client.1.smithi181.stdout:8/202: mkdir d1/d6/d2f/d47 0 2024-04-27T04:09:50.042 INFO:tasks.workunit.client.1.smithi181.stdout:5/250: dwrite d4/d18/d2a/f39 [0,4194304] 0 2024-04-27T04:09:50.042 INFO:tasks.workunit.client.1.smithi181.stdout:2/173: dwrite d4/d9/d16/f26 [4194304,4194304] 0 2024-04-27T04:09:50.043 INFO:tasks.workunit.client.1.smithi181.stdout:1/168: rmdir d1/d2/d5/d1c/d2b 0 2024-04-27T04:09:50.045 INFO:tasks.workunit.client.1.smithi181.stdout:2/174: write d4/d9/f1a [1989532,50616] 0 2024-04-27T04:09:50.047 INFO:tasks.workunit.client.1.smithi181.stdout:0/201: truncate dc/d1e/d26/f28 3321270 0 2024-04-27T04:09:50.059 INFO:tasks.workunit.client.1.smithi181.stdout:9/237: fdatasync d0/f2a 0 2024-04-27T04:09:50.064 INFO:tasks.workunit.client.1.smithi181.stdout:8/203: creat d1/f48 x:0 0 0 2024-04-27T04:09:50.065 INFO:tasks.workunit.client.1.smithi181.stdout:8/204: write d1/d6/d2f/d30/f38 [3589980,105914] 0 2024-04-27T04:09:50.067 INFO:tasks.workunit.client.1.smithi181.stdout:5/251: rmdir d4/de/d2c/d2e/d55 39 2024-04-27T04:09:50.072 INFO:tasks.workunit.client.1.smithi181.stdout:2/175: chown d4/d9/df/d14 0 1 2024-04-27T04:09:50.084 INFO:tasks.workunit.client.1.smithi181.stdout:8/205: mknod d1/d6/d1a/d25/c49 0 2024-04-27T04:09:50.085 INFO:tasks.workunit.client.1.smithi181.stdout:5/252: chown d4/de/d2c/d2e/c49 130 1 2024-04-27T04:09:50.089 INFO:tasks.workunit.client.1.smithi181.stdout:2/176: creat d4/f30 x:0 0 0 2024-04-27T04:09:50.093 INFO:tasks.workunit.client.1.smithi181.stdout:5/253: dwrite d4/de/d2f/f5d [0,4194304] 0 2024-04-27T04:09:50.095 INFO:tasks.workunit.client.1.smithi181.stdout:3/175: creat d0/de/d36/f3d x:0 0 0 2024-04-27T04:09:50.095 INFO:tasks.workunit.client.1.smithi181.stdout:4/129: write d0/d5/d7/ff [2750050,118087] 0 2024-04-27T04:09:50.101 INFO:tasks.workunit.client.1.smithi181.stdout:6/109: link d7/dc/l17 d7/dc/d1e/l24 0 2024-04-27T04:09:50.103 INFO:tasks.workunit.client.1.smithi181.stdout:4/130: dread d0/d15/f1a [0,4194304] 0 2024-04-27T04:09:50.103 INFO:tasks.workunit.client.1.smithi181.stdout:8/206: creat d1/d6/f4a x:0 0 0 2024-04-27T04:09:50.104 INFO:tasks.workunit.client.1.smithi181.stdout:8/207: chown d1 4467661 1 2024-04-27T04:09:50.113 INFO:tasks.workunit.client.1.smithi181.stdout:5/254: mknod d4/d10/d29/c5f 0 2024-04-27T04:09:50.117 INFO:tasks.workunit.client.1.smithi181.stdout:1/169: getdents d1/d2/d5/d1c 0 2024-04-27T04:09:50.122 INFO:tasks.workunit.client.1.smithi181.stdout:4/131: rename d0/d5/d7/ff to d0/d5/f1f 0 2024-04-27T04:09:50.126 INFO:tasks.workunit.client.1.smithi181.stdout:2/177: symlink d4/d9/l31 0 2024-04-27T04:09:50.126 INFO:tasks.workunit.client.1.smithi181.stdout:9/238: dwrite d0/f2a [0,4194304] 0 2024-04-27T04:09:50.127 INFO:tasks.workunit.client.1.smithi181.stdout:5/255: creat d4/de/d2c/d46/f60 x:0 0 0 2024-04-27T04:09:50.130 INFO:tasks.workunit.client.1.smithi181.stdout:3/176: truncate d0/f4 536644 0 2024-04-27T04:09:50.135 INFO:tasks.workunit.client.1.smithi181.stdout:8/208: creat d1/d6/d2f/d47/f4b x:0 0 0 2024-04-27T04:09:50.136 INFO:tasks.workunit.client.1.smithi181.stdout:4/132: symlink d0/d5/l20 0 2024-04-27T04:09:50.137 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:09:49 smithi162 ceph-mon[26841]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' 2024-04-27T04:09:50.137 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:09:49 smithi162 ceph-mon[26841]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' 2024-04-27T04:09:50.137 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:09:49 smithi162 ceph-mon[26841]: pgmap v260: 65 pgs: 65 active+clean; 388 MiB data, 2.9 GiB used, 534 GiB / 536 GiB avail; 1.9 MiB/s rd, 22 MiB/s wr, 329 op/s 2024-04-27T04:09:50.138 INFO:tasks.workunit.client.1.smithi181.stdout:1/170: dread d1/d2/d5/d1c/f1f [0,4194304] 0 2024-04-27T04:09:50.141 INFO:tasks.workunit.client.1.smithi181.stdout:2/178: creat d4/f32 x:0 0 0 2024-04-27T04:09:50.143 INFO:tasks.workunit.client.1.smithi181.stdout:5/256: symlink d4/de/d2c/d46/d4d/l61 0 2024-04-27T04:09:50.144 INFO:tasks.workunit.client.1.smithi181.stdout:5/257: chown d4/d10/d29/c25 9511908 1 2024-04-27T04:09:50.146 INFO:tasks.workunit.client.1.smithi181.stdout:7/128: truncate d1/d2/d1c/f1e 3967927 0 2024-04-27T04:09:50.150 INFO:tasks.workunit.client.1.smithi181.stdout:5/258: dread d4/d10/f59 [0,4194304] 0 2024-04-27T04:09:50.154 INFO:tasks.workunit.client.1.smithi181.stdout:1/171: mknod d1/d2/d5/d6/c34 0 2024-04-27T04:09:50.155 INFO:tasks.workunit.client.1.smithi181.stdout:7/129: dwrite d1/d2/f12 [4194304,4194304] 0 2024-04-27T04:09:50.157 INFO:tasks.workunit.client.1.smithi181.stdout:2/179: rmdir d4/d9 39 2024-04-27T04:09:50.162 INFO:tasks.workunit.client.1.smithi181.stdout:3/177: mkdir d0/d3e 0 2024-04-27T04:09:50.172 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:09:49 smithi181 ceph-mon[33774]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' 2024-04-27T04:09:50.172 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:09:49 smithi181 ceph-mon[33774]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' 2024-04-27T04:09:50.172 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:09:49 smithi181 ceph-mon[33774]: pgmap v260: 65 pgs: 65 active+clean; 388 MiB data, 2.9 GiB used, 534 GiB / 536 GiB avail; 1.9 MiB/s rd, 22 MiB/s wr, 329 op/s 2024-04-27T04:09:50.180 INFO:tasks.workunit.client.1.smithi181.stdout:3/178: read d0/f14 [253883,91336] 0 2024-04-27T04:09:50.180 INFO:tasks.workunit.client.1.smithi181.stdout:5/259: symlink d4/de/d2c/d2e/l62 0 2024-04-27T04:09:50.181 INFO:tasks.workunit.client.1.smithi181.stdout:2/180: dwrite d4/d9/f11 [0,4194304] 0 2024-04-27T04:09:50.182 INFO:tasks.workunit.client.1.smithi181.stdout:2/181: write d4/f32 [172530,42527] 0 2024-04-27T04:09:50.184 INFO:tasks.workunit.client.1.smithi181.stdout:1/172: symlink d1/l35 0 2024-04-27T04:09:50.190 INFO:tasks.workunit.client.1.smithi181.stdout:9/239: write d0/f9 [4621850,56581] 0 2024-04-27T04:09:50.194 INFO:tasks.workunit.client.1.smithi181.stdout:2/182: dwrite d4/d9/d16/f26 [8388608,4194304] 0 2024-04-27T04:09:50.194 INFO:tasks.workunit.client.1.smithi181.stdout:9/240: truncate d0/d44/f4a 1027638 0 2024-04-27T04:09:50.198 INFO:tasks.workunit.client.1.smithi181.stdout:9/241: dread d0/d44/d4b/d3d/f3e [0,4194304] 0 2024-04-27T04:09:50.202 INFO:tasks.workunit.client.1.smithi181.stdout:8/209: truncate d1/d6/d2f/f3a 534504 0 2024-04-27T04:09:50.203 INFO:tasks.workunit.client.1.smithi181.stdout:9/242: dread d0/d44/f4a [0,4194304] 0 2024-04-27T04:09:50.203 INFO:tasks.workunit.client.1.smithi181.stdout:8/210: readlink d1/d6/l2c 0 2024-04-27T04:09:50.206 INFO:tasks.workunit.client.1.smithi181.stdout:4/133: truncate d0/d5/f1f 2450218 0 2024-04-27T04:09:50.207 INFO:tasks.workunit.client.1.smithi181.stdout:3/179: mkdir d0/de/d3f 0 2024-04-27T04:09:50.208 INFO:tasks.workunit.client.1.smithi181.stdout:0/202: sync 2024-04-27T04:09:50.209 INFO:tasks.workunit.client.1.smithi181.stdout:7/130: dread d1/d2/d1c/f1e [0,4194304] 0 2024-04-27T04:09:50.211 INFO:tasks.workunit.client.1.smithi181.stdout:1/173: dread d1/d2/f13 [0,4194304] 0 2024-04-27T04:09:50.220 INFO:tasks.workunit.client.1.smithi181.stdout:8/211: rename d1/d6/f29 to d1/d6/d2f/d40/f4c 0 2024-04-27T04:09:50.220 INFO:tasks.workunit.client.1.smithi181.stdout:8/212: chown d1/d6/d1a/l3b 893277 1 2024-04-27T04:09:50.228 INFO:tasks.workunit.client.1.smithi181.stdout:0/203: creat dc/d1e/f4d x:0 0 0 2024-04-27T04:09:50.229 INFO:tasks.workunit.client.1.smithi181.stdout:5/260: dwrite d4/d18/f41 [0,4194304] 0 2024-04-27T04:09:50.230 INFO:tasks.workunit.client.1.smithi181.stdout:4/134: dread d0/d15/f2 [0,4194304] 0 2024-04-27T04:09:50.231 INFO:tasks.workunit.client.1.smithi181.stdout:2/183: symlink d4/d9/l33 0 2024-04-27T04:09:50.236 INFO:tasks.workunit.client.1.smithi181.stdout:1/174: rmdir d1/d2 39 2024-04-27T04:09:50.238 INFO:tasks.workunit.client.1.smithi181.stdout:4/135: dread d0/d15/f2 [0,4194304] 0 2024-04-27T04:09:50.238 INFO:tasks.workunit.client.1.smithi181.stdout:9/243: write d0/d40/f1c [4695762,61072] 0 2024-04-27T04:09:50.243 INFO:tasks.workunit.client.1.smithi181.stdout:3/180: mkdir d0/d28/d40 0 2024-04-27T04:09:50.247 INFO:tasks.workunit.client.1.smithi181.stdout:8/213: rename d1/d6/d2f/f35 to d1/d6/d1a/f4d 0 2024-04-27T04:09:50.254 INFO:tasks.workunit.client.1.smithi181.stdout:5/261: rmdir d4/d18/d54 39 2024-04-27T04:09:50.261 INFO:tasks.workunit.client.1.smithi181.stdout:4/136: symlink d0/d15/l21 0 2024-04-27T04:09:50.261 INFO:tasks.workunit.client.1.smithi181.stdout:5/262: dwrite d4/de/d2c/d46/f60 [0,4194304] 0 2024-04-27T04:09:50.266 INFO:tasks.workunit.client.1.smithi181.stdout:9/244: creat d0/d44/d4f/f54 x:0 0 0 2024-04-27T04:09:50.266 INFO:tasks.workunit.client.1.smithi181.stdout:9/245: readlink d0/d40/l1d 0 2024-04-27T04:09:50.279 INFO:tasks.workunit.client.1.smithi181.stdout:7/131: link d1/d2/f19 d1/d2/f24 0 2024-04-27T04:09:50.282 INFO:tasks.workunit.client.1.smithi181.stdout:0/204: link dc/ff dc/d1e/d26/d41/f4e 0 2024-04-27T04:09:50.286 INFO:tasks.workunit.client.1.smithi181.stdout:1/175: mkdir d1/d2/d5/d6/db/d2d/d36 0 2024-04-27T04:09:50.286 INFO:tasks.workunit.client.1.smithi181.stdout:1/176: stat d1/d2/l10 0 2024-04-27T04:09:50.290 INFO:tasks.workunit.client.1.smithi181.stdout:4/137: rmdir d0/d5 39 2024-04-27T04:09:50.292 INFO:tasks.workunit.client.1.smithi181.stdout:5/263: mknod d4/de/d2f/c63 0 2024-04-27T04:09:50.294 INFO:tasks.workunit.client.1.smithi181.stdout:3/181: getdents d0/d28/d40 0 2024-04-27T04:09:50.299 INFO:tasks.workunit.client.1.smithi181.stdout:3/182: dread d0/f3c [0,4194304] 0 2024-04-27T04:09:50.299 INFO:tasks.workunit.client.1.smithi181.stdout:7/132: rmdir d1/d22 39 2024-04-27T04:09:50.302 INFO:tasks.workunit.client.1.smithi181.stdout:0/205: creat dc/d1e/d26/d41/f4f x:0 0 0 2024-04-27T04:09:50.308 INFO:tasks.workunit.client.1.smithi181.stdout:5/264: symlink d4/d18/d2a/l64 0 2024-04-27T04:09:50.314 INFO:tasks.workunit.client.1.smithi181.stdout:8/214: dwrite d1/f3c [0,4194304] 0 2024-04-27T04:09:50.319 INFO:tasks.workunit.client.1.smithi181.stdout:1/177: dwrite d1/d2/d5/d1c/f1f [0,4194304] 0 2024-04-27T04:09:50.319 INFO:tasks.workunit.client.1.smithi181.stdout:1/178: stat d1/d2 0 2024-04-27T04:09:50.320 INFO:tasks.workunit.client.1.smithi181.stdout:1/179: fsync d1/d2/f4 0 2024-04-27T04:09:50.321 INFO:tasks.workunit.client.1.smithi181.stdout:0/206: rename dc/d1f/d3b/f3c to dc/d1f/d3b/f50 0 2024-04-27T04:09:50.322 INFO:tasks.workunit.client.1.smithi181.stdout:0/207: dread - dc/f3a zero size 2024-04-27T04:09:50.329 INFO:tasks.workunit.client.1.smithi181.stdout:8/215: rmdir d1/d6/d2f/d30 39 2024-04-27T04:09:50.331 INFO:tasks.workunit.client.1.smithi181.stdout:3/183: symlink d0/l41 0 2024-04-27T04:09:50.333 INFO:tasks.workunit.client.1.smithi181.stdout:7/133: dread d1/d22/f21 [0,4194304] 0 2024-04-27T04:09:50.333 INFO:tasks.workunit.client.1.smithi181.stdout:7/134: chown d1/d22/d15/d13 6176 1 2024-04-27T04:09:50.334 INFO:tasks.workunit.client.1.smithi181.stdout:0/208: creat dc/d1e/f51 x:0 0 0 2024-04-27T04:09:50.335 INFO:tasks.workunit.client.1.smithi181.stdout:0/209: chown dc/d1e/c34 31 1 2024-04-27T04:09:50.336 INFO:tasks.workunit.client.1.smithi181.stdout:4/138: rmdir d0/d15 39 2024-04-27T04:09:50.338 INFO:tasks.workunit.client.1.smithi181.stdout:9/246: link d0/d40/f22 d0/f55 0 2024-04-27T04:09:50.346 INFO:tasks.workunit.client.1.smithi181.stdout:3/184: dwrite d0/f1e [0,4194304] 0 2024-04-27T04:09:50.348 INFO:tasks.workunit.client.1.smithi181.stdout:4/139: rename d0/d5 to d0/d5/d22 22 2024-04-27T04:09:50.354 INFO:tasks.workunit.client.1.smithi181.stdout:9/247: fdatasync d0/f37 0 2024-04-27T04:09:50.355 INFO:tasks.workunit.client.1.smithi181.stdout:6/110: sync 2024-04-27T04:09:50.359 INFO:tasks.workunit.client.1.smithi181.stdout:1/180: truncate d1/f16 259137 0 2024-04-27T04:09:50.362 INFO:tasks.workunit.client.1.smithi181.stdout:0/210: symlink dc/d1f/d3b/d4a/l52 0 2024-04-27T04:09:50.364 INFO:tasks.workunit.client.1.smithi181.stdout:5/265: getdents d4/de/d2f 0 2024-04-27T04:09:50.365 INFO:tasks.workunit.client.1.smithi181.stdout:5/266: chown d4/de/d2f 235 1 2024-04-27T04:09:50.368 INFO:tasks.workunit.client.1.smithi181.stdout:7/135: creat d1/d2/f25 x:0 0 0 2024-04-27T04:09:50.369 INFO:tasks.workunit.client.1.smithi181.stdout:6/111: dread f3 [0,4194304] 0 2024-04-27T04:09:50.371 INFO:tasks.workunit.client.1.smithi181.stdout:4/140: creat d0/d15/f23 x:0 0 0 2024-04-27T04:09:50.373 INFO:tasks.workunit.client.1.smithi181.stdout:0/211: mknod dc/d1f/d3b/c53 0 2024-04-27T04:09:50.373 INFO:tasks.workunit.client.1.smithi181.stdout:4/141: read d0/d5/f14 [147795,51406] 0 2024-04-27T04:09:50.469 INFO:tasks.workunit.client.1.smithi181.stdout:7/136: creat d1/d22/f26 x:0 0 0 2024-04-27T04:09:50.470 INFO:tasks.workunit.client.1.smithi181.stdout:0/212: dread - dc/f3a zero size 2024-04-27T04:09:50.470 INFO:tasks.workunit.client.1.smithi181.stdout:4/142: chown d0/d15/f1a 1112 1 2024-04-27T04:09:50.471 INFO:tasks.workunit.client.1.smithi181.stdout:0/213: readlink dc/d1e/d26/l31 0 2024-04-27T04:09:50.471 INFO:tasks.workunit.client.1.smithi181.stdout:1/181: dread d1/d2/f4 [0,4194304] 0 2024-04-27T04:09:50.471 INFO:tasks.workunit.client.1.smithi181.stdout:3/185: truncate d0/f19 882456 0 2024-04-27T04:09:50.472 INFO:tasks.workunit.client.1.smithi181.stdout:6/112: write d7/dc/ff [3273110,4254] 0 2024-04-27T04:09:50.473 INFO:tasks.workunit.client.1.smithi181.stdout:5/267: mkdir d4/de/d2c/d2e/d55/d65 0 2024-04-27T04:09:50.474 INFO:tasks.workunit.client.1.smithi181.stdout:6/113: fdatasync d7/fb 0 2024-04-27T04:09:50.474 INFO:tasks.workunit.client.1.smithi181.stdout:5/268: write d4/d18/f41 [4674544,53153] 0 2024-04-27T04:09:50.477 INFO:tasks.workunit.client.1.smithi181.stdout:6/114: chown d7/c21 18 1 2024-04-27T04:09:50.480 INFO:tasks.workunit.client.1.smithi181.stdout:4/143: dread d0/d5/d7/f10 [0,4194304] 0 2024-04-27T04:09:50.481 INFO:tasks.workunit.client.1.smithi181.stdout:0/214: mkdir dc/d54 0 2024-04-27T04:09:50.483 INFO:tasks.workunit.client.1.smithi181.stdout:3/186: rename d0/f2f to d0/d28/f42 0 2024-04-27T04:09:50.485 INFO:tasks.workunit.client.1.smithi181.stdout:5/269: creat d4/de/d2c/d46/d4d/f66 x:0 0 0 2024-04-27T04:09:50.486 INFO:tasks.workunit.client.1.smithi181.stdout:3/187: dread d0/de/f10 [0,4194304] 0 2024-04-27T04:09:50.487 INFO:tasks.workunit.client.1.smithi181.stdout:6/115: mknod d7/dc/d1c/c25 0 2024-04-27T04:09:50.490 INFO:tasks.workunit.client.1.smithi181.stdout:7/137: creat d1/d2/f27 x:0 0 0 2024-04-27T04:09:50.493 INFO:tasks.workunit.client.1.smithi181.stdout:0/215: rename dc/d1f/d3b/d4a/l52 to dc/d1e/d26/d41/l55 0 2024-04-27T04:09:50.494 INFO:tasks.workunit.client.1.smithi181.stdout:6/116: dwrite d7/dc/f23 [0,4194304] 0 2024-04-27T04:09:50.495 INFO:tasks.workunit.client.1.smithi181.stdout:0/216: chown dc/d1e/d26/l43 75 1 2024-04-27T04:09:50.496 INFO:tasks.workunit.client.1.smithi181.stdout:7/138: dread d1/d2/f24 [0,4194304] 0 2024-04-27T04:09:50.498 INFO:tasks.workunit.client.1.smithi181.stdout:3/188: symlink d0/de/d36/l43 0 2024-04-27T04:09:50.503 INFO:tasks.workunit.client.1.smithi181.stdout:4/144: symlink d0/d15/de/d19/l24 0 2024-04-27T04:09:50.510 INFO:tasks.workunit.client.1.smithi181.stdout:0/217: symlink dc/d1f/d3b/d4a/l56 0 2024-04-27T04:09:50.513 INFO:tasks.workunit.client.1.smithi181.stdout:7/139: creat d1/d22/d15/f28 x:0 0 0 2024-04-27T04:09:50.514 INFO:tasks.workunit.client.1.smithi181.stdout:1/182: link d1/l35 d1/d2/d5/l37 0 2024-04-27T04:09:50.515 INFO:tasks.workunit.client.1.smithi181.stdout:1/183: write d1/d2/d5/d6/f22 [2774173,119889] 0 2024-04-27T04:09:50.515 INFO:tasks.workunit.client.1.smithi181.stdout:4/145: write d0/d5/d7/f10 [3001163,93256] 0 2024-04-27T04:09:50.516 INFO:tasks.workunit.client.1.smithi181.stdout:1/184: write d1/f33 [854704,107287] 0 2024-04-27T04:09:50.517 INFO:tasks.workunit.client.1.smithi181.stdout:4/146: write d0/d5/f6 [4350495,8599] 0 2024-04-27T04:09:50.519 INFO:tasks.workunit.client.1.smithi181.stdout:4/147: truncate d0/d5/f14 205054 0 2024-04-27T04:09:50.523 INFO:tasks.workunit.client.1.smithi181.stdout:5/270: rename d4/d18/d2a to d4/d18/d67 0 2024-04-27T04:09:50.528 INFO:tasks.workunit.client.1.smithi181.stdout:4/148: mkdir d0/d15/d25 0 2024-04-27T04:09:50.528 INFO:tasks.workunit.client.1.smithi181.stdout:0/218: mkdir dc/d54/d57 0 2024-04-27T04:09:50.530 INFO:tasks.workunit.client.1.smithi181.stdout:0/219: write dc/f18 [121023,19597] 0 2024-04-27T04:09:50.531 INFO:tasks.workunit.client.1.smithi181.stdout:5/271: stat d4/l3b 0 2024-04-27T04:09:50.533 INFO:tasks.workunit.client.1.smithi181.stdout:1/185: dread d1/d2/d5/d6/f22 [0,4194304] 0 2024-04-27T04:09:50.534 INFO:tasks.workunit.client.1.smithi181.stdout:1/186: stat d1/d2/d5/d1c/f1f 0 2024-04-27T04:09:50.534 INFO:tasks.workunit.client.1.smithi181.stdout:1/187: stat d1/d2/d5/d1c/d1e 0 2024-04-27T04:09:50.583 INFO:tasks.workunit.client.1.smithi181.stdout:2/184: sync 2024-04-27T04:09:50.584 INFO:tasks.workunit.client.1.smithi181.stdout:2/185: write d4/f32 [368102,81917] 0 2024-04-27T04:09:50.586 INFO:tasks.workunit.client.1.smithi181.stdout:2/186: read d4/d9/df/d14/f15 [682887,3147] 0 2024-04-27T04:09:50.588 INFO:tasks.workunit.client.1.smithi181.stdout:0/220: truncate dc/d1e/f25 756779 0 2024-04-27T04:09:50.592 INFO:tasks.workunit.client.1.smithi181.stdout:1/188: rename d1/d2/d5/d6/c34 to d1/d2/d5/d2c/c38 0 2024-04-27T04:09:50.595 INFO:tasks.workunit.client.1.smithi181.stdout:5/272: symlink d4/l68 0 2024-04-27T04:09:50.596 INFO:tasks.workunit.client.1.smithi181.stdout:0/221: symlink dc/d1f/d3b/d4a/l58 0 2024-04-27T04:09:50.599 INFO:tasks.workunit.client.1.smithi181.stdout:4/149: link d0/d15/de/l13 d0/d15/d25/l26 0 2024-04-27T04:09:50.602 INFO:tasks.workunit.client.1.smithi181.stdout:0/222: mknod dc/d1f/c59 0 2024-04-27T04:09:50.603 INFO:tasks.workunit.client.1.smithi181.stdout:4/150: dread d0/d5/f6 [0,4194304] 0 2024-04-27T04:09:50.603 INFO:tasks.workunit.client.1.smithi181.stdout:4/151: stat d0/d5/d7 0 2024-04-27T04:09:50.609 INFO:tasks.workunit.client.1.smithi181.stdout:2/187: rename c0 to d4/c34 0 2024-04-27T04:09:50.613 INFO:tasks.workunit.client.1.smithi181.stdout:1/189: link d1/d2/d5/d6/c31 d1/d2/d5/d6/db/d2d/d36/c39 0 2024-04-27T04:09:50.614 INFO:tasks.workunit.client.1.smithi181.stdout:5/273: link d4/l68 d4/de/d2c/d46/l69 0 2024-04-27T04:09:50.619 INFO:tasks.workunit.client.1.smithi181.stdout:2/188: mkdir d4/d9/d35 0 2024-04-27T04:09:50.623 INFO:tasks.workunit.client.1.smithi181.stdout:1/190: mknod d1/d2/d5/c3a 0 2024-04-27T04:09:50.625 INFO:tasks.workunit.client.1.smithi181.stdout:5/274: symlink d4/de/d2c/d2e/d55/d65/l6a 0 2024-04-27T04:09:50.626 INFO:tasks.workunit.client.1.smithi181.stdout:5/275: write d4/de/f1e [2553163,64986] 0 2024-04-27T04:09:50.627 INFO:tasks.workunit.client.1.smithi181.stdout:0/223: getdents dc/d1f/d3b 0 2024-04-27T04:09:50.629 INFO:tasks.workunit.client.1.smithi181.stdout:4/152: getdents d0/d5/d7 0 2024-04-27T04:09:50.632 INFO:tasks.workunit.client.1.smithi181.stdout:2/189: dwrite d4/d9/df/d14/f15 [0,4194304] 0 2024-04-27T04:09:50.633 INFO:tasks.workunit.client.1.smithi181.stdout:5/276: symlink d4/de/d2c/d2e/d55/d65/l6b 0 2024-04-27T04:09:50.635 INFO:tasks.workunit.client.1.smithi181.stdout:0/224: read f8 [1154542,42263] 0 2024-04-27T04:09:50.635 INFO:tasks.workunit.client.1.smithi181.stdout:5/277: dread d4/d10/f59 [0,4194304] 0 2024-04-27T04:09:50.636 INFO:tasks.workunit.client.1.smithi181.stdout:0/225: write dc/d1e/f30 [483299,106161] 0 2024-04-27T04:09:50.637 INFO:tasks.workunit.client.1.smithi181.stdout:4/153: mknod d0/d5/d7/c27 0 2024-04-27T04:09:50.638 INFO:tasks.workunit.client.1.smithi181.stdout:4/154: stat d0/d5/d7 0 2024-04-27T04:09:50.639 INFO:tasks.workunit.client.1.smithi181.stdout:2/190: unlink d4/f7 0 2024-04-27T04:09:50.640 INFO:tasks.workunit.client.1.smithi181.stdout:2/191: truncate d4/f28 802626 0 2024-04-27T04:09:50.645 INFO:tasks.workunit.client.1.smithi181.stdout:0/226: mknod dc/d1e/c5a 0 2024-04-27T04:09:50.648 INFO:tasks.workunit.client.1.smithi181.stdout:2/192: dwrite d4/d9/d16/f29 [0,4194304] 0 2024-04-27T04:09:50.650 INFO:tasks.workunit.client.1.smithi181.stdout:2/193: chown d4/d9/df/d14/c2e 281171713 1 2024-04-27T04:09:50.652 INFO:tasks.workunit.client.1.smithi181.stdout:4/155: dread d0/d5/f6 [0,4194304] 0 2024-04-27T04:09:50.653 INFO:tasks.workunit.client.1.smithi181.stdout:2/194: write d4/d9/df/d14/f15 [1726606,102878] 0 2024-04-27T04:09:50.655 INFO:tasks.workunit.client.1.smithi181.stdout:0/227: fdatasync f8 0 2024-04-27T04:09:50.659 INFO:tasks.workunit.client.1.smithi181.stdout:2/195: dread d4/f6 [0,4194304] 0 2024-04-27T04:09:50.663 INFO:tasks.workunit.client.1.smithi181.stdout:4/156: creat d0/d5/d7/f28 x:0 0 0 2024-04-27T04:09:50.667 INFO:tasks.workunit.client.1.smithi181.stdout:0/228: creat dc/d54/f5b x:0 0 0 2024-04-27T04:09:50.668 INFO:tasks.workunit.client.1.smithi181.stdout:2/196: mknod d4/d9/d16/c36 0 2024-04-27T04:09:50.669 INFO:tasks.workunit.client.1.smithi181.stdout:4/157: dwrite d0/d5/d7/f10 [0,4194304] 0 2024-04-27T04:09:50.671 INFO:tasks.workunit.client.1.smithi181.stdout:2/197: dread d4/f28 [0,4194304] 0 2024-04-27T04:09:50.675 INFO:tasks.workunit.client.1.smithi181.stdout:2/198: dread d4/d9/df/d14/f15 [0,4194304] 0 2024-04-27T04:09:50.676 INFO:tasks.workunit.client.1.smithi181.stdout:2/199: chown d4/d9/df/l22 0 1 2024-04-27T04:09:50.676 INFO:tasks.workunit.client.1.smithi181.stdout:2/200: readlink d4/l5 0 2024-04-27T04:09:50.677 INFO:tasks.workunit.client.1.smithi181.stdout:2/201: readlink d4/d9/df/l13 0 2024-04-27T04:09:50.678 INFO:tasks.workunit.client.1.smithi181.stdout:4/158: rename d0/d15/l21 to d0/d5/d7/l29 0 2024-04-27T04:09:50.680 INFO:tasks.workunit.client.1.smithi181.stdout:4/159: truncate d0/d5/d7/f28 871081 0 2024-04-27T04:09:50.682 INFO:tasks.workunit.client.1.smithi181.stdout:0/229: mknod dc/c5c 0 2024-04-27T04:09:50.684 INFO:tasks.workunit.client.1.smithi181.stdout:2/202: dwrite d4/d9/f11 [0,4194304] 0 2024-04-27T04:09:50.691 INFO:tasks.workunit.client.1.smithi181.stdout:0/230: dwrite dc/f35 [0,4194304] 0 2024-04-27T04:09:50.695 INFO:tasks.workunit.client.1.smithi181.stdout:4/160: mkdir d0/d15/d25/d2a 0 2024-04-27T04:09:50.696 INFO:tasks.workunit.client.1.smithi181.stdout:2/203: mknod d4/d9/df/c37 0 2024-04-27T04:09:50.699 INFO:tasks.workunit.client.1.smithi181.stdout:0/231: creat dc/d1f/d3b/f5d x:0 0 0 2024-04-27T04:09:50.702 INFO:tasks.workunit.client.1.smithi181.stdout:0/232: dread dc/f2c [0,4194304] 0 2024-04-27T04:09:50.703 INFO:tasks.workunit.client.1.smithi181.stdout:0/233: write dc/f10 [814835,49166] 0 2024-04-27T04:09:50.713 INFO:tasks.workunit.client.1.smithi181.stdout:2/204: fsync d4/fb 0 2024-04-27T04:09:50.713 INFO:tasks.workunit.client.1.smithi181.stdout:6/117: truncate d7/dc/ff 2561627 0 2024-04-27T04:09:50.715 INFO:tasks.workunit.client.1.smithi181.stdout:6/118: truncate d7/dc/d1c/f1f 798062 0 2024-04-27T04:09:50.716 INFO:tasks.workunit.client.1.smithi181.stdout:7/140: write d1/d22/f21 [236661,82750] 0 2024-04-27T04:09:50.717 INFO:tasks.workunit.client.1.smithi181.stdout:7/141: stat d1/d2/f12 0 2024-04-27T04:09:50.721 INFO:tasks.workunit.client.1.smithi181.stdout:7/142: write d1/d2/d1c/f1e [249839,98621] 0 2024-04-27T04:09:50.721 INFO:tasks.workunit.client.1.smithi181.stdout:2/205: dwrite d4/d9/f1a [4194304,4194304] 0 2024-04-27T04:09:50.722 INFO:tasks.workunit.client.1.smithi181.stdout:2/206: readlink d4/la 0 2024-04-27T04:09:50.724 INFO:tasks.workunit.client.1.smithi181.stdout:6/119: getdents d7/dc/d1c 0 2024-04-27T04:09:50.729 INFO:tasks.workunit.client.1.smithi181.stdout:2/207: rename d4/d9/df/d14/c1d to d4/d9/d2f/c38 0 2024-04-27T04:09:50.731 INFO:tasks.workunit.client.1.smithi181.stdout:6/120: dwrite f6 [0,4194304] 0 2024-04-27T04:09:50.733 INFO:tasks.workunit.client.1.smithi181.stdout:6/121: chown d7/fe 202321259 1 2024-04-27T04:09:50.734 INFO:tasks.workunit.client.1.smithi181.stdout:6/122: stat d7/d1b 0 2024-04-27T04:09:50.739 INFO:tasks.workunit.client.1.smithi181.stdout:2/208: dwrite d4/d9/f2d [0,4194304] 0 2024-04-27T04:09:50.739 INFO:tasks.workunit.client.1.smithi181.stdout:6/123: rename d7/f9 to d7/f26 0 2024-04-27T04:09:50.740 INFO:tasks.workunit.client.1.smithi181.stdout:6/124: dread - d7/fb zero size 2024-04-27T04:09:50.744 INFO:tasks.workunit.client.1.smithi181.stdout:6/125: mkdir d7/d27 0 2024-04-27T04:09:50.745 INFO:tasks.workunit.client.1.smithi181.stdout:6/126: dread - d7/dc/d1c/f20 zero size 2024-04-27T04:09:50.746 INFO:tasks.workunit.client.1.smithi181.stdout:6/127: chown d7/dc/d1c/c25 338868499 1 2024-04-27T04:09:50.747 INFO:tasks.workunit.client.1.smithi181.stdout:6/128: fsync d7/dc/d1c/f1f 0 2024-04-27T04:09:50.750 INFO:tasks.workunit.client.1.smithi181.stdout:2/209: rename d4/d9/c10 to d4/c39 0 2024-04-27T04:09:50.750 INFO:tasks.workunit.client.1.smithi181.stdout:6/129: rename d7/d27 to d7/d27/d28 22 2024-04-27T04:09:50.756 INFO:tasks.workunit.client.1.smithi181.stdout:2/210: dwrite d4/d9/d16/f23 [0,4194304] 0 2024-04-27T04:09:50.758 INFO:tasks.workunit.client.1.smithi181.stdout:2/211: write d4/d9/d16/f26 [1655121,16427] 0 2024-04-27T04:09:50.765 INFO:tasks.workunit.client.1.smithi181.stdout:2/212: creat d4/d9/d35/f3a x:0 0 0 2024-04-27T04:09:50.767 INFO:tasks.workunit.client.1.smithi181.stdout:5/278: write d4/de/d2c/f5c [2748698,44244] 0 2024-04-27T04:09:50.768 INFO:tasks.workunit.client.1.smithi181.stdout:5/279: fdatasync d4/d18/d67/f50 0 2024-04-27T04:09:50.773 INFO:tasks.workunit.client.1.smithi181.stdout:2/213: dwrite d4/d9/f11 [0,4194304] 0 2024-04-27T04:09:50.773 INFO:tasks.workunit.client.1.smithi181.stdout:5/280: unlink d4/d10/l28 0 2024-04-27T04:09:50.774 INFO:tasks.workunit.client.1.smithi181.stdout:2/214: write d4/d9/d16/f23 [1955432,21732] 0 2024-04-27T04:09:50.777 INFO:tasks.workunit.client.1.smithi181.stdout:5/281: symlink d4/de/d2c/d46/l6c 0 2024-04-27T04:09:50.782 INFO:tasks.workunit.client.1.smithi181.stdout:5/282: dread d4/d18/d67/f39 [0,4194304] 0 2024-04-27T04:09:50.787 INFO:tasks.workunit.client.1.smithi181.stdout:5/283: symlink d4/d18/l6d 0 2024-04-27T04:09:50.788 INFO:tasks.workunit.client.1.smithi181.stdout:5/284: write d4/de/d2f/f5d [5141744,35991] 0 2024-04-27T04:09:50.793 INFO:tasks.workunit.client.1.smithi181.stdout:5/285: unlink d4/de/d2c/d46/l6c 0 2024-04-27T04:09:50.800 INFO:tasks.workunit.client.1.smithi181.stdout:5/286: dwrite d4/d18/d67/f50 [0,4194304] 0 2024-04-27T04:09:50.801 INFO:tasks.workunit.client.1.smithi181.stdout:5/287: fsync d4/de/f1e 0 2024-04-27T04:09:50.802 INFO:tasks.workunit.client.1.smithi181.stdout:5/288: readlink d4/de/d2c/d2e/l62 0 2024-04-27T04:09:50.810 INFO:tasks.workunit.client.1.smithi181.stdout:4/161: dwrite d0/d15/f2 [0,4194304] 0 2024-04-27T04:09:50.813 INFO:tasks.workunit.client.1.smithi181.stdout:0/234: write dc/f16 [1160235,38896] 0 2024-04-27T04:09:50.817 INFO:tasks.workunit.client.1.smithi181.stdout:0/235: write dc/d1f/d3b/f5d [672289,112601] 0 2024-04-27T04:09:50.819 INFO:tasks.workunit.client.1.smithi181.stdout:7/143: dwrite d1/d2/f24 [0,4194304] 0 2024-04-27T04:09:50.821 INFO:tasks.workunit.client.1.smithi181.stdout:7/144: write d1/d2/f24 [93819,18239] 0 2024-04-27T04:09:50.821 INFO:tasks.workunit.client.1.smithi181.stdout:4/162: rename d0/d5 to d0/d15/d25/d2b 0 2024-04-27T04:09:50.823 INFO:tasks.workunit.client.1.smithi181.stdout:7/145: write d1/d2/f12 [8511586,87291] 0 2024-04-27T04:09:50.831 INFO:tasks.workunit.client.1.smithi181.stdout:4/163: creat d0/d15/d25/d2b/f2c x:0 0 0 2024-04-27T04:09:50.834 INFO:tasks.workunit.client.1.smithi181.stdout:7/146: mkdir d1/d22/d15/d29 0 2024-04-27T04:09:50.835 INFO:tasks.workunit.client.1.smithi181.stdout:6/130: truncate d7/f19 3882699 0 2024-04-27T04:09:50.839 INFO:tasks.workunit.client.1.smithi181.stdout:4/164: mknod d0/d15/d25/d2b/d7/c2d 0 2024-04-27T04:09:50.840 INFO:tasks.workunit.client.1.smithi181.stdout:4/165: read d0/d15/d25/d2b/f14 [152037,46615] 0 2024-04-27T04:09:50.840 INFO:tasks.workunit.client.1.smithi181.stdout:0/236: creat dc/f5e x:0 0 0 2024-04-27T04:09:50.850 INFO:tasks.workunit.client.1.smithi181.stdout:7/147: link d1/d22/d15/f28 d1/d22/d15/f2a 0 2024-04-27T04:09:50.852 INFO:tasks.workunit.client.1.smithi181.stdout:2/215: link d4/d9/d2f/c38 d4/c3b 0 2024-04-27T04:09:50.858 INFO:tasks.workunit.client.1.smithi181.stdout:2/216: dwrite d4/d9/f1a [0,4194304] 0 2024-04-27T04:09:50.859 INFO:tasks.workunit.client.1.smithi181.stdout:7/148: mknod d1/d2/d1c/c2b 0 2024-04-27T04:09:50.866 INFO:tasks.workunit.client.1.smithi181.stdout:2/217: mknod d4/c3c 0 2024-04-27T04:09:50.868 INFO:tasks.workunit.client.1.smithi181.stdout:2/218: truncate d4/f32 957257 0 2024-04-27T04:09:50.868 INFO:tasks.workunit.client.1.smithi181.stdout:7/149: dwrite d1/d2/f25 [0,4194304] 0 2024-04-27T04:09:50.872 INFO:tasks.workunit.client.1.smithi181.stdout:7/150: unlink d1/d22/f21 0 2024-04-27T04:09:50.874 INFO:tasks.workunit.client.1.smithi181.stdout:2/219: creat d4/d9/d2f/f3d x:0 0 0 2024-04-27T04:09:50.875 INFO:tasks.workunit.client.1.smithi181.stdout:5/289: dwrite d4/de/f47 [0,4194304] 0 2024-04-27T04:09:50.881 INFO:tasks.workunit.client.1.smithi181.stdout:7/151: creat d1/d2/d1c/f2c x:0 0 0 2024-04-27T04:09:50.883 INFO:tasks.workunit.client.1.smithi181.stdout:8/216: sync 2024-04-27T04:09:50.888 INFO:tasks.workunit.client.1.smithi181.stdout:7/152: dwrite d1/d2/f19 [0,4194304] 0 2024-04-27T04:09:50.890 INFO:tasks.workunit.client.1.smithi181.stdout:7/153: write d1/d2/f25 [3988412,106474] 0 2024-04-27T04:09:50.892 INFO:tasks.workunit.client.1.smithi181.stdout:2/220: mknod d4/c3e 0 2024-04-27T04:09:50.893 INFO:tasks.workunit.client.1.smithi181.stdout:2/221: read - d4/f2b zero size 2024-04-27T04:09:50.893 INFO:tasks.workunit.client.1.smithi181.stdout:8/217: dwrite d1/d6/f44 [0,4194304] 0 2024-04-27T04:09:50.895 INFO:tasks.workunit.client.1.smithi181.stdout:8/218: chown d1/d6/d1a/l3b 119 1 2024-04-27T04:09:50.898 INFO:tasks.workunit.client.1.smithi181.stdout:8/219: read d1/d6/f26 [3357610,13038] 0 2024-04-27T04:09:50.905 INFO:tasks.workunit.client.1.smithi181.stdout:2/222: creat d4/d9/df/d14/f3f x:0 0 0 2024-04-27T04:09:50.908 INFO:tasks.workunit.client.1.smithi181.stdout:8/220: creat d1/d6/f4e x:0 0 0 2024-04-27T04:09:50.909 INFO:tasks.workunit.client.1.smithi181.stdout:8/221: stat d1/d6/d1a/d25/c49 0 2024-04-27T04:09:50.910 INFO:tasks.workunit.client.1.smithi181.stdout:5/290: creat d4/de/f6e x:0 0 0 2024-04-27T04:09:50.913 INFO:tasks.workunit.client.1.smithi181.stdout:0/237: dwrite f6 [0,4194304] 0 2024-04-27T04:09:50.913 INFO:tasks.workunit.client.1.smithi181.stdout:8/222: dread d1/d6/d1a/f4d [0,4194304] 0 2024-04-27T04:09:50.915 INFO:tasks.workunit.client.1.smithi181.stdout:2/223: unlink d4/d9/df/d14/f3f 0 2024-04-27T04:09:50.916 INFO:tasks.workunit.client.1.smithi181.stdout:4/166: truncate d0/d15/f2 345145 0 2024-04-27T04:09:50.923 INFO:tasks.workunit.client.1.smithi181.stdout:5/291: rmdir d4/de/d2c/d46/d4d 39 2024-04-27T04:09:50.925 INFO:tasks.workunit.client.1.smithi181.stdout:8/223: creat d1/d6/d2f/d47/f4f x:0 0 0 2024-04-27T04:09:50.929 INFO:tasks.workunit.client.1.smithi181.stdout:2/224: mkdir d4/d9/d2f/d40 0 2024-04-27T04:09:50.932 INFO:tasks.workunit.client.1.smithi181.stdout:5/292: dwrite d4/d18/d67/f50 [0,4194304] 0 2024-04-27T04:09:50.934 INFO:tasks.workunit.client.1.smithi181.stdout:8/224: dwrite d1/d6/f44 [0,4194304] 0 2024-04-27T04:09:50.937 INFO:tasks.workunit.client.1.smithi181.stdout:2/225: dread d4/d9/f2d [0,4194304] 0 2024-04-27T04:09:50.939 INFO:tasks.workunit.client.1.smithi181.stdout:0/238: link dc/f1b dc/d1f/d3b/f5f 0 2024-04-27T04:09:50.940 INFO:tasks.workunit.client.1.smithi181.stdout:0/239: write dc/f46 [257421,12923] 0 2024-04-27T04:09:50.946 INFO:tasks.workunit.client.1.smithi181.stdout:8/225: symlink d1/d20/l50 0 2024-04-27T04:09:50.951 INFO:tasks.workunit.client.1.smithi181.stdout:4/167: rename d0/d15/l12 to d0/l2e 0 2024-04-27T04:09:50.956 INFO:tasks.workunit.client.1.smithi181.stdout:4/168: dread d0/d15/d25/d2b/f1c [0,4194304] 0 2024-04-27T04:09:50.958 INFO:tasks.workunit.client.1.smithi181.stdout:0/240: mknod dc/c60 0 2024-04-27T04:09:50.961 INFO:tasks.workunit.client.1.smithi181.stdout:8/226: dwrite d1/d6/d2f/d40/d2a/f42 [0,4194304] 0 2024-04-27T04:09:50.961 INFO:tasks.workunit.client.1.smithi181.stdout:0/241: truncate dc/d1e/d26/d41/f4e 4808870 0 2024-04-27T04:09:50.963 INFO:tasks.workunit.client.1.smithi181.stdout:0/242: chown dc/d1e/c21 884395997 1 2024-04-27T04:09:50.963 INFO:tasks.workunit.client.1.smithi181.stdout:5/293: creat d4/de/f6f x:0 0 0 2024-04-27T04:09:50.970 INFO:tasks.workunit.client.1.smithi181.stdout:8/227: creat d1/d20/f51 x:0 0 0 2024-04-27T04:09:50.974 INFO:tasks.workunit.client.1.smithi181.stdout:5/294: rename d4/de/d2f/l20 to d4/d18/d67/l70 0 2024-04-27T04:09:50.978 INFO:tasks.workunit.client.1.smithi181.stdout:5/295: dread d4/d18/d67/f39 [0,4194304] 0 2024-04-27T04:09:50.978 INFO:tasks.workunit.client.1.smithi181.stdout:8/228: creat d1/d6/d1a/d25/f52 x:0 0 0 2024-04-27T04:09:50.982 INFO:tasks.workunit.client.1.smithi181.stdout:5/296: dread d4/d18/f41 [0,4194304] 0 2024-04-27T04:09:50.983 INFO:tasks.workunit.client.1.smithi181.stdout:4/169: getdents d0/d15/de/d19 0 2024-04-27T04:09:50.984 INFO:tasks.workunit.client.1.smithi181.stdout:5/297: write d4/de/f14 [4314591,21532] 0 2024-04-27T04:09:50.986 INFO:tasks.workunit.client.1.smithi181.stdout:2/226: write d4/f6 [1239563,53991] 0 2024-04-27T04:09:50.986 INFO:tasks.workunit.client.1.smithi181.stdout:0/243: link dc/l4b dc/l61 0 2024-04-27T04:09:50.987 INFO:tasks.workunit.client.1.smithi181.stdout:0/244: truncate dc/f22 852086 0 2024-04-27T04:09:50.989 INFO:tasks.workunit.client.1.smithi181.stdout:0/245: readlink dc/d1e/d26/l43 0 2024-04-27T04:09:50.990 INFO:tasks.workunit.client.1.smithi181.stdout:4/170: creat d0/d15/f2f x:0 0 0 2024-04-27T04:09:50.993 INFO:tasks.workunit.client.1.smithi181.stdout:8/229: rmdir d1/d6/d2f/d30 39 2024-04-27T04:09:50.994 INFO:tasks.workunit.client.1.smithi181.stdout:2/227: dread d4/f28 [0,4194304] 0 2024-04-27T04:09:50.996 INFO:tasks.workunit.client.1.smithi181.stdout:0/246: creat dc/d1f/d3b/d4a/f62 x:0 0 0 2024-04-27T04:09:50.997 INFO:tasks.workunit.client.1.smithi181.stdout:0/247: write dc/d1f/d3b/f5d [1147601,68806] 0 2024-04-27T04:09:50.998 INFO:tasks.workunit.client.1.smithi181.stdout:0/248: dread - dc/d1e/f36 zero size 2024-04-27T04:09:50.999 INFO:tasks.workunit.client.1.smithi181.stdout:8/230: creat d1/d6/d1a/f53 x:0 0 0 2024-04-27T04:09:50.999 INFO:tasks.workunit.client.1.smithi181.stdout:4/171: dread d0/d15/d25/d2b/f14 [0,4194304] 0 2024-04-27T04:09:51.003 INFO:tasks.workunit.client.1.smithi181.stdout:8/231: symlink d1/d6/d2f/d40/d2a/l54 0 2024-04-27T04:09:51.006 INFO:tasks.workunit.client.1.smithi181.stdout:2/228: creat d4/d9/df/f41 x:0 0 0 2024-04-27T04:09:51.008 INFO:tasks.workunit.client.1.smithi181.stdout:8/232: rmdir d1/d6/d2f/d40 39 2024-04-27T04:09:51.009 INFO:tasks.workunit.client.1.smithi181.stdout:8/233: truncate d1/d6/d1a/f53 441232 0 2024-04-27T04:09:51.010 INFO:tasks.workunit.client.1.smithi181.stdout:4/172: creat d0/f30 x:0 0 0 2024-04-27T04:09:51.011 INFO:tasks.workunit.client.1.smithi181.stdout:4/173: stat d0/d15/d25 0 2024-04-27T04:09:51.013 INFO:tasks.workunit.client.1.smithi181.stdout:2/229: unlink d4/f28 0 2024-04-27T04:09:51.014 INFO:tasks.workunit.client.1.smithi181.stdout:0/249: getdents dc/d54 0 2024-04-27T04:09:51.014 INFO:tasks.workunit.client.1.smithi181.stdout:2/230: write d4/d9/d16/f23 [1409139,72252] 0 2024-04-27T04:09:51.015 INFO:tasks.workunit.client.1.smithi181.stdout:0/250: write dc/f22 [1007986,124714] 0 2024-04-27T04:09:51.016 INFO:tasks.workunit.client.1.smithi181.stdout:0/251: stat dc/f10 0 2024-04-27T04:09:51.023 INFO:tasks.workunit.client.1.smithi181.stdout:2/231: dwrite d4/d9/f1a [4194304,4194304] 0 2024-04-27T04:09:51.026 INFO:tasks.workunit.client.1.smithi181.stdout:8/234: dwrite d1/d6/d2f/d40/d2a/f37 [0,4194304] 0 2024-04-27T04:09:51.029 INFO:tasks.workunit.client.1.smithi181.stdout:2/232: mkdir d4/d9/d35/d42 0 2024-04-27T04:09:51.031 INFO:tasks.workunit.client.1.smithi181.stdout:8/235: creat d1/d6/f55 x:0 0 0 2024-04-27T04:09:51.032 INFO:tasks.workunit.client.1.smithi181.stdout:2/233: truncate d4/d9/df/d14/f24 4903788 0 2024-04-27T04:09:51.038 INFO:tasks.workunit.client.1.smithi181.stdout:2/234: dwrite d4/f30 [0,4194304] 0 2024-04-27T04:09:51.076 INFO:tasks.workunit.client.1.smithi181.stdout:9/248: sync 2024-04-27T04:09:51.081 INFO:tasks.workunit.client.1.smithi181.stdout:8/236: unlink d1/f34 0 2024-04-27T04:09:51.082 INFO:tasks.workunit.client.1.smithi181.stdout:8/237: chown d1/l22 8504192 1 2024-04-27T04:09:51.082 INFO:tasks.workunit.client.1.smithi181.stdout:8/238: fsync d1/f46 0 2024-04-27T04:09:51.089 INFO:tasks.workunit.client.1.smithi181.stdout:8/239: dwrite d1/d6/f44 [4194304,4194304] 0 2024-04-27T04:09:51.093 INFO:tasks.workunit.client.1.smithi181.stdout:9/249: creat d0/d44/d4b/d43/f56 x:0 0 0 2024-04-27T04:09:51.094 INFO:tasks.workunit.client.1.smithi181.stdout:8/240: creat d1/d20/f56 x:0 0 0 2024-04-27T04:09:51.096 INFO:tasks.workunit.client.1.smithi181.stdout:8/241: truncate d1/d20/f24 9393327 0 2024-04-27T04:09:51.098 INFO:tasks.workunit.client.1.smithi181.stdout:9/250: symlink d0/d44/d4f/l57 0 2024-04-27T04:09:51.104 INFO:tasks.workunit.client.1.smithi181.stdout:9/251: rename d0/d44/d4b/d3d/f45 to d0/d44/d4f/f58 0 2024-04-27T04:09:51.109 INFO:tasks.workunit.client.1.smithi181.stdout:0/252: dwrite dc/d1e/d26/d41/f4e [0,4194304] 0 2024-04-27T04:09:51.110 INFO:tasks.workunit.client.1.smithi181.stdout:2/235: getdents d4/d9/d35 0 2024-04-27T04:09:51.115 INFO:tasks.workunit.client.1.smithi181.stdout:0/253: creat dc/d1e/f63 x:0 0 0 2024-04-27T04:09:51.118 INFO:tasks.workunit.client.1.smithi181.stdout:2/236: dwrite d4/f8 [0,4194304] 0 2024-04-27T04:09:51.121 INFO:tasks.workunit.client.1.smithi181.stdout:9/252: unlink d0/d40/c32 0 2024-04-27T04:09:51.125 INFO:tasks.workunit.client.1.smithi181.stdout:0/254: dwrite dc/f42 [0,4194304] 0 2024-04-27T04:09:51.126 INFO:tasks.workunit.client.1.smithi181.stdout:9/253: truncate d0/d44/f48 4745079 0 2024-04-27T04:09:51.130 INFO:tasks.workunit.client.1.smithi181.stdout:2/237: dread d4/f32 [0,4194304] 0 2024-04-27T04:09:51.132 INFO:tasks.workunit.client.1.smithi181.stdout:9/254: write d0/d44/f48 [3176374,84941] 0 2024-04-27T04:09:51.137 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:09:50 smithi162 ceph-mon[26841]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' 2024-04-27T04:09:51.137 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:09:50 smithi162 ceph-mon[26841]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' 2024-04-27T04:09:51.138 INFO:tasks.workunit.client.1.smithi181.stdout:2/238: mknod d4/c43 0 2024-04-27T04:09:51.140 INFO:tasks.workunit.client.1.smithi181.stdout:8/242: dwrite d1/d6/d2f/d40/f18 [0,4194304] 0 2024-04-27T04:09:51.142 INFO:tasks.workunit.client.1.smithi181.stdout:9/255: fsync d0/d44/f48 0 2024-04-27T04:09:51.146 INFO:tasks.workunit.client.1.smithi181.stdout:8/243: chown d1/d6/d2f/d40/l2e 25353297 1 2024-04-27T04:09:51.148 INFO:tasks.workunit.client.1.smithi181.stdout:2/239: mkdir d4/d9/d44 0 2024-04-27T04:09:51.149 INFO:tasks.workunit.client.1.smithi181.stdout:2/240: dread - d4/d9/df/f41 zero size 2024-04-27T04:09:51.150 INFO:tasks.workunit.client.1.smithi181.stdout:2/241: read d4/d9/f11 [287519,98437] 0 2024-04-27T04:09:51.152 INFO:tasks.workunit.client.1.smithi181.stdout:8/244: mkdir d1/d57 0 2024-04-27T04:09:51.156 INFO:tasks.workunit.client.0.smithi162.stderr:+ pushd ltp-full-20091231/testcases/kernel/fs/fsstress 2024-04-27T04:09:51.156 INFO:tasks.workunit.client.1.smithi181.stdout:9/256: dwrite d0/d44/d4f/f58 [0,4194304] 0 2024-04-27T04:09:51.158 INFO:tasks.workunit.client.1.smithi181.stdout:9/257: stat d0/d44/d4b/d3d/f3e 0 2024-04-27T04:09:51.158 INFO:tasks.workunit.client.1.smithi181.stdout:2/242: dread d4/d9/d16/f29 [0,4194304] 0 2024-04-27T04:09:51.160 INFO:tasks.workunit.client.0.smithi162.stdout:~/cephtest/mnt.0/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/fs/fsstress ~/cephtest/mnt.0/client.0/tmp/fsstress ~/cephtest/mnt.0/client.0/tmp 2024-04-27T04:09:51.161 INFO:tasks.workunit.client.0.smithi162.stderr:+ make 2024-04-27T04:09:51.165 INFO:tasks.workunit.client.1.smithi181.stdout:9/258: symlink d0/l59 0 2024-04-27T04:09:51.167 INFO:tasks.workunit.client.1.smithi181.stdout:2/243: dwrite d4/d9/df/f41 [0,4194304] 0 2024-04-27T04:09:51.171 INFO:tasks.workunit.client.1.smithi181.stdout:8/245: dwrite d1/d6/d1a/f4d [0,4194304] 0 2024-04-27T04:09:51.171 INFO:tasks.workunit.client.1.smithi181.stdout:9/259: write d0/fc [4417775,25935] 0 2024-04-27T04:09:51.172 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:09:50 smithi181 ceph-mon[33774]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' 2024-04-27T04:09:51.172 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:09:50 smithi181 ceph-mon[33774]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' 2024-04-27T04:09:51.174 INFO:tasks.workunit.client.1.smithi181.stdout:2/244: mkdir d4/d9/d16/d45 0 2024-04-27T04:09:51.175 INFO:tasks.workunit.client.1.smithi181.stdout:9/260: fdatasync d0/f15 0 2024-04-27T04:09:51.176 INFO:tasks.workunit.client.1.smithi181.stdout:8/246: symlink d1/d6/d2f/d40/d2a/l58 0 2024-04-27T04:09:51.177 INFO:tasks.workunit.client.1.smithi181.stdout:8/247: read - d1/d6/f4a zero size 2024-04-27T04:09:51.185 INFO:tasks.workunit.client.1.smithi181.stdout:8/248: dwrite d1/d20/f24 [4194304,4194304] 0 2024-04-27T04:09:51.185 INFO:tasks.workunit.client.1.smithi181.stdout:9/261: symlink d0/d44/d4b/d50/l5a 0 2024-04-27T04:09:51.186 INFO:tasks.workunit.client.1.smithi181.stdout:8/249: readlink d1/d20/l50 0 2024-04-27T04:09:51.188 INFO:tasks.workunit.client.1.smithi181.stdout:2/245: getdents d4/d9/d35 0 2024-04-27T04:09:51.189 INFO:tasks.workunit.client.1.smithi181.stdout:2/246: stat d4/f2b 0 2024-04-27T04:09:51.192 INFO:tasks.workunit.client.1.smithi181.stdout:9/262: rename d0/d44/d4b/d50/l5a to d0/d44/l5b 0 2024-04-27T04:09:51.194 INFO:tasks.workunit.client.1.smithi181.stdout:2/247: unlink d4/fb 0 2024-04-27T04:09:51.199 INFO:tasks.workunit.client.1.smithi181.stdout:2/248: creat d4/d9/d44/f46 x:0 0 0 2024-04-27T04:09:51.199 INFO:tasks.workunit.client.1.smithi181.stdout:2/249: readlink d4/d9/l31 0 2024-04-27T04:09:51.209 INFO:tasks.workunit.client.1.smithi181.stdout:3/189: sync 2024-04-27T04:09:51.210 INFO:tasks.workunit.client.1.smithi181.stdout:1/191: sync 2024-04-27T04:09:51.212 INFO:tasks.workunit.client.1.smithi181.stdout:2/250: dwrite d4/f6 [0,4194304] 0 2024-04-27T04:09:51.214 INFO:tasks.workunit.client.1.smithi181.stdout:9/263: dread d0/d44/f48 [0,4194304] 0 2024-04-27T04:09:51.214 INFO:tasks.workunit.client.1.smithi181.stdout:8/250: dwrite d1/d6/f26 [0,4194304] 0 2024-04-27T04:09:51.215 INFO:tasks.workunit.client.0.smithi162.stdout:cc -DNO_XFS -I/home/ubuntu/cephtest/mnt.0/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/fs/fsstress -D_LARGEFILE64_SOURCE -D_GNU_SOURCE -I../../../../include -I../../../../include -L../../../../lib fsstress.c -o fsstress 2024-04-27T04:09:51.215 INFO:tasks.workunit.client.1.smithi181.stdout:3/190: write d0/de/d36/f3d [1031105,88042] 0 2024-04-27T04:09:51.217 INFO:tasks.workunit.client.1.smithi181.stdout:9/264: readlink d0/l21 0 2024-04-27T04:09:51.219 INFO:tasks.workunit.client.1.smithi181.stdout:8/251: mkdir d1/d6/d59 0 2024-04-27T04:09:51.220 INFO:tasks.workunit.client.1.smithi181.stdout:3/191: mkdir d0/de/d36/d44 0 2024-04-27T04:09:51.223 INFO:tasks.workunit.client.1.smithi181.stdout:9/265: read d0/f2a [1425967,114679] 0 2024-04-27T04:09:51.224 INFO:tasks.workunit.client.1.smithi181.stdout:3/192: readlink d0/lc 0 2024-04-27T04:09:51.227 INFO:tasks.workunit.client.1.smithi181.stdout:8/252: symlink d1/d6/d59/l5a 0 2024-04-27T04:09:51.231 INFO:tasks.workunit.client.1.smithi181.stdout:9/266: dwrite d0/d44/d4b/f39 [0,4194304] 0 2024-04-27T04:09:51.233 INFO:tasks.workunit.client.1.smithi181.stdout:2/251: symlink d4/d9/d16/d45/l47 0 2024-04-27T04:09:51.234 INFO:tasks.workunit.client.1.smithi181.stdout:9/267: write d0/d44/d4b/d43/f56 [479402,121866] 0 2024-04-27T04:09:51.240 INFO:tasks.workunit.client.1.smithi181.stdout:2/252: chown d4/d9/d2f/c38 29 1 2024-04-27T04:09:51.242 INFO:tasks.workunit.client.1.smithi181.stdout:9/268: getdents d0/d44/d4b/d3d 0 2024-04-27T04:09:51.247 INFO:tasks.workunit.client.1.smithi181.stdout:9/269: chown d0/d40/c38 7918929 1 2024-04-27T04:09:51.249 INFO:tasks.workunit.client.1.smithi181.stdout:3/193: dwrite d0/f15 [0,4194304] 0 2024-04-27T04:09:51.253 INFO:tasks.workunit.client.1.smithi181.stdout:9/270: mknod d0/d44/d4b/d3d/c5c 0 2024-04-27T04:09:51.254 INFO:tasks.workunit.client.1.smithi181.stdout:8/253: dread d1/d6/d2f/d40/f4c [0,4194304] 0 2024-04-27T04:09:51.254 INFO:tasks.workunit.client.1.smithi181.stdout:2/253: getdents d4/d9/d35 0 2024-04-27T04:09:51.255 INFO:tasks.workunit.client.1.smithi181.stdout:8/254: fdatasync d1/d6/d2f/d40/d2a/f42 0 2024-04-27T04:09:51.258 INFO:tasks.workunit.client.1.smithi181.stdout:9/271: mkdir d0/d44/d4b/d43/d5d 0 2024-04-27T04:09:51.263 INFO:tasks.workunit.client.1.smithi181.stdout:3/194: creat d0/de/d36/d44/f45 x:0 0 0 2024-04-27T04:09:51.264 INFO:tasks.workunit.client.1.smithi181.stdout:8/255: dwrite d1/d6/d1a/d25/f52 [0,4194304] 0 2024-04-27T04:09:51.265 INFO:tasks.workunit.client.1.smithi181.stdout:9/272: creat d0/d44/d4b/d43/f5e x:0 0 0 2024-04-27T04:09:51.267 INFO:tasks.workunit.client.1.smithi181.stdout:2/254: creat d4/d9/df/d14/f48 x:0 0 0 2024-04-27T04:09:51.268 INFO:tasks.workunit.client.1.smithi181.stdout:2/255: write d4/f30 [3404614,17411] 0 2024-04-27T04:09:51.268 INFO:tasks.workunit.client.1.smithi181.stdout:2/256: readlink d4/la 0 2024-04-27T04:09:51.268 INFO:tasks.workunit.client.1.smithi181.stdout:3/195: creat d0/de/d20/f46 x:0 0 0 2024-04-27T04:09:51.271 INFO:tasks.workunit.client.1.smithi181.stdout:8/256: mknod d1/d20/c5b 0 2024-04-27T04:09:51.282 INFO:tasks.workunit.client.1.smithi181.stdout:9/273: mkdir d0/d44/d4b/d5f 0 2024-04-27T04:09:51.285 INFO:tasks.workunit.client.1.smithi181.stdout:2/257: link d4/d9/df/d14/f24 d4/d9/d2f/d40/f49 0 2024-04-27T04:09:51.287 INFO:tasks.workunit.client.1.smithi181.stdout:8/257: mkdir d1/d6/d2f/d30/d5c 0 2024-04-27T04:09:51.292 INFO:tasks.workunit.client.1.smithi181.stdout:2/258: mknod d4/d9/d16/c4a 0 2024-04-27T04:09:51.295 INFO:tasks.workunit.client.1.smithi181.stdout:9/274: symlink d0/d44/d4b/l60 0 2024-04-27T04:09:51.296 INFO:tasks.workunit.client.1.smithi181.stdout:2/259: mknod d4/d9/d35/c4b 0 2024-04-27T04:09:51.297 INFO:tasks.workunit.client.1.smithi181.stdout:2/260: write d4/f2b [1034827,91906] 0 2024-04-27T04:09:51.303 INFO:tasks.workunit.client.1.smithi181.stdout:3/196: truncate d0/f5 736712 0 2024-04-27T04:09:51.303 INFO:tasks.workunit.client.1.smithi181.stdout:8/258: link d1/d6/d1a/c41 d1/d6/d1a/d25/c5d 0 2024-04-27T04:09:51.307 INFO:tasks.workunit.client.1.smithi181.stdout:3/197: creat d0/de/d36/f47 x:0 0 0 2024-04-27T04:09:51.308 INFO:tasks.workunit.client.1.smithi181.stdout:3/198: truncate d0/de/d20/f46 36731 0 2024-04-27T04:09:51.308 INFO:tasks.workunit.client.1.smithi181.stdout:3/199: chown d0/l41 22445 1 2024-04-27T04:09:51.309 INFO:tasks.workunit.client.1.smithi181.stdout:8/259: creat d1/d6/d2f/d40/f5e x:0 0 0 2024-04-27T04:09:51.313 INFO:tasks.workunit.client.1.smithi181.stdout:3/200: symlink d0/de/d20/l48 0 2024-04-27T04:09:51.314 INFO:tasks.workunit.client.1.smithi181.stdout:8/260: creat d1/d6/d1a/d25/f5f x:0 0 0 2024-04-27T04:09:51.315 INFO:tasks.workunit.client.1.smithi181.stdout:8/261: fsync d1/f3c 0 2024-04-27T04:09:51.316 INFO:tasks.workunit.client.1.smithi181.stdout:8/262: fsync d1/d6/d1a/f2b 0 2024-04-27T04:09:51.316 INFO:tasks.workunit.client.1.smithi181.stdout:2/261: link d4/d9/d16/c20 d4/d9/d16/d45/c4c 0 2024-04-27T04:09:51.317 INFO:tasks.workunit.client.1.smithi181.stdout:8/263: read d1/d6/d2f/d40/f18 [4008636,69432] 0 2024-04-27T04:09:51.318 INFO:tasks.workunit.client.1.smithi181.stdout:2/262: read d4/d9/df/d14/f24 [314658,31614] 0 2024-04-27T04:09:51.318 INFO:tasks.workunit.client.1.smithi181.stdout:3/201: read d0/f4 [1240351,13172] 0 2024-04-27T04:09:51.319 INFO:tasks.workunit.client.1.smithi181.stdout:2/263: readlink d4/la 0 2024-04-27T04:09:51.319 INFO:tasks.workunit.client.1.smithi181.stdout:2/264: chown d4/c21 4 1 2024-04-27T04:09:51.326 INFO:tasks.workunit.client.1.smithi181.stdout:8/264: creat d1/d6/d2f/d30/f60 x:0 0 0 2024-04-27T04:09:51.328 INFO:tasks.workunit.client.1.smithi181.stdout:9/275: dwrite d0/f10 [0,4194304] 0 2024-04-27T04:09:51.329 INFO:tasks.workunit.client.1.smithi181.stdout:9/276: chown d0/d40/f25 90 1 2024-04-27T04:09:51.335 INFO:tasks.workunit.client.1.smithi181.stdout:2/265: rename d4/la to d4/d9/d2f/l4d 0 2024-04-27T04:09:51.343 INFO:tasks.workunit.client.1.smithi181.stdout:8/265: dread d1/d6/d2f/f3a [0,4194304] 0 2024-04-27T04:09:51.349 INFO:tasks.workunit.client.1.smithi181.stdout:2/266: dread d4/d9/df/d14/f15 [0,4194304] 0 2024-04-27T04:09:51.353 INFO:tasks.workunit.client.1.smithi181.stdout:3/202: dwrite d0/f3c [0,4194304] 0 2024-04-27T04:09:51.353 INFO:tasks.workunit.client.1.smithi181.stdout:9/277: symlink d0/d44/d4b/d43/d5d/l61 0 2024-04-27T04:09:51.357 INFO:tasks.workunit.client.1.smithi181.stdout:2/267: readlink d4/d9/l33 0 2024-04-27T04:09:51.359 INFO:tasks.workunit.client.1.smithi181.stdout:3/203: stat d0/f16 0 2024-04-27T04:09:51.362 INFO:tasks.workunit.client.1.smithi181.stdout:8/266: creat d1/d6/d2f/d30/d5c/f61 x:0 0 0 2024-04-27T04:09:51.363 INFO:tasks.workunit.client.1.smithi181.stdout:8/267: write d1/d6/f55 [542416,15296] 0 2024-04-27T04:09:51.365 INFO:tasks.workunit.client.1.smithi181.stdout:2/268: creat d4/d9/d16/f4e x:0 0 0 2024-04-27T04:09:51.370 INFO:tasks.workunit.client.1.smithi181.stdout:8/268: chown d1/d6/d1a/c41 2176 1 2024-04-27T04:09:51.374 INFO:tasks.workunit.client.1.smithi181.stdout:9/278: creat d0/d44/d4b/f62 x:0 0 0 2024-04-27T04:09:51.376 INFO:tasks.workunit.client.1.smithi181.stdout:8/269: rename d1/d6/d2f/d40/d2a/l58 to d1/l62 0 2024-04-27T04:09:51.380 INFO:tasks.workunit.client.1.smithi181.stdout:9/279: creat d0/d44/d4b/d43/d5d/f63 x:0 0 0 2024-04-27T04:09:51.385 INFO:tasks.workunit.client.1.smithi181.stdout:9/280: mknod d0/d4d/c64 0 2024-04-27T04:09:51.387 INFO:tasks.workunit.client.1.smithi181.stdout:2/269: getdents d4/d9/df/d14 0 2024-04-27T04:09:51.390 INFO:tasks.workunit.client.1.smithi181.stdout:8/270: link d1/d6/d2f/d40/d2a/f42 d1/d6/d2f/d40/d2a/f63 0 2024-04-27T04:09:51.391 INFO:tasks.workunit.client.1.smithi181.stdout:8/271: read - d1/d6/d1a/d25/f5f zero size 2024-04-27T04:09:51.393 INFO:tasks.workunit.client.1.smithi181.stdout:9/281: unlink d0/d44/d4b/c3a 0 2024-04-27T04:09:51.397 INFO:tasks.workunit.client.1.smithi181.stdout:8/272: creat d1/d6/d2f/d40/d2a/f64 x:0 0 0 2024-04-27T04:09:51.398 INFO:tasks.workunit.client.1.smithi181.stdout:3/204: truncate d0/f23 2592980 0 2024-04-27T04:09:51.399 INFO:tasks.workunit.client.1.smithi181.stdout:9/282: dwrite d0/d44/d4f/f58 [0,4194304] 0 2024-04-27T04:09:51.400 INFO:tasks.workunit.client.1.smithi181.stdout:3/205: dread - d0/f16 zero size 2024-04-27T04:09:51.406 INFO:tasks.workunit.client.1.smithi181.stdout:3/206: symlink d0/de/d36/d44/l49 0 2024-04-27T04:09:51.412 INFO:tasks.workunit.client.1.smithi181.stdout:2/270: rename d4/d9/d2f to d4/d9/d4f 0 2024-04-27T04:09:51.430 INFO:tasks.workunit.client.1.smithi181.stdout:9/283: link d0/d44/d4b/d3d/c5c d0/d40/c65 0 2024-04-27T04:09:51.431 INFO:tasks.workunit.client.1.smithi181.stdout:9/284: write d0/f15 [544939,127822] 0 2024-04-27T04:09:51.433 INFO:tasks.workunit.client.1.smithi181.stdout:8/273: getdents d1/d6/d2f 0 2024-04-27T04:09:51.435 INFO:tasks.workunit.client.1.smithi181.stdout:3/207: rmdir d0/d28/d40 0 2024-04-27T04:09:51.437 INFO:tasks.workunit.client.1.smithi181.stdout:2/271: creat d4/f50 x:0 0 0 2024-04-27T04:09:51.446 INFO:tasks.workunit.client.1.smithi181.stdout:2/272: creat d4/d9/d35/f51 x:0 0 0 2024-04-27T04:09:51.447 INFO:tasks.workunit.client.1.smithi181.stdout:8/274: dwrite d1/d6/d2f/d30/f43 [0,4194304] 0 2024-04-27T04:09:51.448 INFO:tasks.workunit.client.1.smithi181.stdout:8/275: fsync d1/d6/f44 0 2024-04-27T04:09:51.452 INFO:tasks.workunit.client.1.smithi181.stdout:8/276: dread d1/d6/d2f/d40/f27 [0,4194304] 0 2024-04-27T04:09:51.459 INFO:tasks.workunit.client.1.smithi181.stdout:2/273: rename d4/d9/df/d14/c2e to d4/d9/d4f/d40/c52 0 2024-04-27T04:09:51.464 INFO:tasks.workunit.client.1.smithi181.stdout:3/208: write d0/f5 [1680332,100150] 0 2024-04-27T04:09:51.465 INFO:tasks.workunit.client.1.smithi181.stdout:2/274: dwrite d4/d9/d16/f23 [0,4194304] 0 2024-04-27T04:09:51.476 INFO:tasks.workunit.client.1.smithi181.stdout:3/209: rmdir d0/de 39 2024-04-27T04:09:51.477 INFO:tasks.workunit.client.1.smithi181.stdout:8/277: dwrite d1/d6/d2f/d40/d2a/f63 [4194304,4194304] 0 2024-04-27T04:09:51.478 INFO:tasks.workunit.client.1.smithi181.stdout:8/278: chown d1/d6/d2f/d30/f43 146 1 2024-04-27T04:09:51.479 INFO:tasks.workunit.client.1.smithi181.stdout:2/275: creat d4/d9/d4f/d40/f53 x:0 0 0 2024-04-27T04:09:51.483 INFO:tasks.workunit.client.1.smithi181.stdout:3/210: readlink d0/de/l2d 0 2024-04-27T04:09:51.488 INFO:tasks.workunit.client.1.smithi181.stdout:8/279: unlink d1/d6/d2f/d40/f18 0 2024-04-27T04:09:51.493 INFO:tasks.workunit.client.1.smithi181.stdout:9/285: rename d0/d44/d4b/d43/d5d to d0/d40/d66 0 2024-04-27T04:09:51.495 INFO:tasks.workunit.client.1.smithi181.stdout:8/280: dwrite d1/d6/d2f/d30/d5c/f61 [0,4194304] 0 2024-04-27T04:09:51.504 INFO:tasks.workunit.client.1.smithi181.stdout:3/211: getdents d0/de/d3f 0 2024-04-27T04:09:51.506 INFO:tasks.workunit.client.1.smithi181.stdout:8/281: symlink d1/d6/d2f/d30/d5c/l65 0 2024-04-27T04:09:51.509 INFO:tasks.workunit.client.1.smithi181.stdout:9/286: fsync d0/d40/d66/f63 0 2024-04-27T04:09:51.512 INFO:tasks.workunit.client.1.smithi181.stdout:3/212: dread d0/f15 [0,4194304] 0 2024-04-27T04:09:51.515 INFO:tasks.workunit.client.1.smithi181.stdout:2/276: link d4/c34 d4/d9/c54 0 2024-04-27T04:09:51.516 INFO:tasks.workunit.client.1.smithi181.stdout:2/277: stat d4/d9/d4f/d40 0 2024-04-27T04:09:51.517 INFO:tasks.workunit.client.1.smithi181.stdout:3/213: dread - d0/f37 zero size 2024-04-27T04:09:51.519 INFO:tasks.workunit.client.1.smithi181.stdout:9/287: symlink d0/l67 0 2024-04-27T04:09:51.520 INFO:tasks.workunit.client.1.smithi181.stdout:2/278: mknod d4/d9/d16/c55 0 2024-04-27T04:09:51.525 INFO:tasks.workunit.client.1.smithi181.stdout:6/131: sync 2024-04-27T04:09:51.525 INFO:tasks.workunit.client.1.smithi181.stdout:2/279: unlink d4/f2b 0 2024-04-27T04:09:51.525 INFO:tasks.workunit.client.1.smithi181.stdout:8/282: truncate d1/d6/d2f/d40/f4c 351369 0 2024-04-27T04:09:51.530 INFO:tasks.workunit.client.1.smithi181.stdout:2/280: dread d4/f8 [0,4194304] 0 2024-04-27T04:09:51.532 INFO:tasks.workunit.client.1.smithi181.stdout:6/132: creat d7/d1b/f29 x:0 0 0 2024-04-27T04:09:51.532 INFO:tasks.workunit.client.1.smithi181.stdout:8/283: dwrite d1/d6/f55 [0,4194304] 0 2024-04-27T04:09:51.535 INFO:tasks.workunit.client.1.smithi181.stdout:3/214: link d0/de/d20/l48 d0/d3e/l4a 0 2024-04-27T04:09:51.538 INFO:tasks.workunit.client.1.smithi181.stdout:9/288: creat d0/f68 x:0 0 0 2024-04-27T04:09:51.539 INFO:tasks.workunit.client.1.smithi181.stdout:6/133: dread d7/dc/f23 [0,4194304] 0 2024-04-27T04:09:51.544 INFO:tasks.workunit.client.1.smithi181.stdout:2/281: truncate d4/d9/d44/f46 261032 0 2024-04-27T04:09:51.545 INFO:tasks.workunit.client.1.smithi181.stdout:1/192: sync 2024-04-27T04:09:51.545 INFO:tasks.workunit.client.1.smithi181.stdout:8/284: dwrite d1/d6/d2f/d40/f17 [0,4194304] 0 2024-04-27T04:09:51.547 INFO:tasks.workunit.client.1.smithi181.stdout:1/193: readlink d1/d2/d5/d1c/d1e/l2a 0 2024-04-27T04:09:51.548 INFO:tasks.workunit.client.1.smithi181.stdout:2/282: rename d4/d9/d4f/d40 to d4/d9/d4f/d40/d56 22 2024-04-27T04:09:51.548 INFO:tasks.workunit.client.1.smithi181.stdout:4/174: sync 2024-04-27T04:09:51.549 INFO:tasks.workunit.client.1.smithi181.stdout:7/154: sync 2024-04-27T04:09:51.549 INFO:tasks.workunit.client.1.smithi181.stdout:5/298: sync 2024-04-27T04:09:51.549 INFO:tasks.workunit.client.1.smithi181.stdout:0/255: sync 2024-04-27T04:09:51.551 INFO:tasks.workunit.client.1.smithi181.stdout:3/215: dwrite d0/de/d20/f46 [0,4194304] 0 2024-04-27T04:09:51.553 INFO:tasks.workunit.client.1.smithi181.stdout:4/175: dread - d0/d15/d25/d2b/f2c zero size 2024-04-27T04:09:51.554 INFO:tasks.workunit.client.1.smithi181.stdout:6/134: creat d7/dc/d1e/f2a x:0 0 0 2024-04-27T04:09:51.555 INFO:tasks.workunit.client.1.smithi181.stdout:8/285: mkdir d1/d6/d2f/d30/d66 0 2024-04-27T04:09:51.558 INFO:tasks.workunit.client.1.smithi181.stdout:8/286: fdatasync d1/d6/f26 0 2024-04-27T04:09:51.561 INFO:tasks.workunit.client.1.smithi181.stdout:5/299: fsync d4/d18/f41 0 2024-04-27T04:09:51.563 INFO:tasks.workunit.client.1.smithi181.stdout:8/287: write d1/d6/d2f/d40/d2a/f63 [5191547,102213] 0 2024-04-27T04:09:51.563 INFO:tasks.workunit.client.1.smithi181.stdout:1/194: dwrite d1/d2/d5/d1c/d1e/d20/f26 [0,4194304] 0 2024-04-27T04:09:51.563 INFO:tasks.workunit.client.1.smithi181.stdout:5/300: fdatasync d4/d18/d67/f50 0 2024-04-27T04:09:51.565 INFO:tasks.workunit.client.1.smithi181.stdout:6/135: dread f6 [0,4194304] 0 2024-04-27T04:09:51.565 INFO:tasks.workunit.client.1.smithi181.stdout:8/288: fdatasync d1/d6/f44 0 2024-04-27T04:09:51.569 INFO:tasks.workunit.client.1.smithi181.stdout:2/283: chown d4/d9/d4f/d40/c52 2 1 2024-04-27T04:09:51.574 INFO:tasks.workunit.client.1.smithi181.stdout:3/216: creat d0/de/f4b x:0 0 0 2024-04-27T04:09:51.582 INFO:tasks.workunit.client.1.smithi181.stdout:0/256: symlink dc/l64 0 2024-04-27T04:09:51.588 INFO:tasks.workunit.client.1.smithi181.stdout:6/136: readlink d7/dc/l17 0 2024-04-27T04:09:51.590 INFO:tasks.workunit.client.1.smithi181.stdout:1/195: creat d1/d2/d5/d1c/f3b x:0 0 0 2024-04-27T04:09:51.592 INFO:tasks.workunit.client.1.smithi181.stdout:5/301: mkdir d4/d18/d67/d71 0 2024-04-27T04:09:51.592 INFO:tasks.workunit.client.1.smithi181.stdout:9/289: write d0/d44/f4a [973429,8874] 0 2024-04-27T04:09:51.593 INFO:tasks.workunit.client.1.smithi181.stdout:5/302: read d4/d18/f41 [3825634,111693] 0 2024-04-27T04:09:51.594 INFO:tasks.workunit.client.1.smithi181.stdout:5/303: chown d4/de/d2f/c21 566 1 2024-04-27T04:09:51.598 INFO:tasks.workunit.client.1.smithi181.stdout:2/284: readlink d4/d9/l31 0 2024-04-27T04:09:51.598 INFO:tasks.workunit.client.1.smithi181.stdout:2/285: readlink d4/d9/df/l22 0 2024-04-27T04:09:51.605 INFO:tasks.workunit.client.1.smithi181.stdout:0/257: rmdir dc/d1f/d3b/d4a 39 2024-04-27T04:09:51.606 INFO:tasks.workunit.client.1.smithi181.stdout:3/217: dwrite d0/f1 [0,4194304] 0 2024-04-27T04:09:51.608 INFO:tasks.workunit.client.1.smithi181.stdout:6/137: mkdir d7/dc/d1e/d2b 0 2024-04-27T04:09:51.610 INFO:tasks.workunit.client.1.smithi181.stdout:7/155: write d1/d22/d15/f28 [69114,107478] 0 2024-04-27T04:09:51.611 INFO:tasks.workunit.client.1.smithi181.stdout:1/196: mknod d1/d2/d5/d6/db/c3c 0 2024-04-27T04:09:51.681 INFO:tasks.workunit.client.0.smithi162.stderr:++ readlink -f fsstress 2024-04-27T04:09:51.683 INFO:tasks.workunit.client.0.smithi162.stderr:+ BIN=/home/ubuntu/cephtest/mnt.0/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/fs/fsstress/fsstress 2024-04-27T04:09:51.683 INFO:tasks.workunit.client.0.smithi162.stderr:+ popd 2024-04-27T04:09:51.684 INFO:tasks.workunit.client.0.smithi162.stdout:~/cephtest/mnt.0/client.0/tmp/fsstress ~/cephtest/mnt.0/client.0/tmp 2024-04-27T04:09:51.684 INFO:tasks.workunit.client.0.smithi162.stderr:+ popd 2024-04-27T04:09:51.684 INFO:tasks.workunit.client.0.smithi162.stdout:~/cephtest/mnt.0/client.0/tmp 2024-04-27T04:09:51.684 INFO:tasks.workunit.client.0.smithi162.stderr:++ mktemp -d -p . 2024-04-27T04:09:51.687 INFO:tasks.workunit.client.0.smithi162.stderr:+ T=./tmp.jpOSNEf0nQ 2024-04-27T04:09:51.687 INFO:tasks.workunit.client.0.smithi162.stderr:+ /home/ubuntu/cephtest/mnt.0/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/fs/fsstress/fsstress -d ./tmp.jpOSNEf0nQ -l 1 -n 1000 -p 10 -v 2024-04-27T04:09:51.690 INFO:tasks.workunit.client.0.smithi162.stdout:seed = 1713826343 2024-04-27T04:09:51.693 INFO:tasks.workunit.client.0.smithi162.stdout:0/0: dread - no filename 2024-04-27T04:09:51.694 INFO:tasks.workunit.client.0.smithi162.stdout:1/0: write - no filename 2024-04-27T04:09:51.694 INFO:tasks.workunit.client.0.smithi162.stdout:1/1: dread - no filename 2024-04-27T04:09:51.694 INFO:tasks.workunit.client.0.smithi162.stdout:1/2: chown . 19990 1 2024-04-27T04:09:51.694 INFO:tasks.workunit.client.0.smithi162.stdout:1/3: fdatasync - no filename 2024-04-27T04:09:51.695 INFO:tasks.workunit.client.0.smithi162.stdout:0/1: creat f0 x:0 0 0 2024-04-27T04:09:51.696 INFO:tasks.workunit.client.0.smithi162.stdout:1/4: mkdir d0 0 2024-04-27T04:09:51.696 INFO:tasks.workunit.client.0.smithi162.stdout:1/5: chown d0 0 1 2024-04-27T04:09:51.696 INFO:tasks.workunit.client.0.smithi162.stdout:1/6: dwrite - no filename 2024-04-27T04:09:51.696 INFO:tasks.workunit.client.0.smithi162.stdout:1/7: dwrite - no filename 2024-04-27T04:09:51.697 INFO:tasks.workunit.client.0.smithi162.stdout:1/8: stat d0 0 2024-04-27T04:09:51.698 INFO:tasks.workunit.client.0.smithi162.stdout:2/0: mkdir d0 0 2024-04-27T04:09:51.698 INFO:tasks.workunit.client.0.smithi162.stdout:3/0: fdatasync - no filename 2024-04-27T04:09:51.698 INFO:tasks.workunit.client.0.smithi162.stdout:3/1: chown . 422501 1 2024-04-27T04:09:51.698 INFO:tasks.workunit.client.0.smithi162.stdout:3/2: dwrite - no filename 2024-04-27T04:09:51.698 INFO:tasks.workunit.client.0.smithi162.stdout:3/3: rename - no filename 2024-04-27T04:09:51.698 INFO:tasks.workunit.client.0.smithi162.stdout:3/4: unlink - no file 2024-04-27T04:09:51.699 INFO:tasks.workunit.client.0.smithi162.stdout:4/0: dread - no filename 2024-04-27T04:09:51.699 INFO:tasks.workunit.client.0.smithi162.stdout:4/1: dwrite - no filename 2024-04-27T04:09:51.699 INFO:tasks.workunit.client.0.smithi162.stdout:4/2: rmdir - no directory 2024-04-27T04:09:51.699 INFO:tasks.workunit.client.0.smithi162.stdout:4/3: readlink - no filename 2024-04-27T04:09:51.700 INFO:tasks.workunit.client.0.smithi162.stdout:1/9: mknod d0/c1 0 2024-04-27T04:09:51.700 INFO:tasks.workunit.client.0.smithi162.stdout:1/10: dwrite - no filename 2024-04-27T04:09:51.701 INFO:tasks.workunit.client.0.smithi162.stdout:5/0: rmdir - no directory 2024-04-27T04:09:51.701 INFO:tasks.workunit.client.0.smithi162.stdout:5/1: rmdir - no directory 2024-04-27T04:09:51.701 INFO:tasks.workunit.client.0.smithi162.stdout:5/2: readlink - no filename 2024-04-27T04:09:51.701 INFO:tasks.workunit.client.0.smithi162.stdout:5/3: dread - no filename 2024-04-27T04:09:51.702 INFO:tasks.workunit.client.0.smithi162.stdout:5/4: dread - no filename 2024-04-27T04:09:51.702 INFO:tasks.workunit.client.0.smithi162.stdout:5/5: chown . 49846 1 2024-04-27T04:09:51.702 INFO:tasks.workunit.client.0.smithi162.stdout:5/6: stat - no entries 2024-04-27T04:09:51.702 INFO:tasks.workunit.client.0.smithi162.stdout:2/1: creat d0/f1 x:0 0 0 2024-04-27T04:09:51.702 INFO:tasks.workunit.client.0.smithi162.stdout:3/5: creat f0 x:0 0 0 2024-04-27T04:09:51.703 INFO:tasks.workunit.client.1.smithi181.stdout:7/156: mkdir d1/d2/d1c/d2d 0 2024-04-27T04:09:51.703 INFO:tasks.workunit.client.0.smithi162.stdout:0/2: dwrite f0 [0,4194304] 0 2024-04-27T04:09:51.704 INFO:tasks.workunit.client.0.smithi162.stdout:1/11: creat d0/f2 x:0 0 0 2024-04-27T04:09:51.704 INFO:tasks.workunit.client.0.smithi162.stdout:3/6: write f0 [868211,99186] 0 2024-04-27T04:09:51.704 INFO:tasks.workunit.client.0.smithi162.stdout:3/7: rmdir - no directory 2024-04-27T04:09:51.704 INFO:tasks.workunit.client.0.smithi162.stdout:0/3: truncate f0 5051628 0 2024-04-27T04:09:51.704 INFO:tasks.workunit.client.1.smithi181.stdout:1/197: creat d1/d2/d5/d6/f3d x:0 0 0 2024-04-27T04:09:51.704 INFO:tasks.workunit.client.1.smithi181.stdout:9/290: mknod d0/d44/d4b/d5f/c69 0 2024-04-27T04:09:51.705 INFO:tasks.workunit.client.0.smithi162.stdout:3/8: chown f0 92 1 2024-04-27T04:09:51.705 INFO:tasks.workunit.client.1.smithi181.stdout:2/286: symlink d4/d9/l57 0 2024-04-27T04:09:51.705 INFO:tasks.workunit.client.0.smithi162.stdout:6/0: unlink - no file 2024-04-27T04:09:51.706 INFO:tasks.workunit.client.0.smithi162.stdout:2/2: creat d0/f2 x:0 0 0 2024-04-27T04:09:51.707 INFO:tasks.workunit.client.0.smithi162.stdout:2/3: fdatasync d0/f2 0 2024-04-27T04:09:51.707 INFO:tasks.workunit.client.1.smithi181.stdout:4/176: getdents d0/d15/d25 0 2024-04-27T04:09:51.707 INFO:tasks.workunit.client.1.smithi181.stdout:0/258: mknod dc/d1f/d3b/d4a/c65 0 2024-04-27T04:09:51.708 INFO:tasks.workunit.client.1.smithi181.stdout:3/218: creat d0/d3e/f4c x:0 0 0 2024-04-27T04:09:51.708 INFO:tasks.workunit.client.0.smithi162.stdout:1/12: symlink d0/l3 0 2024-04-27T04:09:51.708 INFO:tasks.workunit.client.0.smithi162.stdout:3/9: fdatasync f0 0 2024-04-27T04:09:51.708 INFO:tasks.workunit.client.0.smithi162.stdout:3/10: truncate f0 1004517 0 2024-04-27T04:09:51.709 INFO:tasks.workunit.client.0.smithi162.stdout:5/7: creat f0 x:0 0 0 2024-04-27T04:09:51.710 INFO:tasks.workunit.client.0.smithi162.stdout:0/4: dread f0 [0,4194304] 0 2024-04-27T04:09:51.710 INFO:tasks.workunit.client.0.smithi162.stdout:5/8: write f0 [265011,51178] 0 2024-04-27T04:09:51.712 INFO:tasks.workunit.client.1.smithi181.stdout:1/198: creat d1/d2/d5/d1c/f3e x:0 0 0 2024-04-27T04:09:51.714 INFO:tasks.workunit.client.0.smithi162.stdout:7/0: rename - no filename 2024-04-27T04:09:51.714 INFO:tasks.workunit.client.0.smithi162.stdout:7/1: fsync - no filename 2024-04-27T04:09:51.715 INFO:tasks.workunit.client.1.smithi181.stdout:8/289: dwrite d1/d6/d2f/f45 [0,4194304] 0 2024-04-27T04:09:51.716 INFO:tasks.workunit.client.1.smithi181.stdout:9/291: stat d0/d40/l3f 0 2024-04-27T04:09:51.718 INFO:tasks.workunit.client.0.smithi162.stdout:5/9: dwrite f0 [0,4194304] 0 2024-04-27T04:09:51.723 INFO:tasks.workunit.client.1.smithi181.stdout:1/199: dwrite d1/d2/d5/d1c/f3b [0,4194304] 0 2024-04-27T04:09:51.724 INFO:tasks.workunit.client.1.smithi181.stdout:7/157: symlink d1/l2e 0 2024-04-27T04:09:51.726 INFO:tasks.workunit.client.1.smithi181.stdout:7/158: truncate d1/d22/f26 838656 0 2024-04-27T04:09:51.727 INFO:tasks.workunit.client.1.smithi181.stdout:7/159: chown d1/d2/f27 7 1 2024-04-27T04:09:51.727 INFO:tasks.workunit.client.1.smithi181.stdout:3/219: symlink d0/de/d36/l4d 0 2024-04-27T04:09:51.732 INFO:tasks.workunit.client.0.smithi162.stdout:3/11: mkdir d1 0 2024-04-27T04:09:51.733 INFO:tasks.workunit.client.0.smithi162.stdout:3/12: truncate f0 1157849 0 2024-04-27T04:09:51.734 INFO:tasks.workunit.client.1.smithi181.stdout:4/177: creat d0/d15/d25/d2b/d7/f31 x:0 0 0 2024-04-27T04:09:51.736 INFO:tasks.workunit.client.0.smithi162.stdout:0/5: creat f1 x:0 0 0 2024-04-27T04:09:51.739 INFO:tasks.workunit.client.1.smithi181.stdout:8/290: chown d1/l62 7354 1 2024-04-27T04:09:51.740 INFO:tasks.workunit.client.0.smithi162.stdout:8/0: truncate - no filename 2024-04-27T04:09:51.740 INFO:tasks.workunit.client.0.smithi162.stdout:8/1: fdatasync - no filename 2024-04-27T04:09:51.741 INFO:tasks.workunit.client.0.smithi162.stdout:8/2: chown . 5 1 2024-04-27T04:09:51.741 INFO:tasks.workunit.client.0.smithi162.stdout:8/3: chown . 31791 1 2024-04-27T04:09:51.741 INFO:tasks.workunit.client.1.smithi181.stdout:9/292: creat d0/d44/d4b/d50/f6a x:0 0 0 2024-04-27T04:09:51.742 INFO:tasks.workunit.client.1.smithi181.stdout:6/138: link d7/dc/c1a d7/dc/d1e/d2b/c2c 0 2024-04-27T04:09:51.745 INFO:tasks.workunit.client.0.smithi162.stdout:5/10: mknod c1 0 2024-04-27T04:09:51.746 INFO:tasks.workunit.client.1.smithi181.stdout:2/287: mknod d4/d9/df/c58 0 2024-04-27T04:09:51.747 INFO:tasks.workunit.client.0.smithi162.stdout:6/1: creat f0 x:0 0 0 2024-04-27T04:09:51.749 INFO:tasks.workunit.client.1.smithi181.stdout:1/200: creat d1/d2/d5/d2c/f3f x:0 0 0 2024-04-27T04:09:51.753 INFO:tasks.workunit.client.1.smithi181.stdout:2/288: dwrite d4/f2c [0,4194304] 0 2024-04-27T04:09:51.754 INFO:tasks.workunit.client.0.smithi162.stdout:0/6: creat f2 x:0 0 0 2024-04-27T04:09:51.755 INFO:tasks.workunit.client.1.smithi181.stdout:4/178: creat d0/d15/f32 x:0 0 0 2024-04-27T04:09:51.755 INFO:tasks.workunit.client.1.smithi181.stdout:2/289: write d4/d9/df/f41 [822995,105000] 0 2024-04-27T04:09:51.757 INFO:tasks.workunit.client.0.smithi162.stdout:9/0: dwrite - no filename 2024-04-27T04:09:51.758 INFO:tasks.workunit.client.1.smithi181.stdout:8/291: mknod d1/d6/d2f/c67 0 2024-04-27T04:09:51.760 INFO:tasks.workunit.client.1.smithi181.stdout:7/160: dread d1/f8 [0,4194304] 0 2024-04-27T04:09:51.760 INFO:tasks.workunit.client.1.smithi181.stdout:2/290: dread d4/f8 [0,4194304] 0 2024-04-27T04:09:51.762 INFO:tasks.workunit.client.0.smithi162.stdout:7/2: creat f0 x:0 0 0 2024-04-27T04:09:51.763 INFO:tasks.workunit.client.0.smithi162.stdout:7/3: write f0 [87326,78802] 0 2024-04-27T04:09:51.763 INFO:tasks.workunit.client.1.smithi181.stdout:6/139: mknod d7/dc/d1e/c2d 0 2024-04-27T04:09:51.764 INFO:tasks.workunit.client.1.smithi181.stdout:8/292: dread d1/d20/f24 [4194304,4194304] 0 2024-04-27T04:09:51.765 INFO:tasks.workunit.client.1.smithi181.stdout:8/293: chown d1/d20/l50 24181 1 2024-04-27T04:09:51.765 INFO:tasks.workunit.client.0.smithi162.stdout:5/11: creat f2 x:0 0 0 2024-04-27T04:09:51.765 INFO:tasks.workunit.client.1.smithi181.stdout:6/140: truncate d7/dc/d1e/f2a 499130 0 2024-04-27T04:09:51.766 INFO:tasks.workunit.client.0.smithi162.stdout:5/12: write f2 [853849,29931] 0 2024-04-27T04:09:51.766 INFO:tasks.workunit.client.0.smithi162.stdout:5/13: chown f0 110 1 2024-04-27T04:09:51.768 INFO:tasks.workunit.client.0.smithi162.stdout:6/2: mknod c1 0 2024-04-27T04:09:51.769 INFO:tasks.workunit.client.0.smithi162.stdout:5/14: dread f0 [0,4194304] 0 2024-04-27T04:09:51.770 INFO:tasks.workunit.client.0.smithi162.stdout:5/15: write f0 [1458362,102065] 0 2024-04-27T04:09:51.771 INFO:tasks.workunit.client.0.smithi162.stdout:7/4: dread f0 [0,4194304] 0 2024-04-27T04:09:51.771 INFO:tasks.workunit.client.0.smithi162.stdout:7/5: fdatasync f0 0 2024-04-27T04:09:51.773 INFO:tasks.workunit.client.1.smithi181.stdout:0/259: rename dc/f1d to dc/d1f/d3b/f66 0 2024-04-27T04:09:51.774 INFO:tasks.workunit.client.0.smithi162.stdout:8/4: mkdir d0 0 2024-04-27T04:09:51.780 INFO:tasks.workunit.client.0.smithi162.stdout:6/3: creat f2 x:0 0 0 2024-04-27T04:09:51.780 INFO:tasks.workunit.client.0.smithi162.stdout:6/4: write f2 [151721,124401] 0 2024-04-27T04:09:51.782 INFO:tasks.workunit.client.0.smithi162.stdout:9/1: creat f0 x:0 0 0 2024-04-27T04:09:51.784 INFO:tasks.workunit.client.1.smithi181.stdout:7/161: mknod d1/d22/c2f 0 2024-04-27T04:09:51.785 INFO:tasks.workunit.client.1.smithi181.stdout:7/162: write d1/d2/f12 [4014125,13503] 0 2024-04-27T04:09:51.789 INFO:tasks.workunit.client.1.smithi181.stdout:3/220: creat d0/f4e x:0 0 0 2024-04-27T04:09:51.792 INFO:tasks.workunit.client.0.smithi162.stdout:6/5: rename f0 to f3 0 2024-04-27T04:09:51.793 INFO:tasks.workunit.client.0.smithi162.stdout:9/2: mknod c1 0 2024-04-27T04:09:51.794 INFO:tasks.workunit.client.0.smithi162.stdout:8/5: creat d0/f1 x:0 0 0 2024-04-27T04:09:51.795 INFO:tasks.workunit.client.1.smithi181.stdout:4/179: fsync d0/d15/f2 0 2024-04-27T04:09:51.801 INFO:tasks.workunit.client.0.smithi162.stdout:8/6: dwrite d0/f1 [0,4194304] 0 2024-04-27T04:09:51.804 INFO:tasks.workunit.client.1.smithi181.stdout:6/141: link d7/f8 d7/d27/f2e 0 2024-04-27T04:09:51.805 INFO:tasks.workunit.client.0.smithi162.stdout:8/7: creat d0/f2 x:0 0 0 2024-04-27T04:09:51.806 INFO:tasks.workunit.client.0.smithi162.stdout:8/8: chown d0/f1 1072848 1 2024-04-27T04:09:51.807 INFO:tasks.workunit.client.0.smithi162.stdout:8/9: write d0/f1 [2570003,52386] 0 2024-04-27T04:09:51.807 INFO:tasks.workunit.client.0.smithi162.stdout:8/10: dread - d0/f2 zero size 2024-04-27T04:09:51.808 INFO:tasks.workunit.client.0.smithi162.stdout:8/11: chown d0/f1 19296 1 2024-04-27T04:09:51.808 INFO:tasks.workunit.client.1.smithi181.stdout:3/221: dread d0/de/f10 [0,4194304] 0 2024-04-27T04:09:51.810 INFO:tasks.workunit.client.1.smithi181.stdout:0/260: mknod dc/d54/d57/c67 0 2024-04-27T04:09:51.811 INFO:tasks.workunit.client.1.smithi181.stdout:0/261: write dc/d1e/f2e [540684,124795] 0 2024-04-27T04:09:51.812 INFO:tasks.workunit.client.1.smithi181.stdout:0/262: fdatasync dc/d1e/d26/d41/f4e 0 2024-04-27T04:09:51.813 INFO:tasks.workunit.client.0.smithi162.stdout:2/4: fsync d0/f1 0 2024-04-27T04:09:51.814 INFO:tasks.workunit.client.1.smithi181.stdout:0/263: write dc/d1e/d26/d41/f4e [3162715,94501] 0 2024-04-27T04:09:51.814 INFO:tasks.workunit.client.0.smithi162.stdout:2/5: chown d0/f1 482966 1 2024-04-27T04:09:51.815 INFO:tasks.workunit.client.1.smithi181.stdout:4/180: chown d0/d15/de/l13 2 1 2024-04-27T04:09:51.815 INFO:tasks.workunit.client.0.smithi162.stdout:3/13: fsync f0 0 2024-04-27T04:09:51.815 INFO:tasks.workunit.client.0.smithi162.stdout:1/13: fsync d0/f2 0 2024-04-27T04:09:51.816 INFO:tasks.workunit.client.0.smithi162.stdout:1/14: dread - d0/f2 zero size 2024-04-27T04:09:51.817 INFO:tasks.workunit.client.1.smithi181.stdout:2/291: creat d4/f59 x:0 0 0 2024-04-27T04:09:51.817 INFO:tasks.workunit.client.0.smithi162.stdout:1/15: write d0/f2 [863345,32097] 0 2024-04-27T04:09:51.818 INFO:tasks.workunit.client.0.smithi162.stdout:2/6: mkdir d0/d3 0 2024-04-27T04:09:51.819 INFO:tasks.workunit.client.1.smithi181.stdout:7/163: link d1/d22/d15/f2a d1/d22/d15/d13/f30 0 2024-04-27T04:09:51.820 INFO:tasks.workunit.client.1.smithi181.stdout:7/164: dread - d1/d2/f27 zero size 2024-04-27T04:09:51.820 INFO:tasks.workunit.client.1.smithi181.stdout:7/165: readlink d1/d22/l23 0 2024-04-27T04:09:51.821 INFO:tasks.workunit.client.1.smithi181.stdout:6/142: mkdir d7/d2f 0 2024-04-27T04:09:51.823 INFO:tasks.workunit.client.0.smithi162.stdout:2/7: rename d0/d3 to d0/d4 0 2024-04-27T04:09:51.824 INFO:tasks.workunit.client.0.smithi162.stdout:2/8: fsync d0/f2 0 2024-04-27T04:09:51.824 INFO:tasks.workunit.client.0.smithi162.stdout:2/9: chown d0/f2 1620980 1 2024-04-27T04:09:51.825 INFO:tasks.workunit.client.0.smithi162.stdout:2/10: write d0/f2 [917818,9520] 0 2024-04-27T04:09:51.825 INFO:tasks.workunit.client.0.smithi162.stdout:3/14: mkdir d1/d2 0 2024-04-27T04:09:51.825 INFO:tasks.workunit.client.0.smithi162.stdout:2/11: chown d0 88 1 2024-04-27T04:09:51.826 INFO:tasks.workunit.client.0.smithi162.stdout:3/15: stat d1/d2 0 2024-04-27T04:09:51.829 INFO:tasks.workunit.client.1.smithi181.stdout:0/264: mkdir dc/d1e/d26/d68 0 2024-04-27T04:09:51.830 INFO:tasks.workunit.client.0.smithi162.stdout:0/7: fsync f1 0 2024-04-27T04:09:51.830 INFO:tasks.workunit.client.0.smithi162.stdout:0/8: rmdir - no directory 2024-04-27T04:09:51.831 INFO:tasks.workunit.client.1.smithi181.stdout:4/181: symlink d0/d15/d25/l33 0 2024-04-27T04:09:51.836 INFO:tasks.workunit.client.1.smithi181.stdout:7/166: write d1/d22/d15/f2a [174676,101167] 0 2024-04-27T04:09:51.838 INFO:tasks.workunit.client.0.smithi162.stdout:2/12: rename d0/f2 to d0/d4/f5 0 2024-04-27T04:09:51.838 INFO:tasks.workunit.client.0.smithi162.stdout:2/13: chown d0/f1 60894 1 2024-04-27T04:09:51.840 INFO:tasks.workunit.client.1.smithi181.stdout:6/143: creat d7/dc/d1e/f30 x:0 0 0 2024-04-27T04:09:51.840 INFO:tasks.workunit.client.1.smithi181.stdout:5/304: sync 2024-04-27T04:09:51.841 INFO:tasks.workunit.client.0.smithi162.stdout:0/9: symlink l3 0 2024-04-27T04:09:51.846 INFO:tasks.workunit.client.1.smithi181.stdout:0/265: dread dc/d1f/d3b/f44 [0,4194304] 0 2024-04-27T04:09:51.847 INFO:tasks.workunit.client.1.smithi181.stdout:0/266: read - dc/f5e zero size 2024-04-27T04:09:51.848 INFO:tasks.workunit.client.1.smithi181.stdout:5/305: dwrite d4/d10/f57 [0,4194304] 0 2024-04-27T04:09:51.848 INFO:tasks.workunit.client.0.smithi162.stdout:0/10: symlink l4 0 2024-04-27T04:09:51.849 INFO:tasks.workunit.client.1.smithi181.stdout:0/267: truncate dc/d1e/f30 1219119 0 2024-04-27T04:09:51.853 INFO:tasks.workunit.client.0.smithi162.stdout:2/14: link d0/d4/f5 d0/f6 0 2024-04-27T04:09:51.856 INFO:tasks.workunit.client.1.smithi181.stdout:7/167: mknod d1/df/c31 0 2024-04-27T04:09:51.856 INFO:tasks.workunit.client.0.smithi162.stdout:0/11: dwrite f1 [0,4194304] 0 2024-04-27T04:09:51.856 INFO:tasks.workunit.client.1.smithi181.stdout:7/168: dread - d1/d2/d1c/f2c zero size 2024-04-27T04:09:51.858 INFO:tasks.workunit.client.0.smithi162.stdout:2/15: creat d0/f7 x:0 0 0 2024-04-27T04:09:51.859 INFO:tasks.workunit.client.0.smithi162.stdout:2/16: chown d0/d4/f5 844218999 1 2024-04-27T04:09:51.859 INFO:tasks.workunit.client.1.smithi181.stdout:6/144: dread f6 [0,4194304] 0 2024-04-27T04:09:51.861 INFO:tasks.workunit.client.0.smithi162.stdout:2/17: write d0/f1 [710546,112025] 0 2024-04-27T04:09:51.863 INFO:tasks.workunit.client.1.smithi181.stdout:8/294: readlink d1/d6/d2f/d40/d2a/l54 0 2024-04-27T04:09:51.864 INFO:tasks.workunit.client.1.smithi181.stdout:5/306: mkdir d4/d10/d72 0 2024-04-27T04:09:51.865 INFO:tasks.workunit.client.1.smithi181.stdout:0/268: readlink dc/d1e/d26/d41/l55 0 2024-04-27T04:09:51.867 INFO:tasks.workunit.client.0.smithi162.stdout:2/18: mknod d0/d4/c8 0 2024-04-27T04:09:51.870 INFO:tasks.workunit.client.0.smithi162.stdout:5/16: dwrite f0 [4194304,4194304] 0 2024-04-27T04:09:51.871 INFO:tasks.workunit.client.0.smithi162.stdout:0/12: dwrite f0 [0,4194304] 0 2024-04-27T04:09:51.871 INFO:tasks.workunit.client.1.smithi181.stdout:9/293: dwrite d0/f8 [0,4194304] 0 2024-04-27T04:09:51.872 INFO:tasks.workunit.client.0.smithi162.stdout:5/17: chown f2 480 1 2024-04-27T04:09:51.872 INFO:tasks.workunit.client.1.smithi181.stdout:8/295: symlink d1/d6/d1a/l68 0 2024-04-27T04:09:51.874 INFO:tasks.workunit.client.1.smithi181.stdout:8/296: read - d1/d6/d2f/d40/d2a/f64 zero size 2024-04-27T04:09:51.875 INFO:tasks.workunit.client.1.smithi181.stdout:1/201: truncate d1/d2/d5/d1c/ff 2462406 0 2024-04-27T04:09:51.876 INFO:tasks.workunit.client.0.smithi162.stdout:2/19: dread d0/f6 [0,4194304] 0 2024-04-27T04:09:51.877 INFO:tasks.workunit.client.1.smithi181.stdout:0/269: creat dc/d54/d57/f69 x:0 0 0 2024-04-27T04:09:51.878 INFO:tasks.workunit.client.0.smithi162.stdout:6/6: getdents . 0 2024-04-27T04:09:51.879 INFO:tasks.workunit.client.0.smithi162.stdout:6/7: write f2 [396443,92037] 0 2024-04-27T04:09:51.880 INFO:tasks.workunit.client.1.smithi181.stdout:6/145: dwrite d7/f26 [4194304,4194304] 0 2024-04-27T04:09:51.884 INFO:tasks.workunit.client.1.smithi181.stdout:7/169: getdents d1/df/d1a 0 2024-04-27T04:09:51.886 INFO:tasks.workunit.client.0.smithi162.stdout:0/13: mkdir d5 0 2024-04-27T04:09:51.886 INFO:tasks.workunit.client.1.smithi181.stdout:4/182: write d0/d15/f2 [93650,61306] 0 2024-04-27T04:09:51.887 INFO:tasks.workunit.client.0.smithi162.stdout:0/14: write f0 [1946075,66542] 0 2024-04-27T04:09:51.890 INFO:tasks.workunit.client.0.smithi162.stdout:5/18: rename c1 to c3 0 2024-04-27T04:09:51.891 INFO:tasks.workunit.client.1.smithi181.stdout:8/297: truncate d1/d6/d2f/d30/f60 416446 0 2024-04-27T04:09:51.892 INFO:tasks.workunit.client.1.smithi181.stdout:1/202: creat d1/d2/d5/d6/db/f40 x:0 0 0 2024-04-27T04:09:51.894 INFO:tasks.workunit.client.0.smithi162.stdout:8/12: fsync d0/f2 0 2024-04-27T04:09:51.895 INFO:tasks.workunit.client.0.smithi162.stdout:8/13: fdatasync d0/f2 0 2024-04-27T04:09:51.895 INFO:tasks.workunit.client.0.smithi162.stdout:8/14: dread - d0/f2 zero size 2024-04-27T04:09:51.897 INFO:tasks.workunit.client.1.smithi181.stdout:6/146: mkdir d7/d31 0 2024-04-27T04:09:51.898 INFO:tasks.workunit.client.0.smithi162.stdout:6/8: mknod c4 0 2024-04-27T04:09:51.900 INFO:tasks.workunit.client.1.smithi181.stdout:7/170: mknod d1/df/c32 0 2024-04-27T04:09:51.901 INFO:tasks.workunit.client.1.smithi181.stdout:7/171: write d1/d2/f12 [1389099,76734] 0 2024-04-27T04:09:51.903 INFO:tasks.workunit.client.1.smithi181.stdout:9/294: unlink d0/d40/f1c 0 2024-04-27T04:09:51.905 INFO:tasks.workunit.client.1.smithi181.stdout:6/147: dread f0 [0,4194304] 0 2024-04-27T04:09:51.906 INFO:tasks.workunit.client.1.smithi181.stdout:9/295: truncate d0/d44/f51 901777 0 2024-04-27T04:09:51.906 INFO:tasks.workunit.client.0.smithi162.stdout:0/15: rename l3 to d5/l6 0 2024-04-27T04:09:51.906 INFO:tasks.workunit.client.0.smithi162.stdout:8/15: rename d0 to d0/d3 22 2024-04-27T04:09:51.906 INFO:tasks.workunit.client.0.smithi162.stdout:8/16: stat d0/f1 0 2024-04-27T04:09:51.907 INFO:tasks.workunit.client.0.smithi162.stdout:8/17: dread - d0/f2 zero size 2024-04-27T04:09:51.907 INFO:tasks.workunit.client.0.smithi162.stdout:8/18: write d0/f2 [161266,38782] 0 2024-04-27T04:09:51.907 INFO:tasks.workunit.client.1.smithi181.stdout:4/183: creat d0/d15/de/f34 x:0 0 0 2024-04-27T04:09:51.909 INFO:tasks.workunit.client.0.smithi162.stdout:5/19: rename f2 to f4 0 2024-04-27T04:09:51.912 INFO:tasks.workunit.client.1.smithi181.stdout:9/296: dwrite d0/fc [4194304,4194304] 0 2024-04-27T04:09:51.913 INFO:tasks.workunit.client.0.smithi162.stdout:2/20: link d0/f1 d0/d4/f9 0 2024-04-27T04:09:51.915 INFO:tasks.workunit.client.1.smithi181.stdout:1/203: creat d1/d2/d5/d1c/d1e/d20/f41 x:0 0 0 2024-04-27T04:09:51.916 INFO:tasks.workunit.client.0.smithi162.stdout:1/16: truncate d0/f2 471442 0 2024-04-27T04:09:51.917 INFO:tasks.workunit.client.0.smithi162.stdout:6/9: creat f5 x:0 0 0 2024-04-27T04:09:51.919 INFO:tasks.workunit.client.1.smithi181.stdout:0/270: link dc/d1e/f51 dc/d1e/d26/f6a 0 2024-04-27T04:09:51.922 INFO:tasks.workunit.client.0.smithi162.stdout:0/16: rename f2 to d5/f7 0 2024-04-27T04:09:51.925 INFO:tasks.workunit.client.1.smithi181.stdout:2/292: dwrite d4/d9/df/d14/f15 [0,4194304] 0 2024-04-27T04:09:51.927 INFO:tasks.workunit.client.1.smithi181.stdout:3/222: truncate d0/f1 2225611 0 2024-04-27T04:09:51.929 INFO:tasks.workunit.client.0.smithi162.stdout:5/20: write f4 [1021007,1582] 0 2024-04-27T04:09:51.930 INFO:tasks.workunit.client.0.smithi162.stdout:5/21: truncate f4 1502147 0 2024-04-27T04:09:51.931 INFO:tasks.workunit.client.1.smithi181.stdout:6/148: creat d7/dc/d1c/f32 x:0 0 0 2024-04-27T04:09:51.933 INFO:tasks.workunit.client.0.smithi162.stdout:2/21: creat d0/fa x:0 0 0 2024-04-27T04:09:51.934 INFO:tasks.workunit.client.1.smithi181.stdout:2/293: dwrite d4/d9/d16/f4e [0,4194304] 0 2024-04-27T04:09:51.934 INFO:tasks.workunit.client.1.smithi181.stdout:4/184: mkdir d0/d15/d25/d2b/d7/d35 0 2024-04-27T04:09:51.939 INFO:tasks.workunit.client.1.smithi181.stdout:8/298: symlink d1/d6/d2f/l69 0 2024-04-27T04:09:51.939 INFO:tasks.workunit.client.1.smithi181.stdout:1/204: dread d1/d2/d5/d6/db/f19 [0,4194304] 0 2024-04-27T04:09:51.940 INFO:tasks.workunit.client.0.smithi162.stdout:6/10: mkdir d6 0 2024-04-27T04:09:51.942 INFO:tasks.workunit.client.1.smithi181.stdout:8/299: read d1/d6/d1a/f2b [4090749,108890] 0 2024-04-27T04:09:51.942 INFO:tasks.workunit.client.1.smithi181.stdout:9/297: creat d0/d44/d4f/f6b x:0 0 0 2024-04-27T04:09:51.944 INFO:tasks.workunit.client.1.smithi181.stdout:0/271: mkdir dc/d1f/d6b 0 2024-04-27T04:09:51.945 INFO:tasks.workunit.client.1.smithi181.stdout:2/294: dwrite d4/f50 [0,4194304] 0 2024-04-27T04:09:51.946 INFO:tasks.workunit.client.0.smithi162.stdout:0/17: creat d5/f8 x:0 0 0 2024-04-27T04:09:51.947 INFO:tasks.workunit.client.1.smithi181.stdout:2/295: dread - d4/f59 zero size 2024-04-27T04:09:51.947 INFO:tasks.workunit.client.0.smithi162.stdout:0/18: truncate f1 4498709 0 2024-04-27T04:09:51.948 INFO:tasks.workunit.client.1.smithi181.stdout:7/172: creat d1/d22/d15/d13/d1b/f33 x:0 0 0 2024-04-27T04:09:51.950 INFO:tasks.workunit.client.0.smithi162.stdout:8/19: link d0/f2 d0/f4 0 2024-04-27T04:09:51.951 INFO:tasks.workunit.client.1.smithi181.stdout:2/296: truncate d4/d9/d35/f51 656440 0 2024-04-27T04:09:51.952 INFO:tasks.workunit.client.1.smithi181.stdout:3/223: rmdir d0/d28 39 2024-04-27T04:09:51.954 INFO:tasks.workunit.client.0.smithi162.stdout:5/22: creat f5 x:0 0 0 2024-04-27T04:09:51.954 INFO:tasks.workunit.client.1.smithi181.stdout:0/272: dwrite dc/d1e/f30 [0,4194304] 0 2024-04-27T04:09:51.962 INFO:tasks.workunit.client.1.smithi181.stdout:4/185: rename d0/d15/d25/d2b/d7/d35 to d0/d15/d25/d2a/d36 0 2024-04-27T04:09:51.963 INFO:tasks.workunit.client.1.smithi181.stdout:4/186: dread - d0/d15/f23 zero size 2024-04-27T04:09:51.963 INFO:tasks.workunit.client.0.smithi162.stdout:8/20: symlink d0/l5 0 2024-04-27T04:09:51.964 INFO:tasks.workunit.client.0.smithi162.stdout:8/21: read d0/f2 [181427,127887] 0 2024-04-27T04:09:51.967 INFO:tasks.workunit.client.0.smithi162.stdout:6/11: creat d6/f7 x:0 0 0 2024-04-27T04:09:51.969 INFO:tasks.workunit.client.1.smithi181.stdout:0/273: dread dc/f16 [0,4194304] 0 2024-04-27T04:09:51.970 INFO:tasks.workunit.client.0.smithi162.stdout:0/19: link d5/f7 d5/f9 0 2024-04-27T04:09:51.970 INFO:tasks.workunit.client.1.smithi181.stdout:0/274: chown dc/d1f/d3b/d4a/l56 11 1 2024-04-27T04:09:51.972 INFO:tasks.workunit.client.1.smithi181.stdout:8/300: creat d1/d6/d1a/f6a x:0 0 0 2024-04-27T04:09:51.978 INFO:tasks.workunit.client.0.smithi162.stdout:8/22: creat d0/f6 x:0 0 0 2024-04-27T04:09:51.980 INFO:tasks.workunit.client.0.smithi162.stdout:1/17: truncate d0/f2 1061643 0 2024-04-27T04:09:51.983 INFO:tasks.workunit.client.0.smithi162.stdout:6/12: unlink f3 0 2024-04-27T04:09:51.983 INFO:tasks.workunit.client.0.smithi162.stdout:6/13: chown d6/f7 4003 1 2024-04-27T04:09:51.986 INFO:tasks.workunit.client.0.smithi162.stdout:0/20: creat d5/fa x:0 0 0 2024-04-27T04:09:51.988 INFO:tasks.workunit.client.0.smithi162.stdout:8/23: creat d0/f7 x:0 0 0 2024-04-27T04:09:51.988 INFO:tasks.workunit.client.0.smithi162.stdout:8/24: stat d0/l5 0 2024-04-27T04:09:51.992 INFO:tasks.workunit.client.0.smithi162.stdout:1/18: symlink d0/l4 0 2024-04-27T04:09:51.994 INFO:tasks.workunit.client.0.smithi162.stdout:6/14: mknod d6/c8 0 2024-04-27T04:09:51.995 INFO:tasks.workunit.client.0.smithi162.stdout:6/15: write f5 [123825,76356] 0 2024-04-27T04:09:51.995 INFO:tasks.workunit.client.0.smithi162.stdout:6/16: write f5 [357932,79530] 0 2024-04-27T04:09:51.996 INFO:tasks.workunit.client.1.smithi181.stdout:4/187: symlink d0/d15/d25/d2b/l37 0 2024-04-27T04:09:51.998 INFO:tasks.workunit.client.0.smithi162.stdout:0/21: write d5/f7 [834729,105629] 0 2024-04-27T04:09:51.999 INFO:tasks.workunit.client.0.smithi162.stdout:0/22: chown d5/f7 28653 1 2024-04-27T04:09:52.000 INFO:tasks.workunit.client.0.smithi162.stdout:8/25: write d0/f2 [96734,13812] 0 2024-04-27T04:09:52.001 INFO:tasks.workunit.client.1.smithi181.stdout:1/205: symlink d1/d2/d5/d1c/d1e/l42 0 2024-04-27T04:09:52.002 INFO:tasks.workunit.client.1.smithi181.stdout:1/206: write d1/d2/d5/d1c/f3e [35932,115312] 0 2024-04-27T04:09:52.004 INFO:tasks.workunit.client.0.smithi162.stdout:6/17: dwrite f2 [0,4194304] 0 2024-04-27T04:09:52.004 INFO:tasks.workunit.client.1.smithi181.stdout:0/275: creat dc/d1e/d26/f6c x:0 0 0 2024-04-27T04:09:52.006 INFO:tasks.workunit.client.0.smithi162.stdout:6/18: read f2 [1387686,7216] 0 2024-04-27T04:09:52.011 INFO:tasks.workunit.client.0.smithi162.stdout:8/26: dwrite d0/f7 [0,4194304] 0 2024-04-27T04:09:52.011 INFO:tasks.workunit.client.0.smithi162.stdout:5/23: getdents . 0 2024-04-27T04:09:52.015 INFO:tasks.workunit.client.0.smithi162.stdout:8/27: truncate d0/f2 283852 0 2024-04-27T04:09:52.016 INFO:tasks.workunit.client.0.smithi162.stdout:6/19: dwrite f5 [0,4194304] 0 2024-04-27T04:09:52.018 INFO:tasks.workunit.client.0.smithi162.stdout:8/28: truncate d0/f2 1091810 0 2024-04-27T04:09:52.023 INFO:tasks.workunit.client.0.smithi162.stdout:5/24: dwrite f0 [0,4194304] 0 2024-04-27T04:09:52.028 INFO:tasks.workunit.client.1.smithi181.stdout:9/298: rename d0/d40/d66 to d0/d44/d4b/d50/d6c 0 2024-04-27T04:09:52.030 INFO:tasks.workunit.client.1.smithi181.stdout:4/188: creat d0/d15/d25/d2a/f38 x:0 0 0 2024-04-27T04:09:52.031 INFO:tasks.workunit.client.1.smithi181.stdout:4/189: fsync d0/f30 0 2024-04-27T04:09:52.034 INFO:tasks.workunit.client.1.smithi181.stdout:1/207: symlink d1/d2/d5/d6/db/l43 0 2024-04-27T04:09:52.037 INFO:tasks.workunit.client.1.smithi181.stdout:0/276: mkdir dc/d1e/d6d 0 2024-04-27T04:09:52.042 INFO:tasks.workunit.client.0.smithi162.stdout:8/29: mknod d0/c8 0 2024-04-27T04:09:52.043 INFO:tasks.workunit.client.0.smithi162.stdout:1/19: link d0/l3 d0/l5 0 2024-04-27T04:09:52.044 INFO:tasks.workunit.client.1.smithi181.stdout:3/224: link d0/de/l35 d0/de/d36/l4f 0 2024-04-27T04:09:52.045 INFO:tasks.workunit.client.0.smithi162.stdout:6/20: dwrite f2 [0,4194304] 0 2024-04-27T04:09:52.045 INFO:tasks.workunit.client.1.smithi181.stdout:6/149: getdents d7/d27 0 2024-04-27T04:09:52.046 INFO:tasks.workunit.client.0.smithi162.stdout:6/21: write f2 [2798400,16499] 0 2024-04-27T04:09:52.050 INFO:tasks.workunit.client.0.smithi162.stdout:8/30: unlink d0/f6 0 2024-04-27T04:09:52.054 INFO:tasks.workunit.client.1.smithi181.stdout:8/301: rename d1/d6/d2f/d30/f60 to d1/f6b 0 2024-04-27T04:09:52.054 INFO:tasks.workunit.client.1.smithi181.stdout:8/302: readlink d1/d6/d1a/l68 0 2024-04-27T04:09:52.056 INFO:tasks.workunit.client.1.smithi181.stdout:4/190: chown d0/d15/l1e 444130 1 2024-04-27T04:09:52.058 INFO:tasks.workunit.client.0.smithi162.stdout:5/25: getdents . 0 2024-04-27T04:09:52.059 INFO:tasks.workunit.client.1.smithi181.stdout:4/191: dread d0/d15/d25/d2b/d7/f28 [0,4194304] 0 2024-04-27T04:09:52.059 INFO:tasks.workunit.client.0.smithi162.stdout:5/26: chown f4 249071634 1 2024-04-27T04:09:52.062 INFO:tasks.workunit.client.1.smithi181.stdout:0/277: symlink dc/d54/d57/l6e 0 2024-04-27T04:09:52.062 INFO:tasks.workunit.client.1.smithi181.stdout:0/278: write dc/f11 [2189609,25440] 0 2024-04-27T04:09:52.063 INFO:tasks.workunit.client.0.smithi162.stdout:6/22: creat d6/f9 x:0 0 0 2024-04-27T04:09:52.064 INFO:tasks.workunit.client.1.smithi181.stdout:7/173: getdents d1/d22 0 2024-04-27T04:09:52.067 INFO:tasks.workunit.client.0.smithi162.stdout:8/31: symlink d0/l9 0 2024-04-27T04:09:52.067 INFO:tasks.workunit.client.1.smithi181.stdout:0/279: dread dc/f22 [0,4194304] 0 2024-04-27T04:09:52.068 INFO:tasks.workunit.client.0.smithi162.stdout:0/23: truncate f1 315560 0 2024-04-27T04:09:52.068 INFO:tasks.workunit.client.0.smithi162.stdout:0/24: write d5/f7 [422259,112208] 0 2024-04-27T04:09:52.069 INFO:tasks.workunit.client.1.smithi181.stdout:2/297: getdents d4/d9/df/d14 0 2024-04-27T04:09:52.069 INFO:tasks.workunit.client.0.smithi162.stdout:0/25: write f0 [3246626,37047] 0 2024-04-27T04:09:52.070 INFO:tasks.workunit.client.1.smithi181.stdout:2/298: write d4/d9/d44/f46 [733320,20576] 0 2024-04-27T04:09:52.073 INFO:tasks.workunit.client.0.smithi162.stdout:0/26: dread f0 [0,4194304] 0 2024-04-27T04:09:52.074 INFO:tasks.workunit.client.1.smithi181.stdout:6/150: unlink d7/fe 0 2024-04-27T04:09:52.076 INFO:tasks.workunit.client.0.smithi162.stdout:8/32: rename d0/f1 to d0/fa 0 2024-04-27T04:09:52.076 INFO:tasks.workunit.client.0.smithi162.stdout:8/33: fsync d0/f2 0 2024-04-27T04:09:52.077 INFO:tasks.workunit.client.0.smithi162.stdout:0/27: dread f0 [0,4194304] 0 2024-04-27T04:09:52.077 INFO:tasks.workunit.client.0.smithi162.stdout:8/34: fsync d0/f7 0 2024-04-27T04:09:52.077 INFO:tasks.workunit.client.0.smithi162.stdout:0/28: write f0 [2322090,37356] 0 2024-04-27T04:09:52.080 INFO:tasks.workunit.client.0.smithi162.stdout:6/23: rmdir d6 39 2024-04-27T04:09:52.083 INFO:tasks.workunit.client.1.smithi181.stdout:9/299: write d0/d44/d4f/f6b [991670,92255] 0 2024-04-27T04:09:52.087 INFO:tasks.workunit.client.0.smithi162.stdout:1/20: fdatasync d0/f2 0 2024-04-27T04:09:52.089 INFO:tasks.workunit.client.1.smithi181.stdout:1/208: symlink d1/d2/d5/d1c/d1e/l44 0 2024-04-27T04:09:52.091 INFO:tasks.workunit.client.1.smithi181.stdout:4/192: mkdir d0/d15/de/d39 0 2024-04-27T04:09:52.098 INFO:tasks.workunit.client.0.smithi162.stdout:0/29: creat d5/fb x:0 0 0 2024-04-27T04:09:52.098 INFO:tasks.workunit.client.0.smithi162.stdout:0/30: chown d5 2757545 1 2024-04-27T04:09:52.100 INFO:tasks.workunit.client.1.smithi181.stdout:0/280: rename dc/d54/d57 to dc/d1e/d26/d41/d6f 0 2024-04-27T04:09:52.101 INFO:tasks.workunit.client.0.smithi162.stdout:8/35: dwrite d0/fa [0,4194304] 0 2024-04-27T04:09:52.102 INFO:tasks.workunit.client.0.smithi162.stdout:5/27: link c3 c6 0 2024-04-27T04:09:52.103 INFO:tasks.workunit.client.1.smithi181.stdout:3/225: truncate d0/f23 2420332 0 2024-04-27T04:09:52.109 INFO:tasks.workunit.client.1.smithi181.stdout:6/151: unlink d7/c21 0 2024-04-27T04:09:52.110 INFO:tasks.workunit.client.1.smithi181.stdout:8/303: mknod d1/d57/c6c 0 2024-04-27T04:09:52.112 INFO:tasks.workunit.client.0.smithi162.stdout:0/31: mkdir d5/dc 0 2024-04-27T04:09:52.114 INFO:tasks.workunit.client.0.smithi162.stdout:1/21: dwrite d0/f2 [0,4194304] 0 2024-04-27T04:09:52.115 INFO:tasks.workunit.client.0.smithi162.stdout:6/24: unlink d6/c8 0 2024-04-27T04:09:52.118 INFO:tasks.workunit.client.0.smithi162.stdout:8/36: rmdir d0 39 2024-04-27T04:09:52.119 INFO:tasks.workunit.client.0.smithi162.stdout:1/22: dread d0/f2 [0,4194304] 0 2024-04-27T04:09:52.119 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:09:51 smithi181 ceph-mon[33774]: pgmap v261: 65 pgs: 65 active+clean; 433 MiB data, 3.0 GiB used, 533 GiB / 536 GiB avail; 1.9 MiB/s rd, 27 MiB/s wr, 233 op/s 2024-04-27T04:09:52.120 INFO:tasks.workunit.client.0.smithi162.stdout:5/28: symlink l7 0 2024-04-27T04:09:52.122 INFO:tasks.workunit.client.1.smithi181.stdout:4/193: mknod d0/d15/d25/d2b/d7/c3a 0 2024-04-27T04:09:52.123 INFO:tasks.workunit.client.1.smithi181.stdout:5/307: sync 2024-04-27T04:09:52.124 INFO:tasks.workunit.client.1.smithi181.stdout:5/308: truncate d4/de/f6e 520431 0 2024-04-27T04:09:52.130 INFO:tasks.workunit.client.0.smithi162.stdout:0/32: unlink d5/f7 0 2024-04-27T04:09:52.133 INFO:tasks.workunit.client.1.smithi181.stdout:7/174: mkdir d1/d22/d15/d1f/d34 0 2024-04-27T04:09:52.134 INFO:tasks.workunit.client.1.smithi181.stdout:0/281: rename dc/d1e/c2b to dc/d1e/c70 0 2024-04-27T04:09:52.135 INFO:tasks.workunit.client.1.smithi181.stdout:5/309: dwrite d4/d18/f48 [4194304,4194304] 0 2024-04-27T04:09:52.137 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:09:51 smithi162 ceph-mon[26841]: pgmap v261: 65 pgs: 65 active+clean; 433 MiB data, 3.0 GiB used, 533 GiB / 536 GiB avail; 1.9 MiB/s rd, 27 MiB/s wr, 233 op/s 2024-04-27T04:09:52.138 INFO:tasks.workunit.client.1.smithi181.stdout:2/299: creat d4/d9/d35/d42/f5a x:0 0 0 2024-04-27T04:09:52.139 INFO:tasks.workunit.client.1.smithi181.stdout:2/300: readlink d4/d9/l33 0 2024-04-27T04:09:52.140 INFO:tasks.workunit.client.0.smithi162.stdout:8/37: dread d0/f4 [0,4194304] 0 2024-04-27T04:09:52.140 INFO:tasks.workunit.client.0.smithi162.stdout:1/23: write d0/f2 [2740921,10588] 0 2024-04-27T04:09:52.140 INFO:tasks.workunit.client.0.smithi162.stdout:8/38: readlink d0/l5 0 2024-04-27T04:09:52.142 INFO:tasks.workunit.client.1.smithi181.stdout:3/226: dread d0/f19 [0,4194304] 0 2024-04-27T04:09:52.146 INFO:tasks.workunit.client.0.smithi162.stdout:8/39: dwrite d0/f7 [0,4194304] 0 2024-04-27T04:09:52.149 INFO:tasks.workunit.client.0.smithi162.stdout:1/24: write d0/f2 [551463,118765] 0 2024-04-27T04:09:52.153 INFO:tasks.workunit.client.0.smithi162.stdout:8/40: dwrite d0/f4 [0,4194304] 0 2024-04-27T04:09:52.156 INFO:tasks.workunit.client.0.smithi162.stdout:1/25: mknod d0/c6 0 2024-04-27T04:09:52.161 INFO:tasks.workunit.client.0.smithi162.stdout:6/25: getdents d6 0 2024-04-27T04:09:52.161 INFO:tasks.workunit.client.0.smithi162.stdout:6/26: chown c1 28005339 1 2024-04-27T04:09:52.162 INFO:tasks.workunit.client.1.smithi181.stdout:2/301: creat d4/d9/d4f/f5b x:0 0 0 2024-04-27T04:09:52.163 INFO:tasks.workunit.client.1.smithi181.stdout:2/302: dread - d4/d9/d4f/f3d zero size 2024-04-27T04:09:52.164 INFO:tasks.workunit.client.0.smithi162.stdout:8/41: symlink d0/lb 0 2024-04-27T04:09:52.164 INFO:tasks.workunit.client.1.smithi181.stdout:5/310: write d4/de/d2c/d2e/d55/f3f [869094,35192] 0 2024-04-27T04:09:52.166 INFO:tasks.workunit.client.1.smithi181.stdout:5/311: chown d4/d18/d67/l64 80 1 2024-04-27T04:09:52.167 INFO:tasks.workunit.client.0.smithi162.stdout:8/42: dread d0/fa [0,4194304] 0 2024-04-27T04:09:52.170 INFO:tasks.workunit.client.1.smithi181.stdout:6/152: truncate d7/dc/d1e/f2a 391265 0 2024-04-27T04:09:52.171 INFO:tasks.workunit.client.1.smithi181.stdout:2/303: dwrite d4/d9/d4f/f3d [0,4194304] 0 2024-04-27T04:09:52.172 INFO:tasks.workunit.client.1.smithi181.stdout:2/304: dread - d4/d9/df/d14/f48 zero size 2024-04-27T04:09:52.172 INFO:tasks.workunit.client.0.smithi162.stdout:8/43: dwrite d0/f2 [0,4194304] 0 2024-04-27T04:09:52.177 INFO:tasks.workunit.client.0.smithi162.stdout:1/26: rmdir d0 39 2024-04-27T04:09:52.181 INFO:tasks.workunit.client.0.smithi162.stdout:8/44: dwrite d0/f2 [0,4194304] 0 2024-04-27T04:09:52.182 INFO:tasks.workunit.client.0.smithi162.stdout:6/27: creat d6/fa x:0 0 0 2024-04-27T04:09:52.183 INFO:tasks.workunit.client.1.smithi181.stdout:9/300: creat d0/d44/d4b/d50/d6c/f6d x:0 0 0 2024-04-27T04:09:52.184 INFO:tasks.workunit.client.1.smithi181.stdout:1/209: creat d1/d2/d5/f45 x:0 0 0 2024-04-27T04:09:52.188 INFO:tasks.workunit.client.1.smithi181.stdout:8/304: write d1/d6/d2f/d40/d2a/f63 [1229409,116138] 0 2024-04-27T04:09:52.194 INFO:tasks.workunit.client.0.smithi162.stdout:1/27: write d0/f2 [2765325,68755] 0 2024-04-27T04:09:52.195 INFO:tasks.workunit.client.0.smithi162.stdout:8/45: unlink d0/lb 0 2024-04-27T04:09:52.196 INFO:tasks.workunit.client.0.smithi162.stdout:6/28: mkdir d6/db 0 2024-04-27T04:09:52.197 INFO:tasks.workunit.client.1.smithi181.stdout:9/301: dread - d0/f2b zero size 2024-04-27T04:09:52.198 INFO:tasks.workunit.client.0.smithi162.stdout:1/28: readlink d0/l5 0 2024-04-27T04:09:52.198 INFO:tasks.workunit.client.0.smithi162.stdout:0/33: dwrite d5/f9 [0,4194304] 0 2024-04-27T04:09:52.200 INFO:tasks.workunit.client.1.smithi181.stdout:4/194: creat d0/f3b x:0 0 0 2024-04-27T04:09:52.200 INFO:tasks.workunit.client.0.smithi162.stdout:1/29: fdatasync d0/f2 0 2024-04-27T04:09:52.201 INFO:tasks.workunit.client.1.smithi181.stdout:4/195: write d0/f30 [339752,92113] 0 2024-04-27T04:09:52.202 INFO:tasks.workunit.client.0.smithi162.stdout:6/29: creat d6/fc x:0 0 0 2024-04-27T04:09:52.202 INFO:tasks.workunit.client.1.smithi181.stdout:8/305: symlink d1/d6/d2f/d30/l6d 0 2024-04-27T04:09:52.203 INFO:tasks.workunit.client.0.smithi162.stdout:0/34: chown f1 29879803 1 2024-04-27T04:09:52.205 INFO:tasks.workunit.client.1.smithi181.stdout:3/227: creat d0/f50 x:0 0 0 2024-04-27T04:09:52.206 INFO:tasks.workunit.client.0.smithi162.stdout:1/30: mkdir d0/d7 0 2024-04-27T04:09:52.207 INFO:tasks.workunit.client.0.smithi162.stdout:1/31: write d0/f2 [2701968,53641] 0 2024-04-27T04:09:52.208 INFO:tasks.workunit.client.0.smithi162.stdout:6/30: dwrite f5 [0,4194304] 0 2024-04-27T04:09:52.212 INFO:tasks.workunit.client.0.smithi162.stdout:6/31: mknod d6/cd 0 2024-04-27T04:09:52.213 INFO:tasks.workunit.client.0.smithi162.stdout:1/32: fsync d0/f2 0 2024-04-27T04:09:52.213 INFO:tasks.workunit.client.0.smithi162.stdout:0/35: dread - d5/f8 zero size 2024-04-27T04:09:52.214 INFO:tasks.workunit.client.0.smithi162.stdout:6/32: mkdir d6/db/de 0 2024-04-27T04:09:52.215 INFO:tasks.workunit.client.0.smithi162.stdout:0/36: truncate d5/f8 816122 0 2024-04-27T04:09:52.217 INFO:tasks.workunit.client.0.smithi162.stdout:6/33: mknod d6/db/de/cf 0 2024-04-27T04:09:52.217 INFO:tasks.workunit.client.0.smithi162.stdout:0/37: creat d5/dc/fd x:0 0 0 2024-04-27T04:09:52.218 INFO:tasks.workunit.client.0.smithi162.stdout:1/33: dread d0/f2 [0,4194304] 0 2024-04-27T04:09:52.218 INFO:tasks.workunit.client.0.smithi162.stdout:0/38: chown f0 1 1 2024-04-27T04:09:52.218 INFO:tasks.workunit.client.0.smithi162.stdout:6/34: creat d6/db/de/f10 x:0 0 0 2024-04-27T04:09:52.219 INFO:tasks.workunit.client.0.smithi162.stdout:6/35: readlink - no filename 2024-04-27T04:09:52.220 INFO:tasks.workunit.client.0.smithi162.stdout:1/34: unlink d0/c1 0 2024-04-27T04:09:52.220 INFO:tasks.workunit.client.0.smithi162.stdout:6/36: creat d6/f11 x:0 0 0 2024-04-27T04:09:52.221 INFO:tasks.workunit.client.0.smithi162.stdout:0/39: creat d5/dc/fe x:0 0 0 2024-04-27T04:09:52.222 INFO:tasks.workunit.client.0.smithi162.stdout:6/37: read - d6/db/de/f10 zero size 2024-04-27T04:09:52.223 INFO:tasks.workunit.client.0.smithi162.stdout:6/38: write f5 [3326948,79462] 0 2024-04-27T04:09:52.224 INFO:tasks.workunit.client.0.smithi162.stdout:1/35: dread d0/f2 [0,4194304] 0 2024-04-27T04:09:52.225 INFO:tasks.workunit.client.0.smithi162.stdout:1/36: write d0/f2 [1759114,86425] 0 2024-04-27T04:09:52.225 INFO:tasks.workunit.client.0.smithi162.stdout:0/40: rename l4 to d5/lf 0 2024-04-27T04:09:52.228 INFO:tasks.workunit.client.0.smithi162.stdout:1/37: rename d0/l5 to d0/d7/l8 0 2024-04-27T04:09:52.228 INFO:tasks.workunit.client.0.smithi162.stdout:1/38: chown d0/l4 1 1 2024-04-27T04:09:52.228 INFO:tasks.workunit.client.0.smithi162.stdout:0/41: creat d5/f10 x:0 0 0 2024-04-27T04:09:52.229 INFO:tasks.workunit.client.0.smithi162.stdout:1/39: symlink d0/d7/l9 0 2024-04-27T04:09:52.234 INFO:tasks.workunit.client.1.smithi181.stdout:6/153: link d7/dc/c14 d7/d1b/c33 0 2024-04-27T04:09:52.235 INFO:tasks.workunit.client.0.smithi162.stdout:0/42: creat d5/f11 x:0 0 0 2024-04-27T04:09:52.237 INFO:tasks.workunit.client.1.smithi181.stdout:3/228: mknod d0/d3e/c51 0 2024-04-27T04:09:52.237 INFO:tasks.workunit.client.0.smithi162.stdout:8/46: fdatasync d0/f2 0 2024-04-27T04:09:52.239 INFO:tasks.workunit.client.1.smithi181.stdout:1/210: dread d1/d2/d5/d1c/ff [0,4194304] 0 2024-04-27T04:09:52.243 INFO:tasks.workunit.client.1.smithi181.stdout:8/306: dwrite d1/d6/d2f/d47/f4f [0,4194304] 0 2024-04-27T04:09:52.243 INFO:tasks.workunit.client.1.smithi181.stdout:5/312: write d4/d18/f41 [2944205,126128] 0 2024-04-27T04:09:52.244 INFO:tasks.workunit.client.0.smithi162.stdout:8/47: creat d0/fc x:0 0 0 2024-04-27T04:09:52.245 INFO:tasks.workunit.client.0.smithi162.stdout:8/48: truncate d0/fc 397843 0 2024-04-27T04:09:52.245 INFO:tasks.workunit.client.1.smithi181.stdout:8/307: write d1/d6/d1a/f53 [13508,111089] 0 2024-04-27T04:09:52.248 INFO:tasks.workunit.client.1.smithi181.stdout:8/308: truncate d1/d6/d1a/f53 1272120 0 2024-04-27T04:09:52.249 INFO:tasks.workunit.client.0.smithi162.stdout:1/40: link d0/l3 d0/d7/la 0 2024-04-27T04:09:52.249 INFO:tasks.workunit.client.1.smithi181.stdout:8/309: read - d1/d6/d1a/f6a zero size 2024-04-27T04:09:52.250 INFO:tasks.workunit.client.0.smithi162.stdout:1/41: fsync d0/f2 0 2024-04-27T04:09:52.251 INFO:tasks.workunit.client.1.smithi181.stdout:8/310: dread - d1/d6/d2f/d47/f4b zero size 2024-04-27T04:09:52.251 INFO:tasks.workunit.client.1.smithi181.stdout:3/229: dread - d0/f32 zero size 2024-04-27T04:09:52.252 INFO:tasks.workunit.client.0.smithi162.stdout:8/49: dwrite d0/f4 [0,4194304] 0 2024-04-27T04:09:52.252 INFO:tasks.workunit.client.0.smithi162.stdout:8/50: readlink d0/l5 0 2024-04-27T04:09:52.252 INFO:tasks.workunit.client.1.smithi181.stdout:8/311: truncate d1/d6/d2f/d40/f17 4514134 0 2024-04-27T04:09:52.261 INFO:tasks.workunit.client.1.smithi181.stdout:9/302: creat d0/d44/f6e x:0 0 0 2024-04-27T04:09:52.263 INFO:tasks.workunit.client.0.smithi162.stdout:0/43: link d5/f8 d5/dc/f12 0 2024-04-27T04:09:52.266 INFO:tasks.workunit.client.1.smithi181.stdout:2/305: write d4/d9/f11 [3923164,100221] 0 2024-04-27T04:09:52.266 INFO:tasks.workunit.client.0.smithi162.stdout:1/42: dwrite d0/f2 [0,4194304] 0 2024-04-27T04:09:52.267 INFO:tasks.workunit.client.1.smithi181.stdout:6/154: chown d7/dc/c14 1569786 1 2024-04-27T04:09:52.269 INFO:tasks.workunit.client.1.smithi181.stdout:3/230: chown d0/de/d20/c2b 72 1 2024-04-27T04:09:52.270 INFO:tasks.workunit.client.1.smithi181.stdout:5/313: dwrite d4/d18/d67/f39 [0,4194304] 0 2024-04-27T04:09:52.273 INFO:tasks.workunit.client.1.smithi181.stdout:4/196: getdents d0/d15/d25/d2a 0 2024-04-27T04:09:52.277 INFO:tasks.workunit.client.1.smithi181.stdout:1/211: symlink d1/d2/d5/d1c/d1e/l46 0 2024-04-27T04:09:52.282 INFO:tasks.workunit.client.1.smithi181.stdout:4/197: dwrite d0/d15/f1a [0,4194304] 0 2024-04-27T04:09:52.284 INFO:tasks.workunit.client.1.smithi181.stdout:2/306: mknod d4/d9/d44/c5c 0 2024-04-27T04:09:52.292 INFO:tasks.workunit.client.1.smithi181.stdout:4/198: dwrite d0/d15/f2 [0,4194304] 0 2024-04-27T04:09:52.293 INFO:tasks.workunit.client.1.smithi181.stdout:8/312: link d1/d6/d2f/d40/f27 d1/d6/d2f/d40/f6e 0 2024-04-27T04:09:52.307 INFO:tasks.workunit.client.1.smithi181.stdout:8/313: mkdir d1/d6/d6f 0 2024-04-27T04:09:52.311 INFO:tasks.workunit.client.1.smithi181.stdout:9/303: write d0/f2a [308908,12718] 0 2024-04-27T04:09:52.312 INFO:tasks.workunit.client.1.smithi181.stdout:1/212: mknod d1/d2/d5/c47 0 2024-04-27T04:09:52.314 INFO:tasks.workunit.client.1.smithi181.stdout:2/307: unlink d4/d9/d16/d45/c4c 0 2024-04-27T04:09:52.316 INFO:tasks.workunit.client.1.smithi181.stdout:3/231: dread d0/f23 [0,4194304] 0 2024-04-27T04:09:52.317 INFO:tasks.workunit.client.1.smithi181.stdout:3/232: stat d0/de/d36/d44 0 2024-04-27T04:09:52.318 INFO:tasks.workunit.client.1.smithi181.stdout:8/314: unlink d1/d6/c9 0 2024-04-27T04:09:52.318 INFO:tasks.workunit.client.1.smithi181.stdout:3/233: readlink d0/de/d36/l39 0 2024-04-27T04:09:52.319 INFO:tasks.workunit.client.1.smithi181.stdout:3/234: chown d0/de/ff 6973 1 2024-04-27T04:09:52.322 INFO:tasks.workunit.client.1.smithi181.stdout:2/308: dwrite d4/f59 [0,4194304] 0 2024-04-27T04:09:52.327 INFO:tasks.workunit.client.1.smithi181.stdout:8/315: dwrite d1/f33 [0,4194304] 0 2024-04-27T04:09:52.337 INFO:tasks.workunit.client.1.smithi181.stdout:4/199: link d0/d15/d25/d2b/d7/l11 d0/d15/de/d19/l3c 0 2024-04-27T04:09:52.341 INFO:tasks.workunit.client.1.smithi181.stdout:5/314: getdents d4/de 0 2024-04-27T04:09:52.341 INFO:tasks.workunit.client.1.smithi181.stdout:1/213: rmdir d1/d2/d5 39 2024-04-27T04:09:52.343 INFO:tasks.workunit.client.1.smithi181.stdout:9/304: truncate d0/d40/f22 1046396 0 2024-04-27T04:09:52.345 INFO:tasks.workunit.client.1.smithi181.stdout:4/200: dwrite d0/d15/de/f34 [0,4194304] 0 2024-04-27T04:09:52.351 INFO:tasks.workunit.client.1.smithi181.stdout:2/309: mkdir d4/d9/d5d 0 2024-04-27T04:09:52.358 INFO:tasks.workunit.client.1.smithi181.stdout:8/316: mknod d1/d6/d2f/d30/d66/c70 0 2024-04-27T04:09:52.359 INFO:tasks.workunit.client.1.smithi181.stdout:8/317: readlink d1/d6/l2c 0 2024-04-27T04:09:52.362 INFO:tasks.workunit.client.1.smithi181.stdout:3/235: dwrite d0/f4 [4194304,4194304] 0 2024-04-27T04:09:52.367 INFO:tasks.workunit.client.1.smithi181.stdout:5/315: symlink d4/de/d2c/d46/d4d/l73 0 2024-04-27T04:09:52.370 INFO:tasks.workunit.client.1.smithi181.stdout:8/318: mknod d1/d6/d2f/d47/c71 0 2024-04-27T04:09:52.376 INFO:tasks.workunit.client.1.smithi181.stdout:2/310: mknod d4/d9/df/c5e 0 2024-04-27T04:09:52.379 INFO:tasks.workunit.client.1.smithi181.stdout:3/236: rename d0/de/d36 to d0/d3e/d52 0 2024-04-27T04:09:52.381 INFO:tasks.workunit.client.1.smithi181.stdout:1/214: mkdir d1/d2/d5/d1c/d1e/d48 0 2024-04-27T04:09:52.384 INFO:tasks.workunit.client.1.smithi181.stdout:4/201: dwrite d0/d15/d25/d2b/d7/f28 [0,4194304] 0 2024-04-27T04:09:52.384 INFO:tasks.workunit.client.1.smithi181.stdout:4/202: chown d0/d15/f2f 128 1 2024-04-27T04:09:52.386 INFO:tasks.workunit.client.1.smithi181.stdout:4/203: write d0/d15/d25/d2b/f2c [525265,82541] 0 2024-04-27T04:09:52.391 INFO:tasks.workunit.client.1.smithi181.stdout:2/311: mkdir d4/d9/d4f/d5f 0 2024-04-27T04:09:52.397 INFO:tasks.workunit.client.1.smithi181.stdout:8/319: fsync d1/d6/d2f/f3a 0 2024-04-27T04:09:52.398 INFO:tasks.workunit.client.1.smithi181.stdout:8/320: stat d1/d6/d1a/d25/f52 0 2024-04-27T04:09:52.399 INFO:tasks.workunit.client.1.smithi181.stdout:2/312: write d4/d9/f1a [3829747,40275] 0 2024-04-27T04:09:52.400 INFO:tasks.workunit.client.1.smithi181.stdout:3/237: mknod d0/de/d3f/c53 0 2024-04-27T04:09:52.401 INFO:tasks.workunit.client.1.smithi181.stdout:3/238: dread - d0/f37 zero size 2024-04-27T04:09:52.415 INFO:tasks.workunit.client.1.smithi181.stdout:2/313: creat d4/d9/d35/f60 x:0 0 0 2024-04-27T04:09:52.417 INFO:tasks.workunit.client.1.smithi181.stdout:3/239: truncate d0/de/f17 516547 0 2024-04-27T04:09:52.417 INFO:tasks.workunit.client.1.smithi181.stdout:3/240: dread - d0/f50 zero size 2024-04-27T04:09:52.419 INFO:tasks.workunit.client.1.smithi181.stdout:1/215: creat d1/d2/d5/d1c/d1e/f49 x:0 0 0 2024-04-27T04:09:52.421 INFO:tasks.workunit.client.1.smithi181.stdout:5/316: link d4/l68 d4/d18/d67/d71/l74 0 2024-04-27T04:09:52.423 INFO:tasks.workunit.client.1.smithi181.stdout:4/204: creat d0/f3d x:0 0 0 2024-04-27T04:09:52.425 INFO:tasks.workunit.client.1.smithi181.stdout:2/314: rmdir d4/d9 39 2024-04-27T04:09:52.425 INFO:tasks.workunit.client.1.smithi181.stdout:1/216: dwrite d1/f2f [0,4194304] 0 2024-04-27T04:09:52.427 INFO:tasks.workunit.client.1.smithi181.stdout:1/217: write d1/d2/d5/d1c/d1e/d20/f41 [3667,53951] 0 2024-04-27T04:09:52.427 INFO:tasks.workunit.client.1.smithi181.stdout:3/241: symlink d0/d3e/d52/d44/l54 0 2024-04-27T04:09:52.428 INFO:tasks.workunit.client.1.smithi181.stdout:3/242: read - d0/f37 zero size 2024-04-27T04:09:52.430 INFO:tasks.workunit.client.1.smithi181.stdout:3/243: write d0/de/f4b [434729,128743] 0 2024-04-27T04:09:52.430 INFO:tasks.workunit.client.1.smithi181.stdout:1/218: write d1/d2/d5/d6/db/f40 [262653,76131] 0 2024-04-27T04:09:52.433 INFO:tasks.workunit.client.1.smithi181.stdout:3/244: fsync d0/f1e 0 2024-04-27T04:09:52.433 INFO:tasks.workunit.client.1.smithi181.stdout:8/321: rename d1/d57 to d1/d6/d2f/d72 0 2024-04-27T04:09:52.434 INFO:tasks.workunit.client.1.smithi181.stdout:4/205: mknod d0/d15/d25/d2a/c3e 0 2024-04-27T04:09:52.434 INFO:tasks.workunit.client.1.smithi181.stdout:1/219: write d1/f33 [833537,75373] 0 2024-04-27T04:09:52.436 INFO:tasks.workunit.client.1.smithi181.stdout:3/245: fdatasync d0/f5 0 2024-04-27T04:09:52.436 INFO:tasks.workunit.client.1.smithi181.stdout:2/315: fdatasync d4/f8 0 2024-04-27T04:09:52.437 INFO:tasks.workunit.client.1.smithi181.stdout:1/220: chown d1/f33 421927215 1 2024-04-27T04:09:52.438 INFO:tasks.workunit.client.1.smithi181.stdout:5/317: dwrite d4/d18/d54/f58 [0,4194304] 0 2024-04-27T04:09:52.441 INFO:tasks.workunit.client.1.smithi181.stdout:1/221: truncate d1/d2/d5/d1c/d1e/d20/f41 356216 0 2024-04-27T04:09:52.442 INFO:tasks.workunit.client.1.smithi181.stdout:1/222: readlink d1/d2/d5/d6/l2e 0 2024-04-27T04:09:52.451 INFO:tasks.workunit.client.1.smithi181.stdout:3/246: creat d0/d3e/d52/d44/f55 x:0 0 0 2024-04-27T04:09:52.455 INFO:tasks.workunit.client.1.smithi181.stdout:5/318: creat d4/de/d2c/f75 x:0 0 0 2024-04-27T04:09:52.456 INFO:tasks.workunit.client.1.smithi181.stdout:3/247: dread d0/de/f1d [0,4194304] 0 2024-04-27T04:09:52.459 INFO:tasks.workunit.client.1.smithi181.stdout:4/206: symlink d0/l3f 0 2024-04-27T04:09:52.462 INFO:tasks.workunit.client.1.smithi181.stdout:2/316: rename d4/d9/l57 to d4/d9/d4f/d5f/l61 0 2024-04-27T04:09:52.465 INFO:tasks.workunit.client.1.smithi181.stdout:1/223: dwrite d1/d2/d5/d6/f22 [4194304,4194304] 0 2024-04-27T04:09:52.465 INFO:tasks.workunit.client.1.smithi181.stdout:2/317: readlink d4/d9/l31 0 2024-04-27T04:09:52.466 INFO:tasks.workunit.client.1.smithi181.stdout:2/318: read d4/d9/d16/f23 [1670241,43845] 0 2024-04-27T04:09:52.471 INFO:tasks.workunit.client.1.smithi181.stdout:5/319: symlink d4/de/d2f/l76 0 2024-04-27T04:09:52.472 INFO:tasks.workunit.client.1.smithi181.stdout:5/320: fdatasync d4/de/d2c/f5c 0 2024-04-27T04:09:52.474 INFO:tasks.workunit.client.1.smithi181.stdout:3/248: mkdir d0/d3e/d52/d44/d56 0 2024-04-27T04:09:52.474 INFO:tasks.workunit.client.1.smithi181.stdout:5/321: truncate d4/d18/d54/f58 4756194 0 2024-04-27T04:09:52.478 INFO:tasks.workunit.client.1.smithi181.stdout:3/249: dread d0/de/d20/f46 [0,4194304] 0 2024-04-27T04:09:52.479 INFO:tasks.workunit.client.1.smithi181.stdout:3/250: write d0/f1e [4311805,77734] 0 2024-04-27T04:09:52.479 INFO:tasks.workunit.client.1.smithi181.stdout:4/207: truncate d0/d15/d25/d2b/f1c 1045476 0 2024-04-27T04:09:52.487 INFO:tasks.workunit.client.1.smithi181.stdout:2/319: symlink d4/d9/d35/d42/l62 0 2024-04-27T04:09:52.490 INFO:tasks.workunit.client.1.smithi181.stdout:2/320: dread d4/d9/df/d14/f24 [4194304,4194304] 0 2024-04-27T04:09:52.491 INFO:tasks.workunit.client.1.smithi181.stdout:5/322: chown d4/de/d2c/l4b 2 1 2024-04-27T04:09:52.500 INFO:tasks.workunit.client.1.smithi181.stdout:8/322: truncate d1/d6/d1a/d25/f52 2836397 0 2024-04-27T04:09:52.503 INFO:tasks.workunit.client.1.smithi181.stdout:1/224: creat d1/d2/d5/d6/db/d2d/d36/f4a x:0 0 0 2024-04-27T04:09:52.504 INFO:tasks.workunit.client.1.smithi181.stdout:2/321: dread d4/d9/d4f/d40/f49 [0,4194304] 0 2024-04-27T04:09:52.505 INFO:tasks.workunit.client.1.smithi181.stdout:2/322: chown d4/d9/df/c17 134 1 2024-04-27T04:09:52.505 INFO:tasks.workunit.client.1.smithi181.stdout:2/323: chown d4/f32 0 1 2024-04-27T04:09:52.506 INFO:tasks.workunit.client.1.smithi181.stdout:5/323: rename d4/de/d2c/f5c to d4/d10/d72/f77 0 2024-04-27T04:09:52.511 INFO:tasks.workunit.client.1.smithi181.stdout:2/324: dwrite d4/f8 [4194304,4194304] 0 2024-04-27T04:09:52.520 INFO:tasks.workunit.client.1.smithi181.stdout:5/324: creat d4/d18/d54/f78 x:0 0 0 2024-04-27T04:09:52.522 INFO:tasks.workunit.client.1.smithi181.stdout:3/251: creat d0/f57 x:0 0 0 2024-04-27T04:09:52.527 INFO:tasks.workunit.client.1.smithi181.stdout:2/325: rmdir d4/d9/d35/d42 39 2024-04-27T04:09:52.529 INFO:tasks.workunit.client.1.smithi181.stdout:5/325: dwrite d4/de/d2c/d46/d4d/f52 [0,4194304] 0 2024-04-27T04:09:52.530 INFO:tasks.workunit.client.1.smithi181.stdout:5/326: readlink d4/l32 0 2024-04-27T04:09:52.533 INFO:tasks.workunit.client.1.smithi181.stdout:3/252: dwrite d0/f4 [4194304,4194304] 0 2024-04-27T04:09:52.534 INFO:tasks.workunit.client.1.smithi181.stdout:5/327: read d4/d18/d67/f39 [3919468,120256] 0 2024-04-27T04:09:52.539 INFO:tasks.workunit.client.1.smithi181.stdout:4/208: dwrite d0/d15/d25/d2b/f6 [0,4194304] 0 2024-04-27T04:09:52.547 INFO:tasks.workunit.client.1.smithi181.stdout:1/225: creat d1/d2/d5/d1c/d1e/f4b x:0 0 0 2024-04-27T04:09:52.548 INFO:tasks.workunit.client.1.smithi181.stdout:5/328: dwrite d4/de/d2c/d46/d4d/f66 [0,4194304] 0 2024-04-27T04:09:52.550 INFO:tasks.workunit.client.1.smithi181.stdout:1/226: write d1/d2/d5/d2c/f3f [180361,130113] 0 2024-04-27T04:09:52.555 INFO:tasks.workunit.client.1.smithi181.stdout:2/326: rename d4/d9/d4f/d40 to d4/d9/d44/d63 0 2024-04-27T04:09:52.559 INFO:tasks.workunit.client.1.smithi181.stdout:3/253: mkdir d0/de/d20/d58 0 2024-04-27T04:09:52.561 INFO:tasks.workunit.client.1.smithi181.stdout:4/209: mkdir d0/d15/d25/d40 0 2024-04-27T04:09:52.572 INFO:tasks.workunit.client.1.smithi181.stdout:3/254: creat d0/d3e/d52/f59 x:0 0 0 2024-04-27T04:09:52.575 INFO:tasks.workunit.client.1.smithi181.stdout:4/210: creat d0/d15/d25/d2b/f41 x:0 0 0 2024-04-27T04:09:52.581 INFO:tasks.workunit.client.1.smithi181.stdout:3/255: truncate d0/f23 1443411 0 2024-04-27T04:09:52.583 INFO:tasks.workunit.client.1.smithi181.stdout:5/329: dread d4/de/d2c/d46/f60 [0,4194304] 0 2024-04-27T04:09:52.583 INFO:tasks.workunit.client.1.smithi181.stdout:4/211: creat d0/d15/d25/d2a/d36/f42 x:0 0 0 2024-04-27T04:09:52.586 INFO:tasks.workunit.client.1.smithi181.stdout:2/327: creat d4/d9/f64 x:0 0 0 2024-04-27T04:09:52.589 INFO:tasks.workunit.client.1.smithi181.stdout:3/256: read d0/de/d20/f21 [260641,92458] 0 2024-04-27T04:09:52.589 INFO:tasks.workunit.client.1.smithi181.stdout:5/330: rmdir d4/de/d2f 39 2024-04-27T04:09:52.592 INFO:tasks.workunit.client.1.smithi181.stdout:4/212: read d0/d15/d25/d2b/f1f [1383859,30100] 0 2024-04-27T04:09:52.593 INFO:tasks.workunit.client.1.smithi181.stdout:2/328: fsync d4/d9/d16/f29 0 2024-04-27T04:09:52.593 INFO:tasks.workunit.client.1.smithi181.stdout:4/213: write d0/f3b [630227,86960] 0 2024-04-27T04:09:52.594 INFO:tasks.workunit.client.1.smithi181.stdout:2/329: fsync d4/d9/d35/f3a 0 2024-04-27T04:09:52.594 INFO:tasks.workunit.client.1.smithi181.stdout:1/227: write d1/d2/d5/d1c/ff [2411559,89117] 0 2024-04-27T04:09:52.596 INFO:tasks.workunit.client.1.smithi181.stdout:1/228: dread - d1/d2/d5/d1c/d1e/f49 zero size 2024-04-27T04:09:52.598 INFO:tasks.workunit.client.1.smithi181.stdout:1/229: truncate d1/d2/d5/d6/db/f40 669697 0 2024-04-27T04:09:52.598 INFO:tasks.workunit.client.1.smithi181.stdout:4/214: creat d0/d15/de/f43 x:0 0 0 2024-04-27T04:09:52.600 INFO:tasks.workunit.client.1.smithi181.stdout:2/330: symlink d4/d9/d4f/l65 0 2024-04-27T04:09:52.603 INFO:tasks.workunit.client.1.smithi181.stdout:4/215: truncate d0/d15/d25/d2b/f14 366238 0 2024-04-27T04:09:52.604 INFO:tasks.workunit.client.1.smithi181.stdout:2/331: rmdir d4/d9/df/d14 39 2024-04-27T04:09:52.604 INFO:tasks.workunit.client.1.smithi181.stdout:4/216: fsync d0/d15/d25/d2b/d7/f31 0 2024-04-27T04:09:52.607 INFO:tasks.workunit.client.1.smithi181.stdout:5/331: dwrite d4/de/f6e [0,4194304] 0 2024-04-27T04:09:52.612 INFO:tasks.workunit.client.1.smithi181.stdout:1/230: rename d1/d2/d5/d1c to d1/d4c 0 2024-04-27T04:09:52.613 INFO:tasks.workunit.client.1.smithi181.stdout:1/231: chown d1/d4c/d1e/l2a 6348435 1 2024-04-27T04:09:52.614 INFO:tasks.workunit.client.1.smithi181.stdout:4/217: dwrite d0/d15/f2 [0,4194304] 0 2024-04-27T04:09:52.619 INFO:tasks.workunit.client.1.smithi181.stdout:5/332: creat d4/d10/d29/f79 x:0 0 0 2024-04-27T04:09:52.622 INFO:tasks.workunit.client.1.smithi181.stdout:1/232: rename d1/f33 to d1/d4c/f4d 0 2024-04-27T04:09:52.633 INFO:tasks.workunit.client.1.smithi181.stdout:4/218: rename d0/d15/d25/d2b/d7/c27 to d0/d15/d25/c44 0 2024-04-27T04:09:52.634 INFO:tasks.workunit.client.1.smithi181.stdout:1/233: creat d1/d2/d5/d6/f4e x:0 0 0 2024-04-27T04:09:52.677 INFO:tasks.workunit.client.1.smithi181.stdout:7/175: sync 2024-04-27T04:09:52.677 INFO:tasks.workunit.client.1.smithi181.stdout:5/333: mkdir d4/de/d2f/d7a 0 2024-04-27T04:09:52.677 INFO:tasks.workunit.client.1.smithi181.stdout:5/334: write d4/d18/d54/f78 [749734,48574] 0 2024-04-27T04:09:52.678 INFO:tasks.workunit.client.1.smithi181.stdout:4/219: mknod d0/d15/d25/d2b/d7/c45 0 2024-04-27T04:09:52.679 INFO:tasks.workunit.client.1.smithi181.stdout:4/220: stat d0/d15/d25/d2a 0 2024-04-27T04:09:52.681 INFO:tasks.workunit.client.1.smithi181.stdout:1/234: unlink d1/d4c/ff 0 2024-04-27T04:09:52.689 INFO:tasks.workunit.client.1.smithi181.stdout:5/335: dwrite d4/d10/f59 [0,4194304] 0 2024-04-27T04:09:52.692 INFO:tasks.workunit.client.1.smithi181.stdout:7/176: mkdir d1/d2/d35 0 2024-04-27T04:09:52.697 INFO:tasks.workunit.client.1.smithi181.stdout:5/336: dwrite d4/d18/f48 [4194304,4194304] 0 2024-04-27T04:09:52.701 INFO:tasks.workunit.client.1.smithi181.stdout:1/235: unlink d1/d2/d5/d2c/c38 0 2024-04-27T04:09:52.708 INFO:tasks.workunit.client.1.smithi181.stdout:7/177: creat d1/d22/d15/d1f/f36 x:0 0 0 2024-04-27T04:09:52.714 INFO:tasks.workunit.client.1.smithi181.stdout:2/332: truncate d4/d9/d16/f26 9735406 0 2024-04-27T04:09:52.715 INFO:tasks.workunit.client.1.smithi181.stdout:7/178: mkdir d1/d2/d1c/d37 0 2024-04-27T04:09:52.717 INFO:tasks.workunit.client.1.smithi181.stdout:5/337: creat d4/d18/d67/d71/f7b x:0 0 0 2024-04-27T04:09:52.718 INFO:tasks.workunit.client.1.smithi181.stdout:5/338: fsync d4/de/f6e 0 2024-04-27T04:09:52.720 INFO:tasks.workunit.client.1.smithi181.stdout:2/333: creat d4/d9/d16/d45/f66 x:0 0 0 2024-04-27T04:09:52.723 INFO:tasks.workunit.client.1.smithi181.stdout:2/334: write d4/d9/f1a [6304308,41208] 0 2024-04-27T04:09:52.723 INFO:tasks.workunit.client.1.smithi181.stdout:7/179: rename d1/la to d1/d2/d1c/d2d/l38 0 2024-04-27T04:09:52.723 INFO:tasks.workunit.client.1.smithi181.stdout:2/335: readlink d4/d9/l31 0 2024-04-27T04:09:52.724 INFO:tasks.workunit.client.1.smithi181.stdout:7/180: fdatasync d1/d2/d1c/f1e 0 2024-04-27T04:09:52.728 INFO:tasks.workunit.client.1.smithi181.stdout:1/236: creat d1/d2/d5/f4f x:0 0 0 2024-04-27T04:09:52.728 INFO:tasks.workunit.client.1.smithi181.stdout:1/237: stat d1/d4c/f3e 0 2024-04-27T04:09:52.734 INFO:tasks.workunit.client.1.smithi181.stdout:1/238: dwrite d1/d4c/f3b [0,4194304] 0 2024-04-27T04:09:52.749 INFO:tasks.workunit.client.1.smithi181.stdout:7/181: symlink d1/d2/d35/l39 0 2024-04-27T04:09:52.750 INFO:tasks.workunit.client.1.smithi181.stdout:4/221: dwrite d0/d15/d25/d2b/f14 [0,4194304] 0 2024-04-27T04:09:52.752 INFO:tasks.workunit.client.1.smithi181.stdout:1/239: symlink d1/d2/d5/l50 0 2024-04-27T04:09:52.754 INFO:tasks.workunit.client.1.smithi181.stdout:1/240: read - d1/d2/d5/d6/f3d zero size 2024-04-27T04:09:52.759 INFO:tasks.workunit.client.1.smithi181.stdout:4/222: dwrite d0/d15/d25/d2b/f2c [0,4194304] 0 2024-04-27T04:09:52.759 INFO:tasks.workunit.client.1.smithi181.stdout:2/336: creat d4/d9/f67 x:0 0 0 2024-04-27T04:09:52.768 INFO:tasks.workunit.client.1.smithi181.stdout:1/241: creat d1/d2/d5/d6/db/d2d/d36/f51 x:0 0 0 2024-04-27T04:09:52.770 INFO:tasks.workunit.client.1.smithi181.stdout:1/242: write d1/d2/d5/d6/f22 [4868656,117668] 0 2024-04-27T04:09:52.770 INFO:tasks.workunit.client.1.smithi181.stdout:7/182: dread d1/d2/f19 [0,4194304] 0 2024-04-27T04:09:52.771 INFO:tasks.workunit.client.1.smithi181.stdout:1/243: chown d1/d2/d5/d6/db/f40 1026919716 1 2024-04-27T04:09:52.772 INFO:tasks.workunit.client.1.smithi181.stdout:5/339: rename d4/c1a to d4/de/c7c 0 2024-04-27T04:09:52.775 INFO:tasks.workunit.client.1.smithi181.stdout:7/183: read d1/d2/f12 [1691154,87086] 0 2024-04-27T04:09:52.776 INFO:tasks.workunit.client.1.smithi181.stdout:7/184: dread - d1/d22/d15/d1f/f36 zero size 2024-04-27T04:09:52.777 INFO:tasks.workunit.client.1.smithi181.stdout:4/223: creat d0/d15/d25/d2a/f46 x:0 0 0 2024-04-27T04:09:52.778 INFO:tasks.workunit.client.0.smithi162.stdout:8/51: creat d0/fd x:0 0 0 2024-04-27T04:09:52.780 INFO:tasks.workunit.client.0.smithi162.stdout:8/52: dread d0/fc [0,4194304] 0 2024-04-27T04:09:52.781 INFO:tasks.workunit.client.0.smithi162.stdout:1/43: creat d0/fb x:0 0 0 2024-04-27T04:09:52.783 INFO:tasks.workunit.client.1.smithi181.stdout:1/244: mkdir d1/d4c/d52 0 2024-04-27T04:09:52.785 INFO:tasks.workunit.client.1.smithi181.stdout:5/340: chown d4/de/d2c/d2e/d55/f3f 3 1 2024-04-27T04:09:52.785 INFO:tasks.workunit.client.0.smithi162.stdout:8/53: dwrite d0/fc [0,4194304] 0 2024-04-27T04:09:52.786 INFO:tasks.workunit.client.0.smithi162.stdout:8/54: chown d0/l5 113277 1 2024-04-27T04:09:52.787 INFO:tasks.workunit.client.0.smithi162.stdout:0/44: unlink d5/fb 0 2024-04-27T04:09:52.787 INFO:tasks.workunit.client.0.smithi162.stdout:1/44: dwrite d0/f2 [0,4194304] 0 2024-04-27T04:09:52.788 INFO:tasks.workunit.client.1.smithi181.stdout:2/337: mknod d4/c68 0 2024-04-27T04:09:52.788 INFO:tasks.workunit.client.0.smithi162.stdout:0/45: dread f1 [0,4194304] 0 2024-04-27T04:09:52.790 INFO:tasks.workunit.client.1.smithi181.stdout:4/224: readlink d0/l8 0 2024-04-27T04:09:52.790 INFO:tasks.workunit.client.1.smithi181.stdout:4/225: fdatasync d0/d15/f23 0 2024-04-27T04:09:52.794 INFO:tasks.workunit.client.1.smithi181.stdout:7/185: rename d1/c1d to d1/d22/d15/d13/c3a 0 2024-04-27T04:09:52.797 INFO:tasks.workunit.client.0.smithi162.stdout:8/55: dwrite d0/f4 [4194304,4194304] 0 2024-04-27T04:09:52.799 INFO:tasks.workunit.client.1.smithi181.stdout:1/245: mkdir d1/d4c/d1e/d20/d53 0 2024-04-27T04:09:52.801 INFO:tasks.workunit.client.1.smithi181.stdout:5/341: mknod d4/de/d2c/d2e/d55/d65/c7d 0 2024-04-27T04:09:52.811 INFO:tasks.workunit.client.0.smithi162.stdout:0/46: symlink d5/l13 0 2024-04-27T04:09:52.812 INFO:tasks.workunit.client.1.smithi181.stdout:5/342: mknod d4/de/d2c/c7e 0 2024-04-27T04:09:52.813 INFO:tasks.workunit.client.0.smithi162.stdout:0/47: dread d5/f8 [0,4194304] 0 2024-04-27T04:09:52.813 INFO:tasks.workunit.client.1.smithi181.stdout:0/282: sync 2024-04-27T04:09:52.814 INFO:tasks.workunit.client.1.smithi181.stdout:2/338: symlink d4/d9/df/d14/l69 0 2024-04-27T04:09:52.818 INFO:tasks.workunit.client.1.smithi181.stdout:1/246: symlink d1/d4c/d1e/l54 0 2024-04-27T04:09:52.819 INFO:tasks.workunit.client.0.smithi162.stdout:0/48: truncate d5/fa 91898 0 2024-04-27T04:09:52.820 INFO:tasks.workunit.client.0.smithi162.stdout:0/49: dread - d5/f10 zero size 2024-04-27T04:09:52.821 INFO:tasks.workunit.client.0.smithi162.stdout:1/45: link d0/d7/la d0/lc 0 2024-04-27T04:09:52.821 INFO:tasks.workunit.client.0.smithi162.stdout:1/46: write d0/f2 [886380,24321] 0 2024-04-27T04:09:52.823 INFO:tasks.workunit.client.0.smithi162.stdout:8/56: getdents d0 0 2024-04-27T04:09:52.825 INFO:tasks.workunit.client.1.smithi181.stdout:2/339: unlink d4/c3e 0 2024-04-27T04:09:52.826 INFO:tasks.workunit.client.0.smithi162.stdout:0/50: creat d5/dc/f14 x:0 0 0 2024-04-27T04:09:52.827 INFO:tasks.workunit.client.1.smithi181.stdout:7/186: link d1/d2/l6 d1/d22/d15/d1f/d34/l3b 0 2024-04-27T04:09:52.827 INFO:tasks.workunit.client.1.smithi181.stdout:7/187: dread - d1/d22/d15/d1f/f36 zero size 2024-04-27T04:09:52.828 INFO:tasks.workunit.client.0.smithi162.stdout:8/57: dwrite d0/f2 [0,4194304] 0 2024-04-27T04:09:52.831 INFO:tasks.workunit.client.0.smithi162.stdout:1/47: rename d0/f2 to d0/d7/fd 0 2024-04-27T04:09:52.834 INFO:tasks.workunit.client.1.smithi181.stdout:5/343: truncate d4/d18/d67/f50 298421 0 2024-04-27T04:09:52.836 INFO:tasks.workunit.client.0.smithi162.stdout:0/51: write d5/f9 [4608239,103820] 0 2024-04-27T04:09:52.836 INFO:tasks.workunit.client.0.smithi162.stdout:0/52: stat d5/f9 0 2024-04-27T04:09:52.836 INFO:tasks.workunit.client.0.smithi162.stdout:8/58: rmdir d0 39 2024-04-27T04:09:52.840 INFO:tasks.workunit.client.1.smithi181.stdout:1/247: getdents d1/d4c/d1e/d48 0 2024-04-27T04:09:52.842 INFO:tasks.workunit.client.0.smithi162.stdout:1/48: creat d0/fe x:0 0 0 2024-04-27T04:09:52.842 INFO:tasks.workunit.client.1.smithi181.stdout:1/248: write d1/d2/d5/d6/db/d2d/d36/f51 [1018858,112084] 0 2024-04-27T04:09:52.846 INFO:tasks.workunit.client.1.smithi181.stdout:0/283: dwrite dc/f22 [0,4194304] 0 2024-04-27T04:09:52.847 INFO:tasks.workunit.client.0.smithi162.stdout:0/53: rmdir d5/dc 39 2024-04-27T04:09:52.854 INFO:tasks.workunit.client.1.smithi181.stdout:7/188: link d1/d22/d15/d1f/f36 d1/d22/d15/d1f/d34/f3c 0 2024-04-27T04:09:52.858 INFO:tasks.workunit.client.1.smithi181.stdout:1/249: creat d1/d2/d5/d6/db/d2d/f55 x:0 0 0 2024-04-27T04:09:52.859 INFO:tasks.workunit.client.1.smithi181.stdout:0/284: creat dc/d1e/d26/d41/f71 x:0 0 0 2024-04-27T04:09:52.861 INFO:tasks.workunit.client.0.smithi162.stdout:8/59: getdents d0 0 2024-04-27T04:09:52.863 INFO:tasks.workunit.client.1.smithi181.stdout:0/285: dread dc/d1e/f2e [0,4194304] 0 2024-04-27T04:09:52.867 INFO:tasks.workunit.client.0.smithi162.stdout:1/49: getdents d0/d7 0 2024-04-27T04:09:52.868 INFO:tasks.workunit.client.0.smithi162.stdout:8/60: write d0/fa [4653831,130055] 0 2024-04-27T04:09:52.869 INFO:tasks.workunit.client.0.smithi162.stdout:0/54: rename d5/dc/fd to d5/f15 0 2024-04-27T04:09:52.870 INFO:tasks.workunit.client.0.smithi162.stdout:0/55: dread f1 [0,4194304] 0 2024-04-27T04:09:52.870 INFO:tasks.workunit.client.0.smithi162.stdout:1/50: write d0/d7/fd [3969757,22065] 0 2024-04-27T04:09:52.873 INFO:tasks.workunit.client.1.smithi181.stdout:2/340: getdents d4/d9/d44 0 2024-04-27T04:09:52.873 INFO:tasks.workunit.client.0.smithi162.stdout:1/51: chown d0/d7 2 1 2024-04-27T04:09:52.874 INFO:tasks.workunit.client.1.smithi181.stdout:0/286: fdatasync dc/f16 0 2024-04-27T04:09:52.874 INFO:tasks.workunit.client.1.smithi181.stdout:0/287: write dc/d1e/d26/d41/f4e [3180383,61349] 0 2024-04-27T04:09:52.874 INFO:tasks.workunit.client.1.smithi181.stdout:0/288: write dc/f5e [354254,39036] 0 2024-04-27T04:09:52.874 INFO:tasks.workunit.client.1.smithi181.stdout:1/250: mkdir d1/d4c/d1e/d20/d53/d56 0 2024-04-27T04:09:52.877 INFO:tasks.workunit.client.0.smithi162.stdout:0/56: creat d5/f16 x:0 0 0 2024-04-27T04:09:52.877 INFO:tasks.workunit.client.0.smithi162.stdout:1/52: dwrite d0/fb [0,4194304] 0 2024-04-27T04:09:52.878 INFO:tasks.workunit.client.0.smithi162.stdout:0/57: chown d5/dc 428 1 2024-04-27T04:09:52.879 INFO:tasks.workunit.client.0.smithi162.stdout:0/58: truncate d5/dc/f14 586631 0 2024-04-27T04:09:52.879 INFO:tasks.workunit.client.1.smithi181.stdout:7/189: truncate d1/d2/f25 1911842 0 2024-04-27T04:09:52.880 INFO:tasks.workunit.client.0.smithi162.stdout:0/59: truncate d5/dc/f14 666872 0 2024-04-27T04:09:52.880 INFO:tasks.workunit.client.1.smithi181.stdout:7/190: readlink d1/d2/d35/l39 0 2024-04-27T04:09:52.880 INFO:tasks.workunit.client.1.smithi181.stdout:2/341: mknod d4/d9/d4f/c6a 0 2024-04-27T04:09:52.882 INFO:tasks.workunit.client.0.smithi162.stdout:1/53: symlink d0/lf 0 2024-04-27T04:09:52.886 INFO:tasks.workunit.client.0.smithi162.stdout:1/54: dread d0/d7/fd [0,4194304] 0 2024-04-27T04:09:52.889 INFO:tasks.workunit.client.1.smithi181.stdout:5/344: link d4/de/d2f/l38 d4/l7f 0 2024-04-27T04:09:52.889 INFO:tasks.workunit.client.0.smithi162.stdout:1/55: dread d0/d7/fd [0,4194304] 0 2024-04-27T04:09:52.890 INFO:tasks.workunit.client.1.smithi181.stdout:0/289: dread dc/f42 [0,4194304] 0 2024-04-27T04:09:52.892 INFO:tasks.workunit.client.0.smithi162.stdout:0/60: link d5/dc/f12 d5/dc/f17 0 2024-04-27T04:09:52.893 INFO:tasks.workunit.client.0.smithi162.stdout:1/56: mkdir d0/d7/d10 0 2024-04-27T04:09:52.897 INFO:tasks.workunit.client.1.smithi181.stdout:1/251: mknod d1/d4c/d1e/c57 0 2024-04-27T04:09:52.898 INFO:tasks.workunit.client.0.smithi162.stdout:1/57: mkdir d0/d11 0 2024-04-27T04:09:52.898 INFO:tasks.workunit.client.1.smithi181.stdout:1/252: write d1/d2/d5/d6/f22 [8802484,1576] 0 2024-04-27T04:09:52.899 INFO:tasks.workunit.client.1.smithi181.stdout:1/253: dread - d1/d2/d5/d6/f3d zero size 2024-04-27T04:09:52.899 INFO:tasks.workunit.client.1.smithi181.stdout:5/345: write d4/de/f1e [1636676,24036] 0 2024-04-27T04:09:52.900 INFO:tasks.workunit.client.1.smithi181.stdout:5/346: dread - d4/de/d2c/d2e/f56 zero size 2024-04-27T04:09:52.902 INFO:tasks.workunit.client.0.smithi162.stdout:1/58: stat d0/l3 0 2024-04-27T04:09:52.904 INFO:tasks.workunit.client.0.smithi162.stdout:0/61: dread d5/dc/f17 [0,4194304] 0 2024-04-27T04:09:52.906 INFO:tasks.workunit.client.1.smithi181.stdout:7/191: dwrite d1/d22/d15/d1f/d34/f3c [0,4194304] 0 2024-04-27T04:09:52.908 INFO:tasks.workunit.client.1.smithi181.stdout:0/290: write dc/f1b [3271562,106170] 0 2024-04-27T04:09:52.912 INFO:tasks.workunit.client.1.smithi181.stdout:1/254: mknod d1/d4c/d1e/d20/d53/d56/c58 0 2024-04-27T04:09:52.912 INFO:tasks.workunit.client.1.smithi181.stdout:0/291: read dc/d1e/d26/d41/f4e [3551321,89614] 0 2024-04-27T04:09:52.912 INFO:tasks.workunit.client.1.smithi181.stdout:0/292: stat dc/ff 0 2024-04-27T04:09:52.913 INFO:tasks.workunit.client.1.smithi181.stdout:0/293: dread - dc/d1f/d3b/f50 zero size 2024-04-27T04:09:52.914 INFO:tasks.workunit.client.0.smithi162.stdout:0/62: getdents d5 0 2024-04-27T04:09:52.917 INFO:tasks.workunit.client.1.smithi181.stdout:2/342: rename d4/d9/d35/d42/l62 to d4/l6b 0 2024-04-27T04:09:52.918 INFO:tasks.workunit.client.1.smithi181.stdout:7/192: write d1/df/f18 [5232023,123178] 0 2024-04-27T04:09:52.919 INFO:tasks.workunit.client.1.smithi181.stdout:2/343: chown d4/d9/d16/c4a 493 1 2024-04-27T04:09:52.921 INFO:tasks.workunit.client.1.smithi181.stdout:1/255: dwrite f0 [0,4194304] 0 2024-04-27T04:09:52.921 INFO:tasks.workunit.client.1.smithi181.stdout:0/294: creat dc/d54/f72 x:0 0 0 2024-04-27T04:09:52.922 INFO:tasks.workunit.client.1.smithi181.stdout:2/344: dread - d4/d9/d44/d63/f53 zero size 2024-04-27T04:09:52.922 INFO:tasks.workunit.client.1.smithi181.stdout:0/295: dread - dc/d1e/f63 zero size 2024-04-27T04:09:52.923 INFO:tasks.workunit.client.1.smithi181.stdout:5/347: rename d4/de/d2c/d2e/d55/d65 to d4/d10/d72/d80 0 2024-04-27T04:09:52.923 INFO:tasks.workunit.client.1.smithi181.stdout:2/345: write d4/d9/f11 [3571009,42647] 0 2024-04-27T04:09:52.924 INFO:tasks.workunit.client.0.smithi162.stdout:0/63: getdents d5 0 2024-04-27T04:09:52.924 INFO:tasks.workunit.client.1.smithi181.stdout:0/296: readlink dc/d1e/d26/d41/d6f/l6e 0 2024-04-27T04:09:52.925 INFO:tasks.workunit.client.0.smithi162.stdout:0/64: chown d5/dc/f17 15229249 1 2024-04-27T04:09:52.925 INFO:tasks.workunit.client.0.smithi162.stdout:0/65: dread - d5/f16 zero size 2024-04-27T04:09:52.925 INFO:tasks.workunit.client.0.smithi162.stdout:0/66: dread - d5/f11 zero size 2024-04-27T04:09:52.926 INFO:tasks.workunit.client.1.smithi181.stdout:7/193: mkdir d1/d2/d1c/d2d/d3d 0 2024-04-27T04:09:52.927 INFO:tasks.workunit.client.0.smithi162.stdout:0/67: read d5/f9 [4561811,17272] 0 2024-04-27T04:09:52.927 INFO:tasks.workunit.client.1.smithi181.stdout:1/256: chown d1/l35 1 1 2024-04-27T04:09:52.928 INFO:tasks.workunit.client.1.smithi181.stdout:2/346: creat d4/d9/d16/d45/f6c x:0 0 0 2024-04-27T04:09:52.928 INFO:tasks.workunit.client.1.smithi181.stdout:7/194: read - d1/d2/f27 zero size 2024-04-27T04:09:52.929 INFO:tasks.workunit.client.1.smithi181.stdout:2/347: readlink d4/d9/d4f/l65 0 2024-04-27T04:09:52.929 INFO:tasks.workunit.client.0.smithi162.stdout:0/68: dread d5/dc/f14 [0,4194304] 0 2024-04-27T04:09:52.930 INFO:tasks.workunit.client.1.smithi181.stdout:5/348: mknod d4/de/d2c/d2e/d55/c81 0 2024-04-27T04:09:52.931 INFO:tasks.workunit.client.1.smithi181.stdout:2/348: write d4/f2c [516055,18478] 0 2024-04-27T04:09:52.933 INFO:tasks.workunit.client.1.smithi181.stdout:1/257: dread d1/d2/d5/d6/db/f40 [0,4194304] 0 2024-04-27T04:09:52.933 INFO:tasks.workunit.client.1.smithi181.stdout:5/349: truncate d4/d10/d29/f79 178937 0 2024-04-27T04:09:52.933 INFO:tasks.workunit.client.1.smithi181.stdout:0/297: rmdir dc/d1e/d26/d41 39 2024-04-27T04:09:52.934 INFO:tasks.workunit.client.1.smithi181.stdout:2/349: write d4/d9/df/f41 [1382708,72352] 0 2024-04-27T04:09:52.936 INFO:tasks.workunit.client.0.smithi162.stdout:0/69: creat d5/dc/f18 x:0 0 0 2024-04-27T04:09:52.936 INFO:tasks.workunit.client.1.smithi181.stdout:2/350: write d4/f59 [5181303,47741] 0 2024-04-27T04:09:52.936 INFO:tasks.workunit.client.1.smithi181.stdout:1/258: truncate d1/d4c/d1e/f49 289094 0 2024-04-27T04:09:52.942 INFO:tasks.workunit.client.1.smithi181.stdout:0/298: chown dc/d1e/d26/d41/l55 3 1 2024-04-27T04:09:52.946 INFO:tasks.workunit.client.1.smithi181.stdout:5/350: rename d4/d18/l4e to d4/d10/d72/l82 0 2024-04-27T04:09:52.948 INFO:tasks.workunit.client.1.smithi181.stdout:5/351: dread - d4/de/d2c/d2e/f56 zero size 2024-04-27T04:09:52.949 INFO:tasks.workunit.client.1.smithi181.stdout:0/299: dwrite dc/f22 [0,4194304] 0 2024-04-27T04:09:52.949 INFO:tasks.workunit.client.1.smithi181.stdout:0/300: readlink l5 0 2024-04-27T04:09:52.957 INFO:tasks.workunit.client.1.smithi181.stdout:2/351: creat d4/d9/f6d x:0 0 0 2024-04-27T04:09:52.958 INFO:tasks.workunit.client.1.smithi181.stdout:1/259: creat d1/d2/d5/f59 x:0 0 0 2024-04-27T04:09:52.961 INFO:tasks.workunit.client.0.smithi162.stdout:0/70: dwrite d5/f8 [0,4194304] 0 2024-04-27T04:09:52.961 INFO:tasks.workunit.client.0.smithi162.stdout:0/71: read - d5/dc/f18 zero size 2024-04-27T04:09:52.962 INFO:tasks.workunit.client.1.smithi181.stdout:2/352: chown d4/d9/f2d 0 1 2024-04-27T04:09:52.962 INFO:tasks.workunit.client.0.smithi162.stdout:0/72: dread - d5/dc/f18 zero size 2024-04-27T04:09:52.963 INFO:tasks.workunit.client.1.smithi181.stdout:7/195: truncate d1/d22/d15/d1f/f36 402708 0 2024-04-27T04:09:52.963 INFO:tasks.workunit.client.0.smithi162.stdout:0/73: dread d5/fa [0,4194304] 0 2024-04-27T04:09:52.965 INFO:tasks.workunit.client.1.smithi181.stdout:1/260: dwrite d1/f2f [4194304,4194304] 0 2024-04-27T04:09:52.967 INFO:tasks.workunit.client.1.smithi181.stdout:5/352: link d4/d10/d29/c42 d4/d10/d72/d80/c83 0 2024-04-27T04:09:52.974 INFO:tasks.workunit.client.1.smithi181.stdout:0/301: getdents dc/d1f/d3b 0 2024-04-27T04:09:52.976 INFO:tasks.workunit.client.1.smithi181.stdout:1/261: dwrite d1/d2/d5/d6/f3d [0,4194304] 0 2024-04-27T04:09:52.978 INFO:tasks.workunit.client.1.smithi181.stdout:0/302: chown dc/d1e/d26/d41/f4f 27 1 2024-04-27T04:09:52.979 INFO:tasks.workunit.client.0.smithi162.stdout:0/74: getdents d5/dc 0 2024-04-27T04:09:52.981 INFO:tasks.workunit.client.1.smithi181.stdout:7/196: fdatasync d1/d2/f25 0 2024-04-27T04:09:52.984 INFO:tasks.workunit.client.1.smithi181.stdout:1/262: rmdir d1/d2/d5/d6 39 2024-04-27T04:09:52.985 INFO:tasks.workunit.client.1.smithi181.stdout:5/353: dwrite d4/de/d2c/d46/f60 [0,4194304] 0 2024-04-27T04:09:52.987 INFO:tasks.workunit.client.1.smithi181.stdout:5/354: write d4/de/f1e [2947927,39730] 0 2024-04-27T04:09:52.987 INFO:tasks.workunit.client.0.smithi162.stdout:0/75: rename d5/lf to d5/dc/l19 0 2024-04-27T04:09:52.988 INFO:tasks.workunit.client.1.smithi181.stdout:5/355: readlink d4/l32 0 2024-04-27T04:09:52.988 INFO:tasks.workunit.client.0.smithi162.stdout:0/76: write d5/dc/fe [856417,36283] 0 2024-04-27T04:09:52.992 INFO:tasks.workunit.client.1.smithi181.stdout:0/303: unlink dc/d1e/d26/f6a 0 2024-04-27T04:09:52.994 INFO:tasks.workunit.client.1.smithi181.stdout:7/197: dread d1/f8 [0,4194304] 0 2024-04-27T04:09:52.998 INFO:tasks.workunit.client.0.smithi162.stdout:0/77: dread d5/f9 [0,4194304] 0 2024-04-27T04:09:52.998 INFO:tasks.workunit.client.0.smithi162.stdout:0/78: write d5/f8 [429926,1145] 0 2024-04-27T04:09:52.998 INFO:tasks.workunit.client.1.smithi181.stdout:1/263: write d1/d2/d5/d6/db/d2d/d36/f4a [197393,23928] 0 2024-04-27T04:09:52.999 INFO:tasks.workunit.client.1.smithi181.stdout:5/356: creat d4/d10/d29/f84 x:0 0 0 2024-04-27T04:09:52.999 INFO:tasks.workunit.client.0.smithi162.stdout:0/79: truncate d5/dc/fe 1718756 0 2024-04-27T04:09:53.002 INFO:tasks.workunit.client.1.smithi181.stdout:1/264: read d1/d2/d5/d6/db/d2d/d36/f4a [23529,77314] 0 2024-04-27T04:09:53.002 INFO:tasks.workunit.client.1.smithi181.stdout:0/304: rename dc/d1f/c2a to dc/d1e/c73 0 2024-04-27T04:09:53.004 INFO:tasks.workunit.client.0.smithi162.stdout:0/80: mknod d5/dc/c1a 0 2024-04-27T04:09:53.006 INFO:tasks.workunit.client.1.smithi181.stdout:5/357: dwrite d4/d18/d54/f78 [0,4194304] 0 2024-04-27T04:09:53.006 INFO:tasks.workunit.client.1.smithi181.stdout:7/198: chown d1/d22/d15/d13/c3a 404179 1 2024-04-27T04:09:53.008 INFO:tasks.workunit.client.1.smithi181.stdout:2/353: link d4/c39 d4/d9/c6e 0 2024-04-27T04:09:53.010 INFO:tasks.workunit.client.0.smithi162.stdout:0/81: dwrite d5/f16 [0,4194304] 0 2024-04-27T04:09:53.010 INFO:tasks.workunit.client.1.smithi181.stdout:1/265: dread d1/d2/d5/d6/f22 [4194304,4194304] 0 2024-04-27T04:09:53.011 INFO:tasks.workunit.client.1.smithi181.stdout:1/266: fdatasync d1/d2/d5/f45 0 2024-04-27T04:09:53.015 INFO:tasks.workunit.client.0.smithi162.stdout:0/82: truncate d5/fa 734021 0 2024-04-27T04:09:53.015 INFO:tasks.workunit.client.0.smithi162.stdout:0/83: chown d5/f15 1724 1 2024-04-27T04:09:53.018 INFO:tasks.workunit.client.1.smithi181.stdout:1/267: mknod d1/d2/d5/d2c/c5a 0 2024-04-27T04:09:53.023 INFO:tasks.workunit.client.1.smithi181.stdout:7/199: dread d1/d22/d15/d1f/d34/f3c [0,4194304] 0 2024-04-27T04:09:53.024 INFO:tasks.workunit.client.1.smithi181.stdout:7/200: fsync d1/d22/d15/d13/f30 0 2024-04-27T04:09:53.024 INFO:tasks.workunit.client.1.smithi181.stdout:2/354: mknod d4/d9/c6f 0 2024-04-27T04:09:53.025 INFO:tasks.workunit.client.1.smithi181.stdout:0/305: write dc/f2c [1489963,94011] 0 2024-04-27T04:09:53.027 INFO:tasks.workunit.client.1.smithi181.stdout:1/268: creat d1/d2/d5/d6/f5b x:0 0 0 2024-04-27T04:09:53.027 INFO:tasks.workunit.client.1.smithi181.stdout:2/355: write d4/f30 [2927356,24578] 0 2024-04-27T04:09:53.028 INFO:tasks.workunit.client.1.smithi181.stdout:0/306: truncate dc/d1f/d3b/f50 254518 0 2024-04-27T04:09:53.029 INFO:tasks.workunit.client.0.smithi162.stdout:0/84: getdents d5 0 2024-04-27T04:09:53.030 INFO:tasks.workunit.client.0.smithi162.stdout:0/85: chown d5/f15 47927202 1 2024-04-27T04:09:53.030 INFO:tasks.workunit.client.1.smithi181.stdout:5/358: dread d4/d10/d29/f3c [0,4194304] 0 2024-04-27T04:09:53.030 INFO:tasks.workunit.client.0.smithi162.stdout:0/86: chown d5/dc 4 1 2024-04-27T04:09:53.033 INFO:tasks.workunit.client.1.smithi181.stdout:1/269: creat d1/d4c/d1e/d20/f5c x:0 0 0 2024-04-27T04:09:53.035 INFO:tasks.workunit.client.1.smithi181.stdout:5/359: write d4/d10/d72/f77 [2657021,57892] 0 2024-04-27T04:09:53.035 INFO:tasks.workunit.client.1.smithi181.stdout:2/356: mkdir d4/d70 0 2024-04-27T04:09:53.038 INFO:tasks.workunit.client.1.smithi181.stdout:0/307: mknod dc/d1e/d26/d68/c74 0 2024-04-27T04:09:53.040 INFO:tasks.workunit.client.1.smithi181.stdout:2/357: dread d4/f8 [4194304,4194304] 0 2024-04-27T04:09:53.040 INFO:tasks.workunit.client.1.smithi181.stdout:1/270: rename d1/d2/d5/d6/f5b to d1/d4c/d1e/d48/f5d 0 2024-04-27T04:09:53.040 INFO:tasks.workunit.client.1.smithi181.stdout:5/360: stat d4/de/c33 0 2024-04-27T04:09:53.042 INFO:tasks.workunit.client.1.smithi181.stdout:5/361: write d4/de/f1e [3918105,117178] 0 2024-04-27T04:09:53.043 INFO:tasks.workunit.client.1.smithi181.stdout:7/201: link d1/d22/d15/d1f/d34/f3c d1/d2/d1c/d2d/f3e 0 2024-04-27T04:09:53.043 INFO:tasks.workunit.client.1.smithi181.stdout:5/362: write d4/de/d2c/f75 [345327,87043] 0 2024-04-27T04:09:53.043 INFO:tasks.workunit.client.1.smithi181.stdout:7/202: readlink d1/d2/d35/l39 0 2024-04-27T04:09:53.045 INFO:tasks.workunit.client.1.smithi181.stdout:0/308: mkdir dc/d1f/d3b/d75 0 2024-04-27T04:09:53.047 INFO:tasks.workunit.client.1.smithi181.stdout:0/309: dread - dc/d1e/d26/d41/d6f/f69 zero size 2024-04-27T04:09:53.048 INFO:tasks.workunit.client.1.smithi181.stdout:1/271: creat d1/d2/d5/d6/db/d2d/d36/f5e x:0 0 0 2024-04-27T04:09:53.049 INFO:tasks.workunit.client.0.smithi162.stdout:0/87: dwrite f0 [4194304,4194304] 0 2024-04-27T04:09:53.052 INFO:tasks.workunit.client.1.smithi181.stdout:7/203: unlink d1/d22/d15/d13/f30 0 2024-04-27T04:09:53.053 INFO:tasks.workunit.client.1.smithi181.stdout:5/363: dwrite d4/de/d2f/f5d [4194304,4194304] 0 2024-04-27T04:09:53.060 INFO:tasks.workunit.client.1.smithi181.stdout:2/358: dread d4/f6 [0,4194304] 0 2024-04-27T04:09:53.062 INFO:tasks.workunit.client.0.smithi162.stdout:0/88: link d5/dc/c1a d5/c1b 0 2024-04-27T04:09:53.063 INFO:tasks.workunit.client.1.smithi181.stdout:6/155: sync 2024-04-27T04:09:53.063 INFO:tasks.workunit.client.1.smithi181.stdout:8/323: sync 2024-04-27T04:09:53.063 INFO:tasks.workunit.client.1.smithi181.stdout:3/257: sync 2024-04-27T04:09:53.063 INFO:tasks.workunit.client.1.smithi181.stdout:4/226: sync 2024-04-27T04:09:53.063 INFO:tasks.workunit.client.1.smithi181.stdout:9/305: sync 2024-04-27T04:09:53.064 INFO:tasks.workunit.client.1.smithi181.stdout:1/272: fsync d1/d2/d5/d6/f1a 0 2024-04-27T04:09:53.065 INFO:tasks.workunit.client.1.smithi181.stdout:7/204: creat d1/d2/d1c/d37/f3f x:0 0 0 2024-04-27T04:09:53.065 INFO:tasks.workunit.client.0.smithi162.stdout:0/89: symlink d5/l1c 0 2024-04-27T04:09:53.066 INFO:tasks.workunit.client.1.smithi181.stdout:9/306: chown d0/d44/d4b/d3d/c42 1054860 1 2024-04-27T04:09:53.067 INFO:tasks.workunit.client.1.smithi181.stdout:9/307: chown d0/d4d/c53 80840 1 2024-04-27T04:09:53.067 INFO:tasks.workunit.client.1.smithi181.stdout:1/273: write d1/d2/d5/d6/f4e [153754,10076] 0 2024-04-27T04:09:53.068 INFO:tasks.workunit.client.1.smithi181.stdout:3/258: stat d0/ca 0 2024-04-27T04:09:53.070 INFO:tasks.workunit.client.1.smithi181.stdout:6/156: mkdir d7/dc/d1c/d34 0 2024-04-27T04:09:53.071 INFO:tasks.workunit.client.1.smithi181.stdout:4/227: mknod d0/d15/d25/c47 0 2024-04-27T04:09:53.072 INFO:tasks.workunit.client.1.smithi181.stdout:7/205: creat d1/d22/d15/d13/d1b/f40 x:0 0 0 2024-04-27T04:09:53.074 INFO:tasks.workunit.client.1.smithi181.stdout:1/274: write d1/d2/f13 [1036832,55671] 0 2024-04-27T04:09:53.074 INFO:tasks.workunit.client.1.smithi181.stdout:2/359: dwrite d4/f6 [0,4194304] 0 2024-04-27T04:09:53.077 INFO:tasks.workunit.client.1.smithi181.stdout:6/157: chown d7/dc/d1e/f2a 7664 1 2024-04-27T04:09:53.079 INFO:tasks.workunit.client.1.smithi181.stdout:4/228: mkdir d0/d15/d48 0 2024-04-27T04:09:53.079 INFO:tasks.workunit.client.1.smithi181.stdout:4/229: readlink d0/d15/l1d 0 2024-04-27T04:09:53.084 INFO:tasks.workunit.client.1.smithi181.stdout:8/324: truncate d1/d6/d2f/f45 3831536 0 2024-04-27T04:09:53.085 INFO:tasks.workunit.client.1.smithi181.stdout:6/158: creat d7/dc/d1c/f35 x:0 0 0 2024-04-27T04:09:53.087 INFO:tasks.workunit.client.1.smithi181.stdout:1/275: mkdir d1/d2/d5/d6/db/d2d/d36/d5f 0 2024-04-27T04:09:53.088 INFO:tasks.workunit.client.1.smithi181.stdout:1/276: chown d1/d2/d5/d6/db/d2d/d36 94680 1 2024-04-27T04:09:53.088 INFO:tasks.workunit.client.1.smithi181.stdout:9/308: link d0/d44/d4b/l3b d0/d4d/l6f 0 2024-04-27T04:09:53.090 INFO:tasks.workunit.client.1.smithi181.stdout:5/364: truncate d4/de/f1e 1625685 0 2024-04-27T04:09:53.092 INFO:tasks.workunit.client.1.smithi181.stdout:6/159: symlink d7/d27/l36 0 2024-04-27T04:09:53.094 INFO:tasks.workunit.client.1.smithi181.stdout:4/230: symlink d0/d15/d48/l49 0 2024-04-27T04:09:53.095 INFO:tasks.workunit.client.1.smithi181.stdout:4/231: chown d0/d15/d25/d2b/d7/c3a 105067412 1 2024-04-27T04:09:53.097 INFO:tasks.workunit.client.1.smithi181.stdout:4/232: fsync d0/d15/f2f 0 2024-04-27T04:09:53.099 INFO:tasks.workunit.client.1.smithi181.stdout:5/365: read d4/de/f47 [436661,94678] 0 2024-04-27T04:09:53.099 INFO:tasks.workunit.client.1.smithi181.stdout:9/309: rename d0/d44/f49 to d0/d4d/f70 0 2024-04-27T04:09:53.100 INFO:tasks.workunit.client.1.smithi181.stdout:4/233: write d0/f30 [1270965,90086] 0 2024-04-27T04:09:53.103 INFO:tasks.workunit.client.1.smithi181.stdout:7/206: dread d1/df/f18 [0,4194304] 0 2024-04-27T04:09:53.105 INFO:tasks.workunit.client.1.smithi181.stdout:1/277: dread d1/d2/d5/d6/f1a [0,4194304] 0 2024-04-27T04:09:53.106 INFO:tasks.workunit.client.1.smithi181.stdout:3/259: dwrite d0/de/d20/f46 [0,4194304] 0 2024-04-27T04:09:53.106 INFO:tasks.workunit.client.1.smithi181.stdout:8/325: fsync d1/d6/d2f/f45 0 2024-04-27T04:09:53.107 INFO:tasks.workunit.client.1.smithi181.stdout:8/326: stat d1/d6/d2f/d40/d2a/f37 0 2024-04-27T04:09:53.108 INFO:tasks.workunit.client.1.smithi181.stdout:8/327: chown d1/d6/d2f/d40 0 1 2024-04-27T04:09:53.111 INFO:tasks.workunit.client.1.smithi181.stdout:4/234: rmdir d0/d15/d25 39 2024-04-27T04:09:53.111 INFO:tasks.workunit.client.1.smithi181.stdout:4/235: readlink d0/d15/de/l13 0 2024-04-27T04:09:53.112 INFO:tasks.workunit.client.1.smithi181.stdout:9/310: symlink d0/d44/d4b/d43/l71 0 2024-04-27T04:09:53.116 INFO:tasks.workunit.client.1.smithi181.stdout:7/207: creat d1/d2/d1c/d2d/f41 x:0 0 0 2024-04-27T04:09:53.119 INFO:tasks.workunit.client.1.smithi181.stdout:1/278: mkdir d1/d4c/d60 0 2024-04-27T04:09:53.121 INFO:tasks.workunit.client.1.smithi181.stdout:5/366: unlink d4/de/d2f/l40 0 2024-04-27T04:09:53.124 INFO:tasks.workunit.client.1.smithi181.stdout:4/236: fsync d0/d15/d25/d2b/f2c 0 2024-04-27T04:09:53.126 INFO:tasks.workunit.client.1.smithi181.stdout:9/311: rmdir d0/d44/d4b/d50 39 2024-04-27T04:09:53.126 INFO:tasks.workunit.client.1.smithi181.stdout:6/160: link f3 d7/d31/f37 0 2024-04-27T04:09:53.138 INFO:tasks.workunit.client.1.smithi181.stdout:9/312: write d0/d44/d4b/d43/f56 [436989,53081] 0 2024-04-27T04:09:53.139 INFO:tasks.workunit.client.1.smithi181.stdout:1/279: dwrite d1/d4c/d1e/d20/f26 [0,4194304] 0 2024-04-27T04:09:53.140 INFO:tasks.workunit.client.1.smithi181.stdout:8/328: dwrite d1/f6b [0,4194304] 0 2024-04-27T04:09:53.142 INFO:tasks.workunit.client.1.smithi181.stdout:3/260: write d0/f23 [1689977,43550] 0 2024-04-27T04:09:53.144 INFO:tasks.workunit.client.1.smithi181.stdout:5/367: dwrite d4/d18/f41 [0,4194304] 0 2024-04-27T04:09:53.146 INFO:tasks.workunit.client.1.smithi181.stdout:8/329: readlink d1/d6/d2f/d30/l6d 0 2024-04-27T04:09:53.151 INFO:tasks.workunit.client.1.smithi181.stdout:6/161: dwrite d7/f8 [4194304,4194304] 0 2024-04-27T04:09:53.152 INFO:tasks.workunit.client.1.smithi181.stdout:7/208: truncate d1/d22/d15/d1f/f36 198784 0 2024-04-27T04:09:53.152 INFO:tasks.workunit.client.1.smithi181.stdout:6/162: dread - d7/dc/d1c/f32 zero size 2024-04-27T04:09:53.155 INFO:tasks.workunit.client.1.smithi181.stdout:1/280: dread d1/d2/d5/d6/db/f19 [0,4194304] 0 2024-04-27T04:09:53.156 INFO:tasks.workunit.client.1.smithi181.stdout:5/368: rmdir d4/d10 39 2024-04-27T04:09:53.157 INFO:tasks.workunit.client.1.smithi181.stdout:1/281: write d1/d2/d5/d6/f3d [2566201,13033] 0 2024-04-27T04:09:53.159 INFO:tasks.workunit.client.1.smithi181.stdout:1/282: chown d1/d4c/d1e/l44 855134 1 2024-04-27T04:09:53.160 INFO:tasks.workunit.client.1.smithi181.stdout:9/313: symlink d0/d44/l72 0 2024-04-27T04:09:53.165 INFO:tasks.workunit.client.1.smithi181.stdout:1/283: dread d1/d4c/f4d [0,4194304] 0 2024-04-27T04:09:53.166 INFO:tasks.workunit.client.1.smithi181.stdout:6/163: mkdir d7/d31/d38 0 2024-04-27T04:09:53.167 INFO:tasks.workunit.client.1.smithi181.stdout:1/284: dread d1/d4c/f4d [0,4194304] 0 2024-04-27T04:09:53.169 INFO:tasks.workunit.client.1.smithi181.stdout:8/330: link d1/l22 d1/d6/d2f/d40/l73 0 2024-04-27T04:09:53.171 INFO:tasks.workunit.client.1.smithi181.stdout:6/164: rmdir d7/dc 39 2024-04-27T04:09:53.173 INFO:tasks.workunit.client.1.smithi181.stdout:1/285: creat d1/d4c/d1e/d20/f61 x:0 0 0 2024-04-27T04:09:53.174 INFO:tasks.workunit.client.1.smithi181.stdout:8/331: truncate d1/d6/d2f/d40/f6e 2037756 0 2024-04-27T04:09:53.175 INFO:tasks.workunit.client.1.smithi181.stdout:9/314: dwrite d0/d44/f48 [0,4194304] 0 2024-04-27T04:09:53.177 INFO:tasks.workunit.client.1.smithi181.stdout:4/237: truncate d0/f3b 412391 0 2024-04-27T04:09:53.182 INFO:tasks.workunit.client.1.smithi181.stdout:6/165: symlink d7/d31/d38/l39 0 2024-04-27T04:09:53.188 INFO:tasks.workunit.client.1.smithi181.stdout:8/332: mknod d1/d6/d6f/c74 0 2024-04-27T04:09:53.191 INFO:tasks.workunit.client.1.smithi181.stdout:1/286: dwrite d1/d2/d5/d6/db/d2d/d36/f51 [0,4194304] 0 2024-04-27T04:09:53.193 INFO:tasks.workunit.client.1.smithi181.stdout:3/261: truncate d0/de/d20/f46 2242178 0 2024-04-27T04:09:53.197 INFO:tasks.workunit.client.1.smithi181.stdout:9/315: dwrite d0/f2b [0,4194304] 0 2024-04-27T04:09:53.200 INFO:tasks.workunit.client.1.smithi181.stdout:8/333: symlink d1/d6/d2f/d40/d2a/l75 0 2024-04-27T04:09:53.202 INFO:tasks.workunit.client.1.smithi181.stdout:7/209: write d1/d22/d15/d1f/d34/f3c [106959,62818] 0 2024-04-27T04:09:53.203 INFO:tasks.workunit.client.1.smithi181.stdout:3/262: truncate d0/f19 1279961 0 2024-04-27T04:09:53.207 INFO:tasks.workunit.client.1.smithi181.stdout:1/287: symlink d1/d2/d5/d6/l62 0 2024-04-27T04:09:53.210 INFO:tasks.workunit.client.1.smithi181.stdout:6/166: dread f0 [0,4194304] 0 2024-04-27T04:09:53.211 INFO:tasks.workunit.client.1.smithi181.stdout:9/316: write d0/d4d/f70 [490859,51866] 0 2024-04-27T04:09:53.211 INFO:tasks.workunit.client.1.smithi181.stdout:4/238: dwrite d0/d15/d25/d2b/d7/f31 [0,4194304] 0 2024-04-27T04:09:53.214 INFO:tasks.workunit.client.1.smithi181.stdout:7/210: mknod d1/d2/d1c/d37/c42 0 2024-04-27T04:09:53.216 INFO:tasks.workunit.client.1.smithi181.stdout:3/263: symlink d0/l5a 0 2024-04-27T04:09:53.217 INFO:tasks.workunit.client.1.smithi181.stdout:1/288: unlink d1/d2/d5/d2c/f3f 0 2024-04-27T04:09:53.218 INFO:tasks.workunit.client.1.smithi181.stdout:5/369: dwrite d4/de/d2c/f75 [0,4194304] 0 2024-04-27T04:09:53.220 INFO:tasks.workunit.client.1.smithi181.stdout:7/211: truncate d1/d22/f26 1611115 0 2024-04-27T04:09:53.221 INFO:tasks.workunit.client.1.smithi181.stdout:4/239: chown d0/d15/d25/d2b/l37 56576 1 2024-04-27T04:09:53.223 INFO:tasks.workunit.client.1.smithi181.stdout:8/334: dwrite d1/d6/d1a/d25/f5f [0,4194304] 0 2024-04-27T04:09:53.224 INFO:tasks.workunit.client.1.smithi181.stdout:7/212: fsync d1/d2/d1c/f2c 0 2024-04-27T04:09:53.228 INFO:tasks.workunit.client.1.smithi181.stdout:1/289: write d1/d2/d5/f45 [513451,101517] 0 2024-04-27T04:09:53.228 INFO:tasks.workunit.client.1.smithi181.stdout:9/317: write d0/d40/f29 [1370080,19786] 0 2024-04-27T04:09:53.231 INFO:tasks.workunit.client.1.smithi181.stdout:7/213: truncate d1/d22/d15/d1f/f36 1011981 0 2024-04-27T04:09:53.236 INFO:tasks.workunit.client.1.smithi181.stdout:1/290: dwrite d1/d4c/f1f [4194304,4194304] 0 2024-04-27T04:09:53.238 INFO:tasks.workunit.client.1.smithi181.stdout:7/214: truncate d1/d2/d1c/d2d/f3e 1470094 0 2024-04-27T04:09:53.242 INFO:tasks.workunit.client.1.smithi181.stdout:4/240: rmdir d0/d15/d25/d2b/d7 39 2024-04-27T04:09:53.247 INFO:tasks.workunit.client.1.smithi181.stdout:9/318: chown d0/d40/l1b 101569 1 2024-04-27T04:09:53.250 INFO:tasks.workunit.client.1.smithi181.stdout:3/264: mkdir d0/d3e/d52/d44/d56/d5b 0 2024-04-27T04:09:53.250 INFO:tasks.workunit.client.1.smithi181.stdout:4/241: truncate d0/d15/f2f 426864 0 2024-04-27T04:09:53.252 INFO:tasks.workunit.client.1.smithi181.stdout:5/370: dwrite d4/d10/f59 [0,4194304] 0 2024-04-27T04:09:53.254 INFO:tasks.workunit.client.1.smithi181.stdout:6/167: link d7/dc/d1c/f1f d7/dc/f3a 0 2024-04-27T04:09:53.257 INFO:tasks.workunit.client.1.smithi181.stdout:3/265: truncate d0/de/f34 101045 0 2024-04-27T04:09:53.257 INFO:tasks.workunit.client.1.smithi181.stdout:1/291: dwrite d1/d2/f4 [0,4194304] 0 2024-04-27T04:09:53.257 INFO:tasks.workunit.client.1.smithi181.stdout:9/319: rename d0/d4d/f70 to d0/d44/d4b/d3d/f73 0 2024-04-27T04:09:53.262 INFO:tasks.workunit.client.1.smithi181.stdout:9/320: chown d0/d44/d4b/d43 1078816 1 2024-04-27T04:09:53.263 INFO:tasks.workunit.client.1.smithi181.stdout:4/242: fsync d0/d15/d25/d2b/d7/f28 0 2024-04-27T04:09:53.266 INFO:tasks.workunit.client.1.smithi181.stdout:5/371: mkdir d4/d10/d72/d80/d85 0 2024-04-27T04:09:53.268 INFO:tasks.workunit.client.1.smithi181.stdout:5/372: chown d4/c53 70 1 2024-04-27T04:09:53.270 INFO:tasks.workunit.client.1.smithi181.stdout:8/335: dread d1/d6/f44 [0,4194304] 0 2024-04-27T04:09:53.270 INFO:tasks.workunit.client.1.smithi181.stdout:1/292: dread d1/d2/d5/d6/f3d [0,4194304] 0 2024-04-27T04:09:53.271 INFO:tasks.workunit.client.1.smithi181.stdout:7/215: write d1/d22/d15/f28 [402450,28029] 0 2024-04-27T04:09:53.278 INFO:tasks.workunit.client.1.smithi181.stdout:3/266: creat d0/de/d20/f5c x:0 0 0 2024-04-27T04:09:53.280 INFO:tasks.workunit.client.1.smithi181.stdout:7/216: dwrite d1/d22/d15/d1f/d34/f3c [0,4194304] 0 2024-04-27T04:09:53.282 INFO:tasks.workunit.client.1.smithi181.stdout:6/168: link d7/dc/f12 d7/dc/f3b 0 2024-04-27T04:09:53.284 INFO:tasks.workunit.client.1.smithi181.stdout:5/373: rmdir d4 39 2024-04-27T04:09:53.286 INFO:tasks.workunit.client.1.smithi181.stdout:8/336: mknod d1/d6/d1a/d25/c76 0 2024-04-27T04:09:53.289 INFO:tasks.workunit.client.1.smithi181.stdout:3/267: dwrite d0/f37 [0,4194304] 0 2024-04-27T04:09:53.289 INFO:tasks.workunit.client.1.smithi181.stdout:8/337: readlink d1/d6/d2f/d30/l6d 0 2024-04-27T04:09:53.291 INFO:tasks.workunit.client.1.smithi181.stdout:9/321: symlink d0/d44/d4b/d50/l74 0 2024-04-27T04:09:53.293 INFO:tasks.workunit.client.1.smithi181.stdout:7/217: dwrite d1/d22/d15/d1f/f36 [0,4194304] 0 2024-04-27T04:09:53.293 INFO:tasks.workunit.client.1.smithi181.stdout:8/338: write d1/d6/d2f/d40/d2a/f37 [2834294,79510] 0 2024-04-27T04:09:53.298 INFO:tasks.workunit.client.1.smithi181.stdout:8/339: truncate d1/d6/d1a/f53 1725924 0 2024-04-27T04:09:53.311 INFO:tasks.workunit.client.1.smithi181.stdout:9/322: write d0/f23 [224851,46019] 0 2024-04-27T04:09:53.312 INFO:tasks.workunit.client.1.smithi181.stdout:9/323: write d0/d44/d4b/f62 [750926,96185] 0 2024-04-27T04:09:53.313 INFO:tasks.workunit.client.1.smithi181.stdout:3/268: creat d0/de/d20/f5d x:0 0 0 2024-04-27T04:09:53.314 INFO:tasks.workunit.client.1.smithi181.stdout:7/218: chown d1/d2/d1c/d2d/l38 951564 1 2024-04-27T04:09:53.318 INFO:tasks.workunit.client.1.smithi181.stdout:7/219: dread d1/d22/d15/f28 [0,4194304] 0 2024-04-27T04:09:53.320 INFO:tasks.workunit.client.1.smithi181.stdout:7/220: write d1/d22/d15/d1f/f36 [3937461,112115] 0 2024-04-27T04:09:53.322 INFO:tasks.workunit.client.1.smithi181.stdout:4/243: dwrite d0/d15/d25/d2b/f1c [0,4194304] 0 2024-04-27T04:09:53.323 INFO:tasks.workunit.client.1.smithi181.stdout:7/221: fdatasync d1/d22/d15/d1f/f36 0 2024-04-27T04:09:53.380 INFO:tasks.workunit.client.1.smithi181.stdout:8/340: symlink d1/d6/d2f/d30/l77 0 2024-04-27T04:09:53.382 INFO:tasks.workunit.client.1.smithi181.stdout:1/293: link d1/d2/d5/l50 d1/d4c/d1e/d20/d53/d56/l63 0 2024-04-27T04:09:53.385 INFO:tasks.workunit.client.1.smithi181.stdout:5/374: dread d4/de/d2f/f5d [4194304,4194304] 0 2024-04-27T04:09:53.389 INFO:tasks.workunit.client.1.smithi181.stdout:5/375: dwrite d4/d18/f41 [4194304,4194304] 0 2024-04-27T04:09:53.390 INFO:tasks.workunit.client.1.smithi181.stdout:5/376: readlink d4/de/d2f/l38 0 2024-04-27T04:09:53.391 INFO:tasks.workunit.client.1.smithi181.stdout:5/377: read - d4/d18/d67/d71/f7b zero size 2024-04-27T04:09:53.391 INFO:tasks.workunit.client.1.smithi181.stdout:5/378: stat d4/d18/d67/l35 0 2024-04-27T04:09:53.392 INFO:tasks.workunit.client.1.smithi181.stdout:5/379: fsync d4/d18/d54/f78 0 2024-04-27T04:09:53.393 INFO:tasks.workunit.client.1.smithi181.stdout:5/380: chown d4/de/d2c 31 1 2024-04-27T04:09:53.427 INFO:tasks.workunit.client.1.smithi181.stdout:3/269: unlink d0/f1e 0 2024-04-27T04:09:53.427 INFO:tasks.workunit.client.1.smithi181.stdout:3/270: stat d0/de/d20/f5d 0 2024-04-27T04:09:53.435 INFO:tasks.workunit.client.1.smithi181.stdout:6/169: write d7/fd [1014497,73724] 0 2024-04-27T04:09:53.436 INFO:tasks.workunit.client.1.smithi181.stdout:6/170: dread - d7/fb zero size 2024-04-27T04:09:53.439 INFO:tasks.workunit.client.1.smithi181.stdout:4/244: unlink d0/d15/d25/d2b/f41 0 2024-04-27T04:09:53.443 INFO:tasks.workunit.client.1.smithi181.stdout:8/341: chown d1/d6/d1a/d25/f52 2 1 2024-04-27T04:09:53.443 INFO:tasks.workunit.client.1.smithi181.stdout:8/342: readlink d1/d6/d2f/d30/d5c/l65 0 2024-04-27T04:09:53.447 INFO:tasks.workunit.client.1.smithi181.stdout:1/294: chown d1/d2/d5/d6/db/d2d/d36/c39 6 1 2024-04-27T04:09:53.448 INFO:tasks.workunit.client.1.smithi181.stdout:1/295: write d1/d2/d5/f4f [117068,1705] 0 2024-04-27T04:09:53.461 INFO:tasks.workunit.client.1.smithi181.stdout:4/245: rmdir d0/d15/de/d19 39 2024-04-27T04:09:53.462 INFO:tasks.workunit.client.1.smithi181.stdout:4/246: fsync d0/d15/f2 0 2024-04-27T04:09:53.464 INFO:tasks.workunit.client.1.smithi181.stdout:8/343: symlink d1/d6/d6f/l78 0 2024-04-27T04:09:53.469 INFO:tasks.workunit.client.1.smithi181.stdout:3/271: truncate d0/f19 1416301 0 2024-04-27T04:09:53.469 INFO:tasks.workunit.client.1.smithi181.stdout:3/272: write d0/de/f34 [640525,14543] 0 2024-04-27T04:09:53.470 INFO:tasks.workunit.client.1.smithi181.stdout:3/273: write d0/f57 [493225,108880] 0 2024-04-27T04:09:53.471 INFO:tasks.workunit.client.1.smithi181.stdout:6/171: getdents d7/dc/d1c/d34 0 2024-04-27T04:09:53.471 INFO:tasks.workunit.client.1.smithi181.stdout:6/172: read - d7/fb zero size 2024-04-27T04:09:53.472 INFO:tasks.workunit.client.1.smithi181.stdout:3/274: chown d0/f4 3 1 2024-04-27T04:09:53.474 INFO:tasks.workunit.client.1.smithi181.stdout:3/275: write d0/d3e/d52/d44/f45 [897094,66839] 0 2024-04-27T04:09:53.475 INFO:tasks.workunit.client.1.smithi181.stdout:8/344: symlink d1/d6/d1a/l79 0 2024-04-27T04:09:53.480 INFO:tasks.workunit.client.1.smithi181.stdout:6/173: dwrite f3 [0,4194304] 0 2024-04-27T04:09:53.480 INFO:tasks.workunit.client.1.smithi181.stdout:5/381: creat d4/de/d2c/d2e/f86 x:0 0 0 2024-04-27T04:09:53.482 INFO:tasks.workunit.client.1.smithi181.stdout:6/174: chown d7/dc/d1c/f32 2 1 2024-04-27T04:09:53.484 INFO:tasks.workunit.client.1.smithi181.stdout:8/345: dwrite d1/d6/d1a/d25/f5f [4194304,4194304] 0 2024-04-27T04:09:53.485 INFO:tasks.workunit.client.1.smithi181.stdout:0/310: sync 2024-04-27T04:09:53.485 INFO:tasks.workunit.client.1.smithi181.stdout:2/360: sync 2024-04-27T04:09:53.491 INFO:tasks.workunit.client.1.smithi181.stdout:3/276: truncate d0/d28/f42 872340 0 2024-04-27T04:09:53.493 INFO:tasks.workunit.client.1.smithi181.stdout:0/311: dwrite dc/d1e/d26/f6c [0,4194304] 0 2024-04-27T04:09:53.496 INFO:tasks.workunit.client.1.smithi181.stdout:5/382: rename d4/de/d2f/c63 to d4/de/d2c/d46/d4d/c87 0 2024-04-27T04:09:53.499 INFO:tasks.workunit.client.1.smithi181.stdout:8/346: creat d1/d20/f7a x:0 0 0 2024-04-27T04:09:53.499 INFO:tasks.workunit.client.1.smithi181.stdout:3/277: dwrite d0/de/d20/f5d [0,4194304] 0 2024-04-27T04:09:53.501 INFO:tasks.workunit.client.1.smithi181.stdout:8/347: chown d1/d20/f51 126574209 1 2024-04-27T04:09:53.505 INFO:tasks.workunit.client.1.smithi181.stdout:5/383: dwrite d4/de/d2c/d46/d4d/f66 [0,4194304] 0 2024-04-27T04:09:53.505 INFO:tasks.workunit.client.1.smithi181.stdout:0/312: symlink dc/d54/l76 0 2024-04-27T04:09:53.506 INFO:tasks.workunit.client.1.smithi181.stdout:1/296: truncate d1/d4c/d1e/d20/f41 84093 0 2024-04-27T04:09:53.509 INFO:tasks.workunit.client.1.smithi181.stdout:1/297: truncate d1/d2/d5/d6/f3d 4913687 0 2024-04-27T04:09:53.518 INFO:tasks.workunit.client.1.smithi181.stdout:4/247: truncate d0/d15/de/f34 516050 0 2024-04-27T04:09:53.518 INFO:tasks.workunit.client.1.smithi181.stdout:4/248: dread - d0/d15/f32 zero size 2024-04-27T04:09:53.522 INFO:tasks.workunit.client.1.smithi181.stdout:0/313: rmdir dc/d1f 39 2024-04-27T04:09:53.528 INFO:tasks.workunit.client.1.smithi181.stdout:6/175: link d7/dc/d1e/f2a d7/d2f/f3c 0 2024-04-27T04:09:53.528 INFO:tasks.workunit.client.1.smithi181.stdout:0/314: dwrite dc/f11 [0,4194304] 0 2024-04-27T04:09:53.538 INFO:tasks.workunit.client.1.smithi181.stdout:4/249: creat d0/d15/d25/d2a/d36/f4a x:0 0 0 2024-04-27T04:09:53.538 INFO:tasks.workunit.client.1.smithi181.stdout:8/348: dread d1/d6/d2f/d40/f17 [0,4194304] 0 2024-04-27T04:09:53.543 INFO:tasks.workunit.client.1.smithi181.stdout:5/384: mknod d4/de/d2f/d7a/c88 0 2024-04-27T04:09:53.546 INFO:tasks.workunit.client.1.smithi181.stdout:1/298: mkdir d1/d2/d64 0 2024-04-27T04:09:53.547 INFO:tasks.workunit.client.1.smithi181.stdout:1/299: truncate d1/d2/d5/f45 1475110 0 2024-04-27T04:09:53.548 INFO:tasks.workunit.client.1.smithi181.stdout:0/315: fsync dc/fd 0 2024-04-27T04:09:53.550 INFO:tasks.workunit.client.1.smithi181.stdout:1/300: dread d1/d2/d5/d6/f1a [0,4194304] 0 2024-04-27T04:09:53.551 INFO:tasks.workunit.client.1.smithi181.stdout:6/176: unlink d7/dc/d1e/c2d 0 2024-04-27T04:09:53.554 INFO:tasks.workunit.client.1.smithi181.stdout:8/349: creat d1/d6/f7b x:0 0 0 2024-04-27T04:09:53.555 INFO:tasks.workunit.client.1.smithi181.stdout:3/278: truncate d0/f4 6463569 0 2024-04-27T04:09:53.555 INFO:tasks.workunit.client.1.smithi181.stdout:3/279: chown d0/de/f1d 11267 1 2024-04-27T04:09:53.556 INFO:tasks.workunit.client.1.smithi181.stdout:5/385: dread - d4/de/f6f zero size 2024-04-27T04:09:53.559 INFO:tasks.workunit.client.1.smithi181.stdout:3/280: dread d0/de/f4b [0,4194304] 0 2024-04-27T04:09:53.564 INFO:tasks.workunit.client.1.smithi181.stdout:1/301: rmdir d1/d4c/d1e/d20/d53 39 2024-04-27T04:09:53.567 INFO:tasks.workunit.client.1.smithi181.stdout:6/177: symlink d7/d1b/l3d 0 2024-04-27T04:09:53.568 INFO:tasks.workunit.client.1.smithi181.stdout:4/250: symlink d0/d15/de/d39/l4b 0 2024-04-27T04:09:53.570 INFO:tasks.workunit.client.1.smithi181.stdout:8/350: rename d1/d20/f56 to d1/d6/d2f/d40/d2a/f7c 0 2024-04-27T04:09:53.573 INFO:tasks.workunit.client.1.smithi181.stdout:5/386: rmdir d4 39 2024-04-27T04:09:53.575 INFO:tasks.workunit.client.1.smithi181.stdout:8/351: dwrite d1/d6/f26 [4194304,4194304] 0 2024-04-27T04:09:53.583 INFO:tasks.workunit.client.1.smithi181.stdout:4/251: mkdir d0/d15/d25/d2a/d4c 0 2024-04-27T04:09:53.588 INFO:tasks.workunit.client.1.smithi181.stdout:5/387: truncate d4/d10/d29/f79 878274 0 2024-04-27T04:09:53.592 INFO:tasks.workunit.client.1.smithi181.stdout:3/281: link d0/de/d20/f5d d0/de/d20/f5e 0 2024-04-27T04:09:53.596 INFO:tasks.workunit.client.1.smithi181.stdout:8/352: unlink d1/c10 0 2024-04-27T04:09:53.596 INFO:tasks.workunit.client.1.smithi181.stdout:8/353: readlink d1/l62 0 2024-04-27T04:09:53.602 INFO:tasks.workunit.client.1.smithi181.stdout:8/354: dwrite d1/d6/d1a/f2b [0,4194304] 0 2024-04-27T04:09:53.607 INFO:tasks.workunit.client.1.smithi181.stdout:5/388: rename d4/de/d2c/d2e/f86 to d4/de/d2f/d7a/f89 0 2024-04-27T04:09:53.609 INFO:tasks.workunit.client.1.smithi181.stdout:3/282: creat d0/de/d3f/f5f x:0 0 0 2024-04-27T04:09:53.618 INFO:tasks.workunit.client.1.smithi181.stdout:8/355: rename d1/d6/d1a/l3b to d1/d6/d2f/d47/l7d 0 2024-04-27T04:09:53.619 INFO:tasks.workunit.client.1.smithi181.stdout:8/356: fsync d1/d6/d1a/f4d 0 2024-04-27T04:09:53.624 INFO:tasks.workunit.client.1.smithi181.stdout:7/222: sync 2024-04-27T04:09:53.624 INFO:tasks.workunit.client.1.smithi181.stdout:9/324: sync 2024-04-27T04:09:53.626 INFO:tasks.workunit.client.1.smithi181.stdout:0/316: creat dc/d1f/d3b/f77 x:0 0 0 2024-04-27T04:09:53.627 INFO:tasks.workunit.client.1.smithi181.stdout:4/252: link d0/d15/d25/d2b/l20 d0/d15/d25/l4d 0 2024-04-27T04:09:53.632 INFO:tasks.workunit.client.1.smithi181.stdout:6/178: write d7/d2f/f3c [537638,63730] 0 2024-04-27T04:09:53.636 INFO:tasks.workunit.client.1.smithi181.stdout:3/283: fsync d0/de/d20/f46 0 2024-04-27T04:09:53.639 INFO:tasks.workunit.client.1.smithi181.stdout:9/325: fsync d0/f26 0 2024-04-27T04:09:53.641 INFO:tasks.workunit.client.1.smithi181.stdout:7/223: rmdir d1/df 39 2024-04-27T04:09:53.647 INFO:tasks.workunit.client.1.smithi181.stdout:6/179: mkdir d7/d1b/d3e 0 2024-04-27T04:09:53.650 INFO:tasks.workunit.client.1.smithi181.stdout:5/389: creat d4/d18/f8a x:0 0 0 2024-04-27T04:09:53.652 INFO:tasks.workunit.client.1.smithi181.stdout:5/390: write d4/d18/d54/f58 [3542795,70000] 0 2024-04-27T04:09:53.655 INFO:tasks.workunit.client.1.smithi181.stdout:1/302: dwrite d1/d4c/d1e/d20/f41 [0,4194304] 0 2024-04-27T04:09:53.656 INFO:tasks.workunit.client.1.smithi181.stdout:9/326: dread d0/d44/d4f/f58 [0,4194304] 0 2024-04-27T04:09:53.658 INFO:tasks.workunit.client.1.smithi181.stdout:8/357: write d1/d6/d1a/d25/f52 [545401,37569] 0 2024-04-27T04:09:53.658 INFO:tasks.workunit.client.1.smithi181.stdout:9/327: read d0/d44/d4b/d3d/f3e [107412,41448] 0 2024-04-27T04:09:53.659 INFO:tasks.workunit.client.1.smithi181.stdout:9/328: dread - d0/d44/d4b/d50/d6c/f63 zero size 2024-04-27T04:09:53.668 INFO:tasks.workunit.client.1.smithi181.stdout:6/180: dread f6 [0,4194304] 0 2024-04-27T04:09:53.671 INFO:tasks.workunit.client.1.smithi181.stdout:5/391: fsync d4/d10/d29/f3c 0 2024-04-27T04:09:53.679 INFO:tasks.workunit.client.1.smithi181.stdout:8/358: symlink d1/d6/d2f/d30/d5c/l7e 0 2024-04-27T04:09:53.679 INFO:tasks.workunit.client.1.smithi181.stdout:4/253: truncate d0/f30 401777 0 2024-04-27T04:09:53.680 INFO:tasks.workunit.client.1.smithi181.stdout:4/254: fsync d0/d15/d25/d2a/f38 0 2024-04-27T04:09:53.681 INFO:tasks.workunit.client.1.smithi181.stdout:7/224: creat d1/d22/d15/d29/f43 x:0 0 0 2024-04-27T04:09:53.683 INFO:tasks.workunit.client.1.smithi181.stdout:9/329: mknod d0/d44/d4f/c75 0 2024-04-27T04:09:53.687 INFO:tasks.workunit.client.1.smithi181.stdout:5/392: creat d4/de/d2c/d46/d4d/f8b x:0 0 0 2024-04-27T04:09:53.694 INFO:tasks.workunit.client.1.smithi181.stdout:4/255: chown d0/d15/d25/d2b/l20 30967 1 2024-04-27T04:09:53.696 INFO:tasks.workunit.client.1.smithi181.stdout:7/225: mknod d1/d22/c44 0 2024-04-27T04:09:53.696 INFO:tasks.workunit.client.1.smithi181.stdout:9/330: dread d0/f15 [4194304,4194304] 0 2024-04-27T04:09:53.699 INFO:tasks.workunit.client.1.smithi181.stdout:3/284: write d0/f4 [3869384,55408] 0 2024-04-27T04:09:53.700 INFO:tasks.workunit.client.1.smithi181.stdout:5/393: mknod d4/d18/d67/d71/c8c 0 2024-04-27T04:09:53.702 INFO:tasks.workunit.client.1.smithi181.stdout:5/394: write d4/d10/d29/f79 [610067,62984] 0 2024-04-27T04:09:53.702 INFO:tasks.workunit.client.1.smithi181.stdout:5/395: stat d4/d18/d67 0 2024-04-27T04:09:53.704 INFO:tasks.workunit.client.1.smithi181.stdout:1/303: rmdir d1/d2/d5/d6/db/d2d/d36/d5f 0 2024-04-27T04:09:53.707 INFO:tasks.workunit.client.1.smithi181.stdout:3/285: dread d0/f14 [0,4194304] 0 2024-04-27T04:09:53.714 INFO:tasks.workunit.client.1.smithi181.stdout:5/396: creat d4/d10/d72/d80/f8d x:0 0 0 2024-04-27T04:09:53.714 INFO:tasks.workunit.client.1.smithi181.stdout:9/331: dwrite d0/f1f [0,4194304] 0 2024-04-27T04:09:53.716 INFO:tasks.workunit.client.1.smithi181.stdout:1/304: unlink d1/d2/d5/c47 0 2024-04-27T04:09:53.717 INFO:tasks.workunit.client.1.smithi181.stdout:8/359: link d1/cb d1/d6/d1a/c7f 0 2024-04-27T04:09:53.719 INFO:tasks.workunit.client.1.smithi181.stdout:8/360: readlink d1/d6/d1a/l79 0 2024-04-27T04:09:53.720 INFO:tasks.workunit.client.1.smithi181.stdout:8/361: chown d1/d6/d2f/d30/d5c/l7e 1 1 2024-04-27T04:09:53.722 INFO:tasks.workunit.client.1.smithi181.stdout:3/286: rename d0/de/f1d to d0/d3e/f60 0 2024-04-27T04:09:53.722 INFO:tasks.workunit.client.1.smithi181.stdout:8/362: write d1/f6b [1221433,18449] 0 2024-04-27T04:09:53.723 INFO:tasks.workunit.client.1.smithi181.stdout:3/287: write d0/f15 [4587710,40447] 0 2024-04-27T04:09:53.725 INFO:tasks.workunit.client.1.smithi181.stdout:5/397: rmdir d4/de/d2f/d7a 39 2024-04-27T04:09:53.728 INFO:tasks.workunit.client.1.smithi181.stdout:1/305: rmdir d1/d4c/d1e/d48 39 2024-04-27T04:09:53.728 INFO:tasks.workunit.client.1.smithi181.stdout:1/306: stat d1/d2/l10 0 2024-04-27T04:09:53.729 INFO:tasks.workunit.client.1.smithi181.stdout:4/256: link d0/d15/d25/d2b/f1f d0/d15/d25/d2a/d4c/f4e 0 2024-04-27T04:09:53.730 INFO:tasks.workunit.client.1.smithi181.stdout:1/307: chown d1/d2/d5/d6/f4e 31379169 1 2024-04-27T04:09:53.730 INFO:tasks.workunit.client.1.smithi181.stdout:4/257: write d0/d15/de/f43 [105370,219] 0 2024-04-27T04:09:53.730 INFO:tasks.workunit.client.1.smithi181.stdout:1/308: chown d1/d2/f4 315 1 2024-04-27T04:09:53.731 INFO:tasks.workunit.client.1.smithi181.stdout:4/258: chown d0 0 1 2024-04-27T04:09:53.733 INFO:tasks.workunit.client.1.smithi181.stdout:8/363: mknod d1/d6/d6f/c80 0 2024-04-27T04:09:53.734 INFO:tasks.workunit.client.1.smithi181.stdout:8/364: read d1/d6/f26 [5593224,42498] 0 2024-04-27T04:09:53.749 INFO:tasks.workunit.client.1.smithi181.stdout:8/365: creat d1/d6/d2f/d30/d66/f81 x:0 0 0 2024-04-27T04:09:53.752 INFO:tasks.workunit.client.1.smithi181.stdout:7/226: getdents d1/d22/d15/d1f 0 2024-04-27T04:09:53.756 INFO:tasks.workunit.client.1.smithi181.stdout:7/227: dread d1/f8 [0,4194304] 0 2024-04-27T04:09:53.758 INFO:tasks.workunit.client.1.smithi181.stdout:4/259: rename d0/d15/d25/d2b/f1f to d0/d15/d25/d40/f4f 0 2024-04-27T04:09:53.763 INFO:tasks.workunit.client.1.smithi181.stdout:8/366: truncate d1/d20/f24 9480308 0 2024-04-27T04:09:53.763 INFO:tasks.workunit.client.1.smithi181.stdout:2/361: sync 2024-04-27T04:09:53.763 INFO:tasks.workunit.client.1.smithi181.stdout:8/367: stat d1/d6/d1a/l21 0 2024-04-27T04:09:53.765 INFO:tasks.workunit.client.1.smithi181.stdout:1/309: link d1/d4c/d1e/l44 d1/d4c/d60/l65 0 2024-04-27T04:09:53.766 INFO:tasks.workunit.client.1.smithi181.stdout:1/310: fsync d1/d4c/d1e/f49 0 2024-04-27T04:09:53.768 INFO:tasks.workunit.client.1.smithi181.stdout:1/311: dread d1/d2/d5/d6/f1a [0,4194304] 0 2024-04-27T04:09:53.770 INFO:tasks.workunit.client.1.smithi181.stdout:1/312: dread d1/d4c/d1e/f49 [0,4194304] 0 2024-04-27T04:09:53.772 INFO:tasks.workunit.client.1.smithi181.stdout:2/362: rmdir d4/d9 39 2024-04-27T04:09:53.775 INFO:tasks.workunit.client.1.smithi181.stdout:8/368: creat d1/f82 x:0 0 0 2024-04-27T04:09:53.777 INFO:tasks.workunit.client.1.smithi181.stdout:9/332: truncate d0/f2a 1056247 0 2024-04-27T04:09:53.781 INFO:tasks.workunit.client.1.smithi181.stdout:8/369: dwrite d1/d20/f51 [0,4194304] 0 2024-04-27T04:09:53.786 INFO:tasks.workunit.client.1.smithi181.stdout:3/288: write d0/f3 [655613,24909] 0 2024-04-27T04:09:53.787 INFO:tasks.workunit.client.1.smithi181.stdout:3/289: chown d0/de/d3f/c53 0 1 2024-04-27T04:09:53.788 INFO:tasks.workunit.client.1.smithi181.stdout:7/228: creat d1/f45 x:0 0 0 2024-04-27T04:09:53.796 INFO:tasks.workunit.client.1.smithi181.stdout:4/260: rename d0/d15/d25/d2b/d7/f28 to d0/d15/f50 0 2024-04-27T04:09:53.804 INFO:tasks.workunit.client.1.smithi181.stdout:7/229: creat d1/d22/d15/d13/f46 x:0 0 0 2024-04-27T04:09:53.810 INFO:tasks.workunit.client.1.smithi181.stdout:4/261: creat d0/d15/d25/d2a/d36/f51 x:0 0 0 2024-04-27T04:09:53.811 INFO:tasks.workunit.client.1.smithi181.stdout:7/230: dwrite d1/d2/d1c/d37/f3f [0,4194304] 0 2024-04-27T04:09:53.815 INFO:tasks.workunit.client.1.smithi181.stdout:9/333: write d0/d44/d4f/f58 [1216068,33838] 0 2024-04-27T04:09:53.823 INFO:tasks.workunit.client.1.smithi181.stdout:3/290: write d0/de/f4b [782607,86655] 0 2024-04-27T04:09:53.824 INFO:tasks.workunit.client.1.smithi181.stdout:7/231: creat d1/d22/d15/d13/d1b/f47 x:0 0 0 2024-04-27T04:09:53.824 INFO:tasks.workunit.client.1.smithi181.stdout:3/291: chown d0/d28/f42 1627762639 1 2024-04-27T04:09:53.825 INFO:tasks.workunit.client.1.smithi181.stdout:3/292: write d0/f5 [401318,87629] 0 2024-04-27T04:09:53.826 INFO:tasks.workunit.client.1.smithi181.stdout:8/370: link d1/d6/d1a/l39 d1/d6/d2f/d72/l83 0 2024-04-27T04:09:53.829 INFO:tasks.workunit.client.1.smithi181.stdout:3/293: dread d0/de/d20/f5d [0,4194304] 0 2024-04-27T04:09:53.830 INFO:tasks.workunit.client.1.smithi181.stdout:9/334: mknod d0/d44/d4b/d43/c76 0 2024-04-27T04:09:53.836 INFO:tasks.workunit.client.1.smithi181.stdout:8/371: chown d1/d6/d1a/l39 10 1 2024-04-27T04:09:53.836 INFO:tasks.workunit.client.1.smithi181.stdout:1/313: link d1/d4c/d1e/d20/d53/d56/l63 d1/d4c/d52/l66 0 2024-04-27T04:09:53.836 INFO:tasks.workunit.client.1.smithi181.stdout:1/314: fdatasync f0 0 2024-04-27T04:09:53.841 INFO:tasks.workunit.client.1.smithi181.stdout:2/363: getdents d4/d9/df/d14 0 2024-04-27T04:09:53.842 INFO:tasks.workunit.client.1.smithi181.stdout:9/335: dread d0/d40/f22 [0,4194304] 0 2024-04-27T04:09:53.843 INFO:tasks.workunit.client.1.smithi181.stdout:4/262: rename d0/d15/d48/l49 to d0/l52 0 2024-04-27T04:09:53.847 INFO:tasks.workunit.client.1.smithi181.stdout:1/315: write d1/d2/d5/d6/f3d [5831449,18089] 0 2024-04-27T04:09:53.849 INFO:tasks.workunit.client.1.smithi181.stdout:3/294: write d0/de/d20/f5e [3261805,79688] 0 2024-04-27T04:09:53.851 INFO:tasks.workunit.client.1.smithi181.stdout:1/316: truncate d1/d2/d5/d6/db/d2d/f55 269562 0 2024-04-27T04:09:53.852 INFO:tasks.workunit.client.1.smithi181.stdout:9/336: rmdir d0 39 2024-04-27T04:09:53.854 INFO:tasks.workunit.client.1.smithi181.stdout:8/372: rename d1/d6/d2f/d30/l77 to d1/d6/d2f/d30/d5c/l84 0 2024-04-27T04:09:53.855 INFO:tasks.workunit.client.1.smithi181.stdout:8/373: stat d1/l62 0 2024-04-27T04:09:53.856 INFO:tasks.workunit.client.1.smithi181.stdout:2/364: creat d4/d9/d5d/f71 x:0 0 0 2024-04-27T04:09:53.856 INFO:tasks.workunit.client.1.smithi181.stdout:1/317: dread d1/d2/d5/f45 [0,4194304] 0 2024-04-27T04:09:53.857 INFO:tasks.workunit.client.1.smithi181.stdout:2/365: read d4/f2c [3763965,57459] 0 2024-04-27T04:09:53.859 INFO:tasks.workunit.client.1.smithi181.stdout:7/232: truncate d1/d22/f26 438787 0 2024-04-27T04:09:53.862 INFO:tasks.workunit.client.1.smithi181.stdout:3/295: getdents d0/d3e/d52/d44/d56/d5b 0 2024-04-27T04:09:53.862 INFO:tasks.workunit.client.1.smithi181.stdout:7/233: chown d1/d2/f25 102803 1 2024-04-27T04:09:53.862 INFO:tasks.workunit.client.1.smithi181.stdout:4/263: dread d0/d15/f1a [0,4194304] 0 2024-04-27T04:09:53.864 INFO:tasks.workunit.client.1.smithi181.stdout:7/234: fdatasync d1/d2/d1c/f1e 0 2024-04-27T04:09:53.866 INFO:tasks.workunit.client.1.smithi181.stdout:8/374: dwrite d1/f46 [0,4194304] 0 2024-04-27T04:09:53.866 INFO:tasks.workunit.client.1.smithi181.stdout:7/235: dread - d1/d2/d1c/f2c zero size 2024-04-27T04:09:53.866 INFO:tasks.workunit.client.1.smithi181.stdout:1/318: rename d1/d2/d5/d6/db/d2d/d36/f51 to d1/d2/d5/d6/f67 0 2024-04-27T04:09:53.886 INFO:tasks.workunit.client.1.smithi181.stdout:4/264: symlink d0/d15/de/d39/l53 0 2024-04-27T04:09:53.886 INFO:tasks.workunit.client.1.smithi181.stdout:1/319: write d1/d4c/d1e/d20/f41 [2493287,12470] 0 2024-04-27T04:09:53.889 INFO:tasks.workunit.client.1.smithi181.stdout:4/265: dread - d0/f3d zero size 2024-04-27T04:09:53.889 INFO:tasks.workunit.client.1.smithi181.stdout:1/320: fdatasync d1/d4c/d1e/f4b 0 2024-04-27T04:09:53.889 INFO:tasks.workunit.client.1.smithi181.stdout:9/337: dwrite d0/f2e [0,4194304] 0 2024-04-27T04:09:53.889 INFO:tasks.workunit.client.1.smithi181.stdout:8/375: rename d1/d6/d2f/d30/d5c/l65 to d1/d6/d2f/d72/l85 0 2024-04-27T04:09:53.890 INFO:tasks.workunit.client.1.smithi181.stdout:1/321: chown f0 0 1 2024-04-27T04:09:53.891 INFO:tasks.workunit.client.1.smithi181.stdout:2/366: getdents d4/d9/d16 0 2024-04-27T04:09:53.891 INFO:tasks.workunit.client.1.smithi181.stdout:8/376: stat d1/d6/d2f/d30 0 2024-04-27T04:09:53.895 INFO:tasks.workunit.client.1.smithi181.stdout:9/338: creat d0/d44/d4b/d5f/f77 x:0 0 0 2024-04-27T04:09:53.897 INFO:tasks.workunit.client.1.smithi181.stdout:1/322: truncate d1/d2/d5/f45 1598928 0 2024-04-27T04:09:53.902 INFO:tasks.workunit.client.1.smithi181.stdout:7/236: dwrite d1/f8 [4194304,4194304] 0 2024-04-27T04:09:53.902 INFO:tasks.workunit.client.1.smithi181.stdout:4/266: link d0/d15/de/d39/l4b d0/d15/de/d19/l54 0 2024-04-27T04:09:53.905 INFO:tasks.workunit.client.1.smithi181.stdout:4/267: rename d0 to d0/d15/d25/d40/d55 22 2024-04-27T04:09:53.905 INFO:tasks.workunit.client.1.smithi181.stdout:8/377: dwrite d1/f33 [0,4194304] 0 2024-04-27T04:09:53.906 INFO:tasks.workunit.client.1.smithi181.stdout:9/339: dread d0/d44/d4f/f58 [0,4194304] 0 2024-04-27T04:09:53.906 INFO:tasks.workunit.client.1.smithi181.stdout:7/237: symlink d1/d2/d1c/l48 0 2024-04-27T04:09:53.908 INFO:tasks.workunit.client.1.smithi181.stdout:4/268: rmdir d0/d15/d25/d40 39 2024-04-27T04:09:53.908 INFO:tasks.workunit.client.1.smithi181.stdout:9/340: rmdir d0/d44/d4b/d43 39 2024-04-27T04:09:53.911 INFO:tasks.workunit.client.1.smithi181.stdout:7/238: mknod d1/d2/d1c/c49 0 2024-04-27T04:09:53.914 INFO:tasks.workunit.client.1.smithi181.stdout:9/341: chown d0/c24 134635115 1 2024-04-27T04:09:53.915 INFO:tasks.workunit.client.1.smithi181.stdout:4/269: symlink d0/d15/d25/d40/l56 0 2024-04-27T04:09:53.920 INFO:tasks.workunit.client.1.smithi181.stdout:7/239: truncate d1/d22/d15/d1f/f36 627808 0 2024-04-27T04:09:53.922 INFO:tasks.workunit.client.1.smithi181.stdout:9/342: dread d0/d40/f29 [0,4194304] 0 2024-04-27T04:09:53.922 INFO:tasks.workunit.client.1.smithi181.stdout:4/270: rename d0/d15/de/d39 to d0/d15/d48/d57 0 2024-04-27T04:09:53.923 INFO:tasks.workunit.client.1.smithi181.stdout:7/240: creat d1/d22/d15/d29/f4a x:0 0 0 2024-04-27T04:09:53.924 INFO:tasks.workunit.client.1.smithi181.stdout:1/323: dread f0 [0,4194304] 0 2024-04-27T04:09:53.925 INFO:tasks.workunit.client.1.smithi181.stdout:7/241: dread - d1/d22/d15/d13/d1b/f40 zero size 2024-04-27T04:09:53.926 INFO:tasks.workunit.client.1.smithi181.stdout:7/242: truncate d1/d2/d1c/f2c 758539 0 2024-04-27T04:09:53.926 INFO:tasks.workunit.client.1.smithi181.stdout:7/243: chown d1/d2/f12 1498 1 2024-04-27T04:09:53.927 INFO:tasks.workunit.client.1.smithi181.stdout:8/378: fsync d1/f33 0 2024-04-27T04:09:53.929 INFO:tasks.workunit.client.1.smithi181.stdout:1/324: creat d1/d4c/d52/f68 x:0 0 0 2024-04-27T04:09:53.929 INFO:tasks.workunit.client.1.smithi181.stdout:8/379: creat d1/d6/d2f/d47/f86 x:0 0 0 2024-04-27T04:09:53.932 INFO:tasks.workunit.client.1.smithi181.stdout:1/325: unlink d1/d4c/l28 0 2024-04-27T04:09:53.933 INFO:tasks.workunit.client.1.smithi181.stdout:8/380: symlink d1/d20/l87 0 2024-04-27T04:09:53.934 INFO:tasks.workunit.client.1.smithi181.stdout:7/244: dwrite d1/d22/d15/d13/f46 [0,4194304] 0 2024-04-27T04:09:53.936 INFO:tasks.workunit.client.1.smithi181.stdout:1/326: symlink d1/d4c/d52/l69 0 2024-04-27T04:09:53.938 INFO:tasks.workunit.client.1.smithi181.stdout:8/381: mkdir d1/d6/d88 0 2024-04-27T04:09:53.940 INFO:tasks.workunit.client.1.smithi181.stdout:8/382: chown d1/d6/d2f/d40/d2a/f37 1033762 1 2024-04-27T04:09:53.940 INFO:tasks.workunit.client.1.smithi181.stdout:8/383: chown d1/f46 896 1 2024-04-27T04:09:53.942 INFO:tasks.workunit.client.1.smithi181.stdout:4/271: truncate d0/d15/d25/d2b/f6 1984742 0 2024-04-27T04:09:53.946 INFO:tasks.workunit.client.1.smithi181.stdout:1/327: truncate f0 1221491 0 2024-04-27T04:09:53.949 INFO:tasks.workunit.client.1.smithi181.stdout:4/272: rmdir d0/d15/de 39 2024-04-27T04:09:53.955 INFO:tasks.workunit.client.1.smithi181.stdout:8/384: unlink d1/cb 0 2024-04-27T04:09:53.957 INFO:tasks.workunit.client.1.smithi181.stdout:7/245: getdents d1/d2 0 2024-04-27T04:09:53.962 INFO:tasks.workunit.client.1.smithi181.stdout:8/385: dwrite d1/d6/d1a/f53 [0,4194304] 0 2024-04-27T04:09:53.966 INFO:tasks.workunit.client.1.smithi181.stdout:1/328: getdents d1/d4c/d1e/d20 0 2024-04-27T04:09:53.967 INFO:tasks.workunit.client.1.smithi181.stdout:4/273: getdents d0/d15/d25/d40 0 2024-04-27T04:09:53.969 INFO:tasks.workunit.client.1.smithi181.stdout:7/246: write d1/d22/d15/f2a [1314196,48390] 0 2024-04-27T04:09:53.970 INFO:tasks.workunit.client.1.smithi181.stdout:1/329: truncate d1/d2/d5/f4f 566613 0 2024-04-27T04:09:53.973 INFO:tasks.workunit.client.1.smithi181.stdout:8/386: truncate d1/d6/d2f/d40/f6e 1795277 0 2024-04-27T04:09:53.973 INFO:tasks.workunit.client.1.smithi181.stdout:7/247: symlink d1/d22/d15/d13/l4b 0 2024-04-27T04:09:53.974 INFO:tasks.workunit.client.1.smithi181.stdout:0/317: sync 2024-04-27T04:09:53.974 INFO:tasks.workunit.client.1.smithi181.stdout:1/330: rmdir d1/d4c/d1e/d20/d53 39 2024-04-27T04:09:53.975 INFO:tasks.workunit.client.1.smithi181.stdout:4/274: mknod d0/d15/d25/d2a/d4c/c58 0 2024-04-27T04:09:53.975 INFO:tasks.workunit.client.1.smithi181.stdout:0/318: write dc/f2c [671226,53014] 0 2024-04-27T04:09:53.980 INFO:tasks.workunit.client.1.smithi181.stdout:1/331: dread - d1/d4c/d1e/d20/f5c zero size 2024-04-27T04:09:53.983 INFO:tasks.workunit.client.1.smithi181.stdout:7/248: dwrite d1/f45 [0,4194304] 0 2024-04-27T04:09:53.987 INFO:tasks.workunit.client.1.smithi181.stdout:1/332: mkdir d1/d2/d5/d6/d6a 0 2024-04-27T04:09:53.990 INFO:tasks.workunit.client.1.smithi181.stdout:4/275: dwrite d0/d15/d25/d2a/d36/f42 [0,4194304] 0 2024-04-27T04:09:53.990 INFO:tasks.workunit.client.1.smithi181.stdout:7/249: unlink d1/l14 0 2024-04-27T04:09:53.994 INFO:tasks.workunit.client.1.smithi181.stdout:7/250: write d1/d22/d15/d29/f4a [414572,4683] 0 2024-04-27T04:09:53.997 INFO:tasks.workunit.client.1.smithi181.stdout:4/276: readlink d0/d15/d48/d57/l4b 0 2024-04-27T04:09:53.997 INFO:tasks.workunit.client.1.smithi181.stdout:1/333: symlink d1/d2/d5/l6b 0 2024-04-27T04:09:54.000 INFO:tasks.workunit.client.1.smithi181.stdout:0/319: dwrite f8 [0,4194304] 0 2024-04-27T04:09:54.000 INFO:tasks.workunit.client.1.smithi181.stdout:1/334: write d1/f2f [3154305,103736] 0 2024-04-27T04:09:54.000 INFO:tasks.workunit.client.1.smithi181.stdout:8/387: dread d1/d6/f55 [0,4194304] 0 2024-04-27T04:09:54.004 INFO:tasks.workunit.client.1.smithi181.stdout:0/320: readlink dc/d1e/d26/l31 0 2024-04-27T04:09:54.005 INFO:tasks.workunit.client.1.smithi181.stdout:8/388: mkdir d1/d6/d2f/d47/d89 0 2024-04-27T04:09:54.009 INFO:tasks.workunit.client.1.smithi181.stdout:4/277: mkdir d0/d15/de/d19/d59 0 2024-04-27T04:09:54.011 INFO:tasks.workunit.client.1.smithi181.stdout:1/335: truncate d1/d2/d5/d6/f22 92001 0 2024-04-27T04:09:54.012 INFO:tasks.workunit.client.1.smithi181.stdout:0/321: dwrite dc/f46 [0,4194304] 0 2024-04-27T04:09:54.013 INFO:tasks.workunit.client.1.smithi181.stdout:4/278: write d0/d15/d25/d2a/d36/f51 [951951,41288] 0 2024-04-27T04:09:54.020 INFO:tasks.workunit.client.1.smithi181.stdout:0/322: symlink dc/d1e/d26/d68/l78 0 2024-04-27T04:09:54.022 INFO:tasks.workunit.client.1.smithi181.stdout:4/279: mknod d0/d15/d25/d2a/d36/c5a 0 2024-04-27T04:09:54.028 INFO:tasks.workunit.client.1.smithi181.stdout:4/280: dread d0/d15/d25/d2a/d36/f42 [0,4194304] 0 2024-04-27T04:09:54.033 INFO:tasks.workunit.client.1.smithi181.stdout:0/323: creat dc/d1f/d3b/d75/f79 x:0 0 0 2024-04-27T04:09:54.035 INFO:tasks.workunit.client.1.smithi181.stdout:1/336: dwrite d1/d2/d5/f45 [0,4194304] 0 2024-04-27T04:09:54.042 INFO:tasks.workunit.client.1.smithi181.stdout:1/337: rename d1/l35 to d1/d2/l6c 0 2024-04-27T04:09:54.043 INFO:tasks.workunit.client.1.smithi181.stdout:8/389: dread d1/d6/d2f/d30/d5c/f61 [0,4194304] 0 2024-04-27T04:09:54.047 INFO:tasks.workunit.client.1.smithi181.stdout:8/390: truncate d1/d6/d2f/d40/f5e 63479 0 2024-04-27T04:09:54.047 INFO:tasks.workunit.client.1.smithi181.stdout:8/391: stat d1/d6/d2f/d40/l2e 0 2024-04-27T04:09:54.053 INFO:tasks.workunit.client.1.smithi181.stdout:8/392: rename d1/d6/d2f/d47/l7d to d1/d6/d2f/d72/l8a 0 2024-04-27T04:09:54.053 INFO:tasks.workunit.client.1.smithi181.stdout:4/281: dread d0/d15/d25/d2b/d7/f10 [0,4194304] 0 2024-04-27T04:09:54.053 INFO:tasks.workunit.client.1.smithi181.stdout:8/393: fdatasync d1/d6/f4a 0 2024-04-27T04:09:54.056 INFO:tasks.workunit.client.1.smithi181.stdout:4/282: creat d0/d15/d25/d2b/f5b x:0 0 0 2024-04-27T04:09:54.057 INFO:tasks.workunit.client.1.smithi181.stdout:0/324: dread dc/d1f/d3b/f5f [0,4194304] 0 2024-04-27T04:09:54.058 INFO:tasks.workunit.client.1.smithi181.stdout:8/394: mknod d1/d6/d1a/c8b 0 2024-04-27T04:09:54.059 INFO:tasks.workunit.client.1.smithi181.stdout:4/283: write d0/d15/d25/d2b/d7/f10 [605225,46602] 0 2024-04-27T04:09:54.063 INFO:tasks.workunit.client.1.smithi181.stdout:1/338: dwrite f0 [0,4194304] 0 2024-04-27T04:09:54.064 INFO:tasks.workunit.client.1.smithi181.stdout:4/284: mkdir d0/d15/d5c 0 2024-04-27T04:09:54.064 INFO:tasks.workunit.client.1.smithi181.stdout:1/339: dread - d1/d4c/d1e/f4b zero size 2024-04-27T04:09:54.067 INFO:tasks.workunit.client.1.smithi181.stdout:4/285: write d0/d15/d25/d2a/d36/f4a [558303,4429] 0 2024-04-27T04:09:54.068 INFO:tasks.workunit.client.1.smithi181.stdout:1/340: dread - d1/d4c/d1e/d20/f61 zero size 2024-04-27T04:09:54.072 INFO:tasks.workunit.client.1.smithi181.stdout:1/341: dread - d1/d4c/d1e/d20/f5c zero size 2024-04-27T04:09:54.073 INFO:tasks.workunit.client.1.smithi181.stdout:4/286: rename d0/d15/d25/d2b/d7/cd to d0/d15/d25/d2a/c5d 0 2024-04-27T04:09:54.076 INFO:tasks.workunit.client.1.smithi181.stdout:8/395: dwrite d1/d6/d1a/f53 [0,4194304] 0 2024-04-27T04:09:54.076 INFO:tasks.workunit.client.1.smithi181.stdout:1/342: fdatasync d1/d2/d5/d6/f22 0 2024-04-27T04:09:54.078 INFO:tasks.workunit.client.1.smithi181.stdout:8/396: fdatasync d1/f46 0 2024-04-27T04:09:54.080 INFO:tasks.workunit.client.1.smithi181.stdout:4/287: dread d0/f3b [0,4194304] 0 2024-04-27T04:09:54.080 INFO:tasks.workunit.client.1.smithi181.stdout:4/288: stat d0/d15/f2 0 2024-04-27T04:09:54.081 INFO:tasks.workunit.client.1.smithi181.stdout:8/397: mkdir d1/d8c 0 2024-04-27T04:09:54.082 INFO:tasks.workunit.client.1.smithi181.stdout:4/289: write d0/d15/f23 [787270,73930] 0 2024-04-27T04:09:54.085 INFO:tasks.workunit.client.1.smithi181.stdout:8/398: rmdir d1/d20 39 2024-04-27T04:09:54.088 INFO:tasks.workunit.client.1.smithi181.stdout:8/399: write d1/d6/f7b [411957,35108] 0 2024-04-27T04:09:54.088 INFO:tasks.workunit.client.1.smithi181.stdout:1/343: truncate d1/d2/d5/d6/db/f19 2627904 0 2024-04-27T04:09:54.090 INFO:tasks.workunit.client.1.smithi181.stdout:1/344: readlink d1/l18 0 2024-04-27T04:09:54.091 INFO:tasks.workunit.client.1.smithi181.stdout:4/290: dwrite d0/d15/d25/d2b/f1c [4194304,4194304] 0 2024-04-27T04:09:54.092 INFO:tasks.workunit.client.1.smithi181.stdout:1/345: rename d1/d2 to d1/d2/d5/d6/db/d2d/d6d 22 2024-04-27T04:09:54.094 INFO:tasks.workunit.client.1.smithi181.stdout:8/400: mknod d1/d6/d6f/c8d 0 2024-04-27T04:09:54.094 INFO:tasks.workunit.client.1.smithi181.stdout:4/291: write d0/d15/f23 [1057306,88079] 0 2024-04-27T04:09:54.096 INFO:tasks.workunit.client.1.smithi181.stdout:1/346: dread d1/d4c/d1e/f49 [0,4194304] 0 2024-04-27T04:09:54.097 INFO:tasks.workunit.client.1.smithi181.stdout:4/292: write d0/d15/f2f [722394,59350] 0 2024-04-27T04:09:54.101 INFO:tasks.workunit.client.1.smithi181.stdout:8/401: chown d1/d6/d1a/c7f 444732270 1 2024-04-27T04:09:54.101 INFO:tasks.workunit.client.1.smithi181.stdout:8/402: write d1/d6/f4e [233651,55366] 0 2024-04-27T04:09:54.102 INFO:tasks.workunit.client.1.smithi181.stdout:1/347: rename d1/d4c/d1e/c30 to d1/d2/d5/d2c/c6e 0 2024-04-27T04:09:54.104 INFO:tasks.workunit.client.1.smithi181.stdout:1/348: chown d1/d2/l6c 6013923 1 2024-04-27T04:09:54.105 INFO:tasks.workunit.client.1.smithi181.stdout:8/403: dread d1/d6/d2f/d30/d5c/f61 [0,4194304] 0 2024-04-27T04:09:54.107 INFO:tasks.workunit.client.1.smithi181.stdout:8/404: truncate d1/d6/d2f/d40/d2a/f7c 545406 0 2024-04-27T04:09:54.109 INFO:tasks.workunit.client.1.smithi181.stdout:1/349: dwrite d1/d2/f4 [0,4194304] 0 2024-04-27T04:09:54.110 INFO:tasks.workunit.client.1.smithi181.stdout:8/405: unlink d1/d6/d2f/d30/f43 0 2024-04-27T04:09:54.112 INFO:tasks.workunit.client.1.smithi181.stdout:8/406: creat d1/d6/d2f/d47/f8e x:0 0 0 2024-04-27T04:09:54.113 INFO:tasks.workunit.client.1.smithi181.stdout:1/350: creat d1/d4c/d1e/d20/d53/d56/f6f x:0 0 0 2024-04-27T04:09:54.114 INFO:tasks.workunit.client.1.smithi181.stdout:8/407: symlink d1/d6/d59/l8f 0 2024-04-27T04:09:54.117 INFO:tasks.workunit.client.1.smithi181.stdout:8/408: creat d1/d8c/f90 x:0 0 0 2024-04-27T04:09:54.118 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:09:53 smithi181 ceph-mon[33774]: pgmap v262: 65 pgs: 65 active+clean; 660 MiB data, 3.7 GiB used, 533 GiB / 536 GiB avail; 5.4 MiB/s rd, 66 MiB/s wr, 314 op/s 2024-04-27T04:09:54.120 INFO:tasks.workunit.client.1.smithi181.stdout:8/409: write d1/d6/d1a/f2b [992739,128311] 0 2024-04-27T04:09:54.126 INFO:tasks.workunit.client.1.smithi181.stdout:1/351: write d1/f16 [1292240,12973] 0 2024-04-27T04:09:54.128 INFO:tasks.workunit.client.1.smithi181.stdout:1/352: symlink d1/d4c/d1e/d20/d53/l70 0 2024-04-27T04:09:54.128 INFO:tasks.workunit.client.1.smithi181.stdout:8/410: getdents d1/d8c 0 2024-04-27T04:09:54.130 INFO:tasks.workunit.client.1.smithi181.stdout:8/411: mkdir d1/d6/d6f/d91 0 2024-04-27T04:09:54.132 INFO:tasks.workunit.client.1.smithi181.stdout:8/412: mknod d1/d6/d2f/d40/d2a/c92 0 2024-04-27T04:09:54.135 INFO:tasks.workunit.client.1.smithi181.stdout:8/413: rename d1/d6/d1a/d25/c49 to d1/d8c/c93 0 2024-04-27T04:09:54.137 INFO:tasks.workunit.client.1.smithi181.stdout:6/181: sync 2024-04-27T04:09:54.137 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:09:53 smithi162 ceph-mon[26841]: pgmap v262: 65 pgs: 65 active+clean; 660 MiB data, 3.7 GiB used, 533 GiB / 536 GiB avail; 5.4 MiB/s rd, 66 MiB/s wr, 314 op/s 2024-04-27T04:09:54.140 INFO:tasks.workunit.client.1.smithi181.stdout:8/414: creat d1/d6/d2f/d72/f94 x:0 0 0 2024-04-27T04:09:54.140 INFO:tasks.workunit.client.1.smithi181.stdout:6/182: readlink d7/d31/d38/l39 0 2024-04-27T04:09:54.144 INFO:tasks.workunit.client.1.smithi181.stdout:1/353: dread d1/d2/f13 [0,4194304] 0 2024-04-27T04:09:54.144 INFO:tasks.workunit.client.1.smithi181.stdout:1/354: chown d1/d2/d5/d6/f4e 402551 1 2024-04-27T04:09:54.170 INFO:tasks.workunit.client.1.smithi181.stdout:8/415: fdatasync d1/d6/d2f/d40/f4c 0 2024-04-27T04:09:54.171 INFO:tasks.workunit.client.1.smithi181.stdout:8/416: rename d1/d8c to d1/d8c/d95 22 2024-04-27T04:09:54.172 INFO:tasks.workunit.client.1.smithi181.stdout:8/417: chown d1/d6/d1a/c8b 130 1 2024-04-27T04:09:54.222 INFO:tasks.workunit.client.1.smithi181.stdout:1/355: link d1/d2/l6c d1/d4c/d1e/l71 0 2024-04-27T04:09:54.225 INFO:tasks.workunit.client.1.smithi181.stdout:8/418: truncate d1/f33 3983594 0 2024-04-27T04:09:54.229 INFO:tasks.workunit.client.1.smithi181.stdout:1/356: link d1/d2/d5/d6/f22 d1/d4c/d1e/d20/d53/f72 0 2024-04-27T04:09:54.229 INFO:tasks.workunit.client.1.smithi181.stdout:1/357: stat d1 0 2024-04-27T04:09:54.232 INFO:tasks.workunit.client.1.smithi181.stdout:6/183: write d7/dc/f12 [1640995,77325] 0 2024-04-27T04:09:54.233 INFO:tasks.workunit.client.1.smithi181.stdout:1/358: dread d1/d4c/f3e [0,4194304] 0 2024-04-27T04:09:54.233 INFO:tasks.workunit.client.1.smithi181.stdout:6/184: write d7/dc/d1e/f2a [880228,29813] 0 2024-04-27T04:09:54.234 INFO:tasks.workunit.client.1.smithi181.stdout:1/359: fsync d1/d4c/d1e/f49 0 2024-04-27T04:09:54.235 INFO:tasks.workunit.client.1.smithi181.stdout:6/185: fsync d7/d2f/f3c 0 2024-04-27T04:09:54.238 INFO:tasks.workunit.client.1.smithi181.stdout:1/360: dread d1/d2/d5/d6/f1a [0,4194304] 0 2024-04-27T04:09:54.241 INFO:tasks.workunit.client.1.smithi181.stdout:8/419: write d1/f33 [3817367,38382] 0 2024-04-27T04:09:54.243 INFO:tasks.workunit.client.1.smithi181.stdout:6/186: dwrite d7/d2f/f3c [0,4194304] 0 2024-04-27T04:09:54.255 INFO:tasks.workunit.client.1.smithi181.stdout:1/361: dread d1/d2/d5/d6/f67 [0,4194304] 0 2024-04-27T04:09:54.260 INFO:tasks.workunit.client.1.smithi181.stdout:6/187: dwrite d7/f19 [0,4194304] 0 2024-04-27T04:09:54.272 INFO:tasks.workunit.client.1.smithi181.stdout:6/188: dread d7/dc/f12 [0,4194304] 0 2024-04-27T04:09:54.273 INFO:tasks.workunit.client.1.smithi181.stdout:1/362: dread d1/d4c/d1e/d20/f26 [0,4194304] 0 2024-04-27T04:09:54.274 INFO:tasks.workunit.client.1.smithi181.stdout:1/363: write d1/f16 [1113032,43391] 0 2024-04-27T04:09:54.274 INFO:tasks.workunit.client.1.smithi181.stdout:1/364: readlink d1/d4c/d52/l69 0 2024-04-27T04:09:54.275 INFO:tasks.workunit.client.1.smithi181.stdout:6/189: mkdir d7/dc/d1c/d34/d3f 0 2024-04-27T04:09:54.276 INFO:tasks.workunit.client.1.smithi181.stdout:1/365: creat d1/d2/d5/d6/f73 x:0 0 0 2024-04-27T04:09:54.277 INFO:tasks.workunit.client.1.smithi181.stdout:6/190: mknod d7/dc/c40 0 2024-04-27T04:09:54.278 INFO:tasks.workunit.client.1.smithi181.stdout:6/191: write d7/d31/f37 [1537895,129074] 0 2024-04-27T04:09:54.279 INFO:tasks.workunit.client.1.smithi181.stdout:6/192: read d7/d2f/f3c [1920236,94444] 0 2024-04-27T04:09:54.288 INFO:tasks.workunit.client.1.smithi181.stdout:1/366: getdents d1/d4c/d60 0 2024-04-27T04:09:54.291 INFO:tasks.workunit.client.1.smithi181.stdout:6/193: link d7/d2f/f3c d7/dc/f41 0 2024-04-27T04:09:54.295 INFO:tasks.workunit.client.1.smithi181.stdout:1/367: dwrite d1/d4c/d1e/d20/d53/d56/f6f [0,4194304] 0 2024-04-27T04:09:54.296 INFO:tasks.workunit.client.1.smithi181.stdout:1/368: chown d1/d2 33 1 2024-04-27T04:09:54.298 INFO:tasks.workunit.client.1.smithi181.stdout:6/194: dwrite d7/f26 [0,4194304] 0 2024-04-27T04:09:54.300 INFO:tasks.workunit.client.1.smithi181.stdout:1/369: unlink d1/d4c/f4d 0 2024-04-27T04:09:54.327 INFO:tasks.workunit.client.1.smithi181.stdout:6/195: truncate d7/d2f/f3c 3538510 0 2024-04-27T04:09:54.328 INFO:tasks.workunit.client.1.smithi181.stdout:6/196: stat d7/d1b/l3d 0 2024-04-27T04:09:54.329 INFO:tasks.workunit.client.1.smithi181.stdout:1/370: write d1/d2/d5/d6/db/f19 [2001168,97891] 0 2024-04-27T04:09:54.330 INFO:tasks.workunit.client.1.smithi181.stdout:6/197: creat d7/d1b/f42 x:0 0 0 2024-04-27T04:09:54.333 INFO:tasks.workunit.client.1.smithi181.stdout:6/198: fsync d7/dc/d1c/f1f 0 2024-04-27T04:09:54.334 INFO:tasks.workunit.client.1.smithi181.stdout:6/199: write d7/f26 [7748842,117245] 0 2024-04-27T04:09:54.340 INFO:tasks.workunit.client.1.smithi181.stdout:6/200: dwrite d7/dc/ff [0,4194304] 0 2024-04-27T04:09:54.341 INFO:tasks.workunit.client.1.smithi181.stdout:6/201: write d7/f26 [5486801,64456] 0 2024-04-27T04:09:54.341 INFO:tasks.workunit.client.1.smithi181.stdout:6/202: write f0 [1105476,82426] 0 2024-04-27T04:09:54.351 INFO:tasks.workunit.client.1.smithi181.stdout:6/203: getdents d7/d27 0 2024-04-27T04:09:54.353 INFO:tasks.workunit.client.1.smithi181.stdout:6/204: write d7/dc/d1c/f20 [37912,125326] 0 2024-04-27T04:09:54.353 INFO:tasks.workunit.client.1.smithi181.stdout:5/398: sync 2024-04-27T04:09:54.353 INFO:tasks.workunit.client.1.smithi181.stdout:6/205: chown d7/f19 3 1 2024-04-27T04:09:54.354 INFO:tasks.workunit.client.1.smithi181.stdout:5/399: fdatasync d4/d10/f59 0 2024-04-27T04:09:54.362 INFO:tasks.workunit.client.1.smithi181.stdout:5/400: rename d4/d18 to d4/de/d2c/d46/d4d/d8e 0 2024-04-27T04:09:54.368 INFO:tasks.workunit.client.1.smithi181.stdout:6/206: link d7/d1b/f29 d7/f43 0 2024-04-27T04:09:54.370 INFO:tasks.workunit.client.1.smithi181.stdout:5/401: mkdir d4/de/d2c/d2e/d8f 0 2024-04-27T04:09:54.371 INFO:tasks.workunit.client.1.smithi181.stdout:6/207: creat d7/d27/f44 x:0 0 0 2024-04-27T04:09:54.376 INFO:tasks.workunit.client.1.smithi181.stdout:5/402: mkdir d4/de/d2f/d7a/d90 0 2024-04-27T04:09:54.378 INFO:tasks.workunit.client.1.smithi181.stdout:5/403: read d4/de/d2c/d46/d4d/d8e/d54/f58 [589632,94795] 0 2024-04-27T04:09:54.382 INFO:tasks.workunit.client.1.smithi181.stdout:5/404: rmdir d4/d10/d72 39 2024-04-27T04:09:54.383 INFO:tasks.workunit.client.1.smithi181.stdout:5/405: write d4/de/d2c/d46/d4d/d8e/f8a [312975,121032] 0 2024-04-27T04:09:54.387 INFO:tasks.workunit.client.1.smithi181.stdout:5/406: creat d4/de/d2c/d46/d4d/d8e/d67/d71/f91 x:0 0 0 2024-04-27T04:09:54.391 INFO:tasks.workunit.client.1.smithi181.stdout:5/407: mkdir d4/d10/d29/d92 0 2024-04-27T04:09:54.400 INFO:tasks.workunit.client.1.smithi181.stdout:6/208: fsync d7/d27/f44 0 2024-04-27T04:09:54.405 INFO:tasks.workunit.client.1.smithi181.stdout:6/209: dwrite d7/d1b/f42 [0,4194304] 0 2024-04-27T04:09:54.412 INFO:tasks.workunit.client.1.smithi181.stdout:6/210: dwrite d7/fa [0,4194304] 0 2024-04-27T04:09:54.417 INFO:tasks.workunit.client.1.smithi181.stdout:6/211: rename d7/d2f to d7/d31/d45 0 2024-04-27T04:09:54.428 INFO:tasks.workunit.client.1.smithi181.stdout:5/408: dwrite d4/de/f47 [0,4194304] 0 2024-04-27T04:09:54.431 INFO:tasks.workunit.client.1.smithi181.stdout:5/409: mknod d4/de/d2c/d46/d4d/d8e/d67/c93 0 2024-04-27T04:09:54.432 INFO:tasks.workunit.client.1.smithi181.stdout:5/410: write d4/de/d2c/d46/d4d/d8e/f8a [333775,64977] 0 2024-04-27T04:09:54.434 INFO:tasks.workunit.client.1.smithi181.stdout:5/411: creat d4/de/d2c/d46/d4d/f94 x:0 0 0 2024-04-27T04:09:54.435 INFO:tasks.workunit.client.1.smithi181.stdout:6/212: truncate d7/f8 8371343 0 2024-04-27T04:09:54.436 INFO:tasks.workunit.client.1.smithi181.stdout:6/213: readlink d7/dc/d1c/l22 0 2024-04-27T04:09:54.437 INFO:tasks.workunit.client.1.smithi181.stdout:5/412: mkdir d4/de/d95 0 2024-04-27T04:09:54.439 INFO:tasks.workunit.client.1.smithi181.stdout:6/214: symlink d7/dc/l46 0 2024-04-27T04:09:54.442 INFO:tasks.workunit.client.1.smithi181.stdout:6/215: link d7/dc/f12 d7/d1b/d3e/f47 0 2024-04-27T04:09:54.443 INFO:tasks.workunit.client.1.smithi181.stdout:5/413: getdents d4/de/d2c/d46/d4d/d8e/d54 0 2024-04-27T04:09:54.444 INFO:tasks.workunit.client.1.smithi181.stdout:6/216: mknod d7/dc/d1e/c48 0 2024-04-27T04:09:54.445 INFO:tasks.workunit.client.1.smithi181.stdout:6/217: chown d7/d1b/f42 1544410 1 2024-04-27T04:09:54.447 INFO:tasks.workunit.client.1.smithi181.stdout:5/414: link d4/de/d2c/d2e/d55/f3f d4/f96 0 2024-04-27T04:09:54.449 INFO:tasks.workunit.client.1.smithi181.stdout:6/218: creat d7/f49 x:0 0 0 2024-04-27T04:09:54.449 INFO:tasks.workunit.client.1.smithi181.stdout:5/415: symlink d4/de/d2c/d46/d4d/d8e/d54/l97 0 2024-04-27T04:09:54.451 INFO:tasks.workunit.client.1.smithi181.stdout:5/416: write d4/d10/d72/f77 [2970260,6376] 0 2024-04-27T04:09:54.456 INFO:tasks.workunit.client.1.smithi181.stdout:5/417: dread d4/d10/d29/f79 [0,4194304] 0 2024-04-27T04:09:54.458 INFO:tasks.workunit.client.1.smithi181.stdout:5/418: chown d4/de/d2c/d46/d4d/l61 118 1 2024-04-27T04:09:54.460 INFO:tasks.workunit.client.1.smithi181.stdout:5/419: symlink d4/de/d2c/d2e/l98 0 2024-04-27T04:09:54.465 INFO:tasks.workunit.client.1.smithi181.stdout:6/219: truncate d7/d1b/f42 1871361 0 2024-04-27T04:09:54.469 INFO:tasks.workunit.client.1.smithi181.stdout:5/420: write d4/de/f1e [825495,5945] 0 2024-04-27T04:09:54.470 INFO:tasks.workunit.client.1.smithi181.stdout:5/421: write d4/de/d2c/d46/d4d/d8e/f3a [675824,83632] 0 2024-04-27T04:09:54.475 INFO:tasks.workunit.client.1.smithi181.stdout:5/422: getdents d4/d10/d72 0 2024-04-27T04:09:54.476 INFO:tasks.workunit.client.1.smithi181.stdout:6/220: dwrite d7/dc/d1c/f1f [0,4194304] 0 2024-04-27T04:09:54.479 INFO:tasks.workunit.client.1.smithi181.stdout:5/423: mkdir d4/de/d2c/d46/d99 0 2024-04-27T04:09:54.481 INFO:tasks.workunit.client.1.smithi181.stdout:5/424: read d4/de/d2c/d46/d4d/d8e/d54/f58 [569616,86441] 0 2024-04-27T04:09:54.481 INFO:tasks.workunit.client.1.smithi181.stdout:6/221: fsync d7/d31/d45/f3c 0 2024-04-27T04:09:54.483 INFO:tasks.workunit.client.1.smithi181.stdout:5/425: write d4/de/d2c/d46/d4d/d8e/f8a [1332317,47142] 0 2024-04-27T04:09:54.486 INFO:tasks.workunit.client.1.smithi181.stdout:3/296: sync 2024-04-27T04:09:54.487 INFO:tasks.workunit.client.1.smithi181.stdout:2/367: sync 2024-04-27T04:09:54.487 INFO:tasks.workunit.client.1.smithi181.stdout:9/343: sync 2024-04-27T04:09:54.489 INFO:tasks.workunit.client.1.smithi181.stdout:6/222: dwrite d7/dc/d1c/f1f [0,4194304] 0 2024-04-27T04:09:54.491 INFO:tasks.workunit.client.1.smithi181.stdout:3/297: symlink d0/d3e/d52/l61 0 2024-04-27T04:09:54.492 INFO:tasks.workunit.client.1.smithi181.stdout:3/298: stat d0/d28/l29 0 2024-04-27T04:09:54.492 INFO:tasks.workunit.client.1.smithi181.stdout:2/368: symlink d4/d9/df/l72 0 2024-04-27T04:09:54.494 INFO:tasks.workunit.client.1.smithi181.stdout:9/344: creat d0/d40/f78 x:0 0 0 2024-04-27T04:09:54.495 INFO:tasks.workunit.client.1.smithi181.stdout:9/345: write d0/d40/f78 [723009,72379] 0 2024-04-27T04:09:54.495 INFO:tasks.workunit.client.1.smithi181.stdout:3/299: creat d0/de/d20/f62 x:0 0 0 2024-04-27T04:09:54.496 INFO:tasks.workunit.client.1.smithi181.stdout:5/426: truncate d4/de/d2c/d46/d4d/d8e/f48 872142 0 2024-04-27T04:09:54.500 INFO:tasks.workunit.client.1.smithi181.stdout:6/223: link c2 d7/dc/d1c/d34/c4a 0 2024-04-27T04:09:54.501 INFO:tasks.workunit.client.1.smithi181.stdout:9/346: dread d0/d40/f2c [0,4194304] 0 2024-04-27T04:09:54.502 INFO:tasks.workunit.client.1.smithi181.stdout:5/427: rename d4/de/f1e to d4/de/d2c/d46/d99/f9a 0 2024-04-27T04:09:54.503 INFO:tasks.workunit.client.1.smithi181.stdout:2/369: link d4/d9/df/l13 d4/d9/d16/d45/l73 0 2024-04-27T04:09:54.504 INFO:tasks.workunit.client.1.smithi181.stdout:6/224: dread d7/dc/f3a [0,4194304] 0 2024-04-27T04:09:54.514 INFO:tasks.workunit.client.1.smithi181.stdout:5/428: dwrite d4/de/f14 [0,4194304] 0 2024-04-27T04:09:54.515 INFO:tasks.workunit.client.1.smithi181.stdout:9/347: mknod d0/d40/c79 0 2024-04-27T04:09:54.518 INFO:tasks.workunit.client.1.smithi181.stdout:3/300: truncate d0/f15 4091893 0 2024-04-27T04:09:54.519 INFO:tasks.workunit.client.1.smithi181.stdout:2/370: mknod d4/d9/d35/c74 0 2024-04-27T04:09:54.521 INFO:tasks.workunit.client.1.smithi181.stdout:5/429: symlink d4/d10/d72/d80/l9b 0 2024-04-27T04:09:54.523 INFO:tasks.workunit.client.1.smithi181.stdout:3/301: rename d0/f50 to d0/d3e/d52/d44/d56/f63 0 2024-04-27T04:09:54.528 INFO:tasks.workunit.client.1.smithi181.stdout:5/430: rename d4/de/d2c/d2e/d55/c81 to d4/de/d2c/d46/d4d/d8e/d67/c9c 0 2024-04-27T04:09:54.587 INFO:tasks.workunit.client.1.smithi181.stdout:6/225: getdents d7/d1b/d3e 0 2024-04-27T04:09:54.588 INFO:tasks.workunit.client.1.smithi181.stdout:3/302: mkdir d0/de/d20/d58/d64 0 2024-04-27T04:09:54.590 INFO:tasks.workunit.client.1.smithi181.stdout:6/226: unlink d7/d1b/f42 0 2024-04-27T04:09:54.592 INFO:tasks.workunit.client.1.smithi181.stdout:3/303: rename d0/de/d20/f21 to d0/d3e/d52/f65 0 2024-04-27T04:09:54.592 INFO:tasks.workunit.client.1.smithi181.stdout:5/431: dwrite d4/d10/f59 [0,4194304] 0 2024-04-27T04:09:54.593 INFO:tasks.workunit.client.1.smithi181.stdout:6/227: creat d7/d31/d45/f4b x:0 0 0 2024-04-27T04:09:54.642 INFO:tasks.workunit.client.1.smithi181.stdout:7/251: sync 2024-04-27T04:09:54.643 INFO:tasks.workunit.client.0.smithi162.stdout:4/4: sync 2024-04-27T04:09:54.643 INFO:tasks.workunit.client.0.smithi162.stdout:4/5: write - no filename 2024-04-27T04:09:54.643 INFO:tasks.workunit.client.1.smithi181.stdout:3/304: mkdir d0/d3e/d66 0 2024-04-27T04:09:54.645 INFO:tasks.workunit.client.1.smithi181.stdout:6/228: chown d7/d27/f44 1677 1 2024-04-27T04:09:54.645 INFO:tasks.workunit.client.0.smithi162.stdout:4/6: creat f0 x:0 0 0 2024-04-27T04:09:54.646 INFO:tasks.workunit.client.1.smithi181.stdout:5/432: read d4/de/d2c/d46/d4d/d8e/d54/f78 [3098821,107656] 0 2024-04-27T04:09:54.647 INFO:tasks.workunit.client.1.smithi181.stdout:5/433: stat d4/de/d2c/d2e/f56 0 2024-04-27T04:09:54.648 INFO:tasks.workunit.client.0.smithi162.stdout:4/7: link f0 f1 0 2024-04-27T04:09:54.649 INFO:tasks.workunit.client.1.smithi181.stdout:6/229: link d7/d27/f44 d7/d27/f4c 0 2024-04-27T04:09:54.650 INFO:tasks.workunit.client.1.smithi181.stdout:2/371: dread d4/d9/d16/f26 [0,4194304] 0 2024-04-27T04:09:54.651 INFO:tasks.workunit.client.1.smithi181.stdout:6/230: write d7/dc/ff [1492699,34236] 0 2024-04-27T04:09:54.652 INFO:tasks.workunit.client.1.smithi181.stdout:5/434: mkdir d4/de/d9d 0 2024-04-27T04:09:54.654 INFO:tasks.workunit.client.1.smithi181.stdout:7/252: mkdir d1/d2/d1c/d2d/d3d/d4c 0 2024-04-27T04:09:54.657 INFO:tasks.workunit.client.0.smithi162.stdout:4/8: dwrite f0 [0,4194304] 0 2024-04-27T04:09:54.660 INFO:tasks.workunit.client.1.smithi181.stdout:6/231: dwrite d7/dc/d1c/f35 [0,4194304] 0 2024-04-27T04:09:54.663 INFO:tasks.workunit.client.1.smithi181.stdout:2/372: mknod d4/d9/d35/c75 0 2024-04-27T04:09:54.665 INFO:tasks.workunit.client.0.smithi162.stdout:4/9: mkdir d2 0 2024-04-27T04:09:54.673 INFO:tasks.workunit.client.1.smithi181.stdout:5/435: dread d4/de/d2c/d46/f60 [0,4194304] 0 2024-04-27T04:09:54.674 INFO:tasks.workunit.client.1.smithi181.stdout:2/373: rename d4/c21 to d4/d9/d16/d45/c76 0 2024-04-27T04:09:54.675 INFO:tasks.workunit.client.1.smithi181.stdout:5/436: dread d4/d10/d29/f79 [0,4194304] 0 2024-04-27T04:09:54.675 INFO:tasks.workunit.client.1.smithi181.stdout:5/437: stat d4 0 2024-04-27T04:09:54.676 INFO:tasks.workunit.client.0.smithi162.stdout:4/10: mkdir d2/d3 0 2024-04-27T04:09:54.678 INFO:tasks.workunit.client.1.smithi181.stdout:2/374: creat d4/d9/d4f/d5f/f77 x:0 0 0 2024-04-27T04:09:54.681 INFO:tasks.workunit.client.1.smithi181.stdout:6/232: link d7/f43 d7/dc/d1e/f4d 0 2024-04-27T04:09:54.684 INFO:tasks.workunit.client.1.smithi181.stdout:5/438: link d4/d10/d29/f84 d4/de/d95/f9e 0 2024-04-27T04:09:54.686 INFO:tasks.workunit.client.0.smithi162.stdout:4/11: dwrite f0 [0,4194304] 0 2024-04-27T04:09:54.687 INFO:tasks.workunit.client.0.smithi162.stdout:4/12: write f0 [985916,75101] 0 2024-04-27T04:09:54.691 INFO:tasks.workunit.client.1.smithi181.stdout:5/439: dwrite d4/d10/f57 [0,4194304] 0 2024-04-27T04:09:54.692 INFO:tasks.workunit.client.1.smithi181.stdout:5/440: truncate d4/f96 1111283 0 2024-04-27T04:09:54.692 INFO:tasks.workunit.client.0.smithi162.stdout:4/13: rename f0 to d2/f4 0 2024-04-27T04:09:54.693 INFO:tasks.workunit.client.1.smithi181.stdout:6/233: getdents d7/d31/d38 0 2024-04-27T04:09:54.701 INFO:tasks.workunit.client.1.smithi181.stdout:2/375: dread d4/f8 [4194304,4194304] 0 2024-04-27T04:09:54.702 INFO:tasks.workunit.client.1.smithi181.stdout:7/253: dwrite d1/d2/f12 [4194304,4194304] 0 2024-04-27T04:09:54.702 INFO:tasks.workunit.client.1.smithi181.stdout:2/376: stat d4/d9/c6f 0 2024-04-27T04:09:54.703 INFO:tasks.workunit.client.1.smithi181.stdout:2/377: dread - d4/d9/f64 zero size 2024-04-27T04:09:54.704 INFO:tasks.workunit.client.1.smithi181.stdout:6/234: fdatasync d7/dc/f23 0 2024-04-27T04:09:54.705 INFO:tasks.workunit.client.0.smithi162.stdout:4/14: getdents d2/d3 0 2024-04-27T04:09:54.713 INFO:tasks.workunit.client.1.smithi181.stdout:2/378: dwrite d4/d9/d16/d45/f6c [0,4194304] 0 2024-04-27T04:09:54.718 INFO:tasks.workunit.client.1.smithi181.stdout:6/235: creat d7/dc/d1e/d2b/f4e x:0 0 0 2024-04-27T04:09:54.718 INFO:tasks.workunit.client.0.smithi162.stdout:4/15: fsync f1 0 2024-04-27T04:09:54.720 INFO:tasks.workunit.client.1.smithi181.stdout:6/236: chown d7/d1b/f29 6368330 1 2024-04-27T04:09:54.723 INFO:tasks.workunit.client.0.smithi162.stdout:4/16: rename d2/d3 to d2/d5 0 2024-04-27T04:09:54.723 INFO:tasks.workunit.client.0.smithi162.stdout:5/29: sync 2024-04-27T04:09:54.723 INFO:tasks.workunit.client.0.smithi162.stdout:9/3: sync 2024-04-27T04:09:54.723 INFO:tasks.workunit.client.0.smithi162.stdout:7/6: sync 2024-04-27T04:09:54.723 INFO:tasks.workunit.client.0.smithi162.stdout:2/22: sync 2024-04-27T04:09:54.723 INFO:tasks.workunit.client.0.smithi162.stdout:6/39: sync 2024-04-27T04:09:54.723 INFO:tasks.workunit.client.0.smithi162.stdout:8/61: sync 2024-04-27T04:09:54.723 INFO:tasks.workunit.client.0.smithi162.stdout:3/16: sync 2024-04-27T04:09:54.723 INFO:tasks.workunit.client.0.smithi162.stdout:0/90: sync 2024-04-27T04:09:54.723 INFO:tasks.workunit.client.0.smithi162.stdout:1/59: sync 2024-04-27T04:09:54.723 INFO:tasks.workunit.client.0.smithi162.stdout:4/17: chown f1 207969941 1 2024-04-27T04:09:54.724 INFO:tasks.workunit.client.0.smithi162.stdout:7/7: stat f0 0 2024-04-27T04:09:54.724 INFO:tasks.workunit.client.1.smithi181.stdout:7/254: getdents d1/d2/d1c/d2d 0 2024-04-27T04:09:54.725 INFO:tasks.workunit.client.1.smithi181.stdout:7/255: stat d1/d2/f12 0 2024-04-27T04:09:54.725 INFO:tasks.workunit.client.1.smithi181.stdout:6/237: rename d7/dc/c13 to d7/dc/d1c/c4f 0 2024-04-27T04:09:54.726 INFO:tasks.workunit.client.1.smithi181.stdout:5/441: dwrite d4/f96 [0,4194304] 0 2024-04-27T04:09:54.726 INFO:tasks.workunit.client.0.smithi162.stdout:5/30: mkdir d8 0 2024-04-27T04:09:54.729 INFO:tasks.workunit.client.0.smithi162.stdout:2/23: symlink d0/lb 0 2024-04-27T04:09:54.730 INFO:tasks.workunit.client.0.smithi162.stdout:5/31: dread f0 [4194304,4194304] 0 2024-04-27T04:09:54.730 INFO:tasks.workunit.client.0.smithi162.stdout:2/24: dread - d0/fa zero size 2024-04-27T04:09:54.730 INFO:tasks.workunit.client.1.smithi181.stdout:7/256: mknod d1/d2/d1c/d2d/d3d/c4d 0 2024-04-27T04:09:54.730 INFO:tasks.workunit.client.0.smithi162.stdout:9/4: dread - f0 zero size 2024-04-27T04:09:54.731 INFO:tasks.workunit.client.0.smithi162.stdout:5/32: dread - f5 zero size 2024-04-27T04:09:54.731 INFO:tasks.workunit.client.0.smithi162.stdout:4/18: chown f1 2024 1 2024-04-27T04:09:54.731 INFO:tasks.workunit.client.0.smithi162.stdout:3/17: read f0 [628265,53490] 0 2024-04-27T04:09:54.732 INFO:tasks.workunit.client.1.smithi181.stdout:7/257: dread - d1/d2/d1c/d2d/f41 zero size 2024-04-27T04:09:54.732 INFO:tasks.workunit.client.0.smithi162.stdout:0/91: chown d5/f8 13033 1 2024-04-27T04:09:54.732 INFO:tasks.workunit.client.0.smithi162.stdout:6/40: rmdir d6/db 39 2024-04-27T04:09:54.732 INFO:tasks.workunit.client.0.smithi162.stdout:3/18: stat d1 0 2024-04-27T04:09:54.733 INFO:tasks.workunit.client.0.smithi162.stdout:3/19: chown d1 15 1 2024-04-27T04:09:54.734 INFO:tasks.workunit.client.1.smithi181.stdout:5/442: creat d4/de/d2c/d2e/d55/f9f x:0 0 0 2024-04-27T04:09:54.734 INFO:tasks.workunit.client.0.smithi162.stdout:0/92: write d5/f16 [1734451,11700] 0 2024-04-27T04:09:54.734 INFO:tasks.workunit.client.0.smithi162.stdout:3/20: write f0 [155572,107635] 0 2024-04-27T04:09:54.735 INFO:tasks.workunit.client.0.smithi162.stdout:6/41: truncate d6/fa 571100 0 2024-04-27T04:09:54.736 INFO:tasks.workunit.client.0.smithi162.stdout:0/93: dread f1 [0,4194304] 0 2024-04-27T04:09:54.736 INFO:tasks.workunit.client.0.smithi162.stdout:3/21: truncate f0 1167067 0 2024-04-27T04:09:54.737 INFO:tasks.workunit.client.0.smithi162.stdout:1/60: dread d0/fb [0,4194304] 0 2024-04-27T04:09:54.737 INFO:tasks.workunit.client.0.smithi162.stdout:8/62: dread d0/fa [0,4194304] 0 2024-04-27T04:09:54.739 INFO:tasks.workunit.client.1.smithi181.stdout:7/258: unlink d1/d22/d15/d13/c3a 0 2024-04-27T04:09:54.739 INFO:tasks.workunit.client.1.smithi181.stdout:7/259: chown d1/d2/d35 455800660 1 2024-04-27T04:09:54.742 INFO:tasks.workunit.client.0.smithi162.stdout:7/8: symlink l1 0 2024-04-27T04:09:54.743 INFO:tasks.workunit.client.0.smithi162.stdout:2/25: fsync d0/f1 0 2024-04-27T04:09:54.745 INFO:tasks.workunit.client.1.smithi181.stdout:7/260: creat d1/d22/d15/d13/d1b/f4e x:0 0 0 2024-04-27T04:09:54.748 INFO:tasks.workunit.client.0.smithi162.stdout:5/33: dread f0 [0,4194304] 0 2024-04-27T04:09:54.748 INFO:tasks.workunit.client.1.smithi181.stdout:6/238: getdents d7/d27 0 2024-04-27T04:09:54.749 INFO:tasks.workunit.client.0.smithi162.stdout:4/19: mknod d2/d5/c6 0 2024-04-27T04:09:54.749 INFO:tasks.workunit.client.1.smithi181.stdout:6/239: write d7/dc/d1c/f32 [322094,88393] 0 2024-04-27T04:09:54.749 INFO:tasks.workunit.client.0.smithi162.stdout:4/20: write f1 [3550087,52936] 0 2024-04-27T04:09:54.750 INFO:tasks.workunit.client.1.smithi181.stdout:7/261: creat d1/d2/d1c/d37/f4f x:0 0 0 2024-04-27T04:09:54.750 INFO:tasks.workunit.client.0.smithi162.stdout:4/21: write d2/f4 [2442883,12354] 0 2024-04-27T04:09:54.753 INFO:tasks.workunit.client.1.smithi181.stdout:5/443: creat d4/fa0 x:0 0 0 2024-04-27T04:09:54.754 INFO:tasks.workunit.client.1.smithi181.stdout:5/444: chown d4/de/d2c/d46/l4a 77633830 1 2024-04-27T04:09:54.759 INFO:tasks.workunit.client.1.smithi181.stdout:5/445: unlink d4/d10/d29/f3c 0 2024-04-27T04:09:54.761 INFO:tasks.workunit.client.1.smithi181.stdout:5/446: write d4/de/d2c/d46/d4d/d8e/f8a [2423600,92205] 0 2024-04-27T04:09:54.761 INFO:tasks.workunit.client.0.smithi162.stdout:3/22: creat d1/f3 x:0 0 0 2024-04-27T04:09:54.765 INFO:tasks.workunit.client.1.smithi181.stdout:6/240: dwrite d7/dc/f3a [0,4194304] 0 2024-04-27T04:09:54.766 INFO:tasks.workunit.client.0.smithi162.stdout:7/9: fsync f0 0 2024-04-27T04:09:54.768 INFO:tasks.workunit.client.1.smithi181.stdout:5/447: symlink d4/d10/d72/la1 0 2024-04-27T04:09:54.769 INFO:tasks.workunit.client.1.smithi181.stdout:7/262: link d1/d2/d1c/c49 d1/d2/d1c/d37/c50 0 2024-04-27T04:09:54.770 INFO:tasks.workunit.client.1.smithi181.stdout:5/448: truncate d4/de/d95/f9e 882056 0 2024-04-27T04:09:54.770 INFO:tasks.workunit.client.0.smithi162.stdout:2/26: symlink d0/lc 0 2024-04-27T04:09:54.772 INFO:tasks.workunit.client.1.smithi181.stdout:6/241: unlink d7/dc/f23 0 2024-04-27T04:09:54.776 INFO:tasks.workunit.client.1.smithi181.stdout:5/449: chown d4/de/d2c/d46/d4d/d8e/d67/l70 3047 1 2024-04-27T04:09:54.777 INFO:tasks.workunit.client.1.smithi181.stdout:5/450: chown d4/d10/d29/c5f 9048 1 2024-04-27T04:09:54.780 INFO:tasks.workunit.client.0.smithi162.stdout:6/42: symlink d6/l12 0 2024-04-27T04:09:54.781 INFO:tasks.workunit.client.1.smithi181.stdout:5/451: dread d4/f96 [0,4194304] 0 2024-04-27T04:09:54.783 INFO:tasks.workunit.client.1.smithi181.stdout:5/452: truncate d4/de/d2c/d46/d4d/f94 257003 0 2024-04-27T04:09:54.783 INFO:tasks.workunit.client.0.smithi162.stdout:0/94: fdatasync d5/fa 0 2024-04-27T04:09:54.784 INFO:tasks.workunit.client.0.smithi162.stdout:0/95: write d5/f16 [4124054,14250] 0 2024-04-27T04:09:54.784 INFO:tasks.workunit.client.0.smithi162.stdout:0/96: rename d5 to d5/dc/d1d 22 2024-04-27T04:09:54.786 INFO:tasks.workunit.client.0.smithi162.stdout:3/23: symlink d1/l4 0 2024-04-27T04:09:54.786 INFO:tasks.workunit.client.0.smithi162.stdout:3/24: write d1/f3 [490879,34585] 0 2024-04-27T04:09:54.787 INFO:tasks.workunit.client.1.smithi181.stdout:7/263: rmdir d1/d2/d1c/d2d/d3d/d4c 0 2024-04-27T04:09:54.788 INFO:tasks.workunit.client.1.smithi181.stdout:6/242: write d7/d27/f44 [481513,60742] 0 2024-04-27T04:09:54.789 INFO:tasks.workunit.client.1.smithi181.stdout:6/243: chown d7/d1b/l3d 3646249 1 2024-04-27T04:09:54.794 INFO:tasks.workunit.client.1.smithi181.stdout:5/453: unlink d4/de/d2c/d2e/f56 0 2024-04-27T04:09:54.795 INFO:tasks.workunit.client.1.smithi181.stdout:6/244: dwrite d7/dc/d1c/f1f [0,4194304] 0 2024-04-27T04:09:54.796 INFO:tasks.workunit.client.1.smithi181.stdout:7/264: creat d1/d2/d1c/d2d/d3d/f51 x:0 0 0 2024-04-27T04:09:54.798 INFO:tasks.workunit.client.0.smithi162.stdout:7/10: creat f2 x:0 0 0 2024-04-27T04:09:54.798 INFO:tasks.workunit.client.1.smithi181.stdout:6/245: write d7/dc/d1e/d2b/f4e [871450,78712] 0 2024-04-27T04:09:54.799 INFO:tasks.workunit.client.0.smithi162.stdout:7/11: chown f2 6590 1 2024-04-27T04:09:54.799 INFO:tasks.workunit.client.0.smithi162.stdout:1/61: truncate d0/fb 17303 0 2024-04-27T04:09:54.799 INFO:tasks.workunit.client.1.smithi181.stdout:6/246: truncate d7/fb 477694 0 2024-04-27T04:09:54.800 INFO:tasks.workunit.client.0.smithi162.stdout:8/63: truncate d0/fa 333052 0 2024-04-27T04:09:54.801 INFO:tasks.workunit.client.0.smithi162.stdout:2/27: symlink d0/d4/ld 0 2024-04-27T04:09:54.802 INFO:tasks.workunit.client.1.smithi181.stdout:5/454: stat d4/de/l5b 0 2024-04-27T04:09:54.802 INFO:tasks.workunit.client.0.smithi162.stdout:9/5: link c1 c2 0 2024-04-27T04:09:54.802 INFO:tasks.workunit.client.0.smithi162.stdout:2/28: dread - d0/f7 zero size 2024-04-27T04:09:54.803 INFO:tasks.workunit.client.1.smithi181.stdout:7/265: mknod d1/d22/d15/d29/c52 0 2024-04-27T04:09:54.804 INFO:tasks.workunit.client.0.smithi162.stdout:5/34: mkdir d8/d9 0 2024-04-27T04:09:54.806 INFO:tasks.workunit.client.0.smithi162.stdout:4/22: link d2/f4 d2/d5/f7 0 2024-04-27T04:09:54.808 INFO:tasks.workunit.client.1.smithi181.stdout:7/266: chown d1/d2/d1c/d2d/l38 15 1 2024-04-27T04:09:54.811 INFO:tasks.workunit.client.0.smithi162.stdout:5/35: dwrite f5 [0,4194304] 0 2024-04-27T04:09:54.811 INFO:tasks.workunit.client.1.smithi181.stdout:0/325: sync 2024-04-27T04:09:54.812 INFO:tasks.workunit.client.0.smithi162.stdout:4/23: dread d2/f4 [0,4194304] 0 2024-04-27T04:09:54.814 INFO:tasks.workunit.client.1.smithi181.stdout:5/455: getdents d4/de/d2f/d7a/d90 0 2024-04-27T04:09:54.817 INFO:tasks.workunit.client.0.smithi162.stdout:0/97: readlink d5/dc/l19 0 2024-04-27T04:09:54.817 INFO:tasks.workunit.client.0.smithi162.stdout:0/98: read d5/dc/f14 [452154,49674] 0 2024-04-27T04:09:54.821 INFO:tasks.workunit.client.0.smithi162.stdout:0/99: dread d5/dc/fe [0,4194304] 0 2024-04-27T04:09:54.822 INFO:tasks.workunit.client.1.smithi181.stdout:6/247: write d7/d1b/d3e/f47 [2097681,46159] 0 2024-04-27T04:09:54.825 INFO:tasks.workunit.client.0.smithi162.stdout:7/12: mkdir d3 0 2024-04-27T04:09:54.827 INFO:tasks.workunit.client.1.smithi181.stdout:0/326: mkdir dc/d1f/d3b/d4a/d7a 0 2024-04-27T04:09:54.828 INFO:tasks.workunit.client.0.smithi162.stdout:8/64: mkdir d0/de 0 2024-04-27T04:09:54.829 INFO:tasks.workunit.client.0.smithi162.stdout:9/6: mkdir d3 0 2024-04-27T04:09:54.830 INFO:tasks.workunit.client.1.smithi181.stdout:6/248: creat d7/d1b/d3e/f50 x:0 0 0 2024-04-27T04:09:54.831 INFO:tasks.workunit.client.1.smithi181.stdout:5/456: write d4/de/d2c/d46/d4d/d8e/d54/f58 [1640678,50109] 0 2024-04-27T04:09:54.831 INFO:tasks.workunit.client.0.smithi162.stdout:2/29: creat d0/fe x:0 0 0 2024-04-27T04:09:54.832 INFO:tasks.workunit.client.0.smithi162.stdout:2/30: chown d0/fe 21498 1 2024-04-27T04:09:54.832 INFO:tasks.workunit.client.0.smithi162.stdout:2/31: read d0/d4/f5 [238687,69452] 0 2024-04-27T04:09:54.835 INFO:tasks.workunit.client.0.smithi162.stdout:6/43: creat d6/db/f13 x:0 0 0 2024-04-27T04:09:54.837 INFO:tasks.workunit.client.1.smithi181.stdout:6/249: creat d7/dc/d1e/f51 x:0 0 0 2024-04-27T04:09:54.839 INFO:tasks.workunit.client.1.smithi181.stdout:5/457: creat d4/de/d2c/d46/d4d/d8e/d67/fa2 x:0 0 0 2024-04-27T04:09:54.840 INFO:tasks.workunit.client.1.smithi181.stdout:0/327: dwrite dc/d1f/f48 [4194304,4194304] 0 2024-04-27T04:09:54.841 INFO:tasks.workunit.client.1.smithi181.stdout:7/267: link d1/df/l20 d1/df/d1a/l53 0 2024-04-27T04:09:54.843 INFO:tasks.workunit.client.1.smithi181.stdout:6/250: rmdir d7/dc/d1c/d34 39 2024-04-27T04:09:54.844 INFO:tasks.workunit.client.1.smithi181.stdout:6/251: dread - d7/f49 zero size 2024-04-27T04:09:54.845 INFO:tasks.workunit.client.0.smithi162.stdout:5/36: dwrite f0 [4194304,4194304] 0 2024-04-27T04:09:54.848 INFO:tasks.workunit.client.1.smithi181.stdout:5/458: dwrite d4/de/d2c/d46/d4d/d8e/f3a [0,4194304] 0 2024-04-27T04:09:54.849 INFO:tasks.workunit.client.0.smithi162.stdout:3/25: getdents d1/d2 0 2024-04-27T04:09:54.852 INFO:tasks.workunit.client.0.smithi162.stdout:0/100: mkdir d5/dc/d1e 0 2024-04-27T04:09:54.853 INFO:tasks.workunit.client.0.smithi162.stdout:4/24: write f1 [2379053,27114] 0 2024-04-27T04:09:54.857 INFO:tasks.workunit.client.1.smithi181.stdout:6/252: rename d7/d27/f44 to d7/dc/d1e/f52 0 2024-04-27T04:09:54.860 INFO:tasks.workunit.client.0.smithi162.stdout:8/65: creat d0/ff x:0 0 0 2024-04-27T04:09:54.861 INFO:tasks.workunit.client.0.smithi162.stdout:9/7: unlink c2 0 2024-04-27T04:09:54.862 INFO:tasks.workunit.client.1.smithi181.stdout:7/268: mknod d1/c54 0 2024-04-27T04:09:54.864 INFO:tasks.workunit.client.0.smithi162.stdout:0/101: dwrite d5/dc/f18 [0,4194304] 0 2024-04-27T04:09:54.866 INFO:tasks.workunit.client.0.smithi162.stdout:0/102: readlink d5/dc/l19 0 2024-04-27T04:09:54.870 INFO:tasks.workunit.client.0.smithi162.stdout:8/66: dwrite d0/f7 [0,4194304] 0 2024-04-27T04:09:54.873 INFO:tasks.workunit.client.0.smithi162.stdout:9/8: dwrite f0 [0,4194304] 0 2024-04-27T04:09:54.880 INFO:tasks.workunit.client.0.smithi162.stdout:9/9: dwrite f0 [0,4194304] 0 2024-04-27T04:09:54.895 INFO:tasks.workunit.client.1.smithi181.stdout:5/459: mkdir d4/da3 0 2024-04-27T04:09:54.908 INFO:tasks.workunit.client.0.smithi162.stdout:9/10: fdatasync f0 0 2024-04-27T04:09:54.929 INFO:tasks.workunit.client.0.smithi162.stdout:6/44: symlink d6/db/l14 0 2024-04-27T04:09:54.931 INFO:tasks.workunit.client.1.smithi181.stdout:0/328: dwrite dc/f15 [0,4194304] 0 2024-04-27T04:09:54.934 INFO:tasks.workunit.client.0.smithi162.stdout:6/45: dwrite d6/f7 [0,4194304] 0 2024-04-27T04:09:54.934 INFO:tasks.workunit.client.0.smithi162.stdout:6/46: read - d6/f11 zero size 2024-04-27T04:09:54.935 INFO:tasks.workunit.client.0.smithi162.stdout:6/47: write d6/fc [634124,30810] 0 2024-04-27T04:09:54.936 INFO:tasks.workunit.client.0.smithi162.stdout:5/37: symlink d8/la 0 2024-04-27T04:09:54.936 INFO:tasks.workunit.client.0.smithi162.stdout:6/48: chown d6/f11 190 1 2024-04-27T04:09:54.943 INFO:tasks.workunit.client.1.smithi181.stdout:6/253: link d7/d27/f4c d7/dc/d1e/f53 0 2024-04-27T04:09:54.944 INFO:tasks.workunit.client.0.smithi162.stdout:7/13: symlink d3/l4 0 2024-04-27T04:09:54.945 INFO:tasks.workunit.client.0.smithi162.stdout:7/14: readlink l1 0 2024-04-27T04:09:54.946 INFO:tasks.workunit.client.1.smithi181.stdout:5/460: dwrite d4/d10/d29/f79 [0,4194304] 0 2024-04-27T04:09:54.946 INFO:tasks.workunit.client.1.smithi181.stdout:6/254: write d7/dc/d1c/f32 [1344356,116022] 0 2024-04-27T04:09:54.950 INFO:tasks.workunit.client.1.smithi181.stdout:0/329: creat dc/d1e/d26/d41/d6f/f7b x:0 0 0 2024-04-27T04:09:54.952 INFO:tasks.workunit.client.1.smithi181.stdout:4/293: sync 2024-04-27T04:09:54.952 INFO:tasks.workunit.client.0.smithi162.stdout:4/25: creat d2/d5/f8 x:0 0 0 2024-04-27T04:09:54.953 INFO:tasks.workunit.client.1.smithi181.stdout:4/294: readlink d0/d15/d25/d2b/l20 0 2024-04-27T04:09:54.955 INFO:tasks.workunit.client.0.smithi162.stdout:4/26: dread d2/f4 [0,4194304] 0 2024-04-27T04:09:54.958 INFO:tasks.workunit.client.0.smithi162.stdout:1/62: read d0/fb [10719,68931] 0 2024-04-27T04:09:54.960 INFO:tasks.workunit.client.1.smithi181.stdout:4/295: dwrite d0/d15/d25/d2a/d36/f4a [0,4194304] 0 2024-04-27T04:09:54.961 INFO:tasks.workunit.client.1.smithi181.stdout:4/296: readlink d0/l3f 0 2024-04-27T04:09:54.966 INFO:tasks.workunit.client.1.smithi181.stdout:5/461: rmdir d4/de/d2f/d7a 39 2024-04-27T04:09:54.972 INFO:tasks.workunit.client.0.smithi162.stdout:8/67: mkdir d0/d10 0 2024-04-27T04:09:54.973 INFO:tasks.workunit.client.1.smithi181.stdout:7/269: getdents d1/d22/d15 0 2024-04-27T04:09:54.975 INFO:tasks.workunit.client.1.smithi181.stdout:4/297: creat d0/d15/de/d19/f5e x:0 0 0 2024-04-27T04:09:54.977 INFO:tasks.workunit.client.1.smithi181.stdout:4/298: truncate d0/d15/f2f 1780485 0 2024-04-27T04:09:54.980 INFO:tasks.workunit.client.1.smithi181.stdout:5/462: write d4/de/d2c/d46/f60 [2465450,74331] 0 2024-04-27T04:09:54.985 INFO:tasks.workunit.client.0.smithi162.stdout:5/38: mknod d8/cb 0 2024-04-27T04:09:54.986 INFO:tasks.workunit.client.1.smithi181.stdout:7/270: mkdir d1/d2/d1c/d55 0 2024-04-27T04:09:54.987 INFO:tasks.workunit.client.1.smithi181.stdout:7/271: fdatasync d1/d2/d1c/d2d/f41 0 2024-04-27T04:09:54.987 INFO:tasks.workunit.client.1.smithi181.stdout:2/379: sync 2024-04-27T04:09:54.988 INFO:tasks.workunit.client.1.smithi181.stdout:9/348: sync 2024-04-27T04:09:54.988 INFO:tasks.workunit.client.1.smithi181.stdout:8/420: sync 2024-04-27T04:09:54.988 INFO:tasks.workunit.client.1.smithi181.stdout:3/305: sync 2024-04-27T04:09:54.988 INFO:tasks.workunit.client.1.smithi181.stdout:1/371: sync 2024-04-27T04:09:54.988 INFO:tasks.workunit.client.0.smithi162.stdout:7/15: truncate f0 954826 0 2024-04-27T04:09:54.992 INFO:tasks.workunit.client.0.smithi162.stdout:3/26: sync 2024-04-27T04:09:54.993 INFO:tasks.workunit.client.1.smithi181.stdout:5/463: symlink d4/de/d2c/d46/d99/la4 0 2024-04-27T04:09:54.995 INFO:tasks.workunit.client.0.smithi162.stdout:4/27: creat d2/f9 x:0 0 0 2024-04-27T04:09:54.995 INFO:tasks.workunit.client.0.smithi162.stdout:4/28: chown d2/f9 809300107 1 2024-04-27T04:09:54.999 INFO:tasks.workunit.client.1.smithi181.stdout:7/272: mkdir d1/d2/d1c/d56 0 2024-04-27T04:09:55.000 INFO:tasks.workunit.client.0.smithi162.stdout:0/103: getdents d5/dc/d1e 0 2024-04-27T04:09:55.004 INFO:tasks.workunit.client.1.smithi181.stdout:9/349: creat d0/d44/d4f/f7a x:0 0 0 2024-04-27T04:09:55.005 INFO:tasks.workunit.client.0.smithi162.stdout:9/11: creat d3/f4 x:0 0 0 2024-04-27T04:09:55.006 INFO:tasks.workunit.client.1.smithi181.stdout:2/380: creat d4/d9/df/d14/f78 x:0 0 0 2024-04-27T04:09:55.009 INFO:tasks.workunit.client.1.smithi181.stdout:8/421: unlink d1/d6/d2f/d40/d2a/l75 0 2024-04-27T04:09:55.012 INFO:tasks.workunit.client.0.smithi162.stdout:6/49: link d6/fa d6/db/de/f15 0 2024-04-27T04:09:55.013 INFO:tasks.workunit.client.1.smithi181.stdout:1/372: mknod d1/d4c/d1e/d20/d53/d56/c74 0 2024-04-27T04:09:55.016 INFO:tasks.workunit.client.0.smithi162.stdout:7/16: mkdir d3/d5 0 2024-04-27T04:09:55.016 INFO:tasks.workunit.client.1.smithi181.stdout:6/255: link d7/f26 d7/d1b/d3e/f54 0 2024-04-27T04:09:55.016 INFO:tasks.workunit.client.1.smithi181.stdout:6/256: chown d7/fa 94 1 2024-04-27T04:09:55.017 INFO:tasks.workunit.client.0.smithi162.stdout:6/50: dwrite d6/f7 [0,4194304] 0 2024-04-27T04:09:55.017 INFO:tasks.workunit.client.0.smithi162.stdout:6/51: stat c4 0 2024-04-27T04:09:55.020 INFO:tasks.workunit.client.0.smithi162.stdout:3/27: mknod d1/c5 0 2024-04-27T04:09:55.022 INFO:tasks.workunit.client.1.smithi181.stdout:5/464: dread d4/de/d2c/d2e/d55/f3f [0,4194304] 0 2024-04-27T04:09:55.022 INFO:tasks.workunit.client.1.smithi181.stdout:4/299: creat d0/d15/d25/d2b/f5f x:0 0 0 2024-04-27T04:09:55.024 INFO:tasks.workunit.client.0.smithi162.stdout:3/28: dread f0 [0,4194304] 0 2024-04-27T04:09:55.024 INFO:tasks.workunit.client.0.smithi162.stdout:4/29: mkdir d2/da 0 2024-04-27T04:09:55.024 INFO:tasks.workunit.client.0.smithi162.stdout:3/29: stat d1/f3 0 2024-04-27T04:09:55.026 INFO:tasks.workunit.client.0.smithi162.stdout:1/63: creat d0/d7/d10/f12 x:0 0 0 2024-04-27T04:09:55.026 INFO:tasks.workunit.client.0.smithi162.stdout:1/64: fdatasync d0/fe 0 2024-04-27T04:09:55.027 INFO:tasks.workunit.client.0.smithi162.stdout:1/65: readlink d0/d7/la 0 2024-04-27T04:09:55.029 INFO:tasks.workunit.client.0.smithi162.stdout:1/66: dread d0/fb [0,4194304] 0 2024-04-27T04:09:55.029 INFO:tasks.workunit.client.0.smithi162.stdout:0/104: rename d5/f8 to d5/dc/f1f 0 2024-04-27T04:09:55.032 INFO:tasks.workunit.client.0.smithi162.stdout:2/32: dwrite d0/f6 [0,4194304] 0 2024-04-27T04:09:55.032 INFO:tasks.workunit.client.0.smithi162.stdout:8/68: mknod d0/d10/c11 0 2024-04-27T04:09:55.033 INFO:tasks.workunit.client.0.smithi162.stdout:8/69: chown d0/fc 11148 1 2024-04-27T04:09:55.034 INFO:tasks.workunit.client.0.smithi162.stdout:2/33: truncate d0/fa 722087 0 2024-04-27T04:09:55.036 INFO:tasks.workunit.client.0.smithi162.stdout:9/12: creat d3/f5 x:0 0 0 2024-04-27T04:09:55.037 INFO:tasks.workunit.client.0.smithi162.stdout:0/105: dwrite f0 [4194304,4194304] 0 2024-04-27T04:09:55.037 INFO:tasks.workunit.client.0.smithi162.stdout:9/13: dread - d3/f4 zero size 2024-04-27T04:09:55.039 INFO:tasks.workunit.client.0.smithi162.stdout:5/39: creat d8/d9/fc x:0 0 0 2024-04-27T04:09:55.041 INFO:tasks.workunit.client.1.smithi181.stdout:6/257: symlink d7/dc/l55 0 2024-04-27T04:09:55.042 INFO:tasks.workunit.client.0.smithi162.stdout:2/34: dwrite d0/fa [0,4194304] 0 2024-04-27T04:09:55.044 INFO:tasks.workunit.client.0.smithi162.stdout:2/35: write d0/d4/f5 [3513038,101288] 0 2024-04-27T04:09:55.046 INFO:tasks.workunit.client.1.smithi181.stdout:6/258: dread d7/dc/d1c/f20 [0,4194304] 0 2024-04-27T04:09:55.055 INFO:tasks.workunit.client.1.smithi181.stdout:9/350: symlink d0/d44/d4b/d43/l7b 0 2024-04-27T04:09:55.058 INFO:tasks.workunit.client.1.smithi181.stdout:8/422: symlink d1/d6/d88/l96 0 2024-04-27T04:09:55.060 INFO:tasks.workunit.client.1.smithi181.stdout:1/373: creat d1/d2/d5/d6/d6a/f75 x:0 0 0 2024-04-27T04:09:55.061 INFO:tasks.workunit.client.1.smithi181.stdout:1/374: chown d1/d4c/d1e/d20/f5c 82819243 1 2024-04-27T04:09:55.062 INFO:tasks.workunit.client.1.smithi181.stdout:1/375: dread - d1/d4c/d1e/d20/f61 zero size 2024-04-27T04:09:55.064 INFO:tasks.workunit.client.0.smithi162.stdout:1/67: mkdir d0/d7/d13 0 2024-04-27T04:09:55.068 INFO:tasks.workunit.client.1.smithi181.stdout:4/300: mkdir d0/d15/de/d60 0 2024-04-27T04:09:55.069 INFO:tasks.workunit.client.0.smithi162.stdout:9/14: creat d3/f6 x:0 0 0 2024-04-27T04:09:55.070 INFO:tasks.workunit.client.1.smithi181.stdout:3/306: getdents d0 0 2024-04-27T04:09:55.071 INFO:tasks.workunit.client.0.smithi162.stdout:5/40: symlink d8/ld 0 2024-04-27T04:09:55.071 INFO:tasks.workunit.client.1.smithi181.stdout:3/307: write d0/d3e/d52/f47 [975868,44260] 0 2024-04-27T04:09:55.074 INFO:tasks.workunit.client.1.smithi181.stdout:8/423: creat d1/d6/d2f/d30/d5c/f97 x:0 0 0 2024-04-27T04:09:55.075 INFO:tasks.workunit.client.0.smithi162.stdout:9/15: dwrite d3/f5 [0,4194304] 0 2024-04-27T04:09:55.078 INFO:tasks.workunit.client.0.smithi162.stdout:2/36: rename d0/d4/f5 to d0/d4/ff 0 2024-04-27T04:09:55.079 INFO:tasks.workunit.client.1.smithi181.stdout:1/376: creat d1/d4c/d52/f76 x:0 0 0 2024-04-27T04:09:55.081 INFO:tasks.workunit.client.0.smithi162.stdout:7/17: link f2 d3/f6 0 2024-04-27T04:09:55.083 INFO:tasks.workunit.client.1.smithi181.stdout:6/259: rename d7/dc/d1c/c4f to d7/d31/c56 0 2024-04-27T04:09:55.084 INFO:tasks.workunit.client.1.smithi181.stdout:0/330: truncate dc/d1e/d26/f28 496709 0 2024-04-27T04:09:55.086 INFO:tasks.workunit.client.0.smithi162.stdout:6/52: symlink d6/l16 0 2024-04-27T04:09:55.086 INFO:tasks.workunit.client.1.smithi181.stdout:0/331: write f6 [2937581,73287] 0 2024-04-27T04:09:55.087 INFO:tasks.workunit.client.1.smithi181.stdout:1/377: dwrite d1/d4c/d1e/f4b [0,4194304] 0 2024-04-27T04:09:55.088 INFO:tasks.workunit.client.1.smithi181.stdout:6/260: write d7/dc/d1e/f51 [906104,64277] 0 2024-04-27T04:09:55.088 INFO:tasks.workunit.client.1.smithi181.stdout:0/332: write dc/d1e/d26/d41/f4f [130092,70560] 0 2024-04-27T04:09:55.090 INFO:tasks.workunit.client.1.smithi181.stdout:0/333: chown dc/d1e/d26/d68/l78 1239398 1 2024-04-27T04:09:55.092 INFO:tasks.workunit.client.1.smithi181.stdout:3/308: readlink d0/d3e/d52/l4f 0 2024-04-27T04:09:55.093 INFO:tasks.workunit.client.0.smithi162.stdout:6/53: dwrite f5 [0,4194304] 0 2024-04-27T04:09:55.093 INFO:tasks.workunit.client.0.smithi162.stdout:6/54: chown d6/db 57173158 1 2024-04-27T04:09:55.094 INFO:tasks.workunit.client.0.smithi162.stdout:0/106: mkdir d5/dc/d1e/d20 0 2024-04-27T04:09:55.095 INFO:tasks.workunit.client.0.smithi162.stdout:0/107: read - d5/f15 zero size 2024-04-27T04:09:55.097 INFO:tasks.workunit.client.1.smithi181.stdout:1/378: dwrite d1/d2/d5/d6/db/f19 [0,4194304] 0 2024-04-27T04:09:55.104 INFO:tasks.workunit.client.0.smithi162.stdout:6/55: dwrite d6/f11 [0,4194304] 0 2024-04-27T04:09:55.105 INFO:tasks.workunit.client.1.smithi181.stdout:9/351: creat d0/d44/f7c x:0 0 0 2024-04-27T04:09:55.110 INFO:tasks.workunit.client.0.smithi162.stdout:6/56: dread d6/f11 [0,4194304] 0 2024-04-27T04:09:55.113 INFO:tasks.workunit.client.0.smithi162.stdout:1/68: getdents d0/d11 0 2024-04-27T04:09:55.114 INFO:tasks.workunit.client.0.smithi162.stdout:1/69: truncate d0/d7/fd 4292188 0 2024-04-27T04:09:55.114 INFO:tasks.workunit.client.0.smithi162.stdout:1/70: stat d0 0 2024-04-27T04:09:55.115 INFO:tasks.workunit.client.0.smithi162.stdout:1/71: write d0/d7/d10/f12 [66033,109955] 0 2024-04-27T04:09:55.116 INFO:tasks.workunit.client.1.smithi181.stdout:3/309: rename d0/c3a to d0/d28/c67 0 2024-04-27T04:09:55.126 INFO:tasks.workunit.client.1.smithi181.stdout:9/352: creat d0/d44/d4b/d43/f7d x:0 0 0 2024-04-27T04:09:55.126 INFO:tasks.workunit.client.1.smithi181.stdout:9/353: readlink d0/d44/l72 0 2024-04-27T04:09:55.126 INFO:tasks.workunit.client.1.smithi181.stdout:8/424: symlink d1/d6/d2f/d40/l98 0 2024-04-27T04:09:55.126 INFO:tasks.workunit.client.0.smithi162.stdout:4/30: truncate d2/f4 1941140 0 2024-04-27T04:09:55.127 INFO:tasks.workunit.client.0.smithi162.stdout:4/31: chown d2/da 5505 1 2024-04-27T04:09:55.132 INFO:tasks.workunit.client.0.smithi162.stdout:6/57: fsync d6/fa 0 2024-04-27T04:09:55.134 INFO:tasks.workunit.client.1.smithi181.stdout:1/379: fdatasync d1/d4c/d1e/d20/d53/f72 0 2024-04-27T04:09:55.135 INFO:tasks.workunit.client.1.smithi181.stdout:1/380: chown d1/d4c/d1e/c57 32466138 1 2024-04-27T04:09:55.136 INFO:tasks.workunit.client.1.smithi181.stdout:4/301: getdents d0/d15/d48/d57 0 2024-04-27T04:09:55.137 INFO:tasks.workunit.client.1.smithi181.stdout:7/273: mkdir d1/d57 0 2024-04-27T04:09:55.139 INFO:tasks.workunit.client.1.smithi181.stdout:7/274: truncate d1/d2/d1c/d2d/d3d/f51 798174 0 2024-04-27T04:09:55.140 INFO:tasks.workunit.client.0.smithi162.stdout:1/72: dwrite d0/fb [0,4194304] 0 2024-04-27T04:09:55.143 INFO:tasks.workunit.client.1.smithi181.stdout:8/425: readlink d1/d6/d2f/d72/l8a 0 2024-04-27T04:09:55.143 INFO:tasks.workunit.client.1.smithi181.stdout:7/275: dread - d1/d2/d1c/d2d/f41 zero size 2024-04-27T04:09:55.144 INFO:tasks.workunit.client.1.smithi181.stdout:7/276: stat d1/d2/f19 0 2024-04-27T04:09:55.147 INFO:tasks.workunit.client.0.smithi162.stdout:3/30: sync 2024-04-27T04:09:55.147 INFO:tasks.workunit.client.0.smithi162.stdout:4/32: creat d2/d5/fb x:0 0 0 2024-04-27T04:09:55.152 INFO:tasks.workunit.client.1.smithi181.stdout:5/465: dwrite d4/de/d2c/d2e/d55/f3f [0,4194304] 0 2024-04-27T04:09:55.153 INFO:tasks.workunit.client.1.smithi181.stdout:5/466: chown d4/l1d 10979 1 2024-04-27T04:09:55.162 INFO:tasks.workunit.client.0.smithi162.stdout:5/41: getdents d8 0 2024-04-27T04:09:55.162 INFO:tasks.workunit.client.0.smithi162.stdout:5/42: chown d8 2 1 2024-04-27T04:09:55.165 INFO:tasks.workunit.client.0.smithi162.stdout:2/37: fsync d0/f6 0 2024-04-27T04:09:55.165 INFO:tasks.workunit.client.0.smithi162.stdout:8/70: sync 2024-04-27T04:09:55.167 INFO:tasks.workunit.client.1.smithi181.stdout:8/426: creat d1/d6/d1a/d25/f99 x:0 0 0 2024-04-27T04:09:55.168 INFO:tasks.workunit.client.0.smithi162.stdout:1/73: link d0/d7/fd d0/f14 0 2024-04-27T04:09:55.169 INFO:tasks.workunit.client.1.smithi181.stdout:7/277: write d1/df/f18 [3775083,121627] 0 2024-04-27T04:09:55.171 INFO:tasks.workunit.client.0.smithi162.stdout:2/38: dwrite d0/d4/ff [0,4194304] 0 2024-04-27T04:09:55.171 INFO:tasks.workunit.client.0.smithi162.stdout:3/31: creat d1/d2/f6 x:0 0 0 2024-04-27T04:09:55.173 INFO:tasks.workunit.client.0.smithi162.stdout:2/39: chown d0/lb 15404023 1 2024-04-27T04:09:55.176 INFO:tasks.workunit.client.0.smithi162.stdout:5/43: mknod d8/ce 0 2024-04-27T04:09:55.177 INFO:tasks.workunit.client.0.smithi162.stdout:3/32: dread d1/f3 [0,4194304] 0 2024-04-27T04:09:55.177 INFO:tasks.workunit.client.0.smithi162.stdout:1/74: dwrite d0/fe [0,4194304] 0 2024-04-27T04:09:55.178 INFO:tasks.workunit.client.0.smithi162.stdout:4/33: creat d2/da/fc x:0 0 0 2024-04-27T04:09:55.179 INFO:tasks.workunit.client.1.smithi181.stdout:5/467: symlink d4/de/d2c/la5 0 2024-04-27T04:09:55.181 INFO:tasks.workunit.client.0.smithi162.stdout:6/58: link f5 d6/f17 0 2024-04-27T04:09:55.182 INFO:tasks.workunit.client.1.smithi181.stdout:8/427: symlink d1/d6/l9a 0 2024-04-27T04:09:55.184 INFO:tasks.workunit.client.0.smithi162.stdout:8/71: mkdir d0/d12 0 2024-04-27T04:09:55.184 INFO:tasks.workunit.client.1.smithi181.stdout:7/278: mknod d1/d2/d1c/d2d/d3d/c58 0 2024-04-27T04:09:55.188 INFO:tasks.workunit.client.1.smithi181.stdout:0/334: write dc/d1e/f2e [1531705,3372] 0 2024-04-27T04:09:55.189 INFO:tasks.workunit.client.1.smithi181.stdout:5/468: chown d4/de/c7c 26 1 2024-04-27T04:09:55.191 INFO:tasks.workunit.client.1.smithi181.stdout:7/279: dread d1/d22/d15/f28 [0,4194304] 0 2024-04-27T04:09:55.192 INFO:tasks.workunit.client.1.smithi181.stdout:7/280: chown d1/d2 30196 1 2024-04-27T04:09:55.193 INFO:tasks.workunit.client.1.smithi181.stdout:8/428: mknod d1/d6/d2f/d30/d5c/c9b 0 2024-04-27T04:09:55.193 INFO:tasks.workunit.client.1.smithi181.stdout:7/281: write d1/d2/d1c/d2d/d3d/f51 [494465,20950] 0 2024-04-27T04:09:55.194 INFO:tasks.workunit.client.1.smithi181.stdout:7/282: chown d1 7666 1 2024-04-27T04:09:55.195 INFO:tasks.workunit.client.1.smithi181.stdout:7/283: write d1/d2/d1c/d37/f3f [4845929,60858] 0 2024-04-27T04:09:55.195 INFO:tasks.workunit.client.1.smithi181.stdout:9/354: rmdir d0/d44 39 2024-04-27T04:09:55.201 INFO:tasks.workunit.client.1.smithi181.stdout:1/381: getdents d1/d4c/d1e 0 2024-04-27T04:09:55.202 INFO:tasks.workunit.client.0.smithi162.stdout:2/40: dwrite d0/f7 [0,4194304] 0 2024-04-27T04:09:55.202 INFO:tasks.workunit.client.1.smithi181.stdout:5/469: rename d4 to d4/de/d2c/d2e/da6 22 2024-04-27T04:09:55.204 INFO:tasks.workunit.client.1.smithi181.stdout:2/381: sync 2024-04-27T04:09:55.204 INFO:tasks.workunit.client.0.smithi162.stdout:7/18: write f0 [1018212,14234] 0 2024-04-27T04:09:55.206 INFO:tasks.workunit.client.1.smithi181.stdout:6/261: dwrite d7/d1b/d3e/f54 [4194304,4194304] 0 2024-04-27T04:09:55.207 INFO:tasks.workunit.client.0.smithi162.stdout:0/108: truncate d5/dc/f18 3121512 0 2024-04-27T04:09:55.208 INFO:tasks.workunit.client.1.smithi181.stdout:6/262: chown d7/dc/d1e/f4d 2492401 1 2024-04-27T04:09:55.208 INFO:tasks.workunit.client.0.smithi162.stdout:0/109: fsync d5/f11 0 2024-04-27T04:09:55.210 INFO:tasks.workunit.client.0.smithi162.stdout:0/110: readlink d5/l13 0 2024-04-27T04:09:55.211 INFO:tasks.workunit.client.0.smithi162.stdout:6/59: dwrite d6/fc [0,4194304] 0 2024-04-27T04:09:55.213 INFO:tasks.workunit.client.1.smithi181.stdout:8/429: creat d1/d6/d2f/d30/d66/f9c x:0 0 0 2024-04-27T04:09:55.213 INFO:tasks.workunit.client.0.smithi162.stdout:8/72: dwrite d0/fd [0,4194304] 0 2024-04-27T04:09:55.214 INFO:tasks.workunit.client.1.smithi181.stdout:4/302: write d0/d15/d25/d2a/d36/f42 [1687626,54510] 0 2024-04-27T04:09:55.214 INFO:tasks.workunit.client.1.smithi181.stdout:0/335: dread dc/fd [0,4194304] 0 2024-04-27T04:09:55.215 INFO:tasks.workunit.client.1.smithi181.stdout:8/430: write d1/d6/f7b [577070,19982] 0 2024-04-27T04:09:55.216 INFO:tasks.workunit.client.1.smithi181.stdout:0/336: write dc/d1e/f2e [1955236,43543] 0 2024-04-27T04:09:55.217 INFO:tasks.workunit.client.1.smithi181.stdout:8/431: readlink d1/d6/l9a 0 2024-04-27T04:09:55.221 INFO:tasks.workunit.client.0.smithi162.stdout:2/41: unlink d0/d4/ff 0 2024-04-27T04:09:55.223 INFO:tasks.workunit.client.1.smithi181.stdout:9/355: chown d0/d44/d4f/f54 4527634 1 2024-04-27T04:09:55.224 INFO:tasks.workunit.client.1.smithi181.stdout:4/303: dwrite d0/d15/d25/d2a/d36/f42 [4194304,4194304] 0 2024-04-27T04:09:55.226 INFO:tasks.workunit.client.1.smithi181.stdout:4/304: stat d0/d15/d25/d2b/f14 0 2024-04-27T04:09:55.238 INFO:tasks.workunit.client.0.smithi162.stdout:6/60: truncate d6/fa 1599141 0 2024-04-27T04:09:55.238 INFO:tasks.workunit.client.0.smithi162.stdout:3/33: rmdir d1/d2 39 2024-04-27T04:09:55.238 INFO:tasks.workunit.client.0.smithi162.stdout:6/61: readlink d6/db/l14 0 2024-04-27T04:09:55.239 INFO:tasks.workunit.client.1.smithi181.stdout:2/382: mkdir d4/d9/d4f/d79 0 2024-04-27T04:09:55.242 INFO:tasks.workunit.client.1.smithi181.stdout:2/383: chown d4/d9/d35/c4b 153 1 2024-04-27T04:09:55.242 INFO:tasks.workunit.client.1.smithi181.stdout:3/310: dwrite d0/d28/f42 [0,4194304] 0 2024-04-27T04:09:55.243 INFO:tasks.workunit.client.1.smithi181.stdout:2/384: chown d4/l1b 505468 1 2024-04-27T04:09:55.243 INFO:tasks.workunit.client.0.smithi162.stdout:9/16: sync 2024-04-27T04:09:55.244 INFO:tasks.workunit.client.0.smithi162.stdout:2/42: mknod d0/d4/c10 0 2024-04-27T04:09:55.245 INFO:tasks.workunit.client.0.smithi162.stdout:2/43: chown d0/d4/c10 0 1 2024-04-27T04:09:55.246 INFO:tasks.workunit.client.0.smithi162.stdout:2/44: write d0/f7 [389010,85556] 0 2024-04-27T04:09:55.249 INFO:tasks.workunit.client.0.smithi162.stdout:0/111: chown d5/dc/f18 14 1 2024-04-27T04:09:55.250 INFO:tasks.workunit.client.1.smithi181.stdout:8/432: readlink d1/d6/d1a/l68 0 2024-04-27T04:09:55.251 INFO:tasks.workunit.client.1.smithi181.stdout:8/433: write d1/d6/d2f/d72/f94 [113452,102972] 0 2024-04-27T04:09:55.252 INFO:tasks.workunit.client.0.smithi162.stdout:3/34: fsync d1/f3 0 2024-04-27T04:09:55.252 INFO:tasks.workunit.client.1.smithi181.stdout:0/337: creat dc/d1f/f7c x:0 0 0 2024-04-27T04:09:55.255 INFO:tasks.workunit.client.0.smithi162.stdout:6/62: creat d6/db/de/f18 x:0 0 0 2024-04-27T04:09:55.255 INFO:tasks.workunit.client.0.smithi162.stdout:2/45: dwrite d0/fe [0,4194304] 0 2024-04-27T04:09:55.256 INFO:tasks.workunit.client.1.smithi181.stdout:9/356: creat d0/d4d/f7e x:0 0 0 2024-04-27T04:09:55.257 INFO:tasks.workunit.client.1.smithi181.stdout:9/357: write d0/d44/d4b/d43/f5e [40732,102021] 0 2024-04-27T04:09:55.261 INFO:tasks.workunit.client.0.smithi162.stdout:9/17: symlink d3/l7 0 2024-04-27T04:09:55.262 INFO:tasks.workunit.client.0.smithi162.stdout:9/18: read - d3/f4 zero size 2024-04-27T04:09:55.263 INFO:tasks.workunit.client.0.smithi162.stdout:9/19: fdatasync d3/f4 0 2024-04-27T04:09:55.264 INFO:tasks.workunit.client.0.smithi162.stdout:4/34: dwrite f1 [0,4194304] 0 2024-04-27T04:09:55.265 INFO:tasks.workunit.client.0.smithi162.stdout:4/35: dread - d2/f9 zero size 2024-04-27T04:09:55.266 INFO:tasks.workunit.client.0.smithi162.stdout:5/44: sync 2024-04-27T04:09:55.268 INFO:tasks.workunit.client.0.smithi162.stdout:0/112: rename d5/dc/f12 to d5/dc/d1e/f21 0 2024-04-27T04:09:55.268 INFO:tasks.workunit.client.0.smithi162.stdout:5/45: fsync d8/d9/fc 0 2024-04-27T04:09:55.269 INFO:tasks.workunit.client.0.smithi162.stdout:1/75: truncate d0/d7/d10/f12 1799 0 2024-04-27T04:09:55.270 INFO:tasks.workunit.client.1.smithi181.stdout:5/470: creat d4/da3/fa7 x:0 0 0 2024-04-27T04:09:55.271 INFO:tasks.workunit.client.1.smithi181.stdout:5/471: fdatasync d4/f96 0 2024-04-27T04:09:55.275 INFO:tasks.workunit.client.0.smithi162.stdout:9/20: dwrite d3/f5 [4194304,4194304] 0 2024-04-27T04:09:55.278 INFO:tasks.workunit.client.1.smithi181.stdout:6/263: mknod d7/dc/d1c/c57 0 2024-04-27T04:09:55.278 INFO:tasks.workunit.client.0.smithi162.stdout:9/21: chown d3/f4 0 1 2024-04-27T04:09:55.280 INFO:tasks.workunit.client.1.smithi181.stdout:6/264: write d7/dc/d1e/f30 [874538,38141] 0 2024-04-27T04:09:55.280 INFO:tasks.workunit.client.0.smithi162.stdout:0/113: dwrite f0 [0,4194304] 0 2024-04-27T04:09:55.284 INFO:tasks.workunit.client.0.smithi162.stdout:8/73: link d0/l9 d0/de/l13 0 2024-04-27T04:09:55.287 INFO:tasks.workunit.client.0.smithi162.stdout:7/19: dwrite d3/f6 [0,4194304] 0 2024-04-27T04:09:55.288 INFO:tasks.workunit.client.1.smithi181.stdout:0/338: mkdir dc/d1e/d26/d41/d6f/d7d 0 2024-04-27T04:09:55.291 INFO:tasks.workunit.client.0.smithi162.stdout:4/36: mkdir d2/da/dd 0 2024-04-27T04:09:55.292 INFO:tasks.workunit.client.1.smithi181.stdout:9/358: creat d0/d4d/f7f x:0 0 0 2024-04-27T04:09:55.292 INFO:tasks.workunit.client.0.smithi162.stdout:9/22: dwrite d3/f5 [0,4194304] 0 2024-04-27T04:09:55.293 INFO:tasks.workunit.client.0.smithi162.stdout:9/23: fsync f0 0 2024-04-27T04:09:55.294 INFO:tasks.workunit.client.0.smithi162.stdout:9/24: write f0 [1750,95247] 0 2024-04-27T04:09:55.297 INFO:tasks.workunit.client.1.smithi181.stdout:4/305: unlink d0/c16 0 2024-04-27T04:09:55.300 INFO:tasks.workunit.client.1.smithi181.stdout:7/284: dwrite d1/d22/d15/d1f/d34/f3c [0,4194304] 0 2024-04-27T04:09:55.301 INFO:tasks.workunit.client.1.smithi181.stdout:1/382: link d1/d4c/d52/f68 d1/f77 0 2024-04-27T04:09:55.302 INFO:tasks.workunit.client.0.smithi162.stdout:4/37: dwrite d2/d5/fb [0,4194304] 0 2024-04-27T04:09:55.304 INFO:tasks.workunit.client.0.smithi162.stdout:4/38: truncate d2/da/fc 760654 0 2024-04-27T04:09:55.313 INFO:tasks.workunit.client.0.smithi162.stdout:4/39: dwrite d2/da/fc [0,4194304] 0 2024-04-27T04:09:55.315 INFO:tasks.workunit.client.0.smithi162.stdout:4/40: write d2/d5/f7 [4331586,113490] 0 2024-04-27T04:09:55.316 INFO:tasks.workunit.client.1.smithi181.stdout:6/265: creat d7/d31/f58 x:0 0 0 2024-04-27T04:09:55.325 INFO:tasks.workunit.client.1.smithi181.stdout:8/434: dwrite d1/d6/d2f/d30/f38 [0,4194304] 0 2024-04-27T04:09:55.328 INFO:tasks.workunit.client.1.smithi181.stdout:8/435: dread - d1/d6/d2f/d47/f8e zero size 2024-04-27T04:09:55.330 INFO:tasks.workunit.client.1.smithi181.stdout:6/266: dwrite d7/dc/d1e/f30 [0,4194304] 0 2024-04-27T04:09:55.332 INFO:tasks.workunit.client.1.smithi181.stdout:6/267: write d7/dc/d1e/d2b/f4e [609145,68834] 0 2024-04-27T04:09:55.334 INFO:tasks.workunit.client.1.smithi181.stdout:4/306: mkdir d0/d15/d25/d2a/d61 0 2024-04-27T04:09:55.334 INFO:tasks.workunit.client.1.smithi181.stdout:6/268: read d7/dc/d1e/f52 [534721,14315] 0 2024-04-27T04:09:55.336 INFO:tasks.workunit.client.1.smithi181.stdout:6/269: readlink d7/dc/l16 0 2024-04-27T04:09:55.336 INFO:tasks.workunit.client.1.smithi181.stdout:6/270: stat d7/f43 0 2024-04-27T04:09:55.347 INFO:tasks.workunit.client.1.smithi181.stdout:2/385: creat d4/d9/f7a x:0 0 0 2024-04-27T04:09:55.349 INFO:tasks.workunit.client.1.smithi181.stdout:0/339: mkdir dc/d1f/d6b/d7e 0 2024-04-27T04:09:55.349 INFO:tasks.workunit.client.0.smithi162.stdout:0/114: write f1 [968057,62937] 0 2024-04-27T04:09:55.351 INFO:tasks.workunit.client.1.smithi181.stdout:5/472: dread d4/de/d2f/f5d [4194304,4194304] 0 2024-04-27T04:09:55.359 INFO:tasks.workunit.client.1.smithi181.stdout:7/285: getdents d1/d2/d1c/d56 0 2024-04-27T04:09:55.359 INFO:tasks.workunit.client.1.smithi181.stdout:7/286: chown d1/d22/d15/d13/d1b/f4e 674579035 1 2024-04-27T04:09:55.360 INFO:tasks.workunit.client.1.smithi181.stdout:7/287: dread - d1/d22/d15/d13/d1b/f47 zero size 2024-04-27T04:09:55.367 INFO:tasks.workunit.client.1.smithi181.stdout:2/386: symlink d4/d9/d4f/l7b 0 2024-04-27T04:09:55.368 INFO:tasks.workunit.client.1.smithi181.stdout:6/271: write d7/d27/f4c [79122,99127] 0 2024-04-27T04:09:55.371 INFO:tasks.workunit.client.1.smithi181.stdout:9/359: creat d0/d44/d4b/f80 x:0 0 0 2024-04-27T04:09:55.373 INFO:tasks.workunit.client.1.smithi181.stdout:6/272: dread d7/d1b/d3e/f54 [0,4194304] 0 2024-04-27T04:09:55.375 INFO:tasks.workunit.client.1.smithi181.stdout:8/436: getdents d1/d6/d6f/d91 0 2024-04-27T04:09:55.378 INFO:tasks.workunit.client.1.smithi181.stdout:4/307: rename d0/l52 to d0/d15/d48/l62 0 2024-04-27T04:09:55.379 INFO:tasks.workunit.client.1.smithi181.stdout:9/360: dwrite d0/f8 [0,4194304] 0 2024-04-27T04:09:55.380 INFO:tasks.workunit.client.1.smithi181.stdout:6/273: dread d7/fa [0,4194304] 0 2024-04-27T04:09:55.380 INFO:tasks.workunit.client.1.smithi181.stdout:7/288: creat d1/d2/d1c/d37/f59 x:0 0 0 2024-04-27T04:09:55.383 INFO:tasks.workunit.client.1.smithi181.stdout:1/383: rename d1/d2/d5/f59 to d1/d2/d5/f78 0 2024-04-27T04:09:55.384 INFO:tasks.workunit.client.1.smithi181.stdout:3/311: getdents d0/de 0 2024-04-27T04:09:55.388 INFO:tasks.workunit.client.1.smithi181.stdout:5/473: dread d4/de/f6e [0,4194304] 0 2024-04-27T04:09:55.391 INFO:tasks.workunit.client.0.smithi162.stdout:7/20: rmdir d3 39 2024-04-27T04:09:55.395 INFO:tasks.workunit.client.1.smithi181.stdout:5/474: dwrite d4/de/f47 [0,4194304] 0 2024-04-27T04:09:55.396 INFO:tasks.workunit.client.1.smithi181.stdout:5/475: dread - d4/de/d2c/d46/d4d/d8e/d67/d71/f91 zero size 2024-04-27T04:09:55.396 INFO:tasks.workunit.client.1.smithi181.stdout:8/437: mkdir d1/d6/d2f/d30/d9d 0 2024-04-27T04:09:55.399 INFO:tasks.workunit.client.0.smithi162.stdout:3/35: creat d1/f7 x:0 0 0 2024-04-27T04:09:55.403 INFO:tasks.workunit.client.1.smithi181.stdout:5/476: dwrite d4/d10/f59 [0,4194304] 0 2024-04-27T04:09:55.412 INFO:tasks.workunit.client.1.smithi181.stdout:1/384: creat d1/d4c/d1e/f79 x:0 0 0 2024-04-27T04:09:55.415 INFO:tasks.workunit.client.1.smithi181.stdout:2/387: mknod d4/d9/df/c7c 0 2024-04-27T04:09:55.417 INFO:tasks.workunit.client.0.smithi162.stdout:2/46: link d0/lc d0/d4/l11 0 2024-04-27T04:09:55.422 INFO:tasks.workunit.client.0.smithi162.stdout:0/115: creat d5/dc/f22 x:0 0 0 2024-04-27T04:09:55.422 INFO:tasks.workunit.client.0.smithi162.stdout:0/116: rename d5/dc to d5/dc/d23 22 2024-04-27T04:09:55.424 INFO:tasks.workunit.client.0.smithi162.stdout:8/74: creat d0/d12/f14 x:0 0 0 2024-04-27T04:09:55.425 INFO:tasks.workunit.client.0.smithi162.stdout:9/25: sync 2024-04-27T04:09:55.425 INFO:tasks.workunit.client.0.smithi162.stdout:1/76: sync 2024-04-27T04:09:55.425 INFO:tasks.workunit.client.0.smithi162.stdout:7/21: rename d3 to d3/d5/d7 22 2024-04-27T04:09:55.425 INFO:tasks.workunit.client.0.smithi162.stdout:5/46: link c3 d8/cf 0 2024-04-27T04:09:55.426 INFO:tasks.workunit.client.1.smithi181.stdout:7/289: symlink d1/d2/d1c/d56/l5a 0 2024-04-27T04:09:55.428 INFO:tasks.workunit.client.0.smithi162.stdout:6/63: getdents d6/db/de 0 2024-04-27T04:09:55.429 INFO:tasks.workunit.client.0.smithi162.stdout:6/64: dread - d6/f9 zero size 2024-04-27T04:09:55.429 INFO:tasks.workunit.client.1.smithi181.stdout:5/477: rename d4/d10/d72/la1 to d4/de/d2c/d46/d4d/d8e/d67/d71/la8 0 2024-04-27T04:09:55.431 INFO:tasks.workunit.client.0.smithi162.stdout:4/41: creat d2/da/dd/fe x:0 0 0 2024-04-27T04:09:55.431 INFO:tasks.workunit.client.0.smithi162.stdout:9/26: dread f0 [0,4194304] 0 2024-04-27T04:09:55.432 INFO:tasks.workunit.client.0.smithi162.stdout:9/27: rename d3 to d3/d8 22 2024-04-27T04:09:55.433 INFO:tasks.workunit.client.0.smithi162.stdout:4/42: read d2/da/fc [623819,562] 0 2024-04-27T04:09:55.433 INFO:tasks.workunit.client.0.smithi162.stdout:9/28: chown d3 1356 1 2024-04-27T04:09:55.433 INFO:tasks.workunit.client.1.smithi181.stdout:7/290: dwrite d1/d22/d15/d13/d1b/f33 [0,4194304] 0 2024-04-27T04:09:55.434 INFO:tasks.workunit.client.0.smithi162.stdout:5/47: dwrite f0 [4194304,4194304] 0 2024-04-27T04:09:55.434 INFO:tasks.workunit.client.0.smithi162.stdout:9/29: readlink d3/l7 0 2024-04-27T04:09:55.437 INFO:tasks.workunit.client.0.smithi162.stdout:8/75: creat d0/d12/f15 x:0 0 0 2024-04-27T04:09:55.439 INFO:tasks.workunit.client.0.smithi162.stdout:8/76: write d0/d12/f14 [11364,20499] 0 2024-04-27T04:09:55.440 INFO:tasks.workunit.client.0.smithi162.stdout:4/43: dread d2/d5/f7 [4194304,4194304] 0 2024-04-27T04:09:55.440 INFO:tasks.workunit.client.1.smithi181.stdout:0/340: link dc/d1f/d3b/f44 dc/d1e/d26/d41/d6f/f7f 0 2024-04-27T04:09:55.441 INFO:tasks.workunit.client.0.smithi162.stdout:4/44: truncate d2/f9 423538 0 2024-04-27T04:09:55.442 INFO:tasks.workunit.client.0.smithi162.stdout:4/45: truncate d2/f9 1260796 0 2024-04-27T04:09:55.443 INFO:tasks.workunit.client.0.smithi162.stdout:4/46: chown d2/f9 0 1 2024-04-27T04:09:55.446 INFO:tasks.workunit.client.1.smithi181.stdout:9/361: link d0/d44/d4b/f62 d0/d40/f81 0 2024-04-27T04:09:55.446 INFO:tasks.workunit.client.0.smithi162.stdout:7/22: unlink d3/f6 0 2024-04-27T04:09:55.452 INFO:tasks.workunit.client.0.smithi162.stdout:3/36: symlink d1/d2/l8 0 2024-04-27T04:09:55.452 INFO:tasks.workunit.client.1.smithi181.stdout:9/362: dwrite d0/f23 [0,4194304] 0 2024-04-27T04:09:55.453 INFO:tasks.workunit.client.0.smithi162.stdout:3/37: dread - d1/d2/f6 zero size 2024-04-27T04:09:55.453 INFO:tasks.workunit.client.1.smithi181.stdout:4/308: link d0/d15/d25/d2b/d7/c2d d0/d15/d25/d2b/d7/c63 0 2024-04-27T04:09:55.453 INFO:tasks.workunit.client.1.smithi181.stdout:9/363: stat d0/d44/d4b/f62 0 2024-04-27T04:09:55.455 INFO:tasks.workunit.client.0.smithi162.stdout:6/65: write d6/f11 [557286,23196] 0 2024-04-27T04:09:55.462 INFO:tasks.workunit.client.0.smithi162.stdout:6/66: dwrite d6/db/de/f10 [0,4194304] 0 2024-04-27T04:09:55.463 INFO:tasks.workunit.client.1.smithi181.stdout:6/274: link d7/d27/f2e d7/f59 0 2024-04-27T04:09:55.465 INFO:tasks.workunit.client.1.smithi181.stdout:1/385: write d1/d4c/d1e/d20/f5c [862332,64445] 0 2024-04-27T04:09:55.466 INFO:tasks.workunit.client.1.smithi181.stdout:1/386: chown d1/d2/d5/d6/db/d2d 13527 1 2024-04-27T04:09:55.466 INFO:tasks.workunit.client.0.smithi162.stdout:9/30: symlink d3/l9 0 2024-04-27T04:09:55.468 INFO:tasks.workunit.client.1.smithi181.stdout:3/312: fdatasync d0/d3e/d52/f65 0 2024-04-27T04:09:55.470 INFO:tasks.workunit.client.1.smithi181.stdout:2/388: write d4/d9/df/d14/f24 [4979475,126574] 0 2024-04-27T04:09:55.471 INFO:tasks.workunit.client.0.smithi162.stdout:0/117: dwrite d5/f9 [4194304,4194304] 0 2024-04-27T04:09:55.474 INFO:tasks.workunit.client.1.smithi181.stdout:4/309: chown d0/d15/d25/d40/f4f 13709047 1 2024-04-27T04:09:55.480 INFO:tasks.workunit.client.1.smithi181.stdout:9/364: rename d0/d44/d4b/d50/d6c/f63 to d0/d44/d4b/d43/f82 0 2024-04-27T04:09:55.482 INFO:tasks.workunit.client.1.smithi181.stdout:8/438: truncate d1/d6/d2f/d30/f38 4191596 0 2024-04-27T04:09:55.483 INFO:tasks.workunit.client.1.smithi181.stdout:4/310: dwrite d0/d15/f2f [0,4194304] 0 2024-04-27T04:09:55.483 INFO:tasks.workunit.client.1.smithi181.stdout:8/439: readlink d1/d6/d2f/d40/l98 0 2024-04-27T04:09:55.485 INFO:tasks.workunit.client.1.smithi181.stdout:5/478: mkdir d4/da9 0 2024-04-27T04:09:55.487 INFO:tasks.workunit.client.0.smithi162.stdout:3/38: creat d1/d2/f9 x:0 0 0 2024-04-27T04:09:55.489 INFO:tasks.workunit.client.1.smithi181.stdout:6/275: creat d7/dc/d1e/d2b/f5a x:0 0 0 2024-04-27T04:09:55.490 INFO:tasks.workunit.client.0.smithi162.stdout:2/47: rmdir d0/d4 39 2024-04-27T04:09:55.491 INFO:tasks.workunit.client.1.smithi181.stdout:6/276: readlink d7/d31/d38/l39 0 2024-04-27T04:09:55.491 INFO:tasks.workunit.client.1.smithi181.stdout:9/365: dwrite d0/d44/d4b/d50/d6c/f6d [0,4194304] 0 2024-04-27T04:09:55.496 INFO:tasks.workunit.client.0.smithi162.stdout:6/67: mknod d6/db/de/c19 0 2024-04-27T04:09:55.497 INFO:tasks.workunit.client.0.smithi162.stdout:6/68: chown d6/f7 127290965 1 2024-04-27T04:09:55.503 INFO:tasks.workunit.client.1.smithi181.stdout:2/389: dread d4/d9/f1a [4194304,4194304] 0 2024-04-27T04:09:55.504 INFO:tasks.workunit.client.0.smithi162.stdout:9/31: creat d3/fa x:0 0 0 2024-04-27T04:09:55.504 INFO:tasks.workunit.client.1.smithi181.stdout:2/390: write d4/d9/d5d/f71 [805840,110474] 0 2024-04-27T04:09:55.510 INFO:tasks.workunit.client.1.smithi181.stdout:7/291: truncate d1/d2/d1c/d2d/f3e 1466135 0 2024-04-27T04:09:55.510 INFO:tasks.workunit.client.1.smithi181.stdout:7/292: chown d1/d22/d15/d29/c52 23089280 1 2024-04-27T04:09:55.513 INFO:tasks.workunit.client.1.smithi181.stdout:5/479: symlink d4/de/d2c/d46/d4d/laa 0 2024-04-27T04:09:55.514 INFO:tasks.workunit.client.0.smithi162.stdout:1/77: dwrite d0/d7/d10/f12 [0,4194304] 0 2024-04-27T04:09:55.514 INFO:tasks.workunit.client.0.smithi162.stdout:5/48: truncate f5 641705 0 2024-04-27T04:09:55.515 INFO:tasks.workunit.client.0.smithi162.stdout:7/23: rename f2 to d3/d5/f8 0 2024-04-27T04:09:55.516 INFO:tasks.workunit.client.0.smithi162.stdout:7/24: chown d3/d5 80123013 1 2024-04-27T04:09:55.519 INFO:tasks.workunit.client.0.smithi162.stdout:9/32: creat d3/fb x:0 0 0 2024-04-27T04:09:55.520 INFO:tasks.workunit.client.0.smithi162.stdout:5/49: dwrite f0 [0,4194304] 0 2024-04-27T04:09:55.523 INFO:tasks.workunit.client.0.smithi162.stdout:8/77: truncate d0/fd 18396 0 2024-04-27T04:09:55.530 INFO:tasks.workunit.client.0.smithi162.stdout:2/48: dread d0/d4/f9 [0,4194304] 0 2024-04-27T04:09:55.533 INFO:tasks.workunit.client.1.smithi181.stdout:8/440: dwrite d1/f48 [0,4194304] 0 2024-04-27T04:09:55.535 INFO:tasks.workunit.client.0.smithi162.stdout:1/78: symlink d0/d7/d10/l15 0 2024-04-27T04:09:55.537 INFO:tasks.workunit.client.1.smithi181.stdout:3/313: unlink d0/f15 0 2024-04-27T04:09:55.539 INFO:tasks.workunit.client.1.smithi181.stdout:0/341: link dc/d1f/c40 dc/d1e/d26/d41/d6f/c80 0 2024-04-27T04:09:55.541 INFO:tasks.workunit.client.0.smithi162.stdout:7/25: rmdir d3/d5 39 2024-04-27T04:09:55.541 INFO:tasks.workunit.client.0.smithi162.stdout:1/79: dwrite d0/fe [0,4194304] 0 2024-04-27T04:09:55.543 INFO:tasks.workunit.client.0.smithi162.stdout:9/33: creat d3/fc x:0 0 0 2024-04-27T04:09:55.544 INFO:tasks.workunit.client.1.smithi181.stdout:1/387: truncate d1/d4c/f1f 5506440 0 2024-04-27T04:09:55.548 INFO:tasks.workunit.client.0.smithi162.stdout:9/34: dread d3/f5 [4194304,4194304] 0 2024-04-27T04:09:55.548 INFO:tasks.workunit.client.1.smithi181.stdout:4/311: symlink d0/d15/d25/l64 0 2024-04-27T04:09:55.549 INFO:tasks.workunit.client.0.smithi162.stdout:9/35: write d3/f4 [1006462,57625] 0 2024-04-27T04:09:55.552 INFO:tasks.workunit.client.0.smithi162.stdout:2/49: fdatasync d0/f7 0 2024-04-27T04:09:55.557 INFO:tasks.workunit.client.0.smithi162.stdout:0/118: rename d5/c1b to d5/dc/d1e/c24 0 2024-04-27T04:09:55.559 INFO:tasks.workunit.client.0.smithi162.stdout:1/80: mkdir d0/d7/d10/d16 0 2024-04-27T04:09:55.561 INFO:tasks.workunit.client.1.smithi181.stdout:2/391: write d4/d9/d16/f2a [220292,54220] 0 2024-04-27T04:09:55.566 INFO:tasks.workunit.client.0.smithi162.stdout:9/36: dread f0 [0,4194304] 0 2024-04-27T04:09:55.567 INFO:tasks.workunit.client.1.smithi181.stdout:0/342: dread f6 [0,4194304] 0 2024-04-27T04:09:55.572 INFO:tasks.workunit.client.1.smithi181.stdout:6/277: truncate d7/dc/ff 3434923 0 2024-04-27T04:09:55.572 INFO:tasks.workunit.client.1.smithi181.stdout:6/278: dread - d7/dc/d1e/f4d zero size 2024-04-27T04:09:55.578 INFO:tasks.workunit.client.1.smithi181.stdout:7/293: unlink d1/d22/d15/d1f/f36 0 2024-04-27T04:09:55.578 INFO:tasks.workunit.client.0.smithi162.stdout:9/37: stat c1 0 2024-04-27T04:09:55.580 INFO:tasks.workunit.client.1.smithi181.stdout:4/312: dwrite d0/d15/f50 [0,4194304] 0 2024-04-27T04:09:55.581 INFO:tasks.workunit.client.1.smithi181.stdout:4/313: fdatasync d0/d15/d25/d2b/f1c 0 2024-04-27T04:09:55.582 INFO:tasks.workunit.client.1.smithi181.stdout:5/480: mknod d4/de/d2f/d7a/cab 0 2024-04-27T04:09:55.583 INFO:tasks.workunit.client.1.smithi181.stdout:9/366: creat d0/d44/d4b/f83 x:0 0 0 2024-04-27T04:09:55.585 INFO:tasks.workunit.client.0.smithi162.stdout:5/50: write f5 [28501,102883] 0 2024-04-27T04:09:55.594 INFO:tasks.workunit.client.1.smithi181.stdout:2/392: mkdir d4/d9/d5d/d7d 0 2024-04-27T04:09:55.595 INFO:tasks.workunit.client.1.smithi181.stdout:8/441: symlink d1/d6/d2f/d47/d89/l9e 0 2024-04-27T04:09:55.599 INFO:tasks.workunit.client.1.smithi181.stdout:3/314: creat d0/d3e/d52/f68 x:0 0 0 2024-04-27T04:09:55.599 INFO:tasks.workunit.client.1.smithi181.stdout:0/343: chown dc/d1e/c73 289 1 2024-04-27T04:09:55.600 INFO:tasks.workunit.client.1.smithi181.stdout:1/388: dwrite d1/d2/d5/d6/db/f40 [0,4194304] 0 2024-04-27T04:09:55.603 INFO:tasks.workunit.client.1.smithi181.stdout:8/442: dread d1/d6/d2f/d72/f94 [0,4194304] 0 2024-04-27T04:09:55.604 INFO:tasks.workunit.client.1.smithi181.stdout:0/344: fsync dc/d1e/d26/f6c 0 2024-04-27T04:09:55.605 INFO:tasks.workunit.client.1.smithi181.stdout:7/294: creat d1/d22/d15/d1f/d34/f5b x:0 0 0 2024-04-27T04:09:55.606 INFO:tasks.workunit.client.1.smithi181.stdout:5/481: dread d4/de/d2c/f3e [0,4194304] 0 2024-04-27T04:09:55.608 INFO:tasks.workunit.client.0.smithi162.stdout:4/47: sync 2024-04-27T04:09:55.608 INFO:tasks.workunit.client.1.smithi181.stdout:1/389: fsync d1/d4c/d1e/d20/f61 0 2024-04-27T04:09:55.609 INFO:tasks.workunit.client.1.smithi181.stdout:2/393: dwrite d4/d9/f67 [0,4194304] 0 2024-04-27T04:09:55.610 INFO:tasks.workunit.client.0.smithi162.stdout:9/38: mknod d3/cd 0 2024-04-27T04:09:55.610 INFO:tasks.workunit.client.0.smithi162.stdout:3/39: rename f0 to d1/fa 0 2024-04-27T04:09:55.611 INFO:tasks.workunit.client.1.smithi181.stdout:4/314: rename d0/d15/d25/d2a/d36/c5a to d0/d15/d25/d2b/c65 0 2024-04-27T04:09:55.612 INFO:tasks.workunit.client.0.smithi162.stdout:8/78: dread d0/fd [0,4194304] 0 2024-04-27T04:09:55.612 INFO:tasks.workunit.client.0.smithi162.stdout:9/39: write d3/f6 [890199,47478] 0 2024-04-27T04:09:55.613 INFO:tasks.workunit.client.0.smithi162.stdout:5/51: dwrite f4 [0,4194304] 0 2024-04-27T04:09:55.615 INFO:tasks.workunit.client.0.smithi162.stdout:5/52: write f0 [346199,114186] 0 2024-04-27T04:09:55.617 INFO:tasks.workunit.client.0.smithi162.stdout:5/53: read f0 [4351957,33743] 0 2024-04-27T04:09:55.619 INFO:tasks.workunit.client.0.smithi162.stdout:4/48: creat d2/ff x:0 0 0 2024-04-27T04:09:55.619 INFO:tasks.workunit.client.1.smithi181.stdout:8/443: symlink d1/d6/d2f/d40/d2a/l9f 0 2024-04-27T04:09:55.620 INFO:tasks.workunit.client.1.smithi181.stdout:1/390: dwrite d1/d2/f4 [4194304,4194304] 0 2024-04-27T04:09:55.620 INFO:tasks.workunit.client.0.smithi162.stdout:5/54: write d8/d9/fc [524751,102531] 0 2024-04-27T04:09:55.622 INFO:tasks.workunit.client.0.smithi162.stdout:5/55: write d8/d9/fc [193676,50598] 0 2024-04-27T04:09:55.622 INFO:tasks.workunit.client.0.smithi162.stdout:8/79: dwrite d0/d12/f14 [0,4194304] 0 2024-04-27T04:09:55.623 INFO:tasks.workunit.client.1.smithi181.stdout:8/444: fdatasync d1/f3c 0 2024-04-27T04:09:55.623 INFO:tasks.workunit.client.0.smithi162.stdout:5/56: chown f4 23191 1 2024-04-27T04:09:55.624 INFO:tasks.workunit.client.0.smithi162.stdout:5/57: fdatasync f5 0 2024-04-27T04:09:55.624 INFO:tasks.workunit.client.1.smithi181.stdout:0/345: dwrite dc/f42 [0,4194304] 0 2024-04-27T04:09:55.626 INFO:tasks.workunit.client.1.smithi181.stdout:8/445: stat d1/d6/d1a/d25/c76 0 2024-04-27T04:09:55.626 INFO:tasks.workunit.client.1.smithi181.stdout:0/346: dread - dc/d1e/f63 zero size 2024-04-27T04:09:55.629 INFO:tasks.workunit.client.1.smithi181.stdout:0/347: write dc/d54/f72 [733043,116294] 0 2024-04-27T04:09:55.631 INFO:tasks.workunit.client.1.smithi181.stdout:2/394: unlink d4/d9/d4f/d5f/f77 0 2024-04-27T04:09:55.633 INFO:tasks.workunit.client.0.smithi162.stdout:9/40: creat d3/fe x:0 0 0 2024-04-27T04:09:55.633 INFO:tasks.workunit.client.0.smithi162.stdout:9/41: fdatasync d3/fb 0 2024-04-27T04:09:55.636 INFO:tasks.workunit.client.1.smithi181.stdout:9/367: getdents d0/d44/d4f 0 2024-04-27T04:09:55.640 INFO:tasks.workunit.client.1.smithi181.stdout:9/368: dread - d0/f37 zero size 2024-04-27T04:09:55.641 INFO:tasks.workunit.client.1.smithi181.stdout:3/315: rename d0/d3e/d52/d44/f45 to d0/de/d20/f69 0 2024-04-27T04:09:55.645 INFO:tasks.workunit.client.1.smithi181.stdout:2/395: dwrite d4/d9/df/d14/f24 [0,4194304] 0 2024-04-27T04:09:55.649 INFO:tasks.workunit.client.0.smithi162.stdout:7/26: sync 2024-04-27T04:09:55.649 INFO:tasks.workunit.client.0.smithi162.stdout:7/27: stat d3 0 2024-04-27T04:09:55.650 INFO:tasks.workunit.client.1.smithi181.stdout:2/396: dread d4/f8 [4194304,4194304] 0 2024-04-27T04:09:55.655 INFO:tasks.workunit.client.0.smithi162.stdout:8/80: rmdir d0/de 39 2024-04-27T04:09:55.656 INFO:tasks.workunit.client.1.smithi181.stdout:8/446: mknod d1/d6/d88/ca0 0 2024-04-27T04:09:55.656 INFO:tasks.workunit.client.1.smithi181.stdout:5/482: dwrite d4/f13 [0,4194304] 0 2024-04-27T04:09:55.657 INFO:tasks.workunit.client.0.smithi162.stdout:8/81: dread - d0/ff zero size 2024-04-27T04:09:55.657 INFO:tasks.workunit.client.0.smithi162.stdout:4/49: dwrite d2/da/fc [0,4194304] 0 2024-04-27T04:09:55.657 INFO:tasks.workunit.client.0.smithi162.stdout:5/58: creat d8/d9/f10 x:0 0 0 2024-04-27T04:09:55.658 INFO:tasks.workunit.client.1.smithi181.stdout:5/483: chown d4/de/d2f 95520742 1 2024-04-27T04:09:55.658 INFO:tasks.workunit.client.0.smithi162.stdout:7/28: dwrite f0 [0,4194304] 0 2024-04-27T04:09:55.659 INFO:tasks.workunit.client.1.smithi181.stdout:0/348: mknod dc/d1e/d26/d68/c81 0 2024-04-27T04:09:55.662 INFO:tasks.workunit.client.0.smithi162.stdout:6/69: rename d6/db/de to d6/d1a 0 2024-04-27T04:09:55.668 INFO:tasks.workunit.client.0.smithi162.stdout:6/70: dread d6/fc [0,4194304] 0 2024-04-27T04:09:55.673 INFO:tasks.workunit.client.1.smithi181.stdout:3/316: dread d0/f19 [0,4194304] 0 2024-04-27T04:09:55.673 INFO:tasks.workunit.client.0.smithi162.stdout:5/59: dread f0 [0,4194304] 0 2024-04-27T04:09:55.674 INFO:tasks.workunit.client.1.smithi181.stdout:8/447: rename d1/d6/f55 to d1/d6/d2f/d40/fa1 0 2024-04-27T04:09:55.674 INFO:tasks.workunit.client.0.smithi162.stdout:5/60: write d8/d9/f10 [727011,125927] 0 2024-04-27T04:09:55.675 INFO:tasks.workunit.client.0.smithi162.stdout:3/40: sync 2024-04-27T04:09:55.675 INFO:tasks.workunit.client.0.smithi162.stdout:2/50: sync 2024-04-27T04:09:55.676 INFO:tasks.workunit.client.1.smithi181.stdout:5/484: mkdir d4/de/d2c/d46/d4d/d8e/d67/dac 0 2024-04-27T04:09:55.677 INFO:tasks.workunit.client.1.smithi181.stdout:5/485: read d4/de/f6e [2144976,61594] 0 2024-04-27T04:09:55.679 INFO:tasks.workunit.client.0.smithi162.stdout:0/119: rename f0 to d5/dc/f25 0 2024-04-27T04:09:55.679 INFO:tasks.workunit.client.1.smithi181.stdout:0/349: symlink dc/d1e/d26/d68/l82 0 2024-04-27T04:09:55.680 INFO:tasks.workunit.client.0.smithi162.stdout:0/120: chown d5/f11 60236497 1 2024-04-27T04:09:55.681 INFO:tasks.workunit.client.0.smithi162.stdout:8/82: mknod d0/de/c16 0 2024-04-27T04:09:55.681 INFO:tasks.workunit.client.0.smithi162.stdout:0/121: dread - d5/f10 zero size 2024-04-27T04:09:55.682 INFO:tasks.workunit.client.0.smithi162.stdout:9/42: rmdir d3 39 2024-04-27T04:09:55.683 INFO:tasks.workunit.client.0.smithi162.stdout:3/41: dwrite d1/d2/f6 [0,4194304] 0 2024-04-27T04:09:55.683 INFO:tasks.workunit.client.1.smithi181.stdout:7/295: getdents d1/d22/d15/d1f 0 2024-04-27T04:09:55.686 INFO:tasks.workunit.client.0.smithi162.stdout:7/29: mkdir d3/d5/d9 0 2024-04-27T04:09:55.687 INFO:tasks.workunit.client.1.smithi181.stdout:4/315: creat d0/d15/de/f66 x:0 0 0 2024-04-27T04:09:55.688 INFO:tasks.workunit.client.1.smithi181.stdout:1/391: creat d1/d4c/d1e/f7a x:0 0 0 2024-04-27T04:09:55.689 INFO:tasks.workunit.client.1.smithi181.stdout:1/392: stat d1/d2/d5/d6/l29 0 2024-04-27T04:09:55.690 INFO:tasks.workunit.client.1.smithi181.stdout:1/393: chown d1/d2/d5/d6/db/d2d/d36/f4a 53533 1 2024-04-27T04:09:55.691 INFO:tasks.workunit.client.0.smithi162.stdout:5/61: mkdir d8/d11 0 2024-04-27T04:09:55.692 INFO:tasks.workunit.client.0.smithi162.stdout:0/122: dwrite d5/f10 [0,4194304] 0 2024-04-27T04:09:55.693 INFO:tasks.workunit.client.0.smithi162.stdout:2/51: read d0/f6 [1995541,86091] 0 2024-04-27T04:09:55.694 INFO:tasks.workunit.client.0.smithi162.stdout:0/123: chown d5/dc/f25 5800 1 2024-04-27T04:09:55.694 INFO:tasks.workunit.client.0.smithi162.stdout:2/52: stat d0/d4/c8 0 2024-04-27T04:09:55.695 INFO:tasks.workunit.client.0.smithi162.stdout:0/124: dread d5/dc/f14 [0,4194304] 0 2024-04-27T04:09:55.696 INFO:tasks.workunit.client.1.smithi181.stdout:5/486: chown d4/de/d2c/d46/d4d/d8e/d67/c9c 15 1 2024-04-27T04:09:55.697 INFO:tasks.workunit.client.1.smithi181.stdout:5/487: read d4/f13 [180655,111619] 0 2024-04-27T04:09:55.699 INFO:tasks.workunit.client.0.smithi162.stdout:1/81: rename d0/d7/d10/d16 to d0/d11/d17 0 2024-04-27T04:09:55.701 INFO:tasks.workunit.client.1.smithi181.stdout:0/350: dread dc/d1f/d3b/f50 [0,4194304] 0 2024-04-27T04:09:55.704 INFO:tasks.workunit.client.1.smithi181.stdout:9/369: creat d0/d40/f84 x:0 0 0 2024-04-27T04:09:55.705 INFO:tasks.workunit.client.0.smithi162.stdout:5/62: dread d8/d9/fc [0,4194304] 0 2024-04-27T04:09:55.705 INFO:tasks.workunit.client.0.smithi162.stdout:8/83: mkdir d0/de/d17 0 2024-04-27T04:09:55.709 INFO:tasks.workunit.client.1.smithi181.stdout:4/316: readlink d0/d15/d25/d2b/d7/l29 0 2024-04-27T04:09:55.709 INFO:tasks.workunit.client.1.smithi181.stdout:4/317: stat d0/f3d 0 2024-04-27T04:09:55.709 INFO:tasks.workunit.client.0.smithi162.stdout:9/43: write d3/fe [57092,94275] 0 2024-04-27T04:09:55.711 INFO:tasks.workunit.client.1.smithi181.stdout:1/394: rmdir d1/d2/d5/d6/db/d2d/d36 39 2024-04-27T04:09:55.711 INFO:tasks.workunit.client.0.smithi162.stdout:4/50: dwrite f1 [0,4194304] 0 2024-04-27T04:09:55.712 INFO:tasks.workunit.client.0.smithi162.stdout:4/51: stat d2/d5/fb 0 2024-04-27T04:09:55.714 INFO:tasks.workunit.client.0.smithi162.stdout:7/30: creat d3/d5/fa x:0 0 0 2024-04-27T04:09:55.715 INFO:tasks.workunit.client.1.smithi181.stdout:3/317: creat d0/de/d20/d58/d64/f6a x:0 0 0 2024-04-27T04:09:55.716 INFO:tasks.workunit.client.0.smithi162.stdout:7/31: write f0 [2635705,81170] 0 2024-04-27T04:09:55.716 INFO:tasks.workunit.client.1.smithi181.stdout:1/395: dread d1/d2/d5/d6/db/f40 [0,4194304] 0 2024-04-27T04:09:55.716 INFO:tasks.workunit.client.1.smithi181.stdout:1/396: stat d1/d2 0 2024-04-27T04:09:55.717 INFO:tasks.workunit.client.1.smithi181.stdout:1/397: write d1/d2/d5/d6/f73 [397290,24867] 0 2024-04-27T04:09:55.719 INFO:tasks.workunit.client.0.smithi162.stdout:4/52: dread d2/da/fc [0,4194304] 0 2024-04-27T04:09:55.721 INFO:tasks.workunit.client.1.smithi181.stdout:5/488: rmdir d4/de/d2c 39 2024-04-27T04:09:55.721 INFO:tasks.workunit.client.0.smithi162.stdout:4/53: write d2/da/dd/fe [32466,19877] 0 2024-04-27T04:09:55.721 INFO:tasks.workunit.client.0.smithi162.stdout:4/54: dread - d2/ff zero size 2024-04-27T04:09:55.724 INFO:tasks.workunit.client.1.smithi181.stdout:0/351: creat dc/d1f/d6b/f83 x:0 0 0 2024-04-27T04:09:55.728 INFO:tasks.workunit.client.1.smithi181.stdout:7/296: mknod d1/d2/d1c/d55/c5c 0 2024-04-27T04:09:55.729 INFO:tasks.workunit.client.1.smithi181.stdout:7/297: chown d1/d22/d15/d29/c52 205 1 2024-04-27T04:09:55.732 INFO:tasks.workunit.client.1.smithi181.stdout:8/448: link d1/d6/f4a d1/d20/fa2 0 2024-04-27T04:09:55.735 INFO:tasks.workunit.client.0.smithi162.stdout:9/44: rename d3/cd to d3/cf 0 2024-04-27T04:09:55.736 INFO:tasks.workunit.client.0.smithi162.stdout:9/45: readlink d3/l7 0 2024-04-27T04:09:55.736 INFO:tasks.workunit.client.1.smithi181.stdout:3/318: mkdir d0/de/d20/d6b 0 2024-04-27T04:09:55.736 INFO:tasks.workunit.client.0.smithi162.stdout:9/46: chown d3/fc 35611 1 2024-04-27T04:09:55.737 INFO:tasks.workunit.client.1.smithi181.stdout:9/370: write d0/f15 [167507,81773] 0 2024-04-27T04:09:55.739 INFO:tasks.workunit.client.1.smithi181.stdout:4/318: write d0/f3b [384944,27921] 0 2024-04-27T04:09:55.741 INFO:tasks.workunit.client.0.smithi162.stdout:7/32: rmdir d3/d5 39 2024-04-27T04:09:55.744 INFO:tasks.workunit.client.0.smithi162.stdout:9/47: dwrite d3/fb [0,4194304] 0 2024-04-27T04:09:55.745 INFO:tasks.workunit.client.1.smithi181.stdout:1/398: dwrite d1/d2/d5/d6/f67 [0,4194304] 0 2024-04-27T04:09:55.746 INFO:tasks.workunit.client.1.smithi181.stdout:1/399: chown d1/d2/d5/f78 1 1 2024-04-27T04:09:55.747 INFO:tasks.workunit.client.0.smithi162.stdout:4/55: creat d2/da/dd/f10 x:0 0 0 2024-04-27T04:09:55.749 INFO:tasks.workunit.client.0.smithi162.stdout:4/56: read d2/f4 [197927,39706] 0 2024-04-27T04:09:55.750 INFO:tasks.workunit.client.0.smithi162.stdout:4/57: write d2/f4 [1473188,107180] 0 2024-04-27T04:09:55.750 INFO:tasks.workunit.client.0.smithi162.stdout:0/125: unlink d5/dc/c1a 0 2024-04-27T04:09:55.750 INFO:tasks.workunit.client.1.smithi181.stdout:7/298: dread d1/d22/d15/d1f/d34/f3c [0,4194304] 0 2024-04-27T04:09:55.753 INFO:tasks.workunit.client.0.smithi162.stdout:8/84: mknod d0/c18 0 2024-04-27T04:09:55.754 INFO:tasks.workunit.client.1.smithi181.stdout:9/371: mkdir d0/d4d/d85 0 2024-04-27T04:09:55.755 INFO:tasks.workunit.client.0.smithi162.stdout:3/42: creat d1/fb x:0 0 0 2024-04-27T04:09:55.755 INFO:tasks.workunit.client.0.smithi162.stdout:3/43: dread - d1/f7 zero size 2024-04-27T04:09:55.756 INFO:tasks.workunit.client.1.smithi181.stdout:5/489: mkdir d4/de/d2c/d46/d99/dad 0 2024-04-27T04:09:55.756 INFO:tasks.workunit.client.0.smithi162.stdout:3/44: chown d1/f3 7822477 1 2024-04-27T04:09:55.757 INFO:tasks.workunit.client.0.smithi162.stdout:3/45: chown d1/d2/f6 93 1 2024-04-27T04:09:55.757 INFO:tasks.workunit.client.1.smithi181.stdout:5/490: chown d4/de/d2c/f3e 118482448 1 2024-04-27T04:09:55.758 INFO:tasks.workunit.client.1.smithi181.stdout:0/352: mknod dc/d1f/d6b/d7e/c84 0 2024-04-27T04:09:55.760 INFO:tasks.workunit.client.1.smithi181.stdout:1/400: write d1/d2/d5/f78 [748417,101407] 0 2024-04-27T04:09:55.760 INFO:tasks.workunit.client.0.smithi162.stdout:9/48: truncate f0 2480602 0 2024-04-27T04:09:55.764 INFO:tasks.workunit.client.1.smithi181.stdout:7/299: creat d1/df/f5d x:0 0 0 2024-04-27T04:09:55.765 INFO:tasks.workunit.client.0.smithi162.stdout:6/71: rename c4 to d6/c1b 0 2024-04-27T04:09:55.768 INFO:tasks.workunit.client.0.smithi162.stdout:0/126: unlink d5/dc/f25 0 2024-04-27T04:09:55.769 INFO:tasks.workunit.client.0.smithi162.stdout:1/82: rmdir d0/d7/d13 0 2024-04-27T04:09:55.770 INFO:tasks.workunit.client.1.smithi181.stdout:0/353: mknod dc/d1e/c85 0 2024-04-27T04:09:55.771 INFO:tasks.workunit.client.1.smithi181.stdout:0/354: stat dc/d1e/d26/d41 0 2024-04-27T04:09:55.772 INFO:tasks.workunit.client.0.smithi162.stdout:4/58: rmdir d2 39 2024-04-27T04:09:55.773 INFO:tasks.workunit.client.0.smithi162.stdout:1/83: dread d0/fb [0,4194304] 0 2024-04-27T04:09:55.774 INFO:tasks.workunit.client.1.smithi181.stdout:5/491: rename d4/de/d2c/d46/d4d/d8e/f23 to d4/d10/d72/d80/d85/fae 0 2024-04-27T04:09:55.779 INFO:tasks.workunit.client.0.smithi162.stdout:1/84: dwrite d0/fb [0,4194304] 0 2024-04-27T04:09:55.780 INFO:tasks.workunit.client.1.smithi181.stdout:7/300: mkdir d1/d2/d1c/d5e 0 2024-04-27T04:09:55.781 INFO:tasks.workunit.client.1.smithi181.stdout:5/492: dwrite d4/de/d2c/d46/d4d/d8e/f3a [0,4194304] 0 2024-04-27T04:09:55.781 INFO:tasks.workunit.client.1.smithi181.stdout:7/301: dread - d1/d2/f27 zero size 2024-04-27T04:09:55.781 INFO:tasks.workunit.client.0.smithi162.stdout:2/53: rename d0/lc to d0/l12 0 2024-04-27T04:09:55.784 INFO:tasks.workunit.client.1.smithi181.stdout:3/319: rename d0/d3e to d0/d6c 0 2024-04-27T04:09:55.785 INFO:tasks.workunit.client.0.smithi162.stdout:1/85: dread d0/d7/fd [0,4194304] 0 2024-04-27T04:09:55.785 INFO:tasks.workunit.client.1.smithi181.stdout:3/320: chown d0/d6c/d52/c3b 1 1 2024-04-27T04:09:55.788 INFO:tasks.workunit.client.1.smithi181.stdout:8/449: truncate d1/d6/d1a/d25/f52 649297 0 2024-04-27T04:09:55.791 INFO:tasks.workunit.client.1.smithi181.stdout:4/319: creat d0/d15/d25/d2b/f67 x:0 0 0 2024-04-27T04:09:55.792 INFO:tasks.workunit.client.0.smithi162.stdout:1/86: dread d0/d7/fd [0,4194304] 0 2024-04-27T04:09:55.792 INFO:tasks.workunit.client.0.smithi162.stdout:5/63: rename f0 to d8/f12 0 2024-04-27T04:09:55.792 INFO:tasks.workunit.client.0.smithi162.stdout:3/46: rename d1/d2 to d1/d2/dc 22 2024-04-27T04:09:55.792 INFO:tasks.workunit.client.0.smithi162.stdout:1/87: readlink d0/lf 0 2024-04-27T04:09:55.792 INFO:tasks.workunit.client.0.smithi162.stdout:5/64: chown d8/f12 2 1 2024-04-27T04:09:55.793 INFO:tasks.workunit.client.1.smithi181.stdout:4/320: fdatasync d0/d15/d25/d2b/f1c 0 2024-04-27T04:09:55.793 INFO:tasks.workunit.client.0.smithi162.stdout:1/88: fsync d0/d7/d10/f12 0 2024-04-27T04:09:55.794 INFO:tasks.workunit.client.0.smithi162.stdout:5/65: chown d8/ce 5301 1 2024-04-27T04:09:55.794 INFO:tasks.workunit.client.0.smithi162.stdout:1/89: readlink d0/lc 0 2024-04-27T04:09:55.794 INFO:tasks.workunit.client.0.smithi162.stdout:6/72: creat d6/db/f1c x:0 0 0 2024-04-27T04:09:55.795 INFO:tasks.workunit.client.0.smithi162.stdout:0/127: mkdir d5/d26 0 2024-04-27T04:09:55.802 INFO:tasks.workunit.client.1.smithi181.stdout:1/401: symlink d1/d4c/d1e/d20/d53/l7b 0 2024-04-27T04:09:55.803 INFO:tasks.workunit.client.0.smithi162.stdout:4/59: chown d2/da 2012586 1 2024-04-27T04:09:55.804 INFO:tasks.workunit.client.0.smithi162.stdout:7/33: truncate d3/d5/f8 2979778 0 2024-04-27T04:09:55.809 INFO:tasks.workunit.client.0.smithi162.stdout:4/60: dwrite d2/d5/fb [0,4194304] 0 2024-04-27T04:09:55.811 INFO:tasks.workunit.client.1.smithi181.stdout:9/372: creat d0/d40/f86 x:0 0 0 2024-04-27T04:09:55.812 INFO:tasks.workunit.client.1.smithi181.stdout:9/373: chown d0/f1f 2663133 1 2024-04-27T04:09:55.813 INFO:tasks.workunit.client.0.smithi162.stdout:8/85: write d0/fa [913491,73350] 0 2024-04-27T04:09:55.814 INFO:tasks.workunit.client.0.smithi162.stdout:8/86: write d0/f2 [8373201,85204] 0 2024-04-27T04:09:55.814 INFO:tasks.workunit.client.0.smithi162.stdout:8/87: stat d0 0 2024-04-27T04:09:55.815 INFO:tasks.workunit.client.1.smithi181.stdout:4/321: creat d0/d15/d48/f68 x:0 0 0 2024-04-27T04:09:55.821 INFO:tasks.workunit.client.1.smithi181.stdout:0/355: mknod dc/d1e/d26/d41/d6f/d7d/c86 0 2024-04-27T04:09:55.823 INFO:tasks.workunit.client.1.smithi181.stdout:4/322: dwrite d0/d15/de/f66 [0,4194304] 0 2024-04-27T04:09:55.826 INFO:tasks.workunit.client.1.smithi181.stdout:5/493: mknod d4/de/d2c/d46/caf 0 2024-04-27T04:09:55.828 INFO:tasks.workunit.client.0.smithi162.stdout:5/66: symlink d8/d9/l13 0 2024-04-27T04:09:55.828 INFO:tasks.workunit.client.1.smithi181.stdout:5/494: write d4/d10/d72/f77 [534655,42959] 0 2024-04-27T04:09:55.828 INFO:tasks.workunit.client.0.smithi162.stdout:5/67: chown l7 1 1 2024-04-27T04:09:55.829 INFO:tasks.workunit.client.1.smithi181.stdout:5/495: chown d4/d10/d72/l82 110834 1 2024-04-27T04:09:55.831 INFO:tasks.workunit.client.1.smithi181.stdout:3/321: unlink d0/f3 0 2024-04-27T04:09:55.833 INFO:tasks.workunit.client.1.smithi181.stdout:3/322: write d0/de/d20/f5c [811389,84712] 0 2024-04-27T04:09:55.835 INFO:tasks.workunit.client.0.smithi162.stdout:0/128: truncate d5/fa 602491 0 2024-04-27T04:09:55.835 INFO:tasks.workunit.client.0.smithi162.stdout:6/73: dread d6/d1a/f15 [0,4194304] 0 2024-04-27T04:09:55.836 INFO:tasks.workunit.client.1.smithi181.stdout:3/323: truncate d0/d6c/d52/d44/d56/f63 37282 0 2024-04-27T04:09:55.837 INFO:tasks.workunit.client.1.smithi181.stdout:4/323: dread d0/d15/d25/d2b/d7/f10 [0,4194304] 0 2024-04-27T04:09:55.839 INFO:tasks.workunit.client.1.smithi181.stdout:4/324: write d0/d15/de/d19/f5e [647637,55117] 0 2024-04-27T04:09:55.840 INFO:tasks.workunit.client.1.smithi181.stdout:0/356: mknod dc/d1e/d26/d41/c87 0 2024-04-27T04:09:55.840 INFO:tasks.workunit.client.1.smithi181.stdout:5/496: dread d4/de/d2c/d46/d4d/d8e/f8a [0,4194304] 0 2024-04-27T04:09:55.840 INFO:tasks.workunit.client.1.smithi181.stdout:3/324: truncate d0/d6c/d52/d44/d56/f63 792422 0 2024-04-27T04:09:55.841 INFO:tasks.workunit.client.0.smithi162.stdout:6/74: dwrite d6/f7 [0,4194304] 0 2024-04-27T04:09:55.844 INFO:tasks.workunit.client.1.smithi181.stdout:3/325: dread d0/f19 [0,4194304] 0 2024-04-27T04:09:55.844 INFO:tasks.workunit.client.0.smithi162.stdout:6/75: dread d6/d1a/f15 [0,4194304] 0 2024-04-27T04:09:55.846 INFO:tasks.workunit.client.1.smithi181.stdout:0/357: dwrite dc/f11 [4194304,4194304] 0 2024-04-27T04:09:55.944 INFO:tasks.workunit.client.0.smithi162.stdout:4/61: creat d2/f11 x:0 0 0 2024-04-27T04:09:55.945 INFO:tasks.workunit.client.0.smithi162.stdout:9/49: write f0 [2681091,36184] 0 2024-04-27T04:09:55.947 INFO:tasks.workunit.client.0.smithi162.stdout:2/54: mknod d0/c13 0 2024-04-27T04:09:55.951 INFO:tasks.workunit.client.0.smithi162.stdout:9/50: dwrite d3/fe [0,4194304] 0 2024-04-27T04:09:55.951 INFO:tasks.workunit.client.0.smithi162.stdout:9/51: fdatasync d3/f6 0 2024-04-27T04:09:55.952 INFO:tasks.workunit.client.1.smithi181.stdout:9/374: symlink d0/l87 0 2024-04-27T04:09:55.955 INFO:tasks.workunit.client.1.smithi181.stdout:8/450: rename d1/d6/d2f/d40/f17 to d1/d6/d2f/fa3 0 2024-04-27T04:09:55.958 INFO:tasks.workunit.client.0.smithi162.stdout:5/68: mknod d8/c14 0 2024-04-27T04:09:55.959 INFO:tasks.workunit.client.0.smithi162.stdout:0/129: creat d5/f27 x:0 0 0 2024-04-27T04:09:55.960 INFO:tasks.workunit.client.1.smithi181.stdout:3/326: mknod d0/d6c/c6d 0 2024-04-27T04:09:55.960 INFO:tasks.workunit.client.0.smithi162.stdout:9/52: dwrite d3/f4 [0,4194304] 0 2024-04-27T04:09:55.963 INFO:tasks.workunit.client.1.smithi181.stdout:0/358: unlink dc/d1f/f3d 0 2024-04-27T04:09:55.966 INFO:tasks.workunit.client.0.smithi162.stdout:9/53: dwrite d3/fe [0,4194304] 0 2024-04-27T04:09:55.967 INFO:tasks.workunit.client.0.smithi162.stdout:9/54: write d3/fb [2321263,15567] 0 2024-04-27T04:09:55.969 INFO:tasks.workunit.client.0.smithi162.stdout:7/34: fdatasync d3/d5/f8 0 2024-04-27T04:09:55.971 INFO:tasks.workunit.client.1.smithi181.stdout:7/302: getdents d1/d22 0 2024-04-27T04:09:55.972 INFO:tasks.workunit.client.1.smithi181.stdout:9/375: mkdir d0/d44/d4b/d43/d88 0 2024-04-27T04:09:55.973 INFO:tasks.workunit.client.1.smithi181.stdout:7/303: read d1/d2/f25 [1050315,65976] 0 2024-04-27T04:09:55.973 INFO:tasks.workunit.client.0.smithi162.stdout:2/55: chown d0/l12 40705 1 2024-04-27T04:09:55.974 INFO:tasks.workunit.client.1.smithi181.stdout:9/376: write d0/d44/d4b/d43/f5e [1014426,48050] 0 2024-04-27T04:09:55.975 INFO:tasks.workunit.client.1.smithi181.stdout:4/325: rename d0/d15/d25/d2b/d7 to d0/d15/d5c/d69 0 2024-04-27T04:09:55.977 INFO:tasks.workunit.client.1.smithi181.stdout:8/451: read d1/d6/d2f/fa3 [4271932,117662] 0 2024-04-27T04:09:55.978 INFO:tasks.workunit.client.1.smithi181.stdout:8/452: dread - d1/d6/d2f/d40/d2a/f64 zero size 2024-04-27T04:09:55.980 INFO:tasks.workunit.client.1.smithi181.stdout:0/359: creat dc/d1e/d26/d68/f88 x:0 0 0 2024-04-27T04:09:55.981 INFO:tasks.workunit.client.0.smithi162.stdout:3/47: creat d1/fd x:0 0 0 2024-04-27T04:09:55.982 INFO:tasks.workunit.client.0.smithi162.stdout:3/48: write d1/f3 [447261,14113] 0 2024-04-27T04:09:55.985 INFO:tasks.workunit.client.0.smithi162.stdout:3/49: fsync d1/f3 0 2024-04-27T04:09:55.985 INFO:tasks.workunit.client.0.smithi162.stdout:1/90: rmdir d0/d11/d17 0 2024-04-27T04:09:55.985 INFO:tasks.workunit.client.0.smithi162.stdout:3/50: fdatasync d1/fb 0 2024-04-27T04:09:55.986 INFO:tasks.workunit.client.0.smithi162.stdout:3/51: dread - d1/fd zero size 2024-04-27T04:09:55.986 INFO:tasks.workunit.client.0.smithi162.stdout:5/69: mknod d8/d9/c15 0 2024-04-27T04:09:55.987 INFO:tasks.workunit.client.0.smithi162.stdout:5/70: dread f5 [0,4194304] 0 2024-04-27T04:09:55.989 INFO:tasks.workunit.client.1.smithi181.stdout:7/304: fsync d1/d22/d15/f2a 0 2024-04-27T04:09:55.990 INFO:tasks.workunit.client.0.smithi162.stdout:8/88: truncate d0/f4 1682248 0 2024-04-27T04:09:55.990 INFO:tasks.workunit.client.0.smithi162.stdout:8/89: readlink d0/l5 0 2024-04-27T04:09:55.992 INFO:tasks.workunit.client.0.smithi162.stdout:8/90: write d0/fa [1116211,61664] 0 2024-04-27T04:09:55.992 INFO:tasks.workunit.client.1.smithi181.stdout:9/377: mkdir d0/d44/d4b/d43/d89 0 2024-04-27T04:09:55.993 INFO:tasks.workunit.client.0.smithi162.stdout:0/130: dwrite d5/dc/f17 [4194304,4194304] 0 2024-04-27T04:09:55.994 INFO:tasks.workunit.client.0.smithi162.stdout:6/76: truncate f5 3265019 0 2024-04-27T04:09:55.995 INFO:tasks.workunit.client.1.smithi181.stdout:5/497: creat d4/fb0 x:0 0 0 2024-04-27T04:09:55.996 INFO:tasks.workunit.client.1.smithi181.stdout:5/498: readlink d4/de/d2c/d46/d4d/d8e/d67/l64 0 2024-04-27T04:09:55.997 INFO:tasks.workunit.client.1.smithi181.stdout:5/499: chown d4/d10/d72/d80/l9b 129630660 1 2024-04-27T04:09:56.001 INFO:tasks.workunit.client.1.smithi181.stdout:8/453: mkdir d1/d20/da4 0 2024-04-27T04:09:56.004 INFO:tasks.workunit.client.1.smithi181.stdout:1/402: link d1/d4c/d1e/c57 d1/d2/d5/d6/db/d2d/d36/c7c 0 2024-04-27T04:09:56.006 INFO:tasks.workunit.client.1.smithi181.stdout:1/403: write d1/d2/d5/f45 [2300095,104233] 0 2024-04-27T04:09:56.006 INFO:tasks.workunit.client.0.smithi162.stdout:2/56: mknod d0/d4/c14 0 2024-04-27T04:09:56.011 INFO:tasks.workunit.client.1.smithi181.stdout:7/305: mkdir d1/d22/d15/d5f 0 2024-04-27T04:09:56.013 INFO:tasks.workunit.client.1.smithi181.stdout:9/378: rmdir d0/d44/d4b 39 2024-04-27T04:09:56.014 INFO:tasks.workunit.client.1.smithi181.stdout:9/379: fdatasync d0/d40/f86 0 2024-04-27T04:09:56.016 INFO:tasks.workunit.client.0.smithi162.stdout:3/52: rmdir d1 39 2024-04-27T04:09:56.018 INFO:tasks.workunit.client.1.smithi181.stdout:5/500: dread d4/d10/d72/f77 [0,4194304] 0 2024-04-27T04:09:56.019 INFO:tasks.workunit.client.0.smithi162.stdout:5/71: creat d8/d9/f16 x:0 0 0 2024-04-27T04:09:56.022 INFO:tasks.workunit.client.0.smithi162.stdout:1/91: write d0/f14 [3807202,38294] 0 2024-04-27T04:09:56.025 INFO:tasks.workunit.client.1.smithi181.stdout:0/360: truncate dc/d1e/d26/f6c 598971 0 2024-04-27T04:09:56.026 INFO:tasks.workunit.client.1.smithi181.stdout:0/361: write dc/f2c [1888082,94626] 0 2024-04-27T04:09:56.027 INFO:tasks.workunit.client.0.smithi162.stdout:6/77: rmdir d6 39 2024-04-27T04:09:56.030 INFO:tasks.workunit.client.1.smithi181.stdout:1/404: creat d1/d2/d5/d6/d6a/f7d x:0 0 0 2024-04-27T04:09:56.030 INFO:tasks.workunit.client.1.smithi181.stdout:1/405: readlink d1/l18 0 2024-04-27T04:09:56.031 INFO:tasks.workunit.client.0.smithi162.stdout:7/35: symlink d3/lb 0 2024-04-27T04:09:56.033 INFO:tasks.workunit.client.0.smithi162.stdout:7/36: dread d3/d5/f8 [0,4194304] 0 2024-04-27T04:09:56.034 INFO:tasks.workunit.client.0.smithi162.stdout:7/37: dread - d3/d5/fa zero size 2024-04-27T04:09:56.035 INFO:tasks.workunit.client.1.smithi181.stdout:7/306: rename d1/d22/d15/d13/d1b/f33 to d1/d22/f60 0 2024-04-27T04:09:56.035 INFO:tasks.workunit.client.1.smithi181.stdout:7/307: write d1/d2/f12 [9476150,44878] 0 2024-04-27T04:09:56.036 INFO:tasks.workunit.client.0.smithi162.stdout:2/57: creat d0/d4/f15 x:0 0 0 2024-04-27T04:09:56.037 INFO:tasks.workunit.client.1.smithi181.stdout:9/380: write d0/d44/d4b/f39 [2324720,32260] 0 2024-04-27T04:09:56.037 INFO:tasks.workunit.client.1.smithi181.stdout:4/326: creat d0/d15/f6a x:0 0 0 2024-04-27T04:09:56.038 INFO:tasks.workunit.client.1.smithi181.stdout:9/381: write d0/d44/f7c [441122,92073] 0 2024-04-27T04:09:56.039 INFO:tasks.workunit.client.1.smithi181.stdout:9/382: fsync d0/d44/d4f/f54 0 2024-04-27T04:09:56.040 INFO:tasks.workunit.client.1.smithi181.stdout:4/327: read - d0/d15/d25/d2a/f38 zero size 2024-04-27T04:09:56.041 INFO:tasks.workunit.client.1.smithi181.stdout:3/327: getdents d0/de/d3f 0 2024-04-27T04:09:56.043 INFO:tasks.workunit.client.0.smithi162.stdout:8/91: creat d0/de/d17/f19 x:0 0 0 2024-04-27T04:09:56.044 INFO:tasks.workunit.client.0.smithi162.stdout:1/92: fdatasync d0/fb 0 2024-04-27T04:09:56.048 INFO:tasks.workunit.client.0.smithi162.stdout:4/62: getdents d2/da 0 2024-04-27T04:09:56.051 INFO:tasks.workunit.client.0.smithi162.stdout:7/38: unlink d3/l4 0 2024-04-27T04:09:56.052 INFO:tasks.workunit.client.0.smithi162.stdout:7/39: dread - d3/d5/fa zero size 2024-04-27T04:09:56.053 INFO:tasks.workunit.client.1.smithi181.stdout:9/383: dread d0/f2b [0,4194304] 0 2024-04-27T04:09:56.053 INFO:tasks.workunit.client.0.smithi162.stdout:7/40: chown d3 54928 1 2024-04-27T04:09:56.054 INFO:tasks.workunit.client.0.smithi162.stdout:4/63: dwrite d2/d5/f8 [0,4194304] 0 2024-04-27T04:09:56.058 INFO:tasks.workunit.client.1.smithi181.stdout:5/501: mknod d4/de/d9d/cb1 0 2024-04-27T04:09:56.058 INFO:tasks.workunit.client.1.smithi181.stdout:7/308: dread d1/f45 [0,4194304] 0 2024-04-27T04:09:56.060 INFO:tasks.workunit.client.0.smithi162.stdout:5/72: creat d8/d11/f17 x:0 0 0 2024-04-27T04:09:56.061 INFO:tasks.workunit.client.0.smithi162.stdout:2/58: dwrite d0/fa [0,4194304] 0 2024-04-27T04:09:56.061 INFO:tasks.workunit.client.0.smithi162.stdout:2/59: chown d0/d4 52 1 2024-04-27T04:09:56.065 INFO:tasks.workunit.client.0.smithi162.stdout:1/93: unlink d0/d7/l9 0 2024-04-27T04:09:56.067 INFO:tasks.workunit.client.0.smithi162.stdout:1/94: chown d0/l3 5 1 2024-04-27T04:09:56.067 INFO:tasks.workunit.client.0.smithi162.stdout:0/131: mknod d5/dc/d1e/c28 0 2024-04-27T04:09:56.067 INFO:tasks.workunit.client.1.smithi181.stdout:8/454: link d1/d20/l50 d1/d8c/la5 0 2024-04-27T04:09:56.068 INFO:tasks.workunit.client.1.smithi181.stdout:8/455: read - d1/d6/d2f/d47/f4b zero size 2024-04-27T04:09:56.069 INFO:tasks.workunit.client.0.smithi162.stdout:5/73: dwrite d8/d9/f16 [0,4194304] 0 2024-04-27T04:09:56.069 INFO:tasks.workunit.client.1.smithi181.stdout:8/456: fsync d1/d6/d2f/d30/d66/f81 0 2024-04-27T04:09:56.070 INFO:tasks.workunit.client.1.smithi181.stdout:8/457: fsync d1/f48 0 2024-04-27T04:09:56.071 INFO:tasks.workunit.client.0.smithi162.stdout:9/55: link d3/cf d3/c10 0 2024-04-27T04:09:56.072 INFO:tasks.workunit.client.0.smithi162.stdout:9/56: write d3/fe [3543150,65949] 0 2024-04-27T04:09:56.072 INFO:tasks.workunit.client.1.smithi181.stdout:0/362: rename dc/c1a to dc/d1e/d26/d41/d6f/c89 0 2024-04-27T04:09:56.073 INFO:tasks.workunit.client.0.smithi162.stdout:9/57: chown d3/fa 59 1 2024-04-27T04:09:56.073 INFO:tasks.workunit.client.0.smithi162.stdout:1/95: dread d0/d7/d10/f12 [0,4194304] 0 2024-04-27T04:09:56.074 INFO:tasks.workunit.client.1.smithi181.stdout:0/363: dread - dc/d1e/f4d zero size 2024-04-27T04:09:56.074 INFO:tasks.workunit.client.1.smithi181.stdout:0/364: chown dc/f39 5789 1 2024-04-27T04:09:56.076 INFO:tasks.workunit.client.1.smithi181.stdout:9/384: mkdir d0/d44/d4b/d5f/d8a 0 2024-04-27T04:09:56.078 INFO:tasks.workunit.client.0.smithi162.stdout:4/64: creat d2/da/f12 x:0 0 0 2024-04-27T04:09:56.080 INFO:tasks.workunit.client.1.smithi181.stdout:9/385: dread d0/f8 [0,4194304] 0 2024-04-27T04:09:56.082 INFO:tasks.workunit.client.1.smithi181.stdout:9/386: write d0/d44/f48 [4750148,40193] 0 2024-04-27T04:09:56.083 INFO:tasks.workunit.client.0.smithi162.stdout:3/53: symlink d1/le 0 2024-04-27T04:09:56.084 INFO:tasks.workunit.client.0.smithi162.stdout:4/65: dwrite d2/da/dd/fe [0,4194304] 0 2024-04-27T04:09:56.085 INFO:tasks.workunit.client.0.smithi162.stdout:2/60: creat d0/d4/f16 x:0 0 0 2024-04-27T04:09:56.086 INFO:tasks.workunit.client.1.smithi181.stdout:3/328: write d0/f16 [872567,31999] 0 2024-04-27T04:09:56.087 INFO:tasks.workunit.client.1.smithi181.stdout:3/329: chown d0/de/d20/f69 7 1 2024-04-27T04:09:56.092 INFO:tasks.workunit.client.1.smithi181.stdout:8/458: mkdir d1/d6/da6 0 2024-04-27T04:09:56.093 INFO:tasks.workunit.client.0.smithi162.stdout:0/132: dread d5/fa [0,4194304] 0 2024-04-27T04:09:56.094 INFO:tasks.workunit.client.1.smithi181.stdout:0/365: mknod dc/d1e/d26/d41/d6f/c8a 0 2024-04-27T04:09:56.096 INFO:tasks.workunit.client.0.smithi162.stdout:5/74: creat d8/d9/f18 x:0 0 0 2024-04-27T04:09:56.097 INFO:tasks.workunit.client.0.smithi162.stdout:5/75: chown d8/d9/f16 52779 1 2024-04-27T04:09:56.098 INFO:tasks.workunit.client.0.smithi162.stdout:8/92: dwrite d0/f4 [0,4194304] 0 2024-04-27T04:09:56.099 INFO:tasks.workunit.client.1.smithi181.stdout:5/502: mkdir d4/de/d2c/d46/d99/dad/db2 0 2024-04-27T04:09:56.100 INFO:tasks.workunit.client.0.smithi162.stdout:8/93: dread - d0/de/d17/f19 zero size 2024-04-27T04:09:56.100 INFO:tasks.workunit.client.0.smithi162.stdout:9/58: mkdir d3/d11 0 2024-04-27T04:09:56.101 INFO:tasks.workunit.client.1.smithi181.stdout:0/366: dwrite dc/f18 [0,4194304] 0 2024-04-27T04:09:56.102 INFO:tasks.workunit.client.0.smithi162.stdout:7/41: mkdir d3/d5/d9/dc 0 2024-04-27T04:09:56.104 INFO:tasks.workunit.client.0.smithi162.stdout:3/54: symlink d1/d2/lf 0 2024-04-27T04:09:56.106 INFO:tasks.workunit.client.0.smithi162.stdout:6/78: dwrite d6/f17 [0,4194304] 0 2024-04-27T04:09:56.109 INFO:tasks.workunit.client.0.smithi162.stdout:4/66: creat d2/da/f13 x:0 0 0 2024-04-27T04:09:56.109 INFO:tasks.workunit.client.0.smithi162.stdout:6/79: write d6/d1a/f10 [4345024,96287] 0 2024-04-27T04:09:56.111 INFO:tasks.workunit.client.0.smithi162.stdout:6/80: write d6/db/f1c [906084,67020] 0 2024-04-27T04:09:56.111 INFO:tasks.workunit.client.0.smithi162.stdout:2/61: mkdir d0/d4/d17 0 2024-04-27T04:09:56.111 INFO:tasks.workunit.client.1.smithi181.stdout:9/387: dwrite d0/d44/d4b/d43/f82 [0,4194304] 0 2024-04-27T04:09:56.112 INFO:tasks.workunit.client.0.smithi162.stdout:6/81: fdatasync d6/f7 0 2024-04-27T04:09:56.113 INFO:tasks.workunit.client.1.smithi181.stdout:9/388: readlink d0/l87 0 2024-04-27T04:09:56.115 INFO:tasks.workunit.client.0.smithi162.stdout:0/133: read d5/dc/f18 [21633,107662] 0 2024-04-27T04:09:56.118 INFO:tasks.workunit.client.1.smithi181.stdout:5/503: mkdir d4/d10/d72/db3 0 2024-04-27T04:09:56.120 INFO:tasks.workunit.client.1.smithi181.stdout:0/367: write dc/f1b [1823034,27369] 0 2024-04-27T04:09:56.123 INFO:tasks.workunit.client.1.smithi181.stdout:9/389: dwrite d0/d44/d4b/d43/f5e [0,4194304] 0 2024-04-27T04:09:56.126 INFO:tasks.workunit.client.1.smithi181.stdout:9/390: chown d0/d40/f22 33084363 1 2024-04-27T04:09:56.126 INFO:tasks.workunit.client.1.smithi181.stdout:4/328: getdents d0/d15/d25/d2a/d36 0 2024-04-27T04:09:56.126 INFO:tasks.workunit.client.0.smithi162.stdout:8/94: truncate d0/fd 873005 0 2024-04-27T04:09:56.129 INFO:tasks.workunit.client.0.smithi162.stdout:8/95: write d0/d12/f15 [662226,118820] 0 2024-04-27T04:09:56.131 INFO:tasks.workunit.client.0.smithi162.stdout:3/55: creat d1/d2/f10 x:0 0 0 2024-04-27T04:09:56.133 INFO:tasks.workunit.client.0.smithi162.stdout:3/56: write d1/d2/f9 [735528,22937] 0 2024-04-27T04:09:56.134 INFO:tasks.workunit.client.1.smithi181.stdout:5/504: dwrite d4/d10/d72/d80/f8d [0,4194304] 0 2024-04-27T04:09:56.137 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:09:55 smithi162 ceph-mon[26841]: pgmap v263: 65 pgs: 65 active+clean; 790 MiB data, 4.0 GiB used, 532 GiB / 536 GiB avail; 9.6 MiB/s rd, 89 MiB/s wr, 283 op/s 2024-04-27T04:09:56.138 INFO:tasks.workunit.client.1.smithi181.stdout:9/391: dread d0/d44/d4b/d43/f82 [0,4194304] 0 2024-04-27T04:09:56.140 INFO:tasks.workunit.client.0.smithi162.stdout:4/67: dread d2/f4 [0,4194304] 0 2024-04-27T04:09:56.140 INFO:tasks.workunit.client.1.smithi181.stdout:9/392: write d0/d40/f86 [737717,54868] 0 2024-04-27T04:09:56.141 INFO:tasks.workunit.client.0.smithi162.stdout:4/68: write d2/d5/fb [2566063,97158] 0 2024-04-27T04:09:56.142 INFO:tasks.workunit.client.0.smithi162.stdout:4/69: chown d2/da/f12 156861 1 2024-04-27T04:09:56.145 INFO:tasks.workunit.client.0.smithi162.stdout:6/82: creat d6/d1a/f1d x:0 0 0 2024-04-27T04:09:56.146 INFO:tasks.workunit.client.0.smithi162.stdout:6/83: write f2 [2490453,43161] 0 2024-04-27T04:09:56.148 INFO:tasks.workunit.client.1.smithi181.stdout:4/329: symlink d0/d15/d48/d57/l6b 0 2024-04-27T04:09:56.172 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:09:55 smithi181 ceph-mon[33774]: pgmap v263: 65 pgs: 65 active+clean; 790 MiB data, 4.0 GiB used, 532 GiB / 536 GiB avail; 9.6 MiB/s rd, 89 MiB/s wr, 283 op/s 2024-04-27T04:09:56.193 INFO:tasks.workunit.client.1.smithi181.stdout:7/309: truncate d1/d2/d1c/d2d/d3d/f51 647247 0 2024-04-27T04:09:56.193 INFO:tasks.workunit.client.1.smithi181.stdout:0/368: dread dc/d1e/f30 [0,4194304] 0 2024-04-27T04:09:56.194 INFO:tasks.workunit.client.1.smithi181.stdout:4/330: creat d0/d15/d25/d40/f6c x:0 0 0 2024-04-27T04:09:56.196 INFO:tasks.workunit.client.1.smithi181.stdout:3/330: getdents d0/d28 0 2024-04-27T04:09:56.199 INFO:tasks.workunit.client.1.smithi181.stdout:9/393: rename d0/d40/f86 to d0/f8b 0 2024-04-27T04:09:56.200 INFO:tasks.workunit.client.1.smithi181.stdout:0/369: symlink dc/d1e/d6d/l8b 0 2024-04-27T04:09:56.200 INFO:tasks.workunit.client.0.smithi162.stdout:9/59: mknod d3/d11/c12 0 2024-04-27T04:09:56.201 INFO:tasks.workunit.client.0.smithi162.stdout:2/62: symlink d0/d4/d17/l18 0 2024-04-27T04:09:56.201 INFO:tasks.workunit.client.1.smithi181.stdout:3/331: truncate d0/f4e 535386 0 2024-04-27T04:09:56.202 INFO:tasks.workunit.client.0.smithi162.stdout:5/76: dwrite d8/f12 [8388608,4194304] 0 2024-04-27T04:09:56.202 INFO:tasks.workunit.client.1.smithi181.stdout:7/310: mkdir d1/d2/d1c/d2d/d61 0 2024-04-27T04:09:56.203 INFO:tasks.workunit.client.0.smithi162.stdout:5/77: write f4 [2964268,88231] 0 2024-04-27T04:09:56.204 INFO:tasks.workunit.client.1.smithi181.stdout:5/505: getdents d4/de/d2c/d46/d4d/d8e/d67 0 2024-04-27T04:09:56.207 INFO:tasks.workunit.client.1.smithi181.stdout:4/331: mkdir d0/d15/d25/d2b/d6d 0 2024-04-27T04:09:56.209 INFO:tasks.workunit.client.1.smithi181.stdout:4/332: write d0/d15/d25/d2b/f1c [3516673,73712] 0 2024-04-27T04:09:56.209 INFO:tasks.workunit.client.0.smithi162.stdout:7/42: truncate f0 3784361 0 2024-04-27T04:09:56.210 INFO:tasks.workunit.client.1.smithi181.stdout:4/333: chown d0/d15/d5c/d69/c3a 6760833 1 2024-04-27T04:09:56.212 INFO:tasks.workunit.client.0.smithi162.stdout:6/84: unlink d6/d1a/f10 0 2024-04-27T04:09:56.215 INFO:tasks.workunit.client.1.smithi181.stdout:9/394: chown d0/d40/c38 300168 1 2024-04-27T04:09:56.215 INFO:tasks.workunit.client.0.smithi162.stdout:6/85: dread d6/f7 [0,4194304] 0 2024-04-27T04:09:56.216 INFO:tasks.workunit.client.0.smithi162.stdout:9/60: mkdir d3/d13 0 2024-04-27T04:09:56.216 INFO:tasks.workunit.client.1.smithi181.stdout:9/395: truncate d0/d44/d4b/f62 1706746 0 2024-04-27T04:09:56.216 INFO:tasks.workunit.client.0.smithi162.stdout:4/70: write d2/f4 [4843743,124235] 0 2024-04-27T04:09:56.220 INFO:tasks.workunit.client.1.smithi181.stdout:3/332: mkdir d0/d6c/d6e 0 2024-04-27T04:09:56.222 INFO:tasks.workunit.client.0.smithi162.stdout:4/71: fsync d2/f4 0 2024-04-27T04:09:56.223 INFO:tasks.workunit.client.1.smithi181.stdout:7/311: mkdir d1/df/d62 0 2024-04-27T04:09:56.223 INFO:tasks.workunit.client.0.smithi162.stdout:4/72: dread - d2/ff zero size 2024-04-27T04:09:56.223 INFO:tasks.workunit.client.0.smithi162.stdout:4/73: fdatasync d2/f11 0 2024-04-27T04:09:56.225 INFO:tasks.workunit.client.0.smithi162.stdout:4/74: read d2/f9 [153172,101461] 0 2024-04-27T04:09:56.226 INFO:tasks.workunit.client.1.smithi181.stdout:4/334: symlink d0/d15/d25/d40/l6e 0 2024-04-27T04:09:56.228 INFO:tasks.workunit.client.1.smithi181.stdout:3/333: dwrite d0/de/f34 [0,4194304] 0 2024-04-27T04:09:56.230 INFO:tasks.workunit.client.1.smithi181.stdout:3/334: dread - d0/d6c/d52/f68 zero size 2024-04-27T04:09:56.230 INFO:tasks.workunit.client.1.smithi181.stdout:4/335: dread d0/d15/de/f43 [0,4194304] 0 2024-04-27T04:09:56.233 INFO:tasks.workunit.client.1.smithi181.stdout:5/506: dwrite d4/de/d2c/d46/d4d/d8e/f8a [0,4194304] 0 2024-04-27T04:09:56.234 INFO:tasks.workunit.client.1.smithi181.stdout:4/336: stat d0/d15/d25/l4d 0 2024-04-27T04:09:56.234 INFO:tasks.workunit.client.0.smithi162.stdout:8/96: truncate d0/d12/f14 402398 0 2024-04-27T04:09:56.234 INFO:tasks.workunit.client.0.smithi162.stdout:8/97: fsync d0/f2 0 2024-04-27T04:09:56.237 INFO:tasks.workunit.client.0.smithi162.stdout:6/86: unlink d6/db/f1c 0 2024-04-27T04:09:56.241 INFO:tasks.workunit.client.1.smithi181.stdout:7/312: creat d1/d2/d1c/d56/f63 x:0 0 0 2024-04-27T04:09:56.241 INFO:tasks.workunit.client.1.smithi181.stdout:5/507: dwrite d4/d10/f57 [0,4194304] 0 2024-04-27T04:09:56.242 INFO:tasks.workunit.client.0.smithi162.stdout:1/96: sync 2024-04-27T04:09:56.242 INFO:tasks.workunit.client.1.smithi181.stdout:4/337: dread d0/d15/f2f [0,4194304] 0 2024-04-27T04:09:56.244 INFO:tasks.workunit.client.0.smithi162.stdout:9/61: creat d3/f14 x:0 0 0 2024-04-27T04:09:56.251 INFO:tasks.workunit.client.1.smithi181.stdout:7/313: dwrite d1/d22/d15/d13/f46 [0,4194304] 0 2024-04-27T04:09:56.256 INFO:tasks.workunit.client.1.smithi181.stdout:0/370: dwrite dc/f10 [0,4194304] 0 2024-04-27T04:09:56.256 INFO:tasks.workunit.client.0.smithi162.stdout:0/134: mknod d5/dc/d1e/d20/c29 0 2024-04-27T04:09:56.264 INFO:tasks.workunit.client.1.smithi181.stdout:7/314: dwrite d1/d2/d1c/d37/f4f [0,4194304] 0 2024-04-27T04:09:56.271 INFO:tasks.workunit.client.1.smithi181.stdout:7/315: dread d1/d22/d15/d13/f46 [0,4194304] 0 2024-04-27T04:09:56.271 INFO:tasks.workunit.client.1.smithi181.stdout:7/316: chown d1/d22/c44 2395307 1 2024-04-27T04:09:56.276 INFO:tasks.workunit.client.1.smithi181.stdout:9/396: link d0/d44/d4b/l60 d0/d44/d4b/d3d/l8c 0 2024-04-27T04:09:56.277 INFO:tasks.workunit.client.1.smithi181.stdout:9/397: chown d0/d44/d4b/l41 21140076 1 2024-04-27T04:09:56.278 INFO:tasks.workunit.client.0.smithi162.stdout:6/87: chown d6/c1b 56 1 2024-04-27T04:09:56.280 INFO:tasks.workunit.client.1.smithi181.stdout:3/335: symlink d0/d6c/d52/d44/d56/d5b/l6f 0 2024-04-27T04:09:56.281 INFO:tasks.workunit.client.0.smithi162.stdout:1/97: mknod d0/d11/c18 0 2024-04-27T04:09:56.283 INFO:tasks.workunit.client.0.smithi162.stdout:9/62: rmdir d3/d11 39 2024-04-27T04:09:56.286 INFO:tasks.workunit.client.1.smithi181.stdout:9/398: dwrite d0/d44/d4b/f39 [0,4194304] 0 2024-04-27T04:09:56.290 INFO:tasks.workunit.client.0.smithi162.stdout:2/63: link d0/c13 d0/d4/d17/c19 0 2024-04-27T04:09:56.291 INFO:tasks.workunit.client.0.smithi162.stdout:5/78: truncate d8/d9/f10 359487 0 2024-04-27T04:09:56.291 INFO:tasks.workunit.client.1.smithi181.stdout:5/508: dread d4/d10/d72/d80/d85/fae [0,4194304] 0 2024-04-27T04:09:56.291 INFO:tasks.workunit.client.1.smithi181.stdout:3/336: dwrite d0/d6c/d52/f59 [0,4194304] 0 2024-04-27T04:09:56.292 INFO:tasks.workunit.client.0.smithi162.stdout:0/135: creat d5/dc/d1e/d20/f2a x:0 0 0 2024-04-27T04:09:56.292 INFO:tasks.workunit.client.1.smithi181.stdout:5/509: stat d4/l32 0 2024-04-27T04:09:56.295 INFO:tasks.workunit.client.1.smithi181.stdout:4/338: mkdir d0/d15/de/d60/d6f 0 2024-04-27T04:09:56.296 INFO:tasks.workunit.client.0.smithi162.stdout:7/43: write f0 [1419762,64055] 0 2024-04-27T04:09:56.298 INFO:tasks.workunit.client.0.smithi162.stdout:4/75: dwrite d2/da/fc [0,4194304] 0 2024-04-27T04:09:56.298 INFO:tasks.workunit.client.0.smithi162.stdout:4/76: stat d2/d5/fb 0 2024-04-27T04:09:56.300 INFO:tasks.workunit.client.1.smithi181.stdout:7/317: dwrite d1/d2/f19 [0,4194304] 0 2024-04-27T04:09:56.305 INFO:tasks.workunit.client.0.smithi162.stdout:1/98: creat d0/d7/f19 x:0 0 0 2024-04-27T04:09:56.307 INFO:tasks.workunit.client.1.smithi181.stdout:7/318: dread d1/d22/f60 [0,4194304] 0 2024-04-27T04:09:56.308 INFO:tasks.workunit.client.0.smithi162.stdout:3/57: sync 2024-04-27T04:09:56.310 INFO:tasks.workunit.client.1.smithi181.stdout:3/337: mkdir d0/de/d20/d58/d70 0 2024-04-27T04:09:56.311 INFO:tasks.workunit.client.0.smithi162.stdout:3/58: dread d1/d2/f6 [0,4194304] 0 2024-04-27T04:09:56.313 INFO:tasks.workunit.client.1.smithi181.stdout:5/510: creat d4/de/d2c/d46/d4d/d8e/d67/d71/fb4 x:0 0 0 2024-04-27T04:09:56.314 INFO:tasks.workunit.client.1.smithi181.stdout:5/511: chown d4/l3b 21562808 1 2024-04-27T04:09:56.318 INFO:tasks.workunit.client.1.smithi181.stdout:9/399: creat d0/d4d/d85/f8d x:0 0 0 2024-04-27T04:09:56.319 INFO:tasks.workunit.client.1.smithi181.stdout:7/319: mknod d1/d2/d1c/c64 0 2024-04-27T04:09:56.321 INFO:tasks.workunit.client.0.smithi162.stdout:6/88: write d6/fc [3428671,5851] 0 2024-04-27T04:09:56.322 INFO:tasks.workunit.client.1.smithi181.stdout:4/339: symlink d0/d15/d25/d2b/l70 0 2024-04-27T04:09:56.325 INFO:tasks.workunit.client.1.smithi181.stdout:0/371: getdents dc/d1e/d26/d41/d6f 0 2024-04-27T04:09:56.326 INFO:tasks.workunit.client.1.smithi181.stdout:9/400: creat d0/d4d/d85/f8e x:0 0 0 2024-04-27T04:09:56.327 INFO:tasks.workunit.client.1.smithi181.stdout:9/401: dread - d0/d44/d4b/d43/f7d zero size 2024-04-27T04:09:56.327 INFO:tasks.workunit.client.1.smithi181.stdout:7/320: creat d1/d22/f65 x:0 0 0 2024-04-27T04:09:56.330 INFO:tasks.workunit.client.1.smithi181.stdout:5/512: mkdir d4/de/d95/db5 0 2024-04-27T04:09:56.332 INFO:tasks.workunit.client.1.smithi181.stdout:0/372: dwrite dc/d1f/f48 [0,4194304] 0 2024-04-27T04:09:56.334 INFO:tasks.workunit.client.0.smithi162.stdout:8/98: sync 2024-04-27T04:09:56.337 INFO:tasks.workunit.client.1.smithi181.stdout:3/338: creat d0/f71 x:0 0 0 2024-04-27T04:09:56.340 INFO:tasks.workunit.client.1.smithi181.stdout:9/402: chown d0/f1e 10696 1 2024-04-27T04:09:56.340 INFO:tasks.workunit.client.1.smithi181.stdout:4/340: dread d0/d15/d25/d2b/f6 [0,4194304] 0 2024-04-27T04:09:56.347 INFO:tasks.workunit.client.1.smithi181.stdout:9/403: dwrite d0/fc [4194304,4194304] 0 2024-04-27T04:09:56.348 INFO:tasks.workunit.client.1.smithi181.stdout:0/373: creat dc/d1f/d3b/d75/f8c x:0 0 0 2024-04-27T04:09:56.348 INFO:tasks.workunit.client.1.smithi181.stdout:9/404: readlink d0/d44/d4b/d50/l74 0 2024-04-27T04:09:56.357 INFO:tasks.workunit.client.1.smithi181.stdout:4/341: rename d0/d15/d25/d2b/f14 to d0/d15/d25/d2b/d6d/f71 0 2024-04-27T04:09:56.358 INFO:tasks.workunit.client.1.smithi181.stdout:5/513: creat d4/d10/d72/db3/fb6 x:0 0 0 2024-04-27T04:09:56.359 INFO:tasks.workunit.client.1.smithi181.stdout:0/374: mknod dc/d1e/c8d 0 2024-04-27T04:09:56.360 INFO:tasks.workunit.client.1.smithi181.stdout:9/405: mkdir d0/d44/d4b/d43/d8f 0 2024-04-27T04:09:56.362 INFO:tasks.workunit.client.1.smithi181.stdout:3/339: mknod d0/c72 0 2024-04-27T04:09:56.365 INFO:tasks.workunit.client.1.smithi181.stdout:4/342: fdatasync d0/d15/d25/d2a/d4c/f4e 0 2024-04-27T04:09:56.366 INFO:tasks.workunit.client.1.smithi181.stdout:3/340: dread d0/de/d20/f69 [0,4194304] 0 2024-04-27T04:09:56.367 INFO:tasks.workunit.client.1.smithi181.stdout:4/343: readlink d0/d15/de/d19/l54 0 2024-04-27T04:09:56.367 INFO:tasks.workunit.client.1.smithi181.stdout:5/514: symlink d4/d10/d72/lb7 0 2024-04-27T04:09:56.369 INFO:tasks.workunit.client.1.smithi181.stdout:5/515: dread - d4/de/d2c/d46/d4d/d8e/d67/d71/f7b zero size 2024-04-27T04:09:56.371 INFO:tasks.workunit.client.1.smithi181.stdout:9/406: creat d0/d4d/d85/f90 x:0 0 0 2024-04-27T04:09:56.371 INFO:tasks.workunit.client.1.smithi181.stdout:9/407: read - d0/d4d/d85/f8d zero size 2024-04-27T04:09:56.375 INFO:tasks.workunit.client.1.smithi181.stdout:4/344: creat d0/d15/d48/f72 x:0 0 0 2024-04-27T04:09:56.377 INFO:tasks.workunit.client.1.smithi181.stdout:0/375: symlink dc/l8e 0 2024-04-27T04:09:56.380 INFO:tasks.workunit.client.1.smithi181.stdout:5/516: creat d4/de/d2c/d46/d4d/fb8 x:0 0 0 2024-04-27T04:09:56.380 INFO:tasks.workunit.client.1.smithi181.stdout:7/321: write d1/d22/d15/f2a [2013837,129167] 0 2024-04-27T04:09:56.382 INFO:tasks.workunit.client.1.smithi181.stdout:4/345: dwrite d0/d15/d25/d2b/f2c [0,4194304] 0 2024-04-27T04:09:56.384 INFO:tasks.workunit.client.1.smithi181.stdout:4/346: readlink d0/d15/d25/l26 0 2024-04-27T04:09:56.388 INFO:tasks.workunit.client.1.smithi181.stdout:0/376: fdatasync dc/d1f/f29 0 2024-04-27T04:09:56.393 INFO:tasks.workunit.client.1.smithi181.stdout:5/517: symlink d4/de/d2c/d2e/d55/lb9 0 2024-04-27T04:09:56.395 INFO:tasks.workunit.client.1.smithi181.stdout:4/347: fdatasync d0/d15/d25/d40/f4f 0 2024-04-27T04:09:56.401 INFO:tasks.workunit.client.1.smithi181.stdout:5/518: fsync d4/de/d2f/d7a/f89 0 2024-04-27T04:09:56.403 INFO:tasks.workunit.client.1.smithi181.stdout:5/519: fdatasync d4/de/d2c/d46/d4d/d8e/d67/d71/fb4 0 2024-04-27T04:09:56.408 INFO:tasks.workunit.client.1.smithi181.stdout:4/348: creat d0/d15/de/d19/d59/f73 x:0 0 0 2024-04-27T04:09:56.408 INFO:tasks.workunit.client.1.smithi181.stdout:9/408: rmdir d0/d4d 39 2024-04-27T04:09:56.411 INFO:tasks.workunit.client.1.smithi181.stdout:9/409: fsync d0/d4d/d85/f90 0 2024-04-27T04:09:56.412 INFO:tasks.workunit.client.1.smithi181.stdout:4/349: chown d0/d15/d25/d2b/c65 0 1 2024-04-27T04:09:56.416 INFO:tasks.workunit.client.1.smithi181.stdout:3/341: write d0/f3c [3293631,69136] 0 2024-04-27T04:09:56.418 INFO:tasks.workunit.client.1.smithi181.stdout:4/350: rename d0/d15/d25/d2b/f67 to d0/d15/d25/d2b/d6d/f74 0 2024-04-27T04:09:56.419 INFO:tasks.workunit.client.1.smithi181.stdout:5/520: link d4/de/c22 d4/de/d2c/d46/d4d/d8e/d67/d71/cba 0 2024-04-27T04:09:56.420 INFO:tasks.workunit.client.1.smithi181.stdout:5/521: readlink d4/d10/d72/d80/l9b 0 2024-04-27T04:09:56.421 INFO:tasks.workunit.client.1.smithi181.stdout:5/522: chown d4/de/f6e 3 1 2024-04-27T04:09:56.421 INFO:tasks.workunit.client.1.smithi181.stdout:9/410: mkdir d0/d91 0 2024-04-27T04:09:56.423 INFO:tasks.workunit.client.1.smithi181.stdout:5/523: readlink d4/de/d2c/d2e/d55/l44 0 2024-04-27T04:09:56.424 INFO:tasks.workunit.client.1.smithi181.stdout:5/524: readlink d4/de/d2f/l76 0 2024-04-27T04:09:56.425 INFO:tasks.workunit.client.1.smithi181.stdout:7/322: dwrite d1/d22/f26 [0,4194304] 0 2024-04-27T04:09:56.426 INFO:tasks.workunit.client.1.smithi181.stdout:4/351: symlink d0/d15/d25/d2b/d6d/l75 0 2024-04-27T04:09:56.428 INFO:tasks.workunit.client.1.smithi181.stdout:3/342: dread d0/d6c/d52/f65 [0,4194304] 0 2024-04-27T04:09:56.428 INFO:tasks.workunit.client.1.smithi181.stdout:7/323: write d1/d2/f24 [3718998,127029] 0 2024-04-27T04:09:56.428 INFO:tasks.workunit.client.1.smithi181.stdout:9/411: write d0/f2b [489567,3676] 0 2024-04-27T04:09:56.429 INFO:tasks.workunit.client.1.smithi181.stdout:3/343: fsync d0/de/d20/f5d 0 2024-04-27T04:09:56.438 INFO:tasks.workunit.client.1.smithi181.stdout:4/352: fdatasync d0/d15/de/f43 0 2024-04-27T04:09:56.438 INFO:tasks.workunit.client.1.smithi181.stdout:9/412: dwrite d0/fc [0,4194304] 0 2024-04-27T04:09:56.439 INFO:tasks.workunit.client.1.smithi181.stdout:7/324: dwrite d1/d22/d15/f2a [0,4194304] 0 2024-04-27T04:09:56.451 INFO:tasks.workunit.client.1.smithi181.stdout:5/525: mkdir d4/d10/d29/d92/dbb 0 2024-04-27T04:09:56.453 INFO:tasks.workunit.client.1.smithi181.stdout:3/344: mknod d0/d28/c73 0 2024-04-27T04:09:56.453 INFO:tasks.workunit.client.1.smithi181.stdout:3/345: stat d0/f3c 0 2024-04-27T04:09:56.461 INFO:tasks.workunit.client.1.smithi181.stdout:7/325: creat d1/d2/d1c/d55/f66 x:0 0 0 2024-04-27T04:09:56.465 INFO:tasks.workunit.client.1.smithi181.stdout:9/413: dwrite d0/d44/f51 [0,4194304] 0 2024-04-27T04:09:56.467 INFO:tasks.workunit.client.1.smithi181.stdout:3/346: mkdir d0/d6c/d52/d44/d74 0 2024-04-27T04:09:56.468 INFO:tasks.workunit.client.1.smithi181.stdout:7/326: dread d1/d22/f26 [0,4194304] 0 2024-04-27T04:09:56.470 INFO:tasks.workunit.client.1.smithi181.stdout:4/353: mkdir d0/d15/d76 0 2024-04-27T04:09:56.470 INFO:tasks.workunit.client.1.smithi181.stdout:4/354: fdatasync d0/f3b 0 2024-04-27T04:09:56.485 INFO:tasks.workunit.client.1.smithi181.stdout:7/327: rename d1/d2/d1c/d2d/d3d to d1/d2/d35/d67 0 2024-04-27T04:09:56.488 INFO:tasks.workunit.client.1.smithi181.stdout:4/355: rmdir d0/d15/d48/d57 39 2024-04-27T04:09:56.493 INFO:tasks.workunit.client.1.smithi181.stdout:9/414: unlink d0/f2a 0 2024-04-27T04:09:56.498 INFO:tasks.workunit.client.1.smithi181.stdout:7/328: creat d1/d2/d1c/d2d/d61/f68 x:0 0 0 2024-04-27T04:09:56.500 INFO:tasks.workunit.client.1.smithi181.stdout:9/415: getdents d0/d44/d4b/d43/d89 0 2024-04-27T04:09:56.500 INFO:tasks.workunit.client.1.smithi181.stdout:5/526: write d4/d10/d29/f84 [50536,62658] 0 2024-04-27T04:09:56.502 INFO:tasks.workunit.client.1.smithi181.stdout:5/527: fsync d4/de/d95/f9e 0 2024-04-27T04:09:56.504 INFO:tasks.workunit.client.1.smithi181.stdout:9/416: symlink d0/d44/d4b/d50/d6c/l92 0 2024-04-27T04:09:56.504 INFO:tasks.workunit.client.1.smithi181.stdout:9/417: write d0/f23 [1446271,26137] 0 2024-04-27T04:09:56.509 INFO:tasks.workunit.client.1.smithi181.stdout:5/528: rename d4/de/d95 to d4/da3/dbc 0 2024-04-27T04:09:56.518 INFO:tasks.workunit.client.1.smithi181.stdout:6/279: sync 2024-04-27T04:09:56.518 INFO:tasks.workunit.client.1.smithi181.stdout:5/529: fdatasync d4/d10/d72/d80/d85/fae 0 2024-04-27T04:09:56.519 INFO:tasks.workunit.client.1.smithi181.stdout:3/347: dwrite d0/de/f10 [0,4194304] 0 2024-04-27T04:09:56.521 INFO:tasks.workunit.client.1.smithi181.stdout:3/348: dread - d0/de/d20/d58/d64/f6a zero size 2024-04-27T04:09:56.522 INFO:tasks.workunit.client.1.smithi181.stdout:3/349: chown d0/de/c24 5 1 2024-04-27T04:09:56.524 INFO:tasks.workunit.client.1.smithi181.stdout:7/329: getdents d1/d2/d1c/d56 0 2024-04-27T04:09:56.524 INFO:tasks.workunit.client.1.smithi181.stdout:3/350: write d0/f4e [271583,81055] 0 2024-04-27T04:09:56.525 INFO:tasks.workunit.client.1.smithi181.stdout:3/351: chown d0/de/d20/f46 4 1 2024-04-27T04:09:56.536 INFO:tasks.workunit.client.1.smithi181.stdout:5/530: unlink d4/de/d2c/d46/d4d/d8e/d67/l70 0 2024-04-27T04:09:56.540 INFO:tasks.workunit.client.1.smithi181.stdout:7/330: truncate d1/d22/d15/d1f/d34/f3c 2176464 0 2024-04-27T04:09:56.541 INFO:tasks.workunit.client.1.smithi181.stdout:7/331: write d1/d22/d15/d13/d1b/f47 [9347,25573] 0 2024-04-27T04:09:56.541 INFO:tasks.workunit.client.1.smithi181.stdout:9/418: link d0/d44/d4b/c47 d0/d44/c93 0 2024-04-27T04:09:56.542 INFO:tasks.workunit.client.1.smithi181.stdout:9/419: chown d0/d40/f22 23826161 1 2024-04-27T04:09:56.543 INFO:tasks.workunit.client.1.smithi181.stdout:5/531: creat d4/da3/fbd x:0 0 0 2024-04-27T04:09:56.546 INFO:tasks.workunit.client.1.smithi181.stdout:7/332: creat d1/d2/d35/d67/f69 x:0 0 0 2024-04-27T04:09:56.548 INFO:tasks.workunit.client.0.smithi162.stdout:2/64: rmdir d0 39 2024-04-27T04:09:56.549 INFO:tasks.workunit.client.0.smithi162.stdout:5/79: symlink d8/d9/l19 0 2024-04-27T04:09:56.549 INFO:tasks.workunit.client.0.smithi162.stdout:5/80: dread - d8/d9/f18 zero size 2024-04-27T04:09:56.549 INFO:tasks.workunit.client.0.smithi162.stdout:0/136: symlink d5/dc/d1e/d20/l2b 0 2024-04-27T04:09:56.550 INFO:tasks.workunit.client.1.smithi181.stdout:9/420: mkdir d0/d44/d4b/d43/d89/d94 0 2024-04-27T04:09:56.550 INFO:tasks.workunit.client.0.smithi162.stdout:5/81: stat d8/ld 0 2024-04-27T04:09:56.550 INFO:tasks.workunit.client.0.smithi162.stdout:7/44: mknod d3/d5/cd 0 2024-04-27T04:09:56.551 INFO:tasks.workunit.client.1.smithi181.stdout:7/333: fsync d1/f45 0 2024-04-27T04:09:56.552 INFO:tasks.workunit.client.1.smithi181.stdout:7/334: write d1/d2/d1c/d55/f66 [576243,115884] 0 2024-04-27T04:09:56.552 INFO:tasks.workunit.client.0.smithi162.stdout:4/77: symlink d2/da/dd/l14 0 2024-04-27T04:09:56.552 INFO:tasks.workunit.client.0.smithi162.stdout:7/45: write d3/d5/f8 [419183,25468] 0 2024-04-27T04:09:56.553 INFO:tasks.workunit.client.1.smithi181.stdout:5/532: mknod d4/de/d2f/cbe 0 2024-04-27T04:09:56.553 INFO:tasks.workunit.client.0.smithi162.stdout:4/78: dread - d2/da/f13 zero size 2024-04-27T04:09:56.554 INFO:tasks.workunit.client.1.smithi181.stdout:5/533: stat d4/de/d2c/d46/d4d/d8e/d67/f50 0 2024-04-27T04:09:56.558 INFO:tasks.workunit.client.0.smithi162.stdout:1/99: mkdir d0/d11/d1a 0 2024-04-27T04:09:56.558 INFO:tasks.workunit.client.0.smithi162.stdout:9/63: mkdir d3/d13/d15 0 2024-04-27T04:09:56.558 INFO:tasks.workunit.client.0.smithi162.stdout:0/137: dwrite d5/f27 [0,4194304] 0 2024-04-27T04:09:56.558 INFO:tasks.workunit.client.0.smithi162.stdout:9/64: chown d3/l9 435 1 2024-04-27T04:09:56.560 INFO:tasks.workunit.client.0.smithi162.stdout:1/100: dread d0/fb [0,4194304] 0 2024-04-27T04:09:56.560 INFO:tasks.workunit.client.0.smithi162.stdout:3/59: rename d1/f3 to d1/d2/f11 0 2024-04-27T04:09:56.560 INFO:tasks.workunit.client.0.smithi162.stdout:4/79: rename d2 to d2/da/d15 22 2024-04-27T04:09:56.561 INFO:tasks.workunit.client.1.smithi181.stdout:5/534: creat d4/d10/d29/d92/fbf x:0 0 0 2024-04-27T04:09:56.561 INFO:tasks.workunit.client.0.smithi162.stdout:3/60: chown d1/d2/f10 4 1 2024-04-27T04:09:56.562 INFO:tasks.workunit.client.0.smithi162.stdout:9/65: dwrite d3/fa [0,4194304] 0 2024-04-27T04:09:56.564 INFO:tasks.workunit.client.1.smithi181.stdout:5/535: symlink d4/de/d2c/d2e/d55/lc0 0 2024-04-27T04:09:56.566 INFO:tasks.workunit.client.0.smithi162.stdout:1/101: dwrite d0/fb [0,4194304] 0 2024-04-27T04:09:56.567 INFO:tasks.workunit.client.0.smithi162.stdout:5/82: rename d8/d11 to d8/d1a 0 2024-04-27T04:09:56.571 INFO:tasks.workunit.client.1.smithi181.stdout:5/536: creat d4/da3/dbc/db5/fc1 x:0 0 0 2024-04-27T04:09:56.573 INFO:tasks.workunit.client.0.smithi162.stdout:0/138: fsync d5/dc/f14 0 2024-04-27T04:09:56.574 INFO:tasks.workunit.client.1.smithi181.stdout:5/537: unlink d4/de/d2c/d2e/d55/f9f 0 2024-04-27T04:09:56.575 INFO:tasks.workunit.client.1.smithi181.stdout:5/538: readlink d4/de/d2c/d46/d4d/d8e/l6d 0 2024-04-27T04:09:56.577 INFO:tasks.workunit.client.1.smithi181.stdout:5/539: rmdir d4/de 39 2024-04-27T04:09:56.578 INFO:tasks.workunit.client.0.smithi162.stdout:3/61: creat d1/d2/f12 x:0 0 0 2024-04-27T04:09:56.578 INFO:tasks.workunit.client.0.smithi162.stdout:4/80: dread d2/f9 [0,4194304] 0 2024-04-27T04:09:56.581 INFO:tasks.workunit.client.0.smithi162.stdout:7/46: mkdir d3/d5/d9/dc/de 0 2024-04-27T04:09:56.583 INFO:tasks.workunit.client.0.smithi162.stdout:0/139: rename d5/l6 to d5/l2c 0 2024-04-27T04:09:56.584 INFO:tasks.workunit.client.0.smithi162.stdout:6/89: creat d6/f1e x:0 0 0 2024-04-27T04:09:56.584 INFO:tasks.workunit.client.0.smithi162.stdout:3/62: creat d1/d2/f13 x:0 0 0 2024-04-27T04:09:56.585 INFO:tasks.workunit.client.0.smithi162.stdout:9/66: symlink d3/d11/l16 0 2024-04-27T04:09:56.585 INFO:tasks.workunit.client.0.smithi162.stdout:3/63: dread - d1/f7 zero size 2024-04-27T04:09:56.586 INFO:tasks.workunit.client.0.smithi162.stdout:4/81: rename d2/d5/c6 to d2/da/c16 0 2024-04-27T04:09:56.587 INFO:tasks.workunit.client.0.smithi162.stdout:1/102: link d0/d7/f19 d0/f1b 0 2024-04-27T04:09:56.588 INFO:tasks.workunit.client.0.smithi162.stdout:7/47: dwrite d3/d5/f8 [0,4194304] 0 2024-04-27T04:09:56.589 INFO:tasks.workunit.client.0.smithi162.stdout:0/140: unlink d5/f15 0 2024-04-27T04:09:56.589 INFO:tasks.workunit.client.0.smithi162.stdout:6/90: symlink d6/d1a/l1f 0 2024-04-27T04:09:56.593 INFO:tasks.workunit.client.0.smithi162.stdout:3/64: unlink d1/f7 0 2024-04-27T04:09:56.595 INFO:tasks.workunit.client.0.smithi162.stdout:6/91: dread f2 [0,4194304] 0 2024-04-27T04:09:56.598 INFO:tasks.workunit.client.0.smithi162.stdout:4/82: unlink d2/da/f13 0 2024-04-27T04:09:56.599 INFO:tasks.workunit.client.0.smithi162.stdout:1/103: unlink d0/d7/fd 0 2024-04-27T04:09:56.601 INFO:tasks.workunit.client.0.smithi162.stdout:6/92: dwrite d6/d1a/f18 [0,4194304] 0 2024-04-27T04:09:56.602 INFO:tasks.workunit.client.0.smithi162.stdout:7/48: unlink d3/d5/cd 0 2024-04-27T04:09:56.603 INFO:tasks.workunit.client.0.smithi162.stdout:6/93: write d6/d1a/f18 [383711,119172] 0 2024-04-27T04:09:56.604 INFO:tasks.workunit.client.0.smithi162.stdout:7/49: dread - d3/d5/fa zero size 2024-04-27T04:09:56.605 INFO:tasks.workunit.client.0.smithi162.stdout:0/141: rename d5/dc/f1f to d5/f2d 0 2024-04-27T04:09:56.607 INFO:tasks.workunit.client.0.smithi162.stdout:4/83: rename d2 to d2/da/d17 22 2024-04-27T04:09:56.609 INFO:tasks.workunit.client.0.smithi162.stdout:4/84: chown d2/ff 146346 1 2024-04-27T04:09:56.610 INFO:tasks.workunit.client.1.smithi181.stdout:6/280: write d7/dc/d1e/f30 [4674557,32744] 0 2024-04-27T04:09:56.610 INFO:tasks.workunit.client.0.smithi162.stdout:9/67: mknod d3/d13/d15/c17 0 2024-04-27T04:09:56.612 INFO:tasks.workunit.client.0.smithi162.stdout:4/85: write d2/ff [796207,103329] 0 2024-04-27T04:09:56.613 INFO:tasks.workunit.client.0.smithi162.stdout:1/104: creat d0/d11/f1c x:0 0 0 2024-04-27T04:09:56.615 INFO:tasks.workunit.client.1.smithi181.stdout:6/281: rmdir d7/dc/d1e/d2b 39 2024-04-27T04:09:56.615 INFO:tasks.workunit.client.0.smithi162.stdout:1/105: chown d0/lf 832 1 2024-04-27T04:09:56.617 INFO:tasks.workunit.client.0.smithi162.stdout:7/50: dwrite d3/d5/fa [0,4194304] 0 2024-04-27T04:09:56.620 INFO:tasks.workunit.client.0.smithi162.stdout:9/68: dwrite d3/fa [0,4194304] 0 2024-04-27T04:09:56.622 INFO:tasks.workunit.client.0.smithi162.stdout:6/94: rename d6/fc to d6/d1a/f20 0 2024-04-27T04:09:56.622 INFO:tasks.workunit.client.1.smithi181.stdout:6/282: mknod d7/dc/d1c/c5b 0 2024-04-27T04:09:56.623 INFO:tasks.workunit.client.1.smithi181.stdout:7/335: truncate d1/d22/d15/d1f/d34/f3c 1294502 0 2024-04-27T04:09:56.624 INFO:tasks.workunit.client.0.smithi162.stdout:0/142: mknod d5/dc/c2e 0 2024-04-27T04:09:56.625 INFO:tasks.workunit.client.0.smithi162.stdout:1/106: dwrite d0/d7/d10/f12 [0,4194304] 0 2024-04-27T04:09:56.635 INFO:tasks.workunit.client.0.smithi162.stdout:4/86: symlink d2/da/l18 0 2024-04-27T04:09:56.635 INFO:tasks.workunit.client.0.smithi162.stdout:8/99: stat d0/d12/f14 0 2024-04-27T04:09:56.636 INFO:tasks.workunit.client.0.smithi162.stdout:4/87: chown d2/da/dd/f10 119 1 2024-04-27T04:09:56.636 INFO:tasks.workunit.client.0.smithi162.stdout:4/88: read d2/f4 [637447,89377] 0 2024-04-27T04:09:56.637 INFO:tasks.workunit.client.0.smithi162.stdout:5/83: rmdir d8/d9 39 2024-04-27T04:09:56.637 INFO:tasks.workunit.client.0.smithi162.stdout:4/89: write d2/f4 [4732696,119888] 0 2024-04-27T04:09:56.638 INFO:tasks.workunit.client.1.smithi181.stdout:7/336: mkdir d1/df/d1a/d6a 0 2024-04-27T04:09:56.638 INFO:tasks.workunit.client.1.smithi181.stdout:3/352: dwrite d0/f14 [0,4194304] 0 2024-04-27T04:09:56.638 INFO:tasks.workunit.client.0.smithi162.stdout:4/90: fdatasync d2/d5/fb 0 2024-04-27T04:09:56.645 INFO:tasks.workunit.client.0.smithi162.stdout:9/69: creat d3/d13/d15/f18 x:0 0 0 2024-04-27T04:09:56.646 INFO:tasks.workunit.client.0.smithi162.stdout:9/70: chown d3/fe 1639 1 2024-04-27T04:09:56.648 INFO:tasks.workunit.client.0.smithi162.stdout:6/95: creat d6/db/f21 x:0 0 0 2024-04-27T04:09:56.648 INFO:tasks.workunit.client.1.smithi181.stdout:7/337: mknod d1/d22/d15/d13/d1b/c6b 0 2024-04-27T04:09:56.651 INFO:tasks.workunit.client.1.smithi181.stdout:9/421: write d0/d40/f25 [3927978,49865] 0 2024-04-27T04:09:56.656 INFO:tasks.workunit.client.1.smithi181.stdout:7/338: rename d1/d22/d15/d13/l4b to d1/d2/d1c/d5e/l6c 0 2024-04-27T04:09:56.656 INFO:tasks.workunit.client.0.smithi162.stdout:8/100: mknod d0/de/c1a 0 2024-04-27T04:09:56.657 INFO:tasks.workunit.client.1.smithi181.stdout:7/339: truncate d1/d2/d1c/f1e 4533618 0 2024-04-27T04:09:56.659 INFO:tasks.workunit.client.0.smithi162.stdout:5/84: fsync d8/d9/f16 0 2024-04-27T04:09:56.663 INFO:tasks.workunit.client.0.smithi162.stdout:4/91: symlink d2/da/l19 0 2024-04-27T04:09:56.665 INFO:tasks.workunit.client.1.smithi181.stdout:7/340: creat d1/d22/d15/d29/f6d x:0 0 0 2024-04-27T04:09:56.665 INFO:tasks.workunit.client.0.smithi162.stdout:9/71: creat d3/d13/d15/f19 x:0 0 0 2024-04-27T04:09:56.667 INFO:tasks.workunit.client.1.smithi181.stdout:5/540: write d4/f13 [674566,108681] 0 2024-04-27T04:09:56.668 INFO:tasks.workunit.client.0.smithi162.stdout:6/96: creat d6/db/f22 x:0 0 0 2024-04-27T04:09:56.670 INFO:tasks.workunit.client.1.smithi181.stdout:7/341: symlink d1/df/l6e 0 2024-04-27T04:09:56.671 INFO:tasks.workunit.client.0.smithi162.stdout:3/65: link d1/l4 d1/l14 0 2024-04-27T04:09:56.671 INFO:tasks.workunit.client.1.smithi181.stdout:7/342: fdatasync d1/d22/d15/d1f/d34/f5b 0 2024-04-27T04:09:56.672 INFO:tasks.workunit.client.1.smithi181.stdout:5/541: stat d4/de/d2c/d46/d99/la4 0 2024-04-27T04:09:56.673 INFO:tasks.workunit.client.1.smithi181.stdout:5/542: stat d4/de/d2c/d2e 0 2024-04-27T04:09:56.673 INFO:tasks.workunit.client.0.smithi162.stdout:6/97: dwrite d6/db/f21 [0,4194304] 0 2024-04-27T04:09:56.675 INFO:tasks.workunit.client.1.smithi181.stdout:7/343: dread d1/d22/d15/f28 [0,4194304] 0 2024-04-27T04:09:56.679 INFO:tasks.workunit.client.1.smithi181.stdout:5/543: rename d4/de/d2c/d46/d4d/f52 to d4/da9/fc2 0 2024-04-27T04:09:56.681 INFO:tasks.workunit.client.0.smithi162.stdout:4/92: mkdir d2/da/d1a 0 2024-04-27T04:09:56.682 INFO:tasks.workunit.client.0.smithi162.stdout:2/65: truncate d0/fe 104313 0 2024-04-27T04:09:56.684 INFO:tasks.workunit.client.0.smithi162.stdout:2/66: write d0/d4/f16 [682693,43902] 0 2024-04-27T04:09:56.686 INFO:tasks.workunit.client.0.smithi162.stdout:9/72: symlink d3/d13/d15/l1a 0 2024-04-27T04:09:56.687 INFO:tasks.workunit.client.0.smithi162.stdout:9/73: chown d3/fe 14064462 1 2024-04-27T04:09:56.687 INFO:tasks.workunit.client.0.smithi162.stdout:4/93: dwrite d2/ff [0,4194304] 0 2024-04-27T04:09:56.687 INFO:tasks.workunit.client.0.smithi162.stdout:2/67: fsync d0/d4/f16 0 2024-04-27T04:09:56.688 INFO:tasks.workunit.client.0.smithi162.stdout:2/68: chown d0/d4/ld 266462326 1 2024-04-27T04:09:56.694 INFO:tasks.workunit.client.0.smithi162.stdout:6/98: creat d6/db/f23 x:0 0 0 2024-04-27T04:09:56.697 INFO:tasks.workunit.client.0.smithi162.stdout:7/51: getdents d3/d5/d9/dc 0 2024-04-27T04:09:56.699 INFO:tasks.workunit.client.0.smithi162.stdout:9/74: mkdir d3/d11/d1b 0 2024-04-27T04:09:56.700 INFO:tasks.workunit.client.0.smithi162.stdout:9/75: chown d3/d11/l16 36 1 2024-04-27T04:09:56.701 INFO:tasks.workunit.client.0.smithi162.stdout:2/69: unlink d0/d4/d17/c19 0 2024-04-27T04:09:56.702 INFO:tasks.workunit.client.0.smithi162.stdout:2/70: write d0/d4/f16 [1240018,127763] 0 2024-04-27T04:09:56.705 INFO:tasks.workunit.client.0.smithi162.stdout:8/101: link d0/l5 d0/d10/l1b 0 2024-04-27T04:09:56.707 INFO:tasks.workunit.client.0.smithi162.stdout:9/76: dwrite d3/f6 [0,4194304] 0 2024-04-27T04:09:56.707 INFO:tasks.workunit.client.0.smithi162.stdout:9/77: stat d3 0 2024-04-27T04:09:56.708 INFO:tasks.workunit.client.0.smithi162.stdout:5/85: rename d8/cb to d8/c1b 0 2024-04-27T04:09:56.708 INFO:tasks.workunit.client.0.smithi162.stdout:5/86: stat c6 0 2024-04-27T04:09:56.710 INFO:tasks.workunit.client.0.smithi162.stdout:6/99: unlink d6/d1a/l1f 0 2024-04-27T04:09:56.713 INFO:tasks.workunit.client.0.smithi162.stdout:7/52: rename f0 to d3/d5/ff 0 2024-04-27T04:09:56.714 INFO:tasks.workunit.client.0.smithi162.stdout:2/71: mknod d0/d4/d17/c1a 0 2024-04-27T04:09:56.716 INFO:tasks.workunit.client.1.smithi181.stdout:6/283: dwrite d7/dc/d1c/f20 [0,4194304] 0 2024-04-27T04:09:56.720 INFO:tasks.workunit.client.0.smithi162.stdout:8/102: mknod d0/d10/c1c 0 2024-04-27T04:09:56.721 INFO:tasks.workunit.client.0.smithi162.stdout:9/78: mkdir d3/d13/d1c 0 2024-04-27T04:09:56.724 INFO:tasks.workunit.client.0.smithi162.stdout:5/87: mknod d8/d9/c1c 0 2024-04-27T04:09:56.726 INFO:tasks.workunit.client.0.smithi162.stdout:6/100: creat d6/d1a/f24 x:0 0 0 2024-04-27T04:09:56.726 INFO:tasks.workunit.client.1.smithi181.stdout:6/284: mkdir d7/dc/d1e/d5c 0 2024-04-27T04:09:56.727 INFO:tasks.workunit.client.0.smithi162.stdout:3/66: rmdir d1 39 2024-04-27T04:09:56.730 INFO:tasks.workunit.client.1.smithi181.stdout:3/353: truncate d0/f57 391305 0 2024-04-27T04:09:56.731 INFO:tasks.workunit.client.0.smithi162.stdout:0/143: dwrite d5/fa [0,4194304] 0 2024-04-27T04:09:56.731 INFO:tasks.workunit.client.1.smithi181.stdout:3/354: write d0/de/d20/f62 [822297,82974] 0 2024-04-27T04:09:56.732 INFO:tasks.workunit.client.0.smithi162.stdout:2/72: fdatasync d0/f6 0 2024-04-27T04:09:56.733 INFO:tasks.workunit.client.0.smithi162.stdout:0/144: fdatasync d5/dc/f22 0 2024-04-27T04:09:56.733 INFO:tasks.workunit.client.0.smithi162.stdout:6/101: dread d6/f17 [0,4194304] 0 2024-04-27T04:09:56.759 INFO:tasks.workunit.client.0.smithi162.stdout:1/107: sync 2024-04-27T04:09:56.760 INFO:tasks.workunit.client.1.smithi181.stdout:8/459: sync 2024-04-27T04:09:56.761 INFO:tasks.workunit.client.1.smithi181.stdout:2/397: sync 2024-04-27T04:09:56.761 INFO:tasks.workunit.client.1.smithi181.stdout:1/406: sync 2024-04-27T04:09:56.761 INFO:tasks.workunit.client.1.smithi181.stdout:8/460: stat d1/d20/f51 0 2024-04-27T04:09:56.762 INFO:tasks.workunit.client.1.smithi181.stdout:3/355: chown d0/d6c/d52/l39 150196 1 2024-04-27T04:09:56.766 INFO:tasks.workunit.client.0.smithi162.stdout:9/79: symlink d3/d13/d15/l1d 0 2024-04-27T04:09:56.767 INFO:tasks.workunit.client.1.smithi181.stdout:8/461: dread d1/d6/d2f/d40/f5e [0,4194304] 0 2024-04-27T04:09:56.767 INFO:tasks.workunit.client.1.smithi181.stdout:9/422: truncate d0/d44/d4b/d43/f56 204468 0 2024-04-27T04:09:56.767 INFO:tasks.workunit.client.1.smithi181.stdout:8/462: dread - d1/d6/d2f/d47/f4b zero size 2024-04-27T04:09:56.768 INFO:tasks.workunit.client.0.smithi162.stdout:5/88: symlink d8/d1a/l1d 0 2024-04-27T04:09:56.770 INFO:tasks.workunit.client.0.smithi162.stdout:3/67: stat d1/d2 0 2024-04-27T04:09:56.771 INFO:tasks.workunit.client.1.smithi181.stdout:2/398: symlink d4/d9/d4f/d5f/l7e 0 2024-04-27T04:09:56.781 INFO:tasks.workunit.client.1.smithi181.stdout:8/463: mknod d1/d6/d2f/d72/ca7 0 2024-04-27T04:09:56.782 INFO:tasks.workunit.client.0.smithi162.stdout:8/103: fsync d0/fd 0 2024-04-27T04:09:56.792 INFO:tasks.workunit.client.0.smithi162.stdout:4/94: sync 2024-04-27T04:09:56.795 INFO:tasks.workunit.client.1.smithi181.stdout:9/423: mkdir d0/d44/d4b/d43/d88/d95 0 2024-04-27T04:09:56.795 INFO:tasks.workunit.client.1.smithi181.stdout:9/424: chown d0/f15 3 1 2024-04-27T04:09:56.797 INFO:tasks.workunit.client.1.smithi181.stdout:9/425: read - d0/d4d/f7f zero size 2024-04-27T04:09:56.798 INFO:tasks.workunit.client.0.smithi162.stdout:3/68: rename d1/c5 to d1/d2/c15 0 2024-04-27T04:09:56.798 INFO:tasks.workunit.client.1.smithi181.stdout:3/356: link d0/d6c/d52/f65 d0/de/d3f/f75 0 2024-04-27T04:09:56.800 INFO:tasks.workunit.client.0.smithi162.stdout:3/69: write d1/fd [679024,102135] 0 2024-04-27T04:09:56.801 INFO:tasks.workunit.client.0.smithi162.stdout:3/70: read d1/d2/f11 [340228,22303] 0 2024-04-27T04:09:56.808 INFO:tasks.workunit.client.1.smithi181.stdout:1/407: rename d1/d4c/d1e/l44 to d1/d2/d5/d6/db/d2d/l7e 0 2024-04-27T04:09:56.809 INFO:tasks.workunit.client.1.smithi181.stdout:1/408: write d1/d2/d5/f45 [1546856,97466] 0 2024-04-27T04:09:56.814 INFO:tasks.workunit.client.1.smithi181.stdout:6/285: dwrite d7/dc/ff [0,4194304] 0 2024-04-27T04:09:56.814 INFO:tasks.workunit.client.0.smithi162.stdout:0/145: dwrite d5/dc/d1e/f21 [0,4194304] 0 2024-04-27T04:09:56.815 INFO:tasks.workunit.client.1.smithi181.stdout:3/357: dread d0/de/d20/f5e [0,4194304] 0 2024-04-27T04:09:56.816 INFO:tasks.workunit.client.0.smithi162.stdout:1/108: chown d0/c6 5440 1 2024-04-27T04:09:56.818 INFO:tasks.workunit.client.0.smithi162.stdout:9/80: mkdir d3/d11/d1b/d1e 0 2024-04-27T04:09:56.818 INFO:tasks.workunit.client.1.smithi181.stdout:1/409: rmdir d1/d4c/d1e/d20/d53/d56 39 2024-04-27T04:09:56.822 INFO:tasks.workunit.client.0.smithi162.stdout:7/53: truncate d3/d5/ff 2756802 0 2024-04-27T04:09:56.826 INFO:tasks.workunit.client.0.smithi162.stdout:8/104: write d0/fd [4682,79456] 0 2024-04-27T04:09:56.827 INFO:tasks.workunit.client.1.smithi181.stdout:3/358: creat d0/d6c/d52/d44/d56/f76 x:0 0 0 2024-04-27T04:09:56.829 INFO:tasks.workunit.client.1.smithi181.stdout:1/410: fsync d1/d2/d5/d6/f22 0 2024-04-27T04:09:56.830 INFO:tasks.workunit.client.0.smithi162.stdout:3/71: dread d1/d2/f6 [0,4194304] 0 2024-04-27T04:09:56.831 INFO:tasks.workunit.client.1.smithi181.stdout:1/411: stat d1/d2/d5/d6/db/d2d/d36/f4a 0 2024-04-27T04:09:56.831 INFO:tasks.workunit.client.0.smithi162.stdout:3/72: chown d1/d2/f13 101 1 2024-04-27T04:09:56.832 INFO:tasks.workunit.client.1.smithi181.stdout:3/359: dread d0/de/d20/f30 [0,4194304] 0 2024-04-27T04:09:56.832 INFO:tasks.workunit.client.1.smithi181.stdout:1/412: dread - d1/d4c/d1e/f7a zero size 2024-04-27T04:09:56.833 INFO:tasks.workunit.client.0.smithi162.stdout:8/105: dwrite d0/f4 [0,4194304] 0 2024-04-27T04:09:56.833 INFO:tasks.workunit.client.0.smithi162.stdout:6/102: link d6/f7 d6/f25 0 2024-04-27T04:09:56.833 INFO:tasks.workunit.client.1.smithi181.stdout:3/360: truncate d0/f16 1891114 0 2024-04-27T04:09:56.834 INFO:tasks.workunit.client.1.smithi181.stdout:6/286: creat d7/dc/d1c/d34/f5d x:0 0 0 2024-04-27T04:09:56.835 INFO:tasks.workunit.client.1.smithi181.stdout:6/287: write d7/f19 [4863564,91993] 0 2024-04-27T04:09:56.837 INFO:tasks.workunit.client.1.smithi181.stdout:9/426: getdents d0/d44/d4b/d3d 0 2024-04-27T04:09:56.838 INFO:tasks.workunit.client.1.smithi181.stdout:8/464: truncate d1/f6b 3385112 0 2024-04-27T04:09:56.839 INFO:tasks.workunit.client.1.smithi181.stdout:8/465: dread - d1/d6/d2f/d30/d5c/f97 zero size 2024-04-27T04:09:56.840 INFO:tasks.workunit.client.0.smithi162.stdout:1/109: dread d0/fb [0,4194304] 0 2024-04-27T04:09:56.843 INFO:tasks.workunit.client.0.smithi162.stdout:4/95: creat d2/da/d1a/f1b x:0 0 0 2024-04-27T04:09:56.844 INFO:tasks.workunit.client.1.smithi181.stdout:1/413: creat d1/d2/d5/d6/f7f x:0 0 0 2024-04-27T04:09:56.845 INFO:tasks.workunit.client.1.smithi181.stdout:1/414: fdatasync d1/d4c/d1e/f79 0 2024-04-27T04:09:56.848 INFO:tasks.workunit.client.1.smithi181.stdout:3/361: mkdir d0/d6c/d52/d44/d56/d5b/d77 0 2024-04-27T04:09:56.848 INFO:tasks.workunit.client.1.smithi181.stdout:1/415: read d1/d2/d5/d6/f22 [91087,11676] 0 2024-04-27T04:09:56.851 INFO:tasks.workunit.client.0.smithi162.stdout:2/73: dread d0/fe [0,4194304] 0 2024-04-27T04:09:56.852 INFO:tasks.workunit.client.0.smithi162.stdout:5/89: link l7 d8/d1a/l1e 0 2024-04-27T04:09:56.853 INFO:tasks.workunit.client.1.smithi181.stdout:9/427: symlink d0/d4d/l96 0 2024-04-27T04:09:56.855 INFO:tasks.workunit.client.0.smithi162.stdout:3/73: mknod d1/d2/c16 0 2024-04-27T04:09:56.855 INFO:tasks.workunit.client.1.smithi181.stdout:9/428: write d0/d44/d4f/f54 [651377,53673] 0 2024-04-27T04:09:56.856 INFO:tasks.workunit.client.0.smithi162.stdout:3/74: write d1/d2/f12 [42349,5835] 0 2024-04-27T04:09:56.859 INFO:tasks.workunit.client.0.smithi162.stdout:5/90: dwrite d8/d1a/f17 [0,4194304] 0 2024-04-27T04:09:56.859 INFO:tasks.workunit.client.0.smithi162.stdout:8/106: creat d0/d12/f1d x:0 0 0 2024-04-27T04:09:56.861 INFO:tasks.workunit.client.0.smithi162.stdout:8/107: fdatasync d0/fd 0 2024-04-27T04:09:56.861 INFO:tasks.workunit.client.0.smithi162.stdout:5/91: write d8/d9/f18 [711389,110790] 0 2024-04-27T04:09:56.862 INFO:tasks.workunit.client.0.smithi162.stdout:3/75: dread d1/d2/f6 [0,4194304] 0 2024-04-27T04:09:56.867 INFO:tasks.workunit.client.1.smithi181.stdout:1/416: unlink d1/d4c/d1e/d20/d53/l7b 0 2024-04-27T04:09:56.867 INFO:tasks.workunit.client.1.smithi181.stdout:1/417: chown d1/d2/d5/d6/c31 457 1 2024-04-27T04:09:56.869 INFO:tasks.workunit.client.1.smithi181.stdout:6/288: symlink d7/dc/d1c/d34/d3f/l5e 0 2024-04-27T04:09:56.872 INFO:tasks.workunit.client.0.smithi162.stdout:4/96: mkdir d2/d5/d1c 0 2024-04-27T04:09:56.873 INFO:tasks.workunit.client.0.smithi162.stdout:4/97: write f1 [5619046,80828] 0 2024-04-27T04:09:56.873 INFO:tasks.workunit.client.1.smithi181.stdout:8/466: mknod d1/d6/d2f/d30/d9d/ca8 0 2024-04-27T04:09:56.875 INFO:tasks.workunit.client.1.smithi181.stdout:3/362: mknod d0/de/d20/d58/d70/c78 0 2024-04-27T04:09:56.877 INFO:tasks.workunit.client.1.smithi181.stdout:1/418: mknod d1/d4c/c80 0 2024-04-27T04:09:56.877 INFO:tasks.workunit.client.1.smithi181.stdout:1/419: fdatasync d1/d2/d5/f45 0 2024-04-27T04:09:56.879 INFO:tasks.workunit.client.1.smithi181.stdout:6/289: rename d7/dc/d1c/c25 to d7/dc/d1c/d34/c5f 0 2024-04-27T04:09:56.881 INFO:tasks.workunit.client.0.smithi162.stdout:0/146: truncate d5/dc/d1e/f21 4478808 0 2024-04-27T04:09:56.881 INFO:tasks.workunit.client.0.smithi162.stdout:2/74: dwrite d0/fe [0,4194304] 0 2024-04-27T04:09:56.881 INFO:tasks.workunit.client.1.smithi181.stdout:9/429: symlink d0/d44/d4b/d43/d88/d95/l97 0 2024-04-27T04:09:56.882 INFO:tasks.workunit.client.0.smithi162.stdout:2/75: write d0/d4/f15 [941550,2056] 0 2024-04-27T04:09:56.882 INFO:tasks.workunit.client.0.smithi162.stdout:2/76: fsync d0/f7 0 2024-04-27T04:09:56.886 INFO:tasks.workunit.client.0.smithi162.stdout:8/108: symlink d0/de/d17/l1e 0 2024-04-27T04:09:56.889 INFO:tasks.workunit.client.1.smithi181.stdout:3/363: rmdir d0/de/d20 39 2024-04-27T04:09:56.892 INFO:tasks.workunit.client.0.smithi162.stdout:3/76: mknod d1/d2/c17 0 2024-04-27T04:09:56.893 INFO:tasks.workunit.client.0.smithi162.stdout:3/77: chown d1/fb 60170564 1 2024-04-27T04:09:56.893 INFO:tasks.workunit.client.0.smithi162.stdout:1/110: mkdir d0/d11/d1d 0 2024-04-27T04:09:56.893 INFO:tasks.workunit.client.0.smithi162.stdout:3/78: fdatasync d1/d2/f10 0 2024-04-27T04:09:56.894 INFO:tasks.workunit.client.1.smithi181.stdout:6/290: mknod d7/d1b/d3e/c60 0 2024-04-27T04:09:56.895 INFO:tasks.workunit.client.1.smithi181.stdout:6/291: rename d7/d31/d38 to d7/d31/d38/d61 22 2024-04-27T04:09:56.895 INFO:tasks.workunit.client.1.smithi181.stdout:9/430: chown d0/l1 1 1 2024-04-27T04:09:56.896 INFO:tasks.workunit.client.0.smithi162.stdout:9/81: creat d3/f1f x:0 0 0 2024-04-27T04:09:56.897 INFO:tasks.workunit.client.0.smithi162.stdout:9/82: fdatasync d3/f4 0 2024-04-27T04:09:56.898 INFO:tasks.workunit.client.1.smithi181.stdout:8/467: symlink d1/d6/da6/la9 0 2024-04-27T04:09:56.899 INFO:tasks.workunit.client.1.smithi181.stdout:8/468: readlink d1/d6/da6/la9 0 2024-04-27T04:09:56.901 INFO:tasks.workunit.client.1.smithi181.stdout:3/364: creat d0/d6c/f79 x:0 0 0 2024-04-27T04:09:56.902 INFO:tasks.workunit.client.0.smithi162.stdout:9/83: dwrite d3/fe [0,4194304] 0 2024-04-27T04:09:56.902 INFO:tasks.workunit.client.0.smithi162.stdout:4/98: mknod d2/d5/c1d 0 2024-04-27T04:09:56.903 INFO:tasks.workunit.client.1.smithi181.stdout:8/469: read d1/d6/d1a/f53 [3073379,32001] 0 2024-04-27T04:09:56.903 INFO:tasks.workunit.client.1.smithi181.stdout:9/431: dwrite d0/d44/f7c [0,4194304] 0 2024-04-27T04:09:56.903 INFO:tasks.workunit.client.1.smithi181.stdout:3/365: truncate d0/d6c/f60 4652763 0 2024-04-27T04:09:56.904 INFO:tasks.workunit.client.1.smithi181.stdout:1/420: mknod d1/d2/c81 0 2024-04-27T04:09:56.906 INFO:tasks.workunit.client.1.smithi181.stdout:1/421: write d1/d2/d5/d6/db/d2d/d36/f5e [160858,112069] 0 2024-04-27T04:09:56.908 INFO:tasks.workunit.client.0.smithi162.stdout:4/99: dread d2/da/dd/fe [0,4194304] 0 2024-04-27T04:09:56.908 INFO:tasks.workunit.client.0.smithi162.stdout:7/54: dread d3/d5/ff [0,4194304] 0 2024-04-27T04:09:56.910 INFO:tasks.workunit.client.1.smithi181.stdout:0/377: sync 2024-04-27T04:09:56.910 INFO:tasks.workunit.client.0.smithi162.stdout:6/103: creat d6/f26 x:0 0 0 2024-04-27T04:09:56.911 INFO:tasks.workunit.client.0.smithi162.stdout:6/104: chown d6/d1a/f24 6315362 1 2024-04-27T04:09:56.912 INFO:tasks.workunit.client.0.smithi162.stdout:6/105: chown d6/f17 1235170 1 2024-04-27T04:09:56.914 INFO:tasks.workunit.client.1.smithi181.stdout:3/366: stat d0/d6c/d52/l38 0 2024-04-27T04:09:56.923 INFO:tasks.workunit.client.0.smithi162.stdout:9/84: rename d3/fe to d3/d11/d1b/d1e/f20 0 2024-04-27T04:09:56.924 INFO:tasks.workunit.client.0.smithi162.stdout:4/100: symlink d2/d5/l1e 0 2024-04-27T04:09:56.926 INFO:tasks.workunit.client.0.smithi162.stdout:4/101: dread d2/f9 [0,4194304] 0 2024-04-27T04:09:56.926 INFO:tasks.workunit.client.0.smithi162.stdout:6/106: symlink d6/db/l27 0 2024-04-27T04:09:56.927 INFO:tasks.workunit.client.0.smithi162.stdout:8/109: mknod d0/c1f 0 2024-04-27T04:09:56.928 INFO:tasks.workunit.client.0.smithi162.stdout:6/107: write d6/f9 [329495,117883] 0 2024-04-27T04:09:56.928 INFO:tasks.workunit.client.0.smithi162.stdout:8/110: chown d0/d10 1 1 2024-04-27T04:09:56.929 INFO:tasks.workunit.client.0.smithi162.stdout:8/111: chown d0/f2 0 1 2024-04-27T04:09:56.929 INFO:tasks.workunit.client.0.smithi162.stdout:6/108: write d6/d1a/f1d [227857,54357] 0 2024-04-27T04:09:56.930 INFO:tasks.workunit.client.0.smithi162.stdout:6/109: dread - d6/f26 zero size 2024-04-27T04:09:56.931 INFO:tasks.workunit.client.0.smithi162.stdout:6/110: write d6/db/f23 [903309,63448] 0 2024-04-27T04:09:56.932 INFO:tasks.workunit.client.0.smithi162.stdout:7/55: getdents d3/d5/d9/dc/de 0 2024-04-27T04:09:56.932 INFO:tasks.workunit.client.0.smithi162.stdout:8/112: dread d0/fa [0,4194304] 0 2024-04-27T04:09:56.935 INFO:tasks.workunit.client.0.smithi162.stdout:4/102: unlink d2/d5/l1e 0 2024-04-27T04:09:56.936 INFO:tasks.workunit.client.0.smithi162.stdout:5/92: getdents d8/d1a 0 2024-04-27T04:09:56.938 INFO:tasks.workunit.client.0.smithi162.stdout:3/79: rename d1/d2 to d1/d18 0 2024-04-27T04:09:56.940 INFO:tasks.workunit.client.0.smithi162.stdout:4/103: dwrite d2/d5/f7 [0,4194304] 0 2024-04-27T04:09:56.941 INFO:tasks.workunit.client.0.smithi162.stdout:4/104: dread - d2/da/f12 zero size 2024-04-27T04:09:56.942 INFO:tasks.workunit.client.0.smithi162.stdout:4/105: write f1 [2694816,83836] 0 2024-04-27T04:09:56.942 INFO:tasks.workunit.client.0.smithi162.stdout:3/80: dread d1/d18/f9 [0,4194304] 0 2024-04-27T04:09:56.943 INFO:tasks.workunit.client.0.smithi162.stdout:4/106: write f1 [4796818,28805] 0 2024-04-27T04:09:56.946 INFO:tasks.workunit.client.0.smithi162.stdout:7/56: unlink d3/d5/ff 0 2024-04-27T04:09:56.947 INFO:tasks.workunit.client.0.smithi162.stdout:8/113: mkdir d0/de/d20 0 2024-04-27T04:09:56.950 INFO:tasks.workunit.client.0.smithi162.stdout:1/111: truncate d0/d7/d10/f12 3553173 0 2024-04-27T04:09:56.951 INFO:tasks.workunit.client.0.smithi162.stdout:5/93: mknod d8/d1a/c1f 0 2024-04-27T04:09:56.951 INFO:tasks.workunit.client.0.smithi162.stdout:5/94: fdatasync d8/f12 0 2024-04-27T04:09:56.952 INFO:tasks.workunit.client.0.smithi162.stdout:6/111: write d6/f17 [2722349,88317] 0 2024-04-27T04:09:56.954 INFO:tasks.workunit.client.0.smithi162.stdout:9/85: rmdir d3/d13/d1c 0 2024-04-27T04:09:56.955 INFO:tasks.workunit.client.0.smithi162.stdout:6/112: fdatasync d6/f26 0 2024-04-27T04:09:56.955 INFO:tasks.workunit.client.0.smithi162.stdout:3/81: creat d1/d18/f19 x:0 0 0 2024-04-27T04:09:56.957 INFO:tasks.workunit.client.0.smithi162.stdout:4/107: rename d2/da/dd/fe to d2/f1f 0 2024-04-27T04:09:56.957 INFO:tasks.workunit.client.0.smithi162.stdout:6/113: dread - d6/d1a/f24 zero size 2024-04-27T04:09:56.958 INFO:tasks.workunit.client.1.smithi181.stdout:0/378: mkdir dc/d1e/d26/d41/d6f/d8f 0 2024-04-27T04:09:56.959 INFO:tasks.workunit.client.1.smithi181.stdout:3/367: truncate d0/de/ff 2133316 0 2024-04-27T04:09:56.960 INFO:tasks.workunit.client.0.smithi162.stdout:0/147: dwrite d5/f2d [4194304,4194304] 0 2024-04-27T04:09:56.960 INFO:tasks.workunit.client.1.smithi181.stdout:0/379: fsync dc/d1e/d26/d68/f88 0 2024-04-27T04:09:56.961 INFO:tasks.workunit.client.0.smithi162.stdout:5/95: dwrite d8/f12 [0,4194304] 0 2024-04-27T04:09:56.961 INFO:tasks.workunit.client.0.smithi162.stdout:7/57: mknod d3/d5/d9/dc/de/c10 0 2024-04-27T04:09:56.966 INFO:tasks.workunit.client.0.smithi162.stdout:6/114: read d6/d1a/f1d [5888,129027] 0 2024-04-27T04:09:56.966 INFO:tasks.workunit.client.0.smithi162.stdout:7/58: fsync d3/d5/f8 0 2024-04-27T04:09:56.966 INFO:tasks.workunit.client.0.smithi162.stdout:3/82: creat d1/d18/f1a x:0 0 0 2024-04-27T04:09:56.966 INFO:tasks.workunit.client.1.smithi181.stdout:6/292: getdents d7/d27 0 2024-04-27T04:09:56.966 INFO:tasks.workunit.client.1.smithi181.stdout:3/368: write d0/d6c/d52/f47 [227315,88593] 0 2024-04-27T04:09:56.966 INFO:tasks.workunit.client.1.smithi181.stdout:3/369: chown d0/d6c/d52/d44/d56/f63 539117 1 2024-04-27T04:09:56.966 INFO:tasks.workunit.client.1.smithi181.stdout:9/432: dread d0/f26 [0,4194304] 0 2024-04-27T04:09:56.968 INFO:tasks.workunit.client.1.smithi181.stdout:1/422: rename d1/d4c/d1e/d20/f61 to d1/d4c/d1e/d20/f82 0 2024-04-27T04:09:56.970 INFO:tasks.workunit.client.0.smithi162.stdout:7/59: dread d3/d5/f8 [0,4194304] 0 2024-04-27T04:09:56.971 INFO:tasks.workunit.client.0.smithi162.stdout:7/60: readlink d3/lb 0 2024-04-27T04:09:56.972 INFO:tasks.workunit.client.0.smithi162.stdout:9/86: dwrite d3/d11/d1b/d1e/f20 [0,4194304] 0 2024-04-27T04:09:56.972 INFO:tasks.workunit.client.0.smithi162.stdout:4/108: symlink d2/da/d1a/l20 0 2024-04-27T04:09:56.973 INFO:tasks.workunit.client.0.smithi162.stdout:7/61: read d3/d5/fa [452141,41167] 0 2024-04-27T04:09:56.973 INFO:tasks.workunit.client.0.smithi162.stdout:9/87: stat d3/d13/d15/l1a 0 2024-04-27T04:09:56.974 INFO:tasks.workunit.client.1.smithi181.stdout:0/380: unlink dc/d1e/c2f 0 2024-04-27T04:09:56.974 INFO:tasks.workunit.client.0.smithi162.stdout:4/109: write d2/da/fc [993238,15975] 0 2024-04-27T04:09:56.975 INFO:tasks.workunit.client.0.smithi162.stdout:8/114: truncate d0/fa 157387 0 2024-04-27T04:09:56.978 INFO:tasks.workunit.client.0.smithi162.stdout:0/148: rename d5/dc/l19 to d5/l2f 0 2024-04-27T04:09:56.979 INFO:tasks.workunit.client.1.smithi181.stdout:0/381: dwrite dc/f46 [0,4194304] 0 2024-04-27T04:09:56.981 INFO:tasks.workunit.client.1.smithi181.stdout:0/382: read - dc/d1e/d26/d41/f71 zero size 2024-04-27T04:09:56.984 INFO:tasks.workunit.client.1.smithi181.stdout:9/433: creat d0/d44/d4b/d43/f98 x:0 0 0 2024-04-27T04:09:56.985 INFO:tasks.workunit.client.1.smithi181.stdout:0/383: dread dc/f46 [0,4194304] 0 2024-04-27T04:09:56.986 INFO:tasks.workunit.client.0.smithi162.stdout:6/115: symlink d6/d1a/l28 0 2024-04-27T04:09:56.987 INFO:tasks.workunit.client.1.smithi181.stdout:1/423: unlink d1/d2/d5/d6/db/f19 0 2024-04-27T04:09:56.989 INFO:tasks.workunit.client.0.smithi162.stdout:3/83: mknod d1/d18/c1b 0 2024-04-27T04:09:56.993 INFO:tasks.workunit.client.0.smithi162.stdout:1/112: write d0/d7/d10/f12 [30523,20271] 0 2024-04-27T04:09:56.994 INFO:tasks.workunit.client.1.smithi181.stdout:3/370: symlink d0/de/d20/l7a 0 2024-04-27T04:09:56.995 INFO:tasks.workunit.client.0.smithi162.stdout:3/84: dwrite d1/d18/f19 [0,4194304] 0 2024-04-27T04:09:56.996 INFO:tasks.workunit.client.0.smithi162.stdout:3/85: fsync d1/d18/f10 0 2024-04-27T04:09:57.003 INFO:tasks.workunit.client.1.smithi181.stdout:0/384: rename dc/d1f/f37 to dc/d1f/d3b/d75/f90 0 2024-04-27T04:09:57.004 INFO:tasks.workunit.client.1.smithi181.stdout:0/385: fsync dc/d1e/d26/d41/d6f/f69 0 2024-04-27T04:09:57.005 INFO:tasks.workunit.client.0.smithi162.stdout:7/62: mknod d3/d5/d9/dc/c11 0 2024-04-27T04:09:57.005 INFO:tasks.workunit.client.0.smithi162.stdout:2/77: sync 2024-04-27T04:09:57.007 INFO:tasks.workunit.client.0.smithi162.stdout:4/110: mknod d2/c21 0 2024-04-27T04:09:57.014 INFO:tasks.workunit.client.0.smithi162.stdout:6/116: symlink d6/db/l29 0 2024-04-27T04:09:57.015 INFO:tasks.workunit.client.1.smithi181.stdout:9/434: unlink d0/d40/c33 0 2024-04-27T04:09:57.016 INFO:tasks.workunit.client.1.smithi181.stdout:9/435: write d0/d44/d4b/f80 [219095,63610] 0 2024-04-27T04:09:57.018 INFO:tasks.workunit.client.1.smithi181.stdout:0/386: mkdir dc/d1f/d6b/d7e/d91 0 2024-04-27T04:09:57.020 INFO:tasks.workunit.client.1.smithi181.stdout:1/424: symlink d1/d2/d5/l83 0 2024-04-27T04:09:57.021 INFO:tasks.workunit.client.1.smithi181.stdout:0/387: write dc/d1e/d26/d41/f4f [443182,87266] 0 2024-04-27T04:09:57.023 INFO:tasks.workunit.client.0.smithi162.stdout:8/115: dread d0/fc [0,4194304] 0 2024-04-27T04:09:57.053 INFO:tasks.workunit.client.0.smithi162.stdout:0/149: dread d5/f9 [0,4194304] 0 2024-04-27T04:09:57.054 INFO:tasks.workunit.client.0.smithi162.stdout:7/63: dread d3/d5/fa [0,4194304] 0 2024-04-27T04:09:57.054 INFO:tasks.workunit.client.0.smithi162.stdout:1/113: write d0/d7/f19 [510025,123075] 0 2024-04-27T04:09:57.054 INFO:tasks.workunit.client.0.smithi162.stdout:7/64: chown d3/d5/d9/dc 1 1 2024-04-27T04:09:57.054 INFO:tasks.workunit.client.0.smithi162.stdout:1/114: write d0/d11/f1c [966786,31546] 0 2024-04-27T04:09:57.054 INFO:tasks.workunit.client.0.smithi162.stdout:4/111: rename d2/da/fc to d2/f22 0 2024-04-27T04:09:57.054 INFO:tasks.workunit.client.0.smithi162.stdout:4/112: stat d2/da/dd/l14 0 2024-04-27T04:09:57.054 INFO:tasks.workunit.client.0.smithi162.stdout:7/65: dwrite d3/d5/f8 [0,4194304] 0 2024-04-27T04:09:57.054 INFO:tasks.workunit.client.0.smithi162.stdout:7/66: write d3/d5/f8 [2878052,71746] 0 2024-04-27T04:09:57.054 INFO:tasks.workunit.client.0.smithi162.stdout:5/96: link d8/f12 d8/f20 0 2024-04-27T04:09:57.054 INFO:tasks.workunit.client.0.smithi162.stdout:7/67: dread d3/d5/f8 [0,4194304] 0 2024-04-27T04:09:57.054 INFO:tasks.workunit.client.0.smithi162.stdout:9/88: creat d3/f21 x:0 0 0 2024-04-27T04:09:57.055 INFO:tasks.workunit.client.1.smithi181.stdout:6/293: getdents d7/dc/d1e 0 2024-04-27T04:09:57.055 INFO:tasks.workunit.client.1.smithi181.stdout:9/436: dwrite d0/f2b [0,4194304] 0 2024-04-27T04:09:57.055 INFO:tasks.workunit.client.1.smithi181.stdout:9/437: write d0/d44/f48 [3841245,23783] 0 2024-04-27T04:09:57.055 INFO:tasks.workunit.client.1.smithi181.stdout:0/388: unlink dc/d1e/f24 0 2024-04-27T04:09:57.055 INFO:tasks.workunit.client.1.smithi181.stdout:3/371: dread d0/f37 [0,4194304] 0 2024-04-27T04:09:57.055 INFO:tasks.workunit.client.1.smithi181.stdout:3/372: read d0/f37 [2486345,75947] 0 2024-04-27T04:09:57.055 INFO:tasks.workunit.client.1.smithi181.stdout:6/294: fsync d7/f59 0 2024-04-27T04:09:57.056 INFO:tasks.workunit.client.1.smithi181.stdout:9/438: mknod d0/d44/d4b/d43/d89/c99 0 2024-04-27T04:09:57.056 INFO:tasks.workunit.client.1.smithi181.stdout:0/389: dread dc/fd [0,4194304] 0 2024-04-27T04:09:57.056 INFO:tasks.workunit.client.1.smithi181.stdout:3/373: dwrite d0/de/d20/f62 [0,4194304] 0 2024-04-27T04:09:57.056 INFO:tasks.workunit.client.1.smithi181.stdout:3/374: fsync d0/de/d20/d58/d64/f6a 0 2024-04-27T04:09:57.056 INFO:tasks.workunit.client.1.smithi181.stdout:4/356: sync 2024-04-27T04:09:57.056 INFO:tasks.workunit.client.1.smithi181.stdout:3/375: write d0/de/d20/f5c [94474,68593] 0 2024-04-27T04:09:57.056 INFO:tasks.workunit.client.1.smithi181.stdout:0/390: dread dc/d1e/d26/d41/f4f [0,4194304] 0 2024-04-27T04:09:57.056 INFO:tasks.workunit.client.1.smithi181.stdout:1/425: rename d1/d2/d5/d6/d6a/f7d to d1/d2/d5/f84 0 2024-04-27T04:09:57.056 INFO:tasks.workunit.client.1.smithi181.stdout:0/391: write dc/d1f/d3b/d75/f79 [632427,71821] 0 2024-04-27T04:09:57.056 INFO:tasks.workunit.client.1.smithi181.stdout:1/426: dread - d1/d4c/d52/f76 zero size 2024-04-27T04:09:57.057 INFO:tasks.workunit.client.1.smithi181.stdout:9/439: mkdir d0/d44/d4b/d3d/d9a 0 2024-04-27T04:09:57.057 INFO:tasks.workunit.client.1.smithi181.stdout:9/440: read d0/d44/d4b/f39 [1730112,101097] 0 2024-04-27T04:09:57.058 INFO:tasks.workunit.client.0.smithi162.stdout:2/78: truncate d0/f7 2994291 0 2024-04-27T04:09:57.059 INFO:tasks.workunit.client.0.smithi162.stdout:2/79: write d0/d4/f16 [873277,81979] 0 2024-04-27T04:09:57.061 INFO:tasks.workunit.client.1.smithi181.stdout:6/295: write f6 [2404519,11992] 0 2024-04-27T04:09:57.063 INFO:tasks.workunit.client.0.smithi162.stdout:1/115: dread d0/f14 [0,4194304] 0 2024-04-27T04:09:57.064 INFO:tasks.workunit.client.1.smithi181.stdout:3/376: symlink d0/de/d20/d58/d64/l7b 0 2024-04-27T04:09:57.065 INFO:tasks.workunit.client.0.smithi162.stdout:4/113: mknod d2/da/dd/c23 0 2024-04-27T04:09:57.068 INFO:tasks.workunit.client.1.smithi181.stdout:6/296: dwrite d7/dc/d1e/f52 [0,4194304] 0 2024-04-27T04:09:57.070 INFO:tasks.workunit.client.1.smithi181.stdout:4/357: rename d0/d15/d48 to d0/d15/de/d19/d77 0 2024-04-27T04:09:57.072 INFO:tasks.workunit.client.1.smithi181.stdout:0/392: unlink dc/d1e/f30 0 2024-04-27T04:09:57.075 INFO:tasks.workunit.client.0.smithi162.stdout:5/97: dread d8/d9/fc [0,4194304] 0 2024-04-27T04:09:57.077 INFO:tasks.workunit.client.1.smithi181.stdout:9/441: write d0/f26 [3689674,120006] 0 2024-04-27T04:09:57.078 INFO:tasks.workunit.client.0.smithi162.stdout:7/68: unlink d3/lb 0 2024-04-27T04:09:57.080 INFO:tasks.workunit.client.0.smithi162.stdout:3/86: rename d1/le to d1/l1c 0 2024-04-27T04:09:57.082 INFO:tasks.workunit.client.0.smithi162.stdout:9/89: creat d3/d11/d1b/f22 x:0 0 0 2024-04-27T04:09:57.088 INFO:tasks.workunit.client.1.smithi181.stdout:9/442: rmdir d0/d44/d4b/d50 39 2024-04-27T04:09:57.089 INFO:tasks.workunit.client.1.smithi181.stdout:4/358: dwrite d0/d15/d25/d2b/d6d/f74 [0,4194304] 0 2024-04-27T04:09:57.092 INFO:tasks.workunit.client.1.smithi181.stdout:5/544: sync 2024-04-27T04:09:57.092 INFO:tasks.workunit.client.1.smithi181.stdout:8/470: sync 2024-04-27T04:09:57.092 INFO:tasks.workunit.client.1.smithi181.stdout:2/399: sync 2024-04-27T04:09:57.092 INFO:tasks.workunit.client.1.smithi181.stdout:7/344: sync 2024-04-27T04:09:57.096 INFO:tasks.workunit.client.1.smithi181.stdout:0/393: symlink dc/d1e/d26/d41/d6f/d8f/l92 0 2024-04-27T04:09:57.098 INFO:tasks.workunit.client.0.smithi162.stdout:6/117: rename d6/l12 to d6/l2a 0 2024-04-27T04:09:57.100 INFO:tasks.workunit.client.0.smithi162.stdout:5/98: creat d8/d9/f21 x:0 0 0 2024-04-27T04:09:57.101 INFO:tasks.workunit.client.1.smithi181.stdout:6/297: truncate d7/dc/d1e/f52 3569196 0 2024-04-27T04:09:57.102 INFO:tasks.workunit.client.0.smithi162.stdout:7/69: creat d3/d5/f12 x:0 0 0 2024-04-27T04:09:57.103 INFO:tasks.workunit.client.0.smithi162.stdout:7/70: chown d3/d5/d9/dc/c11 16280 1 2024-04-27T04:09:57.104 INFO:tasks.workunit.client.0.smithi162.stdout:3/87: creat d1/d18/f1d x:0 0 0 2024-04-27T04:09:57.108 INFO:tasks.workunit.client.1.smithi181.stdout:5/545: symlink d4/da3/dbc/lc3 0 2024-04-27T04:09:57.109 INFO:tasks.workunit.client.1.smithi181.stdout:5/546: readlink d4/de/d2c/d46/d4d/l73 0 2024-04-27T04:09:57.109 INFO:tasks.workunit.client.0.smithi162.stdout:0/150: creat d5/dc/d1e/f30 x:0 0 0 2024-04-27T04:09:57.110 INFO:tasks.workunit.client.1.smithi181.stdout:5/547: dread - d4/d10/d29/d92/fbf zero size 2024-04-27T04:09:57.113 INFO:tasks.workunit.client.1.smithi181.stdout:1/427: rename d1/d2/d5/d6/db/c3c to d1/d4c/d1e/d20/d53/c85 0 2024-04-27T04:09:57.116 INFO:tasks.workunit.client.1.smithi181.stdout:0/394: unlink dc/d1f/d3b/f50 0 2024-04-27T04:09:57.117 INFO:tasks.workunit.client.1.smithi181.stdout:2/400: dread d4/d9/df/f41 [0,4194304] 0 2024-04-27T04:09:57.118 INFO:tasks.workunit.client.1.smithi181.stdout:9/443: mknod d0/c9b 0 2024-04-27T04:09:57.119 INFO:tasks.workunit.client.1.smithi181.stdout:9/444: dread - d0/d44/d4b/d43/f98 zero size 2024-04-27T04:09:57.119 INFO:tasks.workunit.client.0.smithi162.stdout:4/114: mkdir d2/d5/d1c/d24 0 2024-04-27T04:09:57.120 INFO:tasks.workunit.client.1.smithi181.stdout:2/401: truncate d4/d9/d16/f4e 4215601 0 2024-04-27T04:09:57.121 INFO:tasks.workunit.client.1.smithi181.stdout:6/298: unlink d7/d31/f58 0 2024-04-27T04:09:57.123 INFO:tasks.workunit.client.0.smithi162.stdout:2/80: truncate d0/d4/f15 69810 0 2024-04-27T04:09:57.124 INFO:tasks.workunit.client.0.smithi162.stdout:6/118: creat d6/d1a/f2b x:0 0 0 2024-04-27T04:09:57.127 INFO:tasks.workunit.client.1.smithi181.stdout:6/299: dwrite d7/dc/ff [0,4194304] 0 2024-04-27T04:09:57.128 INFO:tasks.workunit.client.0.smithi162.stdout:3/88: creat d1/d18/f1e x:0 0 0 2024-04-27T04:09:57.132 INFO:tasks.workunit.client.1.smithi181.stdout:1/428: rmdir d1/d4c/d1e 39 2024-04-27T04:09:57.132 INFO:tasks.workunit.client.0.smithi162.stdout:8/116: link d0/l5 d0/de/d20/l21 0 2024-04-27T04:09:57.135 INFO:tasks.workunit.client.0.smithi162.stdout:3/89: dwrite d1/d18/f1a [0,4194304] 0 2024-04-27T04:09:57.136 INFO:tasks.workunit.client.0.smithi162.stdout:0/151: rename d5/dc/d1e/d20/f2a to d5/dc/f31 0 2024-04-27T04:09:57.140 INFO:tasks.workunit.client.0.smithi162.stdout:1/116: creat d0/f1e x:0 0 0 2024-04-27T04:09:57.141 INFO:tasks.workunit.client.1.smithi181.stdout:2/402: dread d4/d9/df/d14/f15 [0,4194304] 0 2024-04-27T04:09:57.142 INFO:tasks.workunit.client.1.smithi181.stdout:2/403: chown d4/d9/f2d 485903 1 2024-04-27T04:09:57.144 INFO:tasks.workunit.client.0.smithi162.stdout:4/115: rename d2/d5 to d2/d25 0 2024-04-27T04:09:57.144 INFO:tasks.workunit.client.1.smithi181.stdout:9/445: truncate d0/d44/d4b/f39 1953680 0 2024-04-27T04:09:57.148 INFO:tasks.workunit.client.0.smithi162.stdout:7/71: dwrite d3/d5/f8 [0,4194304] 0 2024-04-27T04:09:57.148 INFO:tasks.workunit.client.0.smithi162.stdout:6/119: rmdir d6/d1a 39 2024-04-27T04:09:57.149 INFO:tasks.workunit.client.0.smithi162.stdout:7/72: dread - d3/d5/f12 zero size 2024-04-27T04:09:57.153 INFO:tasks.workunit.client.1.smithi181.stdout:3/377: rename d0/f4 to d0/d6c/d52/d44/d56/d5b/d77/f7c 0 2024-04-27T04:09:57.156 INFO:tasks.workunit.client.0.smithi162.stdout:6/120: dwrite d6/f26 [0,4194304] 0 2024-04-27T04:09:57.156 INFO:tasks.workunit.client.1.smithi181.stdout:1/429: stat d1/d2/l6c 0 2024-04-27T04:09:57.157 INFO:tasks.workunit.client.0.smithi162.stdout:6/121: chown d6/f7 79806 1 2024-04-27T04:09:57.158 INFO:tasks.workunit.client.0.smithi162.stdout:6/122: write d6/db/f22 [318603,123753] 0 2024-04-27T04:09:57.159 INFO:tasks.workunit.client.0.smithi162.stdout:6/123: readlink d6/db/l14 0 2024-04-27T04:09:57.159 INFO:tasks.workunit.client.0.smithi162.stdout:6/124: chown d6/db/f21 697 1 2024-04-27T04:09:57.160 INFO:tasks.workunit.client.1.smithi181.stdout:0/395: mkdir dc/d93 0 2024-04-27T04:09:57.161 INFO:tasks.workunit.client.1.smithi181.stdout:0/396: read - dc/d1f/f7c zero size 2024-04-27T04:09:57.165 INFO:tasks.workunit.client.1.smithi181.stdout:1/430: dwrite d1/d2/d5/d6/f67 [0,4194304] 0 2024-04-27T04:09:57.165 INFO:tasks.workunit.client.0.smithi162.stdout:3/90: unlink d1/fb 0 2024-04-27T04:09:57.167 INFO:tasks.workunit.client.0.smithi162.stdout:3/91: fdatasync d1/d18/f1d 0 2024-04-27T04:09:57.167 INFO:tasks.workunit.client.0.smithi162.stdout:3/92: write d1/d18/f1e [299203,2928] 0 2024-04-27T04:09:57.168 INFO:tasks.workunit.client.1.smithi181.stdout:2/404: symlink d4/d9/d5d/l7f 0 2024-04-27T04:09:57.168 INFO:tasks.workunit.client.1.smithi181.stdout:1/431: read d1/f16 [226004,35201] 0 2024-04-27T04:09:57.169 INFO:tasks.workunit.client.0.smithi162.stdout:9/90: sync 2024-04-27T04:09:57.170 INFO:tasks.workunit.client.0.smithi162.stdout:1/117: creat d0/d7/f1f x:0 0 0 2024-04-27T04:09:57.171 INFO:tasks.workunit.client.0.smithi162.stdout:4/116: mknod d2/d25/d1c/c26 0 2024-04-27T04:09:57.174 INFO:tasks.workunit.client.0.smithi162.stdout:5/99: creat d8/f22 x:0 0 0 2024-04-27T04:09:57.176 INFO:tasks.workunit.client.0.smithi162.stdout:7/73: rename d3/d5/fa to d3/d5/d9/f13 0 2024-04-27T04:09:57.177 INFO:tasks.workunit.client.0.smithi162.stdout:7/74: chown d3/d5/d9/f13 377 1 2024-04-27T04:09:57.179 INFO:tasks.workunit.client.0.smithi162.stdout:0/152: write d5/dc/fe [2514887,36709] 0 2024-04-27T04:09:57.180 INFO:tasks.workunit.client.1.smithi181.stdout:9/446: write d0/f8 [1296275,49339] 0 2024-04-27T04:09:57.181 INFO:tasks.workunit.client.0.smithi162.stdout:7/75: dread d3/d5/f8 [0,4194304] 0 2024-04-27T04:09:57.181 INFO:tasks.workunit.client.0.smithi162.stdout:0/153: chown d5/dc/d1e/c28 121227942 1 2024-04-27T04:09:57.182 INFO:tasks.workunit.client.0.smithi162.stdout:8/117: symlink d0/de/d20/l22 0 2024-04-27T04:09:57.184 INFO:tasks.workunit.client.0.smithi162.stdout:6/125: mknod d6/db/c2c 0 2024-04-27T04:09:57.184 INFO:tasks.workunit.client.0.smithi162.stdout:2/81: write d0/f7 [2583050,49575] 0 2024-04-27T04:09:57.186 INFO:tasks.workunit.client.0.smithi162.stdout:2/82: chown d0/fe 0 1 2024-04-27T04:09:57.187 INFO:tasks.workunit.client.0.smithi162.stdout:7/76: dwrite d3/d5/f8 [0,4194304] 0 2024-04-27T04:09:57.188 INFO:tasks.workunit.client.0.smithi162.stdout:3/93: creat d1/d18/f1f x:0 0 0 2024-04-27T04:09:57.190 INFO:tasks.workunit.client.0.smithi162.stdout:3/94: stat d1/d18/f13 0 2024-04-27T04:09:57.190 INFO:tasks.workunit.client.1.smithi181.stdout:7/345: getdents d1/d2 0 2024-04-27T04:09:57.191 INFO:tasks.workunit.client.0.smithi162.stdout:0/154: dread d5/dc/fe [0,4194304] 0 2024-04-27T04:09:57.192 INFO:tasks.workunit.client.0.smithi162.stdout:5/100: mkdir d8/d1a/d23 0 2024-04-27T04:09:57.192 INFO:tasks.workunit.client.1.smithi181.stdout:5/548: creat d4/d10/fc4 x:0 0 0 2024-04-27T04:09:57.192 INFO:tasks.workunit.client.1.smithi181.stdout:5/549: dread - d4/fb0 zero size 2024-04-27T04:09:57.193 INFO:tasks.workunit.client.0.smithi162.stdout:0/155: chown d5/dc/f31 0 1 2024-04-27T04:09:57.195 INFO:tasks.workunit.client.0.smithi162.stdout:7/77: dwrite d3/d5/f8 [0,4194304] 0 2024-04-27T04:09:57.196 INFO:tasks.workunit.client.1.smithi181.stdout:8/471: rename d1/d6/d2f/d30/l6d to d1/d6/d88/laa 0 2024-04-27T04:09:57.199 INFO:tasks.workunit.client.0.smithi162.stdout:0/156: dread d5/f9 [4194304,4194304] 0 2024-04-27T04:09:57.200 INFO:tasks.workunit.client.1.smithi181.stdout:5/550: dwrite d4/de/d2f/d7a/f89 [0,4194304] 0 2024-04-27T04:09:57.201 INFO:tasks.workunit.client.0.smithi162.stdout:7/78: dread d3/d5/d9/f13 [0,4194304] 0 2024-04-27T04:09:57.205 INFO:tasks.workunit.client.1.smithi181.stdout:1/432: rmdir d1/d2/d5/d2c 39 2024-04-27T04:09:57.206 INFO:tasks.workunit.client.0.smithi162.stdout:4/117: truncate d2/d25/fb 2054782 0 2024-04-27T04:09:57.208 INFO:tasks.workunit.client.1.smithi181.stdout:7/346: mkdir d1/df/d1a/d6f 0 2024-04-27T04:09:57.209 INFO:tasks.workunit.client.0.smithi162.stdout:5/101: creat d8/d1a/f24 x:0 0 0 2024-04-27T04:09:57.210 INFO:tasks.workunit.client.1.smithi181.stdout:4/359: rename d0/d15/de/d19/d77/d57/l4b to d0/d15/d25/d2a/d61/l78 0 2024-04-27T04:09:57.211 INFO:tasks.workunit.client.1.smithi181.stdout:7/347: read d1/f8 [7373940,89104] 0 2024-04-27T04:09:57.211 INFO:tasks.workunit.client.1.smithi181.stdout:4/360: stat d0/d15/d5c/d69/c45 0 2024-04-27T04:09:57.212 INFO:tasks.workunit.client.1.smithi181.stdout:7/348: chown d1/l2e 0 1 2024-04-27T04:09:57.212 INFO:tasks.workunit.client.1.smithi181.stdout:4/361: chown d0/d15/d25/d2b 14863049 1 2024-04-27T04:09:57.212 INFO:tasks.workunit.client.1.smithi181.stdout:8/472: rmdir d1/d6/d2f/d40 39 2024-04-27T04:09:57.214 INFO:tasks.workunit.client.1.smithi181.stdout:8/473: readlink d1/d6/d2f/d47/d89/l9e 0 2024-04-27T04:09:57.214 INFO:tasks.workunit.client.0.smithi162.stdout:8/118: truncate d0/fa 1056445 0 2024-04-27T04:09:57.215 INFO:tasks.workunit.client.0.smithi162.stdout:8/119: readlink d0/de/d20/l21 0 2024-04-27T04:09:57.216 INFO:tasks.workunit.client.0.smithi162.stdout:0/157: mknod d5/dc/d1e/d20/c32 0 2024-04-27T04:09:57.218 INFO:tasks.workunit.client.1.smithi181.stdout:7/349: dread d1/d22/d15/d29/f4a [0,4194304] 0 2024-04-27T04:09:57.219 INFO:tasks.workunit.client.0.smithi162.stdout:8/120: dread d0/d12/f15 [0,4194304] 0 2024-04-27T04:09:57.220 INFO:tasks.workunit.client.0.smithi162.stdout:8/121: chown d0/d12/f15 3606878 1 2024-04-27T04:09:57.221 INFO:tasks.workunit.client.1.smithi181.stdout:7/350: truncate d1/d2/d1c/d55/f66 1205627 0 2024-04-27T04:09:57.221 INFO:tasks.workunit.client.1.smithi181.stdout:8/474: dwrite d1/d20/f7a [0,4194304] 0 2024-04-27T04:09:57.221 INFO:tasks.workunit.client.0.smithi162.stdout:1/118: link d0/fe d0/d11/f20 0 2024-04-27T04:09:57.222 INFO:tasks.workunit.client.0.smithi162.stdout:5/102: unlink d8/d9/l13 0 2024-04-27T04:09:57.223 INFO:tasks.workunit.client.0.smithi162.stdout:7/79: dwrite d3/d5/d9/f13 [4194304,4194304] 0 2024-04-27T04:09:57.225 INFO:tasks.workunit.client.0.smithi162.stdout:9/91: rename d3/d11/d1b to d3/d23 0 2024-04-27T04:09:57.225 INFO:tasks.workunit.client.0.smithi162.stdout:7/80: write d3/d5/f12 [438484,54224] 0 2024-04-27T04:09:57.227 INFO:tasks.workunit.client.0.smithi162.stdout:9/92: write d3/f14 [758126,38220] 0 2024-04-27T04:09:57.228 INFO:tasks.workunit.client.1.smithi181.stdout:0/397: creat dc/f94 x:0 0 0 2024-04-27T04:09:57.229 INFO:tasks.workunit.client.1.smithi181.stdout:5/551: symlink d4/d10/d29/d92/dbb/lc5 0 2024-04-27T04:09:57.229 INFO:tasks.workunit.client.0.smithi162.stdout:9/93: write d3/f6 [3593300,51608] 0 2024-04-27T04:09:57.231 INFO:tasks.workunit.client.0.smithi162.stdout:8/122: rmdir d0/de/d20 39 2024-04-27T04:09:57.231 INFO:tasks.workunit.client.0.smithi162.stdout:9/94: write d3/fc [539228,72154] 0 2024-04-27T04:09:57.233 INFO:tasks.workunit.client.0.smithi162.stdout:1/119: chown d0/f1b 221 1 2024-04-27T04:09:57.233 INFO:tasks.workunit.client.0.smithi162.stdout:5/103: symlink d8/d9/l25 0 2024-04-27T04:09:57.234 INFO:tasks.workunit.client.0.smithi162.stdout:1/120: chown d0/d7/la 460488047 1 2024-04-27T04:09:57.235 INFO:tasks.workunit.client.1.smithi181.stdout:3/378: getdents d0/de/d20/d58/d70 0 2024-04-27T04:09:57.236 INFO:tasks.workunit.client.1.smithi181.stdout:7/351: readlink d1/d2/l6 0 2024-04-27T04:09:57.240 INFO:tasks.workunit.client.1.smithi181.stdout:1/433: dread d1/d4c/f3b [0,4194304] 0 2024-04-27T04:09:57.242 INFO:tasks.workunit.client.0.smithi162.stdout:8/123: mkdir d0/de/d17/d23 0 2024-04-27T04:09:57.243 INFO:tasks.workunit.client.1.smithi181.stdout:9/447: rename d0/d4d/d85/f8d to d0/d40/f9c 0 2024-04-27T04:09:57.244 INFO:tasks.workunit.client.0.smithi162.stdout:9/95: rename d3/d23/d1e/f20 to d3/d11/f24 0 2024-04-27T04:09:57.248 INFO:tasks.workunit.client.0.smithi162.stdout:4/118: link d2/da/c16 d2/d25/d1c/c27 0 2024-04-27T04:09:57.248 INFO:tasks.workunit.client.0.smithi162.stdout:4/119: write d2/da/f12 [14777,124] 0 2024-04-27T04:09:57.251 INFO:tasks.workunit.client.0.smithi162.stdout:9/96: symlink d3/d11/l25 0 2024-04-27T04:09:57.253 INFO:tasks.workunit.client.1.smithi181.stdout:0/398: rename dc/d1e to dc/d1f/d3b/d4a/d95 0 2024-04-27T04:09:57.254 INFO:tasks.workunit.client.0.smithi162.stdout:9/97: dread d3/f6 [0,4194304] 0 2024-04-27T04:09:57.254 INFO:tasks.workunit.client.1.smithi181.stdout:9/448: unlink d0/d40/f22 0 2024-04-27T04:09:57.255 INFO:tasks.workunit.client.0.smithi162.stdout:7/81: link d3/d5/d9/dc/de/c10 d3/d5/d9/dc/de/c14 0 2024-04-27T04:09:57.256 INFO:tasks.workunit.client.0.smithi162.stdout:4/120: mknod d2/d25/c28 0 2024-04-27T04:09:57.257 INFO:tasks.workunit.client.1.smithi181.stdout:8/475: link d1/d6/d1a/f6a d1/d6/d2f/d30/d66/fab 0 2024-04-27T04:09:57.259 INFO:tasks.workunit.client.1.smithi181.stdout:0/399: mknod dc/d1f/d3b/d4a/d95/d26/d41/c96 0 2024-04-27T04:09:57.260 INFO:tasks.workunit.client.1.smithi181.stdout:9/449: fsync d0/d44/d4b/d3d/f73 0 2024-04-27T04:09:57.261 INFO:tasks.workunit.client.0.smithi162.stdout:9/98: creat d3/d13/f26 x:0 0 0 2024-04-27T04:09:57.262 INFO:tasks.workunit.client.1.smithi181.stdout:0/400: read dc/f5e [197537,92922] 0 2024-04-27T04:09:57.262 INFO:tasks.workunit.client.0.smithi162.stdout:9/99: write d3/d13/f26 [847865,17025] 0 2024-04-27T04:09:57.263 INFO:tasks.workunit.client.1.smithi181.stdout:8/476: fdatasync d1/d6/d2f/fa3 0 2024-04-27T04:09:57.265 INFO:tasks.workunit.client.0.smithi162.stdout:9/100: dread d3/fa [0,4194304] 0 2024-04-27T04:09:57.265 INFO:tasks.workunit.client.1.smithi181.stdout:9/450: mkdir d0/d44/d4f/d9d 0 2024-04-27T04:09:57.266 INFO:tasks.workunit.client.1.smithi181.stdout:1/434: dwrite d1/d4c/d1e/f49 [0,4194304] 0 2024-04-27T04:09:57.270 INFO:tasks.workunit.client.0.smithi162.stdout:9/101: dwrite d3/d13/f26 [0,4194304] 0 2024-04-27T04:09:57.270 INFO:tasks.workunit.client.1.smithi181.stdout:8/477: unlink d1/d6/d2f/fa3 0 2024-04-27T04:09:57.271 INFO:tasks.workunit.client.0.smithi162.stdout:9/102: chown d3/l7 1 1 2024-04-27T04:09:57.274 INFO:tasks.workunit.client.1.smithi181.stdout:8/478: dread - d1/d6/d2f/d40/d2a/f64 zero size 2024-04-27T04:09:57.275 INFO:tasks.workunit.client.1.smithi181.stdout:9/451: rename d0/d44/d4b/d43/f7d to d0/d44/f9e 0 2024-04-27T04:09:57.276 INFO:tasks.workunit.client.0.smithi162.stdout:4/121: getdents d2 0 2024-04-27T04:09:57.278 INFO:tasks.workunit.client.1.smithi181.stdout:9/452: creat d0/d44/d4b/d3d/f9f x:0 0 0 2024-04-27T04:09:57.279 INFO:tasks.workunit.client.0.smithi162.stdout:4/122: creat d2/da/d1a/f29 x:0 0 0 2024-04-27T04:09:57.280 INFO:tasks.workunit.client.1.smithi181.stdout:1/435: rename d1/d2/d5/d6/db/d2d/d36/c39 to d1/d2/d5/c86 0 2024-04-27T04:09:57.281 INFO:tasks.workunit.client.1.smithi181.stdout:9/453: mkdir d0/d4d/da0 0 2024-04-27T04:09:57.282 INFO:tasks.workunit.client.1.smithi181.stdout:9/454: read - d0/d4d/f7f zero size 2024-04-27T04:09:57.284 INFO:tasks.workunit.client.0.smithi162.stdout:4/123: link d2/f4 d2/da/dd/f2a 0 2024-04-27T04:09:57.284 INFO:tasks.workunit.client.1.smithi181.stdout:1/436: getdents d1/d4c/d60 0 2024-04-27T04:09:57.285 INFO:tasks.workunit.client.0.smithi162.stdout:4/124: chown d2/d25/f8 1860300 1 2024-04-27T04:09:57.285 INFO:tasks.workunit.client.1.smithi181.stdout:1/437: fdatasync d1/d2/d5/d6/db/d2d/d36/f5e 0 2024-04-27T04:09:57.288 INFO:tasks.workunit.client.1.smithi181.stdout:1/438: creat d1/d4c/d1e/d20/f87 x:0 0 0 2024-04-27T04:09:57.289 INFO:tasks.workunit.client.1.smithi181.stdout:1/439: creat d1/d4c/d60/f88 x:0 0 0 2024-04-27T04:09:57.290 INFO:tasks.workunit.client.1.smithi181.stdout:1/440: readlink d1/d2/d5/d6/l62 0 2024-04-27T04:09:57.298 INFO:tasks.workunit.client.1.smithi181.stdout:1/441: dwrite d1/d2/d5/d6/f73 [0,4194304] 0 2024-04-27T04:09:57.299 INFO:tasks.workunit.client.1.smithi181.stdout:1/442: dread - d1/d4c/d60/f88 zero size 2024-04-27T04:09:57.301 INFO:tasks.workunit.client.1.smithi181.stdout:1/443: mkdir d1/d2/d89 0 2024-04-27T04:09:57.302 INFO:tasks.workunit.client.1.smithi181.stdout:1/444: symlink d1/d4c/d52/l8a 0 2024-04-27T04:09:57.304 INFO:tasks.workunit.client.1.smithi181.stdout:1/445: read - d1/d4c/d1e/d48/f5d zero size 2024-04-27T04:09:57.347 INFO:tasks.workunit.client.1.smithi181.stdout:1/446: dwrite d1/d4c/d1e/f7a [0,4194304] 0 2024-04-27T04:09:57.356 INFO:tasks.workunit.client.1.smithi181.stdout:8/479: dread d1/d6/d2f/d40/fa1 [0,4194304] 0 2024-04-27T04:09:57.357 INFO:tasks.workunit.client.0.smithi162.stdout:3/95: rmdir d1 39 2024-04-27T04:09:57.359 INFO:tasks.workunit.client.1.smithi181.stdout:1/447: dwrite d1/d2/d5/d6/f4e [0,4194304] 0 2024-04-27T04:09:57.360 INFO:tasks.workunit.client.0.smithi162.stdout:9/103: sync 2024-04-27T04:09:57.360 INFO:tasks.workunit.client.0.smithi162.stdout:4/125: sync 2024-04-27T04:09:57.362 INFO:tasks.workunit.client.1.smithi181.stdout:8/480: creat d1/d6/d2f/d47/fac x:0 0 0 2024-04-27T04:09:57.367 INFO:tasks.workunit.client.0.smithi162.stdout:9/104: creat d3/d23/d1e/f27 x:0 0 0 2024-04-27T04:09:57.368 INFO:tasks.workunit.client.1.smithi181.stdout:1/448: dwrite d1/d2/f4 [4194304,4194304] 0 2024-04-27T04:09:57.370 INFO:tasks.workunit.client.1.smithi181.stdout:8/481: mknod d1/d6/d88/cad 0 2024-04-27T04:09:57.372 INFO:tasks.workunit.client.0.smithi162.stdout:3/96: rename d1/d18/f9 to d1/f20 0 2024-04-27T04:09:57.374 INFO:tasks.workunit.client.0.smithi162.stdout:6/126: dwrite d6/f7 [0,4194304] 0 2024-04-27T04:09:57.374 INFO:tasks.workunit.client.0.smithi162.stdout:9/105: symlink d3/l28 0 2024-04-27T04:09:57.379 INFO:tasks.workunit.client.0.smithi162.stdout:4/126: rename d2/da/l18 to d2/l2b 0 2024-04-27T04:09:57.382 INFO:tasks.workunit.client.0.smithi162.stdout:9/106: dwrite d3/d23/d1e/f27 [0,4194304] 0 2024-04-27T04:09:57.382 INFO:tasks.workunit.client.0.smithi162.stdout:3/97: mkdir d1/d21 0 2024-04-27T04:09:57.383 INFO:tasks.workunit.client.1.smithi181.stdout:1/449: dwrite d1/d4c/d1e/d20/d53/d56/f6f [0,4194304] 0 2024-04-27T04:09:57.384 INFO:tasks.workunit.client.0.smithi162.stdout:3/98: chown d1/d18/f1f 2775199 1 2024-04-27T04:09:57.386 INFO:tasks.workunit.client.0.smithi162.stdout:4/127: mkdir d2/da/d2c 0 2024-04-27T04:09:57.387 INFO:tasks.workunit.client.0.smithi162.stdout:9/107: mknod d3/d23/c29 0 2024-04-27T04:09:57.387 INFO:tasks.workunit.client.1.smithi181.stdout:8/482: dwrite d1/d6/d2f/d30/d66/fab [0,4194304] 0 2024-04-27T04:09:57.390 INFO:tasks.workunit.client.1.smithi181.stdout:8/483: truncate d1/d6/f4a 201415 0 2024-04-27T04:09:57.391 INFO:tasks.workunit.client.0.smithi162.stdout:2/83: write d0/f6 [573150,58626] 0 2024-04-27T04:09:57.392 INFO:tasks.workunit.client.0.smithi162.stdout:3/99: symlink d1/l22 0 2024-04-27T04:09:57.393 INFO:tasks.workunit.client.0.smithi162.stdout:3/100: chown d1/d18/c17 4535 1 2024-04-27T04:09:57.393 INFO:tasks.workunit.client.1.smithi181.stdout:8/484: write d1/d6/d2f/d30/d66/f9c [170797,110380] 0 2024-04-27T04:09:57.394 INFO:tasks.workunit.client.1.smithi181.stdout:1/450: dwrite d1/d4c/d1e/f49 [0,4194304] 0 2024-04-27T04:09:57.401 INFO:tasks.workunit.client.1.smithi181.stdout:4/362: rmdir d0/d15/de/d19 39 2024-04-27T04:09:57.403 INFO:tasks.workunit.client.1.smithi181.stdout:8/485: symlink d1/d6/d2f/d40/d2a/lae 0 2024-04-27T04:09:57.406 INFO:tasks.workunit.client.0.smithi162.stdout:2/84: unlink d0/d4/l11 0 2024-04-27T04:09:57.409 INFO:tasks.workunit.client.0.smithi162.stdout:3/101: mkdir d1/d18/d23 0 2024-04-27T04:09:57.410 INFO:tasks.workunit.client.0.smithi162.stdout:3/102: write d1/d18/f12 [392263,1320] 0 2024-04-27T04:09:57.415 INFO:tasks.workunit.client.0.smithi162.stdout:2/85: rmdir d0 39 2024-04-27T04:09:57.417 INFO:tasks.workunit.client.1.smithi181.stdout:4/363: creat d0/d15/de/d19/d59/f79 x:0 0 0 2024-04-27T04:09:57.421 INFO:tasks.workunit.client.0.smithi162.stdout:0/158: write d5/dc/f14 [33455,36964] 0 2024-04-27T04:09:57.423 INFO:tasks.workunit.client.0.smithi162.stdout:3/103: symlink d1/d21/l24 0 2024-04-27T04:09:57.424 INFO:tasks.workunit.client.1.smithi181.stdout:4/364: dwrite d0/d15/d25/d40/f6c [0,4194304] 0 2024-04-27T04:09:57.426 INFO:tasks.workunit.client.0.smithi162.stdout:5/104: getdents d8/d9 0 2024-04-27T04:09:57.428 INFO:tasks.workunit.client.1.smithi181.stdout:4/365: fdatasync d0/d15/d25/d2b/d6d/f71 0 2024-04-27T04:09:57.430 INFO:tasks.workunit.client.0.smithi162.stdout:2/86: truncate d0/f1 202366 0 2024-04-27T04:09:57.431 INFO:tasks.workunit.client.0.smithi162.stdout:3/104: dwrite d1/d18/f1f [0,4194304] 0 2024-04-27T04:09:57.432 INFO:tasks.workunit.client.1.smithi181.stdout:4/366: mkdir d0/d15/d25/d2a/d7a 0 2024-04-27T04:09:57.433 INFO:tasks.workunit.client.0.smithi162.stdout:2/87: stat d0/d4/d17 0 2024-04-27T04:09:57.434 INFO:tasks.workunit.client.0.smithi162.stdout:0/159: dwrite d5/f9 [4194304,4194304] 0 2024-04-27T04:09:57.435 INFO:tasks.workunit.client.1.smithi181.stdout:4/367: mknod d0/d15/de/d19/c7b 0 2024-04-27T04:09:57.436 INFO:tasks.workunit.client.0.smithi162.stdout:3/105: chown d1/d18/c16 20889910 1 2024-04-27T04:09:57.437 INFO:tasks.workunit.client.0.smithi162.stdout:3/106: chown d1/d18/d23 1374400459 1 2024-04-27T04:09:57.438 INFO:tasks.workunit.client.1.smithi181.stdout:4/368: mkdir d0/d15/d25/d2a/d4c/d7c 0 2024-04-27T04:09:57.438 INFO:tasks.workunit.client.1.smithi181.stdout:3/379: write d0/de/f17 [1337661,17004] 0 2024-04-27T04:09:57.439 INFO:tasks.workunit.client.0.smithi162.stdout:5/105: dwrite d8/d1a/f24 [0,4194304] 0 2024-04-27T04:09:57.440 INFO:tasks.workunit.client.0.smithi162.stdout:2/88: mkdir d0/d4/d1b 0 2024-04-27T04:09:57.440 INFO:tasks.workunit.client.0.smithi162.stdout:3/107: truncate d1/d18/f12 969227 0 2024-04-27T04:09:57.440 INFO:tasks.workunit.client.0.smithi162.stdout:5/106: read f4 [1964720,5990] 0 2024-04-27T04:09:57.442 INFO:tasks.workunit.client.0.smithi162.stdout:2/89: fdatasync d0/fe 0 2024-04-27T04:09:57.443 INFO:tasks.workunit.client.0.smithi162.stdout:3/108: write d1/d18/f12 [256210,41698] 0 2024-04-27T04:09:57.445 INFO:tasks.workunit.client.0.smithi162.stdout:3/109: dread - d1/d18/f13 zero size 2024-04-27T04:09:57.445 INFO:tasks.workunit.client.1.smithi181.stdout:5/552: truncate d4/d10/d72/d80/f8d 717579 0 2024-04-27T04:09:57.447 INFO:tasks.workunit.client.1.smithi181.stdout:4/369: creat d0/d15/d5c/f7d x:0 0 0 2024-04-27T04:09:57.448 INFO:tasks.workunit.client.0.smithi162.stdout:1/121: dwrite d0/f14 [4194304,4194304] 0 2024-04-27T04:09:57.449 INFO:tasks.workunit.client.1.smithi181.stdout:5/553: write d4/de/d2c/d46/d4d/d8e/f41 [8889719,65351] 0 2024-04-27T04:09:57.454 INFO:tasks.workunit.client.0.smithi162.stdout:3/110: dwrite d1/fd [0,4194304] 0 2024-04-27T04:09:57.457 INFO:tasks.workunit.client.1.smithi181.stdout:7/352: write d1/d22/d15/d1f/d34/f3c [1379034,24726] 0 2024-04-27T04:09:57.460 INFO:tasks.workunit.client.1.smithi181.stdout:5/554: dwrite d4/d10/d29/f84 [0,4194304] 0 2024-04-27T04:09:57.461 INFO:tasks.workunit.client.1.smithi181.stdout:5/555: write d4/de/d2c/f75 [2495012,107596] 0 2024-04-27T04:09:57.463 INFO:tasks.workunit.client.0.smithi162.stdout:3/111: dwrite d1/d18/f1e [0,4194304] 0 2024-04-27T04:09:57.465 INFO:tasks.workunit.client.0.smithi162.stdout:8/124: truncate d0/f7 3284806 0 2024-04-27T04:09:57.469 INFO:tasks.workunit.client.1.smithi181.stdout:2/405: sync 2024-04-27T04:09:57.469 INFO:tasks.workunit.client.1.smithi181.stdout:0/401: sync 2024-04-27T04:09:57.469 INFO:tasks.workunit.client.1.smithi181.stdout:6/300: sync 2024-04-27T04:09:57.471 INFO:tasks.workunit.client.1.smithi181.stdout:2/406: write d4/d9/f11 [5182520,109031] 0 2024-04-27T04:09:57.473 INFO:tasks.workunit.client.1.smithi181.stdout:2/407: write d4/f30 [1739484,35] 0 2024-04-27T04:09:57.473 INFO:tasks.workunit.client.1.smithi181.stdout:2/408: readlink d4/l5 0 2024-04-27T04:09:57.474 INFO:tasks.workunit.client.1.smithi181.stdout:2/409: stat d4/d9/d16/c25 0 2024-04-27T04:09:57.474 INFO:tasks.workunit.client.0.smithi162.stdout:0/160: mkdir d5/dc/d1e/d33 0 2024-04-27T04:09:57.475 INFO:tasks.workunit.client.1.smithi181.stdout:2/410: write d4/f6 [1576038,7541] 0 2024-04-27T04:09:57.484 INFO:tasks.workunit.client.0.smithi162.stdout:1/122: creat d0/d7/d10/f21 x:0 0 0 2024-04-27T04:09:57.486 INFO:tasks.workunit.client.0.smithi162.stdout:7/82: truncate d3/d5/f12 202123 0 2024-04-27T04:09:57.489 INFO:tasks.workunit.client.1.smithi181.stdout:5/556: mknod d4/de/d2c/cc6 0 2024-04-27T04:09:57.493 INFO:tasks.workunit.client.1.smithi181.stdout:3/380: rename d0/d6c/d6e to d0/de/d7d 0 2024-04-27T04:09:57.494 INFO:tasks.workunit.client.1.smithi181.stdout:5/557: rename d4/de/d2c/d46/d99 to d4/de/d2c/d46/d99/dad/dc7 22 2024-04-27T04:09:57.494 INFO:tasks.workunit.client.1.smithi181.stdout:3/381: stat d0/de/d3f/f5f 0 2024-04-27T04:09:57.495 INFO:tasks.workunit.client.0.smithi162.stdout:3/112: fsync d1/d18/f6 0 2024-04-27T04:09:57.497 INFO:tasks.workunit.client.1.smithi181.stdout:0/402: symlink dc/d1f/d6b/l97 0 2024-04-27T04:09:57.498 INFO:tasks.workunit.client.1.smithi181.stdout:0/403: chown dc/f39 91164 1 2024-04-27T04:09:57.500 INFO:tasks.workunit.client.1.smithi181.stdout:0/404: write dc/d1f/d3b/d4a/d95/f2e [224270,128529] 0 2024-04-27T04:09:57.503 INFO:tasks.workunit.client.0.smithi162.stdout:2/90: unlink d0/l12 0 2024-04-27T04:09:57.505 INFO:tasks.workunit.client.1.smithi181.stdout:6/301: mkdir d7/d31/d45/d62 0 2024-04-27T04:09:57.510 INFO:tasks.workunit.client.1.smithi181.stdout:9/455: dwrite d0/d44/d4b/d43/f82 [0,4194304] 0 2024-04-27T04:09:57.512 INFO:tasks.workunit.client.0.smithi162.stdout:7/83: rename d3/d5/f8 to d3/d5/d9/dc/f15 0 2024-04-27T04:09:57.515 INFO:tasks.workunit.client.0.smithi162.stdout:5/107: creat d8/f26 x:0 0 0 2024-04-27T04:09:57.520 INFO:tasks.workunit.client.0.smithi162.stdout:8/125: rmdir d0/de/d20 39 2024-04-27T04:09:57.522 INFO:tasks.workunit.client.1.smithi181.stdout:3/382: creat d0/de/d20/d58/f7e x:0 0 0 2024-04-27T04:09:57.524 INFO:tasks.workunit.client.0.smithi162.stdout:0/161: creat d5/dc/d1e/d33/f34 x:0 0 0 2024-04-27T04:09:57.525 INFO:tasks.workunit.client.0.smithi162.stdout:6/127: write d6/fa [1903092,43975] 0 2024-04-27T04:09:57.528 INFO:tasks.workunit.client.0.smithi162.stdout:0/162: dread d5/f9 [4194304,4194304] 0 2024-04-27T04:09:57.529 INFO:tasks.workunit.client.1.smithi181.stdout:0/405: dread dc/f15 [0,4194304] 0 2024-04-27T04:09:57.529 INFO:tasks.workunit.client.0.smithi162.stdout:6/128: dread d6/f26 [0,4194304] 0 2024-04-27T04:09:57.530 INFO:tasks.workunit.client.1.smithi181.stdout:7/353: link d1/df/c32 d1/df/d62/c70 0 2024-04-27T04:09:57.531 INFO:tasks.workunit.client.1.smithi181.stdout:7/354: chown d1/d2/d1c/d56 72 1 2024-04-27T04:09:57.532 INFO:tasks.workunit.client.0.smithi162.stdout:0/163: dwrite d5/f16 [0,4194304] 0 2024-04-27T04:09:57.533 INFO:tasks.workunit.client.1.smithi181.stdout:2/411: symlink d4/d9/d35/l80 0 2024-04-27T04:09:57.534 INFO:tasks.workunit.client.0.smithi162.stdout:9/108: write d3/d13/f26 [4610497,1207] 0 2024-04-27T04:09:57.534 INFO:tasks.workunit.client.1.smithi181.stdout:5/558: mknod d4/de/d2c/d2e/cc8 0 2024-04-27T04:09:57.535 INFO:tasks.workunit.client.1.smithi181.stdout:7/355: dread d1/d22/d15/d13/f46 [0,4194304] 0 2024-04-27T04:09:57.535 INFO:tasks.workunit.client.0.smithi162.stdout:0/164: truncate d5/dc/d1e/d33/f34 59369 0 2024-04-27T04:09:57.536 INFO:tasks.workunit.client.0.smithi162.stdout:8/126: fdatasync d0/d12/f14 0 2024-04-27T04:09:57.537 INFO:tasks.workunit.client.0.smithi162.stdout:8/127: stat d0/c18 0 2024-04-27T04:09:57.539 INFO:tasks.workunit.client.0.smithi162.stdout:4/128: dwrite d2/f9 [0,4194304] 0 2024-04-27T04:09:57.539 INFO:tasks.workunit.client.1.smithi181.stdout:1/451: write d1/d4c/d1e/d20/d53/f72 [715358,88753] 0 2024-04-27T04:09:57.539 INFO:tasks.workunit.client.0.smithi162.stdout:4/129: dread - d2/f11 zero size 2024-04-27T04:09:57.540 INFO:tasks.workunit.client.0.smithi162.stdout:0/165: dread d5/f16 [0,4194304] 0 2024-04-27T04:09:57.541 INFO:tasks.workunit.client.0.smithi162.stdout:2/91: mknod d0/c1c 0 2024-04-27T04:09:57.544 INFO:tasks.workunit.client.1.smithi181.stdout:6/302: creat d7/d31/d45/d62/f63 x:0 0 0 2024-04-27T04:09:57.544 INFO:tasks.workunit.client.1.smithi181.stdout:8/486: write d1/d6/d2f/d40/f4c [1215773,90643] 0 2024-04-27T04:09:57.550 INFO:tasks.workunit.client.1.smithi181.stdout:8/487: fsync d1/d6/d2f/d40/d2a/f37 0 2024-04-27T04:09:57.552 INFO:tasks.workunit.client.1.smithi181.stdout:3/383: dwrite d0/f23 [0,4194304] 0 2024-04-27T04:09:57.553 INFO:tasks.workunit.client.1.smithi181.stdout:1/452: dwrite d1/d2/d5/f78 [0,4194304] 0 2024-04-27T04:09:57.553 INFO:tasks.workunit.client.1.smithi181.stdout:3/384: stat d0/de/d7d 0 2024-04-27T04:09:57.559 INFO:tasks.workunit.client.1.smithi181.stdout:7/356: mkdir d1/d22/d15/d13/d71 0 2024-04-27T04:09:57.561 INFO:tasks.workunit.client.0.smithi162.stdout:5/108: mkdir d8/d1a/d23/d27 0 2024-04-27T04:09:57.572 INFO:tasks.workunit.client.1.smithi181.stdout:8/488: creat d1/d6/d2f/d30/faf x:0 0 0 2024-04-27T04:09:57.572 INFO:tasks.workunit.client.1.smithi181.stdout:6/303: dwrite d7/dc/d1e/f53 [0,4194304] 0 2024-04-27T04:09:57.573 INFO:tasks.workunit.client.1.smithi181.stdout:6/304: dread - d7/dc/d1e/d2b/f5a zero size 2024-04-27T04:09:57.574 INFO:tasks.workunit.client.1.smithi181.stdout:6/305: chown d7/f59 10473697 1 2024-04-27T04:09:57.575 INFO:tasks.workunit.client.1.smithi181.stdout:6/306: chown d7/f8 62420787 1 2024-04-27T04:09:57.575 INFO:tasks.workunit.client.1.smithi181.stdout:2/412: mkdir d4/d81 0 2024-04-27T04:09:57.576 INFO:tasks.workunit.client.1.smithi181.stdout:4/370: write d0/d15/d25/d2a/d4c/f4e [2038925,52997] 0 2024-04-27T04:09:57.579 INFO:tasks.workunit.client.0.smithi162.stdout:0/166: creat d5/f35 x:0 0 0 2024-04-27T04:09:57.579 INFO:tasks.workunit.client.1.smithi181.stdout:3/385: truncate d0/f32 959592 0 2024-04-27T04:09:57.583 INFO:tasks.workunit.client.0.smithi162.stdout:2/92: symlink d0/d4/d17/l1d 0 2024-04-27T04:09:57.584 INFO:tasks.workunit.client.1.smithi181.stdout:6/307: dwrite d7/dc/d1c/f32 [0,4194304] 0 2024-04-27T04:09:57.589 INFO:tasks.workunit.client.0.smithi162.stdout:7/84: creat d3/f16 x:0 0 0 2024-04-27T04:09:57.593 INFO:tasks.workunit.client.1.smithi181.stdout:2/413: rename d4/d9/d5d to d4/d9/d16/d82 0 2024-04-27T04:09:57.593 INFO:tasks.workunit.client.0.smithi162.stdout:7/85: dread d3/d5/d9/f13 [0,4194304] 0 2024-04-27T04:09:57.595 INFO:tasks.workunit.client.1.smithi181.stdout:4/371: creat d0/d15/d25/d2a/f7e x:0 0 0 2024-04-27T04:09:57.595 INFO:tasks.workunit.client.1.smithi181.stdout:2/414: truncate d4/d9/df/d14/f78 760607 0 2024-04-27T04:09:57.596 INFO:tasks.workunit.client.1.smithi181.stdout:2/415: chown d4/d9/df/c1c 25383 1 2024-04-27T04:09:57.598 INFO:tasks.workunit.client.0.smithi162.stdout:1/123: fsync d0/d7/d10/f21 0 2024-04-27T04:09:57.602 INFO:tasks.workunit.client.0.smithi162.stdout:4/130: getdents d2/d25/d1c/d24 0 2024-04-27T04:09:57.604 INFO:tasks.workunit.client.1.smithi181.stdout:5/559: creat d4/fc9 x:0 0 0 2024-04-27T04:09:57.606 INFO:tasks.workunit.client.0.smithi162.stdout:2/93: dread d0/f6 [0,4194304] 0 2024-04-27T04:09:57.608 INFO:tasks.workunit.client.1.smithi181.stdout:6/308: creat d7/d31/d45/f64 x:0 0 0 2024-04-27T04:09:57.609 INFO:tasks.workunit.client.1.smithi181.stdout:6/309: write d7/dc/d1c/f35 [3522091,24016] 0 2024-04-27T04:09:57.610 INFO:tasks.workunit.client.1.smithi181.stdout:7/357: mkdir d1/d57/d72 0 2024-04-27T04:09:57.613 INFO:tasks.workunit.client.0.smithi162.stdout:2/94: dread d0/d4/f16 [0,4194304] 0 2024-04-27T04:09:57.613 INFO:tasks.workunit.client.0.smithi162.stdout:6/129: rename d6/db/f23 to d6/f2d 0 2024-04-27T04:09:57.614 INFO:tasks.workunit.client.1.smithi181.stdout:7/358: dread d1/d2/f24 [0,4194304] 0 2024-04-27T04:09:57.616 INFO:tasks.workunit.client.1.smithi181.stdout:4/372: creat d0/d15/d25/d2a/d36/f7f x:0 0 0 2024-04-27T04:09:57.617 INFO:tasks.workunit.client.1.smithi181.stdout:4/373: chown d0/d15/de/d19/d77/d57/l53 6801203 1 2024-04-27T04:09:57.618 INFO:tasks.workunit.client.0.smithi162.stdout:9/109: creat d3/f2a x:0 0 0 2024-04-27T04:09:57.619 INFO:tasks.workunit.client.1.smithi181.stdout:2/416: symlink d4/d9/d44/l83 0 2024-04-27T04:09:57.619 INFO:tasks.workunit.client.1.smithi181.stdout:2/417: chown d4/d70 3870765 1 2024-04-27T04:09:57.620 INFO:tasks.workunit.client.1.smithi181.stdout:2/418: write d4/d9/d16/d82/f71 [1468949,109007] 0 2024-04-27T04:09:57.622 INFO:tasks.workunit.client.0.smithi162.stdout:9/110: dwrite f0 [0,4194304] 0 2024-04-27T04:09:57.623 INFO:tasks.workunit.client.0.smithi162.stdout:4/131: creat d2/d25/f2d x:0 0 0 2024-04-27T04:09:57.624 INFO:tasks.workunit.client.0.smithi162.stdout:9/111: write d3/d13/d15/f18 [1018252,91641] 0 2024-04-27T04:09:57.624 INFO:tasks.workunit.client.1.smithi181.stdout:9/456: chown d0/f26 354 1 2024-04-27T04:09:57.624 INFO:tasks.workunit.client.0.smithi162.stdout:4/132: dread - d2/da/dd/f10 zero size 2024-04-27T04:09:57.627 INFO:tasks.workunit.client.0.smithi162.stdout:3/113: truncate d1/d18/f1f 199335 0 2024-04-27T04:09:57.629 INFO:tasks.workunit.client.1.smithi181.stdout:5/560: mkdir d4/de/d2c/d46/d99/dad/dca 0 2024-04-27T04:09:57.637 INFO:tasks.workunit.client.0.smithi162.stdout:6/130: mknod d6/db/c2e 0 2024-04-27T04:09:57.639 INFO:tasks.workunit.client.1.smithi181.stdout:8/489: creat d1/d6/d2f/fb0 x:0 0 0 2024-04-27T04:09:57.642 INFO:tasks.workunit.client.1.smithi181.stdout:4/374: mkdir d0/d15/de/d19/d59/d80 0 2024-04-27T04:09:57.644 INFO:tasks.workunit.client.1.smithi181.stdout:4/375: write d0/d15/f2 [2880823,67992] 0 2024-04-27T04:09:57.644 INFO:tasks.workunit.client.0.smithi162.stdout:9/112: truncate d3/f5 9166335 0 2024-04-27T04:09:57.645 INFO:tasks.workunit.client.1.smithi181.stdout:0/406: dwrite dc/d1f/d3b/d4a/d95/d26/d41/f4c [0,4194304] 0 2024-04-27T04:09:57.646 INFO:tasks.workunit.client.1.smithi181.stdout:0/407: chown dc/d1f/f7c 9 1 2024-04-27T04:09:57.647 INFO:tasks.workunit.client.0.smithi162.stdout:4/133: creat d2/da/d1a/f2e x:0 0 0 2024-04-27T04:09:57.647 INFO:tasks.workunit.client.1.smithi181.stdout:0/408: fsync dc/f18 0 2024-04-27T04:09:57.651 INFO:tasks.workunit.client.1.smithi181.stdout:6/310: link d7/dc/ff d7/dc/d1e/d5c/f65 0 2024-04-27T04:09:57.653 INFO:tasks.workunit.client.0.smithi162.stdout:6/131: symlink d6/db/l2f 0 2024-04-27T04:09:57.653 INFO:tasks.workunit.client.0.smithi162.stdout:8/128: write d0/d12/f15 [476429,95058] 0 2024-04-27T04:09:57.654 INFO:tasks.workunit.client.0.smithi162.stdout:8/129: chown d0/de/d17/l1e 8701830 1 2024-04-27T04:09:57.655 INFO:tasks.workunit.client.0.smithi162.stdout:9/113: creat d3/d23/d1e/f2b x:0 0 0 2024-04-27T04:09:57.656 INFO:tasks.workunit.client.0.smithi162.stdout:8/130: dread d0/d12/f15 [0,4194304] 0 2024-04-27T04:09:57.662 INFO:tasks.workunit.client.1.smithi181.stdout:1/453: dwrite d1/d4c/d52/f68 [0,4194304] 0 2024-04-27T04:09:57.664 INFO:tasks.workunit.client.1.smithi181.stdout:2/419: creat d4/d81/f84 x:0 0 0 2024-04-27T04:09:57.671 INFO:tasks.workunit.client.0.smithi162.stdout:9/114: symlink d3/d23/d1e/l2c 0 2024-04-27T04:09:57.671 INFO:tasks.workunit.client.1.smithi181.stdout:1/454: dwrite d1/d2/d5/f4f [0,4194304] 0 2024-04-27T04:09:57.672 INFO:tasks.workunit.client.0.smithi162.stdout:0/167: write d5/dc/f18 [2774468,46964] 0 2024-04-27T04:09:57.673 INFO:tasks.workunit.client.1.smithi181.stdout:3/386: dwrite d0/f19 [0,4194304] 0 2024-04-27T04:09:57.677 INFO:tasks.workunit.client.1.smithi181.stdout:0/409: truncate dc/d1f/d3b/d4a/d95/f25 1323138 0 2024-04-27T04:09:57.678 INFO:tasks.workunit.client.0.smithi162.stdout:9/115: dwrite d3/f4 [0,4194304] 0 2024-04-27T04:09:57.679 INFO:tasks.workunit.client.0.smithi162.stdout:8/131: dread d0/fa [0,4194304] 0 2024-04-27T04:09:57.680 INFO:tasks.workunit.client.0.smithi162.stdout:8/132: chown d0/f2 139605 1 2024-04-27T04:09:57.680 INFO:tasks.workunit.client.1.smithi181.stdout:5/561: symlink d4/de/d2c/d2e/lcb 0 2024-04-27T04:09:57.682 INFO:tasks.workunit.client.1.smithi181.stdout:6/311: fdatasync d7/fd 0 2024-04-27T04:09:57.684 INFO:tasks.workunit.client.1.smithi181.stdout:3/387: dread d0/f32 [0,4194304] 0 2024-04-27T04:09:57.695 INFO:tasks.workunit.client.1.smithi181.stdout:2/420: creat d4/d81/f85 x:0 0 0 2024-04-27T04:09:57.696 INFO:tasks.workunit.client.0.smithi162.stdout:9/116: mkdir d3/d13/d2d 0 2024-04-27T04:09:57.700 INFO:tasks.workunit.client.0.smithi162.stdout:6/132: creat d6/f30 x:0 0 0 2024-04-27T04:09:57.700 INFO:tasks.workunit.client.0.smithi162.stdout:5/109: sync 2024-04-27T04:09:57.700 INFO:tasks.workunit.client.0.smithi162.stdout:1/124: sync 2024-04-27T04:09:57.700 INFO:tasks.workunit.client.0.smithi162.stdout:3/114: sync 2024-04-27T04:09:57.703 INFO:tasks.workunit.client.0.smithi162.stdout:0/168: symlink d5/d26/l36 0 2024-04-27T04:09:57.704 INFO:tasks.workunit.client.0.smithi162.stdout:0/169: read d5/dc/d1e/f21 [424815,27205] 0 2024-04-27T04:09:57.706 INFO:tasks.workunit.client.0.smithi162.stdout:8/133: dread d0/d12/f14 [0,4194304] 0 2024-04-27T04:09:57.708 INFO:tasks.workunit.client.0.smithi162.stdout:8/134: dread d0/d12/f14 [0,4194304] 0 2024-04-27T04:09:57.709 INFO:tasks.workunit.client.0.smithi162.stdout:0/170: dwrite d5/f2d [4194304,4194304] 0 2024-04-27T04:09:57.710 INFO:tasks.workunit.client.1.smithi181.stdout:6/312: mkdir d7/dc/d1e/d66 0 2024-04-27T04:09:57.710 INFO:tasks.workunit.client.1.smithi181.stdout:5/562: read d4/de/d2c/d46/d4d/d8e/f48 [820642,67223] 0 2024-04-27T04:09:57.710 INFO:tasks.workunit.client.0.smithi162.stdout:0/171: write d5/f35 [305866,107630] 0 2024-04-27T04:09:57.711 INFO:tasks.workunit.client.1.smithi181.stdout:8/490: write d1/d6/d2f/f3a [693015,88546] 0 2024-04-27T04:09:57.715 INFO:tasks.workunit.client.0.smithi162.stdout:2/95: chown d0/f1 30 1 2024-04-27T04:09:57.715 INFO:tasks.workunit.client.0.smithi162.stdout:1/125: creat d0/d11/f22 x:0 0 0 2024-04-27T04:09:57.715 INFO:tasks.workunit.client.1.smithi181.stdout:7/359: getdents d1/d2/d1c/d2d 0 2024-04-27T04:09:57.717 INFO:tasks.workunit.client.1.smithi181.stdout:7/360: fsync d1/d22/d15/d1f/d34/f3c 0 2024-04-27T04:09:57.717 INFO:tasks.workunit.client.1.smithi181.stdout:4/376: write d0/d15/d25/d2b/f6 [23858,58845] 0 2024-04-27T04:09:57.718 INFO:tasks.workunit.client.0.smithi162.stdout:4/134: dwrite d2/f4 [0,4194304] 0 2024-04-27T04:09:57.719 INFO:tasks.workunit.client.0.smithi162.stdout:4/135: chown d2/da/d1a 65897843 1 2024-04-27T04:09:57.719 INFO:tasks.workunit.client.1.smithi181.stdout:7/361: dread - d1/d22/d15/d29/f43 zero size 2024-04-27T04:09:57.721 INFO:tasks.workunit.client.0.smithi162.stdout:9/117: fdatasync d3/f5 0 2024-04-27T04:09:57.723 INFO:tasks.workunit.client.1.smithi181.stdout:9/457: rename d0/d44/d4b/d50/d6c to d0/d44/d4b/d43/d88/da1 0 2024-04-27T04:09:57.724 INFO:tasks.workunit.client.1.smithi181.stdout:9/458: write d0/d44/f51 [2208048,62126] 0 2024-04-27T04:09:57.729 INFO:tasks.workunit.client.1.smithi181.stdout:5/563: unlink d4/f96 0 2024-04-27T04:09:57.731 INFO:tasks.workunit.client.1.smithi181.stdout:8/491: creat d1/d6/d88/fb1 x:0 0 0 2024-04-27T04:09:57.733 INFO:tasks.workunit.client.0.smithi162.stdout:8/135: rename d0/ff to d0/d12/f24 0 2024-04-27T04:09:57.735 INFO:tasks.workunit.client.0.smithi162.stdout:3/115: mkdir d1/d25 0 2024-04-27T04:09:57.736 INFO:tasks.workunit.client.0.smithi162.stdout:3/116: truncate d1/d18/f10 782636 0 2024-04-27T04:09:57.736 INFO:tasks.workunit.client.0.smithi162.stdout:3/117: fdatasync d1/d18/f1d 0 2024-04-27T04:09:57.737 INFO:tasks.workunit.client.1.smithi181.stdout:1/455: creat d1/d4c/f8b x:0 0 0 2024-04-27T04:09:57.738 INFO:tasks.workunit.client.0.smithi162.stdout:0/172: mkdir d5/dc/d37 0 2024-04-27T04:09:57.738 INFO:tasks.workunit.client.0.smithi162.stdout:0/173: read - d5/f11 zero size 2024-04-27T04:09:57.738 INFO:tasks.workunit.client.1.smithi181.stdout:8/492: dwrite d1/d6/d2f/d47/fac [0,4194304] 0 2024-04-27T04:09:57.739 INFO:tasks.workunit.client.0.smithi162.stdout:0/174: truncate d5/dc/f18 3903936 0 2024-04-27T04:09:57.741 INFO:tasks.workunit.client.0.smithi162.stdout:2/96: read d0/d4/f15 [53851,9546] 0 2024-04-27T04:09:57.742 INFO:tasks.workunit.client.1.smithi181.stdout:4/377: mknod d0/d15/d25/d2a/d61/c81 0 2024-04-27T04:09:57.743 INFO:tasks.workunit.client.0.smithi162.stdout:5/110: write d8/f20 [12237142,63225] 0 2024-04-27T04:09:57.747 INFO:tasks.workunit.client.1.smithi181.stdout:3/388: write d0/d6c/f4c [524493,67574] 0 2024-04-27T04:09:57.754 INFO:tasks.workunit.client.1.smithi181.stdout:7/362: dwrite d1/d22/d15/f2a [4194304,4194304] 0 2024-04-27T04:09:57.755 INFO:tasks.workunit.client.1.smithi181.stdout:4/378: dwrite d0/d15/de/f66 [0,4194304] 0 2024-04-27T04:09:57.755 INFO:tasks.workunit.client.0.smithi162.stdout:9/118: rmdir d3/d23 39 2024-04-27T04:09:57.756 INFO:tasks.workunit.client.0.smithi162.stdout:9/119: readlink d3/d13/d15/l1d 0 2024-04-27T04:09:57.756 INFO:tasks.workunit.client.1.smithi181.stdout:7/363: readlink d1/lc 0 2024-04-27T04:09:57.760 INFO:tasks.workunit.client.0.smithi162.stdout:7/86: write d3/d5/f12 [334953,105345] 0 2024-04-27T04:09:57.761 INFO:tasks.workunit.client.1.smithi181.stdout:8/493: dread d1/f48 [0,4194304] 0 2024-04-27T04:09:57.761 INFO:tasks.workunit.client.1.smithi181.stdout:6/313: symlink d7/dc/d1c/l67 0 2024-04-27T04:09:57.767 INFO:tasks.workunit.client.0.smithi162.stdout:8/136: rename d0/d12 to d0/d10/d25 0 2024-04-27T04:09:57.768 INFO:tasks.workunit.client.0.smithi162.stdout:8/137: read d0/fd [143819,119423] 0 2024-04-27T04:09:57.771 INFO:tasks.workunit.client.1.smithi181.stdout:1/456: symlink d1/d4c/d1e/d48/l8c 0 2024-04-27T04:09:57.774 INFO:tasks.workunit.client.0.smithi162.stdout:4/136: write d2/f1f [3354085,91778] 0 2024-04-27T04:09:57.776 INFO:tasks.workunit.client.0.smithi162.stdout:2/97: mkdir d0/d4/d17/d1e 0 2024-04-27T04:09:57.776 INFO:tasks.workunit.client.1.smithi181.stdout:0/410: getdents dc/d1f/d3b/d4a/d95/d26/d41/d6f/d8f 0 2024-04-27T04:09:57.778 INFO:tasks.workunit.client.0.smithi162.stdout:2/98: dread d0/f1 [0,4194304] 0 2024-04-27T04:09:57.779 INFO:tasks.workunit.client.0.smithi162.stdout:5/111: creat d8/d1a/d23/f28 x:0 0 0 2024-04-27T04:09:57.786 INFO:tasks.workunit.client.1.smithi181.stdout:7/364: symlink d1/d22/d15/d29/l73 0 2024-04-27T04:09:57.787 INFO:tasks.workunit.client.0.smithi162.stdout:9/120: rmdir d3/d11 39 2024-04-27T04:09:57.787 INFO:tasks.workunit.client.0.smithi162.stdout:7/87: stat d3/d5/d9/dc/de/c10 0 2024-04-27T04:09:57.787 INFO:tasks.workunit.client.0.smithi162.stdout:7/88: chown d3/d5/d9/dc 522 1 2024-04-27T04:09:57.789 INFO:tasks.workunit.client.1.smithi181.stdout:6/314: dread d7/dc/f3a [0,4194304] 0 2024-04-27T04:09:57.790 INFO:tasks.workunit.client.0.smithi162.stdout:8/138: creat d0/de/d17/f26 x:0 0 0 2024-04-27T04:09:57.790 INFO:tasks.workunit.client.0.smithi162.stdout:8/139: chown d0/de/d17/d23 1939799673 1 2024-04-27T04:09:57.791 INFO:tasks.workunit.client.0.smithi162.stdout:8/140: chown d0/fc 54783925 1 2024-04-27T04:09:57.794 INFO:tasks.workunit.client.0.smithi162.stdout:4/137: creat d2/f2f x:0 0 0 2024-04-27T04:09:57.795 INFO:tasks.workunit.client.1.smithi181.stdout:0/411: mknod dc/d1f/d3b/d4a/d95/d26/d41/d6f/d8f/c98 0 2024-04-27T04:09:57.796 INFO:tasks.workunit.client.1.smithi181.stdout:8/494: dread d1/d6/d2f/f45 [0,4194304] 0 2024-04-27T04:09:57.796 INFO:tasks.workunit.client.0.smithi162.stdout:1/126: creat d0/f23 x:0 0 0 2024-04-27T04:09:57.797 INFO:tasks.workunit.client.1.smithi181.stdout:9/459: dwrite d0/d44/f46 [0,4194304] 0 2024-04-27T04:09:57.798 INFO:tasks.workunit.client.0.smithi162.stdout:5/112: mkdir d8/d1a/d29 0 2024-04-27T04:09:57.800 INFO:tasks.workunit.client.1.smithi181.stdout:7/365: creat d1/df/d1a/f74 x:0 0 0 2024-04-27T04:09:57.801 INFO:tasks.workunit.client.0.smithi162.stdout:7/89: creat d3/d5/d9/f17 x:0 0 0 2024-04-27T04:09:57.802 INFO:tasks.workunit.client.0.smithi162.stdout:7/90: write d3/d5/d9/f17 [671524,100328] 0 2024-04-27T04:09:57.803 INFO:tasks.workunit.client.1.smithi181.stdout:4/379: getdents d0/d15/de/d60/d6f 0 2024-04-27T04:09:57.804 INFO:tasks.workunit.client.0.smithi162.stdout:3/118: rename d1/l14 to d1/d18/d23/l26 0 2024-04-27T04:09:57.804 INFO:tasks.workunit.client.0.smithi162.stdout:7/91: truncate d3/d5/d9/dc/f15 4386379 0 2024-04-27T04:09:57.805 INFO:tasks.workunit.client.0.smithi162.stdout:3/119: rename d1/d21 to d1/d21/d27 22 2024-04-27T04:09:57.805 INFO:tasks.workunit.client.0.smithi162.stdout:7/92: fdatasync d3/d5/f12 0 2024-04-27T04:09:57.807 INFO:tasks.workunit.client.1.smithi181.stdout:1/457: creat d1/d2/d64/f8d x:0 0 0 2024-04-27T04:09:57.808 INFO:tasks.workunit.client.1.smithi181.stdout:3/389: dwrite d0/f57 [0,4194304] 0 2024-04-27T04:09:57.808 INFO:tasks.workunit.client.0.smithi162.stdout:4/138: creat d2/da/d1a/f30 x:0 0 0 2024-04-27T04:09:57.809 INFO:tasks.workunit.client.0.smithi162.stdout:4/139: write d2/da/dd/f10 [34881,70655] 0 2024-04-27T04:09:57.810 INFO:tasks.workunit.client.0.smithi162.stdout:1/127: fdatasync d0/d11/f20 0 2024-04-27T04:09:57.811 INFO:tasks.workunit.client.0.smithi162.stdout:1/128: fsync d0/d11/f1c 0 2024-04-27T04:09:57.811 INFO:tasks.workunit.client.0.smithi162.stdout:7/93: dwrite d3/d5/d9/f17 [0,4194304] 0 2024-04-27T04:09:57.812 INFO:tasks.workunit.client.0.smithi162.stdout:1/129: chown d0 0 1 2024-04-27T04:09:57.813 INFO:tasks.workunit.client.0.smithi162.stdout:1/130: rename d0/d11/d1d to d0/d11/d1d/d24 22 2024-04-27T04:09:57.816 INFO:tasks.workunit.client.0.smithi162.stdout:8/141: symlink d0/de/d20/l27 0 2024-04-27T04:09:57.819 INFO:tasks.workunit.client.0.smithi162.stdout:6/133: sync 2024-04-27T04:09:57.821 INFO:tasks.workunit.client.0.smithi162.stdout:0/175: getdents d5/dc/d1e/d20 0 2024-04-27T04:09:57.823 INFO:tasks.workunit.client.1.smithi181.stdout:6/315: mkdir d7/dc/d1c/d68 0 2024-04-27T04:09:57.825 INFO:tasks.workunit.client.1.smithi181.stdout:7/366: dread d1/f8 [0,4194304] 0 2024-04-27T04:09:57.825 INFO:tasks.workunit.client.0.smithi162.stdout:3/120: creat d1/d18/d23/f28 x:0 0 0 2024-04-27T04:09:57.826 INFO:tasks.workunit.client.1.smithi181.stdout:7/367: chown d1/d22 1399 1 2024-04-27T04:09:57.826 INFO:tasks.workunit.client.0.smithi162.stdout:3/121: chown d1/d18/f1e 9623369 1 2024-04-27T04:09:57.826 INFO:tasks.workunit.client.0.smithi162.stdout:0/176: dwrite d5/dc/f14 [0,4194304] 0 2024-04-27T04:09:57.827 INFO:tasks.workunit.client.0.smithi162.stdout:0/177: chown f1 173866 1 2024-04-27T04:09:57.829 INFO:tasks.workunit.client.0.smithi162.stdout:4/140: creat d2/da/f31 x:0 0 0 2024-04-27T04:09:57.835 INFO:tasks.workunit.client.0.smithi162.stdout:5/113: dwrite f4 [0,4194304] 0 2024-04-27T04:09:57.841 INFO:tasks.workunit.client.0.smithi162.stdout:5/114: dwrite d8/d9/f16 [0,4194304] 0 2024-04-27T04:09:57.841 INFO:tasks.workunit.client.0.smithi162.stdout:5/115: fsync d8/f26 0 2024-04-27T04:09:57.849 INFO:tasks.workunit.client.0.smithi162.stdout:5/116: dwrite d8/d1a/d23/f28 [0,4194304] 0 2024-04-27T04:09:57.897 INFO:tasks.workunit.client.0.smithi162.stdout:2/99: sync 2024-04-27T04:09:57.918 INFO:tasks.workunit.client.1.smithi181.stdout:8/495: truncate d1/d6/d2f/d30/f38 3219531 0 2024-04-27T04:09:57.920 INFO:tasks.workunit.client.0.smithi162.stdout:1/131: mknod d0/d11/c25 0 2024-04-27T04:09:57.920 INFO:tasks.workunit.client.0.smithi162.stdout:9/121: sync 2024-04-27T04:09:57.921 INFO:tasks.workunit.client.0.smithi162.stdout:1/132: write d0/f1b [92586,69309] 0 2024-04-27T04:09:57.921 INFO:tasks.workunit.client.0.smithi162.stdout:9/122: fsync d3/fb 0 2024-04-27T04:09:57.962 INFO:tasks.workunit.client.0.smithi162.stdout:8/142: symlink d0/de/d17/l28 0 2024-04-27T04:09:57.968 INFO:tasks.workunit.client.1.smithi181.stdout:4/380: truncate d0/d15/f50 80832 0 2024-04-27T04:09:57.968 INFO:tasks.workunit.client.0.smithi162.stdout:0/178: sync 2024-04-27T04:09:57.968 INFO:tasks.workunit.client.1.smithi181.stdout:0/412: dwrite dc/d1f/d3b/d75/f90 [0,4194304] 0 2024-04-27T04:09:57.969 INFO:tasks.workunit.client.0.smithi162.stdout:3/122: symlink d1/d18/d23/l29 0 2024-04-27T04:09:57.970 INFO:tasks.workunit.client.1.smithi181.stdout:4/381: chown d0/d15/de/d19/d77/d57 51686 1 2024-04-27T04:09:57.970 INFO:tasks.workunit.client.1.smithi181.stdout:0/413: write dc/f2c [1977134,76151] 0 2024-04-27T04:09:57.972 INFO:tasks.workunit.client.0.smithi162.stdout:4/141: rename d2/da/l19 to d2/da/dd/l32 0 2024-04-27T04:09:57.975 INFO:tasks.workunit.client.1.smithi181.stdout:9/460: symlink d0/d4d/da0/la2 0 2024-04-27T04:09:57.978 INFO:tasks.workunit.client.1.smithi181.stdout:9/461: readlink d0/d44/d4b/l41 0 2024-04-27T04:09:57.982 INFO:tasks.workunit.client.1.smithi181.stdout:0/414: dwrite dc/d1f/d3b/d75/f8c [0,4194304] 0 2024-04-27T04:09:57.982 INFO:tasks.workunit.client.1.smithi181.stdout:7/368: dwrite d1/d2/f24 [0,4194304] 0 2024-04-27T04:09:57.991 INFO:tasks.workunit.client.1.smithi181.stdout:3/390: creat d0/de/f7f x:0 0 0 2024-04-27T04:09:57.991 INFO:tasks.workunit.client.1.smithi181.stdout:3/391: chown d0/de/f17 61198 1 2024-04-27T04:09:57.993 INFO:tasks.workunit.client.0.smithi162.stdout:9/123: dread - d3/d23/f22 zero size 2024-04-27T04:09:57.995 INFO:tasks.workunit.client.1.smithi181.stdout:1/458: rename d1/d4c/d1e/d20/f87 to d1/f8e 0 2024-04-27T04:09:57.996 INFO:tasks.workunit.client.1.smithi181.stdout:3/392: dread d0/f4e [0,4194304] 0 2024-04-27T04:09:57.996 INFO:tasks.workunit.client.0.smithi162.stdout:6/134: creat d6/d1a/f31 x:0 0 0 2024-04-27T04:09:57.997 INFO:tasks.workunit.client.1.smithi181.stdout:3/393: write d0/d6c/d52/f68 [77242,71795] 0 2024-04-27T04:09:57.998 INFO:tasks.workunit.client.0.smithi162.stdout:3/123: creat d1/d18/d23/f2a x:0 0 0 2024-04-27T04:09:57.999 INFO:tasks.workunit.client.1.smithi181.stdout:6/316: link d7/d27/l36 d7/d1b/l69 0 2024-04-27T04:09:58.001 INFO:tasks.workunit.client.0.smithi162.stdout:1/133: rename d0/f23 to d0/d7/d10/f26 0 2024-04-27T04:09:58.002 INFO:tasks.workunit.client.0.smithi162.stdout:5/117: mknod d8/d1a/d29/c2a 0 2024-04-27T04:09:58.004 INFO:tasks.workunit.client.1.smithi181.stdout:7/369: mknod d1/d2/d1c/d55/c75 0 2024-04-27T04:09:58.005 INFO:tasks.workunit.client.1.smithi181.stdout:0/415: mknod dc/d1f/d3b/d4a/d95/d26/d41/d6f/d7d/c99 0 2024-04-27T04:09:58.008 INFO:tasks.workunit.client.0.smithi162.stdout:9/124: chown d3/d11/l25 13883676 1 2024-04-27T04:09:58.010 INFO:tasks.workunit.client.0.smithi162.stdout:8/143: mkdir d0/de/d29 0 2024-04-27T04:09:58.011 INFO:tasks.workunit.client.0.smithi162.stdout:8/144: chown d0/f4 201685 1 2024-04-27T04:09:58.013 INFO:tasks.workunit.client.0.smithi162.stdout:9/125: dwrite d3/f1f [0,4194304] 0 2024-04-27T04:09:58.014 INFO:tasks.workunit.client.0.smithi162.stdout:9/126: chown d3/d11/l25 0 1 2024-04-27T04:09:58.014 INFO:tasks.workunit.client.0.smithi162.stdout:8/145: dread d0/fa [0,4194304] 0 2024-04-27T04:09:58.014 INFO:tasks.workunit.client.1.smithi181.stdout:8/496: rename d1/d6/d2f/d30/d9d/ca8 to d1/d6/d88/cb2 0 2024-04-27T04:09:58.015 INFO:tasks.workunit.client.0.smithi162.stdout:8/146: dread d0/d10/d25/f14 [0,4194304] 0 2024-04-27T04:09:58.015 INFO:tasks.workunit.client.0.smithi162.stdout:3/124: creat d1/d18/d23/f2b x:0 0 0 2024-04-27T04:09:58.020 INFO:tasks.workunit.client.0.smithi162.stdout:9/127: dwrite d3/d13/d15/f19 [0,4194304] 0 2024-04-27T04:09:58.021 INFO:tasks.workunit.client.1.smithi181.stdout:6/317: mkdir d7/d27/d6a 0 2024-04-27T04:09:58.022 INFO:tasks.workunit.client.1.smithi181.stdout:6/318: stat d7/f59 0 2024-04-27T04:09:58.022 INFO:tasks.workunit.client.0.smithi162.stdout:2/100: rmdir d0/d4/d17/d1e 0 2024-04-27T04:09:58.023 INFO:tasks.workunit.client.1.smithi181.stdout:9/462: getdents d0/d44/d4b/d43/d8f 0 2024-04-27T04:09:58.024 INFO:tasks.workunit.client.0.smithi162.stdout:7/94: getdents d3/d5/d9/dc/de 0 2024-04-27T04:09:58.025 INFO:tasks.workunit.client.1.smithi181.stdout:4/382: dwrite d0/d15/de/f34 [0,4194304] 0 2024-04-27T04:09:58.026 INFO:tasks.workunit.client.1.smithi181.stdout:7/370: unlink d1/d2/d1c/d2d/l38 0 2024-04-27T04:09:58.031 INFO:tasks.workunit.client.1.smithi181.stdout:8/497: mkdir d1/d6/d2f/d40/d2a/db3 0 2024-04-27T04:09:58.031 INFO:tasks.workunit.client.0.smithi162.stdout:7/95: dwrite d3/d5/d9/f17 [0,4194304] 0 2024-04-27T04:09:58.033 INFO:tasks.workunit.client.1.smithi181.stdout:8/498: readlink d1/d20/l36 0 2024-04-27T04:09:58.035 INFO:tasks.workunit.client.0.smithi162.stdout:3/125: creat d1/d21/f2c x:0 0 0 2024-04-27T04:09:58.036 INFO:tasks.workunit.client.0.smithi162.stdout:7/96: dread d3/d5/d9/dc/f15 [0,4194304] 0 2024-04-27T04:09:58.037 INFO:tasks.workunit.client.1.smithi181.stdout:4/383: dwrite d0/d15/d5c/d69/f31 [0,4194304] 0 2024-04-27T04:09:58.039 INFO:tasks.workunit.client.1.smithi181.stdout:4/384: stat d0/d15/d5c/d69/f10 0 2024-04-27T04:09:58.039 INFO:tasks.workunit.client.0.smithi162.stdout:8/147: symlink d0/d10/d25/l2a 0 2024-04-27T04:09:58.040 INFO:tasks.workunit.client.1.smithi181.stdout:4/385: fdatasync d0/d15/d25/d2b/d6d/f71 0 2024-04-27T04:09:58.045 INFO:tasks.workunit.client.0.smithi162.stdout:0/179: rename d5/l2c to d5/dc/d1e/d20/l38 0 2024-04-27T04:09:58.050 INFO:tasks.workunit.client.1.smithi181.stdout:1/459: link d1/d4c/d1e/f49 d1/d4c/d1e/f8f 0 2024-04-27T04:09:58.051 INFO:tasks.workunit.client.0.smithi162.stdout:0/180: dwrite d5/f10 [0,4194304] 0 2024-04-27T04:09:58.052 INFO:tasks.workunit.client.1.smithi181.stdout:8/499: mknod d1/d6/d2f/d40/cb4 0 2024-04-27T04:09:58.052 INFO:tasks.workunit.client.1.smithi181.stdout:1/460: truncate d1/d4c/d1e/d48/f5d 940666 0 2024-04-27T04:09:58.053 INFO:tasks.workunit.client.1.smithi181.stdout:1/461: stat d1/d2/l6c 0 2024-04-27T04:09:58.055 INFO:tasks.workunit.client.0.smithi162.stdout:2/101: truncate d0/d4/f15 1117379 0 2024-04-27T04:09:58.061 INFO:tasks.workunit.client.1.smithi181.stdout:3/394: dwrite d0/f1 [0,4194304] 0 2024-04-27T04:09:58.063 INFO:tasks.workunit.client.1.smithi181.stdout:3/395: chown d0/de/d20/d58/f7e 7012312 1 2024-04-27T04:09:58.064 INFO:tasks.workunit.client.1.smithi181.stdout:3/396: stat d0/d6c/f60 0 2024-04-27T04:09:58.065 INFO:tasks.workunit.client.1.smithi181.stdout:3/397: chown d0/d6c/c6d 238155 1 2024-04-27T04:09:58.066 INFO:tasks.workunit.client.1.smithi181.stdout:3/398: stat d0/d6c/d52/l43 0 2024-04-27T04:09:58.068 INFO:tasks.workunit.client.1.smithi181.stdout:0/416: dread dc/d1f/d3b/f44 [0,4194304] 0 2024-04-27T04:09:58.071 INFO:tasks.workunit.client.1.smithi181.stdout:7/371: mknod d1/c76 0 2024-04-27T04:09:58.073 INFO:tasks.workunit.client.0.smithi162.stdout:7/97: rmdir d3/d5 39 2024-04-27T04:09:58.074 INFO:tasks.workunit.client.0.smithi162.stdout:5/118: truncate d8/d9/f18 205078 0 2024-04-27T04:09:58.075 INFO:tasks.workunit.client.0.smithi162.stdout:5/119: readlink d8/d1a/l1d 0 2024-04-27T04:09:58.075 INFO:tasks.workunit.client.0.smithi162.stdout:5/120: stat d8/d9/l25 0 2024-04-27T04:09:58.075 INFO:tasks.workunit.client.1.smithi181.stdout:3/399: dwrite d0/de/d3f/f5f [0,4194304] 0 2024-04-27T04:09:58.077 INFO:tasks.workunit.client.1.smithi181.stdout:3/400: write d0/f57 [2941763,83789] 0 2024-04-27T04:09:58.078 INFO:tasks.workunit.client.0.smithi162.stdout:8/148: rename d0/de/d17 to d0/d10/d2b 0 2024-04-27T04:09:58.079 INFO:tasks.workunit.client.0.smithi162.stdout:8/149: dread - d0/d10/d25/f24 zero size 2024-04-27T04:09:58.079 INFO:tasks.workunit.client.1.smithi181.stdout:3/401: write d0/de/d20/f5c [485540,31962] 0 2024-04-27T04:09:58.080 INFO:tasks.workunit.client.0.smithi162.stdout:8/150: write d0/d10/d2b/f26 [192057,105988] 0 2024-04-27T04:09:58.081 INFO:tasks.workunit.client.1.smithi181.stdout:1/462: symlink d1/d4c/d60/l90 0 2024-04-27T04:09:58.082 INFO:tasks.workunit.client.1.smithi181.stdout:8/500: read d1/d6/d2f/d30/f38 [680520,105400] 0 2024-04-27T04:09:58.083 INFO:tasks.workunit.client.0.smithi162.stdout:0/181: creat d5/dc/d1e/d33/f39 x:0 0 0 2024-04-27T04:09:58.083 INFO:tasks.workunit.client.1.smithi181.stdout:6/319: rmdir d7/d27/d6a 0 2024-04-27T04:09:58.084 INFO:tasks.workunit.client.0.smithi162.stdout:9/128: mknod d3/c2e 0 2024-04-27T04:09:58.084 INFO:tasks.workunit.client.1.smithi181.stdout:4/386: getdents d0/d15/d25/d2a/d4c/d7c 0 2024-04-27T04:09:58.086 INFO:tasks.workunit.client.1.smithi181.stdout:0/417: creat dc/d1f/d3b/d4a/d95/d26/d41/d6f/d7d/f9a x:0 0 0 2024-04-27T04:09:58.088 INFO:tasks.workunit.client.0.smithi162.stdout:6/135: getdents d6/d1a 0 2024-04-27T04:09:58.090 INFO:tasks.workunit.client.1.smithi181.stdout:3/402: dwrite d0/de/f17 [0,4194304] 0 2024-04-27T04:09:58.091 INFO:tasks.workunit.client.1.smithi181.stdout:0/418: read dc/d1f/d3b/d75/f79 [384588,90090] 0 2024-04-27T04:09:58.093 INFO:tasks.workunit.client.1.smithi181.stdout:0/419: chown dc/d1f/d3b/d4a/d95/l38 15 1 2024-04-27T04:09:58.094 INFO:tasks.workunit.client.1.smithi181.stdout:1/463: creat d1/d2/d5/d6/db/f91 x:0 0 0 2024-04-27T04:09:58.094 INFO:tasks.workunit.client.1.smithi181.stdout:0/420: chown dc/d1f/d3b/d4a/d95/f63 238990 1 2024-04-27T04:09:58.095 INFO:tasks.workunit.client.1.smithi181.stdout:6/320: creat d7/d31/d45/f6b x:0 0 0 2024-04-27T04:09:58.096 INFO:tasks.workunit.client.0.smithi162.stdout:9/129: creat d3/d13/d15/f2f x:0 0 0 2024-04-27T04:09:58.096 INFO:tasks.workunit.client.1.smithi181.stdout:0/421: write dc/f1b [3205464,69435] 0 2024-04-27T04:09:58.097 INFO:tasks.workunit.client.0.smithi162.stdout:9/130: write d3/f14 [1699249,20411] 0 2024-04-27T04:09:58.097 INFO:tasks.workunit.client.1.smithi181.stdout:8/501: truncate d1/d6/f26 6189018 0 2024-04-27T04:09:58.099 INFO:tasks.workunit.client.1.smithi181.stdout:8/502: write d1/d6/d88/fb1 [107829,1896] 0 2024-04-27T04:09:58.101 INFO:tasks.workunit.client.1.smithi181.stdout:8/503: readlink d1/d6/d2f/d40/d2a/l9f 0 2024-04-27T04:09:58.102 INFO:tasks.workunit.client.0.smithi162.stdout:4/142: sync 2024-04-27T04:09:58.102 INFO:tasks.workunit.client.1.smithi181.stdout:4/387: mkdir d0/d15/d25/d2b/d6d/d82 0 2024-04-27T04:09:58.103 INFO:tasks.workunit.client.0.smithi162.stdout:4/143: chown d2/da/dd 14377 1 2024-04-27T04:09:58.103 INFO:tasks.workunit.client.0.smithi162.stdout:4/144: fsync d2/ff 0 2024-04-27T04:09:58.104 INFO:tasks.workunit.client.0.smithi162.stdout:2/102: truncate d0/fe 1734491 0 2024-04-27T04:09:58.104 INFO:tasks.workunit.client.0.smithi162.stdout:4/145: chown d2/d25/f2d 24643 1 2024-04-27T04:09:58.104 INFO:tasks.workunit.client.1.smithi181.stdout:4/388: write d0/d15/d25/d2b/f2c [514368,77026] 0 2024-04-27T04:09:58.107 INFO:tasks.workunit.client.0.smithi162.stdout:6/136: mkdir d6/db/d32 0 2024-04-27T04:09:58.109 INFO:tasks.workunit.client.1.smithi181.stdout:8/504: dwrite d1/d6/d2f/d47/fac [4194304,4194304] 0 2024-04-27T04:09:58.109 INFO:tasks.workunit.client.1.smithi181.stdout:3/403: rename d0/d28/l29 to d0/d6c/d66/l80 0 2024-04-27T04:09:58.110 INFO:tasks.workunit.client.1.smithi181.stdout:8/505: chown d1/d6/d1a 970 1 2024-04-27T04:09:58.110 INFO:tasks.workunit.client.0.smithi162.stdout:4/146: dwrite d2/da/f31 [0,4194304] 0 2024-04-27T04:09:58.110 INFO:tasks.workunit.client.0.smithi162.stdout:4/147: chown d2/f1f 2114178182 1 2024-04-27T04:09:58.111 INFO:tasks.workunit.client.1.smithi181.stdout:8/506: chown d1/d6/d2f/d72/f94 243 1 2024-04-27T04:09:58.113 INFO:tasks.workunit.client.0.smithi162.stdout:3/126: rmdir d1/d25 0 2024-04-27T04:09:58.122 INFO:tasks.workunit.client.0.smithi162.stdout:8/151: mknod d0/d10/d2b/d23/c2c 0 2024-04-27T04:09:58.124 INFO:tasks.workunit.client.1.smithi181.stdout:6/321: dread d7/fa [0,4194304] 0 2024-04-27T04:09:58.124 INFO:tasks.workunit.client.1.smithi181.stdout:0/422: creat dc/d1f/d3b/d4a/d95/d26/d41/d6f/d7d/f9b x:0 0 0 2024-04-27T04:09:58.126 INFO:tasks.workunit.client.0.smithi162.stdout:0/182: creat d5/dc/d37/f3a x:0 0 0 2024-04-27T04:09:58.126 INFO:tasks.workunit.client.0.smithi162.stdout:0/183: chown d5/f11 52 1 2024-04-27T04:09:58.132 INFO:tasks.workunit.client.0.smithi162.stdout:0/184: dwrite d5/dc/d1e/d33/f34 [0,4194304] 0 2024-04-27T04:09:58.132 INFO:tasks.workunit.client.1.smithi181.stdout:4/389: symlink d0/d15/de/d60/l83 0 2024-04-27T04:09:58.132 INFO:tasks.workunit.client.0.smithi162.stdout:0/185: fsync d5/fa 0 2024-04-27T04:09:58.133 INFO:tasks.workunit.client.1.smithi181.stdout:4/390: fsync d0/d15/d25/d2a/d4c/f4e 0 2024-04-27T04:09:58.137 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:09:57 smithi162 ceph-mon[26841]: pgmap v264: 65 pgs: 65 active+clean; 928 MiB data, 4.3 GiB used, 532 GiB / 536 GiB avail; 12 MiB/s rd, 101 MiB/s wr, 266 op/s 2024-04-27T04:09:58.139 INFO:tasks.workunit.client.1.smithi181.stdout:3/404: creat d0/de/d20/d58/d70/f81 x:0 0 0 2024-04-27T04:09:58.139 INFO:tasks.workunit.client.1.smithi181.stdout:4/391: dread d0/d15/de/f66 [0,4194304] 0 2024-04-27T04:09:58.142 INFO:tasks.workunit.client.1.smithi181.stdout:8/507: unlink d1/d6/f7b 0 2024-04-27T04:09:58.143 INFO:tasks.workunit.client.0.smithi162.stdout:6/137: mknod d6/db/c33 0 2024-04-27T04:09:58.144 INFO:tasks.workunit.client.1.smithi181.stdout:7/372: chown d1/d22/d15/d1f/d34 0 1 2024-04-27T04:09:58.145 INFO:tasks.workunit.client.0.smithi162.stdout:4/148: chown d2/d25/fb 2626 1 2024-04-27T04:09:58.148 INFO:tasks.workunit.client.0.smithi162.stdout:5/121: rename d8/d1a/l1e to d8/l2b 0 2024-04-27T04:09:58.149 INFO:tasks.workunit.client.0.smithi162.stdout:5/122: fsync d8/f20 0 2024-04-27T04:09:58.149 INFO:tasks.workunit.client.0.smithi162.stdout:5/123: write d8/d1a/f17 [1562828,42370] 0 2024-04-27T04:09:58.150 INFO:tasks.workunit.client.1.smithi181.stdout:1/464: unlink d1/d2/d5/l50 0 2024-04-27T04:09:58.155 INFO:tasks.workunit.client.0.smithi162.stdout:5/124: dwrite d8/d1a/d23/f28 [0,4194304] 0 2024-04-27T04:09:58.155 INFO:tasks.workunit.client.1.smithi181.stdout:0/423: creat dc/d54/f9c x:0 0 0 2024-04-27T04:09:58.156 INFO:tasks.workunit.client.1.smithi181.stdout:0/424: chown dc/f1b 21462998 1 2024-04-27T04:09:58.156 INFO:tasks.workunit.client.0.smithi162.stdout:9/131: mkdir d3/d13/d2d/d30 0 2024-04-27T04:09:58.161 INFO:tasks.workunit.client.0.smithi162.stdout:9/132: dread d3/d13/d15/f18 [0,4194304] 0 2024-04-27T04:09:58.161 INFO:tasks.workunit.client.0.smithi162.stdout:9/133: dread - d3/d23/f22 zero size 2024-04-27T04:09:58.163 INFO:tasks.workunit.client.1.smithi181.stdout:3/405: symlink d0/d6c/d52/d44/l82 0 2024-04-27T04:09:58.164 INFO:tasks.workunit.client.1.smithi181.stdout:3/406: chown d0 0 1 2024-04-27T04:09:58.164 INFO:tasks.workunit.client.0.smithi162.stdout:0/186: dread d5/dc/fe [0,4194304] 0 2024-04-27T04:09:58.165 INFO:tasks.workunit.client.0.smithi162.stdout:2/103: symlink d0/l1f 0 2024-04-27T04:09:58.165 INFO:tasks.workunit.client.1.smithi181.stdout:3/407: write d0/de/d3f/f5f [3027597,73335] 0 2024-04-27T04:09:58.167 INFO:tasks.workunit.client.1.smithi181.stdout:4/392: creat d0/d15/de/d19/d77/d57/f84 x:0 0 0 2024-04-27T04:09:58.168 INFO:tasks.workunit.client.1.smithi181.stdout:4/393: readlink d0/d15/de/d19/d77/d57/l53 0 2024-04-27T04:09:58.169 INFO:tasks.workunit.client.0.smithi162.stdout:1/134: sync 2024-04-27T04:09:58.169 INFO:tasks.workunit.client.0.smithi162.stdout:1/135: stat d0/d11/c25 0 2024-04-27T04:09:58.170 INFO:tasks.workunit.client.1.smithi181.stdout:8/508: readlink d1/d6/d1a/l79 0 2024-04-27T04:09:58.170 INFO:tasks.workunit.client.0.smithi162.stdout:1/136: readlink d0/d7/la 0 2024-04-27T04:09:58.171 INFO:tasks.workunit.client.1.smithi181.stdout:8/509: truncate d1/d6/d2f/d30/d5c/f97 496185 0 2024-04-27T04:09:58.172 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:09:57 smithi181 ceph-mon[33774]: pgmap v264: 65 pgs: 65 active+clean; 928 MiB data, 4.3 GiB used, 532 GiB / 536 GiB avail; 12 MiB/s rd, 101 MiB/s wr, 266 op/s 2024-04-27T04:09:58.174 INFO:tasks.workunit.client.0.smithi162.stdout:4/149: write d2/f22 [4932155,100718] 0 2024-04-27T04:09:58.175 INFO:tasks.workunit.client.0.smithi162.stdout:8/152: write d0/d10/d25/f14 [526237,44241] 0 2024-04-27T04:09:58.180 INFO:tasks.workunit.client.0.smithi162.stdout:3/127: rename d1/fd to d1/d18/f2d 0 2024-04-27T04:09:58.181 INFO:tasks.workunit.client.1.smithi181.stdout:6/322: dwrite d7/f26 [4194304,4194304] 0 2024-04-27T04:09:58.191 INFO:tasks.workunit.client.1.smithi181.stdout:4/394: dread d0/d15/d25/d2a/d36/f42 [0,4194304] 0 2024-04-27T04:09:58.191 INFO:tasks.workunit.client.0.smithi162.stdout:5/125: truncate d8/d9/f10 1210125 0 2024-04-27T04:09:58.196 INFO:tasks.workunit.client.0.smithi162.stdout:6/138: dwrite d6/d1a/f1d [0,4194304] 0 2024-04-27T04:09:58.205 INFO:tasks.workunit.client.0.smithi162.stdout:4/150: creat d2/d25/d1c/f33 x:0 0 0 2024-04-27T04:09:58.207 INFO:tasks.workunit.client.0.smithi162.stdout:8/153: truncate d0/d10/d25/f15 308661 0 2024-04-27T04:09:58.208 INFO:tasks.workunit.client.1.smithi181.stdout:4/395: dread d0/d15/f2 [0,4194304] 0 2024-04-27T04:09:58.210 INFO:tasks.workunit.client.0.smithi162.stdout:3/128: creat d1/d18/d23/f2e x:0 0 0 2024-04-27T04:09:58.213 INFO:tasks.workunit.client.0.smithi162.stdout:8/154: dwrite d0/f4 [0,4194304] 0 2024-04-27T04:09:58.215 INFO:tasks.workunit.client.0.smithi162.stdout:0/187: dwrite d5/f9 [0,4194304] 0 2024-04-27T04:09:58.217 INFO:tasks.workunit.client.0.smithi162.stdout:6/139: truncate d6/f26 245704 0 2024-04-27T04:09:58.218 INFO:tasks.workunit.client.0.smithi162.stdout:6/140: readlink d6/db/l2f 0 2024-04-27T04:09:58.219 INFO:tasks.workunit.client.1.smithi181.stdout:3/408: mkdir d0/d6c/d83 0 2024-04-27T04:09:58.220 INFO:tasks.workunit.client.0.smithi162.stdout:6/141: read d6/f9 [203461,12248] 0 2024-04-27T04:09:58.220 INFO:tasks.workunit.client.0.smithi162.stdout:8/155: read d0/fd [60952,81275] 0 2024-04-27T04:09:58.223 INFO:tasks.workunit.client.0.smithi162.stdout:2/104: mkdir d0/d20 0 2024-04-27T04:09:58.226 INFO:tasks.workunit.client.1.smithi181.stdout:3/409: dwrite d0/de/d20/f5d [0,4194304] 0 2024-04-27T04:09:58.231 INFO:tasks.workunit.client.0.smithi162.stdout:3/129: mkdir d1/d18/d23/d2f 0 2024-04-27T04:09:58.238 INFO:tasks.workunit.client.0.smithi162.stdout:0/188: creat d5/f3b x:0 0 0 2024-04-27T04:09:58.238 INFO:tasks.workunit.client.0.smithi162.stdout:0/189: fsync d5/dc/f14 0 2024-04-27T04:09:58.239 INFO:tasks.workunit.client.1.smithi181.stdout:1/465: rename d1/d2/d5/d2c to d1/d4c/d52/d92 0 2024-04-27T04:09:58.242 INFO:tasks.workunit.client.1.smithi181.stdout:0/425: mkdir dc/d1f/d3b/d4a/d7a/d9d 0 2024-04-27T04:09:58.243 INFO:tasks.workunit.client.0.smithi162.stdout:6/142: write d6/f2d [1818305,4228] 0 2024-04-27T04:09:58.245 INFO:tasks.workunit.client.0.smithi162.stdout:8/156: mkdir d0/d10/d2b/d2d 0 2024-04-27T04:09:58.247 INFO:tasks.workunit.client.1.smithi181.stdout:7/373: creat d1/d57/f77 x:0 0 0 2024-04-27T04:09:58.248 INFO:tasks.workunit.client.0.smithi162.stdout:4/151: symlink d2/d25/d1c/d24/l34 0 2024-04-27T04:09:58.248 INFO:tasks.workunit.client.0.smithi162.stdout:6/143: dwrite d6/d1a/f2b [0,4194304] 0 2024-04-27T04:09:58.249 INFO:tasks.workunit.client.1.smithi181.stdout:8/510: link d1/f3c d1/d6/d2f/d30/fb5 0 2024-04-27T04:09:58.249 INFO:tasks.workunit.client.0.smithi162.stdout:4/152: write d2/d25/f7 [6195374,100592] 0 2024-04-27T04:09:58.250 INFO:tasks.workunit.client.0.smithi162.stdout:4/153: readlink d2/da/dd/l14 0 2024-04-27T04:09:58.250 INFO:tasks.workunit.client.0.smithi162.stdout:4/154: read d2/f9 [2476099,80657] 0 2024-04-27T04:09:58.251 INFO:tasks.workunit.client.1.smithi181.stdout:4/396: write d0/d15/d5c/d69/f10 [137663,125123] 0 2024-04-27T04:09:58.254 INFO:tasks.workunit.client.0.smithi162.stdout:3/130: rmdir d1/d18 39 2024-04-27T04:09:58.255 INFO:tasks.workunit.client.0.smithi162.stdout:9/134: getdents d3/d11 0 2024-04-27T04:09:58.257 INFO:tasks.workunit.client.1.smithi181.stdout:1/466: creat d1/d4c/d52/d92/f93 x:0 0 0 2024-04-27T04:09:58.258 INFO:tasks.workunit.client.1.smithi181.stdout:0/426: dread - dc/f3a zero size 2024-04-27T04:09:58.260 INFO:tasks.workunit.client.0.smithi162.stdout:7/98: sync 2024-04-27T04:09:58.261 INFO:tasks.workunit.client.1.smithi181.stdout:7/374: read d1/d2/f12 [9229957,84660] 0 2024-04-27T04:09:58.262 INFO:tasks.workunit.client.0.smithi162.stdout:8/157: unlink d0/fa 0 2024-04-27T04:09:58.265 INFO:tasks.workunit.client.1.smithi181.stdout:0/427: dwrite dc/d1f/f48 [4194304,4194304] 0 2024-04-27T04:09:58.265 INFO:tasks.workunit.client.0.smithi162.stdout:8/158: dread d0/fc [0,4194304] 0 2024-04-27T04:09:58.267 INFO:tasks.workunit.client.0.smithi162.stdout:6/144: mkdir d6/d1a/d34 0 2024-04-27T04:09:58.269 INFO:tasks.workunit.client.1.smithi181.stdout:1/467: unlink d1/f16 0 2024-04-27T04:09:58.279 INFO:tasks.workunit.client.1.smithi181.stdout:7/375: truncate d1/f45 5047567 0 2024-04-27T04:09:58.282 INFO:tasks.workunit.client.0.smithi162.stdout:3/131: dwrite d1/d18/f10 [0,4194304] 0 2024-04-27T04:09:58.282 INFO:tasks.workunit.client.0.smithi162.stdout:8/159: mkdir d0/d10/d2b/d2e 0 2024-04-27T04:09:58.283 INFO:tasks.workunit.client.1.smithi181.stdout:0/428: creat dc/d1f/d3b/d4a/d95/d26/d41/f9e x:0 0 0 2024-04-27T04:09:58.286 INFO:tasks.workunit.client.1.smithi181.stdout:3/410: write d0/d6c/d52/f65 [229386,26708] 0 2024-04-27T04:09:58.287 INFO:tasks.workunit.client.0.smithi162.stdout:0/190: dwrite d5/f16 [0,4194304] 0 2024-04-27T04:09:58.290 INFO:tasks.workunit.client.1.smithi181.stdout:6/323: write d7/dc/d1e/f4d [164371,84110] 0 2024-04-27T04:09:58.290 INFO:tasks.workunit.client.0.smithi162.stdout:2/105: link d0/d4/c14 d0/d20/c21 0 2024-04-27T04:09:58.294 INFO:tasks.workunit.client.0.smithi162.stdout:4/155: creat d2/da/d2c/f35 x:0 0 0 2024-04-27T04:09:58.296 INFO:tasks.workunit.client.0.smithi162.stdout:5/126: sync 2024-04-27T04:09:58.296 INFO:tasks.workunit.client.0.smithi162.stdout:1/137: sync 2024-04-27T04:09:58.296 INFO:tasks.workunit.client.0.smithi162.stdout:5/127: write d8/f22 [791395,23212] 0 2024-04-27T04:09:58.299 INFO:tasks.workunit.client.1.smithi181.stdout:6/324: dwrite d7/dc/f12 [0,4194304] 0 2024-04-27T04:09:58.300 INFO:tasks.workunit.client.0.smithi162.stdout:9/135: mkdir d3/d13/d2d/d30/d31 0 2024-04-27T04:09:58.300 INFO:tasks.workunit.client.0.smithi162.stdout:9/136: truncate d3/fb 4633887 0 2024-04-27T04:09:58.302 INFO:tasks.workunit.client.0.smithi162.stdout:5/128: dwrite d8/d1a/d23/f28 [0,4194304] 0 2024-04-27T04:09:58.303 INFO:tasks.workunit.client.0.smithi162.stdout:5/129: write d8/f20 [6891914,25765] 0 2024-04-27T04:09:58.305 INFO:tasks.workunit.client.1.smithi181.stdout:1/468: write d1/d4c/d1e/f49 [367971,24648] 0 2024-04-27T04:09:58.306 INFO:tasks.workunit.client.0.smithi162.stdout:5/130: write d8/f20 [9848525,30663] 0 2024-04-27T04:09:58.309 INFO:tasks.workunit.client.1.smithi181.stdout:3/411: symlink d0/d6c/d52/d44/d56/d5b/d77/l84 0 2024-04-27T04:09:58.310 INFO:tasks.workunit.client.0.smithi162.stdout:9/137: dwrite d3/d13/d15/f2f [0,4194304] 0 2024-04-27T04:09:58.311 INFO:tasks.workunit.client.1.smithi181.stdout:8/511: getdents d1/d6/d2f/d30/d66 0 2024-04-27T04:09:58.311 INFO:tasks.workunit.client.0.smithi162.stdout:3/132: write d1/d18/f2d [3453946,19633] 0 2024-04-27T04:09:58.312 INFO:tasks.workunit.client.0.smithi162.stdout:8/160: write d0/d10/d25/f24 [253965,20244] 0 2024-04-27T04:09:58.315 INFO:tasks.workunit.client.1.smithi181.stdout:7/376: rename d1/df/c31 to d1/d22/d15/d13/d71/c78 0 2024-04-27T04:09:58.316 INFO:tasks.workunit.client.1.smithi181.stdout:4/397: getdents d0/d15/d25/d2a/d4c 0 2024-04-27T04:09:58.317 INFO:tasks.workunit.client.0.smithi162.stdout:8/161: dwrite d0/f4 [0,4194304] 0 2024-04-27T04:09:58.318 INFO:tasks.workunit.client.1.smithi181.stdout:7/377: truncate d1/d22/d15/d13/f46 4585391 0 2024-04-27T04:09:58.319 INFO:tasks.workunit.client.0.smithi162.stdout:8/162: truncate d0/d10/d25/f1d 559086 0 2024-04-27T04:09:58.319 INFO:tasks.workunit.client.1.smithi181.stdout:7/378: stat d1/d22/d15/d13/d1b/f47 0 2024-04-27T04:09:58.320 INFO:tasks.workunit.client.1.smithi181.stdout:6/325: creat d7/dc/d1c/d34/d3f/f6c x:0 0 0 2024-04-27T04:09:58.326 INFO:tasks.workunit.client.0.smithi162.stdout:4/156: creat d2/d25/f36 x:0 0 0 2024-04-27T04:09:58.327 INFO:tasks.workunit.client.1.smithi181.stdout:0/429: write dc/d1f/d3b/f44 [3241444,98406] 0 2024-04-27T04:09:58.328 INFO:tasks.workunit.client.1.smithi181.stdout:1/469: rmdir d1/d2/d5/d6/d6a 39 2024-04-27T04:09:58.332 INFO:tasks.workunit.client.1.smithi181.stdout:8/512: creat d1/d6/d88/fb6 x:0 0 0 2024-04-27T04:09:58.333 INFO:tasks.workunit.client.0.smithi162.stdout:7/99: creat d3/f18 x:0 0 0 2024-04-27T04:09:58.336 INFO:tasks.workunit.client.1.smithi181.stdout:7/379: symlink d1/d22/d15/d13/l79 0 2024-04-27T04:09:58.337 INFO:tasks.workunit.client.0.smithi162.stdout:3/133: symlink d1/d21/l30 0 2024-04-27T04:09:58.340 INFO:tasks.workunit.client.1.smithi181.stdout:1/470: rmdir d1/d4c/d1e 39 2024-04-27T04:09:58.342 INFO:tasks.workunit.client.0.smithi162.stdout:6/145: rename d6/f17 to d6/f35 0 2024-04-27T04:09:58.343 INFO:tasks.workunit.client.0.smithi162.stdout:6/146: write d6/d1a/f24 [448879,54465] 0 2024-04-27T04:09:58.343 INFO:tasks.workunit.client.1.smithi181.stdout:7/380: dwrite d1/d22/f65 [0,4194304] 0 2024-04-27T04:09:58.345 INFO:tasks.workunit.client.0.smithi162.stdout:0/191: link d5/f11 d5/d26/f3c 0 2024-04-27T04:09:58.345 INFO:tasks.workunit.client.1.smithi181.stdout:7/381: write d1/d22/d15/d1f/d34/f3c [2215931,86823] 0 2024-04-27T04:09:58.346 INFO:tasks.workunit.client.1.smithi181.stdout:8/513: symlink d1/d6/d2f/d40/lb7 0 2024-04-27T04:09:58.346 INFO:tasks.workunit.client.1.smithi181.stdout:7/382: stat d1/d2/l6 0 2024-04-27T04:09:58.347 INFO:tasks.workunit.client.1.smithi181.stdout:4/398: symlink d0/l85 0 2024-04-27T04:09:58.348 INFO:tasks.workunit.client.1.smithi181.stdout:6/326: mkdir d7/dc/d1c/d68/d6d 0 2024-04-27T04:09:58.349 INFO:tasks.workunit.client.1.smithi181.stdout:6/327: write d7/f59 [5772185,27031] 0 2024-04-27T04:09:58.350 INFO:tasks.workunit.client.0.smithi162.stdout:4/157: symlink d2/d25/l37 0 2024-04-27T04:09:58.353 INFO:tasks.workunit.client.1.smithi181.stdout:2/421: sync 2024-04-27T04:09:58.353 INFO:tasks.workunit.client.1.smithi181.stdout:5/564: sync 2024-04-27T04:09:58.353 INFO:tasks.workunit.client.1.smithi181.stdout:9/463: sync 2024-04-27T04:09:58.355 INFO:tasks.workunit.client.0.smithi162.stdout:2/106: dwrite d0/f6 [0,4194304] 0 2024-04-27T04:09:58.357 INFO:tasks.workunit.client.1.smithi181.stdout:0/430: dread f8 [0,4194304] 0 2024-04-27T04:09:58.358 INFO:tasks.workunit.client.0.smithi162.stdout:3/134: symlink d1/d18/d23/l31 0 2024-04-27T04:09:58.358 INFO:tasks.workunit.client.0.smithi162.stdout:3/135: read - d1/d18/f13 zero size 2024-04-27T04:09:58.363 INFO:tasks.workunit.client.1.smithi181.stdout:1/471: dwrite d1/d2/d5/d6/db/f40 [0,4194304] 0 2024-04-27T04:09:58.364 INFO:tasks.workunit.client.1.smithi181.stdout:1/472: read - d1/d2/d5/f84 zero size 2024-04-27T04:09:58.365 INFO:tasks.workunit.client.1.smithi181.stdout:1/473: readlink d1/d2/d5/l6b 0 2024-04-27T04:09:58.380 INFO:tasks.workunit.client.1.smithi181.stdout:5/565: dread d4/da9/fc2 [0,4194304] 0 2024-04-27T04:09:58.386 INFO:tasks.workunit.client.1.smithi181.stdout:5/566: dread d4/de/d2c/f3e [0,4194304] 0 2024-04-27T04:09:58.401 INFO:tasks.workunit.client.1.smithi181.stdout:5/567: dread d4/de/d2c/d46/d4d/d8e/f8a [0,4194304] 0 2024-04-27T04:09:58.450 INFO:tasks.workunit.client.0.smithi162.stdout:1/138: sync 2024-04-27T04:09:58.451 INFO:tasks.workunit.client.0.smithi162.stdout:1/139: write d0/f1b [893241,103138] 0 2024-04-27T04:09:58.466 INFO:tasks.workunit.client.1.smithi181.stdout:4/399: unlink d0/f3d 0 2024-04-27T04:09:58.467 INFO:tasks.workunit.client.0.smithi162.stdout:6/147: mknod d6/db/c36 0 2024-04-27T04:09:58.468 INFO:tasks.workunit.client.0.smithi162.stdout:6/148: chown d6/db/c2e 12656412 1 2024-04-27T04:09:58.472 INFO:tasks.workunit.client.0.smithi162.stdout:8/163: truncate d0/fd 332053 0 2024-04-27T04:09:58.473 INFO:tasks.workunit.client.0.smithi162.stdout:0/192: sync 2024-04-27T04:09:58.473 INFO:tasks.workunit.client.1.smithi181.stdout:2/422: fsync d4/d9/d4f/f5b 0 2024-04-27T04:09:58.475 INFO:tasks.workunit.client.0.smithi162.stdout:4/158: chown d2/d25/d1c/c27 2107828 1 2024-04-27T04:09:58.476 INFO:tasks.workunit.client.0.smithi162.stdout:4/159: chown d2/da/f12 6 1 2024-04-27T04:09:58.476 INFO:tasks.workunit.client.1.smithi181.stdout:9/464: symlink d0/d44/d4b/d43/d88/da1/la3 0 2024-04-27T04:09:58.477 INFO:tasks.workunit.client.0.smithi162.stdout:7/100: creat d3/d5/d9/dc/f19 x:0 0 0 2024-04-27T04:09:58.477 INFO:tasks.workunit.client.0.smithi162.stdout:0/193: dread f1 [0,4194304] 0 2024-04-27T04:09:58.478 INFO:tasks.workunit.client.0.smithi162.stdout:7/101: fdatasync d3/f18 0 2024-04-27T04:09:58.481 INFO:tasks.workunit.client.0.smithi162.stdout:5/131: rename d8/d1a/d23/d27 to d8/d2c 0 2024-04-27T04:09:58.483 INFO:tasks.workunit.client.1.smithi181.stdout:3/412: getdents d0/de/d20/d58/d70 0 2024-04-27T04:09:58.483 INFO:tasks.workunit.client.1.smithi181.stdout:3/413: chown d0/de/d20/f62 2928787 1 2024-04-27T04:09:58.485 INFO:tasks.workunit.client.1.smithi181.stdout:7/383: write d1/d2/d1c/d2d/d61/f68 [668318,28085] 0 2024-04-27T04:09:58.487 INFO:tasks.workunit.client.0.smithi162.stdout:9/138: link d3/d13/d15/c17 d3/d23/d1e/c32 0 2024-04-27T04:09:58.487 INFO:tasks.workunit.client.0.smithi162.stdout:9/139: chown d3/f1f 6 1 2024-04-27T04:09:58.490 INFO:tasks.workunit.client.0.smithi162.stdout:1/140: creat d0/d7/f27 x:0 0 0 2024-04-27T04:09:58.494 INFO:tasks.workunit.client.0.smithi162.stdout:4/160: creat d2/f38 x:0 0 0 2024-04-27T04:09:58.496 INFO:tasks.workunit.client.0.smithi162.stdout:0/194: symlink d5/d26/l3d 0 2024-04-27T04:09:58.498 INFO:tasks.workunit.client.1.smithi181.stdout:4/400: dread d0/d15/f1a [0,4194304] 0 2024-04-27T04:09:58.499 INFO:tasks.workunit.client.1.smithi181.stdout:2/423: read - d4/d9/d35/d42/f5a zero size 2024-04-27T04:09:58.500 INFO:tasks.workunit.client.1.smithi181.stdout:2/424: write d4/d9/f7a [461175,15052] 0 2024-04-27T04:09:58.500 INFO:tasks.workunit.client.0.smithi162.stdout:5/132: creat d8/d1a/f2d x:0 0 0 2024-04-27T04:09:58.501 INFO:tasks.workunit.client.0.smithi162.stdout:5/133: readlink d8/d9/l19 0 2024-04-27T04:09:58.501 INFO:tasks.workunit.client.1.smithi181.stdout:0/431: mknod dc/d93/c9f 0 2024-04-27T04:09:58.501 INFO:tasks.workunit.client.0.smithi162.stdout:5/134: write d8/d1a/f24 [773715,47987] 0 2024-04-27T04:09:58.503 INFO:tasks.workunit.client.1.smithi181.stdout:3/414: mknod d0/d6c/d52/d44/d56/d5b/d77/c85 0 2024-04-27T04:09:58.504 INFO:tasks.workunit.client.0.smithi162.stdout:3/136: symlink d1/l32 0 2024-04-27T04:09:58.505 INFO:tasks.workunit.client.1.smithi181.stdout:7/384: mknod d1/d2/d35/c7a 0 2024-04-27T04:09:58.507 INFO:tasks.workunit.client.0.smithi162.stdout:3/137: dread d1/d18/f12 [0,4194304] 0 2024-04-27T04:09:58.507 INFO:tasks.workunit.client.0.smithi162.stdout:3/138: read - d1/d21/f2c zero size 2024-04-27T04:09:58.509 INFO:tasks.workunit.client.1.smithi181.stdout:5/568: creat d4/de/d2c/d46/d99/dad/dca/fcc x:0 0 0 2024-04-27T04:09:58.510 INFO:tasks.workunit.client.0.smithi162.stdout:8/164: mkdir d0/d10/d2b/d2d/d2f 0 2024-04-27T04:09:58.511 INFO:tasks.workunit.client.1.smithi181.stdout:6/328: rename d7/dc/d1e/l24 to d7/dc/d1e/d5c/l6e 0 2024-04-27T04:09:58.513 INFO:tasks.workunit.client.0.smithi162.stdout:2/107: dwrite d0/fe [0,4194304] 0 2024-04-27T04:09:58.513 INFO:tasks.workunit.client.0.smithi162.stdout:4/161: mkdir d2/d25/d1c/d39 0 2024-04-27T04:09:58.513 INFO:tasks.workunit.client.0.smithi162.stdout:2/108: fsync d0/f7 0 2024-04-27T04:09:58.513 INFO:tasks.workunit.client.0.smithi162.stdout:4/162: fsync d2/d25/f2d 0 2024-04-27T04:09:58.515 INFO:tasks.workunit.client.1.smithi181.stdout:0/432: rmdir dc/d1f/d3b/d4a/d95/d26/d41 39 2024-04-27T04:09:58.516 INFO:tasks.workunit.client.1.smithi181.stdout:0/433: readlink l5 0 2024-04-27T04:09:58.518 INFO:tasks.workunit.client.1.smithi181.stdout:3/415: mkdir d0/d28/d86 0 2024-04-27T04:09:58.519 INFO:tasks.workunit.client.0.smithi162.stdout:4/163: dwrite d2/da/d1a/f2e [0,4194304] 0 2024-04-27T04:09:58.520 INFO:tasks.workunit.client.0.smithi162.stdout:4/164: stat d2/d25 0 2024-04-27T04:09:58.520 INFO:tasks.workunit.client.1.smithi181.stdout:2/425: dwrite d4/d9/df/d14/f15 [4194304,4194304] 0 2024-04-27T04:09:58.523 INFO:tasks.workunit.client.1.smithi181.stdout:7/385: symlink d1/d2/d1c/d2d/d61/l7b 0 2024-04-27T04:09:58.523 INFO:tasks.workunit.client.0.smithi162.stdout:5/135: mkdir d8/d2c/d2e 0 2024-04-27T04:09:58.525 INFO:tasks.workunit.client.1.smithi181.stdout:1/474: symlink d1/d4c/d1e/l94 0 2024-04-27T04:09:58.526 INFO:tasks.workunit.client.1.smithi181.stdout:1/475: chown d1/d4c/d1e/d20/d53 10 1 2024-04-27T04:09:58.528 INFO:tasks.workunit.client.1.smithi181.stdout:5/569: creat d4/de/d2c/d46/d99/dad/dca/fcd x:0 0 0 2024-04-27T04:09:58.530 INFO:tasks.workunit.client.0.smithi162.stdout:6/149: sync 2024-04-27T04:09:58.534 INFO:tasks.workunit.client.1.smithi181.stdout:6/329: creat d7/d31/f6f x:0 0 0 2024-04-27T04:09:58.538 INFO:tasks.workunit.client.0.smithi162.stdout:3/139: rmdir d1/d21 39 2024-04-27T04:09:58.539 INFO:tasks.workunit.client.1.smithi181.stdout:2/426: mkdir d4/d9/d35/d42/d86 0 2024-04-27T04:09:58.541 INFO:tasks.workunit.client.0.smithi162.stdout:3/140: dread d1/d18/f1e [0,4194304] 0 2024-04-27T04:09:58.541 INFO:tasks.workunit.client.0.smithi162.stdout:8/165: unlink d0/d10/d25/f24 0 2024-04-27T04:09:58.544 INFO:tasks.workunit.client.1.smithi181.stdout:6/330: creat d7/d31/d45/f70 x:0 0 0 2024-04-27T04:09:58.546 INFO:tasks.workunit.client.0.smithi162.stdout:2/109: creat d0/d4/f22 x:0 0 0 2024-04-27T04:09:58.548 INFO:tasks.workunit.client.1.smithi181.stdout:3/416: fsync d0/de/ff 0 2024-04-27T04:09:58.549 INFO:tasks.workunit.client.0.smithi162.stdout:7/102: link d3/d5/d9/dc/c11 d3/d5/c1a 0 2024-04-27T04:09:58.550 INFO:tasks.workunit.client.0.smithi162.stdout:9/140: truncate d3/f4 3892407 0 2024-04-27T04:09:58.550 INFO:tasks.workunit.client.0.smithi162.stdout:9/141: fdatasync d3/fb 0 2024-04-27T04:09:58.551 INFO:tasks.workunit.client.0.smithi162.stdout:9/142: chown d3/d11/c12 18 1 2024-04-27T04:09:58.552 INFO:tasks.workunit.client.1.smithi181.stdout:2/427: creat d4/d9/d16/d45/f87 x:0 0 0 2024-04-27T04:09:58.554 INFO:tasks.workunit.client.1.smithi181.stdout:6/331: dread f0 [0,4194304] 0 2024-04-27T04:09:58.557 INFO:tasks.workunit.client.0.smithi162.stdout:4/165: creat d2/d25/d1c/d24/f3a x:0 0 0 2024-04-27T04:09:58.559 INFO:tasks.workunit.client.1.smithi181.stdout:3/417: dwrite d0/de/f7f [0,4194304] 0 2024-04-27T04:09:58.559 INFO:tasks.workunit.client.1.smithi181.stdout:1/476: chown d1/d4c/d1e/d20/d53/d56/c58 446 1 2024-04-27T04:09:58.562 INFO:tasks.workunit.client.1.smithi181.stdout:0/434: dwrite fa [8388608,4194304] 0 2024-04-27T04:09:58.562 INFO:tasks.workunit.client.1.smithi181.stdout:1/477: write d1/d2/d5/d6/f3d [384601,67533] 0 2024-04-27T04:09:58.563 INFO:tasks.workunit.client.0.smithi162.stdout:4/166: dwrite d2/da/d1a/f29 [0,4194304] 0 2024-04-27T04:09:58.565 INFO:tasks.workunit.client.0.smithi162.stdout:6/150: write d6/d1a/f20 [2735307,43978] 0 2024-04-27T04:09:58.571 INFO:tasks.workunit.client.0.smithi162.stdout:1/141: rmdir d0/d11/d1d 0 2024-04-27T04:09:58.576 INFO:tasks.workunit.client.1.smithi181.stdout:5/570: truncate d4/de/d2f/d7a/f89 2765175 0 2024-04-27T04:09:58.577 INFO:tasks.workunit.client.1.smithi181.stdout:7/386: dwrite d1/f8 [0,4194304] 0 2024-04-27T04:09:58.582 INFO:tasks.workunit.client.1.smithi181.stdout:2/428: rename d4/d81/f85 to d4/d9/d35/d42/d86/f88 0 2024-04-27T04:09:58.583 INFO:tasks.workunit.client.0.smithi162.stdout:3/141: dwrite d1/d21/f2c [0,4194304] 0 2024-04-27T04:09:58.583 INFO:tasks.workunit.client.0.smithi162.stdout:2/110: mknod d0/d4/c23 0 2024-04-27T04:09:58.584 INFO:tasks.workunit.client.0.smithi162.stdout:2/111: stat d0/c1c 0 2024-04-27T04:09:58.584 INFO:tasks.workunit.client.0.smithi162.stdout:8/166: dread d0/fd [0,4194304] 0 2024-04-27T04:09:58.587 INFO:tasks.workunit.client.0.smithi162.stdout:0/195: creat d5/dc/d1e/f3e x:0 0 0 2024-04-27T04:09:58.588 INFO:tasks.workunit.client.1.smithi181.stdout:3/418: rmdir d0/d6c/d52/d44/d56/d5b 39 2024-04-27T04:09:58.589 INFO:tasks.workunit.client.0.smithi162.stdout:7/103: unlink d3/d5/d9/dc/de/c14 0 2024-04-27T04:09:58.592 INFO:tasks.workunit.client.0.smithi162.stdout:9/143: mknod d3/d23/c33 0 2024-04-27T04:09:58.594 INFO:tasks.workunit.client.0.smithi162.stdout:5/136: unlink d8/d9/f18 0 2024-04-27T04:09:58.596 INFO:tasks.workunit.client.1.smithi181.stdout:5/571: mknod d4/de/d2c/d46/d4d/d8e/d67/cce 0 2024-04-27T04:09:58.598 INFO:tasks.workunit.client.1.smithi181.stdout:6/332: mknod d7/c71 0 2024-04-27T04:09:58.599 INFO:tasks.workunit.client.1.smithi181.stdout:6/333: dread - d7/d31/d45/f70 zero size 2024-04-27T04:09:58.599 INFO:tasks.workunit.client.0.smithi162.stdout:5/137: dwrite d8/d9/f21 [0,4194304] 0 2024-04-27T04:09:58.599 INFO:tasks.workunit.client.1.smithi181.stdout:7/387: mknod d1/d22/d15/d1f/c7c 0 2024-04-27T04:09:58.603 INFO:tasks.workunit.client.1.smithi181.stdout:2/429: rename d4/d9/df/c37 to d4/d9/d4f/c89 0 2024-04-27T04:09:58.606 INFO:tasks.workunit.client.1.smithi181.stdout:5/572: dwrite d4/de/d2c/d46/d99/dad/dca/fcd [0,4194304] 0 2024-04-27T04:09:58.608 INFO:tasks.workunit.client.1.smithi181.stdout:2/430: chown d4/d9/d16/d45/f66 1956152 1 2024-04-27T04:09:58.610 INFO:tasks.workunit.client.0.smithi162.stdout:3/142: symlink d1/d21/l33 0 2024-04-27T04:09:58.611 INFO:tasks.workunit.client.1.smithi181.stdout:6/334: dwrite d7/f26 [4194304,4194304] 0 2024-04-27T04:09:58.614 INFO:tasks.workunit.client.1.smithi181.stdout:6/335: write d7/fd [2498537,79337] 0 2024-04-27T04:09:58.615 INFO:tasks.workunit.client.1.smithi181.stdout:1/478: mkdir d1/d2/d5/d95 0 2024-04-27T04:09:58.616 INFO:tasks.workunit.client.1.smithi181.stdout:1/479: chown d1/d2/d5/c3a 0 1 2024-04-27T04:09:58.616 INFO:tasks.workunit.client.0.smithi162.stdout:8/167: write d0/fc [215048,125712] 0 2024-04-27T04:09:58.618 INFO:tasks.workunit.client.1.smithi181.stdout:0/435: write dc/f15 [952665,79990] 0 2024-04-27T04:09:58.622 INFO:tasks.workunit.client.0.smithi162.stdout:7/104: symlink d3/d5/d9/l1b 0 2024-04-27T04:09:58.625 INFO:tasks.workunit.client.0.smithi162.stdout:9/144: creat d3/d13/d15/f34 x:0 0 0 2024-04-27T04:09:58.626 INFO:tasks.workunit.client.0.smithi162.stdout:0/196: dwrite d5/dc/f31 [0,4194304] 0 2024-04-27T04:09:58.627 INFO:tasks.workunit.client.0.smithi162.stdout:9/145: read d3/d13/d15/f19 [3520107,62090] 0 2024-04-27T04:09:58.627 INFO:tasks.workunit.client.0.smithi162.stdout:9/146: stat f0 0 2024-04-27T04:09:58.630 INFO:tasks.workunit.client.0.smithi162.stdout:4/167: write d2/f9 [4017550,27087] 0 2024-04-27T04:09:58.631 INFO:tasks.workunit.client.0.smithi162.stdout:6/151: mknod d6/d1a/d34/c37 0 2024-04-27T04:09:58.634 INFO:tasks.workunit.client.1.smithi181.stdout:6/336: mkdir d7/d31/d45/d72 0 2024-04-27T04:09:58.635 INFO:tasks.workunit.client.0.smithi162.stdout:1/142: creat d0/d11/d1a/f28 x:0 0 0 2024-04-27T04:09:58.635 INFO:tasks.workunit.client.0.smithi162.stdout:1/143: chown d0/f1e 0 1 2024-04-27T04:09:58.637 INFO:tasks.workunit.client.1.smithi181.stdout:1/480: mknod d1/d2/d5/d6/db/c96 0 2024-04-27T04:09:58.639 INFO:tasks.workunit.client.0.smithi162.stdout:5/138: rename d8/d1a/f24 to d8/d1a/d29/f2f 0 2024-04-27T04:09:58.639 INFO:tasks.workunit.client.0.smithi162.stdout:1/144: dread d0/f1b [0,4194304] 0 2024-04-27T04:09:58.640 INFO:tasks.workunit.client.0.smithi162.stdout:1/145: read d0/d7/f19 [914810,122830] 0 2024-04-27T04:09:58.641 INFO:tasks.workunit.client.1.smithi181.stdout:0/436: unlink dc/d1f/d3b/d4a/d95/c8d 0 2024-04-27T04:09:58.642 INFO:tasks.workunit.client.0.smithi162.stdout:2/112: mkdir d0/d24 0 2024-04-27T04:09:58.644 INFO:tasks.workunit.client.1.smithi181.stdout:3/419: dwrite d0/f14 [0,4194304] 0 2024-04-27T04:09:58.645 INFO:tasks.workunit.client.1.smithi181.stdout:1/481: dwrite d1/d4c/d52/f76 [0,4194304] 0 2024-04-27T04:09:58.674 INFO:tasks.workunit.client.0.smithi162.stdout:8/168: creat d0/d10/d25/f30 x:0 0 0 2024-04-27T04:09:58.678 INFO:tasks.workunit.client.0.smithi162.stdout:7/105: creat d3/d5/d9/dc/de/f1c x:0 0 0 2024-04-27T04:09:58.681 INFO:tasks.workunit.client.1.smithi181.stdout:6/337: write d7/dc/d1e/f53 [3178898,72784] 0 2024-04-27T04:09:58.682 INFO:tasks.workunit.client.0.smithi162.stdout:0/197: mknod d5/dc/d37/c3f 0 2024-04-27T04:09:58.685 INFO:tasks.workunit.client.0.smithi162.stdout:0/198: dread d5/dc/f14 [0,4194304] 0 2024-04-27T04:09:58.688 INFO:tasks.workunit.client.1.smithi181.stdout:7/388: fsync d1/d22/d15/d13/f46 0 2024-04-27T04:09:58.688 INFO:tasks.workunit.client.1.smithi181.stdout:7/389: fsync d1/d57/f77 0 2024-04-27T04:09:58.690 INFO:tasks.workunit.client.0.smithi162.stdout:0/199: dwrite d5/f27 [0,4194304] 0 2024-04-27T04:09:58.691 INFO:tasks.workunit.client.1.smithi181.stdout:1/482: chown d1/d4c/d1e/d20/d53/d56/l63 60 1 2024-04-27T04:09:58.693 INFO:tasks.workunit.client.1.smithi181.stdout:2/431: truncate d4/d9/df/d14/f15 7380523 0 2024-04-27T04:09:58.694 INFO:tasks.workunit.client.1.smithi181.stdout:5/573: rmdir d4/de/d2c/d46/d4d/d8e/d67/dac 0 2024-04-27T04:09:58.696 INFO:tasks.workunit.client.0.smithi162.stdout:5/139: stat d8/c1b 0 2024-04-27T04:09:58.696 INFO:tasks.workunit.client.1.smithi181.stdout:6/338: dread f6 [0,4194304] 0 2024-04-27T04:09:58.698 INFO:tasks.workunit.client.1.smithi181.stdout:6/339: write d7/d27/f2e [5882877,116486] 0 2024-04-27T04:09:58.699 INFO:tasks.workunit.client.0.smithi162.stdout:1/146: symlink d0/d11/d1a/l29 0 2024-04-27T04:09:58.700 INFO:tasks.workunit.client.0.smithi162.stdout:1/147: chown d0/d7/d10 5 1 2024-04-27T04:09:58.701 INFO:tasks.workunit.client.0.smithi162.stdout:3/143: getdents d1/d18/d23/d2f 0 2024-04-27T04:09:58.701 INFO:tasks.workunit.client.0.smithi162.stdout:3/144: chown d1/d18/d23/d2f 25 1 2024-04-27T04:09:58.702 INFO:tasks.workunit.client.0.smithi162.stdout:2/113: unlink d0/fa 0 2024-04-27T04:09:58.703 INFO:tasks.workunit.client.0.smithi162.stdout:3/145: read d1/d18/f6 [370626,95669] 0 2024-04-27T04:09:58.703 INFO:tasks.workunit.client.1.smithi181.stdout:3/420: mknod d0/de/c87 0 2024-04-27T04:09:58.704 INFO:tasks.workunit.client.0.smithi162.stdout:3/146: write d1/d18/f2d [2232147,56926] 0 2024-04-27T04:09:58.705 INFO:tasks.workunit.client.0.smithi162.stdout:8/169: creat d0/d10/d2b/d23/f31 x:0 0 0 2024-04-27T04:09:58.705 INFO:tasks.workunit.client.0.smithi162.stdout:7/106: rmdir d3/d5/d9 39 2024-04-27T04:09:58.706 INFO:tasks.workunit.client.1.smithi181.stdout:1/483: rmdir d1/d4c/d1e/d20/d53/d56 39 2024-04-27T04:09:58.707 INFO:tasks.workunit.client.1.smithi181.stdout:3/421: dread d0/f19 [0,4194304] 0 2024-04-27T04:09:58.709 INFO:tasks.workunit.client.0.smithi162.stdout:2/114: dwrite d0/d4/f22 [0,4194304] 0 2024-04-27T04:09:58.710 INFO:tasks.workunit.client.0.smithi162.stdout:4/168: mkdir d2/d25/d1c/d39/d3b 0 2024-04-27T04:09:58.711 INFO:tasks.workunit.client.1.smithi181.stdout:6/340: mknod d7/dc/c73 0 2024-04-27T04:09:58.713 INFO:tasks.workunit.client.1.smithi181.stdout:3/422: dwrite d0/d6c/d52/f59 [0,4194304] 0 2024-04-27T04:09:58.714 INFO:tasks.workunit.client.0.smithi162.stdout:6/152: creat d6/db/d32/f38 x:0 0 0 2024-04-27T04:09:58.716 INFO:tasks.workunit.client.1.smithi181.stdout:0/437: creat dc/d1f/d3b/d4a/d95/d26/fa0 x:0 0 0 2024-04-27T04:09:58.721 INFO:tasks.workunit.client.0.smithi162.stdout:0/200: rename d5/dc/d1e/d20/l2b to d5/l40 0 2024-04-27T04:09:58.722 INFO:tasks.workunit.client.1.smithi181.stdout:0/438: dread dc/d1f/d3b/d4a/d95/d26/f28 [0,4194304] 0 2024-04-27T04:09:58.725 INFO:tasks.workunit.client.0.smithi162.stdout:1/148: mkdir d0/d7/d2a 0 2024-04-27T04:09:58.725 INFO:tasks.workunit.client.1.smithi181.stdout:1/484: mkdir d1/d4c/d60/d97 0 2024-04-27T04:09:58.729 INFO:tasks.workunit.client.0.smithi162.stdout:0/201: dwrite d5/dc/f22 [0,4194304] 0 2024-04-27T04:09:58.733 INFO:tasks.workunit.client.0.smithi162.stdout:3/147: write d1/d18/f11 [592955,69216] 0 2024-04-27T04:09:58.737 INFO:tasks.workunit.client.1.smithi181.stdout:6/341: rename d7/dc/d1c/f20 to d7/dc/d1c/d34/d3f/f74 0 2024-04-27T04:09:58.738 INFO:tasks.workunit.client.1.smithi181.stdout:2/432: dwrite d4/f8 [4194304,4194304] 0 2024-04-27T04:09:58.738 INFO:tasks.workunit.client.0.smithi162.stdout:8/170: symlink d0/d10/d2b/l32 0 2024-04-27T04:09:58.738 INFO:tasks.workunit.client.0.smithi162.stdout:3/148: dwrite d1/d18/f1a [4194304,4194304] 0 2024-04-27T04:09:58.740 INFO:tasks.workunit.client.1.smithi181.stdout:3/423: mkdir d0/d6c/d66/d88 0 2024-04-27T04:09:58.745 INFO:tasks.workunit.client.0.smithi162.stdout:9/147: rmdir d3/d13/d2d/d30/d31 0 2024-04-27T04:09:58.749 INFO:tasks.workunit.client.0.smithi162.stdout:9/148: dread d3/f6 [0,4194304] 0 2024-04-27T04:09:58.750 INFO:tasks.workunit.client.0.smithi162.stdout:9/149: truncate d3/d13/d15/f34 407496 0 2024-04-27T04:09:58.750 INFO:tasks.workunit.client.1.smithi181.stdout:7/390: link d1/c76 d1/df/d62/c7d 0 2024-04-27T04:09:58.758 INFO:tasks.workunit.client.0.smithi162.stdout:5/140: rename d8/c14 to d8/d2c/d2e/c30 0 2024-04-27T04:09:58.759 INFO:tasks.workunit.client.1.smithi181.stdout:5/574: link d4/d10/d72/f77 d4/de/d2c/d46/d4d/d8e/d54/fcf 0 2024-04-27T04:09:58.761 INFO:tasks.workunit.client.0.smithi162.stdout:1/149: unlink d0/d7/d10/f21 0 2024-04-27T04:09:58.764 INFO:tasks.workunit.client.1.smithi181.stdout:6/342: dread d7/d31/f37 [0,4194304] 0 2024-04-27T04:09:58.765 INFO:tasks.workunit.client.1.smithi181.stdout:6/343: readlink d7/dc/d1c/l67 0 2024-04-27T04:09:58.765 INFO:tasks.workunit.client.1.smithi181.stdout:2/433: rename d4/d9/d35/d42/d86/f88 to d4/d9/d35/d42/d86/f8a 0 2024-04-27T04:09:58.766 INFO:tasks.workunit.client.1.smithi181.stdout:2/434: stat d4/d9/df/c5e 0 2024-04-27T04:09:58.767 INFO:tasks.workunit.client.0.smithi162.stdout:7/107: mkdir d3/d1d 0 2024-04-27T04:09:58.768 INFO:tasks.workunit.client.1.smithi181.stdout:3/424: creat d0/d6c/f89 x:0 0 0 2024-04-27T04:09:58.770 INFO:tasks.workunit.client.1.smithi181.stdout:3/425: write d0/de/d20/d58/f7e [280176,124679] 0 2024-04-27T04:09:58.773 INFO:tasks.workunit.client.0.smithi162.stdout:8/171: chown d0/f7 1 1 2024-04-27T04:09:58.774 INFO:tasks.workunit.client.1.smithi181.stdout:7/391: creat d1/d2/d35/d67/f7e x:0 0 0 2024-04-27T04:09:58.776 INFO:tasks.workunit.client.0.smithi162.stdout:3/149: mkdir d1/d18/d23/d34 0 2024-04-27T04:09:58.778 INFO:tasks.workunit.client.0.smithi162.stdout:2/115: write d0/f1 [1054370,10129] 0 2024-04-27T04:09:58.779 INFO:tasks.workunit.client.0.smithi162.stdout:4/169: write d2/d25/fb [2236971,120804] 0 2024-04-27T04:09:58.780 INFO:tasks.workunit.client.0.smithi162.stdout:2/116: write d0/fe [2080479,27083] 0 2024-04-27T04:09:58.780 INFO:tasks.workunit.client.1.smithi181.stdout:1/485: symlink d1/d2/l98 0 2024-04-27T04:09:58.781 INFO:tasks.workunit.client.0.smithi162.stdout:4/170: write d2/d25/d1c/f33 [718811,57904] 0 2024-04-27T04:09:58.781 INFO:tasks.workunit.client.0.smithi162.stdout:6/153: write f2 [4081134,65156] 0 2024-04-27T04:09:58.786 INFO:tasks.workunit.client.1.smithi181.stdout:6/344: creat d7/d31/f75 x:0 0 0 2024-04-27T04:09:58.787 INFO:tasks.workunit.client.0.smithi162.stdout:5/141: creat d8/d1a/f31 x:0 0 0 2024-04-27T04:09:58.788 INFO:tasks.workunit.client.1.smithi181.stdout:2/435: mknod d4/d9/df/d14/c8b 0 2024-04-27T04:09:58.789 INFO:tasks.workunit.client.1.smithi181.stdout:7/392: symlink d1/d57/l7f 0 2024-04-27T04:09:58.790 INFO:tasks.workunit.client.1.smithi181.stdout:2/436: truncate d4/d9/d16/d45/f6c 4859830 0 2024-04-27T04:09:58.792 INFO:tasks.workunit.client.1.smithi181.stdout:1/486: creat d1/d4c/d52/d92/f99 x:0 0 0 2024-04-27T04:09:58.793 INFO:tasks.workunit.client.0.smithi162.stdout:7/108: read - d3/d5/d9/dc/de/f1c zero size 2024-04-27T04:09:58.796 INFO:tasks.workunit.client.1.smithi181.stdout:6/345: truncate d7/dc/f41 1105813 0 2024-04-27T04:09:58.796 INFO:tasks.workunit.client.1.smithi181.stdout:0/439: dwrite dc/d1f/d3b/d4a/d95/d26/f6c [0,4194304] 0 2024-04-27T04:09:58.797 INFO:tasks.workunit.client.1.smithi181.stdout:0/440: dread - dc/d1f/f7c zero size 2024-04-27T04:09:58.798 INFO:tasks.workunit.client.1.smithi181.stdout:0/441: stat dc/f11 0 2024-04-27T04:09:58.798 INFO:tasks.workunit.client.0.smithi162.stdout:3/150: unlink d1/d18/f19 0 2024-04-27T04:09:58.799 INFO:tasks.workunit.client.0.smithi162.stdout:3/151: chown d1/d21 1206712 1 2024-04-27T04:09:58.799 INFO:tasks.workunit.client.1.smithi181.stdout:7/393: mkdir d1/d2/d1c/d2d/d61/d80 0 2024-04-27T04:09:58.803 INFO:tasks.workunit.client.1.smithi181.stdout:1/487: creat d1/d2/d5/d6/db/d2d/d36/f9a x:0 0 0 2024-04-27T04:09:58.807 INFO:tasks.workunit.client.0.smithi162.stdout:6/154: rmdir d6/d1a 39 2024-04-27T04:09:58.809 INFO:tasks.workunit.client.0.smithi162.stdout:6/155: chown d6/db/c33 117 1 2024-04-27T04:09:58.810 INFO:tasks.workunit.client.1.smithi181.stdout:5/575: link d4/de/d2c/d2e/c49 d4/de/d2f/d7a/d90/cd0 0 2024-04-27T04:09:58.810 INFO:tasks.workunit.client.1.smithi181.stdout:5/576: write d4/de/d2c/d46/d4d/fb8 [531806,278] 0 2024-04-27T04:09:58.810 INFO:tasks.workunit.client.0.smithi162.stdout:4/171: symlink d2/da/dd/l3c 0 2024-04-27T04:09:58.811 INFO:tasks.workunit.client.1.smithi181.stdout:6/346: creat d7/d1b/f76 x:0 0 0 2024-04-27T04:09:58.812 INFO:tasks.workunit.client.1.smithi181.stdout:6/347: dread - d7/d31/d45/f70 zero size 2024-04-27T04:09:58.813 INFO:tasks.workunit.client.1.smithi181.stdout:6/348: read d7/f19 [4329714,2961] 0 2024-04-27T04:09:58.818 INFO:tasks.workunit.client.1.smithi181.stdout:7/394: chown d1/ld 1065869 1 2024-04-27T04:09:58.820 INFO:tasks.workunit.client.1.smithi181.stdout:0/442: dread fa [4194304,4194304] 0 2024-04-27T04:09:58.820 INFO:tasks.workunit.client.0.smithi162.stdout:0/202: rename d5/dc/d1e/d33 to d5/dc/d1e/d41 0 2024-04-27T04:09:58.822 INFO:tasks.workunit.client.1.smithi181.stdout:1/488: rmdir d1/d4c/d60 39 2024-04-27T04:09:58.828 INFO:tasks.workunit.client.0.smithi162.stdout:3/152: unlink d1/d18/d23/f2a 0 2024-04-27T04:09:58.830 INFO:tasks.workunit.client.1.smithi181.stdout:6/349: rmdir d7/dc/d1c/d34/d3f 39 2024-04-27T04:09:58.834 INFO:tasks.workunit.client.1.smithi181.stdout:0/443: write dc/d1f/d3b/d4a/d95/d26/d41/f9e [866363,79028] 0 2024-04-27T04:09:58.835 INFO:tasks.workunit.client.0.smithi162.stdout:4/172: rmdir d2/da 39 2024-04-27T04:09:58.835 INFO:tasks.workunit.client.1.smithi181.stdout:0/444: read dc/fd [772710,26235] 0 2024-04-27T04:09:58.836 INFO:tasks.workunit.client.0.smithi162.stdout:4/173: fdatasync d2/d25/f7 0 2024-04-27T04:09:58.837 INFO:tasks.workunit.client.1.smithi181.stdout:0/445: truncate dc/d1f/d3b/d4a/d95/d26/d68/f88 152020 0 2024-04-27T04:09:58.837 INFO:tasks.workunit.client.1.smithi181.stdout:2/437: link d4/d9/d16/d82/f71 d4/d9/f8c 0 2024-04-27T04:09:58.839 INFO:tasks.workunit.client.0.smithi162.stdout:5/142: unlink d8/d2c/d2e/c30 0 2024-04-27T04:09:58.841 INFO:tasks.workunit.client.0.smithi162.stdout:1/150: creat d0/f2b x:0 0 0 2024-04-27T04:09:58.841 INFO:tasks.workunit.client.0.smithi162.stdout:0/203: mknod d5/dc/d1e/d20/c42 0 2024-04-27T04:09:58.842 INFO:tasks.workunit.client.0.smithi162.stdout:0/204: chown d5/f16 1 1 2024-04-27T04:09:58.844 INFO:tasks.workunit.client.1.smithi181.stdout:7/395: mkdir d1/d22/d15/d5f/d81 0 2024-04-27T04:09:58.844 INFO:tasks.workunit.client.1.smithi181.stdout:0/446: symlink dc/d1f/d6b/la1 0 2024-04-27T04:09:58.845 INFO:tasks.workunit.client.0.smithi162.stdout:9/150: link d3/d13/d15/l1d d3/l35 0 2024-04-27T04:09:58.846 INFO:tasks.workunit.client.1.smithi181.stdout:2/438: symlink d4/d9/d16/d82/l8d 0 2024-04-27T04:09:58.846 INFO:tasks.workunit.client.0.smithi162.stdout:2/117: dwrite d0/d4/f15 [0,4194304] 0 2024-04-27T04:09:58.847 INFO:tasks.workunit.client.1.smithi181.stdout:2/439: write d4/d9/f11 [1330671,102910] 0 2024-04-27T04:09:58.848 INFO:tasks.workunit.client.1.smithi181.stdout:5/577: rename d4/de/d2c/d46/d4d/d8e/d54/l97 to d4/de/d2c/d46/ld1 0 2024-04-27T04:09:58.849 INFO:tasks.workunit.client.1.smithi181.stdout:6/350: symlink d7/dc/d1c/l77 0 2024-04-27T04:09:58.851 INFO:tasks.workunit.client.0.smithi162.stdout:1/151: dwrite d0/d11/f22 [0,4194304] 0 2024-04-27T04:09:58.852 INFO:tasks.workunit.client.1.smithi181.stdout:0/447: creat dc/d1f/d3b/d4a/d95/d6d/fa2 x:0 0 0 2024-04-27T04:09:58.854 INFO:tasks.workunit.client.0.smithi162.stdout:7/109: creat d3/f1e x:0 0 0 2024-04-27T04:09:58.855 INFO:tasks.workunit.client.0.smithi162.stdout:5/143: dread d8/f20 [4194304,4194304] 0 2024-04-27T04:09:58.855 INFO:tasks.workunit.client.0.smithi162.stdout:8/172: getdents d0/de/d20 0 2024-04-27T04:09:58.856 INFO:tasks.workunit.client.0.smithi162.stdout:5/144: dread - d8/d1a/f31 zero size 2024-04-27T04:09:58.863 INFO:tasks.workunit.client.1.smithi181.stdout:7/396: dread d1/d2/d1c/d2d/f3e [0,4194304] 0 2024-04-27T04:09:58.864 INFO:tasks.workunit.client.0.smithi162.stdout:3/153: mknod d1/c35 0 2024-04-27T04:09:58.865 INFO:tasks.workunit.client.1.smithi181.stdout:5/578: mkdir d4/d10/d29/dd2 0 2024-04-27T04:09:58.867 INFO:tasks.workunit.client.0.smithi162.stdout:0/205: symlink d5/dc/d37/l43 0 2024-04-27T04:09:58.870 INFO:tasks.workunit.client.0.smithi162.stdout:6/156: rename d6/db/f21 to d6/f39 0 2024-04-27T04:09:58.871 INFO:tasks.workunit.client.0.smithi162.stdout:6/157: write d6/f2d [655062,56423] 0 2024-04-27T04:09:58.871 INFO:tasks.workunit.client.0.smithi162.stdout:6/158: fsync d6/f11 0 2024-04-27T04:09:58.873 INFO:tasks.workunit.client.0.smithi162.stdout:9/151: mknod d3/d13/d2d/c36 0 2024-04-27T04:09:58.873 INFO:tasks.workunit.client.0.smithi162.stdout:9/152: readlink d3/d23/d1e/l2c 0 2024-04-27T04:09:58.874 INFO:tasks.workunit.client.1.smithi181.stdout:0/448: unlink dc/d1f/d3b/d4a/d95/d26/d68/f88 0 2024-04-27T04:09:58.879 INFO:tasks.workunit.client.1.smithi181.stdout:1/489: rename d1/d2/d5/d6/c31 to d1/d4c/d60/c9b 0 2024-04-27T04:09:58.880 INFO:tasks.workunit.client.1.smithi181.stdout:1/490: fsync d1/d4c/d52/f68 0 2024-04-27T04:09:58.881 INFO:tasks.workunit.client.1.smithi181.stdout:7/397: mkdir d1/d22/d15/d82 0 2024-04-27T04:09:58.883 INFO:tasks.workunit.client.1.smithi181.stdout:1/491: read d1/f2f [2581383,130274] 0 2024-04-27T04:09:58.884 INFO:tasks.workunit.client.0.smithi162.stdout:7/110: mkdir d3/d5/d9/dc/de/d1f 0 2024-04-27T04:09:58.885 INFO:tasks.workunit.client.1.smithi181.stdout:1/492: truncate d1/f8e 391365 0 2024-04-27T04:09:58.889 INFO:tasks.workunit.client.0.smithi162.stdout:3/154: write d1/d18/f1e [3916418,58308] 0 2024-04-27T04:09:58.889 INFO:tasks.workunit.client.1.smithi181.stdout:2/440: dwrite d4/d9/d35/d42/f5a [0,4194304] 0 2024-04-27T04:09:58.889 INFO:tasks.workunit.client.1.smithi181.stdout:0/449: dread dc/f10 [0,4194304] 0 2024-04-27T04:09:58.890 INFO:tasks.workunit.client.0.smithi162.stdout:3/155: dread - d1/d18/f13 zero size 2024-04-27T04:09:58.890 INFO:tasks.workunit.client.1.smithi181.stdout:2/441: readlink d4/d9/l31 0 2024-04-27T04:09:58.891 INFO:tasks.workunit.client.0.smithi162.stdout:5/145: dwrite d8/f20 [0,4194304] 0 2024-04-27T04:09:58.891 INFO:tasks.workunit.client.1.smithi181.stdout:0/450: chown dc/f2c 12232 1 2024-04-27T04:09:58.893 INFO:tasks.workunit.client.1.smithi181.stdout:0/451: write dc/d1f/d3b/d4a/d95/d26/d41/d6f/d7d/f9a [958319,123823] 0 2024-04-27T04:09:58.899 INFO:tasks.workunit.client.0.smithi162.stdout:2/118: write d0/d4/f16 [1235498,71839] 0 2024-04-27T04:09:58.899 INFO:tasks.workunit.client.0.smithi162.stdout:5/146: dwrite f4 [4194304,4194304] 0 2024-04-27T04:09:58.901 INFO:tasks.workunit.client.0.smithi162.stdout:4/174: rename d2/f2f to d2/d25/d1c/f3d 0 2024-04-27T04:09:58.903 INFO:tasks.workunit.client.0.smithi162.stdout:4/175: write d2/d25/d1c/f33 [1262956,109047] 0 2024-04-27T04:09:58.903 INFO:tasks.workunit.client.1.smithi181.stdout:0/452: dwrite dc/d1f/d3b/d4a/d95/d26/d41/f71 [0,4194304] 0 2024-04-27T04:09:58.905 INFO:tasks.workunit.client.1.smithi181.stdout:7/398: readlink d1/df/l20 0 2024-04-27T04:09:58.908 INFO:tasks.workunit.client.0.smithi162.stdout:6/159: write d6/d1a/f1d [824544,122175] 0 2024-04-27T04:09:58.910 INFO:tasks.workunit.client.1.smithi181.stdout:5/579: mknod d4/de/d2c/d2e/d8f/cd3 0 2024-04-27T04:09:58.912 INFO:tasks.workunit.client.1.smithi181.stdout:5/580: write d4/da3/fbd [825582,25823] 0 2024-04-27T04:09:58.913 INFO:tasks.workunit.client.0.smithi162.stdout:3/156: dread d1/d18/f6 [0,4194304] 0 2024-04-27T04:09:58.913 INFO:tasks.workunit.client.0.smithi162.stdout:9/153: fsync d3/f5 0 2024-04-27T04:09:58.914 INFO:tasks.workunit.client.0.smithi162.stdout:3/157: fsync d1/d18/d23/f2b 0 2024-04-27T04:09:58.914 INFO:tasks.workunit.client.0.smithi162.stdout:9/154: write f0 [3241281,104553] 0 2024-04-27T04:09:58.916 INFO:tasks.workunit.client.1.smithi181.stdout:6/351: creat d7/dc/d1c/f78 x:0 0 0 2024-04-27T04:09:58.920 INFO:tasks.workunit.client.0.smithi162.stdout:7/111: mknod d3/d5/d9/dc/c20 0 2024-04-27T04:09:58.921 INFO:tasks.workunit.client.1.smithi181.stdout:2/442: creat d4/d81/f8e x:0 0 0 2024-04-27T04:09:58.924 INFO:tasks.workunit.client.0.smithi162.stdout:5/147: creat d8/d9/f32 x:0 0 0 2024-04-27T04:09:58.925 INFO:tasks.workunit.client.1.smithi181.stdout:1/493: dwrite d1/d2/d5/d6/d6a/f75 [0,4194304] 0 2024-04-27T04:09:58.927 INFO:tasks.workunit.client.1.smithi181.stdout:0/453: unlink dc/d1f/d3b/d4a/d95/f63 0 2024-04-27T04:09:58.928 INFO:tasks.workunit.client.0.smithi162.stdout:8/173: rename d0/d10/d2b/f26 to d0/de/d29/f33 0 2024-04-27T04:09:58.929 INFO:tasks.workunit.client.0.smithi162.stdout:8/174: chown d0/d10/d2b/d2d/d2f 17073563 1 2024-04-27T04:09:58.930 INFO:tasks.workunit.client.1.smithi181.stdout:5/581: mknod d4/de/d2c/d2e/d8f/cd4 0 2024-04-27T04:09:58.930 INFO:tasks.workunit.client.0.smithi162.stdout:4/176: fdatasync d2/da/dd/f10 0 2024-04-27T04:09:58.931 INFO:tasks.workunit.client.1.smithi181.stdout:7/399: dread d1/d22/d15/d1f/d34/f3c [0,4194304] 0 2024-04-27T04:09:58.931 INFO:tasks.workunit.client.0.smithi162.stdout:4/177: write d2/d25/f36 [480172,90617] 0 2024-04-27T04:09:58.932 INFO:tasks.workunit.client.1.smithi181.stdout:9/465: sync 2024-04-27T04:09:58.932 INFO:tasks.workunit.client.1.smithi181.stdout:4/401: sync 2024-04-27T04:09:58.932 INFO:tasks.workunit.client.1.smithi181.stdout:8/514: sync 2024-04-27T04:09:58.934 INFO:tasks.workunit.client.0.smithi162.stdout:6/160: mkdir d6/d1a/d34/d3a 0 2024-04-27T04:09:58.934 INFO:tasks.workunit.client.0.smithi162.stdout:6/161: chown d6/d1a/cf 2 1 2024-04-27T04:09:58.934 INFO:tasks.workunit.client.1.smithi181.stdout:9/466: chown d0/d44/d4b/d43/d88/d95 1025544 1 2024-04-27T04:09:58.938 INFO:tasks.workunit.client.0.smithi162.stdout:9/155: creat d3/d13/d2d/f37 x:0 0 0 2024-04-27T04:09:58.938 INFO:tasks.workunit.client.1.smithi181.stdout:0/454: mknod dc/d1f/d3b/d4a/d95/d26/d68/ca3 0 2024-04-27T04:09:58.939 INFO:tasks.workunit.client.0.smithi162.stdout:1/152: link d0/d11/f20 d0/d11/f2c 0 2024-04-27T04:09:58.939 INFO:tasks.workunit.client.1.smithi181.stdout:0/455: chown dc/d1f/d3b/d75 7 1 2024-04-27T04:09:58.939 INFO:tasks.workunit.client.0.smithi162.stdout:1/153: stat d0/d11/d1a/f28 0 2024-04-27T04:09:58.940 INFO:tasks.workunit.client.1.smithi181.stdout:6/352: rmdir d7/dc/d1c/d34/d3f 39 2024-04-27T04:09:58.941 INFO:tasks.workunit.client.1.smithi181.stdout:6/353: fsync d7/d27/f4c 0 2024-04-27T04:09:58.943 INFO:tasks.workunit.client.1.smithi181.stdout:5/582: unlink d4/de/d2c/d46/d4d/d8e/d67/d71/fb4 0 2024-04-27T04:09:58.944 INFO:tasks.workunit.client.0.smithi162.stdout:5/148: creat d8/d2c/f33 x:0 0 0 2024-04-27T04:09:58.945 INFO:tasks.workunit.client.1.smithi181.stdout:5/583: write d4/fb0 [215466,33853] 0 2024-04-27T04:09:58.945 INFO:tasks.workunit.client.0.smithi162.stdout:5/149: chown d8/d9/f32 0 1 2024-04-27T04:09:58.949 INFO:tasks.workunit.client.1.smithi181.stdout:4/402: truncate d0/d15/f2f 2493827 0 2024-04-27T04:09:58.951 INFO:tasks.workunit.client.0.smithi162.stdout:5/150: dwrite d8/d1a/f2d [0,4194304] 0 2024-04-27T04:09:58.952 INFO:tasks.workunit.client.1.smithi181.stdout:5/584: dwrite d4/da3/fa7 [0,4194304] 0 2024-04-27T04:09:58.952 INFO:tasks.workunit.client.0.smithi162.stdout:0/206: rename d5/dc/d1e/c28 to d5/d26/c44 0 2024-04-27T04:09:58.953 INFO:tasks.workunit.client.1.smithi181.stdout:5/585: chown d4/de/d2c/c7e 232779 1 2024-04-27T04:09:58.955 INFO:tasks.workunit.client.0.smithi162.stdout:8/175: unlink d0/d10/l1b 0 2024-04-27T04:09:58.956 INFO:tasks.workunit.client.0.smithi162.stdout:8/176: read d0/fd [294672,49441] 0 2024-04-27T04:09:58.957 INFO:tasks.workunit.client.1.smithi181.stdout:4/403: dwrite d0/d15/d25/d40/f4f [0,4194304] 0 2024-04-27T04:09:58.961 INFO:tasks.workunit.client.0.smithi162.stdout:4/178: mknod d2/d25/d1c/d24/c3e 0 2024-04-27T04:09:58.962 INFO:tasks.workunit.client.0.smithi162.stdout:4/179: fsync d2/da/dd/f10 0 2024-04-27T04:09:58.963 INFO:tasks.workunit.client.0.smithi162.stdout:6/162: creat d6/db/d32/f3b x:0 0 0 2024-04-27T04:09:58.964 INFO:tasks.workunit.client.0.smithi162.stdout:4/180: write d2/f11 [982973,94995] 0 2024-04-27T04:09:58.965 INFO:tasks.workunit.client.0.smithi162.stdout:4/181: fsync d2/d25/d1c/f33 0 2024-04-27T04:09:58.969 INFO:tasks.workunit.client.0.smithi162.stdout:2/119: link d0/l1f d0/d24/l25 0 2024-04-27T04:09:58.970 INFO:tasks.workunit.client.1.smithi181.stdout:2/443: write d4/d9/d4f/f5b [11631,112585] 0 2024-04-27T04:09:58.970 INFO:tasks.workunit.client.1.smithi181.stdout:0/456: creat dc/d1f/d3b/d4a/d7a/d9d/fa4 x:0 0 0 2024-04-27T04:09:58.975 INFO:tasks.workunit.client.0.smithi162.stdout:4/182: dwrite d2/da/d1a/f30 [0,4194304] 0 2024-04-27T04:09:58.975 INFO:tasks.workunit.client.1.smithi181.stdout:6/354: unlink d7/dc/d1e/c48 0 2024-04-27T04:09:58.979 INFO:tasks.workunit.client.1.smithi181.stdout:7/400: rename d1/d2/d1c/d2d/d61/l7b to d1/l83 0 2024-04-27T04:09:58.982 INFO:tasks.workunit.client.0.smithi162.stdout:8/177: unlink d0/d10/c1c 0 2024-04-27T04:09:58.982 INFO:tasks.workunit.client.0.smithi162.stdout:7/112: write d3/d5/d9/f13 [1842186,35739] 0 2024-04-27T04:09:58.983 INFO:tasks.workunit.client.1.smithi181.stdout:6/355: dwrite d7/f26 [0,4194304] 0 2024-04-27T04:09:58.985 INFO:tasks.workunit.client.1.smithi181.stdout:3/426: sync 2024-04-27T04:09:58.986 INFO:tasks.workunit.client.1.smithi181.stdout:1/494: creat d1/d4c/d1e/d20/d53/d56/f9c x:0 0 0 2024-04-27T04:09:58.988 INFO:tasks.workunit.client.1.smithi181.stdout:0/457: mkdir dc/d1f/da5 0 2024-04-27T04:09:58.989 INFO:tasks.workunit.client.1.smithi181.stdout:3/427: stat d0/d6c/d52/d44/d74 0 2024-04-27T04:09:58.990 INFO:tasks.workunit.client.1.smithi181.stdout:7/401: dwrite d1/d2/f24 [0,4194304] 0 2024-04-27T04:09:58.993 INFO:tasks.workunit.client.0.smithi162.stdout:1/154: creat d0/d7/d2a/f2d x:0 0 0 2024-04-27T04:09:58.998 INFO:tasks.workunit.client.0.smithi162.stdout:2/120: stat d0/l1f 0 2024-04-27T04:09:59.000 INFO:tasks.workunit.client.1.smithi181.stdout:3/428: dwrite d0/de/d20/f5e [0,4194304] 0 2024-04-27T04:09:59.005 INFO:tasks.workunit.client.1.smithi181.stdout:7/402: dwrite d1/d2/d1c/f2c [0,4194304] 0 2024-04-27T04:09:59.005 INFO:tasks.workunit.client.0.smithi162.stdout:2/121: dread d0/d4/f9 [0,4194304] 0 2024-04-27T04:09:59.006 INFO:tasks.workunit.client.1.smithi181.stdout:8/515: rename d1/d6/d6f/c74 to d1/d8c/cb8 0 2024-04-27T04:09:59.008 INFO:tasks.workunit.client.1.smithi181.stdout:7/403: write d1/d2/d35/d67/f7e [473713,19378] 0 2024-04-27T04:09:59.008 INFO:tasks.workunit.client.0.smithi162.stdout:5/151: symlink d8/l34 0 2024-04-27T04:09:59.008 INFO:tasks.workunit.client.0.smithi162.stdout:5/152: chown d8/ce 2058535512 1 2024-04-27T04:09:59.012 INFO:tasks.workunit.client.0.smithi162.stdout:3/158: rename d1/d18/d23/l26 to d1/d21/l36 0 2024-04-27T04:09:59.014 INFO:tasks.workunit.client.1.smithi181.stdout:3/429: dwrite d0/f23 [0,4194304] 0 2024-04-27T04:09:59.014 INFO:tasks.workunit.client.1.smithi181.stdout:6/356: symlink d7/dc/d1e/d5c/l79 0 2024-04-27T04:09:59.015 INFO:tasks.workunit.client.0.smithi162.stdout:8/178: symlink d0/d10/l34 0 2024-04-27T04:09:59.018 INFO:tasks.workunit.client.0.smithi162.stdout:7/113: creat d3/d5/d9/f21 x:0 0 0 2024-04-27T04:09:59.025 INFO:tasks.workunit.client.0.smithi162.stdout:1/155: creat d0/d11/f2e x:0 0 0 2024-04-27T04:09:59.029 INFO:tasks.workunit.client.0.smithi162.stdout:5/153: symlink d8/d2c/d2e/l35 0 2024-04-27T04:09:59.030 INFO:tasks.workunit.client.0.smithi162.stdout:5/154: write d8/d1a/d23/f28 [4112893,103716] 0 2024-04-27T04:09:59.031 INFO:tasks.workunit.client.0.smithi162.stdout:5/155: write f4 [830490,17591] 0 2024-04-27T04:09:59.035 INFO:tasks.workunit.client.1.smithi181.stdout:9/467: rename d0/d44/d4b/d43/d88/d95 to d0/d44/d4b/d5f/da4 0 2024-04-27T04:09:59.036 INFO:tasks.workunit.client.0.smithi162.stdout:5/156: dwrite f4 [4194304,4194304] 0 2024-04-27T04:09:59.039 INFO:tasks.workunit.client.1.smithi181.stdout:6/357: write d7/d31/f37 [4845741,26131] 0 2024-04-27T04:09:59.043 INFO:tasks.workunit.client.0.smithi162.stdout:2/122: symlink d0/l26 0 2024-04-27T04:09:59.047 INFO:tasks.workunit.client.0.smithi162.stdout:0/207: rename d5/dc/d1e/c24 to d5/dc/c45 0 2024-04-27T04:09:59.049 INFO:tasks.workunit.client.1.smithi181.stdout:0/458: dwrite dc/d54/f5b [0,4194304] 0 2024-04-27T04:09:59.050 INFO:tasks.workunit.client.1.smithi181.stdout:2/444: link d4/d9/d16/d82/l7f d4/d9/d4f/l8f 0 2024-04-27T04:09:59.053 INFO:tasks.workunit.client.1.smithi181.stdout:2/445: dread d4/d9/d4f/f5b [0,4194304] 0 2024-04-27T04:09:59.053 INFO:tasks.workunit.client.1.smithi181.stdout:5/586: getdents d4/de 0 2024-04-27T04:09:59.054 INFO:tasks.workunit.client.0.smithi162.stdout:3/159: mkdir d1/d37 0 2024-04-27T04:09:59.056 INFO:tasks.workunit.client.1.smithi181.stdout:8/516: fdatasync d1/d20/f24 0 2024-04-27T04:09:59.058 INFO:tasks.workunit.client.0.smithi162.stdout:8/179: mkdir d0/d10/d2b/d2e/d35 0 2024-04-27T04:09:59.060 INFO:tasks.workunit.client.0.smithi162.stdout:7/114: symlink d3/l22 0 2024-04-27T04:09:59.063 INFO:tasks.workunit.client.0.smithi162.stdout:5/157: symlink d8/d2c/d2e/l36 0 2024-04-27T04:09:59.063 INFO:tasks.workunit.client.1.smithi181.stdout:9/468: dread d0/d44/d4b/d43/f56 [0,4194304] 0 2024-04-27T04:09:59.064 INFO:tasks.workunit.client.0.smithi162.stdout:5/158: chown d8/d9/c15 61 1 2024-04-27T04:09:59.064 INFO:tasks.workunit.client.1.smithi181.stdout:7/404: unlink d1/df/d62/c7d 0 2024-04-27T04:09:59.065 INFO:tasks.workunit.client.0.smithi162.stdout:7/115: dwrite d3/d5/d9/f13 [0,4194304] 0 2024-04-27T04:09:59.065 INFO:tasks.workunit.client.0.smithi162.stdout:6/163: getdents d6/db 0 2024-04-27T04:09:59.066 INFO:tasks.workunit.client.0.smithi162.stdout:6/164: chown d6/c1b 0 1 2024-04-27T04:09:59.067 INFO:tasks.workunit.client.1.smithi181.stdout:7/405: dread d1/d2/d35/d67/f7e [0,4194304] 0 2024-04-27T04:09:59.069 INFO:tasks.workunit.client.0.smithi162.stdout:2/123: rename d0/d4/c8 to d0/d4/c27 0 2024-04-27T04:09:59.069 INFO:tasks.workunit.client.1.smithi181.stdout:3/430: getdents d0/d28/d86 0 2024-04-27T04:09:59.071 INFO:tasks.workunit.client.0.smithi162.stdout:0/208: rmdir d5 39 2024-04-27T04:09:59.072 INFO:tasks.workunit.client.1.smithi181.stdout:1/495: creat d1/d4c/d1e/d20/f9d x:0 0 0 2024-04-27T04:09:59.074 INFO:tasks.workunit.client.0.smithi162.stdout:3/160: rename d1/l22 to d1/d18/d23/l38 0 2024-04-27T04:09:59.075 INFO:tasks.workunit.client.0.smithi162.stdout:6/165: dwrite d6/f11 [0,4194304] 0 2024-04-27T04:09:59.075 INFO:tasks.workunit.client.0.smithi162.stdout:7/116: dwrite d3/d5/d9/f17 [0,4194304] 0 2024-04-27T04:09:59.078 INFO:tasks.workunit.client.0.smithi162.stdout:3/161: write d1/d18/f13 [845961,54284] 0 2024-04-27T04:09:59.078 INFO:tasks.workunit.client.0.smithi162.stdout:8/180: creat d0/d10/d25/f36 x:0 0 0 2024-04-27T04:09:59.080 INFO:tasks.workunit.client.0.smithi162.stdout:6/166: write d6/f25 [1821329,73480] 0 2024-04-27T04:09:59.082 INFO:tasks.workunit.client.0.smithi162.stdout:2/124: dwrite d0/d4/f15 [4194304,4194304] 0 2024-04-27T04:09:59.082 INFO:tasks.workunit.client.1.smithi181.stdout:0/459: symlink dc/d1f/d3b/d4a/d95/la6 0 2024-04-27T04:09:59.084 INFO:tasks.workunit.client.0.smithi162.stdout:2/125: write d0/d4/f22 [2538320,105195] 0 2024-04-27T04:09:59.084 INFO:tasks.workunit.client.0.smithi162.stdout:5/159: symlink d8/d1a/d29/l37 0 2024-04-27T04:09:59.085 INFO:tasks.workunit.client.0.smithi162.stdout:2/126: write d0/d4/f16 [1367449,120879] 0 2024-04-27T04:09:59.086 INFO:tasks.workunit.client.0.smithi162.stdout:2/127: chown d0/d4/c23 3 1 2024-04-27T04:09:59.091 INFO:tasks.workunit.client.1.smithi181.stdout:5/587: rmdir d4/de/d2c/d2e/d55 39 2024-04-27T04:09:59.096 INFO:tasks.workunit.client.1.smithi181.stdout:5/588: read d4/de/d2c/d46/d4d/d8e/f41 [7954252,12004] 0 2024-04-27T04:09:59.097 INFO:tasks.workunit.client.1.smithi181.stdout:5/589: write d4/fc9 [564848,35921] 0 2024-04-27T04:09:59.097 INFO:tasks.workunit.client.0.smithi162.stdout:0/209: read d5/dc/f22 [1716873,23965] 0 2024-04-27T04:09:59.098 INFO:tasks.workunit.client.1.smithi181.stdout:1/496: dread d1/d4c/d1e/f8f [0,4194304] 0 2024-04-27T04:09:59.100 INFO:tasks.workunit.client.1.smithi181.stdout:4/404: rename d0/d15/d25/c44 to d0/d15/d25/d2a/d36/c86 0 2024-04-27T04:09:59.101 INFO:tasks.workunit.client.0.smithi162.stdout:3/162: creat d1/d21/f39 x:0 0 0 2024-04-27T04:09:59.103 INFO:tasks.workunit.client.0.smithi162.stdout:7/117: rename d3/d5/c1a to d3/d5/d9/dc/de/c23 0 2024-04-27T04:09:59.104 INFO:tasks.workunit.client.1.smithi181.stdout:4/405: read d0/d15/d25/d2b/d6d/f74 [541939,37584] 0 2024-04-27T04:09:59.104 INFO:tasks.workunit.client.0.smithi162.stdout:6/167: symlink d6/d1a/d34/l3c 0 2024-04-27T04:09:59.105 INFO:tasks.workunit.client.1.smithi181.stdout:4/406: readlink d0/d15/d25/l64 0 2024-04-27T04:09:59.106 INFO:tasks.workunit.client.0.smithi162.stdout:6/168: write d6/d1a/f20 [267594,127892] 0 2024-04-27T04:09:59.106 INFO:tasks.workunit.client.0.smithi162.stdout:7/118: write d3/d5/d9/dc/de/f1c [233996,19139] 0 2024-04-27T04:09:59.108 INFO:tasks.workunit.client.0.smithi162.stdout:7/119: dread - d3/f16 zero size 2024-04-27T04:09:59.108 INFO:tasks.workunit.client.0.smithi162.stdout:3/163: dwrite d1/d18/f1d [0,4194304] 0 2024-04-27T04:09:59.109 INFO:tasks.workunit.client.1.smithi181.stdout:7/406: creat d1/d22/d15/d29/f84 x:0 0 0 2024-04-27T04:09:59.114 INFO:tasks.workunit.client.1.smithi181.stdout:9/469: dwrite d0/f1f [4194304,4194304] 0 2024-04-27T04:09:59.114 INFO:tasks.workunit.client.0.smithi162.stdout:8/181: rename d0/d10/d25/l2a to d0/d10/l37 0 2024-04-27T04:09:59.114 INFO:tasks.workunit.client.0.smithi162.stdout:3/164: dwrite d1/d18/f1d [0,4194304] 0 2024-04-27T04:09:59.122 INFO:tasks.workunit.client.0.smithi162.stdout:8/182: dwrite d0/f2 [4194304,4194304] 0 2024-04-27T04:09:59.122 INFO:tasks.workunit.client.1.smithi181.stdout:0/460: creat dc/d1f/d3b/d4a/d95/d26/d41/d6f/d7d/fa7 x:0 0 0 2024-04-27T04:09:59.124 INFO:tasks.workunit.client.0.smithi162.stdout:8/183: write d0/d10/d25/f36 [333133,91816] 0 2024-04-27T04:09:59.125 INFO:tasks.workunit.client.1.smithi181.stdout:7/407: dread d1/d2/f25 [0,4194304] 0 2024-04-27T04:09:59.126 INFO:tasks.workunit.client.1.smithi181.stdout:2/446: unlink d4/d9/d4f/c89 0 2024-04-27T04:09:59.126 INFO:tasks.workunit.client.1.smithi181.stdout:7/408: write d1/df/f5d [493197,27935] 0 2024-04-27T04:09:59.129 INFO:tasks.workunit.client.0.smithi162.stdout:6/169: fdatasync d6/f35 0 2024-04-27T04:09:59.135 INFO:tasks.workunit.client.1.smithi181.stdout:8/517: dwrite d1/d20/fa2 [0,4194304] 0 2024-04-27T04:09:59.137 INFO:tasks.workunit.client.1.smithi181.stdout:1/497: creat d1/d4c/d52/f9e x:0 0 0 2024-04-27T04:09:59.140 INFO:tasks.workunit.client.1.smithi181.stdout:5/590: mknod d4/de/d2f/d7a/cd5 0 2024-04-27T04:09:59.141 INFO:tasks.workunit.client.0.smithi162.stdout:0/210: mkdir d5/dc/d1e/d46 0 2024-04-27T04:09:59.143 INFO:tasks.workunit.client.1.smithi181.stdout:6/358: rename d7/dc/d1e/d2b/f4e to d7/d31/d38/f7a 0 2024-04-27T04:09:59.148 INFO:tasks.workunit.client.0.smithi162.stdout:8/184: unlink d0/d10/d2b/l32 0 2024-04-27T04:09:59.153 INFO:tasks.workunit.client.0.smithi162.stdout:2/128: rmdir d0/d4/d1b 0 2024-04-27T04:09:59.155 INFO:tasks.workunit.client.0.smithi162.stdout:0/211: symlink d5/l47 0 2024-04-27T04:09:59.155 INFO:tasks.workunit.client.0.smithi162.stdout:0/212: chown d5/f3b 0 1 2024-04-27T04:09:59.158 INFO:tasks.workunit.client.1.smithi181.stdout:3/431: mknod d0/c8a 0 2024-04-27T04:09:59.159 INFO:tasks.workunit.client.1.smithi181.stdout:9/470: chown d0/l21 45 1 2024-04-27T04:09:59.159 INFO:tasks.workunit.client.1.smithi181.stdout:8/518: dread d1/d20/f7a [0,4194304] 0 2024-04-27T04:09:59.160 INFO:tasks.workunit.client.0.smithi162.stdout:8/185: chown d0/l9 248872 1 2024-04-27T04:09:59.160 INFO:tasks.workunit.client.0.smithi162.stdout:8/186: fdatasync d0/d10/d2b/f19 0 2024-04-27T04:09:59.162 INFO:tasks.workunit.client.1.smithi181.stdout:0/461: creat dc/d1f/d3b/d4a/d95/d26/d41/fa8 x:0 0 0 2024-04-27T04:09:59.163 INFO:tasks.workunit.client.0.smithi162.stdout:6/170: creat d6/d1a/d34/d3a/f3d x:0 0 0 2024-04-27T04:09:59.163 INFO:tasks.workunit.client.0.smithi162.stdout:6/171: fsync d6/fa 0 2024-04-27T04:09:59.166 INFO:tasks.workunit.client.1.smithi181.stdout:7/409: chown d1/df/d62/c70 550050 1 2024-04-27T04:09:59.169 INFO:tasks.workunit.client.1.smithi181.stdout:7/410: read d1/d2/d1c/d37/f4f [2275762,81292] 0 2024-04-27T04:09:59.169 INFO:tasks.workunit.client.0.smithi162.stdout:5/160: rename d8/d9/l19 to d8/l38 0 2024-04-27T04:09:59.170 INFO:tasks.workunit.client.0.smithi162.stdout:8/187: mknod d0/d10/c38 0 2024-04-27T04:09:59.175 INFO:tasks.workunit.client.1.smithi181.stdout:1/498: rmdir d1/d2/d5 39 2024-04-27T04:09:59.177 INFO:tasks.workunit.client.0.smithi162.stdout:6/172: creat d6/d1a/d34/f3e x:0 0 0 2024-04-27T04:09:59.178 INFO:tasks.workunit.client.1.smithi181.stdout:5/591: creat d4/d10/d29/d92/fd6 x:0 0 0 2024-04-27T04:09:59.181 INFO:tasks.workunit.client.0.smithi162.stdout:6/173: dwrite d6/f35 [0,4194304] 0 2024-04-27T04:09:59.181 INFO:tasks.workunit.client.0.smithi162.stdout:6/174: fsync d6/f30 0 2024-04-27T04:09:59.182 INFO:tasks.workunit.client.0.smithi162.stdout:0/213: mkdir d5/dc/d1e/d46/d48 0 2024-04-27T04:09:59.183 INFO:tasks.workunit.client.1.smithi181.stdout:6/359: creat d7/d31/d38/f7b x:0 0 0 2024-04-27T04:09:59.184 INFO:tasks.workunit.client.1.smithi181.stdout:6/360: stat d7/d27/f4c 0 2024-04-27T04:09:59.185 INFO:tasks.workunit.client.1.smithi181.stdout:6/361: readlink d7/d1b/l3d 0 2024-04-27T04:09:59.185 INFO:tasks.workunit.client.0.smithi162.stdout:3/165: rename d1/c35 to d1/d18/d23/d34/c3a 0 2024-04-27T04:09:59.185 INFO:tasks.workunit.client.0.smithi162.stdout:3/166: chown d1/d21 15624 1 2024-04-27T04:09:59.187 INFO:tasks.workunit.client.1.smithi181.stdout:4/407: symlink d0/d15/de/l87 0 2024-04-27T04:09:59.188 INFO:tasks.workunit.client.1.smithi181.stdout:2/447: dwrite d4/d9/d16/f23 [0,4194304] 0 2024-04-27T04:09:59.189 INFO:tasks.workunit.client.0.smithi162.stdout:2/129: link d0/c1c d0/d4/d17/c28 0 2024-04-27T04:09:59.193 INFO:tasks.workunit.client.0.smithi162.stdout:0/214: mknod d5/c49 0 2024-04-27T04:09:59.194 INFO:tasks.workunit.client.1.smithi181.stdout:6/362: dwrite d7/dc/ff [0,4194304] 0 2024-04-27T04:09:59.194 INFO:tasks.workunit.client.0.smithi162.stdout:0/215: dread - d5/f11 zero size 2024-04-27T04:09:59.195 INFO:tasks.workunit.client.1.smithi181.stdout:6/363: fsync d7/dc/d1c/d34/f5d 0 2024-04-27T04:09:59.213 INFO:tasks.workunit.client.0.smithi162.stdout:6/175: mknod d6/c3f 0 2024-04-27T04:09:59.213 INFO:tasks.workunit.client.0.smithi162.stdout:0/216: mkdir d5/d26/d4a 0 2024-04-27T04:09:59.214 INFO:tasks.workunit.client.0.smithi162.stdout:3/167: mknod d1/d18/d23/d2f/c3b 0 2024-04-27T04:09:59.214 INFO:tasks.workunit.client.0.smithi162.stdout:4/183: sync 2024-04-27T04:09:59.214 INFO:tasks.workunit.client.0.smithi162.stdout:9/156: sync 2024-04-27T04:09:59.214 INFO:tasks.workunit.client.0.smithi162.stdout:2/130: mknod d0/d24/c29 0 2024-04-27T04:09:59.215 INFO:tasks.workunit.client.0.smithi162.stdout:9/157: readlink d3/d13/d15/l1d 0 2024-04-27T04:09:59.215 INFO:tasks.workunit.client.0.smithi162.stdout:5/161: getdents d8/d2c 0 2024-04-27T04:09:59.217 INFO:tasks.workunit.client.0.smithi162.stdout:8/188: truncate d0/f2 4300054 0 2024-04-27T04:09:59.218 INFO:tasks.workunit.client.0.smithi162.stdout:4/184: truncate d2/d25/d1c/f3d 683982 0 2024-04-27T04:09:59.218 INFO:tasks.workunit.client.0.smithi162.stdout:9/158: write d3/d23/d1e/f27 [1619581,18633] 0 2024-04-27T04:09:59.220 INFO:tasks.workunit.client.0.smithi162.stdout:3/168: unlink d1/l32 0 2024-04-27T04:09:59.223 INFO:tasks.workunit.client.0.smithi162.stdout:2/131: mkdir d0/d4/d2a 0 2024-04-27T04:09:59.224 INFO:tasks.workunit.client.0.smithi162.stdout:2/132: chown d0/d4/d17 1935 1 2024-04-27T04:09:59.225 INFO:tasks.workunit.client.0.smithi162.stdout:5/162: creat d8/d1a/d23/f39 x:0 0 0 2024-04-27T04:09:59.231 INFO:tasks.workunit.client.0.smithi162.stdout:4/185: mknod d2/d25/c3f 0 2024-04-27T04:09:59.233 INFO:tasks.workunit.client.0.smithi162.stdout:9/159: creat d3/d13/d2d/d30/f38 x:0 0 0 2024-04-27T04:09:59.234 INFO:tasks.workunit.client.0.smithi162.stdout:9/160: write d3/fc [763134,69287] 0 2024-04-27T04:09:59.234 INFO:tasks.workunit.client.0.smithi162.stdout:3/169: unlink d1/d18/f6 0 2024-04-27T04:09:59.235 INFO:tasks.workunit.client.0.smithi162.stdout:9/161: chown d3/d13/d2d 2619 1 2024-04-27T04:09:59.236 INFO:tasks.workunit.client.0.smithi162.stdout:3/170: readlink d1/d18/d23/l29 0 2024-04-27T04:09:59.236 INFO:tasks.workunit.client.0.smithi162.stdout:4/186: dwrite d2/da/dd/f2a [0,4194304] 0 2024-04-27T04:09:59.237 INFO:tasks.workunit.client.0.smithi162.stdout:0/217: symlink d5/dc/d1e/d46/d48/l4b 0 2024-04-27T04:09:59.238 INFO:tasks.workunit.client.0.smithi162.stdout:0/218: write d5/dc/f14 [4310925,93740] 0 2024-04-27T04:09:59.239 INFO:tasks.workunit.client.0.smithi162.stdout:2/133: mkdir d0/d4/d17/d2b 0 2024-04-27T04:09:59.240 INFO:tasks.workunit.client.0.smithi162.stdout:0/219: fdatasync d5/dc/d1e/f3e 0 2024-04-27T04:09:59.242 INFO:tasks.workunit.client.0.smithi162.stdout:5/163: truncate d8/d9/fc 1191378 0 2024-04-27T04:09:59.243 INFO:tasks.workunit.client.0.smithi162.stdout:5/164: readlink d8/la 0 2024-04-27T04:09:59.243 INFO:tasks.workunit.client.1.smithi181.stdout:9/471: fdatasync d0/d44/d4f/f6b 0 2024-04-27T04:09:59.246 INFO:tasks.workunit.client.0.smithi162.stdout:4/187: dwrite d2/ff [0,4194304] 0 2024-04-27T04:09:59.246 INFO:tasks.workunit.client.0.smithi162.stdout:1/156: sync 2024-04-27T04:09:59.246 INFO:tasks.workunit.client.0.smithi162.stdout:7/120: sync 2024-04-27T04:09:59.247 INFO:tasks.workunit.client.1.smithi181.stdout:8/519: creat d1/d6/d2f/d40/fb9 x:0 0 0 2024-04-27T04:09:59.249 INFO:tasks.workunit.client.0.smithi162.stdout:2/134: dwrite d0/f7 [0,4194304] 0 2024-04-27T04:09:59.250 INFO:tasks.workunit.client.0.smithi162.stdout:2/135: chown d0/f6 755122 1 2024-04-27T04:09:59.250 INFO:tasks.workunit.client.0.smithi162.stdout:5/165: dread d8/f12 [0,4194304] 0 2024-04-27T04:09:59.252 INFO:tasks.workunit.client.0.smithi162.stdout:3/171: creat d1/d18/d23/d34/f3c x:0 0 0 2024-04-27T04:09:59.256 INFO:tasks.workunit.client.1.smithi181.stdout:7/411: readlink d1/l2e 0 2024-04-27T04:09:59.256 INFO:tasks.workunit.client.0.smithi162.stdout:8/189: dwrite d0/f7 [0,4194304] 0 2024-04-27T04:09:59.257 INFO:tasks.workunit.client.1.smithi181.stdout:7/412: dread - d1/d22/d15/d29/f43 zero size 2024-04-27T04:09:59.261 INFO:tasks.workunit.client.0.smithi162.stdout:6/176: dwrite d6/f26 [0,4194304] 0 2024-04-27T04:09:59.261 INFO:tasks.workunit.client.1.smithi181.stdout:1/499: stat d1/d2/d5/d6/f3d 0 2024-04-27T04:09:59.263 INFO:tasks.workunit.client.0.smithi162.stdout:4/188: mknod d2/da/d1a/c40 0 2024-04-27T04:09:59.263 INFO:tasks.workunit.client.0.smithi162.stdout:6/177: chown d6/d1a/d34/c37 3 1 2024-04-27T04:09:59.264 INFO:tasks.workunit.client.0.smithi162.stdout:3/172: dwrite d1/d18/d23/f2e [0,4194304] 0 2024-04-27T04:09:59.265 INFO:tasks.workunit.client.1.smithi181.stdout:5/592: creat d4/d10/d72/d80/d85/fd7 x:0 0 0 2024-04-27T04:09:59.265 INFO:tasks.workunit.client.0.smithi162.stdout:4/189: dread - d2/d25/f2d zero size 2024-04-27T04:09:59.266 INFO:tasks.workunit.client.0.smithi162.stdout:3/173: write d1/d18/f1d [4739144,72426] 0 2024-04-27T04:09:59.266 INFO:tasks.workunit.client.0.smithi162.stdout:6/178: chown d6/d1a/d34/l3c 65 1 2024-04-27T04:09:59.269 INFO:tasks.workunit.client.0.smithi162.stdout:9/162: mkdir d3/d39 0 2024-04-27T04:09:59.270 INFO:tasks.workunit.client.0.smithi162.stdout:9/163: chown d3/d11/l25 384 1 2024-04-27T04:09:59.276 INFO:tasks.workunit.client.1.smithi181.stdout:4/408: rename d0/d15/de/d60/l83 to d0/d15/de/d19/d59/l88 0 2024-04-27T04:09:59.279 INFO:tasks.workunit.client.0.smithi162.stdout:9/164: dwrite d3/d23/d1e/f2b [0,4194304] 0 2024-04-27T04:09:59.280 INFO:tasks.workunit.client.1.smithi181.stdout:0/462: write dc/d1f/d3b/d4a/f62 [423645,81903] 0 2024-04-27T04:09:59.282 INFO:tasks.workunit.client.1.smithi181.stdout:3/432: symlink d0/de/l8b 0 2024-04-27T04:09:59.282 INFO:tasks.workunit.client.1.smithi181.stdout:3/433: stat d0/d6c/d52/d44/l49 0 2024-04-27T04:09:59.283 INFO:tasks.workunit.client.0.smithi162.stdout:0/220: creat d5/d26/d4a/f4c x:0 0 0 2024-04-27T04:09:59.283 INFO:tasks.workunit.client.0.smithi162.stdout:9/165: write d3/f21 [767487,125925] 0 2024-04-27T04:09:59.284 INFO:tasks.workunit.client.0.smithi162.stdout:6/179: dread d6/fa [0,4194304] 0 2024-04-27T04:09:59.286 INFO:tasks.workunit.client.0.smithi162.stdout:5/166: dread d8/d9/f10 [0,4194304] 0 2024-04-27T04:09:59.286 INFO:tasks.workunit.client.0.smithi162.stdout:9/166: fdatasync d3/d23/f22 0 2024-04-27T04:09:59.286 INFO:tasks.workunit.client.0.smithi162.stdout:5/167: fsync d8/f26 0 2024-04-27T04:09:59.289 INFO:tasks.workunit.client.1.smithi181.stdout:8/520: truncate d1/d6/d2f/d30/d5c/f61 4374891 0 2024-04-27T04:09:59.290 INFO:tasks.workunit.client.1.smithi181.stdout:8/521: readlink d1/d6/d2f/d30/d5c/l7e 0 2024-04-27T04:09:59.291 INFO:tasks.workunit.client.0.smithi162.stdout:1/157: symlink d0/l2f 0 2024-04-27T04:09:59.291 INFO:tasks.workunit.client.1.smithi181.stdout:2/448: write d4/f32 [585712,106054] 0 2024-04-27T04:09:59.291 INFO:tasks.workunit.client.0.smithi162.stdout:0/221: dwrite d5/dc/d1e/d41/f34 [0,4194304] 0 2024-04-27T04:09:59.293 INFO:tasks.workunit.client.1.smithi181.stdout:8/522: dread d1/d6/d2f/d30/d5c/f97 [0,4194304] 0 2024-04-27T04:09:59.293 INFO:tasks.workunit.client.0.smithi162.stdout:0/222: truncate d5/dc/d1e/f30 997206 0 2024-04-27T04:09:59.294 INFO:tasks.workunit.client.1.smithi181.stdout:7/413: creat d1/d2/d35/f85 x:0 0 0 2024-04-27T04:09:59.295 INFO:tasks.workunit.client.1.smithi181.stdout:7/414: fsync d1/d2/d1c/f2c 0 2024-04-27T04:09:59.296 INFO:tasks.workunit.client.1.smithi181.stdout:7/415: fsync d1/df/d1a/f74 0 2024-04-27T04:09:59.296 INFO:tasks.workunit.client.1.smithi181.stdout:1/500: unlink d1/d4c/d1e/d20/d53/d56/f9c 0 2024-04-27T04:09:59.299 INFO:tasks.workunit.client.1.smithi181.stdout:5/593: mknod d4/da3/dbc/cd8 0 2024-04-27T04:09:59.300 INFO:tasks.workunit.client.0.smithi162.stdout:3/174: chown d1/d21/l36 0 1 2024-04-27T04:09:59.301 INFO:tasks.workunit.client.0.smithi162.stdout:4/190: symlink d2/da/d2c/l41 0 2024-04-27T04:09:59.302 INFO:tasks.workunit.client.0.smithi162.stdout:3/175: write d1/d21/f2c [1427421,28412] 0 2024-04-27T04:09:59.302 INFO:tasks.workunit.client.0.smithi162.stdout:4/191: fdatasync d2/d25/d1c/f33 0 2024-04-27T04:09:59.304 INFO:tasks.workunit.client.0.smithi162.stdout:2/136: symlink d0/d4/d2a/l2c 0 2024-04-27T04:09:59.305 INFO:tasks.workunit.client.1.smithi181.stdout:4/409: readlink d0/d15/de/d19/d59/l88 0 2024-04-27T04:09:59.306 INFO:tasks.workunit.client.1.smithi181.stdout:0/463: dread dc/f15 [0,4194304] 0 2024-04-27T04:09:59.306 INFO:tasks.workunit.client.0.smithi162.stdout:4/192: dread d2/da/dd/f10 [0,4194304] 0 2024-04-27T04:09:59.307 INFO:tasks.workunit.client.0.smithi162.stdout:4/193: fsync d2/da/dd/f10 0 2024-04-27T04:09:59.307 INFO:tasks.workunit.client.1.smithi181.stdout:0/464: write dc/d1f/d3b/d4a/d95/d26/f6c [1176419,18206] 0 2024-04-27T04:09:59.309 INFO:tasks.workunit.client.0.smithi162.stdout:3/176: dwrite d1/d21/f2c [0,4194304] 0 2024-04-27T04:09:59.310 INFO:tasks.workunit.client.0.smithi162.stdout:3/177: read - d1/d18/d23/f28 zero size 2024-04-27T04:09:59.310 INFO:tasks.workunit.client.0.smithi162.stdout:3/178: chown d1/d18/f11 0 1 2024-04-27T04:09:59.313 INFO:tasks.workunit.client.1.smithi181.stdout:2/449: rmdir d4/d9/d4f/d5f 39 2024-04-27T04:09:59.315 INFO:tasks.workunit.client.0.smithi162.stdout:6/180: symlink d6/db/d32/l40 0 2024-04-27T04:09:59.317 INFO:tasks.workunit.client.0.smithi162.stdout:9/167: rmdir d3/d13/d15 39 2024-04-27T04:09:59.318 INFO:tasks.workunit.client.0.smithi162.stdout:9/168: stat d3/cf 0 2024-04-27T04:09:59.319 INFO:tasks.workunit.client.0.smithi162.stdout:5/168: dread d8/d9/f16 [0,4194304] 0 2024-04-27T04:09:59.320 INFO:tasks.workunit.client.1.smithi181.stdout:8/523: rename d1/d6/d2f/d47/d89 to d1/d6/d2f/d47/dba 0 2024-04-27T04:09:59.320 INFO:tasks.workunit.client.1.smithi181.stdout:8/524: chown d1/f3c 12 1 2024-04-27T04:09:59.321 INFO:tasks.workunit.client.0.smithi162.stdout:6/181: dwrite d6/d1a/f24 [0,4194304] 0 2024-04-27T04:09:59.321 INFO:tasks.workunit.client.1.smithi181.stdout:8/525: fsync d1/d6/d2f/d40/fb9 0 2024-04-27T04:09:59.322 INFO:tasks.workunit.client.0.smithi162.stdout:6/182: write d6/f30 [950370,94661] 0 2024-04-27T04:09:59.324 INFO:tasks.workunit.client.0.smithi162.stdout:6/183: truncate d6/d1a/d34/d3a/f3d 774277 0 2024-04-27T04:09:59.328 INFO:tasks.workunit.client.1.smithi181.stdout:1/501: dread d1/d2/d5/d6/f73 [0,4194304] 0 2024-04-27T04:09:59.330 INFO:tasks.workunit.client.1.smithi181.stdout:5/594: creat d4/d10/d29/d92/fd9 x:0 0 0 2024-04-27T04:09:59.331 INFO:tasks.workunit.client.1.smithi181.stdout:6/364: link d7/d1b/l69 d7/dc/d1c/d68/d6d/l7c 0 2024-04-27T04:09:59.333 INFO:tasks.workunit.client.0.smithi162.stdout:0/223: creat d5/d26/f4d x:0 0 0 2024-04-27T04:09:59.334 INFO:tasks.workunit.client.0.smithi162.stdout:0/224: write d5/dc/f18 [952391,58153] 0 2024-04-27T04:09:59.334 INFO:tasks.workunit.client.1.smithi181.stdout:3/434: mknod d0/d6c/d52/d44/d56/c8c 0 2024-04-27T04:09:59.341 INFO:tasks.workunit.client.1.smithi181.stdout:0/465: dread - dc/d1f/f3f zero size 2024-04-27T04:09:59.342 INFO:tasks.workunit.client.0.smithi162.stdout:2/137: symlink d0/d4/d2a/l2d 0 2024-04-27T04:09:59.345 INFO:tasks.workunit.client.0.smithi162.stdout:2/138: dread d0/f6 [0,4194304] 0 2024-04-27T04:09:59.345 INFO:tasks.workunit.client.1.smithi181.stdout:8/526: creat d1/d6/d6f/fbb x:0 0 0 2024-04-27T04:09:59.347 INFO:tasks.workunit.client.0.smithi162.stdout:5/169: fsync d8/d1a/d29/f2f 0 2024-04-27T04:09:59.349 INFO:tasks.workunit.client.0.smithi162.stdout:8/190: truncate d0/f2 2667325 0 2024-04-27T04:09:59.353 INFO:tasks.workunit.client.1.smithi181.stdout:8/527: dwrite d1/d6/d1a/d25/f99 [0,4194304] 0 2024-04-27T04:09:59.353 INFO:tasks.workunit.client.1.smithi181.stdout:7/416: write d1/d2/d35/d67/f51 [14299,69959] 0 2024-04-27T04:09:59.354 INFO:tasks.workunit.client.1.smithi181.stdout:8/528: fdatasync d1/d6/d2f/d47/f4b 0 2024-04-27T04:09:59.355 INFO:tasks.workunit.client.0.smithi162.stdout:1/158: link d0/d7/f1f d0/d7/d2a/f30 0 2024-04-27T04:09:59.356 INFO:tasks.workunit.client.1.smithi181.stdout:7/417: chown d1/d22/d15/d13/d1b/f4e 13865873 1 2024-04-27T04:09:59.356 INFO:tasks.workunit.client.0.smithi162.stdout:9/169: dwrite d3/f5 [0,4194304] 0 2024-04-27T04:09:59.358 INFO:tasks.workunit.client.0.smithi162.stdout:0/225: creat d5/dc/d1e/d41/f4e x:0 0 0 2024-04-27T04:09:59.359 INFO:tasks.workunit.client.1.smithi181.stdout:4/410: link d0/d15/d25/d40/f4f d0/d15/d25/d2a/d36/f89 0 2024-04-27T04:09:59.362 INFO:tasks.workunit.client.0.smithi162.stdout:7/121: getdents d3/d5/d9/dc 0 2024-04-27T04:09:59.364 INFO:tasks.workunit.client.0.smithi162.stdout:7/122: chown d3/f16 19 1 2024-04-27T04:09:59.364 INFO:tasks.workunit.client.0.smithi162.stdout:0/226: readlink d5/dc/d1e/d46/d48/l4b 0 2024-04-27T04:09:59.365 INFO:tasks.workunit.client.1.smithi181.stdout:4/411: dwrite d0/d15/d25/d2a/f38 [0,4194304] 0 2024-04-27T04:09:59.366 INFO:tasks.workunit.client.1.smithi181.stdout:4/412: readlink d0/l3f 0 2024-04-27T04:09:59.366 INFO:tasks.workunit.client.0.smithi162.stdout:9/170: dwrite d3/d13/d2d/d30/f38 [0,4194304] 0 2024-04-27T04:09:59.367 INFO:tasks.workunit.client.1.smithi181.stdout:4/413: write d0/d15/d25/d2b/f5f [407342,35805] 0 2024-04-27T04:09:59.368 INFO:tasks.workunit.client.1.smithi181.stdout:4/414: stat d0/d15/d25/d2a/d4c 0 2024-04-27T04:09:59.423 INFO:tasks.workunit.client.1.smithi181.stdout:2/450: symlink d4/d9/d4f/l90 0 2024-04-27T04:09:59.423 INFO:tasks.workunit.client.0.smithi162.stdout:4/194: mknod d2/d25/d1c/d39/d3b/c42 0 2024-04-27T04:09:59.425 INFO:tasks.workunit.client.1.smithi181.stdout:3/435: write d0/d6c/d52/d44/f55 [477719,34254] 0 2024-04-27T04:09:59.425 INFO:tasks.workunit.client.0.smithi162.stdout:3/179: symlink d1/l3d 0 2024-04-27T04:09:59.428 INFO:tasks.workunit.client.0.smithi162.stdout:2/139: chown d0/d20/c21 3097862 1 2024-04-27T04:09:59.450 INFO:tasks.workunit.client.0.smithi162.stdout:8/191: rename d0/de/d29 to d0/d10/d2b/d2d/d2f/d39 0 2024-04-27T04:09:59.451 INFO:tasks.workunit.client.1.smithi181.stdout:6/365: fsync d7/dc/d1e/f2a 0 2024-04-27T04:09:59.451 INFO:tasks.workunit.client.0.smithi162.stdout:8/192: read d0/d10/d25/f14 [132966,43718] 0 2024-04-27T04:09:59.452 INFO:tasks.workunit.client.0.smithi162.stdout:6/184: mknod d6/c41 0 2024-04-27T04:09:59.453 INFO:tasks.workunit.client.0.smithi162.stdout:6/185: readlink d6/d1a/l28 0 2024-04-27T04:09:59.453 INFO:tasks.workunit.client.0.smithi162.stdout:6/186: dread - d6/d1a/d34/f3e zero size 2024-04-27T04:09:59.453 INFO:tasks.workunit.client.1.smithi181.stdout:8/529: mkdir d1/d6/d2f/d47/dba/dbc 0 2024-04-27T04:09:59.454 INFO:tasks.workunit.client.1.smithi181.stdout:6/366: dread d7/dc/d1e/f4d [0,4194304] 0 2024-04-27T04:09:59.454 INFO:tasks.workunit.client.0.smithi162.stdout:6/187: write d6/d1a/d34/d3a/f3d [19388,1203] 0 2024-04-27T04:09:59.455 INFO:tasks.workunit.client.1.smithi181.stdout:5/595: truncate d4/de/d2c/d46/f60 1389796 0 2024-04-27T04:09:59.460 INFO:tasks.workunit.client.1.smithi181.stdout:5/596: dread d4/de/d2c/d46/d99/dad/dca/fcd [0,4194304] 0 2024-04-27T04:09:59.460 INFO:tasks.workunit.client.0.smithi162.stdout:6/188: dwrite d6/db/d32/f3b [0,4194304] 0 2024-04-27T04:09:59.461 INFO:tasks.workunit.client.0.smithi162.stdout:6/189: write d6/f35 [4007827,26275] 0 2024-04-27T04:09:59.462 INFO:tasks.workunit.client.1.smithi181.stdout:0/466: link dc/d1f/d3b/d4a/d7a/d9d/fa4 dc/d1f/d6b/fa9 0 2024-04-27T04:09:59.466 INFO:tasks.workunit.client.0.smithi162.stdout:7/123: unlink d3/d5/d9/dc/c20 0 2024-04-27T04:09:59.469 INFO:tasks.workunit.client.0.smithi162.stdout:0/227: creat d5/dc/d1e/d41/f4f x:0 0 0 2024-04-27T04:09:59.470 INFO:tasks.workunit.client.1.smithi181.stdout:4/415: mkdir d0/d15/de/d19/d8a 0 2024-04-27T04:09:59.471 INFO:tasks.workunit.client.0.smithi162.stdout:1/159: dread d0/fb [0,4194304] 0 2024-04-27T04:09:59.471 INFO:tasks.workunit.client.1.smithi181.stdout:7/418: dwrite d1/d22/d15/d29/f4a [0,4194304] 0 2024-04-27T04:09:59.472 INFO:tasks.workunit.client.0.smithi162.stdout:9/171: symlink d3/d13/d2d/l3a 0 2024-04-27T04:09:59.472 INFO:tasks.workunit.client.1.smithi181.stdout:7/419: fdatasync d1/d57/f77 0 2024-04-27T04:09:59.475 INFO:tasks.workunit.client.1.smithi181.stdout:2/451: rename d4/d9/d4f/l7b to d4/d9/d16/l91 0 2024-04-27T04:09:59.476 INFO:tasks.workunit.client.1.smithi181.stdout:1/502: creat d1/d4c/d1e/f9f x:0 0 0 2024-04-27T04:09:59.484 INFO:tasks.workunit.client.1.smithi181.stdout:8/530: read d1/d6/d2f/d30/d5c/f61 [2270009,20534] 0 2024-04-27T04:09:59.485 INFO:tasks.workunit.client.1.smithi181.stdout:6/367: write d7/dc/d1c/f1f [2888722,86037] 0 2024-04-27T04:09:59.487 INFO:tasks.workunit.client.0.smithi162.stdout:8/193: symlink d0/d10/d2b/d2e/l3a 0 2024-04-27T04:09:59.488 INFO:tasks.workunit.client.1.smithi181.stdout:5/597: write d4/de/d2c/d46/d4d/d8e/f8a [349385,10754] 0 2024-04-27T04:09:59.489 INFO:tasks.workunit.client.1.smithi181.stdout:5/598: write d4/f13 [5693245,19327] 0 2024-04-27T04:09:59.490 INFO:tasks.workunit.client.1.smithi181.stdout:0/467: mkdir dc/d1f/daa 0 2024-04-27T04:09:59.491 INFO:tasks.workunit.client.1.smithi181.stdout:5/599: fdatasync d4/d10/fc4 0 2024-04-27T04:09:59.492 INFO:tasks.workunit.client.1.smithi181.stdout:0/468: chown dc/d1f/d3b/d4a/d95/d6d/fa2 31373335 1 2024-04-27T04:09:59.493 INFO:tasks.workunit.client.0.smithi162.stdout:6/190: symlink d6/d1a/d34/d3a/l42 0 2024-04-27T04:09:59.493 INFO:tasks.workunit.client.0.smithi162.stdout:6/191: readlink d6/d1a/d34/l3c 0 2024-04-27T04:09:59.494 INFO:tasks.workunit.client.1.smithi181.stdout:0/469: stat dc/d1f/d3b/d4a/d95/d26/fa0 0 2024-04-27T04:09:59.494 INFO:tasks.workunit.client.0.smithi162.stdout:6/192: write d6/db/f13 [196241,109340] 0 2024-04-27T04:09:59.494 INFO:tasks.workunit.client.1.smithi181.stdout:4/416: chown d0/d15/f2f 32 1 2024-04-27T04:09:59.495 INFO:tasks.workunit.client.0.smithi162.stdout:6/193: write d6/d1a/d34/f3e [1003610,45501] 0 2024-04-27T04:09:59.495 INFO:tasks.workunit.client.0.smithi162.stdout:7/124: creat d3/d5/d9/f24 x:0 0 0 2024-04-27T04:09:59.496 INFO:tasks.workunit.client.1.smithi181.stdout:4/417: read d0/d15/d25/d2b/f2c [830586,79814] 0 2024-04-27T04:09:59.496 INFO:tasks.workunit.client.0.smithi162.stdout:7/125: fsync d3/d5/d9/f17 0 2024-04-27T04:09:59.498 INFO:tasks.workunit.client.0.smithi162.stdout:0/228: mkdir d5/d50 0 2024-04-27T04:09:59.499 INFO:tasks.workunit.client.0.smithi162.stdout:0/229: truncate d5/f3b 491805 0 2024-04-27T04:09:59.502 INFO:tasks.workunit.client.0.smithi162.stdout:9/172: mkdir d3/d13/d2d/d3b 0 2024-04-27T04:09:59.505 INFO:tasks.workunit.client.0.smithi162.stdout:2/140: mkdir d0/d20/d2e 0 2024-04-27T04:09:59.507 INFO:tasks.workunit.client.0.smithi162.stdout:9/173: dwrite d3/fb [0,4194304] 0 2024-04-27T04:09:59.511 INFO:tasks.workunit.client.0.smithi162.stdout:8/194: mknod d0/de/d20/c3b 0 2024-04-27T04:09:59.517 INFO:tasks.workunit.client.0.smithi162.stdout:6/194: rmdir d6 39 2024-04-27T04:09:59.517 INFO:tasks.workunit.client.0.smithi162.stdout:8/195: dwrite d0/d10/d25/f30 [0,4194304] 0 2024-04-27T04:09:59.520 INFO:tasks.workunit.client.1.smithi181.stdout:1/503: dread d1/d4c/d1e/d20/f41 [0,4194304] 0 2024-04-27T04:09:59.521 INFO:tasks.workunit.client.0.smithi162.stdout:7/126: unlink d3/d5/d9/l1b 0 2024-04-27T04:09:59.525 INFO:tasks.workunit.client.1.smithi181.stdout:0/470: read dc/f16 [877134,121715] 0 2024-04-27T04:09:59.525 INFO:tasks.workunit.client.0.smithi162.stdout:0/230: rename d5/d26/l36 to d5/dc/d1e/d41/l51 0 2024-04-27T04:09:59.528 INFO:tasks.workunit.client.1.smithi181.stdout:7/420: dwrite d1/d22/d15/d13/d1b/f40 [0,4194304] 0 2024-04-27T04:09:59.529 INFO:tasks.workunit.client.0.smithi162.stdout:1/160: dwrite d0/d7/f1f [0,4194304] 0 2024-04-27T04:09:59.532 INFO:tasks.workunit.client.0.smithi162.stdout:3/180: link d1/d18/l8 d1/d37/l3e 0 2024-04-27T04:09:59.533 INFO:tasks.workunit.client.1.smithi181.stdout:7/421: dread d1/d2/d1c/d2d/f3e [0,4194304] 0 2024-04-27T04:09:59.535 INFO:tasks.workunit.client.0.smithi162.stdout:5/170: getdents d8 0 2024-04-27T04:09:59.535 INFO:tasks.workunit.client.1.smithi181.stdout:4/418: dread d0/d15/d25/d2a/d36/f51 [0,4194304] 0 2024-04-27T04:09:59.536 INFO:tasks.workunit.client.1.smithi181.stdout:7/422: truncate d1/d22/d15/d29/f4a 4997509 0 2024-04-27T04:09:59.537 INFO:tasks.workunit.client.1.smithi181.stdout:4/419: stat d0/d15/d25/d2b/f5f 0 2024-04-27T04:09:59.539 INFO:tasks.workunit.client.1.smithi181.stdout:2/452: mknod d4/d9/c92 0 2024-04-27T04:09:59.539 INFO:tasks.workunit.client.0.smithi162.stdout:9/174: mknod d3/d23/d1e/c3c 0 2024-04-27T04:09:59.540 INFO:tasks.workunit.client.0.smithi162.stdout:4/195: sync 2024-04-27T04:09:59.543 INFO:tasks.workunit.client.1.smithi181.stdout:6/368: symlink d7/dc/d1c/d34/d3f/l7d 0 2024-04-27T04:09:59.544 INFO:tasks.workunit.client.0.smithi162.stdout:4/196: dread d2/da/f12 [0,4194304] 0 2024-04-27T04:09:59.545 INFO:tasks.workunit.client.0.smithi162.stdout:4/197: write d2/d25/fb [868095,37098] 0 2024-04-27T04:09:59.545 INFO:tasks.workunit.client.0.smithi162.stdout:9/175: dread d3/fc [0,4194304] 0 2024-04-27T04:09:59.545 INFO:tasks.workunit.client.1.smithi181.stdout:8/531: dwrite d1/f33 [0,4194304] 0 2024-04-27T04:09:59.546 INFO:tasks.workunit.client.0.smithi162.stdout:4/198: fsync d2/da/d2c/f35 0 2024-04-27T04:09:59.546 INFO:tasks.workunit.client.0.smithi162.stdout:9/176: write d3/d13/d2d/f37 [1000331,47080] 0 2024-04-27T04:09:59.548 INFO:tasks.workunit.client.0.smithi162.stdout:8/196: mkdir d0/d10/d25/d3c 0 2024-04-27T04:09:59.549 INFO:tasks.workunit.client.0.smithi162.stdout:8/197: fdatasync d0/d10/d2b/f19 0 2024-04-27T04:09:59.550 INFO:tasks.workunit.client.0.smithi162.stdout:8/198: readlink d0/de/d20/l27 0 2024-04-27T04:09:59.551 INFO:tasks.workunit.client.1.smithi181.stdout:5/600: dwrite d4/de/d2c/d46/d4d/d8e/f41 [4194304,4194304] 0 2024-04-27T04:09:59.552 INFO:tasks.workunit.client.1.smithi181.stdout:0/471: read - dc/d1f/d3b/d4a/d95/f51 zero size 2024-04-27T04:09:59.554 INFO:tasks.workunit.client.0.smithi162.stdout:4/199: dwrite d2/da/dd/f2a [0,4194304] 0 2024-04-27T04:09:59.554 INFO:tasks.workunit.client.0.smithi162.stdout:1/161: creat d0/d7/d2a/f31 x:0 0 0 2024-04-27T04:09:59.555 INFO:tasks.workunit.client.1.smithi181.stdout:7/423: mkdir d1/df/d86 0 2024-04-27T04:09:59.556 INFO:tasks.workunit.client.0.smithi162.stdout:4/200: chown d2/d25/d1c/d39 108106 1 2024-04-27T04:09:59.556 INFO:tasks.workunit.client.0.smithi162.stdout:1/162: chown d0/d11/d1a/l29 1016 1 2024-04-27T04:09:59.559 INFO:tasks.workunit.client.0.smithi162.stdout:5/171: mknod d8/d9/c3a 0 2024-04-27T04:09:59.562 INFO:tasks.workunit.client.1.smithi181.stdout:6/369: unlink d7/dc/d1c/f1f 0 2024-04-27T04:09:59.563 INFO:tasks.workunit.client.1.smithi181.stdout:4/420: dread d0/f30 [0,4194304] 0 2024-04-27T04:09:59.570 INFO:tasks.workunit.client.1.smithi181.stdout:5/601: symlink d4/d10/d72/db3/lda 0 2024-04-27T04:09:59.572 INFO:tasks.workunit.client.1.smithi181.stdout:5/602: truncate d4/d10/d29/f84 4656253 0 2024-04-27T04:09:59.573 INFO:tasks.workunit.client.1.smithi181.stdout:5/603: stat d4/de/d2c/d46/d4d/d8e/d67 0 2024-04-27T04:09:59.574 INFO:tasks.workunit.client.0.smithi162.stdout:6/195: mkdir d6/d1a/d34/d43 0 2024-04-27T04:09:59.577 INFO:tasks.workunit.client.1.smithi181.stdout:7/424: mknod d1/d2/c87 0 2024-04-27T04:09:59.577 INFO:tasks.workunit.client.1.smithi181.stdout:7/425: chown d1/d2/d1c/d2d/d61/d80 10 1 2024-04-27T04:09:59.579 INFO:tasks.workunit.client.0.smithi162.stdout:2/141: truncate d0/f7 2365111 0 2024-04-27T04:09:59.580 INFO:tasks.workunit.client.0.smithi162.stdout:7/127: mknod d3/c25 0 2024-04-27T04:09:59.585 INFO:tasks.workunit.client.0.smithi162.stdout:2/142: dread d0/d4/f16 [0,4194304] 0 2024-04-27T04:09:59.585 INFO:tasks.workunit.client.0.smithi162.stdout:8/199: symlink d0/d10/d2b/d2d/d2f/l3d 0 2024-04-27T04:09:59.586 INFO:tasks.workunit.client.1.smithi181.stdout:8/532: dwrite d1/d8c/f90 [0,4194304] 0 2024-04-27T04:09:59.588 INFO:tasks.workunit.client.1.smithi181.stdout:6/370: fsync d7/dc/d1c/d34/d3f/f74 0 2024-04-27T04:09:59.590 INFO:tasks.workunit.client.0.smithi162.stdout:1/163: rmdir d0/d11/d1a 39 2024-04-27T04:09:59.592 INFO:tasks.workunit.client.1.smithi181.stdout:1/504: creat d1/d4c/d1e/d20/d53/fa0 x:0 0 0 2024-04-27T04:09:59.595 INFO:tasks.workunit.client.0.smithi162.stdout:4/201: creat d2/d25/d1c/d24/f43 x:0 0 0 2024-04-27T04:09:59.596 INFO:tasks.workunit.client.0.smithi162.stdout:4/202: write d2/d25/f36 [667255,44359] 0 2024-04-27T04:09:59.596 INFO:tasks.workunit.client.1.smithi181.stdout:2/453: truncate d4/f6 1284419 0 2024-04-27T04:09:59.597 INFO:tasks.workunit.client.1.smithi181.stdout:2/454: stat d4/d9/d35/c4b 0 2024-04-27T04:09:59.598 INFO:tasks.workunit.client.0.smithi162.stdout:5/172: symlink d8/d2c/l3b 0 2024-04-27T04:09:59.601 INFO:tasks.workunit.client.0.smithi162.stdout:4/203: dwrite d2/f22 [0,4194304] 0 2024-04-27T04:09:59.610 INFO:tasks.workunit.client.1.smithi181.stdout:8/533: creat d1/d6/d2f/d72/fbd x:0 0 0 2024-04-27T04:09:59.612 INFO:tasks.workunit.client.0.smithi162.stdout:7/128: creat d3/d5/d9/f26 x:0 0 0 2024-04-27T04:09:59.612 INFO:tasks.workunit.client.1.smithi181.stdout:8/534: truncate d1/d6/d2f/f3a 1448906 0 2024-04-27T04:09:59.613 INFO:tasks.workunit.client.0.smithi162.stdout:7/129: truncate d3/d5/f12 1090990 0 2024-04-27T04:09:59.614 INFO:tasks.workunit.client.0.smithi162.stdout:0/231: rmdir d5/d50 0 2024-04-27T04:09:59.614 INFO:tasks.workunit.client.1.smithi181.stdout:7/426: write d1/d22/d15/d1f/d34/f3c [2574867,126995] 0 2024-04-27T04:09:59.615 INFO:tasks.workunit.client.0.smithi162.stdout:4/204: dread d2/f9 [0,4194304] 0 2024-04-27T04:09:59.616 INFO:tasks.workunit.client.0.smithi162.stdout:4/205: write d2/f9 [341470,121332] 0 2024-04-27T04:09:59.617 INFO:tasks.workunit.client.0.smithi162.stdout:4/206: chown d2/da 5 1 2024-04-27T04:09:59.617 INFO:tasks.workunit.client.0.smithi162.stdout:2/143: dread d0/f1 [0,4194304] 0 2024-04-27T04:09:59.618 INFO:tasks.workunit.client.1.smithi181.stdout:4/421: link d0/d15/d5c/d69/f31 d0/d15/d25/d2b/d6d/d82/f8b 0 2024-04-27T04:09:59.618 INFO:tasks.workunit.client.0.smithi162.stdout:4/207: read - d2/d25/f2d zero size 2024-04-27T04:09:59.618 INFO:tasks.workunit.client.0.smithi162.stdout:2/144: chown d0/d4/d17/c1a 40490478 1 2024-04-27T04:09:59.623 INFO:tasks.workunit.client.0.smithi162.stdout:3/181: creat d1/f3f x:0 0 0 2024-04-27T04:09:59.624 INFO:tasks.workunit.client.0.smithi162.stdout:3/182: chown d1/d18/d23/d34 10702320 1 2024-04-27T04:09:59.629 INFO:tasks.workunit.client.0.smithi162.stdout:3/183: dread d1/d21/f2c [0,4194304] 0 2024-04-27T04:09:59.630 INFO:tasks.workunit.client.1.smithi181.stdout:8/535: mknod d1/d6/d1a/d25/cbe 0 2024-04-27T04:09:59.630 INFO:tasks.workunit.client.0.smithi162.stdout:3/184: chown d1/d18/d23/d34 326071 1 2024-04-27T04:09:59.630 INFO:tasks.workunit.client.0.smithi162.stdout:3/185: write d1/d18/f11 [768001,104590] 0 2024-04-27T04:09:59.631 INFO:tasks.workunit.client.0.smithi162.stdout:5/173: symlink d8/d2c/d2e/l3c 0 2024-04-27T04:09:59.632 INFO:tasks.workunit.client.0.smithi162.stdout:6/196: creat d6/d1a/d34/d43/f44 x:0 0 0 2024-04-27T04:09:59.632 INFO:tasks.workunit.client.0.smithi162.stdout:3/186: dwrite d1/d21/f39 [0,4194304] 0 2024-04-27T04:09:59.634 INFO:tasks.workunit.client.1.smithi181.stdout:2/455: dwrite d4/f59 [4194304,4194304] 0 2024-04-27T04:09:59.637 INFO:tasks.workunit.client.0.smithi162.stdout:9/177: link d3/d23/d1e/c3c d3/d23/c3d 0 2024-04-27T04:09:59.644 INFO:tasks.workunit.client.0.smithi162.stdout:7/130: creat d3/d5/d9/dc/f27 x:0 0 0 2024-04-27T04:09:59.645 INFO:tasks.workunit.client.1.smithi181.stdout:5/604: creat d4/d10/fdb x:0 0 0 2024-04-27T04:09:59.646 INFO:tasks.workunit.client.1.smithi181.stdout:8/536: creat d1/d6/d2f/d47/dba/fbf x:0 0 0 2024-04-27T04:09:59.648 INFO:tasks.workunit.client.1.smithi181.stdout:7/427: mknod d1/d57/d72/c88 0 2024-04-27T04:09:59.649 INFO:tasks.workunit.client.1.smithi181.stdout:6/371: mkdir d7/dc/d1c/d34/d3f/d7e 0 2024-04-27T04:09:59.650 INFO:tasks.workunit.client.0.smithi162.stdout:8/200: mknod d0/c3e 0 2024-04-27T04:09:59.651 INFO:tasks.workunit.client.1.smithi181.stdout:2/456: unlink d4/f2c 0 2024-04-27T04:09:59.651 INFO:tasks.workunit.client.0.smithi162.stdout:8/201: dread d0/d10/d25/f1d [0,4194304] 0 2024-04-27T04:09:59.652 INFO:tasks.workunit.client.1.smithi181.stdout:2/457: write d4/d9/f11 [785738,88781] 0 2024-04-27T04:09:59.653 INFO:tasks.workunit.client.1.smithi181.stdout:2/458: chown d4/d9/df/c58 121960721 1 2024-04-27T04:09:59.654 INFO:tasks.workunit.client.0.smithi162.stdout:1/164: link d0/d7/f1f d0/d7/f32 0 2024-04-27T04:09:59.657 INFO:tasks.workunit.client.1.smithi181.stdout:1/505: getdents d1/d4c/d1e/d48 0 2024-04-27T04:09:59.658 INFO:tasks.workunit.client.0.smithi162.stdout:5/174: fdatasync d8/f12 0 2024-04-27T04:09:59.659 INFO:tasks.workunit.client.0.smithi162.stdout:8/202: dread d0/d10/d2b/d2d/d2f/d39/f33 [0,4194304] 0 2024-04-27T04:09:59.660 INFO:tasks.workunit.client.1.smithi181.stdout:7/428: unlink d1/d22/d15/d13/l79 0 2024-04-27T04:09:59.665 INFO:tasks.workunit.client.0.smithi162.stdout:9/178: creat d3/d23/f3e x:0 0 0 2024-04-27T04:09:59.666 INFO:tasks.workunit.client.0.smithi162.stdout:9/179: rename d3/d23/d1e to d3/d23/d1e/d3f 22 2024-04-27T04:09:59.669 INFO:tasks.workunit.client.1.smithi181.stdout:1/506: fdatasync d1/f8e 0 2024-04-27T04:09:59.670 INFO:tasks.workunit.client.0.smithi162.stdout:0/232: write d5/dc/fe [3187744,74040] 0 2024-04-27T04:09:59.670 INFO:tasks.workunit.client.1.smithi181.stdout:1/507: dread - d1/d4c/d1e/d20/d53/fa0 zero size 2024-04-27T04:09:59.672 INFO:tasks.workunit.client.1.smithi181.stdout:1/508: truncate d1/d2/d64/f8d 401624 0 2024-04-27T04:09:59.673 INFO:tasks.workunit.client.1.smithi181.stdout:7/429: mkdir d1/d22/d15/d13/d1b/d89 0 2024-04-27T04:09:59.676 INFO:tasks.workunit.client.1.smithi181.stdout:2/459: dread d4/f6 [0,4194304] 0 2024-04-27T04:09:59.678 INFO:tasks.workunit.client.1.smithi181.stdout:4/422: truncate d0/d15/d5c/d69/f10 2965507 0 2024-04-27T04:09:59.680 INFO:tasks.workunit.client.1.smithi181.stdout:7/430: dwrite d1/d2/d1c/d2d/d61/f68 [0,4194304] 0 2024-04-27T04:09:59.681 INFO:tasks.workunit.client.0.smithi162.stdout:3/187: fsync d1/f20 0 2024-04-27T04:09:59.682 INFO:tasks.workunit.client.1.smithi181.stdout:4/423: read d0/d15/de/f34 [4036034,657] 0 2024-04-27T04:09:59.682 INFO:tasks.workunit.client.0.smithi162.stdout:3/188: chown d1/d18/f13 4367592 1 2024-04-27T04:09:59.682 INFO:tasks.workunit.client.1.smithi181.stdout:5/605: write d4/de/d2f/f5d [1061089,125300] 0 2024-04-27T04:09:59.684 INFO:tasks.workunit.client.0.smithi162.stdout:5/175: unlink d8/d2c/d2e/l3c 0 2024-04-27T04:09:59.684 INFO:tasks.workunit.client.0.smithi162.stdout:5/176: chown c6 10741 1 2024-04-27T04:09:59.687 INFO:tasks.workunit.client.1.smithi181.stdout:8/537: getdents d1/d6/d2f/d47 0 2024-04-27T04:09:59.691 INFO:tasks.workunit.client.1.smithi181.stdout:5/606: dwrite d4/da3/fbd [0,4194304] 0 2024-04-27T04:09:59.691 INFO:tasks.workunit.client.1.smithi181.stdout:7/431: rmdir d1/d2/d1c/d2d 39 2024-04-27T04:09:59.691 INFO:tasks.workunit.client.1.smithi181.stdout:7/432: fdatasync d1/d2/f19 0 2024-04-27T04:09:59.695 INFO:tasks.workunit.client.0.smithi162.stdout:9/180: symlink d3/d13/d2d/d30/l40 0 2024-04-27T04:09:59.695 INFO:tasks.workunit.client.0.smithi162.stdout:9/181: stat d3/fc 0 2024-04-27T04:09:59.696 INFO:tasks.workunit.client.1.smithi181.stdout:4/424: rmdir d0/d15/d25/d2b 39 2024-04-27T04:09:59.700 INFO:tasks.workunit.client.1.smithi181.stdout:8/538: mkdir d1/d6/d2f/d30/d66/dc0 0 2024-04-27T04:09:59.702 INFO:tasks.workunit.client.0.smithi162.stdout:0/233: fsync f1 0 2024-04-27T04:09:59.702 INFO:tasks.workunit.client.0.smithi162.stdout:1/165: dwrite d0/f1b [0,4194304] 0 2024-04-27T04:09:59.704 INFO:tasks.workunit.client.1.smithi181.stdout:7/433: dread d1/d2/d35/d67/f51 [0,4194304] 0 2024-04-27T04:09:59.705 INFO:tasks.workunit.client.0.smithi162.stdout:6/197: truncate d6/f26 3057924 0 2024-04-27T04:09:59.709 INFO:tasks.workunit.client.1.smithi181.stdout:6/372: getdents d7/dc/d1c/d68 0 2024-04-27T04:09:59.710 INFO:tasks.workunit.client.0.smithi162.stdout:3/189: symlink d1/d18/d23/d2f/l40 0 2024-04-27T04:09:59.711 INFO:tasks.workunit.client.0.smithi162.stdout:2/145: dwrite d0/f7 [0,4194304] 0 2024-04-27T04:09:59.712 INFO:tasks.workunit.client.0.smithi162.stdout:5/177: creat d8/d9/f3d x:0 0 0 2024-04-27T04:09:59.714 INFO:tasks.workunit.client.1.smithi181.stdout:7/434: creat d1/d22/d15/d13/d71/f8a x:0 0 0 2024-04-27T04:09:59.719 INFO:tasks.workunit.client.1.smithi181.stdout:8/539: symlink d1/d6/d1a/lc1 0 2024-04-27T04:09:59.720 INFO:tasks.workunit.client.0.smithi162.stdout:4/208: getdents d2/d25/d1c/d24 0 2024-04-27T04:09:59.721 INFO:tasks.workunit.client.1.smithi181.stdout:7/435: mkdir d1/d2/d35/d8b 0 2024-04-27T04:09:59.722 INFO:tasks.workunit.client.1.smithi181.stdout:7/436: fdatasync d1/d22/d15/d13/d1b/f4e 0 2024-04-27T04:09:59.725 INFO:tasks.workunit.client.1.smithi181.stdout:8/540: dread - d1/f82 zero size 2024-04-27T04:09:59.726 INFO:tasks.workunit.client.0.smithi162.stdout:1/166: rmdir d0/d7 39 2024-04-27T04:09:59.729 INFO:tasks.workunit.client.1.smithi181.stdout:7/437: dwrite d1/d2/d35/d67/f69 [0,4194304] 0 2024-04-27T04:09:59.730 INFO:tasks.workunit.client.1.smithi181.stdout:6/373: rename d7/d1b to d7/dc/d1c/d7f 0 2024-04-27T04:09:59.730 INFO:tasks.workunit.client.1.smithi181.stdout:7/438: chown d1/d2/f24 43 1 2024-04-27T04:09:59.731 INFO:tasks.workunit.client.1.smithi181.stdout:7/439: chown d1/d22/d15 4 1 2024-04-27T04:09:59.732 INFO:tasks.workunit.client.0.smithi162.stdout:6/198: creat d6/db/d32/f45 x:0 0 0 2024-04-27T04:09:59.735 INFO:tasks.workunit.client.1.smithi181.stdout:8/541: rename d1/d20/l36 to d1/d6/d2f/d47/dba/dbc/lc2 0 2024-04-27T04:09:59.736 INFO:tasks.workunit.client.0.smithi162.stdout:3/190: rmdir d1/d18/d23/d2f 39 2024-04-27T04:09:59.739 INFO:tasks.workunit.client.0.smithi162.stdout:2/146: creat d0/d4/d17/f2f x:0 0 0 2024-04-27T04:09:59.740 INFO:tasks.workunit.client.0.smithi162.stdout:6/199: dwrite d6/f35 [4194304,4194304] 0 2024-04-27T04:09:59.741 INFO:tasks.workunit.client.0.smithi162.stdout:2/147: write d0/d4/f15 [6086055,4663] 0 2024-04-27T04:09:59.741 INFO:tasks.workunit.client.0.smithi162.stdout:5/178: rename d8/d1a/d29/f2f to d8/d1a/f3e 0 2024-04-27T04:09:59.745 INFO:tasks.workunit.client.0.smithi162.stdout:7/131: getdents d3/d5/d9/dc/de 0 2024-04-27T04:09:59.747 INFO:tasks.workunit.client.1.smithi181.stdout:8/542: symlink d1/d6/d6f/d91/lc3 0 2024-04-27T04:09:59.747 INFO:tasks.workunit.client.0.smithi162.stdout:4/209: rmdir d2/da/d2c 39 2024-04-27T04:09:59.747 INFO:tasks.workunit.client.0.smithi162.stdout:4/210: write d2/f22 [304421,21265] 0 2024-04-27T04:09:59.748 INFO:tasks.workunit.client.0.smithi162.stdout:1/167: chown d0/fe 792980 1 2024-04-27T04:09:59.748 INFO:tasks.workunit.client.1.smithi181.stdout:8/543: chown d1/d6/d2f/d30/d66 89670743 1 2024-04-27T04:09:59.750 INFO:tasks.workunit.client.0.smithi162.stdout:3/191: creat d1/d18/f41 x:0 0 0 2024-04-27T04:09:59.752 INFO:tasks.workunit.client.1.smithi181.stdout:7/440: dwrite d1/d2/d1c/d2d/f41 [0,4194304] 0 2024-04-27T04:09:59.754 INFO:tasks.workunit.client.0.smithi162.stdout:6/200: mkdir d6/d1a/d34/d46 0 2024-04-27T04:09:59.754 INFO:tasks.workunit.client.0.smithi162.stdout:4/211: dwrite d2/f11 [0,4194304] 0 2024-04-27T04:09:59.755 INFO:tasks.workunit.client.0.smithi162.stdout:6/201: stat d6/d1a/l28 0 2024-04-27T04:09:59.756 INFO:tasks.workunit.client.0.smithi162.stdout:2/148: unlink d0/d4/c10 0 2024-04-27T04:09:59.760 INFO:tasks.workunit.client.0.smithi162.stdout:1/168: dread d0/d11/f1c [0,4194304] 0 2024-04-27T04:09:59.761 INFO:tasks.workunit.client.1.smithi181.stdout:7/441: creat d1/d22/d15/d13/d1b/f8c x:0 0 0 2024-04-27T04:09:59.762 INFO:tasks.workunit.client.1.smithi181.stdout:7/442: write d1/d22/f65 [1060033,37199] 0 2024-04-27T04:09:59.763 INFO:tasks.workunit.client.0.smithi162.stdout:2/149: dread d0/f1 [0,4194304] 0 2024-04-27T04:09:59.763 INFO:tasks.workunit.client.0.smithi162.stdout:9/182: creat d3/d13/f41 x:0 0 0 2024-04-27T04:09:59.764 INFO:tasks.workunit.client.1.smithi181.stdout:7/443: chown d1/df/d1a/l53 14526 1 2024-04-27T04:09:59.764 INFO:tasks.workunit.client.1.smithi181.stdout:7/444: chown d1/d2 16248593 1 2024-04-27T04:09:59.766 INFO:tasks.workunit.client.0.smithi162.stdout:2/150: chown d0/d4/f9 861 1 2024-04-27T04:09:59.766 INFO:tasks.workunit.client.0.smithi162.stdout:4/212: dwrite d2/da/dd/f10 [0,4194304] 0 2024-04-27T04:09:59.766 INFO:tasks.workunit.client.0.smithi162.stdout:6/202: dwrite d6/f35 [4194304,4194304] 0 2024-04-27T04:09:59.774 INFO:tasks.workunit.client.0.smithi162.stdout:6/203: dwrite d6/f2d [0,4194304] 0 2024-04-27T04:09:59.775 INFO:tasks.workunit.client.0.smithi162.stdout:1/169: truncate d0/d7/d2a/f31 975233 0 2024-04-27T04:09:59.782 INFO:tasks.workunit.client.0.smithi162.stdout:1/170: dwrite d0/d7/d2a/f2d [0,4194304] 0 2024-04-27T04:09:59.788 INFO:tasks.workunit.client.0.smithi162.stdout:3/192: creat d1/d18/d23/d2f/f42 x:0 0 0 2024-04-27T04:09:59.789 INFO:tasks.workunit.client.1.smithi181.stdout:8/544: dread d1/d6/d1a/f6a [0,4194304] 0 2024-04-27T04:09:59.795 INFO:tasks.workunit.client.0.smithi162.stdout:8/203: sync 2024-04-27T04:09:59.798 INFO:tasks.workunit.client.0.smithi162.stdout:1/171: symlink d0/d7/d10/l33 0 2024-04-27T04:09:59.799 INFO:tasks.workunit.client.0.smithi162.stdout:9/183: truncate d3/f4 3602773 0 2024-04-27T04:09:59.800 INFO:tasks.workunit.client.0.smithi162.stdout:1/172: dread - d0/d7/f27 zero size 2024-04-27T04:09:59.801 INFO:tasks.workunit.client.0.smithi162.stdout:8/204: dwrite d0/f7 [0,4194304] 0 2024-04-27T04:09:59.802 INFO:tasks.workunit.client.1.smithi181.stdout:7/445: dwrite d1/d2/d35/d67/f51 [0,4194304] 0 2024-04-27T04:09:59.803 INFO:tasks.workunit.client.0.smithi162.stdout:3/193: mknod d1/d21/c43 0 2024-04-27T04:09:59.804 INFO:tasks.workunit.client.0.smithi162.stdout:3/194: write d1/f3f [28787,76951] 0 2024-04-27T04:09:59.806 INFO:tasks.workunit.client.0.smithi162.stdout:5/179: getdents d8/d1a/d23 0 2024-04-27T04:09:59.807 INFO:tasks.workunit.client.0.smithi162.stdout:8/205: dread d0/d10/d25/f36 [0,4194304] 0 2024-04-27T04:09:59.809 INFO:tasks.workunit.client.1.smithi181.stdout:7/446: creat d1/d2/d1c/f8d x:0 0 0 2024-04-27T04:09:59.810 INFO:tasks.workunit.client.1.smithi181.stdout:7/447: chown d1/d2/l6 2409 1 2024-04-27T04:09:59.811 INFO:tasks.workunit.client.0.smithi162.stdout:9/184: dread d3/fc [0,4194304] 0 2024-04-27T04:09:59.813 INFO:tasks.workunit.client.0.smithi162.stdout:8/206: stat d0/d10/d2b/f19 0 2024-04-27T04:09:59.813 INFO:tasks.workunit.client.0.smithi162.stdout:4/213: creat d2/da/f44 x:0 0 0 2024-04-27T04:09:59.813 INFO:tasks.workunit.client.0.smithi162.stdout:8/207: readlink d0/l9 0 2024-04-27T04:09:59.814 INFO:tasks.workunit.client.0.smithi162.stdout:5/180: mkdir d8/d2c/d2e/d3f 0 2024-04-27T04:09:59.815 INFO:tasks.workunit.client.0.smithi162.stdout:2/151: getdents d0/d20 0 2024-04-27T04:09:59.816 INFO:tasks.workunit.client.0.smithi162.stdout:5/181: chown d8/d9 41509 1 2024-04-27T04:09:59.816 INFO:tasks.workunit.client.0.smithi162.stdout:3/195: dwrite d1/d18/f11 [0,4194304] 0 2024-04-27T04:09:59.817 INFO:tasks.workunit.client.0.smithi162.stdout:7/132: truncate d3/d5/f12 892636 0 2024-04-27T04:09:59.820 INFO:tasks.workunit.client.1.smithi181.stdout:7/448: getdents d1/df 0 2024-04-27T04:09:59.821 INFO:tasks.workunit.client.0.smithi162.stdout:7/133: read - d3/d5/d9/f21 zero size 2024-04-27T04:09:59.821 INFO:tasks.workunit.client.0.smithi162.stdout:4/214: write d2/da/d2c/f35 [468037,29593] 0 2024-04-27T04:09:59.825 INFO:tasks.workunit.client.0.smithi162.stdout:2/152: dwrite d0/fe [0,4194304] 0 2024-04-27T04:09:59.826 INFO:tasks.workunit.client.0.smithi162.stdout:5/182: dwrite d8/d9/f3d [0,4194304] 0 2024-04-27T04:09:59.828 INFO:tasks.workunit.client.1.smithi181.stdout:7/449: dwrite d1/f45 [0,4194304] 0 2024-04-27T04:09:59.828 INFO:tasks.workunit.client.0.smithi162.stdout:0/234: sync 2024-04-27T04:09:59.829 INFO:tasks.workunit.client.0.smithi162.stdout:8/208: symlink d0/d10/d2b/d2d/l3f 0 2024-04-27T04:09:59.829 INFO:tasks.workunit.client.0.smithi162.stdout:0/235: write d5/f3b [1383980,35197] 0 2024-04-27T04:09:59.834 INFO:tasks.workunit.client.1.smithi181.stdout:8/545: write d1/d6/d1a/d25/f5f [6348207,118165] 0 2024-04-27T04:09:59.834 INFO:tasks.workunit.client.1.smithi181.stdout:8/546: chown d1/d6/d2f/d40/f4c 14847 1 2024-04-27T04:09:59.835 INFO:tasks.workunit.client.0.smithi162.stdout:3/196: read d1/d18/f2d [814073,82284] 0 2024-04-27T04:09:59.835 INFO:tasks.workunit.client.0.smithi162.stdout:3/197: chown d1/d18/d23/l31 7141 1 2024-04-27T04:09:59.835 INFO:tasks.workunit.client.0.smithi162.stdout:7/134: creat d3/d5/d9/f28 x:0 0 0 2024-04-27T04:09:59.839 INFO:tasks.workunit.client.0.smithi162.stdout:3/198: dread - d1/d18/d23/f28 zero size 2024-04-27T04:09:59.841 INFO:tasks.workunit.client.0.smithi162.stdout:6/204: truncate f5 2710529 0 2024-04-27T04:09:59.841 INFO:tasks.workunit.client.1.smithi181.stdout:8/547: symlink d1/d6/d2f/d30/d9d/lc4 0 2024-04-27T04:09:59.842 INFO:tasks.workunit.client.0.smithi162.stdout:3/199: write d1/f3f [847816,69151] 0 2024-04-27T04:09:59.844 INFO:tasks.workunit.client.1.smithi181.stdout:7/450: symlink d1/d2/d1c/d2d/l8e 0 2024-04-27T04:09:59.844 INFO:tasks.workunit.client.0.smithi162.stdout:3/200: write d1/d21/f39 [2116276,88939] 0 2024-04-27T04:09:59.844 INFO:tasks.workunit.client.0.smithi162.stdout:1/173: write d0/d11/f20 [618799,79825] 0 2024-04-27T04:09:59.846 INFO:tasks.workunit.client.1.smithi181.stdout:7/451: truncate d1/d2/d35/f85 86037 0 2024-04-27T04:09:59.846 INFO:tasks.workunit.client.0.smithi162.stdout:1/174: truncate d0/d11/f2c 5093748 0 2024-04-27T04:09:59.846 INFO:tasks.workunit.client.1.smithi181.stdout:7/452: read d1/d2/d1c/f2c [3741514,20170] 0 2024-04-27T04:09:59.848 INFO:tasks.workunit.client.1.smithi181.stdout:8/548: dread d1/d6/d2f/f45 [0,4194304] 0 2024-04-27T04:09:59.848 INFO:tasks.workunit.client.0.smithi162.stdout:8/209: creat d0/d10/d2b/d2e/f40 x:0 0 0 2024-04-27T04:09:59.849 INFO:tasks.workunit.client.0.smithi162.stdout:0/236: creat d5/dc/d1e/d46/f52 x:0 0 0 2024-04-27T04:09:59.850 INFO:tasks.workunit.client.1.smithi181.stdout:8/549: write d1/d6/d2f/d40/d2a/f64 [342384,81225] 0 2024-04-27T04:09:59.851 INFO:tasks.workunit.client.0.smithi162.stdout:9/185: link d3/d23/c33 d3/d13/d2d/d30/c42 0 2024-04-27T04:09:59.853 INFO:tasks.workunit.client.0.smithi162.stdout:5/183: rename d8/d2c/d2e to d8/d1a/d23/d40 0 2024-04-27T04:09:59.854 INFO:tasks.workunit.client.1.smithi181.stdout:8/550: rmdir d1/d6/d6f 39 2024-04-27T04:09:59.855 INFO:tasks.workunit.client.1.smithi181.stdout:8/551: chown d1/d6/d2f/d30/d9d 216565 1 2024-04-27T04:09:59.862 INFO:tasks.workunit.client.0.smithi162.stdout:2/153: symlink d0/d20/d2e/l30 0 2024-04-27T04:09:59.863 INFO:tasks.workunit.client.1.smithi181.stdout:8/552: dwrite d1/d6/d2f/d30/d66/f81 [0,4194304] 0 2024-04-27T04:09:59.864 INFO:tasks.workunit.client.0.smithi162.stdout:3/201: symlink d1/d18/d23/d2f/l44 0 2024-04-27T04:09:59.866 INFO:tasks.workunit.client.0.smithi162.stdout:2/154: dread d0/d4/f15 [4194304,4194304] 0 2024-04-27T04:09:59.871 INFO:tasks.workunit.client.1.smithi181.stdout:7/453: dread d1/d22/d15/f2a [0,4194304] 0 2024-04-27T04:09:59.891 INFO:tasks.workunit.client.0.smithi162.stdout:1/175: readlink d0/d11/d1a/l29 0 2024-04-27T04:09:59.892 INFO:tasks.workunit.client.0.smithi162.stdout:1/176: truncate d0/f2b 852319 0 2024-04-27T04:09:59.899 INFO:tasks.workunit.client.0.smithi162.stdout:1/177: dwrite d0/d11/f2e [0,4194304] 0 2024-04-27T04:09:59.918 INFO:tasks.workunit.client.0.smithi162.stdout:8/210: dread d0/d10/d25/f1d [0,4194304] 0 2024-04-27T04:09:59.919 INFO:tasks.workunit.client.0.smithi162.stdout:0/237: creat d5/dc/d1e/d41/f53 x:0 0 0 2024-04-27T04:09:59.921 INFO:tasks.workunit.client.0.smithi162.stdout:9/186: mkdir d3/d23/d1e/d43 0 2024-04-27T04:09:59.922 INFO:tasks.workunit.client.0.smithi162.stdout:9/187: chown d3/d13/d2d 8 1 2024-04-27T04:09:59.923 INFO:tasks.workunit.client.0.smithi162.stdout:4/215: sync 2024-04-27T04:09:59.924 INFO:tasks.workunit.client.0.smithi162.stdout:4/216: readlink d2/da/dd/l14 0 2024-04-27T04:09:59.924 INFO:tasks.workunit.client.0.smithi162.stdout:4/217: chown d2/da/dd/f2a 5337 1 2024-04-27T04:09:59.925 INFO:tasks.workunit.client.0.smithi162.stdout:5/184: mkdir d8/d2c/d41 0 2024-04-27T04:09:59.930 INFO:tasks.workunit.client.0.smithi162.stdout:4/218: dwrite d2/f38 [0,4194304] 0 2024-04-27T04:09:59.930 INFO:tasks.workunit.client.0.smithi162.stdout:2/155: mkdir d0/d4/d17/d31 0 2024-04-27T04:09:59.932 INFO:tasks.workunit.client.0.smithi162.stdout:8/211: dread d0/d10/d25/f36 [0,4194304] 0 2024-04-27T04:09:59.935 INFO:tasks.workunit.client.0.smithi162.stdout:9/188: fdatasync d3/d13/d15/f34 0 2024-04-27T04:09:59.936 INFO:tasks.workunit.client.0.smithi162.stdout:5/185: creat d8/d1a/d23/d40/f42 x:0 0 0 2024-04-27T04:09:59.943 INFO:tasks.workunit.client.0.smithi162.stdout:3/202: mkdir d1/d45 0 2024-04-27T04:09:59.943 INFO:tasks.workunit.client.0.smithi162.stdout:3/203: readlink d1/d18/d23/l29 0 2024-04-27T04:09:59.944 INFO:tasks.workunit.client.0.smithi162.stdout:3/204: fdatasync d1/d18/f41 0 2024-04-27T04:09:59.945 INFO:tasks.workunit.client.0.smithi162.stdout:4/219: dread d2/da/f12 [0,4194304] 0 2024-04-27T04:09:59.946 INFO:tasks.workunit.client.0.smithi162.stdout:2/156: read d0/d4/f15 [2452895,129402] 0 2024-04-27T04:09:59.947 INFO:tasks.workunit.client.0.smithi162.stdout:2/157: write d0/d4/f22 [2227908,41767] 0 2024-04-27T04:09:59.952 INFO:tasks.workunit.client.0.smithi162.stdout:6/205: getdents d6/d1a/d34/d3a 0 2024-04-27T04:09:59.957 INFO:tasks.workunit.client.0.smithi162.stdout:4/220: creat d2/d25/d1c/d39/f45 x:0 0 0 2024-04-27T04:09:59.960 INFO:tasks.workunit.client.0.smithi162.stdout:8/212: truncate d0/d10/d25/f15 25510 0 2024-04-27T04:09:59.961 INFO:tasks.workunit.client.0.smithi162.stdout:4/221: dread d2/f11 [0,4194304] 0 2024-04-27T04:09:59.961 INFO:tasks.workunit.client.0.smithi162.stdout:4/222: dread - d2/da/f44 zero size 2024-04-27T04:09:59.964 INFO:tasks.workunit.client.0.smithi162.stdout:6/206: creat d6/d1a/f47 x:0 0 0 2024-04-27T04:09:59.966 INFO:tasks.workunit.client.0.smithi162.stdout:6/207: readlink d6/d1a/d34/d3a/l42 0 2024-04-27T04:09:59.967 INFO:tasks.workunit.client.0.smithi162.stdout:8/213: dwrite d0/d10/d2b/f19 [0,4194304] 0 2024-04-27T04:09:59.968 INFO:tasks.workunit.client.0.smithi162.stdout:4/223: creat d2/d25/d1c/d39/f46 x:0 0 0 2024-04-27T04:09:59.974 INFO:tasks.workunit.client.0.smithi162.stdout:4/224: dwrite f1 [0,4194304] 0 2024-04-27T04:09:59.975 INFO:tasks.workunit.client.0.smithi162.stdout:6/208: creat d6/db/f48 x:0 0 0 2024-04-27T04:09:59.976 INFO:tasks.workunit.client.0.smithi162.stdout:4/225: chown d2/da/d1a/c40 260227 1 2024-04-27T04:09:59.976 INFO:tasks.workunit.client.0.smithi162.stdout:6/209: truncate d6/f39 4817768 0 2024-04-27T04:09:59.976 INFO:tasks.workunit.client.0.smithi162.stdout:0/238: read d5/f35 [74809,6277] 0 2024-04-27T04:09:59.978 INFO:tasks.workunit.client.0.smithi162.stdout:8/214: symlink d0/d10/d2b/d23/l41 0 2024-04-27T04:09:59.978 INFO:tasks.workunit.client.0.smithi162.stdout:0/239: write d5/f27 [4405605,54280] 0 2024-04-27T04:09:59.980 INFO:tasks.workunit.client.0.smithi162.stdout:4/226: mkdir d2/d25/d1c/d39/d3b/d47 0 2024-04-27T04:09:59.981 INFO:tasks.workunit.client.0.smithi162.stdout:0/240: write d5/f10 [1911868,6103] 0 2024-04-27T04:09:59.981 INFO:tasks.workunit.client.0.smithi162.stdout:8/215: dread d0/d10/d25/f1d [0,4194304] 0 2024-04-27T04:09:59.983 INFO:tasks.workunit.client.0.smithi162.stdout:0/241: dread - d5/dc/d37/f3a zero size 2024-04-27T04:09:59.983 INFO:tasks.workunit.client.0.smithi162.stdout:4/227: creat d2/d25/d1c/f48 x:0 0 0 2024-04-27T04:09:59.983 INFO:tasks.workunit.client.0.smithi162.stdout:0/242: chown d5 32018 1 2024-04-27T04:09:59.984 INFO:tasks.workunit.client.0.smithi162.stdout:0/243: fsync d5/dc/d1e/d46/f52 0 2024-04-27T04:09:59.985 INFO:tasks.workunit.client.0.smithi162.stdout:8/216: truncate d0/fd 206249 0 2024-04-27T04:09:59.986 INFO:tasks.workunit.client.0.smithi162.stdout:8/217: fdatasync d0/d10/d25/f30 0 2024-04-27T04:09:59.991 INFO:tasks.workunit.client.0.smithi162.stdout:4/228: dwrite d2/da/d2c/f35 [0,4194304] 0 2024-04-27T04:09:59.995 INFO:tasks.workunit.client.0.smithi162.stdout:8/218: dwrite d0/d10/d25/f30 [4194304,4194304] 0 2024-04-27T04:09:59.996 INFO:tasks.workunit.client.0.smithi162.stdout:8/219: fsync d0/d10/d2b/d2e/f40 0 2024-04-27T04:09:59.997 INFO:tasks.workunit.client.0.smithi162.stdout:4/229: rename d2/da/dd/f10 to d2/da/f49 0 2024-04-27T04:09:59.997 INFO:tasks.workunit.client.0.smithi162.stdout:8/220: readlink d0/d10/l34 0 2024-04-27T04:09:59.998 INFO:tasks.workunit.client.1.smithi181.stdout:8/553: write d1/d20/f24 [4099932,81847] 0 2024-04-27T04:09:59.998 INFO:tasks.workunit.client.0.smithi162.stdout:4/230: write d2/d25/d1c/d39/f46 [929715,96537] 0 2024-04-27T04:10:00.000 INFO:tasks.workunit.client.0.smithi162.stdout:4/231: chown d2/da/f44 763582 1 2024-04-27T04:10:00.002 INFO:tasks.workunit.client.1.smithi181.stdout:8/554: mkdir d1/d20/da4/dc5 0 2024-04-27T04:10:00.002 INFO:tasks.workunit.client.0.smithi162.stdout:8/221: mkdir d0/d10/d2b/d2d/d2f/d42 0 2024-04-27T04:10:00.010 INFO:tasks.workunit.client.1.smithi181.stdout:8/555: getdents d1/d6/d2f/d30/d9d 0 2024-04-27T04:10:00.015 INFO:tasks.workunit.client.0.smithi162.stdout:7/135: chown d3/d5/f12 6756164 1 2024-04-27T04:10:00.021 INFO:tasks.workunit.client.0.smithi162.stdout:9/189: truncate d3/fb 893726 0 2024-04-27T04:10:00.024 INFO:tasks.workunit.client.0.smithi162.stdout:2/158: truncate d0/d4/f22 119821 0 2024-04-27T04:10:00.024 INFO:tasks.workunit.client.0.smithi162.stdout:3/205: dwrite d1/d18/f12 [0,4194304] 0 2024-04-27T04:10:00.024 INFO:tasks.workunit.client.0.smithi162.stdout:9/190: mkdir d3/d13/d2d/d44 0 2024-04-27T04:10:00.027 INFO:tasks.workunit.client.0.smithi162.stdout:9/191: mkdir d3/d11/d45 0 2024-04-27T04:10:00.028 INFO:tasks.workunit.client.0.smithi162.stdout:2/159: truncate d0/d4/f15 1929319 0 2024-04-27T04:10:00.029 INFO:tasks.workunit.client.0.smithi162.stdout:8/222: getdents d0/d10/d2b/d23 0 2024-04-27T04:10:00.029 INFO:tasks.workunit.client.0.smithi162.stdout:9/192: write d3/d13/f26 [2430619,95857] 0 2024-04-27T04:10:00.032 INFO:tasks.workunit.client.0.smithi162.stdout:3/206: dwrite d1/d18/f1a [8388608,4194304] 0 2024-04-27T04:10:00.034 INFO:tasks.workunit.client.0.smithi162.stdout:1/178: creat d0/f34 x:0 0 0 2024-04-27T04:10:00.035 INFO:tasks.workunit.client.0.smithi162.stdout:9/193: mknod d3/d39/c46 0 2024-04-27T04:10:00.038 INFO:tasks.workunit.client.0.smithi162.stdout:4/232: rename d2/f1f to d2/d25/d1c/d39/d3b/d47/f4a 0 2024-04-27T04:10:00.040 INFO:tasks.workunit.client.0.smithi162.stdout:9/194: creat d3/d13/d2d/d3b/f47 x:0 0 0 2024-04-27T04:10:00.040 INFO:tasks.workunit.client.0.smithi162.stdout:2/160: dwrite d0/d4/d17/f2f [0,4194304] 0 2024-04-27T04:10:00.042 INFO:tasks.workunit.client.0.smithi162.stdout:4/233: mknod d2/da/d1a/c4b 0 2024-04-27T04:10:00.043 INFO:tasks.workunit.client.0.smithi162.stdout:7/136: rename d3/d5/f12 to d3/d5/d9/dc/de/f29 0 2024-04-27T04:10:00.051 INFO:tasks.workunit.client.1.smithi181.stdout:7/454: rename d1/d22/d15/d13/d71 to d1/d2/d1c/d8f 0 2024-04-27T04:10:00.058 INFO:tasks.workunit.client.0.smithi162.stdout:1/179: rename d0/d11/d1a to d0/d7/d10/d35 0 2024-04-27T04:10:00.059 INFO:tasks.workunit.client.0.smithi162.stdout:9/195: creat d3/d23/d1e/d43/f48 x:0 0 0 2024-04-27T04:10:00.060 INFO:tasks.workunit.client.1.smithi181.stdout:8/556: write d1/d6/d2f/d47/f4f [343636,97643] 0 2024-04-27T04:10:00.061 INFO:tasks.workunit.client.1.smithi181.stdout:8/557: dread - d1/d6/d2f/d72/fbd zero size 2024-04-27T04:10:00.062 INFO:tasks.workunit.client.1.smithi181.stdout:8/558: readlink d1/d6/d2f/d72/l8a 0 2024-04-27T04:10:00.064 INFO:tasks.workunit.client.0.smithi162.stdout:4/234: rename d2/d25/d1c/f48 to d2/d25/d1c/d39/d3b/d47/f4c 0 2024-04-27T04:10:00.065 INFO:tasks.workunit.client.0.smithi162.stdout:1/180: symlink d0/d7/l36 0 2024-04-27T04:10:00.069 INFO:tasks.workunit.client.0.smithi162.stdout:2/161: truncate d0/d4/f22 946364 0 2024-04-27T04:10:00.072 INFO:tasks.workunit.client.0.smithi162.stdout:2/162: dread d0/d4/f9 [0,4194304] 0 2024-04-27T04:10:00.072 INFO:tasks.workunit.client.1.smithi181.stdout:7/455: getdents d1/d2/d35 0 2024-04-27T04:10:00.075 INFO:tasks.workunit.client.0.smithi162.stdout:3/207: dread d1/d18/f1a [0,4194304] 0 2024-04-27T04:10:00.076 INFO:tasks.workunit.client.0.smithi162.stdout:3/208: dread - d1/d18/d23/d2f/f42 zero size 2024-04-27T04:10:00.077 INFO:tasks.workunit.client.0.smithi162.stdout:4/235: rename d2/d25/c3f to d2/d25/c4d 0 2024-04-27T04:10:00.078 INFO:tasks.workunit.client.0.smithi162.stdout:4/236: chown d2/d25/f2d 19963002 1 2024-04-27T04:10:00.079 INFO:tasks.workunit.client.0.smithi162.stdout:4/237: chown d2/d25/d1c/d24/f3a 8155238 1 2024-04-27T04:10:00.079 INFO:tasks.workunit.client.1.smithi181.stdout:7/456: mknod d1/d2/d1c/d56/c90 0 2024-04-27T04:10:00.081 INFO:tasks.workunit.client.0.smithi162.stdout:8/223: write d0/fd [298986,82867] 0 2024-04-27T04:10:00.084 INFO:tasks.workunit.client.1.smithi181.stdout:7/457: rename d1/df/d1a/d6f to d1/d2/d1c/d37/d91 0 2024-04-27T04:10:00.089 INFO:tasks.workunit.client.1.smithi181.stdout:7/458: unlink d1/df/d62/c70 0 2024-04-27T04:10:00.089 INFO:tasks.workunit.client.0.smithi162.stdout:0/244: dread d5/f16 [0,4194304] 0 2024-04-27T04:10:00.091 INFO:tasks.workunit.client.0.smithi162.stdout:0/245: write d5/dc/d1e/f30 [630929,61462] 0 2024-04-27T04:10:00.091 INFO:tasks.workunit.client.0.smithi162.stdout:3/209: rmdir d1/d21 39 2024-04-27T04:10:00.093 INFO:tasks.workunit.client.0.smithi162.stdout:5/186: dread d8/d9/f21 [0,4194304] 0 2024-04-27T04:10:00.096 INFO:tasks.workunit.client.1.smithi181.stdout:7/459: dwrite d1/d22/d15/d13/d1b/f40 [4194304,4194304] 0 2024-04-27T04:10:00.097 INFO:tasks.workunit.client.0.smithi162.stdout:8/224: rename d0/d10/d2b/d23/c2c to d0/d10/d2b/d2d/d2f/c43 0 2024-04-27T04:10:00.097 INFO:tasks.workunit.client.1.smithi181.stdout:7/460: stat d1/d2/f24 0 2024-04-27T04:10:00.099 INFO:tasks.workunit.client.0.smithi162.stdout:4/238: dread f1 [4194304,4194304] 0 2024-04-27T04:10:00.100 INFO:tasks.workunit.client.0.smithi162.stdout:4/239: write d2/d25/d1c/f33 [209920,5012] 0 2024-04-27T04:10:00.107 INFO:tasks.workunit.client.0.smithi162.stdout:9/196: write d3/d13/d15/f18 [435335,122660] 0 2024-04-27T04:10:00.108 INFO:tasks.workunit.client.0.smithi162.stdout:0/246: rename d5/l2f to d5/dc/d1e/d20/l54 0 2024-04-27T04:10:00.111 INFO:tasks.workunit.client.1.smithi181.stdout:8/559: dwrite d1/f46 [0,4194304] 0 2024-04-27T04:10:00.113 INFO:tasks.workunit.client.0.smithi162.stdout:5/187: rmdir d8/d1a/d23/d40/d3f 0 2024-04-27T04:10:00.113 INFO:tasks.workunit.client.1.smithi181.stdout:8/560: read d1/d6/d2f/f3a [883400,98119] 0 2024-04-27T04:10:00.115 INFO:tasks.workunit.client.0.smithi162.stdout:9/197: mknod d3/d13/d2d/d3b/c49 0 2024-04-27T04:10:00.116 INFO:tasks.workunit.client.0.smithi162.stdout:9/198: fdatasync d3/d23/d1e/f27 0 2024-04-27T04:10:00.119 INFO:tasks.workunit.client.0.smithi162.stdout:0/247: read d5/dc/f22 [512192,60977] 0 2024-04-27T04:10:00.121 INFO:tasks.workunit.client.1.smithi181.stdout:8/561: dwrite d1/d6/d2f/d47/fac [0,4194304] 0 2024-04-27T04:10:00.122 INFO:tasks.workunit.client.0.smithi162.stdout:8/225: unlink d0/d10/d25/f15 0 2024-04-27T04:10:00.123 INFO:tasks.workunit.client.1.smithi181.stdout:8/562: chown d1/d6/d2f/d40/d2a/lae 451 1 2024-04-27T04:10:00.127 INFO:tasks.workunit.client.1.smithi181.stdout:8/563: creat d1/d6/d2f/d72/fc6 x:0 0 0 2024-04-27T04:10:00.128 INFO:tasks.workunit.client.0.smithi162.stdout:2/163: dwrite d0/d4/f16 [0,4194304] 0 2024-04-27T04:10:00.128 INFO:tasks.workunit.client.0.smithi162.stdout:0/248: dwrite d5/dc/d1e/f3e [0,4194304] 0 2024-04-27T04:10:00.131 INFO:tasks.workunit.client.0.smithi162.stdout:5/188: fdatasync d8/f20 0 2024-04-27T04:10:00.132 INFO:tasks.workunit.client.0.smithi162.stdout:1/181: truncate d0/d7/d2a/f2d 3243251 0 2024-04-27T04:10:00.133 INFO:tasks.workunit.client.1.smithi181.stdout:8/564: creat d1/d6/d1a/d25/fc7 x:0 0 0 2024-04-27T04:10:00.135 INFO:tasks.workunit.client.0.smithi162.stdout:9/199: creat d3/d23/d1e/f4a x:0 0 0 2024-04-27T04:10:00.136 INFO:tasks.workunit.client.0.smithi162.stdout:8/226: mkdir d0/d10/d44 0 2024-04-27T04:10:00.136 INFO:tasks.workunit.client.0.smithi162.stdout:9/200: write d3/d23/f3e [503337,46619] 0 2024-04-27T04:10:00.137 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:09:59 smithi162 ceph-mon[26841]: pgmap v265: 65 pgs: 65 active+clean; 1.2 GiB data, 5.3 GiB used, 531 GiB / 536 GiB avail; 24 MiB/s rd, 157 MiB/s wr, 372 op/s 2024-04-27T04:10:00.142 INFO:tasks.workunit.client.0.smithi162.stdout:8/227: dwrite d0/f7 [0,4194304] 0 2024-04-27T04:10:00.143 INFO:tasks.workunit.client.1.smithi181.stdout:8/565: creat d1/d6/d2f/d30/d9d/fc8 x:0 0 0 2024-04-27T04:10:00.145 INFO:tasks.workunit.client.0.smithi162.stdout:4/240: link d2/da/dd/l32 d2/da/d1a/l4e 0 2024-04-27T04:10:00.145 INFO:tasks.workunit.client.0.smithi162.stdout:6/210: sync 2024-04-27T04:10:00.149 INFO:tasks.workunit.client.0.smithi162.stdout:6/211: write d6/f2d [3143543,102265] 0 2024-04-27T04:10:00.150 INFO:tasks.workunit.client.0.smithi162.stdout:0/249: creat d5/dc/d1e/d46/f55 x:0 0 0 2024-04-27T04:10:00.152 INFO:tasks.workunit.client.0.smithi162.stdout:5/189: creat d8/d1a/d29/f43 x:0 0 0 2024-04-27T04:10:00.155 INFO:tasks.workunit.client.0.smithi162.stdout:8/228: dwrite d0/fd [0,4194304] 0 2024-04-27T04:10:00.160 INFO:tasks.workunit.client.0.smithi162.stdout:4/241: dwrite d2/d25/f36 [0,4194304] 0 2024-04-27T04:10:00.172 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:09:59 smithi181 ceph-mon[33774]: pgmap v265: 65 pgs: 65 active+clean; 1.2 GiB data, 5.3 GiB used, 531 GiB / 536 GiB avail; 24 MiB/s rd, 157 MiB/s wr, 372 op/s 2024-04-27T04:10:00.242 INFO:tasks.workunit.client.1.smithi181.stdout:9/472: sync 2024-04-27T04:10:00.243 INFO:tasks.workunit.client.1.smithi181.stdout:3/436: sync 2024-04-27T04:10:00.244 INFO:tasks.workunit.client.1.smithi181.stdout:3/437: chown d0/d28/f42 485075898 1 2024-04-27T04:10:00.249 INFO:tasks.workunit.client.0.smithi162.stdout:7/137: sync 2024-04-27T04:10:00.259 INFO:tasks.workunit.client.0.smithi162.stdout:7/138: dread d3/d5/d9/f13 [0,4194304] 0 2024-04-27T04:10:00.260 INFO:tasks.workunit.client.0.smithi162.stdout:7/139: chown d3/f16 30893997 1 2024-04-27T04:10:00.260 INFO:tasks.workunit.client.0.smithi162.stdout:7/140: read - d3/f18 zero size 2024-04-27T04:10:00.263 INFO:tasks.workunit.client.0.smithi162.stdout:9/201: creat d3/d23/f4b x:0 0 0 2024-04-27T04:10:00.264 INFO:tasks.workunit.client.0.smithi162.stdout:9/202: write d3/f5 [7994908,75384] 0 2024-04-27T04:10:00.264 INFO:tasks.workunit.client.0.smithi162.stdout:9/203: readlink d3/l28 0 2024-04-27T04:10:00.265 INFO:tasks.workunit.client.0.smithi162.stdout:9/204: chown d3/cf 723919 1 2024-04-27T04:10:00.265 INFO:tasks.workunit.client.0.smithi162.stdout:9/205: write d3/f5 [8151977,73412] 0 2024-04-27T04:10:00.272 INFO:tasks.workunit.client.0.smithi162.stdout:6/212: rename d6/fa to d6/db/f49 0 2024-04-27T04:10:00.273 INFO:tasks.workunit.client.0.smithi162.stdout:7/141: rename d3/d5 to d3/d5/d9/dc/d2a 22 2024-04-27T04:10:00.274 INFO:tasks.workunit.client.0.smithi162.stdout:6/213: chown d6/db/d32/f45 1436 1 2024-04-27T04:10:00.276 INFO:tasks.workunit.client.0.smithi162.stdout:5/190: mknod d8/d1a/d23/c44 0 2024-04-27T04:10:00.277 INFO:tasks.workunit.client.0.smithi162.stdout:2/164: dwrite d0/f6 [0,4194304] 0 2024-04-27T04:10:00.278 INFO:tasks.workunit.client.0.smithi162.stdout:5/191: write d8/d9/f32 [387162,100405] 0 2024-04-27T04:10:00.279 INFO:tasks.workunit.client.0.smithi162.stdout:8/229: mknod d0/d10/d2b/d23/c45 0 2024-04-27T04:10:00.280 INFO:tasks.workunit.client.0.smithi162.stdout:4/242: symlink d2/d25/d1c/d39/l4f 0 2024-04-27T04:10:00.280 INFO:tasks.workunit.client.0.smithi162.stdout:5/192: fdatasync d8/d9/f32 0 2024-04-27T04:10:00.281 INFO:tasks.workunit.client.0.smithi162.stdout:5/193: chown d8/ld 0 1 2024-04-27T04:10:00.282 INFO:tasks.workunit.client.0.smithi162.stdout:2/165: dread d0/f6 [0,4194304] 0 2024-04-27T04:10:00.288 INFO:tasks.workunit.client.0.smithi162.stdout:0/250: mkdir d5/dc/d1e/d56 0 2024-04-27T04:10:00.289 INFO:tasks.workunit.client.0.smithi162.stdout:6/214: readlink d6/l2a 0 2024-04-27T04:10:00.290 INFO:tasks.workunit.client.0.smithi162.stdout:6/215: write d6/db/d32/f45 [136565,80311] 0 2024-04-27T04:10:00.290 INFO:tasks.workunit.client.0.smithi162.stdout:7/142: mkdir d3/d5/d9/dc/de/d2b 0 2024-04-27T04:10:00.291 INFO:tasks.workunit.client.0.smithi162.stdout:7/143: chown d3/c25 21 1 2024-04-27T04:10:00.292 INFO:tasks.workunit.client.0.smithi162.stdout:1/182: creat d0/f37 x:0 0 0 2024-04-27T04:10:00.294 INFO:tasks.workunit.client.1.smithi181.stdout:9/473: creat d0/d4d/fa5 x:0 0 0 2024-04-27T04:10:00.296 INFO:tasks.workunit.client.1.smithi181.stdout:3/438: write d0/f37 [1584886,111013] 0 2024-04-27T04:10:00.299 INFO:tasks.workunit.client.0.smithi162.stdout:4/243: unlink d2/da/dd/f2a 0 2024-04-27T04:10:00.300 INFO:tasks.workunit.client.0.smithi162.stdout:4/244: write d2/d25/fb [2774492,22547] 0 2024-04-27T04:10:00.301 INFO:tasks.workunit.client.1.smithi181.stdout:9/474: rename d0/d44/d4b/d43/d88/da1/la3 to d0/d44/d4b/d5f/la6 0 2024-04-27T04:10:00.306 INFO:tasks.workunit.client.1.smithi181.stdout:3/439: creat d0/d28/d86/f8d x:0 0 0 2024-04-27T04:10:00.306 INFO:tasks.workunit.client.1.smithi181.stdout:3/440: fdatasync d0/de/f17 0 2024-04-27T04:10:00.308 INFO:tasks.workunit.client.1.smithi181.stdout:9/475: fdatasync d0/d44/f4a 0 2024-04-27T04:10:00.309 INFO:tasks.workunit.client.0.smithi162.stdout:8/230: rmdir d0 39 2024-04-27T04:10:00.313 INFO:tasks.workunit.client.0.smithi162.stdout:7/144: write d3/d5/d9/dc/f15 [5154677,5271] 0 2024-04-27T04:10:00.315 INFO:tasks.workunit.client.0.smithi162.stdout:1/183: rmdir d0/d7/d10 39 2024-04-27T04:10:00.320 INFO:tasks.workunit.client.0.smithi162.stdout:4/245: mknod d2/da/d1a/c50 0 2024-04-27T04:10:00.320 INFO:tasks.workunit.client.0.smithi162.stdout:2/166: mknod d0/c32 0 2024-04-27T04:10:00.321 INFO:tasks.workunit.client.1.smithi181.stdout:3/441: link d0/f4e d0/d6c/d52/d44/d56/d5b/f8e 0 2024-04-27T04:10:00.322 INFO:tasks.workunit.client.1.smithi181.stdout:3/442: write d0/de/d3f/f5f [588610,112278] 0 2024-04-27T04:10:00.323 INFO:tasks.workunit.client.0.smithi162.stdout:0/251: mkdir d5/dc/d1e/d56/d57 0 2024-04-27T04:10:00.331 INFO:tasks.workunit.client.0.smithi162.stdout:4/246: write d2/da/f49 [547646,45609] 0 2024-04-27T04:10:00.334 INFO:tasks.workunit.client.0.smithi162.stdout:9/206: getdents d3/d11 0 2024-04-27T04:10:00.335 INFO:tasks.workunit.client.0.smithi162.stdout:0/252: creat d5/f58 x:0 0 0 2024-04-27T04:10:00.336 INFO:tasks.workunit.client.0.smithi162.stdout:8/231: creat d0/d10/d2b/d2e/f46 x:0 0 0 2024-04-27T04:10:00.337 INFO:tasks.workunit.client.0.smithi162.stdout:7/145: creat d3/d5/d9/dc/de/d2b/f2c x:0 0 0 2024-04-27T04:10:00.339 INFO:tasks.workunit.client.0.smithi162.stdout:7/146: stat d3/f16 0 2024-04-27T04:10:00.342 INFO:tasks.workunit.client.0.smithi162.stdout:1/184: creat d0/f38 x:0 0 0 2024-04-27T04:10:00.344 INFO:tasks.workunit.client.0.smithi162.stdout:9/207: symlink d3/l4c 0 2024-04-27T04:10:00.345 INFO:tasks.workunit.client.0.smithi162.stdout:0/253: dwrite d5/dc/d1e/d46/f55 [0,4194304] 0 2024-04-27T04:10:00.345 INFO:tasks.workunit.client.1.smithi181.stdout:3/443: truncate d0/de/d20/f62 2615665 0 2024-04-27T04:10:00.345 INFO:tasks.workunit.client.0.smithi162.stdout:7/147: write d3/d5/d9/dc/f15 [5648332,34435] 0 2024-04-27T04:10:00.346 INFO:tasks.workunit.client.0.smithi162.stdout:0/254: write d5/dc/f14 [5153741,392] 0 2024-04-27T04:10:00.348 INFO:tasks.workunit.client.0.smithi162.stdout:6/216: dwrite d6/f9 [0,4194304] 0 2024-04-27T04:10:00.348 INFO:tasks.workunit.client.0.smithi162.stdout:7/148: chown d3/d5/d9/f24 16627978 1 2024-04-27T04:10:00.361 INFO:tasks.workunit.client.1.smithi181.stdout:3/444: link d0/lc d0/de/d20/d58/d64/l8f 0 2024-04-27T04:10:00.378 INFO:tasks.workunit.client.0.smithi162.stdout:3/210: sync 2024-04-27T04:10:00.379 INFO:tasks.workunit.client.0.smithi162.stdout:8/232: link d0/d10/d2b/d2d/d2f/d39/f33 d0/d10/d44/f47 0 2024-04-27T04:10:00.379 INFO:tasks.workunit.client.0.smithi162.stdout:4/247: fsync d2/d25/d1c/f3d 0 2024-04-27T04:10:00.380 INFO:tasks.workunit.client.0.smithi162.stdout:6/217: symlink d6/d1a/l4a 0 2024-04-27T04:10:00.381 INFO:tasks.workunit.client.0.smithi162.stdout:1/185: symlink d0/d7/d10/l39 0 2024-04-27T04:10:00.383 INFO:tasks.workunit.client.0.smithi162.stdout:7/149: mkdir d3/d1d/d2d 0 2024-04-27T04:10:00.383 INFO:tasks.workunit.client.0.smithi162.stdout:0/255: dread d5/dc/d1e/f3e [0,4194304] 0 2024-04-27T04:10:00.383 INFO:tasks.workunit.client.0.smithi162.stdout:4/248: mknod d2/da/d1a/c51 0 2024-04-27T04:10:00.384 INFO:tasks.workunit.client.0.smithi162.stdout:2/167: dwrite d0/d4/f15 [0,4194304] 0 2024-04-27T04:10:00.386 INFO:tasks.workunit.client.0.smithi162.stdout:1/186: dread d0/f2b [0,4194304] 0 2024-04-27T04:10:00.387 INFO:tasks.workunit.client.0.smithi162.stdout:8/233: creat d0/d10/d2b/d2d/d2f/d42/f48 x:0 0 0 2024-04-27T04:10:00.387 INFO:tasks.workunit.client.0.smithi162.stdout:7/150: dread - d3/d5/d9/dc/f27 zero size 2024-04-27T04:10:00.388 INFO:tasks.workunit.client.0.smithi162.stdout:8/234: stat d0/c18 0 2024-04-27T04:10:00.389 INFO:tasks.workunit.client.0.smithi162.stdout:7/151: chown d3/c25 6782 1 2024-04-27T04:10:00.389 INFO:tasks.workunit.client.0.smithi162.stdout:1/187: truncate d0/f2b 1266958 0 2024-04-27T04:10:00.390 INFO:tasks.workunit.client.0.smithi162.stdout:2/168: mknod d0/d4/c33 0 2024-04-27T04:10:00.393 INFO:tasks.workunit.client.0.smithi162.stdout:6/218: dwrite d6/d1a/f20 [0,4194304] 0 2024-04-27T04:10:00.397 INFO:tasks.workunit.client.0.smithi162.stdout:7/152: creat d3/d5/d9/dc/f2e x:0 0 0 2024-04-27T04:10:00.398 INFO:tasks.workunit.client.0.smithi162.stdout:1/188: symlink d0/l3a 0 2024-04-27T04:10:00.400 INFO:tasks.workunit.client.0.smithi162.stdout:4/249: dread d2/f9 [0,4194304] 0 2024-04-27T04:10:00.402 INFO:tasks.workunit.client.0.smithi162.stdout:6/219: creat d6/db/d32/f4b x:0 0 0 2024-04-27T04:10:00.402 INFO:tasks.workunit.client.1.smithi181.stdout:0/472: sync 2024-04-27T04:10:00.402 INFO:tasks.workunit.client.0.smithi162.stdout:0/256: dwrite d5/f16 [0,4194304] 0 2024-04-27T04:10:00.404 INFO:tasks.workunit.client.1.smithi181.stdout:0/473: creat dc/d54/fab x:0 0 0 2024-04-27T04:10:00.405 INFO:tasks.workunit.client.0.smithi162.stdout:8/235: dread d0/fd [0,4194304] 0 2024-04-27T04:10:00.405 INFO:tasks.workunit.client.0.smithi162.stdout:4/250: chown d2/da/d1a/f29 4332 1 2024-04-27T04:10:00.406 INFO:tasks.workunit.client.0.smithi162.stdout:7/153: creat d3/d5/d9/dc/f2f x:0 0 0 2024-04-27T04:10:00.408 INFO:tasks.workunit.client.1.smithi181.stdout:0/474: symlink dc/d1f/d3b/d4a/d95/lac 0 2024-04-27T04:10:00.408 INFO:tasks.workunit.client.0.smithi162.stdout:8/236: chown d0/d10/d25/d3c 375915 1 2024-04-27T04:10:00.409 INFO:tasks.workunit.client.0.smithi162.stdout:4/251: stat d2/da/d1a/f29 0 2024-04-27T04:10:00.412 INFO:tasks.workunit.client.0.smithi162.stdout:2/169: dread d0/d4/d17/f2f [0,4194304] 0 2024-04-27T04:10:00.412 INFO:tasks.workunit.client.0.smithi162.stdout:6/220: rename d6/f9 to d6/d1a/d34/d3a/f4c 0 2024-04-27T04:10:00.412 INFO:tasks.workunit.client.0.smithi162.stdout:3/211: dwrite d1/d18/f1f [0,4194304] 0 2024-04-27T04:10:00.414 INFO:tasks.workunit.client.0.smithi162.stdout:6/221: stat d6/d1a/cf 0 2024-04-27T04:10:00.415 INFO:tasks.workunit.client.0.smithi162.stdout:1/189: link d0/d7/f27 d0/f3b 0 2024-04-27T04:10:00.415 INFO:tasks.workunit.client.0.smithi162.stdout:3/212: fsync d1/d18/d23/f2e 0 2024-04-27T04:10:00.417 INFO:tasks.workunit.client.1.smithi181.stdout:0/475: chown dc/d1f/d3b/c53 735 1 2024-04-27T04:10:00.418 INFO:tasks.workunit.client.0.smithi162.stdout:3/213: write d1/d18/f12 [449982,25094] 0 2024-04-27T04:10:00.419 INFO:tasks.workunit.client.0.smithi162.stdout:0/257: dwrite d5/dc/d1e/d41/f4f [0,4194304] 0 2024-04-27T04:10:00.422 INFO:tasks.workunit.client.0.smithi162.stdout:6/222: dread d6/f2d [0,4194304] 0 2024-04-27T04:10:00.422 INFO:tasks.workunit.client.0.smithi162.stdout:8/237: unlink d0/d10/d25/f1d 0 2024-04-27T04:10:00.423 INFO:tasks.workunit.client.0.smithi162.stdout:6/223: chown d6/d1a/d34/l3c 36 1 2024-04-27T04:10:00.424 INFO:tasks.workunit.client.0.smithi162.stdout:7/154: dread d3/d5/d9/dc/de/f29 [0,4194304] 0 2024-04-27T04:10:00.424 INFO:tasks.workunit.client.0.smithi162.stdout:4/252: mkdir d2/d25/d1c/d39/d3b/d52 0 2024-04-27T04:10:00.424 INFO:tasks.workunit.client.0.smithi162.stdout:6/224: chown d6/db/c36 37 1 2024-04-27T04:10:00.425 INFO:tasks.workunit.client.1.smithi181.stdout:0/476: mknod dc/d1f/d3b/d75/cad 0 2024-04-27T04:10:00.429 INFO:tasks.workunit.client.1.smithi181.stdout:0/477: creat dc/d1f/fae x:0 0 0 2024-04-27T04:10:00.433 INFO:tasks.workunit.client.1.smithi181.stdout:0/478: symlink dc/d1f/laf 0 2024-04-27T04:10:00.436 INFO:tasks.workunit.client.0.smithi162.stdout:8/238: symlink d0/d10/d2b/d2d/d2f/d39/l49 0 2024-04-27T04:10:00.437 INFO:tasks.workunit.client.1.smithi181.stdout:0/479: truncate dc/d1f/d3b/d75/f79 440257 0 2024-04-27T04:10:00.437 INFO:tasks.workunit.client.0.smithi162.stdout:8/239: write d0/d10/d2b/d23/f31 [405820,82107] 0 2024-04-27T04:10:00.438 INFO:tasks.workunit.client.0.smithi162.stdout:6/225: mknod d6/d1a/c4d 0 2024-04-27T04:10:00.438 INFO:tasks.workunit.client.1.smithi181.stdout:0/480: dread - dc/d1f/d3b/d4a/d95/d26/d41/d6f/d7d/fa7 zero size 2024-04-27T04:10:00.439 INFO:tasks.workunit.client.0.smithi162.stdout:6/226: chown d6/c1b 122036 1 2024-04-27T04:10:00.439 INFO:tasks.workunit.client.1.smithi181.stdout:0/481: stat dc/d54 0 2024-04-27T04:10:00.440 INFO:tasks.workunit.client.0.smithi162.stdout:7/155: creat d3/d5/d9/dc/de/f30 x:0 0 0 2024-04-27T04:10:00.440 INFO:tasks.workunit.client.0.smithi162.stdout:7/156: read - d3/f18 zero size 2024-04-27T04:10:00.441 INFO:tasks.workunit.client.1.smithi181.stdout:0/482: read - dc/d1f/d3b/f77 zero size 2024-04-27T04:10:00.445 INFO:tasks.workunit.client.0.smithi162.stdout:4/253: symlink d2/da/l53 0 2024-04-27T04:10:00.445 INFO:tasks.workunit.client.0.smithi162.stdout:4/254: read - d2/d25/f2d zero size 2024-04-27T04:10:00.447 INFO:tasks.workunit.client.1.smithi181.stdout:0/483: rename dc/d1f/da5 to dc/d1f/d3b/d4a/d95/d26/d41/db0 0 2024-04-27T04:10:00.453 INFO:tasks.workunit.client.0.smithi162.stdout:7/157: rename d3/d5/d9/dc/de/f29 to d3/d1d/f31 0 2024-04-27T04:10:00.457 INFO:tasks.workunit.client.0.smithi162.stdout:1/190: write d0/d7/d2a/f30 [1839219,18130] 0 2024-04-27T04:10:00.458 INFO:tasks.workunit.client.0.smithi162.stdout:8/240: rmdir d0/d10/d25/d3c 0 2024-04-27T04:10:00.459 INFO:tasks.workunit.client.0.smithi162.stdout:1/191: dread - d0/d7/f27 zero size 2024-04-27T04:10:00.461 INFO:tasks.workunit.client.0.smithi162.stdout:7/158: dwrite d3/d5/d9/dc/de/f30 [0,4194304] 0 2024-04-27T04:10:00.462 INFO:tasks.workunit.client.0.smithi162.stdout:1/192: truncate d0/d11/f2c 5943585 0 2024-04-27T04:10:00.462 INFO:tasks.workunit.client.1.smithi181.stdout:0/484: dread dc/d1f/d3b/d75/f90 [0,4194304] 0 2024-04-27T04:10:00.463 INFO:tasks.workunit.client.0.smithi162.stdout:1/193: chown d0/f34 0 1 2024-04-27T04:10:00.464 INFO:tasks.workunit.client.0.smithi162.stdout:4/255: rename d2/da/d2c to d2/d54 0 2024-04-27T04:10:00.464 INFO:tasks.workunit.client.0.smithi162.stdout:7/159: write d3/d5/d9/f24 [742385,73943] 0 2024-04-27T04:10:00.465 INFO:tasks.workunit.client.0.smithi162.stdout:1/194: chown d0/d11/f2e 84216526 1 2024-04-27T04:10:00.469 INFO:tasks.workunit.client.0.smithi162.stdout:2/170: truncate d0/d4/f22 784164 0 2024-04-27T04:10:00.471 INFO:tasks.workunit.client.0.smithi162.stdout:8/241: rename d0/c18 to d0/d10/d2b/d2d/d2f/d39/c4a 0 2024-04-27T04:10:00.471 INFO:tasks.workunit.client.0.smithi162.stdout:3/214: truncate d1/d18/f1e 3203907 0 2024-04-27T04:10:00.472 INFO:tasks.workunit.client.0.smithi162.stdout:3/215: chown d1/d18/d23/d34 4032 1 2024-04-27T04:10:00.473 INFO:tasks.workunit.client.0.smithi162.stdout:7/160: fsync d3/d5/d9/f13 0 2024-04-27T04:10:00.474 INFO:tasks.workunit.client.0.smithi162.stdout:4/256: creat d2/d25/d1c/f55 x:0 0 0 2024-04-27T04:10:00.481 INFO:tasks.workunit.client.0.smithi162.stdout:7/161: mknod d3/d1d/c32 0 2024-04-27T04:10:00.482 INFO:tasks.workunit.client.0.smithi162.stdout:4/257: dwrite d2/d25/d1c/d39/f45 [0,4194304] 0 2024-04-27T04:10:00.483 INFO:tasks.workunit.client.0.smithi162.stdout:1/195: mkdir d0/d7/d3c 0 2024-04-27T04:10:00.486 INFO:tasks.workunit.client.1.smithi181.stdout:0/485: write dc/fd [3602034,60276] 0 2024-04-27T04:10:00.486 INFO:tasks.workunit.client.0.smithi162.stdout:3/216: dread d1/d21/f39 [0,4194304] 0 2024-04-27T04:10:00.487 INFO:tasks.workunit.client.0.smithi162.stdout:6/227: dwrite d6/f26 [0,4194304] 0 2024-04-27T04:10:00.489 INFO:tasks.workunit.client.0.smithi162.stdout:3/217: dread - d1/d18/d23/f28 zero size 2024-04-27T04:10:00.492 INFO:tasks.workunit.client.0.smithi162.stdout:5/194: sync 2024-04-27T04:10:00.494 INFO:tasks.workunit.client.0.smithi162.stdout:7/162: dread d3/d1d/f31 [0,4194304] 0 2024-04-27T04:10:00.495 INFO:tasks.workunit.client.1.smithi181.stdout:0/486: creat dc/d1f/d3b/d4a/fb1 x:0 0 0 2024-04-27T04:10:00.496 INFO:tasks.workunit.client.1.smithi181.stdout:0/487: chown dc/d1f/d3b/c53 24058062 1 2024-04-27T04:10:00.499 INFO:tasks.workunit.client.0.smithi162.stdout:5/195: dwrite d8/f22 [0,4194304] 0 2024-04-27T04:10:00.500 INFO:tasks.workunit.client.0.smithi162.stdout:6/228: symlink d6/d1a/d34/l4e 0 2024-04-27T04:10:00.503 INFO:tasks.workunit.client.0.smithi162.stdout:3/218: mknod d1/d18/c46 0 2024-04-27T04:10:00.505 INFO:tasks.workunit.client.0.smithi162.stdout:2/171: dwrite d0/f1 [0,4194304] 0 2024-04-27T04:10:00.509 INFO:tasks.workunit.client.1.smithi181.stdout:0/488: rename dc/d1f/d3b/d4a/d7a/d9d to dc/d1f/d3b/db2 0 2024-04-27T04:10:00.510 INFO:tasks.workunit.client.1.smithi181.stdout:0/489: dread - dc/d1f/fae zero size 2024-04-27T04:10:00.513 INFO:tasks.workunit.client.0.smithi162.stdout:7/163: rmdir d3 39 2024-04-27T04:10:00.518 INFO:tasks.workunit.client.0.smithi162.stdout:1/196: link d0/fe d0/d7/d10/f3d 0 2024-04-27T04:10:00.519 INFO:tasks.workunit.client.0.smithi162.stdout:8/242: truncate d0/fc 3875917 0 2024-04-27T04:10:00.520 INFO:tasks.workunit.client.1.smithi181.stdout:0/490: mkdir dc/d1f/d3b/d75/db3 0 2024-04-27T04:10:00.524 INFO:tasks.workunit.client.1.smithi181.stdout:0/491: mknod dc/d1f/d6b/d7e/cb4 0 2024-04-27T04:10:00.526 INFO:tasks.workunit.client.1.smithi181.stdout:0/492: dread - dc/d1f/d3b/d4a/d95/d26/d41/d6f/d7d/fa7 zero size 2024-04-27T04:10:00.526 INFO:tasks.workunit.client.0.smithi162.stdout:3/219: mkdir d1/d21/d47 0 2024-04-27T04:10:00.527 INFO:tasks.workunit.client.1.smithi181.stdout:0/493: write dc/d1f/d3b/d4a/d95/d26/d41/f9e [62189,103689] 0 2024-04-27T04:10:00.528 INFO:tasks.workunit.client.0.smithi162.stdout:2/172: rmdir d0/d4/d2a 39 2024-04-27T04:10:00.528 INFO:tasks.workunit.client.0.smithi162.stdout:5/196: dwrite d8/d1a/f3e [0,4194304] 0 2024-04-27T04:10:00.530 INFO:tasks.workunit.client.0.smithi162.stdout:7/164: read - d3/f16 zero size 2024-04-27T04:10:00.531 INFO:tasks.workunit.client.0.smithi162.stdout:5/197: dread - d8/d1a/d23/d40/f42 zero size 2024-04-27T04:10:00.533 INFO:tasks.workunit.client.0.smithi162.stdout:7/165: write d3/d5/d9/dc/f2f [315554,64376] 0 2024-04-27T04:10:00.534 INFO:tasks.workunit.client.0.smithi162.stdout:4/258: link d2/f4 d2/d25/d1c/d24/f56 0 2024-04-27T04:10:00.536 INFO:tasks.workunit.client.0.smithi162.stdout:2/173: dwrite d0/d4/f16 [0,4194304] 0 2024-04-27T04:10:00.539 INFO:tasks.workunit.client.0.smithi162.stdout:6/229: link d6/d1a/d34/f3e d6/d1a/d34/d46/f4f 0 2024-04-27T04:10:00.541 INFO:tasks.workunit.client.0.smithi162.stdout:6/230: write d6/db/d32/f4b [8666,93169] 0 2024-04-27T04:10:00.544 INFO:tasks.workunit.client.0.smithi162.stdout:4/259: dwrite d2/da/d1a/f2e [4194304,4194304] 0 2024-04-27T04:10:00.545 INFO:tasks.workunit.client.0.smithi162.stdout:3/220: dread d1/d18/f10 [0,4194304] 0 2024-04-27T04:10:00.545 INFO:tasks.workunit.client.1.smithi181.stdout:0/494: link dc/d1f/d3b/d4a/d95/d26/d41/d6f/c67 dc/d1f/d3b/cb5 0 2024-04-27T04:10:00.546 INFO:tasks.workunit.client.0.smithi162.stdout:4/260: read d2/d25/d1c/f3d [236633,41664] 0 2024-04-27T04:10:00.547 INFO:tasks.workunit.client.0.smithi162.stdout:3/221: write d1/d18/f1d [346923,27583] 0 2024-04-27T04:10:00.548 INFO:tasks.workunit.client.0.smithi162.stdout:1/197: mkdir d0/d11/d3e 0 2024-04-27T04:10:00.549 INFO:tasks.workunit.client.0.smithi162.stdout:9/208: sync 2024-04-27T04:10:00.550 INFO:tasks.workunit.client.0.smithi162.stdout:0/258: sync 2024-04-27T04:10:00.550 INFO:tasks.workunit.client.0.smithi162.stdout:9/209: write f0 [3437440,35891] 0 2024-04-27T04:10:00.556 INFO:tasks.workunit.client.0.smithi162.stdout:7/166: chown d3/d5/d9/dc/de/c10 1 1 2024-04-27T04:10:00.556 INFO:tasks.workunit.client.0.smithi162.stdout:7/167: chown d3/d5/d9/dc/de/c10 7 1 2024-04-27T04:10:00.562 INFO:tasks.workunit.client.0.smithi162.stdout:2/174: dwrite d0/d4/d17/f2f [0,4194304] 0 2024-04-27T04:10:00.565 INFO:tasks.workunit.client.0.smithi162.stdout:8/243: creat d0/de/f4b x:0 0 0 2024-04-27T04:10:00.568 INFO:tasks.workunit.client.1.smithi181.stdout:1/509: sync 2024-04-27T04:10:00.568 INFO:tasks.workunit.client.1.smithi181.stdout:2/460: sync 2024-04-27T04:10:00.569 INFO:tasks.workunit.client.1.smithi181.stdout:1/510: fsync d1/d2/d5/d6/f67 0 2024-04-27T04:10:00.569 INFO:tasks.workunit.client.1.smithi181.stdout:2/461: truncate d4/d81/f8e 730878 0 2024-04-27T04:10:00.570 INFO:tasks.workunit.client.1.smithi181.stdout:2/462: chown d4/d9/l31 3 1 2024-04-27T04:10:00.571 INFO:tasks.workunit.client.1.smithi181.stdout:1/511: chown d1/d2/d5/d6/db/l11 175896 1 2024-04-27T04:10:00.571 INFO:tasks.workunit.client.0.smithi162.stdout:4/261: dwrite d2/f4 [0,4194304] 0 2024-04-27T04:10:00.572 INFO:tasks.workunit.client.1.smithi181.stdout:1/512: write d1/d2/d64/f8d [1103368,30888] 0 2024-04-27T04:10:00.579 INFO:tasks.workunit.client.0.smithi162.stdout:3/222: symlink d1/l48 0 2024-04-27T04:10:00.580 INFO:tasks.workunit.client.0.smithi162.stdout:3/223: write d1/d18/f1d [4178947,101729] 0 2024-04-27T04:10:00.588 INFO:tasks.workunit.client.0.smithi162.stdout:0/259: rename d5/dc/d1e/d20/l38 to d5/dc/l59 0 2024-04-27T04:10:00.588 INFO:tasks.workunit.client.0.smithi162.stdout:0/260: write d5/f58 [703669,40574] 0 2024-04-27T04:10:00.589 INFO:tasks.workunit.client.0.smithi162.stdout:0/261: read d5/dc/d1e/f3e [2545812,40196] 0 2024-04-27T04:10:00.590 INFO:tasks.workunit.client.1.smithi181.stdout:0/495: dwrite dc/d1f/d3b/d4a/d95/d26/d41/d6f/f7b [0,4194304] 0 2024-04-27T04:10:00.590 INFO:tasks.workunit.client.1.smithi181.stdout:1/513: creat d1/d2/d5/fa1 x:0 0 0 2024-04-27T04:10:00.591 INFO:tasks.workunit.client.1.smithi181.stdout:1/514: read - d1/d4c/d52/d92/f93 zero size 2024-04-27T04:10:00.593 INFO:tasks.workunit.client.0.smithi162.stdout:4/262: fsync d2/da/f12 0 2024-04-27T04:10:00.594 INFO:tasks.workunit.client.0.smithi162.stdout:5/198: write d8/d9/fc [1552964,57124] 0 2024-04-27T04:10:00.594 INFO:tasks.workunit.client.1.smithi181.stdout:2/463: dread d4/d9/df/d14/f24 [0,4194304] 0 2024-04-27T04:10:00.595 INFO:tasks.workunit.client.0.smithi162.stdout:0/262: dread d5/dc/d1e/d46/f55 [0,4194304] 0 2024-04-27T04:10:00.596 INFO:tasks.workunit.client.0.smithi162.stdout:8/244: dread d0/d10/d25/f30 [0,4194304] 0 2024-04-27T04:10:00.597 INFO:tasks.workunit.client.0.smithi162.stdout:7/168: link d3/f18 d3/d5/d9/dc/f33 0 2024-04-27T04:10:00.599 INFO:tasks.workunit.client.1.smithi181.stdout:0/496: unlink dc/f35 0 2024-04-27T04:10:00.599 INFO:tasks.workunit.client.1.smithi181.stdout:0/497: chown dc/f5e 91210 1 2024-04-27T04:10:00.601 INFO:tasks.workunit.client.0.smithi162.stdout:1/198: dwrite d0/fe [4194304,4194304] 0 2024-04-27T04:10:00.602 INFO:tasks.workunit.client.0.smithi162.stdout:6/231: truncate f5 1956089 0 2024-04-27T04:10:00.603 INFO:tasks.workunit.client.0.smithi162.stdout:3/224: mkdir d1/d18/d23/d34/d49 0 2024-04-27T04:10:00.603 INFO:tasks.workunit.client.1.smithi181.stdout:9/476: sync 2024-04-27T04:10:00.603 INFO:tasks.workunit.client.1.smithi181.stdout:6/374: sync 2024-04-27T04:10:00.603 INFO:tasks.workunit.client.1.smithi181.stdout:4/425: sync 2024-04-27T04:10:00.603 INFO:tasks.workunit.client.1.smithi181.stdout:5/607: sync 2024-04-27T04:10:00.604 INFO:tasks.workunit.client.1.smithi181.stdout:7/461: sync 2024-04-27T04:10:00.604 INFO:tasks.workunit.client.1.smithi181.stdout:3/445: sync 2024-04-27T04:10:00.604 INFO:tasks.workunit.client.1.smithi181.stdout:8/566: sync 2024-04-27T04:10:00.604 INFO:tasks.workunit.client.0.smithi162.stdout:1/199: write d0/d7/d2a/f31 [743617,96955] 0 2024-04-27T04:10:00.604 INFO:tasks.workunit.client.1.smithi181.stdout:7/462: stat d1/d57/f77 0 2024-04-27T04:10:00.605 INFO:tasks.workunit.client.0.smithi162.stdout:3/225: truncate d1/d18/d23/f2b 711770 0 2024-04-27T04:10:00.605 INFO:tasks.workunit.client.1.smithi181.stdout:3/446: chown d0/d28/c2e 642 1 2024-04-27T04:10:00.606 INFO:tasks.workunit.client.1.smithi181.stdout:1/515: mkdir d1/d4c/d1e/d20/da2 0 2024-04-27T04:10:00.607 INFO:tasks.workunit.client.1.smithi181.stdout:7/463: stat d1/d22/d15/d1f/d34/l3b 0 2024-04-27T04:10:00.609 INFO:tasks.workunit.client.1.smithi181.stdout:1/516: fsync d1/d2/d5/f4f 0 2024-04-27T04:10:00.611 INFO:tasks.workunit.client.1.smithi181.stdout:1/517: chown d1/d4c/d1e/d20/f82 16107 1 2024-04-27T04:10:00.611 INFO:tasks.workunit.client.1.smithi181.stdout:7/464: dread d1/df/f5d [0,4194304] 0 2024-04-27T04:10:00.612 INFO:tasks.workunit.client.1.smithi181.stdout:2/464: symlink d4/d9/d4f/d79/l93 0 2024-04-27T04:10:00.613 INFO:tasks.workunit.client.1.smithi181.stdout:7/465: readlink d1/d2/d1c/d2d/l8e 0 2024-04-27T04:10:00.615 INFO:tasks.workunit.client.1.smithi181.stdout:8/567: stat d1/d6/d88/cb2 0 2024-04-27T04:10:00.616 INFO:tasks.workunit.client.1.smithi181.stdout:8/568: write d1/d6/d2f/d40/fb9 [97802,70998] 0 2024-04-27T04:10:00.617 INFO:tasks.workunit.client.1.smithi181.stdout:7/466: read d1/d2/d1c/d37/f4f [2651226,97111] 0 2024-04-27T04:10:00.618 INFO:tasks.workunit.client.0.smithi162.stdout:5/199: creat d8/d1a/f45 x:0 0 0 2024-04-27T04:10:00.618 INFO:tasks.workunit.client.0.smithi162.stdout:5/200: write d8/f22 [3100006,66082] 0 2024-04-27T04:10:00.618 INFO:tasks.workunit.client.1.smithi181.stdout:7/467: write d1/d2/f24 [4696783,54668] 0 2024-04-27T04:10:00.619 INFO:tasks.workunit.client.0.smithi162.stdout:5/201: fsync d8/d1a/d23/d40/f42 0 2024-04-27T04:10:00.626 INFO:tasks.workunit.client.1.smithi181.stdout:7/468: dwrite d1/f45 [4194304,4194304] 0 2024-04-27T04:10:00.627 INFO:tasks.workunit.client.0.smithi162.stdout:8/245: fsync d0/d10/d25/f30 0 2024-04-27T04:10:00.630 INFO:tasks.workunit.client.0.smithi162.stdout:8/246: dread d0/fd [0,4194304] 0 2024-04-27T04:10:00.630 INFO:tasks.workunit.client.0.smithi162.stdout:7/169: fsync d3/d5/d9/dc/f33 0 2024-04-27T04:10:00.636 INFO:tasks.workunit.client.0.smithi162.stdout:7/170: dwrite d3/d5/d9/dc/de/d2b/f2c [0,4194304] 0 2024-04-27T04:10:00.637 INFO:tasks.workunit.client.0.smithi162.stdout:1/200: mknod d0/d7/d10/c3f 0 2024-04-27T04:10:00.637 INFO:tasks.workunit.client.0.smithi162.stdout:1/201: chown d0/fb 0 1 2024-04-27T04:10:00.638 INFO:tasks.workunit.client.0.smithi162.stdout:7/171: write d3/d5/d9/dc/f15 [3574461,8596] 0 2024-04-27T04:10:00.640 INFO:tasks.workunit.client.0.smithi162.stdout:3/226: creat d1/d18/f4a x:0 0 0 2024-04-27T04:10:00.642 INFO:tasks.workunit.client.1.smithi181.stdout:6/375: rmdir d7/dc/d1e/d5c 39 2024-04-27T04:10:00.644 INFO:tasks.workunit.client.0.smithi162.stdout:5/202: dread f4 [4194304,4194304] 0 2024-04-27T04:10:00.645 INFO:tasks.workunit.client.1.smithi181.stdout:8/569: mknod d1/d6/d1a/d25/cc9 0 2024-04-27T04:10:00.646 INFO:tasks.workunit.client.1.smithi181.stdout:9/477: write d0/d44/d4b/f39 [2990850,60418] 0 2024-04-27T04:10:00.646 INFO:tasks.workunit.client.0.smithi162.stdout:9/210: getdents d3/d39 0 2024-04-27T04:10:00.650 INFO:tasks.workunit.client.1.smithi181.stdout:4/426: mknod d0/d15/de/d60/d6f/c8c 0 2024-04-27T04:10:00.650 INFO:tasks.workunit.client.0.smithi162.stdout:9/211: readlink d3/l4c 0 2024-04-27T04:10:00.650 INFO:tasks.workunit.client.0.smithi162.stdout:9/212: read - d3/d13/f41 zero size 2024-04-27T04:10:00.653 INFO:tasks.workunit.client.1.smithi181.stdout:0/498: link dc/d1f/d3b/d4a/d95/d26/l31 dc/d1f/d3b/d75/lb6 0 2024-04-27T04:10:00.656 INFO:tasks.workunit.client.1.smithi181.stdout:4/427: dwrite d0/d15/de/d19/d77/f68 [0,4194304] 0 2024-04-27T04:10:00.656 INFO:tasks.workunit.client.0.smithi162.stdout:8/247: symlink d0/d10/d2b/d2e/l4c 0 2024-04-27T04:10:00.658 INFO:tasks.workunit.client.1.smithi181.stdout:4/428: write d0/d15/d5c/f7d [539559,74605] 0 2024-04-27T04:10:00.663 INFO:tasks.workunit.client.1.smithi181.stdout:3/447: truncate d0/f57 1821013 0 2024-04-27T04:10:00.663 INFO:tasks.workunit.client.1.smithi181.stdout:3/448: readlink d0/d6c/d52/d44/d56/d5b/l6f 0 2024-04-27T04:10:00.665 INFO:tasks.workunit.client.0.smithi162.stdout:4/263: truncate d2/f4 917739 0 2024-04-27T04:10:00.666 INFO:tasks.workunit.client.1.smithi181.stdout:6/376: creat d7/d31/d38/f80 x:0 0 0 2024-04-27T04:10:00.669 INFO:tasks.workunit.client.0.smithi162.stdout:4/264: dread d2/d25/d1c/d39/f45 [0,4194304] 0 2024-04-27T04:10:00.671 INFO:tasks.workunit.client.0.smithi162.stdout:7/172: symlink d3/d5/d9/dc/de/l34 0 2024-04-27T04:10:00.672 INFO:tasks.workunit.client.0.smithi162.stdout:3/227: chown d1/fa 0 1 2024-04-27T04:10:00.673 INFO:tasks.workunit.client.0.smithi162.stdout:3/228: write d1/d18/d23/f2e [844954,79072] 0 2024-04-27T04:10:00.673 INFO:tasks.workunit.client.0.smithi162.stdout:6/232: truncate d6/f26 2531246 0 2024-04-27T04:10:00.675 INFO:tasks.workunit.client.0.smithi162.stdout:6/233: read d6/db/d32/f45 [42481,20129] 0 2024-04-27T04:10:00.677 INFO:tasks.workunit.client.0.smithi162.stdout:2/175: rename d0/d4/d17/l18 to d0/l34 0 2024-04-27T04:10:00.678 INFO:tasks.workunit.client.0.smithi162.stdout:2/176: write d0/fe [3897043,90735] 0 2024-04-27T04:10:00.678 INFO:tasks.workunit.client.1.smithi181.stdout:2/465: truncate d4/d9/df/f41 1662030 0 2024-04-27T04:10:00.680 INFO:tasks.workunit.client.0.smithi162.stdout:5/203: creat d8/d1a/d23/d40/f46 x:0 0 0 2024-04-27T04:10:00.680 INFO:tasks.workunit.client.0.smithi162.stdout:5/204: read - d8/d1a/f31 zero size 2024-04-27T04:10:00.683 INFO:tasks.workunit.client.1.smithi181.stdout:5/608: link d4/de/d2c/d46/d99/dad/dca/fcc d4/de/d2c/d46/d4d/d8e/fdc 0 2024-04-27T04:10:00.685 INFO:tasks.workunit.client.1.smithi181.stdout:8/570: write d1/d6/d2f/f45 [948094,29698] 0 2024-04-27T04:10:00.686 INFO:tasks.workunit.client.0.smithi162.stdout:5/205: dwrite d8/d1a/f2d [0,4194304] 0 2024-04-27T04:10:00.690 INFO:tasks.workunit.client.1.smithi181.stdout:3/449: mkdir d0/d6c/d52/d44/d90 0 2024-04-27T04:10:00.694 INFO:tasks.workunit.client.1.smithi181.stdout:9/478: dwrite d0/d44/d4b/d50/f6a [0,4194304] 0 2024-04-27T04:10:00.696 INFO:tasks.workunit.client.1.smithi181.stdout:9/479: dread - d0/d44/d4b/f83 zero size 2024-04-27T04:10:00.697 INFO:tasks.workunit.client.0.smithi162.stdout:8/248: stat d0/de/d20/l21 0 2024-04-27T04:10:00.697 INFO:tasks.workunit.client.0.smithi162.stdout:8/249: dread - d0/d10/d2b/d2e/f40 zero size 2024-04-27T04:10:00.699 INFO:tasks.workunit.client.1.smithi181.stdout:7/469: truncate d1/d2/d1c/d2d/f41 790334 0 2024-04-27T04:10:00.701 INFO:tasks.workunit.client.1.smithi181.stdout:7/470: truncate d1/d22/d15/d29/f84 909738 0 2024-04-27T04:10:00.705 INFO:tasks.workunit.client.1.smithi181.stdout:0/499: mknod dc/d1f/d3b/cb7 0 2024-04-27T04:10:00.712 INFO:tasks.workunit.client.0.smithi162.stdout:3/229: unlink d1/d18/d23/l29 0 2024-04-27T04:10:00.713 INFO:tasks.workunit.client.1.smithi181.stdout:1/518: getdents d1/d4c/d1e/d20/d53 0 2024-04-27T04:10:00.714 INFO:tasks.workunit.client.1.smithi181.stdout:1/519: stat d1/d2/d5/d6/db/d2d/d36/f5e 0 2024-04-27T04:10:00.715 INFO:tasks.workunit.client.0.smithi162.stdout:6/234: mkdir d6/d1a/d34/d50 0 2024-04-27T04:10:00.717 INFO:tasks.workunit.client.0.smithi162.stdout:9/213: dwrite d3/fc [0,4194304] 0 2024-04-27T04:10:00.718 INFO:tasks.workunit.client.0.smithi162.stdout:9/214: write d3/d23/f4b [1040184,117762] 0 2024-04-27T04:10:00.719 INFO:tasks.workunit.client.1.smithi181.stdout:4/429: link d0/d15/de/d19/d77/f72 d0/d15/d25/d2a/d7a/f8d 0 2024-04-27T04:10:00.720 INFO:tasks.workunit.client.0.smithi162.stdout:9/215: read d3/f5 [5468220,61784] 0 2024-04-27T04:10:00.725 INFO:tasks.workunit.client.1.smithi181.stdout:6/377: symlink d7/dc/d1e/d66/l81 0 2024-04-27T04:10:00.727 INFO:tasks.workunit.client.1.smithi181.stdout:6/378: truncate d7/dc/d1e/d2b/f5a 58371 0 2024-04-27T04:10:00.727 INFO:tasks.workunit.client.1.smithi181.stdout:3/450: rmdir d0/d6c/d52/d44/d56 39 2024-04-27T04:10:00.729 INFO:tasks.workunit.client.1.smithi181.stdout:2/466: write d4/d9/d16/d82/f71 [671529,16015] 0 2024-04-27T04:10:00.731 INFO:tasks.workunit.client.1.smithi181.stdout:9/480: chown d0/d40/f78 59 1 2024-04-27T04:10:00.732 INFO:tasks.workunit.client.1.smithi181.stdout:9/481: write d0/d44/d4b/f83 [73160,17886] 0 2024-04-27T04:10:00.733 INFO:tasks.workunit.client.0.smithi162.stdout:0/263: getdents d5/d26 0 2024-04-27T04:10:00.733 INFO:tasks.workunit.client.0.smithi162.stdout:0/264: stat d5/dc/d1e/d46/d48 0 2024-04-27T04:10:00.734 INFO:tasks.workunit.client.1.smithi181.stdout:9/482: readlink d0/d44/d4b/d3d/l8c 0 2024-04-27T04:10:00.734 INFO:tasks.workunit.client.0.smithi162.stdout:0/265: chown d5/f10 234250742 1 2024-04-27T04:10:00.734 INFO:tasks.workunit.client.0.smithi162.stdout:0/266: read - d5/f11 zero size 2024-04-27T04:10:00.738 INFO:tasks.workunit.client.1.smithi181.stdout:2/467: dwrite d4/d9/f8c [0,4194304] 0 2024-04-27T04:10:00.741 INFO:tasks.workunit.client.1.smithi181.stdout:2/468: write d4/f32 [1615863,2987] 0 2024-04-27T04:10:00.742 INFO:tasks.workunit.client.1.smithi181.stdout:0/500: read dc/f22 [4129742,78090] 0 2024-04-27T04:10:00.746 INFO:tasks.workunit.client.1.smithi181.stdout:8/571: dwrite d1/d6/d2f/d47/f86 [0,4194304] 0 2024-04-27T04:10:00.751 INFO:tasks.workunit.client.0.smithi162.stdout:3/230: rename d1/d21/l36 to d1/d45/l4b 0 2024-04-27T04:10:00.751 INFO:tasks.workunit.client.0.smithi162.stdout:8/250: write d0/d10/d25/f36 [1352640,6083] 0 2024-04-27T04:10:00.757 INFO:tasks.workunit.client.1.smithi181.stdout:7/471: rename d1/l83 to d1/d2/d1c/d2d/d61/l92 0 2024-04-27T04:10:00.759 INFO:tasks.workunit.client.0.smithi162.stdout:3/231: dwrite d1/d18/f1d [0,4194304] 0 2024-04-27T04:10:00.760 INFO:tasks.workunit.client.0.smithi162.stdout:3/232: read - d1/d18/d23/f28 zero size 2024-04-27T04:10:00.760 INFO:tasks.workunit.client.0.smithi162.stdout:3/233: write d1/d18/f12 [4700633,17859] 0 2024-04-27T04:10:00.764 INFO:tasks.workunit.client.0.smithi162.stdout:0/267: dread d5/f9 [4194304,4194304] 0 2024-04-27T04:10:00.765 INFO:tasks.workunit.client.1.smithi181.stdout:2/469: rmdir d4/d9/df 39 2024-04-27T04:10:00.766 INFO:tasks.workunit.client.1.smithi181.stdout:2/470: chown d4/f50 890 1 2024-04-27T04:10:00.769 INFO:tasks.workunit.client.0.smithi162.stdout:1/202: link d0/d7/d10/l15 d0/d7/l40 0 2024-04-27T04:10:00.770 INFO:tasks.workunit.client.1.smithi181.stdout:8/572: mknod d1/d6/d88/cca 0 2024-04-27T04:10:00.774 INFO:tasks.workunit.client.1.smithi181.stdout:6/379: mknod d7/dc/d1c/c82 0 2024-04-27T04:10:00.777 INFO:tasks.workunit.client.1.smithi181.stdout:8/573: dwrite d1/d6/d2f/d47/f8e [0,4194304] 0 2024-04-27T04:10:00.778 INFO:tasks.workunit.client.1.smithi181.stdout:8/574: chown d1/d6/d2f/l69 0 1 2024-04-27T04:10:00.780 INFO:tasks.workunit.client.0.smithi162.stdout:9/216: rename d3/f1f to d3/d11/f4d 0 2024-04-27T04:10:00.785 INFO:tasks.workunit.client.1.smithi181.stdout:7/472: mkdir d1/d22/d15/d5f/d93 0 2024-04-27T04:10:00.787 INFO:tasks.workunit.client.1.smithi181.stdout:5/609: link d4/de/d2f/d7a/d90/cd0 d4/d10/d29/d92/cdd 0 2024-04-27T04:10:00.790 INFO:tasks.workunit.client.1.smithi181.stdout:3/451: truncate d0/de/d3f/f75 88014 0 2024-04-27T04:10:00.790 INFO:tasks.workunit.client.0.smithi162.stdout:5/206: link d8/ld d8/d1a/d23/l47 0 2024-04-27T04:10:00.791 INFO:tasks.workunit.client.1.smithi181.stdout:3/452: chown d0/f23 7747 1 2024-04-27T04:10:00.791 INFO:tasks.workunit.client.0.smithi162.stdout:4/265: sync 2024-04-27T04:10:00.791 INFO:tasks.workunit.client.0.smithi162.stdout:6/235: dwrite d6/f26 [0,4194304] 0 2024-04-27T04:10:00.793 INFO:tasks.workunit.client.1.smithi181.stdout:3/453: write d0/de/f7f [1180383,35979] 0 2024-04-27T04:10:00.793 INFO:tasks.workunit.client.1.smithi181.stdout:6/380: dread d7/dc/d1e/f4d [0,4194304] 0 2024-04-27T04:10:00.793 INFO:tasks.workunit.client.0.smithi162.stdout:3/234: rmdir d1/d18/d23/d34 39 2024-04-27T04:10:00.795 INFO:tasks.workunit.client.1.smithi181.stdout:8/575: creat d1/d6/d2f/d30/d5c/fcb x:0 0 0 2024-04-27T04:10:00.795 INFO:tasks.workunit.client.0.smithi162.stdout:3/235: truncate d1/d18/f4a 1018772 0 2024-04-27T04:10:00.796 INFO:tasks.workunit.client.1.smithi181.stdout:5/610: dwrite d4/da3/fa7 [0,4194304] 0 2024-04-27T04:10:00.797 INFO:tasks.workunit.client.1.smithi181.stdout:8/576: stat d1/d6/d2f/d30/d66/dc0 0 2024-04-27T04:10:00.797 INFO:tasks.workunit.client.1.smithi181.stdout:9/483: creat d0/fa7 x:0 0 0 2024-04-27T04:10:00.797 INFO:tasks.workunit.client.0.smithi162.stdout:3/236: write d1/d18/d23/f28 [330778,27569] 0 2024-04-27T04:10:00.797 INFO:tasks.workunit.client.0.smithi162.stdout:0/268: rmdir d5/d26 39 2024-04-27T04:10:00.798 INFO:tasks.workunit.client.0.smithi162.stdout:3/237: stat d1/d18/d23/f2e 0 2024-04-27T04:10:00.799 INFO:tasks.workunit.client.0.smithi162.stdout:4/266: dwrite d2/d25/f2d [0,4194304] 0 2024-04-27T04:10:00.800 INFO:tasks.workunit.client.0.smithi162.stdout:4/267: readlink d2/d25/d1c/d39/l4f 0 2024-04-27T04:10:00.801 INFO:tasks.workunit.client.0.smithi162.stdout:4/268: read d2/d25/d1c/d39/f45 [1756704,10286] 0 2024-04-27T04:10:00.802 INFO:tasks.workunit.client.1.smithi181.stdout:4/430: rename d0/d15/d25/d2a/d61/l78 to d0/d15/de/d60/d6f/l8e 0 2024-04-27T04:10:00.807 INFO:tasks.workunit.client.0.smithi162.stdout:7/173: getdents d3 0 2024-04-27T04:10:00.808 INFO:tasks.workunit.client.0.smithi162.stdout:7/174: fsync d3/d5/d9/dc/de/f30 0 2024-04-27T04:10:00.815 INFO:tasks.workunit.client.1.smithi181.stdout:6/381: unlink d7/dc/d1c/d34/f5d 0 2024-04-27T04:10:00.816 INFO:tasks.workunit.client.1.smithi181.stdout:6/382: chown d7/dc/d1c/d7f/c1d 287936 1 2024-04-27T04:10:00.816 INFO:tasks.workunit.client.0.smithi162.stdout:8/251: mknod d0/d10/d44/c4d 0 2024-04-27T04:10:00.817 INFO:tasks.workunit.client.1.smithi181.stdout:6/383: dread - d7/dc/d1c/f78 zero size 2024-04-27T04:10:00.819 INFO:tasks.workunit.client.1.smithi181.stdout:8/577: rmdir d1/d6/d2f 39 2024-04-27T04:10:00.820 INFO:tasks.workunit.client.1.smithi181.stdout:8/578: fdatasync d1/f46 0 2024-04-27T04:10:00.822 INFO:tasks.workunit.client.0.smithi162.stdout:2/177: getdents d0/d24 0 2024-04-27T04:10:00.822 INFO:tasks.workunit.client.0.smithi162.stdout:2/178: chown d0 79005 1 2024-04-27T04:10:00.824 INFO:tasks.workunit.client.0.smithi162.stdout:5/207: creat d8/d9/f48 x:0 0 0 2024-04-27T04:10:00.825 INFO:tasks.workunit.client.1.smithi181.stdout:9/484: mknod d0/d4d/d85/ca8 0 2024-04-27T04:10:00.827 INFO:tasks.workunit.client.0.smithi162.stdout:6/236: creat d6/db/d32/f51 x:0 0 0 2024-04-27T04:10:00.827 INFO:tasks.workunit.client.1.smithi181.stdout:7/473: dwrite d1/d2/f25 [0,4194304] 0 2024-04-27T04:10:00.827 INFO:tasks.workunit.client.0.smithi162.stdout:6/237: dread - d6/f1e zero size 2024-04-27T04:10:00.829 INFO:tasks.workunit.client.1.smithi181.stdout:0/501: rename dc/d1f/d3b/d4a/f62 to dc/d1f/d3b/d75/db3/fb8 0 2024-04-27T04:10:00.832 INFO:tasks.workunit.client.1.smithi181.stdout:0/502: dread dc/f15 [0,4194304] 0 2024-04-27T04:10:00.832 INFO:tasks.workunit.client.0.smithi162.stdout:0/269: unlink d5/f10 0 2024-04-27T04:10:00.836 INFO:tasks.workunit.client.0.smithi162.stdout:1/203: creat d0/d11/d3e/f41 x:0 0 0 2024-04-27T04:10:00.838 INFO:tasks.workunit.client.0.smithi162.stdout:4/269: creat d2/da/dd/f57 x:0 0 0 2024-04-27T04:10:00.838 INFO:tasks.workunit.client.0.smithi162.stdout:4/270: chown d2/d25/c1d 610685074 1 2024-04-27T04:10:00.842 INFO:tasks.workunit.client.0.smithi162.stdout:7/175: creat d3/d5/d9/dc/f35 x:0 0 0 2024-04-27T04:10:00.843 INFO:tasks.workunit.client.1.smithi181.stdout:5/611: symlink d4/de/d2c/d46/lde 0 2024-04-27T04:10:00.844 INFO:tasks.workunit.client.1.smithi181.stdout:5/612: readlink d4/de/d2c/d46/d4d/d8e/d67/l35 0 2024-04-27T04:10:00.845 INFO:tasks.workunit.client.1.smithi181.stdout:5/613: readlink d4/de/d2f/l76 0 2024-04-27T04:10:00.845 INFO:tasks.workunit.client.0.smithi162.stdout:8/252: mknod d0/d10/d2b/d2d/d2f/c4e 0 2024-04-27T04:10:00.850 INFO:tasks.workunit.client.0.smithi162.stdout:5/208: rename f5 to d8/d1a/d29/f49 0 2024-04-27T04:10:00.851 INFO:tasks.workunit.client.0.smithi162.stdout:6/238: mknod d6/d1a/d34/d3a/c52 0 2024-04-27T04:10:00.854 INFO:tasks.workunit.client.1.smithi181.stdout:7/474: unlink d1/d2/d1c/f2c 0 2024-04-27T04:10:00.854 INFO:tasks.workunit.client.0.smithi162.stdout:0/270: dread d5/f9 [0,4194304] 0 2024-04-27T04:10:00.857 INFO:tasks.workunit.client.1.smithi181.stdout:2/471: rename d4/d9/d44/d63 to d4/d9/d4f/d79/d94 0 2024-04-27T04:10:00.859 INFO:tasks.workunit.client.1.smithi181.stdout:4/431: mkdir d0/d15/d25/d8f 0 2024-04-27T04:10:00.860 INFO:tasks.workunit.client.0.smithi162.stdout:4/271: mknod d2/d54/c58 0 2024-04-27T04:10:00.861 INFO:tasks.workunit.client.0.smithi162.stdout:7/176: dread - d3/f18 zero size 2024-04-27T04:10:00.862 INFO:tasks.workunit.client.0.smithi162.stdout:7/177: dread - d3/f18 zero size 2024-04-27T04:10:00.863 INFO:tasks.workunit.client.1.smithi181.stdout:5/614: rmdir d4/de/d2c/d46 39 2024-04-27T04:10:00.866 INFO:tasks.workunit.client.0.smithi162.stdout:7/178: dread d3/d5/d9/f13 [0,4194304] 0 2024-04-27T04:10:00.868 INFO:tasks.workunit.client.0.smithi162.stdout:5/209: mkdir d8/d1a/d29/d4a 0 2024-04-27T04:10:00.869 INFO:tasks.workunit.client.0.smithi162.stdout:5/210: write d8/d9/f48 [750564,78381] 0 2024-04-27T04:10:00.870 INFO:tasks.workunit.client.1.smithi181.stdout:2/472: creat d4/d9/d4f/d79/d94/f95 x:0 0 0 2024-04-27T04:10:00.871 INFO:tasks.workunit.client.0.smithi162.stdout:0/271: dread d5/dc/d1e/f21 [4194304,4194304] 0 2024-04-27T04:10:00.874 INFO:tasks.workunit.client.0.smithi162.stdout:6/239: dread d6/f2d [0,4194304] 0 2024-04-27T04:10:00.875 INFO:tasks.workunit.client.0.smithi162.stdout:3/238: rename d1/fa to d1/f4c 0 2024-04-27T04:10:00.877 INFO:tasks.workunit.client.1.smithi181.stdout:6/384: creat d7/dc/d1e/f83 x:0 0 0 2024-04-27T04:10:00.879 INFO:tasks.workunit.client.0.smithi162.stdout:4/272: creat d2/d25/d1c/d39/f59 x:0 0 0 2024-04-27T04:10:00.882 INFO:tasks.workunit.client.0.smithi162.stdout:9/217: getdents d3 0 2024-04-27T04:10:00.882 INFO:tasks.workunit.client.0.smithi162.stdout:8/253: write d0/d10/d44/f47 [1175643,51594] 0 2024-04-27T04:10:00.883 INFO:tasks.workunit.client.0.smithi162.stdout:9/218: read - d3/f2a zero size 2024-04-27T04:10:00.885 INFO:tasks.workunit.client.1.smithi181.stdout:6/385: dwrite d7/d27/f4c [4194304,4194304] 0 2024-04-27T04:10:00.886 INFO:tasks.workunit.client.1.smithi181.stdout:6/386: write d7/d31/d45/f70 [826861,69113] 0 2024-04-27T04:10:00.888 INFO:tasks.workunit.client.1.smithi181.stdout:2/473: mknod d4/d9/d16/d45/c96 0 2024-04-27T04:10:00.890 INFO:tasks.workunit.client.0.smithi162.stdout:2/179: dwrite d0/d4/f22 [0,4194304] 0 2024-04-27T04:10:00.891 INFO:tasks.workunit.client.0.smithi162.stdout:9/219: dwrite d3/d13/d15/f2f [4194304,4194304] 0 2024-04-27T04:10:00.891 INFO:tasks.workunit.client.0.smithi162.stdout:5/211: unlink d8/d1a/l1d 0 2024-04-27T04:10:00.893 INFO:tasks.workunit.client.0.smithi162.stdout:5/212: read d8/d1a/d23/f28 [2902971,104609] 0 2024-04-27T04:10:00.894 INFO:tasks.workunit.client.0.smithi162.stdout:5/213: write d8/d1a/f45 [631322,18517] 0 2024-04-27T04:10:00.900 INFO:tasks.workunit.client.1.smithi181.stdout:5/615: write d4/de/f6f [251369,117639] 0 2024-04-27T04:10:00.902 INFO:tasks.workunit.client.0.smithi162.stdout:6/240: unlink d6/d1a/d34/d3a/l42 0 2024-04-27T04:10:00.904 INFO:tasks.workunit.client.1.smithi181.stdout:8/579: getdents d1/d20/da4 0 2024-04-27T04:10:00.907 INFO:tasks.workunit.client.1.smithi181.stdout:4/432: write d0/d15/f50 [745099,73135] 0 2024-04-27T04:10:00.908 INFO:tasks.workunit.client.0.smithi162.stdout:7/179: dwrite d3/d5/d9/dc/f33 [0,4194304] 0 2024-04-27T04:10:00.908 INFO:tasks.workunit.client.0.smithi162.stdout:1/204: creat d0/d11/f42 x:0 0 0 2024-04-27T04:10:00.914 INFO:tasks.workunit.client.0.smithi162.stdout:7/180: dread d3/d5/d9/f24 [0,4194304] 0 2024-04-27T04:10:00.914 INFO:tasks.workunit.client.1.smithi181.stdout:2/474: creat d4/d9/d44/f97 x:0 0 0 2024-04-27T04:10:00.915 INFO:tasks.workunit.client.0.smithi162.stdout:9/220: fsync d3/f6 0 2024-04-27T04:10:00.915 INFO:tasks.workunit.client.0.smithi162.stdout:7/181: dread - d3/d5/d9/dc/f27 zero size 2024-04-27T04:10:00.915 INFO:tasks.workunit.client.1.smithi181.stdout:0/503: getdents dc/d1f/d3b/d4a/d7a 0 2024-04-27T04:10:00.916 INFO:tasks.workunit.client.0.smithi162.stdout:9/221: write d3/fc [4453193,38975] 0 2024-04-27T04:10:00.916 INFO:tasks.workunit.client.1.smithi181.stdout:0/504: readlink dc/d54/l76 0 2024-04-27T04:10:00.918 INFO:tasks.workunit.client.0.smithi162.stdout:2/180: rename d0/d4/c23 to d0/d4/d17/d2b/c35 0 2024-04-27T04:10:00.919 INFO:tasks.workunit.client.0.smithi162.stdout:2/181: read d0/d4/f9 [3443369,44450] 0 2024-04-27T04:10:00.919 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:10:00 smithi181 ceph-mon[33774]: overall HEALTH_OK 2024-04-27T04:10:00.920 INFO:tasks.workunit.client.1.smithi181.stdout:5/616: fsync d4/da9/fc2 0 2024-04-27T04:10:00.924 INFO:tasks.workunit.client.1.smithi181.stdout:8/580: write d1/d6/d2f/d40/f27 [2330849,31701] 0 2024-04-27T04:10:00.925 INFO:tasks.workunit.client.0.smithi162.stdout:5/214: mkdir d8/d1a/d23/d40/d4b 0 2024-04-27T04:10:00.927 INFO:tasks.workunit.client.0.smithi162.stdout:0/272: mknod d5/d26/d4a/c5a 0 2024-04-27T04:10:00.927 INFO:tasks.workunit.client.0.smithi162.stdout:0/273: dread - d5/dc/d1e/d41/f4e zero size 2024-04-27T04:10:00.928 INFO:tasks.workunit.client.1.smithi181.stdout:7/475: getdents d1/d2/d1c/d2d/d61 0 2024-04-27T04:10:00.929 INFO:tasks.workunit.client.1.smithi181.stdout:7/476: fsync d1/d22/f65 0 2024-04-27T04:10:00.930 INFO:tasks.workunit.client.1.smithi181.stdout:9/485: rename d0/cb to d0/d44/ca9 0 2024-04-27T04:10:00.932 INFO:tasks.workunit.client.0.smithi162.stdout:3/239: chown d1/d18/d23/d34/c3a 235 1 2024-04-27T04:10:00.934 INFO:tasks.workunit.client.1.smithi181.stdout:2/475: mkdir d4/d9/d4f/d79/d98 0 2024-04-27T04:10:00.939 INFO:tasks.workunit.client.0.smithi162.stdout:4/273: fsync f1 0 2024-04-27T04:10:00.939 INFO:tasks.workunit.client.1.smithi181.stdout:0/505: dread dc/f22 [0,4194304] 0 2024-04-27T04:10:00.940 INFO:tasks.workunit.client.0.smithi162.stdout:8/254: symlink d0/de/l4f 0 2024-04-27T04:10:00.945 INFO:tasks.workunit.client.0.smithi162.stdout:7/182: rmdir d3/d1d 39 2024-04-27T04:10:00.946 INFO:tasks.workunit.client.1.smithi181.stdout:7/477: unlink d1/d2/d1c/l48 0 2024-04-27T04:10:00.951 INFO:tasks.workunit.client.1.smithi181.stdout:6/387: creat d7/dc/d1c/d34/f84 x:0 0 0 2024-04-27T04:10:00.951 INFO:tasks.workunit.client.0.smithi162.stdout:2/182: chown d0/l34 3559 1 2024-04-27T04:10:00.953 INFO:tasks.workunit.client.1.smithi181.stdout:8/581: dwrite d1/d6/d2f/d30/d5c/f97 [0,4194304] 0 2024-04-27T04:10:00.956 INFO:tasks.workunit.client.0.smithi162.stdout:8/255: dread d0/d10/d2b/f19 [0,4194304] 0 2024-04-27T04:10:00.957 INFO:tasks.workunit.client.0.smithi162.stdout:2/183: dwrite d0/d4/f22 [0,4194304] 0 2024-04-27T04:10:00.958 INFO:tasks.workunit.client.0.smithi162.stdout:2/184: fsync d0/d4/f16 0 2024-04-27T04:10:00.958 INFO:tasks.workunit.client.1.smithi181.stdout:5/617: truncate d4/de/f6e 2917697 0 2024-04-27T04:10:00.959 INFO:tasks.workunit.client.0.smithi162.stdout:5/215: creat d8/d2c/f4c x:0 0 0 2024-04-27T04:10:00.960 INFO:tasks.workunit.client.0.smithi162.stdout:6/241: creat d6/d1a/d34/d46/f53 x:0 0 0 2024-04-27T04:10:00.961 INFO:tasks.workunit.client.0.smithi162.stdout:5/216: chown d8/d1a/d29/f49 14 1 2024-04-27T04:10:00.961 INFO:tasks.workunit.client.1.smithi181.stdout:4/433: symlink d0/d15/d25/d2b/l90 0 2024-04-27T04:10:00.962 INFO:tasks.workunit.client.0.smithi162.stdout:5/217: chown d8/d2c/f33 38178127 1 2024-04-27T04:10:00.963 INFO:tasks.workunit.client.1.smithi181.stdout:4/434: write d0/d15/de/d19/d59/f79 [197638,50203] 0 2024-04-27T04:10:00.963 INFO:tasks.workunit.client.0.smithi162.stdout:0/274: rename d5/dc/d1e/d20/c29 to d5/dc/d1e/d46/d48/c5b 0 2024-04-27T04:10:00.965 INFO:tasks.workunit.client.0.smithi162.stdout:5/218: read - d8/d1a/d23/d40/f46 zero size 2024-04-27T04:10:00.966 INFO:tasks.workunit.client.0.smithi162.stdout:8/256: dwrite d0/d10/d2b/d23/f31 [0,4194304] 0 2024-04-27T04:10:00.967 INFO:tasks.workunit.client.0.smithi162.stdout:0/275: rename d5/dc to d5/dc/d1e/d20/d5c 22 2024-04-27T04:10:00.970 INFO:tasks.workunit.client.0.smithi162.stdout:1/205: creat d0/d7/d3c/f43 x:0 0 0 2024-04-27T04:10:00.970 INFO:tasks.workunit.client.1.smithi181.stdout:9/486: mknod d0/caa 0 2024-04-27T04:10:00.972 INFO:tasks.workunit.client.0.smithi162.stdout:6/242: dwrite d6/d1a/f47 [0,4194304] 0 2024-04-27T04:10:00.972 INFO:tasks.workunit.client.0.smithi162.stdout:0/276: dread - d5/dc/d1e/d46/f52 zero size 2024-04-27T04:10:00.974 INFO:tasks.workunit.client.1.smithi181.stdout:2/476: creat d4/d9/d4f/d79/d98/f99 x:0 0 0 2024-04-27T04:10:00.975 INFO:tasks.workunit.client.0.smithi162.stdout:8/257: write d0/d10/d2b/f19 [2590782,10573] 0 2024-04-27T04:10:00.977 INFO:tasks.workunit.client.0.smithi162.stdout:1/206: write d0/d7/d10/d35/f28 [337240,122041] 0 2024-04-27T04:10:00.979 INFO:tasks.workunit.client.1.smithi181.stdout:2/477: dread d4/d9/d4f/d79/d94/f49 [0,4194304] 0 2024-04-27T04:10:00.983 INFO:tasks.workunit.client.1.smithi181.stdout:4/435: unlink d0/d15/de/d19/d77/f68 0 2024-04-27T04:10:00.985 INFO:tasks.workunit.client.1.smithi181.stdout:4/436: write d0/d15/de/d19/f5e [654558,87205] 0 2024-04-27T04:10:00.988 INFO:tasks.workunit.client.1.smithi181.stdout:7/478: getdents d1/df/d86 0 2024-04-27T04:10:00.989 INFO:tasks.workunit.client.1.smithi181.stdout:4/437: dread d0/f3b [0,4194304] 0 2024-04-27T04:10:00.993 INFO:tasks.workunit.client.1.smithi181.stdout:6/388: write f6 [1303237,6929] 0 2024-04-27T04:10:00.993 INFO:tasks.workunit.client.1.smithi181.stdout:0/506: dwrite dc/d1f/d6b/fa9 [0,4194304] 0 2024-04-27T04:10:00.993 INFO:tasks.workunit.client.0.smithi162.stdout:3/240: dwrite d1/d21/f2c [0,4194304] 0 2024-04-27T04:10:00.994 INFO:tasks.workunit.client.0.smithi162.stdout:4/274: mkdir d2/d5a 0 2024-04-27T04:10:00.997 INFO:tasks.workunit.client.1.smithi181.stdout:8/582: write d1/d20/f51 [4668590,38728] 0 2024-04-27T04:10:01.002 INFO:tasks.workunit.client.0.smithi162.stdout:6/243: creat d6/d1a/d34/f54 x:0 0 0 2024-04-27T04:10:01.006 INFO:tasks.workunit.client.1.smithi181.stdout:9/487: mknod d0/d44/d4f/d9d/cab 0 2024-04-27T04:10:01.009 INFO:tasks.workunit.client.0.smithi162.stdout:8/258: rename d0/d10/d2b/d2d/d2f/d39/f33 to d0/de/d20/f50 0 2024-04-27T04:10:01.009 INFO:tasks.workunit.client.1.smithi181.stdout:9/488: read d0/d44/f51 [4062406,3531] 0 2024-04-27T04:10:01.011 INFO:tasks.workunit.client.1.smithi181.stdout:5/618: dwrite d4/de/d2c/d46/d4d/f66 [0,4194304] 0 2024-04-27T04:10:01.012 INFO:tasks.workunit.client.0.smithi162.stdout:3/241: creat d1/d18/f4d x:0 0 0 2024-04-27T04:10:01.013 INFO:tasks.workunit.client.1.smithi181.stdout:0/507: fdatasync dc/d1f/f29 0 2024-04-27T04:10:01.015 INFO:tasks.workunit.client.1.smithi181.stdout:6/389: creat d7/dc/d1e/d2b/f85 x:0 0 0 2024-04-27T04:10:01.018 INFO:tasks.workunit.client.0.smithi162.stdout:0/277: mknod d5/dc/d1e/c5d 0 2024-04-27T04:10:01.018 INFO:tasks.workunit.client.0.smithi162.stdout:6/244: read d6/f35 [1509362,19936] 0 2024-04-27T04:10:01.020 INFO:tasks.workunit.client.0.smithi162.stdout:1/207: mkdir d0/d11/d44 0 2024-04-27T04:10:01.021 INFO:tasks.workunit.client.0.smithi162.stdout:1/208: dread - d0/d7/d3c/f43 zero size 2024-04-27T04:10:01.025 INFO:tasks.workunit.client.0.smithi162.stdout:8/259: creat d0/d10/d2b/d23/f51 x:0 0 0 2024-04-27T04:10:01.026 INFO:tasks.workunit.client.1.smithi181.stdout:7/479: symlink d1/d2/d1c/d2d/l94 0 2024-04-27T04:10:01.028 INFO:tasks.workunit.client.0.smithi162.stdout:2/185: rmdir d0/d4/d17/d31 0 2024-04-27T04:10:01.029 INFO:tasks.workunit.client.1.smithi181.stdout:3/454: sync 2024-04-27T04:10:01.029 INFO:tasks.workunit.client.1.smithi181.stdout:1/520: sync 2024-04-27T04:10:01.030 INFO:tasks.workunit.client.1.smithi181.stdout:8/583: write d1/d6/d2f/d30/d5c/f61 [4669374,50605] 0 2024-04-27T04:10:01.031 INFO:tasks.workunit.client.0.smithi162.stdout:4/275: mknod d2/d5a/c5b 0 2024-04-27T04:10:01.031 INFO:tasks.workunit.client.0.smithi162.stdout:4/276: chown f1 619686 1 2024-04-27T04:10:01.032 INFO:tasks.workunit.client.1.smithi181.stdout:8/584: chown d1/d6/d2f/d40/d2a/l54 53468 1 2024-04-27T04:10:01.033 INFO:tasks.workunit.client.1.smithi181.stdout:9/489: unlink d0/d44/d4b/d43/d88/da1/l61 0 2024-04-27T04:10:01.033 INFO:tasks.workunit.client.0.smithi162.stdout:7/183: mknod d3/d1d/d2d/c36 0 2024-04-27T04:10:01.034 INFO:tasks.workunit.client.1.smithi181.stdout:5/619: rmdir d4/de/d2f/d7a 39 2024-04-27T04:10:01.035 INFO:tasks.workunit.client.0.smithi162.stdout:0/278: dread d5/fa [0,4194304] 0 2024-04-27T04:10:01.037 INFO:tasks.workunit.client.1.smithi181.stdout:9/490: dread d0/f1f [4194304,4194304] 0 2024-04-27T04:10:01.037 INFO:tasks.workunit.client.0.smithi162.stdout:3/242: dread d1/d18/d23/f2e [0,4194304] 0 2024-04-27T04:10:01.037 INFO:tasks.workunit.client.0.smithi162.stdout:0/279: dread - d5/dc/d1e/d41/f4e zero size 2024-04-27T04:10:01.040 INFO:tasks.workunit.client.0.smithi162.stdout:0/280: dread d5/f58 [0,4194304] 0 2024-04-27T04:10:01.041 INFO:tasks.workunit.client.1.smithi181.stdout:7/480: read d1/d2/d35/d67/f7e [376152,102586] 0 2024-04-27T04:10:01.046 INFO:tasks.workunit.client.0.smithi162.stdout:0/281: dwrite d5/dc/fe [0,4194304] 0 2024-04-27T04:10:01.047 INFO:tasks.workunit.client.1.smithi181.stdout:0/508: dread dc/d1f/d3b/d4a/d95/d26/d41/f4e [0,4194304] 0 2024-04-27T04:10:01.049 INFO:tasks.workunit.client.0.smithi162.stdout:1/209: write d0/f2b [1241396,36342] 0 2024-04-27T04:10:01.050 INFO:tasks.workunit.client.0.smithi162.stdout:0/282: read d5/dc/d1e/d41/f34 [379114,128821] 0 2024-04-27T04:10:01.051 INFO:tasks.workunit.client.0.smithi162.stdout:1/210: chown d0/d7/d3c 59 1 2024-04-27T04:10:01.051 INFO:tasks.workunit.client.0.smithi162.stdout:0/283: stat d5/l47 0 2024-04-27T04:10:01.052 INFO:tasks.workunit.client.0.smithi162.stdout:2/186: rename d0/d4/f9 to d0/d4/d2a/f36 0 2024-04-27T04:10:01.053 INFO:tasks.workunit.client.1.smithi181.stdout:1/521: write f0 [1557570,113210] 0 2024-04-27T04:10:01.054 INFO:tasks.workunit.client.0.smithi162.stdout:4/277: unlink d2/da/dd/l32 0 2024-04-27T04:10:01.054 INFO:tasks.workunit.client.0.smithi162.stdout:2/187: truncate d0/f6 5134933 0 2024-04-27T04:10:01.057 INFO:tasks.workunit.client.1.smithi181.stdout:5/620: dread - d4/fa0 zero size 2024-04-27T04:10:01.058 INFO:tasks.workunit.client.1.smithi181.stdout:6/390: mknod d7/d31/d45/d72/c86 0 2024-04-27T04:10:01.059 INFO:tasks.workunit.client.1.smithi181.stdout:6/391: chown d7/d31/d45/d62 162 1 2024-04-27T04:10:01.059 INFO:tasks.workunit.client.1.smithi181.stdout:2/478: getdents d4/d9/d35/d42/d86 0 2024-04-27T04:10:01.059 INFO:tasks.workunit.client.0.smithi162.stdout:8/260: dwrite d0/d10/d2b/d23/f31 [4194304,4194304] 0 2024-04-27T04:10:01.059 INFO:tasks.workunit.client.0.smithi162.stdout:4/278: fsync d2/da/d1a/f1b 0 2024-04-27T04:10:01.059 INFO:tasks.workunit.client.0.smithi162.stdout:7/184: creat d3/d5/d9/dc/de/d2b/f37 x:0 0 0 2024-04-27T04:10:01.059 INFO:tasks.workunit.client.0.smithi162.stdout:8/261: readlink d0/de/d20/l21 0 2024-04-27T04:10:01.060 INFO:tasks.workunit.client.1.smithi181.stdout:7/481: creat d1/d22/f95 x:0 0 0 2024-04-27T04:10:01.064 INFO:tasks.workunit.client.1.smithi181.stdout:4/438: getdents d0/d15 0 2024-04-27T04:10:01.066 INFO:tasks.workunit.client.0.smithi162.stdout:1/211: dwrite d0/d7/d10/f26 [0,4194304] 0 2024-04-27T04:10:01.067 INFO:tasks.workunit.client.1.smithi181.stdout:3/455: getdents d0/d6c/d66/d88 0 2024-04-27T04:10:01.068 INFO:tasks.workunit.client.0.smithi162.stdout:2/188: symlink d0/d20/d2e/l37 0 2024-04-27T04:10:01.069 INFO:tasks.workunit.client.0.smithi162.stdout:2/189: fsync d0/f7 0 2024-04-27T04:10:01.069 INFO:tasks.workunit.client.1.smithi181.stdout:0/509: mkdir dc/d1f/d6b/db9 0 2024-04-27T04:10:01.072 INFO:tasks.workunit.client.0.smithi162.stdout:7/185: creat d3/d5/d9/dc/de/f38 x:0 0 0 2024-04-27T04:10:01.072 INFO:tasks.workunit.client.0.smithi162.stdout:4/279: dread d2/d25/f7 [0,4194304] 0 2024-04-27T04:10:01.072 INFO:tasks.workunit.client.0.smithi162.stdout:9/222: sync 2024-04-27T04:10:01.073 INFO:tasks.workunit.client.1.smithi181.stdout:1/522: fsync d1/d4c/d1e/d20/d53/d56/f6f 0 2024-04-27T04:10:01.073 INFO:tasks.workunit.client.0.smithi162.stdout:8/262: rmdir d0/d10/d2b/d2d/d2f 39 2024-04-27T04:10:01.073 INFO:tasks.workunit.client.1.smithi181.stdout:1/523: dread - d1/d4c/d1e/d20/f9d zero size 2024-04-27T04:10:01.079 INFO:tasks.workunit.client.0.smithi162.stdout:7/186: dwrite d3/d5/d9/dc/f2e [0,4194304] 0 2024-04-27T04:10:01.080 INFO:tasks.workunit.client.0.smithi162.stdout:7/187: fdatasync d3/d5/d9/dc/de/f1c 0 2024-04-27T04:10:01.081 INFO:tasks.workunit.client.0.smithi162.stdout:7/188: chown d3/d5/d9 270283 1 2024-04-27T04:10:01.081 INFO:tasks.workunit.client.0.smithi162.stdout:7/189: stat d3/d5/d9/dc/de/f38 0 2024-04-27T04:10:01.082 INFO:tasks.workunit.client.1.smithi181.stdout:6/392: unlink d7/dc/d1c/l67 0 2024-04-27T04:10:01.090 INFO:tasks.workunit.client.1.smithi181.stdout:5/621: write d4/de/d2c/d46/d4d/d8e/d67/fa2 [1023978,104085] 0 2024-04-27T04:10:01.093 INFO:tasks.workunit.client.1.smithi181.stdout:0/510: fsync f6 0 2024-04-27T04:10:01.094 INFO:tasks.workunit.client.0.smithi162.stdout:2/190: creat d0/d24/f38 x:0 0 0 2024-04-27T04:10:01.094 INFO:tasks.workunit.client.1.smithi181.stdout:8/585: link d1/d6/d1a/lc1 d1/d6/d2f/d30/d66/lcc 0 2024-04-27T04:10:01.097 INFO:tasks.workunit.client.0.smithi162.stdout:9/223: unlink d3/d23/d1e/f27 0 2024-04-27T04:10:01.098 INFO:tasks.workunit.client.0.smithi162.stdout:9/224: stat d3/d23/d1e/c32 0 2024-04-27T04:10:01.100 INFO:tasks.workunit.client.1.smithi181.stdout:0/511: dwrite dc/d1f/d3b/d4a/d95/d26/d41/d6f/f69 [0,4194304] 0 2024-04-27T04:10:01.102 INFO:tasks.workunit.client.0.smithi162.stdout:8/263: truncate d0/d10/d2b/d2d/d2f/d42/f48 557192 0 2024-04-27T04:10:01.103 INFO:tasks.workunit.client.0.smithi162.stdout:9/225: dwrite d3/d23/d1e/d43/f48 [0,4194304] 0 2024-04-27T04:10:01.104 INFO:tasks.workunit.client.0.smithi162.stdout:3/243: link d1/d45/l4b d1/d18/d23/d34/l4e 0 2024-04-27T04:10:01.105 INFO:tasks.workunit.client.0.smithi162.stdout:3/244: chown d1/d18/f10 1 1 2024-04-27T04:10:01.106 INFO:tasks.workunit.client.0.smithi162.stdout:4/280: dread d2/da/f49 [0,4194304] 0 2024-04-27T04:10:01.106 INFO:tasks.workunit.client.0.smithi162.stdout:9/226: write d3/d23/d1e/d43/f48 [3802981,53183] 0 2024-04-27T04:10:01.107 INFO:tasks.workunit.client.1.smithi181.stdout:4/439: getdents d0/d15/d76 0 2024-04-27T04:10:01.107 INFO:tasks.workunit.client.0.smithi162.stdout:4/281: chown d2/da/dd 182955970 1 2024-04-27T04:10:01.108 INFO:tasks.workunit.client.0.smithi162.stdout:0/284: link d5/d26/d4a/c5a d5/dc/d1e/d41/c5e 0 2024-04-27T04:10:01.108 INFO:tasks.workunit.client.0.smithi162.stdout:4/282: truncate d2/da/f31 4381481 0 2024-04-27T04:10:01.109 INFO:tasks.workunit.client.1.smithi181.stdout:4/440: read d0/d15/d25/d40/f6c [3950400,33610] 0 2024-04-27T04:10:01.109 INFO:tasks.workunit.client.1.smithi181.stdout:3/456: mkdir d0/de/d20/d6b/d91 0 2024-04-27T04:10:01.110 INFO:tasks.workunit.client.1.smithi181.stdout:4/441: chown d0/d15/d25/d2a/d7a 2747 1 2024-04-27T04:10:01.110 INFO:tasks.workunit.client.1.smithi181.stdout:5/622: rmdir d4/d10/d72 39 2024-04-27T04:10:01.111 INFO:tasks.workunit.client.1.smithi181.stdout:4/442: stat d0/d15/d25/d2b/d6d/l75 0 2024-04-27T04:10:01.112 INFO:tasks.workunit.client.1.smithi181.stdout:8/586: truncate d1/d6/d2f/d40/d2a/f7c 253410 0 2024-04-27T04:10:01.112 INFO:tasks.workunit.client.1.smithi181.stdout:4/443: write d0/d15/d25/d2b/d6d/f71 [5234820,128875] 0 2024-04-27T04:10:01.115 INFO:tasks.workunit.client.0.smithi162.stdout:8/264: symlink d0/d10/d2b/d2e/l52 0 2024-04-27T04:10:01.116 INFO:tasks.workunit.client.0.smithi162.stdout:8/265: chown d0/d10/d2b/d2d/d2f/d42/f48 25405 1 2024-04-27T04:10:01.117 INFO:tasks.workunit.client.0.smithi162.stdout:7/190: mkdir d3/d39 0 2024-04-27T04:10:01.118 INFO:tasks.workunit.client.0.smithi162.stdout:7/191: write d3/d5/d9/f17 [2560379,6302] 0 2024-04-27T04:10:01.119 INFO:tasks.workunit.client.0.smithi162.stdout:4/283: creat d2/d25/f5c x:0 0 0 2024-04-27T04:10:01.120 INFO:tasks.workunit.client.0.smithi162.stdout:4/284: dread - d2/d25/d1c/d39/d3b/d47/f4c zero size 2024-04-27T04:10:01.121 INFO:tasks.workunit.client.0.smithi162.stdout:9/227: dread d3/d13/d2d/f37 [0,4194304] 0 2024-04-27T04:10:01.125 INFO:tasks.workunit.client.1.smithi181.stdout:1/524: rename d1/d4c to d1/d2/d5/d6/da3 0 2024-04-27T04:10:01.126 INFO:tasks.workunit.client.0.smithi162.stdout:8/266: fsync d0/d10/d25/f14 0 2024-04-27T04:10:01.127 INFO:tasks.workunit.client.1.smithi181.stdout:0/512: fsync dc/d1f/d3b/f5d 0 2024-04-27T04:10:01.127 INFO:tasks.workunit.client.1.smithi181.stdout:7/482: getdents d1/d2/d1c/d5e 0 2024-04-27T04:10:01.130 INFO:tasks.workunit.client.1.smithi181.stdout:8/587: mkdir d1/d6/d2f/d30/d66/dc0/dcd 0 2024-04-27T04:10:01.132 INFO:tasks.workunit.client.0.smithi162.stdout:4/285: unlink d2/d54/c58 0 2024-04-27T04:10:01.132 INFO:tasks.workunit.client.0.smithi162.stdout:1/212: getdents d0/d7/d3c 0 2024-04-27T04:10:01.134 INFO:tasks.workunit.client.1.smithi181.stdout:6/393: rename d7/c71 to d7/dc/d1e/d5c/c87 0 2024-04-27T04:10:01.136 INFO:tasks.workunit.client.1.smithi181.stdout:0/513: dread dc/d1f/d3b/d4a/d95/d26/d41/d6f/d7d/f9a [0,4194304] 0 2024-04-27T04:10:01.137 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:10:00 smithi162 ceph-mon[26841]: overall HEALTH_OK 2024-04-27T04:10:01.138 INFO:tasks.workunit.client.1.smithi181.stdout:0/514: chown dc/d1f/d3b/d4a/d95/d26/d41/d6f/d7d/f9b 480511546 1 2024-04-27T04:10:01.143 INFO:tasks.workunit.client.0.smithi162.stdout:8/267: rename d0/de/d20/l21 to d0/d10/d2b/d2e/l53 0 2024-04-27T04:10:01.145 INFO:tasks.workunit.client.1.smithi181.stdout:8/588: symlink d1/d6/d2f/d30/d5c/lce 0 2024-04-27T04:10:01.147 INFO:tasks.workunit.client.1.smithi181.stdout:2/479: dread d4/d9/df/d14/f15 [0,4194304] 0 2024-04-27T04:10:01.147 INFO:tasks.workunit.client.1.smithi181.stdout:1/525: mkdir d1/d2/d5/d6/da3/d60/d97/da4 0 2024-04-27T04:10:01.148 INFO:tasks.workunit.client.0.smithi162.stdout:1/213: dread d0/f1b [0,4194304] 0 2024-04-27T04:10:01.149 INFO:tasks.workunit.client.1.smithi181.stdout:5/623: rename d4/de/d2c/d2e/d55/l44 to d4/de/d2c/d46/d99/ldf 0 2024-04-27T04:10:01.149 INFO:tasks.workunit.client.1.smithi181.stdout:7/483: dread d1/d2/d1c/d37/f4f [0,4194304] 0 2024-04-27T04:10:01.151 INFO:tasks.workunit.client.1.smithi181.stdout:5/624: write d4/d10/d29/f84 [3216543,84631] 0 2024-04-27T04:10:01.152 INFO:tasks.workunit.client.1.smithi181.stdout:5/625: chown d4/de/d2c/d2e/l62 0 1 2024-04-27T04:10:01.154 INFO:tasks.workunit.client.0.smithi162.stdout:9/228: mkdir d3/d11/d45/d4e 0 2024-04-27T04:10:01.155 INFO:tasks.workunit.client.0.smithi162.stdout:9/229: fdatasync d3/d13/d15/f18 0 2024-04-27T04:10:01.157 INFO:tasks.workunit.client.0.smithi162.stdout:3/245: getdents d1 0 2024-04-27T04:10:01.159 INFO:tasks.workunit.client.1.smithi181.stdout:4/444: dwrite d0/f30 [0,4194304] 0 2024-04-27T04:10:01.160 INFO:tasks.workunit.client.1.smithi181.stdout:2/480: mkdir d4/d9/d16/d45/d9a 0 2024-04-27T04:10:01.162 INFO:tasks.workunit.client.0.smithi162.stdout:0/285: getdents d5/d26/d4a 0 2024-04-27T04:10:01.163 INFO:tasks.workunit.client.0.smithi162.stdout:7/192: dwrite d3/d1d/f31 [0,4194304] 0 2024-04-27T04:10:01.165 INFO:tasks.workunit.client.0.smithi162.stdout:1/214: mknod d0/d7/d10/d35/c45 0 2024-04-27T04:10:01.165 INFO:tasks.workunit.client.1.smithi181.stdout:5/626: dwrite d4/de/d2c/d46/d4d/d8e/d67/d71/f91 [0,4194304] 0 2024-04-27T04:10:01.169 INFO:tasks.workunit.client.1.smithi181.stdout:7/484: unlink d1/d2/d35/f85 0 2024-04-27T04:10:01.170 INFO:tasks.workunit.client.1.smithi181.stdout:7/485: chown d1/d2/f27 1871531321 1 2024-04-27T04:10:01.171 INFO:tasks.workunit.client.1.smithi181.stdout:7/486: write d1/d2/d1c/d56/f63 [850570,21893] 0 2024-04-27T04:10:01.172 INFO:tasks.workunit.client.1.smithi181.stdout:7/487: write d1/d22/d15/d13/f46 [3612897,100973] 0 2024-04-27T04:10:01.173 INFO:tasks.workunit.client.1.smithi181.stdout:0/515: mkdir dc/d1f/d6b/db9/dba 0 2024-04-27T04:10:01.173 INFO:tasks.workunit.client.1.smithi181.stdout:0/516: stat dc/c14 0 2024-04-27T04:10:01.174 INFO:tasks.workunit.client.1.smithi181.stdout:0/517: write dc/d1f/f7c [223233,42234] 0 2024-04-27T04:10:01.175 INFO:tasks.workunit.client.1.smithi181.stdout:0/518: chown dc/ff 8383773 1 2024-04-27T04:10:01.176 INFO:tasks.workunit.client.0.smithi162.stdout:9/230: mkdir d3/d13/d2d/d4f 0 2024-04-27T04:10:01.178 INFO:tasks.workunit.client.0.smithi162.stdout:3/246: rmdir d1/d45 39 2024-04-27T04:10:01.179 INFO:tasks.workunit.client.0.smithi162.stdout:8/268: rename d0/f2 to d0/d10/d2b/d2d/d2f/f54 0 2024-04-27T04:10:01.182 INFO:tasks.workunit.client.0.smithi162.stdout:7/193: mknod d3/d5/d9/dc/de/d2b/c3a 0 2024-04-27T04:10:01.183 INFO:tasks.workunit.client.0.smithi162.stdout:7/194: chown d3/d5/d9/dc/de/c10 32358 1 2024-04-27T04:10:01.184 INFO:tasks.workunit.client.0.smithi162.stdout:9/231: mkdir d3/d13/d15/d50 0 2024-04-27T04:10:01.185 INFO:tasks.workunit.client.0.smithi162.stdout:9/232: write d3/d13/d2d/d3b/f47 [828,94138] 0 2024-04-27T04:10:01.186 INFO:tasks.workunit.client.0.smithi162.stdout:0/286: rename d5/dc/d1e/c5d to d5/dc/d1e/d46/d48/c5f 0 2024-04-27T04:10:01.187 INFO:tasks.workunit.client.0.smithi162.stdout:4/286: getdents d2/d25/d1c/d24 0 2024-04-27T04:10:01.188 INFO:tasks.workunit.client.0.smithi162.stdout:7/195: write d3/f18 [2320354,80364] 0 2024-04-27T04:10:01.189 INFO:tasks.workunit.client.0.smithi162.stdout:5/219: sync 2024-04-27T04:10:01.192 INFO:tasks.workunit.client.0.smithi162.stdout:1/215: rename d0/d7/d10/l15 to d0/d7/d2a/l46 0 2024-04-27T04:10:01.193 INFO:tasks.workunit.client.0.smithi162.stdout:8/269: rename d0/d10/d2b/d2d to d0/d10/d2b/d2d/d2f/d42/d55 22 2024-04-27T04:10:01.193 INFO:tasks.workunit.client.0.smithi162.stdout:8/270: stat d0/c3e 0 2024-04-27T04:10:01.194 INFO:tasks.workunit.client.0.smithi162.stdout:8/271: chown d0/de/d20/c3b 37 1 2024-04-27T04:10:01.195 INFO:tasks.workunit.client.0.smithi162.stdout:9/233: mknod d3/d13/d15/d50/c51 0 2024-04-27T04:10:01.196 INFO:tasks.workunit.client.0.smithi162.stdout:5/220: creat d8/d1a/d29/f4d x:0 0 0 2024-04-27T04:10:01.198 INFO:tasks.workunit.client.0.smithi162.stdout:0/287: mkdir d5/d60 0 2024-04-27T04:10:01.199 INFO:tasks.workunit.client.0.smithi162.stdout:1/216: creat d0/d7/d2a/f47 x:0 0 0 2024-04-27T04:10:01.201 INFO:tasks.workunit.client.0.smithi162.stdout:8/272: mknod d0/d10/d2b/c56 0 2024-04-27T04:10:01.205 INFO:tasks.workunit.client.0.smithi162.stdout:5/221: unlink c3 0 2024-04-27T04:10:01.206 INFO:tasks.workunit.client.0.smithi162.stdout:8/273: dread d0/de/d20/f50 [0,4194304] 0 2024-04-27T04:10:01.211 INFO:tasks.workunit.client.0.smithi162.stdout:5/222: dwrite d8/d1a/f3e [0,4194304] 0 2024-04-27T04:10:01.213 INFO:tasks.workunit.client.1.smithi181.stdout:5/627: mknod d4/de/ce0 0 2024-04-27T04:10:01.217 INFO:tasks.workunit.client.0.smithi162.stdout:6/245: sync 2024-04-27T04:10:01.219 INFO:tasks.workunit.client.0.smithi162.stdout:3/247: rmdir d1/d18/d23/d34/d49 0 2024-04-27T04:10:01.223 INFO:tasks.workunit.client.1.smithi181.stdout:6/394: rename d7/dc/d1c/d68 to d7/dc/d1c/d88 0 2024-04-27T04:10:01.224 INFO:tasks.workunit.client.1.smithi181.stdout:7/488: creat d1/d2/d35/d67/f96 x:0 0 0 2024-04-27T04:10:01.225 INFO:tasks.workunit.client.0.smithi162.stdout:5/223: mknod d8/d1a/d23/d40/c4e 0 2024-04-27T04:10:01.227 INFO:tasks.workunit.client.0.smithi162.stdout:1/217: mknod d0/d11/d44/c48 0 2024-04-27T04:10:01.227 INFO:tasks.workunit.client.0.smithi162.stdout:1/218: dread - d0/f34 zero size 2024-04-27T04:10:01.228 INFO:tasks.workunit.client.1.smithi181.stdout:8/589: creat d1/d6/d2f/fcf x:0 0 0 2024-04-27T04:10:01.230 INFO:tasks.workunit.client.0.smithi162.stdout:8/274: truncate d0/f4 3331918 0 2024-04-27T04:10:01.233 INFO:tasks.workunit.client.0.smithi162.stdout:3/248: mkdir d1/d18/d4f 0 2024-04-27T04:10:01.235 INFO:tasks.workunit.client.0.smithi162.stdout:4/287: dwrite f1 [0,4194304] 0 2024-04-27T04:10:01.235 INFO:tasks.workunit.client.0.smithi162.stdout:7/196: getdents d3/d5 0 2024-04-27T04:10:01.239 INFO:tasks.workunit.client.0.smithi162.stdout:5/224: mknod d8/d1a/d23/d40/c4f 0 2024-04-27T04:10:01.243 INFO:tasks.workunit.client.0.smithi162.stdout:6/246: dread d6/d1a/f24 [0,4194304] 0 2024-04-27T04:10:01.245 INFO:tasks.workunit.client.0.smithi162.stdout:9/234: write d3/fb [269280,99971] 0 2024-04-27T04:10:01.251 INFO:tasks.workunit.client.1.smithi181.stdout:1/526: link d1/d2/d5/d6/da3/d1e/l46 d1/d2/d5/d6/d6a/la5 0 2024-04-27T04:10:01.251 INFO:tasks.workunit.client.0.smithi162.stdout:8/275: dread d0/f7 [0,4194304] 0 2024-04-27T04:10:01.253 INFO:tasks.workunit.client.0.smithi162.stdout:9/235: dwrite d3/f2a [0,4194304] 0 2024-04-27T04:10:01.253 INFO:tasks.workunit.client.1.smithi181.stdout:2/481: link d4/d9/d4f/f5b d4/d9/d16/d82/d7d/f9b 0 2024-04-27T04:10:01.255 INFO:tasks.workunit.client.0.smithi162.stdout:0/288: rename d5/dc/d1e/d46/d48/c5b to d5/dc/d1e/c61 0 2024-04-27T04:10:01.255 INFO:tasks.workunit.client.0.smithi162.stdout:0/289: stat d5/f27 0 2024-04-27T04:10:01.256 INFO:tasks.workunit.client.1.smithi181.stdout:6/395: mkdir d7/dc/d1c/d89 0 2024-04-27T04:10:01.261 INFO:tasks.workunit.client.0.smithi162.stdout:7/197: rename d3/d5/d9/dc/f15 to d3/d5/d9/dc/f3b 0 2024-04-27T04:10:01.261 INFO:tasks.workunit.client.1.smithi181.stdout:0/519: link c0 dc/d93/cbb 0 2024-04-27T04:10:01.263 INFO:tasks.workunit.client.0.smithi162.stdout:5/225: creat d8/d1a/d23/f50 x:0 0 0 2024-04-27T04:10:01.265 INFO:tasks.workunit.client.0.smithi162.stdout:6/247: dread d6/d1a/f1d [0,4194304] 0 2024-04-27T04:10:01.265 INFO:tasks.workunit.client.1.smithi181.stdout:7/489: dread d1/d2/f12 [0,4194304] 0 2024-04-27T04:10:01.265 INFO:tasks.workunit.client.1.smithi181.stdout:8/590: unlink d1/l22 0 2024-04-27T04:10:01.266 INFO:tasks.workunit.client.1.smithi181.stdout:4/445: link d0/d15/d25/d40/f4f d0/d15/de/f91 0 2024-04-27T04:10:01.271 INFO:tasks.workunit.client.1.smithi181.stdout:1/527: rmdir d1 39 2024-04-27T04:10:01.274 INFO:tasks.workunit.client.1.smithi181.stdout:2/482: fsync d4/d9/d16/f26 0 2024-04-27T04:10:01.276 INFO:tasks.workunit.client.0.smithi162.stdout:3/249: fdatasync d1/f4c 0 2024-04-27T04:10:01.278 INFO:tasks.workunit.client.1.smithi181.stdout:6/396: rename d7/d31/d38 to d7/dc/d1c/d34/d3f/d7e/d8a 0 2024-04-27T04:10:01.278 INFO:tasks.workunit.client.1.smithi181.stdout:6/397: dread - d7/dc/d1e/d2b/f85 zero size 2024-04-27T04:10:01.282 INFO:tasks.workunit.client.0.smithi162.stdout:1/219: dwrite d0/d7/f27 [0,4194304] 0 2024-04-27T04:10:01.285 INFO:tasks.workunit.client.0.smithi162.stdout:3/250: dwrite d1/d18/d23/d2f/f42 [0,4194304] 0 2024-04-27T04:10:01.288 INFO:tasks.workunit.client.0.smithi162.stdout:8/276: dwrite d0/fd [0,4194304] 0 2024-04-27T04:10:01.288 INFO:tasks.workunit.client.1.smithi181.stdout:5/628: dwrite d4/d10/d72/d80/d85/fae [4194304,4194304] 0 2024-04-27T04:10:01.289 INFO:tasks.workunit.client.0.smithi162.stdout:0/290: write d5/f35 [611753,15202] 0 2024-04-27T04:10:01.289 INFO:tasks.workunit.client.1.smithi181.stdout:5/629: fsync d4/da3/fbd 0 2024-04-27T04:10:01.291 INFO:tasks.workunit.client.1.smithi181.stdout:1/528: chown d1/d2/d5/d6/db/d2d 44 1 2024-04-27T04:10:01.292 INFO:tasks.workunit.client.1.smithi181.stdout:1/529: dread - d1/d2/d5/fa1 zero size 2024-04-27T04:10:01.297 INFO:tasks.workunit.client.0.smithi162.stdout:7/198: mkdir d3/d5/d3c 0 2024-04-27T04:10:01.303 INFO:tasks.workunit.client.0.smithi162.stdout:6/248: symlink d6/d1a/d34/d43/l55 0 2024-04-27T04:10:01.303 INFO:tasks.workunit.client.0.smithi162.stdout:5/226: fdatasync d8/d2c/f4c 0 2024-04-27T04:10:01.305 INFO:tasks.workunit.client.0.smithi162.stdout:5/227: write d8/d1a/d23/f39 [514068,20007] 0 2024-04-27T04:10:01.306 INFO:tasks.workunit.client.0.smithi162.stdout:5/228: read - d8/d1a/d23/f50 zero size 2024-04-27T04:10:01.308 INFO:tasks.workunit.client.0.smithi162.stdout:5/229: chown d8/d1a/d23/d40/f42 1981 1 2024-04-27T04:10:01.309 INFO:tasks.workunit.client.0.smithi162.stdout:3/251: chown d1/d18/d23/l38 2144102 1 2024-04-27T04:10:01.310 INFO:tasks.workunit.client.1.smithi181.stdout:1/530: rename d1/d2/d5/d6/da3/d60/d97 to d1/d2/d5/d6/da3/d1e/d48/da6 0 2024-04-27T04:10:01.311 INFO:tasks.workunit.client.0.smithi162.stdout:4/288: link d2/d25/c1d d2/d25/d1c/d39/d3b/c5d 0 2024-04-27T04:10:01.314 INFO:tasks.workunit.client.0.smithi162.stdout:8/277: rename d0/d10/d2b/d2e/l4c to d0/d10/d44/l57 0 2024-04-27T04:10:01.314 INFO:tasks.workunit.client.0.smithi162.stdout:3/252: dread d1/d18/f1d [0,4194304] 0 2024-04-27T04:10:01.315 INFO:tasks.workunit.client.0.smithi162.stdout:0/291: creat d5/d26/d4a/f62 x:0 0 0 2024-04-27T04:10:01.316 INFO:tasks.workunit.client.1.smithi181.stdout:6/398: mkdir d7/dc/d8b 0 2024-04-27T04:10:01.316 INFO:tasks.workunit.client.1.smithi181.stdout:7/490: getdents d1/d2/d1c/d2d/d61 0 2024-04-27T04:10:01.317 INFO:tasks.workunit.client.1.smithi181.stdout:7/491: stat d1/d22/d15/d29/f43 0 2024-04-27T04:10:01.318 INFO:tasks.workunit.client.0.smithi162.stdout:3/253: chown d1/d18/d23/d34/f3c 782 1 2024-04-27T04:10:01.319 INFO:tasks.workunit.client.0.smithi162.stdout:4/289: dwrite d2/d25/f5c [0,4194304] 0 2024-04-27T04:10:01.322 INFO:tasks.workunit.client.0.smithi162.stdout:4/290: write d2/f22 [4305565,96756] 0 2024-04-27T04:10:01.322 INFO:tasks.workunit.client.1.smithi181.stdout:6/399: dwrite d7/d27/f2e [4194304,4194304] 0 2024-04-27T04:10:01.323 INFO:tasks.workunit.client.1.smithi181.stdout:4/446: getdents d0/d15/de/d60/d6f 0 2024-04-27T04:10:01.325 INFO:tasks.workunit.client.0.smithi162.stdout:8/278: dread d0/d10/d2b/f19 [0,4194304] 0 2024-04-27T04:10:01.326 INFO:tasks.workunit.client.0.smithi162.stdout:0/292: dwrite d5/dc/fe [0,4194304] 0 2024-04-27T04:10:01.328 INFO:tasks.workunit.client.1.smithi181.stdout:1/531: creat d1/d2/d5/d6/da3/d1e/d20/da2/fa7 x:0 0 0 2024-04-27T04:10:01.330 INFO:tasks.workunit.client.0.smithi162.stdout:2/191: sync 2024-04-27T04:10:01.330 INFO:tasks.workunit.client.1.smithi181.stdout:1/532: stat d1/d2/d5/d6/l29 0 2024-04-27T04:10:01.331 INFO:tasks.workunit.client.0.smithi162.stdout:6/249: rename d6/d1a/c4d to d6/d1a/d34/c56 0 2024-04-27T04:10:01.331 INFO:tasks.workunit.client.1.smithi181.stdout:2/483: link d4/d9/df/c1c d4/d9/d35/c9c 0 2024-04-27T04:10:01.332 INFO:tasks.workunit.client.1.smithi181.stdout:2/484: fdatasync d4/d9/d16/d45/f87 0 2024-04-27T04:10:01.332 INFO:tasks.workunit.client.0.smithi162.stdout:8/279: dread d0/de/d20/f50 [0,4194304] 0 2024-04-27T04:10:01.334 INFO:tasks.workunit.client.1.smithi181.stdout:6/400: chown d7/dc/c14 5 1 2024-04-27T04:10:01.337 INFO:tasks.workunit.client.1.smithi181.stdout:4/447: rmdir d0/d15/d25/d2b/d6d/d82 39 2024-04-27T04:10:01.339 INFO:tasks.workunit.client.1.smithi181.stdout:1/533: dwrite d1/d2/d5/d6/da3/d52/d92/f93 [0,4194304] 0 2024-04-27T04:10:01.339 INFO:tasks.workunit.client.1.smithi181.stdout:4/448: readlink d0/d15/de/d19/d77/d57/l6b 0 2024-04-27T04:10:01.341 INFO:tasks.workunit.client.1.smithi181.stdout:4/449: dread - d0/d15/de/d19/d77/d57/f84 zero size 2024-04-27T04:10:01.341 INFO:tasks.workunit.client.0.smithi162.stdout:5/230: getdents d8/d1a/d29/d4a 0 2024-04-27T04:10:01.342 INFO:tasks.workunit.client.1.smithi181.stdout:1/534: write d1/d2/d5/d6/da3/d52/f76 [2823458,106317] 0 2024-04-27T04:10:01.343 INFO:tasks.workunit.client.1.smithi181.stdout:0/520: truncate dc/d54/f5b 232289 0 2024-04-27T04:10:01.346 INFO:tasks.workunit.client.0.smithi162.stdout:4/291: mkdir d2/d25/d1c/d39/d5e 0 2024-04-27T04:10:01.347 INFO:tasks.workunit.client.0.smithi162.stdout:4/292: chown d2/d25/f5c 3514 1 2024-04-27T04:10:01.347 INFO:tasks.workunit.client.0.smithi162.stdout:4/293: chown d2/d25 53816 1 2024-04-27T04:10:01.356 INFO:tasks.workunit.client.0.smithi162.stdout:1/220: truncate d0/f2b 1239333 0 2024-04-27T04:10:01.357 INFO:tasks.workunit.client.0.smithi162.stdout:1/221: write d0/f34 [523521,91817] 0 2024-04-27T04:10:01.357 INFO:tasks.workunit.client.0.smithi162.stdout:6/250: rename d6/db/d32/f4b to d6/d1a/d34/d46/f57 0 2024-04-27T04:10:01.359 INFO:tasks.workunit.client.1.smithi181.stdout:4/450: fdatasync d0/d15/d25/d2a/d36/f4a 0 2024-04-27T04:10:01.359 INFO:tasks.workunit.client.0.smithi162.stdout:6/251: chown d6/d1a/d34/d43/l55 11156 1 2024-04-27T04:10:01.360 INFO:tasks.workunit.client.0.smithi162.stdout:6/252: chown d6/f11 68310 1 2024-04-27T04:10:01.360 INFO:tasks.workunit.client.1.smithi181.stdout:4/451: chown d0/d15/de/f43 837291504 1 2024-04-27T04:10:01.361 INFO:tasks.workunit.client.1.smithi181.stdout:2/485: mkdir d4/d9/d16/d45/d9a/d9d 0 2024-04-27T04:10:01.363 INFO:tasks.workunit.client.1.smithi181.stdout:6/401: fdatasync d7/d27/f2e 0 2024-04-27T04:10:01.364 INFO:tasks.workunit.client.0.smithi162.stdout:0/293: symlink d5/l63 0 2024-04-27T04:10:01.364 INFO:tasks.workunit.client.1.smithi181.stdout:0/521: mkdir dc/d1f/daa/dbc 0 2024-04-27T04:10:01.366 INFO:tasks.workunit.client.1.smithi181.stdout:6/402: chown d7/dc/d1c/f78 64898 1 2024-04-27T04:10:01.366 INFO:tasks.workunit.client.0.smithi162.stdout:6/253: dwrite f2 [0,4194304] 0 2024-04-27T04:10:01.367 INFO:tasks.workunit.client.0.smithi162.stdout:6/254: chown d6/d1a/d34/d43/f44 1 1 2024-04-27T04:10:01.368 INFO:tasks.workunit.client.1.smithi181.stdout:4/452: dwrite d0/d15/de/d19/d59/f73 [0,4194304] 0 2024-04-27T04:10:01.368 INFO:tasks.workunit.client.1.smithi181.stdout:5/630: write d4/de/f14 [524154,68275] 0 2024-04-27T04:10:01.372 INFO:tasks.workunit.client.0.smithi162.stdout:3/254: getdents d1/d21/d47 0 2024-04-27T04:10:01.377 INFO:tasks.workunit.client.1.smithi181.stdout:0/522: mkdir dc/d1f/d6b/db9/dbd 0 2024-04-27T04:10:01.378 INFO:tasks.workunit.client.1.smithi181.stdout:5/631: dwrite d4/d10/d29/f84 [0,4194304] 0 2024-04-27T04:10:01.382 INFO:tasks.workunit.client.1.smithi181.stdout:7/492: truncate d1/d22/d15/d13/d1b/f40 4489290 0 2024-04-27T04:10:01.388 INFO:tasks.workunit.client.0.smithi162.stdout:8/280: write d0/d10/d25/f14 [737376,114674] 0 2024-04-27T04:10:01.389 INFO:tasks.workunit.client.1.smithi181.stdout:2/486: rmdir d4/d70 0 2024-04-27T04:10:01.390 INFO:tasks.workunit.client.1.smithi181.stdout:6/403: dwrite d7/dc/d1c/d34/d3f/d7e/d8a/f7a [0,4194304] 0 2024-04-27T04:10:01.392 INFO:tasks.workunit.client.0.smithi162.stdout:0/294: mknod d5/dc/d37/c64 0 2024-04-27T04:10:01.393 INFO:tasks.workunit.client.1.smithi181.stdout:4/453: dwrite d0/d15/d25/d2b/d6d/f74 [4194304,4194304] 0 2024-04-27T04:10:01.396 INFO:tasks.workunit.client.0.smithi162.stdout:6/255: unlink d6/f7 0 2024-04-27T04:10:01.398 INFO:tasks.workunit.client.0.smithi162.stdout:1/222: symlink d0/d7/l49 0 2024-04-27T04:10:01.399 INFO:tasks.workunit.client.0.smithi162.stdout:1/223: stat d0/d11/f20 0 2024-04-27T04:10:01.399 INFO:tasks.workunit.client.1.smithi181.stdout:7/493: creat d1/d2/d1c/d2d/d61/f97 x:0 0 0 2024-04-27T04:10:01.400 INFO:tasks.workunit.client.1.smithi181.stdout:7/494: chown d1/d2/d1c/d2d 5372026 1 2024-04-27T04:10:01.402 INFO:tasks.workunit.client.0.smithi162.stdout:7/199: mknod d3/d5/d9/dc/de/d1f/c3d 0 2024-04-27T04:10:01.403 INFO:tasks.workunit.client.0.smithi162.stdout:4/294: dwrite d2/f11 [0,4194304] 0 2024-04-27T04:10:01.403 INFO:tasks.workunit.client.0.smithi162.stdout:8/281: mknod d0/d10/d25/c58 0 2024-04-27T04:10:01.405 INFO:tasks.workunit.client.0.smithi162.stdout:6/256: creat d6/d1a/d34/d43/f58 x:0 0 0 2024-04-27T04:10:01.406 INFO:tasks.workunit.client.0.smithi162.stdout:8/282: write d0/d10/d2b/d23/f31 [2251067,107168] 0 2024-04-27T04:10:01.408 INFO:tasks.workunit.client.0.smithi162.stdout:1/224: rename d0/d11/c25 to d0/d7/d3c/c4a 0 2024-04-27T04:10:01.410 INFO:tasks.workunit.client.1.smithi181.stdout:2/487: fsync d4/d9/d4f/f5b 0 2024-04-27T04:10:01.413 INFO:tasks.workunit.client.0.smithi162.stdout:4/295: dwrite d2/d25/d1c/d39/f46 [0,4194304] 0 2024-04-27T04:10:01.414 INFO:tasks.workunit.client.1.smithi181.stdout:4/454: getdents d0/d15/d25/d2a/d4c 0 2024-04-27T04:10:01.415 INFO:tasks.workunit.client.0.smithi162.stdout:7/200: rename d3/c25 to d3/d5/d3c/c3e 0 2024-04-27T04:10:01.416 INFO:tasks.workunit.client.0.smithi162.stdout:4/296: write d2/d25/f7 [4734608,23732] 0 2024-04-27T04:10:01.416 INFO:tasks.workunit.client.0.smithi162.stdout:8/283: getdents d0/d10/d2b/d2d/d2f 0 2024-04-27T04:10:01.416 INFO:tasks.workunit.client.1.smithi181.stdout:7/495: link d1/d2/d1c/d56/c90 d1/d57/c98 0 2024-04-27T04:10:01.417 INFO:tasks.workunit.client.1.smithi181.stdout:5/632: getdents d4/de/d2c 0 2024-04-27T04:10:01.418 INFO:tasks.workunit.client.1.smithi181.stdout:7/496: dread d1/d2/d35/d67/f7e [0,4194304] 0 2024-04-27T04:10:01.419 INFO:tasks.workunit.client.1.smithi181.stdout:4/455: chown d0/d15/de/f91 17908286 1 2024-04-27T04:10:01.419 INFO:tasks.workunit.client.0.smithi162.stdout:1/225: dwrite d0/d7/d2a/f47 [0,4194304] 0 2024-04-27T04:10:01.420 INFO:tasks.workunit.client.1.smithi181.stdout:4/456: truncate d0/d15/de/d19/d59/f79 1275897 0 2024-04-27T04:10:01.422 INFO:tasks.workunit.client.0.smithi162.stdout:7/201: creat d3/d5/d9/dc/f3f x:0 0 0 2024-04-27T04:10:01.431 INFO:tasks.workunit.client.1.smithi181.stdout:2/488: link d4/d9/df/d14/f15 d4/d9/d16/d45/f9e 0 2024-04-27T04:10:01.432 INFO:tasks.workunit.client.1.smithi181.stdout:4/457: creat d0/d15/d25/d2a/d36/f92 x:0 0 0 2024-04-27T04:10:01.433 INFO:tasks.workunit.client.1.smithi181.stdout:7/497: mkdir d1/d2/d1c/d99 0 2024-04-27T04:10:01.433 INFO:tasks.workunit.client.1.smithi181.stdout:4/458: fdatasync d0/f30 0 2024-04-27T04:10:01.438 INFO:tasks.workunit.client.0.smithi162.stdout:8/284: rmdir d0/d10/d2b/d2e/d35 0 2024-04-27T04:10:01.438 INFO:tasks.workunit.client.0.smithi162.stdout:8/285: readlink d0/de/l13 0 2024-04-27T04:10:01.439 INFO:tasks.workunit.client.0.smithi162.stdout:8/286: dread - d0/d10/d2b/d23/f51 zero size 2024-04-27T04:10:01.440 INFO:tasks.workunit.client.0.smithi162.stdout:9/236: sync 2024-04-27T04:10:01.442 INFO:tasks.workunit.client.0.smithi162.stdout:1/226: rename d0/l2f to d0/l4b 0 2024-04-27T04:10:01.444 INFO:tasks.workunit.client.1.smithi181.stdout:7/498: mkdir d1/d2/d1c/d5e/d9a 0 2024-04-27T04:10:01.446 INFO:tasks.workunit.client.0.smithi162.stdout:7/202: creat d3/f40 x:0 0 0 2024-04-27T04:10:01.448 INFO:tasks.workunit.client.0.smithi162.stdout:1/227: creat d0/d7/d10/f4c x:0 0 0 2024-04-27T04:10:01.450 INFO:tasks.workunit.client.0.smithi162.stdout:7/203: readlink d3/l22 0 2024-04-27T04:10:01.450 INFO:tasks.workunit.client.1.smithi181.stdout:4/459: mkdir d0/d15/d25/d2b/d6d/d93 0 2024-04-27T04:10:01.452 INFO:tasks.workunit.client.0.smithi162.stdout:9/237: dwrite d3/d13/d15/f19 [0,4194304] 0 2024-04-27T04:10:01.454 INFO:tasks.workunit.client.0.smithi162.stdout:7/204: creat d3/d1d/f41 x:0 0 0 2024-04-27T04:10:01.456 INFO:tasks.workunit.client.0.smithi162.stdout:9/238: symlink d3/d13/d2d/d44/l52 0 2024-04-27T04:10:01.457 INFO:tasks.workunit.client.1.smithi181.stdout:4/460: dwrite d0/d15/d25/d2a/f38 [0,4194304] 0 2024-04-27T04:10:01.457 INFO:tasks.workunit.client.0.smithi162.stdout:9/239: dread - d3/d23/d1e/f4a zero size 2024-04-27T04:10:01.458 INFO:tasks.workunit.client.1.smithi181.stdout:4/461: write d0/f3b [1319177,115473] 0 2024-04-27T04:10:01.458 INFO:tasks.workunit.client.0.smithi162.stdout:9/240: truncate d3/d13/d2d/d3b/f47 698067 0 2024-04-27T04:10:01.459 INFO:tasks.workunit.client.0.smithi162.stdout:7/205: rename d3/d5/d9/f28 to d3/d5/d9/dc/de/d1f/f42 0 2024-04-27T04:10:01.460 INFO:tasks.workunit.client.0.smithi162.stdout:9/241: write d3/f21 [1513670,102530] 0 2024-04-27T04:10:01.463 INFO:tasks.workunit.client.1.smithi181.stdout:4/462: read d0/d15/d25/d2a/d4c/f4e [1191630,99108] 0 2024-04-27T04:10:01.464 INFO:tasks.workunit.client.0.smithi162.stdout:7/206: dread d3/d5/d9/dc/f2e [0,4194304] 0 2024-04-27T04:10:01.465 INFO:tasks.workunit.client.0.smithi162.stdout:9/242: symlink d3/d39/l53 0 2024-04-27T04:10:01.466 INFO:tasks.workunit.client.0.smithi162.stdout:9/243: fsync d3/d23/f22 0 2024-04-27T04:10:01.468 INFO:tasks.workunit.client.1.smithi181.stdout:4/463: creat d0/d15/de/d19/f94 x:0 0 0 2024-04-27T04:10:01.469 INFO:tasks.workunit.client.0.smithi162.stdout:7/207: creat d3/d5/d9/dc/de/d1f/f43 x:0 0 0 2024-04-27T04:10:01.469 INFO:tasks.workunit.client.1.smithi181.stdout:4/464: read d0/d15/d25/d2a/d36/f51 [105301,37516] 0 2024-04-27T04:10:01.470 INFO:tasks.workunit.client.1.smithi181.stdout:4/465: fsync d0/d15/de/d19/d59/f79 0 2024-04-27T04:10:01.471 INFO:tasks.workunit.client.0.smithi162.stdout:5/231: truncate d8/d9/f48 475167 0 2024-04-27T04:10:01.474 INFO:tasks.workunit.client.0.smithi162.stdout:7/208: creat d3/d5/d9/dc/de/d1f/f44 x:0 0 0 2024-04-27T04:10:01.480 INFO:tasks.workunit.client.1.smithi181.stdout:4/466: rename d0/d15/d25/d2a/d36/f7f to d0/d15/d76/f95 0 2024-04-27T04:10:01.481 INFO:tasks.workunit.client.0.smithi162.stdout:7/209: mknod d3/d5/d9/c45 0 2024-04-27T04:10:01.483 INFO:tasks.workunit.client.0.smithi162.stdout:9/244: dread d3/f21 [0,4194304] 0 2024-04-27T04:10:01.485 INFO:tasks.workunit.client.0.smithi162.stdout:9/245: truncate d3/d13/d15/f2f 8435112 0 2024-04-27T04:10:01.487 INFO:tasks.workunit.client.0.smithi162.stdout:5/232: dread d8/d1a/f17 [0,4194304] 0 2024-04-27T04:10:01.487 INFO:tasks.workunit.client.1.smithi181.stdout:4/467: symlink d0/d15/d25/d2a/l96 0 2024-04-27T04:10:01.491 INFO:tasks.workunit.client.0.smithi162.stdout:3/255: truncate d1/d18/f2d 3051295 0 2024-04-27T04:10:01.492 INFO:tasks.workunit.client.0.smithi162.stdout:9/246: mkdir d3/d13/d2d/d3b/d54 0 2024-04-27T04:10:01.492 INFO:tasks.workunit.client.0.smithi162.stdout:9/247: stat d3/l7 0 2024-04-27T04:10:01.493 INFO:tasks.workunit.client.1.smithi181.stdout:0/523: truncate dc/d1f/d3b/d75/f90 2954473 0 2024-04-27T04:10:01.493 INFO:tasks.workunit.client.0.smithi162.stdout:0/295: sync 2024-04-27T04:10:01.493 INFO:tasks.workunit.client.0.smithi162.stdout:2/192: sync 2024-04-27T04:10:01.494 INFO:tasks.workunit.client.0.smithi162.stdout:0/296: readlink d5/d26/l3d 0 2024-04-27T04:10:01.494 INFO:tasks.workunit.client.1.smithi181.stdout:4/468: mknod d0/d15/d5c/d69/c97 0 2024-04-27T04:10:01.495 INFO:tasks.workunit.client.0.smithi162.stdout:1/228: getdents d0/d7/d3c 0 2024-04-27T04:10:01.496 INFO:tasks.workunit.client.0.smithi162.stdout:5/233: creat d8/d1a/f51 x:0 0 0 2024-04-27T04:10:01.497 INFO:tasks.workunit.client.1.smithi181.stdout:0/524: rmdir dc/d54 39 2024-04-27T04:10:01.500 INFO:tasks.workunit.client.1.smithi181.stdout:4/469: creat d0/d15/de/d60/d6f/f98 x:0 0 0 2024-04-27T04:10:01.503 INFO:tasks.workunit.client.0.smithi162.stdout:9/248: creat d3/d11/d45/f55 x:0 0 0 2024-04-27T04:10:01.504 INFO:tasks.workunit.client.0.smithi162.stdout:2/193: write d0/f1 [5170552,2598] 0 2024-04-27T04:10:01.504 INFO:tasks.workunit.client.0.smithi162.stdout:9/249: write d3/d23/f22 [200301,1133] 0 2024-04-27T04:10:01.507 INFO:tasks.workunit.client.0.smithi162.stdout:9/250: readlink d3/d13/d2d/d44/l52 0 2024-04-27T04:10:01.508 INFO:tasks.workunit.client.0.smithi162.stdout:3/256: symlink d1/d45/l50 0 2024-04-27T04:10:01.511 INFO:tasks.workunit.client.1.smithi181.stdout:5/633: dwrite d4/de/d2c/d2e/d55/f3f [0,4194304] 0 2024-04-27T04:10:01.518 INFO:tasks.workunit.client.1.smithi181.stdout:5/634: creat d4/de/d2c/d46/d99/dad/fe1 x:0 0 0 2024-04-27T04:10:01.520 INFO:tasks.workunit.client.1.smithi181.stdout:2/489: dwrite d4/f6 [0,4194304] 0 2024-04-27T04:10:01.522 INFO:tasks.workunit.client.1.smithi181.stdout:2/490: chown d4/d9/f6d 2913 1 2024-04-27T04:10:01.522 INFO:tasks.workunit.client.1.smithi181.stdout:5/635: creat d4/de/d2c/d46/d99/dad/fe2 x:0 0 0 2024-04-27T04:10:01.524 INFO:tasks.workunit.client.1.smithi181.stdout:5/636: write d4/de/d2c/d46/d4d/d8e/f41 [3501118,105718] 0 2024-04-27T04:10:01.525 INFO:tasks.workunit.client.1.smithi181.stdout:2/491: mkdir d4/d9/d35/d9f 0 2024-04-27T04:10:01.527 INFO:tasks.workunit.client.1.smithi181.stdout:5/637: write d4/de/d2c/d2e/d55/f3f [4990581,29893] 0 2024-04-27T04:10:01.530 INFO:tasks.workunit.client.1.smithi181.stdout:5/638: creat d4/d10/d72/fe3 x:0 0 0 2024-04-27T04:10:01.532 INFO:tasks.workunit.client.1.smithi181.stdout:5/639: creat d4/de/d2c/d46/d99/dad/fe4 x:0 0 0 2024-04-27T04:10:01.533 INFO:tasks.workunit.client.0.smithi162.stdout:4/297: dwrite d2/da/f12 [0,4194304] 0 2024-04-27T04:10:01.533 INFO:tasks.workunit.client.0.smithi162.stdout:7/210: getdents d3/d5/d3c 0 2024-04-27T04:10:01.533 INFO:tasks.workunit.client.0.smithi162.stdout:1/229: creat d0/d11/f4d x:0 0 0 2024-04-27T04:10:01.533 INFO:tasks.workunit.client.0.smithi162.stdout:7/211: chown d3 213 1 2024-04-27T04:10:01.535 INFO:tasks.workunit.client.0.smithi162.stdout:8/287: truncate d0/f7 3844929 0 2024-04-27T04:10:01.535 INFO:tasks.workunit.client.0.smithi162.stdout:0/297: dwrite d5/f9 [4194304,4194304] 0 2024-04-27T04:10:01.538 INFO:tasks.workunit.client.0.smithi162.stdout:5/234: link d8/f12 d8/d1a/d23/d40/d4b/f52 0 2024-04-27T04:10:01.540 INFO:tasks.workunit.client.0.smithi162.stdout:5/235: read d8/d1a/f17 [135749,73417] 0 2024-04-27T04:10:01.540 INFO:tasks.workunit.client.0.smithi162.stdout:7/212: rmdir d3/d5/d9/dc 39 2024-04-27T04:10:01.541 INFO:tasks.workunit.client.0.smithi162.stdout:5/236: fdatasync d8/d9/fc 0 2024-04-27T04:10:01.542 INFO:tasks.workunit.client.0.smithi162.stdout:0/298: fsync d5/dc/f17 0 2024-04-27T04:10:01.544 INFO:tasks.workunit.client.0.smithi162.stdout:8/288: creat d0/d10/d2b/d2d/d2f/d39/f59 x:0 0 0 2024-04-27T04:10:01.546 INFO:tasks.workunit.client.0.smithi162.stdout:4/298: write d2/da/d1a/f1b [347653,117171] 0 2024-04-27T04:10:01.547 INFO:tasks.workunit.client.0.smithi162.stdout:1/230: mknod d0/d11/c4e 0 2024-04-27T04:10:01.547 INFO:tasks.workunit.client.0.smithi162.stdout:3/257: dread d1/f4c [0,4194304] 0 2024-04-27T04:10:01.550 INFO:tasks.workunit.client.1.smithi181.stdout:2/492: dread d4/d9/f2d [0,4194304] 0 2024-04-27T04:10:01.551 INFO:tasks.workunit.client.0.smithi162.stdout:0/299: symlink d5/dc/d37/l65 0 2024-04-27T04:10:01.552 INFO:tasks.workunit.client.0.smithi162.stdout:8/289: creat d0/d10/d2b/d2d/d2f/d39/f5a x:0 0 0 2024-04-27T04:10:01.553 INFO:tasks.workunit.client.1.smithi181.stdout:2/493: symlink d4/d9/d35/d42/la0 0 2024-04-27T04:10:01.555 INFO:tasks.workunit.client.0.smithi162.stdout:7/213: rename l1 to d3/d5/d9/dc/de/l46 0 2024-04-27T04:10:01.555 INFO:tasks.workunit.client.1.smithi181.stdout:2/494: read d4/d9/d16/f23 [3089051,129663] 0 2024-04-27T04:10:01.556 INFO:tasks.workunit.client.1.smithi181.stdout:2/495: write d4/d9/d35/d42/f5a [3735534,13985] 0 2024-04-27T04:10:01.557 INFO:tasks.workunit.client.0.smithi162.stdout:1/231: dwrite d0/d11/d3e/f41 [0,4194304] 0 2024-04-27T04:10:01.557 INFO:tasks.workunit.client.0.smithi162.stdout:9/251: dread d3/d11/f4d [0,4194304] 0 2024-04-27T04:10:01.569 INFO:tasks.workunit.client.1.smithi181.stdout:2/496: stat d4/d9/d35/c9c 0 2024-04-27T04:10:01.570 INFO:tasks.workunit.client.0.smithi162.stdout:7/214: symlink d3/d1d/l47 0 2024-04-27T04:10:01.571 INFO:tasks.workunit.client.1.smithi181.stdout:5/640: dread d4/de/f47 [0,4194304] 0 2024-04-27T04:10:01.572 INFO:tasks.workunit.client.0.smithi162.stdout:9/252: fsync d3/f5 0 2024-04-27T04:10:01.572 INFO:tasks.workunit.client.1.smithi181.stdout:5/641: dread - d4/d10/fc4 zero size 2024-04-27T04:10:01.574 INFO:tasks.workunit.client.0.smithi162.stdout:1/232: rename d0/d11/f22 to d0/d11/d3e/f4f 0 2024-04-27T04:10:01.575 INFO:tasks.workunit.client.1.smithi181.stdout:0/525: dwrite dc/d1f/d3b/d4a/d95/d26/f28 [0,4194304] 0 2024-04-27T04:10:01.576 INFO:tasks.workunit.client.0.smithi162.stdout:1/233: write d0/d7/d10/f4c [787438,98747] 0 2024-04-27T04:10:01.577 INFO:tasks.workunit.client.0.smithi162.stdout:6/257: sync 2024-04-27T04:10:01.577 INFO:tasks.workunit.client.0.smithi162.stdout:5/237: dread d8/f12 [4194304,4194304] 0 2024-04-27T04:10:01.579 INFO:tasks.workunit.client.0.smithi162.stdout:5/238: readlink d8/la 0 2024-04-27T04:10:01.579 INFO:tasks.workunit.client.0.smithi162.stdout:1/234: write d0/d11/f2e [1240981,48229] 0 2024-04-27T04:10:01.580 INFO:tasks.workunit.client.0.smithi162.stdout:3/258: creat d1/d37/f51 x:0 0 0 2024-04-27T04:10:01.581 INFO:tasks.workunit.client.0.smithi162.stdout:9/253: dwrite d3/d13/d2d/d3b/f47 [0,4194304] 0 2024-04-27T04:10:01.581 INFO:tasks.workunit.client.0.smithi162.stdout:1/235: chown d0/d7 5060 1 2024-04-27T04:10:01.584 INFO:tasks.workunit.client.0.smithi162.stdout:0/300: symlink d5/dc/d1e/d56/d57/l66 0 2024-04-27T04:10:01.585 INFO:tasks.workunit.client.1.smithi181.stdout:0/526: dwrite dc/f18 [0,4194304] 0 2024-04-27T04:10:01.585 INFO:tasks.workunit.client.1.smithi181.stdout:2/497: dwrite d4/d9/d35/d42/d86/f8a [0,4194304] 0 2024-04-27T04:10:01.589 INFO:tasks.workunit.client.0.smithi162.stdout:5/239: dwrite d8/f22 [0,4194304] 0 2024-04-27T04:10:01.591 INFO:tasks.workunit.client.0.smithi162.stdout:7/215: mknod d3/d5/d9/dc/c48 0 2024-04-27T04:10:01.601 INFO:tasks.workunit.client.0.smithi162.stdout:7/216: dwrite d3/d5/d9/dc/f19 [0,4194304] 0 2024-04-27T04:10:01.603 INFO:tasks.workunit.client.0.smithi162.stdout:7/217: write d3/d5/d9/dc/f27 [577283,90689] 0 2024-04-27T04:10:01.606 INFO:tasks.workunit.client.0.smithi162.stdout:6/258: truncate f5 1936161 0 2024-04-27T04:10:01.606 INFO:tasks.workunit.client.1.smithi181.stdout:0/527: creat dc/d1f/d6b/db9/dbd/fbe x:0 0 0 2024-04-27T04:10:01.614 INFO:tasks.workunit.client.1.smithi181.stdout:0/528: dwrite dc/f94 [0,4194304] 0 2024-04-27T04:10:01.616 INFO:tasks.workunit.client.0.smithi162.stdout:0/301: unlink d5/dc/d37/f3a 0 2024-04-27T04:10:01.616 INFO:tasks.workunit.client.0.smithi162.stdout:0/302: chown d5/d26/c44 549 1 2024-04-27T04:10:01.620 INFO:tasks.workunit.client.1.smithi181.stdout:5/642: link d4/de/d2f/d7a/cd5 d4/d10/d29/d92/ce5 0 2024-04-27T04:10:01.622 INFO:tasks.workunit.client.0.smithi162.stdout:5/240: truncate d8/d9/f16 1663314 0 2024-04-27T04:10:01.625 INFO:tasks.workunit.client.1.smithi181.stdout:5/643: mknod d4/de/d2c/d2e/d55/ce6 0 2024-04-27T04:10:01.629 INFO:tasks.workunit.client.0.smithi162.stdout:1/236: mkdir d0/d11/d50 0 2024-04-27T04:10:01.630 INFO:tasks.workunit.client.0.smithi162.stdout:1/237: write d0/d11/f2e [465092,108563] 0 2024-04-27T04:10:01.632 INFO:tasks.workunit.client.1.smithi181.stdout:0/529: creat dc/d1f/d3b/fbf x:0 0 0 2024-04-27T04:10:01.634 INFO:tasks.workunit.client.1.smithi181.stdout:5/644: fdatasync d4/d10/d72/f77 0 2024-04-27T04:10:01.635 INFO:tasks.workunit.client.0.smithi162.stdout:1/238: dwrite d0/f3b [0,4194304] 0 2024-04-27T04:10:01.637 INFO:tasks.workunit.client.0.smithi162.stdout:0/303: stat d5/dc/c45 0 2024-04-27T04:10:01.637 INFO:tasks.workunit.client.0.smithi162.stdout:0/304: dread - d5/f11 zero size 2024-04-27T04:10:01.638 INFO:tasks.workunit.client.0.smithi162.stdout:8/290: link d0/l9 d0/l5b 0 2024-04-27T04:10:01.638 INFO:tasks.workunit.client.0.smithi162.stdout:0/305: fsync d5/f16 0 2024-04-27T04:10:01.639 INFO:tasks.workunit.client.0.smithi162.stdout:0/306: stat d5/dc/d1e/d41/f39 0 2024-04-27T04:10:01.640 INFO:tasks.workunit.client.1.smithi181.stdout:0/530: dwrite dc/d1f/d3b/f44 [4194304,4194304] 0 2024-04-27T04:10:01.641 INFO:tasks.workunit.client.0.smithi162.stdout:5/241: mknod d8/d1a/d23/c53 0 2024-04-27T04:10:01.641 INFO:tasks.workunit.client.0.smithi162.stdout:5/242: stat d8/d1a/c1f 0 2024-04-27T04:10:01.643 INFO:tasks.workunit.client.1.smithi181.stdout:2/498: write d4/d9/df/d14/f15 [6399977,54622] 0 2024-04-27T04:10:01.645 INFO:tasks.workunit.client.1.smithi181.stdout:9/491: sync 2024-04-27T04:10:01.645 INFO:tasks.workunit.client.1.smithi181.stdout:9/492: write d0/d4d/f7f [123679,48639] 0 2024-04-27T04:10:01.648 INFO:tasks.workunit.client.1.smithi181.stdout:9/493: read d0/d44/d4b/d43/f56 [17607,62599] 0 2024-04-27T04:10:01.649 INFO:tasks.workunit.client.0.smithi162.stdout:9/254: write d3/f4 [3436491,8042] 0 2024-04-27T04:10:01.650 INFO:tasks.workunit.client.1.smithi181.stdout:9/494: chown d0/d4d/f7e 33066268 1 2024-04-27T04:10:01.651 INFO:tasks.workunit.client.0.smithi162.stdout:6/259: dwrite d6/d1a/f2b [0,4194304] 0 2024-04-27T04:10:01.653 INFO:tasks.workunit.client.0.smithi162.stdout:1/239: symlink d0/d7/d3c/l51 0 2024-04-27T04:10:01.653 INFO:tasks.workunit.client.0.smithi162.stdout:2/194: sync 2024-04-27T04:10:01.654 INFO:tasks.workunit.client.0.smithi162.stdout:6/260: chown d6/d1a/c19 1590 1 2024-04-27T04:10:01.659 INFO:tasks.workunit.client.0.smithi162.stdout:8/291: fdatasync d0/d10/d2b/f19 0 2024-04-27T04:10:01.659 INFO:tasks.workunit.client.0.smithi162.stdout:9/255: dwrite d3/d13/f26 [0,4194304] 0 2024-04-27T04:10:01.661 INFO:tasks.workunit.client.1.smithi181.stdout:5/645: dread d4/da3/fbd [0,4194304] 0 2024-04-27T04:10:01.663 INFO:tasks.workunit.client.1.smithi181.stdout:9/495: symlink d0/d44/d4f/lac 0 2024-04-27T04:10:01.666 INFO:tasks.workunit.client.0.smithi162.stdout:5/243: symlink d8/d9/l54 0 2024-04-27T04:10:01.667 INFO:tasks.workunit.client.1.smithi181.stdout:5/646: dread d4/de/d2c/d46/d4d/d8e/d67/d71/f91 [0,4194304] 0 2024-04-27T04:10:01.670 INFO:tasks.workunit.client.0.smithi162.stdout:0/307: dwrite d5/dc/d1e/d46/f55 [0,4194304] 0 2024-04-27T04:10:01.670 INFO:tasks.workunit.client.0.smithi162.stdout:0/308: fdatasync d5/d26/d4a/f62 0 2024-04-27T04:10:01.672 INFO:tasks.workunit.client.1.smithi181.stdout:9/496: dwrite d0/d40/f25 [0,4194304] 0 2024-04-27T04:10:01.675 INFO:tasks.workunit.client.0.smithi162.stdout:0/309: dread d5/f2d [4194304,4194304] 0 2024-04-27T04:10:01.678 INFO:tasks.workunit.client.0.smithi162.stdout:1/240: rename d0/d7/d3c/l51 to d0/d11/d50/l52 0 2024-04-27T04:10:01.683 INFO:tasks.workunit.client.1.smithi181.stdout:0/531: write dc/d1f/d3b/d4a/d95/f36 [658799,124656] 0 2024-04-27T04:10:01.684 INFO:tasks.workunit.client.1.smithi181.stdout:0/532: chown dc/f10 0 1 2024-04-27T04:10:01.686 INFO:tasks.workunit.client.0.smithi162.stdout:4/299: sync 2024-04-27T04:10:01.686 INFO:tasks.workunit.client.0.smithi162.stdout:3/259: sync 2024-04-27T04:10:01.687 INFO:tasks.workunit.client.0.smithi162.stdout:5/244: readlink d8/l2b 0 2024-04-27T04:10:01.687 INFO:tasks.workunit.client.1.smithi181.stdout:2/499: write d4/d9/d44/f46 [997938,107644] 0 2024-04-27T04:10:01.690 INFO:tasks.workunit.client.0.smithi162.stdout:0/310: rmdir d5/dc/d1e/d41 39 2024-04-27T04:10:01.692 INFO:tasks.workunit.client.0.smithi162.stdout:0/311: write d5/f2d [6093382,119653] 0 2024-04-27T04:10:01.692 INFO:tasks.workunit.client.0.smithi162.stdout:1/241: mkdir d0/d11/d50/d53 0 2024-04-27T04:10:01.695 INFO:tasks.workunit.client.0.smithi162.stdout:5/245: dread d8/f22 [0,4194304] 0 2024-04-27T04:10:01.695 INFO:tasks.workunit.client.0.smithi162.stdout:4/300: dwrite d2/da/f12 [0,4194304] 0 2024-04-27T04:10:01.695 INFO:tasks.workunit.client.0.smithi162.stdout:9/256: mknod d3/d13/c56 0 2024-04-27T04:10:01.695 INFO:tasks.workunit.client.1.smithi181.stdout:2/500: creat d4/d9/fa1 x:0 0 0 2024-04-27T04:10:01.696 INFO:tasks.workunit.client.0.smithi162.stdout:0/312: dread - d5/dc/d1e/d46/f52 zero size 2024-04-27T04:10:01.696 INFO:tasks.workunit.client.1.smithi181.stdout:2/501: stat d4/d9/d4f/d79/d94/f49 0 2024-04-27T04:10:01.697 INFO:tasks.workunit.client.0.smithi162.stdout:3/260: creat d1/d18/f52 x:0 0 0 2024-04-27T04:10:01.700 INFO:tasks.workunit.client.1.smithi181.stdout:2/502: dread d4/d9/d4f/d79/d94/f49 [0,4194304] 0 2024-04-27T04:10:01.700 INFO:tasks.workunit.client.0.smithi162.stdout:2/195: rename d0/d4/d17/d2b/c35 to d0/d20/d2e/c39 0 2024-04-27T04:10:01.700 INFO:tasks.workunit.client.0.smithi162.stdout:5/246: stat d8/d1a/d23/d40/d4b 0 2024-04-27T04:10:01.700 INFO:tasks.workunit.client.0.smithi162.stdout:1/242: mkdir d0/d7/d3c/d54 0 2024-04-27T04:10:01.701 INFO:tasks.workunit.client.0.smithi162.stdout:4/301: chown d2/d25/l37 2 1 2024-04-27T04:10:01.703 INFO:tasks.workunit.client.0.smithi162.stdout:8/292: dwrite d0/d10/d2b/f19 [0,4194304] 0 2024-04-27T04:10:01.704 INFO:tasks.workunit.client.0.smithi162.stdout:9/257: link d3/d23/f4b d3/d13/d15/f57 0 2024-04-27T04:10:01.705 INFO:tasks.workunit.client.0.smithi162.stdout:5/247: write d8/d1a/d23/d40/f46 [505866,76076] 0 2024-04-27T04:10:01.706 INFO:tasks.workunit.client.0.smithi162.stdout:1/243: truncate d0/d7/f32 4493192 0 2024-04-27T04:10:01.706 INFO:tasks.workunit.client.1.smithi181.stdout:2/503: dwrite d4/d9/f11 [4194304,4194304] 0 2024-04-27T04:10:01.707 INFO:tasks.workunit.client.0.smithi162.stdout:8/293: stat d0/d10/d2b/d23/l41 0 2024-04-27T04:10:01.708 INFO:tasks.workunit.client.0.smithi162.stdout:0/313: creat d5/d60/f67 x:0 0 0 2024-04-27T04:10:01.709 INFO:tasks.workunit.client.0.smithi162.stdout:4/302: creat d2/d25/d1c/d39/f5f x:0 0 0 2024-04-27T04:10:01.710 INFO:tasks.workunit.client.1.smithi181.stdout:9/497: write d0/f13 [3004926,91418] 0 2024-04-27T04:10:01.712 INFO:tasks.workunit.client.0.smithi162.stdout:1/244: dwrite d0/d7/d10/f4c [0,4194304] 0 2024-04-27T04:10:01.715 INFO:tasks.workunit.client.0.smithi162.stdout:5/248: mkdir d8/d9/d55 0 2024-04-27T04:10:01.718 INFO:tasks.workunit.client.1.smithi181.stdout:9/498: symlink d0/d44/d4b/d5f/lad 0 2024-04-27T04:10:01.720 INFO:tasks.workunit.client.1.smithi181.stdout:0/533: dwrite dc/f39 [4194304,4194304] 0 2024-04-27T04:10:01.723 INFO:tasks.workunit.client.1.smithi181.stdout:9/499: creat d0/d44/d4f/fae x:0 0 0 2024-04-27T04:10:01.725 INFO:tasks.workunit.client.1.smithi181.stdout:9/500: write d0/d44/d4b/d43/f98 [898145,125765] 0 2024-04-27T04:10:01.731 INFO:tasks.workunit.client.0.smithi162.stdout:4/303: creat d2/da/dd/f60 x:0 0 0 2024-04-27T04:10:01.732 INFO:tasks.workunit.client.1.smithi181.stdout:9/501: dwrite d0/d40/f25 [0,4194304] 0 2024-04-27T04:10:01.732 INFO:tasks.workunit.client.0.smithi162.stdout:2/196: creat d0/d4/f3a x:0 0 0 2024-04-27T04:10:01.734 INFO:tasks.workunit.client.0.smithi162.stdout:7/218: sync 2024-04-27T04:10:01.736 INFO:tasks.workunit.client.0.smithi162.stdout:7/219: chown d3/d5/d9/dc/f2f 314 1 2024-04-27T04:10:01.737 INFO:tasks.workunit.client.0.smithi162.stdout:6/261: rename d6/db/l14 to d6/l59 0 2024-04-27T04:10:01.737 INFO:tasks.workunit.client.1.smithi181.stdout:2/504: write d4/d9/f6d [581686,52513] 0 2024-04-27T04:10:01.737 INFO:tasks.workunit.client.1.smithi181.stdout:9/502: truncate d0/d40/f84 97438 0 2024-04-27T04:10:01.738 INFO:tasks.workunit.client.1.smithi181.stdout:9/503: write d0/f13 [4285407,90227] 0 2024-04-27T04:10:01.739 INFO:tasks.workunit.client.0.smithi162.stdout:9/258: mkdir d3/d11/d45/d4e/d58 0 2024-04-27T04:10:01.742 INFO:tasks.workunit.client.0.smithi162.stdout:4/304: creat d2/da/dd/f61 x:0 0 0 2024-04-27T04:10:01.742 INFO:tasks.workunit.client.1.smithi181.stdout:2/505: read d4/d9/d16/f26 [2115386,61583] 0 2024-04-27T04:10:01.742 INFO:tasks.workunit.client.1.smithi181.stdout:9/504: mkdir d0/d44/d4b/d43/d89/daf 0 2024-04-27T04:10:01.743 INFO:tasks.workunit.client.0.smithi162.stdout:4/305: stat d2/d25/d1c/d39/f59 0 2024-04-27T04:10:01.743 INFO:tasks.workunit.client.1.smithi181.stdout:0/534: fsync dc/f39 0 2024-04-27T04:10:01.745 INFO:tasks.workunit.client.0.smithi162.stdout:2/197: mkdir d0/d4/d2a/d3b 0 2024-04-27T04:10:01.747 INFO:tasks.workunit.client.1.smithi181.stdout:2/506: truncate d4/d9/d16/d45/f66 210508 0 2024-04-27T04:10:01.747 INFO:tasks.workunit.client.0.smithi162.stdout:4/306: dread d2/f11 [0,4194304] 0 2024-04-27T04:10:01.747 INFO:tasks.workunit.client.1.smithi181.stdout:2/507: stat d4/d9/df/l22 0 2024-04-27T04:10:01.749 INFO:tasks.workunit.client.0.smithi162.stdout:6/262: rmdir d6/d1a/d34/d3a 39 2024-04-27T04:10:01.750 INFO:tasks.workunit.client.0.smithi162.stdout:6/263: chown d6/db/c2e 23154004 1 2024-04-27T04:10:01.750 INFO:tasks.workunit.client.0.smithi162.stdout:2/198: dread d0/d4/d2a/f36 [0,4194304] 0 2024-04-27T04:10:01.756 INFO:tasks.workunit.client.1.smithi181.stdout:2/508: unlink d4/d9/d4f/d79/d94/f49 0 2024-04-27T04:10:01.758 INFO:tasks.workunit.client.0.smithi162.stdout:5/249: write d8/f22 [4960149,114263] 0 2024-04-27T04:10:01.759 INFO:tasks.workunit.client.1.smithi181.stdout:2/509: creat d4/d9/d4f/d79/d98/fa2 x:0 0 0 2024-04-27T04:10:01.761 INFO:tasks.workunit.client.0.smithi162.stdout:8/294: write d0/f4 [70899,52082] 0 2024-04-27T04:10:01.762 INFO:tasks.workunit.client.0.smithi162.stdout:8/295: stat d0/d10/d2b/d2e 0 2024-04-27T04:10:01.764 INFO:tasks.workunit.client.0.smithi162.stdout:1/245: truncate d0/d11/f2c 8361903 0 2024-04-27T04:10:01.765 INFO:tasks.workunit.client.0.smithi162.stdout:2/199: creat d0/d4/d17/f3c x:0 0 0 2024-04-27T04:10:01.767 INFO:tasks.workunit.client.1.smithi181.stdout:2/510: link d4/d81/f84 d4/d9/d4f/d79/fa3 0 2024-04-27T04:10:01.768 INFO:tasks.workunit.client.1.smithi181.stdout:2/511: write d4/f8 [6194537,108928] 0 2024-04-27T04:10:01.768 INFO:tasks.workunit.client.0.smithi162.stdout:0/314: getdents d5/dc/d1e/d46 0 2024-04-27T04:10:01.769 INFO:tasks.workunit.client.1.smithi181.stdout:0/535: write dc/d1f/d3b/d4a/d95/d26/d41/f4e [5295901,43315] 0 2024-04-27T04:10:01.770 INFO:tasks.workunit.client.1.smithi181.stdout:2/512: fdatasync d4/d9/df/d14/f78 0 2024-04-27T04:10:01.771 INFO:tasks.workunit.client.1.smithi181.stdout:9/505: truncate d0/d44/d4b/f39 378002 0 2024-04-27T04:10:01.774 INFO:tasks.workunit.client.0.smithi162.stdout:4/307: dread d2/f38 [0,4194304] 0 2024-04-27T04:10:01.774 INFO:tasks.workunit.client.0.smithi162.stdout:7/220: truncate d3/d5/d9/f17 3552346 0 2024-04-27T04:10:01.776 INFO:tasks.workunit.client.0.smithi162.stdout:3/261: rename d1/d21/f39 to d1/d18/d23/f53 0 2024-04-27T04:10:01.777 INFO:tasks.workunit.client.0.smithi162.stdout:3/262: stat d1/f4c 0 2024-04-27T04:10:01.784 INFO:tasks.workunit.client.1.smithi181.stdout:9/506: dread d0/f1e [0,4194304] 0 2024-04-27T04:10:01.784 INFO:tasks.workunit.client.1.smithi181.stdout:0/536: dwrite dc/d1f/d3b/f5f [4194304,4194304] 0 2024-04-27T04:10:01.785 INFO:tasks.workunit.client.0.smithi162.stdout:0/315: mknod d5/dc/d1e/d56/d57/c68 0 2024-04-27T04:10:01.786 INFO:tasks.workunit.client.0.smithi162.stdout:0/316: write d5/dc/d1e/d46/f52 [857709,74239] 0 2024-04-27T04:10:01.787 INFO:tasks.workunit.client.0.smithi162.stdout:5/250: mknod d8/d2c/d41/c56 0 2024-04-27T04:10:01.793 INFO:tasks.workunit.client.1.smithi181.stdout:0/537: dwrite dc/d1f/d3b/d4a/d95/f36 [0,4194304] 0 2024-04-27T04:10:01.794 INFO:tasks.workunit.client.1.smithi181.stdout:0/538: chown l5 110 1 2024-04-27T04:10:01.795 INFO:tasks.workunit.client.1.smithi181.stdout:9/507: unlink d0/d4d/d85/ca8 0 2024-04-27T04:10:01.798 INFO:tasks.workunit.client.0.smithi162.stdout:9/259: rename d3/d11/l16 to d3/d13/d2d/d4f/l59 0 2024-04-27T04:10:01.806 INFO:tasks.workunit.client.1.smithi181.stdout:2/513: dwrite d4/d9/d35/f51 [0,4194304] 0 2024-04-27T04:10:01.806 INFO:tasks.workunit.client.1.smithi181.stdout:0/539: fsync dc/d1f/d3b/f66 0 2024-04-27T04:10:01.807 INFO:tasks.workunit.client.1.smithi181.stdout:0/540: chown dc/d1f/d6b/d7e/d91 30 1 2024-04-27T04:10:01.809 INFO:tasks.workunit.client.1.smithi181.stdout:0/541: fsync dc/d1f/d3b/d4a/d95/d26/d41/d6f/d7d/f9b 0 2024-04-27T04:10:01.812 INFO:tasks.workunit.client.1.smithi181.stdout:9/508: fdatasync d0/f9 0 2024-04-27T04:10:01.814 INFO:tasks.workunit.client.1.smithi181.stdout:2/514: mkdir d4/d9/d44/da4 0 2024-04-27T04:10:01.819 INFO:tasks.workunit.client.1.smithi181.stdout:9/509: symlink d0/d4d/da0/lb0 0 2024-04-27T04:10:01.820 INFO:tasks.workunit.client.1.smithi181.stdout:9/510: dread - d0/d40/f9c zero size 2024-04-27T04:10:01.820 INFO:tasks.workunit.client.1.smithi181.stdout:2/515: stat d4/d9/df/d14/f24 0 2024-04-27T04:10:01.822 INFO:tasks.workunit.client.1.smithi181.stdout:0/542: link dc/f39 dc/d1f/d6b/fc0 0 2024-04-27T04:10:01.823 INFO:tasks.workunit.client.0.smithi162.stdout:3/263: stat d1/f20 0 2024-04-27T04:10:01.823 INFO:tasks.workunit.client.0.smithi162.stdout:8/296: unlink d0/fc 0 2024-04-27T04:10:01.825 INFO:tasks.workunit.client.1.smithi181.stdout:2/516: rename d4/d9/d16/f23 to d4/d9/d16/d82/fa5 0 2024-04-27T04:10:01.825 INFO:tasks.workunit.client.0.smithi162.stdout:1/246: write d0/d7/d10/f3d [4838235,38962] 0 2024-04-27T04:10:01.826 INFO:tasks.workunit.client.1.smithi181.stdout:0/543: creat dc/d1f/d3b/d4a/fc1 x:0 0 0 2024-04-27T04:10:01.827 INFO:tasks.workunit.client.0.smithi162.stdout:2/200: link d0/d20/c21 d0/d20/c3d 0 2024-04-27T04:10:01.828 INFO:tasks.workunit.client.0.smithi162.stdout:0/317: stat d5/dc/d1e/d41/l51 0 2024-04-27T04:10:01.829 INFO:tasks.workunit.client.0.smithi162.stdout:5/251: mkdir d8/d9/d55/d57 0 2024-04-27T04:10:01.829 INFO:tasks.workunit.client.0.smithi162.stdout:2/201: write d0/d4/f22 [2851826,104881] 0 2024-04-27T04:10:01.830 INFO:tasks.workunit.client.1.smithi181.stdout:0/544: creat dc/d1f/d3b/d4a/d95/d26/d41/fc2 x:0 0 0 2024-04-27T04:10:01.830 INFO:tasks.workunit.client.0.smithi162.stdout:5/252: fdatasync d8/d2c/f33 0 2024-04-27T04:10:01.831 INFO:tasks.workunit.client.0.smithi162.stdout:8/297: dwrite d0/d10/d2b/d2d/d2f/d42/f48 [0,4194304] 0 2024-04-27T04:10:01.833 INFO:tasks.workunit.client.1.smithi181.stdout:0/545: symlink dc/d1f/d6b/db9/lc3 0 2024-04-27T04:10:01.834 INFO:tasks.workunit.client.1.smithi181.stdout:9/511: dread d0/d44/f46 [0,4194304] 0 2024-04-27T04:10:01.836 INFO:tasks.workunit.client.0.smithi162.stdout:3/264: rename d1/d21/c43 to d1/d18/c54 0 2024-04-27T04:10:01.836 INFO:tasks.workunit.client.0.smithi162.stdout:3/265: fsync d1/d18/d23/d2f/f42 0 2024-04-27T04:10:01.839 INFO:tasks.workunit.client.1.smithi181.stdout:0/546: creat dc/d1f/d3b/d4a/d95/d6d/fc4 x:0 0 0 2024-04-27T04:10:01.839 INFO:tasks.workunit.client.1.smithi181.stdout:0/547: dread - dc/d1f/d3b/fbf zero size 2024-04-27T04:10:01.841 INFO:tasks.workunit.client.1.smithi181.stdout:0/548: chown dc/d1f/d3b/d75/cad 2714 1 2024-04-27T04:10:01.841 INFO:tasks.workunit.client.1.smithi181.stdout:9/512: mknod d0/d44/d4b/d5f/da4/cb1 0 2024-04-27T04:10:01.845 INFO:tasks.workunit.client.0.smithi162.stdout:0/318: mkdir d5/d26/d69 0 2024-04-27T04:10:01.846 INFO:tasks.workunit.client.0.smithi162.stdout:0/319: write d5/f35 [1586626,110051] 0 2024-04-27T04:10:01.849 INFO:tasks.workunit.client.1.smithi181.stdout:2/517: truncate d4/d9/df/d14/f24 2008874 0 2024-04-27T04:10:01.851 INFO:tasks.workunit.client.1.smithi181.stdout:0/549: unlink dc/d1f/d3b/d4a/d95/d26/d41/d6f/c89 0 2024-04-27T04:10:01.854 INFO:tasks.workunit.client.1.smithi181.stdout:9/513: rename d0/f1f to d0/d40/fb2 0 2024-04-27T04:10:01.854 INFO:tasks.workunit.client.0.smithi162.stdout:9/260: truncate d3/d13/f26 1629911 0 2024-04-27T04:10:01.855 INFO:tasks.workunit.client.1.smithi181.stdout:2/518: rmdir d4/d9/d35 39 2024-04-27T04:10:01.859 INFO:tasks.workunit.client.0.smithi162.stdout:2/202: write d0/f1 [1024096,127911] 0 2024-04-27T04:10:01.860 INFO:tasks.workunit.client.0.smithi162.stdout:6/264: sync 2024-04-27T04:10:01.863 INFO:tasks.workunit.client.1.smithi181.stdout:2/519: symlink d4/d9/d16/la6 0 2024-04-27T04:10:01.865 INFO:tasks.workunit.client.1.smithi181.stdout:2/520: dread d4/d9/d16/d45/f6c [4194304,4194304] 0 2024-04-27T04:10:01.870 INFO:tasks.workunit.client.1.smithi181.stdout:9/514: mknod d0/d44/d4b/d43/d8f/cb3 0 2024-04-27T04:10:01.870 INFO:tasks.workunit.client.1.smithi181.stdout:0/550: write dc/d1f/d3b/d4a/d95/f51 [377792,4594] 0 2024-04-27T04:10:01.870 INFO:tasks.workunit.client.0.smithi162.stdout:0/320: readlink d5/dc/d1e/d20/l54 0 2024-04-27T04:10:01.872 INFO:tasks.workunit.client.0.smithi162.stdout:1/247: truncate d0/d7/f32 4397215 0 2024-04-27T04:10:01.875 INFO:tasks.workunit.client.0.smithi162.stdout:8/298: mknod d0/c5c 0 2024-04-27T04:10:01.877 INFO:tasks.workunit.client.0.smithi162.stdout:8/299: dread d0/d10/d44/f47 [0,4194304] 0 2024-04-27T04:10:01.879 INFO:tasks.workunit.client.0.smithi162.stdout:5/253: mkdir d8/d1a/d23/d40/d4b/d58 0 2024-04-27T04:10:01.880 INFO:tasks.workunit.client.1.smithi181.stdout:0/551: mknod dc/d1f/d3b/db2/cc5 0 2024-04-27T04:10:01.881 INFO:tasks.workunit.client.0.smithi162.stdout:8/300: dwrite d0/fd [0,4194304] 0 2024-04-27T04:10:01.882 INFO:tasks.workunit.client.0.smithi162.stdout:7/221: link d3/d5/d9/dc/de/l46 d3/d5/d9/l49 0 2024-04-27T04:10:01.883 INFO:tasks.workunit.client.0.smithi162.stdout:8/301: stat d0/d10/d2b/d2d/d2f/c4e 0 2024-04-27T04:10:01.885 INFO:tasks.workunit.client.0.smithi162.stdout:9/261: rename d3/d13/d15/f34 to d3/d23/d1e/d43/f5a 0 2024-04-27T04:10:01.886 INFO:tasks.workunit.client.1.smithi181.stdout:2/521: write d4/d9/d4f/f5b [81872,10468] 0 2024-04-27T04:10:01.892 INFO:tasks.workunit.client.0.smithi162.stdout:2/203: chown d0/d20/d2e/c39 2 1 2024-04-27T04:10:01.893 INFO:tasks.workunit.client.0.smithi162.stdout:2/204: chown d0/d4/d2a/f36 0 1 2024-04-27T04:10:01.894 INFO:tasks.workunit.client.0.smithi162.stdout:2/205: write d0/d24/f38 [732265,70861] 0 2024-04-27T04:10:01.895 INFO:tasks.workunit.client.0.smithi162.stdout:6/265: mkdir d6/db/d32/d5a 0 2024-04-27T04:10:01.895 INFO:tasks.workunit.client.0.smithi162.stdout:2/206: truncate d0/d4/d17/f3c 481409 0 2024-04-27T04:10:01.895 INFO:tasks.workunit.client.0.smithi162.stdout:3/266: truncate d1/d18/f11 1888705 0 2024-04-27T04:10:01.896 INFO:tasks.workunit.client.1.smithi181.stdout:2/522: mkdir d4/d9/df/d14/da7 0 2024-04-27T04:10:01.898 INFO:tasks.workunit.client.0.smithi162.stdout:0/321: creat d5/d26/f6a x:0 0 0 2024-04-27T04:10:01.899 INFO:tasks.workunit.client.1.smithi181.stdout:2/523: fsync d4/d9/d16/d45/f6c 0 2024-04-27T04:10:01.900 INFO:tasks.workunit.client.1.smithi181.stdout:2/524: dread - d4/d9/d4f/d79/d94/f95 zero size 2024-04-27T04:10:01.901 INFO:tasks.workunit.client.1.smithi181.stdout:0/552: mknod dc/d1f/d3b/d4a/d95/d26/d41/d6f/d7d/cc6 0 2024-04-27T04:10:01.901 INFO:tasks.workunit.client.1.smithi181.stdout:2/525: readlink d4/d9/d16/la6 0 2024-04-27T04:10:01.903 INFO:tasks.workunit.client.0.smithi162.stdout:5/254: creat d8/d1a/f59 x:0 0 0 2024-04-27T04:10:01.910 INFO:tasks.workunit.client.0.smithi162.stdout:9/262: creat d3/d13/d15/d50/f5b x:0 0 0 2024-04-27T04:10:01.911 INFO:tasks.workunit.client.0.smithi162.stdout:4/308: sync 2024-04-27T04:10:01.912 INFO:tasks.workunit.client.1.smithi181.stdout:0/553: write f6 [4396467,38061] 0 2024-04-27T04:10:01.912 INFO:tasks.workunit.client.0.smithi162.stdout:6/266: readlink d6/l59 0 2024-04-27T04:10:01.913 INFO:tasks.workunit.client.1.smithi181.stdout:0/554: chown dc/d1f/d3b/f44 87640 1 2024-04-27T04:10:01.914 INFO:tasks.workunit.client.0.smithi162.stdout:3/267: mkdir d1/d18/d55 0 2024-04-27T04:10:01.915 INFO:tasks.workunit.client.0.smithi162.stdout:2/207: creat d0/d4/d2a/f3e x:0 0 0 2024-04-27T04:10:01.916 INFO:tasks.workunit.client.0.smithi162.stdout:1/248: symlink d0/d11/d50/d53/l55 0 2024-04-27T04:10:01.916 INFO:tasks.workunit.client.0.smithi162.stdout:1/249: readlink d0/d7/l8 0 2024-04-27T04:10:01.917 INFO:tasks.workunit.client.0.smithi162.stdout:5/255: mkdir d8/d9/d5a 0 2024-04-27T04:10:01.917 INFO:tasks.workunit.client.1.smithi181.stdout:0/555: symlink dc/d1f/d3b/d4a/d95/d26/d41/d6f/d7d/lc7 0 2024-04-27T04:10:01.919 INFO:tasks.workunit.client.1.smithi181.stdout:0/556: write dc/d1f/d3b/d4a/d95/d26/f28 [381086,51160] 0 2024-04-27T04:10:01.920 INFO:tasks.workunit.client.0.smithi162.stdout:9/263: creat d3/d23/f5c x:0 0 0 2024-04-27T04:10:01.921 INFO:tasks.workunit.client.1.smithi181.stdout:2/526: dread d4/d9/d35/d42/f5a [0,4194304] 0 2024-04-27T04:10:01.921 INFO:tasks.workunit.client.0.smithi162.stdout:9/264: dread - d3/d13/d15/d50/f5b zero size 2024-04-27T04:10:01.923 INFO:tasks.workunit.client.0.smithi162.stdout:6/267: fsync d6/db/f49 0 2024-04-27T04:10:01.927 INFO:tasks.workunit.client.0.smithi162.stdout:0/322: symlink d5/dc/d1e/l6b 0 2024-04-27T04:10:01.927 INFO:tasks.workunit.client.1.smithi181.stdout:2/527: dwrite d4/d9/d16/d82/d7d/f9b [0,4194304] 0 2024-04-27T04:10:01.929 INFO:tasks.workunit.client.1.smithi181.stdout:2/528: read - d4/d9/d16/d45/f87 zero size 2024-04-27T04:10:01.930 INFO:tasks.workunit.client.1.smithi181.stdout:2/529: write d4/d9/d44/f46 [2144577,39385] 0 2024-04-27T04:10:01.932 INFO:tasks.workunit.client.1.smithi181.stdout:2/530: read d4/d9/df/d14/f78 [347290,7115] 0 2024-04-27T04:10:01.933 INFO:tasks.workunit.client.0.smithi162.stdout:7/222: truncate d3/d1d/f31 2002859 0 2024-04-27T04:10:01.933 INFO:tasks.workunit.client.1.smithi181.stdout:2/531: stat d4/d9/d4f/d79/d94 0 2024-04-27T04:10:01.934 INFO:tasks.workunit.client.0.smithi162.stdout:0/323: dread d5/f3b [0,4194304] 0 2024-04-27T04:10:01.935 INFO:tasks.workunit.client.0.smithi162.stdout:9/265: mkdir d3/d23/d5d 0 2024-04-27T04:10:01.937 INFO:tasks.workunit.client.0.smithi162.stdout:3/268: mknod d1/c56 0 2024-04-27T04:10:01.938 INFO:tasks.workunit.client.1.smithi181.stdout:0/557: write dc/d1f/d3b/d4a/d95/f36 [4612774,61165] 0 2024-04-27T04:10:01.939 INFO:tasks.workunit.client.1.smithi181.stdout:2/532: mknod d4/d9/d35/ca8 0 2024-04-27T04:10:01.941 INFO:tasks.workunit.client.0.smithi162.stdout:2/208: truncate d0/d4/f16 1362848 0 2024-04-27T04:10:01.942 INFO:tasks.workunit.client.0.smithi162.stdout:2/209: write d0/d4/f15 [4340011,487] 0 2024-04-27T04:10:01.942 INFO:tasks.workunit.client.0.smithi162.stdout:2/210: fsync d0/fe 0 2024-04-27T04:10:01.943 INFO:tasks.workunit.client.1.smithi181.stdout:2/533: read d4/d9/df/f41 [205939,88019] 0 2024-04-27T04:10:01.943 INFO:tasks.workunit.client.1.smithi181.stdout:0/558: dread dc/f5e [0,4194304] 0 2024-04-27T04:10:01.945 INFO:tasks.workunit.client.0.smithi162.stdout:8/302: rename d0/l5b to d0/de/l5d 0 2024-04-27T04:10:01.949 INFO:tasks.workunit.client.0.smithi162.stdout:9/266: creat d3/d11/f5e x:0 0 0 2024-04-27T04:10:01.950 INFO:tasks.workunit.client.1.smithi181.stdout:2/534: fdatasync d4/d9/d35/f60 0 2024-04-27T04:10:01.951 INFO:tasks.workunit.client.0.smithi162.stdout:8/303: dwrite d0/d10/d25/f36 [0,4194304] 0 2024-04-27T04:10:01.951 INFO:tasks.workunit.client.0.smithi162.stdout:6/268: unlink d6/l16 0 2024-04-27T04:10:01.952 INFO:tasks.workunit.client.0.smithi162.stdout:6/269: readlink d6/db/l29 0 2024-04-27T04:10:01.953 INFO:tasks.workunit.client.1.smithi181.stdout:0/559: creat dc/d54/fc8 x:0 0 0 2024-04-27T04:10:01.954 INFO:tasks.workunit.client.0.smithi162.stdout:3/269: truncate d1/d18/f1d 4915548 0 2024-04-27T04:10:01.954 INFO:tasks.workunit.client.0.smithi162.stdout:3/270: readlink d1/d18/d23/l31 0 2024-04-27T04:10:01.958 INFO:tasks.workunit.client.1.smithi181.stdout:2/535: rmdir d4/d9/d16/d82 39 2024-04-27T04:10:01.959 INFO:tasks.workunit.client.0.smithi162.stdout:5/256: dwrite d8/f12 [4194304,4194304] 0 2024-04-27T04:10:01.961 INFO:tasks.workunit.client.1.smithi181.stdout:0/560: mknod dc/d1f/d3b/d4a/d95/d26/d41/cc9 0 2024-04-27T04:10:01.961 INFO:tasks.workunit.client.1.smithi181.stdout:0/561: stat dc/d1f/d6b/fc0 0 2024-04-27T04:10:01.965 INFO:tasks.workunit.client.1.smithi181.stdout:0/562: rename dc/l64 to dc/d1f/d3b/d75/db3/lca 0 2024-04-27T04:10:01.967 INFO:tasks.workunit.client.0.smithi162.stdout:5/257: dwrite d8/d1a/d23/d40/d4b/f52 [4194304,4194304] 0 2024-04-27T04:10:01.967 INFO:tasks.workunit.client.1.smithi181.stdout:0/563: fdatasync dc/f39 0 2024-04-27T04:10:01.969 INFO:tasks.workunit.client.1.smithi181.stdout:2/536: link d4/d9/d16/d45/c96 d4/d9/df/d14/da7/ca9 0 2024-04-27T04:10:01.972 INFO:tasks.workunit.client.1.smithi181.stdout:0/564: dread dc/f22 [0,4194304] 0 2024-04-27T04:10:01.974 INFO:tasks.workunit.client.1.smithi181.stdout:2/537: mknod d4/d9/caa 0 2024-04-27T04:10:01.975 INFO:tasks.workunit.client.0.smithi162.stdout:8/304: symlink d0/d10/d2b/d2d/d2f/d39/l5e 0 2024-04-27T04:10:01.976 INFO:tasks.workunit.client.0.smithi162.stdout:8/305: write d0/d10/d2b/d2e/f46 [471647,36755] 0 2024-04-27T04:10:01.977 INFO:tasks.workunit.client.1.smithi181.stdout:0/565: dread dc/d1f/d3b/d4a/d95/d26/f28 [0,4194304] 0 2024-04-27T04:10:01.978 INFO:tasks.workunit.client.1.smithi181.stdout:2/538: mknod d4/d9/d44/cab 0 2024-04-27T04:10:01.980 INFO:tasks.workunit.client.0.smithi162.stdout:3/271: rename d1/d18/d55 to d1/d21/d47/d57 0 2024-04-27T04:10:01.982 INFO:tasks.workunit.client.1.smithi181.stdout:0/566: dread dc/d1f/d3b/d4a/d95/d26/f28 [0,4194304] 0 2024-04-27T04:10:01.982 INFO:tasks.workunit.client.0.smithi162.stdout:1/250: getdents d0/d7 0 2024-04-27T04:10:01.986 INFO:tasks.workunit.client.0.smithi162.stdout:5/258: mkdir d8/d9/d5b 0 2024-04-27T04:10:01.987 INFO:tasks.workunit.client.1.smithi181.stdout:0/567: dwrite f6 [0,4194304] 0 2024-04-27T04:10:01.987 INFO:tasks.workunit.client.0.smithi162.stdout:7/223: write d3/d1d/f31 [943356,928] 0 2024-04-27T04:10:01.988 INFO:tasks.workunit.client.0.smithi162.stdout:7/224: truncate d3/f40 548515 0 2024-04-27T04:10:01.989 INFO:tasks.workunit.client.0.smithi162.stdout:9/267: symlink d3/d13/l5f 0 2024-04-27T04:10:01.990 INFO:tasks.workunit.client.1.smithi181.stdout:2/539: getdents d4/d9/d4f/d79/d94 0 2024-04-27T04:10:01.991 INFO:tasks.workunit.client.1.smithi181.stdout:2/540: stat d4/d9/d16/f26 0 2024-04-27T04:10:01.992 INFO:tasks.workunit.client.0.smithi162.stdout:9/268: dread d3/f2a [0,4194304] 0 2024-04-27T04:10:01.993 INFO:tasks.workunit.client.1.smithi181.stdout:0/568: creat dc/d1f/daa/fcb x:0 0 0 2024-04-27T04:10:01.999 INFO:tasks.workunit.client.1.smithi181.stdout:0/569: mkdir dc/d1f/d3b/d4a/d95/d26/d41/db0/dcc 0 2024-04-27T04:10:02.000 INFO:tasks.workunit.client.0.smithi162.stdout:0/324: truncate d5/dc/d1e/f21 5954432 0 2024-04-27T04:10:02.001 INFO:tasks.workunit.client.0.smithi162.stdout:0/325: chown d5/dc/fe 0 1 2024-04-27T04:10:02.001 INFO:tasks.workunit.client.0.smithi162.stdout:0/326: chown d5/dc/d1e/d20/c42 24 1 2024-04-27T04:10:02.002 INFO:tasks.workunit.client.1.smithi181.stdout:2/541: creat d4/d9/d4f/fac x:0 0 0 2024-04-27T04:10:02.003 INFO:tasks.workunit.client.1.smithi181.stdout:0/570: write dc/f46 [1841752,7460] 0 2024-04-27T04:10:02.004 INFO:tasks.workunit.client.0.smithi162.stdout:4/309: sync 2024-04-27T04:10:02.005 INFO:tasks.workunit.client.0.smithi162.stdout:1/251: truncate d0/d7/f19 4277091 0 2024-04-27T04:10:02.007 INFO:tasks.workunit.client.1.smithi181.stdout:0/571: fsync dc/d1f/d3b/d4a/d95/d26/d41/f4f 0 2024-04-27T04:10:02.010 INFO:tasks.workunit.client.0.smithi162.stdout:4/310: dwrite d2/d25/d1c/d39/f46 [0,4194304] 0 2024-04-27T04:10:02.010 INFO:tasks.workunit.client.0.smithi162.stdout:7/225: mkdir d3/d5/d3c/d4a 0 2024-04-27T04:10:02.013 INFO:tasks.workunit.client.1.smithi181.stdout:3/457: sync 2024-04-27T04:10:02.015 INFO:tasks.workunit.client.1.smithi181.stdout:0/572: write f8 [970575,76833] 0 2024-04-27T04:10:02.015 INFO:tasks.workunit.client.0.smithi162.stdout:6/270: dwrite d6/f25 [0,4194304] 0 2024-04-27T04:10:02.018 INFO:tasks.workunit.client.0.smithi162.stdout:6/271: chown d6/f25 7 1 2024-04-27T04:10:02.018 INFO:tasks.workunit.client.1.smithi181.stdout:2/542: truncate d4/d9/d16/d45/f9e 6909205 0 2024-04-27T04:10:02.020 INFO:tasks.workunit.client.0.smithi162.stdout:3/272: mkdir d1/d18/d23/d58 0 2024-04-27T04:10:02.020 INFO:tasks.workunit.client.1.smithi181.stdout:2/543: fdatasync d4/d9/d4f/d79/d94/f95 0 2024-04-27T04:10:02.021 INFO:tasks.workunit.client.0.smithi162.stdout:8/306: dwrite d0/d10/d25/f36 [4194304,4194304] 0 2024-04-27T04:10:02.021 INFO:tasks.workunit.client.0.smithi162.stdout:2/211: getdents d0/d20/d2e 0 2024-04-27T04:10:02.021 INFO:tasks.workunit.client.1.smithi181.stdout:2/544: chown d4/d9/df/l22 40251453 1 2024-04-27T04:10:02.022 INFO:tasks.workunit.client.1.smithi181.stdout:0/573: dwrite dc/d1f/d3b/f66 [0,4194304] 0 2024-04-27T04:10:02.029 INFO:tasks.workunit.client.0.smithi162.stdout:3/273: dwrite d1/d21/f2c [0,4194304] 0 2024-04-27T04:10:02.029 INFO:tasks.workunit.client.0.smithi162.stdout:1/252: chown d0/l3 28 1 2024-04-27T04:10:02.034 INFO:tasks.workunit.client.0.smithi162.stdout:5/259: mkdir d8/d1a/d29/d4a/d5c 0 2024-04-27T04:10:02.035 INFO:tasks.workunit.client.0.smithi162.stdout:5/260: read - d8/d2c/f4c zero size 2024-04-27T04:10:02.036 INFO:tasks.workunit.client.0.smithi162.stdout:4/311: mknod d2/d25/c62 0 2024-04-27T04:10:02.038 INFO:tasks.workunit.client.1.smithi181.stdout:2/545: link d4/d9/d44/c5c d4/d9/d4f/d79/cad 0 2024-04-27T04:10:02.038 INFO:tasks.workunit.client.0.smithi162.stdout:1/253: dwrite d0/d7/d10/f3d [0,4194304] 0 2024-04-27T04:10:02.040 INFO:tasks.workunit.client.0.smithi162.stdout:0/327: creat d5/d26/d69/f6c x:0 0 0 2024-04-27T04:10:02.041 INFO:tasks.workunit.client.1.smithi181.stdout:2/546: rmdir d4/d9/d4f/d79/d94 39 2024-04-27T04:10:02.042 INFO:tasks.workunit.client.0.smithi162.stdout:6/272: stat d6/d1a/d34/c56 0 2024-04-27T04:10:02.043 INFO:tasks.workunit.client.1.smithi181.stdout:0/574: getdents dc/d1f/d3b/d4a/d95/d26/d41/d6f 0 2024-04-27T04:10:02.044 INFO:tasks.workunit.client.1.smithi181.stdout:0/575: write dc/f94 [4493440,127694] 0 2024-04-27T04:10:02.047 INFO:tasks.workunit.client.0.smithi162.stdout:8/307: mkdir d0/d10/d44/d5f 0 2024-04-27T04:10:02.052 INFO:tasks.workunit.client.1.smithi181.stdout:0/576: unlink dc/d1f/d3b/d4a/d95/f2e 0 2024-04-27T04:10:02.053 INFO:tasks.workunit.client.1.smithi181.stdout:0/577: dread - dc/d1f/d3b/d4a/d95/d26/d41/fc2 zero size 2024-04-27T04:10:02.055 INFO:tasks.workunit.client.0.smithi162.stdout:4/312: symlink d2/d25/d1c/d24/l63 0 2024-04-27T04:10:02.056 INFO:tasks.workunit.client.1.smithi181.stdout:0/578: read - dc/d1f/d3b/d4a/d95/d6d/fa2 zero size 2024-04-27T04:10:02.056 INFO:tasks.workunit.client.0.smithi162.stdout:4/313: write d2/f4 [1823360,4341] 0 2024-04-27T04:10:02.057 INFO:tasks.workunit.client.0.smithi162.stdout:4/314: truncate d2/f38 5031245 0 2024-04-27T04:10:02.057 INFO:tasks.workunit.client.0.smithi162.stdout:1/254: creat d0/d11/d50/f56 x:0 0 0 2024-04-27T04:10:02.059 INFO:tasks.workunit.client.0.smithi162.stdout:6/273: rmdir d6/d1a/d34 39 2024-04-27T04:10:02.060 INFO:tasks.workunit.client.0.smithi162.stdout:2/212: symlink d0/d4/l3f 0 2024-04-27T04:10:02.060 INFO:tasks.workunit.client.0.smithi162.stdout:6/274: write d6/d1a/f2b [2939816,78953] 0 2024-04-27T04:10:02.061 INFO:tasks.workunit.client.1.smithi181.stdout:0/579: rename dc/d1f/d3b/d4a/d95/d26/d41/d6f/d7d/c86 to dc/d1f/daa/ccd 0 2024-04-27T04:10:02.064 INFO:tasks.workunit.client.0.smithi162.stdout:8/308: rename d0/d10/d2b/d23/f31 to d0/de/d20/f60 0 2024-04-27T04:10:02.064 INFO:tasks.workunit.client.0.smithi162.stdout:1/255: dwrite d0/d7/d10/f4c [0,4194304] 0 2024-04-27T04:10:02.066 INFO:tasks.workunit.client.0.smithi162.stdout:7/226: rename d3/d5/d9/dc to d3/d5/d9/d4b 0 2024-04-27T04:10:02.069 INFO:tasks.workunit.client.0.smithi162.stdout:5/261: getdents d8/d9/d5a 0 2024-04-27T04:10:02.070 INFO:tasks.workunit.client.0.smithi162.stdout:5/262: write d8/d9/f32 [107934,129624] 0 2024-04-27T04:10:02.071 INFO:tasks.workunit.client.0.smithi162.stdout:5/263: write d8/d2c/f4c [748651,85710] 0 2024-04-27T04:10:02.072 INFO:tasks.workunit.client.0.smithi162.stdout:5/264: fsync d8/d1a/f51 0 2024-04-27T04:10:02.076 INFO:tasks.workunit.client.0.smithi162.stdout:6/275: write d6/f11 [1637580,51926] 0 2024-04-27T04:10:02.080 INFO:tasks.workunit.client.0.smithi162.stdout:5/265: dwrite d8/d1a/f2d [0,4194304] 0 2024-04-27T04:10:02.080 INFO:tasks.workunit.client.0.smithi162.stdout:8/309: rename d0/de/f4b to d0/d10/d44/d5f/f61 0 2024-04-27T04:10:02.082 INFO:tasks.workunit.client.0.smithi162.stdout:9/269: dwrite d3/d13/f26 [0,4194304] 0 2024-04-27T04:10:02.083 INFO:tasks.workunit.client.0.smithi162.stdout:0/328: link d5/d26/d4a/f62 d5/dc/d1e/f6d 0 2024-04-27T04:10:02.084 INFO:tasks.workunit.client.0.smithi162.stdout:4/315: link d2/da/f31 d2/d25/d1c/d39/d3b/d52/f64 0 2024-04-27T04:10:02.085 INFO:tasks.workunit.client.1.smithi181.stdout:2/547: write d4/d9/d16/d82/fa5 [3850157,12929] 0 2024-04-27T04:10:02.090 INFO:tasks.workunit.client.0.smithi162.stdout:2/213: truncate d0/f7 630535 0 2024-04-27T04:10:02.092 INFO:tasks.workunit.client.1.smithi181.stdout:2/548: stat d4/d9/d4f/d79/d94/f95 0 2024-04-27T04:10:02.095 INFO:tasks.workunit.client.1.smithi181.stdout:0/580: rmdir dc/d1f/d3b/d4a/d95/d26/d41/d6f/d7d 39 2024-04-27T04:10:02.097 INFO:tasks.workunit.client.1.smithi181.stdout:2/549: symlink d4/d9/d35/d42/d86/lae 0 2024-04-27T04:10:02.099 INFO:tasks.workunit.client.0.smithi162.stdout:6/276: creat d6/db/d32/f5b x:0 0 0 2024-04-27T04:10:02.099 INFO:tasks.workunit.client.1.smithi181.stdout:0/581: stat dc/d1f/d3b/d4a/d95/c3e 0 2024-04-27T04:10:02.103 INFO:tasks.workunit.client.0.smithi162.stdout:8/310: creat d0/d10/d2b/d2d/d2f/d39/f62 x:0 0 0 2024-04-27T04:10:02.103 INFO:tasks.workunit.client.1.smithi181.stdout:0/582: fsync dc/d1f/f3f 0 2024-04-27T04:10:02.105 INFO:tasks.workunit.client.0.smithi162.stdout:5/266: rename d8/d1a/d23/d40/c4f to d8/d1a/d23/d40/c5d 0 2024-04-27T04:10:02.110 INFO:tasks.workunit.client.0.smithi162.stdout:4/316: rename f1 to d2/d5a/f65 0 2024-04-27T04:10:02.110 INFO:tasks.workunit.client.0.smithi162.stdout:4/317: read - d2/d25/d1c/d39/f5f zero size 2024-04-27T04:10:02.111 INFO:tasks.workunit.client.1.smithi181.stdout:0/583: mkdir dc/d1f/d6b/db9/dba/dce 0 2024-04-27T04:10:02.112 INFO:tasks.workunit.client.0.smithi162.stdout:3/274: sync 2024-04-27T04:10:02.112 INFO:tasks.workunit.client.0.smithi162.stdout:2/214: stat d0/d4/d17/c28 0 2024-04-27T04:10:02.112 INFO:tasks.workunit.client.0.smithi162.stdout:3/275: readlink d1/d18/d23/d34/l4e 0 2024-04-27T04:10:02.113 INFO:tasks.workunit.client.0.smithi162.stdout:2/215: dread - d0/d4/f3a zero size 2024-04-27T04:10:02.114 INFO:tasks.workunit.client.0.smithi162.stdout:3/276: truncate d1/d18/f52 463347 0 2024-04-27T04:10:02.114 INFO:tasks.workunit.client.0.smithi162.stdout:3/277: stat d1/d21 0 2024-04-27T04:10:02.114 INFO:tasks.workunit.client.1.smithi181.stdout:0/584: creat dc/d1f/d6b/db9/fcf x:0 0 0 2024-04-27T04:10:02.115 INFO:tasks.workunit.client.0.smithi162.stdout:1/256: write d0/f2b [1448659,100425] 0 2024-04-27T04:10:02.116 INFO:tasks.workunit.client.0.smithi162.stdout:3/278: dread d1/d18/d23/f2b [0,4194304] 0 2024-04-27T04:10:02.119 INFO:tasks.workunit.client.1.smithi181.stdout:2/550: write d4/f50 [3036598,6914] 0 2024-04-27T04:10:02.119 INFO:tasks.workunit.client.1.smithi181.stdout:0/585: mkdir dc/d1f/d3b/dd0 0 2024-04-27T04:10:02.120 INFO:tasks.workunit.client.1.smithi181.stdout:2/551: dread - d4/d9/d4f/d79/d94/f53 zero size 2024-04-27T04:10:02.121 INFO:tasks.workunit.client.0.smithi162.stdout:9/270: symlink d3/d13/d2d/d3b/d54/l60 0 2024-04-27T04:10:02.121 INFO:tasks.workunit.client.1.smithi181.stdout:0/586: readlink dc/d1f/d3b/d4a/d95/d26/d68/l82 0 2024-04-27T04:10:02.121 INFO:tasks.workunit.client.0.smithi162.stdout:5/267: dread d8/d9/f48 [0,4194304] 0 2024-04-27T04:10:02.124 INFO:tasks.workunit.client.0.smithi162.stdout:5/268: dread d8/d1a/d23/d40/f46 [0,4194304] 0 2024-04-27T04:10:02.125 INFO:tasks.workunit.client.1.smithi181.stdout:0/587: chown dc/l61 157 1 2024-04-27T04:10:02.125 INFO:tasks.workunit.client.0.smithi162.stdout:4/318: mknod d2/d5a/c66 0 2024-04-27T04:10:02.125 INFO:tasks.workunit.client.0.smithi162.stdout:6/277: dread d6/f35 [0,4194304] 0 2024-04-27T04:10:02.126 INFO:tasks.workunit.client.0.smithi162.stdout:2/216: mkdir d0/d20/d2e/d40 0 2024-04-27T04:10:02.126 INFO:tasks.workunit.client.1.smithi181.stdout:2/552: dread d4/d9/d16/d45/f9e [0,4194304] 0 2024-04-27T04:10:02.127 INFO:tasks.workunit.client.0.smithi162.stdout:9/271: dwrite d3/fb [0,4194304] 0 2024-04-27T04:10:02.128 INFO:tasks.workunit.client.0.smithi162.stdout:1/257: unlink d0/d11/f2c 0 2024-04-27T04:10:02.129 INFO:tasks.workunit.client.0.smithi162.stdout:1/258: chown d0 36962 1 2024-04-27T04:10:02.129 INFO:tasks.workunit.client.0.smithi162.stdout:9/272: fdatasync d3/d13/f41 0 2024-04-27T04:10:02.130 INFO:tasks.workunit.client.0.smithi162.stdout:7/227: getdents d3/d5/d9/d4b 0 2024-04-27T04:10:02.130 INFO:tasks.workunit.client.0.smithi162.stdout:3/279: mknod d1/d21/c59 0 2024-04-27T04:10:02.132 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:10:01 smithi181 ceph-mon[33774]: pgmap v266: 65 pgs: 65 active+clean; 1.3 GiB data, 5.5 GiB used, 531 GiB / 536 GiB avail; 27 MiB/s rd, 162 MiB/s wr, 307 op/s 2024-04-27T04:10:02.132 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:10:01 smithi181 ceph-mon[33774]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-27T04:10:02.134 INFO:tasks.workunit.client.0.smithi162.stdout:1/259: stat d0/d7/d2a/f31 0 2024-04-27T04:10:02.134 INFO:tasks.workunit.client.1.smithi181.stdout:2/553: creat d4/d9/d16/d45/d9a/d9d/faf x:0 0 0 2024-04-27T04:10:02.135 INFO:tasks.workunit.client.1.smithi181.stdout:0/588: getdents dc/d1f/d3b/d4a/d95/d26/d41 0 2024-04-27T04:10:02.135 INFO:tasks.workunit.client.0.smithi162.stdout:5/269: dwrite d8/f20 [8388608,4194304] 0 2024-04-27T04:10:02.136 INFO:tasks.workunit.client.1.smithi181.stdout:2/554: write d4/d9/d16/d45/f87 [723407,758] 0 2024-04-27T04:10:02.137 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:10:01 smithi162 ceph-mon[26841]: pgmap v266: 65 pgs: 65 active+clean; 1.3 GiB data, 5.5 GiB used, 531 GiB / 536 GiB avail; 27 MiB/s rd, 162 MiB/s wr, 307 op/s 2024-04-27T04:10:02.137 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:10:01 smithi162 ceph-mon[26841]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-27T04:10:02.138 INFO:tasks.workunit.client.0.smithi162.stdout:0/329: link d5/l1c d5/dc/d1e/d20/l6e 0 2024-04-27T04:10:02.140 INFO:tasks.workunit.client.0.smithi162.stdout:9/273: dwrite d3/f14 [0,4194304] 0 2024-04-27T04:10:02.142 INFO:tasks.workunit.client.1.smithi181.stdout:0/589: rmdir dc/d1f/d3b/db2 39 2024-04-27T04:10:02.145 INFO:tasks.workunit.client.0.smithi162.stdout:0/330: dwrite d5/dc/f31 [0,4194304] 0 2024-04-27T04:10:02.147 INFO:tasks.workunit.client.0.smithi162.stdout:4/319: fdatasync d2/d5a/f65 0 2024-04-27T04:10:02.149 INFO:tasks.workunit.client.0.smithi162.stdout:4/320: truncate d2/da/dd/f61 379669 0 2024-04-27T04:10:02.149 INFO:tasks.workunit.client.0.smithi162.stdout:4/321: dread - d2/da/dd/f60 zero size 2024-04-27T04:10:02.151 INFO:tasks.workunit.client.0.smithi162.stdout:2/217: dread d0/fe [0,4194304] 0 2024-04-27T04:10:02.155 INFO:tasks.workunit.client.0.smithi162.stdout:2/218: chown d0/d4/f3a 2012698159 1 2024-04-27T04:10:02.155 INFO:tasks.workunit.client.1.smithi181.stdout:2/555: mknod d4/d9/df/d14/da7/cb0 0 2024-04-27T04:10:02.156 INFO:tasks.workunit.client.1.smithi181.stdout:2/556: fsync d4/d9/f8c 0 2024-04-27T04:10:02.159 INFO:tasks.workunit.client.0.smithi162.stdout:4/322: dwrite d2/da/dd/f61 [0,4194304] 0 2024-04-27T04:10:02.163 INFO:tasks.workunit.client.1.smithi181.stdout:2/557: dwrite d4/d9/f6d [0,4194304] 0 2024-04-27T04:10:02.164 INFO:tasks.workunit.client.0.smithi162.stdout:8/311: link d0/d10/c38 d0/d10/d2b/d23/c63 0 2024-04-27T04:10:02.173 INFO:tasks.workunit.client.0.smithi162.stdout:0/331: creat d5/d26/d69/f6f x:0 0 0 2024-04-27T04:10:02.175 INFO:tasks.workunit.client.0.smithi162.stdout:6/278: creat d6/d1a/d34/d43/f5c x:0 0 0 2024-04-27T04:10:02.177 INFO:tasks.workunit.client.0.smithi162.stdout:2/219: mkdir d0/d4/d2a/d41 0 2024-04-27T04:10:02.179 INFO:tasks.workunit.client.0.smithi162.stdout:8/312: mknod d0/d10/c64 0 2024-04-27T04:10:02.201 INFO:tasks.workunit.client.1.smithi181.stdout:8/591: sync 2024-04-27T04:10:02.202 INFO:tasks.workunit.client.1.smithi181.stdout:8/592: chown d1/d6/d2f/d72/l83 60906 1 2024-04-27T04:10:02.203 INFO:tasks.workunit.client.1.smithi181.stdout:8/593: chown d1/d6/d2f/d47/dba 7 1 2024-04-27T04:10:02.203 INFO:tasks.workunit.client.0.smithi162.stdout:7/228: link d3/d5/d9/d4b/f33 d3/d1d/f4c 0 2024-04-27T04:10:02.208 INFO:tasks.workunit.client.0.smithi162.stdout:1/260: stat d0/d11/f20 0 2024-04-27T04:10:02.209 INFO:tasks.workunit.client.0.smithi162.stdout:0/332: mknod d5/d60/c70 0 2024-04-27T04:10:02.209 INFO:tasks.workunit.client.1.smithi181.stdout:0/590: dwrite dc/f11 [8388608,4194304] 0 2024-04-27T04:10:02.212 INFO:tasks.workunit.client.1.smithi181.stdout:8/594: dwrite d1/d6/d2f/d47/dba/fbf [0,4194304] 0 2024-04-27T04:10:02.214 INFO:tasks.workunit.client.0.smithi162.stdout:0/333: dwrite d5/dc/d1e/d41/f39 [0,4194304] 0 2024-04-27T04:10:02.220 INFO:tasks.workunit.client.0.smithi162.stdout:8/313: mkdir d0/d10/d25/d65 0 2024-04-27T04:10:02.220 INFO:tasks.workunit.client.0.smithi162.stdout:9/274: sync 2024-04-27T04:10:02.222 INFO:tasks.workunit.client.0.smithi162.stdout:9/275: read d3/d13/d15/f18 [562128,60314] 0 2024-04-27T04:10:02.226 INFO:tasks.workunit.client.0.smithi162.stdout:7/229: rename d3/d1d/d2d to d3/d5/d3c/d4d 0 2024-04-27T04:10:02.234 INFO:tasks.workunit.client.0.smithi162.stdout:0/334: creat d5/dc/d1e/d41/f71 x:0 0 0 2024-04-27T04:10:02.237 INFO:tasks.workunit.client.0.smithi162.stdout:3/280: dwrite d1/d18/f1e [0,4194304] 0 2024-04-27T04:10:02.237 INFO:tasks.workunit.client.0.smithi162.stdout:9/276: symlink d3/d13/d15/l61 0 2024-04-27T04:10:02.238 INFO:tasks.workunit.client.0.smithi162.stdout:9/277: write d3/d13/d15/d50/f5b [359606,44552] 0 2024-04-27T04:10:02.239 INFO:tasks.workunit.client.0.smithi162.stdout:7/230: mknod d3/d5/d9/d4b/de/c4e 0 2024-04-27T04:10:02.239 INFO:tasks.workunit.client.0.smithi162.stdout:9/278: chown d3/d13/d15/d50/f5b 6 1 2024-04-27T04:10:02.241 INFO:tasks.workunit.client.0.smithi162.stdout:6/279: creat d6/d1a/f5d x:0 0 0 2024-04-27T04:10:02.243 INFO:tasks.workunit.client.0.smithi162.stdout:2/220: creat d0/d4/f42 x:0 0 0 2024-04-27T04:10:02.245 INFO:tasks.workunit.client.0.smithi162.stdout:4/323: getdents d2 0 2024-04-27T04:10:02.246 INFO:tasks.workunit.client.0.smithi162.stdout:4/324: fsync d2/d25/f2d 0 2024-04-27T04:10:02.247 INFO:tasks.workunit.client.0.smithi162.stdout:0/335: creat d5/d60/f72 x:0 0 0 2024-04-27T04:10:02.248 INFO:tasks.workunit.client.0.smithi162.stdout:0/336: read d5/dc/f22 [513344,10743] 0 2024-04-27T04:10:02.250 INFO:tasks.workunit.client.0.smithi162.stdout:0/337: write d5/dc/d1e/d41/f4f [1924960,2021] 0 2024-04-27T04:10:02.250 INFO:tasks.workunit.client.0.smithi162.stdout:3/281: mknod d1/d21/c5a 0 2024-04-27T04:10:02.252 INFO:tasks.workunit.client.0.smithi162.stdout:9/279: mknod d3/c62 0 2024-04-27T04:10:02.253 INFO:tasks.workunit.client.0.smithi162.stdout:3/282: write d1/d18/f41 [131235,77986] 0 2024-04-27T04:10:02.254 INFO:tasks.workunit.client.0.smithi162.stdout:5/270: dwrite f4 [0,4194304] 0 2024-04-27T04:10:02.256 INFO:tasks.workunit.client.0.smithi162.stdout:5/271: rename d8/d1a to d8/d1a/d29/d5e 22 2024-04-27T04:10:02.258 INFO:tasks.workunit.client.1.smithi181.stdout:8/595: mkdir d1/d6/d2f/d30/d66/dc0/dd0 0 2024-04-27T04:10:02.259 INFO:tasks.workunit.client.0.smithi162.stdout:2/221: creat d0/d4/d2a/f43 x:0 0 0 2024-04-27T04:10:02.263 INFO:tasks.workunit.client.1.smithi181.stdout:0/591: unlink dc/d54/f5b 0 2024-04-27T04:10:02.265 INFO:tasks.workunit.client.0.smithi162.stdout:4/325: symlink d2/l67 0 2024-04-27T04:10:02.265 INFO:tasks.workunit.client.1.smithi181.stdout:0/592: write dc/d1f/d3b/d4a/d95/d26/d41/d6f/f7f [7575941,84783] 0 2024-04-27T04:10:02.265 INFO:tasks.workunit.client.1.smithi181.stdout:0/593: chown dc 28 1 2024-04-27T04:10:02.266 INFO:tasks.workunit.client.1.smithi181.stdout:8/596: mknod d1/d6/d2f/d30/d66/cd1 0 2024-04-27T04:10:02.271 INFO:tasks.workunit.client.0.smithi162.stdout:0/338: creat d5/d26/d69/f73 x:0 0 0 2024-04-27T04:10:02.275 INFO:tasks.workunit.client.0.smithi162.stdout:1/261: write d0/f1b [4506364,24713] 0 2024-04-27T04:10:02.275 INFO:tasks.workunit.client.0.smithi162.stdout:6/280: symlink d6/d1a/d34/d50/l5e 0 2024-04-27T04:10:02.277 INFO:tasks.workunit.client.0.smithi162.stdout:5/272: symlink d8/d2c/l5f 0 2024-04-27T04:10:02.278 INFO:tasks.workunit.client.0.smithi162.stdout:8/314: link d0/d10/d2b/d2e/l53 d0/d10/l66 0 2024-04-27T04:10:02.279 INFO:tasks.workunit.client.0.smithi162.stdout:0/339: dwrite d5/d60/f72 [0,4194304] 0 2024-04-27T04:10:02.281 INFO:tasks.workunit.client.1.smithi181.stdout:8/597: dwrite d1/d6/d2f/d40/fa1 [0,4194304] 0 2024-04-27T04:10:02.283 INFO:tasks.workunit.client.0.smithi162.stdout:1/262: unlink d0/d11/d50/d53/l55 0 2024-04-27T04:10:02.283 INFO:tasks.workunit.client.0.smithi162.stdout:1/263: chown d0/d7/d2a/f47 15 1 2024-04-27T04:10:02.292 INFO:tasks.workunit.client.1.smithi181.stdout:8/598: dwrite d1/d6/d2f/d47/f4b [0,4194304] 0 2024-04-27T04:10:02.295 INFO:tasks.workunit.client.1.smithi181.stdout:2/558: dwrite d4/d9/d16/f2a [0,4194304] 0 2024-04-27T04:10:02.298 INFO:tasks.workunit.client.0.smithi162.stdout:8/315: dwrite d0/de/d20/f50 [0,4194304] 0 2024-04-27T04:10:02.299 INFO:tasks.workunit.client.0.smithi162.stdout:0/340: rename d5/f35 to d5/dc/f74 0 2024-04-27T04:10:02.305 INFO:tasks.workunit.client.0.smithi162.stdout:7/231: sync 2024-04-27T04:10:02.306 INFO:tasks.workunit.client.0.smithi162.stdout:1/264: rmdir d0/d11/d3e 39 2024-04-27T04:10:02.309 INFO:tasks.workunit.client.0.smithi162.stdout:7/232: chown d3/d5/d3c/d4d/c36 468926374 1 2024-04-27T04:10:02.310 INFO:tasks.workunit.client.0.smithi162.stdout:6/281: symlink d6/d1a/d34/d3a/l5f 0 2024-04-27T04:10:02.313 INFO:tasks.workunit.client.0.smithi162.stdout:2/222: creat d0/f44 x:0 0 0 2024-04-27T04:10:02.315 INFO:tasks.workunit.client.1.smithi181.stdout:0/594: write dc/d1f/d3b/d4a/d95/d26/d41/f4f [129640,50001] 0 2024-04-27T04:10:02.316 INFO:tasks.workunit.client.0.smithi162.stdout:3/283: getdents d1/d18/d23/d2f 0 2024-04-27T04:10:02.316 INFO:tasks.workunit.client.0.smithi162.stdout:0/341: dread d5/f9 [0,4194304] 0 2024-04-27T04:10:02.317 INFO:tasks.workunit.client.1.smithi181.stdout:0/595: write dc/d1f/d3b/fbf [994394,43180] 0 2024-04-27T04:10:02.317 INFO:tasks.workunit.client.0.smithi162.stdout:1/265: dwrite d0/f1b [0,4194304] 0 2024-04-27T04:10:02.321 INFO:tasks.workunit.client.0.smithi162.stdout:0/342: chown d5/dc/d1e/d56/d57 736657596 1 2024-04-27T04:10:02.322 INFO:tasks.workunit.client.1.smithi181.stdout:0/596: rename dc/d54/fc8 to dc/d1f/d3b/d4a/fd1 0 2024-04-27T04:10:02.322 INFO:tasks.workunit.client.0.smithi162.stdout:3/284: truncate d1/d18/d23/d34/f3c 140552 0 2024-04-27T04:10:02.323 INFO:tasks.workunit.client.0.smithi162.stdout:0/343: dread - d5/d26/d69/f6f zero size 2024-04-27T04:10:02.323 INFO:tasks.workunit.client.0.smithi162.stdout:8/316: mknod d0/d10/d2b/d2d/d2f/d39/c67 0 2024-04-27T04:10:02.323 INFO:tasks.workunit.client.0.smithi162.stdout:9/280: dwrite d3/d23/d1e/f2b [0,4194304] 0 2024-04-27T04:10:02.336 INFO:tasks.workunit.client.0.smithi162.stdout:6/282: mkdir d6/d1a/d34/d3a/d60 0 2024-04-27T04:10:02.337 INFO:tasks.workunit.client.0.smithi162.stdout:6/283: chown d6/d1a/f5d 479951 1 2024-04-27T04:10:02.337 INFO:tasks.workunit.client.0.smithi162.stdout:6/284: fdatasync d6/db/f49 0 2024-04-27T04:10:02.338 INFO:tasks.workunit.client.1.smithi181.stdout:2/559: dwrite d4/d9/f2d [0,4194304] 0 2024-04-27T04:10:02.340 INFO:tasks.workunit.client.0.smithi162.stdout:1/266: creat d0/d7/d3c/f57 x:0 0 0 2024-04-27T04:10:02.346 INFO:tasks.workunit.client.1.smithi181.stdout:2/560: symlink d4/d9/d44/da4/lb1 0 2024-04-27T04:10:02.347 INFO:tasks.workunit.client.0.smithi162.stdout:0/344: unlink d5/dc/d37/l65 0 2024-04-27T04:10:02.348 INFO:tasks.workunit.client.1.smithi181.stdout:0/597: dwrite dc/d1f/f29 [4194304,4194304] 0 2024-04-27T04:10:02.349 INFO:tasks.workunit.client.0.smithi162.stdout:8/317: unlink d0/d10/d2b/d2d/l3f 0 2024-04-27T04:10:02.349 INFO:tasks.workunit.client.1.smithi181.stdout:0/598: write f6 [5329826,68030] 0 2024-04-27T04:10:02.350 INFO:tasks.workunit.client.0.smithi162.stdout:8/318: dread - d0/d10/d2b/d2e/f40 zero size 2024-04-27T04:10:02.355 INFO:tasks.workunit.client.0.smithi162.stdout:5/273: getdents d8/d9 0 2024-04-27T04:10:02.356 INFO:tasks.workunit.client.0.smithi162.stdout:6/285: fsync d6/d1a/d34/d46/f57 0 2024-04-27T04:10:02.359 INFO:tasks.workunit.client.0.smithi162.stdout:6/286: write d6/d1a/f31 [318757,124538] 0 2024-04-27T04:10:02.359 INFO:tasks.workunit.client.0.smithi162.stdout:3/285: mknod d1/d18/d23/d58/c5b 0 2024-04-27T04:10:02.359 INFO:tasks.workunit.client.0.smithi162.stdout:0/345: symlink d5/dc/d1e/d56/l75 0 2024-04-27T04:10:02.360 INFO:tasks.workunit.client.0.smithi162.stdout:4/326: symlink d2/da/d1a/l68 0 2024-04-27T04:10:02.361 INFO:tasks.workunit.client.0.smithi162.stdout:8/319: mknod d0/d10/d2b/c68 0 2024-04-27T04:10:02.362 INFO:tasks.workunit.client.0.smithi162.stdout:7/233: rename d3/d1d to d3/d4f 0 2024-04-27T04:10:02.365 INFO:tasks.workunit.client.0.smithi162.stdout:0/346: symlink d5/d26/l76 0 2024-04-27T04:10:02.366 INFO:tasks.workunit.client.0.smithi162.stdout:9/281: dread d3/d11/f24 [0,4194304] 0 2024-04-27T04:10:02.366 INFO:tasks.workunit.client.0.smithi162.stdout:2/223: link d0/d4/d17/c28 d0/d4/c45 0 2024-04-27T04:10:02.367 INFO:tasks.workunit.client.0.smithi162.stdout:9/282: chown d3/d11 737816633 1 2024-04-27T04:10:02.368 INFO:tasks.workunit.client.0.smithi162.stdout:5/274: dwrite d8/d1a/d23/d40/f42 [0,4194304] 0 2024-04-27T04:10:02.370 INFO:tasks.workunit.client.0.smithi162.stdout:4/327: creat d2/d25/d1c/d24/f69 x:0 0 0 2024-04-27T04:10:02.370 INFO:tasks.workunit.client.0.smithi162.stdout:7/234: mknod d3/d5/d9/d4b/de/d2b/c50 0 2024-04-27T04:10:02.371 INFO:tasks.workunit.client.1.smithi181.stdout:0/599: dread dc/d1f/f29 [0,4194304] 0 2024-04-27T04:10:02.374 INFO:tasks.workunit.client.0.smithi162.stdout:8/320: rename d0/l5 to d0/d10/d2b/l69 0 2024-04-27T04:10:02.375 INFO:tasks.workunit.client.0.smithi162.stdout:6/287: dwrite d6/d1a/f5d [0,4194304] 0 2024-04-27T04:10:02.376 INFO:tasks.workunit.client.1.smithi181.stdout:0/600: mknod dc/d1f/d6b/db9/dbd/cd2 0 2024-04-27T04:10:02.377 INFO:tasks.workunit.client.0.smithi162.stdout:0/347: dread d5/dc/d1e/d46/f52 [0,4194304] 0 2024-04-27T04:10:02.378 INFO:tasks.workunit.client.0.smithi162.stdout:8/321: chown d0/d10/d2b 14 1 2024-04-27T04:10:02.379 INFO:tasks.workunit.client.0.smithi162.stdout:2/224: mkdir d0/d20/d2e/d46 0 2024-04-27T04:10:02.379 INFO:tasks.workunit.client.0.smithi162.stdout:0/348: chown d5/dc/d1e 105 1 2024-04-27T04:10:02.381 INFO:tasks.workunit.client.0.smithi162.stdout:2/225: chown d0/d4/d17/f2f 58 1 2024-04-27T04:10:02.382 INFO:tasks.workunit.client.0.smithi162.stdout:5/275: creat d8/d1a/d29/f60 x:0 0 0 2024-04-27T04:10:02.382 INFO:tasks.workunit.client.0.smithi162.stdout:1/267: dwrite d0/fe [4194304,4194304] 0 2024-04-27T04:10:02.385 INFO:tasks.workunit.client.1.smithi181.stdout:0/601: mknod dc/d1f/d3b/db2/cd3 0 2024-04-27T04:10:02.385 INFO:tasks.workunit.client.1.smithi181.stdout:0/602: write dc/d1f/d3b/f66 [1673768,62050] 0 2024-04-27T04:10:02.386 INFO:tasks.workunit.client.0.smithi162.stdout:3/286: link d1/d18/d23/d2f/c3b d1/d18/c5c 0 2024-04-27T04:10:02.388 INFO:tasks.workunit.client.0.smithi162.stdout:3/287: read - d1/d37/f51 zero size 2024-04-27T04:10:02.389 INFO:tasks.workunit.client.1.smithi181.stdout:0/603: symlink dc/d1f/d6b/db9/ld4 0 2024-04-27T04:10:02.390 INFO:tasks.workunit.client.0.smithi162.stdout:8/322: chown d0/f7 428 1 2024-04-27T04:10:02.393 INFO:tasks.workunit.client.0.smithi162.stdout:8/323: read - d0/d10/d2b/d2d/d2f/d39/f5a zero size 2024-04-27T04:10:02.393 INFO:tasks.workunit.client.0.smithi162.stdout:6/288: rename d6/d1a/f24 to d6/db/f61 0 2024-04-27T04:10:02.395 INFO:tasks.workunit.client.0.smithi162.stdout:6/289: stat d6/d1a/f18 0 2024-04-27T04:10:02.397 INFO:tasks.workunit.client.0.smithi162.stdout:6/290: dread - d6/db/d32/f5b zero size 2024-04-27T04:10:02.397 INFO:tasks.workunit.client.0.smithi162.stdout:3/288: dwrite d1/d18/f4d [0,4194304] 0 2024-04-27T04:10:02.399 INFO:tasks.workunit.client.0.smithi162.stdout:6/291: write d6/f11 [3976244,111549] 0 2024-04-27T04:10:02.399 INFO:tasks.workunit.client.0.smithi162.stdout:4/328: dread d2/d25/f5c [0,4194304] 0 2024-04-27T04:10:02.404 INFO:tasks.workunit.client.0.smithi162.stdout:2/226: symlink d0/d20/d2e/d46/l47 0 2024-04-27T04:10:02.408 INFO:tasks.workunit.client.0.smithi162.stdout:3/289: dread d1/d18/d23/f2b [0,4194304] 0 2024-04-27T04:10:02.409 INFO:tasks.workunit.client.0.smithi162.stdout:1/268: rename d0/d11/c4e to d0/d11/d3e/c58 0 2024-04-27T04:10:02.410 INFO:tasks.workunit.client.0.smithi162.stdout:7/235: link d3/d4f/c32 d3/d5/d9/d4b/de/d2b/c51 0 2024-04-27T04:10:02.410 INFO:tasks.workunit.client.1.smithi181.stdout:0/604: dread f8 [0,4194304] 0 2024-04-27T04:10:02.411 INFO:tasks.workunit.client.0.smithi162.stdout:9/283: dwrite d3/f14 [4194304,4194304] 0 2024-04-27T04:10:02.412 INFO:tasks.workunit.client.1.smithi181.stdout:0/605: write dc/d1f/d3b/d4a/d95/f36 [800586,107803] 0 2024-04-27T04:10:02.414 INFO:tasks.workunit.client.0.smithi162.stdout:8/324: link d0/d10/d44/d5f/f61 d0/d10/f6a 0 2024-04-27T04:10:02.416 INFO:tasks.workunit.client.0.smithi162.stdout:9/284: dread d3/f14 [0,4194304] 0 2024-04-27T04:10:02.422 INFO:tasks.workunit.client.1.smithi181.stdout:0/606: dread dc/f2c [0,4194304] 0 2024-04-27T04:10:02.426 INFO:tasks.workunit.client.1.smithi181.stdout:0/607: chown dc/d1f/d3b/d4a/d95/d26/d41/d6f/d7d/fa7 119460889 1 2024-04-27T04:10:02.431 INFO:tasks.workunit.client.1.smithi181.stdout:0/608: mknod dc/d1f/d6b/d7e/d91/cd5 0 2024-04-27T04:10:02.432 INFO:tasks.workunit.client.1.smithi181.stdout:0/609: write dc/f11 [2960215,78729] 0 2024-04-27T04:10:02.435 INFO:tasks.workunit.client.1.smithi181.stdout:0/610: symlink dc/d1f/d3b/ld6 0 2024-04-27T04:10:02.436 INFO:tasks.workunit.client.1.smithi181.stdout:0/611: mknod dc/d1f/d6b/db9/cd7 0 2024-04-27T04:10:02.437 INFO:tasks.workunit.client.1.smithi181.stdout:0/612: chown dc/d1f/d3b/d4a/d95/c45 163 1 2024-04-27T04:10:02.441 INFO:tasks.workunit.client.0.smithi162.stdout:1/269: mkdir d0/d11/d3e/d59 0 2024-04-27T04:10:02.441 INFO:tasks.workunit.client.0.smithi162.stdout:0/349: write d5/dc/d1e/f6d [376190,55319] 0 2024-04-27T04:10:02.442 INFO:tasks.workunit.client.0.smithi162.stdout:8/325: mkdir d0/d10/d2b/d2d/d2f/d6b 0 2024-04-27T04:10:02.442 INFO:tasks.workunit.client.0.smithi162.stdout:5/276: link d8/d1a/d23/d40/c5d d8/d1a/d29/d4a/c61 0 2024-04-27T04:10:02.442 INFO:tasks.workunit.client.0.smithi162.stdout:1/270: fdatasync d0/d7/d2a/f31 0 2024-04-27T04:10:02.443 INFO:tasks.workunit.client.1.smithi181.stdout:0/613: dwrite dc/d1f/d3b/d4a/d95/d26/d41/d6f/f7f [0,4194304] 0 2024-04-27T04:10:02.445 INFO:tasks.workunit.client.0.smithi162.stdout:5/277: read d8/d1a/d23/d40/f42 [450185,10501] 0 2024-04-27T04:10:02.446 INFO:tasks.workunit.client.0.smithi162.stdout:4/329: rename d2/da/d1a/c51 to d2/d25/d1c/c6a 0 2024-04-27T04:10:02.446 INFO:tasks.workunit.client.0.smithi162.stdout:7/236: truncate d3/d5/d9/d4b/f33 3061803 0 2024-04-27T04:10:02.447 INFO:tasks.workunit.client.0.smithi162.stdout:1/271: write d0/d7/d3c/f57 [247089,69554] 0 2024-04-27T04:10:02.450 INFO:tasks.workunit.client.1.smithi181.stdout:0/614: link f8 dc/d1f/d3b/d4a/d95/fd8 0 2024-04-27T04:10:02.453 INFO:tasks.workunit.client.0.smithi162.stdout:8/326: mkdir d0/de/d20/d6c 0 2024-04-27T04:10:02.455 INFO:tasks.workunit.client.0.smithi162.stdout:8/327: chown d0/d10/d2b 2475382 1 2024-04-27T04:10:02.457 INFO:tasks.workunit.client.0.smithi162.stdout:2/227: dread d0/d4/f15 [0,4194304] 0 2024-04-27T04:10:02.457 INFO:tasks.workunit.client.0.smithi162.stdout:6/292: dwrite d6/db/f22 [0,4194304] 0 2024-04-27T04:10:02.460 INFO:tasks.workunit.client.0.smithi162.stdout:1/272: rmdir d0/d7/d2a 39 2024-04-27T04:10:02.462 INFO:tasks.workunit.client.0.smithi162.stdout:2/228: fsync d0/d4/d2a/f36 0 2024-04-27T04:10:02.465 INFO:tasks.workunit.client.0.smithi162.stdout:2/229: read - d0/d4/f3a zero size 2024-04-27T04:10:02.467 INFO:tasks.workunit.client.1.smithi181.stdout:0/615: write dc/d54/f72 [1869142,91028] 0 2024-04-27T04:10:02.467 INFO:tasks.workunit.client.0.smithi162.stdout:8/328: creat d0/de/d20/d6c/f6d x:0 0 0 2024-04-27T04:10:02.471 INFO:tasks.workunit.client.0.smithi162.stdout:0/350: link d5/dc/f17 d5/d26/d4a/f77 0 2024-04-27T04:10:02.473 INFO:tasks.workunit.client.0.smithi162.stdout:9/285: dwrite d3/d13/d15/f57 [0,4194304] 0 2024-04-27T04:10:02.473 INFO:tasks.workunit.client.1.smithi181.stdout:0/616: getdents dc/d1f/daa/dbc 0 2024-04-27T04:10:02.474 INFO:tasks.workunit.client.0.smithi162.stdout:5/278: dwrite d8/d1a/d23/d40/f42 [0,4194304] 0 2024-04-27T04:10:02.476 INFO:tasks.workunit.client.0.smithi162.stdout:6/293: dwrite f2 [0,4194304] 0 2024-04-27T04:10:02.476 INFO:tasks.workunit.client.0.smithi162.stdout:0/351: mkdir d5/d60/d78 0 2024-04-27T04:10:02.477 INFO:tasks.workunit.client.1.smithi181.stdout:0/617: rmdir dc/d1f/d3b/d75 39 2024-04-27T04:10:02.481 INFO:tasks.workunit.client.1.smithi181.stdout:0/618: symlink dc/d1f/d3b/d4a/d7a/ld9 0 2024-04-27T04:10:02.482 INFO:tasks.workunit.client.0.smithi162.stdout:9/286: chown d3/f5 1384192 1 2024-04-27T04:10:02.482 INFO:tasks.workunit.client.0.smithi162.stdout:5/279: read d8/f22 [4809759,86422] 0 2024-04-27T04:10:02.486 INFO:tasks.workunit.client.0.smithi162.stdout:7/237: getdents d3/d5/d9/d4b/de/d1f 0 2024-04-27T04:10:02.487 INFO:tasks.workunit.client.0.smithi162.stdout:6/294: rmdir d6/d1a 39 2024-04-27T04:10:02.491 INFO:tasks.workunit.client.1.smithi181.stdout:0/619: rename c0 to dc/d1f/d3b/d4a/d95/d26/d41/d6f/d8f/cda 0 2024-04-27T04:10:02.491 INFO:tasks.workunit.client.0.smithi162.stdout:2/230: dwrite d0/d4/d17/f2f [0,4194304] 0 2024-04-27T04:10:02.493 INFO:tasks.workunit.client.0.smithi162.stdout:7/238: dread - d3/d5/d9/d4b/de/d2b/f37 zero size 2024-04-27T04:10:02.493 INFO:tasks.workunit.client.0.smithi162.stdout:4/330: dread d2/d25/f8 [0,4194304] 0 2024-04-27T04:10:02.494 INFO:tasks.workunit.client.0.smithi162.stdout:0/352: dwrite d5/d26/f6a [0,4194304] 0 2024-04-27T04:10:02.495 INFO:tasks.workunit.client.1.smithi181.stdout:0/620: dread dc/d1f/d3b/f66 [0,4194304] 0 2024-04-27T04:10:02.497 INFO:tasks.workunit.client.0.smithi162.stdout:0/353: readlink d5/dc/d1e/d56/d57/l66 0 2024-04-27T04:10:02.499 INFO:tasks.workunit.client.1.smithi181.stdout:0/621: creat dc/d1f/d3b/db2/fdb x:0 0 0 2024-04-27T04:10:02.499 INFO:tasks.workunit.client.0.smithi162.stdout:8/329: rename d0/d10/d2b/d23 to d0/de/d6e 0 2024-04-27T04:10:02.501 INFO:tasks.workunit.client.0.smithi162.stdout:6/295: dread d6/d1a/d34/d46/f57 [0,4194304] 0 2024-04-27T04:10:02.502 INFO:tasks.workunit.client.0.smithi162.stdout:5/280: creat d8/d1a/d29/d4a/d5c/f62 x:0 0 0 2024-04-27T04:10:02.504 INFO:tasks.workunit.client.0.smithi162.stdout:8/330: write d0/de/d20/f50 [1180890,11053] 0 2024-04-27T04:10:02.505 INFO:tasks.workunit.client.1.smithi181.stdout:0/622: rename dc/d1f/c40 to dc/d1f/d6b/cdc 0 2024-04-27T04:10:02.509 INFO:tasks.workunit.client.0.smithi162.stdout:9/287: dwrite d3/fb [0,4194304] 0 2024-04-27T04:10:02.509 INFO:tasks.workunit.client.0.smithi162.stdout:2/231: symlink d0/d4/d2a/d3b/l48 0 2024-04-27T04:10:02.509 INFO:tasks.workunit.client.1.smithi181.stdout:0/623: symlink dc/d1f/d6b/ldd 0 2024-04-27T04:10:02.510 INFO:tasks.workunit.client.0.smithi162.stdout:4/331: dwrite d2/d25/d1c/f55 [0,4194304] 0 2024-04-27T04:10:02.510 INFO:tasks.workunit.client.1.smithi181.stdout:0/624: write dc/d1f/d6b/fa9 [519844,3984] 0 2024-04-27T04:10:02.511 INFO:tasks.workunit.client.0.smithi162.stdout:1/273: write d0/d7/d2a/f2d [3142862,19934] 0 2024-04-27T04:10:02.511 INFO:tasks.workunit.client.0.smithi162.stdout:4/332: stat d2/d25/f8 0 2024-04-27T04:10:02.511 INFO:tasks.workunit.client.1.smithi181.stdout:0/625: readlink dc/l8e 0 2024-04-27T04:10:02.511 INFO:tasks.workunit.client.1.smithi181.stdout:0/626: stat dc/l27 0 2024-04-27T04:10:02.515 INFO:tasks.workunit.client.1.smithi181.stdout:0/627: rmdir dc/d1f/d3b/d4a/d95/d26/d41/db0 39 2024-04-27T04:10:02.517 INFO:tasks.workunit.client.0.smithi162.stdout:5/281: mkdir d8/d1a/d29/d4a/d5c/d63 0 2024-04-27T04:10:02.519 INFO:tasks.workunit.client.0.smithi162.stdout:7/239: creat d3/d39/f52 x:0 0 0 2024-04-27T04:10:02.520 INFO:tasks.workunit.client.0.smithi162.stdout:4/333: dread d2/d25/f5c [0,4194304] 0 2024-04-27T04:10:02.524 INFO:tasks.workunit.client.0.smithi162.stdout:8/331: mknod d0/d10/d44/c6f 0 2024-04-27T04:10:02.524 INFO:tasks.workunit.client.0.smithi162.stdout:5/282: write d8/f26 [1012151,42629] 0 2024-04-27T04:10:02.525 INFO:tasks.workunit.client.0.smithi162.stdout:2/232: unlink d0/d4/c14 0 2024-04-27T04:10:02.526 INFO:tasks.workunit.client.0.smithi162.stdout:1/274: dwrite d0/fe [0,4194304] 0 2024-04-27T04:10:02.526 INFO:tasks.workunit.client.1.smithi181.stdout:0/628: getdents dc/d1f/d6b/db9/dba 0 2024-04-27T04:10:02.527 INFO:tasks.workunit.client.0.smithi162.stdout:9/288: symlink d3/d13/d2d/d30/l63 0 2024-04-27T04:10:02.528 INFO:tasks.workunit.client.1.smithi181.stdout:0/629: truncate dc/d54/fab 474186 0 2024-04-27T04:10:02.529 INFO:tasks.workunit.client.0.smithi162.stdout:7/240: fsync d3/d5/d9/d4b/f19 0 2024-04-27T04:10:02.529 INFO:tasks.workunit.client.0.smithi162.stdout:0/354: rmdir d5/d60/d78 0 2024-04-27T04:10:02.533 INFO:tasks.workunit.client.0.smithi162.stdout:7/241: truncate d3/d5/d9/d4b/de/d1f/f42 1033610 0 2024-04-27T04:10:02.535 INFO:tasks.workunit.client.1.smithi181.stdout:0/630: mkdir dc/d1f/d6b/db9/dba/dce/dde 0 2024-04-27T04:10:02.537 INFO:tasks.workunit.client.1.smithi181.stdout:6/404: sync 2024-04-27T04:10:02.537 INFO:tasks.workunit.client.1.smithi181.stdout:1/535: sync 2024-04-27T04:10:02.537 INFO:tasks.workunit.client.0.smithi162.stdout:9/289: symlink d3/d13/d2d/l64 0 2024-04-27T04:10:02.539 INFO:tasks.workunit.client.0.smithi162.stdout:4/334: symlink d2/d25/d1c/d39/d5e/l6b 0 2024-04-27T04:10:02.544 INFO:tasks.workunit.client.0.smithi162.stdout:2/233: rename d0/d20/d2e/d46 to d0/d4/d2a/d3b/d49 0 2024-04-27T04:10:02.546 INFO:tasks.workunit.client.0.smithi162.stdout:9/290: chown d3/c10 2 1 2024-04-27T04:10:02.548 INFO:tasks.workunit.client.0.smithi162.stdout:9/291: fsync d3/fb 0 2024-04-27T04:10:02.548 INFO:tasks.workunit.client.0.smithi162.stdout:3/290: sync 2024-04-27T04:10:02.548 INFO:tasks.workunit.client.1.smithi181.stdout:6/405: getdents d7/dc 0 2024-04-27T04:10:02.549 INFO:tasks.workunit.client.0.smithi162.stdout:5/283: creat d8/d1a/d29/d4a/d5c/d63/f64 x:0 0 0 2024-04-27T04:10:02.550 INFO:tasks.workunit.client.0.smithi162.stdout:3/291: dread d1/d18/d23/d34/f3c [0,4194304] 0 2024-04-27T04:10:02.550 INFO:tasks.workunit.client.0.smithi162.stdout:5/284: read d8/d1a/f2d [21725,99693] 0 2024-04-27T04:10:02.551 INFO:tasks.workunit.client.0.smithi162.stdout:5/285: chown d8/d2c/d41/c56 382 1 2024-04-27T04:10:02.551 INFO:tasks.workunit.client.1.smithi181.stdout:6/406: creat d7/d31/f8c x:0 0 0 2024-04-27T04:10:02.552 INFO:tasks.workunit.client.0.smithi162.stdout:5/286: read - d8/d1a/d23/f50 zero size 2024-04-27T04:10:02.552 INFO:tasks.workunit.client.1.smithi181.stdout:6/407: dread - d7/dc/d1c/f78 zero size 2024-04-27T04:10:02.552 INFO:tasks.workunit.client.0.smithi162.stdout:5/287: stat d8/d9/l54 0 2024-04-27T04:10:02.553 INFO:tasks.workunit.client.0.smithi162.stdout:1/275: creat d0/d7/f5a x:0 0 0 2024-04-27T04:10:02.557 INFO:tasks.workunit.client.0.smithi162.stdout:0/355: dread d5/dc/f14 [0,4194304] 0 2024-04-27T04:10:02.558 INFO:tasks.workunit.client.1.smithi181.stdout:6/408: dwrite d7/d31/d45/d62/f63 [0,4194304] 0 2024-04-27T04:10:02.560 INFO:tasks.workunit.client.0.smithi162.stdout:6/296: dread d6/d1a/f20 [0,4194304] 0 2024-04-27T04:10:02.560 INFO:tasks.workunit.client.1.smithi181.stdout:6/409: readlink d7/d27/l36 0 2024-04-27T04:10:02.561 INFO:tasks.workunit.client.0.smithi162.stdout:4/335: fdatasync d2/f38 0 2024-04-27T04:10:02.562 INFO:tasks.workunit.client.0.smithi162.stdout:4/336: readlink d2/da/l53 0 2024-04-27T04:10:02.564 INFO:tasks.workunit.client.0.smithi162.stdout:3/292: mkdir d1/d18/d5d 0 2024-04-27T04:10:02.569 INFO:tasks.workunit.client.0.smithi162.stdout:2/234: creat d0/d4/d2a/d41/f4a x:0 0 0 2024-04-27T04:10:02.570 INFO:tasks.workunit.client.0.smithi162.stdout:2/235: dread - d0/d4/f42 zero size 2024-04-27T04:10:02.570 INFO:tasks.workunit.client.1.smithi181.stdout:6/410: link d7/dc/c14 d7/dc/d1e/d66/c8d 0 2024-04-27T04:10:02.571 INFO:tasks.workunit.client.0.smithi162.stdout:8/332: getdents d0/d10/d2b/d2d/d2f/d39 0 2024-04-27T04:10:02.572 INFO:tasks.workunit.client.0.smithi162.stdout:3/293: dwrite d1/d18/f52 [0,4194304] 0 2024-04-27T04:10:02.572 INFO:tasks.workunit.client.0.smithi162.stdout:0/356: mkdir d5/dc/d1e/d41/d79 0 2024-04-27T04:10:02.574 INFO:tasks.workunit.client.0.smithi162.stdout:7/242: write d3/d5/d9/d4b/f33 [3282083,130780] 0 2024-04-27T04:10:02.578 INFO:tasks.workunit.client.0.smithi162.stdout:7/243: read - d3/d5/d9/d4b/de/f38 zero size 2024-04-27T04:10:02.581 INFO:tasks.workunit.client.0.smithi162.stdout:1/276: mkdir d0/d11/d5b 0 2024-04-27T04:10:02.581 INFO:tasks.workunit.client.0.smithi162.stdout:2/236: dwrite d0/f1 [4194304,4194304] 0 2024-04-27T04:10:02.581 INFO:tasks.workunit.client.0.smithi162.stdout:7/244: fdatasync d3/d5/d9/d4b/de/f30 0 2024-04-27T04:10:02.583 INFO:tasks.workunit.client.0.smithi162.stdout:8/333: dread d0/d10/d25/f36 [4194304,4194304] 0 2024-04-27T04:10:02.583 INFO:tasks.workunit.client.0.smithi162.stdout:2/237: truncate d0/d4/f22 5191141 0 2024-04-27T04:10:02.584 INFO:tasks.workunit.client.0.smithi162.stdout:0/357: creat d5/dc/f7a x:0 0 0 2024-04-27T04:10:02.585 INFO:tasks.workunit.client.0.smithi162.stdout:3/294: creat d1/d21/d47/d57/f5e x:0 0 0 2024-04-27T04:10:02.585 INFO:tasks.workunit.client.0.smithi162.stdout:9/292: link d3/d13/d2d/c36 d3/d13/d15/d50/c65 0 2024-04-27T04:10:02.586 INFO:tasks.workunit.client.0.smithi162.stdout:0/358: fdatasync d5/dc/d1e/f30 0 2024-04-27T04:10:02.590 INFO:tasks.workunit.client.0.smithi162.stdout:5/288: getdents d8/d1a/d23/d40/d4b 0 2024-04-27T04:10:02.594 INFO:tasks.workunit.client.0.smithi162.stdout:1/277: dwrite d0/d11/d3e/f41 [4194304,4194304] 0 2024-04-27T04:10:02.594 INFO:tasks.workunit.client.0.smithi162.stdout:4/337: dread d2/d25/d1c/d39/f45 [0,4194304] 0 2024-04-27T04:10:02.595 INFO:tasks.workunit.client.0.smithi162.stdout:9/293: symlink d3/d13/d15/l66 0 2024-04-27T04:10:02.597 INFO:tasks.workunit.client.0.smithi162.stdout:8/334: unlink d0/d10/d2b/l69 0 2024-04-27T04:10:02.598 INFO:tasks.workunit.client.0.smithi162.stdout:1/278: truncate d0/d7/f5a 466088 0 2024-04-27T04:10:02.598 INFO:tasks.workunit.client.0.smithi162.stdout:2/238: dwrite d0/d4/d17/f3c [0,4194304] 0 2024-04-27T04:10:02.600 INFO:tasks.workunit.client.0.smithi162.stdout:8/335: write d0/d10/d2b/d2e/f46 [1270619,65775] 0 2024-04-27T04:10:02.600 INFO:tasks.workunit.client.0.smithi162.stdout:2/239: chown d0/d24/f38 15700984 1 2024-04-27T04:10:02.601 INFO:tasks.workunit.client.0.smithi162.stdout:5/289: creat d8/d9/d55/f65 x:0 0 0 2024-04-27T04:10:02.601 INFO:tasks.workunit.client.0.smithi162.stdout:6/297: write f5 [1118198,3513] 0 2024-04-27T04:10:02.605 INFO:tasks.workunit.client.0.smithi162.stdout:7/245: creat d3/d5/f53 x:0 0 0 2024-04-27T04:10:02.605 INFO:tasks.workunit.client.1.smithi181.stdout:6/411: truncate d7/dc/d1e/f52 7087031 0 2024-04-27T04:10:02.609 INFO:tasks.workunit.client.0.smithi162.stdout:5/290: dwrite d8/d1a/f45 [0,4194304] 0 2024-04-27T04:10:02.617 INFO:tasks.workunit.client.0.smithi162.stdout:4/338: rename d2/da/dd/f61 to d2/d25/d1c/d39/d3b/d47/f6c 0 2024-04-27T04:10:02.621 INFO:tasks.workunit.client.0.smithi162.stdout:1/279: write d0/d7/f19 [5464857,88813] 0 2024-04-27T04:10:02.623 INFO:tasks.workunit.client.1.smithi181.stdout:6/412: rmdir d7/dc/d1c/d89 0 2024-04-27T04:10:02.626 INFO:tasks.workunit.client.0.smithi162.stdout:3/295: write d1/d18/f11 [1420258,129452] 0 2024-04-27T04:10:02.628 INFO:tasks.workunit.client.0.smithi162.stdout:6/298: unlink d6/db/c2c 0 2024-04-27T04:10:02.629 INFO:tasks.workunit.client.0.smithi162.stdout:8/336: dwrite d0/de/d20/f60 [4194304,4194304] 0 2024-04-27T04:10:02.630 INFO:tasks.workunit.client.1.smithi181.stdout:6/413: dwrite d7/dc/d1c/d7f/d3e/f54 [0,4194304] 0 2024-04-27T04:10:02.632 INFO:tasks.workunit.client.0.smithi162.stdout:0/359: truncate d5/dc/f31 2301976 0 2024-04-27T04:10:02.633 INFO:tasks.workunit.client.0.smithi162.stdout:5/291: fsync d8/d1a/f2d 0 2024-04-27T04:10:02.637 INFO:tasks.workunit.client.0.smithi162.stdout:7/246: dwrite d3/d5/d9/d4b/f35 [0,4194304] 0 2024-04-27T04:10:02.641 INFO:tasks.workunit.client.0.smithi162.stdout:2/240: fdatasync d0/d4/f16 0 2024-04-27T04:10:02.643 INFO:tasks.workunit.client.0.smithi162.stdout:2/241: write d0/fe [2418105,91711] 0 2024-04-27T04:10:02.651 INFO:tasks.workunit.client.0.smithi162.stdout:2/242: dwrite d0/d4/d17/f2f [0,4194304] 0 2024-04-27T04:10:02.652 INFO:tasks.workunit.client.0.smithi162.stdout:0/360: mknod d5/dc/c7b 0 2024-04-27T04:10:02.653 INFO:tasks.workunit.client.0.smithi162.stdout:2/243: write d0/d4/d2a/f36 [4930729,34783] 0 2024-04-27T04:10:02.656 INFO:tasks.workunit.client.0.smithi162.stdout:5/292: mkdir d8/d1a/d23/d40/d4b/d66 0 2024-04-27T04:10:02.659 INFO:tasks.workunit.client.0.smithi162.stdout:4/339: link d2/d25/d1c/f3d d2/d25/d1c/d24/f6d 0 2024-04-27T04:10:02.661 INFO:tasks.workunit.client.0.smithi162.stdout:3/296: getdents d1/d18/d4f 0 2024-04-27T04:10:02.662 INFO:tasks.workunit.client.1.smithi181.stdout:6/414: dread d7/f19 [0,4194304] 0 2024-04-27T04:10:02.666 INFO:tasks.workunit.client.0.smithi162.stdout:5/293: unlink d8/d9/c3a 0 2024-04-27T04:10:02.666 INFO:tasks.workunit.client.1.smithi181.stdout:6/415: dread d7/f26 [4194304,4194304] 0 2024-04-27T04:10:02.667 INFO:tasks.workunit.client.0.smithi162.stdout:2/244: fsync d0/d4/d2a/f36 0 2024-04-27T04:10:02.670 INFO:tasks.workunit.client.0.smithi162.stdout:7/247: mknod d3/c54 0 2024-04-27T04:10:02.671 INFO:tasks.workunit.client.0.smithi162.stdout:4/340: mkdir d2/d25/d1c/d39/d3b/d6e 0 2024-04-27T04:10:02.671 INFO:tasks.workunit.client.0.smithi162.stdout:7/248: write d3/d5/d9/d4b/de/d1f/f44 [1002442,65452] 0 2024-04-27T04:10:02.672 INFO:tasks.workunit.client.1.smithi181.stdout:6/416: creat d7/dc/d1c/d34/d3f/d7e/d8a/f8e x:0 0 0 2024-04-27T04:10:02.676 INFO:tasks.workunit.client.0.smithi162.stdout:6/299: write d6/d1a/f1d [1807488,104767] 0 2024-04-27T04:10:02.677 INFO:tasks.workunit.client.1.smithi181.stdout:6/417: unlink d7/dc/d1c/f78 0 2024-04-27T04:10:02.678 INFO:tasks.workunit.client.0.smithi162.stdout:9/294: truncate d3/fc 1518625 0 2024-04-27T04:10:02.679 INFO:tasks.workunit.client.0.smithi162.stdout:9/295: dread - d3/d11/d45/f55 zero size 2024-04-27T04:10:02.681 INFO:tasks.workunit.client.0.smithi162.stdout:0/361: truncate d5/dc/d1e/f30 24718 0 2024-04-27T04:10:02.683 INFO:tasks.workunit.client.0.smithi162.stdout:5/294: write d8/d1a/d29/f49 [1220756,93024] 0 2024-04-27T04:10:02.684 INFO:tasks.workunit.client.0.smithi162.stdout:5/295: stat d8/d9/l54 0 2024-04-27T04:10:02.684 INFO:tasks.workunit.client.0.smithi162.stdout:5/296: fsync d8/d2c/f33 0 2024-04-27T04:10:02.685 INFO:tasks.workunit.client.0.smithi162.stdout:2/245: rename d0/d4/d2a/f43 to d0/d24/f4b 0 2024-04-27T04:10:02.685 INFO:tasks.workunit.client.0.smithi162.stdout:5/297: write d8/d1a/d29/f49 [443542,95257] 0 2024-04-27T04:10:02.686 INFO:tasks.workunit.client.1.smithi181.stdout:6/418: symlink d7/dc/d1c/d34/l8f 0 2024-04-27T04:10:02.687 INFO:tasks.workunit.client.0.smithi162.stdout:5/298: write d8/d1a/d23/d40/f42 [2664976,77120] 0 2024-04-27T04:10:02.687 INFO:tasks.workunit.client.0.smithi162.stdout:1/280: write d0/d7/d10/f12 [1907366,29560] 0 2024-04-27T04:10:02.688 INFO:tasks.workunit.client.0.smithi162.stdout:1/281: stat d0/d7/d10/f12 0 2024-04-27T04:10:02.690 INFO:tasks.workunit.client.0.smithi162.stdout:8/337: fdatasync d0/d10/d44/f47 0 2024-04-27T04:10:02.693 INFO:tasks.workunit.client.0.smithi162.stdout:3/297: mkdir d1/d18/d5d/d5f 0 2024-04-27T04:10:02.696 INFO:tasks.workunit.client.0.smithi162.stdout:9/296: mkdir d3/d13/d2d/d30/d67 0 2024-04-27T04:10:02.701 INFO:tasks.workunit.client.1.smithi181.stdout:6/419: dread d7/dc/f12 [0,4194304] 0 2024-04-27T04:10:02.702 INFO:tasks.workunit.client.0.smithi162.stdout:4/341: mkdir d2/d25/d1c/d39/d3b/d6e/d6f 0 2024-04-27T04:10:02.702 INFO:tasks.workunit.client.0.smithi162.stdout:9/297: dwrite d3/d23/f3e [0,4194304] 0 2024-04-27T04:10:02.703 INFO:tasks.workunit.client.0.smithi162.stdout:5/299: mknod d8/d2c/c67 0 2024-04-27T04:10:02.704 INFO:tasks.workunit.client.0.smithi162.stdout:7/249: mkdir d3/d5/d3c/d4a/d55 0 2024-04-27T04:10:02.705 INFO:tasks.workunit.client.0.smithi162.stdout:9/298: truncate d3/d13/f41 995107 0 2024-04-27T04:10:02.705 INFO:tasks.workunit.client.0.smithi162.stdout:7/250: chown d3/c54 16673282 1 2024-04-27T04:10:02.706 INFO:tasks.workunit.client.1.smithi181.stdout:6/420: rename d7/d31/f6f to d7/dc/d1e/d66/f90 0 2024-04-27T04:10:02.706 INFO:tasks.workunit.client.0.smithi162.stdout:7/251: fdatasync d3/d5/d9/d4b/f3f 0 2024-04-27T04:10:02.707 INFO:tasks.workunit.client.0.smithi162.stdout:1/282: mknod d0/d11/d3e/c5c 0 2024-04-27T04:10:02.708 INFO:tasks.workunit.client.0.smithi162.stdout:7/252: write d3/d4f/f31 [929091,93525] 0 2024-04-27T04:10:02.710 INFO:tasks.workunit.client.0.smithi162.stdout:6/300: dread d6/d1a/f18 [0,4194304] 0 2024-04-27T04:10:02.711 INFO:tasks.workunit.client.1.smithi181.stdout:5/647: sync 2024-04-27T04:10:02.712 INFO:tasks.workunit.client.1.smithi181.stdout:3/458: sync 2024-04-27T04:10:02.712 INFO:tasks.workunit.client.1.smithi181.stdout:8/599: sync 2024-04-27T04:10:02.712 INFO:tasks.workunit.client.1.smithi181.stdout:9/515: sync 2024-04-27T04:10:02.712 INFO:tasks.workunit.client.1.smithi181.stdout:7/499: sync 2024-04-27T04:10:02.712 INFO:tasks.workunit.client.1.smithi181.stdout:2/561: sync 2024-04-27T04:10:02.712 INFO:tasks.workunit.client.1.smithi181.stdout:4/470: sync 2024-04-27T04:10:02.712 INFO:tasks.workunit.client.1.smithi181.stdout:9/516: stat d0/d4d/fa5 0 2024-04-27T04:10:02.713 INFO:tasks.workunit.client.0.smithi162.stdout:3/298: creat d1/d21/f60 x:0 0 0 2024-04-27T04:10:02.715 INFO:tasks.workunit.client.0.smithi162.stdout:6/301: dwrite d6/db/d32/f3b [0,4194304] 0 2024-04-27T04:10:02.717 INFO:tasks.workunit.client.1.smithi181.stdout:9/517: read d0/d44/d4b/d3d/f73 [174208,109558] 0 2024-04-27T04:10:02.717 INFO:tasks.workunit.client.1.smithi181.stdout:7/500: dread d1/d2/f19 [4194304,4194304] 0 2024-04-27T04:10:02.720 INFO:tasks.workunit.client.0.smithi162.stdout:2/246: symlink d0/l4c 0 2024-04-27T04:10:02.721 INFO:tasks.workunit.client.1.smithi181.stdout:2/562: dwrite d4/d9/d35/f51 [0,4194304] 0 2024-04-27T04:10:02.729 INFO:tasks.workunit.client.1.smithi181.stdout:6/421: write d7/d27/f4c [1207745,48928] 0 2024-04-27T04:10:02.730 INFO:tasks.workunit.client.1.smithi181.stdout:2/563: dwrite d4/d9/d16/d82/fa5 [0,4194304] 0 2024-04-27T04:10:02.731 INFO:tasks.workunit.client.1.smithi181.stdout:6/422: readlink d7/dc/d1c/d34/d3f/l5e 0 2024-04-27T04:10:02.733 INFO:tasks.workunit.client.1.smithi181.stdout:8/600: mkdir d1/d6/d6f/dd2 0 2024-04-27T04:10:02.734 INFO:tasks.workunit.client.0.smithi162.stdout:1/283: creat d0/d7/d2a/f5d x:0 0 0 2024-04-27T04:10:02.735 INFO:tasks.workunit.client.1.smithi181.stdout:9/518: creat d0/d44/d4f/d9d/fb4 x:0 0 0 2024-04-27T04:10:02.737 INFO:tasks.workunit.client.0.smithi162.stdout:7/253: creat d3/d5/d9/d4b/de/d2b/f56 x:0 0 0 2024-04-27T04:10:02.738 INFO:tasks.workunit.client.1.smithi181.stdout:7/501: symlink d1/d2/d1c/d8f/l9b 0 2024-04-27T04:10:02.738 INFO:tasks.workunit.client.0.smithi162.stdout:7/254: truncate d3/d5/d9/f26 980978 0 2024-04-27T04:10:02.739 INFO:tasks.workunit.client.0.smithi162.stdout:7/255: chown d3/d5/d9/d4b 754798 1 2024-04-27T04:10:02.739 INFO:tasks.workunit.client.0.smithi162.stdout:8/338: mkdir d0/d10/d44/d70 0 2024-04-27T04:10:02.740 INFO:tasks.workunit.client.0.smithi162.stdout:7/256: stat d3/d39 0 2024-04-27T04:10:02.740 INFO:tasks.workunit.client.1.smithi181.stdout:7/502: write d1/d22/d15/d29/f43 [809385,115613] 0 2024-04-27T04:10:02.747 INFO:tasks.workunit.client.0.smithi162.stdout:6/302: unlink d6/d1a/d34/d3a/f4c 0 2024-04-27T04:10:02.749 INFO:tasks.workunit.client.0.smithi162.stdout:2/247: creat d0/d24/f4d x:0 0 0 2024-04-27T04:10:02.750 INFO:tasks.workunit.client.0.smithi162.stdout:2/248: fsync d0/d4/d17/f2f 0 2024-04-27T04:10:02.752 INFO:tasks.workunit.client.1.smithi181.stdout:2/564: dwrite d4/d9/d35/d42/f5a [0,4194304] 0 2024-04-27T04:10:02.753 INFO:tasks.workunit.client.1.smithi181.stdout:9/519: rename d0/d4d/f7e to d0/d4d/fb5 0 2024-04-27T04:10:02.754 INFO:tasks.workunit.client.0.smithi162.stdout:5/300: symlink d8/d9/d5a/l68 0 2024-04-27T04:10:02.755 INFO:tasks.workunit.client.0.smithi162.stdout:5/301: fdatasync d8/d2c/f4c 0 2024-04-27T04:10:02.756 INFO:tasks.workunit.client.0.smithi162.stdout:5/302: truncate d8/d1a/d29/f4d 485164 0 2024-04-27T04:10:02.757 INFO:tasks.workunit.client.1.smithi181.stdout:3/459: write d0/d6c/d52/d44/d56/f63 [149027,128094] 0 2024-04-27T04:10:02.758 INFO:tasks.workunit.client.1.smithi181.stdout:4/471: link d0/d15/de/d60/d6f/c8c d0/d15/d25/d2a/c99 0 2024-04-27T04:10:02.760 INFO:tasks.workunit.client.1.smithi181.stdout:3/460: write d0/de/d20/f30 [1553980,126792] 0 2024-04-27T04:10:02.760 INFO:tasks.workunit.client.1.smithi181.stdout:2/565: dwrite d4/d9/d16/d82/f71 [0,4194304] 0 2024-04-27T04:10:02.764 INFO:tasks.workunit.client.1.smithi181.stdout:6/423: write d7/dc/f41 [297220,99100] 0 2024-04-27T04:10:02.765 INFO:tasks.workunit.client.0.smithi162.stdout:8/339: mkdir d0/de/d6e/d71 0 2024-04-27T04:10:02.770 INFO:tasks.workunit.client.0.smithi162.stdout:7/257: rename d3/d5/d9/d4b/de/d1f to d3/d39/d57 0 2024-04-27T04:10:02.771 INFO:tasks.workunit.client.0.smithi162.stdout:7/258: chown d3/d5/f53 230 1 2024-04-27T04:10:02.771 INFO:tasks.workunit.client.0.smithi162.stdout:4/342: truncate d2/d25/f36 3151562 0 2024-04-27T04:10:02.772 INFO:tasks.workunit.client.0.smithi162.stdout:3/299: mknod d1/d18/d5d/d5f/c61 0 2024-04-27T04:10:02.772 INFO:tasks.workunit.client.0.smithi162.stdout:4/343: chown d2/d25/d1c/d39/d3b/d47/f4c 2 1 2024-04-27T04:10:02.773 INFO:tasks.workunit.client.1.smithi181.stdout:7/503: write d1/d22/f60 [783144,37982] 0 2024-04-27T04:10:02.773 INFO:tasks.workunit.client.0.smithi162.stdout:3/300: stat d1/d18/f1e 0 2024-04-27T04:10:02.774 INFO:tasks.workunit.client.1.smithi181.stdout:4/472: mknod d0/d15/d76/c9a 0 2024-04-27T04:10:02.775 INFO:tasks.workunit.client.1.smithi181.stdout:3/461: symlink d0/d6c/d52/d44/d56/d5b/l92 0 2024-04-27T04:10:02.775 INFO:tasks.workunit.client.1.smithi181.stdout:4/473: truncate d0/d15/de/d19/f94 688979 0 2024-04-27T04:10:02.776 INFO:tasks.workunit.client.0.smithi162.stdout:5/303: creat d8/d9/d55/f69 x:0 0 0 2024-04-27T04:10:02.777 INFO:tasks.workunit.client.0.smithi162.stdout:3/301: dwrite d1/d18/f1f [0,4194304] 0 2024-04-27T04:10:02.777 INFO:tasks.workunit.client.1.smithi181.stdout:2/566: symlink d4/d9/d16/d45/d9a/d9d/lb2 0 2024-04-27T04:10:02.781 INFO:tasks.workunit.client.1.smithi181.stdout:8/601: rmdir d1/d6/d2f/d30/d66/dc0/dd0 0 2024-04-27T04:10:02.784 INFO:tasks.workunit.client.1.smithi181.stdout:7/504: mknod d1/df/d1a/c9c 0 2024-04-27T04:10:02.787 INFO:tasks.workunit.client.1.smithi181.stdout:4/474: creat d0/d15/d25/d40/f9b x:0 0 0 2024-04-27T04:10:02.787 INFO:tasks.workunit.client.1.smithi181.stdout:2/567: dwrite d4/d9/d16/d82/d7d/f9b [0,4194304] 0 2024-04-27T04:10:02.789 INFO:tasks.workunit.client.1.smithi181.stdout:4/475: dread - d0/d15/d25/d2a/d36/f92 zero size 2024-04-27T04:10:02.790 INFO:tasks.workunit.client.0.smithi162.stdout:4/344: dwrite d2/f4 [0,4194304] 0 2024-04-27T04:10:02.794 INFO:tasks.workunit.client.0.smithi162.stdout:2/249: mknod d0/c4e 0 2024-04-27T04:10:02.796 INFO:tasks.workunit.client.0.smithi162.stdout:2/250: write d0/fe [783033,10067] 0 2024-04-27T04:10:02.798 INFO:tasks.workunit.client.0.smithi162.stdout:5/304: unlink d8/d9/f32 0 2024-04-27T04:10:02.800 INFO:tasks.workunit.client.0.smithi162.stdout:9/299: getdents d3/d13/d2d/d30 0 2024-04-27T04:10:02.801 INFO:tasks.workunit.client.1.smithi181.stdout:8/602: rmdir d1/d6/d6f/d91 39 2024-04-27T04:10:02.805 INFO:tasks.workunit.client.0.smithi162.stdout:8/340: mknod d0/de/d6e/d71/c72 0 2024-04-27T04:10:02.806 INFO:tasks.workunit.client.0.smithi162.stdout:1/284: truncate d0/f3b 2797626 0 2024-04-27T04:10:02.808 INFO:tasks.workunit.client.0.smithi162.stdout:3/302: dwrite d1/f3f [0,4194304] 0 2024-04-27T04:10:02.810 INFO:tasks.workunit.client.0.smithi162.stdout:4/345: mkdir d2/d25/d1c/d39/d70 0 2024-04-27T04:10:02.814 INFO:tasks.workunit.client.1.smithi181.stdout:2/568: rename d4/d9/df/d14/f15 to d4/d9/d16/d45/d9a/d9d/fb3 0 2024-04-27T04:10:02.816 INFO:tasks.workunit.client.1.smithi181.stdout:6/424: rename d7/dc/d1c/f32 to d7/dc/d1c/f91 0 2024-04-27T04:10:02.819 INFO:tasks.workunit.client.1.smithi181.stdout:9/520: getdents d0/d44/d4b/d3d 0 2024-04-27T04:10:02.821 INFO:tasks.workunit.client.0.smithi162.stdout:5/305: fsync d8/d9/f10 0 2024-04-27T04:10:02.821 INFO:tasks.workunit.client.1.smithi181.stdout:3/462: truncate d0/f1 2635175 0 2024-04-27T04:10:02.825 INFO:tasks.workunit.client.1.smithi181.stdout:7/505: write d1/d2/f12 [9597644,54224] 0 2024-04-27T04:10:02.830 INFO:tasks.workunit.client.0.smithi162.stdout:8/341: creat d0/d10/d44/f73 x:0 0 0 2024-04-27T04:10:02.831 INFO:tasks.workunit.client.1.smithi181.stdout:4/476: dwrite d0/d15/d25/d2a/d36/f89 [0,4194304] 0 2024-04-27T04:10:02.837 INFO:tasks.workunit.client.1.smithi181.stdout:9/521: truncate d0/f9 7068335 0 2024-04-27T04:10:02.841 INFO:tasks.workunit.client.1.smithi181.stdout:8/603: write d1/d6/d2f/f3a [1304812,75194] 0 2024-04-27T04:10:02.841 INFO:tasks.workunit.client.0.smithi162.stdout:4/346: creat d2/d25/d1c/d39/f71 x:0 0 0 2024-04-27T04:10:02.841 INFO:tasks.workunit.client.1.smithi181.stdout:3/463: mknod d0/d6c/d52/d44/d56/d5b/d77/c93 0 2024-04-27T04:10:02.848 INFO:tasks.workunit.client.0.smithi162.stdout:5/306: truncate d8/d1a/f2d 57039 0 2024-04-27T04:10:02.852 INFO:tasks.workunit.client.0.smithi162.stdout:9/300: creat d3/d11/d45/d4e/d58/f68 x:0 0 0 2024-04-27T04:10:02.852 INFO:tasks.workunit.client.1.smithi181.stdout:6/425: dwrite d7/d31/d45/f4b [0,4194304] 0 2024-04-27T04:10:02.854 INFO:tasks.workunit.client.0.smithi162.stdout:1/285: mkdir d0/d7/d5e 0 2024-04-27T04:10:02.855 INFO:tasks.workunit.client.1.smithi181.stdout:3/464: mknod d0/de/d7d/c94 0 2024-04-27T04:10:02.856 INFO:tasks.workunit.client.0.smithi162.stdout:0/362: sync 2024-04-27T04:10:02.856 INFO:tasks.workunit.client.0.smithi162.stdout:8/342: mknod d0/de/d20/d6c/c74 0 2024-04-27T04:10:02.857 INFO:tasks.workunit.client.0.smithi162.stdout:1/286: dread d0/d11/f20 [4194304,4194304] 0 2024-04-27T04:10:02.859 INFO:tasks.workunit.client.0.smithi162.stdout:7/259: getdents d3/d39 0 2024-04-27T04:10:02.863 INFO:tasks.workunit.client.0.smithi162.stdout:3/303: link d1/d18/f13 d1/d18/d5d/f62 0 2024-04-27T04:10:02.864 INFO:tasks.workunit.client.1.smithi181.stdout:4/477: dwrite d0/d15/d25/d2b/f2c [0,4194304] 0 2024-04-27T04:10:02.866 INFO:tasks.workunit.client.1.smithi181.stdout:2/569: rename d4/d9/d44/cab to d4/d9/d35/cb4 0 2024-04-27T04:10:02.874 INFO:tasks.workunit.client.1.smithi181.stdout:4/478: dread d0/d15/f50 [0,4194304] 0 2024-04-27T04:10:02.875 INFO:tasks.workunit.client.1.smithi181.stdout:4/479: stat d0/d15/de/d19/d77/d57/f84 0 2024-04-27T04:10:02.875 INFO:tasks.workunit.client.1.smithi181.stdout:6/426: creat d7/d31/d45/d72/f92 x:0 0 0 2024-04-27T04:10:02.877 INFO:tasks.workunit.client.0.smithi162.stdout:5/307: symlink d8/d1a/d23/l6a 0 2024-04-27T04:10:02.878 INFO:tasks.workunit.client.0.smithi162.stdout:5/308: chown d8/d9/d55/f69 6430 1 2024-04-27T04:10:02.878 INFO:tasks.workunit.client.0.smithi162.stdout:5/309: chown d8/d1a/d29/f49 19949 1 2024-04-27T04:10:02.880 INFO:tasks.workunit.client.0.smithi162.stdout:0/363: readlink d5/l1c 0 2024-04-27T04:10:02.883 INFO:tasks.workunit.client.0.smithi162.stdout:1/287: mkdir d0/d11/d3e/d5f 0 2024-04-27T04:10:02.884 INFO:tasks.workunit.client.1.smithi181.stdout:7/506: link d1/d22/d15/d13/d1b/c6b d1/d57/d72/c9d 0 2024-04-27T04:10:02.885 INFO:tasks.workunit.client.0.smithi162.stdout:7/260: mknod d3/d5/d9/d4b/de/c58 0 2024-04-27T04:10:02.885 INFO:tasks.workunit.client.1.smithi181.stdout:3/465: write d0/d6c/f79 [389635,103563] 0 2024-04-27T04:10:02.887 INFO:tasks.workunit.client.1.smithi181.stdout:3/466: write d0/de/d20/d58/d70/f81 [155305,44676] 0 2024-04-27T04:10:02.889 INFO:tasks.workunit.client.1.smithi181.stdout:8/604: truncate d1/d20/fa2 3012613 0 2024-04-27T04:10:02.892 INFO:tasks.workunit.client.0.smithi162.stdout:4/347: rename d2/d5a to d2/d25/d1c/d72 0 2024-04-27T04:10:02.893 INFO:tasks.workunit.client.1.smithi181.stdout:3/467: dwrite d0/de/f17 [0,4194304] 0 2024-04-27T04:10:02.893 INFO:tasks.workunit.client.1.smithi181.stdout:2/570: creat d4/d9/d4f/d79/d94/fb5 x:0 0 0 2024-04-27T04:10:02.894 INFO:tasks.workunit.client.1.smithi181.stdout:2/571: write d4/d9/d4f/f5b [5104676,55880] 0 2024-04-27T04:10:02.897 INFO:tasks.workunit.client.0.smithi162.stdout:3/304: write d1/d18/d23/f2b [364826,126292] 0 2024-04-27T04:10:02.898 INFO:tasks.workunit.client.1.smithi181.stdout:4/480: mknod d0/d15/d25/d2a/d36/c9c 0 2024-04-27T04:10:02.898 INFO:tasks.workunit.client.0.smithi162.stdout:3/305: chown d1/d37/f51 14 1 2024-04-27T04:10:02.899 INFO:tasks.workunit.client.1.smithi181.stdout:2/572: dread d4/d81/f8e [0,4194304] 0 2024-04-27T04:10:02.899 INFO:tasks.workunit.client.1.smithi181.stdout:6/427: symlink d7/d31/l93 0 2024-04-27T04:10:02.901 INFO:tasks.workunit.client.0.smithi162.stdout:0/364: symlink d5/dc/d1e/d20/l7c 0 2024-04-27T04:10:02.902 INFO:tasks.workunit.client.1.smithi181.stdout:7/507: rmdir d1/d22/d15/d13/d1b 39 2024-04-27T04:10:02.905 INFO:tasks.workunit.client.0.smithi162.stdout:3/306: dwrite d1/d18/f1e [4194304,4194304] 0 2024-04-27T04:10:02.905 INFO:tasks.workunit.client.0.smithi162.stdout:8/343: unlink d0/d10/d2b/d2d/d2f/d39/c4a 0 2024-04-27T04:10:02.908 INFO:tasks.workunit.client.1.smithi181.stdout:9/522: rename d0/d40/l1d to d0/d44/d4b/d43/d8f/lb6 0 2024-04-27T04:10:02.910 INFO:tasks.workunit.client.0.smithi162.stdout:0/365: dwrite d5/dc/d1e/d41/f39 [0,4194304] 0 2024-04-27T04:10:02.912 INFO:tasks.workunit.client.1.smithi181.stdout:8/605: truncate d1/d6/f4e 1265713 0 2024-04-27T04:10:02.912 INFO:tasks.workunit.client.0.smithi162.stdout:0/366: dread - d5/dc/d1e/d41/f71 zero size 2024-04-27T04:10:02.913 INFO:tasks.workunit.client.1.smithi181.stdout:8/606: stat d1/d6/d1a/d25/c76 0 2024-04-27T04:10:02.913 INFO:tasks.workunit.client.1.smithi181.stdout:8/607: dread - d1/f82 zero size 2024-04-27T04:10:02.915 INFO:tasks.workunit.client.0.smithi162.stdout:8/344: dwrite d0/d10/d2b/d2d/d2f/f54 [0,4194304] 0 2024-04-27T04:10:02.915 INFO:tasks.workunit.client.1.smithi181.stdout:8/608: chown d1/d6/d1a/d25/f99 2 1 2024-04-27T04:10:02.917 INFO:tasks.workunit.client.0.smithi162.stdout:5/310: rename d8/d2c/d41/c56 to d8/d1a/d23/d40/d4b/d58/c6b 0 2024-04-27T04:10:02.924 INFO:tasks.workunit.client.1.smithi181.stdout:4/481: unlink d0/l85 0 2024-04-27T04:10:02.928 INFO:tasks.workunit.client.1.smithi181.stdout:2/573: creat d4/d9/df/d14/fb6 x:0 0 0 2024-04-27T04:10:02.930 INFO:tasks.workunit.client.1.smithi181.stdout:6/428: write d7/dc/d1c/d7f/d3e/f47 [3521089,59023] 0 2024-04-27T04:10:02.931 INFO:tasks.workunit.client.0.smithi162.stdout:3/307: mkdir d1/d18/d23/d58/d63 0 2024-04-27T04:10:02.935 INFO:tasks.workunit.client.0.smithi162.stdout:0/367: mknod d5/d26/d4a/c7d 0 2024-04-27T04:10:02.937 INFO:tasks.workunit.client.0.smithi162.stdout:6/303: sync 2024-04-27T04:10:02.937 INFO:tasks.workunit.client.0.smithi162.stdout:6/304: fdatasync d6/db/f13 0 2024-04-27T04:10:02.938 INFO:tasks.workunit.client.0.smithi162.stdout:5/311: mknod d8/d9/d55/c6c 0 2024-04-27T04:10:02.939 INFO:tasks.workunit.client.0.smithi162.stdout:2/251: getdents d0/d20/d2e 0 2024-04-27T04:10:02.939 INFO:tasks.workunit.client.1.smithi181.stdout:8/609: dwrite d1/d6/d2f/d30/d66/fab [0,4194304] 0 2024-04-27T04:10:02.941 INFO:tasks.workunit.client.1.smithi181.stdout:6/429: symlink d7/d31/d45/d62/l94 0 2024-04-27T04:10:02.943 INFO:tasks.workunit.client.0.smithi162.stdout:4/348: symlink d2/d54/l73 0 2024-04-27T04:10:02.952 INFO:tasks.workunit.client.1.smithi181.stdout:7/508: dread d1/d2/d1c/d37/f3f [0,4194304] 0 2024-04-27T04:10:02.964 INFO:tasks.workunit.client.0.smithi162.stdout:9/301: sync 2024-04-27T04:10:02.964 INFO:tasks.workunit.client.0.smithi162.stdout:1/288: sync 2024-04-27T04:10:02.964 INFO:tasks.workunit.client.0.smithi162.stdout:1/289: fsync d0/d11/f1c 0 2024-04-27T04:10:02.967 INFO:tasks.workunit.client.0.smithi162.stdout:9/302: dread d3/d11/f24 [0,4194304] 0 2024-04-27T04:10:02.977 INFO:tasks.workunit.client.1.smithi181.stdout:0/631: sync 2024-04-27T04:10:02.978 INFO:tasks.workunit.client.0.smithi162.stdout:0/368: unlink d5/d26/c44 0 2024-04-27T04:10:02.979 INFO:tasks.workunit.client.1.smithi181.stdout:0/632: fsync dc/d1f/d3b/d4a/d95/d26/d41/d6f/f7f 0 2024-04-27T04:10:02.981 INFO:tasks.workunit.client.1.smithi181.stdout:2/574: mkdir d4/d9/d16/d82/d7d/db7 0 2024-04-27T04:10:02.983 INFO:tasks.workunit.client.0.smithi162.stdout:7/261: truncate d3/d5/d9/d4b/f27 558519 0 2024-04-27T04:10:02.986 INFO:tasks.workunit.client.1.smithi181.stdout:9/523: link d0/caa d0/d44/d4f/cb7 0 2024-04-27T04:10:02.986 INFO:tasks.workunit.client.0.smithi162.stdout:2/252: mkdir d0/d4/d17/d4f 0 2024-04-27T04:10:02.987 INFO:tasks.workunit.client.0.smithi162.stdout:4/349: stat d2/da/f44 0 2024-04-27T04:10:02.989 INFO:tasks.workunit.client.1.smithi181.stdout:3/468: dwrite d0/f4e [0,4194304] 0 2024-04-27T04:10:02.990 INFO:tasks.workunit.client.1.smithi181.stdout:0/633: write dc/d1f/d3b/d75/f8c [4271373,121011] 0 2024-04-27T04:10:02.991 INFO:tasks.workunit.client.1.smithi181.stdout:4/482: link d0/d15/f23 d0/d15/d25/d2a/d61/f9d 0 2024-04-27T04:10:02.992 INFO:tasks.workunit.client.1.smithi181.stdout:4/483: chown d0/d15/d25/d2b/f5f 5475341 1 2024-04-27T04:10:02.994 INFO:tasks.workunit.client.0.smithi162.stdout:8/345: creat d0/d10/d2b/d2d/f75 x:0 0 0 2024-04-27T04:10:03.003 INFO:tasks.workunit.client.0.smithi162.stdout:2/253: fdatasync d0/d4/f16 0 2024-04-27T04:10:03.004 INFO:tasks.workunit.client.0.smithi162.stdout:2/254: dread - d0/d4/f42 zero size 2024-04-27T04:10:03.005 INFO:tasks.workunit.client.0.smithi162.stdout:2/255: write d0/f44 [1029085,58184] 0 2024-04-27T04:10:03.007 INFO:tasks.workunit.client.1.smithi181.stdout:3/469: creat d0/d6c/d52/d44/d56/d5b/d77/f95 x:0 0 0 2024-04-27T04:10:03.011 INFO:tasks.workunit.client.1.smithi181.stdout:0/634: mknod dc/d1f/d3b/d4a/d7a/cdf 0 2024-04-27T04:10:03.011 INFO:tasks.workunit.client.0.smithi162.stdout:4/350: dwrite d2/d25/fb [0,4194304] 0 2024-04-27T04:10:03.013 INFO:tasks.workunit.client.0.smithi162.stdout:0/369: write d5/f11 [748874,51497] 0 2024-04-27T04:10:03.013 INFO:tasks.workunit.client.1.smithi181.stdout:2/575: write d4/d9/df/f41 [1524811,54438] 0 2024-04-27T04:10:03.017 INFO:tasks.workunit.client.1.smithi181.stdout:0/635: dread dc/d1f/d3b/fbf [0,4194304] 0 2024-04-27T04:10:03.018 INFO:tasks.workunit.client.1.smithi181.stdout:8/610: creat d1/d6/d2f/fd3 x:0 0 0 2024-04-27T04:10:03.020 INFO:tasks.workunit.client.1.smithi181.stdout:3/470: dread d0/de/d20/f46 [0,4194304] 0 2024-04-27T04:10:03.022 INFO:tasks.workunit.client.0.smithi162.stdout:6/305: rename d6/db/f22 to d6/f62 0 2024-04-27T04:10:03.023 INFO:tasks.workunit.client.0.smithi162.stdout:6/306: readlink d6/d1a/d34/l3c 0 2024-04-27T04:10:03.023 INFO:tasks.workunit.client.1.smithi181.stdout:9/524: truncate d0/f10 1034482 0 2024-04-27T04:10:03.024 INFO:tasks.workunit.client.0.smithi162.stdout:5/312: link d8/d9/f48 d8/d9/d55/f6d 0 2024-04-27T04:10:03.025 INFO:tasks.workunit.client.1.smithi181.stdout:1/536: sync 2024-04-27T04:10:03.025 INFO:tasks.workunit.client.1.smithi181.stdout:9/525: read d0/d44/d4b/f62 [1696352,10212] 0 2024-04-27T04:10:03.026 INFO:tasks.workunit.client.0.smithi162.stdout:3/308: getdents d1/d18/d23/d58 0 2024-04-27T04:10:03.026 INFO:tasks.workunit.client.1.smithi181.stdout:1/537: write d1/d2/d5/d6/db/f40 [3806855,100961] 0 2024-04-27T04:10:03.028 INFO:tasks.workunit.client.1.smithi181.stdout:4/484: symlink d0/d15/d25/d2a/d7a/l9e 0 2024-04-27T04:10:03.028 INFO:tasks.workunit.client.0.smithi162.stdout:5/313: read d8/f12 [11054657,51463] 0 2024-04-27T04:10:03.029 INFO:tasks.workunit.client.0.smithi162.stdout:1/290: creat d0/d7/d3c/d54/f60 x:0 0 0 2024-04-27T04:10:03.030 INFO:tasks.workunit.client.0.smithi162.stdout:9/303: dwrite d3/fa [4194304,4194304] 0 2024-04-27T04:10:03.109 INFO:tasks.workunit.client.1.smithi181.stdout:0/636: creat dc/d93/fe0 x:0 0 0 2024-04-27T04:10:03.109 INFO:tasks.workunit.client.1.smithi181.stdout:0/637: fsync dc/d1f/d3b/f44 0 2024-04-27T04:10:03.112 INFO:tasks.workunit.client.0.smithi162.stdout:0/370: creat d5/dc/d1e/d41/f7e x:0 0 0 2024-04-27T04:10:03.113 INFO:tasks.workunit.client.1.smithi181.stdout:6/430: getdents d7/dc/d1c/d88/d6d 0 2024-04-27T04:10:03.114 INFO:tasks.workunit.client.1.smithi181.stdout:6/431: chown d7/dc/c40 25655876 1 2024-04-27T04:10:03.114 INFO:tasks.workunit.client.0.smithi162.stdout:8/346: mkdir d0/d10/d76 0 2024-04-27T04:10:03.117 INFO:tasks.workunit.client.1.smithi181.stdout:2/576: write d4/d9/d4f/d79/fa3 [597044,95171] 0 2024-04-27T04:10:03.118 INFO:tasks.workunit.client.1.smithi181.stdout:2/577: chown d4/d9/d16/d45/d9a/d9d 8038 1 2024-04-27T04:10:03.121 INFO:tasks.workunit.client.0.smithi162.stdout:2/256: rename d0/d4/d2a/l2d to d0/d4/d2a/d3b/d49/l50 0 2024-04-27T04:10:03.123 INFO:tasks.workunit.client.1.smithi181.stdout:7/509: rmdir d1/d22/d15/d13/d1b/d89 0 2024-04-27T04:10:03.124 INFO:tasks.workunit.client.0.smithi162.stdout:6/307: creat d6/d1a/d34/d50/f63 x:0 0 0 2024-04-27T04:10:03.125 INFO:tasks.workunit.client.0.smithi162.stdout:6/308: dread - d6/db/d32/f38 zero size 2024-04-27T04:10:03.127 INFO:tasks.workunit.client.1.smithi181.stdout:4/485: creat d0/d15/de/d19/d77/d57/f9f x:0 0 0 2024-04-27T04:10:03.128 INFO:tasks.workunit.client.0.smithi162.stdout:3/309: creat d1/d21/d47/d57/f64 x:0 0 0 2024-04-27T04:10:03.128 INFO:tasks.workunit.client.1.smithi181.stdout:3/471: dwrite d0/f3c [4194304,4194304] 0 2024-04-27T04:10:03.129 INFO:tasks.workunit.client.1.smithi181.stdout:3/472: stat d0/de/d20/c2b 0 2024-04-27T04:10:03.130 INFO:tasks.workunit.client.0.smithi162.stdout:6/309: dwrite d6/db/f48 [0,4194304] 0 2024-04-27T04:10:03.133 INFO:tasks.workunit.client.0.smithi162.stdout:5/314: creat d8/d1a/d23/d40/f6e x:0 0 0 2024-04-27T04:10:03.134 INFO:tasks.workunit.client.1.smithi181.stdout:3/473: read d0/de/f1f [1095601,58565] 0 2024-04-27T04:10:03.134 INFO:tasks.workunit.client.0.smithi162.stdout:1/291: mkdir d0/d7/d3c/d61 0 2024-04-27T04:10:03.135 INFO:tasks.workunit.client.1.smithi181.stdout:4/486: dread d0/d15/d25/d2b/f6 [0,4194304] 0 2024-04-27T04:10:03.137 INFO:tasks.workunit.client.1.smithi181.stdout:6/432: creat d7/dc/d1c/d88/f95 x:0 0 0 2024-04-27T04:10:03.137 INFO:tasks.workunit.client.0.smithi162.stdout:5/315: dread d8/d1a/f45 [0,4194304] 0 2024-04-27T04:10:03.144 INFO:tasks.workunit.client.1.smithi181.stdout:2/578: rename d4/d9/d16/d82/l8d to d4/d9/d35/d42/d86/lb8 0 2024-04-27T04:10:03.145 INFO:tasks.workunit.client.1.smithi181.stdout:2/579: readlink d4/d9/d16/la6 0 2024-04-27T04:10:03.146 INFO:tasks.workunit.client.0.smithi162.stdout:0/371: mknod d5/dc/d1e/d46/c7f 0 2024-04-27T04:10:03.147 INFO:tasks.workunit.client.0.smithi162.stdout:9/304: dwrite d3/f14 [8388608,4194304] 0 2024-04-27T04:10:03.147 INFO:tasks.workunit.client.0.smithi162.stdout:5/316: dread d8/d1a/d23/f39 [0,4194304] 0 2024-04-27T04:10:03.149 INFO:tasks.workunit.client.0.smithi162.stdout:7/262: getdents d3/d5/d3c/d4d 0 2024-04-27T04:10:03.150 INFO:tasks.workunit.client.0.smithi162.stdout:7/263: write d3/d5/d9/d4b/de/d2b/f56 [784862,19019] 0 2024-04-27T04:10:03.151 INFO:tasks.workunit.client.1.smithi181.stdout:8/611: write d1/d6/f4e [1676174,67846] 0 2024-04-27T04:10:03.151 INFO:tasks.workunit.client.0.smithi162.stdout:7/264: chown d3/d5/d9/d4b/de 572 1 2024-04-27T04:10:03.156 INFO:tasks.workunit.client.0.smithi162.stdout:2/257: dwrite d0/f6 [0,4194304] 0 2024-04-27T04:10:03.159 INFO:tasks.workunit.client.1.smithi181.stdout:4/487: mkdir d0/d15/d25/d2a/da0 0 2024-04-27T04:10:03.159 INFO:tasks.workunit.client.1.smithi181.stdout:0/638: dwrite dc/f3a [0,4194304] 0 2024-04-27T04:10:03.162 INFO:tasks.workunit.client.0.smithi162.stdout:4/351: dwrite d2/d25/f36 [0,4194304] 0 2024-04-27T04:10:03.163 INFO:tasks.workunit.client.1.smithi181.stdout:7/510: rename d1/d2/d35/l39 to d1/d22/l9e 0 2024-04-27T04:10:03.165 INFO:tasks.workunit.client.1.smithi181.stdout:5/648: sync 2024-04-27T04:10:03.166 INFO:tasks.workunit.client.1.smithi181.stdout:2/580: creat d4/d9/df/d14/da7/fb9 x:0 0 0 2024-04-27T04:10:03.167 INFO:tasks.workunit.client.1.smithi181.stdout:2/581: fdatasync d4/d9/d44/f97 0 2024-04-27T04:10:03.169 INFO:tasks.workunit.client.1.smithi181.stdout:1/538: creat d1/fa8 x:0 0 0 2024-04-27T04:10:03.169 INFO:tasks.workunit.client.1.smithi181.stdout:3/474: dread d0/de/d20/f69 [0,4194304] 0 2024-04-27T04:10:03.176 INFO:tasks.workunit.client.1.smithi181.stdout:3/475: fdatasync d0/f23 0 2024-04-27T04:10:03.180 INFO:tasks.workunit.client.1.smithi181.stdout:4/488: mkdir d0/d15/d5c/da1 0 2024-04-27T04:10:03.182 INFO:tasks.workunit.client.0.smithi162.stdout:5/317: creat d8/f6f x:0 0 0 2024-04-27T04:10:03.185 INFO:tasks.workunit.client.0.smithi162.stdout:9/305: mkdir d3/d13/d2d/d3b/d69 0 2024-04-27T04:10:03.186 INFO:tasks.workunit.client.1.smithi181.stdout:6/433: rename d7/d31/d45/d62/l94 to d7/d31/d45/l96 0 2024-04-27T04:10:03.189 INFO:tasks.workunit.client.1.smithi181.stdout:5/649: dread d4/de/d2c/d46/d99/f9a [0,4194304] 0 2024-04-27T04:10:03.189 INFO:tasks.workunit.client.0.smithi162.stdout:5/318: dwrite d8/d1a/d23/d40/f6e [0,4194304] 0 2024-04-27T04:10:03.189 INFO:tasks.workunit.client.1.smithi181.stdout:7/511: creat d1/d2/f9f x:0 0 0 2024-04-27T04:10:03.190 INFO:tasks.workunit.client.0.smithi162.stdout:3/310: mkdir d1/d65 0 2024-04-27T04:10:03.191 INFO:tasks.workunit.client.0.smithi162.stdout:6/310: symlink d6/d1a/d34/l64 0 2024-04-27T04:10:03.193 INFO:tasks.workunit.client.1.smithi181.stdout:6/434: dwrite d7/d31/d45/f4b [0,4194304] 0 2024-04-27T04:10:03.194 INFO:tasks.workunit.client.0.smithi162.stdout:1/292: creat d0/d7/d5e/f62 x:0 0 0 2024-04-27T04:10:03.199 INFO:tasks.workunit.client.0.smithi162.stdout:9/306: mknod d3/d13/d2d/d3b/c6a 0 2024-04-27T04:10:03.200 INFO:tasks.workunit.client.1.smithi181.stdout:4/489: dread d0/f30 [0,4194304] 0 2024-04-27T04:10:03.202 INFO:tasks.workunit.client.0.smithi162.stdout:5/319: unlink d8/d1a/d29/f60 0 2024-04-27T04:10:03.203 INFO:tasks.workunit.client.0.smithi162.stdout:3/311: fsync d1/d18/d5d/f62 0 2024-04-27T04:10:03.204 INFO:tasks.workunit.client.0.smithi162.stdout:6/311: symlink d6/d1a/d34/d46/l65 0 2024-04-27T04:10:03.208 INFO:tasks.workunit.client.0.smithi162.stdout:3/312: dwrite d1/d18/d23/d2f/f42 [0,4194304] 0 2024-04-27T04:10:03.209 INFO:tasks.workunit.client.0.smithi162.stdout:3/313: chown d1/d18/d23/f28 2999 1 2024-04-27T04:10:03.210 INFO:tasks.workunit.client.0.smithi162.stdout:7/265: creat d3/d5/d9/f59 x:0 0 0 2024-04-27T04:10:03.213 INFO:tasks.workunit.client.0.smithi162.stdout:5/320: symlink d8/d1a/d29/d4a/d5c/l70 0 2024-04-27T04:10:03.214 INFO:tasks.workunit.client.0.smithi162.stdout:0/372: sync 2024-04-27T04:10:03.215 INFO:tasks.workunit.client.0.smithi162.stdout:4/352: link d2/d25/d1c/d24/l63 d2/d25/d1c/d39/d3b/d6e/l74 0 2024-04-27T04:10:03.217 INFO:tasks.workunit.client.0.smithi162.stdout:4/353: dread d2/d25/d1c/d24/f6d [0,4194304] 0 2024-04-27T04:10:03.219 INFO:tasks.workunit.client.0.smithi162.stdout:0/373: dwrite d5/d26/d4a/f62 [0,4194304] 0 2024-04-27T04:10:03.220 INFO:tasks.workunit.client.0.smithi162.stdout:0/374: chown d5/f11 3730 1 2024-04-27T04:10:03.230 INFO:tasks.workunit.client.0.smithi162.stdout:3/314: mkdir d1/d18/d5d/d5f/d66 0 2024-04-27T04:10:03.239 INFO:tasks.workunit.client.0.smithi162.stdout:7/266: dread - d3/f1e zero size 2024-04-27T04:10:03.240 INFO:tasks.workunit.client.0.smithi162.stdout:7/267: write d3/d5/d9/d4b/f33 [2331112,115320] 0 2024-04-27T04:10:03.241 INFO:tasks.workunit.client.0.smithi162.stdout:7/268: write d3/d5/d9/d4b/f33 [587516,84121] 0 2024-04-27T04:10:03.244 INFO:tasks.workunit.client.0.smithi162.stdout:1/293: dread d0/d7/d10/f12 [0,4194304] 0 2024-04-27T04:10:03.246 INFO:tasks.workunit.client.0.smithi162.stdout:1/294: fsync d0/d7/d2a/f31 0 2024-04-27T04:10:03.249 INFO:tasks.workunit.client.0.smithi162.stdout:7/269: dwrite d3/d5/d9/d4b/f2f [0,4194304] 0 2024-04-27T04:10:03.251 INFO:tasks.workunit.client.0.smithi162.stdout:2/258: dread d0/d4/d2a/f36 [0,4194304] 0 2024-04-27T04:10:03.320 INFO:tasks.workunit.client.0.smithi162.stdout:8/347: dwrite d0/d10/d25/f36 [0,4194304] 0 2024-04-27T04:10:03.320 INFO:tasks.workunit.client.0.smithi162.stdout:6/312: creat d6/d1a/d34/f66 x:0 0 0 2024-04-27T04:10:03.323 INFO:tasks.workunit.client.0.smithi162.stdout:6/313: chown d6/db/f49 1270 1 2024-04-27T04:10:03.324 INFO:tasks.workunit.client.0.smithi162.stdout:3/315: dwrite d1/d18/f10 [4194304,4194304] 0 2024-04-27T04:10:03.330 INFO:tasks.workunit.client.0.smithi162.stdout:6/314: dread d6/f35 [0,4194304] 0 2024-04-27T04:10:03.336 INFO:tasks.workunit.client.0.smithi162.stdout:7/270: fdatasync d3/d39/d57/f42 0 2024-04-27T04:10:03.337 INFO:tasks.workunit.client.0.smithi162.stdout:7/271: chown d3/d39/d57/c3d 10646 1 2024-04-27T04:10:03.341 INFO:tasks.workunit.client.1.smithi181.stdout:5/650: mkdir d4/da9/de7 0 2024-04-27T04:10:03.345 INFO:tasks.workunit.client.0.smithi162.stdout:0/375: mknod d5/dc/d1e/d41/d79/c80 0 2024-04-27T04:10:03.347 INFO:tasks.workunit.client.0.smithi162.stdout:8/348: creat d0/de/d6e/f77 x:0 0 0 2024-04-27T04:10:03.352 INFO:tasks.workunit.client.0.smithi162.stdout:0/376: dwrite d5/f27 [0,4194304] 0 2024-04-27T04:10:03.352 INFO:tasks.workunit.client.1.smithi181.stdout:5/651: creat d4/de/d2c/d2e/d8f/fe8 x:0 0 0 2024-04-27T04:10:03.353 INFO:tasks.workunit.client.0.smithi162.stdout:9/307: dwrite d3/f5 [0,4194304] 0 2024-04-27T04:10:03.353 INFO:tasks.workunit.client.0.smithi162.stdout:0/377: dread - d5/d26/f4d zero size 2024-04-27T04:10:03.353 INFO:tasks.workunit.client.1.smithi181.stdout:7/512: getdents d1/d22/d15/d5f/d81 0 2024-04-27T04:10:03.354 INFO:tasks.workunit.client.0.smithi162.stdout:9/308: dread - d3/d11/d45/d4e/d58/f68 zero size 2024-04-27T04:10:03.355 INFO:tasks.workunit.client.0.smithi162.stdout:6/315: mkdir d6/d1a/d34/d46/d67 0 2024-04-27T04:10:03.356 INFO:tasks.workunit.client.0.smithi162.stdout:6/316: chown d6/db/d32/d5a 20 1 2024-04-27T04:10:03.358 INFO:tasks.workunit.client.1.smithi181.stdout:4/490: creat d0/d15/d25/d2b/d6d/d93/fa2 x:0 0 0 2024-04-27T04:10:03.362 INFO:tasks.workunit.client.1.smithi181.stdout:5/652: unlink d4/de/d2c/d46/d4d/d8e/d67/d71/f91 0 2024-04-27T04:10:03.364 INFO:tasks.workunit.client.1.smithi181.stdout:7/513: dread - d1/d22/d15/d1f/d34/f5b zero size 2024-04-27T04:10:03.364 INFO:tasks.workunit.client.1.smithi181.stdout:5/653: write d4/de/d2c/d46/d4d/d8e/d67/fa2 [181038,103614] 0 2024-04-27T04:10:03.366 INFO:tasks.workunit.client.1.smithi181.stdout:5/654: dread - d4/de/d2c/d46/d99/dad/fe1 zero size 2024-04-27T04:10:03.367 INFO:tasks.workunit.client.0.smithi162.stdout:7/272: truncate d3/d5/d9/f13 230496 0 2024-04-27T04:10:03.367 INFO:tasks.workunit.client.0.smithi162.stdout:7/273: chown d3/d39 147 1 2024-04-27T04:10:03.375 INFO:tasks.workunit.client.0.smithi162.stdout:4/354: write d2/d25/d1c/d39/d3b/d52/f64 [2711549,79650] 0 2024-04-27T04:10:03.376 INFO:tasks.workunit.client.1.smithi181.stdout:6/435: getdents d7/dc/d1e/d2b 0 2024-04-27T04:10:03.377 INFO:tasks.workunit.client.0.smithi162.stdout:0/378: rmdir d5/dc/d1e/d46/d48 39 2024-04-27T04:10:03.378 INFO:tasks.workunit.client.0.smithi162.stdout:0/379: readlink d5/l63 0 2024-04-27T04:10:03.380 INFO:tasks.workunit.client.0.smithi162.stdout:5/321: write d8/d1a/f2d [83588,15985] 0 2024-04-27T04:10:03.384 INFO:tasks.workunit.client.1.smithi181.stdout:6/436: symlink d7/dc/d1e/d5c/l97 0 2024-04-27T04:10:03.387 INFO:tasks.workunit.client.1.smithi181.stdout:1/539: dwrite d1/d2/d5/d6/da3/d60/f88 [0,4194304] 0 2024-04-27T04:10:03.389 INFO:tasks.workunit.client.1.smithi181.stdout:1/540: fdatasync d1/d2/d5/d6/da3/d1e/d20/d53/d56/f6f 0 2024-04-27T04:10:03.389 INFO:tasks.workunit.client.0.smithi162.stdout:3/316: rename d1/d21/l30 to d1/d18/d23/l67 0 2024-04-27T04:10:03.390 INFO:tasks.workunit.client.1.smithi181.stdout:3/476: write d0/f71 [858367,14550] 0 2024-04-27T04:10:03.393 INFO:tasks.workunit.client.1.smithi181.stdout:7/514: rmdir d1/df/d86 0 2024-04-27T04:10:03.394 INFO:tasks.workunit.client.1.smithi181.stdout:0/639: truncate dc/fd 1160071 0 2024-04-27T04:10:03.395 INFO:tasks.workunit.client.0.smithi162.stdout:4/355: mknod d2/d25/d1c/d39/c75 0 2024-04-27T04:10:03.395 INFO:tasks.workunit.client.1.smithi181.stdout:1/541: truncate d1/d2/d5/d6/da3/d1e/d20/d53/fa0 213488 0 2024-04-27T04:10:03.396 INFO:tasks.workunit.client.1.smithi181.stdout:5/655: link d4/d10/d72/db3/fb6 d4/d10/d29/d92/dbb/fe9 0 2024-04-27T04:10:03.399 INFO:tasks.workunit.client.1.smithi181.stdout:3/477: dwrite d0/de/d20/f5e [0,4194304] 0 2024-04-27T04:10:03.402 INFO:tasks.workunit.client.0.smithi162.stdout:9/309: mknod d3/c6b 0 2024-04-27T04:10:03.404 INFO:tasks.workunit.client.1.smithi181.stdout:7/515: mknod d1/d2/d1c/d2d/d61/ca0 0 2024-04-27T04:10:03.405 INFO:tasks.workunit.client.1.smithi181.stdout:5/656: symlink d4/de/d2c/d46/d99/lea 0 2024-04-27T04:10:03.406 INFO:tasks.workunit.client.0.smithi162.stdout:1/295: link d0/l3 d0/d7/l63 0 2024-04-27T04:10:03.408 INFO:tasks.workunit.client.1.smithi181.stdout:1/542: rename d1/d2/d5/d6/da3/d52/d92/c5a to d1/d2/d5/d6/db/ca9 0 2024-04-27T04:10:03.413 INFO:tasks.workunit.client.1.smithi181.stdout:4/491: rmdir d0 39 2024-04-27T04:10:03.413 INFO:tasks.workunit.client.1.smithi181.stdout:5/657: dwrite d4/de/d2c/d46/d4d/fb8 [0,4194304] 0 2024-04-27T04:10:03.415 INFO:tasks.workunit.client.0.smithi162.stdout:2/259: truncate d0/d4/d17/f3c 1385156 0 2024-04-27T04:10:03.415 INFO:tasks.workunit.client.1.smithi181.stdout:1/543: read d1/d2/d5/d6/da3/d52/f68 [284352,35955] 0 2024-04-27T04:10:03.415 INFO:tasks.workunit.client.0.smithi162.stdout:2/260: fdatasync d0/f6 0 2024-04-27T04:10:03.416 INFO:tasks.workunit.client.1.smithi181.stdout:3/478: truncate d0/de/f4b 1509958 0 2024-04-27T04:10:03.416 INFO:tasks.workunit.client.1.smithi181.stdout:3/479: fdatasync d0/de/f17 0 2024-04-27T04:10:03.422 INFO:tasks.workunit.client.0.smithi162.stdout:6/317: rename d6/db/d32/d5a to d6/d1a/d34/d3a/d68 0 2024-04-27T04:10:03.423 INFO:tasks.workunit.client.0.smithi162.stdout:6/318: write d6/f25 [3790514,101865] 0 2024-04-27T04:10:03.430 INFO:tasks.workunit.client.0.smithi162.stdout:6/319: dwrite d6/d1a/d34/d46/f53 [0,4194304] 0 2024-04-27T04:10:03.434 INFO:tasks.workunit.client.0.smithi162.stdout:8/349: dwrite d0/d10/f6a [0,4194304] 0 2024-04-27T04:10:03.441 INFO:tasks.workunit.client.0.smithi162.stdout:5/322: creat d8/d9/d5b/f71 x:0 0 0 2024-04-27T04:10:03.442 INFO:tasks.workunit.client.1.smithi181.stdout:3/480: mknod d0/de/d7d/c96 0 2024-04-27T04:10:03.444 INFO:tasks.workunit.client.0.smithi162.stdout:1/296: creat d0/d7/d10/f64 x:0 0 0 2024-04-27T04:10:03.447 INFO:tasks.workunit.client.0.smithi162.stdout:7/274: truncate d3/d4f/f4c 1484434 0 2024-04-27T04:10:03.448 INFO:tasks.workunit.client.0.smithi162.stdout:7/275: stat d3/d39/d57/f43 0 2024-04-27T04:10:03.448 INFO:tasks.workunit.client.0.smithi162.stdout:7/276: read - d3/f1e zero size 2024-04-27T04:10:03.450 INFO:tasks.workunit.client.1.smithi181.stdout:0/640: link dc/d1f/d3b/d4a/d95/d26/d41/l55 dc/d1f/d3b/d4a/d95/d26/d41/d6f/d8f/le1 0 2024-04-27T04:10:03.452 INFO:tasks.workunit.client.0.smithi162.stdout:3/317: dread d1/d18/d23/f53 [0,4194304] 0 2024-04-27T04:10:03.453 INFO:tasks.workunit.client.0.smithi162.stdout:3/318: fdatasync d1/d21/d47/d57/f64 0 2024-04-27T04:10:03.453 INFO:tasks.workunit.client.1.smithi181.stdout:7/516: write d1/df/f18 [913494,112540] 0 2024-04-27T04:10:03.458 INFO:tasks.workunit.client.1.smithi181.stdout:3/481: mknod d0/de/d20/d58/d70/c97 0 2024-04-27T04:10:03.459 INFO:tasks.workunit.client.0.smithi162.stdout:0/380: truncate d5/f27 2143850 0 2024-04-27T04:10:03.459 INFO:tasks.workunit.client.1.smithi181.stdout:3/482: truncate d0/de/f7f 4373673 0 2024-04-27T04:10:03.464 INFO:tasks.workunit.client.1.smithi181.stdout:1/544: write d1/d2/d5/d6/db/d2d/d36/f4a [1032844,30957] 0 2024-04-27T04:10:03.467 INFO:tasks.workunit.client.0.smithi162.stdout:8/350: readlink d0/de/l5d 0 2024-04-27T04:10:03.469 INFO:tasks.workunit.client.0.smithi162.stdout:9/310: creat d3/d13/d2d/d3b/d54/f6c x:0 0 0 2024-04-27T04:10:03.472 INFO:tasks.workunit.client.1.smithi181.stdout:5/658: creat d4/d10/feb x:0 0 0 2024-04-27T04:10:03.472 INFO:tasks.workunit.client.1.smithi181.stdout:7/517: dwrite d1/d2/f19 [0,4194304] 0 2024-04-27T04:10:03.474 INFO:tasks.workunit.client.0.smithi162.stdout:5/323: mknod d8/d9/d5a/c72 0 2024-04-27T04:10:03.476 INFO:tasks.workunit.client.0.smithi162.stdout:1/297: fdatasync d0/d11/f20 0 2024-04-27T04:10:03.478 INFO:tasks.workunit.client.0.smithi162.stdout:7/277: rmdir d3/d39 39 2024-04-27T04:10:03.480 INFO:tasks.workunit.client.0.smithi162.stdout:5/324: dwrite d8/d1a/f51 [0,4194304] 0 2024-04-27T04:10:03.482 INFO:tasks.workunit.client.0.smithi162.stdout:5/325: fdatasync d8/d1a/d29/f4d 0 2024-04-27T04:10:03.484 INFO:tasks.workunit.client.1.smithi181.stdout:9/526: sync 2024-04-27T04:10:03.485 INFO:tasks.workunit.client.0.smithi162.stdout:7/278: dwrite d3/d5/d9/d4b/de/d2b/f2c [0,4194304] 0 2024-04-27T04:10:03.506 INFO:tasks.workunit.client.1.smithi181.stdout:1/545: stat d1/d2/d5/d6/da3/d1e/l71 0 2024-04-27T04:10:03.519 INFO:tasks.workunit.client.0.smithi162.stdout:6/320: dwrite d6/d1a/d34/f3e [0,4194304] 0 2024-04-27T04:10:03.520 INFO:tasks.workunit.client.1.smithi181.stdout:0/641: dwrite dc/d1f/f3f [0,4194304] 0 2024-04-27T04:10:03.520 INFO:tasks.workunit.client.1.smithi181.stdout:3/483: mkdir d0/d6c/d83/d98 0 2024-04-27T04:10:03.523 INFO:tasks.workunit.client.1.smithi181.stdout:9/527: rename d0/d40/f29 to d0/d44/d4b/d43/d88/fb8 0 2024-04-27T04:10:03.527 INFO:tasks.workunit.client.1.smithi181.stdout:1/546: creat d1/d2/d64/faa x:0 0 0 2024-04-27T04:10:03.528 INFO:tasks.workunit.client.1.smithi181.stdout:4/492: link d0/d15/de/d19/l3c d0/d15/d25/d40/la3 0 2024-04-27T04:10:03.530 INFO:tasks.workunit.client.1.smithi181.stdout:5/659: mkdir d4/d10/dec 0 2024-04-27T04:10:03.533 INFO:tasks.workunit.client.0.smithi162.stdout:9/311: dwrite d3/fb [0,4194304] 0 2024-04-27T04:10:03.533 INFO:tasks.workunit.client.0.smithi162.stdout:1/298: creat d0/d7/d3c/d54/f65 x:0 0 0 2024-04-27T04:10:03.534 INFO:tasks.workunit.client.0.smithi162.stdout:1/299: chown d0/d7/f19 342865110 1 2024-04-27T04:10:03.537 INFO:tasks.workunit.client.1.smithi181.stdout:5/660: dwrite d4/da9/fc2 [0,4194304] 0 2024-04-27T04:10:03.540 INFO:tasks.workunit.client.1.smithi181.stdout:7/518: write d1/d2/f27 [736499,121080] 0 2024-04-27T04:10:03.541 INFO:tasks.workunit.client.1.smithi181.stdout:9/528: symlink d0/d44/d4b/d43/d88/da1/lb9 0 2024-04-27T04:10:03.541 INFO:tasks.workunit.client.0.smithi162.stdout:1/300: dwrite d0/f34 [0,4194304] 0 2024-04-27T04:10:03.545 INFO:tasks.workunit.client.1.smithi181.stdout:1/547: fsync d1/d2/d5/d6/da3/d1e/f49 0 2024-04-27T04:10:03.547 INFO:tasks.workunit.client.0.smithi162.stdout:2/261: fdatasync d0/d4/d17/f3c 0 2024-04-27T04:10:03.556 INFO:tasks.workunit.client.0.smithi162.stdout:0/381: truncate d5/f2d 5152616 0 2024-04-27T04:10:03.559 INFO:tasks.workunit.client.1.smithi181.stdout:7/519: rmdir d1 39 2024-04-27T04:10:03.561 INFO:tasks.workunit.client.1.smithi181.stdout:9/529: rmdir d0/d44/d4b/d3d 39 2024-04-27T04:10:03.562 INFO:tasks.workunit.client.0.smithi162.stdout:4/356: getdents d2/d25/d1c/d24 0 2024-04-27T04:10:03.563 INFO:tasks.workunit.client.0.smithi162.stdout:6/321: symlink d6/d1a/d34/d43/l69 0 2024-04-27T04:10:03.565 INFO:tasks.workunit.client.0.smithi162.stdout:8/351: symlink d0/d10/d2b/d2d/l78 0 2024-04-27T04:10:03.565 INFO:tasks.workunit.client.1.smithi181.stdout:0/642: link dc/d1f/d3b/ld6 dc/d1f/le2 0 2024-04-27T04:10:03.566 INFO:tasks.workunit.client.1.smithi181.stdout:0/643: chown dc/d1f/f29 9695 1 2024-04-27T04:10:03.566 INFO:tasks.workunit.client.1.smithi181.stdout:0/644: chown dc/d54/fab 0 1 2024-04-27T04:10:03.568 INFO:tasks.workunit.client.1.smithi181.stdout:3/484: truncate d0/f3c 5281324 0 2024-04-27T04:10:03.569 INFO:tasks.workunit.client.1.smithi181.stdout:0/645: truncate dc/d1f/d6b/db9/dbd/fbe 310316 0 2024-04-27T04:10:03.569 INFO:tasks.workunit.client.0.smithi162.stdout:9/312: write d3/f14 [8256261,2040] 0 2024-04-27T04:10:03.571 INFO:tasks.workunit.client.0.smithi162.stdout:8/352: dread d0/d10/d25/f30 [0,4194304] 0 2024-04-27T04:10:03.571 INFO:tasks.workunit.client.1.smithi181.stdout:7/520: read d1/d2/d1c/d37/f4f [1102702,120968] 0 2024-04-27T04:10:03.576 INFO:tasks.workunit.client.0.smithi162.stdout:3/319: dwrite d1/d18/f1d [0,4194304] 0 2024-04-27T04:10:03.577 INFO:tasks.workunit.client.1.smithi181.stdout:1/548: creat d1/d2/d89/fab x:0 0 0 2024-04-27T04:10:03.577 INFO:tasks.workunit.client.1.smithi181.stdout:7/521: dwrite d1/df/f18 [0,4194304] 0 2024-04-27T04:10:03.577 INFO:tasks.workunit.client.0.smithi162.stdout:3/320: stat d1/c56 0 2024-04-27T04:10:03.578 INFO:tasks.workunit.client.0.smithi162.stdout:7/279: mknod d3/d5/d3c/d4a/d55/c5a 0 2024-04-27T04:10:03.578 INFO:tasks.workunit.client.0.smithi162.stdout:3/321: chown d1/d21/d47/d57/f5e 7 1 2024-04-27T04:10:03.579 INFO:tasks.workunit.client.0.smithi162.stdout:3/322: fdatasync d1/d18/f52 0 2024-04-27T04:10:03.580 INFO:tasks.workunit.client.0.smithi162.stdout:1/301: mkdir d0/d11/d44/d66 0 2024-04-27T04:10:03.580 INFO:tasks.workunit.client.1.smithi181.stdout:3/485: read d0/de/f34 [1212376,116195] 0 2024-04-27T04:10:03.581 INFO:tasks.workunit.client.0.smithi162.stdout:3/323: fdatasync d1/d18/f41 0 2024-04-27T04:10:03.582 INFO:tasks.workunit.client.0.smithi162.stdout:3/324: write d1/d21/f2c [3631686,1616] 0 2024-04-27T04:10:03.585 INFO:tasks.workunit.client.0.smithi162.stdout:7/280: dwrite d3/d5/d9/f21 [0,4194304] 0 2024-04-27T04:10:03.590 INFO:tasks.workunit.client.1.smithi181.stdout:7/522: dread d1/d22/d15/f2a [0,4194304] 0 2024-04-27T04:10:03.592 INFO:tasks.workunit.client.0.smithi162.stdout:0/382: symlink d5/dc/d1e/d56/d57/l81 0 2024-04-27T04:10:03.593 INFO:tasks.workunit.client.0.smithi162.stdout:3/325: dwrite d1/d21/d47/d57/f5e [0,4194304] 0 2024-04-27T04:10:03.596 INFO:tasks.workunit.client.1.smithi181.stdout:1/549: fdatasync d1/d2/d5/f78 0 2024-04-27T04:10:03.602 INFO:tasks.workunit.client.0.smithi162.stdout:8/353: creat d0/de/d6e/f79 x:0 0 0 2024-04-27T04:10:03.602 INFO:tasks.workunit.client.1.smithi181.stdout:0/646: mknod dc/d1f/d3b/d4a/d95/d26/d41/db0/ce3 0 2024-04-27T04:10:03.603 INFO:tasks.workunit.client.0.smithi162.stdout:5/326: link d8/d1a/d23/f28 d8/d1a/d29/d4a/d5c/d63/f73 0 2024-04-27T04:10:03.603 INFO:tasks.workunit.client.1.smithi181.stdout:0/647: stat dc/d1f/d3b/d4a/d95/d6d 0 2024-04-27T04:10:03.603 INFO:tasks.workunit.client.1.smithi181.stdout:9/530: symlink d0/d44/d4b/d3d/d9a/lba 0 2024-04-27T04:10:03.605 INFO:tasks.workunit.client.0.smithi162.stdout:5/327: read - d8/d1a/d23/f50 zero size 2024-04-27T04:10:03.606 INFO:tasks.workunit.client.1.smithi181.stdout:0/648: write dc/d1f/d6b/db9/dbd/fbe [137384,76316] 0 2024-04-27T04:10:03.606 INFO:tasks.workunit.client.1.smithi181.stdout:9/531: write d0/d44/d4b/d43/f98 [817289,35117] 0 2024-04-27T04:10:03.607 INFO:tasks.workunit.client.0.smithi162.stdout:9/313: dwrite d3/d23/f22 [0,4194304] 0 2024-04-27T04:10:03.607 INFO:tasks.workunit.client.1.smithi181.stdout:0/649: readlink dc/d1f/d6b/l97 0 2024-04-27T04:10:03.608 INFO:tasks.workunit.client.0.smithi162.stdout:9/314: stat d3/d23/d1e/l2c 0 2024-04-27T04:10:03.609 INFO:tasks.workunit.client.0.smithi162.stdout:1/302: chown d0/lf 7 1 2024-04-27T04:10:03.610 INFO:tasks.workunit.client.0.smithi162.stdout:1/303: fdatasync d0/d7/d2a/f5d 0 2024-04-27T04:10:03.610 INFO:tasks.workunit.client.1.smithi181.stdout:5/661: link d4/l68 d4/de/d9d/led 0 2024-04-27T04:10:03.612 INFO:tasks.workunit.client.0.smithi162.stdout:1/304: write d0/d7/d2a/f2d [3391289,126134] 0 2024-04-27T04:10:03.614 INFO:tasks.workunit.client.0.smithi162.stdout:2/262: mkdir d0/d20/d51 0 2024-04-27T04:10:03.617 INFO:tasks.workunit.client.0.smithi162.stdout:9/315: dwrite d3/d23/d1e/d43/f48 [4194304,4194304] 0 2024-04-27T04:10:03.618 INFO:tasks.workunit.client.1.smithi181.stdout:7/523: creat d1/d2/d35/d8b/fa1 x:0 0 0 2024-04-27T04:10:03.622 INFO:tasks.workunit.client.0.smithi162.stdout:4/357: creat d2/d25/d1c/d39/d3b/d6e/d6f/f76 x:0 0 0 2024-04-27T04:10:03.625 INFO:tasks.workunit.client.1.smithi181.stdout:9/532: unlink d0/d4d/l4e 0 2024-04-27T04:10:03.626 INFO:tasks.workunit.client.1.smithi181.stdout:7/524: dwrite d1/d57/f77 [0,4194304] 0 2024-04-27T04:10:03.627 INFO:tasks.workunit.client.1.smithi181.stdout:7/525: stat d1/d22/c44 0 2024-04-27T04:10:03.627 INFO:tasks.workunit.client.0.smithi162.stdout:0/383: rmdir d5/dc/d37 39 2024-04-27T04:10:03.628 INFO:tasks.workunit.client.1.smithi181.stdout:7/526: fdatasync d1/d2/d1c/d2d/d61/f97 0 2024-04-27T04:10:03.628 INFO:tasks.workunit.client.0.smithi162.stdout:0/384: write d5/d60/f67 [181352,129658] 0 2024-04-27T04:10:03.630 INFO:tasks.workunit.client.1.smithi181.stdout:7/527: rename d1/d2/d1c to d1/d2/d1c/d2d/d61/d80/da2 22 2024-04-27T04:10:03.633 INFO:tasks.workunit.client.1.smithi181.stdout:1/550: mknod d1/d2/d5/cac 0 2024-04-27T04:10:03.639 INFO:tasks.workunit.client.1.smithi181.stdout:7/528: dwrite d1/d2/f9f [0,4194304] 0 2024-04-27T04:10:03.639 INFO:tasks.workunit.client.0.smithi162.stdout:1/305: dread d0/d7/d10/f26 [0,4194304] 0 2024-04-27T04:10:03.641 INFO:tasks.workunit.client.0.smithi162.stdout:6/322: dwrite d6/d1a/f18 [4194304,4194304] 0 2024-04-27T04:10:03.642 INFO:tasks.workunit.client.0.smithi162.stdout:6/323: chown d6/d1a/d34/d3a/l5f 130206381 1 2024-04-27T04:10:03.644 INFO:tasks.workunit.client.1.smithi181.stdout:9/533: mknod d0/d44/d4b/d50/cbb 0 2024-04-27T04:10:03.652 INFO:tasks.workunit.client.1.smithi181.stdout:2/582: sync 2024-04-27T04:10:03.652 INFO:tasks.workunit.client.1.smithi181.stdout:8/612: sync 2024-04-27T04:10:03.654 INFO:tasks.workunit.client.1.smithi181.stdout:7/529: mknod d1/d2/d1c/d8f/ca3 0 2024-04-27T04:10:03.655 INFO:tasks.workunit.client.1.smithi181.stdout:3/486: truncate d0/de/f7f 1439934 0 2024-04-27T04:10:03.656 INFO:tasks.workunit.client.1.smithi181.stdout:3/487: truncate d0/f14 4652286 0 2024-04-27T04:10:03.656 INFO:tasks.workunit.client.0.smithi162.stdout:2/263: mknod d0/d4/d2a/d41/c52 0 2024-04-27T04:10:03.659 INFO:tasks.workunit.client.1.smithi181.stdout:0/650: creat dc/d1f/d3b/fe4 x:0 0 0 2024-04-27T04:10:03.660 INFO:tasks.workunit.client.0.smithi162.stdout:7/281: write d3/d5/d9/f24 [471710,7244] 0 2024-04-27T04:10:03.660 INFO:tasks.workunit.client.1.smithi181.stdout:0/651: dread - dc/d1f/d3b/f77 zero size 2024-04-27T04:10:03.661 INFO:tasks.workunit.client.1.smithi181.stdout:9/534: read d0/f9 [1851028,34025] 0 2024-04-27T04:10:03.662 INFO:tasks.workunit.client.1.smithi181.stdout:1/551: mkdir d1/dad 0 2024-04-27T04:10:03.663 INFO:tasks.workunit.client.0.smithi162.stdout:4/358: symlink d2/d25/d1c/d39/l77 0 2024-04-27T04:10:03.664 INFO:tasks.workunit.client.0.smithi162.stdout:4/359: write d2/d25/d1c/d39/f46 [841886,84542] 0 2024-04-27T04:10:03.665 INFO:tasks.workunit.client.0.smithi162.stdout:4/360: read - d2/da/dd/f57 zero size 2024-04-27T04:10:03.665 INFO:tasks.workunit.client.1.smithi181.stdout:2/583: mknod d4/d9/d16/d45/d9a/cba 0 2024-04-27T04:10:03.667 INFO:tasks.workunit.client.0.smithi162.stdout:0/385: unlink d5/dc/d1e/d20/c42 0 2024-04-27T04:10:03.670 INFO:tasks.workunit.client.1.smithi181.stdout:7/530: mknod d1/d57/ca4 0 2024-04-27T04:10:03.674 INFO:tasks.workunit.client.1.smithi181.stdout:0/652: rename dc/d1f/d3b/db2/cc5 to dc/d1f/daa/dbc/ce5 0 2024-04-27T04:10:03.679 INFO:tasks.workunit.client.1.smithi181.stdout:2/584: fdatasync d4/d9/d16/f29 0 2024-04-27T04:10:03.680 INFO:tasks.workunit.client.1.smithi181.stdout:2/585: chown d4/d9/l31 3 1 2024-04-27T04:10:03.686 INFO:tasks.workunit.client.1.smithi181.stdout:7/531: creat d1/d2/d35/d8b/fa5 x:0 0 0 2024-04-27T04:10:03.687 INFO:tasks.workunit.client.1.smithi181.stdout:7/532: fdatasync d1/d2/f27 0 2024-04-27T04:10:03.688 INFO:tasks.workunit.client.1.smithi181.stdout:2/586: dread d4/d9/d44/f46 [0,4194304] 0 2024-04-27T04:10:03.688 INFO:tasks.workunit.client.1.smithi181.stdout:7/533: write d1/d2/f12 [10643037,13353] 0 2024-04-27T04:10:03.694 INFO:tasks.workunit.client.1.smithi181.stdout:0/653: dread - dc/d54/f9c zero size 2024-04-27T04:10:03.696 INFO:tasks.workunit.client.0.smithi162.stdout:3/326: creat d1/d18/d5d/d5f/d66/f68 x:0 0 0 2024-04-27T04:10:03.696 INFO:tasks.workunit.client.0.smithi162.stdout:8/354: creat d0/d10/d2b/d2d/d2f/d6b/f7a x:0 0 0 2024-04-27T04:10:03.696 INFO:tasks.workunit.client.0.smithi162.stdout:4/361: dwrite d2/d25/f36 [0,4194304] 0 2024-04-27T04:10:03.696 INFO:tasks.workunit.client.0.smithi162.stdout:5/328: write d8/d9/f48 [272400,44312] 0 2024-04-27T04:10:03.696 INFO:tasks.workunit.client.0.smithi162.stdout:6/324: creat d6/db/f6a x:0 0 0 2024-04-27T04:10:03.697 INFO:tasks.workunit.client.1.smithi181.stdout:7/534: dwrite d1/d2/d1c/d56/f63 [0,4194304] 0 2024-04-27T04:10:03.698 INFO:tasks.workunit.client.0.smithi162.stdout:9/316: rename d3/d11/f24 to d3/d13/d2d/d30/f6d 0 2024-04-27T04:10:03.699 INFO:tasks.workunit.client.0.smithi162.stdout:9/317: readlink d3/d13/l5f 0 2024-04-27T04:10:03.699 INFO:tasks.workunit.client.0.smithi162.stdout:0/386: mknod d5/dc/d1e/d20/c82 0 2024-04-27T04:10:03.699 INFO:tasks.workunit.client.0.smithi162.stdout:6/325: dread - d6/d1a/d34/d43/f5c zero size 2024-04-27T04:10:03.699 INFO:tasks.workunit.client.0.smithi162.stdout:1/306: mknod d0/d11/d3e/d59/c67 0 2024-04-27T04:10:03.702 INFO:tasks.workunit.client.1.smithi181.stdout:3/488: dwrite d0/de/f1f [0,4194304] 0 2024-04-27T04:10:03.708 INFO:tasks.workunit.client.1.smithi181.stdout:8/613: truncate d1/d20/f51 3375364 0 2024-04-27T04:10:03.717 INFO:tasks.workunit.client.1.smithi181.stdout:7/535: chown d1/d2/d1c/c49 15994 1 2024-04-27T04:10:03.719 INFO:tasks.workunit.client.0.smithi162.stdout:3/327: mkdir d1/d45/d69 0 2024-04-27T04:10:03.719 INFO:tasks.workunit.client.1.smithi181.stdout:1/552: rmdir d1/d2/d5/d95 0 2024-04-27T04:10:03.719 INFO:tasks.workunit.client.0.smithi162.stdout:1/307: chown d0/d11/d50/d53 521045684 1 2024-04-27T04:10:03.720 INFO:tasks.workunit.client.0.smithi162.stdout:7/282: mkdir d3/d5/d5b 0 2024-04-27T04:10:03.721 INFO:tasks.workunit.client.0.smithi162.stdout:3/328: readlink d1/d21/l24 0 2024-04-27T04:10:03.721 INFO:tasks.workunit.client.0.smithi162.stdout:6/326: write d6/d1a/d34/d43/f58 [640560,30547] 0 2024-04-27T04:10:03.722 INFO:tasks.workunit.client.1.smithi181.stdout:0/654: dread dc/d1f/d3b/d4a/d95/d26/d41/d6f/f69 [0,4194304] 0 2024-04-27T04:10:03.722 INFO:tasks.workunit.client.0.smithi162.stdout:3/329: chown d1/d21/d47 45180777 1 2024-04-27T04:10:03.722 INFO:tasks.workunit.client.1.smithi181.stdout:7/536: dread d1/d57/f77 [0,4194304] 0 2024-04-27T04:10:03.723 INFO:tasks.workunit.client.0.smithi162.stdout:5/329: rename d8/d1a/d23/c44 to d8/d1a/d23/c74 0 2024-04-27T04:10:03.724 INFO:tasks.workunit.client.1.smithi181.stdout:7/537: dread - d1/d2/d35/d67/f96 zero size 2024-04-27T04:10:03.724 INFO:tasks.workunit.client.1.smithi181.stdout:0/655: readlink dc/d1f/d3b/d4a/d95/d26/d68/l78 0 2024-04-27T04:10:03.725 INFO:tasks.workunit.client.1.smithi181.stdout:3/489: mkdir d0/d6c/d66/d99 0 2024-04-27T04:10:03.725 INFO:tasks.workunit.client.1.smithi181.stdout:7/538: stat d1/d22/d15/d13/d1b/f4e 0 2024-04-27T04:10:03.726 INFO:tasks.workunit.client.1.smithi181.stdout:0/656: write dc/d1f/d3b/d4a/d95/d26/d41/f4e [4271896,57226] 0 2024-04-27T04:10:03.726 INFO:tasks.workunit.client.1.smithi181.stdout:7/539: readlink d1/d2/d1c/d2d/l94 0 2024-04-27T04:10:03.728 INFO:tasks.workunit.client.0.smithi162.stdout:0/387: dwrite d5/d26/d69/f73 [0,4194304] 0 2024-04-27T04:10:03.728 INFO:tasks.workunit.client.0.smithi162.stdout:3/330: read d1/d18/f1a [10701112,87059] 0 2024-04-27T04:10:03.729 INFO:tasks.workunit.client.1.smithi181.stdout:2/587: mkdir d4/d9/d4f/d5f/dbb 0 2024-04-27T04:10:03.731 INFO:tasks.workunit.client.0.smithi162.stdout:0/388: write d5/dc/f7a [494323,81755] 0 2024-04-27T04:10:03.731 INFO:tasks.workunit.client.0.smithi162.stdout:8/355: dread d0/d10/d25/f14 [0,4194304] 0 2024-04-27T04:10:03.732 INFO:tasks.workunit.client.0.smithi162.stdout:3/331: read d1/d18/f1a [11988755,34707] 0 2024-04-27T04:10:03.733 INFO:tasks.workunit.client.0.smithi162.stdout:6/327: rmdir d6/d1a/d34 39 2024-04-27T04:10:03.734 INFO:tasks.workunit.client.0.smithi162.stdout:9/318: dwrite d3/d23/d1e/d43/f5a [0,4194304] 0 2024-04-27T04:10:03.738 INFO:tasks.workunit.client.1.smithi181.stdout:1/553: symlink d1/d2/d5/d6/da3/d1e/d20/da2/lae 0 2024-04-27T04:10:03.739 INFO:tasks.workunit.client.0.smithi162.stdout:9/319: dread d3/d23/f3e [0,4194304] 0 2024-04-27T04:10:03.744 INFO:tasks.workunit.client.0.smithi162.stdout:5/330: creat d8/d1a/d23/d40/d4b/f75 x:0 0 0 2024-04-27T04:10:03.745 INFO:tasks.workunit.client.1.smithi181.stdout:3/490: rmdir d0/d6c/d66 39 2024-04-27T04:10:03.748 INFO:tasks.workunit.client.1.smithi181.stdout:8/614: creat d1/d6/d6f/dd2/fd4 x:0 0 0 2024-04-27T04:10:03.750 INFO:tasks.workunit.client.0.smithi162.stdout:5/331: dwrite d8/d1a/f3e [0,4194304] 0 2024-04-27T04:10:03.753 INFO:tasks.workunit.client.0.smithi162.stdout:7/283: symlink d3/d5/d9/l5c 0 2024-04-27T04:10:03.755 INFO:tasks.workunit.client.1.smithi181.stdout:0/657: write dc/d1f/d3b/d4a/d95/d26/f28 [2090127,121509] 0 2024-04-27T04:10:03.755 INFO:tasks.workunit.client.0.smithi162.stdout:4/362: dread d2/ff [0,4194304] 0 2024-04-27T04:10:03.758 INFO:tasks.workunit.client.0.smithi162.stdout:3/332: fdatasync d1/d18/f1a 0 2024-04-27T04:10:03.761 INFO:tasks.workunit.client.0.smithi162.stdout:0/389: rename d5/dc/f7a to d5/dc/d1e/d41/f83 0 2024-04-27T04:10:03.764 INFO:tasks.workunit.client.0.smithi162.stdout:8/356: unlink d0/d10/d2b/d2d/f75 0 2024-04-27T04:10:03.767 INFO:tasks.workunit.client.1.smithi181.stdout:3/491: creat d0/d6c/d52/d44/d56/d5b/d77/f9a x:0 0 0 2024-04-27T04:10:03.771 INFO:tasks.workunit.client.0.smithi162.stdout:9/320: symlink d3/d13/d2d/d3b/d54/l6e 0 2024-04-27T04:10:03.773 INFO:tasks.workunit.client.1.smithi181.stdout:0/658: unlink dc/d1f/d3b/d4a/d95/d26/l31 0 2024-04-27T04:10:03.774 INFO:tasks.workunit.client.1.smithi181.stdout:0/659: truncate dc/ff 5576640 0 2024-04-27T04:10:03.774 INFO:tasks.workunit.client.1.smithi181.stdout:7/540: write d1/d2/d1c/d55/f66 [462640,127179] 0 2024-04-27T04:10:03.776 INFO:tasks.workunit.client.0.smithi162.stdout:7/284: mknod d3/d5/d3c/d4a/c5d 0 2024-04-27T04:10:03.780 INFO:tasks.workunit.client.1.smithi181.stdout:8/615: mknod d1/d6/d2f/d30/d66/dc0/dcd/cd5 0 2024-04-27T04:10:03.780 INFO:tasks.workunit.client.0.smithi162.stdout:3/333: dread d1/d18/d23/d34/f3c [0,4194304] 0 2024-04-27T04:10:03.782 INFO:tasks.workunit.client.0.smithi162.stdout:2/264: sync 2024-04-27T04:10:03.783 INFO:tasks.workunit.client.0.smithi162.stdout:9/321: dread d3/d23/f4b [0,4194304] 0 2024-04-27T04:10:03.783 INFO:tasks.workunit.client.0.smithi162.stdout:9/322: chown d3/c6b 5834 1 2024-04-27T04:10:03.785 INFO:tasks.workunit.client.0.smithi162.stdout:9/323: write d3/d23/f4b [2693779,80132] 0 2024-04-27T04:10:03.785 INFO:tasks.workunit.client.0.smithi162.stdout:5/332: rename d8/d9/c15 to d8/d1a/d23/d40/d4b/d66/c76 0 2024-04-27T04:10:03.785 INFO:tasks.workunit.client.1.smithi181.stdout:6/437: sync 2024-04-27T04:10:03.785 INFO:tasks.workunit.client.1.smithi181.stdout:4/493: sync 2024-04-27T04:10:03.787 INFO:tasks.workunit.client.1.smithi181.stdout:4/494: truncate d0/d15/d25/d2a/f38 4419090 0 2024-04-27T04:10:03.792 INFO:tasks.workunit.client.0.smithi162.stdout:6/328: creat d6/d1a/d34/d43/f6b x:0 0 0 2024-04-27T04:10:03.792 INFO:tasks.workunit.client.0.smithi162.stdout:6/329: chown d6/d1a/d34/d46/d67 254873 1 2024-04-27T04:10:03.793 INFO:tasks.workunit.client.1.smithi181.stdout:3/492: creat d0/d6c/d52/d44/d74/f9b x:0 0 0 2024-04-27T04:10:03.793 INFO:tasks.workunit.client.0.smithi162.stdout:6/330: write d6/f11 [679508,67636] 0 2024-04-27T04:10:03.793 INFO:tasks.workunit.client.1.smithi181.stdout:7/541: dwrite d1/d22/d15/f2a [0,4194304] 0 2024-04-27T04:10:03.796 INFO:tasks.workunit.client.1.smithi181.stdout:8/616: rmdir d1/d6/d1a/d25 39 2024-04-27T04:10:03.797 INFO:tasks.workunit.client.1.smithi181.stdout:6/438: mkdir d7/d31/d45/d72/d98 0 2024-04-27T04:10:03.798 INFO:tasks.workunit.client.0.smithi162.stdout:1/308: getdents d0/d7 0 2024-04-27T04:10:03.800 INFO:tasks.workunit.client.1.smithi181.stdout:2/588: link d4/d9/d44/f46 d4/d9/d4f/d79/fbc 0 2024-04-27T04:10:03.802 INFO:tasks.workunit.client.0.smithi162.stdout:4/363: truncate d2/f11 2616098 0 2024-04-27T04:10:03.803 INFO:tasks.workunit.client.1.smithi181.stdout:7/542: dwrite d1/d2/d1c/d8f/f8a [0,4194304] 0 2024-04-27T04:10:03.805 INFO:tasks.workunit.client.1.smithi181.stdout:7/543: dread d1/d2/d1c/d37/f3f [4194304,4194304] 0 2024-04-27T04:10:03.806 INFO:tasks.workunit.client.0.smithi162.stdout:3/334: creat d1/d18/f6a x:0 0 0 2024-04-27T04:10:03.807 INFO:tasks.workunit.client.0.smithi162.stdout:3/335: write d1/d18/f41 [573934,9052] 0 2024-04-27T04:10:03.812 INFO:tasks.workunit.client.1.smithi181.stdout:1/554: link d1/d2/d5/d6/da3/d1e/d20/da2/lae d1/d2/d5/laf 0 2024-04-27T04:10:03.814 INFO:tasks.workunit.client.0.smithi162.stdout:2/265: dwrite d0/d4/f15 [0,4194304] 0 2024-04-27T04:10:03.818 INFO:tasks.workunit.client.0.smithi162.stdout:9/324: mknod d3/d13/d2d/d4f/c6f 0 2024-04-27T04:10:03.824 INFO:tasks.workunit.client.0.smithi162.stdout:5/333: creat d8/d1a/d23/d40/f77 x:0 0 0 2024-04-27T04:10:03.824 INFO:tasks.workunit.client.0.smithi162.stdout:5/334: fdatasync d8/d2c/f33 0 2024-04-27T04:10:03.825 INFO:tasks.workunit.client.1.smithi181.stdout:8/617: creat d1/d6/d2f/d47/dba/fd6 x:0 0 0 2024-04-27T04:10:03.827 INFO:tasks.workunit.client.1.smithi181.stdout:3/493: dread d0/f16 [0,4194304] 0 2024-04-27T04:10:03.827 INFO:tasks.workunit.client.1.smithi181.stdout:4/495: write d0/d15/d25/d2b/f5b [881080,76603] 0 2024-04-27T04:10:03.831 INFO:tasks.workunit.client.0.smithi162.stdout:5/335: dread d8/d9/d55/f6d [0,4194304] 0 2024-04-27T04:10:03.834 INFO:tasks.workunit.client.1.smithi181.stdout:8/618: dwrite d1/d6/d2f/f3a [0,4194304] 0 2024-04-27T04:10:03.836 INFO:tasks.workunit.client.0.smithi162.stdout:6/331: rename d6/d1a/d34/d46 to d6/d1a/d34/d3a/d68/d6c 0 2024-04-27T04:10:03.838 INFO:tasks.workunit.client.1.smithi181.stdout:7/544: rmdir d1/d22 39 2024-04-27T04:10:03.840 INFO:tasks.workunit.client.1.smithi181.stdout:6/439: dread d7/dc/d1c/f35 [0,4194304] 0 2024-04-27T04:10:03.842 INFO:tasks.workunit.client.1.smithi181.stdout:6/440: dread - d7/dc/d1c/d34/d3f/f6c zero size 2024-04-27T04:10:03.843 INFO:tasks.workunit.client.1.smithi181.stdout:0/660: creat dc/d1f/d3b/d4a/d95/fe6 x:0 0 0 2024-04-27T04:10:03.845 INFO:tasks.workunit.client.0.smithi162.stdout:1/309: creat d0/d7/d3c/d54/f68 x:0 0 0 2024-04-27T04:10:03.846 INFO:tasks.workunit.client.0.smithi162.stdout:1/310: chown d0/d11/f2e 43 1 2024-04-27T04:10:03.850 INFO:tasks.workunit.client.1.smithi181.stdout:4/496: mkdir d0/d15/d76/da4 0 2024-04-27T04:10:03.852 INFO:tasks.workunit.client.0.smithi162.stdout:0/390: link d5/dc/d1e/d41/f34 d5/d26/d69/f84 0 2024-04-27T04:10:03.853 INFO:tasks.workunit.client.0.smithi162.stdout:8/357: link d0/d10/d2b/d2d/d2f/d6b/f7a d0/de/f7b 0 2024-04-27T04:10:03.855 INFO:tasks.workunit.client.0.smithi162.stdout:5/336: stat d8/d1a/d29/d4a/c61 0 2024-04-27T04:10:03.855 INFO:tasks.workunit.client.1.smithi181.stdout:8/619: dread d1/d6/d2f/d30/d66/f9c [0,4194304] 0 2024-04-27T04:10:03.856 INFO:tasks.workunit.client.1.smithi181.stdout:0/661: symlink dc/d1f/d3b/d4a/d95/d26/d68/le7 0 2024-04-27T04:10:03.857 INFO:tasks.workunit.client.0.smithi162.stdout:5/337: dread d8/d1a/f2d [0,4194304] 0 2024-04-27T04:10:03.858 INFO:tasks.workunit.client.1.smithi181.stdout:4/497: mkdir d0/d15/da5 0 2024-04-27T04:10:03.859 INFO:tasks.workunit.client.0.smithi162.stdout:8/358: dwrite d0/d10/d2b/d2d/d2f/f54 [0,4194304] 0 2024-04-27T04:10:03.859 INFO:tasks.workunit.client.0.smithi162.stdout:7/285: creat d3/d5/f5e x:0 0 0 2024-04-27T04:10:03.859 INFO:tasks.workunit.client.1.smithi181.stdout:2/589: link d4/d9/d35/ca8 d4/d9/df/d14/cbd 0 2024-04-27T04:10:03.860 INFO:tasks.workunit.client.0.smithi162.stdout:1/311: symlink d0/d7/d3c/d54/l69 0 2024-04-27T04:10:03.861 INFO:tasks.workunit.client.0.smithi162.stdout:1/312: chown d0/l3a 141 1 2024-04-27T04:10:03.861 INFO:tasks.workunit.client.1.smithi181.stdout:7/545: symlink d1/d22/d15/d1f/d34/la6 0 2024-04-27T04:10:03.862 INFO:tasks.workunit.client.1.smithi181.stdout:4/498: dread d0/d15/d25/d2b/f5f [0,4194304] 0 2024-04-27T04:10:03.863 INFO:tasks.workunit.client.1.smithi181.stdout:7/546: write d1/d2/d35/d8b/fa5 [284606,66305] 0 2024-04-27T04:10:03.864 INFO:tasks.workunit.client.1.smithi181.stdout:4/499: chown d0/d15/c1b 122 1 2024-04-27T04:10:03.865 INFO:tasks.workunit.client.1.smithi181.stdout:4/500: fdatasync d0/d15/d25/d40/f9b 0 2024-04-27T04:10:03.866 INFO:tasks.workunit.client.1.smithi181.stdout:0/662: mknod dc/d93/ce8 0 2024-04-27T04:10:03.868 INFO:tasks.workunit.client.1.smithi181.stdout:4/501: write d0/d15/d25/d2b/f5b [733613,93932] 0 2024-04-27T04:10:03.876 INFO:tasks.workunit.client.1.smithi181.stdout:6/441: creat d7/dc/d1c/f99 x:0 0 0 2024-04-27T04:10:03.880 INFO:tasks.workunit.client.1.smithi181.stdout:8/620: unlink d1/d6/d2f/d40/fb9 0 2024-04-27T04:10:03.882 INFO:tasks.workunit.client.0.smithi162.stdout:8/359: unlink d0/c3e 0 2024-04-27T04:10:03.883 INFO:tasks.workunit.client.1.smithi181.stdout:7/547: symlink d1/d2/d35/d67/la7 0 2024-04-27T04:10:03.884 INFO:tasks.workunit.client.1.smithi181.stdout:7/548: write d1/f8 [845111,23714] 0 2024-04-27T04:10:03.884 INFO:tasks.workunit.client.0.smithi162.stdout:7/286: stat d3/d5/d9/f13 0 2024-04-27T04:10:03.885 INFO:tasks.workunit.client.1.smithi181.stdout:7/549: fdatasync d1/d2/f19 0 2024-04-27T04:10:03.889 INFO:tasks.workunit.client.0.smithi162.stdout:2/266: truncate d0/d24/f38 398794 0 2024-04-27T04:10:03.890 INFO:tasks.workunit.client.1.smithi181.stdout:0/663: dread dc/d1f/d3b/f5f [4194304,4194304] 0 2024-04-27T04:10:03.892 INFO:tasks.workunit.client.1.smithi181.stdout:3/494: rename d0/de/f7f to d0/f9c 0 2024-04-27T04:10:03.894 INFO:tasks.workunit.client.0.smithi162.stdout:2/267: dwrite d0/d4/f15 [4194304,4194304] 0 2024-04-27T04:10:03.897 INFO:tasks.workunit.client.1.smithi181.stdout:0/664: dwrite dc/d93/fe0 [0,4194304] 0 2024-04-27T04:10:03.898 INFO:tasks.workunit.client.0.smithi162.stdout:4/364: rename d2/d25/d1c/d39/d3b/d52/f64 to d2/da/f78 0 2024-04-27T04:10:03.898 INFO:tasks.workunit.client.1.smithi181.stdout:0/665: chown dc/f11 5 1 2024-04-27T04:10:03.900 INFO:tasks.workunit.client.0.smithi162.stdout:6/332: truncate d6/db/d32/f3b 771691 0 2024-04-27T04:10:03.901 INFO:tasks.workunit.client.0.smithi162.stdout:2/268: dwrite d0/d24/f4d [0,4194304] 0 2024-04-27T04:10:03.901 INFO:tasks.workunit.client.0.smithi162.stdout:2/269: dread - d0/d24/f4b zero size 2024-04-27T04:10:03.905 INFO:tasks.workunit.client.1.smithi181.stdout:2/590: dwrite d4/d9/d16/f29 [4194304,4194304] 0 2024-04-27T04:10:03.911 INFO:tasks.workunit.client.0.smithi162.stdout:1/313: creat d0/d11/d3e/d5f/f6a x:0 0 0 2024-04-27T04:10:03.915 INFO:tasks.workunit.client.1.smithi181.stdout:2/591: dwrite d4/d9/d4f/d79/d98/fa2 [0,4194304] 0 2024-04-27T04:10:03.917 INFO:tasks.workunit.client.0.smithi162.stdout:9/325: getdents d3/d13/d2d/d44 0 2024-04-27T04:10:03.921 INFO:tasks.workunit.client.1.smithi181.stdout:8/621: creat d1/d6/d2f/d47/dba/fd7 x:0 0 0 2024-04-27T04:10:03.921 INFO:tasks.workunit.client.1.smithi181.stdout:5/662: sync 2024-04-27T04:10:03.922 INFO:tasks.workunit.client.1.smithi181.stdout:9/535: sync 2024-04-27T04:10:03.926 INFO:tasks.workunit.client.0.smithi162.stdout:4/365: creat d2/d25/f79 x:0 0 0 2024-04-27T04:10:03.927 INFO:tasks.workunit.client.1.smithi181.stdout:8/622: dwrite d1/d6/d2f/d30/d5c/f61 [0,4194304] 0 2024-04-27T04:10:03.930 INFO:tasks.workunit.client.1.smithi181.stdout:4/502: truncate d0/d15/d5c/d69/f10 613094 0 2024-04-27T04:10:03.932 INFO:tasks.workunit.client.0.smithi162.stdout:1/314: creat d0/d7/d10/f6b x:0 0 0 2024-04-27T04:10:03.932 INFO:tasks.workunit.client.0.smithi162.stdout:1/315: chown d0/d11/d3e/d5f 83 1 2024-04-27T04:10:03.936 INFO:tasks.workunit.client.0.smithi162.stdout:9/326: unlink d3/d13/d2d/d30/f6d 0 2024-04-27T04:10:03.937 INFO:tasks.workunit.client.0.smithi162.stdout:9/327: chown d3/d11/c12 140 1 2024-04-27T04:10:03.939 INFO:tasks.workunit.client.0.smithi162.stdout:3/336: rename d1/l48 to d1/d18/d5d/d5f/d66/l6b 0 2024-04-27T04:10:03.940 INFO:tasks.workunit.client.0.smithi162.stdout:3/337: fsync d1/d18/d23/f28 0 2024-04-27T04:10:03.941 INFO:tasks.workunit.client.0.smithi162.stdout:3/338: chown d1/d18/f12 222 1 2024-04-27T04:10:03.942 INFO:tasks.workunit.client.0.smithi162.stdout:4/366: readlink d2/da/d1a/l20 0 2024-04-27T04:10:03.946 INFO:tasks.workunit.client.0.smithi162.stdout:6/333: symlink d6/d1a/d34/l6d 0 2024-04-27T04:10:03.946 INFO:tasks.workunit.client.1.smithi181.stdout:2/592: dread - d4/d9/df/d14/f48 zero size 2024-04-27T04:10:03.947 INFO:tasks.workunit.client.0.smithi162.stdout:7/287: creat d3/d5/f5f x:0 0 0 2024-04-27T04:10:03.950 INFO:tasks.workunit.client.0.smithi162.stdout:9/328: rename d3/d23/c3d to d3/d23/c70 0 2024-04-27T04:10:03.950 INFO:tasks.workunit.client.0.smithi162.stdout:9/329: stat d3/d13/d15 0 2024-04-27T04:10:03.951 INFO:tasks.workunit.client.1.smithi181.stdout:9/536: chown d0/c36 28 1 2024-04-27T04:10:03.952 INFO:tasks.workunit.client.0.smithi162.stdout:5/338: rename d8/d9/d55/d57 to d8/d1a/d23/d40/d4b/d78 0 2024-04-27T04:10:03.954 INFO:tasks.workunit.client.0.smithi162.stdout:5/339: chown d8/d9/f16 7610316 1 2024-04-27T04:10:03.955 INFO:tasks.workunit.client.1.smithi181.stdout:7/550: link d1/d2/d35/d67/f51 d1/d2/d1c/d56/fa8 0 2024-04-27T04:10:03.956 INFO:tasks.workunit.client.0.smithi162.stdout:9/330: dwrite d3/fb [0,4194304] 0 2024-04-27T04:10:03.957 INFO:tasks.workunit.client.0.smithi162.stdout:4/367: dread d2/d25/f5c [0,4194304] 0 2024-04-27T04:10:03.961 INFO:tasks.workunit.client.0.smithi162.stdout:6/334: rmdir d6/d1a/d34/d43 39 2024-04-27T04:10:03.963 INFO:tasks.workunit.client.1.smithi181.stdout:3/495: link d0/d6c/f89 d0/de/d20/d58/d70/f9d 0 2024-04-27T04:10:03.964 INFO:tasks.workunit.client.0.smithi162.stdout:9/331: dread d3/d23/f3e [0,4194304] 0 2024-04-27T04:10:03.964 INFO:tasks.workunit.client.0.smithi162.stdout:0/391: sync 2024-04-27T04:10:03.964 INFO:tasks.workunit.client.0.smithi162.stdout:1/316: dread d0/d11/d3e/f4f [0,4194304] 0 2024-04-27T04:10:03.966 INFO:tasks.workunit.client.0.smithi162.stdout:0/392: chown d5/d26/f3c 7426 1 2024-04-27T04:10:03.966 INFO:tasks.workunit.client.0.smithi162.stdout:1/317: stat d0/d7/d2a/f2d 0 2024-04-27T04:10:03.966 INFO:tasks.workunit.client.0.smithi162.stdout:7/288: write d3/d39/d57/f44 [1930107,85562] 0 2024-04-27T04:10:03.968 INFO:tasks.workunit.client.1.smithi181.stdout:4/503: rename d0/d15/d25/d2b/c65 to d0/d15/d25/d40/ca6 0 2024-04-27T04:10:03.972 INFO:tasks.workunit.client.0.smithi162.stdout:0/393: dwrite d5/f11 [0,4194304] 0 2024-04-27T04:10:03.973 INFO:tasks.workunit.client.0.smithi162.stdout:1/318: dwrite d0/d7/d10/f4c [4194304,4194304] 0 2024-04-27T04:10:03.973 INFO:tasks.workunit.client.0.smithi162.stdout:0/394: stat d5/f16 0 2024-04-27T04:10:03.975 INFO:tasks.workunit.client.1.smithi181.stdout:0/666: dwrite dc/d1f/d3b/d4a/d95/d26/d41/f4c [0,4194304] 0 2024-04-27T04:10:03.977 INFO:tasks.workunit.client.1.smithi181.stdout:5/663: dread d4/d10/f57 [0,4194304] 0 2024-04-27T04:10:03.980 INFO:tasks.workunit.client.1.smithi181.stdout:3/496: dwrite d0/d6c/d52/d44/d74/f9b [0,4194304] 0 2024-04-27T04:10:03.980 INFO:tasks.workunit.client.1.smithi181.stdout:6/442: creat d7/dc/d1c/d34/f9a x:0 0 0 2024-04-27T04:10:03.981 INFO:tasks.workunit.client.0.smithi162.stdout:2/270: rename d0/c4e to d0/d4/d17/d2b/c53 0 2024-04-27T04:10:03.982 INFO:tasks.workunit.client.1.smithi181.stdout:7/551: dread d1/d2/d1c/d37/f4f [0,4194304] 0 2024-04-27T04:10:03.983 INFO:tasks.workunit.client.1.smithi181.stdout:2/593: dread - d4/d9/f64 zero size 2024-04-27T04:10:03.986 INFO:tasks.workunit.client.1.smithi181.stdout:2/594: fsync d4/d9/df/d14/f78 0 2024-04-27T04:10:03.986 INFO:tasks.workunit.client.0.smithi162.stdout:5/340: creat d8/d9/d5b/f79 x:0 0 0 2024-04-27T04:10:03.986 INFO:tasks.workunit.client.0.smithi162.stdout:2/271: dread d0/f1 [0,4194304] 0 2024-04-27T04:10:03.986 INFO:tasks.workunit.client.0.smithi162.stdout:5/341: chown d8/d1a/f3e 0 1 2024-04-27T04:10:03.987 INFO:tasks.workunit.client.0.smithi162.stdout:2/272: fdatasync d0/d4/f22 0 2024-04-27T04:10:03.987 INFO:tasks.workunit.client.0.smithi162.stdout:5/342: readlink d8/la 0 2024-04-27T04:10:03.987 INFO:tasks.workunit.client.1.smithi181.stdout:9/537: read d0/d44/d4f/f54 [121117,121645] 0 2024-04-27T04:10:03.988 INFO:tasks.workunit.client.0.smithi162.stdout:5/343: dread - d8/d1a/d23/d40/d4b/f75 zero size 2024-04-27T04:10:03.989 INFO:tasks.workunit.client.0.smithi162.stdout:3/339: link d1/d18/d5d/d5f/d66/f68 d1/d45/d69/f6c 0 2024-04-27T04:10:03.992 INFO:tasks.workunit.client.1.smithi181.stdout:1/555: sync 2024-04-27T04:10:03.993 INFO:tasks.workunit.client.0.smithi162.stdout:4/368: fdatasync d2/d25/d1c/d39/d3b/d47/f4c 0 2024-04-27T04:10:03.993 INFO:tasks.workunit.client.0.smithi162.stdout:8/360: sync 2024-04-27T04:10:03.995 INFO:tasks.workunit.client.1.smithi181.stdout:4/504: readlink d0/d15/de/d60/d6f/l8e 0 2024-04-27T04:10:03.995 INFO:tasks.workunit.client.0.smithi162.stdout:4/369: fsync d2/d25/d1c/f55 0 2024-04-27T04:10:03.995 INFO:tasks.workunit.client.0.smithi162.stdout:2/273: dwrite d0/f1 [4194304,4194304] 0 2024-04-27T04:10:03.996 INFO:tasks.workunit.client.0.smithi162.stdout:6/335: rmdir d6/d1a 39 2024-04-27T04:10:03.997 INFO:tasks.workunit.client.0.smithi162.stdout:5/344: dwrite d8/d1a/d23/d40/f42 [0,4194304] 0 2024-04-27T04:10:03.997 INFO:tasks.workunit.client.0.smithi162.stdout:2/274: dread - d0/d4/d2a/d41/f4a zero size 2024-04-27T04:10:04.002 INFO:tasks.workunit.client.1.smithi181.stdout:0/667: unlink dc/d1f/f29 0 2024-04-27T04:10:04.003 INFO:tasks.workunit.client.1.smithi181.stdout:8/623: write d1/d6/d88/fb1 [16318,66560] 0 2024-04-27T04:10:04.004 INFO:tasks.workunit.client.1.smithi181.stdout:0/668: write dc/f15 [4993632,99372] 0 2024-04-27T04:10:04.007 INFO:tasks.workunit.client.1.smithi181.stdout:3/497: fdatasync d0/f19 0 2024-04-27T04:10:04.011 INFO:tasks.workunit.client.1.smithi181.stdout:6/443: rename d7/dc/f41 to d7/dc/d8b/f9b 0 2024-04-27T04:10:04.012 INFO:tasks.workunit.client.0.smithi162.stdout:7/289: chown d3/f16 54841598 1 2024-04-27T04:10:04.012 INFO:tasks.workunit.client.1.smithi181.stdout:7/552: fsync d1/d2/d1c/d56/fa8 0 2024-04-27T04:10:04.014 INFO:tasks.workunit.client.1.smithi181.stdout:2/595: mkdir d4/d9/d16/d82/d7d/dbe 0 2024-04-27T04:10:04.016 INFO:tasks.workunit.client.1.smithi181.stdout:2/596: write d4/d9/d16/d45/f87 [247120,128460] 0 2024-04-27T04:10:04.016 INFO:tasks.workunit.client.0.smithi162.stdout:1/319: creat d0/d11/d3e/f6c x:0 0 0 2024-04-27T04:10:04.018 INFO:tasks.workunit.client.1.smithi181.stdout:2/597: read - d4/d9/d35/f60 zero size 2024-04-27T04:10:04.022 INFO:tasks.workunit.client.0.smithi162.stdout:3/340: creat d1/d21/f6d x:0 0 0 2024-04-27T04:10:04.023 INFO:tasks.workunit.client.1.smithi181.stdout:8/624: fdatasync d1/d6/d2f/d30/faf 0 2024-04-27T04:10:04.023 INFO:tasks.workunit.client.0.smithi162.stdout:3/341: write d1/d18/d23/d2f/f42 [3430801,112291] 0 2024-04-27T04:10:04.025 INFO:tasks.workunit.client.1.smithi181.stdout:0/669: mknod dc/d1f/ce9 0 2024-04-27T04:10:04.026 INFO:tasks.workunit.client.0.smithi162.stdout:8/361: mkdir d0/d10/d2b/d2d/d2f/d39/d7c 0 2024-04-27T04:10:04.027 INFO:tasks.workunit.client.0.smithi162.stdout:4/370: readlink d2/da/d1a/l4e 0 2024-04-27T04:10:04.029 INFO:tasks.workunit.client.0.smithi162.stdout:5/345: mknod d8/d9/d5a/c7a 0 2024-04-27T04:10:04.030 INFO:tasks.workunit.client.0.smithi162.stdout:5/346: write d8/d1a/d23/f50 [189512,74768] 0 2024-04-27T04:10:04.031 INFO:tasks.workunit.client.0.smithi162.stdout:5/347: fsync d8/d2c/f4c 0 2024-04-27T04:10:04.032 INFO:tasks.workunit.client.0.smithi162.stdout:2/275: rename d0/d4/f42 to d0/d4/d2a/d41/f54 0 2024-04-27T04:10:04.032 INFO:tasks.workunit.client.0.smithi162.stdout:2/276: chown d0 1163 1 2024-04-27T04:10:04.038 INFO:tasks.workunit.client.0.smithi162.stdout:1/320: creat d0/d11/d3e/f6d x:0 0 0 2024-04-27T04:10:04.041 INFO:tasks.workunit.client.0.smithi162.stdout:3/342: mkdir d1/d21/d47/d57/d6e 0 2024-04-27T04:10:04.049 INFO:tasks.workunit.client.0.smithi162.stdout:8/362: rename d0/d10/d44/d5f to d0/d10/d25/d7d 0 2024-04-27T04:10:04.049 INFO:tasks.workunit.client.1.smithi181.stdout:5/664: dwrite d4/de/d2c/d46/d4d/d8e/fdc [0,4194304] 0 2024-04-27T04:10:04.050 INFO:tasks.workunit.client.1.smithi181.stdout:5/665: fsync d4/d10/d29/d92/fd9 0 2024-04-27T04:10:04.051 INFO:tasks.workunit.client.1.smithi181.stdout:4/505: symlink d0/d15/de/la7 0 2024-04-27T04:10:04.051 INFO:tasks.workunit.client.0.smithi162.stdout:2/277: symlink d0/d4/d2a/d3b/l55 0 2024-04-27T04:10:04.052 INFO:tasks.workunit.client.0.smithi162.stdout:2/278: write d0/d4/f3a [208946,24808] 0 2024-04-27T04:10:04.057 INFO:tasks.workunit.client.1.smithi181.stdout:8/625: creat d1/d6/d6f/dd2/fd8 x:0 0 0 2024-04-27T04:10:04.057 INFO:tasks.workunit.client.1.smithi181.stdout:1/556: write d1/d2/d5/d6/da3/d1e/f49 [4209402,120333] 0 2024-04-27T04:10:04.058 INFO:tasks.workunit.client.0.smithi162.stdout:7/290: mkdir d3/d5/d5b/d60 0 2024-04-27T04:10:04.061 INFO:tasks.workunit.client.0.smithi162.stdout:0/395: link d5/dc/d1e/d41/f39 d5/dc/d1e/d46/f85 0 2024-04-27T04:10:04.062 INFO:tasks.workunit.client.1.smithi181.stdout:9/538: dwrite d0/d44/d4b/f80 [0,4194304] 0 2024-04-27T04:10:04.064 INFO:tasks.workunit.client.0.smithi162.stdout:6/336: dread d6/db/d32/f3b [0,4194304] 0 2024-04-27T04:10:04.066 INFO:tasks.workunit.client.0.smithi162.stdout:1/321: mkdir d0/d7/d3c/d6e 0 2024-04-27T04:10:04.067 INFO:tasks.workunit.client.0.smithi162.stdout:1/322: readlink d0/d7/d3c/d54/l69 0 2024-04-27T04:10:04.069 INFO:tasks.workunit.client.1.smithi181.stdout:0/670: dwrite dc/f10 [4194304,4194304] 0 2024-04-27T04:10:04.069 INFO:tasks.workunit.client.1.smithi181.stdout:7/553: mknod d1/d22/d15/d13/ca9 0 2024-04-27T04:10:04.071 INFO:tasks.workunit.client.1.smithi181.stdout:7/554: write d1/d2/f25 [5141063,76539] 0 2024-04-27T04:10:04.071 INFO:tasks.workunit.client.0.smithi162.stdout:3/343: mkdir d1/d18/d5d/d6f 0 2024-04-27T04:10:04.072 INFO:tasks.workunit.client.1.smithi181.stdout:7/555: write d1/d2/d1c/d2d/d61/f68 [3294802,50752] 0 2024-04-27T04:10:04.074 INFO:tasks.workunit.client.1.smithi181.stdout:7/556: chown d1/d2/d1c/d56/l5a 117 1 2024-04-27T04:10:04.079 INFO:tasks.workunit.client.0.smithi162.stdout:8/363: mkdir d0/de/d20/d7e 0 2024-04-27T04:10:04.083 INFO:tasks.workunit.client.0.smithi162.stdout:9/332: getdents d3/d23/d1e 0 2024-04-27T04:10:04.084 INFO:tasks.workunit.client.0.smithi162.stdout:9/333: write d3/d23/d1e/d43/f48 [6883968,92448] 0 2024-04-27T04:10:04.085 INFO:tasks.workunit.client.1.smithi181.stdout:4/506: mkdir d0/d15/de/d19/d77/d57/da8 0 2024-04-27T04:10:04.086 INFO:tasks.workunit.client.0.smithi162.stdout:7/291: rmdir d3/d39/d57 39 2024-04-27T04:10:04.087 INFO:tasks.workunit.client.1.smithi181.stdout:4/507: dread - d0/d15/f32 zero size 2024-04-27T04:10:04.088 INFO:tasks.workunit.client.1.smithi181.stdout:6/444: dwrite d7/dc/f12 [0,4194304] 0 2024-04-27T04:10:04.089 INFO:tasks.workunit.client.1.smithi181.stdout:6/445: chown d7/d31 118 1 2024-04-27T04:10:04.090 INFO:tasks.workunit.client.1.smithi181.stdout:6/446: dread - d7/dc/d1c/d88/f95 zero size 2024-04-27T04:10:04.091 INFO:tasks.workunit.client.1.smithi181.stdout:8/626: mkdir d1/d6/d88/dd9 0 2024-04-27T04:10:04.091 INFO:tasks.workunit.client.0.smithi162.stdout:0/396: dread d5/d26/d69/f84 [0,4194304] 0 2024-04-27T04:10:04.092 INFO:tasks.workunit.client.1.smithi181.stdout:8/627: fsync d1/d6/d6f/dd2/fd4 0 2024-04-27T04:10:04.093 INFO:tasks.workunit.client.0.smithi162.stdout:4/371: dwrite d2/d25/d1c/f3d [0,4194304] 0 2024-04-27T04:10:04.094 INFO:tasks.workunit.client.0.smithi162.stdout:9/334: dwrite d3/d23/d1e/d43/f48 [0,4194304] 0 2024-04-27T04:10:04.096 INFO:tasks.workunit.client.1.smithi181.stdout:3/498: write d0/f57 [324196,127445] 0 2024-04-27T04:10:04.098 INFO:tasks.workunit.client.0.smithi162.stdout:2/279: mkdir d0/d4/d17/d4f/d56 0 2024-04-27T04:10:04.101 INFO:tasks.workunit.client.1.smithi181.stdout:2/598: write d4/d9/d16/d45/f9e [7677342,115455] 0 2024-04-27T04:10:04.105 INFO:tasks.workunit.client.1.smithi181.stdout:0/671: mkdir dc/d1f/d6b/d7e/d91/dea 0 2024-04-27T04:10:04.107 INFO:tasks.workunit.client.1.smithi181.stdout:0/672: dread - dc/d1f/d3b/db2/fdb zero size 2024-04-27T04:10:04.111 INFO:tasks.workunit.client.1.smithi181.stdout:2/599: dwrite d4/d9/f7a [0,4194304] 0 2024-04-27T04:10:04.111 INFO:tasks.workunit.client.1.smithi181.stdout:0/673: read - dc/d1f/d3b/d4a/d95/d26/d41/d6f/d7d/f9b zero size 2024-04-27T04:10:04.112 INFO:tasks.workunit.client.1.smithi181.stdout:7/557: unlink d1/d22/d15/d29/f4a 0 2024-04-27T04:10:04.114 INFO:tasks.workunit.client.0.smithi162.stdout:3/344: mkdir d1/d70 0 2024-04-27T04:10:04.115 INFO:tasks.workunit.client.0.smithi162.stdout:5/348: sync 2024-04-27T04:10:04.115 INFO:tasks.workunit.client.1.smithi181.stdout:5/666: truncate d4/d10/d72/db3/fb6 721571 0 2024-04-27T04:10:04.117 INFO:tasks.workunit.client.0.smithi162.stdout:4/372: mkdir d2/d54/d7a 0 2024-04-27T04:10:04.118 INFO:tasks.workunit.client.0.smithi162.stdout:6/337: dread d6/d1a/f47 [0,4194304] 0 2024-04-27T04:10:04.120 INFO:tasks.workunit.client.0.smithi162.stdout:1/323: truncate d0/d7/f1f 862127 0 2024-04-27T04:10:04.120 INFO:tasks.workunit.client.0.smithi162.stdout:8/364: creat d0/d10/d44/d70/f7f x:0 0 0 2024-04-27T04:10:04.121 INFO:tasks.workunit.client.1.smithi181.stdout:8/628: chown d1/d6/d2f/d40/d2a/f64 6951 1 2024-04-27T04:10:04.121 INFO:tasks.workunit.client.0.smithi162.stdout:1/324: dread - d0/d7/d3c/f43 zero size 2024-04-27T04:10:04.122 INFO:tasks.workunit.client.1.smithi181.stdout:9/539: rename d0/d44/d4f/c75 to d0/d91/cbc 0 2024-04-27T04:10:04.124 INFO:tasks.workunit.client.1.smithi181.stdout:9/540: read d0/d40/f81 [242834,37148] 0 2024-04-27T04:10:04.125 INFO:tasks.workunit.client.0.smithi162.stdout:7/292: creat d3/d5/d5b/d60/f61 x:0 0 0 2024-04-27T04:10:04.127 INFO:tasks.workunit.client.0.smithi162.stdout:0/397: mknod d5/dc/d1e/d46/d48/c86 0 2024-04-27T04:10:04.129 INFO:tasks.workunit.client.0.smithi162.stdout:5/349: chown d8/cf 942041 1 2024-04-27T04:10:04.129 INFO:tasks.workunit.client.0.smithi162.stdout:5/350: chown d8/c1b 4 1 2024-04-27T04:10:04.129 INFO:tasks.workunit.client.0.smithi162.stdout:4/373: write d2/d25/d1c/d39/f45 [2705074,75114] 0 2024-04-27T04:10:04.130 INFO:tasks.workunit.client.1.smithi181.stdout:1/557: dread d1/d2/d5/d6/da3/f3b [0,4194304] 0 2024-04-27T04:10:04.130 INFO:tasks.workunit.client.1.smithi181.stdout:6/447: mknod d7/dc/d1c/c9c 0 2024-04-27T04:10:04.134 INFO:tasks.workunit.client.1.smithi181.stdout:8/629: creat d1/d6/d2f/d30/d5c/fda x:0 0 0 2024-04-27T04:10:04.137 INFO:tasks.workunit.client.1.smithi181.stdout:1/558: read d1/d2/d5/d6/da3/d1e/d20/d53/d56/f6f [1363340,65866] 0 2024-04-27T04:10:04.137 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:10:03 smithi162 ceph-mon[26841]: pgmap v267: 65 pgs: 65 active+clean; 1.7 GiB data, 6.9 GiB used, 530 GiB / 536 GiB avail; 42 MiB/s rd, 224 MiB/s wr, 413 op/s 2024-04-27T04:10:04.142 INFO:tasks.workunit.client.1.smithi181.stdout:7/558: rename d1/c54 to d1/d22/d15/d82/caa 0 2024-04-27T04:10:04.146 INFO:tasks.workunit.client.1.smithi181.stdout:7/559: truncate d1/d2/d35/d8b/fa1 34338 0 2024-04-27T04:10:04.147 INFO:tasks.workunit.client.1.smithi181.stdout:7/560: readlink d1/d2/d1c/d56/l5a 0 2024-04-27T04:10:04.147 INFO:tasks.workunit.client.1.smithi181.stdout:6/448: dwrite d7/dc/d1e/f83 [0,4194304] 0 2024-04-27T04:10:04.149 INFO:tasks.workunit.client.1.smithi181.stdout:7/561: write d1/d22/f65 [611710,94332] 0 2024-04-27T04:10:04.150 INFO:tasks.workunit.client.1.smithi181.stdout:7/562: write d1/d22/f60 [2918576,64047] 0 2024-04-27T04:10:04.156 INFO:tasks.workunit.client.1.smithi181.stdout:8/630: unlink d1/d6/d1a/d25/cc9 0 2024-04-27T04:10:04.160 INFO:tasks.workunit.client.1.smithi181.stdout:6/449: unlink d7/d27/f4c 0 2024-04-27T04:10:04.160 INFO:tasks.workunit.client.1.smithi181.stdout:2/600: getdents d4/d9/d16/d45/d9a/d9d 0 2024-04-27T04:10:04.165 INFO:tasks.workunit.client.1.smithi181.stdout:4/508: dwrite d0/d15/d25/d2b/f6 [0,4194304] 0 2024-04-27T04:10:04.168 INFO:tasks.workunit.client.1.smithi181.stdout:6/450: dread d7/dc/d1e/d2b/f5a [0,4194304] 0 2024-04-27T04:10:04.172 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:10:03 smithi181 ceph-mon[33774]: pgmap v267: 65 pgs: 65 active+clean; 1.7 GiB data, 6.9 GiB used, 530 GiB / 536 GiB avail; 42 MiB/s rd, 224 MiB/s wr, 413 op/s 2024-04-27T04:10:04.179 INFO:tasks.workunit.client.1.smithi181.stdout:8/631: mknod d1/d6/d2f/d40/d2a/db3/cdb 0 2024-04-27T04:10:04.180 INFO:tasks.workunit.client.1.smithi181.stdout:4/509: fdatasync d0/d15/d25/d2a/d36/f51 0 2024-04-27T04:10:04.183 INFO:tasks.workunit.client.1.smithi181.stdout:5/667: rename d4/de/d9d/led to d4/de/d2c/d2e/lee 0 2024-04-27T04:10:04.184 INFO:tasks.workunit.client.1.smithi181.stdout:3/499: dwrite d0/f19 [0,4194304] 0 2024-04-27T04:10:04.190 INFO:tasks.workunit.client.1.smithi181.stdout:0/674: truncate dc/f11 8545830 0 2024-04-27T04:10:04.193 INFO:tasks.workunit.client.1.smithi181.stdout:9/541: write d0/f10 [494355,56300] 0 2024-04-27T04:10:04.194 INFO:tasks.workunit.client.1.smithi181.stdout:1/559: write d1/d2/d5/d6/da3/d1e/d20/d53/f72 [1418256,44099] 0 2024-04-27T04:10:04.196 INFO:tasks.workunit.client.0.smithi162.stdout:1/325: unlink d0/f38 0 2024-04-27T04:10:04.196 INFO:tasks.workunit.client.0.smithi162.stdout:6/338: truncate d6/d1a/d34/d43/f5c 456774 0 2024-04-27T04:10:04.199 INFO:tasks.workunit.client.0.smithi162.stdout:6/339: fsync d6/d1a/d34/d43/f5c 0 2024-04-27T04:10:04.199 INFO:tasks.workunit.client.0.smithi162.stdout:2/280: write d0/d4/f16 [1518197,27448] 0 2024-04-27T04:10:04.200 INFO:tasks.workunit.client.0.smithi162.stdout:9/335: truncate d3/d23/f3e 2347650 0 2024-04-27T04:10:04.202 INFO:tasks.workunit.client.0.smithi162.stdout:5/351: dwrite d8/d1a/f59 [0,4194304] 0 2024-04-27T04:10:04.204 INFO:tasks.workunit.client.0.smithi162.stdout:6/340: write d6/db/f13 [409258,6661] 0 2024-04-27T04:10:04.205 INFO:tasks.workunit.client.0.smithi162.stdout:2/281: write d0/d4/d2a/f3e [677043,92161] 0 2024-04-27T04:10:04.205 INFO:tasks.workunit.client.1.smithi181.stdout:5/668: dread d4/de/f47 [0,4194304] 0 2024-04-27T04:10:04.206 INFO:tasks.workunit.client.0.smithi162.stdout:5/352: chown d8/d2c/f4c 6 1 2024-04-27T04:10:04.207 INFO:tasks.workunit.client.0.smithi162.stdout:1/326: dread d0/d7/d2a/f31 [0,4194304] 0 2024-04-27T04:10:04.210 INFO:tasks.workunit.client.0.smithi162.stdout:5/353: readlink d8/l2b 0 2024-04-27T04:10:04.212 INFO:tasks.workunit.client.0.smithi162.stdout:5/354: fsync d8/d1a/f31 0 2024-04-27T04:10:04.213 INFO:tasks.workunit.client.0.smithi162.stdout:6/341: dread d6/d1a/d34/d3a/f3d [0,4194304] 0 2024-04-27T04:10:04.213 INFO:tasks.workunit.client.0.smithi162.stdout:0/398: dwrite f1 [0,4194304] 0 2024-04-27T04:10:04.215 INFO:tasks.workunit.client.0.smithi162.stdout:9/336: symlink d3/d13/d2d/l71 0 2024-04-27T04:10:04.215 INFO:tasks.workunit.client.0.smithi162.stdout:5/355: truncate d8/d9/d5b/f79 706580 0 2024-04-27T04:10:04.217 INFO:tasks.workunit.client.0.smithi162.stdout:7/293: dread d3/d4f/f31 [0,4194304] 0 2024-04-27T04:10:04.217 INFO:tasks.workunit.client.0.smithi162.stdout:5/356: fdatasync d8/d9/d55/f69 0 2024-04-27T04:10:04.223 INFO:tasks.workunit.client.0.smithi162.stdout:3/345: sync 2024-04-27T04:10:04.223 INFO:tasks.workunit.client.0.smithi162.stdout:2/282: rename d0/d4/d2a/l2c to d0/d4/d17/d2b/l57 0 2024-04-27T04:10:04.225 INFO:tasks.workunit.client.1.smithi181.stdout:8/632: unlink d1/d6/d6f/l78 0 2024-04-27T04:10:04.225 INFO:tasks.workunit.client.0.smithi162.stdout:5/357: dwrite d8/d1a/d29/f43 [0,4194304] 0 2024-04-27T04:10:04.227 INFO:tasks.workunit.client.0.smithi162.stdout:1/327: unlink d0/d7/d2a/f2d 0 2024-04-27T04:10:04.228 INFO:tasks.workunit.client.0.smithi162.stdout:4/374: link d2/d25/d1c/d39/l77 d2/d54/d7a/l7b 0 2024-04-27T04:10:04.229 INFO:tasks.workunit.client.0.smithi162.stdout:3/346: dread d1/d21/d47/d57/f5e [0,4194304] 0 2024-04-27T04:10:04.229 INFO:tasks.workunit.client.0.smithi162.stdout:4/375: write d2/d25/d1c/d72/f65 [4096622,60601] 0 2024-04-27T04:10:04.230 INFO:tasks.workunit.client.1.smithi181.stdout:2/601: rename d4/d9/d16/d82/f71 to d4/d9/d35/d42/fbf 0 2024-04-27T04:10:04.231 INFO:tasks.workunit.client.1.smithi181.stdout:2/602: dread - d4/d9/f64 zero size 2024-04-27T04:10:04.232 INFO:tasks.workunit.client.0.smithi162.stdout:6/342: dread - d6/f1e zero size 2024-04-27T04:10:04.234 INFO:tasks.workunit.client.0.smithi162.stdout:6/343: chown d6/d1a/d34/d3a/d68 0 1 2024-04-27T04:10:04.235 INFO:tasks.workunit.client.0.smithi162.stdout:6/344: write d6/d1a/d34/d50/f63 [269196,1442] 0 2024-04-27T04:10:04.238 INFO:tasks.workunit.client.0.smithi162.stdout:9/337: mkdir d3/d13/d2d/d3b/d72 0 2024-04-27T04:10:04.239 INFO:tasks.workunit.client.0.smithi162.stdout:9/338: fdatasync d3/d23/f22 0 2024-04-27T04:10:04.240 INFO:tasks.workunit.client.0.smithi162.stdout:9/339: stat d3/d13/d2d/d30/l63 0 2024-04-27T04:10:04.240 INFO:tasks.workunit.client.1.smithi181.stdout:5/669: chown d4/de/d2c/d46/d4d/d8e/d67/d71/l74 13 1 2024-04-27T04:10:04.241 INFO:tasks.workunit.client.0.smithi162.stdout:9/340: write d3/d23/d1e/d43/f5a [2691701,91953] 0 2024-04-27T04:10:04.243 INFO:tasks.workunit.client.0.smithi162.stdout:9/341: readlink d3/d13/d15/l1d 0 2024-04-27T04:10:04.247 INFO:tasks.workunit.client.1.smithi181.stdout:8/633: creat d1/d8c/fdc x:0 0 0 2024-04-27T04:10:04.248 INFO:tasks.workunit.client.0.smithi162.stdout:7/294: dwrite d3/d39/d57/f43 [0,4194304] 0 2024-04-27T04:10:04.248 INFO:tasks.workunit.client.1.smithi181.stdout:4/510: creat d0/d15/de/d19/d8a/fa9 x:0 0 0 2024-04-27T04:10:04.249 INFO:tasks.workunit.client.0.smithi162.stdout:9/342: dread d3/fb [0,4194304] 0 2024-04-27T04:10:04.250 INFO:tasks.workunit.client.0.smithi162.stdout:7/295: write d3/d5/f5e [685351,40328] 0 2024-04-27T04:10:04.250 INFO:tasks.workunit.client.0.smithi162.stdout:8/365: sync 2024-04-27T04:10:04.251 INFO:tasks.workunit.client.1.smithi181.stdout:6/451: write d7/dc/f3a [981104,97712] 0 2024-04-27T04:10:04.252 INFO:tasks.workunit.client.0.smithi162.stdout:8/366: chown d0/de/l4f 13 1 2024-04-27T04:10:04.252 INFO:tasks.workunit.client.0.smithi162.stdout:7/296: write d3/d5/d9/d4b/de/f30 [4943391,126821] 0 2024-04-27T04:10:04.253 INFO:tasks.workunit.client.0.smithi162.stdout:1/328: rename d0/f14 to d0/d11/d44/d66/f6f 0 2024-04-27T04:10:04.254 INFO:tasks.workunit.client.0.smithi162.stdout:0/399: dread d5/dc/f18 [0,4194304] 0 2024-04-27T04:10:04.254 INFO:tasks.workunit.client.0.smithi162.stdout:8/367: fdatasync d0/de/d6e/f79 0 2024-04-27T04:10:04.256 INFO:tasks.workunit.client.0.smithi162.stdout:2/283: dread d0/d4/d17/f2f [0,4194304] 0 2024-04-27T04:10:04.256 INFO:tasks.workunit.client.0.smithi162.stdout:7/297: write d3/d5/d9/f24 [1749209,26377] 0 2024-04-27T04:10:04.257 INFO:tasks.workunit.client.0.smithi162.stdout:0/400: write d5/d26/d4a/f4c [334799,96284] 0 2024-04-27T04:10:04.258 INFO:tasks.workunit.client.0.smithi162.stdout:2/284: stat d0/d4/d2a/d3b 0 2024-04-27T04:10:04.258 INFO:tasks.workunit.client.0.smithi162.stdout:7/298: dread - d3/d4f/f41 zero size 2024-04-27T04:10:04.258 INFO:tasks.workunit.client.1.smithi181.stdout:3/500: mkdir d0/de/d20/d6b/d91/d9e 0 2024-04-27T04:10:04.259 INFO:tasks.workunit.client.0.smithi162.stdout:0/401: readlink d5/dc/d1e/l6b 0 2024-04-27T04:10:04.259 INFO:tasks.workunit.client.0.smithi162.stdout:7/299: fsync d3/d4f/f41 0 2024-04-27T04:10:04.261 INFO:tasks.workunit.client.0.smithi162.stdout:3/347: mknod d1/d21/c71 0 2024-04-27T04:10:04.261 INFO:tasks.workunit.client.0.smithi162.stdout:3/348: chown d1/d21/d47/d57 42 1 2024-04-27T04:10:04.262 INFO:tasks.workunit.client.0.smithi162.stdout:0/402: dread f1 [0,4194304] 0 2024-04-27T04:10:04.263 INFO:tasks.workunit.client.0.smithi162.stdout:4/376: symlink d2/d25/d1c/d39/d3b/d52/l7c 0 2024-04-27T04:10:04.264 INFO:tasks.workunit.client.0.smithi162.stdout:4/377: write d2/d25/d1c/d24/f56 [4524123,3191] 0 2024-04-27T04:10:04.267 INFO:tasks.workunit.client.0.smithi162.stdout:6/345: symlink d6/d1a/d34/d3a/d68/l6e 0 2024-04-27T04:10:04.267 INFO:tasks.workunit.client.0.smithi162.stdout:6/346: truncate d6/f62 4712385 0 2024-04-27T04:10:04.269 INFO:tasks.workunit.client.1.smithi181.stdout:0/675: dwrite dc/d1f/d3b/f77 [0,4194304] 0 2024-04-27T04:10:04.279 INFO:tasks.workunit.client.1.smithi181.stdout:3/501: fdatasync d0/de/d20/f5c 0 2024-04-27T04:10:04.281 INFO:tasks.workunit.client.1.smithi181.stdout:1/560: creat d1/d2/d5/fb0 x:0 0 0 2024-04-27T04:10:04.281 INFO:tasks.workunit.client.0.smithi162.stdout:5/358: dread d8/d1a/d23/d40/d4b/f52 [8388608,4194304] 0 2024-04-27T04:10:04.282 INFO:tasks.workunit.client.0.smithi162.stdout:9/343: chown d3/l9 171454 1 2024-04-27T04:10:04.285 INFO:tasks.workunit.client.1.smithi181.stdout:8/634: dread d1/f6b [0,4194304] 0 2024-04-27T04:10:04.285 INFO:tasks.workunit.client.1.smithi181.stdout:5/670: truncate d4/d10/d72/db3/fb6 908736 0 2024-04-27T04:10:04.286 INFO:tasks.workunit.client.1.smithi181.stdout:8/635: chown d1/d6/d2f/d30/d66 732 1 2024-04-27T04:10:04.292 INFO:tasks.workunit.client.1.smithi181.stdout:5/671: dwrite d4/da3/fbd [0,4194304] 0 2024-04-27T04:10:04.295 INFO:tasks.workunit.client.0.smithi162.stdout:2/285: symlink d0/d4/d2a/l58 0 2024-04-27T04:10:04.295 INFO:tasks.workunit.client.1.smithi181.stdout:4/511: write d0/d15/f1a [3979271,89688] 0 2024-04-27T04:10:04.298 INFO:tasks.workunit.client.1.smithi181.stdout:2/603: rename d4/d9/f8c to d4/d9/fc0 0 2024-04-27T04:10:04.301 INFO:tasks.workunit.client.1.smithi181.stdout:5/672: dwrite d4/de/d2c/d2e/d55/f3f [0,4194304] 0 2024-04-27T04:10:04.302 INFO:tasks.workunit.client.0.smithi162.stdout:7/300: creat d3/d5/d9/d4b/de/f62 x:0 0 0 2024-04-27T04:10:04.303 INFO:tasks.workunit.client.0.smithi162.stdout:3/349: symlink d1/d18/d5d/d5f/d66/l72 0 2024-04-27T04:10:04.304 INFO:tasks.workunit.client.0.smithi162.stdout:3/350: write d1/d18/d23/f28 [890361,106335] 0 2024-04-27T04:10:04.305 INFO:tasks.workunit.client.0.smithi162.stdout:3/351: chown d1/d21/f60 240 1 2024-04-27T04:10:04.308 INFO:tasks.workunit.client.0.smithi162.stdout:0/403: rmdir d5/d26 39 2024-04-27T04:10:04.309 INFO:tasks.workunit.client.1.smithi181.stdout:8/636: rmdir d1/d6/d2f/d30/d66 39 2024-04-27T04:10:04.313 INFO:tasks.workunit.client.0.smithi162.stdout:6/347: mknod d6/d1a/d34/d43/c6f 0 2024-04-27T04:10:04.315 INFO:tasks.workunit.client.0.smithi162.stdout:5/359: mkdir d8/d1a/d23/d40/d7b 0 2024-04-27T04:10:04.316 INFO:tasks.workunit.client.1.smithi181.stdout:0/676: dwrite dc/d1f/d3b/fbf [0,4194304] 0 2024-04-27T04:10:04.318 INFO:tasks.workunit.client.1.smithi181.stdout:6/452: dwrite d7/dc/d1e/f53 [0,4194304] 0 2024-04-27T04:10:04.318 INFO:tasks.workunit.client.0.smithi162.stdout:9/344: creat d3/d23/f73 x:0 0 0 2024-04-27T04:10:04.319 INFO:tasks.workunit.client.1.smithi181.stdout:2/604: creat d4/d9/d4f/d5f/fc1 x:0 0 0 2024-04-27T04:10:04.321 INFO:tasks.workunit.client.0.smithi162.stdout:1/329: rename d0/d7/l40 to d0/d7/d3c/d54/l70 0 2024-04-27T04:10:04.322 INFO:tasks.workunit.client.0.smithi162.stdout:1/330: truncate d0/d11/d50/f56 841748 0 2024-04-27T04:10:04.323 INFO:tasks.workunit.client.1.smithi181.stdout:2/605: dread d4/d9/d4f/d79/fa3 [0,4194304] 0 2024-04-27T04:10:04.324 INFO:tasks.workunit.client.1.smithi181.stdout:2/606: readlink d4/d9/df/l72 0 2024-04-27T04:10:04.325 INFO:tasks.workunit.client.1.smithi181.stdout:2/607: write d4/f8 [5912339,44890] 0 2024-04-27T04:10:04.326 INFO:tasks.workunit.client.0.smithi162.stdout:7/301: mknod d3/d4f/c63 0 2024-04-27T04:10:04.329 INFO:tasks.workunit.client.1.smithi181.stdout:8/637: unlink d1/d6/d6f/dd2/fd8 0 2024-04-27T04:10:04.331 INFO:tasks.workunit.client.0.smithi162.stdout:3/352: fsync d1/d18/d5d/f62 0 2024-04-27T04:10:04.332 INFO:tasks.workunit.client.1.smithi181.stdout:4/512: mknod d0/d15/d76/da4/caa 0 2024-04-27T04:10:04.334 INFO:tasks.workunit.client.0.smithi162.stdout:7/302: dwrite d3/d5/d5b/d60/f61 [0,4194304] 0 2024-04-27T04:10:04.338 INFO:tasks.workunit.client.1.smithi181.stdout:3/502: rename d0/d6c/d52/d44/d56/d5b/d77 to d0/d9f 0 2024-04-27T04:10:04.338 INFO:tasks.workunit.client.0.smithi162.stdout:3/353: dwrite d1/d18/f1e [4194304,4194304] 0 2024-04-27T04:10:04.340 INFO:tasks.workunit.client.1.smithi181.stdout:3/503: write d0/d6c/d52/d44/d74/f9b [3019557,58768] 0 2024-04-27T04:10:04.341 INFO:tasks.workunit.client.1.smithi181.stdout:3/504: chown d0/d6c/d52/d44/d74/f9b 474985302 1 2024-04-27T04:10:04.342 INFO:tasks.workunit.client.1.smithi181.stdout:1/561: creat d1/d2/d5/d6/db/d2d/fb1 x:0 0 0 2024-04-27T04:10:04.343 INFO:tasks.workunit.client.1.smithi181.stdout:2/608: mkdir d4/d9/d44/da4/dc2 0 2024-04-27T04:10:04.344 INFO:tasks.workunit.client.0.smithi162.stdout:7/303: dwrite d3/f40 [0,4194304] 0 2024-04-27T04:10:04.345 INFO:tasks.workunit.client.1.smithi181.stdout:8/638: chown d1/d6/d2f/d30/d66/c70 19439 1 2024-04-27T04:10:04.349 INFO:tasks.workunit.client.1.smithi181.stdout:4/513: creat d0/d15/de/d19/fab x:0 0 0 2024-04-27T04:10:04.350 INFO:tasks.workunit.client.0.smithi162.stdout:1/331: truncate d0/fe 8067239 0 2024-04-27T04:10:04.350 INFO:tasks.workunit.client.0.smithi162.stdout:1/332: read - d0/d7/d5e/f62 zero size 2024-04-27T04:10:04.351 INFO:tasks.workunit.client.0.smithi162.stdout:1/333: stat d0/d11/d44/c48 0 2024-04-27T04:10:04.351 INFO:tasks.workunit.client.0.smithi162.stdout:1/334: chown d0/d11/d50/d53 39647246 1 2024-04-27T04:10:04.358 INFO:tasks.workunit.client.0.smithi162.stdout:1/335: dwrite d0/d7/d3c/f57 [0,4194304] 0 2024-04-27T04:10:04.359 INFO:tasks.workunit.client.1.smithi181.stdout:3/505: rmdir d0/d6c/d52/d44/d56 39 2024-04-27T04:10:04.360 INFO:tasks.workunit.client.1.smithi181.stdout:3/506: readlink d0/de/l8b 0 2024-04-27T04:10:04.361 INFO:tasks.workunit.client.1.smithi181.stdout:1/562: mknod d1/d2/d5/d6/db/d2d/d36/cb2 0 2024-04-27T04:10:04.362 INFO:tasks.workunit.client.0.smithi162.stdout:0/404: rmdir d5/dc/d37 39 2024-04-27T04:10:04.363 INFO:tasks.workunit.client.1.smithi181.stdout:1/563: chown d1/d2/d5/d6/da3/d1e/d20/d53/d56/c58 7311517 1 2024-04-27T04:10:04.363 INFO:tasks.workunit.client.1.smithi181.stdout:2/609: rmdir d4/d9/d35/d42/d86 39 2024-04-27T04:10:04.364 INFO:tasks.workunit.client.0.smithi162.stdout:1/336: dread d0/d11/d3e/f4f [0,4194304] 0 2024-04-27T04:10:04.364 INFO:tasks.workunit.client.1.smithi181.stdout:1/564: stat d1/d2/d5/d6/da3/c80 0 2024-04-27T04:10:04.365 INFO:tasks.workunit.client.1.smithi181.stdout:2/610: fdatasync d4/d9/d16/d45/d9a/d9d/faf 0 2024-04-27T04:10:04.366 INFO:tasks.workunit.client.1.smithi181.stdout:2/611: chown d4/d9/d16/la6 1813804 1 2024-04-27T04:10:04.369 INFO:tasks.workunit.client.1.smithi181.stdout:5/673: getdents d4/de 0 2024-04-27T04:10:04.371 INFO:tasks.workunit.client.1.smithi181.stdout:8/639: mkdir d1/d6/d2f/d47/dba/ddd 0 2024-04-27T04:10:04.373 INFO:tasks.workunit.client.0.smithi162.stdout:4/378: creat d2/f7d x:0 0 0 2024-04-27T04:10:04.375 INFO:tasks.workunit.client.0.smithi162.stdout:6/348: mknod d6/d1a/d34/c70 0 2024-04-27T04:10:04.377 INFO:tasks.workunit.client.1.smithi181.stdout:0/677: rename dc/d1f/d3b/d75 to dc/d1f/d6b/db9/dba/deb 0 2024-04-27T04:10:04.378 INFO:tasks.workunit.client.1.smithi181.stdout:0/678: write dc/d1f/fae [405156,3930] 0 2024-04-27T04:10:04.380 INFO:tasks.workunit.client.1.smithi181.stdout:0/679: write dc/d1f/d3b/d4a/d95/d26/d41/d6f/f7f [7269403,92152] 0 2024-04-27T04:10:04.384 INFO:tasks.workunit.client.1.smithi181.stdout:5/674: truncate d4/de/d2c/d46/d4d/d8e/d67/f50 388139 0 2024-04-27T04:10:04.384 INFO:tasks.workunit.client.1.smithi181.stdout:5/675: chown d4/da9/de7 8 1 2024-04-27T04:10:04.387 INFO:tasks.workunit.client.1.smithi181.stdout:8/640: creat d1/d6/d2f/d30/d66/dc0/dcd/fde x:0 0 0 2024-04-27T04:10:04.390 INFO:tasks.workunit.client.0.smithi162.stdout:8/368: link d0/de/d6e/c45 d0/d10/c80 0 2024-04-27T04:10:04.392 INFO:tasks.workunit.client.0.smithi162.stdout:4/379: dread d2/d25/fb [0,4194304] 0 2024-04-27T04:10:04.393 INFO:tasks.workunit.client.1.smithi181.stdout:3/507: rename d0/d6c/d52/d44/d90 to d0/d6c/d52/d44/d74/da0 0 2024-04-27T04:10:04.396 INFO:tasks.workunit.client.0.smithi162.stdout:0/405: unlink d5/dc/d1e/d41/c5e 0 2024-04-27T04:10:04.397 INFO:tasks.workunit.client.1.smithi181.stdout:0/680: creat dc/d1f/d6b/db9/dbd/fec x:0 0 0 2024-04-27T04:10:04.399 INFO:tasks.workunit.client.1.smithi181.stdout:0/681: chown dc/d1f/d3b/d4a/d95/d26/d68/l78 2817673 1 2024-04-27T04:10:04.400 INFO:tasks.workunit.client.1.smithi181.stdout:5/676: symlink d4/d10/d29/d92/dbb/lef 0 2024-04-27T04:10:04.402 INFO:tasks.workunit.client.1.smithi181.stdout:8/641: mkdir d1/d6/d2f/d40/d2a/db3/ddf 0 2024-04-27T04:10:04.404 INFO:tasks.workunit.client.0.smithi162.stdout:3/354: dwrite d1/f20 [0,4194304] 0 2024-04-27T04:10:04.405 INFO:tasks.workunit.client.0.smithi162.stdout:3/355: stat d1/d18/d5d 0 2024-04-27T04:10:04.406 INFO:tasks.workunit.client.1.smithi181.stdout:4/514: write d0/d15/d25/d2a/f7e [958899,40706] 0 2024-04-27T04:10:04.407 INFO:tasks.workunit.client.1.smithi181.stdout:8/642: dread d1/f6b [0,4194304] 0 2024-04-27T04:10:04.407 INFO:tasks.workunit.client.0.smithi162.stdout:3/356: write d1/d21/f60 [346257,103138] 0 2024-04-27T04:10:04.407 INFO:tasks.workunit.client.0.smithi162.stdout:5/360: link d8/d1a/d29/l37 d8/d1a/d29/d4a/l7c 0 2024-04-27T04:10:04.407 INFO:tasks.workunit.client.0.smithi162.stdout:5/361: write f4 [314283,29096] 0 2024-04-27T04:10:04.409 INFO:tasks.workunit.client.0.smithi162.stdout:4/380: dread d2/d54/f35 [0,4194304] 0 2024-04-27T04:10:04.409 INFO:tasks.workunit.client.0.smithi162.stdout:5/362: write d8/d1a/f59 [3530339,108119] 0 2024-04-27T04:10:04.415 INFO:tasks.workunit.client.1.smithi181.stdout:8/643: dread d1/d6/d2f/d30/d5c/f61 [0,4194304] 0 2024-04-27T04:10:04.419 INFO:tasks.workunit.client.1.smithi181.stdout:1/565: dwrite d1/d2/d5/d6/da3/d52/d92/f99 [0,4194304] 0 2024-04-27T04:10:04.420 INFO:tasks.workunit.client.1.smithi181.stdout:5/677: truncate d4/fa0 736168 0 2024-04-27T04:10:04.424 INFO:tasks.workunit.client.0.smithi162.stdout:8/369: symlink d0/d10/d2b/l81 0 2024-04-27T04:10:04.427 INFO:tasks.workunit.client.1.smithi181.stdout:2/612: rename d4/d9/d4f/d5f/l61 to d4/d9/d16/d45/d9a/lc3 0 2024-04-27T04:10:04.430 INFO:tasks.workunit.client.1.smithi181.stdout:4/515: dread d0/d15/d5c/d69/f31 [0,4194304] 0 2024-04-27T04:10:04.431 INFO:tasks.workunit.client.1.smithi181.stdout:4/516: chown d0/d15/de/d19/c7b 62 1 2024-04-27T04:10:04.433 INFO:tasks.workunit.client.1.smithi181.stdout:3/508: mkdir d0/d6c/da1 0 2024-04-27T04:10:04.433 INFO:tasks.workunit.client.1.smithi181.stdout:3/509: dread - d0/d28/d86/f8d zero size 2024-04-27T04:10:04.437 INFO:tasks.workunit.client.1.smithi181.stdout:1/566: chown d1/d2/d5/laf 213 1 2024-04-27T04:10:04.438 INFO:tasks.workunit.client.0.smithi162.stdout:7/304: dwrite d3/d4f/f4c [0,4194304] 0 2024-04-27T04:10:04.439 INFO:tasks.workunit.client.0.smithi162.stdout:3/357: truncate d1/d18/f13 941243 0 2024-04-27T04:10:04.440 INFO:tasks.workunit.client.0.smithi162.stdout:3/358: fdatasync d1/d18/f12 0 2024-04-27T04:10:04.452 INFO:tasks.workunit.client.0.smithi162.stdout:5/363: mkdir d8/d1a/d7d 0 2024-04-27T04:10:04.454 INFO:tasks.workunit.client.0.smithi162.stdout:9/345: link d3/c2e d3/d13/d2d/d3b/d72/c74 0 2024-04-27T04:10:04.457 INFO:tasks.workunit.client.0.smithi162.stdout:3/359: dread d1/d18/d23/f2b [0,4194304] 0 2024-04-27T04:10:04.458 INFO:tasks.workunit.client.0.smithi162.stdout:2/286: rename d0/d4/d17/d2b/l57 to d0/d20/l59 0 2024-04-27T04:10:04.459 INFO:tasks.workunit.client.1.smithi181.stdout:3/510: readlink d0/l5a 0 2024-04-27T04:10:04.460 INFO:tasks.workunit.client.1.smithi181.stdout:3/511: chown d0/d28/c67 12138303 1 2024-04-27T04:10:04.460 INFO:tasks.workunit.client.0.smithi162.stdout:2/287: truncate d0/d4/d17/f2f 5095902 0 2024-04-27T04:10:04.461 INFO:tasks.workunit.client.1.smithi181.stdout:3/512: chown d0/f23 115 1 2024-04-27T04:10:04.463 INFO:tasks.workunit.client.0.smithi162.stdout:3/360: dwrite d1/d18/f52 [0,4194304] 0 2024-04-27T04:10:04.465 INFO:tasks.workunit.client.0.smithi162.stdout:8/370: mknod d0/d10/d2b/c82 0 2024-04-27T04:10:04.465 INFO:tasks.workunit.client.1.smithi181.stdout:5/678: symlink d4/de/d2f/lf0 0 2024-04-27T04:10:04.466 INFO:tasks.workunit.client.0.smithi162.stdout:6/349: sync 2024-04-27T04:10:04.467 INFO:tasks.workunit.client.0.smithi162.stdout:6/350: fdatasync d6/db/f49 0 2024-04-27T04:10:04.469 INFO:tasks.workunit.client.0.smithi162.stdout:3/361: dread d1/d18/f52 [0,4194304] 0 2024-04-27T04:10:04.470 INFO:tasks.workunit.client.0.smithi162.stdout:8/371: truncate d0/d10/d2b/d2d/d2f/d6b/f7a 1021801 0 2024-04-27T04:10:04.471 INFO:tasks.workunit.client.0.smithi162.stdout:3/362: fsync d1/f3f 0 2024-04-27T04:10:04.473 INFO:tasks.workunit.client.1.smithi181.stdout:4/517: dread d0/d15/de/f34 [0,4194304] 0 2024-04-27T04:10:04.474 INFO:tasks.workunit.client.0.smithi162.stdout:3/363: dread d1/d18/d23/f2b [0,4194304] 0 2024-04-27T04:10:04.475 INFO:tasks.workunit.client.1.smithi181.stdout:0/682: rename dc/d1f/d6b/db9/dbd/fec to dc/d1f/d3b/d4a/d95/d26/d41/db0/dcc/fed 0 2024-04-27T04:10:04.476 INFO:tasks.workunit.client.0.smithi162.stdout:8/372: dread d0/d10/d2b/d2d/d2f/f54 [0,4194304] 0 2024-04-27T04:10:04.477 INFO:tasks.workunit.client.0.smithi162.stdout:8/373: dread - d0/d10/d44/f73 zero size 2024-04-27T04:10:04.480 INFO:tasks.workunit.client.0.smithi162.stdout:5/364: dread d8/d1a/d23/d40/f6e [0,4194304] 0 2024-04-27T04:10:04.481 INFO:tasks.workunit.client.1.smithi181.stdout:8/644: write d1/d6/d1a/f53 [1376434,84382] 0 2024-04-27T04:10:04.483 INFO:tasks.workunit.client.0.smithi162.stdout:7/305: write d3/d5/d9/d4b/f2e [4349854,92340] 0 2024-04-27T04:10:04.486 INFO:tasks.workunit.client.1.smithi181.stdout:5/679: mknod d4/de/d2c/d2e/d55/cf1 0 2024-04-27T04:10:04.487 INFO:tasks.workunit.client.0.smithi162.stdout:7/306: dread d3/d5/d9/f26 [0,4194304] 0 2024-04-27T04:10:04.489 INFO:tasks.workunit.client.0.smithi162.stdout:7/307: read - d3/d5/f5f zero size 2024-04-27T04:10:04.489 INFO:tasks.workunit.client.1.smithi181.stdout:1/567: symlink d1/d2/d5/d6/lb3 0 2024-04-27T04:10:04.490 INFO:tasks.workunit.client.0.smithi162.stdout:0/406: dwrite d5/f27 [0,4194304] 0 2024-04-27T04:10:04.491 INFO:tasks.workunit.client.0.smithi162.stdout:9/346: chown d3/l35 114249 1 2024-04-27T04:10:04.491 INFO:tasks.workunit.client.0.smithi162.stdout:0/407: fsync d5/dc/d1e/d41/f4f 0 2024-04-27T04:10:04.492 INFO:tasks.workunit.client.1.smithi181.stdout:4/518: unlink d0/d15/c17 0 2024-04-27T04:10:04.494 INFO:tasks.workunit.client.0.smithi162.stdout:2/288: mkdir d0/d4/d2a/d3b/d49/d5a 0 2024-04-27T04:10:04.496 INFO:tasks.workunit.client.1.smithi181.stdout:1/568: dread d1/d2/d5/d6/db/d2d/d36/f4a [0,4194304] 0 2024-04-27T04:10:04.497 INFO:tasks.workunit.client.0.smithi162.stdout:9/347: read d3/d13/f41 [767555,121287] 0 2024-04-27T04:10:04.497 INFO:tasks.workunit.client.0.smithi162.stdout:4/381: dwrite d2/da/d1a/f29 [0,4194304] 0 2024-04-27T04:10:04.498 INFO:tasks.workunit.client.1.smithi181.stdout:0/683: fsync dc/d1f/d3b/d4a/d95/d26/d41/db0/dcc/fed 0 2024-04-27T04:10:04.501 INFO:tasks.workunit.client.0.smithi162.stdout:6/351: mknod d6/d1a/d34/d43/c71 0 2024-04-27T04:10:04.503 INFO:tasks.workunit.client.0.smithi162.stdout:0/408: dwrite d5/dc/d1e/d41/f4e [0,4194304] 0 2024-04-27T04:10:04.505 INFO:tasks.workunit.client.1.smithi181.stdout:8/645: unlink d1/d6/d2f/d47/dba/fbf 0 2024-04-27T04:10:04.505 INFO:tasks.workunit.client.0.smithi162.stdout:1/337: getdents d0/d11/d3e/d5f 0 2024-04-27T04:10:04.507 INFO:tasks.workunit.client.0.smithi162.stdout:8/374: dread d0/d10/d25/f36 [0,4194304] 0 2024-04-27T04:10:04.508 INFO:tasks.workunit.client.0.smithi162.stdout:1/338: stat d0/d11/d3e/f41 0 2024-04-27T04:10:04.508 INFO:tasks.workunit.client.0.smithi162.stdout:6/352: write d6/d1a/d34/d50/f63 [580715,68446] 0 2024-04-27T04:10:04.514 INFO:tasks.workunit.client.1.smithi181.stdout:5/680: dread d4/de/d2c/d46/d4d/d8e/d67/f50 [0,4194304] 0 2024-04-27T04:10:04.514 INFO:tasks.workunit.client.1.smithi181.stdout:4/519: mknod d0/d15/d5c/cac 0 2024-04-27T04:10:04.516 INFO:tasks.workunit.client.0.smithi162.stdout:6/353: dwrite d6/d1a/f5d [0,4194304] 0 2024-04-27T04:10:04.518 INFO:tasks.workunit.client.0.smithi162.stdout:3/364: creat d1/d18/d23/d2f/f73 x:0 0 0 2024-04-27T04:10:04.519 INFO:tasks.workunit.client.1.smithi181.stdout:1/569: stat d1/d2/d5/d6/da3/d1e/f4b 0 2024-04-27T04:10:04.523 INFO:tasks.workunit.client.0.smithi162.stdout:5/365: mknod d8/d1a/d29/d4a/d5c/d63/c7e 0 2024-04-27T04:10:04.523 INFO:tasks.workunit.client.0.smithi162.stdout:5/366: dread - d8/d1a/d29/d4a/d5c/d63/f64 zero size 2024-04-27T04:10:04.524 INFO:tasks.workunit.client.1.smithi181.stdout:0/684: truncate fa 878287 0 2024-04-27T04:10:04.525 INFO:tasks.workunit.client.1.smithi181.stdout:2/613: getdents d4/d9/d4f/d5f 0 2024-04-27T04:10:04.525 INFO:tasks.workunit.client.0.smithi162.stdout:5/367: write d8/d1a/f59 [3053134,58643] 0 2024-04-27T04:10:04.526 INFO:tasks.workunit.client.0.smithi162.stdout:5/368: write d8/d2c/f33 [634813,1114] 0 2024-04-27T04:10:04.533 INFO:tasks.workunit.client.1.smithi181.stdout:5/681: creat d4/da9/ff2 x:0 0 0 2024-04-27T04:10:04.534 INFO:tasks.workunit.client.1.smithi181.stdout:4/520: truncate d0/d15/d25/d2b/f5f 207280 0 2024-04-27T04:10:04.534 INFO:tasks.workunit.client.1.smithi181.stdout:4/521: stat d0/d15/d25/d2a/d7a 0 2024-04-27T04:10:04.536 INFO:tasks.workunit.client.1.smithi181.stdout:1/570: unlink d1/d2/d5/d6/da3/d1e/f7a 0 2024-04-27T04:10:04.544 INFO:tasks.workunit.client.1.smithi181.stdout:1/571: dwrite d1/d2/d5/d6/da3/d52/d92/f93 [4194304,4194304] 0 2024-04-27T04:10:04.544 INFO:tasks.workunit.client.1.smithi181.stdout:2/614: symlink d4/d9/d4f/d79/d94/lc4 0 2024-04-27T04:10:04.544 INFO:tasks.workunit.client.0.smithi162.stdout:4/382: dwrite d2/f38 [0,4194304] 0 2024-04-27T04:10:04.551 INFO:tasks.workunit.client.1.smithi181.stdout:5/682: creat d4/da3/dbc/ff3 x:0 0 0 2024-04-27T04:10:04.557 INFO:tasks.workunit.client.1.smithi181.stdout:3/513: rename d0/lc to d0/de/la2 0 2024-04-27T04:10:04.566 INFO:tasks.workunit.client.0.smithi162.stdout:6/354: fsync d6/f30 0 2024-04-27T04:10:04.567 INFO:tasks.workunit.client.0.smithi162.stdout:6/355: chown d6/d1a/d34/d50/l5e 2 1 2024-04-27T04:10:04.568 INFO:tasks.workunit.client.1.smithi181.stdout:8/646: link d1/d6/d59/l5a d1/d6/d2f/d40/d2a/db3/ddf/le0 0 2024-04-27T04:10:04.570 INFO:tasks.workunit.client.0.smithi162.stdout:4/383: dread d2/d25/d1c/d72/f65 [0,4194304] 0 2024-04-27T04:10:04.571 INFO:tasks.workunit.client.1.smithi181.stdout:5/683: creat d4/d10/d29/d92/dbb/ff4 x:0 0 0 2024-04-27T04:10:04.572 INFO:tasks.workunit.client.1.smithi181.stdout:2/615: rename d4/d9/d4f/d5f/l7e to d4/d9/d4f/d79/d98/lc5 0 2024-04-27T04:10:04.575 INFO:tasks.workunit.client.0.smithi162.stdout:4/384: dwrite d2/f38 [4194304,4194304] 0 2024-04-27T04:10:04.579 INFO:tasks.workunit.client.0.smithi162.stdout:7/308: symlink d3/l64 0 2024-04-27T04:10:04.583 INFO:tasks.workunit.client.0.smithi162.stdout:9/348: mknod d3/d13/d2d/d3b/d72/c75 0 2024-04-27T04:10:04.588 INFO:tasks.workunit.client.0.smithi162.stdout:0/409: rename d5/dc/d1e/d56/d57/l81 to d5/dc/d37/l87 0 2024-04-27T04:10:04.592 INFO:tasks.workunit.client.0.smithi162.stdout:0/410: dwrite d5/dc/d1e/d41/f4e [0,4194304] 0 2024-04-27T04:10:04.611 INFO:tasks.workunit.client.0.smithi162.stdout:5/369: link d8/d9/d5b/f71 d8/d9/d5b/f7f 0 2024-04-27T04:10:04.611 INFO:tasks.workunit.client.0.smithi162.stdout:8/375: truncate d0/de/d20/f60 5341902 0 2024-04-27T04:10:04.611 INFO:tasks.workunit.client.0.smithi162.stdout:7/309: mkdir d3/d5/d5b/d60/d65 0 2024-04-27T04:10:04.613 INFO:tasks.workunit.client.0.smithi162.stdout:1/339: dwrite d0/d11/f20 [0,4194304] 0 2024-04-27T04:10:04.613 INFO:tasks.workunit.client.0.smithi162.stdout:7/310: chown d3/d5/d9/l49 133 1 2024-04-27T04:10:04.613 INFO:tasks.workunit.client.0.smithi162.stdout:9/349: symlink d3/l76 0 2024-04-27T04:10:04.613 INFO:tasks.workunit.client.0.smithi162.stdout:3/365: getdents d1 0 2024-04-27T04:10:04.621 INFO:tasks.workunit.client.1.smithi181.stdout:7/563: sync 2024-04-27T04:10:04.623 INFO:tasks.workunit.client.1.smithi181.stdout:0/685: write fa [1102487,84709] 0 2024-04-27T04:10:04.625 INFO:tasks.workunit.client.1.smithi181.stdout:0/686: fsync dc/d54/fab 0 2024-04-27T04:10:04.628 INFO:tasks.workunit.client.1.smithi181.stdout:8/647: rmdir d1/d6/d6f/d91 39 2024-04-27T04:10:04.629 INFO:tasks.workunit.client.1.smithi181.stdout:2/616: dread d4/d9/d4f/d79/d98/fa2 [0,4194304] 0 2024-04-27T04:10:04.630 INFO:tasks.workunit.client.1.smithi181.stdout:3/514: creat d0/d6c/d66/d99/fa3 x:0 0 0 2024-04-27T04:10:04.632 INFO:tasks.workunit.client.1.smithi181.stdout:2/617: chown d4/d9/d4f/d79 56 1 2024-04-27T04:10:04.632 INFO:tasks.workunit.client.1.smithi181.stdout:8/648: readlink d1/d6/d2f/d30/d5c/lce 0 2024-04-27T04:10:04.633 INFO:tasks.workunit.client.1.smithi181.stdout:4/522: dwrite d0/d15/de/f66 [0,4194304] 0 2024-04-27T04:10:04.635 INFO:tasks.workunit.client.1.smithi181.stdout:4/523: read d0/d15/d5c/d69/f31 [3443909,1102] 0 2024-04-27T04:10:04.636 INFO:tasks.workunit.client.1.smithi181.stdout:4/524: stat d0 0 2024-04-27T04:10:04.637 INFO:tasks.workunit.client.1.smithi181.stdout:0/687: unlink dc/d1f/d3b/d4a/d95/d26/d41/d6f/f7b 0 2024-04-27T04:10:04.641 INFO:tasks.workunit.client.1.smithi181.stdout:5/684: link d4/de/d2c/d46/d4d/d8e/d67/d71/cba d4/d10/d72/d80/d85/cf5 0 2024-04-27T04:10:04.642 INFO:tasks.workunit.client.1.smithi181.stdout:8/649: mkdir d1/d6/d2f/d30/d66/dc0/dcd/de1 0 2024-04-27T04:10:04.645 INFO:tasks.workunit.client.1.smithi181.stdout:7/564: dwrite d1/df/f5d [0,4194304] 0 2024-04-27T04:10:04.647 INFO:tasks.workunit.client.1.smithi181.stdout:4/525: creat d0/d15/d25/d2a/fad x:0 0 0 2024-04-27T04:10:04.648 INFO:tasks.workunit.client.1.smithi181.stdout:2/618: dwrite d4/f50 [4194304,4194304] 0 2024-04-27T04:10:04.649 INFO:tasks.workunit.client.1.smithi181.stdout:5/685: write d4/d10/d29/d92/fd9 [129838,52646] 0 2024-04-27T04:10:04.653 INFO:tasks.workunit.client.1.smithi181.stdout:5/686: read d4/de/d2c/d46/d4d/d8e/fdc [2210392,47908] 0 2024-04-27T04:10:04.655 INFO:tasks.workunit.client.1.smithi181.stdout:7/565: mknod d1/d22/d15/d13/d1b/cab 0 2024-04-27T04:10:04.657 INFO:tasks.workunit.client.1.smithi181.stdout:0/688: dread dc/f22 [0,4194304] 0 2024-04-27T04:10:04.657 INFO:tasks.workunit.client.1.smithi181.stdout:4/526: fsync d0/d15/f23 0 2024-04-27T04:10:04.658 INFO:tasks.workunit.client.0.smithi162.stdout:2/289: sync 2024-04-27T04:10:04.661 INFO:tasks.workunit.client.1.smithi181.stdout:2/619: creat d4/d9/d16/d45/d9a/fc6 x:0 0 0 2024-04-27T04:10:04.662 INFO:tasks.workunit.client.0.smithi162.stdout:5/370: symlink d8/d9/l80 0 2024-04-27T04:10:04.662 INFO:tasks.workunit.client.1.smithi181.stdout:8/650: getdents d1/d20 0 2024-04-27T04:10:04.663 INFO:tasks.workunit.client.1.smithi181.stdout:5/687: dwrite d4/de/d2c/d46/d4d/fb8 [0,4194304] 0 2024-04-27T04:10:04.663 INFO:tasks.workunit.client.0.smithi162.stdout:6/356: getdents d6/d1a 0 2024-04-27T04:10:04.663 INFO:tasks.workunit.client.0.smithi162.stdout:1/340: chown d0/d7/d10 12096361 1 2024-04-27T04:10:04.664 INFO:tasks.workunit.client.0.smithi162.stdout:5/371: fsync d8/f26 0 2024-04-27T04:10:04.664 INFO:tasks.workunit.client.1.smithi181.stdout:5/688: fsync d4/d10/feb 0 2024-04-27T04:10:04.668 INFO:tasks.workunit.client.1.smithi181.stdout:5/689: read d4/de/f6f [288885,101252] 0 2024-04-27T04:10:04.668 INFO:tasks.workunit.client.1.smithi181.stdout:7/566: unlink d1/d2/d35/d67/f7e 0 2024-04-27T04:10:04.669 INFO:tasks.workunit.client.0.smithi162.stdout:8/376: dread d0/d10/d2b/d2d/d2f/f54 [0,4194304] 0 2024-04-27T04:10:04.670 INFO:tasks.workunit.client.1.smithi181.stdout:7/567: write d1/d22/d15/d29/f43 [521175,94618] 0 2024-04-27T04:10:04.671 INFO:tasks.workunit.client.1.smithi181.stdout:7/568: dread - d1/df/d1a/f74 zero size 2024-04-27T04:10:04.672 INFO:tasks.workunit.client.1.smithi181.stdout:7/569: readlink d1/d22/d15/d29/l73 0 2024-04-27T04:10:04.673 INFO:tasks.workunit.client.1.smithi181.stdout:0/689: mkdir dc/d1f/d3b/d4a/d95/d26/d41/d6f/d7d/dee 0 2024-04-27T04:10:04.674 INFO:tasks.workunit.client.1.smithi181.stdout:4/527: symlink d0/d15/d25/d2a/d4c/lae 0 2024-04-27T04:10:04.676 INFO:tasks.workunit.client.0.smithi162.stdout:3/366: unlink d1/d18/c46 0 2024-04-27T04:10:04.680 INFO:tasks.workunit.client.1.smithi181.stdout:3/515: dread d0/d6c/d52/d44/d56/d5b/f8e [0,4194304] 0 2024-04-27T04:10:04.680 INFO:tasks.workunit.client.0.smithi162.stdout:2/290: symlink d0/d4/d17/d4f/l5b 0 2024-04-27T04:10:04.683 INFO:tasks.workunit.client.0.smithi162.stdout:6/357: symlink d6/d1a/d34/d50/l72 0 2024-04-27T04:10:04.683 INFO:tasks.workunit.client.0.smithi162.stdout:6/358: chown d6/d1a/f2b 57 1 2024-04-27T04:10:04.685 INFO:tasks.workunit.client.0.smithi162.stdout:7/311: dread d3/d5/d9/f17 [0,4194304] 0 2024-04-27T04:10:04.686 INFO:tasks.workunit.client.0.smithi162.stdout:7/312: chown d3/d5/d3c/d4a/c5d 80514 1 2024-04-27T04:10:04.687 INFO:tasks.workunit.client.0.smithi162.stdout:5/372: creat d8/d1a/f81 x:0 0 0 2024-04-27T04:10:04.690 INFO:tasks.workunit.client.0.smithi162.stdout:8/377: mknod d0/de/d6e/d71/c83 0 2024-04-27T04:10:04.691 INFO:tasks.workunit.client.1.smithi181.stdout:5/690: mknod d4/de/d2c/d46/d4d/d8e/d67/cf6 0 2024-04-27T04:10:04.692 INFO:tasks.workunit.client.0.smithi162.stdout:7/313: dwrite d3/d5/d9/d4b/de/f38 [0,4194304] 0 2024-04-27T04:10:04.692 INFO:tasks.workunit.client.0.smithi162.stdout:9/350: dread d3/d13/d2d/d30/f38 [0,4194304] 0 2024-04-27T04:10:04.693 INFO:tasks.workunit.client.1.smithi181.stdout:0/690: read - dc/d1f/d3b/d4a/d95/d26/fa0 zero size 2024-04-27T04:10:04.695 INFO:tasks.workunit.client.1.smithi181.stdout:5/691: dread d4/de/d2c/f3e [0,4194304] 0 2024-04-27T04:10:04.696 INFO:tasks.workunit.client.1.smithi181.stdout:4/528: rename d0/d15/d25/d2a/f46 to d0/d15/d76/da4/faf 0 2024-04-27T04:10:04.697 INFO:tasks.workunit.client.0.smithi162.stdout:6/359: mknod d6/d1a/d34/d50/c73 0 2024-04-27T04:10:04.697 INFO:tasks.workunit.client.0.smithi162.stdout:6/360: stat d6/d1a/d34 0 2024-04-27T04:10:04.697 INFO:tasks.workunit.client.0.smithi162.stdout:6/361: chown d6 402 1 2024-04-27T04:10:04.697 INFO:tasks.workunit.client.1.smithi181.stdout:2/620: symlink d4/d9/d35/lc7 0 2024-04-27T04:10:04.698 INFO:tasks.workunit.client.0.smithi162.stdout:1/341: truncate d0/d7/f27 3605556 0 2024-04-27T04:10:04.700 INFO:tasks.workunit.client.1.smithi181.stdout:3/516: mknod d0/de/d20/d6b/ca4 0 2024-04-27T04:10:04.701 INFO:tasks.workunit.client.0.smithi162.stdout:5/373: mknod d8/d1a/d23/d40/d4b/c82 0 2024-04-27T04:10:04.702 INFO:tasks.workunit.client.1.smithi181.stdout:8/651: symlink d1/d6/d2f/d30/le2 0 2024-04-27T04:10:04.704 INFO:tasks.workunit.client.0.smithi162.stdout:4/385: sync 2024-04-27T04:10:04.705 INFO:tasks.workunit.client.1.smithi181.stdout:7/570: link d1/d2/f27 d1/d22/d15/d13/d1b/fac 0 2024-04-27T04:10:04.706 INFO:tasks.workunit.client.0.smithi162.stdout:3/367: creat d1/d18/d23/d58/d63/f74 x:0 0 0 2024-04-27T04:10:04.707 INFO:tasks.workunit.client.0.smithi162.stdout:3/368: write d1/d18/f4d [363732,28892] 0 2024-04-27T04:10:04.708 INFO:tasks.workunit.client.1.smithi181.stdout:0/691: mkdir dc/d1f/d3b/d4a/def 0 2024-04-27T04:10:04.710 INFO:tasks.workunit.client.1.smithi181.stdout:5/692: rmdir d4/da3/dbc/db5 39 2024-04-27T04:10:04.711 INFO:tasks.workunit.client.0.smithi162.stdout:1/342: fsync d0/d7/d10/f26 0 2024-04-27T04:10:04.711 INFO:tasks.workunit.client.1.smithi181.stdout:5/693: readlink d4/d10/d72/db3/lda 0 2024-04-27T04:10:04.712 INFO:tasks.workunit.client.0.smithi162.stdout:5/374: dread d8/d9/f16 [0,4194304] 0 2024-04-27T04:10:04.712 INFO:tasks.workunit.client.1.smithi181.stdout:7/571: dwrite d1/d2/f9f [0,4194304] 0 2024-04-27T04:10:04.712 INFO:tasks.workunit.client.1.smithi181.stdout:4/529: fdatasync d0/d15/de/d19/d77/f72 0 2024-04-27T04:10:04.713 INFO:tasks.workunit.client.0.smithi162.stdout:8/378: getdents d0/de/d20/d7e 0 2024-04-27T04:10:04.714 INFO:tasks.workunit.client.1.smithi181.stdout:7/572: write d1/df/f5d [4111392,16046] 0 2024-04-27T04:10:04.716 INFO:tasks.workunit.client.1.smithi181.stdout:4/530: truncate d0/d15/d25/d2b/f5b 1122996 0 2024-04-27T04:10:04.718 INFO:tasks.workunit.client.1.smithi181.stdout:3/517: mknod d0/de/d20/ca5 0 2024-04-27T04:10:04.719 INFO:tasks.workunit.client.0.smithi162.stdout:9/351: getdents d3/d23/d5d 0 2024-04-27T04:10:04.719 INFO:tasks.workunit.client.0.smithi162.stdout:9/352: fsync d3/d11/d45/f55 0 2024-04-27T04:10:04.723 INFO:tasks.workunit.client.0.smithi162.stdout:3/369: rmdir d1/d21/d47 39 2024-04-27T04:10:04.732 INFO:tasks.workunit.client.1.smithi181.stdout:0/692: dwrite dc/d1f/d3b/d4a/d95/f51 [0,4194304] 0 2024-04-27T04:10:04.737 INFO:tasks.workunit.client.1.smithi181.stdout:2/621: dread d4/d9/d4f/f3d [0,4194304] 0 2024-04-27T04:10:04.738 INFO:tasks.workunit.client.0.smithi162.stdout:8/379: rmdir d0/de/d6e/d71 39 2024-04-27T04:10:04.738 INFO:tasks.workunit.client.1.smithi181.stdout:2/622: write d4/d9/d44/f97 [815224,72060] 0 2024-04-27T04:10:04.739 INFO:tasks.workunit.client.0.smithi162.stdout:8/380: readlink d0/d10/d2b/d2d/d2f/d39/l5e 0 2024-04-27T04:10:04.744 INFO:tasks.workunit.client.1.smithi181.stdout:8/652: rename d1/d6/d2f/d40/f6e to d1/d6/d2f/d30/d66/dc0/dcd/de1/fe3 0 2024-04-27T04:10:04.745 INFO:tasks.workunit.client.0.smithi162.stdout:2/291: link d0/c13 d0/d4/d17/d4f/c5c 0 2024-04-27T04:10:04.749 INFO:tasks.workunit.client.0.smithi162.stdout:1/343: creat d0/d7/d3c/d6e/f71 x:0 0 0 2024-04-27T04:10:04.749 INFO:tasks.workunit.client.0.smithi162.stdout:4/386: write d2/f9 [1621448,45634] 0 2024-04-27T04:10:04.750 INFO:tasks.workunit.client.1.smithi181.stdout:2/623: symlink d4/d9/d16/d45/d9a/d9d/lc8 0 2024-04-27T04:10:04.752 INFO:tasks.workunit.client.0.smithi162.stdout:5/375: link d8/d1a/d23/f50 d8/d1a/f83 0 2024-04-27T04:10:04.753 INFO:tasks.workunit.client.0.smithi162.stdout:5/376: fsync d8/d2c/f33 0 2024-04-27T04:10:04.754 INFO:tasks.workunit.client.0.smithi162.stdout:8/381: rename d0/d10/d2b/d2e/l3a to d0/d10/d44/d70/l84 0 2024-04-27T04:10:04.756 INFO:tasks.workunit.client.0.smithi162.stdout:2/292: dread d0/d4/f16 [0,4194304] 0 2024-04-27T04:10:04.756 INFO:tasks.workunit.client.0.smithi162.stdout:8/382: chown d0/d10/d44/c6f 70 1 2024-04-27T04:10:04.757 INFO:tasks.workunit.client.0.smithi162.stdout:1/344: dwrite d0/d7/d3c/d54/f65 [0,4194304] 0 2024-04-27T04:10:04.757 INFO:tasks.workunit.client.1.smithi181.stdout:0/693: rename dc/d1f/d3b/d4a/l56 to dc/d1f/d3b/dd0/lf0 0 2024-04-27T04:10:04.759 INFO:tasks.workunit.client.0.smithi162.stdout:6/362: getdents d6/d1a/d34/d3a 0 2024-04-27T04:10:04.759 INFO:tasks.workunit.client.1.smithi181.stdout:8/653: readlink d1/d6/d1a/l68 0 2024-04-27T04:10:04.759 INFO:tasks.workunit.client.0.smithi162.stdout:6/363: read - d6/db/d32/f38 zero size 2024-04-27T04:10:04.760 INFO:tasks.workunit.client.0.smithi162.stdout:6/364: fdatasync d6/db/f49 0 2024-04-27T04:10:04.761 INFO:tasks.workunit.client.1.smithi181.stdout:9/542: sync 2024-04-27T04:10:04.764 INFO:tasks.workunit.client.0.smithi162.stdout:2/293: dwrite d0/d4/d17/f3c [0,4194304] 0 2024-04-27T04:10:04.765 INFO:tasks.workunit.client.0.smithi162.stdout:2/294: readlink d0/d4/d2a/d3b/l55 0 2024-04-27T04:10:04.766 INFO:tasks.workunit.client.1.smithi181.stdout:7/573: creat d1/d22/d15/d13/fad x:0 0 0 2024-04-27T04:10:04.767 INFO:tasks.workunit.client.0.smithi162.stdout:2/295: write d0/f1 [311292,23309] 0 2024-04-27T04:10:04.767 INFO:tasks.workunit.client.1.smithi181.stdout:7/574: readlink d1/l2e 0 2024-04-27T04:10:04.770 INFO:tasks.workunit.client.1.smithi181.stdout:2/624: fdatasync d4/f30 0 2024-04-27T04:10:04.775 INFO:tasks.workunit.client.1.smithi181.stdout:4/531: creat d0/d15/d25/d2b/fb0 x:0 0 0 2024-04-27T04:10:04.778 INFO:tasks.workunit.client.1.smithi181.stdout:3/518: creat d0/d6c/d52/d44/d56/fa6 x:0 0 0 2024-04-27T04:10:04.780 INFO:tasks.workunit.client.0.smithi162.stdout:8/383: symlink d0/d10/d2b/d2d/d2f/l85 0 2024-04-27T04:10:04.781 INFO:tasks.workunit.client.0.smithi162.stdout:6/365: dread d6/d1a/f18 [4194304,4194304] 0 2024-04-27T04:10:04.781 INFO:tasks.workunit.client.1.smithi181.stdout:9/543: dread d0/d40/fb2 [0,4194304] 0 2024-04-27T04:10:04.782 INFO:tasks.workunit.client.0.smithi162.stdout:1/345: rename d0/d7/d3c/c4a to d0/d7/d3c/d6e/c72 0 2024-04-27T04:10:04.783 INFO:tasks.workunit.client.1.smithi181.stdout:0/694: symlink dc/d1f/d3b/d4a/lf1 0 2024-04-27T04:10:04.786 INFO:tasks.workunit.client.1.smithi181.stdout:5/694: getdents d4/d10/d29 0 2024-04-27T04:10:04.787 INFO:tasks.workunit.client.1.smithi181.stdout:5/695: truncate d4/d10/fdb 678792 0 2024-04-27T04:10:04.787 INFO:tasks.workunit.client.1.smithi181.stdout:7/575: rename d1/d2/d1c/d37/c50 to d1/df/cae 0 2024-04-27T04:10:04.788 INFO:tasks.workunit.client.0.smithi162.stdout:1/346: dwrite d0/d7/d10/f26 [4194304,4194304] 0 2024-04-27T04:10:04.788 INFO:tasks.workunit.client.0.smithi162.stdout:5/377: dread d8/d1a/f3e [0,4194304] 0 2024-04-27T04:10:04.790 INFO:tasks.workunit.client.1.smithi181.stdout:2/625: truncate d4/d9/d4f/d79/d94/f53 6695 0 2024-04-27T04:10:04.796 INFO:tasks.workunit.client.1.smithi181.stdout:4/532: mkdir d0/d15/d76/db1 0 2024-04-27T04:10:04.798 INFO:tasks.workunit.client.0.smithi162.stdout:6/366: dread d6/d1a/d34/f3e [0,4194304] 0 2024-04-27T04:10:04.798 INFO:tasks.workunit.client.0.smithi162.stdout:0/411: sync 2024-04-27T04:10:04.798 INFO:tasks.workunit.client.0.smithi162.stdout:6/367: readlink d6/db/l2f 0 2024-04-27T04:10:04.800 INFO:tasks.workunit.client.0.smithi162.stdout:6/368: dread d6/d1a/d34/d43/f5c [0,4194304] 0 2024-04-27T04:10:04.802 INFO:tasks.workunit.client.1.smithi181.stdout:0/695: rmdir dc/d1f/d3b/d4a/d95/d26 39 2024-04-27T04:10:04.802 INFO:tasks.workunit.client.1.smithi181.stdout:0/696: read dc/f22 [1928608,78067] 0 2024-04-27T04:10:04.805 INFO:tasks.workunit.client.1.smithi181.stdout:8/654: mkdir d1/d6/d2f/d30/de4 0 2024-04-27T04:10:04.805 INFO:tasks.workunit.client.1.smithi181.stdout:8/655: fdatasync d1/d6/d6f/fbb 0 2024-04-27T04:10:04.807 INFO:tasks.workunit.client.0.smithi162.stdout:2/296: mkdir d0/d4/d17/d4f/d56/d5d 0 2024-04-27T04:10:04.809 INFO:tasks.workunit.client.1.smithi181.stdout:7/576: mkdir d1/d2/d35/d8b/daf 0 2024-04-27T04:10:04.813 INFO:tasks.workunit.client.0.smithi162.stdout:8/384: write d0/de/f7b [1922262,122814] 0 2024-04-27T04:10:04.813 INFO:tasks.workunit.client.0.smithi162.stdout:9/353: link d3/d13/d2d/d30/c42 d3/d39/c77 0 2024-04-27T04:10:04.814 INFO:tasks.workunit.client.0.smithi162.stdout:9/354: readlink d3/d13/d2d/l71 0 2024-04-27T04:10:04.814 INFO:tasks.workunit.client.1.smithi181.stdout:9/544: write d0/d44/d4b/f62 [2117077,40325] 0 2024-04-27T04:10:04.814 INFO:tasks.workunit.client.1.smithi181.stdout:9/545: chown d0/d44 2960 1 2024-04-27T04:10:04.817 INFO:tasks.workunit.client.0.smithi162.stdout:0/412: mknod d5/d60/c88 0 2024-04-27T04:10:04.818 INFO:tasks.workunit.client.1.smithi181.stdout:8/656: rmdir d1/d6/d2f/d47 39 2024-04-27T04:10:04.818 INFO:tasks.workunit.client.0.smithi162.stdout:8/385: dwrite d0/d10/d44/f47 [0,4194304] 0 2024-04-27T04:10:04.820 INFO:tasks.workunit.client.1.smithi181.stdout:7/577: creat d1/d2/d1c/d37/d91/fb0 x:0 0 0 2024-04-27T04:10:04.821 INFO:tasks.workunit.client.0.smithi162.stdout:2/297: creat d0/d24/f5e x:0 0 0 2024-04-27T04:10:04.821 INFO:tasks.workunit.client.1.smithi181.stdout:7/578: truncate d1/d22/d15/f2a 8553611 0 2024-04-27T04:10:04.822 INFO:tasks.workunit.client.0.smithi162.stdout:1/347: rename d0/lc to d0/d11/d50/l73 0 2024-04-27T04:10:04.823 INFO:tasks.workunit.client.1.smithi181.stdout:7/579: write d1/d2/d35/d67/f96 [902392,69348] 0 2024-04-27T04:10:04.824 INFO:tasks.workunit.client.1.smithi181.stdout:2/626: getdents d4/d9/d16/d82/d7d/dbe 0 2024-04-27T04:10:04.824 INFO:tasks.workunit.client.1.smithi181.stdout:7/580: chown d1/d22/d15/d13/d1b 921115 1 2024-04-27T04:10:04.826 INFO:tasks.workunit.client.1.smithi181.stdout:9/546: mkdir d0/d44/d4b/d3d/dbd 0 2024-04-27T04:10:04.828 INFO:tasks.workunit.client.0.smithi162.stdout:0/413: rmdir d5/dc/d1e/d41 39 2024-04-27T04:10:04.829 INFO:tasks.workunit.client.0.smithi162.stdout:0/414: chown d5/d60/f72 62661 1 2024-04-27T04:10:04.830 INFO:tasks.workunit.client.1.smithi181.stdout:5/696: creat d4/de/d2c/d2e/ff7 x:0 0 0 2024-04-27T04:10:04.831 INFO:tasks.workunit.client.0.smithi162.stdout:9/355: dwrite d3/d13/d2d/d30/f38 [0,4194304] 0 2024-04-27T04:10:04.831 INFO:tasks.workunit.client.0.smithi162.stdout:6/369: dread d6/d1a/f18 [0,4194304] 0 2024-04-27T04:10:04.832 INFO:tasks.workunit.client.0.smithi162.stdout:2/298: unlink d0/fe 0 2024-04-27T04:10:04.832 INFO:tasks.workunit.client.0.smithi162.stdout:2/299: chown d0/d20/d2e 1 1 2024-04-27T04:10:04.833 INFO:tasks.workunit.client.1.smithi181.stdout:8/657: creat d1/d6/d2f/d30/d66/fe5 x:0 0 0 2024-04-27T04:10:04.833 INFO:tasks.workunit.client.0.smithi162.stdout:9/356: write d3/d13/d15/f19 [1871790,50845] 0 2024-04-27T04:10:04.833 INFO:tasks.workunit.client.0.smithi162.stdout:6/370: chown d6/d1a/d34/d50/f63 22709370 1 2024-04-27T04:10:04.834 INFO:tasks.workunit.client.0.smithi162.stdout:2/300: fsync d0/d4/d2a/d41/f4a 0 2024-04-27T04:10:04.834 INFO:tasks.workunit.client.0.smithi162.stdout:1/348: mkdir d0/d11/d50/d53/d74 0 2024-04-27T04:10:04.835 INFO:tasks.workunit.client.0.smithi162.stdout:2/301: write d0/d4/d2a/d41/f4a [581306,14867] 0 2024-04-27T04:10:04.837 INFO:tasks.workunit.client.1.smithi181.stdout:2/627: creat d4/d9/df/d14/fc9 x:0 0 0 2024-04-27T04:10:04.839 INFO:tasks.workunit.client.0.smithi162.stdout:8/386: mkdir d0/de/d6e/d71/d86 0 2024-04-27T04:10:04.839 INFO:tasks.workunit.client.1.smithi181.stdout:7/581: symlink d1/d2/d1c/d37/d91/lb1 0 2024-04-27T04:10:04.840 INFO:tasks.workunit.client.0.smithi162.stdout:2/302: dwrite d0/f6 [0,4194304] 0 2024-04-27T04:10:04.842 INFO:tasks.workunit.client.1.smithi181.stdout:0/697: creat dc/d1f/d3b/d4a/d95/ff2 x:0 0 0 2024-04-27T04:10:04.844 INFO:tasks.workunit.client.1.smithi181.stdout:8/658: truncate d1/d6/f44 3821695 0 2024-04-27T04:10:04.844 INFO:tasks.workunit.client.0.smithi162.stdout:6/371: write d6/f26 [3694024,129788] 0 2024-04-27T04:10:04.847 INFO:tasks.workunit.client.0.smithi162.stdout:1/349: mkdir d0/d7/d10/d75 0 2024-04-27T04:10:04.849 INFO:tasks.workunit.client.0.smithi162.stdout:7/314: sync 2024-04-27T04:10:04.850 INFO:tasks.workunit.client.1.smithi181.stdout:4/533: getdents d0/d15/d5c 0 2024-04-27T04:10:04.850 INFO:tasks.workunit.client.0.smithi162.stdout:0/415: dread d5/dc/f14 [4194304,4194304] 0 2024-04-27T04:10:04.851 INFO:tasks.workunit.client.0.smithi162.stdout:5/378: getdents d8/d1a 0 2024-04-27T04:10:04.852 INFO:tasks.workunit.client.1.smithi181.stdout:0/698: dwrite dc/d1f/d3b/f77 [4194304,4194304] 0 2024-04-27T04:10:04.852 INFO:tasks.workunit.client.0.smithi162.stdout:5/379: write d8/d9/d5b/f79 [570543,15844] 0 2024-04-27T04:10:04.854 INFO:tasks.workunit.client.0.smithi162.stdout:8/387: mkdir d0/d10/d25/d7d/d87 0 2024-04-27T04:10:04.857 INFO:tasks.workunit.client.0.smithi162.stdout:2/303: symlink d0/d4/d2a/l5f 0 2024-04-27T04:10:04.858 INFO:tasks.workunit.client.1.smithi181.stdout:5/697: symlink d4/d10/dec/lf8 0 2024-04-27T04:10:04.859 INFO:tasks.workunit.client.0.smithi162.stdout:2/304: stat d0/d4/d17 0 2024-04-27T04:10:04.864 INFO:tasks.workunit.client.1.smithi181.stdout:8/659: chown d1/d6/d1a/d25/l3e 104747 1 2024-04-27T04:10:04.865 INFO:tasks.workunit.client.1.smithi181.stdout:4/534: fdatasync d0/d15/f23 0 2024-04-27T04:10:04.865 INFO:tasks.workunit.client.1.smithi181.stdout:8/660: readlink d1/d6/d88/l96 0 2024-04-27T04:10:04.866 INFO:tasks.workunit.client.1.smithi181.stdout:5/698: dwrite d4/d10/d29/f84 [0,4194304] 0 2024-04-27T04:10:04.868 INFO:tasks.workunit.client.1.smithi181.stdout:4/535: write d0/d15/d25/d2a/fad [974964,46204] 0 2024-04-27T04:10:04.879 INFO:tasks.workunit.client.1.smithi181.stdout:6/453: sync 2024-04-27T04:10:04.879 INFO:tasks.workunit.client.1.smithi181.stdout:1/572: sync 2024-04-27T04:10:04.889 INFO:tasks.workunit.client.0.smithi162.stdout:1/350: symlink d0/d7/d10/d35/l76 0 2024-04-27T04:10:04.890 INFO:tasks.workunit.client.0.smithi162.stdout:6/372: dread d6/d1a/f1d [0,4194304] 0 2024-04-27T04:10:04.945 INFO:tasks.workunit.client.0.smithi162.stdout:4/387: sync 2024-04-27T04:10:04.946 INFO:tasks.workunit.client.0.smithi162.stdout:3/370: sync 2024-04-27T04:10:04.946 INFO:tasks.workunit.client.0.smithi162.stdout:4/388: dread - d2/f7d zero size 2024-04-27T04:10:04.946 INFO:tasks.workunit.client.0.smithi162.stdout:4/389: chown d2/d54/d7a 162729 1 2024-04-27T04:10:04.946 INFO:tasks.workunit.client.0.smithi162.stdout:4/390: stat d2/d25/d1c/d24/l34 0 2024-04-27T04:10:04.957 INFO:tasks.workunit.client.0.smithi162.stdout:5/380: truncate d8/f12 8395379 0 2024-04-27T04:10:04.962 INFO:tasks.workunit.client.0.smithi162.stdout:8/388: creat d0/d10/d25/f88 x:0 0 0 2024-04-27T04:10:04.963 INFO:tasks.workunit.client.0.smithi162.stdout:8/389: read - d0/d10/d44/d70/f7f zero size 2024-04-27T04:10:04.968 INFO:tasks.workunit.client.0.smithi162.stdout:8/390: dread d0/d10/d2b/d2d/d2f/d6b/f7a [0,4194304] 0 2024-04-27T04:10:04.969 INFO:tasks.workunit.client.0.smithi162.stdout:8/391: read - d0/d10/d2b/d2e/f40 zero size 2024-04-27T04:10:04.970 INFO:tasks.workunit.client.0.smithi162.stdout:8/392: fdatasync d0/d10/d2b/d2d/d2f/d42/f48 0 2024-04-27T04:10:04.972 INFO:tasks.workunit.client.0.smithi162.stdout:1/351: readlink d0/d11/d50/l52 0 2024-04-27T04:10:04.973 INFO:tasks.workunit.client.0.smithi162.stdout:1/352: stat d0/d11/d50/d53 0 2024-04-27T04:10:04.975 INFO:tasks.workunit.client.0.smithi162.stdout:8/393: dwrite d0/d10/d2b/d2d/d2f/d39/f59 [0,4194304] 0 2024-04-27T04:10:04.976 INFO:tasks.workunit.client.0.smithi162.stdout:7/315: sync 2024-04-27T04:10:04.977 INFO:tasks.workunit.client.1.smithi181.stdout:1/573: mkdir d1/d2/d5/d6/da3/d1e/d48/db4 0 2024-04-27T04:10:04.977 INFO:tasks.workunit.client.0.smithi162.stdout:7/316: chown d3/d5/d3c 124306327 1 2024-04-27T04:10:04.979 INFO:tasks.workunit.client.1.smithi181.stdout:1/574: chown d1/d2/d5/d6/da3/d52/d92/f93 82319 1 2024-04-27T04:10:04.980 INFO:tasks.workunit.client.1.smithi181.stdout:1/575: stat d1/d2/d5/d6/da3/d52/f68 0 2024-04-27T04:10:04.983 INFO:tasks.workunit.client.0.smithi162.stdout:4/391: symlink d2/da/dd/l7e 0 2024-04-27T04:10:04.990 INFO:tasks.workunit.client.0.smithi162.stdout:4/392: dwrite d2/d25/d1c/d39/f46 [4194304,4194304] 0 2024-04-27T04:10:04.998 INFO:tasks.workunit.client.0.smithi162.stdout:2/305: getdents d0/d4/d17/d4f/d56/d5d 0 2024-04-27T04:10:05.002 INFO:tasks.workunit.client.0.smithi162.stdout:9/357: getdents d3/d13 0 2024-04-27T04:10:05.003 INFO:tasks.workunit.client.1.smithi181.stdout:1/576: unlink d1/d2/d5/f84 0 2024-04-27T04:10:05.004 INFO:tasks.workunit.client.0.smithi162.stdout:2/306: dwrite d0/d4/f3a [0,4194304] 0 2024-04-27T04:10:05.017 INFO:tasks.workunit.client.0.smithi162.stdout:6/373: mkdir d6/d1a/d34/d3a/d60/d74 0 2024-04-27T04:10:05.017 INFO:tasks.workunit.client.0.smithi162.stdout:6/374: chown d6/d1a/d34/d3a/d68/d6c/f4f 0 1 2024-04-27T04:10:05.018 INFO:tasks.workunit.client.1.smithi181.stdout:1/577: rmdir d1/d2/d5/d6/db/d2d/d36 39 2024-04-27T04:10:05.018 INFO:tasks.workunit.client.0.smithi162.stdout:6/375: chown d6/db/l27 263 1 2024-04-27T04:10:05.019 INFO:tasks.workunit.client.0.smithi162.stdout:8/394: write d0/d10/d25/f36 [7611045,19319] 0 2024-04-27T04:10:05.036 INFO:tasks.workunit.client.0.smithi162.stdout:8/395: creat d0/d10/d44/d70/f89 x:0 0 0 2024-04-27T04:10:05.036 INFO:tasks.workunit.client.0.smithi162.stdout:2/307: fdatasync d0/d24/f38 0 2024-04-27T04:10:05.038 INFO:tasks.workunit.client.0.smithi162.stdout:6/376: rename d6/c41 to d6/d1a/d34/d50/c75 0 2024-04-27T04:10:05.039 INFO:tasks.workunit.client.0.smithi162.stdout:7/317: link d3/c54 d3/d4f/c66 0 2024-04-27T04:10:05.039 INFO:tasks.workunit.client.0.smithi162.stdout:6/377: stat d6/db/d32/f5b 0 2024-04-27T04:10:05.042 INFO:tasks.workunit.client.0.smithi162.stdout:2/308: chown d0/c1c 2928 1 2024-04-27T04:10:05.045 INFO:tasks.workunit.client.0.smithi162.stdout:2/309: symlink d0/d4/d2a/d3b/l60 0 2024-04-27T04:10:05.053 INFO:tasks.workunit.client.0.smithi162.stdout:2/310: dwrite d0/d4/f16 [0,4194304] 0 2024-04-27T04:10:05.054 INFO:tasks.workunit.client.0.smithi162.stdout:2/311: chown d0/l34 520930935 1 2024-04-27T04:10:05.057 INFO:tasks.workunit.client.0.smithi162.stdout:2/312: mkdir d0/d20/d2e/d61 0 2024-04-27T04:10:05.058 INFO:tasks.workunit.client.0.smithi162.stdout:6/378: dread d6/f11 [0,4194304] 0 2024-04-27T04:10:05.061 INFO:tasks.workunit.client.0.smithi162.stdout:2/313: unlink d0/d4/d17/d4f/l5b 0 2024-04-27T04:10:05.064 INFO:tasks.workunit.client.0.smithi162.stdout:6/379: mkdir d6/db/d76 0 2024-04-27T04:10:05.066 INFO:tasks.workunit.client.0.smithi162.stdout:6/380: mknod d6/d1a/d34/d43/c77 0 2024-04-27T04:10:05.067 INFO:tasks.workunit.client.0.smithi162.stdout:2/314: dwrite d0/f44 [0,4194304] 0 2024-04-27T04:10:05.069 INFO:tasks.workunit.client.0.smithi162.stdout:6/381: creat d6/d1a/d34/d3a/d68/f78 x:0 0 0 2024-04-27T04:10:05.076 INFO:tasks.workunit.client.0.smithi162.stdout:9/358: mknod d3/d11/c78 0 2024-04-27T04:10:05.078 INFO:tasks.workunit.client.0.smithi162.stdout:9/359: mknod d3/d11/c79 0 2024-04-27T04:10:05.079 INFO:tasks.workunit.client.0.smithi162.stdout:9/360: write d3/d13/d2d/d3b/d54/f6c [631243,67971] 0 2024-04-27T04:10:05.081 INFO:tasks.workunit.client.0.smithi162.stdout:9/361: write d3/d23/f22 [1996265,97943] 0 2024-04-27T04:10:05.083 INFO:tasks.workunit.client.0.smithi162.stdout:9/362: readlink d3/l7 0 2024-04-27T04:10:05.088 INFO:tasks.workunit.client.0.smithi162.stdout:9/363: dwrite d3/d23/f73 [0,4194304] 0 2024-04-27T04:10:05.089 INFO:tasks.workunit.client.1.smithi181.stdout:8/661: mknod d1/d6/d6f/ce6 0 2024-04-27T04:10:05.092 INFO:tasks.workunit.client.0.smithi162.stdout:9/364: unlink d3/d13/d2d/d30/l63 0 2024-04-27T04:10:05.094 INFO:tasks.workunit.client.1.smithi181.stdout:8/662: symlink d1/d20/le7 0 2024-04-27T04:10:05.095 INFO:tasks.workunit.client.0.smithi162.stdout:2/315: dread d0/f1 [4194304,4194304] 0 2024-04-27T04:10:05.097 INFO:tasks.workunit.client.1.smithi181.stdout:2/628: dwrite d4/d9/df/d14/f24 [0,4194304] 0 2024-04-27T04:10:05.100 INFO:tasks.workunit.client.0.smithi162.stdout:9/365: dwrite d3/f4 [0,4194304] 0 2024-04-27T04:10:05.101 INFO:tasks.workunit.client.0.smithi162.stdout:9/366: readlink d3/d11/l25 0 2024-04-27T04:10:05.105 INFO:tasks.workunit.client.1.smithi181.stdout:2/629: creat d4/d9/d16/d82/d7d/dbe/fca x:0 0 0 2024-04-27T04:10:05.106 INFO:tasks.workunit.client.1.smithi181.stdout:2/630: chown d4/d9/d4f/f3d 10 1 2024-04-27T04:10:05.108 INFO:tasks.workunit.client.0.smithi162.stdout:9/367: dwrite d3/d13/d15/f2f [4194304,4194304] 0 2024-04-27T04:10:05.114 INFO:tasks.workunit.client.1.smithi181.stdout:2/631: read d4/d9/d4f/f5b [5124452,122766] 0 2024-04-27T04:10:05.125 INFO:tasks.workunit.client.0.smithi162.stdout:3/371: dwrite d1/d18/d23/d34/f3c [0,4194304] 0 2024-04-27T04:10:05.126 INFO:tasks.workunit.client.0.smithi162.stdout:3/372: truncate d1/d21/f60 1489212 0 2024-04-27T04:10:05.127 INFO:tasks.workunit.client.0.smithi162.stdout:3/373: fdatasync d1/d18/d23/f28 0 2024-04-27T04:10:05.129 INFO:tasks.workunit.client.1.smithi181.stdout:6/454: creat d7/f9d x:0 0 0 2024-04-27T04:10:05.131 INFO:tasks.workunit.client.1.smithi181.stdout:8/663: dread d1/d6/d2f/d30/f38 [0,4194304] 0 2024-04-27T04:10:05.131 INFO:tasks.workunit.client.1.smithi181.stdout:4/536: write d0/d15/d25/d2b/f5f [321213,102752] 0 2024-04-27T04:10:05.132 INFO:tasks.workunit.client.1.smithi181.stdout:6/455: fsync d7/dc/d1e/f83 0 2024-04-27T04:10:05.135 INFO:tasks.workunit.client.1.smithi181.stdout:4/537: write d0/d15/d25/d2b/d6d/f71 [142822,32202] 0 2024-04-27T04:10:05.136 INFO:tasks.workunit.client.1.smithi181.stdout:1/578: rename d1/d2/d5 to d1/dad/db5 0 2024-04-27T04:10:05.136 INFO:tasks.workunit.client.1.smithi181.stdout:5/699: dwrite d4/d10/d72/d80/f8d [0,4194304] 0 2024-04-27T04:10:05.142 INFO:tasks.workunit.client.0.smithi162.stdout:3/374: link d1/d18/c54 d1/d37/c75 0 2024-04-27T04:10:05.142 INFO:tasks.workunit.client.1.smithi181.stdout:7/582: sync 2024-04-27T04:10:05.142 INFO:tasks.workunit.client.1.smithi181.stdout:3/519: sync 2024-04-27T04:10:05.144 INFO:tasks.workunit.client.1.smithi181.stdout:6/456: creat d7/dc/d1c/d88/f9e x:0 0 0 2024-04-27T04:10:05.145 INFO:tasks.workunit.client.1.smithi181.stdout:7/583: dread - d1/d2/d1c/d37/d91/fb0 zero size 2024-04-27T04:10:05.145 INFO:tasks.workunit.client.1.smithi181.stdout:8/664: dwrite d1/d6/d1a/d25/f5f [0,4194304] 0 2024-04-27T04:10:05.146 INFO:tasks.workunit.client.1.smithi181.stdout:4/538: creat d0/d15/d5c/fb2 x:0 0 0 2024-04-27T04:10:05.148 INFO:tasks.workunit.client.1.smithi181.stdout:8/665: fsync d1/d6/d1a/f6a 0 2024-04-27T04:10:05.148 INFO:tasks.workunit.client.1.smithi181.stdout:7/584: read - d1/d22/d15/d13/fad zero size 2024-04-27T04:10:05.149 INFO:tasks.workunit.client.1.smithi181.stdout:4/539: write d0/d15/d25/d2a/fad [443396,66461] 0 2024-04-27T04:10:05.149 INFO:tasks.workunit.client.1.smithi181.stdout:5/700: creat d4/de/d9d/ff9 x:0 0 0 2024-04-27T04:10:05.151 INFO:tasks.workunit.client.1.smithi181.stdout:2/632: rename d4/d9/d44 to d4/d9/d16/d82/d7d/db7/dcb 0 2024-04-27T04:10:05.151 INFO:tasks.workunit.client.1.smithi181.stdout:5/701: read - d4/d10/d72/fe3 zero size 2024-04-27T04:10:05.155 INFO:tasks.workunit.client.1.smithi181.stdout:8/666: symlink d1/d6/da6/le8 0 2024-04-27T04:10:05.156 INFO:tasks.workunit.client.1.smithi181.stdout:8/667: stat d1/d6/d88 0 2024-04-27T04:10:05.157 INFO:tasks.workunit.client.0.smithi162.stdout:3/375: dread d1/d18/f1a [0,4194304] 0 2024-04-27T04:10:05.158 INFO:tasks.workunit.client.1.smithi181.stdout:7/585: mkdir d1/d22/d15/db2 0 2024-04-27T04:10:05.158 INFO:tasks.workunit.client.0.smithi162.stdout:3/376: fdatasync d1/d18/d23/d34/f3c 0 2024-04-27T04:10:05.158 INFO:tasks.workunit.client.0.smithi162.stdout:0/416: mknod d5/dc/d1e/c89 0 2024-04-27T04:10:05.159 INFO:tasks.workunit.client.1.smithi181.stdout:1/579: mknod d1/dad/db5/d6/da3/d1e/d48/db4/cb6 0 2024-04-27T04:10:05.161 INFO:tasks.workunit.client.1.smithi181.stdout:3/520: mkdir d0/d6c/d83/d98/da7 0 2024-04-27T04:10:05.162 INFO:tasks.workunit.client.1.smithi181.stdout:1/580: stat d1/dad/db5/d6/da3/d1e/d20/f9d 0 2024-04-27T04:10:05.163 INFO:tasks.workunit.client.1.smithi181.stdout:5/702: dwrite d4/de/d2c/d2e/d55/f3f [4194304,4194304] 0 2024-04-27T04:10:05.164 INFO:tasks.workunit.client.0.smithi162.stdout:1/353: write d0/d7/f1f [81237,66409] 0 2024-04-27T04:10:05.166 INFO:tasks.workunit.client.1.smithi181.stdout:4/540: symlink d0/d15/d25/d2a/d4c/lb3 0 2024-04-27T04:10:05.167 INFO:tasks.workunit.client.1.smithi181.stdout:4/541: dread - d0/d15/d5c/fb2 zero size 2024-04-27T04:10:05.169 INFO:tasks.workunit.client.1.smithi181.stdout:6/457: rename d7/fa to d7/dc/d1c/d34/d3f/f9f 0 2024-04-27T04:10:05.172 INFO:tasks.workunit.client.1.smithi181.stdout:2/633: creat d4/d9/d4f/d79/d98/fcc x:0 0 0 2024-04-27T04:10:05.174 INFO:tasks.workunit.client.0.smithi162.stdout:1/354: chown d0/d7/la 225576510 1 2024-04-27T04:10:05.174 INFO:tasks.workunit.client.0.smithi162.stdout:8/396: write d0/f7 [2773945,124483] 0 2024-04-27T04:10:05.175 INFO:tasks.workunit.client.0.smithi162.stdout:1/355: write d0/f1b [2983405,110630] 0 2024-04-27T04:10:05.178 INFO:tasks.workunit.client.1.smithi181.stdout:4/542: dread d0/d15/d25/d2a/d36/f42 [4194304,4194304] 0 2024-04-27T04:10:05.180 INFO:tasks.workunit.client.0.smithi162.stdout:8/397: mkdir d0/d10/d2b/d2e/d8a 0 2024-04-27T04:10:05.181 INFO:tasks.workunit.client.0.smithi162.stdout:8/398: write d0/d10/d25/f88 [633552,43511] 0 2024-04-27T04:10:05.183 INFO:tasks.workunit.client.1.smithi181.stdout:4/543: fdatasync d0/d15/d25/d40/f4f 0 2024-04-27T04:10:05.185 INFO:tasks.workunit.client.1.smithi181.stdout:3/521: dread d0/f57 [0,4194304] 0 2024-04-27T04:10:05.186 INFO:tasks.workunit.client.1.smithi181.stdout:7/586: rename d1/d2/f19 to d1/d2/d1c/d56/fb3 0 2024-04-27T04:10:05.188 INFO:tasks.workunit.client.1.smithi181.stdout:6/458: fdatasync d7/d31/f75 0 2024-04-27T04:10:05.189 INFO:tasks.workunit.client.0.smithi162.stdout:8/399: readlink d0/d10/l34 0 2024-04-27T04:10:05.189 INFO:tasks.workunit.client.0.smithi162.stdout:8/400: fdatasync d0/d10/f6a 0 2024-04-27T04:10:05.194 INFO:tasks.workunit.client.1.smithi181.stdout:9/547: creat d0/fbe x:0 0 0 2024-04-27T04:10:05.195 INFO:tasks.workunit.client.1.smithi181.stdout:6/459: dwrite d7/dc/d1e/f53 [4194304,4194304] 0 2024-04-27T04:10:05.196 INFO:tasks.workunit.client.0.smithi162.stdout:8/401: creat d0/d10/d25/d7d/f8b x:0 0 0 2024-04-27T04:10:05.197 INFO:tasks.workunit.client.1.smithi181.stdout:6/460: write d7/dc/d1e/d2b/f85 [742247,113035] 0 2024-04-27T04:10:05.202 INFO:tasks.workunit.client.1.smithi181.stdout:7/587: dread d1/d2/d1c/d8f/f8a [0,4194304] 0 2024-04-27T04:10:05.208 INFO:tasks.workunit.client.0.smithi162.stdout:7/318: rmdir d3/d5/d5b 39 2024-04-27T04:10:05.209 INFO:tasks.workunit.client.0.smithi162.stdout:6/382: write f5 [435404,39620] 0 2024-04-27T04:10:05.210 INFO:tasks.workunit.client.0.smithi162.stdout:6/383: readlink d6/d1a/d34/d43/l55 0 2024-04-27T04:10:05.211 INFO:tasks.workunit.client.0.smithi162.stdout:7/319: stat d3/d5/d5b/d60/d65 0 2024-04-27T04:10:05.215 INFO:tasks.workunit.client.0.smithi162.stdout:7/320: creat d3/d4f/f67 x:0 0 0 2024-04-27T04:10:05.217 INFO:tasks.workunit.client.0.smithi162.stdout:2/316: sync 2024-04-27T04:10:05.217 INFO:tasks.workunit.client.0.smithi162.stdout:3/377: sync 2024-04-27T04:10:05.218 INFO:tasks.workunit.client.0.smithi162.stdout:6/384: mknod d6/c79 0 2024-04-27T04:10:05.221 INFO:tasks.workunit.client.0.smithi162.stdout:4/393: creat d2/d25/d1c/d39/f7f x:0 0 0 2024-04-27T04:10:05.222 INFO:tasks.workunit.client.0.smithi162.stdout:8/402: dread d0/d10/d2b/f19 [0,4194304] 0 2024-04-27T04:10:05.222 INFO:tasks.workunit.client.0.smithi162.stdout:6/385: symlink d6/d1a/d34/d43/l7a 0 2024-04-27T04:10:05.223 INFO:tasks.workunit.client.0.smithi162.stdout:7/321: dwrite d3/d4f/f41 [0,4194304] 0 2024-04-27T04:10:05.223 INFO:tasks.workunit.client.0.smithi162.stdout:6/386: chown d6/db/d32/l40 2 1 2024-04-27T04:10:05.227 INFO:tasks.workunit.client.0.smithi162.stdout:2/317: truncate d0/f7 702121 0 2024-04-27T04:10:05.231 INFO:tasks.workunit.client.0.smithi162.stdout:4/394: rename d2/d25/d1c/d24/l34 to d2/d25/d1c/d39/d3b/d6e/l80 0 2024-04-27T04:10:05.232 INFO:tasks.workunit.client.0.smithi162.stdout:5/381: dwrite d8/f20 [4194304,4194304] 0 2024-04-27T04:10:05.235 INFO:tasks.workunit.client.0.smithi162.stdout:2/318: dwrite d0/d4/d2a/d41/f4a [0,4194304] 0 2024-04-27T04:10:05.242 INFO:tasks.workunit.client.0.smithi162.stdout:6/387: mkdir d6/d1a/d34/d7b 0 2024-04-27T04:10:05.243 INFO:tasks.workunit.client.0.smithi162.stdout:2/319: dwrite d0/d4/f22 [0,4194304] 0 2024-04-27T04:10:05.244 INFO:tasks.workunit.client.0.smithi162.stdout:5/382: creat d8/d9/d55/f84 x:0 0 0 2024-04-27T04:10:05.244 INFO:tasks.workunit.client.0.smithi162.stdout:9/368: dwrite d3/d13/f41 [0,4194304] 0 2024-04-27T04:10:05.246 INFO:tasks.workunit.client.0.smithi162.stdout:9/369: read d3/d13/d2d/d30/f38 [2961962,6969] 0 2024-04-27T04:10:05.257 INFO:tasks.workunit.client.0.smithi162.stdout:5/383: fdatasync d8/d9/f16 0 2024-04-27T04:10:05.261 INFO:tasks.workunit.client.1.smithi181.stdout:4/544: creat d0/d15/d25/d2a/d36/fb4 x:0 0 0 2024-04-27T04:10:05.262 INFO:tasks.workunit.client.0.smithi162.stdout:2/320: rename d0/d4/l3f to d0/d24/l62 0 2024-04-27T04:10:05.266 INFO:tasks.workunit.client.0.smithi162.stdout:4/395: link d2/d54/f35 d2/d25/d1c/d72/f81 0 2024-04-27T04:10:05.268 INFO:tasks.workunit.client.0.smithi162.stdout:9/370: mkdir d3/d13/d2d/d3b/d7a 0 2024-04-27T04:10:05.269 INFO:tasks.workunit.client.0.smithi162.stdout:8/403: link d0/d10/d2b/d2d/d2f/d39/l5e d0/d10/l8c 0 2024-04-27T04:10:05.270 INFO:tasks.workunit.client.0.smithi162.stdout:8/404: write d0/d10/d25/f36 [7942044,106222] 0 2024-04-27T04:10:05.272 INFO:tasks.workunit.client.1.smithi181.stdout:2/634: symlink d4/d9/d35/d42/d86/lcd 0 2024-04-27T04:10:05.273 INFO:tasks.workunit.client.1.smithi181.stdout:2/635: truncate d4/d9/df/d14/fb6 774950 0 2024-04-27T04:10:05.276 INFO:tasks.workunit.client.0.smithi162.stdout:6/388: creat d6/d1a/d34/d3a/d68/d6c/d67/f7c x:0 0 0 2024-04-27T04:10:05.277 INFO:tasks.workunit.client.0.smithi162.stdout:6/389: truncate d6/d1a/d34/f66 344232 0 2024-04-27T04:10:05.277 INFO:tasks.workunit.client.1.smithi181.stdout:6/461: read d7/fd [1917178,17498] 0 2024-04-27T04:10:05.279 INFO:tasks.workunit.client.1.smithi181.stdout:0/699: link dc/d1f/d3b/d4a/d95/d26/d41/d6f/d7d/f9b dc/d1f/d3b/d4a/ff3 0 2024-04-27T04:10:05.282 INFO:tasks.workunit.client.1.smithi181.stdout:4/545: dread - d0/d15/d76/da4/faf zero size 2024-04-27T04:10:05.283 INFO:tasks.workunit.client.1.smithi181.stdout:5/703: write d4/fc9 [364999,20631] 0 2024-04-27T04:10:05.285 INFO:tasks.workunit.client.0.smithi162.stdout:8/405: symlink d0/d10/d44/l8d 0 2024-04-27T04:10:05.285 INFO:tasks.workunit.client.0.smithi162.stdout:0/417: dwrite d5/dc/d1e/d41/f39 [0,4194304] 0 2024-04-27T04:10:05.290 INFO:tasks.workunit.client.0.smithi162.stdout:2/321: mkdir d0/d4/d2a/d3b/d49/d5a/d63 0 2024-04-27T04:10:05.290 INFO:tasks.workunit.client.0.smithi162.stdout:1/356: dwrite d0/f1e [0,4194304] 0 2024-04-27T04:10:05.291 INFO:tasks.workunit.client.0.smithi162.stdout:1/357: chown d0/d7/d5e/f62 2 1 2024-04-27T04:10:05.292 INFO:tasks.workunit.client.1.smithi181.stdout:9/548: symlink d0/d44/d4b/d43/d89/daf/lbf 0 2024-04-27T04:10:05.292 INFO:tasks.workunit.client.1.smithi181.stdout:5/704: dwrite d4/de/d2c/d46/d4d/fb8 [0,4194304] 0 2024-04-27T04:10:05.293 INFO:tasks.workunit.client.0.smithi162.stdout:6/390: mknod d6/db/d32/c7d 0 2024-04-27T04:10:05.298 INFO:tasks.workunit.client.0.smithi162.stdout:1/358: dwrite d0/d11/f20 [4194304,4194304] 0 2024-04-27T04:10:05.300 INFO:tasks.workunit.client.0.smithi162.stdout:1/359: write d0/d7/d3c/d54/f68 [589025,102846] 0 2024-04-27T04:10:05.300 INFO:tasks.workunit.client.1.smithi181.stdout:6/462: symlink d7/dc/d1c/d88/d6d/la0 0 2024-04-27T04:10:05.302 INFO:tasks.workunit.client.1.smithi181.stdout:0/700: mkdir dc/d1f/d6b/db9/dba/dce/df4 0 2024-04-27T04:10:05.304 INFO:tasks.workunit.client.0.smithi162.stdout:0/418: write d5/d26/f6a [2670092,55371] 0 2024-04-27T04:10:05.304 INFO:tasks.workunit.client.1.smithi181.stdout:1/581: link d1/dad/db5/d6/da3/d52/l66 d1/dad/db5/d6/da3/d1e/lb7 0 2024-04-27T04:10:05.305 INFO:tasks.workunit.client.1.smithi181.stdout:1/582: chown d1/d2/d89 22 1 2024-04-27T04:10:05.306 INFO:tasks.workunit.client.1.smithi181.stdout:1/583: fdatasync d1/dad/db5/d6/db/d2d/fb1 0 2024-04-27T04:10:05.307 INFO:tasks.workunit.client.0.smithi162.stdout:6/391: dread d6/f35 [0,4194304] 0 2024-04-27T04:10:05.311 INFO:tasks.workunit.client.1.smithi181.stdout:8/668: rename d1/d6/d2f/d40/d2a/db3/cdb to d1/d6/d2f/d30/ce9 0 2024-04-27T04:10:05.316 INFO:tasks.workunit.client.0.smithi162.stdout:4/396: link d2/da/dd/c23 d2/d25/d1c/d72/c82 0 2024-04-27T04:10:05.320 INFO:tasks.workunit.client.1.smithi181.stdout:5/705: dread d4/d10/d72/f77 [0,4194304] 0 2024-04-27T04:10:05.321 INFO:tasks.workunit.client.1.smithi181.stdout:6/463: dread d7/dc/d1e/d2b/f5a [0,4194304] 0 2024-04-27T04:10:05.323 INFO:tasks.workunit.client.0.smithi162.stdout:1/360: rename d0/d7/d10/d35 to d0/d7/d3c/d77 0 2024-04-27T04:10:05.323 INFO:tasks.workunit.client.0.smithi162.stdout:1/361: fdatasync d0/d7/d2a/f5d 0 2024-04-27T04:10:05.324 INFO:tasks.workunit.client.0.smithi162.stdout:1/362: stat d0/fe 0 2024-04-27T04:10:05.324 INFO:tasks.workunit.client.1.smithi181.stdout:0/701: rmdir dc/d54 39 2024-04-27T04:10:05.324 INFO:tasks.workunit.client.0.smithi162.stdout:1/363: chown d0/d7/l36 781 1 2024-04-27T04:10:05.325 INFO:tasks.workunit.client.0.smithi162.stdout:8/406: getdents d0/de/d6e/d71/d86 0 2024-04-27T04:10:05.326 INFO:tasks.workunit.client.0.smithi162.stdout:0/419: dread d5/f58 [0,4194304] 0 2024-04-27T04:10:05.327 INFO:tasks.workunit.client.0.smithi162.stdout:0/420: readlink d5/l1c 0 2024-04-27T04:10:05.328 INFO:tasks.workunit.client.0.smithi162.stdout:0/421: write d5/dc/fe [3758985,29190] 0 2024-04-27T04:10:05.328 INFO:tasks.workunit.client.0.smithi162.stdout:0/422: readlink d5/dc/d1e/l6b 0 2024-04-27T04:10:05.329 INFO:tasks.workunit.client.0.smithi162.stdout:0/423: readlink d5/dc/d1e/d20/l6e 0 2024-04-27T04:10:05.329 INFO:tasks.workunit.client.1.smithi181.stdout:7/588: rename d1/d2/d1c/f8d to d1/d22/d15/d5f/d93/fb4 0 2024-04-27T04:10:05.333 INFO:tasks.workunit.client.0.smithi162.stdout:7/322: sync 2024-04-27T04:10:05.333 INFO:tasks.workunit.client.1.smithi181.stdout:3/522: getdents d0/d6c/d66 0 2024-04-27T04:10:05.333 INFO:tasks.workunit.client.0.smithi162.stdout:3/378: truncate d1/d18/f1e 2148130 0 2024-04-27T04:10:05.334 INFO:tasks.workunit.client.0.smithi162.stdout:0/424: dwrite d5/dc/d1e/d41/f39 [0,4194304] 0 2024-04-27T04:10:05.335 INFO:tasks.workunit.client.1.smithi181.stdout:9/549: creat d0/d44/d4b/d5f/d8a/fc0 x:0 0 0 2024-04-27T04:10:05.338 INFO:tasks.workunit.client.1.smithi181.stdout:2/636: creat d4/d9/d4f/d79/fce x:0 0 0 2024-04-27T04:10:05.340 INFO:tasks.workunit.client.1.smithi181.stdout:6/464: fsync d7/dc/d1e/d2b/f5a 0 2024-04-27T04:10:05.347 INFO:tasks.workunit.client.0.smithi162.stdout:0/425: dread d5/dc/f22 [0,4194304] 0 2024-04-27T04:10:05.348 INFO:tasks.workunit.client.0.smithi162.stdout:0/426: truncate d5/dc/fe 4526529 0 2024-04-27T04:10:05.348 INFO:tasks.workunit.client.1.smithi181.stdout:7/589: mknod d1/d22/cb5 0 2024-04-27T04:10:05.348 INFO:tasks.workunit.client.0.smithi162.stdout:0/427: dread - d5/d26/d69/f6c zero size 2024-04-27T04:10:05.351 INFO:tasks.workunit.client.1.smithi181.stdout:3/523: truncate d0/d6c/f4c 1452871 0 2024-04-27T04:10:05.354 INFO:tasks.workunit.client.0.smithi162.stdout:9/371: sync 2024-04-27T04:10:05.354 INFO:tasks.workunit.client.0.smithi162.stdout:4/397: rmdir d2/d25 39 2024-04-27T04:10:05.354 INFO:tasks.workunit.client.1.smithi181.stdout:9/550: creat d0/d4d/d85/fc1 x:0 0 0 2024-04-27T04:10:05.355 INFO:tasks.workunit.client.0.smithi162.stdout:1/364: creat d0/d11/d44/d66/f78 x:0 0 0 2024-04-27T04:10:05.356 INFO:tasks.workunit.client.1.smithi181.stdout:2/637: rmdir d4/d9/d16/d45 39 2024-04-27T04:10:05.359 INFO:tasks.workunit.client.0.smithi162.stdout:6/392: mkdir d6/d1a/d34/d7b/d7e 0 2024-04-27T04:10:05.359 INFO:tasks.workunit.client.0.smithi162.stdout:5/384: write d8/d9/f10 [1053826,71716] 0 2024-04-27T04:10:05.360 INFO:tasks.workunit.client.1.smithi181.stdout:6/465: mknod d7/dc/d1c/d7f/d3e/ca1 0 2024-04-27T04:10:05.364 INFO:tasks.workunit.client.1.smithi181.stdout:1/584: creat d1/dad/db5/fb8 x:0 0 0 2024-04-27T04:10:05.364 INFO:tasks.workunit.client.0.smithi162.stdout:6/393: dwrite d6/d1a/d34/f66 [0,4194304] 0 2024-04-27T04:10:05.366 INFO:tasks.workunit.client.1.smithi181.stdout:7/590: read - d1/d2/d1c/d37/f59 zero size 2024-04-27T04:10:05.366 INFO:tasks.workunit.client.1.smithi181.stdout:7/591: chown d1/d57/d72/c9d 407459407 1 2024-04-27T04:10:05.369 INFO:tasks.workunit.client.1.smithi181.stdout:3/524: fsync d0/de/d20/d58/f7e 0 2024-04-27T04:10:05.370 INFO:tasks.workunit.client.0.smithi162.stdout:6/394: dwrite d6/f26 [4194304,4194304] 0 2024-04-27T04:10:05.372 INFO:tasks.workunit.client.0.smithi162.stdout:7/323: symlink d3/d5/d5b/d60/l68 0 2024-04-27T04:10:05.381 INFO:tasks.workunit.client.1.smithi181.stdout:0/702: creat dc/d1f/d3b/d4a/d95/d26/d41/d6f/ff5 x:0 0 0 2024-04-27T04:10:05.384 INFO:tasks.workunit.client.1.smithi181.stdout:4/546: dwrite d0/d15/f2f [0,4194304] 0 2024-04-27T04:10:05.384 INFO:tasks.workunit.client.0.smithi162.stdout:4/398: chown d2/d25 7482410 1 2024-04-27T04:10:05.386 INFO:tasks.workunit.client.0.smithi162.stdout:9/372: truncate d3/d13/d2d/d30/f38 2895449 0 2024-04-27T04:10:05.387 INFO:tasks.workunit.client.0.smithi162.stdout:9/373: dread - d3/d11/d45/f55 zero size 2024-04-27T04:10:05.387 INFO:tasks.workunit.client.1.smithi181.stdout:8/669: getdents d1/d6/d2f/d47 0 2024-04-27T04:10:05.395 INFO:tasks.workunit.client.0.smithi162.stdout:8/407: creat d0/d10/d25/d65/f8e x:0 0 0 2024-04-27T04:10:05.395 INFO:tasks.workunit.client.0.smithi162.stdout:8/408: chown d0/d10/d2b/d2d/d2f/d39 115 1 2024-04-27T04:10:05.396 INFO:tasks.workunit.client.1.smithi181.stdout:3/525: chown d0/d6c/l4a 0 1 2024-04-27T04:10:05.399 INFO:tasks.workunit.client.0.smithi162.stdout:5/385: rename d8/ce to d8/d9/d5b/c85 0 2024-04-27T04:10:05.402 INFO:tasks.workunit.client.1.smithi181.stdout:1/585: symlink d1/dad/db5/d6/lb9 0 2024-04-27T04:10:05.404 INFO:tasks.workunit.client.0.smithi162.stdout:5/386: dwrite d8/d1a/d23/d40/f42 [0,4194304] 0 2024-04-27T04:10:05.409 INFO:tasks.workunit.client.0.smithi162.stdout:7/324: creat d3/d5/d9/d4b/de/d2b/f69 x:0 0 0 2024-04-27T04:10:05.411 INFO:tasks.workunit.client.1.smithi181.stdout:8/670: unlink d1/d6/d88/cb2 0 2024-04-27T04:10:05.416 INFO:tasks.workunit.client.0.smithi162.stdout:4/399: dread - d2/d25/d1c/d24/f43 zero size 2024-04-27T04:10:05.416 INFO:tasks.workunit.client.1.smithi181.stdout:0/703: dread dc/d1f/d3b/d4a/d95/d26/d41/f71 [0,4194304] 0 2024-04-27T04:10:05.417 INFO:tasks.workunit.client.1.smithi181.stdout:0/704: chown dc/d1f/d3b/d4a/d7a 4 1 2024-04-27T04:10:05.418 INFO:tasks.workunit.client.0.smithi162.stdout:9/374: write d3/d13/d15/f57 [3633181,35697] 0 2024-04-27T04:10:05.419 INFO:tasks.workunit.client.1.smithi181.stdout:1/586: mknod d1/d2/d89/cba 0 2024-04-27T04:10:05.419 INFO:tasks.workunit.client.0.smithi162.stdout:9/375: chown d3/d23/f5c 2 1 2024-04-27T04:10:05.421 INFO:tasks.workunit.client.1.smithi181.stdout:4/547: mkdir d0/d15/d25/d2a/d4c/d7c/db5 0 2024-04-27T04:10:05.422 INFO:tasks.workunit.client.0.smithi162.stdout:2/322: dwrite d0/f7 [0,4194304] 0 2024-04-27T04:10:05.424 INFO:tasks.workunit.client.1.smithi181.stdout:1/587: dread d1/dad/db5/d6/da3/d52/d92/f93 [4194304,4194304] 0 2024-04-27T04:10:05.425 INFO:tasks.workunit.client.1.smithi181.stdout:6/466: write d7/d31/f37 [1065856,81350] 0 2024-04-27T04:10:05.429 INFO:tasks.workunit.client.0.smithi162.stdout:5/387: mkdir d8/d86 0 2024-04-27T04:10:05.431 INFO:tasks.workunit.client.0.smithi162.stdout:0/428: dwrite d5/d26/d4a/f62 [4194304,4194304] 0 2024-04-27T04:10:05.434 INFO:tasks.workunit.client.0.smithi162.stdout:6/395: fsync d6/f2d 0 2024-04-27T04:10:05.435 INFO:tasks.workunit.client.1.smithi181.stdout:3/526: creat d0/d6c/d66/d88/fa8 x:0 0 0 2024-04-27T04:10:05.437 INFO:tasks.workunit.client.0.smithi162.stdout:7/325: creat d3/d5/d3c/d4a/f6a x:0 0 0 2024-04-27T04:10:05.439 INFO:tasks.workunit.client.1.smithi181.stdout:9/551: getdents d0/d44/d4b/d43/d8f 0 2024-04-27T04:10:05.440 INFO:tasks.workunit.client.0.smithi162.stdout:3/379: link d1/d18/d23/f53 d1/d18/d5d/d5f/f76 0 2024-04-27T04:10:05.446 INFO:tasks.workunit.client.1.smithi181.stdout:9/552: dread d0/d4d/f7f [0,4194304] 0 2024-04-27T04:10:05.447 INFO:tasks.workunit.client.0.smithi162.stdout:4/400: dwrite d2/ff [0,4194304] 0 2024-04-27T04:10:05.448 INFO:tasks.workunit.client.1.smithi181.stdout:7/592: link d1/df/cae d1/d2/d1c/d37/cb6 0 2024-04-27T04:10:05.449 INFO:tasks.workunit.client.1.smithi181.stdout:7/593: chown d1/df/d62 272769025 1 2024-04-27T04:10:05.451 INFO:tasks.workunit.client.0.smithi162.stdout:9/376: mkdir d3/d13/d2d/d3b/d7b 0 2024-04-27T04:10:05.452 INFO:tasks.workunit.client.1.smithi181.stdout:1/588: symlink d1/d2/d89/lbb 0 2024-04-27T04:10:05.455 INFO:tasks.workunit.client.0.smithi162.stdout:2/323: mkdir d0/d4/d17/d2b/d64 0 2024-04-27T04:10:05.457 INFO:tasks.workunit.client.0.smithi162.stdout:9/377: dwrite d3/d23/d1e/f2b [0,4194304] 0 2024-04-27T04:10:05.457 INFO:tasks.workunit.client.0.smithi162.stdout:2/324: fsync d0/d4/d2a/d41/f4a 0 2024-04-27T04:10:05.458 INFO:tasks.workunit.client.0.smithi162.stdout:2/325: readlink d0/d4/d2a/l58 0 2024-04-27T04:10:05.459 INFO:tasks.workunit.client.1.smithi181.stdout:3/527: symlink d0/d6c/d52/d44/d74/la9 0 2024-04-27T04:10:05.459 INFO:tasks.workunit.client.0.smithi162.stdout:5/388: rename d8/d2c/d41 to d8/d1a/d7d/d87 0 2024-04-27T04:10:05.462 INFO:tasks.workunit.client.0.smithi162.stdout:0/429: symlink d5/dc/d1e/d56/d57/l8a 0 2024-04-27T04:10:05.463 INFO:tasks.workunit.client.1.smithi181.stdout:8/671: dwrite d1/d8c/f90 [0,4194304] 0 2024-04-27T04:10:05.464 INFO:tasks.workunit.client.0.smithi162.stdout:9/378: dwrite d3/d13/f26 [0,4194304] 0 2024-04-27T04:10:05.466 INFO:tasks.workunit.client.0.smithi162.stdout:9/379: fsync d3/d13/d15/d50/f5b 0 2024-04-27T04:10:05.466 INFO:tasks.workunit.client.1.smithi181.stdout:0/705: mkdir dc/d1f/d6b/db9/dba/deb/df6 0 2024-04-27T04:10:05.470 INFO:tasks.workunit.client.1.smithi181.stdout:4/548: fdatasync d0/d15/d5c/d69/f10 0 2024-04-27T04:10:05.471 INFO:tasks.workunit.client.1.smithi181.stdout:4/549: chown d0/d15/f1a 989 1 2024-04-27T04:10:05.472 INFO:tasks.workunit.client.1.smithi181.stdout:4/550: readlink d0/d15/de/d19/l54 0 2024-04-27T04:10:05.474 INFO:tasks.workunit.client.1.smithi181.stdout:4/551: write d0/d15/d25/d2a/f38 [2992296,55050] 0 2024-04-27T04:10:05.480 INFO:tasks.workunit.client.0.smithi162.stdout:1/365: getdents d0/d11/d44 0 2024-04-27T04:10:05.481 INFO:tasks.workunit.client.1.smithi181.stdout:1/589: dread - d1/dad/db5/d6/da3/d1e/f9f zero size 2024-04-27T04:10:05.481 INFO:tasks.workunit.client.1.smithi181.stdout:1/590: chown d1/d2/d64/f8d 986924962 1 2024-04-27T04:10:05.482 INFO:tasks.workunit.client.0.smithi162.stdout:4/401: write d2/da/f78 [4631537,122351] 0 2024-04-27T04:10:05.483 INFO:tasks.workunit.client.0.smithi162.stdout:2/326: dread d0/d4/d2a/f3e [0,4194304] 0 2024-04-27T04:10:05.486 INFO:tasks.workunit.client.1.smithi181.stdout:8/672: rename d1/d6/d2f/d40/d2a to d1/d6/d2f/d30/d9d/dea 0 2024-04-27T04:10:05.488 INFO:tasks.workunit.client.0.smithi162.stdout:4/402: dwrite d2/d25/d1c/d24/f69 [0,4194304] 0 2024-04-27T04:10:05.490 INFO:tasks.workunit.client.1.smithi181.stdout:4/552: dread d0/d15/d5c/d69/f10 [0,4194304] 0 2024-04-27T04:10:05.493 INFO:tasks.workunit.client.0.smithi162.stdout:2/327: dwrite d0/d4/d17/f2f [4194304,4194304] 0 2024-04-27T04:10:05.494 INFO:tasks.workunit.client.1.smithi181.stdout:7/594: mknod d1/d22/d15/d13/cb7 0 2024-04-27T04:10:05.495 INFO:tasks.workunit.client.1.smithi181.stdout:3/528: dread d0/de/f17 [0,4194304] 0 2024-04-27T04:10:05.499 INFO:tasks.workunit.client.0.smithi162.stdout:7/326: rename d3/d4f/f4c to d3/d5/d5b/d60/d65/f6b 0 2024-04-27T04:10:05.503 INFO:tasks.workunit.client.0.smithi162.stdout:5/389: mkdir d8/d1a/d23/d40/d4b/d66/d88 0 2024-04-27T04:10:05.505 INFO:tasks.workunit.client.1.smithi181.stdout:6/467: link d7/dc/d1c/d7f/d3e/c60 d7/d31/d45/ca2 0 2024-04-27T04:10:05.507 INFO:tasks.workunit.client.1.smithi181.stdout:0/706: mkdir dc/d1f/d3b/df7 0 2024-04-27T04:10:05.510 INFO:tasks.workunit.client.1.smithi181.stdout:8/673: symlink d1/d6/d2f/d30/d66/dc0/dcd/leb 0 2024-04-27T04:10:05.511 INFO:tasks.workunit.client.0.smithi162.stdout:5/390: dwrite d8/d9/fc [0,4194304] 0 2024-04-27T04:10:05.516 INFO:tasks.workunit.client.0.smithi162.stdout:9/380: mkdir d3/d23/d7c 0 2024-04-27T04:10:05.522 INFO:tasks.workunit.client.1.smithi181.stdout:4/553: rename d0/d15/d25/d2b/f6 to d0/d15/d76/db1/fb6 0 2024-04-27T04:10:05.522 INFO:tasks.workunit.client.0.smithi162.stdout:6/396: mkdir d6/d1a/d34/d7f 0 2024-04-27T04:10:05.526 INFO:tasks.workunit.client.0.smithi162.stdout:3/380: link d1/d18/f4d d1/d18/d5d/d5f/f77 0 2024-04-27T04:10:05.527 INFO:tasks.workunit.client.1.smithi181.stdout:3/529: fsync d0/de/d20/d58/d64/f6a 0 2024-04-27T04:10:05.530 INFO:tasks.workunit.client.1.smithi181.stdout:1/591: mkdir d1/dad/db5/d6/da3/d1e/dbc 0 2024-04-27T04:10:05.531 INFO:tasks.workunit.client.1.smithi181.stdout:1/592: write d1/dad/db5/fa1 [808169,87833] 0 2024-04-27T04:10:05.531 INFO:tasks.workunit.client.0.smithi162.stdout:8/409: getdents d0/de 0 2024-04-27T04:10:05.532 INFO:tasks.workunit.client.1.smithi181.stdout:6/468: mkdir d7/dc/d1c/d7f/da3 0 2024-04-27T04:10:05.534 INFO:tasks.workunit.client.1.smithi181.stdout:0/707: creat dc/d93/ff8 x:0 0 0 2024-04-27T04:10:05.536 INFO:tasks.workunit.client.0.smithi162.stdout:4/403: write d2/da/d1a/f2e [2394319,89515] 0 2024-04-27T04:10:05.537 INFO:tasks.workunit.client.1.smithi181.stdout:8/674: read d1/d6/d2f/d30/d9d/dea/f42 [5351603,59064] 0 2024-04-27T04:10:05.537 INFO:tasks.workunit.client.1.smithi181.stdout:9/553: getdents d0/d44/d4b/d5f/d8a 0 2024-04-27T04:10:05.544 INFO:tasks.workunit.client.0.smithi162.stdout:7/327: rename d3/d39/d57/f44 to d3/d5/d5b/f6c 0 2024-04-27T04:10:05.544 INFO:tasks.workunit.client.0.smithi162.stdout:5/391: rename d8/d1a/d23/d40/d4b/d58 to d8/d1a/d23/d40/d4b/d58/d89 22 2024-04-27T04:10:05.546 INFO:tasks.workunit.client.0.smithi162.stdout:7/328: write d3/d5/d9/f24 [2278926,53224] 0 2024-04-27T04:10:05.548 INFO:tasks.workunit.client.0.smithi162.stdout:7/329: write d3/d5/d3c/d4a/f6a [813847,87968] 0 2024-04-27T04:10:05.551 INFO:tasks.workunit.client.1.smithi181.stdout:1/593: symlink d1/dad/lbd 0 2024-04-27T04:10:05.551 INFO:tasks.workunit.client.1.smithi181.stdout:7/595: write d1/d2/d1c/d37/f59 [360190,21327] 0 2024-04-27T04:10:05.552 INFO:tasks.workunit.client.0.smithi162.stdout:3/381: symlink d1/d18/d5d/l78 0 2024-04-27T04:10:05.553 INFO:tasks.workunit.client.1.smithi181.stdout:7/596: write d1/d2/d35/d8b/fa1 [1009903,35971] 0 2024-04-27T04:10:05.554 INFO:tasks.workunit.client.1.smithi181.stdout:7/597: chown d1/d22/d15/d13/d1b/f8c 0 1 2024-04-27T04:10:05.554 INFO:tasks.workunit.client.1.smithi181.stdout:6/469: mknod d7/dc/d1c/d88/ca4 0 2024-04-27T04:10:05.558 INFO:tasks.workunit.client.0.smithi162.stdout:3/382: dwrite d1/d18/f12 [0,4194304] 0 2024-04-27T04:10:05.561 INFO:tasks.workunit.client.1.smithi181.stdout:3/530: dwrite d0/de/d20/d58/d64/f6a [0,4194304] 0 2024-04-27T04:10:05.563 INFO:tasks.workunit.client.1.smithi181.stdout:8/675: mkdir d1/d6/d88/dec 0 2024-04-27T04:10:05.564 INFO:tasks.workunit.client.1.smithi181.stdout:3/531: read d0/de/f1f [3941956,79552] 0 2024-04-27T04:10:05.566 INFO:tasks.workunit.client.0.smithi162.stdout:8/410: rename d0/d10/d2b/l1e to d0/d10/d2b/d2d/d2f/d39/d7c/l8f 0 2024-04-27T04:10:05.567 INFO:tasks.workunit.client.1.smithi181.stdout:4/554: write d0/d15/d25/d40/f6c [3215570,67733] 0 2024-04-27T04:10:05.571 INFO:tasks.workunit.client.0.smithi162.stdout:5/392: mknod d8/d1a/d23/d40/d4b/d58/c8a 0 2024-04-27T04:10:05.573 INFO:tasks.workunit.client.1.smithi181.stdout:1/594: rename d1/dad/db5/c3a to d1/dad/db5/d6/da3/d1e/d48/cbe 0 2024-04-27T04:10:05.575 INFO:tasks.workunit.client.1.smithi181.stdout:4/555: dwrite d0/d15/d25/d2b/fb0 [0,4194304] 0 2024-04-27T04:10:05.576 INFO:tasks.workunit.client.1.smithi181.stdout:6/470: unlink f0 0 2024-04-27T04:10:05.577 INFO:tasks.workunit.client.0.smithi162.stdout:0/430: dwrite d5/dc/d1e/f21 [0,4194304] 0 2024-04-27T04:10:05.578 INFO:tasks.workunit.client.0.smithi162.stdout:6/397: mkdir d6/d1a/d34/d7f/d80 0 2024-04-27T04:10:05.579 INFO:tasks.workunit.client.0.smithi162.stdout:5/393: dwrite f4 [4194304,4194304] 0 2024-04-27T04:10:05.581 INFO:tasks.workunit.client.0.smithi162.stdout:0/431: chown d5/dc/d1e/d20/l7c 4130316 1 2024-04-27T04:10:05.581 INFO:tasks.workunit.client.0.smithi162.stdout:5/394: dread - d8/d9/d55/f84 zero size 2024-04-27T04:10:05.581 INFO:tasks.workunit.client.0.smithi162.stdout:6/398: write d6/d1a/f15 [2145181,15131] 0 2024-04-27T04:10:05.582 INFO:tasks.workunit.client.1.smithi181.stdout:0/708: fdatasync dc/d1f/d3b/d4a/d95/f25 0 2024-04-27T04:10:05.585 INFO:tasks.workunit.client.0.smithi162.stdout:3/383: creat d1/d45/f79 x:0 0 0 2024-04-27T04:10:05.591 INFO:tasks.workunit.client.0.smithi162.stdout:7/330: mknod d3/d39/c6d 0 2024-04-27T04:10:05.593 INFO:tasks.workunit.client.1.smithi181.stdout:3/532: unlink d0/d6c/d52/f68 0 2024-04-27T04:10:05.594 INFO:tasks.workunit.client.0.smithi162.stdout:2/328: getdents d0/d4/d17/d4f 0 2024-04-27T04:10:05.595 INFO:tasks.workunit.client.0.smithi162.stdout:2/329: truncate d0/f44 4322608 0 2024-04-27T04:10:05.598 INFO:tasks.workunit.client.0.smithi162.stdout:5/395: mkdir d8/d1a/d29/d8b 0 2024-04-27T04:10:05.599 INFO:tasks.workunit.client.0.smithi162.stdout:5/396: dread - d8/d9/d55/f65 zero size 2024-04-27T04:10:05.599 INFO:tasks.workunit.client.0.smithi162.stdout:5/397: readlink d8/d9/l80 0 2024-04-27T04:10:05.600 INFO:tasks.workunit.client.0.smithi162.stdout:5/398: chown d8/d1a/d29/d4a/d5c 14939 1 2024-04-27T04:10:05.601 INFO:tasks.workunit.client.0.smithi162.stdout:1/366: sync 2024-04-27T04:10:05.601 INFO:tasks.workunit.client.0.smithi162.stdout:5/399: chown d8/d1a/f83 184 1 2024-04-27T04:10:05.602 INFO:tasks.workunit.client.0.smithi162.stdout:1/367: truncate d0/d7/d10/f4c 8750481 0 2024-04-27T04:10:05.604 INFO:tasks.workunit.client.0.smithi162.stdout:0/432: fsync d5/f2d 0 2024-04-27T04:10:05.604 INFO:tasks.workunit.client.1.smithi181.stdout:7/598: symlink d1/d2/d1c/d2d/lb8 0 2024-04-27T04:10:05.606 INFO:tasks.workunit.client.0.smithi162.stdout:8/411: symlink d0/de/d6e/l90 0 2024-04-27T04:10:05.606 INFO:tasks.workunit.client.0.smithi162.stdout:5/400: dread d8/d1a/d23/d40/d4b/f52 [4194304,4194304] 0 2024-04-27T04:10:05.607 INFO:tasks.workunit.client.0.smithi162.stdout:7/331: creat d3/d5/d9/d4b/f6e x:0 0 0 2024-04-27T04:10:05.608 INFO:tasks.workunit.client.1.smithi181.stdout:9/554: write d0/f68 [372851,58117] 0 2024-04-27T04:10:05.608 INFO:tasks.workunit.client.0.smithi162.stdout:8/412: write d0/de/d20/d6c/f6d [401865,84769] 0 2024-04-27T04:10:05.609 INFO:tasks.workunit.client.0.smithi162.stdout:7/332: chown d3/d39 55011397 1 2024-04-27T04:10:05.609 INFO:tasks.workunit.client.0.smithi162.stdout:5/401: chown d8/d1a/d29/d4a/d5c/f62 1698634 1 2024-04-27T04:10:05.611 INFO:tasks.workunit.client.0.smithi162.stdout:1/368: dwrite d0/d7/f5a [0,4194304] 0 2024-04-27T04:10:05.611 INFO:tasks.workunit.client.0.smithi162.stdout:6/399: creat d6/d1a/d34/d3a/d60/d74/f81 x:0 0 0 2024-04-27T04:10:05.611 INFO:tasks.workunit.client.0.smithi162.stdout:5/402: chown d8/d1a/d29/d8b 1866740 1 2024-04-27T04:10:05.614 INFO:tasks.workunit.client.1.smithi181.stdout:8/676: symlink d1/d20/da4/dc5/led 0 2024-04-27T04:10:05.614 INFO:tasks.workunit.client.0.smithi162.stdout:5/403: write d8/d1a/d29/f4d [1070642,76073] 0 2024-04-27T04:10:05.618 INFO:tasks.workunit.client.1.smithi181.stdout:6/471: write d7/dc/d1e/f4d [727577,9337] 0 2024-04-27T04:10:05.618 INFO:tasks.workunit.client.1.smithi181.stdout:9/555: dwrite d0/d44/d4f/fae [0,4194304] 0 2024-04-27T04:10:05.624 INFO:tasks.workunit.client.0.smithi162.stdout:0/433: mkdir d5/d60/d8b 0 2024-04-27T04:10:05.631 INFO:tasks.workunit.client.0.smithi162.stdout:8/413: creat d0/d10/d25/f91 x:0 0 0 2024-04-27T04:10:05.633 INFO:tasks.workunit.client.0.smithi162.stdout:3/384: write d1/d18/d5d/d5f/f76 [2008868,8835] 0 2024-04-27T04:10:05.637 INFO:tasks.workunit.client.0.smithi162.stdout:7/333: rename d3/d5/d9/d4b/f2f to d3/d5/d5b/d60/f6f 0 2024-04-27T04:10:05.639 INFO:tasks.workunit.client.1.smithi181.stdout:1/595: write d1/dad/db5/d6/db/d2d/d36/f9a [138943,90388] 0 2024-04-27T04:10:05.640 INFO:tasks.workunit.client.1.smithi181.stdout:3/533: dwrite d0/f16 [0,4194304] 0 2024-04-27T04:10:05.640 INFO:tasks.workunit.client.1.smithi181.stdout:9/556: symlink d0/d4d/lc2 0 2024-04-27T04:10:05.644 INFO:tasks.workunit.client.0.smithi162.stdout:2/330: truncate d0/d4/f16 1326575 0 2024-04-27T04:10:05.645 INFO:tasks.workunit.client.0.smithi162.stdout:5/404: symlink d8/d1a/d23/d40/d4b/d66/l8c 0 2024-04-27T04:10:05.647 INFO:tasks.workunit.client.1.smithi181.stdout:4/556: creat d0/d15/de/fb7 x:0 0 0 2024-04-27T04:10:05.649 INFO:tasks.workunit.client.0.smithi162.stdout:0/434: unlink d5/dc/d1e/d46/f85 0 2024-04-27T04:10:05.652 INFO:tasks.workunit.client.1.smithi181.stdout:0/709: creat dc/d1f/d3b/ff9 x:0 0 0 2024-04-27T04:10:05.653 INFO:tasks.workunit.client.0.smithi162.stdout:1/369: dwrite d0/d7/d3c/f57 [4194304,4194304] 0 2024-04-27T04:10:05.653 INFO:tasks.workunit.client.1.smithi181.stdout:8/677: mknod d1/d6/d1a/cee 0 2024-04-27T04:10:05.654 INFO:tasks.workunit.client.0.smithi162.stdout:5/405: dwrite d8/d9/fc [0,4194304] 0 2024-04-27T04:10:05.654 INFO:tasks.workunit.client.1.smithi181.stdout:4/557: dwrite d0/d15/f2 [0,4194304] 0 2024-04-27T04:10:05.654 INFO:tasks.workunit.client.1.smithi181.stdout:0/710: chown dc/f3a 11866635 1 2024-04-27T04:10:05.655 INFO:tasks.workunit.client.0.smithi162.stdout:1/370: stat d0/d11/d3e/c5c 0 2024-04-27T04:10:05.655 INFO:tasks.workunit.client.0.smithi162.stdout:1/371: stat d0/d11/d50 0 2024-04-27T04:10:05.658 INFO:tasks.workunit.client.1.smithi181.stdout:6/472: dwrite d7/dc/d1c/d7f/d3e/f54 [4194304,4194304] 0 2024-04-27T04:10:05.663 INFO:tasks.workunit.client.1.smithi181.stdout:1/596: fsync d1/dad/db5/f78 0 2024-04-27T04:10:05.664 INFO:tasks.workunit.client.0.smithi162.stdout:2/331: creat d0/d4/d17/d2b/f65 x:0 0 0 2024-04-27T04:10:05.670 INFO:tasks.workunit.client.1.smithi181.stdout:3/534: truncate d0/de/d20/f5c 610799 0 2024-04-27T04:10:05.675 INFO:tasks.workunit.client.0.smithi162.stdout:8/414: mknod d0/d10/d76/c92 0 2024-04-27T04:10:05.676 INFO:tasks.workunit.client.0.smithi162.stdout:8/415: write d0/de/d20/d6c/f6d [84404,73240] 0 2024-04-27T04:10:05.682 INFO:tasks.workunit.client.0.smithi162.stdout:0/435: dread d5/d26/d4a/f62 [0,4194304] 0 2024-04-27T04:10:05.683 INFO:tasks.workunit.client.1.smithi181.stdout:4/558: mkdir d0/d15/d25/d2a/d36/db8 0 2024-04-27T04:10:05.683 INFO:tasks.workunit.client.1.smithi181.stdout:9/557: write d0/d44/f51 [4421362,23055] 0 2024-04-27T04:10:05.685 INFO:tasks.workunit.client.1.smithi181.stdout:6/473: mknod d7/dc/d1e/d66/ca5 0 2024-04-27T04:10:05.688 INFO:tasks.workunit.client.1.smithi181.stdout:3/535: stat d0/d9f/f7c 0 2024-04-27T04:10:05.689 INFO:tasks.workunit.client.1.smithi181.stdout:7/599: getdents d1/d2/d1c/d2d 0 2024-04-27T04:10:05.690 INFO:tasks.workunit.client.1.smithi181.stdout:0/711: mkdir dc/d1f/d3b/dfa 0 2024-04-27T04:10:05.691 INFO:tasks.workunit.client.1.smithi181.stdout:0/712: stat dc/d1f/daa/fcb 0 2024-04-27T04:10:05.691 INFO:tasks.workunit.client.1.smithi181.stdout:0/713: dread - dc/d1f/d3b/ff9 zero size 2024-04-27T04:10:05.693 INFO:tasks.workunit.client.1.smithi181.stdout:0/714: write dc/d1f/d3b/d4a/d95/d26/d41/d6f/ff5 [43652,86400] 0 2024-04-27T04:10:05.694 INFO:tasks.workunit.client.1.smithi181.stdout:4/559: mkdir d0/d15/de/d60/d6f/db9 0 2024-04-27T04:10:05.695 INFO:tasks.workunit.client.0.smithi162.stdout:0/436: mknod d5/d60/c8c 0 2024-04-27T04:10:05.695 INFO:tasks.workunit.client.1.smithi181.stdout:0/715: truncate dc/d1f/d6b/db9/fcf 12249 0 2024-04-27T04:10:05.696 INFO:tasks.workunit.client.0.smithi162.stdout:0/437: chown d5/dc/d1e/d56/l75 12789902 1 2024-04-27T04:10:05.699 INFO:tasks.workunit.client.1.smithi181.stdout:0/716: write dc/d1f/d6b/db9/dbd/fbe [98879,116180] 0 2024-04-27T04:10:05.699 INFO:tasks.workunit.client.1.smithi181.stdout:6/474: creat d7/dc/d1c/d34/d3f/d7e/fa6 x:0 0 0 2024-04-27T04:10:05.699 INFO:tasks.workunit.client.0.smithi162.stdout:6/400: getdents d6/d1a/d34/d3a/d68/d6c/d67 0 2024-04-27T04:10:05.701 INFO:tasks.workunit.client.1.smithi181.stdout:6/475: write d7/dc/d1c/d7f/f29 [1170221,124334] 0 2024-04-27T04:10:05.702 INFO:tasks.workunit.client.1.smithi181.stdout:1/597: getdents d1/dad/db5/d6/da3/d1e/dbc 0 2024-04-27T04:10:05.704 INFO:tasks.workunit.client.1.smithi181.stdout:6/476: write d7/dc/d1c/d7f/d3e/f54 [6850790,100543] 0 2024-04-27T04:10:05.706 INFO:tasks.workunit.client.1.smithi181.stdout:4/560: mkdir d0/d15/d25/d2a/d36/dba 0 2024-04-27T04:10:05.706 INFO:tasks.workunit.client.0.smithi162.stdout:0/438: truncate d5/dc/f22 3907855 0 2024-04-27T04:10:05.707 INFO:tasks.workunit.client.0.smithi162.stdout:1/372: creat d0/d11/f79 x:0 0 0 2024-04-27T04:10:05.707 INFO:tasks.workunit.client.0.smithi162.stdout:0/439: readlink d5/dc/d1e/d20/l6e 0 2024-04-27T04:10:05.708 INFO:tasks.workunit.client.1.smithi181.stdout:6/477: truncate d7/dc/d1c/d34/d3f/f6c 495749 0 2024-04-27T04:10:05.709 INFO:tasks.workunit.client.0.smithi162.stdout:6/401: creat d6/d1a/d34/d3a/d68/d6c/f82 x:0 0 0 2024-04-27T04:10:05.710 INFO:tasks.workunit.client.1.smithi181.stdout:6/478: chown d7/dc/d1e/d66/l81 680606 1 2024-04-27T04:10:05.710 INFO:tasks.workunit.client.0.smithi162.stdout:6/402: chown d6/d1a/d34/d3a/d60/d74/f81 35 1 2024-04-27T04:10:05.712 INFO:tasks.workunit.client.0.smithi162.stdout:6/403: chown d6/d1a/d34/d3a/d68/d6c/f53 30 1 2024-04-27T04:10:05.712 INFO:tasks.workunit.client.1.smithi181.stdout:0/717: creat dc/d1f/d3b/dd0/ffb x:0 0 0 2024-04-27T04:10:05.713 INFO:tasks.workunit.client.0.smithi162.stdout:8/416: link d0/d10/d2b/d2d/d2f/d6b/f7a d0/d10/d25/f93 0 2024-04-27T04:10:05.714 INFO:tasks.workunit.client.0.smithi162.stdout:1/373: rmdir d0/d11/d44 39 2024-04-27T04:10:05.714 INFO:tasks.workunit.client.0.smithi162.stdout:8/417: read - d0/d10/d2b/d2d/d2f/d39/f5a zero size 2024-04-27T04:10:05.715 INFO:tasks.workunit.client.1.smithi181.stdout:9/558: dread d0/d44/d4b/d43/d88/fb8 [0,4194304] 0 2024-04-27T04:10:05.715 INFO:tasks.workunit.client.0.smithi162.stdout:1/374: fdatasync d0/d7/d3c/d54/f60 0 2024-04-27T04:10:05.718 INFO:tasks.workunit.client.0.smithi162.stdout:6/404: creat d6/d1a/d34/d7b/f83 x:0 0 0 2024-04-27T04:10:05.719 INFO:tasks.workunit.client.0.smithi162.stdout:9/381: sync 2024-04-27T04:10:05.719 INFO:tasks.workunit.client.0.smithi162.stdout:8/418: symlink d0/d10/d25/d65/l94 0 2024-04-27T04:10:05.719 INFO:tasks.workunit.client.0.smithi162.stdout:6/405: read d6/d1a/f2b [1945043,18744] 0 2024-04-27T04:10:05.721 INFO:tasks.workunit.client.0.smithi162.stdout:6/406: write d6/f30 [1288648,104286] 0 2024-04-27T04:10:05.722 INFO:tasks.workunit.client.0.smithi162.stdout:1/375: unlink d0/d7/l49 0 2024-04-27T04:10:05.722 INFO:tasks.workunit.client.0.smithi162.stdout:0/440: dread d5/f3b [0,4194304] 0 2024-04-27T04:10:05.724 INFO:tasks.workunit.client.0.smithi162.stdout:1/376: write d0/f1b [2273951,105817] 0 2024-04-27T04:10:05.725 INFO:tasks.workunit.client.0.smithi162.stdout:0/441: write d5/f16 [1683173,11693] 0 2024-04-27T04:10:05.725 INFO:tasks.workunit.client.1.smithi181.stdout:4/561: rmdir d0/d15/de/d19/d8a 39 2024-04-27T04:10:05.730 INFO:tasks.workunit.client.0.smithi162.stdout:0/442: dread d5/dc/d1e/f21 [0,4194304] 0 2024-04-27T04:10:05.730 INFO:tasks.workunit.client.0.smithi162.stdout:3/385: truncate d1/d18/f1f 811034 0 2024-04-27T04:10:05.731 INFO:tasks.workunit.client.0.smithi162.stdout:3/386: dread - d1/d18/d23/d58/d63/f74 zero size 2024-04-27T04:10:05.733 INFO:tasks.workunit.client.1.smithi181.stdout:6/479: dread d7/dc/d1e/d5c/f65 [0,4194304] 0 2024-04-27T04:10:05.735 INFO:tasks.workunit.client.0.smithi162.stdout:2/332: write d0/d4/d2a/f3e [243337,5665] 0 2024-04-27T04:10:05.737 INFO:tasks.workunit.client.1.smithi181.stdout:9/559: creat d0/d4d/d85/fc3 x:0 0 0 2024-04-27T04:10:05.738 INFO:tasks.workunit.client.0.smithi162.stdout:5/406: dwrite d8/d9/d5b/f7f [0,4194304] 0 2024-04-27T04:10:05.740 INFO:tasks.workunit.client.1.smithi181.stdout:1/598: mknod d1/dad/db5/d6/da3/d1e/cbf 0 2024-04-27T04:10:05.744 INFO:tasks.workunit.client.0.smithi162.stdout:8/419: rename d0/d10/d44/d70/f89 to d0/de/d6e/d71/f95 0 2024-04-27T04:10:05.744 INFO:tasks.workunit.client.0.smithi162.stdout:8/420: chown d0/f4 8268 1 2024-04-27T04:10:05.745 INFO:tasks.workunit.client.1.smithi181.stdout:4/562: stat d0/d15/d25/d2a/d36/c86 0 2024-04-27T04:10:05.745 INFO:tasks.workunit.client.0.smithi162.stdout:8/421: chown d0/de/d20/d6c/c74 3123821 1 2024-04-27T04:10:05.746 INFO:tasks.workunit.client.1.smithi181.stdout:3/536: write d0/de/d20/d58/f7e [417230,87512] 0 2024-04-27T04:10:05.746 INFO:tasks.workunit.client.0.smithi162.stdout:8/422: stat d0/d10/d25/d65 0 2024-04-27T04:10:05.747 INFO:tasks.workunit.client.1.smithi181.stdout:4/563: chown d0/d15/d25/d40/l56 376359 1 2024-04-27T04:10:05.748 INFO:tasks.workunit.client.0.smithi162.stdout:6/407: symlink d6/d1a/d34/d3a/d68/d6c/d67/l84 0 2024-04-27T04:10:05.749 INFO:tasks.workunit.client.0.smithi162.stdout:8/423: chown d0/d10/d2b/d2e/f40 3523 1 2024-04-27T04:10:05.750 INFO:tasks.workunit.client.0.smithi162.stdout:5/407: dread d8/d2c/f4c [0,4194304] 0 2024-04-27T04:10:05.751 INFO:tasks.workunit.client.0.smithi162.stdout:1/377: creat d0/d11/d3e/d59/f7a x:0 0 0 2024-04-27T04:10:05.753 INFO:tasks.workunit.client.1.smithi181.stdout:7/600: dwrite d1/d22/d15/d1f/d34/f3c [0,4194304] 0 2024-04-27T04:10:05.762 INFO:tasks.workunit.client.0.smithi162.stdout:3/387: creat d1/d18/d5d/d5f/d66/f7a x:0 0 0 2024-04-27T04:10:05.762 INFO:tasks.workunit.client.0.smithi162.stdout:9/382: creat d3/d13/d2d/d3b/d69/f7d x:0 0 0 2024-04-27T04:10:05.766 INFO:tasks.workunit.client.1.smithi181.stdout:0/718: rename dc/d1f/d6b/db9/ld4 to dc/d1f/d3b/d4a/d95/d26/lfc 0 2024-04-27T04:10:05.768 INFO:tasks.workunit.client.0.smithi162.stdout:3/388: dread d1/d18/f12 [4194304,4194304] 0 2024-04-27T04:10:05.769 INFO:tasks.workunit.client.1.smithi181.stdout:4/564: dwrite d0/d15/d76/f95 [0,4194304] 0 2024-04-27T04:10:05.771 INFO:tasks.workunit.client.1.smithi181.stdout:9/560: mkdir d0/d44/d4b/d3d/dbd/dc4 0 2024-04-27T04:10:05.772 INFO:tasks.workunit.client.0.smithi162.stdout:4/404: sync 2024-04-27T04:10:05.772 INFO:tasks.workunit.client.0.smithi162.stdout:7/334: sync 2024-04-27T04:10:05.773 INFO:tasks.workunit.client.0.smithi162.stdout:7/335: write d3/d5/d9/f59 [500074,96199] 0 2024-04-27T04:10:05.776 INFO:tasks.workunit.client.0.smithi162.stdout:8/424: mkdir d0/d10/d25/d65/d96 0 2024-04-27T04:10:05.780 INFO:tasks.workunit.client.0.smithi162.stdout:1/378: creat d0/d7/d3c/d6e/f7b x:0 0 0 2024-04-27T04:10:05.785 INFO:tasks.workunit.client.0.smithi162.stdout:5/408: dread d8/d1a/d23/f39 [0,4194304] 0 2024-04-27T04:10:05.786 INFO:tasks.workunit.client.0.smithi162.stdout:0/443: link d5/dc/fe d5/d26/f8d 0 2024-04-27T04:10:05.791 INFO:tasks.workunit.client.1.smithi181.stdout:6/480: write d7/d31/d45/f6b [503659,26101] 0 2024-04-27T04:10:05.794 INFO:tasks.workunit.client.1.smithi181.stdout:9/561: creat d0/d44/d4b/d43/d89/fc5 x:0 0 0 2024-04-27T04:10:05.798 INFO:tasks.workunit.client.1.smithi181.stdout:3/537: creat d0/d6c/d52/d44/d56/faa x:0 0 0 2024-04-27T04:10:05.802 INFO:tasks.workunit.client.1.smithi181.stdout:9/562: dwrite d0/d44/d4f/d9d/fb4 [0,4194304] 0 2024-04-27T04:10:05.807 INFO:tasks.workunit.client.1.smithi181.stdout:0/719: mkdir dc/d1f/d6b/db9/dba/deb/dfd 0 2024-04-27T04:10:05.810 INFO:tasks.workunit.client.1.smithi181.stdout:0/720: read dc/d1f/d3b/d4a/d95/fd8 [2700564,47385] 0 2024-04-27T04:10:05.812 INFO:tasks.workunit.client.0.smithi162.stdout:7/336: rename d3/d5/d5b/d60/d65/f6b to d3/d5/d3c/d4a/f70 0 2024-04-27T04:10:05.817 INFO:tasks.workunit.client.0.smithi162.stdout:2/333: truncate d0/d4/f3a 3974542 0 2024-04-27T04:10:05.818 INFO:tasks.workunit.client.0.smithi162.stdout:2/334: read - d0/d4/d2a/d41/f54 zero size 2024-04-27T04:10:05.819 INFO:tasks.workunit.client.0.smithi162.stdout:3/389: write d1/d18/f4d [2773962,115900] 0 2024-04-27T04:10:05.819 INFO:tasks.workunit.client.0.smithi162.stdout:5/409: chown d8/d1a/d23/d40/d4b/d66/c76 193707 1 2024-04-27T04:10:05.821 INFO:tasks.workunit.client.1.smithi181.stdout:6/481: dwrite d7/dc/d1e/f2a [0,4194304] 0 2024-04-27T04:10:05.822 INFO:tasks.workunit.client.1.smithi181.stdout:4/565: write d0/d15/d25/d2b/d6d/d82/f8b [3596205,126132] 0 2024-04-27T04:10:05.822 INFO:tasks.workunit.client.0.smithi162.stdout:0/444: mknod d5/dc/d1e/d56/c8e 0 2024-04-27T04:10:05.823 INFO:tasks.workunit.client.0.smithi162.stdout:0/445: chown d5/dc/d1e/f21 315681048 1 2024-04-27T04:10:05.827 INFO:tasks.workunit.client.1.smithi181.stdout:3/538: mknod d0/d6c/d66/d88/cab 0 2024-04-27T04:10:05.827 INFO:tasks.workunit.client.1.smithi181.stdout:4/566: dread d0/d15/f2 [0,4194304] 0 2024-04-27T04:10:05.834 INFO:tasks.workunit.client.0.smithi162.stdout:9/383: dwrite d3/f21 [0,4194304] 0 2024-04-27T04:10:05.851 INFO:tasks.workunit.client.1.smithi181.stdout:5/706: sync 2024-04-27T04:10:05.852 INFO:tasks.workunit.client.0.smithi162.stdout:8/425: rename d0/de/d6e/c63 to d0/d10/d25/d7d/c97 0 2024-04-27T04:10:05.853 INFO:tasks.workunit.client.0.smithi162.stdout:8/426: write d0/fd [701463,32840] 0 2024-04-27T04:10:05.855 INFO:tasks.workunit.client.1.smithi181.stdout:1/599: rename d1/dad/db5/d6/d6a/la5 to d1/dad/db5/d6/lc0 0 2024-04-27T04:10:05.856 INFO:tasks.workunit.client.1.smithi181.stdout:1/600: fsync d1/dad/db5/d6/da3/d60/f88 0 2024-04-27T04:10:05.857 INFO:tasks.workunit.client.0.smithi162.stdout:7/337: mknod d3/d5/d3c/d4d/c71 0 2024-04-27T04:10:05.859 INFO:tasks.workunit.client.1.smithi181.stdout:3/539: write d0/f4e [3805902,32411] 0 2024-04-27T04:10:05.861 INFO:tasks.workunit.client.0.smithi162.stdout:2/335: symlink d0/d4/d2a/d3b/l66 0 2024-04-27T04:10:05.862 INFO:tasks.workunit.client.0.smithi162.stdout:7/338: dread d3/d5/d3c/d4a/f6a [0,4194304] 0 2024-04-27T04:10:05.869 INFO:tasks.workunit.client.0.smithi162.stdout:5/410: dwrite d8/d1a/f45 [0,4194304] 0 2024-04-27T04:10:05.870 INFO:tasks.workunit.client.1.smithi181.stdout:0/721: dwrite dc/d1f/d3b/d4a/d95/d26/d41/d6f/d7d/f9a [0,4194304] 0 2024-04-27T04:10:05.871 INFO:tasks.workunit.client.0.smithi162.stdout:5/411: write d8/d1a/d29/f4d [1059574,45809] 0 2024-04-27T04:10:05.873 INFO:tasks.workunit.client.1.smithi181.stdout:0/722: dread - dc/d1f/daa/fcb zero size 2024-04-27T04:10:05.875 INFO:tasks.workunit.client.1.smithi181.stdout:6/482: creat d7/dc/d8b/fa7 x:0 0 0 2024-04-27T04:10:05.882 INFO:tasks.workunit.client.0.smithi162.stdout:6/408: getdents d6/d1a/d34/d3a/d68/d6c 0 2024-04-27T04:10:05.883 INFO:tasks.workunit.client.1.smithi181.stdout:9/563: dwrite d0/f55 [0,4194304] 0 2024-04-27T04:10:05.895 INFO:tasks.workunit.client.0.smithi162.stdout:8/427: dwrite d0/d10/d25/f14 [0,4194304] 0 2024-04-27T04:10:05.896 INFO:tasks.workunit.client.0.smithi162.stdout:2/336: symlink d0/d4/d2a/l67 0 2024-04-27T04:10:05.896 INFO:tasks.workunit.client.0.smithi162.stdout:3/390: mknod d1/d21/d47/c7b 0 2024-04-27T04:10:05.897 INFO:tasks.workunit.client.0.smithi162.stdout:2/337: truncate d0/f7 5035605 0 2024-04-27T04:10:05.898 INFO:tasks.workunit.client.1.smithi181.stdout:4/567: creat d0/d15/d25/d2a/da0/fbb x:0 0 0 2024-04-27T04:10:05.898 INFO:tasks.workunit.client.0.smithi162.stdout:2/338: write d0/f44 [2309215,112257] 0 2024-04-27T04:10:05.901 INFO:tasks.workunit.client.0.smithi162.stdout:7/339: rename d3/d5/d5b/d60/f61 to d3/d5/d9/d4b/de/d2b/f72 0 2024-04-27T04:10:05.902 INFO:tasks.workunit.client.0.smithi162.stdout:7/340: write d3/d5/d9/d4b/f35 [4415328,82067] 0 2024-04-27T04:10:05.908 INFO:tasks.workunit.client.0.smithi162.stdout:5/412: fdatasync d8/d1a/f45 0 2024-04-27T04:10:05.908 INFO:tasks.workunit.client.1.smithi181.stdout:0/723: mknod dc/d1f/d3b/db2/cfe 0 2024-04-27T04:10:05.908 INFO:tasks.workunit.client.0.smithi162.stdout:5/413: chown d8/l2b 0 1 2024-04-27T04:10:05.913 INFO:tasks.workunit.client.0.smithi162.stdout:4/405: sync 2024-04-27T04:10:05.913 INFO:tasks.workunit.client.1.smithi181.stdout:5/707: creat d4/da3/dbc/db5/ffa x:0 0 0 2024-04-27T04:10:05.913 INFO:tasks.workunit.client.0.smithi162.stdout:4/406: write d2/da/dd/f57 [786964,28716] 0 2024-04-27T04:10:05.915 INFO:tasks.workunit.client.1.smithi181.stdout:5/708: truncate d4/d10/feb 658196 0 2024-04-27T04:10:05.915 INFO:tasks.workunit.client.1.smithi181.stdout:7/601: rename d1/d22/d15/d5f/d93/fb4 to d1/d22/d15/d5f/fb9 0 2024-04-27T04:10:05.915 INFO:tasks.workunit.client.1.smithi181.stdout:0/724: dwrite dc/d1f/d3b/f44 [0,4194304] 0 2024-04-27T04:10:05.918 INFO:tasks.workunit.client.1.smithi181.stdout:1/601: fsync d1/dad/db5/d6/da3/f1f 0 2024-04-27T04:10:05.918 INFO:tasks.workunit.client.0.smithi162.stdout:6/409: write d6/f62 [5337573,44723] 0 2024-04-27T04:10:05.923 INFO:tasks.workunit.client.0.smithi162.stdout:8/428: creat d0/d10/d25/d7d/f98 x:0 0 0 2024-04-27T04:10:05.925 INFO:tasks.workunit.client.0.smithi162.stdout:3/391: creat d1/d18/d23/d2f/f7c x:0 0 0 2024-04-27T04:10:05.931 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:10:05 smithi162 ceph-mon[26841]: pgmap v268: 65 pgs: 65 active+clean; 1.9 GiB data, 7.4 GiB used, 529 GiB / 536 GiB avail; 51 MiB/s rd, 226 MiB/s wr, 364 op/s 2024-04-27T04:10:05.931 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:10:05 smithi162 ceph-mon[26841]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' 2024-04-27T04:10:05.931 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:10:05 smithi162 ceph-mon[26841]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' 2024-04-27T04:10:05.931 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:10:05 smithi162 ceph-mon[26841]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T04:10:05.931 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:10:05 smithi162 ceph-mon[26841]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T04:10:05.931 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:10:05 smithi162 ceph-mon[26841]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' 2024-04-27T04:10:05.931 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:10:05 smithi162 ceph-mon[26841]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T04:10:05.932 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:10:05 smithi162 ceph-mon[26841]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "mgr fail", "who": "smithi162.oennfn"}]: dispatch 2024-04-27T04:10:05.932 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:10:05 smithi162 ceph-mon[26841]: Activating manager daemon smithi181.vtwuhz 2024-04-27T04:10:05.932 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:10:05 smithi162 ceph-mon[26841]: osdmap e47: 6 total, 6 up, 6 in 2024-04-27T04:10:05.932 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:10:05 smithi162 ceph-mon[26841]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' cmd='[{"prefix": "mgr fail", "who": "smithi162.oennfn"}]': finished 2024-04-27T04:10:05.932 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:10:05 smithi162 ceph-mon[26841]: mgrmap e21: smithi181.vtwuhz(active, starting, since 0.0106877s) 2024-04-27T04:10:05.932 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:10:05 smithi162 ceph-mon[26841]: from='mgr.24367 172.21.15.181:0/3181919980' entity='mgr.smithi181.vtwuhz' cmd=[{"prefix": "mon metadata", "id": "smithi162"}]: dispatch 2024-04-27T04:10:05.932 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:10:05 smithi162 ceph-mon[26841]: from='mgr.24367 172.21.15.181:0/3181919980' entity='mgr.smithi181.vtwuhz' cmd=[{"prefix": "mon metadata", "id": "smithi181"}]: dispatch 2024-04-27T04:10:05.932 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:10:05 smithi162 ceph-mon[26841]: from='mgr.24367 172.21.15.181:0/3181919980' entity='mgr.smithi181.vtwuhz' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi162.xcnapk"}]: dispatch 2024-04-27T04:10:05.932 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:10:05 smithi162 ceph-mon[26841]: from='mgr.24367 172.21.15.181:0/3181919980' entity='mgr.smithi181.vtwuhz' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi162.gzlyjj"}]: dispatch 2024-04-27T04:10:05.932 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:10:05 smithi162 ceph-mon[26841]: from='mgr.24367 172.21.15.181:0/3181919980' entity='mgr.smithi181.vtwuhz' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi181.dzomgh"}]: dispatch 2024-04-27T04:10:05.932 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:10:05 smithi162 ceph-mon[26841]: from='mgr.24367 172.21.15.181:0/3181919980' entity='mgr.smithi181.vtwuhz' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi181.bkkcdm"}]: dispatch 2024-04-27T04:10:05.932 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:10:05 smithi162 ceph-mon[26841]: from='mgr.24367 172.21.15.181:0/3181919980' entity='mgr.smithi181.vtwuhz' cmd=[{"prefix": "mgr metadata", "who": "smithi181.vtwuhz", "id": "smithi181.vtwuhz"}]: dispatch 2024-04-27T04:10:05.932 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:10:05 smithi162 ceph-mon[26841]: from='mgr.24367 172.21.15.181:0/3181919980' entity='mgr.smithi181.vtwuhz' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-27T04:10:05.932 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:10:05 smithi162 ceph-mon[26841]: from='mgr.24367 172.21.15.181:0/3181919980' entity='mgr.smithi181.vtwuhz' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-27T04:10:05.933 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:10:05 smithi162 ceph-mon[26841]: from='mgr.24367 172.21.15.181:0/3181919980' entity='mgr.smithi181.vtwuhz' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-27T04:10:05.933 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:10:05 smithi162 ceph-mon[26841]: from='mgr.24367 172.21.15.181:0/3181919980' entity='mgr.smithi181.vtwuhz' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-27T04:10:05.933 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:10:05 smithi162 ceph-mon[26841]: from='mgr.24367 172.21.15.181:0/3181919980' entity='mgr.smithi181.vtwuhz' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-27T04:10:05.933 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:10:05 smithi162 ceph-mon[26841]: from='mgr.24367 172.21.15.181:0/3181919980' entity='mgr.smithi181.vtwuhz' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-27T04:10:05.933 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:10:05 smithi162 ceph-mon[26841]: from='mgr.24367 172.21.15.181:0/3181919980' entity='mgr.smithi181.vtwuhz' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-04-27T04:10:05.933 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:10:05 smithi162 ceph-mon[26841]: from='mgr.24367 172.21.15.181:0/3181919980' entity='mgr.smithi181.vtwuhz' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-04-27T04:10:05.933 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:10:05 smithi162 ceph-mon[26841]: from='mgr.24367 172.21.15.181:0/3181919980' entity='mgr.smithi181.vtwuhz' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-04-27T04:10:05.933 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:10:05 smithi162 ceph-mon[26841]: Manager daemon smithi181.vtwuhz is now available 2024-04-27T04:10:05.933 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:10:05 smithi162 ceph-mon[26841]: from='mgr.24367 172.21.15.181:0/3181919980' entity='mgr.smithi181.vtwuhz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T04:10:05.933 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:10:05 smithi162 ceph-mon[26841]: from='mgr.24367 172.21.15.181:0/3181919980' entity='mgr.smithi181.vtwuhz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-27T04:10:05.933 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:10:05 smithi162 ceph-mon[26841]: from='mgr.24367 172.21.15.181:0/3181919980' entity='mgr.smithi181.vtwuhz' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi181.vtwuhz/mirror_snapshot_schedule"}]: dispatch 2024-04-27T04:10:05.935 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:10:05 smithi181 ceph-mon[33774]: pgmap v268: 65 pgs: 65 active+clean; 1.9 GiB data, 7.4 GiB used, 529 GiB / 536 GiB avail; 51 MiB/s rd, 226 MiB/s wr, 364 op/s 2024-04-27T04:10:05.935 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:10:05 smithi181 ceph-mon[33774]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' 2024-04-27T04:10:05.935 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:10:05 smithi181 ceph-mon[33774]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' 2024-04-27T04:10:05.935 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:10:05 smithi181 ceph-mon[33774]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T04:10:05.935 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:10:05 smithi181 ceph-mon[33774]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T04:10:05.936 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:10:05 smithi181 ceph-mon[33774]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' 2024-04-27T04:10:05.936 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:10:05 smithi181 ceph-mon[33774]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T04:10:05.936 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:10:05 smithi181 ceph-mon[33774]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "mgr fail", "who": "smithi162.oennfn"}]: dispatch 2024-04-27T04:10:05.936 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:10:05 smithi181 ceph-mon[33774]: Activating manager daemon smithi181.vtwuhz 2024-04-27T04:10:05.936 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:10:05 smithi181 ceph-mon[33774]: osdmap e47: 6 total, 6 up, 6 in 2024-04-27T04:10:05.936 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:10:05 smithi181 ceph-mon[33774]: from='mgr.14210 172.21.15.162:0/157500169' entity='mgr.smithi162.oennfn' cmd='[{"prefix": "mgr fail", "who": "smithi162.oennfn"}]': finished 2024-04-27T04:10:05.936 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:10:05 smithi181 ceph-mon[33774]: mgrmap e21: smithi181.vtwuhz(active, starting, since 0.0106877s) 2024-04-27T04:10:05.936 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:10:05 smithi181 ceph-mon[33774]: from='mgr.24367 172.21.15.181:0/3181919980' entity='mgr.smithi181.vtwuhz' cmd=[{"prefix": "mon metadata", "id": "smithi162"}]: dispatch 2024-04-27T04:10:05.936 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:10:05 smithi181 ceph-mon[33774]: from='mgr.24367 172.21.15.181:0/3181919980' entity='mgr.smithi181.vtwuhz' cmd=[{"prefix": "mon metadata", "id": "smithi181"}]: dispatch 2024-04-27T04:10:05.936 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:10:05 smithi181 ceph-mon[33774]: from='mgr.24367 172.21.15.181:0/3181919980' entity='mgr.smithi181.vtwuhz' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi162.xcnapk"}]: dispatch 2024-04-27T04:10:05.936 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:10:05 smithi181 ceph-mon[33774]: from='mgr.24367 172.21.15.181:0/3181919980' entity='mgr.smithi181.vtwuhz' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi162.gzlyjj"}]: dispatch 2024-04-27T04:10:05.936 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:10:05 smithi181 ceph-mon[33774]: from='mgr.24367 172.21.15.181:0/3181919980' entity='mgr.smithi181.vtwuhz' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi181.dzomgh"}]: dispatch 2024-04-27T04:10:05.936 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:10:05 smithi181 ceph-mon[33774]: from='mgr.24367 172.21.15.181:0/3181919980' entity='mgr.smithi181.vtwuhz' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi181.bkkcdm"}]: dispatch 2024-04-27T04:10:05.936 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:10:05 smithi181 ceph-mon[33774]: from='mgr.24367 172.21.15.181:0/3181919980' entity='mgr.smithi181.vtwuhz' cmd=[{"prefix": "mgr metadata", "who": "smithi181.vtwuhz", "id": "smithi181.vtwuhz"}]: dispatch 2024-04-27T04:10:05.937 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:10:05 smithi181 ceph-mon[33774]: from='mgr.24367 172.21.15.181:0/3181919980' entity='mgr.smithi181.vtwuhz' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-27T04:10:05.937 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:10:05 smithi181 ceph-mon[33774]: from='mgr.24367 172.21.15.181:0/3181919980' entity='mgr.smithi181.vtwuhz' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-27T04:10:05.937 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:10:05 smithi181 ceph-mon[33774]: from='mgr.24367 172.21.15.181:0/3181919980' entity='mgr.smithi181.vtwuhz' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-27T04:10:05.937 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:10:05 smithi181 ceph-mon[33774]: from='mgr.24367 172.21.15.181:0/3181919980' entity='mgr.smithi181.vtwuhz' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-27T04:10:05.937 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:10:05 smithi181 ceph-mon[33774]: from='mgr.24367 172.21.15.181:0/3181919980' entity='mgr.smithi181.vtwuhz' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-27T04:10:05.937 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:10:05 smithi181 ceph-mon[33774]: from='mgr.24367 172.21.15.181:0/3181919980' entity='mgr.smithi181.vtwuhz' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-27T04:10:05.937 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:10:05 smithi181 ceph-mon[33774]: from='mgr.24367 172.21.15.181:0/3181919980' entity='mgr.smithi181.vtwuhz' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-04-27T04:10:05.937 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:10:05 smithi181 ceph-mon[33774]: from='mgr.24367 172.21.15.181:0/3181919980' entity='mgr.smithi181.vtwuhz' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-04-27T04:10:05.937 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:10:05 smithi181 ceph-mon[33774]: from='mgr.24367 172.21.15.181:0/3181919980' entity='mgr.smithi181.vtwuhz' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-04-27T04:10:05.937 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:10:05 smithi181 ceph-mon[33774]: Manager daemon smithi181.vtwuhz is now available 2024-04-27T04:10:05.937 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:10:05 smithi181 ceph-mon[33774]: from='mgr.24367 172.21.15.181:0/3181919980' entity='mgr.smithi181.vtwuhz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T04:10:05.937 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:10:05 smithi181 ceph-mon[33774]: from='mgr.24367 172.21.15.181:0/3181919980' entity='mgr.smithi181.vtwuhz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-27T04:10:05.937 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:10:05 smithi181 ceph-mon[33774]: from='mgr.24367 172.21.15.181:0/3181919980' entity='mgr.smithi181.vtwuhz' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi181.vtwuhz/mirror_snapshot_schedule"}]: dispatch 2024-04-27T04:10:05.938 INFO:tasks.workunit.client.0.smithi162.stdout:0/446: dwrite d5/dc/f22 [4194304,4194304] 0 2024-04-27T04:10:05.942 INFO:tasks.workunit.client.0.smithi162.stdout:7/341: write d3/d5/d5b/f6c [1035393,51643] 0 2024-04-27T04:10:05.945 INFO:tasks.workunit.client.0.smithi162.stdout:0/447: dwrite d5/dc/d1e/d41/f7e [0,4194304] 0 2024-04-27T04:10:05.947 INFO:tasks.workunit.client.0.smithi162.stdout:5/414: unlink d8/d1a/d29/f43 0 2024-04-27T04:10:05.948 INFO:tasks.workunit.client.0.smithi162.stdout:5/415: dread - d8/d9/d55/f84 zero size 2024-04-27T04:10:05.956 INFO:tasks.workunit.client.0.smithi162.stdout:3/392: mknod d1/d18/d5d/d5f/d66/c7d 0 2024-04-27T04:10:05.966 INFO:tasks.workunit.client.0.smithi162.stdout:3/393: dwrite d1/d21/f6d [0,4194304] 0 2024-04-27T04:10:05.966 INFO:tasks.workunit.client.0.smithi162.stdout:9/384: getdents d3/d11/d45 0 2024-04-27T04:10:05.966 INFO:tasks.workunit.client.0.smithi162.stdout:6/410: symlink d6/d1a/d34/d7f/d80/l85 0 2024-04-27T04:10:05.968 INFO:tasks.workunit.client.0.smithi162.stdout:8/429: unlink d0/d10/d2b/d2d/d2f/d6b/f7a 0 2024-04-27T04:10:05.971 INFO:tasks.workunit.client.0.smithi162.stdout:0/448: dwrite d5/dc/d1e/d41/f83 [0,4194304] 0 2024-04-27T04:10:05.972 INFO:tasks.workunit.client.0.smithi162.stdout:8/430: fsync d0/d10/d25/d7d/f98 0 2024-04-27T04:10:05.973 INFO:tasks.workunit.client.0.smithi162.stdout:2/339: truncate d0/d4/f15 3852484 0 2024-04-27T04:10:05.976 INFO:tasks.workunit.client.0.smithi162.stdout:3/394: rmdir d1/d18/d5d 39 2024-04-27T04:10:05.980 INFO:tasks.workunit.client.0.smithi162.stdout:3/395: readlink d1/d18/d23/l38 0 2024-04-27T04:10:05.981 INFO:tasks.workunit.client.0.smithi162.stdout:6/411: dwrite d6/db/f48 [0,4194304] 0 2024-04-27T04:10:05.984 INFO:tasks.workunit.client.0.smithi162.stdout:4/407: dwrite d2/d25/f7 [0,4194304] 0 2024-04-27T04:10:05.987 INFO:tasks.workunit.client.0.smithi162.stdout:2/340: symlink d0/d20/d2e/l68 0 2024-04-27T04:10:05.989 INFO:tasks.workunit.client.0.smithi162.stdout:6/412: chown d6/d1a/d34/d3a/f3d 24 1 2024-04-27T04:10:05.992 INFO:tasks.workunit.client.0.smithi162.stdout:3/396: creat d1/d45/f7e x:0 0 0 2024-04-27T04:10:05.997 INFO:tasks.workunit.client.0.smithi162.stdout:4/408: creat d2/d54/d7a/f83 x:0 0 0 2024-04-27T04:10:05.997 INFO:tasks.workunit.client.0.smithi162.stdout:3/397: fsync d1/d18/d23/f28 0 2024-04-27T04:10:05.998 INFO:tasks.workunit.client.0.smithi162.stdout:0/449: dwrite d5/dc/f22 [4194304,4194304] 0 2024-04-27T04:10:05.999 INFO:tasks.workunit.client.0.smithi162.stdout:6/413: mknod d6/d1a/d34/d50/c86 0 2024-04-27T04:10:05.999 INFO:tasks.workunit.client.0.smithi162.stdout:2/341: write d0/d4/d2a/f36 [4499523,14733] 0 2024-04-27T04:10:06.000 INFO:tasks.workunit.client.0.smithi162.stdout:5/416: dwrite d8/f20 [0,4194304] 0 2024-04-27T04:10:06.002 INFO:tasks.workunit.client.0.smithi162.stdout:4/409: dread - d2/d25/d1c/d39/f71 zero size 2024-04-27T04:10:06.002 INFO:tasks.workunit.client.0.smithi162.stdout:9/385: dwrite d3/fc [0,4194304] 0 2024-04-27T04:10:06.004 INFO:tasks.workunit.client.1.smithi181.stdout:2/638: sync 2024-04-27T04:10:06.004 INFO:tasks.workunit.client.1.smithi181.stdout:1/602: fdatasync d1/dad/db5/fa1 0 2024-04-27T04:10:06.004 INFO:tasks.workunit.client.1.smithi181.stdout:4/568: truncate d0/d15/f50 408187 0 2024-04-27T04:10:06.004 INFO:tasks.workunit.client.0.smithi162.stdout:2/342: truncate d0/d4/d17/f2f 8879190 0 2024-04-27T04:10:06.004 INFO:tasks.workunit.client.0.smithi162.stdout:3/398: creat d1/d65/f7f x:0 0 0 2024-04-27T04:10:06.005 INFO:tasks.workunit.client.0.smithi162.stdout:4/410: write d2/da/d1a/f1b [1166751,82037] 0 2024-04-27T04:10:06.006 INFO:tasks.workunit.client.0.smithi162.stdout:6/414: truncate d6/d1a/d34/d3a/d68/f78 688649 0 2024-04-27T04:10:06.006 INFO:tasks.workunit.client.1.smithi181.stdout:8/678: sync 2024-04-27T04:10:06.006 INFO:tasks.workunit.client.1.smithi181.stdout:5/709: fdatasync d4/d10/d72/f77 0 2024-04-27T04:10:06.008 INFO:tasks.workunit.client.0.smithi162.stdout:3/399: write d1/d21/f60 [113953,5447] 0 2024-04-27T04:10:06.009 INFO:tasks.workunit.client.0.smithi162.stdout:2/343: chown d0/d4/d2a/l58 0 1 2024-04-27T04:10:06.009 INFO:tasks.workunit.client.0.smithi162.stdout:6/415: chown d6/f25 55449958 1 2024-04-27T04:10:06.009 INFO:tasks.workunit.client.1.smithi181.stdout:5/710: fsync d4/d10/d72/fe3 0 2024-04-27T04:10:06.010 INFO:tasks.workunit.client.1.smithi181.stdout:4/569: truncate d0/d15/d76/f95 4924562 0 2024-04-27T04:10:06.011 INFO:tasks.workunit.client.1.smithi181.stdout:0/725: rmdir dc/d1f/d6b/db9/dba 39 2024-04-27T04:10:06.012 INFO:tasks.workunit.client.1.smithi181.stdout:4/570: dread - d0/d15/d25/d2a/d36/fb4 zero size 2024-04-27T04:10:06.013 INFO:tasks.workunit.client.1.smithi181.stdout:3/540: truncate d0/de/d20/f5d 1315977 0 2024-04-27T04:10:06.014 INFO:tasks.workunit.client.1.smithi181.stdout:3/541: chown d0/d6c/d83 56774 1 2024-04-27T04:10:06.016 INFO:tasks.workunit.client.1.smithi181.stdout:7/602: read d1/d2/d35/d67/f69 [3189007,115688] 0 2024-04-27T04:10:06.025 INFO:tasks.workunit.client.1.smithi181.stdout:5/711: mknod d4/de/d9d/cfb 0 2024-04-27T04:10:06.038 INFO:tasks.workunit.client.1.smithi181.stdout:1/603: mknod d1/dad/db5/cc1 0 2024-04-27T04:10:06.041 INFO:tasks.workunit.client.1.smithi181.stdout:6/483: rename d7/dc/d1c/d88/f9e to d7/dc/d1c/fa8 0 2024-04-27T04:10:06.044 INFO:tasks.workunit.client.1.smithi181.stdout:7/603: mkdir d1/d22/d15/d5f/dba 0 2024-04-27T04:10:06.050 INFO:tasks.workunit.client.1.smithi181.stdout:3/542: write d0/d6c/d52/d44/d56/f76 [470696,126322] 0 2024-04-27T04:10:06.051 INFO:tasks.workunit.client.1.smithi181.stdout:2/639: dwrite d4/d9/d35/f3a [0,4194304] 0 2024-04-27T04:10:06.054 INFO:tasks.workunit.client.0.smithi162.stdout:1/379: sync 2024-04-27T04:10:06.056 INFO:tasks.workunit.client.1.smithi181.stdout:7/604: chown d1/d2/d1c/d8f/c78 186999178 1 2024-04-27T04:10:06.056 INFO:tasks.workunit.client.1.smithi181.stdout:5/712: symlink d4/de/d2c/d46/d99/dad/db2/lfc 0 2024-04-27T04:10:06.060 INFO:tasks.workunit.client.1.smithi181.stdout:4/571: dwrite d0/d15/d76/da4/faf [0,4194304] 0 2024-04-27T04:10:06.060 INFO:tasks.workunit.client.1.smithi181.stdout:2/640: write d4/d9/d4f/fac [149475,6502] 0 2024-04-27T04:10:06.060 INFO:tasks.workunit.client.1.smithi181.stdout:1/604: symlink d1/dad/db5/d6/da3/d1e/lc2 0 2024-04-27T04:10:06.062 INFO:tasks.workunit.client.1.smithi181.stdout:4/572: readlink d0/d15/de/d19/l24 0 2024-04-27T04:10:06.063 INFO:tasks.workunit.client.0.smithi162.stdout:2/344: chown d0/d4/d2a/d41/f54 820294 1 2024-04-27T04:10:06.064 INFO:tasks.workunit.client.1.smithi181.stdout:4/573: chown d0/d15/d5c/d69/c3a 75708 1 2024-04-27T04:10:06.064 INFO:tasks.workunit.client.0.smithi162.stdout:4/411: mknod d2/d25/c84 0 2024-04-27T04:10:06.065 INFO:tasks.workunit.client.0.smithi162.stdout:6/416: readlink d6/d1a/d34/d50/l72 0 2024-04-27T04:10:06.068 INFO:tasks.workunit.client.0.smithi162.stdout:5/417: dread d8/d9/d5b/f71 [0,4194304] 0 2024-04-27T04:10:06.069 INFO:tasks.workunit.client.1.smithi181.stdout:2/641: dread d4/f50 [4194304,4194304] 0 2024-04-27T04:10:06.070 INFO:tasks.workunit.client.1.smithi181.stdout:0/726: getdents dc/d93 0 2024-04-27T04:10:06.071 INFO:tasks.workunit.client.0.smithi162.stdout:3/400: dwrite d1/d21/f6d [0,4194304] 0 2024-04-27T04:10:06.071 INFO:tasks.workunit.client.1.smithi181.stdout:8/679: getdents d1/d6/d2f/d30/d9d/dea/db3 0 2024-04-27T04:10:06.072 INFO:tasks.workunit.client.1.smithi181.stdout:5/713: mknod d4/de/d2c/d46/d4d/d8e/d54/cfd 0 2024-04-27T04:10:06.072 INFO:tasks.workunit.client.1.smithi181.stdout:3/543: rename d0/de/d20/f30 to d0/d6c/d83/d98/fac 0 2024-04-27T04:10:06.073 INFO:tasks.workunit.client.0.smithi162.stdout:1/380: readlink d0/d7/d2a/l46 0 2024-04-27T04:10:06.074 INFO:tasks.workunit.client.0.smithi162.stdout:8/431: dwrite d0/d10/d25/f93 [0,4194304] 0 2024-04-27T04:10:06.075 INFO:tasks.workunit.client.1.smithi181.stdout:7/605: fsync d1/d22/d15/d29/f6d 0 2024-04-27T04:10:06.080 INFO:tasks.workunit.client.1.smithi181.stdout:4/574: creat d0/d15/d25/d2a/d61/fbc x:0 0 0 2024-04-27T04:10:06.082 INFO:tasks.workunit.client.0.smithi162.stdout:0/450: getdents d5/dc/d1e/d56 0 2024-04-27T04:10:06.090 INFO:tasks.workunit.client.0.smithi162.stdout:5/418: mkdir d8/d1a/d23/d40/d4b/d58/d8d 0 2024-04-27T04:10:06.090 INFO:tasks.workunit.client.0.smithi162.stdout:2/345: dread d0/d4/f16 [0,4194304] 0 2024-04-27T04:10:06.091 INFO:tasks.workunit.client.0.smithi162.stdout:2/346: truncate d0/d4/d17/d2b/f65 660312 0 2024-04-27T04:10:06.091 INFO:tasks.workunit.client.1.smithi181.stdout:8/680: creat d1/d8c/fef x:0 0 0 2024-04-27T04:10:06.092 INFO:tasks.workunit.client.0.smithi162.stdout:1/381: symlink d0/d7/d3c/l7c 0 2024-04-27T04:10:06.094 INFO:tasks.workunit.client.0.smithi162.stdout:8/432: rmdir d0/d10/d76 39 2024-04-27T04:10:06.094 INFO:tasks.workunit.client.0.smithi162.stdout:4/412: dread d2/d25/d1c/f55 [0,4194304] 0 2024-04-27T04:10:06.096 INFO:tasks.workunit.client.1.smithi181.stdout:5/714: symlink d4/de/d2c/d46/d4d/lfe 0 2024-04-27T04:10:06.097 INFO:tasks.workunit.client.1.smithi181.stdout:1/605: dread d1/dad/db5/d6/f73 [0,4194304] 0 2024-04-27T04:10:06.100 INFO:tasks.workunit.client.1.smithi181.stdout:7/606: symlink d1/d2/d1c/d5e/lbb 0 2024-04-27T04:10:06.100 INFO:tasks.workunit.client.1.smithi181.stdout:6/484: getdents d7/dc/d1c/d34/d3f/d7e 0 2024-04-27T04:10:06.101 INFO:tasks.workunit.client.1.smithi181.stdout:7/607: chown d1/df/d1a/d6a 729429 1 2024-04-27T04:10:06.102 INFO:tasks.workunit.client.1.smithi181.stdout:3/544: dread d0/de/f4b [0,4194304] 0 2024-04-27T04:10:06.102 INFO:tasks.workunit.client.0.smithi162.stdout:6/417: rename d6/d1a/d34/l3c to d6/l87 0 2024-04-27T04:10:06.103 INFO:tasks.workunit.client.1.smithi181.stdout:7/608: write d1/d22/d15/f2a [4739124,11307] 0 2024-04-27T04:10:06.103 INFO:tasks.workunit.client.1.smithi181.stdout:3/545: dread - d0/d6c/d52/d44/d56/fa6 zero size 2024-04-27T04:10:06.108 INFO:tasks.workunit.client.0.smithi162.stdout:3/401: creat d1/d18/d5d/d6f/f80 x:0 0 0 2024-04-27T04:10:06.109 INFO:tasks.workunit.client.1.smithi181.stdout:4/575: dread d0/d15/f1a [0,4194304] 0 2024-04-27T04:10:06.111 INFO:tasks.workunit.client.1.smithi181.stdout:4/576: dread - d0/d15/de/d19/d77/d57/f84 zero size 2024-04-27T04:10:06.113 INFO:tasks.workunit.client.1.smithi181.stdout:4/577: truncate d0/d15/d25/d40/f9b 401008 0 2024-04-27T04:10:06.114 INFO:tasks.workunit.client.1.smithi181.stdout:2/642: symlink d4/d9/d16/d45/d9a/lcf 0 2024-04-27T04:10:06.116 INFO:tasks.workunit.client.0.smithi162.stdout:5/419: rename d8/ld to d8/d1a/d23/d40/d4b/l8e 0 2024-04-27T04:10:06.116 INFO:tasks.workunit.client.1.smithi181.stdout:4/578: truncate d0/d15/d25/d2a/da0/fbb 605066 0 2024-04-27T04:10:06.117 INFO:tasks.workunit.client.0.smithi162.stdout:5/420: stat d8/d9/c1c 0 2024-04-27T04:10:06.117 INFO:tasks.workunit.client.1.smithi181.stdout:4/579: fsync d0/f3b 0 2024-04-27T04:10:06.118 INFO:tasks.workunit.client.1.smithi181.stdout:4/580: stat d0/d15/d25/l64 0 2024-04-27T04:10:06.120 INFO:tasks.workunit.client.1.smithi181.stdout:7/609: dread d1/f45 [4194304,4194304] 0 2024-04-27T04:10:06.120 INFO:tasks.workunit.client.0.smithi162.stdout:6/418: creat d6/d1a/f88 x:0 0 0 2024-04-27T04:10:06.122 INFO:tasks.workunit.client.1.smithi181.stdout:7/610: write d1/d2/d35/d67/f96 [399821,87158] 0 2024-04-27T04:10:06.122 INFO:tasks.workunit.client.1.smithi181.stdout:5/715: rmdir d4/de/d2c/d46/d4d/d8e/d67/d71 39 2024-04-27T04:10:06.123 INFO:tasks.workunit.client.0.smithi162.stdout:0/451: dwrite d5/dc/f31 [0,4194304] 0 2024-04-27T04:10:06.126 INFO:tasks.workunit.client.0.smithi162.stdout:3/402: symlink d1/d21/d47/l81 0 2024-04-27T04:10:06.127 INFO:tasks.workunit.client.0.smithi162.stdout:3/403: read d1/d21/f2c [2338954,8763] 0 2024-04-27T04:10:06.128 INFO:tasks.workunit.client.1.smithi181.stdout:1/606: dread d1/dad/db5/d6/db/d2d/d36/f4a [0,4194304] 0 2024-04-27T04:10:06.128 INFO:tasks.workunit.client.0.smithi162.stdout:8/433: mkdir d0/d10/d25/d7d/d99 0 2024-04-27T04:10:06.132 INFO:tasks.workunit.client.1.smithi181.stdout:1/607: readlink d1/dad/db5/l6b 0 2024-04-27T04:10:06.132 INFO:tasks.workunit.client.1.smithi181.stdout:8/681: dwrite d1/f6b [0,4194304] 0 2024-04-27T04:10:06.133 INFO:tasks.workunit.client.0.smithi162.stdout:3/404: dwrite d1/d18/f1d [0,4194304] 0 2024-04-27T04:10:06.138 INFO:tasks.workunit.client.1.smithi181.stdout:7/611: dread d1/d2/d1c/d2d/f3e [0,4194304] 0 2024-04-27T04:10:06.138 INFO:tasks.workunit.client.0.smithi162.stdout:2/347: rename d0/d4/d17/d2b/f65 to d0/d20/d2e/f69 0 2024-04-27T04:10:06.141 INFO:tasks.workunit.client.1.smithi181.stdout:7/612: readlink d1/d2/d1c/d8f/l9b 0 2024-04-27T04:10:06.142 INFO:tasks.workunit.client.1.smithi181.stdout:7/613: write d1/d2/f25 [4237758,114886] 0 2024-04-27T04:10:06.143 INFO:tasks.workunit.client.1.smithi181.stdout:0/727: mkdir dc/d1f/d6b/db9/dba/deb/db3/dff 0 2024-04-27T04:10:06.144 INFO:tasks.workunit.client.0.smithi162.stdout:3/405: dwrite d1/d18/d5d/d5f/f77 [0,4194304] 0 2024-04-27T04:10:06.150 INFO:tasks.workunit.client.0.smithi162.stdout:0/452: creat d5/d26/d69/f8f x:0 0 0 2024-04-27T04:10:06.151 INFO:tasks.workunit.client.0.smithi162.stdout:0/453: chown d5/f58 30 1 2024-04-27T04:10:06.154 INFO:tasks.workunit.client.0.smithi162.stdout:5/421: dwrite d8/d1a/f17 [4194304,4194304] 0 2024-04-27T04:10:06.157 INFO:tasks.workunit.client.1.smithi181.stdout:9/564: sync 2024-04-27T04:10:06.158 INFO:tasks.workunit.client.1.smithi181.stdout:9/565: read d0/f55 [3419546,24747] 0 2024-04-27T04:10:06.162 INFO:tasks.workunit.client.1.smithi181.stdout:8/682: rename d1/d6/d1a/d25/cbe to d1/d6/d59/cf0 0 2024-04-27T04:10:06.162 INFO:tasks.workunit.client.0.smithi162.stdout:3/406: truncate d1/d18/f52 3473929 0 2024-04-27T04:10:06.164 INFO:tasks.workunit.client.0.smithi162.stdout:3/407: chown d1/d65 26387254 1 2024-04-27T04:10:06.165 INFO:tasks.workunit.client.0.smithi162.stdout:5/422: symlink d8/d9/l8f 0 2024-04-27T04:10:06.169 INFO:tasks.workunit.client.0.smithi162.stdout:2/348: mknod d0/d20/d2e/d61/c6a 0 2024-04-27T04:10:06.170 INFO:tasks.workunit.client.0.smithi162.stdout:0/454: dwrite d5/dc/d1e/d41/f4e [4194304,4194304] 0 2024-04-27T04:10:06.170 INFO:tasks.workunit.client.1.smithi181.stdout:5/716: dread d4/da3/fbd [0,4194304] 0 2024-04-27T04:10:06.170 INFO:tasks.workunit.client.1.smithi181.stdout:5/717: readlink d4/d10/d72/l82 0 2024-04-27T04:10:06.170 INFO:tasks.workunit.client.1.smithi181.stdout:7/614: symlink d1/d2/d1c/d56/lbc 0 2024-04-27T04:10:06.171 INFO:tasks.workunit.client.1.smithi181.stdout:0/728: creat dc/d1f/d3b/dd0/f100 x:0 0 0 2024-04-27T04:10:06.171 INFO:tasks.workunit.client.1.smithi181.stdout:2/643: mkdir d4/d9/d35/d9f/dd0 0 2024-04-27T04:10:06.172 INFO:tasks.workunit.client.0.smithi162.stdout:6/419: dwrite d6/f35 [0,4194304] 0 2024-04-27T04:10:06.175 INFO:tasks.workunit.client.1.smithi181.stdout:4/581: truncate d0/d15/d25/d2a/d36/f89 2467806 0 2024-04-27T04:10:06.175 INFO:tasks.workunit.client.1.smithi181.stdout:2/644: chown d4/d9/f1a 15665849 1 2024-04-27T04:10:06.176 INFO:tasks.workunit.client.0.smithi162.stdout:5/423: creat d8/d1a/d23/d40/d4b/f90 x:0 0 0 2024-04-27T04:10:06.178 INFO:tasks.workunit.client.0.smithi162.stdout:5/424: write d8/d1a/f81 [482016,127215] 0 2024-04-27T04:10:06.180 INFO:tasks.workunit.client.0.smithi162.stdout:2/349: symlink d0/d20/d2e/d61/l6b 0 2024-04-27T04:10:06.184 INFO:tasks.workunit.client.0.smithi162.stdout:7/342: sync 2024-04-27T04:10:06.186 INFO:tasks.workunit.client.0.smithi162.stdout:7/343: readlink d3/d5/d9/d4b/de/l34 0 2024-04-27T04:10:06.186 INFO:tasks.workunit.client.0.smithi162.stdout:2/350: dwrite d0/d24/f4d [4194304,4194304] 0 2024-04-27T04:10:06.190 INFO:tasks.workunit.client.0.smithi162.stdout:0/455: creat d5/dc/d1e/d56/f90 x:0 0 0 2024-04-27T04:10:06.191 INFO:tasks.workunit.client.0.smithi162.stdout:0/456: chown d5/dc/d1e/d56/l75 39 1 2024-04-27T04:10:06.196 INFO:tasks.workunit.client.1.smithi181.stdout:7/615: dread d1/d22/d15/d13/d1b/fac [0,4194304] 0 2024-04-27T04:10:06.198 INFO:tasks.workunit.client.1.smithi181.stdout:1/608: dread d1/dad/db5/f4f [0,4194304] 0 2024-04-27T04:10:06.200 INFO:tasks.workunit.client.1.smithi181.stdout:9/566: creat d0/d44/d4b/d5f/da4/fc6 x:0 0 0 2024-04-27T04:10:06.201 INFO:tasks.workunit.client.0.smithi162.stdout:3/408: symlink d1/d21/d47/d57/d6e/l82 0 2024-04-27T04:10:06.203 INFO:tasks.workunit.client.1.smithi181.stdout:3/546: truncate d0/f37 1672865 0 2024-04-27T04:10:06.204 INFO:tasks.workunit.client.1.smithi181.stdout:3/547: stat d0/de/d20/d58/d64/f6a 0 2024-04-27T04:10:06.205 INFO:tasks.workunit.client.0.smithi162.stdout:5/425: creat d8/d1a/d7d/f91 x:0 0 0 2024-04-27T04:10:06.206 INFO:tasks.workunit.client.1.smithi181.stdout:3/548: chown d0/de/l8b 7 1 2024-04-27T04:10:06.207 INFO:tasks.workunit.client.1.smithi181.stdout:5/718: rename d4/de/d2c/d46/d4d/d8e/d54/l5e to d4/d10/d29/dd2/lff 0 2024-04-27T04:10:06.208 INFO:tasks.workunit.client.1.smithi181.stdout:5/719: chown d4/de/d2c/d2e 4710 1 2024-04-27T04:10:06.213 INFO:tasks.workunit.client.0.smithi162.stdout:2/351: rename d0/d4/d17/d4f/d56 to d0/d4/d2a/d3b/d49/d5a/d63/d6c 0 2024-04-27T04:10:06.214 INFO:tasks.workunit.client.0.smithi162.stdout:2/352: write d0/f7 [938482,41859] 0 2024-04-27T04:10:06.216 INFO:tasks.workunit.client.1.smithi181.stdout:5/720: dwrite d4/d10/d72/d80/f8d [0,4194304] 0 2024-04-27T04:10:06.216 INFO:tasks.workunit.client.0.smithi162.stdout:2/353: truncate d0/d20/d2e/f69 700765 0 2024-04-27T04:10:06.219 INFO:tasks.workunit.client.1.smithi181.stdout:2/645: creat d4/d9/d35/d9f/fd1 x:0 0 0 2024-04-27T04:10:06.220 INFO:tasks.workunit.client.1.smithi181.stdout:2/646: dread - d4/d9/df/d14/da7/fb9 zero size 2024-04-27T04:10:06.222 INFO:tasks.workunit.client.0.smithi162.stdout:3/409: mknod d1/d18/d23/d58/d63/c83 0 2024-04-27T04:10:06.222 INFO:tasks.workunit.client.0.smithi162.stdout:3/410: write d1/d18/f6a [376272,52522] 0 2024-04-27T04:10:06.227 INFO:tasks.workunit.client.1.smithi181.stdout:8/683: write d1/d20/f51 [1089794,61277] 0 2024-04-27T04:10:06.228 INFO:tasks.workunit.client.0.smithi162.stdout:5/426: dwrite d8/d1a/d23/d40/f6e [0,4194304] 0 2024-04-27T04:10:06.243 INFO:tasks.workunit.client.0.smithi162.stdout:6/420: creat d6/d1a/d34/d3a/d68/d6c/d67/f89 x:0 0 0 2024-04-27T04:10:06.249 INFO:tasks.workunit.client.0.smithi162.stdout:7/344: link d3/d5/d9/d4b/c48 d3/d5/d3c/c73 0 2024-04-27T04:10:06.249 INFO:tasks.workunit.client.0.smithi162.stdout:7/345: fsync d3/d4f/f67 0 2024-04-27T04:10:06.252 INFO:tasks.workunit.client.0.smithi162.stdout:5/427: rmdir d8/d9 39 2024-04-27T04:10:06.253 INFO:tasks.workunit.client.0.smithi162.stdout:5/428: chown d8/d1a/d29/d4a/d5c/d63/c7e 0 1 2024-04-27T04:10:06.254 INFO:tasks.workunit.client.0.smithi162.stdout:0/457: creat d5/dc/f91 x:0 0 0 2024-04-27T04:10:06.260 INFO:tasks.workunit.client.0.smithi162.stdout:5/429: dread d8/d1a/d29/f49 [0,4194304] 0 2024-04-27T04:10:06.261 INFO:tasks.workunit.client.0.smithi162.stdout:3/411: creat d1/d45/d69/f84 x:0 0 0 2024-04-27T04:10:06.263 INFO:tasks.workunit.client.0.smithi162.stdout:2/354: dwrite d0/d24/f38 [0,4194304] 0 2024-04-27T04:10:06.264 INFO:tasks.workunit.client.0.smithi162.stdout:7/346: creat d3/d5/d9/d4b/de/f74 x:0 0 0 2024-04-27T04:10:06.264 INFO:tasks.workunit.client.0.smithi162.stdout:2/355: chown d0/l1f 33 1 2024-04-27T04:10:06.267 INFO:tasks.workunit.client.0.smithi162.stdout:7/347: dread d3/d5/d9/d4b/de/f1c [0,4194304] 0 2024-04-27T04:10:06.267 INFO:tasks.workunit.client.0.smithi162.stdout:3/412: dread d1/d21/f6d [0,4194304] 0 2024-04-27T04:10:06.269 INFO:tasks.workunit.client.0.smithi162.stdout:3/413: read d1/d21/f60 [1393316,66560] 0 2024-04-27T04:10:06.276 INFO:tasks.workunit.client.0.smithi162.stdout:5/430: mknod d8/d1a/d23/d40/d4b/d78/c92 0 2024-04-27T04:10:06.283 INFO:tasks.workunit.client.0.smithi162.stdout:7/348: creat d3/d5/d3c/f75 x:0 0 0 2024-04-27T04:10:06.289 INFO:tasks.workunit.client.0.smithi162.stdout:3/414: creat d1/d21/d47/d57/f85 x:0 0 0 2024-04-27T04:10:06.293 INFO:tasks.workunit.client.0.smithi162.stdout:0/458: dwrite d5/f9 [0,4194304] 0 2024-04-27T04:10:06.294 INFO:tasks.workunit.client.0.smithi162.stdout:0/459: chown d5/d26/f6a 124848 1 2024-04-27T04:10:06.298 INFO:tasks.workunit.client.1.smithi181.stdout:0/729: symlink dc/d1f/d3b/d4a/d95/d26/d41/d6f/d8f/l101 0 2024-04-27T04:10:06.300 INFO:tasks.workunit.client.0.smithi162.stdout:0/460: dread d5/f2d [0,4194304] 0 2024-04-27T04:10:06.301 INFO:tasks.workunit.client.1.smithi181.stdout:6/485: getdents d7 0 2024-04-27T04:10:06.302 INFO:tasks.workunit.client.0.smithi162.stdout:5/431: dwrite d8/d9/f10 [0,4194304] 0 2024-04-27T04:10:06.303 INFO:tasks.workunit.client.0.smithi162.stdout:3/415: creat d1/d18/d23/d58/d63/f86 x:0 0 0 2024-04-27T04:10:06.305 INFO:tasks.workunit.client.0.smithi162.stdout:6/421: getdents d6/d1a/d34/d3a/d60/d74 0 2024-04-27T04:10:06.309 INFO:tasks.workunit.client.1.smithi181.stdout:7/616: read d1/d22/d15/d13/d1b/f47 [24820,37911] 0 2024-04-27T04:10:06.311 INFO:tasks.workunit.client.0.smithi162.stdout:2/356: rename d0/d4/d2a/d41/c52 to d0/d20/c6d 0 2024-04-27T04:10:06.313 INFO:tasks.workunit.client.0.smithi162.stdout:0/461: creat d5/dc/d37/f92 x:0 0 0 2024-04-27T04:10:06.314 INFO:tasks.workunit.client.0.smithi162.stdout:9/386: sync 2024-04-27T04:10:06.314 INFO:tasks.workunit.client.1.smithi181.stdout:3/549: fdatasync d0/d6c/d52/f59 0 2024-04-27T04:10:06.317 INFO:tasks.workunit.client.0.smithi162.stdout:3/416: rmdir d1/d18/d5d 39 2024-04-27T04:10:06.318 INFO:tasks.workunit.client.1.smithi181.stdout:4/582: mknod d0/cbd 0 2024-04-27T04:10:06.321 INFO:tasks.workunit.client.0.smithi162.stdout:2/357: mkdir d0/d4/d2a/d3b/d49/d5a/d63/d6e 0 2024-04-27T04:10:06.325 INFO:tasks.workunit.client.1.smithi181.stdout:2/647: symlink d4/d9/d4f/d79/d98/ld2 0 2024-04-27T04:10:06.326 INFO:tasks.workunit.client.1.smithi181.stdout:2/648: chown d4/d9/d16/f2a 5555627 1 2024-04-27T04:10:06.327 INFO:tasks.workunit.client.0.smithi162.stdout:2/358: dwrite d0/d24/f5e [0,4194304] 0 2024-04-27T04:10:06.328 INFO:tasks.workunit.client.1.smithi181.stdout:8/684: rename d1/d6/d2f/d47/dba/ddd to d1/d6/d59/df1 0 2024-04-27T04:10:06.329 INFO:tasks.workunit.client.1.smithi181.stdout:7/617: dread d1/d2/f24 [0,4194304] 0 2024-04-27T04:10:06.329 INFO:tasks.workunit.client.1.smithi181.stdout:8/685: chown d1/d6/d88/dd9 1177150810 1 2024-04-27T04:10:06.332 INFO:tasks.workunit.client.1.smithi181.stdout:9/567: write d0/d44/d4b/d43/d88/da1/f6d [98738,103176] 0 2024-04-27T04:10:06.334 INFO:tasks.workunit.client.1.smithi181.stdout:6/486: rmdir d7/dc/d1e/d66 39 2024-04-27T04:10:06.338 INFO:tasks.workunit.client.1.smithi181.stdout:4/583: dread d0/d15/de/d19/d59/f73 [0,4194304] 0 2024-04-27T04:10:06.342 INFO:tasks.workunit.client.0.smithi162.stdout:0/462: write d5/dc/d1e/d46/f52 [1896893,12493] 0 2024-04-27T04:10:06.355 INFO:tasks.workunit.client.1.smithi181.stdout:0/730: symlink dc/d1f/d3b/d4a/d95/d26/l102 0 2024-04-27T04:10:06.356 INFO:tasks.workunit.client.1.smithi181.stdout:3/550: dread d0/f1 [0,4194304] 0 2024-04-27T04:10:06.357 INFO:tasks.workunit.client.1.smithi181.stdout:3/551: dread - d0/d6c/d52/d44/d56/fa6 zero size 2024-04-27T04:10:06.358 INFO:tasks.workunit.client.1.smithi181.stdout:9/568: mknod d0/d44/d4b/d3d/d9a/cc7 0 2024-04-27T04:10:06.362 INFO:tasks.workunit.client.1.smithi181.stdout:6/487: creat d7/dc/d1c/d34/d3f/d7e/d8a/fa9 x:0 0 0 2024-04-27T04:10:06.363 INFO:tasks.workunit.client.0.smithi162.stdout:9/387: mkdir d3/d13/d15/d7e 0 2024-04-27T04:10:06.365 INFO:tasks.workunit.client.0.smithi162.stdout:6/422: truncate d6/d1a/d34/d43/f5c 316284 0 2024-04-27T04:10:06.366 INFO:tasks.workunit.client.1.smithi181.stdout:6/488: dread d7/f8 [4194304,4194304] 0 2024-04-27T04:10:06.368 INFO:tasks.workunit.client.1.smithi181.stdout:6/489: write d7/d31/d45/f6b [47779,103874] 0 2024-04-27T04:10:06.369 INFO:tasks.workunit.client.0.smithi162.stdout:7/349: link d3/d5/d3c/c3e d3/d5/c76 0 2024-04-27T04:10:06.373 INFO:tasks.workunit.client.0.smithi162.stdout:7/350: fsync d3/d5/d9/d4b/de/d2b/f56 0 2024-04-27T04:10:06.374 INFO:tasks.workunit.client.1.smithi181.stdout:5/721: write d4/d10/d29/d92/dbb/fe9 [505266,18659] 0 2024-04-27T04:10:06.375 INFO:tasks.workunit.client.0.smithi162.stdout:7/351: chown d3/d5/d3c/c73 60776854 1 2024-04-27T04:10:06.375 INFO:tasks.workunit.client.0.smithi162.stdout:7/352: chown d3/d5/d9/d4b/de/f30 568075 1 2024-04-27T04:10:06.376 INFO:tasks.workunit.client.1.smithi181.stdout:0/731: dread dc/d1f/d3b/d4a/d95/d26/d41/f4e [0,4194304] 0 2024-04-27T04:10:06.376 INFO:tasks.workunit.client.0.smithi162.stdout:2/359: rename d0/d20/d2e/d61 to d0/d20/d51/d6f 0 2024-04-27T04:10:06.378 INFO:tasks.workunit.client.1.smithi181.stdout:2/649: dwrite d4/d9/d16/f4e [4194304,4194304] 0 2024-04-27T04:10:06.380 INFO:tasks.workunit.client.0.smithi162.stdout:0/463: mknod d5/d26/d69/c93 0 2024-04-27T04:10:06.380 INFO:tasks.workunit.client.1.smithi181.stdout:2/650: readlink d4/d9/d35/lc7 0 2024-04-27T04:10:06.380 INFO:tasks.workunit.client.1.smithi181.stdout:0/732: truncate dc/d1f/d3b/dd0/f100 452343 0 2024-04-27T04:10:06.381 INFO:tasks.workunit.client.1.smithi181.stdout:2/651: readlink d4/d9/df/d14/l69 0 2024-04-27T04:10:06.382 INFO:tasks.workunit.client.1.smithi181.stdout:0/733: readlink dc/d1f/laf 0 2024-04-27T04:10:06.382 INFO:tasks.workunit.client.0.smithi162.stdout:2/360: dwrite d0/d4/d2a/d41/f4a [0,4194304] 0 2024-04-27T04:10:06.383 INFO:tasks.workunit.client.1.smithi181.stdout:2/652: stat d4/d9/df/d14/f78 0 2024-04-27T04:10:06.386 INFO:tasks.workunit.client.0.smithi162.stdout:2/361: write d0/d4/d17/f3c [302658,95778] 0 2024-04-27T04:10:06.387 INFO:tasks.workunit.client.0.smithi162.stdout:9/388: chown d3/d13/d2d/d30/f38 845669321 1 2024-04-27T04:10:06.390 INFO:tasks.workunit.client.0.smithi162.stdout:3/417: write d1/f4c [2102958,4889] 0 2024-04-27T04:10:06.394 INFO:tasks.workunit.client.1.smithi181.stdout:1/609: getdents d1/d2 0 2024-04-27T04:10:06.394 INFO:tasks.workunit.client.0.smithi162.stdout:9/389: dwrite d3/d11/f5e [0,4194304] 0 2024-04-27T04:10:06.398 INFO:tasks.workunit.client.1.smithi181.stdout:8/686: write d1/d6/f44 [2071377,112768] 0 2024-04-27T04:10:06.402 INFO:tasks.workunit.client.1.smithi181.stdout:5/722: rename d4/de/d2c/d2e to d4/da9/d100 0 2024-04-27T04:10:06.404 INFO:tasks.workunit.client.0.smithi162.stdout:0/464: readlink d5/l40 0 2024-04-27T04:10:06.409 INFO:tasks.workunit.client.0.smithi162.stdout:2/362: mkdir d0/d4/d2a/d3b/d49/d5a/d63/d70 0 2024-04-27T04:10:06.410 INFO:tasks.workunit.client.0.smithi162.stdout:6/423: symlink d6/d1a/d34/d7b/d7e/l8a 0 2024-04-27T04:10:06.411 INFO:tasks.workunit.client.1.smithi181.stdout:9/569: dwrite d0/d40/f78 [0,4194304] 0 2024-04-27T04:10:06.413 INFO:tasks.workunit.client.0.smithi162.stdout:9/390: fdatasync d3/f5 0 2024-04-27T04:10:06.415 INFO:tasks.workunit.client.0.smithi162.stdout:9/391: write d3/fc [1375823,98395] 0 2024-04-27T04:10:06.416 INFO:tasks.workunit.client.1.smithi181.stdout:3/552: dwrite d0/de/d20/d58/d70/f9d [0,4194304] 0 2024-04-27T04:10:06.417 INFO:tasks.workunit.client.0.smithi162.stdout:9/392: readlink d3/d39/l53 0 2024-04-27T04:10:06.417 INFO:tasks.workunit.client.0.smithi162.stdout:2/363: dwrite d0/d24/f38 [0,4194304] 0 2024-04-27T04:10:06.420 INFO:tasks.workunit.client.0.smithi162.stdout:6/424: read - d6/db/d32/f51 zero size 2024-04-27T04:10:06.423 INFO:tasks.workunit.client.0.smithi162.stdout:7/353: rename d3/d5/d9/f59 to d3/d39/f77 0 2024-04-27T04:10:06.430 INFO:tasks.workunit.client.0.smithi162.stdout:9/393: unlink d3/d23/d1e/c32 0 2024-04-27T04:10:06.431 INFO:tasks.workunit.client.1.smithi181.stdout:1/610: creat d1/dad/db5/d6/da3/d1e/dbc/fc3 x:0 0 0 2024-04-27T04:10:06.431 INFO:tasks.workunit.client.1.smithi181.stdout:2/653: creat d4/d9/d35/d9f/dd0/fd3 x:0 0 0 2024-04-27T04:10:06.439 INFO:tasks.workunit.client.1.smithi181.stdout:2/654: truncate d4/d9/d4f/d79/fce 725258 0 2024-04-27T04:10:06.442 INFO:tasks.workunit.client.0.smithi162.stdout:7/354: creat d3/d4f/f78 x:0 0 0 2024-04-27T04:10:06.442 INFO:tasks.workunit.client.1.smithi181.stdout:2/655: fdatasync d4/d9/d4f/d79/d98/fcc 0 2024-04-27T04:10:06.445 INFO:tasks.workunit.client.0.smithi162.stdout:2/364: mknod d0/d4/c71 0 2024-04-27T04:10:06.447 INFO:tasks.workunit.client.0.smithi162.stdout:9/394: mkdir d3/d13/d2d/d30/d7f 0 2024-04-27T04:10:06.448 INFO:tasks.workunit.client.1.smithi181.stdout:0/734: dwrite dc/d1f/d3b/d4a/d95/f25 [0,4194304] 0 2024-04-27T04:10:06.448 INFO:tasks.workunit.client.0.smithi162.stdout:6/425: getdents d6/d1a/d34/d3a/d60/d74 0 2024-04-27T04:10:06.449 INFO:tasks.workunit.client.1.smithi181.stdout:6/490: dwrite d7/dc/d1c/d34/f84 [0,4194304] 0 2024-04-27T04:10:06.451 INFO:tasks.workunit.client.0.smithi162.stdout:7/355: rmdir d3/d39 39 2024-04-27T04:10:06.452 INFO:tasks.workunit.client.1.smithi181.stdout:6/491: read d7/dc/d1e/d2b/f85 [746397,107070] 0 2024-04-27T04:10:06.454 INFO:tasks.workunit.client.0.smithi162.stdout:9/395: dwrite d3/d13/d15/f18 [0,4194304] 0 2024-04-27T04:10:06.458 INFO:tasks.workunit.client.0.smithi162.stdout:6/426: write d6/f11 [4475828,110344] 0 2024-04-27T04:10:06.461 INFO:tasks.workunit.client.1.smithi181.stdout:1/611: truncate d1/dad/db5/d6/da3/d1e/f4b 1367417 0 2024-04-27T04:10:06.461 INFO:tasks.workunit.client.0.smithi162.stdout:6/427: stat d6/d1a/d34/d50 0 2024-04-27T04:10:06.463 INFO:tasks.workunit.client.0.smithi162.stdout:7/356: creat d3/d5/d9/d4b/f79 x:0 0 0 2024-04-27T04:10:06.463 INFO:tasks.workunit.client.1.smithi181.stdout:5/723: link d4/d10/d29/d92/cdd d4/d10/d29/d92/dbb/c101 0 2024-04-27T04:10:06.464 INFO:tasks.workunit.client.1.smithi181.stdout:9/570: dwrite d0/d44/d4b/d3d/f9f [0,4194304] 0 2024-04-27T04:10:06.468 INFO:tasks.workunit.client.1.smithi181.stdout:3/553: rmdir d0/d6c/d52/d44/d74/da0 0 2024-04-27T04:10:06.469 INFO:tasks.workunit.client.0.smithi162.stdout:9/396: fsync d3/d13/d2d/d30/f38 0 2024-04-27T04:10:06.471 INFO:tasks.workunit.client.1.smithi181.stdout:8/687: dwrite d1/d6/d1a/f4d [4194304,4194304] 0 2024-04-27T04:10:06.471 INFO:tasks.workunit.client.0.smithi162.stdout:6/428: dwrite d6/f1e [0,4194304] 0 2024-04-27T04:10:06.472 INFO:tasks.workunit.client.1.smithi181.stdout:2/656: rename d4/d9/d35/f51 to d4/d9/d16/d82/d7d/db7/dcb/da4/fd4 0 2024-04-27T04:10:06.475 INFO:tasks.workunit.client.1.smithi181.stdout:2/657: write d4/d9/f11 [6006965,82153] 0 2024-04-27T04:10:06.480 INFO:tasks.workunit.client.0.smithi162.stdout:9/397: dwrite d3/d13/d15/f19 [4194304,4194304] 0 2024-04-27T04:10:06.481 INFO:tasks.workunit.client.0.smithi162.stdout:1/382: sync 2024-04-27T04:10:06.481 INFO:tasks.workunit.client.0.smithi162.stdout:8/434: sync 2024-04-27T04:10:06.482 INFO:tasks.workunit.client.1.smithi181.stdout:9/571: mknod d0/d4d/d85/cc8 0 2024-04-27T04:10:06.484 INFO:tasks.workunit.client.0.smithi162.stdout:4/413: sync 2024-04-27T04:10:06.488 INFO:tasks.workunit.client.0.smithi162.stdout:2/365: getdents d0/d4/d17/d2b 0 2024-04-27T04:10:06.497 INFO:tasks.workunit.client.1.smithi181.stdout:3/554: dread d0/d6c/f4c [0,4194304] 0 2024-04-27T04:10:06.506 INFO:tasks.workunit.client.0.smithi162.stdout:7/357: creat d3/d5/f7a x:0 0 0 2024-04-27T04:10:06.507 INFO:tasks.workunit.client.0.smithi162.stdout:7/358: write d3/d5/d9/d4b/de/f30 [1554413,74013] 0 2024-04-27T04:10:06.508 INFO:tasks.workunit.client.1.smithi181.stdout:1/612: mkdir d1/dad/db5/d6/db/d2d/d36/dc4 0 2024-04-27T04:10:06.510 INFO:tasks.workunit.client.0.smithi162.stdout:7/359: write d3/d5/d9/d4b/de/d2b/f69 [981597,51357] 0 2024-04-27T04:10:06.511 INFO:tasks.workunit.client.0.smithi162.stdout:9/398: creat d3/d11/d45/d4e/d58/f80 x:0 0 0 2024-04-27T04:10:06.513 INFO:tasks.workunit.client.1.smithi181.stdout:5/724: fdatasync d4/de/d2c/d46/d99/dad/dca/fcc 0 2024-04-27T04:10:06.519 INFO:tasks.workunit.client.1.smithi181.stdout:5/725: dwrite d4/da3/dbc/f9e [0,4194304] 0 2024-04-27T04:10:06.521 INFO:tasks.workunit.client.1.smithi181.stdout:5/726: chown d4/de/d2c/cc6 5850600 1 2024-04-27T04:10:06.521 INFO:tasks.workunit.client.1.smithi181.stdout:0/735: rename dc/d1f/d3b/f66 to dc/d1f/daa/f103 0 2024-04-27T04:10:06.536 INFO:tasks.workunit.client.0.smithi162.stdout:8/435: chown d0/de/d20/c3b 29617 1 2024-04-27T04:10:06.538 INFO:tasks.workunit.client.1.smithi181.stdout:9/572: creat d0/d44/d4b/d3d/dbd/fc9 x:0 0 0 2024-04-27T04:10:06.540 INFO:tasks.workunit.client.1.smithi181.stdout:3/555: fdatasync d0/de/f34 0 2024-04-27T04:10:06.542 INFO:tasks.workunit.client.1.smithi181.stdout:3/556: read - d0/d6c/d52/d44/d56/faa zero size 2024-04-27T04:10:06.543 INFO:tasks.workunit.client.1.smithi181.stdout:2/658: dread d4/d81/f84 [0,4194304] 0 2024-04-27T04:10:06.545 INFO:tasks.workunit.client.1.smithi181.stdout:8/688: truncate d1/d20/fa2 1317466 0 2024-04-27T04:10:06.545 INFO:tasks.workunit.client.0.smithi162.stdout:6/429: getdents d6/d1a/d34 0 2024-04-27T04:10:06.546 INFO:tasks.workunit.client.0.smithi162.stdout:6/430: readlink d6/d1a/d34/d7f/d80/l85 0 2024-04-27T04:10:06.547 INFO:tasks.workunit.client.0.smithi162.stdout:6/431: stat d6/d1a/d34/d3a/d68/d6c/f82 0 2024-04-27T04:10:06.548 INFO:tasks.workunit.client.1.smithi181.stdout:6/492: truncate d7/dc/d1e/f52 2190829 0 2024-04-27T04:10:06.549 INFO:tasks.workunit.client.1.smithi181.stdout:5/727: readlink d4/de/d2c/d46/d99/ldf 0 2024-04-27T04:10:06.551 INFO:tasks.workunit.client.0.smithi162.stdout:6/432: stat d6/l2a 0 2024-04-27T04:10:06.553 INFO:tasks.workunit.client.0.smithi162.stdout:8/436: creat d0/d10/f9a x:0 0 0 2024-04-27T04:10:06.553 INFO:tasks.workunit.client.0.smithi162.stdout:1/383: truncate d0/d7/d3c/d54/f65 111292 0 2024-04-27T04:10:06.555 INFO:tasks.workunit.client.0.smithi162.stdout:6/433: symlink d6/d1a/d34/d3a/l8b 0 2024-04-27T04:10:06.558 INFO:tasks.workunit.client.0.smithi162.stdout:8/437: write d0/d10/d2b/d2d/d2f/d39/f5a [419614,128019] 0 2024-04-27T04:10:06.558 INFO:tasks.workunit.client.0.smithi162.stdout:1/384: creat d0/d11/d50/f7d x:0 0 0 2024-04-27T04:10:06.558 INFO:tasks.workunit.client.1.smithi181.stdout:2/659: creat d4/d9/d16/d82/d7d/dbe/fd5 x:0 0 0 2024-04-27T04:10:06.559 INFO:tasks.workunit.client.0.smithi162.stdout:7/360: dwrite d3/d5/d9/f13 [0,4194304] 0 2024-04-27T04:10:06.559 INFO:tasks.workunit.client.1.smithi181.stdout:2/660: write d4/d9/d16/d82/fa5 [1756293,7088] 0 2024-04-27T04:10:06.560 INFO:tasks.workunit.client.1.smithi181.stdout:6/493: mkdir d7/d31/d45/d72/daa 0 2024-04-27T04:10:06.561 INFO:tasks.workunit.client.1.smithi181.stdout:0/736: creat dc/d1f/d6b/db9/dba/dce/dde/f104 x:0 0 0 2024-04-27T04:10:06.562 INFO:tasks.workunit.client.0.smithi162.stdout:7/361: stat d3/d5/d9/d4b/f3b 0 2024-04-27T04:10:06.564 INFO:tasks.workunit.client.0.smithi162.stdout:2/366: dwrite d0/d4/f15 [0,4194304] 0 2024-04-27T04:10:06.564 INFO:tasks.workunit.client.0.smithi162.stdout:6/434: mknod d6/db/d32/c8c 0 2024-04-27T04:10:06.570 INFO:tasks.workunit.client.1.smithi181.stdout:6/494: dwrite d7/dc/d8b/fa7 [0,4194304] 0 2024-04-27T04:10:06.572 INFO:tasks.workunit.client.0.smithi162.stdout:2/367: dwrite d0/d24/f4d [0,4194304] 0 2024-04-27T04:10:06.575 INFO:tasks.workunit.client.0.smithi162.stdout:5/432: sync 2024-04-27T04:10:06.575 INFO:tasks.workunit.client.0.smithi162.stdout:5/433: fsync d8/d9/f10 0 2024-04-27T04:10:06.587 INFO:tasks.workunit.client.0.smithi162.stdout:6/435: dread d6/d1a/d34/d43/f5c [0,4194304] 0 2024-04-27T04:10:06.587 INFO:tasks.workunit.client.1.smithi181.stdout:0/737: chown dc/d1f/d3b/d4a/d95/d26/d41/l55 84 1 2024-04-27T04:10:06.587 INFO:tasks.workunit.client.0.smithi162.stdout:2/368: symlink d0/d4/d2a/l72 0 2024-04-27T04:10:06.593 INFO:tasks.workunit.client.1.smithi181.stdout:3/557: creat d0/d6c/d52/d44/fad x:0 0 0 2024-04-27T04:10:06.594 INFO:tasks.workunit.client.1.smithi181.stdout:6/495: write d7/dc/d1c/f91 [201677,17993] 0 2024-04-27T04:10:06.595 INFO:tasks.workunit.client.0.smithi162.stdout:5/434: creat d8/d9/f93 x:0 0 0 2024-04-27T04:10:06.597 INFO:tasks.workunit.client.0.smithi162.stdout:7/362: symlink d3/d5/l7b 0 2024-04-27T04:10:06.600 INFO:tasks.workunit.client.0.smithi162.stdout:2/369: write d0/d20/d2e/f69 [249651,24227] 0 2024-04-27T04:10:06.600 INFO:tasks.workunit.client.1.smithi181.stdout:8/689: truncate d1/d6/d2f/d47/f4b 3470249 0 2024-04-27T04:10:06.602 INFO:tasks.workunit.client.0.smithi162.stdout:5/435: dwrite d8/d1a/d23/d40/d4b/f52 [0,4194304] 0 2024-04-27T04:10:06.604 INFO:tasks.workunit.client.0.smithi162.stdout:6/436: mknod d6/d1a/d34/d3a/d68/d6c/d67/c8d 0 2024-04-27T04:10:06.605 INFO:tasks.workunit.client.0.smithi162.stdout:1/385: rmdir d0/d11/d50/d53/d74 0 2024-04-27T04:10:06.607 INFO:tasks.workunit.client.1.smithi181.stdout:3/558: read d0/d6c/d52/f47 [663952,62779] 0 2024-04-27T04:10:06.611 INFO:tasks.workunit.client.1.smithi181.stdout:6/496: creat d7/dc/d1c/d7f/fab x:0 0 0 2024-04-27T04:10:06.612 INFO:tasks.workunit.client.0.smithi162.stdout:1/386: dwrite d0/f1e [4194304,4194304] 0 2024-04-27T04:10:06.612 INFO:tasks.workunit.client.1.smithi181.stdout:5/728: write d4/de/d2c/d46/d4d/d8e/d67/d71/f7b [766837,64582] 0 2024-04-27T04:10:06.612 INFO:tasks.workunit.client.1.smithi181.stdout:6/497: write d7/dc/d1c/d34/f9a [786665,77930] 0 2024-04-27T04:10:06.616 INFO:tasks.workunit.client.1.smithi181.stdout:2/661: rename d4/c43 to d4/d9/d4f/cd6 0 2024-04-27T04:10:06.616 INFO:tasks.workunit.client.0.smithi162.stdout:8/438: creat d0/de/d6e/f9b x:0 0 0 2024-04-27T04:10:06.618 INFO:tasks.workunit.client.1.smithi181.stdout:8/690: dread - d1/d6/d1a/d25/fc7 zero size 2024-04-27T04:10:06.621 INFO:tasks.workunit.client.1.smithi181.stdout:8/691: chown d1/d6/d2f/d30/d66/cd1 119511 1 2024-04-27T04:10:06.622 INFO:tasks.workunit.client.1.smithi181.stdout:3/559: readlink d0/de/d20/l22 0 2024-04-27T04:10:06.623 INFO:tasks.workunit.client.1.smithi181.stdout:8/692: chown d1/d6/d2f/d30/d66/dc0/dcd 1651 1 2024-04-27T04:10:06.623 INFO:tasks.workunit.client.0.smithi162.stdout:3/418: sync 2024-04-27T04:10:06.623 INFO:tasks.workunit.client.0.smithi162.stdout:0/465: sync 2024-04-27T04:10:06.623 INFO:tasks.workunit.client.0.smithi162.stdout:4/414: sync 2024-04-27T04:10:06.624 INFO:tasks.workunit.client.0.smithi162.stdout:0/466: readlink d5/l47 0 2024-04-27T04:10:06.626 INFO:tasks.workunit.client.0.smithi162.stdout:0/467: write d5/dc/d1e/d41/f71 [430319,49406] 0 2024-04-27T04:10:06.626 INFO:tasks.workunit.client.1.smithi181.stdout:5/729: mkdir d4/de/d2c/d46/d99/dad/dca/d102 0 2024-04-27T04:10:06.627 INFO:tasks.workunit.client.1.smithi181.stdout:5/730: fsync d4/da3/dbc/db5/ffa 0 2024-04-27T04:10:06.627 INFO:tasks.workunit.client.0.smithi162.stdout:1/387: chown d0/l4 3859315 1 2024-04-27T04:10:06.630 INFO:tasks.workunit.client.1.smithi181.stdout:5/731: dread d4/de/d2c/f3e [0,4194304] 0 2024-04-27T04:10:06.630 INFO:tasks.workunit.client.0.smithi162.stdout:8/439: unlink d0/de/d6e/c45 0 2024-04-27T04:10:06.632 INFO:tasks.workunit.client.0.smithi162.stdout:2/370: mkdir d0/d20/d2e/d40/d73 0 2024-04-27T04:10:06.638 INFO:tasks.workunit.client.0.smithi162.stdout:8/440: dwrite d0/d10/d2b/d2d/d2f/d42/f48 [0,4194304] 0 2024-04-27T04:10:06.641 INFO:tasks.workunit.client.1.smithi181.stdout:0/738: getdents dc/d1f/d3b 0 2024-04-27T04:10:06.641 INFO:tasks.workunit.client.0.smithi162.stdout:4/415: chown d2/d25/c4d 0 1 2024-04-27T04:10:06.644 INFO:tasks.workunit.client.1.smithi181.stdout:8/693: mkdir d1/d6/d6f/dd2/df2 0 2024-04-27T04:10:06.645 INFO:tasks.workunit.client.0.smithi162.stdout:7/363: creat d3/d5/f7c x:0 0 0 2024-04-27T04:10:06.647 INFO:tasks.workunit.client.0.smithi162.stdout:2/371: dread d0/f1 [4194304,4194304] 0 2024-04-27T04:10:06.648 INFO:tasks.workunit.client.0.smithi162.stdout:3/419: creat d1/d45/f87 x:0 0 0 2024-04-27T04:10:06.649 INFO:tasks.workunit.client.0.smithi162.stdout:0/468: rmdir d5/dc/d1e/d41/d79 39 2024-04-27T04:10:06.650 INFO:tasks.workunit.client.1.smithi181.stdout:6/498: truncate d7/f43 1005560 0 2024-04-27T04:10:06.651 INFO:tasks.workunit.client.0.smithi162.stdout:1/388: dread d0/d11/f2e [0,4194304] 0 2024-04-27T04:10:06.651 INFO:tasks.workunit.client.0.smithi162.stdout:5/436: dwrite d8/d1a/d29/d4a/d5c/d63/f73 [0,4194304] 0 2024-04-27T04:10:06.652 INFO:tasks.workunit.client.1.smithi181.stdout:3/560: creat d0/d6c/d83/d98/da7/fae x:0 0 0 2024-04-27T04:10:06.653 INFO:tasks.workunit.client.1.smithi181.stdout:3/561: chown d0/de/d20/c2b 2269 1 2024-04-27T04:10:06.656 INFO:tasks.workunit.client.0.smithi162.stdout:6/437: link d6/db/d32/l40 d6/d1a/d34/l8e 0 2024-04-27T04:10:06.657 INFO:tasks.workunit.client.1.smithi181.stdout:0/739: symlink dc/d1f/l105 0 2024-04-27T04:10:06.658 INFO:tasks.workunit.client.0.smithi162.stdout:6/438: dread - d6/db/f6a zero size 2024-04-27T04:10:06.658 INFO:tasks.workunit.client.0.smithi162.stdout:9/399: sync 2024-04-27T04:10:06.659 INFO:tasks.workunit.client.0.smithi162.stdout:5/437: dwrite d8/d9/d55/f65 [0,4194304] 0 2024-04-27T04:10:06.659 INFO:tasks.workunit.client.0.smithi162.stdout:8/441: rmdir d0/de/d20 39 2024-04-27T04:10:06.660 INFO:tasks.workunit.client.0.smithi162.stdout:6/439: write d6/f25 [1901393,51591] 0 2024-04-27T04:10:06.661 INFO:tasks.workunit.client.0.smithi162.stdout:5/438: stat d8/d1a/d23/d40/l36 0 2024-04-27T04:10:06.664 INFO:tasks.workunit.client.0.smithi162.stdout:4/416: creat d2/d25/d1c/d39/d5e/f85 x:0 0 0 2024-04-27T04:10:06.666 INFO:tasks.workunit.client.0.smithi162.stdout:7/364: write d3/d4f/f31 [2980303,6419] 0 2024-04-27T04:10:06.669 INFO:tasks.workunit.client.1.smithi181.stdout:2/662: link d4/d81/f8e d4/d9/d4f/d79/d94/fd7 0 2024-04-27T04:10:06.669 INFO:tasks.workunit.client.0.smithi162.stdout:3/420: mkdir d1/d18/d23/d58/d63/d88 0 2024-04-27T04:10:06.670 INFO:tasks.workunit.client.0.smithi162.stdout:2/372: creat d0/d4/d2a/d3b/f74 x:0 0 0 2024-04-27T04:10:06.671 INFO:tasks.workunit.client.0.smithi162.stdout:5/439: dwrite d8/d1a/f31 [0,4194304] 0 2024-04-27T04:10:06.671 INFO:tasks.workunit.client.0.smithi162.stdout:0/469: rmdir d5/d26 39 2024-04-27T04:10:06.672 INFO:tasks.workunit.client.0.smithi162.stdout:7/365: write d3/d5/d5b/f6c [1747016,105210] 0 2024-04-27T04:10:06.675 INFO:tasks.workunit.client.1.smithi181.stdout:8/694: mknod d1/d6/d88/dec/cf3 0 2024-04-27T04:10:06.676 INFO:tasks.workunit.client.1.smithi181.stdout:5/732: write d4/de/d2c/d46/d4d/d8e/f8a [4676715,123118] 0 2024-04-27T04:10:06.677 INFO:tasks.workunit.client.0.smithi162.stdout:5/440: write d8/d9/d55/f69 [74890,2690] 0 2024-04-27T04:10:06.677 INFO:tasks.workunit.client.1.smithi181.stdout:5/733: chown d4/da3 1419144 1 2024-04-27T04:10:06.680 INFO:tasks.workunit.client.0.smithi162.stdout:5/441: truncate d8/d1a/d29/d4a/d5c/d63/f64 222143 0 2024-04-27T04:10:06.682 INFO:tasks.workunit.client.0.smithi162.stdout:6/440: dread d6/f11 [0,4194304] 0 2024-04-27T04:10:06.687 INFO:tasks.workunit.client.1.smithi181.stdout:6/499: fsync d7/dc/d1e/f51 0 2024-04-27T04:10:06.687 INFO:tasks.workunit.client.0.smithi162.stdout:5/442: dread d8/d1a/f45 [0,4194304] 0 2024-04-27T04:10:06.687 INFO:tasks.workunit.client.0.smithi162.stdout:1/389: mknod d0/d11/d3e/d59/c7e 0 2024-04-27T04:10:06.688 INFO:tasks.workunit.client.1.smithi181.stdout:2/663: symlink d4/d9/df/d14/da7/ld8 0 2024-04-27T04:10:06.693 INFO:tasks.workunit.client.1.smithi181.stdout:0/740: mkdir dc/d1f/d6b/db9/d106 0 2024-04-27T04:10:06.694 INFO:tasks.workunit.client.1.smithi181.stdout:0/741: stat dc/d1f/d3b/d4a/d95/d26/d68/ca3 0 2024-04-27T04:10:06.696 INFO:tasks.workunit.client.1.smithi181.stdout:0/742: chown dc/d1f/d3b/d4a/d95/d26/d41/fc2 392840868 1 2024-04-27T04:10:06.697 INFO:tasks.workunit.client.0.smithi162.stdout:9/400: write d3/d11/f4d [2211079,72096] 0 2024-04-27T04:10:06.698 INFO:tasks.workunit.client.0.smithi162.stdout:9/401: write d3/f14 [9627801,13247] 0 2024-04-27T04:10:06.702 INFO:tasks.workunit.client.0.smithi162.stdout:9/402: dread - d3/d13/d2d/d3b/d69/f7d zero size 2024-04-27T04:10:06.767 INFO:tasks.workunit.client.0.smithi162.stdout:7/366: sync 2024-04-27T04:10:06.768 INFO:tasks.workunit.client.0.smithi162.stdout:7/367: fdatasync d3/d5/d9/d4b/de/f74 0 2024-04-27T04:10:06.768 INFO:tasks.workunit.client.0.smithi162.stdout:7/368: fdatasync d3/d5/d9/d4b/f35 0 2024-04-27T04:10:06.815 INFO:tasks.workunit.client.1.smithi181.stdout:7/618: sync 2024-04-27T04:10:06.819 INFO:tasks.workunit.client.1.smithi181.stdout:2/664: unlink d4/d9/d35/d9f/dd0/fd3 0 2024-04-27T04:10:06.823 INFO:tasks.workunit.client.0.smithi162.stdout:0/470: chown d5/d26/d69/f6f 3 1 2024-04-27T04:10:06.823 INFO:tasks.workunit.client.0.smithi162.stdout:0/471: stat d5/dc/c7b 0 2024-04-27T04:10:06.826 INFO:tasks.workunit.client.0.smithi162.stdout:3/421: rename d1/d45/d69/f84 to d1/f89 0 2024-04-27T04:10:06.827 INFO:tasks.workunit.client.0.smithi162.stdout:2/373: symlink d0/d4/d2a/d3b/d49/d5a/d63/d6c/l75 0 2024-04-27T04:10:06.831 INFO:tasks.workunit.client.1.smithi181.stdout:8/695: truncate d1/f6b 684930 0 2024-04-27T04:10:06.832 INFO:tasks.workunit.client.0.smithi162.stdout:2/374: dwrite d0/f6 [0,4194304] 0 2024-04-27T04:10:06.832 INFO:tasks.workunit.client.0.smithi162.stdout:1/390: mkdir d0/d11/d3e/d59/d7f 0 2024-04-27T04:10:06.833 INFO:tasks.workunit.client.0.smithi162.stdout:1/391: stat d0/d7/f19 0 2024-04-27T04:10:06.833 INFO:tasks.workunit.client.1.smithi181.stdout:6/500: mknod d7/dc/d1e/d66/cac 0 2024-04-27T04:10:06.834 INFO:tasks.workunit.client.0.smithi162.stdout:1/392: stat d0/d7/l36 0 2024-04-27T04:10:06.838 INFO:tasks.workunit.client.1.smithi181.stdout:7/619: write d1/d22/d15/d29/f84 [1406734,121577] 0 2024-04-27T04:10:06.839 INFO:tasks.workunit.client.0.smithi162.stdout:9/403: write d3/fa [6877326,27212] 0 2024-04-27T04:10:06.841 INFO:tasks.workunit.client.1.smithi181.stdout:0/743: dwrite dc/d1f/d3b/d4a/d95/d26/d41/f4e [4194304,4194304] 0 2024-04-27T04:10:06.845 INFO:tasks.workunit.client.1.smithi181.stdout:6/501: dwrite d7/dc/f3a [0,4194304] 0 2024-04-27T04:10:06.850 INFO:tasks.workunit.client.0.smithi162.stdout:0/472: mknod d5/dc/d37/c94 0 2024-04-27T04:10:06.852 INFO:tasks.workunit.client.1.smithi181.stdout:5/734: link d4/de/d2c/d46/d4d/d8e/d67/d71/cba d4/d10/d72/db3/c103 0 2024-04-27T04:10:06.854 INFO:tasks.workunit.client.0.smithi162.stdout:5/443: rename d8/d1a/d23/d40/d4b/d78/c92 to d8/d1a/d23/d40/d4b/d66/c94 0 2024-04-27T04:10:06.858 INFO:tasks.workunit.client.1.smithi181.stdout:2/665: rename d4/d9/d16/d82/d7d/dbe/fd5 to d4/fd9 0 2024-04-27T04:10:06.864 INFO:tasks.workunit.client.1.smithi181.stdout:5/735: symlink d4/d10/d72/db3/l104 0 2024-04-27T04:10:06.868 INFO:tasks.workunit.client.1.smithi181.stdout:6/502: rename d7/dc/d1c/d7f/fab to d7/dc/d1c/d34/d3f/d7e/d8a/fad 0 2024-04-27T04:10:06.871 INFO:tasks.workunit.client.1.smithi181.stdout:2/666: mknod d4/d9/d35/d9f/dd0/cda 0 2024-04-27T04:10:06.871 INFO:tasks.workunit.client.1.smithi181.stdout:8/696: dwrite d1/d6/d2f/d47/fac [4194304,4194304] 0 2024-04-27T04:10:06.872 INFO:tasks.workunit.client.1.smithi181.stdout:7/620: write d1/d22/d15/d13/d1b/f8c [160842,20696] 0 2024-04-27T04:10:06.879 INFO:tasks.workunit.client.1.smithi181.stdout:6/503: dwrite d7/dc/d1c/f91 [0,4194304] 0 2024-04-27T04:10:06.881 INFO:tasks.workunit.client.0.smithi162.stdout:0/473: creat d5/dc/d37/f95 x:0 0 0 2024-04-27T04:10:06.884 INFO:tasks.workunit.client.0.smithi162.stdout:0/474: read d5/dc/f22 [1884402,96217] 0 2024-04-27T04:10:06.884 INFO:tasks.workunit.client.1.smithi181.stdout:5/736: fdatasync d4/de/d2c/d46/f60 0 2024-04-27T04:10:06.887 INFO:tasks.workunit.client.0.smithi162.stdout:5/444: rename d8/d9/f93 to d8/d1a/d29/f95 0 2024-04-27T04:10:06.887 INFO:tasks.workunit.client.0.smithi162.stdout:2/375: dwrite d0/f1 [4194304,4194304] 0 2024-04-27T04:10:06.888 INFO:tasks.workunit.client.1.smithi181.stdout:2/667: mkdir d4/d9/d35/d9f/dd0/ddb 0 2024-04-27T04:10:06.888 INFO:tasks.workunit.client.0.smithi162.stdout:5/445: stat d8/d1a/d23/f39 0 2024-04-27T04:10:06.889 INFO:tasks.workunit.client.1.smithi181.stdout:2/668: chown d4/d9/df/l22 1918219561 1 2024-04-27T04:10:06.890 INFO:tasks.workunit.client.1.smithi181.stdout:2/669: write d4/d9/df/f41 [2069690,104953] 0 2024-04-27T04:10:06.892 INFO:tasks.workunit.client.0.smithi162.stdout:1/393: mkdir d0/d80 0 2024-04-27T04:10:06.894 INFO:tasks.workunit.client.0.smithi162.stdout:8/442: mknod d0/d10/d2b/d2d/c9c 0 2024-04-27T04:10:06.895 INFO:tasks.workunit.client.0.smithi162.stdout:9/404: getdents d3/d23/d7c 0 2024-04-27T04:10:06.897 INFO:tasks.workunit.client.1.smithi181.stdout:7/621: dwrite d1/d22/d15/d13/d1b/fac [0,4194304] 0 2024-04-27T04:10:06.899 INFO:tasks.workunit.client.0.smithi162.stdout:7/369: symlink d3/d39/l7d 0 2024-04-27T04:10:06.904 INFO:tasks.workunit.client.0.smithi162.stdout:4/417: link d2/d25/d1c/d39/l4f d2/da/l86 0 2024-04-27T04:10:06.904 INFO:tasks.workunit.client.1.smithi181.stdout:5/737: chown d4/de/d2f/d7a/d90/cd0 45153416 1 2024-04-27T04:10:06.909 INFO:tasks.workunit.client.0.smithi162.stdout:3/422: dread d1/f3f [0,4194304] 0 2024-04-27T04:10:06.914 INFO:tasks.workunit.client.0.smithi162.stdout:2/376: chown d0/c13 82803803 1 2024-04-27T04:10:06.916 INFO:tasks.workunit.client.0.smithi162.stdout:5/446: mkdir d8/d9/d5b/d96 0 2024-04-27T04:10:06.916 INFO:tasks.workunit.client.1.smithi181.stdout:8/697: dwrite d1/f46 [0,4194304] 0 2024-04-27T04:10:06.917 INFO:tasks.workunit.client.0.smithi162.stdout:5/447: chown d8/d1a/d29/f4d 0 1 2024-04-27T04:10:06.917 INFO:tasks.workunit.client.1.smithi181.stdout:8/698: dread - d1/d8c/fdc zero size 2024-04-27T04:10:06.919 INFO:tasks.workunit.client.0.smithi162.stdout:1/394: mkdir d0/d11/d50/d81 0 2024-04-27T04:10:06.922 INFO:tasks.workunit.client.1.smithi181.stdout:2/670: dread d4/d9/f7a [0,4194304] 0 2024-04-27T04:10:06.923 INFO:tasks.workunit.client.1.smithi181.stdout:2/671: stat d4/d9/df/d14/da7 0 2024-04-27T04:10:06.924 INFO:tasks.workunit.client.0.smithi162.stdout:5/448: dread d8/f26 [0,4194304] 0 2024-04-27T04:10:06.924 INFO:tasks.workunit.client.0.smithi162.stdout:7/370: creat d3/d5/d3c/f7e x:0 0 0 2024-04-27T04:10:06.924 INFO:tasks.workunit.client.0.smithi162.stdout:4/418: symlink d2/d54/l87 0 2024-04-27T04:10:06.925 INFO:tasks.workunit.client.1.smithi181.stdout:8/699: fsync d1/d6/d2f/d47/f4b 0 2024-04-27T04:10:06.927 INFO:tasks.workunit.client.1.smithi181.stdout:8/700: truncate d1/d8c/fdc 89511 0 2024-04-27T04:10:06.927 INFO:tasks.workunit.client.0.smithi162.stdout:5/449: chown d8/d9/d5b/f71 83 1 2024-04-27T04:10:06.927 INFO:tasks.workunit.client.0.smithi162.stdout:8/443: dwrite d0/d10/f9a [0,4194304] 0 2024-04-27T04:10:06.928 INFO:tasks.workunit.client.1.smithi181.stdout:2/672: rmdir d4/d9/d16/d45/d9a/d9d 39 2024-04-27T04:10:06.928 INFO:tasks.workunit.client.1.smithi181.stdout:2/673: chown d4/d9/l31 270194178 1 2024-04-27T04:10:06.930 INFO:tasks.workunit.client.0.smithi162.stdout:8/444: write d0/d10/d2b/d2d/d2f/d39/f59 [265899,67819] 0 2024-04-27T04:10:06.935 INFO:tasks.workunit.client.1.smithi181.stdout:8/701: mkdir d1/d20/df4 0 2024-04-27T04:10:06.936 INFO:tasks.workunit.client.0.smithi162.stdout:6/441: link c1 d6/d1a/d34/d7b/d7e/c8f 0 2024-04-27T04:10:06.939 INFO:tasks.workunit.client.1.smithi181.stdout:2/674: creat d4/d9/d16/d45/fdc x:0 0 0 2024-04-27T04:10:06.945 INFO:tasks.workunit.client.1.smithi181.stdout:6/504: write d7/dc/d1c/d7f/f29 [333941,116840] 0 2024-04-27T04:10:06.950 INFO:tasks.workunit.client.1.smithi181.stdout:5/738: dread d4/de/f14 [0,4194304] 0 2024-04-27T04:10:06.950 INFO:tasks.workunit.client.1.smithi181.stdout:8/702: dread d1/d6/d1a/d25/f52 [0,4194304] 0 2024-04-27T04:10:06.951 INFO:tasks.workunit.client.1.smithi181.stdout:8/703: chown d1/d6/d6f/c80 27 1 2024-04-27T04:10:06.951 INFO:tasks.workunit.client.1.smithi181.stdout:8/704: readlink d1/d6/da6/la9 0 2024-04-27T04:10:06.956 INFO:tasks.workunit.client.0.smithi162.stdout:0/475: write d5/dc/d1e/f30 [52253,123724] 0 2024-04-27T04:10:06.960 INFO:tasks.workunit.client.0.smithi162.stdout:8/445: dread d0/de/f7b [0,4194304] 0 2024-04-27T04:10:06.960 INFO:tasks.workunit.client.0.smithi162.stdout:2/377: link d0/f7 d0/d4/d2a/d3b/d49/d5a/d63/d6c/f76 0 2024-04-27T04:10:06.961 INFO:tasks.workunit.client.0.smithi162.stdout:8/446: write d0/de/f7b [479687,120260] 0 2024-04-27T04:10:06.962 INFO:tasks.workunit.client.0.smithi162.stdout:9/405: truncate d3/d23/f4b 3918663 0 2024-04-27T04:10:06.963 INFO:tasks.workunit.client.0.smithi162.stdout:7/371: write d3/d5/d9/f17 [1314758,36878] 0 2024-04-27T04:10:06.965 INFO:tasks.workunit.client.1.smithi181.stdout:2/675: rename d4/d9/df/l13 to d4/d9/d16/ldd 0 2024-04-27T04:10:06.965 INFO:tasks.workunit.client.0.smithi162.stdout:6/442: read d6/d1a/f1d [448442,74790] 0 2024-04-27T04:10:06.967 INFO:tasks.workunit.client.1.smithi181.stdout:5/739: write d4/de/f6f [390660,46189] 0 2024-04-27T04:10:06.967 INFO:tasks.workunit.client.1.smithi181.stdout:6/505: truncate d7/dc/f3a 4031820 0 2024-04-27T04:10:06.970 INFO:tasks.workunit.client.0.smithi162.stdout:7/372: dread d3/d5/d9/d4b/de/d2b/f56 [0,4194304] 0 2024-04-27T04:10:06.972 INFO:tasks.workunit.client.0.smithi162.stdout:6/443: dwrite d6/db/f6a [0,4194304] 0 2024-04-27T04:10:06.974 INFO:tasks.workunit.client.1.smithi181.stdout:5/740: creat d4/de/d2c/d46/d99/f105 x:0 0 0 2024-04-27T04:10:06.977 INFO:tasks.workunit.client.1.smithi181.stdout:6/506: truncate d7/dc/d1c/d7f/d3e/f54 2604269 0 2024-04-27T04:10:06.978 INFO:tasks.workunit.client.0.smithi162.stdout:5/450: getdents d8/d1a/d23/d40/d7b 0 2024-04-27T04:10:06.978 INFO:tasks.workunit.client.0.smithi162.stdout:4/419: dwrite d2/da/d1a/f30 [0,4194304] 0 2024-04-27T04:10:06.979 INFO:tasks.workunit.client.0.smithi162.stdout:3/423: truncate d1/d18/f11 396828 0 2024-04-27T04:10:06.980 INFO:tasks.workunit.client.1.smithi181.stdout:8/705: creat d1/d6/d6f/ff5 x:0 0 0 2024-04-27T04:10:06.982 INFO:tasks.workunit.client.1.smithi181.stdout:5/741: rmdir d4/de/d2c/d46 39 2024-04-27T04:10:06.986 INFO:tasks.workunit.client.0.smithi162.stdout:0/476: creat d5/d26/d4a/f96 x:0 0 0 2024-04-27T04:10:06.986 INFO:tasks.workunit.client.0.smithi162.stdout:2/378: rename d0/d20/d2e/d40 to d0/d4/d2a/d41/d77 0 2024-04-27T04:10:06.987 INFO:tasks.workunit.client.0.smithi162.stdout:2/379: fsync d0/f1 0 2024-04-27T04:10:06.987 INFO:tasks.workunit.client.1.smithi181.stdout:8/706: creat d1/d6/d2f/d30/d66/dc0/dcd/ff6 x:0 0 0 2024-04-27T04:10:06.988 INFO:tasks.workunit.client.0.smithi162.stdout:8/447: truncate d0/d10/d2b/d2d/d2f/f54 3243354 0 2024-04-27T04:10:06.989 INFO:tasks.workunit.client.0.smithi162.stdout:0/477: dread d5/f2d [0,4194304] 0 2024-04-27T04:10:06.991 INFO:tasks.workunit.client.0.smithi162.stdout:2/380: dread d0/f1 [4194304,4194304] 0 2024-04-27T04:10:06.993 INFO:tasks.workunit.client.1.smithi181.stdout:2/676: dread d4/d9/d16/f26 [0,4194304] 0 2024-04-27T04:10:06.993 INFO:tasks.workunit.client.1.smithi181.stdout:8/707: truncate d1/d6/d2f/d72/fc6 816567 0 2024-04-27T04:10:06.995 INFO:tasks.workunit.client.0.smithi162.stdout:7/373: unlink d3/d5/d3c/d4a/f70 0 2024-04-27T04:10:06.996 INFO:tasks.workunit.client.1.smithi181.stdout:6/507: link d7/d31/d45/f64 d7/d27/fae 0 2024-04-27T04:10:06.996 INFO:tasks.workunit.client.1.smithi181.stdout:6/508: chown d7/dc/d1c/d7f 904 1 2024-04-27T04:10:07.005 INFO:tasks.workunit.client.0.smithi162.stdout:4/420: symlink d2/d54/d7a/l88 0 2024-04-27T04:10:07.006 INFO:tasks.workunit.client.1.smithi181.stdout:6/509: mknod d7/d31/caf 0 2024-04-27T04:10:07.008 INFO:tasks.workunit.client.1.smithi181.stdout:2/677: unlink d4/d9/d4f/d79/d94/f53 0 2024-04-27T04:10:07.014 INFO:tasks.workunit.client.0.smithi162.stdout:2/381: dread d0/f7 [0,4194304] 0 2024-04-27T04:10:07.017 INFO:tasks.workunit.client.1.smithi181.stdout:4/584: sync 2024-04-27T04:10:07.017 INFO:tasks.workunit.client.1.smithi181.stdout:1/613: sync 2024-04-27T04:10:07.019 INFO:tasks.workunit.client.0.smithi162.stdout:2/382: dwrite d0/f1 [4194304,4194304] 0 2024-04-27T04:10:07.022 INFO:tasks.workunit.client.1.smithi181.stdout:5/742: write d4/da3/dbc/db5/fc1 [398415,81139] 0 2024-04-27T04:10:07.023 INFO:tasks.workunit.client.1.smithi181.stdout:2/678: read d4/f50 [1465351,102377] 0 2024-04-27T04:10:07.027 INFO:tasks.workunit.client.1.smithi181.stdout:4/585: fdatasync d0/d15/d5c/f7d 0 2024-04-27T04:10:07.037 INFO:tasks.workunit.client.1.smithi181.stdout:2/679: mknod d4/d9/df/d14/da7/cde 0 2024-04-27T04:10:07.038 INFO:tasks.workunit.client.0.smithi162.stdout:3/424: truncate d1/d18/f52 4243610 0 2024-04-27T04:10:07.040 INFO:tasks.workunit.client.1.smithi181.stdout:5/743: rename d4/de/c7c to d4/d10/d72/d80/d85/c106 0 2024-04-27T04:10:07.041 INFO:tasks.workunit.client.0.smithi162.stdout:3/425: dread d1/d18/f4d [0,4194304] 0 2024-04-27T04:10:07.042 INFO:tasks.workunit.client.0.smithi162.stdout:3/426: chown d1/d18/f4d 122581116 1 2024-04-27T04:10:07.042 INFO:tasks.workunit.client.1.smithi181.stdout:8/708: dwrite d1/d6/d1a/d25/f99 [4194304,4194304] 0 2024-04-27T04:10:07.044 INFO:tasks.workunit.client.0.smithi162.stdout:5/451: getdents d8/d1a/d23/d40/d4b/d66/d88 0 2024-04-27T04:10:07.046 INFO:tasks.workunit.client.0.smithi162.stdout:5/452: stat d8/d1a/d23/d40/c4e 0 2024-04-27T04:10:07.049 INFO:tasks.workunit.client.0.smithi162.stdout:4/421: dwrite d2/f22 [4194304,4194304] 0 2024-04-27T04:10:07.051 INFO:tasks.workunit.client.0.smithi162.stdout:5/453: truncate d8/d9/d55/f69 579838 0 2024-04-27T04:10:07.051 INFO:tasks.workunit.client.0.smithi162.stdout:9/406: link d3/d23/c29 d3/d13/d2d/d4f/c81 0 2024-04-27T04:10:07.053 INFO:tasks.workunit.client.0.smithi162.stdout:4/422: dread - d2/d25/d1c/d39/f7f zero size 2024-04-27T04:10:07.053 INFO:tasks.workunit.client.0.smithi162.stdout:3/427: dwrite d1/d18/d23/f28 [0,4194304] 0 2024-04-27T04:10:07.054 INFO:tasks.workunit.client.1.smithi181.stdout:2/680: unlink d4/d9/d16/d82/l7f 0 2024-04-27T04:10:07.055 INFO:tasks.workunit.client.1.smithi181.stdout:6/510: truncate d7/d31/d45/d62/f63 640166 0 2024-04-27T04:10:07.055 INFO:tasks.workunit.client.0.smithi162.stdout:9/407: truncate d3/d23/f5c 522076 0 2024-04-27T04:10:07.055 INFO:tasks.workunit.client.1.smithi181.stdout:1/614: dwrite d1/dad/db5/d6/db/d2d/d36/f4a [0,4194304] 0 2024-04-27T04:10:07.057 INFO:tasks.workunit.client.0.smithi162.stdout:1/395: sync 2024-04-27T04:10:07.058 INFO:tasks.workunit.client.0.smithi162.stdout:6/444: creat d6/f90 x:0 0 0 2024-04-27T04:10:07.063 INFO:tasks.workunit.client.1.smithi181.stdout:4/586: creat d0/d15/de/d19/d8a/fbe x:0 0 0 2024-04-27T04:10:07.065 INFO:tasks.workunit.client.0.smithi162.stdout:5/454: dread d8/d1a/d23/d40/f6e [0,4194304] 0 2024-04-27T04:10:07.068 INFO:tasks.workunit.client.1.smithi181.stdout:6/511: mknod d7/d31/d45/d72/cb0 0 2024-04-27T04:10:07.069 INFO:tasks.workunit.client.0.smithi162.stdout:3/428: dwrite d1/d18/d23/f53 [0,4194304] 0 2024-04-27T04:10:07.069 INFO:tasks.workunit.client.1.smithi181.stdout:5/744: getdents d4/da9/de7 0 2024-04-27T04:10:07.070 INFO:tasks.workunit.client.1.smithi181.stdout:6/512: fdatasync d7/dc/d1c/d34/f84 0 2024-04-27T04:10:07.071 INFO:tasks.workunit.client.1.smithi181.stdout:5/745: readlink d4/da9/d100/d55/lc0 0 2024-04-27T04:10:07.072 INFO:tasks.workunit.client.1.smithi181.stdout:5/746: read d4/d10/feb [422020,34837] 0 2024-04-27T04:10:07.073 INFO:tasks.workunit.client.0.smithi162.stdout:1/396: dwrite d0/d11/d3e/f6d [0,4194304] 0 2024-04-27T04:10:07.073 INFO:tasks.workunit.client.1.smithi181.stdout:5/747: chown d4/da9/d100/lee 20 1 2024-04-27T04:10:07.079 INFO:tasks.workunit.client.1.smithi181.stdout:6/513: mknod d7/d27/cb1 0 2024-04-27T04:10:07.081 INFO:tasks.workunit.client.1.smithi181.stdout:6/514: fdatasync d7/dc/d1c/f99 0 2024-04-27T04:10:07.083 INFO:tasks.workunit.client.0.smithi162.stdout:4/423: rename d2/d25/d1c/d39/f71 to d2/d25/f89 0 2024-04-27T04:10:07.089 INFO:tasks.workunit.client.0.smithi162.stdout:9/408: dread d3/d13/d2d/d3b/f47 [0,4194304] 0 2024-04-27T04:10:07.091 INFO:tasks.workunit.client.0.smithi162.stdout:7/374: creat d3/d5/d9/f7f x:0 0 0 2024-04-27T04:10:07.091 INFO:tasks.workunit.client.0.smithi162.stdout:9/409: truncate d3/d13/d15/f19 8472654 0 2024-04-27T04:10:07.092 INFO:tasks.workunit.client.1.smithi181.stdout:2/681: creat d4/d9/d35/fdf x:0 0 0 2024-04-27T04:10:07.093 INFO:tasks.workunit.client.0.smithi162.stdout:9/410: write d3/fc [3471645,25153] 0 2024-04-27T04:10:07.098 INFO:tasks.workunit.client.0.smithi162.stdout:7/375: dwrite d3/d5/d9/d4b/f3f [0,4194304] 0 2024-04-27T04:10:07.098 INFO:tasks.workunit.client.1.smithi181.stdout:6/515: stat d7/f26 0 2024-04-27T04:10:07.099 INFO:tasks.workunit.client.0.smithi162.stdout:7/376: stat d3/d4f/f78 0 2024-04-27T04:10:07.100 INFO:tasks.workunit.client.0.smithi162.stdout:7/377: fdatasync d3/d5/d9/d4b/f79 0 2024-04-27T04:10:07.102 INFO:tasks.workunit.client.1.smithi181.stdout:1/615: creat d1/dad/db5/d6/da3/d1e/fc5 x:0 0 0 2024-04-27T04:10:07.106 INFO:tasks.workunit.client.0.smithi162.stdout:8/448: rmdir d0/d10/d25/d7d/d99 0 2024-04-27T04:10:07.106 INFO:tasks.workunit.client.1.smithi181.stdout:4/587: dwrite d0/d15/de/f34 [4194304,4194304] 0 2024-04-27T04:10:07.111 INFO:tasks.workunit.client.0.smithi162.stdout:3/429: symlink d1/d18/d23/d34/l8a 0 2024-04-27T04:10:07.112 INFO:tasks.workunit.client.1.smithi181.stdout:2/682: creat d4/d9/d16/d82/d7d/fe0 x:0 0 0 2024-04-27T04:10:07.117 INFO:tasks.workunit.client.0.smithi162.stdout:0/478: sync 2024-04-27T04:10:07.120 INFO:tasks.workunit.client.1.smithi181.stdout:1/616: symlink d1/dad/lc6 0 2024-04-27T04:10:07.123 INFO:tasks.workunit.client.0.smithi162.stdout:4/424: rmdir d2/d25/d1c/d39/d3b/d6e/d6f 39 2024-04-27T04:10:07.129 INFO:tasks.workunit.client.0.smithi162.stdout:1/397: dread d0/f3b [0,4194304] 0 2024-04-27T04:10:07.132 INFO:tasks.workunit.client.0.smithi162.stdout:6/445: truncate d6/d1a/f18 5477556 0 2024-04-27T04:10:07.135 INFO:tasks.workunit.client.1.smithi181.stdout:1/617: rename d1/dad/db5/d6/da3/d1e/d48/db4/cb6 to d1/dad/db5/d6/db/d2d/d36/dc4/cc7 0 2024-04-27T04:10:07.136 INFO:tasks.workunit.client.1.smithi181.stdout:1/618: dread - d1/dad/db5/d6/da3/d1e/fc5 zero size 2024-04-27T04:10:07.137 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:10:06 smithi162 ceph-mon[26841]: from='mgr.24367 172.21.15.181:0/3181919980' entity='mgr.smithi181.vtwuhz' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi181.vtwuhz/trash_purge_schedule"}]: dispatch 2024-04-27T04:10:07.137 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:10:06 smithi162 ceph-mon[26841]: from='mgr.24367 172.21.15.181:0/3181919980' entity='mgr.smithi181.vtwuhz' 2024-04-27T04:10:07.137 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:10:06 smithi162 ceph-mon[26841]: from='mgr.24367 172.21.15.181:0/3181919980' entity='mgr.smithi181.vtwuhz' 2024-04-27T04:10:07.137 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:10:06 smithi162 ceph-mon[26841]: mgrmap e22: smithi181.vtwuhz(active, since 1.01498s) 2024-04-27T04:10:07.138 INFO:tasks.workunit.client.1.smithi181.stdout:4/588: creat d0/d15/d25/d2a/d4c/d7c/db5/fbf x:0 0 0 2024-04-27T04:10:07.139 INFO:tasks.workunit.client.0.smithi162.stdout:2/383: getdents d0/d4/d2a 0 2024-04-27T04:10:07.139 INFO:tasks.workunit.client.1.smithi181.stdout:4/589: chown d0/d15/d25/d40/f9b 120 1 2024-04-27T04:10:07.143 INFO:tasks.workunit.client.0.smithi162.stdout:2/384: dread d0/d4/d2a/d3b/d49/d5a/d63/d6c/f76 [0,4194304] 0 2024-04-27T04:10:07.143 INFO:tasks.workunit.client.1.smithi181.stdout:6/516: dwrite d7/dc/d1e/f53 [0,4194304] 0 2024-04-27T04:10:07.143 INFO:tasks.workunit.client.0.smithi162.stdout:7/378: mkdir d3/d5/d5b/d60/d65/d80 0 2024-04-27T04:10:07.144 INFO:tasks.workunit.client.1.smithi181.stdout:6/517: dread - d7/d31/d45/d72/f92 zero size 2024-04-27T04:10:07.146 INFO:tasks.workunit.client.1.smithi181.stdout:6/518: write d7/d31/d45/f6b [590423,2775] 0 2024-04-27T04:10:07.148 INFO:tasks.workunit.client.0.smithi162.stdout:8/449: creat d0/de/d6e/d71/f9d x:0 0 0 2024-04-27T04:10:07.151 INFO:tasks.workunit.client.1.smithi181.stdout:5/748: rename d4/de/d9d to d4/de/d107 0 2024-04-27T04:10:07.153 INFO:tasks.workunit.client.0.smithi162.stdout:0/479: creat d5/dc/d1e/d46/d48/f97 x:0 0 0 2024-04-27T04:10:07.157 INFO:tasks.workunit.client.0.smithi162.stdout:1/398: mknod d0/d7/d3c/d77/c82 0 2024-04-27T04:10:07.159 INFO:tasks.workunit.client.1.smithi181.stdout:6/519: mkdir d7/dc/d1c/d34/d3f/db2 0 2024-04-27T04:10:07.160 INFO:tasks.workunit.client.0.smithi162.stdout:9/411: mknod d3/c82 0 2024-04-27T04:10:07.164 INFO:tasks.workunit.client.0.smithi162.stdout:7/379: unlink d3/d5/d5b/f6c 0 2024-04-27T04:10:07.171 INFO:tasks.workunit.client.0.smithi162.stdout:3/430: unlink d1/d18/f1f 0 2024-04-27T04:10:07.171 INFO:tasks.workunit.client.1.smithi181.stdout:4/590: link d0/d15/de/d19/c7b d0/d15/d25/d40/cc0 0 2024-04-27T04:10:07.172 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:10:06 smithi181 ceph-mon[33774]: from='mgr.24367 172.21.15.181:0/3181919980' entity='mgr.smithi181.vtwuhz' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi181.vtwuhz/trash_purge_schedule"}]: dispatch 2024-04-27T04:10:07.172 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:10:06 smithi181 ceph-mon[33774]: from='mgr.24367 172.21.15.181:0/3181919980' entity='mgr.smithi181.vtwuhz' 2024-04-27T04:10:07.172 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:10:06 smithi181 ceph-mon[33774]: from='mgr.24367 172.21.15.181:0/3181919980' entity='mgr.smithi181.vtwuhz' 2024-04-27T04:10:07.172 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:10:06 smithi181 ceph-mon[33774]: mgrmap e22: smithi181.vtwuhz(active, since 1.01498s) 2024-04-27T04:10:07.173 INFO:tasks.workunit.client.0.smithi162.stdout:0/480: mkdir d5/d60/d98 0 2024-04-27T04:10:07.175 INFO:tasks.workunit.client.1.smithi181.stdout:1/619: getdents d1/d2 0 2024-04-27T04:10:07.178 INFO:tasks.workunit.client.1.smithi181.stdout:5/749: truncate d4/de/d2c/d46/d4d/d8e/d54/fcf 1862572 0 2024-04-27T04:10:07.179 INFO:tasks.workunit.client.1.smithi181.stdout:6/520: write d7/f49 [991417,83410] 0 2024-04-27T04:10:07.181 INFO:tasks.workunit.client.0.smithi162.stdout:6/446: mknod d6/d1a/d34/d3a/d68/d6c/c91 0 2024-04-27T04:10:07.184 INFO:tasks.workunit.client.0.smithi162.stdout:7/380: mkdir d3/d5/d9/d4b/de/d2b/d81 0 2024-04-27T04:10:07.185 INFO:tasks.workunit.client.0.smithi162.stdout:7/381: truncate d3/d4f/f78 678303 0 2024-04-27T04:10:07.185 INFO:tasks.workunit.client.1.smithi181.stdout:5/750: rename d4/d10/d72/d80/c7d to d4/de/d2c/d46/d99/dad/dca/d102/c108 0 2024-04-27T04:10:07.186 INFO:tasks.workunit.client.0.smithi162.stdout:5/455: link d8/d1a/d23/d40/d4b/d66/c94 d8/d1a/d7d/c97 0 2024-04-27T04:10:07.186 INFO:tasks.workunit.client.1.smithi181.stdout:5/751: read - d4/de/d2c/d46/d99/dad/fe2 zero size 2024-04-27T04:10:07.188 INFO:tasks.workunit.client.1.smithi181.stdout:6/521: mkdir d7/dc/d1c/d88/db3 0 2024-04-27T04:10:07.189 INFO:tasks.workunit.client.0.smithi162.stdout:8/450: chown d0/de/d20/f60 27 1 2024-04-27T04:10:07.189 INFO:tasks.workunit.client.0.smithi162.stdout:8/451: fsync d0/f7 0 2024-04-27T04:10:07.192 INFO:tasks.workunit.client.0.smithi162.stdout:3/431: mknod d1/d18/d23/d2f/c8b 0 2024-04-27T04:10:07.196 INFO:tasks.workunit.client.0.smithi162.stdout:4/425: rename d2/d25/d1c/d39/d3b/d52 to d2/d8a 0 2024-04-27T04:10:07.197 INFO:tasks.workunit.client.1.smithi181.stdout:4/591: dwrite d0/d15/d25/d2a/d4c/f4e [0,4194304] 0 2024-04-27T04:10:07.197 INFO:tasks.workunit.client.1.smithi181.stdout:5/752: mkdir d4/d10/dec/d109 0 2024-04-27T04:10:07.199 INFO:tasks.workunit.client.0.smithi162.stdout:1/399: creat d0/d11/d3e/d59/d7f/f83 x:0 0 0 2024-04-27T04:10:07.200 INFO:tasks.workunit.client.1.smithi181.stdout:6/522: mknod d7/d27/cb4 0 2024-04-27T04:10:07.200 INFO:tasks.workunit.client.0.smithi162.stdout:9/412: getdents d3/d13/d15/d7e 0 2024-04-27T04:10:07.201 INFO:tasks.workunit.client.0.smithi162.stdout:6/447: rmdir d6/d1a/d34/d7b/d7e 39 2024-04-27T04:10:07.201 INFO:tasks.workunit.client.0.smithi162.stdout:9/413: write d3/f5 [3094762,110631] 0 2024-04-27T04:10:07.203 INFO:tasks.workunit.client.0.smithi162.stdout:9/414: stat d3/d13/d15 0 2024-04-27T04:10:07.204 INFO:tasks.workunit.client.0.smithi162.stdout:9/415: write d3/d11/f5e [4209420,67379] 0 2024-04-27T04:10:07.208 INFO:tasks.workunit.client.0.smithi162.stdout:1/400: dwrite d0/d7/d10/f64 [0,4194304] 0 2024-04-27T04:10:07.214 INFO:tasks.workunit.client.0.smithi162.stdout:1/401: dwrite d0/d7/d2a/f47 [0,4194304] 0 2024-04-27T04:10:07.237 INFO:tasks.workunit.client.1.smithi181.stdout:9/573: sync 2024-04-27T04:10:07.242 INFO:tasks.workunit.client.0.smithi162.stdout:3/432: truncate d1/d45/f79 957814 0 2024-04-27T04:10:07.242 INFO:tasks.workunit.client.0.smithi162.stdout:0/481: creat d5/d60/d8b/f99 x:0 0 0 2024-04-27T04:10:07.245 INFO:tasks.workunit.client.0.smithi162.stdout:2/385: rename d0/d4/d17/c28 to d0/d4/d2a/d3b/d49/d5a/d63/d70/c78 0 2024-04-27T04:10:07.250 INFO:tasks.workunit.client.0.smithi162.stdout:5/456: dread d8/d9/f3d [0,4194304] 0 2024-04-27T04:10:07.251 INFO:tasks.workunit.client.0.smithi162.stdout:9/416: mknod d3/d13/d2d/c83 0 2024-04-27T04:10:07.254 INFO:tasks.workunit.client.0.smithi162.stdout:9/417: dread d3/d11/f5e [0,4194304] 0 2024-04-27T04:10:07.258 INFO:tasks.workunit.client.0.smithi162.stdout:8/452: creat d0/d10/d25/d65/f9e x:0 0 0 2024-04-27T04:10:07.259 INFO:tasks.workunit.client.0.smithi162.stdout:7/382: sync 2024-04-27T04:10:07.270 INFO:tasks.workunit.client.0.smithi162.stdout:6/448: symlink d6/d1a/d34/d7b/l92 0 2024-04-27T04:10:07.270 INFO:tasks.workunit.client.0.smithi162.stdout:6/449: chown d6/l59 1819407771 1 2024-04-27T04:10:07.275 INFO:tasks.workunit.client.0.smithi162.stdout:1/402: getdents d0/d11/d50/d81 0 2024-04-27T04:10:07.275 INFO:tasks.workunit.client.0.smithi162.stdout:1/403: dread - d0/d7/d3c/f43 zero size 2024-04-27T04:10:07.278 INFO:tasks.workunit.client.0.smithi162.stdout:0/482: fsync d5/dc/d1e/d41/f39 0 2024-04-27T04:10:07.279 INFO:tasks.workunit.client.0.smithi162.stdout:0/483: stat d5/dc/f91 0 2024-04-27T04:10:07.281 INFO:tasks.workunit.client.0.smithi162.stdout:4/426: link d2/d25/d1c/c26 d2/d25/d1c/d24/c8b 0 2024-04-27T04:10:07.281 INFO:tasks.workunit.client.0.smithi162.stdout:2/386: write d0/d4/f16 [474967,55044] 0 2024-04-27T04:10:07.282 INFO:tasks.workunit.client.1.smithi181.stdout:4/592: unlink d0/d15/de/d19/d59/f73 0 2024-04-27T04:10:07.285 INFO:tasks.workunit.client.0.smithi162.stdout:0/484: dwrite d5/d60/d8b/f99 [0,4194304] 0 2024-04-27T04:10:07.285 INFO:tasks.workunit.client.0.smithi162.stdout:6/450: chown d6/cd 498164424 1 2024-04-27T04:10:07.285 INFO:tasks.workunit.client.1.smithi181.stdout:5/753: mkdir d4/de/d2c/d46/d99/dad/dca/d10a 0 2024-04-27T04:10:07.287 INFO:tasks.workunit.client.0.smithi162.stdout:5/457: creat d8/d1a/d23/d40/d4b/d66/d88/f98 x:0 0 0 2024-04-27T04:10:07.289 INFO:tasks.workunit.client.1.smithi181.stdout:9/574: creat d0/d44/d4b/d43/d89/daf/fca x:0 0 0 2024-04-27T04:10:07.290 INFO:tasks.workunit.client.1.smithi181.stdout:1/620: getdents d1/dad/db5/d6/da3/d1e/d48 0 2024-04-27T04:10:07.295 INFO:tasks.workunit.client.1.smithi181.stdout:4/593: rename d0/d15/de/d19/f5e to d0/d15/d25/d2a/da0/fc1 0 2024-04-27T04:10:07.299 INFO:tasks.workunit.client.1.smithi181.stdout:9/575: mknod d0/d44/d4b/d5f/ccb 0 2024-04-27T04:10:07.303 INFO:tasks.workunit.client.1.smithi181.stdout:4/594: symlink d0/d15/d76/da4/lc2 0 2024-04-27T04:10:07.313 INFO:tasks.workunit.client.1.smithi181.stdout:4/595: creat d0/d15/d25/d2b/d6d/d82/fc3 x:0 0 0 2024-04-27T04:10:07.314 INFO:tasks.workunit.client.1.smithi181.stdout:5/754: link d4/da9/d100/d55/lb9 d4/da9/d100/d8f/l10b 0 2024-04-27T04:10:07.316 INFO:tasks.workunit.client.1.smithi181.stdout:1/621: link d1/dad/db5/d6/db/d2d/d36/cb2 d1/dad/db5/d6/da3/d1e/d20/d53/cc8 0 2024-04-27T04:10:07.316 INFO:tasks.workunit.client.1.smithi181.stdout:1/622: chown d1/dad/db5/d6/f4e 1 1 2024-04-27T04:10:07.317 INFO:tasks.workunit.client.1.smithi181.stdout:4/596: truncate d0/d15/f1a 3791224 0 2024-04-27T04:10:07.318 INFO:tasks.workunit.client.1.smithi181.stdout:4/597: truncate d0/d15/de/fb7 814577 0 2024-04-27T04:10:07.319 INFO:tasks.workunit.client.1.smithi181.stdout:5/755: mknod d4/de/d2c/d46/d4d/c10c 0 2024-04-27T04:10:07.324 INFO:tasks.workunit.client.1.smithi181.stdout:4/598: rename d0/d15/f6a to d0/d15/de/d60/fc4 0 2024-04-27T04:10:07.325 INFO:tasks.workunit.client.1.smithi181.stdout:9/576: dwrite d0/d44/d4f/f7a [0,4194304] 0 2024-04-27T04:10:07.331 INFO:tasks.workunit.client.1.smithi181.stdout:5/756: readlink d4/de/d2f/l38 0 2024-04-27T04:10:07.331 INFO:tasks.workunit.client.1.smithi181.stdout:1/623: write d1/dad/db5/d6/f4e [3736612,87774] 0 2024-04-27T04:10:07.332 INFO:tasks.workunit.client.1.smithi181.stdout:5/757: chown d4/de/d2c/l4b 1360 1 2024-04-27T04:10:07.333 INFO:tasks.workunit.client.1.smithi181.stdout:9/577: creat d0/d44/d4b/d3d/dbd/fcc x:0 0 0 2024-04-27T04:10:07.334 INFO:tasks.workunit.client.1.smithi181.stdout:4/599: mkdir d0/d15/de/d19/d77/dc5 0 2024-04-27T04:10:07.336 INFO:tasks.workunit.client.0.smithi162.stdout:3/433: link d1/d18/d23/f53 d1/d70/f8c 0 2024-04-27T04:10:07.337 INFO:tasks.workunit.client.1.smithi181.stdout:1/624: mknod d1/dad/db5/d6/d6a/cc9 0 2024-04-27T04:10:07.337 INFO:tasks.workunit.client.0.smithi162.stdout:0/485: dwrite d5/d60/f72 [0,4194304] 0 2024-04-27T04:10:07.338 INFO:tasks.workunit.client.0.smithi162.stdout:7/383: creat d3/d5/d9/d4b/de/d2b/d81/f82 x:0 0 0 2024-04-27T04:10:07.339 INFO:tasks.workunit.client.0.smithi162.stdout:2/387: unlink d0/f7 0 2024-04-27T04:10:07.340 INFO:tasks.workunit.client.0.smithi162.stdout:9/418: write d3/f6 [4680456,60778] 0 2024-04-27T04:10:07.341 INFO:tasks.workunit.client.1.smithi181.stdout:5/758: fdatasync d4/de/f47 0 2024-04-27T04:10:07.341 INFO:tasks.workunit.client.0.smithi162.stdout:0/486: chown d5/dc/d1e/d20/l54 139677505 1 2024-04-27T04:10:07.342 INFO:tasks.workunit.client.1.smithi181.stdout:4/600: readlink d0/d15/de/d19/d77/l62 0 2024-04-27T04:10:07.343 INFO:tasks.workunit.client.1.smithi181.stdout:4/601: chown d0/d15/d25/d2b/d6d/d93 7881496 1 2024-04-27T04:10:07.349 INFO:tasks.workunit.client.1.smithi181.stdout:1/625: fdatasync d1/dad/db5/d6/da3/d1e/d48/f5d 0 2024-04-27T04:10:07.350 INFO:tasks.workunit.client.0.smithi162.stdout:5/458: dread d8/d2c/f4c [0,4194304] 0 2024-04-27T04:10:07.350 INFO:tasks.workunit.client.0.smithi162.stdout:1/404: dwrite d0/d11/f42 [0,4194304] 0 2024-04-27T04:10:07.352 INFO:tasks.workunit.client.1.smithi181.stdout:5/759: dread - d4/d10/d29/d92/fd6 zero size 2024-04-27T04:10:07.354 INFO:tasks.workunit.client.1.smithi181.stdout:5/760: chown d4/de/d2c/d46/d4d/d8e/l6d 16644223 1 2024-04-27T04:10:07.357 INFO:tasks.workunit.client.1.smithi181.stdout:1/626: dwrite d1/dad/db5/fb8 [0,4194304] 0 2024-04-27T04:10:07.359 INFO:tasks.workunit.client.1.smithi181.stdout:1/627: chown d1/dad/db5/d6/da3/d52/d92/f93 6 1 2024-04-27T04:10:07.364 INFO:tasks.workunit.client.0.smithi162.stdout:3/434: dread - d1/d18/d5d/d5f/d66/f7a zero size 2024-04-27T04:10:07.365 INFO:tasks.workunit.client.1.smithi181.stdout:5/761: mkdir d4/de/d10d 0 2024-04-27T04:10:07.366 INFO:tasks.workunit.client.0.smithi162.stdout:6/451: creat d6/d1a/d34/d50/f93 x:0 0 0 2024-04-27T04:10:07.368 INFO:tasks.workunit.client.1.smithi181.stdout:5/762: mkdir d4/de/d107/d10e 0 2024-04-27T04:10:07.368 INFO:tasks.workunit.client.0.smithi162.stdout:8/453: sync 2024-04-27T04:10:07.375 INFO:tasks.workunit.client.0.smithi162.stdout:7/384: rename d3/d5/d9/d4b to d3/d5/d5b/d83 0 2024-04-27T04:10:07.377 INFO:tasks.workunit.client.0.smithi162.stdout:7/385: write d3/d5/d5b/d83/de/d2b/f56 [641490,113577] 0 2024-04-27T04:10:07.378 INFO:tasks.workunit.client.0.smithi162.stdout:7/386: write d3/d5/f53 [1003670,110038] 0 2024-04-27T04:10:07.380 INFO:tasks.workunit.client.0.smithi162.stdout:1/405: creat d0/d11/d50/d53/f84 x:0 0 0 2024-04-27T04:10:07.383 INFO:tasks.workunit.client.0.smithi162.stdout:0/487: symlink d5/dc/d1e/d41/l9a 0 2024-04-27T04:10:07.384 INFO:tasks.workunit.client.0.smithi162.stdout:3/435: symlink d1/d18/d23/d58/l8d 0 2024-04-27T04:10:07.386 INFO:tasks.workunit.client.0.smithi162.stdout:6/452: rmdir d6/d1a/d34/d3a/d60 39 2024-04-27T04:10:07.389 INFO:tasks.workunit.client.0.smithi162.stdout:9/419: link d3/d23/f22 d3/d13/d2d/d3b/f84 0 2024-04-27T04:10:07.391 INFO:tasks.workunit.client.0.smithi162.stdout:1/406: mknod d0/d7/d3c/d6e/c85 0 2024-04-27T04:10:07.394 INFO:tasks.workunit.client.0.smithi162.stdout:1/407: write d0/d7/d3c/d6e/f7b [454621,119521] 0 2024-04-27T04:10:07.394 INFO:tasks.workunit.client.0.smithi162.stdout:0/488: rmdir d5/dc/d1e/d46 39 2024-04-27T04:10:07.394 INFO:tasks.workunit.client.0.smithi162.stdout:0/489: read d5/f11 [881847,6445] 0 2024-04-27T04:10:07.395 INFO:tasks.workunit.client.0.smithi162.stdout:6/453: chown c1 1897902 1 2024-04-27T04:10:07.397 INFO:tasks.workunit.client.0.smithi162.stdout:0/490: write d5/dc/d1e/d56/f90 [826437,129439] 0 2024-04-27T04:10:07.398 INFO:tasks.workunit.client.0.smithi162.stdout:0/491: readlink d5/d26/l76 0 2024-04-27T04:10:07.398 INFO:tasks.workunit.client.0.smithi162.stdout:9/420: symlink d3/d13/d2d/d3b/l85 0 2024-04-27T04:10:07.405 INFO:tasks.workunit.client.0.smithi162.stdout:2/388: truncate d0/f1 5930147 0 2024-04-27T04:10:07.405 INFO:tasks.workunit.client.0.smithi162.stdout:0/492: dwrite d5/dc/f17 [0,4194304] 0 2024-04-27T04:10:07.409 INFO:tasks.workunit.client.0.smithi162.stdout:9/421: symlink d3/d13/d2d/d3b/l86 0 2024-04-27T04:10:07.413 INFO:tasks.workunit.client.0.smithi162.stdout:5/459: dwrite d8/d9/d55/f6d [0,4194304] 0 2024-04-27T04:10:07.416 INFO:tasks.workunit.client.0.smithi162.stdout:2/389: mknod d0/d20/d51/c79 0 2024-04-27T04:10:07.416 INFO:tasks.workunit.client.0.smithi162.stdout:7/387: truncate d3/d5/f5e 450878 0 2024-04-27T04:10:07.416 INFO:tasks.workunit.client.0.smithi162.stdout:1/408: dread d0/d7/d3c/f57 [0,4194304] 0 2024-04-27T04:10:07.416 INFO:tasks.workunit.client.0.smithi162.stdout:2/390: readlink d0/l26 0 2024-04-27T04:10:07.417 INFO:tasks.workunit.client.0.smithi162.stdout:4/427: sync 2024-04-27T04:10:07.418 INFO:tasks.workunit.client.0.smithi162.stdout:0/493: rmdir d5/dc/d1e/d56 39 2024-04-27T04:10:07.421 INFO:tasks.workunit.client.1.smithi181.stdout:3/562: sync 2024-04-27T04:10:07.422 INFO:tasks.workunit.client.1.smithi181.stdout:9/578: truncate d0/d40/f78 2118332 0 2024-04-27T04:10:07.423 INFO:tasks.workunit.client.1.smithi181.stdout:3/563: chown d0/d6c/d52 27859800 1 2024-04-27T04:10:07.424 INFO:tasks.workunit.client.1.smithi181.stdout:9/579: read d0/f55 [2815964,30212] 0 2024-04-27T04:10:07.424 INFO:tasks.workunit.client.1.smithi181.stdout:4/602: truncate d0/d15/d25/d2b/d6d/d82/f8b 426301 0 2024-04-27T04:10:07.425 INFO:tasks.workunit.client.0.smithi162.stdout:5/460: creat d8/d1a/d29/f99 x:0 0 0 2024-04-27T04:10:07.426 INFO:tasks.workunit.client.0.smithi162.stdout:8/454: dwrite d0/d10/d2b/d2d/d2f/f54 [0,4194304] 0 2024-04-27T04:10:07.427 INFO:tasks.workunit.client.1.smithi181.stdout:1/628: getdents d1/dad/db5/d6 0 2024-04-27T04:10:07.427 INFO:tasks.workunit.client.0.smithi162.stdout:6/454: link d6/c79 d6/d1a/d34/c94 0 2024-04-27T04:10:07.427 INFO:tasks.workunit.client.0.smithi162.stdout:7/388: write d3/d5/d5b/d83/de/d2b/f56 [811587,56272] 0 2024-04-27T04:10:07.430 INFO:tasks.workunit.client.1.smithi181.stdout:3/564: dread d0/d6c/f79 [0,4194304] 0 2024-04-27T04:10:07.432 INFO:tasks.workunit.client.0.smithi162.stdout:9/422: dwrite d3/d13/d15/f2f [4194304,4194304] 0 2024-04-27T04:10:07.432 INFO:tasks.workunit.client.0.smithi162.stdout:8/455: fsync d0/d10/f9a 0 2024-04-27T04:10:07.438 INFO:tasks.workunit.client.0.smithi162.stdout:0/494: mkdir d5/d26/d69/d9b 0 2024-04-27T04:10:07.441 INFO:tasks.workunit.client.0.smithi162.stdout:3/436: dwrite d1/d18/d5d/f62 [0,4194304] 0 2024-04-27T04:10:07.445 INFO:tasks.workunit.client.1.smithi181.stdout:3/565: fsync d0/f32 0 2024-04-27T04:10:07.446 INFO:tasks.workunit.client.0.smithi162.stdout:1/409: mknod d0/d11/d50/d81/c86 0 2024-04-27T04:10:07.447 INFO:tasks.workunit.client.1.smithi181.stdout:4/603: symlink d0/d15/d25/d2b/d6d/lc6 0 2024-04-27T04:10:07.449 INFO:tasks.workunit.client.0.smithi162.stdout:1/410: read - d0/d11/d50/d53/f84 zero size 2024-04-27T04:10:07.449 INFO:tasks.workunit.client.1.smithi181.stdout:1/629: mknod d1/dad/db5/d6/da3/d1e/d20/cca 0 2024-04-27T04:10:07.450 INFO:tasks.workunit.client.1.smithi181.stdout:5/763: write d4/de/d2c/d46/d4d/d8e/f3a [3431535,92876] 0 2024-04-27T04:10:07.451 INFO:tasks.workunit.client.0.smithi162.stdout:0/495: stat d5/dc/d1e/d56/d57/l66 0 2024-04-27T04:10:07.456 INFO:tasks.workunit.client.1.smithi181.stdout:5/764: dread d4/da3/dbc/f9e [0,4194304] 0 2024-04-27T04:10:07.456 INFO:tasks.workunit.client.0.smithi162.stdout:1/411: dwrite d0/d11/d3e/d59/f7a [0,4194304] 0 2024-04-27T04:10:07.457 INFO:tasks.workunit.client.1.smithi181.stdout:5/765: write d4/fc9 [897835,96716] 0 2024-04-27T04:10:07.457 INFO:tasks.workunit.client.0.smithi162.stdout:1/412: readlink d0/d7/d10/l33 0 2024-04-27T04:10:07.458 INFO:tasks.workunit.client.1.smithi181.stdout:3/566: rename d0/de/d20/d58/d70 to d0/d6c/da1/daf 0 2024-04-27T04:10:07.461 INFO:tasks.workunit.client.1.smithi181.stdout:4/604: mkdir d0/d15/d25/d2a/da0/dc7 0 2024-04-27T04:10:07.462 INFO:tasks.workunit.client.0.smithi162.stdout:4/428: dread d2/da/d1a/f2e [0,4194304] 0 2024-04-27T04:10:07.462 INFO:tasks.workunit.client.0.smithi162.stdout:2/391: link d0/d4/f22 d0/d4/d2a/d3b/d49/d5a/d63/d6e/f7a 0 2024-04-27T04:10:07.464 INFO:tasks.workunit.client.1.smithi181.stdout:1/630: mkdir d1/dad/db5/d6/dcb 0 2024-04-27T04:10:07.467 INFO:tasks.workunit.client.0.smithi162.stdout:3/437: mknod d1/d45/d69/c8e 0 2024-04-27T04:10:07.468 INFO:tasks.workunit.client.0.smithi162.stdout:1/413: dwrite d0/d7/d2a/f31 [0,4194304] 0 2024-04-27T04:10:07.469 INFO:tasks.workunit.client.0.smithi162.stdout:7/389: creat d3/d5/d5b/d60/d65/d80/f84 x:0 0 0 2024-04-27T04:10:07.473 INFO:tasks.workunit.client.0.smithi162.stdout:4/429: dwrite d2/da/dd/f57 [0,4194304] 0 2024-04-27T04:10:07.473 INFO:tasks.workunit.client.0.smithi162.stdout:0/496: rename d5/dc/d37/c3f to d5/dc/d1e/d56/d57/c9c 0 2024-04-27T04:10:07.474 INFO:tasks.workunit.client.0.smithi162.stdout:7/390: read d3/d5/d5b/d83/de/d2b/f69 [1016843,56157] 0 2024-04-27T04:10:07.476 INFO:tasks.workunit.client.0.smithi162.stdout:9/423: write d3/d13/d2d/d3b/f84 [2929533,30561] 0 2024-04-27T04:10:07.476 INFO:tasks.workunit.client.0.smithi162.stdout:9/424: readlink d3/d13/d15/l1a 0 2024-04-27T04:10:07.482 INFO:tasks.workunit.client.0.smithi162.stdout:2/392: mkdir d0/d4/d2a/d41/d7b 0 2024-04-27T04:10:07.484 INFO:tasks.workunit.client.0.smithi162.stdout:2/393: read d0/d24/f4d [5359943,59568] 0 2024-04-27T04:10:07.490 INFO:tasks.workunit.client.0.smithi162.stdout:3/438: dread d1/d18/d5d/d5f/f77 [0,4194304] 0 2024-04-27T04:10:07.495 INFO:tasks.workunit.client.1.smithi181.stdout:5/766: write d4/de/d2c/d46/d4d/d8e/d54/f78 [4339413,66369] 0 2024-04-27T04:10:07.496 INFO:tasks.workunit.client.0.smithi162.stdout:3/439: dwrite d1/d45/f87 [0,4194304] 0 2024-04-27T04:10:07.500 INFO:tasks.workunit.client.1.smithi181.stdout:5/767: unlink d4/d10/d72/d80/l9b 0 2024-04-27T04:10:07.502 INFO:tasks.workunit.client.0.smithi162.stdout:2/394: creat d0/d4/d2a/d41/d77/f7c x:0 0 0 2024-04-27T04:10:07.503 INFO:tasks.workunit.client.1.smithi181.stdout:3/567: truncate d0/f16 3442083 0 2024-04-27T04:10:07.505 INFO:tasks.workunit.client.1.smithi181.stdout:1/631: rename d1/dad/db5/d6/db/d2d/d36/c7c to d1/ccc 0 2024-04-27T04:10:07.506 INFO:tasks.workunit.client.0.smithi162.stdout:5/461: getdents d8/d1a/d29/d4a/d5c 0 2024-04-27T04:10:07.513 INFO:tasks.workunit.client.0.smithi162.stdout:5/462: dwrite d8/f6f [0,4194304] 0 2024-04-27T04:10:07.514 INFO:tasks.workunit.client.1.smithi181.stdout:1/632: fdatasync d1/dad/db5/d6/f3d 0 2024-04-27T04:10:07.515 INFO:tasks.workunit.client.0.smithi162.stdout:9/425: truncate d3/d23/f3e 4326 0 2024-04-27T04:10:07.516 INFO:tasks.workunit.client.0.smithi162.stdout:3/440: creat d1/d18/d23/d58/f8f x:0 0 0 2024-04-27T04:10:07.523 INFO:tasks.workunit.client.0.smithi162.stdout:5/463: dwrite d8/d1a/f51 [0,4194304] 0 2024-04-27T04:10:07.527 INFO:tasks.workunit.client.0.smithi162.stdout:5/464: chown d8/d1a/d23/d40/d4b/d66 2 1 2024-04-27T04:10:07.527 INFO:tasks.workunit.client.0.smithi162.stdout:3/441: dwrite d1/f3f [0,4194304] 0 2024-04-27T04:10:07.528 INFO:tasks.workunit.client.0.smithi162.stdout:9/426: dwrite d3/d13/d15/f18 [0,4194304] 0 2024-04-27T04:10:07.528 INFO:tasks.workunit.client.1.smithi181.stdout:1/633: getdents d1/dad/db5/d6/da3/d1e/d48/da6/da4 0 2024-04-27T04:10:07.533 INFO:tasks.workunit.client.1.smithi181.stdout:5/768: dwrite d4/de/d2c/d46/f60 [0,4194304] 0 2024-04-27T04:10:07.538 INFO:tasks.workunit.client.1.smithi181.stdout:3/568: dwrite d0/de/ff [0,4194304] 0 2024-04-27T04:10:07.538 INFO:tasks.workunit.client.0.smithi162.stdout:3/442: dwrite d1/d18/f41 [0,4194304] 0 2024-04-27T04:10:07.543 INFO:tasks.workunit.client.1.smithi181.stdout:1/634: creat d1/dad/db5/d6/da3/d52/fcd x:0 0 0 2024-04-27T04:10:07.544 INFO:tasks.workunit.client.0.smithi162.stdout:4/430: getdents d2/d25/d1c/d24 0 2024-04-27T04:10:07.549 INFO:tasks.workunit.client.0.smithi162.stdout:0/497: getdents d5/d26/d69 0 2024-04-27T04:10:07.554 INFO:tasks.workunit.client.0.smithi162.stdout:9/427: creat d3/d13/d2d/d3b/d54/f87 x:0 0 0 2024-04-27T04:10:07.554 INFO:tasks.workunit.client.0.smithi162.stdout:1/414: dwrite d0/d11/d44/d66/f6f [4194304,4194304] 0 2024-04-27T04:10:07.555 INFO:tasks.workunit.client.0.smithi162.stdout:9/428: fdatasync d3/f6 0 2024-04-27T04:10:07.555 INFO:tasks.workunit.client.0.smithi162.stdout:1/415: fsync d0/d7/d10/f4c 0 2024-04-27T04:10:07.556 INFO:tasks.workunit.client.1.smithi181.stdout:1/635: creat d1/d2/fce x:0 0 0 2024-04-27T04:10:07.556 INFO:tasks.workunit.client.0.smithi162.stdout:1/416: readlink d0/l3 0 2024-04-27T04:10:07.558 INFO:tasks.workunit.client.0.smithi162.stdout:2/395: mkdir d0/d4/d2a/d3b/d49/d5a/d63/d6c/d7d 0 2024-04-27T04:10:07.563 INFO:tasks.workunit.client.0.smithi162.stdout:4/431: rename d2/d25/d1c/d24/f56 to d2/d25/f8c 0 2024-04-27T04:10:07.565 INFO:tasks.workunit.client.0.smithi162.stdout:9/429: dwrite d3/d13/d15/f2f [4194304,4194304] 0 2024-04-27T04:10:07.567 INFO:tasks.workunit.client.0.smithi162.stdout:1/417: dwrite d0/d11/d3e/f41 [0,4194304] 0 2024-04-27T04:10:07.568 INFO:tasks.workunit.client.1.smithi181.stdout:3/569: getdents d0/de/d7d 0 2024-04-27T04:10:07.570 INFO:tasks.workunit.client.1.smithi181.stdout:1/636: link d1/dad/db5/d6/da3/d1e/f79 d1/dad/db5/d6/db/d2d/d36/dc4/fcf 0 2024-04-27T04:10:07.572 INFO:tasks.workunit.client.1.smithi181.stdout:3/570: mknod d0/d6c/d83/cb0 0 2024-04-27T04:10:07.577 INFO:tasks.workunit.client.0.smithi162.stdout:9/430: dwrite d3/d13/d2d/d3b/d54/f87 [0,4194304] 0 2024-04-27T04:10:07.578 INFO:tasks.workunit.client.0.smithi162.stdout:3/443: rename d1/d37/l3e to d1/d18/d5d/l90 0 2024-04-27T04:10:07.578 INFO:tasks.workunit.client.0.smithi162.stdout:9/431: dread - d3/d13/d2d/d3b/d69/f7d zero size 2024-04-27T04:10:07.580 INFO:tasks.workunit.client.0.smithi162.stdout:3/444: read - d1/d18/d23/d58/d63/f74 zero size 2024-04-27T04:10:07.581 INFO:tasks.workunit.client.1.smithi181.stdout:1/637: truncate d1/dad/db5/d6/da3/d1e/f9f 209846 0 2024-04-27T04:10:07.583 INFO:tasks.workunit.client.0.smithi162.stdout:8/456: sync 2024-04-27T04:10:07.583 INFO:tasks.workunit.client.0.smithi162.stdout:6/455: sync 2024-04-27T04:10:07.583 INFO:tasks.workunit.client.0.smithi162.stdout:7/391: sync 2024-04-27T04:10:07.584 INFO:tasks.workunit.client.0.smithi162.stdout:8/457: chown d0/d10/d2b/d2d/d2f/d39 5053 1 2024-04-27T04:10:07.585 INFO:tasks.workunit.client.1.smithi181.stdout:1/638: mknod d1/dad/db5/d6/da3/d1e/d48/db4/cd0 0 2024-04-27T04:10:07.586 INFO:tasks.workunit.client.0.smithi162.stdout:4/432: dread d2/da/d1a/f1b [0,4194304] 0 2024-04-27T04:10:07.587 INFO:tasks.workunit.client.0.smithi162.stdout:5/465: link d8/d1a/d23/d40/d4b/l8e d8/d9/d55/l9a 0 2024-04-27T04:10:07.587 INFO:tasks.workunit.client.0.smithi162.stdout:9/432: dread d3/d13/d15/f18 [0,4194304] 0 2024-04-27T04:10:07.590 INFO:tasks.workunit.client.0.smithi162.stdout:5/466: stat d8/d1a/d23/d40/c4e 0 2024-04-27T04:10:07.590 INFO:tasks.workunit.client.0.smithi162.stdout:6/456: chown d6/d1a/d34/d7b/d7e/l8a 21506 1 2024-04-27T04:10:07.590 INFO:tasks.workunit.client.0.smithi162.stdout:8/458: mknod d0/d10/d2b/c9f 0 2024-04-27T04:10:07.590 INFO:tasks.workunit.client.1.smithi181.stdout:0/744: sync 2024-04-27T04:10:07.591 INFO:tasks.workunit.client.0.smithi162.stdout:5/467: chown d8/d86 44999843 1 2024-04-27T04:10:07.593 INFO:tasks.workunit.client.1.smithi181.stdout:1/639: dwrite d1/dad/db5/d6/da3/d1e/fc5 [0,4194304] 0 2024-04-27T04:10:07.595 INFO:tasks.workunit.client.0.smithi162.stdout:7/392: rename d3/d39/d57/c3d to d3/d5/d5b/d60/d65/c85 0 2024-04-27T04:10:07.596 INFO:tasks.workunit.client.0.smithi162.stdout:7/393: readlink d3/l22 0 2024-04-27T04:10:07.598 INFO:tasks.workunit.client.0.smithi162.stdout:1/418: creat d0/d80/f87 x:0 0 0 2024-04-27T04:10:07.598 INFO:tasks.workunit.client.0.smithi162.stdout:5/468: dread d8/d1a/f2d [0,4194304] 0 2024-04-27T04:10:07.598 INFO:tasks.workunit.client.1.smithi181.stdout:0/745: truncate dc/d1f/f23 4000757 0 2024-04-27T04:10:07.600 INFO:tasks.workunit.client.1.smithi181.stdout:1/640: truncate d1/d2/f13 2996582 0 2024-04-27T04:10:07.602 INFO:tasks.workunit.client.0.smithi162.stdout:3/445: creat d1/d65/f91 x:0 0 0 2024-04-27T04:10:07.604 INFO:tasks.workunit.client.1.smithi181.stdout:1/641: symlink d1/dad/db5/d6/db/ld1 0 2024-04-27T04:10:07.609 INFO:tasks.workunit.client.0.smithi162.stdout:8/459: unlink d0/de/d20/l27 0 2024-04-27T04:10:07.611 INFO:tasks.workunit.client.0.smithi162.stdout:2/396: dread d0/d24/f38 [0,4194304] 0 2024-04-27T04:10:07.612 INFO:tasks.workunit.client.1.smithi181.stdout:5/769: dwrite d4/de/d2c/f3e [0,4194304] 0 2024-04-27T04:10:07.614 INFO:tasks.workunit.client.0.smithi162.stdout:0/498: dread d5/dc/d1e/d41/f39 [0,4194304] 0 2024-04-27T04:10:07.617 INFO:tasks.workunit.client.1.smithi181.stdout:1/642: creat d1/dad/db5/d6/da3/d1e/d48/da6/da4/fd2 x:0 0 0 2024-04-27T04:10:07.617 INFO:tasks.workunit.client.0.smithi162.stdout:1/419: dread d0/d7/d2a/f30 [0,4194304] 0 2024-04-27T04:10:07.620 INFO:tasks.workunit.client.1.smithi181.stdout:5/770: mkdir d4/de/d2c/d46/d4d/d8e/d67/d71/d10f 0 2024-04-27T04:10:07.621 INFO:tasks.workunit.client.1.smithi181.stdout:3/571: dread d0/f19 [0,4194304] 0 2024-04-27T04:10:07.622 INFO:tasks.workunit.client.1.smithi181.stdout:3/572: write d0/d6c/d52/d44/fad [452043,71481] 0 2024-04-27T04:10:07.623 INFO:tasks.workunit.client.1.smithi181.stdout:1/643: symlink d1/dad/db5/d6/da3/d52/d92/ld3 0 2024-04-27T04:10:07.629 INFO:tasks.workunit.client.1.smithi181.stdout:1/644: fdatasync d1/d2/d64/f8d 0 2024-04-27T04:10:07.631 INFO:tasks.workunit.client.0.smithi162.stdout:4/433: creat d2/d25/d1c/d39/d70/f8d x:0 0 0 2024-04-27T04:10:07.631 INFO:tasks.workunit.client.1.smithi181.stdout:3/573: dwrite d0/de/d20/d58/d64/f6a [4194304,4194304] 0 2024-04-27T04:10:07.632 INFO:tasks.workunit.client.1.smithi181.stdout:3/574: read d0/f19 [1432412,71446] 0 2024-04-27T04:10:07.634 INFO:tasks.workunit.client.1.smithi181.stdout:3/575: chown d0/d6c/d83/cb0 243573068 1 2024-04-27T04:10:07.636 INFO:tasks.workunit.client.0.smithi162.stdout:8/460: write d0/d10/d25/f88 [1667896,127712] 0 2024-04-27T04:10:07.639 INFO:tasks.workunit.client.0.smithi162.stdout:6/457: dwrite d6/d1a/d34/f3e [0,4194304] 0 2024-04-27T04:10:07.640 INFO:tasks.workunit.client.1.smithi181.stdout:1/645: creat d1/dad/db5/d6/da3/d1e/d48/da6/fd4 x:0 0 0 2024-04-27T04:10:07.640 INFO:tasks.workunit.client.0.smithi162.stdout:0/499: mknod d5/d60/d8b/c9d 0 2024-04-27T04:10:07.647 INFO:tasks.workunit.client.1.smithi181.stdout:1/646: truncate d1/dad/db5/d6/da3/f3b 1581418 0 2024-04-27T04:10:07.649 INFO:tasks.workunit.client.0.smithi162.stdout:0/500: dwrite d5/dc/f91 [0,4194304] 0 2024-04-27T04:10:07.652 INFO:tasks.workunit.client.0.smithi162.stdout:1/420: dwrite d0/d7/d3c/f57 [4194304,4194304] 0 2024-04-27T04:10:07.654 INFO:tasks.workunit.client.1.smithi181.stdout:1/647: creat d1/dad/db5/d6/da3/d1e/d20/da2/fd5 x:0 0 0 2024-04-27T04:10:07.655 INFO:tasks.workunit.client.1.smithi181.stdout:0/746: dwrite dc/f39 [0,4194304] 0 2024-04-27T04:10:07.655 INFO:tasks.workunit.client.0.smithi162.stdout:9/433: dwrite d3/fb [0,4194304] 0 2024-04-27T04:10:07.656 INFO:tasks.workunit.client.1.smithi181.stdout:0/747: chown dc/d1f/d3b/d4a/d7a 882265 1 2024-04-27T04:10:07.659 INFO:tasks.workunit.client.1.smithi181.stdout:3/576: creat d0/d6c/fb1 x:0 0 0 2024-04-27T04:10:07.660 INFO:tasks.workunit.client.1.smithi181.stdout:3/577: write d0/f23 [4915020,10383] 0 2024-04-27T04:10:07.663 INFO:tasks.workunit.client.1.smithi181.stdout:3/578: chown d0/de/d3f/c53 10536 1 2024-04-27T04:10:07.669 INFO:tasks.workunit.client.1.smithi181.stdout:1/648: dwrite d1/dad/db5/d6/da3/d1e/f49 [4194304,4194304] 0 2024-04-27T04:10:07.669 INFO:tasks.workunit.client.0.smithi162.stdout:2/397: sync 2024-04-27T04:10:07.670 INFO:tasks.workunit.client.1.smithi181.stdout:1/649: readlink d1/dad/db5/l83 0 2024-04-27T04:10:07.673 INFO:tasks.workunit.client.0.smithi162.stdout:7/394: rename d3/f16 to d3/f86 0 2024-04-27T04:10:07.675 INFO:tasks.workunit.client.0.smithi162.stdout:3/446: dwrite d1/d18/d5d/d5f/f76 [4194304,4194304] 0 2024-04-27T04:10:07.675 INFO:tasks.workunit.client.0.smithi162.stdout:8/461: creat d0/d10/d2b/d2d/d2f/d42/fa0 x:0 0 0 2024-04-27T04:10:07.675 INFO:tasks.workunit.client.1.smithi181.stdout:5/771: dwrite d4/d10/d29/d92/fd6 [0,4194304] 0 2024-04-27T04:10:07.676 INFO:tasks.workunit.client.0.smithi162.stdout:3/447: fsync d1/d45/f87 0 2024-04-27T04:10:07.677 INFO:tasks.workunit.client.1.smithi181.stdout:5/772: write d4/de/d2c/d46/d4d/d8e/d67/fa2 [169443,66033] 0 2024-04-27T04:10:07.680 INFO:tasks.workunit.client.1.smithi181.stdout:0/748: mkdir dc/d1f/d3b/d4a/d95/d26/d107 0 2024-04-27T04:10:07.685 INFO:tasks.workunit.client.0.smithi162.stdout:6/458: unlink d6/d1a/f1d 0 2024-04-27T04:10:07.691 INFO:tasks.workunit.client.0.smithi162.stdout:5/469: link d8/d1a/d23/d40/c5d d8/d1a/d23/d40/d4b/d58/c9b 0 2024-04-27T04:10:07.692 INFO:tasks.workunit.client.1.smithi181.stdout:8/709: sync 2024-04-27T04:10:07.692 INFO:tasks.workunit.client.1.smithi181.stdout:7/622: sync 2024-04-27T04:10:07.692 INFO:tasks.workunit.client.1.smithi181.stdout:1/650: fsync d1/dad/db5/f45 0 2024-04-27T04:10:07.695 INFO:tasks.workunit.client.0.smithi162.stdout:4/434: dread d2/f4 [0,4194304] 0 2024-04-27T04:10:07.696 INFO:tasks.workunit.client.1.smithi181.stdout:5/773: dread - d4/de/d2c/d46/d4d/f8b zero size 2024-04-27T04:10:07.697 INFO:tasks.workunit.client.1.smithi181.stdout:0/749: fsync dc/d1f/d3b/d4a/fd1 0 2024-04-27T04:10:07.699 INFO:tasks.workunit.client.0.smithi162.stdout:2/398: unlink d0/d4/d2a/d3b/l55 0 2024-04-27T04:10:07.701 INFO:tasks.workunit.client.1.smithi181.stdout:3/579: fsync d0/de/d20/f62 0 2024-04-27T04:10:07.701 INFO:tasks.workunit.client.0.smithi162.stdout:1/421: dread d0/d7/d10/f3d [4194304,4194304] 0 2024-04-27T04:10:07.702 INFO:tasks.workunit.client.0.smithi162.stdout:7/395: mknod d3/d5/d5b/c87 0 2024-04-27T04:10:07.706 INFO:tasks.workunit.client.1.smithi181.stdout:7/623: creat d1/d2/fbd x:0 0 0 2024-04-27T04:10:07.707 INFO:tasks.workunit.client.1.smithi181.stdout:7/624: chown d1/df/d1a/d6a 189874 1 2024-04-27T04:10:07.713 INFO:tasks.workunit.client.1.smithi181.stdout:9/580: sync 2024-04-27T04:10:07.713 INFO:tasks.workunit.client.1.smithi181.stdout:6/523: sync 2024-04-27T04:10:07.713 INFO:tasks.workunit.client.1.smithi181.stdout:2/683: sync 2024-04-27T04:10:07.713 INFO:tasks.workunit.client.1.smithi181.stdout:4/605: sync 2024-04-27T04:10:07.714 INFO:tasks.workunit.client.1.smithi181.stdout:5/774: creat d4/de/d2c/d46/d4d/d8e/d54/f110 x:0 0 0 2024-04-27T04:10:07.714 INFO:tasks.workunit.client.0.smithi162.stdout:0/501: mkdir d5/dc/d1e/d46/d48/d9e 0 2024-04-27T04:10:07.716 INFO:tasks.workunit.client.1.smithi181.stdout:0/750: fdatasync dc/d1f/d3b/d4a/d95/d26/d41/fa8 0 2024-04-27T04:10:07.718 INFO:tasks.workunit.client.1.smithi181.stdout:7/625: dwrite d1/d2/d1c/d2d/d61/f97 [0,4194304] 0 2024-04-27T04:10:07.726 INFO:tasks.workunit.client.1.smithi181.stdout:8/710: creat d1/d6/d2f/d30/d9d/dea/db3/ddf/ff7 x:0 0 0 2024-04-27T04:10:07.728 INFO:tasks.workunit.client.0.smithi162.stdout:1/422: creat d0/d7/f88 x:0 0 0 2024-04-27T04:10:07.729 INFO:tasks.workunit.client.1.smithi181.stdout:8/711: dread d1/d6/d88/fb1 [0,4194304] 0 2024-04-27T04:10:07.729 INFO:tasks.workunit.client.0.smithi162.stdout:8/462: dread d0/d10/d2b/d2d/d2f/d39/f59 [0,4194304] 0 2024-04-27T04:10:07.734 INFO:tasks.workunit.client.1.smithi181.stdout:9/581: rmdir d0/d4d/da0 39 2024-04-27T04:10:07.737 INFO:tasks.workunit.client.0.smithi162.stdout:3/448: link d1/d18/d23/d34/f3c d1/d70/f92 0 2024-04-27T04:10:07.738 INFO:tasks.workunit.client.1.smithi181.stdout:2/684: fdatasync d4/f6 0 2024-04-27T04:10:07.741 INFO:tasks.workunit.client.1.smithi181.stdout:9/582: dwrite d0/d44/d4f/f7a [4194304,4194304] 0 2024-04-27T04:10:07.742 INFO:tasks.workunit.client.0.smithi162.stdout:5/470: mknod d8/d1a/d29/d8b/c9c 0 2024-04-27T04:10:07.742 INFO:tasks.workunit.client.0.smithi162.stdout:3/449: dwrite d1/f3f [0,4194304] 0 2024-04-27T04:10:07.743 INFO:tasks.workunit.client.1.smithi181.stdout:6/524: creat d7/dc/d1c/d88/d6d/fb5 x:0 0 0 2024-04-27T04:10:07.747 INFO:tasks.workunit.client.1.smithi181.stdout:4/606: creat d0/d15/de/d19/d8a/fc8 x:0 0 0 2024-04-27T04:10:07.747 INFO:tasks.workunit.client.1.smithi181.stdout:6/525: dread - d7/dc/d1c/d34/d3f/d7e/d8a/f8e zero size 2024-04-27T04:10:07.751 INFO:tasks.workunit.client.0.smithi162.stdout:1/423: mkdir d0/d80/d89 0 2024-04-27T04:10:07.752 INFO:tasks.workunit.client.1.smithi181.stdout:4/607: dread d0/d15/d5c/d69/f10 [0,4194304] 0 2024-04-27T04:10:07.754 INFO:tasks.workunit.client.1.smithi181.stdout:3/580: mknod d0/de/d20/d6b/d91/d9e/cb2 0 2024-04-27T04:10:07.760 INFO:tasks.workunit.client.1.smithi181.stdout:8/712: dread - d1/d6/d2f/fcf zero size 2024-04-27T04:10:07.763 INFO:tasks.workunit.client.0.smithi162.stdout:7/396: write d3/d5/d9/f26 [2003404,100700] 0 2024-04-27T04:10:07.765 INFO:tasks.workunit.client.0.smithi162.stdout:5/471: unlink d8/d9/d5b/f71 0 2024-04-27T04:10:07.766 INFO:tasks.workunit.client.0.smithi162.stdout:6/459: write d6/d1a/f2b [1543767,71286] 0 2024-04-27T04:10:07.767 INFO:tasks.workunit.client.0.smithi162.stdout:0/502: getdents d5/d60/d98 0 2024-04-27T04:10:07.767 INFO:tasks.workunit.client.1.smithi181.stdout:9/583: fdatasync d0/d44/f6e 0 2024-04-27T04:10:07.769 INFO:tasks.workunit.client.0.smithi162.stdout:7/397: read d3/d5/d5b/d83/f3f [963046,118548] 0 2024-04-27T04:10:07.771 INFO:tasks.workunit.client.0.smithi162.stdout:5/472: read d8/d1a/d29/d4a/d5c/d63/f73 [1372493,100347] 0 2024-04-27T04:10:07.771 INFO:tasks.workunit.client.0.smithi162.stdout:4/435: dwrite d2/da/dd/f60 [0,4194304] 0 2024-04-27T04:10:07.773 INFO:tasks.workunit.client.0.smithi162.stdout:3/450: creat d1/d21/d47/f93 x:0 0 0 2024-04-27T04:10:07.773 INFO:tasks.workunit.client.1.smithi181.stdout:6/526: mknod d7/dc/d8b/cb6 0 2024-04-27T04:10:07.774 INFO:tasks.workunit.client.0.smithi162.stdout:9/434: dwrite d3/d13/d15/f57 [0,4194304] 0 2024-04-27T04:10:07.775 INFO:tasks.workunit.client.1.smithi181.stdout:6/527: read d7/f26 [1633225,100255] 0 2024-04-27T04:10:07.776 INFO:tasks.workunit.client.0.smithi162.stdout:9/435: write d3/d13/d2d/d3b/f84 [1460141,2166] 0 2024-04-27T04:10:07.781 INFO:tasks.workunit.client.1.smithi181.stdout:5/775: write d4/de/d2f/d7a/f89 [811373,21510] 0 2024-04-27T04:10:07.785 INFO:tasks.workunit.client.0.smithi162.stdout:9/436: dwrite d3/d23/f22 [0,4194304] 0 2024-04-27T04:10:07.785 INFO:tasks.workunit.client.1.smithi181.stdout:4/608: symlink d0/d15/d25/d2a/da0/lc9 0 2024-04-27T04:10:07.786 INFO:tasks.workunit.client.0.smithi162.stdout:7/398: symlink d3/d5/d3c/d4a/d55/l88 0 2024-04-27T04:10:07.788 INFO:tasks.workunit.client.1.smithi181.stdout:3/581: unlink d0/f4e 0 2024-04-27T04:10:07.788 INFO:tasks.workunit.client.0.smithi162.stdout:7/399: write d3/d39/d57/f42 [206283,85168] 0 2024-04-27T04:10:07.790 INFO:tasks.workunit.client.0.smithi162.stdout:5/473: readlink d8/d9/d55/l9a 0 2024-04-27T04:10:07.791 INFO:tasks.workunit.client.1.smithi181.stdout:1/651: getdents d1/dad/db5/d6/da3/d1e/d20/da2 0 2024-04-27T04:10:07.794 INFO:tasks.workunit.client.0.smithi162.stdout:9/437: dwrite d3/d11/d45/d4e/d58/f68 [0,4194304] 0 2024-04-27T04:10:07.797 INFO:tasks.workunit.client.1.smithi181.stdout:7/626: dwrite d1/d22/d15/d5f/fb9 [0,4194304] 0 2024-04-27T04:10:07.801 INFO:tasks.workunit.client.0.smithi162.stdout:6/460: read d6/d1a/d34/f66 [1832600,25370] 0 2024-04-27T04:10:07.806 INFO:tasks.workunit.client.1.smithi181.stdout:6/528: creat d7/d31/d45/d72/fb7 x:0 0 0 2024-04-27T04:10:07.806 INFO:tasks.workunit.client.1.smithi181.stdout:6/529: chown d7/dc/d1e/d5c/f65 4561 1 2024-04-27T04:10:07.806 INFO:tasks.workunit.client.0.smithi162.stdout:9/438: dwrite d3/d11/d45/d4e/d58/f80 [0,4194304] 0 2024-04-27T04:10:07.807 INFO:tasks.workunit.client.0.smithi162.stdout:9/439: write d3/d11/d45/d4e/d58/f80 [2225997,24967] 0 2024-04-27T04:10:07.808 INFO:tasks.workunit.client.0.smithi162.stdout:7/400: creat d3/d5/d3c/d4d/f89 x:0 0 0 2024-04-27T04:10:07.809 INFO:tasks.workunit.client.0.smithi162.stdout:7/401: readlink d3/d5/d5b/d83/de/l34 0 2024-04-27T04:10:07.810 INFO:tasks.workunit.client.0.smithi162.stdout:2/399: sync 2024-04-27T04:10:07.810 INFO:tasks.workunit.client.0.smithi162.stdout:7/402: stat d3/d4f/l47 0 2024-04-27T04:10:07.812 INFO:tasks.workunit.client.0.smithi162.stdout:2/400: dread - d0/d4/d2a/d3b/f74 zero size 2024-04-27T04:10:07.813 INFO:tasks.workunit.client.0.smithi162.stdout:5/474: rename d8/f26 to d8/d1a/d23/d40/d4b/d78/f9d 0 2024-04-27T04:10:07.816 INFO:tasks.workunit.client.0.smithi162.stdout:3/451: link d1/d18/d23/d2f/f73 d1/d45/f94 0 2024-04-27T04:10:07.820 INFO:tasks.workunit.client.0.smithi162.stdout:6/461: symlink d6/db/l95 0 2024-04-27T04:10:07.821 INFO:tasks.workunit.client.1.smithi181.stdout:2/685: stat d4/d9/d4f/d79/cad 0 2024-04-27T04:10:07.822 INFO:tasks.workunit.client.0.smithi162.stdout:0/503: creat d5/dc/d1e/d46/f9f x:0 0 0 2024-04-27T04:10:07.825 INFO:tasks.workunit.client.1.smithi181.stdout:7/627: rmdir d1/d22 39 2024-04-27T04:10:07.829 INFO:tasks.workunit.client.1.smithi181.stdout:9/584: truncate d0/f2e 5860747 0 2024-04-27T04:10:07.831 INFO:tasks.workunit.client.1.smithi181.stdout:5/776: mkdir d4/de/d2c/d46/d111 0 2024-04-27T04:10:07.832 INFO:tasks.workunit.client.0.smithi162.stdout:9/440: dwrite d3/d13/d15/f18 [0,4194304] 0 2024-04-27T04:10:07.833 INFO:tasks.workunit.client.1.smithi181.stdout:8/713: dwrite d1/d6/d1a/d25/fc7 [0,4194304] 0 2024-04-27T04:10:07.836 INFO:tasks.workunit.client.1.smithi181.stdout:7/628: dwrite d1/d2/d1c/d2d/d61/f97 [0,4194304] 0 2024-04-27T04:10:07.836 INFO:tasks.workunit.client.1.smithi181.stdout:0/751: getdents dc/d1f/d3b/d4a/d95/d26/d68 0 2024-04-27T04:10:07.839 INFO:tasks.workunit.client.1.smithi181.stdout:6/530: truncate d7/dc/d1c/d34/d3f/d7e/d8a/f7a 4458141 0 2024-04-27T04:10:07.842 INFO:tasks.workunit.client.0.smithi162.stdout:7/403: dread d3/d5/d5b/d83/de/d2b/f69 [0,4194304] 0 2024-04-27T04:10:07.842 INFO:tasks.workunit.client.0.smithi162.stdout:2/401: mkdir d0/d4/d2a/d3b/d49/d5a/d7e 0 2024-04-27T04:10:07.844 INFO:tasks.workunit.client.0.smithi162.stdout:2/402: chown d0/d4/d2a/d41 3196 1 2024-04-27T04:10:07.845 INFO:tasks.workunit.client.1.smithi181.stdout:4/609: mkdir d0/d15/d25/d2a/d36/db8/dca 0 2024-04-27T04:10:07.847 INFO:tasks.workunit.client.0.smithi162.stdout:5/475: creat d8/d1a/d29/d4a/d5c/f9e x:0 0 0 2024-04-27T04:10:07.849 INFO:tasks.workunit.client.0.smithi162.stdout:7/404: dwrite d3/d39/d57/f43 [4194304,4194304] 0 2024-04-27T04:10:07.850 INFO:tasks.workunit.client.0.smithi162.stdout:3/452: symlink d1/d18/d5d/d6f/l95 0 2024-04-27T04:10:07.850 INFO:tasks.workunit.client.1.smithi181.stdout:1/652: symlink d1/dad/db5/d6/dcb/ld6 0 2024-04-27T04:10:07.856 INFO:tasks.workunit.client.0.smithi162.stdout:0/504: write d5/dc/f18 [3073803,22410] 0 2024-04-27T04:10:07.859 INFO:tasks.workunit.client.0.smithi162.stdout:4/436: getdents d2/d54/d7a 0 2024-04-27T04:10:07.860 INFO:tasks.workunit.client.0.smithi162.stdout:4/437: chown d2/da/d1a/f30 23 1 2024-04-27T04:10:07.862 INFO:tasks.workunit.client.0.smithi162.stdout:9/441: unlink d3/d39/c77 0 2024-04-27T04:10:07.862 INFO:tasks.workunit.client.1.smithi181.stdout:5/777: rmdir d4/d10 39 2024-04-27T04:10:07.865 INFO:tasks.workunit.client.1.smithi181.stdout:8/714: creat d1/d6/da6/ff8 x:0 0 0 2024-04-27T04:10:07.866 INFO:tasks.workunit.client.1.smithi181.stdout:8/715: write d1/d20/f51 [3020664,82937] 0 2024-04-27T04:10:07.868 INFO:tasks.workunit.client.0.smithi162.stdout:9/442: dwrite d3/d13/d2d/d3b/d54/f87 [0,4194304] 0 2024-04-27T04:10:07.869 INFO:tasks.workunit.client.1.smithi181.stdout:9/585: truncate d0/d44/d4b/d5f/f77 143571 0 2024-04-27T04:10:07.870 INFO:tasks.workunit.client.1.smithi181.stdout:9/586: write d0/d44/d4b/f80 [1915509,43106] 0 2024-04-27T04:10:07.871 INFO:tasks.workunit.client.1.smithi181.stdout:7/629: stat d1/d22/d15/d5f/d93 0 2024-04-27T04:10:07.875 INFO:tasks.workunit.client.0.smithi162.stdout:5/476: rename d8/d1a/d29/d4a/d5c/d63 to d8/d1a/d29/d4a/d9f 0 2024-04-27T04:10:07.878 INFO:tasks.workunit.client.0.smithi162.stdout:9/443: dwrite d3/d13/f26 [0,4194304] 0 2024-04-27T04:10:07.878 INFO:tasks.workunit.client.1.smithi181.stdout:7/630: dwrite d1/d2/f27 [4194304,4194304] 0 2024-04-27T04:10:07.879 INFO:tasks.workunit.client.1.smithi181.stdout:0/752: creat dc/d1f/d3b/d4a/d7a/f108 x:0 0 0 2024-04-27T04:10:07.880 INFO:tasks.workunit.client.0.smithi162.stdout:7/405: dread d3/d5/d9/f13 [0,4194304] 0 2024-04-27T04:10:07.881 INFO:tasks.workunit.client.0.smithi162.stdout:7/406: stat d3/d5/d5b/d83/de/f74 0 2024-04-27T04:10:07.884 INFO:tasks.workunit.client.0.smithi162.stdout:3/453: mknod d1/d18/d23/d58/c96 0 2024-04-27T04:10:07.890 INFO:tasks.workunit.client.1.smithi181.stdout:3/582: rename d0/de/f34 to d0/de/fb3 0 2024-04-27T04:10:07.891 INFO:tasks.workunit.client.0.smithi162.stdout:0/505: fdatasync d5/f58 0 2024-04-27T04:10:07.895 INFO:tasks.workunit.client.0.smithi162.stdout:8/463: sync 2024-04-27T04:10:07.895 INFO:tasks.workunit.client.0.smithi162.stdout:1/424: sync 2024-04-27T04:10:07.896 INFO:tasks.workunit.client.0.smithi162.stdout:1/425: readlink d0/d11/d50/l52 0 2024-04-27T04:10:07.900 INFO:tasks.workunit.client.0.smithi162.stdout:6/462: write d6/d1a/d34/f54 [316377,29615] 0 2024-04-27T04:10:07.901 INFO:tasks.workunit.client.0.smithi162.stdout:6/463: write d6/d1a/d34/d3a/d68/d6c/f82 [1045744,59177] 0 2024-04-27T04:10:07.903 INFO:tasks.workunit.client.1.smithi181.stdout:4/610: dwrite d0/d15/d25/d2b/f1c [4194304,4194304] 0 2024-04-27T04:10:07.904 INFO:tasks.workunit.client.0.smithi162.stdout:6/464: read d6/d1a/d34/d3a/d68/d6c/f82 [831444,42651] 0 2024-04-27T04:10:07.905 INFO:tasks.workunit.client.0.smithi162.stdout:6/465: write d6/d1a/d34/d3a/d68/d6c/d67/f7c [401811,10820] 0 2024-04-27T04:10:07.907 INFO:tasks.workunit.client.1.smithi181.stdout:9/587: write d0/f55 [3866644,84560] 0 2024-04-27T04:10:07.911 INFO:tasks.workunit.client.0.smithi162.stdout:9/444: dwrite d3/d13/d2d/d3b/f47 [0,4194304] 0 2024-04-27T04:10:07.913 INFO:tasks.workunit.client.1.smithi181.stdout:7/631: rename d1/d22/d15/d13 to d1/d22/d15/d29/dbe 0 2024-04-27T04:10:07.916 INFO:tasks.workunit.client.0.smithi162.stdout:3/454: rmdir d1/d21 39 2024-04-27T04:10:07.921 INFO:tasks.workunit.client.1.smithi181.stdout:5/778: write d4/de/d2c/d46/d99/dad/fe4 [567246,74276] 0 2024-04-27T04:10:07.922 INFO:tasks.workunit.client.0.smithi162.stdout:2/403: truncate d0/f6 1620719 0 2024-04-27T04:10:07.924 INFO:tasks.workunit.client.0.smithi162.stdout:8/464: fsync d0/d10/d25/f14 0 2024-04-27T04:10:07.926 INFO:tasks.workunit.client.1.smithi181.stdout:3/583: creat d0/d6c/d83/fb4 x:0 0 0 2024-04-27T04:10:07.928 INFO:tasks.workunit.client.1.smithi181.stdout:1/653: mknod d1/dad/db5/d6/db/d2d/cd7 0 2024-04-27T04:10:07.929 INFO:tasks.workunit.client.0.smithi162.stdout:1/426: rename d0/d7/d3c/d54/l70 to d0/d7/d3c/d54/l8a 0 2024-04-27T04:10:07.930 INFO:tasks.workunit.client.1.smithi181.stdout:2/686: creat d4/d9/d16/d45/d9a/fe1 x:0 0 0 2024-04-27T04:10:07.931 INFO:tasks.workunit.client.1.smithi181.stdout:2/687: chown d4/d9/df/c5e 1445458855 1 2024-04-27T04:10:07.931 INFO:tasks.workunit.client.0.smithi162.stdout:5/477: getdents d8/d1a/d23/d40/d7b 0 2024-04-27T04:10:07.932 INFO:tasks.workunit.client.0.smithi162.stdout:5/478: chown d8/d2c/f4c 670754 1 2024-04-27T04:10:07.932 INFO:tasks.workunit.client.1.smithi181.stdout:2/688: read - d4/d9/d16/d45/fdc zero size 2024-04-27T04:10:07.932 INFO:tasks.workunit.client.1.smithi181.stdout:4/611: mknod d0/d15/d25/d2a/d36/ccb 0 2024-04-27T04:10:07.935 INFO:tasks.workunit.client.1.smithi181.stdout:7/632: rename d1/d2/d1c/d5e to d1/d2/d35/d67/dbf 0 2024-04-27T04:10:07.935 INFO:tasks.workunit.client.1.smithi181.stdout:2/689: dread d4/d9/d4f/d79/d94/fd7 [0,4194304] 0 2024-04-27T04:10:07.935 INFO:tasks.workunit.client.0.smithi162.stdout:7/407: mknod d3/d5/d9/c8a 0 2024-04-27T04:10:07.936 INFO:tasks.workunit.client.0.smithi162.stdout:7/408: chown d3/d5/d3c/d4a/d55/c5a 22 1 2024-04-27T04:10:07.936 INFO:tasks.workunit.client.0.smithi162.stdout:9/445: creat d3/d23/f88 x:0 0 0 2024-04-27T04:10:07.937 INFO:tasks.workunit.client.0.smithi162.stdout:7/409: write d3/f1e [332256,43519] 0 2024-04-27T04:10:07.937 INFO:tasks.workunit.client.1.smithi181.stdout:6/531: link d7/dc/d1c/d7f/d3e/ca1 d7/dc/d1c/d7f/da3/cb8 0 2024-04-27T04:10:07.939 INFO:tasks.workunit.client.1.smithi181.stdout:5/779: truncate d4/de/d2c/d46/d4d/f94 103319 0 2024-04-27T04:10:07.939 INFO:tasks.workunit.client.0.smithi162.stdout:2/404: mkdir d0/d4/d17/d4f/d7f 0 2024-04-27T04:10:07.941 INFO:tasks.workunit.client.0.smithi162.stdout:0/506: symlink d5/la0 0 2024-04-27T04:10:07.941 INFO:tasks.workunit.client.1.smithi181.stdout:1/654: mknod d1/dad/cd8 0 2024-04-27T04:10:07.943 INFO:tasks.workunit.client.0.smithi162.stdout:8/465: write d0/d10/f6a [4080171,86740] 0 2024-04-27T04:10:07.946 INFO:tasks.workunit.client.1.smithi181.stdout:2/690: truncate d4/d9/d35/f60 915339 0 2024-04-27T04:10:07.947 INFO:tasks.workunit.client.1.smithi181.stdout:2/691: write d4/d9/d4f/d79/d98/fcc [101211,91419] 0 2024-04-27T04:10:07.948 INFO:tasks.workunit.client.0.smithi162.stdout:1/427: read d0/d7/d3c/d54/f65 [66291,25784] 0 2024-04-27T04:10:07.949 INFO:tasks.workunit.client.0.smithi162.stdout:1/428: readlink d0/l4 0 2024-04-27T04:10:07.951 INFO:tasks.workunit.client.1.smithi181.stdout:0/753: rmdir dc/d1f/d6b/db9/dba/deb/db3/dff 0 2024-04-27T04:10:07.955 INFO:tasks.workunit.client.0.smithi162.stdout:5/479: symlink d8/d1a/d23/d40/d4b/d66/d88/la0 0 2024-04-27T04:10:07.959 INFO:tasks.workunit.client.0.smithi162.stdout:5/480: dread d8/f6f [0,4194304] 0 2024-04-27T04:10:07.959 INFO:tasks.workunit.client.0.smithi162.stdout:5/481: chown d8/d1a/f3e 1 1 2024-04-27T04:10:07.963 INFO:tasks.workunit.client.0.smithi162.stdout:9/446: unlink d3/d13/d2d/d3b/c6a 0 2024-04-27T04:10:07.964 INFO:tasks.workunit.client.1.smithi181.stdout:4/612: getdents d0/d15/de/d19/d77/dc5 0 2024-04-27T04:10:07.967 INFO:tasks.workunit.client.0.smithi162.stdout:2/405: rename d0/d20/d51/d6f to d0/d4/d2a/d3b/d49/d5a/d63/d80 0 2024-04-27T04:10:07.968 INFO:tasks.workunit.client.0.smithi162.stdout:9/447: dwrite d3/f5 [4194304,4194304] 0 2024-04-27T04:10:07.969 INFO:tasks.workunit.client.0.smithi162.stdout:0/507: write d5/dc/d1e/f6d [7722959,2023] 0 2024-04-27T04:10:07.973 INFO:tasks.workunit.client.0.smithi162.stdout:8/466: mknod d0/d10/d44/d70/ca1 0 2024-04-27T04:10:07.976 INFO:tasks.workunit.client.1.smithi181.stdout:6/532: link d7/d31/f8c d7/fb9 0 2024-04-27T04:10:07.976 INFO:tasks.workunit.client.1.smithi181.stdout:2/692: dwrite d4/f50 [0,4194304] 0 2024-04-27T04:10:07.980 INFO:tasks.workunit.client.1.smithi181.stdout:7/633: dwrite d1/d22/f26 [0,4194304] 0 2024-04-27T04:10:07.994 INFO:tasks.workunit.client.1.smithi181.stdout:3/584: dwrite d0/d6c/d52/f65 [0,4194304] 0 2024-04-27T04:10:07.996 INFO:tasks.workunit.client.0.smithi162.stdout:5/482: mkdir d8/d9/da1 0 2024-04-27T04:10:08.002 INFO:tasks.workunit.client.0.smithi162.stdout:3/455: creat d1/d18/d23/f97 x:0 0 0 2024-04-27T04:10:08.003 INFO:tasks.workunit.client.1.smithi181.stdout:6/533: symlink d7/d31/d45/lba 0 2024-04-27T04:10:08.005 INFO:tasks.workunit.client.0.smithi162.stdout:7/410: mknod d3/d5/d9/c8b 0 2024-04-27T04:10:08.006 INFO:tasks.workunit.client.0.smithi162.stdout:7/411: fsync d3/d5/d5b/d83/de/f62 0 2024-04-27T04:10:08.031 INFO:tasks.workunit.client.1.smithi181.stdout:1/655: write d1/dad/db5/d6/f73 [5008052,73555] 0 2024-04-27T04:10:08.032 INFO:tasks.workunit.client.1.smithi181.stdout:5/780: creat d4/f112 x:0 0 0 2024-04-27T04:10:08.033 INFO:tasks.workunit.client.1.smithi181.stdout:1/656: fsync d1/dad/db5/d6/da3/d1e/d20/da2/fd5 0 2024-04-27T04:10:08.034 INFO:tasks.workunit.client.1.smithi181.stdout:7/634: creat d1/d22/fc0 x:0 0 0 2024-04-27T04:10:08.035 INFO:tasks.workunit.client.0.smithi162.stdout:0/508: write f1 [352627,102867] 0 2024-04-27T04:10:08.035 INFO:tasks.workunit.client.1.smithi181.stdout:7/635: chown d1/d22/d15/f28 6370180 1 2024-04-27T04:10:08.036 INFO:tasks.workunit.client.0.smithi162.stdout:4/438: link d2/c21 d2/d25/d1c/d39/d3b/d6e/c8e 0 2024-04-27T04:10:08.041 INFO:tasks.workunit.client.0.smithi162.stdout:0/509: dwrite d5/dc/d1e/d56/f90 [0,4194304] 0 2024-04-27T04:10:08.041 INFO:tasks.workunit.client.1.smithi181.stdout:3/585: creat d0/d6c/d83/d98/fb5 x:0 0 0 2024-04-27T04:10:08.041 INFO:tasks.workunit.client.0.smithi162.stdout:0/510: write d5/dc/d1e/f21 [5882155,38255] 0 2024-04-27T04:10:08.044 INFO:tasks.workunit.client.1.smithi181.stdout:6/534: mknod d7/dc/d1c/d7f/cbb 0 2024-04-27T04:10:08.048 INFO:tasks.workunit.client.1.smithi181.stdout:4/613: write d0/d15/f1a [1559852,5719] 0 2024-04-27T04:10:08.049 INFO:tasks.workunit.client.1.smithi181.stdout:4/614: stat d0/d15/de/d19/d77/dc5 0 2024-04-27T04:10:08.050 INFO:tasks.workunit.client.1.smithi181.stdout:4/615: dread - d0/d15/de/d19/d8a/fa9 zero size 2024-04-27T04:10:08.051 INFO:tasks.workunit.client.0.smithi162.stdout:7/412: rmdir d3/d5/d5b/d60 39 2024-04-27T04:10:08.051 INFO:tasks.workunit.client.0.smithi162.stdout:7/413: chown d3/d39/f52 1749 1 2024-04-27T04:10:08.052 INFO:tasks.workunit.client.1.smithi181.stdout:4/616: write d0/d15/de/d19/d8a/fa9 [810024,95157] 0 2024-04-27T04:10:08.057 INFO:tasks.workunit.client.0.smithi162.stdout:8/467: mknod d0/ca2 0 2024-04-27T04:10:08.057 INFO:tasks.workunit.client.1.smithi181.stdout:3/586: mknod d0/d28/cb6 0 2024-04-27T04:10:08.059 INFO:tasks.workunit.client.1.smithi181.stdout:0/754: getdents dc/d1f/d3b/d4a/d95/d26/d41/d6f/d7d 0 2024-04-27T04:10:08.061 INFO:tasks.workunit.client.1.smithi181.stdout:0/755: write dc/d1f/d3b/d4a/d95/fe6 [72885,98731] 0 2024-04-27T04:10:08.062 INFO:tasks.workunit.client.1.smithi181.stdout:0/756: fdatasync dc/d1f/d3b/d4a/d95/f36 0 2024-04-27T04:10:08.063 INFO:tasks.workunit.client.1.smithi181.stdout:2/693: rename d4/d9/d16/d45/f66 to d4/d9/d4f/d5f/fe2 0 2024-04-27T04:10:08.063 INFO:tasks.workunit.client.0.smithi162.stdout:1/429: link d0/d11/d50/d81/c86 d0/d11/d44/d66/c8b 0 2024-04-27T04:10:08.063 INFO:tasks.workunit.client.1.smithi181.stdout:3/587: dread d0/d6c/d52/f65 [0,4194304] 0 2024-04-27T04:10:08.068 INFO:tasks.workunit.client.0.smithi162.stdout:6/466: link d6/d1a/d34/d43/l69 d6/d1a/d34/d3a/d60/l96 0 2024-04-27T04:10:08.068 INFO:tasks.workunit.client.0.smithi162.stdout:6/467: chown d6/d1a/d34/d50/c73 681942345 1 2024-04-27T04:10:08.069 INFO:tasks.workunit.client.0.smithi162.stdout:6/468: write d6/d1a/d34/d50/f63 [997184,100296] 0 2024-04-27T04:10:08.071 INFO:tasks.workunit.client.0.smithi162.stdout:5/483: write d8/d9/d5b/f79 [931669,115012] 0 2024-04-27T04:10:08.072 INFO:tasks.workunit.client.0.smithi162.stdout:3/456: symlink d1/d21/d47/l98 0 2024-04-27T04:10:08.076 INFO:tasks.workunit.client.1.smithi181.stdout:7/636: dwrite d1/d2/d1c/f1e [0,4194304] 0 2024-04-27T04:10:08.079 INFO:tasks.workunit.client.1.smithi181.stdout:7/637: write d1/d2/d1c/d55/f66 [1184731,121987] 0 2024-04-27T04:10:08.086 INFO:tasks.workunit.client.1.smithi181.stdout:3/588: symlink d0/de/d3f/lb7 0 2024-04-27T04:10:08.089 INFO:tasks.workunit.client.1.smithi181.stdout:2/694: symlink d4/d9/d16/d45/d9a/d9d/le3 0 2024-04-27T04:10:08.097 INFO:tasks.workunit.client.1.smithi181.stdout:3/589: creat d0/d6c/d83/fb8 x:0 0 0 2024-04-27T04:10:08.098 INFO:tasks.workunit.client.1.smithi181.stdout:1/657: mkdir d1/dad/db5/d6/da3/d1e/d20/d53/d56/dd9 0 2024-04-27T04:10:08.100 INFO:tasks.workunit.client.1.smithi181.stdout:5/781: dwrite d4/d10/d72/f77 [0,4194304] 0 2024-04-27T04:10:08.100 INFO:tasks.workunit.client.1.smithi181.stdout:1/658: chown d1/dad/db5/l83 68211 1 2024-04-27T04:10:08.102 INFO:tasks.workunit.client.1.smithi181.stdout:6/535: rename d7/dc/d1c/d7f/l69 to d7/dc/lbc 0 2024-04-27T04:10:08.104 INFO:tasks.workunit.client.1.smithi181.stdout:0/757: creat dc/d1f/d6b/db9/dba/deb/f109 x:0 0 0 2024-04-27T04:10:08.105 INFO:tasks.workunit.client.1.smithi181.stdout:0/758: chown dc/d1f/d6b/d7e/c84 176813643 1 2024-04-27T04:10:08.105 INFO:tasks.workunit.client.0.smithi162.stdout:4/439: write d2/d54/f35 [487331,98740] 0 2024-04-27T04:10:08.106 INFO:tasks.workunit.client.0.smithi162.stdout:0/511: mkdir d5/d26/d69/d9b/da1 0 2024-04-27T04:10:08.106 INFO:tasks.workunit.client.1.smithi181.stdout:0/759: fdatasync dc/f15 0 2024-04-27T04:10:08.107 INFO:tasks.workunit.client.0.smithi162.stdout:5/484: mknod d8/d1a/d29/d4a/ca2 0 2024-04-27T04:10:08.108 INFO:tasks.workunit.client.1.smithi181.stdout:3/590: creat d0/d6c/da1/daf/fb9 x:0 0 0 2024-04-27T04:10:08.108 INFO:tasks.workunit.client.1.smithi181.stdout:5/782: dread - d4/d10/d29/d92/fbf zero size 2024-04-27T04:10:08.109 INFO:tasks.workunit.client.0.smithi162.stdout:2/406: dwrite d0/d4/d2a/f36 [4194304,4194304] 0 2024-04-27T04:10:08.110 INFO:tasks.workunit.client.1.smithi181.stdout:5/783: chown d4/de/d2f/d7a/c88 493690937 1 2024-04-27T04:10:08.110 INFO:tasks.workunit.client.0.smithi162.stdout:3/457: chown d1/d18/l8 79900301 1 2024-04-27T04:10:08.114 INFO:tasks.workunit.client.1.smithi181.stdout:4/617: rename d0/d15/f23 to d0/d15/d25/d2b/d6d/d82/fcc 0 2024-04-27T04:10:08.121 INFO:tasks.workunit.client.1.smithi181.stdout:3/591: mknod d0/d6c/da1/cba 0 2024-04-27T04:10:08.122 INFO:tasks.workunit.client.1.smithi181.stdout:2/695: truncate d4/d9/d16/f29 8119366 0 2024-04-27T04:10:08.125 INFO:tasks.workunit.client.1.smithi181.stdout:0/760: rename dc/d93/ce8 to dc/d1f/d3b/df7/c10a 0 2024-04-27T04:10:08.126 INFO:tasks.workunit.client.1.smithi181.stdout:1/659: getdents d1/dad/db5/d6/da3/d1e/d20/d53/d56/dd9 0 2024-04-27T04:10:08.127 INFO:tasks.workunit.client.0.smithi162.stdout:9/448: link d3/d13/d15/d50/c65 d3/d11/d45/d4e/c89 0 2024-04-27T04:10:08.127 INFO:tasks.workunit.client.1.smithi181.stdout:7/638: getdents d1/d22/d15/d82 0 2024-04-27T04:10:08.128 INFO:tasks.workunit.client.0.smithi162.stdout:3/458: write d1/d65/f7f [754376,68217] 0 2024-04-27T04:10:08.130 INFO:tasks.workunit.client.1.smithi181.stdout:2/696: write d4/d9/fc0 [5095483,46913] 0 2024-04-27T04:10:08.130 INFO:tasks.workunit.client.0.smithi162.stdout:2/407: fdatasync d0/d4/d2a/d41/d77/f7c 0 2024-04-27T04:10:08.132 INFO:tasks.workunit.client.1.smithi181.stdout:3/592: read d0/d28/f42 [527336,88341] 0 2024-04-27T04:10:08.133 INFO:tasks.workunit.client.0.smithi162.stdout:4/440: dread - d2/d25/d1c/d39/d3b/d6e/d6f/f76 zero size 2024-04-27T04:10:08.134 INFO:tasks.workunit.client.0.smithi162.stdout:2/408: chown d0/d20/d2e 211473216 1 2024-04-27T04:10:08.135 INFO:tasks.workunit.client.0.smithi162.stdout:9/449: fdatasync d3/d13/d2d/d3b/d54/f87 0 2024-04-27T04:10:08.135 INFO:tasks.workunit.client.1.smithi181.stdout:1/660: dwrite d1/dad/db5/d6/da3/d1e/d20/da2/fd5 [0,4194304] 0 2024-04-27T04:10:08.137 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:10:07 smithi162 ceph-mon[26841]: Deploying cephadm binary to smithi162 2024-04-27T04:10:08.137 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:10:07 smithi162 ceph-mon[26841]: Deploying cephadm binary to smithi181 2024-04-27T04:10:08.137 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:10:07 smithi162 ceph-mon[26841]: [27/Apr/2024:04:10:06] ENGINE Bus STARTING 2024-04-27T04:10:08.137 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:10:07 smithi162 ceph-mon[26841]: pgmap v3: 65 pgs: 65 active+clean; 2.4 GiB data, 8.6 GiB used, 528 GiB / 536 GiB avail 2024-04-27T04:10:08.137 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:10:07 smithi162 ceph-mon[26841]: [27/Apr/2024:04:10:06] ENGINE Serving on https://172.21.15.181:7150 2024-04-27T04:10:08.137 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:10:07 smithi162 ceph-mon[26841]: [27/Apr/2024:04:10:06] ENGINE Serving on http://172.21.15.181:8765 2024-04-27T04:10:08.137 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:10:07 smithi162 ceph-mon[26841]: [27/Apr/2024:04:10:06] ENGINE Bus STARTED 2024-04-27T04:10:08.140 INFO:tasks.workunit.client.1.smithi181.stdout:1/661: chown d1/dad/db5/d6/db/d2d/f55 214 1 2024-04-27T04:10:08.141 INFO:tasks.workunit.client.0.smithi162.stdout:7/414: dwrite d3/f86 [0,4194304] 0 2024-04-27T04:10:08.141 INFO:tasks.workunit.client.0.smithi162.stdout:8/468: dwrite d0/de/d20/f60 [0,4194304] 0 2024-04-27T04:10:08.141 INFO:tasks.workunit.client.0.smithi162.stdout:6/469: getdents d6/d1a/d34/d7b 0 2024-04-27T04:10:08.143 INFO:tasks.workunit.client.0.smithi162.stdout:5/485: dwrite d8/f20 [0,4194304] 0 2024-04-27T04:10:08.147 INFO:tasks.workunit.client.1.smithi181.stdout:5/784: dwrite d4/d10/fc4 [0,4194304] 0 2024-04-27T04:10:08.155 INFO:tasks.workunit.client.0.smithi162.stdout:4/441: unlink d2/da/d1a/l4e 0 2024-04-27T04:10:08.155 INFO:tasks.workunit.client.0.smithi162.stdout:2/409: dwrite d0/d4/d2a/d41/f4a [0,4194304] 0 2024-04-27T04:10:08.156 INFO:tasks.workunit.client.0.smithi162.stdout:0/512: dwrite d5/dc/d1e/d41/f53 [0,4194304] 0 2024-04-27T04:10:08.157 INFO:tasks.workunit.client.0.smithi162.stdout:0/513: fsync d5/d26/f4d 0 2024-04-27T04:10:08.159 INFO:tasks.workunit.client.0.smithi162.stdout:0/514: write d5/dc/d1e/f6d [5671317,2003] 0 2024-04-27T04:10:08.160 INFO:tasks.workunit.client.1.smithi181.stdout:3/593: rename d0/d6c/d83 to d0/d6c/d52/dbb 0 2024-04-27T04:10:08.160 INFO:tasks.workunit.client.1.smithi181.stdout:6/536: dread d7/dc/d1c/d34/d3f/f9f [0,4194304] 0 2024-04-27T04:10:08.166 INFO:tasks.workunit.client.1.smithi181.stdout:4/618: creat d0/d15/d25/fcd x:0 0 0 2024-04-27T04:10:08.168 INFO:tasks.workunit.client.0.smithi162.stdout:9/450: symlink d3/d13/d2d/d44/l8a 0 2024-04-27T04:10:08.172 INFO:tasks.workunit.client.0.smithi162.stdout:9/451: dread d3/d13/d15/f57 [0,4194304] 0 2024-04-27T04:10:08.172 INFO:tasks.workunit.client.1.smithi181.stdout:8/716: sync 2024-04-27T04:10:08.173 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:10:07 smithi181 ceph-mon[33774]: Deploying cephadm binary to smithi162 2024-04-27T04:10:08.173 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:10:07 smithi181 ceph-mon[33774]: Deploying cephadm binary to smithi181 2024-04-27T04:10:08.173 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:10:07 smithi181 ceph-mon[33774]: [27/Apr/2024:04:10:06] ENGINE Bus STARTING 2024-04-27T04:10:08.173 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:10:07 smithi181 ceph-mon[33774]: pgmap v3: 65 pgs: 65 active+clean; 2.4 GiB data, 8.6 GiB used, 528 GiB / 536 GiB avail 2024-04-27T04:10:08.173 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:10:07 smithi181 ceph-mon[33774]: [27/Apr/2024:04:10:06] ENGINE Serving on https://172.21.15.181:7150 2024-04-27T04:10:08.173 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:10:07 smithi181 ceph-mon[33774]: [27/Apr/2024:04:10:06] ENGINE Serving on http://172.21.15.181:8765 2024-04-27T04:10:08.173 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:10:07 smithi181 ceph-mon[33774]: [27/Apr/2024:04:10:06] ENGINE Bus STARTED 2024-04-27T04:10:08.174 INFO:tasks.workunit.client.0.smithi162.stdout:6/470: rmdir d6 39 2024-04-27T04:10:08.175 INFO:tasks.workunit.client.1.smithi181.stdout:8/717: truncate d1/d6/f4e 2011054 0 2024-04-27T04:10:08.175 INFO:tasks.workunit.client.1.smithi181.stdout:4/619: dwrite d0/d15/de/f66 [0,4194304] 0 2024-04-27T04:10:08.176 INFO:tasks.workunit.client.1.smithi181.stdout:1/662: mkdir d1/dad/db5/d6/da3/d1e/d48/da6/da4/dda 0 2024-04-27T04:10:08.178 INFO:tasks.workunit.client.0.smithi162.stdout:8/469: write d0/de/f7b [1361556,30261] 0 2024-04-27T04:10:08.181 INFO:tasks.workunit.client.1.smithi181.stdout:1/663: write d1/dad/db5/d6/da3/d1e/d48/da6/da4/fd2 [364714,71635] 0 2024-04-27T04:10:08.182 INFO:tasks.workunit.client.1.smithi181.stdout:7/639: write d1/d22/d15/d29/dbe/d1b/f47 [883332,2536] 0 2024-04-27T04:10:08.188 INFO:tasks.workunit.client.1.smithi181.stdout:4/620: dwrite d0/d15/f1a [4194304,4194304] 0 2024-04-27T04:10:08.189 INFO:tasks.workunit.client.1.smithi181.stdout:2/697: write d4/d9/d4f/d79/fa3 [1220638,67762] 0 2024-04-27T04:10:08.189 INFO:tasks.workunit.client.0.smithi162.stdout:2/410: creat d0/d4/d2a/d41/f81 x:0 0 0 2024-04-27T04:10:08.194 INFO:tasks.workunit.client.1.smithi181.stdout:3/594: rmdir d0/d6c/d66/d88 39 2024-04-27T04:10:08.194 INFO:tasks.workunit.client.1.smithi181.stdout:0/761: dread dc/f94 [0,4194304] 0 2024-04-27T04:10:08.198 INFO:tasks.workunit.client.0.smithi162.stdout:9/452: read - d3/d23/d1e/f4a zero size 2024-04-27T04:10:08.200 INFO:tasks.workunit.client.0.smithi162.stdout:6/471: read - d6/d1a/d34/d7b/f83 zero size 2024-04-27T04:10:08.200 INFO:tasks.workunit.client.1.smithi181.stdout:8/718: mknod d1/d6/d88/cf9 0 2024-04-27T04:10:08.200 INFO:tasks.workunit.client.1.smithi181.stdout:1/664: creat d1/dad/fdb x:0 0 0 2024-04-27T04:10:08.201 INFO:tasks.workunit.client.1.smithi181.stdout:4/621: write d0/d15/de/d19/d77/d57/f84 [643840,42243] 0 2024-04-27T04:10:08.201 INFO:tasks.workunit.client.0.smithi162.stdout:8/470: symlink d0/d10/d2b/la3 0 2024-04-27T04:10:08.201 INFO:tasks.workunit.client.0.smithi162.stdout:8/471: chown d0/de/d6e/l90 17077 1 2024-04-27T04:10:08.202 INFO:tasks.workunit.client.1.smithi181.stdout:2/698: creat d4/d9/d35/d9f/dd0/fe4 x:0 0 0 2024-04-27T04:10:08.203 INFO:tasks.workunit.client.1.smithi181.stdout:8/719: stat d1/d6/d2f/d30/d9d/dea/lae 0 2024-04-27T04:10:08.204 INFO:tasks.workunit.client.1.smithi181.stdout:1/665: stat d1/dad/db5/d6/da3/d1e/d20/f5c 0 2024-04-27T04:10:08.205 INFO:tasks.workunit.client.1.smithi181.stdout:8/720: stat d1/d6/d2f/d30/d9d/dea/db3/ddf/ff7 0 2024-04-27T04:10:08.212 INFO:tasks.workunit.client.1.smithi181.stdout:9/588: sync 2024-04-27T04:10:08.213 INFO:tasks.workunit.client.1.smithi181.stdout:9/589: chown d0/d4d/c53 586714 1 2024-04-27T04:10:08.215 INFO:tasks.workunit.client.0.smithi162.stdout:6/472: mkdir d6/d1a/d34/d7b/d7e/d97 0 2024-04-27T04:10:08.216 INFO:tasks.workunit.client.0.smithi162.stdout:6/473: write d6/d1a/f2b [2418673,9776] 0 2024-04-27T04:10:08.217 INFO:tasks.workunit.client.0.smithi162.stdout:8/472: unlink d0/d10/d2b/c82 0 2024-04-27T04:10:08.218 INFO:tasks.workunit.client.0.smithi162.stdout:8/473: chown d0/d10/d2b/d2d/d2f/d39 426018089 1 2024-04-27T04:10:08.221 INFO:tasks.workunit.client.0.smithi162.stdout:3/459: getdents d1/d21/d47/d57/d6e 0 2024-04-27T04:10:08.222 INFO:tasks.workunit.client.0.smithi162.stdout:3/460: write d1/d18/f1a [9543087,116777] 0 2024-04-27T04:10:08.223 INFO:tasks.workunit.client.0.smithi162.stdout:7/415: link d3/d5/d9/c8a d3/d5/d3c/d4d/c8c 0 2024-04-27T04:10:08.224 INFO:tasks.workunit.client.0.smithi162.stdout:7/416: write d3/d5/d9/f7f [378263,123574] 0 2024-04-27T04:10:08.226 INFO:tasks.workunit.client.1.smithi181.stdout:3/595: dread d0/d6c/d52/d44/d74/f9b [0,4194304] 0 2024-04-27T04:10:08.226 INFO:tasks.workunit.client.0.smithi162.stdout:7/417: write d3/d5/d5b/d83/f6e [749211,26612] 0 2024-04-27T04:10:08.234 INFO:tasks.workunit.client.0.smithi162.stdout:4/442: truncate d2/da/f31 2018471 0 2024-04-27T04:10:08.236 INFO:tasks.workunit.client.0.smithi162.stdout:5/486: getdents d8/d1a/d29/d4a/d9f 0 2024-04-27T04:10:08.237 INFO:tasks.workunit.client.0.smithi162.stdout:6/474: dread d6/d1a/f15 [0,4194304] 0 2024-04-27T04:10:08.237 INFO:tasks.workunit.client.0.smithi162.stdout:5/487: read - d8/d1a/d23/d40/d4b/f90 zero size 2024-04-27T04:10:08.238 INFO:tasks.workunit.client.0.smithi162.stdout:3/461: symlink d1/d45/l99 0 2024-04-27T04:10:08.240 INFO:tasks.workunit.client.0.smithi162.stdout:0/515: getdents d5/dc/d1e/d56/d57 0 2024-04-27T04:10:08.245 INFO:tasks.workunit.client.0.smithi162.stdout:8/474: rename d0/d10/d44/d70/l84 to d0/d10/d2b/d2d/d2f/d42/la4 0 2024-04-27T04:10:08.246 INFO:tasks.workunit.client.0.smithi162.stdout:9/453: dwrite d3/d23/d1e/f4a [0,4194304] 0 2024-04-27T04:10:08.248 INFO:tasks.workunit.client.0.smithi162.stdout:6/475: rmdir d6 39 2024-04-27T04:10:08.251 INFO:tasks.workunit.client.1.smithi181.stdout:7/640: mknod d1/d2/d1c/d2d/d61/d80/cc1 0 2024-04-27T04:10:08.251 INFO:tasks.workunit.client.0.smithi162.stdout:5/488: dread d8/d1a/f59 [0,4194304] 0 2024-04-27T04:10:08.254 INFO:tasks.workunit.client.0.smithi162.stdout:3/462: dread d1/d18/d23/d34/f3c [0,4194304] 0 2024-04-27T04:10:08.255 INFO:tasks.workunit.client.0.smithi162.stdout:3/463: chown d1/d18/d5d/f62 29078 1 2024-04-27T04:10:08.256 INFO:tasks.workunit.client.1.smithi181.stdout:7/641: dread d1/d2/f27 [0,4194304] 0 2024-04-27T04:10:08.261 INFO:tasks.workunit.client.1.smithi181.stdout:5/785: creat d4/de/f113 x:0 0 0 2024-04-27T04:10:08.261 INFO:tasks.workunit.client.0.smithi162.stdout:7/418: mkdir d3/d39/d8d 0 2024-04-27T04:10:08.262 INFO:tasks.workunit.client.0.smithi162.stdout:7/419: write d3/d39/d57/f43 [6596014,66771] 0 2024-04-27T04:10:08.266 INFO:tasks.workunit.client.0.smithi162.stdout:7/420: dread d3/d5/d5b/d83/f6e [0,4194304] 0 2024-04-27T04:10:08.268 INFO:tasks.workunit.client.0.smithi162.stdout:8/475: creat d0/de/d20/d6c/fa5 x:0 0 0 2024-04-27T04:10:08.270 INFO:tasks.workunit.client.0.smithi162.stdout:1/430: sync 2024-04-27T04:10:08.270 INFO:tasks.workunit.client.1.smithi181.stdout:8/721: unlink d1/d6/f4e 0 2024-04-27T04:10:08.271 INFO:tasks.workunit.client.0.smithi162.stdout:9/454: creat d3/d13/d15/d50/f8b x:0 0 0 2024-04-27T04:10:08.274 INFO:tasks.workunit.client.0.smithi162.stdout:8/476: dread d0/d10/d25/f88 [0,4194304] 0 2024-04-27T04:10:08.274 INFO:tasks.workunit.client.0.smithi162.stdout:6/476: write d6/d1a/d34/d43/f6b [697999,59263] 0 2024-04-27T04:10:08.274 INFO:tasks.workunit.client.1.smithi181.stdout:9/590: fsync d0/d44/d4b/d43/f56 0 2024-04-27T04:10:08.275 INFO:tasks.workunit.client.0.smithi162.stdout:6/477: chown d6/db/f6a 0 1 2024-04-27T04:10:08.276 INFO:tasks.workunit.client.1.smithi181.stdout:3/596: truncate d0/de/f1f 1637674 0 2024-04-27T04:10:08.278 INFO:tasks.workunit.client.0.smithi162.stdout:5/489: symlink d8/d2c/la3 0 2024-04-27T04:10:08.279 INFO:tasks.workunit.client.1.smithi181.stdout:0/762: dread dc/d1f/d3b/d4a/d95/d26/f6c [0,4194304] 0 2024-04-27T04:10:08.284 INFO:tasks.workunit.client.0.smithi162.stdout:0/516: write d5/dc/fe [2613203,114890] 0 2024-04-27T04:10:08.285 INFO:tasks.workunit.client.1.smithi181.stdout:9/591: dread d0/f26 [4194304,4194304] 0 2024-04-27T04:10:08.285 INFO:tasks.workunit.client.1.smithi181.stdout:7/642: mknod d1/d22/d15/d82/cc2 0 2024-04-27T04:10:08.287 INFO:tasks.workunit.client.0.smithi162.stdout:3/464: unlink d1/d21/l33 0 2024-04-27T04:10:08.290 INFO:tasks.workunit.client.1.smithi181.stdout:7/643: dwrite d1/d2/d1c/d2d/d61/f97 [0,4194304] 0 2024-04-27T04:10:08.292 INFO:tasks.workunit.client.0.smithi162.stdout:0/517: dread d5/dc/d1e/d41/f34 [0,4194304] 0 2024-04-27T04:10:08.296 INFO:tasks.workunit.client.0.smithi162.stdout:7/421: creat d3/d5/d3c/d4d/f8e x:0 0 0 2024-04-27T04:10:08.297 INFO:tasks.workunit.client.0.smithi162.stdout:7/422: chown d3/d5/d5b/d83/de/f62 791 1 2024-04-27T04:10:08.298 INFO:tasks.workunit.client.0.smithi162.stdout:7/423: stat d3/d5/d5b/d83/de/d2b/f37 0 2024-04-27T04:10:08.298 INFO:tasks.workunit.client.1.smithi181.stdout:7/644: dwrite d1/d22/d15/f2a [0,4194304] 0 2024-04-27T04:10:08.300 INFO:tasks.workunit.client.0.smithi162.stdout:7/424: dread d3/f1e [0,4194304] 0 2024-04-27T04:10:08.301 INFO:tasks.workunit.client.0.smithi162.stdout:7/425: write d3/d5/d5b/d83/f35 [5535940,14497] 0 2024-04-27T04:10:08.301 INFO:tasks.workunit.client.1.smithi181.stdout:7/645: truncate d1/d2/d1c/d37/f59 1159923 0 2024-04-27T04:10:08.301 INFO:tasks.workunit.client.1.smithi181.stdout:7/646: readlink d1/d2/l6 0 2024-04-27T04:10:08.308 INFO:tasks.workunit.client.0.smithi162.stdout:4/443: dwrite d2/da/f31 [0,4194304] 0 2024-04-27T04:10:08.308 INFO:tasks.workunit.client.1.smithi181.stdout:8/722: symlink d1/d6/d59/df1/lfa 0 2024-04-27T04:10:08.310 INFO:tasks.workunit.client.0.smithi162.stdout:9/455: unlink d3/d13/d2d/d44/l52 0 2024-04-27T04:10:08.311 INFO:tasks.workunit.client.0.smithi162.stdout:9/456: write d3/d13/d2d/d3b/f47 [2651360,4205] 0 2024-04-27T04:10:08.315 INFO:tasks.workunit.client.0.smithi162.stdout:6/478: unlink d6/d1a/d34/d3a/f3d 0 2024-04-27T04:10:08.318 INFO:tasks.workunit.client.1.smithi181.stdout:3/597: mkdir d0/de/d7d/dbc 0 2024-04-27T04:10:08.319 INFO:tasks.workunit.client.0.smithi162.stdout:5/490: creat d8/d1a/d23/d40/fa4 x:0 0 0 2024-04-27T04:10:08.322 INFO:tasks.workunit.client.1.smithi181.stdout:0/763: truncate dc/d54/fab 479466 0 2024-04-27T04:10:08.322 INFO:tasks.workunit.client.1.smithi181.stdout:6/537: truncate d7/dc/d1e/d5c/f65 1083389 0 2024-04-27T04:10:08.323 INFO:tasks.workunit.client.0.smithi162.stdout:0/518: mkdir d5/d26/da2 0 2024-04-27T04:10:08.328 INFO:tasks.workunit.client.1.smithi181.stdout:0/764: dwrite dc/f10 [0,4194304] 0 2024-04-27T04:10:08.329 INFO:tasks.workunit.client.1.smithi181.stdout:5/786: creat d4/de/d2c/d46/d111/f114 x:0 0 0 2024-04-27T04:10:08.330 INFO:tasks.workunit.client.1.smithi181.stdout:0/765: chown dc/d1f/d3b/d4a/d95/d26/f6c 74211 1 2024-04-27T04:10:08.331 INFO:tasks.workunit.client.0.smithi162.stdout:7/426: write d3/d5/d5b/d83/f6e [88204,106972] 0 2024-04-27T04:10:08.331 INFO:tasks.workunit.client.1.smithi181.stdout:0/766: dread - dc/d1f/d3b/ff9 zero size 2024-04-27T04:10:08.335 INFO:tasks.workunit.client.1.smithi181.stdout:4/622: link d0/d15/d25/l26 d0/d15/de/d19/d77/dc5/lce 0 2024-04-27T04:10:08.336 INFO:tasks.workunit.client.1.smithi181.stdout:4/623: stat d0/d15/de/d19 0 2024-04-27T04:10:08.337 INFO:tasks.workunit.client.0.smithi162.stdout:4/444: fsync d2/da/f78 0 2024-04-27T04:10:08.338 INFO:tasks.workunit.client.0.smithi162.stdout:7/427: dread d3/d5/d9/f7f [0,4194304] 0 2024-04-27T04:10:08.339 INFO:tasks.workunit.client.0.smithi162.stdout:9/457: rmdir d3/d23 39 2024-04-27T04:10:08.342 INFO:tasks.workunit.client.0.smithi162.stdout:4/445: dread d2/da/dd/f60 [0,4194304] 0 2024-04-27T04:10:08.344 INFO:tasks.workunit.client.0.smithi162.stdout:3/465: mkdir d1/d18/d23/d9a 0 2024-04-27T04:10:08.346 INFO:tasks.workunit.client.0.smithi162.stdout:5/491: write d8/d1a/d23/d40/d4b/d78/f9d [1770433,3947] 0 2024-04-27T04:10:08.348 INFO:tasks.workunit.client.0.smithi162.stdout:5/492: truncate d8/d1a/d7d/f91 634203 0 2024-04-27T04:10:08.352 INFO:tasks.workunit.client.1.smithi181.stdout:6/538: creat d7/dc/d1e/d66/fbd x:0 0 0 2024-04-27T04:10:08.353 INFO:tasks.workunit.client.1.smithi181.stdout:6/539: dread - d7/d31/f75 zero size 2024-04-27T04:10:08.357 INFO:tasks.workunit.client.1.smithi181.stdout:5/787: creat d4/d10/d29/dd2/f115 x:0 0 0 2024-04-27T04:10:08.357 INFO:tasks.workunit.client.1.smithi181.stdout:5/788: chown d4/de/d10d 5130 1 2024-04-27T04:10:08.358 INFO:tasks.workunit.client.1.smithi181.stdout:0/767: truncate dc/f42 1345860 0 2024-04-27T04:10:08.360 INFO:tasks.workunit.client.1.smithi181.stdout:1/666: truncate d1/dad/db5/d6/f4e 1253615 0 2024-04-27T04:10:08.360 INFO:tasks.workunit.client.1.smithi181.stdout:2/699: link d4/d9/df/d14/cbd d4/d9/df/d14/da7/ce5 0 2024-04-27T04:10:08.365 INFO:tasks.workunit.client.1.smithi181.stdout:5/789: dwrite d4/de/d2c/d46/d99/dad/fe1 [0,4194304] 0 2024-04-27T04:10:08.369 INFO:tasks.workunit.client.1.smithi181.stdout:2/700: dwrite d4/d9/df/f41 [0,4194304] 0 2024-04-27T04:10:08.373 INFO:tasks.workunit.client.1.smithi181.stdout:9/592: creat d0/d40/fcd x:0 0 0 2024-04-27T04:10:08.374 INFO:tasks.workunit.client.1.smithi181.stdout:2/701: dread d4/d81/f8e [0,4194304] 0 2024-04-27T04:10:08.374 INFO:tasks.workunit.client.1.smithi181.stdout:1/667: mkdir d1/d2/d64/ddc 0 2024-04-27T04:10:08.381 INFO:tasks.workunit.client.1.smithi181.stdout:9/593: dwrite d0/d44/d4b/d3d/dbd/fcc [0,4194304] 0 2024-04-27T04:10:08.385 INFO:tasks.workunit.client.1.smithi181.stdout:7/647: rename d1/d2/d35/d8b to d1/dc3 0 2024-04-27T04:10:08.386 INFO:tasks.workunit.client.0.smithi162.stdout:2/411: sync 2024-04-27T04:10:08.396 INFO:tasks.workunit.client.0.smithi162.stdout:9/458: write d3/d23/d1e/f2b [281571,5737] 0 2024-04-27T04:10:08.397 INFO:tasks.workunit.client.1.smithi181.stdout:3/598: creat d0/d6c/d66/fbd x:0 0 0 2024-04-27T04:10:08.397 INFO:tasks.workunit.client.0.smithi162.stdout:9/459: write d3/f5 [8771413,92863] 0 2024-04-27T04:10:08.398 INFO:tasks.workunit.client.1.smithi181.stdout:3/599: chown d0/de/d20/d58/f7e 411656 1 2024-04-27T04:10:08.399 INFO:tasks.workunit.client.1.smithi181.stdout:3/600: chown d0/d6c/d52/d44/d74 27 1 2024-04-27T04:10:08.400 INFO:tasks.workunit.client.1.smithi181.stdout:3/601: readlink d0/d6c/l4a 0 2024-04-27T04:10:08.401 INFO:tasks.workunit.client.1.smithi181.stdout:3/602: chown d0/de/d20/d58/d64/f6a 13206610 1 2024-04-27T04:10:08.401 INFO:tasks.workunit.client.1.smithi181.stdout:8/723: write d1/d6/f26 [2919951,98310] 0 2024-04-27T04:10:08.402 INFO:tasks.workunit.client.1.smithi181.stdout:8/724: stat d1/d6/d2f/d47 0 2024-04-27T04:10:08.403 INFO:tasks.workunit.client.1.smithi181.stdout:3/603: fdatasync d0/d6c/d52/dbb/d98/da7/fae 0 2024-04-27T04:10:08.403 INFO:tasks.workunit.client.0.smithi162.stdout:4/446: chown d2/d25/d1c/d39/l77 3660969 1 2024-04-27T04:10:08.405 INFO:tasks.workunit.client.0.smithi162.stdout:0/519: creat d5/d60/d98/fa3 x:0 0 0 2024-04-27T04:10:08.407 INFO:tasks.workunit.client.1.smithi181.stdout:1/668: dread d1/f8e [0,4194304] 0 2024-04-27T04:10:08.407 INFO:tasks.workunit.client.1.smithi181.stdout:2/702: write d4/d9/f7a [4266020,125752] 0 2024-04-27T04:10:08.408 INFO:tasks.workunit.client.0.smithi162.stdout:4/447: dwrite d2/da/f31 [0,4194304] 0 2024-04-27T04:10:08.417 INFO:tasks.workunit.client.1.smithi181.stdout:7/648: truncate d1/d2/d1c/d56/fa8 1280518 0 2024-04-27T04:10:08.419 INFO:tasks.workunit.client.1.smithi181.stdout:4/624: link d0/d15/d76/da4/lc2 d0/d15/d25/d2a/da0/dc7/lcf 0 2024-04-27T04:10:08.423 INFO:tasks.workunit.client.1.smithi181.stdout:5/790: link d4/d10/d29/dd2/f115 d4/de/d10d/f116 0 2024-04-27T04:10:08.425 INFO:tasks.workunit.client.1.smithi181.stdout:5/791: stat d4/de/d2c/d46/l69 0 2024-04-27T04:10:08.426 INFO:tasks.workunit.client.0.smithi162.stdout:7/428: rename d3/d5/d5b/d83/c11 to d3/d5/d5b/d83/de/d2b/c8f 0 2024-04-27T04:10:08.426 INFO:tasks.workunit.client.1.smithi181.stdout:4/625: dwrite d0/d15/d25/d2a/f7e [0,4194304] 0 2024-04-27T04:10:08.427 INFO:tasks.workunit.client.1.smithi181.stdout:5/792: chown d4/de/d2c/d46/d4d/c10c 475 1 2024-04-27T04:10:08.428 INFO:tasks.workunit.client.1.smithi181.stdout:4/626: fdatasync d0/d15/de/d19/d8a/fa9 0 2024-04-27T04:10:08.428 INFO:tasks.workunit.client.0.smithi162.stdout:8/477: link d0/d10/l34 d0/d10/d25/d65/la6 0 2024-04-27T04:10:08.429 INFO:tasks.workunit.client.1.smithi181.stdout:4/627: readlink d0/d15/d25/d2a/l96 0 2024-04-27T04:10:08.430 INFO:tasks.workunit.client.1.smithi181.stdout:3/604: creat d0/d6c/d52/fbe x:0 0 0 2024-04-27T04:10:08.434 INFO:tasks.workunit.client.0.smithi162.stdout:2/412: write d0/d24/f4d [9353452,98882] 0 2024-04-27T04:10:08.435 INFO:tasks.workunit.client.1.smithi181.stdout:2/703: write d4/fd9 [470270,12197] 0 2024-04-27T04:10:08.441 INFO:tasks.workunit.client.1.smithi181.stdout:7/649: symlink d1/d2/d1c/d8f/lc4 0 2024-04-27T04:10:08.442 INFO:tasks.workunit.client.0.smithi162.stdout:9/460: dwrite f0 [0,4194304] 0 2024-04-27T04:10:08.443 INFO:tasks.workunit.client.0.smithi162.stdout:7/429: mknod d3/d4f/c90 0 2024-04-27T04:10:08.444 INFO:tasks.workunit.client.1.smithi181.stdout:6/540: read d7/d31/f37 [14911,82995] 0 2024-04-27T04:10:08.448 INFO:tasks.workunit.client.0.smithi162.stdout:9/461: dwrite d3/d13/d15/f18 [0,4194304] 0 2024-04-27T04:10:08.449 INFO:tasks.workunit.client.0.smithi162.stdout:9/462: write d3/d23/f5c [126923,76209] 0 2024-04-27T04:10:08.451 INFO:tasks.workunit.client.0.smithi162.stdout:5/493: rmdir d8/d1a/d23/d40/d7b 0 2024-04-27T04:10:08.451 INFO:tasks.workunit.client.0.smithi162.stdout:1/431: sync 2024-04-27T04:10:08.454 INFO:tasks.workunit.client.0.smithi162.stdout:0/520: symlink d5/d26/d69/d9b/da1/la4 0 2024-04-27T04:10:08.455 INFO:tasks.workunit.client.0.smithi162.stdout:0/521: chown d5/d26/d4a/f4c 2933 1 2024-04-27T04:10:08.455 INFO:tasks.workunit.client.0.smithi162.stdout:0/522: readlink d5/d26/l76 0 2024-04-27T04:10:08.457 INFO:tasks.workunit.client.0.smithi162.stdout:4/448: mknod d2/da/c8f 0 2024-04-27T04:10:08.460 INFO:tasks.workunit.client.0.smithi162.stdout:2/413: mknod d0/d20/d2e/c82 0 2024-04-27T04:10:08.461 INFO:tasks.workunit.client.1.smithi181.stdout:5/793: mknod d4/de/d2f/d7a/c117 0 2024-04-27T04:10:08.461 INFO:tasks.workunit.client.0.smithi162.stdout:2/414: fdatasync d0/d20/d2e/f69 0 2024-04-27T04:10:08.462 INFO:tasks.workunit.client.0.smithi162.stdout:8/478: dwrite d0/d10/d2b/d2e/f46 [0,4194304] 0 2024-04-27T04:10:08.466 INFO:tasks.workunit.client.1.smithi181.stdout:1/669: fsync d1/dad/db5/d6/da3/f3b 0 2024-04-27T04:10:08.467 INFO:tasks.workunit.client.1.smithi181.stdout:2/704: fsync d4/d9/d16/d45/f6c 0 2024-04-27T04:10:08.469 INFO:tasks.workunit.client.1.smithi181.stdout:9/594: link d0/d4d/f7f d0/d44/d4f/d9d/fce 0 2024-04-27T04:10:08.471 INFO:tasks.workunit.client.0.smithi162.stdout:2/415: read d0/d4/f16 [287466,106414] 0 2024-04-27T04:10:08.472 INFO:tasks.workunit.client.1.smithi181.stdout:0/768: rename dc/f39 to dc/d1f/d3b/d4a/d95/f10b 0 2024-04-27T04:10:08.473 INFO:tasks.workunit.client.1.smithi181.stdout:6/541: truncate f3 111524 0 2024-04-27T04:10:08.475 INFO:tasks.workunit.client.0.smithi162.stdout:2/416: read d0/f44 [123505,28978] 0 2024-04-27T04:10:08.478 INFO:tasks.workunit.client.0.smithi162.stdout:4/449: mkdir d2/d25/d1c/d39/d90 0 2024-04-27T04:10:08.479 INFO:tasks.workunit.client.1.smithi181.stdout:5/794: symlink d4/de/d2c/d46/d4d/d8e/d54/l118 0 2024-04-27T04:10:08.484 INFO:tasks.workunit.client.0.smithi162.stdout:7/430: creat d3/d39/d8d/f91 x:0 0 0 2024-04-27T04:10:08.486 INFO:tasks.workunit.client.1.smithi181.stdout:1/670: unlink d1/dad/db5/d6/da3/c80 0 2024-04-27T04:10:08.487 INFO:tasks.workunit.client.1.smithi181.stdout:1/671: readlink d1/dad/db5/d6/da3/d60/l90 0 2024-04-27T04:10:08.487 INFO:tasks.workunit.client.0.smithi162.stdout:3/466: sync 2024-04-27T04:10:08.488 INFO:tasks.workunit.client.0.smithi162.stdout:6/479: sync 2024-04-27T04:10:08.488 INFO:tasks.workunit.client.1.smithi181.stdout:4/628: write d0/d15/d5c/d69/f10 [856973,48943] 0 2024-04-27T04:10:08.489 INFO:tasks.workunit.client.0.smithi162.stdout:6/480: truncate d6/db/f13 522614 0 2024-04-27T04:10:08.491 INFO:tasks.workunit.client.0.smithi162.stdout:6/481: write d6/d1a/d34/d3a/d68/f78 [1093134,60554] 0 2024-04-27T04:10:08.491 INFO:tasks.workunit.client.0.smithi162.stdout:8/479: rmdir d0/d10/d2b/d2e 39 2024-04-27T04:10:08.492 INFO:tasks.workunit.client.1.smithi181.stdout:3/605: dread d0/de/f10 [0,4194304] 0 2024-04-27T04:10:08.494 INFO:tasks.workunit.client.0.smithi162.stdout:0/523: dread d5/d60/d8b/f99 [0,4194304] 0 2024-04-27T04:10:08.494 INFO:tasks.workunit.client.0.smithi162.stdout:8/480: chown d0/d10/d25/f14 13269551 1 2024-04-27T04:10:08.494 INFO:tasks.workunit.client.0.smithi162.stdout:6/482: dread d6/d1a/d34/d43/f5c [0,4194304] 0 2024-04-27T04:10:08.495 INFO:tasks.workunit.client.1.smithi181.stdout:2/705: truncate d4/f59 2518728 0 2024-04-27T04:10:08.496 INFO:tasks.workunit.client.0.smithi162.stdout:8/481: chown d0/d10/d2b/d2d/d2f/d6b 7 1 2024-04-27T04:10:08.496 INFO:tasks.workunit.client.0.smithi162.stdout:6/483: stat d6/d1a/d34/d43/c71 0 2024-04-27T04:10:08.496 INFO:tasks.workunit.client.0.smithi162.stdout:9/463: getdents d3/d13/d15/d7e 0 2024-04-27T04:10:08.496 INFO:tasks.workunit.client.0.smithi162.stdout:1/432: write d0/d11/f2e [1117369,32719] 0 2024-04-27T04:10:08.497 INFO:tasks.workunit.client.1.smithi181.stdout:9/595: symlink d0/d44/d4b/d43/d8f/lcf 0 2024-04-27T04:10:08.505 INFO:tasks.workunit.client.1.smithi181.stdout:8/725: rename d1/d6/d6f/d91/lc3 to d1/d6/d2f/d47/dba/dbc/lfb 0 2024-04-27T04:10:08.506 INFO:tasks.workunit.client.0.smithi162.stdout:7/431: dread d3/d39/d57/f43 [0,4194304] 0 2024-04-27T04:10:08.507 INFO:tasks.workunit.client.0.smithi162.stdout:5/494: link d8/d9/f10 d8/d9/d55/fa5 0 2024-04-27T04:10:08.508 INFO:tasks.workunit.client.1.smithi181.stdout:0/769: rmdir dc/d1f/d6b/d7e/d91 39 2024-04-27T04:10:08.510 INFO:tasks.workunit.client.1.smithi181.stdout:6/542: rmdir d7/d31/d45/d72 39 2024-04-27T04:10:08.511 INFO:tasks.workunit.client.1.smithi181.stdout:6/543: chown d7/d31/d45/f4b 25438 1 2024-04-27T04:10:08.512 INFO:tasks.workunit.client.0.smithi162.stdout:2/417: creat d0/d4/d2a/d41/d77/f83 x:0 0 0 2024-04-27T04:10:08.513 INFO:tasks.workunit.client.0.smithi162.stdout:5/495: dwrite d8/d1a/d29/d4a/d9f/f64 [0,4194304] 0 2024-04-27T04:10:08.515 INFO:tasks.workunit.client.0.smithi162.stdout:4/450: mkdir d2/d25/d1c/d39/d70/d91 0 2024-04-27T04:10:08.516 INFO:tasks.workunit.client.0.smithi162.stdout:2/418: read d0/d4/f22 [1877444,129798] 0 2024-04-27T04:10:08.517 INFO:tasks.workunit.client.0.smithi162.stdout:4/451: truncate d2/d25/f89 841647 0 2024-04-27T04:10:08.518 INFO:tasks.workunit.client.1.smithi181.stdout:4/629: unlink d0/d15/d25/d2a/fad 0 2024-04-27T04:10:08.519 INFO:tasks.workunit.client.0.smithi162.stdout:1/433: dread d0/d7/f19 [0,4194304] 0 2024-04-27T04:10:08.520 INFO:tasks.workunit.client.1.smithi181.stdout:1/672: truncate d1/dad/db5/d6/da3/f1f 1708558 0 2024-04-27T04:10:08.522 INFO:tasks.workunit.client.1.smithi181.stdout:3/606: chown d0/f3c 171873 1 2024-04-27T04:10:08.525 INFO:tasks.workunit.client.1.smithi181.stdout:6/544: dread d7/dc/d1e/f2a [0,4194304] 0 2024-04-27T04:10:08.526 INFO:tasks.workunit.client.1.smithi181.stdout:6/545: write d7/dc/d1e/f52 [3118142,87422] 0 2024-04-27T04:10:08.529 INFO:tasks.workunit.client.0.smithi162.stdout:0/524: write d5/dc/f74 [627965,37288] 0 2024-04-27T04:10:08.531 INFO:tasks.workunit.client.1.smithi181.stdout:7/650: rename d1/d2/d1c/d56/f63 to d1/df/d1a/fc5 0 2024-04-27T04:10:08.537 INFO:tasks.workunit.client.0.smithi162.stdout:9/464: mknod d3/d13/d2d/d3b/c8c 0 2024-04-27T04:10:08.543 INFO:tasks.workunit.client.0.smithi162.stdout:7/432: unlink d3/f86 0 2024-04-27T04:10:08.543 INFO:tasks.workunit.client.0.smithi162.stdout:7/433: readlink d3/d5/d5b/d83/de/l34 0 2024-04-27T04:10:08.548 INFO:tasks.workunit.client.0.smithi162.stdout:9/465: dread d3/d13/d2d/d30/f38 [0,4194304] 0 2024-04-27T04:10:08.549 INFO:tasks.workunit.client.0.smithi162.stdout:9/466: write d3/d23/f88 [873372,75418] 0 2024-04-27T04:10:08.553 INFO:tasks.workunit.client.0.smithi162.stdout:5/496: fsync d8/d1a/f2d 0 2024-04-27T04:10:08.555 INFO:tasks.workunit.client.0.smithi162.stdout:9/467: dwrite d3/d23/d1e/f4a [0,4194304] 0 2024-04-27T04:10:08.555 INFO:tasks.workunit.client.0.smithi162.stdout:5/497: truncate d8/d1a/d23/d40/f77 540794 0 2024-04-27T04:10:08.556 INFO:tasks.workunit.client.1.smithi181.stdout:6/546: symlink d7/dc/d1c/d88/lbe 0 2024-04-27T04:10:08.556 INFO:tasks.workunit.client.0.smithi162.stdout:5/498: chown d8/d86 15 1 2024-04-27T04:10:08.557 INFO:tasks.workunit.client.0.smithi162.stdout:2/419: chown d0/d20/c21 584 1 2024-04-27T04:10:08.558 INFO:tasks.workunit.client.0.smithi162.stdout:2/420: truncate d0/d24/f4b 1039115 0 2024-04-27T04:10:08.558 INFO:tasks.workunit.client.0.smithi162.stdout:2/421: chown d0/l1f 6113 1 2024-04-27T04:10:08.562 INFO:tasks.workunit.client.1.smithi181.stdout:9/596: rename d0/f55 to d0/d44/d4f/d9d/fd0 0 2024-04-27T04:10:08.564 INFO:tasks.workunit.client.1.smithi181.stdout:9/597: write d0/d44/d4b/d3d/f9f [795379,58325] 0 2024-04-27T04:10:08.564 INFO:tasks.workunit.client.0.smithi162.stdout:4/452: mknod d2/d25/d1c/d39/d5e/c92 0 2024-04-27T04:10:08.566 INFO:tasks.workunit.client.0.smithi162.stdout:4/453: dread d2/d25/f89 [0,4194304] 0 2024-04-27T04:10:08.567 INFO:tasks.workunit.client.0.smithi162.stdout:4/454: stat d2/d25/f2d 0 2024-04-27T04:10:08.568 INFO:tasks.workunit.client.0.smithi162.stdout:1/434: symlink d0/d7/d3c/d54/l8c 0 2024-04-27T04:10:08.569 INFO:tasks.workunit.client.0.smithi162.stdout:1/435: dread - d0/d7/d3c/f43 zero size 2024-04-27T04:10:08.571 INFO:tasks.workunit.client.0.smithi162.stdout:3/467: sync 2024-04-27T04:10:08.573 INFO:tasks.workunit.client.0.smithi162.stdout:4/455: dwrite d2/d25/d1c/d39/d3b/d47/f4c [0,4194304] 0 2024-04-27T04:10:08.573 INFO:tasks.workunit.client.1.smithi181.stdout:2/706: truncate d4/f8 3027543 0 2024-04-27T04:10:08.575 INFO:tasks.workunit.client.1.smithi181.stdout:5/795: rmdir d4/de/d107/d10e 0 2024-04-27T04:10:08.577 INFO:tasks.workunit.client.0.smithi162.stdout:6/484: unlink d6/f39 0 2024-04-27T04:10:08.584 INFO:tasks.workunit.client.1.smithi181.stdout:1/673: rename d1/dad/db5/f45 to d1/dad/db5/d6/da3/d1e/d20/d53/d56/dd9/fdd 0 2024-04-27T04:10:08.584 INFO:tasks.workunit.client.0.smithi162.stdout:9/468: creat d3/d13/d2d/d3b/d54/f8d x:0 0 0 2024-04-27T04:10:08.588 INFO:tasks.workunit.client.1.smithi181.stdout:0/770: link dc/d1f/fae dc/d1f/d6b/d7e/f10c 0 2024-04-27T04:10:08.589 INFO:tasks.workunit.client.1.smithi181.stdout:0/771: write dc/d1f/d3b/d4a/d95/fe6 [158955,61125] 0 2024-04-27T04:10:08.590 INFO:tasks.workunit.client.0.smithi162.stdout:9/469: dwrite d3/f14 [0,4194304] 0 2024-04-27T04:10:08.591 INFO:tasks.workunit.client.0.smithi162.stdout:1/436: creat d0/d7/d3c/d77/f8d x:0 0 0 2024-04-27T04:10:08.594 INFO:tasks.workunit.client.1.smithi181.stdout:5/796: truncate d4/de/f6f 1067218 0 2024-04-27T04:10:08.595 INFO:tasks.workunit.client.1.smithi181.stdout:5/797: write d4/da3/dbc/db5/ffa [728271,102307] 0 2024-04-27T04:10:08.596 INFO:tasks.workunit.client.0.smithi162.stdout:0/525: mknod d5/dc/d1e/d41/d79/ca5 0 2024-04-27T04:10:08.596 INFO:tasks.workunit.client.0.smithi162.stdout:3/468: read d1/d21/f2c [3657212,100250] 0 2024-04-27T04:10:08.596 INFO:tasks.workunit.client.1.smithi181.stdout:6/547: truncate d7/d31/d45/d62/f63 1579292 0 2024-04-27T04:10:08.598 INFO:tasks.workunit.client.0.smithi162.stdout:6/485: mknod d6/d1a/d34/d3a/d68/d6c/d67/c98 0 2024-04-27T04:10:08.599 INFO:tasks.workunit.client.1.smithi181.stdout:1/674: symlink d1/d2/d64/lde 0 2024-04-27T04:10:08.600 INFO:tasks.workunit.client.0.smithi162.stdout:0/526: write d5/d26/d4a/f77 [6776497,61061] 0 2024-04-27T04:10:08.600 INFO:tasks.workunit.client.0.smithi162.stdout:7/434: mkdir d3/d92 0 2024-04-27T04:10:08.601 INFO:tasks.workunit.client.0.smithi162.stdout:9/470: dwrite d3/d23/f5c [0,4194304] 0 2024-04-27T04:10:08.603 INFO:tasks.workunit.client.0.smithi162.stdout:1/437: dwrite d0/d11/d50/d53/f84 [0,4194304] 0 2024-04-27T04:10:08.605 INFO:tasks.workunit.client.0.smithi162.stdout:0/527: write d5/dc/d1e/d41/f83 [3957495,125824] 0 2024-04-27T04:10:08.605 INFO:tasks.workunit.client.0.smithi162.stdout:0/528: fdatasync d5/f2d 0 2024-04-27T04:10:08.606 INFO:tasks.workunit.client.1.smithi181.stdout:2/707: unlink d4/d9/d4f/f5b 0 2024-04-27T04:10:08.607 INFO:tasks.workunit.client.0.smithi162.stdout:1/438: write d0/d11/d3e/d5f/f6a [58428,77186] 0 2024-04-27T04:10:08.608 INFO:tasks.workunit.client.1.smithi181.stdout:3/607: getdents d0/de 0 2024-04-27T04:10:08.608 INFO:tasks.workunit.client.0.smithi162.stdout:1/439: truncate d0/d7/d2a/f5d 896161 0 2024-04-27T04:10:08.609 INFO:tasks.workunit.client.0.smithi162.stdout:1/440: chown d0/lf 65248426 1 2024-04-27T04:10:08.609 INFO:tasks.workunit.client.1.smithi181.stdout:3/608: readlink d0/d6c/d52/l61 0 2024-04-27T04:10:08.612 INFO:tasks.workunit.client.1.smithi181.stdout:1/675: unlink d1/dad/db5/d6/da3/f3b 0 2024-04-27T04:10:08.614 INFO:tasks.workunit.client.1.smithi181.stdout:1/676: write d1/dad/db5/d6/da3/d52/f68 [2316090,117609] 0 2024-04-27T04:10:08.618 INFO:tasks.workunit.client.1.smithi181.stdout:2/708: unlink d4/d9/df/c1c 0 2024-04-27T04:10:08.618 INFO:tasks.workunit.client.1.smithi181.stdout:2/709: fdatasync d4/d9/fc0 0 2024-04-27T04:10:08.619 INFO:tasks.workunit.client.0.smithi162.stdout:9/471: creat d3/d11/d45/d4e/d58/f8e x:0 0 0 2024-04-27T04:10:08.619 INFO:tasks.workunit.client.0.smithi162.stdout:9/472: chown d3/d13/d2d/l64 61634 1 2024-04-27T04:10:08.619 INFO:tasks.workunit.client.0.smithi162.stdout:9/473: readlink d3/l9 0 2024-04-27T04:10:08.620 INFO:tasks.workunit.client.0.smithi162.stdout:9/474: chown d3/d13/l5f 7879 1 2024-04-27T04:10:08.624 INFO:tasks.workunit.client.1.smithi181.stdout:5/798: rename d4/de/d2c/d46/d4d/fb8 to d4/f119 0 2024-04-27T04:10:08.625 INFO:tasks.workunit.client.1.smithi181.stdout:5/799: truncate d4/da9/d100/ff7 40480 0 2024-04-27T04:10:08.626 INFO:tasks.workunit.client.1.smithi181.stdout:6/548: mkdir d7/d31/d45/d72/d98/dbf 0 2024-04-27T04:10:08.626 INFO:tasks.workunit.client.0.smithi162.stdout:9/475: dwrite d3/d11/d45/f55 [0,4194304] 0 2024-04-27T04:10:08.627 INFO:tasks.workunit.client.0.smithi162.stdout:0/529: symlink d5/dc/d1e/d46/d48/d9e/la6 0 2024-04-27T04:10:08.629 INFO:tasks.workunit.client.0.smithi162.stdout:9/476: write d3/d13/d2d/d3b/d54/f8d [483275,84186] 0 2024-04-27T04:10:08.630 INFO:tasks.workunit.client.0.smithi162.stdout:9/477: fsync d3/fb 0 2024-04-27T04:10:08.632 INFO:tasks.workunit.client.1.smithi181.stdout:2/710: symlink d4/le6 0 2024-04-27T04:10:08.633 INFO:tasks.workunit.client.0.smithi162.stdout:7/435: creat d3/f93 x:0 0 0 2024-04-27T04:10:08.634 INFO:tasks.workunit.client.0.smithi162.stdout:7/436: truncate d3/d4f/f67 810217 0 2024-04-27T04:10:08.635 INFO:tasks.workunit.client.0.smithi162.stdout:7/437: write d3/d4f/f41 [2436929,93741] 0 2024-04-27T04:10:08.635 INFO:tasks.workunit.client.1.smithi181.stdout:6/549: creat d7/dc/d1c/d7f/d3e/fc0 x:0 0 0 2024-04-27T04:10:08.636 INFO:tasks.workunit.client.1.smithi181.stdout:3/609: link d0/d6c/c6d d0/d6c/da1/cbf 0 2024-04-27T04:10:08.636 INFO:tasks.workunit.client.0.smithi162.stdout:7/438: stat d3/d5/d5b/d83/de/d2b/d81 0 2024-04-27T04:10:08.637 INFO:tasks.workunit.client.0.smithi162.stdout:7/439: chown d3/d5/c76 17 1 2024-04-27T04:10:08.638 INFO:tasks.workunit.client.0.smithi162.stdout:9/478: unlink d3/d13/d15/f2f 0 2024-04-27T04:10:08.640 INFO:tasks.workunit.client.1.smithi181.stdout:2/711: creat d4/d9/d4f/d79/d94/fe7 x:0 0 0 2024-04-27T04:10:08.641 INFO:tasks.workunit.client.1.smithi181.stdout:5/800: mknod d4/de/c11a 0 2024-04-27T04:10:08.643 INFO:tasks.workunit.client.1.smithi181.stdout:5/801: chown d4/de/d2c/f75 16578 1 2024-04-27T04:10:08.646 INFO:tasks.workunit.client.1.smithi181.stdout:2/712: dwrite d4/d9/d35/f3a [4194304,4194304] 0 2024-04-27T04:10:08.646 INFO:tasks.workunit.client.0.smithi162.stdout:0/530: creat d5/dc/d1e/d41/fa7 x:0 0 0 2024-04-27T04:10:08.651 INFO:tasks.workunit.client.0.smithi162.stdout:7/440: creat d3/d5/d5b/d60/f94 x:0 0 0 2024-04-27T04:10:08.652 INFO:tasks.workunit.client.1.smithi181.stdout:3/610: chown d0/d6c/d52/d44/d56/d5b/l6f 90 1 2024-04-27T04:10:08.653 INFO:tasks.workunit.client.1.smithi181.stdout:5/802: dread d4/da3/dbc/f9e [0,4194304] 0 2024-04-27T04:10:08.653 INFO:tasks.workunit.client.0.smithi162.stdout:0/531: mkdir d5/dc/d1e/d56/da8 0 2024-04-27T04:10:08.653 INFO:tasks.workunit.client.0.smithi162.stdout:0/532: chown d5/dc/c2e 65612202 1 2024-04-27T04:10:08.654 INFO:tasks.workunit.client.1.smithi181.stdout:2/713: creat d4/d9/d4f/d79/d98/fe8 x:0 0 0 2024-04-27T04:10:08.654 INFO:tasks.workunit.client.1.smithi181.stdout:5/803: truncate d4/d10/fdb 1412928 0 2024-04-27T04:10:08.655 INFO:tasks.workunit.client.1.smithi181.stdout:2/714: dread - d4/d9/d16/d45/fdc zero size 2024-04-27T04:10:08.655 INFO:tasks.workunit.client.0.smithi162.stdout:7/441: creat d3/d5/d3c/d4a/f95 x:0 0 0 2024-04-27T04:10:08.655 INFO:tasks.workunit.client.1.smithi181.stdout:5/804: write d4/de/f6f [743206,125529] 0 2024-04-27T04:10:08.656 INFO:tasks.workunit.client.0.smithi162.stdout:0/533: creat d5/dc/d37/fa9 x:0 0 0 2024-04-27T04:10:08.659 INFO:tasks.workunit.client.0.smithi162.stdout:7/442: mkdir d3/d5/d5b/d83/de/d2b/d81/d96 0 2024-04-27T04:10:08.660 INFO:tasks.workunit.client.1.smithi181.stdout:6/550: getdents d7 0 2024-04-27T04:10:08.662 INFO:tasks.workunit.client.1.smithi181.stdout:2/715: unlink d4/d9/df/l72 0 2024-04-27T04:10:08.663 INFO:tasks.workunit.client.0.smithi162.stdout:4/456: sync 2024-04-27T04:10:08.663 INFO:tasks.workunit.client.1.smithi181.stdout:2/716: stat d4/d9/d4f/d79/fce 0 2024-04-27T04:10:08.664 INFO:tasks.workunit.client.0.smithi162.stdout:7/443: creat d3/d5/d5b/d60/d65/d80/f97 x:0 0 0 2024-04-27T04:10:08.664 INFO:tasks.workunit.client.1.smithi181.stdout:5/805: fdatasync d4/de/d2c/d46/d99/dad/dca/fcd 0 2024-04-27T04:10:08.666 INFO:tasks.workunit.client.0.smithi162.stdout:4/457: creat d2/da/dd/f93 x:0 0 0 2024-04-27T04:10:08.666 INFO:tasks.workunit.client.1.smithi181.stdout:6/551: fsync d7/dc/d1c/d34/d3f/f74 0 2024-04-27T04:10:08.667 INFO:tasks.workunit.client.1.smithi181.stdout:6/552: dread - d7/d31/d45/d72/fb7 zero size 2024-04-27T04:10:08.669 INFO:tasks.workunit.client.1.smithi181.stdout:2/717: chown d4/d9/d4f/l90 335488 1 2024-04-27T04:10:08.670 INFO:tasks.workunit.client.1.smithi181.stdout:5/806: symlink d4/d10/d29/d92/l11b 0 2024-04-27T04:10:08.673 INFO:tasks.workunit.client.0.smithi162.stdout:0/534: dread d5/dc/f31 [0,4194304] 0 2024-04-27T04:10:08.673 INFO:tasks.workunit.client.1.smithi181.stdout:5/807: chown d4/de/l45 267802746 1 2024-04-27T04:10:08.674 INFO:tasks.workunit.client.0.smithi162.stdout:4/458: dwrite d2/d25/fb [0,4194304] 0 2024-04-27T04:10:08.674 INFO:tasks.workunit.client.0.smithi162.stdout:0/535: fsync d5/dc/d1e/d41/f53 0 2024-04-27T04:10:08.676 INFO:tasks.workunit.client.1.smithi181.stdout:2/718: dwrite d4/d9/f11 [0,4194304] 0 2024-04-27T04:10:08.678 INFO:tasks.workunit.client.1.smithi181.stdout:5/808: symlink d4/da9/d100/l11c 0 2024-04-27T04:10:08.682 INFO:tasks.workunit.client.0.smithi162.stdout:7/444: dread d3/d5/d5b/d83/f19 [0,4194304] 0 2024-04-27T04:10:08.682 INFO:tasks.workunit.client.1.smithi181.stdout:5/809: symlink d4/d10/l11d 0 2024-04-27T04:10:08.683 INFO:tasks.workunit.client.0.smithi162.stdout:7/445: chown d3/d5/d3c/d4d 28969 1 2024-04-27T04:10:08.684 INFO:tasks.workunit.client.0.smithi162.stdout:0/536: getdents d5/d26 0 2024-04-27T04:10:08.684 INFO:tasks.workunit.client.0.smithi162.stdout:7/446: fdatasync d3/d4f/f31 0 2024-04-27T04:10:08.684 INFO:tasks.workunit.client.0.smithi162.stdout:0/537: chown d5/d26 29654 1 2024-04-27T04:10:08.686 INFO:tasks.workunit.client.0.smithi162.stdout:7/447: write d3/d4f/f41 [3537180,28474] 0 2024-04-27T04:10:08.690 INFO:tasks.workunit.client.0.smithi162.stdout:0/538: mknod d5/dc/d1e/d41/d79/caa 0 2024-04-27T04:10:08.691 INFO:tasks.workunit.client.0.smithi162.stdout:0/539: fdatasync d5/f16 0 2024-04-27T04:10:08.691 INFO:tasks.workunit.client.0.smithi162.stdout:7/448: write d3/d5/d9/f13 [4262468,37377] 0 2024-04-27T04:10:08.695 INFO:tasks.workunit.client.1.smithi181.stdout:5/810: dwrite d4/de/f14 [0,4194304] 0 2024-04-27T04:10:08.696 INFO:tasks.workunit.client.0.smithi162.stdout:0/540: dread d5/dc/d1e/d56/f90 [0,4194304] 0 2024-04-27T04:10:08.700 INFO:tasks.workunit.client.1.smithi181.stdout:6/553: dread d7/dc/d1c/d34/d3f/d7e/d8a/f7a [0,4194304] 0 2024-04-27T04:10:08.706 INFO:tasks.workunit.client.1.smithi181.stdout:5/811: mknod d4/da9/d100/d55/c11e 0 2024-04-27T04:10:08.708 INFO:tasks.workunit.client.0.smithi162.stdout:7/449: dwrite d3/d5/d5b/d83/f3b [0,4194304] 0 2024-04-27T04:10:08.708 INFO:tasks.workunit.client.1.smithi181.stdout:7/651: write d1/d2/d1c/d2d/f41 [1718112,59060] 0 2024-04-27T04:10:08.709 INFO:tasks.workunit.client.1.smithi181.stdout:7/652: chown d1/d2/d1c/d8f/l9b 0 1 2024-04-27T04:10:08.713 INFO:tasks.workunit.client.1.smithi181.stdout:6/554: creat d7/d31/d45/d72/d98/dbf/fc1 x:0 0 0 2024-04-27T04:10:08.714 INFO:tasks.workunit.client.1.smithi181.stdout:6/555: write d7/dc/d1c/f99 [338276,125784] 0 2024-04-27T04:10:08.715 INFO:tasks.workunit.client.1.smithi181.stdout:6/556: chown d7/dc/d1c/fa8 1404830 1 2024-04-27T04:10:08.719 INFO:tasks.workunit.client.1.smithi181.stdout:7/653: getdents d1/dc3/daf 0 2024-04-27T04:10:08.720 INFO:tasks.workunit.client.0.smithi162.stdout:8/482: dwrite d0/d10/d25/f30 [4194304,4194304] 0 2024-04-27T04:10:08.721 INFO:tasks.workunit.client.1.smithi181.stdout:6/557: fsync d7/dc/d8b/fa7 0 2024-04-27T04:10:08.724 INFO:tasks.workunit.client.0.smithi162.stdout:7/450: unlink d3/d5/d5b/d83/f27 0 2024-04-27T04:10:08.724 INFO:tasks.workunit.client.1.smithi181.stdout:7/654: mkdir d1/d22/d15/dc6 0 2024-04-27T04:10:08.728 INFO:tasks.workunit.client.1.smithi181.stdout:6/558: fsync d7/dc/d1e/f51 0 2024-04-27T04:10:08.728 INFO:tasks.workunit.client.0.smithi162.stdout:8/483: dwrite d0/d10/d2b/d2d/d2f/d42/f48 [0,4194304] 0 2024-04-27T04:10:08.729 INFO:tasks.workunit.client.0.smithi162.stdout:7/451: chown d3/d5/d5b/d83/de/d2b/c51 1250707 1 2024-04-27T04:10:08.730 INFO:tasks.workunit.client.1.smithi181.stdout:7/655: creat d1/d2/d1c/d2d/d61/d80/fc7 x:0 0 0 2024-04-27T04:10:08.734 INFO:tasks.workunit.client.0.smithi162.stdout:7/452: fdatasync d3/d4f/f41 0 2024-04-27T04:10:08.735 INFO:tasks.workunit.client.1.smithi181.stdout:6/559: creat d7/d27/fc2 x:0 0 0 2024-04-27T04:10:08.736 INFO:tasks.workunit.client.0.smithi162.stdout:7/453: chown d3/d4f/f31 3244 1 2024-04-27T04:10:08.737 INFO:tasks.workunit.client.1.smithi181.stdout:6/560: write d7/dc/d1c/d34/f84 [4718565,125535] 0 2024-04-27T04:10:08.738 INFO:tasks.workunit.client.0.smithi162.stdout:5/499: dwrite d8/d9/d55/fa5 [4194304,4194304] 0 2024-04-27T04:10:08.740 INFO:tasks.workunit.client.0.smithi162.stdout:5/500: read d8/d9/f3d [2229502,116737] 0 2024-04-27T04:10:08.741 INFO:tasks.workunit.client.0.smithi162.stdout:7/454: creat d3/d5/d5b/d83/de/f98 x:0 0 0 2024-04-27T04:10:08.748 INFO:tasks.workunit.client.1.smithi181.stdout:6/561: fdatasync d7/dc/f3a 0 2024-04-27T04:10:08.750 INFO:tasks.workunit.client.1.smithi181.stdout:6/562: truncate d7/dc/d1c/f99 1195142 0 2024-04-27T04:10:08.751 INFO:tasks.workunit.client.0.smithi162.stdout:5/501: fdatasync d8/d1a/d29/d4a/d9f/f73 0 2024-04-27T04:10:08.751 INFO:tasks.workunit.client.1.smithi181.stdout:9/598: write d0/f23 [1277722,39339] 0 2024-04-27T04:10:08.755 INFO:tasks.workunit.client.1.smithi181.stdout:7/656: creat d1/fc8 x:0 0 0 2024-04-27T04:10:08.756 INFO:tasks.workunit.client.1.smithi181.stdout:0/772: write dc/d1f/d3b/d4a/d95/d26/d41/fc2 [343484,53066] 0 2024-04-27T04:10:08.758 INFO:tasks.workunit.client.0.smithi162.stdout:2/422: truncate d0/d4/d2a/d3b/d49/d5a/d63/d6e/f7a 1671170 0 2024-04-27T04:10:08.759 INFO:tasks.workunit.client.1.smithi181.stdout:6/563: dwrite d7/d27/fc2 [0,4194304] 0 2024-04-27T04:10:08.759 INFO:tasks.workunit.client.1.smithi181.stdout:7/657: chown d1/d2/d1c/d37/f4f 20833 1 2024-04-27T04:10:08.764 INFO:tasks.workunit.client.0.smithi162.stdout:5/502: readlink d8/d1a/d29/l37 0 2024-04-27T04:10:08.765 INFO:tasks.workunit.client.0.smithi162.stdout:3/469: write d1/d37/f51 [148967,1898] 0 2024-04-27T04:10:08.767 INFO:tasks.workunit.client.0.smithi162.stdout:8/484: getdents d0/d10/d25 0 2024-04-27T04:10:08.771 INFO:tasks.workunit.client.0.smithi162.stdout:2/423: rename d0/d20/d2e to d0/d4/d2a/d3b/d49/d5a/d63/d84 0 2024-04-27T04:10:08.773 INFO:tasks.workunit.client.1.smithi181.stdout:7/658: rename d1/d22/d15/d5f/fb9 to d1/d2/d35/d67/dbf/fc9 0 2024-04-27T04:10:08.773 INFO:tasks.workunit.client.0.smithi162.stdout:3/470: dread d1/d18/d23/d34/f3c [0,4194304] 0 2024-04-27T04:10:08.776 INFO:tasks.workunit.client.0.smithi162.stdout:6/486: dwrite f2 [0,4194304] 0 2024-04-27T04:10:08.778 INFO:tasks.workunit.client.0.smithi162.stdout:6/487: fsync d6/f90 0 2024-04-27T04:10:08.779 INFO:tasks.workunit.client.0.smithi162.stdout:7/455: dread d3/f18 [0,4194304] 0 2024-04-27T04:10:08.781 INFO:tasks.workunit.client.0.smithi162.stdout:8/485: dwrite d0/de/d6e/d71/f9d [0,4194304] 0 2024-04-27T04:10:08.781 INFO:tasks.workunit.client.1.smithi181.stdout:0/773: mknod dc/d1f/d6b/db9/d106/c10d 0 2024-04-27T04:10:08.787 INFO:tasks.workunit.client.0.smithi162.stdout:3/471: dwrite d1/d18/d23/d58/d63/f86 [0,4194304] 0 2024-04-27T04:10:08.787 INFO:tasks.workunit.client.0.smithi162.stdout:1/441: truncate d0/d7/d2a/f5d 44815 0 2024-04-27T04:10:08.788 INFO:tasks.workunit.client.1.smithi181.stdout:1/677: write d1/dad/db5/d6/f67 [3282582,68595] 0 2024-04-27T04:10:08.790 INFO:tasks.workunit.client.0.smithi162.stdout:4/459: sync 2024-04-27T04:10:08.791 INFO:tasks.workunit.client.1.smithi181.stdout:0/774: dwrite dc/ff [8388608,4194304] 0 2024-04-27T04:10:08.798 INFO:tasks.workunit.client.0.smithi162.stdout:2/424: dread d0/d4/d2a/f3e [0,4194304] 0 2024-04-27T04:10:08.801 INFO:tasks.workunit.client.0.smithi162.stdout:6/488: creat d6/d1a/d34/d50/f99 x:0 0 0 2024-04-27T04:10:08.802 INFO:tasks.workunit.client.0.smithi162.stdout:9/479: dwrite d3/d11/d45/f55 [4194304,4194304] 0 2024-04-27T04:10:08.803 INFO:tasks.workunit.client.1.smithi181.stdout:1/678: chown d1/dad/db5/d6/l62 4580239 1 2024-04-27T04:10:08.803 INFO:tasks.workunit.client.0.smithi162.stdout:6/489: write d6/f62 [4893442,7535] 0 2024-04-27T04:10:08.808 INFO:tasks.workunit.client.1.smithi181.stdout:0/775: creat dc/d1f/d3b/d4a/d95/d26/d41/d6f/d7d/f10e x:0 0 0 2024-04-27T04:10:08.815 INFO:tasks.workunit.client.1.smithi181.stdout:1/679: symlink d1/dad/db5/ldf 0 2024-04-27T04:10:08.817 INFO:tasks.workunit.client.0.smithi162.stdout:1/442: unlink d0/d11/d50/l73 0 2024-04-27T04:10:08.819 INFO:tasks.workunit.client.1.smithi181.stdout:2/719: getdents d4/d9/d4f/d79/d98 0 2024-04-27T04:10:08.821 INFO:tasks.workunit.client.1.smithi181.stdout:1/680: creat d1/dad/db5/d6/da3/d1e/d20/d53/d56/dd9/fe0 x:0 0 0 2024-04-27T04:10:08.825 INFO:tasks.workunit.client.0.smithi162.stdout:2/425: creat d0/d4/d17/d4f/f85 x:0 0 0 2024-04-27T04:10:08.826 INFO:tasks.workunit.client.1.smithi181.stdout:3/611: write d0/de/d3f/f5f [212272,19742] 0 2024-04-27T04:10:08.826 INFO:tasks.workunit.client.1.smithi181.stdout:1/681: creat d1/dad/db5/d6/da3/d1e/d48/da6/fe1 x:0 0 0 2024-04-27T04:10:08.828 INFO:tasks.workunit.client.1.smithi181.stdout:1/682: chown d1/dad/db5/d6/da3/d1e/d48/f5d 23477795 1 2024-04-27T04:10:08.829 INFO:tasks.workunit.client.0.smithi162.stdout:6/490: mknod d6/d1a/d34/d43/c9a 0 2024-04-27T04:10:08.831 INFO:tasks.workunit.client.1.smithi181.stdout:3/612: mknod d0/de/d7d/cc0 0 2024-04-27T04:10:08.832 INFO:tasks.workunit.client.0.smithi162.stdout:7/456: creat d3/d5/d5b/d83/de/d2b/d81/d96/f99 x:0 0 0 2024-04-27T04:10:08.835 INFO:tasks.workunit.client.0.smithi162.stdout:3/472: mknod d1/c9b 0 2024-04-27T04:10:08.836 INFO:tasks.workunit.client.0.smithi162.stdout:3/473: fdatasync d1/d70/f8c 0 2024-04-27T04:10:08.837 INFO:tasks.workunit.client.1.smithi181.stdout:2/720: rmdir d4/d9/d4f/d5f/dbb 0 2024-04-27T04:10:08.837 INFO:tasks.workunit.client.1.smithi181.stdout:3/613: dread d0/de/d20/f46 [0,4194304] 0 2024-04-27T04:10:08.841 INFO:tasks.workunit.client.0.smithi162.stdout:3/474: dwrite d1/d18/f1a [0,4194304] 0 2024-04-27T04:10:08.842 INFO:tasks.workunit.client.1.smithi181.stdout:3/614: creat d0/d6c/d52/d44/d74/fc1 x:0 0 0 2024-04-27T04:10:08.844 INFO:tasks.workunit.client.1.smithi181.stdout:2/721: dwrite d4/d9/df/d14/f78 [0,4194304] 0 2024-04-27T04:10:08.845 INFO:tasks.workunit.client.0.smithi162.stdout:8/486: rename d0/d10/d25/d7d/f61 to d0/fa7 0 2024-04-27T04:10:08.847 INFO:tasks.workunit.client.0.smithi162.stdout:0/541: write d5/fa [4188239,35125] 0 2024-04-27T04:10:08.851 INFO:tasks.workunit.client.1.smithi181.stdout:3/615: truncate d0/d28/f42 2201302 0 2024-04-27T04:10:08.851 INFO:tasks.workunit.client.0.smithi162.stdout:8/487: dwrite d0/d10/d25/f36 [0,4194304] 0 2024-04-27T04:10:08.853 INFO:tasks.workunit.client.1.smithi181.stdout:2/722: dread d4/d81/f84 [0,4194304] 0 2024-04-27T04:10:08.856 INFO:tasks.workunit.client.0.smithi162.stdout:4/460: link d2/d25/f8 d2/d25/d1c/d39/d3b/f94 0 2024-04-27T04:10:08.856 INFO:tasks.workunit.client.1.smithi181.stdout:5/812: dwrite d4/de/d2c/d46/d4d/d8e/f41 [0,4194304] 0 2024-04-27T04:10:08.857 INFO:tasks.workunit.client.0.smithi162.stdout:3/475: read - d1/f89 zero size 2024-04-27T04:10:08.858 INFO:tasks.workunit.client.1.smithi181.stdout:2/723: write d4/d9/d16/d82/fa5 [1413376,8577] 0 2024-04-27T04:10:08.872 INFO:tasks.workunit.client.1.smithi181.stdout:5/813: mknod d4/da9/c11f 0 2024-04-27T04:10:08.875 INFO:tasks.workunit.client.1.smithi181.stdout:2/724: rmdir d4/d9/df/d14 39 2024-04-27T04:10:08.876 INFO:tasks.workunit.client.0.smithi162.stdout:8/488: stat d0/d10/d25/d65/la6 0 2024-04-27T04:10:08.879 INFO:tasks.workunit.client.1.smithi181.stdout:3/616: dread d0/de/d20/f62 [0,4194304] 0 2024-04-27T04:10:08.882 INFO:tasks.workunit.client.0.smithi162.stdout:3/476: creat d1/d45/f9c x:0 0 0 2024-04-27T04:10:08.883 INFO:tasks.workunit.client.1.smithi181.stdout:5/814: mknod d4/de/d2c/d46/d4d/d8e/d67/d71/d10f/c120 0 2024-04-27T04:10:08.883 INFO:tasks.workunit.client.0.smithi162.stdout:3/477: read d1/d18/d5d/d5f/f76 [7011723,50730] 0 2024-04-27T04:10:08.884 INFO:tasks.workunit.client.1.smithi181.stdout:5/815: stat d4/d10/dec/lf8 0 2024-04-27T04:10:08.885 INFO:tasks.workunit.client.0.smithi162.stdout:9/480: getdents d3/d13/d2d/d3b 0 2024-04-27T04:10:08.887 INFO:tasks.workunit.client.1.smithi181.stdout:9/599: dwrite d0/d4d/d85/f8e [0,4194304] 0 2024-04-27T04:10:08.889 INFO:tasks.workunit.client.0.smithi162.stdout:6/491: rename d6/d1a/d34/d43/l69 to d6/d1a/d34/d50/l9b 0 2024-04-27T04:10:08.889 INFO:tasks.workunit.client.0.smithi162.stdout:5/503: write d8/d9/f3d [2374666,50646] 0 2024-04-27T04:10:08.894 INFO:tasks.workunit.client.0.smithi162.stdout:0/542: mknod d5/d26/da2/cab 0 2024-04-27T04:10:08.897 INFO:tasks.workunit.client.0.smithi162.stdout:8/489: symlink d0/d10/d2b/d2d/d2f/d6b/la8 0 2024-04-27T04:10:08.904 INFO:tasks.workunit.client.0.smithi162.stdout:8/490: dwrite d0/d10/d2b/d2d/d2f/f54 [0,4194304] 0 2024-04-27T04:10:08.906 INFO:tasks.workunit.client.0.smithi162.stdout:8/491: chown d0/d10/d2b/d2d/d2f/d42/f48 1 1 2024-04-27T04:10:08.908 INFO:tasks.workunit.client.0.smithi162.stdout:3/478: creat d1/d18/d23/d58/f9d x:0 0 0 2024-04-27T04:10:08.911 INFO:tasks.workunit.client.1.smithi181.stdout:0/776: write dc/d1f/d3b/d4a/d95/d6d/fc4 [216791,120994] 0 2024-04-27T04:10:08.912 INFO:tasks.workunit.client.1.smithi181.stdout:0/777: stat dc/d1f/d3b/d4a/d95/d26/d41/d6f/f7f 0 2024-04-27T04:10:08.913 INFO:tasks.workunit.client.0.smithi162.stdout:6/492: dread d6/f35 [0,4194304] 0 2024-04-27T04:10:08.918 INFO:tasks.workunit.client.1.smithi181.stdout:3/617: rename d0/de/d20/d6b/d91 to d0/d6c/d66/dc2 0 2024-04-27T04:10:08.920 INFO:tasks.workunit.client.1.smithi181.stdout:1/683: write d1/f2f [7794515,67056] 0 2024-04-27T04:10:08.921 INFO:tasks.workunit.client.1.smithi181.stdout:1/684: write d1/dad/db5/fb8 [3971592,71996] 0 2024-04-27T04:10:08.922 INFO:tasks.workunit.client.0.smithi162.stdout:5/504: dwrite d8/d1a/f59 [0,4194304] 0 2024-04-27T04:10:08.926 INFO:tasks.workunit.client.1.smithi181.stdout:5/816: rename d4/da9/d100/l62 to d4/da9/d100/d55/l121 0 2024-04-27T04:10:08.929 INFO:tasks.workunit.client.1.smithi181.stdout:1/685: dwrite d1/dad/db5/d6/da3/d1e/f8f [4194304,4194304] 0 2024-04-27T04:10:08.931 INFO:tasks.workunit.client.0.smithi162.stdout:1/443: dwrite d0/d11/f4d [0,4194304] 0 2024-04-27T04:10:08.931 INFO:tasks.workunit.client.0.smithi162.stdout:2/426: sync 2024-04-27T04:10:08.934 INFO:tasks.workunit.client.1.smithi181.stdout:3/618: dread d0/d6c/f60 [4194304,4194304] 0 2024-04-27T04:10:08.935 INFO:tasks.workunit.client.1.smithi181.stdout:3/619: read d0/de/ff [388720,129796] 0 2024-04-27T04:10:08.936 INFO:tasks.workunit.client.1.smithi181.stdout:0/778: mknod dc/d1f/d6b/db9/c10f 0 2024-04-27T04:10:08.940 INFO:tasks.workunit.client.0.smithi162.stdout:2/427: dwrite d0/d24/f4d [0,4194304] 0 2024-04-27T04:10:08.942 INFO:tasks.workunit.client.0.smithi162.stdout:2/428: fdatasync d0/d4/d2a/d41/f81 0 2024-04-27T04:10:08.945 INFO:tasks.workunit.client.1.smithi181.stdout:3/620: dwrite d0/d6c/d66/d99/fa3 [0,4194304] 0 2024-04-27T04:10:08.948 INFO:tasks.workunit.client.1.smithi181.stdout:2/725: link d4/d9/df/c5e d4/d9/d16/d82/d7d/db7/dcb/da4/dc2/ce9 0 2024-04-27T04:10:08.953 INFO:tasks.workunit.client.0.smithi162.stdout:4/461: getdents d2/d25/d1c/d39 0 2024-04-27T04:10:08.953 INFO:tasks.workunit.client.0.smithi162.stdout:4/462: chown d2/d25/c62 36098418 1 2024-04-27T04:10:08.957 INFO:tasks.workunit.client.1.smithi181.stdout:0/779: mkdir dc/d1f/d6b/db9/d106/d110 0 2024-04-27T04:10:08.958 INFO:tasks.workunit.client.0.smithi162.stdout:8/492: truncate d0/d10/f9a 2018045 0 2024-04-27T04:10:08.963 INFO:tasks.workunit.client.1.smithi181.stdout:3/621: creat d0/d6c/da1/daf/fc3 x:0 0 0 2024-04-27T04:10:08.964 INFO:tasks.workunit.client.0.smithi162.stdout:1/444: readlink d0/l3 0 2024-04-27T04:10:08.968 INFO:tasks.workunit.client.1.smithi181.stdout:1/686: mkdir d1/dad/db5/d6/da3/d1e/d20/d53/d56/de2 0 2024-04-27T04:10:08.970 INFO:tasks.workunit.client.1.smithi181.stdout:9/600: dwrite d0/d44/d4f/f58 [0,4194304] 0 2024-04-27T04:10:08.970 INFO:tasks.workunit.client.1.smithi181.stdout:1/687: write d1/dad/db5/d6/da3/d1e/d48/da6/fe1 [97105,97996] 0 2024-04-27T04:10:08.971 INFO:tasks.workunit.client.1.smithi181.stdout:8/726: sync 2024-04-27T04:10:08.975 INFO:tasks.workunit.client.1.smithi181.stdout:0/780: mkdir dc/d1f/d3b/db2/d111 0 2024-04-27T04:10:08.977 INFO:tasks.workunit.client.0.smithi162.stdout:6/493: fsync d6/d1a/f18 0 2024-04-27T04:10:08.979 INFO:tasks.workunit.client.1.smithi181.stdout:5/817: rename d4/da9 to d4/d10/d29/d122 0 2024-04-27T04:10:08.980 INFO:tasks.workunit.client.1.smithi181.stdout:3/622: rmdir d0/d28 39 2024-04-27T04:10:08.984 INFO:tasks.workunit.client.0.smithi162.stdout:0/543: creat d5/fac x:0 0 0 2024-04-27T04:10:08.985 INFO:tasks.workunit.client.1.smithi181.stdout:9/601: rmdir d0/d40 39 2024-04-27T04:10:08.986 INFO:tasks.workunit.client.1.smithi181.stdout:9/602: chown d0/d44/d4b/d43/d88/da1/f6d 0 1 2024-04-27T04:10:08.989 INFO:tasks.workunit.client.1.smithi181.stdout:8/727: unlink d1/d6/da6/la9 0 2024-04-27T04:10:08.990 INFO:tasks.workunit.client.1.smithi181.stdout:8/728: write d1/d6/d1a/d25/f99 [8245866,57944] 0 2024-04-27T04:10:08.991 INFO:tasks.workunit.client.1.smithi181.stdout:8/729: chown d1/d6/d1a/f53 0 1 2024-04-27T04:10:08.991 INFO:tasks.workunit.client.1.smithi181.stdout:1/688: creat d1/dad/db5/d6/da3/d1e/d48/da6/fe3 x:0 0 0 2024-04-27T04:10:08.992 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:10:08 smithi162 ceph-mon[26841]: pgmap v4: 65 pgs: 65 active+clean; 2.4 GiB data, 8.6 GiB used, 528 GiB / 536 GiB avail 2024-04-27T04:10:08.993 INFO:tasks.workunit.client.1.smithi181.stdout:0/781: mknod dc/d1f/d3b/d4a/d95/d26/d41/d6f/d7d/c112 0 2024-04-27T04:10:08.994 INFO:tasks.workunit.client.0.smithi162.stdout:4/463: dwrite d2/d25/d1c/d39/d3b/d47/f6c [0,4194304] 0 2024-04-27T04:10:08.995 INFO:tasks.workunit.client.0.smithi162.stdout:7/457: sync 2024-04-27T04:10:09.001 INFO:tasks.workunit.client.1.smithi181.stdout:0/782: dwrite dc/d1f/d3b/f77 [0,4194304] 0 2024-04-27T04:10:09.002 INFO:tasks.workunit.client.0.smithi162.stdout:1/445: rmdir d0/d11/d3e/d5f 39 2024-04-27T04:10:09.004 INFO:tasks.workunit.client.1.smithi181.stdout:9/603: chown d0/d40 73193928 1 2024-04-27T04:10:09.005 INFO:tasks.workunit.client.0.smithi162.stdout:7/458: dwrite d3/d5/d3c/f7e [0,4194304] 0 2024-04-27T04:10:09.006 INFO:tasks.workunit.client.1.smithi181.stdout:2/726: dwrite d4/d9/d4f/d79/d98/f99 [0,4194304] 0 2024-04-27T04:10:09.007 INFO:tasks.workunit.client.0.smithi162.stdout:7/459: chown d3/d5/d5b/d83/f19 5566 1 2024-04-27T04:10:09.008 INFO:tasks.workunit.client.0.smithi162.stdout:3/479: dwrite d1/d18/f2d [0,4194304] 0 2024-04-27T04:10:09.008 INFO:tasks.workunit.client.0.smithi162.stdout:9/481: getdents d3/d23/d1e 0 2024-04-27T04:10:09.012 INFO:tasks.workunit.client.0.smithi162.stdout:3/480: write d1/d18/d23/f28 [4332862,60557] 0 2024-04-27T04:10:09.012 INFO:tasks.workunit.client.0.smithi162.stdout:7/460: write d3/d5/d5b/d60/d65/d80/f84 [961615,81520] 0 2024-04-27T04:10:09.018 INFO:tasks.workunit.client.0.smithi162.stdout:6/494: fsync d6/d1a/d34/d43/f44 0 2024-04-27T04:10:09.019 INFO:tasks.workunit.client.0.smithi162.stdout:0/544: dread d5/dc/f91 [0,4194304] 0 2024-04-27T04:10:09.023 INFO:tasks.workunit.client.0.smithi162.stdout:2/429: unlink d0/lb 0 2024-04-27T04:10:09.024 INFO:tasks.workunit.client.0.smithi162.stdout:2/430: stat d0/d4/d2a/d41/f81 0 2024-04-27T04:10:09.025 INFO:tasks.workunit.client.0.smithi162.stdout:2/431: write d0/d4/d2a/d41/d77/f7c [415963,54085] 0 2024-04-27T04:10:09.026 INFO:tasks.workunit.client.1.smithi181.stdout:3/623: symlink d0/d28/d86/lc4 0 2024-04-27T04:10:09.026 INFO:tasks.workunit.client.0.smithi162.stdout:2/432: readlink d0/d4/d2a/l58 0 2024-04-27T04:10:09.033 INFO:tasks.workunit.client.0.smithi162.stdout:4/464: mkdir d2/d54/d7a/d95 0 2024-04-27T04:10:09.034 INFO:tasks.workunit.client.1.smithi181.stdout:9/604: rmdir d0/d44/d4b/d43/d89 39 2024-04-27T04:10:09.038 INFO:tasks.workunit.client.1.smithi181.stdout:8/730: write d1/d6/d2f/d47/f8e [30394,4984] 0 2024-04-27T04:10:09.039 INFO:tasks.workunit.client.0.smithi162.stdout:3/481: dread d1/d21/d47/d57/f5e [0,4194304] 0 2024-04-27T04:10:09.040 INFO:tasks.workunit.client.1.smithi181.stdout:8/731: write d1/d6/d2f/d30/d66/dc0/dcd/ff6 [506243,118052] 0 2024-04-27T04:10:09.040 INFO:tasks.workunit.client.0.smithi162.stdout:1/446: rmdir d0/d11/d50/d53 39 2024-04-27T04:10:09.041 INFO:tasks.workunit.client.1.smithi181.stdout:8/732: write d1/d6/d2f/d47/fac [4284333,66720] 0 2024-04-27T04:10:09.042 INFO:tasks.workunit.client.1.smithi181.stdout:1/689: fdatasync d1/dad/db5/d6/da3/d1e/f4b 0 2024-04-27T04:10:09.043 INFO:tasks.workunit.client.1.smithi181.stdout:0/783: dread dc/d54/f72 [0,4194304] 0 2024-04-27T04:10:09.045 INFO:tasks.workunit.client.1.smithi181.stdout:5/818: creat d4/da3/dbc/f123 x:0 0 0 2024-04-27T04:10:09.047 INFO:tasks.workunit.client.1.smithi181.stdout:4/630: sync 2024-04-27T04:10:09.048 INFO:tasks.workunit.client.1.smithi181.stdout:3/624: mkdir d0/d6c/da1/dc5 0 2024-04-27T04:10:09.050 INFO:tasks.workunit.client.1.smithi181.stdout:9/605: mkdir d0/d44/d4b/d3d/dbd/dd1 0 2024-04-27T04:10:09.055 INFO:tasks.workunit.client.1.smithi181.stdout:2/727: read d4/d9/d35/f3a [1185672,3557] 0 2024-04-27T04:10:09.056 INFO:tasks.workunit.client.0.smithi162.stdout:9/482: dread d3/d13/d15/f19 [4194304,4194304] 0 2024-04-27T04:10:09.057 INFO:tasks.workunit.client.0.smithi162.stdout:9/483: truncate d3/d23/f88 1669539 0 2024-04-27T04:10:09.057 INFO:tasks.workunit.client.0.smithi162.stdout:9/484: chown d3/d13/d2d/d3b/l85 276766 1 2024-04-27T04:10:09.059 INFO:tasks.workunit.client.0.smithi162.stdout:5/505: getdents d8/d9/d5a 0 2024-04-27T04:10:09.059 INFO:tasks.workunit.client.0.smithi162.stdout:9/485: read d3/d13/d2d/d3b/d54/f87 [3430825,71061] 0 2024-04-27T04:10:09.060 INFO:tasks.workunit.client.0.smithi162.stdout:5/506: dread - d8/d9/d55/f84 zero size 2024-04-27T04:10:09.060 INFO:tasks.workunit.client.1.smithi181.stdout:1/690: creat d1/dad/db5/d6/db/d2d/d36/fe4 x:0 0 0 2024-04-27T04:10:09.062 INFO:tasks.workunit.client.1.smithi181.stdout:1/691: chown d1/dad/db5/d6/da3/d52/l8a 1 1 2024-04-27T04:10:09.063 INFO:tasks.workunit.client.1.smithi181.stdout:0/784: rename dc/f2c to dc/d1f/d6b/db9/dbd/f113 0 2024-04-27T04:10:09.063 INFO:tasks.workunit.client.1.smithi181.stdout:0/785: write fa [305798,36475] 0 2024-04-27T04:10:09.064 INFO:tasks.workunit.client.0.smithi162.stdout:4/465: mknod d2/d25/d1c/d39/d5e/c96 0 2024-04-27T04:10:09.064 INFO:tasks.workunit.client.1.smithi181.stdout:0/786: chown dc/d54/l76 0 1 2024-04-27T04:10:09.065 INFO:tasks.workunit.client.0.smithi162.stdout:4/466: readlink d2/l67 0 2024-04-27T04:10:09.065 INFO:tasks.workunit.client.1.smithi181.stdout:5/819: mkdir d4/da3/dbc/db5/d124 0 2024-04-27T04:10:09.066 INFO:tasks.workunit.client.0.smithi162.stdout:4/467: truncate d2/da/dd/f93 1000234 0 2024-04-27T04:10:09.068 INFO:tasks.workunit.client.1.smithi181.stdout:4/631: creat d0/d15/d76/db1/fd0 x:0 0 0 2024-04-27T04:10:09.069 INFO:tasks.workunit.client.1.smithi181.stdout:4/632: fsync d0/d15/d25/d40/f9b 0 2024-04-27T04:10:09.070 INFO:tasks.workunit.client.1.smithi181.stdout:8/733: dread d1/f48 [0,4194304] 0 2024-04-27T04:10:09.073 INFO:tasks.workunit.client.0.smithi162.stdout:1/447: chown d0/d11/d3e/d5f 882 1 2024-04-27T04:10:09.078 INFO:tasks.workunit.client.0.smithi162.stdout:7/461: symlink d3/d5/d5b/l9a 0 2024-04-27T04:10:09.079 INFO:tasks.workunit.client.0.smithi162.stdout:8/493: truncate d0/d10/d2b/d2e/f46 4170472 0 2024-04-27T04:10:09.083 INFO:tasks.workunit.client.0.smithi162.stdout:6/495: mknod d6/d1a/d34/d7f/d80/c9c 0 2024-04-27T04:10:09.084 INFO:tasks.workunit.client.1.smithi181.stdout:0/787: symlink dc/d1f/d3b/l114 0 2024-04-27T04:10:09.085 INFO:tasks.workunit.client.0.smithi162.stdout:8/494: dwrite d0/d10/d25/d7d/f8b [0,4194304] 0 2024-04-27T04:10:09.085 INFO:tasks.workunit.client.0.smithi162.stdout:6/496: write d6/d1a/d34/f3e [1264012,70117] 0 2024-04-27T04:10:09.086 INFO:tasks.workunit.client.1.smithi181.stdout:9/606: dwrite d0/d4d/fa5 [0,4194304] 0 2024-04-27T04:10:09.087 INFO:tasks.workunit.client.1.smithi181.stdout:3/625: dread d0/d6c/d52/d44/d56/d5b/f8e [0,4194304] 0 2024-04-27T04:10:09.087 INFO:tasks.workunit.client.1.smithi181.stdout:5/820: creat d4/d10/d72/d80/f125 x:0 0 0 2024-04-27T04:10:09.088 INFO:tasks.workunit.client.1.smithi181.stdout:5/821: dread - d4/da3/dbc/ff3 zero size 2024-04-27T04:10:09.088 INFO:tasks.workunit.client.0.smithi162.stdout:6/497: read d6/f1e [3849776,45046] 0 2024-04-27T04:10:09.090 INFO:tasks.workunit.client.0.smithi162.stdout:6/498: write d6/d1a/d34/d50/f63 [360773,105348] 0 2024-04-27T04:10:09.091 INFO:tasks.workunit.client.0.smithi162.stdout:6/499: stat d6/db/l95 0 2024-04-27T04:10:09.096 INFO:tasks.workunit.client.1.smithi181.stdout:6/564: sync 2024-04-27T04:10:09.098 INFO:tasks.workunit.client.1.smithi181.stdout:7/659: sync 2024-04-27T04:10:09.100 INFO:tasks.workunit.client.0.smithi162.stdout:1/448: mknod d0/d7/d3c/d77/c8e 0 2024-04-27T04:10:09.101 INFO:tasks.workunit.client.0.smithi162.stdout:1/449: write d0/d7/d10/f4c [7196801,108877] 0 2024-04-27T04:10:09.103 INFO:tasks.workunit.client.1.smithi181.stdout:9/607: unlink d0/d44/f4a 0 2024-04-27T04:10:09.106 INFO:tasks.workunit.client.1.smithi181.stdout:3/626: mkdir d0/de/d20/d58/d64/dc6 0 2024-04-27T04:10:09.106 INFO:tasks.workunit.client.1.smithi181.stdout:2/728: dwrite d4/d9/d16/d82/d7d/dbe/fca [0,4194304] 0 2024-04-27T04:10:09.107 INFO:tasks.workunit.client.0.smithi162.stdout:1/450: dwrite d0/d11/d44/d66/f78 [0,4194304] 0 2024-04-27T04:10:09.115 INFO:tasks.workunit.client.0.smithi162.stdout:2/433: write d0/d4/d2a/d3b/d49/d5a/d63/d6c/f76 [730922,71848] 0 2024-04-27T04:10:09.123 INFO:tasks.workunit.client.0.smithi162.stdout:9/486: write d3/d11/f5e [4513312,112900] 0 2024-04-27T04:10:09.124 INFO:tasks.workunit.client.0.smithi162.stdout:9/487: chown d3/d13/d2d/d3b/d54/f8d 6 1 2024-04-27T04:10:09.129 INFO:tasks.workunit.client.1.smithi181.stdout:4/633: dwrite d0/d15/de/d19/d77/d57/f9f [0,4194304] 0 2024-04-27T04:10:09.131 INFO:tasks.workunit.client.1.smithi181.stdout:3/627: dread d0/de/d3f/f5f [0,4194304] 0 2024-04-27T04:10:09.132 INFO:tasks.workunit.client.1.smithi181.stdout:4/634: dread - d0/d15/d25/d2b/d6d/d82/fc3 zero size 2024-04-27T04:10:09.132 INFO:tasks.workunit.client.1.smithi181.stdout:7/660: dread d1/df/d1a/fc5 [0,4194304] 0 2024-04-27T04:10:09.134 INFO:tasks.workunit.client.1.smithi181.stdout:7/661: fsync d1/d22/d15/d29/f84 0 2024-04-27T04:10:09.135 INFO:tasks.workunit.client.0.smithi162.stdout:4/468: creat d2/d25/d1c/d39/d70/d91/f97 x:0 0 0 2024-04-27T04:10:09.137 INFO:tasks.workunit.client.0.smithi162.stdout:4/469: write d2/d25/d1c/d39/f45 [4833813,101168] 0 2024-04-27T04:10:09.137 INFO:tasks.workunit.client.1.smithi181.stdout:2/729: creat d4/d9/d16/d82/d7d/db7/fea x:0 0 0 2024-04-27T04:10:09.139 INFO:tasks.workunit.client.1.smithi181.stdout:2/730: chown d4/d9/d16/d45/d9a/d9d/le3 18450 1 2024-04-27T04:10:09.139 INFO:tasks.workunit.client.1.smithi181.stdout:7/662: dread d1/d22/d15/f2a [0,4194304] 0 2024-04-27T04:10:09.142 INFO:tasks.workunit.client.1.smithi181.stdout:6/565: write d7/dc/d1c/f35 [2912818,65826] 0 2024-04-27T04:10:09.143 INFO:tasks.workunit.client.1.smithi181.stdout:6/566: stat d7/dc/d1c/f99 0 2024-04-27T04:10:09.143 INFO:tasks.workunit.client.0.smithi162.stdout:7/462: symlink d3/l9b 0 2024-04-27T04:10:09.145 INFO:tasks.workunit.client.0.smithi162.stdout:5/507: truncate d8/d1a/f31 2650065 0 2024-04-27T04:10:09.146 INFO:tasks.workunit.client.1.smithi181.stdout:2/731: dread d4/d9/d16/d82/d7d/db7/dcb/f97 [0,4194304] 0 2024-04-27T04:10:09.147 INFO:tasks.workunit.client.0.smithi162.stdout:0/545: getdents d5/d60 0 2024-04-27T04:10:09.147 INFO:tasks.workunit.client.0.smithi162.stdout:7/463: read d3/d5/d5b/d83/de/f38 [234787,67992] 0 2024-04-27T04:10:09.147 INFO:tasks.workunit.client.1.smithi181.stdout:0/788: dread dc/d1f/d3b/dd0/f100 [0,4194304] 0 2024-04-27T04:10:09.148 INFO:tasks.workunit.client.0.smithi162.stdout:7/464: write d3/d4f/f41 [3245665,11301] 0 2024-04-27T04:10:09.149 INFO:tasks.workunit.client.1.smithi181.stdout:1/692: getdents d1/dad/db5/d6/da3/d52 0 2024-04-27T04:10:09.151 INFO:tasks.workunit.client.0.smithi162.stdout:8/495: unlink d0/d10/l66 0 2024-04-27T04:10:09.152 INFO:tasks.workunit.client.0.smithi162.stdout:8/496: truncate d0/de/d6e/f77 119126 0 2024-04-27T04:10:09.154 INFO:tasks.workunit.client.1.smithi181.stdout:9/608: symlink d0/d44/ld2 0 2024-04-27T04:10:09.157 INFO:tasks.workunit.client.1.smithi181.stdout:3/628: creat d0/d6c/d52/dbb/fc7 x:0 0 0 2024-04-27T04:10:09.158 INFO:tasks.workunit.client.0.smithi162.stdout:6/500: creat d6/d1a/d34/d7b/d7e/d97/f9d x:0 0 0 2024-04-27T04:10:09.161 INFO:tasks.workunit.client.0.smithi162.stdout:9/488: creat d3/d13/d2d/d3b/d54/f8f x:0 0 0 2024-04-27T04:10:09.161 INFO:tasks.workunit.client.0.smithi162.stdout:9/489: readlink d3/d39/l53 0 2024-04-27T04:10:09.162 INFO:tasks.workunit.client.1.smithi181.stdout:4/635: fsync d0/d15/de/d19/d77/f72 0 2024-04-27T04:10:09.169 INFO:tasks.workunit.client.0.smithi162.stdout:3/482: link d1/d18/f11 d1/d21/d47/d57/d6e/f9e 0 2024-04-27T04:10:09.171 INFO:tasks.workunit.client.1.smithi181.stdout:5/822: link d4/d10/l17 d4/d10/d29/d122/de7/l126 0 2024-04-27T04:10:09.172 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:10:08 smithi181 ceph-mon[33774]: pgmap v4: 65 pgs: 65 active+clean; 2.4 GiB data, 8.6 GiB used, 528 GiB / 536 GiB avail 2024-04-27T04:10:09.175 INFO:tasks.workunit.client.0.smithi162.stdout:1/451: creat d0/d80/d89/f8f x:0 0 0 2024-04-27T04:10:09.175 INFO:tasks.workunit.client.0.smithi162.stdout:3/483: dwrite d1/d21/d47/d57/f64 [0,4194304] 0 2024-04-27T04:10:09.176 INFO:tasks.workunit.client.1.smithi181.stdout:2/732: creat d4/d9/d16/d82/feb x:0 0 0 2024-04-27T04:10:09.177 INFO:tasks.workunit.client.0.smithi162.stdout:3/484: fsync d1/d45/f9c 0 2024-04-27T04:10:09.178 INFO:tasks.workunit.client.0.smithi162.stdout:7/465: unlink d3/d5/d5b/d83/f35 0 2024-04-27T04:10:09.179 INFO:tasks.workunit.client.0.smithi162.stdout:1/452: write d0/d7/d2a/f47 [955541,29637] 0 2024-04-27T04:10:09.179 INFO:tasks.workunit.client.1.smithi181.stdout:8/734: rename d1/d6/d1a/c7f to d1/d6/d2f/d30/cfc 0 2024-04-27T04:10:09.179 INFO:tasks.workunit.client.1.smithi181.stdout:8/735: stat d1/d20/c5b 0 2024-04-27T04:10:09.181 INFO:tasks.workunit.client.0.smithi162.stdout:0/546: dwrite d5/d60/d8b/f99 [0,4194304] 0 2024-04-27T04:10:09.181 INFO:tasks.workunit.client.0.smithi162.stdout:8/497: creat d0/d10/d44/d70/fa9 x:0 0 0 2024-04-27T04:10:09.182 INFO:tasks.workunit.client.0.smithi162.stdout:0/547: chown d5/d26/da2 1951 1 2024-04-27T04:10:09.183 INFO:tasks.workunit.client.1.smithi181.stdout:1/693: readlink d1/dad/db5/d6/da3/d1e/lb7 0 2024-04-27T04:10:09.186 INFO:tasks.workunit.client.0.smithi162.stdout:6/501: unlink d6/d1a/d34/d3a/d68/d6c/d67/f89 0 2024-04-27T04:10:09.186 INFO:tasks.workunit.client.0.smithi162.stdout:7/466: dread d3/d5/d5b/d83/de/f30 [4194304,4194304] 0 2024-04-27T04:10:09.187 INFO:tasks.workunit.client.1.smithi181.stdout:9/609: dread - d0/fa7 zero size 2024-04-27T04:10:09.189 INFO:tasks.workunit.client.1.smithi181.stdout:3/629: rmdir d0/de 39 2024-04-27T04:10:09.196 INFO:tasks.workunit.client.1.smithi181.stdout:0/789: write f8 [4351372,101678] 0 2024-04-27T04:10:09.213 INFO:tasks.workunit.client.0.smithi162.stdout:4/470: dwrite d2/d25/d1c/d24/f3a [0,4194304] 0 2024-04-27T04:10:09.216 INFO:tasks.workunit.client.0.smithi162.stdout:1/453: rename d0/fb to d0/d11/d44/d66/f90 0 2024-04-27T04:10:09.217 INFO:tasks.workunit.client.1.smithi181.stdout:6/567: truncate d7/dc/d1c/d7f/f29 536481 0 2024-04-27T04:10:09.219 INFO:tasks.workunit.client.1.smithi181.stdout:8/736: rmdir d1/d6/d6f 39 2024-04-27T04:10:09.223 INFO:tasks.workunit.client.0.smithi162.stdout:8/498: rmdir d0/de/d20 39 2024-04-27T04:10:09.224 INFO:tasks.workunit.client.0.smithi162.stdout:3/485: write d1/d21/f60 [2429221,63954] 0 2024-04-27T04:10:09.224 INFO:tasks.workunit.client.1.smithi181.stdout:2/733: dread d4/f8 [0,4194304] 0 2024-04-27T04:10:09.224 INFO:tasks.workunit.client.0.smithi162.stdout:8/499: write d0/de/d6e/f77 [1021211,13639] 0 2024-04-27T04:10:09.225 INFO:tasks.workunit.client.0.smithi162.stdout:2/434: mkdir d0/d4/d2a/d3b/d49/d5a/d63/d6c/d5d/d86 0 2024-04-27T04:10:09.228 INFO:tasks.workunit.client.0.smithi162.stdout:0/548: rmdir d5/dc/d1e/d46/d48/d9e 39 2024-04-27T04:10:09.229 INFO:tasks.workunit.client.0.smithi162.stdout:0/549: write d5/f2d [4511521,78248] 0 2024-04-27T04:10:09.233 INFO:tasks.workunit.client.1.smithi181.stdout:3/630: read d0/de/d20/d58/f7e [120692,80485] 0 2024-04-27T04:10:09.233 INFO:tasks.workunit.client.0.smithi162.stdout:6/502: symlink d6/d1a/d34/d3a/d60/d74/l9e 0 2024-04-27T04:10:09.234 INFO:tasks.workunit.client.1.smithi181.stdout:3/631: chown d0/de/d20/f69 0 1 2024-04-27T04:10:09.242 INFO:tasks.workunit.client.0.smithi162.stdout:4/471: symlink d2/d25/d1c/d39/d3b/d47/l98 0 2024-04-27T04:10:09.243 INFO:tasks.workunit.client.1.smithi181.stdout:4/636: truncate d0/d15/de/f91 2602228 0 2024-04-27T04:10:09.243 INFO:tasks.workunit.client.1.smithi181.stdout:4/637: fdatasync d0/d15/d25/d2a/f38 0 2024-04-27T04:10:09.245 INFO:tasks.workunit.client.0.smithi162.stdout:1/454: symlink d0/d7/l91 0 2024-04-27T04:10:09.247 INFO:tasks.workunit.client.0.smithi162.stdout:3/486: fdatasync d1/d18/f4a 0 2024-04-27T04:10:09.247 INFO:tasks.workunit.client.0.smithi162.stdout:3/487: stat d1/d18/d23/d34/l4e 0 2024-04-27T04:10:09.251 INFO:tasks.workunit.client.0.smithi162.stdout:0/550: creat d5/d60/d98/fad x:0 0 0 2024-04-27T04:10:09.255 INFO:tasks.workunit.client.0.smithi162.stdout:6/503: mknod d6/d1a/d34/d7b/d7e/c9f 0 2024-04-27T04:10:09.256 INFO:tasks.workunit.client.0.smithi162.stdout:9/490: link d3/d13/d15/d50/c65 d3/d23/d5d/c90 0 2024-04-27T04:10:09.258 INFO:tasks.workunit.client.0.smithi162.stdout:5/508: getdents d8/d1a/d23/d40/d4b 0 2024-04-27T04:10:09.259 INFO:tasks.workunit.client.0.smithi162.stdout:4/472: rmdir d2/d54 39 2024-04-27T04:10:09.260 INFO:tasks.workunit.client.0.smithi162.stdout:8/500: dwrite d0/de/d20/d6c/f6d [0,4194304] 0 2024-04-27T04:10:09.260 INFO:tasks.workunit.client.0.smithi162.stdout:3/488: dwrite d1/d18/d23/d2f/f7c [0,4194304] 0 2024-04-27T04:10:09.262 INFO:tasks.workunit.client.0.smithi162.stdout:2/435: dwrite d0/d24/f38 [0,4194304] 0 2024-04-27T04:10:09.263 INFO:tasks.workunit.client.0.smithi162.stdout:1/455: creat d0/d7/d2a/f92 x:0 0 0 2024-04-27T04:10:09.265 INFO:tasks.workunit.client.0.smithi162.stdout:5/509: read d8/d1a/d23/f28 [1408123,31866] 0 2024-04-27T04:10:09.268 INFO:tasks.workunit.client.0.smithi162.stdout:8/501: chown d0/de/d6e/f9b 493864 1 2024-04-27T04:10:09.268 INFO:tasks.workunit.client.0.smithi162.stdout:6/504: dread d6/db/f13 [0,4194304] 0 2024-04-27T04:10:09.269 INFO:tasks.workunit.client.0.smithi162.stdout:2/436: write d0/d4/d2a/d3b/d49/d5a/d63/d84/f69 [1581746,8576] 0 2024-04-27T04:10:09.270 INFO:tasks.workunit.client.0.smithi162.stdout:8/502: write d0/de/d6e/f79 [367164,114042] 0 2024-04-27T04:10:09.270 INFO:tasks.workunit.client.0.smithi162.stdout:1/456: write d0/d11/d3e/d59/d7f/f83 [726435,58672] 0 2024-04-27T04:10:09.271 INFO:tasks.workunit.client.0.smithi162.stdout:8/503: fsync d0/f4 0 2024-04-27T04:10:09.275 INFO:tasks.workunit.client.0.smithi162.stdout:5/510: dwrite d8/d9/fc [0,4194304] 0 2024-04-27T04:10:09.280 INFO:tasks.workunit.client.0.smithi162.stdout:1/457: dwrite d0/d7/d3c/d6e/f7b [0,4194304] 0 2024-04-27T04:10:09.282 INFO:tasks.workunit.client.0.smithi162.stdout:7/467: getdents d3/d39/d57 0 2024-04-27T04:10:09.283 INFO:tasks.workunit.client.0.smithi162.stdout:7/468: write d3/d4f/f67 [1084873,83852] 0 2024-04-27T04:10:09.287 INFO:tasks.workunit.client.0.smithi162.stdout:6/505: fsync d6/d1a/f18 0 2024-04-27T04:10:09.302 INFO:tasks.workunit.client.0.smithi162.stdout:9/491: dwrite d3/d13/d15/f57 [0,4194304] 0 2024-04-27T04:10:09.305 INFO:tasks.workunit.client.1.smithi181.stdout:7/663: link d1/d2/d1c/d8f/lc4 d1/d22/d15/d5f/dba/lca 0 2024-04-27T04:10:09.306 INFO:tasks.workunit.client.1.smithi181.stdout:7/664: chown d1/d2/d1c/d2d/f41 694 1 2024-04-27T04:10:09.308 INFO:tasks.workunit.client.0.smithi162.stdout:9/492: dwrite d3/d11/f5e [0,4194304] 0 2024-04-27T04:10:09.309 INFO:tasks.workunit.client.1.smithi181.stdout:8/737: creat d1/d6/d2f/d30/d66/dc0/dcd/ffd x:0 0 0 2024-04-27T04:10:09.310 INFO:tasks.workunit.client.0.smithi162.stdout:9/493: chown d3/d13/d2d/d3b/d54/l60 1 1 2024-04-27T04:10:09.310 INFO:tasks.workunit.client.0.smithi162.stdout:9/494: stat d3 0 2024-04-27T04:10:09.312 INFO:tasks.workunit.client.1.smithi181.stdout:2/734: truncate d4/d9/d4f/fac 187852 0 2024-04-27T04:10:09.312 INFO:tasks.workunit.client.0.smithi162.stdout:7/469: creat d3/d5/d5b/d83/de/d2b/d81/f9c x:0 0 0 2024-04-27T04:10:09.315 INFO:tasks.workunit.client.1.smithi181.stdout:9/610: symlink d0/d44/d4b/d3d/dbd/dd1/ld3 0 2024-04-27T04:10:09.317 INFO:tasks.workunit.client.0.smithi162.stdout:6/506: chown d6/c1b 4495 1 2024-04-27T04:10:09.318 INFO:tasks.workunit.client.0.smithi162.stdout:0/551: creat d5/dc/fae x:0 0 0 2024-04-27T04:10:09.323 INFO:tasks.workunit.client.0.smithi162.stdout:3/489: truncate d1/d18/f1a 1298947 0 2024-04-27T04:10:09.324 INFO:tasks.workunit.client.0.smithi162.stdout:8/504: mknod d0/de/d6e/caa 0 2024-04-27T04:10:09.325 INFO:tasks.workunit.client.0.smithi162.stdout:1/458: write d0/f1b [2584334,48865] 0 2024-04-27T04:10:09.326 INFO:tasks.workunit.client.1.smithi181.stdout:4/638: creat d0/d15/d25/d2a/da0/fd1 x:0 0 0 2024-04-27T04:10:09.326 INFO:tasks.workunit.client.0.smithi162.stdout:8/505: chown d0/d10/d2b/d2d/d2f/d42/fa0 152416 1 2024-04-27T04:10:09.327 INFO:tasks.workunit.client.1.smithi181.stdout:4/639: readlink d0/d15/de/d60/d6f/l8e 0 2024-04-27T04:10:09.327 INFO:tasks.workunit.client.0.smithi162.stdout:8/506: chown d0/de/d20/f60 0 1 2024-04-27T04:10:09.333 INFO:tasks.workunit.client.0.smithi162.stdout:9/495: creat d3/d23/d1e/f91 x:0 0 0 2024-04-27T04:10:09.333 INFO:tasks.workunit.client.0.smithi162.stdout:2/437: write d0/d4/f3a [4261058,109253] 0 2024-04-27T04:10:09.334 INFO:tasks.workunit.client.0.smithi162.stdout:7/470: creat d3/d4f/f9d x:0 0 0 2024-04-27T04:10:09.335 INFO:tasks.workunit.client.1.smithi181.stdout:9/611: creat d0/d44/d4b/d5f/da4/fd4 x:0 0 0 2024-04-27T04:10:09.336 INFO:tasks.workunit.client.0.smithi162.stdout:7/471: dread d3/d4f/f78 [0,4194304] 0 2024-04-27T04:10:09.336 INFO:tasks.workunit.client.0.smithi162.stdout:9/496: dread d3/d23/f5c [0,4194304] 0 2024-04-27T04:10:09.336 INFO:tasks.workunit.client.0.smithi162.stdout:6/507: fdatasync d6/db/d32/f51 0 2024-04-27T04:10:09.337 INFO:tasks.workunit.client.1.smithi181.stdout:3/632: fsync d0/f5 0 2024-04-27T04:10:09.340 INFO:tasks.workunit.client.1.smithi181.stdout:5/823: rename d4/d10/d29/d92/ce5 to d4/de/d2c/c127 0 2024-04-27T04:10:09.341 INFO:tasks.workunit.client.1.smithi181.stdout:5/824: stat d4/de/d2f/c21 0 2024-04-27T04:10:09.341 INFO:tasks.workunit.client.1.smithi181.stdout:4/640: mkdir d0/d15/dd2 0 2024-04-27T04:10:09.344 INFO:tasks.workunit.client.0.smithi162.stdout:3/490: rmdir d1/d18/d23/d58/d63 39 2024-04-27T04:10:09.350 INFO:tasks.workunit.client.0.smithi162.stdout:5/511: rmdir d8/d9/da1 0 2024-04-27T04:10:09.351 INFO:tasks.workunit.client.0.smithi162.stdout:5/512: write d8/d9/f10 [7094384,9547] 0 2024-04-27T04:10:09.353 INFO:tasks.workunit.client.1.smithi181.stdout:1/694: getdents d1/dad/db5/d6/db/d2d/d36/dc4 0 2024-04-27T04:10:09.354 INFO:tasks.workunit.client.0.smithi162.stdout:4/473: getdents d2/da 0 2024-04-27T04:10:09.358 INFO:tasks.workunit.client.0.smithi162.stdout:2/438: creat d0/d24/f87 x:0 0 0 2024-04-27T04:10:09.358 INFO:tasks.workunit.client.0.smithi162.stdout:5/513: dwrite d8/d2c/f4c [0,4194304] 0 2024-04-27T04:10:09.358 INFO:tasks.workunit.client.1.smithi181.stdout:3/633: symlink d0/de/d20/d58/lc8 0 2024-04-27T04:10:09.362 INFO:tasks.workunit.client.1.smithi181.stdout:0/790: rename dc/d1f/d6b/db9/dba/dce/dde to dc/d1f/d115 0 2024-04-27T04:10:09.368 INFO:tasks.workunit.client.0.smithi162.stdout:7/472: creat d3/d5/d5b/d83/de/d2b/d81/d96/f9e x:0 0 0 2024-04-27T04:10:09.368 INFO:tasks.workunit.client.1.smithi181.stdout:7/665: dwrite d1/d22/d15/d29/dbe/d1b/f40 [0,4194304] 0 2024-04-27T04:10:09.369 INFO:tasks.workunit.client.0.smithi162.stdout:6/508: creat d6/db/fa0 x:0 0 0 2024-04-27T04:10:09.371 INFO:tasks.workunit.client.1.smithi181.stdout:8/738: link d1/d6/d2f/d30/f38 d1/d6/d59/ffe 0 2024-04-27T04:10:09.374 INFO:tasks.workunit.client.1.smithi181.stdout:2/735: creat d4/d9/d4f/d79/fec x:0 0 0 2024-04-27T04:10:09.375 INFO:tasks.workunit.client.1.smithi181.stdout:8/739: read d1/d6/d2f/d47/f8e [3921317,74375] 0 2024-04-27T04:10:09.375 INFO:tasks.workunit.client.1.smithi181.stdout:2/736: dread - d4/d9/d4f/d79/d94/fe7 zero size 2024-04-27T04:10:09.376 INFO:tasks.workunit.client.1.smithi181.stdout:1/695: fdatasync d1/dad/db5/d6/da3/d1e/d20/d53/d56/f6f 0 2024-04-27T04:10:09.380 INFO:tasks.workunit.client.1.smithi181.stdout:1/696: read d1/dad/db5/d6/db/d2d/d36/f9a [185117,52466] 0 2024-04-27T04:10:09.382 INFO:tasks.workunit.client.1.smithi181.stdout:3/634: chown d0/d6c/c6d 143 1 2024-04-27T04:10:09.382 INFO:tasks.workunit.client.1.smithi181.stdout:3/635: chown d0/d6c/d66/fbd 1 1 2024-04-27T04:10:09.391 INFO:tasks.workunit.client.1.smithi181.stdout:8/740: fdatasync d1/d6/d2f/d47/f4b 0 2024-04-27T04:10:09.392 INFO:tasks.workunit.client.1.smithi181.stdout:8/741: chown d1/d6/d2f/d30/d66/dc0/dcd/fde 6800062 1 2024-04-27T04:10:09.393 INFO:tasks.workunit.client.1.smithi181.stdout:9/612: truncate d0/d44/d4f/f7a 2088059 0 2024-04-27T04:10:09.394 INFO:tasks.workunit.client.1.smithi181.stdout:9/613: truncate d0/d40/fcd 292153 0 2024-04-27T04:10:09.395 INFO:tasks.workunit.client.1.smithi181.stdout:2/737: unlink d4/d9/d16/d82/d7d/db7/dcb/da4/fd4 0 2024-04-27T04:10:09.398 INFO:tasks.workunit.client.1.smithi181.stdout:1/697: mkdir d1/dad/db5/d6/da3/d1e/d48/da6/da4/de5 0 2024-04-27T04:10:09.401 INFO:tasks.workunit.client.1.smithi181.stdout:3/636: unlink d0/d6c/f89 0 2024-04-27T04:10:09.402 INFO:tasks.workunit.client.1.smithi181.stdout:0/791: symlink dc/l116 0 2024-04-27T04:10:09.405 INFO:tasks.workunit.client.1.smithi181.stdout:5/825: creat d4/de/d2c/d46/d4d/d8e/d67/f128 x:0 0 0 2024-04-27T04:10:09.406 INFO:tasks.workunit.client.1.smithi181.stdout:5/826: write d4/da3/dbc/db5/fc1 [1259009,57055] 0 2024-04-27T04:10:09.408 INFO:tasks.workunit.client.1.smithi181.stdout:8/742: write d1/d20/f7a [3326732,103313] 0 2024-04-27T04:10:09.410 INFO:tasks.workunit.client.0.smithi162.stdout:3/491: creat d1/d21/d47/d57/f9f x:0 0 0 2024-04-27T04:10:09.411 INFO:tasks.workunit.client.0.smithi162.stdout:1/459: mkdir d0/d11/d50/d53/d93 0 2024-04-27T04:10:09.412 INFO:tasks.workunit.client.1.smithi181.stdout:2/738: mkdir d4/d9/d16/d82/d7d/dbe/ded 0 2024-04-27T04:10:09.413 INFO:tasks.workunit.client.0.smithi162.stdout:1/460: write d0/d11/d3e/d5f/f6a [244426,107781] 0 2024-04-27T04:10:09.414 INFO:tasks.workunit.client.0.smithi162.stdout:8/507: mknod d0/d10/d2b/cab 0 2024-04-27T04:10:09.414 INFO:tasks.workunit.client.1.smithi181.stdout:1/698: creat d1/dad/db5/d6/da3/d1e/d48/da6/da4/fe6 x:0 0 0 2024-04-27T04:10:09.415 INFO:tasks.workunit.client.0.smithi162.stdout:1/461: write d0/d11/d3e/f6d [899847,123054] 0 2024-04-27T04:10:09.417 INFO:tasks.workunit.client.0.smithi162.stdout:3/492: dwrite d1/d18/f2d [0,4194304] 0 2024-04-27T04:10:09.418 INFO:tasks.workunit.client.0.smithi162.stdout:8/508: truncate d0/d10/d2b/d2d/d2f/d42/fa0 768409 0 2024-04-27T04:10:09.418 INFO:tasks.workunit.client.1.smithi181.stdout:0/792: fdatasync dc/d1f/d3b/ff9 0 2024-04-27T04:10:09.419 INFO:tasks.workunit.client.0.smithi162.stdout:2/439: mkdir d0/d4/d17/d88 0 2024-04-27T04:10:09.421 INFO:tasks.workunit.client.0.smithi162.stdout:2/440: chown d0/d4/d2a/d3b/d49/d5a/d63/d70 11849618 1 2024-04-27T04:10:09.421 INFO:tasks.workunit.client.1.smithi181.stdout:6/568: rename d7/d31/d45 to d7/dc/dc3 0 2024-04-27T04:10:09.427 INFO:tasks.workunit.client.0.smithi162.stdout:7/473: mknod d3/d5/d5b/d83/de/c9f 0 2024-04-27T04:10:09.427 INFO:tasks.workunit.client.0.smithi162.stdout:7/474: chown d3/f40 1 1 2024-04-27T04:10:09.429 INFO:tasks.workunit.client.1.smithi181.stdout:8/743: mkdir d1/d6/d2f/d30/d9d/dea/dff 0 2024-04-27T04:10:09.432 INFO:tasks.workunit.client.0.smithi162.stdout:6/509: creat d6/d1a/d34/d3a/d60/d74/fa1 x:0 0 0 2024-04-27T04:10:09.433 INFO:tasks.workunit.client.0.smithi162.stdout:6/510: chown d6/d1a/d34/d50/f63 0 1 2024-04-27T04:10:09.438 INFO:tasks.workunit.client.0.smithi162.stdout:0/552: creat d5/dc/d1e/faf x:0 0 0 2024-04-27T04:10:09.438 INFO:tasks.workunit.client.1.smithi181.stdout:3/637: creat d0/de/d7d/dbc/fc9 x:0 0 0 2024-04-27T04:10:09.440 INFO:tasks.workunit.client.0.smithi162.stdout:4/474: dread d2/d25/f36 [0,4194304] 0 2024-04-27T04:10:09.441 INFO:tasks.workunit.client.1.smithi181.stdout:1/699: mknod d1/dad/db5/d6/da3/d1e/d48/da6/ce7 0 2024-04-27T04:10:09.445 INFO:tasks.workunit.client.0.smithi162.stdout:7/475: dread d3/d4f/f41 [0,4194304] 0 2024-04-27T04:10:09.446 INFO:tasks.workunit.client.0.smithi162.stdout:6/511: dread d6/db/f6a [0,4194304] 0 2024-04-27T04:10:09.447 INFO:tasks.workunit.client.1.smithi181.stdout:3/638: dwrite d0/d6c/da1/daf/fc3 [0,4194304] 0 2024-04-27T04:10:09.447 INFO:tasks.workunit.client.1.smithi181.stdout:3/639: chown d0/d6c/da1/cbf 19367330 1 2024-04-27T04:10:09.448 INFO:tasks.workunit.client.1.smithi181.stdout:6/569: rmdir d7/dc/d8b 39 2024-04-27T04:10:09.449 INFO:tasks.workunit.client.1.smithi181.stdout:3/640: write d0/de/d20/d58/d64/f6a [5535789,23930] 0 2024-04-27T04:10:09.454 INFO:tasks.workunit.client.1.smithi181.stdout:4/641: rename d0/d15/d25/d2a/d7a/f8d to d0/d15/d25/d2a/d4c/d7c/fd3 0 2024-04-27T04:10:09.457 INFO:tasks.workunit.client.1.smithi181.stdout:5/827: creat d4/d10/d29/d122/de7/f129 x:0 0 0 2024-04-27T04:10:09.458 INFO:tasks.workunit.client.0.smithi162.stdout:1/462: write d0/d11/f20 [2854554,121858] 0 2024-04-27T04:10:09.459 INFO:tasks.workunit.client.1.smithi181.stdout:8/744: mknod d1/d20/da4/c100 0 2024-04-27T04:10:09.461 INFO:tasks.workunit.client.0.smithi162.stdout:8/509: symlink d0/d10/d25/d65/lac 0 2024-04-27T04:10:09.463 INFO:tasks.workunit.client.1.smithi181.stdout:2/739: creat d4/d9/d16/d82/d7d/dbe/ded/fee x:0 0 0 2024-04-27T04:10:09.464 INFO:tasks.workunit.client.1.smithi181.stdout:2/740: chown d4/d81/f84 138431 1 2024-04-27T04:10:09.467 INFO:tasks.workunit.client.1.smithi181.stdout:1/700: mknod d1/dad/db5/d6/da3/d1e/d48/da6/ce8 0 2024-04-27T04:10:09.474 INFO:tasks.workunit.client.0.smithi162.stdout:0/553: mknod d5/d26/d4a/cb0 0 2024-04-27T04:10:09.475 INFO:tasks.workunit.client.1.smithi181.stdout:0/793: truncate dc/d1f/d6b/db9/dbd/f113 1125993 0 2024-04-27T04:10:09.475 INFO:tasks.workunit.client.1.smithi181.stdout:0/794: chown dc/l4b 3 1 2024-04-27T04:10:09.477 INFO:tasks.workunit.client.0.smithi162.stdout:4/475: mknod d2/d25/d1c/d39/d3b/d47/c99 0 2024-04-27T04:10:09.478 INFO:tasks.workunit.client.0.smithi162.stdout:4/476: dread - d2/f7d zero size 2024-04-27T04:10:09.479 INFO:tasks.workunit.client.1.smithi181.stdout:2/741: dread d4/d9/d16/f4e [4194304,4194304] 0 2024-04-27T04:10:09.480 INFO:tasks.workunit.client.0.smithi162.stdout:0/554: dwrite d5/d26/d4a/f77 [0,4194304] 0 2024-04-27T04:10:09.480 INFO:tasks.workunit.client.0.smithi162.stdout:7/476: creat d3/d5/d5b/d60/d65/d80/fa0 x:0 0 0 2024-04-27T04:10:09.481 INFO:tasks.workunit.client.1.smithi181.stdout:6/570: chown d7/dc/c14 55690 1 2024-04-27T04:10:09.488 INFO:tasks.workunit.client.1.smithi181.stdout:7/666: rename d1/d22/d15/db2 to d1/d2/d1c/d2d/d61/d80/dcb 0 2024-04-27T04:10:09.489 INFO:tasks.workunit.client.1.smithi181.stdout:9/614: dwrite d0/f15 [4194304,4194304] 0 2024-04-27T04:10:09.491 INFO:tasks.workunit.client.0.smithi162.stdout:8/510: creat d0/d10/d25/d65/fad x:0 0 0 2024-04-27T04:10:09.492 INFO:tasks.workunit.client.0.smithi162.stdout:8/511: chown d0/de 222533724 1 2024-04-27T04:10:09.494 INFO:tasks.workunit.client.0.smithi162.stdout:9/497: truncate d3/fc 1186502 0 2024-04-27T04:10:09.498 INFO:tasks.workunit.client.0.smithi162.stdout:4/477: dread d2/f22 [0,4194304] 0 2024-04-27T04:10:09.498 INFO:tasks.workunit.client.0.smithi162.stdout:4/478: stat d2/d25/d1c/d39/d3b 0 2024-04-27T04:10:09.500 INFO:tasks.workunit.client.0.smithi162.stdout:3/493: truncate d1/d70/f8c 7992275 0 2024-04-27T04:10:09.504 INFO:tasks.workunit.client.0.smithi162.stdout:2/441: mkdir d0/d4/d17/d2b/d64/d89 0 2024-04-27T04:10:09.505 INFO:tasks.workunit.client.0.smithi162.stdout:2/442: fsync d0/d4/d17/f2f 0 2024-04-27T04:10:09.507 INFO:tasks.workunit.client.1.smithi181.stdout:1/701: creat d1/dad/db5/d6/db/fe9 x:0 0 0 2024-04-27T04:10:09.508 INFO:tasks.workunit.client.1.smithi181.stdout:1/702: chown d1/dad/db5/d6/da3/d1e/d48/da6/fd4 26526331 1 2024-04-27T04:10:09.510 INFO:tasks.workunit.client.1.smithi181.stdout:0/795: unlink dc/f15 0 2024-04-27T04:10:09.514 INFO:tasks.workunit.client.1.smithi181.stdout:2/742: write d4/d9/df/d14/fb6 [388448,24577] 0 2024-04-27T04:10:09.522 INFO:tasks.workunit.client.1.smithi181.stdout:2/743: dwrite d4/d9/d4f/d5f/fc1 [0,4194304] 0 2024-04-27T04:10:09.523 INFO:tasks.workunit.client.1.smithi181.stdout:2/744: stat d4/d9/d35 0 2024-04-27T04:10:09.524 INFO:tasks.workunit.client.1.smithi181.stdout:2/745: truncate d4/d9/d4f/fac 475676 0 2024-04-27T04:10:09.531 INFO:tasks.workunit.client.1.smithi181.stdout:8/745: rename d1/d6/d88/l96 to d1/d20/l101 0 2024-04-27T04:10:09.532 INFO:tasks.workunit.client.1.smithi181.stdout:3/641: write d0/de/f1f [1717005,44729] 0 2024-04-27T04:10:09.532 INFO:tasks.workunit.client.0.smithi162.stdout:1/463: write d0/d7/d3c/d54/f65 [855810,107786] 0 2024-04-27T04:10:09.534 INFO:tasks.workunit.client.1.smithi181.stdout:3/642: chown d0/d6c/d52/d44/l82 71 1 2024-04-27T04:10:09.535 INFO:tasks.workunit.client.0.smithi162.stdout:8/512: fsync d0/de/d6e/d71/f95 0 2024-04-27T04:10:09.536 INFO:tasks.workunit.client.1.smithi181.stdout:9/615: dread - d0/d4d/fb5 zero size 2024-04-27T04:10:09.537 INFO:tasks.workunit.client.1.smithi181.stdout:9/616: readlink d0/d44/d4b/d43/l71 0 2024-04-27T04:10:09.537 INFO:tasks.workunit.client.0.smithi162.stdout:8/513: dread - d0/d10/d44/d70/f7f zero size 2024-04-27T04:10:09.538 INFO:tasks.workunit.client.0.smithi162.stdout:6/512: dwrite d6/d1a/d34/d43/f44 [0,4194304] 0 2024-04-27T04:10:09.543 INFO:tasks.workunit.client.0.smithi162.stdout:3/494: mkdir d1/d18/d23/d2f/da0 0 2024-04-27T04:10:09.545 INFO:tasks.workunit.client.0.smithi162.stdout:8/514: dread d0/d10/d25/d7d/f8b [0,4194304] 0 2024-04-27T04:10:09.545 INFO:tasks.workunit.client.0.smithi162.stdout:3/495: chown d1/d18/d23/f28 21650 1 2024-04-27T04:10:09.548 INFO:tasks.workunit.client.0.smithi162.stdout:4/479: dwrite d2/d25/f8c [0,4194304] 0 2024-04-27T04:10:09.549 INFO:tasks.workunit.client.0.smithi162.stdout:5/514: sync 2024-04-27T04:10:09.551 INFO:tasks.workunit.client.1.smithi181.stdout:7/667: dread d1/d57/f77 [0,4194304] 0 2024-04-27T04:10:09.553 INFO:tasks.workunit.client.1.smithi181.stdout:1/703: rmdir d1/dad/db5/d6/da3/d1e/d48/da6 39 2024-04-27T04:10:09.554 INFO:tasks.workunit.client.1.smithi181.stdout:1/704: truncate d1/d2/fce 279126 0 2024-04-27T04:10:09.558 INFO:tasks.workunit.client.1.smithi181.stdout:0/796: dread dc/d1f/d3b/d4a/d95/d6d/fc4 [0,4194304] 0 2024-04-27T04:10:09.566 INFO:tasks.workunit.client.0.smithi162.stdout:9/498: link d3/d11/d45/d4e/d58/f68 d3/d13/d15/d7e/f92 0 2024-04-27T04:10:09.570 INFO:tasks.workunit.client.1.smithi181.stdout:4/642: truncate d0/d15/d25/d2b/f1c 638707 0 2024-04-27T04:10:09.573 INFO:tasks.workunit.client.1.smithi181.stdout:5/828: write d4/de/f6e [951691,51524] 0 2024-04-27T04:10:09.574 INFO:tasks.workunit.client.0.smithi162.stdout:8/515: unlink d0/de/d20/f60 0 2024-04-27T04:10:09.578 INFO:tasks.workunit.client.1.smithi181.stdout:7/668: symlink d1/d2/d1c/d8f/lcc 0 2024-04-27T04:10:09.579 INFO:tasks.workunit.client.1.smithi181.stdout:7/669: chown d1/d2/f12 312865861 1 2024-04-27T04:10:09.579 INFO:tasks.workunit.client.0.smithi162.stdout:4/480: chown d2/da/d1a/c50 168330 1 2024-04-27T04:10:09.579 INFO:tasks.workunit.client.0.smithi162.stdout:3/496: dread d1/d18/d23/f2b [0,4194304] 0 2024-04-27T04:10:09.580 INFO:tasks.workunit.client.1.smithi181.stdout:1/705: symlink d1/dad/db5/d6/da3/d1e/dbc/lea 0 2024-04-27T04:10:09.581 INFO:tasks.workunit.client.0.smithi162.stdout:5/515: mknod d8/d9/d55/ca6 0 2024-04-27T04:10:09.582 INFO:tasks.workunit.client.1.smithi181.stdout:6/571: truncate d7/dc/d1c/f35 868331 0 2024-04-27T04:10:09.585 INFO:tasks.workunit.client.0.smithi162.stdout:0/555: creat d5/dc/d1e/d46/d48/fb1 x:0 0 0 2024-04-27T04:10:09.585 INFO:tasks.workunit.client.0.smithi162.stdout:3/497: dwrite d1/d18/d23/f97 [0,4194304] 0 2024-04-27T04:10:09.587 INFO:tasks.workunit.client.0.smithi162.stdout:6/513: write d6/db/d32/f51 [628237,18910] 0 2024-04-27T04:10:09.588 INFO:tasks.workunit.client.0.smithi162.stdout:6/514: chown d6/d1a/d34/d50/f63 1711814 1 2024-04-27T04:10:09.589 INFO:tasks.workunit.client.0.smithi162.stdout:1/464: symlink d0/d11/d5b/l94 0 2024-04-27T04:10:09.594 INFO:tasks.workunit.client.1.smithi181.stdout:2/746: mknod d4/d9/d16/d45/d9a/cef 0 2024-04-27T04:10:09.596 INFO:tasks.workunit.client.1.smithi181.stdout:8/746: link d1/d8c/f90 d1/d6/d2f/d30/d5c/f102 0 2024-04-27T04:10:09.598 INFO:tasks.workunit.client.1.smithi181.stdout:3/643: dwrite d0/de/ff [0,4194304] 0 2024-04-27T04:10:09.602 INFO:tasks.workunit.client.1.smithi181.stdout:5/829: fsync d4/de/d2c/d46/d4d/d8e/fdc 0 2024-04-27T04:10:09.605 INFO:tasks.workunit.client.1.smithi181.stdout:0/797: dread dc/f18 [0,4194304] 0 2024-04-27T04:10:09.607 INFO:tasks.workunit.client.1.smithi181.stdout:9/617: mknod d0/d44/d4b/d43/d89/cd5 0 2024-04-27T04:10:09.610 INFO:tasks.workunit.client.1.smithi181.stdout:7/670: symlink d1/d2/d1c/d2d/d61/d80/lcd 0 2024-04-27T04:10:09.614 INFO:tasks.workunit.client.0.smithi162.stdout:4/481: chown d2/d25/f8 15510277 1 2024-04-27T04:10:09.616 INFO:tasks.workunit.client.1.smithi181.stdout:1/706: rename d1/d2/d64/faa to d1/dad/db5/d6/da3/d1e/d20/d53/d56/dd9/feb 0 2024-04-27T04:10:09.619 INFO:tasks.workunit.client.0.smithi162.stdout:5/516: rename d8/la to d8/d1a/d7d/la7 0 2024-04-27T04:10:09.620 INFO:tasks.workunit.client.0.smithi162.stdout:5/517: truncate d8/d1a/d7d/f91 1105270 0 2024-04-27T04:10:09.620 INFO:tasks.workunit.client.0.smithi162.stdout:5/518: dread - d8/d1a/d29/d4a/d5c/f9e zero size 2024-04-27T04:10:09.624 INFO:tasks.workunit.client.1.smithi181.stdout:6/572: creat d7/dc/d1c/d7f/d3e/fc4 x:0 0 0 2024-04-27T04:10:09.624 INFO:tasks.workunit.client.0.smithi162.stdout:1/465: dread d0/d11/d3e/f41 [0,4194304] 0 2024-04-27T04:10:09.624 INFO:tasks.workunit.client.1.smithi181.stdout:1/707: dwrite d1/dad/db5/d6/db/d2d/d36/f4a [0,4194304] 0 2024-04-27T04:10:09.626 INFO:tasks.workunit.client.0.smithi162.stdout:5/519: dwrite d8/d9/d55/f6d [0,4194304] 0 2024-04-27T04:10:09.627 INFO:tasks.workunit.client.0.smithi162.stdout:0/556: dread d5/f27 [0,4194304] 0 2024-04-27T04:10:09.628 INFO:tasks.workunit.client.0.smithi162.stdout:7/477: link d3/d4f/c66 d3/d5/d5b/d83/de/d2b/d81/ca1 0 2024-04-27T04:10:09.630 INFO:tasks.workunit.client.1.smithi181.stdout:4/643: mkdir d0/dd4 0 2024-04-27T04:10:09.637 INFO:tasks.workunit.client.0.smithi162.stdout:3/498: fdatasync d1/d18/f4a 0 2024-04-27T04:10:09.638 INFO:tasks.workunit.client.1.smithi181.stdout:3/644: mkdir d0/d6c/d52/dbb/dca 0 2024-04-27T04:10:09.639 INFO:tasks.workunit.client.1.smithi181.stdout:3/645: chown d0/de/d7d/c94 167 1 2024-04-27T04:10:09.639 INFO:tasks.workunit.client.0.smithi162.stdout:6/515: symlink d6/d1a/d34/d43/la2 0 2024-04-27T04:10:09.640 INFO:tasks.workunit.client.1.smithi181.stdout:3/646: chown d0/d28/l33 7 1 2024-04-27T04:10:09.641 INFO:tasks.workunit.client.0.smithi162.stdout:6/516: write d6/d1a/d34/d43/f6b [1422119,94602] 0 2024-04-27T04:10:09.645 INFO:tasks.workunit.client.1.smithi181.stdout:0/798: stat dc/d1f/daa/ccd 0 2024-04-27T04:10:09.648 INFO:tasks.workunit.client.0.smithi162.stdout:8/516: link d0/d10/d25/f30 d0/de/d6e/d71/fae 0 2024-04-27T04:10:09.649 INFO:tasks.workunit.client.0.smithi162.stdout:2/443: getdents d0/d4/d2a/d3b/d49/d5a/d63/d6e 0 2024-04-27T04:10:09.653 INFO:tasks.workunit.client.0.smithi162.stdout:4/482: creat d2/d25/d1c/d72/f9a x:0 0 0 2024-04-27T04:10:09.657 INFO:tasks.workunit.client.1.smithi181.stdout:8/747: dwrite d1/d6/d2f/d30/d5c/f97 [0,4194304] 0 2024-04-27T04:10:09.657 INFO:tasks.workunit.client.1.smithi181.stdout:1/708: mknod d1/dad/db5/d6/da3/d52/cec 0 2024-04-27T04:10:09.660 INFO:tasks.workunit.client.1.smithi181.stdout:8/748: read d1/d20/f51 [1310480,10543] 0 2024-04-27T04:10:09.660 INFO:tasks.workunit.client.1.smithi181.stdout:5/830: write d4/da3/fa7 [2987220,12070] 0 2024-04-27T04:10:09.664 INFO:tasks.workunit.client.0.smithi162.stdout:5/520: creat d8/d1a/d23/d40/d4b/fa8 x:0 0 0 2024-04-27T04:10:09.666 INFO:tasks.workunit.client.1.smithi181.stdout:2/747: truncate d4/d9/d35/d42/d86/f8a 758045 0 2024-04-27T04:10:09.667 INFO:tasks.workunit.client.0.smithi162.stdout:7/478: dread - d3/d5/d5b/d83/de/d2b/f37 zero size 2024-04-27T04:10:09.667 INFO:tasks.workunit.client.1.smithi181.stdout:2/748: dread - d4/d9/df/d14/f48 zero size 2024-04-27T04:10:09.671 INFO:tasks.workunit.client.0.smithi162.stdout:0/557: write d5/d26/d69/f6f [46307,29573] 0 2024-04-27T04:10:09.675 INFO:tasks.workunit.client.1.smithi181.stdout:3/647: rename d0/d6c/d66/d99/fa3 to d0/d6c/d66/dc2/fcb 0 2024-04-27T04:10:09.676 INFO:tasks.workunit.client.0.smithi162.stdout:0/558: dwrite d5/d60/f72 [0,4194304] 0 2024-04-27T04:10:09.680 INFO:tasks.workunit.client.0.smithi162.stdout:6/517: stat d6/c79 0 2024-04-27T04:10:09.682 INFO:tasks.workunit.client.0.smithi162.stdout:9/499: creat d3/f93 x:0 0 0 2024-04-27T04:10:09.684 INFO:tasks.workunit.client.0.smithi162.stdout:9/500: read d3/fb [418145,50136] 0 2024-04-27T04:10:09.685 INFO:tasks.workunit.client.1.smithi181.stdout:4/644: dwrite d0/d15/d25/d2b/f2c [0,4194304] 0 2024-04-27T04:10:09.687 INFO:tasks.workunit.client.0.smithi162.stdout:9/501: dread d3/d23/f4b [0,4194304] 0 2024-04-27T04:10:09.694 INFO:tasks.workunit.client.0.smithi162.stdout:9/502: dwrite d3/d13/d2d/d3b/d54/f8f [0,4194304] 0 2024-04-27T04:10:09.694 INFO:tasks.workunit.client.1.smithi181.stdout:6/573: symlink d7/dc/d1c/d88/db3/lc5 0 2024-04-27T04:10:09.694 INFO:tasks.workunit.client.1.smithi181.stdout:4/645: dwrite d0/d15/d5c/fb2 [0,4194304] 0 2024-04-27T04:10:09.697 INFO:tasks.workunit.client.1.smithi181.stdout:1/709: fdatasync f0 0 2024-04-27T04:10:09.699 INFO:tasks.workunit.client.1.smithi181.stdout:8/749: stat d1/d6/d2f/d72/fc6 0 2024-04-27T04:10:09.701 INFO:tasks.workunit.client.1.smithi181.stdout:5/831: creat d4/d10/d72/d80/d85/f12a x:0 0 0 2024-04-27T04:10:09.703 INFO:tasks.workunit.client.1.smithi181.stdout:2/749: mkdir d4/d9/d16/d82/d7d/dbe/df0 0 2024-04-27T04:10:09.705 INFO:tasks.workunit.client.0.smithi162.stdout:6/518: write d6/d1a/d34/f66 [490164,124326] 0 2024-04-27T04:10:09.707 INFO:tasks.workunit.client.1.smithi181.stdout:9/618: write d0/d40/f84 [368250,32668] 0 2024-04-27T04:10:09.709 INFO:tasks.workunit.client.1.smithi181.stdout:0/799: dread dc/f42 [0,4194304] 0 2024-04-27T04:10:09.712 INFO:tasks.workunit.client.0.smithi162.stdout:4/483: mknod d2/d25/d1c/d39/d3b/d6e/c9b 0 2024-04-27T04:10:09.713 INFO:tasks.workunit.client.0.smithi162.stdout:2/444: dwrite d0/f44 [0,4194304] 0 2024-04-27T04:10:09.714 INFO:tasks.workunit.client.0.smithi162.stdout:4/484: chown d2/d25/d1c/d72/f65 54677780 1 2024-04-27T04:10:09.715 INFO:tasks.workunit.client.0.smithi162.stdout:1/466: rename d0/d7/d3c/d77/l29 to d0/d11/d3e/l95 0 2024-04-27T04:10:09.717 INFO:tasks.workunit.client.0.smithi162.stdout:9/503: creat d3/d13/d2d/d3b/d72/f94 x:0 0 0 2024-04-27T04:10:09.717 INFO:tasks.workunit.client.0.smithi162.stdout:1/467: read d0/d11/f2e [3496795,67214] 0 2024-04-27T04:10:09.723 INFO:tasks.workunit.client.0.smithi162.stdout:6/519: truncate d6/db/f13 1256035 0 2024-04-27T04:10:09.724 INFO:tasks.workunit.client.0.smithi162.stdout:6/520: stat d6/d1a/d34/d3a/d68/d6c 0 2024-04-27T04:10:09.727 INFO:tasks.workunit.client.0.smithi162.stdout:8/517: link d0/d10/d2b/f19 d0/d10/faf 0 2024-04-27T04:10:09.727 INFO:tasks.workunit.client.0.smithi162.stdout:8/518: chown d0/de/d6e/f77 16030 1 2024-04-27T04:10:09.729 INFO:tasks.workunit.client.0.smithi162.stdout:4/485: mkdir d2/d25/d1c/d39/d3b/d47/d9c 0 2024-04-27T04:10:09.738 INFO:tasks.workunit.client.0.smithi162.stdout:5/521: rmdir d8/d1a/d23/d40/d4b/d58/d8d 0 2024-04-27T04:10:09.739 INFO:tasks.workunit.client.0.smithi162.stdout:5/522: write d8/d1a/f81 [927014,22893] 0 2024-04-27T04:10:09.740 INFO:tasks.workunit.client.1.smithi181.stdout:7/671: getdents d1/d2/d35/d67/dbf 0 2024-04-27T04:10:09.740 INFO:tasks.workunit.client.0.smithi162.stdout:5/523: fsync d8/d1a/d23/d40/d4b/d78/f9d 0 2024-04-27T04:10:09.741 INFO:tasks.workunit.client.1.smithi181.stdout:7/672: write d1/d22/d15/d29/dbe/d1b/f47 [1298172,25655] 0 2024-04-27T04:10:09.743 INFO:tasks.workunit.client.0.smithi162.stdout:8/519: unlink d0/d10/d25/d7d/f98 0 2024-04-27T04:10:09.744 INFO:tasks.workunit.client.1.smithi181.stdout:6/574: mknod d7/dc/d1c/d7f/da3/cc6 0 2024-04-27T04:10:09.745 INFO:tasks.workunit.client.0.smithi162.stdout:8/520: write d0/de/d20/d6c/f6d [2766192,30504] 0 2024-04-27T04:10:09.746 INFO:tasks.workunit.client.0.smithi162.stdout:8/521: dread - d0/d10/d44/d70/fa9 zero size 2024-04-27T04:10:09.748 INFO:tasks.workunit.client.1.smithi181.stdout:6/575: write d7/dc/d1c/d34/d3f/d7e/d8a/fa9 [164295,79595] 0 2024-04-27T04:10:09.751 INFO:tasks.workunit.client.1.smithi181.stdout:8/750: dread d1/d6/d2f/d47/f8e [0,4194304] 0 2024-04-27T04:10:09.751 INFO:tasks.workunit.client.1.smithi181.stdout:5/832: dwrite d4/de/d2c/d46/d4d/f8b [0,4194304] 0 2024-04-27T04:10:09.754 INFO:tasks.workunit.client.0.smithi162.stdout:1/468: mkdir d0/d11/d50/d53/d93/d96 0 2024-04-27T04:10:09.754 INFO:tasks.workunit.client.1.smithi181.stdout:2/750: creat d4/d9/d35/d9f/ff1 x:0 0 0 2024-04-27T04:10:09.755 INFO:tasks.workunit.client.1.smithi181.stdout:9/619: creat d0/d44/d4b/d43/d89/d94/fd6 x:0 0 0 2024-04-27T04:10:09.758 INFO:tasks.workunit.client.1.smithi181.stdout:3/648: rename d0/de/d3f to d0/d6c/dcc 0 2024-04-27T04:10:09.763 INFO:tasks.workunit.client.0.smithi162.stdout:6/521: truncate d6/d1a/f31 821274 0 2024-04-27T04:10:09.764 INFO:tasks.workunit.client.1.smithi181.stdout:1/710: truncate d1/dad/db5/fb8 167566 0 2024-04-27T04:10:09.765 INFO:tasks.workunit.client.0.smithi162.stdout:9/504: write d3/fc [1763345,105537] 0 2024-04-27T04:10:09.766 INFO:tasks.workunit.client.0.smithi162.stdout:9/505: dread - d3/f93 zero size 2024-04-27T04:10:09.771 INFO:tasks.workunit.client.0.smithi162.stdout:3/499: rename d1/d45/f79 to d1/d21/fa1 0 2024-04-27T04:10:09.773 INFO:tasks.workunit.client.1.smithi181.stdout:3/649: dread d0/f57 [0,4194304] 0 2024-04-27T04:10:09.774 INFO:tasks.workunit.client.1.smithi181.stdout:5/833: creat d4/de/d107/f12b x:0 0 0 2024-04-27T04:10:09.774 INFO:tasks.workunit.client.1.smithi181.stdout:3/650: fsync d0/d6c/d52/d44/d56/f76 0 2024-04-27T04:10:09.775 INFO:tasks.workunit.client.0.smithi162.stdout:1/469: creat d0/d7/d3c/f97 x:0 0 0 2024-04-27T04:10:09.775 INFO:tasks.workunit.client.1.smithi181.stdout:8/751: dread d1/d6/d2f/d30/d9d/dea/f64 [0,4194304] 0 2024-04-27T04:10:09.776 INFO:tasks.workunit.client.0.smithi162.stdout:1/470: write d0/d11/d3e/d5f/f6a [611779,13571] 0 2024-04-27T04:10:09.776 INFO:tasks.workunit.client.1.smithi181.stdout:5/834: write d4/d10/d29/d122/de7/f129 [955371,46046] 0 2024-04-27T04:10:09.777 INFO:tasks.workunit.client.0.smithi162.stdout:1/471: chown d0/d7/f19 75926106 1 2024-04-27T04:10:09.777 INFO:tasks.workunit.client.0.smithi162.stdout:1/472: write d0/d7/d2a/f31 [834675,116191] 0 2024-04-27T04:10:09.780 INFO:tasks.workunit.client.0.smithi162.stdout:1/473: read d0/d7/f19 [2250595,68180] 0 2024-04-27T04:10:09.784 INFO:tasks.workunit.client.1.smithi181.stdout:0/800: mknod dc/d1f/d3b/dfa/c117 0 2024-04-27T04:10:09.791 INFO:tasks.workunit.client.1.smithi181.stdout:4/646: getdents d0/d15/d25/d2a/d4c 0 2024-04-27T04:10:09.791 INFO:tasks.workunit.client.1.smithi181.stdout:2/751: write d4/d9/d16/d45/f9e [3543604,26385] 0 2024-04-27T04:10:09.793 INFO:tasks.workunit.client.1.smithi181.stdout:0/801: dwrite dc/d1f/d6b/db9/dbd/fbe [0,4194304] 0 2024-04-27T04:10:09.793 INFO:tasks.workunit.client.0.smithi162.stdout:7/479: rename d3/f40 to d3/d92/fa2 0 2024-04-27T04:10:09.795 INFO:tasks.workunit.client.1.smithi181.stdout:5/835: dread d4/d10/d72/d80/f8d [0,4194304] 0 2024-04-27T04:10:09.795 INFO:tasks.workunit.client.1.smithi181.stdout:6/576: unlink d7/dc/d1e/d5c/l6e 0 2024-04-27T04:10:09.803 INFO:tasks.workunit.client.1.smithi181.stdout:8/752: stat d1/d6/d2f/d40/cb4 0 2024-04-27T04:10:09.804 INFO:tasks.workunit.client.0.smithi162.stdout:5/524: creat d8/fa9 x:0 0 0 2024-04-27T04:10:09.805 INFO:tasks.workunit.client.0.smithi162.stdout:5/525: fdatasync d8/d1a/d29/f99 0 2024-04-27T04:10:09.805 INFO:tasks.workunit.client.0.smithi162.stdout:5/526: chown d8/d9/d5b/f79 1700 1 2024-04-27T04:10:09.808 INFO:tasks.workunit.client.1.smithi181.stdout:9/620: mknod d0/d44/cd7 0 2024-04-27T04:10:09.813 INFO:tasks.workunit.client.0.smithi162.stdout:4/486: getdents d2/d25/d1c/d39/d3b 0 2024-04-27T04:10:09.816 INFO:tasks.workunit.client.0.smithi162.stdout:6/522: truncate d6/db/f48 1200773 0 2024-04-27T04:10:09.817 INFO:tasks.workunit.client.1.smithi181.stdout:2/752: rmdir d4/d81 39 2024-04-27T04:10:09.818 INFO:tasks.workunit.client.1.smithi181.stdout:2/753: stat d4/f50 0 2024-04-27T04:10:09.819 INFO:tasks.workunit.client.1.smithi181.stdout:0/802: stat dc/d1f/d3b/d4a/d95/c21 0 2024-04-27T04:10:09.821 INFO:tasks.workunit.client.0.smithi162.stdout:0/559: rename d5/dc/d1e/d56/d57/l66 to d5/dc/d1e/d56/da8/lb2 0 2024-04-27T04:10:09.821 INFO:tasks.workunit.client.0.smithi162.stdout:2/445: rename d0/d4/d2a/d3b/d49 to d0/d4/d2a/d3b/d49/d5a/d63/d84/d8a 22 2024-04-27T04:10:09.822 INFO:tasks.workunit.client.0.smithi162.stdout:2/446: write d0/d4/d2a/d41/f81 [9543,74834] 0 2024-04-27T04:10:09.824 INFO:tasks.workunit.client.0.smithi162.stdout:7/480: creat d3/d5/d5b/d83/fa3 x:0 0 0 2024-04-27T04:10:09.825 INFO:tasks.workunit.client.1.smithi181.stdout:6/577: symlink d7/dc/d1c/d34/d3f/d7e/d8a/lc7 0 2024-04-27T04:10:09.827 INFO:tasks.workunit.client.0.smithi162.stdout:3/500: mknod d1/d18/d23/d58/ca2 0 2024-04-27T04:10:09.829 INFO:tasks.workunit.client.1.smithi181.stdout:3/651: creat d0/d6c/d66/d88/fcd x:0 0 0 2024-04-27T04:10:09.832 INFO:tasks.workunit.client.1.smithi181.stdout:8/753: unlink d1/d6/d2f/fb0 0 2024-04-27T04:10:09.833 INFO:tasks.workunit.client.0.smithi162.stdout:1/474: link d0/d80/f87 d0/d7/d5e/f98 0 2024-04-27T04:10:09.837 INFO:tasks.workunit.client.1.smithi181.stdout:9/621: mkdir d0/d4d/d85/dd8 0 2024-04-27T04:10:09.839 INFO:tasks.workunit.client.1.smithi181.stdout:7/673: link d1/d57/c98 d1/d57/d72/cce 0 2024-04-27T04:10:09.840 INFO:tasks.workunit.client.1.smithi181.stdout:7/674: fsync d1/d22/d15/d29/dbe/fad 0 2024-04-27T04:10:09.842 INFO:tasks.workunit.client.0.smithi162.stdout:5/527: dwrite d8/d1a/f2d [0,4194304] 0 2024-04-27T04:10:09.842 INFO:tasks.workunit.client.0.smithi162.stdout:5/528: fsync d8/d9/f48 0 2024-04-27T04:10:09.846 INFO:tasks.workunit.client.1.smithi181.stdout:2/754: dread d4/d9/d4f/d79/fa3 [0,4194304] 0 2024-04-27T04:10:09.847 INFO:tasks.workunit.client.0.smithi162.stdout:7/481: creat d3/d5/d3c/d4a/fa4 x:0 0 0 2024-04-27T04:10:09.849 INFO:tasks.workunit.client.0.smithi162.stdout:3/501: mkdir d1/d45/d69/da3 0 2024-04-27T04:10:09.850 INFO:tasks.workunit.client.1.smithi181.stdout:6/578: rmdir d7/dc/d1c/d88/db3 39 2024-04-27T04:10:09.853 INFO:tasks.workunit.client.1.smithi181.stdout:5/836: link d4/f112 d4/de/d2f/d7a/d90/f12c 0 2024-04-27T04:10:09.853 INFO:tasks.workunit.client.1.smithi181.stdout:4/647: dwrite d0/d15/d25/d2a/d61/f9d [0,4194304] 0 2024-04-27T04:10:09.855 INFO:tasks.workunit.client.0.smithi162.stdout:1/475: creat d0/d11/d3e/d5f/f99 x:0 0 0 2024-04-27T04:10:09.858 INFO:tasks.workunit.client.1.smithi181.stdout:1/711: getdents d1/dad 0 2024-04-27T04:10:09.862 INFO:tasks.workunit.client.1.smithi181.stdout:7/675: creat d1/dc3/fcf x:0 0 0 2024-04-27T04:10:09.862 INFO:tasks.workunit.client.0.smithi162.stdout:9/506: getdents d3 0 2024-04-27T04:10:09.862 INFO:tasks.workunit.client.0.smithi162.stdout:6/523: symlink d6/d1a/d34/la3 0 2024-04-27T04:10:09.865 INFO:tasks.workunit.client.0.smithi162.stdout:5/529: symlink d8/d2c/laa 0 2024-04-27T04:10:09.865 INFO:tasks.workunit.client.0.smithi162.stdout:5/530: chown d8/d9/d55/f69 23422 1 2024-04-27T04:10:09.866 INFO:tasks.workunit.client.1.smithi181.stdout:6/579: mknod d7/dc/d1c/d88/d6d/cc8 0 2024-04-27T04:10:09.867 INFO:tasks.workunit.client.0.smithi162.stdout:2/447: symlink d0/d4/d2a/d41/d7b/l8b 0 2024-04-27T04:10:09.867 INFO:tasks.workunit.client.0.smithi162.stdout:5/531: truncate d8/d1a/d29/d4a/d5c/f9e 877797 0 2024-04-27T04:10:09.868 INFO:tasks.workunit.client.0.smithi162.stdout:5/532: truncate d8/d1a/f83 599766 0 2024-04-27T04:10:09.869 INFO:tasks.workunit.client.0.smithi162.stdout:5/533: truncate d8/d9/d55/f65 4565054 0 2024-04-27T04:10:09.869 INFO:tasks.workunit.client.0.smithi162.stdout:5/534: fsync d8/d9/d5b/f79 0 2024-04-27T04:10:09.870 INFO:tasks.workunit.client.0.smithi162.stdout:5/535: stat d8/d1a/d29/d4a/d5c/f9e 0 2024-04-27T04:10:09.871 INFO:tasks.workunit.client.0.smithi162.stdout:5/536: write d8/d9/d5b/f79 [1498277,94427] 0 2024-04-27T04:10:09.871 INFO:tasks.workunit.client.0.smithi162.stdout:8/522: sync 2024-04-27T04:10:09.907 INFO:tasks.workunit.client.0.smithi162.stdout:0/560: sync 2024-04-27T04:10:09.913 INFO:tasks.workunit.client.0.smithi162.stdout:7/482: creat d3/d5/d5b/d60/fa5 x:0 0 0 2024-04-27T04:10:09.972 INFO:tasks.workunit.client.1.smithi181.stdout:9/622: write d0/f1e [1406512,101198] 0 2024-04-27T04:10:09.974 INFO:tasks.workunit.client.0.smithi162.stdout:1/476: dread d0/d7/d3c/d77/f28 [0,4194304] 0 2024-04-27T04:10:09.975 INFO:tasks.workunit.client.1.smithi181.stdout:9/623: write d0/d44/d4b/d3d/dbd/fc9 [113342,118706] 0 2024-04-27T04:10:09.976 INFO:tasks.workunit.client.0.smithi162.stdout:4/487: creat d2/da/d1a/f9d x:0 0 0 2024-04-27T04:10:09.980 INFO:tasks.workunit.client.1.smithi181.stdout:4/648: dread d0/d15/d25/d2a/d4c/f4e [0,4194304] 0 2024-04-27T04:10:09.980 INFO:tasks.workunit.client.0.smithi162.stdout:9/507: rmdir d3/d23/d5d 39 2024-04-27T04:10:09.982 INFO:tasks.workunit.client.1.smithi181.stdout:6/580: creat d7/d27/fc9 x:0 0 0 2024-04-27T04:10:09.984 INFO:tasks.workunit.client.0.smithi162.stdout:6/524: fdatasync d6/d1a/d34/d43/f5c 0 2024-04-27T04:10:09.985 INFO:tasks.workunit.client.0.smithi162.stdout:6/525: dread - d6/db/d32/f38 zero size 2024-04-27T04:10:09.988 INFO:tasks.workunit.client.1.smithi181.stdout:7/676: link d1/d22/d15/d29/f84 d1/df/d62/fd0 0 2024-04-27T04:10:09.989 INFO:tasks.workunit.client.1.smithi181.stdout:7/677: write d1/d22/f26 [4706395,26107] 0 2024-04-27T04:10:09.990 INFO:tasks.workunit.client.1.smithi181.stdout:7/678: dread - d1/fc8 zero size 2024-04-27T04:10:09.993 INFO:tasks.workunit.client.0.smithi162.stdout:5/537: creat d8/d1a/d7d/fab x:0 0 0 2024-04-27T04:10:09.998 INFO:tasks.workunit.client.1.smithi181.stdout:4/649: mknod d0/d15/d25/d2a/da0/cd5 0 2024-04-27T04:10:10.000 INFO:tasks.workunit.client.0.smithi162.stdout:0/561: symlink d5/d26/d69/d9b/lb3 0 2024-04-27T04:10:10.005 INFO:tasks.workunit.client.0.smithi162.stdout:7/483: truncate d3/d5/d5b/d83/de/d2b/f69 35126 0 2024-04-27T04:10:10.006 INFO:tasks.workunit.client.0.smithi162.stdout:7/484: dread - d3/d5/d3c/d4a/f95 zero size 2024-04-27T04:10:10.007 INFO:tasks.workunit.client.1.smithi181.stdout:3/652: getdents d0/d6c/d52/dbb 0 2024-04-27T04:10:10.014 INFO:tasks.workunit.client.1.smithi181.stdout:7/679: fdatasync d1/df/f18 0 2024-04-27T04:10:10.027 INFO:tasks.workunit.client.1.smithi181.stdout:7/680: rmdir d1/d22/d15/d1f/d34 39 2024-04-27T04:10:10.030 INFO:tasks.workunit.client.0.smithi162.stdout:8/523: dread - d0/d10/d2b/d2e/f40 zero size 2024-04-27T04:10:10.031 INFO:tasks.workunit.client.1.smithi181.stdout:4/650: symlink d0/ld6 0 2024-04-27T04:10:10.033 INFO:tasks.workunit.client.0.smithi162.stdout:0/562: rmdir d5/dc/d1e/d46 39 2024-04-27T04:10:10.036 INFO:tasks.workunit.client.0.smithi162.stdout:9/508: mknod d3/d13/d2d/d3b/d7a/c95 0 2024-04-27T04:10:10.036 INFO:tasks.workunit.client.0.smithi162.stdout:0/563: dread d5/dc/d1e/d41/f71 [0,4194304] 0 2024-04-27T04:10:10.037 INFO:tasks.workunit.client.0.smithi162.stdout:0/564: chown d5/la0 3252 1 2024-04-27T04:10:10.046 INFO:tasks.workunit.client.0.smithi162.stdout:8/524: symlink d0/d10/d25/d7d/lb0 0 2024-04-27T04:10:10.046 INFO:tasks.workunit.client.1.smithi181.stdout:4/651: getdents d0/d15/d25/d2a/d36/db8/dca 0 2024-04-27T04:10:10.049 INFO:tasks.workunit.client.1.smithi181.stdout:3/653: creat d0/d6c/da1/daf/fce x:0 0 0 2024-04-27T04:10:10.050 INFO:tasks.workunit.client.0.smithi162.stdout:7/485: mknod d3/ca6 0 2024-04-27T04:10:10.052 INFO:tasks.workunit.client.1.smithi181.stdout:7/681: link d1/dc3/fa5 d1/d2/d1c/d2d/fd1 0 2024-04-27T04:10:10.053 INFO:tasks.workunit.client.1.smithi181.stdout:7/682: readlink d1/d2/d35/d67/dbf/lbb 0 2024-04-27T04:10:10.055 INFO:tasks.workunit.client.0.smithi162.stdout:9/509: creat d3/d11/d45/d4e/f96 x:0 0 0 2024-04-27T04:10:10.055 INFO:tasks.workunit.client.0.smithi162.stdout:9/510: chown d3/d13/d15 40598156 1 2024-04-27T04:10:10.055 INFO:tasks.workunit.client.0.smithi162.stdout:9/511: write d3/d11/f4d [2558081,14846] 0 2024-04-27T04:10:10.058 INFO:tasks.workunit.client.0.smithi162.stdout:0/565: mkdir d5/d26/d69/d9b/db4 0 2024-04-27T04:10:10.062 INFO:tasks.workunit.client.0.smithi162.stdout:9/512: dwrite d3/d11/d45/d4e/d58/f8e [0,4194304] 0 2024-04-27T04:10:10.063 INFO:tasks.workunit.client.0.smithi162.stdout:9/513: write d3/d13/f26 [3267727,33079] 0 2024-04-27T04:10:10.066 INFO:tasks.workunit.client.1.smithi181.stdout:4/652: dread d0/d15/f2f [0,4194304] 0 2024-04-27T04:10:10.066 INFO:tasks.workunit.client.0.smithi162.stdout:9/514: write d3/d13/f26 [4932007,116602] 0 2024-04-27T04:10:10.066 INFO:tasks.workunit.client.0.smithi162.stdout:9/515: chown d3/d13/d2d/d3b/d7a/c95 0 1 2024-04-27T04:10:10.068 INFO:tasks.workunit.client.1.smithi181.stdout:3/654: mkdir d0/d6c/da1/dc5/dcf 0 2024-04-27T04:10:10.072 INFO:tasks.workunit.client.0.smithi162.stdout:7/486: mkdir d3/d5/d5b/d83/de/d2b/da7 0 2024-04-27T04:10:10.073 INFO:tasks.workunit.client.0.smithi162.stdout:7/487: write d3/d5/d5b/d83/f3b [3809268,63500] 0 2024-04-27T04:10:10.076 INFO:tasks.workunit.client.0.smithi162.stdout:4/488: sync 2024-04-27T04:10:10.078 INFO:tasks.workunit.client.1.smithi181.stdout:4/653: creat d0/d15/d25/d2b/d6d/d82/fd7 x:0 0 0 2024-04-27T04:10:10.083 INFO:tasks.workunit.client.0.smithi162.stdout:0/566: symlink d5/d26/d69/d9b/lb5 0 2024-04-27T04:10:10.088 INFO:tasks.workunit.client.1.smithi181.stdout:4/654: mknod d0/d15/de/d60/d6f/cd8 0 2024-04-27T04:10:10.090 INFO:tasks.workunit.client.0.smithi162.stdout:5/538: getdents d8/d1a/d23/d40/d4b/d66/d88 0 2024-04-27T04:10:10.095 INFO:tasks.workunit.client.1.smithi181.stdout:4/655: dwrite d0/d15/d25/d2b/d6d/d82/fcc [4194304,4194304] 0 2024-04-27T04:10:10.097 INFO:tasks.workunit.client.1.smithi181.stdout:7/683: getdents d1/df 0 2024-04-27T04:10:10.097 INFO:tasks.workunit.client.1.smithi181.stdout:4/656: chown d0/d15/d76/c9a 47033373 1 2024-04-27T04:10:10.098 INFO:tasks.workunit.client.1.smithi181.stdout:4/657: fdatasync d0/d15/d25/d2a/d36/fb4 0 2024-04-27T04:10:10.106 INFO:tasks.workunit.client.1.smithi181.stdout:4/658: creat d0/d15/de/d19/d77/d57/fd9 x:0 0 0 2024-04-27T04:10:10.110 INFO:tasks.workunit.client.1.smithi181.stdout:7/684: creat d1/d2/d1c/d99/fd2 x:0 0 0 2024-04-27T04:10:10.113 INFO:tasks.workunit.client.0.smithi162.stdout:4/489: readlink d2/l2b 0 2024-04-27T04:10:10.114 INFO:tasks.workunit.client.0.smithi162.stdout:4/490: fsync d2/d25/fb 0 2024-04-27T04:10:10.114 INFO:tasks.workunit.client.0.smithi162.stdout:4/491: read - d2/f7d zero size 2024-04-27T04:10:10.114 INFO:tasks.workunit.client.0.smithi162.stdout:1/477: link d0/d7/d2a/f30 d0/d11/d44/f9a 0 2024-04-27T04:10:10.116 INFO:tasks.workunit.client.1.smithi181.stdout:4/659: rename d0/d15/d5c/d69/f31 to d0/d15/d25/d2b/d6d/d93/fda 0 2024-04-27T04:10:10.117 INFO:tasks.workunit.client.0.smithi162.stdout:0/567: mknod d5/dc/d1e/d56/cb6 0 2024-04-27T04:10:10.119 INFO:tasks.workunit.client.0.smithi162.stdout:9/516: creat d3/d13/d2d/d30/d67/f97 x:0 0 0 2024-04-27T04:10:10.123 INFO:tasks.workunit.client.0.smithi162.stdout:7/488: mknod d3/d5/ca8 0 2024-04-27T04:10:10.127 INFO:tasks.workunit.client.0.smithi162.stdout:4/492: mknod d2/d25/d1c/d39/d3b/d6e/d6f/c9e 0 2024-04-27T04:10:10.128 INFO:tasks.workunit.client.0.smithi162.stdout:1/478: dread - d0/d7/d10/f6b zero size 2024-04-27T04:10:10.129 INFO:tasks.workunit.client.0.smithi162.stdout:9/517: dread d3/d13/d2d/d3b/f47 [0,4194304] 0 2024-04-27T04:10:10.130 INFO:tasks.workunit.client.0.smithi162.stdout:9/518: write d3/d13/d2d/d3b/d69/f7d [806085,60544] 0 2024-04-27T04:10:10.132 INFO:tasks.workunit.client.0.smithi162.stdout:5/539: symlink d8/d86/lac 0 2024-04-27T04:10:10.133 INFO:tasks.workunit.client.1.smithi181.stdout:4/660: truncate d0/d15/d25/d2b/d6d/d82/f8b 794228 0 2024-04-27T04:10:10.134 INFO:tasks.workunit.client.0.smithi162.stdout:5/540: write d8/d1a/d29/d4a/d5c/f9e [775408,90602] 0 2024-04-27T04:10:10.136 INFO:tasks.workunit.client.1.smithi181.stdout:4/661: dread d0/d15/de/fb7 [0,4194304] 0 2024-04-27T04:10:10.137 INFO:tasks.workunit.client.0.smithi162.stdout:9/519: dwrite d3/fc [0,4194304] 0 2024-04-27T04:10:10.137 INFO:tasks.workunit.client.1.smithi181.stdout:4/662: write d0/d15/d25/d2a/f38 [4840676,31882] 0 2024-04-27T04:10:10.139 INFO:tasks.workunit.client.1.smithi181.stdout:4/663: write d0/d15/de/d19/d8a/fc8 [762528,69287] 0 2024-04-27T04:10:10.143 INFO:tasks.workunit.client.0.smithi162.stdout:0/568: link d5/d26/d69/f6c d5/d26/da2/fb7 0 2024-04-27T04:10:10.145 INFO:tasks.workunit.client.0.smithi162.stdout:4/493: dwrite d2/d54/d7a/f83 [0,4194304] 0 2024-04-27T04:10:10.146 INFO:tasks.workunit.client.0.smithi162.stdout:4/494: chown d2/d25/d1c/d72/f9a 12985335 1 2024-04-27T04:10:10.148 INFO:tasks.workunit.client.0.smithi162.stdout:3/502: rename d1/d18/c15 to d1/d45/ca4 0 2024-04-27T04:10:10.151 INFO:tasks.workunit.client.0.smithi162.stdout:0/569: dwrite d5/dc/d1e/d41/f7e [0,4194304] 0 2024-04-27T04:10:10.153 INFO:tasks.workunit.client.0.smithi162.stdout:9/520: creat d3/d11/d45/d4e/d58/f98 x:0 0 0 2024-04-27T04:10:10.154 INFO:tasks.workunit.client.0.smithi162.stdout:4/495: dwrite d2/da/d1a/f9d [0,4194304] 0 2024-04-27T04:10:10.163 INFO:tasks.workunit.client.0.smithi162.stdout:6/526: unlink d6/db/l27 0 2024-04-27T04:10:10.163 INFO:tasks.workunit.client.0.smithi162.stdout:6/527: readlink d6/db/l29 0 2024-04-27T04:10:10.169 INFO:tasks.workunit.client.1.smithi181.stdout:0/803: write dc/d1f/d3b/d4a/d95/d26/d41/d6f/d7d/fa7 [138751,85108] 0 2024-04-27T04:10:10.169 INFO:tasks.workunit.client.0.smithi162.stdout:8/525: link d0/c5c d0/de/d6e/cb1 0 2024-04-27T04:10:10.170 INFO:tasks.workunit.client.0.smithi162.stdout:8/526: chown d0/de/d20/d6c/f6d 100 1 2024-04-27T04:10:10.171 INFO:tasks.workunit.client.0.smithi162.stdout:1/479: link d0/lf d0/d7/d10/l9b 0 2024-04-27T04:10:10.171 INFO:tasks.workunit.client.0.smithi162.stdout:1/480: fdatasync d0/d11/f4d 0 2024-04-27T04:10:10.172 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:10:09 smithi181 ceph-mon[33774]: mgrmap e23: smithi181.vtwuhz(active, since 3s) 2024-04-27T04:10:10.172 INFO:tasks.workunit.client.0.smithi162.stdout:8/527: write d0/d10/d25/d65/f9e [52335,14809] 0 2024-04-27T04:10:10.173 INFO:tasks.workunit.client.0.smithi162.stdout:1/481: write d0/d7/f5a [677976,16152] 0 2024-04-27T04:10:10.174 INFO:tasks.workunit.client.0.smithi162.stdout:8/528: fdatasync d0/de/d6e/f77 0 2024-04-27T04:10:10.175 INFO:tasks.workunit.client.0.smithi162.stdout:9/521: dread d3/d13/d2d/d3b/f84 [0,4194304] 0 2024-04-27T04:10:10.175 INFO:tasks.workunit.client.0.smithi162.stdout:4/496: rmdir d2/d25/d1c/d39 39 2024-04-27T04:10:10.177 INFO:tasks.workunit.client.0.smithi162.stdout:5/541: creat d8/fad x:0 0 0 2024-04-27T04:10:10.179 INFO:tasks.workunit.client.0.smithi162.stdout:1/482: read d0/d7/d10/f64 [1628835,67528] 0 2024-04-27T04:10:10.180 INFO:tasks.workunit.client.0.smithi162.stdout:7/489: getdents d3/d5/d5b/d83 0 2024-04-27T04:10:10.180 INFO:tasks.workunit.client.0.smithi162.stdout:6/528: mknod d6/d1a/d34/d50/ca4 0 2024-04-27T04:10:10.183 INFO:tasks.workunit.client.0.smithi162.stdout:0/570: dread d5/dc/d1e/d46/f52 [0,4194304] 0 2024-04-27T04:10:10.184 INFO:tasks.workunit.client.0.smithi162.stdout:8/529: dwrite d0/d10/d25/f14 [4194304,4194304] 0 2024-04-27T04:10:10.184 INFO:tasks.workunit.client.0.smithi162.stdout:0/571: chown d5/c49 0 1 2024-04-27T04:10:10.186 INFO:tasks.workunit.client.0.smithi162.stdout:0/572: truncate d5/d26/da2/fb7 706174 0 2024-04-27T04:10:10.189 INFO:tasks.workunit.client.0.smithi162.stdout:9/522: rmdir d3/d13/d2d 39 2024-04-27T04:10:10.193 INFO:tasks.workunit.client.0.smithi162.stdout:8/530: dwrite d0/de/d6e/d71/f95 [0,4194304] 0 2024-04-27T04:10:10.193 INFO:tasks.workunit.client.0.smithi162.stdout:8/531: fsync d0/de/f7b 0 2024-04-27T04:10:10.194 INFO:tasks.workunit.client.0.smithi162.stdout:5/542: rename d8/d9/d55/l9a to d8/d1a/d29/d4a/d9f/lae 0 2024-04-27T04:10:10.196 INFO:tasks.workunit.client.0.smithi162.stdout:1/483: mknod d0/d80/d89/c9c 0 2024-04-27T04:10:10.197 INFO:tasks.workunit.client.0.smithi162.stdout:5/543: write d8/d1a/d23/d40/d4b/f52 [7990024,48061] 0 2024-04-27T04:10:10.199 INFO:tasks.workunit.client.1.smithi181.stdout:6/581: dwrite d7/dc/d1c/d7f/d3e/f47 [0,4194304] 0 2024-04-27T04:10:10.200 INFO:tasks.workunit.client.0.smithi162.stdout:7/490: creat d3/d39/d57/fa9 x:0 0 0 2024-04-27T04:10:10.200 INFO:tasks.workunit.client.0.smithi162.stdout:3/503: dread d1/d18/f12 [0,4194304] 0 2024-04-27T04:10:10.201 INFO:tasks.workunit.client.0.smithi162.stdout:7/491: write d3/d5/d3c/d4a/f95 [758641,7403] 0 2024-04-27T04:10:10.203 INFO:tasks.workunit.client.0.smithi162.stdout:7/492: write d3/d5/d3c/f7e [4371493,90033] 0 2024-04-27T04:10:10.205 INFO:tasks.workunit.client.0.smithi162.stdout:6/529: truncate d6/db/d32/f3b 1050654 0 2024-04-27T04:10:10.206 INFO:tasks.workunit.client.0.smithi162.stdout:6/530: truncate d6/d1a/d34/d50/f93 981862 0 2024-04-27T04:10:10.210 INFO:tasks.workunit.client.0.smithi162.stdout:0/573: rmdir d5/dc/d1e/d41 39 2024-04-27T04:10:10.215 INFO:tasks.workunit.client.0.smithi162.stdout:9/523: creat d3/d11/d45/f99 x:0 0 0 2024-04-27T04:10:10.219 INFO:tasks.workunit.client.1.smithi181.stdout:9/624: dwrite d0/f37 [0,4194304] 0 2024-04-27T04:10:10.222 INFO:tasks.workunit.client.0.smithi162.stdout:1/484: creat d0/d11/d50/d81/f9d x:0 0 0 2024-04-27T04:10:10.227 INFO:tasks.workunit.client.0.smithi162.stdout:2/448: dwrite d0/d4/d2a/d3b/d49/d5a/d63/d6e/f7a [0,4194304] 0 2024-04-27T04:10:10.234 INFO:tasks.workunit.client.0.smithi162.stdout:7/493: mkdir d3/d5/d5b/d83/de/d2b/d81/daa 0 2024-04-27T04:10:10.236 INFO:tasks.workunit.client.0.smithi162.stdout:6/531: mknod d6/d1a/d34/d7b/d7e/ca5 0 2024-04-27T04:10:10.245 INFO:tasks.workunit.client.0.smithi162.stdout:3/504: creat d1/d70/fa5 x:0 0 0 2024-04-27T04:10:10.246 INFO:tasks.workunit.client.0.smithi162.stdout:3/505: truncate d1/d65/f7f 1831677 0 2024-04-27T04:10:10.247 INFO:tasks.workunit.client.0.smithi162.stdout:3/506: truncate d1/d18/d23/d58/f9d 262286 0 2024-04-27T04:10:10.248 INFO:tasks.workunit.client.0.smithi162.stdout:7/494: mkdir d3/d5/d5b/d60/d65/d80/dab 0 2024-04-27T04:10:10.248 INFO:tasks.workunit.client.0.smithi162.stdout:7/495: fsync d3/d5/f5f 0 2024-04-27T04:10:10.251 INFO:tasks.workunit.client.0.smithi162.stdout:9/524: creat d3/d23/d7c/f9a x:0 0 0 2024-04-27T04:10:10.253 INFO:tasks.workunit.client.0.smithi162.stdout:1/485: rename d0/d7/d2a/f30 to d0/d11/d44/f9e 0 2024-04-27T04:10:10.254 INFO:tasks.workunit.client.1.smithi181.stdout:3/655: write d0/d6c/d52/f59 [2272193,56346] 0 2024-04-27T04:10:10.254 INFO:tasks.workunit.client.0.smithi162.stdout:9/525: truncate d3/d11/d45/d4e/d58/f80 5123550 0 2024-04-27T04:10:10.254 INFO:tasks.workunit.client.0.smithi162.stdout:2/449: symlink d0/d4/d17/d4f/d7f/l8c 0 2024-04-27T04:10:10.254 INFO:tasks.workunit.client.0.smithi162.stdout:7/496: dwrite d3/d4f/f67 [0,4194304] 0 2024-04-27T04:10:10.256 INFO:tasks.workunit.client.0.smithi162.stdout:7/497: fsync d3/d5/d9/f26 0 2024-04-27T04:10:10.257 INFO:tasks.workunit.client.0.smithi162.stdout:7/498: stat d3/d5/d5b/d60/d65/d80/f84 0 2024-04-27T04:10:10.258 INFO:tasks.workunit.client.0.smithi162.stdout:7/499: chown d3/d5/d5b/d83/de/f1c 3029 1 2024-04-27T04:10:10.261 INFO:tasks.workunit.client.0.smithi162.stdout:0/574: rename d5/dc/d37 to d5/dc/d1e/d46/d48/d9e/db8 0 2024-04-27T04:10:10.264 INFO:tasks.workunit.client.0.smithi162.stdout:1/486: creat d0/d80/f9f x:0 0 0 2024-04-27T04:10:10.270 INFO:tasks.workunit.client.0.smithi162.stdout:1/487: dwrite d0/d7/d10/f3d [0,4194304] 0 2024-04-27T04:10:10.273 INFO:tasks.workunit.client.0.smithi162.stdout:6/532: creat d6/d1a/fa6 x:0 0 0 2024-04-27T04:10:10.276 INFO:tasks.workunit.client.0.smithi162.stdout:7/500: fdatasync d3/d4f/f67 0 2024-04-27T04:10:10.277 INFO:tasks.workunit.client.0.smithi162.stdout:1/488: dwrite d0/d7/d2a/f31 [0,4194304] 0 2024-04-27T04:10:10.279 INFO:tasks.workunit.client.0.smithi162.stdout:3/507: rename d1/d45/d69/c8e to d1/d18/d4f/ca6 0 2024-04-27T04:10:10.281 INFO:tasks.workunit.client.0.smithi162.stdout:1/489: read d0/d11/d3e/d5f/f6a [126559,61994] 0 2024-04-27T04:10:10.282 INFO:tasks.workunit.client.0.smithi162.stdout:0/575: creat d5/dc/d1e/d46/d48/d9e/db8/fb9 x:0 0 0 2024-04-27T04:10:10.286 INFO:tasks.workunit.client.0.smithi162.stdout:9/526: symlink d3/d13/d2d/d44/l9b 0 2024-04-27T04:10:10.287 INFO:tasks.workunit.client.0.smithi162.stdout:9/527: readlink d3/d13/d2d/l3a 0 2024-04-27T04:10:10.288 INFO:tasks.workunit.client.0.smithi162.stdout:9/528: write d3/d11/d45/d4e/d58/f98 [980952,82244] 0 2024-04-27T04:10:10.290 INFO:tasks.workunit.client.1.smithi181.stdout:0/804: mknod dc/d1f/d6b/db9/c118 0 2024-04-27T04:10:10.292 INFO:tasks.workunit.client.1.smithi181.stdout:6/582: truncate d7/dc/d1c/f35 1647871 0 2024-04-27T04:10:10.295 INFO:tasks.workunit.client.0.smithi162.stdout:5/544: symlink d8/d1a/d29/laf 0 2024-04-27T04:10:10.295 INFO:tasks.workunit.client.0.smithi162.stdout:5/545: fsync d8/d1a/f81 0 2024-04-27T04:10:10.300 INFO:tasks.workunit.client.1.smithi181.stdout:6/583: dwrite d7/d27/fc9 [0,4194304] 0 2024-04-27T04:10:10.302 INFO:tasks.workunit.client.0.smithi162.stdout:3/508: readlink d1/l3d 0 2024-04-27T04:10:10.307 INFO:tasks.workunit.client.0.smithi162.stdout:0/576: mkdir d5/dc/d1e/d20/dba 0 2024-04-27T04:10:10.310 INFO:tasks.workunit.client.0.smithi162.stdout:9/529: mknod d3/d23/d1e/d43/c9c 0 2024-04-27T04:10:10.311 INFO:tasks.workunit.client.1.smithi181.stdout:3/656: rename d0/d6c/f4c to d0/d6c/d52/fd0 0 2024-04-27T04:10:10.318 INFO:tasks.workunit.client.0.smithi162.stdout:5/546: unlink d8/d1a/d29/f49 0 2024-04-27T04:10:10.319 INFO:tasks.workunit.client.1.smithi181.stdout:7/685: mknod d1/d2/d1c/d2d/d61/d80/cd3 0 2024-04-27T04:10:10.324 INFO:tasks.workunit.client.0.smithi162.stdout:3/509: creat d1/d18/d5d/d5f/d66/fa7 x:0 0 0 2024-04-27T04:10:10.326 INFO:tasks.workunit.client.1.smithi181.stdout:7/686: dread d1/d22/d15/d29/f84 [0,4194304] 0 2024-04-27T04:10:10.327 INFO:tasks.workunit.client.1.smithi181.stdout:6/584: dread d7/dc/d8b/fa7 [0,4194304] 0 2024-04-27T04:10:10.328 INFO:tasks.workunit.client.0.smithi162.stdout:8/532: sync 2024-04-27T04:10:10.328 INFO:tasks.workunit.client.0.smithi162.stdout:0/577: write d5/dc/d1e/d41/f83 [4655055,14178] 0 2024-04-27T04:10:10.329 INFO:tasks.workunit.client.1.smithi181.stdout:0/805: link dc/d1f/daa/f103 dc/d1f/d6b/db9/d106/f119 0 2024-04-27T04:10:10.329 INFO:tasks.workunit.client.0.smithi162.stdout:0/578: readlink d5/dc/d1e/d56/l75 0 2024-04-27T04:10:10.330 INFO:tasks.workunit.client.0.smithi162.stdout:9/530: readlink d3/d13/d2d/d4f/l59 0 2024-04-27T04:10:10.331 INFO:tasks.workunit.client.1.smithi181.stdout:3/657: mknod d0/d6c/d52/dbb/dca/cd1 0 2024-04-27T04:10:10.332 INFO:tasks.workunit.client.0.smithi162.stdout:2/450: getdents d0/d4/d2a/d3b/d49/d5a/d63 0 2024-04-27T04:10:10.334 INFO:tasks.workunit.client.0.smithi162.stdout:5/547: truncate d8/d1a/d23/d40/f46 502674 0 2024-04-27T04:10:10.335 INFO:tasks.workunit.client.0.smithi162.stdout:4/497: write d2/da/f12 [2301775,111279] 0 2024-04-27T04:10:10.338 INFO:tasks.workunit.client.1.smithi181.stdout:6/585: mkdir d7/dc/dc3/d72/d98/dbf/dca 0 2024-04-27T04:10:10.339 INFO:tasks.workunit.client.0.smithi162.stdout:2/451: dwrite d0/d24/f38 [0,4194304] 0 2024-04-27T04:10:10.345 INFO:tasks.workunit.client.0.smithi162.stdout:1/490: rename d0/d11/d44/c48 to d0/d11/d50/ca0 0 2024-04-27T04:10:10.347 INFO:tasks.workunit.client.0.smithi162.stdout:8/533: mkdir d0/d10/d44/db2 0 2024-04-27T04:10:10.348 INFO:tasks.workunit.client.0.smithi162.stdout:9/531: dread d3/fb [0,4194304] 0 2024-04-27T04:10:10.348 INFO:tasks.workunit.client.1.smithi181.stdout:7/687: mknod d1/d2/d1c/d2d/cd4 0 2024-04-27T04:10:10.349 INFO:tasks.workunit.client.1.smithi181.stdout:7/688: write d1/d22/f26 [3786167,44737] 0 2024-04-27T04:10:10.350 INFO:tasks.workunit.client.1.smithi181.stdout:7/689: fsync d1/d2/f25 0 2024-04-27T04:10:10.353 INFO:tasks.workunit.client.0.smithi162.stdout:4/498: unlink d2/da/dd/l14 0 2024-04-27T04:10:10.355 INFO:tasks.workunit.client.0.smithi162.stdout:3/510: truncate d1/d18/f1a 1936414 0 2024-04-27T04:10:10.364 INFO:tasks.workunit.client.1.smithi181.stdout:3/658: fsync d0/de/d20/f46 0 2024-04-27T04:10:10.367 INFO:tasks.workunit.client.1.smithi181.stdout:6/586: creat d7/dc/d1c/d88/fcb x:0 0 0 2024-04-27T04:10:10.367 INFO:tasks.workunit.client.1.smithi181.stdout:6/587: read - d7/d31/f75 zero size 2024-04-27T04:10:10.370 INFO:tasks.workunit.client.1.smithi181.stdout:0/806: rename dc/d1f/d6b/db9/dba/deb/f8c to dc/d1f/d6b/db9/dba/deb/f11a 0 2024-04-27T04:10:10.374 INFO:tasks.workunit.client.1.smithi181.stdout:6/588: fsync d7/f59 0 2024-04-27T04:10:10.377 INFO:tasks.workunit.client.1.smithi181.stdout:0/807: symlink dc/d1f/l11b 0 2024-04-27T04:10:10.381 INFO:tasks.workunit.client.0.smithi162.stdout:6/533: sync 2024-04-27T04:10:10.382 INFO:tasks.workunit.client.1.smithi181.stdout:6/589: rmdir d7/dc/d1c/d7f/d3e 39 2024-04-27T04:10:10.386 INFO:tasks.workunit.client.1.smithi181.stdout:3/659: creat d0/fd2 x:0 0 0 2024-04-27T04:10:10.387 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:10:09 smithi162 ceph-mon[26841]: mgrmap e23: smithi181.vtwuhz(active, since 3s) 2024-04-27T04:10:10.389 INFO:tasks.workunit.client.1.smithi181.stdout:6/590: creat d7/dc/dc3/d62/fcc x:0 0 0 2024-04-27T04:10:10.389 INFO:tasks.workunit.client.1.smithi181.stdout:6/591: stat d7/dc/d1e/d66/cac 0 2024-04-27T04:10:10.390 INFO:tasks.workunit.client.1.smithi181.stdout:0/808: truncate dc/d1f/d3b/d4a/d95/d26/f28 3377786 0 2024-04-27T04:10:10.398 INFO:tasks.workunit.client.1.smithi181.stdout:3/660: mknod d0/d6c/d52/d44/d56/cd3 0 2024-04-27T04:10:10.402 INFO:tasks.workunit.client.1.smithi181.stdout:6/592: rename d7/dc/f3b to d7/d27/fcd 0 2024-04-27T04:10:10.404 INFO:tasks.workunit.client.1.smithi181.stdout:0/809: mkdir dc/d1f/d6b/db9/d106/d110/d11c 0 2024-04-27T04:10:10.405 INFO:tasks.workunit.client.1.smithi181.stdout:0/810: readlink dc/d1f/d3b/d4a/d7a/ld9 0 2024-04-27T04:10:10.406 INFO:tasks.workunit.client.1.smithi181.stdout:6/593: dread d7/dc/d1e/f51 [0,4194304] 0 2024-04-27T04:10:10.408 INFO:tasks.workunit.client.0.smithi162.stdout:5/548: sync 2024-04-27T04:10:10.408 INFO:tasks.workunit.client.0.smithi162.stdout:5/549: readlink d8/l2b 0 2024-04-27T04:10:10.408 INFO:tasks.workunit.client.0.smithi162.stdout:5/550: fsync d8/d1a/f81 0 2024-04-27T04:10:10.410 INFO:tasks.workunit.client.0.smithi162.stdout:1/491: mknod d0/d7/d3c/d54/ca1 0 2024-04-27T04:10:10.410 INFO:tasks.workunit.client.0.smithi162.stdout:1/492: chown d0/d11/d3e/d59 1 1 2024-04-27T04:10:10.411 INFO:tasks.workunit.client.0.smithi162.stdout:2/452: truncate d0/d4/f16 1101168 0 2024-04-27T04:10:10.412 INFO:tasks.workunit.client.1.smithi181.stdout:3/661: chown d0/d6c/da1/daf/fb9 6 1 2024-04-27T04:10:10.414 INFO:tasks.workunit.client.0.smithi162.stdout:8/534: rename d0/d10/d25/f91 to d0/de/d6e/d71/fb3 0 2024-04-27T04:10:10.417 INFO:tasks.workunit.client.0.smithi162.stdout:9/532: mknod d3/d13/d2d/d3b/d72/c9d 0 2024-04-27T04:10:10.418 INFO:tasks.workunit.client.1.smithi181.stdout:0/811: truncate dc/d1f/d3b/d4a/d95/f4d 921564 0 2024-04-27T04:10:10.420 INFO:tasks.workunit.client.0.smithi162.stdout:4/499: fdatasync d2/d25/d1c/f3d 0 2024-04-27T04:10:10.422 INFO:tasks.workunit.client.1.smithi181.stdout:9/625: dwrite d0/d44/f6e [0,4194304] 0 2024-04-27T04:10:10.422 INFO:tasks.workunit.client.1.smithi181.stdout:9/626: fdatasync d0/f1e 0 2024-04-27T04:10:10.423 INFO:tasks.workunit.client.1.smithi181.stdout:3/662: unlink d0/de/d20/l7a 0 2024-04-27T04:10:10.426 INFO:tasks.workunit.client.0.smithi162.stdout:3/511: rmdir d1/d18/d23 39 2024-04-27T04:10:10.428 INFO:tasks.workunit.client.0.smithi162.stdout:9/533: dread d3/d13/d2d/d3b/d54/f87 [0,4194304] 0 2024-04-27T04:10:10.429 INFO:tasks.workunit.client.0.smithi162.stdout:6/534: symlink d6/db/la7 0 2024-04-27T04:10:10.434 INFO:tasks.workunit.client.0.smithi162.stdout:5/551: creat d8/d9/fb0 x:0 0 0 2024-04-27T04:10:10.438 INFO:tasks.workunit.client.0.smithi162.stdout:1/493: mknod d0/d11/d3e/d59/d7f/ca2 0 2024-04-27T04:10:10.438 INFO:tasks.workunit.client.0.smithi162.stdout:1/494: chown d0 14 1 2024-04-27T04:10:10.439 INFO:tasks.workunit.client.1.smithi181.stdout:9/627: mknod d0/d44/d4b/d3d/dbd/dd1/cd9 0 2024-04-27T04:10:10.447 INFO:tasks.workunit.client.0.smithi162.stdout:0/579: creat d5/dc/d1e/d46/d48/fbb x:0 0 0 2024-04-27T04:10:10.450 INFO:tasks.workunit.client.1.smithi181.stdout:9/628: dwrite d0/d40/fb2 [4194304,4194304] 0 2024-04-27T04:10:10.451 INFO:tasks.workunit.client.1.smithi181.stdout:9/629: chown d0/d44/d4b/d5f/lad 0 1 2024-04-27T04:10:10.452 INFO:tasks.workunit.client.0.smithi162.stdout:3/512: truncate d1/d18/d5d/d5f/f77 4642861 0 2024-04-27T04:10:10.454 INFO:tasks.workunit.client.0.smithi162.stdout:9/534: symlink d3/d13/d15/d50/l9e 0 2024-04-27T04:10:10.459 INFO:tasks.workunit.client.1.smithi181.stdout:9/630: dwrite d0/f15 [4194304,4194304] 0 2024-04-27T04:10:10.463 INFO:tasks.workunit.client.0.smithi162.stdout:1/495: fsync d0/f37 0 2024-04-27T04:10:10.463 INFO:tasks.workunit.client.1.smithi181.stdout:9/631: dread d0/d44/d4b/d3d/f3e [0,4194304] 0 2024-04-27T04:10:10.464 INFO:tasks.workunit.client.0.smithi162.stdout:1/496: stat d0/d7/d3c/d6e/c85 0 2024-04-27T04:10:10.465 INFO:tasks.workunit.client.1.smithi181.stdout:0/812: dread dc/d1f/d3b/d4a/d95/d26/d41/d6f/f69 [0,4194304] 0 2024-04-27T04:10:10.467 INFO:tasks.workunit.client.0.smithi162.stdout:0/580: creat d5/dc/d1e/d46/d48/d9e/fbc x:0 0 0 2024-04-27T04:10:10.469 INFO:tasks.workunit.client.1.smithi181.stdout:7/690: dwrite d1/d22/d15/f28 [0,4194304] 0 2024-04-27T04:10:10.474 INFO:tasks.workunit.client.0.smithi162.stdout:4/500: mknod d2/d25/d1c/d39/d3b/d47/c9f 0 2024-04-27T04:10:10.474 INFO:tasks.workunit.client.0.smithi162.stdout:3/513: dread d1/d45/f87 [0,4194304] 0 2024-04-27T04:10:10.474 INFO:tasks.workunit.client.1.smithi181.stdout:9/632: readlink d0/d4d/l6f 0 2024-04-27T04:10:10.475 INFO:tasks.workunit.client.1.smithi181.stdout:9/633: write d0/d44/d4b/d43/d89/fc5 [458549,13392] 0 2024-04-27T04:10:10.476 INFO:tasks.workunit.client.0.smithi162.stdout:9/535: creat d3/d23/d5d/f9f x:0 0 0 2024-04-27T04:10:10.487 INFO:tasks.workunit.client.1.smithi181.stdout:7/691: fsync d1/d2/d35/d67/f51 0 2024-04-27T04:10:10.488 INFO:tasks.workunit.client.0.smithi162.stdout:0/581: mknod d5/d60/d98/cbd 0 2024-04-27T04:10:10.494 INFO:tasks.workunit.client.0.smithi162.stdout:4/501: symlink d2/d25/d1c/d39/d5e/la0 0 2024-04-27T04:10:10.496 INFO:tasks.workunit.client.1.smithi181.stdout:7/692: mknod d1/d2/d35/d67/cd5 0 2024-04-27T04:10:10.497 INFO:tasks.workunit.client.0.smithi162.stdout:9/536: truncate d3/d13/d2d/f37 142638 0 2024-04-27T04:10:10.497 INFO:tasks.workunit.client.1.smithi181.stdout:7/693: truncate d1/d2/f27 9003298 0 2024-04-27T04:10:10.502 INFO:tasks.workunit.client.0.smithi162.stdout:0/582: rename d5/l13 to d5/d60/d98/lbe 0 2024-04-27T04:10:10.504 INFO:tasks.workunit.client.1.smithi181.stdout:7/694: rename d1/d2/d35/d67/cd5 to d1/d2/d35/cd6 0 2024-04-27T04:10:10.505 INFO:tasks.workunit.client.1.smithi181.stdout:7/695: chown d1/d22/d15/f28 905 1 2024-04-27T04:10:10.506 INFO:tasks.workunit.client.0.smithi162.stdout:0/583: dread d5/dc/f17 [0,4194304] 0 2024-04-27T04:10:10.507 INFO:tasks.workunit.client.0.smithi162.stdout:0/584: write d5/f9 [185495,31756] 0 2024-04-27T04:10:10.507 INFO:tasks.workunit.client.0.smithi162.stdout:4/502: creat d2/da/fa1 x:0 0 0 2024-04-27T04:10:10.508 INFO:tasks.workunit.client.0.smithi162.stdout:2/453: sync 2024-04-27T04:10:10.508 INFO:tasks.workunit.client.0.smithi162.stdout:9/537: mknod d3/d13/d2d/ca0 0 2024-04-27T04:10:10.509 INFO:tasks.workunit.client.1.smithi181.stdout:7/696: mknod d1/d2/cd7 0 2024-04-27T04:10:10.510 INFO:tasks.workunit.client.0.smithi162.stdout:2/454: write d0/f44 [4821971,475] 0 2024-04-27T04:10:10.510 INFO:tasks.workunit.client.0.smithi162.stdout:1/497: link d0/l3 d0/d7/d3c/d77/la3 0 2024-04-27T04:10:10.511 INFO:tasks.workunit.client.0.smithi162.stdout:9/538: chown d3/d13/d2d/d3b/d72/c9d 257677 1 2024-04-27T04:10:10.511 INFO:tasks.workunit.client.0.smithi162.stdout:2/455: chown d0/d4/d17/d4f/d7f 5 1 2024-04-27T04:10:10.513 INFO:tasks.workunit.client.0.smithi162.stdout:7/501: dread d3/d5/d5b/d83/de/d2b/f69 [0,4194304] 0 2024-04-27T04:10:10.514 INFO:tasks.workunit.client.1.smithi181.stdout:7/697: chown d1/d22/d15/d1f/d34/f3c 36 1 2024-04-27T04:10:10.515 INFO:tasks.workunit.client.0.smithi162.stdout:7/502: fdatasync d3/d5/d5b/d60/d65/d80/f84 0 2024-04-27T04:10:10.516 INFO:tasks.workunit.client.0.smithi162.stdout:7/503: readlink d3/d5/d5b/d60/l68 0 2024-04-27T04:10:10.517 INFO:tasks.workunit.client.0.smithi162.stdout:2/456: dwrite d0/d24/f4d [8388608,4194304] 0 2024-04-27T04:10:10.519 INFO:tasks.workunit.client.0.smithi162.stdout:2/457: write d0/d4/d2a/d3b/f74 [85095,81353] 0 2024-04-27T04:10:10.520 INFO:tasks.workunit.client.0.smithi162.stdout:2/458: readlink d0/d4/d17/d4f/d7f/l8c 0 2024-04-27T04:10:10.524 INFO:tasks.workunit.client.0.smithi162.stdout:0/585: dread d5/dc/d1e/d41/f71 [0,4194304] 0 2024-04-27T04:10:10.525 INFO:tasks.workunit.client.0.smithi162.stdout:1/498: creat d0/d7/d3c/d54/fa4 x:0 0 0 2024-04-27T04:10:10.530 INFO:tasks.workunit.client.1.smithi181.stdout:6/594: dwrite d7/dc/d1c/d88/f95 [0,4194304] 0 2024-04-27T04:10:10.532 INFO:tasks.workunit.client.0.smithi162.stdout:9/539: creat d3/d11/d45/fa1 x:0 0 0 2024-04-27T04:10:10.532 INFO:tasks.workunit.client.1.smithi181.stdout:7/698: unlink d1/d2/d1c/c49 0 2024-04-27T04:10:10.533 INFO:tasks.workunit.client.0.smithi162.stdout:9/540: write d3/d23/f88 [1466185,67209] 0 2024-04-27T04:10:10.543 INFO:tasks.workunit.client.1.smithi181.stdout:7/699: rename d1/d2/d1c/d55 to d1/d22/d15/dc6/dd8 0 2024-04-27T04:10:10.544 INFO:tasks.workunit.client.1.smithi181.stdout:7/700: stat d1/d2/d1c/d37/d91/fb0 0 2024-04-27T04:10:10.545 INFO:tasks.workunit.client.1.smithi181.stdout:6/595: dread d7/dc/dc3/f70 [0,4194304] 0 2024-04-27T04:10:10.546 INFO:tasks.workunit.client.1.smithi181.stdout:3/663: write d0/de/d20/f62 [1143985,87707] 0 2024-04-27T04:10:10.547 INFO:tasks.workunit.client.1.smithi181.stdout:7/701: read - d1/d22/f95 zero size 2024-04-27T04:10:10.548 INFO:tasks.workunit.client.0.smithi162.stdout:7/504: fdatasync d3/d5/d5b/d83/f2e 0 2024-04-27T04:10:10.551 INFO:tasks.workunit.client.0.smithi162.stdout:2/459: mknod d0/d4/d17/d4f/d7f/c8d 0 2024-04-27T04:10:10.553 INFO:tasks.workunit.client.1.smithi181.stdout:6/596: fdatasync d7/f26 0 2024-04-27T04:10:10.554 INFO:tasks.workunit.client.0.smithi162.stdout:8/535: chown d0/d10/d25/f93 3 1 2024-04-27T04:10:10.555 INFO:tasks.workunit.client.0.smithi162.stdout:7/505: dwrite d3/d5/d5b/d83/de/d2b/d81/f9c [0,4194304] 0 2024-04-27T04:10:10.556 INFO:tasks.workunit.client.1.smithi181.stdout:7/702: dread - d1/d2/d1c/d2d/d61/d80/fc7 zero size 2024-04-27T04:10:10.557 INFO:tasks.workunit.client.0.smithi162.stdout:7/506: write d3/d5/d5b/d83/de/f1c [1002997,107684] 0 2024-04-27T04:10:10.557 INFO:tasks.workunit.client.1.smithi181.stdout:7/703: fsync d1/dc3/fcf 0 2024-04-27T04:10:10.559 INFO:tasks.workunit.client.0.smithi162.stdout:1/499: rmdir d0/d11/d44 39 2024-04-27T04:10:10.562 INFO:tasks.workunit.client.1.smithi181.stdout:3/664: mknod d0/d6c/d52/d44/cd4 0 2024-04-27T04:10:10.562 INFO:tasks.workunit.client.0.smithi162.stdout:5/552: fsync d8/d1a/d23/f50 0 2024-04-27T04:10:10.564 INFO:tasks.workunit.client.1.smithi181.stdout:0/813: dwrite dc/d93/fe0 [0,4194304] 0 2024-04-27T04:10:10.564 INFO:tasks.workunit.client.0.smithi162.stdout:6/535: dwrite d6/db/d32/f3b [0,4194304] 0 2024-04-27T04:10:10.576 INFO:tasks.workunit.client.1.smithi181.stdout:9/634: dwrite d0/f2b [0,4194304] 0 2024-04-27T04:10:10.579 INFO:tasks.workunit.client.1.smithi181.stdout:0/814: creat dc/d1f/d3b/d4a/d95/d26/d41/d6f/d7d/dee/f11d x:0 0 0 2024-04-27T04:10:10.581 INFO:tasks.workunit.client.0.smithi162.stdout:2/460: mkdir d0/d4/d2a/d3b/d49/d5a/d63/d70/d8e 0 2024-04-27T04:10:10.582 INFO:tasks.workunit.client.0.smithi162.stdout:2/461: chown d0/d4/d17/d4f/c5c 8372 1 2024-04-27T04:10:10.586 INFO:tasks.workunit.client.1.smithi181.stdout:3/665: rename d0/de/d20 to d0/d6c/d52/d44/dd5 0 2024-04-27T04:10:10.586 INFO:tasks.workunit.client.0.smithi162.stdout:8/536: fdatasync d0/d10/d25/f36 0 2024-04-27T04:10:10.587 INFO:tasks.workunit.client.0.smithi162.stdout:8/537: stat d0/de/d6e 0 2024-04-27T04:10:10.592 INFO:tasks.workunit.client.0.smithi162.stdout:7/507: unlink d3/d5/d3c/d4d/f8e 0 2024-04-27T04:10:10.592 INFO:tasks.workunit.client.0.smithi162.stdout:7/508: readlink d3/d39/l7d 0 2024-04-27T04:10:10.593 INFO:tasks.workunit.client.0.smithi162.stdout:7/509: truncate d3/f93 889374 0 2024-04-27T04:10:10.594 INFO:tasks.workunit.client.1.smithi181.stdout:3/666: dwrite d0/d6c/d52/d44/d56/fa6 [0,4194304] 0 2024-04-27T04:10:10.596 INFO:tasks.workunit.client.0.smithi162.stdout:3/514: write d1/d18/d23/f2e [1503815,92684] 0 2024-04-27T04:10:10.601 INFO:tasks.workunit.client.0.smithi162.stdout:6/536: dread d6/db/d32/f45 [0,4194304] 0 2024-04-27T04:10:10.603 INFO:tasks.workunit.client.0.smithi162.stdout:6/537: readlink d6/d1a/d34/d7b/l92 0 2024-04-27T04:10:10.606 INFO:tasks.workunit.client.0.smithi162.stdout:7/510: dread d3/d5/d9/f24 [0,4194304] 0 2024-04-27T04:10:10.606 INFO:tasks.workunit.client.0.smithi162.stdout:7/511: fsync d3/d4f/f9d 0 2024-04-27T04:10:10.607 INFO:tasks.workunit.client.0.smithi162.stdout:7/512: read - d3/d5/d5b/d60/d65/d80/f97 zero size 2024-04-27T04:10:10.641 INFO:tasks.workunit.client.0.smithi162.stdout:2/462: unlink d0/d24/f4d 0 2024-04-27T04:10:10.643 INFO:tasks.workunit.client.0.smithi162.stdout:2/463: dread d0/d4/d2a/d41/f81 [0,4194304] 0 2024-04-27T04:10:10.643 INFO:tasks.workunit.client.1.smithi181.stdout:3/667: creat d0/d6c/d52/dbb/fd6 x:0 0 0 2024-04-27T04:10:10.645 INFO:tasks.workunit.client.0.smithi162.stdout:8/538: symlink d0/de/d20/lb4 0 2024-04-27T04:10:10.645 INFO:tasks.workunit.client.1.smithi181.stdout:9/635: getdents d0/d44/d4b/d43/d89 0 2024-04-27T04:10:10.647 INFO:tasks.workunit.client.0.smithi162.stdout:0/586: creat d5/dc/d1e/fbf x:0 0 0 2024-04-27T04:10:10.647 INFO:tasks.workunit.client.0.smithi162.stdout:8/539: write d0/de/d20/d6c/fa5 [424811,77587] 0 2024-04-27T04:10:10.649 INFO:tasks.workunit.client.1.smithi181.stdout:9/636: symlink d0/d44/d4b/d3d/lda 0 2024-04-27T04:10:10.649 INFO:tasks.workunit.client.0.smithi162.stdout:2/464: dwrite d0/d4/d2a/d41/d77/f7c [0,4194304] 0 2024-04-27T04:10:10.650 INFO:tasks.workunit.client.0.smithi162.stdout:2/465: readlink d0/l26 0 2024-04-27T04:10:10.653 INFO:tasks.workunit.client.0.smithi162.stdout:3/515: symlink d1/d21/d47/d57/la8 0 2024-04-27T04:10:10.654 INFO:tasks.workunit.client.0.smithi162.stdout:9/541: creat d3/d23/fa2 x:0 0 0 2024-04-27T04:10:10.655 INFO:tasks.workunit.client.0.smithi162.stdout:3/516: readlink d1/d18/d5d/l78 0 2024-04-27T04:10:10.656 INFO:tasks.workunit.client.0.smithi162.stdout:4/503: dwrite d2/f38 [0,4194304] 0 2024-04-27T04:10:10.656 INFO:tasks.workunit.client.1.smithi181.stdout:9/637: mkdir d0/d4d/da0/ddb 0 2024-04-27T04:10:10.657 INFO:tasks.workunit.client.0.smithi162.stdout:0/587: dwrite d5/dc/d1e/d41/f4e [4194304,4194304] 0 2024-04-27T04:10:10.658 INFO:tasks.workunit.client.1.smithi181.stdout:3/668: dread d0/f37 [0,4194304] 0 2024-04-27T04:10:10.659 INFO:tasks.workunit.client.0.smithi162.stdout:6/538: mknod d6/d1a/d34/d3a/d60/d74/ca8 0 2024-04-27T04:10:10.660 INFO:tasks.workunit.client.0.smithi162.stdout:5/553: sync 2024-04-27T04:10:10.660 INFO:tasks.workunit.client.0.smithi162.stdout:6/539: chown d6/d1a/d34/d7b/d7e 39 1 2024-04-27T04:10:10.661 INFO:tasks.workunit.client.0.smithi162.stdout:6/540: fdatasync d6/db/fa0 0 2024-04-27T04:10:10.664 INFO:tasks.workunit.client.1.smithi181.stdout:9/638: truncate d0/d40/f81 3188284 0 2024-04-27T04:10:10.664 INFO:tasks.workunit.client.1.smithi181.stdout:9/639: chown d0/d4d/d85/f90 1048784 1 2024-04-27T04:10:10.679 INFO:tasks.workunit.client.1.smithi181.stdout:3/669: fsync d0/d28/f42 0 2024-04-27T04:10:10.681 INFO:tasks.workunit.client.1.smithi181.stdout:7/704: write d1/d2/d1c/d37/d91/fb0 [407224,35281] 0 2024-04-27T04:10:10.682 INFO:tasks.workunit.client.1.smithi181.stdout:7/705: write d1/d22/fc0 [280165,100219] 0 2024-04-27T04:10:10.688 INFO:tasks.workunit.client.0.smithi162.stdout:3/517: write d1/d21/fa1 [1911814,126109] 0 2024-04-27T04:10:10.689 INFO:tasks.workunit.client.1.smithi181.stdout:7/706: dread d1/dc3/fa1 [0,4194304] 0 2024-04-27T04:10:10.692 INFO:tasks.workunit.client.1.smithi181.stdout:3/670: dread d0/d6c/d52/d44/dd5/f5d [0,4194304] 0 2024-04-27T04:10:10.694 INFO:tasks.workunit.client.1.smithi181.stdout:3/671: dread - d0/d6c/d52/d44/d56/faa zero size 2024-04-27T04:10:10.695 INFO:tasks.workunit.client.1.smithi181.stdout:7/707: dread d1/d22/d15/d29/dbe/d1b/f40 [0,4194304] 0 2024-04-27T04:10:10.696 INFO:tasks.workunit.client.0.smithi162.stdout:0/588: mkdir d5/d26/da2/dc0 0 2024-04-27T04:10:10.696 INFO:tasks.workunit.client.0.smithi162.stdout:1/500: write d0/f1b [3256150,57187] 0 2024-04-27T04:10:10.697 INFO:tasks.workunit.client.0.smithi162.stdout:0/589: chown d5/dc/d1e/d41/l9a 755674 1 2024-04-27T04:10:10.699 INFO:tasks.workunit.client.0.smithi162.stdout:7/513: dread d3/d5/d5b/d83/de/d2b/f2c [0,4194304] 0 2024-04-27T04:10:10.700 INFO:tasks.workunit.client.1.smithi181.stdout:3/672: symlink d0/d28/ld7 0 2024-04-27T04:10:10.701 INFO:tasks.workunit.client.1.smithi181.stdout:0/815: write dc/d1f/d3b/db2/fdb [328565,64208] 0 2024-04-27T04:10:10.701 INFO:tasks.workunit.client.1.smithi181.stdout:0/816: chown dc/d54 45 1 2024-04-27T04:10:10.701 INFO:tasks.workunit.client.0.smithi162.stdout:6/541: rmdir d6/d1a/d34/d7f 39 2024-04-27T04:10:10.704 INFO:tasks.workunit.client.1.smithi181.stdout:7/708: unlink d1/d2/c87 0 2024-04-27T04:10:10.712 INFO:tasks.workunit.client.0.smithi162.stdout:0/590: dread d5/d26/d4a/f62 [0,4194304] 0 2024-04-27T04:10:10.718 INFO:tasks.workunit.client.0.smithi162.stdout:3/518: truncate d1/d18/d23/d58/d63/f74 698963 0 2024-04-27T04:10:10.719 INFO:tasks.workunit.client.0.smithi162.stdout:1/501: mknod d0/d11/d3e/d5f/ca5 0 2024-04-27T04:10:10.720 INFO:tasks.workunit.client.0.smithi162.stdout:1/502: chown d0/f1e 464 1 2024-04-27T04:10:10.721 INFO:tasks.workunit.client.0.smithi162.stdout:1/503: dread - d0/d7/d2a/f92 zero size 2024-04-27T04:10:10.723 INFO:tasks.workunit.client.0.smithi162.stdout:7/514: rmdir d3/d4f 39 2024-04-27T04:10:10.725 INFO:tasks.workunit.client.1.smithi181.stdout:7/709: write d1/d2/d1c/d37/f59 [1657533,111536] 0 2024-04-27T04:10:10.728 INFO:tasks.workunit.client.1.smithi181.stdout:7/710: read d1/d2/f25 [3089180,20323] 0 2024-04-27T04:10:10.730 INFO:tasks.workunit.client.1.smithi181.stdout:9/640: dwrite d0/d4d/d85/f90 [0,4194304] 0 2024-04-27T04:10:10.735 INFO:tasks.workunit.client.1.smithi181.stdout:7/711: creat d1/d22/d15/d5f/d93/fd9 x:0 0 0 2024-04-27T04:10:10.736 INFO:tasks.workunit.client.1.smithi181.stdout:3/673: write d0/d6c/d52/d44/dd5/f46 [404382,66535] 0 2024-04-27T04:10:10.737 INFO:tasks.workunit.client.0.smithi162.stdout:4/504: dwrite d2/d25/d1c/d39/d3b/d47/f4a [4194304,4194304] 0 2024-04-27T04:10:10.737 INFO:tasks.workunit.client.1.smithi181.stdout:0/817: write dc/d1f/f7c [966283,47589] 0 2024-04-27T04:10:10.741 INFO:tasks.workunit.client.1.smithi181.stdout:7/712: write d1/d2/d1c/d37/f59 [764675,94819] 0 2024-04-27T04:10:10.742 INFO:tasks.workunit.client.0.smithi162.stdout:9/542: truncate d3/d23/f5c 1046659 0 2024-04-27T04:10:10.742 INFO:tasks.workunit.client.0.smithi162.stdout:5/554: dwrite d8/d1a/d29/d4a/d5c/f62 [0,4194304] 0 2024-04-27T04:10:10.745 INFO:tasks.workunit.client.0.smithi162.stdout:8/540: getdents d0/de/d6e/d71 0 2024-04-27T04:10:10.746 INFO:tasks.workunit.client.0.smithi162.stdout:2/466: truncate d0/d4/f16 1495197 0 2024-04-27T04:10:10.748 INFO:tasks.workunit.client.0.smithi162.stdout:0/591: mknod d5/dc/d1e/d46/cc1 0 2024-04-27T04:10:10.749 INFO:tasks.workunit.client.1.smithi181.stdout:3/674: truncate d0/d6c/d52/d44/d56/f63 374465 0 2024-04-27T04:10:10.749 INFO:tasks.workunit.client.0.smithi162.stdout:7/515: dwrite d3/d92/fa2 [4194304,4194304] 0 2024-04-27T04:10:10.750 INFO:tasks.workunit.client.0.smithi162.stdout:7/516: chown d3/d5/d3c/d4d 90 1 2024-04-27T04:10:10.750 INFO:tasks.workunit.client.1.smithi181.stdout:3/675: readlink d0/d6c/d52/d44/d56/d5b/l6f 0 2024-04-27T04:10:10.757 INFO:tasks.workunit.client.0.smithi162.stdout:6/542: rename d6/d1a/d34/d7b/d7e/ca5 to d6/ca9 0 2024-04-27T04:10:10.757 INFO:tasks.workunit.client.1.smithi181.stdout:0/818: mkdir dc/d1f/d6b/d7e/d91/dea/d11e 0 2024-04-27T04:10:10.759 INFO:tasks.workunit.client.1.smithi181.stdout:3/676: mknod d0/cd8 0 2024-04-27T04:10:10.760 INFO:tasks.workunit.client.0.smithi162.stdout:5/555: fsync d8/d9/f16 0 2024-04-27T04:10:10.760 INFO:tasks.workunit.client.0.smithi162.stdout:9/543: dread d3/d23/f3e [0,4194304] 0 2024-04-27T04:10:10.762 INFO:tasks.workunit.client.0.smithi162.stdout:8/541: unlink d0/d10/d2b/d2d/d2f/d39/f5a 0 2024-04-27T04:10:10.764 INFO:tasks.workunit.client.0.smithi162.stdout:8/542: fsync d0/d10/d2b/d2d/d2f/f54 0 2024-04-27T04:10:10.766 INFO:tasks.workunit.client.0.smithi162.stdout:5/556: dwrite d8/d1a/f2d [0,4194304] 0 2024-04-27T04:10:10.767 INFO:tasks.workunit.client.1.smithi181.stdout:3/677: creat d0/de/d7d/dbc/fd9 x:0 0 0 2024-04-27T04:10:10.768 INFO:tasks.workunit.client.0.smithi162.stdout:0/592: fdatasync d5/d26/f3c 0 2024-04-27T04:10:10.769 INFO:tasks.workunit.client.0.smithi162.stdout:0/593: chown d5/d26/d69/d9b/db4 110 1 2024-04-27T04:10:10.773 INFO:tasks.workunit.client.0.smithi162.stdout:3/519: dread d1/d21/f6d [0,4194304] 0 2024-04-27T04:10:10.774 INFO:tasks.workunit.client.0.smithi162.stdout:5/557: dwrite d8/d1a/f45 [0,4194304] 0 2024-04-27T04:10:10.775 INFO:tasks.workunit.client.1.smithi181.stdout:3/678: creat d0/fda x:0 0 0 2024-04-27T04:10:10.777 INFO:tasks.workunit.client.0.smithi162.stdout:5/558: truncate d8/d1a/d7d/fab 1028496 0 2024-04-27T04:10:10.778 INFO:tasks.workunit.client.1.smithi181.stdout:9/641: truncate d0/f26 6317164 0 2024-04-27T04:10:10.781 INFO:tasks.workunit.client.1.smithi181.stdout:3/679: mknod d0/d6c/da1/cdb 0 2024-04-27T04:10:10.782 INFO:tasks.workunit.client.1.smithi181.stdout:3/680: read d0/f32 [406144,30285] 0 2024-04-27T04:10:10.784 INFO:tasks.workunit.client.0.smithi162.stdout:5/559: dwrite d8/d1a/d23/d40/f77 [0,4194304] 0 2024-04-27T04:10:10.787 INFO:tasks.workunit.client.0.smithi162.stdout:9/544: creat d3/d13/d2d/d44/fa3 x:0 0 0 2024-04-27T04:10:10.787 INFO:tasks.workunit.client.0.smithi162.stdout:9/545: readlink d3/l7 0 2024-04-27T04:10:10.789 INFO:tasks.workunit.client.0.smithi162.stdout:5/560: dread d8/d1a/d7d/f91 [0,4194304] 0 2024-04-27T04:10:10.789 INFO:tasks.workunit.client.0.smithi162.stdout:2/467: truncate d0/f6 848421 0 2024-04-27T04:10:10.789 INFO:tasks.workunit.client.0.smithi162.stdout:5/561: fsync d8/d9/f3d 0 2024-04-27T04:10:10.789 INFO:tasks.workunit.client.0.smithi162.stdout:2/468: chown d0/l4c 58374 1 2024-04-27T04:10:10.791 INFO:tasks.workunit.client.0.smithi162.stdout:5/562: chown d8/d9/d5a/c72 298999220 1 2024-04-27T04:10:10.794 INFO:tasks.workunit.client.1.smithi181.stdout:9/642: symlink d0/d44/d4b/ldc 0 2024-04-27T04:10:10.795 INFO:tasks.workunit.client.0.smithi162.stdout:2/469: dread d0/d4/d2a/f3e [0,4194304] 0 2024-04-27T04:10:10.795 INFO:tasks.workunit.client.0.smithi162.stdout:9/546: dwrite d3/d13/d2d/d3b/d54/f8f [0,4194304] 0 2024-04-27T04:10:10.797 INFO:tasks.workunit.client.0.smithi162.stdout:2/470: write d0/d24/f87 [223380,18104] 0 2024-04-27T04:10:10.800 INFO:tasks.workunit.client.1.smithi181.stdout:9/643: unlink d0/d4d/da0/la2 0 2024-04-27T04:10:10.802 INFO:tasks.workunit.client.0.smithi162.stdout:0/594: creat d5/dc/d1e/d20/fc2 x:0 0 0 2024-04-27T04:10:10.803 INFO:tasks.workunit.client.1.smithi181.stdout:3/681: link d0/d6c/d66/dc2/d9e/cb2 d0/de/d7d/dbc/cdc 0 2024-04-27T04:10:10.806 INFO:tasks.workunit.client.0.smithi162.stdout:6/543: mknod d6/d1a/d34/caa 0 2024-04-27T04:10:10.806 INFO:tasks.workunit.client.1.smithi181.stdout:0/819: dwrite dc/d1f/d3b/f5d [0,4194304] 0 2024-04-27T04:10:10.807 INFO:tasks.workunit.client.0.smithi162.stdout:6/544: write d6/d1a/d34/d3a/d60/d74/fa1 [554136,2311] 0 2024-04-27T04:10:10.811 INFO:tasks.workunit.client.0.smithi162.stdout:1/504: rename d0/d7/d3c/d77/la3 to d0/d11/d3e/d59/d7f/la6 0 2024-04-27T04:10:10.818 INFO:tasks.workunit.client.1.smithi181.stdout:0/820: truncate dc/d1f/d6b/f83 142507 0 2024-04-27T04:10:10.820 INFO:tasks.workunit.client.1.smithi181.stdout:3/682: mkdir d0/d6c/d52/d44/dd5/d58/d64/dc6/ddd 0 2024-04-27T04:10:10.826 INFO:tasks.workunit.client.1.smithi181.stdout:0/821: readlink dc/d1f/le2 0 2024-04-27T04:10:10.833 INFO:tasks.workunit.client.1.smithi181.stdout:9/644: link d0/c9b d0/d44/d4f/cdd 0 2024-04-27T04:10:10.834 INFO:tasks.workunit.client.0.smithi162.stdout:2/471: rmdir d0/d4/d2a/d3b/d49 39 2024-04-27T04:10:10.834 INFO:tasks.workunit.client.1.smithi181.stdout:9/645: fsync d0/d44/d4b/d3d/dbd/fcc 0 2024-04-27T04:10:10.836 INFO:tasks.workunit.client.0.smithi162.stdout:7/517: creat d3/d5/d9/fac x:0 0 0 2024-04-27T04:10:10.841 INFO:tasks.workunit.client.0.smithi162.stdout:6/545: creat d6/d1a/d34/d43/fab x:0 0 0 2024-04-27T04:10:10.842 INFO:tasks.workunit.client.1.smithi181.stdout:0/822: symlink dc/d1f/d3b/d4a/def/l11f 0 2024-04-27T04:10:10.844 INFO:tasks.workunit.client.0.smithi162.stdout:4/505: rename d2/da/d1a/c50 to d2/d25/d1c/d39/d70/ca2 0 2024-04-27T04:10:10.846 INFO:tasks.workunit.client.0.smithi162.stdout:1/505: fdatasync d0/d11/d3e/f4f 0 2024-04-27T04:10:10.849 INFO:tasks.workunit.client.0.smithi162.stdout:8/543: write d0/d10/f9a [2281876,15705] 0 2024-04-27T04:10:10.849 INFO:tasks.workunit.client.0.smithi162.stdout:9/547: write d3/d23/f73 [2436459,12258] 0 2024-04-27T04:10:10.850 INFO:tasks.workunit.client.0.smithi162.stdout:4/506: dwrite d2/d25/f8c [0,4194304] 0 2024-04-27T04:10:10.856 INFO:tasks.workunit.client.1.smithi181.stdout:3/683: dwrite d0/d6c/d52/fd0 [0,4194304] 0 2024-04-27T04:10:10.857 INFO:tasks.workunit.client.1.smithi181.stdout:0/823: truncate dc/d1f/d3b/d4a/d95/d26/f6c 1072428 0 2024-04-27T04:10:10.862 INFO:tasks.workunit.client.1.smithi181.stdout:9/646: rmdir d0/d44/d4b/d3d/dbd/dc4 0 2024-04-27T04:10:10.863 INFO:tasks.workunit.client.0.smithi162.stdout:6/546: fsync d6/d1a/f18 0 2024-04-27T04:10:10.863 INFO:tasks.workunit.client.1.smithi181.stdout:9/647: fdatasync d0/f68 0 2024-04-27T04:10:10.866 INFO:tasks.workunit.client.0.smithi162.stdout:5/563: rename d8/d1a/d7d/fab to d8/d2c/fb1 0 2024-04-27T04:10:10.868 INFO:tasks.workunit.client.1.smithi181.stdout:0/824: dread dc/d1f/d3b/dd0/f100 [0,4194304] 0 2024-04-27T04:10:10.869 INFO:tasks.workunit.client.1.smithi181.stdout:9/648: dwrite d0/d44/d4b/f80 [0,4194304] 0 2024-04-27T04:10:10.870 INFO:tasks.workunit.client.1.smithi181.stdout:0/825: chown dc/d1f/d3b/d4a/d95/d26/d41/db0/ce3 36 1 2024-04-27T04:10:10.871 INFO:tasks.workunit.client.0.smithi162.stdout:1/506: readlink d0/d7/d3c/d54/l8a 0 2024-04-27T04:10:10.873 INFO:tasks.workunit.client.1.smithi181.stdout:0/826: dread dc/d54/fab [0,4194304] 0 2024-04-27T04:10:10.874 INFO:tasks.workunit.client.0.smithi162.stdout:8/544: dread - d0/d10/d44/f73 zero size 2024-04-27T04:10:10.876 INFO:tasks.workunit.client.1.smithi181.stdout:9/649: mkdir d0/d44/d4b/d3d/dbd/dd1/dde 0 2024-04-27T04:10:10.877 INFO:tasks.workunit.client.0.smithi162.stdout:4/507: dread - d2/d25/d1c/d24/f43 zero size 2024-04-27T04:10:10.879 INFO:tasks.workunit.client.1.smithi181.stdout:3/684: link d0/f37 d0/d6c/fde 0 2024-04-27T04:10:10.879 INFO:tasks.workunit.client.0.smithi162.stdout:9/548: dread d3/d11/d45/d4e/d58/f80 [0,4194304] 0 2024-04-27T04:10:10.882 INFO:tasks.workunit.client.0.smithi162.stdout:0/595: creat d5/dc/d1e/d46/fc3 x:0 0 0 2024-04-27T04:10:10.883 INFO:tasks.workunit.client.1.smithi181.stdout:0/827: rename dc/d1f/l11b to dc/d1f/d6b/d7e/d91/dea/l120 0 2024-04-27T04:10:10.884 INFO:tasks.workunit.client.0.smithi162.stdout:6/547: fsync f5 0 2024-04-27T04:10:10.885 INFO:tasks.workunit.client.0.smithi162.stdout:6/548: readlink d6/d1a/d34/d43/la2 0 2024-04-27T04:10:10.889 INFO:tasks.workunit.client.0.smithi162.stdout:7/518: rename d3/d92/fa2 to d3/d5/d5b/d83/de/d2b/d81/fad 0 2024-04-27T04:10:10.891 INFO:tasks.workunit.client.1.smithi181.stdout:9/650: symlink d0/ldf 0 2024-04-27T04:10:10.891 INFO:tasks.workunit.client.0.smithi162.stdout:7/519: truncate d3/d5/d5b/d60/fa5 895471 0 2024-04-27T04:10:10.891 INFO:tasks.workunit.client.1.smithi181.stdout:9/651: chown d0/d44/d4b/ldc 488545 1 2024-04-27T04:10:10.892 INFO:tasks.workunit.client.0.smithi162.stdout:7/520: truncate d3/d39/d57/fa9 260084 0 2024-04-27T04:10:10.894 INFO:tasks.workunit.client.0.smithi162.stdout:3/520: getdents d1/d18/d5d/d6f 0 2024-04-27T04:10:10.895 INFO:tasks.workunit.client.0.smithi162.stdout:3/521: write d1/d18/d5d/d5f/d66/fa7 [22638,5178] 0 2024-04-27T04:10:10.896 INFO:tasks.workunit.client.0.smithi162.stdout:1/507: creat d0/d7/d10/fa7 x:0 0 0 2024-04-27T04:10:10.897 INFO:tasks.workunit.client.0.smithi162.stdout:0/596: dread d5/d60/d8b/f99 [0,4194304] 0 2024-04-27T04:10:10.898 INFO:tasks.workunit.client.1.smithi181.stdout:9/652: dwrite d0/d40/f84 [0,4194304] 0 2024-04-27T04:10:10.898 INFO:tasks.workunit.client.1.smithi181.stdout:3/685: dread d0/d6c/fde [0,4194304] 0 2024-04-27T04:10:10.900 INFO:tasks.workunit.client.0.smithi162.stdout:9/549: creat d3/d13/d2d/d30/fa4 x:0 0 0 2024-04-27T04:10:10.900 INFO:tasks.workunit.client.1.smithi181.stdout:9/653: write d0/d4d/d85/f8e [41744,58926] 0 2024-04-27T04:10:10.902 INFO:tasks.workunit.client.0.smithi162.stdout:6/549: stat d6/d1a/f20 0 2024-04-27T04:10:10.903 INFO:tasks.workunit.client.1.smithi181.stdout:3/686: mknod d0/d6c/d66/d99/cdf 0 2024-04-27T04:10:10.904 INFO:tasks.workunit.client.1.smithi181.stdout:9/654: fsync d0/f13 0 2024-04-27T04:10:10.905 INFO:tasks.workunit.client.1.smithi181.stdout:3/687: creat d0/d6c/d66/d99/fe0 x:0 0 0 2024-04-27T04:10:10.906 INFO:tasks.workunit.client.0.smithi162.stdout:7/521: mknod d3/d39/d8d/cae 0 2024-04-27T04:10:10.906 INFO:tasks.workunit.client.0.smithi162.stdout:7/522: readlink d3/l22 0 2024-04-27T04:10:10.907 INFO:tasks.workunit.client.0.smithi162.stdout:3/522: creat d1/d18/d5d/d6f/fa9 x:0 0 0 2024-04-27T04:10:10.909 INFO:tasks.workunit.client.1.smithi181.stdout:3/688: fsync d0/d6c/d52/d44/dd5/f5e 0 2024-04-27T04:10:10.910 INFO:tasks.workunit.client.0.smithi162.stdout:8/545: mknod d0/cb5 0 2024-04-27T04:10:10.911 INFO:tasks.workunit.client.0.smithi162.stdout:4/508: symlink d2/da/la3 0 2024-04-27T04:10:10.912 INFO:tasks.workunit.client.0.smithi162.stdout:4/509: write d2/ff [5097178,123860] 0 2024-04-27T04:10:10.913 INFO:tasks.workunit.client.0.smithi162.stdout:0/597: rename d5/dc/d1e/d20/fc2 to d5/d26/da2/fc4 0 2024-04-27T04:10:10.913 INFO:tasks.workunit.client.0.smithi162.stdout:4/510: write d2/d25/fb [4902052,111290] 0 2024-04-27T04:10:10.913 INFO:tasks.workunit.client.1.smithi181.stdout:3/689: fdatasync d0/d6c/d52/d44/d74/f9b 0 2024-04-27T04:10:10.914 INFO:tasks.workunit.client.0.smithi162.stdout:7/523: rename d3 to d3/d5/d5b/d83/de/d2b/d81/daa/daf 22 2024-04-27T04:10:10.915 INFO:tasks.workunit.client.1.smithi181.stdout:9/655: rename d0/d44/d4b/d5f/da4/l97 to d0/d44/le0 0 2024-04-27T04:10:10.916 INFO:tasks.workunit.client.0.smithi162.stdout:0/598: write d5/dc/d1e/d46/d48/f97 [1033523,3338] 0 2024-04-27T04:10:10.916 INFO:tasks.workunit.client.1.smithi181.stdout:9/656: chown d0/f15 0 1 2024-04-27T04:10:10.918 INFO:tasks.workunit.client.1.smithi181.stdout:3/690: dread d0/d6c/d52/d44/dd5/f69 [0,4194304] 0 2024-04-27T04:10:10.922 INFO:tasks.workunit.client.0.smithi162.stdout:0/599: dwrite d5/dc/d1e/d41/fa7 [0,4194304] 0 2024-04-27T04:10:10.925 INFO:tasks.workunit.client.0.smithi162.stdout:8/546: dread d0/d10/d25/f36 [0,4194304] 0 2024-04-27T04:10:10.927 INFO:tasks.workunit.client.0.smithi162.stdout:0/600: dread d5/d60/f72 [0,4194304] 0 2024-04-27T04:10:10.931 INFO:tasks.workunit.client.0.smithi162.stdout:8/547: dwrite d0/d10/d25/f93 [0,4194304] 0 2024-04-27T04:10:10.942 INFO:tasks.workunit.client.1.smithi181.stdout:9/657: creat d0/d44/d4b/d5f/da4/fe1 x:0 0 0 2024-04-27T04:10:10.948 INFO:tasks.workunit.client.0.smithi162.stdout:3/523: rename d1/d37 to d1/d18/d23/d34/daa 0 2024-04-27T04:10:10.967 INFO:tasks.workunit.client.1.smithi181.stdout:8/754: sync 2024-04-27T04:10:10.972 INFO:tasks.workunit.client.0.smithi162.stdout:7/524: creat d3/d5/d3c/d4a/fb0 x:0 0 0 2024-04-27T04:10:10.973 INFO:tasks.workunit.client.0.smithi162.stdout:7/525: write d3/d5/d5b/d83/de/d2b/d81/f9c [2128877,23610] 0 2024-04-27T04:10:10.975 INFO:tasks.workunit.client.0.smithi162.stdout:0/601: mkdir d5/dc/d1e/d20/dc5 0 2024-04-27T04:10:10.977 INFO:tasks.workunit.client.0.smithi162.stdout:8/548: creat d0/de/d20/d6c/fb6 x:0 0 0 2024-04-27T04:10:10.978 INFO:tasks.workunit.client.1.smithi181.stdout:8/755: mknod d1/d6/d2f/d30/c103 0 2024-04-27T04:10:10.978 INFO:tasks.workunit.client.0.smithi162.stdout:8/549: write d0/d10/d2b/d2e/f40 [366674,104028] 0 2024-04-27T04:10:10.981 INFO:tasks.workunit.client.0.smithi162.stdout:4/511: rmdir d2/da/d1a 39 2024-04-27T04:10:10.983 INFO:tasks.workunit.client.0.smithi162.stdout:8/550: read d0/d10/f9a [1612358,48082] 0 2024-04-27T04:10:10.983 INFO:tasks.workunit.client.1.smithi181.stdout:8/756: stat d1/d6/f4a 0 2024-04-27T04:10:10.984 INFO:tasks.workunit.client.0.smithi162.stdout:9/550: dread d3/d13/d15/d7e/f92 [0,4194304] 0 2024-04-27T04:10:10.989 INFO:tasks.workunit.client.0.smithi162.stdout:0/602: dread d5/d26/f8d [0,4194304] 0 2024-04-27T04:10:10.990 INFO:tasks.workunit.client.1.smithi181.stdout:9/658: dread d0/f23 [0,4194304] 0 2024-04-27T04:10:10.992 INFO:tasks.workunit.client.0.smithi162.stdout:0/603: dread d5/d26/f8d [0,4194304] 0 2024-04-27T04:10:10.993 INFO:tasks.workunit.client.1.smithi181.stdout:8/757: rename d1/d6/d2f/d30/d9d/dea/l9f to d1/d6/d2f/d30/d9d/dea/db3/ddf/l104 0 2024-04-27T04:10:10.993 INFO:tasks.workunit.client.0.smithi162.stdout:0/604: write d5/dc/d1e/d46/d48/d9e/db8/fb9 [150300,91200] 0 2024-04-27T04:10:10.994 INFO:tasks.workunit.client.0.smithi162.stdout:0/605: chown d5/dc/d1e/d46/d48 2418 1 2024-04-27T04:10:10.994 INFO:tasks.workunit.client.1.smithi181.stdout:9/659: dread - d0/d40/f9c zero size 2024-04-27T04:10:11.004 INFO:tasks.workunit.client.0.smithi162.stdout:7/526: creat d3/d4f/fb1 x:0 0 0 2024-04-27T04:10:11.004 INFO:tasks.workunit.client.1.smithi181.stdout:9/660: dread d0/f10 [0,4194304] 0 2024-04-27T04:10:11.005 INFO:tasks.workunit.client.0.smithi162.stdout:2/472: write d0/d4/f16 [1576874,114829] 0 2024-04-27T04:10:11.005 INFO:tasks.workunit.client.0.smithi162.stdout:7/527: write d3/d5/d5b/d83/f79 [58774,64518] 0 2024-04-27T04:10:11.005 INFO:tasks.workunit.client.0.smithi162.stdout:6/550: link d6/d1a/f31 d6/db/d76/fac 0 2024-04-27T04:10:11.007 INFO:tasks.workunit.client.0.smithi162.stdout:0/606: creat d5/dc/d1e/d46/d48/d9e/db8/fc6 x:0 0 0 2024-04-27T04:10:11.008 INFO:tasks.workunit.client.1.smithi181.stdout:9/661: mknod d0/d44/d4b/d3d/d9a/ce2 0 2024-04-27T04:10:11.009 INFO:tasks.workunit.client.0.smithi162.stdout:0/607: write d5/d60/d98/fad [328509,130230] 0 2024-04-27T04:10:11.012 INFO:tasks.workunit.client.1.smithi181.stdout:8/758: dread d1/d6/d2f/d30/fb5 [0,4194304] 0 2024-04-27T04:10:11.012 INFO:tasks.workunit.client.0.smithi162.stdout:0/608: dread d5/dc/fe [0,4194304] 0 2024-04-27T04:10:11.013 INFO:tasks.workunit.client.0.smithi162.stdout:0/609: read d5/d26/d69/f6c [101222,120074] 0 2024-04-27T04:10:11.014 INFO:tasks.workunit.client.1.smithi181.stdout:9/662: mkdir d0/d44/d4b/d43/d88/de3 0 2024-04-27T04:10:11.014 INFO:tasks.workunit.client.0.smithi162.stdout:0/610: write d5/d26/f4d [522036,47953] 0 2024-04-27T04:10:11.015 INFO:tasks.workunit.client.0.smithi162.stdout:8/551: creat d0/d10/d76/fb7 x:0 0 0 2024-04-27T04:10:11.016 INFO:tasks.workunit.client.1.smithi181.stdout:8/759: rmdir d1/d8c 39 2024-04-27T04:10:11.018 INFO:tasks.workunit.client.1.smithi181.stdout:9/663: dread d0/d44/f6e [0,4194304] 0 2024-04-27T04:10:11.018 INFO:tasks.workunit.client.0.smithi162.stdout:9/551: mknod d3/d13/ca5 0 2024-04-27T04:10:11.018 INFO:tasks.workunit.client.1.smithi181.stdout:0/828: write dc/f18 [2120980,120908] 0 2024-04-27T04:10:11.019 INFO:tasks.workunit.client.0.smithi162.stdout:9/552: chown d3/d13/d2d/d3b/d72 0 1 2024-04-27T04:10:11.022 INFO:tasks.workunit.client.0.smithi162.stdout:5/564: write d8/d1a/f3e [4606241,63856] 0 2024-04-27T04:10:11.027 INFO:tasks.workunit.client.0.smithi162.stdout:7/528: symlink d3/d5/d3c/d4a/d55/lb2 0 2024-04-27T04:10:11.028 INFO:tasks.workunit.client.0.smithi162.stdout:5/565: dwrite d8/f20 [8388608,4194304] 0 2024-04-27T04:10:11.029 INFO:tasks.workunit.client.0.smithi162.stdout:7/529: stat d3/d5/d5b/d83/de/c4e 0 2024-04-27T04:10:11.032 INFO:tasks.workunit.client.0.smithi162.stdout:2/473: dread d0/f6 [0,4194304] 0 2024-04-27T04:10:11.039 INFO:tasks.workunit.client.0.smithi162.stdout:6/551: dread d6/d1a/f15 [0,4194304] 0 2024-04-27T04:10:11.039 INFO:tasks.workunit.client.0.smithi162.stdout:7/530: creat d3/d5/d5b/d83/fb3 x:0 0 0 2024-04-27T04:10:11.040 INFO:tasks.workunit.client.0.smithi162.stdout:3/524: getdents d1/d18/d5d/d6f 0 2024-04-27T04:10:11.041 INFO:tasks.workunit.client.0.smithi162.stdout:1/508: write d0/f37 [459350,119508] 0 2024-04-27T04:10:11.042 INFO:tasks.workunit.client.0.smithi162.stdout:2/474: symlink d0/d4/d17/d2b/l8f 0 2024-04-27T04:10:11.043 INFO:tasks.workunit.client.0.smithi162.stdout:6/552: write d6/d1a/d34/d3a/d60/d74/fa1 [1309380,24329] 0 2024-04-27T04:10:11.045 INFO:tasks.workunit.client.0.smithi162.stdout:0/611: creat d5/dc/d1e/fc7 x:0 0 0 2024-04-27T04:10:11.047 INFO:tasks.workunit.client.0.smithi162.stdout:5/566: dread d8/f12 [4194304,4194304] 0 2024-04-27T04:10:11.048 INFO:tasks.workunit.client.0.smithi162.stdout:0/612: stat d5/dc/d1e/d56/l75 0 2024-04-27T04:10:11.048 INFO:tasks.workunit.client.0.smithi162.stdout:1/509: chown d0/l4b 3600316 1 2024-04-27T04:10:11.048 INFO:tasks.workunit.client.0.smithi162.stdout:7/531: creat d3/d5/d5b/d83/de/d2b/d81/d96/fb4 x:0 0 0 2024-04-27T04:10:11.050 INFO:tasks.workunit.client.0.smithi162.stdout:2/475: creat d0/d4/d17/d2b/d64/f90 x:0 0 0 2024-04-27T04:10:11.050 INFO:tasks.workunit.client.0.smithi162.stdout:5/567: truncate d8/f20 12840056 0 2024-04-27T04:10:11.050 INFO:tasks.workunit.client.0.smithi162.stdout:6/553: rmdir d6/d1a/d34/d3a 39 2024-04-27T04:10:11.051 INFO:tasks.workunit.client.0.smithi162.stdout:7/532: chown d3/d4f/c90 62 1 2024-04-27T04:10:11.052 INFO:tasks.workunit.client.0.smithi162.stdout:3/525: dwrite d1/d18/f6a [0,4194304] 0 2024-04-27T04:10:11.053 INFO:tasks.workunit.client.0.smithi162.stdout:2/476: chown d0/d4/d2a/d41/d7b/l8b 2494 1 2024-04-27T04:10:11.053 INFO:tasks.workunit.client.0.smithi162.stdout:1/510: rmdir d0/d7 39 2024-04-27T04:10:11.055 INFO:tasks.workunit.client.0.smithi162.stdout:0/613: mknod d5/dc/d1e/d20/cc8 0 2024-04-27T04:10:11.055 INFO:tasks.workunit.client.0.smithi162.stdout:2/477: write d0/f1 [930906,106010] 0 2024-04-27T04:10:11.058 INFO:tasks.workunit.client.0.smithi162.stdout:6/554: mkdir d6/d1a/d34/d50/dad 0 2024-04-27T04:10:11.058 INFO:tasks.workunit.client.0.smithi162.stdout:5/568: creat d8/d9/d5a/fb2 x:0 0 0 2024-04-27T04:10:11.060 INFO:tasks.workunit.client.0.smithi162.stdout:6/555: write d6/db/d32/f3b [4365557,51348] 0 2024-04-27T04:10:11.062 INFO:tasks.workunit.client.0.smithi162.stdout:6/556: write d6/d1a/d34/f54 [281475,6486] 0 2024-04-27T04:10:11.066 INFO:tasks.workunit.client.0.smithi162.stdout:5/569: mkdir d8/d9/db3 0 2024-04-27T04:10:11.068 INFO:tasks.workunit.client.0.smithi162.stdout:3/526: truncate d1/d18/d23/f53 6687228 0 2024-04-27T04:10:11.069 INFO:tasks.workunit.client.0.smithi162.stdout:3/527: chown d1/d18/d23/d34/l8a 15 1 2024-04-27T04:10:11.074 INFO:tasks.workunit.client.0.smithi162.stdout:5/570: write d8/f6f [4342290,69286] 0 2024-04-27T04:10:11.074 INFO:tasks.workunit.client.0.smithi162.stdout:3/528: mkdir d1/d18/d23/d34/daa/dab 0 2024-04-27T04:10:11.077 INFO:tasks.workunit.client.0.smithi162.stdout:2/478: symlink d0/d4/d2a/d3b/d49/d5a/d63/d6c/d5d/d86/l91 0 2024-04-27T04:10:11.080 INFO:tasks.workunit.client.0.smithi162.stdout:0/614: getdents d5/dc/d1e/d46 0 2024-04-27T04:10:11.081 INFO:tasks.workunit.client.0.smithi162.stdout:2/479: symlink d0/d4/d2a/d3b/d49/d5a/d63/d80/l92 0 2024-04-27T04:10:11.083 INFO:tasks.workunit.client.0.smithi162.stdout:2/480: fdatasync d0/d4/d2a/d3b/d49/d5a/d63/d6e/f7a 0 2024-04-27T04:10:11.085 INFO:tasks.workunit.client.0.smithi162.stdout:0/615: truncate d5/dc/d1e/f3e 4934741 0 2024-04-27T04:10:11.086 INFO:tasks.workunit.client.0.smithi162.stdout:0/616: write d5/dc/d1e/d46/d48/fbb [824316,92708] 0 2024-04-27T04:10:11.086 INFO:tasks.workunit.client.0.smithi162.stdout:5/571: creat d8/d9/d5b/fb4 x:0 0 0 2024-04-27T04:10:11.090 INFO:tasks.workunit.client.1.smithi181.stdout:9/664: dwrite d0/d40/f84 [4194304,4194304] 0 2024-04-27T04:10:11.091 INFO:tasks.workunit.client.1.smithi181.stdout:9/665: chown d0/d44/d4b 0 1 2024-04-27T04:10:11.093 INFO:tasks.workunit.client.0.smithi162.stdout:0/617: rename d5/d26/d4a/c5a to d5/dc/d1e/d56/cc9 0 2024-04-27T04:10:11.094 INFO:tasks.workunit.client.0.smithi162.stdout:5/572: creat d8/d9/d5b/fb5 x:0 0 0 2024-04-27T04:10:11.096 INFO:tasks.workunit.client.0.smithi162.stdout:8/552: dwrite d0/d10/faf [0,4194304] 0 2024-04-27T04:10:11.097 INFO:tasks.workunit.client.1.smithi181.stdout:9/666: unlink d0/d44/d4b/l3b 0 2024-04-27T04:10:11.103 INFO:tasks.workunit.client.0.smithi162.stdout:5/573: creat d8/d2c/fb6 x:0 0 0 2024-04-27T04:10:11.104 INFO:tasks.workunit.client.0.smithi162.stdout:2/481: dread d0/d4/d17/f3c [0,4194304] 0 2024-04-27T04:10:11.105 INFO:tasks.workunit.client.0.smithi162.stdout:5/574: chown d8/d1a/d29/d4a/d5c/f9e 269 1 2024-04-27T04:10:11.105 INFO:tasks.workunit.client.0.smithi162.stdout:0/618: creat d5/d26/d69/d9b/db4/fca x:0 0 0 2024-04-27T04:10:11.106 INFO:tasks.workunit.client.0.smithi162.stdout:7/533: rmdir d3/d5/d5b/d83/de/d2b/d81 39 2024-04-27T04:10:11.106 INFO:tasks.workunit.client.0.smithi162.stdout:2/482: fsync d0/d4/f22 0 2024-04-27T04:10:11.107 INFO:tasks.workunit.client.0.smithi162.stdout:8/553: dwrite d0/fd [0,4194304] 0 2024-04-27T04:10:11.108 INFO:tasks.workunit.client.1.smithi181.stdout:9/667: mkdir d0/d44/d4b/d3d/dbd/dd1/dde/de4 0 2024-04-27T04:10:11.114 INFO:tasks.workunit.client.0.smithi162.stdout:8/554: chown d0/d10/d2b/d2e/d8a 143086 1 2024-04-27T04:10:11.118 INFO:tasks.workunit.client.0.smithi162.stdout:5/575: rename d8/d2c/l3b to d8/d1a/d29/d4a/d5c/lb7 0 2024-04-27T04:10:11.118 INFO:tasks.workunit.client.0.smithi162.stdout:7/534: symlink d3/d5/d5b/lb5 0 2024-04-27T04:10:11.118 INFO:tasks.workunit.client.0.smithi162.stdout:0/619: dread d5/f2d [4194304,4194304] 0 2024-04-27T04:10:11.119 INFO:tasks.workunit.client.0.smithi162.stdout:1/511: write d0/d7/d10/f64 [541875,86120] 0 2024-04-27T04:10:11.120 INFO:tasks.workunit.client.0.smithi162.stdout:7/535: mkdir d3/d5/d5b/d60/d65/d80/db6 0 2024-04-27T04:10:11.121 INFO:tasks.workunit.client.0.smithi162.stdout:5/576: read d8/d9/f10 [2106091,121680] 0 2024-04-27T04:10:11.124 INFO:tasks.workunit.client.0.smithi162.stdout:2/483: rename d0/d4/d2a/d3b/d49/d5a/d63/d6c to d0/d4/d2a/d3b/d49/d5a/d63/d84/d93 0 2024-04-27T04:10:11.124 INFO:tasks.workunit.client.0.smithi162.stdout:0/620: dread d5/f2d [0,4194304] 0 2024-04-27T04:10:11.125 INFO:tasks.workunit.client.0.smithi162.stdout:2/484: chown d0/d4/d2a/d41/d7b 393947 1 2024-04-27T04:10:11.126 INFO:tasks.workunit.client.0.smithi162.stdout:8/555: mkdir d0/d10/d25/d7d/db8 0 2024-04-27T04:10:11.127 INFO:tasks.workunit.client.0.smithi162.stdout:7/536: creat d3/d5/d5b/d83/fb7 x:0 0 0 2024-04-27T04:10:11.128 INFO:tasks.workunit.client.0.smithi162.stdout:5/577: rmdir d8/d1a/d23/d40/d4b/d66/d88 39 2024-04-27T04:10:11.129 INFO:tasks.workunit.client.0.smithi162.stdout:7/537: write d3/d5/d9/f17 [3810562,78451] 0 2024-04-27T04:10:11.132 INFO:tasks.workunit.client.0.smithi162.stdout:0/621: creat d5/d26/da2/fcb x:0 0 0 2024-04-27T04:10:11.135 INFO:tasks.workunit.client.0.smithi162.stdout:8/556: dwrite d0/f4 [4194304,4194304] 0 2024-04-27T04:10:11.135 INFO:tasks.workunit.client.0.smithi162.stdout:1/512: link d0/d11/d5b/l94 d0/d11/d3e/d59/d7f/la8 0 2024-04-27T04:10:11.137 INFO:tasks.workunit.client.0.smithi162.stdout:1/513: dread - d0/d7/d10/fa7 zero size 2024-04-27T04:10:11.139 INFO:tasks.workunit.client.0.smithi162.stdout:5/578: dwrite d8/d1a/f83 [0,4194304] 0 2024-04-27T04:10:11.140 INFO:tasks.workunit.client.0.smithi162.stdout:1/514: truncate d0/d80/f9f 147982 0 2024-04-27T04:10:11.144 INFO:tasks.workunit.client.0.smithi162.stdout:7/538: getdents d3/d5/d3c 0 2024-04-27T04:10:11.148 INFO:tasks.workunit.client.0.smithi162.stdout:8/557: symlink d0/d10/d25/d65/lb9 0 2024-04-27T04:10:11.149 INFO:tasks.workunit.client.0.smithi162.stdout:8/558: chown d0/de/d6e/f77 1662 1 2024-04-27T04:10:11.155 INFO:tasks.workunit.client.0.smithi162.stdout:1/515: chown d0/d11/d44/d66/f90 36 1 2024-04-27T04:10:11.156 INFO:tasks.workunit.client.0.smithi162.stdout:8/559: symlink d0/d10/d2b/d2d/d2f/d42/lba 0 2024-04-27T04:10:11.157 INFO:tasks.workunit.client.0.smithi162.stdout:1/516: readlink d0/d11/d3e/l95 0 2024-04-27T04:10:11.158 INFO:tasks.workunit.client.0.smithi162.stdout:1/517: write d0/d11/d3e/f6d [3246135,20013] 0 2024-04-27T04:10:11.159 INFO:tasks.workunit.client.0.smithi162.stdout:4/512: sync 2024-04-27T04:10:11.160 INFO:tasks.workunit.client.0.smithi162.stdout:8/560: creat d0/d10/d44/fbb x:0 0 0 2024-04-27T04:10:11.165 INFO:tasks.workunit.client.0.smithi162.stdout:8/561: readlink d0/d10/d2b/d2d/d2f/d39/d7c/l8f 0 2024-04-27T04:10:11.166 INFO:tasks.workunit.client.0.smithi162.stdout:0/622: truncate d5/dc/d1e/d46/f52 1456365 0 2024-04-27T04:10:11.167 INFO:tasks.workunit.client.0.smithi162.stdout:5/579: truncate d8/d1a/f59 3628151 0 2024-04-27T04:10:11.167 INFO:tasks.workunit.client.0.smithi162.stdout:8/562: write d0/d10/d25/d65/f8e [210748,53125] 0 2024-04-27T04:10:11.168 INFO:tasks.workunit.client.0.smithi162.stdout:7/539: write d3/d5/d5b/d83/de/f30 [5178683,38682] 0 2024-04-27T04:10:11.169 INFO:tasks.workunit.client.0.smithi162.stdout:8/563: chown d0/d10/f6a 47208 1 2024-04-27T04:10:11.170 INFO:tasks.workunit.client.1.smithi181.stdout:9/668: dread d0/d44/d4b/d43/d88/da1/f6d [0,4194304] 0 2024-04-27T04:10:11.170 INFO:tasks.workunit.client.0.smithi162.stdout:5/580: truncate d8/d9/d55/f84 839151 0 2024-04-27T04:10:11.171 INFO:tasks.workunit.client.0.smithi162.stdout:8/564: fdatasync d0/de/d6e/f9b 0 2024-04-27T04:10:11.171 INFO:tasks.workunit.client.0.smithi162.stdout:7/540: write d3/d5/d3c/d4a/fb0 [307754,110735] 0 2024-04-27T04:10:11.172 INFO:tasks.workunit.client.0.smithi162.stdout:1/518: getdents d0/d7/d3c/d77 0 2024-04-27T04:10:11.172 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:10:10 smithi181 ceph-mon[33774]: pgmap v5: 65 pgs: 65 active+clean; 2.4 GiB data, 8.6 GiB used, 528 GiB / 536 GiB avail 2024-04-27T04:10:11.174 INFO:tasks.workunit.client.0.smithi162.stdout:0/623: dread d5/d26/d4a/f77 [0,4194304] 0 2024-04-27T04:10:11.175 INFO:tasks.workunit.client.1.smithi181.stdout:5/837: sync 2024-04-27T04:10:11.176 INFO:tasks.workunit.client.1.smithi181.stdout:2/755: sync 2024-04-27T04:10:11.176 INFO:tasks.workunit.client.1.smithi181.stdout:9/669: rename d0/d44/d4b/d5f/da4/fc6 to d0/d44/d4f/fe5 0 2024-04-27T04:10:11.178 INFO:tasks.workunit.client.0.smithi162.stdout:0/624: dread d5/d26/f4d [0,4194304] 0 2024-04-27T04:10:11.178 INFO:tasks.workunit.client.0.smithi162.stdout:4/513: truncate d2/d54/d7a/f83 3261362 0 2024-04-27T04:10:11.178 INFO:tasks.workunit.client.0.smithi162.stdout:0/625: readlink d5/dc/d1e/d46/d48/l4b 0 2024-04-27T04:10:11.179 INFO:tasks.workunit.client.0.smithi162.stdout:0/626: write d5/d26/da2/fcb [430986,101056] 0 2024-04-27T04:10:11.180 INFO:tasks.workunit.client.0.smithi162.stdout:5/581: symlink d8/d1a/d29/lb8 0 2024-04-27T04:10:11.181 INFO:tasks.workunit.client.0.smithi162.stdout:5/582: chown d8/d1a/d29/d4a/d5c 0 1 2024-04-27T04:10:11.181 INFO:tasks.workunit.client.0.smithi162.stdout:0/627: chown d5/dc/d1e/d41/f71 58 1 2024-04-27T04:10:11.184 INFO:tasks.workunit.client.1.smithi181.stdout:5/838: rmdir d4/da3/dbc/db5 39 2024-04-27T04:10:11.190 INFO:tasks.workunit.client.1.smithi181.stdout:5/839: dwrite d4/fc9 [0,4194304] 0 2024-04-27T04:10:11.192 INFO:tasks.workunit.client.0.smithi162.stdout:4/514: creat d2/d25/d1c/d39/d70/d91/fa4 x:0 0 0 2024-04-27T04:10:11.194 INFO:tasks.workunit.client.1.smithi181.stdout:9/670: dwrite d0/d40/f25 [0,4194304] 0 2024-04-27T04:10:11.197 INFO:tasks.workunit.client.1.smithi181.stdout:5/840: dread - d4/d10/d29/d122/d100/d8f/fe8 zero size 2024-04-27T04:10:11.199 INFO:tasks.workunit.client.0.smithi162.stdout:0/628: mknod d5/dc/d1e/d46/d48/d9e/ccc 0 2024-04-27T04:10:11.200 INFO:tasks.workunit.client.1.smithi181.stdout:2/756: link d4/d9/d16/d82/d7d/db7/dcb/l83 d4/d9/df/d14/lf2 0 2024-04-27T04:10:11.201 INFO:tasks.workunit.client.1.smithi181.stdout:2/757: chown d4/d9/d16/d82/d7d/dbe/ded 50208 1 2024-04-27T04:10:11.202 INFO:tasks.workunit.client.1.smithi181.stdout:9/671: rename d0/d44/le0 to d0/d44/d4b/d5f/d8a/le6 0 2024-04-27T04:10:11.204 INFO:tasks.workunit.client.1.smithi181.stdout:5/841: truncate d4/d10/d29/f84 4678882 0 2024-04-27T04:10:11.211 INFO:tasks.workunit.client.0.smithi162.stdout:7/541: creat d3/d5/d9/fb8 x:0 0 0 2024-04-27T04:10:11.213 INFO:tasks.workunit.client.1.smithi181.stdout:2/758: fsync d4/d9/d4f/d5f/fe2 0 2024-04-27T04:10:11.215 INFO:tasks.workunit.client.0.smithi162.stdout:5/583: creat d8/d9/db3/fb9 x:0 0 0 2024-04-27T04:10:11.216 INFO:tasks.workunit.client.0.smithi162.stdout:7/542: dwrite d3/d39/d8d/f91 [0,4194304] 0 2024-04-27T04:10:11.218 INFO:tasks.workunit.client.0.smithi162.stdout:8/565: creat d0/d10/d2b/fbc x:0 0 0 2024-04-27T04:10:11.218 INFO:tasks.workunit.client.0.smithi162.stdout:1/519: link d0/d7/d3c/d77/c45 d0/d11/d50/d53/ca9 0 2024-04-27T04:10:11.223 INFO:tasks.workunit.client.0.smithi162.stdout:0/629: symlink d5/dc/d1e/lcd 0 2024-04-27T04:10:11.224 INFO:tasks.workunit.client.0.smithi162.stdout:8/566: dwrite d0/de/d20/d6c/fb6 [0,4194304] 0 2024-04-27T04:10:11.226 INFO:tasks.workunit.client.0.smithi162.stdout:5/584: truncate d8/d1a/d23/f28 237376 0 2024-04-27T04:10:11.230 INFO:tasks.workunit.client.0.smithi162.stdout:7/543: unlink d3/d5/d3c/d4a/d55/lb2 0 2024-04-27T04:10:11.239 INFO:tasks.workunit.client.0.smithi162.stdout:0/630: dread d5/f27 [0,4194304] 0 2024-04-27T04:10:11.242 INFO:tasks.workunit.client.0.smithi162.stdout:8/567: creat d0/d10/d44/fbd x:0 0 0 2024-04-27T04:10:11.248 INFO:tasks.workunit.client.0.smithi162.stdout:4/515: dread d2/da/f31 [0,4194304] 0 2024-04-27T04:10:11.249 INFO:tasks.workunit.client.0.smithi162.stdout:1/520: mkdir d0/d11/d50/d53/d93/d96/daa 0 2024-04-27T04:10:11.250 INFO:tasks.workunit.client.1.smithi181.stdout:9/672: dwrite d0/d44/d4b/f83 [0,4194304] 0 2024-04-27T04:10:11.252 INFO:tasks.workunit.client.1.smithi181.stdout:9/673: truncate d0/d44/d4b/d3d/dbd/fc9 530632 0 2024-04-27T04:10:11.253 INFO:tasks.workunit.client.1.smithi181.stdout:9/674: dread - d0/d44/d4f/fe5 zero size 2024-04-27T04:10:11.253 INFO:tasks.workunit.client.0.smithi162.stdout:0/631: mkdir d5/dc/d1e/d46/d48/d9e/dce 0 2024-04-27T04:10:11.254 INFO:tasks.workunit.client.0.smithi162.stdout:0/632: read - d5/dc/d1e/d46/f9f zero size 2024-04-27T04:10:11.255 INFO:tasks.workunit.client.0.smithi162.stdout:0/633: read d5/f27 [1613942,24024] 0 2024-04-27T04:10:11.257 INFO:tasks.workunit.client.0.smithi162.stdout:5/585: symlink d8/d9/d5b/lba 0 2024-04-27T04:10:11.259 INFO:tasks.workunit.client.0.smithi162.stdout:7/544: mknod d3/d5/d5b/d83/de/d2b/d81/cb9 0 2024-04-27T04:10:11.261 INFO:tasks.workunit.client.0.smithi162.stdout:7/545: truncate d3/d5/d5b/d83/de/d2b/d81/f82 341566 0 2024-04-27T04:10:11.262 INFO:tasks.workunit.client.0.smithi162.stdout:7/546: stat d3/d5/d5b/lb5 0 2024-04-27T04:10:11.264 INFO:tasks.workunit.client.0.smithi162.stdout:1/521: dread d0/d7/d2a/f5d [0,4194304] 0 2024-04-27T04:10:11.265 INFO:tasks.workunit.client.0.smithi162.stdout:1/522: read - d0/d11/d3e/d5f/f99 zero size 2024-04-27T04:10:11.265 INFO:tasks.workunit.client.0.smithi162.stdout:4/516: dread d2/da/d1a/f1b [0,4194304] 0 2024-04-27T04:10:11.267 INFO:tasks.workunit.client.0.smithi162.stdout:1/523: chown d0/d7/l36 62 1 2024-04-27T04:10:11.267 INFO:tasks.workunit.client.0.smithi162.stdout:4/517: write d2/ff [2549574,120291] 0 2024-04-27T04:10:11.273 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:10:10 smithi162 ceph-mon[26841]: pgmap v5: 65 pgs: 65 active+clean; 2.4 GiB data, 8.6 GiB used, 528 GiB / 536 GiB avail 2024-04-27T04:10:11.274 INFO:tasks.workunit.client.0.smithi162.stdout:7/547: dread d3/f1e [0,4194304] 0 2024-04-27T04:10:11.277 INFO:tasks.workunit.client.0.smithi162.stdout:1/524: symlink d0/d11/d50/d53/d93/lab 0 2024-04-27T04:10:11.278 INFO:tasks.workunit.client.0.smithi162.stdout:1/525: write d0/d11/d50/d53/f84 [3777086,83597] 0 2024-04-27T04:10:11.281 INFO:tasks.workunit.client.0.smithi162.stdout:4/518: truncate d2/d25/d1c/d39/d3b/f94 3574963 0 2024-04-27T04:10:11.283 INFO:tasks.workunit.client.0.smithi162.stdout:8/568: write d0/de/d6e/d71/fae [7273483,85046] 0 2024-04-27T04:10:11.284 INFO:tasks.workunit.client.0.smithi162.stdout:1/526: dwrite d0/d11/d3e/d5f/f99 [0,4194304] 0 2024-04-27T04:10:11.286 INFO:tasks.workunit.client.0.smithi162.stdout:1/527: fdatasync d0/d11/d44/d66/f6f 0 2024-04-27T04:10:11.290 INFO:tasks.workunit.client.0.smithi162.stdout:0/634: link d5/d26/d69/d9b/lb5 d5/d26/d69/d9b/db4/lcf 0 2024-04-27T04:10:11.292 INFO:tasks.workunit.client.0.smithi162.stdout:9/553: sync 2024-04-27T04:10:11.293 INFO:tasks.workunit.client.0.smithi162.stdout:4/519: symlink d2/d25/la5 0 2024-04-27T04:10:11.297 INFO:tasks.workunit.client.1.smithi181.stdout:9/675: fdatasync d0/d44/d4f/fe5 0 2024-04-27T04:10:11.301 INFO:tasks.workunit.client.1.smithi181.stdout:5/842: rename d4/de/d2c/d46/d111 to d4/d10/d12d 0 2024-04-27T04:10:11.305 INFO:tasks.workunit.client.0.smithi162.stdout:1/528: rename d0/d7/d3c/d54/f65 to d0/d11/d50/d53/d93/d96/fac 0 2024-04-27T04:10:11.310 INFO:tasks.workunit.client.1.smithi181.stdout:9/676: creat d0/d44/d4f/d9d/fe7 x:0 0 0 2024-04-27T04:10:11.311 INFO:tasks.workunit.client.0.smithi162.stdout:0/635: creat d5/d60/fd0 x:0 0 0 2024-04-27T04:10:11.313 INFO:tasks.workunit.client.1.smithi181.stdout:5/843: read d4/da3/fbd [3102321,96851] 0 2024-04-27T04:10:11.314 INFO:tasks.workunit.client.0.smithi162.stdout:9/554: unlink d3/d23/d5d/c90 0 2024-04-27T04:10:11.315 INFO:tasks.workunit.client.0.smithi162.stdout:7/548: write d3/d5/d5b/d83/f19 [701191,54364] 0 2024-04-27T04:10:11.317 INFO:tasks.workunit.client.1.smithi181.stdout:9/677: rmdir d0/d44/d4b/d43/d88/da1 39 2024-04-27T04:10:11.320 INFO:tasks.workunit.client.0.smithi162.stdout:7/549: dwrite d3/d4f/f67 [0,4194304] 0 2024-04-27T04:10:11.322 INFO:tasks.workunit.client.1.smithi181.stdout:9/678: unlink d0/d44/d4f/f58 0 2024-04-27T04:10:11.324 INFO:tasks.workunit.client.0.smithi162.stdout:7/550: dread d3/d4f/f67 [0,4194304] 0 2024-04-27T04:10:11.325 INFO:tasks.workunit.client.0.smithi162.stdout:7/551: chown d3/d5/f5f 321 1 2024-04-27T04:10:11.331 INFO:tasks.workunit.client.0.smithi162.stdout:4/520: dread d2/da/f12 [0,4194304] 0 2024-04-27T04:10:11.333 INFO:tasks.workunit.client.0.smithi162.stdout:4/521: truncate d2/d25/d1c/d39/f45 5456257 0 2024-04-27T04:10:11.333 INFO:tasks.workunit.client.1.smithi181.stdout:2/759: dwrite d4/d81/f8e [0,4194304] 0 2024-04-27T04:10:11.333 INFO:tasks.workunit.client.0.smithi162.stdout:9/555: mknod d3/d23/d1e/ca6 0 2024-04-27T04:10:11.335 INFO:tasks.workunit.client.0.smithi162.stdout:5/586: link d8/d1a/d23/c74 d8/d9/d5b/d96/cbb 0 2024-04-27T04:10:11.338 INFO:tasks.workunit.client.1.smithi181.stdout:9/679: dread d0/d44/d4b/d3d/f73 [0,4194304] 0 2024-04-27T04:10:11.339 INFO:tasks.workunit.client.1.smithi181.stdout:5/844: rename d4/d10/d72/db3/c103 to d4/de/d2c/c12e 0 2024-04-27T04:10:11.341 INFO:tasks.workunit.client.1.smithi181.stdout:2/760: dwrite d4/d9/df/d14/f78 [0,4194304] 0 2024-04-27T04:10:11.342 INFO:tasks.workunit.client.1.smithi181.stdout:2/761: readlink d4/d9/d16/l27 0 2024-04-27T04:10:11.343 INFO:tasks.workunit.client.1.smithi181.stdout:2/762: readlink d4/d9/d4f/d79/d98/ld2 0 2024-04-27T04:10:11.345 INFO:tasks.workunit.client.0.smithi162.stdout:8/569: rmdir d0/d10/d2b/d2e/d8a 0 2024-04-27T04:10:11.353 INFO:tasks.workunit.client.1.smithi181.stdout:5/845: rmdir d4/d10/d29 39 2024-04-27T04:10:11.355 INFO:tasks.workunit.client.0.smithi162.stdout:1/529: write d0/d7/d5e/f62 [105410,89772] 0 2024-04-27T04:10:11.362 INFO:tasks.workunit.client.1.smithi181.stdout:2/763: dwrite d4/d9/d16/d82/d7d/fe0 [0,4194304] 0 2024-04-27T04:10:11.362 INFO:tasks.workunit.client.1.smithi181.stdout:5/846: mkdir d4/de/d12f 0 2024-04-27T04:10:11.362 INFO:tasks.workunit.client.0.smithi162.stdout:5/587: dread d8/d1a/d29/d4a/d9f/f73 [0,4194304] 0 2024-04-27T04:10:11.363 INFO:tasks.workunit.client.0.smithi162.stdout:5/588: readlink d8/l38 0 2024-04-27T04:10:11.364 INFO:tasks.workunit.client.1.smithi181.stdout:2/764: write d4/d9/d4f/d79/d98/fcc [446038,100464] 0 2024-04-27T04:10:11.367 INFO:tasks.workunit.client.1.smithi181.stdout:5/847: chown d4/d10/d29/d122 37740648 1 2024-04-27T04:10:11.369 INFO:tasks.workunit.client.1.smithi181.stdout:5/848: write d4/de/d2c/d46/d99/f105 [790026,121243] 0 2024-04-27T04:10:11.369 INFO:tasks.workunit.client.1.smithi181.stdout:2/765: mkdir d4/d81/df3 0 2024-04-27T04:10:11.373 INFO:tasks.workunit.client.1.smithi181.stdout:1/712: sync 2024-04-27T04:10:11.378 INFO:tasks.workunit.client.1.smithi181.stdout:2/766: mkdir d4/d9/d4f/d79/d94/df4 0 2024-04-27T04:10:11.379 INFO:tasks.workunit.client.1.smithi181.stdout:5/849: read d4/d10/f59 [2867705,51415] 0 2024-04-27T04:10:11.382 INFO:tasks.workunit.client.0.smithi162.stdout:7/552: dwrite d3/d4f/f78 [0,4194304] 0 2024-04-27T04:10:11.384 INFO:tasks.workunit.client.1.smithi181.stdout:9/680: dread d0/d4d/fa5 [0,4194304] 0 2024-04-27T04:10:11.385 INFO:tasks.workunit.client.0.smithi162.stdout:8/570: dread d0/de/d6e/d71/f9d [0,4194304] 0 2024-04-27T04:10:11.387 INFO:tasks.workunit.client.1.smithi181.stdout:1/713: unlink d1/dad/db5/d6/da3/d1e/d20/d53/cc8 0 2024-04-27T04:10:11.390 INFO:tasks.workunit.client.1.smithi181.stdout:5/850: write d4/f119 [2245138,15949] 0 2024-04-27T04:10:11.401 INFO:tasks.workunit.client.1.smithi181.stdout:1/714: truncate d1/dad/db5/d6/da3/d1e/d20/d53/f72 810634 0 2024-04-27T04:10:11.401 INFO:tasks.workunit.client.1.smithi181.stdout:2/767: write d4/d9/f64 [333731,39124] 0 2024-04-27T04:10:11.402 INFO:tasks.workunit.client.0.smithi162.stdout:3/529: sync 2024-04-27T04:10:11.402 INFO:tasks.workunit.client.0.smithi162.stdout:2/485: sync 2024-04-27T04:10:11.402 INFO:tasks.workunit.client.0.smithi162.stdout:6/557: sync 2024-04-27T04:10:11.404 INFO:tasks.workunit.client.0.smithi162.stdout:2/486: fsync d0/d4/d2a/d3b/d49/d5a/d63/d84/f69 0 2024-04-27T04:10:11.405 INFO:tasks.workunit.client.1.smithi181.stdout:5/851: creat d4/de/d2c/d46/f130 x:0 0 0 2024-04-27T04:10:11.405 INFO:tasks.workunit.client.1.smithi181.stdout:9/681: dwrite d0/d44/d4b/d43/d88/da1/f6d [0,4194304] 0 2024-04-27T04:10:11.406 INFO:tasks.workunit.client.1.smithi181.stdout:1/715: mkdir d1/dad/db5/d6/dcb/ded 0 2024-04-27T04:10:11.409 INFO:tasks.workunit.client.1.smithi181.stdout:2/768: rename d4/d9/df/l22 to d4/d9/d16/d45/d9a/d9d/lf5 0 2024-04-27T04:10:11.410 INFO:tasks.workunit.client.1.smithi181.stdout:2/769: chown d4/le6 6440640 1 2024-04-27T04:10:11.410 INFO:tasks.workunit.client.1.smithi181.stdout:5/852: mkdir d4/d10/d29/d92/d131 0 2024-04-27T04:10:11.411 INFO:tasks.workunit.client.1.smithi181.stdout:5/853: readlink d4/de/d2f/lf0 0 2024-04-27T04:10:11.412 INFO:tasks.workunit.client.1.smithi181.stdout:5/854: stat d4/d10/d72/d80/l6b 0 2024-04-27T04:10:11.413 INFO:tasks.workunit.client.1.smithi181.stdout:5/855: read - d4/de/d2c/d46/d4d/d8e/d67/f128 zero size 2024-04-27T04:10:11.415 INFO:tasks.workunit.client.1.smithi181.stdout:2/770: creat d4/d9/df/d14/ff6 x:0 0 0 2024-04-27T04:10:11.416 INFO:tasks.workunit.client.1.smithi181.stdout:2/771: chown d4/d9/d16 182499414 1 2024-04-27T04:10:11.416 INFO:tasks.workunit.client.0.smithi162.stdout:4/522: truncate d2/f11 1243739 0 2024-04-27T04:10:11.417 INFO:tasks.workunit.client.1.smithi181.stdout:2/772: chown d4/d9/d35/d42/d86 26308 1 2024-04-27T04:10:11.417 INFO:tasks.workunit.client.0.smithi162.stdout:4/523: fsync d2/d25/d1c/d39/f45 0 2024-04-27T04:10:11.418 INFO:tasks.workunit.client.0.smithi162.stdout:0/636: sync 2024-04-27T04:10:11.419 INFO:tasks.workunit.client.0.smithi162.stdout:0/637: dread - d5/dc/d1e/d46/fc3 zero size 2024-04-27T04:10:11.421 INFO:tasks.workunit.client.0.smithi162.stdout:5/589: truncate d8/d1a/d29/d4a/d9f/f73 738204 0 2024-04-27T04:10:11.422 INFO:tasks.workunit.client.1.smithi181.stdout:6/597: sync 2024-04-27T04:10:11.422 INFO:tasks.workunit.client.1.smithi181.stdout:7/713: sync 2024-04-27T04:10:11.422 INFO:tasks.workunit.client.1.smithi181.stdout:8/760: sync 2024-04-27T04:10:11.422 INFO:tasks.workunit.client.1.smithi181.stdout:3/691: sync 2024-04-27T04:10:11.422 INFO:tasks.workunit.client.1.smithi181.stdout:4/664: sync 2024-04-27T04:10:11.422 INFO:tasks.workunit.client.1.smithi181.stdout:0/829: sync 2024-04-27T04:10:11.423 INFO:tasks.workunit.client.0.smithi162.stdout:5/590: chown d8/d1a/d29/d4a/d9f/c7e 12573768 1 2024-04-27T04:10:11.423 INFO:tasks.workunit.client.0.smithi162.stdout:5/591: write d8/d9/d55/f84 [553098,79965] 0 2024-04-27T04:10:11.423 INFO:tasks.workunit.client.1.smithi181.stdout:5/856: unlink d4/de/d2c/d46/d4d/l61 0 2024-04-27T04:10:11.424 INFO:tasks.workunit.client.0.smithi162.stdout:5/592: readlink d8/d1a/d29/d4a/l7c 0 2024-04-27T04:10:11.426 INFO:tasks.workunit.client.1.smithi181.stdout:7/714: dread d1/d22/d15/d29/dbe/d1b/fac [8388608,4194304] 0 2024-04-27T04:10:11.428 INFO:tasks.workunit.client.1.smithi181.stdout:9/682: link d0/d40/f84 d0/fe8 0 2024-04-27T04:10:11.428 INFO:tasks.workunit.client.1.smithi181.stdout:9/683: chown d0/d44/d4f/d9d/fd0 1400 1 2024-04-27T04:10:11.433 INFO:tasks.workunit.client.1.smithi181.stdout:1/716: rename d1/dad/db5/d6/da3/d1e/d20/d53/fa0 to d1/dad/db5/d6/d6a/fee 0 2024-04-27T04:10:11.437 INFO:tasks.workunit.client.0.smithi162.stdout:6/558: unlink d6/f90 0 2024-04-27T04:10:11.440 INFO:tasks.workunit.client.1.smithi181.stdout:4/665: fsync d0/d15/d25/d2a/d36/f42 0 2024-04-27T04:10:11.446 INFO:tasks.workunit.client.0.smithi162.stdout:1/530: write d0/d11/d50/f56 [1830201,71885] 0 2024-04-27T04:10:11.447 INFO:tasks.workunit.client.0.smithi162.stdout:1/531: readlink d0/d7/d3c/d54/l8c 0 2024-04-27T04:10:11.454 INFO:tasks.workunit.client.1.smithi181.stdout:7/715: mknod d1/d57/d72/cda 0 2024-04-27T04:10:11.454 INFO:tasks.workunit.client.1.smithi181.stdout:7/716: fdatasync d1/d2/fbd 0 2024-04-27T04:10:11.455 INFO:tasks.workunit.client.0.smithi162.stdout:9/556: link d3/d13/d15/l1a d3/d13/d15/d7e/la7 0 2024-04-27T04:10:11.457 INFO:tasks.workunit.client.1.smithi181.stdout:2/773: fsync d4/d9/d4f/d79/fbc 0 2024-04-27T04:10:11.459 INFO:tasks.workunit.client.1.smithi181.stdout:2/774: write d4/d9/df/d14/da7/fb9 [253197,12729] 0 2024-04-27T04:10:11.461 INFO:tasks.workunit.client.1.smithi181.stdout:1/717: creat d1/d2/d89/fef x:0 0 0 2024-04-27T04:10:11.463 INFO:tasks.workunit.client.0.smithi162.stdout:5/593: rmdir d8/d9/d5a 39 2024-04-27T04:10:11.467 INFO:tasks.workunit.client.0.smithi162.stdout:7/553: getdents d3/d5/d5b/d83/de/d2b/da7 0 2024-04-27T04:10:11.468 INFO:tasks.workunit.client.0.smithi162.stdout:8/571: mkdir d0/d10/d25/d7d/db8/dbe 0 2024-04-27T04:10:11.471 INFO:tasks.workunit.client.0.smithi162.stdout:3/530: mkdir d1/d18/dac 0 2024-04-27T04:10:11.472 INFO:tasks.workunit.client.0.smithi162.stdout:3/531: write d1/d18/f6a [2526400,12601] 0 2024-04-27T04:10:11.472 INFO:tasks.workunit.client.1.smithi181.stdout:8/761: dwrite d1/d6/d2f/d47/f8e [0,4194304] 0 2024-04-27T04:10:11.473 INFO:tasks.workunit.client.0.smithi162.stdout:3/532: chown d1/d18/d23/d58/ca2 1485133022 1 2024-04-27T04:10:11.473 INFO:tasks.workunit.client.1.smithi181.stdout:5/857: write d4/d10/d29/f79 [3346178,20201] 0 2024-04-27T04:10:11.474 INFO:tasks.workunit.client.0.smithi162.stdout:3/533: chown d1/d21 613786 1 2024-04-27T04:10:11.475 INFO:tasks.workunit.client.1.smithi181.stdout:5/858: chown d4/de/d2c/d46/d4d/d8e/d67 376069362 1 2024-04-27T04:10:11.476 INFO:tasks.workunit.client.0.smithi162.stdout:3/534: truncate d1/d18/d5d/d5f/d66/fa7 238357 0 2024-04-27T04:10:11.476 INFO:tasks.workunit.client.1.smithi181.stdout:5/859: chown d4/d10/d72/d80/d85/fae 163560648 1 2024-04-27T04:10:11.480 INFO:tasks.workunit.client.1.smithi181.stdout:1/718: dread d1/d2/f4 [0,4194304] 0 2024-04-27T04:10:11.483 INFO:tasks.workunit.client.0.smithi162.stdout:1/532: creat d0/d11/d5b/fad x:0 0 0 2024-04-27T04:10:11.487 INFO:tasks.workunit.client.1.smithi181.stdout:3/692: dwrite d0/de/f4b [0,4194304] 0 2024-04-27T04:10:11.488 INFO:tasks.workunit.client.1.smithi181.stdout:0/830: write dc/d1f/d6b/db9/d106/f119 [3724461,116644] 0 2024-04-27T04:10:11.489 INFO:tasks.workunit.client.0.smithi162.stdout:2/487: truncate d0/d4/d2a/d3b/d49/d5a/d63/d84/d93/f76 1705314 0 2024-04-27T04:10:11.490 INFO:tasks.workunit.client.0.smithi162.stdout:2/488: chown d0/d4/c71 39 1 2024-04-27T04:10:11.494 INFO:tasks.workunit.client.1.smithi181.stdout:7/717: dwrite d1/d2/d1c/d56/fb3 [0,4194304] 0 2024-04-27T04:10:11.499 INFO:tasks.workunit.client.0.smithi162.stdout:5/594: rename d8/d1a/d23/d40/d4b to d8/d1a/d23/dbc 0 2024-04-27T04:10:11.502 INFO:tasks.workunit.client.1.smithi181.stdout:4/666: mknod d0/d15/de/d60/d6f/db9/cdb 0 2024-04-27T04:10:11.505 INFO:tasks.workunit.client.1.smithi181.stdout:8/762: readlink d1/d6/d2f/d30/d9d/dea/db3/ddf/le0 0 2024-04-27T04:10:11.506 INFO:tasks.workunit.client.0.smithi162.stdout:5/595: dwrite d8/fad [0,4194304] 0 2024-04-27T04:10:11.506 INFO:tasks.workunit.client.1.smithi181.stdout:8/763: dread - d1/d6/d2f/d30/faf zero size 2024-04-27T04:10:11.507 INFO:tasks.workunit.client.0.smithi162.stdout:3/535: mkdir d1/d18/d23/d34/dad 0 2024-04-27T04:10:11.508 INFO:tasks.workunit.client.1.smithi181.stdout:5/860: fsync d4/d10/d29/d122/d100/d8f/fe8 0 2024-04-27T04:10:11.514 INFO:tasks.workunit.client.1.smithi181.stdout:9/684: dwrite d0/d40/f81 [0,4194304] 0 2024-04-27T04:10:11.517 INFO:tasks.workunit.client.0.smithi162.stdout:6/559: truncate d6/d1a/d34/d3a/d68/d6c/f53 1206664 0 2024-04-27T04:10:11.518 INFO:tasks.workunit.client.1.smithi181.stdout:9/685: truncate d0/d44/d4f/d9d/fe7 230216 0 2024-04-27T04:10:11.519 INFO:tasks.workunit.client.1.smithi181.stdout:9/686: readlink d0/d44/d4b/d3d/d9a/lba 0 2024-04-27T04:10:11.519 INFO:tasks.workunit.client.0.smithi162.stdout:1/533: mkdir d0/d7/d3c/d54/dae 0 2024-04-27T04:10:11.527 INFO:tasks.workunit.client.0.smithi162.stdout:2/489: unlink d0/d4/d2a/d3b/l60 0 2024-04-27T04:10:11.529 INFO:tasks.workunit.client.0.smithi162.stdout:9/557: dwrite d3/d13/f41 [4194304,4194304] 0 2024-04-27T04:10:11.530 INFO:tasks.workunit.client.1.smithi181.stdout:3/693: chown d0/de/f17 51828 1 2024-04-27T04:10:11.530 INFO:tasks.workunit.client.1.smithi181.stdout:3/694: stat d0/d6c/c51 0 2024-04-27T04:10:11.530 INFO:tasks.workunit.client.0.smithi162.stdout:9/558: stat d3/cf 0 2024-04-27T04:10:11.532 INFO:tasks.workunit.client.0.smithi162.stdout:0/638: dwrite d5/dc/d1e/f3e [4194304,4194304] 0 2024-04-27T04:10:11.532 INFO:tasks.workunit.client.0.smithi162.stdout:6/560: dwrite d6/d1a/fa6 [0,4194304] 0 2024-04-27T04:10:11.533 INFO:tasks.workunit.client.0.smithi162.stdout:8/572: getdents d0/d10/d25/d65/d96 0 2024-04-27T04:10:11.536 INFO:tasks.workunit.client.0.smithi162.stdout:2/490: readlink d0/d4/d2a/d3b/d49/d5a/d63/d84/d93/d5d/d86/l91 0 2024-04-27T04:10:11.536 INFO:tasks.workunit.client.0.smithi162.stdout:9/559: chown d3/d23/d1e/d43/f5a 27883 1 2024-04-27T04:10:11.539 INFO:tasks.workunit.client.1.smithi181.stdout:2/775: link d4/d9/d35/d9f/ff1 d4/d9/d16/d45/ff7 0 2024-04-27T04:10:11.543 INFO:tasks.workunit.client.1.smithi181.stdout:4/667: fdatasync d0/d15/de/f91 0 2024-04-27T04:10:11.544 INFO:tasks.workunit.client.0.smithi162.stdout:8/573: dwrite d0/de/d6e/d71/fae [4194304,4194304] 0 2024-04-27T04:10:11.545 INFO:tasks.workunit.client.0.smithi162.stdout:2/491: dread d0/d4/d2a/d41/d77/f7c [0,4194304] 0 2024-04-27T04:10:11.546 INFO:tasks.workunit.client.0.smithi162.stdout:3/536: rename d1/d18/d5d/d5f/d66/fa7 to d1/d21/d47/d57/fae 0 2024-04-27T04:10:11.549 INFO:tasks.workunit.client.1.smithi181.stdout:6/598: getdents d7/dc/dc3/d62 0 2024-04-27T04:10:11.550 INFO:tasks.workunit.client.1.smithi181.stdout:6/599: stat d7/dc/dc3/d62/fcc 0 2024-04-27T04:10:11.550 INFO:tasks.workunit.client.0.smithi162.stdout:4/524: getdents d2/d25/d1c/d39/d70 0 2024-04-27T04:10:11.552 INFO:tasks.workunit.client.0.smithi162.stdout:7/554: sync 2024-04-27T04:10:11.553 INFO:tasks.workunit.client.0.smithi162.stdout:4/525: fsync d2/d25/d1c/d39/d70/d91/f97 0 2024-04-27T04:10:11.554 INFO:tasks.workunit.client.0.smithi162.stdout:1/534: creat d0/d80/d89/faf x:0 0 0 2024-04-27T04:10:11.554 INFO:tasks.workunit.client.0.smithi162.stdout:5/596: dread d8/d1a/d23/d40/f46 [0,4194304] 0 2024-04-27T04:10:11.555 INFO:tasks.workunit.client.0.smithi162.stdout:4/526: chown d2/d54/d7a/l88 3605429 1 2024-04-27T04:10:11.557 INFO:tasks.workunit.client.0.smithi162.stdout:7/555: read d3/d39/f77 [359174,101424] 0 2024-04-27T04:10:11.559 INFO:tasks.workunit.client.0.smithi162.stdout:1/535: write d0/d7/d2a/f47 [4190675,104499] 0 2024-04-27T04:10:11.562 INFO:tasks.workunit.client.1.smithi181.stdout:2/776: truncate d4/f32 2214988 0 2024-04-27T04:10:11.563 INFO:tasks.workunit.client.1.smithi181.stdout:2/777: write d4/d9/df/d14/ff6 [1038473,26420] 0 2024-04-27T04:10:11.567 INFO:tasks.workunit.client.1.smithi181.stdout:6/600: fdatasync d7/fb9 0 2024-04-27T04:10:11.568 INFO:tasks.workunit.client.0.smithi162.stdout:9/560: dread d3/d13/f26 [0,4194304] 0 2024-04-27T04:10:11.568 INFO:tasks.workunit.client.0.smithi162.stdout:9/561: stat d3/d13/d2d/l64 0 2024-04-27T04:10:11.569 INFO:tasks.workunit.client.1.smithi181.stdout:8/764: rename d1/d6/d2f/d30/d66/f9c to d1/d6/d6f/dd2/df2/f105 0 2024-04-27T04:10:11.574 INFO:tasks.workunit.client.0.smithi162.stdout:3/537: chown d1/d18/c5c 387099 1 2024-04-27T04:10:11.576 INFO:tasks.workunit.client.0.smithi162.stdout:2/492: mknod d0/d4/d2a/d3b/d49/d5a/d63/d70/c94 0 2024-04-27T04:10:11.577 INFO:tasks.workunit.client.0.smithi162.stdout:2/493: fdatasync d0/d4/d2a/f36 0 2024-04-27T04:10:11.588 INFO:tasks.workunit.client.0.smithi162.stdout:6/561: creat d6/d1a/d34/d50/dad/fae x:0 0 0 2024-04-27T04:10:11.590 INFO:tasks.workunit.client.1.smithi181.stdout:6/601: rename d7/dc/dc3/d72/f92 to d7/dc/dc3/d72/daa/fce 0 2024-04-27T04:10:11.590 INFO:tasks.workunit.client.1.smithi181.stdout:4/668: truncate d0/d15/d76/db1/fb6 1147332 0 2024-04-27T04:10:11.591 INFO:tasks.workunit.client.1.smithi181.stdout:6/602: chown d7/dc/d1c/d7f/c33 531173 1 2024-04-27T04:10:11.592 INFO:tasks.workunit.client.0.smithi162.stdout:1/536: mkdir d0/d7/d3c/d54/db0 0 2024-04-27T04:10:11.592 INFO:tasks.workunit.client.1.smithi181.stdout:8/765: symlink d1/d6/d2f/d30/d66/dc0/l106 0 2024-04-27T04:10:11.593 INFO:tasks.workunit.client.1.smithi181.stdout:8/766: chown d1/l62 8 1 2024-04-27T04:10:11.594 INFO:tasks.workunit.client.0.smithi162.stdout:0/639: mkdir d5/dc/d1e/d46/d48/d9e/dce/dd1 0 2024-04-27T04:10:11.595 INFO:tasks.workunit.client.1.smithi181.stdout:5/861: creat d4/de/d2f/f132 x:0 0 0 2024-04-27T04:10:11.597 INFO:tasks.workunit.client.1.smithi181.stdout:1/719: getdents d1/dad/db5/d6/da3/d1e 0 2024-04-27T04:10:11.598 INFO:tasks.workunit.client.1.smithi181.stdout:0/831: link dc/d1f/d3b/d4a/d95/c70 dc/d1f/d6b/db9/d106/d110/c121 0 2024-04-27T04:10:11.599 INFO:tasks.workunit.client.0.smithi162.stdout:3/538: read d1/d18/d23/d2f/f42 [745590,73761] 0 2024-04-27T04:10:11.599 INFO:tasks.workunit.client.1.smithi181.stdout:2/778: dwrite d4/d9/d16/d82/d7d/db7/dcb/f97 [0,4194304] 0 2024-04-27T04:10:11.604 INFO:tasks.workunit.client.1.smithi181.stdout:4/669: truncate d0/d15/de/d19/fab 541955 0 2024-04-27T04:10:11.605 INFO:tasks.workunit.client.1.smithi181.stdout:4/670: stat d0/d15/d5c/d69/l29 0 2024-04-27T04:10:11.606 INFO:tasks.workunit.client.0.smithi162.stdout:1/537: readlink d0/lf 0 2024-04-27T04:10:11.608 INFO:tasks.workunit.client.0.smithi162.stdout:0/640: creat d5/dc/d1e/d46/d48/d9e/db8/fd2 x:0 0 0 2024-04-27T04:10:11.610 INFO:tasks.workunit.client.0.smithi162.stdout:0/641: write d5/dc/d1e/d46/fc3 [370980,115730] 0 2024-04-27T04:10:11.610 INFO:tasks.workunit.client.0.smithi162.stdout:6/562: dwrite d6/f1e [4194304,4194304] 0 2024-04-27T04:10:11.614 INFO:tasks.workunit.client.1.smithi181.stdout:3/695: getdents d0/d6c/d66/dc2 0 2024-04-27T04:10:11.616 INFO:tasks.workunit.client.0.smithi162.stdout:3/539: fsync d1/f4c 0 2024-04-27T04:10:11.619 INFO:tasks.workunit.client.0.smithi162.stdout:7/556: creat d3/d5/d5b/fba x:0 0 0 2024-04-27T04:10:11.620 INFO:tasks.workunit.client.0.smithi162.stdout:7/557: chown d3/d5/d5b/d83/de/d2b/f56 17490 1 2024-04-27T04:10:11.621 INFO:tasks.workunit.client.0.smithi162.stdout:1/538: creat d0/d11/d3e/d5f/fb1 x:0 0 0 2024-04-27T04:10:11.622 INFO:tasks.workunit.client.0.smithi162.stdout:5/597: truncate d8/d9/d55/f69 449959 0 2024-04-27T04:10:11.626 INFO:tasks.workunit.client.0.smithi162.stdout:7/558: dwrite d3/d5/d3c/d4a/fa4 [0,4194304] 0 2024-04-27T04:10:11.627 INFO:tasks.workunit.client.0.smithi162.stdout:4/527: dread d2/da/dd/f60 [0,4194304] 0 2024-04-27T04:10:11.628 INFO:tasks.workunit.client.1.smithi181.stdout:6/603: dwrite d7/f9d [0,4194304] 0 2024-04-27T04:10:11.629 INFO:tasks.workunit.client.0.smithi162.stdout:0/642: chown d5/dc/l59 28197167 1 2024-04-27T04:10:11.633 INFO:tasks.workunit.client.1.smithi181.stdout:4/671: mknod d0/d15/d76/da4/cdc 0 2024-04-27T04:10:11.633 INFO:tasks.workunit.client.0.smithi162.stdout:8/574: rename d0/de/l5d to d0/lbf 0 2024-04-27T04:10:11.633 INFO:tasks.workunit.client.0.smithi162.stdout:4/528: write d2/d25/d1c/d39/d3b/d47/f4c [3909511,115306] 0 2024-04-27T04:10:11.634 INFO:tasks.workunit.client.0.smithi162.stdout:5/598: dwrite d8/d1a/d29/d4a/d5c/f9e [0,4194304] 0 2024-04-27T04:10:11.634 INFO:tasks.workunit.client.0.smithi162.stdout:8/575: write d0/fd [2464258,101197] 0 2024-04-27T04:10:11.635 INFO:tasks.workunit.client.0.smithi162.stdout:6/563: creat d6/d1a/d34/d7b/d7e/d97/faf x:0 0 0 2024-04-27T04:10:11.638 INFO:tasks.workunit.client.0.smithi162.stdout:5/599: write d8/d9/d5b/f79 [2069850,40473] 0 2024-04-27T04:10:11.641 INFO:tasks.workunit.client.1.smithi181.stdout:3/696: mkdir d0/d6c/d66/dc2/de1 0 2024-04-27T04:10:11.642 INFO:tasks.workunit.client.0.smithi162.stdout:3/540: rmdir d1/d18/d23/d58 39 2024-04-27T04:10:11.644 INFO:tasks.workunit.client.1.smithi181.stdout:5/862: mkdir d4/de/d2c/d133 0 2024-04-27T04:10:11.645 INFO:tasks.workunit.client.1.smithi181.stdout:5/863: chown d4/de/d2c/d46/f60 412191 1 2024-04-27T04:10:11.647 INFO:tasks.workunit.client.0.smithi162.stdout:5/600: dread d8/d1a/d23/f39 [0,4194304] 0 2024-04-27T04:10:11.647 INFO:tasks.workunit.client.1.smithi181.stdout:1/720: rename d1/d2/d64/f8d to d1/dad/db5/d6/da3/d1e/d48/da6/da4/ff0 0 2024-04-27T04:10:11.648 INFO:tasks.workunit.client.0.smithi162.stdout:5/601: dread - d8/d9/fb0 zero size 2024-04-27T04:10:11.650 INFO:tasks.workunit.client.1.smithi181.stdout:0/832: unlink dc/d1f/d3b/d4a/d95/d26/f28 0 2024-04-27T04:10:11.652 INFO:tasks.workunit.client.1.smithi181.stdout:0/833: fdatasync dc/d1f/d3b/d4a/d95/d26/d41/d6f/d7d/fa7 0 2024-04-27T04:10:11.654 INFO:tasks.workunit.client.0.smithi162.stdout:5/602: dwrite d8/f6f [0,4194304] 0 2024-04-27T04:10:11.654 INFO:tasks.workunit.client.0.smithi162.stdout:7/559: unlink d3/d5/d9/l5c 0 2024-04-27T04:10:11.664 INFO:tasks.workunit.client.1.smithi181.stdout:6/604: dread d7/dc/d1e/f53 [0,4194304] 0 2024-04-27T04:10:11.665 INFO:tasks.workunit.client.0.smithi162.stdout:9/562: rename d3/d13/d2d/d3b/c8c to d3/d13/d2d/d3b/d69/ca8 0 2024-04-27T04:10:11.667 INFO:tasks.workunit.client.1.smithi181.stdout:4/672: creat d0/d15/d25/d2a/d4c/d7c/fdd x:0 0 0 2024-04-27T04:10:11.669 INFO:tasks.workunit.client.1.smithi181.stdout:8/767: creat d1/d6/f107 x:0 0 0 2024-04-27T04:10:11.674 INFO:tasks.workunit.client.0.smithi162.stdout:6/564: creat d6/db/fb0 x:0 0 0 2024-04-27T04:10:11.674 INFO:tasks.workunit.client.0.smithi162.stdout:6/565: dread - d6/d1a/d34/d43/fab zero size 2024-04-27T04:10:11.676 INFO:tasks.workunit.client.1.smithi181.stdout:2/779: creat d4/d9/d16/d45/d9a/ff8 x:0 0 0 2024-04-27T04:10:11.679 INFO:tasks.workunit.client.0.smithi162.stdout:3/541: creat d1/d18/d23/d2f/faf x:0 0 0 2024-04-27T04:10:11.681 INFO:tasks.workunit.client.1.smithi181.stdout:6/605: mknod d7/dc/dc3/d72/ccf 0 2024-04-27T04:10:11.683 INFO:tasks.workunit.client.1.smithi181.stdout:4/673: mknod d0/d15/de/d60/d6f/cde 0 2024-04-27T04:10:11.684 INFO:tasks.workunit.client.0.smithi162.stdout:5/603: readlink d8/d1a/d29/d4a/d5c/lb7 0 2024-04-27T04:10:11.685 INFO:tasks.workunit.client.1.smithi181.stdout:8/768: symlink d1/d6/d2f/d30/d9d/dea/db3/ddf/l108 0 2024-04-27T04:10:11.685 INFO:tasks.workunit.client.1.smithi181.stdout:5/864: dwrite d4/d10/d29/d122/fc2 [4194304,4194304] 0 2024-04-27T04:10:11.686 INFO:tasks.workunit.client.1.smithi181.stdout:6/606: truncate d7/dc/dc3/d72/d98/dbf/fc1 870779 0 2024-04-27T04:10:11.687 INFO:tasks.workunit.client.0.smithi162.stdout:7/560: unlink d3/d5/d5b/d83/fa3 0 2024-04-27T04:10:11.687 INFO:tasks.workunit.client.0.smithi162.stdout:7/561: stat d3/d39 0 2024-04-27T04:10:11.688 INFO:tasks.workunit.client.1.smithi181.stdout:8/769: write d1/d6/d6f/ff5 [726520,15547] 0 2024-04-27T04:10:11.689 INFO:tasks.workunit.client.1.smithi181.stdout:1/721: mkdir d1/dad/db5/d6/db/d2d/df1 0 2024-04-27T04:10:11.692 INFO:tasks.workunit.client.1.smithi181.stdout:2/780: write d4/d9/d35/f3a [5412103,23309] 0 2024-04-27T04:10:11.692 INFO:tasks.workunit.client.1.smithi181.stdout:1/722: chown d1/dad/db5/d6/db/d2d/d36/dc4/cc7 3296348 1 2024-04-27T04:10:11.694 INFO:tasks.workunit.client.1.smithi181.stdout:2/781: truncate d4/d9/d16/d45/fdc 979597 0 2024-04-27T04:10:11.698 INFO:tasks.workunit.client.0.smithi162.stdout:2/494: rename d0/d4/d2a/d3b/d49/d5a/d63/d6e/f7a to d0/d4/d17/d2b/f95 0 2024-04-27T04:10:11.698 INFO:tasks.workunit.client.1.smithi181.stdout:0/834: symlink dc/d1f/d6b/db9/dba/deb/l122 0 2024-04-27T04:10:11.700 INFO:tasks.workunit.client.0.smithi162.stdout:8/576: creat d0/de/d6e/d71/d86/fc0 x:0 0 0 2024-04-27T04:10:11.701 INFO:tasks.workunit.client.0.smithi162.stdout:8/577: chown d0/d10/d2b/d2d/c9c 2521313 1 2024-04-27T04:10:11.702 INFO:tasks.workunit.client.0.smithi162.stdout:8/578: chown d0/d10/d25 46282199 1 2024-04-27T04:10:11.702 INFO:tasks.workunit.client.1.smithi181.stdout:6/607: read d7/dc/d1c/d7f/d3e/f54 [781296,84183] 0 2024-04-27T04:10:11.703 INFO:tasks.workunit.client.0.smithi162.stdout:6/566: mknod d6/d1a/d34/d3a/d60/cb1 0 2024-04-27T04:10:11.704 INFO:tasks.workunit.client.1.smithi181.stdout:5/865: fdatasync d4/de/d2c/d46/d99/dad/dca/fcc 0 2024-04-27T04:10:11.705 INFO:tasks.workunit.client.1.smithi181.stdout:5/866: fdatasync d4/d10/d29/d92/dbb/fe9 0 2024-04-27T04:10:11.705 INFO:tasks.workunit.client.0.smithi162.stdout:3/542: rmdir d1/d65 39 2024-04-27T04:10:11.710 INFO:tasks.workunit.client.1.smithi181.stdout:1/723: mkdir d1/dad/db5/d6/db/df2 0 2024-04-27T04:10:11.711 INFO:tasks.workunit.client.0.smithi162.stdout:5/604: fdatasync f4 0 2024-04-27T04:10:11.715 INFO:tasks.workunit.client.0.smithi162.stdout:7/562: creat d3/d5/d5b/d83/de/fbb x:0 0 0 2024-04-27T04:10:11.715 INFO:tasks.workunit.client.1.smithi181.stdout:0/835: read dc/d1f/d3b/f5f [1634600,24211] 0 2024-04-27T04:10:11.717 INFO:tasks.workunit.client.0.smithi162.stdout:0/643: creat d5/dc/d1e/d46/d48/fd3 x:0 0 0 2024-04-27T04:10:11.718 INFO:tasks.workunit.client.0.smithi162.stdout:0/644: truncate d5/dc/d1e/fc7 382135 0 2024-04-27T04:10:11.718 INFO:tasks.workunit.client.1.smithi181.stdout:4/674: write d0/d15/d25/d2b/d6d/f71 [5447780,88456] 0 2024-04-27T04:10:11.719 INFO:tasks.workunit.client.1.smithi181.stdout:4/675: chown d0/d15/d25/d2b/d6d 61 1 2024-04-27T04:10:11.719 INFO:tasks.workunit.client.0.smithi162.stdout:2/495: fsync d0/d4/d2a/f3e 0 2024-04-27T04:10:11.721 INFO:tasks.workunit.client.0.smithi162.stdout:9/563: mknod d3/d13/d15/d7e/ca9 0 2024-04-27T04:10:11.723 INFO:tasks.workunit.client.1.smithi181.stdout:5/867: unlink d4/de/d2f/c21 0 2024-04-27T04:10:11.724 INFO:tasks.workunit.client.0.smithi162.stdout:4/529: creat d2/d25/d1c/d24/fa6 x:0 0 0 2024-04-27T04:10:11.725 INFO:tasks.workunit.client.0.smithi162.stdout:4/530: fdatasync d2/da/dd/f93 0 2024-04-27T04:10:11.726 INFO:tasks.workunit.client.0.smithi162.stdout:8/579: write d0/d10/d25/f30 [8593071,52776] 0 2024-04-27T04:10:11.726 INFO:tasks.workunit.client.1.smithi181.stdout:8/770: dwrite d1/d6/d2f/d47/f4b [0,4194304] 0 2024-04-27T04:10:11.728 INFO:tasks.workunit.client.1.smithi181.stdout:1/724: fsync d1/dad/db5/d6/da3/d1e/d48/f5d 0 2024-04-27T04:10:11.729 INFO:tasks.workunit.client.1.smithi181.stdout:2/782: getdents d4/d9/d35/d9f/dd0/ddb 0 2024-04-27T04:10:11.730 INFO:tasks.workunit.client.1.smithi181.stdout:1/725: chown d1/dad/db5/cc1 685466 1 2024-04-27T04:10:11.733 INFO:tasks.workunit.client.1.smithi181.stdout:5/868: dwrite d4/de/d2c/d46/d4d/d8e/f41 [0,4194304] 0 2024-04-27T04:10:11.734 INFO:tasks.workunit.client.0.smithi162.stdout:1/539: getdents d0/d11/d50/d81 0 2024-04-27T04:10:11.735 INFO:tasks.workunit.client.1.smithi181.stdout:1/726: dread d1/f8e [0,4194304] 0 2024-04-27T04:10:11.738 INFO:tasks.workunit.client.1.smithi181.stdout:0/836: rmdir dc/d1f/d3b/d4a/d95/d26/d41/d6f/d7d 39 2024-04-27T04:10:11.742 INFO:tasks.workunit.client.0.smithi162.stdout:9/564: creat d3/d13/d15/d50/faa x:0 0 0 2024-04-27T04:10:11.742 INFO:tasks.workunit.client.0.smithi162.stdout:9/565: chown d3/d13/d2d/d3b/d72/c9d 2162075 1 2024-04-27T04:10:11.743 INFO:tasks.workunit.client.0.smithi162.stdout:9/566: write d3/d11/d45/f99 [477397,114724] 0 2024-04-27T04:10:11.747 INFO:tasks.workunit.client.1.smithi181.stdout:5/869: creat d4/de/d107/f134 x:0 0 0 2024-04-27T04:10:11.748 INFO:tasks.workunit.client.0.smithi162.stdout:7/563: write d3/d5/d9/f7f [827608,85366] 0 2024-04-27T04:10:11.748 INFO:tasks.workunit.client.0.smithi162.stdout:8/580: symlink d0/de/d20/lc1 0 2024-04-27T04:10:11.749 INFO:tasks.workunit.client.1.smithi181.stdout:7/718: sync 2024-04-27T04:10:11.750 INFO:tasks.workunit.client.1.smithi181.stdout:5/870: truncate d4/d10/d29/d92/dbb/fe9 1684158 0 2024-04-27T04:10:11.750 INFO:tasks.workunit.client.1.smithi181.stdout:6/608: creat d7/dc/dc3/fd0 x:0 0 0 2024-04-27T04:10:11.751 INFO:tasks.workunit.client.0.smithi162.stdout:3/543: mkdir d1/d45/d69/da3/db0 0 2024-04-27T04:10:11.754 INFO:tasks.workunit.client.1.smithi181.stdout:8/771: symlink d1/d6/d2f/d30/de4/l109 0 2024-04-27T04:10:11.755 INFO:tasks.workunit.client.1.smithi181.stdout:8/772: chown d1/d6/d2f/d30/d66 57092 1 2024-04-27T04:10:11.756 INFO:tasks.workunit.client.1.smithi181.stdout:1/727: creat d1/d2/d64/ddc/ff3 x:0 0 0 2024-04-27T04:10:11.760 INFO:tasks.workunit.client.0.smithi162.stdout:0/645: link d5/dc/d1e/d46/d48/fbb d5/d26/da2/fd4 0 2024-04-27T04:10:11.760 INFO:tasks.workunit.client.1.smithi181.stdout:6/609: dwrite d7/dc/d1c/d7f/d3e/fc0 [0,4194304] 0 2024-04-27T04:10:11.760 INFO:tasks.workunit.client.0.smithi162.stdout:0/646: write d5/d60/d98/fa3 [665570,62786] 0 2024-04-27T04:10:11.761 INFO:tasks.workunit.client.0.smithi162.stdout:0/647: chown d5/dc/d1e/d46/d48/fb1 6 1 2024-04-27T04:10:11.761 INFO:tasks.workunit.client.0.smithi162.stdout:2/496: getdents d0/d4/d2a/d3b/d49/d5a/d63/d84/d93/d7d 0 2024-04-27T04:10:11.762 INFO:tasks.workunit.client.1.smithi181.stdout:5/871: symlink d4/da3/l135 0 2024-04-27T04:10:11.764 INFO:tasks.workunit.client.1.smithi181.stdout:5/872: chown d4/d10/d29/d92/cdd 190407 1 2024-04-27T04:10:11.766 INFO:tasks.workunit.client.0.smithi162.stdout:9/567: dread d3/d11/d45/d4e/d58/f80 [0,4194304] 0 2024-04-27T04:10:11.767 INFO:tasks.workunit.client.1.smithi181.stdout:2/783: link d4/d9/d16/d82/d7d/fe0 d4/d9/d16/d45/d9a/ff9 0 2024-04-27T04:10:11.767 INFO:tasks.workunit.client.1.smithi181.stdout:4/676: dread d0/d15/d76/f95 [0,4194304] 0 2024-04-27T04:10:11.767 INFO:tasks.workunit.client.0.smithi162.stdout:4/531: unlink d2/da/f44 0 2024-04-27T04:10:11.769 INFO:tasks.workunit.client.0.smithi162.stdout:7/564: creat d3/d5/d5b/d83/de/fbc x:0 0 0 2024-04-27T04:10:11.776 INFO:tasks.workunit.client.0.smithi162.stdout:1/540: rename d0/d7/l8 to d0/d7/d3c/d54/lb2 0 2024-04-27T04:10:11.777 INFO:tasks.workunit.client.0.smithi162.stdout:1/541: dread - d0/d80/f87 zero size 2024-04-27T04:10:11.777 INFO:tasks.workunit.client.0.smithi162.stdout:1/542: fsync d0/d7/d2a/f47 0 2024-04-27T04:10:11.778 INFO:tasks.workunit.client.0.smithi162.stdout:2/497: mkdir d0/d4/d17/d2b/d64/d96 0 2024-04-27T04:10:11.778 INFO:tasks.workunit.client.0.smithi162.stdout:1/543: stat d0/d11/d50/d53/d93/d96 0 2024-04-27T04:10:11.780 INFO:tasks.workunit.client.0.smithi162.stdout:9/568: creat d3/d13/d2d/fab x:0 0 0 2024-04-27T04:10:11.780 INFO:tasks.workunit.client.1.smithi181.stdout:8/773: mknod d1/d6/d6f/dd2/c10a 0 2024-04-27T04:10:11.781 INFO:tasks.workunit.client.1.smithi181.stdout:6/610: stat d7/dc/d1c/d34/c5f 0 2024-04-27T04:10:11.781 INFO:tasks.workunit.client.1.smithi181.stdout:0/837: creat dc/f123 x:0 0 0 2024-04-27T04:10:11.781 INFO:tasks.workunit.client.1.smithi181.stdout:5/873: creat d4/de/d2c/d46/d4d/d8e/d54/f136 x:0 0 0 2024-04-27T04:10:11.781 INFO:tasks.workunit.client.1.smithi181.stdout:4/677: dread - d0/d15/d25/d2a/d36/f92 zero size 2024-04-27T04:10:11.782 INFO:tasks.workunit.client.0.smithi162.stdout:6/567: getdents d6/d1a/d34/d3a/d68/d6c/d67 0 2024-04-27T04:10:11.784 INFO:tasks.workunit.client.1.smithi181.stdout:4/678: dread - d0/d15/de/d19/d8a/fbe zero size 2024-04-27T04:10:11.784 INFO:tasks.workunit.client.1.smithi181.stdout:6/611: read d7/dc/dc3/f3c [1414977,87529] 0 2024-04-27T04:10:11.784 INFO:tasks.workunit.client.0.smithi162.stdout:8/581: truncate d0/d10/d2b/d2e/f46 416785 0 2024-04-27T04:10:11.786 INFO:tasks.workunit.client.0.smithi162.stdout:8/582: write d0/de/d6e/d71/f95 [1322445,106759] 0 2024-04-27T04:10:11.786 INFO:tasks.workunit.client.1.smithi181.stdout:5/874: fsync d4/f112 0 2024-04-27T04:10:11.787 INFO:tasks.workunit.client.1.smithi181.stdout:5/875: write d4/d10/d29/d122/d100/ff7 [915823,1936] 0 2024-04-27T04:10:11.793 INFO:tasks.workunit.client.0.smithi162.stdout:7/565: rename d3/d5/d5b/d60/d65/d80 to d3/d5/d5b/d83/de/d2b/d81/daa/dbd 0 2024-04-27T04:10:11.794 INFO:tasks.workunit.client.0.smithi162.stdout:7/566: truncate d3/d5/d5b/d60/f94 65353 0 2024-04-27T04:10:11.795 INFO:tasks.workunit.client.1.smithi181.stdout:7/719: dwrite d1/d22/d15/d29/f43 [0,4194304] 0 2024-04-27T04:10:11.795 INFO:tasks.workunit.client.1.smithi181.stdout:2/784: mknod d4/d9/d35/d9f/dd0/ddb/cfa 0 2024-04-27T04:10:11.798 INFO:tasks.workunit.client.1.smithi181.stdout:4/679: dread - d0/d15/de/d60/d6f/f98 zero size 2024-04-27T04:10:11.810 INFO:tasks.workunit.client.1.smithi181.stdout:5/876: truncate d4/d10/d29/d122/d100/d8f/fe8 774450 0 2024-04-27T04:10:11.814 INFO:tasks.workunit.client.0.smithi162.stdout:1/544: symlink d0/d7/d10/lb3 0 2024-04-27T04:10:11.815 INFO:tasks.workunit.client.1.smithi181.stdout:8/774: creat d1/d20/df4/f10b x:0 0 0 2024-04-27T04:10:11.815 INFO:tasks.workunit.client.0.smithi162.stdout:1/545: read d0/d11/d50/d53/f84 [1300300,71009] 0 2024-04-27T04:10:11.816 INFO:tasks.workunit.client.0.smithi162.stdout:9/569: symlink d3/d23/d5d/lac 0 2024-04-27T04:10:11.820 INFO:tasks.workunit.client.1.smithi181.stdout:0/838: dwrite f6 [0,4194304] 0 2024-04-27T04:10:11.820 INFO:tasks.workunit.client.0.smithi162.stdout:4/532: fdatasync d2/d25/d1c/d39/d3b/f94 0 2024-04-27T04:10:11.820 INFO:tasks.workunit.client.0.smithi162.stdout:4/533: readlink d2/l67 0 2024-04-27T04:10:11.822 INFO:tasks.workunit.client.1.smithi181.stdout:0/839: write dc/d1f/d3b/db2/fdb [848424,20447] 0 2024-04-27T04:10:11.822 INFO:tasks.workunit.client.1.smithi181.stdout:7/720: rmdir d1/d2/d1c/d37 39 2024-04-27T04:10:11.825 INFO:tasks.workunit.client.0.smithi162.stdout:6/568: mknod d6/d1a/d34/d3a/d68/d6c/cb2 0 2024-04-27T04:10:11.828 INFO:tasks.workunit.client.0.smithi162.stdout:3/544: symlink d1/d18/d23/d58/lb1 0 2024-04-27T04:10:11.828 INFO:tasks.workunit.client.0.smithi162.stdout:3/545: readlink d1/d18/d23/l31 0 2024-04-27T04:10:11.829 INFO:tasks.workunit.client.0.smithi162.stdout:0/648: truncate d5/dc/d1e/d41/f4e 7563909 0 2024-04-27T04:10:11.830 INFO:tasks.workunit.client.0.smithi162.stdout:0/649: fsync d5/dc/d1e/d46/d48/f97 0 2024-04-27T04:10:11.832 INFO:tasks.workunit.client.0.smithi162.stdout:8/583: mkdir d0/d10/d2b/d2d/d2f/d39/d7c/dc2 0 2024-04-27T04:10:11.833 INFO:tasks.workunit.client.0.smithi162.stdout:5/605: getdents d8/d1a 0 2024-04-27T04:10:11.835 INFO:tasks.workunit.client.1.smithi181.stdout:5/877: creat d4/d10/d29/d92/dbb/f137 x:0 0 0 2024-04-27T04:10:11.837 INFO:tasks.workunit.client.1.smithi181.stdout:8/775: creat d1/d6/d6f/dd2/f10c x:0 0 0 2024-04-27T04:10:11.838 INFO:tasks.workunit.client.0.smithi162.stdout:7/567: mkdir d3/d5/d5b/d83/de/d2b/d81/daa/dbd/dbe 0 2024-04-27T04:10:11.839 INFO:tasks.workunit.client.0.smithi162.stdout:2/498: symlink d0/d4/d2a/d3b/d49/d5a/d7e/l97 0 2024-04-27T04:10:11.840 INFO:tasks.workunit.client.1.smithi181.stdout:0/840: creat dc/d1f/d3b/db2/f124 x:0 0 0 2024-04-27T04:10:11.843 INFO:tasks.workunit.client.1.smithi181.stdout:7/721: fsync d1/d2/d1c/d37/f59 0 2024-04-27T04:10:11.846 INFO:tasks.workunit.client.1.smithi181.stdout:6/612: truncate d7/dc/d1c/d34/f84 280650 0 2024-04-27T04:10:11.848 INFO:tasks.workunit.client.1.smithi181.stdout:4/680: creat d0/d15/d25/d8f/fdf x:0 0 0 2024-04-27T04:10:11.849 INFO:tasks.workunit.client.1.smithi181.stdout:0/841: dwrite dc/f18 [0,4194304] 0 2024-04-27T04:10:11.851 INFO:tasks.workunit.client.1.smithi181.stdout:5/878: creat d4/d10/dec/f138 x:0 0 0 2024-04-27T04:10:11.853 INFO:tasks.workunit.client.0.smithi162.stdout:4/534: read d2/f11 [330102,26169] 0 2024-04-27T04:10:11.854 INFO:tasks.workunit.client.1.smithi181.stdout:8/776: mkdir d1/d20/d10d 0 2024-04-27T04:10:11.854 INFO:tasks.workunit.client.1.smithi181.stdout:2/785: dread d4/d9/f1a [0,4194304] 0 2024-04-27T04:10:11.854 INFO:tasks.workunit.client.1.smithi181.stdout:4/681: dread d0/d15/de/d19/d8a/fc8 [0,4194304] 0 2024-04-27T04:10:11.855 INFO:tasks.workunit.client.0.smithi162.stdout:6/569: chown d6/ca9 50889 1 2024-04-27T04:10:11.861 INFO:tasks.workunit.client.0.smithi162.stdout:3/546: rename d1/d18/d23/d2f/f42 to d1/d45/d69/fb2 0 2024-04-27T04:10:11.865 INFO:tasks.workunit.client.1.smithi181.stdout:7/722: rename d1/d22/d15/dc6/dd8/c5c to d1/d2/d35/d67/dbf/d9a/cdb 0 2024-04-27T04:10:11.866 INFO:tasks.workunit.client.0.smithi162.stdout:8/584: rmdir d0/d10/d2b/d2d/d2f 39 2024-04-27T04:10:11.870 INFO:tasks.workunit.client.1.smithi181.stdout:0/842: rmdir dc/d1f/d6b/db9/d106/d110 39 2024-04-27T04:10:11.872 INFO:tasks.workunit.client.0.smithi162.stdout:5/606: chown d8/d9/d5a/c7a 231 1 2024-04-27T04:10:11.872 INFO:tasks.workunit.client.1.smithi181.stdout:7/723: dwrite d1/d2/d1c/f1e [0,4194304] 0 2024-04-27T04:10:11.873 INFO:tasks.workunit.client.1.smithi181.stdout:5/879: read - d4/de/d107/ff9 zero size 2024-04-27T04:10:11.873 INFO:tasks.workunit.client.0.smithi162.stdout:9/570: write d3/d13/d15/f19 [1915265,88393] 0 2024-04-27T04:10:11.874 INFO:tasks.workunit.client.1.smithi181.stdout:9/687: sync 2024-04-27T04:10:11.874 INFO:tasks.workunit.client.1.smithi181.stdout:3/697: sync 2024-04-27T04:10:11.875 INFO:tasks.workunit.client.1.smithi181.stdout:3/698: readlink d0/d6c/l4a 0 2024-04-27T04:10:11.875 INFO:tasks.workunit.client.0.smithi162.stdout:7/568: rmdir d3/d5/d5b/d83 39 2024-04-27T04:10:11.880 INFO:tasks.workunit.client.1.smithi181.stdout:7/724: dread d1/d2/d1c/f1e [0,4194304] 0 2024-04-27T04:10:11.882 INFO:tasks.workunit.client.1.smithi181.stdout:8/777: mknod d1/d6/d2f/d30/d66/dc0/dcd/de1/c10e 0 2024-04-27T04:10:11.884 INFO:tasks.workunit.client.1.smithi181.stdout:4/682: mkdir d0/d15/d25/d40/de0 0 2024-04-27T04:10:11.885 INFO:tasks.workunit.client.0.smithi162.stdout:1/546: unlink d0/d11/d50/d81/c86 0 2024-04-27T04:10:11.894 INFO:tasks.workunit.client.0.smithi162.stdout:4/535: dread d2/da/f12 [0,4194304] 0 2024-04-27T04:10:11.896 INFO:tasks.workunit.client.0.smithi162.stdout:3/547: chown d1/d70/f8c 1 1 2024-04-27T04:10:11.897 INFO:tasks.workunit.client.1.smithi181.stdout:2/786: dwrite d4/d9/d16/d45/ff7 [0,4194304] 0 2024-04-27T04:10:11.899 INFO:tasks.workunit.client.1.smithi181.stdout:2/787: chown d4/d9/fc0 756 1 2024-04-27T04:10:11.900 INFO:tasks.workunit.client.0.smithi162.stdout:0/650: mknod d5/dc/d1e/d20/dba/cd5 0 2024-04-27T04:10:11.901 INFO:tasks.workunit.client.1.smithi181.stdout:6/613: write d7/d31/f37 [217695,40630] 0 2024-04-27T04:10:11.901 INFO:tasks.workunit.client.1.smithi181.stdout:1/728: sync 2024-04-27T04:10:11.901 INFO:tasks.workunit.client.1.smithi181.stdout:5/880: truncate d4/de/d2c/f75 1117594 0 2024-04-27T04:10:11.906 INFO:tasks.workunit.client.1.smithi181.stdout:9/688: creat d0/d44/d4b/d3d/fe9 x:0 0 0 2024-04-27T04:10:11.906 INFO:tasks.workunit.client.1.smithi181.stdout:3/699: truncate d0/d6c/f79 846624 0 2024-04-27T04:10:11.908 INFO:tasks.workunit.client.1.smithi181.stdout:9/689: chown d0/d44/d4b/d3d/f73 126625006 1 2024-04-27T04:10:11.909 INFO:tasks.workunit.client.0.smithi162.stdout:5/607: rename d8/f6f to d8/d1a/d29/d4a/fbd 0 2024-04-27T04:10:11.910 INFO:tasks.workunit.client.1.smithi181.stdout:6/614: dread d7/dc/d8b/fa7 [0,4194304] 0 2024-04-27T04:10:11.911 INFO:tasks.workunit.client.0.smithi162.stdout:9/571: mknod d3/cad 0 2024-04-27T04:10:11.913 INFO:tasks.workunit.client.1.smithi181.stdout:7/725: fsync d1/d2/d35/d67/f69 0 2024-04-27T04:10:11.914 INFO:tasks.workunit.client.1.smithi181.stdout:0/843: dread dc/d1f/f48 [4194304,4194304] 0 2024-04-27T04:10:11.915 INFO:tasks.workunit.client.1.smithi181.stdout:0/844: chown dc/f1b 855 1 2024-04-27T04:10:11.915 INFO:tasks.workunit.client.0.smithi162.stdout:6/570: dread d6/d1a/f5d [0,4194304] 0 2024-04-27T04:10:11.916 INFO:tasks.workunit.client.1.smithi181.stdout:8/778: symlink d1/d20/da4/dc5/l10f 0 2024-04-27T04:10:11.921 INFO:tasks.workunit.client.0.smithi162.stdout:4/536: truncate d2/d25/d1c/f55 5226645 0 2024-04-27T04:10:11.922 INFO:tasks.workunit.client.0.smithi162.stdout:0/651: creat d5/d60/d8b/fd6 x:0 0 0 2024-04-27T04:10:11.923 INFO:tasks.workunit.client.0.smithi162.stdout:0/652: dread - d5/dc/d1e/fbf zero size 2024-04-27T04:10:11.927 INFO:tasks.workunit.client.0.smithi162.stdout:7/569: write d3/d4f/f67 [3727408,38951] 0 2024-04-27T04:10:11.935 INFO:tasks.workunit.client.0.smithi162.stdout:2/499: dwrite d0/d4/d2a/d3b/d49/d5a/d63/d84/d93/f76 [0,4194304] 0 2024-04-27T04:10:11.936 INFO:tasks.workunit.client.0.smithi162.stdout:1/547: rename d0/d7/d3c/d54/l8c to d0/d80/d89/lb4 0 2024-04-27T04:10:11.937 INFO:tasks.workunit.client.1.smithi181.stdout:2/788: symlink d4/d9/d16/d82/d7d/db7/lfb 0 2024-04-27T04:10:11.943 INFO:tasks.workunit.client.0.smithi162.stdout:2/500: dread d0/d24/f87 [0,4194304] 0 2024-04-27T04:10:11.944 INFO:tasks.workunit.client.0.smithi162.stdout:2/501: chown d0/d4/d17/d2b/f95 4 1 2024-04-27T04:10:11.944 INFO:tasks.workunit.client.0.smithi162.stdout:2/502: chown d0/d24/f38 44863573 1 2024-04-27T04:10:11.947 INFO:tasks.workunit.client.1.smithi181.stdout:3/700: mknod d0/d6c/d52/d44/dd5/d6b/ce2 0 2024-04-27T04:10:11.954 INFO:tasks.workunit.client.0.smithi162.stdout:0/653: unlink d5/dc/fe 0 2024-04-27T04:10:11.955 INFO:tasks.workunit.client.1.smithi181.stdout:6/615: dwrite d7/dc/ff [0,4194304] 0 2024-04-27T04:10:11.957 INFO:tasks.workunit.client.1.smithi181.stdout:0/845: unlink dc/d1f/d3b/d4a/d95/c70 0 2024-04-27T04:10:11.958 INFO:tasks.workunit.client.0.smithi162.stdout:8/585: link d0/c1f d0/d10/d44/cc3 0 2024-04-27T04:10:11.959 INFO:tasks.workunit.client.1.smithi181.stdout:4/683: fdatasync d0/d15/de/d60/fc4 0 2024-04-27T04:10:11.964 INFO:tasks.workunit.client.1.smithi181.stdout:5/881: fdatasync d4/da3/dbc/f9e 0 2024-04-27T04:10:11.966 INFO:tasks.workunit.client.1.smithi181.stdout:5/882: chown d4/de/d2c/d46/d4d/d8e/d54 115 1 2024-04-27T04:10:11.968 INFO:tasks.workunit.client.1.smithi181.stdout:4/684: dwrite d0/d15/d25/d2b/d6d/d82/fc3 [0,4194304] 0 2024-04-27T04:10:11.969 INFO:tasks.workunit.client.1.smithi181.stdout:5/883: write d4/de/d2c/d46/d4d/d8e/f8a [3849580,109025] 0 2024-04-27T04:10:11.975 INFO:tasks.workunit.client.1.smithi181.stdout:3/701: write d0/d6c/dcc/f75 [2178139,33988] 0 2024-04-27T04:10:11.976 INFO:tasks.workunit.client.1.smithi181.stdout:3/702: write d0/d6c/d52/dbb/fc7 [912863,75850] 0 2024-04-27T04:10:11.977 INFO:tasks.workunit.client.0.smithi162.stdout:2/503: creat d0/d4/d2a/d3b/d49/d5a/d63/d6e/f98 x:0 0 0 2024-04-27T04:10:11.977 INFO:tasks.workunit.client.1.smithi181.stdout:3/703: chown d0/d6c/d52/d44/l49 125380 1 2024-04-27T04:10:11.980 INFO:tasks.workunit.client.0.smithi162.stdout:6/571: mkdir d6/d1a/d34/d7f/d80/db3 0 2024-04-27T04:10:11.980 INFO:tasks.workunit.client.0.smithi162.stdout:9/572: dread d3/d13/d2d/d3b/d54/f87 [0,4194304] 0 2024-04-27T04:10:11.982 INFO:tasks.workunit.client.1.smithi181.stdout:3/704: dread d0/f57 [0,4194304] 0 2024-04-27T04:10:11.982 INFO:tasks.workunit.client.0.smithi162.stdout:9/573: chown d3/d13/d2d/d3b/d54/f87 394806454 1 2024-04-27T04:10:11.983 INFO:tasks.workunit.client.1.smithi181.stdout:1/729: write d1/dad/db5/d6/f7f [346640,44298] 0 2024-04-27T04:10:11.984 INFO:tasks.workunit.client.1.smithi181.stdout:7/726: symlink d1/dc3/daf/ldc 0 2024-04-27T04:10:11.985 INFO:tasks.workunit.client.0.smithi162.stdout:5/608: dwrite d8/d9/d55/f69 [0,4194304] 0 2024-04-27T04:10:11.985 INFO:tasks.workunit.client.1.smithi181.stdout:7/727: chown d1/dc3/daf/ldc 462624 1 2024-04-27T04:10:11.986 INFO:tasks.workunit.client.1.smithi181.stdout:7/728: chown d1/d22/d15/d82 3711 1 2024-04-27T04:10:11.988 INFO:tasks.workunit.client.0.smithi162.stdout:7/570: mknod d3/cbf 0 2024-04-27T04:10:11.994 INFO:tasks.workunit.client.0.smithi162.stdout:4/537: truncate d2/d25/d1c/d24/f3a 2537751 0 2024-04-27T04:10:11.996 INFO:tasks.workunit.client.0.smithi162.stdout:0/654: write d5/d26/d4a/f4c [335569,94800] 0 2024-04-27T04:10:11.999 INFO:tasks.workunit.client.1.smithi181.stdout:2/789: creat d4/d9/d4f/d79/d94/df4/ffc x:0 0 0 2024-04-27T04:10:12.000 INFO:tasks.workunit.client.1.smithi181.stdout:2/790: chown d4/d9/d4f/d79/l93 834678896 1 2024-04-27T04:10:12.001 INFO:tasks.workunit.client.1.smithi181.stdout:4/685: fdatasync d0/f30 0 2024-04-27T04:10:12.003 INFO:tasks.workunit.client.0.smithi162.stdout:6/572: creat d6/d1a/d34/d3a/d68/fb4 x:0 0 0 2024-04-27T04:10:12.003 INFO:tasks.workunit.client.1.smithi181.stdout:4/686: readlink d0/d15/de/d19/d77/l62 0 2024-04-27T04:10:12.003 INFO:tasks.workunit.client.0.smithi162.stdout:6/573: chown d6/db/la7 3 1 2024-04-27T04:10:12.004 INFO:tasks.workunit.client.0.smithi162.stdout:6/574: fdatasync d6/d1a/d34/d7b/d7e/d97/f9d 0 2024-04-27T04:10:12.008 INFO:tasks.workunit.client.0.smithi162.stdout:9/574: dwrite d3/d23/f73 [0,4194304] 0 2024-04-27T04:10:12.010 INFO:tasks.workunit.client.1.smithi181.stdout:0/846: dwrite dc/d1f/d3b/d4a/fc1 [0,4194304] 0 2024-04-27T04:10:12.018 INFO:tasks.workunit.client.1.smithi181.stdout:1/730: chown d1/dad/db5/d6/l29 125 1 2024-04-27T04:10:12.022 INFO:tasks.workunit.client.1.smithi181.stdout:3/705: creat d0/d6c/d66/dc2/fe3 x:0 0 0 2024-04-27T04:10:12.024 INFO:tasks.workunit.client.0.smithi162.stdout:3/548: rename d1/d18/d5d/d5f/d66/f7a to d1/d18/d23/d58/fb3 0 2024-04-27T04:10:12.024 INFO:tasks.workunit.client.1.smithi181.stdout:6/616: mkdir d7/dc/dc3/dd1 0 2024-04-27T04:10:12.026 INFO:tasks.workunit.client.0.smithi162.stdout:4/538: mkdir d2/d25/d1c/d39/d3b/da7 0 2024-04-27T04:10:12.026 INFO:tasks.workunit.client.0.smithi162.stdout:0/655: truncate d5/dc/d1e/f21 7550389 0 2024-04-27T04:10:12.028 INFO:tasks.workunit.client.0.smithi162.stdout:2/504: link d0/d4/d17/d4f/f85 d0/d4/d2a/d3b/d49/d5a/d63/d84/f99 0 2024-04-27T04:10:12.030 INFO:tasks.workunit.client.0.smithi162.stdout:2/505: truncate d0/d4/d17/d4f/f85 308129 0 2024-04-27T04:10:12.030 INFO:tasks.workunit.client.0.smithi162.stdout:6/575: mknod d6/d1a/d34/d7b/d7e/d97/cb5 0 2024-04-27T04:10:12.031 INFO:tasks.workunit.client.0.smithi162.stdout:6/576: fsync d6/d1a/d34/d43/f6b 0 2024-04-27T04:10:12.032 INFO:tasks.workunit.client.0.smithi162.stdout:6/577: dread - d6/d1a/d34/d50/f99 zero size 2024-04-27T04:10:12.032 INFO:tasks.workunit.client.0.smithi162.stdout:9/575: rmdir d3/d23 39 2024-04-27T04:10:12.034 INFO:tasks.workunit.client.0.smithi162.stdout:5/609: mknod d8/d1a/d23/dbc/d66/cbe 0 2024-04-27T04:10:12.034 INFO:tasks.workunit.client.1.smithi181.stdout:2/791: symlink d4/d9/d16/d82/d7d/db7/dcb/lfd 0 2024-04-27T04:10:12.036 INFO:tasks.workunit.client.0.smithi162.stdout:8/586: creat d0/d10/d2b/d2d/fc4 x:0 0 0 2024-04-27T04:10:12.036 INFO:tasks.workunit.client.1.smithi181.stdout:2/792: write d4/d9/d16/d45/fdc [1460899,57715] 0 2024-04-27T04:10:12.043 INFO:tasks.workunit.client.1.smithi181.stdout:0/847: truncate dc/f94 691861 0 2024-04-27T04:10:12.044 INFO:tasks.workunit.client.0.smithi162.stdout:7/571: dwrite d3/d5/d3c/d4a/f6a [0,4194304] 0 2024-04-27T04:10:12.045 INFO:tasks.workunit.client.1.smithi181.stdout:9/690: getdents d0/d44/d4b/d43/d8f 0 2024-04-27T04:10:12.048 INFO:tasks.workunit.client.1.smithi181.stdout:1/731: mknod d1/d2/cf4 0 2024-04-27T04:10:12.049 INFO:tasks.workunit.client.0.smithi162.stdout:4/539: symlink d2/d25/la8 0 2024-04-27T04:10:12.052 INFO:tasks.workunit.client.1.smithi181.stdout:3/706: rename d0/d6c/da1 to d0/d6c/d52/d44/dd5/d58/de4 0 2024-04-27T04:10:12.053 INFO:tasks.workunit.client.0.smithi162.stdout:0/656: mknod d5/dc/d1e/d41/d79/cd7 0 2024-04-27T04:10:12.054 INFO:tasks.workunit.client.0.smithi162.stdout:0/657: fdatasync f1 0 2024-04-27T04:10:12.056 INFO:tasks.workunit.client.1.smithi181.stdout:6/617: creat d7/dc/d1c/d34/d3f/d7e/d8a/fd2 x:0 0 0 2024-04-27T04:10:12.057 INFO:tasks.workunit.client.0.smithi162.stdout:5/610: dread d8/d1a/d29/d4a/d5c/f62 [0,4194304] 0 2024-04-27T04:10:12.057 INFO:tasks.workunit.client.1.smithi181.stdout:7/729: mknod d1/d2/d1c/d37/cdd 0 2024-04-27T04:10:12.059 INFO:tasks.workunit.client.1.smithi181.stdout:8/779: getdents d1/d6/d2f/d30/d5c 0 2024-04-27T04:10:12.063 INFO:tasks.workunit.client.1.smithi181.stdout:5/884: link d4/d10/d29/d92/dbb/lc5 d4/d10/d29/d122/de7/l139 0 2024-04-27T04:10:12.066 INFO:tasks.workunit.client.1.smithi181.stdout:0/848: fsync dc/d1f/d6b/fc0 0 2024-04-27T04:10:12.070 INFO:tasks.workunit.client.1.smithi181.stdout:1/732: dread d1/dad/db5/fb8 [0,4194304] 0 2024-04-27T04:10:12.071 INFO:tasks.workunit.client.1.smithi181.stdout:1/733: stat d1/d2/d89/lbb 0 2024-04-27T04:10:12.072 INFO:tasks.workunit.client.1.smithi181.stdout:9/691: rename d0/d44/d4b/l41 to d0/d44/d4b/d43/d88/da1/lea 0 2024-04-27T04:10:12.074 INFO:tasks.workunit.client.1.smithi181.stdout:1/734: read d1/dad/db5/d6/da3/d52/f76 [2759833,72298] 0 2024-04-27T04:10:12.077 INFO:tasks.workunit.client.0.smithi162.stdout:4/540: unlink d2/f22 0 2024-04-27T04:10:12.077 INFO:tasks.workunit.client.0.smithi162.stdout:4/541: read - d2/da/fa1 zero size 2024-04-27T04:10:12.078 INFO:tasks.workunit.client.1.smithi181.stdout:6/618: fdatasync d7/fb9 0 2024-04-27T04:10:12.080 INFO:tasks.workunit.client.1.smithi181.stdout:7/730: fsync d1/d2/d1c/d2d/f41 0 2024-04-27T04:10:12.082 INFO:tasks.workunit.client.1.smithi181.stdout:8/780: symlink d1/d20/df4/l110 0 2024-04-27T04:10:12.083 INFO:tasks.workunit.client.0.smithi162.stdout:2/506: symlink d0/d20/l9a 0 2024-04-27T04:10:12.084 INFO:tasks.workunit.client.0.smithi162.stdout:3/549: write d1/d45/d69/fb2 [4327869,12368] 0 2024-04-27T04:10:12.085 INFO:tasks.workunit.client.0.smithi162.stdout:5/611: read d8/d1a/d23/d40/f42 [2719841,16020] 0 2024-04-27T04:10:12.086 INFO:tasks.workunit.client.0.smithi162.stdout:2/507: truncate d0/d4/d2a/d3b/d49/d5a/d63/d84/f99 536932 0 2024-04-27T04:10:12.086 INFO:tasks.workunit.client.1.smithi181.stdout:5/885: unlink d4/da3/fa7 0 2024-04-27T04:10:12.088 INFO:tasks.workunit.client.1.smithi181.stdout:5/886: chown d4/de/d2c/d46/d4d/d8e/d67/cf6 79 1 2024-04-27T04:10:12.091 INFO:tasks.workunit.client.1.smithi181.stdout:9/692: truncate d0/d44/d4b/d43/f56 576867 0 2024-04-27T04:10:12.092 INFO:tasks.workunit.client.0.smithi162.stdout:8/587: creat d0/d10/d25/d7d/d87/fc5 x:0 0 0 2024-04-27T04:10:12.093 INFO:tasks.workunit.client.1.smithi181.stdout:3/707: dwrite d0/f32 [0,4194304] 0 2024-04-27T04:10:12.094 INFO:tasks.workunit.client.1.smithi181.stdout:6/619: dread - d7/dc/d1c/d7f/f76 zero size 2024-04-27T04:10:12.095 INFO:tasks.workunit.client.0.smithi162.stdout:7/572: mknod d3/d5/d5b/d83/cc0 0 2024-04-27T04:10:12.096 INFO:tasks.workunit.client.0.smithi162.stdout:7/573: chown d3/d5/d9/c8b 3304 1 2024-04-27T04:10:12.098 INFO:tasks.workunit.client.0.smithi162.stdout:4/542: mkdir d2/d25/d1c/d39/da9 0 2024-04-27T04:10:12.100 INFO:tasks.workunit.client.0.smithi162.stdout:1/548: sync 2024-04-27T04:10:12.101 INFO:tasks.workunit.client.0.smithi162.stdout:0/658: stat d5/d26/d4a/c7d 0 2024-04-27T04:10:12.101 INFO:tasks.workunit.client.0.smithi162.stdout:4/543: read d2/d25/fb [3207398,46023] 0 2024-04-27T04:10:12.101 INFO:tasks.workunit.client.0.smithi162.stdout:1/549: dread - d0/d7/f88 zero size 2024-04-27T04:10:12.102 INFO:tasks.workunit.client.0.smithi162.stdout:0/659: stat d5/dc/d1e/d46/d48/d9e/db8/fc6 0 2024-04-27T04:10:12.102 INFO:tasks.workunit.client.1.smithi181.stdout:2/793: link d4/d9/d16/c1e d4/d9/d16/d82/d7d/db7/cfe 0 2024-04-27T04:10:12.104 INFO:tasks.workunit.client.1.smithi181.stdout:0/849: dwrite dc/d54/fab [0,4194304] 0 2024-04-27T04:10:12.105 INFO:tasks.workunit.client.0.smithi162.stdout:6/578: creat d6/d1a/d34/fb6 x:0 0 0 2024-04-27T04:10:12.110 INFO:tasks.workunit.client.1.smithi181.stdout:2/794: dwrite d4/f50 [0,4194304] 0 2024-04-27T04:10:12.112 INFO:tasks.workunit.client.0.smithi162.stdout:2/508: rename d0/d20/c21 to d0/d4/d2a/d3b/d49/d5a/d63/c9b 0 2024-04-27T04:10:12.113 INFO:tasks.workunit.client.1.smithi181.stdout:8/781: write d1/d6/d6f/dd2/fd4 [326393,89829] 0 2024-04-27T04:10:12.115 INFO:tasks.workunit.client.1.smithi181.stdout:9/693: symlink d0/d44/d4b/d5f/da4/leb 0 2024-04-27T04:10:12.116 INFO:tasks.workunit.client.1.smithi181.stdout:8/782: fdatasync d1/d6/d2f/d30/d9d/dea/db3/ddf/ff7 0 2024-04-27T04:10:12.117 INFO:tasks.workunit.client.0.smithi162.stdout:8/588: unlink d0/d10/d25/d65/f9e 0 2024-04-27T04:10:12.119 INFO:tasks.workunit.client.0.smithi162.stdout:8/589: write d0/d10/d2b/d2e/f40 [1194258,95546] 0 2024-04-27T04:10:12.123 INFO:tasks.workunit.client.1.smithi181.stdout:5/887: rename d4/d10/d72 to d4/de/d2c/d46/d99/dad/d13a 0 2024-04-27T04:10:12.124 INFO:tasks.workunit.client.0.smithi162.stdout:3/550: mknod d1/d18/d23/d34/daa/dab/cb4 0 2024-04-27T04:10:12.128 INFO:tasks.workunit.client.0.smithi162.stdout:4/544: creat d2/d25/d1c/d39/d70/faa x:0 0 0 2024-04-27T04:10:12.131 INFO:tasks.workunit.client.0.smithi162.stdout:5/612: link d8/d1a/f51 d8/d1a/fbf 0 2024-04-27T04:10:12.132 INFO:tasks.workunit.client.0.smithi162.stdout:2/509: stat d0/d20/c6d 0 2024-04-27T04:10:12.133 INFO:tasks.workunit.client.0.smithi162.stdout:2/510: fsync d0/d4/d17/f2f 0 2024-04-27T04:10:12.134 INFO:tasks.workunit.client.0.smithi162.stdout:9/576: getdents d3/d13/d2d/d3b/d7a 0 2024-04-27T04:10:12.135 INFO:tasks.workunit.client.1.smithi181.stdout:9/694: creat d0/d44/d4b/d3d/dbd/dd1/fec x:0 0 0 2024-04-27T04:10:12.136 INFO:tasks.workunit.client.1.smithi181.stdout:9/695: stat d0/d44/d4f/d9d/fb4 0 2024-04-27T04:10:12.138 INFO:tasks.workunit.client.1.smithi181.stdout:8/783: symlink d1/d6/d2f/d30/d66/dc0/dcd/l111 0 2024-04-27T04:10:12.141 INFO:tasks.workunit.client.1.smithi181.stdout:6/620: rename d7/dc/d1e/d5c to d7/dc/dc3/d72/daa/dd3 0 2024-04-27T04:10:12.142 INFO:tasks.workunit.client.1.smithi181.stdout:6/621: dread - d7/dc/d1c/d88/fcb zero size 2024-04-27T04:10:12.143 INFO:tasks.workunit.client.0.smithi162.stdout:1/550: dwrite d0/d7/d3c/d6e/f71 [0,4194304] 0 2024-04-27T04:10:12.146 INFO:tasks.workunit.client.1.smithi181.stdout:0/850: mkdir dc/d1f/d6b/db9/dba/deb/df6/d125 0 2024-04-27T04:10:12.147 INFO:tasks.workunit.client.1.smithi181.stdout:0/851: readlink dc/d1f/le2 0 2024-04-27T04:10:12.149 INFO:tasks.workunit.client.1.smithi181.stdout:5/888: mknod d4/de/c13b 0 2024-04-27T04:10:12.150 INFO:tasks.workunit.client.0.smithi162.stdout:0/660: write d5/d26/f8d [1143793,67804] 0 2024-04-27T04:10:12.152 INFO:tasks.workunit.client.0.smithi162.stdout:5/613: mknod d8/d86/cc0 0 2024-04-27T04:10:12.153 INFO:tasks.workunit.client.0.smithi162.stdout:0/661: read d5/dc/f31 [3681842,78795] 0 2024-04-27T04:10:12.158 INFO:tasks.workunit.client.1.smithi181.stdout:0/852: mknod dc/d1f/d6b/d7e/d91/dea/c126 0 2024-04-27T04:10:12.159 INFO:tasks.workunit.client.0.smithi162.stdout:1/551: mknod d0/d80/d89/cb5 0 2024-04-27T04:10:12.160 INFO:tasks.workunit.client.0.smithi162.stdout:3/551: dread d1/d18/f1a [0,4194304] 0 2024-04-27T04:10:12.161 INFO:tasks.workunit.client.1.smithi181.stdout:6/622: symlink d7/dc/ld4 0 2024-04-27T04:10:12.161 INFO:tasks.workunit.client.1.smithi181.stdout:3/708: dwrite d0/d6c/d52/d44/dd5/d58/de4/daf/f81 [0,4194304] 0 2024-04-27T04:10:12.163 INFO:tasks.workunit.client.1.smithi181.stdout:9/696: creat d0/d4d/da0/ddb/fed x:0 0 0 2024-04-27T04:10:12.172 INFO:tasks.workunit.client.1.smithi181.stdout:3/709: dwrite d0/de/d7d/dbc/fd9 [0,4194304] 0 2024-04-27T04:10:12.174 INFO:tasks.workunit.client.1.smithi181.stdout:5/889: symlink d4/de/d2c/l13c 0 2024-04-27T04:10:12.177 INFO:tasks.workunit.client.0.smithi162.stdout:9/577: truncate d3/d23/d1e/d43/f48 6180234 0 2024-04-27T04:10:12.178 INFO:tasks.workunit.client.0.smithi162.stdout:9/578: fsync d3/d23/f73 0 2024-04-27T04:10:12.179 INFO:tasks.workunit.client.1.smithi181.stdout:6/623: rename d7/dc/d1e/d66/cac to d7/d31/cd5 0 2024-04-27T04:10:12.181 INFO:tasks.workunit.client.0.smithi162.stdout:0/662: dread d5/dc/d1e/d41/f34 [0,4194304] 0 2024-04-27T04:10:12.184 INFO:tasks.workunit.client.0.smithi162.stdout:8/590: write d0/d10/d2b/d2e/f46 [1086269,98432] 0 2024-04-27T04:10:12.184 INFO:tasks.workunit.client.1.smithi181.stdout:0/853: mkdir dc/d1f/d3b/df7/d127 0 2024-04-27T04:10:12.185 INFO:tasks.workunit.client.0.smithi162.stdout:8/591: write d0/f4 [8537887,125888] 0 2024-04-27T04:10:12.190 INFO:tasks.workunit.client.0.smithi162.stdout:4/545: creat d2/d25/d1c/fab x:0 0 0 2024-04-27T04:10:12.192 INFO:tasks.workunit.client.1.smithi181.stdout:6/624: mkdir d7/dc/dc3/d72/dd6 0 2024-04-27T04:10:12.192 INFO:tasks.workunit.client.0.smithi162.stdout:5/614: dwrite d8/d1a/f17 [4194304,4194304] 0 2024-04-27T04:10:12.195 INFO:tasks.workunit.client.1.smithi181.stdout:5/890: dread d4/de/d2c/d46/d99/dad/dca/fcc [0,4194304] 0 2024-04-27T04:10:12.198 INFO:tasks.workunit.client.0.smithi162.stdout:5/615: dread d8/fad [0,4194304] 0 2024-04-27T04:10:12.198 INFO:tasks.workunit.client.0.smithi162.stdout:2/511: link d0/c13 d0/d4/d17/d4f/c9c 0 2024-04-27T04:10:12.199 INFO:tasks.workunit.client.0.smithi162.stdout:2/512: read d0/d4/d17/f2f [877678,80919] 0 2024-04-27T04:10:12.201 INFO:tasks.workunit.client.1.smithi181.stdout:9/697: rename d0/d4d/l96 to d0/d44/d4b/d3d/dbd/lee 0 2024-04-27T04:10:12.205 INFO:tasks.workunit.client.0.smithi162.stdout:1/552: dread d0/d11/d3e/f4f [0,4194304] 0 2024-04-27T04:10:12.206 INFO:tasks.workunit.client.0.smithi162.stdout:9/579: mknod d3/d13/d2d/d3b/cae 0 2024-04-27T04:10:12.207 INFO:tasks.workunit.client.1.smithi181.stdout:5/891: symlink d4/d10/d29/d122/d100/d8f/l13d 0 2024-04-27T04:10:12.210 INFO:tasks.workunit.client.1.smithi181.stdout:0/854: creat dc/d1f/d3b/d4a/d95/d26/d41/d6f/d7d/dee/f128 x:0 0 0 2024-04-27T04:10:12.211 INFO:tasks.workunit.client.1.smithi181.stdout:9/698: chown d0/c24 1403 1 2024-04-27T04:10:12.218 INFO:tasks.workunit.client.1.smithi181.stdout:3/710: rename d0/d6c/dcc to d0/d6c/d52/dbb/de5 0 2024-04-27T04:10:12.218 INFO:tasks.workunit.client.1.smithi181.stdout:6/625: creat d7/dc/d1c/d34/d3f/db2/fd7 x:0 0 0 2024-04-27T04:10:12.220 INFO:tasks.workunit.client.1.smithi181.stdout:6/626: dread - d7/dc/d1c/d34/d3f/db2/fd7 zero size 2024-04-27T04:10:12.223 INFO:tasks.workunit.client.1.smithi181.stdout:6/627: chown d7/dc/dc3/f6b 123513046 1 2024-04-27T04:10:12.224 INFO:tasks.workunit.client.0.smithi162.stdout:2/513: dread - d0/d4/d2a/d41/f54 zero size 2024-04-27T04:10:12.227 INFO:tasks.workunit.client.1.smithi181.stdout:5/892: dread d4/de/d2c/f3e [0,4194304] 0 2024-04-27T04:10:12.227 INFO:tasks.workunit.client.0.smithi162.stdout:2/514: write d0/d4/f15 [4818424,70707] 0 2024-04-27T04:10:12.227 INFO:tasks.workunit.client.0.smithi162.stdout:3/552: dwrite d1/d18/f52 [0,4194304] 0 2024-04-27T04:10:12.230 INFO:tasks.workunit.client.0.smithi162.stdout:9/580: read d3/d13/d2d/d3b/d54/f6c [257601,99229] 0 2024-04-27T04:10:12.233 INFO:tasks.workunit.client.0.smithi162.stdout:0/663: write d5/d26/d69/f84 [76537,45748] 0 2024-04-27T04:10:12.233 INFO:tasks.workunit.client.0.smithi162.stdout:5/616: dwrite d8/d1a/d23/dbc/d66/d88/f98 [0,4194304] 0 2024-04-27T04:10:12.234 INFO:tasks.workunit.client.0.smithi162.stdout:9/581: write d3/d13/d15/f19 [4965774,109181] 0 2024-04-27T04:10:12.235 INFO:tasks.workunit.client.0.smithi162.stdout:8/592: fdatasync d0/d10/d2b/d2d/d2f/d39/f59 0 2024-04-27T04:10:12.237 INFO:tasks.workunit.client.0.smithi162.stdout:8/593: write d0/d10/d2b/d2d/d2f/d42/f48 [1230798,101697] 0 2024-04-27T04:10:12.239 INFO:tasks.workunit.client.0.smithi162.stdout:8/594: write d0/d10/d2b/d2d/d2f/d42/fa0 [831542,103207] 0 2024-04-27T04:10:12.240 INFO:tasks.workunit.client.0.smithi162.stdout:4/546: dread d2/d25/d1c/d39/f46 [0,4194304] 0 2024-04-27T04:10:12.244 INFO:tasks.workunit.client.0.smithi162.stdout:1/553: link d0/f1e d0/d7/d3c/fb6 0 2024-04-27T04:10:12.249 INFO:tasks.workunit.client.0.smithi162.stdout:2/515: truncate d0/d24/f87 952105 0 2024-04-27T04:10:12.251 INFO:tasks.workunit.client.0.smithi162.stdout:0/664: creat d5/d60/fd8 x:0 0 0 2024-04-27T04:10:12.253 INFO:tasks.workunit.client.0.smithi162.stdout:5/617: mknod d8/d2c/cc1 0 2024-04-27T04:10:12.253 INFO:tasks.workunit.client.0.smithi162.stdout:3/553: dwrite d1/d18/d23/d58/fb3 [0,4194304] 0 2024-04-27T04:10:12.254 INFO:tasks.workunit.client.0.smithi162.stdout:9/582: fsync d3/f14 0 2024-04-27T04:10:12.258 INFO:tasks.workunit.client.0.smithi162.stdout:4/547: symlink d2/d25/d1c/d39/d3b/lac 0 2024-04-27T04:10:12.260 INFO:tasks.workunit.client.1.smithi181.stdout:3/711: rename d0/d6c/d52/d44/dd5/f46 to d0/d6c/d52/d44/dd5/d58/d64/dc6/fe6 0 2024-04-27T04:10:12.260 INFO:tasks.workunit.client.1.smithi181.stdout:3/712: readlink d0/d6c/d52/d44/d74/la9 0 2024-04-27T04:10:12.262 INFO:tasks.workunit.client.0.smithi162.stdout:1/554: dread d0/d11/f20 [4194304,4194304] 0 2024-04-27T04:10:12.262 INFO:tasks.workunit.client.0.smithi162.stdout:1/555: readlink d0/d7/d3c/d54/l69 0 2024-04-27T04:10:12.263 INFO:tasks.workunit.client.0.smithi162.stdout:7/574: sync 2024-04-27T04:10:12.263 INFO:tasks.workunit.client.0.smithi162.stdout:6/579: sync 2024-04-27T04:10:12.264 INFO:tasks.workunit.client.0.smithi162.stdout:7/575: fsync d3/d5/d9/fac 0 2024-04-27T04:10:12.267 INFO:tasks.workunit.client.1.smithi181.stdout:5/893: creat d4/de/d2c/d46/d4d/d8e/d67/d71/d10f/f13e x:0 0 0 2024-04-27T04:10:12.273 INFO:tasks.workunit.client.0.smithi162.stdout:9/583: creat d3/d13/d2d/d4f/faf x:0 0 0 2024-04-27T04:10:12.274 INFO:tasks.workunit.client.1.smithi181.stdout:9/699: creat d0/d44/d4b/d43/d88/fef x:0 0 0 2024-04-27T04:10:12.279 INFO:tasks.workunit.client.1.smithi181.stdout:3/713: mknod d0/d6c/d66/dc2/de1/ce7 0 2024-04-27T04:10:12.285 INFO:tasks.workunit.client.1.smithi181.stdout:3/714: dwrite d0/d6c/d52/f65 [0,4194304] 0 2024-04-27T04:10:12.294 INFO:tasks.workunit.client.1.smithi181.stdout:0/855: dwrite dc/d1f/d3b/f44 [4194304,4194304] 0 2024-04-27T04:10:12.295 INFO:tasks.workunit.client.0.smithi162.stdout:2/516: link d0/d4/d17/d4f/f85 d0/d4/d17/d88/f9d 0 2024-04-27T04:10:12.296 INFO:tasks.workunit.client.0.smithi162.stdout:2/517: write d0/d4/d2a/d41/f4a [5022872,88692] 0 2024-04-27T04:10:12.299 INFO:tasks.workunit.client.0.smithi162.stdout:8/595: truncate d0/d10/d25/d7d/f8b 2532836 0 2024-04-27T04:10:12.300 INFO:tasks.workunit.client.1.smithi181.stdout:0/856: stat dc/d1f/d3b/d4a/d95/c5a 0 2024-04-27T04:10:12.302 INFO:tasks.workunit.client.0.smithi162.stdout:5/618: creat d8/d9/d5b/d96/fc2 x:0 0 0 2024-04-27T04:10:12.303 INFO:tasks.workunit.client.0.smithi162.stdout:4/548: write d2/f11 [837226,122181] 0 2024-04-27T04:10:12.303 INFO:tasks.workunit.client.1.smithi181.stdout:3/715: mkdir d0/d6c/d52/d44/de8 0 2024-04-27T04:10:12.304 INFO:tasks.workunit.client.0.smithi162.stdout:5/619: fsync d8/d1a/f81 0 2024-04-27T04:10:12.304 INFO:tasks.workunit.client.0.smithi162.stdout:2/518: dwrite d0/d4/d2a/f3e [0,4194304] 0 2024-04-27T04:10:12.304 INFO:tasks.workunit.client.1.smithi181.stdout:5/894: truncate d4/de/d2c/d46/d4d/d8e/f41 6775041 0 2024-04-27T04:10:12.308 INFO:tasks.workunit.client.1.smithi181.stdout:9/700: write d0/f2e [5903685,81717] 0 2024-04-27T04:10:12.310 INFO:tasks.workunit.client.1.smithi181.stdout:9/701: write d0/d44/d4b/d5f/da4/fd4 [848606,107480] 0 2024-04-27T04:10:12.311 INFO:tasks.workunit.client.0.smithi162.stdout:1/556: mkdir d0/d7/d10/d75/db7 0 2024-04-27T04:10:12.311 INFO:tasks.workunit.client.0.smithi162.stdout:7/576: symlink d3/d5/d5b/d83/de/d2b/d81/daa/dbd/dab/lc1 0 2024-04-27T04:10:12.313 INFO:tasks.workunit.client.0.smithi162.stdout:7/577: write d3/d5/d5b/d83/de/d2b/d81/daa/dbd/f84 [1445344,58331] 0 2024-04-27T04:10:12.313 INFO:tasks.workunit.client.1.smithi181.stdout:0/857: unlink dc/d1f/d6b/cdc 0 2024-04-27T04:10:12.318 INFO:tasks.workunit.client.0.smithi162.stdout:0/665: creat d5/dc/fd9 x:0 0 0 2024-04-27T04:10:12.320 INFO:tasks.workunit.client.1.smithi181.stdout:3/716: truncate d0/d6c/d52/d44/d56/f76 125183 0 2024-04-27T04:10:12.320 INFO:tasks.workunit.client.1.smithi181.stdout:5/895: creat d4/de/d12f/f13f x:0 0 0 2024-04-27T04:10:12.322 INFO:tasks.workunit.client.1.smithi181.stdout:5/896: chown d4/da3 15 1 2024-04-27T04:10:12.322 INFO:tasks.workunit.client.1.smithi181.stdout:0/858: dwrite dc/d1f/daa/f103 [0,4194304] 0 2024-04-27T04:10:12.323 INFO:tasks.workunit.client.0.smithi162.stdout:5/620: mknod d8/d1a/d29/d4a/cc3 0 2024-04-27T04:10:12.327 INFO:tasks.workunit.client.0.smithi162.stdout:4/549: rename d2/d25/d1c/d72 to d2/d25/d1c/d39/d90/dad 0 2024-04-27T04:10:12.328 INFO:tasks.workunit.client.0.smithi162.stdout:3/554: link d1/d45/l99 d1/d18/d23/d34/lb5 0 2024-04-27T04:10:12.328 INFO:tasks.workunit.client.0.smithi162.stdout:4/550: dread - d2/d25/d1c/d39/f7f zero size 2024-04-27T04:10:12.329 INFO:tasks.workunit.client.0.smithi162.stdout:6/580: dread d6/db/f61 [0,4194304] 0 2024-04-27T04:10:12.331 INFO:tasks.workunit.client.0.smithi162.stdout:1/557: creat d0/d80/fb8 x:0 0 0 2024-04-27T04:10:12.331 INFO:tasks.workunit.client.0.smithi162.stdout:6/581: truncate d6/d1a/d34/d7b/d7e/d97/f9d 973206 0 2024-04-27T04:10:12.332 INFO:tasks.workunit.client.0.smithi162.stdout:1/558: truncate d0/f37 1301207 0 2024-04-27T04:10:12.332 INFO:tasks.workunit.client.0.smithi162.stdout:6/582: write d6/db/fb0 [697911,87231] 0 2024-04-27T04:10:12.335 INFO:tasks.workunit.client.0.smithi162.stdout:0/666: creat d5/dc/d1e/d56/d57/fda x:0 0 0 2024-04-27T04:10:12.337 INFO:tasks.workunit.client.1.smithi181.stdout:0/859: creat dc/d1f/d6b/d7e/d91/dea/f129 x:0 0 0 2024-04-27T04:10:12.337 INFO:tasks.workunit.client.0.smithi162.stdout:0/667: chown d5/dc/d1e/d41/d79/c80 12086 1 2024-04-27T04:10:12.338 INFO:tasks.workunit.client.1.smithi181.stdout:0/860: write dc/f18 [1908795,61490] 0 2024-04-27T04:10:12.339 INFO:tasks.workunit.client.0.smithi162.stdout:8/596: creat d0/d10/d25/d7d/db8/dbe/fc6 x:0 0 0 2024-04-27T04:10:12.340 INFO:tasks.workunit.client.0.smithi162.stdout:0/668: dread d5/dc/d1e/d46/fc3 [0,4194304] 0 2024-04-27T04:10:12.341 INFO:tasks.workunit.client.1.smithi181.stdout:5/897: rename d4/d10/d29/d122/d100/d8f/cd3 to d4/de/d2c/d46/d99/dad/c140 0 2024-04-27T04:10:12.342 INFO:tasks.workunit.client.1.smithi181.stdout:5/898: fsync d4/de/d2c/d46/d4d/f8b 0 2024-04-27T04:10:12.346 INFO:tasks.workunit.client.0.smithi162.stdout:2/519: rename d0/d4/f22 to d0/d4/d2a/f9e 0 2024-04-27T04:10:12.346 INFO:tasks.workunit.client.1.smithi181.stdout:4/687: sync 2024-04-27T04:10:12.347 INFO:tasks.workunit.client.1.smithi181.stdout:5/899: dread d4/fc9 [0,4194304] 0 2024-04-27T04:10:12.352 INFO:tasks.workunit.client.0.smithi162.stdout:3/555: creat d1/d21/d47/d57/fb6 x:0 0 0 2024-04-27T04:10:12.352 INFO:tasks.workunit.client.0.smithi162.stdout:4/551: truncate d2/d25/f2d 518745 0 2024-04-27T04:10:12.360 INFO:tasks.workunit.client.1.smithi181.stdout:9/702: getdents d0/d44/d4b/d43/d88 0 2024-04-27T04:10:12.366 INFO:tasks.workunit.client.0.smithi162.stdout:7/578: rename d3/d5/d9/f13 to d3/d5/d5b/d83/de/d2b/da7/fc2 0 2024-04-27T04:10:12.373 INFO:tasks.workunit.client.1.smithi181.stdout:0/861: symlink dc/d1f/d3b/df7/d127/l12a 0 2024-04-27T04:10:12.375 INFO:tasks.workunit.client.1.smithi181.stdout:3/717: getdents d0/d6c/d52/d44/d56 0 2024-04-27T04:10:12.375 INFO:tasks.workunit.client.1.smithi181.stdout:3/718: readlink d0/d6c/d52/l61 0 2024-04-27T04:10:12.376 INFO:tasks.workunit.client.1.smithi181.stdout:3/719: stat d0 0 2024-04-27T04:10:12.376 INFO:tasks.workunit.client.1.smithi181.stdout:3/720: chown d0/f23 33226279 1 2024-04-27T04:10:12.390 INFO:tasks.workunit.client.1.smithi181.stdout:0/862: chown dc/d1f/d3b/d4a/d95/d26/fa0 125 1 2024-04-27T04:10:12.391 INFO:tasks.workunit.client.1.smithi181.stdout:3/721: rmdir d0/de/d7d 39 2024-04-27T04:10:12.393 INFO:tasks.workunit.client.1.smithi181.stdout:3/722: readlink d0/d6c/d52/d44/l54 0 2024-04-27T04:10:12.394 INFO:tasks.workunit.client.1.smithi181.stdout:4/688: dwrite d0/d15/d25/d2b/d6d/d93/fda [0,4194304] 0 2024-04-27T04:10:12.394 INFO:tasks.workunit.client.0.smithi162.stdout:4/552: mknod d2/d54/cae 0 2024-04-27T04:10:12.395 INFO:tasks.workunit.client.0.smithi162.stdout:6/583: mknod d6/d1a/cb7 0 2024-04-27T04:10:12.397 INFO:tasks.workunit.client.0.smithi162.stdout:1/559: symlink d0/d11/d50/d53/d93/d96/daa/lb9 0 2024-04-27T04:10:12.398 INFO:tasks.workunit.client.0.smithi162.stdout:1/560: fdatasync d0/d7/d3c/d6e/f7b 0 2024-04-27T04:10:12.399 INFO:tasks.workunit.client.1.smithi181.stdout:5/900: getdents d4/de/d2c/d46/d4d/d8e/d67 0 2024-04-27T04:10:12.402 INFO:tasks.workunit.client.1.smithi181.stdout:4/689: mkdir d0/d15/de1 0 2024-04-27T04:10:12.405 INFO:tasks.workunit.client.1.smithi181.stdout:0/863: mkdir dc/d1f/d6b/d7e/d12b 0 2024-04-27T04:10:12.406 INFO:tasks.workunit.client.1.smithi181.stdout:9/703: getdents d0/d44/d4b/d43/d88 0 2024-04-27T04:10:12.410 INFO:tasks.workunit.client.1.smithi181.stdout:0/864: dread dc/d1f/d6b/db9/dba/deb/db3/fb8 [0,4194304] 0 2024-04-27T04:10:12.411 INFO:tasks.workunit.client.1.smithi181.stdout:5/901: rename d4/d10/d29/c43 to d4/d10/dec/d109/c141 0 2024-04-27T04:10:12.414 INFO:tasks.workunit.client.1.smithi181.stdout:4/690: mknod d0/d15/de/d19/d59/ce2 0 2024-04-27T04:10:12.417 INFO:tasks.workunit.client.0.smithi162.stdout:8/597: truncate d0/f4 108733 0 2024-04-27T04:10:12.418 INFO:tasks.workunit.client.0.smithi162.stdout:8/598: truncate d0/d10/d2b/d2d/fc4 911049 0 2024-04-27T04:10:12.419 INFO:tasks.workunit.client.0.smithi162.stdout:0/669: write d5/dc/d1e/f21 [3108724,100661] 0 2024-04-27T04:10:12.420 INFO:tasks.workunit.client.0.smithi162.stdout:0/670: stat d5/dc/d1e/d41/f4f 0 2024-04-27T04:10:12.420 INFO:tasks.workunit.client.0.smithi162.stdout:5/621: rename d8/d9/d5a/l68 to d8/d1a/d7d/lc4 0 2024-04-27T04:10:12.422 INFO:tasks.workunit.client.0.smithi162.stdout:2/520: write d0/d24/f87 [1815077,90814] 0 2024-04-27T04:10:12.426 INFO:tasks.workunit.client.1.smithi181.stdout:5/902: rename d4/de/d2c/d46/d99/dad/d13a/db3/l104 to d4/de/d2c/d46/d4d/d8e/d67/l142 0 2024-04-27T04:10:12.427 INFO:tasks.workunit.client.1.smithi181.stdout:5/903: dread - d4/de/d2f/f132 zero size 2024-04-27T04:10:12.428 INFO:tasks.workunit.client.1.smithi181.stdout:4/691: truncate d0/d15/d76/da4/faf 2444956 0 2024-04-27T04:10:12.430 INFO:tasks.workunit.client.0.smithi162.stdout:3/556: mknod d1/d18/d23/cb7 0 2024-04-27T04:10:12.431 INFO:tasks.workunit.client.0.smithi162.stdout:3/557: stat d1/d21/c5a 0 2024-04-27T04:10:12.431 INFO:tasks.workunit.client.0.smithi162.stdout:3/558: stat d1/l3d 0 2024-04-27T04:10:12.432 INFO:tasks.workunit.client.1.smithi181.stdout:9/704: dwrite d0/f13 [0,4194304] 0 2024-04-27T04:10:12.434 INFO:tasks.workunit.client.1.smithi181.stdout:9/705: chown d0/d4d/da0/lb0 82841 1 2024-04-27T04:10:12.434 INFO:tasks.workunit.client.0.smithi162.stdout:4/553: write d2/da/d1a/f2e [1790470,116903] 0 2024-04-27T04:10:12.438 INFO:tasks.workunit.client.1.smithi181.stdout:5/904: mkdir d4/da3/d143 0 2024-04-27T04:10:12.439 INFO:tasks.workunit.client.0.smithi162.stdout:7/579: dwrite d3/d5/f7a [0,4194304] 0 2024-04-27T04:10:12.440 INFO:tasks.workunit.client.1.smithi181.stdout:0/865: dwrite dc/d1f/d6b/db9/fcf [0,4194304] 0 2024-04-27T04:10:12.443 INFO:tasks.workunit.client.1.smithi181.stdout:4/692: symlink d0/d15/d25/d2b/d6d/d93/le3 0 2024-04-27T04:10:12.446 INFO:tasks.workunit.client.0.smithi162.stdout:1/561: mkdir d0/d80/dba 0 2024-04-27T04:10:12.449 INFO:tasks.workunit.client.1.smithi181.stdout:9/706: unlink d0/d44/d4f/fe5 0 2024-04-27T04:10:12.454 INFO:tasks.workunit.client.0.smithi162.stdout:9/584: sync 2024-04-27T04:10:12.454 INFO:tasks.workunit.client.0.smithi162.stdout:9/585: dread - d3/d13/d2d/d3b/d72/f94 zero size 2024-04-27T04:10:12.456 INFO:tasks.workunit.client.1.smithi181.stdout:5/905: rename d4/de/d2c/d46/d4d/d8e/d54/fcf to d4/de/d2f/d7a/d90/f144 0 2024-04-27T04:10:12.464 INFO:tasks.workunit.client.0.smithi162.stdout:0/671: rename d5/d60/d8b/f99 to d5/dc/d1e/d46/d48/d9e/fdb 0 2024-04-27T04:10:12.467 INFO:tasks.workunit.client.0.smithi162.stdout:2/521: unlink d0/d4/d2a/d41/f4a 0 2024-04-27T04:10:12.468 INFO:tasks.workunit.client.1.smithi181.stdout:9/707: creat d0/d44/d4b/d3d/dbd/dd1/dde/ff0 x:0 0 0 2024-04-27T04:10:12.473 INFO:tasks.workunit.client.1.smithi181.stdout:4/693: dread d0/d15/d25/d2a/d36/f4a [0,4194304] 0 2024-04-27T04:10:12.476 INFO:tasks.workunit.client.1.smithi181.stdout:0/866: write dc/d1f/d3b/dd0/f100 [574325,49408] 0 2024-04-27T04:10:12.480 INFO:tasks.workunit.client.1.smithi181.stdout:9/708: mknod d0/d44/d4b/d3d/dbd/dd1/cf1 0 2024-04-27T04:10:12.481 INFO:tasks.workunit.client.0.smithi162.stdout:8/599: dwrite d0/f7 [4194304,4194304] 0 2024-04-27T04:10:12.481 INFO:tasks.workunit.client.1.smithi181.stdout:9/709: chown d0/f2b 16758158 1 2024-04-27T04:10:12.482 INFO:tasks.workunit.client.0.smithi162.stdout:8/600: write d0/de/d6e/d71/f95 [1501043,1294] 0 2024-04-27T04:10:12.482 INFO:tasks.workunit.client.0.smithi162.stdout:4/554: dread d2/d25/f5c [0,4194304] 0 2024-04-27T04:10:12.483 INFO:tasks.workunit.client.0.smithi162.stdout:1/562: rmdir d0/d7/d3c/d6e 39 2024-04-27T04:10:12.485 INFO:tasks.workunit.client.0.smithi162.stdout:1/563: fdatasync d0/d7/d3c/d54/fa4 0 2024-04-27T04:10:12.487 INFO:tasks.workunit.client.1.smithi181.stdout:0/867: creat dc/d1f/d3b/dfa/f12c x:0 0 0 2024-04-27T04:10:12.488 INFO:tasks.workunit.client.0.smithi162.stdout:5/622: truncate d8/d9/d5b/f7f 4712725 0 2024-04-27T04:10:12.489 INFO:tasks.workunit.client.1.smithi181.stdout:9/710: symlink d0/d44/d4b/d50/lf2 0 2024-04-27T04:10:12.491 INFO:tasks.workunit.client.0.smithi162.stdout:7/580: dwrite d3/d39/f52 [0,4194304] 0 2024-04-27T04:10:12.493 INFO:tasks.workunit.client.1.smithi181.stdout:9/711: rmdir d0/d4d 39 2024-04-27T04:10:12.495 INFO:tasks.workunit.client.1.smithi181.stdout:5/906: getdents d4/de/d2f/d7a 0 2024-04-27T04:10:12.496 INFO:tasks.workunit.client.1.smithi181.stdout:5/907: write d4/f119 [3718371,2369] 0 2024-04-27T04:10:12.497 INFO:tasks.workunit.client.1.smithi181.stdout:5/908: fdatasync d4/de/f113 0 2024-04-27T04:10:12.501 INFO:tasks.workunit.client.1.smithi181.stdout:0/868: creat dc/d1f/d6b/d7e/d12b/f12d x:0 0 0 2024-04-27T04:10:12.504 INFO:tasks.workunit.client.1.smithi181.stdout:4/694: write d0/d15/d76/db1/fb6 [300590,65165] 0 2024-04-27T04:10:12.505 INFO:tasks.workunit.client.1.smithi181.stdout:4/695: chown d0/d15/d25/d2a/d36/f89 322591 1 2024-04-27T04:10:12.509 INFO:tasks.workunit.client.1.smithi181.stdout:0/869: mkdir dc/d93/d12e 0 2024-04-27T04:10:12.509 INFO:tasks.workunit.client.0.smithi162.stdout:8/601: dread d0/de/d6e/d71/f9d [0,4194304] 0 2024-04-27T04:10:12.511 INFO:tasks.workunit.client.0.smithi162.stdout:9/586: rename d3/d11/d45/d4e/d58/f8e to d3/d13/d2d/d44/fb0 0 2024-04-27T04:10:12.516 INFO:tasks.workunit.client.1.smithi181.stdout:1/735: sync 2024-04-27T04:10:12.516 INFO:tasks.workunit.client.1.smithi181.stdout:7/731: sync 2024-04-27T04:10:12.517 INFO:tasks.workunit.client.0.smithi162.stdout:5/623: dread d8/d9/f48 [0,4194304] 0 2024-04-27T04:10:12.518 INFO:tasks.workunit.client.1.smithi181.stdout:9/712: rename d0/d4d to d0/d44/d4b/d3d/dbd/df3 0 2024-04-27T04:10:12.521 INFO:tasks.workunit.client.0.smithi162.stdout:0/672: symlink d5/d26/da2/ldc 0 2024-04-27T04:10:12.523 INFO:tasks.workunit.client.0.smithi162.stdout:0/673: truncate d5/dc/fae 966376 0 2024-04-27T04:10:12.524 INFO:tasks.workunit.client.0.smithi162.stdout:7/581: creat d3/d4f/fc3 x:0 0 0 2024-04-27T04:10:12.525 INFO:tasks.workunit.client.1.smithi181.stdout:1/736: creat d1/dad/db5/d6/da3/d1e/d48/da6/da4/ff5 x:0 0 0 2024-04-27T04:10:12.525 INFO:tasks.workunit.client.0.smithi162.stdout:0/674: write d5/d26/d4a/f4c [580804,128738] 0 2024-04-27T04:10:12.526 INFO:tasks.workunit.client.0.smithi162.stdout:3/559: creat d1/d21/fb8 x:0 0 0 2024-04-27T04:10:12.526 INFO:tasks.workunit.client.0.smithi162.stdout:4/555: dwrite d2/da/f12 [0,4194304] 0 2024-04-27T04:10:12.527 INFO:tasks.workunit.client.0.smithi162.stdout:3/560: dread - d1/d21/fb8 zero size 2024-04-27T04:10:12.527 INFO:tasks.workunit.client.1.smithi181.stdout:4/696: rename d0/d15/d25/d40/l56 to d0/d15/d25/d2a/da0/le4 0 2024-04-27T04:10:12.528 INFO:tasks.workunit.client.1.smithi181.stdout:4/697: write d0/d15/d25/fcd [924568,10100] 0 2024-04-27T04:10:12.529 INFO:tasks.workunit.client.0.smithi162.stdout:6/584: getdents d6/db/d32 0 2024-04-27T04:10:12.532 INFO:tasks.workunit.client.1.smithi181.stdout:5/909: rmdir d4/de/d2c/d133 0 2024-04-27T04:10:12.534 INFO:tasks.workunit.client.1.smithi181.stdout:0/870: mkdir dc/d1f/d3b/d4a/d95/d26/d41/d12f 0 2024-04-27T04:10:12.536 INFO:tasks.workunit.client.0.smithi162.stdout:1/564: symlink d0/d80/dba/lbb 0 2024-04-27T04:10:12.537 INFO:tasks.workunit.client.0.smithi162.stdout:7/582: dwrite d3/d5/d5b/d83/de/d2b/d81/d96/fb4 [0,4194304] 0 2024-04-27T04:10:12.539 INFO:tasks.workunit.client.0.smithi162.stdout:8/602: readlink d0/de/l4f 0 2024-04-27T04:10:12.539 INFO:tasks.workunit.client.0.smithi162.stdout:3/561: dwrite d1/d21/f60 [0,4194304] 0 2024-04-27T04:10:12.540 INFO:tasks.workunit.client.1.smithi181.stdout:0/871: creat dc/d1f/d3b/df7/d127/f130 x:0 0 0 2024-04-27T04:10:12.540 INFO:tasks.workunit.client.0.smithi162.stdout:6/585: dread d6/db/f61 [0,4194304] 0 2024-04-27T04:10:12.541 INFO:tasks.workunit.client.0.smithi162.stdout:9/587: creat d3/d11/d45/d4e/fb1 x:0 0 0 2024-04-27T04:10:12.542 INFO:tasks.workunit.client.1.smithi181.stdout:7/732: link d1/d22/d15/d1f/c7c d1/df/d1a/d6a/cde 0 2024-04-27T04:10:12.542 INFO:tasks.workunit.client.0.smithi162.stdout:6/586: dread - d6/db/d32/f38 zero size 2024-04-27T04:10:12.545 INFO:tasks.workunit.client.0.smithi162.stdout:7/583: dread d3/d39/d8d/f91 [0,4194304] 0 2024-04-27T04:10:12.547 INFO:tasks.workunit.client.0.smithi162.stdout:7/584: truncate d3/d5/d9/fb8 1047957 0 2024-04-27T04:10:12.547 INFO:tasks.workunit.client.0.smithi162.stdout:7/585: chown d3/d4f/c66 72510 1 2024-04-27T04:10:12.549 INFO:tasks.workunit.client.0.smithi162.stdout:2/522: link d0/d4/d2a/d3b/d49/d5a/d63/d84/l37 d0/d4/d2a/d41/d77/l9f 0 2024-04-27T04:10:12.551 INFO:tasks.workunit.client.1.smithi181.stdout:0/872: chown dc/d1f/d6b/db9/dba/deb/f79 28534241 1 2024-04-27T04:10:12.556 INFO:tasks.workunit.client.1.smithi181.stdout:5/910: link d4/d10/d29/d122/d100/lee d4/de/d2c/d46/d99/dad/d13a/d80/l145 0 2024-04-27T04:10:12.562 INFO:tasks.workunit.client.1.smithi181.stdout:7/733: dread d1/d2/d1c/d37/d91/fb0 [0,4194304] 0 2024-04-27T04:10:12.566 INFO:tasks.workunit.client.0.smithi162.stdout:3/562: creat d1/d18/d5d/fb9 x:0 0 0 2024-04-27T04:10:12.568 INFO:tasks.workunit.client.0.smithi162.stdout:8/603: mkdir d0/d10/d2b/d2d/d2f/d6b/dc7 0 2024-04-27T04:10:12.568 INFO:tasks.workunit.client.0.smithi162.stdout:8/604: readlink d0/de/l4f 0 2024-04-27T04:10:12.569 INFO:tasks.workunit.client.0.smithi162.stdout:6/587: creat d6/d1a/d34/d50/fb8 x:0 0 0 2024-04-27T04:10:12.570 INFO:tasks.workunit.client.1.smithi181.stdout:7/734: creat d1/df/fdf x:0 0 0 2024-04-27T04:10:12.570 INFO:tasks.workunit.client.1.smithi181.stdout:0/873: write dc/fd [339047,77986] 0 2024-04-27T04:10:12.572 INFO:tasks.workunit.client.1.smithi181.stdout:7/735: write d1/d2/d1c/d2d/f41 [393277,80096] 0 2024-04-27T04:10:12.572 INFO:tasks.workunit.client.1.smithi181.stdout:0/874: chown dc/d1f/d3b/d4a/d95/d26/d41/c96 0 1 2024-04-27T04:10:12.573 INFO:tasks.workunit.client.0.smithi162.stdout:5/624: fdatasync d8/d1a/d23/f28 0 2024-04-27T04:10:12.575 INFO:tasks.workunit.client.1.smithi181.stdout:5/911: creat d4/de/f146 x:0 0 0 2024-04-27T04:10:12.576 INFO:tasks.workunit.client.0.smithi162.stdout:7/586: mknod d3/d5/d3c/d4d/cc4 0 2024-04-27T04:10:12.576 INFO:tasks.workunit.client.1.smithi181.stdout:7/736: creat d1/d2/d1c/d2d/d61/fe0 x:0 0 0 2024-04-27T04:10:12.577 INFO:tasks.workunit.client.1.smithi181.stdout:7/737: chown d1/d2/d1c/d2d/f41 128885649 1 2024-04-27T04:10:12.577 INFO:tasks.workunit.client.0.smithi162.stdout:2/523: fdatasync d0/d24/f4b 0 2024-04-27T04:10:12.578 INFO:tasks.workunit.client.1.smithi181.stdout:0/875: fdatasync dc/d1f/d3b/db2/fa4 0 2024-04-27T04:10:12.579 INFO:tasks.workunit.client.0.smithi162.stdout:8/605: dread d0/fd [0,4194304] 0 2024-04-27T04:10:12.579 INFO:tasks.workunit.client.1.smithi181.stdout:0/876: chown dc/d1f/d3b/d4a/d95/f10b 5 1 2024-04-27T04:10:12.581 INFO:tasks.workunit.client.1.smithi181.stdout:0/877: write dc/d1f/d3b/d4a/fd1 [864425,128699] 0 2024-04-27T04:10:12.586 INFO:tasks.workunit.client.1.smithi181.stdout:0/878: fdatasync dc/d1f/d6b/db9/dba/deb/f79 0 2024-04-27T04:10:12.587 INFO:tasks.workunit.client.0.smithi162.stdout:3/563: mknod d1/d21/d47/cba 0 2024-04-27T04:10:12.588 INFO:tasks.workunit.client.1.smithi181.stdout:5/912: write d4/d10/d29/d92/fbf [508165,126453] 0 2024-04-27T04:10:12.588 INFO:tasks.workunit.client.0.smithi162.stdout:3/564: chown d1/d18/d23/d58/fb3 0 1 2024-04-27T04:10:12.590 INFO:tasks.workunit.client.0.smithi162.stdout:3/565: write d1/d18/d23/d58/d63/f86 [1599428,100520] 0 2024-04-27T04:10:12.590 INFO:tasks.workunit.client.0.smithi162.stdout:6/588: creat d6/d1a/d34/d7b/fb9 x:0 0 0 2024-04-27T04:10:12.591 INFO:tasks.workunit.client.0.smithi162.stdout:6/589: fdatasync d6/d1a/d34/d43/fab 0 2024-04-27T04:10:12.593 INFO:tasks.workunit.client.0.smithi162.stdout:9/588: rename d3/d13/d15/d50/c65 to d3/d13/d2d/d3b/d69/cb2 0 2024-04-27T04:10:12.594 INFO:tasks.workunit.client.0.smithi162.stdout:9/589: chown d3/d11/f4d 236 1 2024-04-27T04:10:12.595 INFO:tasks.workunit.client.0.smithi162.stdout:7/587: unlink d3/d5/f7c 0 2024-04-27T04:10:12.598 INFO:tasks.workunit.client.1.smithi181.stdout:0/879: rename dc/d1f/d6b/db9 to dc/d1f/d3b/d4a/d95/d26/d131 0 2024-04-27T04:10:12.599 INFO:tasks.workunit.client.1.smithi181.stdout:0/880: stat dc/d1f/d6b/d7e/d91/dea/c126 0 2024-04-27T04:10:12.600 INFO:tasks.workunit.client.0.smithi162.stdout:8/606: symlink d0/d10/d2b/d2d/d2f/lc8 0 2024-04-27T04:10:12.602 INFO:tasks.workunit.client.1.smithi181.stdout:5/913: getdents d4/d10/d29/d122/d100/d8f 0 2024-04-27T04:10:12.602 INFO:tasks.workunit.client.1.smithi181.stdout:5/914: fdatasync d4/d10/fdb 0 2024-04-27T04:10:12.604 INFO:tasks.workunit.client.1.smithi181.stdout:5/915: chown d4/de/d2c/d46/d4d/d8e/d54/l118 0 1 2024-04-27T04:10:12.609 INFO:tasks.workunit.client.0.smithi162.stdout:3/566: creat d1/d45/fbb x:0 0 0 2024-04-27T04:10:12.610 INFO:tasks.workunit.client.0.smithi162.stdout:3/567: chown d1/d18/d23/d2f/l40 10317000 1 2024-04-27T04:10:12.611 INFO:tasks.workunit.client.1.smithi181.stdout:5/916: mknod d4/d10/d29/dd2/c147 0 2024-04-27T04:10:12.614 INFO:tasks.workunit.client.1.smithi181.stdout:0/881: dread dc/d1f/d3b/f44 [0,4194304] 0 2024-04-27T04:10:12.615 INFO:tasks.workunit.client.0.smithi162.stdout:5/625: rename d8/d9/db3/fb9 to d8/d1a/d7d/fc5 0 2024-04-27T04:10:12.622 INFO:tasks.workunit.client.0.smithi162.stdout:2/524: mkdir d0/d4/d17/d2b/d64/d89/da0 0 2024-04-27T04:10:12.623 INFO:tasks.workunit.client.1.smithi181.stdout:8/784: sync 2024-04-27T04:10:12.623 INFO:tasks.workunit.client.1.smithi181.stdout:2/795: sync 2024-04-27T04:10:12.624 INFO:tasks.workunit.client.1.smithi181.stdout:6/628: sync 2024-04-27T04:10:12.624 INFO:tasks.workunit.client.0.smithi162.stdout:0/675: getdents d5/dc/d1e 0 2024-04-27T04:10:12.624 INFO:tasks.workunit.client.1.smithi181.stdout:5/917: truncate d4/d10/d29/d92/fd6 3780529 0 2024-04-27T04:10:12.625 INFO:tasks.workunit.client.0.smithi162.stdout:4/556: link d2/d25/d1c/f3d d2/da/faf 0 2024-04-27T04:10:12.626 INFO:tasks.workunit.client.0.smithi162.stdout:8/607: chown d0/d10/c38 425410800 1 2024-04-27T04:10:12.627 INFO:tasks.workunit.client.1.smithi181.stdout:8/785: fsync d1/d6/d2f/d72/fbd 0 2024-04-27T04:10:12.628 INFO:tasks.workunit.client.1.smithi181.stdout:6/629: fsync d7/dc/dc3/d72/daa/fce 0 2024-04-27T04:10:12.630 INFO:tasks.workunit.client.1.smithi181.stdout:0/882: dwrite dc/d1f/d3b/d4a/d95/d26/d41/fa8 [0,4194304] 0 2024-04-27T04:10:12.630 INFO:tasks.workunit.client.0.smithi162.stdout:2/525: dwrite d0/d4/d17/d2b/d64/f90 [0,4194304] 0 2024-04-27T04:10:12.634 INFO:tasks.workunit.client.0.smithi162.stdout:0/676: dwrite d5/dc/d1e/fbf [0,4194304] 0 2024-04-27T04:10:12.639 INFO:tasks.workunit.client.0.smithi162.stdout:9/590: creat d3/d13/d2d/d30/d7f/fb3 x:0 0 0 2024-04-27T04:10:12.643 INFO:tasks.workunit.client.0.smithi162.stdout:7/588: symlink d3/d5/lc5 0 2024-04-27T04:10:12.644 INFO:tasks.workunit.client.1.smithi181.stdout:5/918: rename d4/de/d2c/d46/d4d/l73 to d4/de/d2c/d46/d99/dad/d13a/l148 0 2024-04-27T04:10:12.645 INFO:tasks.workunit.client.0.smithi162.stdout:9/591: chown d3/d13/d2d/d30/f38 1772698 1 2024-04-27T04:10:12.645 INFO:tasks.workunit.client.0.smithi162.stdout:2/526: dread d0/d4/d2a/d41/d77/f7c [0,4194304] 0 2024-04-27T04:10:12.646 INFO:tasks.workunit.client.1.smithi181.stdout:8/786: symlink d1/d6/d2f/d30/d9d/dea/db3/ddf/l112 0 2024-04-27T04:10:12.648 INFO:tasks.workunit.client.0.smithi162.stdout:1/565: link d0/d11/d44/d66/f78 d0/d11/d44/d66/fbc 0 2024-04-27T04:10:12.649 INFO:tasks.workunit.client.1.smithi181.stdout:6/630: creat d7/dc/dc3/d72/daa/dd3/fd8 x:0 0 0 2024-04-27T04:10:12.652 INFO:tasks.workunit.client.0.smithi162.stdout:1/566: dread d0/d7/f19 [4194304,4194304] 0 2024-04-27T04:10:12.653 INFO:tasks.workunit.client.1.smithi181.stdout:0/883: truncate dc/d1f/d3b/d4a/d95/d26/d131/dba/deb/f79 1425686 0 2024-04-27T04:10:12.653 INFO:tasks.workunit.client.1.smithi181.stdout:2/796: dread d4/d9/d16/f26 [0,4194304] 0 2024-04-27T04:10:12.656 INFO:tasks.workunit.client.1.smithi181.stdout:5/919: symlink d4/de/d2c/d46/d99/dad/d13a/d80/d85/l149 0 2024-04-27T04:10:12.656 INFO:tasks.workunit.client.1.smithi181.stdout:8/787: mknod d1/d6/da6/c113 0 2024-04-27T04:10:12.657 INFO:tasks.workunit.client.1.smithi181.stdout:5/920: fdatasync d4/de/d2c/d46/d4d/f8b 0 2024-04-27T04:10:12.657 INFO:tasks.workunit.client.1.smithi181.stdout:6/631: write d7/dc/d1c/d34/d3f/d7e/d8a/fad [155914,4728] 0 2024-04-27T04:10:12.659 INFO:tasks.workunit.client.0.smithi162.stdout:9/592: readlink d3/l7 0 2024-04-27T04:10:12.660 INFO:tasks.workunit.client.1.smithi181.stdout:5/921: write d4/d10/d29/d92/dbb/f137 [241496,123411] 0 2024-04-27T04:10:12.661 INFO:tasks.workunit.client.1.smithi181.stdout:2/797: symlink d4/d9/d16/d82/d7d/db7/lff 0 2024-04-27T04:10:12.661 INFO:tasks.workunit.client.0.smithi162.stdout:9/593: truncate d3/d11/d45/d4e/fb1 17190 0 2024-04-27T04:10:12.663 INFO:tasks.workunit.client.1.smithi181.stdout:8/788: mknod d1/d6/d59/c114 0 2024-04-27T04:10:12.663 INFO:tasks.workunit.client.1.smithi181.stdout:2/798: write d4/d9/d4f/d79/d98/f99 [3008345,86121] 0 2024-04-27T04:10:12.665 INFO:tasks.workunit.client.1.smithi181.stdout:0/884: stat dc/d1f/d3b/d4a/d95/d26/d41/d6f/d7d/cc6 0 2024-04-27T04:10:12.666 INFO:tasks.workunit.client.0.smithi162.stdout:8/608: dread d0/de/d20/d6c/f6d [0,4194304] 0 2024-04-27T04:10:12.667 INFO:tasks.workunit.client.0.smithi162.stdout:8/609: stat d0/de/d6e/d71/fb3 0 2024-04-27T04:10:12.669 INFO:tasks.workunit.client.1.smithi181.stdout:6/632: rename d7/dc/d8b to d7/dc/dc3/dd1/dd9 0 2024-04-27T04:10:12.670 INFO:tasks.workunit.client.1.smithi181.stdout:5/922: mkdir d4/de/d2c/d46/d99/dad/d13a/d14a 0 2024-04-27T04:10:12.674 INFO:tasks.workunit.client.0.smithi162.stdout:8/610: dread d0/d10/d2b/d2e/f40 [0,4194304] 0 2024-04-27T04:10:12.675 INFO:tasks.workunit.client.0.smithi162.stdout:8/611: fdatasync d0/d10/d44/d70/f7f 0 2024-04-27T04:10:12.675 INFO:tasks.workunit.client.1.smithi181.stdout:2/799: dwrite d4/d9/d16/d82/d7d/db7/fea [0,4194304] 0 2024-04-27T04:10:12.676 INFO:tasks.workunit.client.1.smithi181.stdout:0/885: dwrite dc/d93/fe0 [0,4194304] 0 2024-04-27T04:10:12.679 INFO:tasks.workunit.client.0.smithi162.stdout:9/594: truncate d3/d13/d2d/d30/f38 1230737 0 2024-04-27T04:10:12.681 INFO:tasks.workunit.client.0.smithi162.stdout:9/595: read d3/d13/d2d/d44/fb0 [1101522,4889] 0 2024-04-27T04:10:12.683 INFO:tasks.workunit.client.1.smithi181.stdout:0/886: dread dc/d93/fe0 [0,4194304] 0 2024-04-27T04:10:12.685 INFO:tasks.workunit.client.0.smithi162.stdout:3/568: rename d1/d18/d5d/d5f/d66/l6b to d1/d21/lbc 0 2024-04-27T04:10:12.692 INFO:tasks.workunit.client.0.smithi162.stdout:5/626: getdents d8/d9/d5b/d96 0 2024-04-27T04:10:12.693 INFO:tasks.workunit.client.1.smithi181.stdout:2/800: mkdir d4/d9/d4f/d5f/d100 0 2024-04-27T04:10:12.697 INFO:tasks.workunit.client.1.smithi181.stdout:8/789: link d1/d6/d2f/d30/d66/dc0/dcd/ff6 d1/d6/d59/df1/f115 0 2024-04-27T04:10:12.700 INFO:tasks.workunit.client.0.smithi162.stdout:0/677: dwrite d5/dc/d1e/d56/f90 [0,4194304] 0 2024-04-27T04:10:12.700 INFO:tasks.workunit.client.0.smithi162.stdout:8/612: dwrite d0/d10/f6a [0,4194304] 0 2024-04-27T04:10:12.701 INFO:tasks.workunit.client.1.smithi181.stdout:0/887: rmdir dc/d1f/d6b/d7e/d12b 39 2024-04-27T04:10:12.704 INFO:tasks.workunit.client.0.smithi162.stdout:6/590: rename d6/db/d32/l40 to d6/d1a/d34/d7b/d7e/d97/lba 0 2024-04-27T04:10:12.717 INFO:tasks.workunit.client.1.smithi181.stdout:6/633: rename d7/dc/dc3/ca2 to d7/dc/d1c/d34/d3f/cda 0 2024-04-27T04:10:12.722 INFO:tasks.workunit.client.1.smithi181.stdout:2/801: fsync d4/d9/d4f/d5f/fe2 0 2024-04-27T04:10:12.723 INFO:tasks.workunit.client.0.smithi162.stdout:9/596: truncate d3/d13/d2d/f37 273405 0 2024-04-27T04:10:12.726 INFO:tasks.workunit.client.1.smithi181.stdout:2/802: dwrite d4/d81/f8e [0,4194304] 0 2024-04-27T04:10:12.727 INFO:tasks.workunit.client.1.smithi181.stdout:2/803: chown d4/d9/fc0 3 1 2024-04-27T04:10:12.736 INFO:tasks.workunit.client.0.smithi162.stdout:4/557: rename d2/ff to d2/d25/d1c/d39/d70/d91/fb0 0 2024-04-27T04:10:12.737 INFO:tasks.workunit.client.0.smithi162.stdout:4/558: dread - d2/da/fa1 zero size 2024-04-27T04:10:12.738 INFO:tasks.workunit.client.0.smithi162.stdout:4/559: readlink d2/d25/d1c/d39/d3b/lac 0 2024-04-27T04:10:12.738 INFO:tasks.workunit.client.1.smithi181.stdout:5/923: dwrite d4/de/d2c/f75 [0,4194304] 0 2024-04-27T04:10:12.739 INFO:tasks.workunit.client.0.smithi162.stdout:9/597: dread d3/d13/d15/f19 [0,4194304] 0 2024-04-27T04:10:12.741 INFO:tasks.workunit.client.0.smithi162.stdout:1/567: link d0/d7/d10/f26 d0/d11/d50/fbd 0 2024-04-27T04:10:12.744 INFO:tasks.workunit.client.0.smithi162.stdout:9/598: read d3/d11/d45/f55 [1844768,63225] 0 2024-04-27T04:10:12.746 INFO:tasks.workunit.client.1.smithi181.stdout:0/888: write dc/d1f/d3b/d4a/d95/d6d/fa2 [537837,108988] 0 2024-04-27T04:10:12.746 INFO:tasks.workunit.client.0.smithi162.stdout:4/560: dwrite d2/f4 [0,4194304] 0 2024-04-27T04:10:12.748 INFO:tasks.workunit.client.1.smithi181.stdout:0/889: write dc/d1f/d3b/d4a/d95/d26/d41/d6f/d7d/fa7 [619981,2251] 0 2024-04-27T04:10:12.751 INFO:tasks.workunit.client.0.smithi162.stdout:0/678: dwrite d5/d26/d69/f6c [0,4194304] 0 2024-04-27T04:10:12.756 INFO:tasks.workunit.client.1.smithi181.stdout:5/924: fdatasync d4/d10/f57 0 2024-04-27T04:10:12.757 INFO:tasks.workunit.client.1.smithi181.stdout:8/790: link d1/d6/l9a d1/d6/d88/dec/l116 0 2024-04-27T04:10:12.758 INFO:tasks.workunit.client.0.smithi162.stdout:8/613: dwrite d0/d10/d44/f73 [0,4194304] 0 2024-04-27T04:10:12.762 INFO:tasks.workunit.client.0.smithi162.stdout:7/589: rename d3/d39/d8d to d3/d5/d3c/d4a/d55/dc6 0 2024-04-27T04:10:12.763 INFO:tasks.workunit.client.0.smithi162.stdout:8/614: dread - d0/d10/d25/d7d/db8/dbe/fc6 zero size 2024-04-27T04:10:12.764 INFO:tasks.workunit.client.1.smithi181.stdout:6/634: dread d7/dc/d1e/f83 [0,4194304] 0 2024-04-27T04:10:12.765 INFO:tasks.workunit.client.0.smithi162.stdout:7/590: chown d3/d5/d5b/d83/f3f 2 1 2024-04-27T04:10:12.766 INFO:tasks.workunit.client.0.smithi162.stdout:0/679: dwrite d5/d26/da2/fcb [0,4194304] 0 2024-04-27T04:10:12.772 INFO:tasks.workunit.client.1.smithi181.stdout:0/890: dread dc/d1f/d3b/d4a/d95/d26/d41/d6f/d7d/fa7 [0,4194304] 0 2024-04-27T04:10:12.772 INFO:tasks.workunit.client.1.smithi181.stdout:2/804: dwrite d4/f6 [0,4194304] 0 2024-04-27T04:10:12.772 INFO:tasks.workunit.client.0.smithi162.stdout:8/615: dread d0/d10/f6a [0,4194304] 0 2024-04-27T04:10:12.774 INFO:tasks.workunit.client.1.smithi181.stdout:8/791: creat d1/d6/d2f/d47/dba/dbc/f117 x:0 0 0 2024-04-27T04:10:12.776 INFO:tasks.workunit.client.0.smithi162.stdout:5/627: creat d8/d9/fc6 x:0 0 0 2024-04-27T04:10:12.778 INFO:tasks.workunit.client.0.smithi162.stdout:5/628: chown d8/d1a/d29/d4a/d9f/c7e 0 1 2024-04-27T04:10:12.778 INFO:tasks.workunit.client.0.smithi162.stdout:6/591: dread d6/f2d [0,4194304] 0 2024-04-27T04:10:12.779 INFO:tasks.workunit.client.0.smithi162.stdout:6/592: chown d6/d1a/d34/f66 3287 1 2024-04-27T04:10:12.780 INFO:tasks.workunit.client.1.smithi181.stdout:6/635: creat d7/dc/d1e/d2b/fdb x:0 0 0 2024-04-27T04:10:12.782 INFO:tasks.workunit.client.1.smithi181.stdout:3/723: sync 2024-04-27T04:10:12.782 INFO:tasks.workunit.client.1.smithi181.stdout:9/713: sync 2024-04-27T04:10:12.782 INFO:tasks.workunit.client.1.smithi181.stdout:5/925: read d4/de/d2c/d46/d4d/d8e/d67/f50 [386425,23644] 0 2024-04-27T04:10:12.786 INFO:tasks.workunit.client.1.smithi181.stdout:3/724: stat d0/d9f/l84 0 2024-04-27T04:10:12.787 INFO:tasks.workunit.client.0.smithi162.stdout:3/569: link d1/l4 d1/d18/d23/d2f/da0/lbd 0 2024-04-27T04:10:12.788 INFO:tasks.workunit.client.0.smithi162.stdout:3/570: truncate d1/d21/fb8 78466 0 2024-04-27T04:10:12.788 INFO:tasks.workunit.client.0.smithi162.stdout:3/571: chown d1/l4 2748 1 2024-04-27T04:10:12.788 INFO:tasks.workunit.client.1.smithi181.stdout:8/792: unlink d1/d6/d2f/d30/d5c/f102 0 2024-04-27T04:10:12.788 INFO:tasks.workunit.client.1.smithi181.stdout:6/636: rmdir d7/dc/dc3/d62 39 2024-04-27T04:10:12.789 INFO:tasks.workunit.client.0.smithi162.stdout:2/527: rename d0/d4/d2a/d3b/d49/d5a/d63/d84/d93/d5d to d0/d4/d2a/da1 0 2024-04-27T04:10:12.790 INFO:tasks.workunit.client.1.smithi181.stdout:5/926: creat d4/d10/dec/f14b x:0 0 0 2024-04-27T04:10:12.792 INFO:tasks.workunit.client.0.smithi162.stdout:0/680: mkdir d5/dc/d1e/d56/ddd 0 2024-04-27T04:10:12.793 INFO:tasks.workunit.client.0.smithi162.stdout:0/681: write d5/dc/d1e/d46/d48/fb1 [709532,12913] 0 2024-04-27T04:10:12.793 INFO:tasks.workunit.client.0.smithi162.stdout:7/591: write d3/d5/d5b/d83/de/d2b/f72 [4869322,40306] 0 2024-04-27T04:10:12.796 INFO:tasks.workunit.client.1.smithi181.stdout:8/793: mknod d1/d6/d2f/d30/d9d/c118 0 2024-04-27T04:10:12.797 INFO:tasks.workunit.client.1.smithi181.stdout:8/794: chown d1/d6/d2f/d72/c6c 3748 1 2024-04-27T04:10:12.798 INFO:tasks.workunit.client.0.smithi162.stdout:5/629: write d8/d1a/d7d/fc5 [794471,45109] 0 2024-04-27T04:10:12.800 INFO:tasks.workunit.client.1.smithi181.stdout:5/927: creat d4/de/d2c/d46/d99/dad/d13a/d80/d85/f14c x:0 0 0 2024-04-27T04:10:12.801 INFO:tasks.workunit.client.0.smithi162.stdout:4/561: fdatasync d2/da/f49 0 2024-04-27T04:10:12.802 INFO:tasks.workunit.client.1.smithi181.stdout:3/725: truncate d0/f16 173444 0 2024-04-27T04:10:12.802 INFO:tasks.workunit.client.1.smithi181.stdout:1/737: sync 2024-04-27T04:10:12.802 INFO:tasks.workunit.client.1.smithi181.stdout:7/738: sync 2024-04-27T04:10:12.802 INFO:tasks.workunit.client.1.smithi181.stdout:4/698: sync 2024-04-27T04:10:12.803 INFO:tasks.workunit.client.0.smithi162.stdout:4/562: chown d2/d25/d1c/c27 14731506 1 2024-04-27T04:10:12.803 INFO:tasks.workunit.client.1.smithi181.stdout:3/726: write d0/d6c/fb1 [407102,20873] 0 2024-04-27T04:10:12.803 INFO:tasks.workunit.client.1.smithi181.stdout:4/699: stat d0/d15 0 2024-04-27T04:10:12.804 INFO:tasks.workunit.client.0.smithi162.stdout:5/630: dwrite d8/d1a/d29/d4a/d9f/f64 [0,4194304] 0 2024-04-27T04:10:12.805 INFO:tasks.workunit.client.0.smithi162.stdout:5/631: write d8/d9/d5a/fb2 [306888,36885] 0 2024-04-27T04:10:12.811 INFO:tasks.workunit.client.1.smithi181.stdout:2/805: rename d4/d9/d35/f60 to d4/d9/d16/d45/d9a/f101 0 2024-04-27T04:10:12.813 INFO:tasks.workunit.client.1.smithi181.stdout:7/739: dwrite d1/d2/d1c/d2d/d61/f97 [0,4194304] 0 2024-04-27T04:10:12.814 INFO:tasks.workunit.client.0.smithi162.stdout:0/682: mknod d5/dc/d1e/d46/cde 0 2024-04-27T04:10:12.814 INFO:tasks.workunit.client.0.smithi162.stdout:2/528: dread d0/d4/f3a [0,4194304] 0 2024-04-27T04:10:12.817 INFO:tasks.workunit.client.1.smithi181.stdout:0/891: write dc/d1f/d3b/d4a/ff3 [458328,85451] 0 2024-04-27T04:10:12.820 INFO:tasks.workunit.client.1.smithi181.stdout:7/740: read d1/df/d62/fd0 [1396957,49966] 0 2024-04-27T04:10:12.821 INFO:tasks.workunit.client.1.smithi181.stdout:4/700: stat d0/d15/de/d19/l3c 0 2024-04-27T04:10:12.823 INFO:tasks.workunit.client.1.smithi181.stdout:4/701: write d0/d15/d25/d2b/d6d/d82/fd7 [323376,68614] 0 2024-04-27T04:10:12.824 INFO:tasks.workunit.client.1.smithi181.stdout:9/714: link d0/d44/d4b/d43/d8f/lcf d0/d44/d4b/d5f/d8a/lf4 0 2024-04-27T04:10:12.830 INFO:tasks.workunit.client.1.smithi181.stdout:9/715: dread d0/d40/f81 [0,4194304] 0 2024-04-27T04:10:12.831 INFO:tasks.workunit.client.0.smithi162.stdout:1/568: dwrite d0/d7/f1f [0,4194304] 0 2024-04-27T04:10:12.832 INFO:tasks.workunit.client.0.smithi162.stdout:3/572: write d1/d18/f11 [1040557,39858] 0 2024-04-27T04:10:12.834 INFO:tasks.workunit.client.1.smithi181.stdout:8/795: truncate d1/d6/d2f/d30/d9d/dea/f7c 987280 0 2024-04-27T04:10:12.839 INFO:tasks.workunit.client.1.smithi181.stdout:0/892: readlink dc/d1f/d3b/d4a/d95/d26/d68/l82 0 2024-04-27T04:10:12.840 INFO:tasks.workunit.client.1.smithi181.stdout:6/637: dwrite d7/dc/d1e/f30 [0,4194304] 0 2024-04-27T04:10:12.848 INFO:tasks.workunit.client.0.smithi162.stdout:8/616: creat d0/de/fc9 x:0 0 0 2024-04-27T04:10:12.848 INFO:tasks.workunit.client.1.smithi181.stdout:3/727: write d0/d28/d86/f8d [545170,50552] 0 2024-04-27T04:10:12.848 INFO:tasks.workunit.client.0.smithi162.stdout:6/593: truncate d6/d1a/f2b 3565918 0 2024-04-27T04:10:12.849 INFO:tasks.workunit.client.1.smithi181.stdout:3/728: chown d0/d9f/l84 2 1 2024-04-27T04:10:12.850 INFO:tasks.workunit.client.0.smithi162.stdout:2/529: dwrite d0/d4/d17/f3c [0,4194304] 0 2024-04-27T04:10:12.851 INFO:tasks.workunit.client.1.smithi181.stdout:1/738: write d1/dad/db5/d6/da3/d52/d92/f99 [2331431,66300] 0 2024-04-27T04:10:12.856 INFO:tasks.workunit.client.0.smithi162.stdout:6/594: dwrite d6/d1a/fa6 [0,4194304] 0 2024-04-27T04:10:12.856 INFO:tasks.workunit.client.0.smithi162.stdout:1/569: fdatasync d0/d7/d3c/d54/f68 0 2024-04-27T04:10:12.856 INFO:tasks.workunit.client.1.smithi181.stdout:9/716: dread - d0/d44/d4b/d3d/dbd/df3/fb5 zero size 2024-04-27T04:10:12.858 INFO:tasks.workunit.client.1.smithi181.stdout:3/729: dwrite d0/d6c/d66/d88/fcd [0,4194304] 0 2024-04-27T04:10:12.860 INFO:tasks.workunit.client.0.smithi162.stdout:3/573: mkdir d1/d18/d5d/d5f/dbe 0 2024-04-27T04:10:12.861 INFO:tasks.workunit.client.0.smithi162.stdout:9/599: rename d3/d13/d2d/d4f/c6f to d3/d23/cb4 0 2024-04-27T04:10:12.862 INFO:tasks.workunit.client.1.smithi181.stdout:5/928: creat d4/de/d2f/f14d x:0 0 0 2024-04-27T04:10:12.862 INFO:tasks.workunit.client.0.smithi162.stdout:5/632: rename d8/d1a/d23 to d8/d1a/d23/dbc/dc7 22 2024-04-27T04:10:12.863 INFO:tasks.workunit.client.0.smithi162.stdout:9/600: chown d3/d13/d2d/d3b/f47 800700 1 2024-04-27T04:10:12.864 INFO:tasks.workunit.client.0.smithi162.stdout:4/563: fsync d2/da/f78 0 2024-04-27T04:10:12.866 INFO:tasks.workunit.client.1.smithi181.stdout:7/741: unlink d1/d2/d1c/d2d/l94 0 2024-04-27T04:10:12.867 INFO:tasks.workunit.client.0.smithi162.stdout:7/592: creat d3/d5/d9/fc7 x:0 0 0 2024-04-27T04:10:12.868 INFO:tasks.workunit.client.0.smithi162.stdout:7/593: chown d3/d5/d3c/d4a/d55 26 1 2024-04-27T04:10:12.871 INFO:tasks.workunit.client.1.smithi181.stdout:4/702: creat d0/d15/de/d19/d59/d80/fe5 x:0 0 0 2024-04-27T04:10:12.872 INFO:tasks.workunit.client.0.smithi162.stdout:2/530: stat d0/c1c 0 2024-04-27T04:10:12.874 INFO:tasks.workunit.client.0.smithi162.stdout:2/531: readlink d0/d4/d2a/da1/d86/l91 0 2024-04-27T04:10:12.875 INFO:tasks.workunit.client.1.smithi181.stdout:7/742: dwrite d1/dc3/fcf [0,4194304] 0 2024-04-27T04:10:12.876 INFO:tasks.workunit.client.0.smithi162.stdout:7/594: dwrite d3/d5/d9/fc7 [0,4194304] 0 2024-04-27T04:10:12.879 INFO:tasks.workunit.client.0.smithi162.stdout:1/570: creat d0/d11/d50/d53/fbe x:0 0 0 2024-04-27T04:10:12.880 INFO:tasks.workunit.client.0.smithi162.stdout:3/574: rmdir d1/d18/d23/d34 39 2024-04-27T04:10:12.886 INFO:tasks.workunit.client.1.smithi181.stdout:2/806: dwrite d4/d9/d35/d42/d86/f8a [0,4194304] 0 2024-04-27T04:10:12.886 INFO:tasks.workunit.client.0.smithi162.stdout:9/601: mknod d3/d11/d45/d4e/d58/cb5 0 2024-04-27T04:10:12.886 INFO:tasks.workunit.client.0.smithi162.stdout:7/595: dwrite d3/d4f/f9d [0,4194304] 0 2024-04-27T04:10:12.890 INFO:tasks.workunit.client.1.smithi181.stdout:5/929: symlink d4/de/d2c/d46/d99/dad/dca/d102/l14e 0 2024-04-27T04:10:12.898 INFO:tasks.workunit.client.1.smithi181.stdout:0/893: mkdir dc/d1f/d3b/d4a/d95/d132 0 2024-04-27T04:10:12.900 INFO:tasks.workunit.client.0.smithi162.stdout:0/683: creat d5/d26/fdf x:0 0 0 2024-04-27T04:10:12.900 INFO:tasks.workunit.client.0.smithi162.stdout:0/684: fdatasync d5/d26/d69/d9b/db4/fca 0 2024-04-27T04:10:12.901 INFO:tasks.workunit.client.0.smithi162.stdout:0/685: fsync d5/d26/d69/f84 0 2024-04-27T04:10:12.907 INFO:tasks.workunit.client.0.smithi162.stdout:1/571: mknod d0/d7/d3c/d54/cbf 0 2024-04-27T04:10:12.911 INFO:tasks.workunit.client.1.smithi181.stdout:3/730: mkdir d0/de/d7d/dbc/de9 0 2024-04-27T04:10:12.912 INFO:tasks.workunit.client.0.smithi162.stdout:3/575: unlink d1/d18/d5d/d5f/d66/c7d 0 2024-04-27T04:10:12.914 INFO:tasks.workunit.client.1.smithi181.stdout:1/739: fdatasync d1/dad/db5/d6/da3/d1e/f79 0 2024-04-27T04:10:12.921 INFO:tasks.workunit.client.1.smithi181.stdout:8/796: rename d1/d8c/c93 to d1/d8c/c119 0 2024-04-27T04:10:12.921 INFO:tasks.workunit.client.0.smithi162.stdout:9/602: dwrite d3/d13/f26 [4194304,4194304] 0 2024-04-27T04:10:12.922 INFO:tasks.workunit.client.1.smithi181.stdout:8/797: stat d1/d6/da6/ff8 0 2024-04-27T04:10:12.927 INFO:tasks.workunit.client.1.smithi181.stdout:2/807: mkdir d4/d81/d102 0 2024-04-27T04:10:12.927 INFO:tasks.workunit.client.1.smithi181.stdout:2/808: write d4/d9/d4f/d79/d98/f99 [618556,61042] 0 2024-04-27T04:10:12.928 INFO:tasks.workunit.client.0.smithi162.stdout:4/564: dwrite d2/da/faf [0,4194304] 0 2024-04-27T04:10:12.930 INFO:tasks.workunit.client.1.smithi181.stdout:0/894: mknod dc/d1f/d3b/d4a/def/c133 0 2024-04-27T04:10:12.930 INFO:tasks.workunit.client.1.smithi181.stdout:2/809: dread d4/d9/f64 [0,4194304] 0 2024-04-27T04:10:12.933 INFO:tasks.workunit.client.1.smithi181.stdout:6/638: creat d7/dc/d1c/d7f/fdc x:0 0 0 2024-04-27T04:10:12.934 INFO:tasks.workunit.client.1.smithi181.stdout:2/810: dread - d4/d9/d16/d45/d9a/fe1 zero size 2024-04-27T04:10:12.935 INFO:tasks.workunit.client.1.smithi181.stdout:4/703: mknod d0/d15/de/d19/d77/ce6 0 2024-04-27T04:10:12.938 INFO:tasks.workunit.client.0.smithi162.stdout:8/617: rename d0/d10/d2b/d2d/d2f/d42 to d0/d10/d2b/d2e/dca 0 2024-04-27T04:10:12.939 INFO:tasks.workunit.client.1.smithi181.stdout:7/743: write d1/d22/d15/d29/f84 [1413370,10865] 0 2024-04-27T04:10:12.940 INFO:tasks.workunit.client.0.smithi162.stdout:1/572: fdatasync d0/d11/f79 0 2024-04-27T04:10:12.942 INFO:tasks.workunit.client.1.smithi181.stdout:3/731: chown d0/de/l35 307004 1 2024-04-27T04:10:12.942 INFO:tasks.workunit.client.1.smithi181.stdout:3/732: chown d0/d6c/d52/dbb/d98 0 1 2024-04-27T04:10:12.943 INFO:tasks.workunit.client.0.smithi162.stdout:5/633: creat d8/d9/d5b/fc8 x:0 0 0 2024-04-27T04:10:12.944 INFO:tasks.workunit.client.1.smithi181.stdout:3/733: chown d0/d6c/d52/d44/dd5/d6b/ca4 112 1 2024-04-27T04:10:12.946 INFO:tasks.workunit.client.1.smithi181.stdout:2/811: dread d4/d81/f84 [0,4194304] 0 2024-04-27T04:10:12.948 INFO:tasks.workunit.client.0.smithi162.stdout:7/596: dwrite d3/f1e [0,4194304] 0 2024-04-27T04:10:12.949 INFO:tasks.workunit.client.0.smithi162.stdout:7/597: truncate d3/d39/f77 622414 0 2024-04-27T04:10:12.950 INFO:tasks.workunit.client.1.smithi181.stdout:8/798: chown d1/d6/d2f/d40/l2e 1049725162 1 2024-04-27T04:10:12.952 INFO:tasks.workunit.client.1.smithi181.stdout:9/717: dwrite d0/d44/d4b/d43/f56 [0,4194304] 0 2024-04-27T04:10:12.953 INFO:tasks.workunit.client.0.smithi162.stdout:0/686: link d5/dc/d1e/d46/d48/d9e/db8/fc6 d5/d26/d69/d9b/db4/fe0 0 2024-04-27T04:10:12.958 INFO:tasks.workunit.client.0.smithi162.stdout:4/565: dread d2/d25/d1c/d39/d3b/d47/f4c [0,4194304] 0 2024-04-27T04:10:12.959 INFO:tasks.workunit.client.0.smithi162.stdout:6/595: rename d6/d1a/d34/d43/f5c to d6/d1a/d34/d7f/d80/db3/fbb 0 2024-04-27T04:10:12.962 INFO:tasks.workunit.client.1.smithi181.stdout:5/930: rename d4/de/d2c/c12e to d4/de/d107/c14f 0 2024-04-27T04:10:12.963 INFO:tasks.workunit.client.0.smithi162.stdout:2/532: truncate d0/d4/d17/f3c 2869671 0 2024-04-27T04:10:12.963 INFO:tasks.workunit.client.1.smithi181.stdout:5/931: dread - d4/de/d12f/f13f zero size 2024-04-27T04:10:12.965 INFO:tasks.workunit.client.1.smithi181.stdout:5/932: chown d4/de/d2c/d46/d4d/d8e/d54/cfd 452 1 2024-04-27T04:10:12.965 INFO:tasks.workunit.client.1.smithi181.stdout:0/895: mkdir dc/d1f/d3b/d4a/d95/d6d/d134 0 2024-04-27T04:10:12.970 INFO:tasks.workunit.client.0.smithi162.stdout:1/573: unlink d0/fe 0 2024-04-27T04:10:12.973 INFO:tasks.workunit.client.1.smithi181.stdout:3/734: write d0/f57 [2335889,65719] 0 2024-04-27T04:10:12.974 INFO:tasks.workunit.client.0.smithi162.stdout:3/576: dwrite d1/d18/f1d [0,4194304] 0 2024-04-27T04:10:12.979 INFO:tasks.workunit.client.1.smithi181.stdout:1/740: dwrite d1/dad/db5/fa1 [0,4194304] 0 2024-04-27T04:10:12.980 INFO:tasks.workunit.client.1.smithi181.stdout:1/741: write d1/dad/fdb [62021,38435] 0 2024-04-27T04:10:12.981 INFO:tasks.workunit.client.1.smithi181.stdout:2/812: unlink d4/d9/d16/d82/d7d/db7/fea 0 2024-04-27T04:10:12.984 INFO:tasks.workunit.client.0.smithi162.stdout:9/603: unlink d3/f2a 0 2024-04-27T04:10:12.989 INFO:tasks.workunit.client.0.smithi162.stdout:7/598: mkdir d3/d5/d5b/d83/de/d2b/d81/d96/dc8 0 2024-04-27T04:10:12.991 INFO:tasks.workunit.client.1.smithi181.stdout:8/799: creat d1/d6/d2f/d30/d66/dc0/dcd/f11a x:0 0 0 2024-04-27T04:10:12.992 INFO:tasks.workunit.client.0.smithi162.stdout:0/687: creat d5/d60/d8b/fe1 x:0 0 0 2024-04-27T04:10:12.994 INFO:tasks.workunit.client.0.smithi162.stdout:8/618: dwrite d0/d10/f9a [0,4194304] 0 2024-04-27T04:10:12.995 INFO:tasks.workunit.client.0.smithi162.stdout:8/619: write d0/d10/faf [4152197,39663] 0 2024-04-27T04:10:12.998 INFO:tasks.workunit.client.0.smithi162.stdout:4/566: fsync d2/d25/d1c/f33 0 2024-04-27T04:10:12.999 INFO:tasks.workunit.client.1.smithi181.stdout:5/933: truncate d4/de/d2f/d7a/d90/f12c 785838 0 2024-04-27T04:10:12.999 INFO:tasks.workunit.client.0.smithi162.stdout:8/620: dread d0/de/d6e/d71/f9d [0,4194304] 0 2024-04-27T04:10:12.999 INFO:tasks.workunit.client.0.smithi162.stdout:6/596: rmdir d6/d1a/d34/d7b 39 2024-04-27T04:10:13.000 INFO:tasks.workunit.client.1.smithi181.stdout:5/934: readlink d4/de/d2c/d46/d4d/laa 0 2024-04-27T04:10:13.004 INFO:tasks.workunit.client.1.smithi181.stdout:3/735: creat d0/d6c/d52/d44/dd5/d58/fea x:0 0 0 2024-04-27T04:10:13.005 INFO:tasks.workunit.client.0.smithi162.stdout:1/574: truncate d0/f2b 1854904 0 2024-04-27T04:10:13.006 INFO:tasks.workunit.client.0.smithi162.stdout:2/533: dwrite d0/d4/d2a/f9e [0,4194304] 0 2024-04-27T04:10:13.008 INFO:tasks.workunit.client.0.smithi162.stdout:2/534: read d0/d4/d2a/d3b/f74 [145202,45459] 0 2024-04-27T04:10:13.009 INFO:tasks.workunit.client.1.smithi181.stdout:1/742: read d1/dad/db5/d6/da3/d1e/d20/d53/f72 [640581,29325] 0 2024-04-27T04:10:13.011 INFO:tasks.workunit.client.0.smithi162.stdout:3/577: rename d1/d18/d5d/d5f/d66/l72 to d1/d70/lbf 0 2024-04-27T04:10:13.012 INFO:tasks.workunit.client.1.smithi181.stdout:2/813: fsync d4/d9/d16/d82/d7d/fe0 0 2024-04-27T04:10:13.014 INFO:tasks.workunit.client.0.smithi162.stdout:7/599: rmdir d3/d5/d5b/d60/d65 39 2024-04-27T04:10:13.015 INFO:tasks.workunit.client.0.smithi162.stdout:7/600: fsync d3/d5/d5b/d83/de/fbb 0 2024-04-27T04:10:13.015 INFO:tasks.workunit.client.0.smithi162.stdout:7/601: dread - d3/d5/f5f zero size 2024-04-27T04:10:13.016 INFO:tasks.workunit.client.1.smithi181.stdout:8/800: fdatasync d1/d6/d2f/fd3 0 2024-04-27T04:10:13.016 INFO:tasks.workunit.client.0.smithi162.stdout:7/602: chown d3/d5/l7b 108513685 1 2024-04-27T04:10:13.018 INFO:tasks.workunit.client.0.smithi162.stdout:0/688: unlink d5/f9 0 2024-04-27T04:10:13.020 INFO:tasks.workunit.client.1.smithi181.stdout:2/814: dwrite d4/d9/d16/d45/f9e [0,4194304] 0 2024-04-27T04:10:13.021 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:10:12 smithi162 ceph-mon[26841]: pgmap v6: 65 pgs: 65 active+clean; 2.6 GiB data, 9.3 GiB used, 527 GiB / 536 GiB avail; 49 MiB/s rd, 115 MiB/s wr, 172 op/s 2024-04-27T04:10:13.022 INFO:tasks.workunit.client.1.smithi181.stdout:2/815: chown d4/d9/d4f/d5f/d100 58937 1 2024-04-27T04:10:13.022 INFO:tasks.workunit.client.1.smithi181.stdout:0/896: creat dc/d1f/d3b/d4a/d95/d26/d41/d12f/f135 x:0 0 0 2024-04-27T04:10:13.030 INFO:tasks.workunit.client.1.smithi181.stdout:1/743: rmdir d1/d2/d64 39 2024-04-27T04:10:13.037 INFO:tasks.workunit.client.0.smithi162.stdout:8/621: creat d0/de/d20/d6c/fcb x:0 0 0 2024-04-27T04:10:13.037 INFO:tasks.workunit.client.1.smithi181.stdout:4/704: truncate d0/d15/d25/d2b/d6d/d82/fcc 1512645 0 2024-04-27T04:10:13.040 INFO:tasks.workunit.client.1.smithi181.stdout:8/801: mknod d1/d6/d6f/d91/c11b 0 2024-04-27T04:10:13.043 INFO:tasks.workunit.client.0.smithi162.stdout:5/634: truncate d8/d1a/f17 1764818 0 2024-04-27T04:10:13.044 INFO:tasks.workunit.client.0.smithi162.stdout:5/635: chown d8/f12 3 1 2024-04-27T04:10:13.045 INFO:tasks.workunit.client.1.smithi181.stdout:2/816: mknod d4/d9/d35/d9f/c103 0 2024-04-27T04:10:13.047 INFO:tasks.workunit.client.0.smithi162.stdout:9/604: rename d3/d13/d2d/l71 to d3/d13/d2d/d3b/d69/lb6 0 2024-04-27T04:10:13.048 INFO:tasks.workunit.client.0.smithi162.stdout:3/578: creat d1/d45/fc0 x:0 0 0 2024-04-27T04:10:13.052 INFO:tasks.workunit.client.1.smithi181.stdout:1/744: mkdir d1/dad/db5/d6/da3/d1e/dbc/df6 0 2024-04-27T04:10:13.054 INFO:tasks.workunit.client.0.smithi162.stdout:2/535: dread d0/d24/f5e [0,4194304] 0 2024-04-27T04:10:13.055 INFO:tasks.workunit.client.1.smithi181.stdout:5/935: write d4/de/d2c/d46/d99/dad/fe2 [62740,67656] 0 2024-04-27T04:10:13.055 INFO:tasks.workunit.client.0.smithi162.stdout:2/536: read d0/d4/d2a/d41/d77/f7c [2217771,130241] 0 2024-04-27T04:10:13.056 INFO:tasks.workunit.client.0.smithi162.stdout:6/597: write d6/db/d32/f5b [137628,522] 0 2024-04-27T04:10:13.057 INFO:tasks.workunit.client.0.smithi162.stdout:6/598: fdatasync d6/d1a/d34/f66 0 2024-04-27T04:10:13.058 INFO:tasks.workunit.client.0.smithi162.stdout:6/599: chown d6/f62 23 1 2024-04-27T04:10:13.059 INFO:tasks.workunit.client.1.smithi181.stdout:8/802: rename d1/d6/d2f/d30/d66/c70 to d1/d8c/c11c 0 2024-04-27T04:10:13.062 INFO:tasks.workunit.client.1.smithi181.stdout:9/718: getdents d0/d44/d4b/d43 0 2024-04-27T04:10:13.064 INFO:tasks.workunit.client.1.smithi181.stdout:0/897: creat dc/d1f/d3b/d4a/d95/d26/d131/dba/deb/df6/d125/f136 x:0 0 0 2024-04-27T04:10:13.070 INFO:tasks.workunit.client.1.smithi181.stdout:5/936: creat d4/de/d2f/d7a/d90/f150 x:0 0 0 2024-04-27T04:10:13.074 INFO:tasks.workunit.client.1.smithi181.stdout:4/705: mkdir d0/d15/d25/d2a/da0/dc7/de7 0 2024-04-27T04:10:13.078 INFO:tasks.workunit.client.1.smithi181.stdout:5/937: dwrite d4/da3/dbc/ff3 [0,4194304] 0 2024-04-27T04:10:13.080 INFO:tasks.workunit.client.1.smithi181.stdout:2/817: symlink d4/l104 0 2024-04-27T04:10:13.080 INFO:tasks.workunit.client.1.smithi181.stdout:2/818: fsync d4/d9/df/f41 0 2024-04-27T04:10:13.083 INFO:tasks.workunit.client.1.smithi181.stdout:9/719: fsync d0/d44/d4f/fae 0 2024-04-27T04:10:13.086 INFO:tasks.workunit.client.0.smithi162.stdout:5/636: write d8/d1a/fbf [3665303,81071] 0 2024-04-27T04:10:13.087 INFO:tasks.workunit.client.0.smithi162.stdout:2/537: creat d0/d4/d2a/d41/fa2 x:0 0 0 2024-04-27T04:10:13.091 INFO:tasks.workunit.client.1.smithi181.stdout:8/803: write d1/d6/d88/fb1 [591302,39231] 0 2024-04-27T04:10:13.093 INFO:tasks.workunit.client.0.smithi162.stdout:9/605: dwrite d3/d23/f4b [0,4194304] 0 2024-04-27T04:10:13.095 INFO:tasks.workunit.client.0.smithi162.stdout:9/606: write d3/d11/d45/d4e/d58/f98 [1966493,47102] 0 2024-04-27T04:10:13.095 INFO:tasks.workunit.client.0.smithi162.stdout:3/579: symlink d1/d65/lc1 0 2024-04-27T04:10:13.099 INFO:tasks.workunit.client.1.smithi181.stdout:1/745: truncate d1/f2f 7822124 0 2024-04-27T04:10:13.100 INFO:tasks.workunit.client.0.smithi162.stdout:7/603: creat d3/d39/fc9 x:0 0 0 2024-04-27T04:10:13.101 INFO:tasks.workunit.client.0.smithi162.stdout:7/604: fsync d3/d5/d9/fb8 0 2024-04-27T04:10:13.103 INFO:tasks.workunit.client.1.smithi181.stdout:0/898: creat dc/d1f/d3b/d4a/d95/d26/d41/d6f/d8f/f137 x:0 0 0 2024-04-27T04:10:13.103 INFO:tasks.workunit.client.1.smithi181.stdout:0/899: stat dc/d1f/d3b/d4a/fc1 0 2024-04-27T04:10:13.104 INFO:tasks.workunit.client.1.smithi181.stdout:4/706: mknod d0/d15/de1/ce8 0 2024-04-27T04:10:13.105 INFO:tasks.workunit.client.1.smithi181.stdout:8/804: symlink d1/d6/d2f/d30/d66/l11d 0 2024-04-27T04:10:13.106 INFO:tasks.workunit.client.0.smithi162.stdout:4/567: getdents d2/d25/d1c/d39/d90 0 2024-04-27T04:10:13.107 INFO:tasks.workunit.client.0.smithi162.stdout:6/600: mknod d6/db/d76/cbc 0 2024-04-27T04:10:13.108 INFO:tasks.workunit.client.1.smithi181.stdout:2/819: mkdir d4/d81/d102/d105 0 2024-04-27T04:10:13.109 INFO:tasks.workunit.client.1.smithi181.stdout:2/820: write d4/fd9 [1120436,37033] 0 2024-04-27T04:10:13.110 INFO:tasks.workunit.client.0.smithi162.stdout:0/689: rename d5/dc/d1e/d20/dba/cd5 to d5/dc/ce2 0 2024-04-27T04:10:13.112 INFO:tasks.workunit.client.0.smithi162.stdout:0/690: truncate d5/dc/d1e/d46/f9f 768150 0 2024-04-27T04:10:13.114 INFO:tasks.workunit.client.0.smithi162.stdout:3/580: unlink d1/d18/d23/d58/d63/f74 0 2024-04-27T04:10:13.115 INFO:tasks.workunit.client.1.smithi181.stdout:2/821: dwrite d4/d9/f11 [0,4194304] 0 2024-04-27T04:10:13.120 INFO:tasks.workunit.client.0.smithi162.stdout:7/605: mknod d3/d39/d57/cca 0 2024-04-27T04:10:13.121 INFO:tasks.workunit.client.0.smithi162.stdout:3/581: dwrite d1/d18/d23/f2e [0,4194304] 0 2024-04-27T04:10:13.123 INFO:tasks.workunit.client.1.smithi181.stdout:5/938: creat d4/d10/d29/d122/d100/f151 x:0 0 0 2024-04-27T04:10:13.126 INFO:tasks.workunit.client.1.smithi181.stdout:5/939: dread d4/d10/d29/d92/dbb/f137 [0,4194304] 0 2024-04-27T04:10:13.127 INFO:tasks.workunit.client.0.smithi162.stdout:2/538: symlink d0/d4/d17/d88/la3 0 2024-04-27T04:10:13.127 INFO:tasks.workunit.client.0.smithi162.stdout:7/606: dread d3/d5/d3c/d4a/fb0 [0,4194304] 0 2024-04-27T04:10:13.136 INFO:tasks.workunit.client.1.smithi181.stdout:2/822: creat d4/d9/df/d14/da7/f106 x:0 0 0 2024-04-27T04:10:13.141 INFO:tasks.workunit.client.1.smithi181.stdout:8/805: write d1/f46 [4291410,108244] 0 2024-04-27T04:10:13.143 INFO:tasks.workunit.client.1.smithi181.stdout:4/707: dwrite d0/d15/d5c/f7d [0,4194304] 0 2024-04-27T04:10:13.145 INFO:tasks.workunit.client.1.smithi181.stdout:5/940: mkdir d4/d10/d29/dd2/d152 0 2024-04-27T04:10:13.147 INFO:tasks.workunit.client.0.smithi162.stdout:5/637: creat d8/d9/d5b/fc9 x:0 0 0 2024-04-27T04:10:13.148 INFO:tasks.workunit.client.0.smithi162.stdout:2/539: creat d0/d24/fa4 x:0 0 0 2024-04-27T04:10:13.151 INFO:tasks.workunit.client.0.smithi162.stdout:2/540: dread d0/d4/d17/d88/f9d [0,4194304] 0 2024-04-27T04:10:13.155 INFO:tasks.workunit.client.0.smithi162.stdout:8/622: rename d0/d10/l8c to d0/de/d20/lcc 0 2024-04-27T04:10:13.156 INFO:tasks.workunit.client.0.smithi162.stdout:8/623: dread - d0/de/d20/d6c/fcb zero size 2024-04-27T04:10:13.156 INFO:tasks.workunit.client.0.smithi162.stdout:4/568: write d2/d25/d1c/d24/f43 [960458,85681] 0 2024-04-27T04:10:13.157 INFO:tasks.workunit.client.0.smithi162.stdout:9/607: creat d3/d13/fb7 x:0 0 0 2024-04-27T04:10:13.157 INFO:tasks.workunit.client.0.smithi162.stdout:4/569: dread - d2/d25/d1c/d39/d90/dad/f9a zero size 2024-04-27T04:10:13.161 INFO:tasks.workunit.client.0.smithi162.stdout:3/582: mknod d1/d18/d5d/d5f/dbe/cc2 0 2024-04-27T04:10:13.163 INFO:tasks.workunit.client.0.smithi162.stdout:5/638: mknod d8/d1a/d7d/d87/cca 0 2024-04-27T04:10:13.165 INFO:tasks.workunit.client.1.smithi181.stdout:4/708: dread d0/d15/d25/d2b/f5f [0,4194304] 0 2024-04-27T04:10:13.172 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:10:12 smithi181 ceph-mon[33774]: pgmap v6: 65 pgs: 65 active+clean; 2.6 GiB data, 9.3 GiB used, 527 GiB / 536 GiB avail; 49 MiB/s rd, 115 MiB/s wr, 172 op/s 2024-04-27T04:10:13.172 INFO:tasks.workunit.client.1.smithi181.stdout:2/823: dwrite d4/f8 [0,4194304] 0 2024-04-27T04:10:13.174 INFO:tasks.workunit.client.1.smithi181.stdout:1/746: dread d1/f2f [0,4194304] 0 2024-04-27T04:10:13.177 INFO:tasks.workunit.client.0.smithi162.stdout:7/607: link d3/d5/d5b/d83/de/d2b/d81/daa/dbd/f84 d3/d39/d57/fcb 0 2024-04-27T04:10:13.181 INFO:tasks.workunit.client.0.smithi162.stdout:1/575: rename d0/d11/d3e/l95 to d0/d11/d3e/d5f/lc0 0 2024-04-27T04:10:13.183 INFO:tasks.workunit.client.1.smithi181.stdout:4/709: creat d0/d15/fe9 x:0 0 0 2024-04-27T04:10:13.184 INFO:tasks.workunit.client.1.smithi181.stdout:0/900: link dc/d1f/d3b/cb7 dc/d1f/d6b/d7e/c138 0 2024-04-27T04:10:13.186 INFO:tasks.workunit.client.0.smithi162.stdout:9/608: symlink d3/d13/d2d/d3b/d7a/lb8 0 2024-04-27T04:10:13.187 INFO:tasks.workunit.client.0.smithi162.stdout:1/576: dwrite d0/d11/d3e/d59/d7f/f83 [0,4194304] 0 2024-04-27T04:10:13.188 INFO:tasks.workunit.client.0.smithi162.stdout:4/570: rmdir d2/d25/d1c/d39/d3b/d6e 39 2024-04-27T04:10:13.188 INFO:tasks.workunit.client.1.smithi181.stdout:8/806: write d1/d6/d6f/dd2/df2/f105 [948225,81255] 0 2024-04-27T04:10:13.189 INFO:tasks.workunit.client.0.smithi162.stdout:1/577: write d0/d7/d3c/d77/f8d [400100,79254] 0 2024-04-27T04:10:13.192 INFO:tasks.workunit.client.0.smithi162.stdout:2/541: dwrite d0/d24/f5e [4194304,4194304] 0 2024-04-27T04:10:13.197 INFO:tasks.workunit.client.0.smithi162.stdout:0/691: rename d5/dc/d1e/d41/l9a to d5/dc/d1e/d56/le3 0 2024-04-27T04:10:13.199 INFO:tasks.workunit.client.0.smithi162.stdout:8/624: mkdir d0/d10/d25/d65/d96/dcd 0 2024-04-27T04:10:13.199 INFO:tasks.workunit.client.0.smithi162.stdout:8/625: stat d0/de/d20/l22 0 2024-04-27T04:10:13.201 INFO:tasks.workunit.client.1.smithi181.stdout:2/824: mknod d4/d81/df3/c107 0 2024-04-27T04:10:13.202 INFO:tasks.workunit.client.1.smithi181.stdout:2/825: fdatasync d4/f8 0 2024-04-27T04:10:13.203 INFO:tasks.workunit.client.1.smithi181.stdout:2/826: stat d4/d81/df3 0 2024-04-27T04:10:13.203 INFO:tasks.workunit.client.1.smithi181.stdout:8/807: chown d1/d6/d1a/l79 1361489 1 2024-04-27T04:10:13.205 INFO:tasks.workunit.client.0.smithi162.stdout:8/626: dwrite d0/d10/d2b/fbc [0,4194304] 0 2024-04-27T04:10:13.206 INFO:tasks.workunit.client.0.smithi162.stdout:2/542: chown d0/d24/l62 286 1 2024-04-27T04:10:13.206 INFO:tasks.workunit.client.0.smithi162.stdout:8/627: stat d0/d10/d25/d65/la6 0 2024-04-27T04:10:13.208 INFO:tasks.workunit.client.1.smithi181.stdout:4/710: getdents d0/dd4 0 2024-04-27T04:10:13.212 INFO:tasks.workunit.client.1.smithi181.stdout:5/941: truncate d4/de/d2c/d46/d4d/d8e/d67/fa2 928192 0 2024-04-27T04:10:13.215 INFO:tasks.workunit.client.0.smithi162.stdout:5/639: rename d8/d1a/fbf to d8/d1a/d29/fcb 0 2024-04-27T04:10:13.217 INFO:tasks.workunit.client.0.smithi162.stdout:0/692: symlink d5/dc/d1e/d20/le4 0 2024-04-27T04:10:13.220 INFO:tasks.workunit.client.0.smithi162.stdout:9/609: mkdir d3/db9 0 2024-04-27T04:10:13.223 INFO:tasks.workunit.client.0.smithi162.stdout:7/608: truncate d3/d5/d5b/d83/f3b 3075533 0 2024-04-27T04:10:13.223 INFO:tasks.workunit.client.0.smithi162.stdout:7/609: readlink d3/d5/d5b/lb5 0 2024-04-27T04:10:13.225 INFO:tasks.workunit.client.1.smithi181.stdout:0/901: link dc/d1f/d3b/d4a/d95/d26/d41/d6f/d7d/dee/f128 dc/d1f/d3b/d4a/d95/d26/d131/f139 0 2024-04-27T04:10:13.232 INFO:tasks.workunit.client.1.smithi181.stdout:6/639: sync 2024-04-27T04:10:13.232 INFO:tasks.workunit.client.1.smithi181.stdout:1/747: getdents d1/dad/db5/d6/db 0 2024-04-27T04:10:13.233 INFO:tasks.workunit.client.1.smithi181.stdout:6/640: read - d7/dc/d1c/d88/fcb zero size 2024-04-27T04:10:13.234 INFO:tasks.workunit.client.0.smithi162.stdout:2/543: creat d0/d4/d2a/da1/fa5 x:0 0 0 2024-04-27T04:10:13.234 INFO:tasks.workunit.client.1.smithi181.stdout:2/827: dread d4/f59 [0,4194304] 0 2024-04-27T04:10:13.242 INFO:tasks.workunit.client.0.smithi162.stdout:9/610: mkdir d3/d13/d2d/dba 0 2024-04-27T04:10:13.244 INFO:tasks.workunit.client.1.smithi181.stdout:5/942: dwrite d4/f13 [4194304,4194304] 0 2024-04-27T04:10:13.244 INFO:tasks.workunit.client.0.smithi162.stdout:1/578: creat d0/d7/d3c/fc1 x:0 0 0 2024-04-27T04:10:13.244 INFO:tasks.workunit.client.1.smithi181.stdout:5/943: chown d4/de/f6e 65835817 1 2024-04-27T04:10:13.250 INFO:tasks.workunit.client.0.smithi162.stdout:2/544: symlink d0/d4/d17/d4f/d7f/la6 0 2024-04-27T04:10:13.256 INFO:tasks.workunit.client.0.smithi162.stdout:0/693: mknod d5/dc/d1e/d46/d48/ce5 0 2024-04-27T04:10:13.258 INFO:tasks.workunit.client.0.smithi162.stdout:9/611: chown d3/l76 5646 1 2024-04-27T04:10:13.260 INFO:tasks.workunit.client.0.smithi162.stdout:9/612: write d3/d13/f41 [5233281,24669] 0 2024-04-27T04:10:13.262 INFO:tasks.workunit.client.0.smithi162.stdout:1/579: dread - d0/d11/d5b/fad zero size 2024-04-27T04:10:13.263 INFO:tasks.workunit.client.0.smithi162.stdout:7/610: mknod d3/d5/d5b/ccc 0 2024-04-27T04:10:13.265 INFO:tasks.workunit.client.0.smithi162.stdout:5/640: truncate d8/d9/d55/f65 4548466 0 2024-04-27T04:10:13.266 INFO:tasks.workunit.client.0.smithi162.stdout:2/545: creat d0/d4/d2a/d3b/d49/d5a/d63/d6e/fa7 x:0 0 0 2024-04-27T04:10:13.267 INFO:tasks.workunit.client.0.smithi162.stdout:8/628: dwrite d0/d10/d25/f88 [0,4194304] 0 2024-04-27T04:10:13.268 INFO:tasks.workunit.client.1.smithi181.stdout:9/720: sync 2024-04-27T04:10:13.268 INFO:tasks.workunit.client.1.smithi181.stdout:3/736: sync 2024-04-27T04:10:13.268 INFO:tasks.workunit.client.1.smithi181.stdout:7/744: sync 2024-04-27T04:10:13.272 INFO:tasks.workunit.client.0.smithi162.stdout:9/613: creat d3/d13/d2d/d3b/d69/fbb x:0 0 0 2024-04-27T04:10:13.277 INFO:tasks.workunit.client.1.smithi181.stdout:7/745: dwrite d1/df/fdf [0,4194304] 0 2024-04-27T04:10:13.278 INFO:tasks.workunit.client.0.smithi162.stdout:7/611: rmdir d3/d5/d3c/d4d 39 2024-04-27T04:10:13.278 INFO:tasks.workunit.client.0.smithi162.stdout:2/546: dwrite d0/d4/d2a/f36 [4194304,4194304] 0 2024-04-27T04:10:13.281 INFO:tasks.workunit.client.0.smithi162.stdout:8/629: dwrite d0/d10/d2b/d2e/dca/fa0 [0,4194304] 0 2024-04-27T04:10:13.281 INFO:tasks.workunit.client.1.smithi181.stdout:7/746: stat d1/dc3/daf/ldc 0 2024-04-27T04:10:13.285 INFO:tasks.workunit.client.0.smithi162.stdout:7/612: chown d3/d39/d57/f42 223 1 2024-04-27T04:10:13.285 INFO:tasks.workunit.client.0.smithi162.stdout:2/547: dread d0/d4/d17/d4f/f85 [0,4194304] 0 2024-04-27T04:10:13.285 INFO:tasks.workunit.client.0.smithi162.stdout:5/641: symlink d8/d1a/d29/d4a/d5c/lcc 0 2024-04-27T04:10:13.294 INFO:tasks.workunit.client.0.smithi162.stdout:1/580: dread d0/d11/d3e/d5f/f6a [0,4194304] 0 2024-04-27T04:10:13.299 INFO:tasks.workunit.client.0.smithi162.stdout:9/614: unlink d3/d11/f5e 0 2024-04-27T04:10:13.300 INFO:tasks.workunit.client.0.smithi162.stdout:0/694: write d5/d26/d4a/f62 [7643080,38734] 0 2024-04-27T04:10:13.304 INFO:tasks.workunit.client.1.smithi181.stdout:7/747: dread d1/d2/d1c/d56/fb3 [0,4194304] 0 2024-04-27T04:10:13.305 INFO:tasks.workunit.client.1.smithi181.stdout:7/748: dread - d1/fc8 zero size 2024-04-27T04:10:13.305 INFO:tasks.workunit.client.1.smithi181.stdout:3/737: dread d0/d6c/d52/f59 [0,4194304] 0 2024-04-27T04:10:13.307 INFO:tasks.workunit.client.0.smithi162.stdout:5/642: creat d8/d1a/d23/d40/fcd x:0 0 0 2024-04-27T04:10:13.311 INFO:tasks.workunit.client.0.smithi162.stdout:7/613: rename d3/d5/d5b/d83/f79 to d3/d5/d3c/d4a/d55/dc6/fcd 0 2024-04-27T04:10:13.312 INFO:tasks.workunit.client.0.smithi162.stdout:6/601: sync 2024-04-27T04:10:13.312 INFO:tasks.workunit.client.0.smithi162.stdout:3/583: sync 2024-04-27T04:10:13.312 INFO:tasks.workunit.client.0.smithi162.stdout:4/571: sync 2024-04-27T04:10:13.314 INFO:tasks.workunit.client.0.smithi162.stdout:2/548: fdatasync d0/f1 0 2024-04-27T04:10:13.315 INFO:tasks.workunit.client.1.smithi181.stdout:3/738: dread d0/d6c/fb1 [0,4194304] 0 2024-04-27T04:10:13.317 INFO:tasks.workunit.client.0.smithi162.stdout:9/615: mknod d3/d13/d2d/d30/d7f/cbc 0 2024-04-27T04:10:13.320 INFO:tasks.workunit.client.0.smithi162.stdout:2/549: dwrite d0/d4/d2a/d41/d77/f83 [0,4194304] 0 2024-04-27T04:10:13.320 INFO:tasks.workunit.client.0.smithi162.stdout:0/695: creat d5/d26/d4a/fe6 x:0 0 0 2024-04-27T04:10:13.320 INFO:tasks.workunit.client.0.smithi162.stdout:2/550: chown d0 226 1 2024-04-27T04:10:13.321 INFO:tasks.workunit.client.0.smithi162.stdout:0/696: chown d5/dc/d1e 3 1 2024-04-27T04:10:13.324 INFO:tasks.workunit.client.0.smithi162.stdout:5/643: mknod d8/d1a/d23/dbc/d78/cce 0 2024-04-27T04:10:13.327 INFO:tasks.workunit.client.1.smithi181.stdout:7/749: dread d1/d22/d15/d29/f43 [0,4194304] 0 2024-04-27T04:10:13.332 INFO:tasks.workunit.client.0.smithi162.stdout:7/614: mkdir d3/d5/d5b/d83/de/dce 0 2024-04-27T04:10:13.338 INFO:tasks.workunit.client.0.smithi162.stdout:6/602: fsync d6/f2d 0 2024-04-27T04:10:13.339 INFO:tasks.workunit.client.0.smithi162.stdout:7/615: dwrite d3/f93 [0,4194304] 0 2024-04-27T04:10:13.340 INFO:tasks.workunit.client.1.smithi181.stdout:9/721: rename d0/d44/d4b/d50/l74 to d0/d44/d4b/d50/lf5 0 2024-04-27T04:10:13.346 INFO:tasks.workunit.client.0.smithi162.stdout:2/551: dread d0/f1 [0,4194304] 0 2024-04-27T04:10:13.348 INFO:tasks.workunit.client.0.smithi162.stdout:2/552: truncate d0/d4/d2a/d3b/d49/d5a/d63/d84/d93/f76 4768876 0 2024-04-27T04:10:13.350 INFO:tasks.workunit.client.1.smithi181.stdout:0/902: dwrite dc/d1f/d3b/d4a/d95/d6d/fc4 [0,4194304] 0 2024-04-27T04:10:13.351 INFO:tasks.workunit.client.0.smithi162.stdout:8/630: creat d0/de/d6e/fce x:0 0 0 2024-04-27T04:10:13.353 INFO:tasks.workunit.client.1.smithi181.stdout:3/739: fdatasync d0/d6c/d52/dbb/fb4 0 2024-04-27T04:10:13.353 INFO:tasks.workunit.client.1.smithi181.stdout:0/903: chown dc/d1f/d3b/d4a/d95/d6d/fc4 502214 1 2024-04-27T04:10:13.356 INFO:tasks.workunit.client.1.smithi181.stdout:6/641: dwrite d7/dc/dc3/f4b [4194304,4194304] 0 2024-04-27T04:10:13.357 INFO:tasks.workunit.client.1.smithi181.stdout:6/642: chown d7/dc/d1e 233569 1 2024-04-27T04:10:13.357 INFO:tasks.workunit.client.1.smithi181.stdout:8/808: rename d1/d6/d88 to d1/d6/d2f/d11e 0 2024-04-27T04:10:13.360 INFO:tasks.workunit.client.0.smithi162.stdout:0/697: creat d5/d60/d98/fe7 x:0 0 0 2024-04-27T04:10:13.361 INFO:tasks.workunit.client.0.smithi162.stdout:8/631: readlink d0/d10/d25/d65/la6 0 2024-04-27T04:10:13.361 INFO:tasks.workunit.client.0.smithi162.stdout:5/644: mknod d8/d9/db3/ccf 0 2024-04-27T04:10:13.362 INFO:tasks.workunit.client.0.smithi162.stdout:5/645: fsync d8/d9/f3d 0 2024-04-27T04:10:13.362 INFO:tasks.workunit.client.0.smithi162.stdout:8/632: chown d0/d10/d44/f73 87 1 2024-04-27T04:10:13.362 INFO:tasks.workunit.client.0.smithi162.stdout:0/698: read - d5/d60/d8b/fe1 zero size 2024-04-27T04:10:13.363 INFO:tasks.workunit.client.0.smithi162.stdout:5/646: dread - d8/d1a/d29/f95 zero size 2024-04-27T04:10:13.363 INFO:tasks.workunit.client.0.smithi162.stdout:8/633: dread - d0/d10/d44/d70/f7f zero size 2024-04-27T04:10:13.364 INFO:tasks.workunit.client.1.smithi181.stdout:1/748: mknod d1/dad/db5/d6/da3/d1e/d20/d53/d56/cf7 0 2024-04-27T04:10:13.366 INFO:tasks.workunit.client.0.smithi162.stdout:8/634: truncate d0/de/d6e/d71/fb3 33577 0 2024-04-27T04:10:13.366 INFO:tasks.workunit.client.1.smithi181.stdout:1/749: chown d1/dad/db5/l37 16 1 2024-04-27T04:10:13.367 INFO:tasks.workunit.client.0.smithi162.stdout:8/635: readlink d0/d10/d25/d65/la6 0 2024-04-27T04:10:13.367 INFO:tasks.workunit.client.1.smithi181.stdout:1/750: write d1/dad/db5/d6/da3/d52/d92/f99 [4981237,98611] 0 2024-04-27T04:10:13.367 INFO:tasks.workunit.client.0.smithi162.stdout:8/636: dread - d0/d10/d25/d7d/db8/dbe/fc6 zero size 2024-04-27T04:10:13.368 INFO:tasks.workunit.client.1.smithi181.stdout:7/750: mknod d1/df/d1a/ce1 0 2024-04-27T04:10:13.369 INFO:tasks.workunit.client.0.smithi162.stdout:8/637: chown d0/de/d6e/f79 1432703827 1 2024-04-27T04:10:13.372 INFO:tasks.workunit.client.0.smithi162.stdout:6/603: mknod d6/d1a/d34/d7f/d80/cbd 0 2024-04-27T04:10:13.373 INFO:tasks.workunit.client.0.smithi162.stdout:6/604: readlink d6/d1a/d34/d43/l7a 0 2024-04-27T04:10:13.373 INFO:tasks.workunit.client.1.smithi181.stdout:5/944: mkdir d4/de/d2c/d46/d4d/d8e/d67/d71/d153 0 2024-04-27T04:10:13.375 INFO:tasks.workunit.client.0.smithi162.stdout:3/584: link d1/d21/d47/d57/f5e d1/d21/d47/d57/fc3 0 2024-04-27T04:10:13.377 INFO:tasks.workunit.client.0.smithi162.stdout:3/585: dread d1/d21/fb8 [0,4194304] 0 2024-04-27T04:10:13.377 INFO:tasks.workunit.client.1.smithi181.stdout:9/722: creat d0/d91/ff6 x:0 0 0 2024-04-27T04:10:13.379 INFO:tasks.workunit.client.1.smithi181.stdout:9/723: write d0/d44/d4b/d43/d88/fef [180038,116946] 0 2024-04-27T04:10:13.380 INFO:tasks.workunit.client.0.smithi162.stdout:9/616: dread d3/f21 [0,4194304] 0 2024-04-27T04:10:13.380 INFO:tasks.workunit.client.0.smithi162.stdout:2/553: unlink d0/d24/fa4 0 2024-04-27T04:10:13.383 INFO:tasks.workunit.client.0.smithi162.stdout:0/699: dread d5/f2d [0,4194304] 0 2024-04-27T04:10:13.384 INFO:tasks.workunit.client.0.smithi162.stdout:0/700: stat d5/dc/d1e/d41/fa7 0 2024-04-27T04:10:13.384 INFO:tasks.workunit.client.0.smithi162.stdout:0/701: chown d5/dc/d1e/d46/d48 185252513 1 2024-04-27T04:10:13.388 INFO:tasks.workunit.client.1.smithi181.stdout:3/740: fdatasync d0/d28/f42 0 2024-04-27T04:10:13.396 INFO:tasks.workunit.client.1.smithi181.stdout:4/711: rename d0/d15/de/d19/l3c to d0/d15/dd2/lea 0 2024-04-27T04:10:13.398 INFO:tasks.workunit.client.1.smithi181.stdout:6/643: fsync d7/dc/d1c/d7f/f76 0 2024-04-27T04:10:13.404 INFO:tasks.workunit.client.1.smithi181.stdout:2/828: link d4/d9/d4f/d79/d94/f95 d4/d9/df/d14/da7/f108 0 2024-04-27T04:10:13.411 INFO:tasks.workunit.client.1.smithi181.stdout:3/741: creat d0/d6c/d52/d44/dd5/d58/de4/feb x:0 0 0 2024-04-27T04:10:13.416 INFO:tasks.workunit.client.1.smithi181.stdout:9/724: write d0/d44/d4f/fae [938612,105683] 0 2024-04-27T04:10:13.419 INFO:tasks.workunit.client.1.smithi181.stdout:7/751: rename d1/d2/d1c/d8f/f8a to d1/d22/d15/d1f/d34/fe2 0 2024-04-27T04:10:13.421 INFO:tasks.workunit.client.1.smithi181.stdout:7/752: read d1/df/d1a/fc5 [2431188,60111] 0 2024-04-27T04:10:13.427 INFO:tasks.workunit.client.1.smithi181.stdout:2/829: creat d4/d9/d35/d9f/dd0/f109 x:0 0 0 2024-04-27T04:10:13.428 INFO:tasks.workunit.client.1.smithi181.stdout:0/904: dwrite dc/d1f/f23 [0,4194304] 0 2024-04-27T04:10:13.434 INFO:tasks.workunit.client.0.smithi162.stdout:5/647: creat d8/d1a/fd0 x:0 0 0 2024-04-27T04:10:13.435 INFO:tasks.workunit.client.0.smithi162.stdout:4/572: creat d2/fb1 x:0 0 0 2024-04-27T04:10:13.437 INFO:tasks.workunit.client.1.smithi181.stdout:4/712: rename d0/d15/de/d60/d6f to d0/d15/dd2/deb 0 2024-04-27T04:10:13.439 INFO:tasks.workunit.client.1.smithi181.stdout:3/742: dwrite d0/f14 [0,4194304] 0 2024-04-27T04:10:13.440 INFO:tasks.workunit.client.0.smithi162.stdout:8/638: creat d0/de/d6e/d71/d86/fcf x:0 0 0 2024-04-27T04:10:13.441 INFO:tasks.workunit.client.1.smithi181.stdout:7/753: unlink d1/d2/d1c/d8f/c78 0 2024-04-27T04:10:13.443 INFO:tasks.workunit.client.1.smithi181.stdout:1/751: link d1/dad/db5/d6/d6a/cc9 d1/dad/db5/d6/dcb/cf8 0 2024-04-27T04:10:13.445 INFO:tasks.workunit.client.1.smithi181.stdout:5/945: creat d4/d10/f154 x:0 0 0 2024-04-27T04:10:13.447 INFO:tasks.workunit.client.0.smithi162.stdout:8/639: dwrite d0/d10/d25/f88 [0,4194304] 0 2024-04-27T04:10:13.451 INFO:tasks.workunit.client.1.smithi181.stdout:6/644: rename d7/dc/dc3/lba to d7/d27/ldd 0 2024-04-27T04:10:13.451 INFO:tasks.workunit.client.0.smithi162.stdout:2/554: rmdir d0/d4/d2a/d41/d7b 39 2024-04-27T04:10:13.452 INFO:tasks.workunit.client.1.smithi181.stdout:3/743: creat d0/d6c/d66/d99/fec x:0 0 0 2024-04-27T04:10:13.454 INFO:tasks.workunit.client.1.smithi181.stdout:3/744: chown d0/d6c/d52/dbb/de5/lb7 95 1 2024-04-27T04:10:13.454 INFO:tasks.workunit.client.0.smithi162.stdout:0/702: creat d5/dc/d1e/d20/dba/fe8 x:0 0 0 2024-04-27T04:10:13.455 INFO:tasks.workunit.client.1.smithi181.stdout:4/713: truncate d0/d15/d25/d2a/d4c/d7c/db5/fbf 934255 0 2024-04-27T04:10:13.455 INFO:tasks.workunit.client.0.smithi162.stdout:0/703: chown d5/dc/d1e/d20/dc5 59 1 2024-04-27T04:10:13.455 INFO:tasks.workunit.client.0.smithi162.stdout:5/648: dread d8/fad [0,4194304] 0 2024-04-27T04:10:13.457 INFO:tasks.workunit.client.0.smithi162.stdout:4/573: unlink d2/da/dd/l3c 0 2024-04-27T04:10:13.458 INFO:tasks.workunit.client.1.smithi181.stdout:8/809: link d1/d6/d2f/d30/d9d/dea/l54 d1/d6/d1a/l11f 0 2024-04-27T04:10:13.460 INFO:tasks.workunit.client.1.smithi181.stdout:8/810: truncate d1/d6/d2f/d30/d66/dc0/dcd/f11a 275632 0 2024-04-27T04:10:13.462 INFO:tasks.workunit.client.1.smithi181.stdout:2/830: creat d4/d81/d102/d105/f10a x:0 0 0 2024-04-27T04:10:13.462 INFO:tasks.workunit.client.1.smithi181.stdout:8/811: write d1/d6/d2f/d47/f4b [235686,99093] 0 2024-04-27T04:10:13.465 INFO:tasks.workunit.client.1.smithi181.stdout:0/905: mknod dc/d1f/d3b/d4a/d95/d26/d131/d106/c13a 0 2024-04-27T04:10:13.465 INFO:tasks.workunit.client.0.smithi162.stdout:1/581: getdents d0/d7/d3c/d6e 0 2024-04-27T04:10:13.466 INFO:tasks.workunit.client.0.smithi162.stdout:1/582: chown d0/d7/d10 6966 1 2024-04-27T04:10:13.467 INFO:tasks.workunit.client.1.smithi181.stdout:6/645: mkdir d7/dc/d1e/d2b/dde 0 2024-04-27T04:10:13.470 INFO:tasks.workunit.client.1.smithi181.stdout:3/745: unlink d0/de/d7d/dbc/fc9 0 2024-04-27T04:10:13.475 INFO:tasks.workunit.client.1.smithi181.stdout:7/754: mkdir d1/d22/d15/d5f/d81/de3 0 2024-04-27T04:10:13.476 INFO:tasks.workunit.client.0.smithi162.stdout:5/649: mknod d8/d9/d5b/d96/cd1 0 2024-04-27T04:10:13.477 INFO:tasks.workunit.client.0.smithi162.stdout:5/650: chown d8/d9/l25 2498023 1 2024-04-27T04:10:13.479 INFO:tasks.workunit.client.0.smithi162.stdout:4/574: creat d2/d25/d1c/d39/d5e/fb2 x:0 0 0 2024-04-27T04:10:13.480 INFO:tasks.workunit.client.0.smithi162.stdout:3/586: sync 2024-04-27T04:10:13.480 INFO:tasks.workunit.client.1.smithi181.stdout:2/831: unlink d4/d9/d4f/d79/fbc 0 2024-04-27T04:10:13.482 INFO:tasks.workunit.client.0.smithi162.stdout:7/616: getdents d3/d5/d5b 0 2024-04-27T04:10:13.483 INFO:tasks.workunit.client.1.smithi181.stdout:8/812: rename d1/d6/d2f/d11e/cad to d1/d6/d6f/dd2/df2/c120 0 2024-04-27T04:10:13.483 INFO:tasks.workunit.client.1.smithi181.stdout:8/813: chown d1/f82 11 1 2024-04-27T04:10:13.485 INFO:tasks.workunit.client.1.smithi181.stdout:0/906: dread - dc/d1f/d115/f104 zero size 2024-04-27T04:10:13.489 INFO:tasks.workunit.client.0.smithi162.stdout:9/617: creat d3/fbd x:0 0 0 2024-04-27T04:10:13.492 INFO:tasks.workunit.client.1.smithi181.stdout:3/746: rmdir d0/d6c/d52/d44/d74 39 2024-04-27T04:10:13.495 INFO:tasks.workunit.client.0.smithi162.stdout:0/704: link d5/d26/da2/fcb d5/dc/d1e/d56/da8/fe9 0 2024-04-27T04:10:13.496 INFO:tasks.workunit.client.0.smithi162.stdout:0/705: truncate d5/d60/d8b/fd6 783046 0 2024-04-27T04:10:13.496 INFO:tasks.workunit.client.0.smithi162.stdout:5/651: rename d8/d9/f10 to d8/d1a/d29/d8b/fd2 0 2024-04-27T04:10:13.500 INFO:tasks.workunit.client.1.smithi181.stdout:2/832: mkdir d4/d9/d16/d82/d7d/dbe/d10b 0 2024-04-27T04:10:13.502 INFO:tasks.workunit.client.0.smithi162.stdout:6/605: getdents d6/d1a/d34/d7f 0 2024-04-27T04:10:13.503 INFO:tasks.workunit.client.1.smithi181.stdout:0/907: write dc/d1f/d3b/d4a/d95/d26/d131/dba/deb/db3/fb8 [1328650,63421] 0 2024-04-27T04:10:13.506 INFO:tasks.workunit.client.1.smithi181.stdout:6/646: mkdir d7/dc/d1c/d88/db3/ddf 0 2024-04-27T04:10:13.506 INFO:tasks.workunit.client.1.smithi181.stdout:5/946: write d4/d10/d29/d122/d100/d8f/fe8 [1063487,125093] 0 2024-04-27T04:10:13.508 INFO:tasks.workunit.client.1.smithi181.stdout:1/752: dwrite d1/dad/db5/d6/db/d2d/d36/dc4/fcf [0,4194304] 0 2024-04-27T04:10:13.509 INFO:tasks.workunit.client.1.smithi181.stdout:3/747: fsync d0/d6c/d52/f59 0 2024-04-27T04:10:13.511 INFO:tasks.workunit.client.1.smithi181.stdout:3/748: dread - d0/d6c/d66/dc2/fe3 zero size 2024-04-27T04:10:13.513 INFO:tasks.workunit.client.1.smithi181.stdout:4/714: write d0/d15/d25/d2a/d61/fbc [905904,611] 0 2024-04-27T04:10:13.513 INFO:tasks.workunit.client.0.smithi162.stdout:2/555: write d0/d4/d17/d88/f9d [1457387,54851] 0 2024-04-27T04:10:13.517 INFO:tasks.workunit.client.0.smithi162.stdout:9/618: unlink d3/d13/d15/l1a 0 2024-04-27T04:10:13.518 INFO:tasks.workunit.client.1.smithi181.stdout:2/833: unlink d4/d9/c6f 0 2024-04-27T04:10:13.519 INFO:tasks.workunit.client.0.smithi162.stdout:1/583: dwrite d0/d11/f20 [8388608,4194304] 0 2024-04-27T04:10:13.520 INFO:tasks.workunit.client.1.smithi181.stdout:2/834: write d4/d9/d4f/d5f/fc1 [2245437,26017] 0 2024-04-27T04:10:13.520 INFO:tasks.workunit.client.0.smithi162.stdout:0/706: creat d5/dc/d1e/d20/fea x:0 0 0 2024-04-27T04:10:13.521 INFO:tasks.workunit.client.1.smithi181.stdout:0/908: chown dc/d1f/d3b/c53 46 1 2024-04-27T04:10:13.532 INFO:tasks.workunit.client.0.smithi162.stdout:4/575: symlink d2/d25/d1c/d24/lb3 0 2024-04-27T04:10:13.533 INFO:tasks.workunit.client.1.smithi181.stdout:5/947: chown d4/de/c33 1 1 2024-04-27T04:10:13.535 INFO:tasks.workunit.client.0.smithi162.stdout:8/640: dwrite d0/f4 [0,4194304] 0 2024-04-27T04:10:13.542 INFO:tasks.workunit.client.1.smithi181.stdout:5/948: dwrite d4/de/d2c/d46/d99/dad/d13a/db3/fb6 [0,4194304] 0 2024-04-27T04:10:13.545 INFO:tasks.workunit.client.0.smithi162.stdout:7/617: creat d3/d5/d3c/d4d/fcf x:0 0 0 2024-04-27T04:10:13.547 INFO:tasks.workunit.client.0.smithi162.stdout:7/618: fsync d3/d5/d5b/d83/de/d2b/d81/daa/dbd/f97 0 2024-04-27T04:10:13.547 INFO:tasks.workunit.client.0.smithi162.stdout:7/619: chown d3/d5/lc5 171 1 2024-04-27T04:10:13.548 INFO:tasks.workunit.client.1.smithi181.stdout:4/715: creat d0/d15/d25/d2a/d36/db8/fec x:0 0 0 2024-04-27T04:10:13.549 INFO:tasks.workunit.client.1.smithi181.stdout:7/755: dwrite d1/f8 [0,4194304] 0 2024-04-27T04:10:13.550 INFO:tasks.workunit.client.0.smithi162.stdout:7/620: chown d3/d5/d5b/d83/de/d2b/d81/f9c 5839289 1 2024-04-27T04:10:13.552 INFO:tasks.workunit.client.0.smithi162.stdout:9/619: creat d3/d11/d45/d4e/fbe x:0 0 0 2024-04-27T04:10:13.559 INFO:tasks.workunit.client.0.smithi162.stdout:1/584: rename d0/d7/d3c/d6e/c85 to d0/d7/cc2 0 2024-04-27T04:10:13.561 INFO:tasks.workunit.client.0.smithi162.stdout:0/707: rmdir d5/dc/d1e/d56/da8 39 2024-04-27T04:10:13.562 INFO:tasks.workunit.client.0.smithi162.stdout:0/708: write d5/dc/d1e/fbf [1278898,28961] 0 2024-04-27T04:10:13.572 INFO:tasks.workunit.client.1.smithi181.stdout:5/949: dread d4/d10/d29/d122/fc2 [0,4194304] 0 2024-04-27T04:10:13.574 INFO:tasks.workunit.client.0.smithi162.stdout:8/641: mkdir d0/d10/d2b/d2d/d2f/d39/d7c/dd0 0 2024-04-27T04:10:13.575 INFO:tasks.workunit.client.0.smithi162.stdout:8/642: chown d0/d10/d2b/d2e/f40 3102 1 2024-04-27T04:10:13.576 INFO:tasks.workunit.client.0.smithi162.stdout:3/587: rmdir d1/d45/d69/da3/db0 0 2024-04-27T04:10:13.578 INFO:tasks.workunit.client.0.smithi162.stdout:6/606: dwrite d6/f35 [0,4194304] 0 2024-04-27T04:10:13.580 INFO:tasks.workunit.client.0.smithi162.stdout:2/556: symlink d0/d4/d2a/d3b/d49/d5a/d63/d70/d8e/la8 0 2024-04-27T04:10:13.583 INFO:tasks.workunit.client.0.smithi162.stdout:7/621: creat d3/d5/d5b/d83/de/d2b/d81/d96/fd0 x:0 0 0 2024-04-27T04:10:13.584 INFO:tasks.workunit.client.1.smithi181.stdout:1/753: dwrite d1/dad/db5/d6/da3/d1e/f4b [0,4194304] 0 2024-04-27T04:10:13.587 INFO:tasks.workunit.client.0.smithi162.stdout:9/620: mknod d3/d23/d7c/cbf 0 2024-04-27T04:10:13.587 INFO:tasks.workunit.client.1.smithi181.stdout:7/756: dwrite d1/d2/d35/d67/dbf/fc9 [0,4194304] 0 2024-04-27T04:10:13.593 INFO:tasks.workunit.client.1.smithi181.stdout:2/835: dwrite d4/d9/d16/d45/f87 [0,4194304] 0 2024-04-27T04:10:13.595 INFO:tasks.workunit.client.1.smithi181.stdout:5/950: mkdir d4/d10/d29/d122/d155 0 2024-04-27T04:10:13.596 INFO:tasks.workunit.client.0.smithi162.stdout:0/709: creat d5/dc/d1e/d20/dba/feb x:0 0 0 2024-04-27T04:10:13.597 INFO:tasks.workunit.client.0.smithi162.stdout:5/652: link d8/d1a/d29/laf d8/d1a/d29/d8b/ld3 0 2024-04-27T04:10:13.598 INFO:tasks.workunit.client.0.smithi162.stdout:5/653: chown d8/d9/d5b/fb5 488985 1 2024-04-27T04:10:13.602 INFO:tasks.workunit.client.0.smithi162.stdout:4/576: creat d2/d25/d1c/d39/d3b/d47/d9c/fb4 x:0 0 0 2024-04-27T04:10:13.604 INFO:tasks.workunit.client.1.smithi181.stdout:3/749: dwrite d0/d6c/f79 [0,4194304] 0 2024-04-27T04:10:13.605 INFO:tasks.workunit.client.0.smithi162.stdout:3/588: rename d1 to d1/d18/d23/d58/dc4 22 2024-04-27T04:10:13.613 INFO:tasks.workunit.client.0.smithi162.stdout:2/557: creat d0/d4/d17/d2b/fa9 x:0 0 0 2024-04-27T04:10:13.614 INFO:tasks.workunit.client.1.smithi181.stdout:5/951: rmdir d4/de/d2f 39 2024-04-27T04:10:13.614 INFO:tasks.workunit.client.0.smithi162.stdout:2/558: chown d0/d4/d2a/d3b/d49/d5a/d63/d80 19351 1 2024-04-27T04:10:13.615 INFO:tasks.workunit.client.0.smithi162.stdout:2/559: readlink d0/d4/d2a/l58 0 2024-04-27T04:10:13.616 INFO:tasks.workunit.client.1.smithi181.stdout:1/754: dwrite d1/f2f [0,4194304] 0 2024-04-27T04:10:13.619 INFO:tasks.workunit.client.0.smithi162.stdout:5/654: dread d8/d9/f48 [0,4194304] 0 2024-04-27T04:10:13.621 INFO:tasks.workunit.client.0.smithi162.stdout:5/655: chown d8/d9/d5b/d96/fc2 22412 1 2024-04-27T04:10:13.624 INFO:tasks.workunit.client.0.smithi162.stdout:7/622: dwrite d3/d5/d5b/d83/f33 [0,4194304] 0 2024-04-27T04:10:13.625 INFO:tasks.workunit.client.1.smithi181.stdout:3/750: unlink d0/d6c/d52/d44/dd5/ca5 0 2024-04-27T04:10:13.626 INFO:tasks.workunit.client.1.smithi181.stdout:0/909: dwrite dc/f94 [0,4194304] 0 2024-04-27T04:10:13.628 INFO:tasks.workunit.client.0.smithi162.stdout:7/623: fsync d3/d5/d3c/d4a/fa4 0 2024-04-27T04:10:13.628 INFO:tasks.workunit.client.0.smithi162.stdout:8/643: creat d0/de/d20/d7e/fd1 x:0 0 0 2024-04-27T04:10:13.629 INFO:tasks.workunit.client.1.smithi181.stdout:0/910: write dc/d1f/d3b/df7/d127/f130 [1459,58150] 0 2024-04-27T04:10:13.630 INFO:tasks.workunit.client.0.smithi162.stdout:5/656: dread d8/d1a/d7d/fc5 [0,4194304] 0 2024-04-27T04:10:13.630 INFO:tasks.workunit.client.0.smithi162.stdout:9/621: dread d3/d11/d45/d4e/d58/f68 [0,4194304] 0 2024-04-27T04:10:13.637 INFO:tasks.workunit.client.1.smithi181.stdout:4/716: dwrite d0/d15/de/d19/fab [0,4194304] 0 2024-04-27T04:10:13.638 INFO:tasks.workunit.client.1.smithi181.stdout:5/952: mkdir d4/de/d2c/d46/d4d/d8e/d54/d156 0 2024-04-27T04:10:13.640 INFO:tasks.workunit.client.0.smithi162.stdout:8/644: dwrite d0/d10/d76/fb7 [0,4194304] 0 2024-04-27T04:10:13.641 INFO:tasks.workunit.client.0.smithi162.stdout:9/622: dwrite d3/d11/d45/d4e/fb1 [0,4194304] 0 2024-04-27T04:10:13.642 INFO:tasks.workunit.client.0.smithi162.stdout:9/623: dread - d3/d11/d45/d4e/fbe zero size 2024-04-27T04:10:13.646 INFO:tasks.workunit.client.1.smithi181.stdout:2/836: write d4/d9/d4f/d79/fce [1281762,76755] 0 2024-04-27T04:10:13.648 INFO:tasks.workunit.client.0.smithi162.stdout:9/624: dread d3/d13/d15/f19 [4194304,4194304] 0 2024-04-27T04:10:13.652 INFO:tasks.workunit.client.1.smithi181.stdout:1/755: chown d1/dad/db5/d6/da3/d1e/d48/da6/da4/ff0 2 1 2024-04-27T04:10:13.654 INFO:tasks.workunit.client.0.smithi162.stdout:9/625: dread d3/d13/d15/f19 [0,4194304] 0 2024-04-27T04:10:13.657 INFO:tasks.workunit.client.1.smithi181.stdout:3/751: write d0/d6c/d52/d44/dd5/d58/de4/daf/fc3 [2035959,56938] 0 2024-04-27T04:10:13.657 INFO:tasks.workunit.client.0.smithi162.stdout:5/657: mknod d8/d9/db3/cd4 0 2024-04-27T04:10:13.659 INFO:tasks.workunit.client.1.smithi181.stdout:7/757: dwrite d1/d2/d1c/d2d/d61/f68 [0,4194304] 0 2024-04-27T04:10:13.659 INFO:tasks.workunit.client.1.smithi181.stdout:0/911: dwrite dc/fd [0,4194304] 0 2024-04-27T04:10:13.659 INFO:tasks.workunit.client.1.smithi181.stdout:5/953: chown d4/de/d2c/d46/d4d/f94 5410 1 2024-04-27T04:10:13.664 INFO:tasks.workunit.client.1.smithi181.stdout:2/837: read d4/d9/f64 [243753,129274] 0 2024-04-27T04:10:13.668 INFO:tasks.workunit.client.0.smithi162.stdout:6/607: creat d6/d1a/d34/d3a/fbe x:0 0 0 2024-04-27T04:10:13.668 INFO:tasks.workunit.client.0.smithi162.stdout:8/645: chown d0/d10/d25/c58 522277 1 2024-04-27T04:10:13.669 INFO:tasks.workunit.client.1.smithi181.stdout:1/756: truncate d1/dad/db5/d6/da3/d1e/d48/da6/fe3 154822 0 2024-04-27T04:10:13.670 INFO:tasks.workunit.client.0.smithi162.stdout:8/646: write d0/d10/d2b/d2d/d2f/d39/f59 [1040435,91389] 0 2024-04-27T04:10:13.671 INFO:tasks.workunit.client.0.smithi162.stdout:8/647: readlink d0/de/d20/lc1 0 2024-04-27T04:10:13.673 INFO:tasks.workunit.client.0.smithi162.stdout:2/560: mkdir d0/d4/d17/d2b/d64/d96/daa 0 2024-04-27T04:10:13.673 INFO:tasks.workunit.client.0.smithi162.stdout:1/585: creat d0/d11/d50/fc3 x:0 0 0 2024-04-27T04:10:13.675 INFO:tasks.workunit.client.0.smithi162.stdout:2/561: stat d0/d4/d2a/da1/fa5 0 2024-04-27T04:10:13.677 INFO:tasks.workunit.client.0.smithi162.stdout:6/608: dwrite d6/d1a/d34/d3a/d60/d74/fa1 [0,4194304] 0 2024-04-27T04:10:13.678 INFO:tasks.workunit.client.1.smithi181.stdout:2/838: fsync d4/d9/df/d14/da7/f108 0 2024-04-27T04:10:13.679 INFO:tasks.workunit.client.0.smithi162.stdout:6/609: write f2 [669026,103510] 0 2024-04-27T04:10:13.680 INFO:tasks.workunit.client.0.smithi162.stdout:9/626: rmdir d3/d13/d2d/d3b/d72 39 2024-04-27T04:10:13.682 INFO:tasks.workunit.client.1.smithi181.stdout:7/758: unlink d1/d22/l9e 0 2024-04-27T04:10:13.683 INFO:tasks.workunit.client.1.smithi181.stdout:3/752: dread d0/f37 [0,4194304] 0 2024-04-27T04:10:13.684 INFO:tasks.workunit.client.0.smithi162.stdout:4/577: rmdir d2/d25/d1c/d39/da9 0 2024-04-27T04:10:13.686 INFO:tasks.workunit.client.1.smithi181.stdout:1/757: rmdir d1/dad/db5/d6/db/d2d/d36 39 2024-04-27T04:10:13.686 INFO:tasks.workunit.client.1.smithi181.stdout:4/717: dread d0/d15/d25/d2b/d6d/d82/fcc [0,4194304] 0 2024-04-27T04:10:13.689 INFO:tasks.workunit.client.1.smithi181.stdout:4/718: write d0/d15/d25/d2b/f2c [1237873,116591] 0 2024-04-27T04:10:13.689 INFO:tasks.workunit.client.0.smithi162.stdout:4/578: dread d2/d25/d1c/f3d [0,4194304] 0 2024-04-27T04:10:13.690 INFO:tasks.workunit.client.1.smithi181.stdout:4/719: fdatasync d0/d15/d5c/fb2 0 2024-04-27T04:10:13.690 INFO:tasks.workunit.client.0.smithi162.stdout:7/624: mknod d3/d5/d5b/d83/de/d2b/d81/d96/dc8/cd1 0 2024-04-27T04:10:13.694 INFO:tasks.workunit.client.1.smithi181.stdout:0/912: rename dc/d1f/d3b/d4a/d95/d26/d41/d6f/c67 to dc/d1f/d3b/d4a/d95/d26/d131/dbd/c13b 0 2024-04-27T04:10:13.695 INFO:tasks.workunit.client.0.smithi162.stdout:3/589: mkdir d1/d18/d23/d34/dad/dc5 0 2024-04-27T04:10:13.696 INFO:tasks.workunit.client.0.smithi162.stdout:3/590: fdatasync d1/d45/fbb 0 2024-04-27T04:10:13.699 INFO:tasks.workunit.client.1.smithi181.stdout:2/839: rmdir d4/d9/d4f/d5f 39 2024-04-27T04:10:13.700 INFO:tasks.workunit.client.1.smithi181.stdout:0/913: dread dc/f5e [0,4194304] 0 2024-04-27T04:10:13.702 INFO:tasks.workunit.client.1.smithi181.stdout:7/759: chown d1/d2/d1c/d37/f4f 113 1 2024-04-27T04:10:13.703 INFO:tasks.workunit.client.1.smithi181.stdout:7/760: write d1/df/d62/fd0 [1676531,51316] 0 2024-04-27T04:10:13.703 INFO:tasks.workunit.client.0.smithi162.stdout:0/710: getdents d5/dc/d1e/d56/d57 0 2024-04-27T04:10:13.709 INFO:tasks.workunit.client.1.smithi181.stdout:1/758: truncate d1/dad/db5/d6/db/d2d/d36/fe4 954914 0 2024-04-27T04:10:13.710 INFO:tasks.workunit.client.1.smithi181.stdout:5/954: dwrite d4/d10/f59 [4194304,4194304] 0 2024-04-27T04:10:13.710 INFO:tasks.workunit.client.0.smithi162.stdout:5/658: write d8/d1a/d23/d40/f42 [1607472,6158] 0 2024-04-27T04:10:13.711 INFO:tasks.workunit.client.1.smithi181.stdout:1/759: chown d1/dad/db5/d6/da3/d1e/d20/d53/d56/f6f 0 1 2024-04-27T04:10:13.712 INFO:tasks.workunit.client.1.smithi181.stdout:1/760: fsync d1/f77 0 2024-04-27T04:10:13.717 INFO:tasks.workunit.client.1.smithi181.stdout:6/647: sync 2024-04-27T04:10:13.717 INFO:tasks.workunit.client.1.smithi181.stdout:8/814: sync 2024-04-27T04:10:13.717 INFO:tasks.workunit.client.1.smithi181.stdout:9/725: sync 2024-04-27T04:10:13.719 INFO:tasks.workunit.client.1.smithi181.stdout:8/815: fdatasync d1/d6/d2f/d30/d5c/f97 0 2024-04-27T04:10:13.721 INFO:tasks.workunit.client.1.smithi181.stdout:2/840: chown d4/d9/d35/l80 1101270565 1 2024-04-27T04:10:13.723 INFO:tasks.workunit.client.0.smithi162.stdout:6/610: write d6/d1a/f15 [711526,76660] 0 2024-04-27T04:10:13.723 INFO:tasks.workunit.client.1.smithi181.stdout:4/720: write d0/d15/de/f66 [161471,15410] 0 2024-04-27T04:10:13.725 INFO:tasks.workunit.client.0.smithi162.stdout:8/648: dwrite d0/d10/d2b/d2d/d2f/d39/f62 [0,4194304] 0 2024-04-27T04:10:13.725 INFO:tasks.workunit.client.0.smithi162.stdout:3/591: mknod d1/d70/cc6 0 2024-04-27T04:10:13.729 INFO:tasks.workunit.client.0.smithi162.stdout:1/586: mkdir d0/d7/d3c/d61/dc4 0 2024-04-27T04:10:13.732 INFO:tasks.workunit.client.1.smithi181.stdout:7/761: creat d1/d22/d15/dc6/dd8/fe4 x:0 0 0 2024-04-27T04:10:13.735 INFO:tasks.workunit.client.1.smithi181.stdout:6/648: mknod d7/dc/dc3/d72/daa/ce0 0 2024-04-27T04:10:13.740 INFO:tasks.workunit.client.1.smithi181.stdout:8/816: creat d1/d6/d2f/d11e/f121 x:0 0 0 2024-04-27T04:10:13.741 INFO:tasks.workunit.client.0.smithi162.stdout:4/579: mkdir d2/d25/d1c/d24/db5 0 2024-04-27T04:10:13.742 INFO:tasks.workunit.client.0.smithi162.stdout:2/562: truncate d0/d4/d2a/f9e 3877904 0 2024-04-27T04:10:13.743 INFO:tasks.workunit.client.1.smithi181.stdout:2/841: fdatasync d4/d9/d16/d45/d9a/fc6 0 2024-04-27T04:10:13.746 INFO:tasks.workunit.client.1.smithi181.stdout:0/914: truncate dc/f11 8231795 0 2024-04-27T04:10:13.749 INFO:tasks.workunit.client.0.smithi162.stdout:7/625: mknod d3/d39/cd2 0 2024-04-27T04:10:13.750 INFO:tasks.workunit.client.1.smithi181.stdout:3/753: creat d0/d6c/d52/d44/fed x:0 0 0 2024-04-27T04:10:13.750 INFO:tasks.workunit.client.1.smithi181.stdout:3/754: chown d0/d9f/c85 28887 1 2024-04-27T04:10:13.751 INFO:tasks.workunit.client.1.smithi181.stdout:9/726: dread d0/fe8 [4194304,4194304] 0 2024-04-27T04:10:13.751 INFO:tasks.workunit.client.0.smithi162.stdout:8/649: creat d0/d10/d2b/d2e/dca/fd2 x:0 0 0 2024-04-27T04:10:13.752 INFO:tasks.workunit.client.1.smithi181.stdout:1/761: creat d1/dad/db5/d6/da3/d1e/d20/d53/d56/de2/ff9 x:0 0 0 2024-04-27T04:10:13.753 INFO:tasks.workunit.client.1.smithi181.stdout:1/762: readlink d1/dad/db5/d6/db/l43 0 2024-04-27T04:10:13.756 INFO:tasks.workunit.client.1.smithi181.stdout:4/721: fsync d0/d15/de/f34 0 2024-04-27T04:10:13.760 INFO:tasks.workunit.client.1.smithi181.stdout:8/817: symlink d1/d6/d2f/d30/d66/dc0/l122 0 2024-04-27T04:10:13.765 INFO:tasks.workunit.client.0.smithi162.stdout:2/563: unlink d0/d4/d2a/d41/f54 0 2024-04-27T04:10:13.766 INFO:tasks.workunit.client.0.smithi162.stdout:2/564: chown d0/d4/f16 7694 1 2024-04-27T04:10:13.767 INFO:tasks.workunit.client.0.smithi162.stdout:6/611: mkdir d6/d1a/d34/d3a/dbf 0 2024-04-27T04:10:13.767 INFO:tasks.workunit.client.1.smithi181.stdout:0/915: rmdir dc/d1f/d3b/df7 39 2024-04-27T04:10:13.770 INFO:tasks.workunit.client.0.smithi162.stdout:8/650: mkdir d0/d10/d2b/d2d/d2f/d6b/dd3 0 2024-04-27T04:10:13.772 INFO:tasks.workunit.client.0.smithi162.stdout:6/612: dwrite d6/f35 [0,4194304] 0 2024-04-27T04:10:13.772 INFO:tasks.workunit.client.1.smithi181.stdout:9/727: rename d0/d44/d4b/d5f/da4 to d0/d44/d4f/df7 0 2024-04-27T04:10:13.778 INFO:tasks.workunit.client.0.smithi162.stdout:9/627: creat d3/d13/d15/d7e/fc0 x:0 0 0 2024-04-27T04:10:13.779 INFO:tasks.workunit.client.1.smithi181.stdout:7/762: dwrite d1/d22/d15/dc6/dd8/f66 [0,4194304] 0 2024-04-27T04:10:13.780 INFO:tasks.workunit.client.1.smithi181.stdout:6/649: creat d7/dc/d1e/d2b/dde/fe1 x:0 0 0 2024-04-27T04:10:13.782 INFO:tasks.workunit.client.0.smithi162.stdout:4/580: creat d2/d25/d1c/d39/d3b/d6e/d6f/fb6 x:0 0 0 2024-04-27T04:10:13.784 INFO:tasks.workunit.client.0.smithi162.stdout:2/565: symlink d0/d4/d17/d4f/d7f/lab 0 2024-04-27T04:10:13.785 INFO:tasks.workunit.client.0.smithi162.stdout:2/566: dread - d0/d4/d2a/d3b/d49/d5a/d63/d6e/f98 zero size 2024-04-27T04:10:13.785 INFO:tasks.workunit.client.1.smithi181.stdout:8/818: fsync d1/d6/d2f/d30/d66/fe5 0 2024-04-27T04:10:13.786 INFO:tasks.workunit.client.0.smithi162.stdout:2/567: write d0/d4/d2a/d41/fa2 [443416,11933] 0 2024-04-27T04:10:13.789 INFO:tasks.workunit.client.1.smithi181.stdout:0/916: mkdir dc/d1f/d115/d13c 0 2024-04-27T04:10:13.789 INFO:tasks.workunit.client.0.smithi162.stdout:2/568: write d0/d4/d2a/d41/fa2 [522946,65427] 0 2024-04-27T04:10:13.790 INFO:tasks.workunit.client.0.smithi162.stdout:2/569: stat d0/d20/l9a 0 2024-04-27T04:10:13.790 INFO:tasks.workunit.client.1.smithi181.stdout:3/755: link d0/f14 d0/d6c/d52/d44/dd5/fee 0 2024-04-27T04:10:13.792 INFO:tasks.workunit.client.1.smithi181.stdout:9/728: creat d0/d44/d4b/d3d/ff8 x:0 0 0 2024-04-27T04:10:13.792 INFO:tasks.workunit.client.0.smithi162.stdout:3/592: dwrite d1/d18/d5d/d5f/f77 [0,4194304] 0 2024-04-27T04:10:13.794 INFO:tasks.workunit.client.1.smithi181.stdout:1/763: mkdir d1/dad/db5/d6/da3/d1e/d48/dfa 0 2024-04-27T04:10:13.794 INFO:tasks.workunit.client.0.smithi162.stdout:3/593: write d1/f4c [2603612,106958] 0 2024-04-27T04:10:13.796 INFO:tasks.workunit.client.0.smithi162.stdout:0/711: getdents d5/d26 0 2024-04-27T04:10:13.798 INFO:tasks.workunit.client.0.smithi162.stdout:9/628: symlink d3/d13/d2d/d30/d67/lc1 0 2024-04-27T04:10:13.799 INFO:tasks.workunit.client.0.smithi162.stdout:9/629: chown d3/d13/d2d/d3b/l86 107501667 1 2024-04-27T04:10:13.799 INFO:tasks.workunit.client.0.smithi162.stdout:4/581: mknod d2/d25/d1c/d39/d70/d91/cb7 0 2024-04-27T04:10:13.801 INFO:tasks.workunit.client.1.smithi181.stdout:7/763: mkdir d1/d22/d15/dc6/de5 0 2024-04-27T04:10:13.803 INFO:tasks.workunit.client.1.smithi181.stdout:7/764: write d1/d22/d15/d29/f84 [1238810,22091] 0 2024-04-27T04:10:13.803 INFO:tasks.workunit.client.0.smithi162.stdout:7/626: creat d3/d5/d5b/d60/fd3 x:0 0 0 2024-04-27T04:10:13.804 INFO:tasks.workunit.client.0.smithi162.stdout:7/627: dread - d3/d4f/fb1 zero size 2024-04-27T04:10:13.806 INFO:tasks.workunit.client.1.smithi181.stdout:6/650: rename d7/dc/d1c/d88/d6d/cc8 to d7/dc/dc3/d72/ce2 0 2024-04-27T04:10:13.809 INFO:tasks.workunit.client.1.smithi181.stdout:8/819: dread - d1/d6/d2f/d47/dba/fd7 zero size 2024-04-27T04:10:13.809 INFO:tasks.workunit.client.0.smithi162.stdout:7/628: dwrite d3/f18 [0,4194304] 0 2024-04-27T04:10:13.812 INFO:tasks.workunit.client.0.smithi162.stdout:2/570: rename d0/d24/f4b to d0/d4/d17/d2b/d64/d96/fac 0 2024-04-27T04:10:13.819 INFO:tasks.workunit.client.0.smithi162.stdout:3/594: creat d1/d18/d5d/d5f/dbe/fc7 x:0 0 0 2024-04-27T04:10:13.820 INFO:tasks.workunit.client.0.smithi162.stdout:0/712: dread d5/dc/d1e/d41/f53 [0,4194304] 0 2024-04-27T04:10:13.820 INFO:tasks.workunit.client.1.smithi181.stdout:9/729: creat d0/d44/d4b/d43/d89/daf/ff9 x:0 0 0 2024-04-27T04:10:13.820 INFO:tasks.workunit.client.0.smithi162.stdout:6/613: dwrite d6/d1a/f20 [0,4194304] 0 2024-04-27T04:10:13.821 INFO:tasks.workunit.client.0.smithi162.stdout:5/659: sync 2024-04-27T04:10:13.821 INFO:tasks.workunit.client.0.smithi162.stdout:1/587: sync 2024-04-27T04:10:13.821 INFO:tasks.workunit.client.0.smithi162.stdout:9/630: write d3/d13/d15/f19 [8797675,107551] 0 2024-04-27T04:10:13.822 INFO:tasks.workunit.client.1.smithi181.stdout:9/730: write d0/d44/d4f/d9d/fe7 [278125,34287] 0 2024-04-27T04:10:13.827 INFO:tasks.workunit.client.0.smithi162.stdout:0/713: dread d5/dc/d1e/d41/f53 [0,4194304] 0 2024-04-27T04:10:13.828 INFO:tasks.workunit.client.1.smithi181.stdout:7/765: unlink d1/d22/d15/d5f/dba/lca 0 2024-04-27T04:10:13.829 INFO:tasks.workunit.client.0.smithi162.stdout:0/714: dread - d5/dc/d1e/d56/d57/fda zero size 2024-04-27T04:10:13.832 INFO:tasks.workunit.client.1.smithi181.stdout:3/756: dread d0/f32 [0,4194304] 0 2024-04-27T04:10:13.833 INFO:tasks.workunit.client.1.smithi181.stdout:3/757: write d0/d6c/d66/dc2/fe3 [203473,115979] 0 2024-04-27T04:10:13.839 INFO:tasks.workunit.client.0.smithi162.stdout:3/595: symlink d1/d18/d5d/d5f/d66/lc8 0 2024-04-27T04:10:13.839 INFO:tasks.workunit.client.0.smithi162.stdout:2/571: dread d0/d4/d2a/d3b/d49/d5a/d63/d84/d93/f76 [0,4194304] 0 2024-04-27T04:10:13.840 INFO:tasks.workunit.client.1.smithi181.stdout:0/917: write dc/d1f/d3b/d4a/d95/d26/d131/dba/deb/db3/fb8 [515914,41066] 0 2024-04-27T04:10:13.840 INFO:tasks.workunit.client.0.smithi162.stdout:9/631: fdatasync d3/fc 0 2024-04-27T04:10:13.841 INFO:tasks.workunit.client.1.smithi181.stdout:1/764: mknod d1/dad/db5/d6/db/df2/cfb 0 2024-04-27T04:10:13.843 INFO:tasks.workunit.client.0.smithi162.stdout:1/588: mkdir d0/d11/d3e/d59/d7f/dc5 0 2024-04-27T04:10:13.845 INFO:tasks.workunit.client.0.smithi162.stdout:5/660: symlink d8/d9/d55/ld5 0 2024-04-27T04:10:13.845 INFO:tasks.workunit.client.0.smithi162.stdout:5/661: chown d8/d9/d5b/d96 88266 1 2024-04-27T04:10:13.848 INFO:tasks.workunit.client.0.smithi162.stdout:0/715: mkdir d5/dc/d1e/d20/dec 0 2024-04-27T04:10:13.848 INFO:tasks.workunit.client.0.smithi162.stdout:0/716: fsync d5/dc/d1e/f6d 0 2024-04-27T04:10:13.849 INFO:tasks.workunit.client.0.smithi162.stdout:0/717: write d5/d60/d8b/fe1 [852819,75188] 0 2024-04-27T04:10:13.849 INFO:tasks.workunit.client.1.smithi181.stdout:7/766: creat d1/d2/d35/d67/fe6 x:0 0 0 2024-04-27T04:10:13.850 INFO:tasks.workunit.client.0.smithi162.stdout:8/651: link d0/d10/c64 d0/d10/d25/d7d/cd4 0 2024-04-27T04:10:13.852 INFO:tasks.workunit.client.1.smithi181.stdout:6/651: link d7/dc/d1c/d7f/d3e/fc0 d7/d27/fe3 0 2024-04-27T04:10:13.852 INFO:tasks.workunit.client.0.smithi162.stdout:7/629: write d3/d4f/f41 [4887907,55530] 0 2024-04-27T04:10:13.853 INFO:tasks.workunit.client.1.smithi181.stdout:6/652: readlink d7/dc/d1c/d88/db3/lc5 0 2024-04-27T04:10:13.853 INFO:tasks.workunit.client.0.smithi162.stdout:7/630: stat d3/d4f/f31 0 2024-04-27T04:10:13.853 INFO:tasks.workunit.client.0.smithi162.stdout:4/582: dread d2/d25/d1c/d24/f69 [0,4194304] 0 2024-04-27T04:10:13.854 INFO:tasks.workunit.client.0.smithi162.stdout:7/631: dread - d3/d5/d5b/d83/de/fbb zero size 2024-04-27T04:10:13.860 INFO:tasks.workunit.client.1.smithi181.stdout:5/955: sync 2024-04-27T04:10:13.861 INFO:tasks.workunit.client.1.smithi181.stdout:0/918: read dc/d1f/d3b/f44 [4740857,119550] 0 2024-04-27T04:10:13.861 INFO:tasks.workunit.client.1.smithi181.stdout:9/731: dread d0/d40/fb2 [0,4194304] 0 2024-04-27T04:10:13.862 INFO:tasks.workunit.client.1.smithi181.stdout:7/767: unlink d1/ld 0 2024-04-27T04:10:13.863 INFO:tasks.workunit.client.1.smithi181.stdout:7/768: truncate d1/fc8 32916 0 2024-04-27T04:10:13.867 INFO:tasks.workunit.client.1.smithi181.stdout:8/820: creat d1/d6/d1a/f123 x:0 0 0 2024-04-27T04:10:13.869 INFO:tasks.workunit.client.1.smithi181.stdout:6/653: rename d7/d27/fcd to d7/dc/d1c/d88/db3/fe4 0 2024-04-27T04:10:13.873 INFO:tasks.workunit.client.0.smithi162.stdout:0/718: truncate d5/dc/d1e/d56/da8/fe9 4400076 0 2024-04-27T04:10:13.875 INFO:tasks.workunit.client.1.smithi181.stdout:1/765: write d1/dad/db5/d6/db/d2d/d36/f5e [630770,80670] 0 2024-04-27T04:10:13.878 INFO:tasks.workunit.client.0.smithi162.stdout:1/589: dread d0/f34 [0,4194304] 0 2024-04-27T04:10:13.880 INFO:tasks.workunit.client.1.smithi181.stdout:7/769: dread d1/d2/d1c/d37/f4f [0,4194304] 0 2024-04-27T04:10:13.881 INFO:tasks.workunit.client.1.smithi181.stdout:3/758: creat d0/d6c/d52/d44/d56/d5b/fef x:0 0 0 2024-04-27T04:10:13.882 INFO:tasks.workunit.client.0.smithi162.stdout:9/632: dwrite d3/d11/d45/d4e/d58/f80 [0,4194304] 0 2024-04-27T04:10:13.885 INFO:tasks.workunit.client.0.smithi162.stdout:1/590: dwrite d0/d7/d3c/f43 [0,4194304] 0 2024-04-27T04:10:13.886 INFO:tasks.workunit.client.0.smithi162.stdout:7/632: creat d3/d5/d5b/d83/de/d2b/d81/d96/dc8/fd4 x:0 0 0 2024-04-27T04:10:13.887 INFO:tasks.workunit.client.1.smithi181.stdout:4/722: sync 2024-04-27T04:10:13.887 INFO:tasks.workunit.client.1.smithi181.stdout:2/842: sync 2024-04-27T04:10:13.889 INFO:tasks.workunit.client.1.smithi181.stdout:5/956: mkdir d4/de/d2c/d46/d157 0 2024-04-27T04:10:13.893 INFO:tasks.workunit.client.0.smithi162.stdout:4/583: dwrite d2/d25/d1c/d39/d3b/f94 [0,4194304] 0 2024-04-27T04:10:13.893 INFO:tasks.workunit.client.1.smithi181.stdout:1/766: mkdir d1/dad/db5/d6/da3/d1e/d48/da6/da4/dfc 0 2024-04-27T04:10:13.895 INFO:tasks.workunit.client.1.smithi181.stdout:0/919: dread dc/f11 [4194304,4194304] 0 2024-04-27T04:10:13.899 INFO:tasks.workunit.client.0.smithi162.stdout:2/572: dwrite d0/d4/d2a/f9e [4194304,4194304] 0 2024-04-27T04:10:13.902 INFO:tasks.workunit.client.0.smithi162.stdout:0/719: mknod d5/d26/d4a/ced 0 2024-04-27T04:10:13.904 INFO:tasks.workunit.client.1.smithi181.stdout:1/767: dwrite d1/dad/db5/d6/da3/d1e/d20/d53/d56/de2/ff9 [0,4194304] 0 2024-04-27T04:10:13.905 INFO:tasks.workunit.client.1.smithi181.stdout:1/768: chown d1/dad/db5/cc1 0 1 2024-04-27T04:10:13.911 INFO:tasks.workunit.client.1.smithi181.stdout:2/843: symlink d4/d9/d35/d42/d86/l10c 0 2024-04-27T04:10:13.913 INFO:tasks.workunit.client.0.smithi162.stdout:9/633: mknod d3/d13/d2d/d3b/d69/cc2 0 2024-04-27T04:10:13.913 INFO:tasks.workunit.client.1.smithi181.stdout:2/844: chown d4/d9/d4f/d79/d98/ld2 3567 1 2024-04-27T04:10:13.915 INFO:tasks.workunit.client.0.smithi162.stdout:1/591: symlink d0/d11/d50/d53/lc6 0 2024-04-27T04:10:13.916 INFO:tasks.workunit.client.1.smithi181.stdout:9/732: mkdir d0/d44/d4b/d3d/dbd/df3/d85/dd8/dfa 0 2024-04-27T04:10:13.917 INFO:tasks.workunit.client.1.smithi181.stdout:7/770: dread d1/d2/d1c/d37/d91/fb0 [0,4194304] 0 2024-04-27T04:10:13.918 INFO:tasks.workunit.client.1.smithi181.stdout:7/771: write d1/df/fdf [5078844,99701] 0 2024-04-27T04:10:13.921 INFO:tasks.workunit.client.0.smithi162.stdout:6/614: getdents d6/d1a/d34/d50 0 2024-04-27T04:10:13.921 INFO:tasks.workunit.client.0.smithi162.stdout:6/615: write d6/d1a/d34/fb6 [698901,122416] 0 2024-04-27T04:10:13.922 INFO:tasks.workunit.client.0.smithi162.stdout:6/616: stat d6/d1a/d34/d50/f99 0 2024-04-27T04:10:13.924 INFO:tasks.workunit.client.1.smithi181.stdout:8/821: creat d1/d6/f124 x:0 0 0 2024-04-27T04:10:13.926 INFO:tasks.workunit.client.0.smithi162.stdout:5/662: link d8/d9/c1c d8/d2c/cd6 0 2024-04-27T04:10:13.926 INFO:tasks.workunit.client.0.smithi162.stdout:6/617: dread d6/d1a/fa6 [0,4194304] 0 2024-04-27T04:10:13.926 INFO:tasks.workunit.client.1.smithi181.stdout:9/733: dwrite d0/d44/d4f/df7/fd4 [0,4194304] 0 2024-04-27T04:10:13.927 INFO:tasks.workunit.client.1.smithi181.stdout:2/845: mknod d4/d9/d16/d45/c10d 0 2024-04-27T04:10:13.928 INFO:tasks.workunit.client.1.smithi181.stdout:8/822: read d1/d6/d2f/d30/fb5 [2603337,127264] 0 2024-04-27T04:10:13.930 INFO:tasks.workunit.client.1.smithi181.stdout:0/920: creat dc/d1f/d3b/db2/f13d x:0 0 0 2024-04-27T04:10:13.933 INFO:tasks.workunit.client.0.smithi162.stdout:9/634: rmdir d3/d11/d45/d4e/d58 39 2024-04-27T04:10:13.936 INFO:tasks.workunit.client.0.smithi162.stdout:3/596: rename d1/d18/d23/d34/daa/c75 to d1/d18/d23/cc9 0 2024-04-27T04:10:13.938 INFO:tasks.workunit.client.0.smithi162.stdout:7/633: creat d3/d5/d5b/d83/de/d2b/d81/daa/dbd/db6/fd5 x:0 0 0 2024-04-27T04:10:13.942 INFO:tasks.workunit.client.0.smithi162.stdout:8/652: getdents d0/d10/d2b/d2e/dca 0 2024-04-27T04:10:13.944 INFO:tasks.workunit.client.0.smithi162.stdout:9/635: mkdir d3/d13/d2d/d30/d7f/dc3 0 2024-04-27T04:10:13.945 INFO:tasks.workunit.client.0.smithi162.stdout:7/634: dwrite d3/d5/d5b/d60/fd3 [0,4194304] 0 2024-04-27T04:10:13.946 INFO:tasks.workunit.client.0.smithi162.stdout:7/635: write d3/d4f/f9d [96884,102571] 0 2024-04-27T04:10:13.947 INFO:tasks.workunit.client.1.smithi181.stdout:1/769: creat d1/dad/db5/d6/db/d2d/ffd x:0 0 0 2024-04-27T04:10:13.947 INFO:tasks.workunit.client.0.smithi162.stdout:7/636: write d3/d5/d5b/d83/f19 [3228536,46478] 0 2024-04-27T04:10:13.948 INFO:tasks.workunit.client.1.smithi181.stdout:3/759: dread d0/f23 [0,4194304] 0 2024-04-27T04:10:13.948 INFO:tasks.workunit.client.0.smithi162.stdout:7/637: chown d3/d5/d3c/d4a/d55/dc6/fcd 8 1 2024-04-27T04:10:13.949 INFO:tasks.workunit.client.0.smithi162.stdout:7/638: stat d3/d4f/f78 0 2024-04-27T04:10:13.949 INFO:tasks.workunit.client.1.smithi181.stdout:4/723: getdents d0/d15 0 2024-04-27T04:10:13.950 INFO:tasks.workunit.client.0.smithi162.stdout:3/597: creat d1/d18/d5d/d5f/fca x:0 0 0 2024-04-27T04:10:13.951 INFO:tasks.workunit.client.1.smithi181.stdout:9/734: fsync d0/fc 0 2024-04-27T04:10:13.953 INFO:tasks.workunit.client.1.smithi181.stdout:9/735: readlink d0/d44/d4b/d43/d8f/lb6 0 2024-04-27T04:10:13.955 INFO:tasks.workunit.client.1.smithi181.stdout:7/772: dread d1/d57/f77 [0,4194304] 0 2024-04-27T04:10:13.958 INFO:tasks.workunit.client.1.smithi181.stdout:5/957: dread d4/de/d2c/d46/d4d/f66 [0,4194304] 0 2024-04-27T04:10:13.960 INFO:tasks.workunit.client.1.smithi181.stdout:1/770: truncate d1/d2/d89/fab 526530 0 2024-04-27T04:10:13.969 INFO:tasks.workunit.client.1.smithi181.stdout:2/846: rename d4/d9/d4f/d79/d94/lc4 to d4/d9/d16/d82/d7d/l10e 0 2024-04-27T04:10:13.971 INFO:tasks.workunit.client.1.smithi181.stdout:2/847: truncate d4/d9/d16/d45/d9a/fe1 538926 0 2024-04-27T04:10:13.976 INFO:tasks.workunit.client.0.smithi162.stdout:8/653: dread d0/de/d20/d6c/fb6 [0,4194304] 0 2024-04-27T04:10:13.979 INFO:tasks.workunit.client.1.smithi181.stdout:2/848: dwrite d4/d9/d4f/d79/fec [0,4194304] 0 2024-04-27T04:10:13.981 INFO:tasks.workunit.client.0.smithi162.stdout:4/584: rename d2/d25/d1c/d24/f69 to d2/da/d1a/fb8 0 2024-04-27T04:10:13.982 INFO:tasks.workunit.client.1.smithi181.stdout:6/654: truncate d7/dc/f12 3628235 0 2024-04-27T04:10:13.982 INFO:tasks.workunit.client.0.smithi162.stdout:2/573: write d0/f6 [1174169,114604] 0 2024-04-27T04:10:13.984 INFO:tasks.workunit.client.0.smithi162.stdout:0/720: write d5/f11 [2813868,19280] 0 2024-04-27T04:10:13.985 INFO:tasks.workunit.client.0.smithi162.stdout:3/598: link d1/d70/fa5 d1/d18/d23/d34/dad/dc5/fcb 0 2024-04-27T04:10:13.988 INFO:tasks.workunit.client.0.smithi162.stdout:5/663: dread d8/d1a/d23/d40/f6e [0,4194304] 0 2024-04-27T04:10:13.992 INFO:tasks.workunit.client.0.smithi162.stdout:6/618: rename d6/d1a/d34/d3a/d60 to d6/db/dc0 0 2024-04-27T04:10:13.994 INFO:tasks.workunit.client.0.smithi162.stdout:4/585: creat d2/d25/d1c/d39/d3b/d47/fb9 x:0 0 0 2024-04-27T04:10:13.995 INFO:tasks.workunit.client.1.smithi181.stdout:8/823: getdents d1/d6/d2f/d11e 0 2024-04-27T04:10:13.997 INFO:tasks.workunit.client.0.smithi162.stdout:2/574: creat d0/d4/d17/d88/fad x:0 0 0 2024-04-27T04:10:13.997 INFO:tasks.workunit.client.0.smithi162.stdout:1/592: sync 2024-04-27T04:10:13.998 INFO:tasks.workunit.client.0.smithi162.stdout:1/593: fdatasync d0/d11/d5b/fad 0 2024-04-27T04:10:13.999 INFO:tasks.workunit.client.0.smithi162.stdout:2/575: dread d0/d4/d17/d4f/f85 [0,4194304] 0 2024-04-27T04:10:14.000 INFO:tasks.workunit.client.0.smithi162.stdout:2/576: chown d0/d4/d2a/d3b 224362857 1 2024-04-27T04:10:14.002 INFO:tasks.workunit.client.1.smithi181.stdout:0/921: write dc/d1f/d3b/d4a/d95/d26/d131/dba/deb/f90 [461134,83538] 0 2024-04-27T04:10:14.002 INFO:tasks.workunit.client.1.smithi181.stdout:3/760: write d0/f23 [4916114,36836] 0 2024-04-27T04:10:14.003 INFO:tasks.workunit.client.1.smithi181.stdout:2/849: symlink d4/d9/d16/d82/d7d/db7/l10f 0 2024-04-27T04:10:14.005 INFO:tasks.workunit.client.0.smithi162.stdout:5/664: creat d8/d1a/d7d/fd7 x:0 0 0 2024-04-27T04:10:14.005 INFO:tasks.workunit.client.0.smithi162.stdout:5/665: chown d8/d1a/d23/dbc/d66 3471 1 2024-04-27T04:10:14.007 INFO:tasks.workunit.client.0.smithi162.stdout:9/636: getdents d3/d13/d2d/d30/d67 0 2024-04-27T04:10:14.008 INFO:tasks.workunit.client.1.smithi181.stdout:4/724: write d0/d15/d25/d2b/d6d/f74 [1807009,83133] 0 2024-04-27T04:10:14.012 INFO:tasks.workunit.client.0.smithi162.stdout:3/599: rename d1/d45/fbb to d1/d18/d23/d34/dad/fcc 0 2024-04-27T04:10:14.012 INFO:tasks.workunit.client.0.smithi162.stdout:7/639: write d3/d5/d9/f21 [1911797,128598] 0 2024-04-27T04:10:14.013 INFO:tasks.workunit.client.1.smithi181.stdout:5/958: dwrite d4/de/d2f/d7a/d90/f144 [0,4194304] 0 2024-04-27T04:10:14.014 INFO:tasks.workunit.client.1.smithi181.stdout:1/771: symlink d1/d2/d64/ddc/lfe 0 2024-04-27T04:10:14.015 INFO:tasks.workunit.client.1.smithi181.stdout:5/959: readlink d4/de/d2c/d46/lde 0 2024-04-27T04:10:14.018 INFO:tasks.workunit.client.0.smithi162.stdout:1/594: dread d0/d11/f20 [0,4194304] 0 2024-04-27T04:10:14.020 INFO:tasks.workunit.client.0.smithi162.stdout:4/586: creat d2/d25/d1c/d39/d5e/fba x:0 0 0 2024-04-27T04:10:14.021 INFO:tasks.workunit.client.1.smithi181.stdout:8/824: read d1/d6/d2f/d72/f94 [101557,47818] 0 2024-04-27T04:10:14.021 INFO:tasks.workunit.client.1.smithi181.stdout:9/736: dwrite d0/fa7 [0,4194304] 0 2024-04-27T04:10:14.023 INFO:tasks.workunit.client.1.smithi181.stdout:9/737: dread - d0/d44/d4b/d43/d89/daf/ff9 zero size 2024-04-27T04:10:14.027 INFO:tasks.workunit.client.0.smithi162.stdout:8/654: write d0/d10/f6a [523448,80529] 0 2024-04-27T04:10:14.028 INFO:tasks.workunit.client.0.smithi162.stdout:8/655: fdatasync d0/d10/d44/d70/fa9 0 2024-04-27T04:10:14.036 INFO:tasks.workunit.client.0.smithi162.stdout:5/666: fsync d8/d1a/f51 0 2024-04-27T04:10:14.036 INFO:tasks.workunit.client.1.smithi181.stdout:7/773: write d1/d22/d15/d1f/d34/fe2 [2849868,130068] 0 2024-04-27T04:10:14.040 INFO:tasks.workunit.client.1.smithi181.stdout:4/725: unlink d0/d15/c3 0 2024-04-27T04:10:14.040 INFO:tasks.workunit.client.0.smithi162.stdout:0/721: dwrite d5/dc/f14 [0,4194304] 0 2024-04-27T04:10:14.041 INFO:tasks.workunit.client.1.smithi181.stdout:6/655: write d7/dc/dc3/f6b [229297,58331] 0 2024-04-27T04:10:14.045 INFO:tasks.workunit.client.0.smithi162.stdout:3/600: creat d1/d21/d47/fcd x:0 0 0 2024-04-27T04:10:14.045 INFO:tasks.workunit.client.1.smithi181.stdout:7/774: dwrite d1/d22/d15/f28 [0,4194304] 0 2024-04-27T04:10:14.046 INFO:tasks.workunit.client.0.smithi162.stdout:3/601: write d1/d18/d23/d58/f9d [530207,29970] 0 2024-04-27T04:10:14.047 INFO:tasks.workunit.client.0.smithi162.stdout:6/619: symlink d6/d1a/d34/d7b/d7e/d97/lc1 0 2024-04-27T04:10:14.052 INFO:tasks.workunit.client.0.smithi162.stdout:4/587: creat d2/d25/d1c/d39/d90/fbb x:0 0 0 2024-04-27T04:10:14.054 INFO:tasks.workunit.client.1.smithi181.stdout:9/738: mknod d0/d44/d4b/d3d/cfb 0 2024-04-27T04:10:14.056 INFO:tasks.workunit.client.1.smithi181.stdout:2/850: symlink d4/d9/d16/l110 0 2024-04-27T04:10:14.057 INFO:tasks.workunit.client.0.smithi162.stdout:6/620: dwrite d6/d1a/d34/d50/f93 [0,4194304] 0 2024-04-27T04:10:14.058 INFO:tasks.workunit.client.1.smithi181.stdout:3/761: dwrite d0/d6c/d52/d44/dd5/f5d [0,4194304] 0 2024-04-27T04:10:14.059 INFO:tasks.workunit.client.0.smithi162.stdout:2/577: mknod d0/d4/d2a/d3b/d49/d5a/d63/d84/d93/cae 0 2024-04-27T04:10:14.064 INFO:tasks.workunit.client.0.smithi162.stdout:5/667: dread d8/d1a/f83 [0,4194304] 0 2024-04-27T04:10:14.065 INFO:tasks.workunit.client.1.smithi181.stdout:9/739: dread d0/d44/d4b/d3d/f73 [0,4194304] 0 2024-04-27T04:10:14.065 INFO:tasks.workunit.client.0.smithi162.stdout:9/637: mkdir d3/d39/dc4 0 2024-04-27T04:10:14.067 INFO:tasks.workunit.client.0.smithi162.stdout:9/638: write d3/d13/fb7 [672698,67870] 0 2024-04-27T04:10:14.070 INFO:tasks.workunit.client.0.smithi162.stdout:0/722: rename d5/d60/d98/fe7 to d5/dc/d1e/d20/dec/fee 0 2024-04-27T04:10:14.072 INFO:tasks.workunit.client.1.smithi181.stdout:4/726: truncate d0/d15/de/d19/d8a/fc8 1528445 0 2024-04-27T04:10:14.076 INFO:tasks.workunit.client.0.smithi162.stdout:0/723: dwrite d5/d26/da2/fb7 [4194304,4194304] 0 2024-04-27T04:10:14.080 INFO:tasks.workunit.client.0.smithi162.stdout:1/595: dread d0/d11/d44/d66/fbc [0,4194304] 0 2024-04-27T04:10:14.086 INFO:tasks.workunit.client.1.smithi181.stdout:3/762: mkdir d0/d6c/d52/d44/dd5/d58/df0 0 2024-04-27T04:10:14.087 INFO:tasks.workunit.client.0.smithi162.stdout:6/621: rmdir d6/d1a/d34/d3a/d68/d6c 39 2024-04-27T04:10:14.087 INFO:tasks.workunit.client.1.smithi181.stdout:3/763: chown d0/d6c/d52/d44/dd5/d58/de4/daf/fce 672 1 2024-04-27T04:10:14.088 INFO:tasks.workunit.client.0.smithi162.stdout:7/640: sync 2024-04-27T04:10:14.089 INFO:tasks.workunit.client.1.smithi181.stdout:2/851: truncate d4/d9/df/d14/da7/f108 165226 0 2024-04-27T04:10:14.089 INFO:tasks.workunit.client.1.smithi181.stdout:2/852: stat d4/d9/d16/d45 0 2024-04-27T04:10:14.091 INFO:tasks.workunit.client.0.smithi162.stdout:2/578: mknod d0/d4/d17/d4f/caf 0 2024-04-27T04:10:14.093 INFO:tasks.workunit.client.1.smithi181.stdout:5/960: creat d4/de/d2c/d46/d4d/d8e/f158 x:0 0 0 2024-04-27T04:10:14.098 INFO:tasks.workunit.client.1.smithi181.stdout:9/740: rmdir d0/d44/d4b/d43/d8f 39 2024-04-27T04:10:14.100 INFO:tasks.workunit.client.1.smithi181.stdout:9/741: write d0/d44/d4b/d3d/dbd/dd1/dde/ff0 [751639,96985] 0 2024-04-27T04:10:14.100 INFO:tasks.workunit.client.1.smithi181.stdout:1/772: creat d1/dad/db5/d6/da3/fff x:0 0 0 2024-04-27T04:10:14.102 INFO:tasks.workunit.client.1.smithi181.stdout:1/773: write d1/dad/db5/d6/da3/d1e/d48/da6/da4/fe6 [1043393,90126] 0 2024-04-27T04:10:14.103 INFO:tasks.workunit.client.1.smithi181.stdout:4/727: symlink d0/d15/de/d60/led 0 2024-04-27T04:10:14.106 INFO:tasks.workunit.client.1.smithi181.stdout:7/775: link d1/d2/d1c/d2d/d61/d80/fc7 d1/df/fe7 0 2024-04-27T04:10:14.108 INFO:tasks.workunit.client.1.smithi181.stdout:0/922: rename dc/d1f/d3b/d4a/d95/d26/d41/d6f/d7d/c99 to dc/d1f/d3b/d4a/d95/d26/d131/d106/d110/c13e 0 2024-04-27T04:10:14.109 INFO:tasks.workunit.client.1.smithi181.stdout:6/656: dwrite d7/dc/d1c/d88/db3/fe4 [0,4194304] 0 2024-04-27T04:10:14.109 INFO:tasks.workunit.client.0.smithi162.stdout:0/724: mkdir d5/d26/d4a/def 0 2024-04-27T04:10:14.113 INFO:tasks.workunit.client.1.smithi181.stdout:3/764: dread - d0/d6c/d66/d88/fa8 zero size 2024-04-27T04:10:14.113 INFO:tasks.workunit.client.1.smithi181.stdout:2/853: creat d4/d9/d4f/d79/d94/df4/f111 x:0 0 0 2024-04-27T04:10:14.114 INFO:tasks.workunit.client.0.smithi162.stdout:1/596: rename d0/l4 to d0/d7/lc7 0 2024-04-27T04:10:14.115 INFO:tasks.workunit.client.0.smithi162.stdout:1/597: stat d0/d80/d89/faf 0 2024-04-27T04:10:14.115 INFO:tasks.workunit.client.1.smithi181.stdout:3/765: chown d0/d6c/d52/d44/dd5/d58/f7e 250 1 2024-04-27T04:10:14.116 INFO:tasks.workunit.client.0.smithi162.stdout:8/656: creat d0/fd5 x:0 0 0 2024-04-27T04:10:14.117 INFO:tasks.workunit.client.1.smithi181.stdout:5/961: readlink d4/de/d2c/d46/d99/dad/d13a/l148 0 2024-04-27T04:10:14.117 INFO:tasks.workunit.client.1.smithi181.stdout:2/854: dread d4/d9/d16/d45/d9a/fe1 [0,4194304] 0 2024-04-27T04:10:14.118 INFO:tasks.workunit.client.0.smithi162.stdout:1/598: truncate d0/d80/d89/f8f 510334 0 2024-04-27T04:10:14.118 INFO:tasks.workunit.client.1.smithi181.stdout:7/776: dwrite d1/d22/d15/d29/dbe/fad [0,4194304] 0 2024-04-27T04:10:14.118 INFO:tasks.workunit.client.0.smithi162.stdout:1/599: stat d0/d11/d50/f56 0 2024-04-27T04:10:14.120 INFO:tasks.workunit.client.1.smithi181.stdout:7/777: write d1/d22/d15/d29/dbe/d1b/f47 [317588,24066] 0 2024-04-27T04:10:14.121 INFO:tasks.workunit.client.0.smithi162.stdout:7/641: mknod d3/d5/d5b/d83/de/d2b/d81/daa/dbd/dab/cd6 0 2024-04-27T04:10:14.125 INFO:tasks.workunit.client.1.smithi181.stdout:1/774: creat d1/dad/db5/d6/da3/d1e/d48/db4/f100 x:0 0 0 2024-04-27T04:10:14.126 INFO:tasks.workunit.client.0.smithi162.stdout:9/639: getdents d3/d13/d2d/d30/d67 0 2024-04-27T04:10:14.130 INFO:tasks.workunit.client.1.smithi181.stdout:0/923: stat dc/d1f/d3b/df7 0 2024-04-27T04:10:14.131 INFO:tasks.workunit.client.0.smithi162.stdout:0/725: mknod d5/d26/cf0 0 2024-04-27T04:10:14.133 INFO:tasks.workunit.client.1.smithi181.stdout:6/657: stat d7/dc/d1e/d2b/c2c 0 2024-04-27T04:10:14.133 INFO:tasks.workunit.client.1.smithi181.stdout:6/658: chown d7/dc/d1c/c5b 51 1 2024-04-27T04:10:14.140 INFO:tasks.workunit.client.0.smithi162.stdout:8/657: symlink d0/d10/d25/d7d/db8/dbe/ld6 0 2024-04-27T04:10:14.140 INFO:tasks.workunit.client.0.smithi162.stdout:8/658: truncate d0/d10/d25/f36 8418845 0 2024-04-27T04:10:14.140 INFO:tasks.workunit.client.0.smithi162.stdout:4/588: link d2/d25/d1c/d39/d3b/d47/f4a d2/d25/d1c/d24/fbc 0 2024-04-27T04:10:14.141 INFO:tasks.workunit.client.1.smithi181.stdout:6/659: dwrite d7/dc/dc3/d72/daa/dd3/fd8 [0,4194304] 0 2024-04-27T04:10:14.142 INFO:tasks.workunit.client.1.smithi181.stdout:6/660: chown d7/d31/f75 209159699 1 2024-04-27T04:10:14.148 INFO:tasks.workunit.client.0.smithi162.stdout:7/642: fdatasync d3/d39/d57/f43 0 2024-04-27T04:10:14.150 INFO:tasks.workunit.client.0.smithi162.stdout:3/602: getdents d1/d18/d23/d34 0 2024-04-27T04:10:14.151 INFO:tasks.workunit.client.1.smithi181.stdout:3/766: dread d0/d6c/d52/d44/f55 [0,4194304] 0 2024-04-27T04:10:14.152 INFO:tasks.workunit.client.1.smithi181.stdout:4/728: dwrite d0/d15/de/d19/d77/d57/f84 [0,4194304] 0 2024-04-27T04:10:14.155 INFO:tasks.workunit.client.1.smithi181.stdout:1/775: symlink d1/dad/db5/d6/da3/d60/l101 0 2024-04-27T04:10:14.155 INFO:tasks.workunit.client.1.smithi181.stdout:3/767: read d0/f37 [591641,125030] 0 2024-04-27T04:10:14.155 INFO:tasks.workunit.client.1.smithi181.stdout:0/924: rename dc/d1f/d3b/d4a/d95/f36 to dc/d1f/d6b/d7e/d91/f13f 0 2024-04-27T04:10:14.161 INFO:tasks.workunit.client.0.smithi162.stdout:2/579: rename d0/d4/d2a/d41/d7b to d0/d4/d17/d4f/db0 0 2024-04-27T04:10:14.162 INFO:tasks.workunit.client.1.smithi181.stdout:7/778: dwrite d1/dc3/fa1 [0,4194304] 0 2024-04-27T04:10:14.162 INFO:tasks.workunit.client.1.smithi181.stdout:7/779: chown d1/d22/d15/d1f/d34/la6 14 1 2024-04-27T04:10:14.166 INFO:tasks.workunit.client.1.smithi181.stdout:5/962: mknod d4/de/d2c/c159 0 2024-04-27T04:10:14.167 INFO:tasks.workunit.client.0.smithi162.stdout:6/622: dwrite d6/d1a/d34/d3a/d68/d6c/d67/f7c [0,4194304] 0 2024-04-27T04:10:14.168 INFO:tasks.workunit.client.0.smithi162.stdout:6/623: fdatasync d6/db/dc0/d74/fa1 0 2024-04-27T04:10:14.169 INFO:tasks.workunit.client.0.smithi162.stdout:6/624: fsync d6/f35 0 2024-04-27T04:10:14.178 INFO:tasks.workunit.client.0.smithi162.stdout:1/600: symlink d0/lc8 0 2024-04-27T04:10:14.187 INFO:tasks.workunit.client.0.smithi162.stdout:5/668: getdents d8/d1a 0 2024-04-27T04:10:14.187 INFO:tasks.workunit.client.1.smithi181.stdout:4/729: creat d0/d15/de/d60/fee x:0 0 0 2024-04-27T04:10:14.192 INFO:tasks.workunit.client.1.smithi181.stdout:9/742: rename d0/d44/d4b/d3d to d0/d44/d4b/d43/d88/da1/dfc 0 2024-04-27T04:10:14.194 INFO:tasks.workunit.client.1.smithi181.stdout:9/743: dread d0/d40/fcd [0,4194304] 0 2024-04-27T04:10:14.194 INFO:tasks.workunit.client.0.smithi162.stdout:3/603: unlink d1/d21/d47/d57/la8 0 2024-04-27T04:10:14.195 INFO:tasks.workunit.client.1.smithi181.stdout:6/661: write d7/dc/d1c/d34/f9a [306884,126021] 0 2024-04-27T04:10:14.204 INFO:tasks.workunit.client.1.smithi181.stdout:0/925: fdatasync dc/d1f/d6b/fa9 0 2024-04-27T04:10:14.205 INFO:tasks.workunit.client.0.smithi162.stdout:9/640: truncate d3/d13/d15/f57 584650 0 2024-04-27T04:10:14.206 INFO:tasks.workunit.client.1.smithi181.stdout:7/780: truncate d1/d22/d15/d1f/d34/f3c 2046216 0 2024-04-27T04:10:14.206 INFO:tasks.workunit.client.0.smithi162.stdout:9/641: write d3/d13/d2d/d3b/d54/f8d [1295300,126767] 0 2024-04-27T04:10:14.208 INFO:tasks.workunit.client.0.smithi162.stdout:0/726: truncate d5/d60/d98/fa3 557259 0 2024-04-27T04:10:14.209 INFO:tasks.workunit.client.0.smithi162.stdout:8/659: mknod d0/d10/d2b/d2d/d2f/d6b/dd3/cd7 0 2024-04-27T04:10:14.209 INFO:tasks.workunit.client.0.smithi162.stdout:8/660: fsync d0/de/d6e/d71/f95 0 2024-04-27T04:10:14.210 INFO:tasks.workunit.client.1.smithi181.stdout:4/730: rmdir d0/d15/d5c/d69 39 2024-04-27T04:10:14.214 INFO:tasks.workunit.client.1.smithi181.stdout:9/744: mkdir d0/d44/d4b/d50/dfd 0 2024-04-27T04:10:14.215 INFO:tasks.workunit.client.1.smithi181.stdout:9/745: chown d0/d44/d4b/d50/dfd 791113 1 2024-04-27T04:10:14.222 INFO:tasks.workunit.client.1.smithi181.stdout:3/768: mkdir d0/d6c/d52/d44/de8/df1 0 2024-04-27T04:10:14.226 INFO:tasks.workunit.client.0.smithi162.stdout:1/601: mkdir d0/d7/d3c/d54/dc9 0 2024-04-27T04:10:14.227 INFO:tasks.workunit.client.0.smithi162.stdout:4/589: truncate d2/d25/f8c 2500582 0 2024-04-27T04:10:14.229 INFO:tasks.workunit.client.1.smithi181.stdout:0/926: mkdir dc/d1f/d115/d140 0 2024-04-27T04:10:14.231 INFO:tasks.workunit.client.0.smithi162.stdout:5/669: fdatasync d8/d1a/d7d/f91 0 2024-04-27T04:10:14.233 INFO:tasks.workunit.client.1.smithi181.stdout:7/781: mkdir d1/df/d1a/de8 0 2024-04-27T04:10:14.234 INFO:tasks.workunit.client.0.smithi162.stdout:7/643: mknod d3/d5/d5b/d60/d65/cd7 0 2024-04-27T04:10:14.235 INFO:tasks.workunit.client.1.smithi181.stdout:5/963: mknod d4/de/d2c/d46/c15a 0 2024-04-27T04:10:14.237 INFO:tasks.workunit.client.0.smithi162.stdout:3/604: creat d1/d18/d23/d34/daa/fce x:0 0 0 2024-04-27T04:10:14.238 INFO:tasks.workunit.client.0.smithi162.stdout:3/605: write d1/d21/d47/d57/fb6 [556642,122305] 0 2024-04-27T04:10:14.241 INFO:tasks.workunit.client.0.smithi162.stdout:1/602: dread d0/d7/d3c/fb6 [0,4194304] 0 2024-04-27T04:10:14.241 INFO:tasks.workunit.client.0.smithi162.stdout:2/580: creat d0/d4/d2a/d41/d77/d73/fb1 x:0 0 0 2024-04-27T04:10:14.243 INFO:tasks.workunit.client.1.smithi181.stdout:5/964: dwrite d4/d10/d29/d122/d100/f151 [0,4194304] 0 2024-04-27T04:10:14.246 INFO:tasks.workunit.client.1.smithi181.stdout:6/662: dwrite d7/f19 [0,4194304] 0 2024-04-27T04:10:14.246 INFO:tasks.workunit.client.1.smithi181.stdout:9/746: readlink d0/d40/l3f 0 2024-04-27T04:10:14.249 INFO:tasks.workunit.client.1.smithi181.stdout:6/663: chown d7/d27/cb4 3092 1 2024-04-27T04:10:14.250 INFO:tasks.workunit.client.1.smithi181.stdout:6/664: read - d7/dc/d1e/d2b/fdb zero size 2024-04-27T04:10:14.252 INFO:tasks.workunit.client.1.smithi181.stdout:6/665: stat d7/dc/f12 0 2024-04-27T04:10:14.255 INFO:tasks.workunit.client.1.smithi181.stdout:6/666: chown d7/dc/dc3/dd1/dd9/f9b 1489378943 1 2024-04-27T04:10:14.257 INFO:tasks.workunit.client.1.smithi181.stdout:5/965: dwrite d4/de/d2f/f132 [0,4194304] 0 2024-04-27T04:10:14.258 INFO:tasks.workunit.client.0.smithi162.stdout:9/642: dread d3/f5 [0,4194304] 0 2024-04-27T04:10:14.260 INFO:tasks.workunit.client.1.smithi181.stdout:9/747: dread d0/f1e [0,4194304] 0 2024-04-27T04:10:14.260 INFO:tasks.workunit.client.0.smithi162.stdout:8/661: truncate d0/d10/d2b/d2e/f40 497129 0 2024-04-27T04:10:14.261 INFO:tasks.workunit.client.1.smithi181.stdout:9/748: stat d0/d44/d4b/d43/d88/fb8 0 2024-04-27T04:10:14.267 INFO:tasks.workunit.client.0.smithi162.stdout:4/590: truncate d2/d25/d1c/d39/f5f 487263 0 2024-04-27T04:10:14.271 INFO:tasks.workunit.client.0.smithi162.stdout:5/670: creat d8/d2c/fd8 x:0 0 0 2024-04-27T04:10:14.284 INFO:tasks.workunit.client.0.smithi162.stdout:9/643: rename d3/d13/d2d/d44/l8a to d3/d13/d2d/d44/lc5 0 2024-04-27T04:10:14.296 INFO:tasks.workunit.client.0.smithi162.stdout:5/671: dread d8/d2c/f33 [0,4194304] 0 2024-04-27T04:10:14.297 INFO:tasks.workunit.client.0.smithi162.stdout:2/581: mkdir d0/d4/d17/d2b/d64/d96/daa/db2 0 2024-04-27T04:10:14.308 INFO:tasks.workunit.client.0.smithi162.stdout:0/727: rename d5/d26/d69/d9b to d5/dc/d1e/d20/dec/df1 0 2024-04-27T04:10:14.312 INFO:tasks.workunit.client.0.smithi162.stdout:8/662: symlink d0/de/ld8 0 2024-04-27T04:10:14.314 INFO:tasks.workunit.client.0.smithi162.stdout:4/591: dwrite d2/d25/f36 [4194304,4194304] 0 2024-04-27T04:10:14.315 INFO:tasks.workunit.client.0.smithi162.stdout:6/625: link d6/d1a/d34/c37 d6/d1a/d34/d7b/d7e/d97/cc2 0 2024-04-27T04:10:14.316 INFO:tasks.workunit.client.0.smithi162.stdout:4/592: chown d2/da/d1a/f30 401 1 2024-04-27T04:10:14.319 INFO:tasks.workunit.client.0.smithi162.stdout:7/644: creat d3/d5/d5b/d60/fd8 x:0 0 0 2024-04-27T04:10:14.320 INFO:tasks.workunit.client.0.smithi162.stdout:7/645: readlink d3/l22 0 2024-04-27T04:10:14.321 INFO:tasks.workunit.client.0.smithi162.stdout:7/646: chown d3/d4f/f9d 108100 1 2024-04-27T04:10:14.323 INFO:tasks.workunit.client.0.smithi162.stdout:1/603: truncate d0/d11/d3e/d59/d7f/f83 3221700 0 2024-04-27T04:10:14.325 INFO:tasks.workunit.client.0.smithi162.stdout:4/593: dwrite d2/d25/d1c/d39/d3b/d47/d9c/fb4 [0,4194304] 0 2024-04-27T04:10:14.328 INFO:tasks.workunit.client.0.smithi162.stdout:5/672: creat d8/d86/fd9 x:0 0 0 2024-04-27T04:10:14.337 INFO:tasks.workunit.client.0.smithi162.stdout:9/644: getdents d3/d13/d2d/d3b/d7b 0 2024-04-27T04:10:14.339 INFO:tasks.workunit.client.0.smithi162.stdout:8/663: mkdir d0/de/d6e/d71/d86/dd9 0 2024-04-27T04:10:14.345 INFO:tasks.workunit.client.0.smithi162.stdout:3/606: getdents d1/d18/d23/d34/daa 0 2024-04-27T04:10:14.357 INFO:tasks.workunit.client.0.smithi162.stdout:0/728: write d5/f58 [579276,12213] 0 2024-04-27T04:10:14.359 INFO:tasks.workunit.client.0.smithi162.stdout:5/673: dread d8/f20 [0,4194304] 0 2024-04-27T04:10:14.360 INFO:tasks.workunit.client.1.smithi181.stdout:0/927: dread dc/d1f/d3b/d4a/d95/d26/d41/d6f/f69 [0,4194304] 0 2024-04-27T04:10:14.360 INFO:tasks.workunit.client.1.smithi181.stdout:1/776: rename d1/dad/db5/d6/da3/d1e/d48/f5d to d1/dad/db5/d6/da3/d1e/d20/f102 0 2024-04-27T04:10:14.367 INFO:tasks.workunit.client.1.smithi181.stdout:5/966: unlink d4/d10/fc4 0 2024-04-27T04:10:14.368 INFO:tasks.workunit.client.1.smithi181.stdout:5/967: chown d4/de/d2c/d46/d99/dad/d13a/d14a 940020001 1 2024-04-27T04:10:14.369 INFO:tasks.workunit.client.1.smithi181.stdout:3/769: mkdir d0/d6c/d52/d44/dd5/d58/de4/dc5/dcf/df2 0 2024-04-27T04:10:14.370 INFO:tasks.workunit.client.1.smithi181.stdout:3/770: chown d0/d6c/d52/d44/dd5/d6b/ca4 75990 1 2024-04-27T04:10:14.372 INFO:tasks.workunit.client.1.smithi181.stdout:7/782: mkdir d1/d22/d15/d5f/d81/de3/de9 0 2024-04-27T04:10:14.373 INFO:tasks.workunit.client.1.smithi181.stdout:4/731: truncate d0/d15/f2f 341777 0 2024-04-27T04:10:14.376 INFO:tasks.workunit.client.1.smithi181.stdout:9/749: rename d0/f2e to d0/d44/d4b/d5f/ffe 0 2024-04-27T04:10:14.380 INFO:tasks.workunit.client.1.smithi181.stdout:1/777: read d1/dad/db5/d6/da3/f1f [1356919,122972] 0 2024-04-27T04:10:14.382 INFO:tasks.workunit.client.1.smithi181.stdout:9/750: dwrite d0/d44/d4f/fae [0,4194304] 0 2024-04-27T04:10:14.386 INFO:tasks.workunit.client.1.smithi181.stdout:4/732: creat d0/d15/d25/d2a/da0/dc7/fef x:0 0 0 2024-04-27T04:10:14.390 INFO:tasks.workunit.client.0.smithi162.stdout:8/664: creat d0/de/d6e/d71/d86/fda x:0 0 0 2024-04-27T04:10:14.391 INFO:tasks.workunit.client.0.smithi162.stdout:7/647: symlink d3/d5/d5b/d83/de/d2b/d81/daa/dbd/dbe/ld9 0 2024-04-27T04:10:14.391 INFO:tasks.workunit.client.1.smithi181.stdout:0/928: mkdir dc/d1f/d3b/d4a/d95/d26/d131/d106/d141 0 2024-04-27T04:10:14.393 INFO:tasks.workunit.client.0.smithi162.stdout:2/582: link d0/d4/d17/d88/la3 d0/d4/d17/d88/lb3 0 2024-04-27T04:10:14.393 INFO:tasks.workunit.client.0.smithi162.stdout:8/665: dread - d0/de/d20/d7e/fd1 zero size 2024-04-27T04:10:14.394 INFO:tasks.workunit.client.0.smithi162.stdout:2/583: chown d0/l26 463785355 1 2024-04-27T04:10:14.394 INFO:tasks.workunit.client.1.smithi181.stdout:4/733: dread d0/d15/d25/fcd [0,4194304] 0 2024-04-27T04:10:14.396 INFO:tasks.workunit.client.0.smithi162.stdout:1/604: dwrite d0/f3b [0,4194304] 0 2024-04-27T04:10:14.406 INFO:tasks.workunit.client.1.smithi181.stdout:0/929: chown dc/d1f/d3b/f5f 1235 1 2024-04-27T04:10:14.409 INFO:tasks.workunit.client.1.smithi181.stdout:8/825: sync 2024-04-27T04:10:14.409 INFO:tasks.workunit.client.1.smithi181.stdout:7/783: creat d1/d2/d1c/d2d/fea x:0 0 0 2024-04-27T04:10:14.413 INFO:tasks.workunit.client.1.smithi181.stdout:9/751: fdatasync d0/d44/d4b/f39 0 2024-04-27T04:10:14.417 INFO:tasks.workunit.client.1.smithi181.stdout:5/968: rename d4/de/d2c/d46/d4d/d8e/d54/f136 to d4/d10/f15b 0 2024-04-27T04:10:14.418 INFO:tasks.workunit.client.1.smithi181.stdout:1/778: write d1/dad/db5/d6/d6a/f75 [5136943,50706] 0 2024-04-27T04:10:14.429 INFO:tasks.workunit.client.1.smithi181.stdout:7/784: dwrite d1/d22/d15/d29/dbe/d1b/fac [0,4194304] 0 2024-04-27T04:10:14.434 INFO:tasks.workunit.client.1.smithi181.stdout:5/969: mkdir d4/da3/d15c 0 2024-04-27T04:10:14.437 INFO:tasks.workunit.client.1.smithi181.stdout:1/779: chown d1/dad/db5/d6/f22 3 1 2024-04-27T04:10:14.438 INFO:tasks.workunit.client.1.smithi181.stdout:0/930: getdents dc/d1f/d3b/d4a/d95/d26/d68 0 2024-04-27T04:10:14.443 INFO:tasks.workunit.client.1.smithi181.stdout:4/734: creat d0/d15/de/d19/ff0 x:0 0 0 2024-04-27T04:10:14.444 INFO:tasks.workunit.client.0.smithi162.stdout:5/674: rmdir d8/d1a/d23/d40 39 2024-04-27T04:10:14.444 INFO:tasks.workunit.client.1.smithi181.stdout:8/826: dwrite d1/d20/fa2 [0,4194304] 0 2024-04-27T04:10:14.446 INFO:tasks.workunit.client.1.smithi181.stdout:8/827: write d1/d6/d6f/ff5 [1086750,85214] 0 2024-04-27T04:10:14.448 INFO:tasks.workunit.client.0.smithi162.stdout:6/626: creat d6/d1a/d34/fc3 x:0 0 0 2024-04-27T04:10:14.449 INFO:tasks.workunit.client.1.smithi181.stdout:9/752: rename d0/d44/d4b/d43/f5e to d0/d44/d4b/d43/d88/da1/dfc/fff 0 2024-04-27T04:10:14.450 INFO:tasks.workunit.client.0.smithi162.stdout:3/607: creat d1/d18/dac/fcf x:0 0 0 2024-04-27T04:10:14.451 INFO:tasks.workunit.client.0.smithi162.stdout:3/608: chown d1/d21/d47 1039 1 2024-04-27T04:10:14.452 INFO:tasks.workunit.client.0.smithi162.stdout:3/609: stat d1/d18/d23/d2f/f7c 0 2024-04-27T04:10:14.452 INFO:tasks.workunit.client.1.smithi181.stdout:5/970: rmdir d4/d10 39 2024-04-27T04:10:14.456 INFO:tasks.workunit.client.0.smithi162.stdout:4/594: link d2/d25/d1c/d24/lb3 d2/d54/lbd 0 2024-04-27T04:10:14.458 INFO:tasks.workunit.client.1.smithi181.stdout:0/931: mknod dc/d93/c142 0 2024-04-27T04:10:14.459 INFO:tasks.workunit.client.0.smithi162.stdout:0/729: symlink d5/d26/lf2 0 2024-04-27T04:10:14.462 INFO:tasks.workunit.client.0.smithi162.stdout:8/666: chown d0/c5c 8221 1 2024-04-27T04:10:14.463 INFO:tasks.workunit.client.1.smithi181.stdout:1/780: dread d1/d2/f13 [0,4194304] 0 2024-04-27T04:10:14.464 INFO:tasks.workunit.client.1.smithi181.stdout:4/735: mknod d0/d15/d76/cf1 0 2024-04-27T04:10:14.465 INFO:tasks.workunit.client.0.smithi162.stdout:2/584: mkdir d0/d4/d17/db4 0 2024-04-27T04:10:14.466 INFO:tasks.workunit.client.0.smithi162.stdout:2/585: dread - d0/d4/d2a/d3b/d49/d5a/d63/d6e/fa7 zero size 2024-04-27T04:10:14.467 INFO:tasks.workunit.client.0.smithi162.stdout:8/667: dwrite d0/d10/d76/fb7 [0,4194304] 0 2024-04-27T04:10:14.468 INFO:tasks.workunit.client.1.smithi181.stdout:8/828: unlink d1/d6/d2f/d30/d66/dc0/dcd/f11a 0 2024-04-27T04:10:14.474 INFO:tasks.workunit.client.0.smithi162.stdout:6/627: creat d6/d1a/d34/d7b/d7e/fc4 x:0 0 0 2024-04-27T04:10:14.475 INFO:tasks.workunit.client.0.smithi162.stdout:6/628: write d6/db/d32/f51 [723927,87212] 0 2024-04-27T04:10:14.476 INFO:tasks.workunit.client.1.smithi181.stdout:7/785: dwrite d1/d22/f95 [0,4194304] 0 2024-04-27T04:10:14.478 INFO:tasks.workunit.client.1.smithi181.stdout:0/932: unlink dc/d1f/daa/f103 0 2024-04-27T04:10:14.479 INFO:tasks.workunit.client.0.smithi162.stdout:3/610: mknod d1/d21/d47/cd0 0 2024-04-27T04:10:14.482 INFO:tasks.workunit.client.0.smithi162.stdout:9/645: sync 2024-04-27T04:10:14.486 INFO:tasks.workunit.client.0.smithi162.stdout:0/730: mkdir d5/dc/d1e/d46/df3 0 2024-04-27T04:10:14.486 INFO:tasks.workunit.client.0.smithi162.stdout:0/731: chown d5/dc/d1e/f21 6 1 2024-04-27T04:10:14.487 INFO:tasks.workunit.client.1.smithi181.stdout:8/829: fsync d1/d20/f51 0 2024-04-27T04:10:14.488 INFO:tasks.workunit.client.0.smithi162.stdout:0/732: chown d5/dc/d1e/d46/d48/d9e/db8/l43 2 1 2024-04-27T04:10:14.488 INFO:tasks.workunit.client.0.smithi162.stdout:0/733: read - d5/dc/fd9 zero size 2024-04-27T04:10:14.489 INFO:tasks.workunit.client.1.smithi181.stdout:8/830: readlink d1/d6/d2f/d30/d66/dc0/l106 0 2024-04-27T04:10:14.490 INFO:tasks.workunit.client.1.smithi181.stdout:5/971: write d4/de/d2c/d46/d4d/d8e/d67/d71/f7b [1810255,18847] 0 2024-04-27T04:10:14.492 INFO:tasks.workunit.client.0.smithi162.stdout:1/605: creat d0/d7/d3c/d54/dc9/fca x:0 0 0 2024-04-27T04:10:14.494 INFO:tasks.workunit.client.0.smithi162.stdout:2/586: mknod d0/d4/d2a/d3b/d49/d5a/d63/d84/cb5 0 2024-04-27T04:10:14.495 INFO:tasks.workunit.client.1.smithi181.stdout:7/786: unlink d1/d2/d1c/d2d/d61/d80/fc7 0 2024-04-27T04:10:14.496 INFO:tasks.workunit.client.0.smithi162.stdout:8/668: rmdir d0/d10/d25/d65/d96 39 2024-04-27T04:10:14.501 INFO:tasks.workunit.client.0.smithi162.stdout:6/629: symlink d6/d1a/d34/d3a/d68/d6c/lc5 0 2024-04-27T04:10:14.502 INFO:tasks.workunit.client.1.smithi181.stdout:7/787: dwrite d1/d2/d35/d67/dbf/fc9 [0,4194304] 0 2024-04-27T04:10:14.510 INFO:tasks.workunit.client.1.smithi181.stdout:8/831: dwrite d1/d6/d2f/f45 [0,4194304] 0 2024-04-27T04:10:14.516 INFO:tasks.workunit.client.0.smithi162.stdout:9/646: rename d3/f21 to d3/d13/d2d/d30/d67/fc6 0 2024-04-27T04:10:14.517 INFO:tasks.workunit.client.1.smithi181.stdout:8/832: dread d1/d6/d2f/d30/d66/dc0/dcd/de1/fe3 [0,4194304] 0 2024-04-27T04:10:14.520 INFO:tasks.workunit.client.1.smithi181.stdout:0/933: creat dc/d1f/d6b/d7e/d12b/f143 x:0 0 0 2024-04-27T04:10:14.522 INFO:tasks.workunit.client.0.smithi162.stdout:0/734: mknod d5/dc/d1e/d46/d48/d9e/cf4 0 2024-04-27T04:10:14.522 INFO:tasks.workunit.client.0.smithi162.stdout:9/647: dwrite d3/fbd [0,4194304] 0 2024-04-27T04:10:14.523 INFO:tasks.workunit.client.1.smithi181.stdout:4/736: link d0/d15/d25/d2b/d6d/lc6 d0/d15/d76/lf2 0 2024-04-27T04:10:14.524 INFO:tasks.workunit.client.0.smithi162.stdout:7/648: link d3/d5/d5b/d83/de/l46 d3/d5/d5b/d83/de/d2b/d81/daa/dbd/db6/lda 0 2024-04-27T04:10:14.526 INFO:tasks.workunit.client.1.smithi181.stdout:7/788: mknod d1/d22/d15/d29/ceb 0 2024-04-27T04:10:14.527 INFO:tasks.workunit.client.1.smithi181.stdout:0/934: dwrite dc/d1f/d3b/d4a/d95/d6d/fc4 [0,4194304] 0 2024-04-27T04:10:14.530 INFO:tasks.workunit.client.0.smithi162.stdout:1/606: mknod d0/d7/d2a/ccb 0 2024-04-27T04:10:14.530 INFO:tasks.workunit.client.1.smithi181.stdout:0/935: write dc/d1f/d3b/d4a/d95/f10b [168677,116358] 0 2024-04-27T04:10:14.531 INFO:tasks.workunit.client.0.smithi162.stdout:1/607: chown d0/d7/d3c/d61/dc4 193747 1 2024-04-27T04:10:14.547 INFO:tasks.workunit.client.0.smithi162.stdout:3/611: dwrite d1/d70/f8c [0,4194304] 0 2024-04-27T04:10:14.549 INFO:tasks.workunit.client.1.smithi181.stdout:5/972: dwrite d4/de/d2c/d46/d4d/d8e/f48 [0,4194304] 0 2024-04-27T04:10:14.550 INFO:tasks.workunit.client.0.smithi162.stdout:6/630: creat d6/db/dc0/fc6 x:0 0 0 2024-04-27T04:10:14.552 INFO:tasks.workunit.client.1.smithi181.stdout:5/973: fsync d4/de/d2f/d7a/d90/f150 0 2024-04-27T04:10:14.561 INFO:tasks.workunit.client.1.smithi181.stdout:7/789: rename d1/d22/d15/d29/dbe/d1b/f4e to d1/d22/d15/d1f/d34/fec 0 2024-04-27T04:10:14.570 INFO:tasks.workunit.client.0.smithi162.stdout:0/735: fsync d5/dc/d1e/d46/d48/d9e/db8/f95 0 2024-04-27T04:10:14.570 INFO:tasks.workunit.client.0.smithi162.stdout:7/649: truncate d3/d5/f53 772286 0 2024-04-27T04:10:14.570 INFO:tasks.workunit.client.0.smithi162.stdout:7/650: fdatasync d3/d5/d5b/d60/fd3 0 2024-04-27T04:10:14.575 INFO:tasks.workunit.client.1.smithi181.stdout:8/833: write d1/d6/d2f/d47/f4f [4477583,64256] 0 2024-04-27T04:10:14.576 INFO:tasks.workunit.client.0.smithi162.stdout:2/587: dwrite d0/d4/d17/d4f/f85 [0,4194304] 0 2024-04-27T04:10:14.577 INFO:tasks.workunit.client.0.smithi162.stdout:2/588: stat d0/d4/d2a 0 2024-04-27T04:10:14.578 INFO:tasks.workunit.client.0.smithi162.stdout:8/669: symlink d0/de/d6e/d71/d86/dd9/ldb 0 2024-04-27T04:10:14.578 INFO:tasks.workunit.client.0.smithi162.stdout:8/670: chown d0/fd 984068 1 2024-04-27T04:10:14.579 INFO:tasks.workunit.client.1.smithi181.stdout:4/737: unlink d0/d15/de/d19/c7b 0 2024-04-27T04:10:14.583 INFO:tasks.workunit.client.0.smithi162.stdout:5/675: getdents d8/d86 0 2024-04-27T04:10:14.588 INFO:tasks.workunit.client.1.smithi181.stdout:0/936: mknod dc/d1f/d3b/d4a/d95/d26/d131/dba/dce/df4/c144 0 2024-04-27T04:10:14.591 INFO:tasks.workunit.client.0.smithi162.stdout:6/631: creat d6/d1a/d34/d7f/d80/fc7 x:0 0 0 2024-04-27T04:10:14.591 INFO:tasks.workunit.client.1.smithi181.stdout:8/834: stat d1/d6/d1a/c41 0 2024-04-27T04:10:14.593 INFO:tasks.workunit.client.0.smithi162.stdout:4/595: getdents d2/d25/d1c/d39/d3b/d47 0 2024-04-27T04:10:14.594 INFO:tasks.workunit.client.0.smithi162.stdout:9/648: fsync d3/d23/f5c 0 2024-04-27T04:10:14.596 INFO:tasks.workunit.client.1.smithi181.stdout:4/738: unlink d0/d15/d25/d2b/d6d/d82/fcc 0 2024-04-27T04:10:14.597 INFO:tasks.workunit.client.1.smithi181.stdout:7/790: creat d1/d22/d15/d5f/d81/de3/de9/fed x:0 0 0 2024-04-27T04:10:14.598 INFO:tasks.workunit.client.1.smithi181.stdout:7/791: chown d1/d2/d1c/d2d/d61/f68 3844 1 2024-04-27T04:10:14.609 INFO:tasks.workunit.client.1.smithi181.stdout:2/855: sync 2024-04-27T04:10:14.609 INFO:tasks.workunit.client.1.smithi181.stdout:6/667: sync 2024-04-27T04:10:14.617 INFO:tasks.workunit.client.1.smithi181.stdout:8/835: mkdir d1/d6/d2f/d30/d66/dc0/d125 0 2024-04-27T04:10:14.617 INFO:tasks.workunit.client.1.smithi181.stdout:0/937: write dc/d54/f72 [1697626,109863] 0 2024-04-27T04:10:14.620 INFO:tasks.workunit.client.0.smithi162.stdout:7/651: rename d3/d5/d9/l49 to d3/d5/d5b/d83/de/ldb 0 2024-04-27T04:10:14.626 INFO:tasks.workunit.client.0.smithi162.stdout:1/608: dwrite d0/f2b [0,4194304] 0 2024-04-27T04:10:14.626 INFO:tasks.workunit.client.0.smithi162.stdout:2/589: rmdir d0/d4/d17/d2b/d64/d96/daa 39 2024-04-27T04:10:14.630 INFO:tasks.workunit.client.1.smithi181.stdout:4/739: write d0/d15/de/d19/d8a/fa9 [449974,25252] 0 2024-04-27T04:10:14.634 INFO:tasks.workunit.client.1.smithi181.stdout:0/938: creat dc/d1f/d3b/d4a/d95/d26/d68/f145 x:0 0 0 2024-04-27T04:10:14.634 INFO:tasks.workunit.client.1.smithi181.stdout:8/836: read d1/d6/d1a/d25/fc7 [2752806,82638] 0 2024-04-27T04:10:14.636 INFO:tasks.workunit.client.0.smithi162.stdout:6/632: unlink d6/db/c36 0 2024-04-27T04:10:14.641 INFO:tasks.workunit.client.0.smithi162.stdout:3/612: truncate d1/d21/d47/d57/fb6 482793 0 2024-04-27T04:10:14.642 INFO:tasks.workunit.client.1.smithi181.stdout:0/939: dwrite dc/d1f/d3b/d4a/d95/f10b [0,4194304] 0 2024-04-27T04:10:14.642 INFO:tasks.workunit.client.0.smithi162.stdout:3/613: readlink d1/d18/d5d/d5f/d66/lc8 0 2024-04-27T04:10:14.643 INFO:tasks.workunit.client.1.smithi181.stdout:0/940: chown dc/l116 32993 1 2024-04-27T04:10:14.643 INFO:tasks.workunit.client.1.smithi181.stdout:6/668: symlink d7/dc/d1c/d7f/le5 0 2024-04-27T04:10:14.649 INFO:tasks.workunit.client.0.smithi162.stdout:9/649: dread d3/d13/d15/d50/f5b [0,4194304] 0 2024-04-27T04:10:14.651 INFO:tasks.workunit.client.1.smithi181.stdout:5/974: rename d4/d10/d29/c42 to d4/de/d2f/c15d 0 2024-04-27T04:10:14.652 INFO:tasks.workunit.client.1.smithi181.stdout:5/975: read - d4/d10/f15b zero size 2024-04-27T04:10:14.654 INFO:tasks.workunit.client.1.smithi181.stdout:5/976: write d4/da3/dbc/ff3 [3622119,23481] 0 2024-04-27T04:10:14.655 INFO:tasks.workunit.client.0.smithi162.stdout:9/650: dwrite d3/d11/f4d [0,4194304] 0 2024-04-27T04:10:14.656 INFO:tasks.workunit.client.1.smithi181.stdout:8/837: fsync d1/d6/d2f/d72/f94 0 2024-04-27T04:10:14.659 INFO:tasks.workunit.client.1.smithi181.stdout:8/838: chown d1/d6/d2f/d72/l83 3443284 1 2024-04-27T04:10:14.659 INFO:tasks.workunit.client.0.smithi162.stdout:7/652: creat d3/d5/d5b/d83/de/d2b/d81/daa/dbd/dab/fdc x:0 0 0 2024-04-27T04:10:14.664 INFO:tasks.workunit.client.1.smithi181.stdout:4/740: write d0/d15/d25/d2a/d36/f51 [1339565,93342] 0 2024-04-27T04:10:14.668 INFO:tasks.workunit.client.0.smithi162.stdout:1/609: symlink d0/d11/d3e/d59/lcc 0 2024-04-27T04:10:14.668 INFO:tasks.workunit.client.1.smithi181.stdout:2/856: link d4/d9/d16/d45/d9a/d9d/le3 d4/d81/d102/l112 0 2024-04-27T04:10:14.670 INFO:tasks.workunit.client.1.smithi181.stdout:2/857: dread d4/d9/f64 [0,4194304] 0 2024-04-27T04:10:14.671 INFO:tasks.workunit.client.0.smithi162.stdout:8/671: unlink d0/d10/l37 0 2024-04-27T04:10:14.674 INFO:tasks.workunit.client.1.smithi181.stdout:0/941: rename dc/d1f/d6b/d7e/d91/dea to dc/d1f/d3b/d4a/d7a/d146 0 2024-04-27T04:10:14.674 INFO:tasks.workunit.client.1.smithi181.stdout:0/942: stat dc/d1f/daa 0 2024-04-27T04:10:14.679 INFO:tasks.workunit.client.1.smithi181.stdout:5/977: mkdir d4/de/d2c/d46/d99/dad/d13a/db3/d15e 0 2024-04-27T04:10:14.680 INFO:tasks.workunit.client.1.smithi181.stdout:5/978: dread - d4/de/d12f/f13f zero size 2024-04-27T04:10:14.680 INFO:tasks.workunit.client.0.smithi162.stdout:4/596: mknod d2/d25/d1c/d39/d3b/d6e/cbe 0 2024-04-27T04:10:14.683 INFO:tasks.workunit.client.0.smithi162.stdout:5/676: write d8/d1a/d23/d40/f46 [158986,127319] 0 2024-04-27T04:10:14.684 INFO:tasks.workunit.client.0.smithi162.stdout:3/614: symlink d1/d18/dac/ld1 0 2024-04-27T04:10:14.685 INFO:tasks.workunit.client.1.smithi181.stdout:6/669: symlink d7/dc/dc3/d72/d98/dbf/dca/le6 0 2024-04-27T04:10:14.686 INFO:tasks.workunit.client.1.smithi181.stdout:4/741: mknod d0/d15/d25/d2a/d36/cf3 0 2024-04-27T04:10:14.688 INFO:tasks.workunit.client.0.smithi162.stdout:9/651: creat d3/d23/d7c/fc7 x:0 0 0 2024-04-27T04:10:14.689 INFO:tasks.workunit.client.1.smithi181.stdout:2/858: unlink d4/d9/d35/d42/f5a 0 2024-04-27T04:10:14.690 INFO:tasks.workunit.client.1.smithi181.stdout:2/859: stat d4/d9/d35/d42/d86/l10c 0 2024-04-27T04:10:14.690 INFO:tasks.workunit.client.0.smithi162.stdout:7/653: creat d3/d5/d5b/d83/fdd x:0 0 0 2024-04-27T04:10:14.691 INFO:tasks.workunit.client.0.smithi162.stdout:1/610: creat d0/d7/fcd x:0 0 0 2024-04-27T04:10:14.693 INFO:tasks.workunit.client.1.smithi181.stdout:8/839: creat d1/d6/d2f/d11e/dd9/f126 x:0 0 0 2024-04-27T04:10:14.694 INFO:tasks.workunit.client.0.smithi162.stdout:9/652: dwrite d3/d23/fa2 [0,4194304] 0 2024-04-27T04:10:14.694 INFO:tasks.workunit.client.0.smithi162.stdout:8/672: mkdir d0/de/d6e/d71/d86/ddc 0 2024-04-27T04:10:14.695 INFO:tasks.workunit.client.1.smithi181.stdout:6/670: unlink c1 0 2024-04-27T04:10:14.697 INFO:tasks.workunit.client.1.smithi181.stdout:6/671: write d7/dc/ff [1654692,76590] 0 2024-04-27T04:10:14.699 INFO:tasks.workunit.client.0.smithi162.stdout:1/611: dwrite d0/d7/f88 [0,4194304] 0 2024-04-27T04:10:14.700 INFO:tasks.workunit.client.0.smithi162.stdout:1/612: dread - d0/d11/d5b/fad zero size 2024-04-27T04:10:14.701 INFO:tasks.workunit.client.0.smithi162.stdout:4/597: symlink d2/d25/d1c/d39/d3b/d6e/d6f/lbf 0 2024-04-27T04:10:14.701 INFO:tasks.workunit.client.1.smithi181.stdout:0/943: creat dc/d1f/d3b/dfa/f147 x:0 0 0 2024-04-27T04:10:14.702 INFO:tasks.workunit.client.1.smithi181.stdout:5/979: mkdir d4/d15f 0 2024-04-27T04:10:14.702 INFO:tasks.workunit.client.0.smithi162.stdout:1/613: chown d0/d7/d3c/d6e/f7b 7364 1 2024-04-27T04:10:14.705 INFO:tasks.workunit.client.1.smithi181.stdout:8/840: readlink d1/d6/d2f/d47/dba/dbc/lfb 0 2024-04-27T04:10:14.706 INFO:tasks.workunit.client.1.smithi181.stdout:8/841: readlink d1/d6/d2f/d30/d66/dc0/l122 0 2024-04-27T04:10:14.709 INFO:tasks.workunit.client.0.smithi162.stdout:3/615: rename d1/d45/d69 to d1/d18/d5d/d5f/d66/dd2 0 2024-04-27T04:10:14.711 INFO:tasks.workunit.client.0.smithi162.stdout:0/736: link d5/dc/d1e/d46/fc3 d5/d26/d4a/def/ff5 0 2024-04-27T04:10:14.722 INFO:tasks.workunit.client.0.smithi162.stdout:8/673: creat d0/d10/d76/fdd x:0 0 0 2024-04-27T04:10:14.723 INFO:tasks.workunit.client.1.smithi181.stdout:5/980: truncate d4/d10/f57 1954539 0 2024-04-27T04:10:14.724 INFO:tasks.workunit.client.1.smithi181.stdout:6/672: dwrite d7/dc/d1e/f53 [4194304,4194304] 0 2024-04-27T04:10:14.725 INFO:tasks.workunit.client.0.smithi162.stdout:9/653: symlink d3/d13/d2d/d30/d7f/lc8 0 2024-04-27T04:10:14.727 INFO:tasks.workunit.client.0.smithi162.stdout:8/674: dwrite d0/de/d20/d6c/fa5 [0,4194304] 0 2024-04-27T04:10:14.732 INFO:tasks.workunit.client.0.smithi162.stdout:6/633: truncate d6/d1a/f20 994354 0 2024-04-27T04:10:14.733 INFO:tasks.workunit.client.0.smithi162.stdout:1/614: fdatasync d0/d7/d10/f4c 0 2024-04-27T04:10:14.736 INFO:tasks.workunit.client.0.smithi162.stdout:5/677: write d8/d1a/d7d/f91 [255544,104854] 0 2024-04-27T04:10:14.739 INFO:tasks.workunit.client.0.smithi162.stdout:3/616: creat d1/d18/d23/d34/fd3 x:0 0 0 2024-04-27T04:10:14.741 INFO:tasks.workunit.client.1.smithi181.stdout:4/742: write d0/d15/de/d19/d77/f72 [1042723,78606] 0 2024-04-27T04:10:14.742 INFO:tasks.workunit.client.0.smithi162.stdout:0/737: unlink d5/dc/d1e/d56/cb6 0 2024-04-27T04:10:14.743 INFO:tasks.workunit.client.1.smithi181.stdout:2/860: write d4/d9/d16/d82/d7d/db7/dcb/f46 [2785786,114983] 0 2024-04-27T04:10:14.743 INFO:tasks.workunit.client.1.smithi181.stdout:5/981: rmdir d4/de/d2c/d46/d99/dad/d13a/db3 39 2024-04-27T04:10:14.746 INFO:tasks.workunit.client.0.smithi162.stdout:2/590: getdents d0/d4/d2a/d3b/d49/d5a/d63/d80 0 2024-04-27T04:10:14.747 INFO:tasks.workunit.client.0.smithi162.stdout:3/617: dwrite d1/d18/d5d/d5f/fca [0,4194304] 0 2024-04-27T04:10:14.754 INFO:tasks.workunit.client.1.smithi181.stdout:0/944: dwrite dc/d93/ff8 [0,4194304] 0 2024-04-27T04:10:14.756 INFO:tasks.workunit.client.1.smithi181.stdout:6/673: dread d7/dc/dc3/d62/f63 [0,4194304] 0 2024-04-27T04:10:14.758 INFO:tasks.workunit.client.1.smithi181.stdout:0/945: readlink dc/d1f/d6b/la1 0 2024-04-27T04:10:14.764 INFO:tasks.workunit.client.0.smithi162.stdout:3/618: dread d1/d21/d47/d57/f5e [0,4194304] 0 2024-04-27T04:10:14.764 INFO:tasks.workunit.client.1.smithi181.stdout:6/674: chown d7/dc/d1e/d66/c8d 1069212 1 2024-04-27T04:10:14.764 INFO:tasks.workunit.client.1.smithi181.stdout:4/743: unlink d0/d15/d25/d2a/da0/le4 0 2024-04-27T04:10:14.764 INFO:tasks.workunit.client.1.smithi181.stdout:8/842: dwrite d1/d6/d1a/d25/f99 [0,4194304] 0 2024-04-27T04:10:14.767 INFO:tasks.workunit.client.0.smithi162.stdout:6/634: symlink d6/d1a/d34/d50/dad/lc8 0 2024-04-27T04:10:14.770 INFO:tasks.workunit.client.0.smithi162.stdout:1/615: creat d0/d11/d50/d53/d93/d96/daa/fce x:0 0 0 2024-04-27T04:10:14.775 INFO:tasks.workunit.client.0.smithi162.stdout:1/616: fdatasync d0/d7/d10/fa7 0 2024-04-27T04:10:14.775 INFO:tasks.workunit.client.0.smithi162.stdout:5/678: rmdir d8/d1a/d29/d4a/d9f 39 2024-04-27T04:10:14.775 INFO:tasks.workunit.client.0.smithi162.stdout:6/635: dread d6/f35 [0,4194304] 0 2024-04-27T04:10:14.775 INFO:tasks.workunit.client.1.smithi181.stdout:0/946: chown dc/d1f/d6b/d7e/c138 6 1 2024-04-27T04:10:14.775 INFO:tasks.workunit.client.1.smithi181.stdout:3/771: sync 2024-04-27T04:10:14.775 INFO:tasks.workunit.client.1.smithi181.stdout:0/947: stat dc/d1f/d3b/dfa 0 2024-04-27T04:10:14.775 INFO:tasks.workunit.client.1.smithi181.stdout:3/772: readlink d0/d28/d86/lc4 0 2024-04-27T04:10:14.775 INFO:tasks.workunit.client.1.smithi181.stdout:5/982: write d4/de/f47 [4552193,75605] 0 2024-04-27T04:10:14.776 INFO:tasks.workunit.client.0.smithi162.stdout:6/636: read d6/db/dc0/d74/fa1 [115523,25067] 0 2024-04-27T04:10:14.776 INFO:tasks.workunit.client.0.smithi162.stdout:6/637: stat d6/db/dc0/d74/l9e 0 2024-04-27T04:10:14.779 INFO:tasks.workunit.client.0.smithi162.stdout:0/738: rmdir d5/d26/d4a/def 39 2024-04-27T04:10:14.779 INFO:tasks.workunit.client.0.smithi162.stdout:5/679: dwrite d8/d9/d5b/fb4 [0,4194304] 0 2024-04-27T04:10:14.781 INFO:tasks.workunit.client.1.smithi181.stdout:2/861: dread d4/d9/d4f/d5f/fc1 [0,4194304] 0 2024-04-27T04:10:14.781 INFO:tasks.workunit.client.1.smithi181.stdout:4/744: symlink d0/d15/d25/d2a/d4c/d7c/db5/lf4 0 2024-04-27T04:10:14.782 INFO:tasks.workunit.client.1.smithi181.stdout:2/862: fdatasync d4/d9/d16/d45/f9e 0 2024-04-27T04:10:14.783 INFO:tasks.workunit.client.0.smithi162.stdout:7/654: creat d3/d5/d9/fde x:0 0 0 2024-04-27T04:10:14.784 INFO:tasks.workunit.client.1.smithi181.stdout:8/843: mkdir d1/d6/d2f/d11e/dd9/d127 0 2024-04-27T04:10:14.785 INFO:tasks.workunit.client.0.smithi162.stdout:7/655: read d3/d5/d5b/d83/f33 [4184950,120104] 0 2024-04-27T04:10:14.786 INFO:tasks.workunit.client.0.smithi162.stdout:2/591: creat d0/d20/d51/fb6 x:0 0 0 2024-04-27T04:10:14.787 INFO:tasks.workunit.client.1.smithi181.stdout:6/675: write d7/dc/d1c/d34/d3f/d7e/d8a/f7a [2456573,39349] 0 2024-04-27T04:10:14.788 INFO:tasks.workunit.client.0.smithi162.stdout:5/680: dwrite d8/d1a/f45 [0,4194304] 0 2024-04-27T04:10:14.792 INFO:tasks.workunit.client.0.smithi162.stdout:9/654: creat d3/d13/d2d/d30/d7f/dc3/fc9 x:0 0 0 2024-04-27T04:10:14.795 INFO:tasks.workunit.client.1.smithi181.stdout:5/983: readlink d4/d10/d29/d122/d100/lee 0 2024-04-27T04:10:14.800 INFO:tasks.workunit.client.1.smithi181.stdout:3/773: rename d0/d6c/d52/d44/dd5/d6b/ce2 to d0/d6c/d52/d44/dd5/d58/de4/dc5/dcf/df2/cf3 0 2024-04-27T04:10:14.802 INFO:tasks.workunit.client.0.smithi162.stdout:3/619: mknod d1/d18/d23/d34/daa/cd4 0 2024-04-27T04:10:14.806 INFO:tasks.workunit.client.1.smithi181.stdout:6/676: creat d7/dc/d1e/d2b/dde/fe7 x:0 0 0 2024-04-27T04:10:14.806 INFO:tasks.workunit.client.0.smithi162.stdout:6/638: chown d6/d1a/d34/l6d 358973 1 2024-04-27T04:10:14.806 INFO:tasks.workunit.client.0.smithi162.stdout:7/656: dread d3/d4f/f78 [0,4194304] 0 2024-04-27T04:10:14.810 INFO:tasks.workunit.client.0.smithi162.stdout:7/657: dwrite d3/d5/d5b/d60/fd3 [4194304,4194304] 0 2024-04-27T04:10:14.811 INFO:tasks.workunit.client.1.smithi181.stdout:3/774: unlink d0/de/f17 0 2024-04-27T04:10:14.811 INFO:tasks.workunit.client.1.smithi181.stdout:3/775: stat d0/d6c/d52/d44 0 2024-04-27T04:10:14.818 INFO:tasks.workunit.client.1.smithi181.stdout:2/863: symlink d4/d9/d4f/d79/l113 0 2024-04-27T04:10:14.821 INFO:tasks.workunit.client.1.smithi181.stdout:2/864: chown d4/d81/f84 624 1 2024-04-27T04:10:14.823 INFO:tasks.workunit.client.0.smithi162.stdout:2/592: mkdir d0/d4/d2a/d3b/d49/db7 0 2024-04-27T04:10:14.824 INFO:tasks.workunit.client.1.smithi181.stdout:8/844: dread d1/d6/d1a/f53 [0,4194304] 0 2024-04-27T04:10:14.824 INFO:tasks.workunit.client.1.smithi181.stdout:6/677: rmdir d7/dc/dc3/d72/d98/dbf 39 2024-04-27T04:10:14.825 INFO:tasks.workunit.client.0.smithi162.stdout:5/681: mknod d8/d1a/d7d/d87/cda 0 2024-04-27T04:10:14.829 INFO:tasks.workunit.client.0.smithi162.stdout:2/593: dwrite d0/d4/d2a/d41/d77/f83 [0,4194304] 0 2024-04-27T04:10:14.829 INFO:tasks.workunit.client.1.smithi181.stdout:3/776: creat d0/d6c/d52/d44/dd5/d58/de4/dc5/dcf/ff4 x:0 0 0 2024-04-27T04:10:14.830 INFO:tasks.workunit.client.0.smithi162.stdout:2/594: write d0/d24/f38 [3400349,70783] 0 2024-04-27T04:10:14.830 INFO:tasks.workunit.client.1.smithi181.stdout:3/777: fsync d0/fd2 0 2024-04-27T04:10:14.833 INFO:tasks.workunit.client.0.smithi162.stdout:4/598: getdents d2/d25/d1c/d39/d70/d91 0 2024-04-27T04:10:14.833 INFO:tasks.workunit.client.1.smithi181.stdout:4/745: dwrite d0/d15/de/d19/d8a/fc8 [0,4194304] 0 2024-04-27T04:10:14.835 INFO:tasks.workunit.client.0.smithi162.stdout:2/595: dwrite d0/f44 [0,4194304] 0 2024-04-27T04:10:14.837 INFO:tasks.workunit.client.0.smithi162.stdout:3/620: unlink d1/d18/d5d/d5f/d66/dd2/fb2 0 2024-04-27T04:10:14.839 INFO:tasks.workunit.client.1.smithi181.stdout:0/948: getdents dc/d1f/d3b/d4a/d95/d6d 0 2024-04-27T04:10:14.840 INFO:tasks.workunit.client.1.smithi181.stdout:9/753: sync 2024-04-27T04:10:14.845 INFO:tasks.workunit.client.0.smithi162.stdout:1/617: rename d0/d7/l63 to d0/d11/d44/lcf 0 2024-04-27T04:10:14.848 INFO:tasks.workunit.client.1.smithi181.stdout:6/678: dread d7/dc/dc3/dd1/dd9/fa7 [0,4194304] 0 2024-04-27T04:10:14.850 INFO:tasks.workunit.client.1.smithi181.stdout:8/845: unlink d1/d6/d2f/d11e/f121 0 2024-04-27T04:10:14.852 INFO:tasks.workunit.client.0.smithi162.stdout:9/655: write d3/d23/d1e/f2b [1036412,61051] 0 2024-04-27T04:10:14.853 INFO:tasks.workunit.client.1.smithi181.stdout:5/984: creat d4/da3/dbc/f160 x:0 0 0 2024-04-27T04:10:14.855 INFO:tasks.workunit.client.1.smithi181.stdout:3/778: mkdir d0/de/d7d/df5 0 2024-04-27T04:10:14.860 INFO:tasks.workunit.client.1.smithi181.stdout:2/865: rename d4/d9/df/d14/da7/ca9 to d4/d9/d4f/d5f/d100/c114 0 2024-04-27T04:10:14.863 INFO:tasks.workunit.client.1.smithi181.stdout:0/949: unlink dc/d1f/c59 0 2024-04-27T04:10:14.864 INFO:tasks.workunit.client.1.smithi181.stdout:3/779: dwrite d0/d6c/d52/d44/dd5/d58/fea [0,4194304] 0 2024-04-27T04:10:14.864 INFO:tasks.workunit.client.1.smithi181.stdout:4/746: dread d0/d15/d76/db1/fb6 [0,4194304] 0 2024-04-27T04:10:14.866 INFO:tasks.workunit.client.1.smithi181.stdout:9/754: unlink d0/d40/fcd 0 2024-04-27T04:10:14.867 INFO:tasks.workunit.client.0.smithi162.stdout:5/682: symlink d8/d1a/d23/d40/ldb 0 2024-04-27T04:10:14.868 INFO:tasks.workunit.client.1.smithi181.stdout:0/950: write dc/d1f/d3b/d4a/d95/d26/d131/dbd/fbe [2793541,9259] 0 2024-04-27T04:10:14.879 INFO:tasks.workunit.client.1.smithi181.stdout:6/679: dread d7/f26 [0,4194304] 0 2024-04-27T04:10:14.880 INFO:tasks.workunit.client.1.smithi181.stdout:9/755: dwrite d0/d44/d4b/d43/d88/da1/dfc/dbd/df3/d85/f90 [0,4194304] 0 2024-04-27T04:10:14.904 INFO:tasks.workunit.client.1.smithi181.stdout:7/792: sync 2024-04-27T04:10:14.904 INFO:tasks.workunit.client.1.smithi181.stdout:1/781: sync 2024-04-27T04:10:14.906 INFO:tasks.workunit.client.1.smithi181.stdout:1/782: chown d1/dad/db5/d6/db/fe9 20604589 1 2024-04-27T04:10:14.909 INFO:tasks.workunit.client.1.smithi181.stdout:7/793: dread d1/d22/d15/dc6/dd8/f66 [0,4194304] 0 2024-04-27T04:10:14.912 INFO:tasks.workunit.client.1.smithi181.stdout:7/794: read d1/dc3/fcf [282744,23596] 0 2024-04-27T04:10:14.914 INFO:tasks.workunit.client.1.smithi181.stdout:7/795: chown d1/d22/d15/d29/dbe/d1b/f47 2 1 2024-04-27T04:10:14.971 INFO:tasks.workunit.client.0.smithi162.stdout:6/639: write d6/d1a/f88 [371139,78094] 0 2024-04-27T04:10:14.978 INFO:tasks.workunit.client.1.smithi181.stdout:2/866: unlink d4/d9/d4f/d79/d98/ld2 0 2024-04-27T04:10:14.982 INFO:tasks.workunit.client.0.smithi162.stdout:1/618: sync 2024-04-27T04:10:14.983 INFO:tasks.workunit.client.0.smithi162.stdout:2/596: creat d0/d4/d17/d4f/d7f/fb8 x:0 0 0 2024-04-27T04:10:14.985 INFO:tasks.workunit.client.0.smithi162.stdout:3/621: mkdir d1/d18/d23/d58/d63/dd5 0 2024-04-27T04:10:14.986 INFO:tasks.workunit.client.1.smithi181.stdout:0/951: truncate dc/f1b 7578089 0 2024-04-27T04:10:14.987 INFO:tasks.workunit.client.1.smithi181.stdout:0/952: chown dc/d1f/d3b/d4a/d95/d26/d68/l82 2 1 2024-04-27T04:10:14.992 INFO:tasks.workunit.client.0.smithi162.stdout:0/739: rmdir d5/dc/d1e/d20/dc5 0 2024-04-27T04:10:14.998 INFO:tasks.workunit.client.0.smithi162.stdout:8/675: link d0/d10/d2b/d2e/l53 d0/d10/lde 0 2024-04-27T04:10:14.999 INFO:tasks.workunit.client.0.smithi162.stdout:8/676: chown d0/d10/d2b/d2d/d2f/d39 172265 1 2024-04-27T04:10:14.999 INFO:tasks.workunit.client.1.smithi181.stdout:1/783: fsync d1/dad/db5/d6/db/d2d/f55 0 2024-04-27T04:10:15.000 INFO:tasks.workunit.client.0.smithi162.stdout:8/677: chown d0/de/d6e/d71/d86/ddc 0 1 2024-04-27T04:10:15.000 INFO:tasks.workunit.client.1.smithi181.stdout:1/784: read - d1/dad/db5/d6/da3/d1e/d20/f82 zero size 2024-04-27T04:10:15.001 INFO:tasks.workunit.client.0.smithi162.stdout:8/678: chown d0/de/l4f 358015202 1 2024-04-27T04:10:15.001 INFO:tasks.workunit.client.1.smithi181.stdout:1/785: stat d1/dad/db5/d6/da3/d1e/d48/dfa 0 2024-04-27T04:10:15.002 INFO:tasks.workunit.client.1.smithi181.stdout:1/786: fdatasync d1/dad/db5/d6/db/d2d/d36/dc4/fcf 0 2024-04-27T04:10:15.003 INFO:tasks.workunit.client.0.smithi162.stdout:6/640: unlink d6/d1a/d34/d50/f63 0 2024-04-27T04:10:15.005 INFO:tasks.workunit.client.0.smithi162.stdout:4/599: mkdir d2/d25/d1c/d24/db5/dc0 0 2024-04-27T04:10:15.012 INFO:tasks.workunit.client.0.smithi162.stdout:2/597: creat d0/d4/d17/d2b/d64/d89/fb9 x:0 0 0 2024-04-27T04:10:15.013 INFO:tasks.workunit.client.1.smithi181.stdout:2/867: fdatasync d4/d9/f2d 0 2024-04-27T04:10:15.016 INFO:tasks.workunit.client.0.smithi162.stdout:3/622: mkdir d1/d21/d47/d57/dd6 0 2024-04-27T04:10:15.018 INFO:tasks.workunit.client.1.smithi181.stdout:0/953: creat dc/d1f/daa/dbc/f148 x:0 0 0 2024-04-27T04:10:15.020 INFO:tasks.workunit.client.1.smithi181.stdout:0/954: write dc/d1f/d3b/d4a/d7a/d146/f129 [731786,78408] 0 2024-04-27T04:10:15.024 INFO:tasks.workunit.client.1.smithi181.stdout:6/680: mkdir d7/dc/de8 0 2024-04-27T04:10:15.026 INFO:tasks.workunit.client.1.smithi181.stdout:0/955: dwrite dc/d1f/d3b/d4a/d95/d26/d41/d12f/f135 [0,4194304] 0 2024-04-27T04:10:15.034 INFO:tasks.workunit.client.0.smithi162.stdout:8/679: rmdir d0/d10/d2b/d2d/d2f/d6b 39 2024-04-27T04:10:15.040 INFO:tasks.workunit.client.0.smithi162.stdout:8/680: dwrite d0/de/f7b [0,4194304] 0 2024-04-27T04:10:15.042 INFO:tasks.workunit.client.0.smithi162.stdout:8/681: write d0/d10/d44/d70/fa9 [129991,119039] 0 2024-04-27T04:10:15.047 INFO:tasks.workunit.client.0.smithi162.stdout:4/600: read d2/d25/f2d [326086,96666] 0 2024-04-27T04:10:15.048 INFO:tasks.workunit.client.0.smithi162.stdout:3/623: unlink d1/d18/d5d/d5f/f76 0 2024-04-27T04:10:15.049 INFO:tasks.workunit.client.0.smithi162.stdout:3/624: fsync d1/d18/d5d/d5f/fca 0 2024-04-27T04:10:15.052 INFO:tasks.workunit.client.1.smithi181.stdout:7/796: chown d1/df/cae 64788 1 2024-04-27T04:10:15.053 INFO:tasks.workunit.client.1.smithi181.stdout:7/797: stat d1/d22/d15/d29/dbe/d1b/cab 0 2024-04-27T04:10:15.054 INFO:tasks.workunit.client.1.smithi181.stdout:7/798: chown d1/d22/d15/d29/f84 0 1 2024-04-27T04:10:15.054 INFO:tasks.workunit.client.0.smithi162.stdout:7/658: getdents d3/d39/d57 0 2024-04-27T04:10:15.062 INFO:tasks.workunit.client.0.smithi162.stdout:6/641: getdents d6/d1a/d34/d3a/d68/d6c/d67 0 2024-04-27T04:10:15.066 INFO:tasks.workunit.client.0.smithi162.stdout:1/619: link d0/d7/d3c/d54/f68 d0/d11/d50/d53/d93/d96/fd0 0 2024-04-27T04:10:15.067 INFO:tasks.workunit.client.0.smithi162.stdout:1/620: read d0/d11/d3e/d5f/f6a [70702,37803] 0 2024-04-27T04:10:15.068 INFO:tasks.workunit.client.0.smithi162.stdout:1/621: chown d0/d7 29 1 2024-04-27T04:10:15.076 INFO:tasks.workunit.client.0.smithi162.stdout:0/740: creat d5/d26/da2/dc0/ff6 x:0 0 0 2024-04-27T04:10:15.077 INFO:tasks.workunit.client.1.smithi181.stdout:4/747: rmdir d0/d15/de/d19/d77/d57 39 2024-04-27T04:10:15.079 INFO:tasks.workunit.client.0.smithi162.stdout:7/659: fsync d3/d39/d57/f42 0 2024-04-27T04:10:15.081 INFO:tasks.workunit.client.0.smithi162.stdout:5/683: link d8/d9/d5b/d96/cbb d8/d1a/d23/dbc/cdc 0 2024-04-27T04:10:15.082 INFO:tasks.workunit.client.0.smithi162.stdout:7/660: write d3/d5/d5b/d83/de/d2b/d81/daa/dbd/fa0 [863828,39776] 0 2024-04-27T04:10:15.086 INFO:tasks.workunit.client.0.smithi162.stdout:6/642: rename f5 to d6/d1a/d34/d50/dad/fc9 0 2024-04-27T04:10:15.087 INFO:tasks.workunit.client.0.smithi162.stdout:4/601: dread d2/da/d1a/f30 [0,4194304] 0 2024-04-27T04:10:15.088 INFO:tasks.workunit.client.0.smithi162.stdout:4/602: stat d2/d25/d1c/d39/d70 0 2024-04-27T04:10:15.089 INFO:tasks.workunit.client.0.smithi162.stdout:4/603: write d2/da/d1a/f2e [8024168,78451] 0 2024-04-27T04:10:15.091 INFO:tasks.workunit.client.1.smithi181.stdout:4/748: creat d0/d15/de/d19/d77/ff5 x:0 0 0 2024-04-27T04:10:15.097 INFO:tasks.workunit.client.0.smithi162.stdout:5/684: dread d8/fad [0,4194304] 0 2024-04-27T04:10:15.099 INFO:tasks.workunit.client.1.smithi181.stdout:4/749: symlink d0/d15/d5c/da1/lf6 0 2024-04-27T04:10:15.104 INFO:tasks.workunit.client.0.smithi162.stdout:8/682: creat d0/de/d6e/fdf x:0 0 0 2024-04-27T04:10:15.109 INFO:tasks.workunit.client.0.smithi162.stdout:5/685: creat d8/d9/db3/fdd x:0 0 0 2024-04-27T04:10:15.111 INFO:tasks.workunit.client.0.smithi162.stdout:5/686: write d8/d1a/d29/fcb [192229,130689] 0 2024-04-27T04:10:15.121 INFO:tasks.workunit.client.0.smithi162.stdout:4/604: creat d2/d25/d1c/d39/d3b/d6e/fc1 x:0 0 0 2024-04-27T04:10:15.122 INFO:tasks.workunit.client.0.smithi162.stdout:0/741: getdents d5/dc/d1e/d20/dec/df1 0 2024-04-27T04:10:15.124 INFO:tasks.workunit.client.0.smithi162.stdout:4/605: truncate d2/d25/d1c/d24/fa6 722459 0 2024-04-27T04:10:15.124 INFO:tasks.workunit.client.0.smithi162.stdout:7/661: creat d3/d5/d9/fdf x:0 0 0 2024-04-27T04:10:15.130 INFO:tasks.workunit.client.0.smithi162.stdout:7/662: dread d3/d5/d9/f7f [0,4194304] 0 2024-04-27T04:10:15.132 INFO:tasks.workunit.client.1.smithi181.stdout:7/799: creat d1/d2/d1c/d2d/d61/fee x:0 0 0 2024-04-27T04:10:15.134 INFO:tasks.workunit.client.1.smithi181.stdout:7/800: read d1/d2/d35/d67/dbf/fc9 [1274479,128938] 0 2024-04-27T04:10:15.134 INFO:tasks.workunit.client.0.smithi162.stdout:7/663: mknod d3/d5/d5b/d83/de/d2b/d81/daa/dbd/db6/ce0 0 2024-04-27T04:10:15.135 INFO:tasks.workunit.client.0.smithi162.stdout:7/664: stat d3/d4f/fc3 0 2024-04-27T04:10:15.141 INFO:tasks.workunit.client.0.smithi162.stdout:2/598: sync 2024-04-27T04:10:15.142 INFO:tasks.workunit.client.0.smithi162.stdout:4/606: dread d2/d25/d1c/d24/fbc [0,4194304] 0 2024-04-27T04:10:15.142 INFO:tasks.workunit.client.1.smithi181.stdout:5/985: rename d4/d10/d29/d92/fd6 to d4/da3/dbc/f161 0 2024-04-27T04:10:15.143 INFO:tasks.workunit.client.1.smithi181.stdout:5/986: dread - d4/de/d2f/f14d zero size 2024-04-27T04:10:15.144 INFO:tasks.workunit.client.0.smithi162.stdout:4/607: readlink d2/d8a/l7c 0 2024-04-27T04:10:15.146 INFO:tasks.workunit.client.0.smithi162.stdout:2/599: dread d0/d4/d2a/d3b/d49/d5a/d63/d84/f99 [0,4194304] 0 2024-04-27T04:10:15.152 INFO:tasks.workunit.client.1.smithi181.stdout:5/987: mkdir d4/d15f/d162 0 2024-04-27T04:10:15.153 INFO:tasks.workunit.client.0.smithi162.stdout:4/608: truncate d2/d25/f8c 1564160 0 2024-04-27T04:10:15.156 INFO:tasks.workunit.client.1.smithi181.stdout:3/780: rename d0/d6c/d66/dc2/fcb to d0/d6c/d52/d44/dd5/d58/de4/daf/ff6 0 2024-04-27T04:10:15.159 INFO:tasks.workunit.client.0.smithi162.stdout:4/609: dwrite d2/da/fa1 [0,4194304] 0 2024-04-27T04:10:15.160 INFO:tasks.workunit.client.1.smithi181.stdout:3/781: unlink d0/d28/ld7 0 2024-04-27T04:10:15.160 INFO:tasks.workunit.client.0.smithi162.stdout:4/610: chown d2/da/f78 388270 1 2024-04-27T04:10:15.165 INFO:tasks.workunit.client.1.smithi181.stdout:5/988: dread d4/da3/dbc/ff3 [0,4194304] 0 2024-04-27T04:10:15.171 INFO:tasks.workunit.client.1.smithi181.stdout:3/782: write d0/f32 [1733231,48363] 0 2024-04-27T04:10:15.171 INFO:tasks.workunit.client.1.smithi181.stdout:2/868: rename d4/l1b to d4/d9/d35/d42/l115 0 2024-04-27T04:10:15.171 INFO:tasks.workunit.client.1.smithi181.stdout:5/989: dread d4/d10/fdb [0,4194304] 0 2024-04-27T04:10:15.171 INFO:tasks.workunit.client.0.smithi162.stdout:9/656: write d3/d13/d2d/d3b/d54/f6c [1624445,66100] 0 2024-04-27T04:10:15.172 INFO:tasks.workunit.client.0.smithi162.stdout:9/657: write d3/d13/d2d/d3b/d69/fbb [364068,21569] 0 2024-04-27T04:10:15.173 INFO:tasks.workunit.client.1.smithi181.stdout:3/783: symlink d0/d6c/d52/d44/dd5/d58/de4/dc5/dcf/df2/lf7 0 2024-04-27T04:10:15.175 INFO:tasks.workunit.client.1.smithi181.stdout:2/869: creat d4/d81/f116 x:0 0 0 2024-04-27T04:10:15.175 INFO:tasks.workunit.client.1.smithi181.stdout:8/846: write d1/f6b [1056282,54061] 0 2024-04-27T04:10:15.177 INFO:tasks.workunit.client.1.smithi181.stdout:8/847: chown d1/d6/d6f/d91 27485 1 2024-04-27T04:10:15.179 INFO:tasks.workunit.client.1.smithi181.stdout:5/990: dwrite d4/da3/dbc/f160 [0,4194304] 0 2024-04-27T04:10:15.182 INFO:tasks.workunit.client.1.smithi181.stdout:8/848: creat d1/d6/d6f/dd2/f128 x:0 0 0 2024-04-27T04:10:15.185 INFO:tasks.workunit.client.1.smithi181.stdout:5/991: creat d4/de/d2c/d46/d99/dad/dca/f163 x:0 0 0 2024-04-27T04:10:15.186 INFO:tasks.workunit.client.1.smithi181.stdout:3/784: getdents d0/d6c/d52/d44/dd5/d58/d64/dc6 0 2024-04-27T04:10:15.189 INFO:tasks.workunit.client.1.smithi181.stdout:8/849: rename d1/d6/d6f/fbb to d1/d6/d2f/d11e/dd9/f129 0 2024-04-27T04:10:15.193 INFO:tasks.workunit.client.1.smithi181.stdout:8/850: symlink d1/d8c/l12a 0 2024-04-27T04:10:15.194 INFO:tasks.workunit.client.1.smithi181.stdout:8/851: chown d1/d20/da4/c100 15803 1 2024-04-27T04:10:15.196 INFO:tasks.workunit.client.1.smithi181.stdout:3/785: creat d0/d6c/d52/d44/de8/df1/ff8 x:0 0 0 2024-04-27T04:10:15.197 INFO:tasks.workunit.client.1.smithi181.stdout:8/852: creat d1/d6/d2f/d30/d9d/dea/db3/ddf/f12b x:0 0 0 2024-04-27T04:10:15.209 INFO:tasks.workunit.client.0.smithi162.stdout:4/611: mknod d2/d25/d1c/d39/cc2 0 2024-04-27T04:10:15.209 INFO:tasks.workunit.client.0.smithi162.stdout:4/612: chown d2/da/fa1 3198 1 2024-04-27T04:10:15.210 INFO:tasks.workunit.client.0.smithi162.stdout:4/613: chown d2/da/d1a/f1b 3 1 2024-04-27T04:10:15.211 INFO:tasks.workunit.client.1.smithi181.stdout:3/786: dread d0/f3c [0,4194304] 0 2024-04-27T04:10:15.211 INFO:tasks.workunit.client.1.smithi181.stdout:1/787: write d1/dad/db5/d6/da3/d1e/d20/d53/d56/f6f [2788140,67885] 0 2024-04-27T04:10:15.217 INFO:tasks.workunit.client.1.smithi181.stdout:9/756: symlink d0/d44/d4b/d43/d88/da1/dfc/dbd/df3/l100 0 2024-04-27T04:10:15.226 INFO:tasks.workunit.client.1.smithi181.stdout:0/956: truncate dc/f94 1733434 0 2024-04-27T04:10:15.231 INFO:tasks.workunit.client.0.smithi162.stdout:3/625: dwrite d1/d70/f92 [0,4194304] 0 2024-04-27T04:10:15.235 INFO:tasks.workunit.client.1.smithi181.stdout:6/681: dwrite d7/dc/dc3/f64 [0,4194304] 0 2024-04-27T04:10:15.236 INFO:tasks.workunit.client.0.smithi162.stdout:3/626: fdatasync d1/d18/d23/d34/dad/dc5/fcb 0 2024-04-27T04:10:15.238 INFO:tasks.workunit.client.0.smithi162.stdout:3/627: fsync d1/d18/d23/d34/fd3 0 2024-04-27T04:10:15.244 INFO:tasks.workunit.client.0.smithi162.stdout:3/628: rename d1/d18/d23/cb7 to d1/d18/d23/d34/daa/dab/cd7 0 2024-04-27T04:10:15.245 INFO:tasks.workunit.client.0.smithi162.stdout:6/643: creat d6/d1a/d34/fca x:0 0 0 2024-04-27T04:10:15.246 INFO:tasks.workunit.client.0.smithi162.stdout:6/644: fsync d6/d1a/d34/d7b/d7e/d97/faf 0 2024-04-27T04:10:15.248 INFO:tasks.workunit.client.1.smithi181.stdout:4/750: write d0/d15/de/f43 [100136,107956] 0 2024-04-27T04:10:15.254 INFO:tasks.workunit.client.0.smithi162.stdout:4/614: rmdir d2/d25/d1c/d24 39 2024-04-27T04:10:15.256 INFO:tasks.workunit.client.0.smithi162.stdout:1/622: write d0/d11/f79 [26469,82753] 0 2024-04-27T04:10:15.260 INFO:tasks.workunit.client.1.smithi181.stdout:7/801: rmdir d1/d22/d15 39 2024-04-27T04:10:15.262 INFO:tasks.workunit.client.0.smithi162.stdout:4/615: dwrite d2/d25/d1c/d39/d3b/d47/d9c/fb4 [0,4194304] 0 2024-04-27T04:10:15.265 INFO:tasks.workunit.client.0.smithi162.stdout:4/616: write d2/d25/d1c/d39/d3b/d6e/d6f/fb6 [969171,75977] 0 2024-04-27T04:10:15.269 INFO:tasks.workunit.client.0.smithi162.stdout:1/623: getdents d0/d11/d50 0 2024-04-27T04:10:15.270 INFO:tasks.workunit.client.0.smithi162.stdout:4/617: chown d2/d54/cae 43527554 1 2024-04-27T04:10:15.272 INFO:tasks.workunit.client.1.smithi181.stdout:6/682: mknod d7/dc/dc3/d72/daa/dd3/ce9 0 2024-04-27T04:10:15.273 INFO:tasks.workunit.client.0.smithi162.stdout:5/687: dwrite d8/d1a/d23/d40/f6e [0,4194304] 0 2024-04-27T04:10:15.273 INFO:tasks.workunit.client.0.smithi162.stdout:6/645: dread d6/f1e [4194304,4194304] 0 2024-04-27T04:10:15.274 INFO:tasks.workunit.client.0.smithi162.stdout:0/742: write d5/f16 [4851162,74960] 0 2024-04-27T04:10:15.275 INFO:tasks.workunit.client.0.smithi162.stdout:8/683: dwrite d0/d10/d25/d7d/f8b [0,4194304] 0 2024-04-27T04:10:15.276 INFO:tasks.workunit.client.0.smithi162.stdout:4/618: fsync d2/d25/d1c/d39/d70/d91/fa4 0 2024-04-27T04:10:15.280 INFO:tasks.workunit.client.1.smithi181.stdout:6/683: symlink d7/dc/dc3/dd1/dd9/lea 0 2024-04-27T04:10:15.281 INFO:tasks.workunit.client.0.smithi162.stdout:2/600: sync 2024-04-27T04:10:15.282 INFO:tasks.workunit.client.0.smithi162.stdout:8/684: readlink d0/d10/d2b/d2d/l78 0 2024-04-27T04:10:15.282 INFO:tasks.workunit.client.0.smithi162.stdout:6/646: readlink d6/d1a/l28 0 2024-04-27T04:10:15.283 INFO:tasks.workunit.client.1.smithi181.stdout:4/751: truncate d0/d15/de/d19/d77/d57/f9f 4802701 0 2024-04-27T04:10:15.284 INFO:tasks.workunit.client.0.smithi162.stdout:5/688: rmdir d8/d1a/d23/dbc/d66 39 2024-04-27T04:10:15.285 INFO:tasks.workunit.client.0.smithi162.stdout:8/685: write d0/de/d6e/fce [77900,6464] 0 2024-04-27T04:10:15.286 INFO:tasks.workunit.client.0.smithi162.stdout:6/647: write d6/d1a/d34/d43/f44 [177892,118094] 0 2024-04-27T04:10:15.287 INFO:tasks.workunit.client.0.smithi162.stdout:1/624: dwrite d0/d80/d89/f8f [0,4194304] 0 2024-04-27T04:10:15.288 INFO:tasks.workunit.client.0.smithi162.stdout:0/743: mknod d5/dc/d1e/d20/cf7 0 2024-04-27T04:10:15.289 INFO:tasks.workunit.client.0.smithi162.stdout:8/686: write d0/d10/d2b/d2d/fc4 [1199289,74361] 0 2024-04-27T04:10:15.293 INFO:tasks.workunit.client.0.smithi162.stdout:2/601: mknod d0/d24/cba 0 2024-04-27T04:10:15.294 INFO:tasks.workunit.client.0.smithi162.stdout:7/665: write d3/d5/d5b/d83/f6e [1811637,60701] 0 2024-04-27T04:10:15.295 INFO:tasks.workunit.client.1.smithi181.stdout:7/802: symlink d1/d22/d15/d29/dbe/lef 0 2024-04-27T04:10:15.298 INFO:tasks.workunit.client.0.smithi162.stdout:5/689: dread - d8/d1a/d23/dbc/f75 zero size 2024-04-27T04:10:15.303 INFO:tasks.workunit.client.0.smithi162.stdout:0/744: mknod d5/d26/d69/cf8 0 2024-04-27T04:10:15.309 INFO:tasks.workunit.client.1.smithi181.stdout:6/684: rmdir d7/dc/de8 0 2024-04-27T04:10:15.312 INFO:tasks.workunit.client.0.smithi162.stdout:7/666: symlink d3/d5/d5b/d83/de/d2b/d81/le1 0 2024-04-27T04:10:15.312 INFO:tasks.workunit.client.0.smithi162.stdout:7/667: dread - d3/d5/d5b/d60/fd8 zero size 2024-04-27T04:10:15.313 INFO:tasks.workunit.client.1.smithi181.stdout:6/685: fdatasync d7/d27/f2e 0 2024-04-27T04:10:15.315 INFO:tasks.workunit.client.0.smithi162.stdout:0/745: readlink d5/dc/d1e/d20/dec/df1/db4/lcf 0 2024-04-27T04:10:15.316 INFO:tasks.workunit.client.0.smithi162.stdout:8/687: creat d0/d10/d44/db2/fe0 x:0 0 0 2024-04-27T04:10:15.318 INFO:tasks.workunit.client.1.smithi181.stdout:6/686: creat d7/dc/dc3/d72/daa/feb x:0 0 0 2024-04-27T04:10:15.321 INFO:tasks.workunit.client.0.smithi162.stdout:9/658: write d3/d13/d2d/d3b/f47 [2420988,19051] 0 2024-04-27T04:10:15.321 INFO:tasks.workunit.client.0.smithi162.stdout:9/659: chown d3/l7 0 1 2024-04-27T04:10:15.322 INFO:tasks.workunit.client.1.smithi181.stdout:6/687: readlink d7/dc/lbc 0 2024-04-27T04:10:15.323 INFO:tasks.workunit.client.0.smithi162.stdout:8/688: readlink d0/de/d20/lcc 0 2024-04-27T04:10:15.324 INFO:tasks.workunit.client.0.smithi162.stdout:8/689: fsync d0/d10/d2b/fbc 0 2024-04-27T04:10:15.324 INFO:tasks.workunit.client.0.smithi162.stdout:8/690: dread - d0/de/d20/d7e/fd1 zero size 2024-04-27T04:10:15.325 INFO:tasks.workunit.client.0.smithi162.stdout:2/602: rename d0/d24/f87 to d0/d20/fbb 0 2024-04-27T04:10:15.326 INFO:tasks.workunit.client.0.smithi162.stdout:2/603: stat d0/d4/d17/d2b/d64/d89 0 2024-04-27T04:10:15.329 INFO:tasks.workunit.client.1.smithi181.stdout:6/688: mknod d7/dc/dc3/d72/d98/dbf/dca/cec 0 2024-04-27T04:10:15.331 INFO:tasks.workunit.client.0.smithi162.stdout:9/660: dwrite d3/d13/d2d/d44/fb0 [0,4194304] 0 2024-04-27T04:10:15.333 INFO:tasks.workunit.client.0.smithi162.stdout:9/661: write d3/fbd [4767537,103766] 0 2024-04-27T04:10:15.334 INFO:tasks.workunit.client.0.smithi162.stdout:6/648: rename d6/d1a/d34/c37 to d6/d1a/d34/d7b/d7e/d97/ccb 0 2024-04-27T04:10:15.337 INFO:tasks.workunit.client.1.smithi181.stdout:8/853: dwrite d1/d6/d2f/d30/d9d/dea/f7c [0,4194304] 0 2024-04-27T04:10:15.338 INFO:tasks.workunit.client.0.smithi162.stdout:8/691: mkdir d0/d10/d2b/d2d/d2f/d39/de1 0 2024-04-27T04:10:15.339 INFO:tasks.workunit.client.0.smithi162.stdout:8/692: dread - d0/d10/d76/fdd zero size 2024-04-27T04:10:15.341 INFO:tasks.workunit.client.0.smithi162.stdout:7/668: creat d3/d5/d9/fe2 x:0 0 0 2024-04-27T04:10:15.345 INFO:tasks.workunit.client.1.smithi181.stdout:5/992: link d4/de/c1f d4/de/d2c/d46/d99/dad/c164 0 2024-04-27T04:10:15.348 INFO:tasks.workunit.client.1.smithi181.stdout:3/787: write d0/d6c/d52/d44/dd5/d58/f7e [569614,100674] 0 2024-04-27T04:10:15.348 INFO:tasks.workunit.client.1.smithi181.stdout:6/689: creat d7/dc/dc3/d72/d98/fed x:0 0 0 2024-04-27T04:10:15.348 INFO:tasks.workunit.client.1.smithi181.stdout:6/690: readlink d7/dc/ld4 0 2024-04-27T04:10:15.349 INFO:tasks.workunit.client.1.smithi181.stdout:6/691: fdatasync d7/dc/d1c/d34/f9a 0 2024-04-27T04:10:15.350 INFO:tasks.workunit.client.1.smithi181.stdout:1/788: rename d1/dad/db5/d6/da3/d1e/d20/d53/d56/dd9/fe0 to d1/dad/db5/d6/db/f103 0 2024-04-27T04:10:15.350 INFO:tasks.workunit.client.1.smithi181.stdout:9/757: rename d0 to d0/d44/d4b/d43/d88/da1/dfc/dbd/dd1/d101 22 2024-04-27T04:10:15.352 INFO:tasks.workunit.client.1.smithi181.stdout:8/854: mknod d1/d6/d6f/dd2/df2/c12c 0 2024-04-27T04:10:15.356 INFO:tasks.workunit.client.0.smithi162.stdout:0/746: rename d5/d60 to d5/dc/d1e/d20/dba/df9 0 2024-04-27T04:10:15.357 INFO:tasks.workunit.client.1.smithi181.stdout:5/993: creat d4/d10/d12d/f165 x:0 0 0 2024-04-27T04:10:15.359 INFO:tasks.workunit.client.0.smithi162.stdout:9/662: creat d3/d11/fca x:0 0 0 2024-04-27T04:10:15.360 INFO:tasks.workunit.client.1.smithi181.stdout:3/788: symlink d0/d6c/d66/dc2/lf9 0 2024-04-27T04:10:15.362 INFO:tasks.workunit.client.0.smithi162.stdout:8/693: creat d0/d10/d2b/d2d/d2f/d39/fe2 x:0 0 0 2024-04-27T04:10:15.365 INFO:tasks.workunit.client.1.smithi181.stdout:6/692: mknod d7/cee 0 2024-04-27T04:10:15.369 INFO:tasks.workunit.client.1.smithi181.stdout:0/957: rename dc/d1f/d3b/d4a/d95/c45 to dc/d1f/d3b/d4a/d95/d26/d41/d6f/d7d/c149 0 2024-04-27T04:10:15.371 INFO:tasks.workunit.client.0.smithi162.stdout:2/604: rename d0/d4/d17/d2b/d64/d96/daa/db2 to d0/d4/d17/d2b/d64/d89/dbc 0 2024-04-27T04:10:15.372 INFO:tasks.workunit.client.0.smithi162.stdout:2/605: dread - d0/d4/d2a/d3b/d49/d5a/d63/d6e/fa7 zero size 2024-04-27T04:10:15.372 INFO:tasks.workunit.client.0.smithi162.stdout:2/606: readlink d0/l1f 0 2024-04-27T04:10:15.373 INFO:tasks.workunit.client.0.smithi162.stdout:2/607: truncate d0/d24/f5e 8689451 0 2024-04-27T04:10:15.374 INFO:tasks.workunit.client.1.smithi181.stdout:8/855: mknod d1/d8c/c12d 0 2024-04-27T04:10:15.375 INFO:tasks.workunit.client.1.smithi181.stdout:8/856: read d1/d20/f51 [1417633,60872] 0 2024-04-27T04:10:15.377 INFO:tasks.workunit.client.1.smithi181.stdout:5/994: creat d4/de/d2c/d46/d4d/d8e/d54/f166 x:0 0 0 2024-04-27T04:10:15.377 INFO:tasks.workunit.client.0.smithi162.stdout:3/629: write d1/d21/d47/d57/f64 [2686690,56292] 0 2024-04-27T04:10:15.378 INFO:tasks.workunit.client.0.smithi162.stdout:1/625: sync 2024-04-27T04:10:15.385 INFO:tasks.workunit.client.0.smithi162.stdout:2/608: chown d0/d4/d2a/d3b/d49/d5a/d63/c9b 334936250 1 2024-04-27T04:10:15.385 INFO:tasks.workunit.client.1.smithi181.stdout:9/758: mkdir d0/d44/d4b/d43/d88/da1/dfc/dbd/df3/d102 0 2024-04-27T04:10:15.389 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:10:14 smithi162 ceph-mon[26841]: pgmap v7: 65 pgs: 65 active+clean; 2.8 GiB data, 9.8 GiB used, 527 GiB / 536 GiB avail; 64 MiB/s rd, 158 MiB/s wr, 359 op/s 2024-04-27T04:10:15.390 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:10:14 smithi162 ceph-mon[26841]: Standby manager daemon smithi162.oennfn started 2024-04-27T04:10:15.390 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:10:14 smithi162 ceph-mon[26841]: from='mgr.? 172.21.15.162:0/1013428318' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi162.oennfn/crt"}]: dispatch 2024-04-27T04:10:15.390 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:10:14 smithi162 ceph-mon[26841]: from='mgr.? 172.21.15.162:0/1013428318' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-27T04:10:15.390 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:10:14 smithi162 ceph-mon[26841]: from='mgr.? 172.21.15.162:0/1013428318' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi162.oennfn/key"}]: dispatch 2024-04-27T04:10:15.390 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:10:14 smithi162 ceph-mon[26841]: from='mgr.? 172.21.15.162:0/1013428318' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-27T04:10:15.390 INFO:tasks.workunit.client.0.smithi162.stdout:2/609: chown d0/d4/d17/d2b/d64/f90 55159903 1 2024-04-27T04:10:15.390 INFO:tasks.workunit.client.1.smithi181.stdout:3/789: symlink d0/d6c/d52/d44/lfa 0 2024-04-27T04:10:15.393 INFO:tasks.workunit.client.0.smithi162.stdout:8/694: truncate d0/de/d6e/f77 1023859 0 2024-04-27T04:10:15.394 INFO:tasks.workunit.client.0.smithi162.stdout:1/626: truncate d0/d80/f87 937646 0 2024-04-27T04:10:15.399 INFO:tasks.workunit.client.0.smithi162.stdout:2/610: mkdir d0/d4/d17/d2b/d64/dbd 0 2024-04-27T04:10:15.402 INFO:tasks.workunit.client.1.smithi181.stdout:3/790: symlink d0/d6c/d66/dc2/de1/lfb 0 2024-04-27T04:10:15.403 INFO:tasks.workunit.client.1.smithi181.stdout:9/759: dwrite d0/fe8 [0,4194304] 0 2024-04-27T04:10:15.404 INFO:tasks.workunit.client.0.smithi162.stdout:2/611: dwrite d0/d4/f16 [0,4194304] 0 2024-04-27T04:10:15.405 INFO:tasks.workunit.client.1.smithi181.stdout:5/995: mkdir d4/de/d2c/d46/d4d/d8e/d67/d167 0 2024-04-27T04:10:15.406 INFO:tasks.workunit.client.1.smithi181.stdout:5/996: chown d4/de/d2c/d46/d99/dad/fe1 10 1 2024-04-27T04:10:15.407 INFO:tasks.workunit.client.0.smithi162.stdout:4/619: dwrite d2/da/faf [0,4194304] 0 2024-04-27T04:10:15.416 INFO:tasks.workunit.client.1.smithi181.stdout:4/752: truncate d0/d15/d25/d2b/f2c 768446 0 2024-04-27T04:10:15.417 INFO:tasks.workunit.client.1.smithi181.stdout:3/791: fsync d0/d6c/d52/d44/dd5/d58/de4/daf/fc3 0 2024-04-27T04:10:15.418 INFO:tasks.workunit.client.1.smithi181.stdout:3/792: readlink d0/d6c/d52/d44/l82 0 2024-04-27T04:10:15.420 INFO:tasks.workunit.client.1.smithi181.stdout:9/760: symlink d0/d44/d4b/d5f/l103 0 2024-04-27T04:10:15.422 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:10:14 smithi181 ceph-mon[33774]: pgmap v7: 65 pgs: 65 active+clean; 2.8 GiB data, 9.8 GiB used, 527 GiB / 536 GiB avail; 64 MiB/s rd, 158 MiB/s wr, 359 op/s 2024-04-27T04:10:15.422 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:10:14 smithi181 ceph-mon[33774]: Standby manager daemon smithi162.oennfn started 2024-04-27T04:10:15.422 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:10:14 smithi181 ceph-mon[33774]: from='mgr.? 172.21.15.162:0/1013428318' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi162.oennfn/crt"}]: dispatch 2024-04-27T04:10:15.422 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:10:14 smithi181 ceph-mon[33774]: from='mgr.? 172.21.15.162:0/1013428318' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-27T04:10:15.422 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:10:14 smithi181 ceph-mon[33774]: from='mgr.? 172.21.15.162:0/1013428318' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi162.oennfn/key"}]: dispatch 2024-04-27T04:10:15.422 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:10:14 smithi181 ceph-mon[33774]: from='mgr.? 172.21.15.162:0/1013428318' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-27T04:10:15.423 INFO:tasks.workunit.client.1.smithi181.stdout:8/857: getdents d1/d6/d2f/d30/d66/dc0/dcd 0 2024-04-27T04:10:15.424 INFO:tasks.workunit.client.1.smithi181.stdout:7/803: dwrite d1/d2/d35/d67/f96 [0,4194304] 0 2024-04-27T04:10:15.426 INFO:tasks.workunit.client.0.smithi162.stdout:5/690: write d8/d1a/f31 [1848236,123404] 0 2024-04-27T04:10:15.430 INFO:tasks.workunit.client.1.smithi181.stdout:4/753: fdatasync d0/d15/d25/fcd 0 2024-04-27T04:10:15.435 INFO:tasks.workunit.client.1.smithi181.stdout:9/761: fdatasync d0/d44/d4b/d50/f6a 0 2024-04-27T04:10:15.450 INFO:tasks.workunit.client.1.smithi181.stdout:5/997: dread d4/de/d2c/d46/d99/dad/dca/fcc [0,4194304] 0 2024-04-27T04:10:15.453 INFO:tasks.workunit.client.1.smithi181.stdout:8/858: symlink d1/d6/d2f/d30/d9d/dea/dff/l12e 0 2024-04-27T04:10:15.457 INFO:tasks.workunit.client.1.smithi181.stdout:3/793: creat d0/d6c/ffc x:0 0 0 2024-04-27T04:10:15.458 INFO:tasks.workunit.client.1.smithi181.stdout:8/859: dread d1/d6/d2f/d30/d66/dc0/dcd/ff6 [0,4194304] 0 2024-04-27T04:10:15.460 INFO:tasks.workunit.client.0.smithi162.stdout:7/669: dwrite d3/d5/d5b/d83/de/f74 [0,4194304] 0 2024-04-27T04:10:15.461 INFO:tasks.workunit.client.1.smithi181.stdout:4/754: creat d0/d15/d25/d2a/d36/db8/dca/ff7 x:0 0 0 2024-04-27T04:10:15.461 INFO:tasks.workunit.client.0.smithi162.stdout:6/649: dwrite d6/d1a/f2b [0,4194304] 0 2024-04-27T04:10:15.462 INFO:tasks.workunit.client.1.smithi181.stdout:4/755: write d0/d15/d25/d2a/da0/dc7/fef [666197,58286] 0 2024-04-27T04:10:15.463 INFO:tasks.workunit.client.0.smithi162.stdout:7/670: truncate d3/d5/d5b/d60/f94 285225 0 2024-04-27T04:10:15.466 INFO:tasks.workunit.client.1.smithi181.stdout:9/762: getdents d0/d44/d4b/d50/dfd 0 2024-04-27T04:10:15.466 INFO:tasks.workunit.client.1.smithi181.stdout:0/958: write dc/d1f/d3b/d4a/d95/d26/d41/fc2 [1240057,111969] 0 2024-04-27T04:10:15.466 INFO:tasks.workunit.client.0.smithi162.stdout:0/747: write d5/dc/d1e/d46/d48/d9e/fdb [2146919,122044] 0 2024-04-27T04:10:15.476 INFO:tasks.workunit.client.1.smithi181.stdout:7/804: rename d1/d22/d15/d29/dbe/cb7 to d1/d2/d1c/d2d/cf0 0 2024-04-27T04:10:15.477 INFO:tasks.workunit.client.1.smithi181.stdout:6/693: dwrite d7/dc/d1c/d34/d3f/f6c [0,4194304] 0 2024-04-27T04:10:15.480 INFO:tasks.workunit.client.0.smithi162.stdout:3/630: write d1/d21/fb8 [561081,76174] 0 2024-04-27T04:10:15.481 INFO:tasks.workunit.client.1.smithi181.stdout:1/789: dwrite d1/dad/db5/d6/da3/d1e/d20/f102 [0,4194304] 0 2024-04-27T04:10:15.485 INFO:tasks.workunit.client.1.smithi181.stdout:4/756: dread d0/d15/de/d19/d77/d57/f84 [0,4194304] 0 2024-04-27T04:10:15.486 INFO:tasks.workunit.client.1.smithi181.stdout:1/790: dread - d1/dad/db5/d6/da3/d1e/d48/db4/f100 zero size 2024-04-27T04:10:15.486 INFO:tasks.workunit.client.1.smithi181.stdout:3/794: mknod d0/d6c/d52/d44/dd5/d58/de4/dc5/dcf/cfd 0 2024-04-27T04:10:15.500 INFO:tasks.workunit.client.1.smithi181.stdout:5/998: rename d4/d10 to d4/de/d2c/d46/d4d/d8e/d54/d168 0 2024-04-27T04:10:15.502 INFO:tasks.workunit.client.0.smithi162.stdout:8/695: truncate d0/de/d6e/f79 346376 0 2024-04-27T04:10:15.506 INFO:tasks.workunit.client.0.smithi162.stdout:9/663: getdents d3/d23 0 2024-04-27T04:10:15.509 INFO:tasks.workunit.client.1.smithi181.stdout:8/860: dread d1/d6/d2f/f3a [0,4194304] 0 2024-04-27T04:10:15.511 INFO:tasks.workunit.client.0.smithi162.stdout:2/612: rename d0/d4/d2a/d3b/d49/d5a/d63/c9b to d0/d4/d2a/d3b/d49/d5a/d7e/cbe 0 2024-04-27T04:10:15.513 INFO:tasks.workunit.client.0.smithi162.stdout:4/620: truncate d2/d25/d1c/f33 39278 0 2024-04-27T04:10:15.517 INFO:tasks.workunit.client.1.smithi181.stdout:9/763: rename d0/d40 to d0/d44/d4b/d43/d88/da1/dfc/dbd/dd1/dde/d104 0 2024-04-27T04:10:15.519 INFO:tasks.workunit.client.1.smithi181.stdout:5/999: symlink d4/de/d2c/d46/d99/dad/d13a/d80/l169 0 2024-04-27T04:10:15.520 INFO:tasks.workunit.client.1.smithi181.stdout:2/870: sync 2024-04-27T04:10:15.520 INFO:tasks.workunit.client.0.smithi162.stdout:7/671: fsync d3/d5/d5b/d83/f3f 0 2024-04-27T04:10:15.521 INFO:tasks.workunit.client.0.smithi162.stdout:7/672: chown d3/d5/d5b/d83/fb7 19251934 1 2024-04-27T04:10:15.522 INFO:tasks.workunit.client.1.smithi181.stdout:7/805: symlink d1/d22/d15/dc6/de5/lf1 0 2024-04-27T04:10:15.522 INFO:tasks.workunit.client.1.smithi181.stdout:2/871: write d4/d9/d16/d82/d7d/db7/dcb/f46 [3209467,7451] 0 2024-04-27T04:10:15.522 INFO:tasks.workunit.client.0.smithi162.stdout:7/673: write d3/d5/d5b/d83/f19 [3652657,4134] 0 2024-04-27T04:10:15.523 INFO:tasks.workunit.client.0.smithi162.stdout:0/748: unlink d5/dc/d1e/d20/l54 0 2024-04-27T04:10:15.525 INFO:tasks.workunit.client.0.smithi162.stdout:7/674: read d3/d5/d3c/d4a/d55/dc6/fcd [387,67565] 0 2024-04-27T04:10:15.526 INFO:tasks.workunit.client.0.smithi162.stdout:3/631: symlink d1/d21/d47/d57/d6e/ld8 0 2024-04-27T04:10:15.527 INFO:tasks.workunit.client.0.smithi162.stdout:3/632: write d1/d18/f6a [3066897,37715] 0 2024-04-27T04:10:15.527 INFO:tasks.workunit.client.1.smithi181.stdout:8/861: creat d1/d6/d2f/d30/d9d/f12f x:0 0 0 2024-04-27T04:10:15.528 INFO:tasks.workunit.client.0.smithi162.stdout:3/633: chown d1/d18/d23/d34/l8a 22 1 2024-04-27T04:10:15.529 INFO:tasks.workunit.client.0.smithi162.stdout:3/634: chown d1/d18/dac/fcf 770824 1 2024-04-27T04:10:15.530 INFO:tasks.workunit.client.1.smithi181.stdout:3/795: mknod d0/d6c/d52/d44/dd5/d58/df0/cfe 0 2024-04-27T04:10:15.531 INFO:tasks.workunit.client.1.smithi181.stdout:3/796: dread - d0/fda zero size 2024-04-27T04:10:15.534 INFO:tasks.workunit.client.0.smithi162.stdout:2/613: creat d0/d4/d2a/d3b/d49/d5a/d63/d84/fbf x:0 0 0 2024-04-27T04:10:15.535 INFO:tasks.workunit.client.1.smithi181.stdout:7/806: rmdir d1/d57 39 2024-04-27T04:10:15.535 INFO:tasks.workunit.client.0.smithi162.stdout:9/664: dread d3/d13/d2d/d3b/d69/f7d [0,4194304] 0 2024-04-27T04:10:15.535 INFO:tasks.workunit.client.0.smithi162.stdout:2/614: dread - d0/d4/d2a/da1/fa5 zero size 2024-04-27T04:10:15.537 INFO:tasks.workunit.client.1.smithi181.stdout:2/872: truncate d4/f30 4861632 0 2024-04-27T04:10:15.537 INFO:tasks.workunit.client.0.smithi162.stdout:5/691: rmdir d8/d1a/d29/d4a/d9f 39 2024-04-27T04:10:15.540 INFO:tasks.workunit.client.1.smithi181.stdout:8/862: mknod d1/d6/d6f/dd2/c130 0 2024-04-27T04:10:15.540 INFO:tasks.workunit.client.1.smithi181.stdout:8/863: stat d1/d20/df4/f10b 0 2024-04-27T04:10:15.542 INFO:tasks.workunit.client.1.smithi181.stdout:0/959: write dc/d1f/d6b/d7e/f10c [1107973,96163] 0 2024-04-27T04:10:15.544 INFO:tasks.workunit.client.0.smithi162.stdout:0/749: unlink d5/dc/d1e/d46/d48/fbb 0 2024-04-27T04:10:15.544 INFO:tasks.workunit.client.1.smithi181.stdout:2/873: dwrite d4/d9/d35/d9f/dd0/f109 [0,4194304] 0 2024-04-27T04:10:15.546 INFO:tasks.workunit.client.1.smithi181.stdout:3/797: fdatasync d0/d6c/d52/dbb/d98/fac 0 2024-04-27T04:10:15.550 INFO:tasks.workunit.client.0.smithi162.stdout:7/675: creat d3/d5/d5b/d83/de/d2b/da7/fe3 x:0 0 0 2024-04-27T04:10:15.550 INFO:tasks.workunit.client.1.smithi181.stdout:7/807: dread d1/d2/d1c/d37/f4f [0,4194304] 0 2024-04-27T04:10:15.557 INFO:tasks.workunit.client.0.smithi162.stdout:1/627: dwrite d0/d11/f20 [4194304,4194304] 0 2024-04-27T04:10:15.564 INFO:tasks.workunit.client.0.smithi162.stdout:2/615: mknod d0/d4/d17/cc0 0 2024-04-27T04:10:15.565 INFO:tasks.workunit.client.0.smithi162.stdout:5/692: fsync d8/d1a/d29/d4a/d5c/f62 0 2024-04-27T04:10:15.566 INFO:tasks.workunit.client.0.smithi162.stdout:5/693: chown d8/d1a/d23/f39 12761 1 2024-04-27T04:10:15.566 INFO:tasks.workunit.client.0.smithi162.stdout:2/616: truncate d0/d4/d17/d2b/fa9 588989 0 2024-04-27T04:10:15.566 INFO:tasks.workunit.client.0.smithi162.stdout:6/650: creat d6/d1a/fcc x:0 0 0 2024-04-27T04:10:15.568 INFO:tasks.workunit.client.0.smithi162.stdout:0/750: creat d5/dc/d1e/d20/dec/df1/db4/ffa x:0 0 0 2024-04-27T04:10:15.569 INFO:tasks.workunit.client.0.smithi162.stdout:1/628: mkdir d0/d7/d3c/d77/dd1 0 2024-04-27T04:10:15.570 INFO:tasks.workunit.client.0.smithi162.stdout:4/621: link d2/da/d1a/f30 d2/d25/d1c/d39/d3b/d47/fc3 0 2024-04-27T04:10:15.573 INFO:tasks.workunit.client.0.smithi162.stdout:2/617: creat d0/d4/d17/d2b/d64/d89/fc1 x:0 0 0 2024-04-27T04:10:15.574 INFO:tasks.workunit.client.0.smithi162.stdout:2/618: chown d0/d4/d2a/da1/d86 5 1 2024-04-27T04:10:15.575 INFO:tasks.workunit.client.0.smithi162.stdout:0/751: rename d5/dc/d1e/d41/f53 to d5/dc/d1e/ffb 0 2024-04-27T04:10:15.575 INFO:tasks.workunit.client.0.smithi162.stdout:3/635: link d1/d21/d47/cba d1/d45/cd9 0 2024-04-27T04:10:15.577 INFO:tasks.workunit.client.0.smithi162.stdout:9/665: creat d3/d11/d45/d4e/fcb x:0 0 0 2024-04-27T04:10:15.580 INFO:tasks.workunit.client.0.smithi162.stdout:7/676: creat d3/d5/d9/fe4 x:0 0 0 2024-04-27T04:10:15.581 INFO:tasks.workunit.client.0.smithi162.stdout:2/619: symlink d0/d24/lc2 0 2024-04-27T04:10:15.583 INFO:tasks.workunit.client.0.smithi162.stdout:3/636: rmdir d1/d70 39 2024-04-27T04:10:15.588 INFO:tasks.workunit.client.0.smithi162.stdout:2/620: unlink d0/d4/d17/l1d 0 2024-04-27T04:10:15.588 INFO:tasks.workunit.client.0.smithi162.stdout:2/621: readlink d0/d4/d17/d88/lb3 0 2024-04-27T04:10:15.589 INFO:tasks.workunit.client.0.smithi162.stdout:8/696: truncate d0/f7 3128319 0 2024-04-27T04:10:15.590 INFO:tasks.workunit.client.0.smithi162.stdout:4/622: creat d2/d25/d1c/d39/d3b/fc4 x:0 0 0 2024-04-27T04:10:15.594 INFO:tasks.workunit.client.0.smithi162.stdout:9/666: mkdir d3/db9/dcc 0 2024-04-27T04:10:15.600 INFO:tasks.workunit.client.0.smithi162.stdout:2/622: rmdir d0/d4/d17/d2b/d64/d89 39 2024-04-27T04:10:15.601 INFO:tasks.workunit.client.0.smithi162.stdout:7/677: creat d3/d5/d9/fe5 x:0 0 0 2024-04-27T04:10:15.602 INFO:tasks.workunit.client.0.smithi162.stdout:8/697: creat d0/d10/d2b/d2d/d2f/d6b/fe3 x:0 0 0 2024-04-27T04:10:15.604 INFO:tasks.workunit.client.0.smithi162.stdout:6/651: dwrite d6/d1a/fa6 [0,4194304] 0 2024-04-27T04:10:15.604 INFO:tasks.workunit.client.0.smithi162.stdout:9/667: rename d3/d11/d45/d4e/d58/f98 to d3/db9/dcc/fcd 0 2024-04-27T04:10:15.605 INFO:tasks.workunit.client.0.smithi162.stdout:5/694: truncate d8/d9/d5b/fb4 1563808 0 2024-04-27T04:10:15.607 INFO:tasks.workunit.client.0.smithi162.stdout:7/678: creat d3/d5/d5b/fe6 x:0 0 0 2024-04-27T04:10:15.611 INFO:tasks.workunit.client.0.smithi162.stdout:2/623: rename d0/d4/d2a/d3b/d49/d5a/d63/d84/d93 to d0/d4/d2a/d41/d77/d73/dc3 0 2024-04-27T04:10:15.614 INFO:tasks.workunit.client.0.smithi162.stdout:1/629: dwrite d0/d7/d5e/f98 [0,4194304] 0 2024-04-27T04:10:15.615 INFO:tasks.workunit.client.0.smithi162.stdout:9/668: creat d3/d13/d2d/fce x:0 0 0 2024-04-27T04:10:15.617 INFO:tasks.workunit.client.0.smithi162.stdout:0/752: creat d5/dc/d1e/d20/ffc x:0 0 0 2024-04-27T04:10:15.619 INFO:tasks.workunit.client.0.smithi162.stdout:3/637: dwrite d1/d45/f94 [0,4194304] 0 2024-04-27T04:10:15.622 INFO:tasks.workunit.client.0.smithi162.stdout:1/630: creat d0/d80/d89/fd2 x:0 0 0 2024-04-27T04:10:15.628 INFO:tasks.workunit.client.0.smithi162.stdout:0/753: write d5/dc/d1e/d46/d48/d9e/db8/fd2 [935929,52981] 0 2024-04-27T04:10:15.628 INFO:tasks.workunit.client.0.smithi162.stdout:9/669: creat d3/d13/d2d/d30/d7f/fcf x:0 0 0 2024-04-27T04:10:15.630 INFO:tasks.workunit.client.0.smithi162.stdout:5/695: unlink d8/d1a/d23/dbc/d66/l8c 0 2024-04-27T04:10:15.631 INFO:tasks.workunit.client.0.smithi162.stdout:4/623: dwrite d2/f7d [0,4194304] 0 2024-04-27T04:10:15.632 INFO:tasks.workunit.client.0.smithi162.stdout:3/638: fdatasync d1/d18/d23/f2b 0 2024-04-27T04:10:15.632 INFO:tasks.workunit.client.0.smithi162.stdout:2/624: dwrite d0/f1 [4194304,4194304] 0 2024-04-27T04:10:15.633 INFO:tasks.workunit.client.0.smithi162.stdout:8/698: link d0/de/d20/lcc d0/d10/d2b/le4 0 2024-04-27T04:10:15.634 INFO:tasks.workunit.client.0.smithi162.stdout:1/631: rmdir d0/d11/d44 39 2024-04-27T04:10:15.634 INFO:tasks.workunit.client.0.smithi162.stdout:5/696: write d8/d9/d55/f6d [1854681,75203] 0 2024-04-27T04:10:15.634 INFO:tasks.workunit.client.0.smithi162.stdout:9/670: write d3/d13/d2d/d30/fa4 [263090,63507] 0 2024-04-27T04:10:15.636 INFO:tasks.workunit.client.0.smithi162.stdout:7/679: creat d3/d5/d9/fe7 x:0 0 0 2024-04-27T04:10:15.637 INFO:tasks.workunit.client.0.smithi162.stdout:6/652: getdents d6/db/dc0 0 2024-04-27T04:10:15.640 INFO:tasks.workunit.client.0.smithi162.stdout:3/639: mknod d1/d18/d5d/d5f/cda 0 2024-04-27T04:10:15.640 INFO:tasks.workunit.client.0.smithi162.stdout:2/625: creat d0/d20/d51/fc4 x:0 0 0 2024-04-27T04:10:15.641 INFO:tasks.workunit.client.0.smithi162.stdout:1/632: mkdir d0/d11/d5b/dd3 0 2024-04-27T04:10:15.643 INFO:tasks.workunit.client.0.smithi162.stdout:4/624: readlink d2/d25/d1c/d39/d3b/d6e/l74 0 2024-04-27T04:10:15.644 INFO:tasks.workunit.client.0.smithi162.stdout:8/699: mknod d0/d10/d2b/d2d/d2f/d39/ce5 0 2024-04-27T04:10:15.644 INFO:tasks.workunit.client.0.smithi162.stdout:5/697: dwrite d8/d1a/f81 [0,4194304] 0 2024-04-27T04:10:15.645 INFO:tasks.workunit.client.0.smithi162.stdout:9/671: dread - d3/d23/d1e/f91 zero size 2024-04-27T04:10:15.646 INFO:tasks.workunit.client.0.smithi162.stdout:6/653: mkdir d6/d1a/d34/d7f/d80/db3/dcd 0 2024-04-27T04:10:15.648 INFO:tasks.workunit.client.0.smithi162.stdout:4/625: symlink d2/d25/d1c/d39/lc5 0 2024-04-27T04:10:15.648 INFO:tasks.workunit.client.0.smithi162.stdout:7/680: fsync d3/d5/d5b/d83/f19 0 2024-04-27T04:10:15.648 INFO:tasks.workunit.client.0.smithi162.stdout:2/626: write d0/d4/d17/d2b/d64/d89/fb9 [213202,109315] 0 2024-04-27T04:10:15.652 INFO:tasks.workunit.client.0.smithi162.stdout:3/640: dread d1/d21/d47/d57/f5e [0,4194304] 0 2024-04-27T04:10:15.652 INFO:tasks.workunit.client.0.smithi162.stdout:9/672: read - d3/d23/d7c/f9a zero size 2024-04-27T04:10:15.653 INFO:tasks.workunit.client.0.smithi162.stdout:2/627: rmdir d0 39 2024-04-27T04:10:15.653 INFO:tasks.workunit.client.0.smithi162.stdout:0/754: link d5/dc/d1e/d41/f4f d5/dc/d1e/d46/d48/d9e/dce/ffd 0 2024-04-27T04:10:15.654 INFO:tasks.workunit.client.0.smithi162.stdout:4/626: creat d2/d25/d1c/d39/d3b/d6e/d6f/fc6 x:0 0 0 2024-04-27T04:10:15.655 INFO:tasks.workunit.client.0.smithi162.stdout:1/633: link d0/lc8 d0/d11/d3e/ld4 0 2024-04-27T04:10:15.655 INFO:tasks.workunit.client.0.smithi162.stdout:7/681: dread - d3/d5/d5b/d83/de/d2b/d81/d96/f9e zero size 2024-04-27T04:10:15.656 INFO:tasks.workunit.client.0.smithi162.stdout:1/634: chown d0/d7/d10/lb3 41670841 1 2024-04-27T04:10:15.656 INFO:tasks.workunit.client.0.smithi162.stdout:7/682: fsync d3/d5/d5b/d83/f3f 0 2024-04-27T04:10:15.657 INFO:tasks.workunit.client.0.smithi162.stdout:8/700: rename d0/d10/d44/db2/fe0 to d0/de/fe6 0 2024-04-27T04:10:15.657 INFO:tasks.workunit.client.0.smithi162.stdout:3/641: symlink d1/d18/d23/d58/d63/ldb 0 2024-04-27T04:10:15.658 INFO:tasks.workunit.client.0.smithi162.stdout:1/635: readlink d0/d11/d50/d53/lc6 0 2024-04-27T04:10:15.659 INFO:tasks.workunit.client.0.smithi162.stdout:4/627: creat d2/d25/d1c/d39/d3b/d6e/d6f/fc7 x:0 0 0 2024-04-27T04:10:15.659 INFO:tasks.workunit.client.0.smithi162.stdout:7/683: stat d3/d5/d5b/d83/de/d2b/da7/fc2 0 2024-04-27T04:10:15.659 INFO:tasks.workunit.client.0.smithi162.stdout:8/701: write d0/d10/d25/d7d/d87/fc5 [1030134,12255] 0 2024-04-27T04:10:15.660 INFO:tasks.workunit.client.0.smithi162.stdout:0/755: stat d5/d26/da2/fc4 0 2024-04-27T04:10:15.660 INFO:tasks.workunit.client.0.smithi162.stdout:2/628: getdents d0/d4/d2a/d41/d77/d73/dc3/d7d 0 2024-04-27T04:10:15.662 INFO:tasks.workunit.client.0.smithi162.stdout:7/684: chown d3/d5/d9/fe4 606 1 2024-04-27T04:10:15.663 INFO:tasks.workunit.client.0.smithi162.stdout:8/702: write d0/d10/d25/f30 [8742538,76465] 0 2024-04-27T04:10:15.663 INFO:tasks.workunit.client.0.smithi162.stdout:3/642: rename d1/d18/d23/d34/c3a to d1/d18/d5d/d5f/d66/cdc 0 2024-04-27T04:10:15.664 INFO:tasks.workunit.client.0.smithi162.stdout:7/685: dread - d3/d5/d5b/d83/fb3 zero size 2024-04-27T04:10:15.664 INFO:tasks.workunit.client.0.smithi162.stdout:2/629: mknod d0/d4/d2a/d3b/d49/d5a/d7e/cc5 0 2024-04-27T04:10:15.665 INFO:tasks.workunit.client.0.smithi162.stdout:7/686: dread - d3/d5/d5b/d83/de/fbb zero size 2024-04-27T04:10:15.689 INFO:tasks.workunit.client.1.smithi181.stdout:6/694: write d7/dc/d1c/d34/d3f/f9f [4314939,81412] 0 2024-04-27T04:10:15.693 INFO:tasks.workunit.client.1.smithi181.stdout:0/960: write dc/d1f/d3b/f5d [5081099,18870] 0 2024-04-27T04:10:15.693 INFO:tasks.workunit.client.1.smithi181.stdout:3/798: creat d0/d6c/d52/d44/dd5/d58/d64/fff x:0 0 0 2024-04-27T04:10:15.695 INFO:tasks.workunit.client.1.smithi181.stdout:9/764: getdents d0/d44/d4b/d43/d88/da1/dfc/dbd/df3/da0/ddb 0 2024-04-27T04:10:15.702 INFO:tasks.workunit.client.1.smithi181.stdout:6/695: unlink d7/dc/d1c/d34/c5f 0 2024-04-27T04:10:15.702 INFO:tasks.workunit.client.1.smithi181.stdout:2/874: mknod d4/d9/d16/d82/d7d/dbe/d10b/c117 0 2024-04-27T04:10:15.704 INFO:tasks.workunit.client.1.smithi181.stdout:7/808: fdatasync d1/d22/d15/dc6/dd8/f66 0 2024-04-27T04:10:15.707 INFO:tasks.workunit.client.1.smithi181.stdout:3/799: mkdir d0/d6c/d66/dc2/d9e/d100 0 2024-04-27T04:10:15.707 INFO:tasks.workunit.client.1.smithi181.stdout:1/791: dwrite d1/dad/db5/d6/da3/f8b [0,4194304] 0 2024-04-27T04:10:15.709 INFO:tasks.workunit.client.1.smithi181.stdout:6/696: chown d7/dc/d1c/d7f/le5 16366 1 2024-04-27T04:10:15.711 INFO:tasks.workunit.client.1.smithi181.stdout:4/757: dwrite d0/d15/de/d19/d77/d57/f9f [4194304,4194304] 0 2024-04-27T04:10:15.713 INFO:tasks.workunit.client.1.smithi181.stdout:9/765: mknod d0/d44/d4b/d43/d88/de3/c105 0 2024-04-27T04:10:15.715 INFO:tasks.workunit.client.1.smithi181.stdout:7/809: mknod d1/d22/d15/d1f/cf2 0 2024-04-27T04:10:15.716 INFO:tasks.workunit.client.1.smithi181.stdout:1/792: creat d1/dad/db5/d6/dcb/ded/f104 x:0 0 0 2024-04-27T04:10:15.718 INFO:tasks.workunit.client.1.smithi181.stdout:1/793: stat d1/dad/db5/d6/da3/d52/cec 0 2024-04-27T04:10:15.718 INFO:tasks.workunit.client.1.smithi181.stdout:4/758: mkdir d0/d15/d25/d2a/df8 0 2024-04-27T04:10:15.721 INFO:tasks.workunit.client.1.smithi181.stdout:9/766: creat d0/d44/d4b/d43/d88/da1/dfc/dbd/df3/da0/ddb/f106 x:0 0 0 2024-04-27T04:10:15.722 INFO:tasks.workunit.client.1.smithi181.stdout:9/767: write d0/f68 [275485,44673] 0 2024-04-27T04:10:15.722 INFO:tasks.workunit.client.1.smithi181.stdout:3/800: symlink d0/d6c/d52/d44/dd5/d58/d64/dc6/ddd/l101 0 2024-04-27T04:10:15.723 INFO:tasks.workunit.client.1.smithi181.stdout:7/810: mkdir d1/d22/d15/d5f/d93/df3 0 2024-04-27T04:10:15.725 INFO:tasks.workunit.client.1.smithi181.stdout:4/759: symlink d0/d15/d25/d2a/lf9 0 2024-04-27T04:10:15.726 INFO:tasks.workunit.client.1.smithi181.stdout:1/794: fdatasync d1/dad/db5/d6/da3/d1e/d20/da2/fa7 0 2024-04-27T04:10:15.726 INFO:tasks.workunit.client.1.smithi181.stdout:4/760: stat d0/d15/d25/d2b/f5f 0 2024-04-27T04:10:15.728 INFO:tasks.workunit.client.1.smithi181.stdout:4/761: write d0/d15/de/d19/d8a/fa9 [967609,64866] 0 2024-04-27T04:10:15.729 INFO:tasks.workunit.client.1.smithi181.stdout:9/768: creat d0/d44/d4b/d43/d88/f107 x:0 0 0 2024-04-27T04:10:15.730 INFO:tasks.workunit.client.1.smithi181.stdout:7/811: creat d1/d22/d15/dc6/de5/ff4 x:0 0 0 2024-04-27T04:10:15.731 INFO:tasks.workunit.client.1.smithi181.stdout:0/961: rmdir dc/d1f/d3b/d4a/d95/d26/d131/d106/d110/d11c 0 2024-04-27T04:10:15.735 INFO:tasks.workunit.client.1.smithi181.stdout:1/795: dwrite d1/d2/d64/ddc/ff3 [0,4194304] 0 2024-04-27T04:10:15.738 INFO:tasks.workunit.client.1.smithi181.stdout:7/812: getdents d1/d2/d1c/d37 0 2024-04-27T04:10:15.740 INFO:tasks.workunit.client.0.smithi162.stdout:8/703: mkdir d0/d10/d2b/d2e/de7 0 2024-04-27T04:10:15.741 INFO:tasks.workunit.client.1.smithi181.stdout:1/796: truncate d1/dad/db5/d6/da3/d52/f9e 919312 0 2024-04-27T04:10:15.741 INFO:tasks.workunit.client.1.smithi181.stdout:7/813: rmdir d1/d57/d72 39 2024-04-27T04:10:15.741 INFO:tasks.workunit.client.1.smithi181.stdout:0/962: dread dc/d1f/f7c [0,4194304] 0 2024-04-27T04:10:15.742 INFO:tasks.workunit.client.0.smithi162.stdout:8/704: dread - d0/de/d20/d7e/fd1 zero size 2024-04-27T04:10:15.744 INFO:tasks.workunit.client.1.smithi181.stdout:1/797: unlink d1/dad/db5/l37 0 2024-04-27T04:10:15.749 INFO:tasks.workunit.client.1.smithi181.stdout:1/798: fdatasync d1/dad/db5/d6/da3/f1f 0 2024-04-27T04:10:15.750 INFO:tasks.workunit.client.0.smithi162.stdout:8/705: dread d0/de/d20/d6c/fa5 [0,4194304] 0 2024-04-27T04:10:15.750 INFO:tasks.workunit.client.1.smithi181.stdout:7/814: creat d1/d22/d15/d5f/d93/df3/ff5 x:0 0 0 2024-04-27T04:10:15.751 INFO:tasks.workunit.client.1.smithi181.stdout:1/799: dread - d1/dad/db5/d6/da3/d1e/d48/db4/f100 zero size 2024-04-27T04:10:15.752 INFO:tasks.workunit.client.1.smithi181.stdout:0/963: creat dc/d1f/d3b/d4a/d95/d26/d131/dba/deb/dfd/f14a x:0 0 0 2024-04-27T04:10:15.752 INFO:tasks.workunit.client.0.smithi162.stdout:4/628: truncate d2/d25/d1c/d24/fbc 4096353 0 2024-04-27T04:10:15.753 INFO:tasks.workunit.client.1.smithi181.stdout:4/762: dread d0/d15/f1a [4194304,4194304] 0 2024-04-27T04:10:15.757 INFO:tasks.workunit.client.1.smithi181.stdout:7/815: read d1/d22/d15/d29/f84 [824462,115864] 0 2024-04-27T04:10:15.758 INFO:tasks.workunit.client.1.smithi181.stdout:7/816: write d1/d2/d35/d67/f96 [1776505,113759] 0 2024-04-27T04:10:15.760 INFO:tasks.workunit.client.0.smithi162.stdout:0/756: rename d5/dc/d1e/d20/dba/df9/c88 to d5/dc/cfe 0 2024-04-27T04:10:15.762 INFO:tasks.workunit.client.0.smithi162.stdout:4/629: symlink d2/d25/lc8 0 2024-04-27T04:10:15.766 INFO:tasks.workunit.client.0.smithi162.stdout:2/630: creat d0/d20/fc6 x:0 0 0 2024-04-27T04:10:15.766 INFO:tasks.workunit.client.0.smithi162.stdout:4/630: read d2/f38 [5455448,120324] 0 2024-04-27T04:10:15.766 INFO:tasks.workunit.client.0.smithi162.stdout:3/643: link d1/d18/c5c d1/d70/cdd 0 2024-04-27T04:10:15.768 INFO:tasks.workunit.client.1.smithi181.stdout:1/800: rename d1/dad/db5/d6/da3/d1e/d20/d53/d56/dd9/feb to d1/dad/db5/d6/dcb/ded/f105 0 2024-04-27T04:10:15.769 INFO:tasks.workunit.client.0.smithi162.stdout:0/757: unlink d5/dc/fd9 0 2024-04-27T04:10:15.771 INFO:tasks.workunit.client.1.smithi181.stdout:4/763: link d0/d15/de/f43 d0/d15/dd2/deb/db9/ffa 0 2024-04-27T04:10:15.771 INFO:tasks.workunit.client.1.smithi181.stdout:0/964: link dc/d1f/d3b/d4a/c65 dc/d1f/d115/d140/c14b 0 2024-04-27T04:10:15.772 INFO:tasks.workunit.client.1.smithi181.stdout:0/965: stat dc/d1f/d3b/db2/cd3 0 2024-04-27T04:10:15.774 INFO:tasks.workunit.client.0.smithi162.stdout:4/631: mknod d2/d25/d1c/d24/db5/cc9 0 2024-04-27T04:10:15.774 INFO:tasks.workunit.client.1.smithi181.stdout:1/801: mknod d1/dad/db5/d6/db/c106 0 2024-04-27T04:10:15.775 INFO:tasks.workunit.client.1.smithi181.stdout:1/802: chown d1/dad/db5/d6/da3/d1e 178083 1 2024-04-27T04:10:15.776 INFO:tasks.workunit.client.1.smithi181.stdout:1/803: truncate d1/dad/fdb 302956 0 2024-04-27T04:10:15.779 INFO:tasks.workunit.client.0.smithi162.stdout:2/631: creat d0/d4/d17/db4/fc7 x:0 0 0 2024-04-27T04:10:15.779 INFO:tasks.workunit.client.0.smithi162.stdout:4/632: mkdir d2/d54/dca 0 2024-04-27T04:10:15.780 INFO:tasks.workunit.client.1.smithi181.stdout:0/966: creat dc/d1f/d115/d140/f14c x:0 0 0 2024-04-27T04:10:15.781 INFO:tasks.workunit.client.0.smithi162.stdout:3/644: mknod d1/d18/d23/d34/dad/dc5/cde 0 2024-04-27T04:10:15.782 INFO:tasks.workunit.client.1.smithi181.stdout:1/804: rename d1/dad/db5/d6/da3/d1e/d48/db4/cd0 to d1/dad/db5/d6/da3/d52/c107 0 2024-04-27T04:10:15.784 INFO:tasks.workunit.client.0.smithi162.stdout:7/687: dread d3/d4f/f31 [0,4194304] 0 2024-04-27T04:10:15.786 INFO:tasks.workunit.client.0.smithi162.stdout:2/632: dread d0/d4/d2a/f36 [4194304,4194304] 0 2024-04-27T04:10:15.787 INFO:tasks.workunit.client.1.smithi181.stdout:1/805: rmdir d1/dad/db5/d6/da3/d1e/dbc 39 2024-04-27T04:10:15.789 INFO:tasks.workunit.client.0.smithi162.stdout:0/758: dread d5/dc/f17 [4194304,4194304] 0 2024-04-27T04:10:15.789 INFO:tasks.workunit.client.0.smithi162.stdout:4/633: creat d2/d25/d1c/d39/d3b/da7/fcb x:0 0 0 2024-04-27T04:10:15.790 INFO:tasks.workunit.client.0.smithi162.stdout:7/688: truncate d3/d5/d5b/d83/de/d2b/d81/d96/fb4 4542823 0 2024-04-27T04:10:15.790 INFO:tasks.workunit.client.1.smithi181.stdout:0/967: dread dc/d1f/d3b/d4a/d95/d26/d41/f4f [0,4194304] 0 2024-04-27T04:10:15.791 INFO:tasks.workunit.client.1.smithi181.stdout:1/806: mknod d1/dad/db5/d6/dcb/c108 0 2024-04-27T04:10:15.794 INFO:tasks.workunit.client.0.smithi162.stdout:0/759: stat d5/dc/d1e/faf 0 2024-04-27T04:10:15.795 INFO:tasks.workunit.client.1.smithi181.stdout:0/968: rename dc/d1f/d3b/db2/fa4 to dc/d1f/d3b/d4a/d95/d26/d41/d6f/d8f/f14d 0 2024-04-27T04:10:15.796 INFO:tasks.workunit.client.0.smithi162.stdout:7/689: rmdir d3/d5/d3c 39 2024-04-27T04:10:15.797 INFO:tasks.workunit.client.1.smithi181.stdout:1/807: write d1/dad/db5/d6/da3/d52/d92/f93 [1634877,43265] 0 2024-04-27T04:10:15.797 INFO:tasks.workunit.client.0.smithi162.stdout:7/690: write d3/d5/d5b/d83/de/d2b/d81/d96/fd0 [443640,74768] 0 2024-04-27T04:10:15.798 INFO:tasks.workunit.client.0.smithi162.stdout:7/691: write d3/d5/d5b/fe6 [994098,111230] 0 2024-04-27T04:10:15.801 INFO:tasks.workunit.client.1.smithi181.stdout:0/969: fsync dc/f5e 0 2024-04-27T04:10:15.802 INFO:tasks.workunit.client.0.smithi162.stdout:3/645: getdents d1/d21/d47 0 2024-04-27T04:10:15.802 INFO:tasks.workunit.client.1.smithi181.stdout:2/875: dwrite d4/d9/f67 [0,4194304] 0 2024-04-27T04:10:15.803 INFO:tasks.workunit.client.0.smithi162.stdout:0/760: chown d5/dc/d1e/d56/d57/fda 67734 1 2024-04-27T04:10:15.805 INFO:tasks.workunit.client.1.smithi181.stdout:1/808: creat d1/dad/db5/d6/da3/d60/f109 x:0 0 0 2024-04-27T04:10:15.808 INFO:tasks.workunit.client.0.smithi162.stdout:7/692: write d3/d5/d5b/d83/de/f38 [1535952,44885] 0 2024-04-27T04:10:15.810 INFO:tasks.workunit.client.1.smithi181.stdout:6/697: truncate d7/d27/fc9 757414 0 2024-04-27T04:10:15.811 INFO:tasks.workunit.client.1.smithi181.stdout:0/970: dwrite dc/d1f/d3b/d4a/d95/d26/d41/d6f/d8f/f137 [0,4194304] 0 2024-04-27T04:10:15.811 INFO:tasks.workunit.client.0.smithi162.stdout:3/646: dread - d1/d45/f7e zero size 2024-04-27T04:10:15.814 INFO:tasks.workunit.client.1.smithi181.stdout:1/809: fsync d1/dad/db5/d6/da3/f1f 0 2024-04-27T04:10:15.816 INFO:tasks.workunit.client.0.smithi162.stdout:2/633: getdents d0/d4/d17/d2b/d64/d89 0 2024-04-27T04:10:15.817 INFO:tasks.workunit.client.1.smithi181.stdout:1/810: truncate d1/d2/d89/fef 483010 0 2024-04-27T04:10:15.821 INFO:tasks.workunit.client.1.smithi181.stdout:3/801: dwrite d0/d6c/fb1 [0,4194304] 0 2024-04-27T04:10:15.822 INFO:tasks.workunit.client.1.smithi181.stdout:6/698: rmdir d7/dc/d1c/d34 39 2024-04-27T04:10:15.825 INFO:tasks.workunit.client.0.smithi162.stdout:3/647: fsync d1/d21/f2c 0 2024-04-27T04:10:15.829 INFO:tasks.workunit.client.0.smithi162.stdout:0/761: symlink d5/d26/da2/lff 0 2024-04-27T04:10:15.833 INFO:tasks.workunit.client.1.smithi181.stdout:7/817: getdents d1/d22/d15/d5f/d93/df3 0 2024-04-27T04:10:15.835 INFO:tasks.workunit.client.0.smithi162.stdout:2/634: dread d0/d4/d2a/d3b/f74 [0,4194304] 0 2024-04-27T04:10:15.836 INFO:tasks.workunit.client.1.smithi181.stdout:3/802: readlink d0/d6c/d52/d44/d56/d5b/l92 0 2024-04-27T04:10:15.840 INFO:tasks.workunit.client.0.smithi162.stdout:8/706: dwrite d0/d10/d44/f47 [0,4194304] 0 2024-04-27T04:10:15.842 INFO:tasks.workunit.client.0.smithi162.stdout:0/762: unlink d5/dc/f14 0 2024-04-27T04:10:15.847 INFO:tasks.workunit.client.0.smithi162.stdout:2/635: symlink d0/d4/d2a/d3b/d49/d5a/d63/d84/lc8 0 2024-04-27T04:10:15.847 INFO:tasks.workunit.client.0.smithi162.stdout:8/707: dwrite d0/d10/faf [0,4194304] 0 2024-04-27T04:10:15.848 INFO:tasks.workunit.client.1.smithi181.stdout:3/803: truncate d0/d6c/d52/d44/dd5/fee 484437 0 2024-04-27T04:10:15.849 INFO:tasks.workunit.client.0.smithi162.stdout:0/763: dwrite d5/dc/d1e/d41/f34 [0,4194304] 0 2024-04-27T04:10:15.850 INFO:tasks.workunit.client.0.smithi162.stdout:8/708: truncate d0/d10/d76/fb7 4345297 0 2024-04-27T04:10:15.858 INFO:tasks.workunit.client.0.smithi162.stdout:2/636: mkdir d0/d4/d17/d88/dc9 0 2024-04-27T04:10:15.863 INFO:tasks.workunit.client.0.smithi162.stdout:2/637: mkdir d0/d4/d2a/da1/dca 0 2024-04-27T04:10:15.865 INFO:tasks.workunit.client.0.smithi162.stdout:3/648: getdents d1/d18/d23/d34/dad/dc5 0 2024-04-27T04:10:15.865 INFO:tasks.workunit.client.1.smithi181.stdout:4/764: dwrite d0/d15/f2 [4194304,4194304] 0 2024-04-27T04:10:15.866 INFO:tasks.workunit.client.0.smithi162.stdout:2/638: write d0/d4/d2a/d41/d77/f7c [1254528,110115] 0 2024-04-27T04:10:15.867 INFO:tasks.workunit.client.0.smithi162.stdout:2/639: chown d0/d4/d2a/d3b/d49/d5a/d63/d6e/f98 168623713 1 2024-04-27T04:10:15.868 INFO:tasks.workunit.client.0.smithi162.stdout:2/640: fsync d0/d4/f15 0 2024-04-27T04:10:15.868 INFO:tasks.workunit.client.0.smithi162.stdout:3/649: rmdir d1/d21/d47 39 2024-04-27T04:10:15.873 INFO:tasks.workunit.client.0.smithi162.stdout:2/641: mknod d0/d24/ccb 0 2024-04-27T04:10:15.875 INFO:tasks.workunit.client.0.smithi162.stdout:3/650: creat d1/d18/d23/d34/dad/fdf x:0 0 0 2024-04-27T04:10:15.876 INFO:tasks.workunit.client.1.smithi181.stdout:0/971: rmdir dc/d1f/d3b/d4a/d95/d26 39 2024-04-27T04:10:15.878 INFO:tasks.workunit.client.0.smithi162.stdout:2/642: unlink d0/d4/c33 0 2024-04-27T04:10:15.879 INFO:tasks.workunit.client.0.smithi162.stdout:3/651: readlink d1/d18/d23/l67 0 2024-04-27T04:10:15.882 INFO:tasks.workunit.client.0.smithi162.stdout:2/643: unlink d0/d4/d2a/d3b/d49/l47 0 2024-04-27T04:10:15.883 INFO:tasks.workunit.client.1.smithi181.stdout:0/972: mkdir dc/d1f/d3b/d4a/d95/d132/d14e 0 2024-04-27T04:10:15.885 INFO:tasks.workunit.client.1.smithi181.stdout:2/876: write d4/d9/d4f/d79/d94/f95 [330317,37792] 0 2024-04-27T04:10:15.888 INFO:tasks.workunit.client.1.smithi181.stdout:1/811: truncate d1/dad/db5/d6/da3/f1f 960129 0 2024-04-27T04:10:15.890 INFO:tasks.workunit.client.0.smithi162.stdout:2/644: truncate d0/d4/d17/f2f 1453038 0 2024-04-27T04:10:15.890 INFO:tasks.workunit.client.0.smithi162.stdout:7/693: truncate d3/d5/d5b/d83/de/f38 1007707 0 2024-04-27T04:10:15.891 INFO:tasks.workunit.client.1.smithi181.stdout:1/812: dread d1/dad/db5/fb8 [0,4194304] 0 2024-04-27T04:10:15.891 INFO:tasks.workunit.client.1.smithi181.stdout:2/877: symlink d4/d9/d16/d82/d7d/dbe/l118 0 2024-04-27T04:10:15.892 INFO:tasks.workunit.client.1.smithi181.stdout:6/699: write d7/dc/d1c/d88/d6d/fb5 [544717,38886] 0 2024-04-27T04:10:15.895 INFO:tasks.workunit.client.0.smithi162.stdout:2/645: creat d0/d4/d17/d2b/d64/d89/fcc x:0 0 0 2024-04-27T04:10:15.896 INFO:tasks.workunit.client.0.smithi162.stdout:2/646: fdatasync d0/d4/d2a/d41/d77/d73/fb1 0 2024-04-27T04:10:15.898 INFO:tasks.workunit.client.0.smithi162.stdout:2/647: write d0/d4/d2a/da1/fa5 [586216,94695] 0 2024-04-27T04:10:15.898 INFO:tasks.workunit.client.1.smithi181.stdout:7/818: dwrite d1/d2/d1c/f1e [0,4194304] 0 2024-04-27T04:10:15.900 INFO:tasks.workunit.client.1.smithi181.stdout:7/819: dread - d1/d22/d15/d5f/d93/df3/ff5 zero size 2024-04-27T04:10:15.902 INFO:tasks.workunit.client.0.smithi162.stdout:7/694: unlink d3/d5/d9/f17 0 2024-04-27T04:10:15.904 INFO:tasks.workunit.client.1.smithi181.stdout:2/878: mknod d4/d9/d35/d42/d86/c119 0 2024-04-27T04:10:15.906 INFO:tasks.workunit.client.0.smithi162.stdout:8/709: write d0/fd [1771507,122006] 0 2024-04-27T04:10:15.907 INFO:tasks.workunit.client.0.smithi162.stdout:2/648: write d0/d4/d17/d4f/f85 [1896031,45981] 0 2024-04-27T04:10:15.908 INFO:tasks.workunit.client.0.smithi162.stdout:0/764: write d5/dc/d1e/d46/d48/d9e/db8/f92 [888809,20934] 0 2024-04-27T04:10:15.909 INFO:tasks.workunit.client.0.smithi162.stdout:7/695: truncate d3/d5/d5b/d83/de/d2b/f69 650927 0 2024-04-27T04:10:15.909 INFO:tasks.workunit.client.0.smithi162.stdout:0/765: stat d5/d26/d4a 0 2024-04-27T04:10:15.910 INFO:tasks.workunit.client.0.smithi162.stdout:7/696: fsync d3/d39/fc9 0 2024-04-27T04:10:15.910 INFO:tasks.workunit.client.1.smithi181.stdout:4/765: write d0/d15/de/f34 [5979858,36302] 0 2024-04-27T04:10:15.911 INFO:tasks.workunit.client.1.smithi181.stdout:6/700: dread d7/dc/f3a [0,4194304] 0 2024-04-27T04:10:15.912 INFO:tasks.workunit.client.1.smithi181.stdout:6/701: fsync d7/dc/dc3/d72/d98/dbf/fc1 0 2024-04-27T04:10:15.912 INFO:tasks.workunit.client.1.smithi181.stdout:6/702: stat d7/dc/d1c/fa8 0 2024-04-27T04:10:15.912 INFO:tasks.workunit.client.1.smithi181.stdout:1/813: creat d1/dad/db5/d6/da3/f10a x:0 0 0 2024-04-27T04:10:15.914 INFO:tasks.workunit.client.0.smithi162.stdout:0/766: read d5/dc/f74 [489619,85158] 0 2024-04-27T04:10:15.915 INFO:tasks.workunit.client.0.smithi162.stdout:8/710: mknod d0/de/d6e/d71/d86/ddc/ce8 0 2024-04-27T04:10:15.916 INFO:tasks.workunit.client.1.smithi181.stdout:2/879: symlink d4/d9/l11a 0 2024-04-27T04:10:15.918 INFO:tasks.workunit.client.1.smithi181.stdout:4/766: chown d0/d15/d5c/d69 763223 1 2024-04-27T04:10:15.922 INFO:tasks.workunit.client.0.smithi162.stdout:2/649: rename d0/d4/d17/d4f/f85 to d0/d20/d51/fcd 0 2024-04-27T04:10:15.922 INFO:tasks.workunit.client.0.smithi162.stdout:2/650: fsync d0/d20/fc6 0 2024-04-27T04:10:15.926 INFO:tasks.workunit.client.0.smithi162.stdout:0/767: mknod d5/dc/d1e/d20/dba/df9/c100 0 2024-04-27T04:10:15.927 INFO:tasks.workunit.client.1.smithi181.stdout:0/973: truncate dc/d1f/d6b/fc0 2342026 0 2024-04-27T04:10:15.928 INFO:tasks.workunit.client.0.smithi162.stdout:3/652: dwrite d1/d65/f91 [0,4194304] 0 2024-04-27T04:10:15.929 INFO:tasks.workunit.client.0.smithi162.stdout:8/711: truncate d0/de/d20/d6c/fa5 3078819 0 2024-04-27T04:10:15.930 INFO:tasks.workunit.client.0.smithi162.stdout:0/768: truncate d5/dc/d1e/d46/d48/d9e/db8/fd2 1760569 0 2024-04-27T04:10:15.931 INFO:tasks.workunit.client.0.smithi162.stdout:0/769: fdatasync d5/dc/d1e/d56/d57/fda 0 2024-04-27T04:10:15.933 INFO:tasks.workunit.client.0.smithi162.stdout:0/770: stat d5/d26/d4a/ced 0 2024-04-27T04:10:15.933 INFO:tasks.workunit.client.0.smithi162.stdout:7/697: rmdir d3/d5/d3c/d4a/d55/dc6 39 2024-04-27T04:10:15.935 INFO:tasks.workunit.client.1.smithi181.stdout:0/974: dwrite dc/d1f/d3b/d4a/d7a/d146/f129 [0,4194304] 0 2024-04-27T04:10:15.938 INFO:tasks.workunit.client.1.smithi181.stdout:2/880: read d4/d9/d4f/d79/d98/fa2 [2204271,71061] 0 2024-04-27T04:10:15.938 INFO:tasks.workunit.client.0.smithi162.stdout:2/651: rmdir d0 39 2024-04-27T04:10:15.943 INFO:tasks.workunit.client.1.smithi181.stdout:1/814: mknod d1/d2/c10b 0 2024-04-27T04:10:15.947 INFO:tasks.workunit.client.1.smithi181.stdout:7/820: link d1/df/fe7 d1/d2/d1c/d8f/ff6 0 2024-04-27T04:10:15.948 INFO:tasks.workunit.client.1.smithi181.stdout:7/821: dread - d1/d22/d15/dc6/dd8/fe4 zero size 2024-04-27T04:10:15.949 INFO:tasks.workunit.client.0.smithi162.stdout:7/698: mknod d3/d5/d5b/d83/de/d2b/d81/daa/dbd/ce8 0 2024-04-27T04:10:15.952 INFO:tasks.workunit.client.0.smithi162.stdout:0/771: rename d5/dc/d1e/d56/c8e to d5/dc/d1e/d46/d48/d9e/dce/dd1/c101 0 2024-04-27T04:10:15.955 INFO:tasks.workunit.client.1.smithi181.stdout:6/703: dwrite d7/fb9 [0,4194304] 0 2024-04-27T04:10:15.960 INFO:tasks.workunit.client.1.smithi181.stdout:4/767: creat d0/d15/de/d19/d77/dc5/ffb x:0 0 0 2024-04-27T04:10:15.962 INFO:tasks.workunit.client.1.smithi181.stdout:1/815: fsync d1/dad/db5/d6/da3/d1e/d48/da6/fd4 0 2024-04-27T04:10:15.964 INFO:tasks.workunit.client.1.smithi181.stdout:6/704: dwrite d7/dc/d1c/d88/db3/fe4 [0,4194304] 0 2024-04-27T04:10:15.971 INFO:tasks.workunit.client.1.smithi181.stdout:0/975: creat dc/d1f/d3b/d4a/d95/d26/d68/f14f x:0 0 0 2024-04-27T04:10:15.978 INFO:tasks.workunit.client.0.smithi162.stdout:3/653: link d1/d18/f41 d1/d45/fe0 0 2024-04-27T04:10:15.980 INFO:tasks.workunit.client.0.smithi162.stdout:8/712: creat d0/d10/d25/d65/fe9 x:0 0 0 2024-04-27T04:10:15.981 INFO:tasks.workunit.client.0.smithi162.stdout:8/713: write d0/fa7 [4618148,72547] 0 2024-04-27T04:10:15.981 INFO:tasks.workunit.client.0.smithi162.stdout:7/699: write d3/d5/d3c/d4a/d55/dc6/fcd [322496,69160] 0 2024-04-27T04:10:15.982 INFO:tasks.workunit.client.1.smithi181.stdout:6/705: truncate d7/dc/d1c/d7f/d3e/f50 173308 0 2024-04-27T04:10:15.984 INFO:tasks.workunit.client.0.smithi162.stdout:2/652: mknod d0/d4/d17/d2b/d64/cce 0 2024-04-27T04:10:15.985 INFO:tasks.workunit.client.1.smithi181.stdout:0/976: fsync dc/d1f/d115/f104 0 2024-04-27T04:10:15.987 INFO:tasks.workunit.client.1.smithi181.stdout:2/881: creat d4/d9/f11b x:0 0 0 2024-04-27T04:10:15.989 INFO:tasks.workunit.client.1.smithi181.stdout:7/822: dwrite d1/d2/d1c/d99/fd2 [0,4194304] 0 2024-04-27T04:10:15.993 INFO:tasks.workunit.client.0.smithi162.stdout:7/700: unlink d3/d5/ca8 0 2024-04-27T04:10:15.993 INFO:tasks.workunit.client.1.smithi181.stdout:6/706: rmdir d7/dc/dc3/dd1 39 2024-04-27T04:10:15.994 INFO:tasks.workunit.client.0.smithi162.stdout:2/653: write d0/d20/fbb [934511,124280] 0 2024-04-27T04:10:15.997 INFO:tasks.workunit.client.1.smithi181.stdout:0/977: fdatasync dc/d1f/d3b/d4a/d95/d26/d41/f9e 0 2024-04-27T04:10:15.998 INFO:tasks.workunit.client.1.smithi181.stdout:2/882: mkdir d4/d9/d35/d9f/dd0/d11c 0 2024-04-27T04:10:16.001 INFO:tasks.workunit.client.0.smithi162.stdout:8/714: creat d0/d10/d2b/d2d/fea x:0 0 0 2024-04-27T04:10:16.001 INFO:tasks.workunit.client.1.smithi181.stdout:0/978: dread dc/d1f/d3b/d4a/d95/d26/d41/d6f/d7d/fa7 [0,4194304] 0 2024-04-27T04:10:16.004 INFO:tasks.workunit.client.0.smithi162.stdout:8/715: mkdir d0/de/d20/d7e/deb 0 2024-04-27T04:10:16.005 INFO:tasks.workunit.client.0.smithi162.stdout:2/654: creat d0/d20/fcf x:0 0 0 2024-04-27T04:10:16.006 INFO:tasks.workunit.client.0.smithi162.stdout:7/701: getdents d3/d5/d5b/d83/de/d2b/d81/daa/dbd/dbe 0 2024-04-27T04:10:16.008 INFO:tasks.workunit.client.1.smithi181.stdout:1/816: getdents d1/dad/db5/d6/da3 0 2024-04-27T04:10:16.010 INFO:tasks.workunit.client.1.smithi181.stdout:2/883: link d4/d9/d35/f3a d4/d9/d16/d82/d7d/db7/dcb/da4/dc2/f11d 0 2024-04-27T04:10:16.011 INFO:tasks.workunit.client.0.smithi162.stdout:8/716: dwrite d0/d10/d25/f36 [4194304,4194304] 0 2024-04-27T04:10:16.011 INFO:tasks.workunit.client.1.smithi181.stdout:7/823: creat d1/d22/d15/d1f/ff7 x:0 0 0 2024-04-27T04:10:16.011 INFO:tasks.workunit.client.0.smithi162.stdout:2/655: dwrite d0/f44 [0,4194304] 0 2024-04-27T04:10:16.012 INFO:tasks.workunit.client.1.smithi181.stdout:7/824: fsync d1/d2/d35/d67/fe6 0 2024-04-27T04:10:16.012 INFO:tasks.workunit.client.0.smithi162.stdout:2/656: stat d0/d4/d2a/d3b/d49/d5a/d63/d70/d8e 0 2024-04-27T04:10:16.013 INFO:tasks.workunit.client.1.smithi181.stdout:0/979: mkdir dc/d1f/d3b/d4a/d95/d132/d14e/d150 0 2024-04-27T04:10:16.014 INFO:tasks.workunit.client.1.smithi181.stdout:8/864: sync 2024-04-27T04:10:16.017 INFO:tasks.workunit.client.0.smithi162.stdout:8/717: creat d0/de/d20/d6c/fec x:0 0 0 2024-04-27T04:10:16.017 INFO:tasks.workunit.client.1.smithi181.stdout:7/825: dread d1/d22/fc0 [0,4194304] 0 2024-04-27T04:10:16.018 INFO:tasks.workunit.client.1.smithi181.stdout:6/707: truncate d7/dc/d1c/d34/f9a 807974 0 2024-04-27T04:10:16.019 INFO:tasks.workunit.client.0.smithi162.stdout:8/718: truncate d0/d10/d2b/d2d/d2f/d39/fe2 194123 0 2024-04-27T04:10:16.020 INFO:tasks.workunit.client.0.smithi162.stdout:2/657: symlink d0/d4/d17/d2b/d64/d96/daa/ld0 0 2024-04-27T04:10:16.020 INFO:tasks.workunit.client.0.smithi162.stdout:8/719: read d0/de/d6e/d71/fb3 [15831,9107] 0 2024-04-27T04:10:16.021 INFO:tasks.workunit.client.1.smithi181.stdout:2/884: readlink d4/d9/d16/d82/d7d/l10e 0 2024-04-27T04:10:16.021 INFO:tasks.workunit.client.0.smithi162.stdout:7/702: link d3/d5/d3c/d4d/c36 d3/d5/d5b/d83/de/d2b/d81/d96/ce9 0 2024-04-27T04:10:16.022 INFO:tasks.workunit.client.1.smithi181.stdout:2/885: chown d4/d9/d4f/d79/l113 0 1 2024-04-27T04:10:16.024 INFO:tasks.workunit.client.0.smithi162.stdout:2/658: rmdir d0/d4/d17/db4 39 2024-04-27T04:10:16.025 INFO:tasks.workunit.client.1.smithi181.stdout:6/708: dwrite d7/d31/f8c [0,4194304] 0 2024-04-27T04:10:16.026 INFO:tasks.workunit.client.1.smithi181.stdout:2/886: write d4/d9/d16/d82/d7d/db7/dcb/f97 [343480,9542] 0 2024-04-27T04:10:16.026 INFO:tasks.workunit.client.0.smithi162.stdout:8/720: dwrite d0/d10/d76/fdd [0,4194304] 0 2024-04-27T04:10:16.027 INFO:tasks.workunit.client.1.smithi181.stdout:2/887: read - d4/d9/d16/d45/d9a/ff8 zero size 2024-04-27T04:10:16.027 INFO:tasks.workunit.client.1.smithi181.stdout:2/888: stat d4/l5 0 2024-04-27T04:10:16.028 INFO:tasks.workunit.client.1.smithi181.stdout:6/709: dread d7/dc/dc3/d62/f63 [0,4194304] 0 2024-04-27T04:10:16.029 INFO:tasks.workunit.client.1.smithi181.stdout:6/710: write d7/fb9 [916765,85827] 0 2024-04-27T04:10:16.030 INFO:tasks.workunit.client.1.smithi181.stdout:7/826: truncate d1/d22/d15/d29/dbe/f46 2537921 0 2024-04-27T04:10:16.033 INFO:tasks.workunit.client.1.smithi181.stdout:0/980: unlink dc/d1f/d3b/c53 0 2024-04-27T04:10:16.033 INFO:tasks.workunit.client.0.smithi162.stdout:8/721: dread - d0/de/d6e/f51 zero size 2024-04-27T04:10:16.033 INFO:tasks.workunit.client.1.smithi181.stdout:8/865: getdents d1/d20/d10d 0 2024-04-27T04:10:16.035 INFO:tasks.workunit.client.1.smithi181.stdout:6/711: symlink d7/dc/d1c/d34/d3f/db2/lef 0 2024-04-27T04:10:16.037 INFO:tasks.workunit.client.1.smithi181.stdout:0/981: mknod dc/d1f/d3b/df7/c151 0 2024-04-27T04:10:16.039 INFO:tasks.workunit.client.1.smithi181.stdout:8/866: mkdir d1/d6/d2f/d11e/dec/d131 0 2024-04-27T04:10:16.040 INFO:tasks.workunit.client.0.smithi162.stdout:2/659: dwrite d0/d4/f3a [0,4194304] 0 2024-04-27T04:10:16.041 INFO:tasks.workunit.client.1.smithi181.stdout:6/712: fsync d7/dc/d1c/d34/d3f/d7e/d8a/fa9 0 2024-04-27T04:10:16.042 INFO:tasks.workunit.client.1.smithi181.stdout:2/889: creat d4/d9/d4f/f11e x:0 0 0 2024-04-27T04:10:16.043 INFO:tasks.workunit.client.1.smithi181.stdout:0/982: symlink dc/d93/l152 0 2024-04-27T04:10:16.046 INFO:tasks.workunit.client.0.smithi162.stdout:8/722: dwrite d0/de/d20/d7e/fd1 [0,4194304] 0 2024-04-27T04:10:16.047 INFO:tasks.workunit.client.1.smithi181.stdout:6/713: rmdir d7/dc/d1c/d34/d3f/db2 39 2024-04-27T04:10:16.049 INFO:tasks.workunit.client.0.smithi162.stdout:8/723: fsync d0/de/d6e/d71/d86/fda 0 2024-04-27T04:10:16.050 INFO:tasks.workunit.client.0.smithi162.stdout:8/724: fsync d0/de/d6e/d71/fae 0 2024-04-27T04:10:16.050 INFO:tasks.workunit.client.1.smithi181.stdout:3/804: sync 2024-04-27T04:10:16.050 INFO:tasks.workunit.client.1.smithi181.stdout:9/769: sync 2024-04-27T04:10:16.051 INFO:tasks.workunit.client.0.smithi162.stdout:2/660: mknod d0/d4/d17/d2b/d64/d96/daa/cd1 0 2024-04-27T04:10:16.053 INFO:tasks.workunit.client.0.smithi162.stdout:2/661: truncate d0/d4/d2a/d3b/d49/d5a/d63/d6e/fa7 881947 0 2024-04-27T04:10:16.056 INFO:tasks.workunit.client.0.smithi162.stdout:0/772: write d5/dc/d1e/d20/dec/df1/db4/fe0 [692273,75152] 0 2024-04-27T04:10:16.057 INFO:tasks.workunit.client.0.smithi162.stdout:8/725: creat d0/d10/d25/d7d/fed x:0 0 0 2024-04-27T04:10:16.057 INFO:tasks.workunit.client.1.smithi181.stdout:0/983: dread dc/d54/f72 [0,4194304] 0 2024-04-27T04:10:16.058 INFO:tasks.workunit.client.0.smithi162.stdout:0/773: truncate d5/dc/d1e/d46/d48/fb1 1577733 0 2024-04-27T04:10:16.064 INFO:tasks.workunit.client.0.smithi162.stdout:8/726: dwrite d0/d10/d2b/d2d/d2f/f54 [0,4194304] 0 2024-04-27T04:10:16.064 INFO:tasks.workunit.client.0.smithi162.stdout:2/662: rmdir d0/d4/d17/d4f 39 2024-04-27T04:10:16.064 INFO:tasks.workunit.client.1.smithi181.stdout:8/867: dread d1/f6b [0,4194304] 0 2024-04-27T04:10:16.068 INFO:tasks.workunit.client.1.smithi181.stdout:3/805: mknod d0/d6c/d52/dbb/de5/c102 0 2024-04-27T04:10:16.070 INFO:tasks.workunit.client.0.smithi162.stdout:0/774: chown d5/dc/d1e/d46/d48/d9e/dce/ffd 14982170 1 2024-04-27T04:10:16.073 INFO:tasks.workunit.client.0.smithi162.stdout:8/727: rename d0/fd5 to d0/d10/d44/d70/fee 0 2024-04-27T04:10:16.079 INFO:tasks.workunit.client.0.smithi162.stdout:0/775: fsync d5/d26/f6a 0 2024-04-27T04:10:16.079 INFO:tasks.workunit.client.0.smithi162.stdout:3/654: dwrite d1/d18/f13 [4194304,4194304] 0 2024-04-27T04:10:16.080 INFO:tasks.workunit.client.1.smithi181.stdout:8/868: symlink d1/d6/d2f/d30/d66/dc0/dcd/de1/l132 0 2024-04-27T04:10:16.080 INFO:tasks.workunit.client.1.smithi181.stdout:4/768: write d0/d15/d25/d2a/d61/f9d [1061301,7821] 0 2024-04-27T04:10:16.081 INFO:tasks.workunit.client.0.smithi162.stdout:8/728: dwrite d0/d10/f9a [4194304,4194304] 0 2024-04-27T04:10:16.081 INFO:tasks.workunit.client.1.smithi181.stdout:8/869: read d1/f6b [510415,97875] 0 2024-04-27T04:10:16.081 INFO:tasks.workunit.client.1.smithi181.stdout:4/769: chown d0/d15/f32 84009 1 2024-04-27T04:10:16.082 INFO:tasks.workunit.client.0.smithi162.stdout:8/729: stat d0/d10/d44 0 2024-04-27T04:10:16.083 INFO:tasks.workunit.client.0.smithi162.stdout:8/730: write d0/de/d6e/f9b [550639,67172] 0 2024-04-27T04:10:16.083 INFO:tasks.workunit.client.1.smithi181.stdout:3/806: fsync d0/d9f/f9a 0 2024-04-27T04:10:16.090 INFO:tasks.workunit.client.0.smithi162.stdout:0/776: fdatasync d5/dc/d1e/fbf 0 2024-04-27T04:10:16.092 INFO:tasks.workunit.client.1.smithi181.stdout:8/870: symlink d1/d6/d2f/d30/d9d/dea/db3/l133 0 2024-04-27T04:10:16.094 INFO:tasks.workunit.client.0.smithi162.stdout:8/731: dread d0/d10/d2b/d2d/d2f/d39/fe2 [0,4194304] 0 2024-04-27T04:10:16.095 INFO:tasks.workunit.client.1.smithi181.stdout:4/770: creat d0/d15/d25/d2a/d36/db8/ffc x:0 0 0 2024-04-27T04:10:16.096 INFO:tasks.workunit.client.0.smithi162.stdout:8/732: write d0/d10/d25/d7d/f8b [641750,56964] 0 2024-04-27T04:10:16.098 INFO:tasks.workunit.client.0.smithi162.stdout:0/777: dwrite d5/dc/d1e/d20/dba/df9/d8b/fe1 [0,4194304] 0 2024-04-27T04:10:16.099 INFO:tasks.workunit.client.0.smithi162.stdout:0/778: write d5/d26/d4a/f4c [196348,11057] 0 2024-04-27T04:10:16.100 INFO:tasks.workunit.client.1.smithi181.stdout:1/817: dwrite d1/dad/db5/f78 [0,4194304] 0 2024-04-27T04:10:16.102 INFO:tasks.workunit.client.1.smithi181.stdout:3/807: rename d0/d6c/d52/d44/d56/faa to d0/de/d7d/df5/f103 0 2024-04-27T04:10:16.105 INFO:tasks.workunit.client.0.smithi162.stdout:8/733: read d0/d10/d2b/d2e/f46 [250555,98767] 0 2024-04-27T04:10:16.106 INFO:tasks.workunit.client.1.smithi181.stdout:0/984: mkdir dc/d1f/d3b/d4a/d95/d26/d131/d106/d141/d153 0 2024-04-27T04:10:16.113 INFO:tasks.workunit.client.0.smithi162.stdout:7/703: dwrite d3/d5/d5b/d83/f3b [0,4194304] 0 2024-04-27T04:10:16.114 INFO:tasks.workunit.client.1.smithi181.stdout:1/818: unlink d1/dad/db5/d6/da3/d52/l66 0 2024-04-27T04:10:16.117 INFO:tasks.workunit.client.1.smithi181.stdout:3/808: unlink d0/f5 0 2024-04-27T04:10:16.123 INFO:tasks.workunit.client.1.smithi181.stdout:7/827: dwrite d1/d2/f9f [4194304,4194304] 0 2024-04-27T04:10:16.125 INFO:tasks.workunit.client.1.smithi181.stdout:4/771: mknod d0/d15/d25/cfd 0 2024-04-27T04:10:16.126 INFO:tasks.workunit.client.1.smithi181.stdout:4/772: fdatasync d0/d15/de/d19/d59/d80/fe5 0 2024-04-27T04:10:16.127 INFO:tasks.workunit.client.1.smithi181.stdout:1/819: truncate d1/dad/db5/d6/da3/d1e/f49 235126 0 2024-04-27T04:10:16.133 INFO:tasks.workunit.client.1.smithi181.stdout:2/890: dwrite d4/d9/d16/f2a [0,4194304] 0 2024-04-27T04:10:16.133 INFO:tasks.workunit.client.0.smithi162.stdout:7/704: creat d3/d5/d5b/d83/de/d2b/fea x:0 0 0 2024-04-27T04:10:16.136 INFO:tasks.workunit.client.1.smithi181.stdout:7/828: fdatasync d1/f45 0 2024-04-27T04:10:16.138 INFO:tasks.workunit.client.1.smithi181.stdout:4/773: fdatasync d0/d15/d25/d2b/f5f 0 2024-04-27T04:10:16.140 INFO:tasks.workunit.client.0.smithi162.stdout:7/705: mkdir d3/d5/d5b/d83/de/d2b/d81/daa/deb 0 2024-04-27T04:10:16.141 INFO:tasks.workunit.client.1.smithi181.stdout:6/714: dwrite d7/dc/dc3/d72/fb7 [0,4194304] 0 2024-04-27T04:10:16.142 INFO:tasks.workunit.client.1.smithi181.stdout:4/774: dread d0/d15/d76/db1/fb6 [0,4194304] 0 2024-04-27T04:10:16.142 INFO:tasks.workunit.client.1.smithi181.stdout:6/715: dread - d7/dc/dc3/d72/daa/feb zero size 2024-04-27T04:10:16.143 INFO:tasks.workunit.client.1.smithi181.stdout:1/820: creat d1/dad/db5/d6/db/d2d/d36/f10c x:0 0 0 2024-04-27T04:10:16.144 INFO:tasks.workunit.client.1.smithi181.stdout:4/775: readlink d0/d15/d25/d2a/da0/lc9 0 2024-04-27T04:10:16.144 INFO:tasks.workunit.client.1.smithi181.stdout:6/716: write d7/dc/dc3/f6b [1454347,40804] 0 2024-04-27T04:10:16.145 INFO:tasks.workunit.client.1.smithi181.stdout:1/821: chown d1/fa8 664961 1 2024-04-27T04:10:16.147 INFO:tasks.workunit.client.1.smithi181.stdout:6/717: truncate d7/dc/d1c/d88/db3/fe4 4999524 0 2024-04-27T04:10:16.148 INFO:tasks.workunit.client.0.smithi162.stdout:7/706: mkdir d3/d5/d5b/d83/de/d2b/d81/daa/dbd/db6/dec 0 2024-04-27T04:10:16.149 INFO:tasks.workunit.client.0.smithi162.stdout:7/707: write d3/d5/d9/fe7 [760773,33842] 0 2024-04-27T04:10:16.151 INFO:tasks.workunit.client.1.smithi181.stdout:7/829: rmdir d1/d2/d1c/d56 39 2024-04-27T04:10:16.153 INFO:tasks.workunit.client.0.smithi162.stdout:7/708: dread - d3/d5/d5b/d83/de/d2b/d81/d96/f99 zero size 2024-04-27T04:10:16.153 INFO:tasks.workunit.client.0.smithi162.stdout:7/709: readlink d3/l64 0 2024-04-27T04:10:16.156 INFO:tasks.workunit.client.1.smithi181.stdout:9/770: dwrite d0/d44/d4b/d43/d88/da1/dfc/dbd/df3/f7f [0,4194304] 0 2024-04-27T04:10:16.157 INFO:tasks.workunit.client.0.smithi162.stdout:7/710: chown d3/d5/d5b/d83/de/d2b/c8f 1621 1 2024-04-27T04:10:16.159 INFO:tasks.workunit.client.0.smithi162.stdout:3/655: dwrite d1/d18/f41 [0,4194304] 0 2024-04-27T04:10:16.161 INFO:tasks.workunit.client.1.smithi181.stdout:6/718: creat d7/dc/d1c/d34/d3f/d7e/d8a/ff0 x:0 0 0 2024-04-27T04:10:16.164 INFO:tasks.workunit.client.0.smithi162.stdout:7/711: fdatasync d3/d39/d57/f42 0 2024-04-27T04:10:16.166 INFO:tasks.workunit.client.0.smithi162.stdout:5/698: sync 2024-04-27T04:10:16.166 INFO:tasks.workunit.client.1.smithi181.stdout:2/891: mknod d4/d9/d16/d82/d7d/db7/dcb/da4/dc2/c11f 0 2024-04-27T04:10:16.167 INFO:tasks.workunit.client.0.smithi162.stdout:5/699: dread - d8/d1a/d23/d40/fcd zero size 2024-04-27T04:10:16.168 INFO:tasks.workunit.client.1.smithi181.stdout:3/809: getdents d0/d6c/d52/dbb/d98 0 2024-04-27T04:10:16.168 INFO:tasks.workunit.client.0.smithi162.stdout:5/700: chown d8/d1a/f83 36 1 2024-04-27T04:10:16.171 INFO:tasks.workunit.client.0.smithi162.stdout:7/712: fdatasync d3/d5/d5b/d83/de/f62 0 2024-04-27T04:10:16.173 INFO:tasks.workunit.client.1.smithi181.stdout:4/776: mkdir d0/d15/de/d19/dfe 0 2024-04-27T04:10:16.173 INFO:tasks.workunit.client.0.smithi162.stdout:5/701: dwrite d8/d86/fd9 [0,4194304] 0 2024-04-27T04:10:16.174 INFO:tasks.workunit.client.1.smithi181.stdout:4/777: chown d0/d15/d25/d2a/d36/f92 71299493 1 2024-04-27T04:10:16.175 INFO:tasks.workunit.client.1.smithi181.stdout:3/810: dwrite d0/d6c/d52/d44/dd5/d58/d64/fff [0,4194304] 0 2024-04-27T04:10:16.176 INFO:tasks.workunit.client.1.smithi181.stdout:3/811: chown d0/d9f/c93 819237 1 2024-04-27T04:10:16.176 INFO:tasks.workunit.client.0.smithi162.stdout:3/656: rename d1/l1c to d1/d18/d23/d58/d63/d88/le1 0 2024-04-27T04:10:16.178 INFO:tasks.workunit.client.0.smithi162.stdout:0/779: write d5/d26/d4a/f77 [3800211,89191] 0 2024-04-27T04:10:16.179 INFO:tasks.workunit.client.1.smithi181.stdout:9/771: rename d0/f23 to d0/d44/d4f/df7/f108 0 2024-04-27T04:10:16.183 INFO:tasks.workunit.client.0.smithi162.stdout:5/702: dwrite d8/d9/d5b/fc9 [0,4194304] 0 2024-04-27T04:10:16.183 INFO:tasks.workunit.client.1.smithi181.stdout:7/830: dread d1/d2/d1c/d37/f3f [0,4194304] 0 2024-04-27T04:10:16.185 INFO:tasks.workunit.client.1.smithi181.stdout:6/719: rmdir d7/dc/dc3 39 2024-04-27T04:10:16.190 INFO:tasks.workunit.client.0.smithi162.stdout:8/734: dwrite d0/d10/d2b/d2e/dca/fa0 [4194304,4194304] 0 2024-04-27T04:10:16.192 INFO:tasks.workunit.client.1.smithi181.stdout:0/985: dwrite dc/d1f/d6b/f83 [0,4194304] 0 2024-04-27T04:10:16.196 INFO:tasks.workunit.client.0.smithi162.stdout:3/657: symlink d1/d18/d23/d34/dad/dc5/le2 0 2024-04-27T04:10:16.196 INFO:tasks.workunit.client.1.smithi181.stdout:3/812: symlink d0/d6c/d66/dc2/l104 0 2024-04-27T04:10:16.200 INFO:tasks.workunit.client.1.smithi181.stdout:2/892: rename d4/l5 to d4/d9/d35/d9f/dd0/l120 0 2024-04-27T04:10:16.205 INFO:tasks.workunit.client.1.smithi181.stdout:7/831: fsync d1/d2/d1c/d37/d91/fb0 0 2024-04-27T04:10:16.209 INFO:tasks.workunit.client.1.smithi181.stdout:6/720: mknod d7/dc/d1c/d88/db3/cf1 0 2024-04-27T04:10:16.211 INFO:tasks.workunit.client.1.smithi181.stdout:1/822: dwrite d1/dad/db5/d6/db/d2d/fb1 [0,4194304] 0 2024-04-27T04:10:16.211 INFO:tasks.workunit.client.1.smithi181.stdout:9/772: dwrite d0/d44/d4b/d43/d88/da1/dfc/dbd/df3/fa5 [0,4194304] 0 2024-04-27T04:10:16.213 INFO:tasks.workunit.client.1.smithi181.stdout:4/778: symlink d0/dd4/lff 0 2024-04-27T04:10:16.214 INFO:tasks.workunit.client.0.smithi162.stdout:6/654: sync 2024-04-27T04:10:16.214 INFO:tasks.workunit.client.0.smithi162.stdout:9/673: sync 2024-04-27T04:10:16.214 INFO:tasks.workunit.client.0.smithi162.stdout:1/636: sync 2024-04-27T04:10:16.215 INFO:tasks.workunit.client.0.smithi162.stdout:1/637: chown d0/f1e 60596 1 2024-04-27T04:10:16.217 INFO:tasks.workunit.client.0.smithi162.stdout:6/655: write d6/d1a/d34/f54 [977129,67548] 0 2024-04-27T04:10:16.219 INFO:tasks.workunit.client.1.smithi181.stdout:3/813: creat d0/d6c/d52/d44/dd5/d58/d64/dc6/f105 x:0 0 0 2024-04-27T04:10:16.223 INFO:tasks.workunit.client.0.smithi162.stdout:5/703: symlink d8/d9/d5b/lde 0 2024-04-27T04:10:16.224 INFO:tasks.workunit.client.0.smithi162.stdout:5/704: fsync d8/d1a/d29/d4a/d5c/f62 0 2024-04-27T04:10:16.226 INFO:tasks.workunit.client.0.smithi162.stdout:7/713: write d3/f18 [2149091,110311] 0 2024-04-27T04:10:16.229 INFO:tasks.workunit.client.0.smithi162.stdout:5/705: read d8/d1a/f59 [1910073,86702] 0 2024-04-27T04:10:16.229 INFO:tasks.workunit.client.1.smithi181.stdout:7/832: mknod d1/d2/d35/d67/dbf/cf8 0 2024-04-27T04:10:16.229 INFO:tasks.workunit.client.0.smithi162.stdout:5/706: readlink d8/l38 0 2024-04-27T04:10:16.230 INFO:tasks.workunit.client.1.smithi181.stdout:7/833: stat d1/d2/d1c/d2d/d61/f68 0 2024-04-27T04:10:16.231 INFO:tasks.workunit.client.1.smithi181.stdout:1/823: creat d1/dad/db5/d6/da3/d60/f10d x:0 0 0 2024-04-27T04:10:16.235 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:10:15 smithi162 ceph-mon[26841]: mgrmap e24: smithi181.vtwuhz(active, since 9s), standbys: smithi162.oennfn 2024-04-27T04:10:16.235 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:10:15 smithi162 ceph-mon[26841]: from='mgr.24367 172.21.15.181:0/3181919980' entity='mgr.smithi181.vtwuhz' cmd=[{"prefix": "mgr metadata", "who": "smithi162.oennfn", "id": "smithi162.oennfn"}]: dispatch 2024-04-27T04:10:16.236 INFO:tasks.workunit.client.1.smithi181.stdout:3/814: truncate d0/d6c/d52/d44/f55 982113 0 2024-04-27T04:10:16.237 INFO:tasks.workunit.client.0.smithi162.stdout:5/707: dread d8/d1a/d23/d40/f46 [0,4194304] 0 2024-04-27T04:10:16.237 INFO:tasks.workunit.client.1.smithi181.stdout:3/815: write d0/d28/d86/f8d [1383750,124313] 0 2024-04-27T04:10:16.240 INFO:tasks.workunit.client.1.smithi181.stdout:0/986: rename dc/d1f/d3b/d4a/d95/d26/l102 to dc/d1f/d6b/d7e/d91/l154 0 2024-04-27T04:10:16.244 INFO:tasks.workunit.client.0.smithi162.stdout:8/735: rename d0/de/c1a to d0/d10/d2b/cef 0 2024-04-27T04:10:16.245 INFO:tasks.workunit.client.0.smithi162.stdout:3/658: write d1/d21/f2c [3734085,47338] 0 2024-04-27T04:10:16.245 INFO:tasks.workunit.client.0.smithi162.stdout:8/736: write d0/d10/d2b/d2d/fc4 [1808452,126097] 0 2024-04-27T04:10:16.246 INFO:tasks.workunit.client.1.smithi181.stdout:6/721: creat d7/dc/d1c/d88/db3/ddf/ff2 x:0 0 0 2024-04-27T04:10:16.247 INFO:tasks.workunit.client.0.smithi162.stdout:0/780: truncate d5/d26/d69/f6c 6171334 0 2024-04-27T04:10:16.249 INFO:tasks.workunit.client.1.smithi181.stdout:7/834: creat d1/d2/ff9 x:0 0 0 2024-04-27T04:10:16.249 INFO:tasks.workunit.client.0.smithi162.stdout:0/781: chown d5/dc/d1e/d46/f9f 51 1 2024-04-27T04:10:16.250 INFO:tasks.workunit.client.1.smithi181.stdout:9/773: symlink d0/d44/d4b/l109 0 2024-04-27T04:10:16.250 INFO:tasks.workunit.client.0.smithi162.stdout:1/638: dread d0/d7/d2a/f47 [0,4194304] 0 2024-04-27T04:10:16.251 INFO:tasks.workunit.client.1.smithi181.stdout:2/893: write d4/d9/d16/f26 [6510152,51423] 0 2024-04-27T04:10:16.252 INFO:tasks.workunit.client.0.smithi162.stdout:1/639: write d0/d7/f88 [3403967,65393] 0 2024-04-27T04:10:16.253 INFO:tasks.workunit.client.0.smithi162.stdout:8/737: dwrite d0/d10/d44/d70/f7f [0,4194304] 0 2024-04-27T04:10:16.254 INFO:tasks.workunit.client.0.smithi162.stdout:1/640: chown d0/d11/f20 3083898 1 2024-04-27T04:10:16.254 INFO:tasks.workunit.client.0.smithi162.stdout:6/656: creat d6/d1a/d34/d3a/fce x:0 0 0 2024-04-27T04:10:16.256 INFO:tasks.workunit.client.1.smithi181.stdout:9/774: dread d0/d44/d4b/d43/d88/fef [0,4194304] 0 2024-04-27T04:10:16.256 INFO:tasks.workunit.client.1.smithi181.stdout:7/835: dwrite d1/d2/ff9 [0,4194304] 0 2024-04-27T04:10:16.257 INFO:tasks.workunit.client.0.smithi162.stdout:7/714: write d3/d39/d57/f43 [2980250,24886] 0 2024-04-27T04:10:16.258 INFO:tasks.workunit.client.1.smithi181.stdout:9/775: read d0/fa7 [1408018,29189] 0 2024-04-27T04:10:16.258 INFO:tasks.workunit.client.0.smithi162.stdout:3/659: mknod d1/d65/ce3 0 2024-04-27T04:10:16.261 INFO:tasks.workunit.client.0.smithi162.stdout:9/674: getdents d3/d13/d2d/d30/d7f/dc3 0 2024-04-27T04:10:16.263 INFO:tasks.workunit.client.0.smithi162.stdout:3/660: truncate d1/d18/d5d/d5f/d66/f68 202777 0 2024-04-27T04:10:16.264 INFO:tasks.workunit.client.0.smithi162.stdout:3/661: fdatasync d1/d70/f92 0 2024-04-27T04:10:16.266 INFO:tasks.workunit.client.0.smithi162.stdout:8/738: dwrite d0/de/d20/d6c/fec [0,4194304] 0 2024-04-27T04:10:16.266 INFO:tasks.workunit.client.0.smithi162.stdout:3/662: write d1/d18/f4d [2136305,25449] 0 2024-04-27T04:10:16.267 INFO:tasks.workunit.client.0.smithi162.stdout:4/634: sync 2024-04-27T04:10:16.267 INFO:tasks.workunit.client.0.smithi162.stdout:2/663: sync 2024-04-27T04:10:16.271 INFO:tasks.workunit.client.0.smithi162.stdout:0/782: mknod d5/dc/d1e/d20/dec/df1/da1/c102 0 2024-04-27T04:10:16.273 INFO:tasks.workunit.client.0.smithi162.stdout:9/675: dwrite d3/d11/d45/d4e/fcb [0,4194304] 0 2024-04-27T04:10:16.273 INFO:tasks.workunit.client.0.smithi162.stdout:1/641: fdatasync d0/d7/d3c/fb6 0 2024-04-27T04:10:16.278 INFO:tasks.workunit.client.0.smithi162.stdout:6/657: rename d6/db/dc0/d74/l9e to d6/d1a/d34/d3a/d68/d6c/lcf 0 2024-04-27T04:10:16.278 INFO:tasks.workunit.client.0.smithi162.stdout:6/658: chown d6/d1a/d34/d43/c6f 230601 1 2024-04-27T04:10:16.285 INFO:tasks.workunit.client.0.smithi162.stdout:5/708: truncate d8/d1a/d23/d40/f77 2282868 0 2024-04-27T04:10:16.286 INFO:tasks.workunit.client.0.smithi162.stdout:4/635: rmdir d2/d25 39 2024-04-27T04:10:16.295 INFO:tasks.workunit.client.0.smithi162.stdout:6/659: creat d6/d1a/d34/d7f/d80/fd0 x:0 0 0 2024-04-27T04:10:16.295 INFO:tasks.workunit.client.1.smithi181.stdout:8/871: sync 2024-04-27T04:10:16.296 INFO:tasks.workunit.client.0.smithi162.stdout:3/663: mknod d1/d21/d47/ce4 0 2024-04-27T04:10:16.298 INFO:tasks.workunit.client.1.smithi181.stdout:3/816: dread - d0/d6c/d52/d44/dd5/d58/de4/daf/fb9 zero size 2024-04-27T04:10:16.298 INFO:tasks.workunit.client.0.smithi162.stdout:4/636: chown d2/d25/d1c/d39/d3b/d6e/cbe 6129 1 2024-04-27T04:10:16.299 INFO:tasks.workunit.client.0.smithi162.stdout:4/637: chown d2/d25/d1c/d39/d3b/d6e/d6f/lbf 770 1 2024-04-27T04:10:16.300 INFO:tasks.workunit.client.1.smithi181.stdout:0/987: symlink dc/d1f/d3b/d4a/d95/d132/d14e/l155 0 2024-04-27T04:10:16.301 INFO:tasks.workunit.client.0.smithi162.stdout:8/739: creat d0/d10/d2b/d2d/d2f/d39/d7c/dd0/ff0 x:0 0 0 2024-04-27T04:10:16.306 INFO:tasks.workunit.client.1.smithi181.stdout:1/824: creat d1/dad/db5/d6/da3/d1e/d48/dfa/f10e x:0 0 0 2024-04-27T04:10:16.308 INFO:tasks.workunit.client.0.smithi162.stdout:1/642: mkdir d0/d7/d3c/d77/dd1/dd5 0 2024-04-27T04:10:16.308 INFO:tasks.workunit.client.1.smithi181.stdout:6/722: dwrite d7/dc/dc3/d72/daa/fce [0,4194304] 0 2024-04-27T04:10:16.313 INFO:tasks.workunit.client.0.smithi162.stdout:4/638: creat d2/d25/d1c/d39/d90/fcc x:0 0 0 2024-04-27T04:10:16.318 INFO:tasks.workunit.client.0.smithi162.stdout:9/676: mkdir d3/d13/d15/d7e/dd0 0 2024-04-27T04:10:16.318 INFO:tasks.workunit.client.1.smithi181.stdout:9/776: truncate d0/d44/d4f/f6b 99020 0 2024-04-27T04:10:16.319 INFO:tasks.workunit.client.1.smithi181.stdout:9/777: dread - d0/d44/d4b/d43/d88/da1/dfc/dbd/dd1/fec zero size 2024-04-27T04:10:16.321 INFO:tasks.workunit.client.0.smithi162.stdout:0/783: write d5/dc/d1e/d41/f4e [5100810,31662] 0 2024-04-27T04:10:16.324 INFO:tasks.workunit.client.0.smithi162.stdout:4/639: fdatasync d2/d25/d1c/d39/d5e/f85 0 2024-04-27T04:10:16.325 INFO:tasks.workunit.client.0.smithi162.stdout:8/740: mknod d0/d10/d25/d65/d96/cf1 0 2024-04-27T04:10:16.328 INFO:tasks.workunit.client.0.smithi162.stdout:8/741: write d0/d10/d25/f93 [1929784,35835] 0 2024-04-27T04:10:16.330 INFO:tasks.workunit.client.0.smithi162.stdout:0/784: dwrite d5/f16 [4194304,4194304] 0 2024-04-27T04:10:16.331 INFO:tasks.workunit.client.1.smithi181.stdout:0/988: dread dc/d1f/d3b/dd0/f100 [0,4194304] 0 2024-04-27T04:10:16.332 INFO:tasks.workunit.client.1.smithi181.stdout:1/825: fsync d1/dad/db5/d6/da3/d1e/fc5 0 2024-04-27T04:10:16.333 INFO:tasks.workunit.client.1.smithi181.stdout:8/872: write d1/d6/d2f/d30/d66/fe5 [700982,81083] 0 2024-04-27T04:10:16.338 INFO:tasks.workunit.client.1.smithi181.stdout:7/836: unlink d1/df/d1a/l53 0 2024-04-27T04:10:16.338 INFO:tasks.workunit.client.1.smithi181.stdout:7/837: fdatasync d1/d2/ff9 0 2024-04-27T04:10:16.339 INFO:tasks.workunit.client.0.smithi162.stdout:3/664: dwrite d1/d18/f4a [0,4194304] 0 2024-04-27T04:10:16.342 INFO:tasks.workunit.client.0.smithi162.stdout:4/640: dwrite d2/d25/d1c/d39/d3b/d6e/d6f/fc6 [0,4194304] 0 2024-04-27T04:10:16.342 INFO:tasks.workunit.client.0.smithi162.stdout:1/643: dread d0/d7/d10/f64 [0,4194304] 0 2024-04-27T04:10:16.343 INFO:tasks.workunit.client.0.smithi162.stdout:3/665: read - d1/d70/fa5 zero size 2024-04-27T04:10:16.345 INFO:tasks.workunit.client.1.smithi181.stdout:2/894: creat d4/d9/d35/f121 x:0 0 0 2024-04-27T04:10:16.349 INFO:tasks.workunit.client.1.smithi181.stdout:1/826: rename d1/dad/db5/f78 to d1/dad/db5/d6/db/d2d/df1/f10f 0 2024-04-27T04:10:16.349 INFO:tasks.workunit.client.1.smithi181.stdout:1/827: stat d1/d2/c23 0 2024-04-27T04:10:16.349 INFO:tasks.workunit.client.0.smithi162.stdout:8/742: dwrite d0/d10/d44/d70/f7f [0,4194304] 0 2024-04-27T04:10:16.353 INFO:tasks.workunit.client.0.smithi162.stdout:3/666: creat d1/d18/d5d/d5f/d66/fe5 x:0 0 0 2024-04-27T04:10:16.354 INFO:tasks.workunit.client.1.smithi181.stdout:6/723: symlink d7/dc/dc3/d72/dd6/lf3 0 2024-04-27T04:10:16.357 INFO:tasks.workunit.client.0.smithi162.stdout:0/785: rename d5/dc/d1e/d20/dba/df9/c8c to d5/dc/d1e/d56/c103 0 2024-04-27T04:10:16.362 INFO:tasks.workunit.client.1.smithi181.stdout:4/779: link d0/d15/d25/d40/f9b d0/d15/d25/d2a/d36/dba/f100 0 2024-04-27T04:10:16.364 INFO:tasks.workunit.client.0.smithi162.stdout:1/644: rename d0/d7/d5e/f98 to d0/d11/d3e/d59/fd6 0 2024-04-27T04:10:16.365 INFO:tasks.workunit.client.0.smithi162.stdout:3/667: dwrite d1/d45/fe0 [0,4194304] 0 2024-04-27T04:10:16.367 INFO:tasks.workunit.client.1.smithi181.stdout:0/989: mknod dc/d1f/d6b/d7e/c156 0 2024-04-27T04:10:16.370 INFO:tasks.workunit.client.1.smithi181.stdout:2/895: rmdir d4/d9/d16/d82/d7d/db7/dcb/da4 39 2024-04-27T04:10:16.371 INFO:tasks.workunit.client.0.smithi162.stdout:4/641: dread d2/d25/d1c/d24/f3a [0,4194304] 0 2024-04-27T04:10:16.373 INFO:tasks.workunit.client.0.smithi162.stdout:3/668: mkdir d1/d18/d23/d34/daa/de6 0 2024-04-27T04:10:16.374 INFO:tasks.workunit.client.0.smithi162.stdout:0/786: symlink d5/dc/d1e/d56/l104 0 2024-04-27T04:10:16.378 INFO:tasks.workunit.client.1.smithi181.stdout:1/828: creat d1/dad/db5/d6/da3/d1e/d48/da6/f110 x:0 0 0 2024-04-27T04:10:16.379 INFO:tasks.workunit.client.1.smithi181.stdout:7/838: write d1/d2/d1c/d37/f4f [2129732,3650] 0 2024-04-27T04:10:16.380 INFO:tasks.workunit.client.1.smithi181.stdout:8/873: mkdir d1/d6/d2f/d40/d134 0 2024-04-27T04:10:16.383 INFO:tasks.workunit.client.0.smithi162.stdout:8/743: truncate d0/d10/d44/f47 2520033 0 2024-04-27T04:10:16.385 INFO:tasks.workunit.client.1.smithi181.stdout:9/778: link d0/fc d0/d44/d4b/d5f/f10a 0 2024-04-27T04:10:16.386 INFO:tasks.workunit.client.0.smithi162.stdout:3/669: creat d1/d18/d23/d58/d63/dd5/fe7 x:0 0 0 2024-04-27T04:10:16.389 INFO:tasks.workunit.client.1.smithi181.stdout:6/724: dread d7/dc/d1e/f4d [0,4194304] 0 2024-04-27T04:10:16.389 INFO:tasks.workunit.client.0.smithi162.stdout:0/787: fsync d5/dc/d1e/f30 0 2024-04-27T04:10:16.389 INFO:tasks.workunit.client.0.smithi162.stdout:1/645: write d0/d11/d3e/d59/f7a [2212829,112138] 0 2024-04-27T04:10:16.389 INFO:tasks.workunit.client.0.smithi162.stdout:8/744: dwrite d0/d10/d2b/d2d/fc4 [0,4194304] 0 2024-04-27T04:10:16.392 INFO:tasks.workunit.client.1.smithi181.stdout:2/896: rmdir d4/d9/d16/d82/d7d/dbe 39 2024-04-27T04:10:16.396 INFO:tasks.workunit.client.1.smithi181.stdout:7/839: mkdir d1/d2/d1c/d2d/d61/d80/dfa 0 2024-04-27T04:10:16.399 INFO:tasks.workunit.client.0.smithi162.stdout:4/642: write d2/d25/d1c/d39/d5e/f85 [967635,2088] 0 2024-04-27T04:10:16.399 INFO:tasks.workunit.client.1.smithi181.stdout:8/874: mknod d1/d6/d2f/d30/d9d/dea/db3/c135 0 2024-04-27T04:10:16.402 INFO:tasks.workunit.client.1.smithi181.stdout:4/780: mkdir d0/d15/de/d19/d101 0 2024-04-27T04:10:16.402 INFO:tasks.workunit.client.1.smithi181.stdout:3/817: link d0/d6c/d52/d44/d56/c8c d0/d6c/d66/dc2/c106 0 2024-04-27T04:10:16.404 INFO:tasks.workunit.client.1.smithi181.stdout:6/725: creat d7/dc/d1c/d88/db3/ff4 x:0 0 0 2024-04-27T04:10:16.407 INFO:tasks.workunit.client.1.smithi181.stdout:0/990: mknod dc/d1f/d3b/d4a/d95/d26/d131/d106/d110/c157 0 2024-04-27T04:10:16.409 INFO:tasks.workunit.client.0.smithi162.stdout:0/788: write d5/dc/d1e/d41/f4f [5039750,119139] 0 2024-04-27T04:10:16.418 INFO:tasks.workunit.client.0.smithi162.stdout:8/745: dwrite d0/de/d6e/d71/f9d [0,4194304] 0 2024-04-27T04:10:16.418 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:10:15 smithi181 ceph-mon[33774]: mgrmap e24: smithi181.vtwuhz(active, since 9s), standbys: smithi162.oennfn 2024-04-27T04:10:16.418 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:10:15 smithi181 ceph-mon[33774]: from='mgr.24367 172.21.15.181:0/3181919980' entity='mgr.smithi181.vtwuhz' cmd=[{"prefix": "mgr metadata", "who": "smithi162.oennfn", "id": "smithi162.oennfn"}]: dispatch 2024-04-27T04:10:16.419 INFO:tasks.workunit.client.1.smithi181.stdout:7/840: creat d1/d2/d35/ffb x:0 0 0 2024-04-27T04:10:16.419 INFO:tasks.workunit.client.1.smithi181.stdout:8/875: read - d1/d6/d2f/d47/dba/fd6 zero size 2024-04-27T04:10:16.419 INFO:tasks.workunit.client.1.smithi181.stdout:7/841: chown d1/d22/d15/d29/dbe 0 1 2024-04-27T04:10:16.420 INFO:tasks.workunit.client.1.smithi181.stdout:9/779: creat d0/d44/d4b/d43/d89/d94/f10b x:0 0 0 2024-04-27T04:10:16.426 INFO:tasks.workunit.client.1.smithi181.stdout:3/818: chown d0/d6c/d52/d44/d56/f63 43510 1 2024-04-27T04:10:16.427 INFO:tasks.workunit.client.1.smithi181.stdout:2/897: write d4/d9/d16/d45/d9a/fe1 [1387383,93769] 0 2024-04-27T04:10:16.428 INFO:tasks.workunit.client.1.smithi181.stdout:6/726: symlink d7/dc/dc3/d72/dd6/lf5 0 2024-04-27T04:10:16.429 INFO:tasks.workunit.client.1.smithi181.stdout:2/898: stat d4/d9/d4f/d79/d98/fcc 0 2024-04-27T04:10:16.430 INFO:tasks.workunit.client.1.smithi181.stdout:2/899: fsync d4/d9/d4f/d79/d94/f95 0 2024-04-27T04:10:16.431 INFO:tasks.workunit.client.1.smithi181.stdout:1/829: creat d1/dad/db5/d6/da3/d1e/d20/d53/f111 x:0 0 0 2024-04-27T04:10:16.433 INFO:tasks.workunit.client.1.smithi181.stdout:8/876: rmdir d1/d6/d2f/d30/d66/dc0/dcd/de1 39 2024-04-27T04:10:16.434 INFO:tasks.workunit.client.1.smithi181.stdout:4/781: write d0/d15/d25/d2a/f7e [2747391,94085] 0 2024-04-27T04:10:16.436 INFO:tasks.workunit.client.1.smithi181.stdout:7/842: symlink d1/d22/d15/d5f/d93/df3/lfc 0 2024-04-27T04:10:16.439 INFO:tasks.workunit.client.1.smithi181.stdout:1/830: dwrite d1/dad/db5/d6/db/d2d/d36/f10c [0,4194304] 0 2024-04-27T04:10:16.442 INFO:tasks.workunit.client.1.smithi181.stdout:6/727: write d7/dc/dc3/f3c [1265861,91264] 0 2024-04-27T04:10:16.447 INFO:tasks.workunit.client.1.smithi181.stdout:2/900: creat d4/d9/d16/d45/f122 x:0 0 0 2024-04-27T04:10:16.448 INFO:tasks.workunit.client.1.smithi181.stdout:2/901: write d4/d81/d102/d105/f10a [883150,86984] 0 2024-04-27T04:10:16.449 INFO:tasks.workunit.client.1.smithi181.stdout:8/877: mknod d1/d8c/c136 0 2024-04-27T04:10:16.452 INFO:tasks.workunit.client.1.smithi181.stdout:9/780: write d0/d44/d4b/f39 [586286,14909] 0 2024-04-27T04:10:16.462 INFO:tasks.workunit.client.1.smithi181.stdout:1/831: unlink d1/dad/db5/d6/dcb/ld6 0 2024-04-27T04:10:16.464 INFO:tasks.workunit.client.1.smithi181.stdout:6/728: fsync d7/dc/d1e/f51 0 2024-04-27T04:10:16.464 INFO:tasks.workunit.client.1.smithi181.stdout:6/729: dread - d7/dc/d1c/d7f/fdc zero size 2024-04-27T04:10:16.466 INFO:tasks.workunit.client.1.smithi181.stdout:8/878: mkdir d1/d6/d2f/d47/d137 0 2024-04-27T04:10:16.468 INFO:tasks.workunit.client.1.smithi181.stdout:2/902: truncate d4/d9/d4f/fac 1379859 0 2024-04-27T04:10:16.470 INFO:tasks.workunit.client.1.smithi181.stdout:4/782: write d0/d15/f32 [852898,73377] 0 2024-04-27T04:10:16.471 INFO:tasks.workunit.client.1.smithi181.stdout:6/730: dwrite d7/dc/dc3/d72/d98/dbf/fc1 [0,4194304] 0 2024-04-27T04:10:16.473 INFO:tasks.workunit.client.1.smithi181.stdout:6/731: fdatasync d7/dc/dc3/d72/daa/feb 0 2024-04-27T04:10:16.475 INFO:tasks.workunit.client.1.smithi181.stdout:6/732: readlink d7/dc/dc3/d72/d98/dbf/dca/le6 0 2024-04-27T04:10:16.478 INFO:tasks.workunit.client.1.smithi181.stdout:1/832: creat d1/dad/db5/d6/da3/d1e/d48/db4/f112 x:0 0 0 2024-04-27T04:10:16.481 INFO:tasks.workunit.client.1.smithi181.stdout:1/833: readlink d1/dad/db5/d6/db/l43 0 2024-04-27T04:10:16.482 INFO:tasks.workunit.client.0.smithi162.stdout:7/715: sync 2024-04-27T04:10:16.482 INFO:tasks.workunit.client.1.smithi181.stdout:7/843: dwrite d1/d22/f26 [0,4194304] 0 2024-04-27T04:10:16.483 INFO:tasks.workunit.client.0.smithi162.stdout:4/643: fsync d2/f38 0 2024-04-27T04:10:16.486 INFO:tasks.workunit.client.1.smithi181.stdout:3/819: dwrite d0/d6c/d52/d44/d56/f76 [0,4194304] 0 2024-04-27T04:10:16.488 INFO:tasks.workunit.client.0.smithi162.stdout:1/646: rename d0/d11/d3e/d59/d7f/la6 to d0/d7/d3c/d77/ld7 0 2024-04-27T04:10:16.490 INFO:tasks.workunit.client.1.smithi181.stdout:9/781: mkdir d0/d44/d4b/d43/d10c 0 2024-04-27T04:10:16.491 INFO:tasks.workunit.client.0.smithi162.stdout:0/789: stat d5/dc/d1e/lcd 0 2024-04-27T04:10:16.492 INFO:tasks.workunit.client.0.smithi162.stdout:0/790: write d5/dc/d1e/d20/dba/fe8 [30783,110226] 0 2024-04-27T04:10:16.493 INFO:tasks.workunit.client.1.smithi181.stdout:4/783: creat d0/d15/d25/d2b/d6d/d82/f102 x:0 0 0 2024-04-27T04:10:16.497 INFO:tasks.workunit.client.1.smithi181.stdout:3/820: dwrite d0/d6c/fb1 [0,4194304] 0 2024-04-27T04:10:16.504 INFO:tasks.workunit.client.1.smithi181.stdout:7/844: mkdir d1/d2/d1c/d99/dfd 0 2024-04-27T04:10:16.504 INFO:tasks.workunit.client.1.smithi181.stdout:7/845: readlink d1/dc3/daf/ldc 0 2024-04-27T04:10:16.505 INFO:tasks.workunit.client.0.smithi162.stdout:7/716: creat d3/d5/d5b/d83/de/d2b/d81/daa/dbd/dab/fed x:0 0 0 2024-04-27T04:10:16.507 INFO:tasks.workunit.client.1.smithi181.stdout:1/834: mknod d1/dad/db5/d6/db/d2d/d36/c113 0 2024-04-27T04:10:16.515 INFO:tasks.workunit.client.1.smithi181.stdout:2/903: truncate d4/d9/df/f41 2579048 0 2024-04-27T04:10:16.518 INFO:tasks.workunit.client.1.smithi181.stdout:9/782: mknod d0/d44/d4b/d50/c10d 0 2024-04-27T04:10:16.523 INFO:tasks.workunit.client.1.smithi181.stdout:6/733: rename d7/dc/d1c/d34/c4a to d7/dc/dc3/d72/d98/dbf/cf6 0 2024-04-27T04:10:16.524 INFO:tasks.workunit.client.1.smithi181.stdout:6/734: read d7/f9d [4025342,14976] 0 2024-04-27T04:10:16.525 INFO:tasks.workunit.client.1.smithi181.stdout:6/735: read - d7/d31/f75 zero size 2024-04-27T04:10:16.526 INFO:tasks.workunit.client.0.smithi162.stdout:4/644: write d2/da/d1a/f9d [4133885,115783] 0 2024-04-27T04:10:16.527 INFO:tasks.workunit.client.0.smithi162.stdout:1/647: creat d0/d11/d3e/d59/d7f/dc5/fd8 x:0 0 0 2024-04-27T04:10:16.529 INFO:tasks.workunit.client.0.smithi162.stdout:1/648: write d0/d11/d3e/d59/d7f/dc5/fd8 [560677,55175] 0 2024-04-27T04:10:16.529 INFO:tasks.workunit.client.0.smithi162.stdout:0/791: rename d5/dc/d1e/l6b to d5/dc/d1e/d41/d79/l105 0 2024-04-27T04:10:16.531 INFO:tasks.workunit.client.1.smithi181.stdout:7/846: creat d1/d22/d15/ffe x:0 0 0 2024-04-27T04:10:16.533 INFO:tasks.workunit.client.0.smithi162.stdout:7/717: mkdir d3/dee 0 2024-04-27T04:10:16.535 INFO:tasks.workunit.client.0.smithi162.stdout:7/718: truncate d3/d4f/fc3 242600 0 2024-04-27T04:10:16.535 INFO:tasks.workunit.client.0.smithi162.stdout:1/649: dwrite d0/d7/d3c/fb6 [4194304,4194304] 0 2024-04-27T04:10:16.537 INFO:tasks.workunit.client.0.smithi162.stdout:8/746: truncate d0/f4 3051470 0 2024-04-27T04:10:16.539 INFO:tasks.workunit.client.0.smithi162.stdout:7/719: write d3/d5/d5b/d83/de/d2b/d81/daa/dbd/db6/fd5 [757651,52031] 0 2024-04-27T04:10:16.539 INFO:tasks.workunit.client.0.smithi162.stdout:4/645: chown d2/d25/d1c/d39/f5f 0 1 2024-04-27T04:10:16.542 INFO:tasks.workunit.client.0.smithi162.stdout:1/650: rmdir d0/d7/d3c/d6e 39 2024-04-27T04:10:16.543 INFO:tasks.workunit.client.0.smithi162.stdout:1/651: chown d0/f2b 73887679 1 2024-04-27T04:10:16.546 INFO:tasks.workunit.client.1.smithi181.stdout:9/783: creat d0/d44/d4b/d43/d88/da1/dfc/dbd/df3/da0/ddb/f10e x:0 0 0 2024-04-27T04:10:16.546 INFO:tasks.workunit.client.0.smithi162.stdout:8/747: mknod d0/de/d6e/d71/d86/ddc/cf2 0 2024-04-27T04:10:16.551 INFO:tasks.workunit.client.1.smithi181.stdout:6/736: chown d7/dc/dc3/dd1/dd9/lea 74083 1 2024-04-27T04:10:16.552 INFO:tasks.workunit.client.1.smithi181.stdout:3/821: dwrite d0/d28/f42 [0,4194304] 0 2024-04-27T04:10:16.552 INFO:tasks.workunit.client.0.smithi162.stdout:7/720: creat d3/d5/d5b/d83/de/d2b/d81/daa/dbd/fef x:0 0 0 2024-04-27T04:10:16.553 INFO:tasks.workunit.client.0.smithi162.stdout:7/721: read - d3/d5/d5b/d83/de/d2b/d81/d96/f99 zero size 2024-04-27T04:10:16.558 INFO:tasks.workunit.client.0.smithi162.stdout:8/748: creat d0/de/d6e/d71/d86/dd9/ff3 x:0 0 0 2024-04-27T04:10:16.559 INFO:tasks.workunit.client.1.smithi181.stdout:7/847: fsync d1/d22/d15/d29/f84 0 2024-04-27T04:10:16.561 INFO:tasks.workunit.client.1.smithi181.stdout:2/904: mkdir d4/d9/d4f/d79/d123 0 2024-04-27T04:10:16.564 INFO:tasks.workunit.client.1.smithi181.stdout:4/784: write d0/d15/d25/d40/f6c [1948303,103447] 0 2024-04-27T04:10:16.564 INFO:tasks.workunit.client.0.smithi162.stdout:0/792: stat d5/dc/d1e/ffb 0 2024-04-27T04:10:16.565 INFO:tasks.workunit.client.1.smithi181.stdout:4/785: chown d0/d15/d76/db1 30581159 1 2024-04-27T04:10:16.566 INFO:tasks.workunit.client.0.smithi162.stdout:0/793: truncate d5/dc/d1e/ffb 4574312 0 2024-04-27T04:10:16.566 INFO:tasks.workunit.client.1.smithi181.stdout:7/848: dwrite d1/fc8 [0,4194304] 0 2024-04-27T04:10:16.567 INFO:tasks.workunit.client.0.smithi162.stdout:8/749: creat d0/d10/d44/ff4 x:0 0 0 2024-04-27T04:10:16.567 INFO:tasks.workunit.client.1.smithi181.stdout:4/786: dread - d0/d15/d25/d2a/d36/f92 zero size 2024-04-27T04:10:16.568 INFO:tasks.workunit.client.0.smithi162.stdout:8/750: rename d0 to d0/d10/d25/d7d/db8/dbe/df5 22 2024-04-27T04:10:16.569 INFO:tasks.workunit.client.0.smithi162.stdout:2/664: sync 2024-04-27T04:10:16.569 INFO:tasks.workunit.client.0.smithi162.stdout:5/709: sync 2024-04-27T04:10:16.573 INFO:tasks.workunit.client.1.smithi181.stdout:7/849: dwrite d1/d2/d1c/d2d/fea [0,4194304] 0 2024-04-27T04:10:16.573 INFO:tasks.workunit.client.0.smithi162.stdout:2/665: dread d0/f44 [0,4194304] 0 2024-04-27T04:10:16.575 INFO:tasks.workunit.client.0.smithi162.stdout:2/666: write d0/d4/d17/d2b/d64/f90 [2158617,94299] 0 2024-04-27T04:10:16.576 INFO:tasks.workunit.client.0.smithi162.stdout:2/667: chown d0/d4/d2a/d3b/d49/d5a/d63/d70 15491392 1 2024-04-27T04:10:16.577 INFO:tasks.workunit.client.0.smithi162.stdout:2/668: readlink d0/d4/d17/d2b/l8f 0 2024-04-27T04:10:16.583 INFO:tasks.workunit.client.0.smithi162.stdout:1/652: symlink d0/d11/d44/d66/ld9 0 2024-04-27T04:10:16.584 INFO:tasks.workunit.client.1.smithi181.stdout:6/737: dread - d7/dc/d1c/d7f/f76 zero size 2024-04-27T04:10:16.585 INFO:tasks.workunit.client.0.smithi162.stdout:7/722: symlink d3/d5/d5b/lf0 0 2024-04-27T04:10:16.587 INFO:tasks.workunit.client.0.smithi162.stdout:0/794: creat d5/d26/d4a/def/f106 x:0 0 0 2024-04-27T04:10:16.587 INFO:tasks.workunit.client.0.smithi162.stdout:7/723: dread - d3/d5/d9/fe2 zero size 2024-04-27T04:10:16.588 INFO:tasks.workunit.client.0.smithi162.stdout:4/646: dwrite d2/d25/d1c/d39/d3b/d47/f6c [4194304,4194304] 0 2024-04-27T04:10:16.591 INFO:tasks.workunit.client.1.smithi181.stdout:9/784: dread d0/d44/d4f/f7a [0,4194304] 0 2024-04-27T04:10:16.593 INFO:tasks.workunit.client.0.smithi162.stdout:5/710: rename d8/d86 to d8/d1a/d7d/ddf 0 2024-04-27T04:10:16.594 INFO:tasks.workunit.client.1.smithi181.stdout:9/785: dread d0/d44/d4f/f7a [0,4194304] 0 2024-04-27T04:10:16.594 INFO:tasks.workunit.client.0.smithi162.stdout:5/711: chown d8/d1a/d23/dbc/d66/c94 6 1 2024-04-27T04:10:16.594 INFO:tasks.workunit.client.1.smithi181.stdout:2/905: fsync d4/d9/df/d14/f48 0 2024-04-27T04:10:16.598 INFO:tasks.workunit.client.1.smithi181.stdout:9/786: chown d0/d44/d4b/d43/d88/da1/dfc/f9f 3992 1 2024-04-27T04:10:16.598 INFO:tasks.workunit.client.1.smithi181.stdout:4/787: creat d0/d15/d25/d2a/d36/db8/dca/f103 x:0 0 0 2024-04-27T04:10:16.603 INFO:tasks.workunit.client.1.smithi181.stdout:6/738: creat d7/dc/dc3/d72/d98/dbf/ff7 x:0 0 0 2024-04-27T04:10:16.604 INFO:tasks.workunit.client.0.smithi162.stdout:0/795: dread d5/dc/d1e/d41/f71 [0,4194304] 0 2024-04-27T04:10:16.607 INFO:tasks.workunit.client.0.smithi162.stdout:8/751: mknod d0/d10/d25/cf6 0 2024-04-27T04:10:16.608 INFO:tasks.workunit.client.1.smithi181.stdout:1/835: getdents d1/d2/d64/ddc 0 2024-04-27T04:10:16.609 INFO:tasks.workunit.client.0.smithi162.stdout:4/647: mkdir d2/d25/d1c/d39/d70/dcd 0 2024-04-27T04:10:16.611 INFO:tasks.workunit.client.0.smithi162.stdout:9/677: sync 2024-04-27T04:10:16.611 INFO:tasks.workunit.client.0.smithi162.stdout:3/670: sync 2024-04-27T04:10:16.611 INFO:tasks.workunit.client.0.smithi162.stdout:6/660: sync 2024-04-27T04:10:16.612 INFO:tasks.workunit.client.0.smithi162.stdout:9/678: chown c1 1138 1 2024-04-27T04:10:16.614 INFO:tasks.workunit.client.0.smithi162.stdout:2/669: rename d0/d4/d17/d2b/c53 to d0/d4/d2a/da1/d86/cd2 0 2024-04-27T04:10:16.618 INFO:tasks.workunit.client.0.smithi162.stdout:2/670: write d0/d4/d17/d88/fad [638141,108029] 0 2024-04-27T04:10:16.618 INFO:tasks.workunit.client.0.smithi162.stdout:1/653: dread d0/d11/d44/d66/f90 [0,4194304] 0 2024-04-27T04:10:16.618 INFO:tasks.workunit.client.0.smithi162.stdout:5/712: dread d8/d1a/d23/f50 [0,4194304] 0 2024-04-27T04:10:16.619 INFO:tasks.workunit.client.1.smithi181.stdout:6/739: rmdir d7/dc/d1e/d66 39 2024-04-27T04:10:16.619 INFO:tasks.workunit.client.1.smithi181.stdout:9/787: dread d0/d44/d4b/d43/f98 [0,4194304] 0 2024-04-27T04:10:16.619 INFO:tasks.workunit.client.0.smithi162.stdout:5/713: stat d8/fad 0 2024-04-27T04:10:16.619 INFO:tasks.workunit.client.0.smithi162.stdout:2/671: truncate d0/d20/d51/fcd 4848091 0 2024-04-27T04:10:16.620 INFO:tasks.workunit.client.1.smithi181.stdout:9/788: fsync d0/d44/d4b/d43/d89/daf/ff9 0 2024-04-27T04:10:16.621 INFO:tasks.workunit.client.1.smithi181.stdout:6/740: write d7/dc/d1c/d34/d3f/d7e/d8a/f7a [4489979,96378] 0 2024-04-27T04:10:16.623 INFO:tasks.workunit.client.1.smithi181.stdout:4/788: dread d0/d15/d25/d2b/d6d/d93/fda [0,4194304] 0 2024-04-27T04:10:16.623 INFO:tasks.workunit.client.0.smithi162.stdout:7/724: dwrite d3/d5/d3c/d4d/f89 [0,4194304] 0 2024-04-27T04:10:16.627 INFO:tasks.workunit.client.1.smithi181.stdout:1/836: unlink d1/dad/db5/d6/da3/d1e/d48/dfa/f10e 0 2024-04-27T04:10:16.630 INFO:tasks.workunit.client.0.smithi162.stdout:3/671: creat d1/d18/d23/d58/d63/d88/fe8 x:0 0 0 2024-04-27T04:10:16.630 INFO:tasks.workunit.client.1.smithi181.stdout:2/906: mknod d4/d9/d16/d82/d7d/dbe/d10b/c124 0 2024-04-27T04:10:16.631 INFO:tasks.workunit.client.0.smithi162.stdout:6/661: truncate d6/d1a/d34/d3a/d68/d6c/f82 706287 0 2024-04-27T04:10:16.632 INFO:tasks.workunit.client.0.smithi162.stdout:9/679: truncate d3/d13/d15/d50/f8b 761444 0 2024-04-27T04:10:16.632 INFO:tasks.workunit.client.0.smithi162.stdout:6/662: write d6/db/fb0 [1802617,45646] 0 2024-04-27T04:10:16.633 INFO:tasks.workunit.client.0.smithi162.stdout:6/663: chown d6/d1a/d34/fb6 24 1 2024-04-27T04:10:16.634 INFO:tasks.workunit.client.0.smithi162.stdout:6/664: readlink d6/d1a/d34/d3a/d68/d6c/l65 0 2024-04-27T04:10:16.635 INFO:tasks.workunit.client.0.smithi162.stdout:6/665: readlink d6/d1a/d34/d3a/l5f 0 2024-04-27T04:10:16.635 INFO:tasks.workunit.client.1.smithi181.stdout:7/850: rename d1/d2/d1c/d2d/d61/l92 to d1/d22/d15/d29/dbe/lff 0 2024-04-27T04:10:16.637 INFO:tasks.workunit.client.1.smithi181.stdout:3/822: getdents d0/d6c/d52/d44/dd5/d58/de4/daf 0 2024-04-27T04:10:16.638 INFO:tasks.workunit.client.0.smithi162.stdout:5/714: rename d8/d1a/f83 to d8/d2c/fe0 0 2024-04-27T04:10:16.641 INFO:tasks.workunit.client.0.smithi162.stdout:2/672: creat d0/d4/d2a/d41/d77/d73/fd3 x:0 0 0 2024-04-27T04:10:16.642 INFO:tasks.workunit.client.1.smithi181.stdout:9/789: mknod d0/d44/d4b/d43/d88/de3/c10f 0 2024-04-27T04:10:16.648 INFO:tasks.workunit.client.0.smithi162.stdout:1/654: write d0/d7/d10/f12 [1394409,120648] 0 2024-04-27T04:10:16.650 INFO:tasks.workunit.client.1.smithi181.stdout:4/789: read d0/d15/d25/d2b/f2c [610845,2592] 0 2024-04-27T04:10:16.651 INFO:tasks.workunit.client.1.smithi181.stdout:4/790: write d0/d15/fe9 [514806,20763] 0 2024-04-27T04:10:16.652 INFO:tasks.workunit.client.1.smithi181.stdout:1/837: mknod d1/dad/db5/d6/db/d2d/df1/c114 0 2024-04-27T04:10:16.654 INFO:tasks.workunit.client.1.smithi181.stdout:1/838: write d1/dad/db5/fa1 [2861991,66748] 0 2024-04-27T04:10:16.655 INFO:tasks.workunit.client.1.smithi181.stdout:2/907: mkdir d4/d9/d4f/d5f/d125 0 2024-04-27T04:10:16.656 INFO:tasks.workunit.client.0.smithi162.stdout:0/796: dread d5/dc/d1e/d20/dba/df9/f72 [0,4194304] 0 2024-04-27T04:10:16.657 INFO:tasks.workunit.client.1.smithi181.stdout:7/851: dread d1/d2/d1c/d2d/d61/f68 [0,4194304] 0 2024-04-27T04:10:16.659 INFO:tasks.workunit.client.1.smithi181.stdout:7/852: truncate d1/d2/d1c/d2d/d61/fee 335762 0 2024-04-27T04:10:16.660 INFO:tasks.workunit.client.1.smithi181.stdout:3/823: creat d0/d6c/d52/dbb/dca/f107 x:0 0 0 2024-04-27T04:10:16.661 INFO:tasks.workunit.client.1.smithi181.stdout:3/824: chown d0/d6c/d52/fd0 13747025 1 2024-04-27T04:10:16.663 INFO:tasks.workunit.client.0.smithi162.stdout:9/680: write d3/d23/d1e/d43/f5a [3912651,117293] 0 2024-04-27T04:10:16.665 INFO:tasks.workunit.client.0.smithi162.stdout:2/673: creat d0/d4/d17/d2b/d64/d89/fd4 x:0 0 0 2024-04-27T04:10:16.667 INFO:tasks.workunit.client.1.smithi181.stdout:9/790: write d0/d44/f7c [3373407,7553] 0 2024-04-27T04:10:16.669 INFO:tasks.workunit.client.1.smithi181.stdout:4/791: write d0/d15/d25/d2a/d36/f92 [842018,22598] 0 2024-04-27T04:10:16.670 INFO:tasks.workunit.client.1.smithi181.stdout:2/908: symlink d4/d81/d102/l126 0 2024-04-27T04:10:16.672 INFO:tasks.workunit.client.0.smithi162.stdout:7/725: write d3/d5/f5e [143058,60069] 0 2024-04-27T04:10:16.673 INFO:tasks.workunit.client.1.smithi181.stdout:1/839: mkdir d1/dad/db5/d6/da3/d1e/d20/d53/d56/de2/d115 0 2024-04-27T04:10:16.673 INFO:tasks.workunit.client.0.smithi162.stdout:4/648: link d2/d54/lbd d2/d54/dca/lce 0 2024-04-27T04:10:16.676 INFO:tasks.workunit.client.1.smithi181.stdout:7/853: creat d1/d22/d15/d5f/d81/de3/f100 x:0 0 0 2024-04-27T04:10:16.677 INFO:tasks.workunit.client.1.smithi181.stdout:2/909: dwrite d4/f50 [4194304,4194304] 0 2024-04-27T04:10:16.681 INFO:tasks.workunit.client.1.smithi181.stdout:7/854: write d1/d22/d15/d1f/d34/fe2 [4566981,112853] 0 2024-04-27T04:10:16.684 INFO:tasks.workunit.client.0.smithi162.stdout:5/715: mknod d8/d9/ce1 0 2024-04-27T04:10:16.685 INFO:tasks.workunit.client.0.smithi162.stdout:1/655: dwrite d0/d11/d3e/f4f [0,4194304] 0 2024-04-27T04:10:16.685 INFO:tasks.workunit.client.1.smithi181.stdout:4/792: chown d0/d15/de/d19/d59/l88 135130203 1 2024-04-27T04:10:16.689 INFO:tasks.workunit.client.0.smithi162.stdout:9/681: mknod d3/d13/d2d/d30/cd1 0 2024-04-27T04:10:16.690 INFO:tasks.workunit.client.0.smithi162.stdout:2/674: write d0/d4/d2a/d3b/d49/d5a/d63/d84/f99 [1165750,12891] 0 2024-04-27T04:10:16.690 INFO:tasks.workunit.client.0.smithi162.stdout:9/682: chown d3/d11/d45/f55 787519537 1 2024-04-27T04:10:16.691 INFO:tasks.workunit.client.0.smithi162.stdout:9/683: stat d3/cad 0 2024-04-27T04:10:16.691 INFO:tasks.workunit.client.0.smithi162.stdout:5/716: dread d8/d1a/d7d/f91 [0,4194304] 0 2024-04-27T04:10:16.692 INFO:tasks.workunit.client.1.smithi181.stdout:4/793: dread d0/d15/d25/d2a/da0/dc7/fef [0,4194304] 0 2024-04-27T04:10:16.692 INFO:tasks.workunit.client.0.smithi162.stdout:9/684: chown d3/db9/dcc/fcd 401 1 2024-04-27T04:10:16.692 INFO:tasks.workunit.client.1.smithi181.stdout:2/910: mkdir d4/d9/d16/d82/d7d/dbe/d127 0 2024-04-27T04:10:16.693 INFO:tasks.workunit.client.1.smithi181.stdout:3/825: dread d0/de/f10 [0,4194304] 0 2024-04-27T04:10:16.694 INFO:tasks.workunit.client.0.smithi162.stdout:7/726: read - d3/d5/d5b/d83/de/d2b/d81/daa/dbd/f97 zero size 2024-04-27T04:10:16.695 INFO:tasks.workunit.client.1.smithi181.stdout:6/741: getdents d7/dc/d1c/d7f 0 2024-04-27T04:10:16.698 INFO:tasks.workunit.client.1.smithi181.stdout:7/855: creat d1/d22/d15/d5f/d93/df3/f101 x:0 0 0 2024-04-27T04:10:16.698 INFO:tasks.workunit.client.1.smithi181.stdout:3/826: write d0/d6c/d52/d44/dd5/d58/de4/daf/f81 [1080329,117651] 0 2024-04-27T04:10:16.699 INFO:tasks.workunit.client.1.smithi181.stdout:9/791: fdatasync d0/d44/d4b/d43/d88/da1/dfc/dbd/dd1/dde/d104/f78 0 2024-04-27T04:10:16.704 INFO:tasks.workunit.client.0.smithi162.stdout:3/672: rename d1/d21/d47/d57/fc3 to d1/d18/d23/fe9 0 2024-04-27T04:10:16.707 INFO:tasks.workunit.client.1.smithi181.stdout:2/911: creat d4/d9/d4f/d79/d94/f128 x:0 0 0 2024-04-27T04:10:16.709 INFO:tasks.workunit.client.1.smithi181.stdout:6/742: symlink d7/dc/d1c/d34/d3f/d7e/lf8 0 2024-04-27T04:10:16.711 INFO:tasks.workunit.client.0.smithi162.stdout:4/649: dwrite d2/d25/f5c [0,4194304] 0 2024-04-27T04:10:16.713 INFO:tasks.workunit.client.0.smithi162.stdout:0/797: write d5/dc/d1e/d20/dba/df9/d98/fa3 [503654,79202] 0 2024-04-27T04:10:16.713 INFO:tasks.workunit.client.0.smithi162.stdout:4/650: fdatasync d2/d25/d1c/d39/d3b/f94 0 2024-04-27T04:10:16.715 INFO:tasks.workunit.client.1.smithi181.stdout:3/827: creat d0/d6c/d66/d88/f108 x:0 0 0 2024-04-27T04:10:16.715 INFO:tasks.workunit.client.0.smithi162.stdout:9/685: dread - d3/d13/d2d/d44/fa3 zero size 2024-04-27T04:10:16.716 INFO:tasks.workunit.client.0.smithi162.stdout:4/651: stat d2/d25/d1c/d39/d3b/d6e/c9b 0 2024-04-27T04:10:16.717 INFO:tasks.workunit.client.0.smithi162.stdout:7/727: unlink d3/d5/d5b/d83/c48 0 2024-04-27T04:10:16.718 INFO:tasks.workunit.client.0.smithi162.stdout:9/686: chown d3/d13/d2d/d30/d67/f97 71234146 1 2024-04-27T04:10:16.719 INFO:tasks.workunit.client.0.smithi162.stdout:6/666: getdents d6/d1a/d34/d50 0 2024-04-27T04:10:16.719 INFO:tasks.workunit.client.1.smithi181.stdout:4/794: mknod d0/d15/d5c/c104 0 2024-04-27T04:10:16.721 INFO:tasks.workunit.client.0.smithi162.stdout:7/728: dread d3/d5/d5b/fe6 [0,4194304] 0 2024-04-27T04:10:16.723 INFO:tasks.workunit.client.0.smithi162.stdout:6/667: dread d6/d1a/f2b [0,4194304] 0 2024-04-27T04:10:16.723 INFO:tasks.workunit.client.0.smithi162.stdout:6/668: chown d6/d1a/d34/d50/l72 325555616 1 2024-04-27T04:10:16.727 INFO:tasks.workunit.client.0.smithi162.stdout:1/656: link d0/d7/d3c/d54/dc9/fca d0/d11/d50/d53/d93/d96/daa/fda 0 2024-04-27T04:10:16.727 INFO:tasks.workunit.client.0.smithi162.stdout:8/752: link d0/l9 d0/d10/d25/d65/lf7 0 2024-04-27T04:10:16.732 INFO:tasks.workunit.client.1.smithi181.stdout:7/856: dwrite d1/d22/d15/d29/dbe/d1b/f8c [0,4194304] 0 2024-04-27T04:10:16.732 INFO:tasks.workunit.client.0.smithi162.stdout:2/675: symlink d0/d4/d17/d4f/db0/ld5 0 2024-04-27T04:10:16.732 INFO:tasks.workunit.client.1.smithi181.stdout:7/857: readlink d1/l2e 0 2024-04-27T04:10:16.733 INFO:tasks.workunit.client.0.smithi162.stdout:3/673: write d1/d18/f1a [1698594,49558] 0 2024-04-27T04:10:16.734 INFO:tasks.workunit.client.0.smithi162.stdout:3/674: chown d1/d18/d23/d34/daa 41 1 2024-04-27T04:10:16.735 INFO:tasks.workunit.client.0.smithi162.stdout:5/717: fsync d8/d9/d5b/f7f 0 2024-04-27T04:10:16.740 INFO:tasks.workunit.client.1.smithi181.stdout:6/743: rename d7/dc/d1c/d34/d3f/db2/lef to d7/dc/d1c/d34/d3f/d7e/d8a/lf9 0 2024-04-27T04:10:16.741 INFO:tasks.workunit.client.1.smithi181.stdout:3/828: write d0/de/f10 [2179566,6951] 0 2024-04-27T04:10:16.742 INFO:tasks.workunit.client.1.smithi181.stdout:6/744: fsync d7/dc/dc3/d72/daa/dd3/fd8 0 2024-04-27T04:10:16.742 INFO:tasks.workunit.client.0.smithi162.stdout:8/753: dread d0/d10/d2b/fbc [0,4194304] 0 2024-04-27T04:10:16.743 INFO:tasks.workunit.client.1.smithi181.stdout:6/745: fdatasync d7/dc/d1e/d2b/dde/fe7 0 2024-04-27T04:10:16.745 INFO:tasks.workunit.client.1.smithi181.stdout:1/840: getdents d1/dad/db5/d6/da3/d1e/dbc 0 2024-04-27T04:10:16.747 INFO:tasks.workunit.client.1.smithi181.stdout:7/858: creat d1/d2/d1c/d2d/d61/f102 x:0 0 0 2024-04-27T04:10:16.748 INFO:tasks.workunit.client.1.smithi181.stdout:7/859: write d1/d2/d1c/d2d/d61/fe0 [69409,45402] 0 2024-04-27T04:10:16.750 INFO:tasks.workunit.client.0.smithi162.stdout:0/798: dwrite d5/dc/d1e/d20/dba/df9/fd0 [0,4194304] 0 2024-04-27T04:10:16.750 INFO:tasks.workunit.client.0.smithi162.stdout:6/669: creat d6/d1a/d34/d7f/fd1 x:0 0 0 2024-04-27T04:10:16.752 INFO:tasks.workunit.client.0.smithi162.stdout:1/657: dread d0/d11/d44/f9a [0,4194304] 0 2024-04-27T04:10:16.753 INFO:tasks.workunit.client.0.smithi162.stdout:2/676: fdatasync d0/d4/d2a/f36 0 2024-04-27T04:10:16.756 INFO:tasks.workunit.client.1.smithi181.stdout:2/912: getdents d4/d9/d16/d82/d7d/dbe/ded 0 2024-04-27T04:10:16.757 INFO:tasks.workunit.client.0.smithi162.stdout:3/675: symlink d1/d18/d4f/lea 0 2024-04-27T04:10:16.758 INFO:tasks.workunit.client.1.smithi181.stdout:2/913: readlink d4/d9/df/d14/da7/ld8 0 2024-04-27T04:10:16.759 INFO:tasks.workunit.client.0.smithi162.stdout:3/676: truncate d1/d18/dac/fcf 15030 0 2024-04-27T04:10:16.759 INFO:tasks.workunit.client.1.smithi181.stdout:3/829: dwrite d0/d6c/d52/d44/dd5/d58/d64/dc6/fe6 [0,4194304] 0 2024-04-27T04:10:16.760 INFO:tasks.workunit.client.0.smithi162.stdout:5/718: creat d8/d2c/fe2 x:0 0 0 2024-04-27T04:10:16.762 INFO:tasks.workunit.client.1.smithi181.stdout:4/795: getdents d0/d15/dd2 0 2024-04-27T04:10:16.763 INFO:tasks.workunit.client.0.smithi162.stdout:1/658: dread d0/f37 [0,4194304] 0 2024-04-27T04:10:16.766 INFO:tasks.workunit.client.1.smithi181.stdout:1/841: truncate d1/dad/db5/d6/db/d2d/d36/dc4/fcf 2956939 0 2024-04-27T04:10:16.766 INFO:tasks.workunit.client.0.smithi162.stdout:7/729: write d3/d5/d3c/d4a/fb0 [1202812,29759] 0 2024-04-27T04:10:16.772 INFO:tasks.workunit.client.0.smithi162.stdout:9/687: mkdir d3/d13/dd2 0 2024-04-27T04:10:16.775 INFO:tasks.workunit.client.1.smithi181.stdout:7/860: truncate d1/d22/f95 2562190 0 2024-04-27T04:10:16.777 INFO:tasks.workunit.client.1.smithi181.stdout:4/796: creat d0/d15/d25/d2a/f105 x:0 0 0 2024-04-27T04:10:16.777 INFO:tasks.workunit.client.1.smithi181.stdout:4/797: fsync d0/d15/fe9 0 2024-04-27T04:10:16.781 INFO:tasks.workunit.client.1.smithi181.stdout:1/842: dread d1/dad/db5/d6/da3/d52/f9e [0,4194304] 0 2024-04-27T04:10:16.786 INFO:tasks.workunit.client.1.smithi181.stdout:7/861: creat d1/d2/d1c/d37/d91/f103 x:0 0 0 2024-04-27T04:10:16.788 INFO:tasks.workunit.client.0.smithi162.stdout:3/677: symlink d1/d18/d5d/d5f/d66/dd2/leb 0 2024-04-27T04:10:16.789 INFO:tasks.workunit.client.0.smithi162.stdout:3/678: chown d1/d18/d5d/d5f 126 1 2024-04-27T04:10:16.789 INFO:tasks.workunit.client.1.smithi181.stdout:4/798: readlink d0/d15/d25/d40/la3 0 2024-04-27T04:10:16.789 INFO:tasks.workunit.client.1.smithi181.stdout:3/830: dwrite d0/de/ff [0,4194304] 0 2024-04-27T04:10:16.790 INFO:tasks.workunit.client.0.smithi162.stdout:5/719: truncate d8/d1a/f17 1358080 0 2024-04-27T04:10:16.790 INFO:tasks.workunit.client.1.smithi181.stdout:3/831: fsync d0/d28/f42 0 2024-04-27T04:10:16.791 INFO:tasks.workunit.client.1.smithi181.stdout:1/843: chown d1/dad/db5/c86 27649489 1 2024-04-27T04:10:16.793 INFO:tasks.workunit.client.0.smithi162.stdout:4/652: creat d2/d25/d1c/d24/fcf x:0 0 0 2024-04-27T04:10:16.793 INFO:tasks.workunit.client.0.smithi162.stdout:4/653: chown d2/d25/f79 0 1 2024-04-27T04:10:16.794 INFO:tasks.workunit.client.0.smithi162.stdout:4/654: write d2/da/d1a/f9d [1145081,101075] 0 2024-04-27T04:10:16.795 INFO:tasks.workunit.client.0.smithi162.stdout:4/655: chown d2/da/f78 28425 1 2024-04-27T04:10:16.795 INFO:tasks.workunit.client.0.smithi162.stdout:7/730: mkdir d3/d5/d5b/d83/de/d2b/d81/daa/df1 0 2024-04-27T04:10:16.796 INFO:tasks.workunit.client.1.smithi181.stdout:3/832: dread d0/d6c/d52/d44/d56/f76 [0,4194304] 0 2024-04-27T04:10:16.797 INFO:tasks.workunit.client.0.smithi162.stdout:7/731: chown d3/d5/d5b/d83/de/d2b/d81/daa/dbd/db6/dec 4006538 1 2024-04-27T04:10:16.798 INFO:tasks.workunit.client.1.smithi181.stdout:2/914: link d4/d9/df/d14/l69 d4/d81/df3/l129 0 2024-04-27T04:10:16.798 INFO:tasks.workunit.client.1.smithi181.stdout:2/915: chown d4/d81/d102/l126 66798231 1 2024-04-27T04:10:16.799 INFO:tasks.workunit.client.0.smithi162.stdout:9/688: creat d3/d23/fd3 x:0 0 0 2024-04-27T04:10:16.800 INFO:tasks.workunit.client.1.smithi181.stdout:2/916: chown d4/d9/df/c58 273957083 1 2024-04-27T04:10:16.800 INFO:tasks.workunit.client.1.smithi181.stdout:7/862: creat d1/d2/d1c/d2d/d61/d80/f104 x:0 0 0 2024-04-27T04:10:16.801 INFO:tasks.workunit.client.0.smithi162.stdout:6/670: mknod d6/d1a/d34/d3a/cd2 0 2024-04-27T04:10:16.804 INFO:tasks.workunit.client.0.smithi162.stdout:0/799: rmdir d5/dc/d1e/d46/d48 39 2024-04-27T04:10:16.805 INFO:tasks.workunit.client.0.smithi162.stdout:0/800: write d5/d26/d4a/fe6 [344354,12325] 0 2024-04-27T04:10:16.806 INFO:tasks.workunit.client.1.smithi181.stdout:4/799: rmdir d0/d15/d25/d2a/d36/db8 39 2024-04-27T04:10:16.809 INFO:tasks.workunit.client.0.smithi162.stdout:2/677: symlink d0/d4/d2a/d3b/d49/db7/ld6 0 2024-04-27T04:10:16.812 INFO:tasks.workunit.client.0.smithi162.stdout:3/679: creat d1/d18/d23/d2f/fec x:0 0 0 2024-04-27T04:10:16.815 INFO:tasks.workunit.client.0.smithi162.stdout:1/659: creat d0/d7/d3c/d61/dc4/fdb x:0 0 0 2024-04-27T04:10:16.816 INFO:tasks.workunit.client.1.smithi181.stdout:7/863: truncate d1/d2/f12 8779533 0 2024-04-27T04:10:16.817 INFO:tasks.workunit.client.0.smithi162.stdout:8/754: dwrite d0/de/d6e/f77 [0,4194304] 0 2024-04-27T04:10:16.817 INFO:tasks.workunit.client.1.smithi181.stdout:3/833: dwrite d0/d6c/d52/d44/dd5/d58/de4/daf/ff6 [0,4194304] 0 2024-04-27T04:10:16.822 INFO:tasks.workunit.client.0.smithi162.stdout:1/660: dread d0/d7/d10/f64 [0,4194304] 0 2024-04-27T04:10:16.822 INFO:tasks.workunit.client.0.smithi162.stdout:4/656: symlink d2/d25/d1c/d39/ld0 0 2024-04-27T04:10:16.827 INFO:tasks.workunit.client.1.smithi181.stdout:1/844: dwrite d1/dad/db5/fb8 [0,4194304] 0 2024-04-27T04:10:16.830 INFO:tasks.workunit.client.1.smithi181.stdout:1/845: truncate d1/dad/db5/d6/db/d2d/ffd 777487 0 2024-04-27T04:10:16.831 INFO:tasks.workunit.client.0.smithi162.stdout:6/671: creat d6/d1a/d34/d7f/d80/db3/fd3 x:0 0 0 2024-04-27T04:10:16.833 INFO:tasks.workunit.client.0.smithi162.stdout:0/801: mknod d5/d26/d69/c107 0 2024-04-27T04:10:16.833 INFO:tasks.workunit.client.0.smithi162.stdout:0/802: chown d5/dc/d1e/d46 10758553 1 2024-04-27T04:10:16.838 INFO:tasks.workunit.client.1.smithi181.stdout:2/917: rmdir d4/d9/d35/d9f/dd0/d11c 0 2024-04-27T04:10:16.840 INFO:tasks.workunit.client.0.smithi162.stdout:5/720: mkdir d8/de3 0 2024-04-27T04:10:16.844 INFO:tasks.workunit.client.0.smithi162.stdout:8/755: mkdir d0/d10/d2b/d2d/d2f/d39/df8 0 2024-04-27T04:10:16.846 INFO:tasks.workunit.client.1.smithi181.stdout:1/846: rename d1/dad/db5/d6/dcb/ded/f104 to d1/dad/db5/d6/da3/d1e/d20/d53/d56/de2/f116 0 2024-04-27T04:10:16.847 INFO:tasks.workunit.client.1.smithi181.stdout:1/847: stat d1/dad/db5/d6/da3/d52/l69 0 2024-04-27T04:10:16.847 INFO:tasks.workunit.client.0.smithi162.stdout:1/661: rmdir d0/d11/d3e/d59/d7f 39 2024-04-27T04:10:16.847 INFO:tasks.workunit.client.1.smithi181.stdout:1/848: dread - d1/dad/db5/d6/db/f91 zero size 2024-04-27T04:10:16.848 INFO:tasks.workunit.client.1.smithi181.stdout:7/864: write d1/d2/d35/d67/f69 [3244980,38943] 0 2024-04-27T04:10:16.849 INFO:tasks.workunit.client.1.smithi181.stdout:1/849: dread - d1/dad/db5/d6/da3/f10a zero size 2024-04-27T04:10:16.849 INFO:tasks.workunit.client.1.smithi181.stdout:1/850: chown d1/dad 104 1 2024-04-27T04:10:16.850 INFO:tasks.workunit.client.0.smithi162.stdout:7/732: dwrite d3/d4f/f31 [0,4194304] 0 2024-04-27T04:10:16.851 INFO:tasks.workunit.client.1.smithi181.stdout:4/800: link d0/d15/d25/d2b/f5f d0/d15/de/d19/d77/f106 0 2024-04-27T04:10:16.854 INFO:tasks.workunit.client.0.smithi162.stdout:6/672: creat d6/d1a/d34/d50/fd4 x:0 0 0 2024-04-27T04:10:16.857 INFO:tasks.workunit.client.0.smithi162.stdout:3/680: creat d1/d18/d23/d9a/fed x:0 0 0 2024-04-27T04:10:16.859 INFO:tasks.workunit.client.1.smithi181.stdout:1/851: rmdir d1/dad/db5/d6/da3/d1e/d48/da6/da4 39 2024-04-27T04:10:16.860 INFO:tasks.workunit.client.0.smithi162.stdout:2/678: dwrite d0/f44 [0,4194304] 0 2024-04-27T04:10:16.861 INFO:tasks.workunit.client.1.smithi181.stdout:3/834: getdents d0/d6c/d52/d44/d56 0 2024-04-27T04:10:16.864 INFO:tasks.workunit.client.1.smithi181.stdout:2/918: link d4/d9/d16/d45/d9a/ff8 d4/d9/d4f/d79/f12a 0 2024-04-27T04:10:16.865 INFO:tasks.workunit.client.0.smithi162.stdout:2/679: chown d0/d4/d17/d4f/d7f/c8d 216 1 2024-04-27T04:10:16.866 INFO:tasks.workunit.client.0.smithi162.stdout:9/689: rmdir d3/d23/d1e 39 2024-04-27T04:10:16.867 INFO:tasks.workunit.client.0.smithi162.stdout:6/673: dwrite d6/d1a/d34/f66 [0,4194304] 0 2024-04-27T04:10:16.868 INFO:tasks.workunit.client.1.smithi181.stdout:1/852: fsync d1/f8e 0 2024-04-27T04:10:16.868 INFO:tasks.workunit.client.0.smithi162.stdout:9/690: stat d3/d13 0 2024-04-27T04:10:16.869 INFO:tasks.workunit.client.0.smithi162.stdout:0/803: dwrite d5/dc/d1e/d46/d48/d9e/dce/ffd [4194304,4194304] 0 2024-04-27T04:10:16.870 INFO:tasks.workunit.client.1.smithi181.stdout:4/801: fsync d0/d15/de/d19/d77/f106 0 2024-04-27T04:10:16.870 INFO:tasks.workunit.client.1.smithi181.stdout:3/835: dwrite d0/d6c/f79 [0,4194304] 0 2024-04-27T04:10:16.873 INFO:tasks.workunit.client.0.smithi162.stdout:3/681: dwrite d1/d21/d47/d57/d6e/f9e [0,4194304] 0 2024-04-27T04:10:16.874 INFO:tasks.workunit.client.0.smithi162.stdout:3/682: write d1/d70/f92 [924783,100715] 0 2024-04-27T04:10:16.876 INFO:tasks.workunit.client.0.smithi162.stdout:7/733: truncate d3/d5/d9/fb8 998637 0 2024-04-27T04:10:16.878 INFO:tasks.workunit.client.0.smithi162.stdout:7/734: write d3/d4f/f67 [2858887,37637] 0 2024-04-27T04:10:16.879 INFO:tasks.workunit.client.1.smithi181.stdout:7/865: creat d1/d22/d15/d1f/f105 x:0 0 0 2024-04-27T04:10:16.880 INFO:tasks.workunit.client.1.smithi181.stdout:7/866: readlink d1/d22/d15/d29/dbe/lef 0 2024-04-27T04:10:16.881 INFO:tasks.workunit.client.0.smithi162.stdout:1/662: rename d0/d7/lc7 to d0/d7/d2a/ldc 0 2024-04-27T04:10:16.883 INFO:tasks.workunit.client.0.smithi162.stdout:2/680: chown d0/d4/d17/db4/fc7 4 1 2024-04-27T04:10:16.885 INFO:tasks.workunit.client.0.smithi162.stdout:2/681: dread - d0/d4/d2a/d3b/d49/d5a/d63/d6e/f98 zero size 2024-04-27T04:10:16.886 INFO:tasks.workunit.client.0.smithi162.stdout:2/682: chown d0/d4/d17/d4f/c5c 54203366 1 2024-04-27T04:10:16.891 INFO:tasks.workunit.client.1.smithi181.stdout:7/867: dwrite d1/d2/d1c/d2d/d61/d80/f104 [0,4194304] 0 2024-04-27T04:10:16.916 INFO:tasks.workunit.client.0.smithi162.stdout:5/721: dread d8/f22 [0,4194304] 0 2024-04-27T04:10:16.998 INFO:tasks.workunit.client.1.smithi181.stdout:8/879: sync 2024-04-27T04:10:16.998 INFO:tasks.workunit.client.1.smithi181.stdout:0/991: sync 2024-04-27T04:10:17.004 INFO:tasks.workunit.client.0.smithi162.stdout:6/674: dread d6/d1a/f31 [0,4194304] 0 2024-04-27T04:10:17.004 INFO:tasks.workunit.client.1.smithi181.stdout:4/802: mknod d0/d15/d76/db1/c107 0 2024-04-27T04:10:17.005 INFO:tasks.workunit.client.0.smithi162.stdout:9/691: symlink d3/d13/d2d/d3b/d54/ld4 0 2024-04-27T04:10:17.009 INFO:tasks.workunit.client.0.smithi162.stdout:3/683: mkdir d1/d65/dee 0 2024-04-27T04:10:17.010 INFO:tasks.workunit.client.1.smithi181.stdout:1/853: write d1/d2/f4 [7467961,106926] 0 2024-04-27T04:10:17.010 INFO:tasks.workunit.client.1.smithi181.stdout:7/868: dread d1/d22/fc0 [0,4194304] 0 2024-04-27T04:10:17.012 INFO:tasks.workunit.client.1.smithi181.stdout:1/854: fdatasync d1/dad/db5/d6/da3/d1e/d48/db4/f100 0 2024-04-27T04:10:17.013 INFO:tasks.workunit.client.1.smithi181.stdout:3/836: write d0/d6c/d52/f3d [1733307,77555] 0 2024-04-27T04:10:17.014 INFO:tasks.workunit.client.1.smithi181.stdout:3/837: write d0/d6c/d66/d88/fcd [2437609,976] 0 2024-04-27T04:10:17.015 INFO:tasks.workunit.client.0.smithi162.stdout:1/663: fdatasync d0/f37 0 2024-04-27T04:10:17.015 INFO:tasks.workunit.client.0.smithi162.stdout:1/664: dread - d0/d7/d10/fa7 zero size 2024-04-27T04:10:17.017 INFO:tasks.workunit.client.0.smithi162.stdout:2/683: creat d0/d4/d2a/d3b/d49/d5a/d63/d84/fd7 x:0 0 0 2024-04-27T04:10:17.019 INFO:tasks.workunit.client.1.smithi181.stdout:2/919: dwrite d4/d9/d16/d82/d7d/db7/dcb/da4/dc2/f11d [4194304,4194304] 0 2024-04-27T04:10:17.019 INFO:tasks.workunit.client.0.smithi162.stdout:8/756: getdents d0/d10/d2b/d2d/d2f/d39/d7c 0 2024-04-27T04:10:17.022 INFO:tasks.workunit.client.0.smithi162.stdout:0/804: symlink d5/dc/d1e/d46/df3/l108 0 2024-04-27T04:10:17.023 INFO:tasks.workunit.client.0.smithi162.stdout:1/665: dwrite d0/d80/d89/fd2 [0,4194304] 0 2024-04-27T04:10:17.025 INFO:tasks.workunit.client.0.smithi162.stdout:6/675: creat d6/db/d32/fd5 x:0 0 0 2024-04-27T04:10:17.026 INFO:tasks.workunit.client.0.smithi162.stdout:2/684: dwrite d0/d4/f3a [0,4194304] 0 2024-04-27T04:10:17.030 INFO:tasks.workunit.client.0.smithi162.stdout:2/685: readlink d0/d24/l25 0 2024-04-27T04:10:17.030 INFO:tasks.workunit.client.0.smithi162.stdout:6/676: write d6/d1a/d34/d43/fab [4920,57273] 0 2024-04-27T04:10:17.030 INFO:tasks.workunit.client.0.smithi162.stdout:3/684: mknod d1/d21/d47/d57/d6e/cef 0 2024-04-27T04:10:17.034 INFO:tasks.workunit.client.0.smithi162.stdout:1/666: rmdir d0/d11/d44/d66 39 2024-04-27T04:10:17.035 INFO:tasks.workunit.client.0.smithi162.stdout:6/677: write d6/d1a/f2b [2680767,100038] 0 2024-04-27T04:10:17.036 INFO:tasks.workunit.client.0.smithi162.stdout:8/757: symlink d0/de/d6e/lf9 0 2024-04-27T04:10:17.038 INFO:tasks.workunit.client.0.smithi162.stdout:7/735: mknod d3/d5/d3c/d4a/cf2 0 2024-04-27T04:10:17.039 INFO:tasks.workunit.client.0.smithi162.stdout:0/805: fdatasync d5/dc/d1e/f30 0 2024-04-27T04:10:17.039 INFO:tasks.workunit.client.0.smithi162.stdout:3/685: write d1/d18/d23/fe9 [4529323,82792] 0 2024-04-27T04:10:17.042 INFO:tasks.workunit.client.0.smithi162.stdout:2/686: dwrite d0/d4/d2a/f36 [0,4194304] 0 2024-04-27T04:10:17.043 INFO:tasks.workunit.client.0.smithi162.stdout:1/667: mknod d0/d7/d2a/cdd 0 2024-04-27T04:10:17.047 INFO:tasks.workunit.client.0.smithi162.stdout:7/736: symlink d3/d39/lf3 0 2024-04-27T04:10:17.049 INFO:tasks.workunit.client.0.smithi162.stdout:8/758: dwrite d0/de/d20/d7e/fd1 [0,4194304] 0 2024-04-27T04:10:17.050 INFO:tasks.workunit.client.0.smithi162.stdout:7/737: stat d3/d5/d5b/d83/de/f62 0 2024-04-27T04:10:17.052 INFO:tasks.workunit.client.0.smithi162.stdout:8/759: readlink d0/de/d6e/d71/d86/dd9/ldb 0 2024-04-27T04:10:17.055 INFO:tasks.workunit.client.0.smithi162.stdout:2/687: creat d0/d4/d2a/d41/d77/d73/dc3/fd8 x:0 0 0 2024-04-27T04:10:17.056 INFO:tasks.workunit.client.0.smithi162.stdout:3/686: getdents d1/d21/d47/d57/d6e 0 2024-04-27T04:10:17.061 INFO:tasks.workunit.client.0.smithi162.stdout:1/668: dwrite d0/d7/d10/f3d [4194304,4194304] 0 2024-04-27T04:10:17.061 INFO:tasks.workunit.client.0.smithi162.stdout:7/738: creat d3/d5/d5b/ff4 x:0 0 0 2024-04-27T04:10:17.064 INFO:tasks.workunit.client.0.smithi162.stdout:0/806: dwrite d5/dc/d1e/fc7 [0,4194304] 0 2024-04-27T04:10:17.065 INFO:tasks.workunit.client.0.smithi162.stdout:6/678: dread d6/d1a/f15 [0,4194304] 0 2024-04-27T04:10:17.070 INFO:tasks.workunit.client.0.smithi162.stdout:0/807: dread - d5/dc/d1e/d20/dec/df1/db4/ffa zero size 2024-04-27T04:10:17.070 INFO:tasks.workunit.client.0.smithi162.stdout:8/760: link d0/d10/d2b/d2e/dca/fa0 d0/d10/d25/d65/d96/dcd/ffa 0 2024-04-27T04:10:17.074 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:10:17.073+0000 7f6ffa958700 1 -- 172.21.15.162:0/351260421 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6ff40fb420 msgr2=0x7f6ff40fb830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:10:17.074 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:10:17.073+0000 7f6ffa958700 1 --2- 172.21.15.162:0/351260421 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6ff40fb420 0x7f6ff40fb830 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f6fe8004940 tx=0x7f6fe8009e30 comp rx=0 tx=0).stop 2024-04-27T04:10:17.074 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:10:17.073+0000 7f6ffa958700 1 -- 172.21.15.162:0/351260421 shutdown_connections 2024-04-27T04:10:17.074 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:10:17.073+0000 7f6ffa958700 1 --2- 172.21.15.162:0/351260421 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6ff40fc6d0 0x7f6ff40fcb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:10:17.074 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:10:17.073+0000 7f6ffa958700 1 --2- 172.21.15.162:0/351260421 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6ff40fb420 0x7f6ff40fb830 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:10:17.074 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:10:17.073+0000 7f6ffa958700 1 -- 172.21.15.162:0/351260421 >> 172.21.15.162:0/351260421 conn(0x7f6ff40f6b80 msgr2=0x7f6ff40f8fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:10:17.074 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:10:17.073+0000 7f6ffa958700 1 -- 172.21.15.162:0/351260421 shutdown_connections 2024-04-27T04:10:17.074 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:10:17.073+0000 7f6ffa958700 1 -- 172.21.15.162:0/351260421 wait complete. 2024-04-27T04:10:17.074 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:10:17.074+0000 7f6ffa958700 1 Processor -- start 2024-04-27T04:10:17.075 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:10:17.074+0000 7f6ffa958700 1 -- start start 2024-04-27T04:10:17.076 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:10:17.075+0000 7f6ffa958700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6ff40fb420 0x7f6ff410f9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:10:17.076 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:10:17.075+0000 7f6ffa958700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6ff40fc6d0 0x7f6ff410ff20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:10:17.076 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:10:17.075+0000 7f6ffa958700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ff4110520 con 0x7f6ff40fc6d0 2024-04-27T04:10:17.076 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:10:17.075+0000 7f6ffa958700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ff4110d70 con 0x7f6ff40fb420 2024-04-27T04:10:17.076 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:10:17.075+0000 7f6ff37fe700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6ff40fc6d0 0x7f6ff410ff20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:10:17.076 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:10:17.075+0000 7f6ff37fe700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6ff40fc6d0 0x7f6ff410ff20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:51794/0 (socket says 172.21.15.162:51794) 2024-04-27T04:10:17.076 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:10:17.075+0000 7f6ff37fe700 1 -- 172.21.15.162:0/12011969 learned_addr learned my addr 172.21.15.162:0/12011969 (peer_addr_for_me v2:172.21.15.162:0/0) 2024-04-27T04:10:17.076 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:10:17.075+0000 7f6ff37fe700 1 -- 172.21.15.162:0/12011969 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6ff40fb420 msgr2=0x7f6ff410f9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:10:17.076 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:10:17.075+0000 7f6ff37fe700 1 --2- 172.21.15.162:0/12011969 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6ff40fb420 0x7f6ff410f9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:10:17.076 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:10:17.075+0000 7f6ff37fe700 1 -- 172.21.15.162:0/12011969 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6fe8005040 con 0x7f6ff40fc6d0 2024-04-27T04:10:17.077 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:10:17.076+0000 7f6ff37fe700 1 --2- 172.21.15.162:0/12011969 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6ff40fc6d0 0x7f6ff410ff20 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f6fe4002a50 tx=0x7f6fe40046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:10:17.077 INFO:tasks.workunit.client.0.smithi162.stdout:1/669: rename d0/f37 to d0/d11/d50/d53/d93/d96/fde 0 2024-04-27T04:10:17.078 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:10:17.076+0000 7f6ff0ff9700 1 -- 172.21.15.162:0/12011969 <== mon.0 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6fe4007cc0 con 0x7f6ff40fc6d0 2024-04-27T04:10:17.078 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:10:17.076+0000 7f6ffa958700 1 -- 172.21.15.162:0/12011969 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6ff4111050 con 0x7f6ff40fc6d0 2024-04-27T04:10:17.078 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:10:17.076+0000 7f6ff0ff9700 1 -- 172.21.15.162:0/12011969 <== mon.0 v2:172.21.15.162:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f6fe400f040 con 0x7f6ff40fc6d0 2024-04-27T04:10:17.078 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:10:17.076+0000 7f6ff0ff9700 1 -- 172.21.15.162:0/12011969 <== mon.0 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6fe4009640 con 0x7f6ff40fc6d0 2024-04-27T04:10:17.078 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:10:17.076+0000 7f6ffa958700 1 -- 172.21.15.162:0/12011969 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6ff4111590 con 0x7f6ff40fc6d0 2024-04-27T04:10:17.078 INFO:tasks.workunit.client.0.smithi162.stdout:0/808: mkdir d5/dc/d1e/d41/d79/d109 0 2024-04-27T04:10:17.079 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:10:17.077+0000 7f6ffa958700 1 -- 172.21.15.162:0/12011969 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6ff4100060 con 0x7f6ff40fc6d0 2024-04-27T04:10:17.079 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:10:17.078+0000 7f6ff0ff9700 1 -- 172.21.15.162:0/12011969 <== mon.0 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 24) v1 ==== 89652+0+0 (secure 0 0 0) 0x7f6fe40097a0 con 0x7f6ff40fc6d0 2024-04-27T04:10:17.079 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:10:17.079+0000 7f6ff0ff9700 1 --2- 172.21.15.162:0/12011969 >> [v2:172.21.15.181:6828/4165336392,v1:172.21.15.181:6829/4165336392] conn(0x7f6fdc06bd40 0x7f6fdc06e1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:10:17.079 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:10:17.079+0000 7f6ff0ff9700 1 -- 172.21.15.162:0/12011969 <== mon.0 v2:172.21.15.162:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5808+0+0 (secure 0 0 0) 0x7f6fe408a800 con 0x7f6ff40fc6d0 2024-04-27T04:10:17.079 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:10:17.079+0000 7f6ff3fff700 1 --2- 172.21.15.162:0/12011969 >> [v2:172.21.15.181:6828/4165336392,v1:172.21.15.181:6829/4165336392] conn(0x7f6fdc06bd40 0x7f6fdc06e1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:10:17.080 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:10:17.080+0000 7f6ff3fff700 1 --2- 172.21.15.162:0/12011969 >> [v2:172.21.15.181:6828/4165336392,v1:172.21.15.181:6829/4165336392] conn(0x7f6fdc06bd40 0x7f6fdc06e1f0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f6fe8009fd0 tx=0x7f6fe8009e30 comp rx=0 tx=0).ready entity=mgr.24367 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:10:17.080 INFO:tasks.workunit.client.0.smithi162.stdout:8/761: unlink d0/d10/d2b/d2d/d2f/d39/d7c/dd0/ff0 0 2024-04-27T04:10:17.082 INFO:tasks.workunit.client.0.smithi162.stdout:1/670: unlink d0/d7/f1f 0 2024-04-27T04:10:17.084 INFO:tasks.workunit.client.0.smithi162.stdout:2/688: link d0/d4/d2a/l67 d0/d4/d17/d4f/d7f/ld9 0 2024-04-27T04:10:17.085 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:10:17.085+0000 7f6ff0ff9700 1 -- 172.21.15.162:0/12011969 <== mon.0 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186652 (secure 0 0 0) 0x7f6fe40578f0 con 0x7f6ff40fc6d0 2024-04-27T04:10:17.086 INFO:tasks.workunit.client.0.smithi162.stdout:9/692: dread d3/d23/d1e/f2b [0,4194304] 0 2024-04-27T04:10:17.088 INFO:tasks.workunit.client.0.smithi162.stdout:9/693: chown d3/d13/d2d/d4f 1 1 2024-04-27T04:10:17.089 INFO:tasks.workunit.client.0.smithi162.stdout:6/679: getdents d6/d1a/d34/d7f 0 2024-04-27T04:10:17.092 INFO:tasks.workunit.client.0.smithi162.stdout:8/762: rename d0/de/l13 to d0/d10/d2b/d2e/de7/lfb 0 2024-04-27T04:10:17.095 INFO:tasks.workunit.client.0.smithi162.stdout:1/671: fdatasync d0/d11/f42 0 2024-04-27T04:10:17.096 INFO:tasks.workunit.client.0.smithi162.stdout:8/763: dread - d0/de/d6e/d71/d86/fc0 zero size 2024-04-27T04:10:17.097 INFO:tasks.workunit.client.0.smithi162.stdout:9/694: dwrite d3/d13/d15/d7e/fc0 [0,4194304] 0 2024-04-27T04:10:17.097 INFO:tasks.workunit.client.0.smithi162.stdout:2/689: creat d0/d4/d17/d88/dc9/fda x:0 0 0 2024-04-27T04:10:17.098 INFO:tasks.workunit.client.1.smithi181.stdout:7/869: rename d1/d22/d15/d1f/ff7 to d1/df/d1a/de8/f106 0 2024-04-27T04:10:17.099 INFO:tasks.workunit.client.1.smithi181.stdout:0/992: symlink dc/d1f/d3b/d4a/d7a/d146/d11e/l158 0 2024-04-27T04:10:17.102 INFO:tasks.workunit.client.0.smithi162.stdout:8/764: dread - d0/de/d20/d6c/fcb zero size 2024-04-27T04:10:17.104 INFO:tasks.workunit.client.1.smithi181.stdout:4/803: dwrite d0/d15/d25/d2a/d36/db8/ffc [0,4194304] 0 2024-04-27T04:10:17.104 INFO:tasks.workunit.client.0.smithi162.stdout:1/672: creat d0/d7/d3c/d54/fdf x:0 0 0 2024-04-27T04:10:17.104 INFO:tasks.workunit.client.0.smithi162.stdout:3/687: dread d1/d18/d5d/d5f/f77 [0,4194304] 0 2024-04-27T04:10:17.106 INFO:tasks.workunit.client.1.smithi181.stdout:0/993: write dc/d1f/d3b/d4a/d95/d26/d41/d12f/f135 [31051,73278] 0 2024-04-27T04:10:17.106 INFO:tasks.workunit.client.1.smithi181.stdout:8/880: dwrite d1/f82 [0,4194304] 0 2024-04-27T04:10:17.107 INFO:tasks.workunit.client.0.smithi162.stdout:6/680: mknod d6/d1a/d34/d3a/d68/cd6 0 2024-04-27T04:10:17.107 INFO:tasks.workunit.client.0.smithi162.stdout:0/809: write d5/f3b [2145621,100749] 0 2024-04-27T04:10:17.108 INFO:tasks.workunit.client.1.smithi181.stdout:2/920: mkdir d4/d9/d16/d82/d7d/dbe/df0/d12b 0 2024-04-27T04:10:17.113 INFO:tasks.workunit.client.1.smithi181.stdout:3/838: mkdir d0/d6c/d52/d44/d74/d109 0 2024-04-27T04:10:17.116 INFO:tasks.workunit.client.1.smithi181.stdout:2/921: dwrite d4/d9/d35/d42/d86/f8a [0,4194304] 0 2024-04-27T04:10:17.116 INFO:tasks.workunit.client.0.smithi162.stdout:7/739: dwrite d3/d39/d57/fa9 [0,4194304] 0 2024-04-27T04:10:17.116 INFO:tasks.workunit.client.0.smithi162.stdout:2/690: creat d0/d4/d2a/d3b/d49/d5a/d7e/fdb x:0 0 0 2024-04-27T04:10:17.119 INFO:tasks.workunit.client.0.smithi162.stdout:8/765: unlink d0/d10/d25/d65/l94 0 2024-04-27T04:10:17.122 INFO:tasks.workunit.client.1.smithi181.stdout:7/870: unlink d1/d2/d35/d67/f69 0 2024-04-27T04:10:17.122 INFO:tasks.workunit.client.0.smithi162.stdout:4/657: sync 2024-04-27T04:10:17.122 INFO:tasks.workunit.client.0.smithi162.stdout:0/810: dread d5/dc/d1e/d20/dba/df9/d8b/fd6 [0,4194304] 0 2024-04-27T04:10:17.123 INFO:tasks.workunit.client.1.smithi181.stdout:7/871: stat d1/dc3/fcf 0 2024-04-27T04:10:17.128 INFO:tasks.workunit.client.1.smithi181.stdout:0/994: symlink dc/d1f/d3b/d4a/def/l159 0 2024-04-27T04:10:17.130 INFO:tasks.workunit.client.0.smithi162.stdout:9/695: dwrite d3/d11/d45/d4e/fcb [0,4194304] 0 2024-04-27T04:10:17.130 INFO:tasks.workunit.client.0.smithi162.stdout:5/722: sync 2024-04-27T04:10:17.130 INFO:tasks.workunit.client.0.smithi162.stdout:2/691: chown d0/d4/d17/d4f/d7f/fb8 402175 1 2024-04-27T04:10:17.131 INFO:tasks.workunit.client.0.smithi162.stdout:4/658: dread - d2/d25/d1c/d39/d3b/d6e/fc1 zero size 2024-04-27T04:10:17.134 INFO:tasks.workunit.client.0.smithi162.stdout:9/696: dread - d3/d13/d15/d50/faa zero size 2024-04-27T04:10:17.134 INFO:tasks.workunit.client.0.smithi162.stdout:0/811: chown d5/dc/d1e/d20/dba/df9/d98/cbd 502068411 1 2024-04-27T04:10:17.135 INFO:tasks.workunit.client.1.smithi181.stdout:3/839: dread - d0/d6c/d66/fbd zero size 2024-04-27T04:10:17.137 INFO:tasks.workunit.client.0.smithi162.stdout:3/688: fsync d1/d18/d23/f97 0 2024-04-27T04:10:17.138 INFO:tasks.workunit.client.0.smithi162.stdout:6/681: mkdir d6/d1a/d34/d3a/dbf/dd7 0 2024-04-27T04:10:17.139 INFO:tasks.workunit.client.1.smithi181.stdout:7/872: mkdir d1/d2/d1c/d37/d107 0 2024-04-27T04:10:17.140 INFO:tasks.workunit.client.0.smithi162.stdout:7/740: dwrite d3/d5/d5b/d83/de/d2b/d81/d96/dc8/fd4 [0,4194304] 0 2024-04-27T04:10:17.140 INFO:tasks.workunit.client.1.smithi181.stdout:7/873: chown d1/d22/d15/dc6 977 1 2024-04-27T04:10:17.142 INFO:tasks.workunit.client.1.smithi181.stdout:2/922: creat d4/d9/d4f/d5f/d125/f12c x:0 0 0 2024-04-27T04:10:17.143 INFO:tasks.workunit.client.0.smithi162.stdout:1/673: dread d0/d11/d44/f9a [0,4194304] 0 2024-04-27T04:10:17.143 INFO:tasks.workunit.client.1.smithi181.stdout:2/923: read d4/d81/d102/d105/f10a [913432,102943] 0 2024-04-27T04:10:17.145 INFO:tasks.workunit.client.1.smithi181.stdout:3/840: creat d0/d6c/d52/d44/dd5/d58/df0/f10a x:0 0 0 2024-04-27T04:10:17.147 INFO:tasks.workunit.client.1.smithi181.stdout:0/995: mknod dc/d93/d12e/c15a 0 2024-04-27T04:10:17.148 INFO:tasks.workunit.client.1.smithi181.stdout:0/996: stat dc/l17 0 2024-04-27T04:10:17.150 INFO:tasks.workunit.client.1.smithi181.stdout:8/881: rename d1/d6/d2f/d11e/cca to d1/d6/d1a/c138 0 2024-04-27T04:10:17.152 INFO:tasks.workunit.client.1.smithi181.stdout:7/874: creat d1/d2/d1c/d2d/d61/d80/f108 x:0 0 0 2024-04-27T04:10:17.153 INFO:tasks.workunit.client.0.smithi162.stdout:5/723: mkdir d8/d1a/d23/d40/de4 0 2024-04-27T04:10:17.155 INFO:tasks.workunit.client.1.smithi181.stdout:2/924: truncate d4/f59 1392420 0 2024-04-27T04:10:17.155 INFO:tasks.workunit.client.0.smithi162.stdout:0/812: mkdir d5/dc/d1e/d20/dec/df1/d10a 0 2024-04-27T04:10:17.157 INFO:tasks.workunit.client.0.smithi162.stdout:3/689: truncate d1/d18/d23/d9a/fed 172554 0 2024-04-27T04:10:17.157 INFO:tasks.workunit.client.0.smithi162.stdout:7/741: creat d3/d4f/ff5 x:0 0 0 2024-04-27T04:10:17.157 INFO:tasks.workunit.client.0.smithi162.stdout:2/692: dwrite d0/d4/d17/d2b/fa9 [0,4194304] 0 2024-04-27T04:10:17.159 INFO:tasks.workunit.client.1.smithi181.stdout:2/925: dread d4/f50 [4194304,4194304] 0 2024-04-27T04:10:17.160 INFO:tasks.workunit.client.1.smithi181.stdout:2/926: chown d4/d9/d16/f26 780 1 2024-04-27T04:10:17.161 INFO:tasks.workunit.client.1.smithi181.stdout:7/875: symlink d1/d2/d1c/d8f/l109 0 2024-04-27T04:10:17.161 INFO:tasks.workunit.client.1.smithi181.stdout:2/927: write d4/f8 [746224,12551] 0 2024-04-27T04:10:17.162 INFO:tasks.workunit.client.0.smithi162.stdout:4/659: dwrite d2/d25/d1c/d24/fcf [0,4194304] 0 2024-04-27T04:10:17.165 INFO:tasks.workunit.client.0.smithi162.stdout:6/682: mkdir d6/d1a/d34/d7f/dd8 0 2024-04-27T04:10:17.166 INFO:tasks.workunit.client.1.smithi181.stdout:0/997: mknod dc/d1f/d3b/d4a/d95/d26/d131/d106/c15b 0 2024-04-27T04:10:17.167 INFO:tasks.workunit.client.1.smithi181.stdout:0/998: chown dc/d93/d12e 37635126 1 2024-04-27T04:10:17.168 INFO:tasks.workunit.client.0.smithi162.stdout:1/674: stat d0/d11/d3e/d59/d7f/f83 0 2024-04-27T04:10:17.168 INFO:tasks.workunit.client.0.smithi162.stdout:6/683: dread - d6/d1a/d34/fca zero size 2024-04-27T04:10:17.169 INFO:tasks.workunit.client.1.smithi181.stdout:0/999: chown dc/d1f/d3b/d4a/d95/d26/d41/d6f/d7d/f9b 14 1 2024-04-27T04:10:17.169 INFO:tasks.workunit.client.0.smithi162.stdout:6/684: fdatasync d6/d1a/d34/d50/f99 0 2024-04-27T04:10:17.179 INFO:tasks.workunit.client.1.smithi181.stdout:9/792: sync 2024-04-27T04:10:17.180 INFO:tasks.workunit.client.1.smithi181.stdout:7/876: readlink d1/df/l6e 0 2024-04-27T04:10:17.186 INFO:tasks.workunit.client.1.smithi181.stdout:2/928: creat d4/d9/d35/d42/d86/f12d x:0 0 0 2024-04-27T04:10:17.188 INFO:tasks.workunit.client.0.smithi162.stdout:8/766: dread d0/de/d20/f50 [0,4194304] 0 2024-04-27T04:10:17.188 INFO:tasks.workunit.client.0.smithi162.stdout:8/767: dread - d0/de/fe6 zero size 2024-04-27T04:10:17.189 INFO:tasks.workunit.client.1.smithi181.stdout:2/929: mkdir d4/d9/d16/d82/d7d/dbe/d10b/d12e 0 2024-04-27T04:10:17.190 INFO:tasks.workunit.client.0.smithi162.stdout:3/690: creat d1/d18/d23/d34/daa/de6/ff0 x:0 0 0 2024-04-27T04:10:17.191 INFO:tasks.workunit.client.1.smithi181.stdout:9/793: fdatasync d0/d44/d4b/d43/d88/da1/dfc/fff 0 2024-04-27T04:10:17.195 INFO:tasks.workunit.client.1.smithi181.stdout:9/794: creat d0/d44/d4b/d43/d88/da1/dfc/dbd/dd1/dde/f110 x:0 0 0 2024-04-27T04:10:17.198 INFO:tasks.workunit.client.0.smithi162.stdout:7/742: symlink d3/d5/d5b/d83/de/d2b/d81/daa/dbd/db6/dec/lf6 0 2024-04-27T04:10:17.200 INFO:tasks.workunit.client.0.smithi162.stdout:5/724: creat d8/d9/d5b/fe5 x:0 0 0 2024-04-27T04:10:17.203 INFO:tasks.workunit.client.0.smithi162.stdout:3/691: mknod d1/d18/d5d/d5f/d66/cf1 0 2024-04-27T04:10:17.210 INFO:tasks.workunit.client.1.smithi181.stdout:2/930: link d4/d9/d35/c74 d4/d9/d16/d82/d7d/db7/dcb/c12f 0 2024-04-27T04:10:17.213 INFO:tasks.workunit.client.0.smithi162.stdout:0/813: creat d5/dc/d1e/f10b x:0 0 0 2024-04-27T04:10:17.214 INFO:tasks.workunit.client.1.smithi181.stdout:7/877: dread d1/d22/f95 [0,4194304] 0 2024-04-27T04:10:17.214 INFO:tasks.workunit.client.0.smithi162.stdout:6/685: dread d6/db/f48 [0,4194304] 0 2024-04-27T04:10:17.215 INFO:tasks.workunit.client.0.smithi162.stdout:6/686: chown d6/d1a/f2b 64 1 2024-04-27T04:10:17.216 INFO:tasks.workunit.client.1.smithi181.stdout:2/931: mknod d4/d9/d16/d82/c130 0 2024-04-27T04:10:17.219 INFO:tasks.workunit.client.1.smithi181.stdout:7/878: creat d1/d2/d35/d67/dbf/d9a/f10a x:0 0 0 2024-04-27T04:10:17.221 INFO:tasks.workunit.client.1.smithi181.stdout:7/879: write d1/d22/d15/d29/dbe/fad [1647327,67249] 0 2024-04-27T04:10:17.224 INFO:tasks.workunit.client.1.smithi181.stdout:7/880: write d1/d2/d1c/d2d/d61/d80/f108 [523980,114245] 0 2024-04-27T04:10:17.224 INFO:tasks.workunit.client.1.smithi181.stdout:8/882: write d1/d8c/fef [133590,29976] 0 2024-04-27T04:10:17.225 INFO:tasks.workunit.client.0.smithi162.stdout:9/697: write d3/d13/d2d/d30/f38 [597006,39595] 0 2024-04-27T04:10:17.228 INFO:tasks.workunit.client.1.smithi181.stdout:3/841: dwrite d0/d6c/d52/dbb/d98/fb5 [0,4194304] 0 2024-04-27T04:10:17.228 INFO:tasks.workunit.client.0.smithi162.stdout:9/698: read d3/d13/d2d/d3b/d69/f7d [643859,126243] 0 2024-04-27T04:10:17.231 INFO:tasks.workunit.client.0.smithi162.stdout:2/693: link d0/d4/d2a/l67 d0/d4/d17/d4f/ldc 0 2024-04-27T04:10:17.232 INFO:tasks.workunit.client.0.smithi162.stdout:3/692: dread d1/d18/d23/d58/d63/f86 [0,4194304] 0 2024-04-27T04:10:17.233 INFO:tasks.workunit.client.1.smithi181.stdout:7/881: dwrite d1/d2/d35/ffb [0,4194304] 0 2024-04-27T04:10:17.235 INFO:tasks.workunit.client.1.smithi181.stdout:8/883: fsync d1/d6/d59/df1/f115 0 2024-04-27T04:10:17.236 INFO:tasks.workunit.client.0.smithi162.stdout:4/660: getdents d2/d25/d1c/d39/d90 0 2024-04-27T04:10:17.236 INFO:tasks.workunit.client.0.smithi162.stdout:0/814: chown d5/dc/d1e/d46/d48/d9e/dce/ffd 1 1 2024-04-27T04:10:17.241 INFO:tasks.workunit.client.0.smithi162.stdout:6/687: mkdir d6/d1a/d34/d7f/dd9 0 2024-04-27T04:10:17.245 INFO:tasks.workunit.client.1.smithi181.stdout:7/882: unlink d1/d57/f77 0 2024-04-27T04:10:17.246 INFO:tasks.workunit.client.0.smithi162.stdout:3/693: dwrite d1/d18/d23/d2f/fec [0,4194304] 0 2024-04-27T04:10:17.247 INFO:tasks.workunit.client.0.smithi162.stdout:1/675: write d0/d11/d50/fbd [2063315,107792] 0 2024-04-27T04:10:17.250 INFO:tasks.workunit.client.1.smithi181.stdout:9/795: write d0/d44/d4f/f6b [1019474,34841] 0 2024-04-27T04:10:17.256 INFO:tasks.workunit.client.0.smithi162.stdout:9/699: fdatasync d3/db9/dcc/fcd 0 2024-04-27T04:10:17.259 INFO:tasks.workunit.client.0.smithi162.stdout:2/694: rmdir d0/d4/d2a/d3b 39 2024-04-27T04:10:17.261 INFO:tasks.workunit.client.0.smithi162.stdout:2/695: dread - d0/d4/d17/d2b/d64/d89/fd4 zero size 2024-04-27T04:10:17.261 INFO:tasks.workunit.client.0.smithi162.stdout:0/815: read - d5/d26/d69/f8f zero size 2024-04-27T04:10:17.265 INFO:tasks.workunit.client.0.smithi162.stdout:4/661: rename d2/l2b to d2/d25/d1c/d39/d70/d91/ld1 0 2024-04-27T04:10:17.265 INFO:tasks.workunit.client.1.smithi181.stdout:2/932: dwrite d4/d9/d4f/d79/d98/fe8 [0,4194304] 0 2024-04-27T04:10:17.267 INFO:tasks.workunit.client.0.smithi162.stdout:6/688: creat d6/d1a/d34/d3a/d68/d6c/fda x:0 0 0 2024-04-27T04:10:17.267 INFO:tasks.workunit.client.0.smithi162.stdout:6/689: readlink d6/d1a/d34/d43/la2 0 2024-04-27T04:10:17.274 INFO:tasks.workunit.client.1.smithi181.stdout:2/933: dwrite d4/d81/f116 [0,4194304] 0 2024-04-27T04:10:17.275 INFO:tasks.workunit.client.1.smithi181.stdout:3/842: dwrite d0/f3c [0,4194304] 0 2024-04-27T04:10:17.280 INFO:tasks.workunit.client.0.smithi162.stdout:8/768: getdents d0/d10/d44/db2 0 2024-04-27T04:10:17.288 INFO:tasks.workunit.client.0.smithi162.stdout:2/696: mkdir d0/d20/d51/ddd 0 2024-04-27T04:10:17.291 INFO:tasks.workunit.client.0.smithi162.stdout:6/690: truncate d6/db/dc0/d74/f81 7309 0 2024-04-27T04:10:17.293 INFO:tasks.workunit.client.0.smithi162.stdout:5/725: getdents d8/d9/d55 0 2024-04-27T04:10:17.295 INFO:tasks.workunit.client.0.smithi162.stdout:8/769: symlink d0/de/d6e/d71/lfc 0 2024-04-27T04:10:17.296 INFO:tasks.workunit.client.0.smithi162.stdout:3/694: mknod d1/d65/dee/cf2 0 2024-04-27T04:10:17.297 INFO:tasks.workunit.client.0.smithi162.stdout:0/816: dwrite d5/dc/d1e/f3e [0,4194304] 0 2024-04-27T04:10:17.297 INFO:tasks.workunit.client.0.smithi162.stdout:9/700: rmdir d3/d23/d5d 39 2024-04-27T04:10:17.299 INFO:tasks.workunit.client.0.smithi162.stdout:3/695: readlink d1/d18/d23/d34/l4e 0 2024-04-27T04:10:17.300 INFO:tasks.workunit.client.0.smithi162.stdout:0/817: write d5/dc/f17 [5257825,88821] 0 2024-04-27T04:10:17.300 INFO:tasks.workunit.client.0.smithi162.stdout:2/697: mkdir d0/d4/d2a/da1/dde 0 2024-04-27T04:10:17.301 INFO:tasks.workunit.client.0.smithi162.stdout:4/662: mkdir d2/d25/d1c/d24/dd2 0 2024-04-27T04:10:17.302 INFO:tasks.workunit.client.0.smithi162.stdout:8/770: read d0/de/f7b [827269,46561] 0 2024-04-27T04:10:17.302 INFO:tasks.workunit.client.0.smithi162.stdout:7/743: getdents d3/d5/d5b/d60 0 2024-04-27T04:10:17.305 INFO:tasks.workunit.client.0.smithi162.stdout:2/698: truncate d0/d4/d2a/d41/d77/f7c 4197558 0 2024-04-27T04:10:17.306 INFO:tasks.workunit.client.0.smithi162.stdout:5/726: unlink d8/d1a/d23/f28 0 2024-04-27T04:10:17.306 INFO:tasks.workunit.client.0.smithi162.stdout:5/727: stat d8/de3 0 2024-04-27T04:10:17.310 INFO:tasks.workunit.client.0.smithi162.stdout:2/699: write d0/d4/d17/d2b/d64/d89/fd4 [274179,39959] 0 2024-04-27T04:10:17.312 INFO:tasks.workunit.client.1.smithi181.stdout:6/746: sync 2024-04-27T04:10:17.313 INFO:tasks.workunit.client.0.smithi162.stdout:0/818: creat d5/dc/d1e/d46/d48/d9e/f10c x:0 0 0 2024-04-27T04:10:17.314 INFO:tasks.workunit.client.0.smithi162.stdout:8/771: dwrite d0/de/d20/d7e/fd1 [0,4194304] 0 2024-04-27T04:10:17.314 INFO:tasks.workunit.client.0.smithi162.stdout:5/728: dread d8/d9/d55/f84 [0,4194304] 0 2024-04-27T04:10:17.316 INFO:tasks.workunit.client.0.smithi162.stdout:0/819: chown d5/dc/l59 206579496 1 2024-04-27T04:10:17.317 INFO:tasks.workunit.client.0.smithi162.stdout:5/729: chown d8/d1a/d23/dbc/d78/cce 1133048850 1 2024-04-27T04:10:17.319 INFO:tasks.workunit.client.0.smithi162.stdout:7/744: creat d3/d5/d5b/d83/de/d2b/da7/ff7 x:0 0 0 2024-04-27T04:10:17.323 INFO:tasks.workunit.client.0.smithi162.stdout:5/730: dwrite d8/d2c/fe2 [0,4194304] 0 2024-04-27T04:10:17.323 INFO:tasks.workunit.client.0.smithi162.stdout:9/701: mknod d3/d13/dd2/cd5 0 2024-04-27T04:10:17.329 INFO:tasks.workunit.client.0.smithi162.stdout:6/691: creat d6/d1a/d34/d3a/d68/fdb x:0 0 0 2024-04-27T04:10:17.330 INFO:tasks.workunit.client.0.smithi162.stdout:4/663: mknod d2/d54/d7a/d95/cd3 0 2024-04-27T04:10:17.331 INFO:tasks.workunit.client.0.smithi162.stdout:1/676: creat d0/d7/d3c/fe0 x:0 0 0 2024-04-27T04:10:17.331 INFO:tasks.workunit.client.0.smithi162.stdout:0/820: truncate d5/dc/f22 3079042 0 2024-04-27T04:10:17.336 INFO:tasks.workunit.client.0.smithi162.stdout:7/745: write d3/d5/d3c/d4d/f89 [775380,25304] 0 2024-04-27T04:10:17.338 INFO:tasks.workunit.client.1.smithi181.stdout:3/843: symlink d0/d6c/d52/d44/dd5/d58/l10b 0 2024-04-27T04:10:17.340 INFO:tasks.workunit.client.0.smithi162.stdout:5/731: rmdir d8/d1a/d7d 39 2024-04-27T04:10:17.342 INFO:tasks.workunit.client.0.smithi162.stdout:4/664: dwrite d2/d25/d1c/d39/d90/fbb [0,4194304] 0 2024-04-27T04:10:17.342 INFO:tasks.workunit.client.1.smithi181.stdout:6/747: creat d7/dc/d1c/d88/db3/ffa x:0 0 0 2024-04-27T04:10:17.343 INFO:tasks.workunit.client.0.smithi162.stdout:9/702: mkdir d3/d13/d2d/d3b/d7a/dd6 0 2024-04-27T04:10:17.345 INFO:tasks.workunit.client.0.smithi162.stdout:2/700: mknod d0/d20/d51/ddd/cdf 0 2024-04-27T04:10:17.345 INFO:tasks.workunit.client.1.smithi181.stdout:7/883: mknod d1/d22/d15/d29/dbe/c10b 0 2024-04-27T04:10:17.350 INFO:tasks.workunit.client.0.smithi162.stdout:6/692: mknod d6/d1a/d34/d7b/d7e/cdc 0 2024-04-27T04:10:17.351 INFO:tasks.workunit.client.1.smithi181.stdout:3/844: symlink d0/d6c/d52/d44/dd5/d58/de4/dc5/dcf/df2/l10c 0 2024-04-27T04:10:17.351 INFO:tasks.workunit.client.0.smithi162.stdout:5/732: dwrite d8/d1a/d29/d4a/d5c/f62 [0,4194304] 0 2024-04-27T04:10:17.352 INFO:tasks.workunit.client.1.smithi181.stdout:3/845: read d0/d6c/d52/d44/dd5/d58/d64/dc6/fe6 [3553888,66540] 0 2024-04-27T04:10:17.355 INFO:tasks.workunit.client.0.smithi162.stdout:5/733: chown d8/d9/fc6 45 1 2024-04-27T04:10:17.356 INFO:tasks.workunit.client.0.smithi162.stdout:8/772: truncate d0/d10/d2b/d2e/dca/fa0 3734521 0 2024-04-27T04:10:17.356 INFO:tasks.workunit.client.1.smithi181.stdout:3/846: readlink d0/d6c/d52/l39 0 2024-04-27T04:10:17.356 INFO:tasks.workunit.client.1.smithi181.stdout:6/748: fdatasync d7/f43 0 2024-04-27T04:10:17.357 INFO:tasks.workunit.client.0.smithi162.stdout:0/821: stat d5/dc/d1e/d20/dba/df9/c70 0 2024-04-27T04:10:17.358 INFO:tasks.workunit.client.0.smithi162.stdout:1/677: symlink d0/d7/d3c/d77/dd1/le1 0 2024-04-27T04:10:17.362 INFO:tasks.workunit.client.1.smithi181.stdout:3/847: dwrite d0/d6c/d52/dbb/dca/f107 [0,4194304] 0 2024-04-27T04:10:17.366 INFO:tasks.workunit.client.0.smithi162.stdout:6/693: dwrite d6/d1a/d34/d3a/d68/fdb [0,4194304] 0 2024-04-27T04:10:17.367 INFO:tasks.workunit.client.1.smithi181.stdout:6/749: write d7/f9d [1077348,1273] 0 2024-04-27T04:10:17.368 INFO:tasks.workunit.client.0.smithi162.stdout:7/746: fsync d3/d5/d5b/d83/de/d2b/d81/fad 0 2024-04-27T04:10:17.368 INFO:tasks.workunit.client.1.smithi181.stdout:7/884: creat d1/d2/d1c/d99/dfd/f10c x:0 0 0 2024-04-27T04:10:17.370 INFO:tasks.workunit.client.0.smithi162.stdout:5/734: dwrite d8/d1a/fd0 [0,4194304] 0 2024-04-27T04:10:17.372 INFO:tasks.workunit.client.1.smithi181.stdout:9/796: getdents d0/d44/d4b/d43/d88/da1/dfc/d9a 0 2024-04-27T04:10:17.382 INFO:tasks.workunit.client.1.smithi181.stdout:6/750: symlink d7/dc/d1c/d34/d3f/d7e/lfb 0 2024-04-27T04:10:17.384 INFO:tasks.workunit.client.1.smithi181.stdout:6/751: fdatasync d7/dc/d1c/d34/d3f/d7e/d8a/f7a 0 2024-04-27T04:10:17.384 INFO:tasks.workunit.client.1.smithi181.stdout:7/885: mkdir d1/d2/d1c/d2d/d61/d10d 0 2024-04-27T04:10:17.385 INFO:tasks.workunit.client.1.smithi181.stdout:7/886: dread - d1/d22/d15/dc6/dd8/fe4 zero size 2024-04-27T04:10:17.387 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:10:16 smithi162 ceph-mon[26841]: pgmap v8: 65 pgs: 65 active+clean; 2.8 GiB data, 9.8 GiB used, 527 GiB / 536 GiB avail; 50 MiB/s rd, 123 MiB/s wr, 279 op/s 2024-04-27T04:10:17.389 INFO:tasks.workunit.client.0.smithi162.stdout:7/747: symlink d3/d5/d5b/d83/de/lf8 0 2024-04-27T04:10:17.391 INFO:tasks.workunit.client.1.smithi181.stdout:6/752: dread d7/dc/dc3/f70 [0,4194304] 0 2024-04-27T04:10:17.392 INFO:tasks.workunit.client.0.smithi162.stdout:4/665: mkdir d2/da/dd4 0 2024-04-27T04:10:17.393 INFO:tasks.workunit.client.0.smithi162.stdout:6/694: chown d6/d1a/d34/d3a/d68/d6c/f53 1658 1 2024-04-27T04:10:17.395 INFO:tasks.workunit.client.0.smithi162.stdout:5/735: creat d8/d9/db3/fe6 x:0 0 0 2024-04-27T04:10:17.396 INFO:tasks.workunit.client.0.smithi162.stdout:2/701: symlink d0/d4/d2a/d41/d77/d73/dc3/d7d/le0 0 2024-04-27T04:10:17.397 INFO:tasks.workunit.client.0.smithi162.stdout:3/696: dwrite d1/d18/f1e [0,4194304] 0 2024-04-27T04:10:17.398 INFO:tasks.workunit.client.1.smithi181.stdout:9/797: unlink d0/l20 0 2024-04-27T04:10:17.400 INFO:tasks.workunit.client.1.smithi181.stdout:8/884: dwrite d1/d6/da6/ff8 [0,4194304] 0 2024-04-27T04:10:17.403 INFO:tasks.workunit.client.1.smithi181.stdout:8/885: stat d1/d6/d2f/d30/d9d/dea/db3/ddf/l112 0 2024-04-27T04:10:17.408 INFO:tasks.workunit.client.1.smithi181.stdout:7/887: mkdir d1/df/d1a/d6a/d10e 0 2024-04-27T04:10:17.411 INFO:tasks.workunit.client.0.smithi162.stdout:2/702: dread d0/d4/d17/d2b/d64/d89/fd4 [0,4194304] 0 2024-04-27T04:10:17.412 INFO:tasks.workunit.client.0.smithi162.stdout:7/748: read d3/d5/d9/f26 [1389323,57409] 0 2024-04-27T04:10:17.413 INFO:tasks.workunit.client.0.smithi162.stdout:8/773: fdatasync d0/d10/d2b/d2d/d2f/f54 0 2024-04-27T04:10:17.413 INFO:tasks.workunit.client.0.smithi162.stdout:6/695: mknod d6/d1a/d34/d7f/d80/db3/cdd 0 2024-04-27T04:10:17.416 INFO:tasks.workunit.client.1.smithi181.stdout:8/886: mkdir d1/d6/d2f/d11e/d139 0 2024-04-27T04:10:17.422 INFO:tasks.workunit.client.1.smithi181.stdout:9/798: mkdir d0/d44/d4b/d43/d8f/d111 0 2024-04-27T04:10:17.422 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:10:16 smithi181 ceph-mon[33774]: pgmap v8: 65 pgs: 65 active+clean; 2.8 GiB data, 9.8 GiB used, 527 GiB / 536 GiB avail; 50 MiB/s rd, 123 MiB/s wr, 279 op/s 2024-04-27T04:10:17.422 INFO:tasks.workunit.client.0.smithi162.stdout:8/774: dwrite d0/d10/d2b/d2d/fea [0,4194304] 0 2024-04-27T04:10:17.423 INFO:tasks.workunit.client.1.smithi181.stdout:8/887: fdatasync d1/d6/d2f/d11e/fb6 0 2024-04-27T04:10:17.424 INFO:tasks.workunit.client.1.smithi181.stdout:8/888: chown d1/d20/l50 135073 1 2024-04-27T04:10:17.425 INFO:tasks.workunit.client.1.smithi181.stdout:7/888: truncate d1/d2/d1c/d2d/fd1 586581 0 2024-04-27T04:10:17.444 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:10:17.445+0000 7f6ffa958700 1 -- 172.21.15.162:0/12011969 --> [v2:172.21.15.181:6828/4165336392,v1:172.21.15.181:6829/4165336392] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f6ff4111800 con 0x7f6fdc06bd40 2024-04-27T04:10:17.445 INFO:tasks.workunit.client.1.smithi181.stdout:4/804: sync 2024-04-27T04:10:17.445 INFO:tasks.workunit.client.1.smithi181.stdout:1/855: sync 2024-04-27T04:10:17.445 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:10:17.447+0000 7f6ff0ff9700 1 -- 172.21.15.162:0/12011969 <== mgr.24367 v2:172.21.15.181:6828/4165336392 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+310 (secure 0 0 0) 0x7f6ff4111800 con 0x7f6fdc06bd40 2024-04-27T04:10:17.448 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:10:17.449+0000 7f6ffa958700 1 -- 172.21.15.162:0/12011969 >> [v2:172.21.15.181:6828/4165336392,v1:172.21.15.181:6829/4165336392] conn(0x7f6fdc06bd40 msgr2=0x7f6fdc06e1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:10:17.448 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:10:17.449+0000 7f6ffa958700 1 --2- 172.21.15.162:0/12011969 >> [v2:172.21.15.181:6828/4165336392,v1:172.21.15.181:6829/4165336392] conn(0x7f6fdc06bd40 0x7f6fdc06e1f0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f6fe8009fd0 tx=0x7f6fe8009e30 comp rx=0 tx=0).stop 2024-04-27T04:10:17.448 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:10:17.449+0000 7f6ffa958700 1 -- 172.21.15.162:0/12011969 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6ff40fc6d0 msgr2=0x7f6ff410ff20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:10:17.448 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:10:17.449+0000 7f6ffa958700 1 --2- 172.21.15.162:0/12011969 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6ff40fc6d0 0x7f6ff410ff20 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f6fe4002a50 tx=0x7f6fe40046b0 comp rx=0 tx=0).stop 2024-04-27T04:10:17.448 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:10:17.449+0000 7f6ffa958700 1 -- 172.21.15.162:0/12011969 shutdown_connections 2024-04-27T04:10:17.449 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:10:17.449+0000 7f6ffa958700 1 --2- 172.21.15.162:0/12011969 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6ff40fb420 0x7f6ff410f9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:10:17.449 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:10:17.449+0000 7f6ffa958700 1 --2- 172.21.15.162:0/12011969 >> [v2:172.21.15.181:6828/4165336392,v1:172.21.15.181:6829/4165336392] conn(0x7f6fdc06bd40 0x7f6fdc06e1f0 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:10:17.449 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:10:17.449+0000 7f6ffa958700 1 --2- 172.21.15.162:0/12011969 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6ff40fc6d0 0x7f6ff410ff20 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:10:17.449 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:10:17.449+0000 7f6ffa958700 1 -- 172.21.15.162:0/12011969 >> 172.21.15.162:0/12011969 conn(0x7f6ff40f6b80 msgr2=0x7f6ff40f6fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:10:17.449 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:10:17.450+0000 7f6ffa958700 1 -- 172.21.15.162:0/12011969 shutdown_connections 2024-04-27T04:10:17.449 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:10:17.450+0000 7f6ffa958700 1 -- 172.21.15.162:0/12011969 wait complete. 2024-04-27T04:10:17.450 INFO:tasks.workunit.client.1.smithi181.stdout:9/799: symlink d0/d44/d4b/d43/d88/da1/dfc/dbd/dd1/dde/d104/l112 0 2024-04-27T04:10:17.451 INFO:tasks.workunit.client.0.smithi162.stdout:2/703: dread d0/d20/d51/fcd [0,4194304] 0 2024-04-27T04:10:17.452 INFO:tasks.workunit.client.0.smithi162.stdout:2/704: write d0/f1 [997738,36650] 0 2024-04-27T04:10:17.452 INFO:tasks.workunit.client.1.smithi181.stdout:9/800: dread - d0/d44/d4b/d43/d88/f107 zero size 2024-04-27T04:10:17.455 INFO:tasks.workunit.client.1.smithi181.stdout:9/801: stat d0/d44/d4b/d43/d88/da1/dfc/dbd/dd1/dde/d104/f2c 0 2024-04-27T04:10:17.455 INFO:tasks.workunit.client.1.smithi181.stdout:3/848: dwrite d0/de/fb3 [0,4194304] 0 2024-04-27T04:10:17.456 INFO:tasks.workunit.client.1.smithi181.stdout:9/802: chown d0/d44/d4b/d43/d8f/d111 48018549 1 2024-04-27T04:10:17.457 INFO:tasks.workunit.client.1.smithi181.stdout:4/805: symlink d0/d15/dd2/deb/db9/l108 0 2024-04-27T04:10:17.459 INFO:tasks.workunit.client.1.smithi181.stdout:4/806: write d0/d15/d25/d2a/f105 [23152,118134] 0 2024-04-27T04:10:17.459 INFO:tasks.workunit.client.1.smithi181.stdout:1/856: truncate d1/dad/db5/d6/da3/d1e/f9f 1139471 0 2024-04-27T04:10:17.460 INFO:teuthology.orchestra.run.smithi162.stdout:true 2024-04-27T04:10:17.464 INFO:tasks.workunit.client.1.smithi181.stdout:7/889: creat d1/d57/f10f x:0 0 0 2024-04-27T04:10:17.467 INFO:tasks.workunit.client.1.smithi181.stdout:9/803: fdatasync d0/d44/d4b/d43/d88/da1/dfc/dbd/dd1/dde/d104/f2c 0 2024-04-27T04:10:17.468 INFO:tasks.workunit.client.1.smithi181.stdout:9/804: chown d0/d44/f51 235206 1 2024-04-27T04:10:17.468 INFO:tasks.workunit.client.1.smithi181.stdout:9/805: read - d0/d44/f9e zero size 2024-04-27T04:10:17.469 INFO:tasks.workunit.client.1.smithi181.stdout:7/890: dread d1/d2/d35/ffb [0,4194304] 0 2024-04-27T04:10:17.470 INFO:tasks.workunit.client.1.smithi181.stdout:7/891: dread d1/d22/fc0 [0,4194304] 0 2024-04-27T04:10:17.472 INFO:tasks.workunit.client.1.smithi181.stdout:1/857: creat d1/dad/db5/d6/da3/d1e/d20/d53/d56/dd9/f117 x:0 0 0 2024-04-27T04:10:17.474 INFO:tasks.workunit.client.1.smithi181.stdout:8/889: rename d1/d6/d2f/c67 to d1/d6/d2f/d30/d66/dc0/dcd/c13a 0 2024-04-27T04:10:17.475 INFO:tasks.workunit.client.1.smithi181.stdout:8/890: write d1/d6/d2f/d30/d5c/f97 [1474954,5314] 0 2024-04-27T04:10:17.487 INFO:tasks.workunit.client.1.smithi181.stdout:1/858: fsync d1/dad/db5/d6/f67 0 2024-04-27T04:10:17.491 INFO:tasks.workunit.client.1.smithi181.stdout:3/849: rename d0/d6c/d52/d44/dd5/d58/de4/dc5/dcf/df2/l10c to d0/de/d7d/df5/l10d 0 2024-04-27T04:10:17.495 INFO:tasks.workunit.client.1.smithi181.stdout:8/891: creat d1/d6/d2f/d30/d5c/f13b x:0 0 0 2024-04-27T04:10:17.511 INFO:tasks.workunit.client.1.smithi181.stdout:7/892: dwrite d1/d2/f25 [0,4194304] 0 2024-04-27T04:10:17.561 INFO:tasks.workunit.client.0.smithi162.stdout:9/703: creat d3/d11/d45/d4e/fd7 x:0 0 0 2024-04-27T04:10:17.561 INFO:tasks.workunit.client.0.smithi162.stdout:4/666: mkdir d2/d25/d1c/d39/d3b/d6e/dd5 0 2024-04-27T04:10:17.562 INFO:tasks.workunit.client.0.smithi162.stdout:2/705: unlink d0/l4c 0 2024-04-27T04:10:17.562 INFO:tasks.workunit.client.0.smithi162.stdout:3/697: link d1/d18/d23/f2e d1/d18/d5d/ff3 0 2024-04-27T04:10:17.562 INFO:tasks.workunit.client.0.smithi162.stdout:9/704: dread d3/d13/f41 [4194304,4194304] 0 2024-04-27T04:10:17.563 INFO:tasks.workunit.client.1.smithi181.stdout:4/807: getdents d0/d15/d25/d2a/d36 0 2024-04-27T04:10:17.563 INFO:tasks.workunit.client.1.smithi181.stdout:4/808: chown d0/d15/d5c/d69/l11 58054987 1 2024-04-27T04:10:17.563 INFO:tasks.workunit.client.1.smithi181.stdout:1/859: fsync d1/dad/db5/d6/da3/d1e/d20/d53/d56/dd9/fdd 0 2024-04-27T04:10:17.563 INFO:tasks.workunit.client.1.smithi181.stdout:1/860: readlink d1/dad/db5/d6/db/l15 0 2024-04-27T04:10:17.563 INFO:tasks.workunit.client.1.smithi181.stdout:2/934: sync 2024-04-27T04:10:17.563 INFO:tasks.workunit.client.1.smithi181.stdout:9/806: rename d0/l1 to d0/d44/d4b/d43/d88/da1/dfc/l113 0 2024-04-27T04:10:17.563 INFO:tasks.workunit.client.1.smithi181.stdout:3/850: write d0/d6c/d52/dbb/d98/da7/fae [642369,99851] 0 2024-04-27T04:10:17.564 INFO:tasks.workunit.client.1.smithi181.stdout:8/892: write d1/d6/f26 [56993,58288] 0 2024-04-27T04:10:17.564 INFO:tasks.workunit.client.1.smithi181.stdout:1/861: creat d1/d2/d89/f118 x:0 0 0 2024-04-27T04:10:17.564 INFO:tasks.workunit.client.1.smithi181.stdout:1/862: write d1/dad/db5/d6/da3/f8b [4098129,123528] 0 2024-04-27T04:10:17.564 INFO:tasks.workunit.client.1.smithi181.stdout:3/851: mknod d0/d28/d86/c10e 0 2024-04-27T04:10:17.564 INFO:tasks.workunit.client.1.smithi181.stdout:8/893: chown d1/d6/d1a/l21 4558 1 2024-04-27T04:10:17.564 INFO:tasks.workunit.client.1.smithi181.stdout:1/863: creat d1/dad/db5/d6/da3/d1e/d48/db4/f119 x:0 0 0 2024-04-27T04:10:17.564 INFO:tasks.workunit.client.1.smithi181.stdout:2/935: creat d4/d9/f131 x:0 0 0 2024-04-27T04:10:17.564 INFO:tasks.workunit.client.1.smithi181.stdout:1/864: dwrite d1/dad/db5/d6/da3/d1e/d48/da6/f110 [0,4194304] 0 2024-04-27T04:10:17.564 INFO:tasks.workunit.client.1.smithi181.stdout:7/893: creat d1/d2/d1c/d37/f110 x:0 0 0 2024-04-27T04:10:17.564 INFO:tasks.workunit.client.1.smithi181.stdout:4/809: dwrite d0/d15/d25/d2b/d6d/f71 [0,4194304] 0 2024-04-27T04:10:17.564 INFO:tasks.workunit.client.1.smithi181.stdout:1/865: creat d1/dad/db5/d6/da3/d1e/d20/d53/d56/de2/f11a x:0 0 0 2024-04-27T04:10:17.565 INFO:tasks.workunit.client.1.smithi181.stdout:7/894: unlink d1/d2/d1c/d56/lbc 0 2024-04-27T04:10:17.565 INFO:tasks.workunit.client.0.smithi162.stdout:7/749: mkdir d3/d5/d5b/d83/de/d2b/df9 0 2024-04-27T04:10:17.567 INFO:tasks.workunit.client.0.smithi162.stdout:5/736: sync 2024-04-27T04:10:17.568 INFO:tasks.workunit.client.0.smithi162.stdout:0/822: sync 2024-04-27T04:10:17.568 INFO:tasks.workunit.client.0.smithi162.stdout:1/678: sync 2024-04-27T04:10:17.569 INFO:tasks.workunit.client.1.smithi181.stdout:9/807: dwrite d0/d44/d4b/d43/d88/da1/dfc/f9f [0,4194304] 0 2024-04-27T04:10:17.569 INFO:tasks.workunit.client.0.smithi162.stdout:6/696: unlink d6/d1a/d34/d7b/d7e/d97/ccb 0 2024-04-27T04:10:17.570 INFO:tasks.workunit.client.1.smithi181.stdout:9/808: readlink d0/d44/d4b/d43/d88/da1/dfc/lda 0 2024-04-27T04:10:17.572 INFO:tasks.workunit.client.1.smithi181.stdout:9/809: read d0/f26 [4720077,106954] 0 2024-04-27T04:10:17.584 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:10:17.585+0000 7f741031c700 1 -- 172.21.15.162:0/292983023 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f74081002a0 msgr2=0x7f7408100710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:10:17.584 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:10:17.585+0000 7f741031c700 1 --2- 172.21.15.162:0/292983023 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f74081002a0 0x7f7408100710 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f73f8004940 tx=0x7f73f8009e30 comp rx=0 tx=0).stop 2024-04-27T04:10:17.584 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:10:17.585+0000 7f741031c700 1 -- 172.21.15.162:0/292983023 shutdown_connections 2024-04-27T04:10:17.584 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:10:17.585+0000 7f741031c700 1 --2- 172.21.15.162:0/292983023 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f74081002a0 0x7f7408100710 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:10:17.584 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:10:17.585+0000 7f741031c700 1 --2- 172.21.15.162:0/292983023 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f74080feff0 0x7f74080ff400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:10:17.585 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:10:17.585+0000 7f741031c700 1 -- 172.21.15.162:0/292983023 >> 172.21.15.162:0/292983023 conn(0x7f74080fa8f0 msgr2=0x7f74080fccd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:10:17.585 INFO:tasks.workunit.client.0.smithi162.stdout:0/823: creat d5/dc/d1e/d20/f10d x:0 0 0 2024-04-27T04:10:17.586 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:10:17.586+0000 7f741031c700 1 -- 172.21.15.162:0/292983023 shutdown_connections 2024-04-27T04:10:17.586 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:10:17.586+0000 7f741031c700 1 -- 172.21.15.162:0/292983023 wait complete. 2024-04-27T04:10:17.586 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:10:17.587+0000 7f741031c700 1 Processor -- start 2024-04-27T04:10:17.586 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:10:17.587+0000 7f741031c700 1 -- start start 2024-04-27T04:10:17.586 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:10:17.587+0000 7f741031c700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f74080feff0 0x7f740810fb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:10:17.586 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:10:17.587+0000 7f741031c700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f74081002a0 0x7f7408110050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:10:17.586 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:10:17.587+0000 7f741031c700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7408110650 con 0x7f74080feff0 2024-04-27T04:10:17.586 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:10:17.587+0000 7f741031c700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74081107c0 con 0x7f74081002a0 2024-04-27T04:10:17.586 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:10:17.587+0000 7f740e0b8700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f74080feff0 0x7f740810fb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:10:17.586 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:10:17.587+0000 7f740e0b8700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f74080feff0 0x7f740810fb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:51812/0 (socket says 172.21.15.162:51812) 2024-04-27T04:10:17.587 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:10:17.587+0000 7f740e0b8700 1 -- 172.21.15.162:0/3679849007 learned_addr learned my addr 172.21.15.162:0/3679849007 (peer_addr_for_me v2:172.21.15.162:0/0) 2024-04-27T04:10:17.587 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:10:17.587+0000 7f740d8b7700 1 --2- 172.21.15.162:0/3679849007 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f74081002a0 0x7f7408110050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:10:17.587 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:10:17.588+0000 7f740e0b8700 1 -- 172.21.15.162:0/3679849007 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f74081002a0 msgr2=0x7f7408110050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:10:17.587 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:10:17.588+0000 7f740e0b8700 1 --2- 172.21.15.162:0/3679849007 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f74081002a0 0x7f7408110050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:10:17.587 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:10:17.588+0000 7f740e0b8700 1 -- 172.21.15.162:0/3679849007 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f73f8005040 con 0x7f74080feff0 2024-04-27T04:10:17.587 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:10:17.588+0000 7f740e0b8700 1 --2- 172.21.15.162:0/3679849007 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f74080feff0 0x7f740810fb10 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f740400eb40 tx=0x7f740400b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:10:17.587 INFO:tasks.workunit.client.0.smithi162.stdout:1/679: creat d0/d80/d89/fe2 x:0 0 0 2024-04-27T04:10:17.588 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:10:17.588+0000 7f73feffd700 1 -- 172.21.15.162:0/3679849007 <== mon.0 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7404007cc0 con 0x7f74080feff0 2024-04-27T04:10:17.588 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:10:17.588+0000 7f741031c700 1 -- 172.21.15.162:0/3679849007 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7408115100 con 0x7f74080feff0 2024-04-27T04:10:17.588 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:10:17.588+0000 7f741031c700 1 -- 172.21.15.162:0/3679849007 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7408115640 con 0x7f74080feff0 2024-04-27T04:10:17.588 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:10:17.588+0000 7f73feffd700 1 -- 172.21.15.162:0/3679849007 <== mon.0 v2:172.21.15.162:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f740400f040 con 0x7f74080feff0 2024-04-27T04:10:17.588 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:10:17.589+0000 7f73feffd700 1 -- 172.21.15.162:0/3679849007 <== mon.0 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f740400ca50 con 0x7f74080feff0 2024-04-27T04:10:17.589 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:10:17.590+0000 7f741031c700 1 -- 172.21.15.162:0/3679849007 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f73ec0052e0 con 0x7f74080feff0 2024-04-27T04:10:17.589 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:10:17.590+0000 7f73feffd700 1 -- 172.21.15.162:0/3679849007 <== mon.0 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 24) v1 ==== 89652+0+0 (secure 0 0 0) 0x7f740400cbb0 con 0x7f74080feff0 2024-04-27T04:10:17.589 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:10:17.591+0000 7f73feffd700 1 --2- 172.21.15.162:0/3679849007 >> [v2:172.21.15.181:6828/4165336392,v1:172.21.15.181:6829/4165336392] conn(0x7f73f406be00 0x7f73f406e2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:10:17.589 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:10:17.591+0000 7f73feffd700 1 -- 172.21.15.162:0/3679849007 <== mon.0 v2:172.21.15.162:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5808+0+0 (secure 0 0 0) 0x7f740408a580 con 0x7f74080feff0 2024-04-27T04:10:17.590 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:10:17.591+0000 7f740d8b7700 1 --2- 172.21.15.162:0/3679849007 >> [v2:172.21.15.181:6828/4165336392,v1:172.21.15.181:6829/4165336392] conn(0x7f73f406be00 0x7f73f406e2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:10:17.590 INFO:tasks.workunit.client.1.smithi181.stdout:9/810: rename d0/d44/d4b/d43/d88/da1/dfc/dbd/dd1/dde/d104/c65 to d0/d44/d4b/d43/d88/da1/dfc/dbd/dd1/dde/de4/c114 0 2024-04-27T04:10:17.590 INFO:tasks.workunit.client.1.smithi181.stdout:9/811: stat d0/d44/d4b/l109 0 2024-04-27T04:10:17.590 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:10:17.592+0000 7f740d8b7700 1 --2- 172.21.15.162:0/3679849007 >> [v2:172.21.15.181:6828/4165336392,v1:172.21.15.181:6829/4165336392] conn(0x7f73f406be00 0x7f73f406e2b0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f73f8013870 tx=0x7f73f8009e30 comp rx=0 tx=0).ready entity=mgr.24367 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:10:17.592 INFO:tasks.workunit.client.1.smithi181.stdout:9/812: chown d0/d44/d4b/d43/d88/da1/dfc/dbd/df3/d85/fc3 7022 1 2024-04-27T04:10:17.592 INFO:tasks.workunit.client.0.smithi162.stdout:8/775: rename d0/de/d20/d6c/fec to d0/d10/d25/d65/ffd 0 2024-04-27T04:10:17.592 INFO:tasks.workunit.client.0.smithi162.stdout:8/776: readlink d0/de/d6e/l90 0 2024-04-27T04:10:17.593 INFO:tasks.workunit.client.0.smithi162.stdout:2/706: mknod d0/d4/d17/d2b/d64/d96/ce1 0 2024-04-27T04:10:17.594 INFO:tasks.workunit.client.0.smithi162.stdout:3/698: creat d1/d18/d23/d2f/da0/ff4 x:0 0 0 2024-04-27T04:10:17.595 INFO:tasks.workunit.client.1.smithi181.stdout:2/936: truncate d4/d9/d35/d9f/dd0/f109 4184200 0 2024-04-27T04:10:17.595 INFO:tasks.workunit.client.0.smithi162.stdout:2/707: write d0/d4/d2a/d41/fa2 [385854,17563] 0 2024-04-27T04:10:17.595 INFO:tasks.workunit.client.1.smithi181.stdout:9/813: readlink d0/d44/d4b/d43/d88/da1/dfc/dbd/dd1/dde/d104/l1b 0 2024-04-27T04:10:17.596 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:10:17.598+0000 7f73feffd700 1 -- 172.21.15.162:0/3679849007 <== mon.0 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186652 (secure 0 0 0) 0x7f7404050930 con 0x7f74080feff0 2024-04-27T04:10:17.597 INFO:tasks.workunit.client.0.smithi162.stdout:2/708: stat d0/d4/d17/d2b/d64/d96/ce1 0 2024-04-27T04:10:17.597 INFO:tasks.workunit.client.0.smithi162.stdout:3/699: fdatasync d1/d18/d23/d34/daa/fce 0 2024-04-27T04:10:17.597 INFO:tasks.workunit.client.0.smithi162.stdout:9/705: fdatasync d3/d13/d2d/d30/d67/fc6 0 2024-04-27T04:10:17.598 INFO:tasks.workunit.client.0.smithi162.stdout:7/750: creat d3/d5/d5b/d83/de/d2b/df9/ffa x:0 0 0 2024-04-27T04:10:17.598 INFO:tasks.workunit.client.1.smithi181.stdout:2/937: creat d4/d81/df3/f132 x:0 0 0 2024-04-27T04:10:17.599 INFO:tasks.workunit.client.0.smithi162.stdout:7/751: fsync d3/d5/d5b/d83/de/d2b/d81/daa/dbd/dab/fed 0 2024-04-27T04:10:17.599 INFO:tasks.workunit.client.1.smithi181.stdout:9/814: chown d0/d44/d4b/d43/d88/da1/dfc/dbd/df3/c53 2337532 1 2024-04-27T04:10:17.600 INFO:tasks.workunit.client.0.smithi162.stdout:7/752: write d3/d5/d9/fe7 [530419,51555] 0 2024-04-27T04:10:17.601 INFO:tasks.workunit.client.0.smithi162.stdout:0/824: creat d5/dc/d1e/d46/d48/d9e/db8/f10e x:0 0 0 2024-04-27T04:10:17.601 INFO:tasks.workunit.client.1.smithi181.stdout:9/815: chown d0/d44/d4b/d50/c10d 80648427 1 2024-04-27T04:10:17.602 INFO:tasks.workunit.client.1.smithi181.stdout:2/938: truncate d4/d9/d4f/d79/d98/fcc 460644 0 2024-04-27T04:10:17.602 INFO:tasks.workunit.client.0.smithi162.stdout:0/825: write d5/dc/d1e/d46/f9f [564052,6488] 0 2024-04-27T04:10:17.603 INFO:tasks.workunit.client.1.smithi181.stdout:2/939: fdatasync d4/d9/d4f/d5f/d125/f12c 0 2024-04-27T04:10:17.605 INFO:tasks.workunit.client.0.smithi162.stdout:8/777: mknod d0/d10/d2b/d2d/d2f/d39/cfe 0 2024-04-27T04:10:17.606 INFO:tasks.workunit.client.1.smithi181.stdout:9/816: unlink d0/d44/d4b/d43/d88/da1/dfc/dbd/dd1/dde/f110 0 2024-04-27T04:10:17.607 INFO:tasks.workunit.client.1.smithi181.stdout:1/866: write d1/dad/db5/d6/da3/d1e/f9f [809293,58710] 0 2024-04-27T04:10:17.608 INFO:tasks.workunit.client.1.smithi181.stdout:4/810: dread d0/d15/de/d19/fab [0,4194304] 0 2024-04-27T04:10:17.611 INFO:tasks.workunit.client.0.smithi162.stdout:6/697: write d6/f30 [417513,50792] 0 2024-04-27T04:10:17.614 INFO:tasks.workunit.client.0.smithi162.stdout:4/667: dwrite d2/da/d1a/fb8 [4194304,4194304] 0 2024-04-27T04:10:17.615 INFO:tasks.workunit.client.0.smithi162.stdout:6/698: write d6/db/d32/f5b [905776,4980] 0 2024-04-27T04:10:17.618 INFO:tasks.workunit.client.0.smithi162.stdout:9/706: mkdir d3/db9/dcc/dd8 0 2024-04-27T04:10:17.622 INFO:tasks.workunit.client.0.smithi162.stdout:5/737: creat d8/d1a/d29/d4a/fe7 x:0 0 0 2024-04-27T04:10:17.628 INFO:tasks.workunit.client.1.smithi181.stdout:9/817: dread d0/f2b [0,4194304] 0 2024-04-27T04:10:17.631 INFO:tasks.workunit.client.1.smithi181.stdout:4/811: rmdir d0/d15/d25/d2a/df8 0 2024-04-27T04:10:17.633 INFO:tasks.workunit.client.0.smithi162.stdout:8/778: dread d0/d10/d25/d65/ffd [0,4194304] 0 2024-04-27T04:10:17.636 INFO:tasks.workunit.client.1.smithi181.stdout:2/940: dwrite d4/d9/d16/d82/feb [0,4194304] 0 2024-04-27T04:10:17.637 INFO:tasks.workunit.client.0.smithi162.stdout:3/700: write d1/d65/f7f [2552974,82533] 0 2024-04-27T04:10:17.645 INFO:tasks.workunit.client.1.smithi181.stdout:1/867: dwrite d1/dad/db5/d6/da3/d1e/d20/d53/f72 [0,4194304] 0 2024-04-27T04:10:17.645 INFO:tasks.workunit.client.1.smithi181.stdout:4/812: rename d0/d15/d25/d2a/d4c to d0/d15/d25/d2a/da0/dc7/d109 0 2024-04-27T04:10:17.650 INFO:tasks.workunit.client.1.smithi181.stdout:2/941: readlink d4/d9/d16/l91 0 2024-04-27T04:10:17.651 INFO:tasks.workunit.client.0.smithi162.stdout:4/668: mknod d2/d54/d7a/cd6 0 2024-04-27T04:10:17.652 INFO:tasks.workunit.client.0.smithi162.stdout:4/669: chown d2 0 1 2024-04-27T04:10:17.653 INFO:tasks.workunit.client.1.smithi181.stdout:9/818: creat d0/f115 x:0 0 0 2024-04-27T04:10:17.662 INFO:tasks.workunit.client.1.smithi181.stdout:1/868: rmdir d1/dad/db5/d6/da3/d1e/d20 39 2024-04-27T04:10:17.665 INFO:tasks.workunit.client.1.smithi181.stdout:2/942: mknod d4/d9/d16/d82/d7d/dbe/df0/c133 0 2024-04-27T04:10:17.666 INFO:tasks.workunit.client.1.smithi181.stdout:1/869: stat d1/dad/db5/d6/db/f103 0 2024-04-27T04:10:17.668 INFO:tasks.workunit.client.1.smithi181.stdout:2/943: creat d4/d9/d16/d82/d7d/dbe/d10b/f134 x:0 0 0 2024-04-27T04:10:17.668 INFO:tasks.workunit.client.1.smithi181.stdout:4/813: link d0/d15/de/l87 d0/d15/de1/l10a 0 2024-04-27T04:10:17.673 INFO:tasks.workunit.client.1.smithi181.stdout:2/944: getdents d4/d81/df3 0 2024-04-27T04:10:17.676 INFO:tasks.workunit.client.1.smithi181.stdout:2/945: truncate d4/d81/f84 297357 0 2024-04-27T04:10:17.679 INFO:tasks.workunit.client.1.smithi181.stdout:2/946: getdents d4/d9/d4f/d5f 0 2024-04-27T04:10:17.681 INFO:tasks.workunit.client.1.smithi181.stdout:4/814: dread d0/d15/d25/d2b/d6d/f74 [4194304,4194304] 0 2024-04-27T04:10:17.684 INFO:tasks.workunit.client.0.smithi162.stdout:3/701: write d1/d18/d23/d34/dad/dc5/fcb [617751,116257] 0 2024-04-27T04:10:17.685 INFO:tasks.workunit.client.0.smithi162.stdout:4/670: creat d2/d25/fd7 x:0 0 0 2024-04-27T04:10:17.686 INFO:tasks.workunit.client.0.smithi162.stdout:0/826: symlink d5/dc/d1e/d41/d79/d109/l10f 0 2024-04-27T04:10:17.686 INFO:tasks.workunit.client.1.smithi181.stdout:2/947: dwrite d4/d9/d16/d45/d9a/fe1 [0,4194304] 0 2024-04-27T04:10:17.688 INFO:tasks.workunit.client.0.smithi162.stdout:6/699: truncate d6/d1a/fa6 2874178 0 2024-04-27T04:10:17.690 INFO:tasks.workunit.client.1.smithi181.stdout:4/815: truncate d0/d15/d25/d2a/da0/fc1 1232603 0 2024-04-27T04:10:17.690 INFO:tasks.workunit.client.0.smithi162.stdout:3/702: chown d1/d18/c17 444116883 1 2024-04-27T04:10:17.691 INFO:tasks.workunit.client.1.smithi181.stdout:2/948: creat d4/d9/d16/d45/f135 x:0 0 0 2024-04-27T04:10:17.692 INFO:tasks.workunit.client.0.smithi162.stdout:1/680: dwrite d0/d7/d2a/f92 [0,4194304] 0 2024-04-27T04:10:17.700 INFO:tasks.workunit.client.0.smithi162.stdout:6/700: chown d6/d1a/d34/d7b 481 1 2024-04-27T04:10:17.700 INFO:tasks.workunit.client.1.smithi181.stdout:2/949: symlink d4/d9/d16/d82/d7d/dbe/df0/d12b/l136 0 2024-04-27T04:10:17.700 INFO:tasks.workunit.client.1.smithi181.stdout:6/753: sync 2024-04-27T04:10:17.701 INFO:tasks.workunit.client.1.smithi181.stdout:4/816: dread d0/d15/de/d19/d77/f72 [0,4194304] 0 2024-04-27T04:10:17.702 INFO:tasks.workunit.client.1.smithi181.stdout:6/754: write d7/dc/dc3/f6b [116024,37420] 0 2024-04-27T04:10:17.704 INFO:tasks.workunit.client.0.smithi162.stdout:5/738: dwrite d8/fa9 [0,4194304] 0 2024-04-27T04:10:17.705 INFO:tasks.workunit.client.1.smithi181.stdout:2/950: truncate d4/d9/d4f/d5f/fe2 860175 0 2024-04-27T04:10:17.706 INFO:tasks.workunit.client.1.smithi181.stdout:6/755: fsync d7/dc/f3a 0 2024-04-27T04:10:17.706 INFO:tasks.workunit.client.0.smithi162.stdout:2/709: dwrite d0/d4/d17/f2f [0,4194304] 0 2024-04-27T04:10:17.708 INFO:tasks.workunit.client.0.smithi162.stdout:1/681: rename d0/d11/d50/d53/d93 to d0/d11/d50/d53/d93/d96/de3 22 2024-04-27T04:10:17.708 INFO:tasks.workunit.client.1.smithi181.stdout:2/951: readlink d4/d9/d4f/l4d 0 2024-04-27T04:10:17.710 INFO:tasks.workunit.client.0.smithi162.stdout:0/827: getdents d5/dc/d1e/d56/d57 0 2024-04-27T04:10:17.710 INFO:tasks.workunit.client.0.smithi162.stdout:7/753: dwrite d3/d5/d5b/fba [0,4194304] 0 2024-04-27T04:10:17.710 INFO:tasks.workunit.client.1.smithi181.stdout:6/756: fsync d7/f59 0 2024-04-27T04:10:17.711 INFO:tasks.workunit.client.1.smithi181.stdout:2/952: truncate d4/d9/df/d14/da7/f106 696059 0 2024-04-27T04:10:17.711 INFO:tasks.workunit.client.0.smithi162.stdout:5/739: chown d8/d9/d55 584105950 1 2024-04-27T04:10:17.713 INFO:tasks.workunit.client.0.smithi162.stdout:5/740: chown d8/d1a/d23/dbc/fa8 688765637 1 2024-04-27T04:10:17.713 INFO:tasks.workunit.client.0.smithi162.stdout:3/703: link d1/d18/f11 d1/d18/d23/d34/daa/ff5 0 2024-04-27T04:10:17.718 INFO:tasks.workunit.client.0.smithi162.stdout:6/701: creat d6/d1a/d34/d3a/d68/d6c/d67/fde x:0 0 0 2024-04-27T04:10:17.719 INFO:tasks.workunit.client.0.smithi162.stdout:5/741: write d8/d1a/d23/d40/f6e [3435085,5606] 0 2024-04-27T04:10:17.720 INFO:tasks.workunit.client.0.smithi162.stdout:5/742: chown d8/d1a/d23/c53 4744661 1 2024-04-27T04:10:17.721 INFO:tasks.workunit.client.1.smithi181.stdout:2/953: dwrite d4/d9/d4f/d5f/d125/f12c [0,4194304] 0 2024-04-27T04:10:17.721 INFO:tasks.workunit.client.0.smithi162.stdout:4/671: dread d2/da/d1a/f30 [0,4194304] 0 2024-04-27T04:10:17.726 INFO:tasks.workunit.client.1.smithi181.stdout:6/757: rename d7/d31/c56 to d7/dc/d1c/d34/d3f/db2/cfc 0 2024-04-27T04:10:17.728 INFO:tasks.workunit.client.0.smithi162.stdout:6/702: symlink d6/db/d32/ldf 0 2024-04-27T04:10:17.729 INFO:tasks.workunit.client.1.smithi181.stdout:6/758: symlink d7/lfd 0 2024-04-27T04:10:17.731 INFO:tasks.workunit.client.1.smithi181.stdout:6/759: read d7/dc/d1c/d88/d6d/fb5 [353710,29821] 0 2024-04-27T04:10:17.733 INFO:tasks.workunit.client.1.smithi181.stdout:6/760: write d7/dc/dc3/f4b [9293070,114878] 0 2024-04-27T04:10:17.735 INFO:tasks.workunit.client.0.smithi162.stdout:7/754: link d3/d39/f52 d3/d39/d57/ffb 0 2024-04-27T04:10:17.737 INFO:tasks.workunit.client.0.smithi162.stdout:4/672: creat d2/d25/d1c/d39/d5e/fd8 x:0 0 0 2024-04-27T04:10:17.738 INFO:tasks.workunit.client.1.smithi181.stdout:6/761: truncate d7/d31/f75 666026 0 2024-04-27T04:10:17.739 INFO:tasks.workunit.client.0.smithi162.stdout:4/673: write d2/d25/d1c/d39/d3b/d6e/d6f/fc7 [523228,30512] 0 2024-04-27T04:10:17.739 INFO:tasks.workunit.client.1.smithi181.stdout:6/762: dread - d7/dc/d1c/d7f/f76 zero size 2024-04-27T04:10:17.741 INFO:tasks.workunit.client.0.smithi162.stdout:2/710: rename d0/d4/d17/d88/lb3 to d0/le2 0 2024-04-27T04:10:17.741 INFO:tasks.workunit.client.0.smithi162.stdout:5/743: mknod d8/d1a/d23/ce8 0 2024-04-27T04:10:17.742 INFO:tasks.workunit.client.0.smithi162.stdout:7/755: fdatasync d3/d5/d5b/d83/de/f30 0 2024-04-27T04:10:17.744 INFO:tasks.workunit.client.0.smithi162.stdout:3/704: creat d1/ff6 x:0 0 0 2024-04-27T04:10:17.746 INFO:tasks.workunit.client.1.smithi181.stdout:6/763: rename d7/dc/d1c/d7f/da3/cb8 to d7/dc/dc3/d72/dd6/cfe 0 2024-04-27T04:10:17.747 INFO:tasks.workunit.client.0.smithi162.stdout:4/674: creat d2/d25/d1c/d39/d70/fd9 x:0 0 0 2024-04-27T04:10:17.751 INFO:tasks.workunit.client.1.smithi181.stdout:4/817: truncate d0/d15/d25/d2b/d6d/d93/fda 1182331 0 2024-04-27T04:10:17.752 INFO:tasks.workunit.client.1.smithi181.stdout:6/764: mknod d7/dc/d1c/cff 0 2024-04-27T04:10:17.753 INFO:tasks.workunit.client.1.smithi181.stdout:6/765: truncate d7/dc/d1e/f2a 4482597 0 2024-04-27T04:10:17.753 INFO:tasks.workunit.client.0.smithi162.stdout:6/703: link d6/d1a/d34/d50/dad/lc8 d6/db/le0 0 2024-04-27T04:10:17.754 INFO:tasks.workunit.client.0.smithi162.stdout:4/675: dwrite d2/fb1 [0,4194304] 0 2024-04-27T04:10:17.755 INFO:tasks.workunit.client.0.smithi162.stdout:7/756: rename d3/d5/d5b/d83/de/dce to d3/d5/d5b/d83/de/dfc 0 2024-04-27T04:10:17.755 INFO:tasks.workunit.client.1.smithi181.stdout:2/954: dread d4/d9/d16/d45/f9e [4194304,4194304] 0 2024-04-27T04:10:17.756 INFO:tasks.workunit.client.1.smithi181.stdout:4/818: creat d0/d15/d25/d2a/f10b x:0 0 0 2024-04-27T04:10:17.758 INFO:tasks.workunit.client.0.smithi162.stdout:3/705: mknod d1/d21/d47/d57/cf7 0 2024-04-27T04:10:17.761 INFO:tasks.workunit.client.0.smithi162.stdout:2/711: rename d0/d4/d17/d4f/caf to d0/d4/d2a/d3b/d49/d5a/d63/d84/ce3 0 2024-04-27T04:10:17.762 INFO:tasks.workunit.client.1.smithi181.stdout:2/955: fdatasync d4/d9/f2d 0 2024-04-27T04:10:17.763 INFO:tasks.workunit.client.0.smithi162.stdout:2/712: read d0/d4/d2a/d41/d77/f83 [4134627,111512] 0 2024-04-27T04:10:17.764 INFO:tasks.workunit.client.0.smithi162.stdout:0/828: write d5/d26/f8d [479803,25747] 0 2024-04-27T04:10:17.765 INFO:tasks.workunit.client.1.smithi181.stdout:4/819: mkdir d0/d15/de/d19/d77/dc5/d10c 0 2024-04-27T04:10:17.765 INFO:tasks.workunit.client.0.smithi162.stdout:0/829: fdatasync d5/dc/d1e/d46/d48/d9e/db8/f92 0 2024-04-27T04:10:17.770 INFO:tasks.workunit.client.1.smithi181.stdout:2/956: dwrite d4/d9/d4f/d79/d94/fd7 [4194304,4194304] 0 2024-04-27T04:10:17.772 INFO:tasks.workunit.client.1.smithi181.stdout:4/820: dread d0/d15/d25/d2b/d6d/f74 [4194304,4194304] 0 2024-04-27T04:10:17.773 INFO:tasks.workunit.client.0.smithi162.stdout:4/676: unlink d2/d25/d1c/d24/f43 0 2024-04-27T04:10:17.776 INFO:tasks.workunit.client.0.smithi162.stdout:7/757: mknod d3/d5/d5b/d83/de/d2b/d81/daa/dbd/dbe/cfd 0 2024-04-27T04:10:17.781 INFO:tasks.workunit.client.1.smithi181.stdout:2/957: dread d4/d9/d4f/d79/fce [0,4194304] 0 2024-04-27T04:10:17.782 INFO:tasks.workunit.client.0.smithi162.stdout:3/706: creat d1/d18/d5d/d5f/d66/ff8 x:0 0 0 2024-04-27T04:10:17.786 INFO:tasks.workunit.client.0.smithi162.stdout:3/707: read d1/d45/f94 [3377889,50931] 0 2024-04-27T04:10:17.787 INFO:tasks.workunit.client.0.smithi162.stdout:3/708: chown d1/d18/d5d/d5f/fca 87527 1 2024-04-27T04:10:17.788 INFO:tasks.workunit.client.0.smithi162.stdout:3/709: chown d1/d18/d5d/d5f/d66/dd2/da3 1 1 2024-04-27T04:10:17.789 INFO:tasks.workunit.client.0.smithi162.stdout:6/704: fsync d6/d1a/d34/d7b/f83 0 2024-04-27T04:10:17.789 INFO:tasks.workunit.client.0.smithi162.stdout:3/710: write d1/d18/d23/d34/dad/dc5/fcb [890209,117051] 0 2024-04-27T04:10:17.790 INFO:tasks.workunit.client.1.smithi181.stdout:6/766: write d7/dc/d1c/f99 [1560033,98261] 0 2024-04-27T04:10:17.791 INFO:tasks.workunit.client.0.smithi162.stdout:6/705: read - d6/d1a/d34/d7b/d7e/fc4 zero size 2024-04-27T04:10:17.792 INFO:tasks.workunit.client.1.smithi181.stdout:2/958: dread d4/d9/d4f/fac [0,4194304] 0 2024-04-27T04:10:17.793 INFO:tasks.workunit.client.0.smithi162.stdout:4/677: creat d2/d25/d1c/d39/d3b/da7/fda x:0 0 0 2024-04-27T04:10:17.795 INFO:tasks.workunit.client.0.smithi162.stdout:0/830: dread d5/d26/f6a [0,4194304] 0 2024-04-27T04:10:17.795 INFO:tasks.workunit.client.0.smithi162.stdout:0/831: stat d5/dc/f18 0 2024-04-27T04:10:17.797 INFO:tasks.workunit.client.1.smithi181.stdout:2/959: mkdir d4/d9/df/d14/d137 0 2024-04-27T04:10:17.800 INFO:tasks.workunit.client.0.smithi162.stdout:5/744: rename d8/d1a/d29/d4a/d9f/f73 to d8/d1a/d23/dbc/d58/fe9 0 2024-04-27T04:10:17.806 INFO:tasks.workunit.client.1.smithi181.stdout:4/821: truncate d0/d15/de/d19/d8a/fa9 319201 0 2024-04-27T04:10:17.807 INFO:tasks.workunit.client.0.smithi162.stdout:3/711: creat d1/d18/d23/d34/daa/dab/ff9 x:0 0 0 2024-04-27T04:10:17.809 INFO:tasks.workunit.client.1.smithi181.stdout:2/960: dread d4/d9/d16/f2a [0,4194304] 0 2024-04-27T04:10:17.809 INFO:tasks.workunit.client.0.smithi162.stdout:6/706: symlink d6/db/dc0/le1 0 2024-04-27T04:10:17.810 INFO:tasks.workunit.client.0.smithi162.stdout:6/707: write d6/d1a/d34/f66 [1230033,25475] 0 2024-04-27T04:10:17.812 INFO:tasks.workunit.client.1.smithi181.stdout:6/767: write d7/dc/dc3/f70 [683136,19727] 0 2024-04-27T04:10:17.813 INFO:tasks.workunit.client.0.smithi162.stdout:6/708: truncate d6/d1a/d34/d3a/d68/d6c/d67/fde 543791 0 2024-04-27T04:10:17.813 INFO:tasks.workunit.client.1.smithi181.stdout:4/822: dwrite d0/d15/d25/d2a/d61/f9d [0,4194304] 0 2024-04-27T04:10:17.814 INFO:tasks.workunit.client.1.smithi181.stdout:2/961: chown d4/c39 3951 1 2024-04-27T04:10:17.819 INFO:tasks.workunit.client.1.smithi181.stdout:2/962: readlink d4/d9/d16/d82/d7d/db7/dcb/l83 0 2024-04-27T04:10:17.819 INFO:tasks.workunit.client.1.smithi181.stdout:2/963: chown d4/d9/d4f/d79/cad 7188472 1 2024-04-27T04:10:17.820 INFO:tasks.workunit.client.1.smithi181.stdout:6/768: mknod d7/dc/d1c/d34/d3f/c100 0 2024-04-27T04:10:17.821 INFO:tasks.workunit.client.0.smithi162.stdout:3/712: mkdir d1/d18/d23/d58/d63/dfa 0 2024-04-27T04:10:17.823 INFO:tasks.workunit.client.0.smithi162.stdout:4/678: truncate d2/d54/d7a/f83 3208200 0 2024-04-27T04:10:17.823 INFO:tasks.workunit.client.1.smithi181.stdout:2/964: dread d4/d9/d16/d45/d9a/d9d/fb3 [4194304,4194304] 0 2024-04-27T04:10:17.824 INFO:tasks.workunit.client.1.smithi181.stdout:6/769: mkdir d7/dc/d1c/d7f/da3/d101 0 2024-04-27T04:10:17.825 INFO:tasks.workunit.client.0.smithi162.stdout:6/709: chown d6/db/dc0/l96 11806577 1 2024-04-27T04:10:17.826 INFO:tasks.workunit.client.0.smithi162.stdout:0/832: link d5/dc/d1e/d56/f90 d5/dc/d1e/d20/f110 0 2024-04-27T04:10:17.827 INFO:tasks.workunit.client.1.smithi181.stdout:6/770: mkdir d7/dc/d1c/d88/db3/d102 0 2024-04-27T04:10:17.829 INFO:tasks.workunit.client.0.smithi162.stdout:2/713: getdents d0/d4/d2a/d3b/d49/d5a/d63/d84 0 2024-04-27T04:10:17.830 INFO:tasks.workunit.client.1.smithi181.stdout:6/771: dread d7/dc/d1c/d34/d3f/d7e/d8a/fad [0,4194304] 0 2024-04-27T04:10:17.831 INFO:tasks.workunit.client.1.smithi181.stdout:6/772: chown d7/dc/f12 840776384 1 2024-04-27T04:10:17.831 INFO:tasks.workunit.client.0.smithi162.stdout:3/713: write d1/d18/f4d [886316,101460] 0 2024-04-27T04:10:17.832 INFO:tasks.workunit.client.1.smithi181.stdout:6/773: chown d7/dc/d1c/d34 120100 1 2024-04-27T04:10:17.832 INFO:tasks.workunit.client.1.smithi181.stdout:6/774: stat d7/dc/d1c/d88/f95 0 2024-04-27T04:10:17.833 INFO:tasks.workunit.client.0.smithi162.stdout:4/679: creat d2/d25/d1c/d39/fdb x:0 0 0 2024-04-27T04:10:17.833 INFO:tasks.workunit.client.0.smithi162.stdout:4/680: readlink d2/d54/lbd 0 2024-04-27T04:10:17.833 INFO:tasks.workunit.client.0.smithi162.stdout:3/714: chown d1/d18/d5d/d5f/c61 31 1 2024-04-27T04:10:17.833 INFO:tasks.workunit.client.0.smithi162.stdout:6/710: readlink d6/d1a/d34/d7b/l92 0 2024-04-27T04:10:17.834 INFO:tasks.workunit.client.0.smithi162.stdout:6/711: write d6/d1a/fcc [979343,20386] 0 2024-04-27T04:10:17.834 INFO:tasks.workunit.client.0.smithi162.stdout:7/758: rename d3/d5/d5b/d83/fdd to d3/d5/d9/ffe 0 2024-04-27T04:10:17.837 INFO:tasks.workunit.client.0.smithi162.stdout:0/833: read d5/d26/f4d [88004,105882] 0 2024-04-27T04:10:17.838 INFO:tasks.workunit.client.1.smithi181.stdout:6/775: dwrite d7/dc/dc3/d72/daa/dd3/fd8 [0,4194304] 0 2024-04-27T04:10:17.839 INFO:tasks.workunit.client.0.smithi162.stdout:5/745: getdents d8/d1a 0 2024-04-27T04:10:17.841 INFO:tasks.workunit.client.1.smithi181.stdout:6/776: mkdir d7/dc/dc3/d72/daa/dd3/d103 0 2024-04-27T04:10:17.842 INFO:tasks.workunit.client.0.smithi162.stdout:4/681: mkdir d2/da/d1a/ddc 0 2024-04-27T04:10:17.842 INFO:tasks.workunit.client.0.smithi162.stdout:6/712: mknod d6/d1a/d34/d50/dad/ce2 0 2024-04-27T04:10:17.845 INFO:tasks.workunit.client.0.smithi162.stdout:7/759: rename d3/d39 to d3/d5/d5b/d83/de/d2b/d81/daa/dbd/dbe/dff 0 2024-04-27T04:10:17.847 INFO:tasks.workunit.client.0.smithi162.stdout:0/834: symlink d5/d26/d69/l111 0 2024-04-27T04:10:17.849 INFO:tasks.workunit.client.0.smithi162.stdout:6/713: write d6/f1e [5627553,19851] 0 2024-04-27T04:10:17.849 INFO:tasks.workunit.client.0.smithi162.stdout:7/760: mknod d3/d5/d5b/d83/de/d2b/d81/daa/dbd/dbe/dff/d57/c100 0 2024-04-27T04:10:17.853 INFO:tasks.workunit.client.0.smithi162.stdout:2/714: getdents d0/d4/d2a/da1/d86 0 2024-04-27T04:10:17.854 INFO:tasks.workunit.client.0.smithi162.stdout:4/682: rename d2/d25/d1c/d39/d70/d91/fb0 to d2/da/fdd 0 2024-04-27T04:10:17.854 INFO:tasks.workunit.client.0.smithi162.stdout:0/835: fdatasync d5/d26/d69/f6c 0 2024-04-27T04:10:17.855 INFO:tasks.workunit.client.0.smithi162.stdout:7/761: unlink d3/d5/d5b/d83/de/d2b/f72 0 2024-04-27T04:10:17.856 INFO:tasks.workunit.client.0.smithi162.stdout:2/715: mkdir d0/d4/d2a/d3b/d49/d5a/d63/d80/de4 0 2024-04-27T04:10:17.857 INFO:tasks.workunit.client.0.smithi162.stdout:0/836: fdatasync d5/dc/d1e/d46/d48/d9e/db8/f10e 0 2024-04-27T04:10:17.857 INFO:tasks.workunit.client.0.smithi162.stdout:7/762: dread - d3/d5/d5b/d83/de/f62 zero size 2024-04-27T04:10:17.858 INFO:tasks.workunit.client.0.smithi162.stdout:4/683: symlink d2/d25/d1c/d39/d90/lde 0 2024-04-27T04:10:17.858 INFO:tasks.workunit.client.0.smithi162.stdout:7/763: chown d3/d5/d5b/d83/de/d2b/d81/d96 0 1 2024-04-27T04:10:17.861 INFO:tasks.workunit.client.0.smithi162.stdout:7/764: mknod d3/dee/c101 0 2024-04-27T04:10:17.861 INFO:tasks.workunit.client.0.smithi162.stdout:2/716: getdents d0/d4/d2a/d3b/d49/d5a/d63 0 2024-04-27T04:10:17.862 INFO:tasks.workunit.client.0.smithi162.stdout:7/765: read - d3/d5/d3c/d4d/fcf zero size 2024-04-27T04:10:17.862 INFO:tasks.workunit.client.0.smithi162.stdout:2/717: chown d0/d24/f5e 49484042 1 2024-04-27T04:10:17.862 INFO:tasks.workunit.client.0.smithi162.stdout:4/684: rename d2/d25/d1c/d39/d70/fd9 to d2/d25/d1c/d39/d5e/fdf 0 2024-04-27T04:10:17.864 INFO:tasks.workunit.client.0.smithi162.stdout:2/718: write d0/d4/d2a/d3b/d49/d5a/d63/d84/fbf [48058,21219] 0 2024-04-27T04:10:17.865 INFO:tasks.workunit.client.0.smithi162.stdout:7/766: creat d3/d5/d5b/d83/de/dfc/f102 x:0 0 0 2024-04-27T04:10:17.865 INFO:tasks.workunit.client.0.smithi162.stdout:4/685: symlink d2/d25/d1c/d24/dd2/le0 0 2024-04-27T04:10:17.867 INFO:tasks.workunit.client.0.smithi162.stdout:7/767: write d3/d5/d5b/d83/de/d2b/df9/ffa [848623,24053] 0 2024-04-27T04:10:17.867 INFO:tasks.workunit.client.0.smithi162.stdout:4/686: creat d2/da/dd/fe1 x:0 0 0 2024-04-27T04:10:17.879 INFO:tasks.workunit.client.0.smithi162.stdout:7/768: dread d3/d4f/f67 [0,4194304] 0 2024-04-27T04:10:17.881 INFO:tasks.workunit.client.0.smithi162.stdout:7/769: mknod d3/d5/d3c/d4d/c103 0 2024-04-27T04:10:17.885 INFO:tasks.workunit.client.0.smithi162.stdout:7/770: rename d3/d5/d5b/d83/de/d2b/d81/daa/dbd/db6/dec to d3/d5/d5b/d83/de/d2b/da7/d104 0 2024-04-27T04:10:17.886 INFO:tasks.workunit.client.0.smithi162.stdout:7/771: write d3/d5/d5b/d83/de/d2b/d81/daa/dbd/fef [841387,83354] 0 2024-04-27T04:10:17.904 INFO:tasks.workunit.client.1.smithi181.stdout:6/777: getdents d7/dc/d1c/d34/d3f 0 2024-04-27T04:10:17.910 INFO:tasks.workunit.client.1.smithi181.stdout:2/965: truncate d4/d9/d16/d45/d9a/fe1 3943769 0 2024-04-27T04:10:17.914 INFO:tasks.workunit.client.0.smithi162.stdout:3/715: write d1/d18/d5d/d5f/d66/dd2/f6c [700628,28345] 0 2024-04-27T04:10:17.914 INFO:tasks.workunit.client.1.smithi181.stdout:2/966: link d4/d9/l33 d4/l138 0 2024-04-27T04:10:17.916 INFO:tasks.workunit.client.0.smithi162.stdout:3/716: fdatasync d1/d18/d5d/ff3 0 2024-04-27T04:10:17.917 INFO:tasks.workunit.client.1.smithi181.stdout:6/778: dread d7/dc/dc3/f64 [0,4194304] 0 2024-04-27T04:10:17.918 INFO:tasks.workunit.client.1.smithi181.stdout:2/967: chown d4/d9/d35/c9c 7 1 2024-04-27T04:10:17.920 INFO:tasks.workunit.client.0.smithi162.stdout:3/717: creat d1/d18/d5d/d6f/ffb x:0 0 0 2024-04-27T04:10:17.923 INFO:tasks.workunit.client.1.smithi181.stdout:2/968: mknod d4/d9/d16/d82/d7d/db7/dcb/c139 0 2024-04-27T04:10:17.924 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:10:17.926+0000 7f741031c700 1 -- 172.21.15.162:0/3679849007 --> [v2:172.21.15.181:6828/4165336392,v1:172.21.15.181:6829/4165336392] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f73ec000bf0 con 0x7f73f406be00 2024-04-27T04:10:17.925 INFO:tasks.workunit.client.1.smithi181.stdout:6/779: dwrite d7/dc/dc3/f64 [0,4194304] 0 2024-04-27T04:10:17.926 INFO:tasks.workunit.client.0.smithi162.stdout:6/714: dwrite d6/d1a/f5d [0,4194304] 0 2024-04-27T04:10:17.926 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:10:17.928+0000 7f73feffd700 1 -- 172.21.15.162:0/3679849007 <== mgr.24367 v2:172.21.15.181:6828/4165336392 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+310 (secure 0 0 0) 0x7f73ec000bf0 con 0x7f73f406be00 2024-04-27T04:10:17.927 INFO:tasks.workunit.client.0.smithi162.stdout:3/718: rename d1/d18/c54 to d1/d18/d23/d9a/cfc 0 2024-04-27T04:10:17.928 INFO:tasks.workunit.client.0.smithi162.stdout:3/719: chown d1/d21/l24 806424471 1 2024-04-27T04:10:17.931 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:10:17.932+0000 7f741031c700 1 -- 172.21.15.162:0/3679849007 >> [v2:172.21.15.181:6828/4165336392,v1:172.21.15.181:6829/4165336392] conn(0x7f73f406be00 msgr2=0x7f73f406e2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:10:17.931 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:10:17.932+0000 7f741031c700 1 --2- 172.21.15.162:0/3679849007 >> [v2:172.21.15.181:6828/4165336392,v1:172.21.15.181:6829/4165336392] conn(0x7f73f406be00 0x7f73f406e2b0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f73f8013870 tx=0x7f73f8009e30 comp rx=0 tx=0).stop 2024-04-27T04:10:17.931 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:10:17.932+0000 7f741031c700 1 -- 172.21.15.162:0/3679849007 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f74080feff0 msgr2=0x7f740810fb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:10:17.931 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:10:17.933+0000 7f741031c700 1 --2- 172.21.15.162:0/3679849007 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f74080feff0 0x7f740810fb10 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f740400eb40 tx=0x7f740400b040 comp rx=0 tx=0).stop 2024-04-27T04:10:17.931 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:10:17.933+0000 7f741031c700 1 -- 172.21.15.162:0/3679849007 shutdown_connections 2024-04-27T04:10:17.932 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:10:17.933+0000 7f741031c700 1 --2- 172.21.15.162:0/3679849007 >> [v2:172.21.15.181:6828/4165336392,v1:172.21.15.181:6829/4165336392] conn(0x7f73f406be00 0x7f73f406e2b0 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:10:17.932 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:10:17.933+0000 7f741031c700 1 --2- 172.21.15.162:0/3679849007 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f74080feff0 0x7f740810fb10 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:10:17.932 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:10:17.933+0000 7f741031c700 1 --2- 172.21.15.162:0/3679849007 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f74081002a0 0x7f7408110050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:10:17.932 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:10:17.933+0000 7f741031c700 1 -- 172.21.15.162:0/3679849007 >> 172.21.15.162:0/3679849007 conn(0x7f74080fa8f0 msgr2=0x7f74080fcf30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:10:17.932 INFO:tasks.workunit.client.1.smithi181.stdout:2/969: mkdir d4/d9/d16/d13a 0 2024-04-27T04:10:17.933 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:10:17.933+0000 7f741031c700 1 -- 172.21.15.162:0/3679849007 shutdown_connections 2024-04-27T04:10:17.933 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:10:17.933+0000 7f741031c700 1 -- 172.21.15.162:0/3679849007 wait complete. 2024-04-27T04:10:17.933 INFO:tasks.workunit.client.0.smithi162.stdout:6/715: mkdir d6/d1a/d34/d3a/d68/d6c/de3 0 2024-04-27T04:10:17.936 INFO:tasks.workunit.client.0.smithi162.stdout:3/720: dwrite d1/d18/d5d/d6f/ffb [0,4194304] 0 2024-04-27T04:10:17.936 INFO:tasks.workunit.client.1.smithi181.stdout:2/970: creat d4/d9/d16/d45/d9a/d9d/f13b x:0 0 0 2024-04-27T04:10:17.938 INFO:tasks.workunit.client.1.smithi181.stdout:2/971: dread - d4/d9/d16/d45/d9a/ff8 zero size 2024-04-27T04:10:17.943 INFO:tasks.workunit.client.0.smithi162.stdout:0/837: dwrite d5/dc/d1e/d46/d48/d9e/db8/fa9 [0,4194304] 0 2024-04-27T04:10:17.944 INFO:tasks.workunit.client.0.smithi162.stdout:3/721: creat d1/d21/d47/d57/ffd x:0 0 0 2024-04-27T04:10:17.946 INFO:tasks.workunit.client.1.smithi181.stdout:2/972: rmdir d4/d9/d4f/d5f/d125 39 2024-04-27T04:10:17.948 INFO:tasks.workunit.client.0.smithi162.stdout:4/687: write d2/da/dd/f93 [816024,61844] 0 2024-04-27T04:10:17.951 INFO:tasks.workunit.client.0.smithi162.stdout:0/838: unlink d5/dc/d1e/d46/d48/d9e/db8/fa9 0 2024-04-27T04:10:17.952 INFO:tasks.workunit.client.0.smithi162.stdout:7/772: rmdir d3/d5/d5b/d83/de/d2b 39 2024-04-27T04:10:17.953 INFO:tasks.workunit.client.1.smithi181.stdout:2/973: mknod d4/d9/d16/d45/d9a/d9d/c13c 0 2024-04-27T04:10:17.955 INFO:tasks.workunit.client.0.smithi162.stdout:3/722: rename d1/d21/d47/d57/d6e/cef to d1/d21/d47/d57/d6e/cfe 0 2024-04-27T04:10:17.964 INFO:tasks.workunit.client.0.smithi162.stdout:0/839: symlink d5/dc/d1e/d46/d48/l112 0 2024-04-27T04:10:17.965 INFO:tasks.workunit.client.0.smithi162.stdout:7/773: rmdir d3/d5/d5b/d83/de/d2b/d81/daa/dbd/db6 39 2024-04-27T04:10:17.970 INFO:tasks.workunit.client.1.smithi181.stdout:6/780: dwrite d7/dc/d1c/d34/d3f/db2/fd7 [0,4194304] 0 2024-04-27T04:10:17.970 INFO:tasks.workunit.client.0.smithi162.stdout:4/688: link d2/da/d1a/c4b d2/d25/d1c/d39/d3b/da7/ce2 0 2024-04-27T04:10:17.978 INFO:tasks.workunit.client.0.smithi162.stdout:7/774: rename d3/d5/d5b/d83/de/d2b/da7/d104 to d3/d5/d5b/d105 0 2024-04-27T04:10:17.979 INFO:tasks.workunit.client.0.smithi162.stdout:6/716: dwrite d6/db/fa0 [0,4194304] 0 2024-04-27T04:10:17.982 INFO:tasks.workunit.client.0.smithi162.stdout:0/840: dread d5/dc/d1e/d41/f34 [0,4194304] 0 2024-04-27T04:10:17.983 INFO:tasks.workunit.client.1.smithi181.stdout:6/781: rename d7/dc/d1e/d2b/fdb to d7/dc/d1c/f104 0 2024-04-27T04:10:17.983 INFO:tasks.workunit.client.0.smithi162.stdout:0/841: readlink d5/dc/d1e/d56/l104 0 2024-04-27T04:10:17.984 INFO:tasks.workunit.client.0.smithi162.stdout:7/775: write d3/d4f/f67 [1592830,20647] 0 2024-04-27T04:10:17.985 INFO:tasks.workunit.client.0.smithi162.stdout:7/776: chown d3/d5/d9/f7f 5 1 2024-04-27T04:10:17.987 INFO:tasks.workunit.client.0.smithi162.stdout:7/777: write d3/d5/d5b/d83/de/d2b/d81/daa/dbd/dab/fed [135998,24969] 0 2024-04-27T04:10:17.990 INFO:tasks.workunit.client.0.smithi162.stdout:0/842: creat d5/dc/d1e/d46/df3/f113 x:0 0 0 2024-04-27T04:10:17.992 INFO:tasks.workunit.client.1.smithi181.stdout:2/974: dwrite d4/d9/df/d14/ff6 [0,4194304] 0 2024-04-27T04:10:17.992 INFO:tasks.workunit.client.0.smithi162.stdout:9/707: sync 2024-04-27T04:10:17.992 INFO:tasks.workunit.client.0.smithi162.stdout:8/779: sync 2024-04-27T04:10:17.993 INFO:tasks.workunit.client.0.smithi162.stdout:6/717: mkdir d6/d1a/d34/d7f/dd9/de4 0 2024-04-27T04:10:17.996 INFO:tasks.workunit.client.0.smithi162.stdout:6/718: truncate d6/d1a/d34/d43/fab 308626 0 2024-04-27T04:10:18.000 INFO:tasks.workunit.client.1.smithi181.stdout:2/975: mknod d4/d9/d16/d82/d7d/db7/c13d 0 2024-04-27T04:10:18.002 INFO:tasks.workunit.client.0.smithi162.stdout:9/708: chown d3/d13/d15/d50/f8b 11 1 2024-04-27T04:10:18.002 INFO:tasks.workunit.client.0.smithi162.stdout:4/689: dread d2/da/fa1 [0,4194304] 0 2024-04-27T04:10:18.003 INFO:tasks.workunit.client.0.smithi162.stdout:8/780: fsync d0/d10/d44/d70/fee 0 2024-04-27T04:10:18.007 INFO:tasks.workunit.client.0.smithi162.stdout:6/719: symlink d6/d1a/d34/d7f/d80/db3/le5 0 2024-04-27T04:10:18.008 INFO:tasks.workunit.client.0.smithi162.stdout:9/709: dwrite d3/d23/d1e/d43/f5a [0,4194304] 0 2024-04-27T04:10:18.010 INFO:tasks.workunit.client.1.smithi181.stdout:6/782: dwrite d7/dc/d1c/d34/f9a [0,4194304] 0 2024-04-27T04:10:18.013 INFO:tasks.workunit.client.1.smithi181.stdout:8/894: sync 2024-04-27T04:10:18.013 INFO:tasks.workunit.client.1.smithi181.stdout:7/895: sync 2024-04-27T04:10:18.013 INFO:tasks.workunit.client.1.smithi181.stdout:3/852: sync 2024-04-27T04:10:18.013 INFO:tasks.workunit.client.1.smithi181.stdout:6/783: write d7/dc/d1e/d2b/dde/fe1 [466851,49696] 0 2024-04-27T04:10:18.014 INFO:tasks.workunit.client.0.smithi162.stdout:7/778: write d3/d5/d5b/d83/de/d2b/f69 [515771,109085] 0 2024-04-27T04:10:18.016 INFO:tasks.workunit.client.1.smithi181.stdout:2/976: link d4/d9/d35/ca8 d4/d9/d16/d82/d7d/dbe/d127/c13e 0 2024-04-27T04:10:18.017 INFO:tasks.workunit.client.0.smithi162.stdout:8/781: creat d0/de/d20/d7e/fff x:0 0 0 2024-04-27T04:10:18.022 INFO:tasks.workunit.client.1.smithi181.stdout:8/895: fsync d1/d6/d6f/dd2/fd4 0 2024-04-27T04:10:18.024 INFO:tasks.workunit.client.0.smithi162.stdout:0/843: dwrite d5/dc/d1e/faf [0,4194304] 0 2024-04-27T04:10:18.024 INFO:tasks.workunit.client.1.smithi181.stdout:3/853: symlink d0/d6c/d66/d99/l10f 0 2024-04-27T04:10:18.029 INFO:tasks.workunit.client.0.smithi162.stdout:8/782: chown d0/d10/d25/d7d/db8/dbe 0 1 2024-04-27T04:10:18.030 INFO:tasks.workunit.client.0.smithi162.stdout:8/783: chown d0/d10/d44/d70/fa9 4702 1 2024-04-27T04:10:18.032 INFO:tasks.workunit.client.0.smithi162.stdout:8/784: fdatasync d0/de/d6e/d71/f9d 0 2024-04-27T04:10:18.036 INFO:tasks.workunit.client.0.smithi162.stdout:0/844: rename d5/dc/d1e/d46/d48/d9e/db8/l43 to d5/dc/d1e/d41/d79/l114 0 2024-04-27T04:10:18.040 INFO:tasks.workunit.client.0.smithi162.stdout:4/690: getdents d2/d54/d7a/d95 0 2024-04-27T04:10:18.040 INFO:tasks.workunit.client.1.smithi181.stdout:7/896: getdents d1/d22/d15/d5f 0 2024-04-27T04:10:18.041 INFO:tasks.workunit.client.0.smithi162.stdout:4/691: readlink d2/d25/d1c/d39/d3b/d6e/d6f/lbf 0 2024-04-27T04:10:18.043 INFO:tasks.workunit.client.0.smithi162.stdout:7/779: dread d3/d5/d5b/d60/fd3 [4194304,4194304] 0 2024-04-27T04:10:18.043 INFO:tasks.workunit.client.0.smithi162.stdout:0/845: symlink d5/dc/d1e/d41/d79/l115 0 2024-04-27T04:10:18.044 INFO:tasks.workunit.client.1.smithi181.stdout:6/784: getdents d7/dc/dc3/d72/d98/dbf 0 2024-04-27T04:10:18.045 INFO:tasks.workunit.client.1.smithi181.stdout:6/785: chown f3 1730809 1 2024-04-27T04:10:18.046 INFO:tasks.workunit.client.0.smithi162.stdout:0/846: dread - d5/dc/d1e/d20/ffc zero size 2024-04-27T04:10:18.047 INFO:tasks.workunit.client.0.smithi162.stdout:4/692: mknod d2/d25/d1c/d39/d3b/da7/ce3 0 2024-04-27T04:10:18.050 INFO:tasks.workunit.client.1.smithi181.stdout:2/977: truncate d4/d9/d4f/d79/fec 1873197 0 2024-04-27T04:10:18.050 INFO:tasks.workunit.client.0.smithi162.stdout:6/720: dwrite d6/db/dc0/d74/f81 [0,4194304] 0 2024-04-27T04:10:18.051 INFO:tasks.workunit.client.1.smithi181.stdout:7/897: read d1/d22/d15/d1f/d34/f3c [1585488,95901] 0 2024-04-27T04:10:18.051 INFO:tasks.workunit.client.1.smithi181.stdout:3/854: dwrite d0/d9f/f7c [0,4194304] 0 2024-04-27T04:10:18.054 INFO:tasks.workunit.client.1.smithi181.stdout:8/896: getdents d1/d6/d2f/d40 0 2024-04-27T04:10:18.054 INFO:tasks.workunit.client.0.smithi162.stdout:9/710: dwrite d3/d23/d1e/d43/f48 [4194304,4194304] 0 2024-04-27T04:10:18.055 INFO:tasks.workunit.client.0.smithi162.stdout:0/847: dread - d5/fac zero size 2024-04-27T04:10:18.055 INFO:tasks.workunit.client.1.smithi181.stdout:7/898: write d1/d22/d15/f28 [4260616,105110] 0 2024-04-27T04:10:18.056 INFO:tasks.workunit.client.1.smithi181.stdout:8/897: dread - d1/d6/d2f/d30/d9d/dea/db3/ddf/f12b zero size 2024-04-27T04:10:18.057 INFO:tasks.workunit.client.1.smithi181.stdout:8/898: read - d1/d6/d2f/d30/d5c/f13b zero size 2024-04-27T04:10:18.061 INFO:tasks.workunit.client.1.smithi181.stdout:3/855: mkdir d0/d9f/d110 0 2024-04-27T04:10:18.068 INFO:tasks.workunit.client.1.smithi181.stdout:8/899: stat d1/d6/d6f/dd2/df2/c120 0 2024-04-27T04:10:18.068 INFO:tasks.workunit.client.0.smithi162.stdout:7/780: dwrite d3/d5/d5b/d60/fd3 [4194304,4194304] 0 2024-04-27T04:10:18.071 INFO:tasks.workunit.client.1.smithi181.stdout:6/786: dread d7/dc/d1c/d88/f95 [0,4194304] 0 2024-04-27T04:10:18.072 INFO:tasks.workunit.client.0.smithi162.stdout:0/848: rename d5/dc/d1e/d56 to d5/dc/d1e/d46/d48/d9e/dce/dd1/d116 0 2024-04-27T04:10:18.073 INFO:tasks.workunit.client.1.smithi181.stdout:3/856: mkdir d0/d9f/d111 0 2024-04-27T04:10:18.074 INFO:tasks.workunit.client.0.smithi162.stdout:6/721: truncate d6/db/dc0/d74/fa1 2846462 0 2024-04-27T04:10:18.075 INFO:tasks.workunit.client.1.smithi181.stdout:7/899: mkdir d1/df/d1a/d6a/d10e/d111 0 2024-04-27T04:10:18.077 INFO:tasks.workunit.client.0.smithi162.stdout:9/711: creat d3/d23/fd9 x:0 0 0 2024-04-27T04:10:18.078 INFO:tasks.workunit.client.1.smithi181.stdout:1/870: sync 2024-04-27T04:10:18.078 INFO:tasks.workunit.client.1.smithi181.stdout:4/823: sync 2024-04-27T04:10:18.078 INFO:tasks.workunit.client.1.smithi181.stdout:9/819: sync 2024-04-27T04:10:18.079 INFO:tasks.workunit.client.0.smithi162.stdout:8/785: dread d0/d10/d25/f36 [0,4194304] 0 2024-04-27T04:10:18.079 INFO:tasks.workunit.client.0.smithi162.stdout:0/849: chown d5/dc/d1e/d46/d48/d9e/dce/dd1/d116/f90 9 1 2024-04-27T04:10:18.079 INFO:tasks.workunit.client.0.smithi162.stdout:7/781: readlink d3/d5/d5b/d83/de/l46 0 2024-04-27T04:10:18.079 INFO:tasks.workunit.client.0.smithi162.stdout:5/746: sync 2024-04-27T04:10:18.079 INFO:tasks.workunit.client.0.smithi162.stdout:1/682: sync 2024-04-27T04:10:18.079 INFO:tasks.workunit.client.0.smithi162.stdout:2/719: sync 2024-04-27T04:10:18.080 INFO:tasks.workunit.client.1.smithi181.stdout:6/787: rename d7/dc/d1e/d2b/dde/fe1 to d7/dc/d1e/d2b/dde/f105 0 2024-04-27T04:10:18.080 INFO:tasks.workunit.client.0.smithi162.stdout:7/782: readlink d3/l64 0 2024-04-27T04:10:18.081 INFO:tasks.workunit.client.0.smithi162.stdout:4/693: dread d2/d25/f5c [0,4194304] 0 2024-04-27T04:10:18.081 INFO:tasks.workunit.client.1.smithi181.stdout:3/857: creat d0/d6c/d52/d44/d56/f112 x:0 0 0 2024-04-27T04:10:18.082 INFO:tasks.workunit.client.0.smithi162.stdout:9/712: mkdir d3/d13/d2d/d3b/d54/dda 0 2024-04-27T04:10:18.083 INFO:tasks.workunit.client.1.smithi181.stdout:2/978: dwrite d4/d9/df/d14/f78 [0,4194304] 0 2024-04-27T04:10:18.084 INFO:tasks.workunit.client.1.smithi181.stdout:7/900: chown d1/d57/d72/c9d 50800349 1 2024-04-27T04:10:18.084 INFO:tasks.workunit.client.1.smithi181.stdout:6/788: dread - d7/dc/d1c/d34/d3f/d7e/d8a/ff0 zero size 2024-04-27T04:10:18.085 INFO:tasks.workunit.client.0.smithi162.stdout:2/720: write d0/d4/d2a/d41/d77/d73/dc3/fd8 [830569,24358] 0 2024-04-27T04:10:18.085 INFO:tasks.workunit.client.0.smithi162.stdout:5/747: symlink d8/d9/db3/lea 0 2024-04-27T04:10:18.086 INFO:tasks.workunit.client.0.smithi162.stdout:7/783: write d3/d5/d5b/d83/de/d2b/da7/fe3 [512565,115061] 0 2024-04-27T04:10:18.090 INFO:tasks.workunit.client.0.smithi162.stdout:9/713: read d3/d23/d1e/d43/f48 [810239,124409] 0 2024-04-27T04:10:18.091 INFO:tasks.workunit.client.1.smithi181.stdout:7/901: chown d1/d2/d1c/d2d/d61/d80/f108 1642107708 1 2024-04-27T04:10:18.092 INFO:tasks.workunit.client.0.smithi162.stdout:8/786: rmdir d0/d10/d2b/d2e 39 2024-04-27T04:10:18.093 INFO:tasks.workunit.client.0.smithi162.stdout:4/694: fsync d2/da/dd/f60 0 2024-04-27T04:10:18.094 INFO:tasks.workunit.client.1.smithi181.stdout:3/858: dwrite d0/d28/d86/f8d [0,4194304] 0 2024-04-27T04:10:18.096 INFO:tasks.workunit.client.0.smithi162.stdout:0/850: dwrite d5/dc/d1e/d20/dba/fe8 [0,4194304] 0 2024-04-27T04:10:18.097 INFO:tasks.workunit.client.1.smithi181.stdout:2/979: creat d4/d9/d35/d9f/f13f x:0 0 0 2024-04-27T04:10:18.097 INFO:tasks.workunit.client.1.smithi181.stdout:6/789: write d7/dc/d1c/d34/d3f/d7e/d8a/ff0 [868965,59558] 0 2024-04-27T04:10:18.098 INFO:tasks.workunit.client.0.smithi162.stdout:0/851: dread - d5/fac zero size 2024-04-27T04:10:18.102 INFO:tasks.workunit.client.0.smithi162.stdout:0/852: truncate d5/dc/d1e/d46/d48/d9e/db8/f92 1421410 0 2024-04-27T04:10:18.103 INFO:tasks.workunit.client.0.smithi162.stdout:5/748: rename d8/d1a/d29/d4a to d8/de3/deb 0 2024-04-27T04:10:18.105 INFO:tasks.workunit.client.0.smithi162.stdout:9/714: stat d3/l35 0 2024-04-27T04:10:18.105 INFO:tasks.workunit.client.1.smithi181.stdout:3/859: dwrite d0/d6c/d52/dbb/dca/f107 [0,4194304] 0 2024-04-27T04:10:18.106 INFO:tasks.workunit.client.0.smithi162.stdout:6/722: dread d6/d1a/f18 [4194304,4194304] 0 2024-04-27T04:10:18.106 INFO:tasks.workunit.client.0.smithi162.stdout:5/749: stat d8/d1a/d23/dbc/d66/d88/f98 0 2024-04-27T04:10:18.106 INFO:tasks.workunit.client.0.smithi162.stdout:5/750: chown d8/de3 33 1 2024-04-27T04:10:18.109 INFO:tasks.workunit.client.1.smithi181.stdout:6/790: stat d7/dc/c73 0 2024-04-27T04:10:18.112 INFO:tasks.workunit.client.1.smithi181.stdout:2/980: creat d4/d9/d16/d82/d7d/dbe/f140 x:0 0 0 2024-04-27T04:10:18.112 INFO:tasks.workunit.client.1.smithi181.stdout:2/981: readlink d4/d81/d102/l126 0 2024-04-27T04:10:18.115 INFO:tasks.workunit.client.1.smithi181.stdout:8/900: write d1/d6/d1a/d25/f52 [22871,1191] 0 2024-04-27T04:10:18.115 INFO:tasks.workunit.client.0.smithi162.stdout:2/721: creat d0/d4/d2a/d3b/d49/d5a/d63/d70/d8e/fe5 x:0 0 0 2024-04-27T04:10:18.116 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:10:18.117+0000 7f8b174b9700 1 -- 172.21.15.162:0/539057610 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8b10102310 msgr2=0x7f8b101046f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:10:18.116 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:10:18.117+0000 7f8b174b9700 1 --2- 172.21.15.162:0/539057610 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8b10102310 0x7f8b101046f0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f8b00004940 tx=0x7f8b00009e30 comp rx=0 tx=0).stop 2024-04-27T04:10:18.116 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:10:18.117+0000 7f8b174b9700 1 -- 172.21.15.162:0/539057610 shutdown_connections 2024-04-27T04:10:18.116 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:10:18.117+0000 7f8b174b9700 1 --2- 172.21.15.162:0/539057610 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8b10102310 0x7f8b101046f0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:10:18.116 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:10:18.117+0000 7f8b174b9700 1 --2- 172.21.15.162:0/539057610 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8b10066ed0 0x7f8b10101dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:10:18.116 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:10:18.117+0000 7f8b174b9700 1 -- 172.21.15.162:0/539057610 >> 172.21.15.162:0/539057610 conn(0x7f8b100f7420 msgr2=0x7f8b100f9870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:10:18.117 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:10:18.118+0000 7f8b174b9700 1 -- 172.21.15.162:0/539057610 shutdown_connections 2024-04-27T04:10:18.117 INFO:tasks.workunit.client.1.smithi181.stdout:4/824: creat d0/d15/de/d19/f10d x:0 0 0 2024-04-27T04:10:18.118 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:10:18.118+0000 7f8b174b9700 1 -- 172.21.15.162:0/539057610 wait complete. 2024-04-27T04:10:18.118 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:10:18.118+0000 7f8b174b9700 1 Processor -- start 2024-04-27T04:10:18.118 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:10:18.118+0000 7f8b174b9700 1 -- start start 2024-04-27T04:10:18.118 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:10:18.119+0000 7f8b174b9700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8b10066ed0 0x7f8b1010fb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:10:18.118 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:10:18.119+0000 7f8b174b9700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8b10102310 0x7f8b10110040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:10:18.118 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:10:18.119+0000 7f8b174b9700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b10110640 con 0x7f8b10066ed0 2024-04-27T04:10:18.119 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:10:18.119+0000 7f8b174b9700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b101107b0 con 0x7f8b10102310 2024-04-27T04:10:18.119 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:10:18.119+0000 7f8b15255700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8b10066ed0 0x7f8b1010fb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:10:18.119 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:10:18.119+0000 7f8b15255700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8b10066ed0 0x7f8b1010fb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:51828/0 (socket says 172.21.15.162:51828) 2024-04-27T04:10:18.119 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:10:18.119+0000 7f8b15255700 1 -- 172.21.15.162:0/564499099 learned_addr learned my addr 172.21.15.162:0/564499099 (peer_addr_for_me v2:172.21.15.162:0/0) 2024-04-27T04:10:18.119 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:10:18.119+0000 7f8b14a54700 1 --2- 172.21.15.162:0/564499099 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8b10102310 0x7f8b10110040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:10:18.119 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:10:18.120+0000 7f8b14a54700 1 -- 172.21.15.162:0/564499099 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8b10066ed0 msgr2=0x7f8b1010fb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:10:18.119 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:10:18.120+0000 7f8b14a54700 1 --2- 172.21.15.162:0/564499099 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8b10066ed0 0x7f8b1010fb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:10:18.119 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:10:18.120+0000 7f8b14a54700 1 -- 172.21.15.162:0/564499099 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b00005040 con 0x7f8b10102310 2024-04-27T04:10:18.120 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:10:18.120+0000 7f8b14a54700 1 --2- 172.21.15.162:0/564499099 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8b10102310 0x7f8b10110040 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f8b00009fd0 tx=0x7f8b00009e30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:10:18.120 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:10:18.121+0000 7f8b05ffb700 1 -- 172.21.15.162:0/564499099 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8b00021070 con 0x7f8b10102310 2024-04-27T04:10:18.120 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:10:18.121+0000 7f8b174b9700 1 -- 172.21.15.162:0/564499099 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8b10115160 con 0x7f8b10102310 2024-04-27T04:10:18.120 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:10:18.121+0000 7f8b174b9700 1 -- 172.21.15.162:0/564499099 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8b101156a0 con 0x7f8b10102310 2024-04-27T04:10:18.120 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:10:18.121+0000 7f8b05ffb700 1 -- 172.21.15.162:0/564499099 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f8b0001c400 con 0x7f8b10102310 2024-04-27T04:10:18.120 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:10:18.121+0000 7f8b05ffb700 1 -- 172.21.15.162:0/564499099 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8b0000e070 con 0x7f8b10102310 2024-04-27T04:10:18.121 INFO:tasks.workunit.client.0.smithi162.stdout:6/723: dread d6/db/d32/f5b [0,4194304] 0 2024-04-27T04:10:18.121 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:10:18.123+0000 7f8b174b9700 1 -- 172.21.15.162:0/564499099 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8b1004c710 con 0x7f8b10102310 2024-04-27T04:10:18.122 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:10:18.123+0000 7f8b05ffb700 1 -- 172.21.15.162:0/564499099 <== mon.1 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 24) v1 ==== 89652+0+0 (secure 0 0 0) 0x7f8b0001ca20 con 0x7f8b10102310 2024-04-27T04:10:18.122 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:10:18.124+0000 7f8b05ffb700 1 --2- 172.21.15.162:0/564499099 >> [v2:172.21.15.181:6828/4165336392,v1:172.21.15.181:6829/4165336392] conn(0x7f8afc06bd00 0x7f8afc06e1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:10:18.123 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:10:18.124+0000 7f8b05ffb700 1 -- 172.21.15.162:0/564499099 <== mon.1 v2:172.21.15.181:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5808+0+0 (secure 0 0 0) 0x7f8b0008bc80 con 0x7f8b10102310 2024-04-27T04:10:18.123 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:10:18.124+0000 7f8b15255700 1 --2- 172.21.15.162:0/564499099 >> [v2:172.21.15.181:6828/4165336392,v1:172.21.15.181:6829/4165336392] conn(0x7f8afc06bd00 0x7f8afc06e1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:10:18.123 INFO:tasks.workunit.client.0.smithi162.stdout:1/683: dread d0/d80/d89/f8f [0,4194304] 0 2024-04-27T04:10:18.124 INFO:tasks.workunit.client.1.smithi181.stdout:3/860: creat d0/de/d7d/dbc/f113 x:0 0 0 2024-04-27T04:10:18.124 INFO:tasks.workunit.client.1.smithi181.stdout:8/901: dwrite d1/d6/d6f/dd2/f128 [0,4194304] 0 2024-04-27T04:10:18.124 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:10:18.126+0000 7f8b15255700 1 --2- 172.21.15.162:0/564499099 >> [v2:172.21.15.181:6828/4165336392,v1:172.21.15.181:6829/4165336392] conn(0x7f8afc06bd00 0x7f8afc06e1b0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f8b0c009b80 tx=0x7f8b0c00b040 comp rx=0 tx=0).ready entity=mgr.24367 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:10:18.125 INFO:tasks.workunit.client.1.smithi181.stdout:3/861: stat d0/d6c/d52/d44/dd5/d58/fea 0 2024-04-27T04:10:18.126 INFO:tasks.workunit.client.1.smithi181.stdout:9/820: write d0/f10 [1785774,18831] 0 2024-04-27T04:10:18.129 INFO:tasks.workunit.client.1.smithi181.stdout:1/871: dread d1/dad/db5/d6/da3/d1e/d20/d53/d56/dd9/fdd [0,4194304] 0 2024-04-27T04:10:18.130 INFO:tasks.workunit.client.1.smithi181.stdout:1/872: chown d1/dad/db5/d6/da3/d60/l90 389038217 1 2024-04-27T04:10:18.131 INFO:tasks.workunit.client.0.smithi162.stdout:5/751: creat d8/d9/d55/fec x:0 0 0 2024-04-27T04:10:18.132 INFO:tasks.workunit.client.0.smithi162.stdout:5/752: read - d8/d2c/fd8 zero size 2024-04-27T04:10:18.132 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:10:18.134+0000 7f8b05ffb700 1 -- 172.21.15.162:0/564499099 <== mon.1 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186652 (secure 0 0 0) 0x7f8b00058d70 con 0x7f8b10102310 2024-04-27T04:10:18.134 INFO:tasks.workunit.client.1.smithi181.stdout:4/825: read d0/d15/de/d19/f94 [104185,130208] 0 2024-04-27T04:10:18.134 INFO:tasks.workunit.client.0.smithi162.stdout:7/784: creat d3/d5/d5b/d83/de/d2b/d81/daa/deb/f106 x:0 0 0 2024-04-27T04:10:18.137 INFO:tasks.workunit.client.0.smithi162.stdout:7/785: write d3/d5/d5b/d83/de/d2b/d81/daa/dbd/dbe/dff/d57/f43 [1017138,120844] 0 2024-04-27T04:10:18.138 INFO:tasks.workunit.client.1.smithi181.stdout:7/902: creat d1/d57/f112 x:0 0 0 2024-04-27T04:10:18.142 INFO:tasks.workunit.client.1.smithi181.stdout:6/791: write d7/f59 [1964143,10223] 0 2024-04-27T04:10:18.144 INFO:tasks.workunit.client.0.smithi162.stdout:3/723: sync 2024-04-27T04:10:18.144 INFO:tasks.workunit.client.1.smithi181.stdout:9/821: symlink d0/d44/d4b/d43/d88/da1/dfc/dbd/dd1/l116 0 2024-04-27T04:10:18.145 INFO:tasks.workunit.client.0.smithi162.stdout:3/724: dread - d1/d21/d47/fcd zero size 2024-04-27T04:10:18.146 INFO:tasks.workunit.client.0.smithi162.stdout:4/695: dread d2/d54/d7a/f83 [0,4194304] 0 2024-04-27T04:10:18.146 INFO:tasks.workunit.client.0.smithi162.stdout:6/724: symlink d6/d1a/d34/d50/le6 0 2024-04-27T04:10:18.146 INFO:tasks.workunit.client.1.smithi181.stdout:2/982: dread d4/d9/f11 [0,4194304] 0 2024-04-27T04:10:18.147 INFO:tasks.workunit.client.0.smithi162.stdout:6/725: write d6/d1a/d34/fc3 [573416,80614] 0 2024-04-27T04:10:18.148 INFO:tasks.workunit.client.0.smithi162.stdout:0/853: mkdir d5/dc/d1e/d46/d48/d9e/dce/dd1/d116/ddd/d117 0 2024-04-27T04:10:18.152 INFO:tasks.workunit.client.1.smithi181.stdout:7/903: rename d1/d2/cd7 to d1/d57/d72/c113 0 2024-04-27T04:10:18.154 INFO:tasks.workunit.client.0.smithi162.stdout:9/715: write d3/d23/f88 [1820496,95312] 0 2024-04-27T04:10:18.154 INFO:tasks.workunit.client.0.smithi162.stdout:9/716: stat c1 0 2024-04-27T04:10:18.154 INFO:tasks.workunit.client.0.smithi162.stdout:9/717: fsync d3/fc 0 2024-04-27T04:10:18.155 INFO:tasks.workunit.client.1.smithi181.stdout:3/862: write d0/d6c/d52/d44/d56/d5b/fef [196777,31336] 0 2024-04-27T04:10:18.156 INFO:tasks.workunit.client.0.smithi162.stdout:9/718: write d3/d11/d45/d4e/fb1 [1811606,70807] 0 2024-04-27T04:10:18.159 INFO:tasks.workunit.client.1.smithi181.stdout:4/826: dread d0/d15/d25/d2b/f1c [0,4194304] 0 2024-04-27T04:10:18.159 INFO:tasks.workunit.client.1.smithi181.stdout:1/873: mknod d1/dad/db5/d6/da3/d1e/c11b 0 2024-04-27T04:10:18.161 INFO:tasks.workunit.client.0.smithi162.stdout:8/787: rmdir d0/de/d20/d7e/deb 0 2024-04-27T04:10:18.162 INFO:tasks.workunit.client.0.smithi162.stdout:1/684: dwrite d0/d11/d3e/d5f/f99 [0,4194304] 0 2024-04-27T04:10:18.163 INFO:tasks.workunit.client.1.smithi181.stdout:9/822: rename d0/d44/d4b/l60 to d0/d44/d4b/d43/d88/da1/dfc/l117 0 2024-04-27T04:10:18.168 INFO:tasks.workunit.client.1.smithi181.stdout:3/863: creat d0/d6c/d66/d88/f114 x:0 0 0 2024-04-27T04:10:18.171 INFO:tasks.workunit.client.1.smithi181.stdout:7/904: dread d1/d22/d15/d29/dbe/d1b/f8c [0,4194304] 0 2024-04-27T04:10:18.171 INFO:tasks.workunit.client.1.smithi181.stdout:6/792: getdents d7/dc/dc3/d72/daa/dd3/d103 0 2024-04-27T04:10:18.172 INFO:tasks.workunit.client.1.smithi181.stdout:6/793: chown d7/dc/d1c/d34/l8f 989406 1 2024-04-27T04:10:18.174 INFO:tasks.workunit.client.1.smithi181.stdout:6/794: read d7/dc/dc3/f6b [845647,77094] 0 2024-04-27T04:10:18.175 INFO:tasks.workunit.client.1.smithi181.stdout:6/795: stat d7/dc/d1c/d7f/cbb 0 2024-04-27T04:10:18.176 INFO:tasks.workunit.client.1.smithi181.stdout:2/983: mkdir d4/d9/d4f/d5f/d125/d141 0 2024-04-27T04:10:18.178 INFO:tasks.workunit.client.0.smithi162.stdout:0/854: creat d5/dc/d1e/d20/dec/df1/db4/f118 x:0 0 0 2024-04-27T04:10:18.180 INFO:tasks.workunit.client.1.smithi181.stdout:9/823: creat d0/d44/d4b/d43/d88/da1/dfc/dbd/df3/da0/ddb/f118 x:0 0 0 2024-04-27T04:10:18.183 INFO:tasks.workunit.client.0.smithi162.stdout:1/685: readlink d0/d7/l36 0 2024-04-27T04:10:18.183 INFO:tasks.workunit.client.0.smithi162.stdout:1/686: chown d0/d7/d3c/d77/f8d 1572641 1 2024-04-27T04:10:18.184 INFO:tasks.workunit.client.1.smithi181.stdout:7/905: truncate d1/d22/f95 3523330 0 2024-04-27T04:10:18.184 INFO:tasks.workunit.client.1.smithi181.stdout:4/827: dwrite d0/d15/d25/d2a/d36/f4a [0,4194304] 0 2024-04-27T04:10:18.186 INFO:tasks.workunit.client.1.smithi181.stdout:7/906: write d1/d22/d15/dc6/dd8/fe4 [878450,3325] 0 2024-04-27T04:10:18.186 INFO:tasks.workunit.client.0.smithi162.stdout:8/788: mkdir d0/de/d6e/d71/d86/dd9/d100 0 2024-04-27T04:10:18.190 INFO:tasks.workunit.client.0.smithi162.stdout:2/722: creat d0/fe6 x:0 0 0 2024-04-27T04:10:18.191 INFO:tasks.workunit.client.0.smithi162.stdout:4/696: symlink d2/le4 0 2024-04-27T04:10:18.193 INFO:tasks.workunit.client.1.smithi181.stdout:6/796: rename d7/dc/dc3/d72/d98/fed to d7/dc/d1c/d34/d3f/d7e/f106 0 2024-04-27T04:10:18.195 INFO:tasks.workunit.client.1.smithi181.stdout:6/797: stat d7/d27/l36 0 2024-04-27T04:10:18.195 INFO:tasks.workunit.client.1.smithi181.stdout:1/874: getdents d1/dad/db5/d6/da3/d1e/d20/d53/d56/de2/d115 0 2024-04-27T04:10:18.196 INFO:tasks.workunit.client.1.smithi181.stdout:7/907: dwrite d1/d2/d1c/d2d/d61/d80/f108 [0,4194304] 0 2024-04-27T04:10:18.197 INFO:tasks.workunit.client.1.smithi181.stdout:9/824: mkdir d0/d44/d4f/df7/d119 0 2024-04-27T04:10:18.197 INFO:tasks.workunit.client.1.smithi181.stdout:1/875: write d1/dad/fdb [86487,35633] 0 2024-04-27T04:10:18.197 INFO:tasks.workunit.client.1.smithi181.stdout:7/908: chown d1/dc3 2839 1 2024-04-27T04:10:18.199 INFO:tasks.workunit.client.0.smithi162.stdout:0/855: mkdir d5/dc/d1e/d41/d79/d109/d119 0 2024-04-27T04:10:18.201 INFO:tasks.workunit.client.1.smithi181.stdout:4/828: truncate d0/d15/f50 1009382 0 2024-04-27T04:10:18.202 INFO:tasks.workunit.client.1.smithi181.stdout:3/864: dread d0/d6c/d52/d44/d56/f76 [0,4194304] 0 2024-04-27T04:10:18.203 INFO:tasks.workunit.client.0.smithi162.stdout:5/753: rename d8/d1a/d23/dbc/d58/c8a to d8/de3/deb/d9f/ced 0 2024-04-27T04:10:18.207 INFO:tasks.workunit.client.0.smithi162.stdout:8/789: write d0/d10/d2b/d2e/dca/fd2 [816199,42555] 0 2024-04-27T04:10:18.208 INFO:tasks.workunit.client.1.smithi181.stdout:4/829: dwrite d0/d15/d25/d2b/d6d/d82/f102 [0,4194304] 0 2024-04-27T04:10:18.209 INFO:tasks.workunit.client.0.smithi162.stdout:3/725: truncate d1/d65/f91 2075540 0 2024-04-27T04:10:18.211 INFO:tasks.workunit.client.0.smithi162.stdout:2/723: creat d0/d4/d2a/d3b/d49/d5a/fe7 x:0 0 0 2024-04-27T04:10:18.218 INFO:tasks.workunit.client.0.smithi162.stdout:0/856: rename d5/d26/d69/cf8 to d5/dc/d1e/d46/d48/d9e/c11a 0 2024-04-27T04:10:18.220 INFO:tasks.workunit.client.1.smithi181.stdout:7/909: creat d1/d22/d15/d29/f114 x:0 0 0 2024-04-27T04:10:18.222 INFO:tasks.workunit.client.0.smithi162.stdout:5/754: truncate d8/d1a/d29/f4d 467723 0 2024-04-27T04:10:18.222 INFO:tasks.workunit.client.0.smithi162.stdout:5/755: dread - d8/d1a/d23/d40/fcd zero size 2024-04-27T04:10:18.224 INFO:tasks.workunit.client.1.smithi181.stdout:1/876: unlink d1/d2/d89/f118 0 2024-04-27T04:10:18.225 INFO:tasks.workunit.client.0.smithi162.stdout:1/687: mkdir d0/d7/d10/de4 0 2024-04-27T04:10:18.226 INFO:tasks.workunit.client.1.smithi181.stdout:7/910: dwrite d1/d22/d15/d29/dbe/d1b/fac [4194304,4194304] 0 2024-04-27T04:10:18.226 INFO:tasks.workunit.client.0.smithi162.stdout:1/688: chown d0/d7/d10/d75 13142 1 2024-04-27T04:10:18.231 INFO:tasks.workunit.client.0.smithi162.stdout:2/724: symlink d0/d4/d2a/d41/d77/le8 0 2024-04-27T04:10:18.233 INFO:tasks.workunit.client.1.smithi181.stdout:9/825: dread d0/d44/d4b/d43/d88/fef [0,4194304] 0 2024-04-27T04:10:18.234 INFO:tasks.workunit.client.1.smithi181.stdout:9/826: chown d0/d44/d4b/d50/lf2 10322 1 2024-04-27T04:10:18.235 INFO:tasks.workunit.client.0.smithi162.stdout:4/697: dwrite d2/f38 [4194304,4194304] 0 2024-04-27T04:10:18.237 INFO:tasks.workunit.client.0.smithi162.stdout:6/726: link d6/d1a/l28 d6/d1a/d34/d3a/le7 0 2024-04-27T04:10:18.237 INFO:tasks.workunit.client.1.smithi181.stdout:4/830: truncate d0/d15/d25/d2a/f38 2131628 0 2024-04-27T04:10:18.238 INFO:tasks.workunit.client.0.smithi162.stdout:2/725: dread d0/d4/d2a/d41/fa2 [0,4194304] 0 2024-04-27T04:10:18.241 INFO:tasks.workunit.client.0.smithi162.stdout:3/726: rename d1/d18/d23/d34/daa/fce to d1/d18/d4f/fff 0 2024-04-27T04:10:18.247 INFO:tasks.workunit.client.0.smithi162.stdout:5/756: mkdir d8/d1a/d23/dbc/d66/d88/dee 0 2024-04-27T04:10:18.248 INFO:tasks.workunit.client.0.smithi162.stdout:5/757: chown d8/d1a/d23/d40/de4 23064 1 2024-04-27T04:10:18.249 INFO:tasks.workunit.client.0.smithi162.stdout:8/790: unlink d0/de/d6e/f79 0 2024-04-27T04:10:18.250 INFO:tasks.workunit.client.0.smithi162.stdout:0/857: read d5/dc/d1e/d20/dba/df9/d98/fad [137197,9326] 0 2024-04-27T04:10:18.250 INFO:tasks.workunit.client.1.smithi181.stdout:6/798: fdatasync d7/dc/d1c/d34/f84 0 2024-04-27T04:10:18.251 INFO:tasks.workunit.client.1.smithi181.stdout:1/877: creat d1/dad/db5/d6/da3/d1e/d20/da2/f11c x:0 0 0 2024-04-27T04:10:18.254 INFO:tasks.workunit.client.0.smithi162.stdout:6/727: fsync d6/d1a/f18 0 2024-04-27T04:10:18.255 INFO:tasks.workunit.client.0.smithi162.stdout:2/726: rmdir d0/d20/d51 39 2024-04-27T04:10:18.255 INFO:tasks.workunit.client.0.smithi162.stdout:6/728: chown d6/d1a/d34/d3a/d68/d6c/f53 562 1 2024-04-27T04:10:18.255 INFO:tasks.workunit.client.0.smithi162.stdout:2/727: stat d0/d4/d2a/d41/f81 0 2024-04-27T04:10:18.256 INFO:tasks.workunit.client.1.smithi181.stdout:9/827: stat d0/d44/d4b/d50/lf5 0 2024-04-27T04:10:18.256 INFO:tasks.workunit.client.0.smithi162.stdout:2/728: chown d0/d20/l9a 5 1 2024-04-27T04:10:18.258 INFO:tasks.workunit.client.1.smithi181.stdout:2/984: getdents d4/d9/d16/d82/d7d/dbe/df0 0 2024-04-27T04:10:18.260 INFO:tasks.workunit.client.0.smithi162.stdout:5/758: symlink d8/d1a/d7d/ddf/lef 0 2024-04-27T04:10:18.266 INFO:tasks.workunit.client.0.smithi162.stdout:3/727: dwrite d1/d18/d23/d34/dad/fcc [0,4194304] 0 2024-04-27T04:10:18.269 INFO:tasks.workunit.client.0.smithi162.stdout:4/698: unlink d2/d25/d1c/d39/d70/d91/ld1 0 2024-04-27T04:10:18.272 INFO:tasks.workunit.client.0.smithi162.stdout:3/728: dread d1/d70/fa5 [0,4194304] 0 2024-04-27T04:10:18.274 INFO:tasks.workunit.client.1.smithi181.stdout:7/911: dread d1/d22/f65 [0,4194304] 0 2024-04-27T04:10:18.282 INFO:tasks.workunit.client.0.smithi162.stdout:8/791: dread d0/f4 [0,4194304] 0 2024-04-27T04:10:18.284 INFO:tasks.workunit.client.0.smithi162.stdout:3/729: symlink d1/d21/d47/d57/l100 0 2024-04-27T04:10:18.285 INFO:tasks.workunit.client.0.smithi162.stdout:3/730: chown d1/d70 3842612 1 2024-04-27T04:10:18.292 INFO:tasks.workunit.client.0.smithi162.stdout:4/699: symlink d2/d25/d1c/d39/d3b/d6e/dd5/le5 0 2024-04-27T04:10:18.293 INFO:tasks.workunit.client.0.smithi162.stdout:4/700: write d2/d25/d1c/d39/d3b/d47/d9c/fb4 [2011809,116896] 0 2024-04-27T04:10:18.294 INFO:tasks.workunit.client.0.smithi162.stdout:8/792: symlink d0/de/d6e/d71/d86/dd9/l101 0 2024-04-27T04:10:18.294 INFO:tasks.workunit.client.0.smithi162.stdout:2/729: truncate d0/d4/f16 2221603 0 2024-04-27T04:10:18.296 INFO:tasks.workunit.client.0.smithi162.stdout:3/731: rename d1/d18/d23/d34/daa/ff5 to d1/d18/d23/d9a/f101 0 2024-04-27T04:10:18.298 INFO:tasks.workunit.client.0.smithi162.stdout:1/689: creat d0/d11/d44/d66/fe5 x:0 0 0 2024-04-27T04:10:18.299 INFO:tasks.workunit.client.0.smithi162.stdout:5/759: creat d8/d1a/d23/ff0 x:0 0 0 2024-04-27T04:10:18.299 INFO:tasks.workunit.client.0.smithi162.stdout:3/732: dread d1/d21/fb8 [0,4194304] 0 2024-04-27T04:10:18.300 INFO:tasks.workunit.client.0.smithi162.stdout:5/760: readlink d8/d1a/d29/l37 0 2024-04-27T04:10:18.301 INFO:tasks.workunit.client.0.smithi162.stdout:3/733: dread - d1/d18/d4f/fff zero size 2024-04-27T04:10:18.301 INFO:tasks.workunit.client.0.smithi162.stdout:4/701: rmdir d2/d25/d1c/d24/dd2 39 2024-04-27T04:10:18.302 INFO:tasks.workunit.client.0.smithi162.stdout:8/793: unlink d0/de/d6e/l41 0 2024-04-27T04:10:18.303 INFO:tasks.workunit.client.0.smithi162.stdout:2/730: symlink d0/d4/d17/d88/dc9/le9 0 2024-04-27T04:10:18.305 INFO:tasks.workunit.client.0.smithi162.stdout:8/794: chown d0/d10/d2b/d2d/d2f/d6b/dd3 0 1 2024-04-27T04:10:18.305 INFO:tasks.workunit.client.0.smithi162.stdout:1/690: dwrite d0/f3b [0,4194304] 0 2024-04-27T04:10:18.305 INFO:tasks.workunit.client.0.smithi162.stdout:3/734: truncate d1/d18/d5d/d5f/d66/ff8 782560 0 2024-04-27T04:10:18.307 INFO:tasks.workunit.client.0.smithi162.stdout:5/761: truncate d8/d1a/f2d 4738782 0 2024-04-27T04:10:18.308 INFO:tasks.workunit.client.0.smithi162.stdout:8/795: readlink d0/d10/d2b/d2d/d2f/d39/d7c/l8f 0 2024-04-27T04:10:18.312 INFO:tasks.workunit.client.0.smithi162.stdout:1/691: rmdir d0/d80 39 2024-04-27T04:10:18.312 INFO:tasks.workunit.client.0.smithi162.stdout:2/731: dwrite d0/d20/fcf [0,4194304] 0 2024-04-27T04:10:18.318 INFO:tasks.workunit.client.0.smithi162.stdout:8/796: unlink d0/d10/d44/l8d 0 2024-04-27T04:10:18.328 INFO:tasks.workunit.client.0.smithi162.stdout:4/702: dread d2/d25/f8 [0,4194304] 0 2024-04-27T04:10:18.329 INFO:tasks.workunit.client.0.smithi162.stdout:5/762: symlink d8/lf1 0 2024-04-27T04:10:18.330 INFO:tasks.workunit.client.1.smithi181.stdout:3/865: creat d0/de/f115 x:0 0 0 2024-04-27T04:10:18.330 INFO:tasks.workunit.client.1.smithi181.stdout:3/866: readlink d0/d6c/d66/d99/l10f 0 2024-04-27T04:10:18.332 INFO:tasks.workunit.client.0.smithi162.stdout:8/797: rename d0/d10/d76/fdd to d0/d10/d25/d7d/d87/f102 0 2024-04-27T04:10:18.334 INFO:tasks.workunit.client.0.smithi162.stdout:8/798: readlink d0/de/d6e/d71/d86/dd9/l101 0 2024-04-27T04:10:18.334 INFO:tasks.workunit.client.0.smithi162.stdout:3/735: creat d1/f102 x:0 0 0 2024-04-27T04:10:18.335 INFO:tasks.workunit.client.0.smithi162.stdout:5/763: dwrite d8/d9/f48 [0,4194304] 0 2024-04-27T04:10:18.335 INFO:tasks.workunit.client.1.smithi181.stdout:9/828: dread - d0/d44/d4b/d43/d88/da1/dfc/dbd/df3/da0/ddb/fed zero size 2024-04-27T04:10:18.337 INFO:tasks.workunit.client.0.smithi162.stdout:2/732: unlink d0/d20/d51/fcd 0 2024-04-27T04:10:18.339 INFO:tasks.workunit.client.1.smithi181.stdout:3/867: dread d0/d6c/d52/d44/dd5/d58/f7e [0,4194304] 0 2024-04-27T04:10:18.339 INFO:tasks.workunit.client.1.smithi181.stdout:2/985: rmdir d4/d9/d4f 39 2024-04-27T04:10:18.340 INFO:tasks.workunit.client.1.smithi181.stdout:3/868: stat d0/d6c/d52/d44/d74/f9b 0 2024-04-27T04:10:18.341 INFO:tasks.workunit.client.0.smithi162.stdout:2/733: readlink d0/d4/d2a/d3b/d49/d5a/d63/d84/l37 0 2024-04-27T04:10:18.341 INFO:tasks.workunit.client.1.smithi181.stdout:3/869: dread - d0/d6c/d66/fbd zero size 2024-04-27T04:10:18.342 INFO:tasks.workunit.client.0.smithi162.stdout:1/692: creat d0/d7/d10/de4/fe6 x:0 0 0 2024-04-27T04:10:18.344 INFO:tasks.workunit.client.0.smithi162.stdout:4/703: mknod d2/d54/ce6 0 2024-04-27T04:10:18.345 INFO:tasks.workunit.client.0.smithi162.stdout:3/736: dwrite d1/d18/d23/d34/dad/fdf [0,4194304] 0 2024-04-27T04:10:18.345 INFO:tasks.workunit.client.0.smithi162.stdout:5/764: dwrite d8/d1a/f51 [0,4194304] 0 2024-04-27T04:10:18.348 INFO:tasks.workunit.client.1.smithi181.stdout:6/799: mkdir d7/dc/d1c/d34/d107 0 2024-04-27T04:10:18.351 INFO:tasks.workunit.client.1.smithi181.stdout:6/800: read d7/d31/f37 [1058387,21417] 0 2024-04-27T04:10:18.358 INFO:tasks.workunit.client.1.smithi181.stdout:4/831: creat d0/d15/d25/d2b/f10e x:0 0 0 2024-04-27T04:10:18.358 INFO:tasks.workunit.client.0.smithi162.stdout:8/799: dread d0/d10/f9a [0,4194304] 0 2024-04-27T04:10:18.360 INFO:tasks.workunit.client.1.smithi181.stdout:9/829: chown d0/d44/d4b/d43/d88/da1/dfc/dbd/lee 0 1 2024-04-27T04:10:18.360 INFO:tasks.workunit.client.0.smithi162.stdout:7/786: sync 2024-04-27T04:10:18.360 INFO:tasks.workunit.client.0.smithi162.stdout:9/719: sync 2024-04-27T04:10:18.367 INFO:tasks.workunit.client.1.smithi181.stdout:3/870: rmdir d0/d6c/d52/d44/dd5 39 2024-04-27T04:10:18.369 INFO:tasks.workunit.client.1.smithi181.stdout:7/912: symlink d1/d22/l115 0 2024-04-27T04:10:18.370 INFO:tasks.workunit.client.0.smithi162.stdout:3/737: creat d1/d18/d5d/d5f/d66/dd2/f103 x:0 0 0 2024-04-27T04:10:18.370 INFO:tasks.workunit.client.0.smithi162.stdout:2/734: chown d0/d4/d2a/l67 1422 1 2024-04-27T04:10:18.372 INFO:tasks.workunit.client.0.smithi162.stdout:3/738: truncate d1/d18/d23/d58/f9d 865626 0 2024-04-27T04:10:18.373 INFO:tasks.workunit.client.1.smithi181.stdout:1/878: creat d1/dad/db5/d6/da3/d1e/d48/f11d x:0 0 0 2024-04-27T04:10:18.374 INFO:tasks.workunit.client.0.smithi162.stdout:5/765: link d8/fad d8/d1a/d23/dbc/d66/d88/dee/ff2 0 2024-04-27T04:10:18.375 INFO:tasks.workunit.client.0.smithi162.stdout:3/739: write d1/d45/fe0 [3168186,70761] 0 2024-04-27T04:10:18.377 INFO:tasks.workunit.client.0.smithi162.stdout:9/720: chown d3/d13/d2d/d44/lc5 239334 1 2024-04-27T04:10:18.378 INFO:tasks.workunit.client.0.smithi162.stdout:7/787: dwrite d3/d5/d3c/d4a/fa4 [0,4194304] 0 2024-04-27T04:10:18.378 INFO:tasks.workunit.client.0.smithi162.stdout:9/721: dread - d3/d23/fd3 zero size 2024-04-27T04:10:18.382 INFO:tasks.workunit.client.1.smithi181.stdout:2/986: write d4/d9/f6d [4465638,121002] 0 2024-04-27T04:10:18.394 INFO:tasks.workunit.client.0.smithi162.stdout:3/740: rename d1/d18/d23/d34/l8a to d1/d18/d5d/d5f/d66/dd2/l104 0 2024-04-27T04:10:18.395 INFO:tasks.workunit.client.1.smithi181.stdout:9/830: write d0/d44/d4b/d43/d88/da1/dfc/dbd/fc9 [137727,114508] 0 2024-04-27T04:10:18.397 INFO:tasks.workunit.client.0.smithi162.stdout:7/788: creat d3/d5/d5b/d83/de/d2b/d81/daa/dbd/dbe/dff/d57/f107 x:0 0 0 2024-04-27T04:10:18.398 INFO:tasks.workunit.client.1.smithi181.stdout:4/832: dwrite d0/d15/d25/d2b/f1c [0,4194304] 0 2024-04-27T04:10:18.399 INFO:tasks.workunit.client.0.smithi162.stdout:8/800: dwrite d0/de/d20/f50 [0,4194304] 0 2024-04-27T04:10:18.400 INFO:tasks.workunit.client.0.smithi162.stdout:7/789: fdatasync d3/d5/d5b/d83/de/d2b/d81/daa/dbd/dab/fdc 0 2024-04-27T04:10:18.401 INFO:tasks.workunit.client.0.smithi162.stdout:5/766: mkdir d8/d9/d5b/df3 0 2024-04-27T04:10:18.401 INFO:tasks.workunit.client.0.smithi162.stdout:3/741: symlink d1/d21/d47/d57/d6e/l105 0 2024-04-27T04:10:18.405 INFO:tasks.workunit.client.1.smithi181.stdout:7/913: fsync d1/d2/d1c/d8f/ff6 0 2024-04-27T04:10:18.405 INFO:tasks.workunit.client.1.smithi181.stdout:8/902: sync 2024-04-27T04:10:18.406 INFO:tasks.workunit.client.1.smithi181.stdout:9/831: dwrite d0/d44/d4f/d9d/fe7 [0,4194304] 0 2024-04-27T04:10:18.407 INFO:tasks.workunit.client.0.smithi162.stdout:9/722: dread d3/d13/d2d/d3b/f84 [0,4194304] 0 2024-04-27T04:10:18.412 INFO:tasks.workunit.client.1.smithi181.stdout:3/871: dread d0/d6c/d52/d44/dd5/f5c [0,4194304] 0 2024-04-27T04:10:18.412 INFO:tasks.workunit.client.1.smithi181.stdout:6/801: getdents d7/dc/dc3/d72/d98/dbf 0 2024-04-27T04:10:18.413 INFO:tasks.workunit.client.1.smithi181.stdout:2/987: rmdir d4/d9/d4f/d79/d98 39 2024-04-27T04:10:18.415 INFO:tasks.workunit.client.0.smithi162.stdout:5/767: write d8/de3/deb/fbd [1321020,44110] 0 2024-04-27T04:10:18.417 INFO:tasks.workunit.client.0.smithi162.stdout:5/768: write d8/d1a/f81 [2302051,19521] 0 2024-04-27T04:10:18.419 INFO:tasks.workunit.client.1.smithi181.stdout:1/879: creat d1/dad/db5/d6/da3/d1e/d48/da6/f11e x:0 0 0 2024-04-27T04:10:18.419 INFO:tasks.workunit.client.1.smithi181.stdout:6/802: dwrite d7/f8 [0,4194304] 0 2024-04-27T04:10:18.421 INFO:tasks.workunit.client.1.smithi181.stdout:6/803: fsync d7/dc/d1c/d34/f9a 0 2024-04-27T04:10:18.421 INFO:tasks.workunit.client.1.smithi181.stdout:7/914: mkdir d1/d22/d15/d29/dbe/d1b/d116 0 2024-04-27T04:10:18.423 INFO:tasks.workunit.client.1.smithi181.stdout:8/903: symlink d1/d6/d2f/d30/d9d/dea/db3/ddf/l13c 0 2024-04-27T04:10:18.428 INFO:tasks.workunit.client.1.smithi181.stdout:4/833: dread d0/d15/de/d19/d59/f79 [0,4194304] 0 2024-04-27T04:10:18.428 INFO:tasks.workunit.client.1.smithi181.stdout:9/832: dread d0/d44/d4b/d5f/ffe [4194304,4194304] 0 2024-04-27T04:10:18.429 INFO:tasks.workunit.client.1.smithi181.stdout:4/834: dread - d0/d15/de/d19/f10d zero size 2024-04-27T04:10:18.431 INFO:tasks.workunit.client.1.smithi181.stdout:4/835: chown d0/d15/d25/d2a/da0/dc7/d109/lae 253 1 2024-04-27T04:10:18.432 INFO:tasks.workunit.client.1.smithi181.stdout:4/836: stat d0/d15/de/d19/d77/ce6 0 2024-04-27T04:10:18.434 INFO:tasks.workunit.client.1.smithi181.stdout:1/880: chown d1/dad/db5/d6/da3/f1f 271030 1 2024-04-27T04:10:18.436 INFO:tasks.workunit.client.1.smithi181.stdout:7/915: mkdir d1/d22/d15/d29/dbe/d1b/d117 0 2024-04-27T04:10:18.438 INFO:tasks.workunit.client.1.smithi181.stdout:6/804: rename d7/dc/dc3/fd0 to d7/dc/dc3/d72/dd6/f108 0 2024-04-27T04:10:18.439 INFO:tasks.workunit.client.1.smithi181.stdout:8/904: mkdir d1/d6/d6f/dd2/d13d 0 2024-04-27T04:10:18.443 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:10:18.444+0000 7f8b174b9700 1 -- 172.21.15.162:0/564499099 --> [v2:172.21.15.181:6828/4165336392,v1:172.21.15.181:6829/4165336392] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f8b100f8f80 con 0x7f8afc06bd00 2024-04-27T04:10:18.446 INFO:tasks.workunit.client.1.smithi181.stdout:4/837: rename d0/d15/de/d19/d59 to d0/d15/de/d19/d101/d10f 0 2024-04-27T04:10:18.448 INFO:tasks.workunit.client.1.smithi181.stdout:6/805: rmdir d7/dc/d1c/d88/d6d 39 2024-04-27T04:10:18.451 INFO:tasks.workunit.client.1.smithi181.stdout:9/833: link d0/d44/d4b/d43/d89/d94/f10b d0/d44/d4b/d50/f11a 0 2024-04-27T04:10:18.455 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:10:18.456+0000 7f8b05ffb700 1 -- 172.21.15.162:0/564499099 <== mgr.24367 v2:172.21.15.181:6828/4165336392 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3388 (secure 0 0 0) 0x7f8b100f8f80 con 0x7f8afc06bd00 2024-04-27T04:10:18.455 INFO:teuthology.orchestra.run.smithi162.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-27T04:10:18.455 INFO:teuthology.orchestra.run.smithi162.stdout:alertmanager.smithi162 smithi162 *:9093,9094 running (7m) 2m ago 8m 24.6M - 0.25.0 c8568f914cd2 61972851fcce 2024-04-27T04:10:18.455 INFO:teuthology.orchestra.run.smithi162.stdout:crash.smithi162 smithi162 running (8m) 2m ago 8m 6660k - 17.2.7-907-g2be3efa9 1c5834633715 228c98df4289 2024-04-27T04:10:18.455 INFO:teuthology.orchestra.run.smithi162.stdout:crash.smithi181 smithi181 running (7m) 29s ago 7m 6665k - 17.2.7-907-g2be3efa9 1c5834633715 c8e553e3df2b 2024-04-27T04:10:18.456 INFO:teuthology.orchestra.run.smithi162.stdout:grafana.smithi162 smithi162 *:3000 running (7m) 2m ago 8m 84.2M - 9.4.7 954c08fa6188 7d3364d5cddc 2024-04-27T04:10:18.456 INFO:teuthology.orchestra.run.smithi162.stdout:mds.cephfs.smithi162.gzlyjj smithi162 running (3m) 2m ago 3m 10.4M - 17.2.7-907-g2be3efa9 1c5834633715 27e5b030dd32 2024-04-27T04:10:18.456 INFO:teuthology.orchestra.run.smithi162.stdout:mds.cephfs.smithi162.xcnapk smithi162 running (3m) 2m ago 3m 13.2M - 17.2.7-907-g2be3efa9 1c5834633715 38f9030df029 2024-04-27T04:10:18.456 INFO:teuthology.orchestra.run.smithi162.stdout:mds.cephfs.smithi181.bkkcdm smithi181 running (3m) 29s ago 3m 12.8M - 17.2.7-907-g2be3efa9 1c5834633715 af55afca1263 2024-04-27T04:10:18.456 INFO:teuthology.orchestra.run.smithi162.stdout:mds.cephfs.smithi181.dzomgh smithi181 running (3m) 29s ago 3m 155M - 17.2.7-907-g2be3efa9 1c5834633715 6733296597f3 2024-04-27T04:10:18.456 INFO:teuthology.orchestra.run.smithi162.stdout:mgr.smithi162.oennfn smithi162 *:9283 running (10m) 2m ago 10m 479M - 17.2.7-907-g2be3efa9 1c5834633715 e768e27ad56e 2024-04-27T04:10:18.456 INFO:teuthology.orchestra.run.smithi162.stdout:mgr.smithi181.vtwuhz smithi181 *:8443,9283 running (50s) 29s ago 7m 96.5M - 19.0.0-3253-g4e213a58 67e414ff4bc4 402cc52dbdbb 2024-04-27T04:10:18.456 INFO:teuthology.orchestra.run.smithi162.stdout:mon.smithi162 smithi162 running (10m) 2m ago 10m 53.9M 2048M 17.2.7-907-g2be3efa9 1c5834633715 0f1e3682b9ab 2024-04-27T04:10:18.456 INFO:teuthology.orchestra.run.smithi162.stdout:mon.smithi181 smithi181 running (7m) 29s ago 7m 36.5M 2048M 17.2.7-907-g2be3efa9 1c5834633715 96e48aa3912f 2024-04-27T04:10:18.456 INFO:teuthology.orchestra.run.smithi162.stdout:node-exporter.smithi162 smithi162 *:9100 running (8m) 2m ago 8m 15.7M - 1.5.0 0da6a335fe13 e4535d340cea 2024-04-27T04:10:18.456 INFO:teuthology.orchestra.run.smithi162.stdout:node-exporter.smithi181 smithi181 *:9100 running (7m) 29s ago 7m 15.7M - 1.5.0 0da6a335fe13 3cc278142657 2024-04-27T04:10:18.456 INFO:teuthology.orchestra.run.smithi162.stdout:osd.0 smithi162 running (6m) 2m ago 6m 48.2M 4288M 17.2.7-907-g2be3efa9 1c5834633715 4a9df3aa938d 2024-04-27T04:10:18.456 INFO:teuthology.orchestra.run.smithi162.stdout:osd.1 smithi162 running (6m) 2m ago 6m 48.4M 4288M 17.2.7-907-g2be3efa9 1c5834633715 aefb464c46e8 2024-04-27T04:10:18.456 INFO:teuthology.orchestra.run.smithi162.stdout:osd.2 smithi162 running (6m) 2m ago 6m 44.5M 4288M 17.2.7-907-g2be3efa9 1c5834633715 70f16d86d38e 2024-04-27T04:10:18.456 INFO:teuthology.orchestra.run.smithi162.stdout:osd.3 smithi181 running (5m) 29s ago 5m 118M 5312M 17.2.7-907-g2be3efa9 1c5834633715 030d829be54e 2024-04-27T04:10:18.456 INFO:teuthology.orchestra.run.smithi162.stdout:osd.4 smithi181 running (5m) 29s ago 5m 108M 5312M 17.2.7-907-g2be3efa9 1c5834633715 512afbee7cf7 2024-04-27T04:10:18.457 INFO:teuthology.orchestra.run.smithi162.stdout:osd.5 smithi181 running (4m) 29s ago 4m 110M 5312M 17.2.7-907-g2be3efa9 1c5834633715 f47a1daac1ea 2024-04-27T04:10:18.457 INFO:teuthology.orchestra.run.smithi162.stdout:prometheus.smithi162 smithi162 *:9095 running (7m) 2m ago 8m 54.6M - 2.43.0 a07b618ecd1d 63755cc04fec 2024-04-27T04:10:18.458 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:10:18.459+0000 7f8b174b9700 1 -- 172.21.15.162:0/564499099 >> [v2:172.21.15.181:6828/4165336392,v1:172.21.15.181:6829/4165336392] conn(0x7f8afc06bd00 msgr2=0x7f8afc06e1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:10:18.458 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:10:18.459+0000 7f8b174b9700 1 --2- 172.21.15.162:0/564499099 >> [v2:172.21.15.181:6828/4165336392,v1:172.21.15.181:6829/4165336392] conn(0x7f8afc06bd00 0x7f8afc06e1b0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f8b0c009b80 tx=0x7f8b0c00b040 comp rx=0 tx=0).stop 2024-04-27T04:10:18.458 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:10:18.459+0000 7f8b174b9700 1 -- 172.21.15.162:0/564499099 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8b10102310 msgr2=0x7f8b10110040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:10:18.458 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:10:18.459+0000 7f8b174b9700 1 --2- 172.21.15.162:0/564499099 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8b10102310 0x7f8b10110040 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f8b00009fd0 tx=0x7f8b00009e30 comp rx=0 tx=0).stop 2024-04-27T04:10:18.458 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:10:18.459+0000 7f8b174b9700 1 -- 172.21.15.162:0/564499099 shutdown_connections 2024-04-27T04:10:18.458 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:10:18.459+0000 7f8b174b9700 1 --2- 172.21.15.162:0/564499099 >> [v2:172.21.15.181:6828/4165336392,v1:172.21.15.181:6829/4165336392] conn(0x7f8afc06bd00 0x7f8afc06e1b0 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:10:18.458 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:10:18.459+0000 7f8b174b9700 1 --2- 172.21.15.162:0/564499099 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8b10066ed0 0x7f8b1010fb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:10:18.458 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:10:18.459+0000 7f8b174b9700 1 --2- 172.21.15.162:0/564499099 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8b10102310 0x7f8b10110040 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:10:18.458 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:10:18.459+0000 7f8b174b9700 1 -- 172.21.15.162:0/564499099 >> 172.21.15.162:0/564499099 conn(0x7f8b100f7420 msgr2=0x7f8b100f7860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:10:18.458 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:10:18.459+0000 7f8b174b9700 1 -- 172.21.15.162:0/564499099 shutdown_connections 2024-04-27T04:10:18.458 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:10:18.459+0000 7f8b174b9700 1 -- 172.21.15.162:0/564499099 wait complete. 2024-04-27T04:10:18.461 INFO:tasks.workunit.client.1.smithi181.stdout:6/806: mknod d7/dc/dc3/dd1/c109 0 2024-04-27T04:10:18.462 INFO:tasks.workunit.client.1.smithi181.stdout:8/905: creat d1/d6/d2f/d11e/dec/d131/f13e x:0 0 0 2024-04-27T04:10:18.463 INFO:tasks.workunit.client.1.smithi181.stdout:3/872: write d0/d6c/d52/d44/dd5/f69 [814204,63535] 0 2024-04-27T04:10:18.464 INFO:tasks.workunit.client.1.smithi181.stdout:9/834: dread - d0/d44/d4b/d43/d88/da1/dfc/dbd/dd1/dde/d104/f9c zero size 2024-04-27T04:10:18.466 INFO:tasks.workunit.client.1.smithi181.stdout:1/881: creat d1/dad/db5/d6/da3/d1e/f11f x:0 0 0 2024-04-27T04:10:18.469 INFO:tasks.workunit.client.1.smithi181.stdout:4/838: symlink d0/l110 0 2024-04-27T04:10:18.474 INFO:tasks.workunit.client.1.smithi181.stdout:8/906: rename d1/d6/d1a to d1/d6/d6f/dd2/d13f 0 2024-04-27T04:10:18.477 INFO:tasks.workunit.client.1.smithi181.stdout:2/988: truncate d4/d9/d16/d82/d7d/db7/dcb/f97 3455658 0 2024-04-27T04:10:18.477 INFO:tasks.workunit.client.0.smithi162.stdout:2/735: link d0/d4/f16 d0/d4/d17/d2b/d64/d96/fea 0 2024-04-27T04:10:18.478 INFO:tasks.workunit.client.0.smithi162.stdout:2/736: write d0/d4/d2a/d41/d77/d73/fb1 [202308,123736] 0 2024-04-27T04:10:18.479 INFO:tasks.workunit.client.0.smithi162.stdout:2/737: fsync d0/d4/d2a/d41/d77/f7c 0 2024-04-27T04:10:18.480 INFO:tasks.workunit.client.1.smithi181.stdout:7/916: getdents d1/df 0 2024-04-27T04:10:18.482 INFO:tasks.workunit.client.1.smithi181.stdout:4/839: rmdir d0/d15/de/d60 39 2024-04-27T04:10:18.484 INFO:tasks.workunit.client.0.smithi162.stdout:3/742: rename d1/d18/d23/f53 to d1/d18/d5d/d5f/f106 0 2024-04-27T04:10:18.486 INFO:tasks.workunit.client.0.smithi162.stdout:5/769: chown d8/de3/deb/d9f/lae 30 1 2024-04-27T04:10:18.487 INFO:tasks.workunit.client.1.smithi181.stdout:8/907: mkdir d1/d8c/d140 0 2024-04-27T04:10:18.490 INFO:tasks.workunit.client.0.smithi162.stdout:8/801: link d0/d10/d2b/le4 d0/de/d6e/d71/d86/dd9/d100/l103 0 2024-04-27T04:10:18.490 INFO:tasks.workunit.client.1.smithi181.stdout:8/908: dread d1/d6/d59/df1/f115 [0,4194304] 0 2024-04-27T04:10:18.491 INFO:tasks.workunit.client.0.smithi162.stdout:8/802: write d0/de/d20/d6c/fcb [47383,30141] 0 2024-04-27T04:10:18.491 INFO:tasks.workunit.client.1.smithi181.stdout:9/835: symlink d0/d44/d4b/d50/dfd/l11b 0 2024-04-27T04:10:18.491 INFO:tasks.workunit.client.0.smithi162.stdout:8/803: chown d0/d10/d25/d7d/d87 1113026345 1 2024-04-27T04:10:18.495 INFO:tasks.workunit.client.0.smithi162.stdout:3/743: mkdir d1/d18/d23/d34/daa/de6/d107 0 2024-04-27T04:10:18.496 INFO:tasks.workunit.client.1.smithi181.stdout:7/917: mknod d1/d22/d15/dc6/de5/c118 0 2024-04-27T04:10:18.497 INFO:tasks.workunit.client.1.smithi181.stdout:7/918: write d1/d2/d1c/d2d/d61/f102 [344007,28734] 0 2024-04-27T04:10:18.500 INFO:tasks.workunit.client.0.smithi162.stdout:7/790: link d3/d5/d5b/d83/de/f38 d3/d5/d5b/d83/de/d2b/d81/daa/dbd/dbe/dff/d57/f108 0 2024-04-27T04:10:18.500 INFO:tasks.workunit.client.1.smithi181.stdout:7/919: chown d1/d22/d15/d29/dbe/d1b/c6b 657 1 2024-04-27T04:10:18.501 INFO:tasks.workunit.client.1.smithi181.stdout:6/807: creat d7/dc/dc3/d72/d98/f10a x:0 0 0 2024-04-27T04:10:18.503 INFO:tasks.workunit.client.0.smithi162.stdout:6/729: sync 2024-04-27T04:10:18.503 INFO:tasks.workunit.client.0.smithi162.stdout:0/858: sync 2024-04-27T04:10:18.503 INFO:tasks.workunit.client.0.smithi162.stdout:2/738: getdents d0/d4/d2a/d3b/d49/d5a/d63/d80/de4 0 2024-04-27T04:10:18.505 INFO:tasks.workunit.client.1.smithi181.stdout:3/873: write d0/d6c/d52/f59 [4841114,81869] 0 2024-04-27T04:10:18.507 INFO:tasks.workunit.client.1.smithi181.stdout:8/909: unlink d1/d6/d6f/dd2/d13f/d25/f99 0 2024-04-27T04:10:18.509 INFO:tasks.workunit.client.1.smithi181.stdout:8/910: chown d1/d6/d2f/d30 56387 1 2024-04-27T04:10:18.511 INFO:tasks.workunit.client.1.smithi181.stdout:9/836: mkdir d0/d44/d4b/d43/d89/d94/d11c 0 2024-04-27T04:10:18.511 INFO:tasks.workunit.client.0.smithi162.stdout:5/770: dread d8/d9/d55/f69 [0,4194304] 0 2024-04-27T04:10:18.512 INFO:tasks.workunit.client.1.smithi181.stdout:3/874: dwrite d0/d9f/f7c [4194304,4194304] 0 2024-04-27T04:10:18.514 INFO:tasks.workunit.client.0.smithi162.stdout:8/804: unlink d0/f7 0 2024-04-27T04:10:18.517 INFO:tasks.workunit.client.0.smithi162.stdout:0/859: fsync d5/f27 0 2024-04-27T04:10:18.521 INFO:tasks.workunit.client.1.smithi181.stdout:2/989: write d4/d9/d4f/d79/d94/fe7 [280071,113036] 0 2024-04-27T04:10:18.522 INFO:tasks.workunit.client.1.smithi181.stdout:7/920: truncate d1/d22/fc0 1357702 0 2024-04-27T04:10:18.525 INFO:tasks.workunit.client.1.smithi181.stdout:6/808: mkdir d7/dc/d1e/d2b/d10b 0 2024-04-27T04:10:18.528 INFO:tasks.workunit.client.0.smithi162.stdout:5/771: rmdir d8 39 2024-04-27T04:10:18.528 INFO:tasks.workunit.client.1.smithi181.stdout:1/882: dwrite d1/dad/db5/d6/d6a/fee [0,4194304] 0 2024-04-27T04:10:18.533 INFO:tasks.workunit.client.1.smithi181.stdout:4/840: dwrite d0/d15/d25/d2a/da0/dc7/d109/d7c/fdd [0,4194304] 0 2024-04-27T04:10:18.536 INFO:tasks.workunit.client.1.smithi181.stdout:4/841: dread - d0/d15/de/d19/d77/dc5/ffb zero size 2024-04-27T04:10:18.536 INFO:tasks.workunit.client.1.smithi181.stdout:9/837: mkdir d0/d44/d4b/d43/d89/d11d 0 2024-04-27T04:10:18.538 INFO:tasks.workunit.client.1.smithi181.stdout:3/875: rmdir d0/d6c/d52/d44/dd5/d58/df0 39 2024-04-27T04:10:18.538 INFO:tasks.workunit.client.1.smithi181.stdout:4/842: write d0/d15/de/d19/d77/d57/f9f [1092361,128474] 0 2024-04-27T04:10:18.540 INFO:tasks.workunit.client.0.smithi162.stdout:6/730: mknod d6/d1a/d34/d7b/ce8 0 2024-04-27T04:10:18.542 INFO:tasks.workunit.client.1.smithi181.stdout:7/921: fdatasync d1/d2/d1c/d2d/d61/f68 0 2024-04-27T04:10:18.543 INFO:tasks.workunit.client.0.smithi162.stdout:3/744: link d1/d21/d47/d57/cf7 d1/d18/d5d/d5f/c108 0 2024-04-27T04:10:18.545 INFO:tasks.workunit.client.0.smithi162.stdout:7/791: creat d3/d5/d5b/d83/de/d2b/d81/daa/dbd/dbe/dff/f109 x:0 0 0 2024-04-27T04:10:18.546 INFO:tasks.workunit.client.0.smithi162.stdout:7/792: chown d3/d5/d5b/d83/fb7 279115749 1 2024-04-27T04:10:18.548 INFO:tasks.workunit.client.0.smithi162.stdout:8/805: mknod d0/d10/d25/d65/d96/dcd/c104 0 2024-04-27T04:10:18.549 INFO:tasks.workunit.client.0.smithi162.stdout:8/806: fdatasync d0/d10/d44/ff4 0 2024-04-27T04:10:18.549 INFO:tasks.workunit.client.0.smithi162.stdout:8/807: fdatasync d0/d10/d44/ff4 0 2024-04-27T04:10:18.550 INFO:tasks.workunit.client.0.smithi162.stdout:0/860: creat d5/dc/d1e/d20/dec/df1/d10a/f11b x:0 0 0 2024-04-27T04:10:18.551 INFO:tasks.workunit.client.1.smithi181.stdout:1/883: mknod d1/dad/db5/d6/da3/d1e/d48/db4/c120 0 2024-04-27T04:10:18.552 INFO:tasks.workunit.client.0.smithi162.stdout:2/739: creat d0/d4/d17/d2b/d64/d96/feb x:0 0 0 2024-04-27T04:10:18.553 INFO:tasks.workunit.client.0.smithi162.stdout:2/740: fsync d0/d4/d17/d2b/f95 0 2024-04-27T04:10:18.553 INFO:tasks.workunit.client.0.smithi162.stdout:1/693: sync 2024-04-27T04:10:18.554 INFO:tasks.workunit.client.1.smithi181.stdout:8/911: dwrite d1/d6/d2f/d40/f27 [0,4194304] 0 2024-04-27T04:10:18.555 INFO:tasks.workunit.client.0.smithi162.stdout:3/745: rename d1/d18/d23/d9a to d1/d18/d23/d34/dad/dc5/d109 0 2024-04-27T04:10:18.559 INFO:tasks.workunit.client.1.smithi181.stdout:3/876: mkdir d0/de/d7d/d116 0 2024-04-27T04:10:18.560 INFO:tasks.workunit.client.0.smithi162.stdout:5/772: mkdir d8/d9/d5b/d96/df4 0 2024-04-27T04:10:18.565 INFO:tasks.workunit.client.0.smithi162.stdout:0/861: creat d5/dc/d1e/d46/d48/d9e/dce/f11c x:0 0 0 2024-04-27T04:10:18.570 INFO:tasks.workunit.client.1.smithi181.stdout:6/809: rename d7/dc/d1c/d34/d3f/db2/cfc to d7/dc/dc3/d72/d98/dbf/c10c 0 2024-04-27T04:10:18.572 INFO:tasks.workunit.client.1.smithi181.stdout:3/877: dwrite d0/de/fb3 [0,4194304] 0 2024-04-27T04:10:18.573 INFO:tasks.workunit.client.0.smithi162.stdout:2/741: rmdir d0/d4/d2a/d3b/d49/d5a/d63/d84 39 2024-04-27T04:10:18.574 INFO:tasks.workunit.client.1.smithi181.stdout:8/912: creat d1/d6/d2f/d47/f141 x:0 0 0 2024-04-27T04:10:18.574 INFO:tasks.workunit.client.0.smithi162.stdout:2/742: write d0/d20/fc6 [306404,42820] 0 2024-04-27T04:10:18.574 INFO:tasks.workunit.client.1.smithi181.stdout:2/990: write d4/f30 [5432925,99819] 0 2024-04-27T04:10:18.575 INFO:tasks.workunit.client.1.smithi181.stdout:2/991: chown d4/d9/d4f 42 1 2024-04-27T04:10:18.576 INFO:tasks.workunit.client.0.smithi162.stdout:3/746: creat d1/d18/d5d/d5f/d66/dd2/f10a x:0 0 0 2024-04-27T04:10:18.579 INFO:tasks.workunit.client.0.smithi162.stdout:7/793: truncate d3/d5/f53 932759 0 2024-04-27T04:10:18.580 INFO:tasks.workunit.client.0.smithi162.stdout:1/694: write d0/f34 [2743678,24659] 0 2024-04-27T04:10:18.581 INFO:tasks.workunit.client.1.smithi181.stdout:4/843: unlink d0/d15/d25/d2a/da0/fc1 0 2024-04-27T04:10:18.581 INFO:tasks.workunit.client.0.smithi162.stdout:3/747: dread d1/d18/f1e [0,4194304] 0 2024-04-27T04:10:18.582 INFO:tasks.workunit.client.0.smithi162.stdout:7/794: write d3/d5/d5b/d83/de/fbb [300199,98329] 0 2024-04-27T04:10:18.582 INFO:tasks.workunit.client.1.smithi181.stdout:4/844: chown d0/d15/d25/d2b/f5f 43354000 1 2024-04-27T04:10:18.582 INFO:tasks.workunit.client.1.smithi181.stdout:4/845: stat d0/l110 0 2024-04-27T04:10:18.583 INFO:tasks.workunit.client.1.smithi181.stdout:6/810: symlink d7/dc/d1c/d34/d3f/d7e/d8a/l10d 0 2024-04-27T04:10:18.584 INFO:tasks.workunit.client.0.smithi162.stdout:0/862: chown d5/dc/d1e/d46/fc3 691128431 1 2024-04-27T04:10:18.584 INFO:tasks.workunit.client.1.smithi181.stdout:4/846: write d0/d15/d25/d2a/d36/db8/dca/f103 [695428,113603] 0 2024-04-27T04:10:18.585 INFO:tasks.workunit.client.0.smithi162.stdout:6/731: creat d6/fe9 x:0 0 0 2024-04-27T04:10:18.585 INFO:tasks.workunit.client.1.smithi181.stdout:6/811: chown d7/dc/d1c/c57 9 1 2024-04-27T04:10:18.585 INFO:tasks.workunit.client.0.smithi162.stdout:0/863: fsync d5/dc/d1e/d20/dba/df9/d98/fa3 0 2024-04-27T04:10:18.589 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:10:18.590+0000 7f06a4268700 1 -- 172.21.15.162:0/2265392732 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f069c0fff70 msgr2=0x7f069c102350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:10:18.589 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:10:18.590+0000 7f06a4268700 1 --2- 172.21.15.162:0/2265392732 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f069c0fff70 0x7f069c102350 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f068c004940 tx=0x7f068c009e30 comp rx=0 tx=0).stop 2024-04-27T04:10:18.590 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:10:18.590+0000 7f06a4268700 1 -- 172.21.15.162:0/2265392732 shutdown_connections 2024-04-27T04:10:18.590 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:10:18.590+0000 7f06a4268700 1 --2- 172.21.15.162:0/2265392732 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f069c0fff70 0x7f069c102350 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:10:18.590 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:10:18.590+0000 7f06a4268700 1 --2- 172.21.15.162:0/2265392732 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f069c066e70 0x7f069c0ffa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:10:18.590 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:10:18.590+0000 7f06a4268700 1 -- 172.21.15.162:0/2265392732 >> 172.21.15.162:0/2265392732 conn(0x7f069c0f7420 msgr2=0x7f069c0f9870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:10:18.590 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:10:18.591+0000 7f06a4268700 1 -- 172.21.15.162:0/2265392732 shutdown_connections 2024-04-27T04:10:18.590 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:10:18.591+0000 7f06a4268700 1 -- 172.21.15.162:0/2265392732 wait complete. 2024-04-27T04:10:18.591 INFO:tasks.workunit.client.1.smithi181.stdout:4/847: symlink d0/d15/d5c/da1/l111 0 2024-04-27T04:10:18.591 INFO:tasks.workunit.client.1.smithi181.stdout:3/878: symlink d0/d6c/d52/d44/dd5/d58/l117 0 2024-04-27T04:10:18.591 INFO:tasks.workunit.client.1.smithi181.stdout:4/848: chown d0/d15/d5c/d69/c97 618834 1 2024-04-27T04:10:18.591 INFO:tasks.workunit.client.1.smithi181.stdout:4/849: fsync d0/d15/d25/d2a/f10b 0 2024-04-27T04:10:18.592 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:10:18.591+0000 7f06a4268700 1 Processor -- start 2024-04-27T04:10:18.593 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:10:18.592+0000 7f06a4268700 1 -- start start 2024-04-27T04:10:18.593 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:10:18.592+0000 7f06a4268700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f069c066e70 0x7f069c0fd230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:10:18.593 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:10:18.592+0000 7f06a4268700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f069c0fff70 0x7f069c0fb930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:10:18.593 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:10:18.592+0000 7f06a4268700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f069c0fd830 con 0x7f069c066e70 2024-04-27T04:10:18.593 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:10:18.592+0000 7f06a4268700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f069c0fbe70 con 0x7f069c0fff70 2024-04-27T04:10:18.593 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:10:18.592+0000 7f06a2004700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f069c066e70 0x7f069c0fd230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:10:18.593 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:10:18.592+0000 7f06a2004700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f069c066e70 0x7f069c0fd230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:51848/0 (socket says 172.21.15.162:51848) 2024-04-27T04:10:18.593 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:10:18.592+0000 7f06a2004700 1 -- 172.21.15.162:0/1347577413 learned_addr learned my addr 172.21.15.162:0/1347577413 (peer_addr_for_me v2:172.21.15.162:0/0) 2024-04-27T04:10:18.593 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:10:18.592+0000 7f06a1803700 1 --2- 172.21.15.162:0/1347577413 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f069c0fff70 0x7f069c0fb930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:10:18.593 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:10:18.592+0000 7f06a2004700 1 -- 172.21.15.162:0/1347577413 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f069c0fff70 msgr2=0x7f069c0fb930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:10:18.593 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:10:18.592+0000 7f06a2004700 1 --2- 172.21.15.162:0/1347577413 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f069c0fff70 0x7f069c0fb930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:10:18.593 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:10:18.592+0000 7f06a2004700 1 -- 172.21.15.162:0/1347577413 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f068c005040 con 0x7f069c066e70 2024-04-27T04:10:18.593 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:10:18.593+0000 7f06a2004700 1 --2- 172.21.15.162:0/1347577413 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f069c066e70 0x7f069c0fd230 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f069800eb70 tx=0x7f069800b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:10:18.594 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:10:18.593+0000 7f0692ffd700 1 -- 172.21.15.162:0/1347577413 <== mon.0 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0698007cc0 con 0x7f069c066e70 2024-04-27T04:10:18.594 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:10:18.593+0000 7f0692ffd700 1 -- 172.21.15.162:0/1347577413 <== mon.0 v2:172.21.15.162:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f069800f040 con 0x7f069c066e70 2024-04-27T04:10:18.594 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:10:18.593+0000 7f06a4268700 1 -- 172.21.15.162:0/1347577413 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f069c0fc0f0 con 0x7f069c066e70 2024-04-27T04:10:18.594 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:10:18.593+0000 7f0692ffd700 1 -- 172.21.15.162:0/1347577413 <== mon.0 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f069800cb20 con 0x7f069c066e70 2024-04-27T04:10:18.594 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:10:18.593+0000 7f06a4268700 1 -- 172.21.15.162:0/1347577413 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f069c0fc630 con 0x7f069c066e70 2024-04-27T04:10:18.594 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:10:18.594+0000 7f06a4268700 1 -- 172.21.15.162:0/1347577413 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f069c04c710 con 0x7f069c066e70 2024-04-27T04:10:18.595 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:10:18.595+0000 7f0692ffd700 1 -- 172.21.15.162:0/1347577413 <== mon.0 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 24) v1 ==== 89652+0+0 (secure 0 0 0) 0x7f069800cc80 con 0x7f069c066e70 2024-04-27T04:10:18.595 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:10:18.595+0000 7f0692ffd700 1 --2- 172.21.15.162:0/1347577413 >> [v2:172.21.15.181:6828/4165336392,v1:172.21.15.181:6829/4165336392] conn(0x7f068806bd00 0x7f068806e1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:10:18.595 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:10:18.595+0000 7f0692ffd700 1 -- 172.21.15.162:0/1347577413 <== mon.0 v2:172.21.15.162:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5808+0+0 (secure 0 0 0) 0x7f069808a470 con 0x7f069c066e70 2024-04-27T04:10:18.595 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:10:18.595+0000 7f06a1803700 1 --2- 172.21.15.162:0/1347577413 >> [v2:172.21.15.181:6828/4165336392,v1:172.21.15.181:6829/4165336392] conn(0x7f068806bd00 0x7f068806e1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:10:18.595 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:10:18.596+0000 7f06a1803700 1 --2- 172.21.15.162:0/1347577413 >> [v2:172.21.15.181:6828/4165336392,v1:172.21.15.181:6829/4165336392] conn(0x7f068806bd00 0x7f068806e1b0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f068c00b0d0 tx=0x7f068c009e30 comp rx=0 tx=0).ready entity=mgr.24367 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:10:18.595 INFO:tasks.workunit.client.1.smithi181.stdout:9/838: getdents d0/d44/d4b/d43 0 2024-04-27T04:10:18.598 INFO:tasks.workunit.client.0.smithi162.stdout:7/795: creat d3/d5/d5b/d83/de/d2b/d81/d96/dc8/f10a x:0 0 0 2024-04-27T04:10:18.599 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:10:18.600+0000 7f0692ffd700 1 -- 172.21.15.162:0/1347577413 <== mon.0 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186652 (secure 0 0 0) 0x7f0698050930 con 0x7f069c066e70 2024-04-27T04:10:18.600 INFO:tasks.workunit.client.0.smithi162.stdout:7/796: dread - d3/d5/d5b/d83/de/d2b/d81/daa/dbd/dbe/dff/d57/f107 zero size 2024-04-27T04:10:18.600 INFO:tasks.workunit.client.1.smithi181.stdout:6/812: mknod d7/dc/d1c/d88/d6d/c10e 0 2024-04-27T04:10:18.602 INFO:tasks.workunit.client.1.smithi181.stdout:2/992: dread d4/d9/d16/d82/fa5 [0,4194304] 0 2024-04-27T04:10:18.603 INFO:tasks.workunit.client.0.smithi162.stdout:0/864: mknod d5/dc/d1e/d20/dec/df1/c11d 0 2024-04-27T04:10:18.603 INFO:tasks.workunit.client.0.smithi162.stdout:3/748: dread d1/d18/d5d/d5f/fca [0,4194304] 0 2024-04-27T04:10:18.604 INFO:tasks.workunit.client.1.smithi181.stdout:9/839: rmdir d0/d44/d4b/d43/d89/daf 39 2024-04-27T04:10:18.605 INFO:tasks.workunit.client.0.smithi162.stdout:2/743: readlink d0/d4/d17/d88/dc9/le9 0 2024-04-27T04:10:18.605 INFO:tasks.workunit.client.0.smithi162.stdout:3/749: chown d1/d18/d5d/d5f/d66/dd2/f103 140508676 1 2024-04-27T04:10:18.606 INFO:tasks.workunit.client.0.smithi162.stdout:2/744: stat d0/d4/d2a/d3b/d49/d5a/d63/d6e/fa7 0 2024-04-27T04:10:18.607 INFO:tasks.workunit.client.0.smithi162.stdout:2/745: fdatasync d0/d24/f38 0 2024-04-27T04:10:18.608 INFO:tasks.workunit.client.0.smithi162.stdout:8/808: getdents d0/d10/d25/d7d/db8 0 2024-04-27T04:10:18.609 INFO:tasks.workunit.client.0.smithi162.stdout:2/746: read d0/d4/d17/d2b/d64/d89/fb9 [211444,15703] 0 2024-04-27T04:10:18.610 INFO:tasks.workunit.client.0.smithi162.stdout:1/695: link d0/d7/d3c/fb6 d0/d11/d50/d53/d93/fe7 0 2024-04-27T04:10:18.613 INFO:tasks.workunit.client.1.smithi181.stdout:3/879: truncate d0/d6c/f60 1943089 0 2024-04-27T04:10:18.615 INFO:tasks.workunit.client.1.smithi181.stdout:4/850: link d0/d15/d5c/d69/f10 d0/d15/d76/da4/f112 0 2024-04-27T04:10:18.618 INFO:tasks.workunit.client.1.smithi181.stdout:6/813: rename d7/dc/dc3/d72/daa/dd3/c87 to d7/dc/dc3/dd1/dd9/c10f 0 2024-04-27T04:10:18.619 INFO:tasks.workunit.client.1.smithi181.stdout:9/840: write d0/d44/d4b/d43/d88/da1/dfc/dbd/df3/da0/ddb/fed [976938,16340] 0 2024-04-27T04:10:18.623 INFO:tasks.workunit.client.0.smithi162.stdout:5/773: getdents d8/d1a/d23/d40 0 2024-04-27T04:10:18.623 INFO:tasks.workunit.client.0.smithi162.stdout:3/750: unlink d1/d21/d47/d57/d6e/l82 0 2024-04-27T04:10:18.624 INFO:tasks.workunit.client.1.smithi181.stdout:6/814: dread d7/dc/d1c/d34/d3f/d7e/d8a/ff0 [0,4194304] 0 2024-04-27T04:10:18.625 INFO:tasks.workunit.client.0.smithi162.stdout:1/696: dread d0/f1e [4194304,4194304] 0 2024-04-27T04:10:18.626 INFO:tasks.workunit.client.1.smithi181.stdout:2/993: truncate d4/d9/d35/d42/d86/f8a 94970 0 2024-04-27T04:10:18.627 INFO:tasks.workunit.client.1.smithi181.stdout:3/880: mknod d0/d6c/d52/d44/d74/c118 0 2024-04-27T04:10:18.633 INFO:tasks.workunit.client.1.smithi181.stdout:2/994: dwrite d4/d9/df/d14/da7/f108 [0,4194304] 0 2024-04-27T04:10:18.633 INFO:tasks.workunit.client.0.smithi162.stdout:9/723: sync 2024-04-27T04:10:18.633 INFO:tasks.workunit.client.0.smithi162.stdout:4/704: sync 2024-04-27T04:10:18.634 INFO:tasks.workunit.client.0.smithi162.stdout:2/747: creat d0/d4/d2a/d3b/d49/d5a/d63/d70/fec x:0 0 0 2024-04-27T04:10:18.634 INFO:tasks.workunit.client.0.smithi162.stdout:9/724: fdatasync d3/d13/d2d/d3b/f47 0 2024-04-27T04:10:18.637 INFO:tasks.workunit.client.0.smithi162.stdout:7/797: creat d3/d5/d5b/d83/de/d2b/d81/daa/df1/f10b x:0 0 0 2024-04-27T04:10:18.638 INFO:tasks.workunit.client.1.smithi181.stdout:6/815: mkdir d7/dc/dc3/d72/daa/d110 0 2024-04-27T04:10:18.639 INFO:tasks.workunit.client.1.smithi181.stdout:6/816: chown d7/dc/d1c/d34/d3f 88245 1 2024-04-27T04:10:18.641 INFO:tasks.workunit.client.1.smithi181.stdout:3/881: creat d0/d6c/d52/d44/dd5/f119 x:0 0 0 2024-04-27T04:10:18.643 INFO:tasks.workunit.client.0.smithi162.stdout:7/798: dread d3/d5/d3c/d4a/d55/dc6/fcd [0,4194304] 0 2024-04-27T04:10:18.644 INFO:tasks.workunit.client.0.smithi162.stdout:0/865: link d5/d26/d4a/f77 d5/dc/d1e/d20/f11e 0 2024-04-27T04:10:18.646 INFO:tasks.workunit.client.1.smithi181.stdout:4/851: dwrite d0/d15/d25/d2a/da0/fbb [0,4194304] 0 2024-04-27T04:10:18.648 INFO:tasks.workunit.client.0.smithi162.stdout:5/774: read d8/d2c/f4c [2293372,30835] 0 2024-04-27T04:10:18.654 INFO:tasks.workunit.client.1.smithi181.stdout:2/995: mknod d4/d9/d16/d82/c142 0 2024-04-27T04:10:18.655 INFO:tasks.workunit.client.0.smithi162.stdout:8/809: symlink d0/de/d6e/l105 0 2024-04-27T04:10:18.657 INFO:tasks.workunit.client.1.smithi181.stdout:6/817: dwrite d7/dc/dc3/d72/daa/feb [0,4194304] 0 2024-04-27T04:10:18.657 INFO:tasks.workunit.client.1.smithi181.stdout:3/882: dwrite d0/d6c/d52/d44/dd5/d58/fea [4194304,4194304] 0 2024-04-27T04:10:18.658 INFO:tasks.workunit.client.1.smithi181.stdout:9/841: dread d0/d44/d4b/f80 [0,4194304] 0 2024-04-27T04:10:18.662 INFO:tasks.workunit.client.1.smithi181.stdout:6/818: dread d7/fb [0,4194304] 0 2024-04-27T04:10:18.664 INFO:tasks.workunit.client.0.smithi162.stdout:4/705: readlink d2/da/l86 0 2024-04-27T04:10:18.665 INFO:tasks.workunit.client.1.smithi181.stdout:4/852: creat d0/d15/d25/d2a/d36/db8/f113 x:0 0 0 2024-04-27T04:10:18.666 INFO:tasks.workunit.client.1.smithi181.stdout:4/853: chown d0/d15/d25/d2a/d36/cf3 515 1 2024-04-27T04:10:18.667 INFO:tasks.workunit.client.1.smithi181.stdout:4/854: stat d0/d15/d25/d2a/da0/dc7/d109/lae 0 2024-04-27T04:10:18.698 INFO:tasks.workunit.client.0.smithi162.stdout:4/706: dread d2/d25/d1c/d39/d3b/d6e/d6f/fc6 [0,4194304] 0 2024-04-27T04:10:18.784 INFO:tasks.workunit.client.1.smithi181.stdout:3/883: unlink d0/d9f/f9a 0 2024-04-27T04:10:18.784 INFO:tasks.workunit.client.0.smithi162.stdout:3/751: write d1/d70/fa5 [1855309,96563] 0 2024-04-27T04:10:18.787 INFO:tasks.workunit.client.0.smithi162.stdout:7/799: rename d3/f1e to d3/d5/d3c/d4a/d55/dc6/f10c 0 2024-04-27T04:10:18.789 INFO:tasks.workunit.client.0.smithi162.stdout:0/866: mknod d5/dc/d1e/d46/df3/c11f 0 2024-04-27T04:10:18.791 INFO:tasks.workunit.client.1.smithi181.stdout:2/996: read d4/d9/d4f/d79/d98/f99 [1038656,74753] 0 2024-04-27T04:10:18.791 INFO:tasks.workunit.client.0.smithi162.stdout:5/775: symlink d8/d9/d5b/d96/lf5 0 2024-04-27T04:10:18.792 INFO:tasks.workunit.client.1.smithi181.stdout:4/855: unlink d0/d15/d25/d2a/da0/dc7/d109/lb3 0 2024-04-27T04:10:18.794 INFO:tasks.workunit.client.0.smithi162.stdout:1/697: mknod d0/d7/d5e/ce8 0 2024-04-27T04:10:18.794 INFO:tasks.workunit.client.1.smithi181.stdout:9/842: mkdir d0/d44/d4b/d43/d89/d11d/d11e 0 2024-04-27T04:10:18.795 INFO:tasks.workunit.client.1.smithi181.stdout:9/843: readlink d0/d44/d4b/d43/d88/da1/dfc/dbd/dd1/ld3 0 2024-04-27T04:10:18.796 INFO:tasks.workunit.client.1.smithi181.stdout:6/819: rename d7/dc/d1c/f35 to d7/dc/d1c/d7f/f111 0 2024-04-27T04:10:18.800 INFO:tasks.workunit.client.1.smithi181.stdout:2/997: rename d4/d9/d35/d9f/ff1 to d4/d9/d4f/d79/d98/f143 0 2024-04-27T04:10:18.802 INFO:tasks.workunit.client.1.smithi181.stdout:2/998: dread - d4/d9/d4f/d79/d94/df4/f111 zero size 2024-04-27T04:10:18.804 INFO:tasks.workunit.client.0.smithi162.stdout:6/732: link d6/d1a/d34/d50/dad/lc8 d6/d1a/d34/lea 0 2024-04-27T04:10:18.806 INFO:tasks.workunit.client.0.smithi162.stdout:3/752: unlink d1/d45/f87 0 2024-04-27T04:10:18.807 INFO:tasks.workunit.client.0.smithi162.stdout:8/810: dread d0/d10/d25/f30 [4194304,4194304] 0 2024-04-27T04:10:18.807 INFO:tasks.workunit.client.1.smithi181.stdout:6/820: mknod d7/dc/d1c/d34/c112 0 2024-04-27T04:10:18.809 INFO:tasks.workunit.client.0.smithi162.stdout:4/707: rename d2/da/d1a to d2/d25/d1c/d39/d70/d91/de7 0 2024-04-27T04:10:18.811 INFO:tasks.workunit.client.0.smithi162.stdout:7/800: rmdir d3/d5/d5b/d83/de/d2b/da7 39 2024-04-27T04:10:18.812 INFO:tasks.workunit.client.0.smithi162.stdout:7/801: chown d3/d5/d5b/d83/de/d2b/d81/daa/dbd/dbe/dff 0 1 2024-04-27T04:10:18.812 INFO:tasks.workunit.client.1.smithi181.stdout:4/856: creat d0/f114 x:0 0 0 2024-04-27T04:10:18.813 INFO:tasks.workunit.client.1.smithi181.stdout:4/857: chown d0/d15/d5c/da1/l111 6 1 2024-04-27T04:10:18.814 INFO:tasks.workunit.client.0.smithi162.stdout:9/725: truncate d3/d13/d15/f19 3761734 0 2024-04-27T04:10:18.816 INFO:tasks.workunit.client.1.smithi181.stdout:3/884: getdents d0/d6c/d52 0 2024-04-27T04:10:18.817 INFO:tasks.workunit.client.0.smithi162.stdout:5/776: mknod d8/d1a/d23/dbc/d66/d88/cf6 0 2024-04-27T04:10:18.817 INFO:tasks.workunit.client.1.smithi181.stdout:4/858: dread d0/d15/f32 [0,4194304] 0 2024-04-27T04:10:18.818 INFO:tasks.workunit.client.1.smithi181.stdout:6/821: rename d7/dc/d1c/d88/f95 to d7/dc/d1e/d2b/d10b/f113 0 2024-04-27T04:10:18.820 INFO:tasks.workunit.client.0.smithi162.stdout:1/698: mkdir d0/d11/d50/d53/d93/d96/daa/de9 0 2024-04-27T04:10:18.820 INFO:tasks.workunit.client.1.smithi181.stdout:2/999: mknod d4/d9/d4f/d79/d123/c144 0 2024-04-27T04:10:18.824 INFO:tasks.workunit.client.1.smithi181.stdout:3/885: truncate d0/d6c/d52/d44/d74/fc1 788177 0 2024-04-27T04:10:18.824 INFO:tasks.workunit.client.1.smithi181.stdout:3/886: stat d0/d6c/d52/d44/dd5/l48 0 2024-04-27T04:10:18.826 INFO:tasks.workunit.client.1.smithi181.stdout:4/859: creat d0/dd4/f115 x:0 0 0 2024-04-27T04:10:18.827 INFO:tasks.workunit.client.1.smithi181.stdout:4/860: fdatasync d0/d15/fe9 0 2024-04-27T04:10:18.831 INFO:tasks.workunit.client.0.smithi162.stdout:0/867: creat d5/d26/da2/dc0/f120 x:0 0 0 2024-04-27T04:10:18.833 INFO:tasks.workunit.client.0.smithi162.stdout:1/699: truncate d0/d11/d44/d66/f90 2066691 0 2024-04-27T04:10:18.834 INFO:tasks.workunit.client.1.smithi181.stdout:6/822: dread d7/f19 [0,4194304] 0 2024-04-27T04:10:18.834 INFO:tasks.workunit.client.0.smithi162.stdout:3/753: dread d1/d65/f91 [0,4194304] 0 2024-04-27T04:10:18.835 INFO:tasks.workunit.client.0.smithi162.stdout:2/748: getdents d0/d4/d2a 0 2024-04-27T04:10:18.836 INFO:tasks.workunit.client.1.smithi181.stdout:3/887: getdents d0/de/d7d 0 2024-04-27T04:10:18.837 INFO:tasks.workunit.client.0.smithi162.stdout:2/749: write d0/d20/fc6 [671301,113422] 0 2024-04-27T04:10:18.837 INFO:tasks.workunit.client.0.smithi162.stdout:3/754: write d1/d18/d5d/d5f/d66/fe5 [552409,95721] 0 2024-04-27T04:10:18.837 INFO:tasks.workunit.client.1.smithi181.stdout:3/888: fdatasync d0/d28/d86/f8d 0 2024-04-27T04:10:18.841 INFO:tasks.workunit.client.0.smithi162.stdout:9/726: symlink d3/d11/d45/d4e/ldb 0 2024-04-27T04:10:18.842 INFO:tasks.workunit.client.0.smithi162.stdout:5/777: mknod d8/d1a/d23/dbc/d66/d88/dee/cf7 0 2024-04-27T04:10:18.843 INFO:tasks.workunit.client.0.smithi162.stdout:7/802: truncate d3/d5/d5b/d83/de/d2b/d81/daa/dbd/dbe/dff/d57/f108 1077522 0 2024-04-27T04:10:18.843 INFO:tasks.workunit.client.1.smithi181.stdout:3/889: chown d0/d6c/d52/dbb/d98/fac 31093468 1 2024-04-27T04:10:18.844 INFO:tasks.workunit.client.1.smithi181.stdout:4/861: dread d0/d15/d5c/fb2 [0,4194304] 0 2024-04-27T04:10:18.844 INFO:tasks.workunit.client.1.smithi181.stdout:3/890: stat d0/d6c/d52/d44/dd5/f5e 0 2024-04-27T04:10:18.846 INFO:tasks.workunit.client.0.smithi162.stdout:6/733: link d6/d1a/d34/d43/la2 d6/d1a/d34/d3a/d68/d6c/de3/leb 0 2024-04-27T04:10:18.846 INFO:tasks.workunit.client.0.smithi162.stdout:5/778: fdatasync d8/d1a/d23/d40/fcd 0 2024-04-27T04:10:18.847 INFO:tasks.workunit.client.0.smithi162.stdout:4/708: dread d2/d25/fb [0,4194304] 0 2024-04-27T04:10:18.847 INFO:tasks.workunit.client.1.smithi181.stdout:3/891: mkdir d0/d28/d86/d11a 0 2024-04-27T04:10:18.848 INFO:tasks.workunit.client.0.smithi162.stdout:6/734: chown d6/d1a/f18 117 1 2024-04-27T04:10:18.849 INFO:tasks.workunit.client.0.smithi162.stdout:2/750: dwrite d0/d4/d17/f2f [0,4194304] 0 2024-04-27T04:10:18.849 INFO:tasks.workunit.client.0.smithi162.stdout:3/755: dwrite d1/d18/d5d/f62 [4194304,4194304] 0 2024-04-27T04:10:18.852 INFO:tasks.workunit.client.1.smithi181.stdout:4/862: rename d0/d15/l1e to d0/d15/d25/d2a/d36/db8/dca/l116 0 2024-04-27T04:10:18.852 INFO:tasks.workunit.client.0.smithi162.stdout:8/811: rename d0/d10/d25/d7d/db8/dbe to d0/d10/d25/d106 0 2024-04-27T04:10:18.854 INFO:tasks.workunit.client.0.smithi162.stdout:2/751: write d0/d4/d2a/d3b/d49/d5a/d63/d70/fec [733057,72016] 0 2024-04-27T04:10:18.861 INFO:tasks.workunit.client.0.smithi162.stdout:2/752: readlink d0/d24/lc2 0 2024-04-27T04:10:18.861 INFO:tasks.workunit.client.1.smithi181.stdout:3/892: creat d0/d6c/d52/d44/dd5/d58/de4/daf/f11b x:0 0 0 2024-04-27T04:10:18.863 INFO:tasks.workunit.client.1.smithi181.stdout:4/863: rmdir d0/d15/de/d19/d77/d57 39 2024-04-27T04:10:18.863 INFO:tasks.workunit.client.1.smithi181.stdout:3/893: fsync d0/d6c/d52/dbb/fd6 0 2024-04-27T04:10:18.865 INFO:tasks.workunit.client.0.smithi162.stdout:0/868: mknod d5/dc/d1e/d46/d48/d9e/dce/dd1/d116/c121 0 2024-04-27T04:10:18.866 INFO:tasks.workunit.client.1.smithi181.stdout:3/894: truncate d0/d6c/d52/d44/dd5/f5e 4355015 0 2024-04-27T04:10:18.866 INFO:tasks.workunit.client.0.smithi162.stdout:4/709: dwrite d2/d25/d1c/d24/fa6 [0,4194304] 0 2024-04-27T04:10:18.868 INFO:tasks.workunit.client.0.smithi162.stdout:1/700: creat d0/d7/d3c/d6e/fea x:0 0 0 2024-04-27T04:10:18.870 INFO:tasks.workunit.client.0.smithi162.stdout:7/803: creat d3/d5/d5b/d83/f10d x:0 0 0 2024-04-27T04:10:18.873 INFO:tasks.workunit.client.1.smithi181.stdout:6/823: write d7/dc/d1c/f91 [2271347,88028] 0 2024-04-27T04:10:18.874 INFO:tasks.workunit.client.1.smithi181.stdout:6/824: chown d7/f43 77 1 2024-04-27T04:10:18.876 INFO:tasks.workunit.client.1.smithi181.stdout:3/895: mknod d0/d6c/d52/d44/dd5/d58/de4/dc5/c11c 0 2024-04-27T04:10:18.879 INFO:tasks.workunit.client.0.smithi162.stdout:7/804: dwrite d3/d5/d5b/d83/f10d [0,4194304] 0 2024-04-27T04:10:18.879 INFO:tasks.workunit.client.1.smithi181.stdout:4/864: creat d0/d15/de/f117 x:0 0 0 2024-04-27T04:10:18.882 INFO:tasks.workunit.client.0.smithi162.stdout:3/756: rename d1/d45/f7e to d1/d21/d47/d57/dd6/f10b 0 2024-04-27T04:10:18.882 INFO:tasks.workunit.client.1.smithi181.stdout:6/825: rmdir d7/dc/dc3 39 2024-04-27T04:10:18.886 INFO:tasks.workunit.client.1.smithi181.stdout:6/826: readlink d7/dc/d1c/l77 0 2024-04-27T04:10:18.886 INFO:tasks.workunit.client.1.smithi181.stdout:3/896: mkdir d0/d6c/d52/d44/dd5/d58/d64/dc6/ddd/d11d 0 2024-04-27T04:10:18.888 INFO:tasks.workunit.client.1.smithi181.stdout:4/865: mknod d0/d15/d25/d2a/d36/db8/dca/c118 0 2024-04-27T04:10:18.893 INFO:tasks.workunit.client.0.smithi162.stdout:9/727: write d3/d13/d2d/d3b/f84 [4648904,60697] 0 2024-04-27T04:10:18.894 INFO:tasks.workunit.client.0.smithi162.stdout:4/710: mknod d2/d25/d1c/d39/d3b/d47/d9c/ce8 0 2024-04-27T04:10:18.895 INFO:tasks.workunit.client.0.smithi162.stdout:4/711: stat d2/d25/f5c 0 2024-04-27T04:10:18.895 INFO:tasks.workunit.client.1.smithi181.stdout:4/866: dwrite d0/d15/d25/d2a/f7e [0,4194304] 0 2024-04-27T04:10:18.897 INFO:tasks.workunit.client.0.smithi162.stdout:1/701: unlink d0/c6 0 2024-04-27T04:10:18.901 INFO:tasks.workunit.client.1.smithi181.stdout:6/827: rename d7/dc/d1c/d34/d3f/d7e/d8a/lf9 to d7/l114 0 2024-04-27T04:10:18.902 INFO:tasks.workunit.client.0.smithi162.stdout:5/779: symlink d8/d9/d5b/d96/df4/lf8 0 2024-04-27T04:10:18.906 INFO:tasks.workunit.client.1.smithi181.stdout:4/867: creat d0/d15/d25/d2a/da0/dc7/d109/d7c/db5/f119 x:0 0 0 2024-04-27T04:10:18.907 INFO:tasks.workunit.client.0.smithi162.stdout:6/735: mknod d6/d1a/d34/d3a/d68/cec 0 2024-04-27T04:10:18.908 INFO:tasks.workunit.client.1.smithi181.stdout:4/868: dread - d0/d15/d25/d2a/d36/db8/dca/ff7 zero size 2024-04-27T04:10:18.909 INFO:tasks.workunit.client.1.smithi181.stdout:6/828: dwrite d7/dc/d1c/d88/db3/ff4 [0,4194304] 0 2024-04-27T04:10:18.910 INFO:tasks.workunit.client.1.smithi181.stdout:6/829: rename d7 to d7/d31/d115 22 2024-04-27T04:10:18.911 INFO:tasks.workunit.client.1.smithi181.stdout:6/830: chown d7/d27/cb4 1005 1 2024-04-27T04:10:18.912 INFO:tasks.workunit.client.0.smithi162.stdout:7/805: creat d3/d5/d5b/d83/de/d2b/d81/d96/dc8/f10e x:0 0 0 2024-04-27T04:10:18.912 INFO:tasks.workunit.client.0.smithi162.stdout:7/806: chown d3/l64 0 1 2024-04-27T04:10:18.915 INFO:tasks.workunit.client.0.smithi162.stdout:3/757: fdatasync d1/d45/f9c 0 2024-04-27T04:10:18.921 INFO:tasks.workunit.client.1.smithi181.stdout:3/897: dwrite d0/f71 [0,4194304] 0 2024-04-27T04:10:18.923 INFO:tasks.workunit.client.1.smithi181.stdout:6/831: creat d7/dc/dc3/d72/f116 x:0 0 0 2024-04-27T04:10:18.929 INFO:tasks.workunit.client.1.smithi181.stdout:3/898: rename d0/d6c/d66/dc2/d9e to d0/de/d7d/dbc/de9/d11e 0 2024-04-27T04:10:18.948 INFO:tasks.workunit.client.1.smithi181.stdout:6/832: dread - d7/dc/d1c/d34/d3f/d7e/d8a/f8e zero size 2024-04-27T04:10:18.948 INFO:tasks.workunit.client.0.smithi162.stdout:5/780: mknod d8/d9/d5b/d96/cf9 0 2024-04-27T04:10:18.949 INFO:tasks.workunit.client.1.smithi181.stdout:4/869: getdents d0/d15/d5c 0 2024-04-27T04:10:18.950 INFO:tasks.workunit.client.0.smithi162.stdout:6/736: mknod d6/db/dc0/d74/ced 0 2024-04-27T04:10:18.950 INFO:tasks.workunit.client.0.smithi162.stdout:6/737: fsync d6/fe9 0 2024-04-27T04:10:18.953 INFO:tasks.workunit.client.0.smithi162.stdout:3/758: mknod d1/d21/d47/d57/c10c 0 2024-04-27T04:10:18.954 INFO:tasks.workunit.client.1.smithi181.stdout:6/833: creat d7/dc/d1c/d34/d107/f117 x:0 0 0 2024-04-27T04:10:18.955 INFO:tasks.workunit.client.0.smithi162.stdout:2/753: creat d0/d4/d17/d2b/d64/fed x:0 0 0 2024-04-27T04:10:18.957 INFO:tasks.workunit.client.0.smithi162.stdout:1/702: creat d0/d7/d10/d75/db7/feb x:0 0 0 2024-04-27T04:10:18.959 INFO:tasks.workunit.client.0.smithi162.stdout:4/712: write d2/d25/d1c/fab [63714,48324] 0 2024-04-27T04:10:18.962 INFO:tasks.workunit.client.0.smithi162.stdout:8/812: rename d0/d10/d44/cc3 to d0/d10/c107 0 2024-04-27T04:10:18.962 INFO:tasks.workunit.client.0.smithi162.stdout:9/728: rename d3 to d3/d11/d45/ddc 22 2024-04-27T04:10:18.963 INFO:tasks.workunit.client.1.smithi181.stdout:4/870: getdents d0/dd4 0 2024-04-27T04:10:18.965 INFO:tasks.workunit.client.1.smithi181.stdout:6/834: mkdir d7/dc/d1c/d7f/d3e/d118 0 2024-04-27T04:10:18.966 INFO:tasks.workunit.client.1.smithi181.stdout:6/835: write d7/d31/f8c [1184942,108988] 0 2024-04-27T04:10:18.967 INFO:tasks.workunit.client.0.smithi162.stdout:7/807: write d3/d5/d5b/d83/de/f1c [803306,106875] 0 2024-04-27T04:10:18.967 INFO:tasks.workunit.client.1.smithi181.stdout:4/871: truncate d0/d15/d25/d2b/d6d/d82/fc3 1576020 0 2024-04-27T04:10:18.968 INFO:tasks.workunit.client.0.smithi162.stdout:8/813: dwrite d0/d10/d2b/d2d/d2f/d39/f59 [0,4194304] 0 2024-04-27T04:10:18.969 INFO:tasks.workunit.client.1.smithi181.stdout:6/836: stat d7/dc/dc3/d72/daa/dd3/l79 0 2024-04-27T04:10:18.969 INFO:tasks.workunit.client.0.smithi162.stdout:7/808: chown d3/d5/d5b/d83/f3b 438 1 2024-04-27T04:10:18.970 INFO:tasks.workunit.client.0.smithi162.stdout:3/759: symlink d1/d18/d5d/d6f/l10d 0 2024-04-27T04:10:18.971 INFO:tasks.workunit.client.0.smithi162.stdout:7/809: chown d3/d5/d5b/d60/f6f 7095 1 2024-04-27T04:10:18.972 INFO:tasks.workunit.client.1.smithi181.stdout:3/899: dwrite d0/d6c/d52/dbb/d98/fac [0,4194304] 0 2024-04-27T04:10:18.973 INFO:tasks.workunit.client.0.smithi162.stdout:0/869: getdents d5/dc/d1e 0 2024-04-27T04:10:18.975 INFO:tasks.workunit.client.0.smithi162.stdout:2/754: rmdir d0/d4/d2a/d3b 39 2024-04-27T04:10:18.978 INFO:tasks.workunit.client.1.smithi181.stdout:3/900: symlink d0/d6c/d52/dbb/de5/l11f 0 2024-04-27T04:10:18.980 INFO:tasks.workunit.client.1.smithi181.stdout:6/837: dread d7/dc/dc3/d62/f63 [0,4194304] 0 2024-04-27T04:10:18.981 INFO:tasks.workunit.client.0.smithi162.stdout:5/781: dwrite d8/d9/f21 [0,4194304] 0 2024-04-27T04:10:18.984 INFO:tasks.workunit.client.0.smithi162.stdout:9/729: symlink d3/d13/d15/d50/ldd 0 2024-04-27T04:10:18.984 INFO:tasks.workunit.client.1.smithi181.stdout:3/901: fdatasync d0/d6c/d52/d44/dd5/d58/de4/daf/f11b 0 2024-04-27T04:10:18.985 INFO:tasks.workunit.client.0.smithi162.stdout:6/738: symlink d6/d1a/d34/lee 0 2024-04-27T04:10:18.986 INFO:tasks.workunit.client.1.smithi181.stdout:4/872: getdents d0/d15/d25/d8f 0 2024-04-27T04:10:18.992 INFO:tasks.workunit.client.1.smithi181.stdout:3/902: dwrite d0/f14 [0,4194304] 0 2024-04-27T04:10:18.997 INFO:tasks.workunit.client.0.smithi162.stdout:4/713: mkdir d2/d25/d1c/d39/d70/d91/de7/de9 0 2024-04-27T04:10:18.999 INFO:tasks.workunit.client.1.smithi181.stdout:3/903: rename d0/d6c/d52/l61 to d0/d9f/l120 0 2024-04-27T04:10:18.999 INFO:tasks.workunit.client.0.smithi162.stdout:4/714: fdatasync d2/d25/d1c/d39/d3b/d47/d9c/fb4 0 2024-04-27T04:10:19.003 INFO:tasks.workunit.client.1.smithi181.stdout:6/838: link d7/d31/f37 d7/dc/d1c/d34/d3f/db2/f119 0 2024-04-27T04:10:19.004 INFO:tasks.workunit.client.1.smithi181.stdout:3/904: dread d0/f14 [0,4194304] 0 2024-04-27T04:10:19.008 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:10:19.009+0000 7f06a4268700 1 -- 172.21.15.162:0/1347577413 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f069c115aa0 con 0x7f069c066e70 2024-04-27T04:10:19.008 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:10:19.010+0000 7f0692ffd700 1 -- 172.21.15.162:0/1347577413 <== mon.0 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+846 (secure 0 0 0) 0x7f0698057610 con 0x7f069c066e70 2024-04-27T04:10:19.009 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-04-27T04:10:19.009 INFO:teuthology.orchestra.run.smithi162.stdout: "mon": { 2024-04-27T04:10:19.009 INFO:teuthology.orchestra.run.smithi162.stdout: "ceph version 17.2.7-907-g2be3efa9 (2be3efa97a7972f8e925bd899b474329979b74d6) quincy (stable)": 2 2024-04-27T04:10:19.009 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-04-27T04:10:19.009 INFO:teuthology.orchestra.run.smithi162.stdout: "mgr": { 2024-04-27T04:10:19.009 INFO:teuthology.orchestra.run.smithi162.stdout: "ceph version 17.2.7-907-g2be3efa9 (2be3efa97a7972f8e925bd899b474329979b74d6) quincy (stable)": 1, 2024-04-27T04:10:19.009 INFO:teuthology.orchestra.run.smithi162.stdout: "ceph version 19.0.0-3253-g4e213a58 (4e213a58e787437ff508e6c890ccd5fd32dc96b5) squid (dev)": 1 2024-04-27T04:10:19.009 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-04-27T04:10:19.010 INFO:teuthology.orchestra.run.smithi162.stdout: "osd": { 2024-04-27T04:10:19.010 INFO:teuthology.orchestra.run.smithi162.stdout: "ceph version 17.2.7-907-g2be3efa9 (2be3efa97a7972f8e925bd899b474329979b74d6) quincy (stable)": 6 2024-04-27T04:10:19.010 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-04-27T04:10:19.010 INFO:teuthology.orchestra.run.smithi162.stdout: "mds": { 2024-04-27T04:10:19.010 INFO:teuthology.orchestra.run.smithi162.stdout: "ceph version 17.2.7-907-g2be3efa9 (2be3efa97a7972f8e925bd899b474329979b74d6) quincy (stable)": 4 2024-04-27T04:10:19.010 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-04-27T04:10:19.010 INFO:teuthology.orchestra.run.smithi162.stdout: "overall": { 2024-04-27T04:10:19.010 INFO:teuthology.orchestra.run.smithi162.stdout: "ceph version 17.2.7-907-g2be3efa9 (2be3efa97a7972f8e925bd899b474329979b74d6) quincy (stable)": 13, 2024-04-27T04:10:19.010 INFO:teuthology.orchestra.run.smithi162.stdout: "ceph version 19.0.0-3253-g4e213a58 (4e213a58e787437ff508e6c890ccd5fd32dc96b5) squid (dev)": 1 2024-04-27T04:10:19.010 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-04-27T04:10:19.011 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-04-27T04:10:19.011 INFO:tasks.workunit.client.0.smithi162.stdout:1/703: creat d0/d11/d3e/fec x:0 0 0 2024-04-27T04:10:19.011 INFO:tasks.workunit.client.1.smithi181.stdout:6/839: symlink d7/d27/l11a 0 2024-04-27T04:10:19.011 INFO:tasks.workunit.client.1.smithi181.stdout:3/905: dwrite d0/d6c/d52/d44/d56/f112 [0,4194304] 0 2024-04-27T04:10:19.012 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:10:19.013+0000 7f06a4268700 1 -- 172.21.15.162:0/1347577413 >> [v2:172.21.15.181:6828/4165336392,v1:172.21.15.181:6829/4165336392] conn(0x7f068806bd00 msgr2=0x7f068806e1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:10:19.012 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:10:19.013+0000 7f06a4268700 1 --2- 172.21.15.162:0/1347577413 >> [v2:172.21.15.181:6828/4165336392,v1:172.21.15.181:6829/4165336392] conn(0x7f068806bd00 0x7f068806e1b0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f068c00b0d0 tx=0x7f068c009e30 comp rx=0 tx=0).stop 2024-04-27T04:10:19.012 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:10:19.013+0000 7f06a4268700 1 -- 172.21.15.162:0/1347577413 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f069c066e70 msgr2=0x7f069c0fd230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:10:19.012 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:10:19.013+0000 7f06a4268700 1 --2- 172.21.15.162:0/1347577413 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f069c066e70 0x7f069c0fd230 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f069800eb70 tx=0x7f069800b040 comp rx=0 tx=0).stop 2024-04-27T04:10:19.012 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:10:19.013+0000 7f06a4268700 1 -- 172.21.15.162:0/1347577413 shutdown_connections 2024-04-27T04:10:19.012 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:10:19.013+0000 7f06a4268700 1 --2- 172.21.15.162:0/1347577413 >> [v2:172.21.15.181:6828/4165336392,v1:172.21.15.181:6829/4165336392] conn(0x7f068806bd00 0x7f068806e1b0 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:10:19.013 INFO:tasks.workunit.client.0.smithi162.stdout:3/760: write d1/d18/d23/d34/dad/dc5/d109/f101 [2617239,3402] 0 2024-04-27T04:10:19.014 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:10:19.013+0000 7f06a4268700 1 --2- 172.21.15.162:0/1347577413 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f069c066e70 0x7f069c0fd230 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:10:19.014 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:10:19.013+0000 7f06a4268700 1 --2- 172.21.15.162:0/1347577413 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f069c0fff70 0x7f069c0fb930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:10:19.014 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:10:19.013+0000 7f06a4268700 1 -- 172.21.15.162:0/1347577413 >> 172.21.15.162:0/1347577413 conn(0x7f069c0f7420 msgr2=0x7f069c0f7860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:10:19.014 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:10:19.013+0000 7f06a4268700 1 -- 172.21.15.162:0/1347577413 shutdown_connections 2024-04-27T04:10:19.014 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:10:19.013+0000 7f06a4268700 1 -- 172.21.15.162:0/1347577413 wait complete. 2024-04-27T04:10:19.014 INFO:tasks.workunit.client.1.smithi181.stdout:4/873: getdents d0/d15/de1 0 2024-04-27T04:10:19.015 INFO:tasks.workunit.client.1.smithi181.stdout:4/874: chown d0/d15/d25/d2a/d36/f42 451810 1 2024-04-27T04:10:19.019 INFO:tasks.workunit.client.0.smithi162.stdout:8/814: write d0/d10/d2b/d2e/f40 [853362,59268] 0 2024-04-27T04:10:19.022 INFO:tasks.workunit.client.0.smithi162.stdout:9/730: mknod d3/d13/d2d/d3b/d54/dda/cde 0 2024-04-27T04:10:19.023 INFO:tasks.workunit.client.1.smithi181.stdout:4/875: mknod d0/d15/d25/d2a/da0/dc7/d109/c11a 0 2024-04-27T04:10:19.025 INFO:tasks.workunit.client.1.smithi181.stdout:6/840: symlink d7/dc/d1c/d88/db3/d102/l11b 0 2024-04-27T04:10:19.026 INFO:tasks.workunit.client.1.smithi181.stdout:3/906: write d0/d6c/d52/dbb/de5/f75 [2855738,126220] 0 2024-04-27T04:10:19.026 INFO:tasks.workunit.client.0.smithi162.stdout:1/704: mknod d0/d7/d3c/d6e/ced 0 2024-04-27T04:10:19.027 INFO:tasks.workunit.client.1.smithi181.stdout:3/907: chown d0/d6c/d52/d44/dd5/c2b 8276166 1 2024-04-27T04:10:19.028 INFO:tasks.workunit.client.0.smithi162.stdout:1/705: truncate d0/d7/d3c/d54/fdf 332171 0 2024-04-27T04:10:19.029 INFO:tasks.workunit.client.0.smithi162.stdout:7/810: rename d3/d5/d5b/d83/de/d2b/d81/daa/dbd/dbe/dff/cd2 to d3/d5/d5b/d83/de/d2b/c10f 0 2024-04-27T04:10:19.030 INFO:tasks.workunit.client.1.smithi181.stdout:4/876: creat d0/d15/de1/f11b x:0 0 0 2024-04-27T04:10:19.031 INFO:tasks.workunit.client.0.smithi162.stdout:3/761: creat d1/d21/d47/d57/dd6/f10e x:0 0 0 2024-04-27T04:10:19.032 INFO:tasks.workunit.client.1.smithi181.stdout:6/841: unlink d7/dc/d1c/d34/d3f/db2/fd7 0 2024-04-27T04:10:19.033 INFO:tasks.workunit.client.0.smithi162.stdout:2/755: creat d0/d4/d17/d2b/d64/d96/fee x:0 0 0 2024-04-27T04:10:19.034 INFO:tasks.workunit.client.0.smithi162.stdout:4/715: creat d2/d54/dca/fea x:0 0 0 2024-04-27T04:10:19.035 INFO:tasks.workunit.client.1.smithi181.stdout:4/877: mknod d0/d15/d25/d2a/da0/dc7/c11c 0 2024-04-27T04:10:19.035 INFO:tasks.workunit.client.1.smithi181.stdout:4/878: dread - d0/d15/de/f117 zero size 2024-04-27T04:10:19.036 INFO:tasks.workunit.client.0.smithi162.stdout:9/731: rmdir d3/d13/d2d/d4f 39 2024-04-27T04:10:19.038 INFO:tasks.workunit.client.0.smithi162.stdout:6/739: creat d6/d1a/d34/d3a/fef x:0 0 0 2024-04-27T04:10:19.039 INFO:tasks.workunit.client.1.smithi181.stdout:3/908: rename d0/d6c/d52/d44/dd5/d58/de4/daf/f9d to d0/d9f/d110/f121 0 2024-04-27T04:10:19.040 INFO:tasks.workunit.client.0.smithi162.stdout:1/706: stat d0/d7/d2a/ldc 0 2024-04-27T04:10:19.040 INFO:tasks.workunit.client.0.smithi162.stdout:6/740: dread d6/d1a/d34/fc3 [0,4194304] 0 2024-04-27T04:10:19.041 INFO:tasks.workunit.client.0.smithi162.stdout:6/741: stat d6/f1e 0 2024-04-27T04:10:19.042 INFO:tasks.workunit.client.0.smithi162.stdout:6/742: read d6/db/f61 [1638689,9802] 0 2024-04-27T04:10:19.043 INFO:tasks.workunit.client.0.smithi162.stdout:6/743: dread - d6/db/d32/fd5 zero size 2024-04-27T04:10:19.043 INFO:tasks.workunit.client.1.smithi181.stdout:6/842: write d7/dc/d1e/d66/f90 [4056,14473] 0 2024-04-27T04:10:19.044 INFO:tasks.workunit.client.0.smithi162.stdout:7/811: fsync d3/d5/f5f 0 2024-04-27T04:10:19.049 INFO:tasks.workunit.client.0.smithi162.stdout:7/812: dread d3/d5/d9/fe7 [0,4194304] 0 2024-04-27T04:10:19.050 INFO:tasks.workunit.client.0.smithi162.stdout:6/744: dread d6/db/fb0 [0,4194304] 0 2024-04-27T04:10:19.051 INFO:tasks.workunit.client.0.smithi162.stdout:4/716: creat d2/da/dd/feb x:0 0 0 2024-04-27T04:10:19.051 INFO:tasks.workunit.client.0.smithi162.stdout:7/813: readlink d3/d5/d5b/d83/de/d2b/d81/le1 0 2024-04-27T04:10:19.053 INFO:tasks.workunit.client.0.smithi162.stdout:8/815: getdents d0/d10/d2b/d2d/d2f/d6b/dc7 0 2024-04-27T04:10:19.054 INFO:tasks.workunit.client.0.smithi162.stdout:7/814: write d3/d5/d5b/d83/de/d2b/d81/daa/deb/f106 [340742,93794] 0 2024-04-27T04:10:19.054 INFO:tasks.workunit.client.0.smithi162.stdout:8/816: stat d0/d10/d2b/d2d/d2f/d39/d7c/dc2 0 2024-04-27T04:10:19.055 INFO:tasks.workunit.client.0.smithi162.stdout:7/815: readlink d3/d5/d5b/d83/de/lf8 0 2024-04-27T04:10:19.056 INFO:tasks.workunit.client.0.smithi162.stdout:9/732: unlink d3/d11/d45/d4e/ldb 0 2024-04-27T04:10:19.057 INFO:tasks.workunit.client.0.smithi162.stdout:9/733: stat d3/d13/d2d/d3b/d54/l60 0 2024-04-27T04:10:19.057 INFO:tasks.workunit.client.1.smithi181.stdout:4/879: write d0/d15/d25/d2b/d6d/d82/fc3 [1629037,114515] 0 2024-04-27T04:10:19.057 INFO:tasks.workunit.client.0.smithi162.stdout:3/762: dread d1/d18/d23/d58/fb3 [0,4194304] 0 2024-04-27T04:10:19.058 INFO:tasks.workunit.client.0.smithi162.stdout:9/734: write d3/d13/d2d/d30/d7f/dc3/fc9 [56505,10633] 0 2024-04-27T04:10:19.059 INFO:tasks.workunit.client.1.smithi181.stdout:3/909: dread d0/d6c/d52/d44/dd5/d58/d64/dc6/fe6 [0,4194304] 0 2024-04-27T04:10:19.060 INFO:tasks.workunit.client.1.smithi181.stdout:6/843: link d7/dc/d1c/c57 d7/d27/c11c 0 2024-04-27T04:10:19.061 INFO:tasks.workunit.client.0.smithi162.stdout:5/782: rename l7 to d8/d9/d5b/df3/lfa 0 2024-04-27T04:10:19.064 INFO:tasks.workunit.client.1.smithi181.stdout:3/910: symlink d0/d6c/d66/dc2/de1/l122 0 2024-04-27T04:10:19.065 INFO:tasks.workunit.client.1.smithi181.stdout:4/880: read - d0/d15/de/d60/fc4 zero size 2024-04-27T04:10:19.067 INFO:tasks.workunit.client.1.smithi181.stdout:6/844: dwrite d7/f59 [0,4194304] 0 2024-04-27T04:10:19.067 INFO:tasks.workunit.client.0.smithi162.stdout:2/756: truncate d0/d4/d2a/d41/d77/d73/fd3 97224 0 2024-04-27T04:10:19.068 INFO:tasks.workunit.client.0.smithi162.stdout:4/717: creat d2/d54/fec x:0 0 0 2024-04-27T04:10:19.069 INFO:tasks.workunit.client.1.smithi181.stdout:3/911: fsync d0/d6c/d52/d44/d74/f9b 0 2024-04-27T04:10:19.070 INFO:tasks.workunit.client.1.smithi181.stdout:3/912: write d0/d9f/f7c [9159298,115877] 0 2024-04-27T04:10:19.073 INFO:tasks.workunit.client.0.smithi162.stdout:8/817: truncate d0/d10/d44/fbd 748583 0 2024-04-27T04:10:19.078 INFO:tasks.workunit.client.0.smithi162.stdout:1/707: creat d0/d11/d50/d53/d93/d96/daa/de9/fee x:0 0 0 2024-04-27T04:10:19.080 INFO:tasks.workunit.client.0.smithi162.stdout:1/708: dread - d0/d11/d5b/fad zero size 2024-04-27T04:10:19.080 INFO:tasks.workunit.client.1.smithi181.stdout:3/913: rmdir d0/d6c/d52/d44/d74 39 2024-04-27T04:10:19.080 INFO:tasks.workunit.client.1.smithi181.stdout:6/845: mknod d7/dc/dc3/c11d 0 2024-04-27T04:10:19.082 INFO:tasks.workunit.client.0.smithi162.stdout:5/783: creat d8/d1a/ffb x:0 0 0 2024-04-27T04:10:19.084 INFO:tasks.workunit.client.1.smithi181.stdout:3/914: symlink d0/d6c/d52/d44/dd5/d58/d64/dc6/ddd/l123 0 2024-04-27T04:10:19.090 INFO:tasks.workunit.client.1.smithi181.stdout:4/881: rename d0/d15/d25/d2a/l96 to d0/d15/de/d19/d77/l11d 0 2024-04-27T04:10:19.093 INFO:tasks.workunit.client.1.smithi181.stdout:4/882: chown d0/d15/dd2/deb/cd8 517 1 2024-04-27T04:10:19.094 INFO:tasks.workunit.client.0.smithi162.stdout:4/718: fdatasync d2/d25/d1c/d39/d90/dad/f9a 0 2024-04-27T04:10:19.094 INFO:tasks.workunit.client.0.smithi162.stdout:2/757: dread d0/d4/d17/d2b/f95 [4194304,4194304] 0 2024-04-27T04:10:19.094 INFO:tasks.workunit.client.0.smithi162.stdout:4/719: chown d2/d25/d1c/d39/d5e/la0 20 1 2024-04-27T04:10:19.097 INFO:tasks.workunit.client.1.smithi181.stdout:4/883: mkdir d0/d15/de/d19/d101/d11e 0 2024-04-27T04:10:19.098 INFO:tasks.workunit.client.1.smithi181.stdout:3/915: symlink d0/d6c/l124 0 2024-04-27T04:10:19.100 INFO:tasks.workunit.client.0.smithi162.stdout:9/735: dread d3/f4 [0,4194304] 0 2024-04-27T04:10:19.100 INFO:tasks.workunit.client.0.smithi162.stdout:1/709: chown d0/d7/f27 1 1 2024-04-27T04:10:19.102 INFO:tasks.workunit.client.1.smithi181.stdout:6/846: dwrite d7/dc/d1c/d34/d3f/d7e/d8a/f7b [0,4194304] 0 2024-04-27T04:10:19.102 INFO:tasks.workunit.client.0.smithi162.stdout:4/720: mknod d2/d25/d1c/d39/d90/ced 0 2024-04-27T04:10:19.105 INFO:tasks.workunit.client.0.smithi162.stdout:7/816: creat d3/d5/d5b/d83/de/f110 x:0 0 0 2024-04-27T04:10:19.108 INFO:tasks.workunit.client.1.smithi181.stdout:3/916: dread d0/f57 [0,4194304] 0 2024-04-27T04:10:19.110 INFO:tasks.workunit.client.1.smithi181.stdout:4/884: rename d0/d15/de/d19/d101/d10f to d0/d15/d25/d2a/d61/d11f 0 2024-04-27T04:10:19.111 INFO:tasks.workunit.client.0.smithi162.stdout:6/745: dwrite d6/d1a/d34/d7f/d80/db3/fbb [0,4194304] 0 2024-04-27T04:10:19.112 INFO:tasks.workunit.client.1.smithi181.stdout:3/917: creat d0/de/d7d/d116/f125 x:0 0 0 2024-04-27T04:10:19.117 INFO:tasks.workunit.client.1.smithi181.stdout:3/918: mkdir d0/de/d7d/d126 0 2024-04-27T04:10:19.118 INFO:tasks.workunit.client.1.smithi181.stdout:3/919: readlink d0/d6c/l4a 0 2024-04-27T04:10:19.118 INFO:tasks.workunit.client.0.smithi162.stdout:5/784: dread d8/d9/d55/f65 [0,4194304] 0 2024-04-27T04:10:19.118 INFO:tasks.workunit.client.0.smithi162.stdout:8/818: dwrite d0/d10/d25/d65/fad [0,4194304] 0 2024-04-27T04:10:19.118 INFO:tasks.workunit.client.1.smithi181.stdout:3/920: fdatasync d0/d6c/f79 0 2024-04-27T04:10:19.119 INFO:tasks.workunit.client.0.smithi162.stdout:9/736: rename d3/d11/f4d to d3/d23/d7c/fdf 0 2024-04-27T04:10:19.122 INFO:tasks.workunit.client.1.smithi181.stdout:4/885: link d0/d15/d5c/d69/c63 d0/d15/dd2/deb/db9/c120 0 2024-04-27T04:10:19.123 INFO:tasks.workunit.client.0.smithi162.stdout:5/785: dread d8/d2c/f33 [0,4194304] 0 2024-04-27T04:10:19.124 INFO:tasks.workunit.client.1.smithi181.stdout:3/921: dread d0/d6c/d52/dbb/d98/fac [0,4194304] 0 2024-04-27T04:10:19.125 INFO:tasks.workunit.client.0.smithi162.stdout:6/746: mkdir d6/db/d32/df0 0 2024-04-27T04:10:19.125 INFO:tasks.workunit.client.0.smithi162.stdout:4/721: dread d2/d25/d1c/d39/d3b/d6e/d6f/fc7 [0,4194304] 0 2024-04-27T04:10:19.128 INFO:tasks.workunit.client.0.smithi162.stdout:4/722: readlink d2/d25/d1c/d39/d3b/d6e/l74 0 2024-04-27T04:10:19.132 INFO:tasks.workunit.client.1.smithi181.stdout:3/922: creat d0/d6c/d52/d44/dd5/d58/d64/dc6/ddd/f127 x:0 0 0 2024-04-27T04:10:19.133 INFO:tasks.workunit.client.0.smithi162.stdout:1/710: dwrite d0/d11/d3e/f6c [0,4194304] 0 2024-04-27T04:10:19.135 INFO:tasks.workunit.client.1.smithi181.stdout:4/886: link d0/ld6 d0/d15/d5c/d69/l121 0 2024-04-27T04:10:19.137 INFO:tasks.workunit.client.0.smithi162.stdout:5/786: write d8/d2c/fb1 [481955,42700] 0 2024-04-27T04:10:19.137 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:10:18 smithi162 ceph-mon[26841]: from='client.14640 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-27T04:10:19.137 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:10:18 smithi162 ceph-mon[26841]: pgmap v9: 65 pgs: 65 active+clean; 3.3 GiB data, 11 GiB used, 525 GiB / 536 GiB avail; 81 MiB/s rd, 200 MiB/s wr, 453 op/s 2024-04-27T04:10:19.137 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:10:18 smithi162 ceph-mon[26841]: from='client.14644 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-27T04:10:19.138 INFO:tasks.workunit.client.0.smithi162.stdout:9/737: dwrite d3/d23/d1e/d43/f48 [0,4194304] 0 2024-04-27T04:10:19.138 INFO:tasks.workunit.client.0.smithi162.stdout:5/787: dread - d8/de3/deb/fe7 zero size 2024-04-27T04:10:19.139 INFO:tasks.workunit.client.0.smithi162.stdout:6/747: rename d6/d1a/d34/d7f/d80/c9c to d6/d1a/d34/d7f/cf1 0 2024-04-27T04:10:19.142 INFO:tasks.workunit.client.1.smithi181.stdout:4/887: rename d0/d15/d25/d2a/d36/f51 to d0/d15/d25/d2a/da0/dc7/d109/d7c/db5/f122 0 2024-04-27T04:10:19.143 INFO:tasks.workunit.client.1.smithi181.stdout:4/888: readlink d0/d15/de/d60/led 0 2024-04-27T04:10:19.150 INFO:tasks.workunit.client.1.smithi181.stdout:4/889: dwrite d0/d15/d25/d2b/d6d/d82/fc3 [0,4194304] 0 2024-04-27T04:10:19.152 INFO:tasks.workunit.client.0.smithi162.stdout:9/738: rename d3/d13/fb7 to d3/d13/d2d/d3b/d54/fe0 0 2024-04-27T04:10:19.152 INFO:tasks.workunit.client.0.smithi162.stdout:8/819: write d0/de/d6e/f51 [110993,115472] 0 2024-04-27T04:10:19.153 INFO:tasks.workunit.client.0.smithi162.stdout:6/748: mknod d6/d1a/d34/d7f/cf2 0 2024-04-27T04:10:19.154 INFO:tasks.workunit.client.0.smithi162.stdout:8/820: write d0/d10/f6a [5366014,78414] 0 2024-04-27T04:10:19.155 INFO:tasks.workunit.client.0.smithi162.stdout:6/749: fdatasync d6/d1a/d34/d7f/d80/db3/fbb 0 2024-04-27T04:10:19.156 INFO:tasks.workunit.client.0.smithi162.stdout:7/817: truncate d3/d5/d5b/d83/f33 1931005 0 2024-04-27T04:10:19.156 INFO:tasks.workunit.client.0.smithi162.stdout:6/750: chown d6/db/d32/f5b 22 1 2024-04-27T04:10:19.158 INFO:tasks.workunit.client.0.smithi162.stdout:8/821: write d0/d10/d2b/d2d/d2f/d39/f59 [1935830,74649] 0 2024-04-27T04:10:19.158 INFO:tasks.workunit.client.1.smithi181.stdout:3/923: write d0/d6c/d52/f47 [1538684,57184] 0 2024-04-27T04:10:19.159 INFO:tasks.workunit.client.0.smithi162.stdout:5/788: dread - d8/d1a/d23/d40/fcd zero size 2024-04-27T04:10:19.164 INFO:tasks.workunit.client.0.smithi162.stdout:6/751: mknod d6/d1a/d34/d7f/cf3 0 2024-04-27T04:10:19.165 INFO:tasks.workunit.client.1.smithi181.stdout:3/924: rename d0/d6c/d52/d44/d56/f63 to d0/de/d7d/dbc/f128 0 2024-04-27T04:10:19.165 INFO:tasks.workunit.client.0.smithi162.stdout:6/752: chown d6/d1a/d34/c94 114279 1 2024-04-27T04:10:19.166 INFO:tasks.workunit.client.0.smithi162.stdout:9/739: mkdir d3/de1 0 2024-04-27T04:10:19.167 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:10:19.168+0000 7ffa31445700 1 -- 172.21.15.162:0/435063146 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ffa2c0fbce0 msgr2=0x7ffa2c0fc0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:10:19.167 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:10:19.168+0000 7ffa31445700 1 --2- 172.21.15.162:0/435063146 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ffa2c0fbce0 0x7ffa2c0fc0f0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7ffa1c004940 tx=0x7ffa1c009e30 comp rx=0 tx=0).stop 2024-04-27T04:10:19.167 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:10:19.168+0000 7ffa31445700 1 -- 172.21.15.162:0/435063146 shutdown_connections 2024-04-27T04:10:19.167 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:10:19.168+0000 7ffa31445700 1 --2- 172.21.15.162:0/435063146 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ffa2c0fcf90 0x7ffa2c0fd400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:10:19.167 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:10:19.168+0000 7ffa31445700 1 --2- 172.21.15.162:0/435063146 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ffa2c0fbce0 0x7ffa2c0fc0f0 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:10:19.167 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:10:19.168+0000 7ffa31445700 1 -- 172.21.15.162:0/435063146 >> 172.21.15.162:0/435063146 conn(0x7ffa2c0f7440 msgr2=0x7ffa2c0f9870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:10:19.167 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:10:19.168+0000 7ffa31445700 1 -- 172.21.15.162:0/435063146 shutdown_connections 2024-04-27T04:10:19.167 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:10:19.168+0000 7ffa31445700 1 -- 172.21.15.162:0/435063146 wait complete. 2024-04-27T04:10:19.168 INFO:tasks.workunit.client.0.smithi162.stdout:6/753: truncate d6/d1a/d34/d7f/d80/fc7 1011607 0 2024-04-27T04:10:19.168 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:10:19.169+0000 7ffa31445700 1 Processor -- start 2024-04-27T04:10:19.168 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:10:19.169+0000 7ffa31445700 1 -- start start 2024-04-27T04:10:19.169 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:10:19.170+0000 7ffa31445700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ffa2c0fbce0 0x7ffa2c10c8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:10:19.169 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:10:19.170+0000 7ffa31445700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ffa2c0fcf90 0x7ffa2c10cde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:10:19.169 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:10:19.170+0000 7ffa31445700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa2c10d3e0 con 0x7ffa2c0fcf90 2024-04-27T04:10:19.169 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:10:19.170+0000 7ffa31445700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa2c10dc30 con 0x7ffa2c0fbce0 2024-04-27T04:10:19.169 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:10:19.170+0000 7ffa2a7fc700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ffa2c0fcf90 0x7ffa2c10cde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:10:19.169 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:10:19.170+0000 7ffa2affd700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ffa2c0fbce0 0x7ffa2c10c8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:10:19.169 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:10:19.170+0000 7ffa2a7fc700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ffa2c0fcf90 0x7ffa2c10cde0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:51872/0 (socket says 172.21.15.162:51872) 2024-04-27T04:10:19.169 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:10:19.170+0000 7ffa2a7fc700 1 -- 172.21.15.162:0/112660346 learned_addr learned my addr 172.21.15.162:0/112660346 (peer_addr_for_me v2:172.21.15.162:0/0) 2024-04-27T04:10:19.169 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:10:19.170+0000 7ffa2a7fc700 1 -- 172.21.15.162:0/112660346 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ffa2c0fbce0 msgr2=0x7ffa2c10c8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:10:19.170 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:10:19.170+0000 7ffa2a7fc700 1 --2- 172.21.15.162:0/112660346 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ffa2c0fbce0 0x7ffa2c10c8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:10:19.170 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:10:19.170+0000 7ffa2a7fc700 1 -- 172.21.15.162:0/112660346 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffa1c005040 con 0x7ffa2c0fcf90 2024-04-27T04:10:19.170 INFO:tasks.workunit.client.0.smithi162.stdout:5/789: dread - d8/d9/d5b/d96/fc2 zero size 2024-04-27T04:10:19.170 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:10:19.171+0000 7ffa2a7fc700 1 --2- 172.21.15.162:0/112660346 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ffa2c0fcf90 0x7ffa2c10cde0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7ffa20010d50 tx=0x7ffa2000b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:10:19.171 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:10:19.171+0000 7ffa137fe700 1 -- 172.21.15.162:0/112660346 <== mon.0 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ffa2000ed60 con 0x7ffa2c0fcf90 2024-04-27T04:10:19.171 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:10:19.171+0000 7ffa31445700 1 -- 172.21.15.162:0/112660346 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffa2c10deb0 con 0x7ffa2c0fcf90 2024-04-27T04:10:19.171 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:10:19.171+0000 7ffa137fe700 1 -- 172.21.15.162:0/112660346 <== mon.0 v2:172.21.15.162:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7ffa200090d0 con 0x7ffa2c0fcf90 2024-04-27T04:10:19.171 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:10:19.171+0000 7ffa137fe700 1 -- 172.21.15.162:0/112660346 <== mon.0 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ffa2000ed60 con 0x7ffa2c0fcf90 2024-04-27T04:10:19.171 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:10:19.171+0000 7ffa31445700 1 -- 172.21.15.162:0/112660346 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffa2c10e3f0 con 0x7ffa2c0fcf90 2024-04-27T04:10:19.171 INFO:tasks.workunit.client.1.smithi181.stdout:3/925: fsync d0/d6c/d66/d88/fa8 0 2024-04-27T04:10:19.172 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:10:18 smithi181 ceph-mon[33774]: from='client.14640 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-27T04:10:19.172 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:10:18 smithi181 ceph-mon[33774]: pgmap v9: 65 pgs: 65 active+clean; 3.3 GiB data, 11 GiB used, 525 GiB / 536 GiB avail; 81 MiB/s rd, 200 MiB/s wr, 453 op/s 2024-04-27T04:10:19.172 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:10:18 smithi181 ceph-mon[33774]: from='client.14644 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-27T04:10:19.173 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:10:19.173+0000 7ffa31445700 1 -- 172.21.15.162:0/112660346 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffa2c05f5c0 con 0x7ffa2c0fcf90 2024-04-27T04:10:19.173 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:10:19.173+0000 7ffa137fe700 1 -- 172.21.15.162:0/112660346 <== mon.0 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 24) v1 ==== 89652+0+0 (secure 0 0 0) 0x7ffa20018030 con 0x7ffa2c0fcf90 2024-04-27T04:10:19.173 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:10:19.174+0000 7ffa137fe700 1 --2- 172.21.15.162:0/112660346 >> [v2:172.21.15.181:6828/4165336392,v1:172.21.15.181:6829/4165336392] conn(0x7ffa1406bd60 0x7ffa1406e210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:10:19.173 INFO:tasks.workunit.client.0.smithi162.stdout:4/723: dread d2/d25/d1c/d39/d90/dad/f81 [0,4194304] 0 2024-04-27T04:10:19.173 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:10:19.174+0000 7ffa2affd700 1 --2- 172.21.15.162:0/112660346 >> [v2:172.21.15.181:6828/4165336392,v1:172.21.15.181:6829/4165336392] conn(0x7ffa1406bd60 0x7ffa1406e210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:10:19.174 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:10:19.174+0000 7ffa137fe700 1 -- 172.21.15.162:0/112660346 <== mon.0 v2:172.21.15.162:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5808+0+0 (secure 0 0 0) 0x7ffa20016070 con 0x7ffa2c0fcf90 2024-04-27T04:10:19.174 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:10:19.175+0000 7ffa2affd700 1 --2- 172.21.15.162:0/112660346 >> [v2:172.21.15.181:6828/4165336392,v1:172.21.15.181:6829/4165336392] conn(0x7ffa1406bd60 0x7ffa1406e210 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7ffa1c00b0d0 tx=0x7ffa1c009e30 comp rx=0 tx=0).ready entity=mgr.24367 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:10:19.179 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:10:19.181+0000 7ffa137fe700 1 -- 172.21.15.162:0/112660346 <== mon.0 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186652 (secure 0 0 0) 0x7ffa20059fa0 con 0x7ffa2c0fcf90 2024-04-27T04:10:19.180 INFO:tasks.workunit.client.0.smithi162.stdout:1/711: dwrite d0/d11/d50/d53/d93/d96/fd0 [0,4194304] 0 2024-04-27T04:10:19.180 INFO:tasks.workunit.client.0.smithi162.stdout:6/754: fdatasync d6/d1a/d34/d3a/d68/d6c/f82 0 2024-04-27T04:10:19.185 INFO:tasks.workunit.client.0.smithi162.stdout:9/740: write d3/d13/f26 [2343736,110557] 0 2024-04-27T04:10:19.189 INFO:tasks.workunit.client.0.smithi162.stdout:5/790: rename d8/d2c/fe2 to d8/de3/deb/ffc 0 2024-04-27T04:10:19.189 INFO:tasks.workunit.client.0.smithi162.stdout:9/741: write d3/d23/f88 [958850,45817] 0 2024-04-27T04:10:19.191 INFO:tasks.workunit.client.0.smithi162.stdout:8/822: dwrite d0/d10/d2b/d2d/d2f/d39/fe2 [0,4194304] 0 2024-04-27T04:10:19.194 INFO:tasks.workunit.client.0.smithi162.stdout:9/742: write d3/d11/d45/d4e/fd7 [713958,122395] 0 2024-04-27T04:10:19.196 INFO:tasks.workunit.client.1.smithi181.stdout:3/926: dwrite d0/f57 [0,4194304] 0 2024-04-27T04:10:19.197 INFO:tasks.workunit.client.1.smithi181.stdout:4/890: dwrite d0/d15/d25/d2a/d36/f42 [4194304,4194304] 0 2024-04-27T04:10:19.198 INFO:tasks.workunit.client.0.smithi162.stdout:6/755: rename d6/d1a/d34/d50/l9b to d6/d1a/d34/d3a/d68/d6c/de3/lf4 0 2024-04-27T04:10:19.203 INFO:tasks.workunit.client.0.smithi162.stdout:8/823: dread d0/d10/d2b/d2d/d2f/d39/f59 [0,4194304] 0 2024-04-27T04:10:19.208 INFO:tasks.workunit.client.0.smithi162.stdout:4/724: dwrite d2/d25/d1c/f55 [4194304,4194304] 0 2024-04-27T04:10:19.212 INFO:tasks.workunit.client.1.smithi181.stdout:4/891: rmdir d0/d15/d25/d2a 39 2024-04-27T04:10:19.214 INFO:tasks.workunit.client.1.smithi181.stdout:3/927: dread d0/d6c/d52/f47 [0,4194304] 0 2024-04-27T04:10:19.215 INFO:tasks.workunit.client.1.smithi181.stdout:3/928: read - d0/d6c/d52/dbb/fd6 zero size 2024-04-27T04:10:19.216 INFO:tasks.workunit.client.0.smithi162.stdout:8/824: readlink d0/l9 0 2024-04-27T04:10:19.216 INFO:tasks.workunit.client.1.smithi181.stdout:4/892: fdatasync d0/d15/de/d19/d8a/fbe 0 2024-04-27T04:10:19.218 INFO:tasks.workunit.client.0.smithi162.stdout:9/743: mknod d3/d13/d2d/d3b/d7a/dd6/ce2 0 2024-04-27T04:10:19.219 INFO:tasks.workunit.client.0.smithi162.stdout:4/725: rename d2/d25/d1c/d39/d3b/d47/c99 to d2/d25/d1c/d39/d3b/d6e/d6f/cee 0 2024-04-27T04:10:19.220 INFO:tasks.workunit.client.0.smithi162.stdout:4/726: chown d2/d25/d1c/d39/d5e/fd8 2840858 1 2024-04-27T04:10:19.221 INFO:tasks.workunit.client.1.smithi181.stdout:4/893: mkdir d0/d15/de/d19/d123 0 2024-04-27T04:10:19.222 INFO:tasks.workunit.client.1.smithi181.stdout:3/929: creat d0/de/f129 x:0 0 0 2024-04-27T04:10:19.225 INFO:tasks.workunit.client.1.smithi181.stdout:3/930: creat d0/de/d7d/df5/f12a x:0 0 0 2024-04-27T04:10:19.226 INFO:tasks.workunit.client.1.smithi181.stdout:4/894: mkdir d0/d15/de/d19/d77/d124 0 2024-04-27T04:10:19.227 INFO:tasks.workunit.client.0.smithi162.stdout:6/756: getdents d6/db/dc0 0 2024-04-27T04:10:19.227 INFO:tasks.workunit.client.0.smithi162.stdout:6/757: chown d6/d1a/d34/f54 133103 1 2024-04-27T04:10:19.228 INFO:tasks.workunit.client.0.smithi162.stdout:9/744: truncate d3/d23/d5d/f9f 911026 0 2024-04-27T04:10:19.229 INFO:tasks.workunit.client.1.smithi181.stdout:4/895: mkdir d0/d15/d76/db1/d125 0 2024-04-27T04:10:19.231 INFO:tasks.workunit.client.0.smithi162.stdout:6/758: creat d6/db/dc0/ff5 x:0 0 0 2024-04-27T04:10:19.232 INFO:tasks.workunit.client.1.smithi181.stdout:3/931: dwrite d0/d6c/d66/dc2/fe3 [0,4194304] 0 2024-04-27T04:10:19.233 INFO:tasks.workunit.client.0.smithi162.stdout:6/759: dread - d6/d1a/d34/d7f/fd1 zero size 2024-04-27T04:10:19.233 INFO:tasks.workunit.client.0.smithi162.stdout:8/825: link d0/d10/d2b/d2d/d2f/c4e d0/d10/d25/d65/c108 0 2024-04-27T04:10:19.234 INFO:tasks.workunit.client.0.smithi162.stdout:8/826: stat d0/de/d6e/d71/d86/ddc/ce8 0 2024-04-27T04:10:19.234 INFO:tasks.workunit.client.0.smithi162.stdout:6/760: dread - d6/d1a/d34/d50/fd4 zero size 2024-04-27T04:10:19.237 INFO:tasks.workunit.client.1.smithi181.stdout:4/896: rename d0/d15/d25/d2b/d6d/d93 to d0/d126 0 2024-04-27T04:10:19.241 INFO:tasks.workunit.client.0.smithi162.stdout:8/827: creat d0/d10/f109 x:0 0 0 2024-04-27T04:10:19.242 INFO:tasks.workunit.client.1.smithi181.stdout:4/897: creat d0/d15/d25/d40/f127 x:0 0 0 2024-04-27T04:10:19.242 INFO:tasks.workunit.client.0.smithi162.stdout:6/761: dwrite d6/d1a/fcc [0,4194304] 0 2024-04-27T04:10:19.244 INFO:tasks.workunit.client.1.smithi181.stdout:4/898: creat d0/d15/d76/db1/f128 x:0 0 0 2024-04-27T04:10:19.244 INFO:tasks.workunit.client.0.smithi162.stdout:4/727: write d2/d25/d1c/d39/f5f [410972,9205] 0 2024-04-27T04:10:19.244 INFO:tasks.workunit.client.0.smithi162.stdout:6/762: fdatasync d6/d1a/d34/f54 0 2024-04-27T04:10:19.247 INFO:tasks.workunit.client.1.smithi181.stdout:4/899: truncate d0/d15/d25/d2a/d36/db8/dca/ff7 151440 0 2024-04-27T04:10:19.248 INFO:tasks.workunit.client.1.smithi181.stdout:4/900: dread - d0/d15/d25/d40/f127 zero size 2024-04-27T04:10:19.249 INFO:tasks.workunit.client.0.smithi162.stdout:5/791: dwrite d8/d1a/d7d/f91 [0,4194304] 0 2024-04-27T04:10:19.249 INFO:tasks.workunit.client.0.smithi162.stdout:8/828: fdatasync d0/de/d6e/d71/d86/dd9/ff3 0 2024-04-27T04:10:19.252 INFO:tasks.workunit.client.0.smithi162.stdout:9/745: dread d3/d11/d45/f55 [0,4194304] 0 2024-04-27T04:10:19.252 INFO:tasks.workunit.client.0.smithi162.stdout:8/829: chown d0/d10/d2b/d2d/d2f/d39/d7c/dc2 9046 1 2024-04-27T04:10:19.254 INFO:tasks.workunit.client.0.smithi162.stdout:4/728: fsync d2/d25/d1c/d39/d3b/d6e/d6f/fc6 0 2024-04-27T04:10:19.256 INFO:tasks.workunit.client.1.smithi181.stdout:4/901: dread d0/d15/d25/d2a/da0/dc7/d109/d7c/fdd [0,4194304] 0 2024-04-27T04:10:19.256 INFO:tasks.workunit.client.0.smithi162.stdout:5/792: mknod d8/d1a/d23/dbc/d58/cfd 0 2024-04-27T04:10:19.258 INFO:tasks.workunit.client.0.smithi162.stdout:8/830: rmdir d0/de/d6e/d71 39 2024-04-27T04:10:19.259 INFO:tasks.workunit.client.0.smithi162.stdout:4/729: mkdir d2/d25/d1c/d39/def 0 2024-04-27T04:10:19.260 INFO:tasks.workunit.client.0.smithi162.stdout:6/763: creat d6/d1a/d34/d7b/ff6 x:0 0 0 2024-04-27T04:10:19.261 INFO:tasks.workunit.client.0.smithi162.stdout:5/793: truncate d8/d9/f16 1997189 0 2024-04-27T04:10:19.263 INFO:tasks.workunit.client.0.smithi162.stdout:8/831: symlink d0/d10/d2b/d2d/d2f/d6b/l10a 0 2024-04-27T04:10:19.263 INFO:tasks.workunit.client.0.smithi162.stdout:5/794: chown d8/de3/deb/d5c/f9e 10021 1 2024-04-27T04:10:19.263 INFO:tasks.workunit.client.0.smithi162.stdout:6/764: symlink d6/d1a/d34/d7f/d80/db3/lf7 0 2024-04-27T04:10:19.264 INFO:tasks.workunit.client.0.smithi162.stdout:8/832: write d0/de/d6e/fdf [508909,99233] 0 2024-04-27T04:10:19.264 INFO:tasks.workunit.client.0.smithi162.stdout:4/730: truncate d2/d25/d1c/d39/d3b/d47/fc3 1781183 0 2024-04-27T04:10:19.270 INFO:tasks.workunit.client.0.smithi162.stdout:5/795: truncate d8/d1a/f17 2328670 0 2024-04-27T04:10:19.273 INFO:tasks.workunit.client.0.smithi162.stdout:8/833: unlink d0/de/d6e/d71/d86/ddc/ce8 0 2024-04-27T04:10:19.274 INFO:tasks.workunit.client.0.smithi162.stdout:4/731: dwrite d2/da/fdd [4194304,4194304] 0 2024-04-27T04:10:19.275 INFO:tasks.workunit.client.0.smithi162.stdout:5/796: fdatasync d8/d9/fc 0 2024-04-27T04:10:19.278 INFO:tasks.workunit.client.0.smithi162.stdout:5/797: stat d8/d1a/d23/dbc/d66/d88/dee/ff2 0 2024-04-27T04:10:19.281 INFO:tasks.workunit.client.0.smithi162.stdout:8/834: dread d0/d10/d44/fbd [0,4194304] 0 2024-04-27T04:10:19.285 INFO:tasks.workunit.client.0.smithi162.stdout:5/798: symlink d8/d9/db3/lfe 0 2024-04-27T04:10:19.286 INFO:tasks.workunit.client.0.smithi162.stdout:5/799: chown d8/d1a/d23/dbc/d66/d88/dee/cf7 26496 1 2024-04-27T04:10:19.287 INFO:tasks.workunit.client.0.smithi162.stdout:8/835: creat d0/d10/d44/f10b x:0 0 0 2024-04-27T04:10:19.289 INFO:tasks.workunit.client.0.smithi162.stdout:5/800: write d8/d1a/d23/f50 [4125733,76051] 0 2024-04-27T04:10:19.292 INFO:tasks.workunit.client.0.smithi162.stdout:5/801: mkdir d8/de3/dff 0 2024-04-27T04:10:19.293 INFO:tasks.workunit.client.0.smithi162.stdout:5/802: readlink d8/d9/l54 0 2024-04-27T04:10:19.296 INFO:tasks.workunit.client.0.smithi162.stdout:5/803: rmdir d8/d1a/d7d 39 2024-04-27T04:10:19.297 INFO:tasks.workunit.client.0.smithi162.stdout:8/836: link d0/d10/d2b/d2d/d2f/d6b/l10a d0/d10/d2b/d2e/dca/l10c 0 2024-04-27T04:10:19.300 INFO:tasks.workunit.client.0.smithi162.stdout:4/732: rename d2/d25/d1c/d39/d70/d91 to d2/d25/d1c/d39/d70/dcd/df0 0 2024-04-27T04:10:19.301 INFO:tasks.workunit.client.0.smithi162.stdout:5/804: fsync d8/d2c/fb6 0 2024-04-27T04:10:19.302 INFO:tasks.workunit.client.0.smithi162.stdout:6/765: write d6/db/d32/f38 [109925,130178] 0 2024-04-27T04:10:19.304 INFO:tasks.workunit.client.0.smithi162.stdout:8/837: rename d0/d10/d2b/d2d/d2f/d39/l49 to d0/d10/d2b/d2d/d2f/d39/d7c/dd0/l10d 0 2024-04-27T04:10:19.305 INFO:tasks.workunit.client.0.smithi162.stdout:8/838: chown d0/de/d6e/cb1 13 1 2024-04-27T04:10:19.306 INFO:tasks.workunit.client.0.smithi162.stdout:8/839: write d0/de/d6e/d71/d86/fda [145157,35767] 0 2024-04-27T04:10:19.311 INFO:tasks.workunit.client.0.smithi162.stdout:4/733: dread d2/d25/f7 [0,4194304] 0 2024-04-27T04:10:19.318 INFO:tasks.workunit.client.0.smithi162.stdout:5/805: unlink d8/d1a/d7d/ddf/cc0 0 2024-04-27T04:10:19.318 INFO:tasks.workunit.client.1.smithi181.stdout:4/902: mknod d0/d15/d25/d2b/d6d/d82/c129 0 2024-04-27T04:10:19.323 INFO:tasks.workunit.client.1.smithi181.stdout:4/903: rename d0/d15/d25/d40 to d0/d15/d25/d2a/da0/dc7/d12a 0 2024-04-27T04:10:19.324 INFO:tasks.workunit.client.1.smithi181.stdout:4/904: chown d0/d15/de/d19/f94 0 1 2024-04-27T04:10:19.331 INFO:tasks.workunit.client.0.smithi162.stdout:8/840: rename d0/d10/d2b/d2d/d2f/d39/d7c/dd0/l10d to d0/d10/d25/d7d/db8/l10e 0 2024-04-27T04:10:19.333 INFO:tasks.workunit.client.0.smithi162.stdout:8/841: creat d0/d10/d2b/d2d/d2f/d6b/f10f x:0 0 0 2024-04-27T04:10:19.334 INFO:tasks.workunit.client.0.smithi162.stdout:6/766: truncate d6/d1a/d34/d50/dad/fc9 950324 0 2024-04-27T04:10:19.335 INFO:tasks.workunit.client.0.smithi162.stdout:6/767: read - d6/d1a/d34/d50/fd4 zero size 2024-04-27T04:10:19.337 INFO:tasks.workunit.client.0.smithi162.stdout:6/768: chown d6/d1a/d34/l64 20702 1 2024-04-27T04:10:19.337 INFO:tasks.workunit.client.0.smithi162.stdout:5/806: write d8/d1a/d29/d8b/fd2 [7210537,79489] 0 2024-04-27T04:10:19.338 INFO:tasks.workunit.client.0.smithi162.stdout:5/807: write d8/d9/d55/fec [53777,71324] 0 2024-04-27T04:10:19.344 INFO:tasks.workunit.client.0.smithi162.stdout:5/808: link d8/f12 d8/d1a/f100 0 2024-04-27T04:10:19.345 INFO:tasks.workunit.client.0.smithi162.stdout:4/734: write d2/d25/d1c/d39/d70/dcd/df0/de7/f30 [2301763,20144] 0 2024-04-27T04:10:19.345 INFO:tasks.workunit.client.0.smithi162.stdout:5/809: readlink d8/d2c/l5f 0 2024-04-27T04:10:19.347 INFO:tasks.workunit.client.0.smithi162.stdout:5/810: fdatasync d8/d1a/f31 0 2024-04-27T04:10:19.348 INFO:tasks.workunit.client.0.smithi162.stdout:5/811: chown d8/d1a/fd0 2226630 1 2024-04-27T04:10:19.352 INFO:tasks.workunit.client.0.smithi162.stdout:5/812: getdents d8/d1a/d23/dbc/d78 0 2024-04-27T04:10:19.352 INFO:tasks.workunit.client.0.smithi162.stdout:5/813: readlink d8/d1a/d23/dbc/l8e 0 2024-04-27T04:10:19.354 INFO:tasks.workunit.client.0.smithi162.stdout:5/814: dread d8/d2c/f33 [0,4194304] 0 2024-04-27T04:10:19.356 INFO:tasks.workunit.client.0.smithi162.stdout:6/769: dread d6/db/f6a [0,4194304] 0 2024-04-27T04:10:19.359 INFO:tasks.workunit.client.0.smithi162.stdout:6/770: mkdir d6/d1a/d34/d7f/dd9/de4/df8 0 2024-04-27T04:10:19.361 INFO:tasks.workunit.client.0.smithi162.stdout:5/815: dwrite d8/de3/deb/fbd [0,4194304] 0 2024-04-27T04:10:19.361 INFO:tasks.workunit.client.0.smithi162.stdout:6/771: mknod d6/d1a/d34/d7f/dd9/cf9 0 2024-04-27T04:10:19.362 INFO:tasks.workunit.client.0.smithi162.stdout:5/816: chown c6 2 1 2024-04-27T04:10:19.363 INFO:tasks.workunit.client.0.smithi162.stdout:5/817: stat d8/d1a/d23 0 2024-04-27T04:10:19.363 INFO:tasks.workunit.client.0.smithi162.stdout:4/735: dread d2/d25/d1c/d24/f6d [0,4194304] 0 2024-04-27T04:10:19.363 INFO:tasks.workunit.client.0.smithi162.stdout:5/818: chown d8/de3/deb/d5c 255872 1 2024-04-27T04:10:19.365 INFO:tasks.workunit.client.0.smithi162.stdout:6/772: readlink d6/d1a/d34/d7b/d7e/d97/lba 0 2024-04-27T04:10:19.368 INFO:tasks.workunit.client.0.smithi162.stdout:5/819: symlink d8/d9/d5b/l101 0 2024-04-27T04:10:19.370 INFO:tasks.workunit.client.0.smithi162.stdout:4/736: dwrite d2/d25/d1c/d39/d3b/fc4 [0,4194304] 0 2024-04-27T04:10:19.370 INFO:tasks.workunit.client.0.smithi162.stdout:6/773: unlink d6/d1a/d34/f54 0 2024-04-27T04:10:19.372 INFO:tasks.workunit.client.0.smithi162.stdout:4/737: fsync d2/f38 0 2024-04-27T04:10:19.374 INFO:tasks.workunit.client.0.smithi162.stdout:6/774: mknod d6/d1a/d34/d7f/dd9/cfa 0 2024-04-27T04:10:19.377 INFO:tasks.workunit.client.0.smithi162.stdout:5/820: link d8/de3/deb/fbd d8/d9/db3/f102 0 2024-04-27T04:10:19.377 INFO:tasks.workunit.client.0.smithi162.stdout:4/738: truncate d2/da/f12 3873282 0 2024-04-27T04:10:19.378 INFO:tasks.workunit.client.0.smithi162.stdout:4/739: dread - d2/da/dd/fe1 zero size 2024-04-27T04:10:19.379 INFO:tasks.workunit.client.0.smithi162.stdout:6/775: creat d6/d1a/d34/d7f/dd9/de4/ffb x:0 0 0 2024-04-27T04:10:19.380 INFO:tasks.workunit.client.0.smithi162.stdout:4/740: chown d2/d25/d1c/d39/d70/dcd/df0/de7/f29 10 1 2024-04-27T04:10:19.383 INFO:tasks.workunit.client.0.smithi162.stdout:6/776: fdatasync d6/f35 0 2024-04-27T04:10:19.384 INFO:tasks.workunit.client.0.smithi162.stdout:6/777: stat d6/d1a/d34/d7f/d80/fc7 0 2024-04-27T04:10:19.384 INFO:tasks.workunit.client.0.smithi162.stdout:6/778: fdatasync d6/d1a/d34/d43/fab 0 2024-04-27T04:10:19.386 INFO:tasks.workunit.client.0.smithi162.stdout:5/821: link d8/d2c/fb6 d8/de3/deb/d9f/f103 0 2024-04-27T04:10:19.389 INFO:tasks.workunit.client.0.smithi162.stdout:5/822: truncate d8/f20 7480667 0 2024-04-27T04:10:19.391 INFO:tasks.workunit.client.0.smithi162.stdout:5/823: mknod d8/de3/c104 0 2024-04-27T04:10:19.393 INFO:tasks.workunit.client.0.smithi162.stdout:5/824: write d8/d1a/f45 [4852370,82225] 0 2024-04-27T04:10:19.393 INFO:tasks.workunit.client.0.smithi162.stdout:5/825: chown d8/d1a/d7d/ddf 1425878 1 2024-04-27T04:10:19.394 INFO:tasks.workunit.client.0.smithi162.stdout:5/826: chown d8/de3/deb/d9f 36416764 1 2024-04-27T04:10:19.397 INFO:tasks.workunit.client.0.smithi162.stdout:5/827: mknod d8/d1a/c105 0 2024-04-27T04:10:19.397 INFO:tasks.workunit.client.0.smithi162.stdout:4/741: truncate d2/d25/d1c/d39/d70/dcd/df0/de7/f2e 1321710 0 2024-04-27T04:10:19.399 INFO:tasks.workunit.client.0.smithi162.stdout:5/828: creat d8/de3/deb/f106 x:0 0 0 2024-04-27T04:10:19.400 INFO:tasks.workunit.client.0.smithi162.stdout:6/779: write d6/d1a/d34/d43/f6b [2271947,127691] 0 2024-04-27T04:10:19.400 INFO:tasks.workunit.client.0.smithi162.stdout:6/780: readlink d6/l59 0 2024-04-27T04:10:19.409 INFO:tasks.workunit.client.0.smithi162.stdout:5/829: dwrite d8/f22 [0,4194304] 0 2024-04-27T04:10:19.445 INFO:tasks.workunit.client.1.smithi181.stdout:7/922: sync 2024-04-27T04:10:19.446 INFO:tasks.workunit.client.0.smithi162.stdout:5/830: rename d8/d1a/d23/dbc/l8e to d8/d1a/d23/dbc/d66/d88/l107 0 2024-04-27T04:10:19.449 INFO:tasks.workunit.client.0.smithi162.stdout:6/781: read d6/f30 [687208,109546] 0 2024-04-27T04:10:19.452 INFO:tasks.workunit.client.1.smithi181.stdout:7/923: dwrite d1/d2/f9f [4194304,4194304] 0 2024-04-27T04:10:19.460 INFO:tasks.workunit.client.1.smithi181.stdout:7/924: dwrite d1/fc8 [0,4194304] 0 2024-04-27T04:10:19.461 INFO:tasks.workunit.client.1.smithi181.stdout:7/925: chown d1/d2/d1c/d2d/d61/d80 3339991 1 2024-04-27T04:10:19.469 INFO:tasks.workunit.client.1.smithi181.stdout:7/926: chown d1/d2/d1c/d37/cdd 1 1 2024-04-27T04:10:19.469 INFO:tasks.workunit.client.1.smithi181.stdout:7/927: chown d1/f45 938 1 2024-04-27T04:10:19.474 INFO:tasks.workunit.client.1.smithi181.stdout:7/928: unlink d1/d2/f27 0 2024-04-27T04:10:19.478 INFO:tasks.workunit.client.1.smithi181.stdout:7/929: creat d1/dc3/f119 x:0 0 0 2024-04-27T04:10:19.487 INFO:tasks.workunit.client.1.smithi181.stdout:7/930: dwrite d1/df/d1a/de8/f106 [0,4194304] 0 2024-04-27T04:10:19.489 INFO:tasks.workunit.client.0.smithi162.stdout:5/831: mkdir d8/d9/d5a/d108 0 2024-04-27T04:10:19.490 INFO:tasks.workunit.client.1.smithi181.stdout:7/931: truncate d1/d22/d15/dc6/de5/ff4 795368 0 2024-04-27T04:10:19.490 INFO:tasks.workunit.client.0.smithi162.stdout:5/832: chown d8/d1a/d23/dbc/d66 23 1 2024-04-27T04:10:19.493 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:10:19.494+0000 7ffa31445700 1 -- 172.21.15.162:0/112660346 --> [v2:172.21.15.181:6828/4165336392,v1:172.21.15.181:6829/4165336392] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7ffa2c10e610 con 0x7ffa1406bd60 2024-04-27T04:10:19.494 INFO:tasks.workunit.client.1.smithi181.stdout:7/932: mknod d1/d2/d1c/d56/c11a 0 2024-04-27T04:10:19.495 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:10:19.496+0000 7ffa137fe700 1 -- 172.21.15.162:0/112660346 <== mgr.24367 v2:172.21.15.181:6828/4165336392 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+310 (secure 0 0 0) 0x7ffa2c10e610 con 0x7ffa1406bd60 2024-04-27T04:10:19.495 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-04-27T04:10:19.495 INFO:teuthology.orchestra.run.smithi162.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:90a3a9fa9ceb8977f6dad9ccd36fe46018d07e2118199820523a6b35301d9c16", 2024-04-27T04:10:19.495 INFO:teuthology.orchestra.run.smithi162.stdout: "in_progress": true, 2024-04-27T04:10:19.495 INFO:teuthology.orchestra.run.smithi162.stdout: "which": "Upgrading daemons of type(s) mgr", 2024-04-27T04:10:19.495 INFO:teuthology.orchestra.run.smithi162.stdout: "services_complete": [], 2024-04-27T04:10:19.495 INFO:teuthology.orchestra.run.smithi162.stdout: "progress": "1/2 daemons upgraded", 2024-04-27T04:10:19.495 INFO:teuthology.orchestra.run.smithi162.stdout: "message": "", 2024-04-27T04:10:19.495 INFO:teuthology.orchestra.run.smithi162.stdout: "is_paused": false 2024-04-27T04:10:19.495 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-04-27T04:10:19.496 INFO:tasks.workunit.client.0.smithi162.stdout:6/782: unlink d6/db/dc0/l96 0 2024-04-27T04:10:19.497 INFO:tasks.workunit.client.0.smithi162.stdout:5/833: creat d8/d1a/d23/dbc/d66/d88/f109 x:0 0 0 2024-04-27T04:10:19.498 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:10:19.499+0000 7ffa31445700 1 -- 172.21.15.162:0/112660346 >> [v2:172.21.15.181:6828/4165336392,v1:172.21.15.181:6829/4165336392] conn(0x7ffa1406bd60 msgr2=0x7ffa1406e210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:10:19.498 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:10:19.499+0000 7ffa31445700 1 --2- 172.21.15.162:0/112660346 >> [v2:172.21.15.181:6828/4165336392,v1:172.21.15.181:6829/4165336392] conn(0x7ffa1406bd60 0x7ffa1406e210 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7ffa1c00b0d0 tx=0x7ffa1c009e30 comp rx=0 tx=0).stop 2024-04-27T04:10:19.498 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:10:19.499+0000 7ffa31445700 1 -- 172.21.15.162:0/112660346 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ffa2c0fcf90 msgr2=0x7ffa2c10cde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:10:19.498 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:10:19.499+0000 7ffa31445700 1 --2- 172.21.15.162:0/112660346 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ffa2c0fcf90 0x7ffa2c10cde0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7ffa20010d50 tx=0x7ffa2000b040 comp rx=0 tx=0).stop 2024-04-27T04:10:19.499 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:10:19.499+0000 7ffa31445700 1 -- 172.21.15.162:0/112660346 shutdown_connections 2024-04-27T04:10:19.499 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:10:19.499+0000 7ffa31445700 1 --2- 172.21.15.162:0/112660346 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ffa2c0fbce0 0x7ffa2c10c8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:10:19.499 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:10:19.499+0000 7ffa31445700 1 --2- 172.21.15.162:0/112660346 >> [v2:172.21.15.181:6828/4165336392,v1:172.21.15.181:6829/4165336392] conn(0x7ffa1406bd60 0x7ffa1406e210 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:10:19.499 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:10:19.499+0000 7ffa31445700 1 --2- 172.21.15.162:0/112660346 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ffa2c0fcf90 0x7ffa2c10cde0 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:10:19.499 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:10:19.499+0000 7ffa31445700 1 -- 172.21.15.162:0/112660346 >> 172.21.15.162:0/112660346 conn(0x7ffa2c0f7440 msgr2=0x7ffa2c0f8a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:10:19.499 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:10:19.499+0000 7ffa31445700 1 -- 172.21.15.162:0/112660346 shutdown_connections 2024-04-27T04:10:19.499 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:10:19.500+0000 7ffa31445700 1 -- 172.21.15.162:0/112660346 wait complete. 2024-04-27T04:10:19.500 INFO:tasks.workunit.client.1.smithi181.stdout:7/933: dwrite d1/d2/d35/d67/f96 [0,4194304] 0 2024-04-27T04:10:19.505 INFO:tasks.workunit.client.1.smithi181.stdout:7/934: readlink d1/d22/d15/d1f/d34/l3b 0 2024-04-27T04:10:19.511 INFO:tasks.workunit.client.1.smithi181.stdout:7/935: unlink d1/d2/d1c/d2d/cf0 0 2024-04-27T04:10:19.514 INFO:tasks.workunit.client.1.smithi181.stdout:7/936: rmdir d1/d2/d1c/d99 39 2024-04-27T04:10:19.516 INFO:tasks.workunit.client.1.smithi181.stdout:7/937: mkdir d1/d2/d11b 0 2024-04-27T04:10:19.517 INFO:tasks.workunit.client.1.smithi181.stdout:7/938: fdatasync d1/d2/ff9 0 2024-04-27T04:10:19.520 INFO:tasks.workunit.client.1.smithi181.stdout:7/939: creat d1/d22/d15/d5f/d81/de3/f11c x:0 0 0 2024-04-27T04:10:19.526 INFO:tasks.workunit.client.0.smithi162.stdout:6/783: dwrite d6/f25 [0,4194304] 0 2024-04-27T04:10:19.529 INFO:tasks.workunit.client.0.smithi162.stdout:5/834: dread d8/d9/fc [0,4194304] 0 2024-04-27T04:10:19.531 INFO:tasks.workunit.client.1.smithi181.stdout:7/940: dwrite d1/d22/d15/dc6/dd8/fe4 [0,4194304] 0 2024-04-27T04:10:19.533 INFO:tasks.workunit.client.0.smithi162.stdout:5/835: unlink d8/d9/d5a/fb2 0 2024-04-27T04:10:19.534 INFO:tasks.workunit.client.0.smithi162.stdout:0/870: sync 2024-04-27T04:10:19.536 INFO:tasks.workunit.client.1.smithi181.stdout:7/941: creat d1/d22/d15/d1f/d34/f11d x:0 0 0 2024-04-27T04:10:19.540 INFO:tasks.workunit.client.0.smithi162.stdout:0/871: truncate d5/dc/d1e/d20/dba/df9/f72 3954955 0 2024-04-27T04:10:19.541 INFO:tasks.workunit.client.1.smithi181.stdout:7/942: read d1/dc3/fcf [460671,22505] 0 2024-04-27T04:10:19.541 INFO:tasks.workunit.client.0.smithi162.stdout:5/836: getdents d8/d9/d5b/d96 0 2024-04-27T04:10:19.544 INFO:tasks.workunit.client.0.smithi162.stdout:0/872: mknod d5/dc/d1e/d41/d79/d109/d119/c122 0 2024-04-27T04:10:19.545 INFO:tasks.workunit.client.0.smithi162.stdout:5/837: rmdir d8/de3 39 2024-04-27T04:10:19.546 INFO:tasks.workunit.client.1.smithi181.stdout:7/943: symlink d1/d22/d15/d5f/d93/l11e 0 2024-04-27T04:10:19.547 INFO:tasks.workunit.client.0.smithi162.stdout:0/873: mknod d5/dc/d1e/d41/d79/d109/c123 0 2024-04-27T04:10:19.548 INFO:tasks.workunit.client.0.smithi162.stdout:6/784: write d6/d1a/d34/d3a/d68/d6c/f4f [4336763,41454] 0 2024-04-27T04:10:19.551 INFO:tasks.workunit.client.0.smithi162.stdout:5/838: dwrite d8/d1a/d29/d8b/fd2 [0,4194304] 0 2024-04-27T04:10:19.553 INFO:tasks.workunit.client.0.smithi162.stdout:6/785: symlink d6/d1a/d34/d50/dad/lfc 0 2024-04-27T04:10:19.554 INFO:tasks.workunit.client.0.smithi162.stdout:0/874: dread d5/dc/d1e/f30 [0,4194304] 0 2024-04-27T04:10:19.556 INFO:tasks.workunit.client.0.smithi162.stdout:6/786: mkdir d6/d1a/d34/d3a/dfd 0 2024-04-27T04:10:19.557 INFO:tasks.workunit.client.0.smithi162.stdout:6/787: readlink d6/l2a 0 2024-04-27T04:10:19.559 INFO:tasks.workunit.client.0.smithi162.stdout:6/788: dread - d6/d1a/d34/d7b/fb9 zero size 2024-04-27T04:10:19.561 INFO:tasks.workunit.client.1.smithi181.stdout:7/944: write d1/d22/d15/dc6/dd8/f66 [661741,106133] 0 2024-04-27T04:10:19.562 INFO:tasks.workunit.client.0.smithi162.stdout:6/789: creat d6/d1a/d34/d3a/d68/d6c/de3/ffe x:0 0 0 2024-04-27T04:10:19.565 INFO:tasks.workunit.client.1.smithi181.stdout:1/884: sync 2024-04-27T04:10:19.565 INFO:tasks.workunit.client.1.smithi181.stdout:8/913: sync 2024-04-27T04:10:19.565 INFO:tasks.workunit.client.1.smithi181.stdout:6/847: sync 2024-04-27T04:10:19.566 INFO:tasks.workunit.client.1.smithi181.stdout:9/844: sync 2024-04-27T04:10:19.566 INFO:tasks.workunit.client.1.smithi181.stdout:7/945: unlink d1/d22/d15/dc6/dd8/f66 0 2024-04-27T04:10:19.567 INFO:tasks.workunit.client.0.smithi162.stdout:6/790: creat d6/d1a/d34/d7b/d7e/fff x:0 0 0 2024-04-27T04:10:19.567 INFO:tasks.workunit.client.1.smithi181.stdout:7/946: chown d1/d2/d35/d67/la7 39 1 2024-04-27T04:10:19.570 INFO:tasks.workunit.client.1.smithi181.stdout:9/845: fdatasync d0/d44/f46 0 2024-04-27T04:10:19.570 INFO:tasks.workunit.client.0.smithi162.stdout:6/791: creat d6/d1a/d34/d3a/d68/d6c/f100 x:0 0 0 2024-04-27T04:10:19.571 INFO:tasks.workunit.client.1.smithi181.stdout:1/885: dread - d1/dad/db5/d6/db/fe9 zero size 2024-04-27T04:10:19.571 INFO:tasks.workunit.client.1.smithi181.stdout:8/914: dread d1/d6/d2f/d72/f94 [0,4194304] 0 2024-04-27T04:10:19.572 INFO:tasks.workunit.client.0.smithi162.stdout:0/875: fsync d5/dc/d1e/d20/dba/df9/f72 0 2024-04-27T04:10:19.572 INFO:tasks.workunit.client.0.smithi162.stdout:6/792: chown d6/d1a/d34/d7f/d80/db3/cdd 1272 1 2024-04-27T04:10:19.572 INFO:tasks.workunit.client.1.smithi181.stdout:7/947: fsync d1/df/f18 0 2024-04-27T04:10:19.574 INFO:tasks.workunit.client.1.smithi181.stdout:8/915: readlink d1/d6/d2f/d30/d9d/dea/db3/ddf/le0 0 2024-04-27T04:10:19.575 INFO:tasks.workunit.client.0.smithi162.stdout:6/793: read d6/d1a/d34/d3a/d68/f78 [801212,17289] 0 2024-04-27T04:10:19.575 INFO:tasks.workunit.client.0.smithi162.stdout:0/876: dread d5/dc/d1e/d20/dba/df9/d98/fad [0,4194304] 0 2024-04-27T04:10:19.576 INFO:tasks.workunit.client.1.smithi181.stdout:9/846: dread d0/d44/d4b/d43/d88/da1/dfc/dbd/df3/da0/ddb/fed [0,4194304] 0 2024-04-27T04:10:19.583 INFO:tasks.workunit.client.1.smithi181.stdout:6/848: truncate d7/dc/dc3/f4b 7585037 0 2024-04-27T04:10:19.584 INFO:tasks.workunit.client.0.smithi162.stdout:0/877: write d5/dc/d1e/d46/d48/d9e/dce/dd1/d116/da8/fe9 [3198021,72640] 0 2024-04-27T04:10:19.587 INFO:tasks.workunit.client.0.smithi162.stdout:6/794: rename d6/d1a/d34/fc3 to d6/d1a/d34/f101 0 2024-04-27T04:10:19.594 INFO:tasks.workunit.client.1.smithi181.stdout:6/849: dwrite d7/dc/dc3/d72/d98/f10a [0,4194304] 0 2024-04-27T04:10:19.600 INFO:tasks.workunit.client.1.smithi181.stdout:9/847: rename d0/d44/d4b/d43/d88/da1/lea to d0/d44/d4b/d43/d89/d11d/l11f 0 2024-04-27T04:10:19.602 INFO:tasks.workunit.client.1.smithi181.stdout:1/886: creat d1/dad/db5/d6/da3/d1e/f121 x:0 0 0 2024-04-27T04:10:19.606 INFO:tasks.workunit.client.1.smithi181.stdout:6/850: stat d7/dc/dc3/d72/daa/dd3/f65 0 2024-04-27T04:10:19.606 INFO:tasks.workunit.client.0.smithi162.stdout:6/795: link d6/d1a/d34/l8e d6/d1a/d34/d7b/l102 0 2024-04-27T04:10:19.607 INFO:tasks.workunit.client.0.smithi162.stdout:6/796: fsync d6/d1a/d34/f3e 0 2024-04-27T04:10:19.607 INFO:tasks.workunit.client.1.smithi181.stdout:9/848: fdatasync d0/d44/d4b/d43/f82 0 2024-04-27T04:10:19.609 INFO:tasks.workunit.client.1.smithi181.stdout:4/905: sync 2024-04-27T04:10:19.609 INFO:tasks.workunit.client.1.smithi181.stdout:3/932: sync 2024-04-27T04:10:19.610 INFO:tasks.workunit.client.1.smithi181.stdout:7/948: dread d1/d2/f12 [4194304,4194304] 0 2024-04-27T04:10:19.611 INFO:tasks.workunit.client.0.smithi162.stdout:0/878: dread d5/dc/d1e/d46/d48/d9e/dce/ffd [4194304,4194304] 0 2024-04-27T04:10:19.612 INFO:tasks.workunit.client.0.smithi162.stdout:0/879: dread - d5/d26/da2/dc0/ff6 zero size 2024-04-27T04:10:19.617 INFO:tasks.workunit.client.1.smithi181.stdout:1/887: read - d1/dad/db5/d6/da3/d1e/d48/da6/da4/ff5 zero size 2024-04-27T04:10:19.618 INFO:tasks.workunit.client.1.smithi181.stdout:1/888: stat d1/d2/d89/fef 0 2024-04-27T04:10:19.618 INFO:tasks.workunit.client.1.smithi181.stdout:3/933: creat d0/d6c/d52/d44/dd5/f12b x:0 0 0 2024-04-27T04:10:19.621 INFO:tasks.workunit.client.0.smithi162.stdout:6/797: write d6/d1a/f18 [887959,54724] 0 2024-04-27T04:10:19.621 INFO:tasks.workunit.client.1.smithi181.stdout:9/849: write d0/d44/d4b/d43/d88/da1/dfc/dbd/dd1/dde/d104/f84 [3193439,120616] 0 2024-04-27T04:10:19.622 INFO:tasks.workunit.client.1.smithi181.stdout:8/916: link d1/d8c/c119 d1/d6/d2f/d30/d66/dc0/dcd/de1/c142 0 2024-04-27T04:10:19.625 INFO:tasks.workunit.client.1.smithi181.stdout:6/851: dread d7/d31/f8c [0,4194304] 0 2024-04-27T04:10:19.625 INFO:tasks.workunit.client.1.smithi181.stdout:6/852: chown d7/dc/dc3/d72/dd6 722533 1 2024-04-27T04:10:19.626 INFO:tasks.workunit.client.1.smithi181.stdout:3/934: rename d0/f32 to d0/d6c/d52/dbb/d98/da7/f12c 0 2024-04-27T04:10:19.629 INFO:tasks.workunit.client.1.smithi181.stdout:4/906: truncate d0/d15/d5c/fb2 29123 0 2024-04-27T04:10:19.631 INFO:tasks.workunit.client.0.smithi162.stdout:0/880: dread d5/d26/da2/fcb [0,4194304] 0 2024-04-27T04:10:19.632 INFO:tasks.workunit.client.0.smithi162.stdout:0/881: chown d5/f27 6 1 2024-04-27T04:10:19.632 INFO:tasks.workunit.client.1.smithi181.stdout:1/889: dwrite d1/dad/db5/d6/da3/d1e/d20/f82 [0,4194304] 0 2024-04-27T04:10:19.633 INFO:tasks.workunit.client.0.smithi162.stdout:6/798: getdents d6/d1a/d34/d7b/d7e/d97 0 2024-04-27T04:10:19.633 INFO:tasks.workunit.client.1.smithi181.stdout:1/890: dread - d1/dad/db5/d6/da3/d1e/d20/d53/d56/de2/f11a zero size 2024-04-27T04:10:19.635 INFO:tasks.workunit.client.1.smithi181.stdout:7/949: symlink d1/d2/d1c/d2d/d61/d80/dfa/l11f 0 2024-04-27T04:10:19.636 INFO:tasks.workunit.client.0.smithi162.stdout:0/882: readlink d5/dc/d1e/d46/d48/d9e/dce/dd1/d116/da8/lb2 0 2024-04-27T04:10:19.636 INFO:tasks.workunit.client.1.smithi181.stdout:1/891: write d1/dad/db5/d6/da3/d1e/d20/d53/d56/dd9/f117 [359094,63391] 0 2024-04-27T04:10:19.637 INFO:tasks.workunit.client.1.smithi181.stdout:8/917: fdatasync d1/d6/f107 0 2024-04-27T04:10:19.638 INFO:tasks.workunit.client.1.smithi181.stdout:7/950: write d1/d22/d15/d5f/d81/de3/f11c [902023,96506] 0 2024-04-27T04:10:19.642 INFO:tasks.workunit.client.0.smithi162.stdout:0/883: creat d5/dc/d1e/d46/d48/d9e/dce/dd1/d116/ddd/f124 x:0 0 0 2024-04-27T04:10:19.643 INFO:tasks.workunit.client.1.smithi181.stdout:6/853: creat d7/dc/dc3/d72/f11e x:0 0 0 2024-04-27T04:10:19.644 INFO:tasks.workunit.client.0.smithi162.stdout:6/799: creat d6/d1a/d34/d7f/dd8/f103 x:0 0 0 2024-04-27T04:10:19.648 INFO:tasks.workunit.client.0.smithi162.stdout:0/884: dwrite d5/dc/d1e/d20/dba/df9/fd0 [0,4194304] 0 2024-04-27T04:10:19.648 INFO:tasks.workunit.client.0.smithi162.stdout:6/800: creat d6/db/dc0/f104 x:0 0 0 2024-04-27T04:10:19.649 INFO:tasks.workunit.client.1.smithi181.stdout:9/850: dwrite d0/f26 [4194304,4194304] 0 2024-04-27T04:10:19.651 INFO:tasks.workunit.client.1.smithi181.stdout:4/907: creat d0/d15/de/d19/d8a/f12b x:0 0 0 2024-04-27T04:10:19.652 INFO:tasks.workunit.client.1.smithi181.stdout:1/892: creat d1/dad/db5/d6/da3/d1e/d20/d53/d56/dd9/f122 x:0 0 0 2024-04-27T04:10:19.654 INFO:tasks.workunit.client.1.smithi181.stdout:1/893: fdatasync d1/dad/db5/d6/da3/f10a 0 2024-04-27T04:10:19.656 INFO:tasks.workunit.client.0.smithi162.stdout:3/763: sync 2024-04-27T04:10:19.656 INFO:tasks.workunit.client.0.smithi162.stdout:2/758: sync 2024-04-27T04:10:19.658 INFO:tasks.workunit.client.0.smithi162.stdout:6/801: symlink d6/d1a/d34/d3a/dbf/l105 0 2024-04-27T04:10:19.658 INFO:tasks.workunit.client.1.smithi181.stdout:7/951: rename d1/df/l6e to d1/df/d1a/de8/l120 0 2024-04-27T04:10:19.662 INFO:tasks.workunit.client.0.smithi162.stdout:0/885: dread d5/dc/d1e/d20/dba/fe8 [0,4194304] 0 2024-04-27T04:10:19.663 INFO:tasks.workunit.client.0.smithi162.stdout:0/886: dread - d5/dc/d1e/d46/d48/d9e/f10c zero size 2024-04-27T04:10:19.664 INFO:tasks.workunit.client.0.smithi162.stdout:3/764: mknod d1/d70/c10f 0 2024-04-27T04:10:19.664 INFO:tasks.workunit.client.1.smithi181.stdout:3/935: mknod d0/d6c/d52/d44/dd5/d58/d64/dc6/ddd/d11d/c12d 0 2024-04-27T04:10:19.670 INFO:tasks.workunit.client.0.smithi162.stdout:3/765: creat d1/d18/d5d/d5f/d66/dd2/da3/f110 x:0 0 0 2024-04-27T04:10:19.671 INFO:tasks.workunit.client.0.smithi162.stdout:2/759: mknod d0/d4/d17/d2b/d64/d96/cef 0 2024-04-27T04:10:19.672 INFO:tasks.workunit.client.0.smithi162.stdout:6/802: symlink d6/d1a/d34/d3a/dfd/l106 0 2024-04-27T04:10:19.674 INFO:tasks.workunit.client.0.smithi162.stdout:0/887: mknod d5/c125 0 2024-04-27T04:10:19.675 INFO:tasks.workunit.client.0.smithi162.stdout:0/888: read d5/dc/d1e/d41/f4f [5899760,101209] 0 2024-04-27T04:10:19.675 INFO:tasks.workunit.client.1.smithi181.stdout:3/936: creat d0/d6c/d52/f12e x:0 0 0 2024-04-27T04:10:19.676 INFO:tasks.workunit.client.0.smithi162.stdout:0/889: stat d5/dc/d1e/d20/ffc 0 2024-04-27T04:10:19.676 INFO:tasks.workunit.client.0.smithi162.stdout:3/766: fsync d1/d18/d23/d58/f8f 0 2024-04-27T04:10:19.677 INFO:tasks.workunit.client.1.smithi181.stdout:3/937: write d0/d6c/d52/d44/dd5/f69 [132542,2614] 0 2024-04-27T04:10:19.677 INFO:tasks.workunit.client.0.smithi162.stdout:3/767: chown d1/d21/d47/fcd 128304 1 2024-04-27T04:10:19.679 INFO:tasks.workunit.client.1.smithi181.stdout:7/952: dread d1/d2/d1c/d2d/fea [0,4194304] 0 2024-04-27T04:10:19.680 INFO:tasks.workunit.client.0.smithi162.stdout:0/890: creat d5/dc/d1e/d20/dba/f126 x:0 0 0 2024-04-27T04:10:19.683 INFO:tasks.workunit.client.1.smithi181.stdout:6/854: creat d7/dc/d1c/d88/f11f x:0 0 0 2024-04-27T04:10:19.683 INFO:tasks.workunit.client.0.smithi162.stdout:3/768: dread - d1/d18/d23/d2f/faf zero size 2024-04-27T04:10:19.684 INFO:tasks.workunit.client.1.smithi181.stdout:8/918: dread d1/d6/d2f/d30/d9d/dea/f37 [0,4194304] 0 2024-04-27T04:10:19.686 INFO:tasks.workunit.client.0.smithi162.stdout:2/760: rename d0/d4/d2a/f36 to d0/d4/d17/db4/ff0 0 2024-04-27T04:10:19.686 INFO:tasks.workunit.client.1.smithi181.stdout:4/908: link d0/d15/de/d19/d77/ce6 d0/d15/c12c 0 2024-04-27T04:10:19.688 INFO:tasks.workunit.client.0.smithi162.stdout:6/803: mknod d6/d1a/d34/c107 0 2024-04-27T04:10:19.689 INFO:tasks.workunit.client.1.smithi181.stdout:3/938: mkdir d0/d6c/d52/d44/dd5/d6b/d12f 0 2024-04-27T04:10:19.690 INFO:tasks.workunit.client.0.smithi162.stdout:0/891: symlink d5/dc/d1e/l127 0 2024-04-27T04:10:19.690 INFO:tasks.workunit.client.1.smithi181.stdout:1/894: link d1/dad/db5/laf d1/dad/db5/d6/da3/d52/l123 0 2024-04-27T04:10:19.701 INFO:tasks.workunit.client.0.smithi162.stdout:3/769: rename d1/d18/d5d/d5f/c108 to d1/d18/d23/d34/daa/de6/c111 0 2024-04-27T04:10:19.703 INFO:tasks.workunit.client.0.smithi162.stdout:2/761: mknod d0/d4/d17/d2b/d64/cf1 0 2024-04-27T04:10:19.704 INFO:tasks.workunit.client.0.smithi162.stdout:3/770: readlink d1/d18/d23/d34/lb5 0 2024-04-27T04:10:19.707 INFO:tasks.workunit.client.0.smithi162.stdout:3/771: rmdir d1/d18/d5d/d5f/d66/dd2 39 2024-04-27T04:10:19.709 INFO:tasks.workunit.client.0.smithi162.stdout:3/772: rmdir d1/d18/d23/d2f 39 2024-04-27T04:10:19.711 INFO:tasks.workunit.client.0.smithi162.stdout:2/762: dwrite d0/d4/d2a/d3b/d49/d5a/d63/d70/fec [0,4194304] 0 2024-04-27T04:10:19.714 INFO:tasks.workunit.client.0.smithi162.stdout:2/763: fdatasync d0/d4/d17/d2b/d64/d89/fc1 0 2024-04-27T04:10:19.715 INFO:tasks.workunit.client.0.smithi162.stdout:0/892: symlink d5/dc/d1e/d20/dba/df9/l128 0 2024-04-27T04:10:19.717 INFO:tasks.workunit.client.0.smithi162.stdout:2/764: stat d0/d20/l59 0 2024-04-27T04:10:19.717 INFO:tasks.workunit.client.0.smithi162.stdout:6/804: dwrite d6/d1a/d34/fb6 [0,4194304] 0 2024-04-27T04:10:19.718 INFO:tasks.workunit.client.0.smithi162.stdout:3/773: dread d1/d18/d5d/d5f/d66/f68 [0,4194304] 0 2024-04-27T04:10:19.719 INFO:tasks.workunit.client.0.smithi162.stdout:0/893: readlink d5/dc/d1e/d41/d79/l115 0 2024-04-27T04:10:19.721 INFO:tasks.workunit.client.0.smithi162.stdout:3/774: mkdir d1/d21/d47/d57/dd6/d112 0 2024-04-27T04:10:19.724 INFO:tasks.workunit.client.0.smithi162.stdout:6/805: dread d6/d1a/d34/d3a/d68/f78 [0,4194304] 0 2024-04-27T04:10:19.725 INFO:tasks.workunit.client.0.smithi162.stdout:3/775: fsync d1/d18/d5d/d6f/fa9 0 2024-04-27T04:10:19.726 INFO:tasks.workunit.client.0.smithi162.stdout:0/894: link d5/dc/d1e/d20/dec/df1/db4/lcf d5/dc/d1e/d41/d79/l129 0 2024-04-27T04:10:19.727 INFO:tasks.workunit.client.0.smithi162.stdout:6/806: unlink d6/d1a/f31 0 2024-04-27T04:10:19.729 INFO:tasks.workunit.client.0.smithi162.stdout:6/807: mkdir d6/d1a/d34/d7f/dd9/d108 0 2024-04-27T04:10:19.730 INFO:tasks.workunit.client.0.smithi162.stdout:3/776: creat d1/d18/d5d/d5f/d66/dd2/da3/f113 x:0 0 0 2024-04-27T04:10:19.731 INFO:tasks.workunit.client.0.smithi162.stdout:6/808: mknod d6/d1a/d34/d7b/d7e/d97/c109 0 2024-04-27T04:10:19.736 INFO:tasks.workunit.client.0.smithi162.stdout:3/777: dwrite d1/d18/d23/d34/daa/de6/ff0 [0,4194304] 0 2024-04-27T04:10:19.737 INFO:tasks.workunit.client.0.smithi162.stdout:3/778: chown d1/d70/c10f 29465 1 2024-04-27T04:10:19.779 INFO:tasks.workunit.client.0.smithi162.stdout:1/712: sync 2024-04-27T04:10:19.779 INFO:tasks.workunit.client.0.smithi162.stdout:9/746: sync 2024-04-27T04:10:19.779 INFO:tasks.workunit.client.0.smithi162.stdout:7/818: sync 2024-04-27T04:10:19.779 INFO:tasks.workunit.client.0.smithi162.stdout:4/742: sync 2024-04-27T04:10:19.779 INFO:tasks.workunit.client.0.smithi162.stdout:8/842: sync 2024-04-27T04:10:19.779 INFO:tasks.workunit.client.0.smithi162.stdout:5/839: sync 2024-04-27T04:10:19.781 INFO:tasks.workunit.client.0.smithi162.stdout:8/843: write d0/d10/d25/d65/fad [980689,76259] 0 2024-04-27T04:10:19.781 INFO:tasks.workunit.client.0.smithi162.stdout:9/747: read d3/d23/f73 [2307129,13834] 0 2024-04-27T04:10:19.783 INFO:tasks.workunit.client.0.smithi162.stdout:1/713: getdents d0/d11/d5b/dd3 0 2024-04-27T04:10:19.785 INFO:tasks.workunit.client.0.smithi162.stdout:8/844: creat d0/d10/f110 x:0 0 0 2024-04-27T04:10:19.786 INFO:tasks.workunit.client.0.smithi162.stdout:5/840: rename d8/d9/fc to d8/de3/deb/f10a 0 2024-04-27T04:10:19.788 INFO:tasks.workunit.client.0.smithi162.stdout:9/748: dwrite d3/fc [0,4194304] 0 2024-04-27T04:10:19.791 INFO:tasks.workunit.client.1.smithi181.stdout:4/909: chown d0/d15/f2f 210 1 2024-04-27T04:10:19.794 INFO:tasks.workunit.client.1.smithi181.stdout:4/910: write d0/f114 [309338,66424] 0 2024-04-27T04:10:19.794 INFO:tasks.workunit.client.1.smithi181.stdout:9/851: dwrite d0/d44/d4b/d43/f82 [0,4194304] 0 2024-04-27T04:10:19.796 INFO:tasks.workunit.client.1.smithi181.stdout:1/895: symlink d1/dad/db5/d6/da3/d1e/d48/da6/l124 0 2024-04-27T04:10:19.812 INFO:tasks.workunit.client.0.smithi162.stdout:5/841: rename d8/d1a/d7d/ddf/lef to d8/d2c/l10b 0 2024-04-27T04:10:19.814 INFO:tasks.workunit.client.0.smithi162.stdout:6/809: write d6/d1a/d34/d50/dad/fae [720797,61756] 0 2024-04-27T04:10:19.815 INFO:tasks.workunit.client.0.smithi162.stdout:8/845: dread d0/de/d6e/d71/fae [4194304,4194304] 0 2024-04-27T04:10:19.817 INFO:tasks.workunit.client.0.smithi162.stdout:8/846: write d0/de/d6e/fdf [857627,34910] 0 2024-04-27T04:10:19.820 INFO:tasks.workunit.client.0.smithi162.stdout:9/749: rename d3/d13/d2d/d30/c42 to d3/d11/d45/d4e/ce3 0 2024-04-27T04:10:19.824 INFO:tasks.workunit.client.0.smithi162.stdout:4/743: dread d2/d25/d1c/d39/d70/dcd/df0/de7/f29 [0,4194304] 0 2024-04-27T04:10:19.825 INFO:tasks.workunit.client.0.smithi162.stdout:6/810: rename d6/d1a/d34/d50/l5e to d6/d1a/d34/d3a/d68/d6c/de3/l10a 0 2024-04-27T04:10:19.826 INFO:tasks.workunit.client.0.smithi162.stdout:4/744: stat d2/d25/d1c/d39/d90/fcc 0 2024-04-27T04:10:19.827 INFO:tasks.workunit.client.0.smithi162.stdout:7/819: dwrite d3/d5/d5b/d83/de/d2b/f2c [0,4194304] 0 2024-04-27T04:10:19.832 INFO:tasks.workunit.client.0.smithi162.stdout:4/745: mknod d2/da/cf1 0 2024-04-27T04:10:19.834 INFO:tasks.workunit.client.0.smithi162.stdout:7/820: rename d3/d5/d5b/d83/de/d2b/d81/ca1 to d3/dee/c111 0 2024-04-27T04:10:19.836 INFO:tasks.workunit.client.0.smithi162.stdout:4/746: unlink d2/da/fa1 0 2024-04-27T04:10:19.839 INFO:tasks.workunit.client.0.smithi162.stdout:6/811: rename d6/d1a/d34/d50/dad/lfc to d6/d1a/d34/d7f/dd9/de4/df8/l10b 0 2024-04-27T04:10:19.839 INFO:tasks.workunit.client.0.smithi162.stdout:4/747: readlink d2/d54/dca/lce 0 2024-04-27T04:10:19.841 INFO:tasks.workunit.client.0.smithi162.stdout:5/842: dwrite d8/d9/d5b/fb5 [0,4194304] 0 2024-04-27T04:10:19.843 INFO:tasks.workunit.client.0.smithi162.stdout:9/750: dwrite d3/d23/f3e [0,4194304] 0 2024-04-27T04:10:19.849 INFO:tasks.workunit.client.0.smithi162.stdout:7/821: write d3/d5/d5b/d83/de/d2b/d81/daa/dbd/fa0 [1898952,12216] 0 2024-04-27T04:10:19.849 INFO:tasks.workunit.client.0.smithi162.stdout:3/779: dread d1/d18/f11 [0,4194304] 0 2024-04-27T04:10:19.850 INFO:tasks.workunit.client.1.smithi181.stdout:8/919: rename d1/d6/d2f/d30/cfc to d1/d6/d2f/d30/d66/dc0/dcd/c143 0 2024-04-27T04:10:19.850 INFO:tasks.workunit.client.1.smithi181.stdout:1/896: write d1/dad/db5/d6/da3/d1e/d20/d53/d56/de2/f11a [480882,55779] 0 2024-04-27T04:10:19.851 INFO:tasks.workunit.client.1.smithi181.stdout:4/911: write d0/f3b [1489515,56390] 0 2024-04-27T04:10:19.852 INFO:tasks.workunit.client.0.smithi162.stdout:7/822: fdatasync d3/d5/d5b/d83/de/d2b/d81/daa/dbd/fef 0 2024-04-27T04:10:19.853 INFO:tasks.workunit.client.1.smithi181.stdout:9/852: write d0/d44/f6e [5184482,116786] 0 2024-04-27T04:10:19.853 INFO:tasks.workunit.client.0.smithi162.stdout:6/812: creat d6/d1a/d34/d3a/dbf/dd7/f10c x:0 0 0 2024-04-27T04:10:19.855 INFO:tasks.workunit.client.0.smithi162.stdout:8/847: dread d0/d10/f6a [0,4194304] 0 2024-04-27T04:10:19.858 INFO:tasks.workunit.client.1.smithi181.stdout:9/853: chown d0/d44/d4b/d43/d88/da1/dfc/dbd/dd1/dde/d104/f84 370594133 1 2024-04-27T04:10:19.859 INFO:tasks.workunit.client.0.smithi162.stdout:3/780: symlink d1/d21/d47/l114 0 2024-04-27T04:10:19.860 INFO:tasks.workunit.client.0.smithi162.stdout:5/843: symlink d8/d9/d5b/d96/l10c 0 2024-04-27T04:10:19.863 INFO:tasks.workunit.client.1.smithi181.stdout:1/897: dwrite d1/dad/db5/d6/da3/d1e/f121 [0,4194304] 0 2024-04-27T04:10:19.864 INFO:tasks.workunit.client.0.smithi162.stdout:7/823: fsync d3/d5/d5b/d60/fa5 0 2024-04-27T04:10:19.866 INFO:tasks.workunit.client.1.smithi181.stdout:4/912: mknod d0/d15/de/d19/dfe/c12d 0 2024-04-27T04:10:19.866 INFO:tasks.workunit.client.1.smithi181.stdout:9/854: truncate d0/d44/d4f/df7/fd4 4993308 0 2024-04-27T04:10:19.868 INFO:tasks.workunit.client.0.smithi162.stdout:7/824: write d3/d5/d5b/d83/de/d2b/d81/daa/dbd/fa0 [2221636,125332] 0 2024-04-27T04:10:19.868 INFO:tasks.workunit.client.0.smithi162.stdout:6/813: creat d6/d1a/d34/d7f/f10d x:0 0 0 2024-04-27T04:10:19.869 INFO:tasks.workunit.client.0.smithi162.stdout:7/825: truncate d3/d4f/fc3 645486 0 2024-04-27T04:10:19.870 INFO:tasks.workunit.client.1.smithi181.stdout:1/898: readlink d1/dad/db5/d6/lc0 0 2024-04-27T04:10:19.873 INFO:tasks.workunit.client.0.smithi162.stdout:2/765: sync 2024-04-27T04:10:19.873 INFO:tasks.workunit.client.0.smithi162.stdout:0/895: sync 2024-04-27T04:10:19.873 INFO:tasks.workunit.client.1.smithi181.stdout:1/899: dread d1/d2/d89/fef [0,4194304] 0 2024-04-27T04:10:19.875 INFO:tasks.workunit.client.1.smithi181.stdout:8/920: dread d1/d6/d59/ffe [0,4194304] 0 2024-04-27T04:10:19.883 INFO:tasks.workunit.client.1.smithi181.stdout:3/939: dread d0/d6c/d52/d44/d56/fa6 [0,4194304] 0 2024-04-27T04:10:19.917 INFO:tasks.workunit.client.0.smithi162.stdout:8/848: mkdir d0/d10/d2b/d2d/d2f/d6b/dd3/d111 0 2024-04-27T04:10:19.919 INFO:tasks.workunit.client.0.smithi162.stdout:8/849: truncate d0/de/d6e/f51 250926 0 2024-04-27T04:10:19.920 INFO:tasks.workunit.client.0.smithi162.stdout:9/751: dwrite d3/d23/d1e/f4a [0,4194304] 0 2024-04-27T04:10:19.923 INFO:tasks.workunit.client.0.smithi162.stdout:8/850: truncate d0/d10/d2b/d2d/d2f/d39/fe2 4833736 0 2024-04-27T04:10:19.939 INFO:tasks.workunit.client.0.smithi162.stdout:1/714: sync 2024-04-27T04:10:19.945 INFO:tasks.workunit.client.1.smithi181.stdout:6/855: sync 2024-04-27T04:10:19.945 INFO:tasks.workunit.client.1.smithi181.stdout:7/953: sync 2024-04-27T04:10:19.974 INFO:tasks.workunit.client.1.smithi181.stdout:4/913: mknod d0/d15/d76/db1/c12e 0 2024-04-27T04:10:19.975 INFO:tasks.workunit.client.0.smithi162.stdout:4/748: write d2/d25/d1c/d39/f7f [54902,129933] 0 2024-04-27T04:10:19.976 INFO:tasks.workunit.client.1.smithi181.stdout:9/855: read d0/d44/d4b/f83 [1193463,32638] 0 2024-04-27T04:10:19.982 INFO:tasks.workunit.client.1.smithi181.stdout:1/900: read d1/dad/db5/d6/da3/d1e/d48/da6/fe3 [79016,76961] 0 2024-04-27T04:10:19.984 INFO:tasks.workunit.client.1.smithi181.stdout:1/901: read d1/dad/db5/d6/da3/d1e/d48/da6/fe3 [69093,122953] 0 2024-04-27T04:10:19.985 INFO:tasks.workunit.client.1.smithi181.stdout:1/902: chown d1/dad/db5/d6/da3/d1e/d20/f9d 37259 1 2024-04-27T04:10:19.995 INFO:tasks.workunit.client.0.smithi162.stdout:2/766: mknod d0/d4/d17/d2b/d64/d89/dbc/cf2 0 2024-04-27T04:10:19.997 INFO:tasks.workunit.client.0.smithi162.stdout:2/767: truncate d0/d4/d17/d2b/d64/fed 171814 0 2024-04-27T04:10:20.002 INFO:tasks.workunit.client.0.smithi162.stdout:1/715: creat d0/d11/d50/d53/d93/d96/fef x:0 0 0 2024-04-27T04:10:20.003 INFO:tasks.workunit.client.0.smithi162.stdout:1/716: fdatasync d0/d11/d3e/f6c 0 2024-04-27T04:10:20.003 INFO:tasks.workunit.client.0.smithi162.stdout:3/781: symlink d1/d18/d23/d34/daa/de6/d107/l115 0 2024-04-27T04:10:20.010 INFO:tasks.workunit.client.0.smithi162.stdout:1/717: dwrite d0/d7/d10/de4/fe6 [0,4194304] 0 2024-04-27T04:10:20.011 INFO:tasks.workunit.client.0.smithi162.stdout:1/718: chown d0/d11/d44/d66/fe5 31155104 1 2024-04-27T04:10:20.013 INFO:tasks.workunit.client.0.smithi162.stdout:0/896: mkdir d5/dc/d1e/d46/d48/d9e/dce/dd1/d116/d57/d12a 0 2024-04-27T04:10:20.016 INFO:tasks.workunit.client.0.smithi162.stdout:9/752: creat d3/d13/d2d/d3b/d7b/fe4 x:0 0 0 2024-04-27T04:10:20.017 INFO:tasks.workunit.client.1.smithi181.stdout:6/856: truncate d7/dc/dc3/dd1/dd9/fa7 1806159 0 2024-04-27T04:10:20.017 INFO:tasks.workunit.client.0.smithi162.stdout:9/753: write d3/d23/fd9 [400774,85847] 0 2024-04-27T04:10:20.024 INFO:tasks.workunit.client.1.smithi181.stdout:3/940: link d0/d6c/d66/dc2/lf9 d0/d6c/d52/dbb/dca/l130 0 2024-04-27T04:10:20.027 INFO:tasks.workunit.client.1.smithi181.stdout:7/954: creat d1/df/d1a/f121 x:0 0 0 2024-04-27T04:10:20.028 INFO:tasks.workunit.client.0.smithi162.stdout:4/749: symlink d2/d25/d1c/lf2 0 2024-04-27T04:10:20.036 INFO:tasks.workunit.client.0.smithi162.stdout:6/814: mkdir d6/db/d76/d10e 0 2024-04-27T04:10:20.038 INFO:tasks.workunit.client.0.smithi162.stdout:1/719: unlink d0/d11/d3e/d59/fd6 0 2024-04-27T04:10:20.040 INFO:tasks.workunit.client.0.smithi162.stdout:0/897: mknod d5/dc/d1e/d20/dba/df9/d8b/c12b 0 2024-04-27T04:10:20.041 INFO:tasks.workunit.client.0.smithi162.stdout:0/898: chown d5/l40 18581545 1 2024-04-27T04:10:20.045 INFO:tasks.workunit.client.0.smithi162.stdout:9/754: rename d3/d13/l5f to d3/d23/d7c/le5 0 2024-04-27T04:10:20.047 INFO:tasks.workunit.client.0.smithi162.stdout:4/750: mkdir d2/d25/d1c/d39/df3 0 2024-04-27T04:10:20.049 INFO:tasks.workunit.client.1.smithi181.stdout:1/903: getdents d1/dad/db5/d6/da3 0 2024-04-27T04:10:20.051 INFO:tasks.workunit.client.1.smithi181.stdout:3/941: link d0/d6c/d52/l38 d0/d28/l131 0 2024-04-27T04:10:20.052 INFO:tasks.workunit.client.0.smithi162.stdout:9/755: dwrite d3/d11/fca [0,4194304] 0 2024-04-27T04:10:20.052 INFO:tasks.workunit.client.1.smithi181.stdout:1/904: read d1/dad/db5/d6/d6a/f75 [3814621,102765] 0 2024-04-27T04:10:20.053 INFO:tasks.workunit.client.0.smithi162.stdout:0/899: mkdir d5/dc/d1e/d41/d79/d12c 0 2024-04-27T04:10:20.053 INFO:tasks.workunit.client.1.smithi181.stdout:1/905: chown d1/dad/db5/d6/db/fe9 1134280756 1 2024-04-27T04:10:20.054 INFO:tasks.workunit.client.0.smithi162.stdout:0/900: dread - d5/d26/da2/dc0/f120 zero size 2024-04-27T04:10:20.056 INFO:tasks.workunit.client.0.smithi162.stdout:6/815: rename d6/db/d32/f5b to d6/d1a/d34/d3a/d68/d6c/f10f 0 2024-04-27T04:10:20.057 INFO:tasks.workunit.client.0.smithi162.stdout:6/816: readlink d6/db/dc0/le1 0 2024-04-27T04:10:20.058 INFO:tasks.workunit.client.1.smithi181.stdout:3/942: dwrite d0/d6c/d52/dbb/d98/da7/fae [0,4194304] 0 2024-04-27T04:10:20.059 INFO:tasks.workunit.client.0.smithi162.stdout:6/817: dread d6/db/d32/f38 [0,4194304] 0 2024-04-27T04:10:20.063 INFO:tasks.workunit.client.1.smithi181.stdout:7/955: getdents d1/d22/d15/d5f/dba 0 2024-04-27T04:10:20.064 INFO:tasks.workunit.client.0.smithi162.stdout:9/756: fdatasync d3/d11/d45/f55 0 2024-04-27T04:10:20.065 INFO:tasks.workunit.client.1.smithi181.stdout:3/943: dread d0/d6c/d52/d44/dd5/f69 [0,4194304] 0 2024-04-27T04:10:20.066 INFO:tasks.workunit.client.1.smithi181.stdout:1/906: dread - d1/dad/db5/d6/da3/d52/fcd zero size 2024-04-27T04:10:20.074 INFO:tasks.workunit.client.1.smithi181.stdout:7/956: creat d1/d2/d35/d67/dbf/f122 x:0 0 0 2024-04-27T04:10:20.075 INFO:tasks.workunit.client.1.smithi181.stdout:1/907: fdatasync d1/dad/db5/d6/db/d2d/f55 0 2024-04-27T04:10:20.078 INFO:tasks.workunit.client.0.smithi162.stdout:9/757: creat d3/d39/fe6 x:0 0 0 2024-04-27T04:10:20.079 INFO:tasks.workunit.client.1.smithi181.stdout:4/914: creat d0/d15/d25/d2a/d61/d11f/f12f x:0 0 0 2024-04-27T04:10:20.081 INFO:tasks.workunit.client.1.smithi181.stdout:7/957: read d1/d22/f95 [1697590,50080] 0 2024-04-27T04:10:20.084 INFO:tasks.workunit.client.0.smithi162.stdout:4/751: creat d2/d25/d1c/ff4 x:0 0 0 2024-04-27T04:10:20.085 INFO:tasks.workunit.client.1.smithi181.stdout:4/915: dwrite d0/d15/f2 [4194304,4194304] 0 2024-04-27T04:10:20.088 INFO:tasks.workunit.client.0.smithi162.stdout:6/818: mkdir d6/d1a/d34/d7f/d80/db3/dcd/d110 0 2024-04-27T04:10:20.091 INFO:tasks.workunit.client.1.smithi181.stdout:7/958: write d1/d22/d15/d29/f43 [2090530,65979] 0 2024-04-27T04:10:20.093 INFO:tasks.workunit.client.1.smithi181.stdout:1/908: dread d1/dad/db5/d6/da3/d1e/d20/f41 [0,4194304] 0 2024-04-27T04:10:20.095 INFO:tasks.workunit.client.1.smithi181.stdout:1/909: dread - d1/dad/db5/d6/db/f91 zero size 2024-04-27T04:10:20.095 INFO:tasks.workunit.client.1.smithi181.stdout:4/916: readlink d0/d15/de/l87 0 2024-04-27T04:10:20.097 INFO:tasks.workunit.client.0.smithi162.stdout:4/752: rename d2/d25/d1c/d39/d90/ced to d2/d54/d7a/d95/cf5 0 2024-04-27T04:10:20.098 INFO:tasks.workunit.client.1.smithi181.stdout:3/944: link d0/d6c/d52/d44/dd5/d58/de4/dc5/dcf/cfd d0/d9f/d110/c132 0 2024-04-27T04:10:20.098 INFO:tasks.workunit.client.0.smithi162.stdout:4/753: read d2/d25/f7 [1292124,92680] 0 2024-04-27T04:10:20.099 INFO:tasks.workunit.client.0.smithi162.stdout:6/819: symlink d6/d1a/d34/d3a/d68/d6c/d67/l111 0 2024-04-27T04:10:20.103 INFO:tasks.workunit.client.0.smithi162.stdout:0/901: getdents d5/d26/d4a/def 0 2024-04-27T04:10:20.103 INFO:tasks.workunit.client.1.smithi181.stdout:3/945: dwrite d0/de/f129 [0,4194304] 0 2024-04-27T04:10:20.106 INFO:tasks.workunit.client.1.smithi181.stdout:1/910: mknod d1/d2/d64/ddc/c125 0 2024-04-27T04:10:20.109 INFO:tasks.workunit.client.0.smithi162.stdout:4/754: unlink d2/da/l86 0 2024-04-27T04:10:20.110 INFO:tasks.workunit.client.1.smithi181.stdout:4/917: fdatasync d0/d15/de/fb7 0 2024-04-27T04:10:20.118 INFO:tasks.workunit.client.0.smithi162.stdout:0/902: mknod d5/dc/d1e/d41/d79/c12d 0 2024-04-27T04:10:20.119 INFO:tasks.workunit.client.0.smithi162.stdout:0/903: read d5/d26/d4a/f4c [638438,104051] 0 2024-04-27T04:10:20.119 INFO:tasks.workunit.client.1.smithi181.stdout:3/946: truncate d0/d9f/f95 295045 0 2024-04-27T04:10:20.122 INFO:tasks.workunit.client.0.smithi162.stdout:4/755: creat d2/d25/d1c/d39/d70/dcd/df0/de7/de9/ff6 x:0 0 0 2024-04-27T04:10:20.126 INFO:tasks.workunit.client.1.smithi181.stdout:4/918: creat d0/d15/d25/d2a/da0/dc7/d109/d7c/f130 x:0 0 0 2024-04-27T04:10:20.128 INFO:tasks.workunit.client.1.smithi181.stdout:7/959: creat d1/d57/f123 x:0 0 0 2024-04-27T04:10:20.129 INFO:tasks.workunit.client.1.smithi181.stdout:7/960: write d1/d2/d35/d67/f96 [2723402,3887] 0 2024-04-27T04:10:20.133 INFO:tasks.workunit.client.1.smithi181.stdout:1/911: fsync d1/dad/db5/d6/da3/d1e/f79 0 2024-04-27T04:10:20.137 INFO:tasks.workunit.client.0.smithi162.stdout:0/904: link d5/fac d5/dc/d1e/d46/d48/d9e/dce/f12e 0 2024-04-27T04:10:20.143 INFO:tasks.workunit.client.1.smithi181.stdout:7/961: dread d1/d22/fc0 [0,4194304] 0 2024-04-27T04:10:20.144 INFO:tasks.workunit.client.1.smithi181.stdout:1/912: mkdir d1/dad/db5/d6/da3/d60/d126 0 2024-04-27T04:10:20.146 INFO:tasks.workunit.client.1.smithi181.stdout:1/913: readlink d1/d2/d89/lbb 0 2024-04-27T04:10:20.146 INFO:tasks.workunit.client.0.smithi162.stdout:0/905: creat d5/dc/d1e/d46/d48/d9e/dce/dd1/d116/ddd/d117/f12f x:0 0 0 2024-04-27T04:10:20.149 INFO:tasks.workunit.client.1.smithi181.stdout:4/919: unlink d0/d15/c12c 0 2024-04-27T04:10:20.150 INFO:tasks.workunit.client.1.smithi181.stdout:7/962: dwrite d1/d2/d1c/d37/f110 [0,4194304] 0 2024-04-27T04:10:20.163 INFO:tasks.workunit.client.0.smithi162.stdout:0/906: getdents d5/dc/d1e/d20/dec/df1/d10a 0 2024-04-27T04:10:20.164 INFO:tasks.workunit.client.1.smithi181.stdout:1/914: dread d1/dad/db5/d6/db/d2d/d36/f5e [0,4194304] 0 2024-04-27T04:10:20.166 INFO:tasks.workunit.client.0.smithi162.stdout:0/907: stat d5/dc/d1e/faf 0 2024-04-27T04:10:20.170 INFO:tasks.workunit.client.0.smithi162.stdout:0/908: fsync d5/dc/d1e/fc7 0 2024-04-27T04:10:20.171 INFO:tasks.workunit.client.1.smithi181.stdout:1/915: symlink d1/dad/db5/d6/da3/d1e/d48/da6/da4/dda/l127 0 2024-04-27T04:10:20.172 INFO:tasks.workunit.client.0.smithi162.stdout:0/909: chown d5/dc/d1e/d46/d48/d9e/dce/dd1/d116/l75 60 1 2024-04-27T04:10:20.173 INFO:tasks.workunit.client.0.smithi162.stdout:6/820: symlink d6/d1a/d34/d7f/d80/l112 0 2024-04-27T04:10:20.174 INFO:tasks.workunit.client.1.smithi181.stdout:1/916: truncate d1/dad/db5/d6/db/f91 668289 0 2024-04-27T04:10:20.180 INFO:tasks.workunit.client.1.smithi181.stdout:1/917: symlink d1/dad/db5/d6/da3/d1e/dbc/df6/l128 0 2024-04-27T04:10:20.182 INFO:tasks.workunit.client.1.smithi181.stdout:1/918: chown d1/dad/db5/d6/da3/d52/l69 62871324 1 2024-04-27T04:10:20.182 INFO:tasks.workunit.client.1.smithi181.stdout:1/919: stat d1/dad/db5/d6 0 2024-04-27T04:10:20.183 INFO:tasks.workunit.client.0.smithi162.stdout:0/910: getdents d5/dc/d1e/d20/dba/df9 0 2024-04-27T04:10:20.185 INFO:tasks.workunit.client.0.smithi162.stdout:9/758: creat d3/d13/d2d/d30/d7f/fe7 x:0 0 0 2024-04-27T04:10:20.188 INFO:tasks.workunit.client.0.smithi162.stdout:9/759: creat d3/d23/d5d/fe8 x:0 0 0 2024-04-27T04:10:20.189 INFO:tasks.workunit.client.0.smithi162.stdout:0/911: dwrite d5/dc/d1e/d46/df3/f113 [0,4194304] 0 2024-04-27T04:10:20.190 INFO:tasks.workunit.client.0.smithi162.stdout:9/760: read d3/db9/dcc/fcd [1193870,694] 0 2024-04-27T04:10:20.194 INFO:tasks.workunit.client.0.smithi162.stdout:0/912: creat d5/dc/d1e/d20/dba/df9/d8b/f130 x:0 0 0 2024-04-27T04:10:20.195 INFO:tasks.workunit.client.1.smithi181.stdout:8/921: dwrite d1/d6/d2f/d72/fc6 [0,4194304] 0 2024-04-27T04:10:20.199 INFO:tasks.workunit.client.1.smithi181.stdout:8/922: mknod d1/d6/d2f/d11e/dd9/c144 0 2024-04-27T04:10:20.201 INFO:tasks.workunit.client.1.smithi181.stdout:9/856: write d0/d44/d4b/d5f/f10a [2047703,37600] 0 2024-04-27T04:10:20.203 INFO:tasks.workunit.client.0.smithi162.stdout:8/851: write d0/de/d20/d6c/f6d [3537411,84534] 0 2024-04-27T04:10:20.204 INFO:tasks.workunit.client.1.smithi181.stdout:8/923: creat d1/d6/d2f/d30/d9d/dea/db3/f145 x:0 0 0 2024-04-27T04:10:20.207 INFO:tasks.workunit.client.1.smithi181.stdout:9/857: chown d0/d44/d4b/d43/d88/da1/dfc/dbd/dd1/dde/d104/l2f 26529 1 2024-04-27T04:10:20.300 INFO:tasks.workunit.client.0.smithi162.stdout:3/782: rmdir d1 39 2024-04-27T04:10:20.302 INFO:tasks.workunit.client.1.smithi181.stdout:9/858: mkdir d0/d44/d4b/d5f/d8a/d120 0 2024-04-27T04:10:20.304 INFO:tasks.workunit.client.0.smithi162.stdout:3/783: chown d1/d21/d47/d57/fae 230689234 1 2024-04-27T04:10:20.305 INFO:tasks.workunit.client.0.smithi162.stdout:8/852: truncate d0/de/d20/d6c/fa5 1233948 0 2024-04-27T04:10:20.307 INFO:tasks.workunit.client.0.smithi162.stdout:8/853: dread d0/de/d6e/d71/fb3 [0,4194304] 0 2024-04-27T04:10:20.309 INFO:tasks.workunit.client.0.smithi162.stdout:3/784: creat d1/d18/d5d/d5f/d66/dd2/f116 x:0 0 0 2024-04-27T04:10:20.309 INFO:tasks.workunit.client.0.smithi162.stdout:5/844: dwrite d8/de3/deb/d9f/f103 [0,4194304] 0 2024-04-27T04:10:20.316 INFO:tasks.workunit.client.0.smithi162.stdout:5/845: fsync d8/d1a/f51 0 2024-04-27T04:10:20.317 INFO:tasks.workunit.client.0.smithi162.stdout:5/846: chown d8/de3/deb/l7c 66207 1 2024-04-27T04:10:20.320 INFO:tasks.workunit.client.1.smithi181.stdout:4/920: mknod d0/d15/d25/d2a/c131 0 2024-04-27T04:10:20.321 INFO:tasks.workunit.client.1.smithi181.stdout:4/921: write d0/d15/d25/d2a/d36/f42 [8296629,71365] 0 2024-04-27T04:10:20.323 INFO:tasks.workunit.client.0.smithi162.stdout:3/785: mkdir d1/d18/d117 0 2024-04-27T04:10:20.326 INFO:tasks.workunit.client.1.smithi181.stdout:6/857: rename d7/dc/d1c/l77 to d7/dc/dc3/d72/daa/d110/l120 0 2024-04-27T04:10:20.327 INFO:tasks.workunit.client.1.smithi181.stdout:6/858: chown d7/dc/dc3 11 1 2024-04-27T04:10:20.337 INFO:tasks.workunit.client.0.smithi162.stdout:1/720: dwrite d0/d7/d10/de4/fe6 [4194304,4194304] 0 2024-04-27T04:10:20.342 INFO:tasks.workunit.client.0.smithi162.stdout:1/721: truncate d0/d7/d3c/d54/f60 418759 0 2024-04-27T04:10:20.343 INFO:tasks.workunit.client.0.smithi162.stdout:2/768: truncate d0/d4/d17/db4/ff0 6524776 0 2024-04-27T04:10:20.343 INFO:tasks.workunit.client.0.smithi162.stdout:5/847: dread d8/de3/deb/d9f/f64 [0,4194304] 0 2024-04-27T04:10:20.343 INFO:tasks.workunit.client.0.smithi162.stdout:5/848: fsync d8/d1a/d23/f50 0 2024-04-27T04:10:20.348 INFO:tasks.workunit.client.0.smithi162.stdout:2/769: dwrite d0/d4/d2a/d3b/d49/d5a/d63/d70/d8e/fe5 [0,4194304] 0 2024-04-27T04:10:20.350 INFO:tasks.workunit.client.0.smithi162.stdout:5/849: fdatasync d8/d9/fb0 0 2024-04-27T04:10:20.353 INFO:tasks.workunit.client.0.smithi162.stdout:1/722: symlink d0/d11/d44/d66/lf0 0 2024-04-27T04:10:20.359 INFO:tasks.workunit.client.0.smithi162.stdout:5/850: rename d8/d1a/d23/d40/fcd to d8/d9/d55/f10d 0 2024-04-27T04:10:20.360 INFO:tasks.workunit.client.0.smithi162.stdout:5/851: chown d8/d9/d55/ld5 325997 1 2024-04-27T04:10:20.362 INFO:tasks.workunit.client.1.smithi181.stdout:4/922: symlink d0/d15/d25/d2b/d6d/d82/l132 0 2024-04-27T04:10:20.364 INFO:tasks.workunit.client.0.smithi162.stdout:5/852: read d8/d1a/f31 [2474414,89548] 0 2024-04-27T04:10:20.364 INFO:tasks.workunit.client.1.smithi181.stdout:9/859: rename d0/d44/d4b/d50/f6a to d0/d44/d4b/d43/d88/da1/dfc/dbd/dd1/f121 0 2024-04-27T04:10:20.365 INFO:tasks.workunit.client.1.smithi181.stdout:6/859: fsync d7/d27/fc2 0 2024-04-27T04:10:20.368 INFO:tasks.workunit.client.1.smithi181.stdout:4/923: chown d0/d15/d25/d2a/d36/c9c 0 1 2024-04-27T04:10:20.370 INFO:tasks.workunit.client.1.smithi181.stdout:6/860: creat d7/d27/f121 x:0 0 0 2024-04-27T04:10:20.370 INFO:tasks.workunit.client.1.smithi181.stdout:9/860: dwrite d0/f115 [0,4194304] 0 2024-04-27T04:10:20.371 INFO:tasks.workunit.client.0.smithi162.stdout:5/853: dwrite d8/d1a/f81 [0,4194304] 0 2024-04-27T04:10:20.371 INFO:tasks.workunit.client.0.smithi162.stdout:5/854: fsync d8/d2c/fe0 0 2024-04-27T04:10:20.373 INFO:tasks.workunit.client.1.smithi181.stdout:4/924: fsync d0/d15/d76/f95 0 2024-04-27T04:10:20.375 INFO:tasks.workunit.client.0.smithi162.stdout:1/723: dread d0/d11/f42 [0,4194304] 0 2024-04-27T04:10:20.376 INFO:tasks.workunit.client.1.smithi181.stdout:6/861: mkdir d7/dc/d1c/d34/d3f/d7e/d122 0 2024-04-27T04:10:20.381 INFO:tasks.workunit.client.0.smithi162.stdout:1/724: fdatasync d0/d11/d50/d53/f84 0 2024-04-27T04:10:20.382 INFO:tasks.workunit.client.0.smithi162.stdout:1/725: chown d0/d7/d10/f26 2802 1 2024-04-27T04:10:20.383 INFO:tasks.workunit.client.0.smithi162.stdout:5/855: mknod d8/d1a/d23/dbc/d66/c10e 0 2024-04-27T04:10:20.387 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:10:19 smithi162 ceph-mon[26841]: from='client.24423 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-27T04:10:20.387 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:10:19 smithi162 ceph-mon[26841]: from='client.? 172.21.15.162:0/1347577413' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-27T04:10:20.388 INFO:tasks.workunit.client.0.smithi162.stdout:1/726: readlink d0/d11/d44/lcf 0 2024-04-27T04:10:20.392 INFO:tasks.workunit.client.1.smithi181.stdout:4/925: creat d0/d15/d25/d2a/da0/dc7/d12a/f133 x:0 0 0 2024-04-27T04:10:20.393 INFO:tasks.workunit.client.0.smithi162.stdout:2/770: rmdir d0/d20 39 2024-04-27T04:10:20.393 INFO:tasks.workunit.client.0.smithi162.stdout:5/856: dread d8/d1a/d23/f50 [0,4194304] 0 2024-04-27T04:10:20.395 INFO:tasks.workunit.client.0.smithi162.stdout:1/727: unlink d0/d11/d3e/d59/d7f/ca2 0 2024-04-27T04:10:20.396 INFO:tasks.workunit.client.0.smithi162.stdout:2/771: mknod d0/d4/d2a/d3b/d49/d5a/d63/d70/cf3 0 2024-04-27T04:10:20.397 INFO:tasks.workunit.client.1.smithi181.stdout:9/861: dread d0/fe8 [0,4194304] 0 2024-04-27T04:10:20.399 INFO:tasks.workunit.client.1.smithi181.stdout:4/926: mkdir d0/d15/da5/d134 0 2024-04-27T04:10:20.402 INFO:tasks.workunit.client.0.smithi162.stdout:2/772: rename d0/d24/ccb to d0/d4/d17/d2b/d64/d89/cf4 0 2024-04-27T04:10:20.404 INFO:tasks.workunit.client.1.smithi181.stdout:9/862: dwrite d0/d44/d4b/d43/d88/da1/dfc/dbd/df3/da0/ddb/f106 [0,4194304] 0 2024-04-27T04:10:20.407 INFO:tasks.workunit.client.1.smithi181.stdout:9/863: fdatasync d0/d44/d4b/d5f/d8a/fc0 0 2024-04-27T04:10:20.409 INFO:tasks.workunit.client.0.smithi162.stdout:2/773: fsync d0/d4/d2a/d3b/d49/d5a/d63/d84/f99 0 2024-04-27T04:10:20.410 INFO:tasks.workunit.client.1.smithi181.stdout:9/864: read d0/d44/d4b/d43/d88/da1/dfc/dbd/df3/fa5 [1681780,32540] 0 2024-04-27T04:10:20.412 INFO:tasks.workunit.client.0.smithi162.stdout:2/774: unlink d0/d4/d2a/d3b/d49/d5a/d63/d70/c94 0 2024-04-27T04:10:20.413 INFO:tasks.workunit.client.0.smithi162.stdout:1/728: dread d0/d80/d89/f8f [0,4194304] 0 2024-04-27T04:10:20.422 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:10:19 smithi181 ceph-mon[33774]: from='client.24423 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-27T04:10:20.422 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:10:19 smithi181 ceph-mon[33774]: from='client.? 172.21.15.162:0/1347577413' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-27T04:10:20.423 INFO:tasks.workunit.client.0.smithi162.stdout:1/729: getdents d0/d7/d3c/d54 0 2024-04-27T04:10:20.424 INFO:tasks.workunit.client.0.smithi162.stdout:1/730: dread - d0/d7/d3c/d54/dc9/fca zero size 2024-04-27T04:10:20.426 INFO:tasks.workunit.client.0.smithi162.stdout:4/756: dwrite d2/d25/d1c/d39/d90/dad/f65 [0,4194304] 0 2024-04-27T04:10:20.427 INFO:tasks.workunit.client.0.smithi162.stdout:3/786: creat d1/d18/d5d/f118 x:0 0 0 2024-04-27T04:10:20.434 INFO:tasks.workunit.client.0.smithi162.stdout:3/787: dwrite d1/d18/d23/d34/dad/dc5/fcb [0,4194304] 0 2024-04-27T04:10:20.459 INFO:tasks.workunit.client.1.smithi181.stdout:9/865: symlink d0/d44/d4b/d43/d89/d11d/l122 0 2024-04-27T04:10:20.459 INFO:tasks.workunit.client.1.smithi181.stdout:3/947: mkdir d0/d6c/d52/d44/dd5/d133 0 2024-04-27T04:10:20.464 INFO:tasks.workunit.client.1.smithi181.stdout:3/948: chown d0/de/d7d/df5/l10d 0 1 2024-04-27T04:10:20.464 INFO:tasks.workunit.client.1.smithi181.stdout:7/963: dwrite d1/d2/d35/d67/f51 [0,4194304] 0 2024-04-27T04:10:20.465 INFO:tasks.workunit.client.1.smithi181.stdout:3/949: write d0/f71 [4258118,112697] 0 2024-04-27T04:10:20.468 INFO:tasks.workunit.client.1.smithi181.stdout:9/866: truncate d0/d44/d4b/d43/d89/fc5 362630 0 2024-04-27T04:10:20.472 INFO:tasks.workunit.client.0.smithi162.stdout:6/821: truncate d6/d1a/d34/d3a/d68/fdb 2970793 0 2024-04-27T04:10:20.477 INFO:tasks.workunit.client.0.smithi162.stdout:1/731: creat d0/d11/d50/d53/ff1 x:0 0 0 2024-04-27T04:10:20.482 INFO:tasks.workunit.client.0.smithi162.stdout:7/826: unlink d3/d5/d5b/fe6 0 2024-04-27T04:10:20.483 INFO:tasks.workunit.client.0.smithi162.stdout:7/827: readlink d3/d5/d5b/d60/l68 0 2024-04-27T04:10:20.484 INFO:tasks.workunit.client.1.smithi181.stdout:3/950: mkdir d0/d6c/d52/d44/d56/d134 0 2024-04-27T04:10:20.485 INFO:tasks.workunit.client.0.smithi162.stdout:9/761: write d3/d13/d15/f18 [3173077,15285] 0 2024-04-27T04:10:20.491 INFO:tasks.workunit.client.0.smithi162.stdout:7/828: symlink d3/d5/d5b/d83/l112 0 2024-04-27T04:10:20.493 INFO:tasks.workunit.client.0.smithi162.stdout:9/762: rmdir d3/d13/d2d/d30/d7f 39 2024-04-27T04:10:20.495 INFO:tasks.workunit.client.1.smithi181.stdout:3/951: getdents d0/d6c/d66/dc2 0 2024-04-27T04:10:20.496 INFO:tasks.workunit.client.1.smithi181.stdout:3/952: write d0/d6c/d52/f65 [818602,5071] 0 2024-04-27T04:10:20.497 INFO:tasks.workunit.client.0.smithi162.stdout:7/829: dwrite d3/d5/d5b/d83/de/d2b/d81/d96/dc8/fd4 [0,4194304] 0 2024-04-27T04:10:20.499 INFO:tasks.workunit.client.0.smithi162.stdout:9/763: creat d3/d23/d1e/fe9 x:0 0 0 2024-04-27T04:10:20.502 INFO:tasks.workunit.client.0.smithi162.stdout:4/757: getdents d2/d25/d1c/d39/d3b/d47 0 2024-04-27T04:10:20.502 INFO:tasks.workunit.client.0.smithi162.stdout:4/758: chown d2/d25/d1c/d39/d70/dcd/df0/de7/f29 816228 1 2024-04-27T04:10:20.505 INFO:tasks.workunit.client.0.smithi162.stdout:6/822: getdents d6/db/dc0 0 2024-04-27T04:10:20.509 INFO:tasks.workunit.client.1.smithi181.stdout:3/953: mkdir d0/d6c/d52/d44/d74/d109/d135 0 2024-04-27T04:10:20.510 INFO:tasks.workunit.client.0.smithi162.stdout:4/759: write d2/da/faf [801861,101849] 0 2024-04-27T04:10:20.512 INFO:tasks.workunit.client.0.smithi162.stdout:7/830: mkdir d3/d5/d113 0 2024-04-27T04:10:20.513 INFO:tasks.workunit.client.1.smithi181.stdout:3/954: truncate d0/de/d7d/dbc/fd9 5088465 0 2024-04-27T04:10:20.514 INFO:tasks.workunit.client.0.smithi162.stdout:9/764: creat d3/d13/d2d/d3b/d72/fea x:0 0 0 2024-04-27T04:10:20.516 INFO:tasks.workunit.client.1.smithi181.stdout:3/955: rmdir d0/d6c/d52/d44/dd5/d6b 39 2024-04-27T04:10:20.521 INFO:tasks.workunit.client.0.smithi162.stdout:4/760: mkdir d2/d25/d1c/d24/dd2/df7 0 2024-04-27T04:10:20.522 INFO:tasks.workunit.client.0.smithi162.stdout:4/761: fsync d2/d25/d1c/d39/d3b/da7/fda 0 2024-04-27T04:10:20.524 INFO:tasks.workunit.client.1.smithi181.stdout:3/956: dread d0/d6c/d52/f59 [4194304,4194304] 0 2024-04-27T04:10:20.530 INFO:tasks.workunit.client.0.smithi162.stdout:9/765: dread d3/d23/d1e/d43/f5a [0,4194304] 0 2024-04-27T04:10:20.535 INFO:tasks.workunit.client.1.smithi181.stdout:8/924: truncate d1/d6/f26 1125120 0 2024-04-27T04:10:20.536 INFO:tasks.workunit.client.1.smithi181.stdout:8/925: stat d1/d6/d59/df1/f115 0 2024-04-27T04:10:20.537 INFO:tasks.workunit.client.0.smithi162.stdout:0/913: sync 2024-04-27T04:10:20.537 INFO:tasks.workunit.client.0.smithi162.stdout:9/766: creat d3/d13/d2d/feb x:0 0 0 2024-04-27T04:10:20.538 INFO:tasks.workunit.client.1.smithi181.stdout:8/926: creat d1/d6/d2f/d11e/dec/f146 x:0 0 0 2024-04-27T04:10:20.539 INFO:tasks.workunit.client.1.smithi181.stdout:8/927: stat d1/d6/d6f/dd2/c130 0 2024-04-27T04:10:20.540 INFO:tasks.workunit.client.0.smithi162.stdout:0/914: fsync d5/dc/d1e/d20/dec/df1/db4/fca 0 2024-04-27T04:10:20.541 INFO:tasks.workunit.client.1.smithi181.stdout:8/928: fdatasync d1/d6/f107 0 2024-04-27T04:10:20.546 INFO:tasks.workunit.client.1.smithi181.stdout:8/929: dwrite d1/f82 [0,4194304] 0 2024-04-27T04:10:20.549 INFO:tasks.workunit.client.1.smithi181.stdout:8/930: creat d1/d6/d6f/dd2/f147 x:0 0 0 2024-04-27T04:10:20.550 INFO:tasks.workunit.client.0.smithi162.stdout:9/767: dread d3/f5 [0,4194304] 0 2024-04-27T04:10:20.550 INFO:tasks.workunit.client.1.smithi181.stdout:6/862: write d7/dc/dc3/dd1/dd9/fa7 [1004765,103339] 0 2024-04-27T04:10:20.559 INFO:tasks.workunit.client.0.smithi162.stdout:9/768: link d3/d11/l25 d3/d13/d2d/d3b/d54/dda/lec 0 2024-04-27T04:10:20.560 INFO:tasks.workunit.client.1.smithi181.stdout:8/931: rmdir d1/d20/d10d 0 2024-04-27T04:10:20.561 INFO:tasks.workunit.client.0.smithi162.stdout:5/857: dwrite d8/d9/d5b/d96/fc2 [0,4194304] 0 2024-04-27T04:10:20.563 INFO:tasks.workunit.client.1.smithi181.stdout:8/932: fdatasync d1/d6/d6f/dd2/d13f/d25/fc7 0 2024-04-27T04:10:20.565 INFO:tasks.workunit.client.0.smithi162.stdout:5/858: chown d8/d9/d5b/fb4 20696823 1 2024-04-27T04:10:20.567 INFO:tasks.workunit.client.0.smithi162.stdout:5/859: write d8/d1a/d23/d40/f46 [1163766,73018] 0 2024-04-27T04:10:20.577 INFO:tasks.workunit.client.1.smithi181.stdout:6/863: dread d7/dc/d1c/d7f/d3e/f47 [0,4194304] 0 2024-04-27T04:10:20.579 INFO:tasks.workunit.client.1.smithi181.stdout:6/864: creat d7/dc/dc3/dd1/dd9/f123 x:0 0 0 2024-04-27T04:10:20.582 INFO:tasks.workunit.client.1.smithi181.stdout:6/865: creat d7/dc/dc3/d72/daa/f124 x:0 0 0 2024-04-27T04:10:20.585 INFO:tasks.workunit.client.1.smithi181.stdout:6/866: creat d7/dc/d1c/d88/d6d/f125 x:0 0 0 2024-04-27T04:10:20.586 INFO:tasks.workunit.client.1.smithi181.stdout:6/867: stat d7/dc/d1c/d88/d6d 0 2024-04-27T04:10:20.586 INFO:tasks.workunit.client.1.smithi181.stdout:9/867: write d0/fbe [173421,59568] 0 2024-04-27T04:10:20.587 INFO:tasks.workunit.client.1.smithi181.stdout:6/868: readlink d7/dc/d1c/d88/d6d/la0 0 2024-04-27T04:10:20.587 INFO:tasks.workunit.client.0.smithi162.stdout:3/788: write d1/d18/d5d/d6f/fa9 [649459,28414] 0 2024-04-27T04:10:20.589 INFO:tasks.workunit.client.0.smithi162.stdout:1/732: write d0/d11/d3e/d5f/f6a [1581603,63016] 0 2024-04-27T04:10:20.590 INFO:tasks.workunit.client.1.smithi181.stdout:7/964: write d1/d22/d15/d29/dbe/d1b/fac [9102316,22937] 0 2024-04-27T04:10:20.592 INFO:tasks.workunit.client.1.smithi181.stdout:7/965: chown d1/d22/d15/d29/dbe/d1b/fac 9060282 1 2024-04-27T04:10:20.594 INFO:tasks.workunit.client.1.smithi181.stdout:6/869: link d7/dc/d1c/d88/d6d/fb5 d7/dc/dc3/d72/daa/f126 0 2024-04-27T04:10:20.595 INFO:tasks.workunit.client.1.smithi181.stdout:7/966: truncate d1/d2/d1c/d99/dfd/f10c 937439 0 2024-04-27T04:10:20.601 INFO:tasks.workunit.client.1.smithi181.stdout:7/967: unlink d1/d22/d15/d29/dbe/d1b/f8c 0 2024-04-27T04:10:20.602 INFO:tasks.workunit.client.1.smithi181.stdout:6/870: link d7/cee d7/dc/d1c/c127 0 2024-04-27T04:10:20.605 INFO:tasks.workunit.client.1.smithi181.stdout:6/871: creat d7/dc/d1c/d88/db3/d102/f128 x:0 0 0 2024-04-27T04:10:20.606 INFO:tasks.workunit.client.1.smithi181.stdout:7/968: mkdir d1/df/d62/d124 0 2024-04-27T04:10:20.607 INFO:tasks.workunit.client.1.smithi181.stdout:6/872: creat d7/dc/d1c/d7f/d3e/f129 x:0 0 0 2024-04-27T04:10:20.608 INFO:tasks.workunit.client.1.smithi181.stdout:7/969: symlink d1/d2/d35/d67/dbf/d9a/l125 0 2024-04-27T04:10:20.608 INFO:tasks.workunit.client.1.smithi181.stdout:6/873: stat d7/dc/d1c/d88/d6d/f125 0 2024-04-27T04:10:20.609 INFO:tasks.workunit.client.1.smithi181.stdout:3/957: write d0/d6c/d52/d44/dd5/d58/de4/daf/fce [435181,8788] 0 2024-04-27T04:10:20.611 INFO:tasks.workunit.client.1.smithi181.stdout:3/958: truncate d0/d6c/d52/f12e 300567 0 2024-04-27T04:10:20.612 INFO:tasks.workunit.client.1.smithi181.stdout:6/874: symlink d7/dc/dc3/d72/daa/dd3/d103/l12a 0 2024-04-27T04:10:20.612 INFO:tasks.workunit.client.1.smithi181.stdout:3/959: chown d0/cd8 17981029 1 2024-04-27T04:10:20.612 INFO:tasks.workunit.client.1.smithi181.stdout:3/960: stat d0/d6c/d52/d44/dd5 0 2024-04-27T04:10:20.617 INFO:tasks.workunit.client.1.smithi181.stdout:3/961: link d0/d6c/d52/dbb/fb8 d0/d6c/d52/d44/d56/d134/f136 0 2024-04-27T04:10:20.625 INFO:tasks.workunit.client.1.smithi181.stdout:9/868: truncate d0/fe8 1577297 0 2024-04-27T04:10:20.628 INFO:tasks.workunit.client.1.smithi181.stdout:9/869: unlink d0/l3c 0 2024-04-27T04:10:20.630 INFO:tasks.workunit.client.1.smithi181.stdout:7/970: write d1/d2/d1c/d2d/d61/f68 [742565,28419] 0 2024-04-27T04:10:20.633 INFO:tasks.workunit.client.1.smithi181.stdout:7/971: symlink d1/d2/d1c/d99/dfd/l126 0 2024-04-27T04:10:20.633 INFO:tasks.workunit.client.1.smithi181.stdout:6/875: write d7/d27/fc2 [3949199,71555] 0 2024-04-27T04:10:20.634 INFO:tasks.workunit.client.1.smithi181.stdout:6/876: chown d7/dc/d1c/d88/f11f 1644895 1 2024-04-27T04:10:20.636 INFO:tasks.workunit.client.1.smithi181.stdout:3/962: write d0/d6c/d52/fd0 [1571032,69821] 0 2024-04-27T04:10:20.638 INFO:tasks.workunit.client.1.smithi181.stdout:7/972: creat d1/d2/d1c/d37/f127 x:0 0 0 2024-04-27T04:10:20.639 INFO:tasks.workunit.client.1.smithi181.stdout:6/877: creat d7/dc/d1e/f12b x:0 0 0 2024-04-27T04:10:20.643 INFO:tasks.workunit.client.1.smithi181.stdout:9/870: write d0/d44/d4b/f62 [2438003,48300] 0 2024-04-27T04:10:20.648 INFO:tasks.workunit.client.0.smithi162.stdout:6/823: write d6/db/d32/f3b [1252589,41852] 0 2024-04-27T04:10:20.649 INFO:tasks.workunit.client.1.smithi181.stdout:3/963: rmdir d0/d9f/d111 0 2024-04-27T04:10:20.649 INFO:tasks.workunit.client.1.smithi181.stdout:7/973: link d1/d57/ca4 d1/d2/d1c/d2d/d61/c128 0 2024-04-27T04:10:20.649 INFO:tasks.workunit.client.0.smithi162.stdout:7/831: write d3/d5/d5b/d83/f33 [2167654,4509] 0 2024-04-27T04:10:20.650 INFO:tasks.workunit.client.1.smithi181.stdout:6/878: creat d7/dc/d1e/f12c x:0 0 0 2024-04-27T04:10:20.651 INFO:tasks.workunit.client.1.smithi181.stdout:9/871: mknod d0/d44/d4b/d43/d10c/c123 0 2024-04-27T04:10:20.651 INFO:tasks.workunit.client.1.smithi181.stdout:7/974: dread - d1/d22/d15/d5f/d93/df3/ff5 zero size 2024-04-27T04:10:20.656 INFO:tasks.workunit.client.1.smithi181.stdout:6/879: fdatasync d7/d31/f75 0 2024-04-27T04:10:20.659 INFO:tasks.workunit.client.1.smithi181.stdout:6/880: unlink d7/dc/dc3/d72/daa/dd3/ce9 0 2024-04-27T04:10:20.666 INFO:tasks.workunit.client.1.smithi181.stdout:3/964: dwrite d0/d6c/d52/d44/dd5/d58/d64/dc6/fe6 [0,4194304] 0 2024-04-27T04:10:20.666 INFO:tasks.workunit.client.1.smithi181.stdout:9/872: dwrite d0/d44/d4b/d43/d89/d94/fd6 [0,4194304] 0 2024-04-27T04:10:20.671 INFO:tasks.workunit.client.1.smithi181.stdout:3/965: mkdir d0/d6c/d52/d44/dd5/d58/d137 0 2024-04-27T04:10:20.672 INFO:tasks.workunit.client.1.smithi181.stdout:9/873: write d0/d44/d4b/d43/d88/da1/dfc/dbd/fcc [4292974,68574] 0 2024-04-27T04:10:20.676 INFO:tasks.workunit.client.1.smithi181.stdout:9/874: mknod d0/d44/d4b/d43/d88/da1/dfc/dbd/dd1/c124 0 2024-04-27T04:10:20.681 INFO:tasks.workunit.client.1.smithi181.stdout:3/966: dread d0/d6c/d52/d44/f55 [0,4194304] 0 2024-04-27T04:10:20.681 INFO:tasks.workunit.client.1.smithi181.stdout:3/967: readlink d0/d6c/d52/d44/l54 0 2024-04-27T04:10:20.682 INFO:tasks.workunit.client.1.smithi181.stdout:3/968: stat d0/d6c/d52/dbb 0 2024-04-27T04:10:20.684 INFO:tasks.workunit.client.1.smithi181.stdout:3/969: symlink d0/d6c/d52/dbb/de5/l138 0 2024-04-27T04:10:20.688 INFO:tasks.workunit.client.1.smithi181.stdout:3/970: link d0/d6c/d52/dbb/d98/da7/f12c d0/d6c/d52/d44/d56/d134/f139 0 2024-04-27T04:10:20.690 INFO:tasks.workunit.client.1.smithi181.stdout:3/971: unlink d0/d6c/d52/d44/dd5/d58/d64/l8f 0 2024-04-27T04:10:20.702 INFO:tasks.workunit.client.0.smithi162.stdout:1/733: rename d0/d7/d10/d75/db7/feb to d0/d7/d5e/ff2 0 2024-04-27T04:10:20.704 INFO:tasks.workunit.client.0.smithi162.stdout:6/824: read d6/d1a/d34/d3a/d68/d6c/f4f [628690,101265] 0 2024-04-27T04:10:20.714 INFO:tasks.workunit.client.0.smithi162.stdout:8/854: dwrite d0/de/d20/d6c/fa5 [0,4194304] 0 2024-04-27T04:10:20.718 INFO:tasks.workunit.client.0.smithi162.stdout:2/775: dwrite d0/f1 [4194304,4194304] 0 2024-04-27T04:10:20.721 INFO:tasks.workunit.client.0.smithi162.stdout:4/762: dwrite d2/d25/fb [4194304,4194304] 0 2024-04-27T04:10:20.721 INFO:tasks.workunit.client.0.smithi162.stdout:0/915: dwrite d5/dc/d1e/d46/f55 [0,4194304] 0 2024-04-27T04:10:20.722 INFO:tasks.workunit.client.0.smithi162.stdout:1/734: mknod d0/d11/d3e/d59/cf3 0 2024-04-27T04:10:20.725 INFO:tasks.workunit.client.0.smithi162.stdout:7/832: write d3/d5/d5b/d83/de/d2b/d81/daa/dbd/f84 [1362087,57873] 0 2024-04-27T04:10:20.726 INFO:tasks.workunit.client.0.smithi162.stdout:5/860: dwrite d8/d1a/d29/f95 [0,4194304] 0 2024-04-27T04:10:20.727 INFO:tasks.workunit.client.0.smithi162.stdout:3/789: dwrite d1/d45/fc0 [0,4194304] 0 2024-04-27T04:10:20.730 INFO:tasks.workunit.client.0.smithi162.stdout:4/763: chown d2/d25/d1c/d24/db5/cc9 1 1 2024-04-27T04:10:20.730 INFO:tasks.workunit.client.1.smithi181.stdout:1/920: sync 2024-04-27T04:10:20.731 INFO:tasks.workunit.client.0.smithi162.stdout:1/735: symlink d0/d11/d50/d81/lf4 0 2024-04-27T04:10:20.731 INFO:tasks.workunit.client.1.smithi181.stdout:3/972: rename d0/f3c to d0/d6c/d52/d44/dd5/d133/f13a 0 2024-04-27T04:10:20.733 INFO:tasks.workunit.client.0.smithi162.stdout:6/825: getdents d6/d1a/d34/d7f/dd8 0 2024-04-27T04:10:20.736 INFO:tasks.workunit.client.0.smithi162.stdout:2/776: mknod d0/d4/d2a/d3b/d49/d5a/d63/cf5 0 2024-04-27T04:10:20.738 INFO:tasks.workunit.client.0.smithi162.stdout:1/736: dread d0/d7/d3c/d77/f28 [0,4194304] 0 2024-04-27T04:10:20.740 INFO:tasks.workunit.client.0.smithi162.stdout:4/764: symlink d2/d25/d1c/d39/d3b/d47/d9c/lf8 0 2024-04-27T04:10:20.741 INFO:tasks.workunit.client.1.smithi181.stdout:1/921: dwrite d1/dad/db5/d6/da3/d1e/d48/f11d [0,4194304] 0 2024-04-27T04:10:20.742 INFO:tasks.workunit.client.0.smithi162.stdout:0/916: write d5/dc/f74 [845691,57047] 0 2024-04-27T04:10:20.744 INFO:tasks.workunit.client.1.smithi181.stdout:3/973: rmdir d0/d6c/d52/d44/dd5/d58/d64/dc6/ddd/d11d 39 2024-04-27T04:10:20.750 INFO:tasks.workunit.client.1.smithi181.stdout:3/974: readlink d0/d28/l131 0 2024-04-27T04:10:20.751 INFO:tasks.workunit.client.1.smithi181.stdout:3/975: fdatasync d0/d6c/d52/d44/dd5/d58/d64/dc6/f105 0 2024-04-27T04:10:20.751 INFO:tasks.workunit.client.0.smithi162.stdout:1/737: mknod d0/d7/d3c/d77/dd1/cf5 0 2024-04-27T04:10:20.752 INFO:tasks.workunit.client.0.smithi162.stdout:1/738: chown d0/d7/d3c/d77/dd1 2078 1 2024-04-27T04:10:20.757 INFO:tasks.workunit.client.1.smithi181.stdout:3/976: creat d0/d6c/d52/d44/dd5/d58/de4/dc5/f13b x:0 0 0 2024-04-27T04:10:20.758 INFO:tasks.workunit.client.0.smithi162.stdout:1/739: dwrite d0/d7/d10/fa7 [0,4194304] 0 2024-04-27T04:10:20.759 INFO:tasks.workunit.client.0.smithi162.stdout:8/855: rename d0/d10/d25/d65/d96/dcd/c104 to d0/d10/d25/d65/c112 0 2024-04-27T04:10:20.761 INFO:tasks.workunit.client.0.smithi162.stdout:0/917: chown d5/dc/d1e/f3e 22 1 2024-04-27T04:10:20.764 INFO:tasks.workunit.client.0.smithi162.stdout:1/740: dwrite d0/d7/d10/f3d [8388608,4194304] 0 2024-04-27T04:10:20.764 INFO:tasks.workunit.client.0.smithi162.stdout:2/777: creat d0/d20/d51/ff6 x:0 0 0 2024-04-27T04:10:20.765 INFO:tasks.workunit.client.1.smithi181.stdout:3/977: dread d0/d6c/d52/d44/d74/fc1 [0,4194304] 0 2024-04-27T04:10:20.767 INFO:tasks.workunit.client.1.smithi181.stdout:1/922: dwrite d1/dad/db5/d6/da3/d1e/f49 [0,4194304] 0 2024-04-27T04:10:20.770 INFO:tasks.workunit.client.0.smithi162.stdout:4/765: creat d2/d25/d1c/d24/db5/dc0/ff9 x:0 0 0 2024-04-27T04:10:20.771 INFO:tasks.workunit.client.0.smithi162.stdout:8/856: mkdir d0/de/d6e/d71/d86/d113 0 2024-04-27T04:10:20.772 INFO:tasks.workunit.client.0.smithi162.stdout:4/766: write d2/d25/fd7 [708790,15422] 0 2024-04-27T04:10:20.772 INFO:tasks.workunit.client.1.smithi181.stdout:3/978: dread - d0/d6c/d52/d44/dd5/d58/de4/feb zero size 2024-04-27T04:10:20.775 INFO:tasks.workunit.client.0.smithi162.stdout:0/918: chown d5/d26/l76 1221 1 2024-04-27T04:10:20.775 INFO:tasks.workunit.client.0.smithi162.stdout:0/919: readlink d5/d26/d69/l111 0 2024-04-27T04:10:20.780 INFO:tasks.workunit.client.1.smithi181.stdout:3/979: dwrite d0/de/d7d/d116/f125 [0,4194304] 0 2024-04-27T04:10:20.783 INFO:tasks.workunit.client.0.smithi162.stdout:8/857: creat d0/d10/d25/d65/d96/f114 x:0 0 0 2024-04-27T04:10:20.784 INFO:tasks.workunit.client.0.smithi162.stdout:7/833: link d3/d5/d5b/d83/de/d2b/d81/daa/dbd/dbe/dff/d57/f108 d3/d5/d5b/d83/de/d2b/f114 0 2024-04-27T04:10:20.785 INFO:tasks.workunit.client.0.smithi162.stdout:7/834: chown d3/d4f 3581 1 2024-04-27T04:10:20.787 INFO:tasks.workunit.client.0.smithi162.stdout:7/835: dread - d3/d5/d5b/d83/de/d2b/d81/d96/f99 zero size 2024-04-27T04:10:20.788 INFO:tasks.workunit.client.1.smithi181.stdout:3/980: dwrite d0/d6c/d52/d44/dd5/d58/fea [0,4194304] 0 2024-04-27T04:10:20.789 INFO:tasks.workunit.client.0.smithi162.stdout:2/778: getdents d0/d20/d51 0 2024-04-27T04:10:20.790 INFO:tasks.workunit.client.1.smithi181.stdout:3/981: chown d0/d6c/d66/d88/cab 16072113 1 2024-04-27T04:10:20.792 INFO:tasks.workunit.client.0.smithi162.stdout:8/858: link d0/d10/d44/d70/fee d0/d10/d2b/f115 0 2024-04-27T04:10:20.793 INFO:tasks.workunit.client.0.smithi162.stdout:2/779: chown d0/d4/d2a/da1/dde 501 1 2024-04-27T04:10:20.794 INFO:tasks.workunit.client.1.smithi181.stdout:3/982: mknod d0/d6c/d52/d44/dd5/d58/de4/dc5/dcf/df2/c13c 0 2024-04-27T04:10:20.797 INFO:tasks.workunit.client.0.smithi162.stdout:7/836: dwrite d3/d5/d5b/d83/de/dfc/f102 [0,4194304] 0 2024-04-27T04:10:20.797 INFO:tasks.workunit.client.0.smithi162.stdout:2/780: chown d0/d20/d51/fb6 16342 1 2024-04-27T04:10:20.799 INFO:tasks.workunit.client.1.smithi181.stdout:3/983: fsync d0/de/f1f 0 2024-04-27T04:10:20.801 INFO:tasks.workunit.client.1.smithi181.stdout:1/923: write d1/dad/db5/d6/db/fe9 [994386,110912] 0 2024-04-27T04:10:20.803 INFO:tasks.workunit.client.1.smithi181.stdout:3/984: rename d0/d6c/d52/d44/d74/la9 to d0/d6c/d66/dc2/l13d 0 2024-04-27T04:10:20.804 INFO:tasks.workunit.client.0.smithi162.stdout:8/859: dwrite d0/d10/d44/f47 [0,4194304] 0 2024-04-27T04:10:20.806 INFO:tasks.workunit.client.1.smithi181.stdout:3/985: read d0/d6c/d52/d44/d56/d134/f139 [1685009,38728] 0 2024-04-27T04:10:20.806 INFO:tasks.workunit.client.0.smithi162.stdout:2/781: unlink d0/d4/d17/f2f 0 2024-04-27T04:10:20.808 INFO:tasks.workunit.client.0.smithi162.stdout:2/782: chown d0/d4/f3a 99730 1 2024-04-27T04:10:20.813 INFO:tasks.workunit.client.0.smithi162.stdout:6/826: dwrite d6/d1a/d34/d7b/d7e/d97/f9d [0,4194304] 0 2024-04-27T04:10:20.817 INFO:tasks.workunit.client.0.smithi162.stdout:1/741: dwrite d0/d7/d10/f6b [0,4194304] 0 2024-04-27T04:10:20.818 INFO:tasks.workunit.client.0.smithi162.stdout:0/920: dread d5/dc/d1e/d41/f4f [0,4194304] 0 2024-04-27T04:10:20.820 INFO:tasks.workunit.client.0.smithi162.stdout:0/921: fsync d5/dc/d1e/d20/f10d 0 2024-04-27T04:10:20.820 INFO:tasks.workunit.client.0.smithi162.stdout:4/767: dread d2/da/f31 [0,4194304] 0 2024-04-27T04:10:20.823 INFO:tasks.workunit.client.0.smithi162.stdout:0/922: chown d5/d26/d4a/fe6 516 1 2024-04-27T04:10:20.824 INFO:tasks.workunit.client.0.smithi162.stdout:2/783: fdatasync d0/d4/d17/d4f/d7f/fb8 0 2024-04-27T04:10:20.824 INFO:tasks.workunit.client.1.smithi181.stdout:1/924: link d1/dad/db5/d6/da3/d1e/d20/f5c d1/dad/db5/d6/dcb/f129 0 2024-04-27T04:10:20.826 INFO:tasks.workunit.client.0.smithi162.stdout:2/784: read - d0/d20/d51/fc4 zero size 2024-04-27T04:10:20.827 INFO:tasks.workunit.client.0.smithi162.stdout:7/837: link d3/d5/d5b/d83/de/d2b/d81/daa/dbd/dbe/dff/f109 d3/d5/d5b/d83/de/d2b/d81/daa/df1/f115 0 2024-04-27T04:10:20.827 INFO:tasks.workunit.client.0.smithi162.stdout:2/785: chown d0/d4/d17/d2b/d64/d96/fac 11494 1 2024-04-27T04:10:20.828 INFO:tasks.workunit.client.0.smithi162.stdout:6/827: truncate d6/d1a/d34/d3a/d68/d6c/f53 1092413 0 2024-04-27T04:10:20.834 INFO:tasks.workunit.client.1.smithi181.stdout:1/925: mkdir d1/dad/db5/d6/da3/d1e/d12a 0 2024-04-27T04:10:20.834 INFO:tasks.workunit.client.0.smithi162.stdout:0/923: truncate d5/f11 2192925 0 2024-04-27T04:10:20.835 INFO:tasks.workunit.client.0.smithi162.stdout:0/924: chown d5/dc/d1e/d46/d48/d9e/la6 1 1 2024-04-27T04:10:20.835 INFO:tasks.workunit.client.0.smithi162.stdout:4/768: mknod d2/d25/d1c/d39/d70/dcd/df0/cfa 0 2024-04-27T04:10:20.836 INFO:tasks.workunit.client.0.smithi162.stdout:4/769: readlink d2/d25/d1c/d39/ld0 0 2024-04-27T04:10:20.840 INFO:tasks.workunit.client.0.smithi162.stdout:2/786: creat d0/d4/d2a/d3b/d49/d5a/d63/d84/ff7 x:0 0 0 2024-04-27T04:10:20.841 INFO:tasks.workunit.client.0.smithi162.stdout:2/787: readlink d0/d4/d2a/d3b/d49/d5a/d63/d80/l92 0 2024-04-27T04:10:20.841 INFO:tasks.workunit.client.1.smithi181.stdout:1/926: getdents d1/dad/db5/d6/da3/d1e/d20/d53/d56/de2/d115 0 2024-04-27T04:10:20.842 INFO:tasks.workunit.client.0.smithi162.stdout:0/925: symlink d5/d26/d4a/def/l131 0 2024-04-27T04:10:20.843 INFO:tasks.workunit.client.1.smithi181.stdout:3/986: dread d0/d28/f42 [0,4194304] 0 2024-04-27T04:10:20.843 INFO:tasks.workunit.client.0.smithi162.stdout:4/770: mkdir d2/d25/d1c/d39/d3b/d47/dfb 0 2024-04-27T04:10:20.844 INFO:tasks.workunit.client.0.smithi162.stdout:4/771: write d2/d25/d1c/d39/d90/fbb [1287289,31514] 0 2024-04-27T04:10:20.845 INFO:tasks.workunit.client.0.smithi162.stdout:6/828: unlink d6/d1a/d34/f101 0 2024-04-27T04:10:20.846 INFO:tasks.workunit.client.0.smithi162.stdout:1/742: link d0/d7/d3c/fc1 d0/d11/d50/d53/d93/d96/ff6 0 2024-04-27T04:10:20.846 INFO:tasks.workunit.client.0.smithi162.stdout:4/772: write d2/d25/d1c/d39/d3b/d47/d9c/fb4 [4973666,11602] 0 2024-04-27T04:10:20.847 INFO:tasks.workunit.client.0.smithi162.stdout:2/788: truncate d0/d20/d51/fb6 18652 0 2024-04-27T04:10:20.848 INFO:tasks.workunit.client.0.smithi162.stdout:2/789: write d0/d24/f38 [4797118,88237] 0 2024-04-27T04:10:20.851 INFO:tasks.workunit.client.0.smithi162.stdout:6/829: unlink d6/d1a/d34/d3a/d68/d6c/f57 0 2024-04-27T04:10:20.851 INFO:tasks.workunit.client.0.smithi162.stdout:2/790: write d0/d4/d17/db4/fc7 [1046429,109054] 0 2024-04-27T04:10:20.851 INFO:tasks.workunit.client.0.smithi162.stdout:1/743: symlink d0/d11/d3e/d59/d7f/lf7 0 2024-04-27T04:10:20.853 INFO:tasks.workunit.client.0.smithi162.stdout:1/744: dread - d0/d7/d3c/d6e/fea zero size 2024-04-27T04:10:20.854 INFO:tasks.workunit.client.0.smithi162.stdout:2/791: chown d0/d4/d2a/d3b/d49/d5a/d63/d80/l92 385554 1 2024-04-27T04:10:20.855 INFO:tasks.workunit.client.0.smithi162.stdout:6/830: write d6/d1a/d34/d7f/dd9/de4/ffb [920030,48100] 0 2024-04-27T04:10:20.856 INFO:tasks.workunit.client.1.smithi181.stdout:3/987: getdents d0/d6c/d52/d44/dd5/d58/d64/dc6 0 2024-04-27T04:10:20.857 INFO:tasks.workunit.client.1.smithi181.stdout:1/927: dread d1/dad/db5/fa1 [0,4194304] 0 2024-04-27T04:10:20.857 INFO:tasks.workunit.client.0.smithi162.stdout:4/773: dwrite d2/d25/d1c/d39/d3b/fc4 [0,4194304] 0 2024-04-27T04:10:20.859 INFO:tasks.workunit.client.1.smithi181.stdout:1/928: dread d1/dad/db5/d6/db/d2d/f55 [0,4194304] 0 2024-04-27T04:10:20.860 INFO:tasks.workunit.client.0.smithi162.stdout:7/838: dread d3/d4f/f9d [0,4194304] 0 2024-04-27T04:10:20.861 INFO:tasks.workunit.client.1.smithi181.stdout:1/929: truncate d1/dad/db5/d6/da3/f10a 384249 0 2024-04-27T04:10:20.861 INFO:tasks.workunit.client.0.smithi162.stdout:6/831: rename d6/cd to d6/d1a/d34/d43/c113 0 2024-04-27T04:10:20.862 INFO:tasks.workunit.client.1.smithi181.stdout:1/930: readlink d1/dad/db5/d6/da3/d52/l69 0 2024-04-27T04:10:20.865 INFO:tasks.workunit.client.1.smithi181.stdout:3/988: unlink d0/de/d7d/dbc/cdc 0 2024-04-27T04:10:20.866 INFO:tasks.workunit.client.0.smithi162.stdout:1/745: dwrite d0/d7/f88 [4194304,4194304] 0 2024-04-27T04:10:20.870 INFO:tasks.workunit.client.0.smithi162.stdout:6/832: symlink d6/db/l114 0 2024-04-27T04:10:20.871 INFO:tasks.workunit.client.0.smithi162.stdout:6/833: chown d6/d1a/d34/d3a/fef 641064 1 2024-04-27T04:10:20.872 INFO:tasks.workunit.client.0.smithi162.stdout:7/839: fdatasync d3/d5/d5b/d60/fd8 0 2024-04-27T04:10:20.873 INFO:tasks.workunit.client.1.smithi181.stdout:3/989: dwrite d0/d6c/d52/dbb/de5/f75 [0,4194304] 0 2024-04-27T04:10:20.873 INFO:tasks.workunit.client.0.smithi162.stdout:6/834: read - d6/d1a/d34/d3a/dbf/dd7/f10c zero size 2024-04-27T04:10:20.874 INFO:tasks.workunit.client.1.smithi181.stdout:1/931: truncate d1/dad/db5/d6/dcb/f129 64143 0 2024-04-27T04:10:20.875 INFO:tasks.workunit.client.1.smithi181.stdout:3/990: dread d0/d6c/d52/f47 [0,4194304] 0 2024-04-27T04:10:20.876 INFO:tasks.workunit.client.1.smithi181.stdout:1/932: stat d1/dad/db5/d6/db/df2 0 2024-04-27T04:10:20.877 INFO:tasks.workunit.client.0.smithi162.stdout:2/792: mknod d0/d4/d2a/d3b/d49/d5a/d63/d80/de4/cf8 0 2024-04-27T04:10:20.879 INFO:tasks.workunit.client.1.smithi181.stdout:3/991: fdatasync d0/d6c/d66/d99/fec 0 2024-04-27T04:10:20.883 INFO:tasks.workunit.client.0.smithi162.stdout:2/793: dwrite d0/d4/d17/d88/dc9/fda [0,4194304] 0 2024-04-27T04:10:20.884 INFO:tasks.workunit.client.1.smithi181.stdout:3/992: creat d0/d6c/d52/d44/dd5/d58/de4/dc5/f13e x:0 0 0 2024-04-27T04:10:20.885 INFO:tasks.workunit.client.0.smithi162.stdout:2/794: readlink d0/d4/d2a/d3b/d49/db7/ld6 0 2024-04-27T04:10:20.888 INFO:tasks.workunit.client.0.smithi162.stdout:7/840: symlink d3/d5/d5b/d83/de/d2b/d81/d96/dc8/l116 0 2024-04-27T04:10:20.889 INFO:tasks.workunit.client.1.smithi181.stdout:3/993: write d0/d6c/d52/d44/dd5/d58/de4/daf/ff6 [3300394,52245] 0 2024-04-27T04:10:20.891 INFO:tasks.workunit.client.1.smithi181.stdout:1/933: creat d1/dad/db5/d6/da3/d1e/d48/f12b x:0 0 0 2024-04-27T04:10:20.891 INFO:tasks.workunit.client.1.smithi181.stdout:1/934: readlink d1/dad/db5/d6/db/ld1 0 2024-04-27T04:10:20.893 INFO:tasks.workunit.client.0.smithi162.stdout:2/795: dwrite d0/d4/d17/d2b/d64/d96/feb [0,4194304] 0 2024-04-27T04:10:20.894 INFO:tasks.workunit.client.1.smithi181.stdout:4/927: sync 2024-04-27T04:10:20.895 INFO:tasks.workunit.client.0.smithi162.stdout:2/796: write d0/d4/d2a/d3b/d49/d5a/fe7 [186942,119833] 0 2024-04-27T04:10:20.897 INFO:tasks.workunit.client.0.smithi162.stdout:2/797: read d0/f44 [4321149,59723] 0 2024-04-27T04:10:20.898 INFO:tasks.workunit.client.1.smithi181.stdout:1/935: fsync d1/fa8 0 2024-04-27T04:10:20.899 INFO:tasks.workunit.client.0.smithi162.stdout:1/746: truncate d0/d11/f2e 1914155 0 2024-04-27T04:10:20.902 INFO:tasks.workunit.client.1.smithi181.stdout:4/928: unlink d0/d15/d25/d2a/d36/db8/ffc 0 2024-04-27T04:10:20.907 INFO:tasks.workunit.client.0.smithi162.stdout:7/841: mkdir d3/d5/d5b/d83/de/d2b/d81/d117 0 2024-04-27T04:10:20.909 INFO:tasks.workunit.client.0.smithi162.stdout:4/774: link d2/d25/d1c/d24/fbc d2/d25/d1c/d39/d3b/d47/ffc 0 2024-04-27T04:10:20.916 INFO:tasks.workunit.client.0.smithi162.stdout:2/798: unlink d0/d4/d17/d2b/d64/d96/daa/ld0 0 2024-04-27T04:10:20.920 INFO:tasks.workunit.client.0.smithi162.stdout:1/747: symlink d0/d11/d50/d53/d93/d96/daa/lf8 0 2024-04-27T04:10:20.921 INFO:tasks.workunit.client.0.smithi162.stdout:3/790: sync 2024-04-27T04:10:20.921 INFO:tasks.workunit.client.0.smithi162.stdout:5/861: sync 2024-04-27T04:10:20.921 INFO:tasks.workunit.client.0.smithi162.stdout:9/769: sync 2024-04-27T04:10:20.921 INFO:tasks.workunit.client.1.smithi181.stdout:4/929: truncate d0/d15/d25/d2a/da0/dc7/d12a/f4f 2444392 0 2024-04-27T04:10:20.923 INFO:tasks.workunit.client.1.smithi181.stdout:3/994: dread d0/d6c/d52/d44/dd5/d58/de4/daf/f81 [0,4194304] 0 2024-04-27T04:10:20.924 INFO:tasks.workunit.client.1.smithi181.stdout:1/936: dread d1/dad/db5/d6/da3/d52/f76 [0,4194304] 0 2024-04-27T04:10:20.925 INFO:tasks.workunit.client.1.smithi181.stdout:1/937: chown d1/dad/db5/d6/da3/d1e/dbc 749000018 1 2024-04-27T04:10:20.926 INFO:tasks.workunit.client.0.smithi162.stdout:4/775: rename d2/d54/d7a/d95 to d2/d25/d1c/d39/d90/dad/dfd 0 2024-04-27T04:10:20.927 INFO:tasks.workunit.client.1.smithi181.stdout:4/930: mkdir d0/d15/de/d19/d101/d135 0 2024-04-27T04:10:20.927 INFO:tasks.workunit.client.0.smithi162.stdout:6/835: dwrite d6/db/dc0/d74/fa1 [0,4194304] 0 2024-04-27T04:10:20.928 INFO:tasks.workunit.client.1.smithi181.stdout:4/931: read - d0/d15/de/d19/d8a/fbe zero size 2024-04-27T04:10:20.928 INFO:tasks.workunit.client.0.smithi162.stdout:6/836: chown d6/f2d 2 1 2024-04-27T04:10:20.929 INFO:tasks.workunit.client.1.smithi181.stdout:3/995: creat d0/d6c/d52/d44/dd5/d58/de4/dc5/dcf/f13f x:0 0 0 2024-04-27T04:10:20.931 INFO:tasks.workunit.client.1.smithi181.stdout:3/996: readlink d0/d6c/d52/d44/dd5/d58/d64/dc6/ddd/l101 0 2024-04-27T04:10:20.931 INFO:tasks.workunit.client.0.smithi162.stdout:6/837: write d6/d1a/d34/d7f/f10d [359582,12304] 0 2024-04-27T04:10:20.932 INFO:tasks.workunit.client.0.smithi162.stdout:6/838: write d6/d1a/d34/d3a/dbf/dd7/f10c [301851,87427] 0 2024-04-27T04:10:20.935 INFO:tasks.workunit.client.0.smithi162.stdout:2/799: dread d0/d4/d2a/d41/d77/f7c [0,4194304] 0 2024-04-27T04:10:20.937 INFO:tasks.workunit.client.0.smithi162.stdout:3/791: creat d1/d18/d23/d34/daa/f119 x:0 0 0 2024-04-27T04:10:20.940 INFO:tasks.workunit.client.0.smithi162.stdout:5/862: truncate d8/d1a/d7d/fc5 1570811 0 2024-04-27T04:10:20.940 INFO:tasks.workunit.client.1.smithi181.stdout:4/932: dread d0/d15/d25/d2b/d6d/d82/f8b [0,4194304] 0 2024-04-27T04:10:20.943 INFO:tasks.workunit.client.0.smithi162.stdout:7/842: write d3/d5/d5b/d83/de/f98 [169243,85489] 0 2024-04-27T04:10:20.943 INFO:tasks.workunit.client.0.smithi162.stdout:1/748: creat d0/d7/d3c/d54/db0/ff9 x:0 0 0 2024-04-27T04:10:20.943 INFO:tasks.workunit.client.0.smithi162.stdout:4/776: truncate d2/d25/d1c/d39/d70/dcd/df0/de7/f29 4638112 0 2024-04-27T04:10:20.945 INFO:tasks.workunit.client.1.smithi181.stdout:4/933: creat d0/d15/de/d19/d77/dc5/d10c/f136 x:0 0 0 2024-04-27T04:10:20.945 INFO:tasks.workunit.client.0.smithi162.stdout:6/839: readlink d6/d1a/d34/d7b/l102 0 2024-04-27T04:10:20.946 INFO:tasks.workunit.client.0.smithi162.stdout:9/770: dwrite d3/d13/d2d/d30/d7f/fcf [0,4194304] 0 2024-04-27T04:10:20.949 INFO:tasks.workunit.client.0.smithi162.stdout:6/840: write d6/f25 [4784858,79546] 0 2024-04-27T04:10:20.949 INFO:tasks.workunit.client.0.smithi162.stdout:2/800: mkdir d0/d4/d17/d2b/d64/d96/daa/df9 0 2024-04-27T04:10:20.949 INFO:tasks.workunit.client.0.smithi162.stdout:3/792: dwrite d1/d18/d5d/f62 [4194304,4194304] 0 2024-04-27T04:10:20.950 INFO:tasks.workunit.client.0.smithi162.stdout:4/777: write d2/d25/d1c/d39/d3b/d47/f6c [2979684,54780] 0 2024-04-27T04:10:20.953 INFO:tasks.workunit.client.1.smithi181.stdout:4/934: mkdir d0/d15/d25/d2b/d137 0 2024-04-27T04:10:20.954 INFO:tasks.workunit.client.0.smithi162.stdout:4/778: truncate d2/da/dd/f93 1967652 0 2024-04-27T04:10:20.957 INFO:tasks.workunit.client.1.smithi181.stdout:1/938: dwrite d1/dad/db5/d6/da3/fff [0,4194304] 0 2024-04-27T04:10:20.958 INFO:tasks.workunit.client.1.smithi181.stdout:3/997: dread d0/d6c/d52/d44/d56/f76 [0,4194304] 0 2024-04-27T04:10:20.958 INFO:tasks.workunit.client.1.smithi181.stdout:1/939: chown d1/dad/db5/fa1 55830 1 2024-04-27T04:10:20.963 INFO:tasks.workunit.client.1.smithi181.stdout:4/935: mkdir d0/d15/de/d19/d101/d135/d138 0 2024-04-27T04:10:20.964 INFO:tasks.workunit.client.1.smithi181.stdout:4/936: chown d0/d15/de/la7 705 1 2024-04-27T04:10:20.965 INFO:tasks.workunit.client.1.smithi181.stdout:3/998: rename d0/d6c/d66/l80 to d0/d6c/d52/dbb/de5/l140 0 2024-04-27T04:10:20.967 INFO:tasks.workunit.client.0.smithi162.stdout:6/841: symlink d6/d1a/d34/d7f/dd9/de4/df8/l115 0 2024-04-27T04:10:20.967 INFO:tasks.workunit.client.0.smithi162.stdout:8/860: sync 2024-04-27T04:10:20.968 INFO:tasks.workunit.client.1.smithi181.stdout:1/940: rename d1/d2/d64/ddc/c125 to d1/dad/db5/d6/da3/c12c 0 2024-04-27T04:10:20.970 INFO:tasks.workunit.client.1.smithi181.stdout:3/999: mknod d0/d6c/d52/d44/dd5/d58/de4/dc5/c141 0 2024-04-27T04:10:20.972 INFO:tasks.workunit.client.1.smithi181.stdout:4/937: rename d0/d15/d5c/d69/l29 to d0/d15/de/d19/d101/d135/l139 0 2024-04-27T04:10:20.973 INFO:tasks.workunit.client.0.smithi162.stdout:4/779: creat d2/d25/d1c/d39/d90/dad/ffe x:0 0 0 2024-04-27T04:10:20.975 INFO:tasks.workunit.client.0.smithi162.stdout:6/842: dwrite d6/d1a/d34/d50/dad/fae [0,4194304] 0 2024-04-27T04:10:20.977 INFO:tasks.workunit.client.0.smithi162.stdout:7/843: mkdir d3/d5/d5b/d118 0 2024-04-27T04:10:20.980 INFO:tasks.workunit.client.1.smithi181.stdout:4/938: truncate d0/d15/de/d19/d77/d57/f84 1155651 0 2024-04-27T04:10:20.980 INFO:tasks.workunit.client.0.smithi162.stdout:7/844: chown d3/d5/d5b/d83/de/d2b/d81/d96/dc8/l116 2 1 2024-04-27T04:10:20.980 INFO:tasks.workunit.client.0.smithi162.stdout:1/749: write d0/f2b [4960619,49074] 0 2024-04-27T04:10:20.981 INFO:tasks.workunit.client.1.smithi181.stdout:4/939: chown d0/d15/d25/d2b/f5b 2809407 1 2024-04-27T04:10:20.981 INFO:tasks.workunit.client.0.smithi162.stdout:9/771: symlink d3/d23/led 0 2024-04-27T04:10:20.983 INFO:tasks.workunit.client.0.smithi162.stdout:3/793: dread d1/d70/f92 [0,4194304] 0 2024-04-27T04:10:20.986 INFO:tasks.workunit.client.0.smithi162.stdout:2/801: mkdir d0/d4/d17/d2b/d64/d96/daa/df9/dfa 0 2024-04-27T04:10:20.988 INFO:tasks.workunit.client.0.smithi162.stdout:1/750: dwrite d0/d80/d89/fe2 [0,4194304] 0 2024-04-27T04:10:20.991 INFO:tasks.workunit.client.0.smithi162.stdout:1/751: write d0/d7/d2a/f47 [1860097,28997] 0 2024-04-27T04:10:20.991 INFO:tasks.workunit.client.0.smithi162.stdout:6/843: rename d6/d1a/d34/caa to d6/d1a/d34/d3a/d68/d6c/de3/c116 0 2024-04-27T04:10:20.992 INFO:tasks.workunit.client.1.smithi181.stdout:4/940: link d0/d126/le3 d0/d15/de/d19/d8a/l13a 0 2024-04-27T04:10:20.993 INFO:tasks.workunit.client.1.smithi181.stdout:4/941: chown d0/d15/de/d19/d8a 7471203 1 2024-04-27T04:10:20.993 INFO:tasks.workunit.client.0.smithi162.stdout:5/863: link d8/d9/db3/ccf d8/d1a/d23/d40/de4/c10f 0 2024-04-27T04:10:20.996 INFO:tasks.workunit.client.1.smithi181.stdout:8/933: sync 2024-04-27T04:10:21.001 INFO:tasks.workunit.client.1.smithi181.stdout:1/941: dwrite d1/dad/db5/d6/da3/d52/f76 [0,4194304] 0 2024-04-27T04:10:21.002 INFO:tasks.workunit.client.1.smithi181.stdout:4/942: dwrite d0/d15/d25/d2b/d6d/f71 [0,4194304] 0 2024-04-27T04:10:21.004 INFO:tasks.workunit.client.1.smithi181.stdout:4/943: fsync d0/d15/d25/d2a/d61/d11f/f12f 0 2024-04-27T04:10:21.005 INFO:tasks.workunit.client.1.smithi181.stdout:8/934: mkdir d1/d6/d148 0 2024-04-27T04:10:21.008 INFO:tasks.workunit.client.0.smithi162.stdout:4/780: mknod d2/da/dd4/cff 0 2024-04-27T04:10:21.011 INFO:tasks.workunit.client.1.smithi181.stdout:4/944: creat d0/d15/d25/d2a/d36/db8/dca/f13b x:0 0 0 2024-04-27T04:10:21.014 INFO:tasks.workunit.client.1.smithi181.stdout:1/942: mkdir d1/dad/db5/d12d 0 2024-04-27T04:10:21.015 INFO:tasks.workunit.client.0.smithi162.stdout:6/844: truncate d6/d1a/d34/d7b/f83 883215 0 2024-04-27T04:10:21.016 INFO:tasks.workunit.client.0.smithi162.stdout:7/845: write d3/d5/d9/fe7 [712377,72746] 0 2024-04-27T04:10:21.017 INFO:tasks.workunit.client.1.smithi181.stdout:8/935: rename d1/d6/d6f/dd2/d13f/d25/f52 to d1/d6/d2f/d11e/dd9/f149 0 2024-04-27T04:10:21.018 INFO:tasks.workunit.client.0.smithi162.stdout:7/846: chown d3/d5/d5b/d83/de/d2b/d81/daa/dbd 28607775 1 2024-04-27T04:10:21.018 INFO:tasks.workunit.client.1.smithi181.stdout:4/945: dwrite d0/d15/d25/d2a/d36/db8/dca/f103 [0,4194304] 0 2024-04-27T04:10:21.019 INFO:tasks.workunit.client.0.smithi162.stdout:5/864: mkdir d8/d9/d5b/df3/d110 0 2024-04-27T04:10:21.020 INFO:tasks.workunit.client.0.smithi162.stdout:5/865: chown d8/fad 52307424 1 2024-04-27T04:10:21.023 INFO:tasks.workunit.client.0.smithi162.stdout:9/772: getdents d3/d13/d15/d7e/dd0 0 2024-04-27T04:10:21.024 INFO:tasks.workunit.client.0.smithi162.stdout:8/861: creat d0/d10/d2b/d2d/d2f/f116 x:0 0 0 2024-04-27T04:10:21.024 INFO:tasks.workunit.client.0.smithi162.stdout:3/794: dwrite d1/d18/d23/d34/fd3 [0,4194304] 0 2024-04-27T04:10:21.025 INFO:tasks.workunit.client.0.smithi162.stdout:3/795: fdatasync d1/d18/f6a 0 2024-04-27T04:10:21.026 INFO:tasks.workunit.client.0.smithi162.stdout:8/862: write d0/de/d6e/d71/f9d [303958,95519] 0 2024-04-27T04:10:21.027 INFO:tasks.workunit.client.1.smithi181.stdout:4/946: rmdir d0/d15/de/d19/d77 39 2024-04-27T04:10:21.029 INFO:tasks.workunit.client.0.smithi162.stdout:2/802: symlink d0/d4/d17/d88/lfb 0 2024-04-27T04:10:21.031 INFO:tasks.workunit.client.1.smithi181.stdout:9/875: sync 2024-04-27T04:10:21.031 INFO:tasks.workunit.client.1.smithi181.stdout:7/975: sync 2024-04-27T04:10:21.031 INFO:tasks.workunit.client.1.smithi181.stdout:6/881: sync 2024-04-27T04:10:21.033 INFO:tasks.workunit.client.1.smithi181.stdout:6/882: chown d7/d31/f37 787292 1 2024-04-27T04:10:21.033 INFO:tasks.workunit.client.0.smithi162.stdout:1/752: truncate d0/d11/d3e/f6c 728136 0 2024-04-27T04:10:21.033 INFO:tasks.workunit.client.1.smithi181.stdout:7/976: chown d1/d2/d1c/d2d/d61/d80/dcb 75787 1 2024-04-27T04:10:21.034 INFO:tasks.workunit.client.1.smithi181.stdout:4/947: unlink d0/d15/d5c/c104 0 2024-04-27T04:10:21.035 INFO:tasks.workunit.client.1.smithi181.stdout:6/883: chown d7/dc/d1c/d34/d3f/d7e/d8a/fad 15 1 2024-04-27T04:10:21.036 INFO:tasks.workunit.client.0.smithi162.stdout:6/845: unlink d6/d1a/d34/l8e 0 2024-04-27T04:10:21.036 INFO:tasks.workunit.client.0.smithi162.stdout:2/803: dwrite d0/d4/d17/d2b/d64/f90 [0,4194304] 0 2024-04-27T04:10:21.036 INFO:tasks.workunit.client.0.smithi162.stdout:0/926: sync 2024-04-27T04:10:21.038 INFO:tasks.workunit.client.1.smithi181.stdout:9/876: symlink d0/d44/d4b/d43/d88/l125 0 2024-04-27T04:10:21.039 INFO:tasks.workunit.client.0.smithi162.stdout:0/927: chown d5/dc/d1e/d46/d48/d9e/dce/dd1/d116/f90 1772652 1 2024-04-27T04:10:21.041 INFO:tasks.workunit.client.0.smithi162.stdout:9/773: unlink d3/d23/d1e/d43/f5a 0 2024-04-27T04:10:21.044 INFO:tasks.workunit.client.0.smithi162.stdout:2/804: dwrite d0/d24/f38 [0,4194304] 0 2024-04-27T04:10:21.046 INFO:tasks.workunit.client.0.smithi162.stdout:2/805: readlink d0/d4/d17/d4f/d7f/la6 0 2024-04-27T04:10:21.046 INFO:tasks.workunit.client.1.smithi181.stdout:1/943: getdents d1/dad/db5 0 2024-04-27T04:10:21.048 INFO:tasks.workunit.client.0.smithi162.stdout:3/796: fdatasync d1/d18/d23/f28 0 2024-04-27T04:10:21.053 INFO:tasks.workunit.client.0.smithi162.stdout:4/781: truncate d2/d25/d1c/d39/d3b/f94 3032609 0 2024-04-27T04:10:21.056 INFO:tasks.workunit.client.1.smithi181.stdout:9/877: rename d0/d44/d4b/d43/c76 to d0/d44/d4b/d43/d89/d94/d11c/c126 0 2024-04-27T04:10:21.059 INFO:tasks.workunit.client.1.smithi181.stdout:8/936: dwrite d1/d20/fa2 [0,4194304] 0 2024-04-27T04:10:21.059 INFO:tasks.workunit.client.0.smithi162.stdout:1/753: chown d0/d7/cc2 5376092 1 2024-04-27T04:10:21.062 INFO:tasks.workunit.client.0.smithi162.stdout:7/847: creat d3/d5/d5b/d83/de/d2b/d81/d117/f119 x:0 0 0 2024-04-27T04:10:21.064 INFO:tasks.workunit.client.1.smithi181.stdout:8/937: dread d1/f82 [0,4194304] 0 2024-04-27T04:10:21.065 INFO:tasks.workunit.client.1.smithi181.stdout:8/938: readlink d1/d6/d2f/d30/de4/l109 0 2024-04-27T04:10:21.066 INFO:tasks.workunit.client.1.smithi181.stdout:9/878: symlink d0/d44/d4f/df7/l127 0 2024-04-27T04:10:21.069 INFO:tasks.workunit.client.0.smithi162.stdout:5/866: dread d8/de3/deb/fbd [0,4194304] 0 2024-04-27T04:10:21.073 INFO:tasks.workunit.client.1.smithi181.stdout:6/884: link d7/dc/d1c/d88/ca4 d7/dc/d1c/d34/d3f/c12d 0 2024-04-27T04:10:21.074 INFO:tasks.workunit.client.1.smithi181.stdout:4/948: dwrite d0/d15/d25/d2b/f5f [0,4194304] 0 2024-04-27T04:10:21.074 INFO:tasks.workunit.client.1.smithi181.stdout:1/944: dread d1/dad/db5/d6/da3/d1e/d20/d53/d56/dd9/fdd [0,4194304] 0 2024-04-27T04:10:21.075 INFO:tasks.workunit.client.1.smithi181.stdout:7/977: truncate d1/d2/d1c/d99/fd2 1759085 0 2024-04-27T04:10:21.076 INFO:tasks.workunit.client.0.smithi162.stdout:1/754: dread d0/d7/d3c/fb6 [0,4194304] 0 2024-04-27T04:10:21.077 INFO:tasks.workunit.client.0.smithi162.stdout:1/755: write d0/d7/d3c/d54/db0/ff9 [520620,83147] 0 2024-04-27T04:10:21.079 INFO:tasks.workunit.client.0.smithi162.stdout:2/806: symlink d0/d4/d2a/d3b/d49/d5a/d63/d80/de4/lfc 0 2024-04-27T04:10:21.081 INFO:tasks.workunit.client.0.smithi162.stdout:3/797: creat d1/d18/d5d/d6f/f11a x:0 0 0 2024-04-27T04:10:21.081 INFO:tasks.workunit.client.0.smithi162.stdout:2/807: chown d0/d4/d2a/d3b/d49/d5a/d63/d84/l37 555 1 2024-04-27T04:10:21.082 INFO:tasks.workunit.client.1.smithi181.stdout:8/939: mkdir d1/d6/d2f/d47/d137/d14a 0 2024-04-27T04:10:21.085 INFO:tasks.workunit.client.1.smithi181.stdout:4/949: creat d0/d15/d25/d2a/f13c x:0 0 0 2024-04-27T04:10:21.088 INFO:tasks.workunit.client.0.smithi162.stdout:4/782: dwrite d2/d25/d1c/d39/d5e/fdf [0,4194304] 0 2024-04-27T04:10:21.090 INFO:tasks.workunit.client.0.smithi162.stdout:7/848: mkdir d3/d5/d5b/d83/de/d2b/d81/d96/dc8/d11a 0 2024-04-27T04:10:21.090 INFO:tasks.workunit.client.0.smithi162.stdout:7/849: chown d3/d4f/f41 0 1 2024-04-27T04:10:21.095 INFO:tasks.workunit.client.0.smithi162.stdout:5/867: rmdir d8/d1a/d23/d40 39 2024-04-27T04:10:21.095 INFO:tasks.workunit.client.1.smithi181.stdout:8/940: symlink d1/d20/da4/dc5/l14b 0 2024-04-27T04:10:21.098 INFO:tasks.workunit.client.1.smithi181.stdout:4/950: mkdir d0/d15/d76/db1/d13d 0 2024-04-27T04:10:21.099 INFO:tasks.workunit.client.1.smithi181.stdout:4/951: chown d0/d15/d25/d2b/f5f 1619707 1 2024-04-27T04:10:21.101 INFO:tasks.workunit.client.1.smithi181.stdout:4/952: read d0/d15/de/f43 [29956,33826] 0 2024-04-27T04:10:21.101 INFO:tasks.workunit.client.0.smithi162.stdout:9/774: rename d3/c2e to d3/db9/dcc/dd8/cee 0 2024-04-27T04:10:21.103 INFO:tasks.workunit.client.0.smithi162.stdout:1/756: mkdir d0/d11/d50/d53/d93/d96/daa/dfa 0 2024-04-27T04:10:21.104 INFO:tasks.workunit.client.1.smithi181.stdout:6/885: symlink d7/dc/d1c/d34/d3f/d7e/d122/l12e 0 2024-04-27T04:10:21.106 INFO:tasks.workunit.client.1.smithi181.stdout:8/941: creat d1/d6/d2f/d11e/dec/f14c x:0 0 0 2024-04-27T04:10:21.107 INFO:tasks.workunit.client.1.smithi181.stdout:9/879: dwrite d0/d44/d4f/df7/fe1 [0,4194304] 0 2024-04-27T04:10:21.107 INFO:tasks.workunit.client.0.smithi162.stdout:3/798: symlink d1/d18/d23/d34/dad/l11b 0 2024-04-27T04:10:21.109 INFO:tasks.workunit.client.0.smithi162.stdout:2/808: symlink d0/d20/d51/lfd 0 2024-04-27T04:10:21.110 INFO:tasks.workunit.client.0.smithi162.stdout:7/850: dread d3/d5/d3c/d4a/fa4 [0,4194304] 0 2024-04-27T04:10:21.111 INFO:tasks.workunit.client.1.smithi181.stdout:7/978: write d1/d22/d15/d1f/d34/f5b [371807,3595] 0 2024-04-27T04:10:21.112 INFO:tasks.workunit.client.0.smithi162.stdout:2/809: dread d0/d4/d2a/d3b/d49/d5a/fe7 [0,4194304] 0 2024-04-27T04:10:21.114 INFO:tasks.workunit.client.0.smithi162.stdout:2/810: fdatasync d0/d4/d2a/d41/d77/d73/fd3 0 2024-04-27T04:10:21.120 INFO:tasks.workunit.client.1.smithi181.stdout:8/942: rename d1/d6/d2f/d11e/ca0 to d1/d6/d2f/d30/d9d/dea/db3/ddf/c14d 0 2024-04-27T04:10:21.122 INFO:tasks.workunit.client.0.smithi162.stdout:6/846: creat d6/d1a/d34/d7b/f117 x:0 0 0 2024-04-27T04:10:21.124 INFO:tasks.workunit.client.1.smithi181.stdout:7/979: mkdir d1/df/d1a/d6a/d129 0 2024-04-27T04:10:21.127 INFO:tasks.workunit.client.1.smithi181.stdout:6/886: getdents d7/dc/d1c/d7f/d3e/d118 0 2024-04-27T04:10:21.130 INFO:tasks.workunit.client.1.smithi181.stdout:8/943: symlink d1/d6/d2f/d47/l14e 0 2024-04-27T04:10:21.139 INFO:tasks.workunit.client.1.smithi181.stdout:6/887: symlink d7/dc/d1c/d7f/d3e/l12f 0 2024-04-27T04:10:21.139 INFO:tasks.workunit.client.1.smithi181.stdout:4/953: dwrite d0/d15/d25/d2b/d6d/d82/f8b [0,4194304] 0 2024-04-27T04:10:21.139 INFO:tasks.workunit.client.1.smithi181.stdout:9/880: creat d0/d44/d4b/d43/d88/da1/dfc/dbd/f128 x:0 0 0 2024-04-27T04:10:21.139 INFO:tasks.workunit.client.1.smithi181.stdout:6/888: dread d7/dc/dc3/f70 [0,4194304] 0 2024-04-27T04:10:21.142 INFO:tasks.workunit.client.1.smithi181.stdout:8/944: mknod d1/d6/d2f/d47/dba/c14f 0 2024-04-27T04:10:21.146 INFO:tasks.workunit.client.1.smithi181.stdout:7/980: write d1/d2/d1c/d2d/f41 [1018849,12169] 0 2024-04-27T04:10:21.149 INFO:tasks.workunit.client.1.smithi181.stdout:4/954: chown d0/d15/de/d19/d101/d135/l139 1447 1 2024-04-27T04:10:21.153 INFO:tasks.workunit.client.1.smithi181.stdout:8/945: creat d1/d6/d2f/d30/de4/f150 x:0 0 0 2024-04-27T04:10:21.155 INFO:tasks.workunit.client.1.smithi181.stdout:4/955: creat d0/d15/d25/d2a/da0/dc7/d109/f13e x:0 0 0 2024-04-27T04:10:21.156 INFO:tasks.workunit.client.1.smithi181.stdout:4/956: write d0/d15/de/f34 [4495957,19430] 0 2024-04-27T04:10:21.156 INFO:tasks.workunit.client.1.smithi181.stdout:9/881: symlink d0/d44/d4b/d43/d89/d11d/l129 0 2024-04-27T04:10:21.158 INFO:tasks.workunit.client.1.smithi181.stdout:6/889: rename d7/dc/dc3/d72/ce2 to d7/dc/d1c/d34/d3f/d7e/d8a/c130 0 2024-04-27T04:10:21.158 INFO:tasks.workunit.client.1.smithi181.stdout:8/946: fsync d1/d6/d2f/d11e/dd9/f149 0 2024-04-27T04:10:21.161 INFO:tasks.workunit.client.1.smithi181.stdout:9/882: readlink d0/d44/d4b/d43/d88/da1/dfc/l117 0 2024-04-27T04:10:21.163 INFO:tasks.workunit.client.1.smithi181.stdout:7/981: rename d1/d2/d35/d67 to d1/df/d62/d124/d12a 0 2024-04-27T04:10:21.164 INFO:tasks.workunit.client.1.smithi181.stdout:6/890: mknod d7/dc/dc3/dd1/c131 0 2024-04-27T04:10:21.167 INFO:tasks.workunit.client.1.smithi181.stdout:9/883: readlink d0/l87 0 2024-04-27T04:10:21.168 INFO:tasks.workunit.client.0.smithi162.stdout:3/799: unlink d1/d21/f60 0 2024-04-27T04:10:21.168 INFO:tasks.workunit.client.0.smithi162.stdout:7/851: mkdir d3/d5/d5b/d83/de/d2b/d81/d96/d11b 0 2024-04-27T04:10:21.168 INFO:tasks.workunit.client.0.smithi162.stdout:2/811: creat d0/d4/d17/d4f/d7f/ffe x:0 0 0 2024-04-27T04:10:21.171 INFO:tasks.workunit.client.0.smithi162.stdout:0/928: getdents d5/dc/d1e/d20/dec/df1/da1 0 2024-04-27T04:10:21.171 INFO:tasks.workunit.client.1.smithi181.stdout:9/884: chown d0/d44/d4b/d43/d89/d94/f10b 36643 1 2024-04-27T04:10:21.171 INFO:tasks.workunit.client.1.smithi181.stdout:7/982: dwrite d1/df/d1a/f121 [0,4194304] 0 2024-04-27T04:10:21.173 INFO:tasks.workunit.client.0.smithi162.stdout:5/868: dwrite d8/d1a/d23/d40/f6e [0,4194304] 0 2024-04-27T04:10:21.176 INFO:tasks.workunit.client.1.smithi181.stdout:9/885: creat d0/d44/d4b/d43/d88/de3/f12a x:0 0 0 2024-04-27T04:10:21.181 INFO:tasks.workunit.client.1.smithi181.stdout:7/983: fsync d1/d2/d1c/d37/f4f 0 2024-04-27T04:10:21.181 INFO:tasks.workunit.client.1.smithi181.stdout:8/947: dread d1/d20/f24 [4194304,4194304] 0 2024-04-27T04:10:21.183 INFO:tasks.workunit.client.1.smithi181.stdout:8/948: read - d1/d6/d2f/fcf zero size 2024-04-27T04:10:21.184 INFO:tasks.workunit.client.1.smithi181.stdout:6/891: rename d7/dc/ff to d7/dc/d1c/d88/f132 0 2024-04-27T04:10:21.185 INFO:tasks.workunit.client.1.smithi181.stdout:4/957: getdents d0/d15/de/d19/d77/dc5/d10c 0 2024-04-27T04:10:21.187 INFO:tasks.workunit.client.1.smithi181.stdout:4/958: write d0/d15/d25/d2b/d6d/d82/f102 [2656029,100384] 0 2024-04-27T04:10:21.187 INFO:tasks.workunit.client.1.smithi181.stdout:9/886: creat d0/d44/d4b/d43/d89/d94/f12b x:0 0 0 2024-04-27T04:10:21.188 INFO:tasks.workunit.client.0.smithi162.stdout:3/800: mkdir d1/d18/d5d/d5f/d66/dd2/da3/d11c 0 2024-04-27T04:10:21.191 INFO:tasks.workunit.client.1.smithi181.stdout:8/949: rmdir d1/d6/d2f/d47 39 2024-04-27T04:10:21.192 INFO:tasks.workunit.client.0.smithi162.stdout:6/847: unlink d6/d1a/c19 0 2024-04-27T04:10:21.194 INFO:tasks.workunit.client.1.smithi181.stdout:6/892: chown d7/d27/fc9 159 1 2024-04-27T04:10:21.194 INFO:tasks.workunit.client.1.smithi181.stdout:7/984: dread d1/d2/f24 [4194304,4194304] 0 2024-04-27T04:10:21.197 INFO:tasks.workunit.client.0.smithi162.stdout:9/775: rmdir d3/d13/d2d/dba 0 2024-04-27T04:10:21.199 INFO:tasks.workunit.client.0.smithi162.stdout:4/783: getdents d2/d25/d1c/d39/d3b 0 2024-04-27T04:10:21.200 INFO:tasks.workunit.client.1.smithi181.stdout:9/887: creat d0/d44/d4b/d43/d88/de3/f12c x:0 0 0 2024-04-27T04:10:21.202 INFO:tasks.workunit.client.1.smithi181.stdout:8/950: fdatasync d1/d6/d6f/dd2/d13f/f53 0 2024-04-27T04:10:21.203 INFO:tasks.workunit.client.0.smithi162.stdout:2/812: mknod d0/d4/d17/d2b/d64/dbd/cff 0 2024-04-27T04:10:21.206 INFO:tasks.workunit.client.0.smithi162.stdout:7/852: write d3/d5/d9/fb8 [309016,72837] 0 2024-04-27T04:10:21.209 INFO:tasks.workunit.client.0.smithi162.stdout:7/853: write d3/d5/d5b/d83/de/d2b/d81/daa/dbd/dbe/dff/fc9 [3181,130155] 0 2024-04-27T04:10:21.210 INFO:tasks.workunit.client.0.smithi162.stdout:5/869: mkdir d8/d9/d5b/d111 0 2024-04-27T04:10:21.212 INFO:tasks.workunit.client.0.smithi162.stdout:3/801: dwrite d1/d21/d47/d57/f9f [0,4194304] 0 2024-04-27T04:10:21.213 INFO:tasks.workunit.client.1.smithi181.stdout:6/893: unlink d7/dc/d1c/d7f/l3d 0 2024-04-27T04:10:21.213 INFO:tasks.workunit.client.1.smithi181.stdout:4/959: dwrite d0/d15/d25/d2a/d61/fbc [0,4194304] 0 2024-04-27T04:10:21.216 INFO:tasks.workunit.client.1.smithi181.stdout:9/888: write d0/f8b [704261,21607] 0 2024-04-27T04:10:21.217 INFO:tasks.workunit.client.1.smithi181.stdout:4/960: dread - d0/d15/d25/d2a/d61/d11f/d80/fe5 zero size 2024-04-27T04:10:21.218 INFO:tasks.workunit.client.0.smithi162.stdout:8/863: sync 2024-04-27T04:10:21.219 INFO:tasks.workunit.client.0.smithi162.stdout:7/854: stat d3/d5/d5b/d60/d65/c85 0 2024-04-27T04:10:21.221 INFO:tasks.workunit.client.1.smithi181.stdout:8/951: creat d1/d6/d148/f151 x:0 0 0 2024-04-27T04:10:21.221 INFO:tasks.workunit.client.1.smithi181.stdout:4/961: rename d0 to d0/d15/de/d19/d8a/d13f 22 2024-04-27T04:10:21.222 INFO:tasks.workunit.client.0.smithi162.stdout:8/864: chown d0/d10/d25/d65/d96/f114 160421 1 2024-04-27T04:10:21.224 INFO:tasks.workunit.client.1.smithi181.stdout:6/894: dwrite d7/dc/dc3/d72/f116 [0,4194304] 0 2024-04-27T04:10:21.224 INFO:tasks.workunit.client.1.smithi181.stdout:4/962: chown d0/d15/d25/d2a/d36/db8/fec 8 1 2024-04-27T04:10:21.224 INFO:tasks.workunit.client.1.smithi181.stdout:7/985: creat d1/d22/d15/d29/dbe/d1b/f12b x:0 0 0 2024-04-27T04:10:21.226 INFO:tasks.workunit.client.1.smithi181.stdout:6/895: chown d7/dc/dc3/f70 6808 1 2024-04-27T04:10:21.226 INFO:tasks.workunit.client.1.smithi181.stdout:7/986: stat d1/d22/d15/d1f/d34 0 2024-04-27T04:10:21.227 INFO:tasks.workunit.client.0.smithi162.stdout:2/813: dwrite d0/d4/d17/d2b/d64/d89/fb9 [0,4194304] 0 2024-04-27T04:10:21.227 INFO:tasks.workunit.client.0.smithi162.stdout:3/802: creat d1/d21/d47/d57/d6e/f11d x:0 0 0 2024-04-27T04:10:21.230 INFO:tasks.workunit.client.0.smithi162.stdout:0/929: rename d5/dc/cfe to d5/dc/d1e/d46/c132 0 2024-04-27T04:10:21.231 INFO:tasks.workunit.client.1.smithi181.stdout:8/952: chown d1/d6/d2f/d47 0 1 2024-04-27T04:10:21.233 INFO:tasks.workunit.client.1.smithi181.stdout:8/953: chown d1/d6/d2f/d40/lb7 23799 1 2024-04-27T04:10:21.233 INFO:tasks.workunit.client.1.smithi181.stdout:7/987: dwrite d1/df/d62/d124/d12a/f51 [0,4194304] 0 2024-04-27T04:10:21.234 INFO:tasks.workunit.client.1.smithi181.stdout:4/963: unlink d0/d15/d25/d2a/da0/fd1 0 2024-04-27T04:10:21.238 INFO:tasks.workunit.client.1.smithi181.stdout:7/988: dread d1/d2/d1c/d2d/d61/fee [0,4194304] 0 2024-04-27T04:10:21.239 INFO:tasks.workunit.client.0.smithi162.stdout:0/930: dread d5/dc/d1e/d41/f71 [0,4194304] 0 2024-04-27T04:10:21.240 INFO:tasks.workunit.client.0.smithi162.stdout:0/931: truncate d5/dc/d1e/d20/dba/fe8 4503480 0 2024-04-27T04:10:21.241 INFO:tasks.workunit.client.0.smithi162.stdout:5/870: mkdir d8/d9/d5b/d111/d112 0 2024-04-27T04:10:21.242 INFO:tasks.workunit.client.1.smithi181.stdout:6/896: mknod d7/dc/d1c/c133 0 2024-04-27T04:10:21.245 INFO:tasks.workunit.client.0.smithi162.stdout:3/803: read - d1/d45/f9c zero size 2024-04-27T04:10:21.245 INFO:tasks.workunit.client.0.smithi162.stdout:2/814: fsync d0/d4/d2a/d3b/d49/d5a/fe7 0 2024-04-27T04:10:21.245 INFO:tasks.workunit.client.1.smithi181.stdout:6/897: readlink d7/dc/d1c/d34/d3f/d7e/d8a/l10d 0 2024-04-27T04:10:21.245 INFO:tasks.workunit.client.1.smithi181.stdout:4/964: creat d0/d15/d25/d2a/d61/d11f/f140 x:0 0 0 2024-04-27T04:10:21.247 INFO:tasks.workunit.client.0.smithi162.stdout:2/815: read d0/d20/fc6 [506406,123800] 0 2024-04-27T04:10:21.247 INFO:tasks.workunit.client.0.smithi162.stdout:9/776: rename d3/d13/d2d/d3b/cae to d3/d23/d1e/d43/cef 0 2024-04-27T04:10:21.248 INFO:tasks.workunit.client.1.smithi181.stdout:7/989: dwrite d1/f45 [0,4194304] 0 2024-04-27T04:10:21.249 INFO:tasks.workunit.client.0.smithi162.stdout:6/848: getdents d6/d1a/d34/d3a/d68 0 2024-04-27T04:10:21.251 INFO:tasks.workunit.client.1.smithi181.stdout:4/965: symlink d0/d126/l141 0 2024-04-27T04:10:21.251 INFO:tasks.workunit.client.0.smithi162.stdout:4/784: getdents d2/d25/d1c/d39/d70/dcd/df0/de7/de9 0 2024-04-27T04:10:21.252 INFO:tasks.workunit.client.1.smithi181.stdout:6/898: symlink d7/dc/dc3/dd1/dd9/l134 0 2024-04-27T04:10:21.255 INFO:tasks.workunit.client.1.smithi181.stdout:7/990: rmdir d1/d2/d1c/d2d/d61/d80 39 2024-04-27T04:10:21.257 INFO:tasks.workunit.client.0.smithi162.stdout:5/871: unlink d8/d1a/d23/dbc/d66/c76 0 2024-04-27T04:10:21.261 INFO:tasks.workunit.client.0.smithi162.stdout:1/757: sync 2024-04-27T04:10:21.262 INFO:tasks.workunit.client.1.smithi181.stdout:4/966: fdatasync d0/d15/d76/f95 0 2024-04-27T04:10:21.262 INFO:tasks.workunit.client.1.smithi181.stdout:8/954: getdents d1/d6/d2f/d30/d9d/dea/dff 0 2024-04-27T04:10:21.262 INFO:tasks.workunit.client.0.smithi162.stdout:3/804: creat d1/d18/d23/d58/d63/dd5/f11e x:0 0 0 2024-04-27T04:10:21.263 INFO:tasks.workunit.client.0.smithi162.stdout:1/758: fsync d0/d11/d50/d53/d93/d96/daa/de9/fee 0 2024-04-27T04:10:21.263 INFO:tasks.workunit.client.1.smithi181.stdout:7/991: creat d1/df/d1a/d6a/d10e/f12c x:0 0 0 2024-04-27T04:10:21.265 INFO:tasks.workunit.client.0.smithi162.stdout:8/865: rename d0/d10/d2b/d2e/dca/lba to d0/d10/d2b/d2d/d2f/d6b/dd3/l117 0 2024-04-27T04:10:21.268 INFO:tasks.workunit.client.1.smithi181.stdout:4/967: dwrite d0/d126/fda [0,4194304] 0 2024-04-27T04:10:21.269 INFO:tasks.workunit.client.0.smithi162.stdout:2/816: fsync d0/d4/d2a/d3b/d49/d5a/fe7 0 2024-04-27T04:10:21.270 INFO:tasks.workunit.client.0.smithi162.stdout:3/805: dwrite d1/d18/f1a [0,4194304] 0 2024-04-27T04:10:21.271 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:10:20 smithi162 ceph-mon[26841]: from='client.14656 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-27T04:10:21.271 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:10:20 smithi162 ceph-mon[26841]: pgmap v10: 65 pgs: 65 active+clean; 3.3 GiB data, 11 GiB used, 525 GiB / 536 GiB avail; 74 MiB/s rd, 183 MiB/s wr, 414 op/s 2024-04-27T04:10:21.272 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:10:20 smithi162 ceph-mon[26841]: from='mgr.24367 172.21.15.181:0/3181919980' entity='mgr.smithi181.vtwuhz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-27T04:10:21.274 INFO:tasks.workunit.client.0.smithi162.stdout:4/785: read d2/d25/d1c/d39/d3b/f94 [2282550,30922] 0 2024-04-27T04:10:21.274 INFO:tasks.workunit.client.0.smithi162.stdout:3/806: write d1/d18/f4a [4914314,93258] 0 2024-04-27T04:10:21.275 INFO:tasks.workunit.client.1.smithi181.stdout:9/889: dwrite d0/d44/d4b/d43/d88/da1/dfc/dbd/df3/fa5 [0,4194304] 0 2024-04-27T04:10:21.280 INFO:tasks.workunit.client.0.smithi162.stdout:3/807: stat d1/d21/d47/f93 0 2024-04-27T04:10:21.283 INFO:tasks.workunit.client.0.smithi162.stdout:2/817: dwrite d0/d4/f15 [0,4194304] 0 2024-04-27T04:10:21.286 INFO:tasks.workunit.client.0.smithi162.stdout:7/855: write d3/d5/d5b/d83/de/d2b/d81/f82 [48850,53092] 0 2024-04-27T04:10:21.291 INFO:tasks.workunit.client.1.smithi181.stdout:1/945: sync 2024-04-27T04:10:21.293 INFO:tasks.workunit.client.0.smithi162.stdout:8/866: dwrite d0/de/fe6 [0,4194304] 0 2024-04-27T04:10:21.296 INFO:tasks.workunit.client.0.smithi162.stdout:1/759: symlink d0/d11/d3e/lfb 0 2024-04-27T04:10:21.302 INFO:tasks.workunit.client.0.smithi162.stdout:6/849: truncate d6/d1a/d34/d7b/d7e/d97/faf 770111 0 2024-04-27T04:10:21.304 INFO:tasks.workunit.client.0.smithi162.stdout:3/808: mknod d1/d18/d5d/d6f/c11f 0 2024-04-27T04:10:21.304 INFO:tasks.workunit.client.1.smithi181.stdout:6/899: write d7/dc/d1c/d7f/d3e/fc0 [2894277,39089] 0 2024-04-27T04:10:21.305 INFO:tasks.workunit.client.0.smithi162.stdout:1/760: dwrite d0/d7/f88 [0,4194304] 0 2024-04-27T04:10:21.307 INFO:tasks.workunit.client.0.smithi162.stdout:2/818: symlink d0/d4/d17/d2b/d64/d96/daa/l100 0 2024-04-27T04:10:21.312 INFO:tasks.workunit.client.1.smithi181.stdout:6/900: dwrite d7/dc/d1c/d88/db3/ffa [0,4194304] 0 2024-04-27T04:10:21.317 INFO:tasks.workunit.client.0.smithi162.stdout:5/872: creat d8/d9/f113 x:0 0 0 2024-04-27T04:10:21.318 INFO:tasks.workunit.client.0.smithi162.stdout:8/867: mknod d0/de/d6e/d71/d86/c118 0 2024-04-27T04:10:21.324 INFO:tasks.workunit.client.0.smithi162.stdout:6/850: unlink d6/d1a/d34/d50/fd4 0 2024-04-27T04:10:21.325 INFO:tasks.workunit.client.0.smithi162.stdout:6/851: readlink d6/d1a/d34/d50/dad/lc8 0 2024-04-27T04:10:21.325 INFO:tasks.workunit.client.0.smithi162.stdout:6/852: dread - d6/d1a/d34/d7b/f117 zero size 2024-04-27T04:10:21.326 INFO:tasks.workunit.client.0.smithi162.stdout:3/809: mknod d1/d70/c120 0 2024-04-27T04:10:21.327 INFO:tasks.workunit.client.0.smithi162.stdout:1/761: rmdir d0/d7/d3c/d61/dc4 39 2024-04-27T04:10:21.330 INFO:tasks.workunit.client.0.smithi162.stdout:9/777: getdents d3/d13/d2d/d3b 0 2024-04-27T04:10:21.331 INFO:tasks.workunit.client.0.smithi162.stdout:9/778: chown d3/d13/d2d/d30/cd1 1086266 1 2024-04-27T04:10:21.332 INFO:tasks.workunit.client.0.smithi162.stdout:7/856: mkdir d3/d5/d11c 0 2024-04-27T04:10:21.337 INFO:tasks.workunit.client.0.smithi162.stdout:0/932: rename d5/dc/d1e/d46/d48/c5f to d5/dc/d1e/d46/c133 0 2024-04-27T04:10:21.337 INFO:tasks.workunit.client.0.smithi162.stdout:0/933: stat d5/l47 0 2024-04-27T04:10:21.342 INFO:tasks.workunit.client.0.smithi162.stdout:3/810: creat d1/d18/d23/d58/d63/d88/f121 x:0 0 0 2024-04-27T04:10:21.345 INFO:tasks.workunit.client.0.smithi162.stdout:2/819: mkdir d0/d101 0 2024-04-27T04:10:21.348 INFO:tasks.workunit.client.0.smithi162.stdout:7/857: mknod d3/d5/d5b/d83/de/d2b/d81/d96/c11d 0 2024-04-27T04:10:21.349 INFO:tasks.workunit.client.0.smithi162.stdout:5/873: write d8/d1a/d23/dbc/fa8 [561996,82865] 0 2024-04-27T04:10:21.354 INFO:tasks.workunit.client.0.smithi162.stdout:6/853: write d6/d1a/d34/d50/dad/fc9 [240454,87028] 0 2024-04-27T04:10:21.355 INFO:tasks.workunit.client.0.smithi162.stdout:8/868: dwrite d0/d10/d2b/f115 [0,4194304] 0 2024-04-27T04:10:21.356 INFO:tasks.workunit.client.0.smithi162.stdout:3/811: symlink d1/d18/d5d/d6f/l122 0 2024-04-27T04:10:21.357 INFO:tasks.workunit.client.0.smithi162.stdout:8/869: write d0/d10/d25/d7d/f8b [4813303,122335] 0 2024-04-27T04:10:21.361 INFO:tasks.workunit.client.1.smithi181.stdout:9/890: dwrite d0/f9 [4194304,4194304] 0 2024-04-27T04:10:21.361 INFO:tasks.workunit.client.1.smithi181.stdout:6/901: truncate d7/d31/f75 1120038 0 2024-04-27T04:10:21.362 INFO:tasks.workunit.client.0.smithi162.stdout:9/779: mkdir d3/d13/df0 0 2024-04-27T04:10:21.364 INFO:tasks.workunit.client.0.smithi162.stdout:9/780: chown d3/d11/d45/d4e/fcb 10 1 2024-04-27T04:10:21.364 INFO:tasks.workunit.client.0.smithi162.stdout:8/870: dwrite d0/de/fe6 [4194304,4194304] 0 2024-04-27T04:10:21.365 INFO:tasks.workunit.client.1.smithi181.stdout:9/891: truncate d0/d44/d4b/d43/d88/da1/dfc/dbd/f128 651185 0 2024-04-27T04:10:21.365 INFO:tasks.workunit.client.0.smithi162.stdout:8/871: stat d0/de/d6e/l90 0 2024-04-27T04:10:21.369 INFO:tasks.workunit.client.0.smithi162.stdout:7/858: creat d3/d5/d5b/d83/de/d2b/d81/daa/f11e x:0 0 0 2024-04-27T04:10:21.371 INFO:tasks.workunit.client.0.smithi162.stdout:8/872: dwrite d0/de/d20/d6c/fa5 [0,4194304] 0 2024-04-27T04:10:21.374 INFO:tasks.workunit.client.1.smithi181.stdout:7/992: dwrite d1/df/d1a/fc5 [0,4194304] 0 2024-04-27T04:10:21.378 INFO:tasks.workunit.client.1.smithi181.stdout:6/902: readlink d7/dc/dc3/d72/daa/d110/l120 0 2024-04-27T04:10:21.382 INFO:tasks.workunit.client.1.smithi181.stdout:8/955: link d1/d6/d2f/d30/d66/dc0/dcd/fde d1/d6/d6f/d91/f152 0 2024-04-27T04:10:21.386 INFO:tasks.workunit.client.1.smithi181.stdout:4/968: creat d0/d15/de/d19/d77/f142 x:0 0 0 2024-04-27T04:10:21.387 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:10:20 smithi181 ceph-mon[33774]: from='client.14656 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-27T04:10:21.387 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:10:20 smithi181 ceph-mon[33774]: pgmap v10: 65 pgs: 65 active+clean; 3.3 GiB data, 11 GiB used, 525 GiB / 536 GiB avail; 74 MiB/s rd, 183 MiB/s wr, 414 op/s 2024-04-27T04:10:21.387 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:10:20 smithi181 ceph-mon[33774]: from='mgr.24367 172.21.15.181:0/3181919980' entity='mgr.smithi181.vtwuhz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-27T04:10:21.388 INFO:tasks.workunit.client.0.smithi162.stdout:3/812: creat d1/d18/d23/d34/daa/f123 x:0 0 0 2024-04-27T04:10:21.389 INFO:tasks.workunit.client.0.smithi162.stdout:3/813: dread - d1/d21/d47/d57/ffd zero size 2024-04-27T04:10:21.390 INFO:tasks.workunit.client.0.smithi162.stdout:2/820: mkdir d0/d4/d17/d102 0 2024-04-27T04:10:21.392 INFO:tasks.workunit.client.0.smithi162.stdout:9/781: rmdir d3/d23 39 2024-04-27T04:10:21.393 INFO:tasks.workunit.client.1.smithi181.stdout:1/946: dwrite d1/dad/db5/d6/da3/d1e/d20/f5c [0,4194304] 0 2024-04-27T04:10:21.395 INFO:tasks.workunit.client.0.smithi162.stdout:5/874: mkdir d8/d9/d5b/d114 0 2024-04-27T04:10:21.396 INFO:tasks.workunit.client.0.smithi162.stdout:5/875: write d8/d9/f21 [4014263,104064] 0 2024-04-27T04:10:21.396 INFO:tasks.workunit.client.1.smithi181.stdout:9/892: rename d0/d44/d4b/d43/d10c/c123 to d0/d44/d4b/d43/d88/c12d 0 2024-04-27T04:10:21.397 INFO:tasks.workunit.client.0.smithi162.stdout:5/876: stat d8/d1a/d29/c2a 0 2024-04-27T04:10:21.398 INFO:tasks.workunit.client.0.smithi162.stdout:5/877: fdatasync d8/d9/d5b/fe5 0 2024-04-27T04:10:21.398 INFO:tasks.workunit.client.0.smithi162.stdout:7/859: mknod d3/d5/d5b/d83/de/d2b/d81/daa/dbd/dbe/dff/d57/c11f 0 2024-04-27T04:10:21.400 INFO:tasks.workunit.client.1.smithi181.stdout:4/969: mkdir d0/d15/d25/d2a/da0/dc7/d109/d7c/d143 0 2024-04-27T04:10:21.400 INFO:tasks.workunit.client.1.smithi181.stdout:4/970: readlink d0/dd4/lff 0 2024-04-27T04:10:21.401 INFO:tasks.workunit.client.0.smithi162.stdout:5/878: truncate d8/d1a/d23/dbc/fa8 995351 0 2024-04-27T04:10:21.403 INFO:tasks.workunit.client.0.smithi162.stdout:5/879: chown c6 151237249 1 2024-04-27T04:10:21.405 INFO:tasks.workunit.client.0.smithi162.stdout:4/786: rename d2/d25/d1c/d39/d70/dcd/df0/de7 to d2/d25/d1c/d24/d100 0 2024-04-27T04:10:21.405 INFO:tasks.workunit.client.0.smithi162.stdout:0/934: creat d5/f134 x:0 0 0 2024-04-27T04:10:21.406 INFO:tasks.workunit.client.0.smithi162.stdout:1/762: getdents d0/d7/d3c/d77 0 2024-04-27T04:10:21.406 INFO:tasks.workunit.client.1.smithi181.stdout:1/947: creat d1/dad/db5/d6/da3/d1e/d48/da6/da4/f12e x:0 0 0 2024-04-27T04:10:21.408 INFO:tasks.workunit.client.0.smithi162.stdout:1/763: dread d0/d7/d3c/d54/fdf [0,4194304] 0 2024-04-27T04:10:21.410 INFO:tasks.workunit.client.0.smithi162.stdout:0/935: truncate d5/dc/d1e/d46/d48/d9e/dce/dd1/d116/ddd/f124 694898 0 2024-04-27T04:10:21.410 INFO:tasks.workunit.client.1.smithi181.stdout:8/956: rename d1/d6/f124 to d1/d6/d6f/dd2/d13d/f153 0 2024-04-27T04:10:21.411 INFO:tasks.workunit.client.1.smithi181.stdout:8/957: chown d1/d6/d2f/d40 12 1 2024-04-27T04:10:21.411 INFO:tasks.workunit.client.1.smithi181.stdout:4/971: creat d0/d15/d25/d2a/f144 x:0 0 0 2024-04-27T04:10:21.413 INFO:tasks.workunit.client.0.smithi162.stdout:7/860: mkdir d3/d5/d5b/d83/de/d2b/d81/daa/dbd/dab/d120 0 2024-04-27T04:10:21.414 INFO:tasks.workunit.client.1.smithi181.stdout:6/903: mknod d7/dc/d1c/d7f/da3/d101/c135 0 2024-04-27T04:10:21.416 INFO:tasks.workunit.client.0.smithi162.stdout:0/936: dwrite d5/f16 [4194304,4194304] 0 2024-04-27T04:10:21.416 INFO:tasks.workunit.client.0.smithi162.stdout:8/873: mknod d0/d10/d25/c119 0 2024-04-27T04:10:21.418 INFO:tasks.workunit.client.0.smithi162.stdout:4/787: chown d2/da/l53 584330289 1 2024-04-27T04:10:21.419 INFO:tasks.workunit.client.1.smithi181.stdout:9/893: dread d0/d44/d4f/fae [0,4194304] 0 2024-04-27T04:10:21.421 INFO:tasks.workunit.client.0.smithi162.stdout:6/854: rename d6/d1a/d34/d7f/dd9/cf9 to d6/d1a/d34/d3a/dbf/c118 0 2024-04-27T04:10:21.427 INFO:tasks.workunit.client.0.smithi162.stdout:3/814: mkdir d1/d18/d5d/d5f/d66/d124 0 2024-04-27T04:10:21.429 INFO:tasks.workunit.client.0.smithi162.stdout:1/764: readlink d0/d11/d3e/ld4 0 2024-04-27T04:10:21.431 INFO:tasks.workunit.client.0.smithi162.stdout:0/937: symlink d5/dc/d1e/d41/d79/d109/d119/l135 0 2024-04-27T04:10:21.431 INFO:tasks.workunit.client.0.smithi162.stdout:7/861: creat d3/d5/d5b/d105/f121 x:0 0 0 2024-04-27T04:10:21.433 INFO:tasks.workunit.client.0.smithi162.stdout:8/874: dwrite d0/de/d20/d7e/fff [0,4194304] 0 2024-04-27T04:10:21.434 INFO:tasks.workunit.client.0.smithi162.stdout:2/821: rename d0/f44 to d0/d4/d17/d102/f103 0 2024-04-27T04:10:21.437 INFO:tasks.workunit.client.0.smithi162.stdout:0/938: fsync d5/dc/d1e/f3e 0 2024-04-27T04:10:21.439 INFO:tasks.workunit.client.0.smithi162.stdout:5/880: truncate d8/d9/f21 798413 0 2024-04-27T04:10:21.439 INFO:tasks.workunit.client.0.smithi162.stdout:9/782: dwrite d3/d13/d15/d50/f5b [0,4194304] 0 2024-04-27T04:10:21.441 INFO:tasks.workunit.client.0.smithi162.stdout:5/881: stat d8/d9/d5b/fb5 0 2024-04-27T04:10:21.441 INFO:tasks.workunit.client.0.smithi162.stdout:9/783: dread - d3/d13/d2d/d3b/d7b/fe4 zero size 2024-04-27T04:10:21.443 INFO:tasks.workunit.client.0.smithi162.stdout:9/784: chown d3/d13/d2d/d3b/f47 0 1 2024-04-27T04:10:21.444 INFO:tasks.workunit.client.0.smithi162.stdout:4/788: dread d2/d25/d1c/d39/d90/dad/f65 [0,4194304] 0 2024-04-27T04:10:21.454 INFO:tasks.workunit.client.1.smithi181.stdout:7/993: getdents d1/d22/d15/d82 0 2024-04-27T04:10:21.455 INFO:tasks.workunit.client.1.smithi181.stdout:9/894: chown d0/d44/cd7 19189453 1 2024-04-27T04:10:21.455 INFO:tasks.workunit.client.1.smithi181.stdout:6/904: dwrite d7/f8 [0,4194304] 0 2024-04-27T04:10:21.458 INFO:tasks.workunit.client.1.smithi181.stdout:4/972: symlink d0/d15/d25/d2b/d6d/l145 0 2024-04-27T04:10:21.460 INFO:tasks.workunit.client.1.smithi181.stdout:9/895: symlink d0/d44/d4f/df7/l12e 0 2024-04-27T04:10:21.464 INFO:tasks.workunit.client.1.smithi181.stdout:8/958: dwrite d1/d6/d2f/d11e/dd9/f129 [0,4194304] 0 2024-04-27T04:10:21.466 INFO:tasks.workunit.client.1.smithi181.stdout:4/973: chown d0/d15/d25/d2a/da0/dc7/d109/f4e 170040458 1 2024-04-27T04:10:21.476 INFO:tasks.workunit.client.0.smithi162.stdout:1/765: dread d0/d7/d10/f26 [4194304,4194304] 0 2024-04-27T04:10:21.477 INFO:tasks.workunit.client.1.smithi181.stdout:8/959: creat d1/d6/d2f/d30/d9d/dea/dff/f154 x:0 0 0 2024-04-27T04:10:21.478 INFO:tasks.workunit.client.1.smithi181.stdout:8/960: fsync d1/d6/d2f/d30/d5c/f97 0 2024-04-27T04:10:21.487 INFO:tasks.workunit.client.1.smithi181.stdout:7/994: link d1/d22/d15/d1f/d34/fec d1/df/d62/d124/f12d 0 2024-04-27T04:10:21.488 INFO:tasks.workunit.client.1.smithi181.stdout:7/995: write d1/d22/d15/d1f/f105 [415877,92375] 0 2024-04-27T04:10:21.493 INFO:tasks.workunit.client.1.smithi181.stdout:8/961: link d1/d6/d2f/d30/d9d/lc4 d1/d6/d2f/d30/d9d/dea/dff/l155 0 2024-04-27T04:10:21.494 INFO:tasks.workunit.client.1.smithi181.stdout:7/996: dwrite d1/d22/d15/d29/f114 [0,4194304] 0 2024-04-27T04:10:21.494 INFO:tasks.workunit.client.1.smithi181.stdout:7/997: chown d1/d22/d15/d1f/d34 0 1 2024-04-27T04:10:21.495 INFO:tasks.workunit.client.0.smithi162.stdout:3/815: dread d1/d21/fb8 [0,4194304] 0 2024-04-27T04:10:21.499 INFO:tasks.workunit.client.1.smithi181.stdout:7/998: mkdir d1/df/d1a/d6a/d10e/d12e 0 2024-04-27T04:10:21.502 INFO:tasks.workunit.client.1.smithi181.stdout:8/962: fdatasync d1/d8c/f90 0 2024-04-27T04:10:21.505 INFO:tasks.workunit.client.0.smithi162.stdout:7/862: truncate d3/d5/d5b/d83/f3b 1061703 0 2024-04-27T04:10:21.508 INFO:tasks.workunit.client.0.smithi162.stdout:0/939: creat d5/dc/d1e/d46/d48/d9e/db8/f136 x:0 0 0 2024-04-27T04:10:21.510 INFO:tasks.workunit.client.1.smithi181.stdout:6/905: truncate d7/dc/dc3/dd1/dd9/f9b 4052975 0 2024-04-27T04:10:21.514 INFO:tasks.workunit.client.1.smithi181.stdout:7/999: dwrite d1/d22/d15/d29/dbe/d1b/f40 [0,4194304] 0 2024-04-27T04:10:21.516 INFO:tasks.workunit.client.1.smithi181.stdout:6/906: creat d7/dc/d1c/d88/db3/f136 x:0 0 0 2024-04-27T04:10:21.522 INFO:tasks.workunit.client.0.smithi162.stdout:7/863: mkdir d3/d5/d5b/d83/de/d2b/df9/d122 0 2024-04-27T04:10:21.522 INFO:tasks.workunit.client.1.smithi181.stdout:8/963: dread d1/d6/d2f/f3a [0,4194304] 0 2024-04-27T04:10:21.523 INFO:tasks.workunit.client.1.smithi181.stdout:8/964: fsync d1/d6/d6f/dd2/f147 0 2024-04-27T04:10:21.525 INFO:tasks.workunit.client.1.smithi181.stdout:4/974: write d0/d15/d25/d2a/da0/dc7/d12a/f9b [992995,64035] 0 2024-04-27T04:10:21.529 INFO:tasks.workunit.client.0.smithi162.stdout:5/882: mkdir d8/d9/d5b/d115 0 2024-04-27T04:10:21.530 INFO:tasks.workunit.client.0.smithi162.stdout:5/883: chown d8/d1a/d29/f99 560700 1 2024-04-27T04:10:21.530 INFO:tasks.workunit.client.0.smithi162.stdout:5/884: stat d8/d1a/c105 0 2024-04-27T04:10:21.547 INFO:tasks.workunit.client.0.smithi162.stdout:1/766: mkdir d0/d11/d44/d66/dfc 0 2024-04-27T04:10:21.547 INFO:tasks.workunit.client.1.smithi181.stdout:4/975: creat d0/d15/de1/f146 x:0 0 0 2024-04-27T04:10:21.548 INFO:tasks.workunit.client.1.smithi181.stdout:4/976: write d0/d15/de/d19/d77/dc5/ffb [926144,28270] 0 2024-04-27T04:10:21.548 INFO:tasks.workunit.client.1.smithi181.stdout:8/965: truncate d1/d6/d2f/d72/f94 423390 0 2024-04-27T04:10:21.552 INFO:tasks.workunit.client.1.smithi181.stdout:4/977: truncate d0/d15/f32 211787 0 2024-04-27T04:10:21.553 INFO:tasks.workunit.client.1.smithi181.stdout:8/966: creat d1/d6/d2f/d11e/dec/f156 x:0 0 0 2024-04-27T04:10:21.554 INFO:tasks.workunit.client.0.smithi162.stdout:2/822: write d0/d4/d17/d2b/d64/d89/fcc [826446,66308] 0 2024-04-27T04:10:21.554 INFO:tasks.workunit.client.1.smithi181.stdout:8/967: dread - d1/d6/d2f/d30/d9d/dea/db3/ddf/f12b zero size 2024-04-27T04:10:21.555 INFO:tasks.workunit.client.0.smithi162.stdout:0/940: fdatasync d5/dc/d1e/f30 0 2024-04-27T04:10:21.556 INFO:tasks.workunit.client.1.smithi181.stdout:4/978: mkdir d0/d15/d25/d2a/da0/dc7/d147 0 2024-04-27T04:10:21.559 INFO:tasks.workunit.client.0.smithi162.stdout:9/785: write d3/d13/d15/d50/f8b [507731,33777] 0 2024-04-27T04:10:21.560 INFO:tasks.workunit.client.0.smithi162.stdout:9/786: stat d3/d13/d2d/l64 0 2024-04-27T04:10:21.561 INFO:tasks.workunit.client.0.smithi162.stdout:3/816: link d1/d18/d5d/d5f/d66/lc8 d1/d18/d23/d34/l125 0 2024-04-27T04:10:21.562 INFO:tasks.workunit.client.1.smithi181.stdout:4/979: dwrite d0/d15/de/d19/d77/f106 [0,4194304] 0 2024-04-27T04:10:21.564 INFO:tasks.workunit.client.1.smithi181.stdout:4/980: dread - d0/d15/de1/f146 zero size 2024-04-27T04:10:21.565 INFO:tasks.workunit.client.1.smithi181.stdout:4/981: write d0/d15/d25/d2a/d61/fbc [648787,103396] 0 2024-04-27T04:10:21.569 INFO:tasks.workunit.client.0.smithi162.stdout:0/941: rename d5/d26/d4a/c7d to d5/dc/d1e/d20/dba/df9/d98/c137 0 2024-04-27T04:10:21.569 INFO:tasks.workunit.client.0.smithi162.stdout:6/855: dwrite d6/d1a/d34/d7b/d7e/d97/faf [0,4194304] 0 2024-04-27T04:10:21.570 INFO:tasks.workunit.client.1.smithi181.stdout:6/907: truncate d7/dc/d1c/d88/db3/ff4 3768459 0 2024-04-27T04:10:21.571 INFO:tasks.workunit.client.0.smithi162.stdout:9/787: dwrite d3/d13/d2d/d3b/f84 [0,4194304] 0 2024-04-27T04:10:21.571 INFO:tasks.workunit.client.0.smithi162.stdout:9/788: stat d3/c62 0 2024-04-27T04:10:21.578 INFO:tasks.workunit.client.0.smithi162.stdout:2/823: symlink d0/d4/d2a/l104 0 2024-04-27T04:10:21.579 INFO:tasks.workunit.client.1.smithi181.stdout:6/908: dwrite d7/dc/dc3/d72/daa/f124 [0,4194304] 0 2024-04-27T04:10:21.582 INFO:tasks.workunit.client.1.smithi181.stdout:6/909: dread - d7/dc/d1c/d88/f11f zero size 2024-04-27T04:10:21.586 INFO:tasks.workunit.client.0.smithi162.stdout:6/856: symlink d6/d1a/d34/d3a/d68/d6c/l119 0 2024-04-27T04:10:21.590 INFO:tasks.workunit.client.0.smithi162.stdout:2/824: mknod d0/d4/d17/d2b/d64/dbd/c105 0 2024-04-27T04:10:21.592 INFO:tasks.workunit.client.1.smithi181.stdout:8/968: write d1/f46 [245892,14055] 0 2024-04-27T04:10:21.593 INFO:tasks.workunit.client.0.smithi162.stdout:7/864: dwrite d3/d5/d9/fe5 [0,4194304] 0 2024-04-27T04:10:21.596 INFO:tasks.workunit.client.0.smithi162.stdout:5/885: dwrite d8/d1a/d23/dbc/d78/f9d [0,4194304] 0 2024-04-27T04:10:21.601 INFO:tasks.workunit.client.0.smithi162.stdout:3/817: getdents d1/d70 0 2024-04-27T04:10:21.601 INFO:tasks.workunit.client.1.smithi181.stdout:8/969: link d1/d6/d59/df1/f115 d1/d6/d2f/d47/dba/dbc/f157 0 2024-04-27T04:10:21.606 INFO:tasks.workunit.client.0.smithi162.stdout:9/789: creat d3/d23/ff1 x:0 0 0 2024-04-27T04:10:21.607 INFO:tasks.workunit.client.1.smithi181.stdout:8/970: symlink d1/d6/l158 0 2024-04-27T04:10:21.608 INFO:tasks.workunit.client.0.smithi162.stdout:2/825: creat d0/d4/d2a/f106 x:0 0 0 2024-04-27T04:10:21.610 INFO:tasks.workunit.client.0.smithi162.stdout:5/886: getdents d8 0 2024-04-27T04:10:21.613 INFO:tasks.workunit.client.0.smithi162.stdout:5/887: creat d8/d1a/d23/d40/f116 x:0 0 0 2024-04-27T04:10:21.622 INFO:tasks.workunit.client.1.smithi181.stdout:8/971: dread d1/d6/d2f/d30/d9d/dea/f42 [0,4194304] 0 2024-04-27T04:10:21.635 INFO:tasks.workunit.client.1.smithi181.stdout:8/972: dread d1/d6/d2f/d30/d66/dc0/dcd/de1/fe3 [0,4194304] 0 2024-04-27T04:10:21.638 INFO:tasks.workunit.client.1.smithi181.stdout:4/982: dwrite d0/d15/d25/d2a/d36/f89 [0,4194304] 0 2024-04-27T04:10:21.640 INFO:tasks.workunit.client.1.smithi181.stdout:4/983: read - d0/d15/de/d19/d8a/fbe zero size 2024-04-27T04:10:21.643 INFO:tasks.workunit.client.1.smithi181.stdout:6/910: dwrite d7/dc/d1c/d34/d3f/d7e/d8a/ff0 [0,4194304] 0 2024-04-27T04:10:21.644 INFO:tasks.workunit.client.1.smithi181.stdout:4/984: write d0/d15/de/f34 [3382607,60923] 0 2024-04-27T04:10:21.647 INFO:tasks.workunit.client.1.smithi181.stdout:6/911: fdatasync d7/fb9 0 2024-04-27T04:10:21.648 INFO:tasks.workunit.client.1.smithi181.stdout:8/973: truncate d1/d6/d2f/d30/d66/dc0/dcd/fde 896052 0 2024-04-27T04:10:21.658 INFO:tasks.workunit.client.0.smithi162.stdout:2/826: dwrite d0/d4/d17/d2b/d64/d89/fcc [0,4194304] 0 2024-04-27T04:10:21.659 INFO:tasks.workunit.client.0.smithi162.stdout:6/857: write d6/d1a/d34/d7b/f83 [402667,12715] 0 2024-04-27T04:10:21.660 INFO:tasks.workunit.client.0.smithi162.stdout:6/858: readlink d6/d1a/d34/lee 0 2024-04-27T04:10:21.662 INFO:tasks.workunit.client.0.smithi162.stdout:5/888: link d8/d9/d55/ca6 d8/d1a/d7d/ddf/c117 0 2024-04-27T04:10:21.666 INFO:tasks.workunit.client.0.smithi162.stdout:2/827: truncate d0/d4/d17/f3c 2923150 0 2024-04-27T04:10:21.668 INFO:tasks.workunit.client.0.smithi162.stdout:9/790: dwrite d3/f4 [4194304,4194304] 0 2024-04-27T04:10:21.669 INFO:tasks.workunit.client.0.smithi162.stdout:6/859: rename d6/d1a/d34/d43/f44 to d6/d1a/d34/d3a/dfd/f11a 0 2024-04-27T04:10:21.670 INFO:tasks.workunit.client.1.smithi181.stdout:4/985: write d0/d15/d25/d2b/f2c [980215,50719] 0 2024-04-27T04:10:21.671 INFO:tasks.workunit.client.1.smithi181.stdout:4/986: chown d0/d15/d25/d2b/f10e 0 1 2024-04-27T04:10:21.671 INFO:tasks.workunit.client.1.smithi181.stdout:8/974: write d1/d6/d2f/d40/f4c [2077519,114403] 0 2024-04-27T04:10:21.672 INFO:tasks.workunit.client.0.smithi162.stdout:5/889: truncate d8/d1a/f3e 697273 0 2024-04-27T04:10:21.673 INFO:tasks.workunit.client.1.smithi181.stdout:6/912: truncate d7/d27/fc2 335727 0 2024-04-27T04:10:21.673 INFO:tasks.workunit.client.0.smithi162.stdout:3/818: dread d1/d70/fa5 [0,4194304] 0 2024-04-27T04:10:21.674 INFO:tasks.workunit.client.0.smithi162.stdout:2/828: creat d0/d20/d51/f107 x:0 0 0 2024-04-27T04:10:21.675 INFO:tasks.workunit.client.0.smithi162.stdout:9/791: mknod d3/d39/cf2 0 2024-04-27T04:10:21.676 INFO:tasks.workunit.client.0.smithi162.stdout:6/860: creat d6/d1a/d34/d3a/d68/d6c/d67/f11b x:0 0 0 2024-04-27T04:10:21.677 INFO:tasks.workunit.client.0.smithi162.stdout:5/890: rename d8/d9/d55/ld5 to d8/l118 0 2024-04-27T04:10:21.678 INFO:tasks.workunit.client.1.smithi181.stdout:6/913: mkdir d7/dc/d1e/d2b/d137 0 2024-04-27T04:10:21.679 INFO:tasks.workunit.client.0.smithi162.stdout:3/819: read - d1/d18/d23/d58/d63/dd5/fe7 zero size 2024-04-27T04:10:21.679 INFO:tasks.workunit.client.0.smithi162.stdout:2/829: rmdir d0 39 2024-04-27T04:10:21.680 INFO:tasks.workunit.client.0.smithi162.stdout:9/792: dwrite d3/f4 [4194304,4194304] 0 2024-04-27T04:10:21.681 INFO:tasks.workunit.client.1.smithi181.stdout:8/975: link d1/d6/d2f/d11e/dec/f156 d1/d6/d2f/d47/d137/d14a/f159 0 2024-04-27T04:10:21.682 INFO:tasks.workunit.client.1.smithi181.stdout:8/976: chown d1/d6/d2f/d30/d66/fab 377003993 1 2024-04-27T04:10:21.683 INFO:tasks.workunit.client.1.smithi181.stdout:4/987: creat d0/d15/d25/d2a/d36/f148 x:0 0 0 2024-04-27T04:10:21.685 INFO:tasks.workunit.client.0.smithi162.stdout:6/861: mknod d6/d1a/d34/d3a/d68/d6c/d67/c11c 0 2024-04-27T04:10:21.686 INFO:tasks.workunit.client.1.smithi181.stdout:6/914: dwrite d7/dc/d1c/d7f/d3e/fc0 [0,4194304] 0 2024-04-27T04:10:21.690 INFO:tasks.workunit.client.1.smithi181.stdout:4/988: dread d0/d126/fda [0,4194304] 0 2024-04-27T04:10:21.691 INFO:tasks.workunit.client.0.smithi162.stdout:3/820: symlink d1/d18/d23/l126 0 2024-04-27T04:10:21.692 INFO:tasks.workunit.client.1.smithi181.stdout:4/989: chown d0/d15/d25/d2a/da0/dc7/d109/d7c/db5/f119 133981876 1 2024-04-27T04:10:21.693 INFO:tasks.workunit.client.1.smithi181.stdout:8/977: truncate d1/d6/d2f/d30/d9d/dea/f37 567672 0 2024-04-27T04:10:21.697 INFO:tasks.workunit.client.1.smithi181.stdout:6/915: truncate d7/dc/f12 5544806 0 2024-04-27T04:10:21.701 INFO:tasks.workunit.client.0.smithi162.stdout:2/830: rename d0/d4/d17/c1a to d0/d4/d2a/d3b/d49/d5a/d63/d80/c108 0 2024-04-27T04:10:21.702 INFO:tasks.workunit.client.0.smithi162.stdout:2/831: chown d0/d4/d2a/d41/d77/le8 360352621 1 2024-04-27T04:10:21.708 INFO:tasks.workunit.client.0.smithi162.stdout:3/821: mknod d1/d18/d117/c127 0 2024-04-27T04:10:21.712 INFO:tasks.workunit.client.0.smithi162.stdout:3/822: fdatasync d1/d18/f12 0 2024-04-27T04:10:21.713 INFO:tasks.workunit.client.0.smithi162.stdout:3/823: stat d1/d21/d47/fcd 0 2024-04-27T04:10:21.713 INFO:tasks.workunit.client.0.smithi162.stdout:9/793: write d3/f14 [1455351,91129] 0 2024-04-27T04:10:21.719 INFO:tasks.workunit.client.1.smithi181.stdout:4/990: link d0/d15/d5c/d69/l121 d0/d15/l149 0 2024-04-27T04:10:21.721 INFO:tasks.workunit.client.0.smithi162.stdout:4/789: sync 2024-04-27T04:10:21.721 INFO:tasks.workunit.client.0.smithi162.stdout:8/875: sync 2024-04-27T04:10:21.722 INFO:tasks.workunit.client.0.smithi162.stdout:6/862: dwrite d6/d1a/d34/d3a/d68/d6c/d67/f7c [0,4194304] 0 2024-04-27T04:10:21.723 INFO:tasks.workunit.client.0.smithi162.stdout:9/794: unlink d3/d13/d15/d50/l9e 0 2024-04-27T04:10:21.724 INFO:tasks.workunit.client.1.smithi181.stdout:8/978: dread d1/d6/d6f/dd2/d13f/f4d [0,4194304] 0 2024-04-27T04:10:21.725 INFO:tasks.workunit.client.1.smithi181.stdout:6/916: dread f3 [0,4194304] 0 2024-04-27T04:10:21.727 INFO:tasks.workunit.client.0.smithi162.stdout:2/832: getdents d0/d20 0 2024-04-27T04:10:21.729 INFO:tasks.workunit.client.1.smithi181.stdout:4/991: read - d0/d15/d25/d2a/d36/db8/fec zero size 2024-04-27T04:10:21.731 INFO:tasks.workunit.client.0.smithi162.stdout:3/824: fdatasync d1/d18/d23/d2f/f7c 0 2024-04-27T04:10:21.732 INFO:tasks.workunit.client.1.smithi181.stdout:8/979: creat d1/d6/d2f/d30/de4/f15a x:0 0 0 2024-04-27T04:10:21.734 INFO:tasks.workunit.client.0.smithi162.stdout:4/790: creat d2/d54/d7a/f101 x:0 0 0 2024-04-27T04:10:21.735 INFO:tasks.workunit.client.0.smithi162.stdout:5/891: dread d8/d1a/d23/dbc/f52 [0,4194304] 0 2024-04-27T04:10:21.736 INFO:tasks.workunit.client.0.smithi162.stdout:8/876: symlink d0/d10/d76/l11a 0 2024-04-27T04:10:21.738 INFO:tasks.workunit.client.0.smithi162.stdout:6/863: fdatasync d6/d1a/d34/d3a/d68/d6c/f82 0 2024-04-27T04:10:21.740 INFO:tasks.workunit.client.1.smithi181.stdout:4/992: rmdir d0/d15/d25/d2a/d36 39 2024-04-27T04:10:21.742 INFO:tasks.workunit.client.0.smithi162.stdout:9/795: rename d3/d13/d2d/d3b/d69/cc2 to d3/d39/dc4/cf3 0 2024-04-27T04:10:21.744 INFO:tasks.workunit.client.1.smithi181.stdout:6/917: symlink d7/dc/d1c/d7f/d3e/d118/l138 0 2024-04-27T04:10:21.748 INFO:tasks.workunit.client.1.smithi181.stdout:8/980: fsync d1/d6/d2f/d47/dba/dbc/f157 0 2024-04-27T04:10:21.751 INFO:tasks.workunit.client.0.smithi162.stdout:8/877: chown d0/d10/d44/l57 4 1 2024-04-27T04:10:21.753 INFO:tasks.workunit.client.0.smithi162.stdout:6/864: unlink d6/d1a/d34/d50/dad/ce2 0 2024-04-27T04:10:21.755 INFO:tasks.workunit.client.0.smithi162.stdout:5/892: dread d8/d1a/f45 [0,4194304] 0 2024-04-27T04:10:21.755 INFO:tasks.workunit.client.0.smithi162.stdout:4/791: write d2/d25/d1c/d39/d70/dcd/df0/f97 [587851,31566] 0 2024-04-27T04:10:21.755 INFO:tasks.workunit.client.1.smithi181.stdout:6/918: truncate d7/dc/d1e/f2a 5022763 0 2024-04-27T04:10:21.757 INFO:tasks.workunit.client.0.smithi162.stdout:4/792: write d2/d54/dca/fea [526673,41293] 0 2024-04-27T04:10:21.757 INFO:tasks.workunit.client.1.smithi181.stdout:4/993: creat d0/d15/de/f14a x:0 0 0 2024-04-27T04:10:21.758 INFO:tasks.workunit.client.0.smithi162.stdout:2/833: mkdir d0/d4/d109 0 2024-04-27T04:10:21.759 INFO:tasks.workunit.client.1.smithi181.stdout:4/994: write d0/d15/d25/d2b/d6d/d82/f102 [1981940,63324] 0 2024-04-27T04:10:21.759 INFO:tasks.workunit.client.1.smithi181.stdout:8/981: write d1/d20/df4/f10b [77805,91322] 0 2024-04-27T04:10:21.760 INFO:tasks.workunit.client.0.smithi162.stdout:2/834: chown d0/d4/ld 3799120 1 2024-04-27T04:10:21.760 INFO:tasks.workunit.client.1.smithi181.stdout:4/995: dread - d0/d15/d25/d2a/d61/d11f/d80/fe5 zero size 2024-04-27T04:10:21.763 INFO:tasks.workunit.client.0.smithi162.stdout:2/835: write d0/d4/d2a/d3b/d49/d5a/d63/d84/fbf [977431,2097] 0 2024-04-27T04:10:21.765 INFO:tasks.workunit.client.0.smithi162.stdout:4/793: dwrite d2/d25/d1c/d39/d3b/d47/f6c [4194304,4194304] 0 2024-04-27T04:10:21.767 INFO:tasks.workunit.client.0.smithi162.stdout:8/878: unlink d0/d10/d2b/d2d/d2f/d6b/f10f 0 2024-04-27T04:10:21.767 INFO:tasks.workunit.client.1.smithi181.stdout:4/996: rename d0/d15/d76/db1/fb6 to d0/d15/de/d19/d8a/f14b 0 2024-04-27T04:10:21.768 INFO:tasks.workunit.client.0.smithi162.stdout:8/879: write d0/d10/d2b/f115 [3295682,121120] 0 2024-04-27T04:10:21.772 INFO:tasks.workunit.client.1.smithi181.stdout:6/919: write d7/dc/dc3/f70 [713509,2954] 0 2024-04-27T04:10:21.773 INFO:tasks.workunit.client.0.smithi162.stdout:5/893: dread d8/d1a/f51 [0,4194304] 0 2024-04-27T04:10:21.774 INFO:tasks.workunit.client.1.smithi181.stdout:8/982: write d1/d6/d2f/d40/fa1 [3231244,68733] 0 2024-04-27T04:10:21.775 INFO:tasks.workunit.client.1.smithi181.stdout:6/920: truncate d7/dc/d1c/d88/db3/f136 687903 0 2024-04-27T04:10:21.776 INFO:tasks.workunit.client.1.smithi181.stdout:4/997: creat d0/d15/da5/f14c x:0 0 0 2024-04-27T04:10:21.779 INFO:tasks.workunit.client.1.smithi181.stdout:8/983: symlink d1/d6/d6f/d91/l15b 0 2024-04-27T04:10:21.779 INFO:tasks.workunit.client.0.smithi162.stdout:9/796: write d3/d13/d15/d7e/f92 [4105100,86611] 0 2024-04-27T04:10:21.780 INFO:tasks.workunit.client.0.smithi162.stdout:2/836: mkdir d0/d4/d17/d2b/d64/d89/dbc/d10a 0 2024-04-27T04:10:21.781 INFO:tasks.workunit.client.1.smithi181.stdout:8/984: truncate d1/d6/d2f/d30/de4/f15a 170815 0 2024-04-27T04:10:21.781 INFO:tasks.workunit.client.1.smithi181.stdout:6/921: fsync d7/dc/d1e/f53 0 2024-04-27T04:10:21.784 INFO:tasks.workunit.client.0.smithi162.stdout:6/865: mkdir d6/db/d32/df0/d11d 0 2024-04-27T04:10:21.787 INFO:tasks.workunit.client.0.smithi162.stdout:9/797: dwrite d3/d23/d1e/f4a [4194304,4194304] 0 2024-04-27T04:10:21.788 INFO:tasks.workunit.client.0.smithi162.stdout:8/880: unlink d0/de/fe6 0 2024-04-27T04:10:21.789 INFO:tasks.workunit.client.0.smithi162.stdout:8/881: write d0/de/d6e/f51 [67402,49167] 0 2024-04-27T04:10:21.789 INFO:tasks.workunit.client.1.smithi181.stdout:6/922: creat d7/dc/dc3/dd1/f139 x:0 0 0 2024-04-27T04:10:21.791 INFO:tasks.workunit.client.0.smithi162.stdout:4/794: write d2/d25/d1c/d39/d3b/da7/fcb [998930,83768] 0 2024-04-27T04:10:21.794 INFO:tasks.workunit.client.0.smithi162.stdout:4/795: read d2/d25/d1c/d39/d3b/f94 [1010177,3851] 0 2024-04-27T04:10:21.794 INFO:tasks.workunit.client.0.smithi162.stdout:5/894: rename d8/d1a/d23/f50 to d8/d1a/d23/d40/f119 0 2024-04-27T04:10:21.795 INFO:tasks.workunit.client.0.smithi162.stdout:8/882: rename d0/d10 to d0/d10/d2b/d2d/d2f/d6b/dc7/d11b 22 2024-04-27T04:10:21.795 INFO:tasks.workunit.client.1.smithi181.stdout:8/985: write d1/d6/d2f/d30/faf [359529,233] 0 2024-04-27T04:10:21.796 INFO:tasks.workunit.client.0.smithi162.stdout:5/895: write d8/d1a/ffb [763176,101016] 0 2024-04-27T04:10:21.797 INFO:tasks.workunit.client.1.smithi181.stdout:8/986: stat d1/d6/d2f/d30/d9d/dea/db3/ddf/le0 0 2024-04-27T04:10:21.799 INFO:tasks.workunit.client.0.smithi162.stdout:2/837: creat d0/d4/d17/d4f/d7f/f10b x:0 0 0 2024-04-27T04:10:21.801 INFO:tasks.workunit.client.0.smithi162.stdout:6/866: symlink d6/d1a/d34/d3a/dbf/l11e 0 2024-04-27T04:10:21.802 INFO:tasks.workunit.client.0.smithi162.stdout:5/896: dwrite d8/d1a/d23/dbc/fa8 [0,4194304] 0 2024-04-27T04:10:21.804 INFO:tasks.workunit.client.0.smithi162.stdout:5/897: read - d8/d1a/d23/d40/f116 zero size 2024-04-27T04:10:21.805 INFO:tasks.workunit.client.0.smithi162.stdout:9/798: fdatasync d3/d11/d45/f55 0 2024-04-27T04:10:21.806 INFO:tasks.workunit.client.1.smithi181.stdout:4/998: dwrite d0/d15/d5c/fb2 [0,4194304] 0 2024-04-27T04:10:21.811 INFO:tasks.workunit.client.1.smithi181.stdout:6/923: getdents d7/dc/d1c/d34 0 2024-04-27T04:10:21.813 INFO:tasks.workunit.client.1.smithi181.stdout:8/987: write d1/d6/d2f/d30/d66/dc0/dcd/ffd [53900,116606] 0 2024-04-27T04:10:21.814 INFO:tasks.workunit.client.0.smithi162.stdout:8/883: symlink d0/d10/d2b/d2e/de7/l11c 0 2024-04-27T04:10:21.814 INFO:tasks.workunit.client.1.smithi181.stdout:6/924: rmdir d7/d27 39 2024-04-27T04:10:21.815 INFO:tasks.workunit.client.1.smithi181.stdout:6/925: chown d7/dc/d1c/d34/d3f/d7e/d8a/f8e 1842875 1 2024-04-27T04:10:21.818 INFO:tasks.workunit.client.0.smithi162.stdout:2/838: rename d0/d4/d17/d4f/d7f to d0/d4/d17/d4f/d10c 0 2024-04-27T04:10:21.821 INFO:tasks.workunit.client.0.smithi162.stdout:6/867: mkdir d6/d1a/d34/d50/d11f 0 2024-04-27T04:10:21.821 INFO:tasks.workunit.client.1.smithi181.stdout:6/926: mknod d7/dc/d1c/d7f/d3e/d118/c13a 0 2024-04-27T04:10:21.825 INFO:tasks.workunit.client.1.smithi181.stdout:6/927: rmdir d7/dc/dc3/dd1 39 2024-04-27T04:10:21.826 INFO:tasks.workunit.client.0.smithi162.stdout:9/799: rmdir d3/d13/d2d/d3b/d7a 39 2024-04-27T04:10:21.826 INFO:tasks.workunit.client.0.smithi162.stdout:6/868: dwrite d6/f35 [0,4194304] 0 2024-04-27T04:10:21.827 INFO:tasks.workunit.client.0.smithi162.stdout:4/796: unlink d2/d25/d1c/d24/c8b 0 2024-04-27T04:10:21.828 INFO:tasks.workunit.client.0.smithi162.stdout:2/839: mkdir d0/d4/d2a/d3b/d49/d5a/d63/d70/d8e/d10d 0 2024-04-27T04:10:21.829 INFO:tasks.workunit.client.1.smithi181.stdout:6/928: mkdir d7/dc/d1c/d34/d3f/d13b 0 2024-04-27T04:10:21.831 INFO:tasks.workunit.client.0.smithi162.stdout:9/800: symlink d3/d23/d5d/lf4 0 2024-04-27T04:10:21.834 INFO:tasks.workunit.client.1.smithi181.stdout:6/929: read d7/dc/f12 [2370669,14137] 0 2024-04-27T04:10:21.834 INFO:tasks.workunit.client.0.smithi162.stdout:8/884: dwrite d0/d10/d25/d7d/d87/f102 [0,4194304] 0 2024-04-27T04:10:21.836 INFO:tasks.workunit.client.0.smithi162.stdout:5/898: write f4 [8897146,24927] 0 2024-04-27T04:10:21.838 INFO:tasks.workunit.client.0.smithi162.stdout:4/797: rename d2/d25/d1c/d39/d90/dad/dfd/cf5 to d2/d25/d1c/d39/d3b/d6e/d6f/c102 0 2024-04-27T04:10:21.838 INFO:tasks.workunit.client.1.smithi181.stdout:8/988: dwrite d1/d6/d6f/dd2/f10c [0,4194304] 0 2024-04-27T04:10:21.841 INFO:tasks.workunit.client.1.smithi181.stdout:6/930: symlink d7/dc/d1c/d88/db3/l13c 0 2024-04-27T04:10:21.842 INFO:tasks.workunit.client.0.smithi162.stdout:8/885: creat d0/d10/d44/db2/f11d x:0 0 0 2024-04-27T04:10:21.846 INFO:tasks.workunit.client.1.smithi181.stdout:6/931: rmdir d7/dc/d1c/d7f 39 2024-04-27T04:10:21.850 INFO:tasks.workunit.client.0.smithi162.stdout:2/840: creat d0/d4/d17/f10e x:0 0 0 2024-04-27T04:10:21.854 INFO:tasks.workunit.client.1.smithi181.stdout:8/989: creat d1/d6/d6f/dd2/d13f/f15c x:0 0 0 2024-04-27T04:10:21.855 INFO:tasks.workunit.client.0.smithi162.stdout:0/942: sync 2024-04-27T04:10:21.855 INFO:tasks.workunit.client.0.smithi162.stdout:7/865: sync 2024-04-27T04:10:21.855 INFO:tasks.workunit.client.0.smithi162.stdout:1/767: sync 2024-04-27T04:10:21.855 INFO:tasks.workunit.client.0.smithi162.stdout:1/768: readlink d0/d7/d3c/d77/l76 0 2024-04-27T04:10:21.856 INFO:tasks.workunit.client.0.smithi162.stdout:8/886: creat d0/d10/d25/d7d/db8/f11e x:0 0 0 2024-04-27T04:10:21.856 INFO:tasks.workunit.client.0.smithi162.stdout:7/866: chown d3/d5/d5b/d83/f2e 1606 1 2024-04-27T04:10:21.857 INFO:tasks.workunit.client.1.smithi181.stdout:6/932: mknod d7/dc/d1c/d88/c13d 0 2024-04-27T04:10:21.857 INFO:tasks.workunit.client.1.smithi181.stdout:6/933: chown d7/dc/d1e/d66/f90 2814 1 2024-04-27T04:10:21.858 INFO:tasks.workunit.client.0.smithi162.stdout:9/801: mkdir d3/d13/d2d/d3b/d72/df5 0 2024-04-27T04:10:21.860 INFO:tasks.workunit.client.0.smithi162.stdout:2/841: fdatasync d0/d4/d2a/f9e 0 2024-04-27T04:10:21.861 INFO:tasks.workunit.client.0.smithi162.stdout:4/798: mknod d2/d25/d1c/d39/df3/c103 0 2024-04-27T04:10:21.861 INFO:tasks.workunit.client.1.smithi181.stdout:6/934: truncate d7/dc/dc3/f4b 7480026 0 2024-04-27T04:10:21.862 INFO:tasks.workunit.client.0.smithi162.stdout:1/769: dwrite d0/d11/d3e/d5f/f6a [0,4194304] 0 2024-04-27T04:10:21.863 INFO:tasks.workunit.client.0.smithi162.stdout:4/799: write d2/d25/d1c/d24/d100/de9/ff6 [229320,70852] 0 2024-04-27T04:10:21.864 INFO:tasks.workunit.client.0.smithi162.stdout:7/867: creat d3/d5/d5b/d83/de/d2b/d81/daa/deb/f123 x:0 0 0 2024-04-27T04:10:21.867 INFO:tasks.workunit.client.0.smithi162.stdout:5/899: dread d8/d1a/d29/f95 [0,4194304] 0 2024-04-27T04:10:21.868 INFO:tasks.workunit.client.0.smithi162.stdout:2/842: mknod d0/d4/d2a/d41/d77/c10f 0 2024-04-27T04:10:21.869 INFO:tasks.workunit.client.0.smithi162.stdout:0/943: symlink d5/dc/d1e/d41/d79/d12c/l138 0 2024-04-27T04:10:21.870 INFO:tasks.workunit.client.0.smithi162.stdout:2/843: dread d0/d4/d2a/d41/d77/d73/fd3 [0,4194304] 0 2024-04-27T04:10:21.871 INFO:tasks.workunit.client.0.smithi162.stdout:2/844: read - d0/d4/d17/f10e zero size 2024-04-27T04:10:21.872 INFO:tasks.workunit.client.0.smithi162.stdout:2/845: write d0/d4/d2a/d3b/d49/d5a/d63/d70/d8e/fe5 [4686344,54489] 0 2024-04-27T04:10:21.873 INFO:tasks.workunit.client.1.smithi181.stdout:6/935: rename d7/l114 to d7/dc/d1c/d88/db3/ddf/l13e 0 2024-04-27T04:10:21.876 INFO:tasks.workunit.client.0.smithi162.stdout:4/800: creat d2/d25/d1c/d39/d90/dad/dfd/f104 x:0 0 0 2024-04-27T04:10:21.876 INFO:tasks.workunit.client.1.smithi181.stdout:8/990: dwrite d1/d6/d6f/d91/f152 [0,4194304] 0 2024-04-27T04:10:21.876 INFO:tasks.workunit.client.0.smithi162.stdout:4/801: stat d2/d25/d1c/d39/c75 0 2024-04-27T04:10:21.876 INFO:tasks.workunit.client.1.smithi181.stdout:8/991: fdatasync d1/d6/d2f/d30/de4/f15a 0 2024-04-27T04:10:21.878 INFO:tasks.workunit.client.0.smithi162.stdout:9/802: unlink d3/d13/d2d/d3b/f47 0 2024-04-27T04:10:21.880 INFO:tasks.workunit.client.1.smithi181.stdout:6/936: rmdir d7/dc/dc3/d72/daa 39 2024-04-27T04:10:21.886 INFO:tasks.workunit.client.1.smithi181.stdout:8/992: unlink d1/d6/l32 0 2024-04-27T04:10:21.896 INFO:tasks.workunit.client.1.smithi181.stdout:6/937: write d7/dc/d1c/d34/d3f/d7e/d8a/f8e [666319,64791] 0 2024-04-27T04:10:21.897 INFO:tasks.workunit.client.1.smithi181.stdout:6/938: write d7/dc/d1c/d34/d3f/d7e/d8a/f7b [4086523,112559] 0 2024-04-27T04:10:21.902 INFO:tasks.workunit.client.1.smithi181.stdout:6/939: write d7/fd [4479872,73930] 0 2024-04-27T04:10:21.905 INFO:tasks.workunit.client.1.smithi181.stdout:6/940: symlink d7/dc/d1e/d2b/d10b/l13f 0 2024-04-27T04:10:21.906 INFO:tasks.workunit.client.0.smithi162.stdout:5/900: unlink d8/d1a/d23/dbc/d66/cbe 0 2024-04-27T04:10:21.907 INFO:tasks.workunit.client.0.smithi162.stdout:0/944: truncate d5/d26/d4a/f96 466429 0 2024-04-27T04:10:21.908 INFO:tasks.workunit.client.0.smithi162.stdout:4/802: symlink d2/d54/dca/l105 0 2024-04-27T04:10:21.909 INFO:tasks.workunit.client.1.smithi181.stdout:6/941: mkdir d7/dc/dc3/d72/daa/dd3/d103/d140 0 2024-04-27T04:10:21.909 INFO:tasks.workunit.client.1.smithi181.stdout:6/942: dread - d7/dc/d1e/d2b/dde/fe7 zero size 2024-04-27T04:10:21.910 INFO:tasks.workunit.client.0.smithi162.stdout:9/803: creat d3/d39/ff6 x:0 0 0 2024-04-27T04:10:21.912 INFO:tasks.workunit.client.1.smithi181.stdout:6/943: truncate d7/dc/d1c/d34/d3f/d7e/d8a/f80 594172 0 2024-04-27T04:10:21.913 INFO:tasks.workunit.client.0.smithi162.stdout:7/868: dread d3/d5/d5b/d83/de/d2b/d81/daa/dbd/dbe/dff/f77 [0,4194304] 0 2024-04-27T04:10:21.914 INFO:tasks.workunit.client.0.smithi162.stdout:4/803: read - d2/d25/d1c/d39/d90/fcc zero size 2024-04-27T04:10:21.914 INFO:tasks.workunit.client.0.smithi162.stdout:5/901: fsync d8/d9/f16 0 2024-04-27T04:10:21.915 INFO:tasks.workunit.client.0.smithi162.stdout:1/770: dwrite d0/d11/d50/d53/d93/d96/daa/fce [0,4194304] 0 2024-04-27T04:10:21.916 INFO:tasks.workunit.client.1.smithi181.stdout:6/944: mknod d7/dc/d1e/d66/c141 0 2024-04-27T04:10:21.918 INFO:tasks.workunit.client.0.smithi162.stdout:9/804: creat d3/d13/d2d/d30/d7f/ff7 x:0 0 0 2024-04-27T04:10:21.919 INFO:tasks.workunit.client.0.smithi162.stdout:1/771: stat d0/d7/d3c/d54/fa4 0 2024-04-27T04:10:21.920 INFO:tasks.workunit.client.0.smithi162.stdout:1/772: fsync d0/f3b 0 2024-04-27T04:10:21.920 INFO:tasks.workunit.client.0.smithi162.stdout:2/846: link d0/d4/d2a/d3b/d49/d5a/d63/d80/c108 d0/d4/d17/d2b/d64/d89/dbc/c110 0 2024-04-27T04:10:21.921 INFO:tasks.workunit.client.1.smithi181.stdout:6/945: truncate d7/dc/d1e/f2a 1458215 0 2024-04-27T04:10:21.922 INFO:tasks.workunit.client.0.smithi162.stdout:5/902: fdatasync d8/d9/fb0 0 2024-04-27T04:10:21.923 INFO:tasks.workunit.client.0.smithi162.stdout:0/945: dwrite d5/dc/d1e/d46/f55 [4194304,4194304] 0 2024-04-27T04:10:21.924 INFO:tasks.workunit.client.1.smithi181.stdout:6/946: fdatasync d7/fb 0 2024-04-27T04:10:21.924 INFO:tasks.workunit.client.0.smithi162.stdout:2/847: readlink d0/le2 0 2024-04-27T04:10:21.927 INFO:tasks.workunit.client.0.smithi162.stdout:5/903: rmdir d8/d9/d5b 39 2024-04-27T04:10:21.930 INFO:tasks.workunit.client.0.smithi162.stdout:2/848: readlink d0/d4/d17/d4f/d10c/ld9 0 2024-04-27T04:10:21.930 INFO:tasks.workunit.client.0.smithi162.stdout:1/773: mkdir d0/d7/d10/d75/db7/dfd 0 2024-04-27T04:10:21.931 INFO:tasks.workunit.client.0.smithi162.stdout:7/869: dwrite d3/d5/d5b/d83/de/d2b/d81/fad [4194304,4194304] 0 2024-04-27T04:10:21.933 INFO:tasks.workunit.client.0.smithi162.stdout:9/805: dread d3/d13/d2d/d3b/f84 [4194304,4194304] 0 2024-04-27T04:10:21.934 INFO:tasks.workunit.client.0.smithi162.stdout:9/806: write d3/f14 [2395344,22643] 0 2024-04-27T04:10:21.936 INFO:tasks.workunit.client.0.smithi162.stdout:0/946: truncate d5/d26/d4a/f96 798766 0 2024-04-27T04:10:21.936 INFO:tasks.workunit.client.0.smithi162.stdout:9/807: chown d3/d13/d2d/d3b/d72/fea 0 1 2024-04-27T04:10:21.937 INFO:tasks.workunit.client.0.smithi162.stdout:2/849: mkdir d0/d4/d17/d2b/d64/d96/daa/d111 0 2024-04-27T04:10:21.938 INFO:tasks.workunit.client.0.smithi162.stdout:9/808: stat d3/c62 0 2024-04-27T04:10:21.938 INFO:tasks.workunit.client.0.smithi162.stdout:2/850: write d0/fe6 [559923,17921] 0 2024-04-27T04:10:21.940 INFO:tasks.workunit.client.0.smithi162.stdout:1/774: unlink d0/d7/d3c/d77/ld7 0 2024-04-27T04:10:21.944 INFO:tasks.workunit.client.0.smithi162.stdout:0/947: unlink d5/dc/d1e/d20/dba/df9/d8b/c12b 0 2024-04-27T04:10:21.944 INFO:tasks.workunit.client.0.smithi162.stdout:9/809: chown d3/d13/d2d/d4f 1925 1 2024-04-27T04:10:21.945 INFO:tasks.workunit.client.1.smithi181.stdout:6/947: dread d7/d27/fae [0,4194304] 0 2024-04-27T04:10:21.947 INFO:tasks.workunit.client.0.smithi162.stdout:1/775: unlink d0/d7/d2a/l46 0 2024-04-27T04:10:21.950 INFO:tasks.workunit.client.0.smithi162.stdout:5/904: write d8/d9/d55/f69 [1141270,64213] 0 2024-04-27T04:10:21.950 INFO:tasks.workunit.client.1.smithi181.stdout:6/948: mknod d7/dc/dc3/d72/c142 0 2024-04-27T04:10:21.953 INFO:tasks.workunit.client.0.smithi162.stdout:1/776: fdatasync d0/d11/d3e/f6d 0 2024-04-27T04:10:21.958 INFO:tasks.workunit.client.1.smithi181.stdout:6/949: creat d7/dc/d1c/d7f/d3e/d118/f143 x:0 0 0 2024-04-27T04:10:21.958 INFO:tasks.workunit.client.0.smithi162.stdout:0/948: rename d5/dc/d1e/d46/d48/d9e/dce/dd1/d116/c103 to d5/dc/d1e/d46/d48/d9e/dce/dd1/d116/da8/c139 0 2024-04-27T04:10:21.960 INFO:tasks.workunit.client.0.smithi162.stdout:7/870: dwrite d3/d5/d5b/d83/de/d2b/d81/d96/f99 [0,4194304] 0 2024-04-27T04:10:21.961 INFO:tasks.workunit.client.1.smithi181.stdout:6/950: symlink d7/dc/d1c/d88/db3/l144 0 2024-04-27T04:10:21.963 INFO:tasks.workunit.client.1.smithi181.stdout:6/951: dread - d7/dc/dc3/d72/dd6/f108 zero size 2024-04-27T04:10:21.964 INFO:tasks.workunit.client.0.smithi162.stdout:2/851: dwrite d0/d4/d17/d2b/d64/d96/fea [0,4194304] 0 2024-04-27T04:10:21.966 INFO:tasks.workunit.client.0.smithi162.stdout:0/949: creat d5/dc/d1e/d46/d48/d9e/dce/dd1/d116/da8/f13a x:0 0 0 2024-04-27T04:10:21.966 INFO:tasks.workunit.client.1.smithi181.stdout:6/952: creat d7/dc/dc3/d72/daa/d110/f145 x:0 0 0 2024-04-27T04:10:21.967 INFO:tasks.workunit.client.1.smithi181.stdout:6/953: readlink d7/dc/d1c/d88/db3/lc5 0 2024-04-27T04:10:21.967 INFO:tasks.workunit.client.0.smithi162.stdout:9/810: link d3/d13/d2d/d30/d67/f97 d3/d13/d2d/d30/d7f/ff8 0 2024-04-27T04:10:21.968 INFO:tasks.workunit.client.1.smithi181.stdout:6/954: chown d7/dc/d1c/d88/d6d/la0 99787031 1 2024-04-27T04:10:21.971 INFO:tasks.workunit.client.0.smithi162.stdout:2/852: mkdir d0/d4/d17/d2b/d112 0 2024-04-27T04:10:21.976 INFO:tasks.workunit.client.1.smithi181.stdout:6/955: dwrite d7/d27/f121 [0,4194304] 0 2024-04-27T04:10:21.979 INFO:tasks.workunit.client.0.smithi162.stdout:2/853: dwrite d0/d4/d17/d2b/d64/d96/feb [0,4194304] 0 2024-04-27T04:10:21.979 INFO:tasks.workunit.client.0.smithi162.stdout:9/811: rename d3/d23/d5d to d3/d23/d1e/d43/df9 0 2024-04-27T04:10:21.981 INFO:tasks.workunit.client.0.smithi162.stdout:2/854: write d0/d4/d17/d2b/d64/d89/fb9 [3224366,60810] 0 2024-04-27T04:10:21.982 INFO:tasks.workunit.client.0.smithi162.stdout:1/777: link d0/d7/d2a/f31 d0/d11/d5b/ffe 0 2024-04-27T04:10:21.985 INFO:tasks.workunit.client.0.smithi162.stdout:1/778: chown d0/d7/d10/f26 23 1 2024-04-27T04:10:21.993 INFO:tasks.workunit.client.0.smithi162.stdout:5/905: getdents d8/d9/d5b/d111 0 2024-04-27T04:10:21.993 INFO:tasks.workunit.client.0.smithi162.stdout:1/779: dwrite d0/d80/d89/fe2 [4194304,4194304] 0 2024-04-27T04:10:21.998 INFO:tasks.workunit.client.0.smithi162.stdout:7/871: truncate d3/f18 2020933 0 2024-04-27T04:10:22.001 INFO:tasks.workunit.client.0.smithi162.stdout:3/825: sync 2024-04-27T04:10:22.003 INFO:tasks.workunit.client.1.smithi181.stdout:6/956: write d7/dc/d1e/d2b/f85 [231015,75808] 0 2024-04-27T04:10:22.004 INFO:tasks.workunit.client.0.smithi162.stdout:0/950: dwrite d5/dc/d1e/f30 [0,4194304] 0 2024-04-27T04:10:22.004 INFO:tasks.workunit.client.0.smithi162.stdout:2/855: link d0/d4/d2a/d3b/d49/l50 d0/d4/d2a/da1/dde/l113 0 2024-04-27T04:10:22.007 INFO:tasks.workunit.client.0.smithi162.stdout:3/826: creat d1/d18/d23/d34/dad/f128 x:0 0 0 2024-04-27T04:10:22.011 INFO:tasks.workunit.client.0.smithi162.stdout:7/872: symlink d3/d5/d5b/d83/de/d2b/da7/l124 0 2024-04-27T04:10:22.011 INFO:tasks.workunit.client.0.smithi162.stdout:5/906: dwrite d8/de3/deb/fbd [0,4194304] 0 2024-04-27T04:10:22.012 INFO:tasks.workunit.client.0.smithi162.stdout:9/812: write d3/d13/d2d/d30/d67/f97 [978894,92736] 0 2024-04-27T04:10:22.013 INFO:tasks.workunit.client.0.smithi162.stdout:5/907: stat d8/d1a/d23/d40/ldb 0 2024-04-27T04:10:22.015 INFO:tasks.workunit.client.0.smithi162.stdout:1/780: write d0/d11/f79 [176068,111631] 0 2024-04-27T04:10:22.016 INFO:tasks.workunit.client.0.smithi162.stdout:7/873: creat d3/d5/d3c/d4d/f125 x:0 0 0 2024-04-27T04:10:22.020 INFO:tasks.workunit.client.1.smithi181.stdout:6/957: write d7/dc/d1e/f53 [2221011,11874] 0 2024-04-27T04:10:22.021 INFO:tasks.workunit.client.0.smithi162.stdout:0/951: write d5/f27 [908000,33461] 0 2024-04-27T04:10:22.022 INFO:tasks.workunit.client.0.smithi162.stdout:7/874: dread d3/d5/d5b/d83/de/d2b/d81/daa/dbd/dab/fed [0,4194304] 0 2024-04-27T04:10:22.022 INFO:tasks.workunit.client.1.smithi181.stdout:6/958: write d7/dc/d1c/f91 [465312,96907] 0 2024-04-27T04:10:22.024 INFO:tasks.workunit.client.0.smithi162.stdout:3/827: write d1/f89 [512355,35182] 0 2024-04-27T04:10:22.027 INFO:tasks.workunit.client.0.smithi162.stdout:9/813: fsync d3/d13/d2d/d3b/d69/f7d 0 2024-04-27T04:10:22.029 INFO:tasks.workunit.client.0.smithi162.stdout:7/875: mknod d3/d5/d5b/d83/de/d2b/d81/d117/c126 0 2024-04-27T04:10:22.029 INFO:tasks.workunit.client.0.smithi162.stdout:1/781: dread d0/d11/d50/fbd [4194304,4194304] 0 2024-04-27T04:10:22.030 INFO:tasks.workunit.client.1.smithi181.stdout:6/959: creat d7/dc/dc3/dd1/dd9/f146 x:0 0 0 2024-04-27T04:10:22.031 INFO:tasks.workunit.client.0.smithi162.stdout:3/828: write d1/d18/d23/d58/d63/d88/fe8 [220571,118515] 0 2024-04-27T04:10:22.032 INFO:tasks.workunit.client.0.smithi162.stdout:7/876: truncate d3/d4f/fc3 1686745 0 2024-04-27T04:10:22.033 INFO:tasks.workunit.client.0.smithi162.stdout:7/877: chown d3/d5/d5b/d83/de/d2b/d81 1760 1 2024-04-27T04:10:22.035 INFO:tasks.workunit.client.0.smithi162.stdout:1/782: creat d0/d11/fff x:0 0 0 2024-04-27T04:10:22.035 INFO:tasks.workunit.client.0.smithi162.stdout:3/829: mknod d1/d18/d23/d58/d63/dfa/c129 0 2024-04-27T04:10:22.036 INFO:tasks.workunit.client.1.smithi181.stdout:6/960: dwrite f3 [0,4194304] 0 2024-04-27T04:10:22.039 INFO:tasks.workunit.client.0.smithi162.stdout:7/878: mkdir d3/d5/d5b/d83/de/d2b/d81/daa/dbd/dbe/d127 0 2024-04-27T04:10:22.039 INFO:tasks.workunit.client.0.smithi162.stdout:9/814: getdents d3 0 2024-04-27T04:10:22.040 INFO:tasks.workunit.client.1.smithi181.stdout:6/961: fdatasync d7/dc/dc3/d62/f63 0 2024-04-27T04:10:22.042 INFO:tasks.workunit.client.0.smithi162.stdout:3/830: truncate d1/d18/d5d/d6f/f80 1009307 0 2024-04-27T04:10:22.043 INFO:tasks.workunit.client.0.smithi162.stdout:9/815: mkdir d3/d23/d1e/dfa 0 2024-04-27T04:10:22.044 INFO:tasks.workunit.client.1.smithi181.stdout:6/962: unlink d7/d27/fe3 0 2024-04-27T04:10:22.046 INFO:tasks.workunit.client.0.smithi162.stdout:5/908: dread f4 [4194304,4194304] 0 2024-04-27T04:10:22.047 INFO:tasks.workunit.client.0.smithi162.stdout:7/879: mkdir d3/d5/d5b/d83/de/d2b/d81/d96/dc8/d128 0 2024-04-27T04:10:22.047 INFO:tasks.workunit.client.1.smithi181.stdout:6/963: unlink d7/dc/f3a 0 2024-04-27T04:10:22.048 INFO:tasks.workunit.client.0.smithi162.stdout:1/783: truncate d0/d11/d50/d53/d93/d96/ff6 969196 0 2024-04-27T04:10:22.054 INFO:tasks.workunit.client.0.smithi162.stdout:9/816: dread d3/d11/d45/d4e/fd7 [0,4194304] 0 2024-04-27T04:10:22.054 INFO:tasks.workunit.client.0.smithi162.stdout:5/909: chown d8/d1a/d29/f95 1 1 2024-04-27T04:10:22.054 INFO:tasks.workunit.client.0.smithi162.stdout:0/952: dwrite d5/dc/d1e/d20/dec/df1/db4/fe0 [0,4194304] 0 2024-04-27T04:10:22.057 INFO:tasks.workunit.client.0.smithi162.stdout:1/784: dread d0/d7/d3c/d77/f28 [0,4194304] 0 2024-04-27T04:10:22.059 INFO:tasks.workunit.client.0.smithi162.stdout:9/817: chown d3/d13/d15/l66 1728071 1 2024-04-27T04:10:22.060 INFO:tasks.workunit.client.0.smithi162.stdout:3/831: dwrite d1/d18/d23/d58/d63/dd5/f11e [0,4194304] 0 2024-04-27T04:10:22.064 INFO:tasks.workunit.client.0.smithi162.stdout:3/832: rmdir d1/d18/d23 39 2024-04-27T04:10:22.065 INFO:tasks.workunit.client.0.smithi162.stdout:7/880: dread d3/d5/d5b/d83/f6e [0,4194304] 0 2024-04-27T04:10:22.066 INFO:tasks.workunit.client.0.smithi162.stdout:0/953: read d5/dc/d1e/d20/f110 [1209222,20323] 0 2024-04-27T04:10:22.066 INFO:tasks.workunit.client.0.smithi162.stdout:5/910: mknod d8/d1a/d23/dbc/c11a 0 2024-04-27T04:10:22.068 INFO:tasks.workunit.client.0.smithi162.stdout:3/833: chown d1/d18/d5d/d5f/dbe/cc2 1728452 1 2024-04-27T04:10:22.069 INFO:tasks.workunit.client.0.smithi162.stdout:0/954: write d5/f3b [1206977,2163] 0 2024-04-27T04:10:22.071 INFO:tasks.workunit.client.0.smithi162.stdout:3/834: fdatasync d1/d21/d47/d57/f5e 0 2024-04-27T04:10:22.071 INFO:tasks.workunit.client.0.smithi162.stdout:7/881: chown d3/l22 12 1 2024-04-27T04:10:22.071 INFO:tasks.workunit.client.0.smithi162.stdout:5/911: write d8/d9/f113 [70500,124585] 0 2024-04-27T04:10:22.073 INFO:tasks.workunit.client.0.smithi162.stdout:1/785: mknod d0/d7/d3c/d54/dae/c100 0 2024-04-27T04:10:22.075 INFO:tasks.workunit.client.0.smithi162.stdout:3/835: mkdir d1/d18/d117/d12a 0 2024-04-27T04:10:22.075 INFO:tasks.workunit.client.1.smithi181.stdout:6/964: dwrite d7/dc/d1e/f83 [0,4194304] 0 2024-04-27T04:10:22.076 INFO:tasks.workunit.client.0.smithi162.stdout:3/836: chown d1/d45/f9c 1 1 2024-04-27T04:10:22.077 INFO:tasks.workunit.client.0.smithi162.stdout:1/786: mkdir d0/d11/d50/d53/d101 0 2024-04-27T04:10:22.080 INFO:tasks.workunit.client.0.smithi162.stdout:0/955: dread d5/dc/d1e/d46/d48/d9e/db8/fb9 [0,4194304] 0 2024-04-27T04:10:22.081 INFO:tasks.workunit.client.0.smithi162.stdout:5/912: creat d8/d9/d5b/d111/d112/f11b x:0 0 0 2024-04-27T04:10:22.083 INFO:tasks.workunit.client.1.smithi181.stdout:6/965: rename d7/dc/d1c/d88/db3/ddf to d7/dc/dc3/d72/daa/dd3/d147 0 2024-04-27T04:10:22.084 INFO:tasks.workunit.client.1.smithi181.stdout:6/966: dread - d7/dc/d1e/f12c zero size 2024-04-27T04:10:22.085 INFO:tasks.workunit.client.0.smithi162.stdout:9/818: dread d3/f5 [0,4194304] 0 2024-04-27T04:10:22.086 INFO:tasks.workunit.client.0.smithi162.stdout:1/787: creat d0/d11/d50/d53/d93/d96/daa/de9/f102 x:0 0 0 2024-04-27T04:10:22.087 INFO:tasks.workunit.client.0.smithi162.stdout:8/887: sync 2024-04-27T04:10:22.087 INFO:tasks.workunit.client.0.smithi162.stdout:4/804: sync 2024-04-27T04:10:22.088 INFO:tasks.workunit.client.0.smithi162.stdout:6/869: sync 2024-04-27T04:10:22.088 INFO:tasks.workunit.client.1.smithi181.stdout:6/967: mknod d7/dc/c148 0 2024-04-27T04:10:22.089 INFO:tasks.workunit.client.1.smithi181.stdout:6/968: chown d7/dc/d1c/d88/db3 12 1 2024-04-27T04:10:22.096 INFO:tasks.workunit.client.1.smithi181.stdout:6/969: write d7/dc/d1c/d34/d3f/d7e/d8a/fad [201757,66105] 0 2024-04-27T04:10:22.096 INFO:tasks.workunit.client.0.smithi162.stdout:6/870: read d6/d1a/d34/d3a/dfd/f11a [396873,13540] 0 2024-04-27T04:10:22.097 INFO:tasks.workunit.client.1.smithi181.stdout:6/970: chown d7/dc/d1e/f83 39792 1 2024-04-27T04:10:22.097 INFO:tasks.workunit.client.0.smithi162.stdout:7/882: dwrite d3/d5/d5b/d83/de/f38 [0,4194304] 0 2024-04-27T04:10:22.101 INFO:tasks.workunit.client.1.smithi181.stdout:6/971: creat d7/dc/d1c/d34/d3f/db2/f149 x:0 0 0 2024-04-27T04:10:22.104 INFO:tasks.workunit.client.0.smithi162.stdout:5/913: link d8/d1a/d23/f39 d8/d9/d5b/df3/f11c 0 2024-04-27T04:10:22.104 INFO:tasks.workunit.client.0.smithi162.stdout:6/871: rmdir d6/d1a/d34/d7f 39 2024-04-27T04:10:22.105 INFO:tasks.workunit.client.0.smithi162.stdout:0/956: dwrite d5/dc/d1e/d46/d48/fd3 [0,4194304] 0 2024-04-27T04:10:22.106 INFO:tasks.workunit.client.1.smithi181.stdout:6/972: write d7/d27/fae [972435,50442] 0 2024-04-27T04:10:22.109 INFO:tasks.workunit.client.0.smithi162.stdout:3/837: getdents d1/d18/d23/d34/dad/dc5 0 2024-04-27T04:10:22.112 INFO:tasks.workunit.client.1.smithi181.stdout:6/973: rename d7/dc/dc3/d72/daa/dd3/d147 to d7/dc/d1e/d2b/d137/d14a 0 2024-04-27T04:10:22.112 INFO:tasks.workunit.client.0.smithi162.stdout:0/957: read d5/dc/d1e/d46/d48/d9e/dce/ffd [50750,28571] 0 2024-04-27T04:10:22.115 INFO:tasks.workunit.client.0.smithi162.stdout:8/888: link d0/d10/d25/f88 d0/d10/d2b/d2e/dca/f11f 0 2024-04-27T04:10:22.116 INFO:tasks.workunit.client.0.smithi162.stdout:5/914: symlink d8/d1a/l11d 0 2024-04-27T04:10:22.116 INFO:tasks.workunit.client.1.smithi181.stdout:6/974: mknod d7/dc/d1c/d88/c14b 0 2024-04-27T04:10:22.117 INFO:tasks.workunit.client.0.smithi162.stdout:9/819: dwrite d3/d23/fa2 [0,4194304] 0 2024-04-27T04:10:22.120 INFO:tasks.workunit.client.0.smithi162.stdout:6/872: fsync d6/d1a/d34/d3a/d68/d6c/f53 0 2024-04-27T04:10:22.122 INFO:tasks.workunit.client.0.smithi162.stdout:3/838: link d1/d45/fc0 d1/d18/d23/d34/dad/f12b 0 2024-04-27T04:10:22.123 INFO:tasks.workunit.client.1.smithi181.stdout:6/975: creat d7/dc/f14c x:0 0 0 2024-04-27T04:10:22.126 INFO:tasks.workunit.client.0.smithi162.stdout:7/883: dwrite d3/d5/d5b/d83/de/d2b/d81/daa/dbd/f84 [0,4194304] 0 2024-04-27T04:10:22.127 INFO:tasks.workunit.client.0.smithi162.stdout:1/788: dwrite d0/d7/d2a/f31 [0,4194304] 0 2024-04-27T04:10:22.129 INFO:tasks.workunit.client.0.smithi162.stdout:5/915: symlink d8/d9/d5b/l11e 0 2024-04-27T04:10:22.130 INFO:tasks.workunit.client.0.smithi162.stdout:1/789: fsync d0/f34 0 2024-04-27T04:10:22.133 INFO:tasks.workunit.client.0.smithi162.stdout:3/839: mkdir d1/d18/d23/d34/dad/dc5/d12c 0 2024-04-27T04:10:22.134 INFO:tasks.workunit.client.1.smithi181.stdout:6/976: link d7/dc/d1c/d34/d3f/cda d7/dc/d1e/d66/c14d 0 2024-04-27T04:10:22.136 INFO:tasks.workunit.client.0.smithi162.stdout:6/873: mkdir d6/d1a/d34/d3a/dbf/d120 0 2024-04-27T04:10:22.138 INFO:tasks.workunit.client.0.smithi162.stdout:3/840: fdatasync d1/d18/d5d/d5f/f77 0 2024-04-27T04:10:22.139 INFO:tasks.workunit.client.0.smithi162.stdout:9/820: dread d3/d13/d2d/d3b/f84 [0,4194304] 0 2024-04-27T04:10:22.139 INFO:tasks.workunit.client.0.smithi162.stdout:8/889: creat d0/d10/d2b/d2d/d2f/d39/de1/f120 x:0 0 0 2024-04-27T04:10:22.141 INFO:tasks.workunit.client.0.smithi162.stdout:5/916: mkdir d8/d1a/d23/d40/de4/d11f 0 2024-04-27T04:10:22.141 INFO:tasks.workunit.client.0.smithi162.stdout:1/790: creat d0/d7/d5e/f103 x:0 0 0 2024-04-27T04:10:22.142 INFO:tasks.workunit.client.1.smithi181.stdout:6/977: link d7/dc/d1c/d88/c14b d7/dc/d1c/d88/d6d/c14e 0 2024-04-27T04:10:22.142 INFO:tasks.workunit.client.0.smithi162.stdout:7/884: truncate d3/d5/d5b/d83/de/d2b/d81/daa/dbd/dbe/dff/fc9 154996 0 2024-04-27T04:10:22.144 INFO:tasks.workunit.client.0.smithi162.stdout:3/841: mknod d1/d18/d23/d58/d63/dd5/c12d 0 2024-04-27T04:10:22.144 INFO:tasks.workunit.client.0.smithi162.stdout:1/791: read d0/f2b [2923905,50509] 0 2024-04-27T04:10:22.145 INFO:tasks.workunit.client.1.smithi181.stdout:6/978: chown d7/d31/f75 106666 1 2024-04-27T04:10:22.145 INFO:tasks.workunit.client.0.smithi162.stdout:3/842: chown d1/d18/d5d/d5f/d66/dd2/da3/f110 6 1 2024-04-27T04:10:22.145 INFO:tasks.workunit.client.0.smithi162.stdout:9/821: chown d3/d23/d1e/d43/cef 9455117 1 2024-04-27T04:10:22.149 INFO:tasks.workunit.client.0.smithi162.stdout:5/917: unlink d8/d9/d5b/d96/df4/lf8 0 2024-04-27T04:10:22.150 INFO:tasks.workunit.client.0.smithi162.stdout:6/874: mkdir d6/db/d32/df0/d11d/d121 0 2024-04-27T04:10:22.150 INFO:tasks.workunit.client.0.smithi162.stdout:0/958: dread d5/dc/d1e/d20/dba/df9/fd0 [0,4194304] 0 2024-04-27T04:10:22.151 INFO:tasks.workunit.client.0.smithi162.stdout:6/875: dread - d6/d1a/d34/d7b/fb9 zero size 2024-04-27T04:10:22.152 INFO:tasks.workunit.client.0.smithi162.stdout:1/792: dread - d0/d11/d50/fc3 zero size 2024-04-27T04:10:22.154 INFO:tasks.workunit.client.0.smithi162.stdout:3/843: rmdir d1/d21 39 2024-04-27T04:10:22.154 INFO:tasks.workunit.client.0.smithi162.stdout:9/822: symlink d3/d13/d2d/d4f/lfb 0 2024-04-27T04:10:22.155 INFO:tasks.workunit.client.0.smithi162.stdout:5/918: rename d8/d9/d55/f65 to d8/d9/d55/f120 0 2024-04-27T04:10:22.155 INFO:tasks.workunit.client.0.smithi162.stdout:9/823: stat d3/d11/d45/d4e/d58 0 2024-04-27T04:10:22.159 INFO:tasks.workunit.client.0.smithi162.stdout:5/919: unlink d8/d9/f3d 0 2024-04-27T04:10:22.163 INFO:tasks.workunit.client.0.smithi162.stdout:0/959: creat d5/dc/d1e/d46/d48/f13b x:0 0 0 2024-04-27T04:10:22.165 INFO:tasks.workunit.client.0.smithi162.stdout:9/824: rmdir d3/d13/d2d/d3b/d7a/dd6 39 2024-04-27T04:10:22.166 INFO:tasks.workunit.client.0.smithi162.stdout:5/920: mknod d8/de3/dff/c121 0 2024-04-27T04:10:22.167 INFO:tasks.workunit.client.0.smithi162.stdout:6/876: link d6/d1a/d34/d3a/d68/fb4 d6/d1a/d34/d3a/dbf/d120/f122 0 2024-04-27T04:10:22.168 INFO:tasks.workunit.client.0.smithi162.stdout:8/890: write d0/d10/d25/d106/fc6 [510002,24662] 0 2024-04-27T04:10:22.169 INFO:tasks.workunit.client.1.smithi181.stdout:6/979: truncate d7/f19 3920616 0 2024-04-27T04:10:22.170 INFO:tasks.workunit.client.0.smithi162.stdout:5/921: mkdir d8/d9/d5a/d108/d122 0 2024-04-27T04:10:22.171 INFO:tasks.workunit.client.0.smithi162.stdout:7/885: write d3/d5/d5b/d60/f94 [208199,17649] 0 2024-04-27T04:10:22.172 INFO:tasks.workunit.client.0.smithi162.stdout:9/825: dwrite d3/d39/ff6 [0,4194304] 0 2024-04-27T04:10:22.172 INFO:tasks.workunit.client.1.smithi181.stdout:6/980: mkdir d7/d27/d14f 0 2024-04-27T04:10:22.174 INFO:tasks.workunit.client.1.smithi181.stdout:6/981: write d7/dc/d1c/d34/d107/f117 [555867,65983] 0 2024-04-27T04:10:22.175 INFO:tasks.workunit.client.1.smithi181.stdout:1/948: sync 2024-04-27T04:10:22.177 INFO:tasks.workunit.client.0.smithi162.stdout:5/922: creat d8/de3/deb/d5c/f123 x:0 0 0 2024-04-27T04:10:22.178 INFO:tasks.workunit.client.0.smithi162.stdout:1/793: truncate d0/d11/d3e/d5f/f6a 1432866 0 2024-04-27T04:10:22.182 INFO:tasks.workunit.client.0.smithi162.stdout:7/886: mknod d3/d92/c129 0 2024-04-27T04:10:22.189 INFO:tasks.workunit.client.0.smithi162.stdout:2/856: sync 2024-04-27T04:10:22.190 INFO:tasks.workunit.client.1.smithi181.stdout:6/982: rename d7/dc/d1c/d34/d3f/l5e to d7/dc/d1c/l150 0 2024-04-27T04:10:22.191 INFO:tasks.workunit.client.0.smithi162.stdout:9/826: symlink d3/de1/lfc 0 2024-04-27T04:10:22.192 INFO:tasks.workunit.client.0.smithi162.stdout:9/827: truncate d3/d23/d1e/fe9 733437 0 2024-04-27T04:10:22.192 INFO:tasks.workunit.client.0.smithi162.stdout:7/887: mknod d3/d5/d3c/d4a/d55/dc6/c12a 0 2024-04-27T04:10:22.195 INFO:tasks.workunit.client.1.smithi181.stdout:6/983: dwrite d7/fb9 [4194304,4194304] 0 2024-04-27T04:10:22.196 INFO:tasks.workunit.client.0.smithi162.stdout:6/877: getdents d6/d1a/d34/d50 0 2024-04-27T04:10:22.197 INFO:tasks.workunit.client.1.smithi181.stdout:6/984: stat d7/dc/dc3/d62 0 2024-04-27T04:10:22.198 INFO:tasks.workunit.client.0.smithi162.stdout:2/857: creat d0/d4/d2a/d41/d77/d73/f114 x:0 0 0 2024-04-27T04:10:22.200 INFO:tasks.workunit.client.1.smithi181.stdout:6/985: mknod d7/d31/c151 0 2024-04-27T04:10:22.201 INFO:tasks.workunit.client.0.smithi162.stdout:7/888: dwrite d3/d5/d5b/d83/de/d2b/d81/daa/dbd/dbe/dff/d57/fcb [0,4194304] 0 2024-04-27T04:10:22.203 INFO:tasks.workunit.client.0.smithi162.stdout:8/891: getdents d0/d10/d25/d7d/d87 0 2024-04-27T04:10:22.204 INFO:tasks.workunit.client.0.smithi162.stdout:5/923: creat d8/d1a/d23/dbc/f124 x:0 0 0 2024-04-27T04:10:22.207 INFO:tasks.workunit.client.0.smithi162.stdout:2/858: rmdir d0/d4/d2a/da1/d86 39 2024-04-27T04:10:22.207 INFO:tasks.workunit.client.0.smithi162.stdout:6/878: rename d6/d1a/d34/d7b/fb9 to d6/d1a/d34/d7b/d7e/f123 0 2024-04-27T04:10:22.211 INFO:tasks.workunit.client.0.smithi162.stdout:1/794: getdents d0/d80/d89 0 2024-04-27T04:10:22.212 INFO:tasks.workunit.client.1.smithi181.stdout:6/986: link d7/dc/d1c/d88/c13d d7/dc/dc3/d72/c152 0 2024-04-27T04:10:22.213 INFO:tasks.workunit.client.0.smithi162.stdout:6/879: dwrite d6/db/dc0/d74/fa1 [0,4194304] 0 2024-04-27T04:10:22.214 INFO:tasks.workunit.client.0.smithi162.stdout:5/924: creat d8/d1a/d7d/d87/f125 x:0 0 0 2024-04-27T04:10:22.218 INFO:tasks.workunit.client.0.smithi162.stdout:7/889: mknod d3/d5/d3c/d4a/d55/c12b 0 2024-04-27T04:10:22.219 INFO:tasks.workunit.client.0.smithi162.stdout:1/795: mknod d0/d7/d5e/c104 0 2024-04-27T04:10:22.220 INFO:tasks.workunit.client.0.smithi162.stdout:5/925: read d8/d9/d5b/fb5 [3306517,107767] 0 2024-04-27T04:10:22.220 INFO:tasks.workunit.client.1.smithi181.stdout:6/987: creat d7/dc/d1c/d7f/f153 x:0 0 0 2024-04-27T04:10:22.221 INFO:tasks.workunit.client.0.smithi162.stdout:5/926: readlink d8/de3/deb/l7c 0 2024-04-27T04:10:22.222 INFO:tasks.workunit.client.0.smithi162.stdout:9/828: truncate d3/d39/ff6 1650945 0 2024-04-27T04:10:22.223 INFO:tasks.workunit.client.0.smithi162.stdout:6/880: truncate d6/d1a/d34/d3a/d68/f78 1810345 0 2024-04-27T04:10:22.225 INFO:tasks.workunit.client.0.smithi162.stdout:7/890: dread d3/d5/d5b/d83/de/f38 [0,4194304] 0 2024-04-27T04:10:22.226 INFO:tasks.workunit.client.0.smithi162.stdout:8/892: write d0/d10/d2b/d2d/fc4 [2174897,30980] 0 2024-04-27T04:10:22.228 INFO:tasks.workunit.client.1.smithi181.stdout:6/988: creat d7/dc/dc3/d72/d98/f154 x:0 0 0 2024-04-27T04:10:22.229 INFO:tasks.workunit.client.0.smithi162.stdout:9/829: dwrite d3/d23/fa2 [0,4194304] 0 2024-04-27T04:10:22.230 INFO:tasks.workunit.client.0.smithi162.stdout:6/881: readlink d6/d1a/d34/d7f/d80/l85 0 2024-04-27T04:10:22.230 INFO:tasks.workunit.client.0.smithi162.stdout:7/891: fdatasync d3/d5/d5b/d83/de/f62 0 2024-04-27T04:10:22.232 INFO:tasks.workunit.client.0.smithi162.stdout:7/892: stat d3/d4f/f41 0 2024-04-27T04:10:22.232 INFO:tasks.workunit.client.0.smithi162.stdout:2/859: write d0/d24/f5e [575492,97066] 0 2024-04-27T04:10:22.232 INFO:tasks.workunit.client.1.smithi181.stdout:6/989: creat d7/dc/d1c/d7f/da3/d101/f155 x:0 0 0 2024-04-27T04:10:22.235 INFO:tasks.workunit.client.0.smithi162.stdout:5/927: mknod d8/d1a/c126 0 2024-04-27T04:10:22.235 INFO:tasks.workunit.client.0.smithi162.stdout:2/860: write d0/d4/d17/d88/dc9/fda [4644925,43175] 0 2024-04-27T04:10:22.236 INFO:tasks.workunit.client.0.smithi162.stdout:5/928: dread - d8/d1a/d23/dbc/f75 zero size 2024-04-27T04:10:22.237 INFO:tasks.workunit.client.0.smithi162.stdout:5/929: chown d8/de3/deb/d9f 243 1 2024-04-27T04:10:22.242 INFO:tasks.workunit.client.1.smithi181.stdout:6/990: mknod d7/dc/d1c/d34/d3f/c156 0 2024-04-27T04:10:22.244 INFO:tasks.workunit.client.0.smithi162.stdout:1/796: creat d0/d7/d3c/f105 x:0 0 0 2024-04-27T04:10:22.245 INFO:tasks.workunit.client.0.smithi162.stdout:1/797: dread - d0/d7/d5e/f103 zero size 2024-04-27T04:10:22.246 INFO:tasks.workunit.client.0.smithi162.stdout:7/893: fsync d3/d5/d5b/d83/de/d2b/d81/d96/fd0 0 2024-04-27T04:10:22.248 INFO:tasks.workunit.client.0.smithi162.stdout:2/861: rmdir d0/d4/d17 39 2024-04-27T04:10:22.249 INFO:tasks.workunit.client.1.smithi181.stdout:6/991: dwrite d7/dc/dc3/d72/d98/f154 [0,4194304] 0 2024-04-27T04:10:22.249 INFO:tasks.workunit.client.0.smithi162.stdout:2/862: dread - d0/d4/d2a/d3b/d49/d5a/d63/d84/ff7 zero size 2024-04-27T04:10:22.252 INFO:tasks.workunit.client.0.smithi162.stdout:5/930: chown d8/d9/d55/ca6 4070 1 2024-04-27T04:10:22.253 INFO:tasks.workunit.client.1.smithi181.stdout:6/992: chown d7/dc/d1c/d7f/d3e/c60 238570 1 2024-04-27T04:10:22.254 INFO:tasks.workunit.client.0.smithi162.stdout:9/830: dread d3/d13/d15/d7e/fc0 [0,4194304] 0 2024-04-27T04:10:22.257 INFO:tasks.workunit.client.1.smithi181.stdout:6/993: mkdir d7/dc/dc3/dd1/dd9/d157 0 2024-04-27T04:10:22.263 INFO:tasks.workunit.client.0.smithi162.stdout:9/831: rename d3/d13/d2d/d3b/d7a/lb8 to d3/d13/d2d/d3b/d72/df5/lfd 0 2024-04-27T04:10:22.266 INFO:tasks.workunit.client.0.smithi162.stdout:1/798: link d0/d80/d89/fe2 d0/d7/d3c/d77/dd1/f106 0 2024-04-27T04:10:22.266 INFO:tasks.workunit.client.0.smithi162.stdout:6/882: dread d6/db/dc0/d74/f81 [0,4194304] 0 2024-04-27T04:10:22.267 INFO:tasks.workunit.client.0.smithi162.stdout:7/894: write d3/d5/d5b/d83/de/f30 [2218034,14753] 0 2024-04-27T04:10:22.268 INFO:tasks.workunit.client.0.smithi162.stdout:6/883: write d6/d1a/d34/d7b/ff6 [543973,7441] 0 2024-04-27T04:10:22.269 INFO:tasks.workunit.client.0.smithi162.stdout:8/893: getdents d0/de/d6e/d71/d86/ddc 0 2024-04-27T04:10:22.271 INFO:tasks.workunit.client.0.smithi162.stdout:2/863: symlink d0/d4/d2a/d3b/d49/d5a/d63/l115 0 2024-04-27T04:10:22.271 INFO:tasks.workunit.client.0.smithi162.stdout:8/894: dread d0/d10/d2b/d2d/d2f/d39/fe2 [4194304,4194304] 0 2024-04-27T04:10:22.272 INFO:tasks.workunit.client.0.smithi162.stdout:8/895: readlink d0/d10/l34 0 2024-04-27T04:10:22.272 INFO:tasks.workunit.client.0.smithi162.stdout:2/864: fsync d0/d4/d2a/d3b/d49/d5a/d63/d6e/f98 0 2024-04-27T04:10:22.276 INFO:tasks.workunit.client.0.smithi162.stdout:7/895: read d3/d5/d9/f7f [447859,64841] 0 2024-04-27T04:10:22.277 INFO:tasks.workunit.client.0.smithi162.stdout:5/931: link d8/d1a/d23/dbc/d66/d88/f109 d8/de3/deb/d9f/f127 0 2024-04-27T04:10:22.279 INFO:tasks.workunit.client.0.smithi162.stdout:2/865: dwrite d0/f1 [0,4194304] 0 2024-04-27T04:10:22.279 INFO:tasks.workunit.client.0.smithi162.stdout:7/896: chown d3/d5/d5b/d83/de/d2b/d81/daa/dbd 541786024 1 2024-04-27T04:10:22.284 INFO:tasks.workunit.client.0.smithi162.stdout:1/799: mkdir d0/d11/d44/d66/d107 0 2024-04-27T04:10:22.289 INFO:tasks.workunit.client.0.smithi162.stdout:9/832: dwrite d3/fa [4194304,4194304] 0 2024-04-27T04:10:22.289 INFO:tasks.workunit.client.0.smithi162.stdout:5/932: chown d8/d1a/d23/dbc/d58/c9b 73577012 1 2024-04-27T04:10:22.295 INFO:tasks.workunit.client.0.smithi162.stdout:2/866: dread d0/d4/d2a/d41/f81 [0,4194304] 0 2024-04-27T04:10:22.295 INFO:tasks.workunit.client.0.smithi162.stdout:7/897: creat d3/d5/d5b/d83/de/d2b/d81/daa/dbd/dbe/dff/d57/f12c x:0 0 0 2024-04-27T04:10:22.296 INFO:tasks.workunit.client.0.smithi162.stdout:8/896: dread d0/de/d6e/d71/f95 [0,4194304] 0 2024-04-27T04:10:22.296 INFO:tasks.workunit.client.0.smithi162.stdout:9/833: mknod d3/d39/cfe 0 2024-04-27T04:10:22.297 INFO:tasks.workunit.client.0.smithi162.stdout:7/898: dread - d3/d5/d5b/d83/de/d2b/d81/daa/dbd/dbe/dff/d57/f12c zero size 2024-04-27T04:10:22.298 INFO:tasks.workunit.client.0.smithi162.stdout:1/800: symlink d0/d7/d3c/d77/dd1/dd5/l108 0 2024-04-27T04:10:22.298 INFO:tasks.workunit.client.0.smithi162.stdout:9/834: truncate d3/d23/ff1 870428 0 2024-04-27T04:10:22.300 INFO:tasks.workunit.client.0.smithi162.stdout:2/867: mkdir d0/d24/d116 0 2024-04-27T04:10:22.301 INFO:tasks.workunit.client.0.smithi162.stdout:8/897: fsync d0/de/d6e/d71/fb3 0 2024-04-27T04:10:22.302 INFO:tasks.workunit.client.0.smithi162.stdout:1/801: dread - d0/d7/fcd zero size 2024-04-27T04:10:22.304 INFO:tasks.workunit.client.0.smithi162.stdout:7/899: fsync d3/d5/d5b/d60/fd8 0 2024-04-27T04:10:22.305 INFO:tasks.workunit.client.0.smithi162.stdout:9/835: symlink d3/d13/d2d/d3b/d72/lff 0 2024-04-27T04:10:22.306 INFO:tasks.workunit.client.0.smithi162.stdout:8/898: fsync d0/d10/d25/d65/f8e 0 2024-04-27T04:10:22.309 INFO:tasks.workunit.client.0.smithi162.stdout:1/802: rename d0/d7/cc2 to d0/d7/d3c/d77/dd1/dd5/c109 0 2024-04-27T04:10:22.313 INFO:tasks.workunit.client.0.smithi162.stdout:8/899: dwrite d0/d10/f109 [0,4194304] 0 2024-04-27T04:10:22.316 INFO:tasks.workunit.client.0.smithi162.stdout:9/836: readlink d3/d13/d2d/d3b/d69/lb6 0 2024-04-27T04:10:22.317 INFO:tasks.workunit.client.0.smithi162.stdout:5/933: dread d8/d9/d55/fa5 [0,4194304] 0 2024-04-27T04:10:22.322 INFO:tasks.workunit.client.0.smithi162.stdout:2/868: dread d0/d4/d2a/f9e [4194304,4194304] 0 2024-04-27T04:10:22.324 INFO:tasks.workunit.client.0.smithi162.stdout:2/869: write d0/d4/d2a/f9e [8771680,5218] 0 2024-04-27T04:10:22.324 INFO:tasks.workunit.client.0.smithi162.stdout:5/934: dwrite d8/d9/d5b/d96/fc2 [4194304,4194304] 0 2024-04-27T04:10:22.329 INFO:tasks.workunit.client.0.smithi162.stdout:9/837: fdatasync d3/d13/d15/d7e/fc0 0 2024-04-27T04:10:22.330 INFO:tasks.workunit.client.0.smithi162.stdout:9/838: chown d3/d11/c79 0 1 2024-04-27T04:10:22.339 INFO:tasks.workunit.client.0.smithi162.stdout:5/935: mkdir d8/d1a/d23/d40/de4/d128 0 2024-04-27T04:10:22.344 INFO:tasks.workunit.client.0.smithi162.stdout:7/900: write d3/d5/d5b/d83/de/d2b/d81/d96/fd0 [1477104,30602] 0 2024-04-27T04:10:22.345 INFO:tasks.workunit.client.1.smithi181.stdout:6/994: getdents d7/dc/dc3/d72 0 2024-04-27T04:10:22.346 INFO:tasks.workunit.client.0.smithi162.stdout:5/936: symlink d8/d1a/l129 0 2024-04-27T04:10:22.346 INFO:tasks.workunit.client.0.smithi162.stdout:5/937: chown d8/de3/deb/d9f 397 1 2024-04-27T04:10:22.357 INFO:tasks.workunit.client.0.smithi162.stdout:7/901: write d3/d5/d5b/d83/fb3 [338958,18032] 0 2024-04-27T04:10:22.359 INFO:tasks.workunit.client.0.smithi162.stdout:1/803: dwrite d0/d11/d50/d53/fbe [0,4194304] 0 2024-04-27T04:10:22.361 INFO:tasks.workunit.client.0.smithi162.stdout:5/938: dwrite d8/d1a/d23/d40/f116 [0,4194304] 0 2024-04-27T04:10:22.362 INFO:tasks.workunit.client.0.smithi162.stdout:8/900: dread d0/de/f7b [0,4194304] 0 2024-04-27T04:10:22.366 INFO:tasks.workunit.client.1.smithi181.stdout:9/896: sync 2024-04-27T04:10:22.366 INFO:tasks.workunit.client.1.smithi181.stdout:8/993: sync 2024-04-27T04:10:22.366 INFO:tasks.workunit.client.1.smithi181.stdout:4/999: sync 2024-04-27T04:10:22.366 INFO:tasks.workunit.client.1.smithi181.stdout:8/994: write d1/d6/d2f/d30/de4/f150 [149053,17573] 0 2024-04-27T04:10:22.367 INFO:tasks.workunit.client.0.smithi162.stdout:2/870: dwrite d0/d4/d2a/d3b/d49/d5a/d63/d84/f99 [0,4194304] 0 2024-04-27T04:10:22.367 INFO:tasks.workunit.client.0.smithi162.stdout:9/839: dread d3/d11/d45/d4e/d58/f80 [0,4194304] 0 2024-04-27T04:10:22.367 INFO:tasks.workunit.client.1.smithi181.stdout:9/897: mknod d0/c12f 0 2024-04-27T04:10:22.370 INFO:tasks.workunit.client.1.smithi181.stdout:8/995: creat d1/d8c/d140/f15d x:0 0 0 2024-04-27T04:10:22.371 INFO:tasks.workunit.client.0.smithi162.stdout:5/939: mkdir d8/d1a/d7d/d12a 0 2024-04-27T04:10:22.371 INFO:tasks.workunit.client.0.smithi162.stdout:1/804: stat d0/d7/d3c/d61/dc4 0 2024-04-27T04:10:22.372 INFO:tasks.workunit.client.0.smithi162.stdout:8/901: rmdir d0/de/d20/d6c 39 2024-04-27T04:10:22.373 INFO:tasks.workunit.client.0.smithi162.stdout:1/805: read - d0/d11/d44/d66/fe5 zero size 2024-04-27T04:10:22.373 INFO:tasks.workunit.client.1.smithi181.stdout:9/898: dread d0/d44/d4b/d43/d88/da1/dfc/dbd/df3/fa5 [0,4194304] 0 2024-04-27T04:10:22.374 INFO:tasks.workunit.client.0.smithi162.stdout:9/840: symlink d3/d13/d2d/d30/d67/l100 0 2024-04-27T04:10:22.375 INFO:tasks.workunit.client.1.smithi181.stdout:8/996: creat d1/d6/d2f/d30/d9d/dea/db3/f15e x:0 0 0 2024-04-27T04:10:22.376 INFO:tasks.workunit.client.1.smithi181.stdout:8/997: fdatasync d1/d6/d2f/d30/d5c/f97 0 2024-04-27T04:10:22.377 INFO:tasks.workunit.client.0.smithi162.stdout:9/841: write d3/d23/fa2 [2872271,86744] 0 2024-04-27T04:10:22.377 INFO:tasks.workunit.client.0.smithi162.stdout:2/871: symlink d0/d4/d17/d2b/d64/dbd/l117 0 2024-04-27T04:10:22.378 INFO:tasks.workunit.client.0.smithi162.stdout:8/902: dread - d0/d10/d25/d7d/fed zero size 2024-04-27T04:10:22.378 INFO:tasks.workunit.client.1.smithi181.stdout:8/998: truncate d1/d6/d2f/d30/d9d/dea/db3/f15e 19471 0 2024-04-27T04:10:22.379 INFO:tasks.workunit.client.0.smithi162.stdout:7/902: dwrite d3/f93 [0,4194304] 0 2024-04-27T04:10:22.380 INFO:tasks.workunit.client.0.smithi162.stdout:2/872: dread - d0/d4/d2a/d41/d77/d73/f114 zero size 2024-04-27T04:10:22.382 INFO:tasks.workunit.client.1.smithi181.stdout:8/999: mknod d1/d6/d2f/d30/c15f 0 2024-04-27T04:10:22.383 INFO:tasks.workunit.client.0.smithi162.stdout:9/842: fdatasync d3/d11/d45/d4e/d58/f80 0 2024-04-27T04:10:22.383 INFO:tasks.workunit.client.0.smithi162.stdout:2/873: readlink d0/d4/d17/d2b/d64/dbd/l117 0 2024-04-27T04:10:22.388 INFO:tasks.workunit.client.0.smithi162.stdout:1/806: getdents d0/d11/d3e/d59 0 2024-04-27T04:10:22.388 INFO:tasks.workunit.client.0.smithi162.stdout:9/843: stat d3/d23/d7c 0 2024-04-27T04:10:22.388 INFO:tasks.workunit.client.0.smithi162.stdout:7/903: creat d3/d5/d5b/d83/de/d2b/d81/daa/dbd/dbe/f12d x:0 0 0 2024-04-27T04:10:22.388 INFO:tasks.workunit.client.0.smithi162.stdout:5/940: dread d8/d1a/d7d/fc5 [0,4194304] 0 2024-04-27T04:10:22.389 INFO:tasks.workunit.client.0.smithi162.stdout:8/903: read d0/d10/d2b/d2d/d2f/f54 [1084918,88787] 0 2024-04-27T04:10:22.390 INFO:tasks.workunit.client.0.smithi162.stdout:1/807: rename d0/f1b to d0/d7/d10/d75/f10a 0 2024-04-27T04:10:22.391 INFO:tasks.workunit.client.0.smithi162.stdout:7/904: creat d3/d5/d5b/d83/de/d2b/da7/f12e x:0 0 0 2024-04-27T04:10:22.392 INFO:tasks.workunit.client.0.smithi162.stdout:5/941: creat d8/d9/d5b/d111/d112/f12b x:0 0 0 2024-04-27T04:10:22.393 INFO:tasks.workunit.client.0.smithi162.stdout:9/844: truncate d3/d13/d2d/fce 227517 0 2024-04-27T04:10:22.394 INFO:tasks.workunit.client.0.smithi162.stdout:1/808: mknod d0/d11/d3e/d59/d7f/dc5/c10b 0 2024-04-27T04:10:22.395 INFO:tasks.workunit.client.0.smithi162.stdout:5/942: rmdir d8/d9/d5a/d108 39 2024-04-27T04:10:22.395 INFO:tasks.workunit.client.0.smithi162.stdout:2/874: dwrite d0/d4/d2a/d3b/d49/d5a/d63/d84/fbf [0,4194304] 0 2024-04-27T04:10:22.396 INFO:tasks.workunit.client.1.smithi181.stdout:9/899: truncate d0/d44/d4f/d9d/fd0 1572420 0 2024-04-27T04:10:22.396 INFO:tasks.workunit.client.0.smithi162.stdout:7/905: chown d3/d5/d5b/d83/de/d2b/d81/daa/dbd/f97 0 1 2024-04-27T04:10:22.397 INFO:tasks.workunit.client.0.smithi162.stdout:9/845: creat d3/d23/d1e/dfa/f101 x:0 0 0 2024-04-27T04:10:22.398 INFO:tasks.workunit.client.1.smithi181.stdout:9/900: fsync d0/d44/d4b/d43/d88/da1/dfc/dbd/df3/da0/ddb/fed 0 2024-04-27T04:10:22.399 INFO:tasks.workunit.client.0.smithi162.stdout:1/809: mkdir d0/d7/d3c/d54/dae/d10c 0 2024-04-27T04:10:22.401 INFO:tasks.workunit.client.0.smithi162.stdout:5/943: rename d8/d2c/f4c to d8/d1a/d7d/ddf/f12c 0 2024-04-27T04:10:22.404 INFO:tasks.workunit.client.0.smithi162.stdout:2/875: dwrite d0/f1 [0,4194304] 0 2024-04-27T04:10:22.404 INFO:tasks.workunit.client.1.smithi181.stdout:9/901: link d0/d44/d4b/d43/d88/da1/dfc/lda d0/d44/d4f/df7/d119/l130 0 2024-04-27T04:10:22.405 INFO:tasks.workunit.client.1.smithi181.stdout:9/902: write d0/d44/f6e [1199866,101422] 0 2024-04-27T04:10:22.406 INFO:tasks.workunit.client.0.smithi162.stdout:2/876: write d0/d20/d51/f107 [9137,119948] 0 2024-04-27T04:10:22.406 INFO:tasks.workunit.client.0.smithi162.stdout:9/846: rename d3/d23/c29 to d3/c102 0 2024-04-27T04:10:22.407 INFO:tasks.workunit.client.0.smithi162.stdout:2/877: readlink d0/d20/l9a 0 2024-04-27T04:10:22.407 INFO:tasks.workunit.client.1.smithi181.stdout:9/903: creat d0/d44/d4b/d43/d88/da1/dfc/d9a/f131 x:0 0 0 2024-04-27T04:10:22.409 INFO:tasks.workunit.client.0.smithi162.stdout:3/844: sync 2024-04-27T04:10:22.409 INFO:tasks.workunit.client.0.smithi162.stdout:4/805: sync 2024-04-27T04:10:22.409 INFO:tasks.workunit.client.0.smithi162.stdout:5/944: mknod d8/d1a/d23/dbc/d66/c12d 0 2024-04-27T04:10:22.410 INFO:tasks.workunit.client.0.smithi162.stdout:2/878: mknod d0/d4/d2a/d3b/d49/d5a/d63/d80/c118 0 2024-04-27T04:10:22.411 INFO:tasks.workunit.client.0.smithi162.stdout:5/945: dread - d8/d2c/fd8 zero size 2024-04-27T04:10:22.412 INFO:tasks.workunit.client.0.smithi162.stdout:1/810: dwrite d0/d11/d44/f9a [0,4194304] 0 2024-04-27T04:10:22.413 INFO:tasks.workunit.client.1.smithi181.stdout:9/904: mkdir d0/d44/d4b/d43/d88/da1/dfc/dbd/dd1/dde/d104/d132 0 2024-04-27T04:10:22.415 INFO:tasks.workunit.client.1.smithi181.stdout:9/905: rmdir d0/d44/d4f/df7/d119 39 2024-04-27T04:10:22.416 INFO:tasks.workunit.client.0.smithi162.stdout:3/845: dread d1/d21/fb8 [0,4194304] 0 2024-04-27T04:10:22.417 INFO:tasks.workunit.client.0.smithi162.stdout:1/811: rmdir d0/d7/d3c/d61/dc4 39 2024-04-27T04:10:22.419 INFO:tasks.workunit.client.0.smithi162.stdout:5/946: dread d8/d1a/d29/d8b/fd2 [0,4194304] 0 2024-04-27T04:10:22.420 INFO:tasks.workunit.client.0.smithi162.stdout:3/846: stat d1/d21/d47/d57/d6e/f11d 0 2024-04-27T04:10:22.421 INFO:tasks.workunit.client.1.smithi181.stdout:9/906: creat d0/d44/d4b/d43/d88/da1/dfc/dbd/dd1/dde/f133 x:0 0 0 2024-04-27T04:10:22.421 INFO:tasks.workunit.client.0.smithi162.stdout:4/806: dwrite d2/f4 [0,4194304] 0 2024-04-27T04:10:22.423 INFO:tasks.workunit.client.0.smithi162.stdout:4/807: chown d2/d25/d1c/d39/d3b/d6e/d6f/c9e 3726 1 2024-04-27T04:10:22.424 INFO:tasks.workunit.client.0.smithi162.stdout:1/812: creat d0/d11/d50/d53/d93/f10d x:0 0 0 2024-04-27T04:10:22.433 INFO:tasks.workunit.client.1.smithi181.stdout:9/907: symlink d0/d44/d4b/d43/d88/da1/dfc/dbd/dd1/dde/d104/l134 0 2024-04-27T04:10:22.434 INFO:tasks.workunit.client.0.smithi162.stdout:7/906: write d3/d5/d9/f24 [2472806,82363] 0 2024-04-27T04:10:22.439 INFO:tasks.workunit.client.1.smithi181.stdout:9/908: creat d0/d44/d4b/d43/d88/f135 x:0 0 0 2024-04-27T04:10:22.439 INFO:tasks.workunit.client.0.smithi162.stdout:2/879: link d0/d4/d2a/d3b/d49/d5a/d63/d84/ce3 d0/d4/d2a/d3b/d49/d5a/d63/d80/c119 0 2024-04-27T04:10:22.448 INFO:tasks.workunit.client.0.smithi162.stdout:4/808: dread d2/fb1 [0,4194304] 0 2024-04-27T04:10:22.449 INFO:tasks.workunit.client.0.smithi162.stdout:4/809: fdatasync d2/d25/d1c/d24/d100/de9/ff6 0 2024-04-27T04:10:22.450 INFO:tasks.workunit.client.1.smithi181.stdout:9/909: link d0/c36 d0/d44/d4b/d50/c136 0 2024-04-27T04:10:22.453 INFO:tasks.workunit.client.0.smithi162.stdout:5/947: dwrite d8/d1a/d23/dbc/f90 [0,4194304] 0 2024-04-27T04:10:22.455 INFO:tasks.workunit.client.1.smithi181.stdout:9/910: creat d0/d44/d4b/d43/d88/da1/dfc/dbd/df3/d102/f137 x:0 0 0 2024-04-27T04:10:22.455 INFO:tasks.workunit.client.0.smithi162.stdout:7/907: dread d3/d5/d5b/d83/de/d2b/da7/fc2 [0,4194304] 0 2024-04-27T04:10:22.455 INFO:tasks.workunit.client.0.smithi162.stdout:4/810: dread d2/d25/d1c/d39/f7f [0,4194304] 0 2024-04-27T04:10:22.455 INFO:tasks.workunit.client.0.smithi162.stdout:5/948: dread - d8/d9/fb0 zero size 2024-04-27T04:10:22.456 INFO:tasks.workunit.client.0.smithi162.stdout:5/949: stat d8/d1a/f45 0 2024-04-27T04:10:22.456 INFO:tasks.workunit.client.0.smithi162.stdout:1/813: dwrite d0/d7/d10/f26 [0,4194304] 0 2024-04-27T04:10:22.458 INFO:tasks.workunit.client.1.smithi181.stdout:9/911: rmdir d0/d44/d4b/d43/d88/da1/dfc/dbd/dd1/dde 39 2024-04-27T04:10:22.464 INFO:tasks.workunit.client.1.smithi181.stdout:9/912: chown d0/d44/l5b 14805362 1 2024-04-27T04:10:22.468 INFO:tasks.workunit.client.1.smithi181.stdout:9/913: readlink d0/d44/d4b/d43/d88/da1/dfc/lda 0 2024-04-27T04:10:22.469 INFO:tasks.workunit.client.1.smithi181.stdout:9/914: dread - d0/d44/d4b/d43/d88/f107 zero size 2024-04-27T04:10:22.475 INFO:tasks.workunit.client.1.smithi181.stdout:9/915: rename d0/c36 to d0/d44/d4b/d43/d89/d11d/c138 0 2024-04-27T04:10:22.476 INFO:tasks.workunit.client.1.smithi181.stdout:9/916: truncate d0/f8b 1148084 0 2024-04-27T04:10:22.492 INFO:tasks.workunit.client.1.smithi181.stdout:1/949: sync 2024-04-27T04:10:22.493 INFO:tasks.workunit.client.1.smithi181.stdout:1/950: read d1/f8e [95083,96369] 0 2024-04-27T04:10:22.496 INFO:tasks.workunit.client.1.smithi181.stdout:9/917: write d0/f15 [4708198,22406] 0 2024-04-27T04:10:22.496 INFO:tasks.workunit.client.1.smithi181.stdout:9/918: dread - d0/d44/f9e zero size 2024-04-27T04:10:22.498 INFO:tasks.workunit.client.1.smithi181.stdout:9/919: dread - d0/d44/d4b/d43/d89/d94/f10b zero size 2024-04-27T04:10:22.498 INFO:tasks.workunit.client.1.smithi181.stdout:9/920: stat d0/d91 0 2024-04-27T04:10:22.501 INFO:tasks.workunit.client.1.smithi181.stdout:1/951: link d1/dad/db5/d6/da3/d1e/f79 d1/dad/db5/d6/da3/d1e/d48/dfa/f12f 0 2024-04-27T04:10:22.504 INFO:tasks.workunit.client.1.smithi181.stdout:1/952: truncate d1/dad/db5/f4f 4724749 0 2024-04-27T04:10:22.506 INFO:tasks.workunit.client.0.smithi162.stdout:9/847: chown d3/d13/d2d/d30/f38 5350764 1 2024-04-27T04:10:22.506 INFO:tasks.workunit.client.1.smithi181.stdout:1/953: truncate d1/dad/db5/d6/da3/d1e/d20/d53/d56/dd9/f122 674722 0 2024-04-27T04:10:22.510 INFO:tasks.workunit.client.1.smithi181.stdout:1/954: rename d1/dad/db5/d6/da3/d1e/l54 to d1/dad/db5/d6/dcb/l130 0 2024-04-27T04:10:22.512 INFO:tasks.workunit.client.0.smithi162.stdout:7/908: stat d3/d5/d5b/d83/de/d2b/c10f 0 2024-04-27T04:10:22.512 INFO:tasks.workunit.client.0.smithi162.stdout:7/909: chown d3/d5/d5b/d83/de/d2b/d81/daa/df1 64784103 1 2024-04-27T04:10:22.513 INFO:tasks.workunit.client.0.smithi162.stdout:7/910: fdatasync d3/d5/d9/f24 0 2024-04-27T04:10:22.513 INFO:tasks.workunit.client.0.smithi162.stdout:5/950: unlink d8/d1a/ffb 0 2024-04-27T04:10:22.515 INFO:tasks.workunit.client.0.smithi162.stdout:2/880: truncate d0/d4/d17/f3c 2044186 0 2024-04-27T04:10:22.517 INFO:tasks.workunit.client.0.smithi162.stdout:4/811: mkdir d2/d25/d1c/d106 0 2024-04-27T04:10:22.521 INFO:tasks.workunit.client.0.smithi162.stdout:5/951: rmdir d8/d9/d5b/d111/d112 39 2024-04-27T04:10:22.523 INFO:tasks.workunit.client.0.smithi162.stdout:4/812: dwrite d2/f38 [4194304,4194304] 0 2024-04-27T04:10:22.524 INFO:tasks.workunit.client.0.smithi162.stdout:9/848: creat d3/d39/f103 x:0 0 0 2024-04-27T04:10:22.525 INFO:tasks.workunit.client.0.smithi162.stdout:7/911: creat d3/d5/d5b/d83/de/d2b/df9/d122/f12f x:0 0 0 2024-04-27T04:10:22.526 INFO:tasks.workunit.client.0.smithi162.stdout:5/952: mkdir d8/d9/d5b/d96/d12e 0 2024-04-27T04:10:22.526 INFO:tasks.workunit.client.0.smithi162.stdout:7/912: chown d3/d5/d5b/d83/de/d2b 1 1 2024-04-27T04:10:22.526 INFO:tasks.workunit.client.0.smithi162.stdout:4/813: rmdir d2/d25/d1c/d24/d100/de9 39 2024-04-27T04:10:22.527 INFO:tasks.workunit.client.0.smithi162.stdout:3/847: write d1/d21/d47/d57/dd6/f10b [964270,12738] 0 2024-04-27T04:10:22.528 INFO:tasks.workunit.client.0.smithi162.stdout:4/814: write d2/da/faf [407831,21630] 0 2024-04-27T04:10:22.529 INFO:tasks.workunit.client.0.smithi162.stdout:4/815: chown d2/d25/d1c/d39/d90 86379 1 2024-04-27T04:10:22.533 INFO:tasks.workunit.client.0.smithi162.stdout:2/881: link d0/d20/l59 d0/d4/d2a/d3b/d49/d5a/d63/d80/de4/l11a 0 2024-04-27T04:10:22.535 INFO:tasks.workunit.client.0.smithi162.stdout:3/848: creat d1/d18/d23/d58/d63/d88/f12e x:0 0 0 2024-04-27T04:10:22.537 INFO:tasks.workunit.client.0.smithi162.stdout:7/913: symlink d3/d5/d5b/l130 0 2024-04-27T04:10:22.539 INFO:tasks.workunit.client.0.smithi162.stdout:9/849: unlink d3/d23/d7c/cbf 0 2024-04-27T04:10:22.540 INFO:tasks.workunit.client.0.smithi162.stdout:2/882: creat d0/d4/d17/d2b/d64/d96/daa/f11b x:0 0 0 2024-04-27T04:10:22.545 INFO:tasks.workunit.client.0.smithi162.stdout:7/914: mkdir d3/d5/d5b/d83/de/d2b/d81/daa/dbd/dab/d131 0 2024-04-27T04:10:22.546 INFO:tasks.workunit.client.0.smithi162.stdout:7/915: chown d3/d5/d3c/d4a/fa4 50 1 2024-04-27T04:10:22.550 INFO:tasks.workunit.client.0.smithi162.stdout:7/916: fdatasync d3/d5/d5b/d83/fb7 0 2024-04-27T04:10:22.553 INFO:tasks.workunit.client.0.smithi162.stdout:2/883: rename d0/d4/d2a/d41/d77/d73/fd3 to d0/d4/d2a/d3b/f11c 0 2024-04-27T04:10:22.554 INFO:tasks.workunit.client.0.smithi162.stdout:2/884: chown d0/d24/f5e 4711 1 2024-04-27T04:10:22.555 INFO:tasks.workunit.client.0.smithi162.stdout:2/885: chown d0/d4/d17/d2b/d64/d89/fcc 199005100 1 2024-04-27T04:10:22.555 INFO:tasks.workunit.client.0.smithi162.stdout:7/917: mknod d3/d5/d5b/d83/de/d2b/d81/daa/df1/c132 0 2024-04-27T04:10:22.556 INFO:tasks.workunit.client.1.smithi181.stdout:1/955: truncate d1/dad/db5/d6/da3/d1e/d20/d53/d56/f6f 945115 0 2024-04-27T04:10:22.556 INFO:tasks.workunit.client.0.smithi162.stdout:5/953: write d8/d1a/d23/dbc/d66/d88/f98 [1941438,84986] 0 2024-04-27T04:10:22.557 INFO:tasks.workunit.client.0.smithi162.stdout:2/886: stat d0/d4/d2a/d3b/d49/d5a/d7e/cbe 0 2024-04-27T04:10:22.559 INFO:tasks.workunit.client.0.smithi162.stdout:5/954: creat d8/d1a/d29/f12f x:0 0 0 2024-04-27T04:10:22.560 INFO:tasks.workunit.client.0.smithi162.stdout:5/955: dread - d8/de3/deb/d5c/f123 zero size 2024-04-27T04:10:22.563 INFO:tasks.workunit.client.0.smithi162.stdout:2/887: symlink d0/d4/d2a/d41/d77/d73/dc3/l11d 0 2024-04-27T04:10:22.565 INFO:tasks.workunit.client.0.smithi162.stdout:5/956: read d8/de3/deb/d5c/f62 [450130,5526] 0 2024-04-27T04:10:22.566 INFO:tasks.workunit.client.0.smithi162.stdout:4/816: dread d2/da/f49 [0,4194304] 0 2024-04-27T04:10:22.566 INFO:tasks.workunit.client.0.smithi162.stdout:4/817: chown d2/d8a/l7c 6763553 1 2024-04-27T04:10:22.571 INFO:tasks.workunit.client.0.smithi162.stdout:7/918: rename d3/d5/d5b/d83/de/d2b/d81/d96/dc8/f10a to d3/d5/d5b/d83/de/d2b/d81/daa/dbd/dbe/dff/f133 0 2024-04-27T04:10:22.571 INFO:tasks.workunit.client.0.smithi162.stdout:9/850: write d3/fb [751447,67343] 0 2024-04-27T04:10:22.571 INFO:tasks.workunit.client.0.smithi162.stdout:4/818: symlink d2/l107 0 2024-04-27T04:10:22.571 INFO:tasks.workunit.client.0.smithi162.stdout:7/919: creat d3/d5/d5b/d83/de/d2b/d81/d96/f134 x:0 0 0 2024-04-27T04:10:22.573 INFO:tasks.workunit.client.0.smithi162.stdout:0/960: sync 2024-04-27T04:10:22.573 INFO:tasks.workunit.client.0.smithi162.stdout:6/884: sync 2024-04-27T04:10:22.574 INFO:tasks.workunit.client.1.smithi181.stdout:1/956: write d1/dad/db5/d6/da3/d1e/d48/da6/fe1 [352703,56687] 0 2024-04-27T04:10:22.577 INFO:tasks.workunit.client.0.smithi162.stdout:3/849: dwrite d1/d18/d23/d34/dad/dc5/d109/fed [0,4194304] 0 2024-04-27T04:10:22.578 INFO:tasks.workunit.client.0.smithi162.stdout:2/888: write d0/d4/d2a/d3b/d49/d5a/fe7 [336129,79299] 0 2024-04-27T04:10:22.581 INFO:tasks.workunit.client.0.smithi162.stdout:4/819: dread d2/da/f49 [0,4194304] 0 2024-04-27T04:10:22.584 INFO:tasks.workunit.client.0.smithi162.stdout:6/885: write d6/d1a/d34/d7b/f83 [1503399,91374] 0 2024-04-27T04:10:22.586 INFO:tasks.workunit.client.0.smithi162.stdout:9/851: dread d3/d13/d2d/d3b/f84 [0,4194304] 0 2024-04-27T04:10:22.587 INFO:tasks.workunit.client.0.smithi162.stdout:5/957: dread d8/d1a/f100 [4194304,4194304] 0 2024-04-27T04:10:22.589 INFO:tasks.workunit.client.0.smithi162.stdout:0/961: write d5/dc/d1e/d46/d48/d9e/db8/fb9 [623230,186] 0 2024-04-27T04:10:22.590 INFO:tasks.workunit.client.0.smithi162.stdout:5/958: readlink d8/d9/d5b/l101 0 2024-04-27T04:10:22.591 INFO:tasks.workunit.client.0.smithi162.stdout:3/850: write d1/d18/d23/f28 [3224552,38862] 0 2024-04-27T04:10:22.592 INFO:tasks.workunit.client.0.smithi162.stdout:4/820: unlink d2/d25/d1c/d39/d90/fbb 0 2024-04-27T04:10:22.593 INFO:tasks.workunit.client.0.smithi162.stdout:6/886: read d6/d1a/f47 [809173,66674] 0 2024-04-27T04:10:22.593 INFO:tasks.workunit.client.1.smithi181.stdout:1/957: dwrite f0 [0,4194304] 0 2024-04-27T04:10:22.595 INFO:tasks.workunit.client.1.smithi181.stdout:1/958: chown d1/dad/db5/d6/da3/d1e/d48 58024383 1 2024-04-27T04:10:22.596 INFO:tasks.workunit.client.1.smithi181.stdout:1/959: chown d1/dad/db5/d6/db/d2d/ffd 101452 1 2024-04-27T04:10:22.597 INFO:tasks.workunit.client.0.smithi162.stdout:5/959: mknod d8/d1a/d7d/c130 0 2024-04-27T04:10:22.597 INFO:tasks.workunit.client.0.smithi162.stdout:2/889: dwrite d0/d4/d17/d2b/f95 [4194304,4194304] 0 2024-04-27T04:10:22.597 INFO:tasks.workunit.client.0.smithi162.stdout:7/920: dwrite d3/d5/d9/fde [0,4194304] 0 2024-04-27T04:10:22.597 INFO:tasks.workunit.client.0.smithi162.stdout:6/887: write d6/f25 [1411031,78571] 0 2024-04-27T04:10:22.598 INFO:tasks.workunit.client.0.smithi162.stdout:9/852: creat d3/d13/d2d/d3b/d54/f104 x:0 0 0 2024-04-27T04:10:22.600 INFO:tasks.workunit.client.0.smithi162.stdout:7/921: readlink d3/d5/lc5 0 2024-04-27T04:10:22.600 INFO:tasks.workunit.client.0.smithi162.stdout:0/962: mkdir d5/d26/da2/d13c 0 2024-04-27T04:10:22.601 INFO:tasks.workunit.client.0.smithi162.stdout:2/890: write d0/fe6 [80493,27798] 0 2024-04-27T04:10:22.601 INFO:tasks.workunit.client.0.smithi162.stdout:9/853: chown d3/d13/d2d/d3b/d69 6422932 1 2024-04-27T04:10:22.605 INFO:tasks.workunit.client.0.smithi162.stdout:3/851: mknod d1/d18/d117/d12a/c12f 0 2024-04-27T04:10:22.605 INFO:tasks.workunit.client.1.smithi181.stdout:1/960: dwrite d1/dad/db5/d6/d6a/fee [0,4194304] 0 2024-04-27T04:10:22.607 INFO:tasks.workunit.client.0.smithi162.stdout:5/960: chown d8/d1a/d23/dbc/d58/fe9 702400759 1 2024-04-27T04:10:22.607 INFO:tasks.workunit.client.0.smithi162.stdout:2/891: write d0/d4/d2a/d41/d77/d73/f114 [811218,15355] 0 2024-04-27T04:10:22.608 INFO:tasks.workunit.client.0.smithi162.stdout:4/821: symlink d2/d25/d1c/d39/d3b/d47/dfb/l108 0 2024-04-27T04:10:22.609 INFO:tasks.workunit.client.0.smithi162.stdout:2/892: chown d0/d4/d17/d88 394 1 2024-04-27T04:10:22.609 INFO:tasks.workunit.client.0.smithi162.stdout:7/922: fsync d3/d5/d5b/d83/de/fbc 0 2024-04-27T04:10:22.611 INFO:tasks.workunit.client.1.smithi181.stdout:1/961: creat d1/dad/db5/d6/da3/d1e/d48/da6/da4/f131 x:0 0 0 2024-04-27T04:10:22.612 INFO:tasks.workunit.client.0.smithi162.stdout:6/888: getdents d6/db/d32/df0/d11d/d121 0 2024-04-27T04:10:22.615 INFO:tasks.workunit.client.0.smithi162.stdout:3/852: mkdir d1/d18/d5d/d6f/d130 0 2024-04-27T04:10:22.616 INFO:tasks.workunit.client.0.smithi162.stdout:0/963: rename d5/dc/d1e/d20/cf7 to d5/d26/d4a/def/c13d 0 2024-04-27T04:10:22.616 INFO:tasks.workunit.client.0.smithi162.stdout:8/904: sync 2024-04-27T04:10:22.616 INFO:tasks.workunit.client.0.smithi162.stdout:4/822: rmdir d2 39 2024-04-27T04:10:22.617 INFO:tasks.workunit.client.0.smithi162.stdout:5/961: fsync d8/d9/d5b/fc8 0 2024-04-27T04:10:22.618 INFO:tasks.workunit.client.1.smithi181.stdout:1/962: mknod d1/dad/db5/d6/da3/d1e/d48/da6/da4/de5/c132 0 2024-04-27T04:10:22.620 INFO:tasks.workunit.client.0.smithi162.stdout:2/893: mkdir d0/d20/d11e 0 2024-04-27T04:10:22.621 INFO:tasks.workunit.client.0.smithi162.stdout:7/923: chown d3/d5/d3c/f7e 382159572 1 2024-04-27T04:10:22.621 INFO:tasks.workunit.client.0.smithi162.stdout:5/962: dread d8/d1a/d7d/fc5 [0,4194304] 0 2024-04-27T04:10:22.622 INFO:tasks.workunit.client.0.smithi162.stdout:9/854: creat d3/d13/d2d/d3b/f105 x:0 0 0 2024-04-27T04:10:22.623 INFO:tasks.workunit.client.0.smithi162.stdout:3/853: unlink d1/d45/l50 0 2024-04-27T04:10:22.623 INFO:tasks.workunit.client.1.smithi181.stdout:1/963: mkdir d1/d2/d133 0 2024-04-27T04:10:22.624 INFO:tasks.workunit.client.0.smithi162.stdout:0/964: dwrite d5/dc/d1e/d46/d48/fd3 [0,4194304] 0 2024-04-27T04:10:22.624 INFO:tasks.workunit.client.1.smithi181.stdout:1/964: stat d1/d2/l21 0 2024-04-27T04:10:22.628 INFO:tasks.workunit.client.1.smithi181.stdout:1/965: truncate d1/dad/db5/d6/da3/d1e/fc5 2982666 0 2024-04-27T04:10:22.633 INFO:tasks.workunit.client.0.smithi162.stdout:9/855: dwrite d3/d23/d1e/d43/df9/fe8 [0,4194304] 0 2024-04-27T04:10:22.635 INFO:tasks.workunit.client.1.smithi181.stdout:1/966: unlink d1/dad/db5/d6/dcb/cf8 0 2024-04-27T04:10:22.636 INFO:tasks.workunit.client.0.smithi162.stdout:4/823: read d2/d25/d1c/d39/d5e/f85 [125340,85243] 0 2024-04-27T04:10:22.638 INFO:tasks.workunit.client.0.smithi162.stdout:8/905: write d0/d10/d2b/d2d/d2f/f54 [1071759,91579] 0 2024-04-27T04:10:22.639 INFO:tasks.workunit.client.0.smithi162.stdout:0/965: truncate d5/dc/d1e/d46/d48/d9e/dce/dd1/d116/d57/fda 265271 0 2024-04-27T04:10:22.642 INFO:tasks.workunit.client.0.smithi162.stdout:6/889: rename d6/d1a/d34/d50/d11f to d6/d124 0 2024-04-27T04:10:22.644 INFO:tasks.workunit.client.0.smithi162.stdout:8/906: mkdir d0/d10/d2b/d2d/d2f/d39/de1/d121 0 2024-04-27T04:10:22.644 INFO:tasks.workunit.client.0.smithi162.stdout:0/966: fsync d5/d26/d69/f73 0 2024-04-27T04:10:22.645 INFO:tasks.workunit.client.0.smithi162.stdout:4/824: symlink d2/d25/d1c/d39/d70/l109 0 2024-04-27T04:10:22.647 INFO:tasks.workunit.client.0.smithi162.stdout:3/854: write d1/d18/d23/d58/d63/f86 [2119703,103723] 0 2024-04-27T04:10:22.649 INFO:tasks.workunit.client.0.smithi162.stdout:7/924: rename d3/d5/d5b/d83/de/d2b/df9 to d3/d5/d5b/d83/de/d2b/d81/d96/d135 0 2024-04-27T04:10:22.649 INFO:tasks.workunit.client.0.smithi162.stdout:2/894: getdents d0/d4/d2a/d3b 0 2024-04-27T04:10:22.650 INFO:tasks.workunit.client.0.smithi162.stdout:2/895: readlink d0/d24/l62 0 2024-04-27T04:10:22.652 INFO:tasks.workunit.client.0.smithi162.stdout:2/896: stat d0/d4/d17/d4f/db0/ld5 0 2024-04-27T04:10:22.654 INFO:tasks.workunit.client.0.smithi162.stdout:8/907: dread d0/de/d20/d6c/fcb [0,4194304] 0 2024-04-27T04:10:22.654 INFO:tasks.workunit.client.0.smithi162.stdout:2/897: read d0/d4/d2a/d3b/f11c [78647,16553] 0 2024-04-27T04:10:22.654 INFO:tasks.workunit.client.0.smithi162.stdout:0/967: chown d5/dc/d1e/d41/d79/l114 57444 1 2024-04-27T04:10:22.654 INFO:tasks.workunit.client.0.smithi162.stdout:4/825: mknod d2/d25/d1c/d39/d3b/da7/c10a 0 2024-04-27T04:10:22.656 INFO:tasks.workunit.client.1.smithi181.stdout:6/995: sync 2024-04-27T04:10:22.657 INFO:tasks.workunit.client.0.smithi162.stdout:9/856: unlink d3/d11/d45/d4e/c89 0 2024-04-27T04:10:22.658 INFO:tasks.workunit.client.0.smithi162.stdout:5/963: dwrite d8/fad [0,4194304] 0 2024-04-27T04:10:22.658 INFO:tasks.workunit.client.0.smithi162.stdout:9/857: readlink d3/d13/d2d/l3a 0 2024-04-27T04:10:22.659 INFO:tasks.workunit.client.0.smithi162.stdout:5/964: stat d8/d9/db3/lea 0 2024-04-27T04:10:22.662 INFO:tasks.workunit.client.0.smithi162.stdout:3/855: write d1/d18/d5d/ff3 [3483132,42807] 0 2024-04-27T04:10:22.663 INFO:tasks.workunit.client.1.smithi181.stdout:6/996: creat d7/f158 x:0 0 0 2024-04-27T04:10:22.667 INFO:tasks.workunit.client.0.smithi162.stdout:2/898: dwrite d0/d4/d17/d2b/d64/f90 [4194304,4194304] 0 2024-04-27T04:10:22.667 INFO:tasks.workunit.client.0.smithi162.stdout:4/826: truncate d2/d25/d1c/d24/fcf 3813846 0 2024-04-27T04:10:22.668 INFO:tasks.workunit.client.1.smithi181.stdout:6/997: truncate d7/dc/dc3/d72/daa/fce 3162928 0 2024-04-27T04:10:22.670 INFO:tasks.workunit.client.0.smithi162.stdout:9/858: read d3/fb [3284829,29788] 0 2024-04-27T04:10:22.671 INFO:tasks.workunit.client.1.smithi181.stdout:6/998: unlink d7/dc/f12 0 2024-04-27T04:10:22.673 INFO:tasks.workunit.client.0.smithi162.stdout:3/856: symlink d1/d45/l131 0 2024-04-27T04:10:22.675 INFO:tasks.workunit.client.0.smithi162.stdout:7/925: creat d3/d5/d5b/d83/de/d2b/d81/daa/dbd/dbe/dff/f136 x:0 0 0 2024-04-27T04:10:22.676 INFO:tasks.workunit.client.0.smithi162.stdout:6/890: dwrite d6/d1a/d34/d3a/d68/d6c/f10f [0,4194304] 0 2024-04-27T04:10:22.677 INFO:tasks.workunit.client.1.smithi181.stdout:6/999: dread d7/f59 [0,4194304] 0 2024-04-27T04:10:22.679 INFO:tasks.workunit.client.0.smithi162.stdout:4/827: creat d2/d54/f10b x:0 0 0 2024-04-27T04:10:22.680 INFO:tasks.workunit.client.0.smithi162.stdout:1/814: sync 2024-04-27T04:10:22.683 INFO:tasks.workunit.client.0.smithi162.stdout:1/815: dread - d0/d11/d50/fc3 zero size 2024-04-27T04:10:22.684 INFO:tasks.workunit.client.0.smithi162.stdout:2/899: mknod d0/d4/d2a/d3b/d49/d5a/d63/d70/c11f 0 2024-04-27T04:10:22.685 INFO:tasks.workunit.client.0.smithi162.stdout:9/859: unlink d3/d13/d15/l1d 0 2024-04-27T04:10:22.686 INFO:tasks.workunit.client.0.smithi162.stdout:7/926: dread d3/d5/d5b/d83/de/d2b/da7/fc2 [0,4194304] 0 2024-04-27T04:10:22.688 INFO:tasks.workunit.client.0.smithi162.stdout:3/857: rename d1/d18/d23/d34 to d1/d21/d47/d57/dd6/d112/d132 0 2024-04-27T04:10:22.692 INFO:tasks.workunit.client.0.smithi162.stdout:8/908: dread d0/d10/d2b/fbc [0,4194304] 0 2024-04-27T04:10:22.694 INFO:tasks.workunit.client.0.smithi162.stdout:2/900: creat d0/d4/d17/d2b/d64/d89/dbc/f120 x:0 0 0 2024-04-27T04:10:22.694 INFO:tasks.workunit.client.0.smithi162.stdout:1/816: rename d0/d11/d50/d53/d101 to d0/d7/d5e/d10e 0 2024-04-27T04:10:22.695 INFO:tasks.workunit.client.0.smithi162.stdout:0/968: dread d5/dc/d1e/f21 [0,4194304] 0 2024-04-27T04:10:22.696 INFO:tasks.workunit.client.0.smithi162.stdout:7/927: mkdir d3/d5/d5b/d83/de/d2b/d81/d96/dc8/d11a/d137 0 2024-04-27T04:10:22.697 INFO:tasks.workunit.client.0.smithi162.stdout:6/891: creat d6/db/d76/f125 x:0 0 0 2024-04-27T04:10:22.698 INFO:tasks.workunit.client.0.smithi162.stdout:9/860: dread d3/d23/d1e/d43/df9/fe8 [0,4194304] 0 2024-04-27T04:10:22.698 INFO:tasks.workunit.client.0.smithi162.stdout:1/817: rmdir d0 39 2024-04-27T04:10:22.702 INFO:tasks.workunit.client.0.smithi162.stdout:7/928: write d3/d5/d3c/d4d/fcf [100086,50683] 0 2024-04-27T04:10:22.703 INFO:tasks.workunit.client.0.smithi162.stdout:5/965: dwrite d8/d1a/d23/dbc/f52 [4194304,4194304] 0 2024-04-27T04:10:22.703 INFO:tasks.workunit.client.0.smithi162.stdout:6/892: fsync d6/d1a/d34/d3a/d68/d6c/f10f 0 2024-04-27T04:10:22.715 INFO:tasks.workunit.client.0.smithi162.stdout:5/966: truncate d8/f20 8682358 0 2024-04-27T04:10:22.716 INFO:tasks.workunit.client.0.smithi162.stdout:4/828: dwrite d2/d25/d1c/d39/d3b/f94 [0,4194304] 0 2024-04-27T04:10:22.716 INFO:tasks.workunit.client.0.smithi162.stdout:6/893: creat d6/d1a/d34/d7f/f126 x:0 0 0 2024-04-27T04:10:22.720 INFO:tasks.workunit.client.0.smithi162.stdout:7/929: rename d3/d5/d5b/d83/de/d2b/d81/d96/dc8/d128 to d3/d5/d5b/d83/de/d2b/da7/d138 0 2024-04-27T04:10:22.722 INFO:tasks.workunit.client.0.smithi162.stdout:8/909: write d0/d10/d2b/d2e/dca/f11f [1154723,78578] 0 2024-04-27T04:10:22.726 INFO:tasks.workunit.client.0.smithi162.stdout:9/861: write d3/d13/d2d/d3b/d69/f7d [1570503,86925] 0 2024-04-27T04:10:22.726 INFO:tasks.workunit.client.0.smithi162.stdout:3/858: dwrite d1/d18/d5d/d6f/f80 [0,4194304] 0 2024-04-27T04:10:22.728 INFO:tasks.workunit.client.0.smithi162.stdout:1/818: write d0/d7/d3c/d6e/f71 [2149826,58604] 0 2024-04-27T04:10:22.730 INFO:tasks.workunit.client.0.smithi162.stdout:2/901: dwrite d0/d4/d2a/d41/d77/f83 [0,4194304] 0 2024-04-27T04:10:22.731 INFO:tasks.workunit.client.0.smithi162.stdout:7/930: write d3/d5/d5b/d83/de/d2b/d81/d96/d135/ffa [688095,101734] 0 2024-04-27T04:10:22.732 INFO:tasks.workunit.client.0.smithi162.stdout:5/967: creat d8/d9/d5b/d114/f131 x:0 0 0 2024-04-27T04:10:22.735 INFO:tasks.workunit.client.0.smithi162.stdout:3/859: chown d1/d21/c59 68721052 1 2024-04-27T04:10:22.735 INFO:tasks.workunit.client.0.smithi162.stdout:7/931: readlink d3/d5/l7b 0 2024-04-27T04:10:22.737 INFO:tasks.workunit.client.0.smithi162.stdout:6/894: link d6/d1a/d34/d43/c6f d6/d1a/d34/d7f/d80/db3/c127 0 2024-04-27T04:10:22.739 INFO:tasks.workunit.client.0.smithi162.stdout:0/969: dwrite d5/dc/d1e/ffb [0,4194304] 0 2024-04-27T04:10:22.739 INFO:tasks.workunit.client.0.smithi162.stdout:8/910: truncate d0/d10/d76/fb7 795818 0 2024-04-27T04:10:22.741 INFO:tasks.workunit.client.0.smithi162.stdout:3/860: write d1/d21/d47/d57/ffd [163950,62433] 0 2024-04-27T04:10:22.741 INFO:tasks.workunit.client.0.smithi162.stdout:9/862: fsync d3/d11/d45/d4e/d58/f68 0 2024-04-27T04:10:22.744 INFO:tasks.workunit.client.0.smithi162.stdout:7/932: mkdir d3/d5/d5b/d83/de/d2b/d81/d139 0 2024-04-27T04:10:22.744 INFO:tasks.workunit.client.0.smithi162.stdout:8/911: write d0/d10/f110 [963597,103067] 0 2024-04-27T04:10:22.747 INFO:tasks.workunit.client.0.smithi162.stdout:2/902: truncate d0/d4/d17/d2b/d64/d89/dbc/f120 397804 0 2024-04-27T04:10:22.749 INFO:tasks.workunit.client.0.smithi162.stdout:3/861: write d1/d18/d23/d58/d63/d88/fe8 [1167309,99753] 0 2024-04-27T04:10:22.750 INFO:tasks.workunit.client.1.smithi181.stdout:9/921: sync 2024-04-27T04:10:22.751 INFO:tasks.workunit.client.0.smithi162.stdout:4/829: dwrite d2/d25/d1c/d24/fcf [0,4194304] 0 2024-04-27T04:10:22.751 INFO:tasks.workunit.client.0.smithi162.stdout:9/863: fsync d3/d13/d2d/d44/fa3 0 2024-04-27T04:10:22.751 INFO:tasks.workunit.client.0.smithi162.stdout:7/933: symlink d3/d5/d5b/d83/de/d2b/d81/d117/l13a 0 2024-04-27T04:10:22.753 INFO:tasks.workunit.client.0.smithi162.stdout:2/903: mkdir d0/d4/d17/d121 0 2024-04-27T04:10:22.753 INFO:tasks.workunit.client.0.smithi162.stdout:7/934: chown d3/d5/d5b/d83/de/f74 95 1 2024-04-27T04:10:22.754 INFO:tasks.workunit.client.1.smithi181.stdout:9/922: unlink d0/d44/d4f/cdd 0 2024-04-27T04:10:22.756 INFO:tasks.workunit.client.0.smithi162.stdout:2/904: chown d0/d4/d2a/f3e 12 1 2024-04-27T04:10:22.758 INFO:tasks.workunit.client.0.smithi162.stdout:4/830: creat d2/d25/d1c/d39/d70/dcd/f10c x:0 0 0 2024-04-27T04:10:22.760 INFO:tasks.workunit.client.0.smithi162.stdout:3/862: getdents d1 0 2024-04-27T04:10:22.764 INFO:tasks.workunit.client.0.smithi162.stdout:2/905: dwrite d0/d4/d17/d2b/d64/d96/daa/f11b [0,4194304] 0 2024-04-27T04:10:22.765 INFO:tasks.workunit.client.0.smithi162.stdout:4/831: dread d2/d25/f8c [0,4194304] 0 2024-04-27T04:10:22.766 INFO:tasks.workunit.client.0.smithi162.stdout:4/832: stat d2/d25/d1c/d39/d5e/l6b 0 2024-04-27T04:10:22.766 INFO:tasks.workunit.client.0.smithi162.stdout:2/906: symlink d0/d4/d2a/d41/l122 0 2024-04-27T04:10:22.769 INFO:tasks.workunit.client.0.smithi162.stdout:4/833: symlink d2/d25/d1c/d39/d3b/d6e/l10d 0 2024-04-27T04:10:22.770 INFO:tasks.workunit.client.0.smithi162.stdout:2/907: fsync d0/d4/d17/f3c 0 2024-04-27T04:10:22.771 INFO:tasks.workunit.client.0.smithi162.stdout:4/834: rmdir d2/d25/d1c/d24/db5/dc0 39 2024-04-27T04:10:22.772 INFO:tasks.workunit.client.0.smithi162.stdout:1/819: dread d0/d80/f87 [0,4194304] 0 2024-04-27T04:10:22.778 INFO:tasks.workunit.client.0.smithi162.stdout:4/835: dwrite d2/d25/d1c/d39/d3b/da7/fcb [0,4194304] 0 2024-04-27T04:10:22.778 INFO:tasks.workunit.client.0.smithi162.stdout:0/970: dread d5/dc/d1e/d46/d48/d9e/fdb [0,4194304] 0 2024-04-27T04:10:22.781 INFO:tasks.workunit.client.0.smithi162.stdout:4/836: mknod d2/d25/d1c/d39/d70/dcd/df0/c10e 0 2024-04-27T04:10:22.782 INFO:tasks.workunit.client.0.smithi162.stdout:0/971: truncate d5/dc/d1e/d20/dba/df9/d8b/fe1 1054048 0 2024-04-27T04:10:22.782 INFO:tasks.workunit.client.0.smithi162.stdout:4/837: dread - d2/d25/d1c/d39/d90/dad/dfd/f104 zero size 2024-04-27T04:10:22.783 INFO:tasks.workunit.client.0.smithi162.stdout:1/820: dread d0/d7/d10/f26 [4194304,4194304] 0 2024-04-27T04:10:22.785 INFO:tasks.workunit.client.0.smithi162.stdout:4/838: unlink d2/d25/d1c/d39/d3b/d6e/d6f/c102 0 2024-04-27T04:10:22.798 INFO:tasks.workunit.client.0.smithi162.stdout:7/935: read d3/d5/d5b/d60/fa5 [817517,43365] 0 2024-04-27T04:10:22.798 INFO:tasks.workunit.client.0.smithi162.stdout:0/972: dread d5/dc/d1e/d20/dba/df9/d98/fad [0,4194304] 0 2024-04-27T04:10:22.799 INFO:tasks.workunit.client.0.smithi162.stdout:7/936: chown d3/d5/d3c/d4a/d55/c12b 3150 1 2024-04-27T04:10:22.807 INFO:tasks.workunit.client.0.smithi162.stdout:7/937: creat d3/d5/d5b/d83/de/d2b/d81/daa/dbd/f13b x:0 0 0 2024-04-27T04:10:22.808 INFO:tasks.workunit.client.1.smithi181.stdout:1/967: sync 2024-04-27T04:10:22.809 INFO:tasks.workunit.client.1.smithi181.stdout:1/968: stat d1/dad/db5/d6/da3/d1e/d20/f82 0 2024-04-27T04:10:22.812 INFO:tasks.workunit.client.0.smithi162.stdout:8/912: dwrite d0/d10/d2b/f19 [0,4194304] 0 2024-04-27T04:10:22.814 INFO:tasks.workunit.client.0.smithi162.stdout:9/864: dwrite d3/d13/d2d/d44/fa3 [0,4194304] 0 2024-04-27T04:10:22.816 INFO:tasks.workunit.client.0.smithi162.stdout:7/938: read d3/d5/d3c/f7e [604194,37486] 0 2024-04-27T04:10:22.819 INFO:tasks.workunit.client.0.smithi162.stdout:9/865: mkdir d3/d13/d2d/d30/d67/d106 0 2024-04-27T04:10:22.821 INFO:tasks.workunit.client.0.smithi162.stdout:1/821: dread d0/f1e [4194304,4194304] 0 2024-04-27T04:10:22.823 INFO:tasks.workunit.client.0.smithi162.stdout:9/866: mkdir d3/d13/d2d/d3b/d107 0 2024-04-27T04:10:22.825 INFO:tasks.workunit.client.0.smithi162.stdout:1/822: unlink d0/d11/d50/l52 0 2024-04-27T04:10:22.827 INFO:tasks.workunit.client.0.smithi162.stdout:8/913: rename d0/d10/d2b/d2d/d2f/d39/ce5 to d0/d10/d25/d65/c122 0 2024-04-27T04:10:22.830 INFO:tasks.workunit.client.0.smithi162.stdout:9/867: mknod d3/d23/d1e/d43/c108 0 2024-04-27T04:10:22.831 INFO:tasks.workunit.client.1.smithi181.stdout:9/923: dwrite d0/d44/d4b/d43/f56 [0,4194304] 0 2024-04-27T04:10:22.831 INFO:tasks.workunit.client.0.smithi162.stdout:9/868: mknod d3/d23/d1e/d43/c109 0 2024-04-27T04:10:22.832 INFO:tasks.workunit.client.1.smithi181.stdout:9/924: fdatasync d0/d44/d4b/d43/d88/da1/dfc/f9f 0 2024-04-27T04:10:22.832 INFO:tasks.workunit.client.0.smithi162.stdout:9/869: write d3/d23/d1e/f4a [1839329,51685] 0 2024-04-27T04:10:22.839 INFO:tasks.workunit.client.1.smithi181.stdout:9/925: dwrite d0/d44/d4f/d9d/fe7 [0,4194304] 0 2024-04-27T04:10:22.844 INFO:tasks.workunit.client.0.smithi162.stdout:2/908: write d0/d4/d2a/da1/fa5 [757252,81173] 0 2024-04-27T04:10:22.847 INFO:tasks.workunit.client.1.smithi181.stdout:1/969: write d1/dad/db5/d6/f4e [139998,49323] 0 2024-04-27T04:10:22.848 INFO:tasks.workunit.client.0.smithi162.stdout:4/839: write d2/d25/d1c/d39/d5e/fb2 [910920,113129] 0 2024-04-27T04:10:22.849 INFO:tasks.workunit.client.1.smithi181.stdout:1/970: chown d1/dad/db5/d6/da3/d1e/d20/d53/d56/de2 130165885 1 2024-04-27T04:10:22.853 INFO:tasks.workunit.client.0.smithi162.stdout:3/863: dwrite d1/d45/fc0 [4194304,4194304] 0 2024-04-27T04:10:22.854 INFO:tasks.workunit.client.0.smithi162.stdout:4/840: fdatasync d2/d25/d1c/d39/d3b/d6e/d6f/f76 0 2024-04-27T04:10:22.855 INFO:tasks.workunit.client.1.smithi181.stdout:1/971: mknod d1/dad/db5/d6/c134 0 2024-04-27T04:10:22.857 INFO:tasks.workunit.client.0.smithi162.stdout:9/870: write d3/d13/d2d/d3b/d54/f8d [341471,89213] 0 2024-04-27T04:10:22.859 INFO:tasks.workunit.client.0.smithi162.stdout:7/939: dwrite d3/d5/d3c/f75 [0,4194304] 0 2024-04-27T04:10:22.863 INFO:tasks.workunit.client.0.smithi162.stdout:9/871: write d3/d13/d2d/d30/d7f/fe7 [144705,107831] 0 2024-04-27T04:10:22.864 INFO:tasks.workunit.client.0.smithi162.stdout:9/872: write d3/d13/d2d/feb [909190,28433] 0 2024-04-27T04:10:22.865 INFO:tasks.workunit.client.1.smithi181.stdout:1/972: dwrite d1/dad/db5/d6/da3/d1e/f49 [4194304,4194304] 0 2024-04-27T04:10:22.867 INFO:tasks.workunit.client.0.smithi162.stdout:4/841: creat d2/f10f x:0 0 0 2024-04-27T04:10:22.869 INFO:tasks.workunit.client.0.smithi162.stdout:1/823: getdents d0/d11/d50/d53/d93/d96/daa 0 2024-04-27T04:10:22.873 INFO:tasks.workunit.client.1.smithi181.stdout:1/973: fsync d1/dad/db5/d6/da3/d1e/d20/d53/d56/de2/f116 0 2024-04-27T04:10:22.875 INFO:tasks.workunit.client.1.smithi181.stdout:9/926: dwrite d0/d44/d4f/d9d/fb4 [4194304,4194304] 0 2024-04-27T04:10:22.875 INFO:tasks.workunit.client.0.smithi162.stdout:9/873: dwrite d3/f4 [4194304,4194304] 0 2024-04-27T04:10:22.878 INFO:tasks.workunit.client.1.smithi181.stdout:9/927: stat d0/d44/cd7 0 2024-04-27T04:10:22.879 INFO:tasks.workunit.client.1.smithi181.stdout:1/974: chown d1/dad/db5/d6/da3/d52/f68 1637 1 2024-04-27T04:10:22.881 INFO:tasks.workunit.client.0.smithi162.stdout:7/940: creat d3/d5/d5b/d83/de/d2b/d81/daa/dbd/dbe/dff/f13c x:0 0 0 2024-04-27T04:10:22.881 INFO:tasks.workunit.client.1.smithi181.stdout:9/928: read d0/d44/d4f/f6b [576974,62879] 0 2024-04-27T04:10:22.885 INFO:tasks.workunit.client.1.smithi181.stdout:1/975: fdatasync d1/dad/db5/d6/db/d2d/ffd 0 2024-04-27T04:10:22.887 INFO:tasks.workunit.client.1.smithi181.stdout:1/976: chown d1/dad/db5/d6/da3/d60/d126 1540600 1 2024-04-27T04:10:22.887 INFO:tasks.workunit.client.0.smithi162.stdout:1/824: fsync d0/d11/f2e 0 2024-04-27T04:10:22.889 INFO:tasks.workunit.client.0.smithi162.stdout:9/874: mknod d3/d13/d2d/d30/c10a 0 2024-04-27T04:10:22.890 INFO:tasks.workunit.client.1.smithi181.stdout:9/929: mknod d0/d44/d4b/d43/d8f/c139 0 2024-04-27T04:10:22.891 INFO:tasks.workunit.client.0.smithi162.stdout:3/864: dwrite d1/d18/f10 [4194304,4194304] 0 2024-04-27T04:10:22.891 INFO:tasks.workunit.client.0.smithi162.stdout:7/941: dread d3/d5/d5b/d83/de/d2b/d81/daa/dbd/dbe/dff/f77 [0,4194304] 0 2024-04-27T04:10:22.891 INFO:tasks.workunit.client.0.smithi162.stdout:1/825: dread - d0/d11/d50/fc3 zero size 2024-04-27T04:10:22.893 INFO:tasks.workunit.client.0.smithi162.stdout:3/865: write d1/d21/d47/d57/ffd [709866,107040] 0 2024-04-27T04:10:22.893 INFO:tasks.workunit.client.0.smithi162.stdout:9/875: symlink d3/d13/d2d/d30/d7f/dc3/l10b 0 2024-04-27T04:10:22.896 INFO:tasks.workunit.client.0.smithi162.stdout:7/942: read d3/d5/d5b/d83/f2e [2730416,29032] 0 2024-04-27T04:10:22.897 INFO:tasks.workunit.client.0.smithi162.stdout:3/866: symlink d1/d21/d47/d57/dd6/l133 0 2024-04-27T04:10:22.897 INFO:tasks.workunit.client.0.smithi162.stdout:1/826: stat d0/d7/d3c/d54/cbf 0 2024-04-27T04:10:22.898 INFO:tasks.workunit.client.1.smithi181.stdout:9/930: dwrite d0/d44/d4b/d43/d88/de3/f12a [0,4194304] 0 2024-04-27T04:10:22.901 INFO:tasks.workunit.client.0.smithi162.stdout:1/827: write d0/d7/d10/de4/fe6 [8493377,16481] 0 2024-04-27T04:10:22.902 INFO:tasks.workunit.client.0.smithi162.stdout:7/943: rmdir d3/d5/d5b/d83/de/d2b/d81/d96/d135/d122 39 2024-04-27T04:10:22.911 INFO:tasks.workunit.client.0.smithi162.stdout:9/876: dwrite d3/d11/d45/d4e/d58/f68 [0,4194304] 0 2024-04-27T04:10:22.912 INFO:tasks.workunit.client.0.smithi162.stdout:8/914: dwrite d0/d10/d76/fb7 [0,4194304] 0 2024-04-27T04:10:22.913 INFO:tasks.workunit.client.0.smithi162.stdout:1/828: creat d0/d7/d3c/d61/dc4/f10f x:0 0 0 2024-04-27T04:10:22.914 INFO:tasks.workunit.client.0.smithi162.stdout:3/867: dread d1/d18/d5d/d6f/f80 [0,4194304] 0 2024-04-27T04:10:22.916 INFO:tasks.workunit.client.0.smithi162.stdout:3/868: stat d1/d21/d47/d57 0 2024-04-27T04:10:22.918 INFO:tasks.workunit.client.0.smithi162.stdout:4/842: dwrite d2/d25/d1c/d39/d70/f8d [0,4194304] 0 2024-04-27T04:10:22.919 INFO:tasks.workunit.client.1.smithi181.stdout:1/977: write d1/dad/db5/d6/f73 [4330254,64759] 0 2024-04-27T04:10:22.921 INFO:tasks.workunit.client.1.smithi181.stdout:1/978: dread - d1/dad/db5/d6/da3/d1e/d48/da6/da4/ff5 zero size 2024-04-27T04:10:22.926 INFO:tasks.workunit.client.0.smithi162.stdout:9/877: creat d3/de1/f10c x:0 0 0 2024-04-27T04:10:22.928 INFO:tasks.workunit.client.1.smithi181.stdout:9/931: write d0/f2b [4656166,54310] 0 2024-04-27T04:10:22.929 INFO:tasks.workunit.client.0.smithi162.stdout:1/829: dwrite d0/d11/d44/f9a [0,4194304] 0 2024-04-27T04:10:22.932 INFO:tasks.workunit.client.0.smithi162.stdout:3/869: dwrite d1/d18/d23/d2f/da0/ff4 [0,4194304] 0 2024-04-27T04:10:22.940 INFO:tasks.workunit.client.0.smithi162.stdout:3/870: rmdir d1/d21/d47/d57/dd6/d112/d132/dad/dc5 39 2024-04-27T04:10:22.940 INFO:tasks.workunit.client.0.smithi162.stdout:4/843: link d2/d25/d1c/d39/d90/dad/f9a d2/d25/d1c/d106/f110 0 2024-04-27T04:10:22.944 INFO:tasks.workunit.client.1.smithi181.stdout:9/932: getdents d0/d44/d4b/d43/d89 0 2024-04-27T04:10:22.944 INFO:tasks.workunit.client.0.smithi162.stdout:9/878: dread d3/d13/d2d/d30/fa4 [0,4194304] 0 2024-04-27T04:10:22.945 INFO:tasks.workunit.client.0.smithi162.stdout:1/830: creat d0/d11/d50/f110 x:0 0 0 2024-04-27T04:10:22.946 INFO:tasks.workunit.client.0.smithi162.stdout:4/844: dread d2/d25/d1c/d39/d70/f8d [0,4194304] 0 2024-04-27T04:10:22.946 INFO:tasks.workunit.client.0.smithi162.stdout:7/944: dread d3/d5/d5b/d83/f19 [0,4194304] 0 2024-04-27T04:10:22.946 INFO:tasks.workunit.client.0.smithi162.stdout:3/871: rename d1/d21/d47/cd0 to d1/d21/d47/d57/dd6/d112/d132/daa/de6/c134 0 2024-04-27T04:10:22.947 INFO:tasks.workunit.client.0.smithi162.stdout:7/945: chown d3/d5/d11c 30587 1 2024-04-27T04:10:22.947 INFO:tasks.workunit.client.0.smithi162.stdout:4/845: dread - d2/d25/d1c/d39/d3b/da7/fda zero size 2024-04-27T04:10:22.948 INFO:tasks.workunit.client.1.smithi181.stdout:9/933: mknod d0/d44/d4b/d43/d8f/d111/c13a 0 2024-04-27T04:10:22.951 INFO:tasks.workunit.client.1.smithi181.stdout:9/934: rmdir d0/d44/d4b/d43/d88/da1/dfc 39 2024-04-27T04:10:22.952 INFO:tasks.workunit.client.1.smithi181.stdout:9/935: chown d0/d44/d4b/d43 79 1 2024-04-27T04:10:22.952 INFO:tasks.workunit.client.0.smithi162.stdout:9/879: symlink d3/d13/d15/d7e/dd0/l10d 0 2024-04-27T04:10:22.953 INFO:tasks.workunit.client.0.smithi162.stdout:7/946: mkdir d3/d5/d5b/d83/de/d2b/d81/daa/dbd/dbe/dff/d57/d13d 0 2024-04-27T04:10:22.954 INFO:tasks.workunit.client.0.smithi162.stdout:8/915: truncate d0/de/d20/d6c/f6d 2513668 0 2024-04-27T04:10:22.955 INFO:tasks.workunit.client.1.smithi181.stdout:9/936: symlink d0/d44/d4f/l13b 0 2024-04-27T04:10:22.956 INFO:tasks.workunit.client.1.smithi181.stdout:1/979: truncate d1/dad/db5/d6/da3/d1e/d20/f5c 3140126 0 2024-04-27T04:10:22.956 INFO:tasks.workunit.client.0.smithi162.stdout:4/846: dwrite d2/d25/d1c/d24/db5/dc0/ff9 [0,4194304] 0 2024-04-27T04:10:22.958 INFO:tasks.workunit.client.1.smithi181.stdout:1/980: write d1/dad/db5/d6/da3/d1e/d20/d53/d56/de2/f116 [583295,127340] 0 2024-04-27T04:10:22.960 INFO:tasks.workunit.client.1.smithi181.stdout:9/937: creat d0/d44/d4b/d43/d88/de3/f13c x:0 0 0 2024-04-27T04:10:22.961 INFO:tasks.workunit.client.0.smithi162.stdout:4/847: dread d2/d25/f8 [0,4194304] 0 2024-04-27T04:10:22.964 INFO:tasks.workunit.client.0.smithi162.stdout:9/880: link d3/d11/fca d3/d13/d15/d50/f10e 0 2024-04-27T04:10:22.965 INFO:tasks.workunit.client.0.smithi162.stdout:9/881: stat d3/d23/d1e/d43 0 2024-04-27T04:10:22.966 INFO:tasks.workunit.client.0.smithi162.stdout:9/882: chown d3/d13/d2d/d44/l9b 25 1 2024-04-27T04:10:22.969 INFO:tasks.workunit.client.0.smithi162.stdout:3/872: write d1/d18/f52 [2737057,127129] 0 2024-04-27T04:10:22.969 INFO:tasks.workunit.client.1.smithi181.stdout:1/981: read d1/dad/db5/d6/da3/d52/d92/f93 [4650275,5452] 0 2024-04-27T04:10:22.971 INFO:tasks.workunit.client.0.smithi162.stdout:1/831: dread d0/d11/d3e/d5f/f6a [0,4194304] 0 2024-04-27T04:10:22.972 INFO:tasks.workunit.client.0.smithi162.stdout:4/848: mknod d2/d25/d1c/d39/d3b/d6e/d6f/c111 0 2024-04-27T04:10:22.973 INFO:tasks.workunit.client.0.smithi162.stdout:9/883: mknod d3/d13/d2d/d44/c10f 0 2024-04-27T04:10:22.979 INFO:tasks.workunit.client.0.smithi162.stdout:4/849: dwrite d2/da/dd/fe1 [0,4194304] 0 2024-04-27T04:10:22.981 INFO:tasks.workunit.client.0.smithi162.stdout:4/850: fdatasync d2/d25/d1c/d39/f5f 0 2024-04-27T04:10:22.982 INFO:tasks.workunit.client.0.smithi162.stdout:1/832: write d0/d7/f32 [3529046,123198] 0 2024-04-27T04:10:22.984 INFO:tasks.workunit.client.1.smithi181.stdout:9/938: dread d0/d44/d4b/f62 [0,4194304] 0 2024-04-27T04:10:22.985 INFO:tasks.workunit.client.0.smithi162.stdout:1/833: readlink d0/d80/d89/lb4 0 2024-04-27T04:10:22.985 INFO:tasks.workunit.client.0.smithi162.stdout:7/947: dread d3/d5/d5b/d60/f6f [0,4194304] 0 2024-04-27T04:10:22.990 INFO:tasks.workunit.client.1.smithi181.stdout:9/939: creat d0/d44/d4b/d43/d10c/f13d x:0 0 0 2024-04-27T04:10:22.993 INFO:tasks.workunit.client.0.smithi162.stdout:4/851: mknod d2/d25/d1c/d24/db5/dc0/c112 0 2024-04-27T04:10:22.998 INFO:tasks.workunit.client.0.smithi162.stdout:3/873: write d1/d70/f8c [5281548,11988] 0 2024-04-27T04:10:23.000 INFO:tasks.workunit.client.1.smithi181.stdout:1/982: dwrite d1/dad/db5/d6/db/f91 [0,4194304] 0 2024-04-27T04:10:23.001 INFO:tasks.workunit.client.0.smithi162.stdout:1/834: dwrite d0/d11/d50/d53/d93/d96/fef [0,4194304] 0 2024-04-27T04:10:23.001 INFO:tasks.workunit.client.1.smithi181.stdout:9/940: mknod d0/d44/d4b/d43/d88/da1/dfc/dbd/df3/d85/dd8/dfa/c13e 0 2024-04-27T04:10:23.005 INFO:tasks.workunit.client.1.smithi181.stdout:1/983: unlink d1/dad/db5/d6/da3/d1e/dbc/lea 0 2024-04-27T04:10:23.011 INFO:tasks.workunit.client.1.smithi181.stdout:1/984: chown d1/dad/db5/d6/l9 56918817 1 2024-04-27T04:10:23.014 INFO:tasks.workunit.client.1.smithi181.stdout:9/941: mknod d0/d44/d4b/d43/d88/da1/dfc/dbd/dd1/dde/d104/d132/c13f 0 2024-04-27T04:10:23.029 INFO:tasks.workunit.client.1.smithi181.stdout:1/985: dwrite d1/d2/d89/fef [0,4194304] 0 2024-04-27T04:10:23.029 INFO:tasks.workunit.client.1.smithi181.stdout:9/942: dwrite d0/d44/d4f/fae [0,4194304] 0 2024-04-27T04:10:23.033 INFO:tasks.workunit.client.1.smithi181.stdout:9/943: chown d0/d44/d4b/d43/d89/d94/d11c/c126 58 1 2024-04-27T04:10:23.035 INFO:tasks.workunit.client.1.smithi181.stdout:1/986: mknod d1/dad/db5/d6/da3/d1e/d48/da6/c135 0 2024-04-27T04:10:23.037 INFO:tasks.workunit.client.1.smithi181.stdout:1/987: stat d1/dad/db5/d6/da3/d1e/l46 0 2024-04-27T04:10:23.037 INFO:tasks.workunit.client.1.smithi181.stdout:9/944: symlink d0/d44/d4b/d43/d88/de3/l140 0 2024-04-27T04:10:23.041 INFO:tasks.workunit.client.1.smithi181.stdout:1/988: mknod d1/dad/db5/d6/da3/d1e/d20/da2/c136 0 2024-04-27T04:10:23.042 INFO:tasks.workunit.client.1.smithi181.stdout:1/989: write d1/dad/db5/d6/f73 [4384993,85546] 0 2024-04-27T04:10:23.047 INFO:tasks.workunit.client.1.smithi181.stdout:1/990: truncate d1/dad/db5/d6/da3/d1e/fc5 2573706 0 2024-04-27T04:10:23.049 INFO:tasks.workunit.client.1.smithi181.stdout:1/991: mknod d1/d2/c137 0 2024-04-27T04:10:23.055 INFO:tasks.workunit.client.0.smithi162.stdout:3/874: rmdir d1/d18/d5d/d6f 39 2024-04-27T04:10:23.056 INFO:tasks.workunit.client.0.smithi162.stdout:4/852: fsync d2/d25/d1c/d24/d100/f2e 0 2024-04-27T04:10:23.056 INFO:tasks.workunit.client.0.smithi162.stdout:9/884: dwrite d3/d13/d2d/d3b/d72/f94 [0,4194304] 0 2024-04-27T04:10:23.059 INFO:tasks.workunit.client.1.smithi181.stdout:9/945: dread d0/d44/d4b/d43/d89/d94/fd6 [0,4194304] 0 2024-04-27T04:10:23.062 INFO:tasks.workunit.client.0.smithi162.stdout:1/835: mknod d0/d11/d44/d66/c111 0 2024-04-27T04:10:23.064 INFO:tasks.workunit.client.0.smithi162.stdout:4/853: mknod d2/d25/c113 0 2024-04-27T04:10:23.064 INFO:tasks.workunit.client.0.smithi162.stdout:7/948: dwrite d3/d5/f5e [0,4194304] 0 2024-04-27T04:10:23.067 INFO:tasks.workunit.client.0.smithi162.stdout:1/836: readlink d0/d7/l91 0 2024-04-27T04:10:23.068 INFO:tasks.workunit.client.1.smithi181.stdout:1/992: dwrite d1/dad/db5/d6/da3/d1e/d48/dfa/f12f [0,4194304] 0 2024-04-27T04:10:23.068 INFO:tasks.workunit.client.0.smithi162.stdout:4/854: chown d2/da/dd/l7e 25995301 1 2024-04-27T04:10:23.068 INFO:tasks.workunit.client.0.smithi162.stdout:3/875: dwrite d1/d70/f8c [0,4194304] 0 2024-04-27T04:10:23.073 INFO:tasks.workunit.client.1.smithi181.stdout:9/946: write d0/d44/f51 [1553074,61808] 0 2024-04-27T04:10:23.073 INFO:tasks.workunit.client.0.smithi162.stdout:4/855: dread d2/d25/fd7 [0,4194304] 0 2024-04-27T04:10:23.075 INFO:tasks.workunit.client.1.smithi181.stdout:9/947: stat d0/d44/d4b/d43/d88/da1/dfc/dbd/lee 0 2024-04-27T04:10:23.076 INFO:tasks.workunit.client.0.smithi162.stdout:7/949: creat d3/d5/d5b/d83/de/d2b/d81/daa/dbd/dab/f13e x:0 0 0 2024-04-27T04:10:23.078 INFO:tasks.workunit.client.0.smithi162.stdout:7/950: chown d3/d5/d5b/d83/de/d2b/da7/fe3 494 1 2024-04-27T04:10:23.080 INFO:tasks.workunit.client.0.smithi162.stdout:1/837: rmdir d0/d11/d50/d53 39 2024-04-27T04:10:23.082 INFO:tasks.workunit.client.1.smithi181.stdout:9/948: mkdir d0/d44/d4f/d141 0 2024-04-27T04:10:23.084 INFO:tasks.workunit.client.1.smithi181.stdout:9/949: chown d0/d44/d4b/d5f/f10a 554 1 2024-04-27T04:10:23.085 INFO:tasks.workunit.client.0.smithi162.stdout:9/885: dread d3/d23/f3e [0,4194304] 0 2024-04-27T04:10:23.085 INFO:tasks.workunit.client.1.smithi181.stdout:1/993: getdents d1/dad/db5/d6/d6a 0 2024-04-27T04:10:23.092 INFO:tasks.workunit.client.1.smithi181.stdout:1/994: dwrite d1/dad/db5/d6/da3/d1e/f8f [0,4194304] 0 2024-04-27T04:10:23.093 INFO:tasks.workunit.client.0.smithi162.stdout:4/856: mkdir d2/da/d114 0 2024-04-27T04:10:23.093 INFO:tasks.workunit.client.0.smithi162.stdout:4/857: chown d2/d25/d1c/d39/d70/l109 229198 1 2024-04-27T04:10:23.095 INFO:tasks.workunit.client.0.smithi162.stdout:3/876: link d1/c9b d1/d21/d47/d57/dd6/d112/d132/daa/c135 0 2024-04-27T04:10:23.096 INFO:tasks.workunit.client.0.smithi162.stdout:9/886: creat d3/d13/d2d/d30/d67/d106/f110 x:0 0 0 2024-04-27T04:10:23.097 INFO:tasks.workunit.client.0.smithi162.stdout:4/858: mkdir d2/d25/d115 0 2024-04-27T04:10:23.098 INFO:tasks.workunit.client.1.smithi181.stdout:1/995: creat d1/f138 x:0 0 0 2024-04-27T04:10:23.101 INFO:tasks.workunit.client.1.smithi181.stdout:1/996: creat d1/dad/db5/d6/db/df2/f139 x:0 0 0 2024-04-27T04:10:23.102 INFO:tasks.workunit.client.0.smithi162.stdout:9/887: creat d3/d13/d2d/d44/f111 x:0 0 0 2024-04-27T04:10:23.103 INFO:tasks.workunit.client.0.smithi162.stdout:1/838: link d0/d7/d3c/d77/dd1/cf5 d0/d7/d3c/d54/dc9/c112 0 2024-04-27T04:10:23.104 INFO:tasks.workunit.client.0.smithi162.stdout:9/888: read - d3/d13/d2d/d30/d67/d106/f110 zero size 2024-04-27T04:10:23.105 INFO:tasks.workunit.client.0.smithi162.stdout:4/859: link d2/d25/d1c/d39/d3b/da7/fcb d2/d54/dca/f116 0 2024-04-27T04:10:23.108 INFO:tasks.workunit.client.0.smithi162.stdout:1/839: mknod d0/d80/dba/c113 0 2024-04-27T04:10:23.109 INFO:tasks.workunit.client.0.smithi162.stdout:3/877: creat d1/d18/f136 x:0 0 0 2024-04-27T04:10:23.110 INFO:tasks.workunit.client.0.smithi162.stdout:3/878: chown d1/d21/d47/d57/f64 124258 1 2024-04-27T04:10:23.112 INFO:tasks.workunit.client.0.smithi162.stdout:9/889: write d3/d23/f3e [2127019,64750] 0 2024-04-27T04:10:23.112 INFO:tasks.workunit.client.1.smithi181.stdout:9/950: dwrite d0/d44/d4b/d43/d88/da1/dfc/dbd/df3/da0/ddb/f10e [0,4194304] 0 2024-04-27T04:10:23.114 INFO:tasks.workunit.client.1.smithi181.stdout:9/951: chown d0/d44/d4b/d43/d88/da1/dfc/d9a/cc7 6 1 2024-04-27T04:10:23.114 INFO:tasks.workunit.client.1.smithi181.stdout:1/997: getdents d1/dad/db5 0 2024-04-27T04:10:23.114 INFO:tasks.workunit.client.0.smithi162.stdout:1/840: creat d0/d80/f114 x:0 0 0 2024-04-27T04:10:23.117 INFO:tasks.workunit.client.0.smithi162.stdout:7/951: dwrite d3/d5/d5b/d83/de/d2b/d81/daa/dbd/db6/fd5 [0,4194304] 0 2024-04-27T04:10:23.117 INFO:tasks.workunit.client.1.smithi181.stdout:1/998: chown d1/dad/db5/d6/db/l15 101833757 1 2024-04-27T04:10:23.118 INFO:tasks.workunit.client.0.smithi162.stdout:3/879: mknod d1/d21/d47/d57/dd6/d112/d132/daa/de6/c137 0 2024-04-27T04:10:23.118 INFO:tasks.workunit.client.0.smithi162.stdout:4/860: creat d2/d25/d1c/d24/dd2/df7/f117 x:0 0 0 2024-04-27T04:10:23.119 INFO:tasks.workunit.client.0.smithi162.stdout:4/861: fdatasync d2/d25/d1c/d39/d3b/d47/fc3 0 2024-04-27T04:10:23.121 INFO:tasks.workunit.client.1.smithi181.stdout:1/999: read d1/dad/db5/d6/da3/d1e/d20/d53/d56/dd9/fdd [1094996,96465] 0 2024-04-27T04:10:23.125 INFO:tasks.workunit.client.0.smithi162.stdout:1/841: chown d0/d7/d3c/d54/db0 439340 1 2024-04-27T04:10:23.127 INFO:tasks.workunit.client.1.smithi181.stdout:9/952: creat d0/f142 x:0 0 0 2024-04-27T04:10:23.128 INFO:tasks.workunit.client.0.smithi162.stdout:3/880: mkdir d1/d18/d23/d58/d63/dfa/d138 0 2024-04-27T04:10:23.133 INFO:tasks.workunit.client.0.smithi162.stdout:9/890: mkdir d3/d39/d112 0 2024-04-27T04:10:23.133 INFO:tasks.workunit.client.1.smithi181.stdout:9/953: dwrite d0/f15 [0,4194304] 0 2024-04-27T04:10:23.136 INFO:tasks.workunit.client.0.smithi162.stdout:3/881: write d1/d18/d5d/d5f/d66/f68 [615801,34572] 0 2024-04-27T04:10:23.137 INFO:tasks.workunit.client.0.smithi162.stdout:4/862: creat d2/d25/d1c/d106/f118 x:0 0 0 2024-04-27T04:10:23.138 INFO:tasks.workunit.client.0.smithi162.stdout:1/842: mkdir d0/d11/d44/d66/d107/d115 0 2024-04-27T04:10:23.139 INFO:tasks.workunit.client.1.smithi181.stdout:9/954: dread d0/d44/d4b/d43/d88/da1/dfc/dbd/df3/da0/ddb/f10e [0,4194304] 0 2024-04-27T04:10:23.140 INFO:tasks.workunit.client.0.smithi162.stdout:7/952: creat d3/f13f x:0 0 0 2024-04-27T04:10:23.141 INFO:tasks.workunit.client.0.smithi162.stdout:4/863: mkdir d2/d25/d1c/d39/d3b/d119 0 2024-04-27T04:10:23.143 INFO:tasks.workunit.client.0.smithi162.stdout:9/891: dwrite d3/d23/d1e/f4a [0,4194304] 0 2024-04-27T04:10:23.143 INFO:tasks.workunit.client.0.smithi162.stdout:0/973: sync 2024-04-27T04:10:23.143 INFO:tasks.workunit.client.0.smithi162.stdout:6/895: sync 2024-04-27T04:10:23.143 INFO:tasks.workunit.client.0.smithi162.stdout:4/864: stat d2/d25/d115 0 2024-04-27T04:10:23.143 INFO:tasks.workunit.client.0.smithi162.stdout:5/968: sync 2024-04-27T04:10:23.147 INFO:tasks.workunit.client.0.smithi162.stdout:0/974: readlink d5/dc/d1e/d41/d79/l115 0 2024-04-27T04:10:23.147 INFO:tasks.workunit.client.0.smithi162.stdout:6/896: dread - d6/db/d32/fd5 zero size 2024-04-27T04:10:23.147 INFO:tasks.workunit.client.0.smithi162.stdout:7/953: chown d3/d5/d5b/d83/de/d2b/d81/f82 65094801 1 2024-04-27T04:10:23.149 INFO:tasks.workunit.client.0.smithi162.stdout:4/865: unlink d2/d54/d7a/f83 0 2024-04-27T04:10:23.149 INFO:tasks.workunit.client.0.smithi162.stdout:9/892: truncate d3/d13/d2d/d30/d7f/ff7 345301 0 2024-04-27T04:10:23.151 INFO:tasks.workunit.client.0.smithi162.stdout:3/882: dwrite d1/d21/d47/d57/dd6/d112/d132/daa/f123 [0,4194304] 0 2024-04-27T04:10:23.151 INFO:tasks.workunit.client.0.smithi162.stdout:5/969: stat d8/d9/d5b/d111/d112/f11b 0 2024-04-27T04:10:23.153 INFO:tasks.workunit.client.0.smithi162.stdout:5/970: chown d8/d1a/d23/dbc/f124 5574231 1 2024-04-27T04:10:23.153 INFO:tasks.workunit.client.0.smithi162.stdout:1/843: link d0/d7/d10/de4/fe6 d0/d7/d3c/d77/dd1/dd5/f116 0 2024-04-27T04:10:23.154 INFO:tasks.workunit.client.0.smithi162.stdout:5/971: stat d8/d1a/d23/dbc/d58/fe9 0 2024-04-27T04:10:23.157 INFO:tasks.workunit.client.0.smithi162.stdout:6/897: unlink d6/d1a/l28 0 2024-04-27T04:10:23.160 INFO:tasks.workunit.client.0.smithi162.stdout:6/898: truncate d6/d1a/d34/d3a/d68/d6c/d67/f11b 838505 0 2024-04-27T04:10:23.160 INFO:tasks.workunit.client.0.smithi162.stdout:7/954: symlink d3/d5/d5b/d83/de/d2b/da7/d138/l140 0 2024-04-27T04:10:23.161 INFO:tasks.workunit.client.0.smithi162.stdout:9/893: mkdir d3/d13/d2d/d30/d7f/dc3/d113 0 2024-04-27T04:10:23.163 INFO:tasks.workunit.client.1.smithi181.stdout:9/955: dwrite d0/d91/ff6 [0,4194304] 0 2024-04-27T04:10:23.163 INFO:tasks.workunit.client.0.smithi162.stdout:4/866: creat d2/d25/d1c/d39/d3b/d47/f11a x:0 0 0 2024-04-27T04:10:23.165 INFO:tasks.workunit.client.0.smithi162.stdout:3/883: mknod d1/d18/d4f/c139 0 2024-04-27T04:10:23.167 INFO:tasks.workunit.client.1.smithi181.stdout:9/956: write d0/d44/d4b/f62 [4066145,809] 0 2024-04-27T04:10:23.168 INFO:tasks.workunit.client.0.smithi162.stdout:1/844: read - d0/d11/d50/d53/d93/d96/daa/fda zero size 2024-04-27T04:10:23.170 INFO:tasks.workunit.client.0.smithi162.stdout:7/955: dwrite d3/d5/d5b/d83/de/d2b/d81/d96/dc8/f10e [0,4194304] 0 2024-04-27T04:10:23.175 INFO:tasks.workunit.client.0.smithi162.stdout:1/845: rmdir d0/d7/d3c/d54/dae 39 2024-04-27T04:10:23.175 INFO:tasks.workunit.client.0.smithi162.stdout:4/867: rmdir d2/d54 39 2024-04-27T04:10:23.175 INFO:tasks.workunit.client.0.smithi162.stdout:6/899: fdatasync d6/d1a/d34/d3a/dbf/d120/f122 0 2024-04-27T04:10:23.177 INFO:tasks.workunit.client.0.smithi162.stdout:3/884: creat d1/d18/d23/d58/d63/dfa/d138/f13a x:0 0 0 2024-04-27T04:10:23.183 INFO:tasks.workunit.client.0.smithi162.stdout:1/846: stat d0/d11/d50/ca0 0 2024-04-27T04:10:23.183 INFO:tasks.workunit.client.0.smithi162.stdout:4/868: symlink d2/d8a/l11b 0 2024-04-27T04:10:23.183 INFO:tasks.workunit.client.0.smithi162.stdout:6/900: mknod d6/d1a/d34/d7f/dd8/c128 0 2024-04-27T04:10:23.184 INFO:tasks.workunit.client.0.smithi162.stdout:3/885: symlink d1/d21/d47/d57/dd6/l13b 0 2024-04-27T04:10:23.185 INFO:tasks.workunit.client.0.smithi162.stdout:3/886: readlink d1/d21/d47/d57/dd6/l133 0 2024-04-27T04:10:23.186 INFO:tasks.workunit.client.0.smithi162.stdout:0/975: dwrite d5/dc/d1e/d46/d48/d9e/dce/dd1/d116/f90 [0,4194304] 0 2024-04-27T04:10:23.187 INFO:tasks.workunit.client.0.smithi162.stdout:9/894: write d3/d13/d2d/d3b/d54/f8f [2600098,78904] 0 2024-04-27T04:10:23.193 INFO:tasks.workunit.client.1.smithi181.stdout:9/957: truncate d0/d44/d4b/d43/d88/da1/dfc/dbd/fc9 380579 0 2024-04-27T04:10:23.196 INFO:tasks.workunit.client.1.smithi181.stdout:9/958: mknod d0/d44/d4b/d50/c143 0 2024-04-27T04:10:23.198 INFO:tasks.workunit.client.0.smithi162.stdout:9/895: creat d3/d13/d2d/d3b/d7a/dd6/f114 x:0 0 0 2024-04-27T04:10:23.199 INFO:tasks.workunit.client.1.smithi181.stdout:9/959: readlink d0/d44/d4b/d43/d88/da1/dfc/dbd/lee 0 2024-04-27T04:10:23.200 INFO:tasks.workunit.client.0.smithi162.stdout:0/976: truncate d5/f11 1868251 0 2024-04-27T04:10:23.202 INFO:tasks.workunit.client.1.smithi181.stdout:9/960: fsync d0/d44/d4b/d43/d88/da1/dfc/f73 0 2024-04-27T04:10:23.203 INFO:tasks.workunit.client.0.smithi162.stdout:0/977: chown d5/dc/d1e/d46/d48/fd3 1672880 1 2024-04-27T04:10:23.205 INFO:tasks.workunit.client.0.smithi162.stdout:1/847: getdents d0/d7/d10/d75 0 2024-04-27T04:10:23.205 INFO:tasks.workunit.client.0.smithi162.stdout:7/956: truncate d3/d5/d5b/d83/de/d2b/d81/daa/dbd/dbe/dff/fc9 110720 0 2024-04-27T04:10:23.205 INFO:tasks.workunit.client.0.smithi162.stdout:0/978: chown d5/dc/d1e/d20/dec/df1/db4/ffa 378 1 2024-04-27T04:10:23.206 INFO:tasks.workunit.client.0.smithi162.stdout:3/887: mkdir d1/d18/d5d/d6f/d130/d13c 0 2024-04-27T04:10:23.206 INFO:tasks.workunit.client.0.smithi162.stdout:6/901: link d6/db/dc0/d74/ced d6/d1a/c129 0 2024-04-27T04:10:23.206 INFO:tasks.workunit.client.0.smithi162.stdout:4/869: dwrite d2/d25/f89 [0,4194304] 0 2024-04-27T04:10:23.208 INFO:tasks.workunit.client.1.smithi181.stdout:9/961: creat d0/d44/d4b/f144 x:0 0 0 2024-04-27T04:10:23.210 INFO:tasks.workunit.client.0.smithi162.stdout:6/902: truncate d6/db/dc0/d74/f81 4720793 0 2024-04-27T04:10:23.210 INFO:tasks.workunit.client.1.smithi181.stdout:9/962: dread d0/d44/d4b/d5f/f77 [0,4194304] 0 2024-04-27T04:10:23.211 INFO:tasks.workunit.client.1.smithi181.stdout:9/963: fsync d0/d44/d4f/d9d/fe7 0 2024-04-27T04:10:23.213 INFO:tasks.workunit.client.0.smithi162.stdout:0/979: symlink d5/d26/da2/dc0/l13e 0 2024-04-27T04:10:23.214 INFO:tasks.workunit.client.0.smithi162.stdout:3/888: truncate d1/d18/d5d/d6f/fa9 861533 0 2024-04-27T04:10:23.215 INFO:tasks.workunit.client.0.smithi162.stdout:0/980: chown d5/dc/d1e/d20/dec/df1 444 1 2024-04-27T04:10:23.217 INFO:tasks.workunit.client.0.smithi162.stdout:1/848: getdents d0/d7/d5e/d10e 0 2024-04-27T04:10:23.218 INFO:tasks.workunit.client.1.smithi181.stdout:9/964: creat d0/d44/d4b/d43/d10c/f145 x:0 0 0 2024-04-27T04:10:23.223 INFO:tasks.workunit.client.1.smithi181.stdout:9/965: creat d0/d44/d4b/d43/d89/d11d/f146 x:0 0 0 2024-04-27T04:10:23.224 INFO:tasks.workunit.client.0.smithi162.stdout:1/849: dwrite d0/d7/d10/f3d [4194304,4194304] 0 2024-04-27T04:10:23.224 INFO:tasks.workunit.client.0.smithi162.stdout:3/889: creat d1/d18/dac/f13d x:0 0 0 2024-04-27T04:10:23.227 INFO:tasks.workunit.client.1.smithi181.stdout:9/966: creat d0/d44/d4b/d43/d88/da1/dfc/dbd/df3/d85/f147 x:0 0 0 2024-04-27T04:10:23.228 INFO:tasks.workunit.client.0.smithi162.stdout:7/957: link d3/d5/d9/fe4 d3/d5/d5b/d83/de/d2b/d81/daa/f141 0 2024-04-27T04:10:23.231 INFO:tasks.workunit.client.0.smithi162.stdout:1/850: creat d0/d11/d5b/f117 x:0 0 0 2024-04-27T04:10:23.231 INFO:tasks.workunit.client.1.smithi181.stdout:9/967: stat d0/d44/d4b/d43/d8f/lcf 0 2024-04-27T04:10:23.232 INFO:tasks.workunit.client.0.smithi162.stdout:3/890: write d1/d70/f92 [41822,40357] 0 2024-04-27T04:10:23.233 INFO:tasks.workunit.client.0.smithi162.stdout:4/870: dread d2/d25/d1c/d39/d90/dad/f81 [0,4194304] 0 2024-04-27T04:10:23.235 INFO:tasks.workunit.client.1.smithi181.stdout:9/968: symlink d0/d44/d4b/d43/d88/da1/dfc/l148 0 2024-04-27T04:10:23.236 INFO:tasks.workunit.client.0.smithi162.stdout:7/958: mkdir d3/d5/d5b/d83/de/d2b/d81/daa/dbd/db6/d142 0 2024-04-27T04:10:23.237 INFO:tasks.workunit.client.1.smithi181.stdout:9/969: mkdir d0/d44/d4b/d43/d88/d149 0 2024-04-27T04:10:23.238 INFO:tasks.workunit.client.0.smithi162.stdout:6/903: dwrite d6/db/d32/f51 [0,4194304] 0 2024-04-27T04:10:23.239 INFO:tasks.workunit.client.0.smithi162.stdout:3/891: creat d1/d65/f13e x:0 0 0 2024-04-27T04:10:23.240 INFO:tasks.workunit.client.0.smithi162.stdout:0/981: creat d5/f13f x:0 0 0 2024-04-27T04:10:23.241 INFO:tasks.workunit.client.1.smithi181.stdout:9/970: symlink d0/d44/d4b/d43/d88/da1/dfc/d9a/l14a 0 2024-04-27T04:10:23.244 INFO:tasks.workunit.client.0.smithi162.stdout:1/851: write d0/d11/d50/fbd [315592,94455] 0 2024-04-27T04:10:23.246 INFO:tasks.workunit.client.0.smithi162.stdout:1/852: fdatasync d0/d7/d2a/f47 0 2024-04-27T04:10:23.247 INFO:tasks.workunit.client.0.smithi162.stdout:7/959: dwrite d3/d5/d5b/d83/de/d2b/d81/daa/dbd/db6/fd5 [0,4194304] 0 2024-04-27T04:10:23.249 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:10:22 smithi181 ceph-mon[33774]: pgmap v11: 65 pgs: 65 active+clean; 3.6 GiB data, 12 GiB used, 524 GiB / 536 GiB avail; 94 MiB/s rd, 234 MiB/s wr, 565 op/s 2024-04-27T04:10:23.250 INFO:tasks.workunit.client.0.smithi162.stdout:3/892: symlink d1/d18/l13f 0 2024-04-27T04:10:23.250 INFO:tasks.workunit.client.1.smithi181.stdout:9/971: creat d0/d44/d4b/d43/d88/de3/f14b x:0 0 0 2024-04-27T04:10:23.251 INFO:tasks.workunit.client.0.smithi162.stdout:4/871: creat d2/d25/d1c/d24/f11c x:0 0 0 2024-04-27T04:10:23.253 INFO:tasks.workunit.client.1.smithi181.stdout:9/972: write d0/d44/d4b/d43/d89/d94/f12b [377375,129018] 0 2024-04-27T04:10:23.254 INFO:tasks.workunit.client.0.smithi162.stdout:3/893: write d1/d18/d5d/d5f/d66/fe5 [1030322,69964] 0 2024-04-27T04:10:23.257 INFO:tasks.workunit.client.0.smithi162.stdout:6/904: link d6/db/d32/f3b d6/d1a/d34/d3a/dbf/dd7/f12a 0 2024-04-27T04:10:23.260 INFO:tasks.workunit.client.0.smithi162.stdout:0/982: dread d5/d26/d69/f6c [0,4194304] 0 2024-04-27T04:10:23.262 INFO:tasks.workunit.client.0.smithi162.stdout:7/960: creat d3/d5/d5b/d83/de/d2b/d81/daa/dbd/dbe/dff/d57/f143 x:0 0 0 2024-04-27T04:10:23.263 INFO:tasks.workunit.client.0.smithi162.stdout:3/894: creat d1/d21/d47/f140 x:0 0 0 2024-04-27T04:10:23.263 INFO:tasks.workunit.client.0.smithi162.stdout:7/961: stat d3/d5/d5b/d83/de/f38 0 2024-04-27T04:10:23.265 INFO:tasks.workunit.client.0.smithi162.stdout:3/895: fdatasync d1/f4c 0 2024-04-27T04:10:23.272 INFO:tasks.workunit.client.0.smithi162.stdout:8/916: sync 2024-04-27T04:10:23.272 INFO:tasks.workunit.client.0.smithi162.stdout:2/909: sync 2024-04-27T04:10:23.273 INFO:tasks.workunit.client.0.smithi162.stdout:4/872: getdents d2/d25/d1c/d39/d3b/d47 0 2024-04-27T04:10:23.274 INFO:tasks.workunit.client.0.smithi162.stdout:8/917: dread - d0/de/d6e/d71/d86/dd9/ff3 zero size 2024-04-27T04:10:23.277 INFO:tasks.workunit.client.0.smithi162.stdout:3/896: rename d1/d70/lbf to d1/d21/d47/d57/l141 0 2024-04-27T04:10:23.282 INFO:tasks.workunit.client.0.smithi162.stdout:3/897: dwrite d1/d70/f8c [0,4194304] 0 2024-04-27T04:10:23.282 INFO:tasks.workunit.client.0.smithi162.stdout:6/905: dread d6/f26 [0,4194304] 0 2024-04-27T04:10:23.287 INFO:tasks.workunit.client.0.smithi162.stdout:1/853: write d0/d7/d5e/f62 [1030983,73314] 0 2024-04-27T04:10:23.287 INFO:tasks.workunit.client.0.smithi162.stdout:1/854: stat d0/d11/d3e/d59 0 2024-04-27T04:10:23.291 INFO:tasks.workunit.client.0.smithi162.stdout:8/918: symlink d0/d10/d2b/d2d/d2f/d39/de1/l123 0 2024-04-27T04:10:23.296 INFO:tasks.workunit.client.0.smithi162.stdout:7/962: rmdir d3/d5/d5b/d83/de/d2b/d81/daa/dbd/db6/d142 0 2024-04-27T04:10:23.296 INFO:tasks.workunit.client.0.smithi162.stdout:7/963: chown d3/d5/d11c 9 1 2024-04-27T04:10:23.299 INFO:tasks.workunit.client.0.smithi162.stdout:6/906: chown d6/d1a/d34/d7b/d7e/d97/lc1 53691 1 2024-04-27T04:10:23.301 INFO:tasks.workunit.client.0.smithi162.stdout:2/910: mkdir d0/d123 0 2024-04-27T04:10:23.303 INFO:tasks.workunit.client.0.smithi162.stdout:4/873: mkdir d2/d25/d1c/d39/d3b/d119/d11d 0 2024-04-27T04:10:23.308 INFO:tasks.workunit.client.0.smithi162.stdout:3/898: mknod d1/d18/d5d/c142 0 2024-04-27T04:10:23.310 INFO:tasks.workunit.client.0.smithi162.stdout:7/964: symlink d3/d5/d5b/d83/de/d2b/d81/d96/dc8/d11a/l144 0 2024-04-27T04:10:23.312 INFO:tasks.workunit.client.0.smithi162.stdout:6/907: creat d6/d1a/d34/d7f/d80/f12b x:0 0 0 2024-04-27T04:10:23.315 INFO:tasks.workunit.client.0.smithi162.stdout:4/874: truncate d2/f11 2235534 0 2024-04-27T04:10:23.316 INFO:tasks.workunit.client.0.smithi162.stdout:4/875: read - d2/d25/d1c/d39/d70/dcd/f10c zero size 2024-04-27T04:10:23.318 INFO:tasks.workunit.client.0.smithi162.stdout:2/911: dread d0/d4/d2a/f9e [0,4194304] 0 2024-04-27T04:10:23.319 INFO:tasks.workunit.client.0.smithi162.stdout:1/855: rename d0/d11/d5b/dd3 to d0/d7/d3c/d54/dae/d10c/d118 0 2024-04-27T04:10:23.320 INFO:tasks.workunit.client.0.smithi162.stdout:1/856: dread - d0/d7/d5e/ff2 zero size 2024-04-27T04:10:23.320 INFO:tasks.workunit.client.0.smithi162.stdout:2/912: chown d0/d4/d2a/d41/f81 0 1 2024-04-27T04:10:23.323 INFO:tasks.workunit.client.0.smithi162.stdout:7/965: symlink d3/d4f/l145 0 2024-04-27T04:10:23.325 INFO:tasks.workunit.client.0.smithi162.stdout:7/966: write d3/d5/d5b/d83/de/d2b/d81/f82 [672825,31463] 0 2024-04-27T04:10:23.333 INFO:tasks.workunit.client.0.smithi162.stdout:4/876: rename d2/d25/d1c/d39/ld0 to d2/d25/d1c/d24/dd2/df7/l11e 0 2024-04-27T04:10:23.347 INFO:tasks.workunit.client.0.smithi162.stdout:1/857: dread d0/d7/d3c/f43 [0,4194304] 0 2024-04-27T04:10:23.348 INFO:tasks.workunit.client.0.smithi162.stdout:4/877: dread - d2/d54/fec zero size 2024-04-27T04:10:23.350 INFO:tasks.workunit.client.0.smithi162.stdout:3/899: creat d1/d18/f143 x:0 0 0 2024-04-27T04:10:23.350 INFO:tasks.workunit.client.0.smithi162.stdout:4/878: truncate d2/d25/d1c/d39/d3b/d6e/d6f/fc7 956606 0 2024-04-27T04:10:23.352 INFO:tasks.workunit.client.0.smithi162.stdout:7/967: mknod d3/d5/d5b/d83/de/d2b/d81/d139/c146 0 2024-04-27T04:10:23.355 INFO:tasks.workunit.client.0.smithi162.stdout:6/908: rename d6/d1a/d34/d3a/d68/d6c/de3/lf4 to d6/db/d32/df0/d11d/l12c 0 2024-04-27T04:10:23.358 INFO:tasks.workunit.client.0.smithi162.stdout:1/858: mknod d0/d7/d5e/d10e/c119 0 2024-04-27T04:10:23.358 INFO:tasks.workunit.client.0.smithi162.stdout:8/919: write d0/d10/d25/f14 [8493336,98317] 0 2024-04-27T04:10:23.359 INFO:tasks.workunit.client.0.smithi162.stdout:8/920: chown d0/c8 230 1 2024-04-27T04:10:23.361 INFO:tasks.workunit.client.0.smithi162.stdout:2/913: creat d0/d4/d17/d88/f124 x:0 0 0 2024-04-27T04:10:23.362 INFO:tasks.workunit.client.0.smithi162.stdout:2/914: chown d0/d4/d17/db4 24674 1 2024-04-27T04:10:23.364 INFO:tasks.workunit.client.0.smithi162.stdout:6/909: dread - d6/d1a/d34/d7f/d80/fd0 zero size 2024-04-27T04:10:23.364 INFO:tasks.workunit.client.0.smithi162.stdout:1/859: creat d0/d7/d10/de4/f11a x:0 0 0 2024-04-27T04:10:23.365 INFO:tasks.workunit.client.0.smithi162.stdout:3/900: rmdir d1/d21/d47/d57/dd6/d112/d132/dad/dc5 39 2024-04-27T04:10:23.367 INFO:tasks.workunit.client.0.smithi162.stdout:7/968: creat d3/d5/d5b/d83/de/d2b/d81/d96/d11b/f147 x:0 0 0 2024-04-27T04:10:23.367 INFO:tasks.workunit.client.0.smithi162.stdout:8/921: dwrite d0/d10/d2b/d2e/dca/f11f [0,4194304] 0 2024-04-27T04:10:23.373 INFO:tasks.workunit.client.0.smithi162.stdout:6/910: creat d6/d1a/d34/d7f/d80/db3/dcd/d110/f12d x:0 0 0 2024-04-27T04:10:23.373 INFO:tasks.workunit.client.0.smithi162.stdout:3/901: mknod d1/d18/d23/d58/d63/c144 0 2024-04-27T04:10:23.373 INFO:tasks.workunit.client.0.smithi162.stdout:8/922: creat d0/d10/d76/f124 x:0 0 0 2024-04-27T04:10:23.375 INFO:tasks.workunit.client.0.smithi162.stdout:2/915: dwrite d0/d4/d2a/d3b/d49/d5a/fe7 [0,4194304] 0 2024-04-27T04:10:23.379 INFO:tasks.workunit.client.0.smithi162.stdout:7/969: rename d3/d5/d9 to d3/d5/d5b/d83/de/d2b/d81/daa/dbd/dbe/d148 0 2024-04-27T04:10:23.381 INFO:tasks.workunit.client.0.smithi162.stdout:3/902: read d1/d21/d47/d57/dd6/d112/d132/dad/fdf [1640602,72444] 0 2024-04-27T04:10:23.386 INFO:tasks.workunit.client.0.smithi162.stdout:2/916: dread d0/fe6 [0,4194304] 0 2024-04-27T04:10:23.391 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:10:22 smithi162 ceph-mon[26841]: pgmap v11: 65 pgs: 65 active+clean; 3.6 GiB data, 12 GiB used, 524 GiB / 536 GiB avail; 94 MiB/s rd, 234 MiB/s wr, 565 op/s 2024-04-27T04:10:23.392 INFO:tasks.workunit.client.0.smithi162.stdout:2/917: fdatasync d0/d4/d2a/d3b/d49/d5a/d63/d84/f99 0 2024-04-27T04:10:23.392 INFO:tasks.workunit.client.0.smithi162.stdout:8/923: dwrite d0/d10/d2b/d2d/d2f/f116 [0,4194304] 0 2024-04-27T04:10:23.392 INFO:tasks.workunit.client.0.smithi162.stdout:3/903: creat d1/d18/d23/d58/f145 x:0 0 0 2024-04-27T04:10:23.392 INFO:tasks.workunit.client.0.smithi162.stdout:7/970: creat d3/d5/d5b/d83/de/d2b/d81/d96/f149 x:0 0 0 2024-04-27T04:10:23.393 INFO:tasks.workunit.client.0.smithi162.stdout:8/924: chown d0/d10/d2b/c9f 2704623 1 2024-04-27T04:10:23.396 INFO:tasks.workunit.client.0.smithi162.stdout:3/904: symlink d1/d21/d47/d57/dd6/d112/d132/daa/de6/l146 0 2024-04-27T04:10:23.396 INFO:tasks.workunit.client.0.smithi162.stdout:2/918: dwrite d0/f1 [0,4194304] 0 2024-04-27T04:10:23.398 INFO:tasks.workunit.client.0.smithi162.stdout:7/971: write d3/d5/d5b/d83/de/d2b/d81/d96/d135/d122/f12f [470484,83071] 0 2024-04-27T04:10:23.403 INFO:tasks.workunit.client.0.smithi162.stdout:8/925: creat d0/d10/d2b/d2d/d2f/d39/de1/d121/f125 x:0 0 0 2024-04-27T04:10:23.404 INFO:tasks.workunit.client.0.smithi162.stdout:3/905: read d1/d18/d23/f2e [1183693,57014] 0 2024-04-27T04:10:23.404 INFO:tasks.workunit.client.0.smithi162.stdout:7/972: chown d3/d5/d5b/d83/de/d2b/d81/daa/dbd/dbe/d148/fe4 7294020 1 2024-04-27T04:10:23.405 INFO:tasks.workunit.client.0.smithi162.stdout:1/860: write d0/d7/f5a [241398,114635] 0 2024-04-27T04:10:23.407 INFO:tasks.workunit.client.0.smithi162.stdout:8/926: rmdir d0/d10/d2b/d2e 39 2024-04-27T04:10:23.409 INFO:tasks.workunit.client.0.smithi162.stdout:3/906: truncate d1/d21/d47/d57/dd6/d112/d132/daa/f119 262063 0 2024-04-27T04:10:23.413 INFO:tasks.workunit.client.0.smithi162.stdout:3/907: symlink d1/d18/d23/d58/d63/dfa/l147 0 2024-04-27T04:10:23.414 INFO:tasks.workunit.client.0.smithi162.stdout:8/927: symlink d0/d10/d2b/d2d/d2f/l126 0 2024-04-27T04:10:23.416 INFO:tasks.workunit.client.0.smithi162.stdout:2/919: truncate d0/d4/d17/d88/f9d 2726166 0 2024-04-27T04:10:23.417 INFO:tasks.workunit.client.0.smithi162.stdout:6/911: dwrite d6/d1a/d34/d3a/d68/d6c/fda [0,4194304] 0 2024-04-27T04:10:23.419 INFO:tasks.workunit.client.0.smithi162.stdout:7/973: link d3/d5/d5b/d83/de/d2b/d81/daa/dbd/dab/fed d3/d5/d5b/d83/de/d2b/d81/d96/dc8/f14a 0 2024-04-27T04:10:23.419 INFO:tasks.workunit.client.0.smithi162.stdout:2/920: dread - d0/d4/d17/d4f/d10c/ffe zero size 2024-04-27T04:10:23.422 INFO:tasks.workunit.client.0.smithi162.stdout:8/928: dread d0/d10/d2b/d2e/dca/fd2 [0,4194304] 0 2024-04-27T04:10:23.422 INFO:tasks.workunit.client.0.smithi162.stdout:1/861: write d0/d7/d3c/f57 [7150222,43300] 0 2024-04-27T04:10:23.423 INFO:tasks.workunit.client.0.smithi162.stdout:5/972: sync 2024-04-27T04:10:23.424 INFO:tasks.workunit.client.0.smithi162.stdout:3/908: creat d1/d21/f148 x:0 0 0 2024-04-27T04:10:23.425 INFO:tasks.workunit.client.0.smithi162.stdout:8/929: fdatasync d0/d10/d25/d7d/f8b 0 2024-04-27T04:10:23.426 INFO:tasks.workunit.client.0.smithi162.stdout:1/862: truncate d0/d11/d44/d66/fe5 703011 0 2024-04-27T04:10:23.427 INFO:tasks.workunit.client.0.smithi162.stdout:8/930: fsync d0/de/d20/f50 0 2024-04-27T04:10:23.427 INFO:tasks.workunit.client.0.smithi162.stdout:2/921: unlink d0/d4/d17/d2b/f95 0 2024-04-27T04:10:23.427 INFO:tasks.workunit.client.0.smithi162.stdout:7/974: fsync d3/d5/d3c/d4a/d55/dc6/f91 0 2024-04-27T04:10:23.427 INFO:tasks.workunit.client.0.smithi162.stdout:2/922: chown d0/d4/d17/db4 772398210 1 2024-04-27T04:10:23.428 INFO:tasks.workunit.client.0.smithi162.stdout:1/863: stat d0/d11/d3e/d59/d7f/dc5 0 2024-04-27T04:10:23.432 INFO:tasks.workunit.client.0.smithi162.stdout:3/909: mkdir d1/d18/d5d/d5f/d66/dd2/d149 0 2024-04-27T04:10:23.434 INFO:tasks.workunit.client.0.smithi162.stdout:2/923: fdatasync d0/d4/f3a 0 2024-04-27T04:10:23.441 INFO:tasks.workunit.client.0.smithi162.stdout:1/864: creat d0/d80/dba/f11b x:0 0 0 2024-04-27T04:10:23.442 INFO:tasks.workunit.client.0.smithi162.stdout:8/931: getdents d0/d10/d2b/d2d/d2f/d39/df8 0 2024-04-27T04:10:23.445 INFO:tasks.workunit.client.0.smithi162.stdout:7/975: dwrite d3/d5/d5b/d60/f94 [0,4194304] 0 2024-04-27T04:10:23.446 INFO:tasks.workunit.client.0.smithi162.stdout:8/932: chown d0/de/d20/d6c/c74 1226 1 2024-04-27T04:10:23.446 INFO:tasks.workunit.client.0.smithi162.stdout:6/912: dwrite d6/d1a/d34/d7f/d80/fd0 [0,4194304] 0 2024-04-27T04:10:23.455 INFO:tasks.workunit.client.0.smithi162.stdout:5/973: dwrite d8/d1a/d7d/ddf/f12c [4194304,4194304] 0 2024-04-27T04:10:23.456 INFO:tasks.workunit.client.0.smithi162.stdout:5/974: chown d8/d9/d5a/c72 3768 1 2024-04-27T04:10:23.459 INFO:tasks.workunit.client.0.smithi162.stdout:7/976: dwrite d3/d5/d5b/d83/de/d2b/d81/daa/dbd/dbe/d148/fe7 [0,4194304] 0 2024-04-27T04:10:23.530 INFO:tasks.workunit.client.0.smithi162.stdout:9/896: sync 2024-04-27T04:10:23.576 INFO:tasks.workunit.client.0.smithi162.stdout:4/879: sync 2024-04-27T04:10:23.576 INFO:tasks.workunit.client.0.smithi162.stdout:0/983: sync 2024-04-27T04:10:23.589 INFO:tasks.workunit.client.0.smithi162.stdout:3/910: mkdir d1/d21/d47/d57/dd6/d112/d14a 0 2024-04-27T04:10:23.590 INFO:tasks.workunit.client.0.smithi162.stdout:2/924: mknod d0/d4/d2a/d3b/d49/d5a/d63/d6e/c125 0 2024-04-27T04:10:23.595 INFO:tasks.workunit.client.0.smithi162.stdout:5/975: write d8/d1a/d29/d8b/fd2 [3523925,48505] 0 2024-04-27T04:10:23.595 INFO:tasks.workunit.client.0.smithi162.stdout:7/977: creat d3/d5/d5b/d60/d65/f14b x:0 0 0 2024-04-27T04:10:23.598 INFO:tasks.workunit.client.0.smithi162.stdout:3/911: write d1/d21/d47/d57/fae [691966,27619] 0 2024-04-27T04:10:23.598 INFO:tasks.workunit.client.0.smithi162.stdout:3/912: read - d1/d18/d23/d58/d63/d88/f121 zero size 2024-04-27T04:10:23.599 INFO:tasks.workunit.client.0.smithi162.stdout:2/925: chown d0/d4/d17/d2b/d64/d89/cf4 1233090 1 2024-04-27T04:10:23.600 INFO:tasks.workunit.client.0.smithi162.stdout:3/913: fdatasync d1/d21/d47/d57/dd6/d112/d132/dad/f128 0 2024-04-27T04:10:23.600 INFO:tasks.workunit.client.0.smithi162.stdout:2/926: chown d0/d4/d2a/d3b/d49/d5a/d7e/cbe 178082 1 2024-04-27T04:10:23.601 INFO:tasks.workunit.client.0.smithi162.stdout:2/927: chown d0/l34 37 1 2024-04-27T04:10:23.601 INFO:tasks.workunit.client.0.smithi162.stdout:5/976: symlink d8/d9/d5b/d111/d112/l132 0 2024-04-27T04:10:23.602 INFO:tasks.workunit.client.0.smithi162.stdout:9/897: mkdir d3/d11/d45/d4e/d115 0 2024-04-27T04:10:23.604 INFO:tasks.workunit.client.0.smithi162.stdout:9/898: chown d3/d13/d2d/d3b/d69/fbb 11725642 1 2024-04-27T04:10:23.605 INFO:tasks.workunit.client.0.smithi162.stdout:7/978: mknod d3/d5/d5b/d83/de/d2b/d81/d139/c14c 0 2024-04-27T04:10:23.606 INFO:tasks.workunit.client.0.smithi162.stdout:7/979: write d3/f13f [109983,64010] 0 2024-04-27T04:10:23.607 INFO:tasks.workunit.client.0.smithi162.stdout:2/928: dwrite d0/d4/d17/d2b/d64/d89/fcc [0,4194304] 0 2024-04-27T04:10:23.608 INFO:tasks.workunit.client.0.smithi162.stdout:6/913: rmdir d6/db/d32/df0/d11d/d121 0 2024-04-27T04:10:23.609 INFO:tasks.workunit.client.0.smithi162.stdout:3/914: symlink d1/d65/l14b 0 2024-04-27T04:10:23.613 INFO:tasks.workunit.client.0.smithi162.stdout:9/899: creat d3/de1/f116 x:0 0 0 2024-04-27T04:10:23.613 INFO:tasks.workunit.client.0.smithi162.stdout:4/880: link d2/d25/f8 d2/da/d114/f11f 0 2024-04-27T04:10:23.615 INFO:tasks.workunit.client.0.smithi162.stdout:7/980: fsync d3/d4f/f9d 0 2024-04-27T04:10:23.623 INFO:tasks.workunit.client.0.smithi162.stdout:4/881: rename d2/d25/d1c/d39/d70/dcd/f10c to d2/d25/d1c/d39/d70/dcd/df0/f120 0 2024-04-27T04:10:23.625 INFO:tasks.workunit.client.0.smithi162.stdout:1/865: sync 2024-04-27T04:10:23.626 INFO:tasks.workunit.client.0.smithi162.stdout:3/915: creat d1/d21/d47/d57/dd6/d112/d132/dad/dc5/d109/f14c x:0 0 0 2024-04-27T04:10:23.626 INFO:tasks.workunit.client.0.smithi162.stdout:1/866: chown d0/d80/f114 22233199 1 2024-04-27T04:10:23.627 INFO:tasks.workunit.client.0.smithi162.stdout:3/916: write d1/d18/f4a [4019927,70748] 0 2024-04-27T04:10:23.628 INFO:tasks.workunit.client.0.smithi162.stdout:3/917: dread - d1/d18/d5d/d5f/d66/dd2/da3/f113 zero size 2024-04-27T04:10:23.629 INFO:tasks.workunit.client.0.smithi162.stdout:4/882: creat d2/d25/d1c/d39/d3b/d119/f121 x:0 0 0 2024-04-27T04:10:23.629 INFO:tasks.workunit.client.0.smithi162.stdout:3/918: chown d1/d21/d47/d57/dd6/d112/d132/dad 1730 1 2024-04-27T04:10:23.633 INFO:tasks.workunit.client.0.smithi162.stdout:6/914: getdents d6/d1a/d34/d43 0 2024-04-27T04:10:23.633 INFO:tasks.workunit.client.0.smithi162.stdout:9/900: getdents d3/d13/d2d/d44 0 2024-04-27T04:10:23.634 INFO:tasks.workunit.client.0.smithi162.stdout:3/919: creat d1/d18/d5d/d5f/dbe/f14d x:0 0 0 2024-04-27T04:10:23.637 INFO:tasks.workunit.client.0.smithi162.stdout:9/901: fsync d3/d13/d2d/f37 0 2024-04-27T04:10:23.643 INFO:tasks.workunit.client.0.smithi162.stdout:8/933: write d0/d10/d25/d65/fe9 [282577,75513] 0 2024-04-27T04:10:23.644 INFO:tasks.workunit.client.0.smithi162.stdout:0/984: write d5/d26/fdf [314203,45070] 0 2024-04-27T04:10:23.648 INFO:tasks.workunit.client.0.smithi162.stdout:8/934: rename d0/d10/d2b/d2d/d2f/d39/df8 to d0/d10/d2b/d2e/d127 0 2024-04-27T04:10:23.649 INFO:tasks.workunit.client.0.smithi162.stdout:0/985: dwrite d5/dc/d1e/fc7 [0,4194304] 0 2024-04-27T04:10:23.650 INFO:tasks.workunit.client.0.smithi162.stdout:8/935: mkdir d0/d10/d25/d65/d128 0 2024-04-27T04:10:23.651 INFO:tasks.workunit.client.0.smithi162.stdout:5/977: write d8/d9/d55/f120 [95415,9161] 0 2024-04-27T04:10:23.652 INFO:tasks.workunit.client.0.smithi162.stdout:7/981: write d3/d5/d5b/d83/de/d2b/d81/daa/df1/f115 [887500,126960] 0 2024-04-27T04:10:23.657 INFO:tasks.workunit.client.0.smithi162.stdout:7/982: truncate d3/d5/d5b/d60/fd3 8306579 0 2024-04-27T04:10:23.659 INFO:tasks.workunit.client.0.smithi162.stdout:7/983: dread - d3/d5/d5b/d83/de/d2b/d81/d96/f9e zero size 2024-04-27T04:10:23.661 INFO:tasks.workunit.client.0.smithi162.stdout:5/978: creat d8/d1a/d23/d40/de4/d11f/f133 x:0 0 0 2024-04-27T04:10:23.661 INFO:tasks.workunit.client.0.smithi162.stdout:1/867: dwrite d0/d11/f42 [0,4194304] 0 2024-04-27T04:10:23.663 INFO:tasks.workunit.client.0.smithi162.stdout:8/936: creat d0/f129 x:0 0 0 2024-04-27T04:10:23.663 INFO:tasks.workunit.client.0.smithi162.stdout:1/868: chown d0/lc8 27947229 1 2024-04-27T04:10:23.696 INFO:tasks.workunit.client.0.smithi162.stdout:5/979: dread - d8/d9/d5b/d114/f131 zero size 2024-04-27T04:10:23.697 INFO:tasks.workunit.client.0.smithi162.stdout:0/986: truncate d5/dc/d1e/d41/f4f 2825588 0 2024-04-27T04:10:23.698 INFO:tasks.workunit.client.0.smithi162.stdout:7/984: stat d3/d5/d5b/d83/de/d2b/d81/daa/dbd/dab/f13e 0 2024-04-27T04:10:23.699 INFO:tasks.workunit.client.0.smithi162.stdout:6/915: dwrite d6/db/fb0 [0,4194304] 0 2024-04-27T04:10:23.700 INFO:tasks.workunit.client.0.smithi162.stdout:8/937: readlink d0/d10/d2b/d2e/dca/la4 0 2024-04-27T04:10:23.703 INFO:tasks.workunit.client.0.smithi162.stdout:1/869: truncate d0/d80/f114 857927 0 2024-04-27T04:10:23.705 INFO:tasks.workunit.client.0.smithi162.stdout:7/985: stat d3/d5/d5b/ccc 0 2024-04-27T04:10:23.708 INFO:tasks.workunit.client.0.smithi162.stdout:0/987: dread - d5/dc/d1e/d20/dec/df1/db4/f118 zero size 2024-04-27T04:10:23.709 INFO:tasks.workunit.client.0.smithi162.stdout:9/902: dwrite d3/d23/f5c [0,4194304] 0 2024-04-27T04:10:23.709 INFO:tasks.workunit.client.0.smithi162.stdout:2/929: dwrite d0/d20/fcf [0,4194304] 0 2024-04-27T04:10:23.713 INFO:tasks.workunit.client.0.smithi162.stdout:8/938: rename d0/f129 to d0/d10/d2b/d2d/d2f/d39/de1/f12a 0 2024-04-27T04:10:23.714 INFO:tasks.workunit.client.1.smithi181.stdout:9/973: sync 2024-04-27T04:10:23.714 INFO:tasks.workunit.client.0.smithi162.stdout:8/939: stat d0/c5c 0 2024-04-27T04:10:23.717 INFO:tasks.workunit.client.0.smithi162.stdout:8/940: stat d0/d10/d25/d7d/f8b 0 2024-04-27T04:10:23.717 INFO:tasks.workunit.client.0.smithi162.stdout:5/980: dwrite d8/d1a/d29/d8b/fd2 [4194304,4194304] 0 2024-04-27T04:10:23.719 INFO:tasks.workunit.client.1.smithi181.stdout:9/974: truncate d0/d44/d4b/d43/d88/da1/dfc/dbd/df3/fb5 287822 0 2024-04-27T04:10:23.720 INFO:tasks.workunit.client.0.smithi162.stdout:8/941: dread - d0/d10/d44/ff4 zero size 2024-04-27T04:10:23.721 INFO:tasks.workunit.client.1.smithi181.stdout:9/975: write d0/d44/d4b/d43/d89/d94/f12b [285545,57817] 0 2024-04-27T04:10:23.722 INFO:tasks.workunit.client.0.smithi162.stdout:9/903: truncate d3/d23/fd3 335835 0 2024-04-27T04:10:23.722 INFO:tasks.workunit.client.0.smithi162.stdout:0/988: dread d5/d26/d69/f6c [0,4194304] 0 2024-04-27T04:10:23.724 INFO:tasks.workunit.client.0.smithi162.stdout:0/989: chown d5/dc/d1e/d46/d48/d9e/dce/dd1/d116/da8/lb2 299732 1 2024-04-27T04:10:23.724 INFO:tasks.workunit.client.0.smithi162.stdout:6/916: truncate d6/f30 679076 0 2024-04-27T04:10:23.729 INFO:tasks.workunit.client.0.smithi162.stdout:1/870: rename d0/d11/d3e/f6c to d0/d7/d10/d75/db7/dfd/f11c 0 2024-04-27T04:10:23.730 INFO:tasks.workunit.client.1.smithi181.stdout:9/976: rmdir d0/d44/d4b/d43/d88/da1/dfc/dbd/df3/d85/dd8/dfa 39 2024-04-27T04:10:23.730 INFO:tasks.workunit.client.0.smithi162.stdout:7/986: write d3/d5/d5b/d83/de/d2b/d81/daa/dbd/dbe/dff/d57/f108 [3490655,92247] 0 2024-04-27T04:10:23.734 INFO:tasks.workunit.client.0.smithi162.stdout:6/917: fsync d6/db/d76/f125 0 2024-04-27T04:10:23.734 INFO:tasks.workunit.client.0.smithi162.stdout:0/990: mknod d5/dc/d1e/d20/dec/c140 0 2024-04-27T04:10:23.734 INFO:tasks.workunit.client.1.smithi181.stdout:9/977: mknod d0/d44/d4b/c14c 0 2024-04-27T04:10:23.735 INFO:tasks.workunit.client.0.smithi162.stdout:5/981: symlink d8/d1a/d23/dbc/d66/l134 0 2024-04-27T04:10:23.737 INFO:tasks.workunit.client.0.smithi162.stdout:8/942: dwrite d0/d10/d25/f36 [4194304,4194304] 0 2024-04-27T04:10:23.737 INFO:tasks.workunit.client.0.smithi162.stdout:2/930: rename d0/c1c to d0/d4/d17/d2b/d64/d96/daa/df9/dfa/c126 0 2024-04-27T04:10:23.737 INFO:tasks.workunit.client.0.smithi162.stdout:1/871: creat d0/d7/d10/f11d x:0 0 0 2024-04-27T04:10:23.738 INFO:tasks.workunit.client.1.smithi181.stdout:9/978: rmdir d0/d44/d4b/d43/d88/da1/dfc/dbd/df3 39 2024-04-27T04:10:23.740 INFO:tasks.workunit.client.1.smithi181.stdout:9/979: rmdir d0/d44/d4b/d43/d89 39 2024-04-27T04:10:23.741 INFO:tasks.workunit.client.0.smithi162.stdout:7/987: symlink d3/d5/d5b/d83/de/d2b/da7/d138/l14d 0 2024-04-27T04:10:23.741 INFO:tasks.workunit.client.0.smithi162.stdout:7/988: chown d3/d4f/c90 1 1 2024-04-27T04:10:23.742 INFO:tasks.workunit.client.0.smithi162.stdout:0/991: stat d5/dc/d1e/d46/d48/d9e/dce/ffd 0 2024-04-27T04:10:23.751 INFO:tasks.workunit.client.0.smithi162.stdout:4/883: sync 2024-04-27T04:10:23.753 INFO:tasks.workunit.client.0.smithi162.stdout:5/982: symlink d8/d9/d5a/d108/l135 0 2024-04-27T04:10:23.754 INFO:tasks.workunit.client.0.smithi162.stdout:5/983: write d8/d1a/d29/f12f [398246,119419] 0 2024-04-27T04:10:23.754 INFO:tasks.workunit.client.0.smithi162.stdout:0/992: mknod d5/dc/d1e/d46/d48/d9e/dce/dd1/d116/ddd/d117/c141 0 2024-04-27T04:10:23.755 INFO:tasks.workunit.client.0.smithi162.stdout:9/904: write d3/fbd [3560752,41596] 0 2024-04-27T04:10:23.756 INFO:tasks.workunit.client.0.smithi162.stdout:9/905: stat d3/d23/d7c 0 2024-04-27T04:10:23.757 INFO:tasks.workunit.client.0.smithi162.stdout:1/872: dread d0/d7/d10/d75/db7/dfd/f11c [0,4194304] 0 2024-04-27T04:10:23.758 INFO:tasks.workunit.client.0.smithi162.stdout:6/918: link d6/d1a/d34/d7b/d7e/fff d6/d1a/d34/d7b/d7e/f12e 0 2024-04-27T04:10:23.762 INFO:tasks.workunit.client.0.smithi162.stdout:0/993: unlink d5/d26/fdf 0 2024-04-27T04:10:23.763 INFO:tasks.workunit.client.0.smithi162.stdout:1/873: creat d0/d7/d5e/f11e x:0 0 0 2024-04-27T04:10:23.763 INFO:tasks.workunit.client.1.smithi181.stdout:9/980: dread d0/d44/d4f/df7/fe1 [0,4194304] 0 2024-04-27T04:10:23.764 INFO:tasks.workunit.client.1.smithi181.stdout:9/981: readlink d0/d44/d4f/lac 0 2024-04-27T04:10:23.765 INFO:tasks.workunit.client.0.smithi162.stdout:7/989: link d3/d5/d5b/d83/de/d2b/d81/d117/c126 d3/d5/d5b/d83/de/d2b/d81/d96/dc8/d11a/d137/c14e 0 2024-04-27T04:10:23.767 INFO:tasks.workunit.client.0.smithi162.stdout:3/920: sync 2024-04-27T04:10:23.768 INFO:tasks.workunit.client.0.smithi162.stdout:5/984: creat d8/d9/f136 x:0 0 0 2024-04-27T04:10:23.769 INFO:tasks.workunit.client.0.smithi162.stdout:2/931: dwrite d0/d4/d2a/d3b/d49/d5a/d7e/fdb [0,4194304] 0 2024-04-27T04:10:23.769 INFO:tasks.workunit.client.0.smithi162.stdout:6/919: creat d6/db/d76/d10e/f12f x:0 0 0 2024-04-27T04:10:23.770 INFO:tasks.workunit.client.0.smithi162.stdout:1/874: rmdir d0/d7/d3c/d77 39 2024-04-27T04:10:23.770 INFO:tasks.workunit.client.0.smithi162.stdout:9/906: dwrite d3/d13/d2d/d30/d7f/fe7 [0,4194304] 0 2024-04-27T04:10:23.770 INFO:tasks.workunit.client.1.smithi181.stdout:9/982: rename d0/d44/d4b/c47 to d0/d44/d4b/d43/d8f/c14d 0 2024-04-27T04:10:23.771 INFO:tasks.workunit.client.0.smithi162.stdout:7/990: stat d3/d5/d5b/d83/de/d2b/d81/daa/f141 0 2024-04-27T04:10:23.776 INFO:tasks.workunit.client.0.smithi162.stdout:4/884: dread d2/d25/d1c/d39/d3b/d47/ffc [0,4194304] 0 2024-04-27T04:10:23.776 INFO:tasks.workunit.client.0.smithi162.stdout:6/920: readlink d6/db/dc0/le1 0 2024-04-27T04:10:23.776 INFO:tasks.workunit.client.0.smithi162.stdout:0/994: fsync d5/dc/d1e/d46/d48/d9e/fdb 0 2024-04-27T04:10:23.776 INFO:tasks.workunit.client.0.smithi162.stdout:9/907: fdatasync d3/fa 0 2024-04-27T04:10:23.779 INFO:tasks.workunit.client.0.smithi162.stdout:8/943: dread d0/de/d20/d6c/fb6 [0,4194304] 0 2024-04-27T04:10:23.785 INFO:tasks.workunit.client.0.smithi162.stdout:5/985: rename d8/d9/d5b/fe5 to d8/d1a/d23/d40/de4/d11f/f137 0 2024-04-27T04:10:23.786 INFO:tasks.workunit.client.0.smithi162.stdout:2/932: fdatasync d0/d4/d2a/f9e 0 2024-04-27T04:10:23.786 INFO:tasks.workunit.client.0.smithi162.stdout:3/921: mknod d1/d21/d47/d57/dd6/d112/d14a/c14e 0 2024-04-27T04:10:23.787 INFO:tasks.workunit.client.1.smithi181.stdout:9/983: write d0/d44/d4f/df7/f108 [5006267,21768] 0 2024-04-27T04:10:23.788 INFO:tasks.workunit.client.0.smithi162.stdout:3/922: chown d1/d70 168885906 1 2024-04-27T04:10:23.789 INFO:tasks.workunit.client.0.smithi162.stdout:7/991: dwrite d3/d5/d5b/d83/de/d2b/d81/daa/df1/f115 [0,4194304] 0 2024-04-27T04:10:23.791 INFO:tasks.workunit.client.0.smithi162.stdout:8/944: mkdir d0/d10/d2b/d2e/de7/d12b 0 2024-04-27T04:10:23.792 INFO:tasks.workunit.client.1.smithi181.stdout:9/984: mknod d0/d44/d4f/d141/c14e 0 2024-04-27T04:10:23.793 INFO:tasks.workunit.client.0.smithi162.stdout:0/995: dwrite d5/dc/d1e/d46/f55 [4194304,4194304] 0 2024-04-27T04:10:23.794 INFO:tasks.workunit.client.0.smithi162.stdout:6/921: link d6/d1a/d34/d3a/d68/d6c/de3/ffe d6/d1a/d34/d50/f130 0 2024-04-27T04:10:23.798 INFO:tasks.workunit.client.0.smithi162.stdout:3/923: dread - d1/d18/d23/d58/d63/d88/f121 zero size 2024-04-27T04:10:23.798 INFO:tasks.workunit.client.1.smithi181.stdout:9/985: creat d0/d44/d4b/d43/d88/da1/dfc/dbd/f14f x:0 0 0 2024-04-27T04:10:23.800 INFO:tasks.workunit.client.0.smithi162.stdout:5/986: symlink d8/d1a/d7d/d12a/l138 0 2024-04-27T04:10:23.801 INFO:tasks.workunit.client.0.smithi162.stdout:4/885: dwrite d2/da/f49 [0,4194304] 0 2024-04-27T04:10:23.801 INFO:tasks.workunit.client.0.smithi162.stdout:0/996: chown d5/dc/d1e/d46/d48/d9e/dce/dd1/d116/d57 68433 1 2024-04-27T04:10:23.802 INFO:tasks.workunit.client.0.smithi162.stdout:3/924: chown d1/d21/d47/d57/dd6/d112/d132/daa/dab 13584 1 2024-04-27T04:10:23.803 INFO:tasks.workunit.client.0.smithi162.stdout:5/987: stat d8/d1a/d23/dbc/d66/d88 0 2024-04-27T04:10:23.804 INFO:tasks.workunit.client.0.smithi162.stdout:2/933: link d0/d4/d2a/f106 d0/d4/d17/d4f/db0/f127 0 2024-04-27T04:10:23.804 INFO:tasks.workunit.client.0.smithi162.stdout:4/886: write d2/d54/dca/fea [256427,91126] 0 2024-04-27T04:10:23.806 INFO:tasks.workunit.client.0.smithi162.stdout:8/945: symlink d0/d10/d25/d65/d96/l12c 0 2024-04-27T04:10:23.806 INFO:tasks.workunit.client.0.smithi162.stdout:6/922: creat d6/d1a/d34/d3a/dbf/dd7/f131 x:0 0 0 2024-04-27T04:10:23.807 INFO:tasks.workunit.client.0.smithi162.stdout:5/988: fdatasync d8/d1a/d23/dbc/f124 0 2024-04-27T04:10:23.807 INFO:tasks.workunit.client.0.smithi162.stdout:1/875: link d0/d7/d3c/d54/ca1 d0/d11/d50/d81/c11f 0 2024-04-27T04:10:23.810 INFO:tasks.workunit.client.0.smithi162.stdout:6/923: dread - d6/d1a/d34/d50/f130 zero size 2024-04-27T04:10:23.810 INFO:tasks.workunit.client.1.smithi181.stdout:9/986: mkdir d0/d44/d4b/d43/d88/da1/dfc/dbd/df3/d85/dd8/d150 0 2024-04-27T04:10:23.811 INFO:tasks.workunit.client.0.smithi162.stdout:1/876: readlink d0/d11/d50/d81/lf4 0 2024-04-27T04:10:23.812 INFO:tasks.workunit.client.0.smithi162.stdout:0/997: creat d5/dc/d1e/d46/d48/d9e/dce/dd1/d116/ddd/d117/f142 x:0 0 0 2024-04-27T04:10:23.813 INFO:tasks.workunit.client.0.smithi162.stdout:3/925: symlink d1/d21/d47/d57/dd6/d112/d132/daa/dab/l14f 0 2024-04-27T04:10:23.817 INFO:tasks.workunit.client.0.smithi162.stdout:5/989: fsync d8/d1a/d23/d40/f77 0 2024-04-27T04:10:23.817 INFO:tasks.workunit.client.0.smithi162.stdout:9/908: dwrite d3/d13/d2d/d3b/d54/f87 [0,4194304] 0 2024-04-27T04:10:23.818 INFO:tasks.workunit.client.0.smithi162.stdout:8/946: unlink d0/d10/d2b/d2e/l52 0 2024-04-27T04:10:23.819 INFO:tasks.workunit.client.0.smithi162.stdout:2/934: dread d0/d4/d2a/d3b/d49/d5a/d63/d6e/fa7 [0,4194304] 0 2024-04-27T04:10:23.819 INFO:tasks.workunit.client.1.smithi181.stdout:9/987: creat d0/d44/d4b/d43/d88/da1/dfc/dbd/f151 x:0 0 0 2024-04-27T04:10:23.820 INFO:tasks.workunit.client.0.smithi162.stdout:4/887: dwrite d2/d54/d7a/f101 [0,4194304] 0 2024-04-27T04:10:23.822 INFO:tasks.workunit.client.0.smithi162.stdout:6/924: dread - d6/d1a/d34/d7b/d7e/fc4 zero size 2024-04-27T04:10:23.824 INFO:tasks.workunit.client.0.smithi162.stdout:6/925: stat d6/d1a/d34/d7f/d80/db3/dcd/d110/f12d 0 2024-04-27T04:10:23.826 INFO:tasks.workunit.client.1.smithi181.stdout:9/988: dwrite d0/d44/d4b/d5f/f10a [0,4194304] 0 2024-04-27T04:10:23.826 INFO:tasks.workunit.client.0.smithi162.stdout:1/877: mknod d0/d7/d10/d75/db7/c120 0 2024-04-27T04:10:23.828 INFO:tasks.workunit.client.0.smithi162.stdout:6/926: write d6/d1a/d34/d3a/d68/d6c/fda [4219502,127345] 0 2024-04-27T04:10:23.829 INFO:tasks.workunit.client.0.smithi162.stdout:3/926: mkdir d1/d21/d47/d57/dd6/d112/d150 0 2024-04-27T04:10:23.834 INFO:tasks.workunit.client.0.smithi162.stdout:2/935: read d0/d20/fc6 [561444,66936] 0 2024-04-27T04:10:23.836 INFO:tasks.workunit.client.1.smithi181.stdout:9/989: symlink d0/d44/d4b/d43/d89/d11d/l152 0 2024-04-27T04:10:23.840 INFO:tasks.workunit.client.0.smithi162.stdout:7/992: dread d3/d5/d5b/d83/de/d2b/da7/fc2 [0,4194304] 0 2024-04-27T04:10:23.841 INFO:tasks.workunit.client.0.smithi162.stdout:7/993: stat d3/d5/d3c/d4d/f125 0 2024-04-27T04:10:23.842 INFO:tasks.workunit.client.1.smithi181.stdout:9/990: write d0/d44/d4b/d43/d88/da1/dfc/dbd/dd1/dde/d104/f81 [2165567,31037] 0 2024-04-27T04:10:23.844 INFO:tasks.workunit.client.0.smithi162.stdout:0/998: write d5/d26/d4a/f4c [348364,41890] 0 2024-04-27T04:10:23.851 INFO:tasks.workunit.client.0.smithi162.stdout:7/994: link d3/d5/d5b/d83/de/d2b/d81/daa/dbd/dbe/d148/fdf d3/d4f/f14f 0 2024-04-27T04:10:23.854 INFO:tasks.workunit.client.0.smithi162.stdout:7/995: rmdir d3/d5/d5b/d83/de/d2b/d81/d96/d135 39 2024-04-27T04:10:23.856 INFO:tasks.workunit.client.1.smithi181.stdout:9/991: write d0/d44/d4f/f6b [1534340,65347] 0 2024-04-27T04:10:23.856 INFO:tasks.workunit.client.0.smithi162.stdout:3/927: write d1/d18/d23/f2b [240445,29987] 0 2024-04-27T04:10:23.857 INFO:tasks.workunit.client.0.smithi162.stdout:0/999: dwrite d5/dc/d1e/d20/f11e [0,4194304] 0 2024-04-27T04:10:23.859 INFO:tasks.workunit.client.0.smithi162.stdout:5/990: truncate d8/d1a/d23/dbc/d66/d88/dee/ff2 377735 0 2024-04-27T04:10:23.860 INFO:tasks.workunit.client.1.smithi181.stdout:9/992: creat d0/d44/d4b/d43/d88/da1/dfc/dbd/df3/d85/dd8/d150/f153 x:0 0 0 2024-04-27T04:10:23.861 INFO:tasks.workunit.client.1.smithi181.stdout:9/993: mknod d0/d44/d4f/d141/c154 0 2024-04-27T04:10:23.865 INFO:tasks.workunit.client.1.smithi181.stdout:9/994: mknod d0/d44/d4b/d43/d88/da1/dfc/dbd/dd1/dde/d104/c155 0 2024-04-27T04:10:23.866 INFO:tasks.workunit.client.0.smithi162.stdout:4/888: dread d2/d25/d1c/d24/d100/f29 [0,4194304] 0 2024-04-27T04:10:23.868 INFO:tasks.workunit.client.0.smithi162.stdout:7/996: mknod d3/d5/d5b/d83/de/d2b/d81/d139/c150 0 2024-04-27T04:10:23.868 INFO:tasks.workunit.client.0.smithi162.stdout:3/928: fdatasync d1/d18/d5d/d6f/f80 0 2024-04-27T04:10:23.868 INFO:tasks.workunit.client.1.smithi181.stdout:9/995: symlink d0/l156 0 2024-04-27T04:10:23.870 INFO:tasks.workunit.client.0.smithi162.stdout:1/878: dread d0/d80/d89/f8f [0,4194304] 0 2024-04-27T04:10:23.871 INFO:tasks.workunit.client.0.smithi162.stdout:9/909: dwrite d3/d13/d2d/d30/d67/fc6 [0,4194304] 0 2024-04-27T04:10:23.873 INFO:tasks.workunit.client.1.smithi181.stdout:9/996: dread d0/fbe [0,4194304] 0 2024-04-27T04:10:23.874 INFO:tasks.workunit.client.0.smithi162.stdout:5/991: rename d8/d2c to d8/d1a/d23/d40/de4/d128/d139 0 2024-04-27T04:10:23.876 INFO:tasks.workunit.client.0.smithi162.stdout:2/936: dwrite d0/d4/d17/d2b/d64/d89/fd4 [0,4194304] 0 2024-04-27T04:10:23.877 INFO:tasks.workunit.client.1.smithi181.stdout:9/997: creat d0/d44/d4b/d5f/d8a/f157 x:0 0 0 2024-04-27T04:10:23.878 INFO:tasks.workunit.client.0.smithi162.stdout:6/927: dwrite d6/d1a/d34/d3a/fce [0,4194304] 0 2024-04-27T04:10:23.879 INFO:tasks.workunit.client.0.smithi162.stdout:8/947: dread d0/fd [0,4194304] 0 2024-04-27T04:10:23.880 INFO:tasks.workunit.client.0.smithi162.stdout:2/937: readlink d0/d4/d2a/d41/d77/le8 0 2024-04-27T04:10:23.881 INFO:tasks.workunit.client.0.smithi162.stdout:8/948: chown d0/de/d20 101 1 2024-04-27T04:10:23.882 INFO:tasks.workunit.client.0.smithi162.stdout:9/910: creat d3/d13/d15/f117 x:0 0 0 2024-04-27T04:10:23.886 INFO:tasks.workunit.client.0.smithi162.stdout:8/949: read - d0/d10/d25/d65/d96/f114 zero size 2024-04-27T04:10:23.887 INFO:tasks.workunit.client.0.smithi162.stdout:3/929: dwrite d1/d18/d5d/d5f/d66/dd2/da3/f113 [0,4194304] 0 2024-04-27T04:10:23.892 INFO:tasks.workunit.client.0.smithi162.stdout:2/938: read - d0/d20/d51/ff6 zero size 2024-04-27T04:10:23.892 INFO:tasks.workunit.client.0.smithi162.stdout:6/928: creat d6/d1a/d34/d3a/d68/f132 x:0 0 0 2024-04-27T04:10:23.893 INFO:tasks.workunit.client.0.smithi162.stdout:2/939: fdatasync d0/d4/d17/db4/fc7 0 2024-04-27T04:10:23.895 INFO:tasks.workunit.client.0.smithi162.stdout:6/929: chown d6/db/dc0/ff5 494 1 2024-04-27T04:10:23.897 INFO:tasks.workunit.client.0.smithi162.stdout:8/950: dwrite d0/d10/d2b/d2d/d2f/d39/de1/f120 [0,4194304] 0 2024-04-27T04:10:23.898 INFO:tasks.workunit.client.0.smithi162.stdout:6/930: fdatasync d6/d1a/d34/d7b/f83 0 2024-04-27T04:10:23.902 INFO:tasks.workunit.client.0.smithi162.stdout:3/930: rename d1/d21/f2c to d1/d18/d23/d58/d63/dfa/d138/f151 0 2024-04-27T04:10:23.904 INFO:tasks.workunit.client.0.smithi162.stdout:4/889: link d2/d25/d1c/d39/d90/lde d2/l122 0 2024-04-27T04:10:23.905 INFO:tasks.workunit.client.1.smithi181.stdout:9/998: write d0/d44/d4b/f39 [1488089,23067] 0 2024-04-27T04:10:23.907 INFO:tasks.workunit.client.0.smithi162.stdout:7/997: dwrite d3/d4f/f67 [0,4194304] 0 2024-04-27T04:10:23.908 INFO:tasks.workunit.client.0.smithi162.stdout:8/951: creat d0/d10/d44/db2/f12d x:0 0 0 2024-04-27T04:10:23.909 INFO:tasks.workunit.client.1.smithi181.stdout:9/999: creat d0/d44/d4b/d43/d89/f158 x:0 0 0 2024-04-27T04:10:23.910 INFO:tasks.workunit.client.0.smithi162.stdout:2/940: mknod d0/d4/d17/d2b/d64/dbd/c128 0 2024-04-27T04:10:23.912 INFO:tasks.workunit.client.0.smithi162.stdout:3/931: rmdir d1/d21/d47/d57 39 2024-04-27T04:10:23.914 INFO:tasks.workunit.client.1.smithi181.stderr:+ rm -rf -- ./tmp.ppze6CcZx3 2024-04-27T04:10:23.917 INFO:tasks.workunit.client.0.smithi162.stdout:8/952: rename d0/d10/d2b/d2d/d2f/d6b/dc7 to d0/d10/d2b/d2d/d2f/d6b/dd3/d111/d12e 0 2024-04-27T04:10:23.919 INFO:tasks.workunit.client.0.smithi162.stdout:8/953: dread - d0/d10/d76/f124 zero size 2024-04-27T04:10:23.922 INFO:tasks.workunit.client.0.smithi162.stdout:9/911: link d3/d23/cb4 d3/d11/c118 0 2024-04-27T04:10:23.922 INFO:tasks.workunit.client.0.smithi162.stdout:3/932: creat d1/d18/d5d/d5f/d66/d124/f152 x:0 0 0 2024-04-27T04:10:23.925 INFO:tasks.workunit.client.0.smithi162.stdout:2/941: rename d0/d4/f15 to d0/d4/d2a/d3b/d49/d5a/d63/f129 0 2024-04-27T04:10:23.926 INFO:tasks.workunit.client.0.smithi162.stdout:7/998: dwrite d3/d5/d5b/d83/de/d2b/d81/daa/dbd/dbe/d148/ffe [0,4194304] 0 2024-04-27T04:10:23.929 INFO:tasks.workunit.client.0.smithi162.stdout:4/890: dread d2/d25/d1c/d39/d3b/da7/fcb [0,4194304] 0 2024-04-27T04:10:23.930 INFO:tasks.workunit.client.0.smithi162.stdout:2/942: symlink d0/d4/d17/d2b/l12a 0 2024-04-27T04:10:23.932 INFO:tasks.workunit.client.0.smithi162.stdout:9/912: getdents d3/d13/dd2 0 2024-04-27T04:10:23.933 INFO:tasks.workunit.client.0.smithi162.stdout:5/992: rename d8/d9/f21 to d8/de3/f13a 0 2024-04-27T04:10:23.940 INFO:tasks.workunit.client.0.smithi162.stdout:2/943: mknod d0/d4/d2a/da1/dca/c12b 0 2024-04-27T04:10:23.942 INFO:tasks.workunit.client.0.smithi162.stdout:9/913: symlink d3/d13/d2d/d3b/d54/l119 0 2024-04-27T04:10:23.942 INFO:tasks.workunit.client.0.smithi162.stdout:5/993: chown d8/d9/d5b/d96/d12e 43 1 2024-04-27T04:10:23.947 INFO:tasks.workunit.client.0.smithi162.stdout:4/891: dwrite d2/d25/d1c/d106/f118 [0,4194304] 0 2024-04-27T04:10:23.949 INFO:tasks.workunit.client.0.smithi162.stdout:7/999: dwrite d3/d5/d5b/d83/de/d2b/d81/daa/dbd/f13b [0,4194304] 0 2024-04-27T04:10:23.949 INFO:tasks.workunit.client.0.smithi162.stdout:1/879: write d0/d7/d3c/d77/dd1/f106 [1001332,36645] 0 2024-04-27T04:10:23.952 INFO:tasks.workunit.client.0.smithi162.stdout:2/944: write d0/d4/d17/d2b/d64/d89/fc1 [791324,95758] 0 2024-04-27T04:10:23.952 INFO:tasks.workunit.client.0.smithi162.stdout:8/954: dread d0/d10/d44/d70/fee [0,4194304] 0 2024-04-27T04:10:23.953 INFO:tasks.workunit.client.0.smithi162.stdout:3/933: dread d1/d18/d23/fe9 [0,4194304] 0 2024-04-27T04:10:23.958 INFO:tasks.workunit.client.0.smithi162.stdout:3/934: dread - d1/d45/f9c zero size 2024-04-27T04:10:23.961 INFO:tasks.workunit.client.0.smithi162.stdout:6/931: truncate d6/d1a/d34/d3a/d68/d6c/fda 4675819 0 2024-04-27T04:10:23.962 INFO:tasks.workunit.client.0.smithi162.stdout:5/994: dwrite d8/d9/f113 [0,4194304] 0 2024-04-27T04:10:23.965 INFO:tasks.workunit.client.0.smithi162.stdout:4/892: dread d2/d25/d1c/d24/d100/f29 [0,4194304] 0 2024-04-27T04:10:23.966 INFO:tasks.workunit.client.0.smithi162.stdout:2/945: read d0/d4/d2a/d3b/d49/d5a/d63/d84/f99 [1375364,98508] 0 2024-04-27T04:10:23.966 INFO:tasks.workunit.client.0.smithi162.stdout:1/880: mknod d0/d11/d3e/c121 0 2024-04-27T04:10:23.967 INFO:tasks.workunit.client.0.smithi162.stdout:8/955: fsync d0/de/d20/d6c/fb6 0 2024-04-27T04:10:23.967 INFO:tasks.workunit.client.0.smithi162.stdout:4/893: dread - d2/d54/f10b zero size 2024-04-27T04:10:23.968 INFO:tasks.workunit.client.0.smithi162.stdout:3/935: mkdir d1/d18/d23/d58/d63/dfa/d138/d153 0 2024-04-27T04:10:23.970 INFO:tasks.workunit.client.0.smithi162.stdout:6/932: dread - d6/d1a/d34/d3a/fbe zero size 2024-04-27T04:10:23.976 INFO:tasks.workunit.client.0.smithi162.stdout:6/933: unlink d6/d1a/d34/d3a/d68/d6c/l119 0 2024-04-27T04:10:23.980 INFO:tasks.workunit.client.0.smithi162.stdout:5/995: link d8/d1a/d23/d40/c4e d8/d1a/d7d/d12a/c13b 0 2024-04-27T04:10:23.981 INFO:tasks.workunit.client.0.smithi162.stdout:5/996: dread - d8/d9/fb0 zero size 2024-04-27T04:10:23.982 INFO:tasks.workunit.client.0.smithi162.stdout:1/881: link d0/d11/d50/fc3 d0/d11/d50/f122 0 2024-04-27T04:10:23.984 INFO:tasks.workunit.client.0.smithi162.stdout:5/997: dread - d8/d1a/d29/f99 zero size 2024-04-27T04:10:23.984 INFO:tasks.workunit.client.0.smithi162.stdout:9/914: truncate d3/d23/fa2 1346053 0 2024-04-27T04:10:23.987 INFO:tasks.workunit.client.0.smithi162.stdout:1/882: rename d0/d7/d3c/d54/dc9 to d0/d11/d50/d53/d93/d96/daa/d123 0 2024-04-27T04:10:23.989 INFO:tasks.workunit.client.0.smithi162.stdout:1/883: chown d0/d11/d3e/d59/d7f/dc5 0 1 2024-04-27T04:10:23.990 INFO:tasks.workunit.client.0.smithi162.stdout:4/894: write d2/d25/d1c/d39/d90/fcc [323052,80270] 0 2024-04-27T04:10:23.992 INFO:tasks.workunit.client.0.smithi162.stdout:5/998: rename d8/d1a/d23/dbc/d66/d88/f98 to d8/d9/d5b/d96/df4/f13c 0 2024-04-27T04:10:23.996 INFO:tasks.workunit.client.0.smithi162.stdout:6/934: dread d6/db/f6a [0,4194304] 0 2024-04-27T04:10:23.996 INFO:tasks.workunit.client.0.smithi162.stdout:3/936: dwrite d1/f3f [4194304,4194304] 0 2024-04-27T04:10:23.998 INFO:tasks.workunit.client.0.smithi162.stdout:8/956: truncate d0/d10/d25/f36 6233534 0 2024-04-27T04:10:24.003 INFO:tasks.workunit.client.0.smithi162.stdout:4/895: chown d2/f11 487133103 1 2024-04-27T04:10:24.004 INFO:tasks.workunit.client.0.smithi162.stdout:1/884: rename d0/d7/d5e/d10e/c119 to d0/d80/d89/c124 0 2024-04-27T04:10:24.004 INFO:tasks.workunit.client.0.smithi162.stdout:1/885: write d0/d7/d3c/d54/db0/ff9 [477778,10413] 0 2024-04-27T04:10:24.005 INFO:tasks.workunit.client.0.smithi162.stdout:9/915: getdents d3/d13/d15/d50 0 2024-04-27T04:10:24.009 INFO:tasks.workunit.client.0.smithi162.stdout:6/935: fdatasync d6/db/d32/f38 0 2024-04-27T04:10:24.010 INFO:tasks.workunit.client.0.smithi162.stdout:5/999: truncate d8/d1a/d23/d40/de4/d128/d139/f33 288004 0 2024-04-27T04:10:24.012 INFO:tasks.workunit.client.0.smithi162.stdout:4/896: rmdir d2/d25/d1c/d39/d5e 39 2024-04-27T04:10:24.019 INFO:tasks.workunit.client.0.smithi162.stdout:1/886: unlink d0/d11/d3e/d5f/f6a 0 2024-04-27T04:10:24.021 INFO:tasks.workunit.client.0.smithi162.stdout:9/916: fdatasync d3/d13/f41 0 2024-04-27T04:10:24.021 INFO:tasks.workunit.client.0.smithi162.stdout:9/917: stat d3/c62 0 2024-04-27T04:10:24.029 INFO:tasks.workunit.client.0.smithi162.stdout:4/897: dread d2/d25/d1c/d39/d3b/d47/d9c/fb4 [0,4194304] 0 2024-04-27T04:10:24.029 INFO:tasks.workunit.client.0.smithi162.stdout:3/937: dwrite d1/d18/d5d/d5f/d66/dd2/f103 [0,4194304] 0 2024-04-27T04:10:24.036 INFO:tasks.workunit.client.0.smithi162.stdout:8/957: truncate d0/d10/d2b/d2e/dca/fd2 231597 0 2024-04-27T04:10:24.038 INFO:tasks.workunit.client.0.smithi162.stdout:8/958: fdatasync d0/d10/d2b/d2d/d2f/d39/de1/f120 0 2024-04-27T04:10:24.038 INFO:tasks.workunit.client.0.smithi162.stdout:9/918: dread d3/d13/d15/f57 [0,4194304] 0 2024-04-27T04:10:24.041 INFO:tasks.workunit.client.0.smithi162.stdout:4/898: chown d2/d25/d1c/d39/d90/lde 4990124 1 2024-04-27T04:10:24.042 INFO:tasks.workunit.client.0.smithi162.stdout:9/919: write d3/d13/d2d/d30/d7f/fcf [3812086,102980] 0 2024-04-27T04:10:24.043 INFO:tasks.workunit.client.0.smithi162.stdout:6/936: link d6/d1a/d34/d43/fab d6/d1a/d34/d3a/dfd/f133 0 2024-04-27T04:10:24.047 INFO:tasks.workunit.client.0.smithi162.stdout:8/959: unlink d0/d10/d44/d70/fee 0 2024-04-27T04:10:24.049 INFO:tasks.workunit.client.0.smithi162.stdout:4/899: dwrite d2/da/dd/f93 [0,4194304] 0 2024-04-27T04:10:24.057 INFO:tasks.workunit.client.0.smithi162.stdout:4/900: dread d2/d25/d1c/d39/d3b/d47/ffc [0,4194304] 0 2024-04-27T04:10:24.059 INFO:tasks.workunit.client.0.smithi162.stdout:9/920: mkdir d3/db9/dcc/dd8/d11a 0 2024-04-27T04:10:24.061 INFO:tasks.workunit.client.0.smithi162.stdout:8/960: mkdir d0/d12f 0 2024-04-27T04:10:24.063 INFO:tasks.workunit.client.0.smithi162.stdout:1/887: dwrite d0/d7/d3c/fc1 [0,4194304] 0 2024-04-27T04:10:24.063 INFO:tasks.workunit.client.0.smithi162.stdout:3/938: dwrite d1/d18/f2d [0,4194304] 0 2024-04-27T04:10:24.063 INFO:tasks.workunit.client.0.smithi162.stdout:3/939: stat d1/d21/c59 0 2024-04-27T04:10:24.064 INFO:tasks.workunit.client.0.smithi162.stdout:8/961: write d0/d10/d25/d65/d96/f114 [39826,117629] 0 2024-04-27T04:10:24.067 INFO:tasks.workunit.client.0.smithi162.stdout:6/937: truncate d6/f35 789918 0 2024-04-27T04:10:24.069 INFO:tasks.workunit.client.0.smithi162.stdout:9/921: mkdir d3/d13/d2d/d3b/d54/d11b 0 2024-04-27T04:10:24.071 INFO:tasks.workunit.client.0.smithi162.stdout:1/888: read - d0/d7/d3c/d61/dc4/fdb zero size 2024-04-27T04:10:24.072 INFO:tasks.workunit.client.0.smithi162.stdout:4/901: write d2/d25/d1c/d39/f45 [4122050,13618] 0 2024-04-27T04:10:24.084 INFO:tasks.workunit.client.0.smithi162.stdout:9/922: rmdir d3/d13/d2d/d3b 39 2024-04-27T04:10:24.085 INFO:tasks.workunit.client.0.smithi162.stdout:3/940: write d1/ff6 [285397,7865] 0 2024-04-27T04:10:24.087 INFO:tasks.workunit.client.0.smithi162.stdout:9/923: creat d3/d23/d1e/dfa/f11c x:0 0 0 2024-04-27T04:10:24.087 INFO:tasks.workunit.client.0.smithi162.stdout:1/889: symlink d0/d11/d44/d66/d107/d115/l125 0 2024-04-27T04:10:24.088 INFO:tasks.workunit.client.0.smithi162.stdout:1/890: fsync d0/d7/f88 0 2024-04-27T04:10:24.089 INFO:tasks.workunit.client.0.smithi162.stdout:3/941: mkdir d1/d18/d23/d58/d63/dfa/d154 0 2024-04-27T04:10:24.089 INFO:tasks.workunit.client.0.smithi162.stdout:3/942: chown d1/d45/f9c 1940337 1 2024-04-27T04:10:24.096 INFO:tasks.workunit.client.0.smithi162.stdout:9/924: creat d3/d13/d2d/f11d x:0 0 0 2024-04-27T04:10:24.097 INFO:tasks.workunit.client.0.smithi162.stdout:1/891: dwrite d0/d11/fff [0,4194304] 0 2024-04-27T04:10:24.100 INFO:tasks.workunit.client.0.smithi162.stdout:8/962: chown d0/d10/d2b/d2d/d2f/d6b/dd3/cd7 867 1 2024-04-27T04:10:24.100 INFO:tasks.workunit.client.0.smithi162.stdout:1/892: fdatasync d0/d7/d3c/d77/dd1/f106 0 2024-04-27T04:10:24.101 INFO:tasks.workunit.client.0.smithi162.stdout:4/902: dread d2/da/f12 [0,4194304] 0 2024-04-27T04:10:24.103 INFO:tasks.workunit.client.0.smithi162.stdout:9/925: rename d3/d39/fe6 to d3/d11/d45/f11e 0 2024-04-27T04:10:24.105 INFO:tasks.workunit.client.0.smithi162.stdout:6/938: dwrite d6/db/d32/f3b [0,4194304] 0 2024-04-27T04:10:24.108 INFO:tasks.workunit.client.0.smithi162.stdout:8/963: mknod d0/d10/c130 0 2024-04-27T04:10:24.111 INFO:tasks.workunit.client.0.smithi162.stdout:6/939: read - d6/d1a/d34/d7f/dd8/f103 zero size 2024-04-27T04:10:24.114 INFO:tasks.workunit.client.0.smithi162.stdout:9/926: write d3/d13/d2d/d3b/f84 [4993252,124751] 0 2024-04-27T04:10:24.117 INFO:tasks.workunit.client.0.smithi162.stdout:4/903: dwrite d2/da/dd/f93 [0,4194304] 0 2024-04-27T04:10:24.117 INFO:tasks.workunit.client.0.smithi162.stdout:8/964: symlink d0/d10/d2b/d2d/d2f/d6b/dd3/d111/d12e/l131 0 2024-04-27T04:10:24.120 INFO:tasks.workunit.client.0.smithi162.stdout:3/943: dread d1/d45/fe0 [0,4194304] 0 2024-04-27T04:10:24.121 INFO:tasks.workunit.client.0.smithi162.stdout:4/904: truncate d2/d25/d1c/d39/d3b/d6e/d6f/fc7 1293165 0 2024-04-27T04:10:24.128 INFO:tasks.workunit.client.0.smithi162.stdout:4/905: chown d2/d25/d1c/c26 3 1 2024-04-27T04:10:24.128 INFO:tasks.workunit.client.0.smithi162.stdout:1/893: write d0/d7/d10/d75/f10a [400987,48795] 0 2024-04-27T04:10:24.130 INFO:tasks.workunit.client.0.smithi162.stdout:6/940: link d6/d1a/d34/d3a/d68/d6c/c91 d6/d1a/d34/d7f/d80/db3/c134 0 2024-04-27T04:10:24.130 INFO:tasks.workunit.client.0.smithi162.stdout:1/894: readlink d0/d11/d50/d53/d93/d96/daa/lf8 0 2024-04-27T04:10:24.132 INFO:tasks.workunit.client.0.smithi162.stdout:9/927: getdents d3/d13/d2d/d30/d7f/dc3 0 2024-04-27T04:10:24.132 INFO:tasks.workunit.client.0.smithi162.stdout:9/928: chown d3/d13/d2d/d44/l9b 1 1 2024-04-27T04:10:24.135 INFO:tasks.workunit.client.0.smithi162.stdout:4/906: mknod d2/d25/d1c/d39/d70/c123 0 2024-04-27T04:10:24.138 INFO:tasks.workunit.client.0.smithi162.stdout:3/944: write d1/f4c [2914021,20781] 0 2024-04-27T04:10:24.139 INFO:tasks.workunit.client.0.smithi162.stdout:1/895: dread d0/d80/f9f [0,4194304] 0 2024-04-27T04:10:24.140 INFO:tasks.workunit.client.0.smithi162.stdout:3/945: write d1/d18/d5d/d6f/f80 [1976556,49752] 0 2024-04-27T04:10:24.141 INFO:tasks.workunit.client.0.smithi162.stdout:3/946: chown d1/d18/d5d/d6f/fa9 217392 1 2024-04-27T04:10:24.142 INFO:tasks.workunit.client.0.smithi162.stdout:8/965: getdents d0/d10/d44/db2 0 2024-04-27T04:10:24.142 INFO:tasks.workunit.client.0.smithi162.stdout:3/947: stat d1/d18/d5d/ff3 0 2024-04-27T04:10:24.144 INFO:tasks.workunit.client.0.smithi162.stdout:4/907: truncate d2/d25/d1c/d39/d3b/d47/f4c 2315628 0 2024-04-27T04:10:24.144 INFO:tasks.workunit.client.0.smithi162.stdout:3/948: read d1/d18/f4a [4353248,23464] 0 2024-04-27T04:10:24.148 INFO:tasks.workunit.client.0.smithi162.stdout:6/941: rename d6/d1a/d34/d7f/cf1 to d6/d1a/d34/d7f/dd9/de4/df8/c135 0 2024-04-27T04:10:24.149 INFO:tasks.workunit.client.0.smithi162.stdout:6/942: readlink d6/db/l95 0 2024-04-27T04:10:24.149 INFO:tasks.workunit.client.0.smithi162.stdout:8/966: dwrite d0/d10/d2b/d2d/d2f/d39/de1/d121/f125 [0,4194304] 0 2024-04-27T04:10:24.151 INFO:tasks.workunit.client.0.smithi162.stdout:8/967: fdatasync d0/d10/d2b/d2d/fc4 0 2024-04-27T04:10:24.156 INFO:tasks.workunit.client.0.smithi162.stdout:3/949: readlink d1/d21/d47/d57/dd6/d112/d132/dad/l11b 0 2024-04-27T04:10:24.157 INFO:tasks.workunit.client.0.smithi162.stdout:3/950: dread - d1/d18/d5d/d5f/d66/dd2/f116 zero size 2024-04-27T04:10:24.158 INFO:tasks.workunit.client.0.smithi162.stdout:3/951: fdatasync d1/d18/d5d/d6f/f11a 0 2024-04-27T04:10:24.163 INFO:tasks.workunit.client.0.smithi162.stdout:6/943: mkdir d6/d1a/d34/d3a/dfd/d136 0 2024-04-27T04:10:24.169 INFO:tasks.workunit.client.0.smithi162.stdout:3/952: symlink d1/d18/dac/l155 0 2024-04-27T04:10:24.172 INFO:tasks.workunit.client.0.smithi162.stdout:9/929: getdents d3/d13/d15/d7e 0 2024-04-27T04:10:24.174 INFO:tasks.workunit.client.0.smithi162.stdout:8/968: creat d0/d10/d2b/d2e/de7/f132 x:0 0 0 2024-04-27T04:10:24.178 INFO:tasks.workunit.client.0.smithi162.stdout:6/944: unlink d6/c3f 0 2024-04-27T04:10:24.179 INFO:tasks.workunit.client.0.smithi162.stdout:8/969: symlink d0/d10/d25/d7d/d87/l133 0 2024-04-27T04:10:24.180 INFO:tasks.workunit.client.0.smithi162.stdout:3/953: rename d1/d21/d47/d57/cf7 to d1/d18/d23/d58/d63/d88/c156 0 2024-04-27T04:10:24.182 INFO:tasks.workunit.client.0.smithi162.stdout:9/930: link d3/de1/f116 d3/d13/d15/d7e/f11f 0 2024-04-27T04:10:24.182 INFO:tasks.workunit.client.0.smithi162.stdout:3/954: readlink d1/d18/d5d/d5f/d66/dd2/leb 0 2024-04-27T04:10:24.183 INFO:tasks.workunit.client.0.smithi162.stdout:1/896: dwrite d0/d11/d50/fc3 [0,4194304] 0 2024-04-27T04:10:24.185 INFO:tasks.workunit.client.0.smithi162.stdout:1/897: write d0/d7/f27 [2420390,43723] 0 2024-04-27T04:10:24.188 INFO:tasks.workunit.client.0.smithi162.stdout:3/955: read d1/d21/d47/d57/dd6/d112/d132/dad/dc5/fcb [3557386,125660] 0 2024-04-27T04:10:24.197 INFO:tasks.workunit.client.0.smithi162.stdout:6/945: dread d6/d1a/f2b [0,4194304] 0 2024-04-27T04:10:24.199 INFO:tasks.workunit.client.0.smithi162.stdout:6/946: mkdir d6/d137 0 2024-04-27T04:10:24.201 INFO:tasks.workunit.client.0.smithi162.stdout:6/947: truncate d6/d1a/d34/d43/fab 94077 0 2024-04-27T04:10:24.202 INFO:tasks.workunit.client.0.smithi162.stdout:6/948: creat d6/db/f138 x:0 0 0 2024-04-27T04:10:24.210 INFO:tasks.workunit.client.0.smithi162.stdout:6/949: dwrite d6/d1a/d34/d7b/f117 [0,4194304] 0 2024-04-27T04:10:24.210 INFO:tasks.workunit.client.0.smithi162.stdout:6/950: write d6/d1a/d34/d3a/dbf/dd7/f131 [512860,14216] 0 2024-04-27T04:10:24.215 INFO:tasks.workunit.client.0.smithi162.stdout:6/951: readlink d6/d1a/d34/d3a/d68/d6c/de3/leb 0 2024-04-27T04:10:24.220 INFO:tasks.workunit.client.0.smithi162.stdout:6/952: unlink d6/d1a/d34/d3a/d68/d6c/f4f 0 2024-04-27T04:10:24.224 INFO:tasks.workunit.client.0.smithi162.stdout:6/953: mknod d6/d1a/d34/d7b/d7e/c139 0 2024-04-27T04:10:24.233 INFO:tasks.workunit.client.0.smithi162.stdout:6/954: creat d6/d1a/d34/d3a/dbf/d120/f13a x:0 0 0 2024-04-27T04:10:24.234 INFO:tasks.workunit.client.0.smithi162.stdout:8/970: write d0/de/d6e/d71/f95 [3077243,35335] 0 2024-04-27T04:10:24.240 INFO:tasks.workunit.client.0.smithi162.stdout:1/898: dread d0/d7/f27 [0,4194304] 0 2024-04-27T04:10:24.240 INFO:tasks.workunit.client.0.smithi162.stdout:8/971: symlink d0/d10/d2b/d2d/d2f/d6b/dd3/d111/d12e/l134 0 2024-04-27T04:10:24.241 INFO:tasks.workunit.client.0.smithi162.stdout:9/931: dwrite d3/d11/d45/d4e/fbe [0,4194304] 0 2024-04-27T04:10:24.243 INFO:tasks.workunit.client.0.smithi162.stdout:9/932: write d3/d13/d2d/d3b/f84 [5488957,72514] 0 2024-04-27T04:10:24.251 INFO:tasks.workunit.client.0.smithi162.stdout:9/933: creat d3/d13/d2d/d3b/d54/f120 x:0 0 0 2024-04-27T04:10:24.251 INFO:tasks.workunit.client.0.smithi162.stdout:9/934: truncate d3/d23/fd9 1477342 0 2024-04-27T04:10:24.254 INFO:tasks.workunit.client.0.smithi162.stdout:1/899: link d0/d7/d10/f12 d0/d7/d10/d75/db7/dfd/f126 0 2024-04-27T04:10:24.256 INFO:tasks.workunit.client.0.smithi162.stdout:6/955: dwrite d6/d1a/d34/d50/f99 [0,4194304] 0 2024-04-27T04:10:24.260 INFO:tasks.workunit.client.0.smithi162.stdout:1/900: mkdir d0/d11/d50/d53/d93/d96/daa/d127 0 2024-04-27T04:10:24.264 INFO:tasks.workunit.client.0.smithi162.stdout:1/901: write d0/d7/f27 [2943368,123934] 0 2024-04-27T04:10:24.264 INFO:tasks.workunit.client.0.smithi162.stdout:1/902: dread - d0/d7/d5e/f103 zero size 2024-04-27T04:10:24.269 INFO:tasks.workunit.client.0.smithi162.stdout:9/935: write d3/f93 [563764,32762] 0 2024-04-27T04:10:24.269 INFO:tasks.workunit.client.0.smithi162.stdout:6/956: getdents d6/d1a/d34/d7f/d80/db3/dcd/d110 0 2024-04-27T04:10:24.273 INFO:tasks.workunit.client.0.smithi162.stdout:6/957: symlink d6/d1a/d34/d7f/dd9/de4/df8/l13b 0 2024-04-27T04:10:24.275 INFO:tasks.workunit.client.0.smithi162.stdout:1/903: rename d0/d11/d50/d53/d93/d96/daa to d0/d11/d50/d128 0 2024-04-27T04:10:24.278 INFO:tasks.workunit.client.0.smithi162.stdout:6/958: dread d6/d1a/d34/d7f/dd9/de4/ffb [0,4194304] 0 2024-04-27T04:10:24.282 INFO:tasks.workunit.client.0.smithi162.stdout:9/936: dwrite d3/d13/d2d/d30/fa4 [0,4194304] 0 2024-04-27T04:10:24.282 INFO:tasks.workunit.client.0.smithi162.stdout:6/959: creat d6/d1a/d34/d7f/dd9/de4/f13c x:0 0 0 2024-04-27T04:10:24.286 INFO:tasks.workunit.client.0.smithi162.stdout:9/937: dread - d3/d11/d45/d4e/f96 zero size 2024-04-27T04:10:24.290 INFO:tasks.workunit.client.0.smithi162.stdout:6/960: dwrite d6/d1a/d34/d3a/fce [0,4194304] 0 2024-04-27T04:10:24.292 INFO:tasks.workunit.client.0.smithi162.stdout:1/904: getdents d0/d11/d50 0 2024-04-27T04:10:24.299 INFO:tasks.workunit.client.0.smithi162.stdout:6/961: dread d6/d1a/d34/d3a/d68/d6c/f82 [0,4194304] 0 2024-04-27T04:10:24.306 INFO:tasks.workunit.client.0.smithi162.stdout:1/905: getdents d0/d7/d3c/d54 0 2024-04-27T04:10:24.307 INFO:tasks.workunit.client.0.smithi162.stdout:1/906: fsync d0/d11/d50/f110 0 2024-04-27T04:10:24.307 INFO:tasks.workunit.client.0.smithi162.stdout:1/907: stat d0/d11/d3e/f4f 0 2024-04-27T04:10:24.312 INFO:tasks.workunit.client.0.smithi162.stdout:9/938: dread d3/d39/ff6 [0,4194304] 0 2024-04-27T04:10:24.314 INFO:tasks.workunit.client.0.smithi162.stdout:9/939: stat d3/d23/fd3 0 2024-04-27T04:10:24.317 INFO:tasks.workunit.client.0.smithi162.stdout:9/940: unlink d3/d13/d2d/d30/d7f/lc8 0 2024-04-27T04:10:24.356 INFO:tasks.workunit.client.0.smithi162.stdout:9/941: write d3/d13/d2d/d3b/d7a/dd6/f114 [232920,30736] 0 2024-04-27T04:10:24.358 INFO:tasks.workunit.client.0.smithi162.stdout:1/908: dwrite d0/d7/d3c/fe0 [0,4194304] 0 2024-04-27T04:10:24.369 INFO:tasks.workunit.client.0.smithi162.stdout:9/942: symlink d3/d23/d1e/d43/df9/l121 0 2024-04-27T04:10:24.411 INFO:tasks.workunit.client.0.smithi162.stdout:2/946: sync 2024-04-27T04:10:24.411 INFO:tasks.workunit.client.0.smithi162.stdout:1/909: mkdir d0/d11/d3e/d59/d7f/d129 0 2024-04-27T04:10:24.414 INFO:tasks.workunit.client.0.smithi162.stdout:1/910: creat d0/d11/d3e/d5f/f12a x:0 0 0 2024-04-27T04:10:24.414 INFO:tasks.workunit.client.0.smithi162.stdout:2/947: mknod d0/d4/d17/d2b/d64/d96/daa/df9/c12c 0 2024-04-27T04:10:24.417 INFO:tasks.workunit.client.0.smithi162.stdout:1/911: symlink d0/d11/d50/d128/dfa/l12b 0 2024-04-27T04:10:24.417 INFO:tasks.workunit.client.0.smithi162.stdout:2/948: write d0/d4/d17/d2b/d64/d96/fea [2740972,34054] 0 2024-04-27T04:10:24.420 INFO:tasks.workunit.client.0.smithi162.stdout:9/943: dwrite d3/d13/d2d/fab [0,4194304] 0 2024-04-27T04:10:24.424 INFO:tasks.workunit.client.0.smithi162.stdout:9/944: mknod d3/d11/c122 0 2024-04-27T04:10:24.427 INFO:tasks.workunit.client.0.smithi162.stdout:1/912: rename d0/d7/d3c/d77/l76 to d0/d11/d3e/l12c 0 2024-04-27T04:10:24.427 INFO:tasks.workunit.client.0.smithi162.stdout:2/949: rename d0/d4/d17/d2b to d0/d4/d17/d2b/d64/d96/daa/df9/dfa/d12d 22 2024-04-27T04:10:24.431 INFO:tasks.workunit.client.0.smithi162.stdout:1/913: rename d0/d11/d3e/d59/c7e to d0/d80/c12d 0 2024-04-27T04:10:24.432 INFO:tasks.workunit.client.0.smithi162.stdout:1/914: readlink d0/d7/d2a/ldc 0 2024-04-27T04:10:24.432 INFO:tasks.workunit.client.0.smithi162.stdout:2/950: dwrite d0/d4/d17/d2b/d64/d96/feb [0,4194304] 0 2024-04-27T04:10:24.445 INFO:tasks.workunit.client.0.smithi162.stdout:1/915: getdents d0/d11/d50/d81 0 2024-04-27T04:10:24.445 INFO:tasks.workunit.client.0.smithi162.stdout:2/951: write d0/d4/d2a/d41/d77/d73/dc3/f76 [4219229,23702] 0 2024-04-27T04:10:24.480 INFO:tasks.workunit.client.0.smithi162.stdout:1/916: rmdir d0/d7/d10/d75/db7/dfd 39 2024-04-27T04:10:24.480 INFO:tasks.workunit.client.0.smithi162.stdout:1/917: stat d0/d7 0 2024-04-27T04:10:24.485 INFO:tasks.workunit.client.0.smithi162.stdout:1/918: dwrite d0/d11/d50/f122 [0,4194304] 0 2024-04-27T04:10:24.486 INFO:tasks.workunit.client.0.smithi162.stdout:1/919: fsync d0/d11/d50/d128/fce 0 2024-04-27T04:10:24.513 INFO:tasks.workunit.client.0.smithi162.stdout:2/952: dwrite d0/d4/d17/d4f/db0/f127 [0,4194304] 0 2024-04-27T04:10:24.514 INFO:tasks.workunit.client.0.smithi162.stdout:1/920: dwrite d0/d7/d3c/d77/dd1/dd5/f116 [4194304,4194304] 0 2024-04-27T04:10:24.515 INFO:tasks.workunit.client.0.smithi162.stdout:1/921: stat d0/d11/d50/f122 0 2024-04-27T04:10:24.516 INFO:tasks.workunit.client.0.smithi162.stdout:2/953: fdatasync d0/d4/d17/d2b/d64/d89/dbc/f120 0 2024-04-27T04:10:24.528 INFO:tasks.workunit.client.0.smithi162.stdout:1/922: symlink d0/d7/d3c/d54/l12e 0 2024-04-27T04:10:24.530 INFO:tasks.workunit.client.0.smithi162.stdout:2/954: unlink d0/d20/c3d 0 2024-04-27T04:10:24.536 INFO:tasks.workunit.client.0.smithi162.stdout:2/955: getdents d0/d4/d17/d2b/d64/d96/daa/df9 0 2024-04-27T04:10:24.538 INFO:tasks.workunit.client.0.smithi162.stdout:4/908: sync 2024-04-27T04:10:24.538 INFO:tasks.workunit.client.0.smithi162.stdout:2/956: mknod d0/d4/c12e 0 2024-04-27T04:10:24.539 INFO:tasks.workunit.client.0.smithi162.stdout:2/957: chown d0/d4/d2a/d3b/l48 1689 1 2024-04-27T04:10:24.547 INFO:tasks.workunit.client.0.smithi162.stdout:2/958: mknod d0/d4/d2a/d3b/d49/d5a/c12f 0 2024-04-27T04:10:24.551 INFO:tasks.workunit.client.0.smithi162.stdout:4/909: truncate d2/d25/d1c/d39/d3b/d47/fc3 2173183 0 2024-04-27T04:10:24.552 INFO:tasks.workunit.client.0.smithi162.stdout:4/910: write d2/da/dd/fe1 [1967865,128969] 0 2024-04-27T04:10:24.559 INFO:tasks.workunit.client.0.smithi162.stdout:1/923: creat d0/d7/d3c/d54/dae/d10c/d118/f12f x:0 0 0 2024-04-27T04:10:24.563 INFO:tasks.workunit.client.0.smithi162.stdout:2/959: link d0/d4/d17/db4/fc7 d0/d4/d17/d2b/d112/f130 0 2024-04-27T04:10:24.565 INFO:tasks.workunit.client.0.smithi162.stdout:2/960: dread d0/d4/d2a/d3b/d49/d5a/d63/d6e/fa7 [0,4194304] 0 2024-04-27T04:10:24.566 INFO:tasks.workunit.client.0.smithi162.stdout:4/911: unlink d2/d25/d1c/d24/l63 0 2024-04-27T04:10:24.568 INFO:tasks.workunit.client.0.smithi162.stdout:4/912: dread d2/da/f12 [0,4194304] 0 2024-04-27T04:10:24.568 INFO:tasks.workunit.client.0.smithi162.stdout:1/924: fsync d0/d7/d3c/d54/f60 0 2024-04-27T04:10:24.570 INFO:tasks.workunit.client.0.smithi162.stdout:4/913: read d2/d25/d1c/d39/f46 [8089300,31067] 0 2024-04-27T04:10:24.572 INFO:tasks.workunit.client.0.smithi162.stdout:4/914: chown d2/d54/dca/lce 23578 1 2024-04-27T04:10:24.577 INFO:tasks.workunit.client.0.smithi162.stdout:4/915: creat d2/d25/d1c/d39/d3b/d6e/d6f/f124 x:0 0 0 2024-04-27T04:10:24.579 INFO:tasks.workunit.client.0.smithi162.stdout:2/961: write d0/d4/d2a/d41/fa2 [1102194,89898] 0 2024-04-27T04:10:24.580 INFO:tasks.workunit.client.0.smithi162.stdout:2/962: write d0/d4/d2a/d41/fa2 [971936,5616] 0 2024-04-27T04:10:24.581 INFO:tasks.workunit.client.0.smithi162.stdout:2/963: chown d0/fe6 447 1 2024-04-27T04:10:24.587 INFO:tasks.workunit.client.0.smithi162.stdout:1/925: creat d0/d11/d3e/d59/d7f/f130 x:0 0 0 2024-04-27T04:10:24.599 INFO:tasks.workunit.client.0.smithi162.stdout:4/916: getdents d2/d25/d1c/d24/d100 0 2024-04-27T04:10:24.604 INFO:tasks.workunit.client.0.smithi162.stdout:1/926: creat d0/d7/d3c/d54/f131 x:0 0 0 2024-04-27T04:10:24.605 INFO:tasks.workunit.client.0.smithi162.stdout:2/964: dread d0/f1 [0,4194304] 0 2024-04-27T04:10:24.608 INFO:tasks.workunit.client.0.smithi162.stdout:4/917: chown d2/d25/d1c/d39/d3b/d6e/l74 43326461 1 2024-04-27T04:10:24.609 INFO:tasks.workunit.client.0.smithi162.stdout:4/918: read - d2/d25/d1c/d39/d3b/d47/fb9 zero size 2024-04-27T04:10:24.612 INFO:tasks.workunit.client.0.smithi162.stdout:3/956: sync 2024-04-27T04:10:24.612 INFO:tasks.workunit.client.0.smithi162.stdout:8/972: sync 2024-04-27T04:10:24.613 INFO:tasks.workunit.client.0.smithi162.stdout:9/945: sync 2024-04-27T04:10:24.613 INFO:tasks.workunit.client.0.smithi162.stdout:6/962: sync 2024-04-27T04:10:24.617 INFO:tasks.workunit.client.0.smithi162.stdout:4/919: symlink d2/d25/d1c/d39/d3b/d6e/dd5/l125 0 2024-04-27T04:10:24.623 INFO:tasks.workunit.client.0.smithi162.stdout:2/965: truncate d0/d4/d17/d2b/d64/f90 3238787 0 2024-04-27T04:10:24.623 INFO:tasks.workunit.client.0.smithi162.stdout:6/963: dread d6/d1a/f88 [0,4194304] 0 2024-04-27T04:10:24.623 INFO:tasks.workunit.client.0.smithi162.stdout:8/973: rename d0/d10/d25/f93 to d0/d10/d2b/d2d/d2f/d39/f135 0 2024-04-27T04:10:24.623 INFO:tasks.workunit.client.0.smithi162.stdout:1/927: dwrite d0/d11/d3e/f41 [0,4194304] 0 2024-04-27T04:10:24.623 INFO:tasks.workunit.client.0.smithi162.stdout:3/957: dwrite d1/f4c [0,4194304] 0 2024-04-27T04:10:24.625 INFO:tasks.workunit.client.0.smithi162.stdout:4/920: rmdir d2/d25/d1c/d39/d3b/d47/d9c 39 2024-04-27T04:10:24.629 INFO:tasks.workunit.client.0.smithi162.stdout:9/946: dwrite d3/d13/d2d/d3b/d54/fe0 [0,4194304] 0 2024-04-27T04:10:24.632 INFO:tasks.workunit.client.0.smithi162.stdout:8/974: symlink d0/d10/d44/db2/l136 0 2024-04-27T04:10:24.638 INFO:tasks.workunit.client.0.smithi162.stdout:4/921: chown d2/d25/d1c/d39/d5e 2544 1 2024-04-27T04:10:24.642 INFO:tasks.workunit.client.0.smithi162.stdout:3/958: unlink d1/d18/d5d/d5f/d66/dd2/f103 0 2024-04-27T04:10:24.643 INFO:tasks.workunit.client.0.smithi162.stdout:3/959: readlink d1/d18/dac/l155 0 2024-04-27T04:10:24.644 INFO:tasks.workunit.client.0.smithi162.stdout:9/947: creat d3/d11/d45/f123 x:0 0 0 2024-04-27T04:10:24.646 INFO:tasks.workunit.client.0.smithi162.stdout:8/975: mkdir d0/d10/d2b/d2e/d127/d137 0 2024-04-27T04:10:24.648 INFO:tasks.workunit.client.0.smithi162.stdout:3/960: truncate d1/d18/d23/f97 929860 0 2024-04-27T04:10:24.648 INFO:tasks.workunit.client.0.smithi162.stdout:9/948: read - d3/d11/d45/fa1 zero size 2024-04-27T04:10:24.649 INFO:tasks.workunit.client.0.smithi162.stdout:2/966: getdents d0/d20/d51/ddd 0 2024-04-27T04:10:24.650 INFO:tasks.workunit.client.0.smithi162.stdout:3/961: mknod d1/d18/d4f/c157 0 2024-04-27T04:10:24.650 INFO:tasks.workunit.client.0.smithi162.stdout:4/922: dread d2/d25/fd7 [0,4194304] 0 2024-04-27T04:10:24.653 INFO:tasks.workunit.client.0.smithi162.stdout:3/962: rmdir d1/d18/d23/d58/d63/dd5 39 2024-04-27T04:10:24.653 INFO:tasks.workunit.client.0.smithi162.stdout:6/964: dread d6/f1e [4194304,4194304] 0 2024-04-27T04:10:24.656 INFO:tasks.workunit.client.0.smithi162.stdout:2/967: read d0/d4/f16 [3275129,65174] 0 2024-04-27T04:10:24.658 INFO:tasks.workunit.client.0.smithi162.stdout:8/976: dwrite d0/d10/d2b/f115 [0,4194304] 0 2024-04-27T04:10:24.658 INFO:tasks.workunit.client.0.smithi162.stdout:3/963: rmdir d1/d21/d47/d57/dd6/d112/d132/daa 39 2024-04-27T04:10:24.659 INFO:tasks.workunit.client.0.smithi162.stdout:1/928: write d0/d7/d3c/f43 [3609645,32944] 0 2024-04-27T04:10:24.660 INFO:tasks.workunit.client.0.smithi162.stdout:2/968: write d0/d4/d17/d88/f124 [916757,70489] 0 2024-04-27T04:10:24.664 INFO:tasks.workunit.client.0.smithi162.stdout:4/923: dread d2/d54/f35 [0,4194304] 0 2024-04-27T04:10:24.667 INFO:tasks.workunit.client.0.smithi162.stdout:3/964: rmdir d1/d18/d23/d58/d63/dfa/d138 39 2024-04-27T04:10:24.672 INFO:tasks.workunit.client.0.smithi162.stdout:2/969: dread - d0/d4/d17/d2b/d64/d96/fee zero size 2024-04-27T04:10:24.675 INFO:tasks.workunit.client.0.smithi162.stdout:4/924: mkdir d2/d25/d1c/d39/d3b/d6e/dd5/d126 0 2024-04-27T04:10:24.678 INFO:tasks.workunit.client.0.smithi162.stdout:6/965: dwrite d6/d1a/d34/fca [0,4194304] 0 2024-04-27T04:10:24.681 INFO:tasks.workunit.client.0.smithi162.stdout:1/929: creat d0/d7/f132 x:0 0 0 2024-04-27T04:10:24.683 INFO:tasks.workunit.client.0.smithi162.stdout:1/930: write d0/d7/d3c/d6e/f71 [2096837,80235] 0 2024-04-27T04:10:24.685 INFO:tasks.workunit.client.0.smithi162.stdout:2/970: mknod d0/d4/d2a/da1/dca/c131 0 2024-04-27T04:10:24.687 INFO:tasks.workunit.client.0.smithi162.stdout:8/977: write d0/d10/d25/d65/d96/dcd/ffa [2448650,29935] 0 2024-04-27T04:10:24.688 INFO:tasks.workunit.client.0.smithi162.stdout:2/971: dread d0/d4/d17/d2b/d64/d96/fac [0,4194304] 0 2024-04-27T04:10:24.690 INFO:tasks.workunit.client.0.smithi162.stdout:6/966: creat d6/d1a/d34/d3a/dbf/f13d x:0 0 0 2024-04-27T04:10:24.694 INFO:tasks.workunit.client.0.smithi162.stdout:1/931: truncate d0/d7/d3c/f97 244533 0 2024-04-27T04:10:24.695 INFO:tasks.workunit.client.0.smithi162.stdout:4/925: write d2/d25/d1c/d39/d3b/d47/d9c/fb4 [5233898,86623] 0 2024-04-27T04:10:24.698 INFO:tasks.workunit.client.0.smithi162.stdout:3/965: rename d1/d18/d5d to d1/d21/d47/d158 0 2024-04-27T04:10:24.699 INFO:tasks.workunit.client.0.smithi162.stdout:3/966: dread - d1/d21/d47/d158/d6f/f11a zero size 2024-04-27T04:10:24.701 INFO:tasks.workunit.client.0.smithi162.stdout:2/972: creat d0/d4/d2a/d3b/d49/f132 x:0 0 0 2024-04-27T04:10:24.702 INFO:tasks.workunit.client.0.smithi162.stdout:6/967: rmdir d6/d1a/d34/d7f/d80/db3 39 2024-04-27T04:10:24.702 INFO:tasks.workunit.client.0.smithi162.stdout:2/973: stat d0/d4/d17/d4f/c9c 0 2024-04-27T04:10:24.703 INFO:tasks.workunit.client.0.smithi162.stdout:1/932: mknod d0/d7/d3c/d77/dd1/c133 0 2024-04-27T04:10:24.703 INFO:tasks.workunit.client.0.smithi162.stdout:1/933: fsync d0/d11/d5b/f117 0 2024-04-27T04:10:24.706 INFO:tasks.workunit.client.0.smithi162.stdout:8/978: link d0/f4 d0/d10/d2b/d2d/d2f/d39/f138 0 2024-04-27T04:10:24.709 INFO:tasks.workunit.client.0.smithi162.stdout:2/974: chown d0/d4/d2a/l72 0 1 2024-04-27T04:10:24.711 INFO:tasks.workunit.client.0.smithi162.stdout:4/926: fdatasync d2/d25/d1c/d39/d3b/d47/fc3 0 2024-04-27T04:10:24.712 INFO:tasks.workunit.client.0.smithi162.stdout:4/927: readlink d2/d25/d1c/d39/l77 0 2024-04-27T04:10:24.717 INFO:tasks.workunit.client.0.smithi162.stdout:3/967: mknod d1/d21/d47/d57/dd6/d112/d132/daa/c159 0 2024-04-27T04:10:24.721 INFO:tasks.workunit.client.0.smithi162.stdout:6/968: mknod d6/d1a/d34/d7f/d80/db3/c13e 0 2024-04-27T04:10:24.724 INFO:tasks.workunit.client.0.smithi162.stdout:4/928: creat d2/da/dd/f127 x:0 0 0 2024-04-27T04:10:24.731 INFO:tasks.workunit.client.0.smithi162.stdout:1/934: creat d0/d11/f134 x:0 0 0 2024-04-27T04:10:24.732 INFO:tasks.workunit.client.0.smithi162.stdout:2/975: dwrite d0/d4/d2a/f3e [0,4194304] 0 2024-04-27T04:10:24.733 INFO:tasks.workunit.client.0.smithi162.stdout:4/929: rename d2/d25/fd7 to d2/d25/d1c/d39/def/f128 0 2024-04-27T04:10:24.734 INFO:tasks.workunit.client.0.smithi162.stdout:4/930: stat d2/d8a/l7c 0 2024-04-27T04:10:24.743 INFO:tasks.workunit.client.0.smithi162.stdout:3/968: mknod d1/d21/d47/d158/d5f/d66/dd2/d149/c15a 0 2024-04-27T04:10:24.743 INFO:tasks.workunit.client.0.smithi162.stdout:8/979: creat d0/f139 x:0 0 0 2024-04-27T04:10:24.743 INFO:tasks.workunit.client.0.smithi162.stdout:2/976: unlink d0/d4/c71 0 2024-04-27T04:10:24.743 INFO:tasks.workunit.client.0.smithi162.stdout:4/931: mkdir d2/da/dd4/d129 0 2024-04-27T04:10:24.745 INFO:tasks.workunit.client.0.smithi162.stdout:2/977: chown d0/d4/d2a/d3b/d49/d5a/d63/d84 1022310945 1 2024-04-27T04:10:24.750 INFO:tasks.workunit.client.0.smithi162.stdout:1/935: link d0/f3b d0/d7/d3c/d61/f135 0 2024-04-27T04:10:24.751 INFO:tasks.workunit.client.0.smithi162.stdout:2/978: dwrite d0/d4/d17/d4f/d10c/ffe [0,4194304] 0 2024-04-27T04:10:24.755 INFO:tasks.workunit.client.0.smithi162.stdout:3/969: rename d1/d21/d47/d158/d5f/d66/dd2/f10a to d1/d21/d47/d57/dd6/d112/d132/dad/dc5/f15b 0 2024-04-27T04:10:24.759 INFO:tasks.workunit.client.0.smithi162.stdout:1/936: creat d0/d11/d50/d53/f136 x:0 0 0 2024-04-27T04:10:24.759 INFO:tasks.workunit.client.0.smithi162.stdout:3/970: write d1/d21/d47/d158/d6f/f11a [287531,47521] 0 2024-04-27T04:10:24.760 INFO:tasks.workunit.client.0.smithi162.stdout:4/932: write d2/d25/f36 [4546790,49650] 0 2024-04-27T04:10:24.761 INFO:tasks.workunit.client.0.smithi162.stdout:2/979: dwrite d0/d24/f38 [0,4194304] 0 2024-04-27T04:10:24.767 INFO:tasks.workunit.client.0.smithi162.stdout:8/980: dread d0/d10/d76/fb7 [0,4194304] 0 2024-04-27T04:10:24.769 INFO:tasks.workunit.client.0.smithi162.stdout:1/937: link d0/d11/d50/d53/f84 d0/d11/d50/d128/de9/f137 0 2024-04-27T04:10:24.770 INFO:tasks.workunit.client.0.smithi162.stdout:3/971: dwrite d1/d21/d47/d158/d5f/d66/dd2/f116 [0,4194304] 0 2024-04-27T04:10:24.780 INFO:tasks.workunit.client.0.smithi162.stdout:8/981: link d0/d10/d2b/d2e/de7/f132 d0/de/d20/d6c/f13a 0 2024-04-27T04:10:24.781 INFO:tasks.workunit.client.0.smithi162.stdout:4/933: write d2/d25/d1c/d39/d5e/fd8 [267048,8710] 0 2024-04-27T04:10:24.781 INFO:tasks.workunit.client.0.smithi162.stdout:2/980: write d0/d4/d17/d102/f103 [2461735,54630] 0 2024-04-27T04:10:24.783 INFO:tasks.workunit.client.0.smithi162.stdout:2/981: write d0/d4/d17/f10e [161689,23611] 0 2024-04-27T04:10:24.788 INFO:tasks.workunit.client.0.smithi162.stdout:8/982: truncate d0/d10/d25/f36 3684380 0 2024-04-27T04:10:24.791 INFO:tasks.workunit.client.0.smithi162.stdout:3/972: getdents d1/d21/d47/d158/d5f/d66/dd2/d149 0 2024-04-27T04:10:24.792 INFO:tasks.workunit.client.0.smithi162.stdout:1/938: truncate d0/d7/d10/d75/f10a 5133383 0 2024-04-27T04:10:24.793 INFO:tasks.workunit.client.0.smithi162.stdout:2/982: creat d0/d4/f133 x:0 0 0 2024-04-27T04:10:24.794 INFO:tasks.workunit.client.0.smithi162.stdout:2/983: chown d0/d4/d2a/l72 92698632 1 2024-04-27T04:10:24.796 INFO:tasks.workunit.client.0.smithi162.stdout:3/973: mkdir d1/d21/d47/d158/d5f/d66/dd2/d149/d15c 0 2024-04-27T04:10:24.805 INFO:tasks.workunit.client.0.smithi162.stdout:4/934: link d2/d25/d1c/d39/d3b/d6e/l10d d2/d25/d1c/d39/d5e/l12a 0 2024-04-27T04:10:24.805 INFO:tasks.workunit.client.0.smithi162.stdout:8/983: dwrite d0/fd [0,4194304] 0 2024-04-27T04:10:24.807 INFO:tasks.workunit.client.0.smithi162.stdout:2/984: creat d0/d4/d2a/d41/d77/d73/dc3/d7d/f134 x:0 0 0 2024-04-27T04:10:24.810 INFO:tasks.workunit.client.0.smithi162.stdout:3/974: truncate d1/d21/d47/d158/d5f/fca 3611080 0 2024-04-27T04:10:24.814 INFO:tasks.workunit.client.0.smithi162.stdout:1/939: unlink d0/d7/d3c/d54/fa4 0 2024-04-27T04:10:24.814 INFO:tasks.workunit.client.0.smithi162.stdout:4/935: dwrite d2/d25/d1c/d24/dd2/df7/f117 [0,4194304] 0 2024-04-27T04:10:24.814 INFO:tasks.workunit.client.0.smithi162.stdout:4/936: chown d2/d8a 106 1 2024-04-27T04:10:24.816 INFO:tasks.workunit.client.0.smithi162.stdout:1/940: write d0/d11/d50/f122 [15945,60648] 0 2024-04-27T04:10:24.819 INFO:tasks.workunit.client.0.smithi162.stdout:1/941: write d0/d7/d2a/f31 [3811607,107167] 0 2024-04-27T04:10:24.826 INFO:tasks.workunit.client.0.smithi162.stdout:4/937: dwrite d2/d25/d1c/d24/f6d [0,4194304] 0 2024-04-27T04:10:24.827 INFO:tasks.workunit.client.0.smithi162.stdout:4/938: read d2/d25/d1c/d106/f118 [434591,58350] 0 2024-04-27T04:10:24.831 INFO:tasks.workunit.client.0.smithi162.stdout:9/949: sync 2024-04-27T04:10:24.833 INFO:tasks.workunit.client.0.smithi162.stdout:3/975: mkdir d1/d18/d23/d58/d63/dfa/d15d 0 2024-04-27T04:10:24.834 INFO:tasks.workunit.client.0.smithi162.stdout:2/985: dread d0/d4/d17/d2b/d64/d89/fcc [0,4194304] 0 2024-04-27T04:10:24.839 INFO:tasks.workunit.client.0.smithi162.stdout:9/950: rename d3/d13/d2d/d30/l40 to d3/d13/dd2/l124 0 2024-04-27T04:10:24.843 INFO:tasks.workunit.client.0.smithi162.stdout:1/942: rmdir d0/d7/d3c/d54 39 2024-04-27T04:10:24.848 INFO:tasks.workunit.client.0.smithi162.stdout:4/939: link d2/d25/d1c/d39/d3b/d47/dfb/l108 d2/d25/d115/l12b 0 2024-04-27T04:10:24.851 INFO:tasks.workunit.client.0.smithi162.stdout:2/986: dwrite d0/d4/f16 [0,4194304] 0 2024-04-27T04:10:24.851 INFO:tasks.workunit.client.0.smithi162.stdout:8/984: dwrite d0/d10/d2b/d2e/f40 [0,4194304] 0 2024-04-27T04:10:24.853 INFO:tasks.workunit.client.0.smithi162.stdout:1/943: write d0/d7/d3c/d54/dae/d10c/d118/f12f [830818,126322] 0 2024-04-27T04:10:24.854 INFO:tasks.workunit.client.0.smithi162.stdout:1/944: fdatasync d0/d7/d3c/f105 0 2024-04-27T04:10:24.860 INFO:tasks.workunit.client.0.smithi162.stdout:2/987: rename d0/d4/d2a/f3e to d0/d4/d17/d121/f135 0 2024-04-27T04:10:24.861 INFO:tasks.workunit.client.0.smithi162.stdout:1/945: dwrite d0/d11/f20 [4194304,4194304] 0 2024-04-27T04:10:24.862 INFO:tasks.workunit.client.0.smithi162.stdout:1/946: fsync d0/d7/f88 0 2024-04-27T04:10:24.864 INFO:tasks.workunit.client.0.smithi162.stdout:1/947: chown d0/d11/d50/f56 6273542 1 2024-04-27T04:10:24.867 INFO:tasks.workunit.client.0.smithi162.stdout:2/988: read d0/d4/d2a/d41/d77/f83 [172906,14083] 0 2024-04-27T04:10:24.871 INFO:tasks.workunit.client.0.smithi162.stdout:3/976: dwrite d1/d18/f41 [4194304,4194304] 0 2024-04-27T04:10:24.889 INFO:tasks.workunit.client.0.smithi162.stdout:6/969: sync 2024-04-27T04:10:24.903 INFO:tasks.workunit.client.0.smithi162.stdout:3/977: dread d1/d18/f13 [0,4194304] 0 2024-04-27T04:10:24.927 INFO:tasks.workunit.client.0.smithi162.stdout:8/985: rename d0/d10/d2b/d2e/de7/l11c to d0/de/d20/d7e/l13b 0 2024-04-27T04:10:24.930 INFO:tasks.workunit.client.0.smithi162.stdout:9/951: dwrite d3/d13/d2d/d30/f38 [0,4194304] 0 2024-04-27T04:10:24.993 INFO:tasks.workunit.client.0.smithi162.stdout:2/989: unlink d0/d4/d17/d4f/d10c/ld9 0 2024-04-27T04:10:25.004 INFO:tasks.workunit.client.0.smithi162.stdout:6/970: symlink d6/d1a/d34/d3a/dfd/l13f 0 2024-04-27T04:10:25.005 INFO:tasks.workunit.client.0.smithi162.stdout:6/971: stat d6/d1a/d34/d43/c9a 0 2024-04-27T04:10:25.009 INFO:tasks.workunit.client.0.smithi162.stdout:8/986: symlink d0/d10/d44/db2/l13c 0 2024-04-27T04:10:25.016 INFO:tasks.workunit.client.0.smithi162.stdout:2/990: rmdir d0/d4/d2a/da1 39 2024-04-27T04:10:25.017 INFO:tasks.workunit.client.0.smithi162.stdout:2/991: write d0/d20/d51/ff6 [824776,30570] 0 2024-04-27T04:10:25.017 INFO:tasks.workunit.client.0.smithi162.stdout:4/940: link d2/d25/d1c/d39/d3b/d6e/d6f/fc6 d2/d25/d1c/d39/def/f12c 0 2024-04-27T04:10:25.029 INFO:tasks.workunit.client.0.smithi162.stdout:8/987: mkdir d0/d10/d25/d65/d128/d13d 0 2024-04-27T04:10:25.033 INFO:tasks.workunit.client.0.smithi162.stdout:2/992: unlink d0/d4/d2a/d3b/d49/d5a/d63/d80/c119 0 2024-04-27T04:10:25.034 INFO:tasks.workunit.client.0.smithi162.stdout:3/978: sync 2024-04-27T04:10:25.036 INFO:tasks.workunit.client.0.smithi162.stdout:8/988: stat d0/d10/d2b/c68 0 2024-04-27T04:10:25.037 INFO:tasks.workunit.client.0.smithi162.stdout:9/952: link d3/c102 d3/db9/dcc/dd8/d11a/c125 0 2024-04-27T04:10:25.040 INFO:tasks.workunit.client.0.smithi162.stdout:3/979: mknod d1/d65/dee/c15e 0 2024-04-27T04:10:25.054 INFO:tasks.workunit.client.0.smithi162.stdout:2/993: dread d0/d4/d2a/d41/d77/f7c [0,4194304] 0 2024-04-27T04:10:25.057 INFO:tasks.workunit.client.0.smithi162.stdout:2/994: symlink d0/d4/d17/d2b/d64/d89/dbc/l136 0 2024-04-27T04:10:25.061 INFO:tasks.workunit.client.0.smithi162.stdout:2/995: getdents d0/d4/d17/d2b 0 2024-04-27T04:10:25.155 INFO:tasks.workunit.client.0.smithi162.stdout:4/941: truncate d2/d25/d1c/d39/d3b/d6e/d6f/fc7 113440 0 2024-04-27T04:10:25.156 INFO:tasks.workunit.client.0.smithi162.stdout:8/989: write d0/d10/d44/d70/fa9 [376519,28817] 0 2024-04-27T04:10:25.157 INFO:tasks.workunit.client.0.smithi162.stdout:3/980: write d1/d18/f1a [5062576,13260] 0 2024-04-27T04:10:25.159 INFO:tasks.workunit.client.0.smithi162.stdout:4/942: link d2/d25/d1c/d24/d100/de9/ff6 d2/d25/d1c/d39/d3b/f12d 0 2024-04-27T04:10:25.162 INFO:tasks.workunit.client.0.smithi162.stdout:3/981: unlink d1/d21/d47/ce4 0 2024-04-27T04:10:25.163 INFO:tasks.workunit.client.0.smithi162.stdout:3/982: chown d1/d21/d47/d158/d5f/d66/d124 2184 1 2024-04-27T04:10:25.168 INFO:tasks.workunit.client.0.smithi162.stdout:4/943: mknod d2/da/d114/c12e 0 2024-04-27T04:10:25.169 INFO:tasks.workunit.client.0.smithi162.stdout:3/983: dwrite d1/d18/d23/f28 [4194304,4194304] 0 2024-04-27T04:10:25.171 INFO:tasks.workunit.client.0.smithi162.stdout:3/984: write d1/d18/d23/d58/d63/d88/fe8 [2125933,31902] 0 2024-04-27T04:10:25.177 INFO:tasks.workunit.client.0.smithi162.stdout:4/944: dwrite d2/f10f [0,4194304] 0 2024-04-27T04:10:25.179 INFO:tasks.workunit.client.0.smithi162.stdout:4/945: readlink d2/d25/la8 0 2024-04-27T04:10:25.183 INFO:tasks.workunit.client.0.smithi162.stdout:3/985: dwrite d1/d21/d47/d158/d5f/d66/dd2/da3/f113 [0,4194304] 0 2024-04-27T04:10:25.186 INFO:tasks.workunit.client.0.smithi162.stdout:2/996: dwrite d0/d4/d17/d2b/d64/d96/fee [0,4194304] 0 2024-04-27T04:10:25.187 INFO:tasks.workunit.client.0.smithi162.stdout:4/946: chown d2/d25/c1d 114847964 1 2024-04-27T04:10:25.187 INFO:tasks.workunit.client.0.smithi162.stdout:2/997: dread - d0/d4/f133 zero size 2024-04-27T04:10:25.193 INFO:tasks.workunit.client.0.smithi162.stdout:4/947: symlink d2/d25/d1c/d106/l12f 0 2024-04-27T04:10:25.197 INFO:tasks.workunit.client.0.smithi162.stdout:2/998: read d0/d4/d17/d2b/d64/d89/fd4 [3463003,101233] 0 2024-04-27T04:10:25.197 INFO:tasks.workunit.client.0.smithi162.stdout:3/986: creat d1/d21/d47/d158/d6f/d130/d13c/f15f x:0 0 0 2024-04-27T04:10:25.200 INFO:tasks.workunit.client.0.smithi162.stdout:4/948: creat d2/d25/d1c/d24/d100/f130 x:0 0 0 2024-04-27T04:10:25.202 INFO:tasks.workunit.client.0.smithi162.stdout:4/949: chown d2/d25/d1c/d39/f7f 1515 1 2024-04-27T04:10:25.202 INFO:tasks.workunit.client.0.smithi162.stdout:4/950: stat d2/f11 0 2024-04-27T04:10:25.202 INFO:tasks.workunit.client.0.smithi162.stdout:4/951: chown d2/da/cf1 513990362 1 2024-04-27T04:10:25.203 INFO:tasks.workunit.client.0.smithi162.stdout:3/987: dread - d1/d18/d4f/fff zero size 2024-04-27T04:10:25.207 INFO:tasks.workunit.client.0.smithi162.stdout:4/952: symlink d2/d25/d1c/d39/d70/dcd/df0/l131 0 2024-04-27T04:10:25.208 INFO:tasks.workunit.client.0.smithi162.stdout:4/953: chown d2/da/fdd 74309 1 2024-04-27T04:10:25.212 INFO:tasks.workunit.client.0.smithi162.stdout:4/954: mknod d2/da/dd4/d129/c132 0 2024-04-27T04:10:25.224 INFO:tasks.workunit.client.0.smithi162.stdout:1/948: rename d0/d7/f5a to d0/d11/d50/d128/dfa/f138 0 2024-04-27T04:10:25.226 INFO:tasks.workunit.client.0.smithi162.stdout:6/972: rename d6/d1a/d34/d3a/d68/cec to d6/d1a/d34/d7f/d80/db3/dcd/c140 0 2024-04-27T04:10:25.229 INFO:tasks.workunit.client.0.smithi162.stdout:2/999: rename d0/d4/d2a/d3b/d49/d5a/d63/f129 to d0/d4/d17/d88/dc9/f137 0 2024-04-27T04:10:25.230 INFO:tasks.workunit.client.0.smithi162.stdout:6/973: rename d6/db/dc0/fc6 to d6/db/dc0/f141 0 2024-04-27T04:10:25.231 INFO:tasks.workunit.client.0.smithi162.stdout:1/949: rename d0/d11/d50/d53/d93/f10d to d0/d11/d3e/d5f/f139 0 2024-04-27T04:10:25.232 INFO:tasks.workunit.client.0.smithi162.stdout:1/950: dread - d0/d11/d50/d128/de9/fee zero size 2024-04-27T04:10:25.234 INFO:tasks.workunit.client.0.smithi162.stdout:1/951: creat d0/d7/d3c/d77/f13a x:0 0 0 2024-04-27T04:10:25.246 INFO:tasks.workunit.client.0.smithi162.stdout:8/990: truncate d0/d10/d25/f88 1192178 0 2024-04-27T04:10:25.247 INFO:tasks.workunit.client.0.smithi162.stdout:8/991: dread - d0/de/d6e/d71/d86/fc0 zero size 2024-04-27T04:10:25.249 INFO:tasks.workunit.client.0.smithi162.stdout:1/952: dread d0/d7/d10/d75/f10a [0,4194304] 0 2024-04-27T04:10:25.250 INFO:tasks.workunit.client.0.smithi162.stdout:1/953: chown d0/d11/d50/d53/d93/lab 864095 1 2024-04-27T04:10:25.257 INFO:tasks.workunit.client.0.smithi162.stdout:1/954: symlink d0/d7/d3c/d61/l13b 0 2024-04-27T04:10:25.259 INFO:tasks.workunit.client.0.smithi162.stdout:1/955: stat d0/d80/c12d 0 2024-04-27T04:10:25.260 INFO:tasks.workunit.client.0.smithi162.stdout:1/956: mkdir d0/d11/d50/d53/d93/d13c 0 2024-04-27T04:10:25.264 INFO:tasks.workunit.client.0.smithi162.stdout:3/988: dwrite d1/d65/f91 [0,4194304] 0 2024-04-27T04:10:25.265 INFO:tasks.workunit.client.0.smithi162.stdout:3/989: write d1/d70/f8c [3238493,72325] 0 2024-04-27T04:10:25.266 INFO:tasks.workunit.client.0.smithi162.stdout:4/955: write d2/d25/d1c/d39/def/f128 [1676698,81598] 0 2024-04-27T04:10:25.267 INFO:tasks.workunit.client.0.smithi162.stdout:4/956: stat d2/d25/c113 0 2024-04-27T04:10:25.271 INFO:tasks.workunit.client.0.smithi162.stdout:4/957: unlink d2/d25/d1c/d39/d3b/d6e/dd5/le5 0 2024-04-27T04:10:25.271 INFO:tasks.workunit.client.0.smithi162.stdout:3/990: dwrite d1/d18/f4d [4194304,4194304] 0 2024-04-27T04:10:25.278 INFO:tasks.workunit.client.0.smithi162.stdout:4/958: mknod d2/da/d114/c133 0 2024-04-27T04:10:25.285 INFO:tasks.workunit.client.0.smithi162.stdout:8/992: write d0/d10/d44/d70/f7f [151421,88077] 0 2024-04-27T04:10:25.287 INFO:tasks.workunit.client.0.smithi162.stdout:8/993: write d0/d10/d2b/d2d/d2f/d39/de1/f120 [2965314,81922] 0 2024-04-27T04:10:25.293 INFO:tasks.workunit.client.0.smithi162.stdout:1/957: dwrite d0/d80/fb8 [0,4194304] 0 2024-04-27T04:10:25.294 INFO:tasks.workunit.client.0.smithi162.stdout:1/958: fsync d0/d7/d3c/fe0 0 2024-04-27T04:10:25.296 INFO:tasks.workunit.client.0.smithi162.stdout:3/991: creat d1/d21/d47/f160 x:0 0 0 2024-04-27T04:10:25.300 INFO:tasks.workunit.client.0.smithi162.stdout:8/994: unlink d0/d10/d44/f47 0 2024-04-27T04:10:25.307 INFO:tasks.workunit.client.0.smithi162.stdout:1/959: chown d0/d7/d3c/d54/ca1 1490366 1 2024-04-27T04:10:25.308 INFO:tasks.workunit.client.0.smithi162.stdout:4/959: truncate d2/d25/d1c/d39/d5e/fd8 151524 0 2024-04-27T04:10:25.313 INFO:tasks.workunit.client.0.smithi162.stdout:8/995: creat d0/de/d6e/d71/d86/dd9/f13e x:0 0 0 2024-04-27T04:10:25.314 INFO:tasks.workunit.client.0.smithi162.stdout:9/953: sync 2024-04-27T04:10:25.316 INFO:tasks.workunit.client.0.smithi162.stdout:1/960: symlink d0/d7/d10/de4/l13d 0 2024-04-27T04:10:25.319 INFO:tasks.workunit.client.0.smithi162.stdout:3/992: symlink d1/d18/d23/l161 0 2024-04-27T04:10:25.319 INFO:tasks.workunit.client.0.smithi162.stdout:9/954: dwrite d3/d13/d2d/d30/d7f/ff7 [0,4194304] 0 2024-04-27T04:10:25.320 INFO:tasks.workunit.client.0.smithi162.stdout:9/955: chown d3/d11/d45/d4e/fb1 0 1 2024-04-27T04:10:25.322 INFO:tasks.workunit.client.0.smithi162.stdout:1/961: chown d0/d7/d10/d75/db7/dfd 846 1 2024-04-27T04:10:25.322 INFO:tasks.workunit.client.0.smithi162.stdout:1/962: stat d0/d11/f2e 0 2024-04-27T04:10:25.327 INFO:tasks.workunit.client.0.smithi162.stdout:8/996: symlink d0/d10/d2b/d2d/d2f/d39/d7c/dc2/l13f 0 2024-04-27T04:10:25.334 INFO:tasks.workunit.client.0.smithi162.stdout:1/963: mknod d0/d7/d3c/d54/dae/d10c/c13e 0 2024-04-27T04:10:25.335 INFO:tasks.workunit.client.0.smithi162.stdout:1/964: dread - d0/d7/d3c/f105 zero size 2024-04-27T04:10:25.341 INFO:tasks.workunit.client.0.smithi162.stdout:8/997: creat d0/d10/d2b/d2e/d127/d137/f140 x:0 0 0 2024-04-27T04:10:25.342 INFO:tasks.workunit.client.0.smithi162.stdout:1/965: symlink d0/d11/d44/l13f 0 2024-04-27T04:10:25.346 INFO:tasks.workunit.client.0.smithi162.stdout:4/960: truncate d2/f38 936269 0 2024-04-27T04:10:25.348 INFO:tasks.workunit.client.0.smithi162.stdout:4/961: write d2/d25/d1c/d39/d70/dcd/df0/f97 [1128950,58341] 0 2024-04-27T04:10:25.352 INFO:tasks.workunit.client.0.smithi162.stdout:4/962: fsync d2/d25/d1c/d39/d3b/f12d 0 2024-04-27T04:10:25.356 INFO:tasks.workunit.client.0.smithi162.stdout:6/974: sync 2024-04-27T04:10:25.358 INFO:tasks.workunit.client.0.smithi162.stdout:4/963: creat d2/d25/d1c/d39/d3b/d119/d11d/f134 x:0 0 0 2024-04-27T04:10:25.359 INFO:tasks.workunit.client.0.smithi162.stdout:8/998: dread d0/d10/d25/d7d/f8b [0,4194304] 0 2024-04-27T04:10:25.360 INFO:tasks.workunit.client.0.smithi162.stdout:1/966: dread d0/d7/d2a/f31 [0,4194304] 0 2024-04-27T04:10:25.363 INFO:tasks.workunit.client.0.smithi162.stdout:4/964: fdatasync d2/d25/d1c/d24/f11c 0 2024-04-27T04:10:25.364 INFO:tasks.workunit.client.0.smithi162.stdout:3/993: dwrite d1/d45/f9c [0,4194304] 0 2024-04-27T04:10:25.368 INFO:tasks.workunit.client.0.smithi162.stdout:1/967: mkdir d0/d11/d3e/d59/d7f/d140 0 2024-04-27T04:10:25.368 INFO:tasks.workunit.client.0.smithi162.stdout:3/994: dread - d1/d18/d23/d58/d63/d88/f121 zero size 2024-04-27T04:10:25.370 INFO:tasks.workunit.client.0.smithi162.stdout:4/965: write d2/d25/d1c/d39/d70/dcd/df0/f97 [1364237,35357] 0 2024-04-27T04:10:25.370 INFO:tasks.workunit.client.0.smithi162.stdout:9/956: dwrite d3/d11/d45/d4e/fb1 [0,4194304] 0 2024-04-27T04:10:25.371 INFO:tasks.workunit.client.0.smithi162.stdout:8/999: rename d0/d10/c38 to d0/de/d6e/d71/d86/dd9/d100/c141 0 2024-04-27T04:10:25.373 INFO:tasks.workunit.client.0.smithi162.stdout:6/975: link d6/c79 d6/d1a/d34/d3a/dbf/dd7/c142 0 2024-04-27T04:10:25.375 INFO:tasks.workunit.client.0.smithi162.stdout:6/976: dread - d6/d1a/d34/d3a/fbe zero size 2024-04-27T04:10:25.378 INFO:tasks.workunit.client.0.smithi162.stdout:4/966: fdatasync d2/f4 0 2024-04-27T04:10:25.382 INFO:tasks.workunit.client.0.smithi162.stdout:6/977: readlink d6/d1a/d34/d3a/l5f 0 2024-04-27T04:10:25.387 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:10:24 smithi162 ceph-mon[26841]: pgmap v12: 65 pgs: 65 active+clean; 3.8 GiB data, 13 GiB used, 524 GiB / 536 GiB avail; 94 MiB/s rd, 224 MiB/s wr, 550 op/s 2024-04-27T04:10:25.388 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:10:24 smithi162 ceph-mon[26841]: from='mgr.24367 172.21.15.181:0/3181919980' entity='mgr.smithi181.vtwuhz' 2024-04-27T04:10:25.388 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:10:24 smithi162 ceph-mon[26841]: from='mgr.24367 172.21.15.181:0/3181919980' entity='mgr.smithi181.vtwuhz' 2024-04-27T04:10:25.388 INFO:tasks.workunit.client.0.smithi162.stdout:3/995: link d1/d21/d47/d158/d5f/f77 d1/d21/d47/d57/dd6/d112/d14a/f162 0 2024-04-27T04:10:25.389 INFO:tasks.workunit.client.0.smithi162.stdout:3/996: dread - d1/d21/d47/f140 zero size 2024-04-27T04:10:25.393 INFO:tasks.workunit.client.0.smithi162.stdout:1/968: getdents d0/d11/d3e 0 2024-04-27T04:10:25.394 INFO:tasks.workunit.client.0.smithi162.stdout:1/969: stat d0/f3b 0 2024-04-27T04:10:25.395 INFO:tasks.workunit.client.0.smithi162.stdout:1/970: dread - d0/d11/d50/d128/d123/fca zero size 2024-04-27T04:10:25.396 INFO:tasks.workunit.client.0.smithi162.stdout:1/971: chown d0/d7/d3c/d77/dd1/f106 9697 1 2024-04-27T04:10:25.397 INFO:tasks.workunit.client.0.smithi162.stdout:1/972: write d0/d11/d50/d53/fbe [268030,31774] 0 2024-04-27T04:10:25.397 INFO:tasks.workunit.client.0.smithi162.stdout:6/978: dread d6/d1a/d34/d50/dad/fc9 [0,4194304] 0 2024-04-27T04:10:25.398 INFO:tasks.workunit.client.0.smithi162.stdout:1/973: chown d0/d11/f20 89561 1 2024-04-27T04:10:25.402 INFO:tasks.workunit.client.0.smithi162.stdout:9/957: link d3/fb d3/d13/d15/d7e/dd0/f126 0 2024-04-27T04:10:25.403 INFO:tasks.workunit.client.0.smithi162.stdout:6/979: dread d6/db/f48 [0,4194304] 0 2024-04-27T04:10:25.403 INFO:tasks.workunit.client.0.smithi162.stdout:9/958: dread - d3/d23/d7c/f9a zero size 2024-04-27T04:10:25.404 INFO:tasks.workunit.client.0.smithi162.stdout:4/967: write d2/d25/d1c/d106/f118 [5217860,43979] 0 2024-04-27T04:10:25.410 INFO:tasks.workunit.client.0.smithi162.stdout:9/959: mkdir d3/db9/dcc/d127 0 2024-04-27T04:10:25.412 INFO:tasks.workunit.client.0.smithi162.stdout:1/974: dread d0/d7/d10/f3d [0,4194304] 0 2024-04-27T04:10:25.414 INFO:tasks.workunit.client.0.smithi162.stdout:3/997: getdents d1/d21/d47/d57/dd6/d112/d132/dad/dc5/d109 0 2024-04-27T04:10:25.415 INFO:tasks.workunit.client.0.smithi162.stdout:3/998: fdatasync d1/d21/d47/d57/f5e 0 2024-04-27T04:10:25.417 INFO:tasks.workunit.client.0.smithi162.stdout:9/960: creat d3/db9/dcc/f128 x:0 0 0 2024-04-27T04:10:25.419 INFO:tasks.workunit.client.0.smithi162.stdout:4/968: fdatasync d2/d25/d1c/d39/d3b/d6e/fc1 0 2024-04-27T04:10:25.422 INFO:tasks.workunit.client.0.smithi162.stdout:1/975: symlink d0/d11/d50/d128/d127/l141 0 2024-04-27T04:10:25.422 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:10:24 smithi181 ceph-mon[33774]: pgmap v12: 65 pgs: 65 active+clean; 3.8 GiB data, 13 GiB used, 524 GiB / 536 GiB avail; 94 MiB/s rd, 224 MiB/s wr, 550 op/s 2024-04-27T04:10:25.422 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:10:24 smithi181 ceph-mon[33774]: from='mgr.24367 172.21.15.181:0/3181919980' entity='mgr.smithi181.vtwuhz' 2024-04-27T04:10:25.422 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:10:24 smithi181 ceph-mon[33774]: from='mgr.24367 172.21.15.181:0/3181919980' entity='mgr.smithi181.vtwuhz' 2024-04-27T04:10:25.423 INFO:tasks.workunit.client.0.smithi162.stdout:4/969: fdatasync d2/d25/d1c/d39/d3b/fc4 0 2024-04-27T04:10:25.425 INFO:tasks.workunit.client.0.smithi162.stdout:3/999: dread d1/d21/d47/d57/f5e [0,4194304] 0 2024-04-27T04:10:25.425 INFO:tasks.workunit.client.0.smithi162.stdout:1/976: read d0/d7/d3c/d77/dd1/f106 [978002,39806] 0 2024-04-27T04:10:25.430 INFO:tasks.workunit.client.0.smithi162.stdout:6/980: dread d6/d1a/d34/d3a/d68/d6c/d67/f7c [0,4194304] 0 2024-04-27T04:10:25.431 INFO:tasks.workunit.client.0.smithi162.stdout:9/961: creat d3/d13/d2d/d3b/d7b/f129 x:0 0 0 2024-04-27T04:10:25.432 INFO:tasks.workunit.client.0.smithi162.stdout:1/977: symlink d0/d7/d5e/l142 0 2024-04-27T04:10:25.436 INFO:tasks.workunit.client.0.smithi162.stdout:1/978: symlink d0/d7/d3c/d6e/l143 0 2024-04-27T04:10:25.441 INFO:tasks.workunit.client.0.smithi162.stdout:4/970: getdents d2/d25/d1c/d39/d3b/d119 0 2024-04-27T04:10:25.441 INFO:tasks.workunit.client.0.smithi162.stdout:6/981: getdents d6/d1a/d34/d3a/d68 0 2024-04-27T04:10:25.442 INFO:tasks.workunit.client.0.smithi162.stdout:9/962: write d3/d23/f88 [1529947,13023] 0 2024-04-27T04:10:25.479 INFO:tasks.workunit.client.0.smithi162.stdout:6/982: write d6/d1a/d34/d3a/dfd/f11a [4098274,58528] 0 2024-04-27T04:10:25.479 INFO:tasks.workunit.client.0.smithi162.stdout:9/963: write d3/d11/d45/d4e/fbe [2410445,2265] 0 2024-04-27T04:10:25.480 INFO:tasks.workunit.client.0.smithi162.stdout:6/983: chown d6/db/d32 254787 1 2024-04-27T04:10:25.533 INFO:tasks.workunit.client.0.smithi162.stdout:4/971: dwrite d2/d54/f10b [0,4194304] 0 2024-04-27T04:10:25.537 INFO:tasks.workunit.client.0.smithi162.stdout:4/972: dread - d2/d25/f79 zero size 2024-04-27T04:10:25.537 INFO:tasks.workunit.client.0.smithi162.stdout:9/964: mknod d3/d13/d2d/d3b/d54/d11b/c12a 0 2024-04-27T04:10:25.538 INFO:tasks.workunit.client.0.smithi162.stdout:6/984: rmdir d6/d1a/d34/d3a/d68/d6c/de3 39 2024-04-27T04:10:25.555 INFO:tasks.workunit.client.0.smithi162.stdout:4/973: dread d2/d25/d1c/d24/fcf [0,4194304] 0 2024-04-27T04:10:25.563 INFO:tasks.workunit.client.0.smithi162.stdout:4/974: truncate d2/d25/d1c/d106/f118 1429369 0 2024-04-27T04:10:25.564 INFO:tasks.workunit.client.0.smithi162.stdout:9/965: dwrite d3/d13/d2d/d30/d7f/fb3 [0,4194304] 0 2024-04-27T04:10:25.565 INFO:tasks.workunit.client.0.smithi162.stdout:6/985: dwrite d6/d1a/d34/d7f/d80/db3/fbb [4194304,4194304] 0 2024-04-27T04:10:25.566 INFO:tasks.workunit.client.0.smithi162.stdout:9/966: chown d3/d23/f73 15474749 1 2024-04-27T04:10:25.582 INFO:tasks.workunit.client.0.smithi162.stdout:4/975: truncate d2/d25/d1c/d39/def/f12c 3257817 0 2024-04-27T04:10:25.584 INFO:tasks.workunit.client.0.smithi162.stdout:1/979: sync 2024-04-27T04:10:25.585 INFO:tasks.workunit.client.0.smithi162.stdout:6/986: write d6/d1a/d34/d3a/d68/d6c/f53 [1139056,100178] 0 2024-04-27T04:10:25.591 INFO:tasks.workunit.client.0.smithi162.stdout:4/976: unlink d2/d25/d1c/d39/d90/dad/dfd/f104 0 2024-04-27T04:10:25.595 INFO:tasks.workunit.client.0.smithi162.stdout:1/980: mknod d0/d11/d44/c144 0 2024-04-27T04:10:25.595 INFO:tasks.workunit.client.0.smithi162.stdout:6/987: dread d6/f26 [0,4194304] 0 2024-04-27T04:10:25.599 INFO:tasks.workunit.client.0.smithi162.stdout:1/981: mkdir d0/d80/dba/d145 0 2024-04-27T04:10:25.601 INFO:tasks.workunit.client.0.smithi162.stdout:1/982: write d0/d7/d3c/d61/dc4/f10f [4650,51687] 0 2024-04-27T04:10:25.603 INFO:tasks.workunit.client.0.smithi162.stdout:9/967: truncate d3/d23/f3e 3322763 0 2024-04-27T04:10:25.604 INFO:tasks.workunit.client.0.smithi162.stdout:1/983: rename d0/f2b to d0/d11/d3e/d59/d7f/dc5/f146 0 2024-04-27T04:10:25.608 INFO:tasks.workunit.client.0.smithi162.stdout:9/968: truncate d3/d11/fca 2473584 0 2024-04-27T04:10:25.609 INFO:tasks.workunit.client.0.smithi162.stdout:9/969: write d3/d23/d1e/f4a [5029395,26429] 0 2024-04-27T04:10:25.612 INFO:tasks.workunit.client.0.smithi162.stdout:9/970: fdatasync d3/d13/d2d/d3b/d54/f8f 0 2024-04-27T04:10:25.622 INFO:tasks.workunit.client.0.smithi162.stdout:9/971: link d3/d23/d1e/dfa/f11c d3/d11/f12b 0 2024-04-27T04:10:25.624 INFO:tasks.workunit.client.0.smithi162.stdout:4/977: write d2/d25/d1c/d39/d3b/da7/fcb [1314683,54835] 0 2024-04-27T04:10:25.625 INFO:tasks.workunit.client.0.smithi162.stdout:4/978: chown d2/d8a/l7c 0 1 2024-04-27T04:10:25.629 INFO:tasks.workunit.client.0.smithi162.stdout:1/984: dread d0/d80/f87 [0,4194304] 0 2024-04-27T04:10:25.630 INFO:tasks.workunit.client.0.smithi162.stdout:6/988: write d6/f30 [1014193,37099] 0 2024-04-27T04:10:25.637 INFO:tasks.workunit.client.0.smithi162.stdout:1/985: dwrite d0/d11/d50/d128/de9/fee [0,4194304] 0 2024-04-27T04:10:25.638 INFO:tasks.workunit.client.0.smithi162.stdout:1/986: readlink d0/d11/d44/l13f 0 2024-04-27T04:10:25.644 INFO:tasks.workunit.client.0.smithi162.stdout:4/979: write d2/d25/d1c/d24/f3a [2982493,87398] 0 2024-04-27T04:10:25.647 INFO:tasks.workunit.client.0.smithi162.stdout:9/972: dwrite d3/d11/d45/f55 [0,4194304] 0 2024-04-27T04:10:25.649 INFO:tasks.workunit.client.0.smithi162.stdout:4/980: write d2/d25/d1c/d39/d70/f8d [3189220,107801] 0 2024-04-27T04:10:25.652 INFO:tasks.workunit.client.0.smithi162.stdout:9/973: mknod d3/d39/d112/c12c 0 2024-04-27T04:10:25.654 INFO:tasks.workunit.client.0.smithi162.stdout:1/987: truncate d0/d80/f87 207112 0 2024-04-27T04:10:25.657 INFO:tasks.workunit.client.0.smithi162.stdout:9/974: fsync d3/d11/d45/f99 0 2024-04-27T04:10:25.659 INFO:tasks.workunit.client.0.smithi162.stdout:1/988: unlink d0/d7/d5e/ff2 0 2024-04-27T04:10:25.659 INFO:tasks.workunit.client.0.smithi162.stdout:6/989: dwrite d6/d1a/d34/d3a/d68/d6c/f82 [0,4194304] 0 2024-04-27T04:10:25.665 INFO:tasks.workunit.client.0.smithi162.stdout:1/989: symlink d0/d7/d10/d75/l147 0 2024-04-27T04:10:25.665 INFO:tasks.workunit.client.0.smithi162.stdout:6/990: creat d6/d1a/d34/d3a/dfd/f143 x:0 0 0 2024-04-27T04:10:25.667 INFO:tasks.workunit.client.0.smithi162.stdout:4/981: getdents d2/d25/d1c/d39/d70/dcd/df0 0 2024-04-27T04:10:25.668 INFO:tasks.workunit.client.0.smithi162.stdout:4/982: stat d2/d25/d1c/d39/d3b/d6e/d6f/lbf 0 2024-04-27T04:10:25.670 INFO:tasks.workunit.client.0.smithi162.stdout:6/991: mknod d6/d1a/d34/d3a/dfd/c144 0 2024-04-27T04:10:25.672 INFO:tasks.workunit.client.0.smithi162.stdout:4/983: truncate d2/d25/d1c/d39/d90/dad/f65 4340743 0 2024-04-27T04:10:25.675 INFO:tasks.workunit.client.0.smithi162.stdout:6/992: dread d6/d1a/d34/d7f/dd9/de4/ffb [0,4194304] 0 2024-04-27T04:10:25.676 INFO:tasks.workunit.client.0.smithi162.stdout:1/990: dread d0/d11/d3e/d59/d7f/dc5/fd8 [0,4194304] 0 2024-04-27T04:10:25.676 INFO:tasks.workunit.client.0.smithi162.stdout:4/984: mkdir d2/d54/dca/d135 0 2024-04-27T04:10:25.683 INFO:tasks.workunit.client.0.smithi162.stdout:4/985: dwrite d2/d25/d1c/d39/d5e/fdf [0,4194304] 0 2024-04-27T04:10:25.686 INFO:tasks.workunit.client.0.smithi162.stdout:1/991: dread d0/d11/f2e [0,4194304] 0 2024-04-27T04:10:25.697 INFO:tasks.workunit.client.0.smithi162.stdout:9/975: write d3/d13/d15/d50/f10e [2123940,78235] 0 2024-04-27T04:10:25.701 INFO:tasks.workunit.client.0.smithi162.stdout:1/992: creat d0/d7/d10/d75/db7/f148 x:0 0 0 2024-04-27T04:10:25.705 INFO:tasks.workunit.client.0.smithi162.stdout:4/986: write d2/d25/d1c/d24/d100/f1b [1620100,101332] 0 2024-04-27T04:10:25.709 INFO:tasks.workunit.client.0.smithi162.stdout:6/993: dwrite d6/d1a/d34/d43/fab [0,4194304] 0 2024-04-27T04:10:25.714 INFO:tasks.workunit.client.0.smithi162.stdout:4/987: getdents d2/d25/d1c/d24/d100/ddc 0 2024-04-27T04:10:25.717 INFO:tasks.workunit.client.0.smithi162.stdout:9/976: dwrite d3/f5 [4194304,4194304] 0 2024-04-27T04:10:25.718 INFO:tasks.workunit.client.0.smithi162.stdout:1/993: dwrite d0/d11/f4d [0,4194304] 0 2024-04-27T04:10:25.719 INFO:tasks.workunit.client.0.smithi162.stdout:4/988: write d2/d25/d1c/d39/f45 [1470920,127065] 0 2024-04-27T04:10:25.722 INFO:tasks.workunit.client.0.smithi162.stdout:1/994: chown d0/d11/d50/ca0 0 1 2024-04-27T04:10:25.727 INFO:tasks.workunit.client.0.smithi162.stdout:6/994: dwrite d6/d1a/d34/d7f/dd9/de4/f13c [0,4194304] 0 2024-04-27T04:10:25.727 INFO:tasks.workunit.client.0.smithi162.stdout:4/989: readlink d2/d25/d1c/d24/d100/l68 0 2024-04-27T04:10:25.730 INFO:tasks.workunit.client.0.smithi162.stdout:1/995: dread d0/d11/d50/d128/de9/fee [0,4194304] 0 2024-04-27T04:10:25.741 INFO:tasks.workunit.client.0.smithi162.stdout:9/977: link d3/d13/d2d/d30/d67/l100 d3/d13/d2d/d3b/d7a/l12d 0 2024-04-27T04:10:25.744 INFO:tasks.workunit.client.0.smithi162.stdout:1/996: rename d0/d7/d3c/d77/dd1/f106 to d0/d11/d3e/d59/f149 0 2024-04-27T04:10:25.749 INFO:tasks.workunit.client.0.smithi162.stdout:1/997: truncate d0/d7/d2a/f47 1117592 0 2024-04-27T04:10:25.754 INFO:tasks.workunit.client.0.smithi162.stdout:6/995: write d6/f62 [127108,15440] 0 2024-04-27T04:10:25.757 INFO:tasks.workunit.client.0.smithi162.stdout:9/978: dread d3/d13/d2d/d3b/d72/f94 [0,4194304] 0 2024-04-27T04:10:25.758 INFO:tasks.workunit.client.0.smithi162.stdout:9/979: write d3/d11/d45/f55 [4945884,51760] 0 2024-04-27T04:10:25.764 INFO:tasks.workunit.client.0.smithi162.stdout:1/998: write d0/d7/d3c/d61/f135 [3731608,105711] 0 2024-04-27T04:10:25.770 INFO:tasks.workunit.client.0.smithi162.stdout:1/999: dwrite d0/d7/d10/f11d [0,4194304] 0 2024-04-27T04:10:25.773 INFO:tasks.workunit.client.0.smithi162.stdout:9/980: write d3/d13/d2d/d4f/faf [315360,37248] 0 2024-04-27T04:10:25.774 INFO:tasks.workunit.client.0.smithi162.stdout:6/996: dwrite d6/d1a/d34/d50/dad/fc9 [0,4194304] 0 2024-04-27T04:10:25.780 INFO:tasks.workunit.client.0.smithi162.stdout:9/981: mknod d3/d13/d2d/d3b/d7a/dd6/c12e 0 2024-04-27T04:10:25.781 INFO:tasks.workunit.client.0.smithi162.stdout:9/982: fdatasync d3/f4 0 2024-04-27T04:10:25.786 INFO:tasks.workunit.client.0.smithi162.stdout:6/997: write d6/db/dc0/ff5 [849169,31307] 0 2024-04-27T04:10:25.790 INFO:tasks.workunit.client.0.smithi162.stdout:6/998: mknod d6/db/dc0/d74/c145 0 2024-04-27T04:10:25.790 INFO:tasks.workunit.client.0.smithi162.stdout:6/999: write d6/f35 [869493,14812] 0 2024-04-27T04:10:25.888 INFO:tasks.workunit.client.0.smithi162.stdout:9/983: sync 2024-04-27T04:10:25.888 INFO:tasks.workunit.client.0.smithi162.stdout:4/990: sync 2024-04-27T04:10:25.888 INFO:tasks.workunit.client.0.smithi162.stdout:9/984: fsync d3/d13/d2d/d3b/d54/f120 0 2024-04-27T04:10:25.891 INFO:tasks.workunit.client.0.smithi162.stdout:4/991: symlink d2/d25/d1c/d24/db5/dc0/l136 0 2024-04-27T04:10:25.895 INFO:tasks.workunit.client.0.smithi162.stdout:9/985: creat d3/d13/d15/d50/f12f x:0 0 0 2024-04-27T04:10:25.895 INFO:tasks.workunit.client.0.smithi162.stdout:4/992: rmdir d2/d25/d1c 39 2024-04-27T04:10:25.896 INFO:tasks.workunit.client.0.smithi162.stdout:9/986: write d3/d13/d15/d50/f8b [703545,94586] 0 2024-04-27T04:10:25.900 INFO:tasks.workunit.client.0.smithi162.stdout:9/987: mknod d3/d13/d15/d7e/c130 0 2024-04-27T04:10:25.901 INFO:tasks.workunit.client.0.smithi162.stdout:9/988: write d3/d13/d15/d50/f12f [619202,102824] 0 2024-04-27T04:10:25.901 INFO:tasks.workunit.client.0.smithi162.stdout:4/993: mkdir d2/d25/d1c/d24/d137 0 2024-04-27T04:10:25.913 INFO:tasks.workunit.client.0.smithi162.stdout:9/989: dread d3/fbd [0,4194304] 0 2024-04-27T04:10:25.913 INFO:tasks.workunit.client.0.smithi162.stdout:9/990: dread - d3/d13/d15/d7e/f11f zero size 2024-04-27T04:10:25.914 INFO:tasks.workunit.client.0.smithi162.stdout:4/994: write d2/da/d114/f11f [4629751,13217] 0 2024-04-27T04:10:25.920 INFO:tasks.workunit.client.0.smithi162.stdout:9/991: dwrite d3/d13/d2d/d30/d7f/ff8 [0,4194304] 0 2024-04-27T04:10:25.921 INFO:tasks.workunit.client.0.smithi162.stdout:4/995: dwrite d2/da/dd/f127 [0,4194304] 0 2024-04-27T04:10:25.923 INFO:tasks.workunit.client.0.smithi162.stdout:9/992: creat d3/d11/f131 x:0 0 0 2024-04-27T04:10:25.923 INFO:tasks.workunit.client.0.smithi162.stdout:9/993: fsync d3/d13/d2d/d30/f38 0 2024-04-27T04:10:25.930 INFO:tasks.workunit.client.0.smithi162.stdout:9/994: symlink d3/d13/d2d/d4f/l132 0 2024-04-27T04:10:25.938 INFO:tasks.workunit.client.0.smithi162.stdout:9/995: rename d3/d11/d45/d4e/fd7 to d3/d13/d15/d7e/f133 0 2024-04-27T04:10:25.940 INFO:tasks.workunit.client.0.smithi162.stdout:9/996: chown d3/d13/d2d/d30/c10a 36683 1 2024-04-27T04:10:25.942 INFO:tasks.workunit.client.0.smithi162.stdout:9/997: symlink d3/d23/d1e/dfa/l134 0 2024-04-27T04:10:25.944 INFO:tasks.workunit.client.0.smithi162.stdout:4/996: dwrite d2/da/f31 [4194304,4194304] 0 2024-04-27T04:10:25.947 INFO:tasks.workunit.client.0.smithi162.stdout:4/997: creat d2/d25/d1c/d39/d3b/d6e/dd5/f138 x:0 0 0 2024-04-27T04:10:25.959 INFO:tasks.workunit.client.0.smithi162.stdout:4/998: dread d2/d25/d1c/d39/d3b/d47/ffc [0,4194304] 0 2024-04-27T04:10:25.959 INFO:tasks.workunit.client.0.smithi162.stdout:4/999: stat d2/da/dd/fe1 0 2024-04-27T04:10:25.973 INFO:tasks.workunit.client.0.smithi162.stdout:9/998: dread d3/d11/d45/d4e/fbe [0,4194304] 0 2024-04-27T04:10:25.984 INFO:tasks.workunit.client.0.smithi162.stdout:9/999: truncate d3/f14 86676 0 2024-04-27T04:10:25.987 INFO:tasks.workunit.client.0.smithi162.stderr:+ rm -rf -- ./tmp.jpOSNEf0nQ 2024-04-27T04:10:27.387 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:10:27 smithi162 ceph-mon[26841]: pgmap v13: 65 pgs: 65 active+clean; 3.8 GiB data, 13 GiB used, 524 GiB / 536 GiB avail; 77 MiB/s rd, 180 MiB/s wr, 413 op/s 2024-04-27T04:10:27.387 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:10:27 smithi162 ceph-mon[26841]: from='mgr.24367 172.21.15.181:0/3181919980' entity='mgr.smithi181.vtwuhz' 2024-04-27T04:10:27.387 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:10:27 smithi162 ceph-mon[26841]: from='mgr.24367 172.21.15.181:0/3181919980' entity='mgr.smithi181.vtwuhz' 2024-04-27T04:10:27.387 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:10:27 smithi162 ceph-mon[26841]: from='mgr.24367 172.21.15.181:0/3181919980' entity='mgr.smithi181.vtwuhz' 2024-04-27T04:10:27.387 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:10:27 smithi162 ceph-mon[26841]: from='mgr.24367 172.21.15.181:0/3181919980' entity='mgr.smithi181.vtwuhz' 2024-04-27T04:10:27.422 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:10:27 smithi181 ceph-mon[33774]: pgmap v13: 65 pgs: 65 active+clean; 3.8 GiB data, 13 GiB used, 524 GiB / 536 GiB avail; 77 MiB/s rd, 180 MiB/s wr, 413 op/s 2024-04-27T04:10:27.422 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:10:27 smithi181 ceph-mon[33774]: from='mgr.24367 172.21.15.181:0/3181919980' entity='mgr.smithi181.vtwuhz' 2024-04-27T04:10:27.422 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:10:27 smithi181 ceph-mon[33774]: from='mgr.24367 172.21.15.181:0/3181919980' entity='mgr.smithi181.vtwuhz' 2024-04-27T04:10:27.422 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:10:27 smithi181 ceph-mon[33774]: from='mgr.24367 172.21.15.181:0/3181919980' entity='mgr.smithi181.vtwuhz' 2024-04-27T04:10:27.423 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:10:27 smithi181 ceph-mon[33774]: from='mgr.24367 172.21.15.181:0/3181919980' entity='mgr.smithi181.vtwuhz' 2024-04-27T04:10:29.322 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:10:29 smithi181 ceph-mon[33774]: pgmap v14: 65 pgs: 65 active+clean; 4.1 GiB data, 14 GiB used, 523 GiB / 536 GiB avail; 108 MiB/s rd, 252 MiB/s wr, 594 op/s 2024-04-27T04:10:29.322 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:10:29 smithi181 ceph-mon[33774]: from='mgr.24367 172.21.15.181:0/3181919980' entity='mgr.smithi181.vtwuhz' 2024-04-27T04:10:29.322 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:10:29 smithi181 ceph-mon[33774]: from='mgr.24367 172.21.15.181:0/3181919980' entity='mgr.smithi181.vtwuhz' 2024-04-27T04:10:29.322 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:10:29 smithi181 ceph-mon[33774]: from='mgr.24367 172.21.15.181:0/3181919980' entity='mgr.smithi181.vtwuhz' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-27T04:10:29.322 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:10:29 smithi181 ceph-mon[33774]: from='mgr.24367 172.21.15.181:0/3181919980' entity='mgr.smithi181.vtwuhz' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-27T04:10:29.322 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:10:29 smithi181 ceph-mon[33774]: from='mgr.24367 172.21.15.181:0/3181919980' entity='mgr.smithi181.vtwuhz' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-27T04:10:29.322 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:10:29 smithi181 ceph-mon[33774]: Adjusting osd_memory_target on smithi162 to 1557M 2024-04-27T04:10:29.322 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:10:29 smithi181 ceph-mon[33774]: from='mgr.24367 172.21.15.181:0/3181919980' entity='mgr.smithi181.vtwuhz' 2024-04-27T04:10:29.387 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:10:29 smithi162 ceph-mon[26841]: pgmap v14: 65 pgs: 65 active+clean; 4.1 GiB data, 14 GiB used, 523 GiB / 536 GiB avail; 108 MiB/s rd, 252 MiB/s wr, 594 op/s 2024-04-27T04:10:29.387 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:10:29 smithi162 ceph-mon[26841]: from='mgr.24367 172.21.15.181:0/3181919980' entity='mgr.smithi181.vtwuhz' 2024-04-27T04:10:29.387 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:10:29 smithi162 ceph-mon[26841]: from='mgr.24367 172.21.15.181:0/3181919980' entity='mgr.smithi181.vtwuhz' 2024-04-27T04:10:29.387 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:10:29 smithi162 ceph-mon[26841]: from='mgr.24367 172.21.15.181:0/3181919980' entity='mgr.smithi181.vtwuhz' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-27T04:10:29.387 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:10:29 smithi162 ceph-mon[26841]: from='mgr.24367 172.21.15.181:0/3181919980' entity='mgr.smithi181.vtwuhz' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-27T04:10:29.387 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:10:29 smithi162 ceph-mon[26841]: from='mgr.24367 172.21.15.181:0/3181919980' entity='mgr.smithi181.vtwuhz' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-27T04:10:29.387 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:10:29 smithi162 ceph-mon[26841]: Adjusting osd_memory_target on smithi162 to 1557M 2024-04-27T04:10:29.388 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:10:29 smithi162 ceph-mon[26841]: from='mgr.24367 172.21.15.181:0/3181919980' entity='mgr.smithi181.vtwuhz' 2024-04-27T04:10:31.172 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:10:31 smithi181 ceph-mon[33774]: pgmap v15: 65 pgs: 65 active+clean; 4.1 GiB data, 14 GiB used, 523 GiB / 536 GiB avail; 71 MiB/s rd, 161 MiB/s wr, 388 op/s 2024-04-27T04:10:31.387 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:10:31 smithi162 ceph-mon[26841]: pgmap v15: 65 pgs: 65 active+clean; 4.1 GiB data, 14 GiB used, 523 GiB / 536 GiB avail; 71 MiB/s rd, 161 MiB/s wr, 388 op/s 2024-04-27T04:10:33.387 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:10:33 smithi162 ceph-mon[26841]: pgmap v16: 65 pgs: 65 active+clean; 3.7 GiB data, 12 GiB used, 524 GiB / 536 GiB avail; 71 MiB/s rd, 161 MiB/s wr, 408 op/s 2024-04-27T04:10:33.422 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:10:33 smithi181 ceph-mon[33774]: pgmap v16: 65 pgs: 65 active+clean; 3.7 GiB data, 12 GiB used, 524 GiB / 536 GiB avail; 71 MiB/s rd, 161 MiB/s wr, 408 op/s 2024-04-27T04:10:34.387 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:10:34 smithi162 ceph-mon[26841]: from='mgr.24367 172.21.15.181:0/3181919980' entity='mgr.smithi181.vtwuhz' 2024-04-27T04:10:34.387 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:10:34 smithi162 ceph-mon[26841]: from='mgr.24367 172.21.15.181:0/3181919980' entity='mgr.smithi181.vtwuhz' 2024-04-27T04:10:34.387 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:10:34 smithi162 ceph-mon[26841]: from='mgr.24367 172.21.15.181:0/3181919980' entity='mgr.smithi181.vtwuhz' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-27T04:10:34.387 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:10:34 smithi162 ceph-mon[26841]: from='mgr.24367 172.21.15.181:0/3181919980' entity='mgr.smithi181.vtwuhz' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-27T04:10:34.387 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:10:34 smithi162 ceph-mon[26841]: from='mgr.24367 172.21.15.181:0/3181919980' entity='mgr.smithi181.vtwuhz' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-27T04:10:34.387 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:10:34 smithi162 ceph-mon[26841]: Adjusting osd_memory_target on smithi181 to 2581M 2024-04-27T04:10:34.387 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:10:34 smithi162 ceph-mon[26841]: from='mgr.24367 172.21.15.181:0/3181919980' entity='mgr.smithi181.vtwuhz' 2024-04-27T04:10:34.387 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:10:34 smithi162 ceph-mon[26841]: from='mgr.24367 172.21.15.181:0/3181919980' entity='mgr.smithi181.vtwuhz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T04:10:34.387 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:10:34 smithi162 ceph-mon[26841]: from='mgr.24367 172.21.15.181:0/3181919980' entity='mgr.smithi181.vtwuhz' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T04:10:34.387 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:10:34 smithi162 ceph-mon[26841]: Updating smithi162:/etc/ceph/ceph.conf 2024-04-27T04:10:34.388 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:10:34 smithi162 ceph-mon[26841]: Updating smithi181:/etc/ceph/ceph.conf 2024-04-27T04:10:34.422 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:10:34 smithi181 ceph-mon[33774]: from='mgr.24367 172.21.15.181:0/3181919980' entity='mgr.smithi181.vtwuhz' 2024-04-27T04:10:34.422 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:10:34 smithi181 ceph-mon[33774]: from='mgr.24367 172.21.15.181:0/3181919980' entity='mgr.smithi181.vtwuhz' 2024-04-27T04:10:34.422 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:10:34 smithi181 ceph-mon[33774]: from='mgr.24367 172.21.15.181:0/3181919980' entity='mgr.smithi181.vtwuhz' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-27T04:10:34.422 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:10:34 smithi181 ceph-mon[33774]: from='mgr.24367 172.21.15.181:0/3181919980' entity='mgr.smithi181.vtwuhz' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-27T04:10:34.422 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:10:34 smithi181 ceph-mon[33774]: from='mgr.24367 172.21.15.181:0/3181919980' entity='mgr.smithi181.vtwuhz' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-27T04:10:34.423 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:10:34 smithi181 ceph-mon[33774]: Adjusting osd_memory_target on smithi181 to 2581M 2024-04-27T04:10:34.423 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:10:34 smithi181 ceph-mon[33774]: from='mgr.24367 172.21.15.181:0/3181919980' entity='mgr.smithi181.vtwuhz' 2024-04-27T04:10:34.423 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:10:34 smithi181 ceph-mon[33774]: from='mgr.24367 172.21.15.181:0/3181919980' entity='mgr.smithi181.vtwuhz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T04:10:34.423 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:10:34 smithi181 ceph-mon[33774]: from='mgr.24367 172.21.15.181:0/3181919980' entity='mgr.smithi181.vtwuhz' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T04:10:34.423 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:10:34 smithi181 ceph-mon[33774]: Updating smithi162:/etc/ceph/ceph.conf 2024-04-27T04:10:34.423 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:10:34 smithi181 ceph-mon[33774]: Updating smithi181:/etc/ceph/ceph.conf 2024-04-27T04:10:35.534 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:10:35 smithi162 ceph-mon[26841]: Updating smithi162:/var/lib/ceph/7f9750d6-044a-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-27T04:10:35.534 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:10:35 smithi162 ceph-mon[26841]: Updating smithi181:/var/lib/ceph/7f9750d6-044a-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-27T04:10:35.535 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:10:35 smithi162 ceph-mon[26841]: Updating smithi162:/etc/ceph/ceph.client.admin.keyring 2024-04-27T04:10:35.535 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:10:35 smithi162 ceph-mon[26841]: Updating smithi181:/etc/ceph/ceph.client.admin.keyring 2024-04-27T04:10:35.535 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:10:35 smithi162 ceph-mon[26841]: pgmap v17: 65 pgs: 65 active+clean; 3.4 GiB data, 11 GiB used, 525 GiB / 536 GiB avail; 51 MiB/s rd, 111 MiB/s wr, 317 op/s 2024-04-27T04:10:35.535 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:10:35 smithi162 ceph-mon[26841]: Updating smithi162:/var/lib/ceph/7f9750d6-044a-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-27T04:10:35.535 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:10:35 smithi162 ceph-mon[26841]: Updating smithi181:/var/lib/ceph/7f9750d6-044a-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-27T04:10:35.535 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:10:35 smithi162 ceph-mon[26841]: from='mgr.24367 172.21.15.181:0/3181919980' entity='mgr.smithi181.vtwuhz' 2024-04-27T04:10:35.535 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:10:35 smithi162 ceph-mon[26841]: from='mgr.24367 172.21.15.181:0/3181919980' entity='mgr.smithi181.vtwuhz' 2024-04-27T04:10:35.535 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:10:35 smithi162 ceph-mon[26841]: from='mgr.24367 172.21.15.181:0/3181919980' entity='mgr.smithi181.vtwuhz' 2024-04-27T04:10:35.535 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:10:35 smithi162 ceph-mon[26841]: from='mgr.24367 172.21.15.181:0/3181919980' entity='mgr.smithi181.vtwuhz' 2024-04-27T04:10:35.535 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:10:35 smithi162 ceph-mon[26841]: from='mgr.24367 172.21.15.181:0/3181919980' entity='mgr.smithi181.vtwuhz' 2024-04-27T04:10:35.535 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:10:35 smithi162 ceph-mon[26841]: from='mgr.24367 172.21.15.181:0/3181919980' entity='mgr.smithi181.vtwuhz' 2024-04-27T04:10:35.535 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:10:35 smithi162 ceph-mon[26841]: from='mgr.24367 172.21.15.181:0/3181919980' entity='mgr.smithi181.vtwuhz' 2024-04-27T04:10:35.535 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:10:35 smithi162 ceph-mon[26841]: from='mgr.24367 172.21.15.181:0/3181919980' entity='mgr.smithi181.vtwuhz' 2024-04-27T04:10:35.535 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:10:35 smithi162 ceph-mon[26841]: from='mgr.24367 172.21.15.181:0/3181919980' entity='mgr.smithi181.vtwuhz' 2024-04-27T04:10:35.672 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:10:35 smithi181 ceph-mon[33774]: Updating smithi162:/var/lib/ceph/7f9750d6-044a-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-27T04:10:35.672 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:10:35 smithi181 ceph-mon[33774]: Updating smithi181:/var/lib/ceph/7f9750d6-044a-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-27T04:10:35.672 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:10:35 smithi181 ceph-mon[33774]: Updating smithi162:/etc/ceph/ceph.client.admin.keyring 2024-04-27T04:10:35.673 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:10:35 smithi181 ceph-mon[33774]: Updating smithi181:/etc/ceph/ceph.client.admin.keyring 2024-04-27T04:10:35.673 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:10:35 smithi181 ceph-mon[33774]: pgmap v17: 65 pgs: 65 active+clean; 3.4 GiB data, 11 GiB used, 525 GiB / 536 GiB avail; 51 MiB/s rd, 111 MiB/s wr, 317 op/s 2024-04-27T04:10:35.673 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:10:35 smithi181 ceph-mon[33774]: Updating smithi162:/var/lib/ceph/7f9750d6-044a-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-27T04:10:35.673 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:10:35 smithi181 ceph-mon[33774]: Updating smithi181:/var/lib/ceph/7f9750d6-044a-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-27T04:10:35.673 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:10:35 smithi181 ceph-mon[33774]: from='mgr.24367 172.21.15.181:0/3181919980' entity='mgr.smithi181.vtwuhz' 2024-04-27T04:10:35.673 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:10:35 smithi181 ceph-mon[33774]: from='mgr.24367 172.21.15.181:0/3181919980' entity='mgr.smithi181.vtwuhz' 2024-04-27T04:10:35.673 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:10:35 smithi181 ceph-mon[33774]: from='mgr.24367 172.21.15.181:0/3181919980' entity='mgr.smithi181.vtwuhz' 2024-04-27T04:10:35.673 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:10:35 smithi181 ceph-mon[33774]: from='mgr.24367 172.21.15.181:0/3181919980' entity='mgr.smithi181.vtwuhz' 2024-04-27T04:10:35.673 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:10:35 smithi181 ceph-mon[33774]: from='mgr.24367 172.21.15.181:0/3181919980' entity='mgr.smithi181.vtwuhz' 2024-04-27T04:10:35.673 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:10:35 smithi181 ceph-mon[33774]: from='mgr.24367 172.21.15.181:0/3181919980' entity='mgr.smithi181.vtwuhz' 2024-04-27T04:10:35.673 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:10:35 smithi181 ceph-mon[33774]: from='mgr.24367 172.21.15.181:0/3181919980' entity='mgr.smithi181.vtwuhz' 2024-04-27T04:10:35.673 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:10:35 smithi181 ceph-mon[33774]: from='mgr.24367 172.21.15.181:0/3181919980' entity='mgr.smithi181.vtwuhz' 2024-04-27T04:10:35.673 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:10:35 smithi181 ceph-mon[33774]: from='mgr.24367 172.21.15.181:0/3181919980' entity='mgr.smithi181.vtwuhz' 2024-04-27T04:10:36.387 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:10:36 smithi162 ceph-mon[26841]: Reconfiguring node-exporter.smithi162 (dependencies changed)... 2024-04-27T04:10:36.387 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:10:36 smithi162 ceph-mon[26841]: Reconfiguring daemon node-exporter.smithi162 on smithi162 2024-04-27T04:10:36.387 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:10:36 smithi162 ceph-mon[26841]: from='mgr.24367 172.21.15.181:0/3181919980' entity='mgr.smithi181.vtwuhz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-27T04:10:36.672 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:10:36 smithi181 ceph-mon[33774]: Reconfiguring node-exporter.smithi162 (dependencies changed)... 2024-04-27T04:10:36.672 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:10:36 smithi181 ceph-mon[33774]: Reconfiguring daemon node-exporter.smithi162 on smithi162 2024-04-27T04:10:36.672 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:10:36 smithi181 ceph-mon[33774]: from='mgr.24367 172.21.15.181:0/3181919980' entity='mgr.smithi181.vtwuhz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-27T04:10:37.637 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:10:37 smithi162 ceph-mon[26841]: pgmap v18: 65 pgs: 65 active+clean; 3.4 GiB data, 11 GiB used, 525 GiB / 536 GiB avail; 31 MiB/s rd, 73 MiB/s wr, 261 op/s 2024-04-27T04:10:37.637 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:10:37 smithi162 ceph-mon[26841]: from='mgr.24367 172.21.15.181:0/3181919980' entity='mgr.smithi181.vtwuhz' 2024-04-27T04:10:37.637 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:10:37 smithi162 ceph-mon[26841]: from='mgr.24367 172.21.15.181:0/3181919980' entity='mgr.smithi181.vtwuhz' 2024-04-27T04:10:37.672 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:10:37 smithi181 ceph-mon[33774]: pgmap v18: 65 pgs: 65 active+clean; 3.4 GiB data, 11 GiB used, 525 GiB / 536 GiB avail; 31 MiB/s rd, 73 MiB/s wr, 261 op/s 2024-04-27T04:10:37.672 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:10:37 smithi181 ceph-mon[33774]: from='mgr.24367 172.21.15.181:0/3181919980' entity='mgr.smithi181.vtwuhz' 2024-04-27T04:10:37.672 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:10:37 smithi181 ceph-mon[33774]: from='mgr.24367 172.21.15.181:0/3181919980' entity='mgr.smithi181.vtwuhz' 2024-04-27T04:10:38.387 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:10:38 smithi162 ceph-mon[26841]: Reconfiguring alertmanager.smithi162 (dependencies changed)... 2024-04-27T04:10:38.387 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:10:38 smithi162 ceph-mon[26841]: Reconfiguring daemon alertmanager.smithi162 on smithi162 2024-04-27T04:10:38.672 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:10:38 smithi181 ceph-mon[33774]: Reconfiguring alertmanager.smithi162 (dependencies changed)... 2024-04-27T04:10:38.672 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:10:38 smithi181 ceph-mon[33774]: Reconfiguring daemon alertmanager.smithi162 on smithi162 2024-04-27T04:10:39.387 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:10:39 smithi162 ceph-mon[26841]: pgmap v19: 65 pgs: 65 active+clean; 2.3 GiB data, 8.4 GiB used, 528 GiB / 536 GiB avail; 32 MiB/s rd, 73 MiB/s wr, 335 op/s 2024-04-27T04:10:39.672 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:10:39 smithi181 ceph-mon[33774]: pgmap v19: 65 pgs: 65 active+clean; 2.3 GiB data, 8.4 GiB used, 528 GiB / 536 GiB avail; 32 MiB/s rd, 73 MiB/s wr, 335 op/s 2024-04-27T04:10:41.387 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:10:41 smithi162 ceph-mon[26841]: pgmap v20: 65 pgs: 65 active+clean; 2.3 GiB data, 8.4 GiB used, 528 GiB / 536 GiB avail; 1.1 MiB/s rd, 1.2 MiB/s wr, 154 op/s 2024-04-27T04:10:41.672 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:10:41 smithi181 ceph-mon[33774]: pgmap v20: 65 pgs: 65 active+clean; 2.3 GiB data, 8.4 GiB used, 528 GiB / 536 GiB avail; 1.1 MiB/s rd, 1.2 MiB/s wr, 154 op/s 2024-04-27T04:10:42.366 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-04-27T04:10:42.366 DEBUG:teuthology.orchestra.run.smithi181:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.1/client.1/tmp 2024-04-27T04:10:43.422 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:10:43 smithi181 ceph-mon[33774]: pgmap v21: 65 pgs: 65 active+clean; 1.6 GiB data, 6.2 GiB used, 530 GiB / 536 GiB avail; 1.4 MiB/s rd, 1.5 MiB/s wr, 200 op/s 2024-04-27T04:10:43.422 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:10:43 smithi181 ceph-mon[33774]: from='mgr.24367 172.21.15.181:0/3181919980' entity='mgr.smithi181.vtwuhz' 2024-04-27T04:10:43.422 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:10:43 smithi181 ceph-mon[33774]: from='mgr.24367 172.21.15.181:0/3181919980' entity='mgr.smithi181.vtwuhz' 2024-04-27T04:10:43.453 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-04-27T04:10:43.453 DEBUG:teuthology.orchestra.run.smithi162:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2024-04-27T04:10:43.517 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:10:43 smithi162 ceph-mon[26841]: pgmap v21: 65 pgs: 65 active+clean; 1.6 GiB data, 6.2 GiB used, 530 GiB / 536 GiB avail; 1.4 MiB/s rd, 1.5 MiB/s wr, 200 op/s 2024-04-27T04:10:43.517 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:10:43 smithi162 ceph-mon[26841]: from='mgr.24367 172.21.15.181:0/3181919980' entity='mgr.smithi181.vtwuhz' 2024-04-27T04:10:43.517 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:10:43 smithi162 ceph-mon[26841]: from='mgr.24367 172.21.15.181:0/3181919980' entity='mgr.smithi181.vtwuhz' 2024-04-27T04:10:44.465 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:10:44 smithi162 ceph-mon[26841]: Reconfiguring grafana.smithi162 (dependencies changed)... 2024-04-27T04:10:44.466 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:10:44 smithi162 ceph-mon[26841]: Reconfiguring daemon grafana.smithi162 on smithi162 2024-04-27T04:10:44.672 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:10:44 smithi181 ceph-mon[33774]: Reconfiguring grafana.smithi162 (dependencies changed)... 2024-04-27T04:10:44.672 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:10:44 smithi181 ceph-mon[33774]: Reconfiguring daemon grafana.smithi162 on smithi162 2024-04-27T04:10:45.548 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:10:45 smithi162 ceph-mon[26841]: pgmap v22: 65 pgs: 65 active+clean; 1.1 GiB data, 5.2 GiB used, 531 GiB / 536 GiB avail; 1.7 MiB/s rd, 1.7 MiB/s wr, 211 op/s 2024-04-27T04:10:45.672 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:10:45 smithi181 ceph-mon[33774]: pgmap v22: 65 pgs: 65 active+clean; 1.1 GiB data, 5.2 GiB used, 531 GiB / 536 GiB avail; 1.7 MiB/s rd, 1.7 MiB/s wr, 211 op/s 2024-04-27T04:10:47.387 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:10:47 smithi162 ceph-mon[26841]: pgmap v23: 65 pgs: 65 active+clean; 1.1 GiB data, 5.2 GiB used, 531 GiB / 536 GiB avail; 1.2 MiB/s rd, 1.1 MiB/s wr, 151 op/s 2024-04-27T04:10:47.672 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:10:47 smithi181 ceph-mon[33774]: pgmap v23: 65 pgs: 65 active+clean; 1.1 GiB data, 5.2 GiB used, 531 GiB / 536 GiB avail; 1.2 MiB/s rd, 1.1 MiB/s wr, 151 op/s 2024-04-27T04:10:49.289 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:10:49 smithi162 ceph-mon[26841]: pgmap v24: 65 pgs: 65 active+clean; 279 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 1.7 MiB/s rd, 1.7 MiB/s wr, 244 op/s 2024-04-27T04:10:49.619 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:10:49.617+0000 7f32a8f3e700 1 -- 172.21.15.162:0/1594417841 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f32a40fbcb0 msgr2=0x7f32a40fc0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:10:49.619 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:10:49.617+0000 7f32a8f3e700 1 --2- 172.21.15.162:0/1594417841 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f32a40fbcb0 0x7f32a40fc0c0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f3294004940 tx=0x7f3294009e30 comp rx=0 tx=0).stop 2024-04-27T04:10:49.619 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:10:49.617+0000 7f32a8f3e700 1 -- 172.21.15.162:0/1594417841 shutdown_connections 2024-04-27T04:10:49.619 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:10:49.617+0000 7f32a8f3e700 1 --2- 172.21.15.162:0/1594417841 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f32a40fcf60 0x7f32a40fd3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:10:49.619 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:10:49.617+0000 7f32a8f3e700 1 --2- 172.21.15.162:0/1594417841 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f32a40fbcb0 0x7f32a40fc0c0 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:10:49.619 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:10:49.617+0000 7f32a8f3e700 1 -- 172.21.15.162:0/1594417841 >> 172.21.15.162:0/1594417841 conn(0x7f32a40f7430 msgr2=0x7f32a40f9840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:10:49.620 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:10:49.617+0000 7f32a8f3e700 1 -- 172.21.15.162:0/1594417841 shutdown_connections 2024-04-27T04:10:49.620 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:10:49.617+0000 7f32a8f3e700 1 -- 172.21.15.162:0/1594417841 wait complete. 2024-04-27T04:10:49.620 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:10:49.618+0000 7f32a8f3e700 1 Processor -- start 2024-04-27T04:10:49.620 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:10:49.618+0000 7f32a8f3e700 1 -- start start 2024-04-27T04:10:49.620 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:10:49.618+0000 7f32a8f3e700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f32a40fbcb0 0x7f32a40ffd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:10:49.620 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:10:49.618+0000 7f32a8f3e700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f32a40fcf60 0x7f32a40fe2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:10:49.620 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:10:49.618+0000 7f32a8f3e700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32a40fe830 con 0x7f32a40fcf60 2024-04-27T04:10:49.620 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:10:49.618+0000 7f32a8f3e700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32a40fe970 con 0x7f32a40fbcb0 2024-04-27T04:10:49.621 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:10:49.618+0000 7f32a1d9b700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f32a40fcf60 0x7f32a40fe2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:10:49.621 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:10:49.618+0000 7f32a259c700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f32a40fbcb0 0x7f32a40ffd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:10:49.621 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:10:49.618+0000 7f32a1d9b700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f32a40fcf60 0x7f32a40fe2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:50854/0 (socket says 172.21.15.162:50854) 2024-04-27T04:10:49.621 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:10:49.618+0000 7f32a1d9b700 1 -- 172.21.15.162:0/2367757291 learned_addr learned my addr 172.21.15.162:0/2367757291 (peer_addr_for_me v2:172.21.15.162:0/0) 2024-04-27T04:10:49.621 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:10:49.619+0000 7f32a259c700 1 -- 172.21.15.162:0/2367757291 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f32a40fcf60 msgr2=0x7f32a40fe2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:10:49.621 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:10:49.619+0000 7f32a259c700 1 --2- 172.21.15.162:0/2367757291 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f32a40fcf60 0x7f32a40fe2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:10:49.621 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:10:49.619+0000 7f32a259c700 1 -- 172.21.15.162:0/2367757291 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3294005040 con 0x7f32a40fbcb0 2024-04-27T04:10:49.622 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:10:49.619+0000 7f32a259c700 1 --2- 172.21.15.162:0/2367757291 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f32a40fbcb0 0x7f32a40ffd80 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f3294000c00 tx=0x7f3294009e30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:10:49.622 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:10:49.619+0000 7f3292ffd700 1 -- 172.21.15.162:0/2367757291 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f329400e030 con 0x7f32a40fbcb0 2024-04-27T04:10:49.622 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:10:49.619+0000 7f32a8f3e700 1 -- 172.21.15.162:0/2367757291 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f32a40febf0 con 0x7f32a40fbcb0 2024-04-27T04:10:49.622 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:10:49.619+0000 7f32a8f3e700 1 -- 172.21.15.162:0/2367757291 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f32a41159f0 con 0x7f32a40fbcb0 2024-04-27T04:10:49.622 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:10:49.619+0000 7f3292ffd700 1 -- 172.21.15.162:0/2367757291 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f3294013de0 con 0x7f32a40fbcb0 2024-04-27T04:10:49.622 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:10:49.620+0000 7f3292ffd700 1 -- 172.21.15.162:0/2367757291 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f329401cdf0 con 0x7f32a40fbcb0 2024-04-27T04:10:49.624 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:10:49.620+0000 7f32a8f3e700 1 -- 172.21.15.162:0/2367757291 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f32840052e0 con 0x7f32a40fbcb0 2024-04-27T04:10:49.624 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:10:49.621+0000 7f3292ffd700 1 -- 172.21.15.162:0/2367757291 <== mon.1 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 24) v1 ==== 89652+0+0 (secure 0 0 0) 0x7f329401c400 con 0x7f32a40fbcb0 2024-04-27T04:10:49.624 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:10:49.621+0000 7f3292ffd700 1 --2- 172.21.15.162:0/2367757291 >> [v2:172.21.15.181:6828/4165336392,v1:172.21.15.181:6829/4165336392] conn(0x7f328c06bcf0 0x7f328c06e1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:10:49.624 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:10:49.621+0000 7f3292ffd700 1 -- 172.21.15.162:0/2367757291 <== mon.1 v2:172.21.15.181:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5808+0+0 (secure 0 0 0) 0x7f329408bfc0 con 0x7f32a40fbcb0 2024-04-27T04:10:49.625 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:10:49.621+0000 7f32a1d9b700 1 --2- 172.21.15.162:0/2367757291 >> [v2:172.21.15.181:6828/4165336392,v1:172.21.15.181:6829/4165336392] conn(0x7f328c06bcf0 0x7f328c06e1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:10:49.625 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:10:49.622+0000 7f32a1d9b700 1 --2- 172.21.15.162:0/2367757291 >> [v2:172.21.15.181:6828/4165336392,v1:172.21.15.181:6829/4165336392] conn(0x7f328c06bcf0 0x7f328c06e1a0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f3298008fd0 tx=0x7f329800b040 comp rx=0 tx=0).ready entity=mgr.24367 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:10:49.627 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:10:49.628+0000 7f3292ffd700 1 -- 172.21.15.162:0/2367757291 <== mon.1 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186652 (secure 0 0 0) 0x7f3294055620 con 0x7f32a40fbcb0 2024-04-27T04:10:49.672 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:10:49 smithi181 ceph-mon[33774]: pgmap v24: 65 pgs: 65 active+clean; 279 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 1.7 MiB/s rd, 1.7 MiB/s wr, 244 op/s 2024-04-27T04:10:49.874 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:10:49.875+0000 7f32a8f3e700 1 -- 172.21.15.162:0/2367757291 --> [v2:172.21.15.181:6828/4165336392,v1:172.21.15.181:6829/4165336392] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f3284000bf0 con 0x7f328c06bcf0 2024-04-27T04:10:49.875 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:10:49.876+0000 7f3292ffd700 1 -- 172.21.15.162:0/2367757291 <== mgr.24367 v2:172.21.15.181:6828/4165336392 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+310 (secure 0 0 0) 0x7f3284000bf0 con 0x7f328c06bcf0 2024-04-27T04:10:49.880 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:10:49.880+0000 7f32a8f3e700 1 -- 172.21.15.162:0/2367757291 >> [v2:172.21.15.181:6828/4165336392,v1:172.21.15.181:6829/4165336392] conn(0x7f328c06bcf0 msgr2=0x7f328c06e1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:10:49.880 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:10:49.880+0000 7f32a8f3e700 1 --2- 172.21.15.162:0/2367757291 >> [v2:172.21.15.181:6828/4165336392,v1:172.21.15.181:6829/4165336392] conn(0x7f328c06bcf0 0x7f328c06e1a0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f3298008fd0 tx=0x7f329800b040 comp rx=0 tx=0).stop 2024-04-27T04:10:49.880 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:10:49.880+0000 7f32a8f3e700 1 -- 172.21.15.162:0/2367757291 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f32a40fbcb0 msgr2=0x7f32a40ffd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:10:49.880 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:10:49.880+0000 7f32a8f3e700 1 --2- 172.21.15.162:0/2367757291 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f32a40fbcb0 0x7f32a40ffd80 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f3294000c00 tx=0x7f3294009e30 comp rx=0 tx=0).stop 2024-04-27T04:10:49.880 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:10:49.880+0000 7f32a8f3e700 1 -- 172.21.15.162:0/2367757291 shutdown_connections 2024-04-27T04:10:49.880 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:10:49.880+0000 7f32a8f3e700 1 --2- 172.21.15.162:0/2367757291 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f32a40fbcb0 0x7f32a40ffd80 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:10:49.880 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:10:49.880+0000 7f32a8f3e700 1 --2- 172.21.15.162:0/2367757291 >> [v2:172.21.15.181:6828/4165336392,v1:172.21.15.181:6829/4165336392] conn(0x7f328c06bcf0 0x7f328c06e1a0 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:10:49.880 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:10:49.880+0000 7f32a8f3e700 1 --2- 172.21.15.162:0/2367757291 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f32a40fcf60 0x7f32a40fe2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:10:49.881 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:10:49.880+0000 7f32a8f3e700 1 -- 172.21.15.162:0/2367757291 >> 172.21.15.162:0/2367757291 conn(0x7f32a40f7430 msgr2=0x7f32a4065b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:10:49.881 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:10:49.880+0000 7f32a8f3e700 1 -- 172.21.15.162:0/2367757291 shutdown_connections 2024-04-27T04:10:49.881 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:10:49.880+0000 7f32a8f3e700 1 -- 172.21.15.162:0/2367757291 wait complete. 2024-04-27T04:10:49.897 INFO:teuthology.orchestra.run.smithi162.stdout:true 2024-04-27T04:10:50.000 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:10:50.000+0000 7fafbbb16700 1 -- 172.21.15.162:0/1990704699 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fafb40fce60 msgr2=0x7fafb40fd270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:10:50.000 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:10:50.000+0000 7fafbbb16700 1 --2- 172.21.15.162:0/1990704699 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fafb40fce60 0x7fafb40fd270 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7fafb0004940 tx=0x7fafb0009e30 comp rx=0 tx=0).stop 2024-04-27T04:10:50.000 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:10:50.000+0000 7fafbbb16700 1 -- 172.21.15.162:0/1990704699 shutdown_connections 2024-04-27T04:10:50.000 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:10:50.000+0000 7fafbbb16700 1 --2- 172.21.15.162:0/1990704699 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fafb40fdfd0 0x7fafb40fe440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:10:50.000 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:10:50.000+0000 7fafbbb16700 1 --2- 172.21.15.162:0/1990704699 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fafb40fce60 0x7fafb40fd270 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:10:50.001 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:10:50.000+0000 7fafbbb16700 1 -- 172.21.15.162:0/1990704699 >> 172.21.15.162:0/1990704699 conn(0x7fafb40f8540 msgr2=0x7fafb40fa950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:10:50.001 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:10:50.001+0000 7fafbbb16700 1 -- 172.21.15.162:0/1990704699 shutdown_connections 2024-04-27T04:10:50.001 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:10:50.001+0000 7fafbbb16700 1 -- 172.21.15.162:0/1990704699 wait complete. 2024-04-27T04:10:50.002 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:10:50.001+0000 7fafbbb16700 1 Processor -- start 2024-04-27T04:10:50.007 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:10:50.001+0000 7fafbbb16700 1 -- start start 2024-04-27T04:10:50.007 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:10:50.001+0000 7fafbbb16700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fafb40fce60 0x7fafb4071c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:10:50.007 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:10:50.001+0000 7fafbbb16700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fafb40fdfd0 0x7fafb40702b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:10:50.007 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:10:50.001+0000 7fafbbb16700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fafb4072200 con 0x7fafb40fce60 2024-04-27T04:10:50.007 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:10:50.001+0000 7fafbbb16700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fafb40707f0 con 0x7fafb40fdfd0 2024-04-27T04:10:50.007 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:10:50.002+0000 7fafb98b2700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fafb40fce60 0x7fafb4071c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:10:50.007 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:10:50.002+0000 7fafb98b2700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fafb40fce60 0x7fafb4071c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:50880/0 (socket says 172.21.15.162:50880) 2024-04-27T04:10:50.007 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:10:50.002+0000 7fafb98b2700 1 -- 172.21.15.162:0/319936008 learned_addr learned my addr 172.21.15.162:0/319936008 (peer_addr_for_me v2:172.21.15.162:0/0) 2024-04-27T04:10:50.007 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:10:50.002+0000 7fafb90b1700 1 --2- 172.21.15.162:0/319936008 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fafb40fdfd0 0x7fafb40702b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:10:50.007 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:10:50.002+0000 7fafb90b1700 1 -- 172.21.15.162:0/319936008 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fafb40fce60 msgr2=0x7fafb4071c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:10:50.008 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:10:50.002+0000 7fafb90b1700 1 --2- 172.21.15.162:0/319936008 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fafb40fce60 0x7fafb4071c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:10:50.008 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:10:50.002+0000 7fafb90b1700 1 -- 172.21.15.162:0/319936008 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fafb0005040 con 0x7fafb40fdfd0 2024-04-27T04:10:50.008 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:10:50.002+0000 7fafb98b2700 1 --2- 172.21.15.162:0/319936008 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fafb40fce60 0x7fafb4071c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-27T04:10:50.008 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:10:50.002+0000 7fafb90b1700 1 --2- 172.21.15.162:0/319936008 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fafb40fdfd0 0x7fafb40702b0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fafa4009fd0 tx=0x7fafa400b040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:10:50.008 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:10:50.002+0000 7fafaa7fc700 1 -- 172.21.15.162:0/319936008 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fafa4004e90 con 0x7fafb40fdfd0 2024-04-27T04:10:50.008 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:10:50.002+0000 7fafbbb16700 1 -- 172.21.15.162:0/319936008 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fafb4070a40 con 0x7fafb40fdfd0 2024-04-27T04:10:50.008 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:10:50.002+0000 7fafaa7fc700 1 -- 172.21.15.162:0/319936008 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fafa400f040 con 0x7fafb40fdfd0 2024-04-27T04:10:50.008 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:10:50.003+0000 7fafaa7fc700 1 -- 172.21.15.162:0/319936008 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fafa400ca50 con 0x7fafb40fdfd0 2024-04-27T04:10:50.008 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:10:50.003+0000 7fafbbb16700 1 -- 172.21.15.162:0/319936008 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fafb4070f80 con 0x7fafb40fdfd0 2024-04-27T04:10:50.009 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:10:50.004+0000 7fafbbb16700 1 -- 172.21.15.162:0/319936008 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fafb404c710 con 0x7fafb40fdfd0 2024-04-27T04:10:50.009 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:10:50.005+0000 7fafaa7fc700 1 -- 172.21.15.162:0/319936008 <== mon.1 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 24) v1 ==== 89652+0+0 (secure 0 0 0) 0x7fafa400cc00 con 0x7fafb40fdfd0 2024-04-27T04:10:50.009 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:10:50.005+0000 7fafaa7fc700 1 --2- 172.21.15.162:0/319936008 >> [v2:172.21.15.181:6828/4165336392,v1:172.21.15.181:6829/4165336392] conn(0x7fafa006bd00 0x7fafa006e1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:10:50.009 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:10:50.005+0000 7fafaa7fc700 1 -- 172.21.15.162:0/319936008 <== mon.1 v2:172.21.15.181:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5808+0+0 (secure 0 0 0) 0x7fafa408a520 con 0x7fafb40fdfd0 2024-04-27T04:10:50.009 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:10:50.005+0000 7fafb98b2700 1 --2- 172.21.15.162:0/319936008 >> [v2:172.21.15.181:6828/4165336392,v1:172.21.15.181:6829/4165336392] conn(0x7fafa006bd00 0x7fafa006e1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:10:50.010 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:10:50.006+0000 7fafb98b2700 1 --2- 172.21.15.162:0/319936008 >> [v2:172.21.15.181:6828/4165336392,v1:172.21.15.181:6829/4165336392] conn(0x7fafa006bd00 0x7fafa006e1b0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fafb000b0d0 tx=0x7fafb0009e30 comp rx=0 tx=0).ready entity=mgr.24367 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:10:50.012 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:10:50.013+0000 7fafaa7fc700 1 -- 172.21.15.162:0/319936008 <== mon.1 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186652 (secure 0 0 0) 0x7fafa4050930 con 0x7fafb40fdfd0 2024-04-27T04:10:50.355 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:10:50.355+0000 7fafbbb16700 1 -- 172.21.15.162:0/319936008 --> [v2:172.21.15.181:6828/4165336392,v1:172.21.15.181:6829/4165336392] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fafb41022a0 con 0x7fafa006bd00 2024-04-27T04:10:50.356 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:10:50.357+0000 7fafaa7fc700 1 -- 172.21.15.162:0/319936008 <== mgr.24367 v2:172.21.15.181:6828/4165336392 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+310 (secure 0 0 0) 0x7fafb41022a0 con 0x7fafa006bd00 2024-04-27T04:10:50.359 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:10:50.359+0000 7fafbbb16700 1 -- 172.21.15.162:0/319936008 >> [v2:172.21.15.181:6828/4165336392,v1:172.21.15.181:6829/4165336392] conn(0x7fafa006bd00 msgr2=0x7fafa006e1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:10:50.359 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:10:50.359+0000 7fafbbb16700 1 --2- 172.21.15.162:0/319936008 >> [v2:172.21.15.181:6828/4165336392,v1:172.21.15.181:6829/4165336392] conn(0x7fafa006bd00 0x7fafa006e1b0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fafb000b0d0 tx=0x7fafb0009e30 comp rx=0 tx=0).stop 2024-04-27T04:10:50.359 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:10:50.359+0000 7fafbbb16700 1 -- 172.21.15.162:0/319936008 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fafb40fdfd0 msgr2=0x7fafb40702b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:10:50.359 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:10:50.359+0000 7fafbbb16700 1 --2- 172.21.15.162:0/319936008 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fafb40fdfd0 0x7fafb40702b0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fafa4009fd0 tx=0x7fafa400b040 comp rx=0 tx=0).stop 2024-04-27T04:10:50.360 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:10:50.359+0000 7fafbbb16700 1 -- 172.21.15.162:0/319936008 shutdown_connections 2024-04-27T04:10:50.360 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:10:50.359+0000 7fafbbb16700 1 --2- 172.21.15.162:0/319936008 >> [v2:172.21.15.181:6828/4165336392,v1:172.21.15.181:6829/4165336392] conn(0x7fafa006bd00 0x7fafa006e1b0 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:10:50.360 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:10:50.359+0000 7fafbbb16700 1 --2- 172.21.15.162:0/319936008 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fafb40fce60 0x7fafb4071c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:10:50.360 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:10:50.359+0000 7fafbbb16700 1 --2- 172.21.15.162:0/319936008 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fafb40fdfd0 0x7fafb40702b0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:10:50.360 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:10:50.359+0000 7fafbbb16700 1 -- 172.21.15.162:0/319936008 >> 172.21.15.162:0/319936008 conn(0x7fafb40f8540 msgr2=0x7fafb40f9b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:10:50.361 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:10:50.359+0000 7fafbbb16700 1 -- 172.21.15.162:0/319936008 shutdown_connections 2024-04-27T04:10:50.361 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:10:50.359+0000 7fafbbb16700 1 -- 172.21.15.162:0/319936008 wait complete. 2024-04-27T04:10:50.472 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:10:50.471+0000 7f9d94158700 1 -- 172.21.15.162:0/3939738108 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9d8c0fc270 msgr2=0x7f9d8c0fc680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:10:50.472 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:10:50.471+0000 7f9d94158700 1 --2- 172.21.15.162:0/3939738108 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9d8c0fc270 0x7f9d8c0fc680 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f9d88004940 tx=0x7f9d88009e30 comp rx=0 tx=0).stop 2024-04-27T04:10:50.472 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:10:50.471+0000 7f9d94158700 1 -- 172.21.15.162:0/3939738108 shutdown_connections 2024-04-27T04:10:50.472 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:10:50.471+0000 7f9d94158700 1 --2- 172.21.15.162:0/3939738108 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9d8c0fcbc0 0x7f9d8c0fd030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:10:50.472 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:10:50.471+0000 7f9d94158700 1 --2- 172.21.15.162:0/3939738108 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9d8c0fc270 0x7f9d8c0fc680 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:10:50.472 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:10:50.471+0000 7f9d94158700 1 -- 172.21.15.162:0/3939738108 >> 172.21.15.162:0/3939738108 conn(0x7f9d8c0f6b80 msgr2=0x7f9d8c0f8fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:10:50.472 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:10:50.472+0000 7f9d94158700 1 -- 172.21.15.162:0/3939738108 shutdown_connections 2024-04-27T04:10:50.472 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:10:50.472+0000 7f9d94158700 1 -- 172.21.15.162:0/3939738108 wait complete. 2024-04-27T04:10:50.472 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:10:50.472+0000 7f9d94158700 1 Processor -- start 2024-04-27T04:10:50.472 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:10:50.472+0000 7f9d94158700 1 -- start start 2024-04-27T04:10:50.474 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:10:50.473+0000 7f9d94158700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9d8c0fc270 0x7f9d8c10fb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:10:50.474 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:10:50.473+0000 7f9d94158700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9d8c0fcbc0 0x7f9d8c1100c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:10:50.474 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:10:50.473+0000 7f9d94158700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d8c1106c0 con 0x7f9d8c0fcbc0 2024-04-27T04:10:50.474 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:10:50.473+0000 7f9d94158700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d8c110f10 con 0x7f9d8c0fc270 2024-04-27T04:10:50.474 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:10:50.473+0000 7f9d916f3700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9d8c0fcbc0 0x7f9d8c1100c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:10:50.474 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:10:50.473+0000 7f9d916f3700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9d8c0fcbc0 0x7f9d8c1100c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:50894/0 (socket says 172.21.15.162:50894) 2024-04-27T04:10:50.474 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:10:50.473+0000 7f9d916f3700 1 -- 172.21.15.162:0/3292950423 learned_addr learned my addr 172.21.15.162:0/3292950423 (peer_addr_for_me v2:172.21.15.162:0/0) 2024-04-27T04:10:50.474 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:10:50.473+0000 7f9d91ef4700 1 --2- 172.21.15.162:0/3292950423 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9d8c0fc270 0x7f9d8c10fb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:10:50.475 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:10:50.473+0000 7f9d916f3700 1 -- 172.21.15.162:0/3292950423 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9d8c0fc270 msgr2=0x7f9d8c10fb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:10:50.475 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:10:50.473+0000 7f9d916f3700 1 --2- 172.21.15.162:0/3292950423 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9d8c0fc270 0x7f9d8c10fb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:10:50.475 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:10:50.473+0000 7f9d916f3700 1 -- 172.21.15.162:0/3292950423 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d88005040 con 0x7f9d8c0fcbc0 2024-04-27T04:10:50.475 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:10:50.474+0000 7f9d916f3700 1 --2- 172.21.15.162:0/3292950423 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9d8c0fcbc0 0x7f9d8c1100c0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f9d7c00eb40 tx=0x7f9d7c00b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:10:50.475 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:10:50.474+0000 7f9d827fc700 1 -- 172.21.15.162:0/3292950423 <== mon.0 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9d7c007cc0 con 0x7f9d8c0fcbc0 2024-04-27T04:10:50.475 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:10:50.474+0000 7f9d94158700 1 -- 172.21.15.162:0/3292950423 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9d8c111190 con 0x7f9d8c0fcbc0 2024-04-27T04:10:50.475 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:10:50.474+0000 7f9d827fc700 1 -- 172.21.15.162:0/3292950423 <== mon.0 v2:172.21.15.162:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f9d7c00f040 con 0x7f9d8c0fcbc0 2024-04-27T04:10:50.475 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:10:50.474+0000 7f9d827fc700 1 -- 172.21.15.162:0/3292950423 <== mon.0 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9d7c00ca10 con 0x7f9d8c0fcbc0 2024-04-27T04:10:50.475 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:10:50.474+0000 7f9d94158700 1 -- 172.21.15.162:0/3292950423 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9d8c1116d0 con 0x7f9d8c0fcbc0 2024-04-27T04:10:50.476 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:10:50.475+0000 7f9d94158700 1 -- 172.21.15.162:0/3292950423 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9d8c1009c0 con 0x7f9d8c0fcbc0 2024-04-27T04:10:50.476 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:10:50.476+0000 7f9d827fc700 1 -- 172.21.15.162:0/3292950423 <== mon.0 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 24) v1 ==== 89652+0+0 (secure 0 0 0) 0x7f9d7c00cc80 con 0x7f9d8c0fcbc0 2024-04-27T04:10:50.476 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:10:50.476+0000 7f9d827fc700 1 --2- 172.21.15.162:0/3292950423 >> [v2:172.21.15.181:6828/4165336392,v1:172.21.15.181:6829/4165336392] conn(0x7f9d7806bcf0 0x7f9d7806e1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:10:50.476 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:10:50.476+0000 7f9d827fc700 1 -- 172.21.15.162:0/3292950423 <== mon.0 v2:172.21.15.162:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5808+0+0 (secure 0 0 0) 0x7f9d7c08a6d0 con 0x7f9d8c0fcbc0 2024-04-27T04:10:50.476 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:10:50.476+0000 7f9d91ef4700 1 --2- 172.21.15.162:0/3292950423 >> [v2:172.21.15.181:6828/4165336392,v1:172.21.15.181:6829/4165336392] conn(0x7f9d7806bcf0 0x7f9d7806e1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:10:50.477 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:10:50.477+0000 7f9d91ef4700 1 --2- 172.21.15.162:0/3292950423 >> [v2:172.21.15.181:6828/4165336392,v1:172.21.15.181:6829/4165336392] conn(0x7f9d7806bcf0 0x7f9d7806e1a0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f9d88009b30 tx=0x7f9d88009e30 comp rx=0 tx=0).ready entity=mgr.24367 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:10:50.481 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:10:50.481+0000 7f9d827fc700 1 -- 172.21.15.162:0/3292950423 <== mon.0 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186652 (secure 0 0 0) 0x7f9d7c050930 con 0x7f9d8c0fcbc0 2024-04-27T04:10:50.637 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:10:50 smithi162 ceph-mon[26841]: from='mgr.24367 172.21.15.181:0/3181919980' entity='mgr.smithi181.vtwuhz' 2024-04-27T04:10:50.637 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:10:50 smithi162 ceph-mon[26841]: from='mgr.24367 172.21.15.181:0/3181919980' entity='mgr.smithi181.vtwuhz' 2024-04-27T04:10:50.637 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:10:50 smithi162 ceph-mon[26841]: Reconfiguring prometheus.smithi162 (dependencies changed)... 2024-04-27T04:10:50.637 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:10:50 smithi162 ceph-mon[26841]: Reconfiguring daemon prometheus.smithi162 on smithi162 2024-04-27T04:10:50.637 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:10:50 smithi162 ceph-mon[26841]: pgmap v25: 65 pgs: 65 active+clean; 279 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 169 op/s 2024-04-27T04:10:50.637 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:10:50 smithi162 ceph-mon[26841]: from='client.24435 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-27T04:10:50.740 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:10:50.740+0000 7f9d94158700 1 -- 172.21.15.162:0/3292950423 --> [v2:172.21.15.181:6828/4165336392,v1:172.21.15.181:6829/4165336392] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f9d8c0fb7d0 con 0x7f9d7806bcf0 2024-04-27T04:10:50.751 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:10:50.751+0000 7f9d827fc700 1 -- 172.21.15.162:0/3292950423 <== mgr.24367 v2:172.21.15.181:6828/4165336392 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3388 (secure 0 0 0) 0x7f9d8c0fb7d0 con 0x7f9d7806bcf0 2024-04-27T04:10:50.751 INFO:teuthology.orchestra.run.smithi162.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-27T04:10:50.751 INFO:teuthology.orchestra.run.smithi162.stdout:alertmanager.smithi162 smithi162 *:9093,9094 starting - - - - 2024-04-27T04:10:50.751 INFO:teuthology.orchestra.run.smithi162.stdout:crash.smithi162 smithi162 running (9m) 23s ago 9m 6660k - 17.2.7-907-g2be3efa9 1c5834633715 228c98df4289 2024-04-27T04:10:50.751 INFO:teuthology.orchestra.run.smithi162.stdout:crash.smithi181 smithi181 running (8m) 26s ago 8m 6665k - 17.2.7-907-g2be3efa9 1c5834633715 c8e553e3df2b 2024-04-27T04:10:50.751 INFO:teuthology.orchestra.run.smithi162.stdout:grafana.smithi162 smithi162 *:3000 starting - - - - 2024-04-27T04:10:50.751 INFO:teuthology.orchestra.run.smithi162.stdout:mds.cephfs.smithi162.gzlyjj smithi162 running (3m) 23s ago 3m 11.9M - 17.2.7-907-g2be3efa9 1c5834633715 27e5b030dd32 2024-04-27T04:10:50.751 INFO:teuthology.orchestra.run.smithi162.stdout:mds.cephfs.smithi162.xcnapk smithi162 running (3m) 23s ago 3m 126M - 17.2.7-907-g2be3efa9 1c5834633715 38f9030df029 2024-04-27T04:10:50.751 INFO:teuthology.orchestra.run.smithi162.stdout:mds.cephfs.smithi181.bkkcdm smithi181 running (3m) 26s ago 3m 13.6M - 17.2.7-907-g2be3efa9 1c5834633715 af55afca1263 2024-04-27T04:10:50.751 INFO:teuthology.orchestra.run.smithi162.stdout:mds.cephfs.smithi181.dzomgh smithi181 running (3m) 26s ago 3m 206M - 17.2.7-907-g2be3efa9 1c5834633715 6733296597f3 2024-04-27T04:10:50.752 INFO:teuthology.orchestra.run.smithi162.stdout:mgr.smithi162.oennfn smithi162 *:9283 running (10m) 23s ago 10m 428M - 17.2.7-907-g2be3efa9 1c5834633715 e768e27ad56e 2024-04-27T04:10:50.752 INFO:teuthology.orchestra.run.smithi162.stdout:mgr.smithi181.vtwuhz smithi181 *:8443,9283 running (82s) 26s ago 8m 491M - 19.0.0-3253-g4e213a58 67e414ff4bc4 402cc52dbdbb 2024-04-27T04:10:50.752 INFO:teuthology.orchestra.run.smithi162.stdout:mon.smithi162 smithi162 running (10m) 23s ago 10m 53.9M 2048M 17.2.7-907-g2be3efa9 1c5834633715 0f1e3682b9ab 2024-04-27T04:10:50.752 INFO:teuthology.orchestra.run.smithi162.stdout:mon.smithi181 smithi181 running (8m) 26s ago 8m 40.1M 2048M 17.2.7-907-g2be3efa9 1c5834633715 96e48aa3912f 2024-04-27T04:10:50.752 INFO:teuthology.orchestra.run.smithi162.stdout:node-exporter.smithi162 smithi162 *:9100 starting - - - - 2024-04-27T04:10:50.752 INFO:teuthology.orchestra.run.smithi162.stdout:node-exporter.smithi181 smithi181 *:9100 running (8m) 26s ago 8m 14.8M - 1.5.0 0da6a335fe13 3cc278142657 2024-04-27T04:10:50.752 INFO:teuthology.orchestra.run.smithi162.stdout:osd.0 smithi162 running (7m) 23s ago 7m 234M 1557M 17.2.7-907-g2be3efa9 1c5834633715 4a9df3aa938d 2024-04-27T04:10:50.752 INFO:teuthology.orchestra.run.smithi162.stdout:osd.1 smithi162 running (7m) 23s ago 7m 233M 1557M 17.2.7-907-g2be3efa9 1c5834633715 aefb464c46e8 2024-04-27T04:10:50.752 INFO:teuthology.orchestra.run.smithi162.stdout:osd.2 smithi162 running (6m) 23s ago 6m 184M 1557M 17.2.7-907-g2be3efa9 1c5834633715 70f16d86d38e 2024-04-27T04:10:50.752 INFO:teuthology.orchestra.run.smithi162.stdout:osd.3 smithi181 running (6m) 26s ago 6m 258M 2581M 17.2.7-907-g2be3efa9 1c5834633715 030d829be54e 2024-04-27T04:10:50.752 INFO:teuthology.orchestra.run.smithi162.stdout:osd.4 smithi181 running (5m) 26s ago 5m 233M 2581M 17.2.7-907-g2be3efa9 1c5834633715 512afbee7cf7 2024-04-27T04:10:50.752 INFO:teuthology.orchestra.run.smithi162.stdout:osd.5 smithi181 running (5m) 26s ago 5m 268M 2581M 17.2.7-907-g2be3efa9 1c5834633715 f47a1daac1ea 2024-04-27T04:10:50.752 INFO:teuthology.orchestra.run.smithi162.stdout:prometheus.smithi162 smithi162 *:9095 running (7m) 23s ago 8m 59.0M - 2.43.0 a07b618ecd1d 63755cc04fec 2024-04-27T04:10:50.753 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:10:50.753+0000 7f9d94158700 1 -- 172.21.15.162:0/3292950423 >> [v2:172.21.15.181:6828/4165336392,v1:172.21.15.181:6829/4165336392] conn(0x7f9d7806bcf0 msgr2=0x7f9d7806e1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:10:50.753 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:10:50.753+0000 7f9d94158700 1 --2- 172.21.15.162:0/3292950423 >> [v2:172.21.15.181:6828/4165336392,v1:172.21.15.181:6829/4165336392] conn(0x7f9d7806bcf0 0x7f9d7806e1a0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f9d88009b30 tx=0x7f9d88009e30 comp rx=0 tx=0).stop 2024-04-27T04:10:50.753 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:10:50.753+0000 7f9d94158700 1 -- 172.21.15.162:0/3292950423 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9d8c0fcbc0 msgr2=0x7f9d8c1100c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:10:50.753 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:10:50.753+0000 7f9d94158700 1 --2- 172.21.15.162:0/3292950423 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9d8c0fcbc0 0x7f9d8c1100c0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f9d7c00eb40 tx=0x7f9d7c00b040 comp rx=0 tx=0).stop 2024-04-27T04:10:50.753 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:10:50.753+0000 7f9d94158700 1 -- 172.21.15.162:0/3292950423 shutdown_connections 2024-04-27T04:10:50.753 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:10:50.753+0000 7f9d94158700 1 --2- 172.21.15.162:0/3292950423 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9d8c0fc270 0x7f9d8c10fb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:10:50.753 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:10:50.753+0000 7f9d94158700 1 --2- 172.21.15.162:0/3292950423 >> [v2:172.21.15.181:6828/4165336392,v1:172.21.15.181:6829/4165336392] conn(0x7f9d7806bcf0 0x7f9d7806e1a0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:10:50.753 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:10:50.753+0000 7f9d94158700 1 --2- 172.21.15.162:0/3292950423 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9d8c0fcbc0 0x7f9d8c1100c0 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:10:50.754 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:10:50.753+0000 7f9d94158700 1 -- 172.21.15.162:0/3292950423 >> 172.21.15.162:0/3292950423 conn(0x7f9d8c0f6b80 msgr2=0x7f9d8c0f8200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:10:50.754 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:10:50.753+0000 7f9d94158700 1 -- 172.21.15.162:0/3292950423 shutdown_connections 2024-04-27T04:10:50.754 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:10:50.753+0000 7f9d94158700 1 -- 172.21.15.162:0/3292950423 wait complete. 2024-04-27T04:10:50.859 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:10:50.858+0000 7f1f2d7ee700 1 -- 172.21.15.162:0/595668186 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1f280662c0 msgr2=0x7f1f280ff9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:10:50.859 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:10:50.858+0000 7f1f2d7ee700 1 --2- 172.21.15.162:0/595668186 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1f280662c0 0x7f1f280ff9a0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f1f18004940 tx=0x7f1f18009e30 comp rx=0 tx=0).stop 2024-04-27T04:10:50.859 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:10:50.858+0000 7f1f2d7ee700 1 -- 172.21.15.162:0/595668186 shutdown_connections 2024-04-27T04:10:50.859 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:10:50.858+0000 7f1f2d7ee700 1 --2- 172.21.15.162:0/595668186 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1f280ffee0 0x7f1f28102300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:10:50.859 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:10:50.858+0000 7f1f2d7ee700 1 --2- 172.21.15.162:0/595668186 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1f280662c0 0x7f1f280ff9a0 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:10:50.859 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:10:50.858+0000 7f1f2d7ee700 1 -- 172.21.15.162:0/595668186 >> 172.21.15.162:0/595668186 conn(0x7f1f280f7430 msgr2=0x7f1f280f9840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:10:50.859 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:10:50.858+0000 7f1f2d7ee700 1 -- 172.21.15.162:0/595668186 shutdown_connections 2024-04-27T04:10:50.859 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:10:50.858+0000 7f1f2d7ee700 1 -- 172.21.15.162:0/595668186 wait complete. 2024-04-27T04:10:50.860 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:10:50.858+0000 7f1f2d7ee700 1 Processor -- start 2024-04-27T04:10:50.860 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:10:50.858+0000 7f1f2d7ee700 1 -- start start 2024-04-27T04:10:50.860 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:10:50.859+0000 7f1f2d7ee700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1f280662c0 0x7f1f2810d9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:10:50.860 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:10:50.859+0000 7f1f2d7ee700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1f280ffee0 0x7f1f2810df20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:10:50.860 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:10:50.859+0000 7f1f2d7ee700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f2810e520 con 0x7f1f280ffee0 2024-04-27T04:10:50.860 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:10:50.859+0000 7f1f2d7ee700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f2810ed20 con 0x7f1f280662c0 2024-04-27T04:10:50.860 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:10:50.859+0000 7f1f267fc700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1f280ffee0 0x7f1f2810df20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:10:50.860 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:10:50.859+0000 7f1f267fc700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1f280ffee0 0x7f1f2810df20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:50912/0 (socket says 172.21.15.162:50912) 2024-04-27T04:10:50.860 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:10:50.859+0000 7f1f267fc700 1 -- 172.21.15.162:0/1821615166 learned_addr learned my addr 172.21.15.162:0/1821615166 (peer_addr_for_me v2:172.21.15.162:0/0) 2024-04-27T04:10:50.860 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:10:50.859+0000 7f1f26ffd700 1 --2- 172.21.15.162:0/1821615166 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1f280662c0 0x7f1f2810d9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:10:50.860 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:10:50.859+0000 7f1f267fc700 1 -- 172.21.15.162:0/1821615166 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1f280662c0 msgr2=0x7f1f2810d9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:10:50.860 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:10:50.859+0000 7f1f267fc700 1 --2- 172.21.15.162:0/1821615166 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1f280662c0 0x7f1f2810d9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:10:50.860 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:10:50.859+0000 7f1f267fc700 1 -- 172.21.15.162:0/1821615166 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f18005040 con 0x7f1f280ffee0 2024-04-27T04:10:50.860 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:10:50.859+0000 7f1f267fc700 1 --2- 172.21.15.162:0/1821615166 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1f280ffee0 0x7f1f2810df20 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f1f1c00eb40 tx=0x7f1f1c00b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:10:50.861 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:10:50.859+0000 7f1f0f7fe700 1 -- 172.21.15.162:0/1821615166 <== mon.0 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1f1c007cc0 con 0x7f1f280ffee0 2024-04-27T04:10:50.861 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:10:50.859+0000 7f1f2d7ee700 1 -- 172.21.15.162:0/1821615166 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1f2810efa0 con 0x7f1f280ffee0 2024-04-27T04:10:50.861 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:10:50.859+0000 7f1f0f7fe700 1 -- 172.21.15.162:0/1821615166 <== mon.0 v2:172.21.15.162:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f1f1c00f040 con 0x7f1f280ffee0 2024-04-27T04:10:50.861 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:10:50.859+0000 7f1f0f7fe700 1 -- 172.21.15.162:0/1821615166 <== mon.0 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1f1c00ca10 con 0x7f1f280ffee0 2024-04-27T04:10:50.861 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:10:50.859+0000 7f1f2d7ee700 1 -- 172.21.15.162:0/1821615166 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1f2810f4e0 con 0x7f1f280ffee0 2024-04-27T04:10:50.862 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:10:50.860+0000 7f1f2d7ee700 1 -- 172.21.15.162:0/1821615166 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1f2805f5c0 con 0x7f1f280ffee0 2024-04-27T04:10:50.862 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:10:50.861+0000 7f1f0f7fe700 1 -- 172.21.15.162:0/1821615166 <== mon.0 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 24) v1 ==== 89652+0+0 (secure 0 0 0) 0x7f1f1c00cc80 con 0x7f1f280ffee0 2024-04-27T04:10:50.862 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:10:50.861+0000 7f1f0f7fe700 1 --2- 172.21.15.162:0/1821615166 >> [v2:172.21.15.181:6828/4165336392,v1:172.21.15.181:6829/4165336392] conn(0x7f1f1006bcf0 0x7f1f1006e1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:10:50.862 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:10:50.861+0000 7f1f0f7fe700 1 -- 172.21.15.162:0/1821615166 <== mon.0 v2:172.21.15.162:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5808+0+0 (secure 0 0 0) 0x7f1f1c08a6d0 con 0x7f1f280ffee0 2024-04-27T04:10:50.862 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:10:50.861+0000 7f1f26ffd700 1 --2- 172.21.15.162:0/1821615166 >> [v2:172.21.15.181:6828/4165336392,v1:172.21.15.181:6829/4165336392] conn(0x7f1f1006bcf0 0x7f1f1006e1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:10:50.862 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:10:50.862+0000 7f1f26ffd700 1 --2- 172.21.15.162:0/1821615166 >> [v2:172.21.15.181:6828/4165336392,v1:172.21.15.181:6829/4165336392] conn(0x7f1f1006bcf0 0x7f1f1006e1a0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f1f18004890 tx=0x7f1f18009e30 comp rx=0 tx=0).ready entity=mgr.24367 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:10:50.865 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:10:50.866+0000 7f1f0f7fe700 1 -- 172.21.15.162:0/1821615166 <== mon.0 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186652 (secure 0 0 0) 0x7f1f1c050930 con 0x7f1f280ffee0 2024-04-27T04:10:50.922 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:10:50 smithi181 ceph-mon[33774]: from='mgr.24367 172.21.15.181:0/3181919980' entity='mgr.smithi181.vtwuhz' 2024-04-27T04:10:50.922 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:10:50 smithi181 ceph-mon[33774]: from='mgr.24367 172.21.15.181:0/3181919980' entity='mgr.smithi181.vtwuhz' 2024-04-27T04:10:50.922 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:10:50 smithi181 ceph-mon[33774]: Reconfiguring prometheus.smithi162 (dependencies changed)... 2024-04-27T04:10:50.922 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:10:50 smithi181 ceph-mon[33774]: Reconfiguring daemon prometheus.smithi162 on smithi162 2024-04-27T04:10:50.922 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:10:50 smithi181 ceph-mon[33774]: pgmap v25: 65 pgs: 65 active+clean; 279 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 169 op/s 2024-04-27T04:10:50.922 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:10:50 smithi181 ceph-mon[33774]: from='client.24435 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-27T04:10:51.236 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:10:51.236+0000 7f1f2d7ee700 1 -- 172.21.15.162:0/1821615166 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f1f2804bed0 con 0x7f1f280ffee0 2024-04-27T04:10:51.236 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:10:51.236+0000 7f1f0f7fe700 1 -- 172.21.15.162:0/1821615166 <== mon.0 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+846 (secure 0 0 0) 0x7f1f1c0577f0 con 0x7f1f280ffee0 2024-04-27T04:10:51.236 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-04-27T04:10:51.236 INFO:teuthology.orchestra.run.smithi162.stdout: "mon": { 2024-04-27T04:10:51.236 INFO:teuthology.orchestra.run.smithi162.stdout: "ceph version 17.2.7-907-g2be3efa9 (2be3efa97a7972f8e925bd899b474329979b74d6) quincy (stable)": 2 2024-04-27T04:10:51.236 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-04-27T04:10:51.237 INFO:teuthology.orchestra.run.smithi162.stdout: "mgr": { 2024-04-27T04:10:51.237 INFO:teuthology.orchestra.run.smithi162.stdout: "ceph version 17.2.7-907-g2be3efa9 (2be3efa97a7972f8e925bd899b474329979b74d6) quincy (stable)": 1, 2024-04-27T04:10:51.237 INFO:teuthology.orchestra.run.smithi162.stdout: "ceph version 19.0.0-3253-g4e213a58 (4e213a58e787437ff508e6c890ccd5fd32dc96b5) squid (dev)": 1 2024-04-27T04:10:51.237 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-04-27T04:10:51.237 INFO:teuthology.orchestra.run.smithi162.stdout: "osd": { 2024-04-27T04:10:51.237 INFO:teuthology.orchestra.run.smithi162.stdout: "ceph version 17.2.7-907-g2be3efa9 (2be3efa97a7972f8e925bd899b474329979b74d6) quincy (stable)": 6 2024-04-27T04:10:51.237 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-04-27T04:10:51.237 INFO:teuthology.orchestra.run.smithi162.stdout: "mds": { 2024-04-27T04:10:51.237 INFO:teuthology.orchestra.run.smithi162.stdout: "ceph version 17.2.7-907-g2be3efa9 (2be3efa97a7972f8e925bd899b474329979b74d6) quincy (stable)": 4 2024-04-27T04:10:51.237 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-04-27T04:10:51.237 INFO:teuthology.orchestra.run.smithi162.stdout: "overall": { 2024-04-27T04:10:51.237 INFO:teuthology.orchestra.run.smithi162.stdout: "ceph version 17.2.7-907-g2be3efa9 (2be3efa97a7972f8e925bd899b474329979b74d6) quincy (stable)": 13, 2024-04-27T04:10:51.237 INFO:teuthology.orchestra.run.smithi162.stdout: "ceph version 19.0.0-3253-g4e213a58 (4e213a58e787437ff508e6c890ccd5fd32dc96b5) squid (dev)": 1 2024-04-27T04:10:51.237 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-04-27T04:10:51.237 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-04-27T04:10:51.239 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:10:51.239+0000 7f1f2d7ee700 1 -- 172.21.15.162:0/1821615166 >> [v2:172.21.15.181:6828/4165336392,v1:172.21.15.181:6829/4165336392] conn(0x7f1f1006bcf0 msgr2=0x7f1f1006e1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:10:51.239 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:10:51.239+0000 7f1f2d7ee700 1 --2- 172.21.15.162:0/1821615166 >> [v2:172.21.15.181:6828/4165336392,v1:172.21.15.181:6829/4165336392] conn(0x7f1f1006bcf0 0x7f1f1006e1a0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f1f18004890 tx=0x7f1f18009e30 comp rx=0 tx=0).stop 2024-04-27T04:10:51.239 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:10:51.239+0000 7f1f2d7ee700 1 -- 172.21.15.162:0/1821615166 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1f280ffee0 msgr2=0x7f1f2810df20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:10:51.239 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:10:51.239+0000 7f1f2d7ee700 1 --2- 172.21.15.162:0/1821615166 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1f280ffee0 0x7f1f2810df20 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f1f1c00eb40 tx=0x7f1f1c00b040 comp rx=0 tx=0).stop 2024-04-27T04:10:51.239 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:10:51.239+0000 7f1f2d7ee700 1 -- 172.21.15.162:0/1821615166 shutdown_connections 2024-04-27T04:10:51.239 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:10:51.239+0000 7f1f2d7ee700 1 --2- 172.21.15.162:0/1821615166 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1f280662c0 0x7f1f2810d9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:10:51.239 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:10:51.239+0000 7f1f2d7ee700 1 --2- 172.21.15.162:0/1821615166 >> [v2:172.21.15.181:6828/4165336392,v1:172.21.15.181:6829/4165336392] conn(0x7f1f1006bcf0 0x7f1f1006e1a0 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:10:51.239 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:10:51.239+0000 7f1f2d7ee700 1 --2- 172.21.15.162:0/1821615166 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1f280ffee0 0x7f1f2810df20 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:10:51.239 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:10:51.239+0000 7f1f2d7ee700 1 -- 172.21.15.162:0/1821615166 >> 172.21.15.162:0/1821615166 conn(0x7f1f280f7430 msgr2=0x7f1f280f7800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:10:51.240 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:10:51.239+0000 7f1f2d7ee700 1 -- 172.21.15.162:0/1821615166 shutdown_connections 2024-04-27T04:10:51.240 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:10:51.239+0000 7f1f2d7ee700 1 -- 172.21.15.162:0/1821615166 wait complete. 2024-04-27T04:10:51.352 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:10:51.351+0000 7f9bbf2ad700 1 -- 172.21.15.162:0/1315732335 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9bb80fdcd0 msgr2=0x7f9bb8100150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:10:51.352 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:10:51.351+0000 7f9bbf2ad700 1 --2- 172.21.15.162:0/1315732335 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9bb80fdcd0 0x7f9bb8100150 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f9bac004940 tx=0x7f9bac009e30 comp rx=0 tx=0).stop 2024-04-27T04:10:51.352 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:10:51.351+0000 7f9bbf2ad700 1 -- 172.21.15.162:0/1315732335 shutdown_connections 2024-04-27T04:10:51.352 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:10:51.351+0000 7f9bbf2ad700 1 --2- 172.21.15.162:0/1315732335 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9bb80fdcd0 0x7f9bb8100150 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:10:51.352 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:10:51.351+0000 7f9bbf2ad700 1 --2- 172.21.15.162:0/1315732335 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9bb80662c0 0x7f9bb80fd790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:10:51.352 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:10:51.351+0000 7f9bbf2ad700 1 -- 172.21.15.162:0/1315732335 >> 172.21.15.162:0/1315732335 conn(0x7f9bb80f7430 msgr2=0x7f9bb80f9840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:10:51.353 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:10:51.351+0000 7f9bbf2ad700 1 -- 172.21.15.162:0/1315732335 shutdown_connections 2024-04-27T04:10:51.353 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:10:51.351+0000 7f9bbf2ad700 1 -- 172.21.15.162:0/1315732335 wait complete. 2024-04-27T04:10:51.353 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:10:51.351+0000 7f9bbf2ad700 1 Processor -- start 2024-04-27T04:10:51.353 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:10:51.351+0000 7f9bbf2ad700 1 -- start start 2024-04-27T04:10:51.353 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:10:51.352+0000 7f9bbf2ad700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9bb80662c0 0x7f9bb8113f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:10:51.353 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:10:51.352+0000 7f9bbf2ad700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9bb80fdcd0 0x7f9bb81144d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:10:51.353 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:10:51.352+0000 7f9bbf2ad700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9bb8114ad0 con 0x7f9bb80fdcd0 2024-04-27T04:10:51.353 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:10:51.352+0000 7f9bbf2ad700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9bb8115320 con 0x7f9bb80662c0 2024-04-27T04:10:51.353 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:10:51.352+0000 7f9bbc848700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9bb80fdcd0 0x7f9bb81144d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:10:51.353 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:10:51.352+0000 7f9bbc848700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9bb80fdcd0 0x7f9bb81144d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:50934/0 (socket says 172.21.15.162:50934) 2024-04-27T04:10:51.353 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:10:51.352+0000 7f9bbc848700 1 -- 172.21.15.162:0/760306221 learned_addr learned my addr 172.21.15.162:0/760306221 (peer_addr_for_me v2:172.21.15.162:0/0) 2024-04-27T04:10:51.353 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:10:51.352+0000 7f9bbd049700 1 --2- 172.21.15.162:0/760306221 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9bb80662c0 0x7f9bb8113f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:10:51.353 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:10:51.352+0000 7f9bbc848700 1 -- 172.21.15.162:0/760306221 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9bb80662c0 msgr2=0x7f9bb8113f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:10:51.354 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:10:51.352+0000 7f9bbc848700 1 --2- 172.21.15.162:0/760306221 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9bb80662c0 0x7f9bb8113f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:10:51.354 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:10:51.352+0000 7f9bbc848700 1 -- 172.21.15.162:0/760306221 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9bac005040 con 0x7f9bb80fdcd0 2024-04-27T04:10:51.354 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:10:51.352+0000 7f9bbc848700 1 --2- 172.21.15.162:0/760306221 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9bb80fdcd0 0x7f9bb81144d0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f9bac013870 tx=0x7f9bac009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:10:51.354 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:10:51.352+0000 7f9bb5ffb700 1 -- 172.21.15.162:0/760306221 <== mon.0 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9bac021070 con 0x7f9bb80fdcd0 2024-04-27T04:10:51.354 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:10:51.352+0000 7f9bb5ffb700 1 -- 172.21.15.162:0/760306221 <== mon.0 v2:172.21.15.162:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f9bac009430 con 0x7f9bb80fdcd0 2024-04-27T04:10:51.354 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:10:51.352+0000 7f9bbf2ad700 1 -- 172.21.15.162:0/760306221 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9bb81155a0 con 0x7f9bb80fdcd0 2024-04-27T04:10:51.354 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:10:51.352+0000 7f9bb5ffb700 1 -- 172.21.15.162:0/760306221 <== mon.0 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9bac00e070 con 0x7f9bb80fdcd0 2024-04-27T04:10:51.354 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:10:51.353+0000 7f9bbf2ad700 1 -- 172.21.15.162:0/760306221 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9bb8115ae0 con 0x7f9bb80fdcd0 2024-04-27T04:10:51.355 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:10:51.354+0000 7f9bbf2ad700 1 -- 172.21.15.162:0/760306221 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9bb805f5c0 con 0x7f9bb80fdcd0 2024-04-27T04:10:51.355 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:10:51.354+0000 7f9bb5ffb700 1 -- 172.21.15.162:0/760306221 <== mon.0 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 24) v1 ==== 89652+0+0 (secure 0 0 0) 0x7f9bac01c750 con 0x7f9bb80fdcd0 2024-04-27T04:10:51.355 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:10:51.354+0000 7f9bb5ffb700 1 --2- 172.21.15.162:0/760306221 >> [v2:172.21.15.181:6828/4165336392,v1:172.21.15.181:6829/4165336392] conn(0x7f9ba006bcf0 0x7f9ba006e1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:10:51.355 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:10:51.355+0000 7f9bb5ffb700 1 -- 172.21.15.162:0/760306221 <== mon.0 v2:172.21.15.162:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5808+0+0 (secure 0 0 0) 0x7f9bac08bba0 con 0x7f9bb80fdcd0 2024-04-27T04:10:51.355 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:10:51.355+0000 7f9bbd049700 1 --2- 172.21.15.162:0/760306221 >> [v2:172.21.15.181:6828/4165336392,v1:172.21.15.181:6829/4165336392] conn(0x7f9ba006bcf0 0x7f9ba006e1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:10:51.355 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:10:51.355+0000 7f9bbd049700 1 --2- 172.21.15.162:0/760306221 >> [v2:172.21.15.181:6828/4165336392,v1:172.21.15.181:6829/4165336392] conn(0x7f9ba006bcf0 0x7f9ba006e1a0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f9bb0009b80 tx=0x7f9bb000b040 comp rx=0 tx=0).ready entity=mgr.24367 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:10:51.358 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:10:51.359+0000 7f9bb5ffb700 1 -- 172.21.15.162:0/760306221 <== mon.0 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186652 (secure 0 0 0) 0x7f9bac058e70 con 0x7f9bb80fdcd0 2024-04-27T04:10:51.596 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:10:51.596+0000 7f9bbf2ad700 1 -- 172.21.15.162:0/760306221 --> [v2:172.21.15.181:6828/4165336392,v1:172.21.15.181:6829/4165336392] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f9bb805f5c0 con 0x7f9ba006bcf0 2024-04-27T04:10:51.596 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:10:51 smithi162 ceph-mon[26841]: from='client.24439 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-27T04:10:51.596 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:10:51 smithi162 ceph-mon[26841]: from='client.14668 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-27T04:10:51.596 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:10:51 smithi162 ceph-mon[26841]: from='mgr.24367 172.21.15.181:0/3181919980' entity='mgr.smithi181.vtwuhz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-27T04:10:51.596 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:10:51 smithi162 ceph-mon[26841]: from='client.? 172.21.15.162:0/1821615166' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-27T04:10:51.597 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:10:51.597+0000 7f9bb5ffb700 1 -- 172.21.15.162:0/760306221 <== mgr.24367 v2:172.21.15.181:6828/4165336392 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+310 (secure 0 0 0) 0x7f9bb805f5c0 con 0x7f9ba006bcf0 2024-04-27T04:10:51.597 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-04-27T04:10:51.597 INFO:teuthology.orchestra.run.smithi162.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:90a3a9fa9ceb8977f6dad9ccd36fe46018d07e2118199820523a6b35301d9c16", 2024-04-27T04:10:51.597 INFO:teuthology.orchestra.run.smithi162.stdout: "in_progress": true, 2024-04-27T04:10:51.597 INFO:teuthology.orchestra.run.smithi162.stdout: "which": "Upgrading daemons of type(s) mgr", 2024-04-27T04:10:51.597 INFO:teuthology.orchestra.run.smithi162.stdout: "services_complete": [], 2024-04-27T04:10:51.597 INFO:teuthology.orchestra.run.smithi162.stdout: "progress": "1/2 daemons upgraded", 2024-04-27T04:10:51.597 INFO:teuthology.orchestra.run.smithi162.stdout: "message": "", 2024-04-27T04:10:51.597 INFO:teuthology.orchestra.run.smithi162.stdout: "is_paused": false 2024-04-27T04:10:51.597 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-04-27T04:10:51.599 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:10:51.600+0000 7f9bbf2ad700 1 -- 172.21.15.162:0/760306221 >> [v2:172.21.15.181:6828/4165336392,v1:172.21.15.181:6829/4165336392] conn(0x7f9ba006bcf0 msgr2=0x7f9ba006e1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:10:51.600 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:10:51.600+0000 7f9bbf2ad700 1 --2- 172.21.15.162:0/760306221 >> [v2:172.21.15.181:6828/4165336392,v1:172.21.15.181:6829/4165336392] conn(0x7f9ba006bcf0 0x7f9ba006e1a0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f9bb0009b80 tx=0x7f9bb000b040 comp rx=0 tx=0).stop 2024-04-27T04:10:51.600 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:10:51.600+0000 7f9bbf2ad700 1 -- 172.21.15.162:0/760306221 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9bb80fdcd0 msgr2=0x7f9bb81144d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:10:51.600 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:10:51.600+0000 7f9bbf2ad700 1 --2- 172.21.15.162:0/760306221 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9bb80fdcd0 0x7f9bb81144d0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f9bac013870 tx=0x7f9bac009e30 comp rx=0 tx=0).stop 2024-04-27T04:10:51.600 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:10:51.600+0000 7f9bbf2ad700 1 -- 172.21.15.162:0/760306221 shutdown_connections 2024-04-27T04:10:51.600 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:10:51.600+0000 7f9bbf2ad700 1 --2- 172.21.15.162:0/760306221 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9bb80662c0 0x7f9bb8113f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:10:51.600 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:10:51.600+0000 7f9bbf2ad700 1 --2- 172.21.15.162:0/760306221 >> [v2:172.21.15.181:6828/4165336392,v1:172.21.15.181:6829/4165336392] conn(0x7f9ba006bcf0 0x7f9ba006e1a0 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:10:51.600 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:10:51.600+0000 7f9bbf2ad700 1 --2- 172.21.15.162:0/760306221 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9bb80fdcd0 0x7f9bb81144d0 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:10:51.600 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:10:51.600+0000 7f9bbf2ad700 1 -- 172.21.15.162:0/760306221 >> 172.21.15.162:0/760306221 conn(0x7f9bb80f7430 msgr2=0x7f9bb80f7800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:10:51.600 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:10:51.600+0000 7f9bbf2ad700 1 -- 172.21.15.162:0/760306221 shutdown_connections 2024-04-27T04:10:51.600 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:10:51.600+0000 7f9bbf2ad700 1 -- 172.21.15.162:0/760306221 wait complete. 2024-04-27T04:10:51.922 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:10:51 smithi181 ceph-mon[33774]: from='client.24439 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-27T04:10:51.922 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:10:51 smithi181 ceph-mon[33774]: from='client.14668 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-27T04:10:51.922 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:10:51 smithi181 ceph-mon[33774]: from='mgr.24367 172.21.15.181:0/3181919980' entity='mgr.smithi181.vtwuhz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-27T04:10:51.922 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:10:51 smithi181 ceph-mon[33774]: from='client.? 172.21.15.162:0/1821615166' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-27T04:10:52.887 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:10:52 smithi162 ceph-mon[26841]: from='client.14676 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-27T04:10:52.887 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:10:52 smithi162 ceph-mon[26841]: pgmap v26: 65 pgs: 65 active+clean; 283 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 1.7 MiB/s rd, 1.7 MiB/s wr, 310 op/s 2024-04-27T04:10:52.922 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:10:52 smithi181 ceph-mon[33774]: from='client.14676 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-27T04:10:52.922 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:10:52 smithi181 ceph-mon[33774]: pgmap v26: 65 pgs: 65 active+clean; 283 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 1.7 MiB/s rd, 1.7 MiB/s wr, 310 op/s 2024-04-27T04:10:54.822 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:10:54 smithi162 ceph-mon[26841]: pgmap v27: 65 pgs: 65 active+clean; 283 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 1.4 MiB/s rd, 1.4 MiB/s wr, 301 op/s 2024-04-27T04:10:55.172 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:10:54 smithi181 ceph-mon[33774]: pgmap v27: 65 pgs: 65 active+clean; 283 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 1.4 MiB/s rd, 1.4 MiB/s wr, 301 op/s 2024-04-27T04:10:57.137 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:10:56 smithi162 ceph-mon[26841]: pgmap v28: 65 pgs: 65 active+clean; 283 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 1.1 MiB/s rd, 1.2 MiB/s wr, 270 op/s 2024-04-27T04:10:57.172 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:10:56 smithi181 ceph-mon[33774]: pgmap v28: 65 pgs: 65 active+clean; 283 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 1.1 MiB/s rd, 1.2 MiB/s wr, 270 op/s 2024-04-27T04:10:58.137 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:10:57 smithi162 ceph-mon[26841]: from='mgr.24367 172.21.15.181:0/3181919980' entity='mgr.smithi181.vtwuhz' 2024-04-27T04:10:58.137 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:10:57 smithi162 ceph-mon[26841]: from='mgr.24367 172.21.15.181:0/3181919980' entity='mgr.smithi181.vtwuhz' 2024-04-27T04:10:58.137 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:10:57 smithi162 ceph-mon[26841]: Reconfiguring node-exporter.smithi181 (dependencies changed)... 2024-04-27T04:10:58.137 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:10:57 smithi162 ceph-mon[26841]: Reconfiguring daemon node-exporter.smithi181 on smithi181 2024-04-27T04:10:58.172 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:10:57 smithi181 ceph-mon[33774]: from='mgr.24367 172.21.15.181:0/3181919980' entity='mgr.smithi181.vtwuhz' 2024-04-27T04:10:58.172 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:10:57 smithi181 ceph-mon[33774]: from='mgr.24367 172.21.15.181:0/3181919980' entity='mgr.smithi181.vtwuhz' 2024-04-27T04:10:58.172 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:10:57 smithi181 ceph-mon[33774]: Reconfiguring node-exporter.smithi181 (dependencies changed)... 2024-04-27T04:10:58.172 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:10:57 smithi181 ceph-mon[33774]: Reconfiguring daemon node-exporter.smithi181 on smithi181 2024-04-27T04:10:59.137 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:10:58 smithi162 ceph-mon[26841]: pgmap v29: 65 pgs: 65 active+clean; 287 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 1.7 MiB/s rd, 1.8 MiB/s wr, 439 op/s 2024-04-27T04:10:59.172 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:10:58 smithi181 ceph-mon[33774]: pgmap v29: 65 pgs: 65 active+clean; 287 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 1.7 MiB/s rd, 1.8 MiB/s wr, 439 op/s 2024-04-27T04:11:01.137 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:11:00 smithi162 ceph-mon[26841]: pgmap v30: 65 pgs: 65 active+clean; 287 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 1.2 MiB/s rd, 1.3 MiB/s wr, 346 op/s 2024-04-27T04:11:01.172 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:11:00 smithi181 ceph-mon[33774]: pgmap v30: 65 pgs: 65 active+clean; 287 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 1.2 MiB/s rd, 1.3 MiB/s wr, 346 op/s 2024-04-27T04:11:02.999 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:11:02 smithi181 ceph-mon[33774]: pgmap v31: 65 pgs: 65 active+clean; 290 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 1.8 MiB/s rd, 1.8 MiB/s wr, 479 op/s 2024-04-27T04:11:03.137 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:11:02 smithi162 ceph-mon[26841]: pgmap v31: 65 pgs: 65 active+clean; 290 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 1.8 MiB/s rd, 1.8 MiB/s wr, 479 op/s 2024-04-27T04:11:05.137 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:11:04 smithi162 ceph-mon[26841]: pgmap v32: 65 pgs: 65 active+clean; 289 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 1.3 MiB/s rd, 1.2 MiB/s wr, 385 op/s 2024-04-27T04:11:05.172 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:11:04 smithi181 ceph-mon[33774]: pgmap v32: 65 pgs: 65 active+clean; 289 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 1.3 MiB/s rd, 1.2 MiB/s wr, 385 op/s 2024-04-27T04:11:06.137 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:11:05 smithi162 ceph-mon[26841]: from='mgr.24367 172.21.15.181:0/3181919980' entity='mgr.smithi181.vtwuhz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-27T04:11:06.172 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:11:05 smithi181 ceph-mon[33774]: from='mgr.24367 172.21.15.181:0/3181919980' entity='mgr.smithi181.vtwuhz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-27T04:11:07.078 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:11:06 smithi181 ceph-mon[33774]: pgmap v33: 65 pgs: 65 active+clean; 289 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 1.2 MiB/s rd, 1.2 MiB/s wr, 348 op/s 2024-04-27T04:11:07.137 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:11:06 smithi162 ceph-mon[26841]: pgmap v33: 65 pgs: 65 active+clean; 289 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 1.2 MiB/s rd, 1.2 MiB/s wr, 348 op/s 2024-04-27T04:11:09.137 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:11:08 smithi162 ceph-mon[26841]: pgmap v34: 65 pgs: 65 active+clean; 289 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 1.6 MiB/s rd, 1.7 MiB/s wr, 478 op/s 2024-04-27T04:11:09.172 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:11:08 smithi181 ceph-mon[33774]: pgmap v34: 65 pgs: 65 active+clean; 289 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 1.6 MiB/s rd, 1.7 MiB/s wr, 478 op/s 2024-04-27T04:11:11.137 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:11:10 smithi162 ceph-mon[26841]: pgmap v35: 65 pgs: 65 active+clean; 289 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 1.0 MiB/s rd, 1.1 MiB/s wr, 309 op/s 2024-04-27T04:11:11.172 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:11:10 smithi181 ceph-mon[33774]: pgmap v35: 65 pgs: 65 active+clean; 289 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 1.0 MiB/s rd, 1.1 MiB/s wr, 309 op/s 2024-04-27T04:11:12.638 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:11:12 smithi162 ceph-mon[26841]: from='mgr.24367 172.21.15.181:0/3181919980' entity='mgr.smithi181.vtwuhz' 2024-04-27T04:11:12.638 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:11:12 smithi162 ceph-mon[26841]: from='mgr.24367 172.21.15.181:0/3181919980' entity='mgr.smithi181.vtwuhz' 2024-04-27T04:11:12.638 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:11:12 smithi162 ceph-mon[26841]: from='mgr.24367 172.21.15.181:0/3181919980' entity='mgr.smithi181.vtwuhz' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-04-27T04:11:12.638 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:11:12 smithi162 ceph-mon[26841]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-04-27T04:11:12.638 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:11:12 smithi162 ceph-mon[26841]: from='mgr.24367 172.21.15.181:0/3181919980' entity='mgr.smithi181.vtwuhz' cmd=[{"prefix": "dashboard set-alertmanager-api-host", "value": "http://smithi162.front.sepia.ceph.com:9093"}]: dispatch 2024-04-27T04:11:12.638 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:11:12 smithi162 ceph-mon[26841]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-alertmanager-api-host", "value": "http://smithi162.front.sepia.ceph.com:9093"}]: dispatch 2024-04-27T04:11:12.638 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:11:12 smithi162 ceph-mon[26841]: from='mgr.24367 172.21.15.181:0/3181919980' entity='mgr.smithi181.vtwuhz' 2024-04-27T04:11:12.638 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:11:12 smithi162 ceph-mon[26841]: from='mgr.24367 172.21.15.181:0/3181919980' entity='mgr.smithi181.vtwuhz' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-04-27T04:11:12.639 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:11:12 smithi162 ceph-mon[26841]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-04-27T04:11:12.639 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:11:12 smithi162 ceph-mon[26841]: from='mgr.24367 172.21.15.181:0/3181919980' entity='mgr.smithi181.vtwuhz' cmd=[{"prefix": "dashboard set-grafana-api-url", "value": "https://smithi162.front.sepia.ceph.com:3000"}]: dispatch 2024-04-27T04:11:12.639 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:11:12 smithi162 ceph-mon[26841]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-grafana-api-url", "value": "https://smithi162.front.sepia.ceph.com:3000"}]: dispatch 2024-04-27T04:11:12.639 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:11:12 smithi162 ceph-mon[26841]: from='mgr.24367 172.21.15.181:0/3181919980' entity='mgr.smithi181.vtwuhz' 2024-04-27T04:11:12.639 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:11:12 smithi162 ceph-mon[26841]: from='mgr.24367 172.21.15.181:0/3181919980' entity='mgr.smithi181.vtwuhz' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-04-27T04:11:12.639 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:11:12 smithi162 ceph-mon[26841]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-04-27T04:11:12.639 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:11:12 smithi162 ceph-mon[26841]: from='mgr.24367 172.21.15.181:0/3181919980' entity='mgr.smithi181.vtwuhz' cmd=[{"prefix": "dashboard set-prometheus-api-host", "value": "http://smithi162.front.sepia.ceph.com:9095"}]: dispatch 2024-04-27T04:11:12.639 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:11:12 smithi162 ceph-mon[26841]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-prometheus-api-host", "value": "http://smithi162.front.sepia.ceph.com:9095"}]: dispatch 2024-04-27T04:11:12.639 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:11:12 smithi162 ceph-mon[26841]: from='mgr.24367 172.21.15.181:0/3181919980' entity='mgr.smithi181.vtwuhz' 2024-04-27T04:11:12.639 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:11:12 smithi162 ceph-mon[26841]: from='mgr.24367 172.21.15.181:0/3181919980' entity='mgr.smithi181.vtwuhz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T04:11:12.639 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:11:12 smithi162 ceph-mon[26841]: Upgrade: Need to upgrade myself (mgr.smithi181.vtwuhz) 2024-04-27T04:11:12.639 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:11:12 smithi162 ceph-mon[26841]: Upgrade: Need to upgrade myself (mgr.smithi181.vtwuhz) 2024-04-27T04:11:12.639 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:11:12 smithi162 ceph-mon[26841]: pgmap v36: 65 pgs: 65 active+clean; 286 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 1.6 MiB/s rd, 1.5 MiB/s wr, 406 op/s 2024-04-27T04:11:12.672 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:11:12 smithi181 ceph-mon[33774]: from='mgr.24367 172.21.15.181:0/3181919980' entity='mgr.smithi181.vtwuhz' 2024-04-27T04:11:12.673 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:11:12 smithi181 ceph-mon[33774]: from='mgr.24367 172.21.15.181:0/3181919980' entity='mgr.smithi181.vtwuhz' 2024-04-27T04:11:12.673 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:11:12 smithi181 ceph-mon[33774]: from='mgr.24367 172.21.15.181:0/3181919980' entity='mgr.smithi181.vtwuhz' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-04-27T04:11:12.673 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:11:12 smithi181 ceph-mon[33774]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-04-27T04:11:12.673 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:11:12 smithi181 ceph-mon[33774]: from='mgr.24367 172.21.15.181:0/3181919980' entity='mgr.smithi181.vtwuhz' cmd=[{"prefix": "dashboard set-alertmanager-api-host", "value": "http://smithi162.front.sepia.ceph.com:9093"}]: dispatch 2024-04-27T04:11:12.673 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:11:12 smithi181 ceph-mon[33774]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-alertmanager-api-host", "value": "http://smithi162.front.sepia.ceph.com:9093"}]: dispatch 2024-04-27T04:11:12.673 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:11:12 smithi181 ceph-mon[33774]: from='mgr.24367 172.21.15.181:0/3181919980' entity='mgr.smithi181.vtwuhz' 2024-04-27T04:11:12.673 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:11:12 smithi181 ceph-mon[33774]: from='mgr.24367 172.21.15.181:0/3181919980' entity='mgr.smithi181.vtwuhz' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-04-27T04:11:12.673 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:11:12 smithi181 ceph-mon[33774]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-04-27T04:11:12.673 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:11:12 smithi181 ceph-mon[33774]: from='mgr.24367 172.21.15.181:0/3181919980' entity='mgr.smithi181.vtwuhz' cmd=[{"prefix": "dashboard set-grafana-api-url", "value": "https://smithi162.front.sepia.ceph.com:3000"}]: dispatch 2024-04-27T04:11:12.673 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:11:12 smithi181 ceph-mon[33774]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-grafana-api-url", "value": "https://smithi162.front.sepia.ceph.com:3000"}]: dispatch 2024-04-27T04:11:12.673 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:11:12 smithi181 ceph-mon[33774]: from='mgr.24367 172.21.15.181:0/3181919980' entity='mgr.smithi181.vtwuhz' 2024-04-27T04:11:12.673 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:11:12 smithi181 ceph-mon[33774]: from='mgr.24367 172.21.15.181:0/3181919980' entity='mgr.smithi181.vtwuhz' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-04-27T04:11:12.674 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:11:12 smithi181 ceph-mon[33774]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-04-27T04:11:12.674 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:11:12 smithi181 ceph-mon[33774]: from='mgr.24367 172.21.15.181:0/3181919980' entity='mgr.smithi181.vtwuhz' cmd=[{"prefix": "dashboard set-prometheus-api-host", "value": "http://smithi162.front.sepia.ceph.com:9095"}]: dispatch 2024-04-27T04:11:12.674 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:11:12 smithi181 ceph-mon[33774]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-prometheus-api-host", "value": "http://smithi162.front.sepia.ceph.com:9095"}]: dispatch 2024-04-27T04:11:12.674 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:11:12 smithi181 ceph-mon[33774]: from='mgr.24367 172.21.15.181:0/3181919980' entity='mgr.smithi181.vtwuhz' 2024-04-27T04:11:12.674 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:11:12 smithi181 ceph-mon[33774]: from='mgr.24367 172.21.15.181:0/3181919980' entity='mgr.smithi181.vtwuhz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T04:11:12.674 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:11:12 smithi181 ceph-mon[33774]: Upgrade: Need to upgrade myself (mgr.smithi181.vtwuhz) 2024-04-27T04:11:12.674 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:11:12 smithi181 ceph-mon[33774]: Upgrade: Need to upgrade myself (mgr.smithi181.vtwuhz) 2024-04-27T04:11:12.674 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:11:12 smithi181 ceph-mon[33774]: pgmap v36: 65 pgs: 65 active+clean; 286 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 1.6 MiB/s rd, 1.5 MiB/s wr, 406 op/s 2024-04-27T04:11:15.353 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:11:15 smithi162 ceph-mon[26841]: pgmap v37: 65 pgs: 65 active+clean; 282 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 1.1 MiB/s rd, 1.2 MiB/s wr, 300 op/s 2024-04-27T04:11:15.353 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:11:15 smithi162 ceph-mon[26841]: Upgrade: Updating mgr.smithi162.oennfn 2024-04-27T04:11:15.353 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:11:15 smithi162 ceph-mon[26841]: from='mgr.24367 172.21.15.181:0/3181919980' entity='mgr.smithi181.vtwuhz' 2024-04-27T04:11:15.353 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:11:15 smithi162 ceph-mon[26841]: from='mgr.24367 172.21.15.181:0/3181919980' entity='mgr.smithi181.vtwuhz' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi162.oennfn", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-27T04:11:15.353 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:11:15 smithi162 ceph-mon[26841]: from='mgr.24367 172.21.15.181:0/3181919980' entity='mgr.smithi181.vtwuhz' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-27T04:11:15.353 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:11:15 smithi162 ceph-mon[26841]: from='mgr.24367 172.21.15.181:0/3181919980' entity='mgr.smithi181.vtwuhz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T04:11:15.353 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:11:15 smithi162 ceph-mon[26841]: Deploying daemon mgr.smithi162.oennfn on smithi162 2024-04-27T04:11:15.422 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:11:15 smithi181 ceph-mon[33774]: pgmap v37: 65 pgs: 65 active+clean; 282 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 1.1 MiB/s rd, 1.2 MiB/s wr, 300 op/s 2024-04-27T04:11:15.422 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:11:15 smithi181 ceph-mon[33774]: Upgrade: Updating mgr.smithi162.oennfn 2024-04-27T04:11:15.422 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:11:15 smithi181 ceph-mon[33774]: from='mgr.24367 172.21.15.181:0/3181919980' entity='mgr.smithi181.vtwuhz' 2024-04-27T04:11:15.423 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:11:15 smithi181 ceph-mon[33774]: from='mgr.24367 172.21.15.181:0/3181919980' entity='mgr.smithi181.vtwuhz' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi162.oennfn", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-27T04:11:15.423 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:11:15 smithi181 ceph-mon[33774]: from='mgr.24367 172.21.15.181:0/3181919980' entity='mgr.smithi181.vtwuhz' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-27T04:11:15.423 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:11:15 smithi181 ceph-mon[33774]: from='mgr.24367 172.21.15.181:0/3181919980' entity='mgr.smithi181.vtwuhz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T04:11:15.423 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:11:15 smithi181 ceph-mon[33774]: Deploying daemon mgr.smithi162.oennfn on smithi162 2024-04-27T04:11:17.387 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:11:17 smithi162 ceph-mon[26841]: pgmap v38: 65 pgs: 65 active+clean; 282 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 1.0 MiB/s rd, 1.1 MiB/s wr, 254 op/s 2024-04-27T04:11:17.422 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:11:17 smithi181 ceph-mon[33774]: pgmap v38: 65 pgs: 65 active+clean; 282 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 1.0 MiB/s rd, 1.1 MiB/s wr, 254 op/s 2024-04-27T04:11:19.196 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:11:19 smithi162 ceph-mon[26841]: pgmap v39: 65 pgs: 65 active+clean; 273 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 1.7 MiB/s rd, 1.9 MiB/s wr, 760 op/s 2024-04-27T04:11:19.422 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:11:19 smithi181 ceph-mon[33774]: pgmap v39: 65 pgs: 65 active+clean; 273 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 1.7 MiB/s rd, 1.9 MiB/s wr, 760 op/s 2024-04-27T04:11:20.637 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:11:20 smithi162 ceph-mon[26841]: from='mgr.24367 172.21.15.181:0/3181919980' entity='mgr.smithi181.vtwuhz' 2024-04-27T04:11:20.637 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:11:20 smithi162 ceph-mon[26841]: from='mgr.24367 172.21.15.181:0/3181919980' entity='mgr.smithi181.vtwuhz' 2024-04-27T04:11:20.637 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:11:20 smithi162 ceph-mon[26841]: from='mgr.24367 172.21.15.181:0/3181919980' entity='mgr.smithi181.vtwuhz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T04:11:20.637 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:11:20 smithi162 ceph-mon[26841]: from='mgr.24367 172.21.15.181:0/3181919980' entity='mgr.smithi181.vtwuhz' 2024-04-27T04:11:20.638 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:11:20 smithi162 ceph-mon[26841]: from='mgr.24367 172.21.15.181:0/3181919980' entity='mgr.smithi181.vtwuhz' 2024-04-27T04:11:20.672 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:11:20 smithi181 ceph-mon[33774]: from='mgr.24367 172.21.15.181:0/3181919980' entity='mgr.smithi181.vtwuhz' 2024-04-27T04:11:20.672 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:11:20 smithi181 ceph-mon[33774]: from='mgr.24367 172.21.15.181:0/3181919980' entity='mgr.smithi181.vtwuhz' 2024-04-27T04:11:20.673 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:11:20 smithi181 ceph-mon[33774]: from='mgr.24367 172.21.15.181:0/3181919980' entity='mgr.smithi181.vtwuhz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T04:11:20.673 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:11:20 smithi181 ceph-mon[33774]: from='mgr.24367 172.21.15.181:0/3181919980' entity='mgr.smithi181.vtwuhz' 2024-04-27T04:11:20.673 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:11:20 smithi181 ceph-mon[33774]: from='mgr.24367 172.21.15.181:0/3181919980' entity='mgr.smithi181.vtwuhz' 2024-04-27T04:11:21.387 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:11:21 smithi162 ceph-mon[26841]: pgmap v40: 65 pgs: 65 active+clean; 273 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 1.2 MiB/s rd, 1.4 MiB/s wr, 630 op/s 2024-04-27T04:11:21.387 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:11:21 smithi162 ceph-mon[26841]: from='mgr.24367 172.21.15.181:0/3181919980' entity='mgr.smithi181.vtwuhz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-27T04:11:21.422 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:11:21 smithi181 ceph-mon[33774]: pgmap v40: 65 pgs: 65 active+clean; 273 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 1.2 MiB/s rd, 1.4 MiB/s wr, 630 op/s 2024-04-27T04:11:21.422 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:11:21 smithi181 ceph-mon[33774]: from='mgr.24367 172.21.15.181:0/3181919980' entity='mgr.smithi181.vtwuhz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-27T04:11:21.783 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:11:21.783+0000 7f8714f37700 1 -- 172.21.15.162:0/1086906990 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8708005870 msgr2=0x7f87080955a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:11:21.783 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:11:21.783+0000 7f8714f37700 1 --2- 172.21.15.162:0/1086906990 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8708005870 0x7f87080955a0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f8710063860 tx=0x7f87100636c0 comp rx=0 tx=0).stop 2024-04-27T04:11:21.783 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:11:21.784+0000 7f8714f37700 1 -- 172.21.15.162:0/1086906990 shutdown_connections 2024-04-27T04:11:21.783 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:11:21.784+0000 7f8714f37700 1 --2- 172.21.15.162:0/1086906990 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8708095ae0 0x7f8708097f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:11:21.783 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:11:21.784+0000 7f8714f37700 1 --2- 172.21.15.162:0/1086906990 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8708005870 0x7f87080955a0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:11:21.784 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:11:21.784+0000 7f8714f37700 1 -- 172.21.15.162:0/1086906990 >> 172.21.15.162:0/1086906990 conn(0x7f870808f200 msgr2=0x7f8708091650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:11:21.784 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:11:21.784+0000 7f8714f37700 1 -- 172.21.15.162:0/1086906990 shutdown_connections 2024-04-27T04:11:21.784 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:11:21.784+0000 7f8714f37700 1 -- 172.21.15.162:0/1086906990 wait complete. 2024-04-27T04:11:21.784 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:11:21.785+0000 7f8714f37700 1 Processor -- start 2024-04-27T04:11:21.785 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:11:21.785+0000 7f8714f37700 1 -- start start 2024-04-27T04:11:21.785 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:11:21.785+0000 7f8714f37700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8708005870 0x7f87080bce20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:11:21.785 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:11:21.785+0000 7f8714f37700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8708095ae0 0x7f87080bd360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:11:21.785 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:11:21.785+0000 7f8714f37700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87080bda40 con 0x7f8708095ae0 2024-04-27T04:11:21.785 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:11:21.785+0000 7f8714f37700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87080c2300 con 0x7f8708005870 2024-04-27T04:11:21.785 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:11:21.785+0000 7f870f7fe700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8708005870 0x7f87080bce20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:11:21.786 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:11:21.785+0000 7f870f7fe700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8708005870 0x7f87080bce20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.162:49818/0 (socket says 172.21.15.162:49818) 2024-04-27T04:11:21.786 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:11:21.785+0000 7f870f7fe700 1 -- 172.21.15.162:0/232288572 learned_addr learned my addr 172.21.15.162:0/232288572 (peer_addr_for_me v2:172.21.15.162:0/0) 2024-04-27T04:11:21.786 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:11:21.786+0000 7f870effd700 1 --2- 172.21.15.162:0/232288572 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8708095ae0 0x7f87080bd360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:11:21.786 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:11:21.786+0000 7f870f7fe700 1 -- 172.21.15.162:0/232288572 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8708095ae0 msgr2=0x7f87080bd360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:11:21.786 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:11:21.786+0000 7f870f7fe700 1 --2- 172.21.15.162:0/232288572 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8708095ae0 0x7f87080bd360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:11:21.786 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:11:21.786+0000 7f870f7fe700 1 -- 172.21.15.162:0/232288572 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87080c2440 con 0x7f8708005870 2024-04-27T04:11:21.786 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:11:21.786+0000 7f870effd700 1 --2- 172.21.15.162:0/232288572 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8708095ae0 0x7f87080bd360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-27T04:11:21.786 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:11:21.786+0000 7f870f7fe700 1 --2- 172.21.15.162:0/232288572 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8708005870 0x7f87080bce20 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f871004d620 tx=0x7f87100636c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:11:21.786 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:11:21.786+0000 7f86fbfff700 1 -- 172.21.15.162:0/232288572 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f871006d070 con 0x7f8708005870 2024-04-27T04:11:21.786 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:11:21.787+0000 7f8714f37700 1 -- 172.21.15.162:0/232288572 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f87080c2690 con 0x7f8708005870 2024-04-27T04:11:21.786 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:11:21.787+0000 7f8714f37700 1 -- 172.21.15.162:0/232288572 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f87080c2a80 con 0x7f8708005870 2024-04-27T04:11:21.787 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:11:21.787+0000 7f86fbfff700 1 -- 172.21.15.162:0/232288572 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f8710064e90 con 0x7f8708005870 2024-04-27T04:11:21.787 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:11:21.787+0000 7f86fbfff700 1 -- 172.21.15.162:0/232288572 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8710076aa0 con 0x7f8708005870 2024-04-27T04:11:21.787 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:11:21.789+0000 7f86fbfff700 1 -- 172.21.15.162:0/232288572 <== mon.1 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 24) v1 ==== 89652+0+0 (secure 0 0 0) 0x7f871007c020 con 0x7f8708005870 2024-04-27T04:11:21.788 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:11:21.790+0000 7f86fbfff700 1 --2- 172.21.15.162:0/232288572 >> [v2:172.21.15.181:6828/4165336392,v1:172.21.15.181:6829/4165336392] conn(0x7f870006be10 0x7f870006e2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:11:21.788 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:11:21.790+0000 7f86fbfff700 1 -- 172.21.15.162:0/232288572 <== mon.1 v2:172.21.15.181:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5808+0+0 (secure 0 0 0) 0x7f87100eba10 con 0x7f8708005870 2024-04-27T04:11:21.791 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:11:21.792+0000 7f8714f37700 1 -- 172.21.15.162:0/232288572 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f86f40052e0 con 0x7f8708005870 2024-04-27T04:11:21.792 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:11:21.792+0000 7f870effd700 1 --2- 172.21.15.162:0/232288572 >> [v2:172.21.15.181:6828/4165336392,v1:172.21.15.181:6829/4165336392] conn(0x7f870006be10 0x7f870006e2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:11:21.796 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:11:21.797+0000 7f870effd700 1 --2- 172.21.15.162:0/232288572 >> [v2:172.21.15.181:6828/4165336392,v1:172.21.15.181:6829/4165336392] conn(0x7f870006be10 0x7f870006e2c0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f8704006fd0 tx=0x7f8704008040 comp rx=0 tx=0).ready entity=mgr.24367 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:11:21.802 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:11:21.803+0000 7f86fbfff700 1 -- 172.21.15.162:0/232288572 <== mon.1 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186652 (secure 0 0 0) 0x7f87100b5060 con 0x7f8708005870 2024-04-27T04:11:22.078 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:11:22.079+0000 7f8714f37700 1 -- 172.21.15.162:0/232288572 --> [v2:172.21.15.181:6828/4165336392,v1:172.21.15.181:6829/4165336392] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f86f4000bf0 con 0x7f870006be10 2024-04-27T04:11:22.079 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:11:22.081+0000 7f86fbfff700 1 -- 172.21.15.162:0/232288572 <== mgr.24367 v2:172.21.15.181:6828/4165336392 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+321 (secure 0 0 0) 0x7f86f4000bf0 con 0x7f870006be10 2024-04-27T04:11:22.083 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:11:22.084+0000 7f8714f37700 1 -- 172.21.15.162:0/232288572 >> [v2:172.21.15.181:6828/4165336392,v1:172.21.15.181:6829/4165336392] conn(0x7f870006be10 msgr2=0x7f870006e2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:11:22.083 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:11:22.084+0000 7f8714f37700 1 --2- 172.21.15.162:0/232288572 >> [v2:172.21.15.181:6828/4165336392,v1:172.21.15.181:6829/4165336392] conn(0x7f870006be10 0x7f870006e2c0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f8704006fd0 tx=0x7f8704008040 comp rx=0 tx=0).stop 2024-04-27T04:11:22.083 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:11:22.084+0000 7f8714f37700 1 -- 172.21.15.162:0/232288572 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8708005870 msgr2=0x7f87080bce20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:11:22.083 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:11:22.084+0000 7f8714f37700 1 --2- 172.21.15.162:0/232288572 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8708005870 0x7f87080bce20 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f871004d620 tx=0x7f87100636c0 comp rx=0 tx=0).stop 2024-04-27T04:11:22.083 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:11:22.084+0000 7f8714f37700 1 -- 172.21.15.162:0/232288572 shutdown_connections 2024-04-27T04:11:22.084 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:11:22.084+0000 7f8714f37700 1 --2- 172.21.15.162:0/232288572 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8708005870 0x7f87080bce20 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:11:22.084 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:11:22.084+0000 7f8714f37700 1 --2- 172.21.15.162:0/232288572 >> [v2:172.21.15.181:6828/4165336392,v1:172.21.15.181:6829/4165336392] conn(0x7f870006be10 0x7f870006e2c0 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:11:22.084 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:11:22.084+0000 7f8714f37700 1 --2- 172.21.15.162:0/232288572 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8708095ae0 0x7f87080bd360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:11:22.084 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:11:22.084+0000 7f8714f37700 1 -- 172.21.15.162:0/232288572 >> 172.21.15.162:0/232288572 conn(0x7f870808f200 msgr2=0x7f8708091650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:11:22.084 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:11:22.084+0000 7f8714f37700 1 -- 172.21.15.162:0/232288572 shutdown_connections 2024-04-27T04:11:22.084 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:11:22.084+0000 7f8714f37700 1 -- 172.21.15.162:0/232288572 wait complete. 2024-04-27T04:11:22.097 INFO:teuthology.orchestra.run.smithi162.stdout:true 2024-04-27T04:11:22.232 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:11:22.234+0000 7f801186d700 1 -- 172.21.15.162:0/1030128142 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f800c1051b0 msgr2=0x7f800c1055c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:11:22.232 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:11:22.234+0000 7f801186d700 1 --2- 172.21.15.162:0/1030128142 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f800c1051b0 0x7f800c1055c0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f7ffc004880 tx=0x7f7ffc009e30 comp rx=0 tx=0).stop 2024-04-27T04:11:22.232 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:11:22.234+0000 7f801186d700 1 -- 172.21.15.162:0/1030128142 shutdown_connections 2024-04-27T04:11:22.232 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:11:22.234+0000 7f801186d700 1 --2- 172.21.15.162:0/1030128142 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f800c105b00 0x7f800c06ee50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:11:22.233 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:11:22.234+0000 7f801186d700 1 --2- 172.21.15.162:0/1030128142 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f800c1051b0 0x7f800c1055c0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:11:22.233 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:11:22.234+0000 7f801186d700 1 -- 172.21.15.162:0/1030128142 >> 172.21.15.162:0/1030128142 conn(0x7f800c06a360 msgr2=0x7f800c06c790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:11:22.233 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:11:22.234+0000 7f801186d700 1 -- 172.21.15.162:0/1030128142 shutdown_connections 2024-04-27T04:11:22.233 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:11:22.235+0000 7f801186d700 1 -- 172.21.15.162:0/1030128142 wait complete. 2024-04-27T04:11:22.234 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:11:22.235+0000 7f801186d700 1 Processor -- start 2024-04-27T04:11:22.234 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:11:22.235+0000 7f801186d700 1 -- start start 2024-04-27T04:11:22.234 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:11:22.236+0000 7f801186d700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f800c1051b0 0x7f800c113f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:11:22.235 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:11:22.236+0000 7f801186d700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f800c105b00 0x7f800c114440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:11:22.235 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:11:22.236+0000 7f801186d700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f800c114a40 con 0x7f800c1051b0 2024-04-27T04:11:22.235 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:11:22.236+0000 7f801186d700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f800c114bb0 con 0x7f800c105b00 2024-04-27T04:11:22.235 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:11:22.236+0000 7f800a7fc700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f800c105b00 0x7f800c114440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:11:22.235 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:11:22.236+0000 7f800a7fc700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f800c105b00 0x7f800c114440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.162:49844/0 (socket says 172.21.15.162:49844) 2024-04-27T04:11:22.235 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:11:22.236+0000 7f800a7fc700 1 -- 172.21.15.162:0/2868971201 learned_addr learned my addr 172.21.15.162:0/2868971201 (peer_addr_for_me v2:172.21.15.162:0/0) 2024-04-27T04:11:22.235 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:11:22.236+0000 7f800affd700 1 --2- 172.21.15.162:0/2868971201 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f800c1051b0 0x7f800c113f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:11:22.236 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:11:22.237+0000 7f800affd700 1 -- 172.21.15.162:0/2868971201 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f800c105b00 msgr2=0x7f800c114440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:11:22.236 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:11:22.237+0000 7f800affd700 1 --2- 172.21.15.162:0/2868971201 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f800c105b00 0x7f800c114440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:11:22.236 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:11:22.237+0000 7f800affd700 1 -- 172.21.15.162:0/2868971201 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ffc005040 con 0x7f800c1051b0 2024-04-27T04:11:22.236 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:11:22.237+0000 7f800a7fc700 1 --2- 172.21.15.162:0/2868971201 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f800c105b00 0x7f800c114440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-27T04:11:22.236 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:11:22.237+0000 7f800affd700 1 --2- 172.21.15.162:0/2868971201 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f800c1051b0 0x7f800c113f00 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f7ffc0138f0 tx=0x7f7ffc009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:11:22.236 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:11:22.237+0000 7f7ff37fe700 1 -- 172.21.15.162:0/2868971201 <== mon.0 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7ffc00e030 con 0x7f800c1051b0 2024-04-27T04:11:22.237 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:11:22.237+0000 7f801186d700 1 -- 172.21.15.162:0/2868971201 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f800c1195b0 con 0x7f800c1051b0 2024-04-27T04:11:22.237 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:11:22.238+0000 7f801186d700 1 -- 172.21.15.162:0/2868971201 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f800c119b70 con 0x7f800c1051b0 2024-04-27T04:11:22.238 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:11:22.239+0000 7f7ff37fe700 1 -- 172.21.15.162:0/2868971201 <== mon.0 v2:172.21.15.162:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f7ffc013b70 con 0x7f800c1051b0 2024-04-27T04:11:22.238 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:11:22.239+0000 7f801186d700 1 -- 172.21.15.162:0/2868971201 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f800c04c710 con 0x7f800c1051b0 2024-04-27T04:11:22.238 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:11:22.239+0000 7f7ff37fe700 1 -- 172.21.15.162:0/2868971201 <== mon.0 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7ffc01a460 con 0x7f800c1051b0 2024-04-27T04:11:22.241 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:11:22.242+0000 7f7ff37fe700 1 -- 172.21.15.162:0/2868971201 <== mon.0 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 24) v1 ==== 89652+0+0 (secure 0 0 0) 0x7f7ffc013ce0 con 0x7f800c1051b0 2024-04-27T04:11:22.241 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:11:22.243+0000 7f7ff37fe700 1 --2- 172.21.15.162:0/2868971201 >> [v2:172.21.15.181:6828/4165336392,v1:172.21.15.181:6829/4165336392] conn(0x7f7ff406bdc0 0x7f7ff406e270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:11:22.242 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:11:22.243+0000 7f800a7fc700 1 --2- 172.21.15.162:0/2868971201 >> [v2:172.21.15.181:6828/4165336392,v1:172.21.15.181:6829/4165336392] conn(0x7f7ff406bdc0 0x7f7ff406e270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:11:22.242 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:11:22.243+0000 7f7ff37fe700 1 -- 172.21.15.162:0/2868971201 <== mon.0 v2:172.21.15.162:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5808+0+0 (secure 0 0 0) 0x7f7ffc08e980 con 0x7f800c1051b0 2024-04-27T04:11:22.242 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:11:22.244+0000 7f800a7fc700 1 --2- 172.21.15.162:0/2868971201 >> [v2:172.21.15.181:6828/4165336392,v1:172.21.15.181:6829/4165336392] conn(0x7f7ff406bdc0 0x7f7ff406e270 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f8000009b80 tx=0x7f800000b040 comp rx=0 tx=0).ready entity=mgr.24367 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:11:22.248 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:11:22.249+0000 7f7ff37fe700 1 -- 172.21.15.162:0/2868971201 <== mon.0 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186652 (secure 0 0 0) 0x7f7ffc05bb90 con 0x7f800c1051b0 2024-04-27T04:11:22.581 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:11:22.582+0000 7f801186d700 1 -- 172.21.15.162:0/2868971201 --> [v2:172.21.15.181:6828/4165336392,v1:172.21.15.181:6829/4165336392] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f800c1092c0 con 0x7f7ff406bdc0 2024-04-27T04:11:22.583 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:11:22.584+0000 7f7ff37fe700 1 -- 172.21.15.162:0/2868971201 <== mgr.24367 v2:172.21.15.181:6828/4165336392 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+321 (secure 0 0 0) 0x7f800c1092c0 con 0x7f7ff406bdc0 2024-04-27T04:11:22.587 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:11:22.588+0000 7f801186d700 1 -- 172.21.15.162:0/2868971201 >> [v2:172.21.15.181:6828/4165336392,v1:172.21.15.181:6829/4165336392] conn(0x7f7ff406bdc0 msgr2=0x7f7ff406e270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:11:22.587 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:11:22.588+0000 7f801186d700 1 --2- 172.21.15.162:0/2868971201 >> [v2:172.21.15.181:6828/4165336392,v1:172.21.15.181:6829/4165336392] conn(0x7f7ff406bdc0 0x7f7ff406e270 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f8000009b80 tx=0x7f800000b040 comp rx=0 tx=0).stop 2024-04-27T04:11:22.587 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:11:22.588+0000 7f801186d700 1 -- 172.21.15.162:0/2868971201 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f800c1051b0 msgr2=0x7f800c113f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:11:22.587 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:11:22.588+0000 7f801186d700 1 --2- 172.21.15.162:0/2868971201 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f800c1051b0 0x7f800c113f00 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f7ffc0138f0 tx=0x7f7ffc009e30 comp rx=0 tx=0).stop 2024-04-27T04:11:22.587 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:11:22.588+0000 7f801186d700 1 -- 172.21.15.162:0/2868971201 shutdown_connections 2024-04-27T04:11:22.587 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:11:22.588+0000 7f801186d700 1 --2- 172.21.15.162:0/2868971201 >> [v2:172.21.15.181:6828/4165336392,v1:172.21.15.181:6829/4165336392] conn(0x7f7ff406bdc0 0x7f7ff406e270 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:11:22.588 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:11:22.589+0000 7f801186d700 1 --2- 172.21.15.162:0/2868971201 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f800c1051b0 0x7f800c113f00 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:11:22.588 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:11:22.589+0000 7f801186d700 1 --2- 172.21.15.162:0/2868971201 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f800c105b00 0x7f800c114440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:11:22.588 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:11:22.589+0000 7f801186d700 1 -- 172.21.15.162:0/2868971201 >> 172.21.15.162:0/2868971201 conn(0x7f800c06a360 msgr2=0x7f800c06ca20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:11:22.588 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:11:22.589+0000 7f801186d700 1 -- 172.21.15.162:0/2868971201 shutdown_connections 2024-04-27T04:11:22.588 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:11:22.589+0000 7f801186d700 1 -- 172.21.15.162:0/2868971201 wait complete. 2024-04-27T04:11:22.718 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:11:22.719+0000 7fd8e58da700 1 -- 172.21.15.162:0/702635550 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd8e00fbd00 msgr2=0x7fd8e00fc110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:11:22.718 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:11:22.719+0000 7fd8e58da700 1 --2- 172.21.15.162:0/702635550 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd8e00fbd00 0x7fd8e00fc110 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7fd8d0004940 tx=0x7fd8d0009e30 comp rx=0 tx=0).stop 2024-04-27T04:11:22.718 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:11:22.719+0000 7fd8e58da700 1 -- 172.21.15.162:0/702635550 shutdown_connections 2024-04-27T04:11:22.718 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:11:22.719+0000 7fd8e58da700 1 --2- 172.21.15.162:0/702635550 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd8e00fcfb0 0x7fd8e00fd420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:11:22.718 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:11:22.719+0000 7fd8e58da700 1 --2- 172.21.15.162:0/702635550 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd8e00fbd00 0x7fd8e00fc110 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:11:22.718 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:11:22.719+0000 7fd8e58da700 1 -- 172.21.15.162:0/702635550 >> 172.21.15.162:0/702635550 conn(0x7fd8e00f7480 msgr2=0x7fd8e00f9890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:11:22.718 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:11:22.719+0000 7fd8e58da700 1 -- 172.21.15.162:0/702635550 shutdown_connections 2024-04-27T04:11:22.718 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:11:22.719+0000 7fd8e58da700 1 -- 172.21.15.162:0/702635550 wait complete. 2024-04-27T04:11:22.720 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:11:22.720+0000 7fd8e58da700 1 Processor -- start 2024-04-27T04:11:22.720 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:11:22.720+0000 7fd8e58da700 1 -- start start 2024-04-27T04:11:22.720 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:11:22.720+0000 7fd8e58da700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd8e00fbd00 0x7fd8e010d930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:11:22.720 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:11:22.720+0000 7fd8e58da700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd8e00fcfb0 0x7fd8e010de70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:11:22.721 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:11:22.720+0000 7fd8e58da700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8e010e470 con 0x7fd8e00fcfb0 2024-04-27T04:11:22.721 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:11:22.720+0000 7fd8e58da700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8e010e5e0 con 0x7fd8e00fbd00 2024-04-27T04:11:22.721 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:11:22.720+0000 7fd8de7fc700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd8e00fcfb0 0x7fd8e010de70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:11:22.721 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:11:22.720+0000 7fd8deffd700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd8e00fbd00 0x7fd8e010d930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:11:22.721 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:11:22.720+0000 7fd8deffd700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd8e00fbd00 0x7fd8e010d930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.162:49862/0 (socket says 172.21.15.162:49862) 2024-04-27T04:11:22.721 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:11:22.720+0000 7fd8de7fc700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd8e00fcfb0 0x7fd8e010de70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:46030/0 (socket says 172.21.15.162:46030) 2024-04-27T04:11:22.721 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:11:22.720+0000 7fd8deffd700 1 -- 172.21.15.162:0/3981858009 learned_addr learned my addr 172.21.15.162:0/3981858009 (peer_addr_for_me v2:172.21.15.162:0/0) 2024-04-27T04:11:22.721 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:11:22.721+0000 7fd8de7fc700 1 -- 172.21.15.162:0/3981858009 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd8e00fbd00 msgr2=0x7fd8e010d930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:11:22.721 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:11:22.721+0000 7fd8de7fc700 1 --2- 172.21.15.162:0/3981858009 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd8e00fbd00 0x7fd8e010d930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:11:22.721 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:11:22.721+0000 7fd8de7fc700 1 -- 172.21.15.162:0/3981858009 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8d0005040 con 0x7fd8e00fcfb0 2024-04-27T04:11:22.721 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:11:22.721+0000 7fd8deffd700 1 --2- 172.21.15.162:0/3981858009 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd8e00fbd00 0x7fd8e010d930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-27T04:11:22.721 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:11:22.721+0000 7fd8de7fc700 1 --2- 172.21.15.162:0/3981858009 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd8e00fcfb0 0x7fd8e010de70 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7fd8d4009fd0 tx=0x7fd8d400b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:11:22.721 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:11:22.721+0000 7fd8c77fe700 1 -- 172.21.15.162:0/3981858009 <== mon.0 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd8d4004e90 con 0x7fd8e00fcfb0 2024-04-27T04:11:22.722 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:11:22.721+0000 7fd8c77fe700 1 -- 172.21.15.162:0/3981858009 <== mon.0 v2:172.21.15.162:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fd8d400f040 con 0x7fd8e00fcfb0 2024-04-27T04:11:22.722 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:11:22.721+0000 7fd8e58da700 1 -- 172.21.15.162:0/3981858009 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd8e0112f40 con 0x7fd8e00fcfb0 2024-04-27T04:11:22.722 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:11:22.721+0000 7fd8c77fe700 1 -- 172.21.15.162:0/3981858009 <== mon.0 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd8d400ca50 con 0x7fd8e00fcfb0 2024-04-27T04:11:22.722 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:11:22.721+0000 7fd8e58da700 1 -- 172.21.15.162:0/3981858009 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd8e0113450 con 0x7fd8e00fcfb0 2024-04-27T04:11:22.723 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:11:22.722+0000 7fd8e58da700 1 -- 172.21.15.162:0/3981858009 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd8e0101b40 con 0x7fd8e00fcfb0 2024-04-27T04:11:22.723 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:11:22.723+0000 7fd8c77fe700 1 -- 172.21.15.162:0/3981858009 <== mon.0 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 24) v1 ==== 89652+0+0 (secure 0 0 0) 0x7fd8d400cbb0 con 0x7fd8e00fcfb0 2024-04-27T04:11:22.723 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:11:22.723+0000 7fd8c77fe700 1 --2- 172.21.15.162:0/3981858009 >> [v2:172.21.15.181:6828/4165336392,v1:172.21.15.181:6829/4165336392] conn(0x7fd8c806bd00 0x7fd8c806e1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:11:22.723 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:11:22.723+0000 7fd8c77fe700 1 -- 172.21.15.162:0/3981858009 <== mon.0 v2:172.21.15.162:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5808+0+0 (secure 0 0 0) 0x7fd8d408a440 con 0x7fd8e00fcfb0 2024-04-27T04:11:22.723 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:11:22.724+0000 7fd8deffd700 1 --2- 172.21.15.162:0/3981858009 >> [v2:172.21.15.181:6828/4165336392,v1:172.21.15.181:6829/4165336392] conn(0x7fd8c806bd00 0x7fd8c806e1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:11:22.723 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:11:22.724+0000 7fd8deffd700 1 --2- 172.21.15.162:0/3981858009 >> [v2:172.21.15.181:6828/4165336392,v1:172.21.15.181:6829/4165336392] conn(0x7fd8c806bd00 0x7fd8c806e1b0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7fd8d000b0d0 tx=0x7fd8d0009e30 comp rx=0 tx=0).ready entity=mgr.24367 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:11:22.727 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:11:22.729+0000 7fd8c77fe700 1 -- 172.21.15.162:0/3981858009 <== mon.0 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186652 (secure 0 0 0) 0x7fd8d4050930 con 0x7fd8e00fcfb0 2024-04-27T04:11:22.962 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:11:22.963+0000 7fd8e58da700 1 -- 172.21.15.162:0/3981858009 --> [v2:172.21.15.181:6828/4165336392,v1:172.21.15.181:6829/4165336392] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7fd8e002b3e0 con 0x7fd8c806bd00 2024-04-27T04:11:22.972 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:11:22.973+0000 7fd8c77fe700 1 -- 172.21.15.162:0/3981858009 <== mgr.24367 v2:172.21.15.181:6828/4165336392 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3520 (secure 0 0 0) 0x7fd8e002b3e0 con 0x7fd8c806bd00 2024-04-27T04:11:22.972 INFO:teuthology.orchestra.run.smithi162.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-27T04:11:22.972 INFO:teuthology.orchestra.run.smithi162.stdout:alertmanager.smithi162 smithi162 *:9093,9094 starting - - - - 2024-04-27T04:11:22.972 INFO:teuthology.orchestra.run.smithi162.stdout:crash.smithi162 smithi162 running (9m) 56s ago 9m 6660k - 17.2.7-907-g2be3efa9 1c5834633715 228c98df4289 2024-04-27T04:11:22.972 INFO:teuthology.orchestra.run.smithi162.stdout:crash.smithi181 smithi181 running (8m) 58s ago 8m 6665k - 17.2.7-907-g2be3efa9 1c5834633715 c8e553e3df2b 2024-04-27T04:11:22.972 INFO:teuthology.orchestra.run.smithi162.stdout:grafana.smithi162 smithi162 *:3000 starting - - - - 2024-04-27T04:11:22.973 INFO:teuthology.orchestra.run.smithi162.stdout:mds.cephfs.smithi162.gzlyjj smithi162 running (4m) 56s ago 4m 11.9M - 17.2.7-907-g2be3efa9 1c5834633715 27e5b030dd32 2024-04-27T04:11:22.973 INFO:teuthology.orchestra.run.smithi162.stdout:mds.cephfs.smithi162.xcnapk smithi162 running (4m) 56s ago 4m 126M - 17.2.7-907-g2be3efa9 1c5834633715 38f9030df029 2024-04-27T04:11:22.973 INFO:teuthology.orchestra.run.smithi162.stdout:mds.cephfs.smithi181.bkkcdm smithi181 running (4m) 58s ago 4m 13.6M - 17.2.7-907-g2be3efa9 1c5834633715 af55afca1263 2024-04-27T04:11:22.973 INFO:teuthology.orchestra.run.smithi162.stdout:mds.cephfs.smithi181.dzomgh smithi181 running (4m) 58s ago 4m 206M - 17.2.7-907-g2be3efa9 1c5834633715 6733296597f3 2024-04-27T04:11:22.973 INFO:teuthology.orchestra.run.smithi162.stdout:mgr.smithi162.oennfn smithi162 *:8443,9283,8765 starting - - - - 2024-04-27T04:11:22.973 INFO:teuthology.orchestra.run.smithi162.stdout:mgr.smithi181.vtwuhz smithi181 *:8443,9283 running (114s) 58s ago 8m 491M - 19.0.0-3253-g4e213a58 67e414ff4bc4 402cc52dbdbb 2024-04-27T04:11:22.973 INFO:teuthology.orchestra.run.smithi162.stdout:mon.smithi162 smithi162 running (11m) 56s ago 11m 53.9M 2048M 17.2.7-907-g2be3efa9 1c5834633715 0f1e3682b9ab 2024-04-27T04:11:22.973 INFO:teuthology.orchestra.run.smithi162.stdout:mon.smithi181 smithi181 running (8m) 58s ago 8m 40.1M 2048M 17.2.7-907-g2be3efa9 1c5834633715 96e48aa3912f 2024-04-27T04:11:22.973 INFO:teuthology.orchestra.run.smithi162.stdout:node-exporter.smithi162 smithi162 *:9100 starting - - - - 2024-04-27T04:11:22.973 INFO:teuthology.orchestra.run.smithi162.stdout:node-exporter.smithi181 smithi181 *:9100 starting - - - - 2024-04-27T04:11:22.973 INFO:teuthology.orchestra.run.smithi162.stdout:osd.0 smithi162 running (7m) 56s ago 7m 234M 1557M 17.2.7-907-g2be3efa9 1c5834633715 4a9df3aa938d 2024-04-27T04:11:22.973 INFO:teuthology.orchestra.run.smithi162.stdout:osd.1 smithi162 running (7m) 56s ago 7m 233M 1557M 17.2.7-907-g2be3efa9 1c5834633715 aefb464c46e8 2024-04-27T04:11:22.973 INFO:teuthology.orchestra.run.smithi162.stdout:osd.2 smithi162 running (7m) 56s ago 7m 184M 1557M 17.2.7-907-g2be3efa9 1c5834633715 70f16d86d38e 2024-04-27T04:11:22.973 INFO:teuthology.orchestra.run.smithi162.stdout:osd.3 smithi181 running (6m) 58s ago 6m 258M 2581M 17.2.7-907-g2be3efa9 1c5834633715 030d829be54e 2024-04-27T04:11:22.974 INFO:teuthology.orchestra.run.smithi162.stdout:osd.4 smithi181 running (6m) 58s ago 6m 233M 2581M 17.2.7-907-g2be3efa9 1c5834633715 512afbee7cf7 2024-04-27T04:11:22.974 INFO:teuthology.orchestra.run.smithi162.stdout:osd.5 smithi181 running (5m) 58s ago 5m 268M 2581M 17.2.7-907-g2be3efa9 1c5834633715 f47a1daac1ea 2024-04-27T04:11:22.974 INFO:teuthology.orchestra.run.smithi162.stdout:prometheus.smithi162 smithi162 *:9095 starting - - - - 2024-04-27T04:11:22.974 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:11:22.976+0000 7fd8e58da700 1 -- 172.21.15.162:0/3981858009 >> [v2:172.21.15.181:6828/4165336392,v1:172.21.15.181:6829/4165336392] conn(0x7fd8c806bd00 msgr2=0x7fd8c806e1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:11:22.975 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:11:22.976+0000 7fd8e58da700 1 --2- 172.21.15.162:0/3981858009 >> [v2:172.21.15.181:6828/4165336392,v1:172.21.15.181:6829/4165336392] conn(0x7fd8c806bd00 0x7fd8c806e1b0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7fd8d000b0d0 tx=0x7fd8d0009e30 comp rx=0 tx=0).stop 2024-04-27T04:11:22.975 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:11:22.976+0000 7fd8e58da700 1 -- 172.21.15.162:0/3981858009 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd8e00fcfb0 msgr2=0x7fd8e010de70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:11:22.975 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:11:22.976+0000 7fd8e58da700 1 --2- 172.21.15.162:0/3981858009 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd8e00fcfb0 0x7fd8e010de70 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7fd8d4009fd0 tx=0x7fd8d400b040 comp rx=0 tx=0).stop 2024-04-27T04:11:22.975 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:11:22.976+0000 7fd8e58da700 1 -- 172.21.15.162:0/3981858009 shutdown_connections 2024-04-27T04:11:22.975 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:11:22.976+0000 7fd8e58da700 1 --2- 172.21.15.162:0/3981858009 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd8e00fbd00 0x7fd8e010d930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:11:22.975 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:11:22.976+0000 7fd8e58da700 1 --2- 172.21.15.162:0/3981858009 >> [v2:172.21.15.181:6828/4165336392,v1:172.21.15.181:6829/4165336392] conn(0x7fd8c806bd00 0x7fd8c806e1b0 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:11:22.975 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:11:22.976+0000 7fd8e58da700 1 --2- 172.21.15.162:0/3981858009 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd8e00fcfb0 0x7fd8e010de70 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:11:22.975 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:11:22.976+0000 7fd8e58da700 1 -- 172.21.15.162:0/3981858009 >> 172.21.15.162:0/3981858009 conn(0x7fd8e00f7480 msgr2=0x7fd8e00666f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:11:22.975 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:11:22.977+0000 7fd8e58da700 1 -- 172.21.15.162:0/3981858009 shutdown_connections 2024-04-27T04:11:22.976 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:11:22.977+0000 7fd8e58da700 1 -- 172.21.15.162:0/3981858009 wait complete. 2024-04-27T04:11:23.082 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:11:23.084+0000 7fd605b78700 1 -- 172.21.15.162:0/2489460562 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd6000fce60 msgr2=0x7fd6000fd270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:11:23.083 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:11:23.084+0000 7fd605b78700 1 --2- 172.21.15.162:0/2489460562 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd6000fce60 0x7fd6000fd270 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7fd5f0004940 tx=0x7fd5f0009e30 comp rx=0 tx=0).stop 2024-04-27T04:11:23.083 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:11:23.084+0000 7fd605b78700 1 -- 172.21.15.162:0/2489460562 shutdown_connections 2024-04-27T04:11:23.083 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:11:23.084+0000 7fd605b78700 1 --2- 172.21.15.162:0/2489460562 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd6000fdfd0 0x7fd6000fe440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:11:23.083 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:11:23.084+0000 7fd605b78700 1 --2- 172.21.15.162:0/2489460562 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd6000fce60 0x7fd6000fd270 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:11:23.083 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:11:23.084+0000 7fd605b78700 1 -- 172.21.15.162:0/2489460562 >> 172.21.15.162:0/2489460562 conn(0x7fd6000f8540 msgr2=0x7fd6000fa950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:11:23.083 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:11:23.084+0000 7fd605b78700 1 -- 172.21.15.162:0/2489460562 shutdown_connections 2024-04-27T04:11:23.083 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:11:23.084+0000 7fd605b78700 1 -- 172.21.15.162:0/2489460562 wait complete. 2024-04-27T04:11:23.085 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:11:23.085+0000 7fd605b78700 1 Processor -- start 2024-04-27T04:11:23.085 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:11:23.085+0000 7fd605b78700 1 -- start start 2024-04-27T04:11:23.085 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:11:23.085+0000 7fd605b78700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd6000fce60 0x7fd60010d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:11:23.085 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:11:23.085+0000 7fd605b78700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd6000fdfd0 0x7fd60010ded0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:11:23.085 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:11:23.085+0000 7fd605b78700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd60010e4d0 con 0x7fd6000fce60 2024-04-27T04:11:23.085 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:11:23.085+0000 7fd605b78700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd60010ed20 con 0x7fd6000fdfd0 2024-04-27T04:11:23.085 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:11:23.085+0000 7fd5ff7fe700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd6000fce60 0x7fd60010d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:11:23.085 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:11:23.085+0000 7fd5ff7fe700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd6000fce60 0x7fd60010d990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:46038/0 (socket says 172.21.15.162:46038) 2024-04-27T04:11:23.085 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:11:23.085+0000 7fd5ff7fe700 1 -- 172.21.15.162:0/1043215927 learned_addr learned my addr 172.21.15.162:0/1043215927 (peer_addr_for_me v2:172.21.15.162:0/0) 2024-04-27T04:11:23.085 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:11:23.085+0000 7fd5feffd700 1 --2- 172.21.15.162:0/1043215927 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd6000fdfd0 0x7fd60010ded0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:11:23.086 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:11:23.085+0000 7fd5feffd700 1 -- 172.21.15.162:0/1043215927 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd6000fce60 msgr2=0x7fd60010d990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:11:23.086 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:11:23.085+0000 7fd5feffd700 1 --2- 172.21.15.162:0/1043215927 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd6000fce60 0x7fd60010d990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:11:23.086 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:11:23.085+0000 7fd5feffd700 1 -- 172.21.15.162:0/1043215927 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5f0005040 con 0x7fd6000fdfd0 2024-04-27T04:11:23.086 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:11:23.085+0000 7fd5ff7fe700 1 --2- 172.21.15.162:0/1043215927 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd6000fce60 0x7fd60010d990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-27T04:11:23.086 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:11:23.086+0000 7fd5feffd700 1 --2- 172.21.15.162:0/1043215927 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd6000fdfd0 0x7fd60010ded0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fd5f4009fd0 tx=0x7fd5f400b040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:11:23.086 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:11:23.086+0000 7fd5e7fff700 1 -- 172.21.15.162:0/1043215927 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd5f4004e90 con 0x7fd6000fdfd0 2024-04-27T04:11:23.086 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:11:23.086+0000 7fd5e7fff700 1 -- 172.21.15.162:0/1043215927 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fd5f400f040 con 0x7fd6000fdfd0 2024-04-27T04:11:23.086 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:11:23.086+0000 7fd605b78700 1 -- 172.21.15.162:0/1043215927 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd60010efa0 con 0x7fd6000fdfd0 2024-04-27T04:11:23.086 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:11:23.086+0000 7fd5e7fff700 1 -- 172.21.15.162:0/1043215927 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd5f400ca50 con 0x7fd6000fdfd0 2024-04-27T04:11:23.086 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:11:23.086+0000 7fd605b78700 1 -- 172.21.15.162:0/1043215927 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd60010f4e0 con 0x7fd6000fdfd0 2024-04-27T04:11:23.087 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:11:23.087+0000 7fd605b78700 1 -- 172.21.15.162:0/1043215927 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd60005f5c0 con 0x7fd6000fdfd0 2024-04-27T04:11:23.087 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:11:23.088+0000 7fd5e7fff700 1 -- 172.21.15.162:0/1043215927 <== mon.1 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 24) v1 ==== 89652+0+0 (secure 0 0 0) 0x7fd5f4007cc0 con 0x7fd6000fdfd0 2024-04-27T04:11:23.087 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:11:23.088+0000 7fd5e7fff700 1 --2- 172.21.15.162:0/1043215927 >> [v2:172.21.15.181:6828/4165336392,v1:172.21.15.181:6829/4165336392] conn(0x7fd5e806bcf0 0x7fd5e806e1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:11:23.087 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:11:23.088+0000 7fd5e7fff700 1 -- 172.21.15.162:0/1043215927 <== mon.1 v2:172.21.15.181:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5808+0+0 (secure 0 0 0) 0x7fd5f408a590 con 0x7fd6000fdfd0 2024-04-27T04:11:23.088 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:11:23.088+0000 7fd5ff7fe700 1 --2- 172.21.15.162:0/1043215927 >> [v2:172.21.15.181:6828/4165336392,v1:172.21.15.181:6829/4165336392] conn(0x7fd5e806bcf0 0x7fd5e806e1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:11:23.088 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:11:23.089+0000 7fd5ff7fe700 1 --2- 172.21.15.162:0/1043215927 >> [v2:172.21.15.181:6828/4165336392,v1:172.21.15.181:6829/4165336392] conn(0x7fd5e806bcf0 0x7fd5e806e1a0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7fd5f0013740 tx=0x7fd5f0009e30 comp rx=0 tx=0).ready entity=mgr.24367 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:11:23.094 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:11:23.095+0000 7fd5e7fff700 1 -- 172.21.15.162:0/1043215927 <== mon.1 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186652 (secure 0 0 0) 0x7fd5f4050930 con 0x7fd6000fdfd0 2024-04-27T04:11:23.335 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:11:23 smithi162 ceph-mon[26841]: pgmap v41: 65 pgs: 65 active+clean; 265 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 1.6 MiB/s rd, 1.6 MiB/s wr, 771 op/s 2024-04-27T04:11:23.335 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:11:23 smithi162 ceph-mon[26841]: from='client.24459 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-27T04:11:23.439 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:11:23.440+0000 7fd605b78700 1 -- 172.21.15.162:0/1043215927 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7fd60010f7c0 con 0x7fd6000fdfd0 2024-04-27T04:11:23.439 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:11:23.441+0000 7fd5e7fff700 1 -- 172.21.15.162:0/1043215927 <== mon.1 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+846 (secure 0 0 0) 0x7fd5f4057630 con 0x7fd6000fdfd0 2024-04-27T04:11:23.440 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-04-27T04:11:23.440 INFO:teuthology.orchestra.run.smithi162.stdout: "mon": { 2024-04-27T04:11:23.440 INFO:teuthology.orchestra.run.smithi162.stdout: "ceph version 17.2.7-907-g2be3efa9 (2be3efa97a7972f8e925bd899b474329979b74d6) quincy (stable)": 2 2024-04-27T04:11:23.440 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-04-27T04:11:23.440 INFO:teuthology.orchestra.run.smithi162.stdout: "mgr": { 2024-04-27T04:11:23.440 INFO:teuthology.orchestra.run.smithi162.stdout: "ceph version 17.2.7-907-g2be3efa9 (2be3efa97a7972f8e925bd899b474329979b74d6) quincy (stable)": 1, 2024-04-27T04:11:23.440 INFO:teuthology.orchestra.run.smithi162.stdout: "ceph version 19.0.0-3253-g4e213a58 (4e213a58e787437ff508e6c890ccd5fd32dc96b5) squid (dev)": 1 2024-04-27T04:11:23.440 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-04-27T04:11:23.440 INFO:teuthology.orchestra.run.smithi162.stdout: "osd": { 2024-04-27T04:11:23.440 INFO:teuthology.orchestra.run.smithi162.stdout: "ceph version 17.2.7-907-g2be3efa9 (2be3efa97a7972f8e925bd899b474329979b74d6) quincy (stable)": 6 2024-04-27T04:11:23.441 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-04-27T04:11:23.441 INFO:teuthology.orchestra.run.smithi162.stdout: "mds": { 2024-04-27T04:11:23.441 INFO:teuthology.orchestra.run.smithi162.stdout: "ceph version 17.2.7-907-g2be3efa9 (2be3efa97a7972f8e925bd899b474329979b74d6) quincy (stable)": 4 2024-04-27T04:11:23.441 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-04-27T04:11:23.441 INFO:teuthology.orchestra.run.smithi162.stdout: "overall": { 2024-04-27T04:11:23.441 INFO:teuthology.orchestra.run.smithi162.stdout: "ceph version 17.2.7-907-g2be3efa9 (2be3efa97a7972f8e925bd899b474329979b74d6) quincy (stable)": 13, 2024-04-27T04:11:23.441 INFO:teuthology.orchestra.run.smithi162.stdout: "ceph version 19.0.0-3253-g4e213a58 (4e213a58e787437ff508e6c890ccd5fd32dc96b5) squid (dev)": 1 2024-04-27T04:11:23.441 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-04-27T04:11:23.441 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-04-27T04:11:23.442 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:11:23.443+0000 7fd605b78700 1 -- 172.21.15.162:0/1043215927 >> [v2:172.21.15.181:6828/4165336392,v1:172.21.15.181:6829/4165336392] conn(0x7fd5e806bcf0 msgr2=0x7fd5e806e1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:11:23.442 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:11:23.443+0000 7fd605b78700 1 --2- 172.21.15.162:0/1043215927 >> [v2:172.21.15.181:6828/4165336392,v1:172.21.15.181:6829/4165336392] conn(0x7fd5e806bcf0 0x7fd5e806e1a0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7fd5f0013740 tx=0x7fd5f0009e30 comp rx=0 tx=0).stop 2024-04-27T04:11:23.442 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:11:23.443+0000 7fd605b78700 1 -- 172.21.15.162:0/1043215927 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd6000fdfd0 msgr2=0x7fd60010ded0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:11:23.442 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:11:23.443+0000 7fd605b78700 1 --2- 172.21.15.162:0/1043215927 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd6000fdfd0 0x7fd60010ded0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fd5f4009fd0 tx=0x7fd5f400b040 comp rx=0 tx=0).stop 2024-04-27T04:11:23.442 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:11:23.443+0000 7fd605b78700 1 -- 172.21.15.162:0/1043215927 shutdown_connections 2024-04-27T04:11:23.443 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:11:23.443+0000 7fd605b78700 1 --2- 172.21.15.162:0/1043215927 >> [v2:172.21.15.181:6828/4165336392,v1:172.21.15.181:6829/4165336392] conn(0x7fd5e806bcf0 0x7fd5e806e1a0 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:11:23.443 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:11:23.443+0000 7fd605b78700 1 --2- 172.21.15.162:0/1043215927 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd6000fce60 0x7fd60010d990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:11:23.443 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:11:23.443+0000 7fd605b78700 1 --2- 172.21.15.162:0/1043215927 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd6000fdfd0 0x7fd60010ded0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:11:23.443 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:11:23.443+0000 7fd605b78700 1 -- 172.21.15.162:0/1043215927 >> 172.21.15.162:0/1043215927 conn(0x7fd6000f8540 msgr2=0x7fd6000f9bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:11:23.443 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:11:23.443+0000 7fd605b78700 1 -- 172.21.15.162:0/1043215927 shutdown_connections 2024-04-27T04:11:23.443 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:11:23.443+0000 7fd605b78700 1 -- 172.21.15.162:0/1043215927 wait complete. 2024-04-27T04:11:23.555 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:11:23.556+0000 7f5d2a4fc700 1 -- 172.21.15.162:0/2698741774 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5d240fcc00 msgr2=0x7f5d240fd070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:11:23.555 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:11:23.556+0000 7f5d2a4fc700 1 --2- 172.21.15.162:0/2698741774 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5d240fcc00 0x7f5d240fd070 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f5d18004940 tx=0x7f5d18009e30 comp rx=0 tx=0).stop 2024-04-27T04:11:23.555 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:11:23.556+0000 7f5d2a4fc700 1 -- 172.21.15.162:0/2698741774 shutdown_connections 2024-04-27T04:11:23.555 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:11:23.556+0000 7f5d2a4fc700 1 --2- 172.21.15.162:0/2698741774 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5d240fcc00 0x7f5d240fd070 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:11:23.555 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:11:23.556+0000 7f5d2a4fc700 1 --2- 172.21.15.162:0/2698741774 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5d240fc2b0 0x7f5d240fc6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:11:23.556 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:11:23.556+0000 7f5d2a4fc700 1 -- 172.21.15.162:0/2698741774 >> 172.21.15.162:0/2698741774 conn(0x7f5d240f6b80 msgr2=0x7f5d240f8fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:11:23.556 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:11:23.557+0000 7f5d2a4fc700 1 -- 172.21.15.162:0/2698741774 shutdown_connections 2024-04-27T04:11:23.556 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:11:23.557+0000 7f5d2a4fc700 1 -- 172.21.15.162:0/2698741774 wait complete. 2024-04-27T04:11:23.556 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:11:23.557+0000 7f5d2a4fc700 1 Processor -- start 2024-04-27T04:11:23.557 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:11:23.557+0000 7f5d2a4fc700 1 -- start start 2024-04-27T04:11:23.557 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:11:23.558+0000 7f5d2a4fc700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5d240fc2b0 0x7f5d24112aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:11:23.557 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:11:23.558+0000 7f5d2a4fc700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5d240fcc00 0x7f5d24114ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:11:23.557 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:11:23.558+0000 7f5d2a4fc700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d240fbb90 con 0x7f5d240fc2b0 2024-04-27T04:11:23.557 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:11:23.558+0000 7f5d2a4fc700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d240fbcd0 con 0x7f5d240fcc00 2024-04-27T04:11:23.557 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:11:23.558+0000 7f5d23fff700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5d240fc2b0 0x7f5d24112aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:11:23.557 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:11:23.558+0000 7f5d23fff700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5d240fc2b0 0x7f5d24112aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:60498/0 (socket says 172.21.15.162:60498) 2024-04-27T04:11:23.557 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:11:23.558+0000 7f5d23fff700 1 -- 172.21.15.162:0/3001429623 learned_addr learned my addr 172.21.15.162:0/3001429623 (peer_addr_for_me v2:172.21.15.162:0/0) 2024-04-27T04:11:23.557 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:11:23.558+0000 7f5d237fe700 1 --2- 172.21.15.162:0/3001429623 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5d240fcc00 0x7f5d24114ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:11:23.557 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:11:23.558+0000 7f5d23fff700 1 -- 172.21.15.162:0/3001429623 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5d240fcc00 msgr2=0x7f5d24114ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:11:23.557 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:11:23.558+0000 7f5d23fff700 1 --2- 172.21.15.162:0/3001429623 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5d240fcc00 0x7f5d24114ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:11:23.558 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:11:23.558+0000 7f5d23fff700 1 -- 172.21.15.162:0/3001429623 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d18005040 con 0x7f5d240fc2b0 2024-04-27T04:11:23.558 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:11:23.559+0000 7f5d23fff700 1 --2- 172.21.15.162:0/3001429623 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5d240fc2b0 0x7f5d24112aa0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f5d1400eb40 tx=0x7f5d1400b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:11:23.559 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:11:23.559+0000 7f5d20ff9700 1 -- 172.21.15.162:0/3001429623 <== mon.0 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5d14007cc0 con 0x7f5d240fc2b0 2024-04-27T04:11:23.559 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:11:23.559+0000 7f5d2a4fc700 1 -- 172.21.15.162:0/3001429623 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5d24115530 con 0x7f5d240fc2b0 2024-04-27T04:11:23.559 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:11:23.559+0000 7f5d20ff9700 1 -- 172.21.15.162:0/3001429623 <== mon.0 v2:172.21.15.162:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f5d1400f040 con 0x7f5d240fc2b0 2024-04-27T04:11:23.559 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:11:23.559+0000 7f5d20ff9700 1 -- 172.21.15.162:0/3001429623 <== mon.0 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5d1400ca10 con 0x7f5d240fc2b0 2024-04-27T04:11:23.559 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:11:23.559+0000 7f5d2a4fc700 1 -- 172.21.15.162:0/3001429623 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5d24115880 con 0x7f5d240fc2b0 2024-04-27T04:11:23.559 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:11:23.560+0000 7f5d2a4fc700 1 -- 172.21.15.162:0/3001429623 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5d2405f5c0 con 0x7f5d240fc2b0 2024-04-27T04:11:23.560 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:11:23.561+0000 7f5d20ff9700 1 -- 172.21.15.162:0/3001429623 <== mon.0 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 24) v1 ==== 89652+0+0 (secure 0 0 0) 0x7f5d1400cc80 con 0x7f5d240fc2b0 2024-04-27T04:11:23.560 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:11:23.561+0000 7f5d20ff9700 1 --2- 172.21.15.162:0/3001429623 >> [v2:172.21.15.181:6828/4165336392,v1:172.21.15.181:6829/4165336392] conn(0x7f5d0c06bcf0 0x7f5d0c06e1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:11:23.560 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:11:23.561+0000 7f5d20ff9700 1 -- 172.21.15.162:0/3001429623 <== mon.0 v2:172.21.15.162:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5808+0+0 (secure 0 0 0) 0x7f5d1408a6e0 con 0x7f5d240fc2b0 2024-04-27T04:11:23.560 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:11:23.561+0000 7f5d237fe700 1 --2- 172.21.15.162:0/3001429623 >> [v2:172.21.15.181:6828/4165336392,v1:172.21.15.181:6829/4165336392] conn(0x7f5d0c06bcf0 0x7f5d0c06e1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:11:23.560 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:11:23.562+0000 7f5d237fe700 1 --2- 172.21.15.162:0/3001429623 >> [v2:172.21.15.181:6828/4165336392,v1:172.21.15.181:6829/4165336392] conn(0x7f5d0c06bcf0 0x7f5d0c06e1a0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f5d18009b30 tx=0x7f5d18009e30 comp rx=0 tx=0).ready entity=mgr.24367 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:11:23.564 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:11:23.566+0000 7f5d20ff9700 1 -- 172.21.15.162:0/3001429623 <== mon.0 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186652 (secure 0 0 0) 0x7f5d14050930 con 0x7f5d240fc2b0 2024-04-27T04:11:23.672 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:11:23 smithi181 ceph-mon[33774]: pgmap v41: 65 pgs: 65 active+clean; 265 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 1.6 MiB/s rd, 1.6 MiB/s wr, 771 op/s 2024-04-27T04:11:23.672 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:11:23 smithi181 ceph-mon[33774]: from='client.24459 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-27T04:11:23.886 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:11:23.888+0000 7f5d2a4fc700 1 -- 172.21.15.162:0/3001429623 --> [v2:172.21.15.181:6828/4165336392,v1:172.21.15.181:6829/4165336392] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f5d241013e0 con 0x7f5d0c06bcf0 2024-04-27T04:11:23.887 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:11:23.889+0000 7f5d20ff9700 1 -- 172.21.15.162:0/3001429623 <== mgr.24367 v2:172.21.15.181:6828/4165336392 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+321 (secure 0 0 0) 0x7f5d241013e0 con 0x7f5d0c06bcf0 2024-04-27T04:11:23.888 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-04-27T04:11:23.888 INFO:teuthology.orchestra.run.smithi162.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:90a3a9fa9ceb8977f6dad9ccd36fe46018d07e2118199820523a6b35301d9c16", 2024-04-27T04:11:23.888 INFO:teuthology.orchestra.run.smithi162.stdout: "in_progress": true, 2024-04-27T04:11:23.888 INFO:teuthology.orchestra.run.smithi162.stdout: "which": "Upgrading daemons of type(s) mgr", 2024-04-27T04:11:23.888 INFO:teuthology.orchestra.run.smithi162.stdout: "services_complete": [], 2024-04-27T04:11:23.888 INFO:teuthology.orchestra.run.smithi162.stdout: "progress": "", 2024-04-27T04:11:23.888 INFO:teuthology.orchestra.run.smithi162.stdout: "message": "Currently upgrading mgr daemons", 2024-04-27T04:11:23.888 INFO:teuthology.orchestra.run.smithi162.stdout: "is_paused": false 2024-04-27T04:11:23.888 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-04-27T04:11:23.890 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:11:23.891+0000 7f5d2a4fc700 1 -- 172.21.15.162:0/3001429623 >> [v2:172.21.15.181:6828/4165336392,v1:172.21.15.181:6829/4165336392] conn(0x7f5d0c06bcf0 msgr2=0x7f5d0c06e1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:11:23.890 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:11:23.891+0000 7f5d2a4fc700 1 --2- 172.21.15.162:0/3001429623 >> [v2:172.21.15.181:6828/4165336392,v1:172.21.15.181:6829/4165336392] conn(0x7f5d0c06bcf0 0x7f5d0c06e1a0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f5d18009b30 tx=0x7f5d18009e30 comp rx=0 tx=0).stop 2024-04-27T04:11:23.890 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:11:23.891+0000 7f5d2a4fc700 1 -- 172.21.15.162:0/3001429623 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5d240fc2b0 msgr2=0x7f5d24112aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:11:23.890 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:11:23.891+0000 7f5d2a4fc700 1 --2- 172.21.15.162:0/3001429623 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5d240fc2b0 0x7f5d24112aa0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f5d1400eb40 tx=0x7f5d1400b040 comp rx=0 tx=0).stop 2024-04-27T04:11:23.891 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:11:23.891+0000 7f5d2a4fc700 1 -- 172.21.15.162:0/3001429623 shutdown_connections 2024-04-27T04:11:23.891 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:11:23.891+0000 7f5d2a4fc700 1 --2- 172.21.15.162:0/3001429623 >> [v2:172.21.15.181:6828/4165336392,v1:172.21.15.181:6829/4165336392] conn(0x7f5d0c06bcf0 0x7f5d0c06e1a0 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:11:23.891 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:11:23.892+0000 7f5d2a4fc700 1 --2- 172.21.15.162:0/3001429623 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5d240fc2b0 0x7f5d24112aa0 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:11:23.891 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:11:23.892+0000 7f5d2a4fc700 1 --2- 172.21.15.162:0/3001429623 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5d240fcc00 0x7f5d24114ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:11:23.891 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:11:23.892+0000 7f5d2a4fc700 1 -- 172.21.15.162:0/3001429623 >> 172.21.15.162:0/3001429623 conn(0x7f5d240f6b80 msgr2=0x7f5d240f8240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:11:23.891 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:11:23.892+0000 7f5d2a4fc700 1 -- 172.21.15.162:0/3001429623 shutdown_connections 2024-04-27T04:11:23.891 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:11:23.892+0000 7f5d2a4fc700 1 -- 172.21.15.162:0/3001429623 wait complete. 2024-04-27T04:11:24.579 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:11:24 smithi162 ceph-mon[26841]: from='client.14684 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-27T04:11:24.580 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:11:24 smithi162 ceph-mon[26841]: from='client.14688 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-27T04:11:24.580 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:11:24 smithi162 ceph-mon[26841]: from='client.? 172.21.15.162:0/1043215927' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-27T04:11:24.580 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:11:24 smithi162 ceph-mon[26841]: from='mgr.24367 172.21.15.181:0/3181919980' entity='mgr.smithi181.vtwuhz' 2024-04-27T04:11:24.580 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:11:24 smithi162 ceph-mon[26841]: from='mgr.24367 172.21.15.181:0/3181919980' entity='mgr.smithi181.vtwuhz' 2024-04-27T04:11:24.672 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:11:24 smithi181 ceph-mon[33774]: from='client.14684 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-27T04:11:24.672 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:11:24 smithi181 ceph-mon[33774]: from='client.14688 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-27T04:11:24.672 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:11:24 smithi181 ceph-mon[33774]: from='client.? 172.21.15.162:0/1043215927' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-27T04:11:24.672 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:11:24 smithi181 ceph-mon[33774]: from='mgr.24367 172.21.15.181:0/3181919980' entity='mgr.smithi181.vtwuhz' 2024-04-27T04:11:24.672 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:11:24 smithi181 ceph-mon[33774]: from='mgr.24367 172.21.15.181:0/3181919980' entity='mgr.smithi181.vtwuhz' 2024-04-27T04:11:25.278 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:11:25 smithi162 ceph-mon[26841]: pgmap v42: 65 pgs: 65 active+clean; 267 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 1.3 MiB/s rd, 1.5 MiB/s wr, 731 op/s 2024-04-27T04:11:25.279 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:11:25 smithi162 ceph-mon[26841]: from='client.14696 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-27T04:11:25.672 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:11:25 smithi181 ceph-mon[33774]: pgmap v42: 65 pgs: 65 active+clean; 267 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 1.3 MiB/s rd, 1.5 MiB/s wr, 731 op/s 2024-04-27T04:11:25.672 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:11:25 smithi181 ceph-mon[33774]: from='client.14696 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-27T04:11:27.472 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:11:27 smithi162 ceph-mon[26841]: pgmap v43: 65 pgs: 65 active+clean; 267 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 1.3 MiB/s rd, 1.4 MiB/s wr, 703 op/s 2024-04-27T04:11:27.473 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:11:27 smithi162 ceph-mon[26841]: from='mgr.24367 172.21.15.181:0/3181919980' entity='mgr.smithi181.vtwuhz' 2024-04-27T04:11:27.473 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:11:27 smithi162 ceph-mon[26841]: from='mgr.24367 172.21.15.181:0/3181919980' entity='mgr.smithi181.vtwuhz' 2024-04-27T04:11:27.672 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:11:27 smithi181 ceph-mon[33774]: pgmap v43: 65 pgs: 65 active+clean; 267 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 1.3 MiB/s rd, 1.4 MiB/s wr, 703 op/s 2024-04-27T04:11:27.672 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:11:27 smithi181 ceph-mon[33774]: from='mgr.24367 172.21.15.181:0/3181919980' entity='mgr.smithi181.vtwuhz' 2024-04-27T04:11:27.672 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:11:27 smithi181 ceph-mon[33774]: from='mgr.24367 172.21.15.181:0/3181919980' entity='mgr.smithi181.vtwuhz' 2024-04-27T04:11:28.512 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:11:28 smithi162 ceph-mon[26841]: Standby manager daemon smithi162.oennfn restarted 2024-04-27T04:11:28.512 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:11:28 smithi162 ceph-mon[26841]: Standby manager daemon smithi162.oennfn started 2024-04-27T04:11:28.512 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:11:28 smithi162 ceph-mon[26841]: from='mgr.? 172.21.15.162:0/501880828' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi162.oennfn/crt"}]: dispatch 2024-04-27T04:11:28.512 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:11:28 smithi162 ceph-mon[26841]: from='mgr.? 172.21.15.162:0/501880828' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-27T04:11:28.512 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:11:28 smithi162 ceph-mon[26841]: from='mgr.? 172.21.15.162:0/501880828' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi162.oennfn/key"}]: dispatch 2024-04-27T04:11:28.512 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:11:28 smithi162 ceph-mon[26841]: from='mgr.? 172.21.15.162:0/501880828' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-27T04:11:28.512 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:11:28 smithi162 ceph-mon[26841]: from='mgr.24367 172.21.15.181:0/3181919980' entity='mgr.smithi181.vtwuhz' 2024-04-27T04:11:28.512 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:11:28 smithi162 ceph-mon[26841]: from='mgr.24367 172.21.15.181:0/3181919980' entity='mgr.smithi181.vtwuhz' 2024-04-27T04:11:28.672 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:11:28 smithi181 ceph-mon[33774]: Standby manager daemon smithi162.oennfn restarted 2024-04-27T04:11:28.673 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:11:28 smithi181 ceph-mon[33774]: Standby manager daemon smithi162.oennfn started 2024-04-27T04:11:28.673 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:11:28 smithi181 ceph-mon[33774]: from='mgr.? 172.21.15.162:0/501880828' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi162.oennfn/crt"}]: dispatch 2024-04-27T04:11:28.673 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:11:28 smithi181 ceph-mon[33774]: from='mgr.? 172.21.15.162:0/501880828' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-27T04:11:28.673 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:11:28 smithi181 ceph-mon[33774]: from='mgr.? 172.21.15.162:0/501880828' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi162.oennfn/key"}]: dispatch 2024-04-27T04:11:28.673 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:11:28 smithi181 ceph-mon[33774]: from='mgr.? 172.21.15.162:0/501880828' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-27T04:11:28.673 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:11:28 smithi181 ceph-mon[33774]: from='mgr.24367 172.21.15.181:0/3181919980' entity='mgr.smithi181.vtwuhz' 2024-04-27T04:11:28.673 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:11:28 smithi181 ceph-mon[33774]: from='mgr.24367 172.21.15.181:0/3181919980' entity='mgr.smithi181.vtwuhz' 2024-04-27T04:11:29.397 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:11:29 smithi162 ceph-mon[26841]: pgmap v44: 65 pgs: 65 active+clean; 260 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 1.8 MiB/s rd, 1.9 MiB/s wr, 949 op/s 2024-04-27T04:11:29.398 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:11:29 smithi162 ceph-mon[26841]: mgrmap e25: smithi181.vtwuhz(active, since 82s), standbys: smithi162.oennfn 2024-04-27T04:11:29.672 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:11:29 smithi181 ceph-mon[33774]: pgmap v44: 65 pgs: 65 active+clean; 260 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 1.8 MiB/s rd, 1.9 MiB/s wr, 949 op/s 2024-04-27T04:11:29.672 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:11:29 smithi181 ceph-mon[33774]: mgrmap e25: smithi181.vtwuhz(active, since 82s), standbys: smithi162.oennfn 2024-04-27T04:11:31.637 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:11:31 smithi162 ceph-mon[26841]: pgmap v45: 65 pgs: 65 active+clean; 260 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 1.2 MiB/s rd, 1.1 MiB/s wr, 443 op/s 2024-04-27T04:11:31.672 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:11:31 smithi181 ceph-mon[33774]: pgmap v45: 65 pgs: 65 active+clean; 260 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 1.2 MiB/s rd, 1.1 MiB/s wr, 443 op/s 2024-04-27T04:11:32.861 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:11:32 smithi181 ceph-mon[33774]: from='mgr.24367 172.21.15.181:0/3181919980' entity='mgr.smithi181.vtwuhz' 2024-04-27T04:11:32.861 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:11:32 smithi181 ceph-mon[33774]: from='mgr.24367 172.21.15.181:0/3181919980' entity='mgr.smithi181.vtwuhz' 2024-04-27T04:11:32.861 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:11:32 smithi181 ceph-mon[33774]: from='mgr.24367 172.21.15.181:0/3181919980' entity='mgr.smithi181.vtwuhz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T04:11:32.861 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:11:32 smithi181 ceph-mon[33774]: from='mgr.24367 172.21.15.181:0/3181919980' entity='mgr.smithi181.vtwuhz' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T04:11:32.861 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:11:32 smithi181 ceph-mon[33774]: from='mgr.24367 172.21.15.181:0/3181919980' entity='mgr.smithi181.vtwuhz' 2024-04-27T04:11:32.862 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:11:32 smithi181 ceph-mon[33774]: from='mgr.24367 172.21.15.181:0/3181919980' entity='mgr.smithi181.vtwuhz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T04:11:32.862 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:11:32 smithi181 ceph-mon[33774]: from='mgr.24367 172.21.15.181:0/3181919980' entity='mgr.smithi181.vtwuhz' cmd=[{"prefix": "mgr fail", "who": "smithi181.vtwuhz"}]: dispatch 2024-04-27T04:11:32.862 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:11:32 smithi181 ceph-mon[33774]: Activating manager daemon smithi162.oennfn 2024-04-27T04:11:32.862 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:11:32 smithi181 ceph-mon[33774]: osdmap e48: 6 total, 6 up, 6 in 2024-04-27T04:11:32.862 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:11:32 smithi181 ceph-mon[33774]: from='mgr.24367 172.21.15.181:0/3181919980' entity='mgr.smithi181.vtwuhz' cmd='[{"prefix": "mgr fail", "who": "smithi181.vtwuhz"}]': finished 2024-04-27T04:11:32.862 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:11:32 smithi181 ceph-mon[33774]: mgrmap e26: smithi162.oennfn(active, starting, since 0.00858877s) 2024-04-27T04:11:32.862 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:11:32 smithi181 ceph-mon[33774]: from='mgr.14680 172.21.15.162:0/501880828' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "mon metadata", "id": "smithi162"}]: dispatch 2024-04-27T04:11:32.862 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:11:32 smithi181 ceph-mon[33774]: from='mgr.14680 172.21.15.162:0/501880828' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "mon metadata", "id": "smithi181"}]: dispatch 2024-04-27T04:11:32.862 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:11:32 smithi181 ceph-mon[33774]: from='mgr.14680 172.21.15.162:0/501880828' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi162.xcnapk"}]: dispatch 2024-04-27T04:11:32.862 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:11:32 smithi181 ceph-mon[33774]: from='mgr.14680 172.21.15.162:0/501880828' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi162.gzlyjj"}]: dispatch 2024-04-27T04:11:32.863 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:11:32 smithi181 ceph-mon[33774]: from='mgr.14680 172.21.15.162:0/501880828' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi181.dzomgh"}]: dispatch 2024-04-27T04:11:32.863 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:11:32 smithi181 ceph-mon[33774]: from='mgr.14680 172.21.15.162:0/501880828' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi181.bkkcdm"}]: dispatch 2024-04-27T04:11:32.863 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:11:32 smithi181 ceph-mon[33774]: from='mgr.14680 172.21.15.162:0/501880828' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "mgr metadata", "who": "smithi162.oennfn", "id": "smithi162.oennfn"}]: dispatch 2024-04-27T04:11:32.863 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:11:32 smithi181 ceph-mon[33774]: from='mgr.14680 172.21.15.162:0/501880828' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-27T04:11:32.863 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:11:32 smithi181 ceph-mon[33774]: from='mgr.14680 172.21.15.162:0/501880828' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-27T04:11:32.863 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:11:32 smithi181 ceph-mon[33774]: from='mgr.14680 172.21.15.162:0/501880828' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-27T04:11:32.864 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:11:32 smithi181 ceph-mon[33774]: from='mgr.14680 172.21.15.162:0/501880828' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-27T04:11:32.864 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:11:32 smithi181 ceph-mon[33774]: from='mgr.14680 172.21.15.162:0/501880828' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-27T04:11:32.864 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:11:32 smithi181 ceph-mon[33774]: from='mgr.14680 172.21.15.162:0/501880828' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-27T04:11:32.864 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:11:32 smithi181 ceph-mon[33774]: from='mgr.14680 172.21.15.162:0/501880828' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-04-27T04:11:32.864 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:11:32 smithi181 ceph-mon[33774]: from='mgr.14680 172.21.15.162:0/501880828' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-04-27T04:11:32.864 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:11:32 smithi181 ceph-mon[33774]: from='mgr.14680 172.21.15.162:0/501880828' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-04-27T04:11:32.864 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:11:32 smithi181 ceph-mon[33774]: Manager daemon smithi162.oennfn is now available 2024-04-27T04:11:32.864 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:11:32 smithi181 ceph-mon[33774]: from='mgr.14680 172.21.15.162:0/501880828' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T04:11:32.864 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:11:32 smithi181 ceph-mon[33774]: from='mgr.14680 172.21.15.162:0/501880828' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-27T04:11:32.865 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:11:32 smithi181 ceph-mon[33774]: from='mgr.14680 172.21.15.162:0/501880828' entity='mgr.smithi162.oennfn' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi162.oennfn/mirror_snapshot_schedule"}]: dispatch 2024-04-27T04:11:32.865 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:11:32 smithi181 ceph-mon[33774]: from='mgr.14680 172.21.15.162:0/501880828' entity='mgr.smithi162.oennfn' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi162.oennfn/trash_purge_schedule"}]: dispatch 2024-04-27T04:11:33.137 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:11:32 smithi162 ceph-mon[26841]: from='mgr.24367 172.21.15.181:0/3181919980' entity='mgr.smithi181.vtwuhz' 2024-04-27T04:11:33.137 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:11:32 smithi162 ceph-mon[26841]: from='mgr.24367 172.21.15.181:0/3181919980' entity='mgr.smithi181.vtwuhz' 2024-04-27T04:11:33.138 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:11:32 smithi162 ceph-mon[26841]: from='mgr.24367 172.21.15.181:0/3181919980' entity='mgr.smithi181.vtwuhz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T04:11:33.138 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:11:32 smithi162 ceph-mon[26841]: from='mgr.24367 172.21.15.181:0/3181919980' entity='mgr.smithi181.vtwuhz' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T04:11:33.138 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:11:32 smithi162 ceph-mon[26841]: from='mgr.24367 172.21.15.181:0/3181919980' entity='mgr.smithi181.vtwuhz' 2024-04-27T04:11:33.138 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:11:32 smithi162 ceph-mon[26841]: from='mgr.24367 172.21.15.181:0/3181919980' entity='mgr.smithi181.vtwuhz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T04:11:33.138 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:11:32 smithi162 ceph-mon[26841]: from='mgr.24367 172.21.15.181:0/3181919980' entity='mgr.smithi181.vtwuhz' cmd=[{"prefix": "mgr fail", "who": "smithi181.vtwuhz"}]: dispatch 2024-04-27T04:11:33.138 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:11:32 smithi162 ceph-mon[26841]: Activating manager daemon smithi162.oennfn 2024-04-27T04:11:33.138 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:11:32 smithi162 ceph-mon[26841]: osdmap e48: 6 total, 6 up, 6 in 2024-04-27T04:11:33.138 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:11:32 smithi162 ceph-mon[26841]: from='mgr.24367 172.21.15.181:0/3181919980' entity='mgr.smithi181.vtwuhz' cmd='[{"prefix": "mgr fail", "who": "smithi181.vtwuhz"}]': finished 2024-04-27T04:11:33.138 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:11:32 smithi162 ceph-mon[26841]: mgrmap e26: smithi162.oennfn(active, starting, since 0.00858877s) 2024-04-27T04:11:33.138 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:11:32 smithi162 ceph-mon[26841]: from='mgr.14680 172.21.15.162:0/501880828' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "mon metadata", "id": "smithi162"}]: dispatch 2024-04-27T04:11:33.138 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:11:32 smithi162 ceph-mon[26841]: from='mgr.14680 172.21.15.162:0/501880828' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "mon metadata", "id": "smithi181"}]: dispatch 2024-04-27T04:11:33.138 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:11:32 smithi162 ceph-mon[26841]: from='mgr.14680 172.21.15.162:0/501880828' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi162.xcnapk"}]: dispatch 2024-04-27T04:11:33.138 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:11:32 smithi162 ceph-mon[26841]: from='mgr.14680 172.21.15.162:0/501880828' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi162.gzlyjj"}]: dispatch 2024-04-27T04:11:33.139 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:11:32 smithi162 ceph-mon[26841]: from='mgr.14680 172.21.15.162:0/501880828' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi181.dzomgh"}]: dispatch 2024-04-27T04:11:33.139 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:11:32 smithi162 ceph-mon[26841]: from='mgr.14680 172.21.15.162:0/501880828' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi181.bkkcdm"}]: dispatch 2024-04-27T04:11:33.139 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:11:32 smithi162 ceph-mon[26841]: from='mgr.14680 172.21.15.162:0/501880828' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "mgr metadata", "who": "smithi162.oennfn", "id": "smithi162.oennfn"}]: dispatch 2024-04-27T04:11:33.139 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:11:32 smithi162 ceph-mon[26841]: from='mgr.14680 172.21.15.162:0/501880828' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-27T04:11:33.139 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:11:32 smithi162 ceph-mon[26841]: from='mgr.14680 172.21.15.162:0/501880828' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-27T04:11:33.139 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:11:32 smithi162 ceph-mon[26841]: from='mgr.14680 172.21.15.162:0/501880828' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-27T04:11:33.139 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:11:32 smithi162 ceph-mon[26841]: from='mgr.14680 172.21.15.162:0/501880828' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-27T04:11:33.139 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:11:32 smithi162 ceph-mon[26841]: from='mgr.14680 172.21.15.162:0/501880828' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-27T04:11:33.139 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:11:32 smithi162 ceph-mon[26841]: from='mgr.14680 172.21.15.162:0/501880828' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-27T04:11:33.139 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:11:32 smithi162 ceph-mon[26841]: from='mgr.14680 172.21.15.162:0/501880828' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-04-27T04:11:33.139 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:11:32 smithi162 ceph-mon[26841]: from='mgr.14680 172.21.15.162:0/501880828' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-04-27T04:11:33.139 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:11:32 smithi162 ceph-mon[26841]: from='mgr.14680 172.21.15.162:0/501880828' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-04-27T04:11:33.139 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:11:32 smithi162 ceph-mon[26841]: Manager daemon smithi162.oennfn is now available 2024-04-27T04:11:33.139 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:11:32 smithi162 ceph-mon[26841]: from='mgr.14680 172.21.15.162:0/501880828' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T04:11:33.140 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:11:32 smithi162 ceph-mon[26841]: from='mgr.14680 172.21.15.162:0/501880828' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-27T04:11:33.140 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:11:32 smithi162 ceph-mon[26841]: from='mgr.14680 172.21.15.162:0/501880828' entity='mgr.smithi162.oennfn' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi162.oennfn/mirror_snapshot_schedule"}]: dispatch 2024-04-27T04:11:33.140 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:11:32 smithi162 ceph-mon[26841]: from='mgr.14680 172.21.15.162:0/501880828' entity='mgr.smithi162.oennfn' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi162.oennfn/trash_purge_schedule"}]: dispatch 2024-04-27T04:11:34.526 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:11:34 smithi162 ceph-mon[26841]: [27/Apr/2024:04:11:33] ENGINE Bus STARTING 2024-04-27T04:11:34.526 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:11:34 smithi162 ceph-mon[26841]: [27/Apr/2024:04:11:33] ENGINE Serving on https://172.21.15.162:7150 2024-04-27T04:11:34.526 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:11:34 smithi162 ceph-mon[26841]: mgrmap e27: smithi162.oennfn(active, since 1.02296s) 2024-04-27T04:11:34.526 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:11:34 smithi162 ceph-mon[26841]: pgmap v3: 65 pgs: 65 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-04-27T04:11:34.526 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:11:34 smithi162 ceph-mon[26841]: [27/Apr/2024:04:11:33] ENGINE Serving on http://172.21.15.162:8765 2024-04-27T04:11:34.526 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:11:34 smithi162 ceph-mon[26841]: [27/Apr/2024:04:11:33] ENGINE Bus STARTED 2024-04-27T04:11:34.609 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:11:34 smithi181 ceph-mon[33774]: [27/Apr/2024:04:11:33] ENGINE Bus STARTING 2024-04-27T04:11:34.609 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:11:34 smithi181 ceph-mon[33774]: [27/Apr/2024:04:11:33] ENGINE Serving on https://172.21.15.162:7150 2024-04-27T04:11:34.609 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:11:34 smithi181 ceph-mon[33774]: mgrmap e27: smithi162.oennfn(active, since 1.02296s) 2024-04-27T04:11:34.609 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:11:34 smithi181 ceph-mon[33774]: pgmap v3: 65 pgs: 65 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-04-27T04:11:34.609 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:11:34 smithi181 ceph-mon[33774]: [27/Apr/2024:04:11:33] ENGINE Serving on http://172.21.15.162:8765 2024-04-27T04:11:34.609 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:11:34 smithi181 ceph-mon[33774]: [27/Apr/2024:04:11:33] ENGINE Bus STARTED 2024-04-27T04:11:35.491 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:11:35 smithi162 ceph-mon[26841]: pgmap v4: 65 pgs: 65 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-04-27T04:11:35.491 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:11:35 smithi162 ceph-mon[26841]: mgrmap e28: smithi162.oennfn(active, since 2s) 2024-04-27T04:11:35.585 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:11:35 smithi181 ceph-mon[33774]: pgmap v4: 65 pgs: 65 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-04-27T04:11:35.585 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:11:35 smithi181 ceph-mon[33774]: mgrmap e28: smithi162.oennfn(active, since 2s) 2024-04-27T04:11:37.367 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:11:37 smithi181 ceph-mon[33774]: pgmap v5: 65 pgs: 65 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-04-27T04:11:37.367 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:11:37 smithi181 ceph-mon[33774]: mgrmap e29: smithi162.oennfn(active, since 4s) 2024-04-27T04:11:37.636 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:11:37 smithi162 ceph-mon[26841]: pgmap v5: 65 pgs: 65 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-04-27T04:11:37.637 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:11:37 smithi162 ceph-mon[26841]: mgrmap e29: smithi162.oennfn(active, since 4s) 2024-04-27T04:11:39.757 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:11:39 smithi181 ceph-mon[33774]: pgmap v6: 65 pgs: 65 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-04-27T04:11:39.758 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:11:39 smithi181 ceph-mon[33774]: from='mgr.14680 172.21.15.162:0/501880828' entity='mgr.smithi162.oennfn' 2024-04-27T04:11:39.758 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:11:39 smithi181 ceph-mon[33774]: from='mgr.14680 172.21.15.162:0/501880828' entity='mgr.smithi162.oennfn' 2024-04-27T04:11:39.758 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:11:39 smithi181 ceph-mon[33774]: from='mgr.14680 172.21.15.162:0/501880828' entity='mgr.smithi162.oennfn' 2024-04-27T04:11:39.758 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:11:39 smithi181 ceph-mon[33774]: from='mgr.14680 172.21.15.162:0/501880828' entity='mgr.smithi162.oennfn' 2024-04-27T04:11:39.887 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:11:39 smithi162 ceph-mon[26841]: pgmap v6: 65 pgs: 65 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-04-27T04:11:39.887 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:11:39 smithi162 ceph-mon[26841]: from='mgr.14680 172.21.15.162:0/501880828' entity='mgr.smithi162.oennfn' 2024-04-27T04:11:39.887 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:11:39 smithi162 ceph-mon[26841]: from='mgr.14680 172.21.15.162:0/501880828' entity='mgr.smithi162.oennfn' 2024-04-27T04:11:39.887 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:11:39 smithi162 ceph-mon[26841]: from='mgr.14680 172.21.15.162:0/501880828' entity='mgr.smithi162.oennfn' 2024-04-27T04:11:39.887 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:11:39 smithi162 ceph-mon[26841]: from='mgr.14680 172.21.15.162:0/501880828' entity='mgr.smithi162.oennfn' 2024-04-27T04:11:40.730 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:11:40 smithi162 ceph-mon[26841]: from='mgr.14680 172.21.15.162:0/501880828' entity='mgr.smithi162.oennfn' 2024-04-27T04:11:40.730 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:11:40 smithi162 ceph-mon[26841]: from='mgr.14680 172.21.15.162:0/501880828' entity='mgr.smithi162.oennfn' 2024-04-27T04:11:40.730 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:11:40 smithi162 ceph-mon[26841]: from='mgr.14680 172.21.15.162:0/501880828' entity='mgr.smithi162.oennfn' 2024-04-27T04:11:40.730 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:11:40 smithi162 ceph-mon[26841]: from='mgr.14680 172.21.15.162:0/501880828' entity='mgr.smithi162.oennfn' 2024-04-27T04:11:41.134 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:11:40 smithi181 ceph-mon[33774]: from='mgr.14680 172.21.15.162:0/501880828' entity='mgr.smithi162.oennfn' 2024-04-27T04:11:41.134 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:11:40 smithi181 ceph-mon[33774]: from='mgr.14680 172.21.15.162:0/501880828' entity='mgr.smithi162.oennfn' 2024-04-27T04:11:41.134 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:11:40 smithi181 ceph-mon[33774]: from='mgr.14680 172.21.15.162:0/501880828' entity='mgr.smithi162.oennfn' 2024-04-27T04:11:41.134 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:11:40 smithi181 ceph-mon[33774]: from='mgr.14680 172.21.15.162:0/501880828' entity='mgr.smithi162.oennfn' 2024-04-27T04:11:42.040 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:11:41 smithi162 ceph-mon[26841]: pgmap v7: 65 pgs: 65 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 686 KiB/s rd, 797 KiB/s wr, 243 op/s 2024-04-27T04:11:42.040 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:11:41 smithi162 ceph-mon[26841]: Standby manager daemon smithi181.vtwuhz started 2024-04-27T04:11:42.040 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:11:41 smithi162 ceph-mon[26841]: from='mgr.? 172.21.15.181:0/2694277941' entity='mgr.smithi181.vtwuhz' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi181.vtwuhz/crt"}]: dispatch 2024-04-27T04:11:42.040 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:11:41 smithi162 ceph-mon[26841]: from='mgr.? 172.21.15.181:0/2694277941' entity='mgr.smithi181.vtwuhz' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-27T04:11:42.040 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:11:41 smithi162 ceph-mon[26841]: from='mgr.? 172.21.15.181:0/2694277941' entity='mgr.smithi181.vtwuhz' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi181.vtwuhz/key"}]: dispatch 2024-04-27T04:11:42.040 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:11:41 smithi162 ceph-mon[26841]: from='mgr.? 172.21.15.181:0/2694277941' entity='mgr.smithi181.vtwuhz' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-27T04:11:42.172 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:11:41 smithi181 ceph-mon[33774]: pgmap v7: 65 pgs: 65 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 686 KiB/s rd, 797 KiB/s wr, 243 op/s 2024-04-27T04:11:42.172 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:11:41 smithi181 ceph-mon[33774]: Standby manager daemon smithi181.vtwuhz started 2024-04-27T04:11:42.172 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:11:41 smithi181 ceph-mon[33774]: from='mgr.? 172.21.15.181:0/2694277941' entity='mgr.smithi181.vtwuhz' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi181.vtwuhz/crt"}]: dispatch 2024-04-27T04:11:42.173 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:11:41 smithi181 ceph-mon[33774]: from='mgr.? 172.21.15.181:0/2694277941' entity='mgr.smithi181.vtwuhz' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-27T04:11:42.173 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:11:41 smithi181 ceph-mon[33774]: from='mgr.? 172.21.15.181:0/2694277941' entity='mgr.smithi181.vtwuhz' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi181.vtwuhz/key"}]: dispatch 2024-04-27T04:11:42.173 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:11:41 smithi181 ceph-mon[33774]: from='mgr.? 172.21.15.181:0/2694277941' entity='mgr.smithi181.vtwuhz' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-27T04:11:43.137 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:11:42 smithi162 ceph-mon[26841]: mgrmap e30: smithi162.oennfn(active, since 9s), standbys: smithi181.vtwuhz 2024-04-27T04:11:43.137 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:11:42 smithi162 ceph-mon[26841]: from='mgr.14680 172.21.15.162:0/501880828' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "mgr metadata", "who": "smithi181.vtwuhz", "id": "smithi181.vtwuhz"}]: dispatch 2024-04-27T04:11:43.172 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:11:42 smithi181 ceph-mon[33774]: mgrmap e30: smithi162.oennfn(active, since 9s), standbys: smithi181.vtwuhz 2024-04-27T04:11:43.172 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:11:42 smithi181 ceph-mon[33774]: from='mgr.14680 172.21.15.162:0/501880828' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "mgr metadata", "who": "smithi181.vtwuhz", "id": "smithi181.vtwuhz"}]: dispatch 2024-04-27T04:11:44.137 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:11:43 smithi162 ceph-mon[26841]: pgmap v8: 65 pgs: 65 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 533 KiB/s rd, 619 KiB/s wr, 188 op/s 2024-04-27T04:11:44.172 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:11:43 smithi181 ceph-mon[33774]: pgmap v8: 65 pgs: 65 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 533 KiB/s rd, 619 KiB/s wr, 188 op/s 2024-04-27T04:11:45.387 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:11:44 smithi162 ceph-mon[26841]: from='mgr.14680 172.21.15.162:0/501880828' entity='mgr.smithi162.oennfn' 2024-04-27T04:11:45.387 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:11:44 smithi162 ceph-mon[26841]: from='mgr.14680 172.21.15.162:0/501880828' entity='mgr.smithi162.oennfn' 2024-04-27T04:11:45.387 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:11:44 smithi162 ceph-mon[26841]: from='mgr.14680 172.21.15.162:0/501880828' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T04:11:45.387 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:11:44 smithi162 ceph-mon[26841]: from='mgr.14680 172.21.15.162:0/501880828' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T04:11:45.387 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:11:44 smithi162 ceph-mon[26841]: Updating smithi162:/etc/ceph/ceph.conf 2024-04-27T04:11:45.387 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:11:44 smithi162 ceph-mon[26841]: Updating smithi181:/etc/ceph/ceph.conf 2024-04-27T04:11:45.422 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:11:44 smithi181 ceph-mon[33774]: from='mgr.14680 172.21.15.162:0/501880828' entity='mgr.smithi162.oennfn' 2024-04-27T04:11:45.422 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:11:44 smithi181 ceph-mon[33774]: from='mgr.14680 172.21.15.162:0/501880828' entity='mgr.smithi162.oennfn' 2024-04-27T04:11:45.422 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:11:44 smithi181 ceph-mon[33774]: from='mgr.14680 172.21.15.162:0/501880828' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T04:11:45.422 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:11:44 smithi181 ceph-mon[33774]: from='mgr.14680 172.21.15.162:0/501880828' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T04:11:45.422 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:11:44 smithi181 ceph-mon[33774]: Updating smithi162:/etc/ceph/ceph.conf 2024-04-27T04:11:45.422 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:11:44 smithi181 ceph-mon[33774]: Updating smithi181:/etc/ceph/ceph.conf 2024-04-27T04:11:46.387 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:11:46 smithi162 ceph-mon[26841]: pgmap v9: 65 pgs: 65 active+clean; 240 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 1021 KiB/s rd, 1.1 MiB/s wr, 321 op/s 2024-04-27T04:11:46.387 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:11:46 smithi162 ceph-mon[26841]: Updating smithi162:/var/lib/ceph/7f9750d6-044a-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-27T04:11:46.387 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:11:46 smithi162 ceph-mon[26841]: Updating smithi181:/var/lib/ceph/7f9750d6-044a-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-27T04:11:46.387 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:11:46 smithi162 ceph-mon[26841]: Updating smithi162:/etc/ceph/ceph.client.admin.keyring 2024-04-27T04:11:46.387 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:11:46 smithi162 ceph-mon[26841]: Updating smithi181:/etc/ceph/ceph.client.admin.keyring 2024-04-27T04:11:46.387 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:11:46 smithi162 ceph-mon[26841]: Updating smithi162:/var/lib/ceph/7f9750d6-044a-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-27T04:11:46.387 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:11:46 smithi162 ceph-mon[26841]: from='mgr.14680 172.21.15.162:0/501880828' entity='mgr.smithi162.oennfn' 2024-04-27T04:11:46.387 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:11:46 smithi162 ceph-mon[26841]: from='mgr.14680 172.21.15.162:0/501880828' entity='mgr.smithi162.oennfn' 2024-04-27T04:11:46.387 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:11:46 smithi162 ceph-mon[26841]: from='mgr.14680 172.21.15.162:0/501880828' entity='mgr.smithi162.oennfn' 2024-04-27T04:11:46.387 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:11:46 smithi162 ceph-mon[26841]: from='mgr.14680 172.21.15.162:0/501880828' entity='mgr.smithi162.oennfn' 2024-04-27T04:11:46.388 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:11:46 smithi162 ceph-mon[26841]: from='mgr.14680 172.21.15.162:0/501880828' entity='mgr.smithi162.oennfn' 2024-04-27T04:11:46.422 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:11:46 smithi181 ceph-mon[33774]: pgmap v9: 65 pgs: 65 active+clean; 240 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 1021 KiB/s rd, 1.1 MiB/s wr, 321 op/s 2024-04-27T04:11:46.422 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:11:46 smithi181 ceph-mon[33774]: Updating smithi162:/var/lib/ceph/7f9750d6-044a-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-27T04:11:46.423 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:11:46 smithi181 ceph-mon[33774]: Updating smithi181:/var/lib/ceph/7f9750d6-044a-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-27T04:11:46.423 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:11:46 smithi181 ceph-mon[33774]: Updating smithi162:/etc/ceph/ceph.client.admin.keyring 2024-04-27T04:11:46.423 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:11:46 smithi181 ceph-mon[33774]: Updating smithi181:/etc/ceph/ceph.client.admin.keyring 2024-04-27T04:11:46.423 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:11:46 smithi181 ceph-mon[33774]: Updating smithi162:/var/lib/ceph/7f9750d6-044a-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-27T04:11:46.423 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:11:46 smithi181 ceph-mon[33774]: from='mgr.14680 172.21.15.162:0/501880828' entity='mgr.smithi162.oennfn' 2024-04-27T04:11:46.423 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:11:46 smithi181 ceph-mon[33774]: from='mgr.14680 172.21.15.162:0/501880828' entity='mgr.smithi162.oennfn' 2024-04-27T04:11:46.423 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:11:46 smithi181 ceph-mon[33774]: from='mgr.14680 172.21.15.162:0/501880828' entity='mgr.smithi162.oennfn' 2024-04-27T04:11:46.423 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:11:46 smithi181 ceph-mon[33774]: from='mgr.14680 172.21.15.162:0/501880828' entity='mgr.smithi162.oennfn' 2024-04-27T04:11:46.423 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:11:46 smithi181 ceph-mon[33774]: from='mgr.14680 172.21.15.162:0/501880828' entity='mgr.smithi162.oennfn' 2024-04-27T04:11:47.387 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:11:47 smithi162 ceph-mon[26841]: Updating smithi181:/var/lib/ceph/7f9750d6-044a-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-27T04:11:47.387 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:11:47 smithi162 ceph-mon[26841]: Reconfiguring prometheus.smithi162 (dependencies changed)... 2024-04-27T04:11:47.387 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:11:47 smithi162 ceph-mon[26841]: Reconfiguring daemon prometheus.smithi162 on smithi162 2024-04-27T04:11:47.422 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:11:47 smithi181 ceph-mon[33774]: Updating smithi181:/var/lib/ceph/7f9750d6-044a-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-27T04:11:47.422 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:11:47 smithi181 ceph-mon[33774]: Reconfiguring prometheus.smithi162 (dependencies changed)... 2024-04-27T04:11:47.422 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:11:47 smithi181 ceph-mon[33774]: Reconfiguring daemon prometheus.smithi162 on smithi162 2024-04-27T04:11:48.388 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:11:48 smithi162 ceph-mon[26841]: pgmap v10: 65 pgs: 65 active+clean; 240 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 934 KiB/s rd, 985 KiB/s wr, 294 op/s 2024-04-27T04:11:48.388 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:11:48 smithi162 ceph-mon[26841]: from='mgr.14680 172.21.15.162:0/501880828' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-27T04:11:48.422 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:11:48 smithi181 ceph-mon[33774]: pgmap v10: 65 pgs: 65 active+clean; 240 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 934 KiB/s rd, 985 KiB/s wr, 294 op/s 2024-04-27T04:11:48.422 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:11:48 smithi181 ceph-mon[33774]: from='mgr.14680 172.21.15.162:0/501880828' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-27T04:11:50.283 INFO:tasks.workunit:Stopping ['suites/fsstress.sh'] on client.1... 2024-04-27T04:11:50.283 DEBUG:teuthology.orchestra.run.smithi181:> sudo rm -rf -- /home/ubuntu/cephtest/workunits.list.client.1 /home/ubuntu/cephtest/clone.client.1 2024-04-27T04:11:50.387 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:11:50 smithi162 ceph-mon[26841]: pgmap v11: 65 pgs: 65 active+clean; 240 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 934 KiB/s rd, 985 KiB/s wr, 294 op/s 2024-04-27T04:11:50.422 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:11:50 smithi181 ceph-mon[33774]: pgmap v11: 65 pgs: 65 active+clean; 240 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 934 KiB/s rd, 985 KiB/s wr, 294 op/s 2024-04-27T04:11:50.596 DEBUG:teuthology.parallel:result is None 2024-04-27T04:11:52.387 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:11:52 smithi162 ceph-mon[26841]: pgmap v12: 65 pgs: 65 active+clean; 233 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 1.3 MiB/s rd, 1.5 MiB/s wr, 441 op/s 2024-04-27T04:11:52.422 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:11:52 smithi181 ceph-mon[33774]: pgmap v12: 65 pgs: 65 active+clean; 233 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 1.3 MiB/s rd, 1.5 MiB/s wr, 441 op/s 2024-04-27T04:11:52.814 INFO:tasks.workunit:Stopping ['suites/fsstress.sh'] on client.0... 2024-04-27T04:11:52.815 DEBUG:teuthology.orchestra.run.smithi162:> sudo rm -rf -- /home/ubuntu/cephtest/workunits.list.client.0 /home/ubuntu/cephtest/clone.client.0 2024-04-27T04:11:53.159 DEBUG:teuthology.parallel:result is None 2024-04-27T04:11:53.159 DEBUG:teuthology.orchestra.run.smithi162:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0 2024-04-27T04:11:53.194 INFO:tasks.workunit:Deleted dir /home/ubuntu/cephtest/mnt.0/client.0 2024-04-27T04:11:53.194 DEBUG:teuthology.orchestra.run.smithi181:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.1/client.1 2024-04-27T04:11:53.228 INFO:tasks.workunit:Deleted dir /home/ubuntu/cephtest/mnt.1/client.1 2024-04-27T04:11:53.228 DEBUG:teuthology.parallel:result is None 2024-04-27T04:11:53.999 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:11:53.999+0000 7fd1799d9700 1 -- 172.21.15.162:0/2282234829 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd1740fce60 msgr2=0x7fd1740fd270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:11:53.999 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:11:53.999+0000 7fd1799d9700 1 --2- 172.21.15.162:0/2282234829 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd1740fce60 0x7fd1740fd270 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7fd164004940 tx=0x7fd164009e30 comp rx=0 tx=0).stop 2024-04-27T04:11:53.999 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:11:53.999+0000 7fd1799d9700 1 -- 172.21.15.162:0/2282234829 shutdown_connections 2024-04-27T04:11:53.999 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:11:53.999+0000 7fd1799d9700 1 --2- 172.21.15.162:0/2282234829 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd1740fdfd0 0x7fd1740fe440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:11:53.999 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:11:53.999+0000 7fd1799d9700 1 --2- 172.21.15.162:0/2282234829 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd1740fce60 0x7fd1740fd270 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:11:54.000 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:11:53.999+0000 7fd1799d9700 1 -- 172.21.15.162:0/2282234829 >> 172.21.15.162:0/2282234829 conn(0x7fd1740f8540 msgr2=0x7fd1740fa950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:11:54.000 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:11:53.999+0000 7fd1799d9700 1 -- 172.21.15.162:0/2282234829 shutdown_connections 2024-04-27T04:11:54.000 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:11:53.999+0000 7fd1799d9700 1 -- 172.21.15.162:0/2282234829 wait complete. 2024-04-27T04:11:54.000 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:11:53.999+0000 7fd1799d9700 1 Processor -- start 2024-04-27T04:11:54.000 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:11:53.999+0000 7fd1799d9700 1 -- start start 2024-04-27T04:11:54.000 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:11:54.000+0000 7fd1799d9700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd1740fce60 0x7fd17410d940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:11:54.000 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:11:54.000+0000 7fd1799d9700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd1740fdfd0 0x7fd17410de80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:11:54.000 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:11:54.000+0000 7fd1799d9700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd17410e480 con 0x7fd1740fdfd0 2024-04-27T04:11:54.000 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:11:54.000+0000 7fd1799d9700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd17410e5f0 con 0x7fd1740fce60 2024-04-27T04:11:54.000 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:11:54.000+0000 7fd16a7fc700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd1740fdfd0 0x7fd17410de80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:11:54.000 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:11:54.000+0000 7fd16a7fc700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd1740fdfd0 0x7fd17410de80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:42352/0 (socket says 172.21.15.162:42352) 2024-04-27T04:11:54.000 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:11:54.000+0000 7fd16a7fc700 1 -- 172.21.15.162:0/1384036148 learned_addr learned my addr 172.21.15.162:0/1384036148 (peer_addr_for_me v2:172.21.15.162:0/0) 2024-04-27T04:11:54.000 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:11:54.000+0000 7fd172ffd700 1 --2- 172.21.15.162:0/1384036148 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd1740fce60 0x7fd17410d940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:11:54.001 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:11:54.000+0000 7fd16a7fc700 1 -- 172.21.15.162:0/1384036148 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd1740fce60 msgr2=0x7fd17410d940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:11:54.001 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:11:54.000+0000 7fd16a7fc700 1 --2- 172.21.15.162:0/1384036148 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd1740fce60 0x7fd17410d940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:11:54.001 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:11:54.000+0000 7fd16a7fc700 1 -- 172.21.15.162:0/1384036148 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd164005040 con 0x7fd1740fdfd0 2024-04-27T04:11:54.001 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:11:54.000+0000 7fd16a7fc700 1 --2- 172.21.15.162:0/1384036148 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd1740fdfd0 0x7fd17410de80 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7fd15c002a50 tx=0x7fd15c0046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:11:54.001 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:11:54.000+0000 7fd16bfff700 1 -- 172.21.15.162:0/1384036148 <== mon.0 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd15c007cc0 con 0x7fd1740fdfd0 2024-04-27T04:11:54.001 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:11:54.000+0000 7fd1799d9700 1 -- 172.21.15.162:0/1384036148 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd174113050 con 0x7fd1740fdfd0 2024-04-27T04:11:54.001 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:11:54.000+0000 7fd16bfff700 1 -- 172.21.15.162:0/1384036148 <== mon.0 v2:172.21.15.162:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fd15c00f040 con 0x7fd1740fdfd0 2024-04-27T04:11:54.001 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:11:54.000+0000 7fd16bfff700 1 -- 172.21.15.162:0/1384036148 <== mon.0 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd15c00da30 con 0x7fd1740fdfd0 2024-04-27T04:11:54.001 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:11:54.001+0000 7fd1799d9700 1 -- 172.21.15.162:0/1384036148 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd1741135e0 con 0x7fd1740fdfd0 2024-04-27T04:11:54.002 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:11:54.001+0000 7fd1799d9700 1 -- 172.21.15.162:0/1384036148 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd17404c710 con 0x7fd1740fdfd0 2024-04-27T04:11:54.002 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:11:54.002+0000 7fd16bfff700 1 -- 172.21.15.162:0/1384036148 <== mon.0 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 30) v1 ==== 92571+0+0 (secure 0 0 0) 0x7fd15c00db90 con 0x7fd1740fdfd0 2024-04-27T04:11:54.002 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:11:54.002+0000 7fd16bfff700 1 --2- 172.21.15.162:0/1384036148 >> [v2:172.21.15.162:6800/901992993,v1:172.21.15.162:6801/901992993] conn(0x7fd16006e630 0x7fd160070ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:11:54.002 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:11:54.002+0000 7fd16bfff700 1 -- 172.21.15.162:0/1384036148 <== mon.0 v2:172.21.15.162:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6152+0+0 (secure 0 0 0) 0x7fd15c08df60 con 0x7fd1740fdfd0 2024-04-27T04:11:54.003 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:11:54.003+0000 7fd172ffd700 1 --2- 172.21.15.162:0/1384036148 >> [v2:172.21.15.162:6800/901992993,v1:172.21.15.162:6801/901992993] conn(0x7fd16006e630 0x7fd160070ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:11:54.003 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:11:54.003+0000 7fd172ffd700 1 --2- 172.21.15.162:0/1384036148 >> [v2:172.21.15.162:6800/901992993,v1:172.21.15.162:6801/901992993] conn(0x7fd16006e630 0x7fd160070ae0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fd164009fd0 tx=0x7fd164009e30 comp rx=0 tx=0).ready entity=mgr.14680 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:11:54.005 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:11:54.007+0000 7fd16bfff700 1 -- 172.21.15.162:0/1384036148 <== mon.0 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186652 (secure 0 0 0) 0x7fd15c0c2930 con 0x7fd1740fdfd0 2024-04-27T04:11:54.222 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:11:54.223+0000 7fd1799d9700 1 -- 172.21.15.162:0/1384036148 --> [v2:172.21.15.162:6800/901992993,v1:172.21.15.162:6801/901992993] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fd174102330 con 0x7fd16006e630 2024-04-27T04:11:54.222 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:11:54 smithi162 ceph-mon[26841]: pgmap v13: 65 pgs: 65 active+clean; 233 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 962 KiB/s rd, 1.0 MiB/s wr, 300 op/s 2024-04-27T04:11:54.222 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:11:54 smithi162 ceph-mon[26841]: from='mgr.14680 172.21.15.162:0/501880828' entity='mgr.smithi162.oennfn' 2024-04-27T04:11:54.222 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:11:54 smithi162 ceph-mon[26841]: from='mgr.14680 172.21.15.162:0/501880828' entity='mgr.smithi162.oennfn' 2024-04-27T04:11:54.222 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:11:54 smithi162 ceph-mon[26841]: from='mgr.14680 172.21.15.162:0/501880828' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-04-27T04:11:54.223 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:11:54 smithi162 ceph-mon[26841]: from='mgr.14680 172.21.15.162:0/501880828' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "dashboard set-prometheus-api-host", "value": "http://smithi162:9095"}]: dispatch 2024-04-27T04:11:54.223 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:11:54 smithi162 ceph-mon[26841]: from='mgr.14680 172.21.15.162:0/501880828' entity='mgr.smithi162.oennfn' 2024-04-27T04:11:54.223 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:11:54 smithi162 ceph-mon[26841]: from='mgr.14680 172.21.15.162:0/501880828' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T04:11:54.223 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:11:54.225+0000 7fd16bfff700 1 -- 172.21.15.162:0/1384036148 <== mgr.14680 v2:172.21.15.162:6800/901992993 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+360 (secure 0 0 0) 0x7fd174102330 con 0x7fd16006e630 2024-04-27T04:11:54.226 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:11:54.227+0000 7fd1799d9700 1 -- 172.21.15.162:0/1384036148 >> [v2:172.21.15.162:6800/901992993,v1:172.21.15.162:6801/901992993] conn(0x7fd16006e630 msgr2=0x7fd160070ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:11:54.226 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:11:54.227+0000 7fd1799d9700 1 --2- 172.21.15.162:0/1384036148 >> [v2:172.21.15.162:6800/901992993,v1:172.21.15.162:6801/901992993] conn(0x7fd16006e630 0x7fd160070ae0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fd164009fd0 tx=0x7fd164009e30 comp rx=0 tx=0).stop 2024-04-27T04:11:54.226 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:11:54.227+0000 7fd1799d9700 1 -- 172.21.15.162:0/1384036148 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd1740fdfd0 msgr2=0x7fd17410de80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:11:54.226 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:11:54.227+0000 7fd1799d9700 1 --2- 172.21.15.162:0/1384036148 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd1740fdfd0 0x7fd17410de80 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7fd15c002a50 tx=0x7fd15c0046b0 comp rx=0 tx=0).stop 2024-04-27T04:11:54.226 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:11:54.227+0000 7fd1799d9700 1 -- 172.21.15.162:0/1384036148 shutdown_connections 2024-04-27T04:11:54.226 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:11:54.227+0000 7fd1799d9700 1 --2- 172.21.15.162:0/1384036148 >> [v2:172.21.15.162:6800/901992993,v1:172.21.15.162:6801/901992993] conn(0x7fd16006e630 0x7fd160070ae0 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:11:54.226 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:11:54.227+0000 7fd1799d9700 1 --2- 172.21.15.162:0/1384036148 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd1740fce60 0x7fd17410d940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:11:54.226 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:11:54.227+0000 7fd1799d9700 1 --2- 172.21.15.162:0/1384036148 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd1740fdfd0 0x7fd17410de80 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:11:54.227 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:11:54.227+0000 7fd1799d9700 1 -- 172.21.15.162:0/1384036148 >> 172.21.15.162:0/1384036148 conn(0x7fd1740f8540 msgr2=0x7fd1740f9c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:11:54.227 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:11:54.227+0000 7fd1799d9700 1 -- 172.21.15.162:0/1384036148 shutdown_connections 2024-04-27T04:11:54.227 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:11:54.228+0000 7fd1799d9700 1 -- 172.21.15.162:0/1384036148 wait complete. 2024-04-27T04:11:54.236 INFO:teuthology.orchestra.run.smithi162.stdout:true 2024-04-27T04:11:54.331 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:11:54.330+0000 7f429b064700 1 -- 172.21.15.162:0/3529984729 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f42940fcf70 msgr2=0x7f42940fd3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:11:54.331 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:11:54.330+0000 7f429b064700 1 --2- 172.21.15.162:0/3529984729 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f42940fcf70 0x7f42940fd3e0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f4280004940 tx=0x7f4280009e30 comp rx=0 tx=0).stop 2024-04-27T04:11:54.331 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:11:54.330+0000 7f429b064700 1 -- 172.21.15.162:0/3529984729 shutdown_connections 2024-04-27T04:11:54.331 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:11:54.330+0000 7f429b064700 1 --2- 172.21.15.162:0/3529984729 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f42940fcf70 0x7f42940fd3e0 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:11:54.331 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:11:54.330+0000 7f429b064700 1 --2- 172.21.15.162:0/3529984729 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f42940fbcc0 0x7f42940fc0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:11:54.331 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:11:54.330+0000 7f429b064700 1 -- 172.21.15.162:0/3529984729 >> 172.21.15.162:0/3529984729 conn(0x7f42940f7440 msgr2=0x7f42940f9850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:11:54.331 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:11:54.330+0000 7f429b064700 1 -- 172.21.15.162:0/3529984729 shutdown_connections 2024-04-27T04:11:54.331 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:11:54.330+0000 7f429b064700 1 -- 172.21.15.162:0/3529984729 wait complete. 2024-04-27T04:11:54.331 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:11:54.331+0000 7f429b064700 1 Processor -- start 2024-04-27T04:11:54.332 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:11:54.331+0000 7f429b064700 1 -- start start 2024-04-27T04:11:54.332 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:11:54.331+0000 7f429b064700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f42940fbcc0 0x7f4294066d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:11:54.332 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:11:54.331+0000 7f429b064700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f42940fcf70 0x7f4294112650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:11:54.332 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:11:54.331+0000 7f429b064700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4294112b90 con 0x7f42940fcf70 2024-04-27T04:11:54.332 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:11:54.331+0000 7f429b064700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4294112cd0 con 0x7f42940fbcc0 2024-04-27T04:11:54.332 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:11:54.331+0000 7f4293fff700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f42940fcf70 0x7f4294112650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:11:54.332 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:11:54.331+0000 7f4293fff700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f42940fcf70 0x7f4294112650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:42370/0 (socket says 172.21.15.162:42370) 2024-04-27T04:11:54.332 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:11:54.331+0000 7f4293fff700 1 -- 172.21.15.162:0/3107632542 learned_addr learned my addr 172.21.15.162:0/3107632542 (peer_addr_for_me v2:172.21.15.162:0/0) 2024-04-27T04:11:54.332 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:11:54.331+0000 7f4298e00700 1 --2- 172.21.15.162:0/3107632542 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f42940fbcc0 0x7f4294066d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:11:54.332 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:11:54.332+0000 7f4293fff700 1 -- 172.21.15.162:0/3107632542 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f42940fbcc0 msgr2=0x7f4294066d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:11:54.332 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:11:54.332+0000 7f4293fff700 1 --2- 172.21.15.162:0/3107632542 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f42940fbcc0 0x7f4294066d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:11:54.332 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:11:54.332+0000 7f4293fff700 1 -- 172.21.15.162:0/3107632542 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4280005040 con 0x7f42940fcf70 2024-04-27T04:11:54.332 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:11:54.332+0000 7f4293fff700 1 --2- 172.21.15.162:0/3107632542 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f42940fcf70 0x7f4294112650 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f428000b3b0 tx=0x7f4280009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:11:54.333 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:11:54.332+0000 7f42917fa700 1 -- 172.21.15.162:0/3107632542 <== mon.0 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f428000e040 con 0x7f42940fcf70 2024-04-27T04:11:54.333 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:11:54.332+0000 7f42917fa700 1 -- 172.21.15.162:0/3107632542 <== mon.0 v2:172.21.15.162:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f428000bbf0 con 0x7f42940fcf70 2024-04-27T04:11:54.333 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:11:54.332+0000 7f429b064700 1 -- 172.21.15.162:0/3107632542 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4294112f50 con 0x7f42940fcf70 2024-04-27T04:11:54.333 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:11:54.332+0000 7f42917fa700 1 -- 172.21.15.162:0/3107632542 <== mon.0 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f428001c630 con 0x7f42940fcf70 2024-04-27T04:11:54.333 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:11:54.332+0000 7f429b064700 1 -- 172.21.15.162:0/3107632542 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f42941134e0 con 0x7f42940fcf70 2024-04-27T04:11:54.334 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:11:54.334+0000 7f429b064700 1 -- 172.21.15.162:0/3107632542 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f429404c710 con 0x7f42940fcf70 2024-04-27T04:11:54.334 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:11:54.334+0000 7f42917fa700 1 -- 172.21.15.162:0/3107632542 <== mon.0 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 30) v1 ==== 92571+0+0 (secure 0 0 0) 0x7f428000bd60 con 0x7f42940fcf70 2024-04-27T04:11:54.334 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:11:54.334+0000 7f42917fa700 1 --2- 172.21.15.162:0/3107632542 >> [v2:172.21.15.162:6800/901992993,v1:172.21.15.162:6801/901992993] conn(0x7f428406e630 0x7f4284070ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:11:54.334 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:11:54.334+0000 7f42917fa700 1 -- 172.21.15.162:0/3107632542 <== mon.0 v2:172.21.15.162:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6152+0+0 (secure 0 0 0) 0x7f428008ec60 con 0x7f42940fcf70 2024-04-27T04:11:54.334 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:11:54.334+0000 7f4298e00700 1 --2- 172.21.15.162:0/3107632542 >> [v2:172.21.15.162:6800/901992993,v1:172.21.15.162:6801/901992993] conn(0x7f428406e630 0x7f4284070ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:11:54.334 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:11:54.335+0000 7f4298e00700 1 --2- 172.21.15.162:0/3107632542 >> [v2:172.21.15.162:6800/901992993,v1:172.21.15.162:6801/901992993] conn(0x7f428406e630 0x7f4284070ae0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f4288009fd0 tx=0x7f428800b040 comp rx=0 tx=0).ready entity=mgr.14680 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:11:54.337 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:11:54.339+0000 7f42917fa700 1 -- 172.21.15.162:0/3107632542 <== mon.0 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186652 (secure 0 0 0) 0x7f428005bc90 con 0x7f42940fcf70 2024-04-27T04:11:54.419 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:11:54 smithi181 ceph-mon[33774]: pgmap v13: 65 pgs: 65 active+clean; 233 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 962 KiB/s rd, 1.0 MiB/s wr, 300 op/s 2024-04-27T04:11:54.419 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:11:54 smithi181 ceph-mon[33774]: from='mgr.14680 172.21.15.162:0/501880828' entity='mgr.smithi162.oennfn' 2024-04-27T04:11:54.419 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:11:54 smithi181 ceph-mon[33774]: from='mgr.14680 172.21.15.162:0/501880828' entity='mgr.smithi162.oennfn' 2024-04-27T04:11:54.419 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:11:54 smithi181 ceph-mon[33774]: from='mgr.14680 172.21.15.162:0/501880828' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-04-27T04:11:54.420 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:11:54 smithi181 ceph-mon[33774]: from='mgr.14680 172.21.15.162:0/501880828' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "dashboard set-prometheus-api-host", "value": "http://smithi162:9095"}]: dispatch 2024-04-27T04:11:54.420 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:11:54 smithi181 ceph-mon[33774]: from='mgr.14680 172.21.15.162:0/501880828' entity='mgr.smithi162.oennfn' 2024-04-27T04:11:54.420 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:11:54 smithi181 ceph-mon[33774]: from='mgr.14680 172.21.15.162:0/501880828' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T04:11:54.553 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:11:54.554+0000 7f429b064700 1 -- 172.21.15.162:0/3107632542 --> [v2:172.21.15.162:6800/901992993,v1:172.21.15.162:6801/901992993] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f4294101210 con 0x7f428406e630 2024-04-27T04:11:54.554 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:11:54.556+0000 7f42917fa700 1 -- 172.21.15.162:0/3107632542 <== mgr.14680 v2:172.21.15.162:6800/901992993 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+360 (secure 0 0 0) 0x7f4294101210 con 0x7f428406e630 2024-04-27T04:11:54.557 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:11:54.558+0000 7f429b064700 1 -- 172.21.15.162:0/3107632542 >> [v2:172.21.15.162:6800/901992993,v1:172.21.15.162:6801/901992993] conn(0x7f428406e630 msgr2=0x7f4284070ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:11:54.557 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:11:54.558+0000 7f429b064700 1 --2- 172.21.15.162:0/3107632542 >> [v2:172.21.15.162:6800/901992993,v1:172.21.15.162:6801/901992993] conn(0x7f428406e630 0x7f4284070ae0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f4288009fd0 tx=0x7f428800b040 comp rx=0 tx=0).stop 2024-04-27T04:11:54.557 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:11:54.558+0000 7f429b064700 1 -- 172.21.15.162:0/3107632542 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f42940fcf70 msgr2=0x7f4294112650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:11:54.557 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:11:54.558+0000 7f429b064700 1 --2- 172.21.15.162:0/3107632542 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f42940fcf70 0x7f4294112650 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f428000b3b0 tx=0x7f4280009e30 comp rx=0 tx=0).stop 2024-04-27T04:11:54.557 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:11:54.558+0000 7f429b064700 1 -- 172.21.15.162:0/3107632542 shutdown_connections 2024-04-27T04:11:54.557 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:11:54.558+0000 7f429b064700 1 --2- 172.21.15.162:0/3107632542 >> [v2:172.21.15.162:6800/901992993,v1:172.21.15.162:6801/901992993] conn(0x7f428406e630 0x7f4284070ae0 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:11:54.557 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:11:54.558+0000 7f429b064700 1 --2- 172.21.15.162:0/3107632542 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f42940fbcc0 0x7f4294066d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:11:54.557 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:11:54.558+0000 7f429b064700 1 --2- 172.21.15.162:0/3107632542 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f42940fcf70 0x7f4294112650 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:11:54.557 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:11:54.558+0000 7f429b064700 1 -- 172.21.15.162:0/3107632542 >> 172.21.15.162:0/3107632542 conn(0x7f42940f7440 msgr2=0x7f42940f8ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:11:54.558 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:11:54.558+0000 7f429b064700 1 -- 172.21.15.162:0/3107632542 shutdown_connections 2024-04-27T04:11:54.558 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:11:54.558+0000 7f429b064700 1 -- 172.21.15.162:0/3107632542 wait complete. 2024-04-27T04:11:54.660 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:11:54.659+0000 7f457e629700 1 -- 172.21.15.162:0/3800324460 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f45780fbce0 msgr2=0x7f45780fc0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:11:54.660 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:11:54.659+0000 7f457e629700 1 --2- 172.21.15.162:0/3800324460 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f45780fbce0 0x7f45780fc0f0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f4568004940 tx=0x7f4568009e30 comp rx=0 tx=0).stop 2024-04-27T04:11:54.660 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:11:54.659+0000 7f457e629700 1 -- 172.21.15.162:0/3800324460 shutdown_connections 2024-04-27T04:11:54.660 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:11:54.659+0000 7f457e629700 1 --2- 172.21.15.162:0/3800324460 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f45780fcf90 0x7f45780fd400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:11:54.660 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:11:54.659+0000 7f457e629700 1 --2- 172.21.15.162:0/3800324460 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f45780fbce0 0x7f45780fc0f0 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:11:54.660 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:11:54.659+0000 7f457e629700 1 -- 172.21.15.162:0/3800324460 >> 172.21.15.162:0/3800324460 conn(0x7f45780f7440 msgr2=0x7f45780f9870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:11:54.660 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:11:54.659+0000 7f457e629700 1 -- 172.21.15.162:0/3800324460 shutdown_connections 2024-04-27T04:11:54.660 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:11:54.659+0000 7f457e629700 1 -- 172.21.15.162:0/3800324460 wait complete. 2024-04-27T04:11:54.661 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:11:54.660+0000 7f457e629700 1 Processor -- start 2024-04-27T04:11:54.661 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:11:54.660+0000 7f457e629700 1 -- start start 2024-04-27T04:11:54.661 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:11:54.660+0000 7f457e629700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f45780fbce0 0x7f457810da00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:11:54.661 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:11:54.660+0000 7f457e629700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f45780fcf90 0x7f457810df40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:11:54.661 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:11:54.660+0000 7f457e629700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f457810e540 con 0x7f45780fcf90 2024-04-27T04:11:54.661 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:11:54.660+0000 7f457e629700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f457810ed90 con 0x7f45780fbce0 2024-04-27T04:11:54.661 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:11:54.660+0000 7f45777fe700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f45780fcf90 0x7f457810df40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:11:54.661 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:11:54.660+0000 7f45777fe700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f45780fcf90 0x7f457810df40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:42386/0 (socket says 172.21.15.162:42386) 2024-04-27T04:11:54.661 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:11:54.660+0000 7f45777fe700 1 -- 172.21.15.162:0/3671951864 learned_addr learned my addr 172.21.15.162:0/3671951864 (peer_addr_for_me v2:172.21.15.162:0/0) 2024-04-27T04:11:54.661 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:11:54.661+0000 7f4577fff700 1 --2- 172.21.15.162:0/3671951864 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f45780fbce0 0x7f457810da00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:11:54.662 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:11:54.661+0000 7f45777fe700 1 -- 172.21.15.162:0/3671951864 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f45780fbce0 msgr2=0x7f457810da00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:11:54.662 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:11:54.661+0000 7f45777fe700 1 --2- 172.21.15.162:0/3671951864 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f45780fbce0 0x7f457810da00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:11:54.662 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:11:54.661+0000 7f45777fe700 1 -- 172.21.15.162:0/3671951864 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4568005040 con 0x7f45780fcf90 2024-04-27T04:11:54.662 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:11:54.661+0000 7f45777fe700 1 --2- 172.21.15.162:0/3671951864 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f45780fcf90 0x7f457810df40 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f456c002a50 tx=0x7f456c0046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:11:54.662 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:11:54.661+0000 7f4574ff9700 1 -- 172.21.15.162:0/3671951864 <== mon.0 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f456c007cc0 con 0x7f45780fcf90 2024-04-27T04:11:54.662 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:11:54.661+0000 7f457e629700 1 -- 172.21.15.162:0/3671951864 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f457810f070 con 0x7f45780fcf90 2024-04-27T04:11:54.662 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:11:54.661+0000 7f4574ff9700 1 -- 172.21.15.162:0/3671951864 <== mon.0 v2:172.21.15.162:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f456c00f040 con 0x7f45780fcf90 2024-04-27T04:11:54.662 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:11:54.661+0000 7f4574ff9700 1 -- 172.21.15.162:0/3671951864 <== mon.0 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f456c00da30 con 0x7f45780fcf90 2024-04-27T04:11:54.662 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:11:54.661+0000 7f457e629700 1 -- 172.21.15.162:0/3671951864 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f457810f5b0 con 0x7f45780fcf90 2024-04-27T04:11:54.664 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:11:54.662+0000 7f457e629700 1 -- 172.21.15.162:0/3671951864 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4578101b20 con 0x7f45780fcf90 2024-04-27T04:11:54.664 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:11:54.663+0000 7f4574ff9700 1 -- 172.21.15.162:0/3671951864 <== mon.0 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 30) v1 ==== 92571+0+0 (secure 0 0 0) 0x7f456c00db90 con 0x7f45780fcf90 2024-04-27T04:11:54.664 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:11:54.663+0000 7f4574ff9700 1 --2- 172.21.15.162:0/3671951864 >> [v2:172.21.15.162:6800/901992993,v1:172.21.15.162:6801/901992993] conn(0x7f456006e620 0x7f4560070ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:11:54.664 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:11:54.663+0000 7f4574ff9700 1 -- 172.21.15.162:0/3671951864 <== mon.0 v2:172.21.15.162:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6152+0+0 (secure 0 0 0) 0x7f456c08df40 con 0x7f45780fcf90 2024-04-27T04:11:54.664 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:11:54.663+0000 7f4577fff700 1 --2- 172.21.15.162:0/3671951864 >> [v2:172.21.15.162:6800/901992993,v1:172.21.15.162:6801/901992993] conn(0x7f456006e620 0x7f4560070ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:11:54.665 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:11:54.664+0000 7f4577fff700 1 --2- 172.21.15.162:0/3671951864 >> [v2:172.21.15.162:6800/901992993,v1:172.21.15.162:6801/901992993] conn(0x7f456006e620 0x7f4560070ad0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f456800b970 tx=0x7f4568009e30 comp rx=0 tx=0).ready entity=mgr.14680 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:11:54.666 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:11:54.668+0000 7f4574ff9700 1 -- 172.21.15.162:0/3671951864 <== mon.0 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186652 (secure 0 0 0) 0x7f456c0c2930 con 0x7f45780fcf90 2024-04-27T04:11:54.961 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:11:54.962+0000 7f457e629700 1 -- 172.21.15.162:0/3671951864 --> [v2:172.21.15.162:6800/901992993,v1:172.21.15.162:6801/901992993] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f457810f7d0 con 0x7f456006e620 2024-04-27T04:11:54.971 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:11:54.972+0000 7f4574ff9700 1 -- 172.21.15.162:0/3671951864 <== mgr.14680 v2:172.21.15.162:6800/901992993 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3498 (secure 0 0 0) 0x7f457810f7d0 con 0x7f456006e620 2024-04-27T04:11:54.971 INFO:teuthology.orchestra.run.smithi162.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-27T04:11:54.971 INFO:teuthology.orchestra.run.smithi162.stdout:alertmanager.smithi162 smithi162 *:9093,9094 running (72s) 16s ago 10m 16.6M - 0.25.0 c8568f914cd2 36702db3ad50 2024-04-27T04:11:54.971 INFO:teuthology.orchestra.run.smithi162.stdout:crash.smithi162 smithi162 running (10m) 16s ago 10m 6660k - 17.2.7-907-g2be3efa9 1c5834633715 228c98df4289 2024-04-27T04:11:54.971 INFO:teuthology.orchestra.run.smithi162.stdout:crash.smithi181 smithi181 running (9m) 15s ago 9m 6665k - 17.2.7-907-g2be3efa9 1c5834633715 c8e553e3df2b 2024-04-27T04:11:54.971 INFO:teuthology.orchestra.run.smithi162.stdout:grafana.smithi162 smithi162 *:3000 running (66s) 16s ago 9m 82.4M - 9.4.7 954c08fa6188 746f08b3fb5b 2024-04-27T04:11:54.972 INFO:teuthology.orchestra.run.smithi162.stdout:mds.cephfs.smithi162.gzlyjj smithi162 running (4m) 16s ago 4m 12.3M - 17.2.7-907-g2be3efa9 1c5834633715 27e5b030dd32 2024-04-27T04:11:54.972 INFO:teuthology.orchestra.run.smithi162.stdout:mds.cephfs.smithi162.xcnapk smithi162 running (4m) 16s ago 4m 116M - 17.2.7-907-g2be3efa9 1c5834633715 38f9030df029 2024-04-27T04:11:54.972 INFO:teuthology.orchestra.run.smithi162.stdout:mds.cephfs.smithi181.bkkcdm smithi181 running (4m) 15s ago 4m 14.0M - 17.2.7-907-g2be3efa9 1c5834633715 af55afca1263 2024-04-27T04:11:54.972 INFO:teuthology.orchestra.run.smithi162.stdout:mds.cephfs.smithi181.dzomgh smithi181 running (4m) 15s ago 4m 198M - 17.2.7-907-g2be3efa9 1c5834633715 6733296597f3 2024-04-27T04:11:54.972 INFO:teuthology.orchestra.run.smithi162.stdout:mgr.smithi162.oennfn smithi162 *:8443,9283,8765 running (36s) 16s ago 11m 474M - 19.0.0-3253-g4e213a58 67e414ff4bc4 58a748d3637b 2024-04-27T04:11:54.972 INFO:teuthology.orchestra.run.smithi162.stdout:mgr.smithi181.vtwuhz smithi181 *:8443,9283 running (2m) 15s ago 9m 67.0M - 19.0.0-3253-g4e213a58 67e414ff4bc4 402cc52dbdbb 2024-04-27T04:11:54.972 INFO:teuthology.orchestra.run.smithi162.stdout:mon.smithi162 smithi162 running (11m) 16s ago 11m 55.4M 2048M 17.2.7-907-g2be3efa9 1c5834633715 0f1e3682b9ab 2024-04-27T04:11:54.972 INFO:teuthology.orchestra.run.smithi162.stdout:mon.smithi181 smithi181 running (9m) 15s ago 9m 43.0M 2048M 17.2.7-907-g2be3efa9 1c5834633715 96e48aa3912f 2024-04-27T04:11:54.972 INFO:teuthology.orchestra.run.smithi162.stdout:node-exporter.smithi162 smithi162 *:9100 running (78s) 16s ago 10m 8414k - 1.5.0 0da6a335fe13 6cc38cb8cfd1 2024-04-27T04:11:54.972 INFO:teuthology.orchestra.run.smithi162.stdout:node-exporter.smithi181 smithi181 *:9100 running (47s) 15s ago 9m 8711k - 1.5.0 0da6a335fe13 dc5ca8e89d75 2024-04-27T04:11:54.972 INFO:teuthology.orchestra.run.smithi162.stdout:osd.0 smithi162 running (8m) 16s ago 8m 222M 1557M 17.2.7-907-g2be3efa9 1c5834633715 4a9df3aa938d 2024-04-27T04:11:54.972 INFO:teuthology.orchestra.run.smithi162.stdout:osd.1 smithi162 running (8m) 16s ago 8m 236M 1557M 17.2.7-907-g2be3efa9 1c5834633715 aefb464c46e8 2024-04-27T04:11:54.972 INFO:teuthology.orchestra.run.smithi162.stdout:osd.2 smithi162 running (7m) 16s ago 7m 194M 1557M 17.2.7-907-g2be3efa9 1c5834633715 70f16d86d38e 2024-04-27T04:11:54.972 INFO:teuthology.orchestra.run.smithi162.stdout:osd.3 smithi181 running (7m) 15s ago 7m 266M 2581M 17.2.7-907-g2be3efa9 1c5834633715 030d829be54e 2024-04-27T04:11:54.973 INFO:teuthology.orchestra.run.smithi162.stdout:osd.4 smithi181 running (6m) 15s ago 6m 237M 2581M 17.2.7-907-g2be3efa9 1c5834633715 512afbee7cf7 2024-04-27T04:11:54.973 INFO:teuthology.orchestra.run.smithi162.stdout:osd.5 smithi181 running (6m) 15s ago 6m 214M 2581M 17.2.7-907-g2be3efa9 1c5834633715 f47a1daac1ea 2024-04-27T04:11:54.973 INFO:teuthology.orchestra.run.smithi162.stdout:prometheus.smithi162 smithi162 *:9095 starting - - - - 2024-04-27T04:11:54.973 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:11:54.975+0000 7f457e629700 1 -- 172.21.15.162:0/3671951864 >> [v2:172.21.15.162:6800/901992993,v1:172.21.15.162:6801/901992993] conn(0x7f456006e620 msgr2=0x7f4560070ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:11:54.973 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:11:54.975+0000 7f457e629700 1 --2- 172.21.15.162:0/3671951864 >> [v2:172.21.15.162:6800/901992993,v1:172.21.15.162:6801/901992993] conn(0x7f456006e620 0x7f4560070ad0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f456800b970 tx=0x7f4568009e30 comp rx=0 tx=0).stop 2024-04-27T04:11:54.973 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:11:54.975+0000 7f457e629700 1 -- 172.21.15.162:0/3671951864 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f45780fcf90 msgr2=0x7f457810df40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:11:54.973 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:11:54.975+0000 7f457e629700 1 --2- 172.21.15.162:0/3671951864 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f45780fcf90 0x7f457810df40 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f456c002a50 tx=0x7f456c0046b0 comp rx=0 tx=0).stop 2024-04-27T04:11:54.974 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:11:54.975+0000 7f457e629700 1 -- 172.21.15.162:0/3671951864 shutdown_connections 2024-04-27T04:11:54.974 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:11:54.975+0000 7f457e629700 1 --2- 172.21.15.162:0/3671951864 >> [v2:172.21.15.162:6800/901992993,v1:172.21.15.162:6801/901992993] conn(0x7f456006e620 0x7f4560070ad0 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:11:54.974 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:11:54.975+0000 7f457e629700 1 --2- 172.21.15.162:0/3671951864 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f45780fbce0 0x7f457810da00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:11:54.974 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:11:54.975+0000 7f457e629700 1 --2- 172.21.15.162:0/3671951864 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f45780fcf90 0x7f457810df40 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:11:54.974 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:11:54.975+0000 7f457e629700 1 -- 172.21.15.162:0/3671951864 >> 172.21.15.162:0/3671951864 conn(0x7f45780f7440 msgr2=0x7f4578066410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:11:54.974 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:11:54.975+0000 7f457e629700 1 -- 172.21.15.162:0/3671951864 shutdown_connections 2024-04-27T04:11:54.974 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:11:54.975+0000 7f457e629700 1 -- 172.21.15.162:0/3671951864 wait complete. 2024-04-27T04:11:55.081 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:11:55.081+0000 7fb2a67d5700 1 -- 172.21.15.162:0/3448845416 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb2a00fcf60 msgr2=0x7fb2a00fd3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:11:55.081 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:11:55.081+0000 7fb2a67d5700 1 --2- 172.21.15.162:0/3448845416 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb2a00fcf60 0x7fb2a00fd3d0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7fb294004940 tx=0x7fb294009e30 comp rx=0 tx=0).stop 2024-04-27T04:11:55.081 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:11:55.081+0000 7fb2a67d5700 1 -- 172.21.15.162:0/3448845416 shutdown_connections 2024-04-27T04:11:55.081 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:11:55.081+0000 7fb2a67d5700 1 --2- 172.21.15.162:0/3448845416 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb2a00fcf60 0x7fb2a00fd3d0 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:11:55.081 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:11:55.081+0000 7fb2a67d5700 1 --2- 172.21.15.162:0/3448845416 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb2a00fbcb0 0x7fb2a00fc0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:11:55.081 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:11:55.081+0000 7fb2a67d5700 1 -- 172.21.15.162:0/3448845416 >> 172.21.15.162:0/3448845416 conn(0x7fb2a00f7430 msgr2=0x7fb2a00f9840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:11:55.081 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:11:55.081+0000 7fb2a67d5700 1 -- 172.21.15.162:0/3448845416 shutdown_connections 2024-04-27T04:11:55.081 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:11:55.081+0000 7fb2a67d5700 1 -- 172.21.15.162:0/3448845416 wait complete. 2024-04-27T04:11:55.081 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:11:55.082+0000 7fb2a67d5700 1 Processor -- start 2024-04-27T04:11:55.082 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:11:55.082+0000 7fb2a67d5700 1 -- start start 2024-04-27T04:11:55.082 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:11:55.082+0000 7fb2a67d5700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb2a00fbcb0 0x7fb2a010d880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:11:55.082 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:11:55.082+0000 7fb2a67d5700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb2a00fcf60 0x7fb2a010ddc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:11:55.082 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:11:55.082+0000 7fb2a67d5700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2a010e3c0 con 0x7fb2a00fcf60 2024-04-27T04:11:55.082 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:11:55.082+0000 7fb2a67d5700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2a010e530 con 0x7fb2a00fbcb0 2024-04-27T04:11:55.082 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:11:55.083+0000 7fb29f7fe700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb2a00fcf60 0x7fb2a010ddc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:11:55.082 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:11:55.083+0000 7fb29f7fe700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb2a00fcf60 0x7fb2a010ddc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:42396/0 (socket says 172.21.15.162:42396) 2024-04-27T04:11:55.082 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:11:55.083+0000 7fb29f7fe700 1 -- 172.21.15.162:0/1467813466 learned_addr learned my addr 172.21.15.162:0/1467813466 (peer_addr_for_me v2:172.21.15.162:0/0) 2024-04-27T04:11:55.083 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:11:55.083+0000 7fb29ffff700 1 --2- 172.21.15.162:0/1467813466 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb2a00fbcb0 0x7fb2a010d880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:11:55.083 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:11:55.083+0000 7fb29f7fe700 1 -- 172.21.15.162:0/1467813466 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb2a00fbcb0 msgr2=0x7fb2a010d880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:11:55.084 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:11:55.083+0000 7fb29f7fe700 1 --2- 172.21.15.162:0/1467813466 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb2a00fbcb0 0x7fb2a010d880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:11:55.084 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:11:55.083+0000 7fb29f7fe700 1 -- 172.21.15.162:0/1467813466 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb294005040 con 0x7fb2a00fcf60 2024-04-27T04:11:55.084 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:11:55.083+0000 7fb29f7fe700 1 --2- 172.21.15.162:0/1467813466 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb2a00fcf60 0x7fb2a010ddc0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7fb294013870 tx=0x7fb294009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:11:55.084 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:11:55.083+0000 7fb29cff9700 1 -- 172.21.15.162:0/1467813466 <== mon.0 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb29400e070 con 0x7fb2a00fcf60 2024-04-27T04:11:55.084 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:11:55.083+0000 7fb29cff9700 1 -- 172.21.15.162:0/1467813466 <== mon.0 v2:172.21.15.162:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fb294013de0 con 0x7fb2a00fcf60 2024-04-27T04:11:55.084 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:11:55.083+0000 7fb2a67d5700 1 -- 172.21.15.162:0/1467813466 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb2a0112f30 con 0x7fb2a00fcf60 2024-04-27T04:11:55.084 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:11:55.083+0000 7fb29cff9700 1 -- 172.21.15.162:0/1467813466 <== mon.0 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb29401cea0 con 0x7fb2a00fcf60 2024-04-27T04:11:55.084 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:11:55.084+0000 7fb2a67d5700 1 -- 172.21.15.162:0/1467813466 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb2a01133e0 con 0x7fb2a00fcf60 2024-04-27T04:11:55.084 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:11:55.084+0000 7fb2a67d5700 1 -- 172.21.15.162:0/1467813466 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb2a004c710 con 0x7fb2a00fcf60 2024-04-27T04:11:55.084 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:11:55.085+0000 7fb29cff9700 1 -- 172.21.15.162:0/1467813466 <== mon.0 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 30) v1 ==== 92571+0+0 (secure 0 0 0) 0x7fb29401c690 con 0x7fb2a00fcf60 2024-04-27T04:11:55.085 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:11:55.085+0000 7fb29cff9700 1 --2- 172.21.15.162:0/1467813466 >> [v2:172.21.15.162:6800/901992993,v1:172.21.15.162:6801/901992993] conn(0x7fb28806e630 0x7fb288070ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:11:55.085 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:11:55.086+0000 7fb29cff9700 1 -- 172.21.15.162:0/1467813466 <== mon.0 v2:172.21.15.162:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6152+0+0 (secure 0 0 0) 0x7fb29408f880 con 0x7fb2a00fcf60 2024-04-27T04:11:55.085 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:11:55.086+0000 7fb29ffff700 1 --2- 172.21.15.162:0/1467813466 >> [v2:172.21.15.162:6800/901992993,v1:172.21.15.162:6801/901992993] conn(0x7fb28806e630 0x7fb288070ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:11:55.085 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:11:55.086+0000 7fb29ffff700 1 --2- 172.21.15.162:0/1467813466 >> [v2:172.21.15.162:6800/901992993,v1:172.21.15.162:6801/901992993] conn(0x7fb28806e630 0x7fb288070ae0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fb290009b80 tx=0x7fb29000b040 comp rx=0 tx=0).ready entity=mgr.14680 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:11:55.089 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:11:55.090+0000 7fb29cff9700 1 -- 172.21.15.162:0/1467813466 <== mon.0 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186652 (secure 0 0 0) 0x7fb29405c810 con 0x7fb2a00fcf60 2024-04-27T04:11:55.254 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:11:55 smithi162 ceph-mon[26841]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-04-27T04:11:55.254 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:11:55 smithi162 ceph-mon[26841]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-prometheus-api-host", "value": "http://smithi162:9095"}]: dispatch 2024-04-27T04:11:55.332 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:11:55 smithi181 ceph-mon[33774]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-04-27T04:11:55.332 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:11:55 smithi181 ceph-mon[33774]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-prometheus-api-host", "value": "http://smithi162:9095"}]: dispatch 2024-04-27T04:11:55.380 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:11:55.382+0000 7fb2a67d5700 1 -- 172.21.15.162:0/1467813466 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7fb2a01058b0 con 0x7fb2a00fcf60 2024-04-27T04:11:55.381 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:11:55.382+0000 7fb29cff9700 1 -- 172.21.15.162:0/1467813466 <== mon.0 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+739 (secure 0 0 0) 0x7fb294004490 con 0x7fb2a00fcf60 2024-04-27T04:11:55.381 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-04-27T04:11:55.381 INFO:teuthology.orchestra.run.smithi162.stdout: "mon": { 2024-04-27T04:11:55.381 INFO:teuthology.orchestra.run.smithi162.stdout: "ceph version 17.2.7-907-g2be3efa9 (2be3efa97a7972f8e925bd899b474329979b74d6) quincy (stable)": 2 2024-04-27T04:11:55.381 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-04-27T04:11:55.381 INFO:teuthology.orchestra.run.smithi162.stdout: "mgr": { 2024-04-27T04:11:55.382 INFO:teuthology.orchestra.run.smithi162.stdout: "ceph version 19.0.0-3253-g4e213a58 (4e213a58e787437ff508e6c890ccd5fd32dc96b5) squid (dev)": 2 2024-04-27T04:11:55.382 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-04-27T04:11:55.382 INFO:teuthology.orchestra.run.smithi162.stdout: "osd": { 2024-04-27T04:11:55.382 INFO:teuthology.orchestra.run.smithi162.stdout: "ceph version 17.2.7-907-g2be3efa9 (2be3efa97a7972f8e925bd899b474329979b74d6) quincy (stable)": 6 2024-04-27T04:11:55.382 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-04-27T04:11:55.382 INFO:teuthology.orchestra.run.smithi162.stdout: "mds": { 2024-04-27T04:11:55.382 INFO:teuthology.orchestra.run.smithi162.stdout: "ceph version 17.2.7-907-g2be3efa9 (2be3efa97a7972f8e925bd899b474329979b74d6) quincy (stable)": 4 2024-04-27T04:11:55.382 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-04-27T04:11:55.382 INFO:teuthology.orchestra.run.smithi162.stdout: "overall": { 2024-04-27T04:11:55.382 INFO:teuthology.orchestra.run.smithi162.stdout: "ceph version 17.2.7-907-g2be3efa9 (2be3efa97a7972f8e925bd899b474329979b74d6) quincy (stable)": 12, 2024-04-27T04:11:55.382 INFO:teuthology.orchestra.run.smithi162.stdout: "ceph version 19.0.0-3253-g4e213a58 (4e213a58e787437ff508e6c890ccd5fd32dc96b5) squid (dev)": 2 2024-04-27T04:11:55.382 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-04-27T04:11:55.382 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-04-27T04:11:55.384 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:11:55.385+0000 7fb2a67d5700 1 -- 172.21.15.162:0/1467813466 >> [v2:172.21.15.162:6800/901992993,v1:172.21.15.162:6801/901992993] conn(0x7fb28806e630 msgr2=0x7fb288070ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:11:55.384 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:11:55.385+0000 7fb2a67d5700 1 --2- 172.21.15.162:0/1467813466 >> [v2:172.21.15.162:6800/901992993,v1:172.21.15.162:6801/901992993] conn(0x7fb28806e630 0x7fb288070ae0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fb290009b80 tx=0x7fb29000b040 comp rx=0 tx=0).stop 2024-04-27T04:11:55.384 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:11:55.385+0000 7fb2a67d5700 1 -- 172.21.15.162:0/1467813466 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb2a00fcf60 msgr2=0x7fb2a010ddc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:11:55.384 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:11:55.385+0000 7fb2a67d5700 1 --2- 172.21.15.162:0/1467813466 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb2a00fcf60 0x7fb2a010ddc0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7fb294013870 tx=0x7fb294009e30 comp rx=0 tx=0).stop 2024-04-27T04:11:55.384 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:11:55.385+0000 7fb2a67d5700 1 -- 172.21.15.162:0/1467813466 shutdown_connections 2024-04-27T04:11:55.384 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:11:55.385+0000 7fb2a67d5700 1 --2- 172.21.15.162:0/1467813466 >> [v2:172.21.15.162:6800/901992993,v1:172.21.15.162:6801/901992993] conn(0x7fb28806e630 0x7fb288070ae0 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:11:55.384 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:11:55.385+0000 7fb2a67d5700 1 --2- 172.21.15.162:0/1467813466 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb2a00fbcb0 0x7fb2a010d880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:11:55.384 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:11:55.385+0000 7fb2a67d5700 1 --2- 172.21.15.162:0/1467813466 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb2a00fcf60 0x7fb2a010ddc0 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:11:55.384 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:11:55.385+0000 7fb2a67d5700 1 -- 172.21.15.162:0/1467813466 >> 172.21.15.162:0/1467813466 conn(0x7fb2a00f7430 msgr2=0x7fb2a00f8a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:11:55.384 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:11:55.385+0000 7fb2a67d5700 1 -- 172.21.15.162:0/1467813466 shutdown_connections 2024-04-27T04:11:55.384 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:11:55.385+0000 7fb2a67d5700 1 -- 172.21.15.162:0/1467813466 wait complete. 2024-04-27T04:11:55.493 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:11:55.493+0000 7fad7bd56700 1 -- 172.21.15.162:0/2170220175 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fad740fcf90 msgr2=0x7fad740fd400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:11:55.493 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:11:55.493+0000 7fad7bd56700 1 --2- 172.21.15.162:0/2170220175 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fad740fcf90 0x7fad740fd400 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7fad64004940 tx=0x7fad64009e30 comp rx=0 tx=0).stop 2024-04-27T04:11:55.493 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:11:55.493+0000 7fad7bd56700 1 -- 172.21.15.162:0/2170220175 shutdown_connections 2024-04-27T04:11:55.493 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:11:55.493+0000 7fad7bd56700 1 --2- 172.21.15.162:0/2170220175 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fad740fcf90 0x7fad740fd400 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:11:55.493 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:11:55.493+0000 7fad7bd56700 1 --2- 172.21.15.162:0/2170220175 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fad740fbce0 0x7fad740fc0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:11:55.494 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:11:55.493+0000 7fad7bd56700 1 -- 172.21.15.162:0/2170220175 >> 172.21.15.162:0/2170220175 conn(0x7fad740f7420 msgr2=0x7fad740f9870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:11:55.494 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:11:55.493+0000 7fad7bd56700 1 -- 172.21.15.162:0/2170220175 shutdown_connections 2024-04-27T04:11:55.494 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:11:55.493+0000 7fad7bd56700 1 -- 172.21.15.162:0/2170220175 wait complete. 2024-04-27T04:11:55.494 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:11:55.494+0000 7fad7bd56700 1 Processor -- start 2024-04-27T04:11:55.494 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:11:55.494+0000 7fad7bd56700 1 -- start start 2024-04-27T04:11:55.494 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:11:55.494+0000 7fad7bd56700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fad740fbce0 0x7fad7410da00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:11:55.494 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:11:55.494+0000 7fad7bd56700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fad740fcf90 0x7fad7410df40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:11:55.494 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:11:55.494+0000 7fad7bd56700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad7410e540 con 0x7fad740fbce0 2024-04-27T04:11:55.494 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:11:55.494+0000 7fad7bd56700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad7410e6b0 con 0x7fad740fcf90 2024-04-27T04:11:55.494 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:11:55.494+0000 7fad79af2700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fad740fbce0 0x7fad7410da00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:11:55.494 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:11:55.494+0000 7fad79af2700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fad740fbce0 0x7fad7410da00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:42412/0 (socket says 172.21.15.162:42412) 2024-04-27T04:11:55.494 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:11:55.494+0000 7fad79af2700 1 -- 172.21.15.162:0/752617478 learned_addr learned my addr 172.21.15.162:0/752617478 (peer_addr_for_me v2:172.21.15.162:0/0) 2024-04-27T04:11:55.495 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:11:55.494+0000 7fad792f1700 1 --2- 172.21.15.162:0/752617478 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fad740fcf90 0x7fad7410df40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:11:55.495 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:11:55.494+0000 7fad79af2700 1 -- 172.21.15.162:0/752617478 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fad740fcf90 msgr2=0x7fad7410df40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:11:55.495 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:11:55.494+0000 7fad79af2700 1 --2- 172.21.15.162:0/752617478 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fad740fcf90 0x7fad7410df40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:11:55.495 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:11:55.494+0000 7fad79af2700 1 -- 172.21.15.162:0/752617478 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad64005040 con 0x7fad740fbce0 2024-04-27T04:11:55.495 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:11:55.494+0000 7fad79af2700 1 --2- 172.21.15.162:0/752617478 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fad740fbce0 0x7fad7410da00 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7fad70002a50 tx=0x7fad700046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:11:55.495 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:11:55.494+0000 7fad6a7fc700 1 -- 172.21.15.162:0/752617478 <== mon.0 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fad70007cc0 con 0x7fad740fbce0 2024-04-27T04:11:55.495 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:11:55.495+0000 7fad6a7fc700 1 -- 172.21.15.162:0/752617478 <== mon.0 v2:172.21.15.162:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fad7000f040 con 0x7fad740fbce0 2024-04-27T04:11:55.495 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:11:55.495+0000 7fad7bd56700 1 -- 172.21.15.162:0/752617478 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fad74113070 con 0x7fad740fbce0 2024-04-27T04:11:55.496 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:11:55.495+0000 7fad6a7fc700 1 -- 172.21.15.162:0/752617478 <== mon.0 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fad7000da30 con 0x7fad740fbce0 2024-04-27T04:11:55.496 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:11:55.495+0000 7fad7bd56700 1 -- 172.21.15.162:0/752617478 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fad74113630 con 0x7fad740fbce0 2024-04-27T04:11:55.496 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:11:55.496+0000 7fad6a7fc700 1 -- 172.21.15.162:0/752617478 <== mon.0 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 30) v1 ==== 92571+0+0 (secure 0 0 0) 0x7fad7000dc70 con 0x7fad740fbce0 2024-04-27T04:11:55.496 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:11:55.496+0000 7fad7bd56700 1 -- 172.21.15.162:0/752617478 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fad74101b20 con 0x7fad740fbce0 2024-04-27T04:11:55.496 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:11:55.496+0000 7fad6a7fc700 1 --2- 172.21.15.162:0/752617478 >> [v2:172.21.15.162:6800/901992993,v1:172.21.15.162:6801/901992993] conn(0x7fad6006e420 0x7fad600708d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:11:55.496 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:11:55.497+0000 7fad6a7fc700 1 -- 172.21.15.162:0/752617478 <== mon.0 v2:172.21.15.162:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6152+0+0 (secure 0 0 0) 0x7fad7008d740 con 0x7fad740fbce0 2024-04-27T04:11:55.497 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:11:55.497+0000 7fad792f1700 1 --2- 172.21.15.162:0/752617478 >> [v2:172.21.15.162:6800/901992993,v1:172.21.15.162:6801/901992993] conn(0x7fad6006e420 0x7fad600708d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:11:55.497 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:11:55.497+0000 7fad792f1700 1 --2- 172.21.15.162:0/752617478 >> [v2:172.21.15.162:6800/901992993,v1:172.21.15.162:6801/901992993] conn(0x7fad6006e420 0x7fad600708d0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fad6400b3b0 tx=0x7fad64009e30 comp rx=0 tx=0).ready entity=mgr.14680 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:11:55.499 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:11:55.501+0000 7fad6a7fc700 1 -- 172.21.15.162:0/752617478 <== mon.0 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186652 (secure 0 0 0) 0x7fad7005a640 con 0x7fad740fbce0 2024-04-27T04:11:55.726 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:11:55.728+0000 7fad7bd56700 1 -- 172.21.15.162:0/752617478 --> [v2:172.21.15.162:6800/901992993,v1:172.21.15.162:6801/901992993] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fad7402b300 con 0x7fad6006e420 2024-04-27T04:11:55.727 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:11:55.729+0000 7fad6a7fc700 1 -- 172.21.15.162:0/752617478 <== mgr.14680 v2:172.21.15.162:6800/901992993 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+360 (secure 0 0 0) 0x7fad7402b300 con 0x7fad6006e420 2024-04-27T04:11:55.728 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-04-27T04:11:55.728 INFO:teuthology.orchestra.run.smithi162.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:90a3a9fa9ceb8977f6dad9ccd36fe46018d07e2118199820523a6b35301d9c16", 2024-04-27T04:11:55.728 INFO:teuthology.orchestra.run.smithi162.stdout: "in_progress": true, 2024-04-27T04:11:55.728 INFO:teuthology.orchestra.run.smithi162.stdout: "which": "Upgrading daemons of type(s) mgr", 2024-04-27T04:11:55.728 INFO:teuthology.orchestra.run.smithi162.stdout: "services_complete": [ 2024-04-27T04:11:55.728 INFO:teuthology.orchestra.run.smithi162.stdout: "mgr" 2024-04-27T04:11:55.728 INFO:teuthology.orchestra.run.smithi162.stdout: ], 2024-04-27T04:11:55.728 INFO:teuthology.orchestra.run.smithi162.stdout: "progress": "2/2 daemons upgraded", 2024-04-27T04:11:55.728 INFO:teuthology.orchestra.run.smithi162.stdout: "message": "Currently upgrading mgr daemons", 2024-04-27T04:11:55.728 INFO:teuthology.orchestra.run.smithi162.stdout: "is_paused": false 2024-04-27T04:11:55.728 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-04-27T04:11:55.730 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:11:55.731+0000 7fad7bd56700 1 -- 172.21.15.162:0/752617478 >> [v2:172.21.15.162:6800/901992993,v1:172.21.15.162:6801/901992993] conn(0x7fad6006e420 msgr2=0x7fad600708d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:11:55.730 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:11:55.731+0000 7fad7bd56700 1 --2- 172.21.15.162:0/752617478 >> [v2:172.21.15.162:6800/901992993,v1:172.21.15.162:6801/901992993] conn(0x7fad6006e420 0x7fad600708d0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fad6400b3b0 tx=0x7fad64009e30 comp rx=0 tx=0).stop 2024-04-27T04:11:55.730 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:11:55.731+0000 7fad7bd56700 1 -- 172.21.15.162:0/752617478 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fad740fbce0 msgr2=0x7fad7410da00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:11:55.730 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:11:55.731+0000 7fad7bd56700 1 --2- 172.21.15.162:0/752617478 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fad740fbce0 0x7fad7410da00 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7fad70002a50 tx=0x7fad700046b0 comp rx=0 tx=0).stop 2024-04-27T04:11:55.730 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:11:55.731+0000 7fad7bd56700 1 -- 172.21.15.162:0/752617478 shutdown_connections 2024-04-27T04:11:55.730 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:11:55.731+0000 7fad7bd56700 1 --2- 172.21.15.162:0/752617478 >> [v2:172.21.15.162:6800/901992993,v1:172.21.15.162:6801/901992993] conn(0x7fad6006e420 0x7fad600708d0 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:11:55.730 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:11:55.731+0000 7fad7bd56700 1 --2- 172.21.15.162:0/752617478 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fad740fbce0 0x7fad7410da00 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:11:55.730 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:11:55.731+0000 7fad7bd56700 1 --2- 172.21.15.162:0/752617478 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fad740fcf90 0x7fad7410df40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:11:55.731 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:11:55.731+0000 7fad7bd56700 1 -- 172.21.15.162:0/752617478 >> 172.21.15.162:0/752617478 conn(0x7fad740f7420 msgr2=0x7fad740667c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:11:55.731 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:11:55.732+0000 7fad7bd56700 1 -- 172.21.15.162:0/752617478 shutdown_connections 2024-04-27T04:11:55.731 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:11:55.732+0000 7fad7bd56700 1 -- 172.21.15.162:0/752617478 wait complete. 2024-04-27T04:11:56.252 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:11:56 smithi181 ceph-mon[33774]: from='client.14726 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-27T04:11:56.253 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:11:56 smithi181 ceph-mon[33774]: pgmap v14: 65 pgs: 65 active+clean; 221 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 1.5 MiB/s rd, 1.5 MiB/s wr, 442 op/s 2024-04-27T04:11:56.253 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:11:56 smithi181 ceph-mon[33774]: from='client.14730 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-27T04:11:56.253 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:11:56 smithi181 ceph-mon[33774]: from='client.14734 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-27T04:11:56.253 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:11:56 smithi181 ceph-mon[33774]: from='client.? 172.21.15.162:0/1467813466' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-27T04:11:56.253 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:11:56 smithi181 ceph-mon[33774]: Upgrade: Updating mgr.smithi181.vtwuhz 2024-04-27T04:11:56.253 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:11:56 smithi181 ceph-mon[33774]: from='mgr.14680 172.21.15.162:0/501880828' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi181.vtwuhz", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-27T04:11:56.253 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:11:56 smithi181 ceph-mon[33774]: from='mgr.14680 172.21.15.162:0/501880828' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-27T04:11:56.253 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:11:56 smithi181 ceph-mon[33774]: from='mgr.14680 172.21.15.162:0/501880828' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T04:11:56.253 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:11:56 smithi181 ceph-mon[33774]: Deploying daemon mgr.smithi181.vtwuhz on smithi181 2024-04-27T04:11:56.253 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:11:56 smithi181 ceph-mon[33774]: from='client.14742 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-27T04:11:56.387 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:11:56 smithi162 ceph-mon[26841]: from='client.14726 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-27T04:11:56.387 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:11:56 smithi162 ceph-mon[26841]: pgmap v14: 65 pgs: 65 active+clean; 221 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 1.5 MiB/s rd, 1.5 MiB/s wr, 442 op/s 2024-04-27T04:11:56.387 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:11:56 smithi162 ceph-mon[26841]: from='client.14730 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-27T04:11:56.387 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:11:56 smithi162 ceph-mon[26841]: from='client.14734 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-27T04:11:56.388 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:11:56 smithi162 ceph-mon[26841]: from='client.? 172.21.15.162:0/1467813466' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-27T04:11:56.388 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:11:56 smithi162 ceph-mon[26841]: Upgrade: Updating mgr.smithi181.vtwuhz 2024-04-27T04:11:56.388 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:11:56 smithi162 ceph-mon[26841]: from='mgr.14680 172.21.15.162:0/501880828' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi181.vtwuhz", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-27T04:11:56.388 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:11:56 smithi162 ceph-mon[26841]: from='mgr.14680 172.21.15.162:0/501880828' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-27T04:11:56.388 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:11:56 smithi162 ceph-mon[26841]: from='mgr.14680 172.21.15.162:0/501880828' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T04:11:56.388 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:11:56 smithi162 ceph-mon[26841]: Deploying daemon mgr.smithi181.vtwuhz on smithi181 2024-04-27T04:11:56.388 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:11:56 smithi162 ceph-mon[26841]: from='client.14742 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-27T04:11:57.260 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:11:57 smithi181 ceph-mon[33774]: pgmap v15: 65 pgs: 65 active+clean; 221 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 1016 KiB/s rd, 1.0 MiB/s wr, 290 op/s 2024-04-27T04:11:57.387 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:11:57 smithi162 ceph-mon[26841]: pgmap v15: 65 pgs: 65 active+clean; 221 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 1016 KiB/s rd, 1.0 MiB/s wr, 290 op/s 2024-04-27T04:11:59.361 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:11:59 smithi181 ceph-mon[33774]: pgmap v16: 65 pgs: 65 active+clean; 221 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 1016 KiB/s rd, 1.0 MiB/s wr, 290 op/s 2024-04-27T04:11:59.361 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:11:59 smithi181 ceph-mon[33774]: from='mgr.14680 172.21.15.162:0/501880828' entity='mgr.smithi162.oennfn' 2024-04-27T04:11:59.361 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:11:59 smithi181 ceph-mon[33774]: from='mgr.14680 172.21.15.162:0/501880828' entity='mgr.smithi162.oennfn' 2024-04-27T04:11:59.361 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:11:59 smithi181 ceph-mon[33774]: from='mgr.14680 172.21.15.162:0/501880828' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T04:11:59.637 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:11:59 smithi162 ceph-mon[26841]: pgmap v16: 65 pgs: 65 active+clean; 221 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 1016 KiB/s rd, 1.0 MiB/s wr, 290 op/s 2024-04-27T04:11:59.637 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:11:59 smithi162 ceph-mon[26841]: from='mgr.14680 172.21.15.162:0/501880828' entity='mgr.smithi162.oennfn' 2024-04-27T04:11:59.637 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:11:59 smithi162 ceph-mon[26841]: from='mgr.14680 172.21.15.162:0/501880828' entity='mgr.smithi162.oennfn' 2024-04-27T04:11:59.637 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:11:59 smithi162 ceph-mon[26841]: from='mgr.14680 172.21.15.162:0/501880828' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T04:12:01.500 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:12:01 smithi181 ceph-mon[33774]: pgmap v17: 65 pgs: 65 active+clean; 208 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 1.0 MiB/s rd, 1.1 MiB/s wr, 343 op/s 2024-04-27T04:12:01.637 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:12:01 smithi162 ceph-mon[26841]: pgmap v17: 65 pgs: 65 active+clean; 208 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 1.0 MiB/s rd, 1.1 MiB/s wr, 343 op/s 2024-04-27T04:12:03.385 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:12:03 smithi162 ceph-mon[26841]: from='mgr.14680 172.21.15.162:0/501880828' entity='mgr.smithi162.oennfn' 2024-04-27T04:12:03.386 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:12:03 smithi162 ceph-mon[26841]: from='mgr.14680 172.21.15.162:0/501880828' entity='mgr.smithi162.oennfn' 2024-04-27T04:12:03.386 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:12:03 smithi162 ceph-mon[26841]: pgmap v18: 65 pgs: 65 active+clean; 208 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 646 KiB/s rd, 559 KiB/s wr, 196 op/s 2024-04-27T04:12:03.386 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:12:03 smithi162 ceph-mon[26841]: from='mgr.14680 172.21.15.162:0/501880828' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-27T04:12:03.422 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:12:03 smithi181 ceph-mon[33774]: from='mgr.14680 172.21.15.162:0/501880828' entity='mgr.smithi162.oennfn' 2024-04-27T04:12:03.422 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:12:03 smithi181 ceph-mon[33774]: from='mgr.14680 172.21.15.162:0/501880828' entity='mgr.smithi162.oennfn' 2024-04-27T04:12:03.422 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:12:03 smithi181 ceph-mon[33774]: pgmap v18: 65 pgs: 65 active+clean; 208 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 646 KiB/s rd, 559 KiB/s wr, 196 op/s 2024-04-27T04:12:03.423 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:12:03 smithi181 ceph-mon[33774]: from='mgr.14680 172.21.15.162:0/501880828' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-27T04:12:04.422 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:12:04 smithi181 ceph-mon[33774]: from='mgr.14680 172.21.15.162:0/501880828' entity='mgr.smithi162.oennfn' 2024-04-27T04:12:04.423 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:12:04 smithi181 ceph-mon[33774]: from='mgr.14680 172.21.15.162:0/501880828' entity='mgr.smithi162.oennfn' 2024-04-27T04:12:04.423 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:12:04 smithi181 ceph-mon[33774]: from='mgr.14680 172.21.15.162:0/501880828' entity='mgr.smithi162.oennfn' 2024-04-27T04:12:04.423 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:12:04 smithi181 ceph-mon[33774]: from='mgr.14680 172.21.15.162:0/501880828' entity='mgr.smithi162.oennfn' 2024-04-27T04:12:04.637 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:12:04 smithi162 ceph-mon[26841]: from='mgr.14680 172.21.15.162:0/501880828' entity='mgr.smithi162.oennfn' 2024-04-27T04:12:04.637 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:12:04 smithi162 ceph-mon[26841]: from='mgr.14680 172.21.15.162:0/501880828' entity='mgr.smithi162.oennfn' 2024-04-27T04:12:04.637 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:12:04 smithi162 ceph-mon[26841]: from='mgr.14680 172.21.15.162:0/501880828' entity='mgr.smithi162.oennfn' 2024-04-27T04:12:04.637 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:12:04 smithi162 ceph-mon[26841]: from='mgr.14680 172.21.15.162:0/501880828' entity='mgr.smithi162.oennfn' 2024-04-27T04:12:06.243 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:12:06 smithi181 ceph-mon[33774]: pgmap v19: 65 pgs: 65 active+clean; 208 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 647 KiB/s rd, 559 KiB/s wr, 197 op/s 2024-04-27T04:12:06.387 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:12:06 smithi162 ceph-mon[26841]: pgmap v19: 65 pgs: 65 active+clean; 208 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 647 KiB/s rd, 559 KiB/s wr, 197 op/s 2024-04-27T04:12:07.387 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:12:07 smithi162 ceph-mon[26841]: Standby manager daemon smithi181.vtwuhz restarted 2024-04-27T04:12:07.387 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:12:07 smithi162 ceph-mon[26841]: Standby manager daemon smithi181.vtwuhz started 2024-04-27T04:12:07.387 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:12:07 smithi162 ceph-mon[26841]: from='mgr.? 172.21.15.181:0/101709953' entity='mgr.smithi181.vtwuhz' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi181.vtwuhz/crt"}]: dispatch 2024-04-27T04:12:07.387 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:12:07 smithi162 ceph-mon[26841]: from='mgr.? 172.21.15.181:0/101709953' entity='mgr.smithi181.vtwuhz' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-27T04:12:07.387 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:12:07 smithi162 ceph-mon[26841]: from='mgr.? 172.21.15.181:0/101709953' entity='mgr.smithi181.vtwuhz' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi181.vtwuhz/key"}]: dispatch 2024-04-27T04:12:07.387 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:12:07 smithi162 ceph-mon[26841]: from='mgr.? 172.21.15.181:0/101709953' entity='mgr.smithi181.vtwuhz' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-27T04:12:07.422 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:12:07 smithi181 ceph-mon[33774]: Standby manager daemon smithi181.vtwuhz restarted 2024-04-27T04:12:07.422 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:12:07 smithi181 ceph-mon[33774]: Standby manager daemon smithi181.vtwuhz started 2024-04-27T04:12:07.423 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:12:07 smithi181 ceph-mon[33774]: from='mgr.? 172.21.15.181:0/101709953' entity='mgr.smithi181.vtwuhz' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi181.vtwuhz/crt"}]: dispatch 2024-04-27T04:12:07.423 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:12:07 smithi181 ceph-mon[33774]: from='mgr.? 172.21.15.181:0/101709953' entity='mgr.smithi181.vtwuhz' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-27T04:12:07.423 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:12:07 smithi181 ceph-mon[33774]: from='mgr.? 172.21.15.181:0/101709953' entity='mgr.smithi181.vtwuhz' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi181.vtwuhz/key"}]: dispatch 2024-04-27T04:12:07.423 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:12:07 smithi181 ceph-mon[33774]: from='mgr.? 172.21.15.181:0/101709953' entity='mgr.smithi181.vtwuhz' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-27T04:12:08.137 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:12:08 smithi162 ceph-mon[26841]: pgmap v20: 65 pgs: 65 active+clean; 208 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 58 KiB/s rd, 40 KiB/s wr, 54 op/s 2024-04-27T04:12:08.137 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:12:08 smithi162 ceph-mon[26841]: mgrmap e31: smithi162.oennfn(active, since 34s), standbys: smithi181.vtwuhz 2024-04-27T04:12:08.137 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:12:08 smithi162 ceph-mon[26841]: from='mgr.14680 172.21.15.162:0/501880828' entity='mgr.smithi162.oennfn' 2024-04-27T04:12:08.137 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:12:08 smithi162 ceph-mon[26841]: from='mgr.14680 172.21.15.162:0/501880828' entity='mgr.smithi162.oennfn' 2024-04-27T04:12:08.137 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:12:08 smithi162 ceph-mon[26841]: from='mgr.14680 172.21.15.162:0/501880828' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T04:12:08.137 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:12:08 smithi162 ceph-mon[26841]: from='mgr.14680 172.21.15.162:0/501880828' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T04:12:08.137 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:12:08 smithi162 ceph-mon[26841]: from='mgr.14680 172.21.15.162:0/501880828' entity='mgr.smithi162.oennfn' 2024-04-27T04:12:08.138 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:12:08 smithi162 ceph-mon[26841]: from='mgr.14680 172.21.15.162:0/501880828' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T04:12:08.138 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:12:08 smithi162 ceph-mon[26841]: from='mgr.14680 172.21.15.162:0/501880828' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "versions"}]: dispatch 2024-04-27T04:12:08.138 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:12:08 smithi162 ceph-mon[26841]: from='mgr.14680 172.21.15.162:0/501880828' entity='mgr.smithi162.oennfn' 2024-04-27T04:12:08.138 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:12:08 smithi162 ceph-mon[26841]: from='mgr.14680 172.21.15.162:0/501880828' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mgr.smithi162.oennfn"}]: dispatch 2024-04-27T04:12:08.138 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:12:08 smithi162 ceph-mon[26841]: from='mgr.14680 172.21.15.162:0/501880828' entity='mgr.smithi162.oennfn' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mgr.smithi162.oennfn"}]': finished 2024-04-27T04:12:08.138 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:12:08 smithi162 ceph-mon[26841]: from='mgr.14680 172.21.15.162:0/501880828' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mgr.smithi181.vtwuhz"}]: dispatch 2024-04-27T04:12:08.138 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:12:08 smithi162 ceph-mon[26841]: from='mgr.14680 172.21.15.162:0/501880828' entity='mgr.smithi162.oennfn' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mgr.smithi181.vtwuhz"}]': finished 2024-04-27T04:12:08.138 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:12:08 smithi162 ceph-mon[26841]: from='mgr.14680 172.21.15.162:0/501880828' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "versions"}]: dispatch 2024-04-27T04:12:08.138 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:12:08 smithi162 ceph-mon[26841]: from='mgr.14680 172.21.15.162:0/501880828' entity='mgr.smithi162.oennfn' 2024-04-27T04:12:08.138 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:12:08 smithi162 ceph-mon[26841]: from='mgr.14680 172.21.15.162:0/501880828' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "versions"}]: dispatch 2024-04-27T04:12:08.138 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:12:08 smithi162 ceph-mon[26841]: from='mgr.14680 172.21.15.162:0/501880828' entity='mgr.smithi162.oennfn' 2024-04-27T04:12:08.138 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:12:08 smithi162 ceph-mon[26841]: from='mgr.14680 172.21.15.162:0/501880828' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "versions"}]: dispatch 2024-04-27T04:12:08.138 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:12:08 smithi162 ceph-mon[26841]: from='mgr.14680 172.21.15.162:0/501880828' entity='mgr.smithi162.oennfn' 2024-04-27T04:12:08.138 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:12:08 smithi162 ceph-mon[26841]: from='mgr.14680 172.21.15.162:0/501880828' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "versions"}]: dispatch 2024-04-27T04:12:08.139 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:12:08 smithi162 ceph-mon[26841]: from='mgr.14680 172.21.15.162:0/501880828' entity='mgr.smithi162.oennfn' 2024-04-27T04:12:08.139 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:12:08 smithi162 ceph-mon[26841]: from='mgr.14680 172.21.15.162:0/501880828' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "versions"}]: dispatch 2024-04-27T04:12:08.139 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:12:08 smithi162 ceph-mon[26841]: from='mgr.14680 172.21.15.162:0/501880828' entity='mgr.smithi162.oennfn' 2024-04-27T04:12:08.139 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:12:08 smithi162 ceph-mon[26841]: from='mgr.14680 172.21.15.162:0/501880828' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "versions"}]: dispatch 2024-04-27T04:12:08.139 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:12:08 smithi162 ceph-mon[26841]: from='mgr.14680 172.21.15.162:0/501880828' entity='mgr.smithi162.oennfn' 2024-04-27T04:12:08.139 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:12:08 smithi162 ceph-mon[26841]: from='mgr.14680 172.21.15.162:0/501880828' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "versions"}]: dispatch 2024-04-27T04:12:08.139 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:12:08 smithi162 ceph-mon[26841]: from='mgr.14680 172.21.15.162:0/501880828' entity='mgr.smithi162.oennfn' 2024-04-27T04:12:08.422 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:12:08 smithi181 ceph-mon[33774]: pgmap v20: 65 pgs: 65 active+clean; 208 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 58 KiB/s rd, 40 KiB/s wr, 54 op/s 2024-04-27T04:12:08.423 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:12:08 smithi181 ceph-mon[33774]: mgrmap e31: smithi162.oennfn(active, since 34s), standbys: smithi181.vtwuhz 2024-04-27T04:12:08.423 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:12:08 smithi181 ceph-mon[33774]: from='mgr.14680 172.21.15.162:0/501880828' entity='mgr.smithi162.oennfn' 2024-04-27T04:12:08.423 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:12:08 smithi181 ceph-mon[33774]: from='mgr.14680 172.21.15.162:0/501880828' entity='mgr.smithi162.oennfn' 2024-04-27T04:12:08.423 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:12:08 smithi181 ceph-mon[33774]: from='mgr.14680 172.21.15.162:0/501880828' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T04:12:08.423 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:12:08 smithi181 ceph-mon[33774]: from='mgr.14680 172.21.15.162:0/501880828' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T04:12:08.423 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:12:08 smithi181 ceph-mon[33774]: from='mgr.14680 172.21.15.162:0/501880828' entity='mgr.smithi162.oennfn' 2024-04-27T04:12:08.423 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:12:08 smithi181 ceph-mon[33774]: from='mgr.14680 172.21.15.162:0/501880828' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T04:12:08.423 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:12:08 smithi181 ceph-mon[33774]: from='mgr.14680 172.21.15.162:0/501880828' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "versions"}]: dispatch 2024-04-27T04:12:08.423 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:12:08 smithi181 ceph-mon[33774]: from='mgr.14680 172.21.15.162:0/501880828' entity='mgr.smithi162.oennfn' 2024-04-27T04:12:08.423 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:12:08 smithi181 ceph-mon[33774]: from='mgr.14680 172.21.15.162:0/501880828' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mgr.smithi162.oennfn"}]: dispatch 2024-04-27T04:12:08.423 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:12:08 smithi181 ceph-mon[33774]: from='mgr.14680 172.21.15.162:0/501880828' entity='mgr.smithi162.oennfn' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mgr.smithi162.oennfn"}]': finished 2024-04-27T04:12:08.423 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:12:08 smithi181 ceph-mon[33774]: from='mgr.14680 172.21.15.162:0/501880828' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mgr.smithi181.vtwuhz"}]: dispatch 2024-04-27T04:12:08.423 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:12:08 smithi181 ceph-mon[33774]: from='mgr.14680 172.21.15.162:0/501880828' entity='mgr.smithi162.oennfn' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mgr.smithi181.vtwuhz"}]': finished 2024-04-27T04:12:08.423 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:12:08 smithi181 ceph-mon[33774]: from='mgr.14680 172.21.15.162:0/501880828' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "versions"}]: dispatch 2024-04-27T04:12:08.424 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:12:08 smithi181 ceph-mon[33774]: from='mgr.14680 172.21.15.162:0/501880828' entity='mgr.smithi162.oennfn' 2024-04-27T04:12:08.424 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:12:08 smithi181 ceph-mon[33774]: from='mgr.14680 172.21.15.162:0/501880828' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "versions"}]: dispatch 2024-04-27T04:12:08.424 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:12:08 smithi181 ceph-mon[33774]: from='mgr.14680 172.21.15.162:0/501880828' entity='mgr.smithi162.oennfn' 2024-04-27T04:12:08.424 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:12:08 smithi181 ceph-mon[33774]: from='mgr.14680 172.21.15.162:0/501880828' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "versions"}]: dispatch 2024-04-27T04:12:08.424 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:12:08 smithi181 ceph-mon[33774]: from='mgr.14680 172.21.15.162:0/501880828' entity='mgr.smithi162.oennfn' 2024-04-27T04:12:08.424 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:12:08 smithi181 ceph-mon[33774]: from='mgr.14680 172.21.15.162:0/501880828' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "versions"}]: dispatch 2024-04-27T04:12:08.424 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:12:08 smithi181 ceph-mon[33774]: from='mgr.14680 172.21.15.162:0/501880828' entity='mgr.smithi162.oennfn' 2024-04-27T04:12:08.424 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:12:08 smithi181 ceph-mon[33774]: from='mgr.14680 172.21.15.162:0/501880828' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "versions"}]: dispatch 2024-04-27T04:12:08.424 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:12:08 smithi181 ceph-mon[33774]: from='mgr.14680 172.21.15.162:0/501880828' entity='mgr.smithi162.oennfn' 2024-04-27T04:12:08.424 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:12:08 smithi181 ceph-mon[33774]: from='mgr.14680 172.21.15.162:0/501880828' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "versions"}]: dispatch 2024-04-27T04:12:08.424 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:12:08 smithi181 ceph-mon[33774]: from='mgr.14680 172.21.15.162:0/501880828' entity='mgr.smithi162.oennfn' 2024-04-27T04:12:08.424 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:12:08 smithi181 ceph-mon[33774]: from='mgr.14680 172.21.15.162:0/501880828' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "versions"}]: dispatch 2024-04-27T04:12:08.424 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:12:08 smithi181 ceph-mon[33774]: from='mgr.14680 172.21.15.162:0/501880828' entity='mgr.smithi162.oennfn' 2024-04-27T04:12:09.387 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:12:09 smithi162 ceph-mon[26841]: Upgrade: Setting container_image for all mgr 2024-04-27T04:12:09.387 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:12:09 smithi162 ceph-mon[26841]: Upgrade: Setting container_image for all rgw 2024-04-27T04:12:09.387 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:12:09 smithi162 ceph-mon[26841]: Upgrade: Setting container_image for all rbd-mirror 2024-04-27T04:12:09.387 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:12:09 smithi162 ceph-mon[26841]: Upgrade: Setting container_image for all cephfs-mirror 2024-04-27T04:12:09.387 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:12:09 smithi162 ceph-mon[26841]: Upgrade: Setting container_image for all ceph-exporter 2024-04-27T04:12:09.387 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:12:09 smithi162 ceph-mon[26841]: Upgrade: Setting container_image for all iscsi 2024-04-27T04:12:09.387 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:12:09 smithi162 ceph-mon[26841]: Upgrade: Setting container_image for all nfs 2024-04-27T04:12:09.387 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:12:09 smithi162 ceph-mon[26841]: Upgrade: Setting container_image for all nvmeof 2024-04-27T04:12:09.422 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:12:09 smithi181 ceph-mon[33774]: Upgrade: Setting container_image for all mgr 2024-04-27T04:12:09.422 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:12:09 smithi181 ceph-mon[33774]: Upgrade: Setting container_image for all rgw 2024-04-27T04:12:09.422 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:12:09 smithi181 ceph-mon[33774]: Upgrade: Setting container_image for all rbd-mirror 2024-04-27T04:12:09.422 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:12:09 smithi181 ceph-mon[33774]: Upgrade: Setting container_image for all cephfs-mirror 2024-04-27T04:12:09.422 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:12:09 smithi181 ceph-mon[33774]: Upgrade: Setting container_image for all ceph-exporter 2024-04-27T04:12:09.422 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:12:09 smithi181 ceph-mon[33774]: Upgrade: Setting container_image for all iscsi 2024-04-27T04:12:09.422 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:12:09 smithi181 ceph-mon[33774]: Upgrade: Setting container_image for all nfs 2024-04-27T04:12:09.422 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:12:09 smithi181 ceph-mon[33774]: Upgrade: Setting container_image for all nvmeof 2024-04-27T04:12:10.387 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:12:10 smithi162 ceph-mon[26841]: pgmap v21: 65 pgs: 65 active+clean; 208 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 58 KiB/s rd, 40 KiB/s wr, 54 op/s 2024-04-27T04:12:10.387 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:12:10 smithi162 ceph-mon[26841]: Upgrade: Updating node-exporter.smithi162 (1/2) 2024-04-27T04:12:10.388 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:12:10 smithi162 ceph-mon[26841]: Deploying daemon node-exporter.smithi162 on smithi162 2024-04-27T04:12:10.422 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:12:10 smithi181 ceph-mon[33774]: pgmap v21: 65 pgs: 65 active+clean; 208 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 58 KiB/s rd, 40 KiB/s wr, 54 op/s 2024-04-27T04:12:10.422 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:12:10 smithi181 ceph-mon[33774]: Upgrade: Updating node-exporter.smithi162 (1/2) 2024-04-27T04:12:10.422 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:12:10 smithi181 ceph-mon[33774]: Deploying daemon node-exporter.smithi162 on smithi162 2024-04-27T04:12:11.362 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:12:11 smithi162 ceph-mon[26841]: pgmap v22: 65 pgs: 65 active+clean; 208 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 59 KiB/s rd, 40 KiB/s wr, 55 op/s 2024-04-27T04:12:11.422 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:12:11 smithi181 ceph-mon[33774]: pgmap v22: 65 pgs: 65 active+clean; 208 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 59 KiB/s rd, 40 KiB/s wr, 55 op/s 2024-04-27T04:12:13.172 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:12:12 smithi181 ceph-mon[33774]: from='mgr.14680 172.21.15.162:0/501880828' entity='mgr.smithi162.oennfn' 2024-04-27T04:12:13.172 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:12:12 smithi181 ceph-mon[33774]: from='mgr.14680 172.21.15.162:0/501880828' entity='mgr.smithi162.oennfn' 2024-04-27T04:12:13.387 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:12:12 smithi162 ceph-mon[26841]: from='mgr.14680 172.21.15.162:0/501880828' entity='mgr.smithi162.oennfn' 2024-04-27T04:12:13.387 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:12:12 smithi162 ceph-mon[26841]: from='mgr.14680 172.21.15.162:0/501880828' entity='mgr.smithi162.oennfn' 2024-04-27T04:12:14.343 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:12:13 smithi181 ceph-mon[33774]: pgmap v23: 65 pgs: 65 active+clean; 208 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 1.1 KiB/s rd, 255 B/s wr, 1 op/s 2024-04-27T04:12:14.387 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:12:13 smithi162 ceph-mon[26841]: pgmap v23: 65 pgs: 65 active+clean; 208 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 1.1 KiB/s rd, 255 B/s wr, 1 op/s 2024-04-27T04:12:15.172 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:12:14 smithi181 ceph-mon[33774]: Upgrade: Updating node-exporter.smithi181 (2/2) 2024-04-27T04:12:15.172 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:12:14 smithi181 ceph-mon[33774]: Deploying daemon node-exporter.smithi181 on smithi181 2024-04-27T04:12:15.387 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:12:14 smithi162 ceph-mon[26841]: Upgrade: Updating node-exporter.smithi181 (2/2) 2024-04-27T04:12:15.387 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:12:14 smithi162 ceph-mon[26841]: Deploying daemon node-exporter.smithi181 on smithi181 2024-04-27T04:12:16.387 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:12:15 smithi162 ceph-mon[26841]: pgmap v24: 65 pgs: 65 active+clean; 208 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 1.5 KiB/s rd, 255 B/s wr, 2 op/s 2024-04-27T04:12:16.422 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:12:15 smithi181 ceph-mon[33774]: pgmap v24: 65 pgs: 65 active+clean; 208 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 1.5 KiB/s rd, 255 B/s wr, 2 op/s 2024-04-27T04:12:17.387 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:12:17 smithi162 ceph-mon[26841]: from='mgr.14680 172.21.15.162:0/501880828' entity='mgr.smithi162.oennfn' 2024-04-27T04:12:17.387 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:12:17 smithi162 ceph-mon[26841]: from='mgr.14680 172.21.15.162:0/501880828' entity='mgr.smithi162.oennfn' 2024-04-27T04:12:17.387 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:12:17 smithi162 ceph-mon[26841]: from='mgr.14680 172.21.15.162:0/501880828' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T04:12:17.387 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:12:17 smithi162 ceph-mon[26841]: pgmap v25: 65 pgs: 65 active+clean; 208 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:12:17.422 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:12:17 smithi181 ceph-mon[33774]: from='mgr.14680 172.21.15.162:0/501880828' entity='mgr.smithi162.oennfn' 2024-04-27T04:12:17.422 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:12:17 smithi181 ceph-mon[33774]: from='mgr.14680 172.21.15.162:0/501880828' entity='mgr.smithi162.oennfn' 2024-04-27T04:12:17.422 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:12:17 smithi181 ceph-mon[33774]: from='mgr.14680 172.21.15.162:0/501880828' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T04:12:17.422 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:12:17 smithi181 ceph-mon[33774]: pgmap v25: 65 pgs: 65 active+clean; 208 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:12:18.422 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:12:18 smithi181 ceph-mon[33774]: from='mgr.14680 172.21.15.162:0/501880828' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-27T04:12:18.480 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:12:18 smithi162 ceph-mon[26841]: from='mgr.14680 172.21.15.162:0/501880828' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-27T04:12:19.451 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:12:19 smithi162 ceph-mon[26841]: pgmap v26: 65 pgs: 65 active+clean; 208 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:12:19.672 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:12:19 smithi181 ceph-mon[33774]: pgmap v26: 65 pgs: 65 active+clean; 208 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:12:20.387 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:12:20 smithi162 ceph-mon[26841]: from='mgr.14680 172.21.15.162:0/501880828' entity='mgr.smithi162.oennfn' 2024-04-27T04:12:20.413 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:12:20 smithi162 ceph-mon[26841]: from='mgr.14680 172.21.15.162:0/501880828' entity='mgr.smithi162.oennfn' 2024-04-27T04:12:20.672 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:12:20 smithi181 ceph-mon[33774]: from='mgr.14680 172.21.15.162:0/501880828' entity='mgr.smithi162.oennfn' 2024-04-27T04:12:20.672 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:12:20 smithi181 ceph-mon[33774]: from='mgr.14680 172.21.15.162:0/501880828' entity='mgr.smithi162.oennfn' 2024-04-27T04:12:21.422 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:12:21 smithi181 ceph-mon[33774]: pgmap v27: 65 pgs: 65 active+clean; 208 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:12:21.422 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:12:21 smithi181 ceph-mon[33774]: from='mgr.14680 172.21.15.162:0/501880828' entity='mgr.smithi162.oennfn' 2024-04-27T04:12:21.422 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:12:21 smithi181 ceph-mon[33774]: from='mgr.14680 172.21.15.162:0/501880828' entity='mgr.smithi162.oennfn' 2024-04-27T04:12:21.637 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:12:21 smithi162 ceph-mon[26841]: pgmap v27: 65 pgs: 65 active+clean; 208 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:12:21.637 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:12:21 smithi162 ceph-mon[26841]: from='mgr.14680 172.21.15.162:0/501880828' entity='mgr.smithi162.oennfn' 2024-04-27T04:12:21.637 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:12:21 smithi162 ceph-mon[26841]: from='mgr.14680 172.21.15.162:0/501880828' entity='mgr.smithi162.oennfn' 2024-04-27T04:12:22.922 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:12:22 smithi181 ceph-mon[33774]: from='mgr.14680 172.21.15.162:0/501880828' entity='mgr.smithi162.oennfn' 2024-04-27T04:12:22.922 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:12:22 smithi181 ceph-mon[33774]: from='mgr.14680 172.21.15.162:0/501880828' entity='mgr.smithi162.oennfn' 2024-04-27T04:12:23.137 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:12:22 smithi162 ceph-mon[26841]: from='mgr.14680 172.21.15.162:0/501880828' entity='mgr.smithi162.oennfn' 2024-04-27T04:12:23.137 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:12:22 smithi162 ceph-mon[26841]: from='mgr.14680 172.21.15.162:0/501880828' entity='mgr.smithi162.oennfn' 2024-04-27T04:12:24.137 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:12:23 smithi162 ceph-mon[26841]: pgmap v28: 65 pgs: 65 active+clean; 208 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:12:24.137 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:12:23 smithi162 ceph-mon[26841]: from='mgr.14680 172.21.15.162:0/501880828' entity='mgr.smithi162.oennfn' 2024-04-27T04:12:24.137 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:12:23 smithi162 ceph-mon[26841]: from='mgr.14680 172.21.15.162:0/501880828' entity='mgr.smithi162.oennfn' 2024-04-27T04:12:24.172 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:12:23 smithi181 ceph-mon[33774]: pgmap v28: 65 pgs: 65 active+clean; 208 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:12:24.172 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:12:23 smithi181 ceph-mon[33774]: from='mgr.14680 172.21.15.162:0/501880828' entity='mgr.smithi162.oennfn' 2024-04-27T04:12:24.172 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:12:23 smithi181 ceph-mon[33774]: from='mgr.14680 172.21.15.162:0/501880828' entity='mgr.smithi162.oennfn' 2024-04-27T04:12:24.962 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:12:24 smithi162 ceph-mon[26841]: from='mgr.14680 172.21.15.162:0/501880828' entity='mgr.smithi162.oennfn' 2024-04-27T04:12:24.962 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:12:24 smithi162 ceph-mon[26841]: from='mgr.14680 172.21.15.162:0/501880828' entity='mgr.smithi162.oennfn' 2024-04-27T04:12:25.172 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:12:24 smithi181 ceph-mon[33774]: from='mgr.14680 172.21.15.162:0/501880828' entity='mgr.smithi162.oennfn' 2024-04-27T04:12:25.172 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:12:24 smithi181 ceph-mon[33774]: from='mgr.14680 172.21.15.162:0/501880828' entity='mgr.smithi162.oennfn' 2024-04-27T04:12:25.839 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:12:25.838+0000 7fb0b02cc700 1 -- 172.21.15.162:0/4276827947 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb0a80fbcb0 msgr2=0x7fb0a80fc0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:12:25.839 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:12:25.838+0000 7fb0b02cc700 1 --2- 172.21.15.162:0/4276827947 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb0a80fbcb0 0x7fb0a80fc0c0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7fb0a4004940 tx=0x7fb0a4009e30 comp rx=0 tx=0).stop 2024-04-27T04:12:25.840 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:12:25.838+0000 7fb0b02cc700 1 -- 172.21.15.162:0/4276827947 shutdown_connections 2024-04-27T04:12:25.840 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:12:25.838+0000 7fb0b02cc700 1 --2- 172.21.15.162:0/4276827947 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb0a80fcf60 0x7fb0a80fd3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:12:25.840 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:12:25.838+0000 7fb0b02cc700 1 --2- 172.21.15.162:0/4276827947 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb0a80fbcb0 0x7fb0a80fc0c0 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:12:25.840 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:12:25.838+0000 7fb0b02cc700 1 -- 172.21.15.162:0/4276827947 >> 172.21.15.162:0/4276827947 conn(0x7fb0a80f7430 msgr2=0x7fb0a80f9840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:12:25.840 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:12:25.839+0000 7fb0b02cc700 1 -- 172.21.15.162:0/4276827947 shutdown_connections 2024-04-27T04:12:25.840 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:12:25.839+0000 7fb0b02cc700 1 -- 172.21.15.162:0/4276827947 wait complete. 2024-04-27T04:12:25.840 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:12:25.839+0000 7fb0b02cc700 1 Processor -- start 2024-04-27T04:12:25.840 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:12:25.839+0000 7fb0b02cc700 1 -- start start 2024-04-27T04:12:25.840 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:12:25.839+0000 7fb0b02cc700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb0a80fbcb0 0x7fb0a8110760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:12:25.840 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:12:25.839+0000 7fb0b02cc700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb0a80fcf60 0x7fb0a8112cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:12:25.840 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:12:25.839+0000 7fb0b02cc700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0a8066370 con 0x7fb0a80fcf60 2024-04-27T04:12:25.840 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:12:25.839+0000 7fb0b02cc700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0a80664b0 con 0x7fb0a80fbcb0 2024-04-27T04:12:25.840 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:12:25.839+0000 7fb0ad867700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb0a80fcf60 0x7fb0a8112cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:12:25.841 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:12:25.839+0000 7fb0ad867700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb0a80fcf60 0x7fb0a8112cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:59436/0 (socket says 172.21.15.162:59436) 2024-04-27T04:12:25.841 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:12:25.839+0000 7fb0ad867700 1 -- 172.21.15.162:0/3862020911 learned_addr learned my addr 172.21.15.162:0/3862020911 (peer_addr_for_me v2:172.21.15.162:0/0) 2024-04-27T04:12:25.841 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:12:25.840+0000 7fb0ad867700 1 -- 172.21.15.162:0/3862020911 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb0a80fbcb0 msgr2=0x7fb0a8110760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:12:25.841 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:12:25.840+0000 7fb0ad867700 1 --2- 172.21.15.162:0/3862020911 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb0a80fbcb0 0x7fb0a8110760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:12:25.841 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:12:25.840+0000 7fb0ad867700 1 -- 172.21.15.162:0/3862020911 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0a4005040 con 0x7fb0a80fcf60 2024-04-27T04:12:25.841 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:12:25.840+0000 7fb0ad867700 1 --2- 172.21.15.162:0/3862020911 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb0a80fcf60 0x7fb0a8112cb0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7fb09800b790 tx=0x7fb098009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:12:25.841 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:12:25.840+0000 7fb09effd700 1 -- 172.21.15.162:0/3862020911 <== mon.0 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb09800f770 con 0x7fb0a80fcf60 2024-04-27T04:12:25.841 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:12:25.840+0000 7fb0b02cc700 1 -- 172.21.15.162:0/3862020911 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb0a81132b0 con 0x7fb0a80fcf60 2024-04-27T04:12:25.841 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:12:25.840+0000 7fb09effd700 1 -- 172.21.15.162:0/3862020911 <== mon.0 v2:172.21.15.162:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fb09800fda0 con 0x7fb0a80fcf60 2024-04-27T04:12:25.841 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:12:25.840+0000 7fb09effd700 1 -- 172.21.15.162:0/3862020911 <== mon.0 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb09800d310 con 0x7fb0a80fcf60 2024-04-27T04:12:25.841 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:12:25.840+0000 7fb0b02cc700 1 -- 172.21.15.162:0/3862020911 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb0a8113870 con 0x7fb0a80fcf60 2024-04-27T04:12:25.842 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:12:25.841+0000 7fb0b02cc700 1 -- 172.21.15.162:0/3862020911 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb0a805f5c0 con 0x7fb0a80fcf60 2024-04-27T04:12:25.842 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:12:25.842+0000 7fb09effd700 1 -- 172.21.15.162:0/3862020911 <== mon.0 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 31) v1 ==== 92571+0+0 (secure 0 0 0) 0x7fb098010420 con 0x7fb0a80fcf60 2024-04-27T04:12:25.842 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:12:25.842+0000 7fb09effd700 1 --2- 172.21.15.162:0/3862020911 >> [v2:172.21.15.162:6800/901992993,v1:172.21.15.162:6801/901992993] conn(0x7fb09406e620 0x7fb094070ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:12:25.843 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:12:25.842+0000 7fb09effd700 1 -- 172.21.15.162:0/3862020911 <== mon.0 v2:172.21.15.162:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6152+0+0 (secure 0 0 0) 0x7fb09808e0e0 con 0x7fb0a80fcf60 2024-04-27T04:12:25.843 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:12:25.842+0000 7fb0ae068700 1 --2- 172.21.15.162:0/3862020911 >> [v2:172.21.15.162:6800/901992993,v1:172.21.15.162:6801/901992993] conn(0x7fb09406e620 0x7fb094070ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:12:25.843 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:12:25.843+0000 7fb0ae068700 1 --2- 172.21.15.162:0/3862020911 >> [v2:172.21.15.162:6800/901992993,v1:172.21.15.162:6801/901992993] conn(0x7fb09406e620 0x7fb094070ad0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fb0a4004f90 tx=0x7fb0a4009e30 comp rx=0 tx=0).ready entity=mgr.14680 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:12:25.846 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:12:25.847+0000 7fb09effd700 1 -- 172.21.15.162:0/3862020911 <== mon.0 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186652 (secure 0 0 0) 0x7fb098057650 con 0x7fb0a80fcf60 2024-04-27T04:12:25.997 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:12:25 smithi162 ceph-mon[26841]: pgmap v29: 65 pgs: 65 active+clean; 208 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:12:26.077 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:12:26.077+0000 7fb0b02cc700 1 -- 172.21.15.162:0/3862020911 --> [v2:172.21.15.162:6800/901992993,v1:172.21.15.162:6801/901992993] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fb0a8101250 con 0x7fb09406e620 2024-04-27T04:12:26.078 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:12:26.079+0000 7fb09effd700 1 -- 172.21.15.162:0/3862020911 <== mgr.14680 v2:172.21.15.162:6800/901992993 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+370 (secure 0 0 0) 0x7fb0a8101250 con 0x7fb09406e620 2024-04-27T04:12:26.080 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:12:26.081+0000 7fb0b02cc700 1 -- 172.21.15.162:0/3862020911 >> [v2:172.21.15.162:6800/901992993,v1:172.21.15.162:6801/901992993] conn(0x7fb09406e620 msgr2=0x7fb094070ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:12:26.080 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:12:26.081+0000 7fb0b02cc700 1 --2- 172.21.15.162:0/3862020911 >> [v2:172.21.15.162:6800/901992993,v1:172.21.15.162:6801/901992993] conn(0x7fb09406e620 0x7fb094070ad0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fb0a4004f90 tx=0x7fb0a4009e30 comp rx=0 tx=0).stop 2024-04-27T04:12:26.081 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:12:26.081+0000 7fb0b02cc700 1 -- 172.21.15.162:0/3862020911 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb0a80fcf60 msgr2=0x7fb0a8112cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:12:26.081 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:12:26.081+0000 7fb0b02cc700 1 --2- 172.21.15.162:0/3862020911 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb0a80fcf60 0x7fb0a8112cb0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7fb09800b790 tx=0x7fb098009250 comp rx=0 tx=0).stop 2024-04-27T04:12:26.081 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:12:26.081+0000 7fb0b02cc700 1 -- 172.21.15.162:0/3862020911 shutdown_connections 2024-04-27T04:12:26.081 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:12:26.081+0000 7fb0b02cc700 1 --2- 172.21.15.162:0/3862020911 >> [v2:172.21.15.162:6800/901992993,v1:172.21.15.162:6801/901992993] conn(0x7fb09406e620 0x7fb094070ad0 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:12:26.081 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:12:26.081+0000 7fb0b02cc700 1 --2- 172.21.15.162:0/3862020911 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb0a80fbcb0 0x7fb0a8110760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:12:26.081 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:12:26.081+0000 7fb0b02cc700 1 --2- 172.21.15.162:0/3862020911 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb0a80fcf60 0x7fb0a8112cb0 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:12:26.081 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:12:26.081+0000 7fb0b02cc700 1 -- 172.21.15.162:0/3862020911 >> 172.21.15.162:0/3862020911 conn(0x7fb0a80f7430 msgr2=0x7fb0a80f8a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:12:26.081 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:12:26.081+0000 7fb0b02cc700 1 -- 172.21.15.162:0/3862020911 shutdown_connections 2024-04-27T04:12:26.081 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:12:26.081+0000 7fb0b02cc700 1 -- 172.21.15.162:0/3862020911 wait complete. 2024-04-27T04:12:26.091 INFO:teuthology.orchestra.run.smithi162.stdout:true 2024-04-27T04:12:26.172 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:12:25 smithi181 ceph-mon[33774]: pgmap v29: 65 pgs: 65 active+clean; 208 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:12:26.189 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:12:26.188+0000 7fa8babdd700 1 -- 172.21.15.162:0/4119656389 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa8b40fce60 msgr2=0x7fa8b40fd270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:12:26.189 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:12:26.188+0000 7fa8babdd700 1 --2- 172.21.15.162:0/4119656389 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa8b40fce60 0x7fa8b40fd270 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7fa8a8004940 tx=0x7fa8a8009e30 comp rx=0 tx=0).stop 2024-04-27T04:12:26.189 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:12:26.188+0000 7fa8babdd700 1 -- 172.21.15.162:0/4119656389 shutdown_connections 2024-04-27T04:12:26.189 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:12:26.188+0000 7fa8babdd700 1 --2- 172.21.15.162:0/4119656389 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa8b40fdfd0 0x7fa8b40fe440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:12:26.189 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:12:26.188+0000 7fa8babdd700 1 --2- 172.21.15.162:0/4119656389 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa8b40fce60 0x7fa8b40fd270 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:12:26.189 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:12:26.188+0000 7fa8babdd700 1 -- 172.21.15.162:0/4119656389 >> 172.21.15.162:0/4119656389 conn(0x7fa8b40f8540 msgr2=0x7fa8b40fa950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:12:26.190 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:12:26.188+0000 7fa8babdd700 1 -- 172.21.15.162:0/4119656389 shutdown_connections 2024-04-27T04:12:26.190 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:12:26.188+0000 7fa8babdd700 1 -- 172.21.15.162:0/4119656389 wait complete. 2024-04-27T04:12:26.190 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:12:26.189+0000 7fa8babdd700 1 Processor -- start 2024-04-27T04:12:26.190 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:12:26.189+0000 7fa8babdd700 1 -- start start 2024-04-27T04:12:26.190 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:12:26.189+0000 7fa8babdd700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa8b40fce60 0x7fa8b4071de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:12:26.190 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:12:26.189+0000 7fa8babdd700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa8b40fdfd0 0x7fa8b40702b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:12:26.190 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:12:26.189+0000 7fa8babdd700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8b4070860 con 0x7fa8b40fdfd0 2024-04-27T04:12:26.190 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:12:26.189+0000 7fa8babdd700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8b40709d0 con 0x7fa8b40fce60 2024-04-27T04:12:26.190 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:12:26.189+0000 7fa8b3fff700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa8b40fdfd0 0x7fa8b40702b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:12:26.190 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:12:26.189+0000 7fa8b3fff700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa8b40fdfd0 0x7fa8b40702b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:59462/0 (socket says 172.21.15.162:59462) 2024-04-27T04:12:26.190 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:12:26.189+0000 7fa8b3fff700 1 -- 172.21.15.162:0/2262346084 learned_addr learned my addr 172.21.15.162:0/2262346084 (peer_addr_for_me v2:172.21.15.162:0/0) 2024-04-27T04:12:26.190 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:12:26.189+0000 7fa8b8979700 1 --2- 172.21.15.162:0/2262346084 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa8b40fce60 0x7fa8b4071de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:12:26.190 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:12:26.189+0000 7fa8b3fff700 1 -- 172.21.15.162:0/2262346084 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa8b40fce60 msgr2=0x7fa8b4071de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:12:26.190 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:12:26.189+0000 7fa8b3fff700 1 --2- 172.21.15.162:0/2262346084 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa8b40fce60 0x7fa8b4071de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:12:26.191 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:12:26.189+0000 7fa8b3fff700 1 -- 172.21.15.162:0/2262346084 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa8a8005040 con 0x7fa8b40fdfd0 2024-04-27T04:12:26.191 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:12:26.190+0000 7fa8b3fff700 1 --2- 172.21.15.162:0/2262346084 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa8b40fdfd0 0x7fa8b40702b0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7fa8a0002a50 tx=0x7fa8a00046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:12:26.191 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:12:26.190+0000 7fa8b17fa700 1 -- 172.21.15.162:0/2262346084 <== mon.0 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa8a0007cc0 con 0x7fa8b40fdfd0 2024-04-27T04:12:26.191 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:12:26.190+0000 7fa8b17fa700 1 -- 172.21.15.162:0/2262346084 <== mon.0 v2:172.21.15.162:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fa8a000f040 con 0x7fa8b40fdfd0 2024-04-27T04:12:26.191 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:12:26.190+0000 7fa8babdd700 1 -- 172.21.15.162:0/2262346084 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa8b4070cb0 con 0x7fa8b40fdfd0 2024-04-27T04:12:26.191 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:12:26.190+0000 7fa8b17fa700 1 -- 172.21.15.162:0/2262346084 <== mon.0 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa8a000da30 con 0x7fa8b40fdfd0 2024-04-27T04:12:26.191 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:12:26.190+0000 7fa8babdd700 1 -- 172.21.15.162:0/2262346084 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa8b4115a30 con 0x7fa8b40fdfd0 2024-04-27T04:12:26.192 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:12:26.191+0000 7fa8babdd700 1 -- 172.21.15.162:0/2262346084 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa8b405f5c0 con 0x7fa8b40fdfd0 2024-04-27T04:12:26.192 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:12:26.191+0000 7fa8b17fa700 1 -- 172.21.15.162:0/2262346084 <== mon.0 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 31) v1 ==== 92571+0+0 (secure 0 0 0) 0x7fa8a000db90 con 0x7fa8b40fdfd0 2024-04-27T04:12:26.192 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:12:26.192+0000 7fa8b17fa700 1 --2- 172.21.15.162:0/2262346084 >> [v2:172.21.15.162:6800/901992993,v1:172.21.15.162:6801/901992993] conn(0x7fa8a406e620 0x7fa8a4070ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:12:26.192 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:12:26.192+0000 7fa8b17fa700 1 -- 172.21.15.162:0/2262346084 <== mon.0 v2:172.21.15.162:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6152+0+0 (secure 0 0 0) 0x7fa8a008df50 con 0x7fa8b40fdfd0 2024-04-27T04:12:26.192 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:12:26.192+0000 7fa8b8979700 1 --2- 172.21.15.162:0/2262346084 >> [v2:172.21.15.162:6800/901992993,v1:172.21.15.162:6801/901992993] conn(0x7fa8a406e620 0x7fa8a4070ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:12:26.192 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:12:26.192+0000 7fa8b8979700 1 --2- 172.21.15.162:0/2262346084 >> [v2:172.21.15.162:6800/901992993,v1:172.21.15.162:6801/901992993] conn(0x7fa8a406e620 0x7fa8a4070ad0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fa8a8004890 tx=0x7fa8a8009e30 comp rx=0 tx=0).ready entity=mgr.14680 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:12:26.196 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:12:26.197+0000 7fa8b17fa700 1 -- 172.21.15.162:0/2262346084 <== mon.0 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186652 (secure 0 0 0) 0x7fa8a00c2930 con 0x7fa8b40fdfd0 2024-04-27T04:12:26.424 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:12:26.424+0000 7fa8babdd700 1 -- 172.21.15.162:0/2262346084 --> [v2:172.21.15.162:6800/901992993,v1:172.21.15.162:6801/901992993] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fa8b41022f0 con 0x7fa8a406e620 2024-04-27T04:12:26.424 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:12:26.425+0000 7fa8b17fa700 1 -- 172.21.15.162:0/2262346084 <== mgr.14680 v2:172.21.15.162:6800/901992993 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+370 (secure 0 0 0) 0x7fa8b41022f0 con 0x7fa8a406e620 2024-04-27T04:12:26.430 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:12:26.427+0000 7fa8babdd700 1 -- 172.21.15.162:0/2262346084 >> [v2:172.21.15.162:6800/901992993,v1:172.21.15.162:6801/901992993] conn(0x7fa8a406e620 msgr2=0x7fa8a4070ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:12:26.430 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:12:26.427+0000 7fa8babdd700 1 --2- 172.21.15.162:0/2262346084 >> [v2:172.21.15.162:6800/901992993,v1:172.21.15.162:6801/901992993] conn(0x7fa8a406e620 0x7fa8a4070ad0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fa8a8004890 tx=0x7fa8a8009e30 comp rx=0 tx=0).stop 2024-04-27T04:12:26.430 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:12:26.427+0000 7fa8babdd700 1 -- 172.21.15.162:0/2262346084 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa8b40fdfd0 msgr2=0x7fa8b40702b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:12:26.430 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:12:26.427+0000 7fa8babdd700 1 --2- 172.21.15.162:0/2262346084 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa8b40fdfd0 0x7fa8b40702b0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7fa8a0002a50 tx=0x7fa8a00046b0 comp rx=0 tx=0).stop 2024-04-27T04:12:26.430 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:12:26.428+0000 7fa8babdd700 1 -- 172.21.15.162:0/2262346084 shutdown_connections 2024-04-27T04:12:26.430 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:12:26.428+0000 7fa8babdd700 1 --2- 172.21.15.162:0/2262346084 >> [v2:172.21.15.162:6800/901992993,v1:172.21.15.162:6801/901992993] conn(0x7fa8a406e620 0x7fa8a4070ad0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:12:26.430 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:12:26.428+0000 7fa8babdd700 1 --2- 172.21.15.162:0/2262346084 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa8b40fce60 0x7fa8b4071de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:12:26.430 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:12:26.428+0000 7fa8babdd700 1 --2- 172.21.15.162:0/2262346084 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa8b40fdfd0 0x7fa8b40702b0 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:12:26.430 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:12:26.428+0000 7fa8babdd700 1 -- 172.21.15.162:0/2262346084 >> 172.21.15.162:0/2262346084 conn(0x7fa8b40f8540 msgr2=0x7fa8b40f9c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:12:26.430 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:12:26.428+0000 7fa8babdd700 1 -- 172.21.15.162:0/2262346084 shutdown_connections 2024-04-27T04:12:26.430 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:12:26.428+0000 7fa8babdd700 1 -- 172.21.15.162:0/2262346084 wait complete. 2024-04-27T04:12:26.539 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:12:26.538+0000 7fb64c3c7700 1 -- 172.21.15.162:0/1041780060 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb6440fdfd0 msgr2=0x7fb6440fe440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:12:26.539 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:12:26.538+0000 7fb64c3c7700 1 --2- 172.21.15.162:0/1041780060 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb6440fdfd0 0x7fb6440fe440 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7fb634004940 tx=0x7fb634009e30 comp rx=0 tx=0).stop 2024-04-27T04:12:26.539 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:12:26.538+0000 7fb64c3c7700 1 -- 172.21.15.162:0/1041780060 shutdown_connections 2024-04-27T04:12:26.540 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:12:26.538+0000 7fb64c3c7700 1 --2- 172.21.15.162:0/1041780060 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb6440fdfd0 0x7fb6440fe440 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:12:26.540 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:12:26.538+0000 7fb64c3c7700 1 --2- 172.21.15.162:0/1041780060 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb6440fce60 0x7fb6440fd270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:12:26.540 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:12:26.538+0000 7fb64c3c7700 1 -- 172.21.15.162:0/1041780060 >> 172.21.15.162:0/1041780060 conn(0x7fb6440f8540 msgr2=0x7fb6440fa950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:12:26.540 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:12:26.539+0000 7fb64c3c7700 1 -- 172.21.15.162:0/1041780060 shutdown_connections 2024-04-27T04:12:26.540 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:12:26.539+0000 7fb64c3c7700 1 -- 172.21.15.162:0/1041780060 wait complete. 2024-04-27T04:12:26.540 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:12:26.539+0000 7fb64c3c7700 1 Processor -- start 2024-04-27T04:12:26.540 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:12:26.539+0000 7fb64c3c7700 1 -- start start 2024-04-27T04:12:26.540 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:12:26.539+0000 7fb64c3c7700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb6440fce60 0x7fb64410d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:12:26.540 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:12:26.539+0000 7fb64c3c7700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb6440fdfd0 0x7fb64410df00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:12:26.541 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:12:26.539+0000 7fb64c3c7700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb64410e500 con 0x7fb6440fce60 2024-04-27T04:12:26.541 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:12:26.539+0000 7fb64c3c7700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb64410ed00 con 0x7fb6440fdfd0 2024-04-27T04:12:26.541 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:12:26.539+0000 7fb64a163700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb6440fce60 0x7fb64410d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:12:26.541 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:12:26.540+0000 7fb64a163700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb6440fce60 0x7fb64410d9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:59480/0 (socket says 172.21.15.162:59480) 2024-04-27T04:12:26.541 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:12:26.540+0000 7fb64a163700 1 -- 172.21.15.162:0/1199544026 learned_addr learned my addr 172.21.15.162:0/1199544026 (peer_addr_for_me v2:172.21.15.162:0/0) 2024-04-27T04:12:26.541 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:12:26.540+0000 7fb649962700 1 --2- 172.21.15.162:0/1199544026 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb6440fdfd0 0x7fb64410df00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:12:26.541 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:12:26.540+0000 7fb64a163700 1 -- 172.21.15.162:0/1199544026 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb6440fdfd0 msgr2=0x7fb64410df00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:12:26.541 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:12:26.540+0000 7fb64a163700 1 --2- 172.21.15.162:0/1199544026 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb6440fdfd0 0x7fb64410df00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:12:26.541 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:12:26.540+0000 7fb64a163700 1 -- 172.21.15.162:0/1199544026 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb634005040 con 0x7fb6440fce60 2024-04-27T04:12:26.541 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:12:26.540+0000 7fb64a163700 1 --2- 172.21.15.162:0/1199544026 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb6440fce60 0x7fb64410d9c0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7fb640002a80 tx=0x7fb6400046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:12:26.542 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:12:26.540+0000 7fb63affd700 1 -- 172.21.15.162:0/1199544026 <== mon.0 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb640007cc0 con 0x7fb6440fce60 2024-04-27T04:12:26.542 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:12:26.540+0000 7fb63affd700 1 -- 172.21.15.162:0/1199544026 <== mon.0 v2:172.21.15.162:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fb64000f040 con 0x7fb6440fce60 2024-04-27T04:12:26.542 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:12:26.540+0000 7fb64c3c7700 1 -- 172.21.15.162:0/1199544026 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb64410efe0 con 0x7fb6440fce60 2024-04-27T04:12:26.543 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:12:26.540+0000 7fb63affd700 1 -- 172.21.15.162:0/1199544026 <== mon.0 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb6400096f0 con 0x7fb6440fce60 2024-04-27T04:12:26.543 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:12:26.540+0000 7fb64c3c7700 1 -- 172.21.15.162:0/1199544026 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb64410f520 con 0x7fb6440fce60 2024-04-27T04:12:26.543 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:12:26.541+0000 7fb64c3c7700 1 -- 172.21.15.162:0/1199544026 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb64405f5c0 con 0x7fb6440fce60 2024-04-27T04:12:26.543 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:12:26.542+0000 7fb63affd700 1 -- 172.21.15.162:0/1199544026 <== mon.0 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 31) v1 ==== 92571+0+0 (secure 0 0 0) 0x7fb64000d690 con 0x7fb6440fce60 2024-04-27T04:12:26.543 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:12:26.542+0000 7fb63affd700 1 --2- 172.21.15.162:0/1199544026 >> [v2:172.21.15.162:6800/901992993,v1:172.21.15.162:6801/901992993] conn(0x7fb63006e620 0x7fb630070ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:12:26.543 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:12:26.542+0000 7fb63affd700 1 -- 172.21.15.162:0/1199544026 <== mon.0 v2:172.21.15.162:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6152+0+0 (secure 0 0 0) 0x7fb64008d8c0 con 0x7fb6440fce60 2024-04-27T04:12:26.543 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:12:26.542+0000 7fb649962700 1 --2- 172.21.15.162:0/1199544026 >> [v2:172.21.15.162:6800/901992993,v1:172.21.15.162:6801/901992993] conn(0x7fb63006e620 0x7fb630070ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:12:26.543 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:12:26.543+0000 7fb649962700 1 --2- 172.21.15.162:0/1199544026 >> [v2:172.21.15.162:6800/901992993,v1:172.21.15.162:6801/901992993] conn(0x7fb63006e620 0x7fb630070ad0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fb63400b3b0 tx=0x7fb634009e30 comp rx=0 tx=0).ready entity=mgr.14680 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:12:26.549 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:12:26.550+0000 7fb63affd700 1 -- 172.21.15.162:0/1199544026 <== mon.0 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186652 (secure 0 0 0) 0x7fb64005a850 con 0x7fb6440fce60 2024-04-27T04:12:26.770 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:12:26.770+0000 7fb64c3c7700 1 -- 172.21.15.162:0/1199544026 --> [v2:172.21.15.162:6800/901992993,v1:172.21.15.162:6801/901992993] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7fb64410f8f0 con 0x7fb63006e620 2024-04-27T04:12:26.781 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:12:26.781+0000 7fb63affd700 1 -- 172.21.15.162:0/1199544026 <== mgr.14680 v2:172.21.15.162:6800/901992993 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3520 (secure 0 0 0) 0x7fb64410f8f0 con 0x7fb63006e620 2024-04-27T04:12:26.781 INFO:teuthology.orchestra.run.smithi162.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-27T04:12:26.781 INFO:teuthology.orchestra.run.smithi162.stdout:alertmanager.smithi162 smithi162 *:9093,9094 running (104s) 5s ago 10m 16.6M - 0.25.0 c8568f914cd2 36702db3ad50 2024-04-27T04:12:26.781 INFO:teuthology.orchestra.run.smithi162.stdout:crash.smithi162 smithi162 running (10m) 5s ago 10m 6660k - 17.2.7-907-g2be3efa9 1c5834633715 228c98df4289 2024-04-27T04:12:26.781 INFO:teuthology.orchestra.run.smithi162.stdout:crash.smithi181 smithi181 running (10m) 7s ago 10m 6665k - 17.2.7-907-g2be3efa9 1c5834633715 c8e553e3df2b 2024-04-27T04:12:26.782 INFO:teuthology.orchestra.run.smithi162.stdout:grafana.smithi162 smithi162 *:3000 running (97s) 5s ago 10m 79.6M - 9.4.7 954c08fa6188 746f08b3fb5b 2024-04-27T04:12:26.782 INFO:teuthology.orchestra.run.smithi162.stdout:mds.cephfs.smithi162.gzlyjj smithi162 running (5m) 5s ago 5m 12.7M - 17.2.7-907-g2be3efa9 1c5834633715 27e5b030dd32 2024-04-27T04:12:26.782 INFO:teuthology.orchestra.run.smithi162.stdout:mds.cephfs.smithi162.xcnapk smithi162 running (5m) 5s ago 5m 100M - 17.2.7-907-g2be3efa9 1c5834633715 38f9030df029 2024-04-27T04:12:26.782 INFO:teuthology.orchestra.run.smithi162.stdout:mds.cephfs.smithi181.bkkcdm smithi181 running (5m) 7s ago 5m 14.3M - 17.2.7-907-g2be3efa9 1c5834633715 af55afca1263 2024-04-27T04:12:26.782 INFO:teuthology.orchestra.run.smithi162.stdout:mds.cephfs.smithi181.dzomgh smithi181 running (5m) 7s ago 5m 145M - 17.2.7-907-g2be3efa9 1c5834633715 6733296597f3 2024-04-27T04:12:26.782 INFO:teuthology.orchestra.run.smithi162.stdout:mgr.smithi162.oennfn smithi162 *:8443,9283,8765 running (68s) 5s ago 12m 496M - 19.0.0-3253-g4e213a58 67e414ff4bc4 58a748d3637b 2024-04-27T04:12:26.782 INFO:teuthology.orchestra.run.smithi162.stdout:mgr.smithi181.vtwuhz smithi181 *:8443,9283,8765 running (28s) 7s ago 9m 451M - 19.0.0-3253-g4e213a58 67e414ff4bc4 813584297b1d 2024-04-27T04:12:26.782 INFO:teuthology.orchestra.run.smithi162.stdout:mon.smithi162 smithi162 running (12m) 5s ago 12m 57.0M 2048M 17.2.7-907-g2be3efa9 1c5834633715 0f1e3682b9ab 2024-04-27T04:12:26.782 INFO:teuthology.orchestra.run.smithi162.stdout:mon.smithi181 smithi181 running (9m) 7s ago 9m 42.7M 2048M 17.2.7-907-g2be3efa9 1c5834633715 96e48aa3912f 2024-04-27T04:12:26.782 INFO:teuthology.orchestra.run.smithi162.stdout:node-exporter.smithi162 smithi162 *:9100 running (15s) 5s ago 10m 3657k - 1.5.0 0da6a335fe13 5c4c5ea9c1d7 2024-04-27T04:12:26.782 INFO:teuthology.orchestra.run.smithi162.stdout:node-exporter.smithi181 smithi181 *:9100 running (11s) 7s ago 9m 3670k - 1.5.0 0da6a335fe13 9067f14c0519 2024-04-27T04:12:26.782 INFO:teuthology.orchestra.run.smithi162.stdout:osd.0 smithi162 running (9m) 5s ago 9m 231M 1557M 17.2.7-907-g2be3efa9 1c5834633715 4a9df3aa938d 2024-04-27T04:12:26.782 INFO:teuthology.orchestra.run.smithi162.stdout:osd.1 smithi162 running (8m) 5s ago 8m 243M 1557M 17.2.7-907-g2be3efa9 1c5834633715 aefb464c46e8 2024-04-27T04:12:26.782 INFO:teuthology.orchestra.run.smithi162.stdout:osd.2 smithi162 running (8m) 5s ago 8m 202M 1557M 17.2.7-907-g2be3efa9 1c5834633715 70f16d86d38e 2024-04-27T04:12:26.783 INFO:teuthology.orchestra.run.smithi162.stdout:osd.3 smithi181 running (7m) 7s ago 7m 276M 2581M 17.2.7-907-g2be3efa9 1c5834633715 030d829be54e 2024-04-27T04:12:26.783 INFO:teuthology.orchestra.run.smithi162.stdout:osd.4 smithi181 running (7m) 7s ago 7m 244M 2581M 17.2.7-907-g2be3efa9 1c5834633715 512afbee7cf7 2024-04-27T04:12:26.783 INFO:teuthology.orchestra.run.smithi162.stdout:osd.5 smithi181 running (6m) 7s ago 6m 221M 2581M 17.2.7-907-g2be3efa9 1c5834633715 f47a1daac1ea 2024-04-27T04:12:26.783 INFO:teuthology.orchestra.run.smithi162.stdout:prometheus.smithi162 smithi162 *:9095 running (33s) 5s ago 10m 48.1M - 2.43.0 a07b618ecd1d 8c0bb82ad078 2024-04-27T04:12:26.784 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:12:26.784+0000 7fb64c3c7700 1 -- 172.21.15.162:0/1199544026 >> [v2:172.21.15.162:6800/901992993,v1:172.21.15.162:6801/901992993] conn(0x7fb63006e620 msgr2=0x7fb630070ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:12:26.784 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:12:26.784+0000 7fb64c3c7700 1 --2- 172.21.15.162:0/1199544026 >> [v2:172.21.15.162:6800/901992993,v1:172.21.15.162:6801/901992993] conn(0x7fb63006e620 0x7fb630070ad0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fb63400b3b0 tx=0x7fb634009e30 comp rx=0 tx=0).stop 2024-04-27T04:12:26.784 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:12:26.784+0000 7fb64c3c7700 1 -- 172.21.15.162:0/1199544026 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb6440fce60 msgr2=0x7fb64410d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:12:26.784 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:12:26.784+0000 7fb64c3c7700 1 --2- 172.21.15.162:0/1199544026 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb6440fce60 0x7fb64410d9c0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7fb640002a80 tx=0x7fb6400046b0 comp rx=0 tx=0).stop 2024-04-27T04:12:26.784 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:12:26.784+0000 7fb64c3c7700 1 -- 172.21.15.162:0/1199544026 shutdown_connections 2024-04-27T04:12:26.784 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:12:26.784+0000 7fb64c3c7700 1 --2- 172.21.15.162:0/1199544026 >> [v2:172.21.15.162:6800/901992993,v1:172.21.15.162:6801/901992993] conn(0x7fb63006e620 0x7fb630070ad0 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:12:26.784 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:12:26.784+0000 7fb64c3c7700 1 --2- 172.21.15.162:0/1199544026 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb6440fce60 0x7fb64410d9c0 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:12:26.784 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:12:26.784+0000 7fb64c3c7700 1 --2- 172.21.15.162:0/1199544026 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb6440fdfd0 0x7fb64410df00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:12:26.784 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:12:26.784+0000 7fb64c3c7700 1 -- 172.21.15.162:0/1199544026 >> 172.21.15.162:0/1199544026 conn(0x7fb6440f8540 msgr2=0x7fb6440f9b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:12:26.784 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:12:26.784+0000 7fb64c3c7700 1 -- 172.21.15.162:0/1199544026 shutdown_connections 2024-04-27T04:12:26.784 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:12:26.784+0000 7fb64c3c7700 1 -- 172.21.15.162:0/1199544026 wait complete. 2024-04-27T04:12:26.889 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:12:26.890+0000 7f032e7f9700 1 -- 172.21.15.162:0/1996746885 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f03280fbcb0 msgr2=0x7f03280fc0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:12:26.889 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:12:26.890+0000 7f032e7f9700 1 --2- 172.21.15.162:0/1996746885 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f03280fbcb0 0x7f03280fc0c0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f0318004940 tx=0x7f0318009e30 comp rx=0 tx=0).stop 2024-04-27T04:12:26.890 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:12:26.890+0000 7f032e7f9700 1 -- 172.21.15.162:0/1996746885 shutdown_connections 2024-04-27T04:12:26.890 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:12:26.890+0000 7f032e7f9700 1 --2- 172.21.15.162:0/1996746885 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f03280fcf60 0x7f03280fd3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:12:26.890 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:12:26.890+0000 7f032e7f9700 1 --2- 172.21.15.162:0/1996746885 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f03280fbcb0 0x7f03280fc0c0 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:12:26.890 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:12:26.890+0000 7f032e7f9700 1 -- 172.21.15.162:0/1996746885 >> 172.21.15.162:0/1996746885 conn(0x7f03280f7430 msgr2=0x7f03280f9840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:12:26.891 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:12:26.891+0000 7f032e7f9700 1 -- 172.21.15.162:0/1996746885 shutdown_connections 2024-04-27T04:12:26.891 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:12:26.891+0000 7f032e7f9700 1 -- 172.21.15.162:0/1996746885 wait complete. 2024-04-27T04:12:26.891 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:12:26.891+0000 7f032e7f9700 1 Processor -- start 2024-04-27T04:12:26.893 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:12:26.891+0000 7f032e7f9700 1 -- start start 2024-04-27T04:12:26.893 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:12:26.892+0000 7f032e7f9700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f03280fbcb0 0x7f032810c900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:12:26.893 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:12:26.892+0000 7f032e7f9700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f03280fcf60 0x7f032810ce40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:12:26.893 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:12:26.892+0000 7f032e7f9700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f032810d440 con 0x7f03280fcf60 2024-04-27T04:12:26.893 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:12:26.892+0000 7f032e7f9700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f032810dc90 con 0x7f03280fbcb0 2024-04-27T04:12:26.893 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:12:26.892+0000 7f03277fe700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f03280fcf60 0x7f032810ce40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:12:26.893 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:12:26.892+0000 7f03277fe700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f03280fcf60 0x7f032810ce40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:59504/0 (socket says 172.21.15.162:59504) 2024-04-27T04:12:26.893 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:12:26.892+0000 7f03277fe700 1 -- 172.21.15.162:0/248115955 learned_addr learned my addr 172.21.15.162:0/248115955 (peer_addr_for_me v2:172.21.15.162:0/0) 2024-04-27T04:12:26.893 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:12:26.892+0000 7f0327fff700 1 --2- 172.21.15.162:0/248115955 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f03280fbcb0 0x7f032810c900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:12:26.893 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:12:26.892+0000 7f03277fe700 1 -- 172.21.15.162:0/248115955 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f03280fbcb0 msgr2=0x7f032810c900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:12:26.893 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:12:26.892+0000 7f03277fe700 1 --2- 172.21.15.162:0/248115955 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f03280fbcb0 0x7f032810c900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:12:26.893 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:12:26.892+0000 7f03277fe700 1 -- 172.21.15.162:0/248115955 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0318005040 con 0x7f03280fcf60 2024-04-27T04:12:26.893 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:12:26.892+0000 7f0327fff700 1 --2- 172.21.15.162:0/248115955 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f03280fbcb0 0x7f032810c900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-27T04:12:26.894 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:12:26.892+0000 7f03277fe700 1 --2- 172.21.15.162:0/248115955 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f03280fcf60 0x7f032810ce40 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f031c00c8c0 tx=0x7f031c0044c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:12:26.894 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:12:26.893+0000 7f0324ff9700 1 -- 172.21.15.162:0/248115955 <== mon.0 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f031c00a980 con 0x7f03280fcf60 2024-04-27T04:12:26.894 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:12:26.893+0000 7f0324ff9700 1 -- 172.21.15.162:0/248115955 <== mon.0 v2:172.21.15.162:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f031c00ce60 con 0x7f03280fcf60 2024-04-27T04:12:26.894 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:12:26.893+0000 7f032e7f9700 1 -- 172.21.15.162:0/248115955 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f032810df70 con 0x7f03280fcf60 2024-04-27T04:12:26.894 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:12:26.893+0000 7f032e7f9700 1 -- 172.21.15.162:0/248115955 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f032810e4b0 con 0x7f03280fcf60 2024-04-27T04:12:26.894 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:12:26.893+0000 7f0324ff9700 1 -- 172.21.15.162:0/248115955 <== mon.0 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f031c0115e0 con 0x7f03280fcf60 2024-04-27T04:12:26.895 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:12:26.894+0000 7f032e7f9700 1 -- 172.21.15.162:0/248115955 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f032805f5c0 con 0x7f03280fcf60 2024-04-27T04:12:26.895 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:12:26.894+0000 7f0324ff9700 1 -- 172.21.15.162:0/248115955 <== mon.0 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 31) v1 ==== 92571+0+0 (secure 0 0 0) 0x7f031c012400 con 0x7f03280fcf60 2024-04-27T04:12:26.895 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:12:26.895+0000 7f0324ff9700 1 --2- 172.21.15.162:0/248115955 >> [v2:172.21.15.162:6800/901992993,v1:172.21.15.162:6801/901992993] conn(0x7f031006e620 0x7f0310070ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:12:26.895 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:12:26.895+0000 7f0324ff9700 1 -- 172.21.15.162:0/248115955 <== mon.0 v2:172.21.15.162:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6152+0+0 (secure 0 0 0) 0x7f031c08e190 con 0x7f03280fcf60 2024-04-27T04:12:26.895 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:12:26.895+0000 7f0327fff700 1 --2- 172.21.15.162:0/248115955 >> [v2:172.21.15.162:6800/901992993,v1:172.21.15.162:6801/901992993] conn(0x7f031006e620 0x7f0310070ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:12:26.895 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:12:26.895+0000 7f0327fff700 1 --2- 172.21.15.162:0/248115955 >> [v2:172.21.15.162:6800/901992993,v1:172.21.15.162:6801/901992993] conn(0x7f031006e620 0x7f0310070ad0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f031800b3b0 tx=0x7f0318009e30 comp rx=0 tx=0).ready entity=mgr.14680 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:12:26.899 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:12:26.900+0000 7f0324ff9700 1 -- 172.21.15.162:0/248115955 <== mon.0 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186652 (secure 0 0 0) 0x7f031c05b230 con 0x7f03280fcf60 2024-04-27T04:12:27.218 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:12:27.219+0000 7f032e7f9700 1 -- 172.21.15.162:0/248115955 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f032810e840 con 0x7f03280fcf60 2024-04-27T04:12:27.218 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:12:27.219+0000 7f0324ff9700 1 -- 172.21.15.162:0/248115955 <== mon.0 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+739 (secure 0 0 0) 0x7f031c0126b0 con 0x7f03280fcf60 2024-04-27T04:12:27.219 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-04-27T04:12:27.219 INFO:teuthology.orchestra.run.smithi162.stdout: "mon": { 2024-04-27T04:12:27.219 INFO:teuthology.orchestra.run.smithi162.stdout: "ceph version 17.2.7-907-g2be3efa9 (2be3efa97a7972f8e925bd899b474329979b74d6) quincy (stable)": 2 2024-04-27T04:12:27.219 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-04-27T04:12:27.219 INFO:teuthology.orchestra.run.smithi162.stdout: "mgr": { 2024-04-27T04:12:27.219 INFO:teuthology.orchestra.run.smithi162.stdout: "ceph version 19.0.0-3253-g4e213a58 (4e213a58e787437ff508e6c890ccd5fd32dc96b5) squid (dev)": 2 2024-04-27T04:12:27.219 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-04-27T04:12:27.219 INFO:teuthology.orchestra.run.smithi162.stdout: "osd": { 2024-04-27T04:12:27.219 INFO:teuthology.orchestra.run.smithi162.stdout: "ceph version 17.2.7-907-g2be3efa9 (2be3efa97a7972f8e925bd899b474329979b74d6) quincy (stable)": 6 2024-04-27T04:12:27.219 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-04-27T04:12:27.219 INFO:teuthology.orchestra.run.smithi162.stdout: "mds": { 2024-04-27T04:12:27.219 INFO:teuthology.orchestra.run.smithi162.stdout: "ceph version 17.2.7-907-g2be3efa9 (2be3efa97a7972f8e925bd899b474329979b74d6) quincy (stable)": 4 2024-04-27T04:12:27.219 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-04-27T04:12:27.219 INFO:teuthology.orchestra.run.smithi162.stdout: "overall": { 2024-04-27T04:12:27.220 INFO:teuthology.orchestra.run.smithi162.stdout: "ceph version 17.2.7-907-g2be3efa9 (2be3efa97a7972f8e925bd899b474329979b74d6) quincy (stable)": 12, 2024-04-27T04:12:27.220 INFO:teuthology.orchestra.run.smithi162.stdout: "ceph version 19.0.0-3253-g4e213a58 (4e213a58e787437ff508e6c890ccd5fd32dc96b5) squid (dev)": 2 2024-04-27T04:12:27.220 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-04-27T04:12:27.220 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-04-27T04:12:27.221 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:12:27.221+0000 7f032e7f9700 1 -- 172.21.15.162:0/248115955 >> [v2:172.21.15.162:6800/901992993,v1:172.21.15.162:6801/901992993] conn(0x7f031006e620 msgr2=0x7f0310070ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:12:27.221 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:12:27.221+0000 7f032e7f9700 1 --2- 172.21.15.162:0/248115955 >> [v2:172.21.15.162:6800/901992993,v1:172.21.15.162:6801/901992993] conn(0x7f031006e620 0x7f0310070ad0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f031800b3b0 tx=0x7f0318009e30 comp rx=0 tx=0).stop 2024-04-27T04:12:27.221 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:12:27.221+0000 7f032e7f9700 1 -- 172.21.15.162:0/248115955 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f03280fcf60 msgr2=0x7f032810ce40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:12:27.221 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:12:27.221+0000 7f032e7f9700 1 --2- 172.21.15.162:0/248115955 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f03280fcf60 0x7f032810ce40 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f031c00c8c0 tx=0x7f031c0044c0 comp rx=0 tx=0).stop 2024-04-27T04:12:27.221 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:12:27.221+0000 7f032e7f9700 1 -- 172.21.15.162:0/248115955 shutdown_connections 2024-04-27T04:12:27.221 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:12:27.221+0000 7f032e7f9700 1 --2- 172.21.15.162:0/248115955 >> [v2:172.21.15.162:6800/901992993,v1:172.21.15.162:6801/901992993] conn(0x7f031006e620 0x7f0310070ad0 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:12:27.221 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:12:27.221+0000 7f032e7f9700 1 --2- 172.21.15.162:0/248115955 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f03280fbcb0 0x7f032810c900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:12:27.221 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:12:27.221+0000 7f032e7f9700 1 --2- 172.21.15.162:0/248115955 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f03280fcf60 0x7f032810ce40 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:12:27.222 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:12:27.221+0000 7f032e7f9700 1 -- 172.21.15.162:0/248115955 >> 172.21.15.162:0/248115955 conn(0x7f03280f7430 msgr2=0x7f03280f8b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:12:27.222 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:12:27.221+0000 7f032e7f9700 1 -- 172.21.15.162:0/248115955 shutdown_connections 2024-04-27T04:12:27.222 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:12:27.222+0000 7f032e7f9700 1 -- 172.21.15.162:0/248115955 wait complete. 2024-04-27T04:12:27.329 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:12:27.325+0000 7f7114c76700 1 -- 172.21.15.162:0/3695284856 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f71100fce80 msgr2=0x7f71100fd290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:12:27.329 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:12:27.325+0000 7f7114c76700 1 --2- 172.21.15.162:0/3695284856 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f71100fce80 0x7f71100fd290 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f7100004940 tx=0x7f7100009e30 comp rx=0 tx=0).stop 2024-04-27T04:12:27.329 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:12:27.325+0000 7f7114c76700 1 -- 172.21.15.162:0/3695284856 shutdown_connections 2024-04-27T04:12:27.329 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:12:27.325+0000 7f7114c76700 1 --2- 172.21.15.162:0/3695284856 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f71100fe080 0x7f71100fe4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:12:27.329 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:12:27.325+0000 7f7114c76700 1 --2- 172.21.15.162:0/3695284856 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f71100fce80 0x7f71100fd290 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:12:27.329 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:12:27.325+0000 7f7114c76700 1 -- 172.21.15.162:0/3695284856 >> 172.21.15.162:0/3695284856 conn(0x7f71100f8590 msgr2=0x7f71100fa9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:12:27.329 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:12:27.325+0000 7f7114c76700 1 -- 172.21.15.162:0/3695284856 shutdown_connections 2024-04-27T04:12:27.329 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:12:27.325+0000 7f7114c76700 1 -- 172.21.15.162:0/3695284856 wait complete. 2024-04-27T04:12:27.329 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:12:27.325+0000 7f7114c76700 1 Processor -- start 2024-04-27T04:12:27.330 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:12:27.325+0000 7f7114c76700 1 -- start start 2024-04-27T04:12:27.330 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:12:27.326+0000 7f7114c76700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f71100fce80 0x7f7110071de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:12:27.330 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:12:27.326+0000 7f7114c76700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f71100fe080 0x7f71100702b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:12:27.330 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:12:27.326+0000 7f7114c76700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7110070860 con 0x7f71100fce80 2024-04-27T04:12:27.330 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:12:27.326+0000 7f7114c76700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71100709d0 con 0x7f71100fe080 2024-04-27T04:12:27.330 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:12:27.326+0000 7f710e59c700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f71100fce80 0x7f7110071de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:12:27.330 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:12:27.326+0000 7f710e59c700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f71100fce80 0x7f7110071de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:59524/0 (socket says 172.21.15.162:59524) 2024-04-27T04:12:27.330 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:12:27.326+0000 7f710e59c700 1 -- 172.21.15.162:0/4098791282 learned_addr learned my addr 172.21.15.162:0/4098791282 (peer_addr_for_me v2:172.21.15.162:0/0) 2024-04-27T04:12:27.330 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:12:27.326+0000 7f710dd9b700 1 --2- 172.21.15.162:0/4098791282 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f71100fe080 0x7f71100702b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:12:27.331 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:12:27.326+0000 7f710e59c700 1 -- 172.21.15.162:0/4098791282 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f71100fe080 msgr2=0x7f71100702b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:12:27.331 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:12:27.326+0000 7f710e59c700 1 --2- 172.21.15.162:0/4098791282 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f71100fe080 0x7f71100702b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:12:27.331 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:12:27.326+0000 7f710e59c700 1 -- 172.21.15.162:0/4098791282 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7100005040 con 0x7f71100fce80 2024-04-27T04:12:27.331 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:12:27.326+0000 7f710dd9b700 1 --2- 172.21.15.162:0/4098791282 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f71100fe080 0x7f71100702b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-27T04:12:27.331 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:12:27.326+0000 7f710e59c700 1 --2- 172.21.15.162:0/4098791282 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f71100fce80 0x7f7110071de0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f710000b3b0 tx=0x7f7100009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:12:27.331 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:12:27.326+0000 7f70feffd700 1 -- 172.21.15.162:0/4098791282 <== mon.0 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f710000e040 con 0x7f71100fce80 2024-04-27T04:12:27.331 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:12:27.326+0000 7f7114c76700 1 -- 172.21.15.162:0/4098791282 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7110070c50 con 0x7f71100fce80 2024-04-27T04:12:27.331 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:12:27.326+0000 7f70feffd700 1 -- 172.21.15.162:0/4098791282 <== mon.0 v2:172.21.15.162:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f710000bcc0 con 0x7f71100fce80 2024-04-27T04:12:27.331 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:12:27.326+0000 7f70feffd700 1 -- 172.21.15.162:0/4098791282 <== mon.0 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f710001c630 con 0x7f71100fce80 2024-04-27T04:12:27.331 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:12:27.326+0000 7f7114c76700 1 -- 172.21.15.162:0/4098791282 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7110115a70 con 0x7f71100fce80 2024-04-27T04:12:27.331 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:12:27.327+0000 7f7114c76700 1 -- 172.21.15.162:0/4098791282 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f711005f5c0 con 0x7f71100fce80 2024-04-27T04:12:27.331 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:12:27.328+0000 7f70feffd700 1 -- 172.21.15.162:0/4098791282 <== mon.0 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 31) v1 ==== 92571+0+0 (secure 0 0 0) 0x7f71000139f0 con 0x7f71100fce80 2024-04-27T04:12:27.331 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:12:27.328+0000 7f70feffd700 1 --2- 172.21.15.162:0/4098791282 >> [v2:172.21.15.162:6800/901992993,v1:172.21.15.162:6801/901992993] conn(0x7f70f806e620 0x7f70f8070ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:12:27.331 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:12:27.329+0000 7f70feffd700 1 -- 172.21.15.162:0/4098791282 <== mon.0 v2:172.21.15.162:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6152+0+0 (secure 0 0 0) 0x7f710008ed50 con 0x7f71100fce80 2024-04-27T04:12:27.331 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:12:27.329+0000 7f710dd9b700 1 --2- 172.21.15.162:0/4098791282 >> [v2:172.21.15.162:6800/901992993,v1:172.21.15.162:6801/901992993] conn(0x7f70f806e620 0x7f70f8070ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:12:27.332 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:12:27.329+0000 7f710dd9b700 1 --2- 172.21.15.162:0/4098791282 >> [v2:172.21.15.162:6800/901992993,v1:172.21.15.162:6801/901992993] conn(0x7f70f806e620 0x7f70f8070ad0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f71040095c0 tx=0x7f7104007f20 comp rx=0 tx=0).ready entity=mgr.14680 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:12:27.332 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:12:27.333+0000 7f70feffd700 1 -- 172.21.15.162:0/4098791282 <== mon.0 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186652 (secure 0 0 0) 0x7f7100058250 con 0x7f71100fce80 2024-04-27T04:12:27.564 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:12:27.564+0000 7f7114c76700 1 -- 172.21.15.162:0/4098791282 --> [v2:172.21.15.162:6800/901992993,v1:172.21.15.162:6801/901992993] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f7110102440 con 0x7f70f806e620 2024-04-27T04:12:27.565 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:12:27 smithi162 ceph-mon[26841]: from='client.14750 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-27T04:12:27.565 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:12:27 smithi162 ceph-mon[26841]: pgmap v30: 65 pgs: 65 active+clean; 208 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:12:27.565 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:12:27 smithi162 ceph-mon[26841]: from='client.14754 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-27T04:12:27.565 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:12:27 smithi162 ceph-mon[26841]: from='mgr.14680 172.21.15.162:0/501880828' entity='mgr.smithi162.oennfn' 2024-04-27T04:12:27.565 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:12:27 smithi162 ceph-mon[26841]: from='mgr.14680 172.21.15.162:0/501880828' entity='mgr.smithi162.oennfn' 2024-04-27T04:12:27.565 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:12:27 smithi162 ceph-mon[26841]: from='mgr.14680 172.21.15.162:0/501880828' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T04:12:27.566 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:12:27 smithi162 ceph-mon[26841]: from='mgr.14680 172.21.15.162:0/501880828' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T04:12:27.566 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:12:27 smithi162 ceph-mon[26841]: from='mgr.14680 172.21.15.162:0/501880828' entity='mgr.smithi162.oennfn' 2024-04-27T04:12:27.566 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:12:27 smithi162 ceph-mon[26841]: from='mgr.14680 172.21.15.162:0/501880828' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T04:12:27.566 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:12:27 smithi162 ceph-mon[26841]: from='mgr.14680 172.21.15.162:0/501880828' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "versions"}]: dispatch 2024-04-27T04:12:27.566 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:12:27 smithi162 ceph-mon[26841]: from='mgr.14680 172.21.15.162:0/501880828' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "versions"}]: dispatch 2024-04-27T04:12:27.567 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:12:27 smithi162 ceph-mon[26841]: from='mgr.14680 172.21.15.162:0/501880828' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "versions"}]: dispatch 2024-04-27T04:12:27.567 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:12:27 smithi162 ceph-mon[26841]: from='mgr.14680 172.21.15.162:0/501880828' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "versions"}]: dispatch 2024-04-27T04:12:27.567 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:12:27 smithi162 ceph-mon[26841]: from='mgr.14680 172.21.15.162:0/501880828' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "versions"}]: dispatch 2024-04-27T04:12:27.567 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:12:27 smithi162 ceph-mon[26841]: from='mgr.14680 172.21.15.162:0/501880828' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "versions"}]: dispatch 2024-04-27T04:12:27.567 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:12:27 smithi162 ceph-mon[26841]: from='mgr.14680 172.21.15.162:0/501880828' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "versions"}]: dispatch 2024-04-27T04:12:27.567 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:12:27 smithi162 ceph-mon[26841]: from='mgr.14680 172.21.15.162:0/501880828' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "versions"}]: dispatch 2024-04-27T04:12:27.567 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:12:27 smithi162 ceph-mon[26841]: from='mgr.14680 172.21.15.162:0/501880828' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "versions"}]: dispatch 2024-04-27T04:12:27.567 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:12:27 smithi162 ceph-mon[26841]: from='client.14758 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-27T04:12:27.567 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:12:27 smithi162 ceph-mon[26841]: from='client.? 172.21.15.162:0/248115955' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-27T04:12:27.567 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:12:27.565+0000 7f70feffd700 1 -- 172.21.15.162:0/4098791282 <== mgr.14680 v2:172.21.15.162:6800/901992993 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+367 (secure 0 0 0) 0x7f7110102440 con 0x7f70f806e620 2024-04-27T04:12:27.568 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-04-27T04:12:27.568 INFO:teuthology.orchestra.run.smithi162.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:90a3a9fa9ceb8977f6dad9ccd36fe46018d07e2118199820523a6b35301d9c16", 2024-04-27T04:12:27.568 INFO:teuthology.orchestra.run.smithi162.stdout: "in_progress": true, 2024-04-27T04:12:27.568 INFO:teuthology.orchestra.run.smithi162.stdout: "which": "Upgrading daemons of type(s) mgr", 2024-04-27T04:12:27.568 INFO:teuthology.orchestra.run.smithi162.stdout: "services_complete": [ 2024-04-27T04:12:27.568 INFO:teuthology.orchestra.run.smithi162.stdout: "mgr" 2024-04-27T04:12:27.568 INFO:teuthology.orchestra.run.smithi162.stdout: ], 2024-04-27T04:12:27.568 INFO:teuthology.orchestra.run.smithi162.stdout: "progress": "2/2 daemons upgraded", 2024-04-27T04:12:27.568 INFO:teuthology.orchestra.run.smithi162.stdout: "message": "Currently upgrading prometheus daemons", 2024-04-27T04:12:27.568 INFO:teuthology.orchestra.run.smithi162.stdout: "is_paused": false 2024-04-27T04:12:27.568 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-04-27T04:12:27.569 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:12:27.568+0000 7f7114c76700 1 -- 172.21.15.162:0/4098791282 >> [v2:172.21.15.162:6800/901992993,v1:172.21.15.162:6801/901992993] conn(0x7f70f806e620 msgr2=0x7f70f8070ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:12:27.569 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:12:27.568+0000 7f7114c76700 1 --2- 172.21.15.162:0/4098791282 >> [v2:172.21.15.162:6800/901992993,v1:172.21.15.162:6801/901992993] conn(0x7f70f806e620 0x7f70f8070ad0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f71040095c0 tx=0x7f7104007f20 comp rx=0 tx=0).stop 2024-04-27T04:12:27.569 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:12:27.568+0000 7f7114c76700 1 -- 172.21.15.162:0/4098791282 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f71100fce80 msgr2=0x7f7110071de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:12:27.570 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:12:27.568+0000 7f7114c76700 1 --2- 172.21.15.162:0/4098791282 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f71100fce80 0x7f7110071de0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f710000b3b0 tx=0x7f7100009e30 comp rx=0 tx=0).stop 2024-04-27T04:12:27.570 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:12:27.568+0000 7f7114c76700 1 -- 172.21.15.162:0/4098791282 shutdown_connections 2024-04-27T04:12:27.570 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:12:27.568+0000 7f7114c76700 1 --2- 172.21.15.162:0/4098791282 >> [v2:172.21.15.162:6800/901992993,v1:172.21.15.162:6801/901992993] conn(0x7f70f806e620 0x7f70f8070ad0 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:12:27.570 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:12:27.568+0000 7f7114c76700 1 --2- 172.21.15.162:0/4098791282 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f71100fce80 0x7f7110071de0 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:12:27.570 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:12:27.568+0000 7f7114c76700 1 --2- 172.21.15.162:0/4098791282 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f71100fe080 0x7f71100702b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:12:27.570 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:12:27.568+0000 7f7114c76700 1 -- 172.21.15.162:0/4098791282 >> 172.21.15.162:0/4098791282 conn(0x7f71100f8590 msgr2=0x7f71100f9d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:12:27.570 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:12:27.568+0000 7f7114c76700 1 -- 172.21.15.162:0/4098791282 shutdown_connections 2024-04-27T04:12:27.570 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:12:27.568+0000 7f7114c76700 1 -- 172.21.15.162:0/4098791282 wait complete. 2024-04-27T04:12:27.922 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:12:27 smithi181 ceph-mon[33774]: from='client.14750 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-27T04:12:27.923 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:12:27 smithi181 ceph-mon[33774]: pgmap v30: 65 pgs: 65 active+clean; 208 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:12:27.923 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:12:27 smithi181 ceph-mon[33774]: from='client.14754 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-27T04:12:27.923 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:12:27 smithi181 ceph-mon[33774]: from='mgr.14680 172.21.15.162:0/501880828' entity='mgr.smithi162.oennfn' 2024-04-27T04:12:27.923 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:12:27 smithi181 ceph-mon[33774]: from='mgr.14680 172.21.15.162:0/501880828' entity='mgr.smithi162.oennfn' 2024-04-27T04:12:27.923 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:12:27 smithi181 ceph-mon[33774]: from='mgr.14680 172.21.15.162:0/501880828' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T04:12:27.923 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:12:27 smithi181 ceph-mon[33774]: from='mgr.14680 172.21.15.162:0/501880828' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T04:12:27.923 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:12:27 smithi181 ceph-mon[33774]: from='mgr.14680 172.21.15.162:0/501880828' entity='mgr.smithi162.oennfn' 2024-04-27T04:12:27.923 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:12:27 smithi181 ceph-mon[33774]: from='mgr.14680 172.21.15.162:0/501880828' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T04:12:27.923 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:12:27 smithi181 ceph-mon[33774]: from='mgr.14680 172.21.15.162:0/501880828' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "versions"}]: dispatch 2024-04-27T04:12:27.923 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:12:27 smithi181 ceph-mon[33774]: from='mgr.14680 172.21.15.162:0/501880828' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "versions"}]: dispatch 2024-04-27T04:12:27.923 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:12:27 smithi181 ceph-mon[33774]: from='mgr.14680 172.21.15.162:0/501880828' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "versions"}]: dispatch 2024-04-27T04:12:27.923 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:12:27 smithi181 ceph-mon[33774]: from='mgr.14680 172.21.15.162:0/501880828' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "versions"}]: dispatch 2024-04-27T04:12:27.924 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:12:27 smithi181 ceph-mon[33774]: from='mgr.14680 172.21.15.162:0/501880828' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "versions"}]: dispatch 2024-04-27T04:12:27.924 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:12:27 smithi181 ceph-mon[33774]: from='mgr.14680 172.21.15.162:0/501880828' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "versions"}]: dispatch 2024-04-27T04:12:27.924 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:12:27 smithi181 ceph-mon[33774]: from='mgr.14680 172.21.15.162:0/501880828' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "versions"}]: dispatch 2024-04-27T04:12:27.924 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:12:27 smithi181 ceph-mon[33774]: from='mgr.14680 172.21.15.162:0/501880828' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "versions"}]: dispatch 2024-04-27T04:12:27.924 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:12:27 smithi181 ceph-mon[33774]: from='mgr.14680 172.21.15.162:0/501880828' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "versions"}]: dispatch 2024-04-27T04:12:27.924 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:12:27 smithi181 ceph-mon[33774]: from='client.14758 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-27T04:12:27.924 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:12:27 smithi181 ceph-mon[33774]: from='client.? 172.21.15.162:0/248115955' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-27T04:12:28.636 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:12:28 smithi162 ceph-mon[26841]: from='client.14766 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-27T04:12:28.922 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:12:28 smithi181 ceph-mon[33774]: from='client.14766 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-27T04:12:29.789 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:12:29 smithi162 ceph-mon[26841]: pgmap v31: 65 pgs: 65 active+clean; 208 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:12:29.789 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:12:29 smithi162 ceph-mon[26841]: Upgrade: Updating prometheus.smithi162 2024-04-27T04:12:29.789 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:12:29 smithi162 ceph-mon[26841]: Deploying daemon prometheus.smithi162 on smithi162 2024-04-27T04:12:29.922 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:12:29 smithi181 ceph-mon[33774]: pgmap v31: 65 pgs: 65 active+clean; 208 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:12:29.922 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:12:29 smithi181 ceph-mon[33774]: Upgrade: Updating prometheus.smithi162 2024-04-27T04:12:29.922 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:12:29 smithi181 ceph-mon[33774]: Deploying daemon prometheus.smithi162 on smithi162 2024-04-27T04:12:31.887 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:12:31 smithi162 ceph-mon[26841]: pgmap v32: 65 pgs: 65 active+clean; 208 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:12:31.922 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:12:31 smithi181 ceph-mon[33774]: pgmap v32: 65 pgs: 65 active+clean; 208 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:12:32.685 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:12:32 smithi162 ceph-mon[26841]: from='mgr.14680 172.21.15.162:0/501880828' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-27T04:12:32.823 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:12:32 smithi181 ceph-mon[33774]: from='mgr.14680 172.21.15.162:0/501880828' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-27T04:12:33.809 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:12:33 smithi162 ceph-mon[26841]: pgmap v33: 65 pgs: 65 active+clean; 208 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:12:33.922 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:12:33 smithi181 ceph-mon[33774]: pgmap v33: 65 pgs: 65 active+clean; 208 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:12:35.637 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:12:35 smithi162 ceph-mon[26841]: pgmap v34: 65 pgs: 65 active+clean; 208 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:12:35.637 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:12:35 smithi162 ceph-mon[26841]: from='mgr.14680 172.21.15.162:0/501880828' entity='mgr.smithi162.oennfn' 2024-04-27T04:12:35.637 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:12:35 smithi162 ceph-mon[26841]: from='mgr.14680 172.21.15.162:0/501880828' entity='mgr.smithi162.oennfn' 2024-04-27T04:12:35.637 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:12:35 smithi162 ceph-mon[26841]: from='mgr.14680 172.21.15.162:0/501880828' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T04:12:35.637 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:12:35 smithi162 ceph-mon[26841]: from='mgr.14680 172.21.15.162:0/501880828' entity='mgr.smithi162.oennfn' 2024-04-27T04:12:35.637 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:12:35 smithi162 ceph-mon[26841]: from='mgr.14680 172.21.15.162:0/501880828' entity='mgr.smithi162.oennfn' 2024-04-27T04:12:35.672 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:12:35 smithi181 ceph-mon[33774]: pgmap v34: 65 pgs: 65 active+clean; 208 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:12:35.672 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:12:35 smithi181 ceph-mon[33774]: from='mgr.14680 172.21.15.162:0/501880828' entity='mgr.smithi162.oennfn' 2024-04-27T04:12:35.672 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:12:35 smithi181 ceph-mon[33774]: from='mgr.14680 172.21.15.162:0/501880828' entity='mgr.smithi162.oennfn' 2024-04-27T04:12:35.672 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:12:35 smithi181 ceph-mon[33774]: from='mgr.14680 172.21.15.162:0/501880828' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T04:12:35.672 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:12:35 smithi181 ceph-mon[33774]: from='mgr.14680 172.21.15.162:0/501880828' entity='mgr.smithi162.oennfn' 2024-04-27T04:12:35.673 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:12:35 smithi181 ceph-mon[33774]: from='mgr.14680 172.21.15.162:0/501880828' entity='mgr.smithi162.oennfn' 2024-04-27T04:12:37.637 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:12:37 smithi162 ceph-mon[26841]: pgmap v35: 65 pgs: 65 active+clean; 208 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:12:37.672 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:12:37 smithi181 ceph-mon[33774]: pgmap v35: 65 pgs: 65 active+clean; 208 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:12:39.637 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:12:39 smithi162 ceph-mon[26841]: pgmap v36: 65 pgs: 65 active+clean; 208 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:12:39.672 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:12:39 smithi181 ceph-mon[33774]: pgmap v36: 65 pgs: 65 active+clean; 208 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:12:40.887 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:12:40 smithi162 ceph-mon[26841]: from='mgr.14680 172.21.15.162:0/501880828' entity='mgr.smithi162.oennfn' 2024-04-27T04:12:40.887 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:12:40 smithi162 ceph-mon[26841]: from='mgr.14680 172.21.15.162:0/501880828' entity='mgr.smithi162.oennfn' 2024-04-27T04:12:40.923 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:12:40 smithi181 ceph-mon[33774]: from='mgr.14680 172.21.15.162:0/501880828' entity='mgr.smithi162.oennfn' 2024-04-27T04:12:40.923 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:12:40 smithi181 ceph-mon[33774]: from='mgr.14680 172.21.15.162:0/501880828' entity='mgr.smithi162.oennfn' 2024-04-27T04:12:41.887 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:12:41 smithi162 ceph-mon[26841]: pgmap v37: 65 pgs: 65 active+clean; 208 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:12:41.887 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:12:41 smithi162 ceph-mon[26841]: from='mgr.14680 172.21.15.162:0/501880828' entity='mgr.smithi162.oennfn' 2024-04-27T04:12:41.887 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:12:41 smithi162 ceph-mon[26841]: from='mgr.14680 172.21.15.162:0/501880828' entity='mgr.smithi162.oennfn' 2024-04-27T04:12:41.922 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:12:41 smithi181 ceph-mon[33774]: pgmap v37: 65 pgs: 65 active+clean; 208 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:12:41.922 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:12:41 smithi181 ceph-mon[33774]: from='mgr.14680 172.21.15.162:0/501880828' entity='mgr.smithi162.oennfn' 2024-04-27T04:12:41.922 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:12:41 smithi181 ceph-mon[33774]: from='mgr.14680 172.21.15.162:0/501880828' entity='mgr.smithi162.oennfn' 2024-04-27T04:12:43.742 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:12:43 smithi162 ceph-mon[26841]: pgmap v38: 65 pgs: 65 active+clean; 208 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:12:43.922 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:12:43 smithi181 ceph-mon[33774]: pgmap v38: 65 pgs: 65 active+clean; 208 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:12:45.637 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:12:45 smithi162 ceph-mon[26841]: pgmap v39: 65 pgs: 65 active+clean; 208 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:12:45.637 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:12:45 smithi162 ceph-mon[26841]: from='mgr.14680 172.21.15.162:0/501880828' entity='mgr.smithi162.oennfn' 2024-04-27T04:12:45.637 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:12:45 smithi162 ceph-mon[26841]: from='mgr.14680 172.21.15.162:0/501880828' entity='mgr.smithi162.oennfn' 2024-04-27T04:12:45.638 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:12:45 smithi162 ceph-mon[26841]: from='mgr.14680 172.21.15.162:0/501880828' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T04:12:45.638 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:12:45 smithi162 ceph-mon[26841]: from='mgr.14680 172.21.15.162:0/501880828' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T04:12:45.638 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:12:45 smithi162 ceph-mon[26841]: from='mgr.14680 172.21.15.162:0/501880828' entity='mgr.smithi162.oennfn' 2024-04-27T04:12:45.638 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:12:45 smithi162 ceph-mon[26841]: from='mgr.14680 172.21.15.162:0/501880828' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-04-27T04:12:45.638 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:12:45 smithi162 ceph-mon[26841]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-04-27T04:12:45.638 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:12:45 smithi162 ceph-mon[26841]: from='mgr.14680 172.21.15.162:0/501880828' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T04:12:45.638 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:12:45 smithi162 ceph-mon[26841]: from='mgr.14680 172.21.15.162:0/501880828' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "versions"}]: dispatch 2024-04-27T04:12:45.638 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:12:45 smithi162 ceph-mon[26841]: from='mgr.14680 172.21.15.162:0/501880828' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "versions"}]: dispatch 2024-04-27T04:12:45.638 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:12:45 smithi162 ceph-mon[26841]: from='mgr.14680 172.21.15.162:0/501880828' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "versions"}]: dispatch 2024-04-27T04:12:45.638 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:12:45 smithi162 ceph-mon[26841]: from='mgr.14680 172.21.15.162:0/501880828' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "versions"}]: dispatch 2024-04-27T04:12:45.639 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:12:45 smithi162 ceph-mon[26841]: from='mgr.14680 172.21.15.162:0/501880828' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "versions"}]: dispatch 2024-04-27T04:12:45.639 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:12:45 smithi162 ceph-mon[26841]: from='mgr.14680 172.21.15.162:0/501880828' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "versions"}]: dispatch 2024-04-27T04:12:45.639 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:12:45 smithi162 ceph-mon[26841]: from='mgr.14680 172.21.15.162:0/501880828' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "versions"}]: dispatch 2024-04-27T04:12:45.639 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:12:45 smithi162 ceph-mon[26841]: from='mgr.14680 172.21.15.162:0/501880828' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "versions"}]: dispatch 2024-04-27T04:12:45.639 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:12:45 smithi162 ceph-mon[26841]: from='mgr.14680 172.21.15.162:0/501880828' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "versions"}]: dispatch 2024-04-27T04:12:45.639 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:12:45 smithi162 ceph-mon[26841]: from='mgr.14680 172.21.15.162:0/501880828' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "versions"}]: dispatch 2024-04-27T04:12:45.672 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:12:45 smithi181 ceph-mon[33774]: pgmap v39: 65 pgs: 65 active+clean; 208 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:12:45.672 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:12:45 smithi181 ceph-mon[33774]: from='mgr.14680 172.21.15.162:0/501880828' entity='mgr.smithi162.oennfn' 2024-04-27T04:12:45.673 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:12:45 smithi181 ceph-mon[33774]: from='mgr.14680 172.21.15.162:0/501880828' entity='mgr.smithi162.oennfn' 2024-04-27T04:12:45.673 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:12:45 smithi181 ceph-mon[33774]: from='mgr.14680 172.21.15.162:0/501880828' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T04:12:45.673 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:12:45 smithi181 ceph-mon[33774]: from='mgr.14680 172.21.15.162:0/501880828' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T04:12:45.673 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:12:45 smithi181 ceph-mon[33774]: from='mgr.14680 172.21.15.162:0/501880828' entity='mgr.smithi162.oennfn' 2024-04-27T04:12:45.673 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:12:45 smithi181 ceph-mon[33774]: from='mgr.14680 172.21.15.162:0/501880828' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-04-27T04:12:45.673 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:12:45 smithi181 ceph-mon[33774]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-04-27T04:12:45.673 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:12:45 smithi181 ceph-mon[33774]: from='mgr.14680 172.21.15.162:0/501880828' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T04:12:45.673 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:12:45 smithi181 ceph-mon[33774]: from='mgr.14680 172.21.15.162:0/501880828' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "versions"}]: dispatch 2024-04-27T04:12:45.673 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:12:45 smithi181 ceph-mon[33774]: from='mgr.14680 172.21.15.162:0/501880828' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "versions"}]: dispatch 2024-04-27T04:12:45.673 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:12:45 smithi181 ceph-mon[33774]: from='mgr.14680 172.21.15.162:0/501880828' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "versions"}]: dispatch 2024-04-27T04:12:45.673 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:12:45 smithi181 ceph-mon[33774]: from='mgr.14680 172.21.15.162:0/501880828' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "versions"}]: dispatch 2024-04-27T04:12:45.673 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:12:45 smithi181 ceph-mon[33774]: from='mgr.14680 172.21.15.162:0/501880828' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "versions"}]: dispatch 2024-04-27T04:12:45.674 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:12:45 smithi181 ceph-mon[33774]: from='mgr.14680 172.21.15.162:0/501880828' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "versions"}]: dispatch 2024-04-27T04:12:45.674 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:12:45 smithi181 ceph-mon[33774]: from='mgr.14680 172.21.15.162:0/501880828' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "versions"}]: dispatch 2024-04-27T04:12:45.674 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:12:45 smithi181 ceph-mon[33774]: from='mgr.14680 172.21.15.162:0/501880828' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "versions"}]: dispatch 2024-04-27T04:12:45.674 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:12:45 smithi181 ceph-mon[33774]: from='mgr.14680 172.21.15.162:0/501880828' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "versions"}]: dispatch 2024-04-27T04:12:45.674 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:12:45 smithi181 ceph-mon[33774]: from='mgr.14680 172.21.15.162:0/501880828' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "versions"}]: dispatch 2024-04-27T04:12:46.574 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:12:46 smithi162 ceph-mon[26841]: Upgrade: Updating alertmanager.smithi162 2024-04-27T04:12:46.574 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:12:46 smithi162 ceph-mon[26841]: Deploying daemon alertmanager.smithi162 on smithi162 2024-04-27T04:12:46.672 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:12:46 smithi181 ceph-mon[33774]: Upgrade: Updating alertmanager.smithi162 2024-04-27T04:12:46.672 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:12:46 smithi181 ceph-mon[33774]: Deploying daemon alertmanager.smithi162 on smithi162 2024-04-27T04:12:47.637 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:12:47 smithi162 ceph-mon[26841]: pgmap v40: 65 pgs: 65 active+clean; 208 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:12:47.672 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:12:47 smithi181 ceph-mon[33774]: pgmap v40: 65 pgs: 65 active+clean; 208 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:12:48.387 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:12:48 smithi162 ceph-mon[26841]: from='mgr.14680 172.21.15.162:0/501880828' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-27T04:12:48.672 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:12:48 smithi181 ceph-mon[33774]: from='mgr.14680 172.21.15.162:0/501880828' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-27T04:12:49.637 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:12:49 smithi162 ceph-mon[26841]: pgmap v41: 65 pgs: 65 active+clean; 208 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:12:49.672 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:12:49 smithi181 ceph-mon[33774]: pgmap v41: 65 pgs: 65 active+clean; 208 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:12:51.637 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:12:51 smithi162 ceph-mon[26841]: pgmap v42: 65 pgs: 65 active+clean; 208 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:12:51.672 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:12:51 smithi181 ceph-mon[33774]: pgmap v42: 65 pgs: 65 active+clean; 208 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:12:53.137 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:12:52 smithi162 ceph-mon[26841]: from='mgr.14680 172.21.15.162:0/501880828' entity='mgr.smithi162.oennfn' 2024-04-27T04:12:53.137 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:12:52 smithi162 ceph-mon[26841]: from='mgr.14680 172.21.15.162:0/501880828' entity='mgr.smithi162.oennfn' 2024-04-27T04:12:53.137 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:12:52 smithi162 ceph-mon[26841]: from='mgr.14680 172.21.15.162:0/501880828' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T04:12:53.422 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:12:52 smithi181 ceph-mon[33774]: from='mgr.14680 172.21.15.162:0/501880828' entity='mgr.smithi162.oennfn' 2024-04-27T04:12:53.422 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:12:52 smithi181 ceph-mon[33774]: from='mgr.14680 172.21.15.162:0/501880828' entity='mgr.smithi162.oennfn' 2024-04-27T04:12:53.423 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:12:52 smithi181 ceph-mon[33774]: from='mgr.14680 172.21.15.162:0/501880828' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T04:12:54.136 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:12:53 smithi162 ceph-mon[26841]: pgmap v43: 65 pgs: 65 active+clean; 208 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:12:54.422 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:12:53 smithi181 ceph-mon[33774]: pgmap v43: 65 pgs: 65 active+clean; 208 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:12:56.136 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:12:55 smithi162 ceph-mon[26841]: pgmap v44: 65 pgs: 65 active+clean; 208 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:12:56.422 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:12:55 smithi181 ceph-mon[33774]: pgmap v44: 65 pgs: 65 active+clean; 208 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:12:57.676 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:12:57.678+0000 7f1f0d842700 1 -- 172.21.15.162:0/1492031175 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1f080fe080 msgr2=0x7f1f080fe4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:12:57.676 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:12:57.678+0000 7f1f0d842700 1 --2- 172.21.15.162:0/1492031175 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1f080fe080 0x7f1f080fe4f0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f1efc004940 tx=0x7f1efc009e30 comp rx=0 tx=0).stop 2024-04-27T04:12:57.677 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:12:57.678+0000 7f1f0d842700 1 -- 172.21.15.162:0/1492031175 shutdown_connections 2024-04-27T04:12:57.677 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:12:57.678+0000 7f1f0d842700 1 --2- 172.21.15.162:0/1492031175 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1f080fe080 0x7f1f080fe4f0 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:12:57.677 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:12:57.678+0000 7f1f0d842700 1 --2- 172.21.15.162:0/1492031175 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1f080fcf10 0x7f1f080fd320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:12:57.677 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:12:57.678+0000 7f1f0d842700 1 -- 172.21.15.162:0/1492031175 >> 172.21.15.162:0/1492031175 conn(0x7f1f080f8560 msgr2=0x7f1f080fa970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:12:57.677 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:12:57.678+0000 7f1f0d842700 1 -- 172.21.15.162:0/1492031175 shutdown_connections 2024-04-27T04:12:57.677 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:12:57.678+0000 7f1f0d842700 1 -- 172.21.15.162:0/1492031175 wait complete. 2024-04-27T04:12:57.679 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:12:57.678+0000 7f1f0d842700 1 Processor -- start 2024-04-27T04:12:57.679 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:12:57.679+0000 7f1f0d842700 1 -- start start 2024-04-27T04:12:57.679 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:12:57.679+0000 7f1f0d842700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1f080fcf10 0x7f1f0810d9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:12:57.679 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:12:57.679+0000 7f1f0d842700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1f080fe080 0x7f1f0810df20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:12:57.679 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:12:57.679+0000 7f1f0d842700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f0810e520 con 0x7f1f080fcf10 2024-04-27T04:12:57.679 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:12:57.679+0000 7f1f0d842700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f0810ed70 con 0x7f1f080fe080 2024-04-27T04:12:57.679 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:12:57.679+0000 7f1f06ffd700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1f080fcf10 0x7f1f0810d9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:12:57.679 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:12:57.679+0000 7f1f06ffd700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1f080fcf10 0x7f1f0810d9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:57010/0 (socket says 172.21.15.162:57010) 2024-04-27T04:12:57.680 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:12:57.679+0000 7f1f06ffd700 1 -- 172.21.15.162:0/809234487 learned_addr learned my addr 172.21.15.162:0/809234487 (peer_addr_for_me v2:172.21.15.162:0/0) 2024-04-27T04:12:57.680 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:12:57.679+0000 7f1f067fc700 1 --2- 172.21.15.162:0/809234487 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1f080fe080 0x7f1f0810df20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:12:57.680 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:12:57.679+0000 7f1f06ffd700 1 -- 172.21.15.162:0/809234487 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1f080fe080 msgr2=0x7f1f0810df20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:12:57.680 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:12:57.679+0000 7f1f06ffd700 1 --2- 172.21.15.162:0/809234487 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1f080fe080 0x7f1f0810df20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:12:57.680 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:12:57.679+0000 7f1f06ffd700 1 -- 172.21.15.162:0/809234487 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1efc005040 con 0x7f1f080fcf10 2024-04-27T04:12:57.680 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:12:57.679+0000 7f1f06ffd700 1 --2- 172.21.15.162:0/809234487 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1f080fcf10 0x7f1f0810d9e0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f1ef800c830 tx=0x7f1ef8004470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:12:57.680 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:12:57.680+0000 7f1eef7fe700 1 -- 172.21.15.162:0/809234487 <== mon.0 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1ef800a760 con 0x7f1f080fcf10 2024-04-27T04:12:57.680 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:12:57.680+0000 7f1f0d842700 1 -- 172.21.15.162:0/809234487 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1f0810f050 con 0x7f1f080fcf10 2024-04-27T04:12:57.680 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:12:57.680+0000 7f1eef7fe700 1 -- 172.21.15.162:0/809234487 <== mon.0 v2:172.21.15.162:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f1ef800ad90 con 0x7f1f080fcf10 2024-04-27T04:12:57.680 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:12:57.680+0000 7f1eef7fe700 1 -- 172.21.15.162:0/809234487 <== mon.0 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1ef80113a0 con 0x7f1f080fcf10 2024-04-27T04:12:57.680 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:12:57.680+0000 7f1f0d842700 1 -- 172.21.15.162:0/809234487 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1f0810f590 con 0x7f1f080fcf10 2024-04-27T04:12:57.680 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:12:57.681+0000 7f1f0d842700 1 -- 172.21.15.162:0/809234487 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1f0805f5c0 con 0x7f1f080fcf10 2024-04-27T04:12:57.681 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:12:57.681+0000 7f1eef7fe700 1 -- 172.21.15.162:0/809234487 <== mon.0 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 31) v1 ==== 92571+0+0 (secure 0 0 0) 0x7f1ef8012400 con 0x7f1f080fcf10 2024-04-27T04:12:57.681 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:12:57.681+0000 7f1eef7fe700 1 --2- 172.21.15.162:0/809234487 >> [v2:172.21.15.162:6800/901992993,v1:172.21.15.162:6801/901992993] conn(0x7f1ef006e620 0x7f1ef0070ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:12:57.681 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:12:57.682+0000 7f1eef7fe700 1 -- 172.21.15.162:0/809234487 <== mon.0 v2:172.21.15.162:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6152+0+0 (secure 0 0 0) 0x7f1ef808e0a0 con 0x7f1f080fcf10 2024-04-27T04:12:57.681 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:12:57.682+0000 7f1f067fc700 1 --2- 172.21.15.162:0/809234487 >> [v2:172.21.15.162:6800/901992993,v1:172.21.15.162:6801/901992993] conn(0x7f1ef006e620 0x7f1ef0070ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:12:57.681 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:12:57.682+0000 7f1f067fc700 1 --2- 172.21.15.162:0/809234487 >> [v2:172.21.15.162:6800/901992993,v1:172.21.15.162:6801/901992993] conn(0x7f1ef006e620 0x7f1ef0070ad0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f1efc00b970 tx=0x7f1efc009e30 comp rx=0 tx=0).ready entity=mgr.14680 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:12:57.685 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:12:57.687+0000 7f1eef7fe700 1 -- 172.21.15.162:0/809234487 <== mon.0 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186652 (secure 0 0 0) 0x7f1ef805b140 con 0x7f1f080fcf10 2024-04-27T04:12:57.908 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:12:57.910+0000 7f1f0d842700 1 -- 172.21.15.162:0/809234487 --> [v2:172.21.15.162:6800/901992993,v1:172.21.15.162:6801/901992993] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f1f0810f970 con 0x7f1ef006e620 2024-04-27T04:12:57.909 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:12:57.911+0000 7f1eef7fe700 1 -- 172.21.15.162:0/809234487 <== mgr.14680 v2:172.21.15.162:6800/901992993 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+369 (secure 0 0 0) 0x7f1f0810f970 con 0x7f1ef006e620 2024-04-27T04:12:57.911 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:12:57.913+0000 7f1f0d842700 1 -- 172.21.15.162:0/809234487 >> [v2:172.21.15.162:6800/901992993,v1:172.21.15.162:6801/901992993] conn(0x7f1ef006e620 msgr2=0x7f1ef0070ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:12:57.912 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:12:57.913+0000 7f1f0d842700 1 --2- 172.21.15.162:0/809234487 >> [v2:172.21.15.162:6800/901992993,v1:172.21.15.162:6801/901992993] conn(0x7f1ef006e620 0x7f1ef0070ad0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f1efc00b970 tx=0x7f1efc009e30 comp rx=0 tx=0).stop 2024-04-27T04:12:57.912 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:12:57.913+0000 7f1f0d842700 1 -- 172.21.15.162:0/809234487 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1f080fcf10 msgr2=0x7f1f0810d9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:12:57.912 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:12:57.913+0000 7f1f0d842700 1 --2- 172.21.15.162:0/809234487 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1f080fcf10 0x7f1f0810d9e0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f1ef800c830 tx=0x7f1ef8004470 comp rx=0 tx=0).stop 2024-04-27T04:12:57.912 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:12:57.913+0000 7f1f0d842700 1 -- 172.21.15.162:0/809234487 shutdown_connections 2024-04-27T04:12:57.912 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:12:57.913+0000 7f1f0d842700 1 --2- 172.21.15.162:0/809234487 >> [v2:172.21.15.162:6800/901992993,v1:172.21.15.162:6801/901992993] conn(0x7f1ef006e620 0x7f1ef0070ad0 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:12:57.913 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:12:57.913+0000 7f1f0d842700 1 --2- 172.21.15.162:0/809234487 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1f080fcf10 0x7f1f0810d9e0 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:12:57.913 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:12:57.913+0000 7f1f0d842700 1 --2- 172.21.15.162:0/809234487 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1f080fe080 0x7f1f0810df20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:12:57.913 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:12:57.913+0000 7f1f0d842700 1 -- 172.21.15.162:0/809234487 >> 172.21.15.162:0/809234487 conn(0x7f1f080f8560 msgr2=0x7f1f080f9c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:12:57.913 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:12:57.913+0000 7f1f0d842700 1 -- 172.21.15.162:0/809234487 shutdown_connections 2024-04-27T04:12:57.913 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:12:57.913+0000 7f1f0d842700 1 -- 172.21.15.162:0/809234487 wait complete. 2024-04-27T04:12:57.922 INFO:teuthology.orchestra.run.smithi162.stdout:true 2024-04-27T04:12:58.017 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:12:58.019+0000 7f3377e41700 1 -- 172.21.15.162:0/3983952180 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f33700fcff0 msgr2=0x7f33700fd460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:12:58.018 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:12:58.019+0000 7f3377e41700 1 --2- 172.21.15.162:0/3983952180 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f33700fcff0 0x7f33700fd460 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f336c004940 tx=0x7f336c009e30 comp rx=0 tx=0).stop 2024-04-27T04:12:58.018 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:12:58.019+0000 7f3377e41700 1 -- 172.21.15.162:0/3983952180 shutdown_connections 2024-04-27T04:12:58.018 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:12:58.019+0000 7f3377e41700 1 --2- 172.21.15.162:0/3983952180 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f33700fcff0 0x7f33700fd460 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:12:58.018 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:12:58.019+0000 7f3377e41700 1 --2- 172.21.15.162:0/3983952180 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f33700fbd40 0x7f33700fc150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:12:58.018 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:12:58.019+0000 7f3377e41700 1 -- 172.21.15.162:0/3983952180 >> 172.21.15.162:0/3983952180 conn(0x7f33700f74c0 msgr2=0x7f33700f98d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:12:58.018 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:12:58.019+0000 7f3377e41700 1 -- 172.21.15.162:0/3983952180 shutdown_connections 2024-04-27T04:12:58.018 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:12:58.019+0000 7f3377e41700 1 -- 172.21.15.162:0/3983952180 wait complete. 2024-04-27T04:12:58.020 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:12:58.020+0000 7f3377e41700 1 Processor -- start 2024-04-27T04:12:58.020 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:12:58.020+0000 7f3377e41700 1 -- start start 2024-04-27T04:12:58.020 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:12:58.020+0000 7f3377e41700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f33700fbd40 0x7f3370110710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:12:58.020 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:12:58.020+0000 7f3377e41700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f33700fcff0 0x7f3370112c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:12:58.020 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:12:58.020+0000 7f3377e41700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3370066c50 con 0x7f33700fcff0 2024-04-27T04:12:58.020 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:12:58.020+0000 7f3377e41700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3370066d90 con 0x7f33700fbd40 2024-04-27T04:12:58.020 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:12:58.020+0000 7f33753dc700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f33700fcff0 0x7f3370112c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:12:58.020 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:12:58.020+0000 7f33753dc700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f33700fcff0 0x7f3370112c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:57026/0 (socket says 172.21.15.162:57026) 2024-04-27T04:12:58.021 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:12:58.020+0000 7f33753dc700 1 -- 172.21.15.162:0/524899086 learned_addr learned my addr 172.21.15.162:0/524899086 (peer_addr_for_me v2:172.21.15.162:0/0) 2024-04-27T04:12:58.021 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:12:58.020+0000 7f3375bdd700 1 --2- 172.21.15.162:0/524899086 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f33700fbd40 0x7f3370110710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:12:58.021 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:12:58.020+0000 7f33753dc700 1 -- 172.21.15.162:0/524899086 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f33700fbd40 msgr2=0x7f3370110710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:12:58.021 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:12:58.020+0000 7f33753dc700 1 --2- 172.21.15.162:0/524899086 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f33700fbd40 0x7f3370110710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:12:58.021 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:12:58.020+0000 7f33753dc700 1 -- 172.21.15.162:0/524899086 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f336c005040 con 0x7f33700fcff0 2024-04-27T04:12:58.021 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:12:58.021+0000 7f33753dc700 1 --2- 172.21.15.162:0/524899086 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f33700fcff0 0x7f3370112c60 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f336c00b970 tx=0x7f336c009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:12:58.021 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:12:58.021+0000 7f33667fc700 1 -- 172.21.15.162:0/524899086 <== mon.0 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f336c00e040 con 0x7f33700fcff0 2024-04-27T04:12:58.021 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:12:58.021+0000 7f3377e41700 1 -- 172.21.15.162:0/524899086 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3370113260 con 0x7f33700fcff0 2024-04-27T04:12:58.021 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:12:58.021+0000 7f3377e41700 1 -- 172.21.15.162:0/524899086 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f33701137f0 con 0x7f33700fcff0 2024-04-27T04:12:58.021 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:12:58.021+0000 7f33667fc700 1 -- 172.21.15.162:0/524899086 <== mon.0 v2:172.21.15.162:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f336c004290 con 0x7f33700fcff0 2024-04-27T04:12:58.022 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:12:58.022+0000 7f33667fc700 1 -- 172.21.15.162:0/524899086 <== mon.0 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f336c025640 con 0x7f33700fcff0 2024-04-27T04:12:58.022 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:12:58.022+0000 7f3377e41700 1 -- 172.21.15.162:0/524899086 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f33540052e0 con 0x7f33700fcff0 2024-04-27T04:12:58.022 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:12:58.023+0000 7f33667fc700 1 -- 172.21.15.162:0/524899086 <== mon.0 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 31) v1 ==== 92571+0+0 (secure 0 0 0) 0x7f336c01c3f0 con 0x7f33700fcff0 2024-04-27T04:12:58.023 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:12:58.023+0000 7f33667fc700 1 --2- 172.21.15.162:0/524899086 >> [v2:172.21.15.162:6800/901992993,v1:172.21.15.162:6801/901992993] conn(0x7f335c06e620 0x7f335c070ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:12:58.023 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:12:58.023+0000 7f3375bdd700 1 --2- 172.21.15.162:0/524899086 >> [v2:172.21.15.162:6800/901992993,v1:172.21.15.162:6801/901992993] conn(0x7f335c06e620 0x7f335c070ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:12:58.023 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:12:58.023+0000 7f33667fc700 1 -- 172.21.15.162:0/524899086 <== mon.0 v2:172.21.15.162:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6152+0+0 (secure 0 0 0) 0x7f336c08f320 con 0x7f33700fcff0 2024-04-27T04:12:58.023 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:12:58.024+0000 7f3375bdd700 1 --2- 172.21.15.162:0/524899086 >> [v2:172.21.15.162:6800/901992993,v1:172.21.15.162:6801/901992993] conn(0x7f335c06e620 0x7f335c070ad0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f3360006fd0 tx=0x7f3360008040 comp rx=0 tx=0).ready entity=mgr.14680 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:12:58.026 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:12:58.028+0000 7f33667fc700 1 -- 172.21.15.162:0/524899086 <== mon.0 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186652 (secure 0 0 0) 0x7f336c058820 con 0x7f33700fcff0 2024-04-27T04:12:58.257 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:12:58.259+0000 7f3377e41700 1 -- 172.21.15.162:0/524899086 --> [v2:172.21.15.162:6800/901992993,v1:172.21.15.162:6801/901992993] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f3354000bf0 con 0x7f335c06e620 2024-04-27T04:12:58.258 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:12:58.260+0000 7f33667fc700 1 -- 172.21.15.162:0/524899086 <== mgr.14680 v2:172.21.15.162:6800/901992993 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+369 (secure 0 0 0) 0x7f3354000bf0 con 0x7f335c06e620 2024-04-27T04:12:58.261 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:12:58.262+0000 7f3377e41700 1 -- 172.21.15.162:0/524899086 >> [v2:172.21.15.162:6800/901992993,v1:172.21.15.162:6801/901992993] conn(0x7f335c06e620 msgr2=0x7f335c070ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:12:58.261 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:12:58.262+0000 7f3377e41700 1 --2- 172.21.15.162:0/524899086 >> [v2:172.21.15.162:6800/901992993,v1:172.21.15.162:6801/901992993] conn(0x7f335c06e620 0x7f335c070ad0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f3360006fd0 tx=0x7f3360008040 comp rx=0 tx=0).stop 2024-04-27T04:12:58.261 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:12:58.262+0000 7f3377e41700 1 -- 172.21.15.162:0/524899086 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f33700fcff0 msgr2=0x7f3370112c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:12:58.261 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:12:58.262+0000 7f3377e41700 1 --2- 172.21.15.162:0/524899086 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f33700fcff0 0x7f3370112c60 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f336c00b970 tx=0x7f336c009e30 comp rx=0 tx=0).stop 2024-04-27T04:12:58.261 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:12:58.262+0000 7f3377e41700 1 -- 172.21.15.162:0/524899086 shutdown_connections 2024-04-27T04:12:58.261 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:12:58.262+0000 7f3377e41700 1 --2- 172.21.15.162:0/524899086 >> [v2:172.21.15.162:6800/901992993,v1:172.21.15.162:6801/901992993] conn(0x7f335c06e620 0x7f335c070ad0 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:12:58.261 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:12:58.262+0000 7f3377e41700 1 --2- 172.21.15.162:0/524899086 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f33700fbd40 0x7f3370110710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:12:58.261 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:12:58.262+0000 7f3377e41700 1 --2- 172.21.15.162:0/524899086 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f33700fcff0 0x7f3370112c60 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:12:58.262 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:12:58.262+0000 7f3377e41700 1 -- 172.21.15.162:0/524899086 >> 172.21.15.162:0/524899086 conn(0x7f33700f74c0 msgr2=0x7f33700f8b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:12:58.262 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:12:58.262+0000 7f3377e41700 1 -- 172.21.15.162:0/524899086 shutdown_connections 2024-04-27T04:12:58.262 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:12:58.262+0000 7f3377e41700 1 -- 172.21.15.162:0/524899086 wait complete. 2024-04-27T04:12:58.366 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:12:58.367+0000 7f37f846e700 1 -- 172.21.15.162:0/400112696 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f37f00fce80 msgr2=0x7f37f00fd290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:12:58.366 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:12:58.367+0000 7f37f846e700 1 --2- 172.21.15.162:0/400112696 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f37f00fce80 0x7f37f00fd290 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f37ec004940 tx=0x7f37ec009e30 comp rx=0 tx=0).stop 2024-04-27T04:12:58.366 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:12:58.367+0000 7f37f846e700 1 -- 172.21.15.162:0/400112696 shutdown_connections 2024-04-27T04:12:58.366 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:12:58.367+0000 7f37f846e700 1 --2- 172.21.15.162:0/400112696 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f37f00fdff0 0x7f37f00fe460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:12:58.366 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:12:58.367+0000 7f37f846e700 1 --2- 172.21.15.162:0/400112696 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f37f00fce80 0x7f37f00fd290 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:12:58.366 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:12:58.367+0000 7f37f846e700 1 -- 172.21.15.162:0/400112696 >> 172.21.15.162:0/400112696 conn(0x7f37f00f8580 msgr2=0x7f37f00fa990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:12:58.367 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:12:58 smithi162 ceph-mon[26841]: pgmap v45: 65 pgs: 65 active+clean; 208 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:12:58.367 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:12:58 smithi162 ceph-mon[26841]: from='mgr.14680 172.21.15.162:0/501880828' entity='mgr.smithi162.oennfn' 2024-04-27T04:12:58.367 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:12:58 smithi162 ceph-mon[26841]: from='mgr.14680 172.21.15.162:0/501880828' entity='mgr.smithi162.oennfn' 2024-04-27T04:12:58.368 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:12:58.368+0000 7f37f846e700 1 -- 172.21.15.162:0/400112696 shutdown_connections 2024-04-27T04:12:58.368 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:12:58.368+0000 7f37f846e700 1 -- 172.21.15.162:0/400112696 wait complete. 2024-04-27T04:12:58.368 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:12:58.368+0000 7f37f846e700 1 Processor -- start 2024-04-27T04:12:58.368 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:12:58.368+0000 7f37f846e700 1 -- start start 2024-04-27T04:12:58.368 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:12:58.369+0000 7f37f846e700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f37f00fce80 0x7f37f0071c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:12:58.368 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:12:58.369+0000 7f37f846e700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f37f00fdff0 0x7f37f00702b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:12:58.368 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:12:58.369+0000 7f37f846e700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37f00707f0 con 0x7f37f00fdff0 2024-04-27T04:12:58.368 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:12:58.369+0000 7f37f846e700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37f0070960 con 0x7f37f00fce80 2024-04-27T04:12:58.368 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:12:58.369+0000 7f37f5a09700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f37f00fdff0 0x7f37f00702b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:12:58.368 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:12:58.369+0000 7f37f620a700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f37f00fce80 0x7f37f0071c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:12:58.369 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:12:58.369+0000 7f37f5a09700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f37f00fdff0 0x7f37f00702b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:57052/0 (socket says 172.21.15.162:57052) 2024-04-27T04:12:58.369 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:12:58.369+0000 7f37f5a09700 1 -- 172.21.15.162:0/137852857 learned_addr learned my addr 172.21.15.162:0/137852857 (peer_addr_for_me v2:172.21.15.162:0/0) 2024-04-27T04:12:58.369 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:12:58.369+0000 7f37f620a700 1 --2- 172.21.15.162:0/137852857 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f37f00fce80 0x7f37f0071c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.162:40956/0 (socket says 172.21.15.162:40956) 2024-04-27T04:12:58.369 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:12:58.369+0000 7f37f5a09700 1 -- 172.21.15.162:0/137852857 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f37f00fce80 msgr2=0x7f37f0071c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:12:58.369 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:12:58.369+0000 7f37f5a09700 1 --2- 172.21.15.162:0/137852857 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f37f00fce80 0x7f37f0071c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:12:58.369 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:12:58.369+0000 7f37f5a09700 1 -- 172.21.15.162:0/137852857 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f37ec005040 con 0x7f37f00fdff0 2024-04-27T04:12:58.369 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:12:58.369+0000 7f37f5a09700 1 --2- 172.21.15.162:0/137852857 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f37f00fdff0 0x7f37f00702b0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f37e000b790 tx=0x7f37e0009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:12:58.369 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:12:58.370+0000 7f37e6ffd700 1 -- 172.21.15.162:0/137852857 <== mon.0 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f37e000f6f0 con 0x7f37f00fdff0 2024-04-27T04:12:58.369 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:12:58.370+0000 7f37e6ffd700 1 -- 172.21.15.162:0/137852857 <== mon.0 v2:172.21.15.162:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f37e000fd20 con 0x7f37f00fdff0 2024-04-27T04:12:58.370 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:12:58.370+0000 7f37e6ffd700 1 -- 172.21.15.162:0/137852857 <== mon.0 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f37e00103a0 con 0x7f37f00fdff0 2024-04-27T04:12:58.370 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:12:58.370+0000 7f37f846e700 1 -- 172.21.15.162:0/137852857 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f37f0070c40 con 0x7f37f00fdff0 2024-04-27T04:12:58.370 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:12:58.370+0000 7f37f846e700 1 -- 172.21.15.162:0/137852857 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f37f0115a30 con 0x7f37f00fdff0 2024-04-27T04:12:58.370 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:12:58.371+0000 7f37e6ffd700 1 -- 172.21.15.162:0/137852857 <== mon.0 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 31) v1 ==== 92571+0+0 (secure 0 0 0) 0x7f37e0011400 con 0x7f37f00fdff0 2024-04-27T04:12:58.370 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:12:58.371+0000 7f37f846e700 1 -- 172.21.15.162:0/137852857 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f37f004c710 con 0x7f37f00fdff0 2024-04-27T04:12:58.370 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:12:58.372+0000 7f37e6ffd700 1 --2- 172.21.15.162:0/137852857 >> [v2:172.21.15.162:6800/901992993,v1:172.21.15.162:6801/901992993] conn(0x7f37dc06e630 0x7f37dc070ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:12:58.371 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:12:58.372+0000 7f37e6ffd700 1 -- 172.21.15.162:0/137852857 <== mon.0 v2:172.21.15.162:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6152+0+0 (secure 0 0 0) 0x7f37e008db60 con 0x7f37f00fdff0 2024-04-27T04:12:58.371 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:12:58.372+0000 7f37f620a700 1 --2- 172.21.15.162:0/137852857 >> [v2:172.21.15.162:6800/901992993,v1:172.21.15.162:6801/901992993] conn(0x7f37dc06e630 0x7f37dc070ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:12:58.371 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:12:58.372+0000 7f37f620a700 1 --2- 172.21.15.162:0/137852857 >> [v2:172.21.15.162:6800/901992993,v1:172.21.15.162:6801/901992993] conn(0x7f37dc06e630 0x7f37dc070ae0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f37ec00b970 tx=0x7f37ec009e30 comp rx=0 tx=0).ready entity=mgr.14680 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:12:58.375 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:12:58.377+0000 7f37e6ffd700 1 -- 172.21.15.162:0/137852857 <== mon.0 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186652 (secure 0 0 0) 0x7f37e005aca0 con 0x7f37f00fdff0 2024-04-27T04:12:58.422 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:12:58 smithi181 ceph-mon[33774]: pgmap v45: 65 pgs: 65 active+clean; 208 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:12:58.422 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:12:58 smithi181 ceph-mon[33774]: from='mgr.14680 172.21.15.162:0/501880828' entity='mgr.smithi162.oennfn' 2024-04-27T04:12:58.422 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:12:58 smithi181 ceph-mon[33774]: from='mgr.14680 172.21.15.162:0/501880828' entity='mgr.smithi162.oennfn' 2024-04-27T04:12:58.586 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:12:58.588+0000 7f37f846e700 1 -- 172.21.15.162:0/137852857 --> [v2:172.21.15.162:6800/901992993,v1:172.21.15.162:6801/901992993] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f37f0102290 con 0x7f37dc06e630 2024-04-27T04:12:58.605 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:12:58.607+0000 7f37e6ffd700 1 -- 172.21.15.162:0/137852857 <== mgr.14680 v2:172.21.15.162:6800/901992993 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3498 (secure 0 0 0) 0x7f37f0102290 con 0x7f37dc06e630 2024-04-27T04:12:58.606 INFO:teuthology.orchestra.run.smithi162.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-27T04:12:58.606 INFO:teuthology.orchestra.run.smithi162.stdout:alertmanager.smithi162 smithi162 *:9093,9094 running (7s) 1s ago 11m 16.4M - 0.25.0 c8568f914cd2 40e334659017 2024-04-27T04:12:58.606 INFO:teuthology.orchestra.run.smithi162.stdout:crash.smithi162 smithi162 running (11m) 1s ago 11m 6660k - 17.2.7-907-g2be3efa9 1c5834633715 228c98df4289 2024-04-27T04:12:58.606 INFO:teuthology.orchestra.run.smithi162.stdout:crash.smithi181 smithi181 running (10m) 39s ago 10m 6665k - 17.2.7-907-g2be3efa9 1c5834633715 c8e553e3df2b 2024-04-27T04:12:58.606 INFO:teuthology.orchestra.run.smithi162.stdout:grafana.smithi162 smithi162 *:3000 running (2m) 1s ago 10m 81.2M - 9.4.7 954c08fa6188 746f08b3fb5b 2024-04-27T04:12:58.606 INFO:teuthology.orchestra.run.smithi162.stdout:mds.cephfs.smithi162.gzlyjj smithi162 running (6m) 1s ago 6m 12.8M - 17.2.7-907-g2be3efa9 1c5834633715 27e5b030dd32 2024-04-27T04:12:58.606 INFO:teuthology.orchestra.run.smithi162.stdout:mds.cephfs.smithi162.xcnapk smithi162 running (5m) 1s ago 5m 100M - 17.2.7-907-g2be3efa9 1c5834633715 38f9030df029 2024-04-27T04:12:58.607 INFO:teuthology.orchestra.run.smithi162.stdout:mds.cephfs.smithi181.bkkcdm smithi181 running (5m) 39s ago 5m 14.3M - 17.2.7-907-g2be3efa9 1c5834633715 af55afca1263 2024-04-27T04:12:58.607 INFO:teuthology.orchestra.run.smithi162.stdout:mds.cephfs.smithi181.dzomgh smithi181 running (5m) 39s ago 5m 145M - 17.2.7-907-g2be3efa9 1c5834633715 6733296597f3 2024-04-27T04:12:58.607 INFO:teuthology.orchestra.run.smithi162.stdout:mgr.smithi162.oennfn smithi162 *:8443,9283,8765 running (99s) 1s ago 12m 500M - 19.0.0-3253-g4e213a58 67e414ff4bc4 58a748d3637b 2024-04-27T04:12:58.607 INFO:teuthology.orchestra.run.smithi162.stdout:mgr.smithi181.vtwuhz smithi181 *:8443,9283,8765 running (59s) 39s ago 10m 451M - 19.0.0-3253-g4e213a58 67e414ff4bc4 813584297b1d 2024-04-27T04:12:58.607 INFO:teuthology.orchestra.run.smithi162.stdout:mon.smithi162 smithi162 running (12m) 1s ago 12m 54.0M 2048M 17.2.7-907-g2be3efa9 1c5834633715 0f1e3682b9ab 2024-04-27T04:12:58.607 INFO:teuthology.orchestra.run.smithi162.stdout:mon.smithi181 smithi181 running (10m) 39s ago 10m 42.7M 2048M 17.2.7-907-g2be3efa9 1c5834633715 96e48aa3912f 2024-04-27T04:12:58.607 INFO:teuthology.orchestra.run.smithi162.stdout:node-exporter.smithi162 smithi162 *:9100 running (47s) 1s ago 11m 7880k - 1.5.0 0da6a335fe13 5c4c5ea9c1d7 2024-04-27T04:12:58.607 INFO:teuthology.orchestra.run.smithi162.stdout:node-exporter.smithi181 smithi181 *:9100 running (42s) 39s ago 10m 3670k - 1.5.0 0da6a335fe13 9067f14c0519 2024-04-27T04:12:58.607 INFO:teuthology.orchestra.run.smithi162.stdout:osd.0 smithi162 running (9m) 1s ago 9m 231M 1557M 17.2.7-907-g2be3efa9 1c5834633715 4a9df3aa938d 2024-04-27T04:12:58.607 INFO:teuthology.orchestra.run.smithi162.stdout:osd.1 smithi162 running (9m) 1s ago 9m 243M 1557M 17.2.7-907-g2be3efa9 1c5834633715 aefb464c46e8 2024-04-27T04:12:58.607 INFO:teuthology.orchestra.run.smithi162.stdout:osd.2 smithi162 running (8m) 1s ago 8m 202M 1557M 17.2.7-907-g2be3efa9 1c5834633715 70f16d86d38e 2024-04-27T04:12:58.607 INFO:teuthology.orchestra.run.smithi162.stdout:osd.3 smithi181 running (8m) 39s ago 8m 276M 2581M 17.2.7-907-g2be3efa9 1c5834633715 030d829be54e 2024-04-27T04:12:58.607 INFO:teuthology.orchestra.run.smithi162.stdout:osd.4 smithi181 running (7m) 39s ago 7m 244M 2581M 17.2.7-907-g2be3efa9 1c5834633715 512afbee7cf7 2024-04-27T04:12:58.607 INFO:teuthology.orchestra.run.smithi162.stdout:osd.5 smithi181 running (7m) 39s ago 7m 221M 2581M 17.2.7-907-g2be3efa9 1c5834633715 f47a1daac1ea 2024-04-27T04:12:58.608 INFO:teuthology.orchestra.run.smithi162.stdout:prometheus.smithi162 smithi162 *:9095 running (24s) 1s ago 10m 44.5M - 2.43.0 a07b618ecd1d f30657a43324 2024-04-27T04:12:58.608 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:12:58.609+0000 7f37f846e700 1 -- 172.21.15.162:0/137852857 >> [v2:172.21.15.162:6800/901992993,v1:172.21.15.162:6801/901992993] conn(0x7f37dc06e630 msgr2=0x7f37dc070ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:12:58.608 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:12:58.609+0000 7f37f846e700 1 --2- 172.21.15.162:0/137852857 >> [v2:172.21.15.162:6800/901992993,v1:172.21.15.162:6801/901992993] conn(0x7f37dc06e630 0x7f37dc070ae0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f37ec00b970 tx=0x7f37ec009e30 comp rx=0 tx=0).stop 2024-04-27T04:12:58.609 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:12:58.609+0000 7f37f846e700 1 -- 172.21.15.162:0/137852857 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f37f00fdff0 msgr2=0x7f37f00702b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:12:58.609 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:12:58.609+0000 7f37f846e700 1 --2- 172.21.15.162:0/137852857 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f37f00fdff0 0x7f37f00702b0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f37e000b790 tx=0x7f37e0009250 comp rx=0 tx=0).stop 2024-04-27T04:12:58.609 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:12:58.609+0000 7f37f846e700 1 -- 172.21.15.162:0/137852857 shutdown_connections 2024-04-27T04:12:58.609 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:12:58.609+0000 7f37f846e700 1 --2- 172.21.15.162:0/137852857 >> [v2:172.21.15.162:6800/901992993,v1:172.21.15.162:6801/901992993] conn(0x7f37dc06e630 0x7f37dc070ae0 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:12:58.609 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:12:58.609+0000 7f37f846e700 1 --2- 172.21.15.162:0/137852857 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f37f00fce80 0x7f37f0071c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:12:58.609 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:12:58.609+0000 7f37f846e700 1 --2- 172.21.15.162:0/137852857 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f37f00fdff0 0x7f37f00702b0 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:12:58.609 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:12:58.609+0000 7f37f846e700 1 -- 172.21.15.162:0/137852857 >> 172.21.15.162:0/137852857 conn(0x7f37f00f8580 msgr2=0x7f37f00f9ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:12:58.609 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:12:58.610+0000 7f37f846e700 1 -- 172.21.15.162:0/137852857 shutdown_connections 2024-04-27T04:12:58.609 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:12:58.610+0000 7f37f846e700 1 -- 172.21.15.162:0/137852857 wait complete. 2024-04-27T04:12:58.717 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:12:58.718+0000 7f465b193700 1 -- 172.21.15.162:0/1095223674 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f46540ffee0 msgr2=0x7f4654102300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:12:58.717 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:12:58.718+0000 7f465b193700 1 --2- 172.21.15.162:0/1095223674 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f46540ffee0 0x7f4654102300 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f4640004940 tx=0x7f4640009e30 comp rx=0 tx=0).stop 2024-04-27T04:12:58.717 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:12:58.718+0000 7f465b193700 1 -- 172.21.15.162:0/1095223674 shutdown_connections 2024-04-27T04:12:58.717 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:12:58.718+0000 7f465b193700 1 --2- 172.21.15.162:0/1095223674 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f46540ffee0 0x7f4654102300 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:12:58.718 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:12:58.718+0000 7f465b193700 1 --2- 172.21.15.162:0/1095223674 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f46540662c0 0x7f46540ff9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:12:58.718 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:12:58.718+0000 7f465b193700 1 -- 172.21.15.162:0/1095223674 >> 172.21.15.162:0/1095223674 conn(0x7f46540f7430 msgr2=0x7f46540f9840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:12:58.718 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:12:58.719+0000 7f465b193700 1 -- 172.21.15.162:0/1095223674 shutdown_connections 2024-04-27T04:12:58.718 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:12:58.719+0000 7f465b193700 1 -- 172.21.15.162:0/1095223674 wait complete. 2024-04-27T04:12:58.718 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:12:58.719+0000 7f465b193700 1 Processor -- start 2024-04-27T04:12:58.718 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:12:58.719+0000 7f465b193700 1 -- start start 2024-04-27T04:12:58.723 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:12:58.719+0000 7f465b193700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f46540662c0 0x7f46540fd3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:12:58.723 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:12:58.719+0000 7f465b193700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f46540ffee0 0x7f46540fb8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:12:58.723 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:12:58.719+0000 7f465b193700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46540fbe50 con 0x7f46540ffee0 2024-04-27T04:12:58.723 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:12:58.719+0000 7f465b193700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46540fbfc0 con 0x7f46540662c0 2024-04-27T04:12:58.723 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:12:58.719+0000 7f4653fff700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f46540ffee0 0x7f46540fb8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:12:58.723 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:12:58.720+0000 7f4653fff700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f46540ffee0 0x7f46540fb8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:57082/0 (socket says 172.21.15.162:57082) 2024-04-27T04:12:58.723 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:12:58.720+0000 7f4653fff700 1 -- 172.21.15.162:0/3295885114 learned_addr learned my addr 172.21.15.162:0/3295885114 (peer_addr_for_me v2:172.21.15.162:0/0) 2024-04-27T04:12:58.723 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:12:58.720+0000 7f4658f2f700 1 --2- 172.21.15.162:0/3295885114 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f46540662c0 0x7f46540fd3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:12:58.723 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:12:58.720+0000 7f4653fff700 1 -- 172.21.15.162:0/3295885114 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f46540662c0 msgr2=0x7f46540fd3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:12:58.723 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:12:58.720+0000 7f4653fff700 1 --2- 172.21.15.162:0/3295885114 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f46540662c0 0x7f46540fd3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:12:58.723 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:12:58.720+0000 7f4653fff700 1 -- 172.21.15.162:0/3295885114 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4640005040 con 0x7f46540ffee0 2024-04-27T04:12:58.724 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:12:58.720+0000 7f4658f2f700 1 --2- 172.21.15.162:0/3295885114 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f46540662c0 0x7f46540fd3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-27T04:12:58.724 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:12:58.720+0000 7f4653fff700 1 --2- 172.21.15.162:0/3295885114 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f46540ffee0 0x7f46540fb8a0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f464000b3b0 tx=0x7f4640009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:12:58.724 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:12:58.720+0000 7f46517fa700 1 -- 172.21.15.162:0/3295885114 <== mon.0 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f464000e040 con 0x7f46540ffee0 2024-04-27T04:12:58.724 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:12:58.720+0000 7f46517fa700 1 -- 172.21.15.162:0/3295885114 <== mon.0 v2:172.21.15.162:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f464000bcc0 con 0x7f46540ffee0 2024-04-27T04:12:58.724 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:12:58.720+0000 7f465b193700 1 -- 172.21.15.162:0/3295885114 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f46540fc240 con 0x7f46540ffee0 2024-04-27T04:12:58.724 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:12:58.720+0000 7f46517fa700 1 -- 172.21.15.162:0/3295885114 <== mon.0 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f464001c630 con 0x7f46540ffee0 2024-04-27T04:12:58.724 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:12:58.720+0000 7f465b193700 1 -- 172.21.15.162:0/3295885114 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4654115a30 con 0x7f46540ffee0 2024-04-27T04:12:58.724 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:12:58.721+0000 7f465b193700 1 -- 172.21.15.162:0/3295885114 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f465405f5c0 con 0x7f46540ffee0 2024-04-27T04:12:58.724 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:12:58.722+0000 7f46517fa700 1 -- 172.21.15.162:0/3295885114 <== mon.0 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 31) v1 ==== 92571+0+0 (secure 0 0 0) 0x7f4640013420 con 0x7f46540ffee0 2024-04-27T04:12:58.724 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:12:58.722+0000 7f46517fa700 1 --2- 172.21.15.162:0/3295885114 >> [v2:172.21.15.162:6800/901992993,v1:172.21.15.162:6801/901992993] conn(0x7f464406e620 0x7f4644070ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:12:58.724 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:12:58.722+0000 7f46517fa700 1 -- 172.21.15.162:0/3295885114 <== mon.0 v2:172.21.15.162:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6152+0+0 (secure 0 0 0) 0x7f464008ee90 con 0x7f46540ffee0 2024-04-27T04:12:58.724 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:12:58.722+0000 7f4658f2f700 1 --2- 172.21.15.162:0/3295885114 >> [v2:172.21.15.162:6800/901992993,v1:172.21.15.162:6801/901992993] conn(0x7f464406e620 0x7f4644070ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:12:58.724 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:12:58.723+0000 7f4658f2f700 1 --2- 172.21.15.162:0/3295885114 >> [v2:172.21.15.162:6800/901992993,v1:172.21.15.162:6801/901992993] conn(0x7f464406e620 0x7f4644070ad0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f4648007fd0 tx=0x7f4648009040 comp rx=0 tx=0).ready entity=mgr.14680 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:12:58.725 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:12:58.727+0000 7f46517fa700 1 -- 172.21.15.162:0/3295885114 <== mon.0 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186652 (secure 0 0 0) 0x7f4640058270 con 0x7f46540ffee0 2024-04-27T04:12:59.019 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:12:59.020+0000 7f465b193700 1 -- 172.21.15.162:0/3295885114 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f46541058b0 con 0x7f46540ffee0 2024-04-27T04:12:59.019 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:12:59.020+0000 7f46517fa700 1 -- 172.21.15.162:0/3295885114 <== mon.0 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+739 (secure 0 0 0) 0x7f464005b930 con 0x7f46540ffee0 2024-04-27T04:12:59.019 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-04-27T04:12:59.019 INFO:teuthology.orchestra.run.smithi162.stdout: "mon": { 2024-04-27T04:12:59.019 INFO:teuthology.orchestra.run.smithi162.stdout: "ceph version 17.2.7-907-g2be3efa9 (2be3efa97a7972f8e925bd899b474329979b74d6) quincy (stable)": 2 2024-04-27T04:12:59.019 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-04-27T04:12:59.019 INFO:teuthology.orchestra.run.smithi162.stdout: "mgr": { 2024-04-27T04:12:59.019 INFO:teuthology.orchestra.run.smithi162.stdout: "ceph version 19.0.0-3253-g4e213a58 (4e213a58e787437ff508e6c890ccd5fd32dc96b5) squid (dev)": 2 2024-04-27T04:12:59.019 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-04-27T04:12:59.020 INFO:teuthology.orchestra.run.smithi162.stdout: "osd": { 2024-04-27T04:12:59.020 INFO:teuthology.orchestra.run.smithi162.stdout: "ceph version 17.2.7-907-g2be3efa9 (2be3efa97a7972f8e925bd899b474329979b74d6) quincy (stable)": 6 2024-04-27T04:12:59.020 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-04-27T04:12:59.020 INFO:teuthology.orchestra.run.smithi162.stdout: "mds": { 2024-04-27T04:12:59.020 INFO:teuthology.orchestra.run.smithi162.stdout: "ceph version 17.2.7-907-g2be3efa9 (2be3efa97a7972f8e925bd899b474329979b74d6) quincy (stable)": 4 2024-04-27T04:12:59.020 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-04-27T04:12:59.020 INFO:teuthology.orchestra.run.smithi162.stdout: "overall": { 2024-04-27T04:12:59.020 INFO:teuthology.orchestra.run.smithi162.stdout: "ceph version 17.2.7-907-g2be3efa9 (2be3efa97a7972f8e925bd899b474329979b74d6) quincy (stable)": 12, 2024-04-27T04:12:59.020 INFO:teuthology.orchestra.run.smithi162.stdout: "ceph version 19.0.0-3253-g4e213a58 (4e213a58e787437ff508e6c890ccd5fd32dc96b5) squid (dev)": 2 2024-04-27T04:12:59.020 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-04-27T04:12:59.020 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-04-27T04:12:59.021 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:12:59.022+0000 7f465b193700 1 -- 172.21.15.162:0/3295885114 >> [v2:172.21.15.162:6800/901992993,v1:172.21.15.162:6801/901992993] conn(0x7f464406e620 msgr2=0x7f4644070ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:12:59.021 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:12:59.022+0000 7f465b193700 1 --2- 172.21.15.162:0/3295885114 >> [v2:172.21.15.162:6800/901992993,v1:172.21.15.162:6801/901992993] conn(0x7f464406e620 0x7f4644070ad0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f4648007fd0 tx=0x7f4648009040 comp rx=0 tx=0).stop 2024-04-27T04:12:59.021 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:12:59.022+0000 7f465b193700 1 -- 172.21.15.162:0/3295885114 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f46540ffee0 msgr2=0x7f46540fb8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:12:59.021 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:12:59.022+0000 7f465b193700 1 --2- 172.21.15.162:0/3295885114 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f46540ffee0 0x7f46540fb8a0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f464000b3b0 tx=0x7f4640009e30 comp rx=0 tx=0).stop 2024-04-27T04:12:59.021 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:12:59.023+0000 7f465b193700 1 -- 172.21.15.162:0/3295885114 shutdown_connections 2024-04-27T04:12:59.021 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:12:59.023+0000 7f465b193700 1 --2- 172.21.15.162:0/3295885114 >> [v2:172.21.15.162:6800/901992993,v1:172.21.15.162:6801/901992993] conn(0x7f464406e620 0x7f4644070ad0 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:12:59.021 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:12:59.023+0000 7f465b193700 1 --2- 172.21.15.162:0/3295885114 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f46540662c0 0x7f46540fd3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:12:59.021 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:12:59.023+0000 7f465b193700 1 --2- 172.21.15.162:0/3295885114 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f46540ffee0 0x7f46540fb8a0 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:12:59.021 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:12:59.023+0000 7f465b193700 1 -- 172.21.15.162:0/3295885114 >> 172.21.15.162:0/3295885114 conn(0x7f46540f7430 msgr2=0x7f46540f7800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:12:59.021 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:12:59.023+0000 7f465b193700 1 -- 172.21.15.162:0/3295885114 shutdown_connections 2024-04-27T04:12:59.021 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:12:59.023+0000 7f465b193700 1 -- 172.21.15.162:0/3295885114 wait complete. 2024-04-27T04:12:59.135 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:12:59.135+0000 7f610030d700 1 -- 172.21.15.162:0/3548208409 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f60f80fdfd0 msgr2=0x7f60f80fe440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:12:59.137 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:12:59.136+0000 7f610030d700 1 --2- 172.21.15.162:0/3548208409 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f60f80fdfd0 0x7f60f80fe440 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f60e8004940 tx=0x7f60e8009e30 comp rx=0 tx=0).stop 2024-04-27T04:12:59.137 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:12:59.136+0000 7f610030d700 1 -- 172.21.15.162:0/3548208409 shutdown_connections 2024-04-27T04:12:59.137 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:12:59.136+0000 7f610030d700 1 --2- 172.21.15.162:0/3548208409 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f60f80fdfd0 0x7f60f80fe440 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:12:59.137 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:12:59.136+0000 7f610030d700 1 --2- 172.21.15.162:0/3548208409 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f60f80fce60 0x7f60f80fd270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:12:59.138 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:12:59.136+0000 7f610030d700 1 -- 172.21.15.162:0/3548208409 >> 172.21.15.162:0/3548208409 conn(0x7f60f80f8540 msgr2=0x7f60f80fa950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:12:59.138 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:12:59.136+0000 7f610030d700 1 -- 172.21.15.162:0/3548208409 shutdown_connections 2024-04-27T04:12:59.138 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:12:59.136+0000 7f610030d700 1 -- 172.21.15.162:0/3548208409 wait complete. 2024-04-27T04:12:59.138 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:12:59.136+0000 7f610030d700 1 Processor -- start 2024-04-27T04:12:59.138 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:12:59.136+0000 7f610030d700 1 -- start start 2024-04-27T04:12:59.138 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:12:59.136+0000 7f610030d700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f60f80fce60 0x7f60f810d900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:12:59.139 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:12:59.137+0000 7f610030d700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f60f80fdfd0 0x7f60f810de40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:12:59.139 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:12:59.137+0000 7f610030d700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60f810e4d0 con 0x7f60f80fdfd0 2024-04-27T04:12:59.139 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:12:59.137+0000 7f610030d700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60f8112e00 con 0x7f60f80fce60 2024-04-27T04:12:59.139 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:12:59.137+0000 7f60fd8a8700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f60f80fdfd0 0x7f60f810de40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:12:59.139 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:12:59.137+0000 7f60fd8a8700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f60f80fdfd0 0x7f60f810de40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:57100/0 (socket says 172.21.15.162:57100) 2024-04-27T04:12:59.139 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:12:59.137+0000 7f60fd8a8700 1 -- 172.21.15.162:0/1639585302 learned_addr learned my addr 172.21.15.162:0/1639585302 (peer_addr_for_me v2:172.21.15.162:0/0) 2024-04-27T04:12:59.139 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:12:59.137+0000 7f60fe0a9700 1 --2- 172.21.15.162:0/1639585302 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f60f80fce60 0x7f60f810d900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:12:59.139 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:12:59.137+0000 7f60fd8a8700 1 -- 172.21.15.162:0/1639585302 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f60f80fce60 msgr2=0x7f60f810d900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:12:59.139 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:12:59.137+0000 7f60fd8a8700 1 --2- 172.21.15.162:0/1639585302 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f60f80fce60 0x7f60f810d900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:12:59.139 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:12:59.137+0000 7f60fd8a8700 1 -- 172.21.15.162:0/1639585302 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f60e8005040 con 0x7f60f80fdfd0 2024-04-27T04:12:59.139 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:12:59.137+0000 7f60fd8a8700 1 --2- 172.21.15.162:0/1639585302 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f60f80fdfd0 0x7f60f810de40 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f60e8009150 tx=0x7f60e8009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:12:59.139 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:12:59.137+0000 7f60eeffd700 1 -- 172.21.15.162:0/1639585302 <== mon.0 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f60e800e040 con 0x7f60f80fdfd0 2024-04-27T04:12:59.140 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:12:59.137+0000 7f60eeffd700 1 -- 172.21.15.162:0/1639585302 <== mon.0 v2:172.21.15.162:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f60e800bcc0 con 0x7f60f80fdfd0 2024-04-27T04:12:59.140 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:12:59.137+0000 7f610030d700 1 -- 172.21.15.162:0/1639585302 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f60f8113000 con 0x7f60f80fdfd0 2024-04-27T04:12:59.140 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:12:59.137+0000 7f60eeffd700 1 -- 172.21.15.162:0/1639585302 <== mon.0 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f60e801c630 con 0x7f60f80fdfd0 2024-04-27T04:12:59.140 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:12:59.137+0000 7f610030d700 1 -- 172.21.15.162:0/1639585302 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f60f8113570 con 0x7f60f80fdfd0 2024-04-27T04:12:59.141 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:12:59.138+0000 7f610030d700 1 -- 172.21.15.162:0/1639585302 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f60f804c710 con 0x7f60f80fdfd0 2024-04-27T04:12:59.141 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:12:59.139+0000 7f60eeffd700 1 -- 172.21.15.162:0/1639585302 <== mon.0 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 31) v1 ==== 92571+0+0 (secure 0 0 0) 0x7f60e8013420 con 0x7f60f80fdfd0 2024-04-27T04:12:59.141 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:12:59.139+0000 7f60eeffd700 1 --2- 172.21.15.162:0/1639585302 >> [v2:172.21.15.162:6800/901992993,v1:172.21.15.162:6801/901992993] conn(0x7f60e406e630 0x7f60e4070ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:12:59.141 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:12:59.140+0000 7f60eeffd700 1 -- 172.21.15.162:0/1639585302 <== mon.0 v2:172.21.15.162:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6152+0+0 (secure 0 0 0) 0x7f60e808ee50 con 0x7f60f80fdfd0 2024-04-27T04:12:59.141 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:12:59.140+0000 7f60fe0a9700 1 --2- 172.21.15.162:0/1639585302 >> [v2:172.21.15.162:6800/901992993,v1:172.21.15.162:6801/901992993] conn(0x7f60e406e630 0x7f60e4070ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:12:59.141 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:12:59.140+0000 7f60fe0a9700 1 --2- 172.21.15.162:0/1639585302 >> [v2:172.21.15.162:6800/901992993,v1:172.21.15.162:6801/901992993] conn(0x7f60e406e630 0x7f60e4070ae0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f60f4006fd0 tx=0x7f60f4008040 comp rx=0 tx=0).ready entity=mgr.14680 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:12:59.142 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:12:59.144+0000 7f60eeffd700 1 -- 172.21.15.162:0/1639585302 <== mon.0 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186652 (secure 0 0 0) 0x7f60e805bde0 con 0x7f60f80fdfd0 2024-04-27T04:12:59.366 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:12:59.368+0000 7f610030d700 1 -- 172.21.15.162:0/1639585302 --> [v2:172.21.15.162:6800/901992993,v1:172.21.15.162:6801/901992993] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f60f8102270 con 0x7f60e406e630 2024-04-27T04:12:59.366 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:12:59 smithi162 ceph-mon[26841]: from='client.14770 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-27T04:12:59.367 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:12:59 smithi162 ceph-mon[26841]: from='mgr.14680 172.21.15.162:0/501880828' entity='mgr.smithi162.oennfn' 2024-04-27T04:12:59.367 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:12:59 smithi162 ceph-mon[26841]: from='mgr.14680 172.21.15.162:0/501880828' entity='mgr.smithi162.oennfn' 2024-04-27T04:12:59.367 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:12:59 smithi162 ceph-mon[26841]: from='client.? 172.21.15.162:0/3295885114' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-27T04:12:59.367 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:12:59.369+0000 7f60eeffd700 1 -- 172.21.15.162:0/1639585302 <== mgr.14680 v2:172.21.15.162:6800/901992993 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+369 (secure 0 0 0) 0x7f60f8102270 con 0x7f60e406e630 2024-04-27T04:12:59.367 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-04-27T04:12:59.367 INFO:teuthology.orchestra.run.smithi162.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:90a3a9fa9ceb8977f6dad9ccd36fe46018d07e2118199820523a6b35301d9c16", 2024-04-27T04:12:59.368 INFO:teuthology.orchestra.run.smithi162.stdout: "in_progress": true, 2024-04-27T04:12:59.368 INFO:teuthology.orchestra.run.smithi162.stdout: "which": "Upgrading daemons of type(s) mgr", 2024-04-27T04:12:59.368 INFO:teuthology.orchestra.run.smithi162.stdout: "services_complete": [ 2024-04-27T04:12:59.368 INFO:teuthology.orchestra.run.smithi162.stdout: "mgr" 2024-04-27T04:12:59.368 INFO:teuthology.orchestra.run.smithi162.stdout: ], 2024-04-27T04:12:59.368 INFO:teuthology.orchestra.run.smithi162.stdout: "progress": "2/2 daemons upgraded", 2024-04-27T04:12:59.368 INFO:teuthology.orchestra.run.smithi162.stdout: "message": "Currently upgrading alertmanager daemons", 2024-04-27T04:12:59.368 INFO:teuthology.orchestra.run.smithi162.stdout: "is_paused": false 2024-04-27T04:12:59.368 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-04-27T04:12:59.370 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:12:59.371+0000 7f610030d700 1 -- 172.21.15.162:0/1639585302 >> [v2:172.21.15.162:6800/901992993,v1:172.21.15.162:6801/901992993] conn(0x7f60e406e630 msgr2=0x7f60e4070ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:12:59.370 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:12:59.371+0000 7f610030d700 1 --2- 172.21.15.162:0/1639585302 >> [v2:172.21.15.162:6800/901992993,v1:172.21.15.162:6801/901992993] conn(0x7f60e406e630 0x7f60e4070ae0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f60f4006fd0 tx=0x7f60f4008040 comp rx=0 tx=0).stop 2024-04-27T04:12:59.370 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:12:59.371+0000 7f610030d700 1 -- 172.21.15.162:0/1639585302 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f60f80fdfd0 msgr2=0x7f60f810de40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:12:59.371 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:12:59.371+0000 7f610030d700 1 --2- 172.21.15.162:0/1639585302 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f60f80fdfd0 0x7f60f810de40 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f60e8009150 tx=0x7f60e8009e30 comp rx=0 tx=0).stop 2024-04-27T04:12:59.371 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:12:59.372+0000 7f610030d700 1 -- 172.21.15.162:0/1639585302 shutdown_connections 2024-04-27T04:12:59.371 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:12:59.372+0000 7f610030d700 1 --2- 172.21.15.162:0/1639585302 >> [v2:172.21.15.162:6800/901992993,v1:172.21.15.162:6801/901992993] conn(0x7f60e406e630 0x7f60e4070ae0 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:12:59.371 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:12:59.372+0000 7f610030d700 1 --2- 172.21.15.162:0/1639585302 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f60f80fce60 0x7f60f810d900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:12:59.371 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:12:59.372+0000 7f610030d700 1 --2- 172.21.15.162:0/1639585302 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f60f80fdfd0 0x7f60f810de40 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:12:59.371 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:12:59.372+0000 7f610030d700 1 -- 172.21.15.162:0/1639585302 >> 172.21.15.162:0/1639585302 conn(0x7f60f80f8540 msgr2=0x7f60f80f9ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:12:59.371 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:12:59.372+0000 7f610030d700 1 -- 172.21.15.162:0/1639585302 shutdown_connections 2024-04-27T04:12:59.371 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:12:59.372+0000 7f610030d700 1 -- 172.21.15.162:0/1639585302 wait complete. 2024-04-27T04:12:59.422 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:12:59 smithi181 ceph-mon[33774]: from='client.14770 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-27T04:12:59.422 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:12:59 smithi181 ceph-mon[33774]: from='mgr.14680 172.21.15.162:0/501880828' entity='mgr.smithi162.oennfn' 2024-04-27T04:12:59.422 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:12:59 smithi181 ceph-mon[33774]: from='mgr.14680 172.21.15.162:0/501880828' entity='mgr.smithi162.oennfn' 2024-04-27T04:12:59.422 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:12:59 smithi181 ceph-mon[33774]: from='client.? 172.21.15.162:0/3295885114' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-27T04:13:00.100 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:13:00 smithi162 ceph-mon[26841]: pgmap v46: 65 pgs: 65 active+clean; 208 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:13:00.100 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:13:00 smithi162 ceph-mon[26841]: from='client.14774 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-27T04:13:00.101 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:13:00 smithi162 ceph-mon[26841]: from='client.14778 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-27T04:13:00.422 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:13:00 smithi181 ceph-mon[33774]: pgmap v46: 65 pgs: 65 active+clean; 208 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:13:00.422 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:13:00 smithi181 ceph-mon[33774]: from='client.14774 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-27T04:13:00.422 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:13:00 smithi181 ceph-mon[33774]: from='client.14778 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-27T04:13:01.118 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:13:01 smithi162 ceph-mon[26841]: from='client.14786 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-27T04:13:01.348 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:13:01 smithi181 ceph-mon[33774]: from='client.14786 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-27T04:13:02.362 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:13:02 smithi162 ceph-mon[26841]: pgmap v47: 65 pgs: 65 active+clean; 208 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:13:02.362 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:13:02 smithi162 ceph-mon[26841]: from='mgr.14680 172.21.15.162:0/501880828' entity='mgr.smithi162.oennfn' 2024-04-27T04:13:02.362 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:13:02 smithi162 ceph-mon[26841]: from='mgr.14680 172.21.15.162:0/501880828' entity='mgr.smithi162.oennfn' 2024-04-27T04:13:02.362 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:13:02 smithi162 ceph-mon[26841]: from='mgr.14680 172.21.15.162:0/501880828' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T04:13:02.362 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:13:02 smithi162 ceph-mon[26841]: from='mgr.14680 172.21.15.162:0/501880828' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T04:13:02.362 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:13:02 smithi162 ceph-mon[26841]: from='mgr.14680 172.21.15.162:0/501880828' entity='mgr.smithi162.oennfn' 2024-04-27T04:13:02.362 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:13:02 smithi162 ceph-mon[26841]: from='mgr.14680 172.21.15.162:0/501880828' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-04-27T04:13:02.363 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:13:02 smithi162 ceph-mon[26841]: from='mgr.14680 172.21.15.162:0/501880828' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "dashboard set-alertmanager-api-host", "value": "http://smithi162:9093"}]: dispatch 2024-04-27T04:13:02.363 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:13:02 smithi162 ceph-mon[26841]: from='mgr.14680 172.21.15.162:0/501880828' entity='mgr.smithi162.oennfn' 2024-04-27T04:13:02.363 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:13:02 smithi162 ceph-mon[26841]: from='mgr.14680 172.21.15.162:0/501880828' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T04:13:02.363 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:13:02 smithi162 ceph-mon[26841]: from='mgr.14680 172.21.15.162:0/501880828' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "versions"}]: dispatch 2024-04-27T04:13:02.363 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:13:02 smithi162 ceph-mon[26841]: from='mgr.14680 172.21.15.162:0/501880828' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "versions"}]: dispatch 2024-04-27T04:13:02.363 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:13:02 smithi162 ceph-mon[26841]: from='mgr.14680 172.21.15.162:0/501880828' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "versions"}]: dispatch 2024-04-27T04:13:02.363 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:13:02 smithi162 ceph-mon[26841]: from='mgr.14680 172.21.15.162:0/501880828' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "versions"}]: dispatch 2024-04-27T04:13:02.363 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:13:02 smithi162 ceph-mon[26841]: from='mgr.14680 172.21.15.162:0/501880828' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "versions"}]: dispatch 2024-04-27T04:13:02.363 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:13:02 smithi162 ceph-mon[26841]: from='mgr.14680 172.21.15.162:0/501880828' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "versions"}]: dispatch 2024-04-27T04:13:02.363 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:13:02 smithi162 ceph-mon[26841]: from='mgr.14680 172.21.15.162:0/501880828' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "versions"}]: dispatch 2024-04-27T04:13:02.363 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:13:02 smithi162 ceph-mon[26841]: from='mgr.14680 172.21.15.162:0/501880828' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "versions"}]: dispatch 2024-04-27T04:13:02.363 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:13:02 smithi162 ceph-mon[26841]: from='mgr.14680 172.21.15.162:0/501880828' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "versions"}]: dispatch 2024-04-27T04:13:02.363 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:13:02 smithi162 ceph-mon[26841]: from='mgr.14680 172.21.15.162:0/501880828' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "versions"}]: dispatch 2024-04-27T04:13:02.363 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:13:02 smithi162 ceph-mon[26841]: from='mgr.14680 172.21.15.162:0/501880828' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "versions"}]: dispatch 2024-04-27T04:13:02.423 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:13:02 smithi181 ceph-mon[33774]: pgmap v47: 65 pgs: 65 active+clean; 208 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:13:02.423 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:13:02 smithi181 ceph-mon[33774]: from='mgr.14680 172.21.15.162:0/501880828' entity='mgr.smithi162.oennfn' 2024-04-27T04:13:02.423 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:13:02 smithi181 ceph-mon[33774]: from='mgr.14680 172.21.15.162:0/501880828' entity='mgr.smithi162.oennfn' 2024-04-27T04:13:02.423 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:13:02 smithi181 ceph-mon[33774]: from='mgr.14680 172.21.15.162:0/501880828' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T04:13:02.423 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:13:02 smithi181 ceph-mon[33774]: from='mgr.14680 172.21.15.162:0/501880828' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T04:13:02.424 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:13:02 smithi181 ceph-mon[33774]: from='mgr.14680 172.21.15.162:0/501880828' entity='mgr.smithi162.oennfn' 2024-04-27T04:13:02.424 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:13:02 smithi181 ceph-mon[33774]: from='mgr.14680 172.21.15.162:0/501880828' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-04-27T04:13:02.424 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:13:02 smithi181 ceph-mon[33774]: from='mgr.14680 172.21.15.162:0/501880828' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "dashboard set-alertmanager-api-host", "value": "http://smithi162:9093"}]: dispatch 2024-04-27T04:13:02.424 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:13:02 smithi181 ceph-mon[33774]: from='mgr.14680 172.21.15.162:0/501880828' entity='mgr.smithi162.oennfn' 2024-04-27T04:13:02.424 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:13:02 smithi181 ceph-mon[33774]: from='mgr.14680 172.21.15.162:0/501880828' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T04:13:02.424 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:13:02 smithi181 ceph-mon[33774]: from='mgr.14680 172.21.15.162:0/501880828' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "versions"}]: dispatch 2024-04-27T04:13:02.424 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:13:02 smithi181 ceph-mon[33774]: from='mgr.14680 172.21.15.162:0/501880828' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "versions"}]: dispatch 2024-04-27T04:13:02.425 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:13:02 smithi181 ceph-mon[33774]: from='mgr.14680 172.21.15.162:0/501880828' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "versions"}]: dispatch 2024-04-27T04:13:02.425 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:13:02 smithi181 ceph-mon[33774]: from='mgr.14680 172.21.15.162:0/501880828' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "versions"}]: dispatch 2024-04-27T04:13:02.425 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:13:02 smithi181 ceph-mon[33774]: from='mgr.14680 172.21.15.162:0/501880828' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "versions"}]: dispatch 2024-04-27T04:13:02.425 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:13:02 smithi181 ceph-mon[33774]: from='mgr.14680 172.21.15.162:0/501880828' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "versions"}]: dispatch 2024-04-27T04:13:02.425 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:13:02 smithi181 ceph-mon[33774]: from='mgr.14680 172.21.15.162:0/501880828' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "versions"}]: dispatch 2024-04-27T04:13:02.425 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:13:02 smithi181 ceph-mon[33774]: from='mgr.14680 172.21.15.162:0/501880828' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "versions"}]: dispatch 2024-04-27T04:13:02.426 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:13:02 smithi181 ceph-mon[33774]: from='mgr.14680 172.21.15.162:0/501880828' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "versions"}]: dispatch 2024-04-27T04:13:02.426 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:13:02 smithi181 ceph-mon[33774]: from='mgr.14680 172.21.15.162:0/501880828' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "versions"}]: dispatch 2024-04-27T04:13:02.426 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:13:02 smithi181 ceph-mon[33774]: from='mgr.14680 172.21.15.162:0/501880828' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "versions"}]: dispatch 2024-04-27T04:13:03.387 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:13:03 smithi162 ceph-mon[26841]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-04-27T04:13:03.387 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:13:03 smithi162 ceph-mon[26841]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-alertmanager-api-host", "value": "http://smithi162:9093"}]: dispatch 2024-04-27T04:13:03.387 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:13:03 smithi162 ceph-mon[26841]: from='mgr.14680 172.21.15.162:0/501880828' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-27T04:13:03.422 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:13:03 smithi181 ceph-mon[33774]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-04-27T04:13:03.422 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:13:03 smithi181 ceph-mon[33774]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-alertmanager-api-host", "value": "http://smithi162:9093"}]: dispatch 2024-04-27T04:13:03.422 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:13:03 smithi181 ceph-mon[33774]: from='mgr.14680 172.21.15.162:0/501880828' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-27T04:13:04.387 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:13:04 smithi162 ceph-mon[26841]: pgmap v48: 65 pgs: 65 active+clean; 208 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:13:04.422 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:13:04 smithi181 ceph-mon[33774]: pgmap v48: 65 pgs: 65 active+clean; 208 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:13:05.387 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:13:05 smithi162 ceph-mon[26841]: Upgrade: Updating grafana.smithi162 2024-04-27T04:13:05.387 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:13:05 smithi162 ceph-mon[26841]: Deploying daemon grafana.smithi162 on smithi162 2024-04-27T04:13:05.387 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:13:05 smithi162 ceph-mon[26841]: mgrmap e32: smithi162.oennfn(active, since 91s), standbys: smithi181.vtwuhz 2024-04-27T04:13:05.422 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:13:05 smithi181 ceph-mon[33774]: Upgrade: Updating grafana.smithi162 2024-04-27T04:13:05.422 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:13:05 smithi181 ceph-mon[33774]: Deploying daemon grafana.smithi162 on smithi162 2024-04-27T04:13:05.422 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:13:05 smithi181 ceph-mon[33774]: mgrmap e32: smithi162.oennfn(active, since 91s), standbys: smithi181.vtwuhz 2024-04-27T04:13:06.386 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:13:06 smithi162 ceph-mon[26841]: pgmap v49: 65 pgs: 65 active+clean; 208 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 1.3 KiB/s rd, 255 B/s wr, 2 op/s 2024-04-27T04:13:06.422 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:13:06 smithi181 ceph-mon[33774]: pgmap v49: 65 pgs: 65 active+clean; 208 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 1.3 KiB/s rd, 255 B/s wr, 2 op/s 2024-04-27T04:13:07.386 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:13:07 smithi162 ceph-mon[26841]: pgmap v50: 65 pgs: 65 active+clean; 208 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 938 B/s rd, 255 B/s wr, 1 op/s 2024-04-27T04:13:07.422 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:13:07 smithi181 ceph-mon[33774]: pgmap v50: 65 pgs: 65 active+clean; 208 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 938 B/s rd, 255 B/s wr, 1 op/s 2024-04-27T04:13:09.636 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:13:09 smithi162 ceph-mon[26841]: pgmap v51: 65 pgs: 65 active+clean; 208 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 938 B/s rd, 255 B/s wr, 2 op/s 2024-04-27T04:13:09.672 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:13:09 smithi181 ceph-mon[33774]: pgmap v51: 65 pgs: 65 active+clean; 208 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 938 B/s rd, 255 B/s wr, 2 op/s 2024-04-27T04:13:11.637 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:13:11 smithi162 ceph-mon[26841]: pgmap v52: 65 pgs: 65 active+clean; 208 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 1.3 KiB/s rd, 255 B/s wr, 2 op/s 2024-04-27T04:13:11.710 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:13:11 smithi181 ceph-mon[33774]: pgmap v52: 65 pgs: 65 active+clean; 208 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 1.3 KiB/s rd, 255 B/s wr, 2 op/s 2024-04-27T04:13:13.422 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:13:13 smithi181 ceph-mon[33774]: pgmap v53: 65 pgs: 65 active+clean; 208 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 938 B/s rd, 255 B/s wr, 1 op/s 2024-04-27T04:13:13.635 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:13:13 smithi162 ceph-mon[26841]: pgmap v53: 65 pgs: 65 active+clean; 208 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 938 B/s rd, 255 B/s wr, 1 op/s 2024-04-27T04:13:15.637 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:13:15 smithi162 ceph-mon[26841]: pgmap v54: 65 pgs: 65 active+clean; 208 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 1.3 KiB/s rd, 255 B/s wr, 2 op/s 2024-04-27T04:13:15.637 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:13:15 smithi162 ceph-mon[26841]: from='mgr.14680 172.21.15.162:0/501880828' entity='mgr.smithi162.oennfn' 2024-04-27T04:13:15.637 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:13:15 smithi162 ceph-mon[26841]: from='mgr.14680 172.21.15.162:0/501880828' entity='mgr.smithi162.oennfn' 2024-04-27T04:13:15.637 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:13:15 smithi162 ceph-mon[26841]: from='mgr.14680 172.21.15.162:0/501880828' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T04:13:15.672 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:13:15 smithi181 ceph-mon[33774]: pgmap v54: 65 pgs: 65 active+clean; 208 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 1.3 KiB/s rd, 255 B/s wr, 2 op/s 2024-04-27T04:13:15.672 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:13:15 smithi181 ceph-mon[33774]: from='mgr.14680 172.21.15.162:0/501880828' entity='mgr.smithi162.oennfn' 2024-04-27T04:13:15.672 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:13:15 smithi181 ceph-mon[33774]: from='mgr.14680 172.21.15.162:0/501880828' entity='mgr.smithi162.oennfn' 2024-04-27T04:13:15.672 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:13:15 smithi181 ceph-mon[33774]: from='mgr.14680 172.21.15.162:0/501880828' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T04:13:17.637 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:13:17 smithi162 ceph-mon[26841]: pgmap v55: 65 pgs: 65 active+clean; 208 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:13:17.637 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:13:17 smithi162 ceph-mon[26841]: from='mgr.14680 172.21.15.162:0/501880828' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-27T04:13:17.672 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:13:17 smithi181 ceph-mon[33774]: pgmap v55: 65 pgs: 65 active+clean; 208 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:13:17.672 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:13:17 smithi181 ceph-mon[33774]: from='mgr.14680 172.21.15.162:0/501880828' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-27T04:13:19.637 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:13:19 smithi162 ceph-mon[26841]: pgmap v56: 65 pgs: 65 active+clean; 208 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-04-27T04:13:19.672 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:13:19 smithi181 ceph-mon[33774]: pgmap v56: 65 pgs: 65 active+clean; 208 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-04-27T04:13:21.636 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:13:21 smithi162 ceph-mon[26841]: pgmap v57: 65 pgs: 65 active+clean; 208 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:13:21.637 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:13:21 smithi162 ceph-mon[26841]: from='mgr.14680 172.21.15.162:0/501880828' entity='mgr.smithi162.oennfn' 2024-04-27T04:13:21.637 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:13:21 smithi162 ceph-mon[26841]: from='mgr.14680 172.21.15.162:0/501880828' entity='mgr.smithi162.oennfn' 2024-04-27T04:13:21.672 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:13:21 smithi181 ceph-mon[33774]: pgmap v57: 65 pgs: 65 active+clean; 208 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:13:21.672 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:13:21 smithi181 ceph-mon[33774]: from='mgr.14680 172.21.15.162:0/501880828' entity='mgr.smithi162.oennfn' 2024-04-27T04:13:21.672 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:13:21 smithi181 ceph-mon[33774]: from='mgr.14680 172.21.15.162:0/501880828' entity='mgr.smithi162.oennfn' 2024-04-27T04:13:22.816 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:13:22 smithi162 ceph-mon[26841]: from='mgr.14680 172.21.15.162:0/501880828' entity='mgr.smithi162.oennfn' 2024-04-27T04:13:22.817 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:13:22 smithi162 ceph-mon[26841]: from='mgr.14680 172.21.15.162:0/501880828' entity='mgr.smithi162.oennfn' 2024-04-27T04:13:22.922 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:13:22 smithi181 ceph-mon[33774]: from='mgr.14680 172.21.15.162:0/501880828' entity='mgr.smithi162.oennfn' 2024-04-27T04:13:22.922 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:13:22 smithi181 ceph-mon[33774]: from='mgr.14680 172.21.15.162:0/501880828' entity='mgr.smithi162.oennfn' 2024-04-27T04:13:23.705 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:13:23 smithi162 ceph-mon[26841]: pgmap v58: 65 pgs: 65 active+clean; 208 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:13:23.922 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:13:23 smithi181 ceph-mon[33774]: pgmap v58: 65 pgs: 65 active+clean; 208 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:13:25.537 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:13:25 smithi162 ceph-mon[26841]: pgmap v59: 65 pgs: 65 active+clean; 208 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:13:25.922 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:13:25 smithi181 ceph-mon[33774]: pgmap v59: 65 pgs: 65 active+clean; 208 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:13:26.887 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:13:26 smithi162 ceph-mon[26841]: from='mgr.14680 172.21.15.162:0/501880828' entity='mgr.smithi162.oennfn' 2024-04-27T04:13:26.888 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:13:26 smithi162 ceph-mon[26841]: from='mgr.14680 172.21.15.162:0/501880828' entity='mgr.smithi162.oennfn' 2024-04-27T04:13:26.888 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:13:26 smithi162 ceph-mon[26841]: from='mgr.14680 172.21.15.162:0/501880828' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T04:13:26.888 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:13:26 smithi162 ceph-mon[26841]: from='mgr.14680 172.21.15.162:0/501880828' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T04:13:26.888 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:13:26 smithi162 ceph-mon[26841]: from='mgr.14680 172.21.15.162:0/501880828' entity='mgr.smithi162.oennfn' 2024-04-27T04:13:26.888 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:13:26 smithi162 ceph-mon[26841]: from='mgr.14680 172.21.15.162:0/501880828' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-04-27T04:13:26.888 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:13:26 smithi162 ceph-mon[26841]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-04-27T04:13:26.888 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:13:26 smithi162 ceph-mon[26841]: from='mgr.14680 172.21.15.162:0/501880828' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "dashboard set-grafana-api-url", "value": "https://smithi162:3000"}]: dispatch 2024-04-27T04:13:26.888 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:13:26 smithi162 ceph-mon[26841]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-grafana-api-url", "value": "https://smithi162:3000"}]: dispatch 2024-04-27T04:13:26.888 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:13:26 smithi162 ceph-mon[26841]: from='mgr.14680 172.21.15.162:0/501880828' entity='mgr.smithi162.oennfn' 2024-04-27T04:13:26.888 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:13:26 smithi162 ceph-mon[26841]: from='mgr.14680 172.21.15.162:0/501880828' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T04:13:26.888 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:13:26 smithi162 ceph-mon[26841]: from='mgr.14680 172.21.15.162:0/501880828' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "versions"}]: dispatch 2024-04-27T04:13:26.889 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:13:26 smithi162 ceph-mon[26841]: from='mgr.14680 172.21.15.162:0/501880828' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "versions"}]: dispatch 2024-04-27T04:13:26.889 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:13:26 smithi162 ceph-mon[26841]: from='mgr.14680 172.21.15.162:0/501880828' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "versions"}]: dispatch 2024-04-27T04:13:26.889 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:13:26 smithi162 ceph-mon[26841]: from='mgr.14680 172.21.15.162:0/501880828' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "versions"}]: dispatch 2024-04-27T04:13:26.889 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:13:26 smithi162 ceph-mon[26841]: from='mgr.14680 172.21.15.162:0/501880828' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "versions"}]: dispatch 2024-04-27T04:13:26.889 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:13:26 smithi162 ceph-mon[26841]: from='mgr.14680 172.21.15.162:0/501880828' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "versions"}]: dispatch 2024-04-27T04:13:26.889 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:13:26 smithi162 ceph-mon[26841]: from='mgr.14680 172.21.15.162:0/501880828' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "versions"}]: dispatch 2024-04-27T04:13:26.889 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:13:26 smithi162 ceph-mon[26841]: from='mgr.14680 172.21.15.162:0/501880828' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "versions"}]: dispatch 2024-04-27T04:13:26.889 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:13:26 smithi162 ceph-mon[26841]: from='mgr.14680 172.21.15.162:0/501880828' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "versions"}]: dispatch 2024-04-27T04:13:26.889 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:13:26 smithi162 ceph-mon[26841]: from='mgr.14680 172.21.15.162:0/501880828' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "versions"}]: dispatch 2024-04-27T04:13:26.889 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:13:26 smithi162 ceph-mon[26841]: from='mgr.14680 172.21.15.162:0/501880828' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "versions"}]: dispatch 2024-04-27T04:13:26.889 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:13:26 smithi162 ceph-mon[26841]: from='mgr.14680 172.21.15.162:0/501880828' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "versions"}]: dispatch 2024-04-27T04:13:26.889 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:13:26 smithi162 ceph-mon[26841]: from='mgr.14680 172.21.15.162:0/501880828' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "versions"}]: dispatch 2024-04-27T04:13:26.889 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:13:26 smithi162 ceph-mon[26841]: from='mgr.14680 172.21.15.162:0/501880828' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "versions"}]: dispatch 2024-04-27T04:13:26.889 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:13:26 smithi162 ceph-mon[26841]: Upgrade: Finalizing container_image settings 2024-04-27T04:13:26.890 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:13:26 smithi162 ceph-mon[26841]: from='mgr.14680 172.21.15.162:0/501880828' entity='mgr.smithi162.oennfn' 2024-04-27T04:13:26.890 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:13:26 smithi162 ceph-mon[26841]: from='mgr.14680 172.21.15.162:0/501880828' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mgr"}]: dispatch 2024-04-27T04:13:26.890 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:13:26 smithi162 ceph-mon[26841]: from='mgr.14680 172.21.15.162:0/501880828' entity='mgr.smithi162.oennfn' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mgr"}]': finished 2024-04-27T04:13:26.890 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:13:26 smithi162 ceph-mon[26841]: from='mgr.14680 172.21.15.162:0/501880828' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-04-27T04:13:26.890 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:13:26 smithi162 ceph-mon[26841]: from='mgr.14680 172.21.15.162:0/501880828' entity='mgr.smithi162.oennfn' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mon"}]': finished 2024-04-27T04:13:26.890 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:13:26 smithi162 ceph-mon[26841]: from='mgr.14680 172.21.15.162:0/501880828' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.crash"}]: dispatch 2024-04-27T04:13:26.890 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:13:26 smithi162 ceph-mon[26841]: from='mgr.14680 172.21.15.162:0/501880828' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd"}]: dispatch 2024-04-27T04:13:26.890 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:13:26 smithi162 ceph-mon[26841]: from='mgr.14680 172.21.15.162:0/501880828' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mds"}]: dispatch 2024-04-27T04:13:26.890 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:13:26 smithi162 ceph-mon[26841]: from='mgr.14680 172.21.15.162:0/501880828' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.rgw"}]: dispatch 2024-04-27T04:13:26.890 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:13:26 smithi162 ceph-mon[26841]: from='mgr.14680 172.21.15.162:0/501880828' entity='mgr.smithi162.oennfn' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.rgw"}]': finished 2024-04-27T04:13:26.890 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:13:26 smithi162 ceph-mon[26841]: from='mgr.14680 172.21.15.162:0/501880828' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.rbd-mirror"}]: dispatch 2024-04-27T04:13:26.890 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:13:26 smithi162 ceph-mon[26841]: from='mgr.14680 172.21.15.162:0/501880828' entity='mgr.smithi162.oennfn' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.rbd-mirror"}]': finished 2024-04-27T04:13:26.890 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:13:26 smithi162 ceph-mon[26841]: from='mgr.14680 172.21.15.162:0/501880828' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-04-27T04:13:26.890 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:13:26 smithi162 ceph-mon[26841]: from='mgr.14680 172.21.15.162:0/501880828' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.ceph-exporter"}]: dispatch 2024-04-27T04:13:26.891 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:13:26 smithi162 ceph-mon[26841]: from='mgr.14680 172.21.15.162:0/501880828' entity='mgr.smithi162.oennfn' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.ceph-exporter"}]': finished 2024-04-27T04:13:26.891 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:13:26 smithi162 ceph-mon[26841]: from='mgr.14680 172.21.15.162:0/501880828' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.iscsi"}]: dispatch 2024-04-27T04:13:26.891 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:13:26 smithi162 ceph-mon[26841]: from='mgr.14680 172.21.15.162:0/501880828' entity='mgr.smithi162.oennfn' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.iscsi"}]': finished 2024-04-27T04:13:26.891 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:13:26 smithi162 ceph-mon[26841]: from='mgr.14680 172.21.15.162:0/501880828' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.nfs"}]: dispatch 2024-04-27T04:13:26.891 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:13:26 smithi162 ceph-mon[26841]: from='mgr.14680 172.21.15.162:0/501880828' entity='mgr.smithi162.oennfn' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.nfs"}]': finished 2024-04-27T04:13:26.891 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:13:26 smithi162 ceph-mon[26841]: from='mgr.14680 172.21.15.162:0/501880828' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.nvmeof"}]: dispatch 2024-04-27T04:13:26.891 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:13:26 smithi162 ceph-mon[26841]: from='mgr.14680 172.21.15.162:0/501880828' entity='mgr.smithi162.oennfn' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.nvmeof"}]': finished 2024-04-27T04:13:26.891 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:13:26 smithi162 ceph-mon[26841]: from='mgr.14680 172.21.15.162:0/501880828' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-04-27T04:13:26.891 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:13:26 smithi162 ceph-mon[26841]: from='mgr.14680 172.21.15.162:0/501880828' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-04-27T04:13:26.891 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:13:26 smithi162 ceph-mon[26841]: from='mgr.14680 172.21.15.162:0/501880828' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-04-27T04:13:26.892 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:13:26 smithi162 ceph-mon[26841]: from='mgr.14680 172.21.15.162:0/501880828' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-04-27T04:13:26.892 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:13:26 smithi162 ceph-mon[26841]: from='mgr.14680 172.21.15.162:0/501880828' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-04-27T04:13:26.892 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:13:26 smithi162 ceph-mon[26841]: from='mgr.14680 172.21.15.162:0/501880828' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-04-27T04:13:26.892 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:13:26 smithi162 ceph-mon[26841]: Upgrade: Complete! 2024-04-27T04:13:26.892 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:13:26 smithi162 ceph-mon[26841]: from='mgr.14680 172.21.15.162:0/501880828' entity='mgr.smithi162.oennfn' cmd=[{"prefix":"config-key del","key":"mgr/cephadm/upgrade_state"}]: dispatch 2024-04-27T04:13:26.892 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:13:26 smithi162 ceph-mon[26841]: from='mgr.14680 172.21.15.162:0/501880828' entity='mgr.smithi162.oennfn' cmd='[{"prefix":"config-key del","key":"mgr/cephadm/upgrade_state"}]': finished 2024-04-27T04:13:26.892 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:13:26 smithi162 ceph-mon[26841]: from='mgr.14680 172.21.15.162:0/501880828' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T04:13:26.892 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:13:26 smithi162 ceph-mon[26841]: from='mgr.14680 172.21.15.162:0/501880828' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T04:13:26.892 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:13:26 smithi162 ceph-mon[26841]: from='mgr.14680 172.21.15.162:0/501880828' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T04:13:26.892 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:13:26 smithi162 ceph-mon[26841]: from='mgr.14680 172.21.15.162:0/501880828' entity='mgr.smithi162.oennfn' 2024-04-27T04:13:26.892 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:13:26 smithi162 ceph-mon[26841]: from='mgr.14680 172.21.15.162:0/501880828' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T04:13:26.892 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:13:26 smithi162 ceph-mon[26841]: from='mgr.14680 172.21.15.162:0/501880828' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T04:13:26.892 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:13:26 smithi162 ceph-mon[26841]: from='mgr.14680 172.21.15.162:0/501880828' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T04:13:26.892 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:13:26 smithi162 ceph-mon[26841]: from='mgr.14680 172.21.15.162:0/501880828' entity='mgr.smithi162.oennfn' 2024-04-27T04:13:26.923 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:13:26 smithi181 ceph-mon[33774]: from='mgr.14680 172.21.15.162:0/501880828' entity='mgr.smithi162.oennfn' 2024-04-27T04:13:26.923 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:13:26 smithi181 ceph-mon[33774]: from='mgr.14680 172.21.15.162:0/501880828' entity='mgr.smithi162.oennfn' 2024-04-27T04:13:26.923 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:13:26 smithi181 ceph-mon[33774]: from='mgr.14680 172.21.15.162:0/501880828' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T04:13:26.923 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:13:26 smithi181 ceph-mon[33774]: from='mgr.14680 172.21.15.162:0/501880828' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T04:13:26.923 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:13:26 smithi181 ceph-mon[33774]: from='mgr.14680 172.21.15.162:0/501880828' entity='mgr.smithi162.oennfn' 2024-04-27T04:13:26.923 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:13:26 smithi181 ceph-mon[33774]: from='mgr.14680 172.21.15.162:0/501880828' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-04-27T04:13:26.923 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:13:26 smithi181 ceph-mon[33774]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-04-27T04:13:26.923 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:13:26 smithi181 ceph-mon[33774]: from='mgr.14680 172.21.15.162:0/501880828' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "dashboard set-grafana-api-url", "value": "https://smithi162:3000"}]: dispatch 2024-04-27T04:13:26.923 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:13:26 smithi181 ceph-mon[33774]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-grafana-api-url", "value": "https://smithi162:3000"}]: dispatch 2024-04-27T04:13:26.923 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:13:26 smithi181 ceph-mon[33774]: from='mgr.14680 172.21.15.162:0/501880828' entity='mgr.smithi162.oennfn' 2024-04-27T04:13:26.924 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:13:26 smithi181 ceph-mon[33774]: from='mgr.14680 172.21.15.162:0/501880828' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T04:13:26.924 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:13:26 smithi181 ceph-mon[33774]: from='mgr.14680 172.21.15.162:0/501880828' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "versions"}]: dispatch 2024-04-27T04:13:26.924 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:13:26 smithi181 ceph-mon[33774]: from='mgr.14680 172.21.15.162:0/501880828' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "versions"}]: dispatch 2024-04-27T04:13:26.924 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:13:26 smithi181 ceph-mon[33774]: from='mgr.14680 172.21.15.162:0/501880828' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "versions"}]: dispatch 2024-04-27T04:13:26.924 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:13:26 smithi181 ceph-mon[33774]: from='mgr.14680 172.21.15.162:0/501880828' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "versions"}]: dispatch 2024-04-27T04:13:26.924 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:13:26 smithi181 ceph-mon[33774]: from='mgr.14680 172.21.15.162:0/501880828' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "versions"}]: dispatch 2024-04-27T04:13:26.924 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:13:26 smithi181 ceph-mon[33774]: from='mgr.14680 172.21.15.162:0/501880828' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "versions"}]: dispatch 2024-04-27T04:13:26.924 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:13:26 smithi181 ceph-mon[33774]: from='mgr.14680 172.21.15.162:0/501880828' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "versions"}]: dispatch 2024-04-27T04:13:26.924 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:13:26 smithi181 ceph-mon[33774]: from='mgr.14680 172.21.15.162:0/501880828' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "versions"}]: dispatch 2024-04-27T04:13:26.924 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:13:26 smithi181 ceph-mon[33774]: from='mgr.14680 172.21.15.162:0/501880828' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "versions"}]: dispatch 2024-04-27T04:13:26.924 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:13:26 smithi181 ceph-mon[33774]: from='mgr.14680 172.21.15.162:0/501880828' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "versions"}]: dispatch 2024-04-27T04:13:26.924 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:13:26 smithi181 ceph-mon[33774]: from='mgr.14680 172.21.15.162:0/501880828' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "versions"}]: dispatch 2024-04-27T04:13:26.924 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:13:26 smithi181 ceph-mon[33774]: from='mgr.14680 172.21.15.162:0/501880828' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "versions"}]: dispatch 2024-04-27T04:13:26.924 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:13:26 smithi181 ceph-mon[33774]: from='mgr.14680 172.21.15.162:0/501880828' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "versions"}]: dispatch 2024-04-27T04:13:26.925 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:13:26 smithi181 ceph-mon[33774]: from='mgr.14680 172.21.15.162:0/501880828' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "versions"}]: dispatch 2024-04-27T04:13:26.925 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:13:26 smithi181 ceph-mon[33774]: Upgrade: Finalizing container_image settings 2024-04-27T04:13:26.925 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:13:26 smithi181 ceph-mon[33774]: from='mgr.14680 172.21.15.162:0/501880828' entity='mgr.smithi162.oennfn' 2024-04-27T04:13:26.925 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:13:26 smithi181 ceph-mon[33774]: from='mgr.14680 172.21.15.162:0/501880828' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mgr"}]: dispatch 2024-04-27T04:13:26.925 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:13:26 smithi181 ceph-mon[33774]: from='mgr.14680 172.21.15.162:0/501880828' entity='mgr.smithi162.oennfn' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mgr"}]': finished 2024-04-27T04:13:26.925 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:13:26 smithi181 ceph-mon[33774]: from='mgr.14680 172.21.15.162:0/501880828' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-04-27T04:13:26.925 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:13:26 smithi181 ceph-mon[33774]: from='mgr.14680 172.21.15.162:0/501880828' entity='mgr.smithi162.oennfn' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mon"}]': finished 2024-04-27T04:13:26.925 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:13:26 smithi181 ceph-mon[33774]: from='mgr.14680 172.21.15.162:0/501880828' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.crash"}]: dispatch 2024-04-27T04:13:26.925 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:13:26 smithi181 ceph-mon[33774]: from='mgr.14680 172.21.15.162:0/501880828' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd"}]: dispatch 2024-04-27T04:13:26.925 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:13:26 smithi181 ceph-mon[33774]: from='mgr.14680 172.21.15.162:0/501880828' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mds"}]: dispatch 2024-04-27T04:13:26.925 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:13:26 smithi181 ceph-mon[33774]: from='mgr.14680 172.21.15.162:0/501880828' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.rgw"}]: dispatch 2024-04-27T04:13:26.925 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:13:26 smithi181 ceph-mon[33774]: from='mgr.14680 172.21.15.162:0/501880828' entity='mgr.smithi162.oennfn' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.rgw"}]': finished 2024-04-27T04:13:26.925 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:13:26 smithi181 ceph-mon[33774]: from='mgr.14680 172.21.15.162:0/501880828' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.rbd-mirror"}]: dispatch 2024-04-27T04:13:26.925 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:13:26 smithi181 ceph-mon[33774]: from='mgr.14680 172.21.15.162:0/501880828' entity='mgr.smithi162.oennfn' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.rbd-mirror"}]': finished 2024-04-27T04:13:26.926 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:13:26 smithi181 ceph-mon[33774]: from='mgr.14680 172.21.15.162:0/501880828' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-04-27T04:13:26.926 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:13:26 smithi181 ceph-mon[33774]: from='mgr.14680 172.21.15.162:0/501880828' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.ceph-exporter"}]: dispatch 2024-04-27T04:13:26.926 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:13:26 smithi181 ceph-mon[33774]: from='mgr.14680 172.21.15.162:0/501880828' entity='mgr.smithi162.oennfn' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.ceph-exporter"}]': finished 2024-04-27T04:13:26.926 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:13:26 smithi181 ceph-mon[33774]: from='mgr.14680 172.21.15.162:0/501880828' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.iscsi"}]: dispatch 2024-04-27T04:13:26.926 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:13:26 smithi181 ceph-mon[33774]: from='mgr.14680 172.21.15.162:0/501880828' entity='mgr.smithi162.oennfn' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.iscsi"}]': finished 2024-04-27T04:13:26.926 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:13:26 smithi181 ceph-mon[33774]: from='mgr.14680 172.21.15.162:0/501880828' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.nfs"}]: dispatch 2024-04-27T04:13:26.926 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:13:26 smithi181 ceph-mon[33774]: from='mgr.14680 172.21.15.162:0/501880828' entity='mgr.smithi162.oennfn' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.nfs"}]': finished 2024-04-27T04:13:26.926 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:13:26 smithi181 ceph-mon[33774]: from='mgr.14680 172.21.15.162:0/501880828' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.nvmeof"}]: dispatch 2024-04-27T04:13:26.926 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:13:26 smithi181 ceph-mon[33774]: from='mgr.14680 172.21.15.162:0/501880828' entity='mgr.smithi162.oennfn' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.nvmeof"}]': finished 2024-04-27T04:13:26.926 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:13:26 smithi181 ceph-mon[33774]: from='mgr.14680 172.21.15.162:0/501880828' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-04-27T04:13:26.926 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:13:26 smithi181 ceph-mon[33774]: from='mgr.14680 172.21.15.162:0/501880828' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-04-27T04:13:26.926 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:13:26 smithi181 ceph-mon[33774]: from='mgr.14680 172.21.15.162:0/501880828' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-04-27T04:13:26.926 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:13:26 smithi181 ceph-mon[33774]: from='mgr.14680 172.21.15.162:0/501880828' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-04-27T04:13:26.927 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:13:26 smithi181 ceph-mon[33774]: from='mgr.14680 172.21.15.162:0/501880828' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-04-27T04:13:26.927 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:13:26 smithi181 ceph-mon[33774]: from='mgr.14680 172.21.15.162:0/501880828' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-04-27T04:13:26.927 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:13:26 smithi181 ceph-mon[33774]: Upgrade: Complete! 2024-04-27T04:13:26.927 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:13:26 smithi181 ceph-mon[33774]: from='mgr.14680 172.21.15.162:0/501880828' entity='mgr.smithi162.oennfn' cmd=[{"prefix":"config-key del","key":"mgr/cephadm/upgrade_state"}]: dispatch 2024-04-27T04:13:26.927 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:13:26 smithi181 ceph-mon[33774]: from='mgr.14680 172.21.15.162:0/501880828' entity='mgr.smithi162.oennfn' cmd='[{"prefix":"config-key del","key":"mgr/cephadm/upgrade_state"}]': finished 2024-04-27T04:13:26.927 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:13:26 smithi181 ceph-mon[33774]: from='mgr.14680 172.21.15.162:0/501880828' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T04:13:26.927 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:13:26 smithi181 ceph-mon[33774]: from='mgr.14680 172.21.15.162:0/501880828' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T04:13:26.927 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:13:26 smithi181 ceph-mon[33774]: from='mgr.14680 172.21.15.162:0/501880828' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T04:13:26.927 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:13:26 smithi181 ceph-mon[33774]: from='mgr.14680 172.21.15.162:0/501880828' entity='mgr.smithi162.oennfn' 2024-04-27T04:13:26.927 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:13:26 smithi181 ceph-mon[33774]: from='mgr.14680 172.21.15.162:0/501880828' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T04:13:26.927 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:13:26 smithi181 ceph-mon[33774]: from='mgr.14680 172.21.15.162:0/501880828' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T04:13:26.927 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:13:26 smithi181 ceph-mon[33774]: from='mgr.14680 172.21.15.162:0/501880828' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T04:13:26.927 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:13:26 smithi181 ceph-mon[33774]: from='mgr.14680 172.21.15.162:0/501880828' entity='mgr.smithi162.oennfn' 2024-04-27T04:13:28.137 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:13:27 smithi162 ceph-mon[26841]: pgmap v60: 65 pgs: 65 active+clean; 208 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:13:28.137 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:13:27 smithi162 ceph-mon[26841]: from='mgr.14680 172.21.15.162:0/501880828' entity='mgr.smithi162.oennfn' 2024-04-27T04:13:28.172 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:13:27 smithi181 ceph-mon[33774]: pgmap v60: 65 pgs: 65 active+clean; 208 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:13:28.172 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:13:27 smithi181 ceph-mon[33774]: from='mgr.14680 172.21.15.162:0/501880828' entity='mgr.smithi162.oennfn' 2024-04-27T04:13:29.485 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:13:29.487+0000 7ff96e398700 1 -- 172.21.15.162:0/1067965369 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff9680fdfd0 msgr2=0x7ff9680fe440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:13:29.486 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:13:29.487+0000 7ff96e398700 1 --2- 172.21.15.162:0/1067965369 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff9680fdfd0 0x7ff9680fe440 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7ff95c004940 tx=0x7ff95c009e30 comp rx=0 tx=0).stop 2024-04-27T04:13:29.486 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:13:29.487+0000 7ff96e398700 1 -- 172.21.15.162:0/1067965369 shutdown_connections 2024-04-27T04:13:29.486 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:13:29.487+0000 7ff96e398700 1 --2- 172.21.15.162:0/1067965369 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff9680fdfd0 0x7ff9680fe440 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:13:29.486 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:13:29.487+0000 7ff96e398700 1 --2- 172.21.15.162:0/1067965369 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff9680fce60 0x7ff9680fd270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:13:29.486 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:13:29.487+0000 7ff96e398700 1 -- 172.21.15.162:0/1067965369 >> 172.21.15.162:0/1067965369 conn(0x7ff9680f8520 msgr2=0x7ff9680fa950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:13:29.486 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:13:29.487+0000 7ff96e398700 1 -- 172.21.15.162:0/1067965369 shutdown_connections 2024-04-27T04:13:29.486 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:13:29.487+0000 7ff96e398700 1 -- 172.21.15.162:0/1067965369 wait complete. 2024-04-27T04:13:29.486 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:13:29.487+0000 7ff96e398700 1 Processor -- start 2024-04-27T04:13:29.488 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:13:29.488+0000 7ff96e398700 1 -- start start 2024-04-27T04:13:29.488 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:13:29.488+0000 7ff96e398700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff9680fce60 0x7ff96810d8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:13:29.488 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:13:29.488+0000 7ff96e398700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff9680fdfd0 0x7ff96810de30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:13:29.488 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:13:29.488+0000 7ff96e398700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff96810e430 con 0x7ff9680fce60 2024-04-27T04:13:29.488 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:13:29.488+0000 7ff96e398700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff96810e5a0 con 0x7ff9680fdfd0 2024-04-27T04:13:29.488 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:13:29.488+0000 7ff967fff700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff9680fce60 0x7ff96810d8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:13:29.488 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:13:29.488+0000 7ff967fff700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff9680fce60 0x7ff96810d8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:34920/0 (socket says 172.21.15.162:34920) 2024-04-27T04:13:29.488 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:13:29.488+0000 7ff967fff700 1 -- 172.21.15.162:0/3059571515 learned_addr learned my addr 172.21.15.162:0/3059571515 (peer_addr_for_me v2:172.21.15.162:0/0) 2024-04-27T04:13:29.488 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:13:29.488+0000 7ff9677fe700 1 --2- 172.21.15.162:0/3059571515 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff9680fdfd0 0x7ff96810de30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:13:29.488 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:13:29.488+0000 7ff967fff700 1 -- 172.21.15.162:0/3059571515 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff9680fdfd0 msgr2=0x7ff96810de30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:13:29.488 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:13:29.488+0000 7ff967fff700 1 --2- 172.21.15.162:0/3059571515 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff9680fdfd0 0x7ff96810de30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:13:29.489 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:13:29.488+0000 7ff967fff700 1 -- 172.21.15.162:0/3059571515 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff95c005040 con 0x7ff9680fce60 2024-04-27T04:13:29.489 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:13:29.489+0000 7ff967fff700 1 --2- 172.21.15.162:0/3059571515 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff9680fce60 0x7ff96810d8f0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7ff95800c830 tx=0x7ff958004470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:13:29.489 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:13:29.489+0000 7ff964ff9700 1 -- 172.21.15.162:0/3059571515 <== mon.0 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff9580106f0 con 0x7ff9680fce60 2024-04-27T04:13:29.489 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:13:29.489+0000 7ff96e398700 1 -- 172.21.15.162:0/3059571515 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff968112fa0 con 0x7ff9680fce60 2024-04-27T04:13:29.489 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:13:29.489+0000 7ff964ff9700 1 -- 172.21.15.162:0/3059571515 <== mon.0 v2:172.21.15.162:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7ff958010d20 con 0x7ff9680fce60 2024-04-27T04:13:29.489 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:13:29.489+0000 7ff964ff9700 1 -- 172.21.15.162:0/3059571515 <== mon.0 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff9580113a0 con 0x7ff9680fce60 2024-04-27T04:13:29.489 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:13:29.489+0000 7ff96e398700 1 -- 172.21.15.162:0/3059571515 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff968113530 con 0x7ff9680fce60 2024-04-27T04:13:29.490 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:13:29.490+0000 7ff96e398700 1 -- 172.21.15.162:0/3059571515 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff96804c710 con 0x7ff9680fce60 2024-04-27T04:13:29.490 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:13:29.491+0000 7ff964ff9700 1 -- 172.21.15.162:0/3059571515 <== mon.0 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 32) v1 ==== 92469+0+0 (secure 0 0 0) 0x7ff958012400 con 0x7ff9680fce60 2024-04-27T04:13:29.490 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:13:29.491+0000 7ff964ff9700 1 --2- 172.21.15.162:0/3059571515 >> [v2:172.21.15.162:6800/901992993,v1:172.21.15.162:6801/901992993] conn(0x7ff95006e500 0x7ff9500709b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:13:29.490 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:13:29.491+0000 7ff964ff9700 1 -- 172.21.15.162:0/3059571515 <== mon.0 v2:172.21.15.162:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6152+0+0 (secure 0 0 0) 0x7ff95808ded0 con 0x7ff9680fce60 2024-04-27T04:13:29.490 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:13:29.491+0000 7ff9677fe700 1 --2- 172.21.15.162:0/3059571515 >> [v2:172.21.15.162:6800/901992993,v1:172.21.15.162:6801/901992993] conn(0x7ff95006e500 0x7ff9500709b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:13:29.490 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:13:29.492+0000 7ff9677fe700 1 --2- 172.21.15.162:0/3059571515 >> [v2:172.21.15.162:6800/901992993,v1:172.21.15.162:6801/901992993] conn(0x7ff95006e500 0x7ff9500709b0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7ff95c00b970 tx=0x7ff95c009e30 comp rx=0 tx=0).ready entity=mgr.14680 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:13:29.494 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:13:29.496+0000 7ff964ff9700 1 -- 172.21.15.162:0/3059571515 <== mon.0 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186652 (secure 0 0 0) 0x7ff95805b080 con 0x7ff9680fce60 2024-04-27T04:13:29.712 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:13:29.714+0000 7ff96e398700 1 -- 172.21.15.162:0/3059571515 --> [v2:172.21.15.162:6800/901992993,v1:172.21.15.162:6801/901992993] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7ff968102380 con 0x7ff95006e500 2024-04-27T04:13:29.713 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:13:29.715+0000 7ff964ff9700 1 -- 172.21.15.162:0/3059571515 <== mgr.14680 v2:172.21.15.162:6800/901992993 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+175 (secure 0 0 0) 0x7ff968102380 con 0x7ff95006e500 2024-04-27T04:13:29.716 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:13:29.717+0000 7ff96e398700 1 -- 172.21.15.162:0/3059571515 >> [v2:172.21.15.162:6800/901992993,v1:172.21.15.162:6801/901992993] conn(0x7ff95006e500 msgr2=0x7ff9500709b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:13:29.716 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:13:29.717+0000 7ff96e398700 1 --2- 172.21.15.162:0/3059571515 >> [v2:172.21.15.162:6800/901992993,v1:172.21.15.162:6801/901992993] conn(0x7ff95006e500 0x7ff9500709b0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7ff95c00b970 tx=0x7ff95c009e30 comp rx=0 tx=0).stop 2024-04-27T04:13:29.716 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:13:29.717+0000 7ff96e398700 1 -- 172.21.15.162:0/3059571515 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff9680fce60 msgr2=0x7ff96810d8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:13:29.716 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:13:29.717+0000 7ff96e398700 1 --2- 172.21.15.162:0/3059571515 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff9680fce60 0x7ff96810d8f0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7ff95800c830 tx=0x7ff958004470 comp rx=0 tx=0).stop 2024-04-27T04:13:29.716 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:13:29.717+0000 7ff96e398700 1 -- 172.21.15.162:0/3059571515 shutdown_connections 2024-04-27T04:13:29.716 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:13:29.717+0000 7ff96e398700 1 --2- 172.21.15.162:0/3059571515 >> [v2:172.21.15.162:6800/901992993,v1:172.21.15.162:6801/901992993] conn(0x7ff95006e500 0x7ff9500709b0 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:13:29.716 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:13:29.717+0000 7ff96e398700 1 --2- 172.21.15.162:0/3059571515 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff9680fce60 0x7ff96810d8f0 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:13:29.716 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:13:29.717+0000 7ff96e398700 1 --2- 172.21.15.162:0/3059571515 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff9680fdfd0 0x7ff96810de30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:13:29.716 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:13:29.717+0000 7ff96e398700 1 -- 172.21.15.162:0/3059571515 >> 172.21.15.162:0/3059571515 conn(0x7ff9680f8520 msgr2=0x7ff9680f9c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:13:29.716 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:13:29.717+0000 7ff96e398700 1 -- 172.21.15.162:0/3059571515 shutdown_connections 2024-04-27T04:13:29.716 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:13:29.717+0000 7ff96e398700 1 -- 172.21.15.162:0/3059571515 wait complete. 2024-04-27T04:13:29.997 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:13:29 smithi162 ceph-mon[26841]: pgmap v61: 65 pgs: 65 active+clean; 208 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-04-27T04:13:30.172 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:13:29 smithi181 ceph-mon[33774]: pgmap v61: 65 pgs: 65 active+clean; 208 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-04-27T04:13:30.384 DEBUG:teuthology.orchestra.run.smithi162:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 7f9750d6-044a-11ef-bc93-c7b262605968 -e sha1=4e213a58e787437ff508e6c890ccd5fd32dc96b5 -- bash -c 'ceph versions | jq -e '"'"'.mgr | length == 1'"'"'' 2024-04-27T04:13:30.684 INFO:teuthology.orchestra.run.smithi162.stderr:Inferring config /var/lib/ceph/7f9750d6-044a-11ef-bc93-c7b262605968/mon.smithi162/config 2024-04-27T04:13:31.070 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:13:30 smithi162 ceph-mon[26841]: from='client.14790 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-27T04:13:31.172 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:13:30 smithi181 ceph-mon[33774]: from='client.14790 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-27T04:13:32.122 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:13:31 smithi162 ceph-mon[26841]: pgmap v62: 65 pgs: 65 active+clean; 208 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:13:32.172 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:13:31 smithi181 ceph-mon[33774]: pgmap v62: 65 pgs: 65 active+clean; 208 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:13:32.347 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:13:32.347+0000 7f43f56a5700 1 -- 172.21.15.162:0/1997270806 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f43f0101d90 msgr2=0x7f43f0102160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:13:32.347 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:13:32.347+0000 7f43f56a5700 1 --2- 172.21.15.162:0/1997270806 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f43f0101d90 0x7f43f0102160 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f43e0004940 tx=0x7f43e0009e30 comp rx=0 tx=0).stop 2024-04-27T04:13:32.347 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:13:32.347+0000 7f43f56a5700 1 -- 172.21.15.162:0/1997270806 shutdown_connections 2024-04-27T04:13:32.347 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:13:32.347+0000 7f43f56a5700 1 --2- 172.21.15.162:0/1997270806 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f43f00fbd40 0x7f43f00fc1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:13:32.347 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:13:32.347+0000 7f43f56a5700 1 --2- 172.21.15.162:0/1997270806 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f43f0101d90 0x7f43f0102160 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:13:32.347 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:13:32.347+0000 7f43f56a5700 1 -- 172.21.15.162:0/1997270806 >> 172.21.15.162:0/1997270806 conn(0x7f43f0071d00 msgr2=0x7f43f0072100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:13:32.347 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:13:32.347+0000 7f43f56a5700 1 -- 172.21.15.162:0/1997270806 shutdown_connections 2024-04-27T04:13:32.347 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:13:32.347+0000 7f43f56a5700 1 -- 172.21.15.162:0/1997270806 wait complete. 2024-04-27T04:13:32.347 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:13:32.347+0000 7f43f56a5700 1 Processor -- start 2024-04-27T04:13:32.348 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:13:32.348+0000 7f43f56a5700 1 -- start start 2024-04-27T04:13:32.348 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:13:32.348+0000 7f43f56a5700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f43f00fbd40 0x7f43f010ec80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:13:32.348 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:13:32.348+0000 7f43f56a5700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f43f0101d90 0x7f43f010f1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:13:32.348 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:13:32.348+0000 7f43f56a5700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43f010f790 con 0x7f43f0101d90 2024-04-27T04:13:32.348 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:13:32.348+0000 7f43f56a5700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43f010f8d0 con 0x7f43f00fbd40 2024-04-27T04:13:32.348 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:13:32.348+0000 7f43ee7fc700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f43f0101d90 0x7f43f010f1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:13:32.348 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:13:32.348+0000 7f43ee7fc700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f43f0101d90 0x7f43f010f1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:34936/0 (socket says 172.21.15.162:34936) 2024-04-27T04:13:32.348 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:13:32.348+0000 7f43ee7fc700 1 -- 172.21.15.162:0/1902762312 learned_addr learned my addr 172.21.15.162:0/1902762312 (peer_addr_for_me v2:172.21.15.162:0/0) 2024-04-27T04:13:32.348 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:13:32.348+0000 7f43eeffd700 1 --2- 172.21.15.162:0/1902762312 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f43f00fbd40 0x7f43f010ec80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:13:32.348 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:13:32.348+0000 7f43ee7fc700 1 -- 172.21.15.162:0/1902762312 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f43f00fbd40 msgr2=0x7f43f010ec80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:13:32.348 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:13:32.348+0000 7f43ee7fc700 1 --2- 172.21.15.162:0/1902762312 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f43f00fbd40 0x7f43f010ec80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:13:32.348 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:13:32.348+0000 7f43ee7fc700 1 -- 172.21.15.162:0/1902762312 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43e0005040 con 0x7f43f0101d90 2024-04-27T04:13:32.348 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:13:32.348+0000 7f43ee7fc700 1 --2- 172.21.15.162:0/1902762312 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f43f0101d90 0x7f43f010f1c0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f43e400b790 tx=0x7f43e4009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:13:32.348 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:13:32.348+0000 7f43d77fe700 1 -- 172.21.15.162:0/1902762312 <== mon.0 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f43e400f770 con 0x7f43f0101d90 2024-04-27T04:13:32.349 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:13:32.349+0000 7f43d77fe700 1 -- 172.21.15.162:0/1902762312 <== mon.0 v2:172.21.15.162:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f43e400fda0 con 0x7f43f0101d90 2024-04-27T04:13:32.349 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:13:32.349+0000 7f43f56a5700 1 -- 172.21.15.162:0/1902762312 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f43f0117bf0 con 0x7f43f0101d90 2024-04-27T04:13:32.349 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:13:32.349+0000 7f43d77fe700 1 -- 172.21.15.162:0/1902762312 <== mon.0 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f43e400d310 con 0x7f43f0101d90 2024-04-27T04:13:32.349 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:13:32.349+0000 7f43f56a5700 1 -- 172.21.15.162:0/1902762312 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f43f0118180 con 0x7f43f0101d90 2024-04-27T04:13:32.350 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:13:32.349+0000 7f43f56a5700 1 -- 172.21.15.162:0/1902762312 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f43f00fd430 con 0x7f43f0101d90 2024-04-27T04:13:32.350 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:13:32.350+0000 7f43d77fe700 1 -- 172.21.15.162:0/1902762312 <== mon.0 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 32) v1 ==== 92469+0+0 (secure 0 0 0) 0x7f43e4010420 con 0x7f43f0101d90 2024-04-27T04:13:32.350 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:13:32.351+0000 7f43d77fe700 1 --2- 172.21.15.162:0/1902762312 >> [v2:172.21.15.162:6800/901992993,v1:172.21.15.162:6801/901992993] conn(0x7f43d806e500 0x7f43d80709b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:13:32.350 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:13:32.351+0000 7f43eeffd700 1 --2- 172.21.15.162:0/1902762312 >> [v2:172.21.15.162:6800/901992993,v1:172.21.15.162:6801/901992993] conn(0x7f43d806e500 0x7f43d80709b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:13:32.350 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:13:32.351+0000 7f43d77fe700 1 -- 172.21.15.162:0/1902762312 <== mon.0 v2:172.21.15.162:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6152+0+0 (secure 0 0 0) 0x7f43e408df70 con 0x7f43f0101d90 2024-04-27T04:13:32.350 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:13:32.351+0000 7f43eeffd700 1 --2- 172.21.15.162:0/1902762312 >> [v2:172.21.15.162:6800/901992993,v1:172.21.15.162:6801/901992993] conn(0x7f43d806e500 0x7f43d80709b0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f43e000b3b0 tx=0x7f43e0009e30 comp rx=0 tx=0).ready entity=mgr.14680 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:13:32.353 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:13:32.355+0000 7f43d77fe700 1 -- 172.21.15.162:0/1902762312 <== mon.0 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186652 (secure 0 0 0) 0x7f43e40c2930 con 0x7f43f0101d90 2024-04-27T04:13:32.640 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:13:32.641+0000 7f43f56a5700 1 -- 172.21.15.162:0/1902762312 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f43f005f5c0 con 0x7f43f0101d90 2024-04-27T04:13:32.640 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:13:32.642+0000 7f43d77fe700 1 -- 172.21.15.162:0/1902762312 <== mon.0 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+739 (secure 0 0 0) 0x7f43e40575f0 con 0x7f43f0101d90 2024-04-27T04:13:32.642 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:13:32.644+0000 7f43f56a5700 1 -- 172.21.15.162:0/1902762312 >> [v2:172.21.15.162:6800/901992993,v1:172.21.15.162:6801/901992993] conn(0x7f43d806e500 msgr2=0x7f43d80709b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:13:32.642 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:13:32.644+0000 7f43f56a5700 1 --2- 172.21.15.162:0/1902762312 >> [v2:172.21.15.162:6800/901992993,v1:172.21.15.162:6801/901992993] conn(0x7f43d806e500 0x7f43d80709b0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f43e000b3b0 tx=0x7f43e0009e30 comp rx=0 tx=0).stop 2024-04-27T04:13:32.643 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:13:32.644+0000 7f43f56a5700 1 -- 172.21.15.162:0/1902762312 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f43f0101d90 msgr2=0x7f43f010f1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:13:32.643 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:13:32.644+0000 7f43f56a5700 1 --2- 172.21.15.162:0/1902762312 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f43f0101d90 0x7f43f010f1c0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f43e400b790 tx=0x7f43e4009250 comp rx=0 tx=0).stop 2024-04-27T04:13:32.643 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:13:32.644+0000 7f43f56a5700 1 -- 172.21.15.162:0/1902762312 shutdown_connections 2024-04-27T04:13:32.643 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:13:32.644+0000 7f43f56a5700 1 --2- 172.21.15.162:0/1902762312 >> [v2:172.21.15.162:6800/901992993,v1:172.21.15.162:6801/901992993] conn(0x7f43d806e500 0x7f43d80709b0 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:13:32.643 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:13:32.644+0000 7f43f56a5700 1 --2- 172.21.15.162:0/1902762312 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f43f00fbd40 0x7f43f010ec80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:13:32.643 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:13:32.644+0000 7f43f56a5700 1 --2- 172.21.15.162:0/1902762312 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f43f0101d90 0x7f43f010f1c0 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:13:32.643 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:13:32.644+0000 7f43f56a5700 1 -- 172.21.15.162:0/1902762312 >> 172.21.15.162:0/1902762312 conn(0x7f43f0071d00 msgr2=0x7f43f00fa120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:13:32.643 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:13:32.644+0000 7f43f56a5700 1 -- 172.21.15.162:0/1902762312 shutdown_connections 2024-04-27T04:13:32.643 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:13:32.644+0000 7f43f56a5700 1 -- 172.21.15.162:0/1902762312 wait complete. 2024-04-27T04:13:32.652 INFO:teuthology.orchestra.run.smithi162.stdout:true 2024-04-27T04:13:32.948 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:13:32 smithi162 ceph-mon[26841]: from='mgr.14680 172.21.15.162:0/501880828' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-27T04:13:32.948 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:13:32 smithi162 ceph-mon[26841]: from='client.? 172.21.15.162:0/1902762312' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-27T04:13:33.172 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:13:32 smithi181 ceph-mon[33774]: from='mgr.14680 172.21.15.162:0/501880828' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-27T04:13:33.172 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:13:32 smithi181 ceph-mon[33774]: from='client.? 172.21.15.162:0/1902762312' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-27T04:13:33.307 DEBUG:teuthology.orchestra.run.smithi162:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 7f9750d6-044a-11ef-bc93-c7b262605968 -e sha1=4e213a58e787437ff508e6c890ccd5fd32dc96b5 -- bash -c 'ceph versions | jq -e '"'"'.mgr | keys'"'"' | grep $sha1' 2024-04-27T04:13:33.567 INFO:teuthology.orchestra.run.smithi162.stderr:Inferring config /var/lib/ceph/7f9750d6-044a-11ef-bc93-c7b262605968/mon.smithi162/config 2024-04-27T04:13:34.026 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:13:33 smithi162 ceph-mon[26841]: pgmap v63: 65 pgs: 65 active+clean; 208 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:13:34.172 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:13:33 smithi181 ceph-mon[33774]: pgmap v63: 65 pgs: 65 active+clean; 208 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:13:35.223 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:13:35.208+0000 7f5652c2e700 1 -- 172.21.15.162:0/1814176129 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f564c101d90 msgr2=0x7f564c102160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:13:35.223 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:13:35.208+0000 7f5652c2e700 1 --2- 172.21.15.162:0/1814176129 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f564c101d90 0x7f564c102160 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f5644009ce0 tx=0x7f5644004d10 comp rx=0 tx=0).stop 2024-04-27T04:13:35.223 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:13:35.208+0000 7f5652c2e700 1 -- 172.21.15.162:0/1814176129 shutdown_connections 2024-04-27T04:13:35.223 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:13:35.208+0000 7f5652c2e700 1 --2- 172.21.15.162:0/1814176129 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f564c102730 0x7f564c0fbda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:13:35.223 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:13:35.208+0000 7f5652c2e700 1 --2- 172.21.15.162:0/1814176129 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f564c101d90 0x7f564c102160 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:13:35.223 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:13:35.208+0000 7f5652c2e700 1 -- 172.21.15.162:0/1814176129 >> 172.21.15.162:0/1814176129 conn(0x7f564c0f7940 msgr2=0x7f564c0f9d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:13:35.224 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:13:35.209+0000 7f5652c2e700 1 -- 172.21.15.162:0/1814176129 shutdown_connections 2024-04-27T04:13:35.224 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:13:35.209+0000 7f5652c2e700 1 -- 172.21.15.162:0/1814176129 wait complete. 2024-04-27T04:13:35.224 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:13:35.209+0000 7f5652c2e700 1 Processor -- start 2024-04-27T04:13:35.224 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:13:35.209+0000 7f5652c2e700 1 -- start start 2024-04-27T04:13:35.224 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:13:35.210+0000 7f5652c2e700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f564c101d90 0x7f564c113a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:13:35.224 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:13:35.210+0000 7f5652c2e700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f564c102730 0x7f564c113f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:13:35.224 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:13:35.210+0000 7f5652c2e700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f564c114670 con 0x7f564c101d90 2024-04-27T04:13:35.224 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:13:35.210+0000 7f5652c2e700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f564c10dc00 con 0x7f564c102730 2024-04-27T04:13:35.224 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:13:35.210+0000 7f56509ca700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f564c101d90 0x7f564c113a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:13:35.224 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:13:35.210+0000 7f56509ca700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f564c101d90 0x7f564c113a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:33782/0 (socket says 172.21.15.162:33782) 2024-04-27T04:13:35.225 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:13:35.210+0000 7f56509ca700 1 -- 172.21.15.162:0/2014234931 learned_addr learned my addr 172.21.15.162:0/2014234931 (peer_addr_for_me v2:172.21.15.162:0/0) 2024-04-27T04:13:35.225 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:13:35.210+0000 7f564bfff700 1 --2- 172.21.15.162:0/2014234931 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f564c102730 0x7f564c113f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:13:35.225 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:13:35.210+0000 7f56509ca700 1 -- 172.21.15.162:0/2014234931 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f564c102730 msgr2=0x7f564c113f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:13:35.225 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:13:35.210+0000 7f56509ca700 1 --2- 172.21.15.162:0/2014234931 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f564c102730 0x7f564c113f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:13:35.225 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:13:35.210+0000 7f56509ca700 1 -- 172.21.15.162:0/2014234931 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5644009940 con 0x7f564c101d90 2024-04-27T04:13:35.225 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:13:35.210+0000 7f56509ca700 1 --2- 172.21.15.162:0/2014234931 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f564c101d90 0x7f564c113a50 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f564400be90 tx=0x7f5644004d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:13:35.225 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:13:35.210+0000 7f56497fa700 1 -- 172.21.15.162:0/2014234931 <== mon.0 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f564400e040 con 0x7f564c101d90 2024-04-27T04:13:35.225 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:13:35.210+0000 7f5652c2e700 1 -- 172.21.15.162:0/2014234931 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f564c10de80 con 0x7f564c101d90 2024-04-27T04:13:35.225 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:13:35.210+0000 7f56497fa700 1 -- 172.21.15.162:0/2014234931 <== mon.0 v2:172.21.15.162:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f5644004610 con 0x7f564c101d90 2024-04-27T04:13:35.225 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:13:35.210+0000 7f56497fa700 1 -- 172.21.15.162:0/2014234931 <== mon.0 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f564401c6d0 con 0x7f564c101d90 2024-04-27T04:13:35.225 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:13:35.211+0000 7f5652c2e700 1 -- 172.21.15.162:0/2014234931 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f564c10e440 con 0x7f564c101d90 2024-04-27T04:13:35.225 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:13:35.212+0000 7f5652c2e700 1 -- 172.21.15.162:0/2014234931 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f564c0fd4e0 con 0x7f564c101d90 2024-04-27T04:13:35.225 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:13:35.212+0000 7f56497fa700 1 -- 172.21.15.162:0/2014234931 <== mon.0 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 32) v1 ==== 92469+0+0 (secure 0 0 0) 0x7f564401c940 con 0x7f564c101d90 2024-04-27T04:13:35.225 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:13:35.212+0000 7f56497fa700 1 --2- 172.21.15.162:0/2014234931 >> [v2:172.21.15.162:6800/901992993,v1:172.21.15.162:6801/901992993] conn(0x7f563806e500 0x7f56380709b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:13:35.226 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:13:35.212+0000 7f56497fa700 1 -- 172.21.15.162:0/2014234931 <== mon.0 v2:172.21.15.162:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6152+0+0 (secure 0 0 0) 0x7f564408fa40 con 0x7f564c101d90 2024-04-27T04:13:35.226 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:13:35.212+0000 7f564bfff700 1 --2- 172.21.15.162:0/2014234931 >> [v2:172.21.15.162:6800/901992993,v1:172.21.15.162:6801/901992993] conn(0x7f563806e500 0x7f56380709b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:13:35.226 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:13:35.213+0000 7f564bfff700 1 --2- 172.21.15.162:0/2014234931 >> [v2:172.21.15.162:6800/901992993,v1:172.21.15.162:6801/901992993] conn(0x7f563806e500 0x7f56380709b0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f564c10f330 tx=0x7f563c0091a0 comp rx=0 tx=0).ready entity=mgr.14680 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:13:35.226 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:13:35.217+0000 7f56497fa700 1 -- 172.21.15.162:0/2014234931 <== mon.0 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186652 (secure 0 0 0) 0x7f5644059040 con 0x7f564c101d90 2024-04-27T04:13:35.499 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:13:35.500+0000 7f5652c2e700 1 -- 172.21.15.162:0/2014234931 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f564c04c710 con 0x7f564c101d90 2024-04-27T04:13:35.499 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:13:35.501+0000 7f56497fa700 1 -- 172.21.15.162:0/2014234931 <== mon.0 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+739 (secure 0 0 0) 0x7f5644004e10 con 0x7f564c101d90 2024-04-27T04:13:35.501 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:13:35.503+0000 7f5652c2e700 1 -- 172.21.15.162:0/2014234931 >> [v2:172.21.15.162:6800/901992993,v1:172.21.15.162:6801/901992993] conn(0x7f563806e500 msgr2=0x7f56380709b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:13:35.501 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:13:35.503+0000 7f5652c2e700 1 --2- 172.21.15.162:0/2014234931 >> [v2:172.21.15.162:6800/901992993,v1:172.21.15.162:6801/901992993] conn(0x7f563806e500 0x7f56380709b0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f564c10f330 tx=0x7f563c0091a0 comp rx=0 tx=0).stop 2024-04-27T04:13:35.501 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:13:35.503+0000 7f5652c2e700 1 -- 172.21.15.162:0/2014234931 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f564c101d90 msgr2=0x7f564c113a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:13:35.501 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:13:35.503+0000 7f5652c2e700 1 --2- 172.21.15.162:0/2014234931 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f564c101d90 0x7f564c113a50 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f564400be90 tx=0x7f5644004d10 comp rx=0 tx=0).stop 2024-04-27T04:13:35.501 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:13:35.503+0000 7f5652c2e700 1 -- 172.21.15.162:0/2014234931 shutdown_connections 2024-04-27T04:13:35.501 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:13:35.503+0000 7f5652c2e700 1 --2- 172.21.15.162:0/2014234931 >> [v2:172.21.15.162:6800/901992993,v1:172.21.15.162:6801/901992993] conn(0x7f563806e500 0x7f56380709b0 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:13:35.502 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:13:35.503+0000 7f5652c2e700 1 --2- 172.21.15.162:0/2014234931 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f564c101d90 0x7f564c113a50 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:13:35.502 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:13:35.503+0000 7f5652c2e700 1 --2- 172.21.15.162:0/2014234931 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f564c102730 0x7f564c113f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:13:35.502 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:13:35.503+0000 7f5652c2e700 1 -- 172.21.15.162:0/2014234931 >> 172.21.15.162:0/2014234931 conn(0x7f564c0f7940 msgr2=0x7f564c0664c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:13:35.502 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:13:35.503+0000 7f5652c2e700 1 -- 172.21.15.162:0/2014234931 shutdown_connections 2024-04-27T04:13:35.502 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:13:35.503+0000 7f5652c2e700 1 -- 172.21.15.162:0/2014234931 wait complete. 2024-04-27T04:13:35.511 INFO:teuthology.orchestra.run.smithi162.stdout: "ceph version 19.0.0-3253-g4e213a58 (4e213a58e787437ff508e6c890ccd5fd32dc96b5) squid (dev)" 2024-04-27T04:13:36.071 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:13:35 smithi162 ceph-mon[26841]: pgmap v64: 65 pgs: 65 active+clean; 208 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:13:36.071 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:13:35 smithi162 ceph-mon[26841]: from='client.? 172.21.15.162:0/2014234931' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-27T04:13:36.097 DEBUG:teuthology.orchestra.run.smithi162:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 7f9750d6-044a-11ef-bc93-c7b262605968 -e sha1=4e213a58e787437ff508e6c890ccd5fd32dc96b5 -- bash -c 'ceph versions | jq -e '"'"'.overall | length == 2'"'"'' 2024-04-27T04:13:36.172 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:13:35 smithi181 ceph-mon[33774]: pgmap v64: 65 pgs: 65 active+clean; 208 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:13:36.172 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:13:35 smithi181 ceph-mon[33774]: from='client.? 172.21.15.162:0/2014234931' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-27T04:13:36.356 INFO:teuthology.orchestra.run.smithi162.stderr:Inferring config /var/lib/ceph/7f9750d6-044a-11ef-bc93-c7b262605968/mon.smithi162/config 2024-04-27T04:13:38.064 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:13:38.064+0000 7ff181bc3700 1 -- 172.21.15.162:0/3250440661 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff17c0fbd40 msgr2=0x7ff17c0fc1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:13:38.064 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:13:38.064+0000 7ff181bc3700 1 --2- 172.21.15.162:0/3250440661 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff17c0fbd40 0x7ff17c0fc1b0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7ff164004940 tx=0x7ff164009e30 comp rx=0 tx=0).stop 2024-04-27T04:13:38.064 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:13:38.064+0000 7ff181bc3700 1 -- 172.21.15.162:0/3250440661 shutdown_connections 2024-04-27T04:13:38.064 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:13:38.064+0000 7ff181bc3700 1 --2- 172.21.15.162:0/3250440661 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff17c0fbd40 0x7ff17c0fc1b0 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:13:38.064 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:13:38.064+0000 7ff181bc3700 1 --2- 172.21.15.162:0/3250440661 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff17c101d90 0x7ff17c102160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:13:38.065 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:13:38.064+0000 7ff181bc3700 1 -- 172.21.15.162:0/3250440661 >> 172.21.15.162:0/3250440661 conn(0x7ff17c071d00 msgr2=0x7ff17c072100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:13:38.065 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:13:38.064+0000 7ff181bc3700 1 -- 172.21.15.162:0/3250440661 shutdown_connections 2024-04-27T04:13:38.065 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:13:38.064+0000 7ff181bc3700 1 -- 172.21.15.162:0/3250440661 wait complete. 2024-04-27T04:13:38.065 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:13:38.065+0000 7ff181bc3700 1 Processor -- start 2024-04-27T04:13:38.065 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:13:38.065+0000 7ff181bc3700 1 -- start start 2024-04-27T04:13:38.065 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:13:38.065+0000 7ff181bc3700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff17c0fbd40 0x7ff17c112ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:13:38.065 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:13:38.065+0000 7ff181bc3700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff17c101d90 0x7ff17c112ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:13:38.065 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:13:38.065+0000 7ff181bc3700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff17c113640 con 0x7ff17c0fbd40 2024-04-27T04:13:38.065 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:13:38.065+0000 7ff181bc3700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff17c10ca30 con 0x7ff17c101d90 2024-04-27T04:13:38.065 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:13:38.065+0000 7ff17b7fe700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff17c0fbd40 0x7ff17c112ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:13:38.065 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:13:38.065+0000 7ff17b7fe700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff17c0fbd40 0x7ff17c112ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:33802/0 (socket says 172.21.15.162:33802) 2024-04-27T04:13:38.065 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:13:38.065+0000 7ff17b7fe700 1 -- 172.21.15.162:0/4098087047 learned_addr learned my addr 172.21.15.162:0/4098087047 (peer_addr_for_me v2:172.21.15.162:0/0) 2024-04-27T04:13:38.065 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:13:38.065+0000 7ff17affd700 1 --2- 172.21.15.162:0/4098087047 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff17c101d90 0x7ff17c112ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:13:38.065 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:13:38.065+0000 7ff17b7fe700 1 -- 172.21.15.162:0/4098087047 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff17c101d90 msgr2=0x7ff17c112ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:13:38.066 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:13:38.065+0000 7ff17b7fe700 1 --2- 172.21.15.162:0/4098087047 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff17c101d90 0x7ff17c112ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:13:38.066 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:13:38.065+0000 7ff17b7fe700 1 -- 172.21.15.162:0/4098087047 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff164005040 con 0x7ff17c0fbd40 2024-04-27T04:13:38.066 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:13:38.065+0000 7ff17affd700 1 --2- 172.21.15.162:0/4098087047 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff17c101d90 0x7ff17c112ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-27T04:13:38.066 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:13:38.066+0000 7ff17b7fe700 1 --2- 172.21.15.162:0/4098087047 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff17c0fbd40 0x7ff17c112ab0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7ff17400fe10 tx=0x7ff174009f30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:13:38.066 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:13:38.066+0000 7ff173fff700 1 -- 172.21.15.162:0/4098087047 <== mon.0 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff174004020 con 0x7ff17c0fbd40 2024-04-27T04:13:38.066 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:13:38.066+0000 7ff181bc3700 1 -- 172.21.15.162:0/4098087047 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff17c10ccb0 con 0x7ff17c0fbd40 2024-04-27T04:13:38.066 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:13:38.066+0000 7ff173fff700 1 -- 172.21.15.162:0/4098087047 <== mon.0 v2:172.21.15.162:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7ff174007df0 con 0x7ff17c0fbd40 2024-04-27T04:13:38.066 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:13:38.066+0000 7ff173fff700 1 -- 172.21.15.162:0/4098087047 <== mon.0 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff1740126a0 con 0x7ff17c0fbd40 2024-04-27T04:13:38.066 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:13:38.066+0000 7ff181bc3700 1 -- 172.21.15.162:0/4098087047 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff17c10d270 con 0x7ff17c0fbd40 2024-04-27T04:13:38.067 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:13:38.067+0000 7ff181bc3700 1 -- 172.21.15.162:0/4098087047 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff17c0fd430 con 0x7ff17c0fbd40 2024-04-27T04:13:38.067 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:13:38.067+0000 7ff173fff700 1 -- 172.21.15.162:0/4098087047 <== mon.0 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 32) v1 ==== 92469+0+0 (secure 0 0 0) 0x7ff174004180 con 0x7ff17c0fbd40 2024-04-27T04:13:38.067 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:13:38.068+0000 7ff173fff700 1 --2- 172.21.15.162:0/4098087047 >> [v2:172.21.15.162:6800/901992993,v1:172.21.15.162:6801/901992993] conn(0x7ff16806e500 0x7ff1680709b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:13:38.067 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:13:38.068+0000 7ff173fff700 1 -- 172.21.15.162:0/4098087047 <== mon.0 v2:172.21.15.162:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6152+0+0 (secure 0 0 0) 0x7ff174016030 con 0x7ff17c0fbd40 2024-04-27T04:13:38.067 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:13:38.068+0000 7ff17affd700 1 --2- 172.21.15.162:0/4098087047 >> [v2:172.21.15.162:6800/901992993,v1:172.21.15.162:6801/901992993] conn(0x7ff16806e500 0x7ff1680709b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:13:38.067 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:13:38.068+0000 7ff17affd700 1 --2- 172.21.15.162:0/4098087047 >> [v2:172.21.15.162:6800/901992993,v1:172.21.15.162:6801/901992993] conn(0x7ff16806e500 0x7ff1680709b0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7ff164009fd0 tx=0x7ff164009e30 comp rx=0 tx=0).ready entity=mgr.14680 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:13:38.070 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:13:38.072+0000 7ff173fff700 1 -- 172.21.15.162:0/4098087047 <== mon.0 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186652 (secure 0 0 0) 0x7ff17405b340 con 0x7ff17c0fbd40 2024-04-27T04:13:38.136 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:13:37 smithi162 ceph-mon[26841]: pgmap v65: 65 pgs: 65 active+clean; 208 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:13:38.172 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:13:37 smithi181 ceph-mon[33774]: pgmap v65: 65 pgs: 65 active+clean; 208 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:13:38.353 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:13:38.355+0000 7ff181bc3700 1 -- 172.21.15.162:0/4098087047 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7ff17c04bed0 con 0x7ff17c0fbd40 2024-04-27T04:13:38.353 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:13:38.355+0000 7ff173fff700 1 -- 172.21.15.162:0/4098087047 <== mon.0 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+739 (secure 0 0 0) 0x7ff17405aed0 con 0x7ff17c0fbd40 2024-04-27T04:13:38.356 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:13:38.357+0000 7ff181bc3700 1 -- 172.21.15.162:0/4098087047 >> [v2:172.21.15.162:6800/901992993,v1:172.21.15.162:6801/901992993] conn(0x7ff16806e500 msgr2=0x7ff1680709b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:13:38.356 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:13:38.357+0000 7ff181bc3700 1 --2- 172.21.15.162:0/4098087047 >> [v2:172.21.15.162:6800/901992993,v1:172.21.15.162:6801/901992993] conn(0x7ff16806e500 0x7ff1680709b0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7ff164009fd0 tx=0x7ff164009e30 comp rx=0 tx=0).stop 2024-04-27T04:13:38.356 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:13:38.357+0000 7ff181bc3700 1 -- 172.21.15.162:0/4098087047 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff17c0fbd40 msgr2=0x7ff17c112ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:13:38.356 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:13:38.357+0000 7ff181bc3700 1 --2- 172.21.15.162:0/4098087047 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff17c0fbd40 0x7ff17c112ab0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7ff17400fe10 tx=0x7ff174009f30 comp rx=0 tx=0).stop 2024-04-27T04:13:38.356 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:13:38.357+0000 7ff181bc3700 1 -- 172.21.15.162:0/4098087047 shutdown_connections 2024-04-27T04:13:38.356 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:13:38.357+0000 7ff181bc3700 1 --2- 172.21.15.162:0/4098087047 >> [v2:172.21.15.162:6800/901992993,v1:172.21.15.162:6801/901992993] conn(0x7ff16806e500 0x7ff1680709b0 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:13:38.356 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:13:38.357+0000 7ff181bc3700 1 --2- 172.21.15.162:0/4098087047 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff17c0fbd40 0x7ff17c112ab0 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:13:38.356 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:13:38.357+0000 7ff181bc3700 1 --2- 172.21.15.162:0/4098087047 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff17c101d90 0x7ff17c112ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:13:38.356 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:13:38.357+0000 7ff181bc3700 1 -- 172.21.15.162:0/4098087047 >> 172.21.15.162:0/4098087047 conn(0x7ff17c071d00 msgr2=0x7ff17c104a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:13:38.357 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:13:38.358+0000 7ff181bc3700 1 -- 172.21.15.162:0/4098087047 shutdown_connections 2024-04-27T04:13:38.357 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:13:38.358+0000 7ff181bc3700 1 -- 172.21.15.162:0/4098087047 wait complete. 2024-04-27T04:13:38.366 INFO:teuthology.orchestra.run.smithi162.stdout:true 2024-04-27T04:13:39.011 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:13:38 smithi162 ceph-mon[26841]: from='client.? 172.21.15.162:0/4098087047' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-27T04:13:39.038 DEBUG:teuthology.orchestra.run.smithi162:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 7f9750d6-044a-11ef-bc93-c7b262605968 -e sha1=4e213a58e787437ff508e6c890ccd5fd32dc96b5 -- bash -c 'ceph orch upgrade check quay.ceph.io/ceph-ci/ceph:$sha1 | jq -e '"'"'.up_to_date | length == 2'"'"'' 2024-04-27T04:13:39.172 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:13:38 smithi181 ceph-mon[33774]: from='client.? 172.21.15.162:0/4098087047' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-27T04:13:39.295 INFO:teuthology.orchestra.run.smithi162.stderr:Inferring config /var/lib/ceph/7f9750d6-044a-11ef-bc93-c7b262605968/mon.smithi162/config 2024-04-27T04:13:40.084 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:13:39 smithi162 ceph-mon[26841]: pgmap v66: 65 pgs: 65 active+clean; 208 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-04-27T04:13:40.172 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:13:39 smithi181 ceph-mon[33774]: pgmap v66: 65 pgs: 65 active+clean; 208 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-04-27T04:13:41.033 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:13:41.034+0000 7f7e015c8700 1 -- 172.21.15.162:0/3832696234 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7dfc101d70 msgr2=0x7f7dfc102140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:13:41.033 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:13:41.034+0000 7f7e015c8700 1 --2- 172.21.15.162:0/3832696234 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7dfc101d70 0x7f7dfc102140 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f7de4004940 tx=0x7f7de4009e30 comp rx=0 tx=0).stop 2024-04-27T04:13:41.033 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:13:41.034+0000 7f7e015c8700 1 -- 172.21.15.162:0/3832696234 shutdown_connections 2024-04-27T04:13:41.033 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:13:41.034+0000 7f7e015c8700 1 --2- 172.21.15.162:0/3832696234 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7dfc102710 0x7f7dfc0fbd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:13:41.033 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:13:41.034+0000 7f7e015c8700 1 --2- 172.21.15.162:0/3832696234 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7dfc101d70 0x7f7dfc102140 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:13:41.033 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:13:41.034+0000 7f7e015c8700 1 -- 172.21.15.162:0/3832696234 >> 172.21.15.162:0/3832696234 conn(0x7f7dfc0f78e0 msgr2=0x7f7dfc0f9cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:13:41.033 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:13:41.034+0000 7f7e015c8700 1 -- 172.21.15.162:0/3832696234 shutdown_connections 2024-04-27T04:13:41.033 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:13:41.034+0000 7f7e015c8700 1 -- 172.21.15.162:0/3832696234 wait complete. 2024-04-27T04:13:41.035 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:13:41.035+0000 7f7e015c8700 1 Processor -- start 2024-04-27T04:13:41.035 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:13:41.035+0000 7f7e015c8700 1 -- start start 2024-04-27T04:13:41.035 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:13:41.035+0000 7f7e015c8700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7dfc101d70 0x7f7dfc115af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:13:41.035 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:13:41.035+0000 7f7e015c8700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7dfc102710 0x7f7dfc116030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:13:41.035 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:13:41.035+0000 7f7e015c8700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7dfc116680 con 0x7f7dfc101d70 2024-04-27T04:13:41.035 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:13:41.035+0000 7f7e015c8700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7dfc10fc60 con 0x7f7dfc102710 2024-04-27T04:13:41.035 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:13:41.035+0000 7f7dfaffd700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7dfc101d70 0x7f7dfc115af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:13:41.036 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:13:41.035+0000 7f7dfaffd700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7dfc101d70 0x7f7dfc115af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:33814/0 (socket says 172.21.15.162:33814) 2024-04-27T04:13:41.036 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:13:41.035+0000 7f7dfaffd700 1 -- 172.21.15.162:0/1455081901 learned_addr learned my addr 172.21.15.162:0/1455081901 (peer_addr_for_me v2:172.21.15.162:0/0) 2024-04-27T04:13:41.036 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:13:41.035+0000 7f7dfa7fc700 1 --2- 172.21.15.162:0/1455081901 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7dfc102710 0x7f7dfc116030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:13:41.036 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:13:41.035+0000 7f7dfaffd700 1 -- 172.21.15.162:0/1455081901 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7dfc102710 msgr2=0x7f7dfc116030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:13:41.036 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:13:41.035+0000 7f7dfaffd700 1 --2- 172.21.15.162:0/1455081901 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7dfc102710 0x7f7dfc116030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:13:41.036 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:13:41.035+0000 7f7dfaffd700 1 -- 172.21.15.162:0/1455081901 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7de4005040 con 0x7f7dfc101d70 2024-04-27T04:13:41.036 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:13:41.036+0000 7f7dfaffd700 1 --2- 172.21.15.162:0/1455081901 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7dfc101d70 0x7f7dfc115af0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f7de400b3b0 tx=0x7f7de4009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:13:41.036 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:13:41.036+0000 7f7df37fe700 1 -- 172.21.15.162:0/1455081901 <== mon.0 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7de400e040 con 0x7f7dfc101d70 2024-04-27T04:13:41.036 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:13:41.036+0000 7f7e015c8700 1 -- 172.21.15.162:0/1455081901 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7dfc10fee0 con 0x7f7dfc101d70 2024-04-27T04:13:41.036 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:13:41.036+0000 7f7df37fe700 1 -- 172.21.15.162:0/1455081901 <== mon.0 v2:172.21.15.162:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f7de400bbc0 con 0x7f7dfc101d70 2024-04-27T04:13:41.036 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:13:41.036+0000 7f7df37fe700 1 -- 172.21.15.162:0/1455081901 <== mon.0 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7de401c3f0 con 0x7f7dfc101d70 2024-04-27T04:13:41.036 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:13:41.036+0000 7f7e015c8700 1 -- 172.21.15.162:0/1455081901 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7dfc1104a0 con 0x7f7dfc101d70 2024-04-27T04:13:41.036 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:13:41.037+0000 7f7e015c8700 1 -- 172.21.15.162:0/1455081901 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7dfc0fd4c0 con 0x7f7dfc101d70 2024-04-27T04:13:41.037 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:13:41.037+0000 7f7df37fe700 1 -- 172.21.15.162:0/1455081901 <== mon.0 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 32) v1 ==== 92469+0+0 (secure 0 0 0) 0x7f7de4013420 con 0x7f7dfc101d70 2024-04-27T04:13:41.037 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:13:41.038+0000 7f7df37fe700 1 --2- 172.21.15.162:0/1455081901 >> [v2:172.21.15.162:6800/901992993,v1:172.21.15.162:6801/901992993] conn(0x7f7de806e500 0x7f7de80709b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:13:41.037 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:13:41.038+0000 7f7df37fe700 1 -- 172.21.15.162:0/1455081901 <== mon.0 v2:172.21.15.162:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6152+0+0 (secure 0 0 0) 0x7f7de408f1d0 con 0x7f7dfc101d70 2024-04-27T04:13:41.038 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:13:41.038+0000 7f7dfa7fc700 1 --2- 172.21.15.162:0/1455081901 >> [v2:172.21.15.162:6800/901992993,v1:172.21.15.162:6801/901992993] conn(0x7f7de806e500 0x7f7de80709b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:13:41.038 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:13:41.038+0000 7f7dfa7fc700 1 --2- 172.21.15.162:0/1455081901 >> [v2:172.21.15.162:6800/901992993,v1:172.21.15.162:6801/901992993] conn(0x7f7de806e500 0x7f7de80709b0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f7dfc111390 tx=0x7f7dec009ee0 comp rx=0 tx=0).ready entity=mgr.14680 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:13:41.040 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:13:41.042+0000 7f7df37fe700 1 -- 172.21.15.162:0/1455081901 <== mon.0 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186652 (secure 0 0 0) 0x7f7de405c300 con 0x7f7dfc101d70 2024-04-27T04:13:41.261 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:13:41.262+0000 7f7e015c8700 1 -- 172.21.15.162:0/1455081901 --> [v2:172.21.15.162:6800/901992993,v1:172.21.15.162:6801/901992993] -- mgr_command(tid 0: {"prefix": "orch upgrade check", "image": "quay.ceph.io/ceph-ci/ceph:4e213a58e787437ff508e6c890ccd5fd32dc96b5", "target": ["mon-mgr", ""]}) v1 -- 0x7f7dfc108f10 con 0x7f7de806e500 2024-04-27T04:13:42.137 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:13:41 smithi162 ceph-mon[26841]: pgmap v67: 65 pgs: 65 active+clean; 208 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:13:42.172 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:13:41 smithi181 ceph-mon[33774]: pgmap v67: 65 pgs: 65 active+clean; 208 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:13:43.137 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:13:42 smithi162 ceph-mon[26841]: from='client.14806 -' entity='client.admin' cmd=[{"prefix": "orch upgrade check", "image": "quay.ceph.io/ceph-ci/ceph:4e213a58e787437ff508e6c890ccd5fd32dc96b5", "target": ["mon-mgr", ""]}]: dispatch 2024-04-27T04:13:43.172 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:13:42 smithi181 ceph-mon[33774]: from='client.14806 -' entity='client.admin' cmd=[{"prefix": "orch upgrade check", "image": "quay.ceph.io/ceph-ci/ceph:4e213a58e787437ff508e6c890ccd5fd32dc96b5", "target": ["mon-mgr", ""]}]: dispatch 2024-04-27T04:13:43.331 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:13:43.333+0000 7f7df37fe700 1 -- 172.21.15.162:0/1455081901 <== mgr.14680 v2:172.21.15.162:6800/901992993 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+5092 (secure 0 0 0) 0x7f7dec004610 con 0x7f7de806e500 2024-04-27T04:13:43.334 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:13:43.336+0000 7f7e015c8700 1 -- 172.21.15.162:0/1455081901 >> [v2:172.21.15.162:6800/901992993,v1:172.21.15.162:6801/901992993] conn(0x7f7de806e500 msgr2=0x7f7de80709b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:13:43.334 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:13:43.336+0000 7f7e015c8700 1 --2- 172.21.15.162:0/1455081901 >> [v2:172.21.15.162:6800/901992993,v1:172.21.15.162:6801/901992993] conn(0x7f7de806e500 0x7f7de80709b0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f7dfc111390 tx=0x7f7dec009ee0 comp rx=0 tx=0).stop 2024-04-27T04:13:43.334 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:13:43.336+0000 7f7e015c8700 1 -- 172.21.15.162:0/1455081901 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7dfc101d70 msgr2=0x7f7dfc115af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:13:43.334 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:13:43.336+0000 7f7e015c8700 1 --2- 172.21.15.162:0/1455081901 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7dfc101d70 0x7f7dfc115af0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f7de400b3b0 tx=0x7f7de4009e30 comp rx=0 tx=0).stop 2024-04-27T04:13:43.335 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:13:43.336+0000 7f7e015c8700 1 -- 172.21.15.162:0/1455081901 shutdown_connections 2024-04-27T04:13:43.335 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:13:43.336+0000 7f7e015c8700 1 --2- 172.21.15.162:0/1455081901 >> [v2:172.21.15.162:6800/901992993,v1:172.21.15.162:6801/901992993] conn(0x7f7de806e500 0x7f7de80709b0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:13:43.335 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:13:43.336+0000 7f7e015c8700 1 --2- 172.21.15.162:0/1455081901 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7dfc101d70 0x7f7dfc115af0 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:13:43.335 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:13:43.336+0000 7f7e015c8700 1 --2- 172.21.15.162:0/1455081901 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7dfc102710 0x7f7dfc116030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:13:43.335 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:13:43.336+0000 7f7e015c8700 1 -- 172.21.15.162:0/1455081901 >> 172.21.15.162:0/1455081901 conn(0x7f7dfc0f78e0 msgr2=0x7f7dfc066530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:13:43.335 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:13:43.336+0000 7f7e015c8700 1 -- 172.21.15.162:0/1455081901 shutdown_connections 2024-04-27T04:13:43.335 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:13:43.336+0000 7f7e015c8700 1 -- 172.21.15.162:0/1455081901 wait complete. 2024-04-27T04:13:43.348 INFO:teuthology.orchestra.run.smithi162.stdout:true 2024-04-27T04:13:43.935 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:13:43 smithi162 ceph-mon[26841]: pgmap v68: 65 pgs: 65 active+clean; 208 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:13:43.960 DEBUG:teuthology.orchestra.run.smithi162:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 7f9750d6-044a-11ef-bc93-c7b262605968 -e sha1=4e213a58e787437ff508e6c890ccd5fd32dc96b5 -- bash -c 'ceph orch ps' 2024-04-27T04:13:44.172 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:13:43 smithi181 ceph-mon[33774]: pgmap v68: 65 pgs: 65 active+clean; 208 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:13:44.214 INFO:teuthology.orchestra.run.smithi162.stderr:Inferring config /var/lib/ceph/7f9750d6-044a-11ef-bc93-c7b262605968/mon.smithi162/config 2024-04-27T04:13:45.812 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:13:45.812+0000 7f311ec0c700 1 -- 172.21.15.162:0/3221650693 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3118102de0 msgr2=0x7f31181031b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:13:45.812 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:13:45.812+0000 7f311ec0c700 1 --2- 172.21.15.162:0/3221650693 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3118102de0 0x7f31181031b0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f310c004940 tx=0x7f310c009e30 comp rx=0 tx=0).stop 2024-04-27T04:13:45.813 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:13:45.812+0000 7f311ec0c700 1 -- 172.21.15.162:0/3221650693 shutdown_connections 2024-04-27T04:13:45.813 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:13:45.812+0000 7f311ec0c700 1 --2- 172.21.15.162:0/3221650693 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3118103780 0x7f31180fcdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:13:45.813 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:13:45.812+0000 7f311ec0c700 1 --2- 172.21.15.162:0/3221650693 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3118102de0 0x7f31181031b0 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:13:45.813 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:13:45.812+0000 7f311ec0c700 1 -- 172.21.15.162:0/3221650693 >> 172.21.15.162:0/3221650693 conn(0x7f31180f8990 msgr2=0x7f31180fada0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:13:45.813 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:13:45.812+0000 7f311ec0c700 1 -- 172.21.15.162:0/3221650693 shutdown_connections 2024-04-27T04:13:45.813 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:13:45.812+0000 7f311ec0c700 1 -- 172.21.15.162:0/3221650693 wait complete. 2024-04-27T04:13:45.813 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:13:45.813+0000 7f311ec0c700 1 Processor -- start 2024-04-27T04:13:45.813 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:13:45.813+0000 7f311ec0c700 1 -- start start 2024-04-27T04:13:45.813 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:13:45.813+0000 7f311ec0c700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3118102de0 0x7f3118070260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:13:45.813 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:13:45.813+0000 7f311ec0c700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3118103780 0x7f31180707a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:13:45.813 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:13:45.813+0000 7f311ec0c700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3118072240 con 0x7f3118102de0 2024-04-27T04:13:45.813 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:13:45.813+0000 7f311ec0c700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3118070ce0 con 0x7f3118103780 2024-04-27T04:13:45.813 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:13:45.813+0000 7f311c9a8700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3118102de0 0x7f3118070260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:13:45.814 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:13:45.813+0000 7f311c9a8700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3118102de0 0x7f3118070260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:40872/0 (socket says 172.21.15.162:40872) 2024-04-27T04:13:45.814 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:13:45.813+0000 7f311c9a8700 1 -- 172.21.15.162:0/2900144774 learned_addr learned my addr 172.21.15.162:0/2900144774 (peer_addr_for_me v2:172.21.15.162:0/0) 2024-04-27T04:13:45.814 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:13:45.814+0000 7f3117fff700 1 --2- 172.21.15.162:0/2900144774 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3118103780 0x7f31180707a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:13:45.814 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:13:45.814+0000 7f311c9a8700 1 -- 172.21.15.162:0/2900144774 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3118103780 msgr2=0x7f31180707a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:13:45.814 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:13:45.814+0000 7f311c9a8700 1 --2- 172.21.15.162:0/2900144774 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3118103780 0x7f31180707a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:13:45.814 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:13:45.814+0000 7f311c9a8700 1 -- 172.21.15.162:0/2900144774 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f310c005040 con 0x7f3118102de0 2024-04-27T04:13:45.814 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:13:45.814+0000 7f311c9a8700 1 --2- 172.21.15.162:0/2900144774 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3118102de0 0x7f3118070260 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f310c00b3b0 tx=0x7f310c009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:13:45.814 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:13:45.814+0000 7f31157fa700 1 -- 172.21.15.162:0/2900144774 <== mon.0 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f310c00e040 con 0x7f3118102de0 2024-04-27T04:13:45.814 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:13:45.814+0000 7f311ec0c700 1 -- 172.21.15.162:0/2900144774 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3118070f60 con 0x7f3118102de0 2024-04-27T04:13:45.814 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:13:45.814+0000 7f31157fa700 1 -- 172.21.15.162:0/2900144774 <== mon.0 v2:172.21.15.162:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f310c00bbc0 con 0x7f3118102de0 2024-04-27T04:13:45.814 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:13:45.814+0000 7f31157fa700 1 -- 172.21.15.162:0/2900144774 <== mon.0 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f310c01c3f0 con 0x7f3118102de0 2024-04-27T04:13:45.814 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:13:45.814+0000 7f311ec0c700 1 -- 172.21.15.162:0/2900144774 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f311811a130 con 0x7f3118102de0 2024-04-27T04:13:45.815 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:13:45.815+0000 7f311ec0c700 1 -- 172.21.15.162:0/2900144774 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f31181000f0 con 0x7f3118102de0 2024-04-27T04:13:45.815 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:13:45.815+0000 7f31157fa700 1 -- 172.21.15.162:0/2900144774 <== mon.0 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 32) v1 ==== 92469+0+0 (secure 0 0 0) 0x7f310c013420 con 0x7f3118102de0 2024-04-27T04:13:45.816 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:13:45.816+0000 7f31157fa700 1 --2- 172.21.15.162:0/2900144774 >> [v2:172.21.15.162:6800/901992993,v1:172.21.15.162:6801/901992993] conn(0x7f310806e500 0x7f31080709b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:13:45.816 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:13:45.816+0000 7f31157fa700 1 -- 172.21.15.162:0/2900144774 <== mon.0 v2:172.21.15.162:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6152+0+0 (secure 0 0 0) 0x7f310c08f120 con 0x7f3118102de0 2024-04-27T04:13:45.816 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:13:45.816+0000 7f3117fff700 1 --2- 172.21.15.162:0/2900144774 >> [v2:172.21.15.162:6800/901992993,v1:172.21.15.162:6801/901992993] conn(0x7f310806e500 0x7f31080709b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:13:45.816 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:13:45.816+0000 7f3117fff700 1 --2- 172.21.15.162:0/2900144774 >> [v2:172.21.15.162:6800/901992993,v1:172.21.15.162:6801/901992993] conn(0x7f310806e500 0x7f31080709b0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f3104006fd0 tx=0x7f3104008040 comp rx=0 tx=0).ready entity=mgr.14680 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:13:45.819 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:13:45.821+0000 7f31157fa700 1 -- 172.21.15.162:0/2900144774 <== mon.0 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186652 (secure 0 0 0) 0x7f310c0587a0 con 0x7f3118102de0 2024-04-27T04:13:45.886 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:13:45 smithi162 ceph-mon[26841]: pgmap v69: 65 pgs: 65 active+clean; 208 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:13:46.028 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:13:46.030+0000 7f311ec0c700 1 -- 172.21.15.162:0/2900144774 --> [v2:172.21.15.162:6800/901992993,v1:172.21.15.162:6801/901992993] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f3118071740 con 0x7f310806e500 2024-04-27T04:13:46.038 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:13:46.040+0000 7f31157fa700 1 -- 172.21.15.162:0/2900144774 <== mgr.14680 v2:172.21.15.162:6800/901992993 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3520 (secure 0 0 0) 0x7f3118071740 con 0x7f310806e500 2024-04-27T04:13:46.039 INFO:teuthology.orchestra.run.smithi162.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-27T04:13:46.039 INFO:teuthology.orchestra.run.smithi162.stdout:alertmanager.smithi162 smithi162 *:9093,9094 running (54s) 25s ago 12m 16.5M - 0.25.0 c8568f914cd2 40e334659017 2024-04-27T04:13:46.039 INFO:teuthology.orchestra.run.smithi162.stdout:crash.smithi162 smithi162 running (12m) 25s ago 12m 6660k - 17.2.7-907-g2be3efa9 1c5834633715 228c98df4289 2024-04-27T04:13:46.039 INFO:teuthology.orchestra.run.smithi162.stdout:crash.smithi181 smithi181 running (11m) 86s ago 11m 6665k - 17.2.7-907-g2be3efa9 1c5834633715 c8e553e3df2b 2024-04-27T04:13:46.039 INFO:teuthology.orchestra.run.smithi162.stdout:grafana.smithi162 smithi162 *:3000 running (31s) 25s ago 11m 52.9M - 9.4.12 2bacad6d85d8 3d78c478f03f 2024-04-27T04:13:46.039 INFO:teuthology.orchestra.run.smithi162.stdout:mds.cephfs.smithi162.gzlyjj smithi162 running (6m) 25s ago 6m 12.9M - 17.2.7-907-g2be3efa9 1c5834633715 27e5b030dd32 2024-04-27T04:13:46.039 INFO:teuthology.orchestra.run.smithi162.stdout:mds.cephfs.smithi162.xcnapk smithi162 running (6m) 25s ago 6m 100M - 17.2.7-907-g2be3efa9 1c5834633715 38f9030df029 2024-04-27T04:13:46.039 INFO:teuthology.orchestra.run.smithi162.stdout:mds.cephfs.smithi181.bkkcdm smithi181 running (6m) 86s ago 6m 14.3M - 17.2.7-907-g2be3efa9 1c5834633715 af55afca1263 2024-04-27T04:13:46.039 INFO:teuthology.orchestra.run.smithi162.stdout:mds.cephfs.smithi181.dzomgh smithi181 running (6m) 86s ago 6m 145M - 17.2.7-907-g2be3efa9 1c5834633715 6733296597f3 2024-04-27T04:13:46.039 INFO:teuthology.orchestra.run.smithi162.stdout:mgr.smithi162.oennfn smithi162 *:8443,9283,8765 running (2m) 25s ago 13m 508M - 19.0.0-3253-g4e213a58 67e414ff4bc4 58a748d3637b 2024-04-27T04:13:46.039 INFO:teuthology.orchestra.run.smithi162.stdout:mgr.smithi181.vtwuhz smithi181 *:8443,9283,8765 running (107s) 86s ago 11m 451M - 19.0.0-3253-g4e213a58 67e414ff4bc4 813584297b1d 2024-04-27T04:13:46.039 INFO:teuthology.orchestra.run.smithi162.stdout:mon.smithi162 smithi162 running (13m) 25s ago 13m 55.0M 2048M 17.2.7-907-g2be3efa9 1c5834633715 0f1e3682b9ab 2024-04-27T04:13:46.039 INFO:teuthology.orchestra.run.smithi162.stdout:mon.smithi181 smithi181 running (11m) 86s ago 11m 42.7M 2048M 17.2.7-907-g2be3efa9 1c5834633715 96e48aa3912f 2024-04-27T04:13:46.040 INFO:teuthology.orchestra.run.smithi162.stdout:node-exporter.smithi162 smithi162 *:9100 running (94s) 25s ago 12m 8376k - 1.5.0 0da6a335fe13 5c4c5ea9c1d7 2024-04-27T04:13:46.040 INFO:teuthology.orchestra.run.smithi162.stdout:node-exporter.smithi181 smithi181 *:9100 running (90s) 86s ago 11m 3670k - 1.5.0 0da6a335fe13 9067f14c0519 2024-04-27T04:13:46.040 INFO:teuthology.orchestra.run.smithi162.stdout:osd.0 smithi162 running (10m) 25s ago 10m 232M 1557M 17.2.7-907-g2be3efa9 1c5834633715 4a9df3aa938d 2024-04-27T04:13:46.040 INFO:teuthology.orchestra.run.smithi162.stdout:osd.1 smithi162 running (9m) 25s ago 9m 244M 1557M 17.2.7-907-g2be3efa9 1c5834633715 aefb464c46e8 2024-04-27T04:13:46.040 INFO:teuthology.orchestra.run.smithi162.stdout:osd.2 smithi162 running (9m) 25s ago 9m 203M 1557M 17.2.7-907-g2be3efa9 1c5834633715 70f16d86d38e 2024-04-27T04:13:46.040 INFO:teuthology.orchestra.run.smithi162.stdout:osd.3 smithi181 running (9m) 86s ago 9m 276M 2581M 17.2.7-907-g2be3efa9 1c5834633715 030d829be54e 2024-04-27T04:13:46.040 INFO:teuthology.orchestra.run.smithi162.stdout:osd.4 smithi181 running (8m) 86s ago 8m 244M 2581M 17.2.7-907-g2be3efa9 1c5834633715 512afbee7cf7 2024-04-27T04:13:46.040 INFO:teuthology.orchestra.run.smithi162.stdout:osd.5 smithi181 running (8m) 86s ago 8m 221M 2581M 17.2.7-907-g2be3efa9 1c5834633715 f47a1daac1ea 2024-04-27T04:13:46.040 INFO:teuthology.orchestra.run.smithi162.stdout:prometheus.smithi162 smithi162 *:9095 running (72s) 25s ago 11m 51.4M - 2.43.0 a07b618ecd1d f30657a43324 2024-04-27T04:13:46.041 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:13:46.042+0000 7f311ec0c700 1 -- 172.21.15.162:0/2900144774 >> [v2:172.21.15.162:6800/901992993,v1:172.21.15.162:6801/901992993] conn(0x7f310806e500 msgr2=0x7f31080709b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:13:46.041 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:13:46.042+0000 7f311ec0c700 1 --2- 172.21.15.162:0/2900144774 >> [v2:172.21.15.162:6800/901992993,v1:172.21.15.162:6801/901992993] conn(0x7f310806e500 0x7f31080709b0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f3104006fd0 tx=0x7f3104008040 comp rx=0 tx=0).stop 2024-04-27T04:13:46.041 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:13:46.042+0000 7f311ec0c700 1 -- 172.21.15.162:0/2900144774 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3118102de0 msgr2=0x7f3118070260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:13:46.041 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:13:46.042+0000 7f311ec0c700 1 --2- 172.21.15.162:0/2900144774 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3118102de0 0x7f3118070260 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f310c00b3b0 tx=0x7f310c009e30 comp rx=0 tx=0).stop 2024-04-27T04:13:46.041 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:13:46.042+0000 7f311ec0c700 1 -- 172.21.15.162:0/2900144774 shutdown_connections 2024-04-27T04:13:46.041 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:13:46.042+0000 7f311ec0c700 1 --2- 172.21.15.162:0/2900144774 >> [v2:172.21.15.162:6800/901992993,v1:172.21.15.162:6801/901992993] conn(0x7f310806e500 0x7f31080709b0 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:13:46.041 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:13:46.042+0000 7f311ec0c700 1 --2- 172.21.15.162:0/2900144774 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3118102de0 0x7f3118070260 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:13:46.042 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:13:46.042+0000 7f311ec0c700 1 --2- 172.21.15.162:0/2900144774 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3118103780 0x7f31180707a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:13:46.042 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:13:46.042+0000 7f311ec0c700 1 -- 172.21.15.162:0/2900144774 >> 172.21.15.162:0/2900144774 conn(0x7f31180f8990 msgr2=0x7f31180fa6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:13:46.042 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:13:46.042+0000 7f311ec0c700 1 -- 172.21.15.162:0/2900144774 shutdown_connections 2024-04-27T04:13:46.042 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:13:46.042+0000 7f311ec0c700 1 -- 172.21.15.162:0/2900144774 wait complete. 2024-04-27T04:13:46.172 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:13:45 smithi181 ceph-mon[33774]: pgmap v69: 65 pgs: 65 active+clean; 208 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:13:46.699 INFO:teuthology.task.sequential:In sequential, running task cephadm.shell... 2024-04-27T04:13:46.700 INFO:tasks.cephadm:Running commands on role host.a host ubuntu@smithi162.front.sepia.ceph.com 2024-04-27T04:13:46.700 DEBUG:teuthology.orchestra.run.smithi162:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 7f9750d6-044a-11ef-bc93-c7b262605968 -e sha1=4e213a58e787437ff508e6c890ccd5fd32dc96b5 -- bash -c 'ceph config set mgr mgr/orchestrator/fail_fs true' 2024-04-27T04:13:46.955 INFO:teuthology.orchestra.run.smithi162.stderr:Inferring config /var/lib/ceph/7f9750d6-044a-11ef-bc93-c7b262605968/mon.smithi162/config 2024-04-27T04:13:47.117 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:13:46 smithi162 ceph-mon[26841]: from='client.14810 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-27T04:13:47.172 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:13:46 smithi181 ceph-mon[33774]: from='client.14810 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-27T04:13:48.137 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:13:47 smithi162 ceph-mon[26841]: pgmap v70: 65 pgs: 65 active+clean; 208 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:13:48.137 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:13:47 smithi162 ceph-mon[26841]: from='mgr.14680 172.21.15.162:0/501880828' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-27T04:13:48.172 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:13:47 smithi181 ceph-mon[33774]: pgmap v70: 65 pgs: 65 active+clean; 208 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:13:48.172 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:13:47 smithi181 ceph-mon[33774]: from='mgr.14680 172.21.15.162:0/501880828' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-27T04:13:48.879 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:13:48.880+0000 7fdd8f690700 1 -- 172.21.15.162:0/3107317724 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fdd88102de0 msgr2=0x7fdd881031b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:13:48.879 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:13:48.880+0000 7fdd8f690700 1 --2- 172.21.15.162:0/3107317724 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fdd88102de0 0x7fdd881031b0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7fdd84004940 tx=0x7fdd84009e30 comp rx=0 tx=0).stop 2024-04-27T04:13:48.879 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:13:48.880+0000 7fdd8f690700 1 -- 172.21.15.162:0/3107317724 shutdown_connections 2024-04-27T04:13:48.879 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:13:48.880+0000 7fdd8f690700 1 --2- 172.21.15.162:0/3107317724 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdd88103780 0x7fdd880fcdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:13:48.879 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:13:48.880+0000 7fdd8f690700 1 --2- 172.21.15.162:0/3107317724 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fdd88102de0 0x7fdd881031b0 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:13:48.879 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:13:48.880+0000 7fdd8f690700 1 -- 172.21.15.162:0/3107317724 >> 172.21.15.162:0/3107317724 conn(0x7fdd880f8990 msgr2=0x7fdd880fada0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:13:48.879 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:13:48.881+0000 7fdd8f690700 1 -- 172.21.15.162:0/3107317724 shutdown_connections 2024-04-27T04:13:48.880 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:13:48.881+0000 7fdd8f690700 1 -- 172.21.15.162:0/3107317724 wait complete. 2024-04-27T04:13:48.880 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:13:48.881+0000 7fdd8f690700 1 Processor -- start 2024-04-27T04:13:48.880 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:13:48.881+0000 7fdd8f690700 1 -- start start 2024-04-27T04:13:48.881 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:13:48.881+0000 7fdd8f690700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fdd88102de0 0x7fdd88070260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:13:48.881 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:13:48.881+0000 7fdd8f690700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdd88103780 0x7fdd880707a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:13:48.881 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:13:48.881+0000 7fdd8f690700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd88072240 con 0x7fdd88102de0 2024-04-27T04:13:48.881 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:13:48.881+0000 7fdd8f690700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd88070ce0 con 0x7fdd88103780 2024-04-27T04:13:48.881 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:13:48.881+0000 7fdd8d42c700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fdd88102de0 0x7fdd88070260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:13:48.882 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:13:48.881+0000 7fdd8d42c700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fdd88102de0 0x7fdd88070260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:40902/0 (socket says 172.21.15.162:40902) 2024-04-27T04:13:48.882 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:13:48.881+0000 7fdd8d42c700 1 -- 172.21.15.162:0/1441557457 learned_addr learned my addr 172.21.15.162:0/1441557457 (peer_addr_for_me v2:172.21.15.162:0/0) 2024-04-27T04:13:48.882 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:13:48.882+0000 7fdd8cc2b700 1 --2- 172.21.15.162:0/1441557457 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdd88103780 0x7fdd880707a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:13:48.882 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:13:48.882+0000 7fdd8d42c700 1 -- 172.21.15.162:0/1441557457 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdd88103780 msgr2=0x7fdd880707a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:13:48.882 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:13:48.882+0000 7fdd8d42c700 1 --2- 172.21.15.162:0/1441557457 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdd88103780 0x7fdd880707a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:13:48.882 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:13:48.882+0000 7fdd8d42c700 1 -- 172.21.15.162:0/1441557457 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd84005040 con 0x7fdd88102de0 2024-04-27T04:13:48.882 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:13:48.882+0000 7fdd8d42c700 1 --2- 172.21.15.162:0/1441557457 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fdd88102de0 0x7fdd88070260 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7fdd8400b3b0 tx=0x7fdd84009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:13:48.883 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:13:48.882+0000 7fdd7dffb700 1 -- 172.21.15.162:0/1441557457 <== mon.0 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fdd8400e040 con 0x7fdd88102de0 2024-04-27T04:13:48.883 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:13:48.882+0000 7fdd8f690700 1 -- 172.21.15.162:0/1441557457 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdd88070f60 con 0x7fdd88102de0 2024-04-27T04:13:48.883 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:13:48.882+0000 7fdd7dffb700 1 -- 172.21.15.162:0/1441557457 <== mon.0 v2:172.21.15.162:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fdd8400bbc0 con 0x7fdd88102de0 2024-04-27T04:13:48.883 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:13:48.882+0000 7fdd7dffb700 1 -- 172.21.15.162:0/1441557457 <== mon.0 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fdd8401c3f0 con 0x7fdd88102de0 2024-04-27T04:13:48.883 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:13:48.882+0000 7fdd8f690700 1 -- 172.21.15.162:0/1441557457 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdd8811a130 con 0x7fdd88102de0 2024-04-27T04:13:48.883 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:13:48.883+0000 7fdd8f690700 1 -- 172.21.15.162:0/1441557457 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdd881000f0 con 0x7fdd88102de0 2024-04-27T04:13:48.884 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:13:48.884+0000 7fdd7dffb700 1 -- 172.21.15.162:0/1441557457 <== mon.0 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 32) v1 ==== 92469+0+0 (secure 0 0 0) 0x7fdd8400bd30 con 0x7fdd88102de0 2024-04-27T04:13:48.884 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:13:48.884+0000 7fdd7dffb700 1 --2- 172.21.15.162:0/1441557457 >> [v2:172.21.15.162:6800/901992993,v1:172.21.15.162:6801/901992993] conn(0x7fdd7406e500 0x7fdd740709b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:13:48.884 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:13:48.884+0000 7fdd7dffb700 1 -- 172.21.15.162:0/1441557457 <== mon.0 v2:172.21.15.162:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6152+0+0 (secure 0 0 0) 0x7fdd8408f1e0 con 0x7fdd88102de0 2024-04-27T04:13:48.884 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:13:48.884+0000 7fdd8cc2b700 1 --2- 172.21.15.162:0/1441557457 >> [v2:172.21.15.162:6800/901992993,v1:172.21.15.162:6801/901992993] conn(0x7fdd7406e500 0x7fdd740709b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:13:48.884 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:13:48.885+0000 7fdd8cc2b700 1 --2- 172.21.15.162:0/1441557457 >> [v2:172.21.15.162:6800/901992993,v1:172.21.15.162:6801/901992993] conn(0x7fdd7406e500 0x7fdd740709b0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fdd78006fd0 tx=0x7fdd78008040 comp rx=0 tx=0).ready entity=mgr.14680 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:13:48.887 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:13:48.889+0000 7fdd7dffb700 1 -- 172.21.15.162:0/1441557457 <== mon.0 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186652 (secure 0 0 0) 0x7fdd840587e0 con 0x7fdd88102de0 2024-04-27T04:13:49.085 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:13:49.086+0000 7fdd8f690700 1 -- 172.21.15.162:0/1441557457 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command([{prefix=config set, name=mgr/orchestrator/fail_fs}] v 0) v1 -- 0x7fdd880fe530 con 0x7fdd88102de0 2024-04-27T04:13:49.089 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:13:49.091+0000 7fdd7dffb700 1 -- 172.21.15.162:0/1441557457 <== mon.0 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/orchestrator/fail_fs}]=0 v52)=0 v52) v1 ==== 125+0+0 (secure 0 0 0) 0x7fdd880fe530 con 0x7fdd88102de0 2024-04-27T04:13:49.092 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:13:49.094+0000 7fdd8f690700 1 -- 172.21.15.162:0/1441557457 >> [v2:172.21.15.162:6800/901992993,v1:172.21.15.162:6801/901992993] conn(0x7fdd7406e500 msgr2=0x7fdd740709b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:13:49.093 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:13:49.094+0000 7fdd8f690700 1 --2- 172.21.15.162:0/1441557457 >> [v2:172.21.15.162:6800/901992993,v1:172.21.15.162:6801/901992993] conn(0x7fdd7406e500 0x7fdd740709b0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fdd78006fd0 tx=0x7fdd78008040 comp rx=0 tx=0).stop 2024-04-27T04:13:49.093 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:13:49.094+0000 7fdd8f690700 1 -- 172.21.15.162:0/1441557457 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fdd88102de0 msgr2=0x7fdd88070260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:13:49.093 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:13:49.094+0000 7fdd8f690700 1 --2- 172.21.15.162:0/1441557457 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fdd88102de0 0x7fdd88070260 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7fdd8400b3b0 tx=0x7fdd84009e30 comp rx=0 tx=0).stop 2024-04-27T04:13:49.093 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:13:49.094+0000 7fdd8f690700 1 -- 172.21.15.162:0/1441557457 shutdown_connections 2024-04-27T04:13:49.093 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:13:49.094+0000 7fdd8f690700 1 --2- 172.21.15.162:0/1441557457 >> [v2:172.21.15.162:6800/901992993,v1:172.21.15.162:6801/901992993] conn(0x7fdd7406e500 0x7fdd740709b0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:13:49.093 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:13:49.094+0000 7fdd8f690700 1 --2- 172.21.15.162:0/1441557457 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fdd88102de0 0x7fdd88070260 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:13:49.093 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:13:49.094+0000 7fdd8f690700 1 --2- 172.21.15.162:0/1441557457 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdd88103780 0x7fdd880707a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:13:49.093 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:13:49.094+0000 7fdd8f690700 1 -- 172.21.15.162:0/1441557457 >> 172.21.15.162:0/1441557457 conn(0x7fdd880f8990 msgr2=0x7fdd880fa6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:13:49.093 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:13:49.094+0000 7fdd8f690700 1 -- 172.21.15.162:0/1441557457 shutdown_connections 2024-04-27T04:13:49.093 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:13:49.094+0000 7fdd8f690700 1 -- 172.21.15.162:0/1441557457 wait complete. 2024-04-27T04:13:49.698 INFO:teuthology.task.sequential:In sequential, running task cephadm.shell... 2024-04-27T04:13:49.698 INFO:tasks.cephadm:Running commands on role host.a host ubuntu@smithi162.front.sepia.ceph.com 2024-04-27T04:13:49.698 DEBUG:teuthology.orchestra.run.smithi162:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 7f9750d6-044a-11ef-bc93-c7b262605968 -e sha1=4e213a58e787437ff508e6c890ccd5fd32dc96b5 -- bash -c 'ceph config set mon mon_warn_on_insecure_global_id_reclaim false --force' 2024-04-27T04:13:49.953 INFO:teuthology.orchestra.run.smithi162.stderr:Inferring config /var/lib/ceph/7f9750d6-044a-11ef-bc93-c7b262605968/mon.smithi162/config 2024-04-27T04:13:50.327 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:13:50 smithi162 ceph-mon[26841]: pgmap v71: 65 pgs: 65 active+clean; 208 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-04-27T04:13:50.327 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:13:50 smithi162 ceph-mon[26841]: from='client.? 172.21.15.162:0/1441557457' entity='client.admin' 2024-04-27T04:13:50.327 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:13:50 smithi162 ceph-mon[26841]: from='mgr.14680 172.21.15.162:0/501880828' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T04:13:50.328 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:13:50 smithi162 ceph-mon[26841]: from='mgr.14680 172.21.15.162:0/501880828' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T04:13:50.328 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:13:50 smithi162 ceph-mon[26841]: from='mgr.14680 172.21.15.162:0/501880828' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T04:13:50.328 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:13:50 smithi162 ceph-mon[26841]: from='mgr.14680 172.21.15.162:0/501880828' entity='mgr.smithi162.oennfn' 2024-04-27T04:13:50.422 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:13:50 smithi181 ceph-mon[33774]: pgmap v71: 65 pgs: 65 active+clean; 208 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-04-27T04:13:50.422 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:13:50 smithi181 ceph-mon[33774]: from='client.? 172.21.15.162:0/1441557457' entity='client.admin' 2024-04-27T04:13:50.422 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:13:50 smithi181 ceph-mon[33774]: from='mgr.14680 172.21.15.162:0/501880828' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T04:13:50.422 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:13:50 smithi181 ceph-mon[33774]: from='mgr.14680 172.21.15.162:0/501880828' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T04:13:50.422 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:13:50 smithi181 ceph-mon[33774]: from='mgr.14680 172.21.15.162:0/501880828' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T04:13:50.422 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:13:50 smithi181 ceph-mon[33774]: from='mgr.14680 172.21.15.162:0/501880828' entity='mgr.smithi162.oennfn' 2024-04-27T04:13:51.561 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:13:51.546+0000 7f10fc3af700 1 -- 172.21.15.162:0/2310478168 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f10f4065640 msgr2=0x7f10f4065ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:13:51.561 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:13:51.546+0000 7f10fc3af700 1 --2- 172.21.15.162:0/2310478168 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f10f4065640 0x7f10f4065ab0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f10e4004940 tx=0x7f10e4009e30 comp rx=0 tx=0).stop 2024-04-27T04:13:51.561 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:13:51.547+0000 7f10fc3af700 1 -- 172.21.15.162:0/2310478168 shutdown_connections 2024-04-27T04:13:51.561 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:13:51.547+0000 7f10fc3af700 1 --2- 172.21.15.162:0/2310478168 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f10f4065640 0x7f10f4065ab0 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:13:51.561 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:13:51.547+0000 7f10fc3af700 1 --2- 172.21.15.162:0/2310478168 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f10f4102e50 0x7f10f4103220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:13:51.561 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:13:51.547+0000 7f10fc3af700 1 -- 172.21.15.162:0/2310478168 >> 172.21.15.162:0/2310478168 conn(0x7f10f4071d00 msgr2=0x7f10f4072100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:13:51.561 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:13:51.547+0000 7f10fc3af700 1 -- 172.21.15.162:0/2310478168 shutdown_connections 2024-04-27T04:13:51.562 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:13:51.547+0000 7f10fc3af700 1 -- 172.21.15.162:0/2310478168 wait complete. 2024-04-27T04:13:51.562 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:13:51.547+0000 7f10fc3af700 1 Processor -- start 2024-04-27T04:13:51.562 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:13:51.547+0000 7f10fc3af700 1 -- start start 2024-04-27T04:13:51.562 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:13:51.547+0000 7f10fc3af700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f10f4065640 0x7f10f4113b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:13:51.562 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:13:51.547+0000 7f10fc3af700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f10f4102e50 0x7f10f41140a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:13:51.562 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:13:51.547+0000 7f10fc3af700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10f4114780 con 0x7f10f4065640 2024-04-27T04:13:51.562 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:13:51.547+0000 7f10fc3af700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10f410dae0 con 0x7f10f4102e50 2024-04-27T04:13:51.562 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:13:51.547+0000 7f10fa14b700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f10f4065640 0x7f10f4113b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:13:51.562 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:13:51.548+0000 7f10fa14b700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f10f4065640 0x7f10f4113b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:40916/0 (socket says 172.21.15.162:40916) 2024-04-27T04:13:51.562 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:13:51.548+0000 7f10fa14b700 1 -- 172.21.15.162:0/4247197818 learned_addr learned my addr 172.21.15.162:0/4247197818 (peer_addr_for_me v2:172.21.15.162:0/0) 2024-04-27T04:13:51.562 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:13:51.548+0000 7f10f994a700 1 --2- 172.21.15.162:0/4247197818 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f10f4102e50 0x7f10f41140a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:13:51.562 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:13:51.548+0000 7f10fa14b700 1 -- 172.21.15.162:0/4247197818 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f10f4102e50 msgr2=0x7f10f41140a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:13:51.562 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:13:51.548+0000 7f10fa14b700 1 --2- 172.21.15.162:0/4247197818 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f10f4102e50 0x7f10f41140a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:13:51.563 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:13:51.548+0000 7f10fa14b700 1 -- 172.21.15.162:0/4247197818 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f10e4005040 con 0x7f10f4065640 2024-04-27T04:13:51.563 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:13:51.548+0000 7f10fa14b700 1 --2- 172.21.15.162:0/4247197818 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f10f4065640 0x7f10f4113b60 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f10f000b790 tx=0x7f10f0009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:13:51.563 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:13:51.548+0000 7f10eaffd700 1 -- 172.21.15.162:0/4247197818 <== mon.0 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f10f000f770 con 0x7f10f4065640 2024-04-27T04:13:51.563 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:13:51.548+0000 7f10eaffd700 1 -- 172.21.15.162:0/4247197818 <== mon.0 v2:172.21.15.162:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f10f000fda0 con 0x7f10f4065640 2024-04-27T04:13:51.563 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:13:51.548+0000 7f10eaffd700 1 -- 172.21.15.162:0/4247197818 <== mon.0 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f10f000d310 con 0x7f10f4065640 2024-04-27T04:13:51.563 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:13:51.548+0000 7f10fc3af700 1 -- 172.21.15.162:0/4247197818 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f10f410ddc0 con 0x7f10f4065640 2024-04-27T04:13:51.563 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:13:51.548+0000 7f10fc3af700 1 -- 172.21.15.162:0/4247197818 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f10f410e380 con 0x7f10f4065640 2024-04-27T04:13:51.563 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:13:51.549+0000 7f10fc3af700 1 -- 172.21.15.162:0/4247197818 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f10f40fe540 con 0x7f10f4065640 2024-04-27T04:13:51.563 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:13:51.550+0000 7f10eaffd700 1 -- 172.21.15.162:0/4247197818 <== mon.0 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 32) v1 ==== 92469+0+0 (secure 0 0 0) 0x7f10f0010420 con 0x7f10f4065640 2024-04-27T04:13:51.563 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:13:51.550+0000 7f10eaffd700 1 --2- 172.21.15.162:0/4247197818 >> [v2:172.21.15.162:6800/901992993,v1:172.21.15.162:6801/901992993] conn(0x7f10e006e500 0x7f10e00709b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:13:51.563 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:13:51.550+0000 7f10eaffd700 1 -- 172.21.15.162:0/4247197818 <== mon.0 v2:172.21.15.162:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6152+0+0 (secure 0 0 0) 0x7f10f008de70 con 0x7f10f4065640 2024-04-27T04:13:51.563 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:13:51.550+0000 7f10f994a700 1 --2- 172.21.15.162:0/4247197818 >> [v2:172.21.15.162:6800/901992993,v1:172.21.15.162:6801/901992993] conn(0x7f10e006e500 0x7f10e00709b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:13:51.563 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:13:51.550+0000 7f10f994a700 1 --2- 172.21.15.162:0/4247197818 >> [v2:172.21.15.162:6800/901992993,v1:172.21.15.162:6801/901992993] conn(0x7f10e006e500 0x7f10e00709b0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f10e400b3b0 tx=0x7f10e4009e30 comp rx=0 tx=0).ready entity=mgr.14680 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:13:51.563 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:13:51.554+0000 7f10eaffd700 1 -- 172.21.15.162:0/4247197818 <== mon.0 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186652 (secure 0 0 0) 0x7f10f005b0c0 con 0x7f10f4065640 2024-04-27T04:13:51.750 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:13:51.752+0000 7f10fc3af700 1 -- 172.21.15.162:0/4247197818 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command([{prefix=config set, name=mon_warn_on_insecure_global_id_reclaim}] v 0) v1 -- 0x7f10f405f5c0 con 0x7f10f4065640 2024-04-27T04:13:51.750 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:13:51.752+0000 7f10eaffd700 1 -- 172.21.15.162:0/4247197818 <== mon.0 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_warn_on_insecure_global_id_reclaim}]=0 v52)=0 v52) v1 ==== 155+0+0 (secure 0 0 0) 0x7f10f0055e60 con 0x7f10f4065640 2024-04-27T04:13:51.753 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:13:51.754+0000 7f10fc3af700 1 -- 172.21.15.162:0/4247197818 >> [v2:172.21.15.162:6800/901992993,v1:172.21.15.162:6801/901992993] conn(0x7f10e006e500 msgr2=0x7f10e00709b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:13:51.753 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:13:51.754+0000 7f10fc3af700 1 --2- 172.21.15.162:0/4247197818 >> [v2:172.21.15.162:6800/901992993,v1:172.21.15.162:6801/901992993] conn(0x7f10e006e500 0x7f10e00709b0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f10e400b3b0 tx=0x7f10e4009e30 comp rx=0 tx=0).stop 2024-04-27T04:13:51.753 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:13:51.754+0000 7f10fc3af700 1 -- 172.21.15.162:0/4247197818 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f10f4065640 msgr2=0x7f10f4113b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:13:51.753 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:13:51.754+0000 7f10fc3af700 1 --2- 172.21.15.162:0/4247197818 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f10f4065640 0x7f10f4113b60 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f10f000b790 tx=0x7f10f0009250 comp rx=0 tx=0).stop 2024-04-27T04:13:51.753 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:13:51.754+0000 7f10fc3af700 1 -- 172.21.15.162:0/4247197818 shutdown_connections 2024-04-27T04:13:51.753 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:13:51.754+0000 7f10fc3af700 1 --2- 172.21.15.162:0/4247197818 >> [v2:172.21.15.162:6800/901992993,v1:172.21.15.162:6801/901992993] conn(0x7f10e006e500 0x7f10e00709b0 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:13:51.753 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:13:51.754+0000 7f10fc3af700 1 --2- 172.21.15.162:0/4247197818 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f10f4065640 0x7f10f4113b60 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:13:51.753 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:13:51.754+0000 7f10fc3af700 1 --2- 172.21.15.162:0/4247197818 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f10f4102e50 0x7f10f41140a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:13:51.753 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:13:51.754+0000 7f10fc3af700 1 -- 172.21.15.162:0/4247197818 >> 172.21.15.162:0/4247197818 conn(0x7f10f4071d00 msgr2=0x7f10f40fa000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:13:51.753 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:13:51.755+0000 7f10fc3af700 1 -- 172.21.15.162:0/4247197818 shutdown_connections 2024-04-27T04:13:51.753 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:13:51.755+0000 7f10fc3af700 1 -- 172.21.15.162:0/4247197818 wait complete. 2024-04-27T04:13:52.344 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:13:52 smithi162 ceph-mon[26841]: pgmap v72: 65 pgs: 65 active+clean; 208 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:13:52.345 DEBUG:teuthology.orchestra.run.smithi162:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 7f9750d6-044a-11ef-bc93-c7b262605968 -e sha1=4e213a58e787437ff508e6c890ccd5fd32dc96b5 -- bash -c 'ceph config set mon mon_warn_on_insecure_global_id_reclaim_allowed false --force' 2024-04-27T04:13:52.422 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:13:52 smithi181 ceph-mon[33774]: pgmap v72: 65 pgs: 65 active+clean; 208 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:13:52.616 INFO:teuthology.orchestra.run.smithi162.stderr:Inferring config /var/lib/ceph/7f9750d6-044a-11ef-bc93-c7b262605968/mon.smithi162/config 2024-04-27T04:13:53.386 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:13:53 smithi162 ceph-mon[26841]: pgmap v73: 65 pgs: 65 active+clean; 208 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:13:53.422 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:13:53 smithi181 ceph-mon[33774]: pgmap v73: 65 pgs: 65 active+clean; 208 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:13:54.253 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:13:54.253+0000 7f3f01c1d700 1 -- 172.21.15.162:0/2163455794 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3efc0fbd40 msgr2=0x7f3efc0fc1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:13:54.254 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:13:54.253+0000 7f3f01c1d700 1 --2- 172.21.15.162:0/2163455794 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3efc0fbd40 0x7f3efc0fc1b0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f3eec004940 tx=0x7f3eec009e30 comp rx=0 tx=0).stop 2024-04-27T04:13:54.254 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:13:54.253+0000 7f3f01c1d700 1 -- 172.21.15.162:0/2163455794 shutdown_connections 2024-04-27T04:13:54.254 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:13:54.253+0000 7f3f01c1d700 1 --2- 172.21.15.162:0/2163455794 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3efc0fbd40 0x7f3efc0fc1b0 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:13:54.254 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:13:54.253+0000 7f3f01c1d700 1 --2- 172.21.15.162:0/2163455794 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3efc101d90 0x7f3efc102160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:13:54.254 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:13:54.253+0000 7f3f01c1d700 1 -- 172.21.15.162:0/2163455794 >> 172.21.15.162:0/2163455794 conn(0x7f3efc071d00 msgr2=0x7f3efc072100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:13:54.254 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:13:54.253+0000 7f3f01c1d700 1 -- 172.21.15.162:0/2163455794 shutdown_connections 2024-04-27T04:13:54.254 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:13:54.253+0000 7f3f01c1d700 1 -- 172.21.15.162:0/2163455794 wait complete. 2024-04-27T04:13:54.254 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:13:54.254+0000 7f3f01c1d700 1 Processor -- start 2024-04-27T04:13:54.254 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:13:54.254+0000 7f3f01c1d700 1 -- start start 2024-04-27T04:13:54.254 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:13:54.254+0000 7f3f01c1d700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3efc0fbd40 0x7f3efc10eb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:13:54.254 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:13:54.254+0000 7f3f01c1d700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3efc101d90 0x7f3efc10f080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:13:54.254 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:13:54.254+0000 7f3f01c1d700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3efc10f6d0 con 0x7f3efc0fbd40 2024-04-27T04:13:54.254 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:13:54.254+0000 7f3f01c1d700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3efc10f810 con 0x7f3efc101d90 2024-04-27T04:13:54.255 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:13:54.254+0000 7f3efb7fe700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3efc0fbd40 0x7f3efc10eb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:13:54.255 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:13:54.254+0000 7f3efb7fe700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3efc0fbd40 0x7f3efc10eb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:51432/0 (socket says 172.21.15.162:51432) 2024-04-27T04:13:54.255 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:13:54.254+0000 7f3efb7fe700 1 -- 172.21.15.162:0/1651137452 learned_addr learned my addr 172.21.15.162:0/1651137452 (peer_addr_for_me v2:172.21.15.162:0/0) 2024-04-27T04:13:54.255 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:13:54.254+0000 7f3efaffd700 1 --2- 172.21.15.162:0/1651137452 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3efc101d90 0x7f3efc10f080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:13:54.255 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:13:54.255+0000 7f3efb7fe700 1 -- 172.21.15.162:0/1651137452 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3efc101d90 msgr2=0x7f3efc10f080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:13:54.255 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:13:54.255+0000 7f3efb7fe700 1 --2- 172.21.15.162:0/1651137452 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3efc101d90 0x7f3efc10f080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:13:54.255 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:13:54.255+0000 7f3efb7fe700 1 -- 172.21.15.162:0/1651137452 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3eec005040 con 0x7f3efc0fbd40 2024-04-27T04:13:54.255 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:13:54.255+0000 7f3efb7fe700 1 --2- 172.21.15.162:0/1651137452 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3efc0fbd40 0x7f3efc10eb40 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f3ef0002a50 tx=0x7f3ef00046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:13:54.255 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:13:54.255+0000 7f3ee3fff700 1 -- 172.21.15.162:0/1651137452 <== mon.0 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3ef0007cc0 con 0x7f3efc0fbd40 2024-04-27T04:13:54.255 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:13:54.255+0000 7f3ee3fff700 1 -- 172.21.15.162:0/1651137452 <== mon.0 v2:172.21.15.162:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f3ef000f040 con 0x7f3efc0fbd40 2024-04-27T04:13:54.255 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:13:54.255+0000 7f3f01c1d700 1 -- 172.21.15.162:0/1651137452 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3efc117bb0 con 0x7f3efc0fbd40 2024-04-27T04:13:54.255 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:13:54.255+0000 7f3ee3fff700 1 -- 172.21.15.162:0/1651137452 <== mon.0 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3ef000da30 con 0x7f3efc0fbd40 2024-04-27T04:13:54.255 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:13:54.255+0000 7f3f01c1d700 1 -- 172.21.15.162:0/1651137452 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3efc118110 con 0x7f3efc0fbd40 2024-04-27T04:13:54.256 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:13:54.256+0000 7f3f01c1d700 1 -- 172.21.15.162:0/1651137452 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3efc0fd430 con 0x7f3efc0fbd40 2024-04-27T04:13:54.256 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:13:54.257+0000 7f3ee3fff700 1 -- 172.21.15.162:0/1651137452 <== mon.0 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 32) v1 ==== 92469+0+0 (secure 0 0 0) 0x7f3ef000db90 con 0x7f3efc0fbd40 2024-04-27T04:13:54.256 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:13:54.257+0000 7f3ee3fff700 1 --2- 172.21.15.162:0/1651137452 >> [v2:172.21.15.162:6800/901992993,v1:172.21.15.162:6801/901992993] conn(0x7f3ee406e500 0x7f3ee40709b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:13:54.257 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:13:54.257+0000 7f3ee3fff700 1 -- 172.21.15.162:0/1651137452 <== mon.0 v2:172.21.15.162:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6152+0+0 (secure 0 0 0) 0x7f3ef008dbf0 con 0x7f3efc0fbd40 2024-04-27T04:13:54.257 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:13:54.257+0000 7f3efaffd700 1 --2- 172.21.15.162:0/1651137452 >> [v2:172.21.15.162:6800/901992993,v1:172.21.15.162:6801/901992993] conn(0x7f3ee406e500 0x7f3ee40709b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:13:54.257 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:13:54.258+0000 7f3efaffd700 1 --2- 172.21.15.162:0/1651137452 >> [v2:172.21.15.162:6800/901992993,v1:172.21.15.162:6801/901992993] conn(0x7f3ee406e500 0x7f3ee40709b0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f3efc065fe0 tx=0x7f3eec009e30 comp rx=0 tx=0).ready entity=mgr.14680 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:13:54.260 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:13:54.262+0000 7f3ee3fff700 1 -- 172.21.15.162:0/1651137452 <== mon.0 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186652 (secure 0 0 0) 0x7f3ef0057270 con 0x7f3efc0fbd40 2024-04-27T04:13:54.461 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:13:54.462+0000 7f3f01c1d700 1 -- 172.21.15.162:0/1651137452 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command([{prefix=config set, name=mon_warn_on_insecure_global_id_reclaim_allowed}] v 0) v1 -- 0x7f3efc05f5c0 con 0x7f3efc0fbd40 2024-04-27T04:13:54.461 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:13:54.463+0000 7f3ee3fff700 1 -- 172.21.15.162:0/1651137452 <== mon.0 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_warn_on_insecure_global_id_reclaim_allowed}]=0 v52)=0 v52) v1 ==== 163+0+0 (secure 0 0 0) 0x7f3ef005a930 con 0x7f3efc0fbd40 2024-04-27T04:13:54.463 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:13:54.465+0000 7f3f01c1d700 1 -- 172.21.15.162:0/1651137452 >> [v2:172.21.15.162:6800/901992993,v1:172.21.15.162:6801/901992993] conn(0x7f3ee406e500 msgr2=0x7f3ee40709b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:13:54.463 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:13:54.465+0000 7f3f01c1d700 1 --2- 172.21.15.162:0/1651137452 >> [v2:172.21.15.162:6800/901992993,v1:172.21.15.162:6801/901992993] conn(0x7f3ee406e500 0x7f3ee40709b0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f3efc065fe0 tx=0x7f3eec009e30 comp rx=0 tx=0).stop 2024-04-27T04:13:54.463 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:13:54.465+0000 7f3f01c1d700 1 -- 172.21.15.162:0/1651137452 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3efc0fbd40 msgr2=0x7f3efc10eb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:13:54.464 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:13:54.465+0000 7f3f01c1d700 1 --2- 172.21.15.162:0/1651137452 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3efc0fbd40 0x7f3efc10eb40 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f3ef0002a50 tx=0x7f3ef00046b0 comp rx=0 tx=0).stop 2024-04-27T04:13:54.464 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:13:54.465+0000 7f3f01c1d700 1 -- 172.21.15.162:0/1651137452 shutdown_connections 2024-04-27T04:13:54.464 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:13:54.465+0000 7f3f01c1d700 1 --2- 172.21.15.162:0/1651137452 >> [v2:172.21.15.162:6800/901992993,v1:172.21.15.162:6801/901992993] conn(0x7f3ee406e500 0x7f3ee40709b0 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:13:54.464 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:13:54.465+0000 7f3f01c1d700 1 --2- 172.21.15.162:0/1651137452 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3efc0fbd40 0x7f3efc10eb40 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:13:54.464 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:13:54.465+0000 7f3f01c1d700 1 --2- 172.21.15.162:0/1651137452 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3efc101d90 0x7f3efc10f080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:13:54.464 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:13:54.465+0000 7f3f01c1d700 1 -- 172.21.15.162:0/1651137452 >> 172.21.15.162:0/1651137452 conn(0x7f3efc071d00 msgr2=0x7f3efc0fa0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:13:54.464 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:13:54.465+0000 7f3f01c1d700 1 -- 172.21.15.162:0/1651137452 shutdown_connections 2024-04-27T04:13:54.464 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:13:54.465+0000 7f3f01c1d700 1 -- 172.21.15.162:0/1651137452 wait complete. 2024-04-27T04:13:55.188 DEBUG:teuthology.orchestra.run.smithi162:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 7f9750d6-044a-11ef-bc93-c7b262605968 -e sha1=4e213a58e787437ff508e6c890ccd5fd32dc96b5 -- bash -c 'ceph config set global log_to_journald false --force' 2024-04-27T04:13:55.445 INFO:teuthology.orchestra.run.smithi162.stderr:Inferring config /var/lib/ceph/7f9750d6-044a-11ef-bc93-c7b262605968/mon.smithi162/config 2024-04-27T04:13:55.568 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:13:55 smithi162 ceph-mon[26841]: pgmap v74: 65 pgs: 65 active+clean; 208 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:13:55.672 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:13:55 smithi181 ceph-mon[33774]: pgmap v74: 65 pgs: 65 active+clean; 208 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:13:57.059 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:13:57.059+0000 7fe78ce9d700 1 -- 172.21.15.162:0/374492186 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe788101d90 msgr2=0x7fe788102160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:13:57.060 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:13:57.059+0000 7fe78ce9d700 1 --2- 172.21.15.162:0/374492186 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe788101d90 0x7fe788102160 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7fe778009ce0 tx=0x7fe778004d10 comp rx=0 tx=0).stop 2024-04-27T04:13:57.060 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:13:57.059+0000 7fe78ce9d700 1 -- 172.21.15.162:0/374492186 shutdown_connections 2024-04-27T04:13:57.060 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:13:57.059+0000 7fe78ce9d700 1 --2- 172.21.15.162:0/374492186 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe7880fbd40 0x7fe7880fc1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:13:57.060 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:13:57.059+0000 7fe78ce9d700 1 --2- 172.21.15.162:0/374492186 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe788101d90 0x7fe788102160 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:13:57.060 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:13:57.059+0000 7fe78ce9d700 1 -- 172.21.15.162:0/374492186 >> 172.21.15.162:0/374492186 conn(0x7fe788071d00 msgr2=0x7fe788072100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:13:57.060 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:13:57.059+0000 7fe78ce9d700 1 -- 172.21.15.162:0/374492186 shutdown_connections 2024-04-27T04:13:57.060 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:13:57.059+0000 7fe78ce9d700 1 -- 172.21.15.162:0/374492186 wait complete. 2024-04-27T04:13:57.060 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:13:57.059+0000 7fe78ce9d700 1 Processor -- start 2024-04-27T04:13:57.060 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:13:57.060+0000 7fe78ce9d700 1 -- start start 2024-04-27T04:13:57.060 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:13:57.060+0000 7fe78ce9d700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe7880fbd40 0x7fe788112c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:13:57.060 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:13:57.060+0000 7fe78ce9d700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe788101d90 0x7fe788113140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:13:57.060 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:13:57.060+0000 7fe78ce9d700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe788113790 con 0x7fe788101d90 2024-04-27T04:13:57.060 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:13:57.060+0000 7fe78ce9d700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe78810cb80 con 0x7fe7880fbd40 2024-04-27T04:13:57.061 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:13:57.060+0000 7fe785d9b700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe788101d90 0x7fe788113140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:13:57.061 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:13:57.060+0000 7fe785d9b700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe788101d90 0x7fe788113140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:51454/0 (socket says 172.21.15.162:51454) 2024-04-27T04:13:57.061 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:13:57.060+0000 7fe785d9b700 1 -- 172.21.15.162:0/2804611699 learned_addr learned my addr 172.21.15.162:0/2804611699 (peer_addr_for_me v2:172.21.15.162:0/0) 2024-04-27T04:13:57.061 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:13:57.060+0000 7fe78659c700 1 --2- 172.21.15.162:0/2804611699 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe7880fbd40 0x7fe788112c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:13:57.061 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:13:57.060+0000 7fe785d9b700 1 -- 172.21.15.162:0/2804611699 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe7880fbd40 msgr2=0x7fe788112c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:13:57.061 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:13:57.060+0000 7fe785d9b700 1 --2- 172.21.15.162:0/2804611699 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe7880fbd40 0x7fe788112c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:13:57.061 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:13:57.060+0000 7fe785d9b700 1 -- 172.21.15.162:0/2804611699 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe778009940 con 0x7fe788101d90 2024-04-27T04:13:57.061 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:13:57.060+0000 7fe785d9b700 1 --2- 172.21.15.162:0/2804611699 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe788101d90 0x7fe788113140 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7fe77000c830 tx=0x7fe770004470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:13:57.061 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:13:57.060+0000 7fe77effd700 1 -- 172.21.15.162:0/2804611699 <== mon.0 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe77000a760 con 0x7fe788101d90 2024-04-27T04:13:57.061 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:13:57.060+0000 7fe77effd700 1 -- 172.21.15.162:0/2804611699 <== mon.0 v2:172.21.15.162:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fe77000ad90 con 0x7fe788101d90 2024-04-27T04:13:57.061 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:13:57.060+0000 7fe77effd700 1 -- 172.21.15.162:0/2804611699 <== mon.0 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe7700113a0 con 0x7fe788101d90 2024-04-27T04:13:57.061 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:13:57.060+0000 7fe78ce9d700 1 -- 172.21.15.162:0/2804611699 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe78810ce60 con 0x7fe788101d90 2024-04-27T04:13:57.061 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:13:57.061+0000 7fe78ce9d700 1 -- 172.21.15.162:0/2804611699 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe78810d2f0 con 0x7fe788101d90 2024-04-27T04:13:57.062 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:13:57.062+0000 7fe78ce9d700 1 -- 172.21.15.162:0/2804611699 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe7880fd430 con 0x7fe788101d90 2024-04-27T04:13:57.062 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:13:57.062+0000 7fe77effd700 1 -- 172.21.15.162:0/2804611699 <== mon.0 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 32) v1 ==== 92469+0+0 (secure 0 0 0) 0x7fe770012400 con 0x7fe788101d90 2024-04-27T04:13:57.062 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:13:57.062+0000 7fe77effd700 1 --2- 172.21.15.162:0/2804611699 >> [v2:172.21.15.162:6800/901992993,v1:172.21.15.162:6801/901992993] conn(0x7fe77406e500 0x7fe7740709b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:13:57.063 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:13:57.063+0000 7fe77effd700 1 -- 172.21.15.162:0/2804611699 <== mon.0 v2:172.21.15.162:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6152+0+0 (secure 0 0 0) 0x7fe77008df90 con 0x7fe788101d90 2024-04-27T04:13:57.063 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:13:57.063+0000 7fe78659c700 1 --2- 172.21.15.162:0/2804611699 >> [v2:172.21.15.162:6800/901992993,v1:172.21.15.162:6801/901992993] conn(0x7fe77406e500 0x7fe7740709b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:13:57.063 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:13:57.063+0000 7fe78659c700 1 --2- 172.21.15.162:0/2804611699 >> [v2:172.21.15.162:6800/901992993,v1:172.21.15.162:6801/901992993] conn(0x7fe77406e500 0x7fe7740709b0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fe77800b740 tx=0x7fe778004d10 comp rx=0 tx=0).ready entity=mgr.14680 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:13:57.066 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:13:57.067+0000 7fe77effd700 1 -- 172.21.15.162:0/2804611699 <== mon.0 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186652 (secure 0 0 0) 0x7fe77005b140 con 0x7fe788101d90 2024-04-27T04:13:57.266 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:13:57.266+0000 7fe78ce9d700 1 -- 172.21.15.162:0/2804611699 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command([{prefix=config set, name=log_to_journald}] v 0) v1 -- 0x7fe78804bf40 con 0x7fe788101d90 2024-04-27T04:13:57.266 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:13:57.267+0000 7fe77effd700 1 -- 172.21.15.162:0/2804611699 <== mon.0 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{prefix=config set, name=log_to_journald}]=0 v52)=0 v52) v1 ==== 135+0+0 (secure 0 0 0) 0x7fe77005acd0 con 0x7fe788101d90 2024-04-27T04:13:57.268 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:13:57.269+0000 7fe78ce9d700 1 -- 172.21.15.162:0/2804611699 >> [v2:172.21.15.162:6800/901992993,v1:172.21.15.162:6801/901992993] conn(0x7fe77406e500 msgr2=0x7fe7740709b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:13:57.268 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:13:57.269+0000 7fe78ce9d700 1 --2- 172.21.15.162:0/2804611699 >> [v2:172.21.15.162:6800/901992993,v1:172.21.15.162:6801/901992993] conn(0x7fe77406e500 0x7fe7740709b0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fe77800b740 tx=0x7fe778004d10 comp rx=0 tx=0).stop 2024-04-27T04:13:57.268 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:13:57.269+0000 7fe78ce9d700 1 -- 172.21.15.162:0/2804611699 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe788101d90 msgr2=0x7fe788113140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:13:57.268 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:13:57.269+0000 7fe78ce9d700 1 --2- 172.21.15.162:0/2804611699 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe788101d90 0x7fe788113140 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7fe77000c830 tx=0x7fe770004470 comp rx=0 tx=0).stop 2024-04-27T04:13:57.268 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:13:57.269+0000 7fe78ce9d700 1 -- 172.21.15.162:0/2804611699 shutdown_connections 2024-04-27T04:13:57.269 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:13:57.269+0000 7fe78ce9d700 1 --2- 172.21.15.162:0/2804611699 >> [v2:172.21.15.162:6800/901992993,v1:172.21.15.162:6801/901992993] conn(0x7fe77406e500 0x7fe7740709b0 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:13:57.269 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:13:57.269+0000 7fe78ce9d700 1 --2- 172.21.15.162:0/2804611699 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe7880fbd40 0x7fe788112c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:13:57.269 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:13:57.269+0000 7fe78ce9d700 1 --2- 172.21.15.162:0/2804611699 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe788101d90 0x7fe788113140 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:13:57.269 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:13:57.269+0000 7fe78ce9d700 1 -- 172.21.15.162:0/2804611699 >> 172.21.15.162:0/2804611699 conn(0x7fe788071d00 msgr2=0x7fe7880fa140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:13:57.269 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:13:57.269+0000 7fe78ce9d700 1 -- 172.21.15.162:0/2804611699 shutdown_connections 2024-04-27T04:13:57.269 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:13:57.269+0000 7fe78ce9d700 1 -- 172.21.15.162:0/2804611699 wait complete. 2024-04-27T04:13:57.598 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:13:57 smithi162 ceph-mon[26841]: pgmap v75: 65 pgs: 65 active+clean; 208 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:13:57.672 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:13:57 smithi181 ceph-mon[33774]: pgmap v75: 65 pgs: 65 active+clean; 208 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:13:57.896 DEBUG:teuthology.orchestra.run.smithi162:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 7f9750d6-044a-11ef-bc93-c7b262605968 -e sha1=4e213a58e787437ff508e6c890ccd5fd32dc96b5 -- bash -c 'ceph orch upgrade start --image quay.ceph.io/ceph-ci/ceph:$sha1' 2024-04-27T04:13:58.158 INFO:teuthology.orchestra.run.smithi162.stderr:Inferring config /var/lib/ceph/7f9750d6-044a-11ef-bc93-c7b262605968/mon.smithi162/config 2024-04-27T04:13:59.574 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:13:59 smithi162 ceph-mon[26841]: pgmap v76: 65 pgs: 65 active+clean; 208 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-04-27T04:13:59.672 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:13:59 smithi181 ceph-mon[33774]: pgmap v76: 65 pgs: 65 active+clean; 208 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-04-27T04:13:59.780 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:13:59.781+0000 7f25cd7a9700 1 -- 172.21.15.162:0/1418297619 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f25c8070b90 msgr2=0x7f25c8109170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:13:59.780 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:13:59.781+0000 7f25cd7a9700 1 --2- 172.21.15.162:0/1418297619 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f25c8070b90 0x7f25c8109170 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f25bc004940 tx=0x7f25bc009e30 comp rx=0 tx=0).stop 2024-04-27T04:13:59.780 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:13:59.781+0000 7f25cd7a9700 1 -- 172.21.15.162:0/1418297619 shutdown_connections 2024-04-27T04:13:59.780 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:13:59.781+0000 7f25cd7a9700 1 --2- 172.21.15.162:0/1418297619 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f25c8070b90 0x7f25c8109170 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:13:59.780 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:13:59.781+0000 7f25cd7a9700 1 --2- 172.21.15.162:0/1418297619 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f25c8070280 0x7f25c8070650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:13:59.780 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:13:59.781+0000 7f25cd7a9700 1 -- 172.21.15.162:0/1418297619 >> 172.21.15.162:0/1418297619 conn(0x7f25c80f8990 msgr2=0x7f25c80fada0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:13:59.781 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:13:59.781+0000 7f25cd7a9700 1 -- 172.21.15.162:0/1418297619 shutdown_connections 2024-04-27T04:13:59.781 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:13:59.781+0000 7f25cd7a9700 1 -- 172.21.15.162:0/1418297619 wait complete. 2024-04-27T04:13:59.782 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:13:59.781+0000 7f25cd7a9700 1 Processor -- start 2024-04-27T04:13:59.782 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:13:59.781+0000 7f25cd7a9700 1 -- start start 2024-04-27T04:13:59.782 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:13:59.782+0000 7f25cd7a9700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f25c8070280 0x7f25c8115c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:13:59.782 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:13:59.782+0000 7f25cd7a9700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f25c8070b90 0x7f25c8116140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:13:59.783 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:13:59.782+0000 7f25cd7a9700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25c8116790 con 0x7f25c8070b90 2024-04-27T04:13:59.783 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:13:59.782+0000 7f25cd7a9700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25c810fd70 con 0x7f25c8070280 2024-04-27T04:13:59.783 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:13:59.782+0000 7f25c67fc700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f25c8070b90 0x7f25c8116140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:13:59.783 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:13:59.782+0000 7f25c67fc700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f25c8070b90 0x7f25c8116140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:51464/0 (socket says 172.21.15.162:51464) 2024-04-27T04:13:59.783 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:13:59.782+0000 7f25c67fc700 1 -- 172.21.15.162:0/1277574261 learned_addr learned my addr 172.21.15.162:0/1277574261 (peer_addr_for_me v2:172.21.15.162:0/0) 2024-04-27T04:13:59.783 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:13:59.782+0000 7f25c6ffd700 1 --2- 172.21.15.162:0/1277574261 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f25c8070280 0x7f25c8115c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:13:59.783 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:13:59.782+0000 7f25c67fc700 1 -- 172.21.15.162:0/1277574261 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f25c8070280 msgr2=0x7f25c8115c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:13:59.783 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:13:59.782+0000 7f25c67fc700 1 --2- 172.21.15.162:0/1277574261 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f25c8070280 0x7f25c8115c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:13:59.783 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:13:59.782+0000 7f25c67fc700 1 -- 172.21.15.162:0/1277574261 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25bc005040 con 0x7f25c8070b90 2024-04-27T04:13:59.783 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:13:59.782+0000 7f25c67fc700 1 --2- 172.21.15.162:0/1277574261 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f25c8070b90 0x7f25c8116140 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f25bc00b3b0 tx=0x7f25bc009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:13:59.783 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:13:59.782+0000 7f25af7fe700 1 -- 172.21.15.162:0/1277574261 <== mon.0 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f25bc00e040 con 0x7f25c8070b90 2024-04-27T04:13:59.783 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:13:59.782+0000 7f25cd7a9700 1 -- 172.21.15.162:0/1277574261 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f25c810fff0 con 0x7f25c8070b90 2024-04-27T04:13:59.783 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:13:59.782+0000 7f25af7fe700 1 -- 172.21.15.162:0/1277574261 <== mon.0 v2:172.21.15.162:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f25bc013420 con 0x7f25c8070b90 2024-04-27T04:13:59.783 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:13:59.782+0000 7f25cd7a9700 1 -- 172.21.15.162:0/1277574261 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f25c8110530 con 0x7f25c8070b90 2024-04-27T04:13:59.784 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:13:59.782+0000 7f25af7fe700 1 -- 172.21.15.162:0/1277574261 <== mon.0 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f25bc01c630 con 0x7f25c8070b90 2024-04-27T04:13:59.785 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:13:59.784+0000 7f25cd7a9700 1 -- 172.21.15.162:0/1277574261 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f25c80719c0 con 0x7f25c8070b90 2024-04-27T04:13:59.785 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:13:59.784+0000 7f25af7fe700 1 -- 172.21.15.162:0/1277574261 <== mon.0 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 32) v1 ==== 92469+0+0 (secure 0 0 0) 0x7f25bc013590 con 0x7f25c8070b90 2024-04-27T04:13:59.785 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:13:59.784+0000 7f25af7fe700 1 --2- 172.21.15.162:0/1277574261 >> [v2:172.21.15.162:6800/901992993,v1:172.21.15.162:6801/901992993] conn(0x7f25b006e500 0x7f25b00709b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:13:59.785 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:13:59.784+0000 7f25af7fe700 1 -- 172.21.15.162:0/1277574261 <== mon.0 v2:172.21.15.162:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6152+0+0 (secure 0 0 0) 0x7f25bc08f110 con 0x7f25c8070b90 2024-04-27T04:13:59.785 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:13:59.785+0000 7f25c6ffd700 1 --2- 172.21.15.162:0/1277574261 >> [v2:172.21.15.162:6800/901992993,v1:172.21.15.162:6801/901992993] conn(0x7f25b006e500 0x7f25b00709b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:13:59.785 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:13:59.785+0000 7f25c6ffd700 1 --2- 172.21.15.162:0/1277574261 >> [v2:172.21.15.162:6800/901992993,v1:172.21.15.162:6801/901992993] conn(0x7f25b006e500 0x7f25b00709b0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f25b8004f40 tx=0x7f25b800b040 comp rx=0 tx=0).ready entity=mgr.14680 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:13:59.788 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:13:59.789+0000 7f25af7fe700 1 -- 172.21.15.162:0/1277574261 <== mon.0 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186652 (secure 0 0 0) 0x7f25bc05c2c0 con 0x7f25c8070b90 2024-04-27T04:14:00.011 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:14:00.011+0000 7f25cd7a9700 1 -- 172.21.15.162:0/1277574261 --> [v2:172.21.15.162:6800/901992993,v1:172.21.15.162:6801/901992993] -- mgr_command(tid 0: {"prefix": "orch upgrade start", "image": "quay.ceph.io/ceph-ci/ceph:4e213a58e787437ff508e6c890ccd5fd32dc96b5", "target": ["mon-mgr", ""]}) v1 -- 0x7f25c810c0c0 con 0x7f25b006e500 2024-04-27T04:14:00.016 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:14:00.017+0000 7f25af7fe700 1 -- 172.21.15.162:0/1277574261 <== mgr.14680 v2:172.21.15.162:6800/901992993 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+89 (secure 0 0 0) 0x7f25c810c0c0 con 0x7f25b006e500 2024-04-27T04:14:00.016 INFO:teuthology.orchestra.run.smithi162.stdout:Initiating upgrade to quay.ceph.io/ceph-ci/ceph:4e213a58e787437ff508e6c890ccd5fd32dc96b5 2024-04-27T04:14:00.018 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:14:00.019+0000 7f25cd7a9700 1 -- 172.21.15.162:0/1277574261 >> [v2:172.21.15.162:6800/901992993,v1:172.21.15.162:6801/901992993] conn(0x7f25b006e500 msgr2=0x7f25b00709b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:14:00.019 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:14:00.019+0000 7f25cd7a9700 1 --2- 172.21.15.162:0/1277574261 >> [v2:172.21.15.162:6800/901992993,v1:172.21.15.162:6801/901992993] conn(0x7f25b006e500 0x7f25b00709b0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f25b8004f40 tx=0x7f25b800b040 comp rx=0 tx=0).stop 2024-04-27T04:14:00.019 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:14:00.019+0000 7f25cd7a9700 1 -- 172.21.15.162:0/1277574261 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f25c8070b90 msgr2=0x7f25c8116140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:14:00.019 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:14:00.019+0000 7f25cd7a9700 1 --2- 172.21.15.162:0/1277574261 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f25c8070b90 0x7f25c8116140 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f25bc00b3b0 tx=0x7f25bc009e30 comp rx=0 tx=0).stop 2024-04-27T04:14:00.019 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:14:00.019+0000 7f25cd7a9700 1 -- 172.21.15.162:0/1277574261 shutdown_connections 2024-04-27T04:14:00.019 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:14:00.019+0000 7f25cd7a9700 1 --2- 172.21.15.162:0/1277574261 >> [v2:172.21.15.162:6800/901992993,v1:172.21.15.162:6801/901992993] conn(0x7f25b006e500 0x7f25b00709b0 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:14:00.019 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:14:00.019+0000 7f25cd7a9700 1 --2- 172.21.15.162:0/1277574261 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f25c8070280 0x7f25c8115c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:14:00.019 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:14:00.019+0000 7f25cd7a9700 1 --2- 172.21.15.162:0/1277574261 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f25c8070b90 0x7f25c8116140 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:14:00.019 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:14:00.019+0000 7f25cd7a9700 1 -- 172.21.15.162:0/1277574261 >> 172.21.15.162:0/1277574261 conn(0x7f25c80f8990 msgr2=0x7f25c80f9e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:14:00.019 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:14:00.019+0000 7f25cd7a9700 1 -- 172.21.15.162:0/1277574261 shutdown_connections 2024-04-27T04:14:00.019 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:14:00.019+0000 7f25cd7a9700 1 -- 172.21.15.162:0/1277574261 wait complete. 2024-04-27T04:14:00.669 INFO:teuthology.task.sequential:In sequential, running task cephadm.shell... 2024-04-27T04:14:00.670 INFO:tasks.cephadm:Running commands on role host.a host ubuntu@smithi162.front.sepia.ceph.com 2024-04-27T04:14:00.670 DEBUG:teuthology.orchestra.run.smithi162:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 7f9750d6-044a-11ef-bc93-c7b262605968 -e sha1=4e213a58e787437ff508e6c890ccd5fd32dc96b5 -- bash -c 'while ceph orch upgrade status | jq '"'"'.in_progress'"'"' | grep true && ! ceph orch upgrade status | jq '"'"'.message'"'"' | grep Error ; do ceph orch ps ; ceph versions ; ceph fs dump; ceph orch upgrade status ; ceph health detail ; sleep 30 ; done' 2024-04-27T04:14:00.959 INFO:teuthology.orchestra.run.smithi162.stderr:Inferring config /var/lib/ceph/7f9750d6-044a-11ef-bc93-c7b262605968/mon.smithi162/config 2024-04-27T04:14:01.331 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:14:01 smithi162 ceph-mon[26841]: from='client.14830 -' entity='client.admin' cmd=[{"prefix": "orch upgrade start", "image": "quay.ceph.io/ceph-ci/ceph:4e213a58e787437ff508e6c890ccd5fd32dc96b5", "target": ["mon-mgr", ""]}]: dispatch 2024-04-27T04:14:01.332 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:14:01 smithi162 ceph-mon[26841]: Upgrade: Started with target quay.ceph.io/ceph-ci/ceph:4e213a58e787437ff508e6c890ccd5fd32dc96b5 2024-04-27T04:14:01.332 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:14:01 smithi162 ceph-mon[26841]: from='mgr.14680 172.21.15.162:0/501880828' entity='mgr.smithi162.oennfn' 2024-04-27T04:14:01.332 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:14:01 smithi162 ceph-mon[26841]: from='mgr.14680 172.21.15.162:0/501880828' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T04:14:01.332 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:14:01 smithi162 ceph-mon[26841]: from='mgr.14680 172.21.15.162:0/501880828' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T04:14:01.332 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:14:01 smithi162 ceph-mon[26841]: from='mgr.14680 172.21.15.162:0/501880828' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T04:14:01.332 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:14:01 smithi162 ceph-mon[26841]: from='mgr.14680 172.21.15.162:0/501880828' entity='mgr.smithi162.oennfn' 2024-04-27T04:14:01.422 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:14:01 smithi181 ceph-mon[33774]: from='client.14830 -' entity='client.admin' cmd=[{"prefix": "orch upgrade start", "image": "quay.ceph.io/ceph-ci/ceph:4e213a58e787437ff508e6c890ccd5fd32dc96b5", "target": ["mon-mgr", ""]}]: dispatch 2024-04-27T04:14:01.422 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:14:01 smithi181 ceph-mon[33774]: Upgrade: Started with target quay.ceph.io/ceph-ci/ceph:4e213a58e787437ff508e6c890ccd5fd32dc96b5 2024-04-27T04:14:01.422 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:14:01 smithi181 ceph-mon[33774]: from='mgr.14680 172.21.15.162:0/501880828' entity='mgr.smithi162.oennfn' 2024-04-27T04:14:01.422 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:14:01 smithi181 ceph-mon[33774]: from='mgr.14680 172.21.15.162:0/501880828' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T04:14:01.423 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:14:01 smithi181 ceph-mon[33774]: from='mgr.14680 172.21.15.162:0/501880828' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T04:14:01.423 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:14:01 smithi181 ceph-mon[33774]: from='mgr.14680 172.21.15.162:0/501880828' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T04:14:01.423 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:14:01 smithi181 ceph-mon[33774]: from='mgr.14680 172.21.15.162:0/501880828' entity='mgr.smithi162.oennfn' 2024-04-27T04:14:02.323 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:14:02 smithi181 ceph-mon[33774]: Upgrade: First pull of quay.ceph.io/ceph-ci/ceph:4e213a58e787437ff508e6c890ccd5fd32dc96b5 2024-04-27T04:14:02.323 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:14:02 smithi181 ceph-mon[33774]: pgmap v77: 65 pgs: 65 active+clean; 208 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:14:02.387 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:14:02 smithi162 ceph-mon[26841]: Upgrade: First pull of quay.ceph.io/ceph-ci/ceph:4e213a58e787437ff508e6c890ccd5fd32dc96b5 2024-04-27T04:14:02.387 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:14:02 smithi162 ceph-mon[26841]: pgmap v77: 65 pgs: 65 active+clean; 208 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:14:03.275 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:14:03 smithi162 ceph-mon[26841]: from='mgr.14680 172.21.15.162:0/501880828' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-27T04:14:03.383 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:14:03.376+0000 7f13b32bb700 1 -- 172.21.15.162:0/2987949183 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f13ac101d00 msgr2=0x7f13ac1020d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:14:03.383 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:14:03.376+0000 7f13b32bb700 1 --2- 172.21.15.162:0/2987949183 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f13ac101d00 0x7f13ac1020d0 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f13a8009ce0 tx=0x7f13a8004d10 comp rx=0 tx=0).stop 2024-04-27T04:14:03.383 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:14:03.376+0000 7f13b32bb700 1 -- 172.21.15.162:0/2987949183 shutdown_connections 2024-04-27T04:14:03.384 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:14:03.376+0000 7f13b32bb700 1 --2- 172.21.15.162:0/2987949183 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f13ac1026a0 0x7f13ac0fbd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:14:03.384 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:14:03.376+0000 7f13b32bb700 1 --2- 172.21.15.162:0/2987949183 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f13ac101d00 0x7f13ac1020d0 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:14:03.384 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:14:03.376+0000 7f13b32bb700 1 -- 172.21.15.162:0/2987949183 >> 172.21.15.162:0/2987949183 conn(0x7f13ac0f7870 msgr2=0x7f13ac0f9c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:14:03.384 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:14:03.376+0000 7f13b32bb700 1 -- 172.21.15.162:0/2987949183 shutdown_connections 2024-04-27T04:14:03.384 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:14:03.376+0000 7f13b32bb700 1 -- 172.21.15.162:0/2987949183 wait complete. 2024-04-27T04:14:03.384 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:14:03.377+0000 7f13b32bb700 1 Processor -- start 2024-04-27T04:14:03.384 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:14:03.377+0000 7f13b32bb700 1 -- start start 2024-04-27T04:14:03.384 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:14:03.377+0000 7f13b32bb700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f13ac101d00 0x7f13ac112a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:14:03.384 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:14:03.377+0000 7f13b32bb700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f13ac1026a0 0x7f13ac112f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:14:03.384 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:14:03.377+0000 7f13b32bb700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13ac113650 con 0x7f13ac101d00 2024-04-27T04:14:03.384 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:14:03.377+0000 7f13b32bb700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13ac10cba0 con 0x7f13ac1026a0 2024-04-27T04:14:03.384 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:14:03.377+0000 7f13b1057700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f13ac101d00 0x7f13ac112a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:14:03.384 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:14:03.377+0000 7f13b1057700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f13ac101d00 0x7f13ac112a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:51482/0 (socket says 172.21.15.162:51482) 2024-04-27T04:14:03.384 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:14:03.377+0000 7f13b1057700 1 -- 172.21.15.162:0/2219244277 learned_addr learned my addr 172.21.15.162:0/2219244277 (peer_addr_for_me v2:172.21.15.162:0/0) 2024-04-27T04:14:03.385 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:14:03.377+0000 7f13b0856700 1 --2- 172.21.15.162:0/2219244277 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f13ac1026a0 0x7f13ac112f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:14:03.385 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:14:03.377+0000 7f13b1057700 1 -- 172.21.15.162:0/2219244277 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f13ac1026a0 msgr2=0x7f13ac112f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:14:03.385 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:14:03.377+0000 7f13b1057700 1 --2- 172.21.15.162:0/2219244277 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f13ac1026a0 0x7f13ac112f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:14:03.385 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:14:03.377+0000 7f13b1057700 1 -- 172.21.15.162:0/2219244277 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f13a8009940 con 0x7f13ac101d00 2024-04-27T04:14:03.385 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:14:03.377+0000 7f13b1057700 1 --2- 172.21.15.162:0/2219244277 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f13ac101d00 0x7f13ac112a30 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f13a800be90 tx=0x7f13a8004d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:14:03.385 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:14:03.377+0000 7f13a1ffb700 1 -- 172.21.15.162:0/2219244277 <== mon.0 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f13a800e040 con 0x7f13ac101d00 2024-04-27T04:14:03.385 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:14:03.377+0000 7f13a1ffb700 1 -- 172.21.15.162:0/2219244277 <== mon.0 v2:172.21.15.162:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f13a8008ea0 con 0x7f13ac101d00 2024-04-27T04:14:03.385 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:14:03.377+0000 7f13a1ffb700 1 -- 172.21.15.162:0/2219244277 <== mon.0 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f13a801c810 con 0x7f13ac101d00 2024-04-27T04:14:03.385 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:14:03.377+0000 7f13b32bb700 1 -- 172.21.15.162:0/2219244277 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f13ac10cdc0 con 0x7f13ac101d00 2024-04-27T04:14:03.385 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:14:03.378+0000 7f13b32bb700 1 -- 172.21.15.162:0/2219244277 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f13ac10d380 con 0x7f13ac101d00 2024-04-27T04:14:03.385 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:14:03.378+0000 7f13b32bb700 1 -- 172.21.15.162:0/2219244277 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f13ac0fd450 con 0x7f13ac101d00 2024-04-27T04:14:03.385 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:14:03.379+0000 7f13a1ffb700 1 -- 172.21.15.162:0/2219244277 <== mon.0 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 32) v1 ==== 92469+0+0 (secure 0 0 0) 0x7f13a8012c00 con 0x7f13ac101d00 2024-04-27T04:14:03.385 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:14:03.380+0000 7f13a1ffb700 1 --2- 172.21.15.162:0/2219244277 >> [v2:172.21.15.162:6800/901992993,v1:172.21.15.162:6801/901992993] conn(0x7f139806e500 0x7f13980709b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:14:03.385 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:14:03.380+0000 7f13a1ffb700 1 -- 172.21.15.162:0/2219244277 <== mon.0 v2:172.21.15.162:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6152+0+0 (secure 0 0 0) 0x7f13a808fb50 con 0x7f13ac101d00 2024-04-27T04:14:03.385 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:14:03.380+0000 7f13b0856700 1 --2- 172.21.15.162:0/2219244277 >> [v2:172.21.15.162:6800/901992993,v1:172.21.15.162:6801/901992993] conn(0x7f139806e500 0x7f13980709b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:14:03.386 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:14:03.380+0000 7f13b0856700 1 --2- 172.21.15.162:0/2219244277 >> [v2:172.21.15.162:6800/901992993,v1:172.21.15.162:6801/901992993] conn(0x7f139806e500 0x7f13980709b0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f139c006fd0 tx=0x7f139c008040 comp rx=0 tx=0).ready entity=mgr.14680 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:14:03.386 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:14:03.384+0000 7f13a1ffb700 1 -- 172.21.15.162:0/2219244277 <== mon.0 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186652 (secure 0 0 0) 0x7f13a805cc80 con 0x7f13ac101d00 2024-04-27T04:14:03.422 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:14:03 smithi181 ceph-mon[33774]: from='mgr.14680 172.21.15.162:0/501880828' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-27T04:14:03.617 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:14:03.617+0000 7f13b32bb700 1 -- 172.21.15.162:0/2219244277 --> [v2:172.21.15.162:6800/901992993,v1:172.21.15.162:6801/901992993] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f13ac10d5a0 con 0x7f139806e500 2024-04-27T04:14:03.617 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:14:03.618+0000 7f13a1ffb700 1 -- 172.21.15.162:0/2219244277 <== mgr.14680 v2:172.21.15.162:6800/901992993 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+358 (secure 0 0 0) 0x7f13ac10d5a0 con 0x7f139806e500 2024-04-27T04:14:03.620 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:14:03.620+0000 7f13b32bb700 1 -- 172.21.15.162:0/2219244277 >> [v2:172.21.15.162:6800/901992993,v1:172.21.15.162:6801/901992993] conn(0x7f139806e500 msgr2=0x7f13980709b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:14:03.620 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:14:03.620+0000 7f13b32bb700 1 --2- 172.21.15.162:0/2219244277 >> [v2:172.21.15.162:6800/901992993,v1:172.21.15.162:6801/901992993] conn(0x7f139806e500 0x7f13980709b0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f139c006fd0 tx=0x7f139c008040 comp rx=0 tx=0).stop 2024-04-27T04:14:03.620 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:14:03.620+0000 7f13b32bb700 1 -- 172.21.15.162:0/2219244277 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f13ac101d00 msgr2=0x7f13ac112a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:14:03.621 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:14:03.620+0000 7f13b32bb700 1 --2- 172.21.15.162:0/2219244277 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f13ac101d00 0x7f13ac112a30 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f13a800be90 tx=0x7f13a8004d10 comp rx=0 tx=0).stop 2024-04-27T04:14:03.621 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:14:03.620+0000 7f13b32bb700 1 -- 172.21.15.162:0/2219244277 shutdown_connections 2024-04-27T04:14:03.621 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:14:03.620+0000 7f13b32bb700 1 --2- 172.21.15.162:0/2219244277 >> [v2:172.21.15.162:6800/901992993,v1:172.21.15.162:6801/901992993] conn(0x7f139806e500 0x7f13980709b0 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:14:03.621 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:14:03.620+0000 7f13b32bb700 1 --2- 172.21.15.162:0/2219244277 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f13ac101d00 0x7f13ac112a30 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:14:03.621 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:14:03.620+0000 7f13b32bb700 1 --2- 172.21.15.162:0/2219244277 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f13ac1026a0 0x7f13ac112f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:14:03.622 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:14:03.620+0000 7f13b32bb700 1 -- 172.21.15.162:0/2219244277 >> 172.21.15.162:0/2219244277 conn(0x7f13ac0f7870 msgr2=0x7f13ac0f87d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:14:03.622 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:14:03.620+0000 7f13b32bb700 1 -- 172.21.15.162:0/2219244277 shutdown_connections 2024-04-27T04:14:03.622 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:14:03.621+0000 7f13b32bb700 1 -- 172.21.15.162:0/2219244277 wait complete. 2024-04-27T04:14:03.630 INFO:teuthology.orchestra.run.smithi162.stdout:true 2024-04-27T04:14:03.723 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:14:03.723+0000 7f348b7da700 1 -- 172.21.15.162:0/414103370 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f34840fbd40 msgr2=0x7f34840fc1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:14:03.723 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:14:03.723+0000 7f348b7da700 1 --2- 172.21.15.162:0/414103370 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f34840fbd40 0x7f34840fc1b0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f3480004940 tx=0x7f3480009e30 comp rx=0 tx=0).stop 2024-04-27T04:14:03.723 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:14:03.723+0000 7f348b7da700 1 -- 172.21.15.162:0/414103370 shutdown_connections 2024-04-27T04:14:03.723 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:14:03.723+0000 7f348b7da700 1 --2- 172.21.15.162:0/414103370 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f34840fbd40 0x7f34840fc1b0 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:14:03.723 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:14:03.723+0000 7f348b7da700 1 --2- 172.21.15.162:0/414103370 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3484101d90 0x7f3484102160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:14:03.723 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:14:03.723+0000 7f348b7da700 1 -- 172.21.15.162:0/414103370 >> 172.21.15.162:0/414103370 conn(0x7f3484071d00 msgr2=0x7f3484072100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:14:03.723 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:14:03.724+0000 7f348b7da700 1 -- 172.21.15.162:0/414103370 shutdown_connections 2024-04-27T04:14:03.724 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:14:03.724+0000 7f348b7da700 1 -- 172.21.15.162:0/414103370 wait complete. 2024-04-27T04:14:03.724 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:14:03.724+0000 7f348b7da700 1 Processor -- start 2024-04-27T04:14:03.724 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:14:03.724+0000 7f348b7da700 1 -- start start 2024-04-27T04:14:03.725 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:14:03.724+0000 7f348b7da700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f34840fbd40 0x7f3484115d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:14:03.725 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:14:03.724+0000 7f348b7da700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3484101d90 0x7f34841162c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:14:03.725 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:14:03.724+0000 7f348b7da700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3484116910 con 0x7f3484101d90 2024-04-27T04:14:03.726 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:14:03.724+0000 7f348b7da700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f348410fd20 con 0x7f34840fbd40 2024-04-27T04:14:03.726 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:14:03.724+0000 7f3488d75700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3484101d90 0x7f34841162c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:14:03.726 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:14:03.724+0000 7f3488d75700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3484101d90 0x7f34841162c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:47860/0 (socket says 172.21.15.162:47860) 2024-04-27T04:14:03.726 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:14:03.724+0000 7f3488d75700 1 -- 172.21.15.162:0/4245186960 learned_addr learned my addr 172.21.15.162:0/4245186960 (peer_addr_for_me v2:172.21.15.162:0/0) 2024-04-27T04:14:03.726 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:14:03.725+0000 7f3488d75700 1 -- 172.21.15.162:0/4245186960 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f34840fbd40 msgr2=0x7f3484115d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:14:03.726 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:14:03.725+0000 7f3488d75700 1 --2- 172.21.15.162:0/4245186960 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f34840fbd40 0x7f3484115d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:14:03.726 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:14:03.725+0000 7f3488d75700 1 -- 172.21.15.162:0/4245186960 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3480005040 con 0x7f3484101d90 2024-04-27T04:14:03.726 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:14:03.725+0000 7f3488d75700 1 --2- 172.21.15.162:0/4245186960 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3484101d90 0x7f34841162c0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f348000b3b0 tx=0x7f3480009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:14:03.726 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:14:03.725+0000 7f3479ffb700 1 -- 172.21.15.162:0/4245186960 <== mon.0 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f348000e040 con 0x7f3484101d90 2024-04-27T04:14:03.726 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:14:03.725+0000 7f3479ffb700 1 -- 172.21.15.162:0/4245186960 <== mon.0 v2:172.21.15.162:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f348000bbc0 con 0x7f3484101d90 2024-04-27T04:14:03.726 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:14:03.725+0000 7f348b7da700 1 -- 172.21.15.162:0/4245186960 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f348410ffa0 con 0x7f3484101d90 2024-04-27T04:14:03.726 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:14:03.725+0000 7f3479ffb700 1 -- 172.21.15.162:0/4245186960 <== mon.0 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f348001c3f0 con 0x7f3484101d90 2024-04-27T04:14:03.726 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:14:03.725+0000 7f348b7da700 1 -- 172.21.15.162:0/4245186960 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3484110560 con 0x7f3484101d90 2024-04-27T04:14:03.727 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:14:03.726+0000 7f348b7da700 1 -- 172.21.15.162:0/4245186960 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f34840fd430 con 0x7f3484101d90 2024-04-27T04:14:03.727 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:14:03.727+0000 7f3479ffb700 1 -- 172.21.15.162:0/4245186960 <== mon.0 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 32) v1 ==== 92469+0+0 (secure 0 0 0) 0x7f348000bd30 con 0x7f3484101d90 2024-04-27T04:14:03.728 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:14:03.727+0000 7f3479ffb700 1 --2- 172.21.15.162:0/4245186960 >> [v2:172.21.15.162:6800/901992993,v1:172.21.15.162:6801/901992993] conn(0x7f347006e500 0x7f34700709b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:14:03.728 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:14:03.728+0000 7f3489576700 1 --2- 172.21.15.162:0/4245186960 >> [v2:172.21.15.162:6800/901992993,v1:172.21.15.162:6801/901992993] conn(0x7f347006e500 0x7f34700709b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:14:03.728 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:14:03.728+0000 7f3479ffb700 1 -- 172.21.15.162:0/4245186960 <== mon.0 v2:172.21.15.162:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6152+0+0 (secure 0 0 0) 0x7f348008f190 con 0x7f3484101d90 2024-04-27T04:14:03.728 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:14:03.728+0000 7f3489576700 1 --2- 172.21.15.162:0/4245186960 >> [v2:172.21.15.162:6800/901992993,v1:172.21.15.162:6801/901992993] conn(0x7f347006e500 0x7f34700709b0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f3474004d00 tx=0x7f3474009ce0 comp rx=0 tx=0).ready entity=mgr.14680 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:14:03.731 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:14:03.732+0000 7f3479ffb700 1 -- 172.21.15.162:0/4245186960 <== mon.0 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186652 (secure 0 0 0) 0x7f3480058810 con 0x7f3484101d90 2024-04-27T04:14:03.956 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:14:03.957+0000 7f348b7da700 1 -- 172.21.15.162:0/4245186960 --> [v2:172.21.15.162:6800/901992993,v1:172.21.15.162:6801/901992993] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f3484108e60 con 0x7f347006e500 2024-04-27T04:14:03.957 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:14:03.958+0000 7f3479ffb700 1 -- 172.21.15.162:0/4245186960 <== mgr.14680 v2:172.21.15.162:6800/901992993 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+368 (secure 0 0 0) 0x7f3484108e60 con 0x7f347006e500 2024-04-27T04:14:03.963 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:14:03.960+0000 7f348b7da700 1 -- 172.21.15.162:0/4245186960 >> [v2:172.21.15.162:6800/901992993,v1:172.21.15.162:6801/901992993] conn(0x7f347006e500 msgr2=0x7f34700709b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:14:03.963 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:14:03.960+0000 7f348b7da700 1 --2- 172.21.15.162:0/4245186960 >> [v2:172.21.15.162:6800/901992993,v1:172.21.15.162:6801/901992993] conn(0x7f347006e500 0x7f34700709b0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f3474004d00 tx=0x7f3474009ce0 comp rx=0 tx=0).stop 2024-04-27T04:14:03.964 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:14:03.960+0000 7f348b7da700 1 -- 172.21.15.162:0/4245186960 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3484101d90 msgr2=0x7f34841162c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:14:03.964 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:14:03.960+0000 7f348b7da700 1 --2- 172.21.15.162:0/4245186960 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3484101d90 0x7f34841162c0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f348000b3b0 tx=0x7f3480009e30 comp rx=0 tx=0).stop 2024-04-27T04:14:03.964 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:14:03.960+0000 7f348b7da700 1 -- 172.21.15.162:0/4245186960 shutdown_connections 2024-04-27T04:14:03.964 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:14:03.960+0000 7f348b7da700 1 --2- 172.21.15.162:0/4245186960 >> [v2:172.21.15.162:6800/901992993,v1:172.21.15.162:6801/901992993] conn(0x7f347006e500 0x7f34700709b0 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:14:03.964 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:14:03.960+0000 7f348b7da700 1 --2- 172.21.15.162:0/4245186960 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f34840fbd40 0x7f3484115d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:14:03.964 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:14:03.960+0000 7f348b7da700 1 --2- 172.21.15.162:0/4245186960 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3484101d90 0x7f34841162c0 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:14:03.965 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:14:03.961+0000 7f348b7da700 1 -- 172.21.15.162:0/4245186960 >> 172.21.15.162:0/4245186960 conn(0x7f3484071d00 msgr2=0x7f3484065600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:14:03.965 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:14:03.961+0000 7f348b7da700 1 -- 172.21.15.162:0/4245186960 shutdown_connections 2024-04-27T04:14:03.965 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:14:03.961+0000 7f348b7da700 1 -- 172.21.15.162:0/4245186960 wait complete. 2024-04-27T04:14:04.067 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:14:04.067+0000 7fa82f2c4700 1 -- 172.21.15.162:0/2101416263 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa828065640 msgr2=0x7fa828065ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:14:04.067 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:14:04.067+0000 7fa82f2c4700 1 --2- 172.21.15.162:0/2101416263 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa828065640 0x7fa828065ab0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7fa818004940 tx=0x7fa818009e30 comp rx=0 tx=0).stop 2024-04-27T04:14:04.068 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:14:04.067+0000 7fa82f2c4700 1 -- 172.21.15.162:0/2101416263 shutdown_connections 2024-04-27T04:14:04.068 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:14:04.067+0000 7fa82f2c4700 1 --2- 172.21.15.162:0/2101416263 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa828065640 0x7fa828065ab0 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:14:04.068 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:14:04.067+0000 7fa82f2c4700 1 --2- 172.21.15.162:0/2101416263 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa828102e50 0x7fa828103220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:14:04.068 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:14:04.067+0000 7fa82f2c4700 1 -- 172.21.15.162:0/2101416263 >> 172.21.15.162:0/2101416263 conn(0x7fa828071d00 msgr2=0x7fa828072100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:14:04.068 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:14:04.067+0000 7fa82f2c4700 1 -- 172.21.15.162:0/2101416263 shutdown_connections 2024-04-27T04:14:04.068 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:14:04.067+0000 7fa82f2c4700 1 -- 172.21.15.162:0/2101416263 wait complete. 2024-04-27T04:14:04.069 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:14:04 smithi162 ceph-mon[26841]: pgmap v78: 65 pgs: 65 active+clean; 208 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:14:04.069 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:14:04 smithi162 ceph-mon[26841]: from='mgr.14680 172.21.15.162:0/501880828' entity='mgr.smithi162.oennfn' 2024-04-27T04:14:04.069 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:14:04 smithi162 ceph-mon[26841]: from='mgr.14680 172.21.15.162:0/501880828' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T04:14:04.069 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:14:04 smithi162 ceph-mon[26841]: from='mgr.14680 172.21.15.162:0/501880828' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "versions"}]: dispatch 2024-04-27T04:14:04.069 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:14:04 smithi162 ceph-mon[26841]: from='mgr.14680 172.21.15.162:0/501880828' entity='mgr.smithi162.oennfn' 2024-04-27T04:14:04.069 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:14:04 smithi162 ceph-mon[26841]: from='mgr.14680 172.21.15.162:0/501880828' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "quorum_status"}]: dispatch 2024-04-27T04:14:04.071 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:14:04.068+0000 7fa82f2c4700 1 Processor -- start 2024-04-27T04:14:04.071 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:14:04.068+0000 7fa82f2c4700 1 -- start start 2024-04-27T04:14:04.071 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:14:04.068+0000 7fa82f2c4700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa828065640 0x7fa8280fbdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:14:04.071 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:14:04.068+0000 7fa82f2c4700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa828102e50 0x7fa8280fc2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:14:04.072 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:14:04.068+0000 7fa82f2c4700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8280fc880 con 0x7fa828065640 2024-04-27T04:14:04.072 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:14:04.068+0000 7fa82f2c4700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8280fc9c0 con 0x7fa828102e50 2024-04-27T04:14:04.072 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:14:04.068+0000 7fa82d060700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa828065640 0x7fa8280fbdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:14:04.072 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:14:04.068+0000 7fa82d060700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa828065640 0x7fa8280fbdb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:47874/0 (socket says 172.21.15.162:47874) 2024-04-27T04:14:04.072 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:14:04.068+0000 7fa82d060700 1 -- 172.21.15.162:0/2643713060 learned_addr learned my addr 172.21.15.162:0/2643713060 (peer_addr_for_me v2:172.21.15.162:0/0) 2024-04-27T04:14:04.072 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:14:04.068+0000 7fa82c85f700 1 --2- 172.21.15.162:0/2643713060 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa828102e50 0x7fa8280fc2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:14:04.072 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:14:04.069+0000 7fa82d060700 1 -- 172.21.15.162:0/2643713060 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa828102e50 msgr2=0x7fa8280fc2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:14:04.072 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:14:04.069+0000 7fa82d060700 1 --2- 172.21.15.162:0/2643713060 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa828102e50 0x7fa8280fc2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:14:04.072 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:14:04.069+0000 7fa82d060700 1 -- 172.21.15.162:0/2643713060 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa818005040 con 0x7fa828065640 2024-04-27T04:14:04.072 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:14:04.069+0000 7fa82c85f700 1 --2- 172.21.15.162:0/2643713060 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa828102e50 0x7fa8280fc2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-27T04:14:04.072 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:14:04.069+0000 7fa82d060700 1 --2- 172.21.15.162:0/2643713060 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa828065640 0x7fa8280fbdb0 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7fa824002a50 tx=0x7fa8240046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:14:04.072 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:14:04.069+0000 7fa81dffb700 1 -- 172.21.15.162:0/2643713060 <== mon.0 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa824007cc0 con 0x7fa828065640 2024-04-27T04:14:04.072 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:14:04.069+0000 7fa81dffb700 1 -- 172.21.15.162:0/2643713060 <== mon.0 v2:172.21.15.162:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fa82400f040 con 0x7fa828065640 2024-04-27T04:14:04.072 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:14:04.069+0000 7fa82f2c4700 1 -- 172.21.15.162:0/2643713060 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa828119e70 con 0x7fa828065640 2024-04-27T04:14:04.073 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:14:04.069+0000 7fa81dffb700 1 -- 172.21.15.162:0/2643713060 <== mon.0 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa824009640 con 0x7fa828065640 2024-04-27T04:14:04.073 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:14:04.069+0000 7fa82f2c4700 1 -- 172.21.15.162:0/2643713060 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa82811a2f0 con 0x7fa828065640 2024-04-27T04:14:04.073 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:14:04.071+0000 7fa82f2c4700 1 -- 172.21.15.162:0/2643713060 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa828100100 con 0x7fa828065640 2024-04-27T04:14:04.073 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:14:04.071+0000 7fa81dffb700 1 -- 172.21.15.162:0/2643713060 <== mon.0 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 32) v1 ==== 92469+0+0 (secure 0 0 0) 0x7fa82400d690 con 0x7fa828065640 2024-04-27T04:14:04.073 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:14:04.072+0000 7fa81dffb700 1 --2- 172.21.15.162:0/2643713060 >> [v2:172.21.15.162:6800/901992993,v1:172.21.15.162:6801/901992993] conn(0x7fa81406e500 0x7fa8140709b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:14:04.073 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:14:04.072+0000 7fa81dffb700 1 -- 172.21.15.162:0/2643713060 <== mon.0 v2:172.21.15.162:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6152+0+0 (secure 0 0 0) 0x7fa82408dcc0 con 0x7fa828065640 2024-04-27T04:14:04.073 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:14:04.072+0000 7fa82c85f700 1 --2- 172.21.15.162:0/2643713060 >> [v2:172.21.15.162:6800/901992993,v1:172.21.15.162:6801/901992993] conn(0x7fa81406e500 0x7fa8140709b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:14:04.073 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:14:04.072+0000 7fa82c85f700 1 --2- 172.21.15.162:0/2643713060 >> [v2:172.21.15.162:6800/901992993,v1:172.21.15.162:6801/901992993] conn(0x7fa81406e500 0x7fa8140709b0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fa818009fd0 tx=0x7fa818009e30 comp rx=0 tx=0).ready entity=mgr.14680 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:14:04.076 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:14:04.077+0000 7fa81dffb700 1 -- 172.21.15.162:0/2643713060 <== mon.0 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186652 (secure 0 0 0) 0x7fa8240572c0 con 0x7fa828065640 2024-04-27T04:14:04.294 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:14:04.295+0000 7fa82f2c4700 1 -- 172.21.15.162:0/2643713060 --> [v2:172.21.15.162:6800/901992993,v1:172.21.15.162:6801/901992993] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7fa8280fcb50 con 0x7fa81406e500 2024-04-27T04:14:04.305 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:14:04.305+0000 7fa81dffb700 1 -- 172.21.15.162:0/2643713060 <== mgr.14680 v2:172.21.15.162:6800/901992993 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3520 (secure 0 0 0) 0x7fa8280fcb50 con 0x7fa81406e500 2024-04-27T04:14:04.305 INFO:teuthology.orchestra.run.smithi162.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-27T04:14:04.305 INFO:teuthology.orchestra.run.smithi162.stdout:alertmanager.smithi162 smithi162 *:9093,9094 running (73s) 43s ago 12m 16.5M - 0.25.0 c8568f914cd2 40e334659017 2024-04-27T04:14:04.305 INFO:teuthology.orchestra.run.smithi162.stdout:crash.smithi162 smithi162 running (12m) 43s ago 12m 6660k - 17.2.7-907-g2be3efa9 1c5834633715 228c98df4289 2024-04-27T04:14:04.305 INFO:teuthology.orchestra.run.smithi162.stdout:crash.smithi181 smithi181 running (11m) 105s ago 11m 6665k - 17.2.7-907-g2be3efa9 1c5834633715 c8e553e3df2b 2024-04-27T04:14:04.305 INFO:teuthology.orchestra.run.smithi162.stdout:grafana.smithi162 smithi162 *:3000 running (49s) 43s ago 12m 52.9M - 9.4.12 2bacad6d85d8 3d78c478f03f 2024-04-27T04:14:04.305 INFO:teuthology.orchestra.run.smithi162.stdout:mds.cephfs.smithi162.gzlyjj smithi162 running (7m) 43s ago 7m 12.9M - 17.2.7-907-g2be3efa9 1c5834633715 27e5b030dd32 2024-04-27T04:14:04.305 INFO:teuthology.orchestra.run.smithi162.stdout:mds.cephfs.smithi162.xcnapk smithi162 running (7m) 43s ago 7m 100M - 17.2.7-907-g2be3efa9 1c5834633715 38f9030df029 2024-04-27T04:14:04.305 INFO:teuthology.orchestra.run.smithi162.stdout:mds.cephfs.smithi181.bkkcdm smithi181 running (6m) 105s ago 6m 14.3M - 17.2.7-907-g2be3efa9 1c5834633715 af55afca1263 2024-04-27T04:14:04.305 INFO:teuthology.orchestra.run.smithi162.stdout:mds.cephfs.smithi181.dzomgh smithi181 running (7m) 105s ago 7m 145M - 17.2.7-907-g2be3efa9 1c5834633715 6733296597f3 2024-04-27T04:14:04.305 INFO:teuthology.orchestra.run.smithi162.stdout:mgr.smithi162.oennfn smithi162 *:8443,9283,8765 running (2m) 43s ago 13m 508M - 19.0.0-3253-g4e213a58 67e414ff4bc4 58a748d3637b 2024-04-27T04:14:04.306 INFO:teuthology.orchestra.run.smithi162.stdout:mgr.smithi181.vtwuhz smithi181 *:8443,9283,8765 running (2m) 105s ago 11m 451M - 19.0.0-3253-g4e213a58 67e414ff4bc4 813584297b1d 2024-04-27T04:14:04.306 INFO:teuthology.orchestra.run.smithi162.stdout:mon.smithi162 smithi162 running (13m) 43s ago 13m 55.0M 2048M 17.2.7-907-g2be3efa9 1c5834633715 0f1e3682b9ab 2024-04-27T04:14:04.306 INFO:teuthology.orchestra.run.smithi162.stdout:mon.smithi181 smithi181 running (11m) 105s ago 11m 42.7M 2048M 17.2.7-907-g2be3efa9 1c5834633715 96e48aa3912f 2024-04-27T04:14:04.306 INFO:teuthology.orchestra.run.smithi162.stdout:node-exporter.smithi162 smithi162 *:9100 running (112s) 43s ago 12m 8376k - 1.5.0 0da6a335fe13 5c4c5ea9c1d7 2024-04-27T04:14:04.306 INFO:teuthology.orchestra.run.smithi162.stdout:node-exporter.smithi181 smithi181 *:9100 running (108s) 105s ago 11m 3670k - 1.5.0 0da6a335fe13 9067f14c0519 2024-04-27T04:14:04.306 INFO:teuthology.orchestra.run.smithi162.stdout:osd.0 smithi162 running (10m) 43s ago 10m 232M 1557M 17.2.7-907-g2be3efa9 1c5834633715 4a9df3aa938d 2024-04-27T04:14:04.306 INFO:teuthology.orchestra.run.smithi162.stdout:osd.1 smithi162 running (10m) 43s ago 10m 244M 1557M 17.2.7-907-g2be3efa9 1c5834633715 aefb464c46e8 2024-04-27T04:14:04.306 INFO:teuthology.orchestra.run.smithi162.stdout:osd.2 smithi162 running (9m) 43s ago 9m 203M 1557M 17.2.7-907-g2be3efa9 1c5834633715 70f16d86d38e 2024-04-27T04:14:04.306 INFO:teuthology.orchestra.run.smithi162.stdout:osd.3 smithi181 running (9m) 105s ago 9m 276M 2581M 17.2.7-907-g2be3efa9 1c5834633715 030d829be54e 2024-04-27T04:14:04.306 INFO:teuthology.orchestra.run.smithi162.stdout:osd.4 smithi181 running (8m) 105s ago 8m 244M 2581M 17.2.7-907-g2be3efa9 1c5834633715 512afbee7cf7 2024-04-27T04:14:04.306 INFO:teuthology.orchestra.run.smithi162.stdout:osd.5 smithi181 running (8m) 105s ago 8m 221M 2581M 17.2.7-907-g2be3efa9 1c5834633715 f47a1daac1ea 2024-04-27T04:14:04.306 INFO:teuthology.orchestra.run.smithi162.stdout:prometheus.smithi162 smithi162 *:9095 running (90s) 43s ago 11m 51.4M - 2.43.0 a07b618ecd1d f30657a43324 2024-04-27T04:14:04.307 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:14:04.308+0000 7fa82f2c4700 1 -- 172.21.15.162:0/2643713060 >> [v2:172.21.15.162:6800/901992993,v1:172.21.15.162:6801/901992993] conn(0x7fa81406e500 msgr2=0x7fa8140709b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:14:04.307 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:14:04.308+0000 7fa82f2c4700 1 --2- 172.21.15.162:0/2643713060 >> [v2:172.21.15.162:6800/901992993,v1:172.21.15.162:6801/901992993] conn(0x7fa81406e500 0x7fa8140709b0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fa818009fd0 tx=0x7fa818009e30 comp rx=0 tx=0).stop 2024-04-27T04:14:04.307 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:14:04.308+0000 7fa82f2c4700 1 -- 172.21.15.162:0/2643713060 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa828065640 msgr2=0x7fa8280fbdb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:14:04.307 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:14:04.308+0000 7fa82f2c4700 1 --2- 172.21.15.162:0/2643713060 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa828065640 0x7fa8280fbdb0 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7fa824002a50 tx=0x7fa8240046b0 comp rx=0 tx=0).stop 2024-04-27T04:14:04.307 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:14:04.308+0000 7fa82f2c4700 1 -- 172.21.15.162:0/2643713060 shutdown_connections 2024-04-27T04:14:04.307 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:14:04.308+0000 7fa82f2c4700 1 --2- 172.21.15.162:0/2643713060 >> [v2:172.21.15.162:6800/901992993,v1:172.21.15.162:6801/901992993] conn(0x7fa81406e500 0x7fa8140709b0 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:14:04.307 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:14:04.308+0000 7fa82f2c4700 1 --2- 172.21.15.162:0/2643713060 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa828065640 0x7fa8280fbdb0 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:14:04.307 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:14:04.308+0000 7fa82f2c4700 1 --2- 172.21.15.162:0/2643713060 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa828102e50 0x7fa8280fc2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:14:04.307 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:14:04.308+0000 7fa82f2c4700 1 -- 172.21.15.162:0/2643713060 >> 172.21.15.162:0/2643713060 conn(0x7fa828071d00 msgr2=0x7fa8280fb000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:14:04.308 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:14:04.308+0000 7fa82f2c4700 1 -- 172.21.15.162:0/2643713060 shutdown_connections 2024-04-27T04:14:04.308 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:14:04.308+0000 7fa82f2c4700 1 -- 172.21.15.162:0/2643713060 wait complete. 2024-04-27T04:14:04.422 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:14:04 smithi181 ceph-mon[33774]: pgmap v78: 65 pgs: 65 active+clean; 208 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:14:04.422 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:14:04 smithi181 ceph-mon[33774]: from='mgr.14680 172.21.15.162:0/501880828' entity='mgr.smithi162.oennfn' 2024-04-27T04:14:04.422 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:14:04 smithi181 ceph-mon[33774]: from='mgr.14680 172.21.15.162:0/501880828' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T04:14:04.422 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:14:04 smithi181 ceph-mon[33774]: from='mgr.14680 172.21.15.162:0/501880828' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "versions"}]: dispatch 2024-04-27T04:14:04.423 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:14:04 smithi181 ceph-mon[33774]: from='mgr.14680 172.21.15.162:0/501880828' entity='mgr.smithi162.oennfn' 2024-04-27T04:14:04.423 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:14:04 smithi181 ceph-mon[33774]: from='mgr.14680 172.21.15.162:0/501880828' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "quorum_status"}]: dispatch 2024-04-27T04:14:04.431 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:14:04.430+0000 7fd6e8292700 1 -- 172.21.15.162:0/1966723344 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd6e01037c0 msgr2=0x7fd6e00fb3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:14:04.431 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:14:04.430+0000 7fd6e8292700 1 --2- 172.21.15.162:0/1966723344 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd6e01037c0 0x7fd6e00fb3c0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7fd6d0004940 tx=0x7fd6d0009e30 comp rx=0 tx=0).stop 2024-04-27T04:14:04.431 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:14:04.430+0000 7fd6e8292700 1 -- 172.21.15.162:0/1966723344 shutdown_connections 2024-04-27T04:14:04.431 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:14:04.430+0000 7fd6e8292700 1 --2- 172.21.15.162:0/1966723344 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd6e01037c0 0x7fd6e00fb3c0 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:14:04.432 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:14:04.430+0000 7fd6e8292700 1 --2- 172.21.15.162:0/1966723344 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd6e0102e20 0x7fd6e01031f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:14:04.432 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:14:04.430+0000 7fd6e8292700 1 -- 172.21.15.162:0/1966723344 >> 172.21.15.162:0/1966723344 conn(0x7fd6e0072380 msgr2=0x7fd6e0072780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:14:04.432 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:14:04.430+0000 7fd6e8292700 1 -- 172.21.15.162:0/1966723344 shutdown_connections 2024-04-27T04:14:04.432 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:14:04.430+0000 7fd6e8292700 1 -- 172.21.15.162:0/1966723344 wait complete. 2024-04-27T04:14:04.432 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:14:04.431+0000 7fd6e8292700 1 Processor -- start 2024-04-27T04:14:04.432 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:14:04.431+0000 7fd6e8292700 1 -- start start 2024-04-27T04:14:04.432 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:14:04.431+0000 7fd6e8292700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd6e0102e20 0x7fd6e0113a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:14:04.432 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:14:04.431+0000 7fd6e8292700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd6e01037c0 0x7fd6e0113fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:14:04.432 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:14:04.431+0000 7fd6e8292700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6e01145f0 con 0x7fd6e0102e20 2024-04-27T04:14:04.432 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:14:04.431+0000 7fd6e8292700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6e010dc10 con 0x7fd6e01037c0 2024-04-27T04:14:04.432 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:14:04.431+0000 7fd6e602e700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd6e0102e20 0x7fd6e0113a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:14:04.432 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:14:04.431+0000 7fd6e602e700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd6e0102e20 0x7fd6e0113a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:47892/0 (socket says 172.21.15.162:47892) 2024-04-27T04:14:04.432 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:14:04.431+0000 7fd6e602e700 1 -- 172.21.15.162:0/16176343 learned_addr learned my addr 172.21.15.162:0/16176343 (peer_addr_for_me v2:172.21.15.162:0/0) 2024-04-27T04:14:04.433 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:14:04.432+0000 7fd6e582d700 1 --2- 172.21.15.162:0/16176343 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd6e01037c0 0x7fd6e0113fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:14:04.433 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:14:04.432+0000 7fd6e602e700 1 -- 172.21.15.162:0/16176343 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd6e01037c0 msgr2=0x7fd6e0113fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:14:04.433 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:14:04.432+0000 7fd6e602e700 1 --2- 172.21.15.162:0/16176343 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd6e01037c0 0x7fd6e0113fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:14:04.433 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:14:04.432+0000 7fd6e602e700 1 -- 172.21.15.162:0/16176343 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6d0005040 con 0x7fd6e0102e20 2024-04-27T04:14:04.433 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:14:04.432+0000 7fd6e602e700 1 --2- 172.21.15.162:0/16176343 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd6e0102e20 0x7fd6e0113a60 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7fd6dc002a50 tx=0x7fd6dc004600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:14:04.433 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:14:04.432+0000 7fd6d6ffd700 1 -- 172.21.15.162:0/16176343 <== mon.0 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd6dc007cc0 con 0x7fd6e0102e20 2024-04-27T04:14:04.433 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:14:04.432+0000 7fd6d6ffd700 1 -- 172.21.15.162:0/16176343 <== mon.0 v2:172.21.15.162:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fd6dc00f040 con 0x7fd6e0102e20 2024-04-27T04:14:04.433 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:14:04.432+0000 7fd6e8292700 1 -- 172.21.15.162:0/16176343 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd6e010def0 con 0x7fd6e0102e20 2024-04-27T04:14:04.433 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:14:04.432+0000 7fd6d6ffd700 1 -- 172.21.15.162:0/16176343 <== mon.0 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd6dc00da40 con 0x7fd6e0102e20 2024-04-27T04:14:04.433 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:14:04.432+0000 7fd6e8292700 1 -- 172.21.15.162:0/16176343 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd6e010e430 con 0x7fd6e0102e20 2024-04-27T04:14:04.434 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:14:04.433+0000 7fd6e8292700 1 -- 172.21.15.162:0/16176343 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd6e0100160 con 0x7fd6e0102e20 2024-04-27T04:14:04.434 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:14:04.434+0000 7fd6d6ffd700 1 -- 172.21.15.162:0/16176343 <== mon.0 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 32) v1 ==== 92469+0+0 (secure 0 0 0) 0x7fd6dc0097c0 con 0x7fd6e0102e20 2024-04-27T04:14:04.434 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:14:04.434+0000 7fd6d6ffd700 1 --2- 172.21.15.162:0/16176343 >> [v2:172.21.15.162:6800/901992993,v1:172.21.15.162:6801/901992993] conn(0x7fd6cc06e500 0x7fd6cc0709b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:14:04.434 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:14:04.434+0000 7fd6d6ffd700 1 -- 172.21.15.162:0/16176343 <== mon.0 v2:172.21.15.162:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6152+0+0 (secure 0 0 0) 0x7fd6dc08e610 con 0x7fd6e0102e20 2024-04-27T04:14:04.435 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:14:04.434+0000 7fd6e582d700 1 --2- 172.21.15.162:0/16176343 >> [v2:172.21.15.162:6800/901992993,v1:172.21.15.162:6801/901992993] conn(0x7fd6cc06e500 0x7fd6cc0709b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:14:04.435 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:14:04.435+0000 7fd6e582d700 1 --2- 172.21.15.162:0/16176343 >> [v2:172.21.15.162:6800/901992993,v1:172.21.15.162:6801/901992993] conn(0x7fd6cc06e500 0x7fd6cc0709b0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fd6d000b3b0 tx=0x7fd6d0009e30 comp rx=0 tx=0).ready entity=mgr.14680 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:14:04.438 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:14:04.439+0000 7fd6d6ffd700 1 -- 172.21.15.162:0/16176343 <== mon.0 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186652 (secure 0 0 0) 0x7fd6dc05b860 con 0x7fd6e0102e20 2024-04-27T04:14:04.731 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:14:04.731+0000 7fd6e8292700 1 -- 172.21.15.162:0/16176343 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7fd6e0109f40 con 0x7fd6e0102e20 2024-04-27T04:14:04.731 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:14:04.732+0000 7fd6d6ffd700 1 -- 172.21.15.162:0/16176343 <== mon.0 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+739 (secure 0 0 0) 0x7fd6dc05b3f0 con 0x7fd6e0102e20 2024-04-27T04:14:04.731 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-04-27T04:14:04.731 INFO:teuthology.orchestra.run.smithi162.stdout: "mon": { 2024-04-27T04:14:04.731 INFO:teuthology.orchestra.run.smithi162.stdout: "ceph version 17.2.7-907-g2be3efa9 (2be3efa97a7972f8e925bd899b474329979b74d6) quincy (stable)": 2 2024-04-27T04:14:04.731 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-04-27T04:14:04.731 INFO:teuthology.orchestra.run.smithi162.stdout: "mgr": { 2024-04-27T04:14:04.731 INFO:teuthology.orchestra.run.smithi162.stdout: "ceph version 19.0.0-3253-g4e213a58 (4e213a58e787437ff508e6c890ccd5fd32dc96b5) squid (dev)": 2 2024-04-27T04:14:04.731 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-04-27T04:14:04.732 INFO:teuthology.orchestra.run.smithi162.stdout: "osd": { 2024-04-27T04:14:04.732 INFO:teuthology.orchestra.run.smithi162.stdout: "ceph version 17.2.7-907-g2be3efa9 (2be3efa97a7972f8e925bd899b474329979b74d6) quincy (stable)": 6 2024-04-27T04:14:04.732 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-04-27T04:14:04.732 INFO:teuthology.orchestra.run.smithi162.stdout: "mds": { 2024-04-27T04:14:04.732 INFO:teuthology.orchestra.run.smithi162.stdout: "ceph version 17.2.7-907-g2be3efa9 (2be3efa97a7972f8e925bd899b474329979b74d6) quincy (stable)": 4 2024-04-27T04:14:04.732 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-04-27T04:14:04.732 INFO:teuthology.orchestra.run.smithi162.stdout: "overall": { 2024-04-27T04:14:04.732 INFO:teuthology.orchestra.run.smithi162.stdout: "ceph version 17.2.7-907-g2be3efa9 (2be3efa97a7972f8e925bd899b474329979b74d6) quincy (stable)": 12, 2024-04-27T04:14:04.732 INFO:teuthology.orchestra.run.smithi162.stdout: "ceph version 19.0.0-3253-g4e213a58 (4e213a58e787437ff508e6c890ccd5fd32dc96b5) squid (dev)": 2 2024-04-27T04:14:04.733 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-04-27T04:14:04.733 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-04-27T04:14:04.733 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:14:04.734+0000 7fd6e8292700 1 -- 172.21.15.162:0/16176343 >> [v2:172.21.15.162:6800/901992993,v1:172.21.15.162:6801/901992993] conn(0x7fd6cc06e500 msgr2=0x7fd6cc0709b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:14:04.733 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:14:04.734+0000 7fd6e8292700 1 --2- 172.21.15.162:0/16176343 >> [v2:172.21.15.162:6800/901992993,v1:172.21.15.162:6801/901992993] conn(0x7fd6cc06e500 0x7fd6cc0709b0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fd6d000b3b0 tx=0x7fd6d0009e30 comp rx=0 tx=0).stop 2024-04-27T04:14:04.733 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:14:04.734+0000 7fd6e8292700 1 -- 172.21.15.162:0/16176343 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd6e0102e20 msgr2=0x7fd6e0113a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:14:04.733 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:14:04.734+0000 7fd6e8292700 1 --2- 172.21.15.162:0/16176343 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd6e0102e20 0x7fd6e0113a60 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7fd6dc002a50 tx=0x7fd6dc004600 comp rx=0 tx=0).stop 2024-04-27T04:14:04.733 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:14:04.734+0000 7fd6e8292700 1 -- 172.21.15.162:0/16176343 shutdown_connections 2024-04-27T04:14:04.734 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:14:04.734+0000 7fd6e8292700 1 --2- 172.21.15.162:0/16176343 >> [v2:172.21.15.162:6800/901992993,v1:172.21.15.162:6801/901992993] conn(0x7fd6cc06e500 0x7fd6cc0709b0 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:14:04.734 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:14:04.734+0000 7fd6e8292700 1 --2- 172.21.15.162:0/16176343 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd6e0102e20 0x7fd6e0113a60 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:14:04.734 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:14:04.734+0000 7fd6e8292700 1 --2- 172.21.15.162:0/16176343 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd6e01037c0 0x7fd6e0113fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:14:04.734 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:14:04.734+0000 7fd6e8292700 1 -- 172.21.15.162:0/16176343 >> 172.21.15.162:0/16176343 conn(0x7fd6e0072380 msgr2=0x7fd6e00fad10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:14:04.734 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:14:04.734+0000 7fd6e8292700 1 -- 172.21.15.162:0/16176343 shutdown_connections 2024-04-27T04:14:04.734 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:14:04.734+0000 7fd6e8292700 1 -- 172.21.15.162:0/16176343 wait complete. 2024-04-27T04:14:04.842 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:14:04.841+0000 7f0f820d0700 1 -- 172.21.15.162:0/2903827873 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0f7c103790 msgr2=0x7f0f7c0fce00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:14:04.843 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:14:04.841+0000 7f0f820d0700 1 --2- 172.21.15.162:0/2903827873 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0f7c103790 0x7f0f7c0fce00 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f0f70004940 tx=0x7f0f70009e30 comp rx=0 tx=0).stop 2024-04-27T04:14:04.843 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:14:04.841+0000 7f0f820d0700 1 -- 172.21.15.162:0/2903827873 shutdown_connections 2024-04-27T04:14:04.843 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:14:04.841+0000 7f0f820d0700 1 --2- 172.21.15.162:0/2903827873 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0f7c103790 0x7f0f7c0fce00 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:14:04.843 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:14:04.841+0000 7f0f820d0700 1 --2- 172.21.15.162:0/2903827873 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0f7c102df0 0x7f0f7c1031c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:14:04.843 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:14:04.841+0000 7f0f820d0700 1 -- 172.21.15.162:0/2903827873 >> 172.21.15.162:0/2903827873 conn(0x7f0f7c0f8980 msgr2=0x7f0f7c0fad90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:14:04.843 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:14:04.841+0000 7f0f820d0700 1 -- 172.21.15.162:0/2903827873 shutdown_connections 2024-04-27T04:14:04.843 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:14:04.841+0000 7f0f820d0700 1 -- 172.21.15.162:0/2903827873 wait complete. 2024-04-27T04:14:04.843 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:14:04.842+0000 7f0f820d0700 1 Processor -- start 2024-04-27T04:14:04.843 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:14:04.842+0000 7f0f820d0700 1 -- start start 2024-04-27T04:14:04.843 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:14:04.842+0000 7f0f820d0700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0f7c102df0 0x7f0f7c113b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:14:04.843 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:14:04.842+0000 7f0f820d0700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0f7c103790 0x7f0f7c1140d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:14:04.843 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:14:04.842+0000 7f0f820d0700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f7c114720 con 0x7f0f7c102df0 2024-04-27T04:14:04.843 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:14:04.842+0000 7f0f820d0700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f7c10db10 con 0x7f0f7c103790 2024-04-27T04:14:04.844 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:14:04.842+0000 7f0f7b7fe700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0f7c102df0 0x7f0f7c113b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:14:04.844 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:14:04.842+0000 7f0f7b7fe700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0f7c102df0 0x7f0f7c113b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:47916/0 (socket says 172.21.15.162:47916) 2024-04-27T04:14:04.844 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:14:04.842+0000 7f0f7b7fe700 1 -- 172.21.15.162:0/666293446 learned_addr learned my addr 172.21.15.162:0/666293446 (peer_addr_for_me v2:172.21.15.162:0/0) 2024-04-27T04:14:04.844 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:14:04.842+0000 7f0f7b7fe700 1 -- 172.21.15.162:0/666293446 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0f7c103790 msgr2=0x7f0f7c1140d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:14:04.844 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:14:04.842+0000 7f0f7b7fe700 1 --2- 172.21.15.162:0/666293446 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0f7c103790 0x7f0f7c1140d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:14:04.844 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:14:04.842+0000 7f0f7b7fe700 1 -- 172.21.15.162:0/666293446 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0f70005040 con 0x7f0f7c102df0 2024-04-27T04:14:04.844 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:14:04.843+0000 7f0f7b7fe700 1 --2- 172.21.15.162:0/666293446 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0f7c102df0 0x7f0f7c113b90 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f0f6c002a50 tx=0x7f0f6c0046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:14:04.844 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:14:04.843+0000 7f0f63fff700 1 -- 172.21.15.162:0/666293446 <== mon.0 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0f6c007cc0 con 0x7f0f7c102df0 2024-04-27T04:14:04.844 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:14:04.843+0000 7f0f63fff700 1 -- 172.21.15.162:0/666293446 <== mon.0 v2:172.21.15.162:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f0f6c00f040 con 0x7f0f7c102df0 2024-04-27T04:14:04.844 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:14:04.843+0000 7f0f820d0700 1 -- 172.21.15.162:0/666293446 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0f7c10ddf0 con 0x7f0f7c102df0 2024-04-27T04:14:04.844 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:14:04.843+0000 7f0f63fff700 1 -- 172.21.15.162:0/666293446 <== mon.0 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0f6c009640 con 0x7f0f7c102df0 2024-04-27T04:14:04.844 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:14:04.843+0000 7f0f820d0700 1 -- 172.21.15.162:0/666293446 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0f7c10e3b0 con 0x7f0f7c102df0 2024-04-27T04:14:04.845 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:14:04.844+0000 7f0f820d0700 1 -- 172.21.15.162:0/666293446 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0f7c0fe540 con 0x7f0f7c102df0 2024-04-27T04:14:04.846 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:14:04.844+0000 7f0f63fff700 1 -- 172.21.15.162:0/666293446 <== mon.0 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 32) v1 ==== 92469+0+0 (secure 0 0 0) 0x7f0f6c0097c0 con 0x7f0f7c102df0 2024-04-27T04:14:04.846 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:14:04.845+0000 7f0f63fff700 1 --2- 172.21.15.162:0/666293446 >> [v2:172.21.15.162:6800/901992993,v1:172.21.15.162:6801/901992993] conn(0x7f0f6406e500 0x7f0f640709b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:14:04.846 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:14:04.845+0000 7f0f63fff700 1 -- 172.21.15.162:0/666293446 <== mon.0 v2:172.21.15.162:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6152+0+0 (secure 0 0 0) 0x7f0f6c08dc60 con 0x7f0f7c102df0 2024-04-27T04:14:04.846 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:14:04.845+0000 7f0f7affd700 1 --2- 172.21.15.162:0/666293446 >> [v2:172.21.15.162:6800/901992993,v1:172.21.15.162:6801/901992993] conn(0x7f0f6406e500 0x7f0f640709b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:14:04.846 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:14:04.845+0000 7f0f7affd700 1 --2- 172.21.15.162:0/666293446 >> [v2:172.21.15.162:6800/901992993,v1:172.21.15.162:6801/901992993] conn(0x7f0f6406e500 0x7f0f640709b0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f0f70004f90 tx=0x7f0f70009e30 comp rx=0 tx=0).ready entity=mgr.14680 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:14:04.848 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:14:04.849+0000 7f0f63fff700 1 -- 172.21.15.162:0/666293446 <== mon.0 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186652 (secure 0 0 0) 0x7f0f6c05af60 con 0x7f0f7c102df0 2024-04-27T04:14:05.105 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:14:05.105+0000 7f0f820d0700 1 -- 172.21.15.162:0/666293446 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f0f7c109ed0 con 0x7f0f7c102df0 2024-04-27T04:14:05.105 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:14:05.106+0000 7f0f63fff700 1 -- 172.21.15.162:0/666293446 <== mon.0 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 15 v15) v1 ==== 76+0+1841 (secure 0 0 0) 0x7f0f6c05aaf0 con 0x7f0f7c102df0 2024-04-27T04:14:05.105 INFO:teuthology.orchestra.run.smithi162.stdout:e15 2024-04-27T04:14:05.105 INFO:teuthology.orchestra.run.smithi162.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-04-27T04:14:05.105 INFO:teuthology.orchestra.run.smithi162.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-27T04:14:05.106 INFO:teuthology.orchestra.run.smithi162.stdout:legacy client fscid: 1 2024-04-27T04:14:05.106 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-04-27T04:14:05.106 INFO:teuthology.orchestra.run.smithi162.stdout:Filesystem 'cephfs' (1) 2024-04-27T04:14:05.106 INFO:teuthology.orchestra.run.smithi162.stdout:fs_name cephfs 2024-04-27T04:14:05.106 INFO:teuthology.orchestra.run.smithi162.stdout:epoch 15 2024-04-27T04:14:05.106 INFO:teuthology.orchestra.run.smithi162.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-04-27T04:14:05.106 INFO:teuthology.orchestra.run.smithi162.stdout:created 2024-04-27T04:06:53.095749+0000 2024-04-27T04:14:05.106 INFO:teuthology.orchestra.run.smithi162.stdout:modified 2024-04-27T04:07:11.285033+0000 2024-04-27T04:14:05.106 INFO:teuthology.orchestra.run.smithi162.stdout:tableserver 0 2024-04-27T04:14:05.106 INFO:teuthology.orchestra.run.smithi162.stdout:root 0 2024-04-27T04:14:05.106 INFO:teuthology.orchestra.run.smithi162.stdout:session_timeout 60 2024-04-27T04:14:05.106 INFO:teuthology.orchestra.run.smithi162.stdout:session_autoclose 300 2024-04-27T04:14:05.106 INFO:teuthology.orchestra.run.smithi162.stdout:max_file_size 1099511627776 2024-04-27T04:14:05.106 INFO:teuthology.orchestra.run.smithi162.stdout:required_client_features {} 2024-04-27T04:14:05.106 INFO:teuthology.orchestra.run.smithi162.stdout:last_failure 0 2024-04-27T04:14:05.107 INFO:teuthology.orchestra.run.smithi162.stdout:last_failure_osd_epoch 46 2024-04-27T04:14:05.107 INFO:teuthology.orchestra.run.smithi162.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-27T04:14:05.107 INFO:teuthology.orchestra.run.smithi162.stdout:max_mds 1 2024-04-27T04:14:05.107 INFO:teuthology.orchestra.run.smithi162.stdout:in 0 2024-04-27T04:14:05.107 INFO:teuthology.orchestra.run.smithi162.stdout:up {0=24273} 2024-04-27T04:14:05.107 INFO:teuthology.orchestra.run.smithi162.stdout:failed 2024-04-27T04:14:05.107 INFO:teuthology.orchestra.run.smithi162.stdout:damaged 2024-04-27T04:14:05.107 INFO:teuthology.orchestra.run.smithi162.stdout:stopped 2024-04-27T04:14:05.107 INFO:teuthology.orchestra.run.smithi162.stdout:data_pools [3] 2024-04-27T04:14:05.107 INFO:teuthology.orchestra.run.smithi162.stdout:metadata_pool 2 2024-04-27T04:14:05.107 INFO:teuthology.orchestra.run.smithi162.stdout:inline_data disabled 2024-04-27T04:14:05.107 INFO:teuthology.orchestra.run.smithi162.stdout:balancer 2024-04-27T04:14:05.107 INFO:teuthology.orchestra.run.smithi162.stdout:standby_count_wanted 1 2024-04-27T04:14:05.107 INFO:teuthology.orchestra.run.smithi162.stdout:[mds.cephfs.smithi181.dzomgh{0:24273} state up:active seq 6 join_fscid=1 addr [v2:172.21.15.181:6824/658214993,v1:172.21.15.181:6825/658214993] compat {c=[1],r=[1],i=[7ff]}] 2024-04-27T04:14:05.107 INFO:teuthology.orchestra.run.smithi162.stdout:[mds.cephfs.smithi162.xcnapk{0:14476} state up:standby-replay seq 2 join_fscid=1 addr [v2:172.21.15.162:6828/1296160676,v1:172.21.15.162:6829/1296160676] compat {c=[1],r=[1],i=[7ff]}] 2024-04-27T04:14:05.108 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-04-27T04:14:05.108 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-04-27T04:14:05.108 INFO:teuthology.orchestra.run.smithi162.stdout:Standby daemons: 2024-04-27T04:14:05.108 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-04-27T04:14:05.108 INFO:teuthology.orchestra.run.smithi162.stdout:[mds.cephfs.smithi162.gzlyjj{-1:14488} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.162:6826/2629103894,v1:172.21.15.162:6827/2629103894] compat {c=[1],r=[1],i=[7ff]}] 2024-04-27T04:14:05.108 INFO:teuthology.orchestra.run.smithi162.stdout:[mds.cephfs.smithi181.bkkcdm{-1:24287} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.181:6826/3756475453,v1:172.21.15.181:6827/3756475453] compat {c=[1],r=[1],i=[7ff]}] 2024-04-27T04:14:05.109 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:14:05.108+0000 7f0f820d0700 1 -- 172.21.15.162:0/666293446 >> [v2:172.21.15.162:6800/901992993,v1:172.21.15.162:6801/901992993] conn(0x7f0f6406e500 msgr2=0x7f0f640709b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:14:05.109 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:14:05.108+0000 7f0f820d0700 1 --2- 172.21.15.162:0/666293446 >> [v2:172.21.15.162:6800/901992993,v1:172.21.15.162:6801/901992993] conn(0x7f0f6406e500 0x7f0f640709b0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f0f70004f90 tx=0x7f0f70009e30 comp rx=0 tx=0).stop 2024-04-27T04:14:05.109 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:14:05.108+0000 7f0f820d0700 1 -- 172.21.15.162:0/666293446 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0f7c102df0 msgr2=0x7f0f7c113b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:14:05.109 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:14:05.108+0000 7f0f820d0700 1 --2- 172.21.15.162:0/666293446 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0f7c102df0 0x7f0f7c113b90 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f0f6c002a50 tx=0x7f0f6c0046b0 comp rx=0 tx=0).stop 2024-04-27T04:14:05.109 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:14:05.108+0000 7f0f820d0700 1 -- 172.21.15.162:0/666293446 shutdown_connections 2024-04-27T04:14:05.109 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:14:05.108+0000 7f0f820d0700 1 --2- 172.21.15.162:0/666293446 >> [v2:172.21.15.162:6800/901992993,v1:172.21.15.162:6801/901992993] conn(0x7f0f6406e500 0x7f0f640709b0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:14:05.109 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:14:05.108+0000 7f0f820d0700 1 --2- 172.21.15.162:0/666293446 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0f7c102df0 0x7f0f7c113b90 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:14:05.109 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:14:05.108+0000 7f0f820d0700 1 --2- 172.21.15.162:0/666293446 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0f7c103790 0x7f0f7c1140d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:14:05.109 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:14:05.108+0000 7f0f820d0700 1 -- 172.21.15.162:0/666293446 >> 172.21.15.162:0/666293446 conn(0x7f0f7c0f8980 msgr2=0x7f0f7c0f9990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:14:05.109 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:14:05.108+0000 7f0f820d0700 1 -- 172.21.15.162:0/666293446 shutdown_connections 2024-04-27T04:14:05.109 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:14:05.108+0000 7f0f820d0700 1 -- 172.21.15.162:0/666293446 wait complete. 2024-04-27T04:14:05.109 INFO:teuthology.orchestra.run.smithi162.stderr:dumped fsmap epoch 15 2024-04-27T04:14:05.213 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:14:05.213+0000 7f02b5d1d700 1 -- 172.21.15.162:0/1076233437 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f02b0101ce0 msgr2=0x7f02b01020b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:14:05.213 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:14:05.213+0000 7f02b5d1d700 1 --2- 172.21.15.162:0/1076233437 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f02b0101ce0 0x7f02b01020b0 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f02a0004940 tx=0x7f02a0009e30 comp rx=0 tx=0).stop 2024-04-27T04:14:05.213 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:14:05.213+0000 7f02b5d1d700 1 -- 172.21.15.162:0/1076233437 shutdown_connections 2024-04-27T04:14:05.214 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:14:05.213+0000 7f02b5d1d700 1 --2- 172.21.15.162:0/1076233437 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f02b0102680 0x7f02b00fbcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:14:05.214 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:14:05.213+0000 7f02b5d1d700 1 --2- 172.21.15.162:0/1076233437 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f02b0101ce0 0x7f02b01020b0 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:14:05.214 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:14:05.213+0000 7f02b5d1d700 1 -- 172.21.15.162:0/1076233437 >> 172.21.15.162:0/1076233437 conn(0x7f02b00f7870 msgr2=0x7f02b00f9c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:14:05.214 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:14:05.214+0000 7f02b5d1d700 1 -- 172.21.15.162:0/1076233437 shutdown_connections 2024-04-27T04:14:05.214 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:14:05.214+0000 7f02b5d1d700 1 -- 172.21.15.162:0/1076233437 wait complete. 2024-04-27T04:14:05.214 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:14:05.214+0000 7f02b5d1d700 1 Processor -- start 2024-04-27T04:14:05.214 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:14:05.214+0000 7f02b5d1d700 1 -- start start 2024-04-27T04:14:05.215 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:14:05.215+0000 7f02b5d1d700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f02b0101ce0 0x7f02b0115b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:14:05.215 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:14:05.215+0000 7f02b5d1d700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f02b0102680 0x7f02b01160a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:14:05.216 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:14:05.215+0000 7f02b5d1d700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02b0116780 con 0x7f02b0102680 2024-04-27T04:14:05.216 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:14:05.215+0000 7f02b5d1d700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02b010fcd0 con 0x7f02b0101ce0 2024-04-27T04:14:05.216 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:14:05.215+0000 7f02aeffd700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f02b0102680 0x7f02b01160a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:14:05.216 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:14:05.215+0000 7f02aeffd700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f02b0102680 0x7f02b01160a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:47944/0 (socket says 172.21.15.162:47944) 2024-04-27T04:14:05.216 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:14:05.215+0000 7f02aeffd700 1 -- 172.21.15.162:0/2359190773 learned_addr learned my addr 172.21.15.162:0/2359190773 (peer_addr_for_me v2:172.21.15.162:0/0) 2024-04-27T04:14:05.216 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:14:05.215+0000 7f02af7fe700 1 --2- 172.21.15.162:0/2359190773 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f02b0101ce0 0x7f02b0115b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:14:05.216 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:14:05.215+0000 7f02aeffd700 1 -- 172.21.15.162:0/2359190773 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f02b0101ce0 msgr2=0x7f02b0115b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:14:05.216 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:14:05.215+0000 7f02aeffd700 1 --2- 172.21.15.162:0/2359190773 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f02b0101ce0 0x7f02b0115b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:14:05.216 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:14:05.215+0000 7f02aeffd700 1 -- 172.21.15.162:0/2359190773 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f02a0005040 con 0x7f02b0102680 2024-04-27T04:14:05.216 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:14:05.215+0000 7f02aeffd700 1 --2- 172.21.15.162:0/2359190773 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f02b0102680 0x7f02b01160a0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f02a400b790 tx=0x7f02a4009220 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:14:05.216 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:14:05.215+0000 7f0297fff700 1 -- 172.21.15.162:0/2359190773 <== mon.0 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f02a400f6f0 con 0x7f02b0102680 2024-04-27T04:14:05.216 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:14:05.215+0000 7f0297fff700 1 -- 172.21.15.162:0/2359190773 <== mon.0 v2:172.21.15.162:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f02a400fd20 con 0x7f02b0102680 2024-04-27T04:14:05.216 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:14:05.215+0000 7f0297fff700 1 -- 172.21.15.162:0/2359190773 <== mon.0 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f02a40103a0 con 0x7f02b0102680 2024-04-27T04:14:05.217 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:14:05.215+0000 7f02b5d1d700 1 -- 172.21.15.162:0/2359190773 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f02b010ff50 con 0x7f02b0102680 2024-04-27T04:14:05.217 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:14:05.215+0000 7f02b5d1d700 1 -- 172.21.15.162:0/2359190773 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f02b0110510 con 0x7f02b0102680 2024-04-27T04:14:05.217 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:14:05.216+0000 7f02b5d1d700 1 -- 172.21.15.162:0/2359190773 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f02b00fd430 con 0x7f02b0102680 2024-04-27T04:14:05.217 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:14:05.217+0000 7f0297fff700 1 -- 172.21.15.162:0/2359190773 <== mon.0 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 32) v1 ==== 92469+0+0 (secure 0 0 0) 0x7f02a4011400 con 0x7f02b0102680 2024-04-27T04:14:05.217 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:14:05.217+0000 7f0297fff700 1 --2- 172.21.15.162:0/2359190773 >> [v2:172.21.15.162:6800/901992993,v1:172.21.15.162:6801/901992993] conn(0x7f029806e500 0x7f02980709b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:14:05.217 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:14:05.217+0000 7f0297fff700 1 -- 172.21.15.162:0/2359190773 <== mon.0 v2:172.21.15.162:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6152+0+0 (secure 0 0 0) 0x7f02a408dc00 con 0x7f02b0102680 2024-04-27T04:14:05.218 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:14:05.217+0000 7f02af7fe700 1 --2- 172.21.15.162:0/2359190773 >> [v2:172.21.15.162:6800/901992993,v1:172.21.15.162:6801/901992993] conn(0x7f029806e500 0x7f02980709b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:14:05.218 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:14:05.218+0000 7f02af7fe700 1 --2- 172.21.15.162:0/2359190773 >> [v2:172.21.15.162:6800/901992993,v1:172.21.15.162:6801/901992993] conn(0x7f029806e500 0x7f02980709b0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f02a0004f90 tx=0x7f02a0009e30 comp rx=0 tx=0).ready entity=mgr.14680 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:14:05.221 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:14:05.222+0000 7f0297fff700 1 -- 172.21.15.162:0/2359190773 <== mon.0 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186652 (secure 0 0 0) 0x7f02a4093020 con 0x7f02b0102680 2024-04-27T04:14:05.386 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:14:05 smithi162 ceph-mon[26841]: from='client.14834 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-27T04:14:05.387 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:14:05 smithi162 ceph-mon[26841]: Upgrade: Target is version 19.0.0-3253-g4e213a58 (squid) 2024-04-27T04:14:05.387 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:14:05 smithi162 ceph-mon[26841]: Upgrade: Target container is quay.ceph.io/ceph-ci/ceph@sha256:90a3a9fa9ceb8977f6dad9ccd36fe46018d07e2118199820523a6b35301d9c16, digests ['quay.ceph.io/ceph-ci/ceph@sha256:90a3a9fa9ceb8977f6dad9ccd36fe46018d07e2118199820523a6b35301d9c16'] 2024-04-27T04:14:05.387 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:14:05 smithi162 ceph-mon[26841]: Upgrade: Setting container_image for all mgr 2024-04-27T04:14:05.387 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:14:05 smithi162 ceph-mon[26841]: from='client.14838 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-27T04:14:05.387 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:14:05 smithi162 ceph-mon[26841]: from='client.? 172.21.15.162:0/16176343' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-27T04:14:05.422 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:14:05 smithi181 ceph-mon[33774]: from='client.14834 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-27T04:14:05.422 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:14:05 smithi181 ceph-mon[33774]: Upgrade: Target is version 19.0.0-3253-g4e213a58 (squid) 2024-04-27T04:14:05.422 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:14:05 smithi181 ceph-mon[33774]: Upgrade: Target container is quay.ceph.io/ceph-ci/ceph@sha256:90a3a9fa9ceb8977f6dad9ccd36fe46018d07e2118199820523a6b35301d9c16, digests ['quay.ceph.io/ceph-ci/ceph@sha256:90a3a9fa9ceb8977f6dad9ccd36fe46018d07e2118199820523a6b35301d9c16'] 2024-04-27T04:14:05.422 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:14:05 smithi181 ceph-mon[33774]: Upgrade: Setting container_image for all mgr 2024-04-27T04:14:05.422 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:14:05 smithi181 ceph-mon[33774]: from='client.14838 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-27T04:14:05.422 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:14:05 smithi181 ceph-mon[33774]: from='client.? 172.21.15.162:0/16176343' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-27T04:14:05.443 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:14:05.444+0000 7f02b5d1d700 1 -- 172.21.15.162:0/2359190773 --> [v2:172.21.15.162:6800/901992993,v1:172.21.15.162:6801/901992993] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f02b0108ec0 con 0x7f029806e500 2024-04-27T04:14:05.445 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:14:05.446+0000 7f0297fff700 1 -- 172.21.15.162:0/2359190773 <== mgr.14680 v2:172.21.15.162:6800/901992993 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+368 (secure 0 0 0) 0x7f02b0108ec0 con 0x7f029806e500 2024-04-27T04:14:05.445 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-04-27T04:14:05.445 INFO:teuthology.orchestra.run.smithi162.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:90a3a9fa9ceb8977f6dad9ccd36fe46018d07e2118199820523a6b35301d9c16", 2024-04-27T04:14:05.445 INFO:teuthology.orchestra.run.smithi162.stdout: "in_progress": true, 2024-04-27T04:14:05.445 INFO:teuthology.orchestra.run.smithi162.stdout: "which": "Upgrading all daemon types on all hosts", 2024-04-27T04:14:05.445 INFO:teuthology.orchestra.run.smithi162.stdout: "services_complete": [ 2024-04-27T04:14:05.445 INFO:teuthology.orchestra.run.smithi162.stdout: "mgr" 2024-04-27T04:14:05.445 INFO:teuthology.orchestra.run.smithi162.stdout: ], 2024-04-27T04:14:05.445 INFO:teuthology.orchestra.run.smithi162.stdout: "progress": "2/21 daemons upgraded", 2024-04-27T04:14:05.445 INFO:teuthology.orchestra.run.smithi162.stdout: "message": "Currently upgrading mon daemons", 2024-04-27T04:14:05.445 INFO:teuthology.orchestra.run.smithi162.stdout: "is_paused": false 2024-04-27T04:14:05.446 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-04-27T04:14:05.447 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:14:05.448+0000 7f02b5d1d700 1 -- 172.21.15.162:0/2359190773 >> [v2:172.21.15.162:6800/901992993,v1:172.21.15.162:6801/901992993] conn(0x7f029806e500 msgr2=0x7f02980709b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:14:05.447 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:14:05.448+0000 7f02b5d1d700 1 --2- 172.21.15.162:0/2359190773 >> [v2:172.21.15.162:6800/901992993,v1:172.21.15.162:6801/901992993] conn(0x7f029806e500 0x7f02980709b0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f02a0004f90 tx=0x7f02a0009e30 comp rx=0 tx=0).stop 2024-04-27T04:14:05.447 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:14:05.448+0000 7f02b5d1d700 1 -- 172.21.15.162:0/2359190773 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f02b0102680 msgr2=0x7f02b01160a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:14:05.447 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:14:05.448+0000 7f02b5d1d700 1 --2- 172.21.15.162:0/2359190773 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f02b0102680 0x7f02b01160a0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f02a400b790 tx=0x7f02a4009220 comp rx=0 tx=0).stop 2024-04-27T04:14:05.447 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:14:05.448+0000 7f02b5d1d700 1 -- 172.21.15.162:0/2359190773 shutdown_connections 2024-04-27T04:14:05.448 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:14:05.448+0000 7f02b5d1d700 1 --2- 172.21.15.162:0/2359190773 >> [v2:172.21.15.162:6800/901992993,v1:172.21.15.162:6801/901992993] conn(0x7f029806e500 0x7f02980709b0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:14:05.448 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:14:05.448+0000 7f02b5d1d700 1 --2- 172.21.15.162:0/2359190773 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f02b0101ce0 0x7f02b0115b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:14:05.448 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:14:05.448+0000 7f02b5d1d700 1 --2- 172.21.15.162:0/2359190773 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f02b0102680 0x7f02b01160a0 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:14:05.448 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:14:05.448+0000 7f02b5d1d700 1 -- 172.21.15.162:0/2359190773 >> 172.21.15.162:0/2359190773 conn(0x7f02b00f7870 msgr2=0x7f02b00662e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:14:05.448 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:14:05.448+0000 7f02b5d1d700 1 -- 172.21.15.162:0/2359190773 shutdown_connections 2024-04-27T04:14:05.448 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:14:05.448+0000 7f02b5d1d700 1 -- 172.21.15.162:0/2359190773 wait complete. 2024-04-27T04:14:05.556 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:14:05.557+0000 7f8db46b9700 1 -- 172.21.15.162:0/3155074336 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8dac0fe800 msgr2=0x7f8dac106e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:14:05.556 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:14:05.557+0000 7f8db46b9700 1 --2- 172.21.15.162:0/3155074336 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8dac0fe800 0x7f8dac106e40 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f8d9c004940 tx=0x7f8d9c009e30 comp rx=0 tx=0).stop 2024-04-27T04:14:05.556 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:14:05.557+0000 7f8db46b9700 1 -- 172.21.15.162:0/3155074336 shutdown_connections 2024-04-27T04:14:05.557 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:14:05.557+0000 7f8db46b9700 1 --2- 172.21.15.162:0/3155074336 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8dac0fe800 0x7f8dac106e40 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:14:05.557 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:14:05.557+0000 7f8db46b9700 1 --2- 172.21.15.162:0/3155074336 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8dac0fde60 0x7f8dac0fe230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:14:05.557 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:14:05.557+0000 7f8db46b9700 1 -- 172.21.15.162:0/3155074336 >> 172.21.15.162:0/3155074336 conn(0x7f8dac0f78d0 msgr2=0x7f8dac0f9ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:14:05.557 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:14:05.557+0000 7f8db46b9700 1 -- 172.21.15.162:0/3155074336 shutdown_connections 2024-04-27T04:14:05.557 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:14:05.557+0000 7f8db46b9700 1 -- 172.21.15.162:0/3155074336 wait complete. 2024-04-27T04:14:05.558 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:14:05.558+0000 7f8db46b9700 1 Processor -- start 2024-04-27T04:14:05.558 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:14:05.558+0000 7f8db46b9700 1 -- start start 2024-04-27T04:14:05.559 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:14:05.558+0000 7f8db46b9700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8dac0fde60 0x7f8dac113b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:14:05.559 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:14:05.558+0000 7f8db46b9700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8dac0fe800 0x7f8dac114070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:14:05.559 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:14:05.558+0000 7f8db46b9700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8dac114750 con 0x7f8dac0fe800 2024-04-27T04:14:05.559 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:14:05.558+0000 7f8db46b9700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8dac10dab0 con 0x7f8dac0fde60 2024-04-27T04:14:05.559 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:14:05.558+0000 7f8db1c54700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8dac0fe800 0x7f8dac114070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:14:05.559 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:14:05.558+0000 7f8db1c54700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8dac0fe800 0x7f8dac114070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:47968/0 (socket says 172.21.15.162:47968) 2024-04-27T04:14:05.559 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:14:05.558+0000 7f8db1c54700 1 -- 172.21.15.162:0/4012531731 learned_addr learned my addr 172.21.15.162:0/4012531731 (peer_addr_for_me v2:172.21.15.162:0/0) 2024-04-27T04:14:05.559 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:14:05.558+0000 7f8db2455700 1 --2- 172.21.15.162:0/4012531731 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8dac0fde60 0x7f8dac113b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:14:05.559 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:14:05.558+0000 7f8db1c54700 1 -- 172.21.15.162:0/4012531731 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8dac0fde60 msgr2=0x7f8dac113b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:14:05.559 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:14:05.558+0000 7f8db1c54700 1 --2- 172.21.15.162:0/4012531731 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8dac0fde60 0x7f8dac113b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:14:05.559 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:14:05.558+0000 7f8db1c54700 1 -- 172.21.15.162:0/4012531731 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d9c005040 con 0x7f8dac0fe800 2024-04-27T04:14:05.559 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:14:05.558+0000 7f8db2455700 1 --2- 172.21.15.162:0/4012531731 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8dac0fde60 0x7f8dac113b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-27T04:14:05.559 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:14:05.558+0000 7f8db1c54700 1 --2- 172.21.15.162:0/4012531731 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8dac0fe800 0x7f8dac114070 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f8d9c00b3b0 tx=0x7f8d9c009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:14:05.559 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:14:05.559+0000 7f8da2ffd700 1 -- 172.21.15.162:0/4012531731 <== mon.0 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8d9c00e040 con 0x7f8dac0fe800 2024-04-27T04:14:05.560 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:14:05.559+0000 7f8da2ffd700 1 -- 172.21.15.162:0/4012531731 <== mon.0 v2:172.21.15.162:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f8d9c013420 con 0x7f8dac0fe800 2024-04-27T04:14:05.560 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:14:05.559+0000 7f8da2ffd700 1 -- 172.21.15.162:0/4012531731 <== mon.0 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8d9c01c630 con 0x7f8dac0fe800 2024-04-27T04:14:05.560 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:14:05.559+0000 7f8db46b9700 1 -- 172.21.15.162:0/4012531731 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8dac10dd90 con 0x7f8dac0fe800 2024-04-27T04:14:05.560 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:14:05.559+0000 7f8db46b9700 1 -- 172.21.15.162:0/4012531731 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8dac10e2d0 con 0x7f8dac0fe800 2024-04-27T04:14:05.561 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:14:05.560+0000 7f8db46b9700 1 -- 172.21.15.162:0/4012531731 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8dac0666f0 con 0x7f8dac0fe800 2024-04-27T04:14:05.561 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:14:05.560+0000 7f8da2ffd700 1 -- 172.21.15.162:0/4012531731 <== mon.0 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 32) v1 ==== 92469+0+0 (secure 0 0 0) 0x7f8d9c00baf0 con 0x7f8dac0fe800 2024-04-27T04:14:05.561 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:14:05.561+0000 7f8da2ffd700 1 --2- 172.21.15.162:0/4012531731 >> [v2:172.21.15.162:6800/901992993,v1:172.21.15.162:6801/901992993] conn(0x7f8d9806e510 0x7f8d980709c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:14:05.561 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:14:05.561+0000 7f8da2ffd700 1 -- 172.21.15.162:0/4012531731 <== mon.0 v2:172.21.15.162:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6152+0+0 (secure 0 0 0) 0x7f8d9c08f1b0 con 0x7f8dac0fe800 2024-04-27T04:14:05.561 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:14:05.561+0000 7f8db2455700 1 --2- 172.21.15.162:0/4012531731 >> [v2:172.21.15.162:6800/901992993,v1:172.21.15.162:6801/901992993] conn(0x7f8d9806e510 0x7f8d980709c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:14:05.561 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:14:05.561+0000 7f8db2455700 1 --2- 172.21.15.162:0/4012531731 >> [v2:172.21.15.162:6800/901992993,v1:172.21.15.162:6801/901992993] conn(0x7f8d9806e510 0x7f8d980709c0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f8da8009b10 tx=0x7f8da8008040 comp rx=0 tx=0).ready entity=mgr.14680 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:14:05.564 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:14:05.565+0000 7f8da2ffd700 1 -- 172.21.15.162:0/4012531731 <== mon.0 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186652 (secure 0 0 0) 0x7f8d9c05c340 con 0x7f8dac0fe800 2024-04-27T04:14:05.855 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:14:05.855+0000 7f8db46b9700 1 -- 172.21.15.162:0/4012531731 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f8dac05f5c0 con 0x7f8dac0fe800 2024-04-27T04:14:05.855 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:14:05.856+0000 7f8da2ffd700 1 -- 172.21.15.162:0/4012531731 <== mon.0 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+10 (secure 0 0 0) 0x7f8d9c05bed0 con 0x7f8dac0fe800 2024-04-27T04:14:05.855 INFO:teuthology.orchestra.run.smithi162.stdout:HEALTH_OK 2024-04-27T04:14:05.857 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:14:05.858+0000 7f8db46b9700 1 -- 172.21.15.162:0/4012531731 >> [v2:172.21.15.162:6800/901992993,v1:172.21.15.162:6801/901992993] conn(0x7f8d9806e510 msgr2=0x7f8d980709c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:14:05.858 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:14:05.858+0000 7f8db46b9700 1 --2- 172.21.15.162:0/4012531731 >> [v2:172.21.15.162:6800/901992993,v1:172.21.15.162:6801/901992993] conn(0x7f8d9806e510 0x7f8d980709c0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f8da8009b10 tx=0x7f8da8008040 comp rx=0 tx=0).stop 2024-04-27T04:14:05.858 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:14:05.858+0000 7f8db46b9700 1 -- 172.21.15.162:0/4012531731 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8dac0fe800 msgr2=0x7f8dac114070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:14:05.858 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:14:05.858+0000 7f8db46b9700 1 --2- 172.21.15.162:0/4012531731 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8dac0fe800 0x7f8dac114070 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f8d9c00b3b0 tx=0x7f8d9c009e30 comp rx=0 tx=0).stop 2024-04-27T04:14:05.858 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:14:05.858+0000 7f8db46b9700 1 -- 172.21.15.162:0/4012531731 shutdown_connections 2024-04-27T04:14:05.858 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:14:05.858+0000 7f8db46b9700 1 --2- 172.21.15.162:0/4012531731 >> [v2:172.21.15.162:6800/901992993,v1:172.21.15.162:6801/901992993] conn(0x7f8d9806e510 0x7f8d980709c0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:14:05.858 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:14:05.858+0000 7f8db46b9700 1 --2- 172.21.15.162:0/4012531731 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8dac0fde60 0x7f8dac113b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:14:05.858 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:14:05.858+0000 7f8db46b9700 1 --2- 172.21.15.162:0/4012531731 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8dac0fe800 0x7f8dac114070 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:14:05.858 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:14:05.858+0000 7f8db46b9700 1 -- 172.21.15.162:0/4012531731 >> 172.21.15.162:0/4012531731 conn(0x7f8dac0f78d0 msgr2=0x7f8dac0f8e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:14:05.858 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:14:05.858+0000 7f8db46b9700 1 -- 172.21.15.162:0/4012531731 shutdown_connections 2024-04-27T04:14:05.858 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:14:05.859+0000 7f8db46b9700 1 -- 172.21.15.162:0/4012531731 wait complete. 2024-04-27T04:14:06.120 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:14:06 smithi162 ceph-mon[26841]: pgmap v79: 65 pgs: 65 active+clean; 208 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:14:06.121 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:14:06 smithi162 ceph-mon[26841]: from='client.14842 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-27T04:14:06.121 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:14:06 smithi162 ceph-mon[26841]: from='client.? 172.21.15.162:0/666293446' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-27T04:14:06.121 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:14:06 smithi162 ceph-mon[26841]: from='mgr.14680 172.21.15.162:0/501880828' entity='mgr.smithi162.oennfn' 2024-04-27T04:14:06.121 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:14:06 smithi162 ceph-mon[26841]: from='mgr.14680 172.21.15.162:0/501880828' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-27T04:14:06.121 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:14:06 smithi162 ceph-mon[26841]: from='mgr.14680 172.21.15.162:0/501880828' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-27T04:14:06.121 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:14:06 smithi162 ceph-mon[26841]: from='mgr.14680 172.21.15.162:0/501880828' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T04:14:06.121 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:14:06 smithi162 ceph-mon[26841]: from='client.? 172.21.15.162:0/4012531731' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-27T04:14:06.422 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:14:06 smithi181 ceph-mon[33774]: pgmap v79: 65 pgs: 65 active+clean; 208 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:14:06.422 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:14:06 smithi181 ceph-mon[33774]: from='client.14842 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-27T04:14:06.422 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:14:06 smithi181 ceph-mon[33774]: from='client.? 172.21.15.162:0/666293446' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-27T04:14:06.423 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:14:06 smithi181 ceph-mon[33774]: from='mgr.14680 172.21.15.162:0/501880828' entity='mgr.smithi162.oennfn' 2024-04-27T04:14:06.423 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:14:06 smithi181 ceph-mon[33774]: from='mgr.14680 172.21.15.162:0/501880828' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-27T04:14:06.423 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:14:06 smithi181 ceph-mon[33774]: from='mgr.14680 172.21.15.162:0/501880828' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-27T04:14:06.423 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:14:06 smithi181 ceph-mon[33774]: from='mgr.14680 172.21.15.162:0/501880828' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T04:14:06.423 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:14:06 smithi181 ceph-mon[33774]: from='client.? 172.21.15.162:0/4012531731' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-27T04:14:07.328 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:14:07 smithi162 ceph-mon[26841]: from='client.14854 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-27T04:14:07.328 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:14:07 smithi162 ceph-mon[26841]: Upgrade: Updating mon.smithi162 2024-04-27T04:14:07.328 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:14:07 smithi162 ceph-mon[26841]: Deploying daemon mon.smithi162 on smithi162 2024-04-27T04:14:07.422 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:14:07 smithi181 ceph-mon[33774]: from='client.14854 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-27T04:14:07.422 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:14:07 smithi181 ceph-mon[33774]: Upgrade: Updating mon.smithi162 2024-04-27T04:14:07.422 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:14:07 smithi181 ceph-mon[33774]: Deploying daemon mon.smithi162 on smithi162 2024-04-27T04:14:07.582 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:14:07 smithi162 systemd[1]: Stopping Ceph mon.smithi162 for 7f9750d6-044a-11ef-bc93-c7b262605968... 2024-04-27T04:14:07.886 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:14:07 smithi162 ceph-7f9750d6-044a-11ef-bc93-c7b262605968-mon-smithi162[26837]: 2024-04-27T04:14:07.582+0000 7f82be9b6700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.smithi162 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-04-27T04:14:07.887 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:14:07 smithi162 ceph-7f9750d6-044a-11ef-bc93-c7b262605968-mon-smithi162[26837]: 2024-04-27T04:14:07.582+0000 7f82be9b6700 -1 mon.smithi162@0(leader) e2 *** Got Signal Terminated *** 2024-04-27T04:14:07.887 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:14:07 smithi162 podman[110467]: 2024-04-27 04:14:07.746935017 +0000 UTC m=+0.274095192 container died 0f1e3682b9abe73f7036e1738bc1dc3ccb64fdc5377b198e5b3f4cd759ea5a3c (image=quay.ceph.io/ceph-ci/ceph:quincy, name=ceph-7f9750d6-044a-11ef-bc93-c7b262605968-mon-smithi162, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.build-date=20240424, ceph=True, maintainer=Guillaume Abrioux , RELEASE=quincy-2be3efa, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b) 2024-04-27T04:14:08.365 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:14:08 smithi162 podman[110467]: 2024-04-27 04:14:08.116691762 +0000 UTC m=+0.643851922 container cleanup 0f1e3682b9abe73f7036e1738bc1dc3ccb64fdc5377b198e5b3f4cd759ea5a3c (image=quay.ceph.io/ceph-ci/ceph:quincy, name=ceph-7f9750d6-044a-11ef-bc93-c7b262605968-mon-smithi162, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240424, io.buildah.version=1.33.5, RELEASE=quincy-2be3efa, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True) 2024-04-27T04:14:08.366 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:14:08 smithi162 bash[110467]: ceph-7f9750d6-044a-11ef-bc93-c7b262605968-mon-smithi162 2024-04-27T04:14:08.636 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:14:08 smithi162 podman[110492]: 2024-04-27 04:14:08.367880414 +0000 UTC m=+0.623198306 container remove 0f1e3682b9abe73f7036e1738bc1dc3ccb64fdc5377b198e5b3f4cd759ea5a3c (image=quay.ceph.io/ceph-ci/ceph:quincy, name=ceph-7f9750d6-044a-11ef-bc93-c7b262605968-mon-smithi162, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240424, GIT_CLEAN=True, ceph=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=quincy-2be3efa, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD) 2024-04-27T04:14:08.637 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:14:08 smithi162 systemd[1]: ceph-7f9750d6-044a-11ef-bc93-c7b262605968@mon.smithi162.service: Deactivated successfully. 2024-04-27T04:14:08.637 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:14:08 smithi162 systemd[1]: Stopped Ceph mon.smithi162 for 7f9750d6-044a-11ef-bc93-c7b262605968. 2024-04-27T04:14:08.637 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:14:08 smithi162 systemd[1]: ceph-7f9750d6-044a-11ef-bc93-c7b262605968@mon.smithi162.service: Consumed 12.617s CPU time. 2024-04-27T04:14:08.983 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:14:08 smithi162 systemd[1]: Starting Ceph mon.smithi162 for 7f9750d6-044a-11ef-bc93-c7b262605968... 2024-04-27T04:14:09.246 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:14:08 smithi162 podman[110622]: 2024-04-27 04:14:08.98586076 +0000 UTC m=+0.114663419 container create 375adaf4468f7520330d3d23debb783bfbda1c09ddf0808a70b1c616606c5800 (image=quay.ceph.io/ceph-ci/ceph@sha256:90a3a9fa9ceb8977f6dad9ccd36fe46018d07e2118199820523a6b35301d9c16, name=ceph-7f9750d6-044a-11ef-bc93-c7b262605968-mon-smithi162, ceph=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240424, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.vendor=CentOS, RELEASE=main-4e213a5, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-27T04:14:09.246 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:14:08 smithi162 podman[110622]: 2024-04-27 04:14:08.88972776 +0000 UTC m=+0.018530368 image pull 67e414ff4bc41092cb31c84c98a9563a32813d02583028679c46b40362afeb3e quay.ceph.io/ceph-ci/ceph@sha256:90a3a9fa9ceb8977f6dad9ccd36fe46018d07e2118199820523a6b35301d9c16 2024-04-27T04:14:09.246 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:14:09 smithi162 podman[110622]: 2024-04-27 04:14:09.221754913 +0000 UTC m=+0.350557510 container init 375adaf4468f7520330d3d23debb783bfbda1c09ddf0808a70b1c616606c5800 (image=quay.ceph.io/ceph-ci/ceph@sha256:90a3a9fa9ceb8977f6dad9ccd36fe46018d07e2118199820523a6b35301d9c16, name=ceph-7f9750d6-044a-11ef-bc93-c7b262605968-mon-smithi162, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_CLEAN=True, RELEASE=main-4e213a5, org.label-schema.build-date=20240424, ceph=True, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux ) 2024-04-27T04:14:09.246 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:14:09 smithi162 podman[110622]: 2024-04-27 04:14:09.224695082 +0000 UTC m=+0.353497679 container start 375adaf4468f7520330d3d23debb783bfbda1c09ddf0808a70b1c616606c5800 (image=quay.ceph.io/ceph-ci/ceph@sha256:90a3a9fa9ceb8977f6dad9ccd36fe46018d07e2118199820523a6b35301d9c16, name=ceph-7f9750d6-044a-11ef-bc93-c7b262605968-mon-smithi162, org.label-schema.build-date=20240424, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, RELEASE=main-4e213a5, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_BRANCH=HEAD) 2024-04-27T04:14:09.637 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:14:09 smithi162 ceph-mon[110641]: set uid:gid to 167:167 (ceph:ceph) 2024-04-27T04:14:09.637 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:14:09 smithi162 ceph-mon[110641]: ceph version 19.0.0-3253-g4e213a58 (4e213a58e787437ff508e6c890ccd5fd32dc96b5) squid (dev), process ceph-mon, pid 2 2024-04-27T04:14:09.637 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:14:09 smithi162 ceph-mon[110641]: pidfile_write: ignore empty --pid-file 2024-04-27T04:14:09.637 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:14:09 smithi162 ceph-mon[110641]: load: jerasure load: lrc 2024-04-27T04:14:09.637 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:14:09 smithi162 ceph-mon[110641]: rocksdb: RocksDB version: 7.9.2 2024-04-27T04:14:09.637 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:14:09 smithi162 ceph-mon[110641]: rocksdb: Git sha 0 2024-04-27T04:14:09.637 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:14:09 smithi162 ceph-mon[110641]: rocksdb: Compile date 2024-04-24 22:40:16 2024-04-27T04:14:09.638 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:14:09 smithi162 ceph-mon[110641]: rocksdb: DB SUMMARY 2024-04-27T04:14:09.638 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:14:09 smithi162 ceph-mon[110641]: rocksdb: DB Session ID: 3YPTOS294AN7DCM78MUO 2024-04-27T04:14:09.638 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:14:09 smithi162 ceph-mon[110641]: rocksdb: CURRENT file: CURRENT 2024-04-27T04:14:09.638 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:14:09 smithi162 ceph-mon[110641]: rocksdb: IDENTITY file: IDENTITY 2024-04-27T04:14:09.638 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:14:09 smithi162 ceph-mon[110641]: rocksdb: MANIFEST file: MANIFEST-000015 size: 1159 Bytes 2024-04-27T04:14:09.638 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:14:09 smithi162 ceph-mon[110641]: rocksdb: SST files in /var/lib/ceph/mon/ceph-smithi162/store.db dir, Total Num: 1, files: 000033.sst 2024-04-27T04:14:09.638 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:14:09 smithi162 ceph-mon[110641]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-smithi162/store.db: 000031.log size: 1812684 ; 2024-04-27T04:14:09.638 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:14:09 smithi162 ceph-mon[110641]: rocksdb: Options.error_if_exists: 0 2024-04-27T04:14:09.638 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:14:09 smithi162 ceph-mon[110641]: rocksdb: Options.create_if_missing: 0 2024-04-27T04:14:09.638 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:14:09 smithi162 ceph-mon[110641]: rocksdb: Options.paranoid_checks: 1 2024-04-27T04:14:09.638 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:14:09 smithi162 ceph-mon[110641]: rocksdb: Options.flush_verify_memtable_count: 1 2024-04-27T04:14:09.638 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:14:09 smithi162 ceph-mon[110641]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-04-27T04:14:09.638 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:14:09 smithi162 ceph-mon[110641]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-04-27T04:14:09.638 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:14:09 smithi162 ceph-mon[110641]: rocksdb: Options.env: 0x56301c096cc0 2024-04-27T04:14:09.639 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:14:09 smithi162 ceph-mon[110641]: rocksdb: Options.fs: PosixFileSystem 2024-04-27T04:14:09.639 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:14:09 smithi162 ceph-mon[110641]: rocksdb: Options.info_log: 0x56301dd19c60 2024-04-27T04:14:09.639 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:14:09 smithi162 ceph-mon[110641]: rocksdb: Options.max_file_opening_threads: 16 2024-04-27T04:14:09.639 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:14:09 smithi162 ceph-mon[110641]: rocksdb: Options.statistics: (nil) 2024-04-27T04:14:09.639 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:14:09 smithi162 ceph-mon[110641]: rocksdb: Options.use_fsync: 0 2024-04-27T04:14:09.639 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:14:09 smithi162 ceph-mon[110641]: rocksdb: Options.max_log_file_size: 0 2024-04-27T04:14:09.639 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:14:09 smithi162 ceph-mon[110641]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-04-27T04:14:09.639 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:14:09 smithi162 ceph-mon[110641]: rocksdb: Options.log_file_time_to_roll: 0 2024-04-27T04:14:09.639 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:14:09 smithi162 ceph-mon[110641]: rocksdb: Options.keep_log_file_num: 1000 2024-04-27T04:14:09.639 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:14:09 smithi162 ceph-mon[110641]: rocksdb: Options.recycle_log_file_num: 0 2024-04-27T04:14:09.639 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:14:09 smithi162 ceph-mon[110641]: rocksdb: Options.allow_fallocate: 1 2024-04-27T04:14:09.639 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:14:09 smithi162 ceph-mon[110641]: rocksdb: Options.allow_mmap_reads: 0 2024-04-27T04:14:09.639 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:14:09 smithi162 ceph-mon[110641]: rocksdb: Options.allow_mmap_writes: 0 2024-04-27T04:14:09.639 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:14:09 smithi162 ceph-mon[110641]: rocksdb: Options.use_direct_reads: 0 2024-04-27T04:14:09.639 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:14:09 smithi162 ceph-mon[110641]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-04-27T04:14:09.640 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:14:09 smithi162 ceph-mon[110641]: rocksdb: Options.create_missing_column_families: 0 2024-04-27T04:14:09.640 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:14:09 smithi162 ceph-mon[110641]: rocksdb: Options.db_log_dir: 2024-04-27T04:14:09.640 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:14:09 smithi162 ceph-mon[110641]: rocksdb: Options.wal_dir: 2024-04-27T04:14:09.641 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:14:09 smithi162 ceph-mon[110641]: rocksdb: Options.table_cache_numshardbits: 6 2024-04-27T04:14:09.641 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:14:09 smithi162 ceph-mon[110641]: rocksdb: Options.WAL_ttl_seconds: 0 2024-04-27T04:14:09.641 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:14:09 smithi162 ceph-mon[110641]: rocksdb: Options.WAL_size_limit_MB: 0 2024-04-27T04:14:09.641 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:14:09 smithi162 ceph-mon[110641]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-04-27T04:14:09.641 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:14:09 smithi162 ceph-mon[110641]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-04-27T04:14:09.641 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:14:09 smithi162 ceph-mon[110641]: rocksdb: Options.is_fd_close_on_exec: 1 2024-04-27T04:14:09.641 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:14:09 smithi162 ceph-mon[110641]: rocksdb: Options.advise_random_on_open: 1 2024-04-27T04:14:09.642 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:14:09 smithi162 ceph-mon[110641]: rocksdb: Options.db_write_buffer_size: 0 2024-04-27T04:14:09.642 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:14:09 smithi162 ceph-mon[110641]: rocksdb: Options.write_buffer_manager: 0x56301dceb0e0 2024-04-27T04:14:09.642 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:14:09 smithi162 ceph-mon[110641]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-04-27T04:14:09.642 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:14:09 smithi162 ceph-mon[110641]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-04-27T04:14:09.642 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:14:09 smithi162 ceph-mon[110641]: rocksdb: Options.use_adaptive_mutex: 0 2024-04-27T04:14:09.642 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:14:09 smithi162 ceph-mon[110641]: rocksdb: Options.rate_limiter: (nil) 2024-04-27T04:14:09.642 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:14:09 smithi162 ceph-mon[110641]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-04-27T04:14:09.642 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:14:09 smithi162 ceph-mon[110641]: rocksdb: Options.wal_recovery_mode: 2 2024-04-27T04:14:09.642 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:14:09 smithi162 ceph-mon[110641]: rocksdb: Options.enable_thread_tracking: 0 2024-04-27T04:14:09.642 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:14:09 smithi162 ceph-mon[110641]: rocksdb: Options.enable_pipelined_write: 0 2024-04-27T04:14:09.642 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:14:09 smithi162 ceph-mon[110641]: rocksdb: Options.unordered_write: 0 2024-04-27T04:14:09.642 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:14:09 smithi162 ceph-mon[110641]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-04-27T04:14:09.642 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:14:09 smithi162 ceph-mon[110641]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-04-27T04:14:09.642 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:14:09 smithi162 ceph-mon[110641]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-04-27T04:14:09.642 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:14:09 smithi162 ceph-mon[110641]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-04-27T04:14:09.643 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:14:09 smithi162 ceph-mon[110641]: rocksdb: Options.row_cache: None 2024-04-27T04:14:09.643 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:14:09 smithi162 ceph-mon[110641]: rocksdb: Options.wal_filter: None 2024-04-27T04:14:09.643 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:14:09 smithi162 ceph-mon[110641]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-04-27T04:14:09.643 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:14:09 smithi162 ceph-mon[110641]: rocksdb: Options.allow_ingest_behind: 0 2024-04-27T04:14:09.643 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:14:09 smithi162 ceph-mon[110641]: rocksdb: Options.two_write_queues: 0 2024-04-27T04:14:09.643 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:14:09 smithi162 ceph-mon[110641]: rocksdb: Options.manual_wal_flush: 0 2024-04-27T04:14:09.643 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:14:09 smithi162 ceph-mon[110641]: rocksdb: Options.wal_compression: 0 2024-04-27T04:14:09.643 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:14:09 smithi162 ceph-mon[110641]: rocksdb: Options.atomic_flush: 0 2024-04-27T04:14:09.643 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:14:09 smithi162 ceph-mon[110641]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-04-27T04:14:09.643 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:14:09 smithi162 ceph-mon[110641]: rocksdb: Options.persist_stats_to_disk: 0 2024-04-27T04:14:09.643 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:14:09 smithi162 ceph-mon[110641]: rocksdb: Options.write_dbid_to_manifest: 0 2024-04-27T04:14:09.643 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:14:09 smithi162 ceph-mon[110641]: rocksdb: Options.log_readahead_size: 0 2024-04-27T04:14:09.643 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:14:09 smithi162 ceph-mon[110641]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-04-27T04:14:09.643 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:14:09 smithi162 ceph-mon[110641]: rocksdb: Options.best_efforts_recovery: 0 2024-04-27T04:14:09.643 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:14:09 smithi162 ceph-mon[110641]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-04-27T04:14:09.644 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:14:09 smithi162 ceph-mon[110641]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-04-27T04:14:09.644 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:14:09 smithi162 ceph-mon[110641]: rocksdb: Options.allow_data_in_errors: 0 2024-04-27T04:14:09.644 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:14:09 smithi162 ceph-mon[110641]: rocksdb: Options.db_host_id: __hostname__ 2024-04-27T04:14:09.644 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:14:09 smithi162 ceph-mon[110641]: rocksdb: Options.enforce_single_del_contracts: true 2024-04-27T04:14:09.644 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:14:09 smithi162 ceph-mon[110641]: rocksdb: Options.max_background_jobs: 2 2024-04-27T04:14:09.644 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:14:09 smithi162 ceph-mon[110641]: rocksdb: Options.max_background_compactions: -1 2024-04-27T04:14:09.644 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:14:09 smithi162 ceph-mon[110641]: rocksdb: Options.max_subcompactions: 1 2024-04-27T04:14:09.644 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:14:09 smithi162 ceph-mon[110641]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-04-27T04:14:09.644 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:14:09 smithi162 ceph-mon[110641]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-04-27T04:14:09.644 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:14:09 smithi162 ceph-mon[110641]: rocksdb: Options.delayed_write_rate : 16777216 2024-04-27T04:14:09.644 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:14:09 smithi162 ceph-mon[110641]: rocksdb: Options.max_total_wal_size: 0 2024-04-27T04:14:09.644 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:14:09 smithi162 ceph-mon[110641]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-04-27T04:14:09.644 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:14:09 smithi162 ceph-mon[110641]: rocksdb: Options.stats_dump_period_sec: 600 2024-04-27T04:14:09.644 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:14:09 smithi162 ceph-mon[110641]: rocksdb: Options.stats_persist_period_sec: 600 2024-04-27T04:14:09.644 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:14:09 smithi162 ceph-mon[110641]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-04-27T04:14:09.644 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:14:09 smithi162 ceph-mon[110641]: rocksdb: Options.max_open_files: -1 2024-04-27T04:14:09.645 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:14:09 smithi162 ceph-mon[110641]: rocksdb: Options.bytes_per_sync: 0 2024-04-27T04:14:09.645 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:14:09 smithi162 ceph-mon[110641]: rocksdb: Options.wal_bytes_per_sync: 0 2024-04-27T04:14:09.645 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:14:09 smithi162 ceph-mon[110641]: rocksdb: Options.strict_bytes_per_sync: 0 2024-04-27T04:14:09.645 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:14:09 smithi162 ceph-mon[110641]: rocksdb: Options.compaction_readahead_size: 0 2024-04-27T04:14:09.645 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:14:09 smithi162 ceph-mon[110641]: rocksdb: Options.max_background_flushes: -1 2024-04-27T04:14:09.645 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:14:09 smithi162 ceph-mon[110641]: rocksdb: Compression algorithms supported: 2024-04-27T04:14:09.645 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:14:09 smithi162 ceph-mon[110641]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-04-27T04:14:09.645 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:14:09 smithi162 ceph-mon[110641]: rocksdb: kZSTD supported: 0 2024-04-27T04:14:09.645 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:14:09 smithi162 ceph-mon[110641]: rocksdb: kXpressCompression supported: 0 2024-04-27T04:14:09.645 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:14:09 smithi162 ceph-mon[110641]: rocksdb: kLZ4HCCompression supported: 1 2024-04-27T04:14:09.645 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:14:09 smithi162 ceph-mon[110641]: rocksdb: kZlibCompression supported: 1 2024-04-27T04:14:09.645 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:14:09 smithi162 ceph-mon[110641]: rocksdb: kSnappyCompression supported: 1 2024-04-27T04:14:09.645 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:14:09 smithi162 ceph-mon[110641]: rocksdb: kLZ4Compression supported: 1 2024-04-27T04:14:09.645 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:14:09 smithi162 ceph-mon[110641]: rocksdb: kBZip2Compression supported: 0 2024-04-27T04:14:09.645 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:14:09 smithi162 ceph-mon[110641]: rocksdb: Fast CRC32 supported: Supported on x86 2024-04-27T04:14:09.646 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:14:09 smithi162 ceph-mon[110641]: rocksdb: DMutex implementation: pthread_mutex_t 2024-04-27T04:14:09.646 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:14:09 smithi162 ceph-mon[110641]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-smithi162/store.db/MANIFEST-000015 2024-04-27T04:14:09.646 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:14:09 smithi162 ceph-mon[110641]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-04-27T04:14:09.646 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:14:09 smithi162 ceph-mon[110641]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-04-27T04:14:09.646 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:14:09 smithi162 ceph-mon[110641]: rocksdb: Options.merge_operator: 2024-04-27T04:14:09.646 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:14:09 smithi162 ceph-mon[110641]: rocksdb: Options.compaction_filter: None 2024-04-27T04:14:09.646 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:14:09 smithi162 ceph-mon[110641]: rocksdb: Options.compaction_filter_factory: None 2024-04-27T04:14:09.646 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:14:09 smithi162 ceph-mon[110641]: rocksdb: Options.sst_partitioner_factory: None 2024-04-27T04:14:09.646 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:14:09 smithi162 ceph-mon[110641]: rocksdb: Options.memtable_factory: SkipListFactory 2024-04-27T04:14:09.646 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:14:09 smithi162 ceph-mon[110641]: rocksdb: Options.table_factory: BlockBasedTable 2024-04-27T04:14:09.646 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:14:09 smithi162 ceph-mon[110641]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x56301dd19e20) 2024-04-27T04:14:09.646 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout: cache_index_and_filter_blocks: 1 2024-04-27T04:14:09.646 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-04-27T04:14:09.646 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-04-27T04:14:09.647 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout: pin_top_level_index_and_filter: 1 2024-04-27T04:14:09.647 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout: index_type: 0 2024-04-27T04:14:09.647 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout: data_block_index_type: 0 2024-04-27T04:14:09.647 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout: index_shortening: 1 2024-04-27T04:14:09.647 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout: data_block_hash_table_util_ratio: 0.750000 2024-04-27T04:14:09.647 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout: checksum: 4 2024-04-27T04:14:09.647 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout: no_block_cache: 0 2024-04-27T04:14:09.647 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout: block_cache: 0x56301dd1d1f0 2024-04-27T04:14:09.647 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout: block_cache_name: BinnedLRUCache 2024-04-27T04:14:09.647 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout: block_cache_options: 2024-04-27T04:14:09.647 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout: capacity : 536870912 2024-04-27T04:14:09.647 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout: num_shard_bits : 4 2024-04-27T04:14:09.647 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout: strict_capacity_limit : 0 2024-04-27T04:14:09.648 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout: high_pri_pool_ratio: 0.000 2024-04-27T04:14:09.648 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout: block_cache_compressed: (nil) 2024-04-27T04:14:09.648 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout: persistent_cache: (nil) 2024-04-27T04:14:09.648 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout: block_size: 4096 2024-04-27T04:14:09.648 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout: block_size_deviation: 10 2024-04-27T04:14:09.648 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout: block_restart_interval: 16 2024-04-27T04:14:09.648 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout: index_block_restart_interval: 1 2024-04-27T04:14:09.648 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout: metadata_block_size: 4096 2024-04-27T04:14:09.648 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout: partition_filters: 0 2024-04-27T04:14:09.648 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout: use_delta_encoding: 1 2024-04-27T04:14:09.648 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout: filter_policy: bloomfilter 2024-04-27T04:14:09.648 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout: whole_key_filtering: 1 2024-04-27T04:14:09.648 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout: verify_compression: 0 2024-04-27T04:14:09.648 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout: read_amp_bytes_per_bit: 0 2024-04-27T04:14:09.648 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout: format_version: 5 2024-04-27T04:14:09.649 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout: enable_index_compression: 1 2024-04-27T04:14:09.649 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout: block_align: 0 2024-04-27T04:14:09.649 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout: max_auto_readahead_size: 262144 2024-04-27T04:14:09.649 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout: prepopulate_block_cache: 0 2024-04-27T04:14:09.649 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout: initial_auto_readahead_size: 8192 2024-04-27T04:14:09.649 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout: num_file_reads_for_auto_readahead: 2 2024-04-27T04:14:09.649 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:14:09 smithi162 ceph-mon[110641]: rocksdb: Options.write_buffer_size: 33554432 2024-04-27T04:14:09.649 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:14:09 smithi162 ceph-mon[110641]: rocksdb: Options.max_write_buffer_number: 2 2024-04-27T04:14:09.649 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:14:09 smithi162 ceph-mon[110641]: rocksdb: Options.compression: NoCompression 2024-04-27T04:14:09.649 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:14:09 smithi162 ceph-mon[110641]: rocksdb: Options.bottommost_compression: Disabled 2024-04-27T04:14:09.649 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:14:09 smithi162 ceph-mon[110641]: rocksdb: Options.prefix_extractor: nullptr 2024-04-27T04:14:09.649 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:14:09 smithi162 ceph-mon[110641]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-04-27T04:14:09.649 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:14:09 smithi162 ceph-mon[110641]: rocksdb: Options.num_levels: 7 2024-04-27T04:14:09.649 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:14:09 smithi162 ceph-mon[110641]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-04-27T04:14:09.649 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:14:09 smithi162 ceph-mon[110641]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-04-27T04:14:09.649 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:14:09 smithi162 ceph-mon[110641]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-04-27T04:14:09.650 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:14:09 smithi162 ceph-mon[110641]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-04-27T04:14:09.650 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:14:09 smithi162 ceph-mon[110641]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-04-27T04:14:09.650 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:14:09 smithi162 ceph-mon[110641]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-04-27T04:14:09.650 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:14:09 smithi162 ceph-mon[110641]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-04-27T04:14:09.650 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:14:09 smithi162 ceph-mon[110641]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-04-27T04:14:09.650 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:14:09 smithi162 ceph-mon[110641]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-04-27T04:14:09.650 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:14:09 smithi162 ceph-mon[110641]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-04-27T04:14:09.650 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:14:09 smithi162 ceph-mon[110641]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-04-27T04:14:09.650 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:14:09 smithi162 ceph-mon[110641]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-04-27T04:14:09.650 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:14:09 smithi162 ceph-mon[110641]: rocksdb: Options.compression_opts.window_bits: -14 2024-04-27T04:14:09.650 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:14:09 smithi162 ceph-mon[110641]: rocksdb: Options.compression_opts.level: 32767 2024-04-27T04:14:09.650 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:14:09 smithi162 ceph-mon[110641]: rocksdb: Options.compression_opts.strategy: 0 2024-04-27T04:14:09.650 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:14:09 smithi162 ceph-mon[110641]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-04-27T04:14:09.651 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:14:09 smithi162 ceph-mon[110641]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-04-27T04:14:09.651 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:14:09 smithi162 ceph-mon[110641]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-04-27T04:14:09.651 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:14:09 smithi162 ceph-mon[110641]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-04-27T04:14:09.651 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:14:09 smithi162 ceph-mon[110641]: rocksdb: Options.compression_opts.enabled: false 2024-04-27T04:14:09.651 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:14:09 smithi162 ceph-mon[110641]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-04-27T04:14:09.651 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:14:09 smithi162 ceph-mon[110641]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-04-27T04:14:09.651 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:14:09 smithi162 ceph-mon[110641]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-04-27T04:14:09.651 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:14:09 smithi162 ceph-mon[110641]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-04-27T04:14:09.651 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:14:09 smithi162 ceph-mon[110641]: rocksdb: Options.target_file_size_base: 67108864 2024-04-27T04:14:09.651 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:14:09 smithi162 ceph-mon[110641]: rocksdb: Options.target_file_size_multiplier: 1 2024-04-27T04:14:09.651 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:14:09 smithi162 ceph-mon[110641]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-04-27T04:14:09.651 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:14:09 smithi162 ceph-mon[110641]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-04-27T04:14:09.651 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:14:09 smithi162 ceph-mon[110641]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-04-27T04:14:09.651 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:14:09 smithi162 ceph-mon[110641]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-04-27T04:14:09.651 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:14:09 smithi162 ceph-mon[110641]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-04-27T04:14:09.651 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:14:09 smithi162 ceph-mon[110641]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-04-27T04:14:09.652 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:14:09 smithi162 ceph-mon[110641]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-04-27T04:14:09.652 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:14:09 smithi162 ceph-mon[110641]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-04-27T04:14:09.652 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:14:09 smithi162 ceph-mon[110641]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-04-27T04:14:09.652 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:14:09 smithi162 ceph-mon[110641]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-04-27T04:14:09.652 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:14:09 smithi162 ceph-mon[110641]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-04-27T04:14:09.652 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:14:09 smithi162 ceph-mon[110641]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-04-27T04:14:09.652 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:14:09 smithi162 ceph-mon[110641]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-04-27T04:14:09.652 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:14:09 smithi162 ceph-mon[110641]: rocksdb: Options.arena_block_size: 1048576 2024-04-27T04:14:09.652 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:14:09 smithi162 ceph-mon[110641]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-04-27T04:14:09.652 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:14:09 smithi162 ceph-mon[110641]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-04-27T04:14:09.652 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:14:09 smithi162 ceph-mon[110641]: rocksdb: Options.disable_auto_compactions: 0 2024-04-27T04:14:09.652 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:14:09 smithi162 ceph-mon[110641]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-04-27T04:14:09.652 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:14:09 smithi162 ceph-mon[110641]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-04-27T04:14:09.652 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:14:09 smithi162 ceph-mon[110641]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-04-27T04:14:09.653 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:14:09 smithi162 ceph-mon[110641]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-04-27T04:14:09.653 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:14:09 smithi162 ceph-mon[110641]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-04-27T04:14:09.653 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:14:09 smithi162 ceph-mon[110641]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-04-27T04:14:09.653 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:14:09 smithi162 ceph-mon[110641]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-04-27T04:14:09.653 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:14:09 smithi162 ceph-mon[110641]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-04-27T04:14:09.653 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:14:09 smithi162 ceph-mon[110641]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-04-27T04:14:09.653 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:14:09 smithi162 ceph-mon[110641]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-04-27T04:14:09.653 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:14:09 smithi162 ceph-mon[110641]: rocksdb: Options.table_properties_collectors: CompactOnDeletionCollector (Sliding window size = 32768 Deletion trigger = 16384 Deletion ratio = 0); 2024-04-27T04:14:09.653 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:14:09 smithi162 ceph-mon[110641]: rocksdb: Options.inplace_update_support: 0 2024-04-27T04:14:09.653 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:14:09 smithi162 ceph-mon[110641]: rocksdb: Options.inplace_update_num_locks: 10000 2024-04-27T04:14:09.653 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:14:09 smithi162 ceph-mon[110641]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-04-27T04:14:09.653 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:14:09 smithi162 ceph-mon[110641]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-04-27T04:14:09.653 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:14:09 smithi162 ceph-mon[110641]: rocksdb: Options.memtable_huge_page_size: 0 2024-04-27T04:14:09.653 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:14:09 smithi162 ceph-mon[110641]: rocksdb: Options.bloom_locality: 0 2024-04-27T04:14:09.654 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:14:09 smithi162 ceph-mon[110641]: rocksdb: Options.max_successive_merges: 0 2024-04-27T04:14:09.654 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:14:09 smithi162 ceph-mon[110641]: rocksdb: Options.optimize_filters_for_hits: 0 2024-04-27T04:14:09.654 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:14:09 smithi162 ceph-mon[110641]: rocksdb: Options.paranoid_file_checks: 0 2024-04-27T04:14:09.654 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:14:09 smithi162 ceph-mon[110641]: rocksdb: Options.force_consistency_checks: 1 2024-04-27T04:14:09.654 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:14:09 smithi162 ceph-mon[110641]: rocksdb: Options.report_bg_io_stats: 0 2024-04-27T04:14:09.654 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:14:09 smithi162 ceph-mon[110641]: rocksdb: Options.ttl: 2592000 2024-04-27T04:14:09.654 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:14:09 smithi162 ceph-mon[110641]: rocksdb: Options.periodic_compaction_seconds: 0 2024-04-27T04:14:09.654 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:14:09 smithi162 ceph-mon[110641]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-04-27T04:14:09.654 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:14:09 smithi162 ceph-mon[110641]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-04-27T04:14:09.654 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:14:09 smithi162 ceph-mon[110641]: rocksdb: Options.enable_blob_files: false 2024-04-27T04:14:09.654 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:14:09 smithi162 ceph-mon[110641]: rocksdb: Options.min_blob_size: 0 2024-04-27T04:14:09.654 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:14:09 smithi162 ceph-mon[110641]: rocksdb: Options.blob_file_size: 268435456 2024-04-27T04:14:09.654 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:14:09 smithi162 ceph-mon[110641]: rocksdb: Options.blob_compression_type: NoCompression 2024-04-27T04:14:09.654 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:14:09 smithi162 ceph-mon[110641]: rocksdb: Options.enable_blob_garbage_collection: false 2024-04-27T04:14:09.654 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:14:09 smithi162 ceph-mon[110641]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-04-27T04:14:09.655 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:14:09 smithi162 ceph-mon[110641]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-04-27T04:14:09.655 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:14:09 smithi162 ceph-mon[110641]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-04-27T04:14:09.655 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:14:09 smithi162 ceph-mon[110641]: rocksdb: Options.blob_file_starting_level: 0 2024-04-27T04:14:09.655 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:14:09 smithi162 ceph-mon[110641]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-04-27T04:14:09.655 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:14:09 smithi162 ceph-mon[110641]: rocksdb: [table/block_based/block_based_table_reader.cc:721] At least one SST file opened without unique ID to verify: 33.sst 2024-04-27T04:14:09.655 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:14:09 smithi162 ceph-mon[110641]: rocksdb: [db/version_set.cc:4390] More existing levels in DB than needed. max_bytes_for_level_multiplier may not be guaranteed. 2024-04-27T04:14:09.655 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:14:09 smithi162 ceph-mon[110641]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-smithi162/store.db/MANIFEST-000015 succeeded,manifest_file_number is 15, next_file_number is 35, last_sequence is 11983, log_number is 31,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 0 2024-04-27T04:14:09.655 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:14:09 smithi162 ceph-mon[110641]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 31 2024-04-27T04:14:09.655 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:14:09 smithi162 ceph-mon[110641]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: e34ebe7d-df06-467d-81b8-43c81d228e86 2024-04-27T04:14:09.655 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:14:09 smithi162 ceph-mon[110641]: rocksdb: EVENT_LOG_v1 {"time_micros": 1714191249254884, "job": 1, "event": "recovery_started", "wal_files": [31]} 2024-04-27T04:14:09.655 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:14:09 smithi162 ceph-mon[110641]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #31 mode 2 2024-04-27T04:14:09.655 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:14:09 smithi162 ceph-mon[110641]: rocksdb: EVENT_LOG_v1 {"time_micros": 1714191249263026, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 36, "file_size": 1617092, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 11984, "largest_seqno": 13475, "table_properties": {"data_size": 1610234, "index_size": 4088, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 1733, "raw_key_size": 16464, "raw_average_key_size": 24, "raw_value_size": 1595883, "raw_average_value_size": 2346, "num_data_blocks": 186, "num_entries": 680, "num_filter_entries": 680, "num_deletions": 7, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[CompactOnDeletionCollector]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1714191249, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "e34ebe7d-df06-467d-81b8-43c81d228e86", "db_session_id": "3YPTOS294AN7DCM78MUO", "orig_file_number": 36, "seqno_to_time_mapping": "N/A"}} 2024-04-27T04:14:09.655 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:14:09 smithi162 ceph-mon[110641]: rocksdb: EVENT_LOG_v1 {"time_micros": 1714191249263107, "job": 1, "event": "recovery_finished"} 2024-04-27T04:14:09.655 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:14:09 smithi162 ceph-mon[110641]: rocksdb: [db/version_set.cc:5047] Creating manifest 38 2024-04-27T04:14:09.656 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:14:09 smithi162 ceph-mon[110641]: rocksdb: [db/version_set.cc:4390] More existing levels in DB than needed. max_bytes_for_level_multiplier may not be guaranteed. 2024-04-27T04:14:09.656 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:14:09 smithi162 ceph-mon[110641]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-smithi162/store.db/000031.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-04-27T04:14:09.656 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:14:09 smithi162 ceph-mon[110641]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x56301de3a000 2024-04-27T04:14:09.656 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:14:09 smithi162 ceph-mon[110641]: rocksdb: DB pointer 0x56301de24000 2024-04-27T04:14:09.656 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:14:09 smithi162 ceph-mon[110641]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-04-27T04:14:09.656 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:14:09 smithi162 ceph-mon[110641]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-04-27T04:14:09.656 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout: ** DB Stats ** 2024-04-27T04:14:09.656 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-04-27T04:14:09.656 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-04-27T04:14:09.656 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-04-27T04:14:09.656 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-04-27T04:14:09.656 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-04-27T04:14:09.656 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-04-27T04:14:09.656 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-04-27T04:14:09.656 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout: 2024-04-27T04:14:09.657 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout: ** Compaction Stats [default] ** 2024-04-27T04:14:09.657 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-04-27T04:14:09.657 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-04-27T04:14:09.657 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout: L0 1/0 1.54 MB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 458.2 0.00 0.00 1 0.003 0 0 0.0 0.0 2024-04-27T04:14:09.657 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout: L6 1/0 6.72 MB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 2024-04-27T04:14:09.657 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout: Sum 2/0 8.26 MB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 458.2 0.00 0.00 1 0.003 0 0 0.0 0.0 2024-04-27T04:14:09.657 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 458.2 0.00 0.00 1 0.003 0 0 0.0 0.0 2024-04-27T04:14:09.657 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout: 2024-04-27T04:14:09.657 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout: ** Compaction Stats [default] ** 2024-04-27T04:14:09.657 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-04-27T04:14:09.657 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-04-27T04:14:09.657 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 458.2 0.00 0.00 1 0.003 0 0 0.0 0.0 2024-04-27T04:14:09.657 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout: 2024-04-27T04:14:09.657 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-04-27T04:14:09.657 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout: 2024-04-27T04:14:09.658 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-04-27T04:14:09.658 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout: Flush(GB): cumulative 0.002, interval 0.002 2024-04-27T04:14:09.658 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-04-27T04:14:09.658 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-04-27T04:14:09.658 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-04-27T04:14:09.658 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout: AddFile(Keys): cumulative 0, interval 0 2024-04-27T04:14:09.658 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout: Cumulative compaction: 0.00 GB write, 121.89 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-04-27T04:14:09.658 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout: Interval compaction: 0.00 GB write, 121.89 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-04-27T04:14:09.658 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-04-27T04:14:09.658 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout: Block cache BinnedLRUCache@0x56301dd1d1f0#2 capacity: 512.00 MB usage: 5.86 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 9e-06 secs_since: 0 2024-04-27T04:14:09.658 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout: Block cache entry stats(count,size,portion): FilterBlock(1,1.78 KB,0.000339746%) IndexBlock(1,4.08 KB,0.000777841%) Misc(1,0.00 KB,0%) 2024-04-27T04:14:09.658 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout: 2024-04-27T04:14:09.658 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout: ** File Read Latency Histogram By Level [default] ** 2024-04-27T04:14:09.658 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:14:09 smithi162 ceph-mon[110641]: starting mon.smithi162 rank 0 at public addrs [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] at bind addrs [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon_data /var/lib/ceph/mon/ceph-smithi162 fsid 7f9750d6-044a-11ef-bc93-c7b262605968 2024-04-27T04:14:09.658 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:14:09 smithi162 ceph-mon[110641]: mon.smithi162@-1(???) e2 preinit fsid 7f9750d6-044a-11ef-bc93-c7b262605968 2024-04-27T04:14:09.659 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:14:09 smithi162 ceph-mon[110641]: mon.smithi162@-1(???).mds e15 new map 2024-04-27T04:14:09.659 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:14:09 smithi162 ceph-mon[110641]: mon.smithi162@-1(???).mds e15 print_map 2024-04-27T04:14:09.659 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout: e15 2024-04-27T04:14:09.659 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-04-27T04:14:09.659 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-27T04:14:09.659 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout: legacy client fscid: 1 2024-04-27T04:14:09.659 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout: 2024-04-27T04:14:09.659 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout: Filesystem 'cephfs' (1) 2024-04-27T04:14:09.659 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout: fs_name cephfs 2024-04-27T04:14:09.659 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout: epoch 15 2024-04-27T04:14:09.659 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout: flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-04-27T04:14:09.659 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout: created 2024-04-27T04:06:53.095749+0000 2024-04-27T04:14:09.659 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout: modified 2024-04-27T04:07:11.285033+0000 2024-04-27T04:14:09.659 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout: tableserver 0 2024-04-27T04:14:09.659 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout: root 0 2024-04-27T04:14:09.660 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout: session_timeout 60 2024-04-27T04:14:09.660 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout: session_autoclose 300 2024-04-27T04:14:09.660 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout: max_file_size 1099511627776 2024-04-27T04:14:09.660 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout: max_xattr_size 65536 2024-04-27T04:14:09.660 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout: required_client_features {} 2024-04-27T04:14:09.660 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout: last_failure 0 2024-04-27T04:14:09.660 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout: last_failure_osd_epoch 46 2024-04-27T04:14:09.660 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout: compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-27T04:14:09.660 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout: max_mds 1 2024-04-27T04:14:09.660 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout: in 0 2024-04-27T04:14:09.660 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout: up {0=24273} 2024-04-27T04:14:09.660 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout: failed 2024-04-27T04:14:09.660 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout: damaged 2024-04-27T04:14:09.660 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout: stopped 2024-04-27T04:14:09.661 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout: data_pools [3] 2024-04-27T04:14:09.661 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout: metadata_pool 2 2024-04-27T04:14:09.661 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout: inline_data disabled 2024-04-27T04:14:09.661 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout: balancer 2024-04-27T04:14:09.661 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout: bal_rank_mask -1 2024-04-27T04:14:09.661 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout: standby_count_wanted 1 2024-04-27T04:14:09.661 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout: qdb_cluster leader: 0 members: 2024-04-27T04:14:09.661 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout: [mds.cephfs.smithi181.dzomgh{0:24273} state up:active seq 6 join_fscid=1 addr [v2:172.21.15.181:6824/658214993,v1:172.21.15.181:6825/658214993] compat {c=[1],r=[1],i=[7ff]}] 2024-04-27T04:14:09.661 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout: [mds.cephfs.smithi162.xcnapk{0:14476} state up:standby-replay seq 2 join_fscid=1 addr [v2:172.21.15.162:6828/1296160676,v1:172.21.15.162:6829/1296160676] compat {c=[1],r=[1],i=[7ff]}] 2024-04-27T04:14:09.661 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout: 2024-04-27T04:14:09.661 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout: 2024-04-27T04:14:09.662 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout: Standby daemons: 2024-04-27T04:14:09.662 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout: 2024-04-27T04:14:09.662 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout: [mds.cephfs.smithi162.gzlyjj{-1:14488} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.162:6826/2629103894,v1:172.21.15.162:6827/2629103894] compat {c=[1],r=[1],i=[7ff]}] 2024-04-27T04:14:09.662 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout: [mds.cephfs.smithi181.bkkcdm{-1:24287} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.181:6826/3756475453,v1:172.21.15.181:6827/3756475453] compat {c=[1],r=[1],i=[7ff]}] 2024-04-27T04:14:09.662 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:14:09 smithi162 ceph-mon[110641]: mon.smithi162@-1(???).osd e48 crush map has features 3314933000852226048, adjusting msgr requires 2024-04-27T04:14:09.662 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:14:09 smithi162 ceph-mon[110641]: mon.smithi162@-1(???).osd e48 crush map has features 288514051259236352, adjusting msgr requires 2024-04-27T04:14:09.662 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:14:09 smithi162 ceph-mon[110641]: mon.smithi162@-1(???).osd e48 crush map has features 288514051259236352, adjusting msgr requires 2024-04-27T04:14:09.662 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:14:09 smithi162 ceph-mon[110641]: mon.smithi162@-1(???).osd e48 crush map has features 288514051259236352, adjusting msgr requires 2024-04-27T04:14:09.662 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:14:09 smithi162 ceph-mon[110641]: mon.smithi162@-1(???).paxosservice(auth 1..21) refresh upgraded, format 0 -> 3 2024-04-27T04:14:09.662 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:14:09 smithi162 bash[110622]: 375adaf4468f7520330d3d23debb783bfbda1c09ddf0808a70b1c616606c5800 2024-04-27T04:14:09.662 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:14:09 smithi162 systemd[1]: Started Ceph mon.smithi162 for 7f9750d6-044a-11ef-bc93-c7b262605968. 2024-04-27T04:14:11.056 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:14:10 smithi181 ceph-mon[33774]: from='mgr.14680 172.21.15.162:0/501880828' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "mon metadata", "id": "smithi162"}]: dispatch 2024-04-27T04:14:11.056 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:14:10 smithi181 ceph-mon[33774]: from='mgr.14680 172.21.15.162:0/501880828' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "mon metadata", "id": "smithi181"}]: dispatch 2024-04-27T04:14:11.056 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:14:10 smithi181 ceph-mon[33774]: pgmap v81: 65 pgs: 65 active+clean; 208 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-04-27T04:14:11.056 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:14:10 smithi181 ceph-mon[33774]: mon.smithi162 calling monitor election 2024-04-27T04:14:11.056 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:14:10 smithi181 ceph-mon[33774]: mon.smithi162 is new leader, mons smithi162,smithi181 in quorum (ranks 0,1) 2024-04-27T04:14:11.056 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:14:10 smithi181 ceph-mon[33774]: monmap epoch 2 2024-04-27T04:14:11.056 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:14:10 smithi181 ceph-mon[33774]: fsid 7f9750d6-044a-11ef-bc93-c7b262605968 2024-04-27T04:14:11.056 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:14:10 smithi181 ceph-mon[33774]: last_changed 2024-04-27T04:02:40.633767+0000 2024-04-27T04:14:11.056 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:14:10 smithi181 ceph-mon[33774]: created 2024-04-27T04:00:03.626312+0000 2024-04-27T04:14:11.056 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:14:10 smithi181 ceph-mon[33774]: min_mon_release 17 (quincy) 2024-04-27T04:14:11.057 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:14:10 smithi181 ceph-mon[33774]: election_strategy: 1 2024-04-27T04:14:11.057 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:14:10 smithi181 ceph-mon[33774]: 0: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.smithi162 2024-04-27T04:14:11.057 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:14:10 smithi181 ceph-mon[33774]: 1: [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] mon.smithi181 2024-04-27T04:14:11.057 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:14:10 smithi181 ceph-mon[33774]: fsmap cephfs:1 {0=cephfs.smithi181.dzomgh=up:active} 1 up:standby-replay 2 up:standby 2024-04-27T04:14:11.057 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:14:10 smithi181 ceph-mon[33774]: osdmap e48: 6 total, 6 up, 6 in 2024-04-27T04:14:11.057 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:14:10 smithi181 ceph-mon[33774]: mgrmap e32: smithi162.oennfn(active, since 2m), standbys: smithi181.vtwuhz 2024-04-27T04:14:11.057 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:14:10 smithi181 ceph-mon[33774]: overall HEALTH_OK 2024-04-27T04:14:11.057 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:14:10 smithi181 ceph-mon[33774]: from='mgr.14680 ' entity='' 2024-04-27T04:14:11.057 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:14:10 smithi181 ceph-mon[33774]: mgrmap e33: smithi162.oennfn(active, since 2m), standbys: smithi181.vtwuhz 2024-04-27T04:14:11.060 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:14:10 smithi162 ceph-mon[110641]: from='mgr.14680 172.21.15.162:0/501880828' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "mon metadata", "id": "smithi162"}]: dispatch 2024-04-27T04:14:11.060 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:14:10 smithi162 ceph-mon[110641]: from='mgr.14680 172.21.15.162:0/501880828' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "mon metadata", "id": "smithi181"}]: dispatch 2024-04-27T04:14:11.060 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:14:10 smithi162 ceph-mon[110641]: pgmap v81: 65 pgs: 65 active+clean; 208 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-04-27T04:14:11.060 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:14:10 smithi162 ceph-mon[110641]: mon.smithi162 calling monitor election 2024-04-27T04:14:11.060 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:14:10 smithi162 ceph-mon[110641]: mon.smithi162 is new leader, mons smithi162,smithi181 in quorum (ranks 0,1) 2024-04-27T04:14:11.060 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:14:10 smithi162 ceph-mon[110641]: monmap epoch 2 2024-04-27T04:14:11.060 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:14:10 smithi162 ceph-mon[110641]: fsid 7f9750d6-044a-11ef-bc93-c7b262605968 2024-04-27T04:14:11.061 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:14:10 smithi162 ceph-mon[110641]: last_changed 2024-04-27T04:02:40.633767+0000 2024-04-27T04:14:11.061 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:14:10 smithi162 ceph-mon[110641]: created 2024-04-27T04:00:03.626312+0000 2024-04-27T04:14:11.061 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:14:10 smithi162 ceph-mon[110641]: min_mon_release 17 (quincy) 2024-04-27T04:14:11.061 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:14:10 smithi162 ceph-mon[110641]: election_strategy: 1 2024-04-27T04:14:11.061 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:14:10 smithi162 ceph-mon[110641]: 0: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.smithi162 2024-04-27T04:14:11.061 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:14:10 smithi162 ceph-mon[110641]: 1: [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] mon.smithi181 2024-04-27T04:14:11.061 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:14:10 smithi162 ceph-mon[110641]: fsmap cephfs:1 {0=cephfs.smithi181.dzomgh=up:active} 1 up:standby-replay 2 up:standby 2024-04-27T04:14:11.061 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:14:10 smithi162 ceph-mon[110641]: osdmap e48: 6 total, 6 up, 6 in 2024-04-27T04:14:11.061 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:14:10 smithi162 ceph-mon[110641]: mgrmap e32: smithi162.oennfn(active, since 2m), standbys: smithi181.vtwuhz 2024-04-27T04:14:11.061 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:14:10 smithi162 ceph-mon[110641]: overall HEALTH_OK 2024-04-27T04:14:11.061 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:14:10 smithi162 ceph-mon[110641]: from='mgr.14680 ' entity='' 2024-04-27T04:14:11.061 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:14:10 smithi162 ceph-mon[110641]: mgrmap e33: smithi162.oennfn(active, since 2m), standbys: smithi181.vtwuhz 2024-04-27T04:14:17.637 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:14:17 smithi162 ceph-mon[110641]: Standby manager daemon smithi181.vtwuhz restarted 2024-04-27T04:14:17.637 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:14:17 smithi162 ceph-mon[110641]: Standby manager daemon smithi181.vtwuhz started 2024-04-27T04:14:17.637 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:14:17 smithi162 ceph-mon[110641]: from='mgr.? 172.21.15.181:0/2870590046' entity='mgr.smithi181.vtwuhz' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi181.vtwuhz/crt"}]: dispatch 2024-04-27T04:14:17.637 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:14:17 smithi162 ceph-mon[110641]: from='mgr.? 172.21.15.181:0/2870590046' entity='mgr.smithi181.vtwuhz' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-27T04:14:17.637 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:14:17 smithi162 ceph-mon[110641]: from='mgr.? 172.21.15.181:0/2870590046' entity='mgr.smithi181.vtwuhz' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi181.vtwuhz/key"}]: dispatch 2024-04-27T04:14:17.637 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:14:17 smithi162 ceph-mon[110641]: from='mgr.? 172.21.15.181:0/2870590046' entity='mgr.smithi181.vtwuhz' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-27T04:14:17.922 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:14:17 smithi181 ceph-mon[33774]: Standby manager daemon smithi181.vtwuhz restarted 2024-04-27T04:14:17.922 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:14:17 smithi181 ceph-mon[33774]: Standby manager daemon smithi181.vtwuhz started 2024-04-27T04:14:17.923 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:14:17 smithi181 ceph-mon[33774]: from='mgr.? 172.21.15.181:0/2870590046' entity='mgr.smithi181.vtwuhz' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi181.vtwuhz/crt"}]: dispatch 2024-04-27T04:14:17.923 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:14:17 smithi181 ceph-mon[33774]: from='mgr.? 172.21.15.181:0/2870590046' entity='mgr.smithi181.vtwuhz' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-27T04:14:17.923 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:14:17 smithi181 ceph-mon[33774]: from='mgr.? 172.21.15.181:0/2870590046' entity='mgr.smithi181.vtwuhz' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi181.vtwuhz/key"}]: dispatch 2024-04-27T04:14:17.923 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:14:17 smithi181 ceph-mon[33774]: from='mgr.? 172.21.15.181:0/2870590046' entity='mgr.smithi181.vtwuhz' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-27T04:14:18.887 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:14:18 smithi162 ceph-mon[110641]: mgrmap e34: smithi162.oennfn(active, since 2m), standbys: smithi181.vtwuhz 2024-04-27T04:14:18.887 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:14:18 smithi162 ceph-mon[110641]: Active manager daemon smithi162.oennfn restarted 2024-04-27T04:14:18.887 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:14:18 smithi162 ceph-mon[110641]: Activating manager daemon smithi162.oennfn 2024-04-27T04:14:18.887 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:14:18 smithi162 ceph-mon[110641]: osdmap e49: 6 total, 6 up, 6 in 2024-04-27T04:14:18.887 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:14:18 smithi162 ceph-mon[110641]: mgrmap e35: smithi162.oennfn(active, starting, since 0.00600496s), standbys: smithi181.vtwuhz 2024-04-27T04:14:18.887 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:14:18 smithi162 ceph-mon[110641]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "mon metadata", "id": "smithi162"}]: dispatch 2024-04-27T04:14:18.887 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:14:18 smithi162 ceph-mon[110641]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "mon metadata", "id": "smithi181"}]: dispatch 2024-04-27T04:14:18.887 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:14:18 smithi162 ceph-mon[110641]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi162.xcnapk"}]: dispatch 2024-04-27T04:14:18.887 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:14:18 smithi162 ceph-mon[110641]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi162.gzlyjj"}]: dispatch 2024-04-27T04:14:18.888 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:14:18 smithi162 ceph-mon[110641]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi181.dzomgh"}]: dispatch 2024-04-27T04:14:18.888 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:14:18 smithi162 ceph-mon[110641]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi181.bkkcdm"}]: dispatch 2024-04-27T04:14:18.888 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:14:18 smithi162 ceph-mon[110641]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "mgr metadata", "who": "smithi162.oennfn", "id": "smithi162.oennfn"}]: dispatch 2024-04-27T04:14:18.888 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:14:18 smithi162 ceph-mon[110641]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "mgr metadata", "who": "smithi181.vtwuhz", "id": "smithi181.vtwuhz"}]: dispatch 2024-04-27T04:14:18.888 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:14:18 smithi162 ceph-mon[110641]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-27T04:14:18.888 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:14:18 smithi162 ceph-mon[110641]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-27T04:14:18.888 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:14:18 smithi162 ceph-mon[110641]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-27T04:14:18.888 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:14:18 smithi162 ceph-mon[110641]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-27T04:14:18.888 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:14:18 smithi162 ceph-mon[110641]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-27T04:14:18.888 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:14:18 smithi162 ceph-mon[110641]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-27T04:14:18.888 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:14:18 smithi162 ceph-mon[110641]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-04-27T04:14:18.888 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:14:18 smithi162 ceph-mon[110641]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-04-27T04:14:18.888 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:14:18 smithi162 ceph-mon[110641]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-04-27T04:14:18.888 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:14:18 smithi162 ceph-mon[110641]: Manager daemon smithi162.oennfn is now available 2024-04-27T04:14:18.889 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:14:18 smithi162 ceph-mon[110641]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T04:14:18.889 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:14:18 smithi162 ceph-mon[110641]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-27T04:14:18.889 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:14:18 smithi162 ceph-mon[110641]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi162.oennfn/mirror_snapshot_schedule"}]: dispatch 2024-04-27T04:14:18.889 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:14:18 smithi162 ceph-mon[110641]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi162.oennfn/trash_purge_schedule"}]: dispatch 2024-04-27T04:14:18.907 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:14:18 smithi181 ceph-mon[33774]: mgrmap e34: smithi162.oennfn(active, since 2m), standbys: smithi181.vtwuhz 2024-04-27T04:14:18.907 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:14:18 smithi181 ceph-mon[33774]: Active manager daemon smithi162.oennfn restarted 2024-04-27T04:14:18.907 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:14:18 smithi181 ceph-mon[33774]: Activating manager daemon smithi162.oennfn 2024-04-27T04:14:18.907 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:14:18 smithi181 ceph-mon[33774]: osdmap e49: 6 total, 6 up, 6 in 2024-04-27T04:14:18.907 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:14:18 smithi181 ceph-mon[33774]: mgrmap e35: smithi162.oennfn(active, starting, since 0.00600496s), standbys: smithi181.vtwuhz 2024-04-27T04:14:18.908 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:14:18 smithi181 ceph-mon[33774]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "mon metadata", "id": "smithi162"}]: dispatch 2024-04-27T04:14:18.908 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:14:18 smithi181 ceph-mon[33774]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "mon metadata", "id": "smithi181"}]: dispatch 2024-04-27T04:14:18.908 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:14:18 smithi181 ceph-mon[33774]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi162.xcnapk"}]: dispatch 2024-04-27T04:14:18.908 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:14:18 smithi181 ceph-mon[33774]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi162.gzlyjj"}]: dispatch 2024-04-27T04:14:18.908 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:14:18 smithi181 ceph-mon[33774]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi181.dzomgh"}]: dispatch 2024-04-27T04:14:18.908 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:14:18 smithi181 ceph-mon[33774]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi181.bkkcdm"}]: dispatch 2024-04-27T04:14:18.909 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:14:18 smithi181 ceph-mon[33774]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "mgr metadata", "who": "smithi162.oennfn", "id": "smithi162.oennfn"}]: dispatch 2024-04-27T04:14:18.909 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:14:18 smithi181 ceph-mon[33774]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "mgr metadata", "who": "smithi181.vtwuhz", "id": "smithi181.vtwuhz"}]: dispatch 2024-04-27T04:14:18.909 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:14:18 smithi181 ceph-mon[33774]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-27T04:14:18.909 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:14:18 smithi181 ceph-mon[33774]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-27T04:14:18.909 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:14:18 smithi181 ceph-mon[33774]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-27T04:14:18.909 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:14:18 smithi181 ceph-mon[33774]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-27T04:14:18.910 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:14:18 smithi181 ceph-mon[33774]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-27T04:14:18.910 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:14:18 smithi181 ceph-mon[33774]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-27T04:14:18.910 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:14:18 smithi181 ceph-mon[33774]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-04-27T04:14:18.910 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:14:18 smithi181 ceph-mon[33774]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-04-27T04:14:18.910 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:14:18 smithi181 ceph-mon[33774]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-04-27T04:14:18.910 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:14:18 smithi181 ceph-mon[33774]: Manager daemon smithi162.oennfn is now available 2024-04-27T04:14:18.911 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:14:18 smithi181 ceph-mon[33774]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T04:14:18.911 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:14:18 smithi181 ceph-mon[33774]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-27T04:14:18.911 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:14:18 smithi181 ceph-mon[33774]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi162.oennfn/mirror_snapshot_schedule"}]: dispatch 2024-04-27T04:14:18.911 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:14:18 smithi181 ceph-mon[33774]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi162.oennfn/trash_purge_schedule"}]: dispatch 2024-04-27T04:14:19.859 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:14:19 smithi162 ceph-mon[110641]: [27/Apr/2024:04:14:18] ENGINE Bus STARTING 2024-04-27T04:14:19.859 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:14:19 smithi162 ceph-mon[110641]: [27/Apr/2024:04:14:18] ENGINE Serving on https://172.21.15.162:7150 2024-04-27T04:14:19.859 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:14:19 smithi162 ceph-mon[110641]: [27/Apr/2024:04:14:18] ENGINE Serving on http://172.21.15.162:8765 2024-04-27T04:14:19.859 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:14:19 smithi162 ceph-mon[110641]: [27/Apr/2024:04:14:18] ENGINE Bus STARTED 2024-04-27T04:14:19.859 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:14:19 smithi162 ceph-mon[110641]: mgrmap e36: smithi162.oennfn(active, since 1.0095s), standbys: smithi181.vtwuhz 2024-04-27T04:14:19.859 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:14:19 smithi162 ceph-mon[110641]: pgmap v3: 65 pgs: 65 active+clean; 208 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail 2024-04-27T04:14:19.922 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:14:19 smithi181 ceph-mon[33774]: [27/Apr/2024:04:14:18] ENGINE Bus STARTING 2024-04-27T04:14:19.922 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:14:19 smithi181 ceph-mon[33774]: [27/Apr/2024:04:14:18] ENGINE Serving on https://172.21.15.162:7150 2024-04-27T04:14:19.922 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:14:19 smithi181 ceph-mon[33774]: [27/Apr/2024:04:14:18] ENGINE Serving on http://172.21.15.162:8765 2024-04-27T04:14:19.922 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:14:19 smithi181 ceph-mon[33774]: [27/Apr/2024:04:14:18] ENGINE Bus STARTED 2024-04-27T04:14:19.922 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:14:19 smithi181 ceph-mon[33774]: mgrmap e36: smithi162.oennfn(active, since 1.0095s), standbys: smithi181.vtwuhz 2024-04-27T04:14:19.923 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:14:19 smithi181 ceph-mon[33774]: pgmap v3: 65 pgs: 65 active+clean; 208 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail 2024-04-27T04:14:20.887 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:14:20 smithi162 ceph-mon[110641]: pgmap v4: 65 pgs: 65 active+clean; 208 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail 2024-04-27T04:14:20.887 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:14:20 smithi162 ceph-mon[110641]: mgrmap e37: smithi162.oennfn(active, since 2s), standbys: smithi181.vtwuhz 2024-04-27T04:14:20.922 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:14:20 smithi181 ceph-mon[33774]: pgmap v4: 65 pgs: 65 active+clean; 208 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail 2024-04-27T04:14:20.922 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:14:20 smithi181 ceph-mon[33774]: mgrmap e37: smithi162.oennfn(active, since 2s), standbys: smithi181.vtwuhz 2024-04-27T04:14:22.010 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:14:21 smithi162 ceph-mon[110641]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' 2024-04-27T04:14:22.010 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:14:21 smithi162 ceph-mon[110641]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' 2024-04-27T04:14:22.010 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:14:21 smithi162 ceph-mon[110641]: pgmap v5: 65 pgs: 65 active+clean; 208 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail 2024-04-27T04:14:22.172 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:14:21 smithi181 ceph-mon[33774]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' 2024-04-27T04:14:22.172 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:14:21 smithi181 ceph-mon[33774]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' 2024-04-27T04:14:22.172 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:14:21 smithi181 ceph-mon[33774]: pgmap v5: 65 pgs: 65 active+clean; 208 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail 2024-04-27T04:14:23.172 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:14:22 smithi181 ceph-mon[33774]: mgrmap e38: smithi162.oennfn(active, since 4s), standbys: smithi181.vtwuhz 2024-04-27T04:14:23.172 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:14:22 smithi181 ceph-mon[33774]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' 2024-04-27T04:14:23.172 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:14:22 smithi181 ceph-mon[33774]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' 2024-04-27T04:14:23.386 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:14:22 smithi162 ceph-mon[110641]: mgrmap e38: smithi162.oennfn(active, since 4s), standbys: smithi181.vtwuhz 2024-04-27T04:14:23.386 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:14:22 smithi162 ceph-mon[110641]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' 2024-04-27T04:14:23.387 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:14:22 smithi162 ceph-mon[110641]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' 2024-04-27T04:14:24.423 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:14:24 smithi181 ceph-mon[33774]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' 2024-04-27T04:14:24.423 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:14:24 smithi181 ceph-mon[33774]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' 2024-04-27T04:14:24.423 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:14:24 smithi181 ceph-mon[33774]: pgmap v6: 65 pgs: 65 active+clean; 208 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 42 KiB/s rd, 0 B/s wr, 24 op/s 2024-04-27T04:14:24.636 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:14:24 smithi162 ceph-mon[110641]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' 2024-04-27T04:14:24.637 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:14:24 smithi162 ceph-mon[110641]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' 2024-04-27T04:14:24.637 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:14:24 smithi162 ceph-mon[110641]: pgmap v6: 65 pgs: 65 active+clean; 208 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 42 KiB/s rd, 0 B/s wr, 24 op/s 2024-04-27T04:14:25.636 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:14:25 smithi162 ceph-mon[110641]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' 2024-04-27T04:14:25.637 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:14:25 smithi162 ceph-mon[110641]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' 2024-04-27T04:14:25.672 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:14:25 smithi181 ceph-mon[33774]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' 2024-04-27T04:14:25.672 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:14:25 smithi181 ceph-mon[33774]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' 2024-04-27T04:14:26.636 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:14:26 smithi162 ceph-mon[110641]: pgmap v7: 65 pgs: 65 active+clean; 208 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 30 KiB/s rd, 0 B/s wr, 18 op/s 2024-04-27T04:14:26.672 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:14:26 smithi181 ceph-mon[33774]: pgmap v7: 65 pgs: 65 active+clean; 208 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 30 KiB/s rd, 0 B/s wr, 18 op/s 2024-04-27T04:14:29.038 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:14:28 smithi162 ceph-mon[110641]: pgmap v8: 65 pgs: 65 active+clean; 208 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 23 KiB/s rd, 0 B/s wr, 14 op/s 2024-04-27T04:14:29.039 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:14:28 smithi162 ceph-mon[110641]: Detected new or changed devices on smithi162 2024-04-27T04:14:29.039 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:14:28 smithi162 ceph-mon[110641]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' 2024-04-27T04:14:29.039 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:14:28 smithi162 ceph-mon[110641]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' 2024-04-27T04:14:29.039 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:14:28 smithi162 ceph-mon[110641]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T04:14:29.039 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:14:28 smithi162 ceph-mon[110641]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T04:14:29.039 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:14:28 smithi162 ceph-mon[110641]: Updating smithi162:/etc/ceph/ceph.conf 2024-04-27T04:14:29.039 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:14:28 smithi162 ceph-mon[110641]: Updating smithi181:/etc/ceph/ceph.conf 2024-04-27T04:14:29.039 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:14:28 smithi162 ceph-mon[110641]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' 2024-04-27T04:14:29.039 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:14:28 smithi162 ceph-mon[110641]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' 2024-04-27T04:14:29.039 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:14:28 smithi162 ceph-mon[110641]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' 2024-04-27T04:14:29.040 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:14:28 smithi162 ceph-mon[110641]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' 2024-04-27T04:14:29.040 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:14:28 smithi162 ceph-mon[110641]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' 2024-04-27T04:14:29.172 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:14:28 smithi181 ceph-mon[33774]: pgmap v8: 65 pgs: 65 active+clean; 208 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 23 KiB/s rd, 0 B/s wr, 14 op/s 2024-04-27T04:14:29.172 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:14:28 smithi181 ceph-mon[33774]: Detected new or changed devices on smithi162 2024-04-27T04:14:29.172 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:14:28 smithi181 ceph-mon[33774]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' 2024-04-27T04:14:29.172 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:14:28 smithi181 ceph-mon[33774]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' 2024-04-27T04:14:29.172 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:14:28 smithi181 ceph-mon[33774]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T04:14:29.172 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:14:28 smithi181 ceph-mon[33774]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T04:14:29.172 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:14:28 smithi181 ceph-mon[33774]: Updating smithi162:/etc/ceph/ceph.conf 2024-04-27T04:14:29.173 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:14:28 smithi181 ceph-mon[33774]: Updating smithi181:/etc/ceph/ceph.conf 2024-04-27T04:14:29.173 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:14:28 smithi181 ceph-mon[33774]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' 2024-04-27T04:14:29.173 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:14:28 smithi181 ceph-mon[33774]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' 2024-04-27T04:14:29.173 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:14:28 smithi181 ceph-mon[33774]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' 2024-04-27T04:14:29.173 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:14:28 smithi181 ceph-mon[33774]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' 2024-04-27T04:14:29.173 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:14:28 smithi181 ceph-mon[33774]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' 2024-04-27T04:14:29.922 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:14:29 smithi181 ceph-mon[33774]: Updating smithi162:/var/lib/ceph/7f9750d6-044a-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-27T04:14:29.922 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:14:29 smithi181 ceph-mon[33774]: Updating smithi181:/var/lib/ceph/7f9750d6-044a-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-27T04:14:29.923 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:14:29 smithi181 ceph-mon[33774]: Updating smithi162:/etc/ceph/ceph.client.admin.keyring 2024-04-27T04:14:29.923 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:14:29 smithi181 ceph-mon[33774]: Updating smithi181:/etc/ceph/ceph.client.admin.keyring 2024-04-27T04:14:29.923 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:14:29 smithi181 ceph-mon[33774]: Updating smithi162:/var/lib/ceph/7f9750d6-044a-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-27T04:14:29.923 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:14:29 smithi181 ceph-mon[33774]: Updating smithi181:/var/lib/ceph/7f9750d6-044a-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-27T04:14:29.923 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:14:29 smithi181 ceph-mon[33774]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T04:14:29.923 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:14:29 smithi181 ceph-mon[33774]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "versions"}]: dispatch 2024-04-27T04:14:29.923 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:14:29 smithi181 ceph-mon[33774]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "quorum_status"}]: dispatch 2024-04-27T04:14:30.050 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:14:29 smithi162 ceph-mon[110641]: Updating smithi162:/var/lib/ceph/7f9750d6-044a-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-27T04:14:30.051 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:14:29 smithi162 ceph-mon[110641]: Updating smithi181:/var/lib/ceph/7f9750d6-044a-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-27T04:14:30.051 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:14:29 smithi162 ceph-mon[110641]: Updating smithi162:/etc/ceph/ceph.client.admin.keyring 2024-04-27T04:14:30.051 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:14:29 smithi162 ceph-mon[110641]: Updating smithi181:/etc/ceph/ceph.client.admin.keyring 2024-04-27T04:14:30.051 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:14:29 smithi162 ceph-mon[110641]: Updating smithi162:/var/lib/ceph/7f9750d6-044a-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-27T04:14:30.051 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:14:29 smithi162 ceph-mon[110641]: Updating smithi181:/var/lib/ceph/7f9750d6-044a-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-27T04:14:30.051 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:14:29 smithi162 ceph-mon[110641]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T04:14:30.051 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:14:29 smithi162 ceph-mon[110641]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "versions"}]: dispatch 2024-04-27T04:14:30.051 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:14:29 smithi162 ceph-mon[110641]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "quorum_status"}]: dispatch 2024-04-27T04:14:31.059 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:14:30 smithi181 ceph-mon[33774]: pgmap v9: 65 pgs: 65 active+clean; 208 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 20 KiB/s rd, 0 B/s wr, 12 op/s 2024-04-27T04:14:31.059 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:14:30 smithi181 ceph-mon[33774]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' 2024-04-27T04:14:31.059 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:14:30 smithi181 ceph-mon[33774]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-27T04:14:31.059 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:14:30 smithi181 ceph-mon[33774]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-27T04:14:31.059 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:14:30 smithi181 ceph-mon[33774]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T04:14:31.137 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:14:30 smithi162 ceph-mon[110641]: pgmap v9: 65 pgs: 65 active+clean; 208 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 20 KiB/s rd, 0 B/s wr, 12 op/s 2024-04-27T04:14:31.137 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:14:30 smithi162 ceph-mon[110641]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' 2024-04-27T04:14:31.137 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:14:30 smithi162 ceph-mon[110641]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-27T04:14:31.137 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:14:30 smithi162 ceph-mon[110641]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-27T04:14:31.137 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:14:30 smithi162 ceph-mon[110641]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T04:14:31.922 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:14:31 smithi181 ceph-mon[33774]: Upgrade: Updating mon.smithi181 2024-04-27T04:14:31.922 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:14:31 smithi181 ceph-mon[33774]: Deploying daemon mon.smithi181 on smithi181 2024-04-27T04:14:32.137 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:14:31 smithi162 ceph-mon[110641]: Upgrade: Updating mon.smithi181 2024-04-27T04:14:32.137 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:14:31 smithi162 ceph-mon[110641]: Deploying daemon mon.smithi181 on smithi181 2024-04-27T04:14:32.506 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:14:32 smithi181 systemd[1]: Stopping Ceph mon.smithi181 for 7f9750d6-044a-11ef-bc93-c7b262605968... 2024-04-27T04:14:32.922 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:14:32 smithi181 ceph-7f9750d6-044a-11ef-bc93-c7b262605968-mon-smithi181[33756]: 2024-04-27T04:14:32.507+0000 7fb8ef6f5700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.smithi181 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-04-27T04:14:32.923 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:14:32 smithi181 ceph-7f9750d6-044a-11ef-bc93-c7b262605968-mon-smithi181[33756]: 2024-04-27T04:14:32.507+0000 7fb8ef6f5700 -1 mon.smithi181@1(peon) e2 *** Got Signal Terminated *** 2024-04-27T04:14:32.923 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:14:32 smithi181 podman[80971]: 2024-04-27 04:14:32.721608201 +0000 UTC m=+0.326211878 container died 96e48aa3912ff515ec9f6dd66d494324ee1054ead5cec21affb7c30d81421ddb (image=quay.ceph.io/ceph-ci/ceph@sha256:d31f278ece6dbd17d1fa5e598f509017f54b76fafe1c4416391fcc51b9718ee9, name=ceph-7f9750d6-044a-11ef-bc93-c7b262605968-mon-smithi181, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=quincy-2be3efa, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_BRANCH=HEAD, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.build-date=20240424, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2) 2024-04-27T04:14:33.281 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:14:33 smithi181 podman[80971]: 2024-04-27 04:14:33.073640231 +0000 UTC m=+0.678243907 container cleanup 96e48aa3912ff515ec9f6dd66d494324ee1054ead5cec21affb7c30d81421ddb (image=quay.ceph.io/ceph-ci/ceph@sha256:d31f278ece6dbd17d1fa5e598f509017f54b76fafe1c4416391fcc51b9718ee9, name=ceph-7f9750d6-044a-11ef-bc93-c7b262605968-mon-smithi181, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.build-date=20240424, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, RELEASE=quincy-2be3efa, ceph=True) 2024-04-27T04:14:33.281 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:14:33 smithi181 bash[80971]: ceph-7f9750d6-044a-11ef-bc93-c7b262605968-mon-smithi181 2024-04-27T04:14:33.594 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:14:33 smithi181 podman[80986]: 2024-04-27 04:14:33.283373815 +0000 UTC m=+0.562506525 container remove 96e48aa3912ff515ec9f6dd66d494324ee1054ead5cec21affb7c30d81421ddb (image=quay.ceph.io/ceph-ci/ceph@sha256:d31f278ece6dbd17d1fa5e598f509017f54b76fafe1c4416391fcc51b9718ee9, name=ceph-7f9750d6-044a-11ef-bc93-c7b262605968-mon-smithi181, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=quincy-2be3efa, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.build-date=20240424, org.label-schema.license=GPLv2, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, ceph=True, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0) 2024-04-27T04:14:33.594 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:14:33 smithi181 systemd[1]: ceph-7f9750d6-044a-11ef-bc93-c7b262605968@mon.smithi181.service: Deactivated successfully. 2024-04-27T04:14:33.594 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:14:33 smithi181 systemd[1]: Stopped Ceph mon.smithi181 for 7f9750d6-044a-11ef-bc93-c7b262605968. 2024-04-27T04:14:33.594 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:14:33 smithi181 systemd[1]: ceph-7f9750d6-044a-11ef-bc93-c7b262605968@mon.smithi181.service: Consumed 6.791s CPU time. 2024-04-27T04:14:33.595 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:14:33 smithi181 systemd[1]: Starting Ceph mon.smithi181 for 7f9750d6-044a-11ef-bc93-c7b262605968... 2024-04-27T04:14:33.922 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:14:33 smithi181 podman[81115]: 2024-04-27 04:14:33.713756176 +0000 UTC m=+0.018125446 image pull 67e414ff4bc41092cb31c84c98a9563a32813d02583028679c46b40362afeb3e quay.ceph.io/ceph-ci/ceph@sha256:90a3a9fa9ceb8977f6dad9ccd36fe46018d07e2118199820523a6b35301d9c16 2024-04-27T04:14:33.922 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:14:33 smithi181 podman[81115]: 2024-04-27 04:14:33.834401829 +0000 UTC m=+0.138771082 container create 77b71f6d522fbf9721875667252bef513a31e55d8e307eeda8c4a035133b1815 (image=quay.ceph.io/ceph-ci/ceph@sha256:90a3a9fa9ceb8977f6dad9ccd36fe46018d07e2118199820523a6b35301d9c16, name=ceph-7f9750d6-044a-11ef-bc93-c7b262605968-mon-smithi181, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD, ceph=True, GIT_CLEAN=True, io.buildah.version=1.33.5, RELEASE=main-4e213a5, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240424) 2024-04-27T04:14:34.105 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:14:34 smithi181 podman[81115]: 2024-04-27 04:14:34.071238699 +0000 UTC m=+0.375607950 container init 77b71f6d522fbf9721875667252bef513a31e55d8e307eeda8c4a035133b1815 (image=quay.ceph.io/ceph-ci/ceph@sha256:90a3a9fa9ceb8977f6dad9ccd36fe46018d07e2118199820523a6b35301d9c16, name=ceph-7f9750d6-044a-11ef-bc93-c7b262605968-mon-smithi181, org.label-schema.build-date=20240424, ceph=True, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD, RELEASE=main-4e213a5, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0) 2024-04-27T04:14:34.105 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:14:34 smithi181 podman[81115]: 2024-04-27 04:14:34.074358178 +0000 UTC m=+0.378727436 container start 77b71f6d522fbf9721875667252bef513a31e55d8e307eeda8c4a035133b1815 (image=quay.ceph.io/ceph-ci/ceph@sha256:90a3a9fa9ceb8977f6dad9ccd36fe46018d07e2118199820523a6b35301d9c16, name=ceph-7f9750d6-044a-11ef-bc93-c7b262605968-mon-smithi181, org.label-schema.build-date=20240424, maintainer=Guillaume Abrioux , RELEASE=main-4e213a5, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD, ceph=True, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=) 2024-04-27T04:14:34.105 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:14:34 smithi181 ceph-mon[81134]: set uid:gid to 167:167 (ceph:ceph) 2024-04-27T04:14:34.105 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:14:34 smithi181 ceph-mon[81134]: ceph version 19.0.0-3253-g4e213a58 (4e213a58e787437ff508e6c890ccd5fd32dc96b5) squid (dev), process ceph-mon, pid 2 2024-04-27T04:14:34.105 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:14:34 smithi181 ceph-mon[81134]: pidfile_write: ignore empty --pid-file 2024-04-27T04:14:34.105 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:14:34 smithi181 ceph-mon[81134]: load: jerasure load: lrc 2024-04-27T04:14:34.105 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:14:34 smithi181 ceph-mon[81134]: rocksdb: RocksDB version: 7.9.2 2024-04-27T04:14:34.105 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:14:34 smithi181 ceph-mon[81134]: rocksdb: Git sha 0 2024-04-27T04:14:34.106 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:14:34 smithi181 ceph-mon[81134]: rocksdb: Compile date 2024-04-24 22:40:16 2024-04-27T04:14:34.106 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:14:34 smithi181 ceph-mon[81134]: rocksdb: DB SUMMARY 2024-04-27T04:14:34.106 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:14:34 smithi181 ceph-mon[81134]: rocksdb: DB Session ID: ICNYFWCMNZSVXW3R0VPN 2024-04-27T04:14:34.106 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:14:34 smithi181 ceph-mon[81134]: rocksdb: CURRENT file: CURRENT 2024-04-27T04:14:34.106 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:14:34 smithi181 ceph-mon[81134]: rocksdb: IDENTITY file: IDENTITY 2024-04-27T04:14:34.106 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:14:34 smithi181 ceph-mon[81134]: rocksdb: MANIFEST file: MANIFEST-000009 size: 1084 Bytes 2024-04-27T04:14:34.106 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:14:34 smithi181 ceph-mon[81134]: rocksdb: SST files in /var/lib/ceph/mon/ceph-smithi181/store.db dir, Total Num: 1, files: 000027.sst 2024-04-27T04:14:34.106 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:14:34 smithi181 ceph-mon[81134]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-smithi181/store.db: 000025.log size: 5591561 ; 2024-04-27T04:14:34.106 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:14:34 smithi181 ceph-mon[81134]: rocksdb: Options.error_if_exists: 0 2024-04-27T04:14:34.106 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:14:34 smithi181 ceph-mon[81134]: rocksdb: Options.create_if_missing: 0 2024-04-27T04:14:34.106 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:14:34 smithi181 ceph-mon[81134]: rocksdb: Options.paranoid_checks: 1 2024-04-27T04:14:34.107 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:14:34 smithi181 ceph-mon[81134]: rocksdb: Options.flush_verify_memtable_count: 1 2024-04-27T04:14:34.107 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:14:34 smithi181 ceph-mon[81134]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-04-27T04:14:34.107 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:14:34 smithi181 ceph-mon[81134]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-04-27T04:14:34.107 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:14:34 smithi181 ceph-mon[81134]: rocksdb: Options.env: 0x55c04c8fecc0 2024-04-27T04:14:34.107 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:14:34 smithi181 ceph-mon[81134]: rocksdb: Options.fs: PosixFileSystem 2024-04-27T04:14:34.107 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:14:34 smithi181 ceph-mon[81134]: rocksdb: Options.info_log: 0x55c04e129c60 2024-04-27T04:14:34.107 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:14:34 smithi181 ceph-mon[81134]: rocksdb: Options.max_file_opening_threads: 16 2024-04-27T04:14:34.108 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:14:34 smithi181 ceph-mon[81134]: rocksdb: Options.statistics: (nil) 2024-04-27T04:14:34.108 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:14:34 smithi181 ceph-mon[81134]: rocksdb: Options.use_fsync: 0 2024-04-27T04:14:34.108 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:14:34 smithi181 ceph-mon[81134]: rocksdb: Options.max_log_file_size: 0 2024-04-27T04:14:34.108 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:14:34 smithi181 ceph-mon[81134]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-04-27T04:14:34.108 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:14:34 smithi181 ceph-mon[81134]: rocksdb: Options.log_file_time_to_roll: 0 2024-04-27T04:14:34.108 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:14:34 smithi181 ceph-mon[81134]: rocksdb: Options.keep_log_file_num: 1000 2024-04-27T04:14:34.108 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:14:34 smithi181 ceph-mon[81134]: rocksdb: Options.recycle_log_file_num: 0 2024-04-27T04:14:34.109 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:14:34 smithi181 ceph-mon[81134]: rocksdb: Options.allow_fallocate: 1 2024-04-27T04:14:34.109 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:14:34 smithi181 ceph-mon[81134]: rocksdb: Options.allow_mmap_reads: 0 2024-04-27T04:14:34.109 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:14:34 smithi181 ceph-mon[81134]: rocksdb: Options.allow_mmap_writes: 0 2024-04-27T04:14:34.109 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:14:34 smithi181 ceph-mon[81134]: rocksdb: Options.use_direct_reads: 0 2024-04-27T04:14:34.109 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:14:34 smithi181 ceph-mon[81134]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-04-27T04:14:34.109 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:14:34 smithi181 ceph-mon[81134]: rocksdb: Options.create_missing_column_families: 0 2024-04-27T04:14:34.109 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:14:34 smithi181 ceph-mon[81134]: rocksdb: Options.db_log_dir: 2024-04-27T04:14:34.109 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:14:34 smithi181 ceph-mon[81134]: rocksdb: Options.wal_dir: 2024-04-27T04:14:34.109 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:14:34 smithi181 ceph-mon[81134]: rocksdb: Options.table_cache_numshardbits: 6 2024-04-27T04:14:34.110 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:14:34 smithi181 ceph-mon[81134]: rocksdb: Options.WAL_ttl_seconds: 0 2024-04-27T04:14:34.110 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:14:34 smithi181 ceph-mon[81134]: rocksdb: Options.WAL_size_limit_MB: 0 2024-04-27T04:14:34.110 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:14:34 smithi181 ceph-mon[81134]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-04-27T04:14:34.110 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:14:34 smithi181 ceph-mon[81134]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-04-27T04:14:34.110 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:14:34 smithi181 ceph-mon[81134]: rocksdb: Options.is_fd_close_on_exec: 1 2024-04-27T04:14:34.110 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:14:34 smithi181 ceph-mon[81134]: rocksdb: Options.advise_random_on_open: 1 2024-04-27T04:14:34.110 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:14:34 smithi181 ceph-mon[81134]: rocksdb: Options.db_write_buffer_size: 0 2024-04-27T04:14:34.110 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:14:34 smithi181 ceph-mon[81134]: rocksdb: Options.write_buffer_manager: 0x55c04e0fb0e0 2024-04-27T04:14:34.111 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:14:34 smithi181 ceph-mon[81134]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-04-27T04:14:34.111 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:14:34 smithi181 ceph-mon[81134]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-04-27T04:14:34.111 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:14:34 smithi181 ceph-mon[81134]: rocksdb: Options.use_adaptive_mutex: 0 2024-04-27T04:14:34.111 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:14:34 smithi181 ceph-mon[81134]: rocksdb: Options.rate_limiter: (nil) 2024-04-27T04:14:34.111 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:14:34 smithi181 ceph-mon[81134]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-04-27T04:14:34.111 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:14:34 smithi181 ceph-mon[81134]: rocksdb: Options.wal_recovery_mode: 2 2024-04-27T04:14:34.111 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:14:34 smithi181 ceph-mon[81134]: rocksdb: Options.enable_thread_tracking: 0 2024-04-27T04:14:34.111 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:14:34 smithi181 ceph-mon[81134]: rocksdb: Options.enable_pipelined_write: 0 2024-04-27T04:14:34.111 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:14:34 smithi181 ceph-mon[81134]: rocksdb: Options.unordered_write: 0 2024-04-27T04:14:34.111 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:14:34 smithi181 ceph-mon[81134]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-04-27T04:14:34.111 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:14:34 smithi181 ceph-mon[81134]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-04-27T04:14:34.111 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:14:34 smithi181 ceph-mon[81134]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-04-27T04:14:34.111 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:14:34 smithi181 ceph-mon[81134]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-04-27T04:14:34.112 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:14:34 smithi181 ceph-mon[81134]: rocksdb: Options.row_cache: None 2024-04-27T04:14:34.112 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:14:34 smithi181 ceph-mon[81134]: rocksdb: Options.wal_filter: None 2024-04-27T04:14:34.112 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:14:34 smithi181 ceph-mon[81134]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-04-27T04:14:34.112 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:14:34 smithi181 ceph-mon[81134]: rocksdb: Options.allow_ingest_behind: 0 2024-04-27T04:14:34.112 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:14:34 smithi181 ceph-mon[81134]: rocksdb: Options.two_write_queues: 0 2024-04-27T04:14:34.112 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:14:34 smithi181 ceph-mon[81134]: rocksdb: Options.manual_wal_flush: 0 2024-04-27T04:14:34.112 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:14:34 smithi181 ceph-mon[81134]: rocksdb: Options.wal_compression: 0 2024-04-27T04:14:34.112 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:14:34 smithi181 ceph-mon[81134]: rocksdb: Options.atomic_flush: 0 2024-04-27T04:14:34.112 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:14:34 smithi181 ceph-mon[81134]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-04-27T04:14:34.112 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:14:34 smithi181 ceph-mon[81134]: rocksdb: Options.persist_stats_to_disk: 0 2024-04-27T04:14:34.112 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:14:34 smithi181 ceph-mon[81134]: rocksdb: Options.write_dbid_to_manifest: 0 2024-04-27T04:14:34.112 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:14:34 smithi181 ceph-mon[81134]: rocksdb: Options.log_readahead_size: 0 2024-04-27T04:14:34.112 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:14:34 smithi181 ceph-mon[81134]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-04-27T04:14:34.113 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:14:34 smithi181 ceph-mon[81134]: rocksdb: Options.best_efforts_recovery: 0 2024-04-27T04:14:34.113 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:14:34 smithi181 ceph-mon[81134]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-04-27T04:14:34.113 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:14:34 smithi181 ceph-mon[81134]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-04-27T04:14:34.113 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:14:34 smithi181 ceph-mon[81134]: rocksdb: Options.allow_data_in_errors: 0 2024-04-27T04:14:34.113 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:14:34 smithi181 ceph-mon[81134]: rocksdb: Options.db_host_id: __hostname__ 2024-04-27T04:14:34.113 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:14:34 smithi181 ceph-mon[81134]: rocksdb: Options.enforce_single_del_contracts: true 2024-04-27T04:14:34.113 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:14:34 smithi181 ceph-mon[81134]: rocksdb: Options.max_background_jobs: 2 2024-04-27T04:14:34.113 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:14:34 smithi181 ceph-mon[81134]: rocksdb: Options.max_background_compactions: -1 2024-04-27T04:14:34.113 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:14:34 smithi181 ceph-mon[81134]: rocksdb: Options.max_subcompactions: 1 2024-04-27T04:14:34.113 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:14:34 smithi181 ceph-mon[81134]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-04-27T04:14:34.114 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:14:34 smithi181 ceph-mon[81134]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-04-27T04:14:34.114 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:14:34 smithi181 ceph-mon[81134]: rocksdb: Options.delayed_write_rate : 16777216 2024-04-27T04:14:34.114 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:14:34 smithi181 ceph-mon[81134]: rocksdb: Options.max_total_wal_size: 0 2024-04-27T04:14:34.114 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:14:34 smithi181 ceph-mon[81134]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-04-27T04:14:34.114 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:14:34 smithi181 ceph-mon[81134]: rocksdb: Options.stats_dump_period_sec: 600 2024-04-27T04:14:34.114 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:14:34 smithi181 ceph-mon[81134]: rocksdb: Options.stats_persist_period_sec: 600 2024-04-27T04:14:34.114 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:14:34 smithi181 ceph-mon[81134]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-04-27T04:14:34.114 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:14:34 smithi181 ceph-mon[81134]: rocksdb: Options.max_open_files: -1 2024-04-27T04:14:34.114 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:14:34 smithi181 ceph-mon[81134]: rocksdb: Options.bytes_per_sync: 0 2024-04-27T04:14:34.114 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:14:34 smithi181 ceph-mon[81134]: rocksdb: Options.wal_bytes_per_sync: 0 2024-04-27T04:14:34.114 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:14:34 smithi181 ceph-mon[81134]: rocksdb: Options.strict_bytes_per_sync: 0 2024-04-27T04:14:34.114 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:14:34 smithi181 ceph-mon[81134]: rocksdb: Options.compaction_readahead_size: 0 2024-04-27T04:14:34.115 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:14:34 smithi181 ceph-mon[81134]: rocksdb: Options.max_background_flushes: -1 2024-04-27T04:14:34.115 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:14:34 smithi181 ceph-mon[81134]: rocksdb: Compression algorithms supported: 2024-04-27T04:14:34.115 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:14:34 smithi181 ceph-mon[81134]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-04-27T04:14:34.115 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:14:34 smithi181 ceph-mon[81134]: rocksdb: kZSTD supported: 0 2024-04-27T04:14:34.115 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:14:34 smithi181 ceph-mon[81134]: rocksdb: kXpressCompression supported: 0 2024-04-27T04:14:34.115 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:14:34 smithi181 ceph-mon[81134]: rocksdb: kLZ4HCCompression supported: 1 2024-04-27T04:14:34.115 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:14:34 smithi181 ceph-mon[81134]: rocksdb: kZlibCompression supported: 1 2024-04-27T04:14:34.115 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:14:34 smithi181 ceph-mon[81134]: rocksdb: kSnappyCompression supported: 1 2024-04-27T04:14:34.116 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:14:34 smithi181 ceph-mon[81134]: rocksdb: kLZ4Compression supported: 1 2024-04-27T04:14:34.116 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:14:34 smithi181 ceph-mon[81134]: rocksdb: kBZip2Compression supported: 0 2024-04-27T04:14:34.116 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:14:34 smithi181 ceph-mon[81134]: rocksdb: Fast CRC32 supported: Supported on x86 2024-04-27T04:14:34.116 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:14:34 smithi181 ceph-mon[81134]: rocksdb: DMutex implementation: pthread_mutex_t 2024-04-27T04:14:34.116 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:14:34 smithi181 ceph-mon[81134]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-smithi181/store.db/MANIFEST-000009 2024-04-27T04:14:34.116 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:14:34 smithi181 ceph-mon[81134]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-04-27T04:14:34.116 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:14:34 smithi181 ceph-mon[81134]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-04-27T04:14:34.116 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:14:34 smithi181 ceph-mon[81134]: rocksdb: Options.merge_operator: 2024-04-27T04:14:34.117 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:14:34 smithi181 ceph-mon[81134]: rocksdb: Options.compaction_filter: None 2024-04-27T04:14:34.117 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:14:34 smithi181 ceph-mon[81134]: rocksdb: Options.compaction_filter_factory: None 2024-04-27T04:14:34.117 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:14:34 smithi181 ceph-mon[81134]: rocksdb: Options.sst_partitioner_factory: None 2024-04-27T04:14:34.117 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:14:34 smithi181 ceph-mon[81134]: rocksdb: Options.memtable_factory: SkipListFactory 2024-04-27T04:14:34.117 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:14:34 smithi181 ceph-mon[81134]: rocksdb: Options.table_factory: BlockBasedTable 2024-04-27T04:14:34.118 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:14:34 smithi181 ceph-mon[81134]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x55c04e129e20) 2024-04-27T04:14:34.118 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout: cache_index_and_filter_blocks: 1 2024-04-27T04:14:34.118 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-04-27T04:14:34.118 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-04-27T04:14:34.118 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout: pin_top_level_index_and_filter: 1 2024-04-27T04:14:34.118 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout: index_type: 0 2024-04-27T04:14:34.118 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout: data_block_index_type: 0 2024-04-27T04:14:34.118 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout: index_shortening: 1 2024-04-27T04:14:34.118 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout: data_block_hash_table_util_ratio: 0.750000 2024-04-27T04:14:34.118 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout: checksum: 4 2024-04-27T04:14:34.118 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout: no_block_cache: 0 2024-04-27T04:14:34.119 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout: block_cache: 0x55c04e12d1f0 2024-04-27T04:14:34.119 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout: block_cache_name: BinnedLRUCache 2024-04-27T04:14:34.119 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout: block_cache_options: 2024-04-27T04:14:34.119 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout: capacity : 536870912 2024-04-27T04:14:34.119 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout: num_shard_bits : 4 2024-04-27T04:14:34.119 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout: strict_capacity_limit : 0 2024-04-27T04:14:34.119 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout: high_pri_pool_ratio: 0.000 2024-04-27T04:14:34.119 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout: block_cache_compressed: (nil) 2024-04-27T04:14:34.119 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout: persistent_cache: (nil) 2024-04-27T04:14:34.120 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout: block_size: 4096 2024-04-27T04:14:34.120 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout: block_size_deviation: 10 2024-04-27T04:14:34.120 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout: block_restart_interval: 16 2024-04-27T04:14:34.120 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout: index_block_restart_interval: 1 2024-04-27T04:14:34.120 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout: metadata_block_size: 4096 2024-04-27T04:14:34.120 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout: partition_filters: 0 2024-04-27T04:14:34.120 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout: use_delta_encoding: 1 2024-04-27T04:14:34.120 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout: filter_policy: bloomfilter 2024-04-27T04:14:34.120 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout: whole_key_filtering: 1 2024-04-27T04:14:34.120 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout: verify_compression: 0 2024-04-27T04:14:34.120 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout: read_amp_bytes_per_bit: 0 2024-04-27T04:14:34.120 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout: format_version: 5 2024-04-27T04:14:34.121 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout: enable_index_compression: 1 2024-04-27T04:14:34.121 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout: block_align: 0 2024-04-27T04:14:34.121 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout: max_auto_readahead_size: 262144 2024-04-27T04:14:34.121 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout: prepopulate_block_cache: 0 2024-04-27T04:14:34.121 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout: initial_auto_readahead_size: 8192 2024-04-27T04:14:34.121 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout: num_file_reads_for_auto_readahead: 2 2024-04-27T04:14:34.121 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:14:34 smithi181 ceph-mon[81134]: rocksdb: Options.write_buffer_size: 33554432 2024-04-27T04:14:34.121 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:14:34 smithi181 ceph-mon[81134]: rocksdb: Options.max_write_buffer_number: 2 2024-04-27T04:14:34.121 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:14:34 smithi181 ceph-mon[81134]: rocksdb: Options.compression: NoCompression 2024-04-27T04:14:34.121 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:14:34 smithi181 ceph-mon[81134]: rocksdb: Options.bottommost_compression: Disabled 2024-04-27T04:14:34.122 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:14:34 smithi181 ceph-mon[81134]: rocksdb: Options.prefix_extractor: nullptr 2024-04-27T04:14:34.122 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:14:34 smithi181 ceph-mon[81134]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-04-27T04:14:34.122 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:14:34 smithi181 ceph-mon[81134]: rocksdb: Options.num_levels: 7 2024-04-27T04:14:34.122 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:14:34 smithi181 ceph-mon[81134]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-04-27T04:14:34.122 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:14:34 smithi181 ceph-mon[81134]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-04-27T04:14:34.122 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:14:34 smithi181 ceph-mon[81134]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-04-27T04:14:34.122 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:14:34 smithi181 ceph-mon[81134]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-04-27T04:14:34.122 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:14:34 smithi181 ceph-mon[81134]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-04-27T04:14:34.122 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:14:34 smithi181 ceph-mon[81134]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-04-27T04:14:34.122 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:14:34 smithi181 ceph-mon[81134]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-04-27T04:14:34.122 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:14:34 smithi181 ceph-mon[81134]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-04-27T04:14:34.122 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:14:34 smithi181 ceph-mon[81134]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-04-27T04:14:34.123 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:14:34 smithi181 ceph-mon[81134]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-04-27T04:14:34.123 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:14:34 smithi181 ceph-mon[81134]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-04-27T04:14:34.123 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:14:34 smithi181 ceph-mon[81134]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-04-27T04:14:34.123 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:14:34 smithi181 ceph-mon[81134]: rocksdb: Options.compression_opts.window_bits: -14 2024-04-27T04:14:34.123 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:14:34 smithi181 ceph-mon[81134]: rocksdb: Options.compression_opts.level: 32767 2024-04-27T04:14:34.123 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:14:34 smithi181 ceph-mon[81134]: rocksdb: Options.compression_opts.strategy: 0 2024-04-27T04:14:34.123 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:14:34 smithi181 ceph-mon[81134]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-04-27T04:14:34.123 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:14:34 smithi181 ceph-mon[81134]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-04-27T04:14:34.123 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:14:34 smithi181 ceph-mon[81134]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-04-27T04:14:34.123 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:14:34 smithi181 ceph-mon[81134]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-04-27T04:14:34.123 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:14:34 smithi181 ceph-mon[81134]: rocksdb: Options.compression_opts.enabled: false 2024-04-27T04:14:34.123 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:14:34 smithi181 ceph-mon[81134]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-04-27T04:14:34.124 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:14:34 smithi181 ceph-mon[81134]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-04-27T04:14:34.124 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:14:34 smithi181 ceph-mon[81134]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-04-27T04:14:34.124 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:14:34 smithi181 ceph-mon[81134]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-04-27T04:14:34.124 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:14:34 smithi181 ceph-mon[81134]: rocksdb: Options.target_file_size_base: 67108864 2024-04-27T04:14:34.124 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:14:34 smithi181 ceph-mon[81134]: rocksdb: Options.target_file_size_multiplier: 1 2024-04-27T04:14:34.124 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:14:34 smithi181 ceph-mon[81134]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-04-27T04:14:34.124 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:14:34 smithi181 ceph-mon[81134]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-04-27T04:14:34.124 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:14:34 smithi181 ceph-mon[81134]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-04-27T04:14:34.124 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:14:34 smithi181 ceph-mon[81134]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-04-27T04:14:34.125 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:14:34 smithi181 ceph-mon[81134]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-04-27T04:14:34.125 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:14:34 smithi181 ceph-mon[81134]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-04-27T04:14:34.125 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:14:34 smithi181 ceph-mon[81134]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-04-27T04:14:34.125 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:14:34 smithi181 ceph-mon[81134]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-04-27T04:14:34.125 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:14:34 smithi181 ceph-mon[81134]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-04-27T04:14:34.126 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:14:34 smithi181 ceph-mon[81134]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-04-27T04:14:34.126 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:14:34 smithi181 ceph-mon[81134]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-04-27T04:14:34.126 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:14:34 smithi181 ceph-mon[81134]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-04-27T04:14:34.126 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:14:34 smithi181 ceph-mon[81134]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-04-27T04:14:34.126 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:14:34 smithi181 ceph-mon[81134]: rocksdb: Options.arena_block_size: 1048576 2024-04-27T04:14:34.126 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:14:34 smithi181 ceph-mon[81134]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-04-27T04:14:34.126 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:14:34 smithi181 ceph-mon[81134]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-04-27T04:14:34.126 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:14:34 smithi181 ceph-mon[81134]: rocksdb: Options.disable_auto_compactions: 0 2024-04-27T04:14:34.126 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:14:34 smithi181 ceph-mon[81134]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-04-27T04:14:34.126 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:14:34 smithi181 ceph-mon[81134]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-04-27T04:14:34.126 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:14:34 smithi181 ceph-mon[81134]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-04-27T04:14:34.127 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:14:34 smithi181 ceph-mon[81134]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-04-27T04:14:34.127 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:14:34 smithi181 ceph-mon[81134]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-04-27T04:14:34.127 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:14:34 smithi181 ceph-mon[81134]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-04-27T04:14:34.127 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:14:34 smithi181 ceph-mon[81134]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-04-27T04:14:34.127 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:14:34 smithi181 ceph-mon[81134]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-04-27T04:14:34.127 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:14:34 smithi181 ceph-mon[81134]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-04-27T04:14:34.127 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:14:34 smithi181 ceph-mon[81134]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-04-27T04:14:34.127 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:14:34 smithi181 ceph-mon[81134]: rocksdb: Options.table_properties_collectors: CompactOnDeletionCollector (Sliding window size = 32768 Deletion trigger = 16384 Deletion ratio = 0); 2024-04-27T04:14:34.127 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:14:34 smithi181 ceph-mon[81134]: rocksdb: Options.inplace_update_support: 0 2024-04-27T04:14:34.127 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:14:34 smithi181 ceph-mon[81134]: rocksdb: Options.inplace_update_num_locks: 10000 2024-04-27T04:14:34.127 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:14:34 smithi181 ceph-mon[81134]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-04-27T04:14:34.127 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:14:34 smithi181 ceph-mon[81134]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-04-27T04:14:34.128 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:14:34 smithi181 ceph-mon[81134]: rocksdb: Options.memtable_huge_page_size: 0 2024-04-27T04:14:34.128 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:14:34 smithi181 ceph-mon[81134]: rocksdb: Options.bloom_locality: 0 2024-04-27T04:14:34.128 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:14:34 smithi181 ceph-mon[81134]: rocksdb: Options.max_successive_merges: 0 2024-04-27T04:14:34.128 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:14:34 smithi181 ceph-mon[81134]: rocksdb: Options.optimize_filters_for_hits: 0 2024-04-27T04:14:34.128 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:14:34 smithi181 ceph-mon[81134]: rocksdb: Options.paranoid_file_checks: 0 2024-04-27T04:14:34.128 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:14:34 smithi181 ceph-mon[81134]: rocksdb: Options.force_consistency_checks: 1 2024-04-27T04:14:34.128 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:14:34 smithi181 ceph-mon[81134]: rocksdb: Options.report_bg_io_stats: 0 2024-04-27T04:14:34.128 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:14:34 smithi181 ceph-mon[81134]: rocksdb: Options.ttl: 2592000 2024-04-27T04:14:34.128 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:14:34 smithi181 ceph-mon[81134]: rocksdb: Options.periodic_compaction_seconds: 0 2024-04-27T04:14:34.128 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:14:34 smithi181 ceph-mon[81134]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-04-27T04:14:34.128 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:14:34 smithi181 ceph-mon[81134]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-04-27T04:14:34.128 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:14:34 smithi181 ceph-mon[81134]: rocksdb: Options.enable_blob_files: false 2024-04-27T04:14:34.129 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:14:34 smithi181 ceph-mon[81134]: rocksdb: Options.min_blob_size: 0 2024-04-27T04:14:34.129 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:14:34 smithi181 ceph-mon[81134]: rocksdb: Options.blob_file_size: 268435456 2024-04-27T04:14:34.129 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:14:34 smithi181 ceph-mon[81134]: rocksdb: Options.blob_compression_type: NoCompression 2024-04-27T04:14:34.129 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:14:34 smithi181 ceph-mon[81134]: rocksdb: Options.enable_blob_garbage_collection: false 2024-04-27T04:14:34.129 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:14:34 smithi181 ceph-mon[81134]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-04-27T04:14:34.129 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:14:34 smithi181 ceph-mon[81134]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-04-27T04:14:34.129 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:14:34 smithi181 ceph-mon[81134]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-04-27T04:14:34.129 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:14:34 smithi181 ceph-mon[81134]: rocksdb: Options.blob_file_starting_level: 0 2024-04-27T04:14:34.129 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:14:34 smithi181 ceph-mon[81134]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-04-27T04:14:34.129 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:14:34 smithi181 ceph-mon[81134]: rocksdb: [table/block_based/block_based_table_reader.cc:721] At least one SST file opened without unique ID to verify: 27.sst 2024-04-27T04:14:34.129 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:14:34 smithi181 ceph-mon[81134]: rocksdb: [db/version_set.cc:4390] More existing levels in DB than needed. max_bytes_for_level_multiplier may not be guaranteed. 2024-04-27T04:14:34.129 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:14:34 smithi181 ceph-mon[81134]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-smithi181/store.db/MANIFEST-000009 succeeded,manifest_file_number is 9, next_file_number is 29, last_sequence is 12261, log_number is 25,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 0 2024-04-27T04:14:34.130 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:14:34 smithi181 ceph-mon[81134]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 25 2024-04-27T04:14:34.130 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:14:34 smithi181 ceph-mon[81134]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: abc600b7-a152-4e8f-bf9e-7ef9a31ac24d 2024-04-27T04:14:34.130 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:14:34 smithi181 ceph-mon[81134]: rocksdb: EVENT_LOG_v1 {"time_micros": 1714191274103797, "job": 1, "event": "recovery_started", "wal_files": [25]} 2024-04-27T04:14:34.130 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:14:34 smithi181 ceph-mon[81134]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #25 mode 2 2024-04-27T04:14:34.397 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:14:34 smithi181 ceph-mon[81134]: rocksdb: EVENT_LOG_v1 {"time_micros": 1714191274118480, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 30, "file_size": 3433006, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 12266, "largest_seqno": 14413, "table_properties": {"data_size": 3424098, "index_size": 5622, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 2245, "raw_key_size": 21456, "raw_average_key_size": 24, "raw_value_size": 3405094, "raw_average_value_size": 3813, "num_data_blocks": 254, "num_entries": 893, "num_filter_entries": 893, "num_deletions": 7, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[CompactOnDeletionCollector]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1714191274, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "abc600b7-a152-4e8f-bf9e-7ef9a31ac24d", "db_session_id": "ICNYFWCMNZSVXW3R0VPN", "orig_file_number": 30, "seqno_to_time_mapping": "N/A"}} 2024-04-27T04:14:34.397 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:14:34 smithi181 ceph-mon[81134]: rocksdb: EVENT_LOG_v1 {"time_micros": 1714191274118590, "job": 1, "event": "recovery_finished"} 2024-04-27T04:14:34.398 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:14:34 smithi181 ceph-mon[81134]: rocksdb: [db/version_set.cc:5047] Creating manifest 32 2024-04-27T04:14:34.398 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:14:34 smithi181 ceph-mon[81134]: rocksdb: [db/version_set.cc:4390] More existing levels in DB than needed. max_bytes_for_level_multiplier may not be guaranteed. 2024-04-27T04:14:34.398 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:14:34 smithi181 ceph-mon[81134]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-smithi181/store.db/000025.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-04-27T04:14:34.398 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:14:34 smithi181 ceph-mon[81134]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x55c04e24a000 2024-04-27T04:14:34.398 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:14:34 smithi181 ceph-mon[81134]: rocksdb: DB pointer 0x55c04e234000 2024-04-27T04:14:34.398 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:14:34 smithi181 ceph-mon[81134]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-04-27T04:14:34.398 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:14:34 smithi181 ceph-mon[81134]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-04-27T04:14:34.398 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout: ** DB Stats ** 2024-04-27T04:14:34.399 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-04-27T04:14:34.399 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-04-27T04:14:34.399 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-04-27T04:14:34.399 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-04-27T04:14:34.399 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-04-27T04:14:34.399 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-04-27T04:14:34.399 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-04-27T04:14:34.399 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout: 2024-04-27T04:14:34.399 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout: ** Compaction Stats [default] ** 2024-04-27T04:14:34.399 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-04-27T04:14:34.399 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-04-27T04:14:34.399 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout: L0 1/0 3.27 MB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 646.9 0.01 0.00 1 0.005 0 0 0.0 0.0 2024-04-27T04:14:34.399 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout: L6 1/0 6.72 MB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 2024-04-27T04:14:34.399 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout: Sum 2/0 9.99 MB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 646.9 0.01 0.00 1 0.005 0 0 0.0 0.0 2024-04-27T04:14:34.400 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 646.9 0.01 0.00 1 0.005 0 0 0.0 0.0 2024-04-27T04:14:34.400 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout: 2024-04-27T04:14:34.400 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout: ** Compaction Stats [default] ** 2024-04-27T04:14:34.400 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-04-27T04:14:34.400 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-04-27T04:14:34.400 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 646.9 0.01 0.00 1 0.005 0 0 0.0 0.0 2024-04-27T04:14:34.400 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout: 2024-04-27T04:14:34.400 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-04-27T04:14:34.400 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout: 2024-04-27T04:14:34.400 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-04-27T04:14:34.400 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout: Flush(GB): cumulative 0.003, interval 0.003 2024-04-27T04:14:34.400 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-04-27T04:14:34.401 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-04-27T04:14:34.401 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-04-27T04:14:34.401 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout: AddFile(Keys): cumulative 0, interval 0 2024-04-27T04:14:34.401 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout: Cumulative compaction: 0.00 GB write, 154.48 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-04-27T04:14:34.401 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout: Interval compaction: 0.00 GB write, 154.48 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-04-27T04:14:34.401 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-04-27T04:14:34.401 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout: Block cache BinnedLRUCache@0x55c04e12d1f0#2 capacity: 512.00 MB usage: 8.36 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 1.1e-05 secs_since: 0 2024-04-27T04:14:34.401 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout: Block cache entry stats(count,size,portion): FilterBlock(1,2.28 KB,0.000435114%) IndexBlock(1,6.08 KB,0.00115931%) Misc(1,0.00 KB,0%) 2024-04-27T04:14:34.401 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout: 2024-04-27T04:14:34.401 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout: ** File Read Latency Histogram By Level [default] ** 2024-04-27T04:14:34.401 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:14:34 smithi181 ceph-mon[81134]: starting mon.smithi181 rank 1 at public addrs [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] at bind addrs [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] mon_data /var/lib/ceph/mon/ceph-smithi181 fsid 7f9750d6-044a-11ef-bc93-c7b262605968 2024-04-27T04:14:34.401 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:14:34 smithi181 ceph-mon[81134]: mon.smithi181@-1(???) e2 preinit fsid 7f9750d6-044a-11ef-bc93-c7b262605968 2024-04-27T04:14:34.401 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:14:34 smithi181 ceph-mon[81134]: mon.smithi181@-1(???).mds e15 new map 2024-04-27T04:14:34.402 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:14:34 smithi181 ceph-mon[81134]: mon.smithi181@-1(???).mds e15 print_map 2024-04-27T04:14:34.402 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout: e15 2024-04-27T04:14:34.402 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-04-27T04:14:34.402 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-27T04:14:34.402 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout: legacy client fscid: 1 2024-04-27T04:14:34.402 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout: 2024-04-27T04:14:34.402 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout: Filesystem 'cephfs' (1) 2024-04-27T04:14:34.402 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout: fs_name cephfs 2024-04-27T04:14:34.402 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout: epoch 15 2024-04-27T04:14:34.402 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout: flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-04-27T04:14:34.402 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout: created 2024-04-27T04:06:53.095749+0000 2024-04-27T04:14:34.402 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout: modified 2024-04-27T04:07:11.285033+0000 2024-04-27T04:14:34.402 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout: tableserver 0 2024-04-27T04:14:34.403 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout: root 0 2024-04-27T04:14:34.403 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout: session_timeout 60 2024-04-27T04:14:34.403 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout: session_autoclose 300 2024-04-27T04:14:34.403 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout: max_file_size 1099511627776 2024-04-27T04:14:34.403 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout: max_xattr_size 65536 2024-04-27T04:14:34.403 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout: required_client_features {} 2024-04-27T04:14:34.403 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout: last_failure 0 2024-04-27T04:14:34.403 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout: last_failure_osd_epoch 46 2024-04-27T04:14:34.403 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout: compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-27T04:14:34.403 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout: max_mds 1 2024-04-27T04:14:34.403 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout: in 0 2024-04-27T04:14:34.403 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout: up {0=24273} 2024-04-27T04:14:34.404 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout: failed 2024-04-27T04:14:34.404 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout: damaged 2024-04-27T04:14:34.404 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout: stopped 2024-04-27T04:14:34.404 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout: data_pools [3] 2024-04-27T04:14:34.404 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout: metadata_pool 2 2024-04-27T04:14:34.404 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout: inline_data disabled 2024-04-27T04:14:34.404 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout: balancer 2024-04-27T04:14:34.404 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout: bal_rank_mask -1 2024-04-27T04:14:34.404 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout: standby_count_wanted 1 2024-04-27T04:14:34.404 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout: qdb_cluster leader: 0 members: 2024-04-27T04:14:34.404 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout: [mds.cephfs.smithi181.dzomgh{0:24273} state up:active seq 6 join_fscid=1 addr [v2:172.21.15.181:6824/658214993,v1:172.21.15.181:6825/658214993] compat {c=[1],r=[1],i=[7ff]}] 2024-04-27T04:14:34.405 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout: [mds.cephfs.smithi162.xcnapk{0:14476} state up:standby-replay seq 2 join_fscid=1 addr [v2:172.21.15.162:6828/1296160676,v1:172.21.15.162:6829/1296160676] compat {c=[1],r=[1],i=[7ff]}] 2024-04-27T04:14:34.405 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout: 2024-04-27T04:14:34.405 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout: 2024-04-27T04:14:34.405 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout: Standby daemons: 2024-04-27T04:14:34.405 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout: 2024-04-27T04:14:34.405 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout: [mds.cephfs.smithi162.gzlyjj{-1:14488} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.162:6826/2629103894,v1:172.21.15.162:6827/2629103894] compat {c=[1],r=[1],i=[7ff]}] 2024-04-27T04:14:34.405 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout: [mds.cephfs.smithi181.bkkcdm{-1:24287} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.181:6826/3756475453,v1:172.21.15.181:6827/3756475453] compat {c=[1],r=[1],i=[7ff]}] 2024-04-27T04:14:34.405 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:14:34 smithi181 ceph-mon[81134]: mon.smithi181@-1(???).osd e49 crush map has features 3314933000852226048, adjusting msgr requires 2024-04-27T04:14:34.405 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:14:34 smithi181 ceph-mon[81134]: mon.smithi181@-1(???).osd e49 crush map has features 288514051259236352, adjusting msgr requires 2024-04-27T04:14:34.405 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:14:34 smithi181 ceph-mon[81134]: mon.smithi181@-1(???).osd e49 crush map has features 288514051259236352, adjusting msgr requires 2024-04-27T04:14:34.405 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:14:34 smithi181 ceph-mon[81134]: mon.smithi181@-1(???).osd e49 crush map has features 288514051259236352, adjusting msgr requires 2024-04-27T04:14:34.405 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:14:34 smithi181 ceph-mon[81134]: mon.smithi181@-1(???).paxosservice(auth 1..22) refresh upgraded, format 0 -> 3 2024-04-27T04:14:34.405 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:14:34 smithi181 bash[81115]: 77b71f6d522fbf9721875667252bef513a31e55d8e307eeda8c4a035133b1815 2024-04-27T04:14:34.406 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:14:34 smithi181 systemd[1]: Started Ceph mon.smithi181 for 7f9750d6-044a-11ef-bc93-c7b262605968. 2024-04-27T04:14:35.922 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:14:35 smithi181 ceph-mon[81134]: pgmap v10: 65 pgs: 65 active+clean; 208 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 18 KiB/s rd, 0 B/s wr, 11 op/s 2024-04-27T04:14:35.922 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:14:35 smithi181 ceph-mon[81134]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-27T04:14:35.975 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:14:35 smithi162 ceph-mon[110641]: pgmap v10: 65 pgs: 65 active+clean; 208 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 18 KiB/s rd, 0 B/s wr, 11 op/s 2024-04-27T04:14:35.975 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:14:35 smithi162 ceph-mon[110641]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-27T04:14:35.976 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:14:35.976+0000 7febafc05700 1 -- 172.21.15.162:0/1020810214 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7feba8070bb0 msgr2=0x7feba8109150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:14:35.977 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:14:35.976+0000 7febafc05700 1 --2- 172.21.15.162:0/1020810214 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7feba8070bb0 0x7feba8109150 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7feb98004970 tx=0x7feb98009e30 comp rx=0 tx=0).stop 2024-04-27T04:14:35.977 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:14:35.976+0000 7febafc05700 1 -- 172.21.15.162:0/1020810214 shutdown_connections 2024-04-27T04:14:35.977 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:14:35.976+0000 7febafc05700 1 --2- 172.21.15.162:0/1020810214 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7feba8070bb0 0x7feba8109150 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:14:35.977 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:14:35.976+0000 7febafc05700 1 --2- 172.21.15.162:0/1020810214 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7feba80702a0 0x7feba8070670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:14:35.977 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:14:35.976+0000 7febafc05700 1 -- 172.21.15.162:0/1020810214 >> 172.21.15.162:0/1020810214 conn(0x7feba80f8990 msgr2=0x7feba80fada0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:14:35.977 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:14:35.976+0000 7febafc05700 1 -- 172.21.15.162:0/1020810214 shutdown_connections 2024-04-27T04:14:35.977 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:14:35.976+0000 7febafc05700 1 -- 172.21.15.162:0/1020810214 wait complete. 2024-04-27T04:14:35.977 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:14:35.977+0000 7febafc05700 1 Processor -- start 2024-04-27T04:14:35.977 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:14:35.977+0000 7febafc05700 1 -- start start 2024-04-27T04:14:35.977 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:14:35.977+0000 7febafc05700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7feba80702a0 0x7feba8115b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:14:35.977 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:14:35.977+0000 7febafc05700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7feba8070bb0 0x7feba81160b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:14:35.978 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:14:35.977+0000 7febafc05700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feba8116700 con 0x7feba8070bb0 2024-04-27T04:14:35.978 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:14:35.977+0000 7febafc05700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feba810fce0 con 0x7feba80702a0 2024-04-27T04:14:35.978 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:14:35.977+0000 7febad1a0700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7feba8070bb0 0x7feba81160b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:14:35.978 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:14:35.977+0000 7febad1a0700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7feba8070bb0 0x7feba81160b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:58174/0 (socket says 172.21.15.162:58174) 2024-04-27T04:14:35.978 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:14:35.977+0000 7febad1a0700 1 -- 172.21.15.162:0/1919518601 learned_addr learned my addr 172.21.15.162:0/1919518601 (peer_addr_for_me v2:172.21.15.162:0/0) 2024-04-27T04:14:35.978 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:14:35.977+0000 7febad9a1700 1 --2- 172.21.15.162:0/1919518601 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7feba80702a0 0x7feba8115b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:14:35.978 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:14:35.977+0000 7febad9a1700 1 -- 172.21.15.162:0/1919518601 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7feba80702a0 msgr2=0x7feba8115b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 12 2024-04-27T04:14:35.978 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:14:35.977+0000 7febad9a1700 1 -- 172.21.15.162:0/1919518601 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7feba80702a0 msgr2=0x7feba8115b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-04-27T04:14:35.978 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:14:35.977+0000 7febad9a1700 1 --2- 172.21.15.162:0/1919518601 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7feba80702a0 0x7feba8115b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2024-04-27T04:14:35.978 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:14:35.977+0000 7febad9a1700 1 --2- 172.21.15.162:0/1919518601 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7feba80702a0 0x7feba8115b70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-27T04:14:35.978 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:14:35.977+0000 7febad1a0700 1 -- 172.21.15.162:0/1919518601 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7feba80702a0 msgr2=0x7feba8115b70 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-27T04:14:35.978 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:14:35.977+0000 7febad1a0700 1 --2- 172.21.15.162:0/1919518601 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7feba80702a0 0x7feba8115b70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:14:35.978 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:14:35.977+0000 7febad1a0700 1 -- 172.21.15.162:0/1919518601 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feb98005040 con 0x7feba8070bb0 2024-04-27T04:14:35.979 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:14:35.978+0000 7febad1a0700 1 --2- 172.21.15.162:0/1919518601 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7feba8070bb0 0x7feba81160b0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7feb98000c00 tx=0x7feb98009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:14:35.979 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:14:35.978+0000 7feb9e7fc700 1 -- 172.21.15.162:0/1919518601 <== mon.0 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7feb9800e040 con 0x7feba8070bb0 2024-04-27T04:14:35.979 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:14:35.978+0000 7feb9e7fc700 1 -- 172.21.15.162:0/1919518601 <== mon.0 v2:172.21.15.162:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7feb9800bc10 con 0x7feba8070bb0 2024-04-27T04:14:35.979 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:14:35.978+0000 7feb9e7fc700 1 -- 172.21.15.162:0/1919518601 <== mon.0 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7feb9801c630 con 0x7feba8070bb0 2024-04-27T04:14:35.979 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:14:35.978+0000 7febafc05700 1 -- 172.21.15.162:0/1919518601 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feba810ff60 con 0x7feba8070bb0 2024-04-27T04:14:35.979 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:14:35.978+0000 7febafc05700 1 -- 172.21.15.162:0/1919518601 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feba81104a0 con 0x7feba8070bb0 2024-04-27T04:14:35.980 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:14:35.979+0000 7febafc05700 1 -- 172.21.15.162:0/1919518601 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feba80719e0 con 0x7feba8070bb0 2024-04-27T04:14:35.980 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:14:35.979+0000 7feb9e7fc700 1 -- 172.21.15.162:0/1919518601 <== mon.0 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 38) v1 ==== 92849+0+0 (secure 0 0 0) 0x7feb9800bd80 con 0x7feba8070bb0 2024-04-27T04:14:35.980 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:14:35.980+0000 7feb9e7fc700 1 --2- 172.21.15.162:0/1919518601 >> [v2:172.21.15.162:6800/1396866108,v1:172.21.15.162:6801/1396866108] conn(0x7feb9406ed30 0x7feb940711e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:14:35.980 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:14:35.980+0000 7feb9e7fc700 1 -- 172.21.15.162:0/1919518601 <== mon.0 v2:172.21.15.162:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6410+0+0 (secure 0 0 0) 0x7feb9808fdd0 con 0x7feba8070bb0 2024-04-27T04:14:35.980 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:14:35.980+0000 7febad9a1700 1 --2- 172.21.15.162:0/1919518601 >> [v2:172.21.15.162:6800/1396866108,v1:172.21.15.162:6801/1396866108] conn(0x7feb9406ed30 0x7feb940711e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:14:35.981 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:14:35.980+0000 7febad9a1700 1 --2- 172.21.15.162:0/1919518601 >> [v2:172.21.15.162:6800/1396866108,v1:172.21.15.162:6801/1396866108] conn(0x7feb9406ed30 0x7feb940711e0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7feba4009ce0 tx=0x7feba4009f70 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:14:35.983 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:14:35.985+0000 7feb9e7fc700 1 -- 172.21.15.162:0/1919518601 <== mon.0 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7feb9805c560 con 0x7feba8070bb0 2024-04-27T04:14:36.200 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:14:36.202+0000 7febafc05700 1 -- 172.21.15.162:0/1919518601 --> [v2:172.21.15.162:6800/1396866108,v1:172.21.15.162:6801/1396866108] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7feba810c170 con 0x7feb9406ed30 2024-04-27T04:14:36.201 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:14:36.203+0000 7feb9e7fc700 1 -- 172.21.15.162:0/1919518601 <== mgr.34104 v2:172.21.15.162:6800/1396866108 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+368 (secure 0 0 0) 0x7feba810c170 con 0x7feb9406ed30 2024-04-27T04:14:36.204 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:14:36.205+0000 7febafc05700 1 -- 172.21.15.162:0/1919518601 >> [v2:172.21.15.162:6800/1396866108,v1:172.21.15.162:6801/1396866108] conn(0x7feb9406ed30 msgr2=0x7feb940711e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:14:36.204 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:14:36.205+0000 7febafc05700 1 --2- 172.21.15.162:0/1919518601 >> [v2:172.21.15.162:6800/1396866108,v1:172.21.15.162:6801/1396866108] conn(0x7feb9406ed30 0x7feb940711e0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7feba4009ce0 tx=0x7feba4009f70 comp rx=0 tx=0).stop 2024-04-27T04:14:36.204 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:14:36.205+0000 7febafc05700 1 -- 172.21.15.162:0/1919518601 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7feba8070bb0 msgr2=0x7feba81160b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:14:36.204 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:14:36.205+0000 7febafc05700 1 --2- 172.21.15.162:0/1919518601 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7feba8070bb0 0x7feba81160b0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7feb98000c00 tx=0x7feb98009e30 comp rx=0 tx=0).stop 2024-04-27T04:14:36.204 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:14:36.205+0000 7febafc05700 1 -- 172.21.15.162:0/1919518601 shutdown_connections 2024-04-27T04:14:36.204 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:14:36.205+0000 7febafc05700 1 --2- 172.21.15.162:0/1919518601 >> [v2:172.21.15.162:6800/1396866108,v1:172.21.15.162:6801/1396866108] conn(0x7feb9406ed30 0x7feb940711e0 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:14:36.204 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:14:36.205+0000 7febafc05700 1 --2- 172.21.15.162:0/1919518601 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7feba80702a0 0x7feba8115b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:14:36.204 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:14:36.205+0000 7febafc05700 1 --2- 172.21.15.162:0/1919518601 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7feba8070bb0 0x7feba81160b0 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:14:36.204 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:14:36.205+0000 7febafc05700 1 -- 172.21.15.162:0/1919518601 >> 172.21.15.162:0/1919518601 conn(0x7feba80f8990 msgr2=0x7feba80f9d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:14:36.204 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:14:36.205+0000 7febafc05700 1 -- 172.21.15.162:0/1919518601 shutdown_connections 2024-04-27T04:14:36.204 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:14:36.205+0000 7febafc05700 1 -- 172.21.15.162:0/1919518601 wait complete. 2024-04-27T04:14:36.213 INFO:teuthology.orchestra.run.smithi162.stdout:true 2024-04-27T04:14:36.306 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:14:36.308+0000 7fa2c63e2700 1 -- 172.21.15.162:0/1755209221 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa2c0102730 msgr2=0x7fa2c00fbda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:14:36.307 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:14:36.308+0000 7fa2c63e2700 1 --2- 172.21.15.162:0/1755209221 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa2c0102730 0x7fa2c00fbda0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fa2b4004940 tx=0x7fa2b4009e30 comp rx=0 tx=0).stop 2024-04-27T04:14:36.307 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:14:36.308+0000 7fa2c63e2700 1 -- 172.21.15.162:0/1755209221 shutdown_connections 2024-04-27T04:14:36.307 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:14:36.308+0000 7fa2c63e2700 1 --2- 172.21.15.162:0/1755209221 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa2c0102730 0x7fa2c00fbda0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:14:36.307 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:14:36.308+0000 7fa2c63e2700 1 --2- 172.21.15.162:0/1755209221 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa2c0101d90 0x7fa2c0102160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:14:36.307 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:14:36.308+0000 7fa2c63e2700 1 -- 172.21.15.162:0/1755209221 >> 172.21.15.162:0/1755209221 conn(0x7fa2c00f7940 msgr2=0x7fa2c00f9d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:14:36.307 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:14:36.308+0000 7fa2c63e2700 1 -- 172.21.15.162:0/1755209221 shutdown_connections 2024-04-27T04:14:36.307 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:14:36.308+0000 7fa2c63e2700 1 -- 172.21.15.162:0/1755209221 wait complete. 2024-04-27T04:14:36.307 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:14:36.308+0000 7fa2c63e2700 1 Processor -- start 2024-04-27T04:14:36.309 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:14:36.309+0000 7fa2c63e2700 1 -- start start 2024-04-27T04:14:36.309 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:14:36.309+0000 7fa2c63e2700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa2c0101d90 0x7fa2c0115b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:14:36.309 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:14:36.309+0000 7fa2c63e2700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa2c0102730 0x7fa2c0116070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:14:36.309 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:14:36.309+0000 7fa2c63e2700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2c01166c0 con 0x7fa2c0101d90 2024-04-27T04:14:36.310 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:14:36.309+0000 7fa2c63e2700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2c010fce0 con 0x7fa2c0102730 2024-04-27T04:14:36.310 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:14:36.309+0000 7fa2bffff700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa2c0101d90 0x7fa2c0115b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:14:36.310 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:14:36.309+0000 7fa2bffff700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa2c0101d90 0x7fa2c0115b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:58190/0 (socket says 172.21.15.162:58190) 2024-04-27T04:14:36.310 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:14:36.309+0000 7fa2bffff700 1 -- 172.21.15.162:0/949590728 learned_addr learned my addr 172.21.15.162:0/949590728 (peer_addr_for_me v2:172.21.15.162:0/0) 2024-04-27T04:14:36.310 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:14:36.309+0000 7fa2bf7fe700 1 --2- 172.21.15.162:0/949590728 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa2c0102730 0x7fa2c0116070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:14:36.310 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:14:36.309+0000 7fa2bf7fe700 1 -- 172.21.15.162:0/949590728 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa2c0102730 msgr2=0x7fa2c0116070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 13 2024-04-27T04:14:36.310 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:14:36.309+0000 7fa2bf7fe700 1 -- 172.21.15.162:0/949590728 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa2c0102730 msgr2=0x7fa2c0116070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-04-27T04:14:36.310 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:14:36.309+0000 7fa2bf7fe700 1 --2- 172.21.15.162:0/949590728 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa2c0102730 0x7fa2c0116070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2024-04-27T04:14:36.310 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:14:36.309+0000 7fa2bf7fe700 1 --2- 172.21.15.162:0/949590728 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa2c0102730 0x7fa2c0116070 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-27T04:14:36.310 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:14:36.310+0000 7fa2bffff700 1 -- 172.21.15.162:0/949590728 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa2c0102730 msgr2=0x7fa2c0116070 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-27T04:14:36.310 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:14:36.310+0000 7fa2bffff700 1 --2- 172.21.15.162:0/949590728 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa2c0102730 0x7fa2c0116070 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:14:36.310 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:14:36.310+0000 7fa2bffff700 1 -- 172.21.15.162:0/949590728 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2b4005040 con 0x7fa2c0101d90 2024-04-27T04:14:36.310 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:14:36.310+0000 7fa2bffff700 1 --2- 172.21.15.162:0/949590728 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa2c0101d90 0x7fa2c0115b30 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fa2b0002a50 tx=0x7fa2b00046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:14:36.311 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:14:36.310+0000 7fa2bcff9700 1 -- 172.21.15.162:0/949590728 <== mon.0 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa2b0007cc0 con 0x7fa2c0101d90 2024-04-27T04:14:36.311 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:14:36.310+0000 7fa2bcff9700 1 -- 172.21.15.162:0/949590728 <== mon.0 v2:172.21.15.162:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fa2b000f040 con 0x7fa2c0101d90 2024-04-27T04:14:36.311 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:14:36.310+0000 7fa2bcff9700 1 -- 172.21.15.162:0/949590728 <== mon.0 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa2b000db70 con 0x7fa2c0101d90 2024-04-27T04:14:36.311 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:14:36.310+0000 7fa2c63e2700 1 -- 172.21.15.162:0/949590728 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa2c010ffc0 con 0x7fa2c0101d90 2024-04-27T04:14:36.311 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:14:36.310+0000 7fa2c63e2700 1 -- 172.21.15.162:0/949590728 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa2c0110580 con 0x7fa2c0101d90 2024-04-27T04:14:36.311 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:14:36.311+0000 7fa2c63e2700 1 -- 172.21.15.162:0/949590728 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa2c00fd4e0 con 0x7fa2c0101d90 2024-04-27T04:14:36.312 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:14:36.312+0000 7fa2bcff9700 1 -- 172.21.15.162:0/949590728 <== mon.0 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 38) v1 ==== 92849+0+0 (secure 0 0 0) 0x7fa2b000dd10 con 0x7fa2c0101d90 2024-04-27T04:14:36.312 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:14:36.312+0000 7fa2bcff9700 1 --2- 172.21.15.162:0/949590728 >> [v2:172.21.15.162:6800/1396866108,v1:172.21.15.162:6801/1396866108] conn(0x7fa2a806ed30 0x7fa2a80711e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:14:36.313 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:14:36.312+0000 7fa2bcff9700 1 -- 172.21.15.162:0/949590728 <== mon.0 v2:172.21.15.162:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6410+0+0 (secure 0 0 0) 0x7fa2b008e0e0 con 0x7fa2c0101d90 2024-04-27T04:14:36.313 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:14:36.312+0000 7fa2bf7fe700 1 --2- 172.21.15.162:0/949590728 >> [v2:172.21.15.162:6800/1396866108,v1:172.21.15.162:6801/1396866108] conn(0x7fa2a806ed30 0x7fa2a80711e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:14:36.313 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:14:36.313+0000 7fa2bf7fe700 1 --2- 172.21.15.162:0/949590728 >> [v2:172.21.15.162:6800/1396866108,v1:172.21.15.162:6801/1396866108] conn(0x7fa2a806ed30 0x7fa2a80711e0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fa2b400b3b0 tx=0x7fa2b4009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:14:36.315 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:14:36.317+0000 7fa2bcff9700 1 -- 172.21.15.162:0/949590728 <== mon.0 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7fa2b0056d40 con 0x7fa2c0101d90 2024-04-27T04:14:36.532 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:14:36.534+0000 7fa2c63e2700 1 -- 172.21.15.162:0/949590728 --> [v2:172.21.15.162:6800/1396866108,v1:172.21.15.162:6801/1396866108] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fa2c0111230 con 0x7fa2a806ed30 2024-04-27T04:14:36.533 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:14:36.535+0000 7fa2bcff9700 1 -- 172.21.15.162:0/949590728 <== mgr.34104 v2:172.21.15.162:6800/1396866108 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+368 (secure 0 0 0) 0x7fa2c0111230 con 0x7fa2a806ed30 2024-04-27T04:14:36.536 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:14:36.537+0000 7fa2c63e2700 1 -- 172.21.15.162:0/949590728 >> [v2:172.21.15.162:6800/1396866108,v1:172.21.15.162:6801/1396866108] conn(0x7fa2a806ed30 msgr2=0x7fa2a80711e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:14:36.536 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:14:36.537+0000 7fa2c63e2700 1 --2- 172.21.15.162:0/949590728 >> [v2:172.21.15.162:6800/1396866108,v1:172.21.15.162:6801/1396866108] conn(0x7fa2a806ed30 0x7fa2a80711e0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fa2b400b3b0 tx=0x7fa2b4009e30 comp rx=0 tx=0).stop 2024-04-27T04:14:36.536 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:14:36.537+0000 7fa2c63e2700 1 -- 172.21.15.162:0/949590728 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa2c0101d90 msgr2=0x7fa2c0115b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:14:36.536 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:14:36.537+0000 7fa2c63e2700 1 --2- 172.21.15.162:0/949590728 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa2c0101d90 0x7fa2c0115b30 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fa2b0002a50 tx=0x7fa2b00046b0 comp rx=0 tx=0).stop 2024-04-27T04:14:36.536 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:14:36.537+0000 7fa2c63e2700 1 -- 172.21.15.162:0/949590728 shutdown_connections 2024-04-27T04:14:36.536 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:14:36.537+0000 7fa2c63e2700 1 --2- 172.21.15.162:0/949590728 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa2c0101d90 0x7fa2c0115b30 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:14:36.536 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:14:36.537+0000 7fa2c63e2700 1 --2- 172.21.15.162:0/949590728 >> [v2:172.21.15.162:6800/1396866108,v1:172.21.15.162:6801/1396866108] conn(0x7fa2a806ed30 0x7fa2a80711e0 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:14:36.536 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:14:36.537+0000 7fa2c63e2700 1 --2- 172.21.15.162:0/949590728 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa2c0102730 0x7fa2c0116070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:14:36.537 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:14:36.537+0000 7fa2c63e2700 1 -- 172.21.15.162:0/949590728 >> 172.21.15.162:0/949590728 conn(0x7fa2c00f7940 msgr2=0x7fa2c0066560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:14:36.537 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:14:36.538+0000 7fa2c63e2700 1 -- 172.21.15.162:0/949590728 shutdown_connections 2024-04-27T04:14:36.537 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:14:36.538+0000 7fa2c63e2700 1 -- 172.21.15.162:0/949590728 wait complete. 2024-04-27T04:14:36.638 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:14:36.638+0000 7f72eb7db700 1 -- 172.21.15.162:0/3283946487 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f72e40fbd40 msgr2=0x7f72e40fc1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:14:36.638 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:14:36.638+0000 7f72eb7db700 1 --2- 172.21.15.162:0/3283946487 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f72e40fbd40 0x7f72e40fc1b0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f72e0004940 tx=0x7f72e0009e30 comp rx=0 tx=0).stop 2024-04-27T04:14:36.639 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:14:36.638+0000 7f72eb7db700 1 -- 172.21.15.162:0/3283946487 shutdown_connections 2024-04-27T04:14:36.639 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:14:36.638+0000 7f72eb7db700 1 --2- 172.21.15.162:0/3283946487 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f72e40fbd40 0x7f72e40fc1b0 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:14:36.639 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:14:36.638+0000 7f72eb7db700 1 --2- 172.21.15.162:0/3283946487 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f72e4101d90 0x7f72e4102160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:14:36.639 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:14:36.638+0000 7f72eb7db700 1 -- 172.21.15.162:0/3283946487 >> 172.21.15.162:0/3283946487 conn(0x7f72e4071d00 msgr2=0x7f72e4072100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:14:36.639 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:14:36.638+0000 7f72eb7db700 1 -- 172.21.15.162:0/3283946487 shutdown_connections 2024-04-27T04:14:36.639 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:14:36.638+0000 7f72eb7db700 1 -- 172.21.15.162:0/3283946487 wait complete. 2024-04-27T04:14:36.639 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:14:36.639+0000 7f72eb7db700 1 Processor -- start 2024-04-27T04:14:36.639 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:14:36.639+0000 7f72eb7db700 1 -- start start 2024-04-27T04:14:36.639 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:14:36.639+0000 7f72eb7db700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f72e40fbd40 0x7f72e4113bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:14:36.639 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:14:36.639+0000 7f72eb7db700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f72e4101d90 0x7f72e41140f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:14:36.639 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:14:36.639+0000 7f72eb7db700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72e4114740 con 0x7f72e4101d90 2024-04-27T04:14:36.640 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:14:36.639+0000 7f72eb7db700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72e410db30 con 0x7f72e40fbd40 2024-04-27T04:14:36.640 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:14:36.639+0000 7f72e8d76700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f72e4101d90 0x7f72e41140f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:14:36.640 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:14:36.639+0000 7f72e8d76700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f72e4101d90 0x7f72e41140f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:58204/0 (socket says 172.21.15.162:58204) 2024-04-27T04:14:36.640 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:14:36.639+0000 7f72e8d76700 1 -- 172.21.15.162:0/2869123881 learned_addr learned my addr 172.21.15.162:0/2869123881 (peer_addr_for_me v2:172.21.15.162:0/0) 2024-04-27T04:14:36.640 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:14:36.639+0000 7f72e9577700 1 --2- 172.21.15.162:0/2869123881 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f72e40fbd40 0x7f72e4113bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:14:36.640 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:14:36.639+0000 7f72e9577700 1 -- 172.21.15.162:0/2869123881 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f72e40fbd40 msgr2=0x7f72e4113bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 12 2024-04-27T04:14:36.640 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:14:36.639+0000 7f72e9577700 1 -- 172.21.15.162:0/2869123881 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f72e40fbd40 msgr2=0x7f72e4113bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-04-27T04:14:36.640 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:14:36.639+0000 7f72e9577700 1 --2- 172.21.15.162:0/2869123881 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f72e40fbd40 0x7f72e4113bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2024-04-27T04:14:36.640 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:14:36.639+0000 7f72e9577700 1 --2- 172.21.15.162:0/2869123881 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f72e40fbd40 0x7f72e4113bb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-27T04:14:36.640 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:14:36.639+0000 7f72e8d76700 1 -- 172.21.15.162:0/2869123881 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f72e40fbd40 msgr2=0x7f72e4113bb0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-27T04:14:36.641 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:14:36.639+0000 7f72e8d76700 1 --2- 172.21.15.162:0/2869123881 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f72e40fbd40 0x7f72e4113bb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:14:36.641 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:14:36.639+0000 7f72e8d76700 1 -- 172.21.15.162:0/2869123881 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f72e0005040 con 0x7f72e4101d90 2024-04-27T04:14:36.641 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:14:36.639+0000 7f72e8d76700 1 --2- 172.21.15.162:0/2869123881 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f72e4101d90 0x7f72e41140f0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f72e0009150 tx=0x7f72e0009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:14:36.641 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:14:36.640+0000 7f72d9ffb700 1 -- 172.21.15.162:0/2869123881 <== mon.0 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f72e000e040 con 0x7f72e4101d90 2024-04-27T04:14:36.641 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:14:36.640+0000 7f72d9ffb700 1 -- 172.21.15.162:0/2869123881 <== mon.0 v2:172.21.15.162:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f72e000bbc0 con 0x7f72e4101d90 2024-04-27T04:14:36.641 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:14:36.640+0000 7f72eb7db700 1 -- 172.21.15.162:0/2869123881 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f72e410ddb0 con 0x7f72e4101d90 2024-04-27T04:14:36.641 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:14:36.640+0000 7f72d9ffb700 1 -- 172.21.15.162:0/2869123881 <== mon.0 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f72e001c630 con 0x7f72e4101d90 2024-04-27T04:14:36.641 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:14:36.640+0000 7f72eb7db700 1 -- 172.21.15.162:0/2869123881 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f72e410e370 con 0x7f72e4101d90 2024-04-27T04:14:36.642 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:14:36.641+0000 7f72eb7db700 1 -- 172.21.15.162:0/2869123881 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f72e40fd430 con 0x7f72e4101d90 2024-04-27T04:14:36.642 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:14:36.641+0000 7f72d9ffb700 1 -- 172.21.15.162:0/2869123881 <== mon.0 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 38) v1 ==== 92849+0+0 (secure 0 0 0) 0x7f72e000bd30 con 0x7f72e4101d90 2024-04-27T04:14:36.642 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:14:36.642+0000 7f72d9ffb700 1 --2- 172.21.15.162:0/2869123881 >> [v2:172.21.15.162:6800/1396866108,v1:172.21.15.162:6801/1396866108] conn(0x7f72d006ed20 0x7f72d00711d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:14:36.642 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:14:36.642+0000 7f72d9ffb700 1 -- 172.21.15.162:0/2869123881 <== mon.0 v2:172.21.15.162:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6410+0+0 (secure 0 0 0) 0x7f72e008fcb0 con 0x7f72e4101d90 2024-04-27T04:14:36.642 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:14:36.642+0000 7f72e9577700 1 --2- 172.21.15.162:0/2869123881 >> [v2:172.21.15.162:6800/1396866108,v1:172.21.15.162:6801/1396866108] conn(0x7f72d006ed20 0x7f72d00711d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:14:36.642 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:14:36.642+0000 7f72e9577700 1 --2- 172.21.15.162:0/2869123881 >> [v2:172.21.15.162:6800/1396866108,v1:172.21.15.162:6801/1396866108] conn(0x7f72d006ed20 0x7f72d00711d0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f72d4009e50 tx=0x7f72d4009f10 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:14:36.645 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:14:36.647+0000 7f72d9ffb700 1 -- 172.21.15.162:0/2869123881 <== mon.0 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7f72e005c4e0 con 0x7f72e4101d90 2024-04-27T04:14:36.850 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:14:36 smithi181 ceph-mon[81134]: pgmap v12: 65 pgs: 65 active+clean; 208 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-04-27T04:14:36.850 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:14:36 smithi181 ceph-mon[81134]: mgrmap e38: smithi162.oennfn(active, since 18s), standbys: smithi181.vtwuhz 2024-04-27T04:14:36.850 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:14:36 smithi181 ceph-mon[81134]: overall HEALTH_OK 2024-04-27T04:14:36.850 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:14:36 smithi181 ceph-mon[81134]: mon.smithi181 calling monitor election 2024-04-27T04:14:36.851 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:14:36 smithi181 ceph-mon[81134]: mon.smithi162 calling monitor election 2024-04-27T04:14:36.851 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:14:36 smithi181 ceph-mon[81134]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "mon metadata", "id": "smithi162"}]: dispatch 2024-04-27T04:14:36.851 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:14:36 smithi181 ceph-mon[81134]: mon.smithi162 is new leader, mons smithi162,smithi181 in quorum (ranks 0,1) 2024-04-27T04:14:36.851 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:14:36 smithi181 ceph-mon[81134]: monmap epoch 3 2024-04-27T04:14:36.851 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:14:36 smithi181 ceph-mon[81134]: fsid 7f9750d6-044a-11ef-bc93-c7b262605968 2024-04-27T04:14:36.851 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:14:36 smithi181 ceph-mon[81134]: last_changed 2024-04-27T04:14:35.692174+0000 2024-04-27T04:14:36.851 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:14:36 smithi181 ceph-mon[81134]: created 2024-04-27T04:00:03.626312+0000 2024-04-27T04:14:36.851 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:14:36 smithi181 ceph-mon[81134]: min_mon_release 19 (squid) 2024-04-27T04:14:36.851 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:14:36 smithi181 ceph-mon[81134]: election_strategy: 1 2024-04-27T04:14:36.851 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:14:36 smithi181 ceph-mon[81134]: 0: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.smithi162 2024-04-27T04:14:36.851 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:14:36 smithi181 ceph-mon[81134]: 1: [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] mon.smithi181 2024-04-27T04:14:36.851 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:14:36 smithi181 ceph-mon[81134]: fsmap cephfs:1 {0=cephfs.smithi181.dzomgh=up:active} 1 up:standby-replay 2 up:standby 2024-04-27T04:14:36.851 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:14:36 smithi181 ceph-mon[81134]: osdmap e49: 6 total, 6 up, 6 in 2024-04-27T04:14:36.851 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:14:36 smithi181 ceph-mon[81134]: mgrmap e38: smithi162.oennfn(active, since 18s), standbys: smithi181.vtwuhz 2024-04-27T04:14:36.852 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:14:36 smithi181 ceph-mon[81134]: overall HEALTH_OK 2024-04-27T04:14:36.852 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:14:36 smithi181 ceph-mon[81134]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "mon metadata", "id": "smithi181"}]: dispatch 2024-04-27T04:14:36.852 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:14:36 smithi181 ceph-mon[81134]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' 2024-04-27T04:14:36.852 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:14:36 smithi181 ceph-mon[81134]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' 2024-04-27T04:14:36.852 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:14:36 smithi181 ceph-mon[81134]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T04:14:36.856 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:14:36.858+0000 7f72eb7db700 1 -- 172.21.15.162:0/2869123881 --> [v2:172.21.15.162:6800/1396866108,v1:172.21.15.162:6801/1396866108] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f72e4108df0 con 0x7f72d006ed20 2024-04-27T04:14:36.867 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:14:36.868+0000 7f72d9ffb700 1 -- 172.21.15.162:0/2869123881 <== mgr.34104 v2:172.21.15.162:6800/1396866108 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3520 (secure 0 0 0) 0x7f72e4108df0 con 0x7f72d006ed20 2024-04-27T04:14:36.867 INFO:teuthology.orchestra.run.smithi162.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-27T04:14:36.867 INFO:teuthology.orchestra.run.smithi162.stdout:alertmanager.smithi162 smithi162 *:9093,9094 running (105s) 13s ago 12m 16.9M - 0.25.0 c8568f914cd2 40e334659017 2024-04-27T04:14:36.867 INFO:teuthology.orchestra.run.smithi162.stdout:crash.smithi162 smithi162 running (13m) 13s ago 13m 6660k - 17.2.7-907-g2be3efa9 1c5834633715 228c98df4289 2024-04-27T04:14:36.867 INFO:teuthology.orchestra.run.smithi162.stdout:crash.smithi181 smithi181 running (12m) 15s ago 12m 6665k - 17.2.7-907-g2be3efa9 1c5834633715 c8e553e3df2b 2024-04-27T04:14:36.868 INFO:teuthology.orchestra.run.smithi162.stdout:grafana.smithi162 smithi162 *:3000 running (82s) 13s ago 12m 75.4M - 9.4.12 2bacad6d85d8 3d78c478f03f 2024-04-27T04:14:36.868 INFO:teuthology.orchestra.run.smithi162.stdout:mds.cephfs.smithi162.gzlyjj smithi162 running (7m) 13s ago 7m 13.4M - 17.2.7-907-g2be3efa9 1c5834633715 27e5b030dd32 2024-04-27T04:14:36.868 INFO:teuthology.orchestra.run.smithi162.stdout:mds.cephfs.smithi162.xcnapk smithi162 running (7m) 13s ago 7m 100M - 17.2.7-907-g2be3efa9 1c5834633715 38f9030df029 2024-04-27T04:14:36.868 INFO:teuthology.orchestra.run.smithi162.stdout:mds.cephfs.smithi181.bkkcdm smithi181 running (7m) 15s ago 7m 14.9M - 17.2.7-907-g2be3efa9 1c5834633715 af55afca1263 2024-04-27T04:14:36.868 INFO:teuthology.orchestra.run.smithi162.stdout:mds.cephfs.smithi181.dzomgh smithi181 running (7m) 15s ago 7m 145M - 17.2.7-907-g2be3efa9 1c5834633715 6733296597f3 2024-04-27T04:14:36.868 INFO:teuthology.orchestra.run.smithi162.stdout:mgr.smithi162.oennfn smithi162 *:8443,9283,8765 running (3m) 13s ago 14m 475M - 19.0.0-3253-g4e213a58 67e414ff4bc4 58a748d3637b 2024-04-27T04:14:36.868 INFO:teuthology.orchestra.run.smithi162.stdout:mgr.smithi181.vtwuhz smithi181 *:8443,9283,8765 running (2m) 15s ago 12m 447M - 19.0.0-3253-g4e213a58 67e414ff4bc4 813584297b1d 2024-04-27T04:14:36.868 INFO:teuthology.orchestra.run.smithi162.stdout:mon.smithi162 smithi162 running (27s) 13s ago 14m 27.1M 2048M 19.0.0-3253-g4e213a58 67e414ff4bc4 375adaf4468f 2024-04-27T04:14:36.868 INFO:teuthology.orchestra.run.smithi162.stdout:mon.smithi181 smithi181 starting - - - 2048M 2024-04-27T04:14:36.868 INFO:teuthology.orchestra.run.smithi162.stdout:node-exporter.smithi162 smithi162 *:9100 running (2m) 13s ago 13m 15.6M - 1.5.0 0da6a335fe13 5c4c5ea9c1d7 2024-04-27T04:14:36.868 INFO:teuthology.orchestra.run.smithi162.stdout:node-exporter.smithi181 smithi181 *:9100 running (2m) 15s ago 12m 15.5M - 1.5.0 0da6a335fe13 9067f14c0519 2024-04-27T04:14:36.869 INFO:teuthology.orchestra.run.smithi162.stdout:osd.0 smithi162 running (11m) 13s ago 11m 231M 1557M 17.2.7-907-g2be3efa9 1c5834633715 4a9df3aa938d 2024-04-27T04:14:36.869 INFO:teuthology.orchestra.run.smithi162.stdout:osd.1 smithi162 running (10m) 13s ago 10m 246M 1557M 17.2.7-907-g2be3efa9 1c5834633715 aefb464c46e8 2024-04-27T04:14:36.869 INFO:teuthology.orchestra.run.smithi162.stdout:osd.2 smithi162 running (10m) 13s ago 10m 205M 1557M 17.2.7-907-g2be3efa9 1c5834633715 70f16d86d38e 2024-04-27T04:14:36.869 INFO:teuthology.orchestra.run.smithi162.stdout:osd.3 smithi181 running (9m) 15s ago 9m 277M 2581M 17.2.7-907-g2be3efa9 1c5834633715 030d829be54e 2024-04-27T04:14:36.869 INFO:teuthology.orchestra.run.smithi162.stdout:osd.4 smithi181 running (9m) 15s ago 9m 246M 2581M 17.2.7-907-g2be3efa9 1c5834633715 512afbee7cf7 2024-04-27T04:14:36.869 INFO:teuthology.orchestra.run.smithi162.stdout:osd.5 smithi181 running (9m) 15s ago 9m 224M 2581M 17.2.7-907-g2be3efa9 1c5834633715 f47a1daac1ea 2024-04-27T04:14:36.869 INFO:teuthology.orchestra.run.smithi162.stdout:prometheus.smithi162 smithi162 *:9095 running (2m) 13s ago 12m 56.4M - 2.43.0 a07b618ecd1d f30657a43324 2024-04-27T04:14:36.870 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:14:36.871+0000 7f72eb7db700 1 -- 172.21.15.162:0/2869123881 >> [v2:172.21.15.162:6800/1396866108,v1:172.21.15.162:6801/1396866108] conn(0x7f72d006ed20 msgr2=0x7f72d00711d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:14:36.870 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:14:36.871+0000 7f72eb7db700 1 --2- 172.21.15.162:0/2869123881 >> [v2:172.21.15.162:6800/1396866108,v1:172.21.15.162:6801/1396866108] conn(0x7f72d006ed20 0x7f72d00711d0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f72d4009e50 tx=0x7f72d4009f10 comp rx=0 tx=0).stop 2024-04-27T04:14:36.870 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:14:36.871+0000 7f72eb7db700 1 -- 172.21.15.162:0/2869123881 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f72e4101d90 msgr2=0x7f72e41140f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:14:36.870 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:14:36.871+0000 7f72eb7db700 1 --2- 172.21.15.162:0/2869123881 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f72e4101d90 0x7f72e41140f0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f72e0009150 tx=0x7f72e0009e30 comp rx=0 tx=0).stop 2024-04-27T04:14:36.871 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:14:36.871+0000 7f72eb7db700 1 -- 172.21.15.162:0/2869123881 shutdown_connections 2024-04-27T04:14:36.871 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:14:36.871+0000 7f72eb7db700 1 --2- 172.21.15.162:0/2869123881 >> [v2:172.21.15.162:6800/1396866108,v1:172.21.15.162:6801/1396866108] conn(0x7f72d006ed20 0x7f72d00711d0 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:14:36.871 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:14:36.871+0000 7f72eb7db700 1 --2- 172.21.15.162:0/2869123881 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f72e40fbd40 0x7f72e4113bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:14:36.871 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:14:36.871+0000 7f72eb7db700 1 --2- 172.21.15.162:0/2869123881 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f72e4101d90 0x7f72e41140f0 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:14:36.871 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:14:36.871+0000 7f72eb7db700 1 -- 172.21.15.162:0/2869123881 >> 172.21.15.162:0/2869123881 conn(0x7f72e4071d00 msgr2=0x7f72e40655a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:14:36.871 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:14:36.871+0000 7f72eb7db700 1 -- 172.21.15.162:0/2869123881 shutdown_connections 2024-04-27T04:14:36.871 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:14:36.871+0000 7f72eb7db700 1 -- 172.21.15.162:0/2869123881 wait complete. 2024-04-27T04:14:36.886 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:14:36 smithi162 ceph-mon[110641]: pgmap v12: 65 pgs: 65 active+clean; 208 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-04-27T04:14:36.887 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:14:36 smithi162 ceph-mon[110641]: mgrmap e38: smithi162.oennfn(active, since 18s), standbys: smithi181.vtwuhz 2024-04-27T04:14:36.887 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:14:36 smithi162 ceph-mon[110641]: overall HEALTH_OK 2024-04-27T04:14:36.887 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:14:36 smithi162 ceph-mon[110641]: mon.smithi181 calling monitor election 2024-04-27T04:14:36.887 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:14:36 smithi162 ceph-mon[110641]: mon.smithi162 calling monitor election 2024-04-27T04:14:36.888 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:14:36 smithi162 ceph-mon[110641]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "mon metadata", "id": "smithi162"}]: dispatch 2024-04-27T04:14:36.888 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:14:36 smithi162 ceph-mon[110641]: mon.smithi162 is new leader, mons smithi162,smithi181 in quorum (ranks 0,1) 2024-04-27T04:14:36.888 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:14:36 smithi162 ceph-mon[110641]: monmap epoch 3 2024-04-27T04:14:36.888 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:14:36 smithi162 ceph-mon[110641]: fsid 7f9750d6-044a-11ef-bc93-c7b262605968 2024-04-27T04:14:36.888 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:14:36 smithi162 ceph-mon[110641]: last_changed 2024-04-27T04:14:35.692174+0000 2024-04-27T04:14:36.888 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:14:36 smithi162 ceph-mon[110641]: created 2024-04-27T04:00:03.626312+0000 2024-04-27T04:14:36.888 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:14:36 smithi162 ceph-mon[110641]: min_mon_release 19 (squid) 2024-04-27T04:14:36.888 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:14:36 smithi162 ceph-mon[110641]: election_strategy: 1 2024-04-27T04:14:36.888 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:14:36 smithi162 ceph-mon[110641]: 0: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.smithi162 2024-04-27T04:14:36.888 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:14:36 smithi162 ceph-mon[110641]: 1: [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] mon.smithi181 2024-04-27T04:14:36.888 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:14:36 smithi162 ceph-mon[110641]: fsmap cephfs:1 {0=cephfs.smithi181.dzomgh=up:active} 1 up:standby-replay 2 up:standby 2024-04-27T04:14:36.888 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:14:36 smithi162 ceph-mon[110641]: osdmap e49: 6 total, 6 up, 6 in 2024-04-27T04:14:36.889 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:14:36 smithi162 ceph-mon[110641]: mgrmap e38: smithi162.oennfn(active, since 18s), standbys: smithi181.vtwuhz 2024-04-27T04:14:36.889 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:14:36 smithi162 ceph-mon[110641]: overall HEALTH_OK 2024-04-27T04:14:36.889 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:14:36 smithi162 ceph-mon[110641]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "mon metadata", "id": "smithi181"}]: dispatch 2024-04-27T04:14:36.889 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:14:36 smithi162 ceph-mon[110641]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' 2024-04-27T04:14:36.889 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:14:36 smithi162 ceph-mon[110641]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' 2024-04-27T04:14:36.889 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:14:36 smithi162 ceph-mon[110641]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T04:14:36.970 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:14:36.970+0000 7f17c605d700 1 -- 172.21.15.162:0/142687339 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f17c0102de0 msgr2=0x7f17c01031b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:14:36.970 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:14:36.970+0000 7f17c605d700 1 --2- 172.21.15.162:0/142687339 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f17c0102de0 0x7f17c01031b0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f17b0004940 tx=0x7f17b0009e30 comp rx=0 tx=0).stop 2024-04-27T04:14:36.970 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:14:36.970+0000 7f17c605d700 1 -- 172.21.15.162:0/142687339 shutdown_connections 2024-04-27T04:14:36.970 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:14:36.970+0000 7f17c605d700 1 --2- 172.21.15.162:0/142687339 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f17c0103780 0x7f17c00fcdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:14:36.970 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:14:36.970+0000 7f17c605d700 1 --2- 172.21.15.162:0/142687339 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f17c0102de0 0x7f17c01031b0 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:14:36.971 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:14:36.970+0000 7f17c605d700 1 -- 172.21.15.162:0/142687339 >> 172.21.15.162:0/142687339 conn(0x7f17c00f8990 msgr2=0x7f17c00fada0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:14:36.971 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:14:36.970+0000 7f17c605d700 1 -- 172.21.15.162:0/142687339 shutdown_connections 2024-04-27T04:14:36.971 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:14:36.970+0000 7f17c605d700 1 -- 172.21.15.162:0/142687339 wait complete. 2024-04-27T04:14:36.971 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:14:36.970+0000 7f17c605d700 1 Processor -- start 2024-04-27T04:14:36.971 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:14:36.971+0000 7f17c605d700 1 -- start start 2024-04-27T04:14:36.971 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:14:36.971+0000 7f17c605d700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f17c0102de0 0x7f17c0113990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:14:36.971 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:14:36.971+0000 7f17c605d700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f17c0103780 0x7f17c0113ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:14:36.971 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:14:36.971+0000 7f17c605d700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17c01145b0 con 0x7f17c0103780 2024-04-27T04:14:36.971 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:14:36.971+0000 7f17c605d700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17c010db20 con 0x7f17c0102de0 2024-04-27T04:14:36.971 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:14:36.971+0000 7f17b6ffd700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f17c0103780 0x7f17c0113ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:14:36.971 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:14:36.971+0000 7f17b6ffd700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f17c0103780 0x7f17c0113ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:58234/0 (socket says 172.21.15.162:58234) 2024-04-27T04:14:36.972 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:14:36.971+0000 7f17b6ffd700 1 -- 172.21.15.162:0/2581691340 learned_addr learned my addr 172.21.15.162:0/2581691340 (peer_addr_for_me v2:172.21.15.162:0/0) 2024-04-27T04:14:36.972 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:14:36.971+0000 7f17bf7fe700 1 --2- 172.21.15.162:0/2581691340 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f17c0102de0 0x7f17c0113990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:14:36.972 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:14:36.971+0000 7f17bf7fe700 1 -- 172.21.15.162:0/2581691340 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f17c0102de0 msgr2=0x7f17c0113990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 12 2024-04-27T04:14:36.972 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:14:36.971+0000 7f17bf7fe700 1 -- 172.21.15.162:0/2581691340 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f17c0102de0 msgr2=0x7f17c0113990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-04-27T04:14:36.972 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:14:36.971+0000 7f17bf7fe700 1 --2- 172.21.15.162:0/2581691340 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f17c0102de0 0x7f17c0113990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2024-04-27T04:14:36.972 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:14:36.971+0000 7f17b6ffd700 1 -- 172.21.15.162:0/2581691340 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f17c0102de0 msgr2=0x7f17c0113990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:14:36.972 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:14:36.971+0000 7f17bf7fe700 1 --2- 172.21.15.162:0/2581691340 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f17c0102de0 0x7f17c0113990 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-27T04:14:36.972 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:14:36.971+0000 7f17b6ffd700 1 --2- 172.21.15.162:0/2581691340 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f17c0102de0 0x7f17c0113990 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:14:36.972 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:14:36.971+0000 7f17b6ffd700 1 -- 172.21.15.162:0/2581691340 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f17b0005040 con 0x7f17c0103780 2024-04-27T04:14:36.972 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:14:36.972+0000 7f17b6ffd700 1 --2- 172.21.15.162:0/2581691340 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f17c0103780 0x7f17c0113ed0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f17a800b790 tx=0x7f17a8009220 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:14:36.972 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:14:36.972+0000 7f17bcff9700 1 -- 172.21.15.162:0/2581691340 <== mon.0 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f17a800f6f0 con 0x7f17c0103780 2024-04-27T04:14:36.972 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:14:36.972+0000 7f17c605d700 1 -- 172.21.15.162:0/2581691340 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f17c010de00 con 0x7f17c0103780 2024-04-27T04:14:36.972 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:14:36.972+0000 7f17bcff9700 1 -- 172.21.15.162:0/2581691340 <== mon.0 v2:172.21.15.162:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f17a800fd20 con 0x7f17c0103780 2024-04-27T04:14:36.973 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:14:36.972+0000 7f17bcff9700 1 -- 172.21.15.162:0/2581691340 <== mon.0 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f17a80103a0 con 0x7f17c0103780 2024-04-27T04:14:36.973 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:14:36.972+0000 7f17c605d700 1 -- 172.21.15.162:0/2581691340 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f17c010e3c0 con 0x7f17c0103780 2024-04-27T04:14:36.973 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:14:36.973+0000 7f17c605d700 1 -- 172.21.15.162:0/2581691340 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f17c01000f0 con 0x7f17c0103780 2024-04-27T04:14:36.973 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:14:36.973+0000 7f17bcff9700 1 -- 172.21.15.162:0/2581691340 <== mon.0 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 38) v1 ==== 92849+0+0 (secure 0 0 0) 0x7f17a8011400 con 0x7f17c0103780 2024-04-27T04:14:36.974 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:14:36.974+0000 7f17bcff9700 1 --2- 172.21.15.162:0/2581691340 >> [v2:172.21.15.162:6800/1396866108,v1:172.21.15.162:6801/1396866108] conn(0x7f17ac06ed30 0x7f17ac0711e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:14:36.974 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:14:36.974+0000 7f17bcff9700 1 -- 172.21.15.162:0/2581691340 <== mon.0 v2:172.21.15.162:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6410+0+0 (secure 0 0 0) 0x7f17a808e8b0 con 0x7f17c0103780 2024-04-27T04:14:36.974 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:14:36.974+0000 7f17bf7fe700 1 --2- 172.21.15.162:0/2581691340 >> [v2:172.21.15.162:6800/1396866108,v1:172.21.15.162:6801/1396866108] conn(0x7f17ac06ed30 0x7f17ac0711e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:14:36.974 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:14:36.974+0000 7f17bf7fe700 1 --2- 172.21.15.162:0/2581691340 >> [v2:172.21.15.162:6800/1396866108,v1:172.21.15.162:6801/1396866108] conn(0x7f17ac06ed30 0x7f17ac0711e0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f17b0004f90 tx=0x7f17b0009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:14:36.976 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:14:36.978+0000 7f17bcff9700 1 -- 172.21.15.162:0/2581691340 <== mon.0 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7f17a805b070 con 0x7f17c0103780 2024-04-27T04:14:37.270 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:14:37.272+0000 7f17c605d700 1 -- 172.21.15.162:0/2581691340 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f17c010efe0 con 0x7f17c0103780 2024-04-27T04:14:37.271 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:14:37.272+0000 7f17bcff9700 1 -- 172.21.15.162:0/2581691340 <== mon.0 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+736 (secure 0 0 0) 0x7f17a805ac00 con 0x7f17c0103780 2024-04-27T04:14:37.271 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-04-27T04:14:37.271 INFO:teuthology.orchestra.run.smithi162.stdout: "mon": { 2024-04-27T04:14:37.271 INFO:teuthology.orchestra.run.smithi162.stdout: "ceph version 19.0.0-3253-g4e213a58 (4e213a58e787437ff508e6c890ccd5fd32dc96b5) squid (dev)": 2 2024-04-27T04:14:37.271 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-04-27T04:14:37.271 INFO:teuthology.orchestra.run.smithi162.stdout: "mgr": { 2024-04-27T04:14:37.271 INFO:teuthology.orchestra.run.smithi162.stdout: "ceph version 19.0.0-3253-g4e213a58 (4e213a58e787437ff508e6c890ccd5fd32dc96b5) squid (dev)": 2 2024-04-27T04:14:37.271 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-04-27T04:14:37.271 INFO:teuthology.orchestra.run.smithi162.stdout: "osd": { 2024-04-27T04:14:37.272 INFO:teuthology.orchestra.run.smithi162.stdout: "ceph version 17.2.7-907-g2be3efa9 (2be3efa97a7972f8e925bd899b474329979b74d6) quincy (stable)": 6 2024-04-27T04:14:37.272 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-04-27T04:14:37.272 INFO:teuthology.orchestra.run.smithi162.stdout: "mds": { 2024-04-27T04:14:37.272 INFO:teuthology.orchestra.run.smithi162.stdout: "ceph version 17.2.7-907-g2be3efa9 (2be3efa97a7972f8e925bd899b474329979b74d6) quincy (stable)": 4 2024-04-27T04:14:37.272 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-04-27T04:14:37.272 INFO:teuthology.orchestra.run.smithi162.stdout: "overall": { 2024-04-27T04:14:37.272 INFO:teuthology.orchestra.run.smithi162.stdout: "ceph version 17.2.7-907-g2be3efa9 (2be3efa97a7972f8e925bd899b474329979b74d6) quincy (stable)": 10, 2024-04-27T04:14:37.272 INFO:teuthology.orchestra.run.smithi162.stdout: "ceph version 19.0.0-3253-g4e213a58 (4e213a58e787437ff508e6c890ccd5fd32dc96b5) squid (dev)": 4 2024-04-27T04:14:37.272 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-04-27T04:14:37.272 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-04-27T04:14:37.273 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:14:37.275+0000 7f17c605d700 1 -- 172.21.15.162:0/2581691340 >> [v2:172.21.15.162:6800/1396866108,v1:172.21.15.162:6801/1396866108] conn(0x7f17ac06ed30 msgr2=0x7f17ac0711e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:14:37.273 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:14:37.275+0000 7f17c605d700 1 --2- 172.21.15.162:0/2581691340 >> [v2:172.21.15.162:6800/1396866108,v1:172.21.15.162:6801/1396866108] conn(0x7f17ac06ed30 0x7f17ac0711e0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f17b0004f90 tx=0x7f17b0009e30 comp rx=0 tx=0).stop 2024-04-27T04:14:37.274 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:14:37.275+0000 7f17c605d700 1 -- 172.21.15.162:0/2581691340 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f17c0103780 msgr2=0x7f17c0113ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:14:37.274 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:14:37.275+0000 7f17c605d700 1 --2- 172.21.15.162:0/2581691340 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f17c0103780 0x7f17c0113ed0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f17a800b790 tx=0x7f17a8009220 comp rx=0 tx=0).stop 2024-04-27T04:14:37.274 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:14:37.275+0000 7f17c605d700 1 -- 172.21.15.162:0/2581691340 shutdown_connections 2024-04-27T04:14:37.274 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:14:37.275+0000 7f17c605d700 1 --2- 172.21.15.162:0/2581691340 >> [v2:172.21.15.162:6800/1396866108,v1:172.21.15.162:6801/1396866108] conn(0x7f17ac06ed30 0x7f17ac0711e0 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:14:37.274 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:14:37.275+0000 7f17c605d700 1 --2- 172.21.15.162:0/2581691340 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f17c0102de0 0x7f17c0113990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:14:37.274 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:14:37.275+0000 7f17c605d700 1 --2- 172.21.15.162:0/2581691340 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f17c0103780 0x7f17c0113ed0 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:14:37.274 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:14:37.275+0000 7f17c605d700 1 -- 172.21.15.162:0/2581691340 >> 172.21.15.162:0/2581691340 conn(0x7f17c00f8990 msgr2=0x7f17c0107330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:14:37.274 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:14:37.275+0000 7f17c605d700 1 -- 172.21.15.162:0/2581691340 shutdown_connections 2024-04-27T04:14:37.274 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:14:37.275+0000 7f17c605d700 1 -- 172.21.15.162:0/2581691340 wait complete. 2024-04-27T04:14:37.376 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:14:37.376+0000 7f456a6af700 1 -- 172.21.15.162:0/2811704692 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4564101d90 msgr2=0x7f4564102160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:14:37.376 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:14:37.376+0000 7f456a6af700 1 --2- 172.21.15.162:0/2811704692 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4564101d90 0x7f4564102160 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f4554004940 tx=0x7f4554009e30 comp rx=0 tx=0).stop 2024-04-27T04:14:37.376 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:14:37.376+0000 7f456a6af700 1 -- 172.21.15.162:0/2811704692 shutdown_connections 2024-04-27T04:14:37.376 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:14:37.376+0000 7f456a6af700 1 --2- 172.21.15.162:0/2811704692 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f45640fbd40 0x7f45640fc1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:14:37.377 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:14:37.376+0000 7f456a6af700 1 --2- 172.21.15.162:0/2811704692 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4564101d90 0x7f4564102160 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:14:37.377 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:14:37.376+0000 7f456a6af700 1 -- 172.21.15.162:0/2811704692 >> 172.21.15.162:0/2811704692 conn(0x7f4564071d00 msgr2=0x7f4564072100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:14:37.377 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:14:37.376+0000 7f456a6af700 1 -- 172.21.15.162:0/2811704692 shutdown_connections 2024-04-27T04:14:37.377 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:14:37.376+0000 7f456a6af700 1 -- 172.21.15.162:0/2811704692 wait complete. 2024-04-27T04:14:37.377 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:14:37.376+0000 7f456a6af700 1 Processor -- start 2024-04-27T04:14:37.377 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:14:37.377+0000 7f456a6af700 1 -- start start 2024-04-27T04:14:37.377 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:14:37.377+0000 7f456a6af700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f45640fbd40 0x7f456410eb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:14:37.377 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:14:37.377+0000 7f456a6af700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4564101d90 0x7f456410f060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:14:37.377 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:14:37.377+0000 7f456a6af700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f456410f740 con 0x7f45640fbd40 2024-04-27T04:14:37.377 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:14:37.377+0000 7f456a6af700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f456410f880 con 0x7f4564101d90 2024-04-27T04:14:37.377 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:14:37.377+0000 7f4563fff700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f45640fbd40 0x7f456410eb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:14:37.377 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:14:37.377+0000 7f4563fff700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f45640fbd40 0x7f456410eb20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:58262/0 (socket says 172.21.15.162:58262) 2024-04-27T04:14:37.378 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:14:37.377+0000 7f4563fff700 1 -- 172.21.15.162:0/3111984482 learned_addr learned my addr 172.21.15.162:0/3111984482 (peer_addr_for_me v2:172.21.15.162:0/0) 2024-04-27T04:14:37.378 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:14:37.377+0000 7f45637fe700 1 --2- 172.21.15.162:0/3111984482 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4564101d90 0x7f456410f060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:14:37.378 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:14:37.377+0000 7f4563fff700 1 -- 172.21.15.162:0/3111984482 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4564101d90 msgr2=0x7f456410f060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:14:37.378 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:14:37.377+0000 7f4563fff700 1 --2- 172.21.15.162:0/3111984482 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4564101d90 0x7f456410f060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:14:37.378 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:14:37.377+0000 7f4563fff700 1 -- 172.21.15.162:0/3111984482 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4554005040 con 0x7f45640fbd40 2024-04-27T04:14:37.378 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:14:37.377+0000 7f4563fff700 1 --2- 172.21.15.162:0/3111984482 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f45640fbd40 0x7f456410eb20 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f455400b3b0 tx=0x7f4554009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:14:37.378 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:14:37.378+0000 7f4560ff9700 1 -- 172.21.15.162:0/3111984482 <== mon.0 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f455400e040 con 0x7f45640fbd40 2024-04-27T04:14:37.378 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:14:37.378+0000 7f4560ff9700 1 -- 172.21.15.162:0/3111984482 <== mon.0 v2:172.21.15.162:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f455400bbc0 con 0x7f45640fbd40 2024-04-27T04:14:37.378 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:14:37.378+0000 7f456a6af700 1 -- 172.21.15.162:0/3111984482 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4564117b90 con 0x7f45640fbd40 2024-04-27T04:14:37.378 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:14:37.378+0000 7f4560ff9700 1 -- 172.21.15.162:0/3111984482 <== mon.0 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f455401c3f0 con 0x7f45640fbd40 2024-04-27T04:14:37.378 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:14:37.378+0000 7f456a6af700 1 -- 172.21.15.162:0/3111984482 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4564118120 con 0x7f45640fbd40 2024-04-27T04:14:37.379 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:14:37.379+0000 7f456a6af700 1 -- 172.21.15.162:0/3111984482 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f45640fd430 con 0x7f45640fbd40 2024-04-27T04:14:37.379 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:14:37.379+0000 7f4560ff9700 1 -- 172.21.15.162:0/3111984482 <== mon.0 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 38) v1 ==== 92849+0+0 (secure 0 0 0) 0x7f455400bd30 con 0x7f45640fbd40 2024-04-27T04:14:37.380 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:14:37.380+0000 7f4560ff9700 1 --2- 172.21.15.162:0/3111984482 >> [v2:172.21.15.162:6800/1396866108,v1:172.21.15.162:6801/1396866108] conn(0x7f454c06ed20 0x7f454c0711d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:14:37.380 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:14:37.380+0000 7f4560ff9700 1 -- 172.21.15.162:0/3111984482 <== mon.0 v2:172.21.15.162:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6410+0+0 (secure 0 0 0) 0x7f455408fdb0 con 0x7f45640fbd40 2024-04-27T04:14:37.380 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:14:37.380+0000 7f45637fe700 1 --2- 172.21.15.162:0/3111984482 >> [v2:172.21.15.162:6800/1396866108,v1:172.21.15.162:6801/1396866108] conn(0x7f454c06ed20 0x7f454c0711d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:14:37.380 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:14:37.380+0000 7f45637fe700 1 --2- 172.21.15.162:0/3111984482 >> [v2:172.21.15.162:6800/1396866108,v1:172.21.15.162:6801/1396866108] conn(0x7f454c06ed20 0x7f454c0711d0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f4558006fd0 tx=0x7f4558008040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:14:37.382 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:14:37.384+0000 7f4560ff9700 1 -- 172.21.15.162:0/3111984482 <== mon.0 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7f4554058990 con 0x7f45640fbd40 2024-04-27T04:14:37.626 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:14:37.627+0000 7f456a6af700 1 -- 172.21.15.162:0/3111984482 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f456404bf60 con 0x7f45640fbd40 2024-04-27T04:14:37.626 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:14:37.628+0000 7f4560ff9700 1 -- 172.21.15.162:0/3111984482 <== mon.0 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 15 v15) v1 ==== 76+0+1911 (secure 0 0 0) 0x7f455405c050 con 0x7f45640fbd40 2024-04-27T04:14:37.626 INFO:teuthology.orchestra.run.smithi162.stdout:e15 2024-04-27T04:14:37.626 INFO:teuthology.orchestra.run.smithi162.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-04-27T04:14:37.626 INFO:teuthology.orchestra.run.smithi162.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-27T04:14:37.626 INFO:teuthology.orchestra.run.smithi162.stdout:legacy client fscid: 1 2024-04-27T04:14:37.627 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-04-27T04:14:37.627 INFO:teuthology.orchestra.run.smithi162.stdout:Filesystem 'cephfs' (1) 2024-04-27T04:14:37.627 INFO:teuthology.orchestra.run.smithi162.stdout:fs_name cephfs 2024-04-27T04:14:37.627 INFO:teuthology.orchestra.run.smithi162.stdout:epoch 15 2024-04-27T04:14:37.627 INFO:teuthology.orchestra.run.smithi162.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-04-27T04:14:37.627 INFO:teuthology.orchestra.run.smithi162.stdout:created 2024-04-27T04:06:53.095749+0000 2024-04-27T04:14:37.627 INFO:teuthology.orchestra.run.smithi162.stdout:modified 2024-04-27T04:07:11.285033+0000 2024-04-27T04:14:37.627 INFO:teuthology.orchestra.run.smithi162.stdout:tableserver 0 2024-04-27T04:14:37.627 INFO:teuthology.orchestra.run.smithi162.stdout:root 0 2024-04-27T04:14:37.627 INFO:teuthology.orchestra.run.smithi162.stdout:session_timeout 60 2024-04-27T04:14:37.627 INFO:teuthology.orchestra.run.smithi162.stdout:session_autoclose 300 2024-04-27T04:14:37.627 INFO:teuthology.orchestra.run.smithi162.stdout:max_file_size 1099511627776 2024-04-27T04:14:37.627 INFO:teuthology.orchestra.run.smithi162.stdout:max_xattr_size 65536 2024-04-27T04:14:37.627 INFO:teuthology.orchestra.run.smithi162.stdout:required_client_features {} 2024-04-27T04:14:37.628 INFO:teuthology.orchestra.run.smithi162.stdout:last_failure 0 2024-04-27T04:14:37.628 INFO:teuthology.orchestra.run.smithi162.stdout:last_failure_osd_epoch 46 2024-04-27T04:14:37.628 INFO:teuthology.orchestra.run.smithi162.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-27T04:14:37.628 INFO:teuthology.orchestra.run.smithi162.stdout:max_mds 1 2024-04-27T04:14:37.628 INFO:teuthology.orchestra.run.smithi162.stdout:in 0 2024-04-27T04:14:37.628 INFO:teuthology.orchestra.run.smithi162.stdout:up {0=24273} 2024-04-27T04:14:37.628 INFO:teuthology.orchestra.run.smithi162.stdout:failed 2024-04-27T04:14:37.628 INFO:teuthology.orchestra.run.smithi162.stdout:damaged 2024-04-27T04:14:37.628 INFO:teuthology.orchestra.run.smithi162.stdout:stopped 2024-04-27T04:14:37.628 INFO:teuthology.orchestra.run.smithi162.stdout:data_pools [3] 2024-04-27T04:14:37.628 INFO:teuthology.orchestra.run.smithi162.stdout:metadata_pool 2 2024-04-27T04:14:37.628 INFO:teuthology.orchestra.run.smithi162.stdout:inline_data disabled 2024-04-27T04:14:37.628 INFO:teuthology.orchestra.run.smithi162.stdout:balancer 2024-04-27T04:14:37.628 INFO:teuthology.orchestra.run.smithi162.stdout:bal_rank_mask -1 2024-04-27T04:14:37.629 INFO:teuthology.orchestra.run.smithi162.stdout:standby_count_wanted 1 2024-04-27T04:14:37.629 INFO:teuthology.orchestra.run.smithi162.stdout:qdb_cluster leader: 0 members: 2024-04-27T04:14:37.629 INFO:teuthology.orchestra.run.smithi162.stdout:[mds.cephfs.smithi181.dzomgh{0:24273} state up:active seq 6 join_fscid=1 addr [v2:172.21.15.181:6824/658214993,v1:172.21.15.181:6825/658214993] compat {c=[1],r=[1],i=[7ff]}] 2024-04-27T04:14:37.629 INFO:teuthology.orchestra.run.smithi162.stdout:[mds.cephfs.smithi162.xcnapk{0:14476} state up:standby-replay seq 2 join_fscid=1 addr [v2:172.21.15.162:6828/1296160676,v1:172.21.15.162:6829/1296160676] compat {c=[1],r=[1],i=[7ff]}] 2024-04-27T04:14:37.629 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-04-27T04:14:37.629 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-04-27T04:14:37.629 INFO:teuthology.orchestra.run.smithi162.stdout:Standby daemons: 2024-04-27T04:14:37.629 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-04-27T04:14:37.629 INFO:teuthology.orchestra.run.smithi162.stdout:[mds.cephfs.smithi162.gzlyjj{-1:14488} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.162:6826/2629103894,v1:172.21.15.162:6827/2629103894] compat {c=[1],r=[1],i=[7ff]}] 2024-04-27T04:14:37.629 INFO:teuthology.orchestra.run.smithi162.stdout:[mds.cephfs.smithi181.bkkcdm{-1:24287} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.181:6826/3756475453,v1:172.21.15.181:6827/3756475453] compat {c=[1],r=[1],i=[7ff]}] 2024-04-27T04:14:37.630 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:14:37.630+0000 7f456a6af700 1 -- 172.21.15.162:0/3111984482 >> [v2:172.21.15.162:6800/1396866108,v1:172.21.15.162:6801/1396866108] conn(0x7f454c06ed20 msgr2=0x7f454c0711d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:14:37.630 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:14:37.630+0000 7f456a6af700 1 --2- 172.21.15.162:0/3111984482 >> [v2:172.21.15.162:6800/1396866108,v1:172.21.15.162:6801/1396866108] conn(0x7f454c06ed20 0x7f454c0711d0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f4558006fd0 tx=0x7f4558008040 comp rx=0 tx=0).stop 2024-04-27T04:14:37.630 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:14:37.630+0000 7f456a6af700 1 -- 172.21.15.162:0/3111984482 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f45640fbd40 msgr2=0x7f456410eb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:14:37.630 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:14:37.630+0000 7f456a6af700 1 --2- 172.21.15.162:0/3111984482 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f45640fbd40 0x7f456410eb20 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f455400b3b0 tx=0x7f4554009e30 comp rx=0 tx=0).stop 2024-04-27T04:14:37.630 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:14:37.630+0000 7f456a6af700 1 -- 172.21.15.162:0/3111984482 shutdown_connections 2024-04-27T04:14:37.630 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:14:37.630+0000 7f456a6af700 1 --2- 172.21.15.162:0/3111984482 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f45640fbd40 0x7f456410eb20 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:14:37.631 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:14:37.630+0000 7f456a6af700 1 --2- 172.21.15.162:0/3111984482 >> [v2:172.21.15.162:6800/1396866108,v1:172.21.15.162:6801/1396866108] conn(0x7f454c06ed20 0x7f454c0711d0 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:14:37.631 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:14:37.630+0000 7f456a6af700 1 --2- 172.21.15.162:0/3111984482 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4564101d90 0x7f456410f060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:14:37.631 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:14:37.630+0000 7f456a6af700 1 -- 172.21.15.162:0/3111984482 >> 172.21.15.162:0/3111984482 conn(0x7f4564071d00 msgr2=0x7f45640fa120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:14:37.631 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:14:37.631+0000 7f456a6af700 1 -- 172.21.15.162:0/3111984482 shutdown_connections 2024-04-27T04:14:37.631 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:14:37.631+0000 7f456a6af700 1 -- 172.21.15.162:0/3111984482 wait complete. 2024-04-27T04:14:37.631 INFO:teuthology.orchestra.run.smithi162.stderr:dumped fsmap epoch 15 2024-04-27T04:14:37.735 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:14:37.737+0000 7f9cca11a700 1 -- 172.21.15.162:0/1027617813 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9cc40fc5a0 msgr2=0x7f9cc4100bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:14:37.735 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:14:37.737+0000 7f9cca11a700 1 --2- 172.21.15.162:0/1027617813 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9cc40fc5a0 0x7f9cc4100bc0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f9cb8004940 tx=0x7f9cb8009e30 comp rx=0 tx=0).stop 2024-04-27T04:14:37.735 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:14:37.737+0000 7f9cca11a700 1 -- 172.21.15.162:0/1027617813 shutdown_connections 2024-04-27T04:14:37.735 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:14:37.737+0000 7f9cca11a700 1 --2- 172.21.15.162:0/1027617813 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9cc40fc5a0 0x7f9cc4100bc0 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:14:37.735 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:14:37.737+0000 7f9cca11a700 1 --2- 172.21.15.162:0/1027617813 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9cc40fbc00 0x7f9cc40fbfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:14:37.736 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:14:37.737+0000 7f9cca11a700 1 -- 172.21.15.162:0/1027617813 >> 172.21.15.162:0/1027617813 conn(0x7f9cc40f7860 msgr2=0x7f9cc40f9c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:14:37.736 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:14:37.737+0000 7f9cca11a700 1 -- 172.21.15.162:0/1027617813 shutdown_connections 2024-04-27T04:14:37.736 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:14:37.737+0000 7f9cca11a700 1 -- 172.21.15.162:0/1027617813 wait complete. 2024-04-27T04:14:37.736 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:14:37.737+0000 7f9cca11a700 1 Processor -- start 2024-04-27T04:14:37.738 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:14:37.737+0000 7f9cca11a700 1 -- start start 2024-04-27T04:14:37.738 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:14:37.738+0000 7f9cca11a700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9cc40fbc00 0x7f9cc411a030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:14:37.738 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:14:37.738+0000 7f9cca11a700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9cc40fc5a0 0x7f9cc411a570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:14:37.738 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:14:37.738+0000 7f9cca11a700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9cc411abc0 con 0x7f9cc40fc5a0 2024-04-27T04:14:37.738 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:14:37.738+0000 7f9cca11a700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9cc41141a0 con 0x7f9cc40fbc00 2024-04-27T04:14:37.738 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:14:37.738+0000 7f9cc2ffd700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9cc40fc5a0 0x7f9cc411a570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:14:37.738 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:14:37.738+0000 7f9cc2ffd700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9cc40fc5a0 0x7f9cc411a570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:58276/0 (socket says 172.21.15.162:58276) 2024-04-27T04:14:37.738 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:14:37.738+0000 7f9cc2ffd700 1 -- 172.21.15.162:0/373683125 learned_addr learned my addr 172.21.15.162:0/373683125 (peer_addr_for_me v2:172.21.15.162:0/0) 2024-04-27T04:14:37.738 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:14:37.738+0000 7f9cc37fe700 1 --2- 172.21.15.162:0/373683125 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9cc40fbc00 0x7f9cc411a030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:14:37.738 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:14:37.738+0000 7f9cc37fe700 1 -- 172.21.15.162:0/373683125 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9cc40fbc00 msgr2=0x7f9cc411a030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 12 2024-04-27T04:14:37.739 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:14:37.738+0000 7f9cc37fe700 1 -- 172.21.15.162:0/373683125 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9cc40fbc00 msgr2=0x7f9cc411a030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-04-27T04:14:37.739 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:14:37.738+0000 7f9cc37fe700 1 --2- 172.21.15.162:0/373683125 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9cc40fbc00 0x7f9cc411a030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2024-04-27T04:14:37.739 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:14:37.738+0000 7f9cc37fe700 1 --2- 172.21.15.162:0/373683125 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9cc40fbc00 0x7f9cc411a030 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-27T04:14:37.739 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:14:37.738+0000 7f9cc2ffd700 1 -- 172.21.15.162:0/373683125 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9cc40fbc00 msgr2=0x7f9cc411a030 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-27T04:14:37.739 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:14:37.738+0000 7f9cc2ffd700 1 --2- 172.21.15.162:0/373683125 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9cc40fbc00 0x7f9cc411a030 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:14:37.739 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:14:37.738+0000 7f9cc2ffd700 1 -- 172.21.15.162:0/373683125 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9cb8005040 con 0x7f9cc40fc5a0 2024-04-27T04:14:37.739 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:14:37.738+0000 7f9cc2ffd700 1 --2- 172.21.15.162:0/373683125 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9cc40fc5a0 0x7f9cc411a570 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f9cb800b3b0 tx=0x7f9cb8009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:14:37.739 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:14:37.738+0000 7f9cabfff700 1 -- 172.21.15.162:0/373683125 <== mon.0 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9cb800e040 con 0x7f9cc40fc5a0 2024-04-27T04:14:37.739 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:14:37.738+0000 7f9cabfff700 1 -- 172.21.15.162:0/373683125 <== mon.0 v2:172.21.15.162:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f9cb800bbc0 con 0x7f9cc40fc5a0 2024-04-27T04:14:37.739 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:14:37.738+0000 7f9cca11a700 1 -- 172.21.15.162:0/373683125 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9cc4114420 con 0x7f9cc40fc5a0 2024-04-27T04:14:37.739 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:14:37.738+0000 7f9cabfff700 1 -- 172.21.15.162:0/373683125 <== mon.0 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9cb801c3f0 con 0x7f9cc40fc5a0 2024-04-27T04:14:37.739 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:14:37.738+0000 7f9cca11a700 1 -- 172.21.15.162:0/373683125 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9cc4114960 con 0x7f9cc40fc5a0 2024-04-27T04:14:37.739 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:14:37.739+0000 7f9cca11a700 1 -- 172.21.15.162:0/373683125 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9cc4066b50 con 0x7f9cc40fc5a0 2024-04-27T04:14:37.739 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:14:37.740+0000 7f9cabfff700 1 -- 172.21.15.162:0/373683125 <== mon.0 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 38) v1 ==== 92849+0+0 (secure 0 0 0) 0x7f9cb800bd30 con 0x7f9cc40fc5a0 2024-04-27T04:14:37.740 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:14:37.740+0000 7f9cabfff700 1 --2- 172.21.15.162:0/373683125 >> [v2:172.21.15.162:6800/1396866108,v1:172.21.15.162:6801/1396866108] conn(0x7f9cac06ed30 0x7f9cac0711e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:14:37.740 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:14:37.740+0000 7f9cabfff700 1 -- 172.21.15.162:0/373683125 <== mon.0 v2:172.21.15.162:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6410+0+0 (secure 0 0 0) 0x7f9cb808fd90 con 0x7f9cc40fc5a0 2024-04-27T04:14:37.740 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:14:37.740+0000 7f9cc37fe700 1 --2- 172.21.15.162:0/373683125 >> [v2:172.21.15.162:6800/1396866108,v1:172.21.15.162:6801/1396866108] conn(0x7f9cac06ed30 0x7f9cac0711e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:14:37.741 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:14:37.741+0000 7f9cc37fe700 1 --2- 172.21.15.162:0/373683125 >> [v2:172.21.15.162:6800/1396866108,v1:172.21.15.162:6801/1396866108] conn(0x7f9cac06ed30 0x7f9cac0711e0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f9cb4009ce0 tx=0x7f9cb4009f70 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:14:37.743 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:14:37.745+0000 7f9cabfff700 1 -- 172.21.15.162:0/373683125 <== mon.0 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7f9cb805c4a0 con 0x7f9cc40fc5a0 2024-04-27T04:14:37.886 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:14:37 smithi162 ceph-mon[110641]: from='client.34132 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-27T04:14:37.887 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:14:37 smithi162 ceph-mon[110641]: from='client.34136 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-27T04:14:37.887 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:14:37 smithi162 ceph-mon[110641]: from='client.? 172.21.15.162:0/2581691340' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-27T04:14:37.887 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:14:37 smithi162 ceph-mon[110641]: from='client.? 172.21.15.162:0/3111984482' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-27T04:14:37.963 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:14:37.964+0000 7f9cca11a700 1 -- 172.21.15.162:0/373683125 --> [v2:172.21.15.162:6800/1396866108,v1:172.21.15.162:6801/1396866108] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f9cc4110530 con 0x7f9cac06ed30 2024-04-27T04:14:37.964 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:14:37.966+0000 7f9cabfff700 1 -- 172.21.15.162:0/373683125 <== mgr.34104 v2:172.21.15.162:6800/1396866108 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+368 (secure 0 0 0) 0x7f9cc4110530 con 0x7f9cac06ed30 2024-04-27T04:14:37.964 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-04-27T04:14:37.964 INFO:teuthology.orchestra.run.smithi162.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:90a3a9fa9ceb8977f6dad9ccd36fe46018d07e2118199820523a6b35301d9c16", 2024-04-27T04:14:37.964 INFO:teuthology.orchestra.run.smithi162.stdout: "in_progress": true, 2024-04-27T04:14:37.964 INFO:teuthology.orchestra.run.smithi162.stdout: "which": "Upgrading all daemon types on all hosts", 2024-04-27T04:14:37.964 INFO:teuthology.orchestra.run.smithi162.stdout: "services_complete": [ 2024-04-27T04:14:37.964 INFO:teuthology.orchestra.run.smithi162.stdout: "mgr" 2024-04-27T04:14:37.964 INFO:teuthology.orchestra.run.smithi162.stdout: ], 2024-04-27T04:14:37.964 INFO:teuthology.orchestra.run.smithi162.stdout: "progress": "3/21 daemons upgraded", 2024-04-27T04:14:37.964 INFO:teuthology.orchestra.run.smithi162.stdout: "message": "Currently upgrading mon daemons", 2024-04-27T04:14:37.964 INFO:teuthology.orchestra.run.smithi162.stdout: "is_paused": false 2024-04-27T04:14:37.964 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-04-27T04:14:37.966 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:14:37.968+0000 7f9cca11a700 1 -- 172.21.15.162:0/373683125 >> [v2:172.21.15.162:6800/1396866108,v1:172.21.15.162:6801/1396866108] conn(0x7f9cac06ed30 msgr2=0x7f9cac0711e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:14:37.966 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:14:37.968+0000 7f9cca11a700 1 --2- 172.21.15.162:0/373683125 >> [v2:172.21.15.162:6800/1396866108,v1:172.21.15.162:6801/1396866108] conn(0x7f9cac06ed30 0x7f9cac0711e0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f9cb4009ce0 tx=0x7f9cb4009f70 comp rx=0 tx=0).stop 2024-04-27T04:14:37.966 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:14:37.968+0000 7f9cca11a700 1 -- 172.21.15.162:0/373683125 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9cc40fc5a0 msgr2=0x7f9cc411a570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:14:37.966 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:14:37.968+0000 7f9cca11a700 1 --2- 172.21.15.162:0/373683125 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9cc40fc5a0 0x7f9cc411a570 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f9cb800b3b0 tx=0x7f9cb8009e30 comp rx=0 tx=0).stop 2024-04-27T04:14:37.967 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:14:37.968+0000 7f9cca11a700 1 -- 172.21.15.162:0/373683125 shutdown_connections 2024-04-27T04:14:37.967 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:14:37.968+0000 7f9cca11a700 1 --2- 172.21.15.162:0/373683125 >> [v2:172.21.15.162:6800/1396866108,v1:172.21.15.162:6801/1396866108] conn(0x7f9cac06ed30 0x7f9cac0711e0 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:14:37.967 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:14:37.968+0000 7f9cca11a700 1 --2- 172.21.15.162:0/373683125 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9cc40fbc00 0x7f9cc411a030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:14:37.967 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:14:37.968+0000 7f9cca11a700 1 --2- 172.21.15.162:0/373683125 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9cc40fc5a0 0x7f9cc411a570 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:14:37.967 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:14:37.968+0000 7f9cca11a700 1 -- 172.21.15.162:0/373683125 >> 172.21.15.162:0/373683125 conn(0x7f9cc40f7860 msgr2=0x7f9cc40f8d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:14:37.967 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:14:37.968+0000 7f9cca11a700 1 -- 172.21.15.162:0/373683125 shutdown_connections 2024-04-27T04:14:37.967 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:14:37.968+0000 7f9cca11a700 1 -- 172.21.15.162:0/373683125 wait complete. 2024-04-27T04:14:38.019 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:14:37 smithi181 ceph-mon[81134]: from='client.34132 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-27T04:14:38.019 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:14:37 smithi181 ceph-mon[81134]: from='client.34136 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-27T04:14:38.019 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:14:37 smithi181 ceph-mon[81134]: from='client.? 172.21.15.162:0/2581691340' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-27T04:14:38.019 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:14:37 smithi181 ceph-mon[81134]: from='client.? 172.21.15.162:0/3111984482' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-27T04:14:38.069 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:14:38.068+0000 7f3669651700 1 -- 172.21.15.162:0/3997293370 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f36640fb3c0 msgr2=0x7f36640fb830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:14:38.069 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:14:38.068+0000 7f3669651700 1 --2- 172.21.15.162:0/3997293370 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f36640fb3c0 0x7f36640fb830 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f3658009ce0 tx=0x7f3658004d10 comp rx=0 tx=0).stop 2024-04-27T04:14:38.069 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:14:38.068+0000 7f3669651700 1 -- 172.21.15.162:0/3997293370 shutdown_connections 2024-04-27T04:14:38.069 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:14:38.068+0000 7f3669651700 1 --2- 172.21.15.162:0/3997293370 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f36640fb3c0 0x7f36640fb830 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:14:38.069 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:14:38.068+0000 7f3669651700 1 --2- 172.21.15.162:0/3997293370 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3664102eb0 0x7f3664103280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:14:38.069 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:14:38.068+0000 7f3669651700 1 -- 172.21.15.162:0/3997293370 >> 172.21.15.162:0/3997293370 conn(0x7f3664072380 msgr2=0x7f3664072780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:14:38.069 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:14:38.068+0000 7f3669651700 1 -- 172.21.15.162:0/3997293370 shutdown_connections 2024-04-27T04:14:38.069 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:14:38.068+0000 7f3669651700 1 -- 172.21.15.162:0/3997293370 wait complete. 2024-04-27T04:14:38.069 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:14:38.069+0000 7f3669651700 1 Processor -- start 2024-04-27T04:14:38.069 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:14:38.069+0000 7f3669651700 1 -- start start 2024-04-27T04:14:38.069 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:14:38.069+0000 7f3669651700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f36640fb3c0 0x7f3664113b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:14:38.069 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:14:38.069+0000 7f3669651700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3664102eb0 0x7f3664114080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:14:38.070 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:14:38.069+0000 7f3669651700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36641146d0 con 0x7f36640fb3c0 2024-04-27T04:14:38.070 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:14:38.069+0000 7f3669651700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f366410dcb0 con 0x7f3664102eb0 2024-04-27T04:14:38.070 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:14:38.069+0000 7f3662ffd700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f36640fb3c0 0x7f3664113b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:14:38.070 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:14:38.069+0000 7f3662ffd700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f36640fb3c0 0x7f3664113b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:58284/0 (socket says 172.21.15.162:58284) 2024-04-27T04:14:38.070 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:14:38.069+0000 7f3662ffd700 1 -- 172.21.15.162:0/982866571 learned_addr learned my addr 172.21.15.162:0/982866571 (peer_addr_for_me v2:172.21.15.162:0/0) 2024-04-27T04:14:38.070 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:14:38.070+0000 7f36627fc700 1 --2- 172.21.15.162:0/982866571 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3664102eb0 0x7f3664114080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:14:38.070 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:14:38.070+0000 7f36627fc700 1 -- 172.21.15.162:0/982866571 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3664102eb0 msgr2=0x7f3664114080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 13 2024-04-27T04:14:38.070 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:14:38.070+0000 7f36627fc700 1 -- 172.21.15.162:0/982866571 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3664102eb0 msgr2=0x7f3664114080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-04-27T04:14:38.070 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:14:38.070+0000 7f36627fc700 1 --2- 172.21.15.162:0/982866571 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3664102eb0 0x7f3664114080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2024-04-27T04:14:38.070 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:14:38.070+0000 7f36627fc700 1 --2- 172.21.15.162:0/982866571 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3664102eb0 0x7f3664114080 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-27T04:14:38.070 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:14:38.070+0000 7f3662ffd700 1 -- 172.21.15.162:0/982866571 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3664102eb0 msgr2=0x7f3664114080 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-27T04:14:38.070 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:14:38.070+0000 7f3662ffd700 1 --2- 172.21.15.162:0/982866571 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3664102eb0 0x7f3664114080 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:14:38.070 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:14:38.070+0000 7f3662ffd700 1 -- 172.21.15.162:0/982866571 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3658009940 con 0x7f36640fb3c0 2024-04-27T04:14:38.071 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:14:38.070+0000 7f3662ffd700 1 --2- 172.21.15.162:0/982866571 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f36640fb3c0 0x7f3664113b40 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f365c002a50 tx=0x7f365c004630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:14:38.071 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:14:38.070+0000 7f364b7fe700 1 -- 172.21.15.162:0/982866571 <== mon.0 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f365c007cc0 con 0x7f36640fb3c0 2024-04-27T04:14:38.071 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:14:38.070+0000 7f364b7fe700 1 -- 172.21.15.162:0/982866571 <== mon.0 v2:172.21.15.162:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f365c00f040 con 0x7f36640fb3c0 2024-04-27T04:14:38.071 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:14:38.070+0000 7f3669651700 1 -- 172.21.15.162:0/982866571 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f366410df90 con 0x7f36640fb3c0 2024-04-27T04:14:38.071 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:14:38.070+0000 7f364b7fe700 1 -- 172.21.15.162:0/982866571 <== mon.0 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f365c00db70 con 0x7f36640fb3c0 2024-04-27T04:14:38.071 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:14:38.070+0000 7f3669651700 1 -- 172.21.15.162:0/982866571 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f366410e550 con 0x7f36640fb3c0 2024-04-27T04:14:38.072 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:14:38.071+0000 7f3669651700 1 -- 172.21.15.162:0/982866571 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3664100160 con 0x7f36640fb3c0 2024-04-27T04:14:38.072 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:14:38.072+0000 7f364b7fe700 1 -- 172.21.15.162:0/982866571 <== mon.0 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 38) v1 ==== 92849+0+0 (secure 0 0 0) 0x7f365c00dd10 con 0x7f36640fb3c0 2024-04-27T04:14:38.072 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:14:38.072+0000 7f364b7fe700 1 --2- 172.21.15.162:0/982866571 >> [v2:172.21.15.162:6800/1396866108,v1:172.21.15.162:6801/1396866108] conn(0x7f364c06ed30 0x7f364c0711e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:14:38.072 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:14:38.072+0000 7f364b7fe700 1 -- 172.21.15.162:0/982866571 <== mon.0 v2:172.21.15.162:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6410+0+0 (secure 0 0 0) 0x7f365c08e180 con 0x7f36640fb3c0 2024-04-27T04:14:38.072 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:14:38.072+0000 7f36627fc700 1 --2- 172.21.15.162:0/982866571 >> [v2:172.21.15.162:6800/1396866108,v1:172.21.15.162:6801/1396866108] conn(0x7f364c06ed30 0x7f364c0711e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:14:38.072 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:14:38.073+0000 7f36627fc700 1 --2- 172.21.15.162:0/982866571 >> [v2:172.21.15.162:6800/1396866108,v1:172.21.15.162:6801/1396866108] conn(0x7f364c06ed30 0x7f364c0711e0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f366410f3e0 tx=0x7f3658004d10 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:14:38.075 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:14:38.077+0000 7f364b7fe700 1 -- 172.21.15.162:0/982866571 <== mon.0 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7f365c05a890 con 0x7f36640fb3c0 2024-04-27T04:14:38.360 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:14:38.362+0000 7f3669651700 1 -- 172.21.15.162:0/982866571 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f36640fe6e0 con 0x7f36640fb3c0 2024-04-27T04:14:38.360 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:14:38.362+0000 7f364b7fe700 1 -- 172.21.15.162:0/982866571 <== mon.0 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+10 (secure 0 0 0) 0x7f365c05a420 con 0x7f36640fb3c0 2024-04-27T04:14:38.361 INFO:teuthology.orchestra.run.smithi162.stdout:HEALTH_OK 2024-04-27T04:14:38.364 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:14:38.365+0000 7f3669651700 1 -- 172.21.15.162:0/982866571 >> [v2:172.21.15.162:6800/1396866108,v1:172.21.15.162:6801/1396866108] conn(0x7f364c06ed30 msgr2=0x7f364c0711e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:14:38.364 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:14:38.365+0000 7f3669651700 1 --2- 172.21.15.162:0/982866571 >> [v2:172.21.15.162:6800/1396866108,v1:172.21.15.162:6801/1396866108] conn(0x7f364c06ed30 0x7f364c0711e0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f366410f3e0 tx=0x7f3658004d10 comp rx=0 tx=0).stop 2024-04-27T04:14:38.364 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:14:38.365+0000 7f3669651700 1 -- 172.21.15.162:0/982866571 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f36640fb3c0 msgr2=0x7f3664113b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:14:38.364 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:14:38.365+0000 7f3669651700 1 --2- 172.21.15.162:0/982866571 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f36640fb3c0 0x7f3664113b40 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f365c002a50 tx=0x7f365c004630 comp rx=0 tx=0).stop 2024-04-27T04:14:38.364 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:14:38.366+0000 7f3669651700 1 -- 172.21.15.162:0/982866571 shutdown_connections 2024-04-27T04:14:38.364 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:14:38.366+0000 7f3669651700 1 --2- 172.21.15.162:0/982866571 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f36640fb3c0 0x7f3664113b40 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:14:38.364 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:14:38.366+0000 7f3669651700 1 --2- 172.21.15.162:0/982866571 >> [v2:172.21.15.162:6800/1396866108,v1:172.21.15.162:6801/1396866108] conn(0x7f364c06ed30 0x7f364c0711e0 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:14:38.364 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:14:38.366+0000 7f3669651700 1 --2- 172.21.15.162:0/982866571 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3664102eb0 0x7f3664114080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:14:38.364 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:14:38.366+0000 7f3669651700 1 -- 172.21.15.162:0/982866571 >> 172.21.15.162:0/982866571 conn(0x7f3664072380 msgr2=0x7f3664107370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:14:38.364 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:14:38.366+0000 7f3669651700 1 -- 172.21.15.162:0/982866571 shutdown_connections 2024-04-27T04:14:38.365 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:14:38.366+0000 7f3669651700 1 -- 172.21.15.162:0/982866571 wait complete. 2024-04-27T04:14:39.129 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:14:38 smithi162 ceph-mon[110641]: from='client.34140 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-27T04:14:39.130 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:14:38 smithi162 ceph-mon[110641]: pgmap v13: 65 pgs: 65 active+clean; 208 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:14:39.130 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:14:38 smithi162 ceph-mon[110641]: from='client.? 172.21.15.162:0/982866571' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-27T04:14:39.172 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:14:38 smithi181 ceph-mon[81134]: from='client.34140 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-27T04:14:39.172 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:14:38 smithi181 ceph-mon[81134]: pgmap v13: 65 pgs: 65 active+clean; 208 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:14:39.172 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:14:38 smithi181 ceph-mon[81134]: from='client.? 172.21.15.162:0/982866571' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-27T04:14:40.136 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:14:39 smithi162 ceph-mon[110641]: from='client.34152 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-27T04:14:40.137 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:14:39 smithi162 ceph-mon[110641]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' 2024-04-27T04:14:40.137 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:14:39 smithi162 ceph-mon[110641]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' 2024-04-27T04:14:40.172 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:14:39 smithi181 ceph-mon[81134]: from='client.34152 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-27T04:14:40.172 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:14:39 smithi181 ceph-mon[81134]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' 2024-04-27T04:14:40.172 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:14:39 smithi181 ceph-mon[81134]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' 2024-04-27T04:14:41.136 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:14:40 smithi162 ceph-mon[110641]: pgmap v14: 65 pgs: 65 active+clean; 208 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:14:41.137 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:14:40 smithi162 ceph-mon[110641]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' 2024-04-27T04:14:41.137 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:14:40 smithi162 ceph-mon[110641]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' 2024-04-27T04:14:41.172 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:14:40 smithi181 ceph-mon[81134]: pgmap v14: 65 pgs: 65 active+clean; 208 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:14:41.172 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:14:40 smithi181 ceph-mon[81134]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' 2024-04-27T04:14:41.172 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:14:40 smithi181 ceph-mon[81134]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' 2024-04-27T04:14:41.922 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:14:41 smithi181 ceph-mon[81134]: pgmap v15: 65 pgs: 65 active+clean; 208 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:14:42.136 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:14:41 smithi162 ceph-mon[110641]: pgmap v15: 65 pgs: 65 active+clean; 208 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:14:44.636 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:14:44 smithi162 ceph-mon[110641]: Detected new or changed devices on smithi181 2024-04-27T04:14:44.637 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:14:44 smithi162 ceph-mon[110641]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' 2024-04-27T04:14:44.637 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:14:44 smithi162 ceph-mon[110641]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' 2024-04-27T04:14:44.637 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:14:44 smithi162 ceph-mon[110641]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T04:14:44.637 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:14:44 smithi162 ceph-mon[110641]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T04:14:44.637 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:14:44 smithi162 ceph-mon[110641]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' 2024-04-27T04:14:44.637 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:14:44 smithi162 ceph-mon[110641]: Reconfiguring mon.smithi162 (monmap changed)... 2024-04-27T04:14:44.637 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:14:44 smithi162 ceph-mon[110641]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-27T04:14:44.637 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:14:44 smithi162 ceph-mon[110641]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-27T04:14:44.637 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:14:44 smithi162 ceph-mon[110641]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T04:14:44.637 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:14:44 smithi162 ceph-mon[110641]: Reconfiguring daemon mon.smithi162 on smithi162 2024-04-27T04:14:44.637 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:14:44 smithi162 ceph-mon[110641]: pgmap v16: 65 pgs: 65 active+clean; 208 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:14:44.672 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:14:44 smithi181 ceph-mon[81134]: Detected new or changed devices on smithi181 2024-04-27T04:14:44.672 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:14:44 smithi181 ceph-mon[81134]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' 2024-04-27T04:14:44.672 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:14:44 smithi181 ceph-mon[81134]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' 2024-04-27T04:14:44.672 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:14:44 smithi181 ceph-mon[81134]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T04:14:44.672 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:14:44 smithi181 ceph-mon[81134]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T04:14:44.673 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:14:44 smithi181 ceph-mon[81134]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' 2024-04-27T04:14:44.673 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:14:44 smithi181 ceph-mon[81134]: Reconfiguring mon.smithi162 (monmap changed)... 2024-04-27T04:14:44.673 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:14:44 smithi181 ceph-mon[81134]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-27T04:14:44.673 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:14:44 smithi181 ceph-mon[81134]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-27T04:14:44.673 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:14:44 smithi181 ceph-mon[81134]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T04:14:44.673 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:14:44 smithi181 ceph-mon[81134]: Reconfiguring daemon mon.smithi162 on smithi162 2024-04-27T04:14:44.673 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:14:44 smithi181 ceph-mon[81134]: pgmap v16: 65 pgs: 65 active+clean; 208 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:14:46.136 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:14:45 smithi162 ceph-mon[110641]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' 2024-04-27T04:14:46.137 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:14:45 smithi162 ceph-mon[110641]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' 2024-04-27T04:14:46.137 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:14:45 smithi162 ceph-mon[110641]: Reconfiguring mgr.smithi162.oennfn (monmap changed)... 2024-04-27T04:14:46.137 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:14:45 smithi162 ceph-mon[110641]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi162.oennfn", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-27T04:14:46.137 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:14:45 smithi162 ceph-mon[110641]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-27T04:14:46.137 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:14:45 smithi162 ceph-mon[110641]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T04:14:46.137 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:14:45 smithi162 ceph-mon[110641]: Reconfiguring daemon mgr.smithi162.oennfn on smithi162 2024-04-27T04:14:46.137 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:14:45 smithi162 ceph-mon[110641]: pgmap v17: 65 pgs: 65 active+clean; 208 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-04-27T04:14:46.422 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:14:45 smithi181 ceph-mon[81134]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' 2024-04-27T04:14:46.422 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:14:45 smithi181 ceph-mon[81134]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' 2024-04-27T04:14:46.422 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:14:45 smithi181 ceph-mon[81134]: Reconfiguring mgr.smithi162.oennfn (monmap changed)... 2024-04-27T04:14:46.422 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:14:45 smithi181 ceph-mon[81134]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi162.oennfn", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-27T04:14:46.423 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:14:45 smithi181 ceph-mon[81134]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-27T04:14:46.423 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:14:45 smithi181 ceph-mon[81134]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T04:14:46.423 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:14:45 smithi181 ceph-mon[81134]: Reconfiguring daemon mgr.smithi162.oennfn on smithi162 2024-04-27T04:14:46.423 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:14:45 smithi181 ceph-mon[81134]: pgmap v17: 65 pgs: 65 active+clean; 208 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-04-27T04:14:47.637 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:14:47 smithi162 ceph-mon[110641]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' 2024-04-27T04:14:47.637 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:14:47 smithi162 ceph-mon[110641]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' 2024-04-27T04:14:47.637 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:14:47 smithi162 ceph-mon[110641]: Reconfiguring crash.smithi162 (monmap changed)... 2024-04-27T04:14:47.637 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:14:47 smithi162 ceph-mon[110641]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi162", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-04-27T04:14:47.637 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:14:47 smithi162 ceph-mon[110641]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T04:14:47.637 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:14:47 smithi162 ceph-mon[110641]: Reconfiguring daemon crash.smithi162 on smithi162 2024-04-27T04:14:47.922 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:14:47 smithi181 ceph-mon[81134]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' 2024-04-27T04:14:47.922 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:14:47 smithi181 ceph-mon[81134]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' 2024-04-27T04:14:47.922 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:14:47 smithi181 ceph-mon[81134]: Reconfiguring crash.smithi162 (monmap changed)... 2024-04-27T04:14:47.922 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:14:47 smithi181 ceph-mon[81134]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi162", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-04-27T04:14:47.922 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:14:47 smithi181 ceph-mon[81134]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T04:14:47.922 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:14:47 smithi181 ceph-mon[81134]: Reconfiguring daemon crash.smithi162 on smithi162 2024-04-27T04:14:48.791 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:14:48 smithi162 ceph-mon[110641]: pgmap v18: 65 pgs: 65 active+clean; 208 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:14:48.791 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:14:48 smithi162 ceph-mon[110641]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-27T04:14:48.791 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:14:48 smithi162 ceph-mon[110641]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' 2024-04-27T04:14:48.792 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:14:48 smithi162 ceph-mon[110641]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' 2024-04-27T04:14:48.792 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:14:48 smithi162 ceph-mon[110641]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-04-27T04:14:48.792 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:14:48 smithi162 ceph-mon[110641]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T04:14:48.922 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:14:48 smithi181 ceph-mon[81134]: pgmap v18: 65 pgs: 65 active+clean; 208 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:14:48.922 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:14:48 smithi181 ceph-mon[81134]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-27T04:14:48.922 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:14:48 smithi181 ceph-mon[81134]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' 2024-04-27T04:14:48.922 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:14:48 smithi181 ceph-mon[81134]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' 2024-04-27T04:14:48.922 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:14:48 smithi181 ceph-mon[81134]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-04-27T04:14:48.922 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:14:48 smithi181 ceph-mon[81134]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T04:14:49.615 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:14:49 smithi162 ceph-mon[110641]: Reconfiguring osd.0 (monmap changed)... 2024-04-27T04:14:49.615 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:14:49 smithi162 ceph-mon[110641]: Reconfiguring daemon osd.0 on smithi162 2024-04-27T04:14:49.922 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:14:49 smithi181 ceph-mon[81134]: Reconfiguring osd.0 (monmap changed)... 2024-04-27T04:14:49.922 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:14:49 smithi181 ceph-mon[81134]: Reconfiguring daemon osd.0 on smithi162 2024-04-27T04:14:50.886 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:14:50 smithi162 ceph-mon[110641]: pgmap v19: 65 pgs: 65 active+clean; 208 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:14:50.886 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:14:50 smithi162 ceph-mon[110641]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' 2024-04-27T04:14:50.887 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:14:50 smithi162 ceph-mon[110641]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' 2024-04-27T04:14:50.887 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:14:50 smithi162 ceph-mon[110641]: Reconfiguring osd.1 (monmap changed)... 2024-04-27T04:14:50.887 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:14:50 smithi162 ceph-mon[110641]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-04-27T04:14:50.887 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:14:50 smithi162 ceph-mon[110641]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T04:14:50.887 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:14:50 smithi162 ceph-mon[110641]: Reconfiguring daemon osd.1 on smithi162 2024-04-27T04:14:51.172 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:14:50 smithi181 ceph-mon[81134]: pgmap v19: 65 pgs: 65 active+clean; 208 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:14:51.172 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:14:50 smithi181 ceph-mon[81134]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' 2024-04-27T04:14:51.172 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:14:50 smithi181 ceph-mon[81134]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' 2024-04-27T04:14:51.172 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:14:50 smithi181 ceph-mon[81134]: Reconfiguring osd.1 (monmap changed)... 2024-04-27T04:14:51.172 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:14:50 smithi181 ceph-mon[81134]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-04-27T04:14:51.172 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:14:50 smithi181 ceph-mon[81134]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T04:14:51.172 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:14:50 smithi181 ceph-mon[81134]: Reconfiguring daemon osd.1 on smithi162 2024-04-27T04:14:52.636 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:14:52 smithi162 ceph-mon[110641]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' 2024-04-27T04:14:52.637 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:14:52 smithi162 ceph-mon[110641]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' 2024-04-27T04:14:52.637 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:14:52 smithi162 ceph-mon[110641]: Reconfiguring osd.2 (monmap changed)... 2024-04-27T04:14:52.637 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:14:52 smithi162 ceph-mon[110641]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-04-27T04:14:52.637 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:14:52 smithi162 ceph-mon[110641]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T04:14:52.637 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:14:52 smithi162 ceph-mon[110641]: Reconfiguring daemon osd.2 on smithi162 2024-04-27T04:14:52.637 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:14:52 smithi162 ceph-mon[110641]: pgmap v20: 65 pgs: 65 active+clean; 208 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:14:52.672 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:14:52 smithi181 ceph-mon[81134]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' 2024-04-27T04:14:52.672 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:14:52 smithi181 ceph-mon[81134]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' 2024-04-27T04:14:52.672 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:14:52 smithi181 ceph-mon[81134]: Reconfiguring osd.2 (monmap changed)... 2024-04-27T04:14:52.672 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:14:52 smithi181 ceph-mon[81134]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-04-27T04:14:52.672 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:14:52 smithi181 ceph-mon[81134]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T04:14:52.672 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:14:52 smithi181 ceph-mon[81134]: Reconfiguring daemon osd.2 on smithi162 2024-04-27T04:14:52.672 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:14:52 smithi181 ceph-mon[81134]: pgmap v20: 65 pgs: 65 active+clean; 208 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:14:54.387 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:14:54 smithi162 ceph-mon[110641]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' 2024-04-27T04:14:54.387 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:14:54 smithi162 ceph-mon[110641]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' 2024-04-27T04:14:54.387 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:14:54 smithi162 ceph-mon[110641]: Reconfiguring mds.cephfs.smithi162.gzlyjj (monmap changed)... 2024-04-27T04:14:54.387 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:14:54 smithi162 ceph-mon[110641]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi162.gzlyjj", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-27T04:14:54.387 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:14:54 smithi162 ceph-mon[110641]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T04:14:54.387 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:14:54 smithi162 ceph-mon[110641]: Reconfiguring daemon mds.cephfs.smithi162.gzlyjj on smithi162 2024-04-27T04:14:54.387 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:14:54 smithi162 ceph-mon[110641]: pgmap v21: 65 pgs: 65 active+clean; 208 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:14:54.422 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:14:54 smithi181 ceph-mon[81134]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' 2024-04-27T04:14:54.422 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:14:54 smithi181 ceph-mon[81134]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' 2024-04-27T04:14:54.422 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:14:54 smithi181 ceph-mon[81134]: Reconfiguring mds.cephfs.smithi162.gzlyjj (monmap changed)... 2024-04-27T04:14:54.422 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:14:54 smithi181 ceph-mon[81134]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi162.gzlyjj", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-27T04:14:54.422 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:14:54 smithi181 ceph-mon[81134]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T04:14:54.422 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:14:54 smithi181 ceph-mon[81134]: Reconfiguring daemon mds.cephfs.smithi162.gzlyjj on smithi162 2024-04-27T04:14:54.423 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:14:54 smithi181 ceph-mon[81134]: pgmap v21: 65 pgs: 65 active+clean; 208 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:14:55.859 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:14:55 smithi162 ceph-mon[110641]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' 2024-04-27T04:14:55.859 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:14:55 smithi162 ceph-mon[110641]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' 2024-04-27T04:14:55.859 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:14:55 smithi162 ceph-mon[110641]: Reconfiguring mds.cephfs.smithi162.xcnapk (monmap changed)... 2024-04-27T04:14:55.859 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:14:55 smithi162 ceph-mon[110641]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi162.xcnapk", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-27T04:14:55.860 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:14:55 smithi162 ceph-mon[110641]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T04:14:55.860 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:14:55 smithi162 ceph-mon[110641]: Reconfiguring daemon mds.cephfs.smithi162.xcnapk on smithi162 2024-04-27T04:14:55.922 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:14:55 smithi181 ceph-mon[81134]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' 2024-04-27T04:14:55.922 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:14:55 smithi181 ceph-mon[81134]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' 2024-04-27T04:14:55.922 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:14:55 smithi181 ceph-mon[81134]: Reconfiguring mds.cephfs.smithi162.xcnapk (monmap changed)... 2024-04-27T04:14:55.922 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:14:55 smithi181 ceph-mon[81134]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi162.xcnapk", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-27T04:14:55.922 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:14:55 smithi181 ceph-mon[81134]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T04:14:55.922 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:14:55 smithi181 ceph-mon[81134]: Reconfiguring daemon mds.cephfs.smithi162.xcnapk on smithi162 2024-04-27T04:14:56.922 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:14:56 smithi181 ceph-mon[81134]: pgmap v22: 65 pgs: 65 active+clean; 208 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:14:56.922 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:14:56 smithi181 ceph-mon[81134]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' 2024-04-27T04:14:56.922 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:14:56 smithi181 ceph-mon[81134]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' 2024-04-27T04:14:56.922 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:14:56 smithi181 ceph-mon[81134]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi181", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-04-27T04:14:56.922 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:14:56 smithi181 ceph-mon[81134]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T04:14:57.137 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:14:56 smithi162 ceph-mon[110641]: pgmap v22: 65 pgs: 65 active+clean; 208 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:14:57.137 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:14:56 smithi162 ceph-mon[110641]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' 2024-04-27T04:14:57.137 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:14:56 smithi162 ceph-mon[110641]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' 2024-04-27T04:14:57.137 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:14:56 smithi162 ceph-mon[110641]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi181", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-04-27T04:14:57.137 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:14:56 smithi162 ceph-mon[110641]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T04:14:57.922 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:14:57 smithi181 ceph-mon[81134]: Reconfiguring crash.smithi181 (monmap changed)... 2024-04-27T04:14:57.922 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:14:57 smithi181 ceph-mon[81134]: Reconfiguring daemon crash.smithi181 on smithi181 2024-04-27T04:14:58.136 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:14:57 smithi162 ceph-mon[110641]: Reconfiguring crash.smithi181 (monmap changed)... 2024-04-27T04:14:58.137 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:14:57 smithi162 ceph-mon[110641]: Reconfiguring daemon crash.smithi181 on smithi181 2024-04-27T04:14:58.922 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:14:58 smithi181 ceph-mon[81134]: pgmap v23: 65 pgs: 65 active+clean; 208 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:14:58.922 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:14:58 smithi181 ceph-mon[81134]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' 2024-04-27T04:14:58.922 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:14:58 smithi181 ceph-mon[81134]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' 2024-04-27T04:14:58.922 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:14:58 smithi181 ceph-mon[81134]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi181.vtwuhz", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-27T04:14:58.922 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:14:58 smithi181 ceph-mon[81134]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-27T04:14:58.922 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:14:58 smithi181 ceph-mon[81134]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T04:14:59.037 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:14:58 smithi162 ceph-mon[110641]: pgmap v23: 65 pgs: 65 active+clean; 208 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:14:59.038 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:14:58 smithi162 ceph-mon[110641]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' 2024-04-27T04:14:59.038 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:14:58 smithi162 ceph-mon[110641]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' 2024-04-27T04:14:59.038 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:14:58 smithi162 ceph-mon[110641]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi181.vtwuhz", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-27T04:14:59.038 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:14:58 smithi162 ceph-mon[110641]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-27T04:14:59.038 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:14:58 smithi162 ceph-mon[110641]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T04:15:00.054 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:14:59 smithi162 ceph-mon[110641]: Reconfiguring mgr.smithi181.vtwuhz (monmap changed)... 2024-04-27T04:15:00.055 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:14:59 smithi162 ceph-mon[110641]: Reconfiguring daemon mgr.smithi181.vtwuhz on smithi181 2024-04-27T04:15:00.055 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:14:59 smithi162 ceph-mon[110641]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' 2024-04-27T04:15:00.055 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:14:59 smithi162 ceph-mon[110641]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' 2024-04-27T04:15:00.055 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:14:59 smithi162 ceph-mon[110641]: Reconfiguring mon.smithi181 (monmap changed)... 2024-04-27T04:15:00.055 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:14:59 smithi162 ceph-mon[110641]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-27T04:15:00.055 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:14:59 smithi162 ceph-mon[110641]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-27T04:15:00.055 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:14:59 smithi162 ceph-mon[110641]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T04:15:00.055 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:14:59 smithi162 ceph-mon[110641]: Reconfiguring daemon mon.smithi181 on smithi181 2024-04-27T04:15:00.055 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:14:59 smithi162 ceph-mon[110641]: pgmap v24: 65 pgs: 65 active+clean; 208 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:15:00.078 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:14:59 smithi181 ceph-mon[81134]: Reconfiguring mgr.smithi181.vtwuhz (monmap changed)... 2024-04-27T04:15:00.078 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:14:59 smithi181 ceph-mon[81134]: Reconfiguring daemon mgr.smithi181.vtwuhz on smithi181 2024-04-27T04:15:00.078 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:14:59 smithi181 ceph-mon[81134]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' 2024-04-27T04:15:00.078 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:14:59 smithi181 ceph-mon[81134]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' 2024-04-27T04:15:00.078 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:14:59 smithi181 ceph-mon[81134]: Reconfiguring mon.smithi181 (monmap changed)... 2024-04-27T04:15:00.079 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:14:59 smithi181 ceph-mon[81134]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-27T04:15:00.079 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:14:59 smithi181 ceph-mon[81134]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-27T04:15:00.079 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:14:59 smithi181 ceph-mon[81134]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T04:15:00.079 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:14:59 smithi181 ceph-mon[81134]: Reconfiguring daemon mon.smithi181 on smithi181 2024-04-27T04:15:00.079 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:14:59 smithi181 ceph-mon[81134]: pgmap v24: 65 pgs: 65 active+clean; 208 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:15:02.387 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:15:02 smithi162 ceph-mon[110641]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' 2024-04-27T04:15:02.387 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:15:02 smithi162 ceph-mon[110641]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' 2024-04-27T04:15:02.387 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:15:02 smithi162 ceph-mon[110641]: Reconfiguring osd.3 (monmap changed)... 2024-04-27T04:15:02.387 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:15:02 smithi162 ceph-mon[110641]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-04-27T04:15:02.387 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:15:02 smithi162 ceph-mon[110641]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T04:15:02.387 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:15:02 smithi162 ceph-mon[110641]: Reconfiguring daemon osd.3 on smithi181 2024-04-27T04:15:02.388 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:15:02 smithi162 ceph-mon[110641]: pgmap v25: 65 pgs: 65 active+clean; 208 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:15:02.398 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:15:02 smithi181 ceph-mon[81134]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' 2024-04-27T04:15:02.398 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:15:02 smithi181 ceph-mon[81134]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' 2024-04-27T04:15:02.398 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:15:02 smithi181 ceph-mon[81134]: Reconfiguring osd.3 (monmap changed)... 2024-04-27T04:15:02.398 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:15:02 smithi181 ceph-mon[81134]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-04-27T04:15:02.398 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:15:02 smithi181 ceph-mon[81134]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T04:15:02.398 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:15:02 smithi181 ceph-mon[81134]: Reconfiguring daemon osd.3 on smithi181 2024-04-27T04:15:02.398 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:15:02 smithi181 ceph-mon[81134]: pgmap v25: 65 pgs: 65 active+clean; 208 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:15:03.375 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:15:03 smithi181 ceph-mon[81134]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-27T04:15:03.375 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:15:03 smithi181 ceph-mon[81134]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' 2024-04-27T04:15:03.375 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:15:03 smithi181 ceph-mon[81134]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' 2024-04-27T04:15:03.375 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:15:03 smithi181 ceph-mon[81134]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-04-27T04:15:03.375 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:15:03 smithi181 ceph-mon[81134]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T04:15:03.386 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:15:03 smithi162 ceph-mon[110641]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-27T04:15:03.386 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:15:03 smithi162 ceph-mon[110641]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' 2024-04-27T04:15:03.387 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:15:03 smithi162 ceph-mon[110641]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' 2024-04-27T04:15:03.387 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:15:03 smithi162 ceph-mon[110641]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-04-27T04:15:03.387 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:15:03 smithi162 ceph-mon[110641]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T04:15:04.339 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:15:04 smithi181 ceph-mon[81134]: Reconfiguring osd.4 (monmap changed)... 2024-04-27T04:15:04.339 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:15:04 smithi181 ceph-mon[81134]: Reconfiguring daemon osd.4 on smithi181 2024-04-27T04:15:04.339 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:15:04 smithi181 ceph-mon[81134]: pgmap v26: 65 pgs: 65 active+clean; 208 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:15:04.386 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:15:04 smithi162 ceph-mon[110641]: Reconfiguring osd.4 (monmap changed)... 2024-04-27T04:15:04.387 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:15:04 smithi162 ceph-mon[110641]: Reconfiguring daemon osd.4 on smithi181 2024-04-27T04:15:04.387 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:15:04 smithi162 ceph-mon[110641]: pgmap v26: 65 pgs: 65 active+clean; 208 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:15:05.672 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:15:05 smithi181 ceph-mon[81134]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' 2024-04-27T04:15:05.672 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:15:05 smithi181 ceph-mon[81134]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' 2024-04-27T04:15:05.672 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:15:05 smithi181 ceph-mon[81134]: Reconfiguring osd.5 (monmap changed)... 2024-04-27T04:15:05.672 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:15:05 smithi181 ceph-mon[81134]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-04-27T04:15:05.672 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:15:05 smithi181 ceph-mon[81134]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T04:15:05.673 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:15:05 smithi181 ceph-mon[81134]: Reconfiguring daemon osd.5 on smithi181 2024-04-27T04:15:05.886 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:15:05 smithi162 ceph-mon[110641]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' 2024-04-27T04:15:05.887 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:15:05 smithi162 ceph-mon[110641]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' 2024-04-27T04:15:05.887 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:15:05 smithi162 ceph-mon[110641]: Reconfiguring osd.5 (monmap changed)... 2024-04-27T04:15:05.887 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:15:05 smithi162 ceph-mon[110641]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-04-27T04:15:05.887 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:15:05 smithi162 ceph-mon[110641]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T04:15:05.887 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:15:05 smithi162 ceph-mon[110641]: Reconfiguring daemon osd.5 on smithi181 2024-04-27T04:15:06.764 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:15:06 smithi181 ceph-mon[81134]: pgmap v27: 65 pgs: 65 active+clean; 208 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:15:06.764 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:15:06 smithi181 ceph-mon[81134]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' 2024-04-27T04:15:06.764 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:15:06 smithi181 ceph-mon[81134]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' 2024-04-27T04:15:06.764 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:15:06 smithi181 ceph-mon[81134]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi181.dzomgh", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-27T04:15:06.765 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:15:06 smithi181 ceph-mon[81134]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T04:15:06.886 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:15:06 smithi162 ceph-mon[110641]: pgmap v27: 65 pgs: 65 active+clean; 208 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:15:06.887 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:15:06 smithi162 ceph-mon[110641]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' 2024-04-27T04:15:06.887 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:15:06 smithi162 ceph-mon[110641]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' 2024-04-27T04:15:06.887 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:15:06 smithi162 ceph-mon[110641]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi181.dzomgh", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-27T04:15:06.887 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:15:06 smithi162 ceph-mon[110641]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T04:15:07.694 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:15:07 smithi181 ceph-mon[81134]: Reconfiguring mds.cephfs.smithi181.dzomgh (monmap changed)... 2024-04-27T04:15:07.695 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:15:07 smithi181 ceph-mon[81134]: Reconfiguring daemon mds.cephfs.smithi181.dzomgh on smithi181 2024-04-27T04:15:07.886 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:15:07 smithi162 ceph-mon[110641]: Reconfiguring mds.cephfs.smithi181.dzomgh (monmap changed)... 2024-04-27T04:15:07.887 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:15:07 smithi162 ceph-mon[110641]: Reconfiguring daemon mds.cephfs.smithi181.dzomgh on smithi181 2024-04-27T04:15:08.483 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:15:08.484+0000 7fad7e827700 1 -- 172.21.15.162:0/2107013452 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fad78103780 msgr2=0x7fad780fcdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:15:08.483 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:15:08.484+0000 7fad7e827700 1 --2- 172.21.15.162:0/2107013452 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fad78103780 0x7fad780fcdf0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fad6c004940 tx=0x7fad6c009e30 comp rx=0 tx=0).stop 2024-04-27T04:15:08.483 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:15:08.484+0000 7fad7e827700 1 -- 172.21.15.162:0/2107013452 shutdown_connections 2024-04-27T04:15:08.483 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:15:08.484+0000 7fad7e827700 1 --2- 172.21.15.162:0/2107013452 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fad78103780 0x7fad780fcdf0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:15:08.484 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:15:08.484+0000 7fad7e827700 1 --2- 172.21.15.162:0/2107013452 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fad78102de0 0x7fad781031b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:15:08.484 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:15:08.484+0000 7fad7e827700 1 -- 172.21.15.162:0/2107013452 >> 172.21.15.162:0/2107013452 conn(0x7fad780f8990 msgr2=0x7fad780fada0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:15:08.484 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:15:08.484+0000 7fad7e827700 1 -- 172.21.15.162:0/2107013452 shutdown_connections 2024-04-27T04:15:08.484 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:15:08.484+0000 7fad7e827700 1 -- 172.21.15.162:0/2107013452 wait complete. 2024-04-27T04:15:08.484 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:15:08.484+0000 7fad7e827700 1 Processor -- start 2024-04-27T04:15:08.484 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:15:08.485+0000 7fad7e827700 1 -- start start 2024-04-27T04:15:08.484 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:15:08.485+0000 7fad7e827700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fad78102de0 0x7fad781138d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:15:08.484 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:15:08.485+0000 7fad7e827700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fad78103780 0x7fad78113e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:15:08.484 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:15:08.485+0000 7fad7e827700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad78114460 con 0x7fad78102de0 2024-04-27T04:15:08.484 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:15:08.485+0000 7fad7e827700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad7810da60 con 0x7fad78103780 2024-04-27T04:15:08.484 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:15:08.485+0000 7fad77fff700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fad78102de0 0x7fad781138d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:15:08.484 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:15:08.485+0000 7fad77fff700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fad78102de0 0x7fad781138d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:33674/0 (socket says 172.21.15.162:33674) 2024-04-27T04:15:08.484 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:15:08.485+0000 7fad77fff700 1 -- 172.21.15.162:0/3036577334 learned_addr learned my addr 172.21.15.162:0/3036577334 (peer_addr_for_me v2:172.21.15.162:0/0) 2024-04-27T04:15:08.484 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:15:08.485+0000 7fad777fe700 1 --2- 172.21.15.162:0/3036577334 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fad78103780 0x7fad78113e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:15:08.484 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:15:08.485+0000 7fad77fff700 1 -- 172.21.15.162:0/3036577334 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fad78103780 msgr2=0x7fad78113e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:15:08.485 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:15:08.485+0000 7fad77fff700 1 --2- 172.21.15.162:0/3036577334 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fad78103780 0x7fad78113e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:15:08.485 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:15:08.485+0000 7fad77fff700 1 -- 172.21.15.162:0/3036577334 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad6c005040 con 0x7fad78102de0 2024-04-27T04:15:08.486 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:15:08.485+0000 7fad77fff700 1 --2- 172.21.15.162:0/3036577334 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fad78102de0 0x7fad781138d0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fad6800eb40 tx=0x7fad6800b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:15:08.486 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:15:08.486+0000 7fad74ff9700 1 -- 172.21.15.162:0/3036577334 <== mon.0 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fad68007cc0 con 0x7fad78102de0 2024-04-27T04:15:08.486 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:15:08.486+0000 7fad7e827700 1 -- 172.21.15.162:0/3036577334 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fad7810dce0 con 0x7fad78102de0 2024-04-27T04:15:08.486 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:15:08.486+0000 7fad74ff9700 1 -- 172.21.15.162:0/3036577334 <== mon.0 v2:172.21.15.162:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fad6800f040 con 0x7fad78102de0 2024-04-27T04:15:08.486 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:15:08.486+0000 7fad74ff9700 1 -- 172.21.15.162:0/3036577334 <== mon.0 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fad6800ca10 con 0x7fad78102de0 2024-04-27T04:15:08.486 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:15:08.486+0000 7fad7e827700 1 -- 172.21.15.162:0/3036577334 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fad7810e1f0 con 0x7fad78102de0 2024-04-27T04:15:08.486 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:15:08.487+0000 7fad7e827700 1 -- 172.21.15.162:0/3036577334 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fad780fe530 con 0x7fad78102de0 2024-04-27T04:15:08.486 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:15:08.487+0000 7fad74ff9700 1 -- 172.21.15.162:0/3036577334 <== mon.0 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 38) v1 ==== 92849+0+0 (secure 0 0 0) 0x7fad6800cc80 con 0x7fad78102de0 2024-04-27T04:15:08.486 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:15:08.488+0000 7fad74ff9700 1 --2- 172.21.15.162:0/3036577334 >> [v2:172.21.15.162:6800/1396866108,v1:172.21.15.162:6801/1396866108] conn(0x7fad6006ed30 0x7fad600711e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:15:08.487 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:15:08.488+0000 7fad74ff9700 1 -- 172.21.15.162:0/3036577334 <== mon.0 v2:172.21.15.162:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6410+0+0 (secure 0 0 0) 0x7fad6808e6e0 con 0x7fad78102de0 2024-04-27T04:15:08.487 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:15:08.488+0000 7fad777fe700 1 --2- 172.21.15.162:0/3036577334 >> [v2:172.21.15.162:6800/1396866108,v1:172.21.15.162:6801/1396866108] conn(0x7fad6006ed30 0x7fad600711e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:15:08.487 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:15:08.488+0000 7fad777fe700 1 --2- 172.21.15.162:0/3036577334 >> [v2:172.21.15.162:6800/1396866108,v1:172.21.15.162:6801/1396866108] conn(0x7fad6006ed30 0x7fad600711e0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fad6c004890 tx=0x7fad6c009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:15:08.490 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:15:08.493+0000 7fad74ff9700 1 -- 172.21.15.162:0/3036577334 <== mon.0 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7fad6805aea0 con 0x7fad78102de0 2024-04-27T04:15:08.713 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:15:08.714+0000 7fad7e827700 1 -- 172.21.15.162:0/3036577334 --> [v2:172.21.15.162:6800/1396866108,v1:172.21.15.162:6801/1396866108] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fad78109f10 con 0x7fad6006ed30 2024-04-27T04:15:08.713 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:15:08.715+0000 7fad74ff9700 1 -- 172.21.15.162:0/3036577334 <== mgr.34104 v2:172.21.15.162:6800/1396866108 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+383 (secure 0 0 0) 0x7fad78109f10 con 0x7fad6006ed30 2024-04-27T04:15:08.716 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:15:08.718+0000 7fad7e827700 1 -- 172.21.15.162:0/3036577334 >> [v2:172.21.15.162:6800/1396866108,v1:172.21.15.162:6801/1396866108] conn(0x7fad6006ed30 msgr2=0x7fad600711e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:15:08.716 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:15:08.718+0000 7fad7e827700 1 --2- 172.21.15.162:0/3036577334 >> [v2:172.21.15.162:6800/1396866108,v1:172.21.15.162:6801/1396866108] conn(0x7fad6006ed30 0x7fad600711e0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fad6c004890 tx=0x7fad6c009e30 comp rx=0 tx=0).stop 2024-04-27T04:15:08.716 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:15:08.718+0000 7fad7e827700 1 -- 172.21.15.162:0/3036577334 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fad78102de0 msgr2=0x7fad781138d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:15:08.716 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:15:08.718+0000 7fad7e827700 1 --2- 172.21.15.162:0/3036577334 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fad78102de0 0x7fad781138d0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fad6800eb40 tx=0x7fad6800b040 comp rx=0 tx=0).stop 2024-04-27T04:15:08.716 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:15:08.718+0000 7fad7e827700 1 -- 172.21.15.162:0/3036577334 shutdown_connections 2024-04-27T04:15:08.716 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:15:08.718+0000 7fad7e827700 1 --2- 172.21.15.162:0/3036577334 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fad78102de0 0x7fad781138d0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:15:08.716 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:15:08.718+0000 7fad7e827700 1 --2- 172.21.15.162:0/3036577334 >> [v2:172.21.15.162:6800/1396866108,v1:172.21.15.162:6801/1396866108] conn(0x7fad6006ed30 0x7fad600711e0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:15:08.716 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:15:08.718+0000 7fad7e827700 1 --2- 172.21.15.162:0/3036577334 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fad78103780 0x7fad78113e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:15:08.716 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:15:08.718+0000 7fad7e827700 1 -- 172.21.15.162:0/3036577334 >> 172.21.15.162:0/3036577334 conn(0x7fad780f8990 msgr2=0x7fad780f98a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:15:08.716 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:15:08.718+0000 7fad7e827700 1 -- 172.21.15.162:0/3036577334 shutdown_connections 2024-04-27T04:15:08.717 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:15:08.718+0000 7fad7e827700 1 -- 172.21.15.162:0/3036577334 wait complete. 2024-04-27T04:15:08.726 INFO:teuthology.orchestra.run.smithi162.stdout:true 2024-04-27T04:15:08.820 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:15:08.820+0000 7fbb80fee700 1 -- 172.21.15.162:0/485478413 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fbb7c0fb520 msgr2=0x7fbb7c0fb990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:15:08.820 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:15:08.820+0000 7fbb80fee700 1 --2- 172.21.15.162:0/485478413 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fbb7c0fb520 0x7fbb7c0fb990 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fbb6c004940 tx=0x7fbb6c009e30 comp rx=0 tx=0).stop 2024-04-27T04:15:08.820 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:15:08.820+0000 7fbb80fee700 1 -- 172.21.15.162:0/485478413 shutdown_connections 2024-04-27T04:15:08.820 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:15:08.820+0000 7fbb80fee700 1 --2- 172.21.15.162:0/485478413 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fbb7c0fb520 0x7fbb7c0fb990 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:15:08.820 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:15:08.820+0000 7fbb80fee700 1 --2- 172.21.15.162:0/485478413 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbb7c102f60 0x7fbb7c103330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:15:08.820 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:15:08.820+0000 7fbb80fee700 1 -- 172.21.15.162:0/485478413 >> 172.21.15.162:0/485478413 conn(0x7fbb7c0714d0 msgr2=0x7fbb7c0718d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:15:08.820 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:15:08.820+0000 7fbb80fee700 1 -- 172.21.15.162:0/485478413 shutdown_connections 2024-04-27T04:15:08.821 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:15:08.821+0000 7fbb80fee700 1 -- 172.21.15.162:0/485478413 wait complete. 2024-04-27T04:15:08.821 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:15:08.821+0000 7fbb80fee700 1 Processor -- start 2024-04-27T04:15:08.821 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:15:08.821+0000 7fbb80fee700 1 -- start start 2024-04-27T04:15:08.821 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:15:08.821+0000 7fbb80fee700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbb7c0fb520 0x7fbb7c113a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:15:08.821 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:15:08.821+0000 7fbb80fee700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fbb7c102f60 0x7fbb7c113f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:15:08.821 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:15:08.821+0000 7fbb80fee700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb7c114590 con 0x7fbb7c102f60 2024-04-27T04:15:08.821 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:15:08.821+0000 7fbb80fee700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb7c10dbb0 con 0x7fbb7c0fb520 2024-04-27T04:15:08.821 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:15:08.821+0000 7fbb79d9b700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fbb7c102f60 0x7fbb7c113f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:15:08.821 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:15:08.821+0000 7fbb79d9b700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fbb7c102f60 0x7fbb7c113f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:33704/0 (socket says 172.21.15.162:33704) 2024-04-27T04:15:08.821 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:15:08.821+0000 7fbb79d9b700 1 -- 172.21.15.162:0/4070709443 learned_addr learned my addr 172.21.15.162:0/4070709443 (peer_addr_for_me v2:172.21.15.162:0/0) 2024-04-27T04:15:08.821 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:15:08.821+0000 7fbb7a59c700 1 --2- 172.21.15.162:0/4070709443 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbb7c0fb520 0x7fbb7c113a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:15:08.821 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:15:08.822+0000 7fbb79d9b700 1 -- 172.21.15.162:0/4070709443 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbb7c0fb520 msgr2=0x7fbb7c113a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:15:08.821 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:15:08.822+0000 7fbb79d9b700 1 --2- 172.21.15.162:0/4070709443 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbb7c0fb520 0x7fbb7c113a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:15:08.821 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:15:08.822+0000 7fbb79d9b700 1 -- 172.21.15.162:0/4070709443 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb6c005040 con 0x7fbb7c102f60 2024-04-27T04:15:08.822 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:15:08.822+0000 7fbb79d9b700 1 --2- 172.21.15.162:0/4070709443 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fbb7c102f60 0x7fbb7c113f40 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fbb6c009150 tx=0x7fbb6c009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:15:08.822 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:15:08.822+0000 7fbb6affd700 1 -- 172.21.15.162:0/4070709443 <== mon.0 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbb6c021070 con 0x7fbb7c102f60 2024-04-27T04:15:08.822 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:15:08.822+0000 7fbb80fee700 1 -- 172.21.15.162:0/4070709443 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbb7c10de30 con 0x7fbb7c102f60 2024-04-27T04:15:08.822 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:15:08.822+0000 7fbb6affd700 1 -- 172.21.15.162:0/4070709443 <== mon.0 v2:172.21.15.162:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fbb6c013c00 con 0x7fbb7c102f60 2024-04-27T04:15:08.822 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:15:08.822+0000 7fbb6affd700 1 -- 172.21.15.162:0/4070709443 <== mon.0 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbb6c00e070 con 0x7fbb7c102f60 2024-04-27T04:15:08.822 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:15:08.822+0000 7fbb80fee700 1 -- 172.21.15.162:0/4070709443 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbb7c10e310 con 0x7fbb7c102f60 2024-04-27T04:15:08.823 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:15:08.823+0000 7fbb80fee700 1 -- 172.21.15.162:0/4070709443 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbb7c100250 con 0x7fbb7c102f60 2024-04-27T04:15:08.823 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:15:08.823+0000 7fbb6affd700 1 -- 172.21.15.162:0/4070709443 <== mon.0 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 38) v1 ==== 92849+0+0 (secure 0 0 0) 0x7fbb6c01c400 con 0x7fbb7c102f60 2024-04-27T04:15:08.823 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:15:08.824+0000 7fbb6affd700 1 --2- 172.21.15.162:0/4070709443 >> [v2:172.21.15.162:6800/1396866108,v1:172.21.15.162:6801/1396866108] conn(0x7fbb6406ed30 0x7fbb640711e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:15:08.823 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:15:08.824+0000 7fbb6affd700 1 -- 172.21.15.162:0/4070709443 <== mon.0 v2:172.21.15.162:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6410+0+0 (secure 0 0 0) 0x7fbb6c08fed0 con 0x7fbb7c102f60 2024-04-27T04:15:08.823 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:15:08.824+0000 7fbb7a59c700 1 --2- 172.21.15.162:0/4070709443 >> [v2:172.21.15.162:6800/1396866108,v1:172.21.15.162:6801/1396866108] conn(0x7fbb6406ed30 0x7fbb640711e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:15:08.823 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:15:08.824+0000 7fbb7a59c700 1 --2- 172.21.15.162:0/4070709443 >> [v2:172.21.15.162:6800/1396866108,v1:172.21.15.162:6801/1396866108] conn(0x7fbb6406ed30 0x7fbb640711e0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fbb70009b80 tx=0x7fbb7000b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:15:08.826 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:15:08.829+0000 7fbb6affd700 1 -- 172.21.15.162:0/4070709443 <== mon.0 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7fbb6c058b30 con 0x7fbb7c102f60 2024-04-27T04:15:08.922 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:15:08 smithi181 ceph-mon[81134]: pgmap v28: 65 pgs: 65 active+clean; 208 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:15:08.922 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:15:08 smithi181 ceph-mon[81134]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' 2024-04-27T04:15:08.922 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:15:08 smithi181 ceph-mon[81134]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' 2024-04-27T04:15:08.922 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:15:08 smithi181 ceph-mon[81134]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi181.bkkcdm", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-27T04:15:08.922 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:15:08 smithi181 ceph-mon[81134]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T04:15:09.044 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:15:09.046+0000 7fbb80fee700 1 -- 172.21.15.162:0/4070709443 --> [v2:172.21.15.162:6800/1396866108,v1:172.21.15.162:6801/1396866108] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fbb7c10f090 con 0x7fbb6406ed30 2024-04-27T04:15:09.044 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:15:08 smithi162 ceph-mon[110641]: pgmap v28: 65 pgs: 65 active+clean; 208 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:15:09.045 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:15:08 smithi162 ceph-mon[110641]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' 2024-04-27T04:15:09.045 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:15:08 smithi162 ceph-mon[110641]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' 2024-04-27T04:15:09.045 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:15:08 smithi162 ceph-mon[110641]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi181.bkkcdm", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-27T04:15:09.045 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:15:08 smithi162 ceph-mon[110641]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T04:15:09.046 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:15:09.048+0000 7fbb6affd700 1 -- 172.21.15.162:0/4070709443 <== mgr.34104 v2:172.21.15.162:6800/1396866108 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+383 (secure 0 0 0) 0x7fbb7c10f090 con 0x7fbb6406ed30 2024-04-27T04:15:09.048 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:15:09.050+0000 7fbb80fee700 1 -- 172.21.15.162:0/4070709443 >> [v2:172.21.15.162:6800/1396866108,v1:172.21.15.162:6801/1396866108] conn(0x7fbb6406ed30 msgr2=0x7fbb640711e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:15:09.048 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:15:09.050+0000 7fbb80fee700 1 --2- 172.21.15.162:0/4070709443 >> [v2:172.21.15.162:6800/1396866108,v1:172.21.15.162:6801/1396866108] conn(0x7fbb6406ed30 0x7fbb640711e0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fbb70009b80 tx=0x7fbb7000b040 comp rx=0 tx=0).stop 2024-04-27T04:15:09.049 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:15:09.050+0000 7fbb80fee700 1 -- 172.21.15.162:0/4070709443 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fbb7c102f60 msgr2=0x7fbb7c113f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:15:09.049 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:15:09.050+0000 7fbb80fee700 1 --2- 172.21.15.162:0/4070709443 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fbb7c102f60 0x7fbb7c113f40 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fbb6c009150 tx=0x7fbb6c009e30 comp rx=0 tx=0).stop 2024-04-27T04:15:09.049 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:15:09.050+0000 7fbb80fee700 1 -- 172.21.15.162:0/4070709443 shutdown_connections 2024-04-27T04:15:09.049 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:15:09.050+0000 7fbb80fee700 1 --2- 172.21.15.162:0/4070709443 >> [v2:172.21.15.162:6800/1396866108,v1:172.21.15.162:6801/1396866108] conn(0x7fbb6406ed30 0x7fbb640711e0 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:15:09.049 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:15:09.050+0000 7fbb80fee700 1 --2- 172.21.15.162:0/4070709443 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbb7c0fb520 0x7fbb7c113a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:15:09.049 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:15:09.050+0000 7fbb80fee700 1 --2- 172.21.15.162:0/4070709443 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fbb7c102f60 0x7fbb7c113f40 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:15:09.049 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:15:09.050+0000 7fbb80fee700 1 -- 172.21.15.162:0/4070709443 >> 172.21.15.162:0/4070709443 conn(0x7fbb7c0714d0 msgr2=0x7fbb7c0fa850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:15:09.049 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:15:09.050+0000 7fbb80fee700 1 -- 172.21.15.162:0/4070709443 shutdown_connections 2024-04-27T04:15:09.049 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:15:09.050+0000 7fbb80fee700 1 -- 172.21.15.162:0/4070709443 wait complete. 2024-04-27T04:15:09.152 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:15:09.153+0000 7fa2e8fff700 1 -- 172.21.15.162:0/1659506164 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa2e4101cd0 msgr2=0x7fa2e41020a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:15:09.152 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:15:09.153+0000 7fa2e8fff700 1 --2- 172.21.15.162:0/1659506164 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa2e4101cd0 0x7fa2e41020a0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fa2d8004940 tx=0x7fa2d8009e30 comp rx=0 tx=0).stop 2024-04-27T04:15:09.152 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:15:09.153+0000 7fa2e8fff700 1 -- 172.21.15.162:0/1659506164 shutdown_connections 2024-04-27T04:15:09.152 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:15:09.153+0000 7fa2e8fff700 1 --2- 172.21.15.162:0/1659506164 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa2e4102670 0x7fa2e40fb600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:15:09.152 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:15:09.153+0000 7fa2e8fff700 1 --2- 172.21.15.162:0/1659506164 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa2e4101cd0 0x7fa2e41020a0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:15:09.152 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:15:09.153+0000 7fa2e8fff700 1 -- 172.21.15.162:0/1659506164 >> 172.21.15.162:0/1659506164 conn(0x7fa2e4071460 msgr2=0x7fa2e4071860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:15:09.152 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:15:09.153+0000 7fa2e8fff700 1 -- 172.21.15.162:0/1659506164 shutdown_connections 2024-04-27T04:15:09.152 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:15:09.153+0000 7fa2e8fff700 1 -- 172.21.15.162:0/1659506164 wait complete. 2024-04-27T04:15:09.152 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:15:09.154+0000 7fa2e8fff700 1 Processor -- start 2024-04-27T04:15:09.152 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:15:09.154+0000 7fa2e8fff700 1 -- start start 2024-04-27T04:15:09.154 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:15:09.154+0000 7fa2e8fff700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa2e4101cd0 0x7fa2e4113af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:15:09.154 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:15:09.154+0000 7fa2e8fff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa2e4102670 0x7fa2e4114030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:15:09.154 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:15:09.154+0000 7fa2e8fff700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2e4114680 con 0x7fa2e4101cd0 2024-04-27T04:15:09.154 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:15:09.154+0000 7fa2e8fff700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2e410dac0 con 0x7fa2e4102670 2024-04-27T04:15:09.154 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:15:09.154+0000 7fa2e259c700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa2e4101cd0 0x7fa2e4113af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:15:09.155 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:15:09.154+0000 7fa2e259c700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa2e4101cd0 0x7fa2e4113af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:33710/0 (socket says 172.21.15.162:33710) 2024-04-27T04:15:09.155 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:15:09.154+0000 7fa2e259c700 1 -- 172.21.15.162:0/2305244880 learned_addr learned my addr 172.21.15.162:0/2305244880 (peer_addr_for_me v2:172.21.15.162:0/0) 2024-04-27T04:15:09.155 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:15:09.154+0000 7fa2e1d9b700 1 --2- 172.21.15.162:0/2305244880 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa2e4102670 0x7fa2e4114030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:15:09.155 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:15:09.155+0000 7fa2e259c700 1 -- 172.21.15.162:0/2305244880 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa2e4102670 msgr2=0x7fa2e4114030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:15:09.155 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:15:09.155+0000 7fa2e259c700 1 --2- 172.21.15.162:0/2305244880 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa2e4102670 0x7fa2e4114030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:15:09.155 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:15:09.155+0000 7fa2e259c700 1 -- 172.21.15.162:0/2305244880 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2d8005040 con 0x7fa2e4101cd0 2024-04-27T04:15:09.155 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:15:09.155+0000 7fa2e259c700 1 --2- 172.21.15.162:0/2305244880 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa2e4101cd0 0x7fa2e4113af0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fa2d8009b30 tx=0x7fa2d8009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:15:09.155 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:15:09.155+0000 7fa2d2ffd700 1 -- 172.21.15.162:0/2305244880 <== mon.0 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa2d8021070 con 0x7fa2e4101cd0 2024-04-27T04:15:09.155 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:15:09.155+0000 7fa2d2ffd700 1 -- 172.21.15.162:0/2305244880 <== mon.0 v2:172.21.15.162:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fa2d8013c00 con 0x7fa2e4101cd0 2024-04-27T04:15:09.155 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:15:09.155+0000 7fa2e8fff700 1 -- 172.21.15.162:0/2305244880 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa2e410dd40 con 0x7fa2e4101cd0 2024-04-27T04:15:09.155 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:15:09.155+0000 7fa2d2ffd700 1 -- 172.21.15.162:0/2305244880 <== mon.0 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa2d800e070 con 0x7fa2e4101cd0 2024-04-27T04:15:09.155 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:15:09.155+0000 7fa2e8fff700 1 -- 172.21.15.162:0/2305244880 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa2e410e220 con 0x7fa2e4101cd0 2024-04-27T04:15:09.155 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:15:09.156+0000 7fa2e8fff700 1 -- 172.21.15.162:0/2305244880 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa2e41052e0 con 0x7fa2e4101cd0 2024-04-27T04:15:09.155 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:15:09.156+0000 7fa2d2ffd700 1 -- 172.21.15.162:0/2305244880 <== mon.0 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 38) v1 ==== 92849+0+0 (secure 0 0 0) 0x7fa2d800b790 con 0x7fa2e4101cd0 2024-04-27T04:15:09.156 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:15:09.157+0000 7fa2d2ffd700 1 --2- 172.21.15.162:0/2305244880 >> [v2:172.21.15.162:6800/1396866108,v1:172.21.15.162:6801/1396866108] conn(0x7fa2cc06ed20 0x7fa2cc0711d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:15:09.156 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:15:09.157+0000 7fa2d2ffd700 1 -- 172.21.15.162:0/2305244880 <== mon.0 v2:172.21.15.162:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6410+0+0 (secure 0 0 0) 0x7fa2d808ff20 con 0x7fa2e4101cd0 2024-04-27T04:15:09.156 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:15:09.157+0000 7fa2e1d9b700 1 --2- 172.21.15.162:0/2305244880 >> [v2:172.21.15.162:6800/1396866108,v1:172.21.15.162:6801/1396866108] conn(0x7fa2cc06ed20 0x7fa2cc0711d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:15:09.156 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:15:09.157+0000 7fa2e1d9b700 1 --2- 172.21.15.162:0/2305244880 >> [v2:172.21.15.162:6800/1396866108,v1:172.21.15.162:6801/1396866108] conn(0x7fa2cc06ed20 0x7fa2cc0711d0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fa2e410ec30 tx=0x7fa2d400b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:15:09.159 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:15:09.161+0000 7fa2d2ffd700 1 -- 172.21.15.162:0/2305244880 <== mon.0 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7fa2d80c4ff0 con 0x7fa2e4101cd0 2024-04-27T04:15:09.386 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:15:09.388+0000 7fa2e8fff700 1 -- 172.21.15.162:0/2305244880 --> [v2:172.21.15.162:6800/1396866108,v1:172.21.15.162:6801/1396866108] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7fa2e410f140 con 0x7fa2cc06ed20 2024-04-27T04:15:09.396 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:15:09.398+0000 7fa2d2ffd700 1 -- 172.21.15.162:0/2305244880 <== mgr.34104 v2:172.21.15.162:6800/1396866108 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3520 (secure 0 0 0) 0x7fa2e410f140 con 0x7fa2cc06ed20 2024-04-27T04:15:09.396 INFO:teuthology.orchestra.run.smithi162.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-27T04:15:09.396 INFO:teuthology.orchestra.run.smithi162.stdout:alertmanager.smithi162 smithi162 *:9093,9094 running (2m) 46s ago 13m 16.9M - 0.25.0 c8568f914cd2 40e334659017 2024-04-27T04:15:09.396 INFO:teuthology.orchestra.run.smithi162.stdout:crash.smithi162 smithi162 running (13m) 46s ago 13m 6660k - 17.2.7-907-g2be3efa9 1c5834633715 228c98df4289 2024-04-27T04:15:09.396 INFO:teuthology.orchestra.run.smithi162.stdout:crash.smithi181 smithi181 running (12m) 30s ago 12m 6665k - 17.2.7-907-g2be3efa9 1c5834633715 c8e553e3df2b 2024-04-27T04:15:09.396 INFO:teuthology.orchestra.run.smithi162.stdout:grafana.smithi162 smithi162 *:3000 running (114s) 46s ago 13m 75.4M - 9.4.12 2bacad6d85d8 3d78c478f03f 2024-04-27T04:15:09.397 INFO:teuthology.orchestra.run.smithi162.stdout:mds.cephfs.smithi162.gzlyjj smithi162 running (8m) 46s ago 8m 13.4M - 17.2.7-907-g2be3efa9 1c5834633715 27e5b030dd32 2024-04-27T04:15:09.397 INFO:teuthology.orchestra.run.smithi162.stdout:mds.cephfs.smithi162.xcnapk smithi162 running (8m) 46s ago 8m 100M - 17.2.7-907-g2be3efa9 1c5834633715 38f9030df029 2024-04-27T04:15:09.397 INFO:teuthology.orchestra.run.smithi162.stdout:mds.cephfs.smithi181.bkkcdm smithi181 running (8m) 30s ago 8m 15.1M - 17.2.7-907-g2be3efa9 1c5834633715 af55afca1263 2024-04-27T04:15:09.397 INFO:teuthology.orchestra.run.smithi162.stdout:mds.cephfs.smithi181.dzomgh smithi181 running (8m) 30s ago 8m 143M - 17.2.7-907-g2be3efa9 1c5834633715 6733296597f3 2024-04-27T04:15:09.397 INFO:teuthology.orchestra.run.smithi162.stdout:mgr.smithi162.oennfn smithi162 *:8443,9283,8765 running (3m) 46s ago 14m 475M - 19.0.0-3253-g4e213a58 67e414ff4bc4 58a748d3637b 2024-04-27T04:15:09.397 INFO:teuthology.orchestra.run.smithi162.stdout:mgr.smithi181.vtwuhz smithi181 *:8443,9283,8765 running (3m) 30s ago 12m 449M - 19.0.0-3253-g4e213a58 67e414ff4bc4 813584297b1d 2024-04-27T04:15:09.397 INFO:teuthology.orchestra.run.smithi162.stdout:mon.smithi162 smithi162 running (60s) 46s ago 15m 27.1M 2048M 19.0.0-3253-g4e213a58 67e414ff4bc4 375adaf4468f 2024-04-27T04:15:09.397 INFO:teuthology.orchestra.run.smithi162.stdout:mon.smithi181 smithi181 running (35s) 30s ago 12m 22.0M 2048M 19.0.0-3253-g4e213a58 67e414ff4bc4 77b71f6d522f 2024-04-27T04:15:09.397 INFO:teuthology.orchestra.run.smithi162.stdout:node-exporter.smithi162 smithi162 *:9100 running (2m) 46s ago 13m 15.6M - 1.5.0 0da6a335fe13 5c4c5ea9c1d7 2024-04-27T04:15:09.397 INFO:teuthology.orchestra.run.smithi162.stdout:node-exporter.smithi181 smithi181 *:9100 running (2m) 30s ago 12m 15.8M - 1.5.0 0da6a335fe13 9067f14c0519 2024-04-27T04:15:09.397 INFO:teuthology.orchestra.run.smithi162.stdout:osd.0 smithi162 running (11m) 46s ago 11m 231M 1557M 17.2.7-907-g2be3efa9 1c5834633715 4a9df3aa938d 2024-04-27T04:15:09.397 INFO:teuthology.orchestra.run.smithi162.stdout:osd.1 smithi162 running (11m) 46s ago 11m 246M 1557M 17.2.7-907-g2be3efa9 1c5834633715 aefb464c46e8 2024-04-27T04:15:09.397 INFO:teuthology.orchestra.run.smithi162.stdout:osd.2 smithi162 running (10m) 46s ago 10m 205M 1557M 17.2.7-907-g2be3efa9 1c5834633715 70f16d86d38e 2024-04-27T04:15:09.397 INFO:teuthology.orchestra.run.smithi162.stdout:osd.3 smithi181 running (10m) 30s ago 10m 277M 2581M 17.2.7-907-g2be3efa9 1c5834633715 030d829be54e 2024-04-27T04:15:09.398 INFO:teuthology.orchestra.run.smithi162.stdout:osd.4 smithi181 running (10m) 30s ago 10m 246M 2581M 17.2.7-907-g2be3efa9 1c5834633715 512afbee7cf7 2024-04-27T04:15:09.398 INFO:teuthology.orchestra.run.smithi162.stdout:osd.5 smithi181 running (9m) 30s ago 9m 224M 2581M 17.2.7-907-g2be3efa9 1c5834633715 f47a1daac1ea 2024-04-27T04:15:09.398 INFO:teuthology.orchestra.run.smithi162.stdout:prometheus.smithi162 smithi162 *:9095 running (2m) 46s ago 13m 56.4M - 2.43.0 a07b618ecd1d f30657a43324 2024-04-27T04:15:09.398 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:15:09.400+0000 7fa2e8fff700 1 -- 172.21.15.162:0/2305244880 >> [v2:172.21.15.162:6800/1396866108,v1:172.21.15.162:6801/1396866108] conn(0x7fa2cc06ed20 msgr2=0x7fa2cc0711d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:15:09.398 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:15:09.400+0000 7fa2e8fff700 1 --2- 172.21.15.162:0/2305244880 >> [v2:172.21.15.162:6800/1396866108,v1:172.21.15.162:6801/1396866108] conn(0x7fa2cc06ed20 0x7fa2cc0711d0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fa2e410ec30 tx=0x7fa2d400b040 comp rx=0 tx=0).stop 2024-04-27T04:15:09.399 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:15:09.400+0000 7fa2e8fff700 1 -- 172.21.15.162:0/2305244880 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa2e4101cd0 msgr2=0x7fa2e4113af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:15:09.399 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:15:09.400+0000 7fa2e8fff700 1 --2- 172.21.15.162:0/2305244880 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa2e4101cd0 0x7fa2e4113af0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fa2d8009b30 tx=0x7fa2d8009e30 comp rx=0 tx=0).stop 2024-04-27T04:15:09.399 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:15:09.400+0000 7fa2e8fff700 1 -- 172.21.15.162:0/2305244880 shutdown_connections 2024-04-27T04:15:09.399 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:15:09.400+0000 7fa2e8fff700 1 --2- 172.21.15.162:0/2305244880 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa2e4101cd0 0x7fa2e4113af0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:15:09.399 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:15:09.400+0000 7fa2e8fff700 1 --2- 172.21.15.162:0/2305244880 >> [v2:172.21.15.162:6800/1396866108,v1:172.21.15.162:6801/1396866108] conn(0x7fa2cc06ed20 0x7fa2cc0711d0 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:15:09.399 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:15:09.400+0000 7fa2e8fff700 1 --2- 172.21.15.162:0/2305244880 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa2e4102670 0x7fa2e4114030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:15:09.399 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:15:09.400+0000 7fa2e8fff700 1 -- 172.21.15.162:0/2305244880 >> 172.21.15.162:0/2305244880 conn(0x7fa2e4071460 msgr2=0x7fa2e40fabe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:15:09.399 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:15:09.400+0000 7fa2e8fff700 1 -- 172.21.15.162:0/2305244880 shutdown_connections 2024-04-27T04:15:09.399 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:15:09.400+0000 7fa2e8fff700 1 -- 172.21.15.162:0/2305244880 wait complete. 2024-04-27T04:15:09.505 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:15:09.505+0000 7ffaf9a6d700 1 -- 172.21.15.162:0/2844308778 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ffaf40fbd40 msgr2=0x7ffaf40fc1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:15:09.506 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:15:09.505+0000 7ffaf9a6d700 1 --2- 172.21.15.162:0/2844308778 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ffaf40fbd40 0x7ffaf40fc1b0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7ffae8004940 tx=0x7ffae8009e30 comp rx=0 tx=0).stop 2024-04-27T04:15:09.506 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:15:09.505+0000 7ffaf9a6d700 1 -- 172.21.15.162:0/2844308778 shutdown_connections 2024-04-27T04:15:09.506 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:15:09.505+0000 7ffaf9a6d700 1 --2- 172.21.15.162:0/2844308778 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ffaf40fbd40 0x7ffaf40fc1b0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:15:09.506 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:15:09.505+0000 7ffaf9a6d700 1 --2- 172.21.15.162:0/2844308778 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ffaf4101d90 0x7ffaf4102160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:15:09.506 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:15:09.505+0000 7ffaf9a6d700 1 -- 172.21.15.162:0/2844308778 >> 172.21.15.162:0/2844308778 conn(0x7ffaf4071d00 msgr2=0x7ffaf4072100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:15:09.507 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:15:09.505+0000 7ffaf9a6d700 1 -- 172.21.15.162:0/2844308778 shutdown_connections 2024-04-27T04:15:09.507 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:15:09.505+0000 7ffaf9a6d700 1 -- 172.21.15.162:0/2844308778 wait complete. 2024-04-27T04:15:09.507 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:15:09.505+0000 7ffaf9a6d700 1 Processor -- start 2024-04-27T04:15:09.507 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:15:09.505+0000 7ffaf9a6d700 1 -- start start 2024-04-27T04:15:09.509 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:15:09.506+0000 7ffaf9a6d700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ffaf40fbd40 0x7ffaf410ec00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:15:09.509 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:15:09.506+0000 7ffaf9a6d700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ffaf4101d90 0x7ffaf410f140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:15:09.509 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:15:09.506+0000 7ffaf9a6d700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffaf410f680 con 0x7ffaf40fbd40 2024-04-27T04:15:09.509 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:15:09.506+0000 7ffaf9a6d700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffaf410f7c0 con 0x7ffaf4101d90 2024-04-27T04:15:09.509 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:15:09.506+0000 7ffaf37fe700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ffaf40fbd40 0x7ffaf410ec00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:15:09.509 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:15:09.506+0000 7ffaf37fe700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ffaf40fbd40 0x7ffaf410ec00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:33722/0 (socket says 172.21.15.162:33722) 2024-04-27T04:15:09.509 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:15:09.506+0000 7ffaf2ffd700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ffaf4101d90 0x7ffaf410f140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:15:09.509 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:15:09.506+0000 7ffaf37fe700 1 -- 172.21.15.162:0/2815764875 learned_addr learned my addr 172.21.15.162:0/2815764875 (peer_addr_for_me v2:172.21.15.162:0/0) 2024-04-27T04:15:09.509 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:15:09.506+0000 7ffaf37fe700 1 -- 172.21.15.162:0/2815764875 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ffaf4101d90 msgr2=0x7ffaf410f140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:15:09.510 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:15:09.506+0000 7ffaf37fe700 1 --2- 172.21.15.162:0/2815764875 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ffaf4101d90 0x7ffaf410f140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:15:09.510 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:15:09.506+0000 7ffaf37fe700 1 -- 172.21.15.162:0/2815764875 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffae8005040 con 0x7ffaf40fbd40 2024-04-27T04:15:09.510 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:15:09.506+0000 7ffaf37fe700 1 --2- 172.21.15.162:0/2815764875 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ffaf40fbd40 0x7ffaf410ec00 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7ffae400eb40 tx=0x7ffae400b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:15:09.510 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:15:09.506+0000 7ffadbfff700 1 -- 172.21.15.162:0/2815764875 <== mon.0 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ffae4007cc0 con 0x7ffaf40fbd40 2024-04-27T04:15:09.510 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:15:09.506+0000 7ffaf9a6d700 1 -- 172.21.15.162:0/2815764875 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffaf4117ab0 con 0x7ffaf40fbd40 2024-04-27T04:15:09.510 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:15:09.507+0000 7ffadbfff700 1 -- 172.21.15.162:0/2815764875 <== mon.0 v2:172.21.15.162:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7ffae400f040 con 0x7ffaf40fbd40 2024-04-27T04:15:09.510 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:15:09.507+0000 7ffadbfff700 1 -- 172.21.15.162:0/2815764875 <== mon.0 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ffae400ca10 con 0x7ffaf40fbd40 2024-04-27T04:15:09.510 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:15:09.507+0000 7ffaf9a6d700 1 -- 172.21.15.162:0/2815764875 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffaf4117f60 con 0x7ffaf40fbd40 2024-04-27T04:15:09.510 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:15:09.507+0000 7ffaf9a6d700 1 -- 172.21.15.162:0/2815764875 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffaf40fd430 con 0x7ffaf40fbd40 2024-04-27T04:15:09.510 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:15:09.508+0000 7ffadbfff700 1 -- 172.21.15.162:0/2815764875 <== mon.0 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 38) v1 ==== 92849+0+0 (secure 0 0 0) 0x7ffae400cb70 con 0x7ffaf40fbd40 2024-04-27T04:15:09.510 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:15:09.509+0000 7ffadbfff700 1 --2- 172.21.15.162:0/2815764875 >> [v2:172.21.15.162:6800/1396866108,v1:172.21.15.162:6801/1396866108] conn(0x7ffadc06ed20 0x7ffadc0711d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:15:09.510 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:15:09.509+0000 7ffadbfff700 1 -- 172.21.15.162:0/2815764875 <== mon.0 v2:172.21.15.162:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6410+0+0 (secure 0 0 0) 0x7ffae408e780 con 0x7ffaf40fbd40 2024-04-27T04:15:09.510 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:15:09.509+0000 7ffaf2ffd700 1 --2- 172.21.15.162:0/2815764875 >> [v2:172.21.15.162:6800/1396866108,v1:172.21.15.162:6801/1396866108] conn(0x7ffadc06ed20 0x7ffadc0711d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:15:09.511 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:15:09.509+0000 7ffaf2ffd700 1 --2- 172.21.15.162:0/2815764875 >> [v2:172.21.15.162:6800/1396866108,v1:172.21.15.162:6801/1396866108] conn(0x7ffadc06ed20 0x7ffadc0711d0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7ffae8009b30 tx=0x7ffae8009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:15:09.511 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:15:09.513+0000 7ffadbfff700 1 -- 172.21.15.162:0/2815764875 <== mon.0 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7ffae4057530 con 0x7ffaf40fbd40 2024-04-27T04:15:09.824 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:15:09.823+0000 7ffaf9a6d700 1 -- 172.21.15.162:0/2815764875 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7ffaf404bf00 con 0x7ffaf40fbd40 2024-04-27T04:15:09.824 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:15:09.823+0000 7ffadbfff700 1 -- 172.21.15.162:0/2815764875 <== mon.0 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+736 (secure 0 0 0) 0x7ffae405abf0 con 0x7ffaf40fbd40 2024-04-27T04:15:09.825 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-04-27T04:15:09.825 INFO:teuthology.orchestra.run.smithi162.stdout: "mon": { 2024-04-27T04:15:09.825 INFO:teuthology.orchestra.run.smithi162.stdout: "ceph version 19.0.0-3253-g4e213a58 (4e213a58e787437ff508e6c890ccd5fd32dc96b5) squid (dev)": 2 2024-04-27T04:15:09.825 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-04-27T04:15:09.825 INFO:teuthology.orchestra.run.smithi162.stdout: "mgr": { 2024-04-27T04:15:09.825 INFO:teuthology.orchestra.run.smithi162.stdout: "ceph version 19.0.0-3253-g4e213a58 (4e213a58e787437ff508e6c890ccd5fd32dc96b5) squid (dev)": 2 2024-04-27T04:15:09.825 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-04-27T04:15:09.825 INFO:teuthology.orchestra.run.smithi162.stdout: "osd": { 2024-04-27T04:15:09.826 INFO:teuthology.orchestra.run.smithi162.stdout: "ceph version 17.2.7-907-g2be3efa9 (2be3efa97a7972f8e925bd899b474329979b74d6) quincy (stable)": 6 2024-04-27T04:15:09.826 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-04-27T04:15:09.826 INFO:teuthology.orchestra.run.smithi162.stdout: "mds": { 2024-04-27T04:15:09.826 INFO:teuthology.orchestra.run.smithi162.stdout: "ceph version 17.2.7-907-g2be3efa9 (2be3efa97a7972f8e925bd899b474329979b74d6) quincy (stable)": 4 2024-04-27T04:15:09.826 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-04-27T04:15:09.826 INFO:teuthology.orchestra.run.smithi162.stdout: "overall": { 2024-04-27T04:15:09.826 INFO:teuthology.orchestra.run.smithi162.stdout: "ceph version 17.2.7-907-g2be3efa9 (2be3efa97a7972f8e925bd899b474329979b74d6) quincy (stable)": 10, 2024-04-27T04:15:09.826 INFO:teuthology.orchestra.run.smithi162.stdout: "ceph version 19.0.0-3253-g4e213a58 (4e213a58e787437ff508e6c890ccd5fd32dc96b5) squid (dev)": 4 2024-04-27T04:15:09.827 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-04-27T04:15:09.827 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-04-27T04:15:09.827 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:15:09.825+0000 7ffaf9a6d700 1 -- 172.21.15.162:0/2815764875 >> [v2:172.21.15.162:6800/1396866108,v1:172.21.15.162:6801/1396866108] conn(0x7ffadc06ed20 msgr2=0x7ffadc0711d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:15:09.827 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:15:09.826+0000 7ffaf9a6d700 1 --2- 172.21.15.162:0/2815764875 >> [v2:172.21.15.162:6800/1396866108,v1:172.21.15.162:6801/1396866108] conn(0x7ffadc06ed20 0x7ffadc0711d0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7ffae8009b30 tx=0x7ffae8009e30 comp rx=0 tx=0).stop 2024-04-27T04:15:09.828 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:15:09.826+0000 7ffaf9a6d700 1 -- 172.21.15.162:0/2815764875 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ffaf40fbd40 msgr2=0x7ffaf410ec00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:15:09.828 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:15:09.826+0000 7ffaf9a6d700 1 --2- 172.21.15.162:0/2815764875 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ffaf40fbd40 0x7ffaf410ec00 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7ffae400eb40 tx=0x7ffae400b040 comp rx=0 tx=0).stop 2024-04-27T04:15:09.828 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:15:09.826+0000 7ffaf9a6d700 1 -- 172.21.15.162:0/2815764875 shutdown_connections 2024-04-27T04:15:09.828 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:15:09.826+0000 7ffaf9a6d700 1 --2- 172.21.15.162:0/2815764875 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ffaf40fbd40 0x7ffaf410ec00 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:15:09.828 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:15:09.826+0000 7ffaf9a6d700 1 --2- 172.21.15.162:0/2815764875 >> [v2:172.21.15.162:6800/1396866108,v1:172.21.15.162:6801/1396866108] conn(0x7ffadc06ed20 0x7ffadc0711d0 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:15:09.828 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:15:09.826+0000 7ffaf9a6d700 1 --2- 172.21.15.162:0/2815764875 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ffaf4101d90 0x7ffaf410f140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:15:09.828 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:15:09.826+0000 7ffaf9a6d700 1 -- 172.21.15.162:0/2815764875 >> 172.21.15.162:0/2815764875 conn(0x7ffaf4071d00 msgr2=0x7ffaf40fa0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:15:09.828 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:15:09.826+0000 7ffaf9a6d700 1 -- 172.21.15.162:0/2815764875 shutdown_connections 2024-04-27T04:15:09.828 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:15:09.826+0000 7ffaf9a6d700 1 -- 172.21.15.162:0/2815764875 wait complete. 2024-04-27T04:15:09.925 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:15:09.926+0000 7f6e2be0a700 1 -- 172.21.15.162:0/3924039842 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6e240fb7f0 msgr2=0x7f6e240fbbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:15:09.925 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:15:09.926+0000 7f6e2be0a700 1 --2- 172.21.15.162:0/3924039842 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6e240fb7f0 0x7f6e240fbbc0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f6e14004940 tx=0x7f6e14009e30 comp rx=0 tx=0).stop 2024-04-27T04:15:09.925 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:15:09.926+0000 7f6e2be0a700 1 -- 172.21.15.162:0/3924039842 shutdown_connections 2024-04-27T04:15:09.925 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:15:09.926+0000 7f6e2be0a700 1 --2- 172.21.15.162:0/3924039842 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6e240fc190 0x7f6e24101a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:15:09.926 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:15:09.926+0000 7f6e2be0a700 1 --2- 172.21.15.162:0/3924039842 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6e240fb7f0 0x7f6e240fbbc0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:15:09.926 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:15:09.926+0000 7f6e2be0a700 1 -- 172.21.15.162:0/3924039842 >> 172.21.15.162:0/3924039842 conn(0x7f6e24071460 msgr2=0x7f6e24071860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:15:09.926 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:15:09.926+0000 7f6e2be0a700 1 -- 172.21.15.162:0/3924039842 shutdown_connections 2024-04-27T04:15:09.926 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:15:09.926+0000 7f6e2be0a700 1 -- 172.21.15.162:0/3924039842 wait complete. 2024-04-27T04:15:09.932 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:15:09.927+0000 7f6e2be0a700 1 Processor -- start 2024-04-27T04:15:09.933 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:15:09.927+0000 7f6e2be0a700 1 -- start start 2024-04-27T04:15:09.933 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:15:09.927+0000 7f6e2be0a700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6e240fb7f0 0x7f6e240fc780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:15:09.933 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:15:09.927+0000 7f6e2be0a700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6e240fc190 0x7f6e240fccc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:15:09.933 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:15:09.927+0000 7f6e2be0a700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e240fe720 con 0x7f6e240fb7f0 2024-04-27T04:15:09.933 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:15:09.927+0000 7f6e2be0a700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e240fd200 con 0x7f6e240fc190 2024-04-27T04:15:09.933 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:15:09.927+0000 7f6e29ba6700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6e240fb7f0 0x7f6e240fc780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:15:09.933 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:15:09.927+0000 7f6e29ba6700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6e240fb7f0 0x7f6e240fc780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:33740/0 (socket says 172.21.15.162:33740) 2024-04-27T04:15:09.933 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:15:09.927+0000 7f6e29ba6700 1 -- 172.21.15.162:0/1115238275 learned_addr learned my addr 172.21.15.162:0/1115238275 (peer_addr_for_me v2:172.21.15.162:0/0) 2024-04-27T04:15:09.933 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:15:09.927+0000 7f6e293a5700 1 --2- 172.21.15.162:0/1115238275 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6e240fc190 0x7f6e240fccc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:15:09.933 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:15:09.928+0000 7f6e29ba6700 1 -- 172.21.15.162:0/1115238275 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6e240fc190 msgr2=0x7f6e240fccc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:15:09.934 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:15:09.928+0000 7f6e29ba6700 1 --2- 172.21.15.162:0/1115238275 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6e240fc190 0x7f6e240fccc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:15:09.934 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:15:09.928+0000 7f6e29ba6700 1 -- 172.21.15.162:0/1115238275 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e14005040 con 0x7f6e240fb7f0 2024-04-27T04:15:09.934 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:15:09.928+0000 7f6e293a5700 1 --2- 172.21.15.162:0/1115238275 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6e240fc190 0x7f6e240fccc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-27T04:15:09.938 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:15:09.928+0000 7f6e29ba6700 1 --2- 172.21.15.162:0/1115238275 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6e240fb7f0 0x7f6e240fc780 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f6e1400b3b0 tx=0x7f6e14009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:15:09.938 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:15:09.928+0000 7f6e1a7fc700 1 -- 172.21.15.162:0/1115238275 <== mon.0 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6e1400e040 con 0x7f6e240fb7f0 2024-04-27T04:15:09.938 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:15:09.929+0000 7f6e1a7fc700 1 -- 172.21.15.162:0/1115238275 <== mon.0 v2:172.21.15.162:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f6e1400bbc0 con 0x7f6e240fb7f0 2024-04-27T04:15:09.938 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:15:09.929+0000 7f6e2be0a700 1 -- 172.21.15.162:0/1115238275 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6e240fd450 con 0x7f6e240fb7f0 2024-04-27T04:15:09.938 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:15:09.929+0000 7f6e1a7fc700 1 -- 172.21.15.162:0/1115238275 <== mon.0 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6e1401c630 con 0x7f6e240fb7f0 2024-04-27T04:15:09.938 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:15:09.929+0000 7f6e2be0a700 1 -- 172.21.15.162:0/1115238275 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6e2411a1b0 con 0x7f6e240fb7f0 2024-04-27T04:15:09.938 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:15:09.930+0000 7f6e1a7fc700 1 -- 172.21.15.162:0/1115238275 <== mon.0 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 38) v1 ==== 92849+0+0 (secure 0 0 0) 0x7f6e1400bd30 con 0x7f6e240fb7f0 2024-04-27T04:15:09.939 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:15:09.930+0000 7f6e2be0a700 1 -- 172.21.15.162:0/1115238275 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6e24065f30 con 0x7f6e240fb7f0 2024-04-27T04:15:09.939 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:15:09.931+0000 7f6e1a7fc700 1 --2- 172.21.15.162:0/1115238275 >> [v2:172.21.15.162:6800/1396866108,v1:172.21.15.162:6801/1396866108] conn(0x7f6e1006ed30 0x7f6e100711e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:15:09.939 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:15:09.931+0000 7f6e1a7fc700 1 -- 172.21.15.162:0/1115238275 <== mon.0 v2:172.21.15.162:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6410+0+0 (secure 0 0 0) 0x7f6e1408fd00 con 0x7f6e240fb7f0 2024-04-27T04:15:09.939 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:15:09.931+0000 7f6e293a5700 1 --2- 172.21.15.162:0/1115238275 >> [v2:172.21.15.162:6800/1396866108,v1:172.21.15.162:6801/1396866108] conn(0x7f6e1006ed30 0x7f6e100711e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:15:09.939 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:15:09.932+0000 7f6e293a5700 1 --2- 172.21.15.162:0/1115238275 >> [v2:172.21.15.162:6800/1396866108,v1:172.21.15.162:6801/1396866108] conn(0x7f6e1006ed30 0x7f6e100711e0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f6e20007fd0 tx=0x7f6e20009040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:15:09.939 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:15:09.936+0000 7f6e1a7fc700 1 -- 172.21.15.162:0/1115238275 <== mon.0 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7f6e14093030 con 0x7f6e240fb7f0 2024-04-27T04:15:10.044 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:15:09 smithi162 ceph-mon[110641]: Reconfiguring mds.cephfs.smithi181.bkkcdm (monmap changed)... 2024-04-27T04:15:10.045 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:15:09 smithi162 ceph-mon[110641]: Reconfiguring daemon mds.cephfs.smithi181.bkkcdm on smithi181 2024-04-27T04:15:10.045 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:15:09 smithi162 ceph-mon[110641]: from='client.34160 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-27T04:15:10.045 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:15:09 smithi162 ceph-mon[110641]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' 2024-04-27T04:15:10.045 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:15:09 smithi162 ceph-mon[110641]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' 2024-04-27T04:15:10.045 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:15:09 smithi162 ceph-mon[110641]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T04:15:10.045 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:15:09 smithi162 ceph-mon[110641]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "versions"}]: dispatch 2024-04-27T04:15:10.045 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:15:09 smithi162 ceph-mon[110641]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "versions"}]: dispatch 2024-04-27T04:15:10.045 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:15:09 smithi162 ceph-mon[110641]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' 2024-04-27T04:15:10.045 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:15:09 smithi162 ceph-mon[110641]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon.smithi162"}]: dispatch 2024-04-27T04:15:10.046 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:15:09 smithi162 ceph-mon[110641]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mon.smithi162"}]': finished 2024-04-27T04:15:10.046 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:15:09 smithi162 ceph-mon[110641]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon.smithi181"}]: dispatch 2024-04-27T04:15:10.046 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:15:09 smithi162 ceph-mon[110641]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mon.smithi181"}]': finished 2024-04-27T04:15:10.172 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:15:09 smithi181 ceph-mon[81134]: Reconfiguring mds.cephfs.smithi181.bkkcdm (monmap changed)... 2024-04-27T04:15:10.172 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:15:09 smithi181 ceph-mon[81134]: Reconfiguring daemon mds.cephfs.smithi181.bkkcdm on smithi181 2024-04-27T04:15:10.172 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:15:09 smithi181 ceph-mon[81134]: from='client.34160 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-27T04:15:10.172 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:15:09 smithi181 ceph-mon[81134]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' 2024-04-27T04:15:10.172 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:15:09 smithi181 ceph-mon[81134]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' 2024-04-27T04:15:10.173 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:15:09 smithi181 ceph-mon[81134]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T04:15:10.173 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:15:09 smithi181 ceph-mon[81134]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "versions"}]: dispatch 2024-04-27T04:15:10.173 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:15:09 smithi181 ceph-mon[81134]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "versions"}]: dispatch 2024-04-27T04:15:10.173 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:15:09 smithi181 ceph-mon[81134]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' 2024-04-27T04:15:10.173 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:15:09 smithi181 ceph-mon[81134]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon.smithi162"}]: dispatch 2024-04-27T04:15:10.173 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:15:09 smithi181 ceph-mon[81134]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mon.smithi162"}]': finished 2024-04-27T04:15:10.173 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:15:09 smithi181 ceph-mon[81134]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon.smithi181"}]: dispatch 2024-04-27T04:15:10.173 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:15:09 smithi181 ceph-mon[81134]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mon.smithi181"}]': finished 2024-04-27T04:15:10.196 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:15:10.193+0000 7f6e2be0a700 1 -- 172.21.15.162:0/1115238275 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f6e240fdc60 con 0x7f6e240fb7f0 2024-04-27T04:15:10.196 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:15:10.194+0000 7f6e1a7fc700 1 -- 172.21.15.162:0/1115238275 <== mon.0 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 15 v15) v1 ==== 76+0+1911 (secure 0 0 0) 0x7f6e14058960 con 0x7f6e240fb7f0 2024-04-27T04:15:10.197 INFO:teuthology.orchestra.run.smithi162.stdout:e15 2024-04-27T04:15:10.197 INFO:teuthology.orchestra.run.smithi162.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-04-27T04:15:10.197 INFO:teuthology.orchestra.run.smithi162.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-27T04:15:10.197 INFO:teuthology.orchestra.run.smithi162.stdout:legacy client fscid: 1 2024-04-27T04:15:10.197 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-04-27T04:15:10.197 INFO:teuthology.orchestra.run.smithi162.stdout:Filesystem 'cephfs' (1) 2024-04-27T04:15:10.197 INFO:teuthology.orchestra.run.smithi162.stdout:fs_name cephfs 2024-04-27T04:15:10.197 INFO:teuthology.orchestra.run.smithi162.stdout:epoch 15 2024-04-27T04:15:10.197 INFO:teuthology.orchestra.run.smithi162.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-04-27T04:15:10.197 INFO:teuthology.orchestra.run.smithi162.stdout:created 2024-04-27T04:06:53.095749+0000 2024-04-27T04:15:10.197 INFO:teuthology.orchestra.run.smithi162.stdout:modified 2024-04-27T04:07:11.285033+0000 2024-04-27T04:15:10.197 INFO:teuthology.orchestra.run.smithi162.stdout:tableserver 0 2024-04-27T04:15:10.197 INFO:teuthology.orchestra.run.smithi162.stdout:root 0 2024-04-27T04:15:10.197 INFO:teuthology.orchestra.run.smithi162.stdout:session_timeout 60 2024-04-27T04:15:10.198 INFO:teuthology.orchestra.run.smithi162.stdout:session_autoclose 300 2024-04-27T04:15:10.198 INFO:teuthology.orchestra.run.smithi162.stdout:max_file_size 1099511627776 2024-04-27T04:15:10.198 INFO:teuthology.orchestra.run.smithi162.stdout:max_xattr_size 65536 2024-04-27T04:15:10.198 INFO:teuthology.orchestra.run.smithi162.stdout:required_client_features {} 2024-04-27T04:15:10.198 INFO:teuthology.orchestra.run.smithi162.stdout:last_failure 0 2024-04-27T04:15:10.198 INFO:teuthology.orchestra.run.smithi162.stdout:last_failure_osd_epoch 46 2024-04-27T04:15:10.198 INFO:teuthology.orchestra.run.smithi162.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-27T04:15:10.198 INFO:teuthology.orchestra.run.smithi162.stdout:max_mds 1 2024-04-27T04:15:10.198 INFO:teuthology.orchestra.run.smithi162.stdout:in 0 2024-04-27T04:15:10.198 INFO:teuthology.orchestra.run.smithi162.stdout:up {0=24273} 2024-04-27T04:15:10.198 INFO:teuthology.orchestra.run.smithi162.stdout:failed 2024-04-27T04:15:10.198 INFO:teuthology.orchestra.run.smithi162.stdout:damaged 2024-04-27T04:15:10.198 INFO:teuthology.orchestra.run.smithi162.stdout:stopped 2024-04-27T04:15:10.198 INFO:teuthology.orchestra.run.smithi162.stdout:data_pools [3] 2024-04-27T04:15:10.199 INFO:teuthology.orchestra.run.smithi162.stdout:metadata_pool 2 2024-04-27T04:15:10.199 INFO:teuthology.orchestra.run.smithi162.stdout:inline_data disabled 2024-04-27T04:15:10.199 INFO:teuthology.orchestra.run.smithi162.stdout:balancer 2024-04-27T04:15:10.199 INFO:teuthology.orchestra.run.smithi162.stdout:bal_rank_mask -1 2024-04-27T04:15:10.199 INFO:teuthology.orchestra.run.smithi162.stdout:standby_count_wanted 1 2024-04-27T04:15:10.199 INFO:teuthology.orchestra.run.smithi162.stdout:qdb_cluster leader: 0 members: 2024-04-27T04:15:10.199 INFO:teuthology.orchestra.run.smithi162.stdout:[mds.cephfs.smithi181.dzomgh{0:24273} state up:active seq 6 join_fscid=1 addr [v2:172.21.15.181:6824/658214993,v1:172.21.15.181:6825/658214993] compat {c=[1],r=[1],i=[7ff]}] 2024-04-27T04:15:10.199 INFO:teuthology.orchestra.run.smithi162.stdout:[mds.cephfs.smithi162.xcnapk{0:14476} state up:standby-replay seq 2 join_fscid=1 addr [v2:172.21.15.162:6828/1296160676,v1:172.21.15.162:6829/1296160676] compat {c=[1],r=[1],i=[7ff]}] 2024-04-27T04:15:10.199 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-04-27T04:15:10.199 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-04-27T04:15:10.199 INFO:teuthology.orchestra.run.smithi162.stdout:Standby daemons: 2024-04-27T04:15:10.199 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-04-27T04:15:10.199 INFO:teuthology.orchestra.run.smithi162.stdout:[mds.cephfs.smithi162.gzlyjj{-1:14488} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.162:6826/2629103894,v1:172.21.15.162:6827/2629103894] compat {c=[1],r=[1],i=[7ff]}] 2024-04-27T04:15:10.199 INFO:teuthology.orchestra.run.smithi162.stdout:[mds.cephfs.smithi181.bkkcdm{-1:24287} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.181:6826/3756475453,v1:172.21.15.181:6827/3756475453] compat {c=[1],r=[1],i=[7ff]}] 2024-04-27T04:15:10.200 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:15:10.196+0000 7f6e2be0a700 1 -- 172.21.15.162:0/1115238275 >> [v2:172.21.15.162:6800/1396866108,v1:172.21.15.162:6801/1396866108] conn(0x7f6e1006ed30 msgr2=0x7f6e100711e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:15:10.200 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:15:10.196+0000 7f6e2be0a700 1 --2- 172.21.15.162:0/1115238275 >> [v2:172.21.15.162:6800/1396866108,v1:172.21.15.162:6801/1396866108] conn(0x7f6e1006ed30 0x7f6e100711e0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f6e20007fd0 tx=0x7f6e20009040 comp rx=0 tx=0).stop 2024-04-27T04:15:10.200 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:15:10.196+0000 7f6e2be0a700 1 -- 172.21.15.162:0/1115238275 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6e240fb7f0 msgr2=0x7f6e240fc780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:15:10.200 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:15:10.196+0000 7f6e2be0a700 1 --2- 172.21.15.162:0/1115238275 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6e240fb7f0 0x7f6e240fc780 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f6e1400b3b0 tx=0x7f6e14009e30 comp rx=0 tx=0).stop 2024-04-27T04:15:10.201 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:15:10.196+0000 7f6e2be0a700 1 -- 172.21.15.162:0/1115238275 shutdown_connections 2024-04-27T04:15:10.201 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:15:10.196+0000 7f6e2be0a700 1 --2- 172.21.15.162:0/1115238275 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6e240fb7f0 0x7f6e240fc780 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:15:10.201 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:15:10.196+0000 7f6e2be0a700 1 --2- 172.21.15.162:0/1115238275 >> [v2:172.21.15.162:6800/1396866108,v1:172.21.15.162:6801/1396866108] conn(0x7f6e1006ed30 0x7f6e100711e0 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:15:10.201 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:15:10.196+0000 7f6e2be0a700 1 --2- 172.21.15.162:0/1115238275 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6e240fc190 0x7f6e240fccc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:15:10.201 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:15:10.196+0000 7f6e2be0a700 1 -- 172.21.15.162:0/1115238275 >> 172.21.15.162:0/1115238275 conn(0x7f6e24071460 msgr2=0x7f6e240f93d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:15:10.201 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:15:10.196+0000 7f6e2be0a700 1 -- 172.21.15.162:0/1115238275 shutdown_connections 2024-04-27T04:15:10.201 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:15:10.196+0000 7f6e2be0a700 1 -- 172.21.15.162:0/1115238275 wait complete. 2024-04-27T04:15:10.201 INFO:teuthology.orchestra.run.smithi162.stderr:dumped fsmap epoch 15 2024-04-27T04:15:10.302 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:15:10.303+0000 7f9f1a291700 1 -- 172.21.15.162:0/4252158856 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9f14101d00 msgr2=0x7f9f141020d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:15:10.302 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:15:10.303+0000 7f9f1a291700 1 --2- 172.21.15.162:0/4252158856 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9f14101d00 0x7f9f141020d0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f9f08004880 tx=0x7f9f08009e30 comp rx=0 tx=0).stop 2024-04-27T04:15:10.302 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:15:10.303+0000 7f9f1a291700 1 -- 172.21.15.162:0/4252158856 shutdown_connections 2024-04-27T04:15:10.302 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:15:10.303+0000 7f9f1a291700 1 --2- 172.21.15.162:0/4252158856 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9f141026a0 0x7f9f140fbd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:15:10.302 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:15:10.303+0000 7f9f1a291700 1 --2- 172.21.15.162:0/4252158856 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9f14101d00 0x7f9f141020d0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:15:10.302 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:15:10.303+0000 7f9f1a291700 1 -- 172.21.15.162:0/4252158856 >> 172.21.15.162:0/4252158856 conn(0x7f9f140f78d0 msgr2=0x7f9f140f9ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:15:10.302 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:15:10.304+0000 7f9f1a291700 1 -- 172.21.15.162:0/4252158856 shutdown_connections 2024-04-27T04:15:10.302 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:15:10.304+0000 7f9f1a291700 1 -- 172.21.15.162:0/4252158856 wait complete. 2024-04-27T04:15:10.303 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:15:10.304+0000 7f9f1a291700 1 Processor -- start 2024-04-27T04:15:10.304 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:15:10.304+0000 7f9f1a291700 1 -- start start 2024-04-27T04:15:10.304 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:15:10.305+0000 7f9f1a291700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9f14101d00 0x7f9f14113ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:15:10.304 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:15:10.305+0000 7f9f1a291700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9f141026a0 0x7f9f141140e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:15:10.304 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:15:10.305+0000 7f9f1a291700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f14114730 con 0x7f9f141026a0 2024-04-27T04:15:10.304 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:15:10.305+0000 7f9f1a291700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f1410db40 con 0x7f9f14101d00 2024-04-27T04:15:10.304 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:15:10.305+0000 7f9f137fe700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9f141026a0 0x7f9f141140e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:15:10.304 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:15:10.305+0000 7f9f137fe700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9f141026a0 0x7f9f141140e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:33746/0 (socket says 172.21.15.162:33746) 2024-04-27T04:15:10.304 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:15:10.305+0000 7f9f13fff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9f14101d00 0x7f9f14113ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:15:10.304 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:15:10.305+0000 7f9f137fe700 1 -- 172.21.15.162:0/2189229050 learned_addr learned my addr 172.21.15.162:0/2189229050 (peer_addr_for_me v2:172.21.15.162:0/0) 2024-04-27T04:15:10.304 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:15:10.305+0000 7f9f13fff700 1 -- 172.21.15.162:0/2189229050 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9f141026a0 msgr2=0x7f9f141140e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:15:10.304 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:15:10.305+0000 7f9f13fff700 1 --2- 172.21.15.162:0/2189229050 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9f141026a0 0x7f9f141140e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:15:10.304 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:15:10.305+0000 7f9f13fff700 1 -- 172.21.15.162:0/2189229050 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f08005040 con 0x7f9f14101d00 2024-04-27T04:15:10.305 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:15:10.305+0000 7f9f137fe700 1 --2- 172.21.15.162:0/2189229050 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9f141026a0 0x7f9f141140e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-27T04:15:10.305 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:15:10.306+0000 7f9f13fff700 1 --2- 172.21.15.162:0/2189229050 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9f14101d00 0x7f9f14113ba0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f9f08013000 tx=0x7f9f08009e30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:15:10.307 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:15:10.306+0000 7f9f10ff9700 1 -- 172.21.15.162:0/2189229050 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9f0800e030 con 0x7f9f14101d00 2024-04-27T04:15:10.307 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:15:10.306+0000 7f9f10ff9700 1 -- 172.21.15.162:0/2189229050 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f9f08013d60 con 0x7f9f14101d00 2024-04-27T04:15:10.307 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:15:10.306+0000 7f9f1a291700 1 -- 172.21.15.162:0/2189229050 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9f1410ddc0 con 0x7f9f14101d00 2024-04-27T04:15:10.307 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:15:10.306+0000 7f9f10ff9700 1 -- 172.21.15.162:0/2189229050 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9f0801cd30 con 0x7f9f14101d00 2024-04-27T04:15:10.307 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:15:10.306+0000 7f9f1a291700 1 -- 172.21.15.162:0/2189229050 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9f1410e380 con 0x7f9f14101d00 2024-04-27T04:15:10.307 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:15:10.307+0000 7f9f1a291700 1 -- 172.21.15.162:0/2189229050 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9f140fd450 con 0x7f9f14101d00 2024-04-27T04:15:10.307 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:15:10.308+0000 7f9f10ff9700 1 -- 172.21.15.162:0/2189229050 <== mon.1 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 38) v1 ==== 92849+0+0 (secure 0 0 0) 0x7f9f08013890 con 0x7f9f14101d00 2024-04-27T04:15:10.307 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:15:10.308+0000 7f9f10ff9700 1 --2- 172.21.15.162:0/2189229050 >> [v2:172.21.15.162:6800/1396866108,v1:172.21.15.162:6801/1396866108] conn(0x7f9efc06ed20 0x7f9efc0711d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:15:10.307 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:15:10.308+0000 7f9f10ff9700 1 -- 172.21.15.162:0/2189229050 <== mon.1 v2:172.21.15.181:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6410+0+0 (secure 0 0 0) 0x7f9f08090110 con 0x7f9f14101d00 2024-04-27T04:15:10.307 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:15:10.308+0000 7f9f137fe700 1 --2- 172.21.15.162:0/2189229050 >> [v2:172.21.15.162:6800/1396866108,v1:172.21.15.162:6801/1396866108] conn(0x7f9efc06ed20 0x7f9efc0711d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:15:10.308 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:15:10.309+0000 7f9f137fe700 1 --2- 172.21.15.162:0/2189229050 >> [v2:172.21.15.162:6800/1396866108,v1:172.21.15.162:6801/1396866108] conn(0x7f9efc06ed20 0x7f9efc0711d0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f9f1410f440 tx=0x7f9f0400b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:15:10.312 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:15:10.314+0000 7f9f10ff9700 1 -- 172.21.15.162:0/2189229050 <== mon.1 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7f9f08058e30 con 0x7f9f14101d00 2024-04-27T04:15:10.536 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:15:10.538+0000 7f9f1a291700 1 -- 172.21.15.162:0/2189229050 --> [v2:172.21.15.162:6800/1396866108,v1:172.21.15.162:6801/1396866108] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f9f1404bf40 con 0x7f9efc06ed20 2024-04-27T04:15:10.537 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:15:10.539+0000 7f9f10ff9700 1 -- 172.21.15.162:0/2189229050 <== mgr.34104 v2:172.21.15.162:6800/1396866108 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+385 (secure 0 0 0) 0x7f9f1404bf40 con 0x7f9efc06ed20 2024-04-27T04:15:10.537 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-04-27T04:15:10.537 INFO:teuthology.orchestra.run.smithi162.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:90a3a9fa9ceb8977f6dad9ccd36fe46018d07e2118199820523a6b35301d9c16", 2024-04-27T04:15:10.537 INFO:teuthology.orchestra.run.smithi162.stdout: "in_progress": true, 2024-04-27T04:15:10.538 INFO:teuthology.orchestra.run.smithi162.stdout: "which": "Upgrading all daemon types on all hosts", 2024-04-27T04:15:10.538 INFO:teuthology.orchestra.run.smithi162.stdout: "services_complete": [ 2024-04-27T04:15:10.538 INFO:teuthology.orchestra.run.smithi162.stdout: "mon", 2024-04-27T04:15:10.538 INFO:teuthology.orchestra.run.smithi162.stdout: "mgr" 2024-04-27T04:15:10.538 INFO:teuthology.orchestra.run.smithi162.stdout: ], 2024-04-27T04:15:10.538 INFO:teuthology.orchestra.run.smithi162.stdout: "progress": "4/21 daemons upgraded", 2024-04-27T04:15:10.538 INFO:teuthology.orchestra.run.smithi162.stdout: "message": "Currently upgrading crash daemons", 2024-04-27T04:15:10.538 INFO:teuthology.orchestra.run.smithi162.stdout: "is_paused": false 2024-04-27T04:15:10.538 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-04-27T04:15:10.540 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:15:10.541+0000 7f9f1a291700 1 -- 172.21.15.162:0/2189229050 >> [v2:172.21.15.162:6800/1396866108,v1:172.21.15.162:6801/1396866108] conn(0x7f9efc06ed20 msgr2=0x7f9efc0711d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:15:10.540 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:15:10.541+0000 7f9f1a291700 1 --2- 172.21.15.162:0/2189229050 >> [v2:172.21.15.162:6800/1396866108,v1:172.21.15.162:6801/1396866108] conn(0x7f9efc06ed20 0x7f9efc0711d0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f9f1410f440 tx=0x7f9f0400b040 comp rx=0 tx=0).stop 2024-04-27T04:15:10.540 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:15:10.541+0000 7f9f1a291700 1 -- 172.21.15.162:0/2189229050 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9f14101d00 msgr2=0x7f9f14113ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:15:10.540 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:15:10.541+0000 7f9f1a291700 1 --2- 172.21.15.162:0/2189229050 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9f14101d00 0x7f9f14113ba0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f9f08013000 tx=0x7f9f08009e30 comp rx=0 tx=0).stop 2024-04-27T04:15:10.540 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:15:10.541+0000 7f9f1a291700 1 -- 172.21.15.162:0/2189229050 shutdown_connections 2024-04-27T04:15:10.540 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:15:10.541+0000 7f9f1a291700 1 --2- 172.21.15.162:0/2189229050 >> [v2:172.21.15.162:6800/1396866108,v1:172.21.15.162:6801/1396866108] conn(0x7f9efc06ed20 0x7f9efc0711d0 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:15:10.540 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:15:10.541+0000 7f9f1a291700 1 --2- 172.21.15.162:0/2189229050 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9f14101d00 0x7f9f14113ba0 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:15:10.540 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:15:10.541+0000 7f9f1a291700 1 --2- 172.21.15.162:0/2189229050 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9f141026a0 0x7f9f141140e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:15:10.540 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:15:10.541+0000 7f9f1a291700 1 -- 172.21.15.162:0/2189229050 >> 172.21.15.162:0/2189229050 conn(0x7f9f140f78d0 msgr2=0x7f9f140f8f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:15:10.540 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:15:10.541+0000 7f9f1a291700 1 -- 172.21.15.162:0/2189229050 shutdown_connections 2024-04-27T04:15:10.540 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:15:10.541+0000 7f9f1a291700 1 -- 172.21.15.162:0/2189229050 wait complete. 2024-04-27T04:15:10.641 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:15:10.643+0000 7fc63705b700 1 -- 172.21.15.162:0/3158626292 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc630102de0 msgr2=0x7fc6301031b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:15:10.642 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:15:10.643+0000 7fc63705b700 1 --2- 172.21.15.162:0/3158626292 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc630102de0 0x7fc6301031b0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fc624004940 tx=0x7fc624009e30 comp rx=0 tx=0).stop 2024-04-27T04:15:10.642 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:15:10.643+0000 7fc63705b700 1 -- 172.21.15.162:0/3158626292 shutdown_connections 2024-04-27T04:15:10.642 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:15:10.643+0000 7fc63705b700 1 --2- 172.21.15.162:0/3158626292 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc630103780 0x7fc6300fcdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:15:10.642 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:15:10.643+0000 7fc63705b700 1 --2- 172.21.15.162:0/3158626292 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc630102de0 0x7fc6301031b0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:15:10.642 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:15:10.643+0000 7fc63705b700 1 -- 172.21.15.162:0/3158626292 >> 172.21.15.162:0/3158626292 conn(0x7fc6300f89b0 msgr2=0x7fc6300fadc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:15:10.642 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:15:10.643+0000 7fc63705b700 1 -- 172.21.15.162:0/3158626292 shutdown_connections 2024-04-27T04:15:10.642 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:15:10.643+0000 7fc63705b700 1 -- 172.21.15.162:0/3158626292 wait complete. 2024-04-27T04:15:10.644 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:15:10.644+0000 7fc63705b700 1 Processor -- start 2024-04-27T04:15:10.644 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:15:10.644+0000 7fc63705b700 1 -- start start 2024-04-27T04:15:10.644 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:15:10.644+0000 7fc63705b700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc630102de0 0x7fc630113b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:15:10.645 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:15:10.644+0000 7fc63705b700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc630103780 0x7fc630114090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:15:10.645 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:15:10.644+0000 7fc63705b700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6301146e0 con 0x7fc630102de0 2024-04-27T04:15:10.645 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:15:10.644+0000 7fc63705b700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc63010dad0 con 0x7fc630103780 2024-04-27T04:15:10.645 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:15:10.644+0000 7fc634df7700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc630102de0 0x7fc630113b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:15:10.645 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:15:10.644+0000 7fc634df7700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc630102de0 0x7fc630113b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:33768/0 (socket says 172.21.15.162:33768) 2024-04-27T04:15:10.645 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:15:10.644+0000 7fc634df7700 1 -- 172.21.15.162:0/3749171725 learned_addr learned my addr 172.21.15.162:0/3749171725 (peer_addr_for_me v2:172.21.15.162:0/0) 2024-04-27T04:15:10.645 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:15:10.645+0000 7fc62ffff700 1 --2- 172.21.15.162:0/3749171725 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc630103780 0x7fc630114090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:15:10.645 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:15:10.645+0000 7fc634df7700 1 -- 172.21.15.162:0/3749171725 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc630103780 msgr2=0x7fc630114090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:15:10.645 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:15:10.645+0000 7fc634df7700 1 --2- 172.21.15.162:0/3749171725 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc630103780 0x7fc630114090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:15:10.645 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:15:10.645+0000 7fc634df7700 1 -- 172.21.15.162:0/3749171725 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc624005040 con 0x7fc630102de0 2024-04-27T04:15:10.645 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:15:10.645+0000 7fc634df7700 1 --2- 172.21.15.162:0/3749171725 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc630102de0 0x7fc630113b50 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fc62400b3b0 tx=0x7fc624009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:15:10.645 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:15:10.645+0000 7fc62d7fa700 1 -- 172.21.15.162:0/3749171725 <== mon.0 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc62400e040 con 0x7fc630102de0 2024-04-27T04:15:10.645 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:15:10.645+0000 7fc62d7fa700 1 -- 172.21.15.162:0/3749171725 <== mon.0 v2:172.21.15.162:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fc62400bbc0 con 0x7fc630102de0 2024-04-27T04:15:10.646 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:15:10.645+0000 7fc63705b700 1 -- 172.21.15.162:0/3749171725 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc63010dd50 con 0x7fc630102de0 2024-04-27T04:15:10.646 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:15:10.645+0000 7fc62d7fa700 1 -- 172.21.15.162:0/3749171725 <== mon.0 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc62401c3f0 con 0x7fc630102de0 2024-04-27T04:15:10.646 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:15:10.645+0000 7fc63705b700 1 -- 172.21.15.162:0/3749171725 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc63010e310 con 0x7fc630102de0 2024-04-27T04:15:10.646 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:15:10.646+0000 7fc63705b700 1 -- 172.21.15.162:0/3749171725 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc6301000f0 con 0x7fc630102de0 2024-04-27T04:15:10.647 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:15:10.647+0000 7fc62d7fa700 1 -- 172.21.15.162:0/3749171725 <== mon.0 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 38) v1 ==== 92849+0+0 (secure 0 0 0) 0x7fc62400bd30 con 0x7fc630102de0 2024-04-27T04:15:10.647 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:15:10.647+0000 7fc62d7fa700 1 --2- 172.21.15.162:0/3749171725 >> [v2:172.21.15.162:6800/1396866108,v1:172.21.15.162:6801/1396866108] conn(0x7fc62006ed20 0x7fc6200711d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:15:10.647 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:15:10.647+0000 7fc62ffff700 1 --2- 172.21.15.162:0/3749171725 >> [v2:172.21.15.162:6800/1396866108,v1:172.21.15.162:6801/1396866108] conn(0x7fc62006ed20 0x7fc6200711d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:15:10.647 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:15:10.647+0000 7fc62d7fa700 1 -- 172.21.15.162:0/3749171725 <== mon.0 v2:172.21.15.162:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6410+0+0 (secure 0 0 0) 0x7fc62408fd90 con 0x7fc630102de0 2024-04-27T04:15:10.647 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:15:10.648+0000 7fc62ffff700 1 --2- 172.21.15.162:0/3749171725 >> [v2:172.21.15.162:6800/1396866108,v1:172.21.15.162:6801/1396866108] conn(0x7fc62006ed20 0x7fc6200711d0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7fc63010ec90 tx=0x7fc61c009ee0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:15:10.650 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:15:10.652+0000 7fc62d7fa700 1 -- 172.21.15.162:0/3749171725 <== mon.0 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7fc62405c4a0 con 0x7fc630102de0 2024-04-27T04:15:10.929 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:15:10 smithi162 ceph-mon[110641]: from='client.34164 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-27T04:15:10.929 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:15:10 smithi162 ceph-mon[110641]: Upgrade: Setting container_image for all mon 2024-04-27T04:15:10.929 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:15:10 smithi162 ceph-mon[110641]: from='client.34168 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-27T04:15:10.929 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:15:10 smithi162 ceph-mon[110641]: pgmap v29: 65 pgs: 65 active+clean; 208 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:15:10.929 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:15:10 smithi162 ceph-mon[110641]: from='client.? 172.21.15.162:0/2815764875' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-27T04:15:10.930 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:15:10 smithi162 ceph-mon[110641]: from='client.? 172.21.15.162:0/1115238275' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-27T04:15:10.942 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:15:10.944+0000 7fc63705b700 1 -- 172.21.15.162:0/3749171725 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7fc63010f1a0 con 0x7fc630102de0 2024-04-27T04:15:10.942 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:15:10.944+0000 7fc62d7fa700 1 -- 172.21.15.162:0/3749171725 <== mon.0 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+10 (secure 0 0 0) 0x7fc62405c030 con 0x7fc630102de0 2024-04-27T04:15:10.943 INFO:teuthology.orchestra.run.smithi162.stdout:HEALTH_OK 2024-04-27T04:15:10.945 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:15:10.946+0000 7fc63705b700 1 -- 172.21.15.162:0/3749171725 >> [v2:172.21.15.162:6800/1396866108,v1:172.21.15.162:6801/1396866108] conn(0x7fc62006ed20 msgr2=0x7fc6200711d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:15:10.945 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:15:10.947+0000 7fc63705b700 1 --2- 172.21.15.162:0/3749171725 >> [v2:172.21.15.162:6800/1396866108,v1:172.21.15.162:6801/1396866108] conn(0x7fc62006ed20 0x7fc6200711d0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7fc63010ec90 tx=0x7fc61c009ee0 comp rx=0 tx=0).stop 2024-04-27T04:15:10.945 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:15:10.947+0000 7fc63705b700 1 -- 172.21.15.162:0/3749171725 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc630102de0 msgr2=0x7fc630113b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:15:10.945 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:15:10.947+0000 7fc63705b700 1 --2- 172.21.15.162:0/3749171725 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc630102de0 0x7fc630113b50 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fc62400b3b0 tx=0x7fc624009e30 comp rx=0 tx=0).stop 2024-04-27T04:15:10.945 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:15:10.947+0000 7fc63705b700 1 -- 172.21.15.162:0/3749171725 shutdown_connections 2024-04-27T04:15:10.945 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:15:10.947+0000 7fc63705b700 1 --2- 172.21.15.162:0/3749171725 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc630102de0 0x7fc630113b50 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:15:10.945 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:15:10.947+0000 7fc63705b700 1 --2- 172.21.15.162:0/3749171725 >> [v2:172.21.15.162:6800/1396866108,v1:172.21.15.162:6801/1396866108] conn(0x7fc62006ed20 0x7fc6200711d0 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:15:10.945 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:15:10.947+0000 7fc63705b700 1 --2- 172.21.15.162:0/3749171725 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc630103780 0x7fc630114090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:15:10.945 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:15:10.947+0000 7fc63705b700 1 -- 172.21.15.162:0/3749171725 >> 172.21.15.162:0/3749171725 conn(0x7fc6300f89b0 msgr2=0x7fc6300fa730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:15:10.945 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:15:10.947+0000 7fc63705b700 1 -- 172.21.15.162:0/3749171725 shutdown_connections 2024-04-27T04:15:10.945 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:15:10.947+0000 7fc63705b700 1 -- 172.21.15.162:0/3749171725 wait complete. 2024-04-27T04:15:11.172 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:15:10 smithi181 ceph-mon[81134]: from='client.34164 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-27T04:15:11.172 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:15:10 smithi181 ceph-mon[81134]: Upgrade: Setting container_image for all mon 2024-04-27T04:15:11.172 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:15:10 smithi181 ceph-mon[81134]: from='client.34168 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-27T04:15:11.172 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:15:10 smithi181 ceph-mon[81134]: pgmap v29: 65 pgs: 65 active+clean; 208 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:15:11.172 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:15:10 smithi181 ceph-mon[81134]: from='client.? 172.21.15.162:0/2815764875' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-27T04:15:11.172 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:15:10 smithi181 ceph-mon[81134]: from='client.? 172.21.15.162:0/1115238275' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-27T04:15:11.886 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:15:11 smithi162 ceph-mon[110641]: from='client.44117 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-27T04:15:11.887 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:15:11 smithi162 ceph-mon[110641]: from='client.? 172.21.15.162:0/3749171725' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-27T04:15:11.887 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:15:11 smithi162 ceph-mon[110641]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' 2024-04-27T04:15:11.887 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:15:11 smithi162 ceph-mon[110641]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi162", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-04-27T04:15:11.887 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:15:11 smithi162 ceph-mon[110641]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T04:15:12.172 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:15:11 smithi181 ceph-mon[81134]: from='client.44117 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-27T04:15:12.172 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:15:11 smithi181 ceph-mon[81134]: from='client.? 172.21.15.162:0/3749171725' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-27T04:15:12.172 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:15:11 smithi181 ceph-mon[81134]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' 2024-04-27T04:15:12.172 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:15:11 smithi181 ceph-mon[81134]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi162", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-04-27T04:15:12.172 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:15:11 smithi181 ceph-mon[81134]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T04:15:12.887 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:15:12 smithi162 ceph-mon[110641]: Upgrade: Updating crash.smithi162 (1/2) 2024-04-27T04:15:12.887 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:15:12 smithi162 ceph-mon[110641]: Deploying daemon crash.smithi162 on smithi162 2024-04-27T04:15:12.887 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:15:12 smithi162 ceph-mon[110641]: pgmap v30: 65 pgs: 65 active+clean; 208 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:15:13.172 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:15:12 smithi181 ceph-mon[81134]: Upgrade: Updating crash.smithi162 (1/2) 2024-04-27T04:15:13.172 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:15:12 smithi181 ceph-mon[81134]: Deploying daemon crash.smithi162 on smithi162 2024-04-27T04:15:13.172 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:15:12 smithi181 ceph-mon[81134]: pgmap v30: 65 pgs: 65 active+clean; 208 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:15:15.137 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:15:14 smithi162 ceph-mon[110641]: pgmap v31: 65 pgs: 65 active+clean; 208 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:15:15.137 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:15:14 smithi162 ceph-mon[110641]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' 2024-04-27T04:15:15.137 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:15:14 smithi162 ceph-mon[110641]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' 2024-04-27T04:15:15.172 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:15:14 smithi181 ceph-mon[81134]: pgmap v31: 65 pgs: 65 active+clean; 208 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:15:15.172 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:15:14 smithi181 ceph-mon[81134]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' 2024-04-27T04:15:15.172 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:15:14 smithi181 ceph-mon[81134]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' 2024-04-27T04:15:16.997 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:15:16 smithi162 ceph-mon[110641]: pgmap v32: 65 pgs: 65 active+clean; 208 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:15:16.997 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:15:16 smithi162 ceph-mon[110641]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' 2024-04-27T04:15:16.997 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:15:16 smithi162 ceph-mon[110641]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi181", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-04-27T04:15:16.997 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:15:16 smithi162 ceph-mon[110641]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T04:15:17.011 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:15:16 smithi181 ceph-mon[81134]: pgmap v32: 65 pgs: 65 active+clean; 208 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:15:17.011 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:15:16 smithi181 ceph-mon[81134]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' 2024-04-27T04:15:17.011 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:15:16 smithi181 ceph-mon[81134]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi181", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-04-27T04:15:17.011 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:15:16 smithi181 ceph-mon[81134]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T04:15:17.799 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:15:17 smithi181 ceph-mon[81134]: Upgrade: Updating crash.smithi181 (2/2) 2024-04-27T04:15:17.799 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:15:17 smithi181 ceph-mon[81134]: Deploying daemon crash.smithi181 on smithi181 2024-04-27T04:15:17.799 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:15:17 smithi181 ceph-mon[81134]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-27T04:15:18.137 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:15:17 smithi162 ceph-mon[110641]: Upgrade: Updating crash.smithi181 (2/2) 2024-04-27T04:15:18.137 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:15:17 smithi162 ceph-mon[110641]: Deploying daemon crash.smithi181 on smithi181 2024-04-27T04:15:18.137 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:15:17 smithi162 ceph-mon[110641]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-27T04:15:18.922 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:15:18 smithi181 ceph-mon[81134]: pgmap v33: 65 pgs: 65 active+clean; 208 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:15:19.136 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:15:18 smithi162 ceph-mon[110641]: pgmap v33: 65 pgs: 65 active+clean; 208 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:15:19.918 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:15:19 smithi181 ceph-mon[81134]: pgmap v34: 65 pgs: 65 active+clean; 208 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:15:20.136 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:15:19 smithi162 ceph-mon[110641]: pgmap v34: 65 pgs: 65 active+clean; 208 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:15:21.133 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:15:21 smithi162 ceph-mon[110641]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' 2024-04-27T04:15:21.133 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:15:21 smithi162 ceph-mon[110641]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' 2024-04-27T04:15:21.133 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:15:21 smithi162 ceph-mon[110641]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T04:15:21.164 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:15:21 smithi181 ceph-mon[81134]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' 2024-04-27T04:15:21.164 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:15:21 smithi181 ceph-mon[81134]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' 2024-04-27T04:15:21.165 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:15:21 smithi181 ceph-mon[81134]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T04:15:22.386 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:15:22 smithi162 ceph-mon[110641]: pgmap v35: 65 pgs: 65 active+clean; 208 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:15:22.388 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:15:22 smithi181 ceph-mon[81134]: pgmap v35: 65 pgs: 65 active+clean; 208 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:15:24.386 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:15:24 smithi162 ceph-mon[110641]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' 2024-04-27T04:15:24.387 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:15:24 smithi162 ceph-mon[110641]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' 2024-04-27T04:15:24.387 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:15:24 smithi162 ceph-mon[110641]: pgmap v36: 65 pgs: 65 active+clean; 208 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:15:24.422 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:15:24 smithi181 ceph-mon[81134]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' 2024-04-27T04:15:24.422 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:15:24 smithi181 ceph-mon[81134]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' 2024-04-27T04:15:24.422 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:15:24 smithi181 ceph-mon[81134]: pgmap v36: 65 pgs: 65 active+clean; 208 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:15:25.422 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:15:25 smithi181 ceph-mon[81134]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' 2024-04-27T04:15:25.422 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:15:25 smithi181 ceph-mon[81134]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' 2024-04-27T04:15:25.636 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:15:25 smithi162 ceph-mon[110641]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' 2024-04-27T04:15:25.636 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:15:25 smithi162 ceph-mon[110641]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' 2024-04-27T04:15:26.422 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:15:26 smithi181 ceph-mon[81134]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' 2024-04-27T04:15:26.422 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:15:26 smithi181 ceph-mon[81134]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' 2024-04-27T04:15:26.422 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:15:26 smithi181 ceph-mon[81134]: pgmap v37: 65 pgs: 65 active+clean; 208 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:15:26.636 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:15:26 smithi162 ceph-mon[110641]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' 2024-04-27T04:15:26.636 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:15:26 smithi162 ceph-mon[110641]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' 2024-04-27T04:15:26.637 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:15:26 smithi162 ceph-mon[110641]: pgmap v37: 65 pgs: 65 active+clean; 208 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:15:27.422 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:15:27 smithi181 ceph-mon[81134]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' 2024-04-27T04:15:27.422 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:15:27 smithi181 ceph-mon[81134]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' 2024-04-27T04:15:27.565 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:15:27 smithi162 ceph-mon[110641]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' 2024-04-27T04:15:27.565 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:15:27 smithi162 ceph-mon[110641]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' 2024-04-27T04:15:28.886 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:15:28 smithi162 ceph-mon[110641]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' 2024-04-27T04:15:28.886 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:15:28 smithi162 ceph-mon[110641]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' 2024-04-27T04:15:28.887 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:15:28 smithi162 ceph-mon[110641]: pgmap v38: 65 pgs: 65 active+clean; 208 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:15:28.922 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:15:28 smithi181 ceph-mon[81134]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' 2024-04-27T04:15:28.922 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:15:28 smithi181 ceph-mon[81134]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' 2024-04-27T04:15:28.922 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:15:28 smithi181 ceph-mon[81134]: pgmap v38: 65 pgs: 65 active+clean; 208 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:15:31.138 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:15:30 smithi162 ceph-mon[110641]: pgmap v39: 65 pgs: 65 active+clean; 208 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:15:31.138 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:15:30 smithi162 ceph-mon[110641]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' 2024-04-27T04:15:31.138 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:15:30 smithi162 ceph-mon[110641]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' 2024-04-27T04:15:31.138 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:15:30 smithi162 ceph-mon[110641]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T04:15:31.138 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:15:30 smithi162 ceph-mon[110641]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T04:15:31.138 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:15:30 smithi162 ceph-mon[110641]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' 2024-04-27T04:15:31.138 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:15:30 smithi162 ceph-mon[110641]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T04:15:31.139 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:15:30 smithi162 ceph-mon[110641]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "versions"}]: dispatch 2024-04-27T04:15:31.139 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:15:30 smithi162 ceph-mon[110641]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "versions"}]: dispatch 2024-04-27T04:15:31.139 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:15:30 smithi162 ceph-mon[110641]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "versions"}]: dispatch 2024-04-27T04:15:31.139 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:15:30 smithi162 ceph-mon[110641]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' 2024-04-27T04:15:31.139 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:15:30 smithi162 ceph-mon[110641]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.crash.smithi162"}]: dispatch 2024-04-27T04:15:31.139 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:15:30 smithi162 ceph-mon[110641]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.crash.smithi162"}]': finished 2024-04-27T04:15:31.139 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:15:30 smithi162 ceph-mon[110641]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.crash.smithi181"}]: dispatch 2024-04-27T04:15:31.139 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:15:30 smithi162 ceph-mon[110641]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.crash.smithi181"}]': finished 2024-04-27T04:15:31.139 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:15:30 smithi162 ceph-mon[110641]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "osd ok-to-stop", "ids": ["0"], "max": 16}]: dispatch 2024-04-27T04:15:31.172 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:15:30 smithi181 ceph-mon[81134]: pgmap v39: 65 pgs: 65 active+clean; 208 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:15:31.172 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:15:30 smithi181 ceph-mon[81134]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' 2024-04-27T04:15:31.172 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:15:30 smithi181 ceph-mon[81134]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' 2024-04-27T04:15:31.173 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:15:30 smithi181 ceph-mon[81134]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T04:15:31.173 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:15:30 smithi181 ceph-mon[81134]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T04:15:31.173 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:15:30 smithi181 ceph-mon[81134]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' 2024-04-27T04:15:31.173 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:15:30 smithi181 ceph-mon[81134]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T04:15:31.173 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:15:30 smithi181 ceph-mon[81134]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "versions"}]: dispatch 2024-04-27T04:15:31.173 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:15:30 smithi181 ceph-mon[81134]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "versions"}]: dispatch 2024-04-27T04:15:31.173 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:15:30 smithi181 ceph-mon[81134]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "versions"}]: dispatch 2024-04-27T04:15:31.173 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:15:30 smithi181 ceph-mon[81134]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' 2024-04-27T04:15:31.173 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:15:30 smithi181 ceph-mon[81134]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.crash.smithi162"}]: dispatch 2024-04-27T04:15:31.173 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:15:30 smithi181 ceph-mon[81134]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.crash.smithi162"}]': finished 2024-04-27T04:15:31.173 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:15:30 smithi181 ceph-mon[81134]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.crash.smithi181"}]: dispatch 2024-04-27T04:15:31.173 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:15:30 smithi181 ceph-mon[81134]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.crash.smithi181"}]': finished 2024-04-27T04:15:31.173 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:15:30 smithi181 ceph-mon[81134]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "osd ok-to-stop", "ids": ["0"], "max": 16}]: dispatch 2024-04-27T04:15:31.887 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:15:31 smithi162 ceph-mon[110641]: Upgrade: Setting container_image for all crash 2024-04-27T04:15:31.887 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:15:31 smithi162 ceph-mon[110641]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["0"], "max": 16}]: dispatch 2024-04-27T04:15:31.887 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:15:31 smithi162 ceph-mon[110641]: Upgrade: osd.0 is safe to restart 2024-04-27T04:15:31.887 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:15:31 smithi162 ceph-mon[110641]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' 2024-04-27T04:15:31.888 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:15:31 smithi162 ceph-mon[110641]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-04-27T04:15:31.888 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:15:31 smithi162 ceph-mon[110641]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T04:15:32.075 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:15:31 smithi181 ceph-mon[81134]: Upgrade: Setting container_image for all crash 2024-04-27T04:15:32.075 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:15:31 smithi181 ceph-mon[81134]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["0"], "max": 16}]: dispatch 2024-04-27T04:15:32.075 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:15:31 smithi181 ceph-mon[81134]: Upgrade: osd.0 is safe to restart 2024-04-27T04:15:32.075 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:15:31 smithi181 ceph-mon[81134]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' 2024-04-27T04:15:32.076 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:15:31 smithi181 ceph-mon[81134]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-04-27T04:15:32.076 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:15:31 smithi181 ceph-mon[81134]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T04:15:32.879 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:15:32 smithi162 ceph-mon[110641]: Upgrade: Updating osd.0 2024-04-27T04:15:32.879 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:15:32 smithi162 ceph-mon[110641]: Deploying daemon osd.0 on smithi162 2024-04-27T04:15:32.879 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:15:32 smithi162 ceph-mon[110641]: pgmap v40: 65 pgs: 65 active+clean; 208 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:15:32.879 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:15:32 smithi162 ceph-mon[110641]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-27T04:15:33.172 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:15:32 smithi181 ceph-mon[81134]: Upgrade: Updating osd.0 2024-04-27T04:15:33.172 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:15:32 smithi181 ceph-mon[81134]: Deploying daemon osd.0 on smithi162 2024-04-27T04:15:33.172 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:15:32 smithi181 ceph-mon[81134]: pgmap v40: 65 pgs: 65 active+clean; 208 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:15:33.172 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:15:32 smithi181 ceph-mon[81134]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-27T04:15:34.754 INFO:journalctl@ceph.osd.0.smithi162.stdout:Apr 27 04:15:34 smithi162 systemd[1]: Stopping Ceph osd.0 for 7f9750d6-044a-11ef-bc93-c7b262605968... 2024-04-27T04:15:35.136 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:15:34 smithi162 ceph-mon[110641]: pgmap v41: 65 pgs: 65 active+clean; 208 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:15:35.137 INFO:journalctl@ceph.osd.0.smithi162.stdout:Apr 27 04:15:34 smithi162 ceph-7f9750d6-044a-11ef-bc93-c7b262605968-osd-0[45256]: 2024-04-27T04:15:34.758+0000 7f1a86db4700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.0 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-27T04:15:35.137 INFO:journalctl@ceph.osd.0.smithi162.stdout:Apr 27 04:15:34 smithi162 ceph-7f9750d6-044a-11ef-bc93-c7b262605968-osd-0[45256]: 2024-04-27T04:15:34.758+0000 7f1a86db4700 -1 osd.0 49 *** Got signal Terminated *** 2024-04-27T04:15:35.137 INFO:journalctl@ceph.osd.0.smithi162.stdout:Apr 27 04:15:34 smithi162 ceph-7f9750d6-044a-11ef-bc93-c7b262605968-osd-0[45256]: 2024-04-27T04:15:34.758+0000 7f1a86db4700 -1 osd.0 49 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-27T04:15:35.172 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:15:34 smithi181 ceph-mon[81134]: pgmap v41: 65 pgs: 65 active+clean; 208 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:15:36.137 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:15:35 smithi162 ceph-mon[110641]: osd.0 marked itself down and dead 2024-04-27T04:15:36.172 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:15:35 smithi181 ceph-mon[81134]: osd.0 marked itself down and dead 2024-04-27T04:15:36.458 INFO:journalctl@ceph.osd.0.smithi162.stdout:Apr 27 04:15:36 smithi162 podman[121114]: 2024-04-27 04:15:36.199778194 +0000 UTC m=+1.552838235 container died 4a9df3aa938d16746515b75f48f9f6857e755bad4d330d812368968f73c2cb86 (image=quay.ceph.io/ceph-ci/ceph@sha256:d31f278ece6dbd17d1fa5e598f509017f54b76fafe1c4416391fcc51b9718ee9, name=ceph-7f9750d6-044a-11ef-bc93-c7b262605968-osd-0, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.build-date=20240424, RELEASE=quincy-2be3efa, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.schema-version=1.0) 2024-04-27T04:15:36.756 INFO:journalctl@ceph.osd.0.smithi162.stdout:Apr 27 04:15:36 smithi162 podman[121114]: 2024-04-27 04:15:36.61739006 +0000 UTC m=+1.970450098 container cleanup 4a9df3aa938d16746515b75f48f9f6857e755bad4d330d812368968f73c2cb86 (image=quay.ceph.io/ceph-ci/ceph@sha256:d31f278ece6dbd17d1fa5e598f509017f54b76fafe1c4416391fcc51b9718ee9, name=ceph-7f9750d6-044a-11ef-bc93-c7b262605968-osd-0, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.build-date=20240424, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, RELEASE=quincy-2be3efa, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, ceph=True) 2024-04-27T04:15:36.757 INFO:journalctl@ceph.osd.0.smithi162.stdout:Apr 27 04:15:36 smithi162 bash[121114]: ceph-7f9750d6-044a-11ef-bc93-c7b262605968-osd-0 2024-04-27T04:15:37.136 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:15:36 smithi162 ceph-mon[110641]: pgmap v42: 65 pgs: 65 active+clean; 208 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:15:37.137 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:15:36 smithi162 ceph-mon[110641]: Health check failed: 1 osds down (OSD_DOWN) 2024-04-27T04:15:37.137 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:15:36 smithi162 ceph-mon[110641]: osdmap e50: 6 total, 5 up, 6 in 2024-04-27T04:15:37.137 INFO:journalctl@ceph.osd.0.smithi162.stdout:Apr 27 04:15:36 smithi162 podman[121131]: 2024-04-27 04:15:36.869209819 +0000 UTC m=+0.663914591 container remove 4a9df3aa938d16746515b75f48f9f6857e755bad4d330d812368968f73c2cb86 (image=quay.ceph.io/ceph-ci/ceph@sha256:d31f278ece6dbd17d1fa5e598f509017f54b76fafe1c4416391fcc51b9718ee9, name=ceph-7f9750d6-044a-11ef-bc93-c7b262605968-osd-0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, ceph=True, io.buildah.version=1.33.5, org.label-schema.build-date=20240424, RELEASE=quincy-2be3efa) 2024-04-27T04:15:37.172 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:15:36 smithi181 ceph-mon[81134]: pgmap v42: 65 pgs: 65 active+clean; 208 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:15:37.172 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:15:36 smithi181 ceph-mon[81134]: Health check failed: 1 osds down (OSD_DOWN) 2024-04-27T04:15:37.172 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:15:36 smithi181 ceph-mon[81134]: osdmap e50: 6 total, 5 up, 6 in 2024-04-27T04:15:37.503 INFO:journalctl@ceph.osd.0.smithi162.stdout:Apr 27 04:15:37 smithi162 podman[121221]: 2024-04-27 04:15:37.228295446 +0000 UTC m=+0.111318205 container create 3394aa40dd27585bee8f28f40a1972cf629f3648c50640012930461fd87b094c (image=quay.ceph.io/ceph-ci/ceph@sha256:90a3a9fa9ceb8977f6dad9ccd36fe46018d07e2118199820523a6b35301d9c16, name=ceph-7f9750d6-044a-11ef-bc93-c7b262605968-osd-0-deactivate, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, ceph=True, RELEASE=main-4e213a5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.build-date=20240424, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2) 2024-04-27T04:15:37.503 INFO:journalctl@ceph.osd.0.smithi162.stdout:Apr 27 04:15:37 smithi162 podman[121221]: 2024-04-27 04:15:37.135341252 +0000 UTC m=+0.018364020 image pull 67e414ff4bc41092cb31c84c98a9563a32813d02583028679c46b40362afeb3e quay.ceph.io/ceph-ci/ceph@sha256:90a3a9fa9ceb8977f6dad9ccd36fe46018d07e2118199820523a6b35301d9c16 2024-04-27T04:15:37.503 INFO:journalctl@ceph.osd.0.smithi162.stdout:Apr 27 04:15:37 smithi162 podman[121221]: 2024-04-27 04:15:37.422233435 +0000 UTC m=+0.305256193 container init 3394aa40dd27585bee8f28f40a1972cf629f3648c50640012930461fd87b094c (image=quay.ceph.io/ceph-ci/ceph@sha256:90a3a9fa9ceb8977f6dad9ccd36fe46018d07e2118199820523a6b35301d9c16, name=ceph-7f9750d6-044a-11ef-bc93-c7b262605968-osd-0-deactivate, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.license=GPLv2, RELEASE=main-4e213a5, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240424) 2024-04-27T04:15:37.503 INFO:journalctl@ceph.osd.0.smithi162.stdout:Apr 27 04:15:37 smithi162 podman[121221]: 2024-04-27 04:15:37.425297346 +0000 UTC m=+0.308320115 container start 3394aa40dd27585bee8f28f40a1972cf629f3648c50640012930461fd87b094c (image=quay.ceph.io/ceph-ci/ceph@sha256:90a3a9fa9ceb8977f6dad9ccd36fe46018d07e2118199820523a6b35301d9c16, name=ceph-7f9750d6-044a-11ef-bc93-c7b262605968-osd-0-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, RELEASE=main-4e213a5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240424, CEPH_POINT_RELEASE=, GIT_CLEAN=True) 2024-04-27T04:15:37.503 INFO:journalctl@ceph.osd.0.smithi162.stdout:Apr 27 04:15:37 smithi162 podman[121221]: 2024-04-27 04:15:37.505631649 +0000 UTC m=+0.388654418 container attach 3394aa40dd27585bee8f28f40a1972cf629f3648c50640012930461fd87b094c (image=quay.ceph.io/ceph-ci/ceph@sha256:90a3a9fa9ceb8977f6dad9ccd36fe46018d07e2118199820523a6b35301d9c16, name=ceph-7f9750d6-044a-11ef-bc93-c7b262605968-osd-0-deactivate, org.label-schema.vendor=CentOS, RELEASE=main-4e213a5, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240424) 2024-04-27T04:15:37.769 INFO:journalctl@ceph.osd.0.smithi162.stdout:Apr 27 04:15:37 smithi162 podman[121221]: 2024-04-27 04:15:37.722725527 +0000 UTC m=+0.605748286 container died 3394aa40dd27585bee8f28f40a1972cf629f3648c50640012930461fd87b094c (image=quay.ceph.io/ceph-ci/ceph@sha256:90a3a9fa9ceb8977f6dad9ccd36fe46018d07e2118199820523a6b35301d9c16, name=ceph-7f9750d6-044a-11ef-bc93-c7b262605968-osd-0-deactivate, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.build-date=20240424, ceph=True, CEPH_POINT_RELEASE=, RELEASE=main-4e213a5, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD) 2024-04-27T04:15:38.056 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:15:37 smithi162 ceph-mon[110641]: osdmap e51: 6 total, 5 up, 6 in 2024-04-27T04:15:38.172 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:15:37 smithi181 ceph-mon[81134]: osdmap e51: 6 total, 5 up, 6 in 2024-04-27T04:15:38.637 INFO:journalctl@ceph.osd.0.smithi162.stdout:Apr 27 04:15:38 smithi162 podman[121247]: 2024-04-27 04:15:38.359831343 +0000 UTC m=+0.631597600 container remove 3394aa40dd27585bee8f28f40a1972cf629f3648c50640012930461fd87b094c (image=quay.ceph.io/ceph-ci/ceph@sha256:90a3a9fa9ceb8977f6dad9ccd36fe46018d07e2118199820523a6b35301d9c16, name=ceph-7f9750d6-044a-11ef-bc93-c7b262605968-osd-0-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=main-4e213a5, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240424) 2024-04-27T04:15:38.637 INFO:journalctl@ceph.osd.0.smithi162.stdout:Apr 27 04:15:38 smithi162 systemd[1]: ceph-7f9750d6-044a-11ef-bc93-c7b262605968@osd.0.service: Deactivated successfully. 2024-04-27T04:15:38.637 INFO:journalctl@ceph.osd.0.smithi162.stdout:Apr 27 04:15:38 smithi162 systemd[1]: Stopped Ceph osd.0 for 7f9750d6-044a-11ef-bc93-c7b262605968. 2024-04-27T04:15:38.637 INFO:journalctl@ceph.osd.0.smithi162.stdout:Apr 27 04:15:38 smithi162 systemd[1]: ceph-7f9750d6-044a-11ef-bc93-c7b262605968@osd.0.service: Consumed 1min 1.364s CPU time. 2024-04-27T04:15:38.637 INFO:journalctl@ceph.osd.0.smithi162.stdout:Apr 27 04:15:38 smithi162 systemd[1]: Starting Ceph osd.0 for 7f9750d6-044a-11ef-bc93-c7b262605968... 2024-04-27T04:15:38.967 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:15:38 smithi162 ceph-mon[110641]: pgmap v45: 65 pgs: 9 stale+active+clean, 56 active+clean; 208 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 639 B/s rd, 1 op/s 2024-04-27T04:15:38.967 INFO:journalctl@ceph.osd.0.smithi162.stdout:Apr 27 04:15:38 smithi162 podman[121351]: 2024-04-27 04:15:38.716227542 +0000 UTC m=+0.018362975 image pull 67e414ff4bc41092cb31c84c98a9563a32813d02583028679c46b40362afeb3e quay.ceph.io/ceph-ci/ceph@sha256:90a3a9fa9ceb8977f6dad9ccd36fe46018d07e2118199820523a6b35301d9c16 2024-04-27T04:15:38.967 INFO:journalctl@ceph.osd.0.smithi162.stdout:Apr 27 04:15:38 smithi162 podman[121351]: 2024-04-27 04:15:38.869168395 +0000 UTC m=+0.171303821 container create 491a29987934cfc16e060bf92379e9bdc498c53ad9f26720018c7310d1a37179 (image=quay.ceph.io/ceph-ci/ceph@sha256:90a3a9fa9ceb8977f6dad9ccd36fe46018d07e2118199820523a6b35301d9c16, name=ceph-7f9750d6-044a-11ef-bc93-c7b262605968-osd-0-activate, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, ceph=True, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=main-4e213a5, org.label-schema.build-date=20240424, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=) 2024-04-27T04:15:39.172 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:15:38 smithi181 ceph-mon[81134]: pgmap v45: 65 pgs: 9 stale+active+clean, 56 active+clean; 208 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 639 B/s rd, 1 op/s 2024-04-27T04:15:39.386 INFO:journalctl@ceph.osd.0.smithi162.stdout:Apr 27 04:15:39 smithi162 podman[121351]: 2024-04-27 04:15:39.063134145 +0000 UTC m=+0.365269571 container init 491a29987934cfc16e060bf92379e9bdc498c53ad9f26720018c7310d1a37179 (image=quay.ceph.io/ceph-ci/ceph@sha256:90a3a9fa9ceb8977f6dad9ccd36fe46018d07e2118199820523a6b35301d9c16, name=ceph-7f9750d6-044a-11ef-bc93-c7b262605968-osd-0-activate, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240424, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, RELEASE=main-4e213a5, maintainer=Guillaume Abrioux , ceph=True) 2024-04-27T04:15:39.387 INFO:journalctl@ceph.osd.0.smithi162.stdout:Apr 27 04:15:39 smithi162 podman[121351]: 2024-04-27 04:15:39.066304078 +0000 UTC m=+0.368439513 container start 491a29987934cfc16e060bf92379e9bdc498c53ad9f26720018c7310d1a37179 (image=quay.ceph.io/ceph-ci/ceph@sha256:90a3a9fa9ceb8977f6dad9ccd36fe46018d07e2118199820523a6b35301d9c16, name=ceph-7f9750d6-044a-11ef-bc93-c7b262605968-osd-0-activate, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, RELEASE=main-4e213a5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240424, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-27T04:15:39.387 INFO:journalctl@ceph.osd.0.smithi162.stdout:Apr 27 04:15:39 smithi162 podman[121351]: 2024-04-27 04:15:39.146444443 +0000 UTC m=+0.448579887 container attach 491a29987934cfc16e060bf92379e9bdc498c53ad9f26720018c7310d1a37179 (image=quay.ceph.io/ceph-ci/ceph@sha256:90a3a9fa9ceb8977f6dad9ccd36fe46018d07e2118199820523a6b35301d9c16, name=ceph-7f9750d6-044a-11ef-bc93-c7b262605968-osd-0-activate, org.label-schema.build-date=20240424, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_BRANCH=HEAD, RELEASE=main-4e213a5) 2024-04-27T04:15:39.886 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:15:39 smithi162 ceph-mon[110641]: pgmap v46: 65 pgs: 18 active+undersized+degraded, 16 active+undersized, 31 active+clean; 208 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 1.1 KiB/s rd, 1 op/s; 41/231 objects degraded (17.749%) 2024-04-27T04:15:39.887 INFO:journalctl@ceph.osd.0.smithi162.stdout:Apr 27 04:15:39 smithi162 ceph-7f9750d6-044a-11ef-bc93-c7b262605968-osd-0-activate[121366]: Running command: /usr/bin/ceph-authtool --gen-print-key 2024-04-27T04:15:39.887 INFO:journalctl@ceph.osd.0.smithi162.stdout:Apr 27 04:15:39 smithi162 ceph-7f9750d6-044a-11ef-bc93-c7b262605968-osd-0-activate[121366]: Running command: /usr/bin/ceph-authtool --gen-print-key 2024-04-27T04:15:39.887 INFO:journalctl@ceph.osd.0.smithi162.stdout:Apr 27 04:15:39 smithi162 bash[121351]: Running command: /usr/bin/ceph-authtool --gen-print-key 2024-04-27T04:15:39.887 INFO:journalctl@ceph.osd.0.smithi162.stdout:Apr 27 04:15:39 smithi162 bash[121351]: Running command: /usr/bin/ceph-authtool --gen-print-key 2024-04-27T04:15:39.887 INFO:journalctl@ceph.osd.0.smithi162.stdout:Apr 27 04:15:39 smithi162 bash[121351]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2024-04-27T04:15:39.887 INFO:journalctl@ceph.osd.0.smithi162.stdout:Apr 27 04:15:39 smithi162 bash[121351]: Running command: /usr/bin/ceph-bluestore-tool prime-osd-dir --path /var/lib/ceph/osd/ceph-0 --no-mon-config --dev /dev/mapper/vg_nvme-lv_4 2024-04-27T04:15:39.887 INFO:journalctl@ceph.osd.0.smithi162.stdout:Apr 27 04:15:39 smithi162 bash[121351]: Running command: /usr/bin/chown -h ceph:ceph /dev/mapper/vg_nvme-lv_4 2024-04-27T04:15:39.887 INFO:journalctl@ceph.osd.0.smithi162.stdout:Apr 27 04:15:39 smithi162 bash[121351]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-3 2024-04-27T04:15:39.887 INFO:journalctl@ceph.osd.0.smithi162.stdout:Apr 27 04:15:39 smithi162 bash[121351]: Running command: /usr/bin/ln -s /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-0/block 2024-04-27T04:15:39.887 INFO:journalctl@ceph.osd.0.smithi162.stdout:Apr 27 04:15:39 smithi162 bash[121351]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2024-04-27T04:15:39.888 INFO:journalctl@ceph.osd.0.smithi162.stdout:Apr 27 04:15:39 smithi162 bash[121351]: --> ceph-volume raw activate successful for osd ID: 0 2024-04-27T04:15:39.888 INFO:journalctl@ceph.osd.0.smithi162.stdout:Apr 27 04:15:39 smithi162 ceph-7f9750d6-044a-11ef-bc93-c7b262605968-osd-0-activate[121366]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2024-04-27T04:15:39.888 INFO:journalctl@ceph.osd.0.smithi162.stdout:Apr 27 04:15:39 smithi162 ceph-7f9750d6-044a-11ef-bc93-c7b262605968-osd-0-activate[121366]: Running command: /usr/bin/ceph-bluestore-tool prime-osd-dir --path /var/lib/ceph/osd/ceph-0 --no-mon-config --dev /dev/mapper/vg_nvme-lv_4 2024-04-27T04:15:39.888 INFO:journalctl@ceph.osd.0.smithi162.stdout:Apr 27 04:15:39 smithi162 ceph-7f9750d6-044a-11ef-bc93-c7b262605968-osd-0-activate[121366]: Running command: /usr/bin/chown -h ceph:ceph /dev/mapper/vg_nvme-lv_4 2024-04-27T04:15:39.888 INFO:journalctl@ceph.osd.0.smithi162.stdout:Apr 27 04:15:39 smithi162 ceph-7f9750d6-044a-11ef-bc93-c7b262605968-osd-0-activate[121366]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-3 2024-04-27T04:15:39.888 INFO:journalctl@ceph.osd.0.smithi162.stdout:Apr 27 04:15:39 smithi162 ceph-7f9750d6-044a-11ef-bc93-c7b262605968-osd-0-activate[121366]: Running command: /usr/bin/ln -s /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-0/block 2024-04-27T04:15:39.888 INFO:journalctl@ceph.osd.0.smithi162.stdout:Apr 27 04:15:39 smithi162 ceph-7f9750d6-044a-11ef-bc93-c7b262605968-osd-0-activate[121366]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2024-04-27T04:15:39.888 INFO:journalctl@ceph.osd.0.smithi162.stdout:Apr 27 04:15:39 smithi162 ceph-7f9750d6-044a-11ef-bc93-c7b262605968-osd-0-activate[121366]: --> ceph-volume raw activate successful for osd ID: 0 2024-04-27T04:15:39.888 INFO:journalctl@ceph.osd.0.smithi162.stdout:Apr 27 04:15:39 smithi162 podman[121351]: 2024-04-27 04:15:39.573231736 +0000 UTC m=+0.875367164 container died 491a29987934cfc16e060bf92379e9bdc498c53ad9f26720018c7310d1a37179 (image=quay.ceph.io/ceph-ci/ceph@sha256:90a3a9fa9ceb8977f6dad9ccd36fe46018d07e2118199820523a6b35301d9c16, name=ceph-7f9750d6-044a-11ef-bc93-c7b262605968-osd-0-activate, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=main-4e213a5, org.label-schema.build-date=20240424, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-27T04:15:40.172 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:15:39 smithi181 ceph-mon[81134]: pgmap v46: 65 pgs: 18 active+undersized+degraded, 16 active+undersized, 31 active+clean; 208 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 1.1 KiB/s rd, 1 op/s; 41/231 objects degraded (17.749%) 2024-04-27T04:15:40.626 INFO:journalctl@ceph.osd.0.smithi162.stdout:Apr 27 04:15:40 smithi162 podman[121506]: 2024-04-27 04:15:40.284031284 +0000 UTC m=+0.705697676 container remove 491a29987934cfc16e060bf92379e9bdc498c53ad9f26720018c7310d1a37179 (image=quay.ceph.io/ceph-ci/ceph@sha256:90a3a9fa9ceb8977f6dad9ccd36fe46018d07e2118199820523a6b35301d9c16, name=ceph-7f9750d6-044a-11ef-bc93-c7b262605968-osd-0-activate, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240424, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, ceph=True, io.buildah.version=1.33.5, RELEASE=main-4e213a5) 2024-04-27T04:15:40.626 INFO:journalctl@ceph.osd.0.smithi162.stdout:Apr 27 04:15:40 smithi162 podman[121573]: 2024-04-27 04:15:40.401058749 +0000 UTC m=+0.018514177 image pull 67e414ff4bc41092cb31c84c98a9563a32813d02583028679c46b40362afeb3e quay.ceph.io/ceph-ci/ceph@sha256:90a3a9fa9ceb8977f6dad9ccd36fe46018d07e2118199820523a6b35301d9c16 2024-04-27T04:15:40.626 INFO:journalctl@ceph.osd.0.smithi162.stdout:Apr 27 04:15:40 smithi162 podman[121573]: 2024-04-27 04:15:40.526705615 +0000 UTC m=+0.144161046 container create 2744aff3cd885623e08b77f5e5c0477e8489290d8922e4ccccd400f92ba53294 (image=quay.ceph.io/ceph-ci/ceph@sha256:90a3a9fa9ceb8977f6dad9ccd36fe46018d07e2118199820523a6b35301d9c16, name=ceph-7f9750d6-044a-11ef-bc93-c7b262605968-osd-0, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240424, ceph=True, GIT_CLEAN=True, RELEASE=main-4e213a5, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-27T04:15:40.886 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:15:40 smithi162 ceph-mon[110641]: Health check failed: Degraded data redundancy: 41/231 objects degraded (17.749%), 18 pgs degraded (PG_DEGRADED) 2024-04-27T04:15:40.887 INFO:journalctl@ceph.osd.0.smithi162.stdout:Apr 27 04:15:40 smithi162 podman[121573]: 2024-04-27 04:15:40.737189641 +0000 UTC m=+0.354645029 container init 2744aff3cd885623e08b77f5e5c0477e8489290d8922e4ccccd400f92ba53294 (image=quay.ceph.io/ceph-ci/ceph@sha256:90a3a9fa9ceb8977f6dad9ccd36fe46018d07e2118199820523a6b35301d9c16, name=ceph-7f9750d6-044a-11ef-bc93-c7b262605968-osd-0, GIT_CLEAN=True, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240424, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.vendor=CentOS, RELEASE=main-4e213a5, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-27T04:15:40.887 INFO:journalctl@ceph.osd.0.smithi162.stdout:Apr 27 04:15:40 smithi162 podman[121573]: 2024-04-27 04:15:40.740257764 +0000 UTC m=+0.357713163 container start 2744aff3cd885623e08b77f5e5c0477e8489290d8922e4ccccd400f92ba53294 (image=quay.ceph.io/ceph-ci/ceph@sha256:90a3a9fa9ceb8977f6dad9ccd36fe46018d07e2118199820523a6b35301d9c16, name=ceph-7f9750d6-044a-11ef-bc93-c7b262605968-osd-0, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.build-date=20240424, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, RELEASE=main-4e213a5, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.schema-version=1.0) 2024-04-27T04:15:40.887 INFO:journalctl@ceph.osd.0.smithi162.stdout:Apr 27 04:15:40 smithi162 ceph-osd[121592]: -- 172.21.15.162:0/3534724224 <== mon.0 v2:172.21.15.162:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55daa1179680 con 0x55daa1142c00 2024-04-27T04:15:40.887 INFO:journalctl@ceph.osd.0.smithi162.stdout:Apr 27 04:15:40 smithi162 bash[121573]: 2744aff3cd885623e08b77f5e5c0477e8489290d8922e4ccccd400f92ba53294 2024-04-27T04:15:40.887 INFO:journalctl@ceph.osd.0.smithi162.stdout:Apr 27 04:15:40 smithi162 systemd[1]: Started Ceph osd.0 for 7f9750d6-044a-11ef-bc93-c7b262605968. 2024-04-27T04:15:41.057 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:15:41.055+0000 7fd105a8d700 1 -- 172.21.15.162:0/3902382596 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd1001014d0 msgr2=0x7fd1001018a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:15:41.057 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:15:41.055+0000 7fd105a8d700 1 --2- 172.21.15.162:0/3902382596 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd1001014d0 0x7fd1001018a0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fd0f4004940 tx=0x7fd0f4009e30 comp rx=0 tx=0).stop 2024-04-27T04:15:41.057 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:15:41.055+0000 7fd105a8d700 1 -- 172.21.15.162:0/3902382596 shutdown_connections 2024-04-27T04:15:41.057 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:15:41.055+0000 7fd105a8d700 1 --2- 172.21.15.162:0/3902382596 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd1000fb600 0x7fd1000fba70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:15:41.057 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:15:41.055+0000 7fd105a8d700 1 --2- 172.21.15.162:0/3902382596 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd1001014d0 0x7fd1001018a0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:15:41.057 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:15:41.055+0000 7fd105a8d700 1 -- 172.21.15.162:0/3902382596 >> 172.21.15.162:0/3902382596 conn(0x7fd100071460 msgr2=0x7fd100071860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:15:41.057 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:15:41.055+0000 7fd105a8d700 1 -- 172.21.15.162:0/3902382596 shutdown_connections 2024-04-27T04:15:41.057 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:15:41.055+0000 7fd105a8d700 1 -- 172.21.15.162:0/3902382596 wait complete. 2024-04-27T04:15:41.057 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:15:41.056+0000 7fd105a8d700 1 Processor -- start 2024-04-27T04:15:41.058 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:15:41.056+0000 7fd105a8d700 1 -- start start 2024-04-27T04:15:41.058 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:15:41.056+0000 7fd105a8d700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd1000fb600 0x7fd100113b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:15:41.058 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:15:41.056+0000 7fd105a8d700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd1001014d0 0x7fd1001140a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:15:41.058 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:15:41.056+0000 7fd105a8d700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1001146f0 con 0x7fd1000fb600 2024-04-27T04:15:41.058 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:15:41.056+0000 7fd105a8d700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd10010dae0 con 0x7fd1001014d0 2024-04-27T04:15:41.058 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:15:41.056+0000 7fd0ff7fe700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd1000fb600 0x7fd100113b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:15:41.058 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:15:41.056+0000 7fd0ff7fe700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd1000fb600 0x7fd100113b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:49968/0 (socket says 172.21.15.162:49968) 2024-04-27T04:15:41.058 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:15:41.056+0000 7fd0ff7fe700 1 -- 172.21.15.162:0/2577830087 learned_addr learned my addr 172.21.15.162:0/2577830087 (peer_addr_for_me v2:172.21.15.162:0/0) 2024-04-27T04:15:41.058 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:15:41.057+0000 7fd0feffd700 1 --2- 172.21.15.162:0/2577830087 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd1001014d0 0x7fd1001140a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:15:41.058 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:15:41.057+0000 7fd0ff7fe700 1 -- 172.21.15.162:0/2577830087 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd1001014d0 msgr2=0x7fd1001140a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:15:41.058 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:15:41.057+0000 7fd0ff7fe700 1 --2- 172.21.15.162:0/2577830087 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd1001014d0 0x7fd1001140a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:15:41.058 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:15:41.057+0000 7fd0ff7fe700 1 -- 172.21.15.162:0/2577830087 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd0f4005040 con 0x7fd1000fb600 2024-04-27T04:15:41.058 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:15:41.057+0000 7fd0feffd700 1 --2- 172.21.15.162:0/2577830087 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd1001014d0 0x7fd1001140a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-27T04:15:41.059 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:15:41.057+0000 7fd0ff7fe700 1 --2- 172.21.15.162:0/2577830087 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd1000fb600 0x7fd100113b60 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fd0f400b3b0 tx=0x7fd0f4009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:15:41.059 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:15:41.057+0000 7fd0e7fff700 1 -- 172.21.15.162:0/2577830087 <== mon.0 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd0f400e040 con 0x7fd1000fb600 2024-04-27T04:15:41.059 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:15:41.057+0000 7fd0e7fff700 1 -- 172.21.15.162:0/2577830087 <== mon.0 v2:172.21.15.162:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fd0f400bbc0 con 0x7fd1000fb600 2024-04-27T04:15:41.059 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:15:41.057+0000 7fd105a8d700 1 -- 172.21.15.162:0/2577830087 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd10010dd60 con 0x7fd1000fb600 2024-04-27T04:15:41.059 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:15:41.057+0000 7fd0e7fff700 1 -- 172.21.15.162:0/2577830087 <== mon.0 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd0f401c3f0 con 0x7fd1000fb600 2024-04-27T04:15:41.059 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:15:41.057+0000 7fd105a8d700 1 -- 172.21.15.162:0/2577830087 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd10010e320 con 0x7fd1000fb600 2024-04-27T04:15:41.060 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:15:41.059+0000 7fd105a8d700 1 -- 172.21.15.162:0/2577830087 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd100104fd0 con 0x7fd1000fb600 2024-04-27T04:15:41.060 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:15:41.059+0000 7fd0e7fff700 1 -- 172.21.15.162:0/2577830087 <== mon.0 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 38) v1 ==== 92849+0+0 (secure 0 0 0) 0x7fd0f400bd30 con 0x7fd1000fb600 2024-04-27T04:15:41.060 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:15:41.059+0000 7fd0e7fff700 1 --2- 172.21.15.162:0/2577830087 >> [v2:172.21.15.162:6800/1396866108,v1:172.21.15.162:6801/1396866108] conn(0x7fd0e806ed20 0x7fd0e80711d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:15:41.060 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:15:41.060+0000 7fd0e7fff700 1 -- 172.21.15.162:0/2577830087 <== mon.0 v2:172.21.15.162:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 6410+0+0 (secure 0 0 0) 0x7fd0f408fd90 con 0x7fd1000fb600 2024-04-27T04:15:41.060 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:15:41.060+0000 7fd0feffd700 1 --2- 172.21.15.162:0/2577830087 >> [v2:172.21.15.162:6800/1396866108,v1:172.21.15.162:6801/1396866108] conn(0x7fd0e806ed20 0x7fd0e80711d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:15:41.060 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:15:41.060+0000 7fd0feffd700 1 --2- 172.21.15.162:0/2577830087 >> [v2:172.21.15.162:6800/1396866108,v1:172.21.15.162:6801/1396866108] conn(0x7fd0e806ed20 0x7fd0e80711d0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fd0f0009fd0 tx=0x7fd0f000b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:15:41.063 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:15:41.064+0000 7fd0e7fff700 1 -- 172.21.15.162:0/2577830087 <== mon.0 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7fd0f405c4a0 con 0x7fd1000fb600 2024-04-27T04:15:41.172 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:15:40 smithi181 ceph-mon[81134]: Health check failed: Degraded data redundancy: 41/231 objects degraded (17.749%), 18 pgs degraded (PG_DEGRADED) 2024-04-27T04:15:41.291 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:15:41.292+0000 7fd105a8d700 1 -- 172.21.15.162:0/2577830087 --> [v2:172.21.15.162:6800/1396866108,v1:172.21.15.162:6801/1396866108] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fd10010f1b0 con 0x7fd0e806ed20 2024-04-27T04:15:41.293 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:15:41.294+0000 7fd0e7fff700 1 -- 172.21.15.162:0/2577830087 <== mgr.34104 v2:172.21.15.162:6800/1396866108 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+400 (secure 0 0 0) 0x7fd10010f1b0 con 0x7fd0e806ed20 2024-04-27T04:15:41.295 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:15:41.296+0000 7fd105a8d700 1 -- 172.21.15.162:0/2577830087 >> [v2:172.21.15.162:6800/1396866108,v1:172.21.15.162:6801/1396866108] conn(0x7fd0e806ed20 msgr2=0x7fd0e80711d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:15:41.296 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:15:41.296+0000 7fd105a8d700 1 --2- 172.21.15.162:0/2577830087 >> [v2:172.21.15.162:6800/1396866108,v1:172.21.15.162:6801/1396866108] conn(0x7fd0e806ed20 0x7fd0e80711d0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fd0f0009fd0 tx=0x7fd0f000b040 comp rx=0 tx=0).stop 2024-04-27T04:15:41.296 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:15:41.296+0000 7fd105a8d700 1 -- 172.21.15.162:0/2577830087 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd1000fb600 msgr2=0x7fd100113b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:15:41.296 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:15:41.296+0000 7fd105a8d700 1 --2- 172.21.15.162:0/2577830087 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd1000fb600 0x7fd100113b60 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fd0f400b3b0 tx=0x7fd0f4009e30 comp rx=0 tx=0).stop 2024-04-27T04:15:41.296 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:15:41.296+0000 7fd105a8d700 1 -- 172.21.15.162:0/2577830087 shutdown_connections 2024-04-27T04:15:41.296 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:15:41.296+0000 7fd105a8d700 1 --2- 172.21.15.162:0/2577830087 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd1000fb600 0x7fd100113b60 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:15:41.296 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:15:41.296+0000 7fd105a8d700 1 --2- 172.21.15.162:0/2577830087 >> [v2:172.21.15.162:6800/1396866108,v1:172.21.15.162:6801/1396866108] conn(0x7fd0e806ed20 0x7fd0e80711d0 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:15:41.296 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:15:41.296+0000 7fd105a8d700 1 --2- 172.21.15.162:0/2577830087 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd1001014d0 0x7fd1001140a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:15:41.296 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:15:41.296+0000 7fd105a8d700 1 -- 172.21.15.162:0/2577830087 >> 172.21.15.162:0/2577830087 conn(0x7fd100071460 msgr2=0x7fd100102150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:15:41.296 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:15:41.297+0000 7fd105a8d700 1 -- 172.21.15.162:0/2577830087 shutdown_connections 2024-04-27T04:15:41.296 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:15:41.297+0000 7fd105a8d700 1 -- 172.21.15.162:0/2577830087 wait complete. 2024-04-27T04:15:41.307 INFO:teuthology.orchestra.run.smithi162.stdout:true 2024-04-27T04:15:41.307 INFO:journalctl@ceph.osd.0.smithi162.stdout:Apr 27 04:15:41 smithi162 ceph-7f9750d6-044a-11ef-bc93-c7b262605968-osd-0[121588]: 2024-04-27T04:15:41.277+0000 7f199620f7c0 -1 Falling back to public interface 2024-04-27T04:15:41.406 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:15:41.406+0000 7f552a85b700 1 -- 172.21.15.162:0/1498796781 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5524103780 msgr2=0x7f55240fcdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:15:41.406 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:15:41.406+0000 7f552a85b700 1 --2- 172.21.15.162:0/1498796781 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5524103780 0x7f55240fcdf0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f5518004940 tx=0x7f5518009e30 comp rx=0 tx=0).stop 2024-04-27T04:15:41.406 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:15:41.406+0000 7f552a85b700 1 -- 172.21.15.162:0/1498796781 shutdown_connections 2024-04-27T04:15:41.406 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:15:41.406+0000 7f552a85b700 1 --2- 172.21.15.162:0/1498796781 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5524103780 0x7f55240fcdf0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:15:41.406 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:15:41.406+0000 7f552a85b700 1 --2- 172.21.15.162:0/1498796781 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5524102de0 0x7f55241031b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:15:41.406 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:15:41.406+0000 7f552a85b700 1 -- 172.21.15.162:0/1498796781 >> 172.21.15.162:0/1498796781 conn(0x7f55240f89b0 msgr2=0x7f55240fadc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:15:41.407 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:15:41.406+0000 7f552a85b700 1 -- 172.21.15.162:0/1498796781 shutdown_connections 2024-04-27T04:15:41.407 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:15:41.406+0000 7f552a85b700 1 -- 172.21.15.162:0/1498796781 wait complete. 2024-04-27T04:15:41.407 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:15:41.407+0000 7f552a85b700 1 Processor -- start 2024-04-27T04:15:41.408 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:15:41.407+0000 7f552a85b700 1 -- start start 2024-04-27T04:15:41.408 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:15:41.408+0000 7f552a85b700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5524102de0 0x7f5524113b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:15:41.408 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:15:41.408+0000 7f552a85b700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5524103780 0x7f55241140c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:15:41.408 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:15:41.408+0000 7f552a85b700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5524114710 con 0x7f5524102de0 2024-04-27T04:15:41.409 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:15:41.408+0000 7f552a85b700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f552410db00 con 0x7f5524103780 2024-04-27T04:15:41.409 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:15:41.408+0000 7f5523fff700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5524102de0 0x7f5524113b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:15:41.409 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:15:41.408+0000 7f5523fff700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5524102de0 0x7f5524113b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:49986/0 (socket says 172.21.15.162:49986) 2024-04-27T04:15:41.409 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:15:41.408+0000 7f5523fff700 1 -- 172.21.15.162:0/2364751048 learned_addr learned my addr 172.21.15.162:0/2364751048 (peer_addr_for_me v2:172.21.15.162:0/0) 2024-04-27T04:15:41.409 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:15:41.408+0000 7f55237fe700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5524103780 0x7f55241140c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:15:41.409 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:15:41.408+0000 7f5523fff700 1 -- 172.21.15.162:0/2364751048 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5524103780 msgr2=0x7f55241140c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:15:41.409 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:15:41.408+0000 7f5523fff700 1 --2- 172.21.15.162:0/2364751048 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5524103780 0x7f55241140c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:15:41.409 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:15:41.408+0000 7f5523fff700 1 -- 172.21.15.162:0/2364751048 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5518005040 con 0x7f5524102de0 2024-04-27T04:15:41.409 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:15:41.408+0000 7f5523fff700 1 --2- 172.21.15.162:0/2364751048 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5524102de0 0x7f5524113b80 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f5514002a50 tx=0x7f55140046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:15:41.409 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:15:41.408+0000 7f5520ff9700 1 -- 172.21.15.162:0/2364751048 <== mon.0 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5514007cc0 con 0x7f5524102de0 2024-04-27T04:15:41.409 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:15:41.408+0000 7f5520ff9700 1 -- 172.21.15.162:0/2364751048 <== mon.0 v2:172.21.15.162:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f551400f040 con 0x7f5524102de0 2024-04-27T04:15:41.409 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:15:41.408+0000 7f5520ff9700 1 -- 172.21.15.162:0/2364751048 <== mon.0 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f551400da30 con 0x7f5524102de0 2024-04-27T04:15:41.409 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:15:41.409+0000 7f552a85b700 1 -- 172.21.15.162:0/2364751048 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f552410dde0 con 0x7f5524102de0 2024-04-27T04:15:41.409 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:15:41.409+0000 7f552a85b700 1 -- 172.21.15.162:0/2364751048 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f552410e370 con 0x7f5524102de0 2024-04-27T04:15:41.410 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:15:41.410+0000 7f552a85b700 1 -- 172.21.15.162:0/2364751048 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f55240fe530 con 0x7f5524102de0 2024-04-27T04:15:41.410 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:15:41.410+0000 7f5520ff9700 1 -- 172.21.15.162:0/2364751048 <== mon.0 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 38) v1 ==== 92849+0+0 (secure 0 0 0) 0x7f551400db90 con 0x7f5524102de0 2024-04-27T04:15:41.410 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:15:41.411+0000 7f5520ff9700 1 --2- 172.21.15.162:0/2364751048 >> [v2:172.21.15.162:6800/1396866108,v1:172.21.15.162:6801/1396866108] conn(0x7f550c06ed20 0x7f550c0711d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:15:41.410 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:15:41.411+0000 7f55237fe700 1 --2- 172.21.15.162:0/2364751048 >> [v2:172.21.15.162:6800/1396866108,v1:172.21.15.162:6801/1396866108] conn(0x7f550c06ed20 0x7f550c0711d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:15:41.411 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:15:41.411+0000 7f5520ff9700 1 -- 172.21.15.162:0/2364751048 <== mon.0 v2:172.21.15.162:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 6410+0+0 (secure 0 0 0) 0x7f551408e900 con 0x7f5524102de0 2024-04-27T04:15:41.411 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:15:41.411+0000 7f55237fe700 1 --2- 172.21.15.162:0/2364751048 >> [v2:172.21.15.162:6800/1396866108,v1:172.21.15.162:6801/1396866108] conn(0x7f550c06ed20 0x7f550c0711d0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f5518004890 tx=0x7f5518009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:15:41.415 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:15:41.416+0000 7f5520ff9700 1 -- 172.21.15.162:0/2364751048 <== mon.0 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7f551405af70 con 0x7f5524102de0 2024-04-27T04:15:41.654 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:15:41.654+0000 7f552a85b700 1 -- 172.21.15.162:0/2364751048 --> [v2:172.21.15.162:6800/1396866108,v1:172.21.15.162:6801/1396866108] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f5524109f10 con 0x7f550c06ed20 2024-04-27T04:15:41.654 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:15:41.655+0000 7f5520ff9700 1 -- 172.21.15.162:0/2364751048 <== mgr.34104 v2:172.21.15.162:6800/1396866108 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+400 (secure 0 0 0) 0x7f5524109f10 con 0x7f550c06ed20 2024-04-27T04:15:41.656 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:15:41.657+0000 7f552a85b700 1 -- 172.21.15.162:0/2364751048 >> [v2:172.21.15.162:6800/1396866108,v1:172.21.15.162:6801/1396866108] conn(0x7f550c06ed20 msgr2=0x7f550c0711d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:15:41.657 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:15:41.657+0000 7f552a85b700 1 --2- 172.21.15.162:0/2364751048 >> [v2:172.21.15.162:6800/1396866108,v1:172.21.15.162:6801/1396866108] conn(0x7f550c06ed20 0x7f550c0711d0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f5518004890 tx=0x7f5518009e30 comp rx=0 tx=0).stop 2024-04-27T04:15:41.657 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:15:41.657+0000 7f552a85b700 1 -- 172.21.15.162:0/2364751048 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5524102de0 msgr2=0x7f5524113b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:15:41.657 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:15:41.657+0000 7f552a85b700 1 --2- 172.21.15.162:0/2364751048 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5524102de0 0x7f5524113b80 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f5514002a50 tx=0x7f55140046b0 comp rx=0 tx=0).stop 2024-04-27T04:15:41.657 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:15:41.657+0000 7f552a85b700 1 -- 172.21.15.162:0/2364751048 shutdown_connections 2024-04-27T04:15:41.657 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:15:41.657+0000 7f552a85b700 1 --2- 172.21.15.162:0/2364751048 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5524102de0 0x7f5524113b80 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:15:41.657 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:15:41.657+0000 7f552a85b700 1 --2- 172.21.15.162:0/2364751048 >> [v2:172.21.15.162:6800/1396866108,v1:172.21.15.162:6801/1396866108] conn(0x7f550c06ed20 0x7f550c0711d0 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:15:41.657 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:15:41.657+0000 7f552a85b700 1 --2- 172.21.15.162:0/2364751048 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5524103780 0x7f55241140c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:15:41.657 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:15:41.658+0000 7f552a85b700 1 -- 172.21.15.162:0/2364751048 >> 172.21.15.162:0/2364751048 conn(0x7f55240f89b0 msgr2=0x7f55240f9900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:15:41.657 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:15:41.658+0000 7f552a85b700 1 -- 172.21.15.162:0/2364751048 shutdown_connections 2024-04-27T04:15:41.657 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:15:41.658+0000 7f552a85b700 1 -- 172.21.15.162:0/2364751048 wait complete. 2024-04-27T04:15:41.764 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:15:41.762+0000 7f88075ed700 1 -- 172.21.15.162:0/2589474712 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8800102680 msgr2=0x7f88000fbcf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:15:41.764 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:15:41.762+0000 7f88075ed700 1 --2- 172.21.15.162:0/2589474712 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8800102680 0x7f88000fbcf0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f87f0004940 tx=0x7f87f0009e30 comp rx=0 tx=0).stop 2024-04-27T04:15:41.764 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:15:41.762+0000 7f88075ed700 1 -- 172.21.15.162:0/2589474712 shutdown_connections 2024-04-27T04:15:41.765 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:15:41.762+0000 7f88075ed700 1 --2- 172.21.15.162:0/2589474712 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8800102680 0x7f88000fbcf0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:15:41.765 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:15:41.762+0000 7f88075ed700 1 --2- 172.21.15.162:0/2589474712 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8800101ce0 0x7f88001020b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:15:41.765 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:15:41.762+0000 7f88075ed700 1 -- 172.21.15.162:0/2589474712 >> 172.21.15.162:0/2589474712 conn(0x7f88000f7850 msgr2=0x7f88000f9c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:15:41.765 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:15:41.763+0000 7f88075ed700 1 -- 172.21.15.162:0/2589474712 shutdown_connections 2024-04-27T04:15:41.765 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:15:41.763+0000 7f88075ed700 1 -- 172.21.15.162:0/2589474712 wait complete. 2024-04-27T04:15:41.765 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:15:41.763+0000 7f88075ed700 1 Processor -- start 2024-04-27T04:15:41.765 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:15:41.763+0000 7f88075ed700 1 -- start start 2024-04-27T04:15:41.765 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:15:41.764+0000 7f88075ed700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8800101ce0 0x7f880010c930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:15:41.765 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:15:41.764+0000 7f88075ed700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8800102680 0x7f880010ce70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:15:41.765 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:15:41.764+0000 7f88075ed700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f880010d3b0 con 0x7f8800102680 2024-04-27T04:15:41.765 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:15:41.764+0000 7f88075ed700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f880010d520 con 0x7f8800101ce0 2024-04-27T04:15:41.765 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:15:41.764+0000 7f8804b88700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8800102680 0x7f880010ce70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:15:41.765 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:15:41.764+0000 7f8804b88700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8800102680 0x7f880010ce70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:50002/0 (socket says 172.21.15.162:50002) 2024-04-27T04:15:41.765 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:15:41.764+0000 7f8804b88700 1 -- 172.21.15.162:0/134455695 learned_addr learned my addr 172.21.15.162:0/134455695 (peer_addr_for_me v2:172.21.15.162:0/0) 2024-04-27T04:15:41.766 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:15:41.764+0000 7f8805389700 1 --2- 172.21.15.162:0/134455695 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8800101ce0 0x7f880010c930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:15:41.766 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:15:41.764+0000 7f8804b88700 1 -- 172.21.15.162:0/134455695 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8800101ce0 msgr2=0x7f880010c930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:15:41.766 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:15:41.764+0000 7f8804b88700 1 --2- 172.21.15.162:0/134455695 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8800101ce0 0x7f880010c930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:15:41.766 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:15:41.764+0000 7f8804b88700 1 -- 172.21.15.162:0/134455695 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87f0005040 con 0x7f8800102680 2024-04-27T04:15:41.766 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:15:41.764+0000 7f8805389700 1 --2- 172.21.15.162:0/134455695 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8800101ce0 0x7f880010c930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-27T04:15:41.766 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:15:41.765+0000 7f8804b88700 1 --2- 172.21.15.162:0/134455695 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8800102680 0x7f880010ce70 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f87f0009100 tx=0x7f87f0009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:15:41.766 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:15:41.765+0000 7f87f5ffb700 1 -- 172.21.15.162:0/134455695 <== mon.0 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f87f000e040 con 0x7f8800102680 2024-04-27T04:15:41.766 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:15:41.765+0000 7f88075ed700 1 -- 172.21.15.162:0/134455695 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f880010d7a0 con 0x7f8800102680 2024-04-27T04:15:41.766 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:15:41.765+0000 7f87f5ffb700 1 -- 172.21.15.162:0/134455695 <== mon.0 v2:172.21.15.162:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f87f0013420 con 0x7f8800102680 2024-04-27T04:15:41.766 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:15:41.765+0000 7f87f5ffb700 1 -- 172.21.15.162:0/134455695 <== mon.0 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f87f001c610 con 0x7f8800102680 2024-04-27T04:15:41.767 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:15:41.765+0000 7f88075ed700 1 -- 172.21.15.162:0/134455695 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f880011a320 con 0x7f8800102680 2024-04-27T04:15:41.767 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:15:41.767+0000 7f88075ed700 1 -- 172.21.15.162:0/134455695 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f88000fd430 con 0x7f8800102680 2024-04-27T04:15:41.767 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:15:41.767+0000 7f87f5ffb700 1 -- 172.21.15.162:0/134455695 <== mon.0 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 38) v1 ==== 92849+0+0 (secure 0 0 0) 0x7f87f0013590 con 0x7f8800102680 2024-04-27T04:15:41.767 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:15:41.767+0000 7f87f5ffb700 1 --2- 172.21.15.162:0/134455695 >> [v2:172.21.15.162:6800/1396866108,v1:172.21.15.162:6801/1396866108] conn(0x7f87ec06ed20 0x7f87ec0711d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:15:41.768 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:15:41.767+0000 7f87f5ffb700 1 -- 172.21.15.162:0/134455695 <== mon.0 v2:172.21.15.162:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 6410+0+0 (secure 0 0 0) 0x7f87f008fad0 con 0x7f8800102680 2024-04-27T04:15:41.768 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:15:41.768+0000 7f8805389700 1 --2- 172.21.15.162:0/134455695 >> [v2:172.21.15.162:6800/1396866108,v1:172.21.15.162:6801/1396866108] conn(0x7f87ec06ed20 0x7f87ec0711d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:15:41.768 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:15:41.768+0000 7f8805389700 1 --2- 172.21.15.162:0/134455695 >> [v2:172.21.15.162:6800/1396866108,v1:172.21.15.162:6801/1396866108] conn(0x7f87ec06ed20 0x7f87ec0711d0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f87fc009fd0 tx=0x7f87fc009e50 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:15:41.772 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:15:41.773+0000 7f87f5ffb700 1 -- 172.21.15.162:0/134455695 <== mon.0 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7f87f005c260 con 0x7f8800102680 2024-04-27T04:15:41.920 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:15:41 smithi162 ceph-mon[110641]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' 2024-04-27T04:15:41.920 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:15:41 smithi162 ceph-mon[110641]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' 2024-04-27T04:15:41.921 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:15:41 smithi162 ceph-mon[110641]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T04:15:41.921 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:15:41 smithi162 ceph-mon[110641]: from='client.34190 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-27T04:15:41.921 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:15:41 smithi162 ceph-mon[110641]: pgmap v47: 65 pgs: 18 active+undersized+degraded, 16 active+undersized, 31 active+clean; 208 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 511 B/s rd, 0 op/s; 41/231 objects degraded (17.749%) 2024-04-27T04:15:41.921 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:15:41 smithi162 ceph-mon[110641]: from='client.34194 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-27T04:15:41.994 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:15:41.995+0000 7f88075ed700 1 -- 172.21.15.162:0/134455695 --> [v2:172.21.15.162:6800/1396866108,v1:172.21.15.162:6801/1396866108] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f8800108e30 con 0x7f87ec06ed20 2024-04-27T04:15:42.004 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:15:42.005+0000 7f87f5ffb700 1 -- 172.21.15.162:0/134455695 <== mgr.34104 v2:172.21.15.162:6800/1396866108 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3498 (secure 0 0 0) 0x7f8800108e30 con 0x7f87ec06ed20 2024-04-27T04:15:42.004 INFO:teuthology.orchestra.run.smithi162.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-27T04:15:42.004 INFO:teuthology.orchestra.run.smithi162.stdout:alertmanager.smithi162 smithi162 *:9093,9094 running (2m) 16s ago 13m 27.6M - 0.25.0 c8568f914cd2 40e334659017 2024-04-27T04:15:42.005 INFO:teuthology.orchestra.run.smithi162.stdout:crash.smithi162 smithi162 running (27s) 16s ago 14m 6660k - 19.0.0-3253-g4e213a58 67e414ff4bc4 1aa8f4d5a28b 2024-04-27T04:15:42.005 INFO:teuthology.orchestra.run.smithi162.stdout:crash.smithi181 smithi181 running (22s) 18s ago 13m 6656k - 19.0.0-3253-g4e213a58 67e414ff4bc4 832d9cbc5c30 2024-04-27T04:15:42.005 INFO:teuthology.orchestra.run.smithi162.stdout:grafana.smithi162 smithi162 *:3000 running (2m) 16s ago 13m 75.3M - 9.4.12 2bacad6d85d8 3d78c478f03f 2024-04-27T04:15:42.005 INFO:teuthology.orchestra.run.smithi162.stdout:mds.cephfs.smithi162.gzlyjj smithi162 running (8m) 16s ago 8m 13.9M - 17.2.7-907-g2be3efa9 1c5834633715 27e5b030dd32 2024-04-27T04:15:42.005 INFO:teuthology.orchestra.run.smithi162.stdout:mds.cephfs.smithi162.xcnapk smithi162 running (8m) 16s ago 8m 100M - 17.2.7-907-g2be3efa9 1c5834633715 38f9030df029 2024-04-27T04:15:42.005 INFO:teuthology.orchestra.run.smithi162.stdout:mds.cephfs.smithi181.bkkcdm smithi181 running (8m) 18s ago 8m 15.3M - 17.2.7-907-g2be3efa9 1c5834633715 af55afca1263 2024-04-27T04:15:42.005 INFO:teuthology.orchestra.run.smithi162.stdout:mds.cephfs.smithi181.dzomgh smithi181 running (8m) 18s ago 8m 143M - 17.2.7-907-g2be3efa9 1c5834633715 6733296597f3 2024-04-27T04:15:42.005 INFO:teuthology.orchestra.run.smithi162.stdout:mgr.smithi162.oennfn smithi162 *:8443,9283,8765 running (4m) 16s ago 15m 496M - 19.0.0-3253-g4e213a58 67e414ff4bc4 58a748d3637b 2024-04-27T04:15:42.005 INFO:teuthology.orchestra.run.smithi162.stdout:mgr.smithi181.vtwuhz smithi181 *:8443,9283,8765 running (3m) 18s ago 13m 450M - 19.0.0-3253-g4e213a58 67e414ff4bc4 813584297b1d 2024-04-27T04:15:42.005 INFO:teuthology.orchestra.run.smithi162.stdout:mon.smithi162 smithi162 running (93s) 16s ago 15m 40.6M 2048M 19.0.0-3253-g4e213a58 67e414ff4bc4 375adaf4468f 2024-04-27T04:15:42.005 INFO:teuthology.orchestra.run.smithi162.stdout:mon.smithi181 smithi181 running (68s) 18s ago 13m 33.1M 2048M 19.0.0-3253-g4e213a58 67e414ff4bc4 77b71f6d522f 2024-04-27T04:15:42.005 INFO:teuthology.orchestra.run.smithi162.stdout:node-exporter.smithi162 smithi162 *:9100 running (3m) 16s ago 14m 15.3M - 1.5.0 0da6a335fe13 5c4c5ea9c1d7 2024-04-27T04:15:42.005 INFO:teuthology.orchestra.run.smithi162.stdout:node-exporter.smithi181 smithi181 *:9100 running (3m) 18s ago 13m 15.7M - 1.5.0 0da6a335fe13 9067f14c0519 2024-04-27T04:15:42.006 INFO:teuthology.orchestra.run.smithi162.stdout:osd.0 smithi162 starting - - - 1557M 2024-04-27T04:15:42.006 INFO:teuthology.orchestra.run.smithi162.stdout:osd.1 smithi162 running (11m) 16s ago 11m 247M 1557M 17.2.7-907-g2be3efa9 1c5834633715 aefb464c46e8 2024-04-27T04:15:42.006 INFO:teuthology.orchestra.run.smithi162.stdout:osd.2 smithi162 running (11m) 16s ago 11m 206M 1557M 17.2.7-907-g2be3efa9 1c5834633715 70f16d86d38e 2024-04-27T04:15:42.006 INFO:teuthology.orchestra.run.smithi162.stdout:osd.3 smithi181 running (11m) 18s ago 11m 278M 2581M 17.2.7-907-g2be3efa9 1c5834633715 030d829be54e 2024-04-27T04:15:42.006 INFO:teuthology.orchestra.run.smithi162.stdout:osd.4 smithi181 running (10m) 18s ago 10m 247M 2581M 17.2.7-907-g2be3efa9 1c5834633715 512afbee7cf7 2024-04-27T04:15:42.006 INFO:teuthology.orchestra.run.smithi162.stdout:osd.5 smithi181 running (10m) 18s ago 10m 224M 2581M 17.2.7-907-g2be3efa9 1c5834633715 f47a1daac1ea 2024-04-27T04:15:42.006 INFO:teuthology.orchestra.run.smithi162.stdout:prometheus.smithi162 smithi162 *:9095 running (3m) 16s ago 13m 67.5M - 2.43.0 a07b618ecd1d f30657a43324 2024-04-27T04:15:42.007 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:15:42.007+0000 7f88075ed700 1 -- 172.21.15.162:0/134455695 >> [v2:172.21.15.162:6800/1396866108,v1:172.21.15.162:6801/1396866108] conn(0x7f87ec06ed20 msgr2=0x7f87ec0711d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:15:42.007 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:15:42.007+0000 7f88075ed700 1 --2- 172.21.15.162:0/134455695 >> [v2:172.21.15.162:6800/1396866108,v1:172.21.15.162:6801/1396866108] conn(0x7f87ec06ed20 0x7f87ec0711d0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f87fc009fd0 tx=0x7f87fc009e50 comp rx=0 tx=0).stop 2024-04-27T04:15:42.007 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:15:42.007+0000 7f88075ed700 1 -- 172.21.15.162:0/134455695 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8800102680 msgr2=0x7f880010ce70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:15:42.007 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:15:42.007+0000 7f88075ed700 1 --2- 172.21.15.162:0/134455695 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8800102680 0x7f880010ce70 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f87f0009100 tx=0x7f87f0009e30 comp rx=0 tx=0).stop 2024-04-27T04:15:42.007 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:15:42.007+0000 7f88075ed700 1 -- 172.21.15.162:0/134455695 shutdown_connections 2024-04-27T04:15:42.007 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:15:42.007+0000 7f88075ed700 1 --2- 172.21.15.162:0/134455695 >> [v2:172.21.15.162:6800/1396866108,v1:172.21.15.162:6801/1396866108] conn(0x7f87ec06ed20 0x7f87ec0711d0 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:15:42.007 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:15:42.007+0000 7f88075ed700 1 --2- 172.21.15.162:0/134455695 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8800101ce0 0x7f880010c930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:15:42.007 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:15:42.007+0000 7f88075ed700 1 --2- 172.21.15.162:0/134455695 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8800102680 0x7f880010ce70 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:15:42.007 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:15:42.007+0000 7f88075ed700 1 -- 172.21.15.162:0/134455695 >> 172.21.15.162:0/134455695 conn(0x7f88000f7850 msgr2=0x7f88000f8820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:15:42.007 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:15:42.008+0000 7f88075ed700 1 -- 172.21.15.162:0/134455695 shutdown_connections 2024-04-27T04:15:42.008 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:15:42.008+0000 7f88075ed700 1 -- 172.21.15.162:0/134455695 wait complete. 2024-04-27T04:15:42.113 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:15:42.113+0000 7fcee8ef7700 1 -- 172.21.15.162:0/3048608701 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fcee4101cf0 msgr2=0x7fcee41020c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:15:42.113 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:15:42.113+0000 7fcee8ef7700 1 --2- 172.21.15.162:0/3048608701 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fcee4101cf0 0x7fcee41020c0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fced4004940 tx=0x7fced4009e30 comp rx=0 tx=0).stop 2024-04-27T04:15:42.113 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:15:42.113+0000 7fcee8ef7700 1 -- 172.21.15.162:0/3048608701 shutdown_connections 2024-04-27T04:15:42.113 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:15:42.113+0000 7fcee8ef7700 1 --2- 172.21.15.162:0/3048608701 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcee4102690 0x7fcee40fbd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:15:42.113 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:15:42.113+0000 7fcee8ef7700 1 --2- 172.21.15.162:0/3048608701 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fcee4101cf0 0x7fcee41020c0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:15:42.113 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:15:42.113+0000 7fcee8ef7700 1 -- 172.21.15.162:0/3048608701 >> 172.21.15.162:0/3048608701 conn(0x7fcee40f7860 msgr2=0x7fcee40f9c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:15:42.113 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:15:42.113+0000 7fcee8ef7700 1 -- 172.21.15.162:0/3048608701 shutdown_connections 2024-04-27T04:15:42.114 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:15:42.113+0000 7fcee8ef7700 1 -- 172.21.15.162:0/3048608701 wait complete. 2024-04-27T04:15:42.114 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:15:42.114+0000 7fcee8ef7700 1 Processor -- start 2024-04-27T04:15:42.114 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:15:42.114+0000 7fcee8ef7700 1 -- start start 2024-04-27T04:15:42.115 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:15:42.114+0000 7fcee8ef7700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fcee4101cf0 0x7fcee4112860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:15:42.116 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:15:42.114+0000 7fcee8ef7700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcee4102690 0x7fcee4112da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:15:42.116 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:15:42.114+0000 7fcee8ef7700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcee41133f0 con 0x7fcee4101cf0 2024-04-27T04:15:42.116 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:15:42.114+0000 7fcee8ef7700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcee410c9f0 con 0x7fcee4102690 2024-04-27T04:15:42.116 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:15:42.115+0000 7fcee259c700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fcee4101cf0 0x7fcee4112860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:15:42.116 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:15:42.115+0000 7fcee259c700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fcee4101cf0 0x7fcee4112860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:50026/0 (socket says 172.21.15.162:50026) 2024-04-27T04:15:42.116 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:15:42.115+0000 7fcee259c700 1 -- 172.21.15.162:0/2694661282 learned_addr learned my addr 172.21.15.162:0/2694661282 (peer_addr_for_me v2:172.21.15.162:0/0) 2024-04-27T04:15:42.116 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:15:42.115+0000 7fcee1d9b700 1 --2- 172.21.15.162:0/2694661282 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcee4102690 0x7fcee4112da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:15:42.116 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:15:42.115+0000 7fcee259c700 1 -- 172.21.15.162:0/2694661282 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcee4102690 msgr2=0x7fcee4112da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:15:42.116 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:15:42.115+0000 7fcee259c700 1 --2- 172.21.15.162:0/2694661282 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcee4102690 0x7fcee4112da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:15:42.116 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:15:42.115+0000 7fcee259c700 1 -- 172.21.15.162:0/2694661282 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fced4005040 con 0x7fcee4101cf0 2024-04-27T04:15:42.116 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:15:42.115+0000 7fcee1d9b700 1 --2- 172.21.15.162:0/2694661282 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcee4102690 0x7fcee4112da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-27T04:15:42.116 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:15:42.115+0000 7fcee259c700 1 --2- 172.21.15.162:0/2694661282 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fcee4101cf0 0x7fcee4112860 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fced400b3b0 tx=0x7fced4009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:15:42.117 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:15:42.115+0000 7fced2ffd700 1 -- 172.21.15.162:0/2694661282 <== mon.0 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fced400e040 con 0x7fcee4101cf0 2024-04-27T04:15:42.117 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:15:42.115+0000 7fced2ffd700 1 -- 172.21.15.162:0/2694661282 <== mon.0 v2:172.21.15.162:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fced400bbc0 con 0x7fcee4101cf0 2024-04-27T04:15:42.117 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:15:42.115+0000 7fcee8ef7700 1 -- 172.21.15.162:0/2694661282 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcee410cc70 con 0x7fcee4101cf0 2024-04-27T04:15:42.117 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:15:42.115+0000 7fced2ffd700 1 -- 172.21.15.162:0/2694661282 <== mon.0 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fced401c3f0 con 0x7fcee4101cf0 2024-04-27T04:15:42.117 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:15:42.115+0000 7fcee8ef7700 1 -- 172.21.15.162:0/2694661282 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcee410d230 con 0x7fcee4101cf0 2024-04-27T04:15:42.117 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:15:42.116+0000 7fcee8ef7700 1 -- 172.21.15.162:0/2694661282 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcee40fd440 con 0x7fcee4101cf0 2024-04-27T04:15:42.117 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:15:42.117+0000 7fced2ffd700 1 -- 172.21.15.162:0/2694661282 <== mon.0 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 38) v1 ==== 92849+0+0 (secure 0 0 0) 0x7fced400bd30 con 0x7fcee4101cf0 2024-04-27T04:15:42.118 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:15:42.117+0000 7fced2ffd700 1 --2- 172.21.15.162:0/2694661282 >> [v2:172.21.15.162:6800/1396866108,v1:172.21.15.162:6801/1396866108] conn(0x7fcecc06ed30 0x7fcecc0711e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:15:42.118 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:15:42.117+0000 7fced2ffd700 1 -- 172.21.15.162:0/2694661282 <== mon.0 v2:172.21.15.162:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 6410+0+0 (secure 0 0 0) 0x7fced408fca0 con 0x7fcee4101cf0 2024-04-27T04:15:42.118 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:15:42.117+0000 7fcee1d9b700 1 --2- 172.21.15.162:0/2694661282 >> [v2:172.21.15.162:6800/1396866108,v1:172.21.15.162:6801/1396866108] conn(0x7fcecc06ed30 0x7fcecc0711e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:15:42.118 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:15:42.118+0000 7fcee1d9b700 1 --2- 172.21.15.162:0/2694661282 >> [v2:172.21.15.162:6800/1396866108,v1:172.21.15.162:6801/1396866108] conn(0x7fcecc06ed30 0x7fcecc0711e0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fced8007fd0 tx=0x7fced8009040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:15:42.120 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:15:42.122+0000 7fced2ffd700 1 -- 172.21.15.162:0/2694661282 <== mon.0 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7fced4094070 con 0x7fcee4101cf0 2024-04-27T04:15:42.172 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:15:41 smithi181 ceph-mon[81134]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' 2024-04-27T04:15:42.172 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:15:41 smithi181 ceph-mon[81134]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' 2024-04-27T04:15:42.172 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:15:41 smithi181 ceph-mon[81134]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T04:15:42.172 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:15:41 smithi181 ceph-mon[81134]: from='client.34190 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-27T04:15:42.172 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:15:41 smithi181 ceph-mon[81134]: pgmap v47: 65 pgs: 18 active+undersized+degraded, 16 active+undersized, 31 active+clean; 208 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 511 B/s rd, 0 op/s; 41/231 objects degraded (17.749%) 2024-04-27T04:15:42.172 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:15:41 smithi181 ceph-mon[81134]: from='client.34194 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-27T04:15:42.446 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:15:42.447+0000 7fcee8ef7700 1 -- 172.21.15.162:0/2694661282 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7fcee4108e80 con 0x7fcee4101cf0 2024-04-27T04:15:42.447 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:15:42.448+0000 7fced2ffd700 1 -- 172.21.15.162:0/2694661282 <== mon.0 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+735 (secure 0 0 0) 0x7fced405c430 con 0x7fcee4101cf0 2024-04-27T04:15:42.447 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-04-27T04:15:42.447 INFO:teuthology.orchestra.run.smithi162.stdout: "mon": { 2024-04-27T04:15:42.447 INFO:teuthology.orchestra.run.smithi162.stdout: "ceph version 19.0.0-3253-g4e213a58 (4e213a58e787437ff508e6c890ccd5fd32dc96b5) squid (dev)": 2 2024-04-27T04:15:42.447 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-04-27T04:15:42.447 INFO:teuthology.orchestra.run.smithi162.stdout: "mgr": { 2024-04-27T04:15:42.447 INFO:teuthology.orchestra.run.smithi162.stdout: "ceph version 19.0.0-3253-g4e213a58 (4e213a58e787437ff508e6c890ccd5fd32dc96b5) squid (dev)": 2 2024-04-27T04:15:42.448 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-04-27T04:15:42.448 INFO:teuthology.orchestra.run.smithi162.stdout: "osd": { 2024-04-27T04:15:42.448 INFO:teuthology.orchestra.run.smithi162.stdout: "ceph version 17.2.7-907-g2be3efa9 (2be3efa97a7972f8e925bd899b474329979b74d6) quincy (stable)": 5 2024-04-27T04:15:42.448 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-04-27T04:15:42.448 INFO:teuthology.orchestra.run.smithi162.stdout: "mds": { 2024-04-27T04:15:42.448 INFO:teuthology.orchestra.run.smithi162.stdout: "ceph version 17.2.7-907-g2be3efa9 (2be3efa97a7972f8e925bd899b474329979b74d6) quincy (stable)": 4 2024-04-27T04:15:42.448 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-04-27T04:15:42.448 INFO:teuthology.orchestra.run.smithi162.stdout: "overall": { 2024-04-27T04:15:42.448 INFO:teuthology.orchestra.run.smithi162.stdout: "ceph version 17.2.7-907-g2be3efa9 (2be3efa97a7972f8e925bd899b474329979b74d6) quincy (stable)": 9, 2024-04-27T04:15:42.448 INFO:teuthology.orchestra.run.smithi162.stdout: "ceph version 19.0.0-3253-g4e213a58 (4e213a58e787437ff508e6c890ccd5fd32dc96b5) squid (dev)": 4 2024-04-27T04:15:42.448 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-04-27T04:15:42.448 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-04-27T04:15:42.450 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:15:42.450+0000 7fcee8ef7700 1 -- 172.21.15.162:0/2694661282 >> [v2:172.21.15.162:6800/1396866108,v1:172.21.15.162:6801/1396866108] conn(0x7fcecc06ed30 msgr2=0x7fcecc0711e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:15:42.450 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:15:42.450+0000 7fcee8ef7700 1 --2- 172.21.15.162:0/2694661282 >> [v2:172.21.15.162:6800/1396866108,v1:172.21.15.162:6801/1396866108] conn(0x7fcecc06ed30 0x7fcecc0711e0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fced8007fd0 tx=0x7fced8009040 comp rx=0 tx=0).stop 2024-04-27T04:15:42.450 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:15:42.451+0000 7fcee8ef7700 1 -- 172.21.15.162:0/2694661282 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fcee4101cf0 msgr2=0x7fcee4112860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:15:42.450 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:15:42.451+0000 7fcee8ef7700 1 --2- 172.21.15.162:0/2694661282 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fcee4101cf0 0x7fcee4112860 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fced400b3b0 tx=0x7fced4009e30 comp rx=0 tx=0).stop 2024-04-27T04:15:42.450 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:15:42.451+0000 7fcee8ef7700 1 -- 172.21.15.162:0/2694661282 shutdown_connections 2024-04-27T04:15:42.450 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:15:42.451+0000 7fcee8ef7700 1 --2- 172.21.15.162:0/2694661282 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fcee4101cf0 0x7fcee4112860 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:15:42.450 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:15:42.451+0000 7fcee8ef7700 1 --2- 172.21.15.162:0/2694661282 >> [v2:172.21.15.162:6800/1396866108,v1:172.21.15.162:6801/1396866108] conn(0x7fcecc06ed30 0x7fcecc0711e0 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:15:42.450 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:15:42.451+0000 7fcee8ef7700 1 --2- 172.21.15.162:0/2694661282 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcee4102690 0x7fcee4112da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:15:42.450 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:15:42.451+0000 7fcee8ef7700 1 -- 172.21.15.162:0/2694661282 >> 172.21.15.162:0/2694661282 conn(0x7fcee40f7860 msgr2=0x7fcee40f8750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:15:42.450 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:15:42.451+0000 7fcee8ef7700 1 -- 172.21.15.162:0/2694661282 shutdown_connections 2024-04-27T04:15:42.451 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:15:42.451+0000 7fcee8ef7700 1 -- 172.21.15.162:0/2694661282 wait complete. 2024-04-27T04:15:42.553 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:15:42.554+0000 7fed0a0bf700 1 -- 172.21.15.162:0/2523845267 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fed04103760 msgr2=0x7fed040fcdd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:15:42.553 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:15:42.554+0000 7fed0a0bf700 1 --2- 172.21.15.162:0/2523845267 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fed04103760 0x7fed040fcdd0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fecf8004940 tx=0x7fecf8009e30 comp rx=0 tx=0).stop 2024-04-27T04:15:42.553 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:15:42.554+0000 7fed0a0bf700 1 -- 172.21.15.162:0/2523845267 shutdown_connections 2024-04-27T04:15:42.553 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:15:42.554+0000 7fed0a0bf700 1 --2- 172.21.15.162:0/2523845267 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fed04103760 0x7fed040fcdd0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:15:42.553 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:15:42.554+0000 7fed0a0bf700 1 --2- 172.21.15.162:0/2523845267 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fed04102dc0 0x7fed04103190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:15:42.553 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:15:42.554+0000 7fed0a0bf700 1 -- 172.21.15.162:0/2523845267 >> 172.21.15.162:0/2523845267 conn(0x7fed040f8930 msgr2=0x7fed040fad40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:15:42.553 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:15:42.554+0000 7fed0a0bf700 1 -- 172.21.15.162:0/2523845267 shutdown_connections 2024-04-27T04:15:42.554 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:15:42.554+0000 7fed0a0bf700 1 -- 172.21.15.162:0/2523845267 wait complete. 2024-04-27T04:15:42.554 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:15:42.554+0000 7fed0a0bf700 1 Processor -- start 2024-04-27T04:15:42.555 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:15:42.554+0000 7fed0a0bf700 1 -- start start 2024-04-27T04:15:42.555 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:15:42.555+0000 7fed0a0bf700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fed04102dc0 0x7fed04115b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:15:42.556 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:15:42.555+0000 7fed0a0bf700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fed04103760 0x7fed04116090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:15:42.556 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:15:42.555+0000 7fed0a0bf700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed041166e0 con 0x7fed04103760 2024-04-27T04:15:42.556 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:15:42.555+0000 7fed0a0bf700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed0410fcc0 con 0x7fed04102dc0 2024-04-27T04:15:42.556 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:15:42.555+0000 7fed02ffd700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fed04103760 0x7fed04116090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:15:42.556 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:15:42.555+0000 7fed02ffd700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fed04103760 0x7fed04116090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:50048/0 (socket says 172.21.15.162:50048) 2024-04-27T04:15:42.556 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:15:42.555+0000 7fed02ffd700 1 -- 172.21.15.162:0/1051910817 learned_addr learned my addr 172.21.15.162:0/1051910817 (peer_addr_for_me v2:172.21.15.162:0/0) 2024-04-27T04:15:42.556 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:15:42.555+0000 7fed037fe700 1 --2- 172.21.15.162:0/1051910817 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fed04102dc0 0x7fed04115b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:15:42.556 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:15:42.555+0000 7fed02ffd700 1 -- 172.21.15.162:0/1051910817 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fed04102dc0 msgr2=0x7fed04115b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:15:42.556 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:15:42.555+0000 7fed02ffd700 1 --2- 172.21.15.162:0/1051910817 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fed04102dc0 0x7fed04115b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:15:42.556 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:15:42.555+0000 7fed02ffd700 1 -- 172.21.15.162:0/1051910817 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fecf8005040 con 0x7fed04103760 2024-04-27T04:15:42.556 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:15:42.555+0000 7fed02ffd700 1 --2- 172.21.15.162:0/1051910817 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fed04103760 0x7fed04116090 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fecf8009180 tx=0x7fecf8009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:15:42.556 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:15:42.555+0000 7fecebfff700 1 -- 172.21.15.162:0/1051910817 <== mon.0 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fecf800e040 con 0x7fed04103760 2024-04-27T04:15:42.556 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:15:42.555+0000 7fecebfff700 1 -- 172.21.15.162:0/1051910817 <== mon.0 v2:172.21.15.162:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fecf8013420 con 0x7fed04103760 2024-04-27T04:15:42.556 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:15:42.555+0000 7fed0a0bf700 1 -- 172.21.15.162:0/1051910817 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fed0410ff40 con 0x7fed04103760 2024-04-27T04:15:42.557 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:15:42.555+0000 7fecebfff700 1 -- 172.21.15.162:0/1051910817 <== mon.0 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fecf801c610 con 0x7fed04103760 2024-04-27T04:15:42.557 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:15:42.555+0000 7fed0a0bf700 1 -- 172.21.15.162:0/1051910817 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fed04110500 con 0x7fed04103760 2024-04-27T04:15:42.557 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:15:42.557+0000 7fed0a0bf700 1 -- 172.21.15.162:0/1051910817 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fed040fe510 con 0x7fed04103760 2024-04-27T04:15:42.557 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:15:42.557+0000 7fecebfff700 1 -- 172.21.15.162:0/1051910817 <== mon.0 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 38) v1 ==== 92849+0+0 (secure 0 0 0) 0x7fecf8013a00 con 0x7fed04103760 2024-04-27T04:15:42.557 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:15:42.557+0000 7fecebfff700 1 --2- 172.21.15.162:0/1051910817 >> [v2:172.21.15.162:6800/1396866108,v1:172.21.15.162:6801/1396866108] conn(0x7fecec06ed30 0x7fecec0711e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:15:42.558 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:15:42.557+0000 7fecebfff700 1 -- 172.21.15.162:0/1051910817 <== mon.0 v2:172.21.15.162:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 6410+0+0 (secure 0 0 0) 0x7fecf808fbd0 con 0x7fed04103760 2024-04-27T04:15:42.558 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:15:42.558+0000 7fed037fe700 1 --2- 172.21.15.162:0/1051910817 >> [v2:172.21.15.162:6800/1396866108,v1:172.21.15.162:6801/1396866108] conn(0x7fecec06ed30 0x7fecec0711e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:15:42.558 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:15:42.558+0000 7fed037fe700 1 --2- 172.21.15.162:0/1051910817 >> [v2:172.21.15.162:6800/1396866108,v1:172.21.15.162:6801/1396866108] conn(0x7fecec06ed30 0x7fecec0711e0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fecf4004f40 tx=0x7fecf400b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:15:42.561 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:15:42.562+0000 7fecebfff700 1 -- 172.21.15.162:0/1051910817 <== mon.0 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7fecf805c410 con 0x7fed04103760 2024-04-27T04:15:42.819 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:15:42.820+0000 7fed0a0bf700 1 -- 172.21.15.162:0/1051910817 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7fed04109ea0 con 0x7fed04103760 2024-04-27T04:15:42.819 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:15:42.820+0000 7fecebfff700 1 -- 172.21.15.162:0/1051910817 <== mon.0 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 15 v15) v1 ==== 76+0+1911 (secure 0 0 0) 0x7fecf805bfa0 con 0x7fed04103760 2024-04-27T04:15:42.819 INFO:teuthology.orchestra.run.smithi162.stdout:e15 2024-04-27T04:15:42.820 INFO:teuthology.orchestra.run.smithi162.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-04-27T04:15:42.820 INFO:teuthology.orchestra.run.smithi162.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-27T04:15:42.820 INFO:teuthology.orchestra.run.smithi162.stdout:legacy client fscid: 1 2024-04-27T04:15:42.820 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-04-27T04:15:42.820 INFO:teuthology.orchestra.run.smithi162.stdout:Filesystem 'cephfs' (1) 2024-04-27T04:15:42.820 INFO:teuthology.orchestra.run.smithi162.stdout:fs_name cephfs 2024-04-27T04:15:42.820 INFO:teuthology.orchestra.run.smithi162.stdout:epoch 15 2024-04-27T04:15:42.820 INFO:teuthology.orchestra.run.smithi162.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-04-27T04:15:42.820 INFO:teuthology.orchestra.run.smithi162.stdout:created 2024-04-27T04:06:53.095749+0000 2024-04-27T04:15:42.820 INFO:teuthology.orchestra.run.smithi162.stdout:modified 2024-04-27T04:07:11.285033+0000 2024-04-27T04:15:42.820 INFO:teuthology.orchestra.run.smithi162.stdout:tableserver 0 2024-04-27T04:15:42.820 INFO:teuthology.orchestra.run.smithi162.stdout:root 0 2024-04-27T04:15:42.820 INFO:teuthology.orchestra.run.smithi162.stdout:session_timeout 60 2024-04-27T04:15:42.820 INFO:teuthology.orchestra.run.smithi162.stdout:session_autoclose 300 2024-04-27T04:15:42.821 INFO:teuthology.orchestra.run.smithi162.stdout:max_file_size 1099511627776 2024-04-27T04:15:42.821 INFO:teuthology.orchestra.run.smithi162.stdout:max_xattr_size 65536 2024-04-27T04:15:42.821 INFO:teuthology.orchestra.run.smithi162.stdout:required_client_features {} 2024-04-27T04:15:42.821 INFO:teuthology.orchestra.run.smithi162.stdout:last_failure 0 2024-04-27T04:15:42.821 INFO:teuthology.orchestra.run.smithi162.stdout:last_failure_osd_epoch 46 2024-04-27T04:15:42.821 INFO:teuthology.orchestra.run.smithi162.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-27T04:15:42.821 INFO:teuthology.orchestra.run.smithi162.stdout:max_mds 1 2024-04-27T04:15:42.821 INFO:teuthology.orchestra.run.smithi162.stdout:in 0 2024-04-27T04:15:42.821 INFO:teuthology.orchestra.run.smithi162.stdout:up {0=24273} 2024-04-27T04:15:42.821 INFO:teuthology.orchestra.run.smithi162.stdout:failed 2024-04-27T04:15:42.821 INFO:teuthology.orchestra.run.smithi162.stdout:damaged 2024-04-27T04:15:42.821 INFO:teuthology.orchestra.run.smithi162.stdout:stopped 2024-04-27T04:15:42.821 INFO:teuthology.orchestra.run.smithi162.stdout:data_pools [3] 2024-04-27T04:15:42.821 INFO:teuthology.orchestra.run.smithi162.stdout:metadata_pool 2 2024-04-27T04:15:42.821 INFO:teuthology.orchestra.run.smithi162.stdout:inline_data disabled 2024-04-27T04:15:42.822 INFO:teuthology.orchestra.run.smithi162.stdout:balancer 2024-04-27T04:15:42.822 INFO:teuthology.orchestra.run.smithi162.stdout:bal_rank_mask -1 2024-04-27T04:15:42.822 INFO:teuthology.orchestra.run.smithi162.stdout:standby_count_wanted 1 2024-04-27T04:15:42.822 INFO:teuthology.orchestra.run.smithi162.stdout:qdb_cluster leader: 0 members: 2024-04-27T04:15:42.822 INFO:teuthology.orchestra.run.smithi162.stdout:[mds.cephfs.smithi181.dzomgh{0:24273} state up:active seq 6 join_fscid=1 addr [v2:172.21.15.181:6824/658214993,v1:172.21.15.181:6825/658214993] compat {c=[1],r=[1],i=[7ff]}] 2024-04-27T04:15:42.822 INFO:teuthology.orchestra.run.smithi162.stdout:[mds.cephfs.smithi162.xcnapk{0:14476} state up:standby-replay seq 2 join_fscid=1 addr [v2:172.21.15.162:6828/1296160676,v1:172.21.15.162:6829/1296160676] compat {c=[1],r=[1],i=[7ff]}] 2024-04-27T04:15:42.822 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-04-27T04:15:42.822 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-04-27T04:15:42.822 INFO:teuthology.orchestra.run.smithi162.stdout:Standby daemons: 2024-04-27T04:15:42.822 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-04-27T04:15:42.822 INFO:teuthology.orchestra.run.smithi162.stdout:[mds.cephfs.smithi162.gzlyjj{-1:14488} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.162:6826/2629103894,v1:172.21.15.162:6827/2629103894] compat {c=[1],r=[1],i=[7ff]}] 2024-04-27T04:15:42.822 INFO:teuthology.orchestra.run.smithi162.stdout:[mds.cephfs.smithi181.bkkcdm{-1:24287} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.181:6826/3756475453,v1:172.21.15.181:6827/3756475453] compat {c=[1],r=[1],i=[7ff]}] 2024-04-27T04:15:42.823 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:15:42.822+0000 7fed0a0bf700 1 -- 172.21.15.162:0/1051910817 >> [v2:172.21.15.162:6800/1396866108,v1:172.21.15.162:6801/1396866108] conn(0x7fecec06ed30 msgr2=0x7fecec0711e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:15:42.823 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:15:42.822+0000 7fed0a0bf700 1 --2- 172.21.15.162:0/1051910817 >> [v2:172.21.15.162:6800/1396866108,v1:172.21.15.162:6801/1396866108] conn(0x7fecec06ed30 0x7fecec0711e0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fecf4004f40 tx=0x7fecf400b040 comp rx=0 tx=0).stop 2024-04-27T04:15:42.823 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:15:42.822+0000 7fed0a0bf700 1 -- 172.21.15.162:0/1051910817 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fed04103760 msgr2=0x7fed04116090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:15:42.823 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:15:42.822+0000 7fed0a0bf700 1 --2- 172.21.15.162:0/1051910817 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fed04103760 0x7fed04116090 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fecf8009180 tx=0x7fecf8009e30 comp rx=0 tx=0).stop 2024-04-27T04:15:42.823 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:15:42.822+0000 7fed0a0bf700 1 -- 172.21.15.162:0/1051910817 shutdown_connections 2024-04-27T04:15:42.823 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:15:42.822+0000 7fed0a0bf700 1 --2- 172.21.15.162:0/1051910817 >> [v2:172.21.15.162:6800/1396866108,v1:172.21.15.162:6801/1396866108] conn(0x7fecec06ed30 0x7fecec0711e0 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:15:42.824 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:15:42.822+0000 7fed0a0bf700 1 --2- 172.21.15.162:0/1051910817 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fed04102dc0 0x7fed04115b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:15:42.824 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:15:42.822+0000 7fed0a0bf700 1 --2- 172.21.15.162:0/1051910817 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fed04103760 0x7fed04116090 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:15:42.824 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:15:42.822+0000 7fed0a0bf700 1 -- 172.21.15.162:0/1051910817 >> 172.21.15.162:0/1051910817 conn(0x7fed040f8930 msgr2=0x7fed040f9860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:15:42.824 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:15:42.823+0000 7fed0a0bf700 1 -- 172.21.15.162:0/1051910817 shutdown_connections 2024-04-27T04:15:42.824 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:15:42.823+0000 7fed0a0bf700 1 -- 172.21.15.162:0/1051910817 wait complete. 2024-04-27T04:15:42.824 INFO:teuthology.orchestra.run.smithi162.stderr:dumped fsmap epoch 15 2024-04-27T04:15:42.928 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:15:42.929+0000 7f93fc5fd700 1 -- 172.21.15.162:0/1683306893 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f93f4101d90 msgr2=0x7f93f4102160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:15:42.929 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:15:42.929+0000 7f93fc5fd700 1 --2- 172.21.15.162:0/1683306893 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f93f4101d90 0x7f93f4102160 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f93ec004940 tx=0x7f93ec009e30 comp rx=0 tx=0).stop 2024-04-27T04:15:42.929 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:15:42.929+0000 7f93fc5fd700 1 -- 172.21.15.162:0/1683306893 shutdown_connections 2024-04-27T04:15:42.929 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:15:42.929+0000 7f93fc5fd700 1 --2- 172.21.15.162:0/1683306893 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f93f40fbd40 0x7f93f40fc1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:15:42.929 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:15:42.929+0000 7f93fc5fd700 1 --2- 172.21.15.162:0/1683306893 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f93f4101d90 0x7f93f4102160 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:15:42.929 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:15:42.929+0000 7f93fc5fd700 1 -- 172.21.15.162:0/1683306893 >> 172.21.15.162:0/1683306893 conn(0x7f93f4071d00 msgr2=0x7f93f4072100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:15:42.929 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:15:42.929+0000 7f93fc5fd700 1 -- 172.21.15.162:0/1683306893 shutdown_connections 2024-04-27T04:15:42.929 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:15:42.929+0000 7f93fc5fd700 1 -- 172.21.15.162:0/1683306893 wait complete. 2024-04-27T04:15:42.929 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:15:42.930+0000 7f93fc5fd700 1 Processor -- start 2024-04-27T04:15:42.929 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:15:42.930+0000 7f93fc5fd700 1 -- start start 2024-04-27T04:15:42.931 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:15:42.930+0000 7f93fc5fd700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f93f40fbd40 0x7f93f410eb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:15:42.931 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:15:42.930+0000 7f93fc5fd700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f93f4101d90 0x7f93f410f080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:15:42.931 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:15:42.930+0000 7f93fc5fd700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93f410f6d0 con 0x7f93f40fbd40 2024-04-27T04:15:42.931 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:15:42.930+0000 7f93fc5fd700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93f410f840 con 0x7f93f4101d90 2024-04-27T04:15:42.931 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:15:42.930+0000 7f93fa399700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f93f40fbd40 0x7f93f410eb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:15:42.931 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:15:42.930+0000 7f93fa399700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f93f40fbd40 0x7f93f410eb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:50076/0 (socket says 172.21.15.162:50076) 2024-04-27T04:15:42.931 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:15:42.930+0000 7f93fa399700 1 -- 172.21.15.162:0/4185763612 learned_addr learned my addr 172.21.15.162:0/4185763612 (peer_addr_for_me v2:172.21.15.162:0/0) 2024-04-27T04:15:42.931 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:15:42.930+0000 7f93fa399700 1 -- 172.21.15.162:0/4185763612 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f93f4101d90 msgr2=0x7f93f410f080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:15:42.931 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:15:42.930+0000 7f93fa399700 1 --2- 172.21.15.162:0/4185763612 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f93f4101d90 0x7f93f410f080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:15:42.932 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:15:42.930+0000 7f93fa399700 1 -- 172.21.15.162:0/4185763612 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f93ec005040 con 0x7f93f40fbd40 2024-04-27T04:15:42.932 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:15:42.930+0000 7f93fa399700 1 --2- 172.21.15.162:0/4185763612 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f93f40fbd40 0x7f93f410eb40 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f93ec00b3b0 tx=0x7f93ec009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:15:42.932 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:15:42.931+0000 7f93eaffd700 1 -- 172.21.15.162:0/4185763612 <== mon.0 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f93ec00e040 con 0x7f93f40fbd40 2024-04-27T04:15:42.932 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:15:42.931+0000 7f93eaffd700 1 -- 172.21.15.162:0/4185763612 <== mon.0 v2:172.21.15.162:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f93ec00bbc0 con 0x7f93f40fbd40 2024-04-27T04:15:42.932 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:15:42.931+0000 7f93fc5fd700 1 -- 172.21.15.162:0/4185763612 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f93f4117bb0 con 0x7f93f40fbd40 2024-04-27T04:15:42.932 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:15:42.931+0000 7f93eaffd700 1 -- 172.21.15.162:0/4185763612 <== mon.0 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f93ec01c3f0 con 0x7f93f40fbd40 2024-04-27T04:15:42.932 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:15:42.931+0000 7f93fc5fd700 1 -- 172.21.15.162:0/4185763612 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f93f4118090 con 0x7f93f40fbd40 2024-04-27T04:15:42.932 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:15:42.932+0000 7f93fc5fd700 1 -- 172.21.15.162:0/4185763612 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f93f40fd430 con 0x7f93f40fbd40 2024-04-27T04:15:42.932 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:15:42.932+0000 7f93eaffd700 1 -- 172.21.15.162:0/4185763612 <== mon.0 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 38) v1 ==== 92849+0+0 (secure 0 0 0) 0x7f93ec00bd30 con 0x7f93f40fbd40 2024-04-27T04:15:42.933 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:15:42.932+0000 7f93eaffd700 1 --2- 172.21.15.162:0/4185763612 >> [v2:172.21.15.162:6800/1396866108,v1:172.21.15.162:6801/1396866108] conn(0x7f93e006ed20 0x7f93e00711d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:15:42.933 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:15:42.933+0000 7f93eaffd700 1 -- 172.21.15.162:0/4185763612 <== mon.0 v2:172.21.15.162:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 6410+0+0 (secure 0 0 0) 0x7f93ec08fcb0 con 0x7f93f40fbd40 2024-04-27T04:15:42.933 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:15:42.933+0000 7f93f9b98700 1 --2- 172.21.15.162:0/4185763612 >> [v2:172.21.15.162:6800/1396866108,v1:172.21.15.162:6801/1396866108] conn(0x7f93e006ed20 0x7f93e00711d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:15:42.933 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:15:42.933+0000 7f93f9b98700 1 --2- 172.21.15.162:0/4185763612 >> [v2:172.21.15.162:6800/1396866108,v1:172.21.15.162:6801/1396866108] conn(0x7f93e006ed20 0x7f93e00711d0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f93f4065f90 tx=0x7f93e4009ce0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:15:42.936 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:15:42.937+0000 7f93eaffd700 1 -- 172.21.15.162:0/4185763612 <== mon.0 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7f93ec058910 con 0x7f93f40fbd40 2024-04-27T04:15:43.136 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:15:42 smithi162 ceph-mon[110641]: from='client.34198 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-27T04:15:43.136 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:15:42 smithi162 ceph-mon[110641]: from='client.? 172.21.15.162:0/2694661282' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-27T04:15:43.137 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:15:42 smithi162 ceph-mon[110641]: from='client.? 172.21.15.162:0/1051910817' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-27T04:15:43.148 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:15:42 smithi181 ceph-mon[81134]: from='client.34198 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-27T04:15:43.148 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:15:42 smithi181 ceph-mon[81134]: from='client.? 172.21.15.162:0/2694661282' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-27T04:15:43.148 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:15:42 smithi181 ceph-mon[81134]: from='client.? 172.21.15.162:0/1051910817' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-27T04:15:43.175 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:15:43.176+0000 7f93fc5fd700 1 -- 172.21.15.162:0/4185763612 --> [v2:172.21.15.162:6800/1396866108,v1:172.21.15.162:6801/1396866108] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f93f40fac20 con 0x7f93e006ed20 2024-04-27T04:15:43.177 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:15:43.178+0000 7f93eaffd700 1 -- 172.21.15.162:0/4185763612 <== mgr.34104 v2:172.21.15.162:6800/1396866108 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+400 (secure 0 0 0) 0x7f93f40fac20 con 0x7f93e006ed20 2024-04-27T04:15:43.177 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-04-27T04:15:43.177 INFO:teuthology.orchestra.run.smithi162.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:90a3a9fa9ceb8977f6dad9ccd36fe46018d07e2118199820523a6b35301d9c16", 2024-04-27T04:15:43.177 INFO:teuthology.orchestra.run.smithi162.stdout: "in_progress": true, 2024-04-27T04:15:43.177 INFO:teuthology.orchestra.run.smithi162.stdout: "which": "Upgrading all daemon types on all hosts", 2024-04-27T04:15:43.177 INFO:teuthology.orchestra.run.smithi162.stdout: "services_complete": [ 2024-04-27T04:15:43.177 INFO:teuthology.orchestra.run.smithi162.stdout: "crash", 2024-04-27T04:15:43.177 INFO:teuthology.orchestra.run.smithi162.stdout: "mon", 2024-04-27T04:15:43.177 INFO:teuthology.orchestra.run.smithi162.stdout: "mgr" 2024-04-27T04:15:43.177 INFO:teuthology.orchestra.run.smithi162.stdout: ], 2024-04-27T04:15:43.177 INFO:teuthology.orchestra.run.smithi162.stdout: "progress": "6/21 daemons upgraded", 2024-04-27T04:15:43.178 INFO:teuthology.orchestra.run.smithi162.stdout: "message": "Currently upgrading osd daemons", 2024-04-27T04:15:43.178 INFO:teuthology.orchestra.run.smithi162.stdout: "is_paused": false 2024-04-27T04:15:43.178 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-04-27T04:15:43.179 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:15:43.180+0000 7f93fc5fd700 1 -- 172.21.15.162:0/4185763612 >> [v2:172.21.15.162:6800/1396866108,v1:172.21.15.162:6801/1396866108] conn(0x7f93e006ed20 msgr2=0x7f93e00711d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:15:43.179 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:15:43.180+0000 7f93fc5fd700 1 --2- 172.21.15.162:0/4185763612 >> [v2:172.21.15.162:6800/1396866108,v1:172.21.15.162:6801/1396866108] conn(0x7f93e006ed20 0x7f93e00711d0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f93f4065f90 tx=0x7f93e4009ce0 comp rx=0 tx=0).stop 2024-04-27T04:15:43.179 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:15:43.180+0000 7f93fc5fd700 1 -- 172.21.15.162:0/4185763612 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f93f40fbd40 msgr2=0x7f93f410eb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:15:43.180 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:15:43.180+0000 7f93fc5fd700 1 --2- 172.21.15.162:0/4185763612 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f93f40fbd40 0x7f93f410eb40 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f93ec00b3b0 tx=0x7f93ec009e30 comp rx=0 tx=0).stop 2024-04-27T04:15:43.180 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:15:43.180+0000 7f93fc5fd700 1 -- 172.21.15.162:0/4185763612 shutdown_connections 2024-04-27T04:15:43.180 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:15:43.180+0000 7f93fc5fd700 1 --2- 172.21.15.162:0/4185763612 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f93f40fbd40 0x7f93f410eb40 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:15:43.180 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:15:43.180+0000 7f93fc5fd700 1 --2- 172.21.15.162:0/4185763612 >> [v2:172.21.15.162:6800/1396866108,v1:172.21.15.162:6801/1396866108] conn(0x7f93e006ed20 0x7f93e00711d0 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:15:43.180 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:15:43.180+0000 7f93fc5fd700 1 --2- 172.21.15.162:0/4185763612 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f93f4101d90 0x7f93f410f080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:15:43.180 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:15:43.180+0000 7f93fc5fd700 1 -- 172.21.15.162:0/4185763612 >> 172.21.15.162:0/4185763612 conn(0x7f93f4071d00 msgr2=0x7f93f4104a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:15:43.180 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:15:43.180+0000 7f93fc5fd700 1 -- 172.21.15.162:0/4185763612 shutdown_connections 2024-04-27T04:15:43.180 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:15:43.180+0000 7f93fc5fd700 1 -- 172.21.15.162:0/4185763612 wait complete. 2024-04-27T04:15:43.285 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:15:43.285+0000 7fe957498700 1 -- 172.21.15.162:0/3781490673 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe950101ce0 msgr2=0x7fe9501020b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:15:43.286 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:15:43.285+0000 7fe957498700 1 --2- 172.21.15.162:0/3781490673 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe950101ce0 0x7fe9501020b0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fe948004940 tx=0x7fe948009e30 comp rx=0 tx=0).stop 2024-04-27T04:15:43.286 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:15:43.285+0000 7fe957498700 1 -- 172.21.15.162:0/3781490673 shutdown_connections 2024-04-27T04:15:43.286 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:15:43.285+0000 7fe957498700 1 --2- 172.21.15.162:0/3781490673 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe950102680 0x7fe9500fbcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:15:43.286 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:15:43.285+0000 7fe957498700 1 --2- 172.21.15.162:0/3781490673 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe950101ce0 0x7fe9501020b0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:15:43.286 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:15:43.285+0000 7fe957498700 1 -- 172.21.15.162:0/3781490673 >> 172.21.15.162:0/3781490673 conn(0x7fe9500f7850 msgr2=0x7fe9500f9c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:15:43.286 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:15:43.285+0000 7fe957498700 1 -- 172.21.15.162:0/3781490673 shutdown_connections 2024-04-27T04:15:43.286 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:15:43.285+0000 7fe957498700 1 -- 172.21.15.162:0/3781490673 wait complete. 2024-04-27T04:15:43.286 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:15:43.285+0000 7fe957498700 1 Processor -- start 2024-04-27T04:15:43.286 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:15:43.286+0000 7fe957498700 1 -- start start 2024-04-27T04:15:43.286 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:15:43.286+0000 7fe957498700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe950101ce0 0x7fe950112a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:15:43.286 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:15:43.286+0000 7fe957498700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe950102680 0x7fe950112fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:15:43.287 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:15:43.286+0000 7fe957498700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9501136a0 con 0x7fe950102680 2024-04-27T04:15:43.287 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:15:43.286+0000 7fe957498700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe95010ca00 con 0x7fe950101ce0 2024-04-27T04:15:43.287 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:15:43.286+0000 7fe954a33700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe950102680 0x7fe950112fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:15:43.287 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:15:43.286+0000 7fe954a33700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe950102680 0x7fe950112fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:50090/0 (socket says 172.21.15.162:50090) 2024-04-27T04:15:43.287 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:15:43.286+0000 7fe954a33700 1 -- 172.21.15.162:0/2753963468 learned_addr learned my addr 172.21.15.162:0/2753963468 (peer_addr_for_me v2:172.21.15.162:0/0) 2024-04-27T04:15:43.287 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:15:43.286+0000 7fe955234700 1 --2- 172.21.15.162:0/2753963468 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe950101ce0 0x7fe950112a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:15:43.287 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:15:43.286+0000 7fe954a33700 1 -- 172.21.15.162:0/2753963468 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe950101ce0 msgr2=0x7fe950112a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:15:43.287 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:15:43.286+0000 7fe954a33700 1 --2- 172.21.15.162:0/2753963468 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe950101ce0 0x7fe950112a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:15:43.287 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:15:43.286+0000 7fe954a33700 1 -- 172.21.15.162:0/2753963468 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe948005040 con 0x7fe950102680 2024-04-27T04:15:43.287 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:15:43.286+0000 7fe954a33700 1 --2- 172.21.15.162:0/2753963468 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe950102680 0x7fe950112fc0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fe94000b790 tx=0x7fe940009220 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:15:43.287 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:15:43.287+0000 7fe945ffb700 1 -- 172.21.15.162:0/2753963468 <== mon.0 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe94000f6f0 con 0x7fe950102680 2024-04-27T04:15:43.287 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:15:43.287+0000 7fe945ffb700 1 -- 172.21.15.162:0/2753963468 <== mon.0 v2:172.21.15.162:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fe94000fd20 con 0x7fe950102680 2024-04-27T04:15:43.287 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:15:43.287+0000 7fe945ffb700 1 -- 172.21.15.162:0/2753963468 <== mon.0 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe9400103a0 con 0x7fe950102680 2024-04-27T04:15:43.287 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:15:43.287+0000 7fe957498700 1 -- 172.21.15.162:0/2753963468 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe95010cce0 con 0x7fe950102680 2024-04-27T04:15:43.288 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:15:43.287+0000 7fe957498700 1 -- 172.21.15.162:0/2753963468 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe95010d250 con 0x7fe950102680 2024-04-27T04:15:43.288 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:15:43.288+0000 7fe957498700 1 -- 172.21.15.162:0/2753963468 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe9500fd430 con 0x7fe950102680 2024-04-27T04:15:43.289 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:15:43.288+0000 7fe945ffb700 1 -- 172.21.15.162:0/2753963468 <== mon.0 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 38) v1 ==== 92849+0+0 (secure 0 0 0) 0x7fe940011400 con 0x7fe950102680 2024-04-27T04:15:43.289 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:15:43.288+0000 7fe945ffb700 1 --2- 172.21.15.162:0/2753963468 >> [v2:172.21.15.162:6800/1396866108,v1:172.21.15.162:6801/1396866108] conn(0x7fe93c06ed20 0x7fe93c0711d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:15:43.289 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:15:43.289+0000 7fe945ffb700 1 -- 172.21.15.162:0/2753963468 <== mon.0 v2:172.21.15.162:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 6410+0+0 (secure 0 0 0) 0x7fe94008e7a0 con 0x7fe950102680 2024-04-27T04:15:43.289 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:15:43.289+0000 7fe955234700 1 --2- 172.21.15.162:0/2753963468 >> [v2:172.21.15.162:6800/1396866108,v1:172.21.15.162:6801/1396866108] conn(0x7fe93c06ed20 0x7fe93c0711d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:15:43.289 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:15:43.289+0000 7fe955234700 1 --2- 172.21.15.162:0/2753963468 >> [v2:172.21.15.162:6800/1396866108,v1:172.21.15.162:6801/1396866108] conn(0x7fe93c06ed20 0x7fe93c0711d0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fe948004f90 tx=0x7fe948009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:15:43.292 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:15:43.293+0000 7fe945ffb700 1 -- 172.21.15.162:0/2753963468 <== mon.0 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7fe94005b040 con 0x7fe950102680 2024-04-27T04:15:43.586 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:15:43.586+0000 7fe957498700 1 -- 172.21.15.162:0/2753963468 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7fe95005f5c0 con 0x7fe950102680 2024-04-27T04:15:43.586 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:15:43.587+0000 7fe945ffb700 1 -- 172.21.15.162:0/2753963468 <== mon.0 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+1271 (secure 0 0 0) 0x7fe94005abd0 con 0x7fe950102680 2024-04-27T04:15:43.586 INFO:teuthology.orchestra.run.smithi162.stdout:HEALTH_WARN 1 osds down; Degraded data redundancy: 41/231 objects degraded (17.749%), 18 pgs degraded 2024-04-27T04:15:43.586 INFO:teuthology.orchestra.run.smithi162.stdout:[WRN] OSD_DOWN: 1 osds down 2024-04-27T04:15:43.586 INFO:teuthology.orchestra.run.smithi162.stdout: osd.0 (root=default,host=smithi162) is down 2024-04-27T04:15:43.586 INFO:teuthology.orchestra.run.smithi162.stdout:[WRN] PG_DEGRADED: Degraded data redundancy: 41/231 objects degraded (17.749%), 18 pgs degraded 2024-04-27T04:15:43.586 INFO:teuthology.orchestra.run.smithi162.stdout: pg 1.0 is active+undersized+degraded, acting [3,1] 2024-04-27T04:15:43.586 INFO:teuthology.orchestra.run.smithi162.stdout: pg 2.0 is active+undersized+degraded, acting [3,1] 2024-04-27T04:15:43.586 INFO:teuthology.orchestra.run.smithi162.stdout: pg 2.1 is active+undersized+degraded, acting [2,1] 2024-04-27T04:15:43.587 INFO:teuthology.orchestra.run.smithi162.stdout: pg 2.2 is active+undersized+degraded, acting [5,1] 2024-04-27T04:15:43.587 INFO:teuthology.orchestra.run.smithi162.stdout: pg 2.4 is active+undersized+degraded, acting [1,4] 2024-04-27T04:15:43.587 INFO:teuthology.orchestra.run.smithi162.stdout: pg 2.5 is active+undersized+degraded, acting [3,4] 2024-04-27T04:15:43.587 INFO:teuthology.orchestra.run.smithi162.stdout: pg 2.8 is active+undersized+degraded, acting [3,5] 2024-04-27T04:15:43.587 INFO:teuthology.orchestra.run.smithi162.stdout: pg 2.9 is active+undersized+degraded, acting [1,4] 2024-04-27T04:15:43.587 INFO:teuthology.orchestra.run.smithi162.stdout: pg 2.c is active+undersized+degraded, acting [2,3] 2024-04-27T04:15:43.587 INFO:teuthology.orchestra.run.smithi162.stdout: pg 2.e is active+undersized+degraded, acting [2,3] 2024-04-27T04:15:43.587 INFO:teuthology.orchestra.run.smithi162.stdout: pg 2.f is active+undersized+degraded, acting [4,5] 2024-04-27T04:15:43.587 INFO:teuthology.orchestra.run.smithi162.stdout: pg 2.10 is active+undersized+degraded, acting [2,1] 2024-04-27T04:15:43.587 INFO:teuthology.orchestra.run.smithi162.stdout: pg 2.13 is active+undersized+degraded, acting [4,2] 2024-04-27T04:15:43.587 INFO:teuthology.orchestra.run.smithi162.stdout: pg 2.15 is active+undersized+degraded, acting [1,3] 2024-04-27T04:15:43.587 INFO:teuthology.orchestra.run.smithi162.stdout: pg 2.19 is active+undersized+degraded, acting [4,2] 2024-04-27T04:15:43.587 INFO:teuthology.orchestra.run.smithi162.stdout: pg 2.1d is active+undersized+degraded, acting [3,5] 2024-04-27T04:15:43.588 INFO:teuthology.orchestra.run.smithi162.stdout: pg 2.1e is active+undersized+degraded, acting [2,5] 2024-04-27T04:15:43.588 INFO:teuthology.orchestra.run.smithi162.stdout: pg 2.1f is active+undersized+degraded, acting [3,4] 2024-04-27T04:15:43.588 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:15:43.589+0000 7fe957498700 1 -- 172.21.15.162:0/2753963468 >> [v2:172.21.15.162:6800/1396866108,v1:172.21.15.162:6801/1396866108] conn(0x7fe93c06ed20 msgr2=0x7fe93c0711d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:15:43.588 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:15:43.589+0000 7fe957498700 1 --2- 172.21.15.162:0/2753963468 >> [v2:172.21.15.162:6800/1396866108,v1:172.21.15.162:6801/1396866108] conn(0x7fe93c06ed20 0x7fe93c0711d0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fe948004f90 tx=0x7fe948009e30 comp rx=0 tx=0).stop 2024-04-27T04:15:43.588 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:15:43.589+0000 7fe957498700 1 -- 172.21.15.162:0/2753963468 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe950102680 msgr2=0x7fe950112fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:15:43.589 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:15:43.589+0000 7fe957498700 1 --2- 172.21.15.162:0/2753963468 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe950102680 0x7fe950112fc0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fe94000b790 tx=0x7fe940009220 comp rx=0 tx=0).stop 2024-04-27T04:15:43.589 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:15:43.589+0000 7fe957498700 1 -- 172.21.15.162:0/2753963468 shutdown_connections 2024-04-27T04:15:43.589 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:15:43.589+0000 7fe957498700 1 --2- 172.21.15.162:0/2753963468 >> [v2:172.21.15.162:6800/1396866108,v1:172.21.15.162:6801/1396866108] conn(0x7fe93c06ed20 0x7fe93c0711d0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:15:43.589 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:15:43.589+0000 7fe957498700 1 --2- 172.21.15.162:0/2753963468 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe950101ce0 0x7fe950112a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:15:43.589 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:15:43.589+0000 7fe957498700 1 --2- 172.21.15.162:0/2753963468 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe950102680 0x7fe950112fc0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:15:43.589 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:15:43.589+0000 7fe957498700 1 -- 172.21.15.162:0/2753963468 >> 172.21.15.162:0/2753963468 conn(0x7fe9500f7850 msgr2=0x7fe9500f8800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:15:43.589 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:15:43.589+0000 7fe957498700 1 -- 172.21.15.162:0/2753963468 shutdown_connections 2024-04-27T04:15:43.589 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:15:43.589+0000 7fe957498700 1 -- 172.21.15.162:0/2753963468 wait complete. 2024-04-27T04:15:44.172 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:15:43 smithi181 ceph-mon[81134]: from='client.34210 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-27T04:15:44.172 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:15:43 smithi181 ceph-mon[81134]: from='client.? 172.21.15.162:0/2753963468' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-27T04:15:44.172 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:15:43 smithi181 ceph-mon[81134]: pgmap v48: 65 pgs: 18 active+undersized+degraded, 16 active+undersized, 31 active+clean; 208 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 1.1 KiB/s rd, 1 op/s; 41/231 objects degraded (17.749%) 2024-04-27T04:15:44.262 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:15:43 smithi162 ceph-mon[110641]: from='client.34210 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-27T04:15:44.262 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:15:43 smithi162 ceph-mon[110641]: from='client.? 172.21.15.162:0/2753963468' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-27T04:15:44.262 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:15:43 smithi162 ceph-mon[110641]: pgmap v48: 65 pgs: 18 active+undersized+degraded, 16 active+undersized, 31 active+clean; 208 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 1.1 KiB/s rd, 1 op/s; 41/231 objects degraded (17.749%) 2024-04-27T04:15:46.136 INFO:journalctl@ceph.osd.0.smithi162.stdout:Apr 27 04:15:45 smithi162 ceph-7f9750d6-044a-11ef-bc93-c7b262605968-osd-0[121588]: 2024-04-27T04:15:45.969+0000 7f199620f7c0 -1 osd.0 49 log_to_monitors true 2024-04-27T04:15:47.136 INFO:journalctl@ceph.osd.0.smithi162.stdout:Apr 27 04:15:46 smithi162 ceph-7f9750d6-044a-11ef-bc93-c7b262605968-osd-0[121588]: 2024-04-27T04:15:46.696+0000 7f1987667700 -1 osd.0 49 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-27T04:15:47.137 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:15:46 smithi162 ceph-mon[110641]: pgmap v49: 65 pgs: 18 active+undersized+degraded, 16 active+undersized, 31 active+clean; 208 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 934 B/s rd, 1 op/s; 41/231 objects degraded (17.749%) 2024-04-27T04:15:47.137 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:15:46 smithi162 ceph-mon[110641]: from='osd.0 [v2:172.21.15.162:6802/1840117633,v1:172.21.15.162:6803/1840117633]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-04-27T04:15:47.137 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:15:46 smithi162 ceph-mon[110641]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' 2024-04-27T04:15:47.137 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:15:46 smithi162 ceph-mon[110641]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' 2024-04-27T04:15:47.172 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:15:46 smithi181 ceph-mon[81134]: pgmap v49: 65 pgs: 18 active+undersized+degraded, 16 active+undersized, 31 active+clean; 208 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 934 B/s rd, 1 op/s; 41/231 objects degraded (17.749%) 2024-04-27T04:15:47.172 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:15:46 smithi181 ceph-mon[81134]: from='osd.0 [v2:172.21.15.162:6802/1840117633,v1:172.21.15.162:6803/1840117633]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-04-27T04:15:47.172 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:15:46 smithi181 ceph-mon[81134]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' 2024-04-27T04:15:47.172 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:15:46 smithi181 ceph-mon[81134]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' 2024-04-27T04:15:47.945 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:15:47 smithi162 ceph-mon[110641]: from='osd.0 [v2:172.21.15.162:6802/1840117633,v1:172.21.15.162:6803/1840117633]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-04-27T04:15:47.945 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:15:47 smithi162 ceph-mon[110641]: osdmap e52: 6 total, 5 up, 6 in 2024-04-27T04:15:47.945 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:15:47 smithi162 ceph-mon[110641]: from='osd.0 [v2:172.21.15.162:6802/1840117633,v1:172.21.15.162:6803/1840117633]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi162", "root=default"]}]: dispatch 2024-04-27T04:15:47.945 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:15:47 smithi162 ceph-mon[110641]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' 2024-04-27T04:15:47.945 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:15:47 smithi162 ceph-mon[110641]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' 2024-04-27T04:15:47.946 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:15:47 smithi162 ceph-mon[110641]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' 2024-04-27T04:15:47.946 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:15:47 smithi162 ceph-mon[110641]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-27T04:15:48.172 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:15:47 smithi181 ceph-mon[81134]: from='osd.0 [v2:172.21.15.162:6802/1840117633,v1:172.21.15.162:6803/1840117633]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-04-27T04:15:48.172 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:15:47 smithi181 ceph-mon[81134]: osdmap e52: 6 total, 5 up, 6 in 2024-04-27T04:15:48.172 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:15:47 smithi181 ceph-mon[81134]: from='osd.0 [v2:172.21.15.162:6802/1840117633,v1:172.21.15.162:6803/1840117633]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi162", "root=default"]}]: dispatch 2024-04-27T04:15:48.172 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:15:47 smithi181 ceph-mon[81134]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' 2024-04-27T04:15:48.172 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:15:47 smithi181 ceph-mon[81134]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' 2024-04-27T04:15:48.172 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:15:47 smithi181 ceph-mon[81134]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' 2024-04-27T04:15:48.172 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:15:47 smithi181 ceph-mon[81134]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-27T04:15:48.954 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:15:48 smithi162 ceph-mon[110641]: pgmap v51: 65 pgs: 18 active+undersized+degraded, 16 active+undersized, 31 active+clean; 208 MiB data, 2.1 GiB used, 534 GiB / 536 GiB avail; 921 B/s rd, 1 op/s; 41/231 objects degraded (17.749%) 2024-04-27T04:15:48.954 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:15:48 smithi162 ceph-mon[110641]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-04-27T04:15:48.954 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:15:48 smithi162 ceph-mon[110641]: osd.0 [v2:172.21.15.162:6802/1840117633,v1:172.21.15.162:6803/1840117633] boot 2024-04-27T04:15:48.954 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:15:48 smithi162 ceph-mon[110641]: osdmap e53: 6 total, 6 up, 6 in 2024-04-27T04:15:48.954 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:15:48 smithi162 ceph-mon[110641]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-27T04:15:48.954 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:15:48 smithi162 ceph-mon[110641]: osdmap e54: 6 total, 6 up, 6 in 2024-04-27T04:15:49.172 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:15:48 smithi181 ceph-mon[81134]: pgmap v51: 65 pgs: 18 active+undersized+degraded, 16 active+undersized, 31 active+clean; 208 MiB data, 2.1 GiB used, 534 GiB / 536 GiB avail; 921 B/s rd, 1 op/s; 41/231 objects degraded (17.749%) 2024-04-27T04:15:49.172 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:15:48 smithi181 ceph-mon[81134]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-04-27T04:15:49.172 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:15:48 smithi181 ceph-mon[81134]: osd.0 [v2:172.21.15.162:6802/1840117633,v1:172.21.15.162:6803/1840117633] boot 2024-04-27T04:15:49.172 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:15:48 smithi181 ceph-mon[81134]: osdmap e53: 6 total, 6 up, 6 in 2024-04-27T04:15:49.172 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:15:48 smithi181 ceph-mon[81134]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-27T04:15:49.172 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:15:48 smithi181 ceph-mon[81134]: osdmap e54: 6 total, 6 up, 6 in 2024-04-27T04:15:50.030 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:15:49 smithi162 ceph-mon[110641]: Health check update: Degraded data redundancy: 7/231 objects degraded (3.030%), 3 pgs degraded (PG_DEGRADED) 2024-04-27T04:15:50.030 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:15:49 smithi162 ceph-mon[110641]: mgrmap e39: smithi162.oennfn(active, since 92s), standbys: smithi181.vtwuhz 2024-04-27T04:15:50.172 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:15:49 smithi181 ceph-mon[81134]: Health check update: Degraded data redundancy: 7/231 objects degraded (3.030%), 3 pgs degraded (PG_DEGRADED) 2024-04-27T04:15:50.172 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:15:49 smithi181 ceph-mon[81134]: mgrmap e39: smithi162.oennfn(active, since 92s), standbys: smithi181.vtwuhz 2024-04-27T04:15:51.136 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:15:50 smithi162 ceph-mon[110641]: pgmap v54: 65 pgs: 25 peering, 3 active+undersized+degraded, 6 active+undersized, 31 active+clean; 208 MiB data, 2.1 GiB used, 534 GiB / 536 GiB avail; 853 B/s rd, 170 B/s wr, 1 op/s; 7/231 objects degraded (3.030%) 2024-04-27T04:15:51.172 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:15:50 smithi181 ceph-mon[81134]: pgmap v54: 65 pgs: 25 peering, 3 active+undersized+degraded, 6 active+undersized, 31 active+clean; 208 MiB data, 2.1 GiB used, 534 GiB / 536 GiB avail; 853 B/s rd, 170 B/s wr, 1 op/s; 7/231 objects degraded (3.030%) 2024-04-27T04:15:52.137 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:15:51 smithi162 ceph-mon[110641]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' 2024-04-27T04:15:52.137 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:15:51 smithi162 ceph-mon[110641]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' 2024-04-27T04:15:52.137 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:15:51 smithi162 ceph-mon[110641]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T04:15:52.137 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:15:51 smithi162 ceph-mon[110641]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T04:15:52.137 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:15:51 smithi162 ceph-mon[110641]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' 2024-04-27T04:15:52.137 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:15:51 smithi162 ceph-mon[110641]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T04:15:52.137 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:15:51 smithi162 ceph-mon[110641]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "versions"}]: dispatch 2024-04-27T04:15:52.137 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:15:51 smithi162 ceph-mon[110641]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "versions"}]: dispatch 2024-04-27T04:15:52.137 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:15:51 smithi162 ceph-mon[110641]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "versions"}]: dispatch 2024-04-27T04:15:52.137 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:15:51 smithi162 ceph-mon[110641]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "osd ok-to-stop", "ids": ["1"], "max": 16}]: dispatch 2024-04-27T04:15:52.138 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:15:51 smithi162 ceph-mon[110641]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["1"], "max": 16}]: dispatch 2024-04-27T04:15:52.138 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:15:51 smithi162 ceph-mon[110641]: Upgrade: unsafe to stop osd(s) at this time (15 PGs are or would become offline) 2024-04-27T04:15:52.138 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:15:51 smithi162 ceph-mon[110641]: pgmap v55: 65 pgs: 25 peering, 3 active+undersized+degraded, 6 active+undersized, 31 active+clean; 208 MiB data, 2.1 GiB used, 534 GiB / 536 GiB avail; 853 B/s rd, 170 B/s wr, 1 op/s; 7/231 objects degraded (3.030%) 2024-04-27T04:15:52.172 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:15:51 smithi181 ceph-mon[81134]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' 2024-04-27T04:15:52.172 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:15:51 smithi181 ceph-mon[81134]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' 2024-04-27T04:15:52.172 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:15:51 smithi181 ceph-mon[81134]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T04:15:52.172 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:15:51 smithi181 ceph-mon[81134]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T04:15:52.172 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:15:51 smithi181 ceph-mon[81134]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' 2024-04-27T04:15:52.172 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:15:51 smithi181 ceph-mon[81134]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T04:15:52.172 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:15:51 smithi181 ceph-mon[81134]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "versions"}]: dispatch 2024-04-27T04:15:52.173 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:15:51 smithi181 ceph-mon[81134]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "versions"}]: dispatch 2024-04-27T04:15:52.173 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:15:51 smithi181 ceph-mon[81134]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "versions"}]: dispatch 2024-04-27T04:15:52.173 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:15:51 smithi181 ceph-mon[81134]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "osd ok-to-stop", "ids": ["1"], "max": 16}]: dispatch 2024-04-27T04:15:52.173 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:15:51 smithi181 ceph-mon[81134]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["1"], "max": 16}]: dispatch 2024-04-27T04:15:52.173 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:15:51 smithi181 ceph-mon[81134]: Upgrade: unsafe to stop osd(s) at this time (15 PGs are or would become offline) 2024-04-27T04:15:52.173 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:15:51 smithi181 ceph-mon[81134]: pgmap v55: 65 pgs: 25 peering, 3 active+undersized+degraded, 6 active+undersized, 31 active+clean; 208 MiB data, 2.1 GiB used, 534 GiB / 536 GiB avail; 853 B/s rd, 170 B/s wr, 1 op/s; 7/231 objects degraded (3.030%) 2024-04-27T04:15:54.136 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:15:53 smithi162 ceph-mon[110641]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 7/231 objects degraded (3.030%), 3 pgs degraded) 2024-04-27T04:15:54.137 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:15:53 smithi162 ceph-mon[110641]: Cluster is now healthy 2024-04-27T04:15:54.172 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:15:53 smithi181 ceph-mon[81134]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 7/231 objects degraded (3.030%), 3 pgs degraded) 2024-04-27T04:15:54.172 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:15:53 smithi181 ceph-mon[81134]: Cluster is now healthy 2024-04-27T04:15:55.136 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:15:54 smithi162 ceph-mon[110641]: pgmap v56: 65 pgs: 65 active+clean; 208 MiB data, 2.1 GiB used, 534 GiB / 536 GiB avail; 1.4 KiB/s rd, 442 B/s wr, 3 op/s 2024-04-27T04:15:55.172 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:15:54 smithi181 ceph-mon[81134]: pgmap v56: 65 pgs: 65 active+clean; 208 MiB data, 2.1 GiB used, 534 GiB / 536 GiB avail; 1.4 KiB/s rd, 442 B/s wr, 3 op/s 2024-04-27T04:15:57.136 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:15:56 smithi162 ceph-mon[110641]: pgmap v57: 65 pgs: 65 active+clean; 208 MiB data, 2.1 GiB used, 534 GiB / 536 GiB avail; 1.2 KiB/s rd, 383 B/s wr, 2 op/s 2024-04-27T04:15:57.171 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:15:56 smithi181 ceph-mon[81134]: pgmap v57: 65 pgs: 65 active+clean; 208 MiB data, 2.1 GiB used, 534 GiB / 536 GiB avail; 1.2 KiB/s rd, 383 B/s wr, 2 op/s 2024-04-27T04:15:59.136 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:15:58 smithi162 ceph-mon[110641]: pgmap v58: 65 pgs: 65 active+clean; 208 MiB data, 2.1 GiB used, 534 GiB / 536 GiB avail; 1.0 KiB/s rd, 309 B/s wr, 2 op/s 2024-04-27T04:15:59.172 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:15:58 smithi181 ceph-mon[81134]: pgmap v58: 65 pgs: 65 active+clean; 208 MiB data, 2.1 GiB used, 534 GiB / 536 GiB avail; 1.0 KiB/s rd, 309 B/s wr, 2 op/s 2024-04-27T04:16:01.136 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:16:00 smithi162 ceph-mon[110641]: pgmap v59: 65 pgs: 65 active+clean; 208 MiB data, 2.1 GiB used, 534 GiB / 536 GiB avail; 936 B/s rd, 187 B/s wr, 2 op/s 2024-04-27T04:16:01.172 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:16:00 smithi181 ceph-mon[81134]: pgmap v59: 65 pgs: 65 active+clean; 208 MiB data, 2.1 GiB used, 534 GiB / 536 GiB avail; 936 B/s rd, 187 B/s wr, 2 op/s 2024-04-27T04:16:03.136 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:16:02 smithi162 ceph-mon[110641]: pgmap v60: 65 pgs: 65 active+clean; 208 MiB data, 2.1 GiB used, 534 GiB / 536 GiB avail; 853 B/s rd, 170 B/s wr, 1 op/s 2024-04-27T04:16:03.137 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:16:02 smithi162 ceph-mon[110641]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' 2024-04-27T04:16:03.137 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:16:02 smithi162 ceph-mon[110641]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-27T04:16:03.137 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:16:02 smithi162 ceph-mon[110641]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' 2024-04-27T04:16:03.172 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:16:02 smithi181 ceph-mon[81134]: pgmap v60: 65 pgs: 65 active+clean; 208 MiB data, 2.1 GiB used, 534 GiB / 536 GiB avail; 853 B/s rd, 170 B/s wr, 1 op/s 2024-04-27T04:16:03.172 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:16:02 smithi181 ceph-mon[81134]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' 2024-04-27T04:16:03.172 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:16:02 smithi181 ceph-mon[81134]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-27T04:16:03.173 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:16:02 smithi181 ceph-mon[81134]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' 2024-04-27T04:16:05.136 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:16:04 smithi162 ceph-mon[110641]: pgmap v61: 65 pgs: 65 active+clean; 208 MiB data, 2.1 GiB used, 534 GiB / 536 GiB avail; 1.2 KiB/s rd, 170 B/s wr, 2 op/s 2024-04-27T04:16:05.157 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:16:04 smithi181 ceph-mon[81134]: pgmap v61: 65 pgs: 65 active+clean; 208 MiB data, 2.1 GiB used, 534 GiB / 536 GiB avail; 1.2 KiB/s rd, 170 B/s wr, 2 op/s 2024-04-27T04:16:06.826 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:16:06 smithi162 ceph-mon[110641]: pgmap v62: 65 pgs: 65 active+clean; 208 MiB data, 2.1 GiB used, 534 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:16:06.826 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:16:06 smithi162 ceph-mon[110641]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "osd ok-to-stop", "ids": ["1"], "max": 16}]: dispatch 2024-04-27T04:16:07.172 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:16:06 smithi181 ceph-mon[81134]: pgmap v62: 65 pgs: 65 active+clean; 208 MiB data, 2.1 GiB used, 534 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:16:07.172 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:16:06 smithi181 ceph-mon[81134]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "osd ok-to-stop", "ids": ["1"], "max": 16}]: dispatch 2024-04-27T04:16:07.886 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:16:07 smithi162 ceph-mon[110641]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["1"], "max": 16}]: dispatch 2024-04-27T04:16:07.886 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:16:07 smithi162 ceph-mon[110641]: Upgrade: osd.1 is safe to restart 2024-04-27T04:16:07.887 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:16:07 smithi162 ceph-mon[110641]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' 2024-04-27T04:16:07.887 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:16:07 smithi162 ceph-mon[110641]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-04-27T04:16:07.887 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:16:07 smithi162 ceph-mon[110641]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T04:16:08.172 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:16:07 smithi181 ceph-mon[81134]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["1"], "max": 16}]: dispatch 2024-04-27T04:16:08.172 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:16:07 smithi181 ceph-mon[81134]: Upgrade: osd.1 is safe to restart 2024-04-27T04:16:08.172 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:16:07 smithi181 ceph-mon[81134]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' 2024-04-27T04:16:08.172 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:16:07 smithi181 ceph-mon[81134]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-04-27T04:16:08.172 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:16:07 smithi181 ceph-mon[81134]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T04:16:09.084 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:16:08 smithi162 ceph-mon[110641]: Upgrade: Updating osd.1 2024-04-27T04:16:09.085 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:16:08 smithi162 ceph-mon[110641]: Deploying daemon osd.1 on smithi162 2024-04-27T04:16:09.085 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:16:08 smithi162 ceph-mon[110641]: pgmap v63: 65 pgs: 65 active+clean; 208 MiB data, 2.1 GiB used, 534 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:16:09.172 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:16:08 smithi181 ceph-mon[81134]: Upgrade: Updating osd.1 2024-04-27T04:16:09.172 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:16:08 smithi181 ceph-mon[81134]: Deploying daemon osd.1 on smithi162 2024-04-27T04:16:09.172 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:16:08 smithi181 ceph-mon[81134]: pgmap v63: 65 pgs: 65 active+clean; 208 MiB data, 2.1 GiB used, 534 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:16:10.954 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:16:10 smithi162 ceph-mon[110641]: pgmap v64: 65 pgs: 65 active+clean; 208 MiB data, 2.1 GiB used, 534 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:16:10.954 INFO:journalctl@ceph.osd.1.smithi162.stdout:Apr 27 04:16:10 smithi162 systemd[1]: Stopping Ceph osd.1 for 7f9750d6-044a-11ef-bc93-c7b262605968... 2024-04-27T04:16:11.172 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:16:10 smithi181 ceph-mon[81134]: pgmap v64: 65 pgs: 65 active+clean; 208 MiB data, 2.1 GiB used, 534 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:16:11.386 INFO:journalctl@ceph.osd.1.smithi162.stdout:Apr 27 04:16:10 smithi162 ceph-7f9750d6-044a-11ef-bc93-c7b262605968-osd-1[52837]: 2024-04-27T04:16:10.955+0000 7fbcba051700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.1 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-27T04:16:11.387 INFO:journalctl@ceph.osd.1.smithi162.stdout:Apr 27 04:16:10 smithi162 ceph-7f9750d6-044a-11ef-bc93-c7b262605968-osd-1[52837]: 2024-04-27T04:16:10.955+0000 7fbcba051700 -1 osd.1 54 *** Got signal Terminated *** 2024-04-27T04:16:11.387 INFO:journalctl@ceph.osd.1.smithi162.stdout:Apr 27 04:16:10 smithi162 ceph-7f9750d6-044a-11ef-bc93-c7b262605968-osd-1[52837]: 2024-04-27T04:16:10.955+0000 7fbcba051700 -1 osd.1 54 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-27T04:16:12.119 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:16:11 smithi162 ceph-mon[110641]: osd.1 marked itself down and dead 2024-04-27T04:16:12.172 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:16:11 smithi181 ceph-mon[81134]: osd.1 marked itself down and dead 2024-04-27T04:16:12.379 INFO:journalctl@ceph.osd.1.smithi162.stdout:Apr 27 04:16:12 smithi162 podman[126486]: 2024-04-27 04:16:12.120578175 +0000 UTC m=+1.291010069 container died aefb464c46e851c6a3ecca37ac0f877300f87f2fdc17c7065aaeb4ec1b0af466 (image=quay.ceph.io/ceph-ci/ceph@sha256:d31f278ece6dbd17d1fa5e598f509017f54b76fafe1c4416391fcc51b9718ee9, name=ceph-7f9750d6-044a-11ef-bc93-c7b262605968-osd-1, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240424, RELEASE=quincy-2be3efa, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b) 2024-04-27T04:16:12.636 INFO:journalctl@ceph.osd.1.smithi162.stdout:Apr 27 04:16:12 smithi162 podman[126486]: 2024-04-27 04:16:12.555725225 +0000 UTC m=+1.726157119 container cleanup aefb464c46e851c6a3ecca37ac0f877300f87f2fdc17c7065aaeb4ec1b0af466 (image=quay.ceph.io/ceph-ci/ceph@sha256:d31f278ece6dbd17d1fa5e598f509017f54b76fafe1c4416391fcc51b9718ee9, name=ceph-7f9750d6-044a-11ef-bc93-c7b262605968-osd-1, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, ceph=True, org.label-schema.schema-version=1.0, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240424, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=quincy-2be3efa, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.vendor=CentOS) 2024-04-27T04:16:12.637 INFO:journalctl@ceph.osd.1.smithi162.stdout:Apr 27 04:16:12 smithi162 bash[126486]: ceph-7f9750d6-044a-11ef-bc93-c7b262605968-osd-1 2024-04-27T04:16:13.136 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:16:12 smithi162 ceph-mon[110641]: pgmap v65: 65 pgs: 65 active+clean; 208 MiB data, 2.1 GiB used, 534 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:16:13.136 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:16:12 smithi162 ceph-mon[110641]: Health check failed: 1 osds down (OSD_DOWN) 2024-04-27T04:16:13.137 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:16:12 smithi162 ceph-mon[110641]: osdmap e55: 6 total, 5 up, 6 in 2024-04-27T04:16:13.137 INFO:journalctl@ceph.osd.1.smithi162.stdout:Apr 27 04:16:12 smithi162 podman[126503]: 2024-04-27 04:16:12.757381558 +0000 UTC m=+0.636307908 container remove aefb464c46e851c6a3ecca37ac0f877300f87f2fdc17c7065aaeb4ec1b0af466 (image=quay.ceph.io/ceph-ci/ceph@sha256:d31f278ece6dbd17d1fa5e598f509017f54b76fafe1c4416391fcc51b9718ee9, name=ceph-7f9750d6-044a-11ef-bc93-c7b262605968-osd-1, GIT_CLEAN=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, ceph=True, org.label-schema.build-date=20240424, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, RELEASE=quincy-2be3efa, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-27T04:16:13.172 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:16:12 smithi181 ceph-mon[81134]: pgmap v65: 65 pgs: 65 active+clean; 208 MiB data, 2.1 GiB used, 534 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:16:13.172 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:16:12 smithi181 ceph-mon[81134]: Health check failed: 1 osds down (OSD_DOWN) 2024-04-27T04:16:13.172 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:16:12 smithi181 ceph-mon[81134]: osdmap e55: 6 total, 5 up, 6 in 2024-04-27T04:16:13.517 INFO:journalctl@ceph.osd.1.smithi162.stdout:Apr 27 04:16:13 smithi162 podman[126595]: 2024-04-27 04:16:13.092786614 +0000 UTC m=+0.022824274 image pull 67e414ff4bc41092cb31c84c98a9563a32813d02583028679c46b40362afeb3e quay.ceph.io/ceph-ci/ceph@sha256:90a3a9fa9ceb8977f6dad9ccd36fe46018d07e2118199820523a6b35301d9c16 2024-04-27T04:16:13.517 INFO:journalctl@ceph.osd.1.smithi162.stdout:Apr 27 04:16:13 smithi162 podman[126595]: 2024-04-27 04:16:13.216735441 +0000 UTC m=+0.146773088 container create 0b051b64e2afdd95189353b317ff61355abd70d8bc2a6eb104a74d4f86e4898d (image=quay.ceph.io/ceph-ci/ceph@sha256:90a3a9fa9ceb8977f6dad9ccd36fe46018d07e2118199820523a6b35301d9c16, name=ceph-7f9750d6-044a-11ef-bc93-c7b262605968-osd-1-deactivate, org.label-schema.build-date=20240424, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, RELEASE=main-4e213a5, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , ceph=True) 2024-04-27T04:16:13.517 INFO:journalctl@ceph.osd.1.smithi162.stdout:Apr 27 04:16:13 smithi162 podman[126595]: 2024-04-27 04:16:13.419570705 +0000 UTC m=+0.349608356 container init 0b051b64e2afdd95189353b317ff61355abd70d8bc2a6eb104a74d4f86e4898d (image=quay.ceph.io/ceph-ci/ceph@sha256:90a3a9fa9ceb8977f6dad9ccd36fe46018d07e2118199820523a6b35301d9c16, name=ceph-7f9750d6-044a-11ef-bc93-c7b262605968-osd-1-deactivate, org.label-schema.schema-version=1.0, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD, org.label-schema.build-date=20240424, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , RELEASE=main-4e213a5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-27T04:16:13.517 INFO:journalctl@ceph.osd.1.smithi162.stdout:Apr 27 04:16:13 smithi162 podman[126595]: 2024-04-27 04:16:13.42416033 +0000 UTC m=+0.354197977 container start 0b051b64e2afdd95189353b317ff61355abd70d8bc2a6eb104a74d4f86e4898d (image=quay.ceph.io/ceph-ci/ceph@sha256:90a3a9fa9ceb8977f6dad9ccd36fe46018d07e2118199820523a6b35301d9c16, name=ceph-7f9750d6-044a-11ef-bc93-c7b262605968-osd-1-deactivate, GIT_BRANCH=HEAD, org.label-schema.build-date=20240424, GIT_CLEAN=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, RELEASE=main-4e213a5, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-27T04:16:13.517 INFO:journalctl@ceph.osd.1.smithi162.stdout:Apr 27 04:16:13 smithi162 podman[126595]: 2024-04-27 04:16:13.519339049 +0000 UTC m=+0.449376707 container attach 0b051b64e2afdd95189353b317ff61355abd70d8bc2a6eb104a74d4f86e4898d (image=quay.ceph.io/ceph-ci/ceph@sha256:90a3a9fa9ceb8977f6dad9ccd36fe46018d07e2118199820523a6b35301d9c16, name=ceph-7f9750d6-044a-11ef-bc93-c7b262605968-osd-1-deactivate, RELEASE=main-4e213a5, org.label-schema.build-date=20240424, GIT_CLEAN=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux ) 2024-04-27T04:16:13.699 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:16:13.699+0000 7f89cc553700 1 -- 172.21.15.162:0/27301717 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f89c4065640 msgr2=0x7f89c4065ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:16:13.699 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:16:13.699+0000 7f89cc553700 1 --2- 172.21.15.162:0/27301717 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f89c4065640 0x7f89c4065ab0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f89b4004940 tx=0x7f89b4009e30 comp rx=0 tx=0).stop 2024-04-27T04:16:13.699 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:16:13.699+0000 7f89cc553700 1 -- 172.21.15.162:0/27301717 shutdown_connections 2024-04-27T04:16:13.700 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:16:13.699+0000 7f89cc553700 1 --2- 172.21.15.162:0/27301717 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f89c4065640 0x7f89c4065ab0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:16:13.700 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:16:13.699+0000 7f89cc553700 1 --2- 172.21.15.162:0/27301717 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f89c4102e50 0x7f89c4103220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:16:13.700 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:16:13.699+0000 7f89cc553700 1 -- 172.21.15.162:0/27301717 >> 172.21.15.162:0/27301717 conn(0x7f89c4071d00 msgr2=0x7f89c4072100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:16:13.700 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:16:13.699+0000 7f89cc553700 1 -- 172.21.15.162:0/27301717 shutdown_connections 2024-04-27T04:16:13.700 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:16:13.699+0000 7f89cc553700 1 -- 172.21.15.162:0/27301717 wait complete. 2024-04-27T04:16:13.700 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:16:13.700+0000 7f89cc553700 1 Processor -- start 2024-04-27T04:16:13.700 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:16:13.700+0000 7f89cc553700 1 -- start start 2024-04-27T04:16:13.700 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:16:13.700+0000 7f89cc553700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f89c4065640 0x7f89c4113b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:16:13.700 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:16:13.700+0000 7f89cc553700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f89c4102e50 0x7f89c41140d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:16:13.700 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:16:13.700+0000 7f89cc553700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89c4114720 con 0x7f89c4065640 2024-04-27T04:16:13.700 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:16:13.700+0000 7f89cc553700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89c410db10 con 0x7f89c4102e50 2024-04-27T04:16:13.700 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:16:13.700+0000 7f89ca2ef700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f89c4065640 0x7f89c4113b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:16:13.700 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:16:13.700+0000 7f89ca2ef700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f89c4065640 0x7f89c4113b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:49590/0 (socket says 172.21.15.162:49590) 2024-04-27T04:16:13.701 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:16:13.700+0000 7f89ca2ef700 1 -- 172.21.15.162:0/2952489299 learned_addr learned my addr 172.21.15.162:0/2952489299 (peer_addr_for_me v2:172.21.15.162:0/0) 2024-04-27T04:16:13.701 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:16:13.700+0000 7f89c9aee700 1 --2- 172.21.15.162:0/2952489299 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f89c4102e50 0x7f89c41140d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:16:13.701 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:16:13.700+0000 7f89ca2ef700 1 -- 172.21.15.162:0/2952489299 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f89c4102e50 msgr2=0x7f89c41140d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:16:13.701 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:16:13.700+0000 7f89ca2ef700 1 --2- 172.21.15.162:0/2952489299 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f89c4102e50 0x7f89c41140d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:16:13.701 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:16:13.700+0000 7f89ca2ef700 1 -- 172.21.15.162:0/2952489299 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f89b4005040 con 0x7f89c4065640 2024-04-27T04:16:13.701 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:16:13.701+0000 7f89ca2ef700 1 --2- 172.21.15.162:0/2952489299 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f89c4065640 0x7f89c4113b90 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f89c000c810 tx=0x7f89c0004470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:16:13.701 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:16:13.701+0000 7f89baffd700 1 -- 172.21.15.162:0/2952489299 <== mon.0 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f89c0012810 con 0x7f89c4065640 2024-04-27T04:16:13.701 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:16:13.701+0000 7f89cc553700 1 -- 172.21.15.162:0/2952489299 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f89c410ddf0 con 0x7f89c4065640 2024-04-27T04:16:13.701 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:16:13.701+0000 7f89baffd700 1 -- 172.21.15.162:0/2952489299 <== mon.0 v2:172.21.15.162:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f89c0012e40 con 0x7f89c4065640 2024-04-27T04:16:13.701 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:16:13.701+0000 7f89baffd700 1 -- 172.21.15.162:0/2952489299 <== mon.0 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f89c000a370 con 0x7f89c4065640 2024-04-27T04:16:13.701 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:16:13.701+0000 7f89cc553700 1 -- 172.21.15.162:0/2952489299 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f89c410e3b0 con 0x7f89c4065640 2024-04-27T04:16:13.702 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:16:13.702+0000 7f89cc553700 1 -- 172.21.15.162:0/2952489299 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f89c4100100 con 0x7f89c4065640 2024-04-27T04:16:13.702 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:16:13.703+0000 7f89baffd700 1 -- 172.21.15.162:0/2952489299 <== mon.0 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 39) v1 ==== 92747+0+0 (secure 0 0 0) 0x7f89c000a550 con 0x7f89c4065640 2024-04-27T04:16:13.702 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:16:13.703+0000 7f89baffd700 1 --2- 172.21.15.162:0/2952489299 >> [v2:172.21.15.162:6800/1396866108,v1:172.21.15.162:6801/1396866108] conn(0x7f89b006ec00 0x7f89b00710b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:16:13.702 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:16:13.703+0000 7f89baffd700 1 -- 172.21.15.162:0/2952489299 <== mon.0 v2:172.21.15.162:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 6410+0+0 (secure 0 0 0) 0x7f89c008ecc0 con 0x7f89c4065640 2024-04-27T04:16:13.702 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:16:13.703+0000 7f89c9aee700 1 --2- 172.21.15.162:0/2952489299 >> [v2:172.21.15.162:6800/1396866108,v1:172.21.15.162:6801/1396866108] conn(0x7f89b006ec00 0x7f89b00710b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:16:13.702 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:16:13.704+0000 7f89c9aee700 1 --2- 172.21.15.162:0/2952489299 >> [v2:172.21.15.162:6800/1396866108,v1:172.21.15.162:6801/1396866108] conn(0x7f89b006ec00 0x7f89b00710b0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f89b400b3b0 tx=0x7f89b4009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:16:13.706 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:16:13.708+0000 7f89baffd700 1 -- 172.21.15.162:0/2952489299 <== mon.0 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7f89c005b660 con 0x7f89c4065640 2024-04-27T04:16:13.769 INFO:journalctl@ceph.osd.1.smithi162.stdout:Apr 27 04:16:13 smithi162 podman[126595]: 2024-04-27 04:16:13.727325837 +0000 UTC m=+0.657363482 container died 0b051b64e2afdd95189353b317ff61355abd70d8bc2a6eb104a74d4f86e4898d (image=quay.ceph.io/ceph-ci/ceph@sha256:90a3a9fa9ceb8977f6dad9ccd36fe46018d07e2118199820523a6b35301d9c16, name=ceph-7f9750d6-044a-11ef-bc93-c7b262605968-osd-1-deactivate, RELEASE=main-4e213a5, CEPH_POINT_RELEASE=, GIT_CLEAN=True, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240424, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-27T04:16:13.931 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:16:13.933+0000 7f89cc553700 1 -- 172.21.15.162:0/2952489299 --> [v2:172.21.15.162:6800/1396866108,v1:172.21.15.162:6801/1396866108] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f89c410f1e0 con 0x7f89b006ec00 2024-04-27T04:16:13.932 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:16:13.934+0000 7f89baffd700 1 -- 172.21.15.162:0/2952489299 <== mgr.34104 v2:172.21.15.162:6800/1396866108 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+400 (secure 0 0 0) 0x7f89c410f1e0 con 0x7f89b006ec00 2024-04-27T04:16:13.934 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:16:13.936+0000 7f89cc553700 1 -- 172.21.15.162:0/2952489299 >> [v2:172.21.15.162:6800/1396866108,v1:172.21.15.162:6801/1396866108] conn(0x7f89b006ec00 msgr2=0x7f89b00710b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:16:13.935 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:16:13.936+0000 7f89cc553700 1 --2- 172.21.15.162:0/2952489299 >> [v2:172.21.15.162:6800/1396866108,v1:172.21.15.162:6801/1396866108] conn(0x7f89b006ec00 0x7f89b00710b0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f89b400b3b0 tx=0x7f89b4009e30 comp rx=0 tx=0).stop 2024-04-27T04:16:13.935 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:16:13.936+0000 7f89cc553700 1 -- 172.21.15.162:0/2952489299 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f89c4065640 msgr2=0x7f89c4113b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:16:13.935 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:16:13.936+0000 7f89cc553700 1 --2- 172.21.15.162:0/2952489299 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f89c4065640 0x7f89c4113b90 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f89c000c810 tx=0x7f89c0004470 comp rx=0 tx=0).stop 2024-04-27T04:16:13.935 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:16:13.936+0000 7f89cc553700 1 -- 172.21.15.162:0/2952489299 shutdown_connections 2024-04-27T04:16:13.935 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:16:13.936+0000 7f89cc553700 1 --2- 172.21.15.162:0/2952489299 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f89c4065640 0x7f89c4113b90 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:16:13.935 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:16:13.936+0000 7f89cc553700 1 --2- 172.21.15.162:0/2952489299 >> [v2:172.21.15.162:6800/1396866108,v1:172.21.15.162:6801/1396866108] conn(0x7f89b006ec00 0x7f89b00710b0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:16:13.935 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:16:13.936+0000 7f89cc553700 1 --2- 172.21.15.162:0/2952489299 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f89c4102e50 0x7f89c41140d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:16:13.935 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:16:13.936+0000 7f89cc553700 1 -- 172.21.15.162:0/2952489299 >> 172.21.15.162:0/2952489299 conn(0x7f89c4071d00 msgr2=0x7f89c40faf60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:16:13.935 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:16:13.936+0000 7f89cc553700 1 -- 172.21.15.162:0/2952489299 shutdown_connections 2024-04-27T04:16:13.935 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:16:13.936+0000 7f89cc553700 1 -- 172.21.15.162:0/2952489299 wait complete. 2024-04-27T04:16:13.945 INFO:teuthology.orchestra.run.smithi162.stdout:true 2024-04-27T04:16:14.039 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:16:14.041+0000 7fc36c584700 1 -- 172.21.15.162:0/1973075417 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc364103780 msgr2=0x7fc3640fcdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:16:14.040 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:16:14.041+0000 7fc36c584700 1 --2- 172.21.15.162:0/1973075417 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc364103780 0x7fc3640fcdf0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fc354004940 tx=0x7fc354009e30 comp rx=0 tx=0).stop 2024-04-27T04:16:14.040 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:16:13 smithi162 ceph-mon[110641]: osdmap e56: 6 total, 5 up, 6 in 2024-04-27T04:16:14.042 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:16:14.042+0000 7fc36c584700 1 -- 172.21.15.162:0/1973075417 shutdown_connections 2024-04-27T04:16:14.042 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:16:14.042+0000 7fc36c584700 1 --2- 172.21.15.162:0/1973075417 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc364103780 0x7fc3640fcdf0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:16:14.042 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:16:14.042+0000 7fc36c584700 1 --2- 172.21.15.162:0/1973075417 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc364102de0 0x7fc3641031b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:16:14.042 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:16:14.042+0000 7fc36c584700 1 -- 172.21.15.162:0/1973075417 >> 172.21.15.162:0/1973075417 conn(0x7fc3640f8990 msgr2=0x7fc3640fada0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:16:14.042 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:16:14.042+0000 7fc36c584700 1 -- 172.21.15.162:0/1973075417 shutdown_connections 2024-04-27T04:16:14.042 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:16:14.042+0000 7fc36c584700 1 -- 172.21.15.162:0/1973075417 wait complete. 2024-04-27T04:16:14.042 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:16:14.042+0000 7fc36c584700 1 Processor -- start 2024-04-27T04:16:14.042 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:16:14.042+0000 7fc36c584700 1 -- start start 2024-04-27T04:16:14.042 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:16:14.042+0000 7fc36c584700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc364102de0 0x7fc364113970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:16:14.042 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:16:14.042+0000 7fc36c584700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc364103780 0x7fc364113eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:16:14.042 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:16:14.043+0000 7fc36c584700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc364114590 con 0x7fc364103780 2024-04-27T04:16:14.042 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:16:14.043+0000 7fc36c584700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc36410db20 con 0x7fc364102de0 2024-04-27T04:16:14.043 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:16:14.043+0000 7fc369b1f700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc364103780 0x7fc364113eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:16:14.043 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:16:14.043+0000 7fc369b1f700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc364103780 0x7fc364113eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:49606/0 (socket says 172.21.15.162:49606) 2024-04-27T04:16:14.043 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:16:14.043+0000 7fc369b1f700 1 -- 172.21.15.162:0/1015189561 learned_addr learned my addr 172.21.15.162:0/1015189561 (peer_addr_for_me v2:172.21.15.162:0/0) 2024-04-27T04:16:14.043 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:16:14.043+0000 7fc36a320700 1 --2- 172.21.15.162:0/1015189561 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc364102de0 0x7fc364113970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:16:14.043 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:16:14.043+0000 7fc369b1f700 1 -- 172.21.15.162:0/1015189561 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc364102de0 msgr2=0x7fc364113970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:16:14.043 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:16:14.043+0000 7fc369b1f700 1 --2- 172.21.15.162:0/1015189561 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc364102de0 0x7fc364113970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:16:14.043 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:16:14.043+0000 7fc369b1f700 1 -- 172.21.15.162:0/1015189561 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc354005040 con 0x7fc364103780 2024-04-27T04:16:14.043 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:16:14.043+0000 7fc369b1f700 1 --2- 172.21.15.162:0/1015189561 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc364103780 0x7fc364113eb0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fc354009100 tx=0x7fc354009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:16:14.043 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:16:14.043+0000 7fc35affd700 1 -- 172.21.15.162:0/1015189561 <== mon.0 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc35400e040 con 0x7fc364103780 2024-04-27T04:16:14.043 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:16:14.043+0000 7fc35affd700 1 -- 172.21.15.162:0/1015189561 <== mon.0 v2:172.21.15.162:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fc354004290 con 0x7fc364103780 2024-04-27T04:16:14.043 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:16:14.043+0000 7fc36c584700 1 -- 172.21.15.162:0/1015189561 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc36410dda0 con 0x7fc364103780 2024-04-27T04:16:14.043 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:16:14.043+0000 7fc35affd700 1 -- 172.21.15.162:0/1015189561 <== mon.0 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc35401c3f0 con 0x7fc364103780 2024-04-27T04:16:14.043 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:16:14.043+0000 7fc36c584700 1 -- 172.21.15.162:0/1015189561 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc36410e360 con 0x7fc364103780 2024-04-27T04:16:14.044 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:16:14.044+0000 7fc36c584700 1 -- 172.21.15.162:0/1015189561 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc3641000f0 con 0x7fc364103780 2024-04-27T04:16:14.044 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:16:14.045+0000 7fc35affd700 1 -- 172.21.15.162:0/1015189561 <== mon.0 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 39) v1 ==== 92747+0+0 (secure 0 0 0) 0x7fc354013690 con 0x7fc364103780 2024-04-27T04:16:14.044 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:16:14.045+0000 7fc35affd700 1 --2- 172.21.15.162:0/1015189561 >> [v2:172.21.15.162:6800/1396866108,v1:172.21.15.162:6801/1396866108] conn(0x7fc35006ec00 0x7fc3500710b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:16:14.044 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:16:14.045+0000 7fc35affd700 1 -- 172.21.15.162:0/1015189561 <== mon.0 v2:172.21.15.162:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 6410+0+0 (secure 0 0 0) 0x7fc35408f9b0 con 0x7fc364103780 2024-04-27T04:16:14.044 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:16:14.045+0000 7fc36a320700 1 --2- 172.21.15.162:0/1015189561 >> [v2:172.21.15.162:6800/1396866108,v1:172.21.15.162:6801/1396866108] conn(0x7fc35006ec00 0x7fc3500710b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:16:14.045 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:16:14.046+0000 7fc36a320700 1 --2- 172.21.15.162:0/1015189561 >> [v2:172.21.15.162:6800/1396866108,v1:172.21.15.162:6801/1396866108] conn(0x7fc35006ec00 0x7fc3500710b0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fc360009bf0 tx=0x7fc3600091a0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:16:14.048 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:16:14.050+0000 7fc35affd700 1 -- 172.21.15.162:0/1015189561 <== mon.0 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7fc354058830 con 0x7fc364103780 2024-04-27T04:16:14.172 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:16:13 smithi181 ceph-mon[81134]: osdmap e56: 6 total, 5 up, 6 in 2024-04-27T04:16:14.269 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:16:14.271+0000 7fc36c584700 1 -- 172.21.15.162:0/1015189561 --> [v2:172.21.15.162:6800/1396866108,v1:172.21.15.162:6801/1396866108] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fc364109f30 con 0x7fc35006ec00 2024-04-27T04:16:14.270 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:16:14.272+0000 7fc35affd700 1 -- 172.21.15.162:0/1015189561 <== mgr.34104 v2:172.21.15.162:6800/1396866108 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+400 (secure 0 0 0) 0x7fc364109f30 con 0x7fc35006ec00 2024-04-27T04:16:14.273 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:16:14.274+0000 7fc36c584700 1 -- 172.21.15.162:0/1015189561 >> [v2:172.21.15.162:6800/1396866108,v1:172.21.15.162:6801/1396866108] conn(0x7fc35006ec00 msgr2=0x7fc3500710b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:16:14.273 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:16:14.274+0000 7fc36c584700 1 --2- 172.21.15.162:0/1015189561 >> [v2:172.21.15.162:6800/1396866108,v1:172.21.15.162:6801/1396866108] conn(0x7fc35006ec00 0x7fc3500710b0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fc360009bf0 tx=0x7fc3600091a0 comp rx=0 tx=0).stop 2024-04-27T04:16:14.273 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:16:14.274+0000 7fc36c584700 1 -- 172.21.15.162:0/1015189561 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc364103780 msgr2=0x7fc364113eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:16:14.273 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:16:14.274+0000 7fc36c584700 1 --2- 172.21.15.162:0/1015189561 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc364103780 0x7fc364113eb0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fc354009100 tx=0x7fc354009e30 comp rx=0 tx=0).stop 2024-04-27T04:16:14.273 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:16:14.274+0000 7fc36c584700 1 -- 172.21.15.162:0/1015189561 shutdown_connections 2024-04-27T04:16:14.273 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:16:14.274+0000 7fc36c584700 1 --2- 172.21.15.162:0/1015189561 >> [v2:172.21.15.162:6800/1396866108,v1:172.21.15.162:6801/1396866108] conn(0x7fc35006ec00 0x7fc3500710b0 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:16:14.273 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:16:14.274+0000 7fc36c584700 1 --2- 172.21.15.162:0/1015189561 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc364102de0 0x7fc364113970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:16:14.273 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:16:14.274+0000 7fc36c584700 1 --2- 172.21.15.162:0/1015189561 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc364103780 0x7fc364113eb0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:16:14.273 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:16:14.274+0000 7fc36c584700 1 -- 172.21.15.162:0/1015189561 >> 172.21.15.162:0/1015189561 conn(0x7fc3640f8990 msgr2=0x7fc3640fa750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:16:14.273 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:16:14.275+0000 7fc36c584700 1 -- 172.21.15.162:0/1015189561 shutdown_connections 2024-04-27T04:16:14.273 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:16:14.275+0000 7fc36c584700 1 -- 172.21.15.162:0/1015189561 wait complete. 2024-04-27T04:16:14.376 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:16:14.378+0000 7f62ba83f700 1 -- 172.21.15.162:0/2017201799 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f62b4103780 msgr2=0x7f62b40fcdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:16:14.376 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:16:14.378+0000 7f62ba83f700 1 --2- 172.21.15.162:0/2017201799 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f62b4103780 0x7f62b40fcdf0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f62a4004940 tx=0x7f62a4009e30 comp rx=0 tx=0).stop 2024-04-27T04:16:14.376 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:16:14.378+0000 7f62ba83f700 1 -- 172.21.15.162:0/2017201799 shutdown_connections 2024-04-27T04:16:14.377 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:16:14.378+0000 7f62ba83f700 1 --2- 172.21.15.162:0/2017201799 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f62b4103780 0x7f62b40fcdf0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:16:14.377 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:16:14.378+0000 7f62ba83f700 1 --2- 172.21.15.162:0/2017201799 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f62b4102de0 0x7f62b41031b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:16:14.377 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:16:14.378+0000 7f62ba83f700 1 -- 172.21.15.162:0/2017201799 >> 172.21.15.162:0/2017201799 conn(0x7f62b40f8990 msgr2=0x7f62b40fada0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:16:14.377 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:16:14.378+0000 7f62ba83f700 1 -- 172.21.15.162:0/2017201799 shutdown_connections 2024-04-27T04:16:14.377 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:16:14.378+0000 7f62ba83f700 1 -- 172.21.15.162:0/2017201799 wait complete. 2024-04-27T04:16:14.379 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:16:14.379+0000 7f62ba83f700 1 Processor -- start 2024-04-27T04:16:14.379 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:16:14.379+0000 7f62ba83f700 1 -- start start 2024-04-27T04:16:14.379 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:16:14.379+0000 7f62ba83f700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f62b4102de0 0x7f62b4070260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:16:14.379 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:16:14.379+0000 7f62ba83f700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f62b4103780 0x7f62b40707a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:16:14.379 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:16:14.379+0000 7f62ba83f700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62b4072190 con 0x7f62b4102de0 2024-04-27T04:16:14.379 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:16:14.379+0000 7f62ba83f700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62b4070ce0 con 0x7f62b4103780 2024-04-27T04:16:14.379 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:16:14.379+0000 7f62b3fff700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f62b4102de0 0x7f62b4070260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:16:14.379 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:16:14.379+0000 7f62b3fff700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f62b4102de0 0x7f62b4070260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:49614/0 (socket says 172.21.15.162:49614) 2024-04-27T04:16:14.380 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:16:14.379+0000 7f62b3fff700 1 -- 172.21.15.162:0/1620458973 learned_addr learned my addr 172.21.15.162:0/1620458973 (peer_addr_for_me v2:172.21.15.162:0/0) 2024-04-27T04:16:14.380 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:16:14.380+0000 7f62b37fe700 1 --2- 172.21.15.162:0/1620458973 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f62b4103780 0x7f62b40707a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:16:14.380 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:16:14.380+0000 7f62b3fff700 1 -- 172.21.15.162:0/1620458973 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f62b4103780 msgr2=0x7f62b40707a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:16:14.380 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:16:14.380+0000 7f62b3fff700 1 --2- 172.21.15.162:0/1620458973 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f62b4103780 0x7f62b40707a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:16:14.380 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:16:14.380+0000 7f62b3fff700 1 -- 172.21.15.162:0/1620458973 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f62a4005040 con 0x7f62b4102de0 2024-04-27T04:16:14.380 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:16:14.380+0000 7f62b37fe700 1 --2- 172.21.15.162:0/1620458973 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f62b4103780 0x7f62b40707a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-27T04:16:14.380 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:16:14.380+0000 7f62b3fff700 1 --2- 172.21.15.162:0/1620458973 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f62b4102de0 0x7f62b4070260 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f62a800eb40 tx=0x7f62a800b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:16:14.380 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:16:14.380+0000 7f62b0ff9700 1 -- 172.21.15.162:0/1620458973 <== mon.0 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f62a8007cc0 con 0x7f62b4102de0 2024-04-27T04:16:14.380 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:16:14.380+0000 7f62b0ff9700 1 -- 172.21.15.162:0/1620458973 <== mon.0 v2:172.21.15.162:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f62a800f040 con 0x7f62b4102de0 2024-04-27T04:16:14.380 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:16:14.380+0000 7f62ba83f700 1 -- 172.21.15.162:0/1620458973 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f62b4070f60 con 0x7f62b4102de0 2024-04-27T04:16:14.380 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:16:14.380+0000 7f62b0ff9700 1 -- 172.21.15.162:0/1620458973 <== mon.0 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f62a8009620 con 0x7f62b4102de0 2024-04-27T04:16:14.380 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:16:14.380+0000 7f62ba83f700 1 -- 172.21.15.162:0/1620458973 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f62b411a020 con 0x7f62b4102de0 2024-04-27T04:16:14.381 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:16:14.381+0000 7f62ba83f700 1 -- 172.21.15.162:0/1620458973 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f62b40fe530 con 0x7f62b4102de0 2024-04-27T04:16:14.381 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:16:14.382+0000 7f62b0ff9700 1 -- 172.21.15.162:0/1620458973 <== mon.0 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 39) v1 ==== 92747+0+0 (secure 0 0 0) 0x7f62a8009780 con 0x7f62b4102de0 2024-04-27T04:16:14.381 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:16:14.382+0000 7f62b0ff9700 1 --2- 172.21.15.162:0/1620458973 >> [v2:172.21.15.162:6800/1396866108,v1:172.21.15.162:6801/1396866108] conn(0x7f629c06ec00 0x7f629c0710b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:16:14.381 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:16:14.383+0000 7f62b0ff9700 1 -- 172.21.15.162:0/1620458973 <== mon.0 v2:172.21.15.162:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 6410+0+0 (secure 0 0 0) 0x7f62a808e520 con 0x7f62b4102de0 2024-04-27T04:16:14.381 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:16:14.383+0000 7f62b37fe700 1 --2- 172.21.15.162:0/1620458973 >> [v2:172.21.15.162:6800/1396866108,v1:172.21.15.162:6801/1396866108] conn(0x7f629c06ec00 0x7f629c0710b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:16:14.382 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:16:14.383+0000 7f62b37fe700 1 --2- 172.21.15.162:0/1620458973 >> [v2:172.21.15.162:6800/1396866108,v1:172.21.15.162:6801/1396866108] conn(0x7f629c06ec00 0x7f629c0710b0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f62a400b0d0 tx=0x7f62a4009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:16:14.386 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:16:14.388+0000 7f62b0ff9700 1 -- 172.21.15.162:0/1620458973 <== mon.0 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7f62a80573d0 con 0x7f62b4102de0 2024-04-27T04:16:14.616 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:16:14.617+0000 7f62ba83f700 1 -- 172.21.15.162:0/1620458973 --> [v2:172.21.15.162:6800/1396866108,v1:172.21.15.162:6801/1396866108] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f62b40f9f90 con 0x7f629c06ec00 2024-04-27T04:16:14.626 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:16:14.628+0000 7f62b0ff9700 1 -- 172.21.15.162:0/1620458973 <== mgr.34104 v2:172.21.15.162:6800/1396866108 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3520 (secure 0 0 0) 0x7f62b40f9f90 con 0x7f629c06ec00 2024-04-27T04:16:14.626 INFO:teuthology.orchestra.run.smithi162.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-27T04:16:14.626 INFO:teuthology.orchestra.run.smithi162.stdout:alertmanager.smithi162 smithi162 *:9093,9094 running (3m) 28s ago 14m 27.6M - 0.25.0 c8568f914cd2 40e334659017 2024-04-27T04:16:14.626 INFO:teuthology.orchestra.run.smithi162.stdout:crash.smithi162 smithi162 running (60s) 28s ago 14m 6660k - 19.0.0-3253-g4e213a58 67e414ff4bc4 1aa8f4d5a28b 2024-04-27T04:16:14.626 INFO:teuthology.orchestra.run.smithi162.stdout:crash.smithi181 smithi181 running (55s) 51s ago 13m 6656k - 19.0.0-3253-g4e213a58 67e414ff4bc4 832d9cbc5c30 2024-04-27T04:16:14.626 INFO:teuthology.orchestra.run.smithi162.stdout:grafana.smithi162 smithi162 *:3000 running (2m) 28s ago 14m 75.5M - 9.4.12 2bacad6d85d8 3d78c478f03f 2024-04-27T04:16:14.627 INFO:teuthology.orchestra.run.smithi162.stdout:mds.cephfs.smithi162.gzlyjj smithi162 running (9m) 28s ago 9m 14.0M - 17.2.7-907-g2be3efa9 1c5834633715 27e5b030dd32 2024-04-27T04:16:14.627 INFO:teuthology.orchestra.run.smithi162.stdout:mds.cephfs.smithi162.xcnapk smithi162 running (9m) 28s ago 9m 100M - 17.2.7-907-g2be3efa9 1c5834633715 38f9030df029 2024-04-27T04:16:14.627 INFO:teuthology.orchestra.run.smithi162.stdout:mds.cephfs.smithi181.bkkcdm smithi181 running (9m) 51s ago 9m 15.3M - 17.2.7-907-g2be3efa9 1c5834633715 af55afca1263 2024-04-27T04:16:14.627 INFO:teuthology.orchestra.run.smithi162.stdout:mds.cephfs.smithi181.dzomgh smithi181 running (9m) 51s ago 9m 143M - 17.2.7-907-g2be3efa9 1c5834633715 6733296597f3 2024-04-27T04:16:14.627 INFO:teuthology.orchestra.run.smithi162.stdout:mgr.smithi162.oennfn smithi162 *:8443,9283,8765 running (4m) 28s ago 16m 499M - 19.0.0-3253-g4e213a58 67e414ff4bc4 58a748d3637b 2024-04-27T04:16:14.627 INFO:teuthology.orchestra.run.smithi162.stdout:mgr.smithi181.vtwuhz smithi181 *:8443,9283,8765 running (4m) 51s ago 13m 450M - 19.0.0-3253-g4e213a58 67e414ff4bc4 813584297b1d 2024-04-27T04:16:14.627 INFO:teuthology.orchestra.run.smithi162.stdout:mon.smithi162 smithi162 running (2m) 28s ago 16m 42.5M 2048M 19.0.0-3253-g4e213a58 67e414ff4bc4 375adaf4468f 2024-04-27T04:16:14.627 INFO:teuthology.orchestra.run.smithi162.stdout:mon.smithi181 smithi181 running (100s) 51s ago 13m 33.1M 2048M 19.0.0-3253-g4e213a58 67e414ff4bc4 77b71f6d522f 2024-04-27T04:16:14.627 INFO:teuthology.orchestra.run.smithi162.stdout:node-exporter.smithi162 smithi162 *:9100 running (4m) 28s ago 14m 16.2M - 1.5.0 0da6a335fe13 5c4c5ea9c1d7 2024-04-27T04:16:14.627 INFO:teuthology.orchestra.run.smithi162.stdout:node-exporter.smithi181 smithi181 *:9100 running (3m) 51s ago 13m 15.7M - 1.5.0 0da6a335fe13 9067f14c0519 2024-04-27T04:16:14.627 INFO:teuthology.orchestra.run.smithi162.stdout:osd.0 smithi162 running (34s) 28s ago 12m 12.6M 1557M 19.0.0-3253-g4e213a58 67e414ff4bc4 2744aff3cd88 2024-04-27T04:16:14.627 INFO:teuthology.orchestra.run.smithi162.stdout:osd.1 smithi162 running (12m) 28s ago 12m 247M 1557M 17.2.7-907-g2be3efa9 1c5834633715 aefb464c46e8 2024-04-27T04:16:14.627 INFO:teuthology.orchestra.run.smithi162.stdout:osd.2 smithi162 running (11m) 28s ago 11m 206M 1557M 17.2.7-907-g2be3efa9 1c5834633715 70f16d86d38e 2024-04-27T04:16:14.627 INFO:teuthology.orchestra.run.smithi162.stdout:osd.3 smithi181 running (11m) 51s ago 11m 278M 2581M 17.2.7-907-g2be3efa9 1c5834633715 030d829be54e 2024-04-27T04:16:14.627 INFO:teuthology.orchestra.run.smithi162.stdout:osd.4 smithi181 running (11m) 51s ago 11m 247M 2581M 17.2.7-907-g2be3efa9 1c5834633715 512afbee7cf7 2024-04-27T04:16:14.628 INFO:teuthology.orchestra.run.smithi162.stdout:osd.5 smithi181 running (10m) 51s ago 10m 224M 2581M 17.2.7-907-g2be3efa9 1c5834633715 f47a1daac1ea 2024-04-27T04:16:14.628 INFO:teuthology.orchestra.run.smithi162.stdout:prometheus.smithi162 smithi162 *:9095 running (3m) 28s ago 14m 67.5M - 2.43.0 a07b618ecd1d f30657a43324 2024-04-27T04:16:14.628 INFO:journalctl@ceph.osd.1.smithi162.stdout:Apr 27 04:16:14 smithi162 podman[126644]: 2024-04-27 04:16:14.381754428 +0000 UTC m=+0.644732761 container remove 0b051b64e2afdd95189353b317ff61355abd70d8bc2a6eb104a74d4f86e4898d (image=quay.ceph.io/ceph-ci/ceph@sha256:90a3a9fa9ceb8977f6dad9ccd36fe46018d07e2118199820523a6b35301d9c16, name=ceph-7f9750d6-044a-11ef-bc93-c7b262605968-osd-1-deactivate, GIT_BRANCH=HEAD, RELEASE=main-4e213a5, org.label-schema.build-date=20240424, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-27T04:16:14.628 INFO:journalctl@ceph.osd.1.smithi162.stdout:Apr 27 04:16:14 smithi162 systemd[1]: ceph-7f9750d6-044a-11ef-bc93-c7b262605968@osd.1.service: Deactivated successfully. 2024-04-27T04:16:14.628 INFO:journalctl@ceph.osd.1.smithi162.stdout:Apr 27 04:16:14 smithi162 systemd[1]: Stopped Ceph osd.1 for 7f9750d6-044a-11ef-bc93-c7b262605968. 2024-04-27T04:16:14.628 INFO:journalctl@ceph.osd.1.smithi162.stdout:Apr 27 04:16:14 smithi162 systemd[1]: ceph-7f9750d6-044a-11ef-bc93-c7b262605968@osd.1.service: Consumed 1min 7.644s CPU time. 2024-04-27T04:16:14.628 INFO:journalctl@ceph.osd.1.smithi162.stdout:Apr 27 04:16:14 smithi162 systemd[1]: Starting Ceph osd.1 for 7f9750d6-044a-11ef-bc93-c7b262605968... 2024-04-27T04:16:14.629 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:16:14.631+0000 7f62ba83f700 1 -- 172.21.15.162:0/1620458973 >> [v2:172.21.15.162:6800/1396866108,v1:172.21.15.162:6801/1396866108] conn(0x7f629c06ec00 msgr2=0x7f629c0710b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:16:14.629 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:16:14.631+0000 7f62ba83f700 1 --2- 172.21.15.162:0/1620458973 >> [v2:172.21.15.162:6800/1396866108,v1:172.21.15.162:6801/1396866108] conn(0x7f629c06ec00 0x7f629c0710b0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f62a400b0d0 tx=0x7f62a4009e30 comp rx=0 tx=0).stop 2024-04-27T04:16:14.630 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:16:14.631+0000 7f62ba83f700 1 -- 172.21.15.162:0/1620458973 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f62b4102de0 msgr2=0x7f62b4070260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:16:14.630 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:16:14.631+0000 7f62ba83f700 1 --2- 172.21.15.162:0/1620458973 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f62b4102de0 0x7f62b4070260 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f62a800eb40 tx=0x7f62a800b040 comp rx=0 tx=0).stop 2024-04-27T04:16:14.630 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:16:14.631+0000 7f62ba83f700 1 -- 172.21.15.162:0/1620458973 shutdown_connections 2024-04-27T04:16:14.630 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:16:14.631+0000 7f62ba83f700 1 --2- 172.21.15.162:0/1620458973 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f62b4102de0 0x7f62b4070260 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:16:14.630 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:16:14.631+0000 7f62ba83f700 1 --2- 172.21.15.162:0/1620458973 >> [v2:172.21.15.162:6800/1396866108,v1:172.21.15.162:6801/1396866108] conn(0x7f629c06ec00 0x7f629c0710b0 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:16:14.630 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:16:14.631+0000 7f62ba83f700 1 --2- 172.21.15.162:0/1620458973 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f62b4103780 0x7f62b40707a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:16:14.630 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:16:14.631+0000 7f62ba83f700 1 -- 172.21.15.162:0/1620458973 >> 172.21.15.162:0/1620458973 conn(0x7f62b40f8990 msgr2=0x7f62b40f9880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:16:14.630 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:16:14.632+0000 7f62ba83f700 1 -- 172.21.15.162:0/1620458973 shutdown_connections 2024-04-27T04:16:14.630 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:16:14.632+0000 7f62ba83f700 1 -- 172.21.15.162:0/1620458973 wait complete. 2024-04-27T04:16:14.753 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:16:14.755+0000 7fd76d9b4700 1 -- 172.21.15.162:0/2855027512 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd768103780 msgr2=0x7fd7680fcdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:16:14.753 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:16:14.755+0000 7fd76d9b4700 1 --2- 172.21.15.162:0/2855027512 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd768103780 0x7fd7680fcdf0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fd75c004940 tx=0x7fd75c009e30 comp rx=0 tx=0).stop 2024-04-27T04:16:14.753 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:16:14.755+0000 7fd76d9b4700 1 -- 172.21.15.162:0/2855027512 shutdown_connections 2024-04-27T04:16:14.753 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:16:14.755+0000 7fd76d9b4700 1 --2- 172.21.15.162:0/2855027512 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd768103780 0x7fd7680fcdf0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:16:14.754 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:16:14.755+0000 7fd76d9b4700 1 --2- 172.21.15.162:0/2855027512 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd768102de0 0x7fd7681031b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:16:14.754 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:16:14.755+0000 7fd76d9b4700 1 -- 172.21.15.162:0/2855027512 >> 172.21.15.162:0/2855027512 conn(0x7fd7680f8950 msgr2=0x7fd7680fad60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:16:14.754 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:16:14.755+0000 7fd76d9b4700 1 -- 172.21.15.162:0/2855027512 shutdown_connections 2024-04-27T04:16:14.754 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:16:14.755+0000 7fd76d9b4700 1 -- 172.21.15.162:0/2855027512 wait complete. 2024-04-27T04:16:14.754 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:16:14.755+0000 7fd76d9b4700 1 Processor -- start 2024-04-27T04:16:14.754 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:16:14.755+0000 7fd76d9b4700 1 -- start start 2024-04-27T04:16:14.755 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:16:14.756+0000 7fd76d9b4700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd768102de0 0x7fd768113b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:16:14.756 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:16:14.756+0000 7fd76d9b4700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd768103780 0x7fd768114080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:16:14.756 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:16:14.756+0000 7fd76d9b4700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd768114760 con 0x7fd768102de0 2024-04-27T04:16:14.756 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:16:14.756+0000 7fd76d9b4700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd76810dac0 con 0x7fd768103780 2024-04-27T04:16:14.756 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:16:14.756+0000 7fd766ffd700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd768102de0 0x7fd768113b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:16:14.756 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:16:14.756+0000 7fd766ffd700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd768102de0 0x7fd768113b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:49632/0 (socket says 172.21.15.162:49632) 2024-04-27T04:16:14.756 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:16:14.756+0000 7fd766ffd700 1 -- 172.21.15.162:0/3273604472 learned_addr learned my addr 172.21.15.162:0/3273604472 (peer_addr_for_me v2:172.21.15.162:0/0) 2024-04-27T04:16:14.756 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:16:14.756+0000 7fd7667fc700 1 --2- 172.21.15.162:0/3273604472 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd768103780 0x7fd768114080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:16:14.756 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:16:14.756+0000 7fd766ffd700 1 -- 172.21.15.162:0/3273604472 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd768103780 msgr2=0x7fd768114080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:16:14.756 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:16:14.756+0000 7fd766ffd700 1 --2- 172.21.15.162:0/3273604472 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd768103780 0x7fd768114080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:16:14.756 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:16:14.756+0000 7fd766ffd700 1 -- 172.21.15.162:0/3273604472 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd75c005040 con 0x7fd768102de0 2024-04-27T04:16:14.756 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:16:14.756+0000 7fd7667fc700 1 --2- 172.21.15.162:0/3273604472 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd768103780 0x7fd768114080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-27T04:16:14.756 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:16:14.756+0000 7fd766ffd700 1 --2- 172.21.15.162:0/3273604472 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd768102de0 0x7fd768113b40 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fd75800b790 tx=0x7fd758009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:16:14.756 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:16:14.756+0000 7fd74f7fe700 1 -- 172.21.15.162:0/3273604472 <== mon.0 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd75800f770 con 0x7fd768102de0 2024-04-27T04:16:14.756 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:16:14.756+0000 7fd76d9b4700 1 -- 172.21.15.162:0/3273604472 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd76810dda0 con 0x7fd768102de0 2024-04-27T04:16:14.757 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:16:14.756+0000 7fd74f7fe700 1 -- 172.21.15.162:0/3273604472 <== mon.0 v2:172.21.15.162:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fd75800fda0 con 0x7fd768102de0 2024-04-27T04:16:14.757 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:16:14.756+0000 7fd74f7fe700 1 -- 172.21.15.162:0/3273604472 <== mon.0 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd75800d310 con 0x7fd768102de0 2024-04-27T04:16:14.757 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:16:14.756+0000 7fd76d9b4700 1 -- 172.21.15.162:0/3273604472 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd76810e360 con 0x7fd768102de0 2024-04-27T04:16:14.757 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:16:14.757+0000 7fd76d9b4700 1 -- 172.21.15.162:0/3273604472 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd7681000f0 con 0x7fd768102de0 2024-04-27T04:16:14.757 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:16:14.758+0000 7fd74f7fe700 1 -- 172.21.15.162:0/3273604472 <== mon.0 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 39) v1 ==== 92747+0+0 (secure 0 0 0) 0x7fd758010420 con 0x7fd768102de0 2024-04-27T04:16:14.758 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:16:14.758+0000 7fd74f7fe700 1 --2- 172.21.15.162:0/3273604472 >> [v2:172.21.15.162:6800/1396866108,v1:172.21.15.162:6801/1396866108] conn(0x7fd75006ec00 0x7fd7500710b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:16:14.758 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:16:14.758+0000 7fd74f7fe700 1 -- 172.21.15.162:0/3273604472 <== mon.0 v2:172.21.15.162:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 6410+0+0 (secure 0 0 0) 0x7fd75808e790 con 0x7fd768102de0 2024-04-27T04:16:14.758 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:16:14.758+0000 7fd7667fc700 1 --2- 172.21.15.162:0/3273604472 >> [v2:172.21.15.162:6800/1396866108,v1:172.21.15.162:6801/1396866108] conn(0x7fd75006ec00 0x7fd7500710b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:16:14.758 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:16:14.759+0000 7fd7667fc700 1 --2- 172.21.15.162:0/3273604472 >> [v2:172.21.15.162:6800/1396866108,v1:172.21.15.162:6801/1396866108] conn(0x7fd75006ec00 0x7fd7500710b0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fd76810ec80 tx=0x7fd75c009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:16:14.761 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:16:14.763+0000 7fd74f7fe700 1 -- 172.21.15.162:0/3273604472 <== mon.0 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7fd75805b1e0 con 0x7fd768102de0 2024-04-27T04:16:14.886 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:16:14 smithi162 ceph-mon[110641]: pgmap v68: 65 pgs: 11 active+undersized+degraded, 3 stale+active+clean, 18 active+undersized, 33 active+clean; 208 MiB data, 2.1 GiB used, 534 GiB / 536 GiB avail; 27/231 objects degraded (11.688%) 2024-04-27T04:16:14.886 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:16:14 smithi162 ceph-mon[110641]: Health check failed: Degraded data redundancy: 27/231 objects degraded (11.688%), 11 pgs degraded (PG_DEGRADED) 2024-04-27T04:16:14.887 INFO:journalctl@ceph.osd.1.smithi162.stdout:Apr 27 04:16:14 smithi162 podman[126796]: 2024-04-27 04:16:14.744803673 +0000 UTC m=+0.018527499 image pull 67e414ff4bc41092cb31c84c98a9563a32813d02583028679c46b40362afeb3e quay.ceph.io/ceph-ci/ceph@sha256:90a3a9fa9ceb8977f6dad9ccd36fe46018d07e2118199820523a6b35301d9c16 2024-04-27T04:16:14.887 INFO:journalctl@ceph.osd.1.smithi162.stdout:Apr 27 04:16:14 smithi162 podman[126796]: 2024-04-27 04:16:14.857818265 +0000 UTC m=+0.131542111 container create 613936c9f1cbaf4954bb98f77a0f82b73afe9094474c518a12304ff03a2565e9 (image=quay.ceph.io/ceph-ci/ceph@sha256:90a3a9fa9ceb8977f6dad9ccd36fe46018d07e2118199820523a6b35301d9c16, name=ceph-7f9750d6-044a-11ef-bc93-c7b262605968-osd-1-activate, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, RELEASE=main-4e213a5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240424) 2024-04-27T04:16:14.922 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:16:14 smithi181 ceph-mon[81134]: pgmap v68: 65 pgs: 11 active+undersized+degraded, 3 stale+active+clean, 18 active+undersized, 33 active+clean; 208 MiB data, 2.1 GiB used, 534 GiB / 536 GiB avail; 27/231 objects degraded (11.688%) 2024-04-27T04:16:14.922 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:16:14 smithi181 ceph-mon[81134]: Health check failed: Degraded data redundancy: 27/231 objects degraded (11.688%), 11 pgs degraded (PG_DEGRADED) 2024-04-27T04:16:15.059 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:16:15.061+0000 7fd76d9b4700 1 -- 172.21.15.162:0/3273604472 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7fd76810f190 con 0x7fd768102de0 2024-04-27T04:16:15.059 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:16:15.061+0000 7fd74f7fe700 1 -- 172.21.15.162:0/3273604472 <== mon.0 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+839 (secure 0 0 0) 0x7fd75805ad70 con 0x7fd768102de0 2024-04-27T04:16:15.060 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-04-27T04:16:15.060 INFO:teuthology.orchestra.run.smithi162.stdout: "mon": { 2024-04-27T04:16:15.060 INFO:teuthology.orchestra.run.smithi162.stdout: "ceph version 19.0.0-3253-g4e213a58 (4e213a58e787437ff508e6c890ccd5fd32dc96b5) squid (dev)": 2 2024-04-27T04:16:15.060 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-04-27T04:16:15.060 INFO:teuthology.orchestra.run.smithi162.stdout: "mgr": { 2024-04-27T04:16:15.060 INFO:teuthology.orchestra.run.smithi162.stdout: "ceph version 19.0.0-3253-g4e213a58 (4e213a58e787437ff508e6c890ccd5fd32dc96b5) squid (dev)": 2 2024-04-27T04:16:15.060 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-04-27T04:16:15.060 INFO:teuthology.orchestra.run.smithi162.stdout: "osd": { 2024-04-27T04:16:15.060 INFO:teuthology.orchestra.run.smithi162.stdout: "ceph version 17.2.7-907-g2be3efa9 (2be3efa97a7972f8e925bd899b474329979b74d6) quincy (stable)": 4, 2024-04-27T04:16:15.060 INFO:teuthology.orchestra.run.smithi162.stdout: "ceph version 19.0.0-3253-g4e213a58 (4e213a58e787437ff508e6c890ccd5fd32dc96b5) squid (dev)": 1 2024-04-27T04:16:15.060 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-04-27T04:16:15.060 INFO:teuthology.orchestra.run.smithi162.stdout: "mds": { 2024-04-27T04:16:15.061 INFO:teuthology.orchestra.run.smithi162.stdout: "ceph version 17.2.7-907-g2be3efa9 (2be3efa97a7972f8e925bd899b474329979b74d6) quincy (stable)": 4 2024-04-27T04:16:15.061 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-04-27T04:16:15.061 INFO:teuthology.orchestra.run.smithi162.stdout: "overall": { 2024-04-27T04:16:15.061 INFO:teuthology.orchestra.run.smithi162.stdout: "ceph version 17.2.7-907-g2be3efa9 (2be3efa97a7972f8e925bd899b474329979b74d6) quincy (stable)": 8, 2024-04-27T04:16:15.061 INFO:teuthology.orchestra.run.smithi162.stdout: "ceph version 19.0.0-3253-g4e213a58 (4e213a58e787437ff508e6c890ccd5fd32dc96b5) squid (dev)": 5 2024-04-27T04:16:15.061 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-04-27T04:16:15.061 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-04-27T04:16:15.062 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:16:15.063+0000 7fd76d9b4700 1 -- 172.21.15.162:0/3273604472 >> [v2:172.21.15.162:6800/1396866108,v1:172.21.15.162:6801/1396866108] conn(0x7fd75006ec00 msgr2=0x7fd7500710b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:16:15.062 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:16:15.063+0000 7fd76d9b4700 1 --2- 172.21.15.162:0/3273604472 >> [v2:172.21.15.162:6800/1396866108,v1:172.21.15.162:6801/1396866108] conn(0x7fd75006ec00 0x7fd7500710b0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fd76810ec80 tx=0x7fd75c009e30 comp rx=0 tx=0).stop 2024-04-27T04:16:15.062 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:16:15.063+0000 7fd76d9b4700 1 -- 172.21.15.162:0/3273604472 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd768102de0 msgr2=0x7fd768113b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:16:15.062 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:16:15.063+0000 7fd76d9b4700 1 --2- 172.21.15.162:0/3273604472 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd768102de0 0x7fd768113b40 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fd75800b790 tx=0x7fd758009250 comp rx=0 tx=0).stop 2024-04-27T04:16:15.062 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:16:15.064+0000 7fd76d9b4700 1 -- 172.21.15.162:0/3273604472 shutdown_connections 2024-04-27T04:16:15.062 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:16:15.064+0000 7fd76d9b4700 1 --2- 172.21.15.162:0/3273604472 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd768102de0 0x7fd768113b40 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:16:15.062 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:16:15.064+0000 7fd76d9b4700 1 --2- 172.21.15.162:0/3273604472 >> [v2:172.21.15.162:6800/1396866108,v1:172.21.15.162:6801/1396866108] conn(0x7fd75006ec00 0x7fd7500710b0 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:16:15.062 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:16:15.064+0000 7fd76d9b4700 1 --2- 172.21.15.162:0/3273604472 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd768103780 0x7fd768114080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:16:15.062 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:16:15.064+0000 7fd76d9b4700 1 -- 172.21.15.162:0/3273604472 >> 172.21.15.162:0/3273604472 conn(0x7fd7680f8950 msgr2=0x7fd7680fa690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:16:15.062 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:16:15.064+0000 7fd76d9b4700 1 -- 172.21.15.162:0/3273604472 shutdown_connections 2024-04-27T04:16:15.062 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:16:15.064+0000 7fd76d9b4700 1 -- 172.21.15.162:0/3273604472 wait complete. 2024-04-27T04:16:15.166 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:16:15.167+0000 7fce844d0700 1 -- 172.21.15.162:0/4234860005 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fce7c102de0 msgr2=0x7fce7c1031b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:16:15.166 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:16:15.167+0000 7fce844d0700 1 --2- 172.21.15.162:0/4234860005 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fce7c102de0 0x7fce7c1031b0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fce78004940 tx=0x7fce78009e30 comp rx=0 tx=0).stop 2024-04-27T04:16:15.166 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:16:15.167+0000 7fce844d0700 1 -- 172.21.15.162:0/4234860005 shutdown_connections 2024-04-27T04:16:15.166 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:16:15.167+0000 7fce844d0700 1 --2- 172.21.15.162:0/4234860005 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fce7c103780 0x7fce7c0fcdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:16:15.166 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:16:15.167+0000 7fce844d0700 1 --2- 172.21.15.162:0/4234860005 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fce7c102de0 0x7fce7c1031b0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:16:15.166 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:16:15.167+0000 7fce844d0700 1 -- 172.21.15.162:0/4234860005 >> 172.21.15.162:0/4234860005 conn(0x7fce7c0f8990 msgr2=0x7fce7c0fada0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:16:15.166 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:16:15.167+0000 7fce844d0700 1 -- 172.21.15.162:0/4234860005 shutdown_connections 2024-04-27T04:16:15.167 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:16:15.167+0000 7fce844d0700 1 -- 172.21.15.162:0/4234860005 wait complete. 2024-04-27T04:16:15.167 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:16:15.168+0000 7fce844d0700 1 Processor -- start 2024-04-27T04:16:15.167 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:16:15.168+0000 7fce844d0700 1 -- start start 2024-04-27T04:16:15.167 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:16:15.168+0000 7fce844d0700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fce7c102de0 0x7fce7c113970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:16:15.167 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:16:15.168+0000 7fce844d0700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fce7c103780 0x7fce7c113eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:16:15.167 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:16:15.168+0000 7fce844d0700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce7c114500 con 0x7fce7c103780 2024-04-27T04:16:15.167 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:16:15.168+0000 7fce844d0700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce7c10db50 con 0x7fce7c102de0 2024-04-27T04:16:15.167 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:16:15.168+0000 7fce81a6b700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fce7c103780 0x7fce7c113eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:16:15.167 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:16:15.168+0000 7fce81a6b700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fce7c103780 0x7fce7c113eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:49652/0 (socket says 172.21.15.162:49652) 2024-04-27T04:16:15.167 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:16:15.168+0000 7fce81a6b700 1 -- 172.21.15.162:0/2710272664 learned_addr learned my addr 172.21.15.162:0/2710272664 (peer_addr_for_me v2:172.21.15.162:0/0) 2024-04-27T04:16:15.167 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:16:15.168+0000 7fce8226c700 1 --2- 172.21.15.162:0/2710272664 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fce7c102de0 0x7fce7c113970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:16:15.167 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:16:15.168+0000 7fce81a6b700 1 -- 172.21.15.162:0/2710272664 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fce7c102de0 msgr2=0x7fce7c113970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:16:15.167 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:16:15.168+0000 7fce81a6b700 1 --2- 172.21.15.162:0/2710272664 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fce7c102de0 0x7fce7c113970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:16:15.167 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:16:15.168+0000 7fce81a6b700 1 -- 172.21.15.162:0/2710272664 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce78005040 con 0x7fce7c103780 2024-04-27T04:16:15.169 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:16:15.168+0000 7fce81a6b700 1 --2- 172.21.15.162:0/2710272664 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fce7c103780 0x7fce7c113eb0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fce6c002a50 tx=0x7fce6c004600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:16:15.169 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:16:15.169+0000 7fce72ffd700 1 -- 172.21.15.162:0/2710272664 <== mon.0 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fce6c007cc0 con 0x7fce7c103780 2024-04-27T04:16:15.169 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:16:15.169+0000 7fce72ffd700 1 -- 172.21.15.162:0/2710272664 <== mon.0 v2:172.21.15.162:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fce6c00f040 con 0x7fce7c103780 2024-04-27T04:16:15.169 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:16:15.169+0000 7fce844d0700 1 -- 172.21.15.162:0/2710272664 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fce7c10de30 con 0x7fce7c103780 2024-04-27T04:16:15.169 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:16:15.169+0000 7fce72ffd700 1 -- 172.21.15.162:0/2710272664 <== mon.0 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fce6c00d9c0 con 0x7fce7c103780 2024-04-27T04:16:15.169 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:16:15.169+0000 7fce844d0700 1 -- 172.21.15.162:0/2710272664 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fce7c10e3f0 con 0x7fce7c103780 2024-04-27T04:16:15.169 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:16:15.170+0000 7fce844d0700 1 -- 172.21.15.162:0/2710272664 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fce7c1000f0 con 0x7fce7c103780 2024-04-27T04:16:15.169 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:16:15.170+0000 7fce72ffd700 1 -- 172.21.15.162:0/2710272664 <== mon.0 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 39) v1 ==== 92747+0+0 (secure 0 0 0) 0x7fce6c009730 con 0x7fce7c103780 2024-04-27T04:16:15.169 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:16:15.171+0000 7fce72ffd700 1 --2- 172.21.15.162:0/2710272664 >> [v2:172.21.15.162:6800/1396866108,v1:172.21.15.162:6801/1396866108] conn(0x7fce6806ec00 0x7fce680710b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:16:15.169 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:16:15.171+0000 7fce72ffd700 1 -- 172.21.15.162:0/2710272664 <== mon.0 v2:172.21.15.162:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 6410+0+0 (secure 0 0 0) 0x7fce6c08e4e0 con 0x7fce7c103780 2024-04-27T04:16:15.170 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:16:15.171+0000 7fce8226c700 1 --2- 172.21.15.162:0/2710272664 >> [v2:172.21.15.162:6800/1396866108,v1:172.21.15.162:6801/1396866108] conn(0x7fce6806ec00 0x7fce680710b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:16:15.170 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:16:15.171+0000 7fce8226c700 1 --2- 172.21.15.162:0/2710272664 >> [v2:172.21.15.162:6800/1396866108,v1:172.21.15.162:6801/1396866108] conn(0x7fce6806ec00 0x7fce680710b0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fce78004f90 tx=0x7fce78009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:16:15.173 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:16:15.175+0000 7fce72ffd700 1 -- 172.21.15.162:0/2710272664 <== mon.0 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7fce6c05ae90 con 0x7fce7c103780 2024-04-27T04:16:15.386 INFO:journalctl@ceph.osd.1.smithi162.stdout:Apr 27 04:16:15 smithi162 podman[126796]: 2024-04-27 04:16:15.177790132 +0000 UTC m=+0.451513936 container init 613936c9f1cbaf4954bb98f77a0f82b73afe9094474c518a12304ff03a2565e9 (image=quay.ceph.io/ceph-ci/ceph@sha256:90a3a9fa9ceb8977f6dad9ccd36fe46018d07e2118199820523a6b35301d9c16, name=ceph-7f9750d6-044a-11ef-bc93-c7b262605968-osd-1-activate, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, RELEASE=main-4e213a5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.build-date=20240424, GIT_CLEAN=True) 2024-04-27T04:16:15.387 INFO:journalctl@ceph.osd.1.smithi162.stdout:Apr 27 04:16:15 smithi162 podman[126796]: 2024-04-27 04:16:15.180972207 +0000 UTC m=+0.454696016 container start 613936c9f1cbaf4954bb98f77a0f82b73afe9094474c518a12304ff03a2565e9 (image=quay.ceph.io/ceph-ci/ceph@sha256:90a3a9fa9ceb8977f6dad9ccd36fe46018d07e2118199820523a6b35301d9c16, name=ceph-7f9750d6-044a-11ef-bc93-c7b262605968-osd-1-activate, ceph=True, org.label-schema.build-date=20240424, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, RELEASE=main-4e213a5, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD) 2024-04-27T04:16:15.387 INFO:journalctl@ceph.osd.1.smithi162.stdout:Apr 27 04:16:15 smithi162 podman[126796]: 2024-04-27 04:16:15.261026338 +0000 UTC m=+0.534750142 container attach 613936c9f1cbaf4954bb98f77a0f82b73afe9094474c518a12304ff03a2565e9 (image=quay.ceph.io/ceph-ci/ceph@sha256:90a3a9fa9ceb8977f6dad9ccd36fe46018d07e2118199820523a6b35301d9c16, name=ceph-7f9750d6-044a-11ef-bc93-c7b262605968-osd-1-activate, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD, org.label-schema.build-date=20240424, RELEASE=main-4e213a5, ceph=True, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2) 2024-04-27T04:16:15.432 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:16:15.433+0000 7fce844d0700 1 -- 172.21.15.162:0/2710272664 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7fce7c10efc0 con 0x7fce7c103780 2024-04-27T04:16:15.432 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:16:15.434+0000 7fce72ffd700 1 -- 172.21.15.162:0/2710272664 <== mon.0 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 15 v15) v1 ==== 76+0+1911 (secure 0 0 0) 0x7fce6c05aa20 con 0x7fce7c103780 2024-04-27T04:16:15.432 INFO:teuthology.orchestra.run.smithi162.stdout:e15 2024-04-27T04:16:15.432 INFO:teuthology.orchestra.run.smithi162.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-04-27T04:16:15.433 INFO:teuthology.orchestra.run.smithi162.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-27T04:16:15.433 INFO:teuthology.orchestra.run.smithi162.stdout:legacy client fscid: 1 2024-04-27T04:16:15.433 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-04-27T04:16:15.433 INFO:teuthology.orchestra.run.smithi162.stdout:Filesystem 'cephfs' (1) 2024-04-27T04:16:15.433 INFO:teuthology.orchestra.run.smithi162.stdout:fs_name cephfs 2024-04-27T04:16:15.433 INFO:teuthology.orchestra.run.smithi162.stdout:epoch 15 2024-04-27T04:16:15.433 INFO:teuthology.orchestra.run.smithi162.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-04-27T04:16:15.433 INFO:teuthology.orchestra.run.smithi162.stdout:created 2024-04-27T04:06:53.095749+0000 2024-04-27T04:16:15.433 INFO:teuthology.orchestra.run.smithi162.stdout:modified 2024-04-27T04:07:11.285033+0000 2024-04-27T04:16:15.433 INFO:teuthology.orchestra.run.smithi162.stdout:tableserver 0 2024-04-27T04:16:15.433 INFO:teuthology.orchestra.run.smithi162.stdout:root 0 2024-04-27T04:16:15.433 INFO:teuthology.orchestra.run.smithi162.stdout:session_timeout 60 2024-04-27T04:16:15.433 INFO:teuthology.orchestra.run.smithi162.stdout:session_autoclose 300 2024-04-27T04:16:15.433 INFO:teuthology.orchestra.run.smithi162.stdout:max_file_size 1099511627776 2024-04-27T04:16:15.433 INFO:teuthology.orchestra.run.smithi162.stdout:max_xattr_size 65536 2024-04-27T04:16:15.433 INFO:teuthology.orchestra.run.smithi162.stdout:required_client_features {} 2024-04-27T04:16:15.434 INFO:teuthology.orchestra.run.smithi162.stdout:last_failure 0 2024-04-27T04:16:15.434 INFO:teuthology.orchestra.run.smithi162.stdout:last_failure_osd_epoch 46 2024-04-27T04:16:15.434 INFO:teuthology.orchestra.run.smithi162.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-27T04:16:15.434 INFO:teuthology.orchestra.run.smithi162.stdout:max_mds 1 2024-04-27T04:16:15.434 INFO:teuthology.orchestra.run.smithi162.stdout:in 0 2024-04-27T04:16:15.434 INFO:teuthology.orchestra.run.smithi162.stdout:up {0=24273} 2024-04-27T04:16:15.434 INFO:teuthology.orchestra.run.smithi162.stdout:failed 2024-04-27T04:16:15.434 INFO:teuthology.orchestra.run.smithi162.stdout:damaged 2024-04-27T04:16:15.434 INFO:teuthology.orchestra.run.smithi162.stdout:stopped 2024-04-27T04:16:15.434 INFO:teuthology.orchestra.run.smithi162.stdout:data_pools [3] 2024-04-27T04:16:15.434 INFO:teuthology.orchestra.run.smithi162.stdout:metadata_pool 2 2024-04-27T04:16:15.434 INFO:teuthology.orchestra.run.smithi162.stdout:inline_data disabled 2024-04-27T04:16:15.434 INFO:teuthology.orchestra.run.smithi162.stdout:balancer 2024-04-27T04:16:15.434 INFO:teuthology.orchestra.run.smithi162.stdout:bal_rank_mask -1 2024-04-27T04:16:15.435 INFO:teuthology.orchestra.run.smithi162.stdout:standby_count_wanted 1 2024-04-27T04:16:15.435 INFO:teuthology.orchestra.run.smithi162.stdout:qdb_cluster leader: 0 members: 2024-04-27T04:16:15.435 INFO:teuthology.orchestra.run.smithi162.stdout:[mds.cephfs.smithi181.dzomgh{0:24273} state up:active seq 6 join_fscid=1 addr [v2:172.21.15.181:6824/658214993,v1:172.21.15.181:6825/658214993] compat {c=[1],r=[1],i=[7ff]}] 2024-04-27T04:16:15.435 INFO:teuthology.orchestra.run.smithi162.stdout:[mds.cephfs.smithi162.xcnapk{0:14476} state up:standby-replay seq 2 join_fscid=1 addr [v2:172.21.15.162:6828/1296160676,v1:172.21.15.162:6829/1296160676] compat {c=[1],r=[1],i=[7ff]}] 2024-04-27T04:16:15.435 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-04-27T04:16:15.435 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-04-27T04:16:15.435 INFO:teuthology.orchestra.run.smithi162.stdout:Standby daemons: 2024-04-27T04:16:15.435 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-04-27T04:16:15.435 INFO:teuthology.orchestra.run.smithi162.stdout:[mds.cephfs.smithi162.gzlyjj{-1:14488} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.162:6826/2629103894,v1:172.21.15.162:6827/2629103894] compat {c=[1],r=[1],i=[7ff]}] 2024-04-27T04:16:15.435 INFO:teuthology.orchestra.run.smithi162.stdout:[mds.cephfs.smithi181.bkkcdm{-1:24287} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.181:6826/3756475453,v1:172.21.15.181:6827/3756475453] compat {c=[1],r=[1],i=[7ff]}] 2024-04-27T04:16:15.436 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:16:15.436+0000 7fce844d0700 1 -- 172.21.15.162:0/2710272664 >> [v2:172.21.15.162:6800/1396866108,v1:172.21.15.162:6801/1396866108] conn(0x7fce6806ec00 msgr2=0x7fce680710b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:16:15.436 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:16:15.436+0000 7fce844d0700 1 --2- 172.21.15.162:0/2710272664 >> [v2:172.21.15.162:6800/1396866108,v1:172.21.15.162:6801/1396866108] conn(0x7fce6806ec00 0x7fce680710b0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fce78004f90 tx=0x7fce78009e30 comp rx=0 tx=0).stop 2024-04-27T04:16:15.436 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:16:15.436+0000 7fce844d0700 1 -- 172.21.15.162:0/2710272664 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fce7c103780 msgr2=0x7fce7c113eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:16:15.436 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:16:15.436+0000 7fce844d0700 1 --2- 172.21.15.162:0/2710272664 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fce7c103780 0x7fce7c113eb0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fce6c002a50 tx=0x7fce6c004600 comp rx=0 tx=0).stop 2024-04-27T04:16:15.436 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:16:15.436+0000 7fce844d0700 1 -- 172.21.15.162:0/2710272664 shutdown_connections 2024-04-27T04:16:15.436 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:16:15.436+0000 7fce844d0700 1 --2- 172.21.15.162:0/2710272664 >> [v2:172.21.15.162:6800/1396866108,v1:172.21.15.162:6801/1396866108] conn(0x7fce6806ec00 0x7fce680710b0 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:16:15.436 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:16:15.436+0000 7fce844d0700 1 --2- 172.21.15.162:0/2710272664 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fce7c102de0 0x7fce7c113970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:16:15.436 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:16:15.436+0000 7fce844d0700 1 --2- 172.21.15.162:0/2710272664 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fce7c103780 0x7fce7c113eb0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:16:15.436 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:16:15.436+0000 7fce844d0700 1 -- 172.21.15.162:0/2710272664 >> 172.21.15.162:0/2710272664 conn(0x7fce7c0f8990 msgr2=0x7fce7c107380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:16:15.436 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:16:15.436+0000 7fce844d0700 1 -- 172.21.15.162:0/2710272664 shutdown_connections 2024-04-27T04:16:15.437 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:16:15.436+0000 7fce844d0700 1 -- 172.21.15.162:0/2710272664 wait complete. 2024-04-27T04:16:15.437 INFO:teuthology.orchestra.run.smithi162.stderr:dumped fsmap epoch 15 2024-04-27T04:16:15.544 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:16:15.546+0000 7ff732347700 1 -- 172.21.15.162:0/1777994321 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff72c1037a0 msgr2=0x7ff72c0fce10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:16:15.544 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:16:15.546+0000 7ff732347700 1 --2- 172.21.15.162:0/1777994321 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff72c1037a0 0x7ff72c0fce10 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7ff720004940 tx=0x7ff720009e30 comp rx=0 tx=0).stop 2024-04-27T04:16:15.544 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:16:15.546+0000 7ff732347700 1 -- 172.21.15.162:0/1777994321 shutdown_connections 2024-04-27T04:16:15.544 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:16:15.546+0000 7ff732347700 1 --2- 172.21.15.162:0/1777994321 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff72c1037a0 0x7ff72c0fce10 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:16:15.544 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:16:15.546+0000 7ff732347700 1 --2- 172.21.15.162:0/1777994321 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff72c102e00 0x7ff72c1031d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:16:15.544 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:16:15.546+0000 7ff732347700 1 -- 172.21.15.162:0/1777994321 >> 172.21.15.162:0/1777994321 conn(0x7ff72c0f89b0 msgr2=0x7ff72c0fadc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:16:15.544 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:16:15.546+0000 7ff732347700 1 -- 172.21.15.162:0/1777994321 shutdown_connections 2024-04-27T04:16:15.545 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:16:15.546+0000 7ff732347700 1 -- 172.21.15.162:0/1777994321 wait complete. 2024-04-27T04:16:15.545 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:16:15.546+0000 7ff732347700 1 Processor -- start 2024-04-27T04:16:15.546 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:16:15.547+0000 7ff732347700 1 -- start start 2024-04-27T04:16:15.546 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:16:15.547+0000 7ff732347700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff72c102e00 0x7ff72c113b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:16:15.547 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:16:15.547+0000 7ff732347700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff72c1037a0 0x7ff72c114080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:16:15.547 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:16:15.547+0000 7ff732347700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff72c114760 con 0x7ff72c1037a0 2024-04-27T04:16:15.547 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:16:15.547+0000 7ff732347700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff72c10dac0 con 0x7ff72c102e00 2024-04-27T04:16:15.547 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:16:15.547+0000 7ff72b7fe700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff72c1037a0 0x7ff72c114080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:16:15.547 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:16:15.547+0000 7ff72b7fe700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff72c1037a0 0x7ff72c114080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:49658/0 (socket says 172.21.15.162:49658) 2024-04-27T04:16:15.547 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:16:15.547+0000 7ff72b7fe700 1 -- 172.21.15.162:0/2580566294 learned_addr learned my addr 172.21.15.162:0/2580566294 (peer_addr_for_me v2:172.21.15.162:0/0) 2024-04-27T04:16:15.547 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:16:15.547+0000 7ff72bfff700 1 --2- 172.21.15.162:0/2580566294 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff72c102e00 0x7ff72c113b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:16:15.547 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:16:15.547+0000 7ff72b7fe700 1 -- 172.21.15.162:0/2580566294 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff72c102e00 msgr2=0x7ff72c113b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:16:15.547 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:16:15.547+0000 7ff72b7fe700 1 --2- 172.21.15.162:0/2580566294 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff72c102e00 0x7ff72c113b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:16:15.547 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:16:15.547+0000 7ff72b7fe700 1 -- 172.21.15.162:0/2580566294 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff720005040 con 0x7ff72c1037a0 2024-04-27T04:16:15.547 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:16:15.547+0000 7ff72b7fe700 1 --2- 172.21.15.162:0/2580566294 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff72c1037a0 0x7ff72c114080 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7ff720009100 tx=0x7ff720009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:16:15.547 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:16:15.547+0000 7ff728ff9700 1 -- 172.21.15.162:0/2580566294 <== mon.0 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff72000e040 con 0x7ff72c1037a0 2024-04-27T04:16:15.547 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:16:15.548+0000 7ff732347700 1 -- 172.21.15.162:0/2580566294 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff72c10dd40 con 0x7ff72c1037a0 2024-04-27T04:16:15.547 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:16:15.548+0000 7ff728ff9700 1 -- 172.21.15.162:0/2580566294 <== mon.0 v2:172.21.15.162:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7ff720009af0 con 0x7ff72c1037a0 2024-04-27T04:16:15.548 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:16:15.548+0000 7ff728ff9700 1 -- 172.21.15.162:0/2580566294 <== mon.0 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff72001c3f0 con 0x7ff72c1037a0 2024-04-27T04:16:15.548 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:16:15.548+0000 7ff732347700 1 -- 172.21.15.162:0/2580566294 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff72c10e300 con 0x7ff72c1037a0 2024-04-27T04:16:15.548 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:16:15.549+0000 7ff732347700 1 -- 172.21.15.162:0/2580566294 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff72c100110 con 0x7ff72c1037a0 2024-04-27T04:16:15.548 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:16:15.549+0000 7ff728ff9700 1 -- 172.21.15.162:0/2580566294 <== mon.0 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 39) v1 ==== 92747+0+0 (secure 0 0 0) 0x7ff72000baf0 con 0x7ff72c1037a0 2024-04-27T04:16:15.548 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:16:15.550+0000 7ff728ff9700 1 --2- 172.21.15.162:0/2580566294 >> [v2:172.21.15.162:6800/1396866108,v1:172.21.15.162:6801/1396866108] conn(0x7ff71406ec00 0x7ff7140710b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:16:15.549 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:16:15.550+0000 7ff728ff9700 1 -- 172.21.15.162:0/2580566294 <== mon.0 v2:172.21.15.162:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 6410+0+0 (secure 0 0 0) 0x7ff72008f9a0 con 0x7ff72c1037a0 2024-04-27T04:16:15.549 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:16:15.550+0000 7ff72bfff700 1 --2- 172.21.15.162:0/2580566294 >> [v2:172.21.15.162:6800/1396866108,v1:172.21.15.162:6801/1396866108] conn(0x7ff71406ec00 0x7ff7140710b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:16:15.549 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:16:15.550+0000 7ff72bfff700 1 --2- 172.21.15.162:0/2580566294 >> [v2:172.21.15.162:6800/1396866108,v1:172.21.15.162:6801/1396866108] conn(0x7ff71406ec00 0x7ff7140710b0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7ff71c009b60 tx=0x7ff71c0091a0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:16:15.552 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:16:15.554+0000 7ff728ff9700 1 -- 172.21.15.162:0/2580566294 <== mon.0 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7ff72005c3f0 con 0x7ff72c1037a0 2024-04-27T04:16:15.697 INFO:journalctl@ceph.osd.1.smithi162.stdout:Apr 27 04:16:15 smithi162 ceph-7f9750d6-044a-11ef-bc93-c7b262605968-osd-1-activate[126830]: Running command: /usr/bin/ceph-authtool --gen-print-key 2024-04-27T04:16:15.697 INFO:journalctl@ceph.osd.1.smithi162.stdout:Apr 27 04:16:15 smithi162 ceph-7f9750d6-044a-11ef-bc93-c7b262605968-osd-1-activate[126830]: Running command: /usr/bin/ceph-authtool --gen-print-key 2024-04-27T04:16:15.697 INFO:journalctl@ceph.osd.1.smithi162.stdout:Apr 27 04:16:15 smithi162 bash[126796]: Running command: /usr/bin/ceph-authtool --gen-print-key 2024-04-27T04:16:15.697 INFO:journalctl@ceph.osd.1.smithi162.stdout:Apr 27 04:16:15 smithi162 bash[126796]: Running command: /usr/bin/ceph-authtool --gen-print-key 2024-04-27T04:16:15.697 INFO:journalctl@ceph.osd.1.smithi162.stdout:Apr 27 04:16:15 smithi162 bash[126796]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2024-04-27T04:16:15.697 INFO:journalctl@ceph.osd.1.smithi162.stdout:Apr 27 04:16:15 smithi162 bash[126796]: Running command: /usr/bin/ceph-bluestore-tool prime-osd-dir --path /var/lib/ceph/osd/ceph-1 --no-mon-config --dev /dev/mapper/vg_nvme-lv_3 2024-04-27T04:16:15.697 INFO:journalctl@ceph.osd.1.smithi162.stdout:Apr 27 04:16:15 smithi162 bash[126796]: Running command: /usr/bin/chown -h ceph:ceph /dev/mapper/vg_nvme-lv_3 2024-04-27T04:16:15.697 INFO:journalctl@ceph.osd.1.smithi162.stdout:Apr 27 04:16:15 smithi162 bash[126796]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-2 2024-04-27T04:16:15.697 INFO:journalctl@ceph.osd.1.smithi162.stdout:Apr 27 04:16:15 smithi162 bash[126796]: Running command: /usr/bin/ln -s /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-1/block 2024-04-27T04:16:15.697 INFO:journalctl@ceph.osd.1.smithi162.stdout:Apr 27 04:16:15 smithi162 bash[126796]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2024-04-27T04:16:15.697 INFO:journalctl@ceph.osd.1.smithi162.stdout:Apr 27 04:16:15 smithi162 bash[126796]: --> ceph-volume raw activate successful for osd ID: 1 2024-04-27T04:16:15.698 INFO:journalctl@ceph.osd.1.smithi162.stdout:Apr 27 04:16:15 smithi162 ceph-7f9750d6-044a-11ef-bc93-c7b262605968-osd-1-activate[126830]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2024-04-27T04:16:15.698 INFO:journalctl@ceph.osd.1.smithi162.stdout:Apr 27 04:16:15 smithi162 ceph-7f9750d6-044a-11ef-bc93-c7b262605968-osd-1-activate[126830]: Running command: /usr/bin/ceph-bluestore-tool prime-osd-dir --path /var/lib/ceph/osd/ceph-1 --no-mon-config --dev /dev/mapper/vg_nvme-lv_3 2024-04-27T04:16:15.698 INFO:journalctl@ceph.osd.1.smithi162.stdout:Apr 27 04:16:15 smithi162 ceph-7f9750d6-044a-11ef-bc93-c7b262605968-osd-1-activate[126830]: Running command: /usr/bin/chown -h ceph:ceph /dev/mapper/vg_nvme-lv_3 2024-04-27T04:16:15.698 INFO:journalctl@ceph.osd.1.smithi162.stdout:Apr 27 04:16:15 smithi162 ceph-7f9750d6-044a-11ef-bc93-c7b262605968-osd-1-activate[126830]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-2 2024-04-27T04:16:15.698 INFO:journalctl@ceph.osd.1.smithi162.stdout:Apr 27 04:16:15 smithi162 ceph-7f9750d6-044a-11ef-bc93-c7b262605968-osd-1-activate[126830]: Running command: /usr/bin/ln -s /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-1/block 2024-04-27T04:16:15.698 INFO:journalctl@ceph.osd.1.smithi162.stdout:Apr 27 04:16:15 smithi162 ceph-7f9750d6-044a-11ef-bc93-c7b262605968-osd-1-activate[126830]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2024-04-27T04:16:15.698 INFO:journalctl@ceph.osd.1.smithi162.stdout:Apr 27 04:16:15 smithi162 ceph-7f9750d6-044a-11ef-bc93-c7b262605968-osd-1-activate[126830]: --> ceph-volume raw activate successful for osd ID: 1 2024-04-27T04:16:15.788 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:16:15.790+0000 7ff732347700 1 -- 172.21.15.162:0/2580566294 --> [v2:172.21.15.162:6800/1396866108,v1:172.21.15.162:6801/1396866108] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7ff72c10f190 con 0x7ff71406ec00 2024-04-27T04:16:15.789 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:16:15.792+0000 7ff728ff9700 1 -- 172.21.15.162:0/2580566294 <== mgr.34104 v2:172.21.15.162:6800/1396866108 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+400 (secure 0 0 0) 0x7ff72c10f190 con 0x7ff71406ec00 2024-04-27T04:16:15.790 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-04-27T04:16:15.790 INFO:teuthology.orchestra.run.smithi162.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:90a3a9fa9ceb8977f6dad9ccd36fe46018d07e2118199820523a6b35301d9c16", 2024-04-27T04:16:15.790 INFO:teuthology.orchestra.run.smithi162.stdout: "in_progress": true, 2024-04-27T04:16:15.790 INFO:teuthology.orchestra.run.smithi162.stdout: "which": "Upgrading all daemon types on all hosts", 2024-04-27T04:16:15.790 INFO:teuthology.orchestra.run.smithi162.stdout: "services_complete": [ 2024-04-27T04:16:15.790 INFO:teuthology.orchestra.run.smithi162.stdout: "crash", 2024-04-27T04:16:15.790 INFO:teuthology.orchestra.run.smithi162.stdout: "mon", 2024-04-27T04:16:15.790 INFO:teuthology.orchestra.run.smithi162.stdout: "mgr" 2024-04-27T04:16:15.790 INFO:teuthology.orchestra.run.smithi162.stdout: ], 2024-04-27T04:16:15.790 INFO:teuthology.orchestra.run.smithi162.stdout: "progress": "7/21 daemons upgraded", 2024-04-27T04:16:15.790 INFO:teuthology.orchestra.run.smithi162.stdout: "message": "Currently upgrading osd daemons", 2024-04-27T04:16:15.790 INFO:teuthology.orchestra.run.smithi162.stdout: "is_paused": false 2024-04-27T04:16:15.791 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-04-27T04:16:15.792 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:16:15.794+0000 7ff732347700 1 -- 172.21.15.162:0/2580566294 >> [v2:172.21.15.162:6800/1396866108,v1:172.21.15.162:6801/1396866108] conn(0x7ff71406ec00 msgr2=0x7ff7140710b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:16:15.792 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:16:15.794+0000 7ff732347700 1 --2- 172.21.15.162:0/2580566294 >> [v2:172.21.15.162:6800/1396866108,v1:172.21.15.162:6801/1396866108] conn(0x7ff71406ec00 0x7ff7140710b0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7ff71c009b60 tx=0x7ff71c0091a0 comp rx=0 tx=0).stop 2024-04-27T04:16:15.792 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:16:15.794+0000 7ff732347700 1 -- 172.21.15.162:0/2580566294 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff72c1037a0 msgr2=0x7ff72c114080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:16:15.792 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:16:15.794+0000 7ff732347700 1 --2- 172.21.15.162:0/2580566294 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff72c1037a0 0x7ff72c114080 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7ff720009100 tx=0x7ff720009e30 comp rx=0 tx=0).stop 2024-04-27T04:16:15.792 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:16:15.794+0000 7ff732347700 1 -- 172.21.15.162:0/2580566294 shutdown_connections 2024-04-27T04:16:15.792 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:16:15.794+0000 7ff732347700 1 --2- 172.21.15.162:0/2580566294 >> [v2:172.21.15.162:6800/1396866108,v1:172.21.15.162:6801/1396866108] conn(0x7ff71406ec00 0x7ff7140710b0 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:16:15.792 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:16:15.794+0000 7ff732347700 1 --2- 172.21.15.162:0/2580566294 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff72c102e00 0x7ff72c113b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:16:15.793 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:16:15.794+0000 7ff732347700 1 --2- 172.21.15.162:0/2580566294 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff72c1037a0 0x7ff72c114080 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:16:15.793 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:16:15.794+0000 7ff732347700 1 -- 172.21.15.162:0/2580566294 >> 172.21.15.162:0/2580566294 conn(0x7ff72c0f89b0 msgr2=0x7ff72c0fa6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:16:15.793 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:16:15.794+0000 7ff732347700 1 -- 172.21.15.162:0/2580566294 shutdown_connections 2024-04-27T04:16:15.793 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:16:15.794+0000 7ff732347700 1 -- 172.21.15.162:0/2580566294 wait complete. 2024-04-27T04:16:15.896 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:16:15.896+0000 7f1c1b8fe700 1 -- 172.21.15.162:0/929158979 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1c14101d90 msgr2=0x7f1c14102160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:16:15.896 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:16:15.896+0000 7f1c1b8fe700 1 --2- 172.21.15.162:0/929158979 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1c14101d90 0x7f1c14102160 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f1c10004940 tx=0x7f1c10009e30 comp rx=0 tx=0).stop 2024-04-27T04:16:15.896 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:16:15.896+0000 7f1c1b8fe700 1 -- 172.21.15.162:0/929158979 shutdown_connections 2024-04-27T04:16:15.896 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:16:15.896+0000 7f1c1b8fe700 1 --2- 172.21.15.162:0/929158979 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1c140fbd40 0x7f1c140fc1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:16:15.896 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:16:15.896+0000 7f1c1b8fe700 1 --2- 172.21.15.162:0/929158979 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1c14101d90 0x7f1c14102160 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:16:15.896 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:16:15.896+0000 7f1c1b8fe700 1 -- 172.21.15.162:0/929158979 >> 172.21.15.162:0/929158979 conn(0x7f1c14071d00 msgr2=0x7f1c14072100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:16:15.896 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:16:15.896+0000 7f1c1b8fe700 1 -- 172.21.15.162:0/929158979 shutdown_connections 2024-04-27T04:16:15.896 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:16:15.896+0000 7f1c1b8fe700 1 -- 172.21.15.162:0/929158979 wait complete. 2024-04-27T04:16:15.896 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:16:15.897+0000 7f1c1b8fe700 1 Processor -- start 2024-04-27T04:16:15.896 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:16:15.897+0000 7f1c1b8fe700 1 -- start start 2024-04-27T04:16:15.896 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:16:15.897+0000 7f1c1b8fe700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1c140fbd40 0x7f1c14112aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:16:15.897 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:16:15.897+0000 7f1c1b8fe700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1c14101d90 0x7f1c14112fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:16:15.897 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:16:15.897+0000 7f1c1b8fe700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c14113630 con 0x7f1c14101d90 2024-04-27T04:16:15.897 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:16:15.897+0000 7f1c1b8fe700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c1410ca20 con 0x7f1c140fbd40 2024-04-27T04:16:15.897 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:16:15.897+0000 7f1c18e99700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1c14101d90 0x7f1c14112fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:16:15.897 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:16:15.897+0000 7f1c18e99700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1c14101d90 0x7f1c14112fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:49678/0 (socket says 172.21.15.162:49678) 2024-04-27T04:16:15.897 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:16:15.897+0000 7f1c18e99700 1 -- 172.21.15.162:0/780875060 learned_addr learned my addr 172.21.15.162:0/780875060 (peer_addr_for_me v2:172.21.15.162:0/0) 2024-04-27T04:16:15.897 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:16:15.897+0000 7f1c1969a700 1 --2- 172.21.15.162:0/780875060 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1c140fbd40 0x7f1c14112aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:16:15.897 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:16:15.897+0000 7f1c18e99700 1 -- 172.21.15.162:0/780875060 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1c140fbd40 msgr2=0x7f1c14112aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:16:15.897 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:16:15.897+0000 7f1c18e99700 1 --2- 172.21.15.162:0/780875060 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1c140fbd40 0x7f1c14112aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:16:15.897 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:16:15.897+0000 7f1c18e99700 1 -- 172.21.15.162:0/780875060 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c10005040 con 0x7f1c14101d90 2024-04-27T04:16:15.897 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:16:15.898+0000 7f1c18e99700 1 --2- 172.21.15.162:0/780875060 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1c14101d90 0x7f1c14112fe0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f1c04002a50 tx=0x7f1c040046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:16:15.897 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:16:15.898+0000 7f1c09ffb700 1 -- 172.21.15.162:0/780875060 <== mon.0 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1c04007cc0 con 0x7f1c14101d90 2024-04-27T04:16:15.897 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:16:15.898+0000 7f1c09ffb700 1 -- 172.21.15.162:0/780875060 <== mon.0 v2:172.21.15.162:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f1c0400f040 con 0x7f1c14101d90 2024-04-27T04:16:15.897 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:16:15.898+0000 7f1c09ffb700 1 -- 172.21.15.162:0/780875060 <== mon.0 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1c0400da30 con 0x7f1c14101d90 2024-04-27T04:16:15.897 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:16:15.898+0000 7f1c1b8fe700 1 -- 172.21.15.162:0/780875060 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1c1410cd00 con 0x7f1c14101d90 2024-04-27T04:16:15.898 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:16:15.898+0000 7f1c1b8fe700 1 -- 172.21.15.162:0/780875060 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1c1410d2c0 con 0x7f1c14101d90 2024-04-27T04:16:15.898 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:16:15.899+0000 7f1c1b8fe700 1 -- 172.21.15.162:0/780875060 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1c140fd430 con 0x7f1c14101d90 2024-04-27T04:16:15.899 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:16:15.899+0000 7f1c09ffb700 1 -- 172.21.15.162:0/780875060 <== mon.0 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 39) v1 ==== 92747+0+0 (secure 0 0 0) 0x7f1c0400db90 con 0x7f1c14101d90 2024-04-27T04:16:15.899 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:16:15.900+0000 7f1c09ffb700 1 --2- 172.21.15.162:0/780875060 >> [v2:172.21.15.162:6800/1396866108,v1:172.21.15.162:6801/1396866108] conn(0x7f1c0006ec00 0x7f1c000710b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:16:15.899 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:16:15.900+0000 7f1c09ffb700 1 -- 172.21.15.162:0/780875060 <== mon.0 v2:172.21.15.162:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 6410+0+0 (secure 0 0 0) 0x7f1c0408e5d0 con 0x7f1c14101d90 2024-04-27T04:16:15.899 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:16:15.900+0000 7f1c1969a700 1 --2- 172.21.15.162:0/780875060 >> [v2:172.21.15.162:6800/1396866108,v1:172.21.15.162:6801/1396866108] conn(0x7f1c0006ec00 0x7f1c000710b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:16:15.899 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:16:15.900+0000 7f1c1969a700 1 --2- 172.21.15.162:0/780875060 >> [v2:172.21.15.162:6800/1396866108,v1:172.21.15.162:6801/1396866108] conn(0x7f1c0006ec00 0x7f1c000710b0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f1c10004560 tx=0x7f1c10009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:16:15.902 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:16:15.905+0000 7f1c09ffb700 1 -- 172.21.15.162:0/780875060 <== mon.0 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7f1c0405af00 con 0x7f1c14101d90 2024-04-27T04:16:15.987 INFO:journalctl@ceph.osd.1.smithi162.stdout:Apr 27 04:16:15 smithi162 podman[127012]: 2024-04-27 04:16:15.730260251 +0000 UTC m=+0.019313361 container died 613936c9f1cbaf4954bb98f77a0f82b73afe9094474c518a12304ff03a2565e9 (image=quay.ceph.io/ceph-ci/ceph@sha256:90a3a9fa9ceb8977f6dad9ccd36fe46018d07e2118199820523a6b35301d9c16, name=ceph-7f9750d6-044a-11ef-bc93-c7b262605968-osd-1-activate, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, ceph=True, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, RELEASE=main-4e213a5, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.build-date=20240424, org.label-schema.schema-version=1.0) 2024-04-27T04:16:15.987 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:16:15 smithi162 ceph-mon[110641]: from='client.34220 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-27T04:16:15.987 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:16:15 smithi162 ceph-mon[110641]: from='client.34224 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-27T04:16:15.988 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:16:15 smithi162 ceph-mon[110641]: from='client.34228 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-27T04:16:15.988 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:16:15 smithi162 ceph-mon[110641]: from='client.? 172.21.15.162:0/3273604472' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-27T04:16:15.988 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:16:15 smithi162 ceph-mon[110641]: from='client.? 172.21.15.162:0/2710272664' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-27T04:16:16.172 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:16:15 smithi181 ceph-mon[81134]: from='client.34220 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-27T04:16:16.172 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:16:15 smithi181 ceph-mon[81134]: from='client.34224 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-27T04:16:16.172 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:16:15 smithi181 ceph-mon[81134]: from='client.34228 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-27T04:16:16.172 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:16:15 smithi181 ceph-mon[81134]: from='client.? 172.21.15.162:0/3273604472' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-27T04:16:16.172 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:16:15 smithi181 ceph-mon[81134]: from='client.? 172.21.15.162:0/2710272664' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-27T04:16:16.192 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:16:16.194+0000 7f1c1b8fe700 1 -- 172.21.15.162:0/780875060 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f1c14108df0 con 0x7f1c14101d90 2024-04-27T04:16:16.193 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:16:16.195+0000 7f1c09ffb700 1 -- 172.21.15.162:0/780875060 <== mon.0 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+881 (secure 0 0 0) 0x7f1c0405aa90 con 0x7f1c14101d90 2024-04-27T04:16:16.193 INFO:teuthology.orchestra.run.smithi162.stdout:HEALTH_WARN 1 osds down; Degraded data redundancy: 27/231 objects degraded (11.688%), 11 pgs degraded 2024-04-27T04:16:16.193 INFO:teuthology.orchestra.run.smithi162.stdout:[WRN] OSD_DOWN: 1 osds down 2024-04-27T04:16:16.193 INFO:teuthology.orchestra.run.smithi162.stdout: osd.1 (root=default,host=smithi162) is down 2024-04-27T04:16:16.193 INFO:teuthology.orchestra.run.smithi162.stdout:[WRN] PG_DEGRADED: Degraded data redundancy: 27/231 objects degraded (11.688%), 11 pgs degraded 2024-04-27T04:16:16.193 INFO:teuthology.orchestra.run.smithi162.stdout: pg 1.0 is active+undersized+degraded, acting [3,0] 2024-04-27T04:16:16.193 INFO:teuthology.orchestra.run.smithi162.stdout: pg 2.0 is active+undersized+degraded, acting [3,0] 2024-04-27T04:16:16.193 INFO:teuthology.orchestra.run.smithi162.stdout: pg 2.1 is active+undersized+degraded, acting [2,0] 2024-04-27T04:16:16.193 INFO:teuthology.orchestra.run.smithi162.stdout: pg 2.2 is active+undersized+degraded, acting [5,0] 2024-04-27T04:16:16.193 INFO:teuthology.orchestra.run.smithi162.stdout: pg 2.3 is active+undersized+degraded, acting [5,2] 2024-04-27T04:16:16.194 INFO:teuthology.orchestra.run.smithi162.stdout: pg 2.6 is active+undersized+degraded, acting [3,4] 2024-04-27T04:16:16.194 INFO:teuthology.orchestra.run.smithi162.stdout: pg 2.9 is active+undersized+degraded, acting [4,0] 2024-04-27T04:16:16.194 INFO:teuthology.orchestra.run.smithi162.stdout: pg 2.a is active+undersized+degraded, acting [4,3] 2024-04-27T04:16:16.194 INFO:teuthology.orchestra.run.smithi162.stdout: pg 2.d is active+undersized+degraded, acting [3,2] 2024-04-27T04:16:16.194 INFO:teuthology.orchestra.run.smithi162.stdout: pg 2.10 is active+undersized+degraded, acting [2,0] 2024-04-27T04:16:16.194 INFO:teuthology.orchestra.run.smithi162.stdout: pg 2.15 is active+undersized+degraded, acting [3,0] 2024-04-27T04:16:16.195 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:16:16.197+0000 7f1c1b8fe700 1 -- 172.21.15.162:0/780875060 >> [v2:172.21.15.162:6800/1396866108,v1:172.21.15.162:6801/1396866108] conn(0x7f1c0006ec00 msgr2=0x7f1c000710b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:16:16.195 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:16:16.197+0000 7f1c1b8fe700 1 --2- 172.21.15.162:0/780875060 >> [v2:172.21.15.162:6800/1396866108,v1:172.21.15.162:6801/1396866108] conn(0x7f1c0006ec00 0x7f1c000710b0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f1c10004560 tx=0x7f1c10009e30 comp rx=0 tx=0).stop 2024-04-27T04:16:16.195 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:16:16.197+0000 7f1c1b8fe700 1 -- 172.21.15.162:0/780875060 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1c14101d90 msgr2=0x7f1c14112fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:16:16.195 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:16:16.197+0000 7f1c1b8fe700 1 --2- 172.21.15.162:0/780875060 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1c14101d90 0x7f1c14112fe0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f1c04002a50 tx=0x7f1c040046b0 comp rx=0 tx=0).stop 2024-04-27T04:16:16.195 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:16:16.197+0000 7f1c1b8fe700 1 -- 172.21.15.162:0/780875060 shutdown_connections 2024-04-27T04:16:16.195 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:16:16.197+0000 7f1c1b8fe700 1 --2- 172.21.15.162:0/780875060 >> [v2:172.21.15.162:6800/1396866108,v1:172.21.15.162:6801/1396866108] conn(0x7f1c0006ec00 0x7f1c000710b0 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:16:16.195 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:16:16.197+0000 7f1c1b8fe700 1 --2- 172.21.15.162:0/780875060 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1c140fbd40 0x7f1c14112aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:16:16.196 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:16:16.197+0000 7f1c1b8fe700 1 --2- 172.21.15.162:0/780875060 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1c14101d90 0x7f1c14112fe0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:16:16.196 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:16:16.197+0000 7f1c1b8fe700 1 -- 172.21.15.162:0/780875060 >> 172.21.15.162:0/780875060 conn(0x7f1c14071d00 msgr2=0x7f1c140fa140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:16:16.196 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:16:16.197+0000 7f1c1b8fe700 1 -- 172.21.15.162:0/780875060 shutdown_connections 2024-04-27T04:16:16.196 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:16:16.197+0000 7f1c1b8fe700 1 -- 172.21.15.162:0/780875060 wait complete. 2024-04-27T04:16:16.637 INFO:journalctl@ceph.osd.1.smithi162.stdout:Apr 27 04:16:16 smithi162 podman[127012]: 2024-04-27 04:16:16.373853787 +0000 UTC m=+0.662906942 container remove 613936c9f1cbaf4954bb98f77a0f82b73afe9094474c518a12304ff03a2565e9 (image=quay.ceph.io/ceph-ci/ceph@sha256:90a3a9fa9ceb8977f6dad9ccd36fe46018d07e2118199820523a6b35301d9c16, name=ceph-7f9750d6-044a-11ef-bc93-c7b262605968-osd-1-activate, RELEASE=main-4e213a5, org.label-schema.license=GPLv2, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.build-date=20240424, CEPH_POINT_RELEASE=, ceph=True) 2024-04-27T04:16:16.637 INFO:journalctl@ceph.osd.1.smithi162.stdout:Apr 27 04:16:16 smithi162 podman[127104]: 2024-04-27 04:16:16.488453108 +0000 UTC m=+0.018034497 image pull 67e414ff4bc41092cb31c84c98a9563a32813d02583028679c46b40362afeb3e quay.ceph.io/ceph-ci/ceph@sha256:90a3a9fa9ceb8977f6dad9ccd36fe46018d07e2118199820523a6b35301d9c16 2024-04-27T04:16:16.637 INFO:journalctl@ceph.osd.1.smithi162.stdout:Apr 27 04:16:16 smithi162 podman[127104]: 2024-04-27 04:16:16.59982078 +0000 UTC m=+0.129402170 container create 668e16faad6233fa9895f18d0284aad8a73737d64d405561bb0da15b674ddbe7 (image=quay.ceph.io/ceph-ci/ceph@sha256:90a3a9fa9ceb8977f6dad9ccd36fe46018d07e2118199820523a6b35301d9c16, name=ceph-7f9750d6-044a-11ef-bc93-c7b262605968-osd-1, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, RELEASE=main-4e213a5, GIT_CLEAN=True, org.label-schema.build-date=20240424, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , ceph=True) 2024-04-27T04:16:17.137 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:16:16 smithi162 ceph-mon[110641]: pgmap v69: 65 pgs: 11 active+undersized+degraded, 3 stale+active+clean, 18 active+undersized, 33 active+clean; 208 MiB data, 2.1 GiB used, 534 GiB / 536 GiB avail; 27/231 objects degraded (11.688%) 2024-04-27T04:16:17.137 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:16:16 smithi162 ceph-mon[110641]: from='client.? 172.21.15.162:0/780875060' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-27T04:16:17.137 INFO:journalctl@ceph.osd.1.smithi162.stdout:Apr 27 04:16:16 smithi162 podman[127104]: 2024-04-27 04:16:16.812722579 +0000 UTC m=+0.342303970 container init 668e16faad6233fa9895f18d0284aad8a73737d64d405561bb0da15b674ddbe7 (image=quay.ceph.io/ceph-ci/ceph@sha256:90a3a9fa9ceb8977f6dad9ccd36fe46018d07e2118199820523a6b35301d9c16, name=ceph-7f9750d6-044a-11ef-bc93-c7b262605968-osd-1, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=main-4e213a5, GIT_BRANCH=HEAD, org.label-schema.build-date=20240424, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_CLEAN=True, io.buildah.version=1.33.5) 2024-04-27T04:16:17.138 INFO:journalctl@ceph.osd.1.smithi162.stdout:Apr 27 04:16:16 smithi162 podman[127104]: 2024-04-27 04:16:16.815840601 +0000 UTC m=+0.345421992 container start 668e16faad6233fa9895f18d0284aad8a73737d64d405561bb0da15b674ddbe7 (image=quay.ceph.io/ceph-ci/ceph@sha256:90a3a9fa9ceb8977f6dad9ccd36fe46018d07e2118199820523a6b35301d9c16, name=ceph-7f9750d6-044a-11ef-bc93-c7b262605968-osd-1, RELEASE=main-4e213a5, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.build-date=20240424) 2024-04-27T04:16:17.138 INFO:journalctl@ceph.osd.1.smithi162.stdout:Apr 27 04:16:16 smithi162 ceph-osd[127123]: -- 172.21.15.162:0/1878540833 <== mon.0 v2:172.21.15.162:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5623e6ea3680 con 0x5623e6e6d000 2024-04-27T04:16:17.138 INFO:journalctl@ceph.osd.1.smithi162.stdout:Apr 27 04:16:16 smithi162 bash[127104]: 668e16faad6233fa9895f18d0284aad8a73737d64d405561bb0da15b674ddbe7 2024-04-27T04:16:17.138 INFO:journalctl@ceph.osd.1.smithi162.stdout:Apr 27 04:16:16 smithi162 systemd[1]: Started Ceph osd.1 for 7f9750d6-044a-11ef-bc93-c7b262605968. 2024-04-27T04:16:17.172 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:16:16 smithi181 ceph-mon[81134]: pgmap v69: 65 pgs: 11 active+undersized+degraded, 3 stale+active+clean, 18 active+undersized, 33 active+clean; 208 MiB data, 2.1 GiB used, 534 GiB / 536 GiB avail; 27/231 objects degraded (11.688%) 2024-04-27T04:16:17.172 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:16:16 smithi181 ceph-mon[81134]: from='client.? 172.21.15.162:0/780875060' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-27T04:16:17.636 INFO:journalctl@ceph.osd.1.smithi162.stdout:Apr 27 04:16:17 smithi162 ceph-7f9750d6-044a-11ef-bc93-c7b262605968-osd-1[127119]: 2024-04-27T04:16:17.371+0000 7f2fa53a87c0 -1 Falling back to public interface 2024-04-27T04:16:18.136 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:16:17 smithi162 ceph-mon[110641]: from='client.34240 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-27T04:16:18.137 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:16:17 smithi162 ceph-mon[110641]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' 2024-04-27T04:16:18.137 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:16:17 smithi162 ceph-mon[110641]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' 2024-04-27T04:16:18.137 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:16:17 smithi162 ceph-mon[110641]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T04:16:18.137 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:16:17 smithi162 ceph-mon[110641]: pgmap v70: 65 pgs: 12 active+undersized+degraded, 22 active+undersized, 31 active+clean; 208 MiB data, 2.1 GiB used, 534 GiB / 536 GiB avail; 28/231 objects degraded (12.121%) 2024-04-27T04:16:18.137 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:16:17 smithi162 ceph-mon[110641]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' 2024-04-27T04:16:18.137 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:16:17 smithi162 ceph-mon[110641]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-27T04:16:18.172 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:16:17 smithi181 ceph-mon[81134]: from='client.34240 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-27T04:16:18.172 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:16:17 smithi181 ceph-mon[81134]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' 2024-04-27T04:16:18.172 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:16:17 smithi181 ceph-mon[81134]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' 2024-04-27T04:16:18.172 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:16:17 smithi181 ceph-mon[81134]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T04:16:18.172 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:16:17 smithi181 ceph-mon[81134]: pgmap v70: 65 pgs: 12 active+undersized+degraded, 22 active+undersized, 31 active+clean; 208 MiB data, 2.1 GiB used, 534 GiB / 536 GiB avail; 28/231 objects degraded (12.121%) 2024-04-27T04:16:18.172 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:16:17 smithi181 ceph-mon[81134]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' 2024-04-27T04:16:18.172 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:16:17 smithi181 ceph-mon[81134]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-27T04:16:19.636 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:16:19 smithi162 ceph-mon[110641]: Health check update: Degraded data redundancy: 28/231 objects degraded (12.121%), 12 pgs degraded (PG_DEGRADED) 2024-04-27T04:16:19.672 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:16:19 smithi181 ceph-mon[81134]: Health check update: Degraded data redundancy: 28/231 objects degraded (12.121%), 12 pgs degraded (PG_DEGRADED) 2024-04-27T04:16:20.636 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:16:20 smithi162 ceph-mon[110641]: pgmap v71: 65 pgs: 12 active+undersized+degraded, 22 active+undersized, 31 active+clean; 208 MiB data, 2.1 GiB used, 534 GiB / 536 GiB avail; 28/231 objects degraded (12.121%) 2024-04-27T04:16:20.672 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:16:20 smithi181 ceph-mon[81134]: pgmap v71: 65 pgs: 12 active+undersized+degraded, 22 active+undersized, 31 active+clean; 208 MiB data, 2.1 GiB used, 534 GiB / 536 GiB avail; 28/231 objects degraded (12.121%) 2024-04-27T04:16:22.886 INFO:journalctl@ceph.osd.1.smithi162.stdout:Apr 27 04:16:22 smithi162 ceph-7f9750d6-044a-11ef-bc93-c7b262605968-osd-1[127119]: 2024-04-27T04:16:22.456+0000 7f2fa53a87c0 -1 osd.1 54 log_to_monitors true 2024-04-27T04:16:22.887 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:16:22 smithi162 ceph-mon[110641]: pgmap v72: 65 pgs: 12 active+undersized+degraded, 22 active+undersized, 31 active+clean; 208 MiB data, 2.1 GiB used, 534 GiB / 536 GiB avail; 28/231 objects degraded (12.121%) 2024-04-27T04:16:22.887 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:16:22 smithi162 ceph-mon[110641]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' 2024-04-27T04:16:22.887 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:16:22 smithi162 ceph-mon[110641]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' 2024-04-27T04:16:22.887 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:16:22 smithi162 ceph-mon[110641]: from='osd.1 [v2:172.21.15.162:6810/2144127279,v1:172.21.15.162:6811/2144127279]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-04-27T04:16:23.172 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:16:22 smithi181 ceph-mon[81134]: pgmap v72: 65 pgs: 12 active+undersized+degraded, 22 active+undersized, 31 active+clean; 208 MiB data, 2.1 GiB used, 534 GiB / 536 GiB avail; 28/231 objects degraded (12.121%) 2024-04-27T04:16:23.172 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:16:22 smithi181 ceph-mon[81134]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' 2024-04-27T04:16:23.172 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:16:22 smithi181 ceph-mon[81134]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' 2024-04-27T04:16:23.172 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:16:22 smithi181 ceph-mon[81134]: from='osd.1 [v2:172.21.15.162:6810/2144127279,v1:172.21.15.162:6811/2144127279]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-04-27T04:16:23.636 INFO:journalctl@ceph.osd.1.smithi162.stdout:Apr 27 04:16:23 smithi162 ceph-7f9750d6-044a-11ef-bc93-c7b262605968-osd-1[127119]: 2024-04-27T04:16:23.254+0000 7f2f96800700 -1 osd.1 54 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-27T04:16:24.636 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:16:24 smithi162 ceph-mon[110641]: from='osd.1 [v2:172.21.15.162:6810/2144127279,v1:172.21.15.162:6811/2144127279]' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-04-27T04:16:24.637 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:16:24 smithi162 ceph-mon[110641]: osdmap e57: 6 total, 5 up, 6 in 2024-04-27T04:16:24.637 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:16:24 smithi162 ceph-mon[110641]: from='osd.1 [v2:172.21.15.162:6810/2144127279,v1:172.21.15.162:6811/2144127279]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi162", "root=default"]}]: dispatch 2024-04-27T04:16:24.637 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:16:24 smithi162 ceph-mon[110641]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' 2024-04-27T04:16:24.637 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:16:24 smithi162 ceph-mon[110641]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' 2024-04-27T04:16:24.637 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:16:24 smithi162 ceph-mon[110641]: pgmap v74: 65 pgs: 12 active+undersized+degraded, 22 active+undersized, 31 active+clean; 208 MiB data, 1.8 GiB used, 535 GiB / 536 GiB avail; 1023 B/s rd, 1 op/s; 28/231 objects degraded (12.121%) 2024-04-27T04:16:24.672 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:16:24 smithi181 ceph-mon[81134]: from='osd.1 [v2:172.21.15.162:6810/2144127279,v1:172.21.15.162:6811/2144127279]' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-04-27T04:16:24.672 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:16:24 smithi181 ceph-mon[81134]: osdmap e57: 6 total, 5 up, 6 in 2024-04-27T04:16:24.672 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:16:24 smithi181 ceph-mon[81134]: from='osd.1 [v2:172.21.15.162:6810/2144127279,v1:172.21.15.162:6811/2144127279]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi162", "root=default"]}]: dispatch 2024-04-27T04:16:24.672 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:16:24 smithi181 ceph-mon[81134]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' 2024-04-27T04:16:24.672 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:16:24 smithi181 ceph-mon[81134]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' 2024-04-27T04:16:24.672 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:16:24 smithi181 ceph-mon[81134]: pgmap v74: 65 pgs: 12 active+undersized+degraded, 22 active+undersized, 31 active+clean; 208 MiB data, 1.8 GiB used, 535 GiB / 536 GiB avail; 1023 B/s rd, 1 op/s; 28/231 objects degraded (12.121%) 2024-04-27T04:16:25.535 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:16:25 smithi162 ceph-mon[110641]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-04-27T04:16:25.535 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:16:25 smithi162 ceph-mon[110641]: osd.1 [v2:172.21.15.162:6810/2144127279,v1:172.21.15.162:6811/2144127279] boot 2024-04-27T04:16:25.535 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:16:25 smithi162 ceph-mon[110641]: osdmap e58: 6 total, 6 up, 6 in 2024-04-27T04:16:25.536 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:16:25 smithi162 ceph-mon[110641]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-27T04:16:25.536 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:16:25 smithi162 ceph-mon[110641]: osdmap e59: 6 total, 6 up, 6 in 2024-04-27T04:16:25.672 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:16:25 smithi181 ceph-mon[81134]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-04-27T04:16:25.672 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:16:25 smithi181 ceph-mon[81134]: osd.1 [v2:172.21.15.162:6810/2144127279,v1:172.21.15.162:6811/2144127279] boot 2024-04-27T04:16:25.672 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:16:25 smithi181 ceph-mon[81134]: osdmap e58: 6 total, 6 up, 6 in 2024-04-27T04:16:25.672 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:16:25 smithi181 ceph-mon[81134]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-27T04:16:25.672 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:16:25 smithi181 ceph-mon[81134]: osdmap e59: 6 total, 6 up, 6 in 2024-04-27T04:16:26.524 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:16:26 smithi162 ceph-mon[110641]: pgmap v77: 65 pgs: 12 active+undersized+degraded, 22 active+undersized, 31 active+clean; 208 MiB data, 1.8 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s; 28/231 objects degraded (12.121%) 2024-04-27T04:16:26.672 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:16:26 smithi181 ceph-mon[81134]: pgmap v77: 65 pgs: 12 active+undersized+degraded, 22 active+undersized, 31 active+clean; 208 MiB data, 1.8 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s; 28/231 objects degraded (12.121%) 2024-04-27T04:16:28.422 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:16:28 smithi181 ceph-mon[81134]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' 2024-04-27T04:16:28.422 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:16:28 smithi181 ceph-mon[81134]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' 2024-04-27T04:16:28.422 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:16:28 smithi181 ceph-mon[81134]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T04:16:28.422 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:16:28 smithi181 ceph-mon[81134]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T04:16:28.422 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:16:28 smithi181 ceph-mon[81134]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' 2024-04-27T04:16:28.422 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:16:28 smithi181 ceph-mon[81134]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T04:16:28.423 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:16:28 smithi181 ceph-mon[81134]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "versions"}]: dispatch 2024-04-27T04:16:28.423 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:16:28 smithi181 ceph-mon[81134]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "versions"}]: dispatch 2024-04-27T04:16:28.423 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:16:28 smithi181 ceph-mon[81134]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "versions"}]: dispatch 2024-04-27T04:16:28.423 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:16:28 smithi181 ceph-mon[81134]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "osd ok-to-stop", "ids": ["2"], "max": 16}]: dispatch 2024-04-27T04:16:28.423 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:16:28 smithi181 ceph-mon[81134]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["2"], "max": 16}]: dispatch 2024-04-27T04:16:28.423 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:16:28 smithi181 ceph-mon[81134]: Upgrade: unsafe to stop osd(s) at this time (12 PGs are or would become offline) 2024-04-27T04:16:28.423 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:16:28 smithi181 ceph-mon[81134]: pgmap v78: 65 pgs: 12 active+undersized+degraded, 20 active+undersized, 33 active+clean; 208 MiB data, 1.8 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s; 28/231 objects degraded (12.121%) 2024-04-27T04:16:28.636 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:16:28 smithi162 ceph-mon[110641]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' 2024-04-27T04:16:28.637 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:16:28 smithi162 ceph-mon[110641]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' 2024-04-27T04:16:28.637 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:16:28 smithi162 ceph-mon[110641]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T04:16:28.637 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:16:28 smithi162 ceph-mon[110641]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T04:16:28.637 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:16:28 smithi162 ceph-mon[110641]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' 2024-04-27T04:16:28.637 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:16:28 smithi162 ceph-mon[110641]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T04:16:28.637 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:16:28 smithi162 ceph-mon[110641]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "versions"}]: dispatch 2024-04-27T04:16:28.637 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:16:28 smithi162 ceph-mon[110641]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "versions"}]: dispatch 2024-04-27T04:16:28.637 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:16:28 smithi162 ceph-mon[110641]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "versions"}]: dispatch 2024-04-27T04:16:28.637 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:16:28 smithi162 ceph-mon[110641]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "osd ok-to-stop", "ids": ["2"], "max": 16}]: dispatch 2024-04-27T04:16:28.637 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:16:28 smithi162 ceph-mon[110641]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["2"], "max": 16}]: dispatch 2024-04-27T04:16:28.637 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:16:28 smithi162 ceph-mon[110641]: Upgrade: unsafe to stop osd(s) at this time (12 PGs are or would become offline) 2024-04-27T04:16:28.638 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:16:28 smithi162 ceph-mon[110641]: pgmap v78: 65 pgs: 12 active+undersized+degraded, 20 active+undersized, 33 active+clean; 208 MiB data, 1.8 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s; 28/231 objects degraded (12.121%) 2024-04-27T04:16:30.136 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:16:29 smithi162 ceph-mon[110641]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 28/231 objects degraded (12.121%), 12 pgs degraded) 2024-04-27T04:16:30.136 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:16:29 smithi162 ceph-mon[110641]: Cluster is now healthy 2024-04-27T04:16:30.171 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:16:29 smithi181 ceph-mon[81134]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 28/231 objects degraded (12.121%), 12 pgs degraded) 2024-04-27T04:16:30.172 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:16:29 smithi181 ceph-mon[81134]: Cluster is now healthy 2024-04-27T04:16:31.136 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:16:30 smithi162 ceph-mon[110641]: pgmap v79: 65 pgs: 65 active+clean; 208 MiB data, 1.8 GiB used, 535 GiB / 536 GiB avail; 1.6 KiB/s rd, 1 op/s 2024-04-27T04:16:31.172 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:16:30 smithi181 ceph-mon[81134]: pgmap v79: 65 pgs: 65 active+clean; 208 MiB data, 1.8 GiB used, 535 GiB / 536 GiB avail; 1.6 KiB/s rd, 1 op/s 2024-04-27T04:16:33.137 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:16:32 smithi162 ceph-mon[110641]: pgmap v80: 65 pgs: 65 active+clean; 208 MiB data, 1.8 GiB used, 535 GiB / 536 GiB avail 2024-04-27T04:16:33.137 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:16:32 smithi162 ceph-mon[110641]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' 2024-04-27T04:16:33.137 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:16:32 smithi162 ceph-mon[110641]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-27T04:16:33.172 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:16:32 smithi181 ceph-mon[81134]: pgmap v80: 65 pgs: 65 active+clean; 208 MiB data, 1.8 GiB used, 535 GiB / 536 GiB avail 2024-04-27T04:16:33.172 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:16:32 smithi181 ceph-mon[81134]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' 2024-04-27T04:16:33.172 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:16:32 smithi181 ceph-mon[81134]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-27T04:16:35.136 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:16:34 smithi162 ceph-mon[110641]: pgmap v81: 65 pgs: 65 active+clean; 208 MiB data, 1.8 GiB used, 535 GiB / 536 GiB avail; 1.1 KiB/s rd, 0 op/s 2024-04-27T04:16:35.172 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:16:34 smithi181 ceph-mon[81134]: pgmap v81: 65 pgs: 65 active+clean; 208 MiB data, 1.8 GiB used, 535 GiB / 536 GiB avail; 1.1 KiB/s rd, 0 op/s 2024-04-27T04:16:37.138 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:16:36 smithi162 ceph-mon[110641]: pgmap v82: 65 pgs: 65 active+clean; 208 MiB data, 1.8 GiB used, 535 GiB / 536 GiB avail; 984 B/s rd, 0 op/s 2024-04-27T04:16:37.171 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:16:36 smithi181 ceph-mon[81134]: pgmap v82: 65 pgs: 65 active+clean; 208 MiB data, 1.8 GiB used, 535 GiB / 536 GiB avail; 984 B/s rd, 0 op/s 2024-04-27T04:16:39.136 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:16:38 smithi162 ceph-mon[110641]: pgmap v83: 65 pgs: 65 active+clean; 208 MiB data, 1.8 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 0 op/s 2024-04-27T04:16:39.137 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:16:38 smithi162 ceph-mon[110641]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' 2024-04-27T04:16:39.172 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:16:38 smithi181 ceph-mon[81134]: pgmap v83: 65 pgs: 65 active+clean; 208 MiB data, 1.8 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 0 op/s 2024-04-27T04:16:39.172 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:16:38 smithi181 ceph-mon[81134]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' 2024-04-27T04:16:41.136 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:16:40 smithi162 ceph-mon[110641]: pgmap v84: 65 pgs: 65 active+clean; 208 MiB data, 1.8 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-04-27T04:16:41.172 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:16:40 smithi181 ceph-mon[81134]: pgmap v84: 65 pgs: 65 active+clean; 208 MiB data, 1.8 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-04-27T04:16:43.007 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:16:42 smithi162 ceph-mon[110641]: pgmap v85: 65 pgs: 65 active+clean; 208 MiB data, 1.8 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:16:43.007 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:16:42 smithi162 ceph-mon[110641]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "osd ok-to-stop", "ids": ["2"], "max": 16}]: dispatch 2024-04-27T04:16:43.172 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:16:42 smithi181 ceph-mon[81134]: pgmap v85: 65 pgs: 65 active+clean; 208 MiB data, 1.8 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:16:43.172 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:16:42 smithi181 ceph-mon[81134]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "osd ok-to-stop", "ids": ["2"], "max": 16}]: dispatch 2024-04-27T04:16:44.136 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:16:43 smithi162 ceph-mon[110641]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["2"], "max": 16}]: dispatch 2024-04-27T04:16:44.136 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:16:43 smithi162 ceph-mon[110641]: Upgrade: osd.2 is safe to restart 2024-04-27T04:16:44.172 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:16:43 smithi181 ceph-mon[81134]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["2"], "max": 16}]: dispatch 2024-04-27T04:16:44.172 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:16:43 smithi181 ceph-mon[81134]: Upgrade: osd.2 is safe to restart 2024-04-27T04:16:44.807 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:16:44 smithi162 ceph-mon[110641]: pgmap v86: 65 pgs: 65 active+clean; 208 MiB data, 1.8 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:16:44.807 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:16:44 smithi162 ceph-mon[110641]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' 2024-04-27T04:16:44.807 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:16:44 smithi162 ceph-mon[110641]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-04-27T04:16:44.807 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:16:44 smithi162 ceph-mon[110641]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T04:16:45.172 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:16:44 smithi181 ceph-mon[81134]: pgmap v86: 65 pgs: 65 active+clean; 208 MiB data, 1.8 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:16:45.172 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:16:44 smithi181 ceph-mon[81134]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' 2024-04-27T04:16:45.172 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:16:44 smithi181 ceph-mon[81134]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-04-27T04:16:45.172 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:16:44 smithi181 ceph-mon[81134]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T04:16:45.886 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:16:45 smithi162 ceph-mon[110641]: Upgrade: Updating osd.2 2024-04-27T04:16:45.886 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:16:45 smithi162 ceph-mon[110641]: Deploying daemon osd.2 on smithi162 2024-04-27T04:16:46.172 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:16:45 smithi181 ceph-mon[81134]: Upgrade: Updating osd.2 2024-04-27T04:16:46.172 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:16:45 smithi181 ceph-mon[81134]: Deploying daemon osd.2 on smithi162 2024-04-27T04:16:46.304 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:16:46.306+0000 7f4456cbe700 1 -- 172.21.15.162:0/881602604 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4450101d90 msgr2=0x7f4450102160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:16:46.304 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:16:46.306+0000 7f4456cbe700 1 --2- 172.21.15.162:0/881602604 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4450101d90 0x7f4450102160 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f4444004940 tx=0x7f4444009e30 comp rx=0 tx=0).stop 2024-04-27T04:16:46.304 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:16:46.306+0000 7f4456cbe700 1 -- 172.21.15.162:0/881602604 shutdown_connections 2024-04-27T04:16:46.305 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:16:46.306+0000 7f4456cbe700 1 --2- 172.21.15.162:0/881602604 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f44500fbd40 0x7f44500fc1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:16:46.305 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:16:46.306+0000 7f4456cbe700 1 --2- 172.21.15.162:0/881602604 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4450101d90 0x7f4450102160 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:16:46.305 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:16:46.306+0000 7f4456cbe700 1 -- 172.21.15.162:0/881602604 >> 172.21.15.162:0/881602604 conn(0x7f4450071d00 msgr2=0x7f4450072100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:16:46.305 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:16:46.306+0000 7f4456cbe700 1 -- 172.21.15.162:0/881602604 shutdown_connections 2024-04-27T04:16:46.305 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:16:46.306+0000 7f4456cbe700 1 -- 172.21.15.162:0/881602604 wait complete. 2024-04-27T04:16:46.305 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:16:46.306+0000 7f4456cbe700 1 Processor -- start 2024-04-27T04:16:46.305 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:16:46.307+0000 7f4456cbe700 1 -- start start 2024-04-27T04:16:46.307 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:16:46.307+0000 7f4456cbe700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f44500fbd40 0x7f445010ea80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:16:46.307 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:16:46.307+0000 7f4456cbe700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4450101d90 0x7f445010efc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:16:46.307 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:16:46.307+0000 7f4456cbe700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f445010f610 con 0x7f44500fbd40 2024-04-27T04:16:46.307 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:16:46.307+0000 7f4456cbe700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f445010f750 con 0x7f4450101d90 2024-04-27T04:16:46.307 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:16:46.307+0000 7f4454a5a700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f44500fbd40 0x7f445010ea80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:16:46.307 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:16:46.307+0000 7f4454a5a700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f44500fbd40 0x7f445010ea80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:38130/0 (socket says 172.21.15.162:38130) 2024-04-27T04:16:46.307 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:16:46.307+0000 7f4454a5a700 1 -- 172.21.15.162:0/2335060055 learned_addr learned my addr 172.21.15.162:0/2335060055 (peer_addr_for_me v2:172.21.15.162:0/0) 2024-04-27T04:16:46.307 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:16:46.307+0000 7f4454a5a700 1 -- 172.21.15.162:0/2335060055 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4450101d90 msgr2=0x7f445010efc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:16:46.307 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:16:46.307+0000 7f4454a5a700 1 --2- 172.21.15.162:0/2335060055 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4450101d90 0x7f445010efc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:16:46.307 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:16:46.307+0000 7f4454a5a700 1 -- 172.21.15.162:0/2335060055 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4444005040 con 0x7f44500fbd40 2024-04-27T04:16:46.307 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:16:46.307+0000 7f4454a5a700 1 --2- 172.21.15.162:0/2335060055 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f44500fbd40 0x7f445010ea80 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f4444009180 tx=0x7f4444009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:16:46.307 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:16:46.308+0000 7f444d7fa700 1 -- 172.21.15.162:0/2335060055 <== mon.0 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f444400e040 con 0x7f44500fbd40 2024-04-27T04:16:46.307 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:16:46.308+0000 7f4456cbe700 1 -- 172.21.15.162:0/2335060055 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4450117af0 con 0x7f44500fbd40 2024-04-27T04:16:46.308 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:16:46.308+0000 7f444d7fa700 1 -- 172.21.15.162:0/2335060055 <== mon.0 v2:172.21.15.162:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f444400bc90 con 0x7f44500fbd40 2024-04-27T04:16:46.308 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:16:46.308+0000 7f444d7fa700 1 -- 172.21.15.162:0/2335060055 <== mon.0 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f444401c630 con 0x7f44500fbd40 2024-04-27T04:16:46.308 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:16:46.308+0000 7f4456cbe700 1 -- 172.21.15.162:0/2335060055 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4450118000 con 0x7f44500fbd40 2024-04-27T04:16:46.308 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:16:46.309+0000 7f4456cbe700 1 -- 172.21.15.162:0/2335060055 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f44500fd430 con 0x7f44500fbd40 2024-04-27T04:16:46.308 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:16:46.309+0000 7f444d7fa700 1 -- 172.21.15.162:0/2335060055 <== mon.0 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 39) v1 ==== 92747+0+0 (secure 0 0 0) 0x7f4444013980 con 0x7f44500fbd40 2024-04-27T04:16:46.309 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:16:46.310+0000 7f444d7fa700 1 --2- 172.21.15.162:0/2335060055 >> [v2:172.21.15.162:6800/1396866108,v1:172.21.15.162:6801/1396866108] conn(0x7f444006ec10 0x7f44400710c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:16:46.309 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:16:46.310+0000 7f444d7fa700 1 -- 172.21.15.162:0/2335060055 <== mon.0 v2:172.21.15.162:3300/0 5 ==== osd_map(59..59 src has 1..59) v4 ==== 6410+0+0 (secure 0 0 0) 0x7f444408fcd0 con 0x7f44500fbd40 2024-04-27T04:16:46.309 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:16:46.310+0000 7f444ffff700 1 --2- 172.21.15.162:0/2335060055 >> [v2:172.21.15.162:6800/1396866108,v1:172.21.15.162:6801/1396866108] conn(0x7f444006ec10 0x7f44400710c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:16:46.309 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:16:46.310+0000 7f444ffff700 1 --2- 172.21.15.162:0/2335060055 >> [v2:172.21.15.162:6800/1396866108,v1:172.21.15.162:6801/1396866108] conn(0x7f444006ec10 0x7f44400710c0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f443c009b80 tx=0x7f443c009d60 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:16:46.312 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:16:46.315+0000 7f444d7fa700 1 -- 172.21.15.162:0/2335060055 <== mon.0 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7f4444058b50 con 0x7f44500fbd40 2024-04-27T04:16:46.536 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:16:46.538+0000 7f4456cbe700 1 -- 172.21.15.162:0/2335060055 --> [v2:172.21.15.162:6800/1396866108,v1:172.21.15.162:6801/1396866108] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f445004bed0 con 0x7f444006ec10 2024-04-27T04:16:46.537 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:16:46.539+0000 7f444d7fa700 1 -- 172.21.15.162:0/2335060055 <== mgr.34104 v2:172.21.15.162:6800/1396866108 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+400 (secure 0 0 0) 0x7f445004bed0 con 0x7f444006ec10 2024-04-27T04:16:46.540 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:16:46.542+0000 7f4456cbe700 1 -- 172.21.15.162:0/2335060055 >> [v2:172.21.15.162:6800/1396866108,v1:172.21.15.162:6801/1396866108] conn(0x7f444006ec10 msgr2=0x7f44400710c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:16:46.540 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:16:46.542+0000 7f4456cbe700 1 --2- 172.21.15.162:0/2335060055 >> [v2:172.21.15.162:6800/1396866108,v1:172.21.15.162:6801/1396866108] conn(0x7f444006ec10 0x7f44400710c0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f443c009b80 tx=0x7f443c009d60 comp rx=0 tx=0).stop 2024-04-27T04:16:46.540 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:16:46.542+0000 7f4456cbe700 1 -- 172.21.15.162:0/2335060055 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f44500fbd40 msgr2=0x7f445010ea80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:16:46.540 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:16:46.542+0000 7f4456cbe700 1 --2- 172.21.15.162:0/2335060055 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f44500fbd40 0x7f445010ea80 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f4444009180 tx=0x7f4444009e30 comp rx=0 tx=0).stop 2024-04-27T04:16:46.540 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:16:46.542+0000 7f4456cbe700 1 -- 172.21.15.162:0/2335060055 shutdown_connections 2024-04-27T04:16:46.540 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:16:46.542+0000 7f4456cbe700 1 --2- 172.21.15.162:0/2335060055 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f44500fbd40 0x7f445010ea80 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:16:46.540 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:16:46.542+0000 7f4456cbe700 1 --2- 172.21.15.162:0/2335060055 >> [v2:172.21.15.162:6800/1396866108,v1:172.21.15.162:6801/1396866108] conn(0x7f444006ec10 0x7f44400710c0 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:16:46.540 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:16:46.542+0000 7f4456cbe700 1 --2- 172.21.15.162:0/2335060055 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4450101d90 0x7f445010efc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:16:46.540 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:16:46.542+0000 7f4456cbe700 1 -- 172.21.15.162:0/2335060055 >> 172.21.15.162:0/2335060055 conn(0x7f4450071d00 msgr2=0x7f44500fa0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:16:46.540 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:16:46.542+0000 7f4456cbe700 1 -- 172.21.15.162:0/2335060055 shutdown_connections 2024-04-27T04:16:46.540 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:16:46.542+0000 7f4456cbe700 1 -- 172.21.15.162:0/2335060055 wait complete. 2024-04-27T04:16:46.550 INFO:teuthology.orchestra.run.smithi162.stdout:true 2024-04-27T04:16:46.645 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:16:46.645+0000 7f5807a68700 1 -- 172.21.15.162:0/96711177 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f58000fc080 msgr2=0x7f580010d470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:16:46.645 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:16:46.645+0000 7f5807a68700 1 --2- 172.21.15.162:0/96711177 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f58000fc080 0x7f580010d470 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f57f0004970 tx=0x7f57f0009e30 comp rx=0 tx=0).stop 2024-04-27T04:16:46.645 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:16:46.645+0000 7f5807a68700 1 -- 172.21.15.162:0/96711177 shutdown_connections 2024-04-27T04:16:46.645 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:16:46.645+0000 7f5807a68700 1 --2- 172.21.15.162:0/96711177 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f58000fc080 0x7f580010d470 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:16:46.645 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:16:46.645+0000 7f5807a68700 1 --2- 172.21.15.162:0/96711177 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f58000fb770 0x7f58000fbb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:16:46.645 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:16:46.645+0000 7f5807a68700 1 -- 172.21.15.162:0/96711177 >> 172.21.15.162:0/96711177 conn(0x7f58000714d0 msgr2=0x7f58000718d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:16:46.645 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:16:46.645+0000 7f5807a68700 1 -- 172.21.15.162:0/96711177 shutdown_connections 2024-04-27T04:16:46.645 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:16:46.645+0000 7f5807a68700 1 -- 172.21.15.162:0/96711177 wait complete. 2024-04-27T04:16:46.645 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:16:46.646+0000 7f5807a68700 1 Processor -- start 2024-04-27T04:16:46.645 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:16:46.646+0000 7f5807a68700 1 -- start start 2024-04-27T04:16:46.646 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:16:46.646+0000 7f5807a68700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f58000fb770 0x7f580011a000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:16:46.646 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:16:46.646+0000 7f5807a68700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f58000fc080 0x7f580011a540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:16:46.646 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:16:46.646+0000 7f5807a68700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f580011ab90 con 0x7f58000fc080 2024-04-27T04:16:46.646 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:16:46.646+0000 7f5807a68700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5800114170 con 0x7f58000fb770 2024-04-27T04:16:46.646 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:16:46.646+0000 7f5805003700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f58000fc080 0x7f580011a540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:16:46.646 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:16:46.646+0000 7f5805003700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f58000fc080 0x7f580011a540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:38140/0 (socket says 172.21.15.162:38140) 2024-04-27T04:16:46.646 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:16:46.646+0000 7f5805003700 1 -- 172.21.15.162:0/1174655549 learned_addr learned my addr 172.21.15.162:0/1174655549 (peer_addr_for_me v2:172.21.15.162:0/0) 2024-04-27T04:16:46.646 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:16:46.646+0000 7f5805804700 1 --2- 172.21.15.162:0/1174655549 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f58000fb770 0x7f580011a000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:16:46.646 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:16:46.646+0000 7f5805003700 1 -- 172.21.15.162:0/1174655549 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f58000fb770 msgr2=0x7f580011a000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:16:46.646 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:16:46.646+0000 7f5805003700 1 --2- 172.21.15.162:0/1174655549 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f58000fb770 0x7f580011a000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:16:46.646 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:16:46.646+0000 7f5805003700 1 -- 172.21.15.162:0/1174655549 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57f0005040 con 0x7f58000fc080 2024-04-27T04:16:46.646 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:16:46.647+0000 7f5805003700 1 --2- 172.21.15.162:0/1174655549 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f58000fc080 0x7f580011a540 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f57f0005010 tx=0x7f57f0009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:16:46.646 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:16:46.647+0000 7f57f67fc700 1 -- 172.21.15.162:0/1174655549 <== mon.0 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f57f000e040 con 0x7f58000fc080 2024-04-27T04:16:46.646 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:16:46.647+0000 7f5807a68700 1 -- 172.21.15.162:0/1174655549 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f58001143f0 con 0x7f58000fc080 2024-04-27T04:16:46.647 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:16:46.647+0000 7f57f67fc700 1 -- 172.21.15.162:0/1174655549 <== mon.0 v2:172.21.15.162:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f57f0004290 con 0x7f58000fc080 2024-04-27T04:16:46.647 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:16:46.647+0000 7f57f67fc700 1 -- 172.21.15.162:0/1174655549 <== mon.0 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f57f001c3f0 con 0x7f58000fc080 2024-04-27T04:16:46.647 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:16:46.647+0000 7f5807a68700 1 -- 172.21.15.162:0/1174655549 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5800114930 con 0x7f58000fc080 2024-04-27T04:16:46.647 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:16:46.648+0000 7f5807a68700 1 -- 172.21.15.162:0/1174655549 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f58000fe0f0 con 0x7f58000fc080 2024-04-27T04:16:46.648 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:16:46.648+0000 7f57f67fc700 1 -- 172.21.15.162:0/1174655549 <== mon.0 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 39) v1 ==== 92747+0+0 (secure 0 0 0) 0x7f57f0013800 con 0x7f58000fc080 2024-04-27T04:16:46.648 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:16:46.649+0000 7f57f67fc700 1 --2- 172.21.15.162:0/1174655549 >> [v2:172.21.15.162:6800/1396866108,v1:172.21.15.162:6801/1396866108] conn(0x7f57ec06ec00 0x7f57ec0710b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:16:46.648 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:16:46.649+0000 7f57f67fc700 1 -- 172.21.15.162:0/1174655549 <== mon.0 v2:172.21.15.162:3300/0 5 ==== osd_map(59..59 src has 1..59) v4 ==== 6410+0+0 (secure 0 0 0) 0x7f57f008fa50 con 0x7f58000fc080 2024-04-27T04:16:46.648 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:16:46.649+0000 7f5805804700 1 --2- 172.21.15.162:0/1174655549 >> [v2:172.21.15.162:6800/1396866108,v1:172.21.15.162:6801/1396866108] conn(0x7f57ec06ec00 0x7f57ec0710b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:16:46.648 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:16:46.649+0000 7f5805804700 1 --2- 172.21.15.162:0/1174655549 >> [v2:172.21.15.162:6800/1396866108,v1:172.21.15.162:6801/1396866108] conn(0x7f57ec06ec00 0x7f57ec0710b0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f57fc009fd0 tx=0x7f57fc00b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:16:46.651 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:16:46.653+0000 7f57f67fc700 1 -- 172.21.15.162:0/1174655549 <== mon.0 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7f57f005c400 con 0x7f58000fc080 2024-04-27T04:16:46.834 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:16:46 smithi162 ceph-mon[110641]: pgmap v87: 65 pgs: 65 active+clean; 208 MiB data, 1.8 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:16:46.873 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:16:46.875+0000 7f5807a68700 1 -- 172.21.15.162:0/1174655549 --> [v2:172.21.15.162:6800/1396866108,v1:172.21.15.162:6801/1396866108] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f580004c710 con 0x7f57ec06ec00 2024-04-27T04:16:46.874 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:16:46.877+0000 7f57f67fc700 1 -- 172.21.15.162:0/1174655549 <== mgr.34104 v2:172.21.15.162:6800/1396866108 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+400 (secure 0 0 0) 0x7f580004c710 con 0x7f57ec06ec00 2024-04-27T04:16:46.877 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:16:46.879+0000 7f5807a68700 1 -- 172.21.15.162:0/1174655549 >> [v2:172.21.15.162:6800/1396866108,v1:172.21.15.162:6801/1396866108] conn(0x7f57ec06ec00 msgr2=0x7f57ec0710b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:16:46.877 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:16:46.879+0000 7f5807a68700 1 --2- 172.21.15.162:0/1174655549 >> [v2:172.21.15.162:6800/1396866108,v1:172.21.15.162:6801/1396866108] conn(0x7f57ec06ec00 0x7f57ec0710b0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f57fc009fd0 tx=0x7f57fc00b040 comp rx=0 tx=0).stop 2024-04-27T04:16:46.877 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:16:46.879+0000 7f5807a68700 1 -- 172.21.15.162:0/1174655549 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f58000fc080 msgr2=0x7f580011a540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:16:46.877 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:16:46.879+0000 7f5807a68700 1 --2- 172.21.15.162:0/1174655549 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f58000fc080 0x7f580011a540 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f57f0005010 tx=0x7f57f0009e30 comp rx=0 tx=0).stop 2024-04-27T04:16:46.877 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:16:46.879+0000 7f5807a68700 1 -- 172.21.15.162:0/1174655549 shutdown_connections 2024-04-27T04:16:46.877 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:16:46.879+0000 7f5807a68700 1 --2- 172.21.15.162:0/1174655549 >> [v2:172.21.15.162:6800/1396866108,v1:172.21.15.162:6801/1396866108] conn(0x7f57ec06ec00 0x7f57ec0710b0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:16:46.877 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:16:46.879+0000 7f5807a68700 1 --2- 172.21.15.162:0/1174655549 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f58000fb770 0x7f580011a000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:16:46.878 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:16:46.879+0000 7f5807a68700 1 --2- 172.21.15.162:0/1174655549 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f58000fc080 0x7f580011a540 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:16:46.878 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:16:46.879+0000 7f5807a68700 1 -- 172.21.15.162:0/1174655549 >> 172.21.15.162:0/1174655549 conn(0x7f58000714d0 msgr2=0x7f5800102050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:16:46.878 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:16:46.879+0000 7f5807a68700 1 -- 172.21.15.162:0/1174655549 shutdown_connections 2024-04-27T04:16:46.878 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:16:46.879+0000 7f5807a68700 1 -- 172.21.15.162:0/1174655549 wait complete. 2024-04-27T04:16:46.984 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:16:46.984+0000 7fc6b478f700 1 -- 172.21.15.162:0/2946954388 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc6ac101d90 msgr2=0x7fc6ac102160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:16:46.984 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:16:46.984+0000 7fc6b478f700 1 --2- 172.21.15.162:0/2946954388 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc6ac101d90 0x7fc6ac102160 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fc6a8004940 tx=0x7fc6a8009e30 comp rx=0 tx=0).stop 2024-04-27T04:16:46.984 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:16:46.985+0000 7fc6b478f700 1 -- 172.21.15.162:0/2946954388 shutdown_connections 2024-04-27T04:16:46.984 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:16:46.985+0000 7fc6b478f700 1 --2- 172.21.15.162:0/2946954388 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc6ac0fbd40 0x7fc6ac0fc1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:16:46.984 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:16:46.985+0000 7fc6b478f700 1 --2- 172.21.15.162:0/2946954388 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc6ac101d90 0x7fc6ac102160 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:16:46.984 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:16:46.985+0000 7fc6b478f700 1 -- 172.21.15.162:0/2946954388 >> 172.21.15.162:0/2946954388 conn(0x7fc6ac071d00 msgr2=0x7fc6ac072100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:16:46.984 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:16:46.985+0000 7fc6b478f700 1 -- 172.21.15.162:0/2946954388 shutdown_connections 2024-04-27T04:16:46.984 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:16:46.985+0000 7fc6b478f700 1 -- 172.21.15.162:0/2946954388 wait complete. 2024-04-27T04:16:46.985 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:16:46.985+0000 7fc6b478f700 1 Processor -- start 2024-04-27T04:16:46.985 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:16:46.985+0000 7fc6b478f700 1 -- start start 2024-04-27T04:16:46.985 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:16:46.985+0000 7fc6b478f700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc6ac0fbd40 0x7fc6ac113ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:16:46.985 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:16:46.985+0000 7fc6b478f700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc6ac101d90 0x7fc6ac1140e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:16:46.985 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:16:46.985+0000 7fc6b478f700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6ac114730 con 0x7fc6ac0fbd40 2024-04-27T04:16:46.985 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:16:46.985+0000 7fc6b478f700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6ac10db20 con 0x7fc6ac101d90 2024-04-27T04:16:46.985 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:16:46.985+0000 7fc6b252b700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc6ac0fbd40 0x7fc6ac113ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:16:46.985 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:16:46.985+0000 7fc6b252b700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc6ac0fbd40 0x7fc6ac113ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:38160/0 (socket says 172.21.15.162:38160) 2024-04-27T04:16:46.985 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:16:46.985+0000 7fc6b252b700 1 -- 172.21.15.162:0/1608726830 learned_addr learned my addr 172.21.15.162:0/1608726830 (peer_addr_for_me v2:172.21.15.162:0/0) 2024-04-27T04:16:46.985 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:16:46.986+0000 7fc6b252b700 1 -- 172.21.15.162:0/1608726830 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc6ac101d90 msgr2=0x7fc6ac1140e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:16:46.985 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:16:46.986+0000 7fc6b252b700 1 --2- 172.21.15.162:0/1608726830 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc6ac101d90 0x7fc6ac1140e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:16:46.985 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:16:46.986+0000 7fc6b252b700 1 -- 172.21.15.162:0/1608726830 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc6a8005040 con 0x7fc6ac0fbd40 2024-04-27T04:16:46.985 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:16:46.986+0000 7fc6b252b700 1 --2- 172.21.15.162:0/1608726830 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc6ac0fbd40 0x7fc6ac113ba0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fc6a8009150 tx=0x7fc6a8009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:16:46.985 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:16:46.986+0000 7fc6a2ffd700 1 -- 172.21.15.162:0/1608726830 <== mon.0 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc6a800e040 con 0x7fc6ac0fbd40 2024-04-27T04:16:46.985 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:16:46.986+0000 7fc6a2ffd700 1 -- 172.21.15.162:0/1608726830 <== mon.0 v2:172.21.15.162:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fc6a800bbc0 con 0x7fc6ac0fbd40 2024-04-27T04:16:46.986 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:16:46.986+0000 7fc6b478f700 1 -- 172.21.15.162:0/1608726830 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc6ac10dda0 con 0x7fc6ac0fbd40 2024-04-27T04:16:46.986 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:16:46.986+0000 7fc6a2ffd700 1 -- 172.21.15.162:0/1608726830 <== mon.0 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc6a801c630 con 0x7fc6ac0fbd40 2024-04-27T04:16:46.986 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:16:46.986+0000 7fc6b478f700 1 -- 172.21.15.162:0/1608726830 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc6ac10e360 con 0x7fc6ac0fbd40 2024-04-27T04:16:46.987 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:16:46.987+0000 7fc6b478f700 1 -- 172.21.15.162:0/1608726830 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc6ac0fd430 con 0x7fc6ac0fbd40 2024-04-27T04:16:46.987 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:16:46.988+0000 7fc6a2ffd700 1 -- 172.21.15.162:0/1608726830 <== mon.0 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 39) v1 ==== 92747+0+0 (secure 0 0 0) 0x7fc6a800bd30 con 0x7fc6ac0fbd40 2024-04-27T04:16:46.987 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:16:46.988+0000 7fc6a2ffd700 1 --2- 172.21.15.162:0/1608726830 >> [v2:172.21.15.162:6800/1396866108,v1:172.21.15.162:6801/1396866108] conn(0x7fc69806ec10 0x7fc6980710c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:16:46.987 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:16:46.988+0000 7fc6a2ffd700 1 -- 172.21.15.162:0/1608726830 <== mon.0 v2:172.21.15.162:3300/0 5 ==== osd_map(59..59 src has 1..59) v4 ==== 6410+0+0 (secure 0 0 0) 0x7fc6a8090540 con 0x7fc6ac0fbd40 2024-04-27T04:16:46.987 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:16:46.988+0000 7fc6b1d2a700 1 --2- 172.21.15.162:0/1608726830 >> [v2:172.21.15.162:6800/1396866108,v1:172.21.15.162:6801/1396866108] conn(0x7fc69806ec10 0x7fc6980710c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:16:46.987 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:16:46.989+0000 7fc6b1d2a700 1 --2- 172.21.15.162:0/1608726830 >> [v2:172.21.15.162:6800/1396866108,v1:172.21.15.162:6801/1396866108] conn(0x7fc69806ec10 0x7fc6980710c0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fc6ac10ece0 tx=0x7fc69c009ce0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:16:46.990 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:16:46.992+0000 7fc6a2ffd700 1 -- 172.21.15.162:0/1608726830 <== mon.0 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7fc6a805cf90 con 0x7fc6ac0fbd40 2024-04-27T04:16:47.172 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:16:46 smithi181 ceph-mon[81134]: pgmap v87: 65 pgs: 65 active+clean; 208 MiB data, 1.8 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:16:47.213 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:16:47.215+0000 7fc6b478f700 1 -- 172.21.15.162:0/1608726830 --> [v2:172.21.15.162:6800/1396866108,v1:172.21.15.162:6801/1396866108] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7fc6ac108df0 con 0x7fc69806ec10 2024-04-27T04:16:47.224 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:16:47.226+0000 7fc6a2ffd700 1 -- 172.21.15.162:0/1608726830 <== mgr.34104 v2:172.21.15.162:6800/1396866108 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3498 (secure 0 0 0) 0x7fc6ac108df0 con 0x7fc69806ec10 2024-04-27T04:16:47.224 INFO:teuthology.orchestra.run.smithi162.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-27T04:16:47.224 INFO:teuthology.orchestra.run.smithi162.stdout:alertmanager.smithi162 smithi162 *:9093,9094 running (3m) 25s ago 15m 27.2M - 0.25.0 c8568f914cd2 40e334659017 2024-04-27T04:16:47.224 INFO:teuthology.orchestra.run.smithi162.stdout:crash.smithi162 smithi162 running (92s) 25s ago 15m 6660k - 19.0.0-3253-g4e213a58 67e414ff4bc4 1aa8f4d5a28b 2024-04-27T04:16:47.224 INFO:teuthology.orchestra.run.smithi162.stdout:crash.smithi181 smithi181 running (87s) 84s ago 14m 6656k - 19.0.0-3253-g4e213a58 67e414ff4bc4 832d9cbc5c30 2024-04-27T04:16:47.224 INFO:teuthology.orchestra.run.smithi162.stdout:grafana.smithi162 smithi162 *:3000 running (3m) 25s ago 14m 78.9M - 9.4.12 2bacad6d85d8 3d78c478f03f 2024-04-27T04:16:47.224 INFO:teuthology.orchestra.run.smithi162.stdout:mds.cephfs.smithi162.gzlyjj smithi162 running (9m) 25s ago 9m 14.1M - 17.2.7-907-g2be3efa9 1c5834633715 27e5b030dd32 2024-04-27T04:16:47.224 INFO:teuthology.orchestra.run.smithi162.stdout:mds.cephfs.smithi162.xcnapk smithi162 running (9m) 25s ago 9m 100M - 17.2.7-907-g2be3efa9 1c5834633715 38f9030df029 2024-04-27T04:16:47.225 INFO:teuthology.orchestra.run.smithi162.stdout:mds.cephfs.smithi181.bkkcdm smithi181 running (9m) 84s ago 9m 15.3M - 17.2.7-907-g2be3efa9 1c5834633715 af55afca1263 2024-04-27T04:16:47.225 INFO:teuthology.orchestra.run.smithi162.stdout:mds.cephfs.smithi181.dzomgh smithi181 running (9m) 84s ago 9m 143M - 17.2.7-907-g2be3efa9 1c5834633715 6733296597f3 2024-04-27T04:16:47.225 INFO:teuthology.orchestra.run.smithi162.stdout:mgr.smithi162.oennfn smithi162 *:8443,9283,8765 running (5m) 25s ago 16m 505M - 19.0.0-3253-g4e213a58 67e414ff4bc4 58a748d3637b 2024-04-27T04:16:47.225 INFO:teuthology.orchestra.run.smithi162.stdout:mgr.smithi181.vtwuhz smithi181 *:8443,9283,8765 running (4m) 84s ago 14m 450M - 19.0.0-3253-g4e213a58 67e414ff4bc4 813584297b1d 2024-04-27T04:16:47.225 INFO:teuthology.orchestra.run.smithi162.stdout:mon.smithi162 smithi162 running (2m) 25s ago 16m 44.4M 2048M 19.0.0-3253-g4e213a58 67e414ff4bc4 375adaf4468f 2024-04-27T04:16:47.225 INFO:teuthology.orchestra.run.smithi162.stdout:mon.smithi181 smithi181 running (2m) 84s ago 14m 33.1M 2048M 19.0.0-3253-g4e213a58 67e414ff4bc4 77b71f6d522f 2024-04-27T04:16:47.225 INFO:teuthology.orchestra.run.smithi162.stdout:node-exporter.smithi162 smithi162 *:9100 running (4m) 25s ago 15m 15.2M - 1.5.0 0da6a335fe13 5c4c5ea9c1d7 2024-04-27T04:16:47.225 INFO:teuthology.orchestra.run.smithi162.stdout:node-exporter.smithi181 smithi181 *:9100 running (4m) 84s ago 14m 15.7M - 1.5.0 0da6a335fe13 9067f14c0519 2024-04-27T04:16:47.225 INFO:teuthology.orchestra.run.smithi162.stdout:osd.0 smithi162 running (66s) 25s ago 13m 86.8M 1557M 19.0.0-3253-g4e213a58 67e414ff4bc4 2744aff3cd88 2024-04-27T04:16:47.225 INFO:teuthology.orchestra.run.smithi162.stdout:osd.1 smithi162 running (30s) 25s ago 12m 12.2M 1557M 19.0.0-3253-g4e213a58 67e414ff4bc4 668e16faad62 2024-04-27T04:16:47.225 INFO:teuthology.orchestra.run.smithi162.stdout:osd.2 smithi162 running (12m) 25s ago 12m 203M 1557M 17.2.7-907-g2be3efa9 1c5834633715 70f16d86d38e 2024-04-27T04:16:47.225 INFO:teuthology.orchestra.run.smithi162.stdout:osd.3 smithi181 running (12m) 84s ago 12m 278M 2581M 17.2.7-907-g2be3efa9 1c5834633715 030d829be54e 2024-04-27T04:16:47.225 INFO:teuthology.orchestra.run.smithi162.stdout:osd.4 smithi181 running (11m) 84s ago 11m 247M 2581M 17.2.7-907-g2be3efa9 1c5834633715 512afbee7cf7 2024-04-27T04:16:47.225 INFO:teuthology.orchestra.run.smithi162.stdout:osd.5 smithi181 running (11m) 84s ago 11m 224M 2581M 17.2.7-907-g2be3efa9 1c5834633715 f47a1daac1ea 2024-04-27T04:16:47.225 INFO:teuthology.orchestra.run.smithi162.stdout:prometheus.smithi162 smithi162 *:9095 running (4m) 25s ago 14m 67.6M - 2.43.0 a07b618ecd1d f30657a43324 2024-04-27T04:16:47.227 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:16:47.229+0000 7fc6b478f700 1 -- 172.21.15.162:0/1608726830 >> [v2:172.21.15.162:6800/1396866108,v1:172.21.15.162:6801/1396866108] conn(0x7fc69806ec10 msgr2=0x7fc6980710c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:16:47.227 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:16:47.229+0000 7fc6b478f700 1 --2- 172.21.15.162:0/1608726830 >> [v2:172.21.15.162:6800/1396866108,v1:172.21.15.162:6801/1396866108] conn(0x7fc69806ec10 0x7fc6980710c0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fc6ac10ece0 tx=0x7fc69c009ce0 comp rx=0 tx=0).stop 2024-04-27T04:16:47.227 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:16:47.229+0000 7fc6b478f700 1 -- 172.21.15.162:0/1608726830 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc6ac0fbd40 msgr2=0x7fc6ac113ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:16:47.227 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:16:47.229+0000 7fc6b478f700 1 --2- 172.21.15.162:0/1608726830 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc6ac0fbd40 0x7fc6ac113ba0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fc6a8009150 tx=0x7fc6a8009e30 comp rx=0 tx=0).stop 2024-04-27T04:16:47.228 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:16:47.229+0000 7fc6b478f700 1 -- 172.21.15.162:0/1608726830 shutdown_connections 2024-04-27T04:16:47.228 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:16:47.229+0000 7fc6b478f700 1 --2- 172.21.15.162:0/1608726830 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc6ac0fbd40 0x7fc6ac113ba0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:16:47.228 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:16:47.229+0000 7fc6b478f700 1 --2- 172.21.15.162:0/1608726830 >> [v2:172.21.15.162:6800/1396866108,v1:172.21.15.162:6801/1396866108] conn(0x7fc69806ec10 0x7fc6980710c0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:16:47.228 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:16:47.229+0000 7fc6b478f700 1 --2- 172.21.15.162:0/1608726830 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc6ac101d90 0x7fc6ac1140e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:16:47.228 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:16:47.229+0000 7fc6b478f700 1 -- 172.21.15.162:0/1608726830 >> 172.21.15.162:0/1608726830 conn(0x7fc6ac071d00 msgr2=0x7fc6ac0fa850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:16:47.228 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:16:47.229+0000 7fc6b478f700 1 -- 172.21.15.162:0/1608726830 shutdown_connections 2024-04-27T04:16:47.228 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:16:47.229+0000 7fc6b478f700 1 -- 172.21.15.162:0/1608726830 wait complete. 2024-04-27T04:16:47.339 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:16:47.339+0000 7fcbc6fcd700 1 -- 172.21.15.162:0/2270400543 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fcbc0101d10 msgr2=0x7fcbc01020e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:16:47.339 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:16:47.339+0000 7fcbc6fcd700 1 --2- 172.21.15.162:0/2270400543 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fcbc0101d10 0x7fcbc01020e0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fcbb4004940 tx=0x7fcbb4009e30 comp rx=0 tx=0).stop 2024-04-27T04:16:47.339 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:16:47.339+0000 7fcbc6fcd700 1 -- 172.21.15.162:0/2270400543 shutdown_connections 2024-04-27T04:16:47.339 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:16:47.339+0000 7fcbc6fcd700 1 --2- 172.21.15.162:0/2270400543 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcbc01026b0 0x7fcbc00fbd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:16:47.339 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:16:47.339+0000 7fcbc6fcd700 1 --2- 172.21.15.162:0/2270400543 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fcbc0101d10 0x7fcbc01020e0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:16:47.339 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:16:47.339+0000 7fcbc6fcd700 1 -- 172.21.15.162:0/2270400543 >> 172.21.15.162:0/2270400543 conn(0x7fcbc00f7860 msgr2=0x7fcbc00f9c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:16:47.339 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:16:47.339+0000 7fcbc6fcd700 1 -- 172.21.15.162:0/2270400543 shutdown_connections 2024-04-27T04:16:47.339 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:16:47.339+0000 7fcbc6fcd700 1 -- 172.21.15.162:0/2270400543 wait complete. 2024-04-27T04:16:47.340 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:16:47.339+0000 7fcbc6fcd700 1 Processor -- start 2024-04-27T04:16:47.340 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:16:47.339+0000 7fcbc6fcd700 1 -- start start 2024-04-27T04:16:47.340 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:16:47.340+0000 7fcbc6fcd700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fcbc0101d10 0x7fcbc0106bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:16:47.340 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:16:47.340+0000 7fcbc6fcd700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcbc01026b0 0x7fcbc01070f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:16:47.340 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:16:47.340+0000 7fcbc6fcd700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcbc0108ab0 con 0x7fcbc0101d10 2024-04-27T04:16:47.340 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:16:47.340+0000 7fcbc6fcd700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcbc0107630 con 0x7fcbc01026b0 2024-04-27T04:16:47.340 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:16:47.340+0000 7fcbc4d69700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fcbc0101d10 0x7fcbc0106bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:16:47.340 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:16:47.340+0000 7fcbc4d69700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fcbc0101d10 0x7fcbc0106bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:38178/0 (socket says 172.21.15.162:38178) 2024-04-27T04:16:47.340 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:16:47.340+0000 7fcbc4d69700 1 -- 172.21.15.162:0/2779021551 learned_addr learned my addr 172.21.15.162:0/2779021551 (peer_addr_for_me v2:172.21.15.162:0/0) 2024-04-27T04:16:47.340 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:16:47.340+0000 7fcbbffff700 1 --2- 172.21.15.162:0/2779021551 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcbc01026b0 0x7fcbc01070f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:16:47.340 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:16:47.340+0000 7fcbc4d69700 1 -- 172.21.15.162:0/2779021551 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcbc01026b0 msgr2=0x7fcbc01070f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:16:47.340 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:16:47.340+0000 7fcbc4d69700 1 --2- 172.21.15.162:0/2779021551 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcbc01026b0 0x7fcbc01070f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:16:47.341 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:16:47.340+0000 7fcbc4d69700 1 -- 172.21.15.162:0/2779021551 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcbb4005040 con 0x7fcbc0101d10 2024-04-27T04:16:47.341 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:16:47.341+0000 7fcbc4d69700 1 --2- 172.21.15.162:0/2779021551 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fcbc0101d10 0x7fcbc0106bb0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fcbb400b3b0 tx=0x7fcbb4009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:16:47.341 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:16:47.341+0000 7fcbbd7fa700 1 -- 172.21.15.162:0/2779021551 <== mon.0 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fcbb400e040 con 0x7fcbc0101d10 2024-04-27T04:16:47.341 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:16:47.341+0000 7fcbbd7fa700 1 -- 172.21.15.162:0/2779021551 <== mon.0 v2:172.21.15.162:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fcbb400bbc0 con 0x7fcbc0101d10 2024-04-27T04:16:47.341 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:16:47.341+0000 7fcbbd7fa700 1 -- 172.21.15.162:0/2779021551 <== mon.0 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fcbb401c3f0 con 0x7fcbc0101d10 2024-04-27T04:16:47.341 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:16:47.341+0000 7fcbc6fcd700 1 -- 172.21.15.162:0/2779021551 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcbc01078b0 con 0x7fcbc0101d10 2024-04-27T04:16:47.341 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:16:47.341+0000 7fcbc6fcd700 1 -- 172.21.15.162:0/2779021551 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcbc011a0e0 con 0x7fcbc0101d10 2024-04-27T04:16:47.342 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:16:47.342+0000 7fcbc6fcd700 1 -- 172.21.15.162:0/2779021551 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcbc00ff020 con 0x7fcbc0101d10 2024-04-27T04:16:47.342 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:16:47.343+0000 7fcbbd7fa700 1 -- 172.21.15.162:0/2779021551 <== mon.0 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 39) v1 ==== 92747+0+0 (secure 0 0 0) 0x7fcbb400bd30 con 0x7fcbc0101d10 2024-04-27T04:16:47.342 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:16:47.343+0000 7fcbbd7fa700 1 --2- 172.21.15.162:0/2779021551 >> [v2:172.21.15.162:6800/1396866108,v1:172.21.15.162:6801/1396866108] conn(0x7fcbb006ec00 0x7fcbb00710b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:16:47.342 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:16:47.344+0000 7fcbbd7fa700 1 -- 172.21.15.162:0/2779021551 <== mon.0 v2:172.21.15.162:3300/0 5 ==== osd_map(59..59 src has 1..59) v4 ==== 6410+0+0 (secure 0 0 0) 0x7fcbb408fa00 con 0x7fcbc0101d10 2024-04-27T04:16:47.342 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:16:47.344+0000 7fcbbffff700 1 --2- 172.21.15.162:0/2779021551 >> [v2:172.21.15.162:6800/1396866108,v1:172.21.15.162:6801/1396866108] conn(0x7fcbb006ec00 0x7fcbb00710b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:16:47.342 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:16:47.344+0000 7fcbbffff700 1 --2- 172.21.15.162:0/2779021551 >> [v2:172.21.15.162:6800/1396866108,v1:172.21.15.162:6801/1396866108] conn(0x7fcbb006ec00 0x7fcbb00710b0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fcbac007fd0 tx=0x7fcbac009040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:16:47.346 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:16:47.348+0000 7fcbbd7fa700 1 -- 172.21.15.162:0/2779021551 <== mon.0 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7fcbb4058880 con 0x7fcbc0101d10 2024-04-27T04:16:47.637 INFO:journalctl@ceph.osd.2.smithi162.stdout:Apr 27 04:16:47 smithi162 systemd[1]: Stopping Ceph osd.2 for 7f9750d6-044a-11ef-bc93-c7b262605968... 2024-04-27T04:16:47.637 INFO:journalctl@ceph.osd.2.smithi162.stdout:Apr 27 04:16:47 smithi162 ceph-7f9750d6-044a-11ef-bc93-c7b262605968-osd-2[60531]: 2024-04-27T04:16:47.458+0000 7f4a5c32b700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.2 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-27T04:16:47.637 INFO:journalctl@ceph.osd.2.smithi162.stdout:Apr 27 04:16:47 smithi162 ceph-7f9750d6-044a-11ef-bc93-c7b262605968-osd-2[60531]: 2024-04-27T04:16:47.458+0000 7f4a5c32b700 -1 osd.2 59 *** Got signal Terminated *** 2024-04-27T04:16:47.637 INFO:journalctl@ceph.osd.2.smithi162.stdout:Apr 27 04:16:47 smithi162 ceph-7f9750d6-044a-11ef-bc93-c7b262605968-osd-2[60531]: 2024-04-27T04:16:47.458+0000 7f4a5c32b700 -1 osd.2 59 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-27T04:16:47.646 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:16:47.648+0000 7fcbc6fcd700 1 -- 172.21.15.162:0/2779021551 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7fcbc00ff020 con 0x7fcbc0101d10 2024-04-27T04:16:47.646 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:16:47.648+0000 7fcbbd7fa700 1 -- 172.21.15.162:0/2779021551 <== mon.0 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+839 (secure 0 0 0) 0x7fcbb405bf40 con 0x7fcbc0101d10 2024-04-27T04:16:47.646 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-04-27T04:16:47.647 INFO:teuthology.orchestra.run.smithi162.stdout: "mon": { 2024-04-27T04:16:47.647 INFO:teuthology.orchestra.run.smithi162.stdout: "ceph version 19.0.0-3253-g4e213a58 (4e213a58e787437ff508e6c890ccd5fd32dc96b5) squid (dev)": 2 2024-04-27T04:16:47.647 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-04-27T04:16:47.647 INFO:teuthology.orchestra.run.smithi162.stdout: "mgr": { 2024-04-27T04:16:47.647 INFO:teuthology.orchestra.run.smithi162.stdout: "ceph version 19.0.0-3253-g4e213a58 (4e213a58e787437ff508e6c890ccd5fd32dc96b5) squid (dev)": 2 2024-04-27T04:16:47.647 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-04-27T04:16:47.647 INFO:teuthology.orchestra.run.smithi162.stdout: "osd": { 2024-04-27T04:16:47.647 INFO:teuthology.orchestra.run.smithi162.stdout: "ceph version 17.2.7-907-g2be3efa9 (2be3efa97a7972f8e925bd899b474329979b74d6) quincy (stable)": 4, 2024-04-27T04:16:47.647 INFO:teuthology.orchestra.run.smithi162.stdout: "ceph version 19.0.0-3253-g4e213a58 (4e213a58e787437ff508e6c890ccd5fd32dc96b5) squid (dev)": 2 2024-04-27T04:16:47.647 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-04-27T04:16:47.647 INFO:teuthology.orchestra.run.smithi162.stdout: "mds": { 2024-04-27T04:16:47.647 INFO:teuthology.orchestra.run.smithi162.stdout: "ceph version 17.2.7-907-g2be3efa9 (2be3efa97a7972f8e925bd899b474329979b74d6) quincy (stable)": 4 2024-04-27T04:16:47.647 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-04-27T04:16:47.647 INFO:teuthology.orchestra.run.smithi162.stdout: "overall": { 2024-04-27T04:16:47.648 INFO:teuthology.orchestra.run.smithi162.stdout: "ceph version 17.2.7-907-g2be3efa9 (2be3efa97a7972f8e925bd899b474329979b74d6) quincy (stable)": 8, 2024-04-27T04:16:47.648 INFO:teuthology.orchestra.run.smithi162.stdout: "ceph version 19.0.0-3253-g4e213a58 (4e213a58e787437ff508e6c890ccd5fd32dc96b5) squid (dev)": 6 2024-04-27T04:16:47.648 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-04-27T04:16:47.648 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-04-27T04:16:47.648 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:16:47.650+0000 7fcbc6fcd700 1 -- 172.21.15.162:0/2779021551 >> [v2:172.21.15.162:6800/1396866108,v1:172.21.15.162:6801/1396866108] conn(0x7fcbb006ec00 msgr2=0x7fcbb00710b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:16:47.649 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:16:47.650+0000 7fcbc6fcd700 1 --2- 172.21.15.162:0/2779021551 >> [v2:172.21.15.162:6800/1396866108,v1:172.21.15.162:6801/1396866108] conn(0x7fcbb006ec00 0x7fcbb00710b0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fcbac007fd0 tx=0x7fcbac009040 comp rx=0 tx=0).stop 2024-04-27T04:16:47.649 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:16:47.650+0000 7fcbc6fcd700 1 -- 172.21.15.162:0/2779021551 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fcbc0101d10 msgr2=0x7fcbc0106bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:16:47.649 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:16:47.650+0000 7fcbc6fcd700 1 --2- 172.21.15.162:0/2779021551 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fcbc0101d10 0x7fcbc0106bb0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fcbb400b3b0 tx=0x7fcbb4009e30 comp rx=0 tx=0).stop 2024-04-27T04:16:47.649 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:16:47.651+0000 7fcbc6fcd700 1 -- 172.21.15.162:0/2779021551 shutdown_connections 2024-04-27T04:16:47.649 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:16:47.651+0000 7fcbc6fcd700 1 --2- 172.21.15.162:0/2779021551 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fcbc0101d10 0x7fcbc0106bb0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:16:47.649 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:16:47.651+0000 7fcbc6fcd700 1 --2- 172.21.15.162:0/2779021551 >> [v2:172.21.15.162:6800/1396866108,v1:172.21.15.162:6801/1396866108] conn(0x7fcbb006ec00 0x7fcbb00710b0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:16:47.649 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:16:47.651+0000 7fcbc6fcd700 1 --2- 172.21.15.162:0/2779021551 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcbc01026b0 0x7fcbc01070f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:16:47.649 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:16:47.651+0000 7fcbc6fcd700 1 -- 172.21.15.162:0/2779021551 >> 172.21.15.162:0/2779021551 conn(0x7fcbc00f7860 msgr2=0x7fcbc01062b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:16:47.649 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:16:47.651+0000 7fcbc6fcd700 1 -- 172.21.15.162:0/2779021551 shutdown_connections 2024-04-27T04:16:47.649 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:16:47.651+0000 7fcbc6fcd700 1 -- 172.21.15.162:0/2779021551 wait complete. 2024-04-27T04:16:47.757 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:16:47.758+0000 7fa3d31e4700 1 -- 172.21.15.162:0/2196755903 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa3cc101d00 msgr2=0x7fa3cc1020d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:16:47.757 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:16:47.758+0000 7fa3d31e4700 1 --2- 172.21.15.162:0/2196755903 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa3cc101d00 0x7fa3cc1020d0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fa3c0004940 tx=0x7fa3c0009e30 comp rx=0 tx=0).stop 2024-04-27T04:16:47.757 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:16:47.758+0000 7fa3d31e4700 1 -- 172.21.15.162:0/2196755903 shutdown_connections 2024-04-27T04:16:47.757 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:16:47.758+0000 7fa3d31e4700 1 --2- 172.21.15.162:0/2196755903 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa3cc1026a0 0x7fa3cc0fbd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:16:47.757 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:16:47.758+0000 7fa3d31e4700 1 --2- 172.21.15.162:0/2196755903 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa3cc101d00 0x7fa3cc1020d0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:16:47.757 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:16:47.758+0000 7fa3d31e4700 1 -- 172.21.15.162:0/2196755903 >> 172.21.15.162:0/2196755903 conn(0x7fa3cc0f7870 msgr2=0x7fa3cc0f9c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:16:47.757 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:16:47.758+0000 7fa3d31e4700 1 -- 172.21.15.162:0/2196755903 shutdown_connections 2024-04-27T04:16:47.757 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:16:47.758+0000 7fa3d31e4700 1 -- 172.21.15.162:0/2196755903 wait complete. 2024-04-27T04:16:47.758 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:16:47.759+0000 7fa3d31e4700 1 Processor -- start 2024-04-27T04:16:47.758 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:16:47.759+0000 7fa3d31e4700 1 -- start start 2024-04-27T04:16:47.759 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:16:47.759+0000 7fa3d31e4700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa3cc101d00 0x7fa3cc112890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:16:47.759 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:16:47.760+0000 7fa3d31e4700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa3cc1026a0 0x7fa3cc112dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:16:47.759 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:16:47.760+0000 7fa3d31e4700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3cc1134b0 con 0x7fa3cc101d00 2024-04-27T04:16:47.759 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:16:47.760+0000 7fa3d31e4700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3cc10ca20 con 0x7fa3cc1026a0 2024-04-27T04:16:47.759 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:16:47.760+0000 7fa3d0f80700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa3cc101d00 0x7fa3cc112890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:16:47.759 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:16:47.760+0000 7fa3d0f80700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa3cc101d00 0x7fa3cc112890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:38204/0 (socket says 172.21.15.162:38204) 2024-04-27T04:16:47.760 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:16:47.760+0000 7fa3d0f80700 1 -- 172.21.15.162:0/674530927 learned_addr learned my addr 172.21.15.162:0/674530927 (peer_addr_for_me v2:172.21.15.162:0/0) 2024-04-27T04:16:47.760 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:16:47.760+0000 7fa3cbfff700 1 --2- 172.21.15.162:0/674530927 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa3cc1026a0 0x7fa3cc112dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:16:47.760 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:16:47.760+0000 7fa3d0f80700 1 -- 172.21.15.162:0/674530927 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa3cc1026a0 msgr2=0x7fa3cc112dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:16:47.760 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:16:47.760+0000 7fa3d0f80700 1 --2- 172.21.15.162:0/674530927 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa3cc1026a0 0x7fa3cc112dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:16:47.760 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:16:47.760+0000 7fa3d0f80700 1 -- 172.21.15.162:0/674530927 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa3c0005040 con 0x7fa3cc101d00 2024-04-27T04:16:47.760 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:16:47.760+0000 7fa3d0f80700 1 --2- 172.21.15.162:0/674530927 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa3cc101d00 0x7fa3cc112890 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fa3c0009180 tx=0x7fa3c0009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:16:47.760 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:16:47.760+0000 7fa3c97fa700 1 -- 172.21.15.162:0/674530927 <== mon.0 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa3c000e040 con 0x7fa3cc101d00 2024-04-27T04:16:47.760 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:16:47.760+0000 7fa3d31e4700 1 -- 172.21.15.162:0/674530927 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa3cc10cca0 con 0x7fa3cc101d00 2024-04-27T04:16:47.760 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:16:47.760+0000 7fa3c97fa700 1 -- 172.21.15.162:0/674530927 <== mon.0 v2:172.21.15.162:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fa3c000bbc0 con 0x7fa3cc101d00 2024-04-27T04:16:47.760 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:16:47.760+0000 7fa3c97fa700 1 -- 172.21.15.162:0/674530927 <== mon.0 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa3c001c630 con 0x7fa3cc101d00 2024-04-27T04:16:47.760 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:16:47.760+0000 7fa3d31e4700 1 -- 172.21.15.162:0/674530927 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa3cc10d260 con 0x7fa3cc101d00 2024-04-27T04:16:47.760 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:16:47.761+0000 7fa3d31e4700 1 -- 172.21.15.162:0/674530927 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa3cc0fd450 con 0x7fa3cc101d00 2024-04-27T04:16:47.761 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:16:47.762+0000 7fa3c97fa700 1 -- 172.21.15.162:0/674530927 <== mon.0 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 39) v1 ==== 92747+0+0 (secure 0 0 0) 0x7fa3c000bd30 con 0x7fa3cc101d00 2024-04-27T04:16:47.761 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:16:47.762+0000 7fa3c97fa700 1 --2- 172.21.15.162:0/674530927 >> [v2:172.21.15.162:6800/1396866108,v1:172.21.15.162:6801/1396866108] conn(0x7fa3bc06ec00 0x7fa3bc0710b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:16:47.761 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:16:47.762+0000 7fa3c97fa700 1 -- 172.21.15.162:0/674530927 <== mon.0 v2:172.21.15.162:3300/0 5 ==== osd_map(60..60 src has 1..60) v4 ==== 6410+0+0 (secure 0 0 0) 0x7fa3c008fc40 con 0x7fa3cc101d00 2024-04-27T04:16:47.761 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:16:47.762+0000 7fa3cbfff700 1 --2- 172.21.15.162:0/674530927 >> [v2:172.21.15.162:6800/1396866108,v1:172.21.15.162:6801/1396866108] conn(0x7fa3bc06ec00 0x7fa3bc0710b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:16:47.761 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:16:47.763+0000 7fa3cbfff700 1 --2- 172.21.15.162:0/674530927 >> [v2:172.21.15.162:6800/1396866108,v1:172.21.15.162:6801/1396866108] conn(0x7fa3bc06ec00 0x7fa3bc0710b0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fa3b8006fd0 tx=0x7fa3b8008040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:16:47.765 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:16:47.767+0000 7fa3c97fa700 1 -- 172.21.15.162:0/674530927 <== mon.0 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7fa3c005c5f0 con 0x7fa3cc101d00 2024-04-27T04:16:48.014 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:16:48.016+0000 7fa3d31e4700 1 -- 172.21.15.162:0/674530927 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7fa3cc108f00 con 0x7fa3cc101d00 2024-04-27T04:16:48.015 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:16:47 smithi162 ceph-mon[110641]: from='client.34252 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-27T04:16:48.015 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:16:47 smithi162 ceph-mon[110641]: osd.2 marked itself down and dead 2024-04-27T04:16:48.015 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:16:47 smithi162 ceph-mon[110641]: from='client.? 172.21.15.162:0/2779021551' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-27T04:16:48.015 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:16:47 smithi162 ceph-mon[110641]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-27T04:16:48.015 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:16:48.017+0000 7fa3c97fa700 1 -- 172.21.15.162:0/674530927 <== mon.0 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 15 v15) v1 ==== 76+0+1911 (secure 0 0 0) 0x7fa3c005c180 con 0x7fa3cc101d00 2024-04-27T04:16:48.015 INFO:teuthology.orchestra.run.smithi162.stdout:e15 2024-04-27T04:16:48.016 INFO:teuthology.orchestra.run.smithi162.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-04-27T04:16:48.016 INFO:teuthology.orchestra.run.smithi162.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-27T04:16:48.016 INFO:teuthology.orchestra.run.smithi162.stdout:legacy client fscid: 1 2024-04-27T04:16:48.016 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-04-27T04:16:48.016 INFO:teuthology.orchestra.run.smithi162.stdout:Filesystem 'cephfs' (1) 2024-04-27T04:16:48.016 INFO:teuthology.orchestra.run.smithi162.stdout:fs_name cephfs 2024-04-27T04:16:48.016 INFO:teuthology.orchestra.run.smithi162.stdout:epoch 15 2024-04-27T04:16:48.016 INFO:teuthology.orchestra.run.smithi162.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-04-27T04:16:48.016 INFO:teuthology.orchestra.run.smithi162.stdout:created 2024-04-27T04:06:53.095749+0000 2024-04-27T04:16:48.016 INFO:teuthology.orchestra.run.smithi162.stdout:modified 2024-04-27T04:07:11.285033+0000 2024-04-27T04:16:48.016 INFO:teuthology.orchestra.run.smithi162.stdout:tableserver 0 2024-04-27T04:16:48.016 INFO:teuthology.orchestra.run.smithi162.stdout:root 0 2024-04-27T04:16:48.016 INFO:teuthology.orchestra.run.smithi162.stdout:session_timeout 60 2024-04-27T04:16:48.016 INFO:teuthology.orchestra.run.smithi162.stdout:session_autoclose 300 2024-04-27T04:16:48.016 INFO:teuthology.orchestra.run.smithi162.stdout:max_file_size 1099511627776 2024-04-27T04:16:48.017 INFO:teuthology.orchestra.run.smithi162.stdout:max_xattr_size 65536 2024-04-27T04:16:48.017 INFO:teuthology.orchestra.run.smithi162.stdout:required_client_features {} 2024-04-27T04:16:48.017 INFO:teuthology.orchestra.run.smithi162.stdout:last_failure 0 2024-04-27T04:16:48.017 INFO:teuthology.orchestra.run.smithi162.stdout:last_failure_osd_epoch 46 2024-04-27T04:16:48.017 INFO:teuthology.orchestra.run.smithi162.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-27T04:16:48.017 INFO:teuthology.orchestra.run.smithi162.stdout:max_mds 1 2024-04-27T04:16:48.017 INFO:teuthology.orchestra.run.smithi162.stdout:in 0 2024-04-27T04:16:48.017 INFO:teuthology.orchestra.run.smithi162.stdout:up {0=24273} 2024-04-27T04:16:48.017 INFO:teuthology.orchestra.run.smithi162.stdout:failed 2024-04-27T04:16:48.017 INFO:teuthology.orchestra.run.smithi162.stdout:damaged 2024-04-27T04:16:48.017 INFO:teuthology.orchestra.run.smithi162.stdout:stopped 2024-04-27T04:16:48.017 INFO:teuthology.orchestra.run.smithi162.stdout:data_pools [3] 2024-04-27T04:16:48.017 INFO:teuthology.orchestra.run.smithi162.stdout:metadata_pool 2 2024-04-27T04:16:48.017 INFO:teuthology.orchestra.run.smithi162.stdout:inline_data disabled 2024-04-27T04:16:48.017 INFO:teuthology.orchestra.run.smithi162.stdout:balancer 2024-04-27T04:16:48.018 INFO:teuthology.orchestra.run.smithi162.stdout:bal_rank_mask -1 2024-04-27T04:16:48.018 INFO:teuthology.orchestra.run.smithi162.stdout:standby_count_wanted 1 2024-04-27T04:16:48.018 INFO:teuthology.orchestra.run.smithi162.stdout:qdb_cluster leader: 0 members: 2024-04-27T04:16:48.018 INFO:teuthology.orchestra.run.smithi162.stdout:[mds.cephfs.smithi181.dzomgh{0:24273} state up:active seq 6 join_fscid=1 addr [v2:172.21.15.181:6824/658214993,v1:172.21.15.181:6825/658214993] compat {c=[1],r=[1],i=[7ff]}] 2024-04-27T04:16:48.018 INFO:teuthology.orchestra.run.smithi162.stdout:[mds.cephfs.smithi162.xcnapk{0:14476} state up:standby-replay seq 2 join_fscid=1 addr [v2:172.21.15.162:6828/1296160676,v1:172.21.15.162:6829/1296160676] compat {c=[1],r=[1],i=[7ff]}] 2024-04-27T04:16:48.018 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-04-27T04:16:48.018 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-04-27T04:16:48.018 INFO:teuthology.orchestra.run.smithi162.stdout:Standby daemons: 2024-04-27T04:16:48.018 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-04-27T04:16:48.018 INFO:teuthology.orchestra.run.smithi162.stdout:[mds.cephfs.smithi162.gzlyjj{-1:14488} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.162:6826/2629103894,v1:172.21.15.162:6827/2629103894] compat {c=[1],r=[1],i=[7ff]}] 2024-04-27T04:16:48.018 INFO:teuthology.orchestra.run.smithi162.stdout:[mds.cephfs.smithi181.bkkcdm{-1:24287} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.181:6826/3756475453,v1:172.21.15.181:6827/3756475453] compat {c=[1],r=[1],i=[7ff]}] 2024-04-27T04:16:48.019 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:16:48.019+0000 7fa3d31e4700 1 -- 172.21.15.162:0/674530927 >> [v2:172.21.15.162:6800/1396866108,v1:172.21.15.162:6801/1396866108] conn(0x7fa3bc06ec00 msgr2=0x7fa3bc0710b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:16:48.019 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:16:48.019+0000 7fa3d31e4700 1 --2- 172.21.15.162:0/674530927 >> [v2:172.21.15.162:6800/1396866108,v1:172.21.15.162:6801/1396866108] conn(0x7fa3bc06ec00 0x7fa3bc0710b0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fa3b8006fd0 tx=0x7fa3b8008040 comp rx=0 tx=0).stop 2024-04-27T04:16:48.019 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:16:48.019+0000 7fa3d31e4700 1 -- 172.21.15.162:0/674530927 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa3cc101d00 msgr2=0x7fa3cc112890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:16:48.019 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:16:48.019+0000 7fa3d31e4700 1 --2- 172.21.15.162:0/674530927 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa3cc101d00 0x7fa3cc112890 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fa3c0009180 tx=0x7fa3c0009e30 comp rx=0 tx=0).stop 2024-04-27T04:16:48.019 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:16:48.019+0000 7fa3d31e4700 1 -- 172.21.15.162:0/674530927 shutdown_connections 2024-04-27T04:16:48.019 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:16:48.019+0000 7fa3d31e4700 1 --2- 172.21.15.162:0/674530927 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa3cc101d00 0x7fa3cc112890 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:16:48.019 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:16:48.019+0000 7fa3d31e4700 1 --2- 172.21.15.162:0/674530927 >> [v2:172.21.15.162:6800/1396866108,v1:172.21.15.162:6801/1396866108] conn(0x7fa3bc06ec00 0x7fa3bc0710b0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:16:48.019 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:16:48.019+0000 7fa3d31e4700 1 --2- 172.21.15.162:0/674530927 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa3cc1026a0 0x7fa3cc112dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:16:48.020 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:16:48.019+0000 7fa3d31e4700 1 -- 172.21.15.162:0/674530927 >> 172.21.15.162:0/674530927 conn(0x7fa3cc0f7870 msgr2=0x7fa3cc0f8800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:16:48.020 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:16:48.019+0000 7fa3d31e4700 1 -- 172.21.15.162:0/674530927 shutdown_connections 2024-04-27T04:16:48.020 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:16:48.019+0000 7fa3d31e4700 1 -- 172.21.15.162:0/674530927 wait complete. 2024-04-27T04:16:48.020 INFO:teuthology.orchestra.run.smithi162.stderr:dumped fsmap epoch 15 2024-04-27T04:16:48.125 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:16:48.126+0000 7f0e142b7700 1 -- 172.21.15.162:0/2211867077 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0e0c102de0 msgr2=0x7f0e0c1031b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:16:48.125 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:16:48.126+0000 7f0e142b7700 1 --2- 172.21.15.162:0/2211867077 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0e0c102de0 0x7f0e0c1031b0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f0e08004940 tx=0x7f0e08009e30 comp rx=0 tx=0).stop 2024-04-27T04:16:48.125 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:16:48.127+0000 7f0e142b7700 1 -- 172.21.15.162:0/2211867077 shutdown_connections 2024-04-27T04:16:48.125 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:16:48.127+0000 7f0e142b7700 1 --2- 172.21.15.162:0/2211867077 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0e0c103780 0x7f0e0c0fcdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:16:48.125 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:16:48.127+0000 7f0e142b7700 1 --2- 172.21.15.162:0/2211867077 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0e0c102de0 0x7f0e0c1031b0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:16:48.125 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:16:48.127+0000 7f0e142b7700 1 -- 172.21.15.162:0/2211867077 >> 172.21.15.162:0/2211867077 conn(0x7f0e0c0f8990 msgr2=0x7f0e0c0fada0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:16:48.125 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:16:48.127+0000 7f0e142b7700 1 -- 172.21.15.162:0/2211867077 shutdown_connections 2024-04-27T04:16:48.125 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:16:48.127+0000 7f0e142b7700 1 -- 172.21.15.162:0/2211867077 wait complete. 2024-04-27T04:16:48.127 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:16:48.127+0000 7f0e142b7700 1 Processor -- start 2024-04-27T04:16:48.127 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:16:48.127+0000 7f0e142b7700 1 -- start start 2024-04-27T04:16:48.127 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:16:48.128+0000 7f0e142b7700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0e0c102de0 0x7f0e0c113970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:16:48.127 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:16:48.128+0000 7f0e142b7700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0e0c103780 0x7f0e0c113eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:16:48.128 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:16:48.128+0000 7f0e142b7700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e0c114500 con 0x7f0e0c102de0 2024-04-27T04:16:48.128 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:16:48.128+0000 7f0e142b7700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e0c10db50 con 0x7f0e0c103780 2024-04-27T04:16:48.128 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:16:48.128+0000 7f0e12053700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0e0c102de0 0x7f0e0c113970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:16:48.128 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:16:48.128+0000 7f0e12053700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0e0c102de0 0x7f0e0c113970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:38226/0 (socket says 172.21.15.162:38226) 2024-04-27T04:16:48.128 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:16:48.128+0000 7f0e12053700 1 -- 172.21.15.162:0/2222842159 learned_addr learned my addr 172.21.15.162:0/2222842159 (peer_addr_for_me v2:172.21.15.162:0/0) 2024-04-27T04:16:48.128 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:16:48.128+0000 7f0e12053700 1 -- 172.21.15.162:0/2222842159 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0e0c103780 msgr2=0x7f0e0c113eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:16:48.128 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:16:48.128+0000 7f0e11852700 1 --2- 172.21.15.162:0/2222842159 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0e0c103780 0x7f0e0c113eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:16:48.128 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:16:48.128+0000 7f0e12053700 1 --2- 172.21.15.162:0/2222842159 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0e0c103780 0x7f0e0c113eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:16:48.128 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:16:48.128+0000 7f0e12053700 1 -- 172.21.15.162:0/2222842159 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e08005040 con 0x7f0e0c102de0 2024-04-27T04:16:48.128 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:16:48.128+0000 7f0e11852700 1 --2- 172.21.15.162:0/2222842159 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0e0c103780 0x7f0e0c113eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-27T04:16:48.128 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:16:48.128+0000 7f0e12053700 1 --2- 172.21.15.162:0/2222842159 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0e0c102de0 0x7f0e0c113970 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f0e0800b3b0 tx=0x7f0e08009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:16:48.128 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:16:48.128+0000 7f0e02ffd700 1 -- 172.21.15.162:0/2222842159 <== mon.0 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0e0800e040 con 0x7f0e0c102de0 2024-04-27T04:16:48.128 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:16:48.128+0000 7f0e142b7700 1 -- 172.21.15.162:0/2222842159 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0e0c10ddd0 con 0x7f0e0c102de0 2024-04-27T04:16:48.128 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:16:48.128+0000 7f0e02ffd700 1 -- 172.21.15.162:0/2222842159 <== mon.0 v2:172.21.15.162:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f0e0800bbc0 con 0x7f0e0c102de0 2024-04-27T04:16:48.129 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:16:48.128+0000 7f0e02ffd700 1 -- 172.21.15.162:0/2222842159 <== mon.0 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0e0801c3f0 con 0x7f0e0c102de0 2024-04-27T04:16:48.129 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:16:48.128+0000 7f0e142b7700 1 -- 172.21.15.162:0/2222842159 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0e0c10e390 con 0x7f0e0c102de0 2024-04-27T04:16:48.129 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:16:48.129+0000 7f0e142b7700 1 -- 172.21.15.162:0/2222842159 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0e0c1000f0 con 0x7f0e0c102de0 2024-04-27T04:16:48.129 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:16:48.130+0000 7f0e02ffd700 1 -- 172.21.15.162:0/2222842159 <== mon.0 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 39) v1 ==== 92747+0+0 (secure 0 0 0) 0x7f0e0800bd30 con 0x7f0e0c102de0 2024-04-27T04:16:48.129 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:16:48.130+0000 7f0e02ffd700 1 --2- 172.21.15.162:0/2222842159 >> [v2:172.21.15.162:6800/1396866108,v1:172.21.15.162:6801/1396866108] conn(0x7f0df806ec00 0x7f0df80710b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:16:48.130 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:16:48.130+0000 7f0e02ffd700 1 -- 172.21.15.162:0/2222842159 <== mon.0 v2:172.21.15.162:3300/0 5 ==== osd_map(60..60 src has 1..60) v4 ==== 6410+0+0 (secure 0 0 0) 0x7f0e0808fb00 con 0x7f0e0c102de0 2024-04-27T04:16:48.130 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:16:48.130+0000 7f0e11852700 1 --2- 172.21.15.162:0/2222842159 >> [v2:172.21.15.162:6800/1396866108,v1:172.21.15.162:6801/1396866108] conn(0x7f0df806ec00 0x7f0df80710b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:16:48.130 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:16:48.131+0000 7f0e11852700 1 --2- 172.21.15.162:0/2222842159 >> [v2:172.21.15.162:6800/1396866108,v1:172.21.15.162:6801/1396866108] conn(0x7f0df806ec00 0x7f0df80710b0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f0dfc007fd0 tx=0x7f0dfc009040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:16:48.133 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:16:48.135+0000 7f0e02ffd700 1 -- 172.21.15.162:0/2222842159 <== mon.0 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7f0e0805c430 con 0x7f0e0c102de0 2024-04-27T04:16:48.172 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:16:47 smithi181 ceph-mon[81134]: from='client.34252 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-27T04:16:48.172 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:16:47 smithi181 ceph-mon[81134]: osd.2 marked itself down and dead 2024-04-27T04:16:48.172 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:16:47 smithi181 ceph-mon[81134]: from='client.? 172.21.15.162:0/2779021551' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-27T04:16:48.172 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:16:47 smithi181 ceph-mon[81134]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-27T04:16:48.340 INFO:journalctl@ceph.osd.2.smithi162.stdout:Apr 27 04:16:48 smithi162 podman[132360]: 2024-04-27 04:16:48.088102804 +0000 UTC m=+0.723060387 container died 70f16d86d38e96c0d1b624f6d1bd488d7691dc54ee4a6bd1f99e063d422ecf5d (image=quay.ceph.io/ceph-ci/ceph@sha256:d31f278ece6dbd17d1fa5e598f509017f54b76fafe1c4416391fcc51b9718ee9, name=ceph-7f9750d6-044a-11ef-bc93-c7b262605968-osd-2, org.label-schema.license=GPLv2, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240424, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.vendor=CentOS, RELEASE=quincy-2be3efa) 2024-04-27T04:16:48.356 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:16:48.358+0000 7f0e142b7700 1 -- 172.21.15.162:0/2222842159 --> [v2:172.21.15.162:6800/1396866108,v1:172.21.15.162:6801/1396866108] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f0e0c10f030 con 0x7f0df806ec00 2024-04-27T04:16:48.357 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:16:48.359+0000 7f0e02ffd700 1 -- 172.21.15.162:0/2222842159 <== mgr.34104 v2:172.21.15.162:6800/1396866108 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+400 (secure 0 0 0) 0x7f0e0c10f030 con 0x7f0df806ec00 2024-04-27T04:16:48.357 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-04-27T04:16:48.357 INFO:teuthology.orchestra.run.smithi162.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:90a3a9fa9ceb8977f6dad9ccd36fe46018d07e2118199820523a6b35301d9c16", 2024-04-27T04:16:48.357 INFO:teuthology.orchestra.run.smithi162.stdout: "in_progress": true, 2024-04-27T04:16:48.357 INFO:teuthology.orchestra.run.smithi162.stdout: "which": "Upgrading all daemon types on all hosts", 2024-04-27T04:16:48.357 INFO:teuthology.orchestra.run.smithi162.stdout: "services_complete": [ 2024-04-27T04:16:48.357 INFO:teuthology.orchestra.run.smithi162.stdout: "crash", 2024-04-27T04:16:48.357 INFO:teuthology.orchestra.run.smithi162.stdout: "mon", 2024-04-27T04:16:48.357 INFO:teuthology.orchestra.run.smithi162.stdout: "mgr" 2024-04-27T04:16:48.357 INFO:teuthology.orchestra.run.smithi162.stdout: ], 2024-04-27T04:16:48.358 INFO:teuthology.orchestra.run.smithi162.stdout: "progress": "8/21 daemons upgraded", 2024-04-27T04:16:48.358 INFO:teuthology.orchestra.run.smithi162.stdout: "message": "Currently upgrading osd daemons", 2024-04-27T04:16:48.358 INFO:teuthology.orchestra.run.smithi162.stdout: "is_paused": false 2024-04-27T04:16:48.358 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-04-27T04:16:48.359 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:16:48.361+0000 7f0e142b7700 1 -- 172.21.15.162:0/2222842159 >> [v2:172.21.15.162:6800/1396866108,v1:172.21.15.162:6801/1396866108] conn(0x7f0df806ec00 msgr2=0x7f0df80710b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:16:48.359 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:16:48.361+0000 7f0e142b7700 1 --2- 172.21.15.162:0/2222842159 >> [v2:172.21.15.162:6800/1396866108,v1:172.21.15.162:6801/1396866108] conn(0x7f0df806ec00 0x7f0df80710b0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f0dfc007fd0 tx=0x7f0dfc009040 comp rx=0 tx=0).stop 2024-04-27T04:16:48.359 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:16:48.361+0000 7f0e142b7700 1 -- 172.21.15.162:0/2222842159 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0e0c102de0 msgr2=0x7f0e0c113970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:16:48.359 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:16:48.361+0000 7f0e142b7700 1 --2- 172.21.15.162:0/2222842159 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0e0c102de0 0x7f0e0c113970 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f0e0800b3b0 tx=0x7f0e08009e30 comp rx=0 tx=0).stop 2024-04-27T04:16:48.359 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:16:48.361+0000 7f0e142b7700 1 -- 172.21.15.162:0/2222842159 shutdown_connections 2024-04-27T04:16:48.360 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:16:48.361+0000 7f0e142b7700 1 --2- 172.21.15.162:0/2222842159 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0e0c102de0 0x7f0e0c113970 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:16:48.360 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:16:48.361+0000 7f0e142b7700 1 --2- 172.21.15.162:0/2222842159 >> [v2:172.21.15.162:6800/1396866108,v1:172.21.15.162:6801/1396866108] conn(0x7f0df806ec00 0x7f0df80710b0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:16:48.360 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:16:48.361+0000 7f0e142b7700 1 --2- 172.21.15.162:0/2222842159 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0e0c103780 0x7f0e0c113eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:16:48.360 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:16:48.361+0000 7f0e142b7700 1 -- 172.21.15.162:0/2222842159 >> 172.21.15.162:0/2222842159 conn(0x7f0e0c0f8990 msgr2=0x7f0e0c107380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:16:48.360 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:16:48.361+0000 7f0e142b7700 1 -- 172.21.15.162:0/2222842159 shutdown_connections 2024-04-27T04:16:48.360 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:16:48.361+0000 7f0e142b7700 1 -- 172.21.15.162:0/2222842159 wait complete. 2024-04-27T04:16:48.462 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:16:48.463+0000 7f7eec93a700 1 -- 172.21.15.162:0/1744487963 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7ee80fbd40 msgr2=0x7f7ee80fc1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:16:48.463 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:16:48.463+0000 7f7eec93a700 1 --2- 172.21.15.162:0/1744487963 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7ee80fbd40 0x7f7ee80fc1b0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f7edc004940 tx=0x7f7edc009e30 comp rx=0 tx=0).stop 2024-04-27T04:16:48.463 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:16:48.463+0000 7f7eec93a700 1 -- 172.21.15.162:0/1744487963 shutdown_connections 2024-04-27T04:16:48.463 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:16:48.463+0000 7f7eec93a700 1 --2- 172.21.15.162:0/1744487963 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7ee80fbd40 0x7f7ee80fc1b0 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:16:48.463 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:16:48.463+0000 7f7eec93a700 1 --2- 172.21.15.162:0/1744487963 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7ee8101d90 0x7f7ee8102160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:16:48.463 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:16:48.463+0000 7f7eec93a700 1 -- 172.21.15.162:0/1744487963 >> 172.21.15.162:0/1744487963 conn(0x7f7ee8071d00 msgr2=0x7f7ee8072100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:16:48.463 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:16:48.463+0000 7f7eec93a700 1 -- 172.21.15.162:0/1744487963 shutdown_connections 2024-04-27T04:16:48.463 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:16:48.463+0000 7f7eec93a700 1 -- 172.21.15.162:0/1744487963 wait complete. 2024-04-27T04:16:48.463 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:16:48.463+0000 7f7eec93a700 1 Processor -- start 2024-04-27T04:16:48.463 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:16:48.463+0000 7f7eec93a700 1 -- start start 2024-04-27T04:16:48.463 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:16:48.464+0000 7f7eec93a700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7ee80fbd40 0x7f7ee8113bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:16:48.463 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:16:48.464+0000 7f7eec93a700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7ee8101d90 0x7f7ee81140f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:16:48.463 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:16:48.464+0000 7f7eec93a700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ee8114740 con 0x7f7ee80fbd40 2024-04-27T04:16:48.463 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:16:48.464+0000 7f7eec93a700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ee810db30 con 0x7f7ee8101d90 2024-04-27T04:16:48.464 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:16:48.464+0000 7f7ee659c700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7ee80fbd40 0x7f7ee8113bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:16:48.464 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:16:48.464+0000 7f7ee659c700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7ee80fbd40 0x7f7ee8113bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:38238/0 (socket says 172.21.15.162:38238) 2024-04-27T04:16:48.464 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:16:48.464+0000 7f7ee659c700 1 -- 172.21.15.162:0/3896266071 learned_addr learned my addr 172.21.15.162:0/3896266071 (peer_addr_for_me v2:172.21.15.162:0/0) 2024-04-27T04:16:48.464 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:16:48.464+0000 7f7ee5d9b700 1 --2- 172.21.15.162:0/3896266071 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7ee8101d90 0x7f7ee81140f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:16:48.464 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:16:48.464+0000 7f7ee659c700 1 -- 172.21.15.162:0/3896266071 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7ee8101d90 msgr2=0x7f7ee81140f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:16:48.464 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:16:48.464+0000 7f7ee659c700 1 --2- 172.21.15.162:0/3896266071 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7ee8101d90 0x7f7ee81140f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:16:48.464 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:16:48.464+0000 7f7ee659c700 1 -- 172.21.15.162:0/3896266071 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7edc005040 con 0x7f7ee80fbd40 2024-04-27T04:16:48.464 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:16:48.464+0000 7f7ee659c700 1 --2- 172.21.15.162:0/3896266071 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7ee80fbd40 0x7f7ee8113bb0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f7ed800c8c0 tx=0x7f7ed80044c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:16:48.464 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:16:48.464+0000 7f7ed6ffd700 1 -- 172.21.15.162:0/3896266071 <== mon.0 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7ed800a980 con 0x7f7ee80fbd40 2024-04-27T04:16:48.464 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:16:48.464+0000 7f7ed6ffd700 1 -- 172.21.15.162:0/3896266071 <== mon.0 v2:172.21.15.162:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f7ed800ce60 con 0x7f7ee80fbd40 2024-04-27T04:16:48.464 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:16:48.464+0000 7f7eec93a700 1 -- 172.21.15.162:0/3896266071 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7ee810de10 con 0x7f7ee80fbd40 2024-04-27T04:16:48.464 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:16:48.465+0000 7f7ed6ffd700 1 -- 172.21.15.162:0/3896266071 <== mon.0 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7ed80115e0 con 0x7f7ee80fbd40 2024-04-27T04:16:48.464 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:16:48.465+0000 7f7eec93a700 1 -- 172.21.15.162:0/3896266071 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7ee810e3d0 con 0x7f7ee80fbd40 2024-04-27T04:16:48.465 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:16:48.466+0000 7f7eec93a700 1 -- 172.21.15.162:0/3896266071 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7ee80fd430 con 0x7f7ee80fbd40 2024-04-27T04:16:48.465 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:16:48.466+0000 7f7ed6ffd700 1 -- 172.21.15.162:0/3896266071 <== mon.0 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 39) v1 ==== 92747+0+0 (secure 0 0 0) 0x7f7ed8012400 con 0x7f7ee80fbd40 2024-04-27T04:16:48.465 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:16:48.466+0000 7f7ed6ffd700 1 --2- 172.21.15.162:0/3896266071 >> [v2:172.21.15.162:6800/1396866108,v1:172.21.15.162:6801/1396866108] conn(0x7f7ed006ec10 0x7f7ed00710c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:16:48.465 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:16:48.467+0000 7f7ee5d9b700 1 --2- 172.21.15.162:0/3896266071 >> [v2:172.21.15.162:6800/1396866108,v1:172.21.15.162:6801/1396866108] conn(0x7f7ed006ec10 0x7f7ed00710c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:16:48.465 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:16:48.467+0000 7f7ed6ffd700 1 -- 172.21.15.162:0/3896266071 <== mon.0 v2:172.21.15.162:3300/0 5 ==== osd_map(60..60 src has 1..60) v4 ==== 6410+0+0 (secure 0 0 0) 0x7f7ed808e9c0 con 0x7f7ee80fbd40 2024-04-27T04:16:48.466 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:16:48.467+0000 7f7ee5d9b700 1 --2- 172.21.15.162:0/3896266071 >> [v2:172.21.15.162:6800/1396866108,v1:172.21.15.162:6801/1396866108] conn(0x7f7ed006ec10 0x7f7ed00710c0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f7edc00b020 tx=0x7f7edc009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:16:48.469 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:16:48.471+0000 7f7ed6ffd700 1 -- 172.21.15.162:0/3896266071 <== mon.0 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7f7ed805b2d0 con 0x7f7ee80fbd40 2024-04-27T04:16:48.636 INFO:journalctl@ceph.osd.2.smithi162.stdout:Apr 27 04:16:48 smithi162 podman[132360]: 2024-04-27 04:16:48.466513902 +0000 UTC m=+1.101471484 container cleanup 70f16d86d38e96c0d1b624f6d1bd488d7691dc54ee4a6bd1f99e063d422ecf5d (image=quay.ceph.io/ceph-ci/ceph@sha256:d31f278ece6dbd17d1fa5e598f509017f54b76fafe1c4416391fcc51b9718ee9, name=ceph-7f9750d6-044a-11ef-bc93-c7b262605968-osd-2, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=quincy-2be3efa, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240424, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, io.buildah.version=1.33.5) 2024-04-27T04:16:48.636 INFO:journalctl@ceph.osd.2.smithi162.stdout:Apr 27 04:16:48 smithi162 bash[132360]: ceph-7f9750d6-044a-11ef-bc93-c7b262605968-osd-2 2024-04-27T04:16:48.782 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:16:48.784+0000 7f7eec93a700 1 -- 172.21.15.162:0/3896266071 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f7ee8108df0 con 0x7f7ee80fbd40 2024-04-27T04:16:48.783 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:16:48.785+0000 7f7ed6ffd700 1 -- 172.21.15.162:0/3896266071 <== mon.0 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+100 (secure 0 0 0) 0x7f7ed80126b0 con 0x7f7ee80fbd40 2024-04-27T04:16:48.783 INFO:teuthology.orchestra.run.smithi162.stdout:HEALTH_WARN 1 osds down 2024-04-27T04:16:48.783 INFO:teuthology.orchestra.run.smithi162.stdout:[WRN] OSD_DOWN: 1 osds down 2024-04-27T04:16:48.783 INFO:teuthology.orchestra.run.smithi162.stdout: osd.2 (root=default,host=smithi162) is down 2024-04-27T04:16:48.785 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:16:48.787+0000 7f7eec93a700 1 -- 172.21.15.162:0/3896266071 >> [v2:172.21.15.162:6800/1396866108,v1:172.21.15.162:6801/1396866108] conn(0x7f7ed006ec10 msgr2=0x7f7ed00710c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:16:48.785 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:16:48.787+0000 7f7eec93a700 1 --2- 172.21.15.162:0/3896266071 >> [v2:172.21.15.162:6800/1396866108,v1:172.21.15.162:6801/1396866108] conn(0x7f7ed006ec10 0x7f7ed00710c0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f7edc00b020 tx=0x7f7edc009e30 comp rx=0 tx=0).stop 2024-04-27T04:16:48.785 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:16:48.787+0000 7f7eec93a700 1 -- 172.21.15.162:0/3896266071 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7ee80fbd40 msgr2=0x7f7ee8113bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:16:48.785 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:16:48.787+0000 7f7eec93a700 1 --2- 172.21.15.162:0/3896266071 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7ee80fbd40 0x7f7ee8113bb0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f7ed800c8c0 tx=0x7f7ed80044c0 comp rx=0 tx=0).stop 2024-04-27T04:16:48.786 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:16:48.787+0000 7f7eec93a700 1 -- 172.21.15.162:0/3896266071 shutdown_connections 2024-04-27T04:16:48.786 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:16:48.787+0000 7f7eec93a700 1 --2- 172.21.15.162:0/3896266071 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7ee80fbd40 0x7f7ee8113bb0 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:16:48.786 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:16:48.787+0000 7f7eec93a700 1 --2- 172.21.15.162:0/3896266071 >> [v2:172.21.15.162:6800/1396866108,v1:172.21.15.162:6801/1396866108] conn(0x7f7ed006ec10 0x7f7ed00710c0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:16:48.786 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:16:48.787+0000 7f7eec93a700 1 --2- 172.21.15.162:0/3896266071 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7ee8101d90 0x7f7ee81140f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:16:48.786 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:16:48.787+0000 7f7eec93a700 1 -- 172.21.15.162:0/3896266071 >> 172.21.15.162:0/3896266071 conn(0x7f7ee8071d00 msgr2=0x7f7ee80655a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:16:48.786 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:16:48.787+0000 7f7eec93a700 1 -- 172.21.15.162:0/3896266071 shutdown_connections 2024-04-27T04:16:48.786 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:16:48.787+0000 7f7eec93a700 1 -- 172.21.15.162:0/3896266071 wait complete. 2024-04-27T04:16:49.075 INFO:journalctl@ceph.osd.2.smithi162.stdout:Apr 27 04:16:48 smithi162 podman[132403]: 2024-04-27 04:16:48.751663887 +0000 UTC m=+0.665419571 container remove 70f16d86d38e96c0d1b624f6d1bd488d7691dc54ee4a6bd1f99e063d422ecf5d (image=quay.ceph.io/ceph-ci/ceph@sha256:d31f278ece6dbd17d1fa5e598f509017f54b76fafe1c4416391fcc51b9718ee9, name=ceph-7f9750d6-044a-11ef-bc93-c7b262605968-osd-2, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_CLEAN=True, GIT_BRANCH=HEAD, RELEASE=quincy-2be3efa, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.build-date=20240424, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, ceph=True) 2024-04-27T04:16:49.075 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:16:48 smithi162 ceph-mon[110641]: from='client.34256 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-27T04:16:49.075 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:16:48 smithi162 ceph-mon[110641]: from='client.34260 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-27T04:16:49.075 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:16:48 smithi162 ceph-mon[110641]: pgmap v88: 65 pgs: 65 active+clean; 208 MiB data, 1.8 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:16:49.075 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:16:48 smithi162 ceph-mon[110641]: Health check failed: 1 osds down (OSD_DOWN) 2024-04-27T04:16:49.075 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:16:48 smithi162 ceph-mon[110641]: osdmap e60: 6 total, 5 up, 6 in 2024-04-27T04:16:49.076 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:16:48 smithi162 ceph-mon[110641]: from='client.? 172.21.15.162:0/674530927' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-27T04:16:49.172 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:16:48 smithi181 ceph-mon[81134]: from='client.34256 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-27T04:16:49.172 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:16:48 smithi181 ceph-mon[81134]: from='client.34260 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-27T04:16:49.172 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:16:48 smithi181 ceph-mon[81134]: pgmap v88: 65 pgs: 65 active+clean; 208 MiB data, 1.8 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:16:49.172 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:16:48 smithi181 ceph-mon[81134]: Health check failed: 1 osds down (OSD_DOWN) 2024-04-27T04:16:49.172 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:16:48 smithi181 ceph-mon[81134]: osdmap e60: 6 total, 5 up, 6 in 2024-04-27T04:16:49.172 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:16:48 smithi181 ceph-mon[81134]: from='client.? 172.21.15.162:0/674530927' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-27T04:16:49.386 INFO:journalctl@ceph.osd.2.smithi162.stdout:Apr 27 04:16:49 smithi162 podman[132538]: 2024-04-27 04:16:49.07751522 +0000 UTC m=+0.110335084 container create 24f5799fab21f987793d1fb54bdec842edc96b48035d063a56ba53b9e6228a3e (image=quay.ceph.io/ceph-ci/ceph@sha256:90a3a9fa9ceb8977f6dad9ccd36fe46018d07e2118199820523a6b35301d9c16, name=ceph-7f9750d6-044a-11ef-bc93-c7b262605968-osd-2-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=main-4e213a5, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.build-date=20240424, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , ceph=True, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-27T04:16:49.387 INFO:journalctl@ceph.osd.2.smithi162.stdout:Apr 27 04:16:49 smithi162 podman[132538]: 2024-04-27 04:16:48.985591685 +0000 UTC m=+0.018411559 image pull 67e414ff4bc41092cb31c84c98a9563a32813d02583028679c46b40362afeb3e quay.ceph.io/ceph-ci/ceph@sha256:90a3a9fa9ceb8977f6dad9ccd36fe46018d07e2118199820523a6b35301d9c16 2024-04-27T04:16:49.387 INFO:journalctl@ceph.osd.2.smithi162.stdout:Apr 27 04:16:49 smithi162 podman[132538]: 2024-04-27 04:16:49.322787551 +0000 UTC m=+0.355607422 container init 24f5799fab21f987793d1fb54bdec842edc96b48035d063a56ba53b9e6228a3e (image=quay.ceph.io/ceph-ci/ceph@sha256:90a3a9fa9ceb8977f6dad9ccd36fe46018d07e2118199820523a6b35301d9c16, name=ceph-7f9750d6-044a-11ef-bc93-c7b262605968-osd-2-deactivate, RELEASE=main-4e213a5, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240424, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_CLEAN=True, maintainer=Guillaume Abrioux ) 2024-04-27T04:16:49.387 INFO:journalctl@ceph.osd.2.smithi162.stdout:Apr 27 04:16:49 smithi162 podman[132538]: 2024-04-27 04:16:49.326263183 +0000 UTC m=+0.359083048 container start 24f5799fab21f987793d1fb54bdec842edc96b48035d063a56ba53b9e6228a3e (image=quay.ceph.io/ceph-ci/ceph@sha256:90a3a9fa9ceb8977f6dad9ccd36fe46018d07e2118199820523a6b35301d9c16, name=ceph-7f9750d6-044a-11ef-bc93-c7b262605968-osd-2-deactivate, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=main-4e213a5, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.build-date=20240424, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_CLEAN=True) 2024-04-27T04:16:49.722 INFO:journalctl@ceph.osd.2.smithi162.stdout:Apr 27 04:16:49 smithi162 podman[132538]: 2024-04-27 04:16:49.397069192 +0000 UTC m=+0.429889059 container attach 24f5799fab21f987793d1fb54bdec842edc96b48035d063a56ba53b9e6228a3e (image=quay.ceph.io/ceph-ci/ceph@sha256:90a3a9fa9ceb8977f6dad9ccd36fe46018d07e2118199820523a6b35301d9c16, name=ceph-7f9750d6-044a-11ef-bc93-c7b262605968-osd-2-deactivate, ceph=True, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=main-4e213a5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240424, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_BRANCH=HEAD) 2024-04-27T04:16:49.722 INFO:journalctl@ceph.osd.2.smithi162.stdout:Apr 27 04:16:49 smithi162 podman[132538]: 2024-04-27 04:16:49.616170767 +0000 UTC m=+0.648990623 container died 24f5799fab21f987793d1fb54bdec842edc96b48035d063a56ba53b9e6228a3e (image=quay.ceph.io/ceph-ci/ceph@sha256:90a3a9fa9ceb8977f6dad9ccd36fe46018d07e2118199820523a6b35301d9c16, name=ceph-7f9750d6-044a-11ef-bc93-c7b262605968-osd-2-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, ceph=True, org.label-schema.build-date=20240424, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, RELEASE=main-4e213a5, maintainer=Guillaume Abrioux ) 2024-04-27T04:16:50.136 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:16:49 smithi162 ceph-mon[110641]: from='client.34272 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-27T04:16:50.136 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:16:49 smithi162 ceph-mon[110641]: osdmap e61: 6 total, 5 up, 6 in 2024-04-27T04:16:50.136 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:16:49 smithi162 ceph-mon[110641]: from='client.? 172.21.15.162:0/3896266071' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-27T04:16:50.172 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:16:49 smithi181 ceph-mon[81134]: from='client.34272 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-27T04:16:50.172 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:16:49 smithi181 ceph-mon[81134]: osdmap e61: 6 total, 5 up, 6 in 2024-04-27T04:16:50.172 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:16:49 smithi181 ceph-mon[81134]: from='client.? 172.21.15.162:0/3896266071' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-27T04:16:50.637 INFO:journalctl@ceph.osd.2.smithi162.stdout:Apr 27 04:16:50 smithi162 podman[132564]: 2024-04-27 04:16:50.259902266 +0000 UTC m=+0.637860379 container remove 24f5799fab21f987793d1fb54bdec842edc96b48035d063a56ba53b9e6228a3e (image=quay.ceph.io/ceph-ci/ceph@sha256:90a3a9fa9ceb8977f6dad9ccd36fe46018d07e2118199820523a6b35301d9c16, name=ceph-7f9750d6-044a-11ef-bc93-c7b262605968-osd-2-deactivate, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240424, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , RELEASE=main-4e213a5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, ceph=True) 2024-04-27T04:16:50.637 INFO:journalctl@ceph.osd.2.smithi162.stdout:Apr 27 04:16:50 smithi162 systemd[1]: ceph-7f9750d6-044a-11ef-bc93-c7b262605968@osd.2.service: Deactivated successfully. 2024-04-27T04:16:50.637 INFO:journalctl@ceph.osd.2.smithi162.stdout:Apr 27 04:16:50 smithi162 systemd[1]: Stopped Ceph osd.2 for 7f9750d6-044a-11ef-bc93-c7b262605968. 2024-04-27T04:16:50.637 INFO:journalctl@ceph.osd.2.smithi162.stdout:Apr 27 04:16:50 smithi162 systemd[1]: ceph-7f9750d6-044a-11ef-bc93-c7b262605968@osd.2.service: Consumed 56.089s CPU time. 2024-04-27T04:16:50.637 INFO:journalctl@ceph.osd.2.smithi162.stdout:Apr 27 04:16:50 smithi162 systemd[1]: Starting Ceph osd.2 for 7f9750d6-044a-11ef-bc93-c7b262605968... 2024-04-27T04:16:50.969 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:16:50 smithi162 ceph-mon[110641]: pgmap v91: 65 pgs: 19 peering, 3 stale+active+clean, 43 active+clean; 208 MiB data, 1.8 GiB used, 535 GiB / 536 GiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T04:16:50.969 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:16:50 smithi162 ceph-mon[110641]: Health check failed: Reduced data availability: 6 pgs peering (PG_AVAILABILITY) 2024-04-27T04:16:50.969 INFO:journalctl@ceph.osd.2.smithi162.stdout:Apr 27 04:16:50 smithi162 podman[132672]: 2024-04-27 04:16:50.613446046 +0000 UTC m=+0.018192753 image pull 67e414ff4bc41092cb31c84c98a9563a32813d02583028679c46b40362afeb3e quay.ceph.io/ceph-ci/ceph@sha256:90a3a9fa9ceb8977f6dad9ccd36fe46018d07e2118199820523a6b35301d9c16 2024-04-27T04:16:50.969 INFO:journalctl@ceph.osd.2.smithi162.stdout:Apr 27 04:16:50 smithi162 podman[132672]: 2024-04-27 04:16:50.727439051 +0000 UTC m=+0.132185734 container create 3b7e9a8f65fd9f1cb4253db8b5bf6e62d89eed9f1818852b995c347cc1ce01d8 (image=quay.ceph.io/ceph-ci/ceph@sha256:90a3a9fa9ceb8977f6dad9ccd36fe46018d07e2118199820523a6b35301d9c16, name=ceph-7f9750d6-044a-11ef-bc93-c7b262605968-osd-2-activate, ceph=True, org.label-schema.schema-version=1.0, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, RELEASE=main-4e213a5, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240424, org.label-schema.vendor=CentOS) 2024-04-27T04:16:50.970 INFO:journalctl@ceph.osd.2.smithi162.stdout:Apr 27 04:16:50 smithi162 podman[132672]: 2024-04-27 04:16:50.971841145 +0000 UTC m=+0.376587834 container init 3b7e9a8f65fd9f1cb4253db8b5bf6e62d89eed9f1818852b995c347cc1ce01d8 (image=quay.ceph.io/ceph-ci/ceph@sha256:90a3a9fa9ceb8977f6dad9ccd36fe46018d07e2118199820523a6b35301d9c16, name=ceph-7f9750d6-044a-11ef-bc93-c7b262605968-osd-2-activate, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, ceph=True, org.label-schema.build-date=20240424, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, RELEASE=main-4e213a5, org.label-schema.vendor=CentOS) 2024-04-27T04:16:51.172 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:16:50 smithi181 ceph-mon[81134]: pgmap v91: 65 pgs: 19 peering, 3 stale+active+clean, 43 active+clean; 208 MiB data, 1.8 GiB used, 535 GiB / 536 GiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T04:16:51.172 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:16:50 smithi181 ceph-mon[81134]: Health check failed: Reduced data availability: 6 pgs peering (PG_AVAILABILITY) 2024-04-27T04:16:51.386 INFO:journalctl@ceph.osd.2.smithi162.stdout:Apr 27 04:16:50 smithi162 podman[132672]: 2024-04-27 04:16:50.975127569 +0000 UTC m=+0.379874253 container start 3b7e9a8f65fd9f1cb4253db8b5bf6e62d89eed9f1818852b995c347cc1ce01d8 (image=quay.ceph.io/ceph-ci/ceph@sha256:90a3a9fa9ceb8977f6dad9ccd36fe46018d07e2118199820523a6b35301d9c16, name=ceph-7f9750d6-044a-11ef-bc93-c7b262605968-osd-2-activate, io.buildah.version=1.33.5, org.label-schema.build-date=20240424, maintainer=Guillaume Abrioux , ceph=True, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=main-4e213a5, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-27T04:16:51.386 INFO:journalctl@ceph.osd.2.smithi162.stdout:Apr 27 04:16:51 smithi162 podman[132672]: 2024-04-27 04:16:51.046729193 +0000 UTC m=+0.451475906 container attach 3b7e9a8f65fd9f1cb4253db8b5bf6e62d89eed9f1818852b995c347cc1ce01d8 (image=quay.ceph.io/ceph-ci/ceph@sha256:90a3a9fa9ceb8977f6dad9ccd36fe46018d07e2118199820523a6b35301d9c16, name=ceph-7f9750d6-044a-11ef-bc93-c7b262605968-osd-2-activate, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, ceph=True, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=main-4e213a5, org.label-schema.license=GPLv2, org.label-schema.build-date=20240424, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS) 2024-04-27T04:16:51.791 INFO:journalctl@ceph.osd.2.smithi162.stdout:Apr 27 04:16:51 smithi162 ceph-7f9750d6-044a-11ef-bc93-c7b262605968-osd-2-activate[132686]: Running command: /usr/bin/ceph-authtool --gen-print-key 2024-04-27T04:16:51.791 INFO:journalctl@ceph.osd.2.smithi162.stdout:Apr 27 04:16:51 smithi162 ceph-7f9750d6-044a-11ef-bc93-c7b262605968-osd-2-activate[132686]: Running command: /usr/bin/ceph-authtool --gen-print-key 2024-04-27T04:16:51.791 INFO:journalctl@ceph.osd.2.smithi162.stdout:Apr 27 04:16:51 smithi162 bash[132672]: Running command: /usr/bin/ceph-authtool --gen-print-key 2024-04-27T04:16:51.791 INFO:journalctl@ceph.osd.2.smithi162.stdout:Apr 27 04:16:51 smithi162 bash[132672]: Running command: /usr/bin/ceph-authtool --gen-print-key 2024-04-27T04:16:51.791 INFO:journalctl@ceph.osd.2.smithi162.stdout:Apr 27 04:16:51 smithi162 bash[132672]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2024-04-27T04:16:51.791 INFO:journalctl@ceph.osd.2.smithi162.stdout:Apr 27 04:16:51 smithi162 bash[132672]: Running command: /usr/bin/ceph-bluestore-tool prime-osd-dir --path /var/lib/ceph/osd/ceph-2 --no-mon-config --dev /dev/mapper/vg_nvme-lv_2 2024-04-27T04:16:51.791 INFO:journalctl@ceph.osd.2.smithi162.stdout:Apr 27 04:16:51 smithi162 bash[132672]: Running command: /usr/bin/chown -h ceph:ceph /dev/mapper/vg_nvme-lv_2 2024-04-27T04:16:51.791 INFO:journalctl@ceph.osd.2.smithi162.stdout:Apr 27 04:16:51 smithi162 bash[132672]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-1 2024-04-27T04:16:51.791 INFO:journalctl@ceph.osd.2.smithi162.stdout:Apr 27 04:16:51 smithi162 bash[132672]: Running command: /usr/bin/ln -s /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-2/block 2024-04-27T04:16:51.791 INFO:journalctl@ceph.osd.2.smithi162.stdout:Apr 27 04:16:51 smithi162 bash[132672]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2024-04-27T04:16:51.792 INFO:journalctl@ceph.osd.2.smithi162.stdout:Apr 27 04:16:51 smithi162 bash[132672]: --> ceph-volume raw activate successful for osd ID: 2 2024-04-27T04:16:51.792 INFO:journalctl@ceph.osd.2.smithi162.stdout:Apr 27 04:16:51 smithi162 ceph-7f9750d6-044a-11ef-bc93-c7b262605968-osd-2-activate[132686]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2024-04-27T04:16:51.792 INFO:journalctl@ceph.osd.2.smithi162.stdout:Apr 27 04:16:51 smithi162 ceph-7f9750d6-044a-11ef-bc93-c7b262605968-osd-2-activate[132686]: Running command: /usr/bin/ceph-bluestore-tool prime-osd-dir --path /var/lib/ceph/osd/ceph-2 --no-mon-config --dev /dev/mapper/vg_nvme-lv_2 2024-04-27T04:16:51.792 INFO:journalctl@ceph.osd.2.smithi162.stdout:Apr 27 04:16:51 smithi162 ceph-7f9750d6-044a-11ef-bc93-c7b262605968-osd-2-activate[132686]: Running command: /usr/bin/chown -h ceph:ceph /dev/mapper/vg_nvme-lv_2 2024-04-27T04:16:51.792 INFO:journalctl@ceph.osd.2.smithi162.stdout:Apr 27 04:16:51 smithi162 ceph-7f9750d6-044a-11ef-bc93-c7b262605968-osd-2-activate[132686]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-1 2024-04-27T04:16:51.792 INFO:journalctl@ceph.osd.2.smithi162.stdout:Apr 27 04:16:51 smithi162 ceph-7f9750d6-044a-11ef-bc93-c7b262605968-osd-2-activate[132686]: Running command: /usr/bin/ln -s /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-2/block 2024-04-27T04:16:51.792 INFO:journalctl@ceph.osd.2.smithi162.stdout:Apr 27 04:16:51 smithi162 ceph-7f9750d6-044a-11ef-bc93-c7b262605968-osd-2-activate[132686]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2024-04-27T04:16:51.792 INFO:journalctl@ceph.osd.2.smithi162.stdout:Apr 27 04:16:51 smithi162 ceph-7f9750d6-044a-11ef-bc93-c7b262605968-osd-2-activate[132686]: --> ceph-volume raw activate successful for osd ID: 2 2024-04-27T04:16:51.792 INFO:journalctl@ceph.osd.2.smithi162.stdout:Apr 27 04:16:51 smithi162 podman[132672]: 2024-04-27 04:16:51.47882393 +0000 UTC m=+0.883570623 container died 3b7e9a8f65fd9f1cb4253db8b5bf6e62d89eed9f1818852b995c347cc1ce01d8 (image=quay.ceph.io/ceph-ci/ceph@sha256:90a3a9fa9ceb8977f6dad9ccd36fe46018d07e2118199820523a6b35301d9c16, name=ceph-7f9750d6-044a-11ef-bc93-c7b262605968-osd-2-activate, ceph=True, org.label-schema.build-date=20240424, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=main-4e213a5, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5) 2024-04-27T04:16:52.521 INFO:journalctl@ceph.osd.2.smithi162.stdout:Apr 27 04:16:52 smithi162 podman[132816]: 2024-04-27 04:16:52.193048218 +0000 UTC m=+0.705136748 container remove 3b7e9a8f65fd9f1cb4253db8b5bf6e62d89eed9f1818852b995c347cc1ce01d8 (image=quay.ceph.io/ceph-ci/ceph@sha256:90a3a9fa9ceb8977f6dad9ccd36fe46018d07e2118199820523a6b35301d9c16, name=ceph-7f9750d6-044a-11ef-bc93-c7b262605968-osd-2-activate, io.buildah.version=1.33.5, org.label-schema.build-date=20240424, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, RELEASE=main-4e213a5, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0) 2024-04-27T04:16:52.521 INFO:journalctl@ceph.osd.2.smithi162.stdout:Apr 27 04:16:52 smithi162 podman[132880]: 2024-04-27 04:16:52.307580313 +0000 UTC m=+0.018574470 image pull 67e414ff4bc41092cb31c84c98a9563a32813d02583028679c46b40362afeb3e quay.ceph.io/ceph-ci/ceph@sha256:90a3a9fa9ceb8977f6dad9ccd36fe46018d07e2118199820523a6b35301d9c16 2024-04-27T04:16:52.521 INFO:journalctl@ceph.osd.2.smithi162.stdout:Apr 27 04:16:52 smithi162 podman[132880]: 2024-04-27 04:16:52.418963397 +0000 UTC m=+0.129957569 container create d24878d461e7102a217196422c598c253a67d0ce73605f8073d54a1b38da2b3e (image=quay.ceph.io/ceph-ci/ceph@sha256:90a3a9fa9ceb8977f6dad9ccd36fe46018d07e2118199820523a6b35301d9c16, name=ceph-7f9750d6-044a-11ef-bc93-c7b262605968-osd-2, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, ceph=True, io.buildah.version=1.33.5, GIT_CLEAN=True, RELEASE=main-4e213a5, org.label-schema.build-date=20240424, org.label-schema.license=GPLv2) 2024-04-27T04:16:52.887 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:16:52 smithi162 ceph-mon[110641]: pgmap v92: 65 pgs: 19 peering, 3 stale+active+clean, 43 active+clean; 208 MiB data, 1.8 GiB used, 535 GiB / 536 GiB avail; 511 B/s rd, 1 op/s 2024-04-27T04:16:52.887 INFO:journalctl@ceph.osd.2.smithi162.stdout:Apr 27 04:16:52 smithi162 podman[132880]: 2024-04-27 04:16:52.629579638 +0000 UTC m=+0.340573769 container init d24878d461e7102a217196422c598c253a67d0ce73605f8073d54a1b38da2b3e (image=quay.ceph.io/ceph-ci/ceph@sha256:90a3a9fa9ceb8977f6dad9ccd36fe46018d07e2118199820523a6b35301d9c16, name=ceph-7f9750d6-044a-11ef-bc93-c7b262605968-osd-2, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240424, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=main-4e213a5, GIT_CLEAN=True, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5) 2024-04-27T04:16:52.887 INFO:journalctl@ceph.osd.2.smithi162.stdout:Apr 27 04:16:52 smithi162 podman[132880]: 2024-04-27 04:16:52.632611372 +0000 UTC m=+0.343605509 container start d24878d461e7102a217196422c598c253a67d0ce73605f8073d54a1b38da2b3e (image=quay.ceph.io/ceph-ci/ceph@sha256:90a3a9fa9ceb8977f6dad9ccd36fe46018d07e2118199820523a6b35301d9c16, name=ceph-7f9750d6-044a-11ef-bc93-c7b262605968-osd-2, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.build-date=20240424, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=main-4e213a5, org.label-schema.vendor=CentOS) 2024-04-27T04:16:52.888 INFO:journalctl@ceph.osd.2.smithi162.stdout:Apr 27 04:16:52 smithi162 ceph-osd[132899]: -- 172.21.15.162:0/606442076 <== mon.0 v2:172.21.15.162:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55aaf5e2b680 con 0x55aaf5df5000 2024-04-27T04:16:52.888 INFO:journalctl@ceph.osd.2.smithi162.stdout:Apr 27 04:16:52 smithi162 bash[132880]: d24878d461e7102a217196422c598c253a67d0ce73605f8073d54a1b38da2b3e 2024-04-27T04:16:52.888 INFO:journalctl@ceph.osd.2.smithi162.stdout:Apr 27 04:16:52 smithi162 systemd[1]: Started Ceph osd.2 for 7f9750d6-044a-11ef-bc93-c7b262605968. 2024-04-27T04:16:53.173 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:16:52 smithi181 ceph-mon[81134]: pgmap v92: 65 pgs: 19 peering, 3 stale+active+clean, 43 active+clean; 208 MiB data, 1.8 GiB used, 535 GiB / 536 GiB avail; 511 B/s rd, 1 op/s 2024-04-27T04:16:53.590 INFO:journalctl@ceph.osd.2.smithi162.stdout:Apr 27 04:16:53 smithi162 ceph-7f9750d6-044a-11ef-bc93-c7b262605968-osd-2[132895]: 2024-04-27T04:16:53.183+0000 7ff01da7a7c0 -1 Falling back to public interface 2024-04-27T04:16:53.872 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:16:53 smithi162 ceph-mon[110641]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' 2024-04-27T04:16:53.872 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:16:53 smithi162 ceph-mon[110641]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' 2024-04-27T04:16:53.872 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:16:53 smithi162 ceph-mon[110641]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T04:16:54.172 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:16:53 smithi181 ceph-mon[81134]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' 2024-04-27T04:16:54.172 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:16:53 smithi181 ceph-mon[81134]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' 2024-04-27T04:16:54.172 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:16:53 smithi181 ceph-mon[81134]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T04:16:55.136 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:16:54 smithi162 ceph-mon[110641]: pgmap v93: 65 pgs: 12 active+undersized+degraded, 15 active+undersized, 38 active+clean; 208 MiB data, 1.8 GiB used, 535 GiB / 536 GiB avail; 0 B/s rd, 2 op/s; 30/231 objects degraded (12.987%) 2024-04-27T04:16:55.137 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:16:54 smithi162 ceph-mon[110641]: Health check failed: Degraded data redundancy: 30/231 objects degraded (12.987%), 12 pgs degraded (PG_DEGRADED) 2024-04-27T04:16:55.137 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:16:54 smithi162 ceph-mon[110641]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 6 pgs peering) 2024-04-27T04:16:55.172 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:16:54 smithi181 ceph-mon[81134]: pgmap v93: 65 pgs: 12 active+undersized+degraded, 15 active+undersized, 38 active+clean; 208 MiB data, 1.8 GiB used, 535 GiB / 536 GiB avail; 0 B/s rd, 2 op/s; 30/231 objects degraded (12.987%) 2024-04-27T04:16:55.172 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:16:54 smithi181 ceph-mon[81134]: Health check failed: Degraded data redundancy: 30/231 objects degraded (12.987%), 12 pgs degraded (PG_DEGRADED) 2024-04-27T04:16:55.172 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:16:54 smithi181 ceph-mon[81134]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 6 pgs peering) 2024-04-27T04:16:56.136 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:16:55 smithi162 ceph-mon[110641]: pgmap v94: 65 pgs: 12 active+undersized+degraded, 15 active+undersized, 38 active+clean; 208 MiB data, 1.8 GiB used, 535 GiB / 536 GiB avail; 0 B/s rd, 2 op/s; 30/231 objects degraded (12.987%) 2024-04-27T04:16:56.173 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:16:55 smithi181 ceph-mon[81134]: pgmap v94: 65 pgs: 12 active+undersized+degraded, 15 active+undersized, 38 active+clean; 208 MiB data, 1.8 GiB used, 535 GiB / 536 GiB avail; 0 B/s rd, 2 op/s; 30/231 objects degraded (12.987%) 2024-04-27T04:16:58.136 INFO:journalctl@ceph.osd.2.smithi162.stdout:Apr 27 04:16:57 smithi162 ceph-7f9750d6-044a-11ef-bc93-c7b262605968-osd-2[132895]: 2024-04-27T04:16:57.809+0000 7ff01da7a7c0 -1 osd.2 59 log_to_monitors true 2024-04-27T04:16:59.136 INFO:journalctl@ceph.osd.2.smithi162.stdout:Apr 27 04:16:58 smithi162 ceph-7f9750d6-044a-11ef-bc93-c7b262605968-osd-2[132895]: 2024-04-27T04:16:58.714+0000 7ff00eed2700 -1 osd.2 59 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-27T04:16:59.137 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:16:58 smithi162 ceph-mon[110641]: pgmap v95: 65 pgs: 12 active+undersized+degraded, 15 active+undersized, 38 active+clean; 208 MiB data, 1.8 GiB used, 535 GiB / 536 GiB avail; 0 B/s rd, 1 op/s; 30/231 objects degraded (12.987%) 2024-04-27T04:16:59.137 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:16:58 smithi162 ceph-mon[110641]: from='osd.2 [v2:172.21.15.162:6818/3115093010,v1:172.21.15.162:6819/3115093010]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-04-27T04:16:59.137 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:16:58 smithi162 ceph-mon[110641]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' 2024-04-27T04:16:59.137 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:16:58 smithi162 ceph-mon[110641]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' 2024-04-27T04:16:59.172 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:16:58 smithi181 ceph-mon[81134]: pgmap v95: 65 pgs: 12 active+undersized+degraded, 15 active+undersized, 38 active+clean; 208 MiB data, 1.8 GiB used, 535 GiB / 536 GiB avail; 0 B/s rd, 1 op/s; 30/231 objects degraded (12.987%) 2024-04-27T04:16:59.172 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:16:58 smithi181 ceph-mon[81134]: from='osd.2 [v2:172.21.15.162:6818/3115093010,v1:172.21.15.162:6819/3115093010]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-04-27T04:16:59.172 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:16:58 smithi181 ceph-mon[81134]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' 2024-04-27T04:16:59.172 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:16:58 smithi181 ceph-mon[81134]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' 2024-04-27T04:17:00.036 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:16:59 smithi162 ceph-mon[110641]: from='osd.2 [v2:172.21.15.162:6818/3115093010,v1:172.21.15.162:6819/3115093010]' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-04-27T04:17:00.036 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:16:59 smithi162 ceph-mon[110641]: osdmap e62: 6 total, 5 up, 6 in 2024-04-27T04:17:00.036 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:16:59 smithi162 ceph-mon[110641]: from='osd.2 [v2:172.21.15.162:6818/3115093010,v1:172.21.15.162:6819/3115093010]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi162", "root=default"]}]: dispatch 2024-04-27T04:17:00.036 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:16:59 smithi162 ceph-mon[110641]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' 2024-04-27T04:17:00.037 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:16:59 smithi162 ceph-mon[110641]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' 2024-04-27T04:17:00.172 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:16:59 smithi181 ceph-mon[81134]: from='osd.2 [v2:172.21.15.162:6818/3115093010,v1:172.21.15.162:6819/3115093010]' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-04-27T04:17:00.172 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:16:59 smithi181 ceph-mon[81134]: osdmap e62: 6 total, 5 up, 6 in 2024-04-27T04:17:00.172 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:16:59 smithi181 ceph-mon[81134]: from='osd.2 [v2:172.21.15.162:6818/3115093010,v1:172.21.15.162:6819/3115093010]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi162", "root=default"]}]: dispatch 2024-04-27T04:17:00.172 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:16:59 smithi181 ceph-mon[81134]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' 2024-04-27T04:17:00.172 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:16:59 smithi181 ceph-mon[81134]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' 2024-04-27T04:17:01.070 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:17:00 smithi162 ceph-mon[110641]: pgmap v97: 65 pgs: 12 active+undersized+degraded, 15 active+undersized, 38 active+clean; 208 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 0 B/s rd, 1 op/s; 30/231 objects degraded (12.987%) 2024-04-27T04:17:01.070 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:17:00 smithi162 ceph-mon[110641]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-04-27T04:17:01.070 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:17:00 smithi162 ceph-mon[110641]: osd.2 [v2:172.21.15.162:6818/3115093010,v1:172.21.15.162:6819/3115093010] boot 2024-04-27T04:17:01.070 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:17:00 smithi162 ceph-mon[110641]: osdmap e63: 6 total, 6 up, 6 in 2024-04-27T04:17:01.070 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:17:00 smithi162 ceph-mon[110641]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-27T04:17:01.070 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:17:00 smithi162 ceph-mon[110641]: osdmap e64: 6 total, 6 up, 6 in 2024-04-27T04:17:01.172 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:17:00 smithi181 ceph-mon[81134]: pgmap v97: 65 pgs: 12 active+undersized+degraded, 15 active+undersized, 38 active+clean; 208 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 0 B/s rd, 1 op/s; 30/231 objects degraded (12.987%) 2024-04-27T04:17:01.172 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:17:00 smithi181 ceph-mon[81134]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-04-27T04:17:01.172 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:17:00 smithi181 ceph-mon[81134]: osd.2 [v2:172.21.15.162:6818/3115093010,v1:172.21.15.162:6819/3115093010] boot 2024-04-27T04:17:01.172 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:17:00 smithi181 ceph-mon[81134]: osdmap e63: 6 total, 6 up, 6 in 2024-04-27T04:17:01.172 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:17:00 smithi181 ceph-mon[81134]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-27T04:17:01.172 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:17:00 smithi181 ceph-mon[81134]: osdmap e64: 6 total, 6 up, 6 in 2024-04-27T04:17:02.974 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:17:02 smithi181 ceph-mon[81134]: pgmap v100: 65 pgs: 12 active+undersized+degraded, 15 active+undersized, 38 active+clean; 208 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s; 30/231 objects degraded (12.987%) 2024-04-27T04:17:02.974 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:17:02 smithi181 ceph-mon[81134]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' 2024-04-27T04:17:02.974 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:17:02 smithi181 ceph-mon[81134]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-27T04:17:03.136 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:17:02 smithi162 ceph-mon[110641]: pgmap v100: 65 pgs: 12 active+undersized+degraded, 15 active+undersized, 38 active+clean; 208 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s; 30/231 objects degraded (12.987%) 2024-04-27T04:17:03.136 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:17:02 smithi162 ceph-mon[110641]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' 2024-04-27T04:17:03.136 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:17:02 smithi162 ceph-mon[110641]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-27T04:17:04.136 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:17:03 smithi162 ceph-mon[110641]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' 2024-04-27T04:17:04.137 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:17:03 smithi162 ceph-mon[110641]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' 2024-04-27T04:17:04.137 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:17:03 smithi162 ceph-mon[110641]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T04:17:04.137 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:17:03 smithi162 ceph-mon[110641]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T04:17:04.137 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:17:03 smithi162 ceph-mon[110641]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' 2024-04-27T04:17:04.137 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:17:03 smithi162 ceph-mon[110641]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T04:17:04.137 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:17:03 smithi162 ceph-mon[110641]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "versions"}]: dispatch 2024-04-27T04:17:04.137 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:17:03 smithi162 ceph-mon[110641]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "versions"}]: dispatch 2024-04-27T04:17:04.137 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:17:03 smithi162 ceph-mon[110641]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "versions"}]: dispatch 2024-04-27T04:17:04.137 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:17:03 smithi162 ceph-mon[110641]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "osd ok-to-stop", "ids": ["3"], "max": 16}]: dispatch 2024-04-27T04:17:04.172 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:17:03 smithi181 ceph-mon[81134]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' 2024-04-27T04:17:04.172 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:17:03 smithi181 ceph-mon[81134]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' 2024-04-27T04:17:04.172 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:17:03 smithi181 ceph-mon[81134]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T04:17:04.172 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:17:03 smithi181 ceph-mon[81134]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T04:17:04.172 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:17:03 smithi181 ceph-mon[81134]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' 2024-04-27T04:17:04.172 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:17:03 smithi181 ceph-mon[81134]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T04:17:04.172 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:17:03 smithi181 ceph-mon[81134]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "versions"}]: dispatch 2024-04-27T04:17:04.172 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:17:03 smithi181 ceph-mon[81134]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "versions"}]: dispatch 2024-04-27T04:17:04.172 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:17:03 smithi181 ceph-mon[81134]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "versions"}]: dispatch 2024-04-27T04:17:04.172 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:17:03 smithi181 ceph-mon[81134]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "osd ok-to-stop", "ids": ["3"], "max": 16}]: dispatch 2024-04-27T04:17:05.136 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:17:04 smithi162 ceph-mon[110641]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["3"], "max": 16}]: dispatch 2024-04-27T04:17:05.137 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:17:04 smithi162 ceph-mon[110641]: Upgrade: unsafe to stop osd(s) at this time (11 PGs are or would become offline) 2024-04-27T04:17:05.137 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:17:04 smithi162 ceph-mon[110641]: pgmap v101: 65 pgs: 5 active+undersized+degraded, 4 active+undersized, 56 active+clean; 208 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s; 14/231 objects degraded (6.061%) 2024-04-27T04:17:05.137 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:17:04 smithi162 ceph-mon[110641]: Health check update: Degraded data redundancy: 14/231 objects degraded (6.061%), 5 pgs degraded (PG_DEGRADED) 2024-04-27T04:17:05.172 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:17:04 smithi181 ceph-mon[81134]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["3"], "max": 16}]: dispatch 2024-04-27T04:17:05.172 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:17:04 smithi181 ceph-mon[81134]: Upgrade: unsafe to stop osd(s) at this time (11 PGs are or would become offline) 2024-04-27T04:17:05.172 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:17:04 smithi181 ceph-mon[81134]: pgmap v101: 65 pgs: 5 active+undersized+degraded, 4 active+undersized, 56 active+clean; 208 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s; 14/231 objects degraded (6.061%) 2024-04-27T04:17:05.172 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:17:04 smithi181 ceph-mon[81134]: Health check update: Degraded data redundancy: 14/231 objects degraded (6.061%), 5 pgs degraded (PG_DEGRADED) 2024-04-27T04:17:06.136 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:17:05 smithi162 ceph-mon[110641]: pgmap v102: 65 pgs: 65 active+clean; 208 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 736 B/s rd, 1 op/s 2024-04-27T04:17:06.136 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:17:05 smithi162 ceph-mon[110641]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 14/231 objects degraded (6.061%), 5 pgs degraded) 2024-04-27T04:17:06.137 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:17:05 smithi162 ceph-mon[110641]: Cluster is now healthy 2024-04-27T04:17:06.171 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:17:05 smithi181 ceph-mon[81134]: pgmap v102: 65 pgs: 65 active+clean; 208 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 736 B/s rd, 1 op/s 2024-04-27T04:17:06.172 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:17:05 smithi181 ceph-mon[81134]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 14/231 objects degraded (6.061%), 5 pgs degraded) 2024-04-27T04:17:06.172 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:17:05 smithi181 ceph-mon[81134]: Cluster is now healthy 2024-04-27T04:17:09.136 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:17:08 smithi162 ceph-mon[110641]: pgmap v103: 65 pgs: 65 active+clean; 208 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 639 B/s rd, 1 op/s 2024-04-27T04:17:09.172 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:17:08 smithi181 ceph-mon[81134]: pgmap v103: 65 pgs: 65 active+clean; 208 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 639 B/s rd, 1 op/s 2024-04-27T04:17:11.136 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:17:10 smithi162 ceph-mon[110641]: pgmap v104: 65 pgs: 65 active+clean; 208 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 1.0 KiB/s rd, 2 op/s 2024-04-27T04:17:11.171 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:17:10 smithi181 ceph-mon[81134]: pgmap v104: 65 pgs: 65 active+clean; 208 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 1.0 KiB/s rd, 2 op/s 2024-04-27T04:17:13.136 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:17:12 smithi162 ceph-mon[110641]: pgmap v105: 65 pgs: 65 active+clean; 208 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 936 B/s rd, 1 op/s 2024-04-27T04:17:13.136 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:17:12 smithi162 ceph-mon[110641]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' 2024-04-27T04:17:13.171 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:17:12 smithi181 ceph-mon[81134]: pgmap v105: 65 pgs: 65 active+clean; 208 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 936 B/s rd, 1 op/s 2024-04-27T04:17:13.171 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:17:12 smithi181 ceph-mon[81134]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' 2024-04-27T04:17:15.136 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:17:14 smithi162 ceph-mon[110641]: pgmap v106: 65 pgs: 65 active+clean; 208 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:17:15.171 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:17:14 smithi181 ceph-mon[81134]: pgmap v106: 65 pgs: 65 active+clean; 208 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:17:17.136 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:17:16 smithi162 ceph-mon[110641]: pgmap v107: 65 pgs: 65 active+clean; 208 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:17:17.172 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:17:16 smithi181 ceph-mon[81134]: pgmap v107: 65 pgs: 65 active+clean; 208 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:17:18.800 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:17:18 smithi181 ceph-mon[81134]: pgmap v108: 65 pgs: 65 active+clean; 208 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:17:18.800 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:17:18 smithi181 ceph-mon[81134]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' 2024-04-27T04:17:18.800 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:17:18 smithi181 ceph-mon[81134]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-27T04:17:18.801 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:17:18 smithi181 ceph-mon[81134]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "osd ok-to-stop", "ids": ["3"], "max": 16}]: dispatch 2024-04-27T04:17:18.907 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:17:18.906+0000 7f17685e8700 1 -- 172.21.15.162:0/1315681604 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f17600fbd40 msgr2=0x7f17600fc1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:17:18.907 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:17:18.906+0000 7f17685e8700 1 --2- 172.21.15.162:0/1315681604 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f17600fbd40 0x7f17600fc1b0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f175c004940 tx=0x7f175c009e30 comp rx=0 tx=0).stop 2024-04-27T04:17:18.907 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:17:18.906+0000 7f17685e8700 1 -- 172.21.15.162:0/1315681604 shutdown_connections 2024-04-27T04:17:18.907 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:17:18.906+0000 7f17685e8700 1 --2- 172.21.15.162:0/1315681604 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f17600fbd40 0x7f17600fc1b0 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:17:18.908 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:17:18.906+0000 7f17685e8700 1 --2- 172.21.15.162:0/1315681604 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1760101d90 0x7f1760102160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:17:18.908 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:17:18.906+0000 7f17685e8700 1 -- 172.21.15.162:0/1315681604 >> 172.21.15.162:0/1315681604 conn(0x7f1760071d00 msgr2=0x7f1760072100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:17:18.908 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:17:18.907+0000 7f17685e8700 1 -- 172.21.15.162:0/1315681604 shutdown_connections 2024-04-27T04:17:18.908 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:17:18.907+0000 7f17685e8700 1 -- 172.21.15.162:0/1315681604 wait complete. 2024-04-27T04:17:18.908 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:17:18.907+0000 7f17685e8700 1 Processor -- start 2024-04-27T04:17:18.908 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:17:18.907+0000 7f17685e8700 1 -- start start 2024-04-27T04:17:18.908 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:17:18.907+0000 7f17685e8700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f17600fbd40 0x7f1760113b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:17:18.908 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:17:18.907+0000 7f17685e8700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1760101d90 0x7f1760114040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:17:18.908 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:17:18.907+0000 7f17685e8700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1760114690 con 0x7f17600fbd40 2024-04-27T04:17:18.908 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:17:18.907+0000 7f17685e8700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f176010da80 con 0x7f1760101d90 2024-04-27T04:17:18.908 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:17:18.907+0000 7f1766384700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f17600fbd40 0x7f1760113b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:17:18.908 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:17:18.908+0000 7f1766384700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f17600fbd40 0x7f1760113b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:42522/0 (socket says 172.21.15.162:42522) 2024-04-27T04:17:18.908 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:17:18.908+0000 7f1766384700 1 -- 172.21.15.162:0/15592591 learned_addr learned my addr 172.21.15.162:0/15592591 (peer_addr_for_me v2:172.21.15.162:0/0) 2024-04-27T04:17:18.908 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:17:18.908+0000 7f1765b83700 1 --2- 172.21.15.162:0/15592591 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1760101d90 0x7f1760114040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:17:18.909 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:17:18.908+0000 7f1766384700 1 -- 172.21.15.162:0/15592591 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1760101d90 msgr2=0x7f1760114040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:17:18.909 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:17:18.908+0000 7f1766384700 1 --2- 172.21.15.162:0/15592591 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1760101d90 0x7f1760114040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:17:18.909 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:17:18.908+0000 7f1766384700 1 -- 172.21.15.162:0/15592591 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f175c005040 con 0x7f17600fbd40 2024-04-27T04:17:18.909 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:17:18.908+0000 7f1766384700 1 --2- 172.21.15.162:0/15592591 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f17600fbd40 0x7f1760113b00 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f175000eb40 tx=0x7f175000b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:17:18.909 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:17:18.908+0000 7f1756ffd700 1 -- 172.21.15.162:0/15592591 <== mon.0 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1750007cc0 con 0x7f17600fbd40 2024-04-27T04:17:18.909 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:17:18.908+0000 7f17685e8700 1 -- 172.21.15.162:0/15592591 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f176010dd00 con 0x7f17600fbd40 2024-04-27T04:17:18.909 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:17:18.908+0000 7f1756ffd700 1 -- 172.21.15.162:0/15592591 <== mon.0 v2:172.21.15.162:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f175000f040 con 0x7f17600fbd40 2024-04-27T04:17:18.909 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:17:18.908+0000 7f1756ffd700 1 -- 172.21.15.162:0/15592591 <== mon.0 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f175000ca10 con 0x7f17600fbd40 2024-04-27T04:17:18.909 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:17:18.908+0000 7f17685e8700 1 -- 172.21.15.162:0/15592591 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f176010e210 con 0x7f17600fbd40 2024-04-27T04:17:18.910 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:17:18.909+0000 7f17685e8700 1 -- 172.21.15.162:0/15592591 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f17600fd430 con 0x7f17600fbd40 2024-04-27T04:17:18.910 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:17:18.910+0000 7f1756ffd700 1 -- 172.21.15.162:0/15592591 <== mon.0 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 39) v1 ==== 92747+0+0 (secure 0 0 0) 0x7f175000cc80 con 0x7f17600fbd40 2024-04-27T04:17:18.910 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:17:18.910+0000 7f1756ffd700 1 --2- 172.21.15.162:0/15592591 >> [v2:172.21.15.162:6800/1396866108,v1:172.21.15.162:6801/1396866108] conn(0x7f174c06ec00 0x7f174c0710b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:17:18.911 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:17:18.910+0000 7f1756ffd700 1 -- 172.21.15.162:0/15592591 <== mon.0 v2:172.21.15.162:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 6410+0+0 (secure 0 0 0) 0x7f175008e420 con 0x7f17600fbd40 2024-04-27T04:17:18.911 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:17:18.910+0000 7f1765b83700 1 --2- 172.21.15.162:0/15592591 >> [v2:172.21.15.162:6800/1396866108,v1:172.21.15.162:6801/1396866108] conn(0x7f174c06ec00 0x7f174c0710b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:17:18.911 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:17:18.911+0000 7f1765b83700 1 --2- 172.21.15.162:0/15592591 >> [v2:172.21.15.162:6800/1396866108,v1:172.21.15.162:6801/1396866108] conn(0x7f174c06ec00 0x7f174c0710b0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f175c009b30 tx=0x7f175c009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:17:18.914 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:17:18.915+0000 7f1756ffd700 1 -- 172.21.15.162:0/15592591 <== mon.0 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7f175005ae00 con 0x7f17600fbd40 2024-04-27T04:17:19.135 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:17:19.136+0000 7f17685e8700 1 -- 172.21.15.162:0/15592591 --> [v2:172.21.15.162:6800/1396866108,v1:172.21.15.162:6801/1396866108] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f1760108df0 con 0x7f174c06ec00 2024-04-27T04:17:19.135 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:17:18 smithi162 ceph-mon[110641]: pgmap v108: 65 pgs: 65 active+clean; 208 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:17:19.135 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:17:18 smithi162 ceph-mon[110641]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' 2024-04-27T04:17:19.135 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:17:18 smithi162 ceph-mon[110641]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-27T04:17:19.135 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:17:18 smithi162 ceph-mon[110641]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "osd ok-to-stop", "ids": ["3"], "max": 16}]: dispatch 2024-04-27T04:17:19.136 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:17:19.137+0000 7f1756ffd700 1 -- 172.21.15.162:0/15592591 <== mgr.34104 v2:172.21.15.162:6800/1396866108 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+400 (secure 0 0 0) 0x7f1760108df0 con 0x7f174c06ec00 2024-04-27T04:17:19.138 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:17:19.139+0000 7f17685e8700 1 -- 172.21.15.162:0/15592591 >> [v2:172.21.15.162:6800/1396866108,v1:172.21.15.162:6801/1396866108] conn(0x7f174c06ec00 msgr2=0x7f174c0710b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:17:19.138 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:17:19.139+0000 7f17685e8700 1 --2- 172.21.15.162:0/15592591 >> [v2:172.21.15.162:6800/1396866108,v1:172.21.15.162:6801/1396866108] conn(0x7f174c06ec00 0x7f174c0710b0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f175c009b30 tx=0x7f175c009e30 comp rx=0 tx=0).stop 2024-04-27T04:17:19.138 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:17:19.139+0000 7f17685e8700 1 -- 172.21.15.162:0/15592591 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f17600fbd40 msgr2=0x7f1760113b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:17:19.139 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:17:19.139+0000 7f17685e8700 1 --2- 172.21.15.162:0/15592591 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f17600fbd40 0x7f1760113b00 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f175000eb40 tx=0x7f175000b040 comp rx=0 tx=0).stop 2024-04-27T04:17:19.139 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:17:19.139+0000 7f17685e8700 1 -- 172.21.15.162:0/15592591 shutdown_connections 2024-04-27T04:17:19.139 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:17:19.139+0000 7f17685e8700 1 --2- 172.21.15.162:0/15592591 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f17600fbd40 0x7f1760113b00 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:17:19.139 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:17:19.139+0000 7f17685e8700 1 --2- 172.21.15.162:0/15592591 >> [v2:172.21.15.162:6800/1396866108,v1:172.21.15.162:6801/1396866108] conn(0x7f174c06ec00 0x7f174c0710b0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:17:19.139 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:17:19.139+0000 7f17685e8700 1 --2- 172.21.15.162:0/15592591 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1760101d90 0x7f1760114040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:17:19.139 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:17:19.139+0000 7f17685e8700 1 -- 172.21.15.162:0/15592591 >> 172.21.15.162:0/15592591 conn(0x7f1760071d00 msgr2=0x7f1760065620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:17:19.139 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:17:19.139+0000 7f17685e8700 1 -- 172.21.15.162:0/15592591 shutdown_connections 2024-04-27T04:17:19.139 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:17:19.139+0000 7f17685e8700 1 -- 172.21.15.162:0/15592591 wait complete. 2024-04-27T04:17:19.149 INFO:teuthology.orchestra.run.smithi162.stdout:true 2024-04-27T04:17:19.242 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:17:19.241+0000 7f02d0b94700 1 -- 172.21.15.162:0/2411946443 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f02cc1026b0 msgr2=0x7f02cc0fbd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:17:19.242 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:17:19.241+0000 7f02d0b94700 1 --2- 172.21.15.162:0/2411946443 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f02cc1026b0 0x7f02cc0fbd20 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f02c0004940 tx=0x7f02c0009e30 comp rx=0 tx=0).stop 2024-04-27T04:17:19.242 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:17:19.241+0000 7f02d0b94700 1 -- 172.21.15.162:0/2411946443 shutdown_connections 2024-04-27T04:17:19.242 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:17:19.241+0000 7f02d0b94700 1 --2- 172.21.15.162:0/2411946443 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f02cc1026b0 0x7f02cc0fbd20 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:17:19.242 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:17:19.241+0000 7f02d0b94700 1 --2- 172.21.15.162:0/2411946443 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f02cc101d10 0x7f02cc1020e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:17:19.242 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:17:19.241+0000 7f02d0b94700 1 -- 172.21.15.162:0/2411946443 >> 172.21.15.162:0/2411946443 conn(0x7f02cc0f7820 msgr2=0x7f02cc0f9c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:17:19.242 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:17:19.242+0000 7f02d0b94700 1 -- 172.21.15.162:0/2411946443 shutdown_connections 2024-04-27T04:17:19.242 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:17:19.242+0000 7f02d0b94700 1 -- 172.21.15.162:0/2411946443 wait complete. 2024-04-27T04:17:19.243 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:17:19.242+0000 7f02d0b94700 1 Processor -- start 2024-04-27T04:17:19.243 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:17:19.242+0000 7f02d0b94700 1 -- start start 2024-04-27T04:17:19.243 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:17:19.242+0000 7f02d0b94700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f02cc101d10 0x7f02cc10ee60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:17:19.243 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:17:19.242+0000 7f02d0b94700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f02cc1026b0 0x7f02cc10f3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:17:19.243 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:17:19.242+0000 7f02d0b94700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02cc10f8e0 con 0x7f02cc101d10 2024-04-27T04:17:19.243 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:17:19.242+0000 7f02d0b94700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02cc10fa20 con 0x7f02cc1026b0 2024-04-27T04:17:19.243 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:17:19.242+0000 7f02ca59c700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f02cc101d10 0x7f02cc10ee60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:17:19.243 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:17:19.242+0000 7f02ca59c700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f02cc101d10 0x7f02cc10ee60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:42540/0 (socket says 172.21.15.162:42540) 2024-04-27T04:17:19.243 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:17:19.242+0000 7f02ca59c700 1 -- 172.21.15.162:0/2204638180 learned_addr learned my addr 172.21.15.162:0/2204638180 (peer_addr_for_me v2:172.21.15.162:0/0) 2024-04-27T04:17:19.243 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:17:19.243+0000 7f02c9d9b700 1 --2- 172.21.15.162:0/2204638180 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f02cc1026b0 0x7f02cc10f3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:17:19.243 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:17:19.243+0000 7f02ca59c700 1 -- 172.21.15.162:0/2204638180 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f02cc1026b0 msgr2=0x7f02cc10f3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:17:19.243 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:17:19.243+0000 7f02ca59c700 1 --2- 172.21.15.162:0/2204638180 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f02cc1026b0 0x7f02cc10f3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:17:19.243 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:17:19.243+0000 7f02ca59c700 1 -- 172.21.15.162:0/2204638180 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f02c0005040 con 0x7f02cc101d10 2024-04-27T04:17:19.243 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:17:19.243+0000 7f02c9d9b700 1 --2- 172.21.15.162:0/2204638180 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f02cc1026b0 0x7f02cc10f3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-27T04:17:19.244 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:17:19.243+0000 7f02ca59c700 1 --2- 172.21.15.162:0/2204638180 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f02cc101d10 0x7f02cc10ee60 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f02bc00c830 tx=0x7f02bc004470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:17:19.244 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:17:19.243+0000 7f02baffd700 1 -- 172.21.15.162:0/2204638180 <== mon.0 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f02bc00a760 con 0x7f02cc101d10 2024-04-27T04:17:19.244 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:17:19.243+0000 7f02baffd700 1 -- 172.21.15.162:0/2204638180 <== mon.0 v2:172.21.15.162:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f02bc00ad90 con 0x7f02cc101d10 2024-04-27T04:17:19.244 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:17:19.243+0000 7f02d0b94700 1 -- 172.21.15.162:0/2204638180 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f02cc117ed0 con 0x7f02cc101d10 2024-04-27T04:17:19.244 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:17:19.243+0000 7f02baffd700 1 -- 172.21.15.162:0/2204638180 <== mon.0 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f02bc0113a0 con 0x7f02cc101d10 2024-04-27T04:17:19.244 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:17:19.243+0000 7f02d0b94700 1 -- 172.21.15.162:0/2204638180 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f02cc1182a0 con 0x7f02cc101d10 2024-04-27T04:17:19.245 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:17:19.244+0000 7f02d0b94700 1 -- 172.21.15.162:0/2204638180 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f02cc0fd460 con 0x7f02cc101d10 2024-04-27T04:17:19.245 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:17:19.245+0000 7f02baffd700 1 -- 172.21.15.162:0/2204638180 <== mon.0 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 39) v1 ==== 92747+0+0 (secure 0 0 0) 0x7f02bc012400 con 0x7f02cc101d10 2024-04-27T04:17:19.245 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:17:19.245+0000 7f02baffd700 1 --2- 172.21.15.162:0/2204638180 >> [v2:172.21.15.162:6800/1396866108,v1:172.21.15.162:6801/1396866108] conn(0x7f02b406ec00 0x7f02b40710b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:17:19.245 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:17:19.245+0000 7f02c9d9b700 1 --2- 172.21.15.162:0/2204638180 >> [v2:172.21.15.162:6800/1396866108,v1:172.21.15.162:6801/1396866108] conn(0x7f02b406ec00 0x7f02b40710b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:17:19.245 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:17:19.245+0000 7f02baffd700 1 -- 172.21.15.162:0/2204638180 <== mon.0 v2:172.21.15.162:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 6410+0+0 (secure 0 0 0) 0x7f02bc08e7a0 con 0x7f02cc101d10 2024-04-27T04:17:19.245 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:17:19.245+0000 7f02c9d9b700 1 --2- 172.21.15.162:0/2204638180 >> [v2:172.21.15.162:6800/1396866108,v1:172.21.15.162:6801/1396866108] conn(0x7f02b406ec00 0x7f02b40710b0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f02c000b3b0 tx=0x7f02c0009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:17:19.248 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:17:19.250+0000 7f02baffd700 1 -- 172.21.15.162:0/2204638180 <== mon.0 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7f02bc0576c0 con 0x7f02cc101d10 2024-04-27T04:17:19.470 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:17:19.471+0000 7f02d0b94700 1 -- 172.21.15.162:0/2204638180 --> [v2:172.21.15.162:6800/1396866108,v1:172.21.15.162:6801/1396866108] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f02cc0668b0 con 0x7f02b406ec00 2024-04-27T04:17:19.471 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:17:19.472+0000 7f02baffd700 1 -- 172.21.15.162:0/2204638180 <== mgr.34104 v2:172.21.15.162:6800/1396866108 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+400 (secure 0 0 0) 0x7f02cc0668b0 con 0x7f02b406ec00 2024-04-27T04:17:19.474 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:17:19.475+0000 7f02d0b94700 1 -- 172.21.15.162:0/2204638180 >> [v2:172.21.15.162:6800/1396866108,v1:172.21.15.162:6801/1396866108] conn(0x7f02b406ec00 msgr2=0x7f02b40710b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:17:19.474 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:17:19.475+0000 7f02d0b94700 1 --2- 172.21.15.162:0/2204638180 >> [v2:172.21.15.162:6800/1396866108,v1:172.21.15.162:6801/1396866108] conn(0x7f02b406ec00 0x7f02b40710b0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f02c000b3b0 tx=0x7f02c0009e30 comp rx=0 tx=0).stop 2024-04-27T04:17:19.474 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:17:19.475+0000 7f02d0b94700 1 -- 172.21.15.162:0/2204638180 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f02cc101d10 msgr2=0x7f02cc10ee60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:17:19.474 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:17:19.475+0000 7f02d0b94700 1 --2- 172.21.15.162:0/2204638180 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f02cc101d10 0x7f02cc10ee60 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f02bc00c830 tx=0x7f02bc004470 comp rx=0 tx=0).stop 2024-04-27T04:17:19.474 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:17:19.475+0000 7f02d0b94700 1 -- 172.21.15.162:0/2204638180 shutdown_connections 2024-04-27T04:17:19.474 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:17:19.475+0000 7f02d0b94700 1 --2- 172.21.15.162:0/2204638180 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f02cc101d10 0x7f02cc10ee60 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:17:19.474 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:17:19.475+0000 7f02d0b94700 1 --2- 172.21.15.162:0/2204638180 >> [v2:172.21.15.162:6800/1396866108,v1:172.21.15.162:6801/1396866108] conn(0x7f02b406ec00 0x7f02b40710b0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:17:19.474 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:17:19.475+0000 7f02d0b94700 1 --2- 172.21.15.162:0/2204638180 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f02cc1026b0 0x7f02cc10f3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:17:19.474 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:17:19.475+0000 7f02d0b94700 1 -- 172.21.15.162:0/2204638180 >> 172.21.15.162:0/2204638180 conn(0x7f02cc0f7820 msgr2=0x7f02cc0f8830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:17:19.474 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:17:19.475+0000 7f02d0b94700 1 -- 172.21.15.162:0/2204638180 shutdown_connections 2024-04-27T04:17:19.475 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:17:19.475+0000 7f02d0b94700 1 -- 172.21.15.162:0/2204638180 wait complete. 2024-04-27T04:17:19.576 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:17:19.577+0000 7fc8157a1700 1 -- 172.21.15.162:0/422428538 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc810102bb0 msgr2=0x7fc810102f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:17:19.576 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:17:19.577+0000 7fc8157a1700 1 --2- 172.21.15.162:0/422428538 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc810102bb0 0x7fc810102f80 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fc800004940 tx=0x7fc800009e30 comp rx=0 tx=0).stop 2024-04-27T04:17:19.576 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:17:19.577+0000 7fc8157a1700 1 -- 172.21.15.162:0/422428538 shutdown_connections 2024-04-27T04:17:19.576 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:17:19.577+0000 7fc8157a1700 1 --2- 172.21.15.162:0/422428538 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc810103550 0x7fc8100fb0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:17:19.576 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:17:19.577+0000 7fc8157a1700 1 --2- 172.21.15.162:0/422428538 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc810102bb0 0x7fc810102f80 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:17:19.576 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:17:19.577+0000 7fc8157a1700 1 -- 172.21.15.162:0/422428538 >> 172.21.15.162:0/422428538 conn(0x7fc810074ff0 msgr2=0x7fc8100f9270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:17:19.576 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:17:19.577+0000 7fc8157a1700 1 -- 172.21.15.162:0/422428538 shutdown_connections 2024-04-27T04:17:19.577 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:17:19.577+0000 7fc8157a1700 1 -- 172.21.15.162:0/422428538 wait complete. 2024-04-27T04:17:19.577 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:17:19.577+0000 7fc8157a1700 1 Processor -- start 2024-04-27T04:17:19.578 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:17:19.578+0000 7fc8157a1700 1 -- start start 2024-04-27T04:17:19.578 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:17:19.578+0000 7fc8157a1700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc810102bb0 0x7fc8100fc2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:17:19.578 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:17:19.578+0000 7fc8157a1700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc810103550 0x7fc8100fc7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:17:19.579 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:17:19.578+0000 7fc8157a1700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8100fcd70 con 0x7fc810103550 2024-04-27T04:17:19.579 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:17:19.578+0000 7fc8157a1700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8100fceb0 con 0x7fc810102bb0 2024-04-27T04:17:19.579 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:17:19.578+0000 7fc80f7fe700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc810103550 0x7fc8100fc7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:17:19.579 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:17:19.578+0000 7fc80f7fe700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc810103550 0x7fc8100fc7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:42554/0 (socket says 172.21.15.162:42554) 2024-04-27T04:17:19.579 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:17:19.578+0000 7fc80f7fe700 1 -- 172.21.15.162:0/2251972035 learned_addr learned my addr 172.21.15.162:0/2251972035 (peer_addr_for_me v2:172.21.15.162:0/0) 2024-04-27T04:17:19.579 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:17:19.578+0000 7fc80ffff700 1 --2- 172.21.15.162:0/2251972035 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc810102bb0 0x7fc8100fc2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:17:19.579 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:17:19.578+0000 7fc80f7fe700 1 -- 172.21.15.162:0/2251972035 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc810102bb0 msgr2=0x7fc8100fc2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:17:19.579 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:17:19.578+0000 7fc80f7fe700 1 --2- 172.21.15.162:0/2251972035 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc810102bb0 0x7fc8100fc2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:17:19.579 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:17:19.578+0000 7fc80f7fe700 1 -- 172.21.15.162:0/2251972035 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc800005040 con 0x7fc810103550 2024-04-27T04:17:19.579 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:17:19.578+0000 7fc80ffff700 1 --2- 172.21.15.162:0/2251972035 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc810102bb0 0x7fc8100fc2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-27T04:17:19.579 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:17:19.578+0000 7fc80f7fe700 1 --2- 172.21.15.162:0/2251972035 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc810103550 0x7fc8100fc7e0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fc80400eb40 tx=0x7fc80400b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:17:19.579 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:17:19.579+0000 7fc80cff9700 1 -- 172.21.15.162:0/2251972035 <== mon.0 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc804007cc0 con 0x7fc810103550 2024-04-27T04:17:19.579 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:17:19.579+0000 7fc80cff9700 1 -- 172.21.15.162:0/2251972035 <== mon.0 v2:172.21.15.162:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fc80400f040 con 0x7fc810103550 2024-04-27T04:17:19.580 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:17:19.579+0000 7fc80cff9700 1 -- 172.21.15.162:0/2251972035 <== mon.0 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc80400cad0 con 0x7fc810103550 2024-04-27T04:17:19.580 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:17:19.579+0000 7fc8157a1700 1 -- 172.21.15.162:0/2251972035 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc8100fd100 con 0x7fc810103550 2024-04-27T04:17:19.580 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:17:19.579+0000 7fc8157a1700 1 -- 172.21.15.162:0/2251972035 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc81011a010 con 0x7fc810103550 2024-04-27T04:17:19.580 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:17:19.580+0000 7fc8157a1700 1 -- 172.21.15.162:0/2251972035 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc8100fe3e0 con 0x7fc810103550 2024-04-27T04:17:19.580 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:17:19.581+0000 7fc80cff9700 1 -- 172.21.15.162:0/2251972035 <== mon.0 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 39) v1 ==== 92747+0+0 (secure 0 0 0) 0x7fc80400cc30 con 0x7fc810103550 2024-04-27T04:17:19.581 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:17:19.582+0000 7fc80cff9700 1 --2- 172.21.15.162:0/2251972035 >> [v2:172.21.15.162:6800/1396866108,v1:172.21.15.162:6801/1396866108] conn(0x7fc7f806ec00 0x7fc7f80710b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:17:19.581 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:17:19.582+0000 7fc80cff9700 1 -- 172.21.15.162:0/2251972035 <== mon.0 v2:172.21.15.162:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 6410+0+0 (secure 0 0 0) 0x7fc80408e300 con 0x7fc810103550 2024-04-27T04:17:19.581 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:17:19.582+0000 7fc80ffff700 1 --2- 172.21.15.162:0/2251972035 >> [v2:172.21.15.162:6800/1396866108,v1:172.21.15.162:6801/1396866108] conn(0x7fc7f806ec00 0x7fc7f80710b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:17:19.581 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:17:19.582+0000 7fc80ffff700 1 --2- 172.21.15.162:0/2251972035 >> [v2:172.21.15.162:6800/1396866108,v1:172.21.15.162:6801/1396866108] conn(0x7fc7f806ec00 0x7fc7f80710b0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fc800009b30 tx=0x7fc800009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:17:19.588 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:17:19.589+0000 7fc80cff9700 1 -- 172.21.15.162:0/2251972035 <== mon.0 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7fc804057180 con 0x7fc810103550 2024-04-27T04:17:19.796 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:17:19.797+0000 7fc8157a1700 1 -- 172.21.15.162:0/2251972035 --> [v2:172.21.15.162:6800/1396866108,v1:172.21.15.162:6801/1396866108] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7fc81004bed0 con 0x7fc7f806ec00 2024-04-27T04:17:19.796 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:17:19 smithi162 ceph-mon[110641]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["3"], "max": 16}]: dispatch 2024-04-27T04:17:19.797 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:17:19 smithi162 ceph-mon[110641]: Upgrade: osd.3 is safe to restart 2024-04-27T04:17:19.797 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:17:19 smithi162 ceph-mon[110641]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' 2024-04-27T04:17:19.797 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:17:19 smithi162 ceph-mon[110641]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-04-27T04:17:19.797 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:17:19 smithi162 ceph-mon[110641]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T04:17:19.807 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:17:19.808+0000 7fc80cff9700 1 -- 172.21.15.162:0/2251972035 <== mgr.34104 v2:172.21.15.162:6800/1396866108 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3498 (secure 0 0 0) 0x7fc81004bed0 con 0x7fc7f806ec00 2024-04-27T04:17:19.807 INFO:teuthology.orchestra.run.smithi162.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-27T04:17:19.807 INFO:teuthology.orchestra.run.smithi162.stdout:alertmanager.smithi162 smithi162 *:9093,9094 running (4m) 21s ago 15m 27.2M - 0.25.0 c8568f914cd2 40e334659017 2024-04-27T04:17:19.807 INFO:teuthology.orchestra.run.smithi162.stdout:crash.smithi162 smithi162 running (2m) 21s ago 15m 6660k - 19.0.0-3253-g4e213a58 67e414ff4bc4 1aa8f4d5a28b 2024-04-27T04:17:19.807 INFO:teuthology.orchestra.run.smithi162.stdout:crash.smithi181 smithi181 running (2m) 116s ago 14m 6656k - 19.0.0-3253-g4e213a58 67e414ff4bc4 832d9cbc5c30 2024-04-27T04:17:19.808 INFO:teuthology.orchestra.run.smithi162.stdout:grafana.smithi162 smithi162 *:3000 running (4m) 21s ago 15m 80.5M - 9.4.12 2bacad6d85d8 3d78c478f03f 2024-04-27T04:17:19.808 INFO:teuthology.orchestra.run.smithi162.stdout:mds.cephfs.smithi162.gzlyjj smithi162 running (10m) 21s ago 10m 14.3M - 17.2.7-907-g2be3efa9 1c5834633715 27e5b030dd32 2024-04-27T04:17:19.808 INFO:teuthology.orchestra.run.smithi162.stdout:mds.cephfs.smithi162.xcnapk smithi162 running (10m) 21s ago 10m 100M - 17.2.7-907-g2be3efa9 1c5834633715 38f9030df029 2024-04-27T04:17:19.808 INFO:teuthology.orchestra.run.smithi162.stdout:mds.cephfs.smithi181.bkkcdm smithi181 running (10m) 116s ago 10m 15.3M - 17.2.7-907-g2be3efa9 1c5834633715 af55afca1263 2024-04-27T04:17:19.808 INFO:teuthology.orchestra.run.smithi162.stdout:mds.cephfs.smithi181.dzomgh smithi181 running (10m) 116s ago 10m 143M - 17.2.7-907-g2be3efa9 1c5834633715 6733296597f3 2024-04-27T04:17:19.808 INFO:teuthology.orchestra.run.smithi162.stdout:mgr.smithi162.oennfn smithi162 *:8443,9283,8765 running (6m) 21s ago 17m 505M - 19.0.0-3253-g4e213a58 67e414ff4bc4 58a748d3637b 2024-04-27T04:17:19.808 INFO:teuthology.orchestra.run.smithi162.stdout:mgr.smithi181.vtwuhz smithi181 *:8443,9283,8765 running (5m) 116s ago 14m 450M - 19.0.0-3253-g4e213a58 67e414ff4bc4 813584297b1d 2024-04-27T04:17:19.808 INFO:teuthology.orchestra.run.smithi162.stdout:mon.smithi162 smithi162 running (3m) 21s ago 17m 50.4M 2048M 19.0.0-3253-g4e213a58 67e414ff4bc4 375adaf4468f 2024-04-27T04:17:19.808 INFO:teuthology.orchestra.run.smithi162.stdout:mon.smithi181 smithi181 running (2m) 116s ago 14m 33.1M 2048M 19.0.0-3253-g4e213a58 67e414ff4bc4 77b71f6d522f 2024-04-27T04:17:19.808 INFO:teuthology.orchestra.run.smithi162.stdout:node-exporter.smithi162 smithi162 *:9100 running (5m) 21s ago 15m 15.4M - 1.5.0 0da6a335fe13 5c4c5ea9c1d7 2024-04-27T04:17:19.808 INFO:teuthology.orchestra.run.smithi162.stdout:node-exporter.smithi181 smithi181 *:9100 running (5m) 116s ago 14m 15.7M - 1.5.0 0da6a335fe13 9067f14c0519 2024-04-27T04:17:19.808 INFO:teuthology.orchestra.run.smithi162.stdout:osd.0 smithi162 running (99s) 21s ago 13m 87.7M 1557M 19.0.0-3253-g4e213a58 67e414ff4bc4 2744aff3cd88 2024-04-27T04:17:19.808 INFO:teuthology.orchestra.run.smithi162.stdout:osd.1 smithi162 running (63s) 21s ago 13m 93.8M 1557M 19.0.0-3253-g4e213a58 67e414ff4bc4 668e16faad62 2024-04-27T04:17:19.808 INFO:teuthology.orchestra.run.smithi162.stdout:osd.2 smithi162 running (27s) 21s ago 13m 15.1M 1557M 19.0.0-3253-g4e213a58 67e414ff4bc4 d24878d461e7 2024-04-27T04:17:19.809 INFO:teuthology.orchestra.run.smithi162.stdout:osd.3 smithi181 running (12m) 116s ago 12m 278M 2581M 17.2.7-907-g2be3efa9 1c5834633715 030d829be54e 2024-04-27T04:17:19.809 INFO:teuthology.orchestra.run.smithi162.stdout:osd.4 smithi181 running (12m) 116s ago 12m 247M 2581M 17.2.7-907-g2be3efa9 1c5834633715 512afbee7cf7 2024-04-27T04:17:19.809 INFO:teuthology.orchestra.run.smithi162.stdout:osd.5 smithi181 running (11m) 116s ago 11m 224M 2581M 17.2.7-907-g2be3efa9 1c5834633715 f47a1daac1ea 2024-04-27T04:17:19.809 INFO:teuthology.orchestra.run.smithi162.stdout:prometheus.smithi162 smithi162 *:9095 running (4m) 21s ago 15m 67.6M - 2.43.0 a07b618ecd1d f30657a43324 2024-04-27T04:17:19.810 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:17:19.810+0000 7fc8157a1700 1 -- 172.21.15.162:0/2251972035 >> [v2:172.21.15.162:6800/1396866108,v1:172.21.15.162:6801/1396866108] conn(0x7fc7f806ec00 msgr2=0x7fc7f80710b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:17:19.810 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:17:19.810+0000 7fc8157a1700 1 --2- 172.21.15.162:0/2251972035 >> [v2:172.21.15.162:6800/1396866108,v1:172.21.15.162:6801/1396866108] conn(0x7fc7f806ec00 0x7fc7f80710b0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fc800009b30 tx=0x7fc800009e30 comp rx=0 tx=0).stop 2024-04-27T04:17:19.810 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:17:19.810+0000 7fc8157a1700 1 -- 172.21.15.162:0/2251972035 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc810103550 msgr2=0x7fc8100fc7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:17:19.810 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:17:19.810+0000 7fc8157a1700 1 --2- 172.21.15.162:0/2251972035 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc810103550 0x7fc8100fc7e0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fc80400eb40 tx=0x7fc80400b040 comp rx=0 tx=0).stop 2024-04-27T04:17:19.810 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:17:19.810+0000 7fc8157a1700 1 -- 172.21.15.162:0/2251972035 shutdown_connections 2024-04-27T04:17:19.810 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:17:19.810+0000 7fc8157a1700 1 --2- 172.21.15.162:0/2251972035 >> [v2:172.21.15.162:6800/1396866108,v1:172.21.15.162:6801/1396866108] conn(0x7fc7f806ec00 0x7fc7f80710b0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:17:19.810 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:17:19.810+0000 7fc8157a1700 1 --2- 172.21.15.162:0/2251972035 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc810102bb0 0x7fc8100fc2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:17:19.810 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:17:19.810+0000 7fc8157a1700 1 --2- 172.21.15.162:0/2251972035 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc810103550 0x7fc8100fc7e0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:17:19.810 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:17:19.810+0000 7fc8157a1700 1 -- 172.21.15.162:0/2251972035 >> 172.21.15.162:0/2251972035 conn(0x7fc810074ff0 msgr2=0x7fc810105930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:17:19.810 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:17:19.810+0000 7fc8157a1700 1 -- 172.21.15.162:0/2251972035 shutdown_connections 2024-04-27T04:17:19.811 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:17:19.810+0000 7fc8157a1700 1 -- 172.21.15.162:0/2251972035 wait complete. 2024-04-27T04:17:19.911 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:17:19.911+0000 7fd144ebc700 1 -- 172.21.15.162:0/2452015923 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd1400fddc0 msgr2=0x7fd1400fe190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:17:19.912 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:17:19.911+0000 7fd144ebc700 1 --2- 172.21.15.162:0/2452015923 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd1400fddc0 0x7fd1400fe190 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fd130004940 tx=0x7fd130009e30 comp rx=0 tx=0).stop 2024-04-27T04:17:19.912 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:17:19.911+0000 7fd144ebc700 1 -- 172.21.15.162:0/2452015923 shutdown_connections 2024-04-27T04:17:19.912 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:17:19.911+0000 7fd144ebc700 1 --2- 172.21.15.162:0/2452015923 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd1400fe760 0x7fd140106e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:17:19.912 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:17:19.911+0000 7fd144ebc700 1 --2- 172.21.15.162:0/2452015923 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd1400fddc0 0x7fd1400fe190 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:17:19.912 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:17:19.911+0000 7fd144ebc700 1 -- 172.21.15.162:0/2452015923 >> 172.21.15.162:0/2452015923 conn(0x7fd1400f7860 msgr2=0x7fd1400f9c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:17:19.912 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:17:19.911+0000 7fd144ebc700 1 -- 172.21.15.162:0/2452015923 shutdown_connections 2024-04-27T04:17:19.912 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:17:19.911+0000 7fd144ebc700 1 -- 172.21.15.162:0/2452015923 wait complete. 2024-04-27T04:17:19.912 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:17:19.912+0000 7fd144ebc700 1 Processor -- start 2024-04-27T04:17:19.912 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:17:19.912+0000 7fd144ebc700 1 -- start start 2024-04-27T04:17:19.912 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:17:19.912+0000 7fd144ebc700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd1400fddc0 0x7fd140113970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:17:19.912 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:17:19.912+0000 7fd144ebc700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd1400fe760 0x7fd140113eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:17:19.912 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:17:19.912+0000 7fd144ebc700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd140114500 con 0x7fd1400fe760 2024-04-27T04:17:19.913 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:17:19.912+0000 7fd144ebc700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd14010db20 con 0x7fd1400fddc0 2024-04-27T04:17:19.913 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:17:19.912+0000 7fd13dd9b700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd1400fe760 0x7fd140113eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:17:19.913 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:17:19.912+0000 7fd13dd9b700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd1400fe760 0x7fd140113eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:42568/0 (socket says 172.21.15.162:42568) 2024-04-27T04:17:19.913 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:17:19.912+0000 7fd13dd9b700 1 -- 172.21.15.162:0/2703576298 learned_addr learned my addr 172.21.15.162:0/2703576298 (peer_addr_for_me v2:172.21.15.162:0/0) 2024-04-27T04:17:19.913 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:17:19.912+0000 7fd13e59c700 1 --2- 172.21.15.162:0/2703576298 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd1400fddc0 0x7fd140113970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:17:19.913 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:17:19.912+0000 7fd13dd9b700 1 -- 172.21.15.162:0/2703576298 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd1400fddc0 msgr2=0x7fd140113970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:17:19.913 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:17:19.912+0000 7fd13dd9b700 1 --2- 172.21.15.162:0/2703576298 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd1400fddc0 0x7fd140113970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:17:19.913 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:17:19.912+0000 7fd13dd9b700 1 -- 172.21.15.162:0/2703576298 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd130005040 con 0x7fd1400fe760 2024-04-27T04:17:19.913 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:17:19.912+0000 7fd13dd9b700 1 --2- 172.21.15.162:0/2703576298 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd1400fe760 0x7fd140113eb0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fd134002a50 tx=0x7fd1340046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:17:19.913 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:17:19.912+0000 7fd12effd700 1 -- 172.21.15.162:0/2703576298 <== mon.0 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd134007cc0 con 0x7fd1400fe760 2024-04-27T04:17:19.913 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:17:19.912+0000 7fd144ebc700 1 -- 172.21.15.162:0/2703576298 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd14010de00 con 0x7fd1400fe760 2024-04-27T04:17:19.913 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:17:19.912+0000 7fd12effd700 1 -- 172.21.15.162:0/2703576298 <== mon.0 v2:172.21.15.162:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fd13400f040 con 0x7fd1400fe760 2024-04-27T04:17:19.913 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:17:19.913+0000 7fd12effd700 1 -- 172.21.15.162:0/2703576298 <== mon.0 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd13400da30 con 0x7fd1400fe760 2024-04-27T04:17:19.913 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:17:19.913+0000 7fd144ebc700 1 -- 172.21.15.162:0/2703576298 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd14010e340 con 0x7fd1400fe760 2024-04-27T04:17:19.914 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:17:19.913+0000 7fd144ebc700 1 -- 172.21.15.162:0/2703576298 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd140066af0 con 0x7fd1400fe760 2024-04-27T04:17:19.914 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:17:19.914+0000 7fd12effd700 1 -- 172.21.15.162:0/2703576298 <== mon.0 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 39) v1 ==== 92747+0+0 (secure 0 0 0) 0x7fd13400db90 con 0x7fd1400fe760 2024-04-27T04:17:19.915 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:17:19.914+0000 7fd12effd700 1 --2- 172.21.15.162:0/2703576298 >> [v2:172.21.15.162:6800/1396866108,v1:172.21.15.162:6801/1396866108] conn(0x7fd12806ec00 0x7fd1280710b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:17:19.915 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:17:19.914+0000 7fd12effd700 1 -- 172.21.15.162:0/2703576298 <== mon.0 v2:172.21.15.162:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 6410+0+0 (secure 0 0 0) 0x7fd13408e5d0 con 0x7fd1400fe760 2024-04-27T04:17:19.915 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:17:19.914+0000 7fd13e59c700 1 --2- 172.21.15.162:0/2703576298 >> [v2:172.21.15.162:6800/1396866108,v1:172.21.15.162:6801/1396866108] conn(0x7fd12806ec00 0x7fd1280710b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:17:19.915 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:17:19.915+0000 7fd13e59c700 1 --2- 172.21.15.162:0/2703576298 >> [v2:172.21.15.162:6800/1396866108,v1:172.21.15.162:6801/1396866108] conn(0x7fd12806ec00 0x7fd1280710b0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fd130004f90 tx=0x7fd130009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:17:19.918 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:17:19.919+0000 7fd12effd700 1 -- 172.21.15.162:0/2703576298 <== mon.0 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7fd1340573d0 con 0x7fd1400fe760 2024-04-27T04:17:19.922 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:17:19 smithi181 ceph-mon[81134]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["3"], "max": 16}]: dispatch 2024-04-27T04:17:19.922 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:17:19 smithi181 ceph-mon[81134]: Upgrade: osd.3 is safe to restart 2024-04-27T04:17:19.922 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:17:19 smithi181 ceph-mon[81134]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' 2024-04-27T04:17:19.922 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:17:19 smithi181 ceph-mon[81134]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-04-27T04:17:19.922 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:17:19 smithi181 ceph-mon[81134]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T04:17:20.201 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:17:20.202+0000 7fd144ebc700 1 -- 172.21.15.162:0/2703576298 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7fd140109e50 con 0x7fd1400fe760 2024-04-27T04:17:20.202 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:17:20.203+0000 7fd12effd700 1 -- 172.21.15.162:0/2703576298 <== mon.0 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+839 (secure 0 0 0) 0x7fd13405aa90 con 0x7fd1400fe760 2024-04-27T04:17:20.202 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-04-27T04:17:20.202 INFO:teuthology.orchestra.run.smithi162.stdout: "mon": { 2024-04-27T04:17:20.202 INFO:teuthology.orchestra.run.smithi162.stdout: "ceph version 19.0.0-3253-g4e213a58 (4e213a58e787437ff508e6c890ccd5fd32dc96b5) squid (dev)": 2 2024-04-27T04:17:20.202 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-04-27T04:17:20.202 INFO:teuthology.orchestra.run.smithi162.stdout: "mgr": { 2024-04-27T04:17:20.202 INFO:teuthology.orchestra.run.smithi162.stdout: "ceph version 19.0.0-3253-g4e213a58 (4e213a58e787437ff508e6c890ccd5fd32dc96b5) squid (dev)": 2 2024-04-27T04:17:20.202 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-04-27T04:17:20.203 INFO:teuthology.orchestra.run.smithi162.stdout: "osd": { 2024-04-27T04:17:20.203 INFO:teuthology.orchestra.run.smithi162.stdout: "ceph version 17.2.7-907-g2be3efa9 (2be3efa97a7972f8e925bd899b474329979b74d6) quincy (stable)": 3, 2024-04-27T04:17:20.203 INFO:teuthology.orchestra.run.smithi162.stdout: "ceph version 19.0.0-3253-g4e213a58 (4e213a58e787437ff508e6c890ccd5fd32dc96b5) squid (dev)": 3 2024-04-27T04:17:20.203 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-04-27T04:17:20.203 INFO:teuthology.orchestra.run.smithi162.stdout: "mds": { 2024-04-27T04:17:20.203 INFO:teuthology.orchestra.run.smithi162.stdout: "ceph version 17.2.7-907-g2be3efa9 (2be3efa97a7972f8e925bd899b474329979b74d6) quincy (stable)": 4 2024-04-27T04:17:20.203 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-04-27T04:17:20.203 INFO:teuthology.orchestra.run.smithi162.stdout: "overall": { 2024-04-27T04:17:20.203 INFO:teuthology.orchestra.run.smithi162.stdout: "ceph version 17.2.7-907-g2be3efa9 (2be3efa97a7972f8e925bd899b474329979b74d6) quincy (stable)": 7, 2024-04-27T04:17:20.203 INFO:teuthology.orchestra.run.smithi162.stdout: "ceph version 19.0.0-3253-g4e213a58 (4e213a58e787437ff508e6c890ccd5fd32dc96b5) squid (dev)": 7 2024-04-27T04:17:20.203 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-04-27T04:17:20.203 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-04-27T04:17:20.204 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:17:20.205+0000 7fd144ebc700 1 -- 172.21.15.162:0/2703576298 >> [v2:172.21.15.162:6800/1396866108,v1:172.21.15.162:6801/1396866108] conn(0x7fd12806ec00 msgr2=0x7fd1280710b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:17:20.204 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:17:20.205+0000 7fd144ebc700 1 --2- 172.21.15.162:0/2703576298 >> [v2:172.21.15.162:6800/1396866108,v1:172.21.15.162:6801/1396866108] conn(0x7fd12806ec00 0x7fd1280710b0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fd130004f90 tx=0x7fd130009e30 comp rx=0 tx=0).stop 2024-04-27T04:17:20.204 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:17:20.205+0000 7fd144ebc700 1 -- 172.21.15.162:0/2703576298 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd1400fe760 msgr2=0x7fd140113eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:17:20.204 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:17:20.205+0000 7fd144ebc700 1 --2- 172.21.15.162:0/2703576298 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd1400fe760 0x7fd140113eb0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fd134002a50 tx=0x7fd1340046b0 comp rx=0 tx=0).stop 2024-04-27T04:17:20.204 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:17:20.205+0000 7fd144ebc700 1 -- 172.21.15.162:0/2703576298 shutdown_connections 2024-04-27T04:17:20.204 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:17:20.205+0000 7fd144ebc700 1 --2- 172.21.15.162:0/2703576298 >> [v2:172.21.15.162:6800/1396866108,v1:172.21.15.162:6801/1396866108] conn(0x7fd12806ec00 0x7fd1280710b0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:17:20.204 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:17:20.205+0000 7fd144ebc700 1 --2- 172.21.15.162:0/2703576298 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd1400fddc0 0x7fd140113970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:17:20.205 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:17:20.205+0000 7fd144ebc700 1 --2- 172.21.15.162:0/2703576298 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd1400fe760 0x7fd140113eb0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:17:20.205 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:17:20.205+0000 7fd144ebc700 1 -- 172.21.15.162:0/2703576298 >> 172.21.15.162:0/2703576298 conn(0x7fd1400f7860 msgr2=0x7fd1400f8d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:17:20.205 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:17:20.205+0000 7fd144ebc700 1 -- 172.21.15.162:0/2703576298 shutdown_connections 2024-04-27T04:17:20.205 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:17:20.205+0000 7fd144ebc700 1 -- 172.21.15.162:0/2703576298 wait complete. 2024-04-27T04:17:20.305 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:17:20.306+0000 7f4b922e8700 1 -- 172.21.15.162:0/1760233958 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4b8c0fe790 msgr2=0x7f4b8c106e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:17:20.305 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:17:20.306+0000 7f4b922e8700 1 --2- 172.21.15.162:0/1760233958 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4b8c0fe790 0x7f4b8c106e60 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f4b80004940 tx=0x7f4b80009e30 comp rx=0 tx=0).stop 2024-04-27T04:17:20.305 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:17:20.306+0000 7f4b922e8700 1 -- 172.21.15.162:0/1760233958 shutdown_connections 2024-04-27T04:17:20.306 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:17:20.306+0000 7f4b922e8700 1 --2- 172.21.15.162:0/1760233958 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4b8c0fe790 0x7f4b8c106e60 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:17:20.306 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:17:20.306+0000 7f4b922e8700 1 --2- 172.21.15.162:0/1760233958 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4b8c0fddf0 0x7f4b8c0fe1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:17:20.306 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:17:20.306+0000 7f4b922e8700 1 -- 172.21.15.162:0/1760233958 >> 172.21.15.162:0/1760233958 conn(0x7f4b8c0f7870 msgr2=0x7f4b8c0f9c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:17:20.306 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:17:20.306+0000 7f4b922e8700 1 -- 172.21.15.162:0/1760233958 shutdown_connections 2024-04-27T04:17:20.306 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:17:20.306+0000 7f4b922e8700 1 -- 172.21.15.162:0/1760233958 wait complete. 2024-04-27T04:17:20.308 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:17:20.307+0000 7f4b922e8700 1 Processor -- start 2024-04-27T04:17:20.308 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:17:20.307+0000 7f4b922e8700 1 -- start start 2024-04-27T04:17:20.308 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:17:20.307+0000 7f4b922e8700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4b8c0fddf0 0x7f4b8c113970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:17:20.308 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:17:20.307+0000 7f4b922e8700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4b8c0fe790 0x7f4b8c113eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:17:20.308 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:17:20.307+0000 7f4b922e8700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b8c114500 con 0x7f4b8c0fddf0 2024-04-27T04:17:20.308 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:17:20.307+0000 7f4b922e8700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b8c10db20 con 0x7f4b8c0fe790 2024-04-27T04:17:20.308 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:17:20.307+0000 7f4b8bfff700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4b8c0fddf0 0x7f4b8c113970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:17:20.308 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:17:20.307+0000 7f4b8bfff700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4b8c0fddf0 0x7f4b8c113970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:42590/0 (socket says 172.21.15.162:42590) 2024-04-27T04:17:20.308 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:17:20.307+0000 7f4b8bfff700 1 -- 172.21.15.162:0/2453089240 learned_addr learned my addr 172.21.15.162:0/2453089240 (peer_addr_for_me v2:172.21.15.162:0/0) 2024-04-27T04:17:20.308 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:17:20.308+0000 7f4b8b7fe700 1 --2- 172.21.15.162:0/2453089240 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4b8c0fe790 0x7f4b8c113eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:17:20.308 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:17:20.308+0000 7f4b8bfff700 1 -- 172.21.15.162:0/2453089240 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4b8c0fe790 msgr2=0x7f4b8c113eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:17:20.309 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:17:20.308+0000 7f4b8bfff700 1 --2- 172.21.15.162:0/2453089240 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4b8c0fe790 0x7f4b8c113eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:17:20.309 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:17:20.308+0000 7f4b8bfff700 1 -- 172.21.15.162:0/2453089240 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b80005040 con 0x7f4b8c0fddf0 2024-04-27T04:17:20.309 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:17:20.308+0000 7f4b8bfff700 1 --2- 172.21.15.162:0/2453089240 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4b8c0fddf0 0x7f4b8c113970 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f4b7c00c8c0 tx=0x7f4b7c0044c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:17:20.309 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:17:20.308+0000 7f4b88ff9700 1 -- 172.21.15.162:0/2453089240 <== mon.0 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4b7c00a980 con 0x7f4b8c0fddf0 2024-04-27T04:17:20.309 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:17:20.308+0000 7f4b88ff9700 1 -- 172.21.15.162:0/2453089240 <== mon.0 v2:172.21.15.162:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f4b7c00ce60 con 0x7f4b8c0fddf0 2024-04-27T04:17:20.309 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:17:20.308+0000 7f4b88ff9700 1 -- 172.21.15.162:0/2453089240 <== mon.0 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4b7c0115e0 con 0x7f4b8c0fddf0 2024-04-27T04:17:20.309 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:17:20.308+0000 7f4b922e8700 1 -- 172.21.15.162:0/2453089240 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4b8c10de00 con 0x7f4b8c0fddf0 2024-04-27T04:17:20.309 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:17:20.308+0000 7f4b922e8700 1 -- 172.21.15.162:0/2453089240 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4b8c10e2f0 con 0x7f4b8c0fddf0 2024-04-27T04:17:20.309 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:17:20.309+0000 7f4b922e8700 1 -- 172.21.15.162:0/2453089240 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4b8c066ab0 con 0x7f4b8c0fddf0 2024-04-27T04:17:20.310 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:17:20.310+0000 7f4b88ff9700 1 -- 172.21.15.162:0/2453089240 <== mon.0 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 39) v1 ==== 92747+0+0 (secure 0 0 0) 0x7f4b7c012400 con 0x7f4b8c0fddf0 2024-04-27T04:17:20.310 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:17:20.310+0000 7f4b88ff9700 1 --2- 172.21.15.162:0/2453089240 >> [v2:172.21.15.162:6800/1396866108,v1:172.21.15.162:6801/1396866108] conn(0x7f4b7406ec00 0x7f4b740710b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:17:20.310 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:17:20.310+0000 7f4b88ff9700 1 -- 172.21.15.162:0/2453089240 <== mon.0 v2:172.21.15.162:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 6410+0+0 (secure 0 0 0) 0x7f4b7c08e930 con 0x7f4b8c0fddf0 2024-04-27T04:17:20.310 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:17:20.310+0000 7f4b8b7fe700 1 --2- 172.21.15.162:0/2453089240 >> [v2:172.21.15.162:6800/1396866108,v1:172.21.15.162:6801/1396866108] conn(0x7f4b7406ec00 0x7f4b740710b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:17:20.310 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:17:20.310+0000 7f4b8b7fe700 1 --2- 172.21.15.162:0/2453089240 >> [v2:172.21.15.162:6800/1396866108,v1:172.21.15.162:6801/1396866108] conn(0x7f4b7406ec00 0x7f4b740710b0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f4b8c10f210 tx=0x7f4b80009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:17:20.313 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:17:20.315+0000 7f4b88ff9700 1 -- 172.21.15.162:0/2453089240 <== mon.0 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7f4b7c0577b0 con 0x7f4b8c0fddf0 2024-04-27T04:17:20.560 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:17:20.560+0000 7f4b922e8700 1 -- 172.21.15.162:0/2453089240 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f4b8c04c710 con 0x7f4b8c0fddf0 2024-04-27T04:17:20.560 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:17:20.561+0000 7f4b88ff9700 1 -- 172.21.15.162:0/2453089240 <== mon.0 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 15 v15) v1 ==== 76+0+1911 (secure 0 0 0) 0x7f4b7c0126b0 con 0x7f4b8c0fddf0 2024-04-27T04:17:20.560 INFO:teuthology.orchestra.run.smithi162.stdout:e15 2024-04-27T04:17:20.560 INFO:teuthology.orchestra.run.smithi162.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-04-27T04:17:20.560 INFO:teuthology.orchestra.run.smithi162.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-27T04:17:20.560 INFO:teuthology.orchestra.run.smithi162.stdout:legacy client fscid: 1 2024-04-27T04:17:20.560 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-04-27T04:17:20.561 INFO:teuthology.orchestra.run.smithi162.stdout:Filesystem 'cephfs' (1) 2024-04-27T04:17:20.561 INFO:teuthology.orchestra.run.smithi162.stdout:fs_name cephfs 2024-04-27T04:17:20.561 INFO:teuthology.orchestra.run.smithi162.stdout:epoch 15 2024-04-27T04:17:20.561 INFO:teuthology.orchestra.run.smithi162.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-04-27T04:17:20.561 INFO:teuthology.orchestra.run.smithi162.stdout:created 2024-04-27T04:06:53.095749+0000 2024-04-27T04:17:20.561 INFO:teuthology.orchestra.run.smithi162.stdout:modified 2024-04-27T04:07:11.285033+0000 2024-04-27T04:17:20.561 INFO:teuthology.orchestra.run.smithi162.stdout:tableserver 0 2024-04-27T04:17:20.561 INFO:teuthology.orchestra.run.smithi162.stdout:root 0 2024-04-27T04:17:20.561 INFO:teuthology.orchestra.run.smithi162.stdout:session_timeout 60 2024-04-27T04:17:20.561 INFO:teuthology.orchestra.run.smithi162.stdout:session_autoclose 300 2024-04-27T04:17:20.561 INFO:teuthology.orchestra.run.smithi162.stdout:max_file_size 1099511627776 2024-04-27T04:17:20.561 INFO:teuthology.orchestra.run.smithi162.stdout:max_xattr_size 65536 2024-04-27T04:17:20.561 INFO:teuthology.orchestra.run.smithi162.stdout:required_client_features {} 2024-04-27T04:17:20.561 INFO:teuthology.orchestra.run.smithi162.stdout:last_failure 0 2024-04-27T04:17:20.562 INFO:teuthology.orchestra.run.smithi162.stdout:last_failure_osd_epoch 46 2024-04-27T04:17:20.562 INFO:teuthology.orchestra.run.smithi162.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-27T04:17:20.562 INFO:teuthology.orchestra.run.smithi162.stdout:max_mds 1 2024-04-27T04:17:20.562 INFO:teuthology.orchestra.run.smithi162.stdout:in 0 2024-04-27T04:17:20.562 INFO:teuthology.orchestra.run.smithi162.stdout:up {0=24273} 2024-04-27T04:17:20.562 INFO:teuthology.orchestra.run.smithi162.stdout:failed 2024-04-27T04:17:20.562 INFO:teuthology.orchestra.run.smithi162.stdout:damaged 2024-04-27T04:17:20.562 INFO:teuthology.orchestra.run.smithi162.stdout:stopped 2024-04-27T04:17:20.562 INFO:teuthology.orchestra.run.smithi162.stdout:data_pools [3] 2024-04-27T04:17:20.562 INFO:teuthology.orchestra.run.smithi162.stdout:metadata_pool 2 2024-04-27T04:17:20.562 INFO:teuthology.orchestra.run.smithi162.stdout:inline_data disabled 2024-04-27T04:17:20.562 INFO:teuthology.orchestra.run.smithi162.stdout:balancer 2024-04-27T04:17:20.562 INFO:teuthology.orchestra.run.smithi162.stdout:bal_rank_mask -1 2024-04-27T04:17:20.562 INFO:teuthology.orchestra.run.smithi162.stdout:standby_count_wanted 1 2024-04-27T04:17:20.563 INFO:teuthology.orchestra.run.smithi162.stdout:qdb_cluster leader: 0 members: 2024-04-27T04:17:20.563 INFO:teuthology.orchestra.run.smithi162.stdout:[mds.cephfs.smithi181.dzomgh{0:24273} state up:active seq 6 join_fscid=1 addr [v2:172.21.15.181:6824/658214993,v1:172.21.15.181:6825/658214993] compat {c=[1],r=[1],i=[7ff]}] 2024-04-27T04:17:20.563 INFO:teuthology.orchestra.run.smithi162.stdout:[mds.cephfs.smithi162.xcnapk{0:14476} state up:standby-replay seq 2 join_fscid=1 addr [v2:172.21.15.162:6828/1296160676,v1:172.21.15.162:6829/1296160676] compat {c=[1],r=[1],i=[7ff]}] 2024-04-27T04:17:20.563 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-04-27T04:17:20.563 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-04-27T04:17:20.563 INFO:teuthology.orchestra.run.smithi162.stdout:Standby daemons: 2024-04-27T04:17:20.563 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-04-27T04:17:20.563 INFO:teuthology.orchestra.run.smithi162.stdout:[mds.cephfs.smithi162.gzlyjj{-1:14488} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.162:6826/2629103894,v1:172.21.15.162:6827/2629103894] compat {c=[1],r=[1],i=[7ff]}] 2024-04-27T04:17:20.563 INFO:teuthology.orchestra.run.smithi162.stdout:[mds.cephfs.smithi181.bkkcdm{-1:24287} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.181:6826/3756475453,v1:172.21.15.181:6827/3756475453] compat {c=[1],r=[1],i=[7ff]}] 2024-04-27T04:17:20.564 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:17:20.563+0000 7f4b922e8700 1 -- 172.21.15.162:0/2453089240 >> [v2:172.21.15.162:6800/1396866108,v1:172.21.15.162:6801/1396866108] conn(0x7f4b7406ec00 msgr2=0x7f4b740710b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:17:20.564 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:17:20.563+0000 7f4b922e8700 1 --2- 172.21.15.162:0/2453089240 >> [v2:172.21.15.162:6800/1396866108,v1:172.21.15.162:6801/1396866108] conn(0x7f4b7406ec00 0x7f4b740710b0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f4b8c10f210 tx=0x7f4b80009e30 comp rx=0 tx=0).stop 2024-04-27T04:17:20.564 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:17:20.563+0000 7f4b922e8700 1 -- 172.21.15.162:0/2453089240 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4b8c0fddf0 msgr2=0x7f4b8c113970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:17:20.564 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:17:20.563+0000 7f4b922e8700 1 --2- 172.21.15.162:0/2453089240 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4b8c0fddf0 0x7f4b8c113970 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f4b7c00c8c0 tx=0x7f4b7c0044c0 comp rx=0 tx=0).stop 2024-04-27T04:17:20.564 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:17:20.563+0000 7f4b922e8700 1 -- 172.21.15.162:0/2453089240 shutdown_connections 2024-04-27T04:17:20.564 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:17:20.563+0000 7f4b922e8700 1 --2- 172.21.15.162:0/2453089240 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4b8c0fddf0 0x7f4b8c113970 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:17:20.564 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:17:20.563+0000 7f4b922e8700 1 --2- 172.21.15.162:0/2453089240 >> [v2:172.21.15.162:6800/1396866108,v1:172.21.15.162:6801/1396866108] conn(0x7f4b7406ec00 0x7f4b740710b0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:17:20.564 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:17:20.563+0000 7f4b922e8700 1 --2- 172.21.15.162:0/2453089240 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4b8c0fe790 0x7f4b8c113eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:17:20.564 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:17:20.563+0000 7f4b922e8700 1 -- 172.21.15.162:0/2453089240 >> 172.21.15.162:0/2453089240 conn(0x7f4b8c0f7870 msgr2=0x7f4b8c0f8d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:17:20.565 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:17:20.563+0000 7f4b922e8700 1 -- 172.21.15.162:0/2453089240 shutdown_connections 2024-04-27T04:17:20.565 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:17:20.563+0000 7f4b922e8700 1 -- 172.21.15.162:0/2453089240 wait complete. 2024-04-27T04:17:20.565 INFO:teuthology.orchestra.run.smithi162.stderr:dumped fsmap epoch 15 2024-04-27T04:17:20.668 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:17:20.667+0000 7f7ba13ed700 1 -- 172.21.15.162:0/3027517801 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7b9c1026a0 msgr2=0x7f7b9c0fbd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:17:20.668 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:17:20.667+0000 7f7ba13ed700 1 --2- 172.21.15.162:0/3027517801 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7b9c1026a0 0x7f7b9c0fbd10 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f7b90004940 tx=0x7f7b90009e30 comp rx=0 tx=0).stop 2024-04-27T04:17:20.668 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:17:20.667+0000 7f7ba13ed700 1 -- 172.21.15.162:0/3027517801 shutdown_connections 2024-04-27T04:17:20.668 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:17:20.667+0000 7f7ba13ed700 1 --2- 172.21.15.162:0/3027517801 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7b9c1026a0 0x7f7b9c0fbd10 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:17:20.668 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:17:20.667+0000 7f7ba13ed700 1 --2- 172.21.15.162:0/3027517801 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7b9c101d00 0x7f7b9c1020d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:17:20.668 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:17:20.667+0000 7f7ba13ed700 1 -- 172.21.15.162:0/3027517801 >> 172.21.15.162:0/3027517801 conn(0x7f7b9c0f7870 msgr2=0x7f7b9c0f9c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:17:20.668 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:17:20.667+0000 7f7ba13ed700 1 -- 172.21.15.162:0/3027517801 shutdown_connections 2024-04-27T04:17:20.668 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:17:20.667+0000 7f7ba13ed700 1 -- 172.21.15.162:0/3027517801 wait complete. 2024-04-27T04:17:20.668 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:17:20.668+0000 7f7ba13ed700 1 Processor -- start 2024-04-27T04:17:20.669 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:17:20.668+0000 7f7ba13ed700 1 -- start start 2024-04-27T04:17:20.669 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:17:20.668+0000 7f7ba13ed700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7b9c101d00 0x7f7b9c112880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:17:20.669 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:17:20.668+0000 7f7ba13ed700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7b9c1026a0 0x7f7b9c112dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:17:20.669 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:17:20.668+0000 7f7ba13ed700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b9c113410 con 0x7f7b9c1026a0 2024-04-27T04:17:20.669 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:17:20.668+0000 7f7ba13ed700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b9c10ca10 con 0x7f7b9c101d00 2024-04-27T04:17:20.669 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:17:20.668+0000 7f7b9a7fc700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7b9c1026a0 0x7f7b9c112dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:17:20.669 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:17:20.668+0000 7f7b9a7fc700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7b9c1026a0 0x7f7b9c112dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:42616/0 (socket says 172.21.15.162:42616) 2024-04-27T04:17:20.669 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:17:20.668+0000 7f7b9a7fc700 1 -- 172.21.15.162:0/304349235 learned_addr learned my addr 172.21.15.162:0/304349235 (peer_addr_for_me v2:172.21.15.162:0/0) 2024-04-27T04:17:20.669 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:17:20.668+0000 7f7b9affd700 1 --2- 172.21.15.162:0/304349235 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7b9c101d00 0x7f7b9c112880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:17:20.669 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:17:20.668+0000 7f7b9a7fc700 1 -- 172.21.15.162:0/304349235 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7b9c101d00 msgr2=0x7f7b9c112880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:17:20.669 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:17:20.668+0000 7f7b9a7fc700 1 --2- 172.21.15.162:0/304349235 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7b9c101d00 0x7f7b9c112880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:17:20.669 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:17:20.668+0000 7f7b9a7fc700 1 -- 172.21.15.162:0/304349235 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b90005040 con 0x7f7b9c1026a0 2024-04-27T04:17:20.669 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:17:20.668+0000 7f7b9a7fc700 1 --2- 172.21.15.162:0/304349235 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7b9c1026a0 0x7f7b9c112dc0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f7b90000c00 tx=0x7f7b90009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:17:20.670 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:17:20.669+0000 7f7b837fe700 1 -- 172.21.15.162:0/304349235 <== mon.0 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7b9000e040 con 0x7f7b9c1026a0 2024-04-27T04:17:20.670 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:17:20.669+0000 7f7b837fe700 1 -- 172.21.15.162:0/304349235 <== mon.0 v2:172.21.15.162:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f7b9000bd00 con 0x7f7b9c1026a0 2024-04-27T04:17:20.670 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:17:20.669+0000 7f7ba13ed700 1 -- 172.21.15.162:0/304349235 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7b9c10cc90 con 0x7f7b9c1026a0 2024-04-27T04:17:20.670 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:17:20.669+0000 7f7b837fe700 1 -- 172.21.15.162:0/304349235 <== mon.0 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7b90013660 con 0x7f7b9c1026a0 2024-04-27T04:17:20.670 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:17:20.669+0000 7f7ba13ed700 1 -- 172.21.15.162:0/304349235 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7b9c10d250 con 0x7f7b9c1026a0 2024-04-27T04:17:20.671 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:17:20.670+0000 7f7ba13ed700 1 -- 172.21.15.162:0/304349235 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7b9c0fd450 con 0x7f7b9c1026a0 2024-04-27T04:17:20.671 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:17:20.670+0000 7f7b837fe700 1 -- 172.21.15.162:0/304349235 <== mon.0 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 39) v1 ==== 92747+0+0 (secure 0 0 0) 0x7f7b9001e9f0 con 0x7f7b9c1026a0 2024-04-27T04:17:20.671 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:17:20.671+0000 7f7b837fe700 1 --2- 172.21.15.162:0/304349235 >> [v2:172.21.15.162:6800/1396866108,v1:172.21.15.162:6801/1396866108] conn(0x7f7b8406ec00 0x7f7b840710b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:17:20.671 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:17:20.671+0000 7f7b837fe700 1 -- 172.21.15.162:0/304349235 <== mon.0 v2:172.21.15.162:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 6410+0+0 (secure 0 0 0) 0x7f7b90092310 con 0x7f7b9c1026a0 2024-04-27T04:17:20.671 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:17:20.671+0000 7f7b9affd700 1 --2- 172.21.15.162:0/304349235 >> [v2:172.21.15.162:6800/1396866108,v1:172.21.15.162:6801/1396866108] conn(0x7f7b8406ec00 0x7f7b840710b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:17:20.671 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:17:20.671+0000 7f7b9affd700 1 --2- 172.21.15.162:0/304349235 >> [v2:172.21.15.162:6800/1396866108,v1:172.21.15.162:6801/1396866108] conn(0x7f7b8406ec00 0x7f7b840710b0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f7b8c0041d0 tx=0x7f7b8c0043c0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:17:20.674 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:17:20.676+0000 7f7b837fe700 1 -- 172.21.15.162:0/304349235 <== mon.0 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7f7b9005ec40 con 0x7f7b9c1026a0 2024-04-27T04:17:20.694 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:17:20 smithi181 ceph-mon[81134]: from='client.34284 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-27T04:17:20.694 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:17:20 smithi181 ceph-mon[81134]: from='client.34288 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-27T04:17:20.694 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:17:20 smithi181 ceph-mon[81134]: Upgrade: Updating osd.3 2024-04-27T04:17:20.694 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:17:20 smithi181 ceph-mon[81134]: Deploying daemon osd.3 on smithi181 2024-04-27T04:17:20.694 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:17:20 smithi181 ceph-mon[81134]: pgmap v109: 65 pgs: 65 active+clean; 208 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:17:20.694 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:17:20 smithi181 ceph-mon[81134]: from='client.? 172.21.15.162:0/2703576298' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-27T04:17:20.694 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:17:20 smithi181 ceph-mon[81134]: from='client.? 172.21.15.162:0/2453089240' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-27T04:17:20.886 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:17:20 smithi162 ceph-mon[110641]: from='client.34284 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-27T04:17:20.886 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:17:20 smithi162 ceph-mon[110641]: from='client.34288 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-27T04:17:20.886 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:17:20 smithi162 ceph-mon[110641]: Upgrade: Updating osd.3 2024-04-27T04:17:20.886 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:17:20 smithi162 ceph-mon[110641]: Deploying daemon osd.3 on smithi181 2024-04-27T04:17:20.887 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:17:20 smithi162 ceph-mon[110641]: pgmap v109: 65 pgs: 65 active+clean; 208 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:17:20.887 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:17:20 smithi162 ceph-mon[110641]: from='client.? 172.21.15.162:0/2703576298' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-27T04:17:20.887 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:17:20 smithi162 ceph-mon[110641]: from='client.? 172.21.15.162:0/2453089240' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-27T04:17:20.894 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:17:20.895+0000 7f7ba13ed700 1 -- 172.21.15.162:0/304349235 --> [v2:172.21.15.162:6800/1396866108,v1:172.21.15.162:6801/1396866108] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f7b9c108de0 con 0x7f7b8406ec00 2024-04-27T04:17:20.895 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:17:20.896+0000 7f7b837fe700 1 -- 172.21.15.162:0/304349235 <== mgr.34104 v2:172.21.15.162:6800/1396866108 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+400 (secure 0 0 0) 0x7f7b9c108de0 con 0x7f7b8406ec00 2024-04-27T04:17:20.895 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-04-27T04:17:20.895 INFO:teuthology.orchestra.run.smithi162.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:90a3a9fa9ceb8977f6dad9ccd36fe46018d07e2118199820523a6b35301d9c16", 2024-04-27T04:17:20.895 INFO:teuthology.orchestra.run.smithi162.stdout: "in_progress": true, 2024-04-27T04:17:20.895 INFO:teuthology.orchestra.run.smithi162.stdout: "which": "Upgrading all daemon types on all hosts", 2024-04-27T04:17:20.896 INFO:teuthology.orchestra.run.smithi162.stdout: "services_complete": [ 2024-04-27T04:17:20.896 INFO:teuthology.orchestra.run.smithi162.stdout: "crash", 2024-04-27T04:17:20.896 INFO:teuthology.orchestra.run.smithi162.stdout: "mon", 2024-04-27T04:17:20.896 INFO:teuthology.orchestra.run.smithi162.stdout: "mgr" 2024-04-27T04:17:20.896 INFO:teuthology.orchestra.run.smithi162.stdout: ], 2024-04-27T04:17:20.896 INFO:teuthology.orchestra.run.smithi162.stdout: "progress": "9/21 daemons upgraded", 2024-04-27T04:17:20.896 INFO:teuthology.orchestra.run.smithi162.stdout: "message": "Currently upgrading osd daemons", 2024-04-27T04:17:20.896 INFO:teuthology.orchestra.run.smithi162.stdout: "is_paused": false 2024-04-27T04:17:20.896 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-04-27T04:17:20.897 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:17:20.898+0000 7f7ba13ed700 1 -- 172.21.15.162:0/304349235 >> [v2:172.21.15.162:6800/1396866108,v1:172.21.15.162:6801/1396866108] conn(0x7f7b8406ec00 msgr2=0x7f7b840710b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:17:20.898 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:17:20.898+0000 7f7ba13ed700 1 --2- 172.21.15.162:0/304349235 >> [v2:172.21.15.162:6800/1396866108,v1:172.21.15.162:6801/1396866108] conn(0x7f7b8406ec00 0x7f7b840710b0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f7b8c0041d0 tx=0x7f7b8c0043c0 comp rx=0 tx=0).stop 2024-04-27T04:17:20.898 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:17:20.898+0000 7f7ba13ed700 1 -- 172.21.15.162:0/304349235 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7b9c1026a0 msgr2=0x7f7b9c112dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:17:20.898 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:17:20.898+0000 7f7ba13ed700 1 --2- 172.21.15.162:0/304349235 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7b9c1026a0 0x7f7b9c112dc0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f7b90000c00 tx=0x7f7b90009e30 comp rx=0 tx=0).stop 2024-04-27T04:17:20.898 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:17:20.898+0000 7f7ba13ed700 1 -- 172.21.15.162:0/304349235 shutdown_connections 2024-04-27T04:17:20.898 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:17:20.898+0000 7f7ba13ed700 1 --2- 172.21.15.162:0/304349235 >> [v2:172.21.15.162:6800/1396866108,v1:172.21.15.162:6801/1396866108] conn(0x7f7b8406ec00 0x7f7b840710b0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:17:20.898 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:17:20.898+0000 7f7ba13ed700 1 --2- 172.21.15.162:0/304349235 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7b9c101d00 0x7f7b9c112880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:17:20.898 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:17:20.898+0000 7f7ba13ed700 1 --2- 172.21.15.162:0/304349235 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7b9c1026a0 0x7f7b9c112dc0 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:17:20.898 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:17:20.898+0000 7f7ba13ed700 1 -- 172.21.15.162:0/304349235 >> 172.21.15.162:0/304349235 conn(0x7f7b9c0f7870 msgr2=0x7f7b9c0f8810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:17:20.898 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:17:20.898+0000 7f7ba13ed700 1 -- 172.21.15.162:0/304349235 shutdown_connections 2024-04-27T04:17:20.898 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:17:20.899+0000 7f7ba13ed700 1 -- 172.21.15.162:0/304349235 wait complete. 2024-04-27T04:17:21.000 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:17:21.000+0000 7fe895d40700 1 -- 172.21.15.162:0/2209857823 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe8900fc6b0 msgr2=0x7fe8900fcb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:17:21.000 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:17:21.000+0000 7fe895d40700 1 --2- 172.21.15.162:0/2209857823 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe8900fc6b0 0x7fe8900fcb20 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fe878004940 tx=0x7fe878009e30 comp rx=0 tx=0).stop 2024-04-27T04:17:21.001 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:17:21.000+0000 7fe895d40700 1 -- 172.21.15.162:0/2209857823 shutdown_connections 2024-04-27T04:17:21.001 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:17:21.000+0000 7fe895d40700 1 --2- 172.21.15.162:0/2209857823 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe8900fc6b0 0x7fe8900fcb20 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:17:21.001 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:17:21.000+0000 7fe895d40700 1 --2- 172.21.15.162:0/2209857823 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe890101d90 0x7fe890102160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:17:21.001 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:17:21.000+0000 7fe895d40700 1 -- 172.21.15.162:0/2209857823 >> 172.21.15.162:0/2209857823 conn(0x7fe890071460 msgr2=0x7fe890071860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:17:21.001 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:17:21.000+0000 7fe895d40700 1 -- 172.21.15.162:0/2209857823 shutdown_connections 2024-04-27T04:17:21.001 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:17:21.000+0000 7fe895d40700 1 -- 172.21.15.162:0/2209857823 wait complete. 2024-04-27T04:17:21.001 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:17:21.000+0000 7fe895d40700 1 Processor -- start 2024-04-27T04:17:21.001 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:17:21.001+0000 7fe895d40700 1 -- start start 2024-04-27T04:17:21.001 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:17:21.001+0000 7fe895d40700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe8900fc6b0 0x7fe890113910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:17:21.001 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:17:21.001+0000 7fe895d40700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe890101d90 0x7fe890113e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:17:21.001 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:17:21.001+0000 7fe895d40700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8901144a0 con 0x7fe8900fc6b0 2024-04-27T04:17:21.001 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:17:21.001+0000 7fe895d40700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe89010daa0 con 0x7fe890101d90 2024-04-27T04:17:21.001 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:17:21.001+0000 7fe88f7fe700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe8900fc6b0 0x7fe890113910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:17:21.001 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:17:21.001+0000 7fe88f7fe700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe8900fc6b0 0x7fe890113910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:42642/0 (socket says 172.21.15.162:42642) 2024-04-27T04:17:21.002 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:17:21.001+0000 7fe88f7fe700 1 -- 172.21.15.162:0/1808007434 learned_addr learned my addr 172.21.15.162:0/1808007434 (peer_addr_for_me v2:172.21.15.162:0/0) 2024-04-27T04:17:21.002 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:17:21.001+0000 7fe887fff700 1 --2- 172.21.15.162:0/1808007434 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe890101d90 0x7fe890113e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:17:21.002 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:17:21.001+0000 7fe88f7fe700 1 -- 172.21.15.162:0/1808007434 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe890101d90 msgr2=0x7fe890113e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:17:21.002 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:17:21.001+0000 7fe88f7fe700 1 --2- 172.21.15.162:0/1808007434 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe890101d90 0x7fe890113e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:17:21.002 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:17:21.001+0000 7fe88f7fe700 1 -- 172.21.15.162:0/1808007434 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe878005040 con 0x7fe8900fc6b0 2024-04-27T04:17:21.002 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:17:21.001+0000 7fe88f7fe700 1 --2- 172.21.15.162:0/1808007434 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe8900fc6b0 0x7fe890113910 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fe88000eb40 tx=0x7fe88000b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:17:21.002 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:17:21.001+0000 7fe88cff9700 1 -- 172.21.15.162:0/1808007434 <== mon.0 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe880007cc0 con 0x7fe8900fc6b0 2024-04-27T04:17:21.002 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:17:21.001+0000 7fe88cff9700 1 -- 172.21.15.162:0/1808007434 <== mon.0 v2:172.21.15.162:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fe88000f040 con 0x7fe8900fc6b0 2024-04-27T04:17:21.002 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:17:21.001+0000 7fe895d40700 1 -- 172.21.15.162:0/1808007434 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe89010dd20 con 0x7fe8900fc6b0 2024-04-27T04:17:21.002 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:17:21.001+0000 7fe88cff9700 1 -- 172.21.15.162:0/1808007434 <== mon.0 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe880009620 con 0x7fe8900fc6b0 2024-04-27T04:17:21.002 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:17:21.001+0000 7fe895d40700 1 -- 172.21.15.162:0/1808007434 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe89010e260 con 0x7fe8900fc6b0 2024-04-27T04:17:21.003 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:17:21.002+0000 7fe895d40700 1 -- 172.21.15.162:0/1808007434 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe890066010 con 0x7fe8900fc6b0 2024-04-27T04:17:21.003 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:17:21.003+0000 7fe88cff9700 1 -- 172.21.15.162:0/1808007434 <== mon.0 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 39) v1 ==== 92747+0+0 (secure 0 0 0) 0x7fe880009780 con 0x7fe8900fc6b0 2024-04-27T04:17:21.003 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:17:21.003+0000 7fe88cff9700 1 --2- 172.21.15.162:0/1808007434 >> [v2:172.21.15.162:6800/1396866108,v1:172.21.15.162:6801/1396866108] conn(0x7fe87c06ec00 0x7fe87c0710b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:17:21.003 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:17:21.003+0000 7fe88cff9700 1 -- 172.21.15.162:0/1808007434 <== mon.0 v2:172.21.15.162:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 6410+0+0 (secure 0 0 0) 0x7fe88008e4f0 con 0x7fe8900fc6b0 2024-04-27T04:17:21.003 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:17:21.003+0000 7fe887fff700 1 --2- 172.21.15.162:0/1808007434 >> [v2:172.21.15.162:6800/1396866108,v1:172.21.15.162:6801/1396866108] conn(0x7fe87c06ec00 0x7fe87c0710b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:17:21.003 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:17:21.004+0000 7fe887fff700 1 --2- 172.21.15.162:0/1808007434 >> [v2:172.21.15.162:6800/1396866108,v1:172.21.15.162:6801/1396866108] conn(0x7fe87c06ec00 0x7fe87c0710b0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fe87800b0d0 tx=0x7fe878009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:17:21.007 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:17:21.008+0000 7fe88cff9700 1 -- 172.21.15.162:0/1808007434 <== mon.0 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7fe88005aed0 con 0x7fe8900fc6b0 2024-04-27T04:17:21.291 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:17:21.292+0000 7fe895d40700 1 -- 172.21.15.162:0/1808007434 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7fe890108e70 con 0x7fe8900fc6b0 2024-04-27T04:17:21.291 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:17:21.292+0000 7fe88cff9700 1 -- 172.21.15.162:0/1808007434 <== mon.0 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+10 (secure 0 0 0) 0x7fe88005aa60 con 0x7fe8900fc6b0 2024-04-27T04:17:21.291 INFO:teuthology.orchestra.run.smithi162.stdout:HEALTH_OK 2024-04-27T04:17:21.293 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:17:21.294+0000 7fe895d40700 1 -- 172.21.15.162:0/1808007434 >> [v2:172.21.15.162:6800/1396866108,v1:172.21.15.162:6801/1396866108] conn(0x7fe87c06ec00 msgr2=0x7fe87c0710b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:17:21.294 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:17:21.294+0000 7fe895d40700 1 --2- 172.21.15.162:0/1808007434 >> [v2:172.21.15.162:6800/1396866108,v1:172.21.15.162:6801/1396866108] conn(0x7fe87c06ec00 0x7fe87c0710b0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fe87800b0d0 tx=0x7fe878009e30 comp rx=0 tx=0).stop 2024-04-27T04:17:21.294 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:17:21.294+0000 7fe895d40700 1 -- 172.21.15.162:0/1808007434 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe8900fc6b0 msgr2=0x7fe890113910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:17:21.294 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:17:21.294+0000 7fe895d40700 1 --2- 172.21.15.162:0/1808007434 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe8900fc6b0 0x7fe890113910 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fe88000eb40 tx=0x7fe88000b040 comp rx=0 tx=0).stop 2024-04-27T04:17:21.294 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:17:21.294+0000 7fe895d40700 1 -- 172.21.15.162:0/1808007434 shutdown_connections 2024-04-27T04:17:21.294 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:17:21.294+0000 7fe895d40700 1 --2- 172.21.15.162:0/1808007434 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe8900fc6b0 0x7fe890113910 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:17:21.294 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:17:21.294+0000 7fe895d40700 1 --2- 172.21.15.162:0/1808007434 >> [v2:172.21.15.162:6800/1396866108,v1:172.21.15.162:6801/1396866108] conn(0x7fe87c06ec00 0x7fe87c0710b0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:17:21.294 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:17:21.294+0000 7fe895d40700 1 --2- 172.21.15.162:0/1808007434 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe890101d90 0x7fe890113e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:17:21.294 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:17:21.294+0000 7fe895d40700 1 -- 172.21.15.162:0/1808007434 >> 172.21.15.162:0/1808007434 conn(0x7fe890071460 msgr2=0x7fe8900fb8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:17:21.294 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:17:21.295+0000 7fe895d40700 1 -- 172.21.15.162:0/1808007434 shutdown_connections 2024-04-27T04:17:21.294 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:17:21.295+0000 7fe895d40700 1 -- 172.21.15.162:0/1808007434 wait complete. 2024-04-27T04:17:21.921 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:17:21 smithi181 ceph-mon[81134]: from='client.34292 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-27T04:17:21.922 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:17:21 smithi181 ceph-mon[81134]: from='client.? 172.21.15.162:0/1808007434' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-27T04:17:22.138 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:17:21 smithi162 ceph-mon[110641]: from='client.34292 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-27T04:17:22.138 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:17:21 smithi162 ceph-mon[110641]: from='client.? 172.21.15.162:0/1808007434' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-27T04:17:23.027 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:17:22 smithi181 ceph-mon[81134]: from='client.34304 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-27T04:17:23.027 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:17:22 smithi181 ceph-mon[81134]: pgmap v110: 65 pgs: 65 active+clean; 208 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:17:23.027 INFO:journalctl@ceph.osd.3.smithi181.stdout:Apr 27 04:17:22 smithi181 systemd[1]: Stopping Ceph osd.3 for 7f9750d6-044a-11ef-bc93-c7b262605968... 2024-04-27T04:17:23.136 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:17:22 smithi162 ceph-mon[110641]: from='client.34304 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-27T04:17:23.136 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:17:22 smithi162 ceph-mon[110641]: pgmap v110: 65 pgs: 65 active+clean; 208 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:17:23.422 INFO:journalctl@ceph.osd.3.smithi181.stdout:Apr 27 04:17:23 smithi181 ceph-7f9750d6-044a-11ef-bc93-c7b262605968-osd-3[39355]: 2024-04-27T04:17:23.051+0000 7f9b475f0700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.3 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-27T04:17:23.422 INFO:journalctl@ceph.osd.3.smithi181.stdout:Apr 27 04:17:23 smithi181 ceph-7f9750d6-044a-11ef-bc93-c7b262605968-osd-3[39355]: 2024-04-27T04:17:23.051+0000 7f9b475f0700 -1 osd.3 64 *** Got signal Terminated *** 2024-04-27T04:17:23.422 INFO:journalctl@ceph.osd.3.smithi181.stdout:Apr 27 04:17:23 smithi181 ceph-7f9750d6-044a-11ef-bc93-c7b262605968-osd-3[39355]: 2024-04-27T04:17:23.051+0000 7f9b475f0700 -1 osd.3 64 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-27T04:17:23.963 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:17:23 smithi181 ceph-mon[81134]: osd.3 marked itself down and dead 2024-04-27T04:17:24.136 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:17:23 smithi162 ceph-mon[110641]: osd.3 marked itself down and dead 2024-04-27T04:17:24.400 INFO:journalctl@ceph.osd.3.smithi181.stdout:Apr 27 04:17:24 smithi181 podman[89931]: 2024-04-27 04:17:24.089812343 +0000 UTC m=+1.162529304 container died 030d829be54ee1ee8d909e94894ea657848304429a4b86265684d94bb3165537 (image=quay.ceph.io/ceph-ci/ceph@sha256:d31f278ece6dbd17d1fa5e598f509017f54b76fafe1c4416391fcc51b9718ee9, name=ceph-7f9750d6-044a-11ef-bc93-c7b262605968-osd-3, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_CLEAN=True, RELEASE=quincy-2be3efa, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.build-date=20240424, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b) 2024-04-27T04:17:24.671 INFO:journalctl@ceph.osd.3.smithi181.stdout:Apr 27 04:17:24 smithi181 podman[89931]: 2024-04-27 04:17:24.576944204 +0000 UTC m=+1.649661212 container cleanup 030d829be54ee1ee8d909e94894ea657848304429a4b86265684d94bb3165537 (image=quay.ceph.io/ceph-ci/ceph@sha256:d31f278ece6dbd17d1fa5e598f509017f54b76fafe1c4416391fcc51b9718ee9, name=ceph-7f9750d6-044a-11ef-bc93-c7b262605968-osd-3, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240424, org.label-schema.vendor=CentOS, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=quincy-2be3efa, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_CLEAN=True) 2024-04-27T04:17:24.672 INFO:journalctl@ceph.osd.3.smithi181.stdout:Apr 27 04:17:24 smithi181 bash[89931]: ceph-7f9750d6-044a-11ef-bc93-c7b262605968-osd-3 2024-04-27T04:17:25.136 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:17:24 smithi162 ceph-mon[110641]: pgmap v111: 65 pgs: 65 active+clean; 208 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:17:25.136 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:17:24 smithi162 ceph-mon[110641]: Health check failed: 1 osds down (OSD_DOWN) 2024-04-27T04:17:25.136 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:17:24 smithi162 ceph-mon[110641]: osdmap e65: 6 total, 5 up, 6 in 2024-04-27T04:17:25.171 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:17:24 smithi181 ceph-mon[81134]: pgmap v111: 65 pgs: 65 active+clean; 208 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:17:25.172 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:17:24 smithi181 ceph-mon[81134]: Health check failed: 1 osds down (OSD_DOWN) 2024-04-27T04:17:25.172 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:17:24 smithi181 ceph-mon[81134]: osdmap e65: 6 total, 5 up, 6 in 2024-04-27T04:17:25.172 INFO:journalctl@ceph.osd.3.smithi181.stdout:Apr 27 04:17:24 smithi181 podman[89960]: 2024-04-27 04:17:24.862078155 +0000 UTC m=+0.773063976 container remove 030d829be54ee1ee8d909e94894ea657848304429a4b86265684d94bb3165537 (image=quay.ceph.io/ceph-ci/ceph@sha256:d31f278ece6dbd17d1fa5e598f509017f54b76fafe1c4416391fcc51b9718ee9, name=ceph-7f9750d6-044a-11ef-bc93-c7b262605968-osd-3, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240424, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, ceph=True, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=quincy-2be3efa) 2024-04-27T04:17:25.505 INFO:journalctl@ceph.osd.3.smithi181.stdout:Apr 27 04:17:25 smithi181 podman[90052]: 2024-04-27 04:17:25.090753344 +0000 UTC m=+0.017999660 image pull 67e414ff4bc41092cb31c84c98a9563a32813d02583028679c46b40362afeb3e quay.ceph.io/ceph-ci/ceph@sha256:90a3a9fa9ceb8977f6dad9ccd36fe46018d07e2118199820523a6b35301d9c16 2024-04-27T04:17:25.505 INFO:journalctl@ceph.osd.3.smithi181.stdout:Apr 27 04:17:25 smithi181 podman[90052]: 2024-04-27 04:17:25.229597644 +0000 UTC m=+0.156843959 container create 35463df5a2d3b2675e116ea1f53961cf49f3b7307d4ad412a7ec9c792d344d38 (image=quay.ceph.io/ceph-ci/ceph@sha256:90a3a9fa9ceb8977f6dad9ccd36fe46018d07e2118199820523a6b35301d9c16, name=ceph-7f9750d6-044a-11ef-bc93-c7b262605968-osd-3-deactivate, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_CLEAN=True, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , ceph=True, RELEASE=main-4e213a5, org.label-schema.build-date=20240424, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2) 2024-04-27T04:17:25.505 INFO:journalctl@ceph.osd.3.smithi181.stdout:Apr 27 04:17:25 smithi181 podman[90052]: 2024-04-27 04:17:25.432014983 +0000 UTC m=+0.359261298 container init 35463df5a2d3b2675e116ea1f53961cf49f3b7307d4ad412a7ec9c792d344d38 (image=quay.ceph.io/ceph-ci/ceph@sha256:90a3a9fa9ceb8977f6dad9ccd36fe46018d07e2118199820523a6b35301d9c16, name=ceph-7f9750d6-044a-11ef-bc93-c7b262605968-osd-3-deactivate, io.buildah.version=1.33.5, RELEASE=main-4e213a5, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.build-date=20240424, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-27T04:17:25.505 INFO:journalctl@ceph.osd.3.smithi181.stdout:Apr 27 04:17:25 smithi181 podman[90052]: 2024-04-27 04:17:25.435255369 +0000 UTC m=+0.362501685 container start 35463df5a2d3b2675e116ea1f53961cf49f3b7307d4ad412a7ec9c792d344d38 (image=quay.ceph.io/ceph-ci/ceph@sha256:90a3a9fa9ceb8977f6dad9ccd36fe46018d07e2118199820523a6b35301d9c16, name=ceph-7f9750d6-044a-11ef-bc93-c7b262605968-osd-3-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.build-date=20240424, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, RELEASE=main-4e213a5, GIT_BRANCH=HEAD, ceph=True, org.label-schema.vendor=CentOS) 2024-04-27T04:17:25.505 INFO:journalctl@ceph.osd.3.smithi181.stdout:Apr 27 04:17:25 smithi181 podman[90052]: 2024-04-27 04:17:25.506935918 +0000 UTC m=+0.434182242 container attach 35463df5a2d3b2675e116ea1f53961cf49f3b7307d4ad412a7ec9c792d344d38 (image=quay.ceph.io/ceph-ci/ceph@sha256:90a3a9fa9ceb8977f6dad9ccd36fe46018d07e2118199820523a6b35301d9c16, name=ceph-7f9750d6-044a-11ef-bc93-c7b262605968-osd-3-deactivate, io.buildah.version=1.33.5, RELEASE=main-4e213a5, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.build-date=20240424, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-27T04:17:25.922 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:17:25 smithi181 ceph-mon[81134]: osdmap e66: 6 total, 5 up, 6 in 2024-04-27T04:17:25.922 INFO:journalctl@ceph.osd.3.smithi181.stdout:Apr 27 04:17:25 smithi181 podman[90052]: 2024-04-27 04:17:25.734067668 +0000 UTC m=+0.661313982 container died 35463df5a2d3b2675e116ea1f53961cf49f3b7307d4ad412a7ec9c792d344d38 (image=quay.ceph.io/ceph-ci/ceph@sha256:90a3a9fa9ceb8977f6dad9ccd36fe46018d07e2118199820523a6b35301d9c16, name=ceph-7f9750d6-044a-11ef-bc93-c7b262605968-osd-3-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, ceph=True, io.buildah.version=1.33.5, org.label-schema.build-date=20240424, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, RELEASE=main-4e213a5, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, maintainer=Guillaume Abrioux ) 2024-04-27T04:17:26.136 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:17:25 smithi162 ceph-mon[110641]: osdmap e66: 6 total, 5 up, 6 in 2024-04-27T04:17:26.672 INFO:journalctl@ceph.osd.3.smithi181.stdout:Apr 27 04:17:26 smithi181 podman[90079]: 2024-04-27 04:17:26.337926666 +0000 UTC m=+0.598637400 container remove 35463df5a2d3b2675e116ea1f53961cf49f3b7307d4ad412a7ec9c792d344d38 (image=quay.ceph.io/ceph-ci/ceph@sha256:90a3a9fa9ceb8977f6dad9ccd36fe46018d07e2118199820523a6b35301d9c16, name=ceph-7f9750d6-044a-11ef-bc93-c7b262605968-osd-3-deactivate, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, ceph=True, GIT_CLEAN=True, org.label-schema.license=GPLv2, RELEASE=main-4e213a5, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.build-date=20240424, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-27T04:17:26.672 INFO:journalctl@ceph.osd.3.smithi181.stdout:Apr 27 04:17:26 smithi181 systemd[1]: ceph-7f9750d6-044a-11ef-bc93-c7b262605968@osd.3.service: Deactivated successfully. 2024-04-27T04:17:26.672 INFO:journalctl@ceph.osd.3.smithi181.stdout:Apr 27 04:17:26 smithi181 systemd[1]: Stopped Ceph osd.3 for 7f9750d6-044a-11ef-bc93-c7b262605968. 2024-04-27T04:17:26.672 INFO:journalctl@ceph.osd.3.smithi181.stdout:Apr 27 04:17:26 smithi181 systemd[1]: ceph-7f9750d6-044a-11ef-bc93-c7b262605968@osd.3.service: Consumed 1min 20.183s CPU time. 2024-04-27T04:17:26.672 INFO:journalctl@ceph.osd.3.smithi181.stdout:Apr 27 04:17:26 smithi181 systemd[1]: Starting Ceph osd.3 for 7f9750d6-044a-11ef-bc93-c7b262605968... 2024-04-27T04:17:27.012 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:17:26 smithi181 ceph-mon[81134]: pgmap v114: 65 pgs: 4 peering, 15 stale+active+clean, 46 active+clean; 208 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:17:27.012 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:17:26 smithi181 ceph-mon[81134]: Health check failed: Reduced data availability: 1 pg peering (PG_AVAILABILITY) 2024-04-27T04:17:27.013 INFO:journalctl@ceph.osd.3.smithi181.stdout:Apr 27 04:17:26 smithi181 podman[90182]: 2024-04-27 04:17:26.678375785 +0000 UTC m=+0.018175665 image pull 67e414ff4bc41092cb31c84c98a9563a32813d02583028679c46b40362afeb3e quay.ceph.io/ceph-ci/ceph@sha256:90a3a9fa9ceb8977f6dad9ccd36fe46018d07e2118199820523a6b35301d9c16 2024-04-27T04:17:27.013 INFO:journalctl@ceph.osd.3.smithi181.stdout:Apr 27 04:17:26 smithi181 podman[90182]: 2024-04-27 04:17:26.811962394 +0000 UTC m=+0.151762305 container create 3f96871134701018d5ec13659a870572f7cb3eff69380fc0825de777bdd0bb9c (image=quay.ceph.io/ceph-ci/ceph@sha256:90a3a9fa9ceb8977f6dad9ccd36fe46018d07e2118199820523a6b35301d9c16, name=ceph-7f9750d6-044a-11ef-bc93-c7b262605968-osd-3-activate, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, RELEASE=main-4e213a5, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240424) 2024-04-27T04:17:27.013 INFO:journalctl@ceph.osd.3.smithi181.stdout:Apr 27 04:17:27 smithi181 podman[90182]: 2024-04-27 04:17:27.01464192 +0000 UTC m=+0.354441790 container init 3f96871134701018d5ec13659a870572f7cb3eff69380fc0825de777bdd0bb9c (image=quay.ceph.io/ceph-ci/ceph@sha256:90a3a9fa9ceb8977f6dad9ccd36fe46018d07e2118199820523a6b35301d9c16, name=ceph-7f9750d6-044a-11ef-bc93-c7b262605968-osd-3-activate, ceph=True, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240424, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, RELEASE=main-4e213a5) 2024-04-27T04:17:27.136 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:17:26 smithi162 ceph-mon[110641]: pgmap v114: 65 pgs: 4 peering, 15 stale+active+clean, 46 active+clean; 208 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:17:27.136 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:17:26 smithi162 ceph-mon[110641]: Health check failed: Reduced data availability: 1 pg peering (PG_AVAILABILITY) 2024-04-27T04:17:27.422 INFO:journalctl@ceph.osd.3.smithi181.stdout:Apr 27 04:17:27 smithi181 podman[90182]: 2024-04-27 04:17:27.017887278 +0000 UTC m=+0.357687150 container start 3f96871134701018d5ec13659a870572f7cb3eff69380fc0825de777bdd0bb9c (image=quay.ceph.io/ceph-ci/ceph@sha256:90a3a9fa9ceb8977f6dad9ccd36fe46018d07e2118199820523a6b35301d9c16, name=ceph-7f9750d6-044a-11ef-bc93-c7b262605968-osd-3-activate, io.buildah.version=1.33.5, RELEASE=main-4e213a5, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.build-date=20240424, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-27T04:17:27.422 INFO:journalctl@ceph.osd.3.smithi181.stdout:Apr 27 04:17:27 smithi181 podman[90182]: 2024-04-27 04:17:27.097957541 +0000 UTC m=+0.437757429 container attach 3f96871134701018d5ec13659a870572f7cb3eff69380fc0825de777bdd0bb9c (image=quay.ceph.io/ceph-ci/ceph@sha256:90a3a9fa9ceb8977f6dad9ccd36fe46018d07e2118199820523a6b35301d9c16, name=ceph-7f9750d6-044a-11ef-bc93-c7b262605968-osd-3-activate, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, RELEASE=main-4e213a5, org.label-schema.build-date=20240424, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, CEPH_POINT_RELEASE=) 2024-04-27T04:17:27.749 INFO:journalctl@ceph.osd.3.smithi181.stdout:Apr 27 04:17:27 smithi181 ceph-7f9750d6-044a-11ef-bc93-c7b262605968-osd-3-activate[90197]: Running command: /usr/bin/ceph-authtool --gen-print-key 2024-04-27T04:17:27.749 INFO:journalctl@ceph.osd.3.smithi181.stdout:Apr 27 04:17:27 smithi181 ceph-7f9750d6-044a-11ef-bc93-c7b262605968-osd-3-activate[90197]: Running command: /usr/bin/ceph-authtool --gen-print-key 2024-04-27T04:17:27.749 INFO:journalctl@ceph.osd.3.smithi181.stdout:Apr 27 04:17:27 smithi181 bash[90182]: Running command: /usr/bin/ceph-authtool --gen-print-key 2024-04-27T04:17:27.750 INFO:journalctl@ceph.osd.3.smithi181.stdout:Apr 27 04:17:27 smithi181 bash[90182]: Running command: /usr/bin/ceph-authtool --gen-print-key 2024-04-27T04:17:27.750 INFO:journalctl@ceph.osd.3.smithi181.stdout:Apr 27 04:17:27 smithi181 bash[90182]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2024-04-27T04:17:27.750 INFO:journalctl@ceph.osd.3.smithi181.stdout:Apr 27 04:17:27 smithi181 bash[90182]: Running command: /usr/bin/ceph-bluestore-tool prime-osd-dir --path /var/lib/ceph/osd/ceph-3 --no-mon-config --dev /dev/mapper/vg_nvme-lv_4 2024-04-27T04:17:27.750 INFO:journalctl@ceph.osd.3.smithi181.stdout:Apr 27 04:17:27 smithi181 bash[90182]: Running command: /usr/bin/chown -h ceph:ceph /dev/mapper/vg_nvme-lv_4 2024-04-27T04:17:27.750 INFO:journalctl@ceph.osd.3.smithi181.stdout:Apr 27 04:17:27 smithi181 bash[90182]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-3 2024-04-27T04:17:27.750 INFO:journalctl@ceph.osd.3.smithi181.stdout:Apr 27 04:17:27 smithi181 bash[90182]: Running command: /usr/bin/ln -s /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3/block 2024-04-27T04:17:27.750 INFO:journalctl@ceph.osd.3.smithi181.stdout:Apr 27 04:17:27 smithi181 bash[90182]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2024-04-27T04:17:27.750 INFO:journalctl@ceph.osd.3.smithi181.stdout:Apr 27 04:17:27 smithi181 bash[90182]: --> ceph-volume raw activate successful for osd ID: 3 2024-04-27T04:17:27.750 INFO:journalctl@ceph.osd.3.smithi181.stdout:Apr 27 04:17:27 smithi181 ceph-7f9750d6-044a-11ef-bc93-c7b262605968-osd-3-activate[90197]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2024-04-27T04:17:27.750 INFO:journalctl@ceph.osd.3.smithi181.stdout:Apr 27 04:17:27 smithi181 ceph-7f9750d6-044a-11ef-bc93-c7b262605968-osd-3-activate[90197]: Running command: /usr/bin/ceph-bluestore-tool prime-osd-dir --path /var/lib/ceph/osd/ceph-3 --no-mon-config --dev /dev/mapper/vg_nvme-lv_4 2024-04-27T04:17:27.750 INFO:journalctl@ceph.osd.3.smithi181.stdout:Apr 27 04:17:27 smithi181 ceph-7f9750d6-044a-11ef-bc93-c7b262605968-osd-3-activate[90197]: Running command: /usr/bin/chown -h ceph:ceph /dev/mapper/vg_nvme-lv_4 2024-04-27T04:17:27.750 INFO:journalctl@ceph.osd.3.smithi181.stdout:Apr 27 04:17:27 smithi181 ceph-7f9750d6-044a-11ef-bc93-c7b262605968-osd-3-activate[90197]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-3 2024-04-27T04:17:27.750 INFO:journalctl@ceph.osd.3.smithi181.stdout:Apr 27 04:17:27 smithi181 ceph-7f9750d6-044a-11ef-bc93-c7b262605968-osd-3-activate[90197]: Running command: /usr/bin/ln -s /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3/block 2024-04-27T04:17:27.750 INFO:journalctl@ceph.osd.3.smithi181.stdout:Apr 27 04:17:27 smithi181 ceph-7f9750d6-044a-11ef-bc93-c7b262605968-osd-3-activate[90197]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2024-04-27T04:17:27.750 INFO:journalctl@ceph.osd.3.smithi181.stdout:Apr 27 04:17:27 smithi181 ceph-7f9750d6-044a-11ef-bc93-c7b262605968-osd-3-activate[90197]: --> ceph-volume raw activate successful for osd ID: 3 2024-04-27T04:17:27.751 INFO:journalctl@ceph.osd.3.smithi181.stdout:Apr 27 04:17:27 smithi181 podman[90182]: 2024-04-27 04:17:27.518774706 +0000 UTC m=+0.858574578 container died 3f96871134701018d5ec13659a870572f7cb3eff69380fc0825de777bdd0bb9c (image=quay.ceph.io/ceph-ci/ceph@sha256:90a3a9fa9ceb8977f6dad9ccd36fe46018d07e2118199820523a6b35301d9c16, name=ceph-7f9750d6-044a-11ef-bc93-c7b262605968-osd-3-activate, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , RELEASE=main-4e213a5, org.label-schema.build-date=20240424, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, GIT_CLEAN=True) 2024-04-27T04:17:28.133 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:17:27 smithi181 ceph-mon[81134]: Health check failed: Degraded data redundancy: 7/231 objects degraded (3.030%), 3 pgs degraded (PG_DEGRADED) 2024-04-27T04:17:28.136 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:17:27 smithi162 ceph-mon[110641]: Health check failed: Degraded data redundancy: 7/231 objects degraded (3.030%), 3 pgs degraded (PG_DEGRADED) 2024-04-27T04:17:28.422 INFO:journalctl@ceph.osd.3.smithi181.stdout:Apr 27 04:17:28 smithi181 podman[90323]: 2024-04-27 04:17:28.135672885 +0000 UTC m=+0.611802531 container remove 3f96871134701018d5ec13659a870572f7cb3eff69380fc0825de777bdd0bb9c (image=quay.ceph.io/ceph-ci/ceph@sha256:90a3a9fa9ceb8977f6dad9ccd36fe46018d07e2118199820523a6b35301d9c16, name=ceph-7f9750d6-044a-11ef-bc93-c7b262605968-osd-3-activate, RELEASE=main-4e213a5, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240424, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD, GIT_CLEAN=True, io.buildah.version=1.33.5, ceph=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-27T04:17:28.422 INFO:journalctl@ceph.osd.3.smithi181.stdout:Apr 27 04:17:28 smithi181 podman[90386]: 2024-04-27 04:17:28.252906276 +0000 UTC m=+0.018099227 image pull 67e414ff4bc41092cb31c84c98a9563a32813d02583028679c46b40362afeb3e quay.ceph.io/ceph-ci/ceph@sha256:90a3a9fa9ceb8977f6dad9ccd36fe46018d07e2118199820523a6b35301d9c16 2024-04-27T04:17:28.422 INFO:journalctl@ceph.osd.3.smithi181.stdout:Apr 27 04:17:28 smithi181 podman[90386]: 2024-04-27 04:17:28.369830732 +0000 UTC m=+0.135023681 container create ab473cfd54d26de1cca563d0c68ad497b05a5291422771be5378898a3ab2abda (image=quay.ceph.io/ceph-ci/ceph@sha256:90a3a9fa9ceb8977f6dad9ccd36fe46018d07e2118199820523a6b35301d9c16, name=ceph-7f9750d6-044a-11ef-bc93-c7b262605968-osd-3, ceph=True, RELEASE=main-4e213a5, org.label-schema.build-date=20240424, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux ) 2024-04-27T04:17:28.750 INFO:journalctl@ceph.osd.3.smithi181.stdout:Apr 27 04:17:28 smithi181 podman[90386]: 2024-04-27 04:17:28.555421216 +0000 UTC m=+0.320614157 container init ab473cfd54d26de1cca563d0c68ad497b05a5291422771be5378898a3ab2abda (image=quay.ceph.io/ceph-ci/ceph@sha256:90a3a9fa9ceb8977f6dad9ccd36fe46018d07e2118199820523a6b35301d9c16, name=ceph-7f9750d6-044a-11ef-bc93-c7b262605968-osd-3, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240424, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, ceph=True, RELEASE=main-4e213a5, org.label-schema.license=GPLv2, io.buildah.version=1.33.5) 2024-04-27T04:17:28.750 INFO:journalctl@ceph.osd.3.smithi181.stdout:Apr 27 04:17:28 smithi181 podman[90386]: 2024-04-27 04:17:28.5581915 +0000 UTC m=+0.323384444 container start ab473cfd54d26de1cca563d0c68ad497b05a5291422771be5378898a3ab2abda (image=quay.ceph.io/ceph-ci/ceph@sha256:90a3a9fa9ceb8977f6dad9ccd36fe46018d07e2118199820523a6b35301d9c16, name=ceph-7f9750d6-044a-11ef-bc93-c7b262605968-osd-3, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, ceph=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, RELEASE=main-4e213a5, org.label-schema.build-date=20240424, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2) 2024-04-27T04:17:28.750 INFO:journalctl@ceph.osd.3.smithi181.stdout:Apr 27 04:17:28 smithi181 ceph-osd[90404]: -- 172.21.15.181:0/238704941 <== mon.1 v2:172.21.15.181:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5593da5c7680 con 0x5593da591000 2024-04-27T04:17:28.750 INFO:journalctl@ceph.osd.3.smithi181.stdout:Apr 27 04:17:28 smithi181 bash[90386]: ab473cfd54d26de1cca563d0c68ad497b05a5291422771be5378898a3ab2abda 2024-04-27T04:17:28.750 INFO:journalctl@ceph.osd.3.smithi181.stdout:Apr 27 04:17:28 smithi181 systemd[1]: Started Ceph osd.3 for 7f9750d6-044a-11ef-bc93-c7b262605968. 2024-04-27T04:17:29.103 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:17:28 smithi181 ceph-mon[81134]: pgmap v115: 65 pgs: 3 active+undersized+degraded, 4 peering, 12 stale+active+clean, 4 active+undersized, 42 active+clean; 208 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 7/231 objects degraded (3.030%) 2024-04-27T04:17:29.104 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:17:28 smithi181 ceph-mon[81134]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' 2024-04-27T04:17:29.104 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:17:28 smithi181 ceph-mon[81134]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' 2024-04-27T04:17:29.104 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:17:28 smithi181 ceph-mon[81134]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T04:17:29.136 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:17:28 smithi162 ceph-mon[110641]: pgmap v115: 65 pgs: 3 active+undersized+degraded, 4 peering, 12 stale+active+clean, 4 active+undersized, 42 active+clean; 208 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 7/231 objects degraded (3.030%) 2024-04-27T04:17:29.136 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:17:28 smithi162 ceph-mon[110641]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' 2024-04-27T04:17:29.136 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:17:28 smithi162 ceph-mon[110641]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' 2024-04-27T04:17:29.137 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:17:28 smithi162 ceph-mon[110641]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T04:17:29.422 INFO:journalctl@ceph.osd.3.smithi181.stdout:Apr 27 04:17:29 smithi181 ceph-7f9750d6-044a-11ef-bc93-c7b262605968-osd-3[90400]: 2024-04-27T04:17:29.104+0000 7fbc67e737c0 -1 Falling back to public interface 2024-04-27T04:17:30.060 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:17:29 smithi162 ceph-mon[110641]: pgmap v116: 65 pgs: 18 active+undersized+degraded, 20 active+undersized, 27 active+clean; 208 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 55/231 objects degraded (23.810%) 2024-04-27T04:17:30.060 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:17:29 smithi162 ceph-mon[110641]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg peering) 2024-04-27T04:17:30.167 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:17:29 smithi181 ceph-mon[81134]: pgmap v116: 65 pgs: 18 active+undersized+degraded, 20 active+undersized, 27 active+clean; 208 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 55/231 objects degraded (23.810%) 2024-04-27T04:17:30.167 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:17:29 smithi181 ceph-mon[81134]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg peering) 2024-04-27T04:17:32.886 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:17:32 smithi162 ceph-mon[110641]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' 2024-04-27T04:17:32.886 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:17:32 smithi162 ceph-mon[110641]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' 2024-04-27T04:17:32.887 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:17:32 smithi162 ceph-mon[110641]: pgmap v117: 65 pgs: 18 active+undersized+degraded, 20 active+undersized, 27 active+clean; 208 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 55/231 objects degraded (23.810%) 2024-04-27T04:17:32.921 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:17:32 smithi181 ceph-mon[81134]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' 2024-04-27T04:17:32.921 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:17:32 smithi181 ceph-mon[81134]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' 2024-04-27T04:17:32.921 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:17:32 smithi181 ceph-mon[81134]: pgmap v117: 65 pgs: 18 active+undersized+degraded, 20 active+undersized, 27 active+clean; 208 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 55/231 objects degraded (23.810%) 2024-04-27T04:17:33.668 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:17:33 smithi162 ceph-mon[110641]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' 2024-04-27T04:17:33.669 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:17:33 smithi162 ceph-mon[110641]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-27T04:17:33.669 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:17:33 smithi162 ceph-mon[110641]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' 2024-04-27T04:17:33.669 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:17:33 smithi162 ceph-mon[110641]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' 2024-04-27T04:17:33.922 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:17:33 smithi181 ceph-mon[81134]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' 2024-04-27T04:17:33.922 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:17:33 smithi181 ceph-mon[81134]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-27T04:17:33.922 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:17:33 smithi181 ceph-mon[81134]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' 2024-04-27T04:17:33.922 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:17:33 smithi181 ceph-mon[81134]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' 2024-04-27T04:17:34.564 INFO:journalctl@ceph.osd.3.smithi181.stdout:Apr 27 04:17:34 smithi181 ceph-7f9750d6-044a-11ef-bc93-c7b262605968-osd-3[90400]: 2024-04-27T04:17:34.259+0000 7fbc67e737c0 -1 osd.3 64 log_to_monitors true 2024-04-27T04:17:34.859 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:17:34 smithi181 ceph-mon[81134]: pgmap v118: 65 pgs: 18 active+undersized+degraded, 20 active+undersized, 27 active+clean; 208 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 55/231 objects degraded (23.810%) 2024-04-27T04:17:34.859 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:17:34 smithi181 ceph-mon[81134]: from='osd.3 [v2:172.21.15.181:6800/114683138,v1:172.21.15.181:6801/114683138]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-04-27T04:17:34.859 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:17:34 smithi181 ceph-mon[81134]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-04-27T04:17:34.860 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:17:34 smithi181 ceph-mon[81134]: Health check update: Degraded data redundancy: 55/231 objects degraded (23.810%), 18 pgs degraded (PG_DEGRADED) 2024-04-27T04:17:35.136 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:17:34 smithi162 ceph-mon[110641]: pgmap v118: 65 pgs: 18 active+undersized+degraded, 20 active+undersized, 27 active+clean; 208 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 55/231 objects degraded (23.810%) 2024-04-27T04:17:35.137 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:17:34 smithi162 ceph-mon[110641]: from='osd.3 [v2:172.21.15.181:6800/114683138,v1:172.21.15.181:6801/114683138]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-04-27T04:17:35.137 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:17:34 smithi162 ceph-mon[110641]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-04-27T04:17:35.137 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:17:34 smithi162 ceph-mon[110641]: Health check update: Degraded data redundancy: 55/231 objects degraded (23.810%), 18 pgs degraded (PG_DEGRADED) 2024-04-27T04:17:36.021 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:17:35 smithi181 ceph-mon[81134]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-04-27T04:17:36.021 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:17:35 smithi181 ceph-mon[81134]: from='osd.3 [v2:172.21.15.181:6800/114683138,v1:172.21.15.181:6801/114683138]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi181", "root=default"]}]: dispatch 2024-04-27T04:17:36.021 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:17:35 smithi181 ceph-mon[81134]: osdmap e67: 6 total, 5 up, 6 in 2024-04-27T04:17:36.021 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:17:35 smithi181 ceph-mon[81134]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi181", "root=default"]}]: dispatch 2024-04-27T04:17:36.021 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:17:35 smithi181 ceph-mon[81134]: from='osd.3 ' entity='osd.3' 2024-04-27T04:17:36.021 INFO:journalctl@ceph.osd.3.smithi181.stdout:Apr 27 04:17:35 smithi181 ceph-7f9750d6-044a-11ef-bc93-c7b262605968-osd-3[90400]: 2024-04-27T04:17:35.703+0000 7fbc552c3700 -1 osd.3 64 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-27T04:17:36.136 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:17:35 smithi162 ceph-mon[110641]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-04-27T04:17:36.137 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:17:35 smithi162 ceph-mon[110641]: from='osd.3 [v2:172.21.15.181:6800/114683138,v1:172.21.15.181:6801/114683138]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi181", "root=default"]}]: dispatch 2024-04-27T04:17:36.137 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:17:35 smithi162 ceph-mon[110641]: osdmap e67: 6 total, 5 up, 6 in 2024-04-27T04:17:36.137 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:17:35 smithi162 ceph-mon[110641]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi181", "root=default"]}]: dispatch 2024-04-27T04:17:36.137 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:17:35 smithi162 ceph-mon[110641]: from='osd.3 ' entity='osd.3' 2024-04-27T04:17:36.886 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:17:36 smithi162 ceph-mon[110641]: pgmap v120: 65 pgs: 18 active+undersized+degraded, 20 active+undersized, 27 active+clean; 208 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 55/231 objects degraded (23.810%) 2024-04-27T04:17:36.887 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:17:36 smithi162 ceph-mon[110641]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' 2024-04-27T04:17:36.887 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:17:36 smithi162 ceph-mon[110641]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' 2024-04-27T04:17:36.887 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:17:36 smithi162 ceph-mon[110641]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T04:17:36.887 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:17:36 smithi162 ceph-mon[110641]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T04:17:36.887 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:17:36 smithi162 ceph-mon[110641]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' 2024-04-27T04:17:36.887 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:17:36 smithi162 ceph-mon[110641]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T04:17:36.887 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:17:36 smithi162 ceph-mon[110641]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "versions"}]: dispatch 2024-04-27T04:17:36.887 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:17:36 smithi162 ceph-mon[110641]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "versions"}]: dispatch 2024-04-27T04:17:36.887 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:17:36 smithi162 ceph-mon[110641]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "versions"}]: dispatch 2024-04-27T04:17:36.887 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:17:36 smithi162 ceph-mon[110641]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "osd ok-to-stop", "ids": ["4"], "max": 16}]: dispatch 2024-04-27T04:17:36.887 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:17:36 smithi162 ceph-mon[110641]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-04-27T04:17:36.887 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:17:36 smithi162 ceph-mon[110641]: osd.3 [v2:172.21.15.181:6800/114683138,v1:172.21.15.181:6801/114683138] boot 2024-04-27T04:17:36.888 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:17:36 smithi162 ceph-mon[110641]: osdmap e68: 6 total, 6 up, 6 in 2024-04-27T04:17:36.888 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:17:36 smithi162 ceph-mon[110641]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-27T04:17:37.172 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:17:36 smithi181 ceph-mon[81134]: pgmap v120: 65 pgs: 18 active+undersized+degraded, 20 active+undersized, 27 active+clean; 208 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 55/231 objects degraded (23.810%) 2024-04-27T04:17:37.172 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:17:36 smithi181 ceph-mon[81134]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' 2024-04-27T04:17:37.172 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:17:36 smithi181 ceph-mon[81134]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' 2024-04-27T04:17:37.172 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:17:36 smithi181 ceph-mon[81134]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T04:17:37.172 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:17:36 smithi181 ceph-mon[81134]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T04:17:37.172 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:17:36 smithi181 ceph-mon[81134]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' 2024-04-27T04:17:37.172 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:17:36 smithi181 ceph-mon[81134]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T04:17:37.172 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:17:36 smithi181 ceph-mon[81134]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "versions"}]: dispatch 2024-04-27T04:17:37.173 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:17:36 smithi181 ceph-mon[81134]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "versions"}]: dispatch 2024-04-27T04:17:37.173 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:17:36 smithi181 ceph-mon[81134]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "versions"}]: dispatch 2024-04-27T04:17:37.173 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:17:36 smithi181 ceph-mon[81134]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "osd ok-to-stop", "ids": ["4"], "max": 16}]: dispatch 2024-04-27T04:17:37.173 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:17:36 smithi181 ceph-mon[81134]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-04-27T04:17:37.173 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:17:36 smithi181 ceph-mon[81134]: osd.3 [v2:172.21.15.181:6800/114683138,v1:172.21.15.181:6801/114683138] boot 2024-04-27T04:17:37.173 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:17:36 smithi181 ceph-mon[81134]: osdmap e68: 6 total, 6 up, 6 in 2024-04-27T04:17:37.173 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:17:36 smithi181 ceph-mon[81134]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-27T04:17:38.136 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:17:37 smithi162 ceph-mon[110641]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["4"], "max": 16}]: dispatch 2024-04-27T04:17:38.136 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:17:37 smithi162 ceph-mon[110641]: Upgrade: unsafe to stop osd(s) at this time (18 PGs are or would become offline) 2024-04-27T04:17:38.136 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:17:37 smithi162 ceph-mon[110641]: osdmap e69: 6 total, 6 up, 6 in 2024-04-27T04:17:38.171 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:17:37 smithi181 ceph-mon[81134]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["4"], "max": 16}]: dispatch 2024-04-27T04:17:38.172 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:17:37 smithi181 ceph-mon[81134]: Upgrade: unsafe to stop osd(s) at this time (18 PGs are or would become offline) 2024-04-27T04:17:38.172 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:17:37 smithi181 ceph-mon[81134]: osdmap e69: 6 total, 6 up, 6 in 2024-04-27T04:17:39.136 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:17:38 smithi162 ceph-mon[110641]: pgmap v122: 65 pgs: 18 active+undersized+degraded, 20 active+undersized, 27 active+clean; 208 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 639 B/s rd, 1 op/s; 55/231 objects degraded (23.810%) 2024-04-27T04:17:39.172 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:17:38 smithi181 ceph-mon[81134]: pgmap v122: 65 pgs: 18 active+undersized+degraded, 20 active+undersized, 27 active+clean; 208 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 639 B/s rd, 1 op/s; 55/231 objects degraded (23.810%) 2024-04-27T04:17:40.136 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:17:39 smithi162 ceph-mon[110641]: pgmap v124: 65 pgs: 10 active+undersized+degraded, 10 active+undersized, 45 active+clean; 208 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 0 B/s rd, 0 op/s; 28/231 objects degraded (12.121%) 2024-04-27T04:17:40.136 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:17:39 smithi162 ceph-mon[110641]: Health check update: Degraded data redundancy: 28/231 objects degraded (12.121%), 10 pgs degraded (PG_DEGRADED) 2024-04-27T04:17:40.172 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:17:39 smithi181 ceph-mon[81134]: pgmap v124: 65 pgs: 10 active+undersized+degraded, 10 active+undersized, 45 active+clean; 208 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 0 B/s rd, 0 op/s; 28/231 objects degraded (12.121%) 2024-04-27T04:17:40.172 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:17:39 smithi181 ceph-mon[81134]: Health check update: Degraded data redundancy: 28/231 objects degraded (12.121%), 10 pgs degraded (PG_DEGRADED) 2024-04-27T04:17:42.974 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:17:42 smithi181 ceph-mon[81134]: pgmap v125: 65 pgs: 1 active+undersized+degraded, 3 active+undersized, 61 active+clean; 208 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 742 B/s rd, 1 op/s; 3/231 objects degraded (1.299%) 2024-04-27T04:17:43.136 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:17:42 smithi162 ceph-mon[110641]: pgmap v125: 65 pgs: 1 active+undersized+degraded, 3 active+undersized, 61 active+clean; 208 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 742 B/s rd, 1 op/s; 3/231 objects degraded (1.299%) 2024-04-27T04:17:44.136 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:17:43 smithi162 ceph-mon[110641]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 3/231 objects degraded (1.299%), 1 pg degraded) 2024-04-27T04:17:44.137 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:17:43 smithi162 ceph-mon[110641]: Cluster is now healthy 2024-04-27T04:17:44.171 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:17:43 smithi181 ceph-mon[81134]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 3/231 objects degraded (1.299%), 1 pg degraded) 2024-04-27T04:17:44.171 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:17:43 smithi181 ceph-mon[81134]: Cluster is now healthy 2024-04-27T04:17:45.136 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:17:44 smithi162 ceph-mon[110641]: pgmap v126: 65 pgs: 65 active+clean; 208 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 639 B/s rd, 1 op/s 2024-04-27T04:17:45.172 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:17:44 smithi181 ceph-mon[81134]: pgmap v126: 65 pgs: 65 active+clean; 208 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 639 B/s rd, 1 op/s 2024-04-27T04:17:47.136 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:17:46 smithi162 ceph-mon[110641]: pgmap v127: 65 pgs: 65 active+clean; 208 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T04:17:47.171 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:17:46 smithi181 ceph-mon[81134]: pgmap v127: 65 pgs: 65 active+clean; 208 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T04:17:48.922 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:17:48 smithi181 ceph-mon[81134]: pgmap v128: 65 pgs: 65 active+clean; 208 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1023 B/s rd, 1 op/s 2024-04-27T04:17:48.922 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:17:48 smithi181 ceph-mon[81134]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' 2024-04-27T04:17:48.922 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:17:48 smithi181 ceph-mon[81134]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-27T04:17:49.136 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:17:48 smithi162 ceph-mon[110641]: pgmap v128: 65 pgs: 65 active+clean; 208 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1023 B/s rd, 1 op/s 2024-04-27T04:17:49.136 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:17:48 smithi162 ceph-mon[110641]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' 2024-04-27T04:17:49.136 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:17:48 smithi162 ceph-mon[110641]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-27T04:17:51.136 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:17:50 smithi162 ceph-mon[110641]: pgmap v129: 65 pgs: 65 active+clean; 208 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 856 B/s rd, 1 op/s 2024-04-27T04:17:51.171 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:17:50 smithi181 ceph-mon[81134]: pgmap v129: 65 pgs: 65 active+clean; 208 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 856 B/s rd, 1 op/s 2024-04-27T04:17:51.411 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:17:51.411+0000 7f1f14b07700 1 -- 172.21.15.162:0/51919208 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1f101026b0 msgr2=0x7f1f100fbd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:17:51.411 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:17:51.411+0000 7f1f14b07700 1 --2- 172.21.15.162:0/51919208 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1f101026b0 0x7f1f100fbd20 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f1f04004940 tx=0x7f1f04009e30 comp rx=0 tx=0).stop 2024-04-27T04:17:51.411 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:17:51.411+0000 7f1f14b07700 1 -- 172.21.15.162:0/51919208 shutdown_connections 2024-04-27T04:17:51.412 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:17:51.411+0000 7f1f14b07700 1 --2- 172.21.15.162:0/51919208 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1f101026b0 0x7f1f100fbd20 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:17:51.412 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:17:51.411+0000 7f1f14b07700 1 --2- 172.21.15.162:0/51919208 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1f10101d10 0x7f1f101020e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:17:51.412 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:17:51.411+0000 7f1f14b07700 1 -- 172.21.15.162:0/51919208 >> 172.21.15.162:0/51919208 conn(0x7f1f100f7860 msgr2=0x7f1f100f9c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:17:51.412 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:17:51.411+0000 7f1f14b07700 1 -- 172.21.15.162:0/51919208 shutdown_connections 2024-04-27T04:17:51.412 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:17:51.411+0000 7f1f14b07700 1 -- 172.21.15.162:0/51919208 wait complete. 2024-04-27T04:17:51.412 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:17:51.411+0000 7f1f14b07700 1 Processor -- start 2024-04-27T04:17:51.412 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:17:51.411+0000 7f1f14b07700 1 -- start start 2024-04-27T04:17:51.412 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:17:51.412+0000 7f1f14b07700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1f10101d10 0x7f1f10113a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:17:51.412 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:17:51.412+0000 7f1f14b07700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1f101026b0 0x7f1f10113f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:17:51.413 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:17:51.412+0000 7f1f14b07700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f10114650 con 0x7f1f10101d10 2024-04-27T04:17:51.413 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:17:51.412+0000 7f1f14b07700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f1010dba0 con 0x7f1f101026b0 2024-04-27T04:17:51.413 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:17:51.412+0000 7f1f0e59c700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1f10101d10 0x7f1f10113a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:17:51.413 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:17:51.412+0000 7f1f0e59c700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1f10101d10 0x7f1f10113a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:47686/0 (socket says 172.21.15.162:47686) 2024-04-27T04:17:51.413 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:17:51.412+0000 7f1f0e59c700 1 -- 172.21.15.162:0/714892422 learned_addr learned my addr 172.21.15.162:0/714892422 (peer_addr_for_me v2:172.21.15.162:0/0) 2024-04-27T04:17:51.413 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:17:51.412+0000 7f1f0dd9b700 1 --2- 172.21.15.162:0/714892422 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1f101026b0 0x7f1f10113f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:17:51.413 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:17:51.412+0000 7f1f0e59c700 1 -- 172.21.15.162:0/714892422 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1f101026b0 msgr2=0x7f1f10113f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:17:51.413 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:17:51.412+0000 7f1f0e59c700 1 --2- 172.21.15.162:0/714892422 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1f101026b0 0x7f1f10113f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:17:51.413 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:17:51.412+0000 7f1f0e59c700 1 -- 172.21.15.162:0/714892422 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f04005040 con 0x7f1f10101d10 2024-04-27T04:17:51.413 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:17:51.412+0000 7f1f0e59c700 1 --2- 172.21.15.162:0/714892422 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1f10101d10 0x7f1f10113a30 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f1f0000b790 tx=0x7f1f00009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:17:51.413 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:17:51.412+0000 7f1efeffd700 1 -- 172.21.15.162:0/714892422 <== mon.0 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1f0000f6f0 con 0x7f1f10101d10 2024-04-27T04:17:51.413 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:17:51.412+0000 7f1efeffd700 1 -- 172.21.15.162:0/714892422 <== mon.0 v2:172.21.15.162:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f1f0000fd20 con 0x7f1f10101d10 2024-04-27T04:17:51.413 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:17:51.412+0000 7f1f14b07700 1 -- 172.21.15.162:0/714892422 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1f1010de20 con 0x7f1f10101d10 2024-04-27T04:17:51.414 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:17:51.413+0000 7f1efeffd700 1 -- 172.21.15.162:0/714892422 <== mon.0 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1f000103a0 con 0x7f1f10101d10 2024-04-27T04:17:51.414 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:17:51.413+0000 7f1f14b07700 1 -- 172.21.15.162:0/714892422 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1f1010e360 con 0x7f1f10101d10 2024-04-27T04:17:51.414 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:17:51.413+0000 7f1f14b07700 1 -- 172.21.15.162:0/714892422 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1f100fd460 con 0x7f1f10101d10 2024-04-27T04:17:51.415 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:17:51.414+0000 7f1efeffd700 1 -- 172.21.15.162:0/714892422 <== mon.0 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 39) v1 ==== 92747+0+0 (secure 0 0 0) 0x7f1f00011400 con 0x7f1f10101d10 2024-04-27T04:17:51.415 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:17:51.414+0000 7f1efeffd700 1 --2- 172.21.15.162:0/714892422 >> [v2:172.21.15.162:6800/1396866108,v1:172.21.15.162:6801/1396866108] conn(0x7f1ef806ec00 0x7f1ef80710b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:17:51.415 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:17:51.415+0000 7f1efeffd700 1 -- 172.21.15.162:0/714892422 <== mon.0 v2:172.21.15.162:3300/0 5 ==== osd_map(69..69 src has 1..69) v4 ==== 6410+0+0 (secure 0 0 0) 0x7f1f0008e660 con 0x7f1f10101d10 2024-04-27T04:17:51.415 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:17:51.415+0000 7f1f0dd9b700 1 --2- 172.21.15.162:0/714892422 >> [v2:172.21.15.162:6800/1396866108,v1:172.21.15.162:6801/1396866108] conn(0x7f1ef806ec00 0x7f1ef80710b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:17:51.415 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:17:51.415+0000 7f1f0dd9b700 1 --2- 172.21.15.162:0/714892422 >> [v2:172.21.15.162:6800/1396866108,v1:172.21.15.162:6801/1396866108] conn(0x7f1ef806ec00 0x7f1ef80710b0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f1f1010f2d0 tx=0x7f1f04009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:17:51.417 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:17:51.419+0000 7f1efeffd700 1 -- 172.21.15.162:0/714892422 <== mon.0 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7f1f0005b010 con 0x7f1f10101d10 2024-04-27T04:17:51.634 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:17:51.636+0000 7f1f14b07700 1 -- 172.21.15.162:0/714892422 --> [v2:172.21.15.162:6800/1396866108,v1:172.21.15.162:6801/1396866108] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f1f1010f140 con 0x7f1ef806ec00 2024-04-27T04:17:51.635 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:17:51.637+0000 7f1efeffd700 1 -- 172.21.15.162:0/714892422 <== mgr.34104 v2:172.21.15.162:6800/1396866108 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+401 (secure 0 0 0) 0x7f1f1010f140 con 0x7f1ef806ec00 2024-04-27T04:17:51.637 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:17:51.639+0000 7f1f14b07700 1 -- 172.21.15.162:0/714892422 >> [v2:172.21.15.162:6800/1396866108,v1:172.21.15.162:6801/1396866108] conn(0x7f1ef806ec00 msgr2=0x7f1ef80710b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:17:51.637 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:17:51.639+0000 7f1f14b07700 1 --2- 172.21.15.162:0/714892422 >> [v2:172.21.15.162:6800/1396866108,v1:172.21.15.162:6801/1396866108] conn(0x7f1ef806ec00 0x7f1ef80710b0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f1f1010f2d0 tx=0x7f1f04009e30 comp rx=0 tx=0).stop 2024-04-27T04:17:51.638 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:17:51.639+0000 7f1f14b07700 1 -- 172.21.15.162:0/714892422 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1f10101d10 msgr2=0x7f1f10113a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:17:51.638 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:17:51.639+0000 7f1f14b07700 1 --2- 172.21.15.162:0/714892422 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1f10101d10 0x7f1f10113a30 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f1f0000b790 tx=0x7f1f00009250 comp rx=0 tx=0).stop 2024-04-27T04:17:51.638 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:17:51.639+0000 7f1f14b07700 1 -- 172.21.15.162:0/714892422 shutdown_connections 2024-04-27T04:17:51.638 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:17:51.639+0000 7f1f14b07700 1 --2- 172.21.15.162:0/714892422 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1f10101d10 0x7f1f10113a30 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:17:51.638 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:17:51.639+0000 7f1f14b07700 1 --2- 172.21.15.162:0/714892422 >> [v2:172.21.15.162:6800/1396866108,v1:172.21.15.162:6801/1396866108] conn(0x7f1ef806ec00 0x7f1ef80710b0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:17:51.638 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:17:51.639+0000 7f1f14b07700 1 --2- 172.21.15.162:0/714892422 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1f101026b0 0x7f1f10113f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:17:51.638 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:17:51.639+0000 7f1f14b07700 1 -- 172.21.15.162:0/714892422 >> 172.21.15.162:0/714892422 conn(0x7f1f100f7860 msgr2=0x7f1f100f8ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:17:51.638 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:17:51.640+0000 7f1f14b07700 1 -- 172.21.15.162:0/714892422 shutdown_connections 2024-04-27T04:17:51.638 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:17:51.640+0000 7f1f14b07700 1 -- 172.21.15.162:0/714892422 wait complete. 2024-04-27T04:17:51.648 INFO:teuthology.orchestra.run.smithi162.stdout:true 2024-04-27T04:17:51.746 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:17:51.746+0000 7f152c390700 1 -- 172.21.15.162:0/3081710412 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1524101d90 msgr2=0x7f1524102160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:17:51.746 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:17:51.746+0000 7f152c390700 1 --2- 172.21.15.162:0/3081710412 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1524101d90 0x7f1524102160 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f1520004940 tx=0x7f1520009e30 comp rx=0 tx=0).stop 2024-04-27T04:17:51.746 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:17:51.747+0000 7f152c390700 1 -- 172.21.15.162:0/3081710412 shutdown_connections 2024-04-27T04:17:51.747 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:17:51.747+0000 7f152c390700 1 --2- 172.21.15.162:0/3081710412 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f15240fbd40 0x7f15240fc1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:17:51.747 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:17:51.747+0000 7f152c390700 1 --2- 172.21.15.162:0/3081710412 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1524101d90 0x7f1524102160 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:17:51.747 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:17:51.747+0000 7f152c390700 1 -- 172.21.15.162:0/3081710412 >> 172.21.15.162:0/3081710412 conn(0x7f1524071d00 msgr2=0x7f1524072100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:17:51.747 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:17:51.747+0000 7f152c390700 1 -- 172.21.15.162:0/3081710412 shutdown_connections 2024-04-27T04:17:51.747 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:17:51.747+0000 7f152c390700 1 -- 172.21.15.162:0/3081710412 wait complete. 2024-04-27T04:17:51.747 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:17:51.747+0000 7f152c390700 1 Processor -- start 2024-04-27T04:17:51.747 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:17:51.747+0000 7f152c390700 1 -- start start 2024-04-27T04:17:51.747 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:17:51.747+0000 7f152c390700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f15240fbd40 0x7f1524115cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:17:51.747 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:17:51.747+0000 7f152c390700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1524101d90 0x7f1524116200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:17:51.747 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:17:51.747+0000 7f152c390700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1524116850 con 0x7f1524101d90 2024-04-27T04:17:51.747 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:17:51.747+0000 7f152c390700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f152410fc20 con 0x7f15240fbd40 2024-04-27T04:17:51.748 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:17:51.747+0000 7f152992b700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1524101d90 0x7f1524116200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:17:51.748 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:17:51.747+0000 7f152992b700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1524101d90 0x7f1524116200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:47698/0 (socket says 172.21.15.162:47698) 2024-04-27T04:17:51.748 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:17:51.748+0000 7f152992b700 1 -- 172.21.15.162:0/461699048 learned_addr learned my addr 172.21.15.162:0/461699048 (peer_addr_for_me v2:172.21.15.162:0/0) 2024-04-27T04:17:51.748 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:17:51.748+0000 7f152a12c700 1 --2- 172.21.15.162:0/461699048 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f15240fbd40 0x7f1524115cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:17:51.748 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:17:51.748+0000 7f152992b700 1 -- 172.21.15.162:0/461699048 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f15240fbd40 msgr2=0x7f1524115cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:17:51.748 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:17:51.748+0000 7f152992b700 1 --2- 172.21.15.162:0/461699048 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f15240fbd40 0x7f1524115cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:17:51.748 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:17:51.748+0000 7f152992b700 1 -- 172.21.15.162:0/461699048 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1520005040 con 0x7f1524101d90 2024-04-27T04:17:51.748 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:17:51.748+0000 7f152992b700 1 --2- 172.21.15.162:0/461699048 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1524101d90 0x7f1524116200 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f151400eb40 tx=0x7f151400b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:17:51.748 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:17:51.748+0000 7f151affd700 1 -- 172.21.15.162:0/461699048 <== mon.0 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1514007cc0 con 0x7f1524101d90 2024-04-27T04:17:51.748 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:17:51.748+0000 7f152c390700 1 -- 172.21.15.162:0/461699048 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f152410fea0 con 0x7f1524101d90 2024-04-27T04:17:51.748 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:17:51.748+0000 7f151affd700 1 -- 172.21.15.162:0/461699048 <== mon.0 v2:172.21.15.162:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f151400f040 con 0x7f1524101d90 2024-04-27T04:17:51.749 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:17:51.748+0000 7f151affd700 1 -- 172.21.15.162:0/461699048 <== mon.0 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f151400ca10 con 0x7f1524101d90 2024-04-27T04:17:51.749 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:17:51.748+0000 7f152c390700 1 -- 172.21.15.162:0/461699048 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f15241103e0 con 0x7f1524101d90 2024-04-27T04:17:51.750 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:17:51.749+0000 7f152c390700 1 -- 172.21.15.162:0/461699048 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f15241061c0 con 0x7f1524101d90 2024-04-27T04:17:51.750 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:17:51.750+0000 7f151affd700 1 -- 172.21.15.162:0/461699048 <== mon.0 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 39) v1 ==== 92747+0+0 (secure 0 0 0) 0x7f151400cc80 con 0x7f1524101d90 2024-04-27T04:17:51.750 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:17:51.750+0000 7f151affd700 1 --2- 172.21.15.162:0/461699048 >> [v2:172.21.15.162:6800/1396866108,v1:172.21.15.162:6801/1396866108] conn(0x7f151006ec10 0x7f15100710c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:17:51.750 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:17:51.750+0000 7f151affd700 1 -- 172.21.15.162:0/461699048 <== mon.0 v2:172.21.15.162:3300/0 5 ==== osd_map(69..69 src has 1..69) v4 ==== 6410+0+0 (secure 0 0 0) 0x7f151408e420 con 0x7f1524101d90 2024-04-27T04:17:51.750 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:17:51.750+0000 7f152a12c700 1 --2- 172.21.15.162:0/461699048 >> [v2:172.21.15.162:6800/1396866108,v1:172.21.15.162:6801/1396866108] conn(0x7f151006ec10 0x7f15100710c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:17:51.750 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:17:51.751+0000 7f152a12c700 1 --2- 172.21.15.162:0/461699048 >> [v2:172.21.15.162:6800/1396866108,v1:172.21.15.162:6801/1396866108] conn(0x7f151006ec10 0x7f15100710c0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f152000b0d0 tx=0x7f1520009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:17:51.752 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:17:51.755+0000 7f151affd700 1 -- 172.21.15.162:0/461699048 <== mon.0 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7f151405ae00 con 0x7f1524101d90 2024-04-27T04:17:51.969 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:17:51.971+0000 7f152c390700 1 -- 172.21.15.162:0/461699048 --> [v2:172.21.15.162:6800/1396866108,v1:172.21.15.162:6801/1396866108] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f1524108df0 con 0x7f151006ec10 2024-04-27T04:17:51.969 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:17:51 smithi162 ceph-mon[110641]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "osd ok-to-stop", "ids": ["4"], "max": 16}]: dispatch 2024-04-27T04:17:51.970 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:17:51.973+0000 7f151affd700 1 -- 172.21.15.162:0/461699048 <== mgr.34104 v2:172.21.15.162:6800/1396866108 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+401 (secure 0 0 0) 0x7f1524108df0 con 0x7f151006ec10 2024-04-27T04:17:51.973 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:17:51.975+0000 7f152c390700 1 -- 172.21.15.162:0/461699048 >> [v2:172.21.15.162:6800/1396866108,v1:172.21.15.162:6801/1396866108] conn(0x7f151006ec10 msgr2=0x7f15100710c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:17:51.973 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:17:51.975+0000 7f152c390700 1 --2- 172.21.15.162:0/461699048 >> [v2:172.21.15.162:6800/1396866108,v1:172.21.15.162:6801/1396866108] conn(0x7f151006ec10 0x7f15100710c0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f152000b0d0 tx=0x7f1520009e30 comp rx=0 tx=0).stop 2024-04-27T04:17:51.973 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:17:51.975+0000 7f152c390700 1 -- 172.21.15.162:0/461699048 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1524101d90 msgr2=0x7f1524116200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:17:51.973 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:17:51.975+0000 7f152c390700 1 --2- 172.21.15.162:0/461699048 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1524101d90 0x7f1524116200 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f151400eb40 tx=0x7f151400b040 comp rx=0 tx=0).stop 2024-04-27T04:17:51.973 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:17:51.975+0000 7f152c390700 1 -- 172.21.15.162:0/461699048 shutdown_connections 2024-04-27T04:17:51.973 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:17:51.975+0000 7f152c390700 1 --2- 172.21.15.162:0/461699048 >> [v2:172.21.15.162:6800/1396866108,v1:172.21.15.162:6801/1396866108] conn(0x7f151006ec10 0x7f15100710c0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:17:51.973 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:17:51.975+0000 7f152c390700 1 --2- 172.21.15.162:0/461699048 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f15240fbd40 0x7f1524115cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:17:51.973 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:17:51.975+0000 7f152c390700 1 --2- 172.21.15.162:0/461699048 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1524101d90 0x7f1524116200 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:17:51.973 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:17:51.975+0000 7f152c390700 1 -- 172.21.15.162:0/461699048 >> 172.21.15.162:0/461699048 conn(0x7f1524071d00 msgr2=0x7f15240fb100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:17:51.973 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:17:51.975+0000 7f152c390700 1 -- 172.21.15.162:0/461699048 shutdown_connections 2024-04-27T04:17:51.973 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:17:51.975+0000 7f152c390700 1 -- 172.21.15.162:0/461699048 wait complete. 2024-04-27T04:17:52.075 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:17:52.077+0000 7efdb261f700 1 -- 172.21.15.162:0/1043282060 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7efdac103780 msgr2=0x7efdac0fcdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:17:52.075 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:17:52.077+0000 7efdb261f700 1 --2- 172.21.15.162:0/1043282060 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7efdac103780 0x7efdac0fcdf0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7efd9c004940 tx=0x7efd9c009e30 comp rx=0 tx=0).stop 2024-04-27T04:17:52.075 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:17:52.077+0000 7efdb261f700 1 -- 172.21.15.162:0/1043282060 shutdown_connections 2024-04-27T04:17:52.075 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:17:52.077+0000 7efdb261f700 1 --2- 172.21.15.162:0/1043282060 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7efdac103780 0x7efdac0fcdf0 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:17:52.076 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:17:52.077+0000 7efdb261f700 1 --2- 172.21.15.162:0/1043282060 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7efdac102de0 0x7efdac1031b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:17:52.076 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:17:52.077+0000 7efdb261f700 1 -- 172.21.15.162:0/1043282060 >> 172.21.15.162:0/1043282060 conn(0x7efdac0f8990 msgr2=0x7efdac0fada0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:17:52.076 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:17:52.077+0000 7efdb261f700 1 -- 172.21.15.162:0/1043282060 shutdown_connections 2024-04-27T04:17:52.076 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:17:52.077+0000 7efdb261f700 1 -- 172.21.15.162:0/1043282060 wait complete. 2024-04-27T04:17:52.076 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:17:52.078+0000 7efdb261f700 1 Processor -- start 2024-04-27T04:17:52.077 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:17:52.078+0000 7efdb261f700 1 -- start start 2024-04-27T04:17:52.077 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:17:52.078+0000 7efdb261f700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7efdac102de0 0x7efdac113900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:17:52.078 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:17:52.078+0000 7efdb261f700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7efdac103780 0x7efdac113e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:17:52.078 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:17:52.078+0000 7efdb261f700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efdac114490 con 0x7efdac103780 2024-04-27T04:17:52.078 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:17:52.078+0000 7efdb261f700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efdac10da90 con 0x7efdac102de0 2024-04-27T04:17:52.078 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:17:52.078+0000 7efdab7fe700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7efdac103780 0x7efdac113e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:17:52.078 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:17:52.078+0000 7efdab7fe700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7efdac103780 0x7efdac113e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:47716/0 (socket says 172.21.15.162:47716) 2024-04-27T04:17:52.078 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:17:52.078+0000 7efdab7fe700 1 -- 172.21.15.162:0/1616020440 learned_addr learned my addr 172.21.15.162:0/1616020440 (peer_addr_for_me v2:172.21.15.162:0/0) 2024-04-27T04:17:52.078 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:17:52.078+0000 7efdabfff700 1 --2- 172.21.15.162:0/1616020440 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7efdac102de0 0x7efdac113900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:17:52.078 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:17:52.079+0000 7efdab7fe700 1 -- 172.21.15.162:0/1616020440 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7efdac102de0 msgr2=0x7efdac113900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:17:52.078 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:17:52.079+0000 7efdab7fe700 1 --2- 172.21.15.162:0/1616020440 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7efdac102de0 0x7efdac113900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:17:52.078 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:17:52.079+0000 7efdab7fe700 1 -- 172.21.15.162:0/1616020440 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd9c005040 con 0x7efdac103780 2024-04-27T04:17:52.078 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:17:52.079+0000 7efdab7fe700 1 --2- 172.21.15.162:0/1616020440 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7efdac103780 0x7efdac113e40 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7efd9c000c00 tx=0x7efd9c009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:17:52.078 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:17:52.079+0000 7efda8ff9700 1 -- 172.21.15.162:0/1616020440 <== mon.0 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7efd9c021070 con 0x7efdac103780 2024-04-27T04:17:52.078 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:17:52.079+0000 7efdb261f700 1 -- 172.21.15.162:0/1616020440 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efdac10dd10 con 0x7efdac103780 2024-04-27T04:17:52.079 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:17:52.079+0000 7efda8ff9700 1 -- 172.21.15.162:0/1616020440 <== mon.0 v2:172.21.15.162:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7efd9c013c00 con 0x7efdac103780 2024-04-27T04:17:52.079 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:17:52.079+0000 7efda8ff9700 1 -- 172.21.15.162:0/1616020440 <== mon.0 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7efd9c00e070 con 0x7efdac103780 2024-04-27T04:17:52.079 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:17:52.079+0000 7efdb261f700 1 -- 172.21.15.162:0/1616020440 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efdac10e1f0 con 0x7efdac103780 2024-04-27T04:17:52.080 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:17:52.080+0000 7efdb261f700 1 -- 172.21.15.162:0/1616020440 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efdac0fe530 con 0x7efdac103780 2024-04-27T04:17:52.080 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:17:52.081+0000 7efda8ff9700 1 -- 172.21.15.162:0/1616020440 <== mon.0 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 39) v1 ==== 92747+0+0 (secure 0 0 0) 0x7efd9c01ca60 con 0x7efdac103780 2024-04-27T04:17:52.080 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:17:52.081+0000 7efda8ff9700 1 --2- 172.21.15.162:0/1616020440 >> [v2:172.21.15.162:6800/1396866108,v1:172.21.15.162:6801/1396866108] conn(0x7efd9406ec00 0x7efd940710b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:17:52.080 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:17:52.081+0000 7efda8ff9700 1 -- 172.21.15.162:0/1616020440 <== mon.0 v2:172.21.15.162:3300/0 5 ==== osd_map(69..69 src has 1..69) v4 ==== 6410+0+0 (secure 0 0 0) 0x7efd9c08fca0 con 0x7efdac103780 2024-04-27T04:17:52.080 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:17:52.081+0000 7efdabfff700 1 --2- 172.21.15.162:0/1616020440 >> [v2:172.21.15.162:6800/1396866108,v1:172.21.15.162:6801/1396866108] conn(0x7efd9406ec00 0x7efd940710b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:17:52.080 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:17:52.082+0000 7efdabfff700 1 --2- 172.21.15.162:0/1616020440 >> [v2:172.21.15.162:6800/1396866108,v1:172.21.15.162:6801/1396866108] conn(0x7efd9406ec00 0x7efd940710b0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7efda0009b80 tx=0x7efda000b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:17:52.083 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:17:52.086+0000 7efda8ff9700 1 -- 172.21.15.162:0/1616020440 <== mon.0 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7efd9c094070 con 0x7efdac103780 2024-04-27T04:17:52.131 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:17:51 smithi181 ceph-mon[81134]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "osd ok-to-stop", "ids": ["4"], "max": 16}]: dispatch 2024-04-27T04:17:52.293 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:17:52.295+0000 7efdb261f700 1 -- 172.21.15.162:0/1616020440 --> [v2:172.21.15.162:6800/1396866108,v1:172.21.15.162:6801/1396866108] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7efdac109ec0 con 0x7efd9406ec00 2024-04-27T04:17:52.303 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:17:52.305+0000 7efda8ff9700 1 -- 172.21.15.162:0/1616020440 <== mgr.34104 v2:172.21.15.162:6800/1396866108 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3498 (secure 0 0 0) 0x7efdac109ec0 con 0x7efd9406ec00 2024-04-27T04:17:52.303 INFO:teuthology.orchestra.run.smithi162.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-27T04:17:52.303 INFO:teuthology.orchestra.run.smithi162.stdout:alertmanager.smithi162 smithi162 *:9093,9094 running (5m) 54s ago 16m 27.2M - 0.25.0 c8568f914cd2 40e334659017 2024-04-27T04:17:52.303 INFO:teuthology.orchestra.run.smithi162.stdout:crash.smithi162 smithi162 running (2m) 54s ago 16m 6660k - 19.0.0-3253-g4e213a58 67e414ff4bc4 1aa8f4d5a28b 2024-04-27T04:17:52.303 INFO:teuthology.orchestra.run.smithi162.stdout:crash.smithi181 smithi181 running (2m) 20s ago 15m 6656k - 19.0.0-3253-g4e213a58 67e414ff4bc4 832d9cbc5c30 2024-04-27T04:17:52.303 INFO:teuthology.orchestra.run.smithi162.stdout:grafana.smithi162 smithi162 *:3000 running (4m) 54s ago 15m 80.5M - 9.4.12 2bacad6d85d8 3d78c478f03f 2024-04-27T04:17:52.304 INFO:teuthology.orchestra.run.smithi162.stdout:mds.cephfs.smithi162.gzlyjj smithi162 running (10m) 54s ago 10m 14.3M - 17.2.7-907-g2be3efa9 1c5834633715 27e5b030dd32 2024-04-27T04:17:52.304 INFO:teuthology.orchestra.run.smithi162.stdout:mds.cephfs.smithi162.xcnapk smithi162 running (10m) 54s ago 10m 100M - 17.2.7-907-g2be3efa9 1c5834633715 38f9030df029 2024-04-27T04:17:52.304 INFO:teuthology.orchestra.run.smithi162.stdout:mds.cephfs.smithi181.bkkcdm smithi181 running (10m) 20s ago 10m 15.8M - 17.2.7-907-g2be3efa9 1c5834633715 af55afca1263 2024-04-27T04:17:52.304 INFO:teuthology.orchestra.run.smithi162.stdout:mds.cephfs.smithi181.dzomgh smithi181 running (10m) 20s ago 10m 143M - 17.2.7-907-g2be3efa9 1c5834633715 6733296597f3 2024-04-27T04:17:52.304 INFO:teuthology.orchestra.run.smithi162.stdout:mgr.smithi162.oennfn smithi162 *:8443,9283,8765 running (6m) 54s ago 17m 505M - 19.0.0-3253-g4e213a58 67e414ff4bc4 58a748d3637b 2024-04-27T04:17:52.304 INFO:teuthology.orchestra.run.smithi162.stdout:mgr.smithi181.vtwuhz smithi181 *:8443,9283,8765 running (5m) 20s ago 15m 451M - 19.0.0-3253-g4e213a58 67e414ff4bc4 813584297b1d 2024-04-27T04:17:52.304 INFO:teuthology.orchestra.run.smithi162.stdout:mon.smithi162 smithi162 running (3m) 54s ago 17m 50.4M 2048M 19.0.0-3253-g4e213a58 67e414ff4bc4 375adaf4468f 2024-04-27T04:17:52.304 INFO:teuthology.orchestra.run.smithi162.stdout:mon.smithi181 smithi181 running (3m) 20s ago 15m 35.0M 2048M 19.0.0-3253-g4e213a58 67e414ff4bc4 77b71f6d522f 2024-04-27T04:17:52.304 INFO:teuthology.orchestra.run.smithi162.stdout:node-exporter.smithi162 smithi162 *:9100 running (5m) 54s ago 16m 15.4M - 1.5.0 0da6a335fe13 5c4c5ea9c1d7 2024-04-27T04:17:52.304 INFO:teuthology.orchestra.run.smithi162.stdout:node-exporter.smithi181 smithi181 *:9100 running (5m) 20s ago 15m 15.6M - 1.5.0 0da6a335fe13 9067f14c0519 2024-04-27T04:17:52.304 INFO:teuthology.orchestra.run.smithi162.stdout:osd.0 smithi162 running (2m) 54s ago 14m 87.7M 1557M 19.0.0-3253-g4e213a58 67e414ff4bc4 2744aff3cd88 2024-04-27T04:17:52.304 INFO:teuthology.orchestra.run.smithi162.stdout:osd.1 smithi162 running (95s) 54s ago 14m 93.8M 1557M 19.0.0-3253-g4e213a58 67e414ff4bc4 668e16faad62 2024-04-27T04:17:52.304 INFO:teuthology.orchestra.run.smithi162.stdout:osd.2 smithi162 running (59s) 54s ago 13m 15.1M 1557M 19.0.0-3253-g4e213a58 67e414ff4bc4 d24878d461e7 2024-04-27T04:17:52.304 INFO:teuthology.orchestra.run.smithi162.stdout:osd.3 smithi181 running (24s) 20s ago 13m 14.8M 2581M 19.0.0-3253-g4e213a58 67e414ff4bc4 ab473cfd54d2 2024-04-27T04:17:52.304 INFO:teuthology.orchestra.run.smithi162.stdout:osd.4 smithi181 running (12m) 20s ago 12m 248M 2581M 17.2.7-907-g2be3efa9 1c5834633715 512afbee7cf7 2024-04-27T04:17:52.305 INFO:teuthology.orchestra.run.smithi162.stdout:osd.5 smithi181 running (12m) 20s ago 12m 223M 2581M 17.2.7-907-g2be3efa9 1c5834633715 f47a1daac1ea 2024-04-27T04:17:52.305 INFO:teuthology.orchestra.run.smithi162.stdout:prometheus.smithi162 smithi162 *:9095 running (5m) 54s ago 15m 67.6M - 2.43.0 a07b618ecd1d f30657a43324 2024-04-27T04:17:52.305 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:17:52.307+0000 7efdb261f700 1 -- 172.21.15.162:0/1616020440 >> [v2:172.21.15.162:6800/1396866108,v1:172.21.15.162:6801/1396866108] conn(0x7efd9406ec00 msgr2=0x7efd940710b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:17:52.305 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:17:52.307+0000 7efdb261f700 1 --2- 172.21.15.162:0/1616020440 >> [v2:172.21.15.162:6800/1396866108,v1:172.21.15.162:6801/1396866108] conn(0x7efd9406ec00 0x7efd940710b0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7efda0009b80 tx=0x7efda000b040 comp rx=0 tx=0).stop 2024-04-27T04:17:52.306 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:17:52.307+0000 7efdb261f700 1 -- 172.21.15.162:0/1616020440 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7efdac103780 msgr2=0x7efdac113e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:17:52.306 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:17:52.307+0000 7efdb261f700 1 --2- 172.21.15.162:0/1616020440 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7efdac103780 0x7efdac113e40 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7efd9c000c00 tx=0x7efd9c009e30 comp rx=0 tx=0).stop 2024-04-27T04:17:52.306 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:17:52.307+0000 7efdb261f700 1 -- 172.21.15.162:0/1616020440 shutdown_connections 2024-04-27T04:17:52.306 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:17:52.307+0000 7efdb261f700 1 --2- 172.21.15.162:0/1616020440 >> [v2:172.21.15.162:6800/1396866108,v1:172.21.15.162:6801/1396866108] conn(0x7efd9406ec00 0x7efd940710b0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:17:52.306 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:17:52.307+0000 7efdb261f700 1 --2- 172.21.15.162:0/1616020440 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7efdac102de0 0x7efdac113900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:17:52.306 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:17:52.307+0000 7efdb261f700 1 --2- 172.21.15.162:0/1616020440 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7efdac103780 0x7efdac113e40 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:17:52.306 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:17:52.307+0000 7efdb261f700 1 -- 172.21.15.162:0/1616020440 >> 172.21.15.162:0/1616020440 conn(0x7efdac0f8990 msgr2=0x7efdac0f9880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:17:52.306 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:17:52.308+0000 7efdb261f700 1 -- 172.21.15.162:0/1616020440 shutdown_connections 2024-04-27T04:17:52.306 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:17:52.308+0000 7efdb261f700 1 -- 172.21.15.162:0/1616020440 wait complete. 2024-04-27T04:17:52.407 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:17:52.408+0000 7f8937cee700 1 -- 172.21.15.162:0/2863250481 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f89300fbd40 msgr2=0x7f89300fc1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:17:52.407 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:17:52.408+0000 7f8937cee700 1 --2- 172.21.15.162:0/2863250481 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f89300fbd40 0x7f89300fc1b0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f8920004940 tx=0x7f8920009e30 comp rx=0 tx=0).stop 2024-04-27T04:17:52.407 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:17:52.408+0000 7f8937cee700 1 -- 172.21.15.162:0/2863250481 shutdown_connections 2024-04-27T04:17:52.407 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:17:52.408+0000 7f8937cee700 1 --2- 172.21.15.162:0/2863250481 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f89300fbd40 0x7f89300fc1b0 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:17:52.408 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:17:52.408+0000 7f8937cee700 1 --2- 172.21.15.162:0/2863250481 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8930101d90 0x7f8930102160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:17:52.408 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:17:52.408+0000 7f8937cee700 1 -- 172.21.15.162:0/2863250481 >> 172.21.15.162:0/2863250481 conn(0x7f8930071d00 msgr2=0x7f8930072100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:17:52.408 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:17:52.408+0000 7f8937cee700 1 -- 172.21.15.162:0/2863250481 shutdown_connections 2024-04-27T04:17:52.408 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:17:52.408+0000 7f8937cee700 1 -- 172.21.15.162:0/2863250481 wait complete. 2024-04-27T04:17:52.408 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:17:52.408+0000 7f8937cee700 1 Processor -- start 2024-04-27T04:17:52.408 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:17:52.408+0000 7f8937cee700 1 -- start start 2024-04-27T04:17:52.408 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:17:52.409+0000 7f8937cee700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f89300fbd40 0x7f893010eaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:17:52.408 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:17:52.409+0000 7f8937cee700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8930101d90 0x7f893010efe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:17:52.408 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:17:52.409+0000 7f8937cee700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f893010f630 con 0x7f89300fbd40 2024-04-27T04:17:52.408 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:17:52.409+0000 7f8937cee700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f893010f770 con 0x7f8930101d90 2024-04-27T04:17:52.408 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:17:52.409+0000 7f8935a8a700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f89300fbd40 0x7f893010eaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:17:52.408 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:17:52.409+0000 7f8935a8a700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f89300fbd40 0x7f893010eaa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:47746/0 (socket says 172.21.15.162:47746) 2024-04-27T04:17:52.408 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:17:52.409+0000 7f8935a8a700 1 -- 172.21.15.162:0/3671935529 learned_addr learned my addr 172.21.15.162:0/3671935529 (peer_addr_for_me v2:172.21.15.162:0/0) 2024-04-27T04:17:52.408 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:17:52.409+0000 7f8935289700 1 --2- 172.21.15.162:0/3671935529 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8930101d90 0x7f893010efe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:17:52.409 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:17:52.409+0000 7f8935a8a700 1 -- 172.21.15.162:0/3671935529 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8930101d90 msgr2=0x7f893010efe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:17:52.409 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:17:52.409+0000 7f8935a8a700 1 --2- 172.21.15.162:0/3671935529 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8930101d90 0x7f893010efe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:17:52.409 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:17:52.409+0000 7f8935a8a700 1 -- 172.21.15.162:0/3671935529 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8920005040 con 0x7f89300fbd40 2024-04-27T04:17:52.409 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:17:52.409+0000 7f8935289700 1 --2- 172.21.15.162:0/3671935529 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8930101d90 0x7f893010efe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-27T04:17:52.409 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:17:52.409+0000 7f8935a8a700 1 --2- 172.21.15.162:0/3671935529 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f89300fbd40 0x7f893010eaa0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f892c00eb40 tx=0x7f892c00b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:17:52.409 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:17:52.409+0000 7f89267fc700 1 -- 172.21.15.162:0/3671935529 <== mon.0 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f892c007cc0 con 0x7f89300fbd40 2024-04-27T04:17:52.409 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:17:52.409+0000 7f89267fc700 1 -- 172.21.15.162:0/3671935529 <== mon.0 v2:172.21.15.162:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f892c00f040 con 0x7f89300fbd40 2024-04-27T04:17:52.409 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:17:52.409+0000 7f8937cee700 1 -- 172.21.15.162:0/3671935529 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8930117b10 con 0x7f89300fbd40 2024-04-27T04:17:52.409 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:17:52.409+0000 7f89267fc700 1 -- 172.21.15.162:0/3671935529 <== mon.0 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f892c009620 con 0x7f89300fbd40 2024-04-27T04:17:52.409 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:17:52.409+0000 7f8937cee700 1 -- 172.21.15.162:0/3671935529 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8930117fa0 con 0x7f89300fbd40 2024-04-27T04:17:52.410 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:17:52.410+0000 7f8937cee700 1 -- 172.21.15.162:0/3671935529 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f89300fd430 con 0x7f89300fbd40 2024-04-27T04:17:52.410 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:17:52.411+0000 7f89267fc700 1 -- 172.21.15.162:0/3671935529 <== mon.0 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 39) v1 ==== 92747+0+0 (secure 0 0 0) 0x7f892c009780 con 0x7f89300fbd40 2024-04-27T04:17:52.410 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:17:52.411+0000 7f89267fc700 1 --2- 172.21.15.162:0/3671935529 >> [v2:172.21.15.162:6800/1396866108,v1:172.21.15.162:6801/1396866108] conn(0x7f891c06ec00 0x7f891c0710b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:17:52.410 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:17:52.411+0000 7f89267fc700 1 -- 172.21.15.162:0/3671935529 <== mon.0 v2:172.21.15.162:3300/0 5 ==== osd_map(69..69 src has 1..69) v4 ==== 6410+0+0 (secure 0 0 0) 0x7f892c08e4f0 con 0x7f89300fbd40 2024-04-27T04:17:52.410 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:17:52.412+0000 7f8935289700 1 --2- 172.21.15.162:0/3671935529 >> [v2:172.21.15.162:6800/1396866108,v1:172.21.15.162:6801/1396866108] conn(0x7f891c06ec00 0x7f891c0710b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:17:52.410 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:17:52.412+0000 7f8935289700 1 --2- 172.21.15.162:0/3671935529 >> [v2:172.21.15.162:6800/1396866108,v1:172.21.15.162:6801/1396866108] conn(0x7f891c06ec00 0x7f891c0710b0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f8930065fe0 tx=0x7f8920009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:17:52.414 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:17:52.416+0000 7f89267fc700 1 -- 172.21.15.162:0/3671935529 <== mon.0 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7f892c0573a0 con 0x7f89300fbd40 2024-04-27T04:17:52.699 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:17:52.701+0000 7f8937cee700 1 -- 172.21.15.162:0/3671935529 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f893004bf60 con 0x7f89300fbd40 2024-04-27T04:17:52.700 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:17:52.702+0000 7f89267fc700 1 -- 172.21.15.162:0/3671935529 <== mon.0 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+839 (secure 0 0 0) 0x7f892c05aa60 con 0x7f89300fbd40 2024-04-27T04:17:52.700 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-04-27T04:17:52.700 INFO:teuthology.orchestra.run.smithi162.stdout: "mon": { 2024-04-27T04:17:52.700 INFO:teuthology.orchestra.run.smithi162.stdout: "ceph version 19.0.0-3253-g4e213a58 (4e213a58e787437ff508e6c890ccd5fd32dc96b5) squid (dev)": 2 2024-04-27T04:17:52.700 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-04-27T04:17:52.700 INFO:teuthology.orchestra.run.smithi162.stdout: "mgr": { 2024-04-27T04:17:52.700 INFO:teuthology.orchestra.run.smithi162.stdout: "ceph version 19.0.0-3253-g4e213a58 (4e213a58e787437ff508e6c890ccd5fd32dc96b5) squid (dev)": 2 2024-04-27T04:17:52.700 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-04-27T04:17:52.700 INFO:teuthology.orchestra.run.smithi162.stdout: "osd": { 2024-04-27T04:17:52.701 INFO:teuthology.orchestra.run.smithi162.stdout: "ceph version 17.2.7-907-g2be3efa9 (2be3efa97a7972f8e925bd899b474329979b74d6) quincy (stable)": 2, 2024-04-27T04:17:52.701 INFO:teuthology.orchestra.run.smithi162.stdout: "ceph version 19.0.0-3253-g4e213a58 (4e213a58e787437ff508e6c890ccd5fd32dc96b5) squid (dev)": 4 2024-04-27T04:17:52.701 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-04-27T04:17:52.701 INFO:teuthology.orchestra.run.smithi162.stdout: "mds": { 2024-04-27T04:17:52.701 INFO:teuthology.orchestra.run.smithi162.stdout: "ceph version 17.2.7-907-g2be3efa9 (2be3efa97a7972f8e925bd899b474329979b74d6) quincy (stable)": 4 2024-04-27T04:17:52.701 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-04-27T04:17:52.701 INFO:teuthology.orchestra.run.smithi162.stdout: "overall": { 2024-04-27T04:17:52.701 INFO:teuthology.orchestra.run.smithi162.stdout: "ceph version 17.2.7-907-g2be3efa9 (2be3efa97a7972f8e925bd899b474329979b74d6) quincy (stable)": 6, 2024-04-27T04:17:52.701 INFO:teuthology.orchestra.run.smithi162.stdout: "ceph version 19.0.0-3253-g4e213a58 (4e213a58e787437ff508e6c890ccd5fd32dc96b5) squid (dev)": 8 2024-04-27T04:17:52.701 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-04-27T04:17:52.701 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-04-27T04:17:52.702 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:17:52.704+0000 7f8937cee700 1 -- 172.21.15.162:0/3671935529 >> [v2:172.21.15.162:6800/1396866108,v1:172.21.15.162:6801/1396866108] conn(0x7f891c06ec00 msgr2=0x7f891c0710b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:17:52.702 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:17:52.704+0000 7f8937cee700 1 --2- 172.21.15.162:0/3671935529 >> [v2:172.21.15.162:6800/1396866108,v1:172.21.15.162:6801/1396866108] conn(0x7f891c06ec00 0x7f891c0710b0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f8930065fe0 tx=0x7f8920009e30 comp rx=0 tx=0).stop 2024-04-27T04:17:52.702 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:17:52.704+0000 7f8937cee700 1 -- 172.21.15.162:0/3671935529 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f89300fbd40 msgr2=0x7f893010eaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:17:52.702 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:17:52.704+0000 7f8937cee700 1 --2- 172.21.15.162:0/3671935529 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f89300fbd40 0x7f893010eaa0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f892c00eb40 tx=0x7f892c00b040 comp rx=0 tx=0).stop 2024-04-27T04:17:52.703 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:17:52.704+0000 7f8937cee700 1 -- 172.21.15.162:0/3671935529 shutdown_connections 2024-04-27T04:17:52.703 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:17:52.704+0000 7f8937cee700 1 --2- 172.21.15.162:0/3671935529 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f89300fbd40 0x7f893010eaa0 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:17:52.703 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:17:52.704+0000 7f8937cee700 1 --2- 172.21.15.162:0/3671935529 >> [v2:172.21.15.162:6800/1396866108,v1:172.21.15.162:6801/1396866108] conn(0x7f891c06ec00 0x7f891c0710b0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:17:52.703 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:17:52.704+0000 7f8937cee700 1 --2- 172.21.15.162:0/3671935529 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8930101d90 0x7f893010efe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:17:52.703 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:17:52.704+0000 7f8937cee700 1 -- 172.21.15.162:0/3671935529 >> 172.21.15.162:0/3671935529 conn(0x7f8930071d00 msgr2=0x7f89300fa0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:17:52.703 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:17:52.704+0000 7f8937cee700 1 -- 172.21.15.162:0/3671935529 shutdown_connections 2024-04-27T04:17:52.703 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:17:52.704+0000 7f8937cee700 1 -- 172.21.15.162:0/3671935529 wait complete. 2024-04-27T04:17:52.805 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:17:52.805+0000 7f2f23d2a700 1 -- 172.21.15.162:0/896083480 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2f1c102de0 msgr2=0x7f2f1c1031b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:17:52.805 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:17:52.805+0000 7f2f23d2a700 1 --2- 172.21.15.162:0/896083480 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2f1c102de0 0x7f2f1c1031b0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f2f18004940 tx=0x7f2f18009e30 comp rx=0 tx=0).stop 2024-04-27T04:17:52.805 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:17:52.805+0000 7f2f23d2a700 1 -- 172.21.15.162:0/896083480 shutdown_connections 2024-04-27T04:17:52.805 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:17:52.805+0000 7f2f23d2a700 1 --2- 172.21.15.162:0/896083480 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2f1c103780 0x7f2f1c0fcdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:17:52.805 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:17:52.805+0000 7f2f23d2a700 1 --2- 172.21.15.162:0/896083480 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2f1c102de0 0x7f2f1c1031b0 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:17:52.805 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:17:52.805+0000 7f2f23d2a700 1 -- 172.21.15.162:0/896083480 >> 172.21.15.162:0/896083480 conn(0x7f2f1c0f8990 msgr2=0x7f2f1c0fada0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:17:52.805 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:17:52.805+0000 7f2f23d2a700 1 -- 172.21.15.162:0/896083480 shutdown_connections 2024-04-27T04:17:52.806 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:17:52.805+0000 7f2f23d2a700 1 -- 172.21.15.162:0/896083480 wait complete. 2024-04-27T04:17:52.806 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:17:52.806+0000 7f2f23d2a700 1 Processor -- start 2024-04-27T04:17:52.806 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:17:52.806+0000 7f2f23d2a700 1 -- start start 2024-04-27T04:17:52.806 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:17:52.806+0000 7f2f23d2a700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2f1c102de0 0x7f2f1c1138e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:17:52.806 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:17:52.806+0000 7f2f23d2a700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2f1c103780 0x7f2f1c113e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:17:52.806 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:17:52.806+0000 7f2f23d2a700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f1c114470 con 0x7f2f1c103780 2024-04-27T04:17:52.806 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:17:52.806+0000 7f2f23d2a700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f1c10da70 con 0x7f2f1c102de0 2024-04-27T04:17:52.806 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:17:52.806+0000 7f2f212c5700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2f1c103780 0x7f2f1c113e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:17:52.806 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:17:52.806+0000 7f2f212c5700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2f1c103780 0x7f2f1c113e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:47764/0 (socket says 172.21.15.162:47764) 2024-04-27T04:17:52.806 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:17:52.806+0000 7f2f212c5700 1 -- 172.21.15.162:0/2234611580 learned_addr learned my addr 172.21.15.162:0/2234611580 (peer_addr_for_me v2:172.21.15.162:0/0) 2024-04-27T04:17:52.806 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:17:52.806+0000 7f2f21ac6700 1 --2- 172.21.15.162:0/2234611580 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2f1c102de0 0x7f2f1c1138e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:17:52.806 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:17:52.807+0000 7f2f212c5700 1 -- 172.21.15.162:0/2234611580 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2f1c102de0 msgr2=0x7f2f1c1138e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:17:52.806 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:17:52.807+0000 7f2f212c5700 1 --2- 172.21.15.162:0/2234611580 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2f1c102de0 0x7f2f1c1138e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:17:52.806 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:17:52.807+0000 7f2f212c5700 1 -- 172.21.15.162:0/2234611580 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f18005040 con 0x7f2f1c103780 2024-04-27T04:17:52.807 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:17:52.807+0000 7f2f212c5700 1 --2- 172.21.15.162:0/2234611580 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2f1c103780 0x7f2f1c113e20 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f2f0c00eb40 tx=0x7f2f0c00b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:17:52.807 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:17:52.807+0000 7f2f127fc700 1 -- 172.21.15.162:0/2234611580 <== mon.0 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2f0c007cc0 con 0x7f2f1c103780 2024-04-27T04:17:52.807 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:17:52.807+0000 7f2f23d2a700 1 -- 172.21.15.162:0/2234611580 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2f1c10dcf0 con 0x7f2f1c103780 2024-04-27T04:17:52.807 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:17:52.807+0000 7f2f127fc700 1 -- 172.21.15.162:0/2234611580 <== mon.0 v2:172.21.15.162:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f2f0c00f040 con 0x7f2f1c103780 2024-04-27T04:17:52.807 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:17:52.807+0000 7f2f127fc700 1 -- 172.21.15.162:0/2234611580 <== mon.0 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2f0c00c9d0 con 0x7f2f1c103780 2024-04-27T04:17:52.807 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:17:52.807+0000 7f2f23d2a700 1 -- 172.21.15.162:0/2234611580 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2f1c10e230 con 0x7f2f1c103780 2024-04-27T04:17:52.808 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:17:52.808+0000 7f2f23d2a700 1 -- 172.21.15.162:0/2234611580 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2f1c1000f0 con 0x7f2f1c103780 2024-04-27T04:17:52.808 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:17:52.808+0000 7f2f127fc700 1 -- 172.21.15.162:0/2234611580 <== mon.0 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 39) v1 ==== 92747+0+0 (secure 0 0 0) 0x7f2f0c00cc40 con 0x7f2f1c103780 2024-04-27T04:17:52.808 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:17:52.809+0000 7f2f127fc700 1 --2- 172.21.15.162:0/2234611580 >> [v2:172.21.15.162:6800/1396866108,v1:172.21.15.162:6801/1396866108] conn(0x7f2f0806ec00 0x7f2f080710b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:17:52.808 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:17:52.809+0000 7f2f127fc700 1 -- 172.21.15.162:0/2234611580 <== mon.0 v2:172.21.15.162:3300/0 5 ==== osd_map(69..69 src has 1..69) v4 ==== 6410+0+0 (secure 0 0 0) 0x7f2f0c08e2b0 con 0x7f2f1c103780 2024-04-27T04:17:52.808 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:17:52.809+0000 7f2f21ac6700 1 --2- 172.21.15.162:0/2234611580 >> [v2:172.21.15.162:6800/1396866108,v1:172.21.15.162:6801/1396866108] conn(0x7f2f0806ec00 0x7f2f080710b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:17:52.808 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:17:52.809+0000 7f2f21ac6700 1 --2- 172.21.15.162:0/2234611580 >> [v2:172.21.15.162:6800/1396866108,v1:172.21.15.162:6801/1396866108] conn(0x7f2f0806ec00 0x7f2f080710b0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f2f18007f50 tx=0x7f2f18009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:17:52.811 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:17:52.814+0000 7f2f127fc700 1 -- 172.21.15.162:0/2234611580 <== mon.0 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7f2f0c05ac90 con 0x7f2f1c103780 2024-04-27T04:17:53.030 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:17:52 smithi181 ceph-mon[81134]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["4"], "max": 16}]: dispatch 2024-04-27T04:17:53.030 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:17:52 smithi181 ceph-mon[81134]: Upgrade: osd.4 is safe to restart 2024-04-27T04:17:53.030 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:17:52 smithi181 ceph-mon[81134]: from='client.34314 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-27T04:17:53.030 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:17:52 smithi181 ceph-mon[81134]: pgmap v130: 65 pgs: 65 active+clean; 208 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:17:53.052 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:17:53.054+0000 7f2f23d2a700 1 -- 172.21.15.162:0/2234611580 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f2f1c0fe530 con 0x7f2f1c103780 2024-04-27T04:17:53.052 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:17:52 smithi162 ceph-mon[110641]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["4"], "max": 16}]: dispatch 2024-04-27T04:17:53.052 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:17:52 smithi162 ceph-mon[110641]: Upgrade: osd.4 is safe to restart 2024-04-27T04:17:53.053 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:17:52 smithi162 ceph-mon[110641]: from='client.34314 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-27T04:17:53.053 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:17:52 smithi162 ceph-mon[110641]: pgmap v130: 65 pgs: 65 active+clean; 208 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:17:53.053 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:17:53.055+0000 7f2f127fc700 1 -- 172.21.15.162:0/2234611580 <== mon.0 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 15 v15) v1 ==== 76+0+1911 (secure 0 0 0) 0x7f2f0c05a820 con 0x7f2f1c103780 2024-04-27T04:17:53.053 INFO:teuthology.orchestra.run.smithi162.stdout:e15 2024-04-27T04:17:53.053 INFO:teuthology.orchestra.run.smithi162.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-04-27T04:17:53.053 INFO:teuthology.orchestra.run.smithi162.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-27T04:17:53.053 INFO:teuthology.orchestra.run.smithi162.stdout:legacy client fscid: 1 2024-04-27T04:17:53.053 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-04-27T04:17:53.054 INFO:teuthology.orchestra.run.smithi162.stdout:Filesystem 'cephfs' (1) 2024-04-27T04:17:53.054 INFO:teuthology.orchestra.run.smithi162.stdout:fs_name cephfs 2024-04-27T04:17:53.054 INFO:teuthology.orchestra.run.smithi162.stdout:epoch 15 2024-04-27T04:17:53.054 INFO:teuthology.orchestra.run.smithi162.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-04-27T04:17:53.054 INFO:teuthology.orchestra.run.smithi162.stdout:created 2024-04-27T04:06:53.095749+0000 2024-04-27T04:17:53.054 INFO:teuthology.orchestra.run.smithi162.stdout:modified 2024-04-27T04:07:11.285033+0000 2024-04-27T04:17:53.054 INFO:teuthology.orchestra.run.smithi162.stdout:tableserver 0 2024-04-27T04:17:53.054 INFO:teuthology.orchestra.run.smithi162.stdout:root 0 2024-04-27T04:17:53.054 INFO:teuthology.orchestra.run.smithi162.stdout:session_timeout 60 2024-04-27T04:17:53.054 INFO:teuthology.orchestra.run.smithi162.stdout:session_autoclose 300 2024-04-27T04:17:53.054 INFO:teuthology.orchestra.run.smithi162.stdout:max_file_size 1099511627776 2024-04-27T04:17:53.054 INFO:teuthology.orchestra.run.smithi162.stdout:max_xattr_size 65536 2024-04-27T04:17:53.054 INFO:teuthology.orchestra.run.smithi162.stdout:required_client_features {} 2024-04-27T04:17:53.055 INFO:teuthology.orchestra.run.smithi162.stdout:last_failure 0 2024-04-27T04:17:53.055 INFO:teuthology.orchestra.run.smithi162.stdout:last_failure_osd_epoch 46 2024-04-27T04:17:53.055 INFO:teuthology.orchestra.run.smithi162.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-27T04:17:53.055 INFO:teuthology.orchestra.run.smithi162.stdout:max_mds 1 2024-04-27T04:17:53.055 INFO:teuthology.orchestra.run.smithi162.stdout:in 0 2024-04-27T04:17:53.055 INFO:teuthology.orchestra.run.smithi162.stdout:up {0=24273} 2024-04-27T04:17:53.055 INFO:teuthology.orchestra.run.smithi162.stdout:failed 2024-04-27T04:17:53.055 INFO:teuthology.orchestra.run.smithi162.stdout:damaged 2024-04-27T04:17:53.055 INFO:teuthology.orchestra.run.smithi162.stdout:stopped 2024-04-27T04:17:53.055 INFO:teuthology.orchestra.run.smithi162.stdout:data_pools [3] 2024-04-27T04:17:53.055 INFO:teuthology.orchestra.run.smithi162.stdout:metadata_pool 2 2024-04-27T04:17:53.055 INFO:teuthology.orchestra.run.smithi162.stdout:inline_data disabled 2024-04-27T04:17:53.055 INFO:teuthology.orchestra.run.smithi162.stdout:balancer 2024-04-27T04:17:53.055 INFO:teuthology.orchestra.run.smithi162.stdout:bal_rank_mask -1 2024-04-27T04:17:53.055 INFO:teuthology.orchestra.run.smithi162.stdout:standby_count_wanted 1 2024-04-27T04:17:53.055 INFO:teuthology.orchestra.run.smithi162.stdout:qdb_cluster leader: 0 members: 2024-04-27T04:17:53.056 INFO:teuthology.orchestra.run.smithi162.stdout:[mds.cephfs.smithi181.dzomgh{0:24273} state up:active seq 6 join_fscid=1 addr [v2:172.21.15.181:6824/658214993,v1:172.21.15.181:6825/658214993] compat {c=[1],r=[1],i=[7ff]}] 2024-04-27T04:17:53.056 INFO:teuthology.orchestra.run.smithi162.stdout:[mds.cephfs.smithi162.xcnapk{0:14476} state up:standby-replay seq 2 join_fscid=1 addr [v2:172.21.15.162:6828/1296160676,v1:172.21.15.162:6829/1296160676] compat {c=[1],r=[1],i=[7ff]}] 2024-04-27T04:17:53.056 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-04-27T04:17:53.056 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-04-27T04:17:53.056 INFO:teuthology.orchestra.run.smithi162.stdout:Standby daemons: 2024-04-27T04:17:53.056 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-04-27T04:17:53.056 INFO:teuthology.orchestra.run.smithi162.stdout:[mds.cephfs.smithi162.gzlyjj{-1:14488} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.162:6826/2629103894,v1:172.21.15.162:6827/2629103894] compat {c=[1],r=[1],i=[7ff]}] 2024-04-27T04:17:53.056 INFO:teuthology.orchestra.run.smithi162.stdout:[mds.cephfs.smithi181.bkkcdm{-1:24287} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.181:6826/3756475453,v1:172.21.15.181:6827/3756475453] compat {c=[1],r=[1],i=[7ff]}] 2024-04-27T04:17:53.057 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:17:53.057+0000 7f2f23d2a700 1 -- 172.21.15.162:0/2234611580 >> [v2:172.21.15.162:6800/1396866108,v1:172.21.15.162:6801/1396866108] conn(0x7f2f0806ec00 msgr2=0x7f2f080710b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:17:53.057 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:17:53.057+0000 7f2f23d2a700 1 --2- 172.21.15.162:0/2234611580 >> [v2:172.21.15.162:6800/1396866108,v1:172.21.15.162:6801/1396866108] conn(0x7f2f0806ec00 0x7f2f080710b0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f2f18007f50 tx=0x7f2f18009e30 comp rx=0 tx=0).stop 2024-04-27T04:17:53.057 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:17:53.057+0000 7f2f23d2a700 1 -- 172.21.15.162:0/2234611580 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2f1c103780 msgr2=0x7f2f1c113e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:17:53.057 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:17:53.057+0000 7f2f23d2a700 1 --2- 172.21.15.162:0/2234611580 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2f1c103780 0x7f2f1c113e20 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f2f0c00eb40 tx=0x7f2f0c00b040 comp rx=0 tx=0).stop 2024-04-27T04:17:53.057 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:17:53.057+0000 7f2f23d2a700 1 -- 172.21.15.162:0/2234611580 shutdown_connections 2024-04-27T04:17:53.057 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:17:53.057+0000 7f2f23d2a700 1 --2- 172.21.15.162:0/2234611580 >> [v2:172.21.15.162:6800/1396866108,v1:172.21.15.162:6801/1396866108] conn(0x7f2f0806ec00 0x7f2f080710b0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:17:53.057 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:17:53.057+0000 7f2f23d2a700 1 --2- 172.21.15.162:0/2234611580 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2f1c102de0 0x7f2f1c1138e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:17:53.057 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:17:53.057+0000 7f2f23d2a700 1 --2- 172.21.15.162:0/2234611580 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2f1c103780 0x7f2f1c113e20 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:17:53.057 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:17:53.057+0000 7f2f23d2a700 1 -- 172.21.15.162:0/2234611580 >> 172.21.15.162:0/2234611580 conn(0x7f2f1c0f8990 msgr2=0x7f2f1c0fa6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:17:53.058 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:17:53.058+0000 7f2f23d2a700 1 -- 172.21.15.162:0/2234611580 shutdown_connections 2024-04-27T04:17:53.058 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:17:53.058+0000 7f2f23d2a700 1 -- 172.21.15.162:0/2234611580 wait complete. 2024-04-27T04:17:53.058 INFO:teuthology.orchestra.run.smithi162.stderr:dumped fsmap epoch 15 2024-04-27T04:17:53.161 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:17:53.163+0000 7efe0e416700 1 -- 172.21.15.162:0/3891368080 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7efe08102d80 msgr2=0x7efe08103150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:17:53.161 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:17:53.163+0000 7efe0e416700 1 --2- 172.21.15.162:0/3891368080 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7efe08102d80 0x7efe08103150 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7efdf8004940 tx=0x7efdf8009e30 comp rx=0 tx=0).stop 2024-04-27T04:17:53.162 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:17:53.163+0000 7efe0e416700 1 -- 172.21.15.162:0/3891368080 shutdown_connections 2024-04-27T04:17:53.162 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:17:53.163+0000 7efe0e416700 1 --2- 172.21.15.162:0/3891368080 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7efe08103720 0x7efe080fcd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:17:53.162 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:17:53.163+0000 7efe0e416700 1 --2- 172.21.15.162:0/3891368080 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7efe08102d80 0x7efe08103150 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:17:53.162 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:17:53.163+0000 7efe0e416700 1 -- 172.21.15.162:0/3891368080 >> 172.21.15.162:0/3891368080 conn(0x7efe080f8950 msgr2=0x7efe080fad60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:17:53.162 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:17:53.163+0000 7efe0e416700 1 -- 172.21.15.162:0/3891368080 shutdown_connections 2024-04-27T04:17:53.162 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:17:53.163+0000 7efe0e416700 1 -- 172.21.15.162:0/3891368080 wait complete. 2024-04-27T04:17:53.162 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:17:53.164+0000 7efe0e416700 1 Processor -- start 2024-04-27T04:17:53.162 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:17:53.164+0000 7efe0e416700 1 -- start start 2024-04-27T04:17:53.164 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:17:53.164+0000 7efe0e416700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7efe08102d80 0x7efe08113ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:17:53.164 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:17:53.164+0000 7efe0e416700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7efe08103720 0x7efe08113ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:17:53.164 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:17:53.164+0000 7efe0e416700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe08114640 con 0x7efe08103720 2024-04-27T04:17:53.164 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:17:53.164+0000 7efe0e416700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe0810da30 con 0x7efe08102d80 2024-04-27T04:17:53.164 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:17:53.164+0000 7efe077fe700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7efe08103720 0x7efe08113ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:17:53.164 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:17:53.164+0000 7efe077fe700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7efe08103720 0x7efe08113ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:47780/0 (socket says 172.21.15.162:47780) 2024-04-27T04:17:53.164 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:17:53.164+0000 7efe077fe700 1 -- 172.21.15.162:0/3445483250 learned_addr learned my addr 172.21.15.162:0/3445483250 (peer_addr_for_me v2:172.21.15.162:0/0) 2024-04-27T04:17:53.164 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:17:53.164+0000 7efe07fff700 1 --2- 172.21.15.162:0/3445483250 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7efe08102d80 0x7efe08113ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:17:53.164 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:17:53.164+0000 7efe077fe700 1 -- 172.21.15.162:0/3445483250 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7efe08102d80 msgr2=0x7efe08113ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:17:53.164 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:17:53.164+0000 7efe077fe700 1 --2- 172.21.15.162:0/3445483250 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7efe08102d80 0x7efe08113ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:17:53.164 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:17:53.164+0000 7efe077fe700 1 -- 172.21.15.162:0/3445483250 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efdf8005040 con 0x7efe08103720 2024-04-27T04:17:53.164 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:17:53.165+0000 7efe077fe700 1 --2- 172.21.15.162:0/3445483250 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7efe08103720 0x7efe08113ff0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7efdfc00eb40 tx=0x7efdfc00b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:17:53.165 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:17:53.165+0000 7efe04ff9700 1 -- 172.21.15.162:0/3445483250 <== mon.0 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7efdfc007cc0 con 0x7efe08103720 2024-04-27T04:17:53.165 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:17:53.165+0000 7efe04ff9700 1 -- 172.21.15.162:0/3445483250 <== mon.0 v2:172.21.15.162:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7efdfc00f040 con 0x7efe08103720 2024-04-27T04:17:53.165 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:17:53.165+0000 7efe0e416700 1 -- 172.21.15.162:0/3445483250 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efe0810dcb0 con 0x7efe08103720 2024-04-27T04:17:53.165 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:17:53.165+0000 7efe04ff9700 1 -- 172.21.15.162:0/3445483250 <== mon.0 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7efdfc00c9d0 con 0x7efe08103720 2024-04-27T04:17:53.165 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:17:53.165+0000 7efe0e416700 1 -- 172.21.15.162:0/3445483250 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efe0810e1f0 con 0x7efe08103720 2024-04-27T04:17:53.165 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:17:53.166+0000 7efe0e416700 1 -- 172.21.15.162:0/3445483250 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efe080fe4d0 con 0x7efe08103720 2024-04-27T04:17:53.166 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:17:53.166+0000 7efe04ff9700 1 -- 172.21.15.162:0/3445483250 <== mon.0 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 39) v1 ==== 92747+0+0 (secure 0 0 0) 0x7efdfc00cb30 con 0x7efe08103720 2024-04-27T04:17:53.166 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:17:53.167+0000 7efe04ff9700 1 --2- 172.21.15.162:0/3445483250 >> [v2:172.21.15.162:6800/1396866108,v1:172.21.15.162:6801/1396866108] conn(0x7efdf006ec10 0x7efdf00710c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:17:53.166 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:17:53.167+0000 7efe04ff9700 1 -- 172.21.15.162:0/3445483250 <== mon.0 v2:172.21.15.162:3300/0 5 ==== osd_map(69..69 src has 1..69) v4 ==== 6410+0+0 (secure 0 0 0) 0x7efdfc08e170 con 0x7efe08103720 2024-04-27T04:17:53.166 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:17:53.167+0000 7efe07fff700 1 --2- 172.21.15.162:0/3445483250 >> [v2:172.21.15.162:6800/1396866108,v1:172.21.15.162:6801/1396866108] conn(0x7efdf006ec10 0x7efdf00710c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:17:53.166 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:17:53.167+0000 7efe07fff700 1 --2- 172.21.15.162:0/3445483250 >> [v2:172.21.15.162:6800/1396866108,v1:172.21.15.162:6801/1396866108] conn(0x7efdf006ec10 0x7efdf00710c0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7efdf8007f50 tx=0x7efdf8009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:17:53.169 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:17:53.171+0000 7efe04ff9700 1 -- 172.21.15.162:0/3445483250 <== mon.0 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7efdfc05abd0 con 0x7efe08103720 2024-04-27T04:17:53.387 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:17:53.389+0000 7efe0e416700 1 -- 172.21.15.162:0/3445483250 --> [v2:172.21.15.162:6800/1396866108,v1:172.21.15.162:6801/1396866108] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7efe08109e80 con 0x7efdf006ec10 2024-04-27T04:17:53.388 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:17:53.391+0000 7efe04ff9700 1 -- 172.21.15.162:0/3445483250 <== mgr.34104 v2:172.21.15.162:6800/1396866108 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+401 (secure 0 0 0) 0x7efe08109e80 con 0x7efdf006ec10 2024-04-27T04:17:53.389 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-04-27T04:17:53.389 INFO:teuthology.orchestra.run.smithi162.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:90a3a9fa9ceb8977f6dad9ccd36fe46018d07e2118199820523a6b35301d9c16", 2024-04-27T04:17:53.389 INFO:teuthology.orchestra.run.smithi162.stdout: "in_progress": true, 2024-04-27T04:17:53.389 INFO:teuthology.orchestra.run.smithi162.stdout: "which": "Upgrading all daemon types on all hosts", 2024-04-27T04:17:53.389 INFO:teuthology.orchestra.run.smithi162.stdout: "services_complete": [ 2024-04-27T04:17:53.389 INFO:teuthology.orchestra.run.smithi162.stdout: "crash", 2024-04-27T04:17:53.389 INFO:teuthology.orchestra.run.smithi162.stdout: "mon", 2024-04-27T04:17:53.389 INFO:teuthology.orchestra.run.smithi162.stdout: "mgr" 2024-04-27T04:17:53.389 INFO:teuthology.orchestra.run.smithi162.stdout: ], 2024-04-27T04:17:53.389 INFO:teuthology.orchestra.run.smithi162.stdout: "progress": "10/21 daemons upgraded", 2024-04-27T04:17:53.390 INFO:teuthology.orchestra.run.smithi162.stdout: "message": "Currently upgrading osd daemons", 2024-04-27T04:17:53.390 INFO:teuthology.orchestra.run.smithi162.stdout: "is_paused": false 2024-04-27T04:17:53.390 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-04-27T04:17:53.391 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:17:53.393+0000 7efe0e416700 1 -- 172.21.15.162:0/3445483250 >> [v2:172.21.15.162:6800/1396866108,v1:172.21.15.162:6801/1396866108] conn(0x7efdf006ec10 msgr2=0x7efdf00710c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:17:53.391 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:17:53.393+0000 7efe0e416700 1 --2- 172.21.15.162:0/3445483250 >> [v2:172.21.15.162:6800/1396866108,v1:172.21.15.162:6801/1396866108] conn(0x7efdf006ec10 0x7efdf00710c0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7efdf8007f50 tx=0x7efdf8009e30 comp rx=0 tx=0).stop 2024-04-27T04:17:53.391 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:17:53.393+0000 7efe0e416700 1 -- 172.21.15.162:0/3445483250 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7efe08103720 msgr2=0x7efe08113ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:17:53.391 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:17:53.393+0000 7efe0e416700 1 --2- 172.21.15.162:0/3445483250 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7efe08103720 0x7efe08113ff0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7efdfc00eb40 tx=0x7efdfc00b040 comp rx=0 tx=0).stop 2024-04-27T04:17:53.391 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:17:53.393+0000 7efe0e416700 1 -- 172.21.15.162:0/3445483250 shutdown_connections 2024-04-27T04:17:53.391 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:17:53.393+0000 7efe0e416700 1 --2- 172.21.15.162:0/3445483250 >> [v2:172.21.15.162:6800/1396866108,v1:172.21.15.162:6801/1396866108] conn(0x7efdf006ec10 0x7efdf00710c0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:17:53.391 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:17:53.393+0000 7efe0e416700 1 --2- 172.21.15.162:0/3445483250 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7efe08102d80 0x7efe08113ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:17:53.391 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:17:53.393+0000 7efe0e416700 1 --2- 172.21.15.162:0/3445483250 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7efe08103720 0x7efe08113ff0 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:17:53.391 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:17:53.393+0000 7efe0e416700 1 -- 172.21.15.162:0/3445483250 >> 172.21.15.162:0/3445483250 conn(0x7efe080f8950 msgr2=0x7efe080f98e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:17:53.392 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:17:53.393+0000 7efe0e416700 1 -- 172.21.15.162:0/3445483250 shutdown_connections 2024-04-27T04:17:53.392 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:17:53.393+0000 7efe0e416700 1 -- 172.21.15.162:0/3445483250 wait complete. 2024-04-27T04:17:53.492 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:17:53.494+0000 7f5e26a90700 1 -- 172.21.15.162:0/2855467978 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5e200fbd40 msgr2=0x7f5e200fc1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:17:53.492 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:17:53.494+0000 7f5e26a90700 1 --2- 172.21.15.162:0/2855467978 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5e200fbd40 0x7f5e200fc1b0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f5e10004940 tx=0x7f5e10009e30 comp rx=0 tx=0).stop 2024-04-27T04:17:53.492 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:17:53.494+0000 7f5e26a90700 1 -- 172.21.15.162:0/2855467978 shutdown_connections 2024-04-27T04:17:53.492 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:17:53.494+0000 7f5e26a90700 1 --2- 172.21.15.162:0/2855467978 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5e200fbd40 0x7f5e200fc1b0 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:17:53.493 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:17:53.494+0000 7f5e26a90700 1 --2- 172.21.15.162:0/2855467978 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5e20101d90 0x7f5e20102160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:17:53.493 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:17:53.494+0000 7f5e26a90700 1 -- 172.21.15.162:0/2855467978 >> 172.21.15.162:0/2855467978 conn(0x7f5e20071d00 msgr2=0x7f5e20072100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:17:53.493 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:17:53.494+0000 7f5e26a90700 1 -- 172.21.15.162:0/2855467978 shutdown_connections 2024-04-27T04:17:53.493 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:17:53.494+0000 7f5e26a90700 1 -- 172.21.15.162:0/2855467978 wait complete. 2024-04-27T04:17:53.493 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:17:53.494+0000 7f5e26a90700 1 Processor -- start 2024-04-27T04:17:53.493 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:17:53.494+0000 7f5e26a90700 1 -- start start 2024-04-27T04:17:53.495 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:17:53.495+0000 7f5e26a90700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5e200fbd40 0x7f5e20112a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:17:53.495 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:17:53.495+0000 7f5e26a90700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5e20101d90 0x7f5e20112f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:17:53.495 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:17:53.495+0000 7f5e26a90700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e20113590 con 0x7f5e200fbd40 2024-04-27T04:17:53.495 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:17:53.495+0000 7f5e26a90700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e2010c980 con 0x7f5e20101d90 2024-04-27T04:17:53.495 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:17:53.495+0000 7f5e2482c700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5e200fbd40 0x7f5e20112a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:17:53.495 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:17:53.495+0000 7f5e2482c700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5e200fbd40 0x7f5e20112a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:52142/0 (socket says 172.21.15.162:52142) 2024-04-27T04:17:53.495 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:17:53.495+0000 7f5e2482c700 1 -- 172.21.15.162:0/1413311 learned_addr learned my addr 172.21.15.162:0/1413311 (peer_addr_for_me v2:172.21.15.162:0/0) 2024-04-27T04:17:53.495 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:17:53.495+0000 7f5e17fff700 1 --2- 172.21.15.162:0/1413311 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5e20101d90 0x7f5e20112f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:17:53.495 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:17:53.495+0000 7f5e2482c700 1 -- 172.21.15.162:0/1413311 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5e20101d90 msgr2=0x7f5e20112f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:17:53.495 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:17:53.495+0000 7f5e2482c700 1 --2- 172.21.15.162:0/1413311 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5e20101d90 0x7f5e20112f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:17:53.495 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:17:53.495+0000 7f5e2482c700 1 -- 172.21.15.162:0/1413311 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e10005040 con 0x7f5e200fbd40 2024-04-27T04:17:53.495 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:17:53.495+0000 7f5e2482c700 1 --2- 172.21.15.162:0/1413311 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5e200fbd40 0x7f5e20112a00 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f5e1c010ea0 tx=0x7f5e1c00b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:17:53.495 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:17:53.496+0000 7f5e157fa700 1 -- 172.21.15.162:0/1413311 <== mon.0 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5e1c007920 con 0x7f5e200fbd40 2024-04-27T04:17:53.496 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:17:53.496+0000 7f5e157fa700 1 -- 172.21.15.162:0/1413311 <== mon.0 v2:172.21.15.162:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f5e1c0090d0 con 0x7f5e200fbd40 2024-04-27T04:17:53.496 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:17:53.496+0000 7f5e26a90700 1 -- 172.21.15.162:0/1413311 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5e2010cc00 con 0x7f5e200fbd40 2024-04-27T04:17:53.496 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:17:53.496+0000 7f5e157fa700 1 -- 172.21.15.162:0/1413311 <== mon.0 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5e1c012830 con 0x7f5e200fbd40 2024-04-27T04:17:53.496 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:17:53.496+0000 7f5e26a90700 1 -- 172.21.15.162:0/1413311 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5e2010d110 con 0x7f5e200fbd40 2024-04-27T04:17:53.496 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:17:53.497+0000 7f5e26a90700 1 -- 172.21.15.162:0/1413311 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5e200fd430 con 0x7f5e200fbd40 2024-04-27T04:17:53.496 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:17:53.497+0000 7f5e157fa700 1 -- 172.21.15.162:0/1413311 <== mon.0 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 39) v1 ==== 92747+0+0 (secure 0 0 0) 0x7f5e1c007a80 con 0x7f5e200fbd40 2024-04-27T04:17:53.496 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:17:53.497+0000 7f5e157fa700 1 --2- 172.21.15.162:0/1413311 >> [v2:172.21.15.162:6800/1396866108,v1:172.21.15.162:6801/1396866108] conn(0x7f5e0806ec00 0x7f5e080710b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:17:53.497 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:17:53.498+0000 7f5e157fa700 1 -- 172.21.15.162:0/1413311 <== mon.0 v2:172.21.15.162:3300/0 5 ==== osd_map(69..69 src has 1..69) v4 ==== 6410+0+0 (secure 0 0 0) 0x7f5e1c016070 con 0x7f5e200fbd40 2024-04-27T04:17:53.497 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:17:53.498+0000 7f5e17fff700 1 --2- 172.21.15.162:0/1413311 >> [v2:172.21.15.162:6800/1396866108,v1:172.21.15.162:6801/1396866108] conn(0x7f5e0806ec00 0x7f5e080710b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:17:53.497 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:17:53.498+0000 7f5e17fff700 1 --2- 172.21.15.162:0/1413311 >> [v2:172.21.15.162:6800/1396866108,v1:172.21.15.162:6801/1396866108] conn(0x7f5e0806ec00 0x7f5e080710b0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f5e10009b30 tx=0x7f5e10009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:17:53.500 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:17:53.502+0000 7f5e157fa700 1 -- 172.21.15.162:0/1413311 <== mon.0 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7f5e1c05dae0 con 0x7f5e200fbd40 2024-04-27T04:17:53.791 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:17:53.792+0000 7f5e26a90700 1 -- 172.21.15.162:0/1413311 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f5e20108e80 con 0x7f5e200fbd40 2024-04-27T04:17:53.791 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:17:53.792+0000 7f5e157fa700 1 -- 172.21.15.162:0/1413311 <== mon.0 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+10 (secure 0 0 0) 0x7f5e1c05d670 con 0x7f5e200fbd40 2024-04-27T04:17:53.792 INFO:teuthology.orchestra.run.smithi162.stdout:HEALTH_OK 2024-04-27T04:17:53.794 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:17:53.795+0000 7f5e26a90700 1 -- 172.21.15.162:0/1413311 >> [v2:172.21.15.162:6800/1396866108,v1:172.21.15.162:6801/1396866108] conn(0x7f5e0806ec00 msgr2=0x7f5e080710b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:17:53.794 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:17:53.795+0000 7f5e26a90700 1 --2- 172.21.15.162:0/1413311 >> [v2:172.21.15.162:6800/1396866108,v1:172.21.15.162:6801/1396866108] conn(0x7f5e0806ec00 0x7f5e080710b0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f5e10009b30 tx=0x7f5e10009e30 comp rx=0 tx=0).stop 2024-04-27T04:17:53.794 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:17:53.795+0000 7f5e26a90700 1 -- 172.21.15.162:0/1413311 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5e200fbd40 msgr2=0x7f5e20112a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:17:53.794 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:17:53.795+0000 7f5e26a90700 1 --2- 172.21.15.162:0/1413311 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5e200fbd40 0x7f5e20112a00 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f5e1c010ea0 tx=0x7f5e1c00b040 comp rx=0 tx=0).stop 2024-04-27T04:17:53.794 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:17:53.795+0000 7f5e26a90700 1 -- 172.21.15.162:0/1413311 shutdown_connections 2024-04-27T04:17:53.794 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:17:53.795+0000 7f5e26a90700 1 --2- 172.21.15.162:0/1413311 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5e200fbd40 0x7f5e20112a00 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:17:53.794 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:17:53.795+0000 7f5e26a90700 1 --2- 172.21.15.162:0/1413311 >> [v2:172.21.15.162:6800/1396866108,v1:172.21.15.162:6801/1396866108] conn(0x7f5e0806ec00 0x7f5e080710b0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:17:53.794 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:17:53.795+0000 7f5e26a90700 1 --2- 172.21.15.162:0/1413311 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5e20101d90 0x7f5e20112f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:17:53.794 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:17:53.795+0000 7f5e26a90700 1 -- 172.21.15.162:0/1413311 >> 172.21.15.162:0/1413311 conn(0x7f5e20071d00 msgr2=0x7f5e200fa0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:17:53.794 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:17:53.795+0000 7f5e26a90700 1 -- 172.21.15.162:0/1413311 shutdown_connections 2024-04-27T04:17:53.794 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:17:53.795+0000 7f5e26a90700 1 -- 172.21.15.162:0/1413311 wait complete. 2024-04-27T04:17:53.921 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:17:53 smithi181 ceph-mon[81134]: from='client.34318 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-27T04:17:53.922 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:17:53 smithi181 ceph-mon[81134]: from='client.34322 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-27T04:17:53.922 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:17:53 smithi181 ceph-mon[81134]: from='client.? 172.21.15.162:0/3671935529' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-27T04:17:53.922 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:17:53 smithi181 ceph-mon[81134]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' 2024-04-27T04:17:53.922 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:17:53 smithi181 ceph-mon[81134]: from='client.? 172.21.15.162:0/2234611580' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-27T04:17:53.922 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:17:53 smithi181 ceph-mon[81134]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' 2024-04-27T04:17:53.922 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:17:53 smithi181 ceph-mon[81134]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-04-27T04:17:53.923 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:17:53 smithi181 ceph-mon[81134]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T04:17:54.136 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:17:53 smithi162 ceph-mon[110641]: from='client.34318 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-27T04:17:54.137 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:17:53 smithi162 ceph-mon[110641]: from='client.34322 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-27T04:17:54.137 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:17:53 smithi162 ceph-mon[110641]: from='client.? 172.21.15.162:0/3671935529' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-27T04:17:54.137 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:17:53 smithi162 ceph-mon[110641]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' 2024-04-27T04:17:54.137 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:17:53 smithi162 ceph-mon[110641]: from='client.? 172.21.15.162:0/2234611580' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-27T04:17:54.137 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:17:53 smithi162 ceph-mon[110641]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' 2024-04-27T04:17:54.137 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:17:53 smithi162 ceph-mon[110641]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-04-27T04:17:54.137 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:17:53 smithi162 ceph-mon[110641]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T04:17:54.922 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:17:54 smithi181 ceph-mon[81134]: Upgrade: Updating osd.4 2024-04-27T04:17:54.922 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:17:54 smithi181 ceph-mon[81134]: Deploying daemon osd.4 on smithi181 2024-04-27T04:17:54.922 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:17:54 smithi181 ceph-mon[81134]: from='client.34334 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-27T04:17:54.922 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:17:54 smithi181 ceph-mon[81134]: pgmap v131: 65 pgs: 65 active+clean; 208 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-04-27T04:17:54.923 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:17:54 smithi181 ceph-mon[81134]: from='client.? 172.21.15.162:0/1413311' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-27T04:17:55.136 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:17:54 smithi162 ceph-mon[110641]: Upgrade: Updating osd.4 2024-04-27T04:17:55.136 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:17:54 smithi162 ceph-mon[110641]: Deploying daemon osd.4 on smithi181 2024-04-27T04:17:55.137 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:17:54 smithi162 ceph-mon[110641]: from='client.34334 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-27T04:17:55.137 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:17:54 smithi162 ceph-mon[110641]: pgmap v131: 65 pgs: 65 active+clean; 208 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-04-27T04:17:55.137 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:17:54 smithi162 ceph-mon[110641]: from='client.? 172.21.15.162:0/1413311' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-27T04:17:56.625 INFO:journalctl@ceph.osd.4.smithi181.stdout:Apr 27 04:17:56 smithi181 systemd[1]: Stopping Ceph osd.4 for 7f9750d6-044a-11ef-bc93-c7b262605968... 2024-04-27T04:17:56.625 INFO:journalctl@ceph.osd.4.smithi181.stdout:Apr 27 04:17:56 smithi181 ceph-7f9750d6-044a-11ef-bc93-c7b262605968-osd-4[46239]: 2024-04-27T04:17:56.609+0000 7f8819300700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.4 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-27T04:17:56.625 INFO:journalctl@ceph.osd.4.smithi181.stdout:Apr 27 04:17:56 smithi181 ceph-7f9750d6-044a-11ef-bc93-c7b262605968-osd-4[46239]: 2024-04-27T04:17:56.609+0000 7f8819300700 -1 osd.4 69 *** Got signal Terminated *** 2024-04-27T04:17:56.626 INFO:journalctl@ceph.osd.4.smithi181.stdout:Apr 27 04:17:56 smithi181 ceph-7f9750d6-044a-11ef-bc93-c7b262605968-osd-4[46239]: 2024-04-27T04:17:56.609+0000 7f8819300700 -1 osd.4 69 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-27T04:17:56.922 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:17:56 smithi181 ceph-mon[81134]: pgmap v132: 65 pgs: 65 active+clean; 208 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:17:56.922 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:17:56 smithi181 ceph-mon[81134]: osd.4 marked itself down and dead 2024-04-27T04:17:57.136 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:17:56 smithi162 ceph-mon[110641]: pgmap v132: 65 pgs: 65 active+clean; 208 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:17:57.136 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:17:56 smithi162 ceph-mon[110641]: osd.4 marked itself down and dead 2024-04-27T04:17:57.422 INFO:journalctl@ceph.osd.4.smithi181.stdout:Apr 27 04:17:57 smithi181 podman[95218]: 2024-04-27 04:17:57.162582658 +0000 UTC m=+0.649087613 container died 512afbee7cf78a0eb9b7614cf1c7704486a16476421900fa4fe6abc7090a7422 (image=quay.ceph.io/ceph-ci/ceph@sha256:d31f278ece6dbd17d1fa5e598f509017f54b76fafe1c4416391fcc51b9718ee9, name=ceph-7f9750d6-044a-11ef-bc93-c7b262605968-osd-4, GIT_CLEAN=True, io.buildah.version=1.33.5, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, RELEASE=quincy-2be3efa, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.build-date=20240424) 2024-04-27T04:17:57.767 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:17:57 smithi181 ceph-mon[81134]: Health check failed: 1 osds down (OSD_DOWN) 2024-04-27T04:17:57.767 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:17:57 smithi181 ceph-mon[81134]: osdmap e70: 6 total, 5 up, 6 in 2024-04-27T04:17:57.767 INFO:journalctl@ceph.osd.4.smithi181.stdout:Apr 27 04:17:57 smithi181 podman[95218]: 2024-04-27 04:17:57.584760178 +0000 UTC m=+1.071265133 container cleanup 512afbee7cf78a0eb9b7614cf1c7704486a16476421900fa4fe6abc7090a7422 (image=quay.ceph.io/ceph-ci/ceph@sha256:d31f278ece6dbd17d1fa5e598f509017f54b76fafe1c4416391fcc51b9718ee9, name=ceph-7f9750d6-044a-11ef-bc93-c7b262605968-osd-4, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=quincy-2be3efa, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240424, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, ceph=True, GIT_CLEAN=True) 2024-04-27T04:17:57.768 INFO:journalctl@ceph.osd.4.smithi181.stdout:Apr 27 04:17:57 smithi181 bash[95218]: ceph-7f9750d6-044a-11ef-bc93-c7b262605968-osd-4 2024-04-27T04:17:58.136 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:17:57 smithi162 ceph-mon[110641]: Health check failed: 1 osds down (OSD_DOWN) 2024-04-27T04:17:58.136 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:17:57 smithi162 ceph-mon[110641]: osdmap e70: 6 total, 5 up, 6 in 2024-04-27T04:17:58.171 INFO:journalctl@ceph.osd.4.smithi181.stdout:Apr 27 04:17:57 smithi181 podman[95236]: 2024-04-27 04:17:57.769486309 +0000 UTC m=+0.596253855 container remove 512afbee7cf78a0eb9b7614cf1c7704486a16476421900fa4fe6abc7090a7422 (image=quay.ceph.io/ceph-ci/ceph@sha256:d31f278ece6dbd17d1fa5e598f509017f54b76fafe1c4416391fcc51b9718ee9, name=ceph-7f9750d6-044a-11ef-bc93-c7b262605968-osd-4, ceph=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240424, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=quincy-2be3efa, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, io.buildah.version=1.33.5, GIT_CLEAN=True) 2024-04-27T04:17:58.445 INFO:journalctl@ceph.osd.4.smithi181.stdout:Apr 27 04:17:58 smithi181 podman[95329]: 2024-04-27 04:17:58.096682094 +0000 UTC m=+0.017731111 image pull 67e414ff4bc41092cb31c84c98a9563a32813d02583028679c46b40362afeb3e quay.ceph.io/ceph-ci/ceph@sha256:90a3a9fa9ceb8977f6dad9ccd36fe46018d07e2118199820523a6b35301d9c16 2024-04-27T04:17:58.445 INFO:journalctl@ceph.osd.4.smithi181.stdout:Apr 27 04:17:58 smithi181 podman[95329]: 2024-04-27 04:17:58.220500195 +0000 UTC m=+0.141549208 container create 114fcb2ae4b17bfe7918202cbc7a3886a65dadf7f33f4e61588251ac134b046c (image=quay.ceph.io/ceph-ci/ceph@sha256:90a3a9fa9ceb8977f6dad9ccd36fe46018d07e2118199820523a6b35301d9c16, name=ceph-7f9750d6-044a-11ef-bc93-c7b262605968-osd-4-deactivate, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240424, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, maintainer=Guillaume Abrioux , RELEASE=main-4e213a5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_CLEAN=True) 2024-04-27T04:17:58.445 INFO:journalctl@ceph.osd.4.smithi181.stdout:Apr 27 04:17:58 smithi181 podman[95329]: 2024-04-27 04:17:58.447998451 +0000 UTC m=+0.369047465 container init 114fcb2ae4b17bfe7918202cbc7a3886a65dadf7f33f4e61588251ac134b046c (image=quay.ceph.io/ceph-ci/ceph@sha256:90a3a9fa9ceb8977f6dad9ccd36fe46018d07e2118199820523a6b35301d9c16, name=ceph-7f9750d6-044a-11ef-bc93-c7b262605968-osd-4-deactivate, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_CLEAN=True, maintainer=Guillaume Abrioux , RELEASE=main-4e213a5, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.build-date=20240424) 2024-04-27T04:17:58.699 INFO:journalctl@ceph.osd.4.smithi181.stdout:Apr 27 04:17:58 smithi181 podman[95329]: 2024-04-27 04:17:58.451213158 +0000 UTC m=+0.372262173 container start 114fcb2ae4b17bfe7918202cbc7a3886a65dadf7f33f4e61588251ac134b046c (image=quay.ceph.io/ceph-ci/ceph@sha256:90a3a9fa9ceb8977f6dad9ccd36fe46018d07e2118199820523a6b35301d9c16, name=ceph-7f9750d6-044a-11ef-bc93-c7b262605968-osd-4-deactivate, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240424, maintainer=Guillaume Abrioux , RELEASE=main-4e213a5, org.label-schema.license=GPLv2, ceph=True, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_CLEAN=True) 2024-04-27T04:17:58.699 INFO:journalctl@ceph.osd.4.smithi181.stdout:Apr 27 04:17:58 smithi181 podman[95329]: 2024-04-27 04:17:58.536853239 +0000 UTC m=+0.457902269 container attach 114fcb2ae4b17bfe7918202cbc7a3886a65dadf7f33f4e61588251ac134b046c (image=quay.ceph.io/ceph-ci/ceph@sha256:90a3a9fa9ceb8977f6dad9ccd36fe46018d07e2118199820523a6b35301d9c16, name=ceph-7f9750d6-044a-11ef-bc93-c7b262605968-osd-4-deactivate, io.buildah.version=1.33.5, RELEASE=main-4e213a5, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240424, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0) 2024-04-27T04:17:59.090 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:17:58 smithi181 ceph-mon[81134]: pgmap v134: 65 pgs: 6 stale+active+clean, 59 active+clean; 208 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1023 B/s rd, 1 op/s 2024-04-27T04:17:59.090 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:17:58 smithi181 ceph-mon[81134]: osdmap e71: 6 total, 5 up, 6 in 2024-04-27T04:17:59.090 INFO:journalctl@ceph.osd.4.smithi181.stdout:Apr 27 04:17:58 smithi181 podman[95329]: 2024-04-27 04:17:58.747973804 +0000 UTC m=+0.669022810 container died 114fcb2ae4b17bfe7918202cbc7a3886a65dadf7f33f4e61588251ac134b046c (image=quay.ceph.io/ceph-ci/ceph@sha256:90a3a9fa9ceb8977f6dad9ccd36fe46018d07e2118199820523a6b35301d9c16, name=ceph-7f9750d6-044a-11ef-bc93-c7b262605968-osd-4-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=main-4e213a5, GIT_CLEAN=True, ceph=True, org.label-schema.build-date=20240424, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=) 2024-04-27T04:17:59.136 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:17:58 smithi162 ceph-mon[110641]: pgmap v134: 65 pgs: 6 stale+active+clean, 59 active+clean; 208 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1023 B/s rd, 1 op/s 2024-04-27T04:17:59.136 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:17:58 smithi162 ceph-mon[110641]: osdmap e71: 6 total, 5 up, 6 in 2024-04-27T04:17:59.671 INFO:journalctl@ceph.osd.4.smithi181.stdout:Apr 27 04:17:59 smithi181 podman[95355]: 2024-04-27 04:17:59.343173424 +0000 UTC m=+0.590206414 container remove 114fcb2ae4b17bfe7918202cbc7a3886a65dadf7f33f4e61588251ac134b046c (image=quay.ceph.io/ceph-ci/ceph@sha256:90a3a9fa9ceb8977f6dad9ccd36fe46018d07e2118199820523a6b35301d9c16, name=ceph-7f9750d6-044a-11ef-bc93-c7b262605968-osd-4-deactivate, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, RELEASE=main-4e213a5, io.buildah.version=1.33.5, org.label-schema.build-date=20240424, ceph=True, org.label-schema.vendor=CentOS, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-27T04:17:59.672 INFO:journalctl@ceph.osd.4.smithi181.stdout:Apr 27 04:17:59 smithi181 systemd[1]: ceph-7f9750d6-044a-11ef-bc93-c7b262605968@osd.4.service: Deactivated successfully. 2024-04-27T04:17:59.672 INFO:journalctl@ceph.osd.4.smithi181.stdout:Apr 27 04:17:59 smithi181 systemd[1]: Stopped Ceph osd.4 for 7f9750d6-044a-11ef-bc93-c7b262605968. 2024-04-27T04:17:59.672 INFO:journalctl@ceph.osd.4.smithi181.stdout:Apr 27 04:17:59 smithi181 systemd[1]: ceph-7f9750d6-044a-11ef-bc93-c7b262605968@osd.4.service: Consumed 1min 5.470s CPU time. 2024-04-27T04:17:59.672 INFO:journalctl@ceph.osd.4.smithi181.stdout:Apr 27 04:17:59 smithi181 systemd[1]: Starting Ceph osd.4 for 7f9750d6-044a-11ef-bc93-c7b262605968... 2024-04-27T04:18:00.038 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:17:59 smithi162 ceph-mon[110641]: Health check failed: Degraded data redundancy: 14/231 objects degraded (6.061%), 6 pgs degraded (PG_DEGRADED) 2024-04-27T04:18:00.044 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:17:59 smithi181 ceph-mon[81134]: Health check failed: Degraded data redundancy: 14/231 objects degraded (6.061%), 6 pgs degraded (PG_DEGRADED) 2024-04-27T04:18:00.044 INFO:journalctl@ceph.osd.4.smithi181.stdout:Apr 27 04:17:59 smithi181 podman[95460]: 2024-04-27 04:17:59.692064046 +0000 UTC m=+0.018461974 image pull 67e414ff4bc41092cb31c84c98a9563a32813d02583028679c46b40362afeb3e quay.ceph.io/ceph-ci/ceph@sha256:90a3a9fa9ceb8977f6dad9ccd36fe46018d07e2118199820523a6b35301d9c16 2024-04-27T04:18:00.045 INFO:journalctl@ceph.osd.4.smithi181.stdout:Apr 27 04:17:59 smithi181 podman[95460]: 2024-04-27 04:17:59.819302926 +0000 UTC m=+0.145700829 container create e6908f33c26c2c2f3d9b5eb4c9d164ecf5cc99d9ac251e26f2e6f5f45831b8e6 (image=quay.ceph.io/ceph-ci/ceph@sha256:90a3a9fa9ceb8977f6dad9ccd36fe46018d07e2118199820523a6b35301d9c16, name=ceph-7f9750d6-044a-11ef-bc93-c7b262605968-osd-4-activate, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, ceph=True, org.label-schema.license=GPLv2, RELEASE=main-4e213a5, org.label-schema.build-date=20240424, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-27T04:18:00.422 INFO:journalctl@ceph.osd.4.smithi181.stdout:Apr 27 04:18:00 smithi181 podman[95460]: 2024-04-27 04:18:00.046463164 +0000 UTC m=+0.372861077 container init e6908f33c26c2c2f3d9b5eb4c9d164ecf5cc99d9ac251e26f2e6f5f45831b8e6 (image=quay.ceph.io/ceph-ci/ceph@sha256:90a3a9fa9ceb8977f6dad9ccd36fe46018d07e2118199820523a6b35301d9c16, name=ceph-7f9750d6-044a-11ef-bc93-c7b262605968-osd-4-activate, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240424, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, RELEASE=main-4e213a5, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-27T04:18:00.422 INFO:journalctl@ceph.osd.4.smithi181.stdout:Apr 27 04:18:00 smithi181 podman[95460]: 2024-04-27 04:18:00.050642749 +0000 UTC m=+0.377040654 container start e6908f33c26c2c2f3d9b5eb4c9d164ecf5cc99d9ac251e26f2e6f5f45831b8e6 (image=quay.ceph.io/ceph-ci/ceph@sha256:90a3a9fa9ceb8977f6dad9ccd36fe46018d07e2118199820523a6b35301d9c16, name=ceph-7f9750d6-044a-11ef-bc93-c7b262605968-osd-4-activate, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , RELEASE=main-4e213a5, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240424, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=) 2024-04-27T04:18:00.422 INFO:journalctl@ceph.osd.4.smithi181.stdout:Apr 27 04:18:00 smithi181 podman[95460]: 2024-04-27 04:18:00.138390379 +0000 UTC m=+0.464788303 container attach e6908f33c26c2c2f3d9b5eb4c9d164ecf5cc99d9ac251e26f2e6f5f45831b8e6 (image=quay.ceph.io/ceph-ci/ceph@sha256:90a3a9fa9ceb8977f6dad9ccd36fe46018d07e2118199820523a6b35301d9c16, name=ceph-7f9750d6-044a-11ef-bc93-c7b262605968-osd-4-activate, org.label-schema.schema-version=1.0, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=main-4e213a5, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.build-date=20240424, maintainer=Guillaume Abrioux ) 2024-04-27T04:18:00.823 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:18:00 smithi181 ceph-mon[81134]: pgmap v136: 65 pgs: 6 active+undersized+degraded, 3 stale+active+clean, 9 active+undersized, 47 active+clean; 208 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 639 B/s rd, 1 op/s; 14/231 objects degraded (6.061%) 2024-04-27T04:18:00.823 INFO:journalctl@ceph.osd.4.smithi181.stdout:Apr 27 04:18:00 smithi181 ceph-7f9750d6-044a-11ef-bc93-c7b262605968-osd-4-activate[95475]: Running command: /usr/bin/ceph-authtool --gen-print-key 2024-04-27T04:18:00.823 INFO:journalctl@ceph.osd.4.smithi181.stdout:Apr 27 04:18:00 smithi181 ceph-7f9750d6-044a-11ef-bc93-c7b262605968-osd-4-activate[95475]: Running command: /usr/bin/ceph-authtool --gen-print-key 2024-04-27T04:18:00.823 INFO:journalctl@ceph.osd.4.smithi181.stdout:Apr 27 04:18:00 smithi181 bash[95460]: Running command: /usr/bin/ceph-authtool --gen-print-key 2024-04-27T04:18:00.823 INFO:journalctl@ceph.osd.4.smithi181.stdout:Apr 27 04:18:00 smithi181 bash[95460]: Running command: /usr/bin/ceph-authtool --gen-print-key 2024-04-27T04:18:00.823 INFO:journalctl@ceph.osd.4.smithi181.stdout:Apr 27 04:18:00 smithi181 bash[95460]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2024-04-27T04:18:00.824 INFO:journalctl@ceph.osd.4.smithi181.stdout:Apr 27 04:18:00 smithi181 bash[95460]: Running command: /usr/bin/ceph-bluestore-tool prime-osd-dir --path /var/lib/ceph/osd/ceph-4 --no-mon-config --dev /dev/mapper/vg_nvme-lv_3 2024-04-27T04:18:00.824 INFO:journalctl@ceph.osd.4.smithi181.stdout:Apr 27 04:18:00 smithi181 bash[95460]: Running command: /usr/bin/chown -h ceph:ceph /dev/mapper/vg_nvme-lv_3 2024-04-27T04:18:00.824 INFO:journalctl@ceph.osd.4.smithi181.stdout:Apr 27 04:18:00 smithi181 bash[95460]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-2 2024-04-27T04:18:00.824 INFO:journalctl@ceph.osd.4.smithi181.stdout:Apr 27 04:18:00 smithi181 bash[95460]: Running command: /usr/bin/ln -s /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-4/block 2024-04-27T04:18:00.824 INFO:journalctl@ceph.osd.4.smithi181.stdout:Apr 27 04:18:00 smithi181 bash[95460]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2024-04-27T04:18:00.824 INFO:journalctl@ceph.osd.4.smithi181.stdout:Apr 27 04:18:00 smithi181 bash[95460]: --> ceph-volume raw activate successful for osd ID: 4 2024-04-27T04:18:00.824 INFO:journalctl@ceph.osd.4.smithi181.stdout:Apr 27 04:18:00 smithi181 ceph-7f9750d6-044a-11ef-bc93-c7b262605968-osd-4-activate[95475]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2024-04-27T04:18:00.824 INFO:journalctl@ceph.osd.4.smithi181.stdout:Apr 27 04:18:00 smithi181 ceph-7f9750d6-044a-11ef-bc93-c7b262605968-osd-4-activate[95475]: Running command: /usr/bin/ceph-bluestore-tool prime-osd-dir --path /var/lib/ceph/osd/ceph-4 --no-mon-config --dev /dev/mapper/vg_nvme-lv_3 2024-04-27T04:18:00.824 INFO:journalctl@ceph.osd.4.smithi181.stdout:Apr 27 04:18:00 smithi181 ceph-7f9750d6-044a-11ef-bc93-c7b262605968-osd-4-activate[95475]: Running command: /usr/bin/chown -h ceph:ceph /dev/mapper/vg_nvme-lv_3 2024-04-27T04:18:00.824 INFO:journalctl@ceph.osd.4.smithi181.stdout:Apr 27 04:18:00 smithi181 ceph-7f9750d6-044a-11ef-bc93-c7b262605968-osd-4-activate[95475]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-2 2024-04-27T04:18:00.824 INFO:journalctl@ceph.osd.4.smithi181.stdout:Apr 27 04:18:00 smithi181 ceph-7f9750d6-044a-11ef-bc93-c7b262605968-osd-4-activate[95475]: Running command: /usr/bin/ln -s /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-4/block 2024-04-27T04:18:00.824 INFO:journalctl@ceph.osd.4.smithi181.stdout:Apr 27 04:18:00 smithi181 ceph-7f9750d6-044a-11ef-bc93-c7b262605968-osd-4-activate[95475]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2024-04-27T04:18:00.824 INFO:journalctl@ceph.osd.4.smithi181.stdout:Apr 27 04:18:00 smithi181 ceph-7f9750d6-044a-11ef-bc93-c7b262605968-osd-4-activate[95475]: --> ceph-volume raw activate successful for osd ID: 4 2024-04-27T04:18:00.824 INFO:journalctl@ceph.osd.4.smithi181.stdout:Apr 27 04:18:00 smithi181 podman[95460]: 2024-04-27 04:18:00.559831021 +0000 UTC m=+0.886228931 container died e6908f33c26c2c2f3d9b5eb4c9d164ecf5cc99d9ac251e26f2e6f5f45831b8e6 (image=quay.ceph.io/ceph-ci/ceph@sha256:90a3a9fa9ceb8977f6dad9ccd36fe46018d07e2118199820523a6b35301d9c16, name=ceph-7f9750d6-044a-11ef-bc93-c7b262605968-osd-4-activate, maintainer=Guillaume Abrioux , GIT_CLEAN=True, RELEASE=main-4e213a5, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240424, ceph=True, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-27T04:18:01.137 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:18:00 smithi162 ceph-mon[110641]: pgmap v136: 65 pgs: 6 active+undersized+degraded, 3 stale+active+clean, 9 active+undersized, 47 active+clean; 208 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 639 B/s rd, 1 op/s; 14/231 objects degraded (6.061%) 2024-04-27T04:18:01.422 INFO:journalctl@ceph.osd.4.smithi181.stdout:Apr 27 04:18:01 smithi181 podman[95616]: 2024-04-27 04:18:01.134067295 +0000 UTC m=+0.566914560 container remove e6908f33c26c2c2f3d9b5eb4c9d164ecf5cc99d9ac251e26f2e6f5f45831b8e6 (image=quay.ceph.io/ceph-ci/ceph@sha256:90a3a9fa9ceb8977f6dad9ccd36fe46018d07e2118199820523a6b35301d9c16, name=ceph-7f9750d6-044a-11ef-bc93-c7b262605968-osd-4-activate, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240424, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , RELEASE=main-4e213a5, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, ceph=True, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-27T04:18:01.422 INFO:journalctl@ceph.osd.4.smithi181.stdout:Apr 27 04:18:01 smithi181 podman[95682]: 2024-04-27 04:18:01.249233869 +0000 UTC m=+0.018775172 image pull 67e414ff4bc41092cb31c84c98a9563a32813d02583028679c46b40362afeb3e quay.ceph.io/ceph-ci/ceph@sha256:90a3a9fa9ceb8977f6dad9ccd36fe46018d07e2118199820523a6b35301d9c16 2024-04-27T04:18:01.422 INFO:journalctl@ceph.osd.4.smithi181.stdout:Apr 27 04:18:01 smithi181 podman[95682]: 2024-04-27 04:18:01.360037929 +0000 UTC m=+0.129579278 container create de3e10f2d26d90be9fd0cd7f7bf01bd9ffd95a455849a6d3da7311decd0cf5a1 (image=quay.ceph.io/ceph-ci/ceph@sha256:90a3a9fa9ceb8977f6dad9ccd36fe46018d07e2118199820523a6b35301d9c16, name=ceph-7f9750d6-044a-11ef-bc93-c7b262605968-osd-4, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.build-date=20240424, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=main-4e213a5, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-27T04:18:01.922 INFO:journalctl@ceph.osd.4.smithi181.stdout:Apr 27 04:18:01 smithi181 podman[95682]: 2024-04-27 04:18:01.595990423 +0000 UTC m=+0.365531735 container init de3e10f2d26d90be9fd0cd7f7bf01bd9ffd95a455849a6d3da7311decd0cf5a1 (image=quay.ceph.io/ceph-ci/ceph@sha256:90a3a9fa9ceb8977f6dad9ccd36fe46018d07e2118199820523a6b35301d9c16, name=ceph-7f9750d6-044a-11ef-bc93-c7b262605968-osd-4, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_CLEAN=True, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, RELEASE=main-4e213a5, org.label-schema.build-date=20240424, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-27T04:18:01.922 INFO:journalctl@ceph.osd.4.smithi181.stdout:Apr 27 04:18:01 smithi181 podman[95682]: 2024-04-27 04:18:01.599132194 +0000 UTC m=+0.368673498 container start de3e10f2d26d90be9fd0cd7f7bf01bd9ffd95a455849a6d3da7311decd0cf5a1 (image=quay.ceph.io/ceph-ci/ceph@sha256:90a3a9fa9ceb8977f6dad9ccd36fe46018d07e2118199820523a6b35301d9c16, name=ceph-7f9750d6-044a-11ef-bc93-c7b262605968-osd-4, org.label-schema.license=GPLv2, RELEASE=main-4e213a5, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, ceph=True, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.build-date=20240424) 2024-04-27T04:18:01.922 INFO:journalctl@ceph.osd.4.smithi181.stdout:Apr 27 04:18:01 smithi181 ceph-osd[95702]: -- 172.21.15.181:0/3758223241 <== mon.1 v2:172.21.15.181:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55c5fcb59680 con 0x55c5fcb23000 2024-04-27T04:18:01.922 INFO:journalctl@ceph.osd.4.smithi181.stdout:Apr 27 04:18:01 smithi181 bash[95682]: de3e10f2d26d90be9fd0cd7f7bf01bd9ffd95a455849a6d3da7311decd0cf5a1 2024-04-27T04:18:01.922 INFO:journalctl@ceph.osd.4.smithi181.stdout:Apr 27 04:18:01 smithi181 systemd[1]: Started Ceph osd.4 for 7f9750d6-044a-11ef-bc93-c7b262605968. 2024-04-27T04:18:02.421 INFO:journalctl@ceph.osd.4.smithi181.stdout:Apr 27 04:18:02 smithi181 ceph-7f9750d6-044a-11ef-bc93-c7b262605968-osd-4[95698]: 2024-04-27T04:18:02.148+0000 7ffb0f61a7c0 -1 Falling back to public interface 2024-04-27T04:18:03.136 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:18:02 smithi162 ceph-mon[110641]: pgmap v137: 65 pgs: 11 active+undersized+degraded, 2 stale+active+clean, 15 active+undersized, 37 active+clean; 208 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 767 B/s rd, 1 op/s; 32/231 objects degraded (13.853%) 2024-04-27T04:18:03.136 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:18:02 smithi162 ceph-mon[110641]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' 2024-04-27T04:18:03.137 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:18:02 smithi162 ceph-mon[110641]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' 2024-04-27T04:18:03.137 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:18:02 smithi162 ceph-mon[110641]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T04:18:03.137 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:18:02 smithi162 ceph-mon[110641]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' 2024-04-27T04:18:03.137 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:18:02 smithi162 ceph-mon[110641]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-27T04:18:03.172 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:18:02 smithi181 ceph-mon[81134]: pgmap v137: 65 pgs: 11 active+undersized+degraded, 2 stale+active+clean, 15 active+undersized, 37 active+clean; 208 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 767 B/s rd, 1 op/s; 32/231 objects degraded (13.853%) 2024-04-27T04:18:03.172 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:18:02 smithi181 ceph-mon[81134]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' 2024-04-27T04:18:03.172 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:18:02 smithi181 ceph-mon[81134]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' 2024-04-27T04:18:03.172 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:18:02 smithi181 ceph-mon[81134]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T04:18:03.172 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:18:02 smithi181 ceph-mon[81134]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' 2024-04-27T04:18:03.172 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:18:02 smithi181 ceph-mon[81134]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-27T04:18:05.136 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:18:04 smithi162 ceph-mon[110641]: pgmap v138: 65 pgs: 15 active+undersized+degraded, 19 active+undersized, 31 active+clean; 208 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 127 B/s rd, 1 op/s; 41/231 objects degraded (17.749%) 2024-04-27T04:18:05.136 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:18:04 smithi162 ceph-mon[110641]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' 2024-04-27T04:18:05.136 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:18:04 smithi162 ceph-mon[110641]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' 2024-04-27T04:18:05.172 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:18:04 smithi181 ceph-mon[81134]: pgmap v138: 65 pgs: 15 active+undersized+degraded, 19 active+undersized, 31 active+clean; 208 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 127 B/s rd, 1 op/s; 41/231 objects degraded (17.749%) 2024-04-27T04:18:05.172 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:18:04 smithi181 ceph-mon[81134]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' 2024-04-27T04:18:05.172 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:18:04 smithi181 ceph-mon[81134]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' 2024-04-27T04:18:07.136 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:18:06 smithi162 ceph-mon[110641]: pgmap v139: 65 pgs: 15 active+undersized+degraded, 19 active+undersized, 31 active+clean; 208 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 685 B/s rd, 1 op/s; 41/231 objects degraded (17.749%) 2024-04-27T04:18:07.137 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:18:06 smithi162 ceph-mon[110641]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' 2024-04-27T04:18:07.137 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:18:06 smithi162 ceph-mon[110641]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' 2024-04-27T04:18:07.171 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:18:06 smithi181 ceph-mon[81134]: pgmap v139: 65 pgs: 15 active+undersized+degraded, 19 active+undersized, 31 active+clean; 208 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 685 B/s rd, 1 op/s; 41/231 objects degraded (17.749%) 2024-04-27T04:18:07.172 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:18:06 smithi181 ceph-mon[81134]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' 2024-04-27T04:18:07.172 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:18:06 smithi181 ceph-mon[81134]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' 2024-04-27T04:18:07.474 INFO:journalctl@ceph.osd.4.smithi181.stdout:Apr 27 04:18:07 smithi181 ceph-7f9750d6-044a-11ef-bc93-c7b262605968-osd-4[95698]: 2024-04-27T04:18:07.221+0000 7ffb0f61a7c0 -1 osd.4 69 log_to_monitors true 2024-04-27T04:18:08.136 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:18:07 smithi162 ceph-mon[110641]: from='osd.4 [v2:172.21.15.181:6808/1781129695,v1:172.21.15.181:6809/1781129695]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-04-27T04:18:08.136 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:18:07 smithi162 ceph-mon[110641]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-04-27T04:18:08.172 INFO:journalctl@ceph.osd.4.smithi181.stdout:Apr 27 04:18:07 smithi181 ceph-7f9750d6-044a-11ef-bc93-c7b262605968-osd-4[95698]: 2024-04-27T04:18:07.782+0000 7ffb00a72700 -1 osd.4 69 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-27T04:18:08.172 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:18:07 smithi181 ceph-mon[81134]: from='osd.4 [v2:172.21.15.181:6808/1781129695,v1:172.21.15.181:6809/1781129695]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-04-27T04:18:08.172 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:18:07 smithi181 ceph-mon[81134]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-04-27T04:18:08.868 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:18:08 smithi181 ceph-mon[81134]: pgmap v140: 65 pgs: 15 active+undersized+degraded, 19 active+undersized, 31 active+clean; 208 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 614 B/s rd, 1 op/s; 41/231 objects degraded (17.749%) 2024-04-27T04:18:08.868 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:18:08 smithi181 ceph-mon[81134]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-04-27T04:18:08.868 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:18:08 smithi181 ceph-mon[81134]: osdmap e72: 6 total, 5 up, 6 in 2024-04-27T04:18:08.868 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:18:08 smithi181 ceph-mon[81134]: from='osd.4 [v2:172.21.15.181:6808/1781129695,v1:172.21.15.181:6809/1781129695]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi181", "root=default"]}]: dispatch 2024-04-27T04:18:08.868 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:18:08 smithi181 ceph-mon[81134]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi181", "root=default"]}]: dispatch 2024-04-27T04:18:09.136 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:18:08 smithi162 ceph-mon[110641]: pgmap v140: 65 pgs: 15 active+undersized+degraded, 19 active+undersized, 31 active+clean; 208 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 614 B/s rd, 1 op/s; 41/231 objects degraded (17.749%) 2024-04-27T04:18:09.137 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:18:08 smithi162 ceph-mon[110641]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-04-27T04:18:09.137 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:18:08 smithi162 ceph-mon[110641]: osdmap e72: 6 total, 5 up, 6 in 2024-04-27T04:18:09.137 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:18:08 smithi162 ceph-mon[110641]: from='osd.4 [v2:172.21.15.181:6808/1781129695,v1:172.21.15.181:6809/1781129695]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi181", "root=default"]}]: dispatch 2024-04-27T04:18:09.137 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:18:08 smithi162 ceph-mon[110641]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi181", "root=default"]}]: dispatch 2024-04-27T04:18:10.137 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:18:09 smithi162 ceph-mon[110641]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-04-27T04:18:10.137 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:18:09 smithi162 ceph-mon[110641]: osd.4 [v2:172.21.15.181:6808/1781129695,v1:172.21.15.181:6809/1781129695] boot 2024-04-27T04:18:10.137 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:18:09 smithi162 ceph-mon[110641]: osdmap e73: 6 total, 6 up, 6 in 2024-04-27T04:18:10.137 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:18:09 smithi162 ceph-mon[110641]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-27T04:18:10.137 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:18:09 smithi162 ceph-mon[110641]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' 2024-04-27T04:18:10.137 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:18:09 smithi162 ceph-mon[110641]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' 2024-04-27T04:18:10.137 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:18:09 smithi162 ceph-mon[110641]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T04:18:10.137 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:18:09 smithi162 ceph-mon[110641]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T04:18:10.137 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:18:09 smithi162 ceph-mon[110641]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' 2024-04-27T04:18:10.137 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:18:09 smithi162 ceph-mon[110641]: Health check update: Degraded data redundancy: 41/231 objects degraded (17.749%), 15 pgs degraded (PG_DEGRADED) 2024-04-27T04:18:10.137 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:18:09 smithi162 ceph-mon[110641]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T04:18:10.138 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:18:09 smithi162 ceph-mon[110641]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "versions"}]: dispatch 2024-04-27T04:18:10.138 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:18:09 smithi162 ceph-mon[110641]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "versions"}]: dispatch 2024-04-27T04:18:10.138 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:18:09 smithi162 ceph-mon[110641]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "versions"}]: dispatch 2024-04-27T04:18:10.138 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:18:09 smithi162 ceph-mon[110641]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "osd ok-to-stop", "ids": ["5"], "max": 16}]: dispatch 2024-04-27T04:18:10.172 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:18:09 smithi181 ceph-mon[81134]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-04-27T04:18:10.172 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:18:09 smithi181 ceph-mon[81134]: osd.4 [v2:172.21.15.181:6808/1781129695,v1:172.21.15.181:6809/1781129695] boot 2024-04-27T04:18:10.172 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:18:09 smithi181 ceph-mon[81134]: osdmap e73: 6 total, 6 up, 6 in 2024-04-27T04:18:10.172 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:18:09 smithi181 ceph-mon[81134]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-27T04:18:10.172 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:18:09 smithi181 ceph-mon[81134]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' 2024-04-27T04:18:10.172 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:18:09 smithi181 ceph-mon[81134]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' 2024-04-27T04:18:10.172 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:18:09 smithi181 ceph-mon[81134]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T04:18:10.172 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:18:09 smithi181 ceph-mon[81134]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T04:18:10.172 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:18:09 smithi181 ceph-mon[81134]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' 2024-04-27T04:18:10.173 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:18:09 smithi181 ceph-mon[81134]: Health check update: Degraded data redundancy: 41/231 objects degraded (17.749%), 15 pgs degraded (PG_DEGRADED) 2024-04-27T04:18:10.173 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:18:09 smithi181 ceph-mon[81134]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T04:18:10.173 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:18:09 smithi181 ceph-mon[81134]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "versions"}]: dispatch 2024-04-27T04:18:10.173 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:18:09 smithi181 ceph-mon[81134]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "versions"}]: dispatch 2024-04-27T04:18:10.173 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:18:09 smithi181 ceph-mon[81134]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "versions"}]: dispatch 2024-04-27T04:18:10.173 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:18:09 smithi181 ceph-mon[81134]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "osd ok-to-stop", "ids": ["5"], "max": 16}]: dispatch 2024-04-27T04:18:11.136 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:18:10 smithi162 ceph-mon[110641]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["5"], "max": 16}]: dispatch 2024-04-27T04:18:11.136 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:18:10 smithi162 ceph-mon[110641]: Upgrade: unsafe to stop osd(s) at this time (11 PGs are or would become offline) 2024-04-27T04:18:11.136 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:18:10 smithi162 ceph-mon[110641]: pgmap v143: 65 pgs: 15 active+undersized+degraded, 19 active+undersized, 31 active+clean; 208 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 639 B/s rd, 1 op/s; 41/231 objects degraded (17.749%) 2024-04-27T04:18:11.136 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:18:10 smithi162 ceph-mon[110641]: osdmap e74: 6 total, 6 up, 6 in 2024-04-27T04:18:11.171 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:18:10 smithi181 ceph-mon[81134]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["5"], "max": 16}]: dispatch 2024-04-27T04:18:11.172 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:18:10 smithi181 ceph-mon[81134]: Upgrade: unsafe to stop osd(s) at this time (11 PGs are or would become offline) 2024-04-27T04:18:11.172 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:18:10 smithi181 ceph-mon[81134]: pgmap v143: 65 pgs: 15 active+undersized+degraded, 19 active+undersized, 31 active+clean; 208 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 639 B/s rd, 1 op/s; 41/231 objects degraded (17.749%) 2024-04-27T04:18:11.172 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:18:10 smithi181 ceph-mon[81134]: osdmap e74: 6 total, 6 up, 6 in 2024-04-27T04:18:13.136 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:18:12 smithi162 ceph-mon[110641]: pgmap v145: 65 pgs: 2 active, 10 active+undersized+degraded, 14 active+undersized, 39 active+clean; 208 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 682 B/s rd, 1 op/s; 23/231 objects degraded (9.957%) 2024-04-27T04:18:13.171 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:18:12 smithi181 ceph-mon[81134]: pgmap v145: 65 pgs: 2 active, 10 active+undersized+degraded, 14 active+undersized, 39 active+clean; 208 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 682 B/s rd, 1 op/s; 23/231 objects degraded (9.957%) 2024-04-27T04:18:14.136 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:18:13 smithi162 ceph-mon[110641]: pgmap v146: 65 pgs: 2 active, 1 active+undersized, 62 active+clean; 208 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 682 B/s rd, 2 op/s 2024-04-27T04:18:14.136 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:18:13 smithi162 ceph-mon[110641]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 23/231 objects degraded (9.957%), 10 pgs degraded) 2024-04-27T04:18:14.137 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:18:13 smithi162 ceph-mon[110641]: Cluster is now healthy 2024-04-27T04:18:14.171 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:18:13 smithi181 ceph-mon[81134]: pgmap v146: 65 pgs: 2 active, 1 active+undersized, 62 active+clean; 208 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 682 B/s rd, 2 op/s 2024-04-27T04:18:14.172 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:18:13 smithi181 ceph-mon[81134]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 23/231 objects degraded (9.957%), 10 pgs degraded) 2024-04-27T04:18:14.172 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:18:13 smithi181 ceph-mon[81134]: Cluster is now healthy 2024-04-27T04:18:17.136 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:18:16 smithi162 ceph-mon[110641]: pgmap v147: 65 pgs: 65 active+clean; 208 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 1.0 KiB/s rd, 2 op/s 2024-04-27T04:18:17.172 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:18:16 smithi181 ceph-mon[81134]: pgmap v147: 65 pgs: 65 active+clean; 208 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 1.0 KiB/s rd, 2 op/s 2024-04-27T04:18:18.922 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:18:18 smithi181 ceph-mon[81134]: pgmap v148: 65 pgs: 65 active+clean; 208 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 920 B/s rd, 1 op/s 2024-04-27T04:18:18.922 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:18:18 smithi181 ceph-mon[81134]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' 2024-04-27T04:18:18.922 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:18:18 smithi181 ceph-mon[81134]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-27T04:18:19.136 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:18:18 smithi162 ceph-mon[110641]: pgmap v148: 65 pgs: 65 active+clean; 208 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 920 B/s rd, 1 op/s 2024-04-27T04:18:19.136 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:18:18 smithi162 ceph-mon[110641]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' 2024-04-27T04:18:19.136 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:18:18 smithi162 ceph-mon[110641]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-27T04:18:21.136 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:18:20 smithi162 ceph-mon[110641]: pgmap v149: 65 pgs: 65 active+clean; 208 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 818 B/s rd, 1 op/s 2024-04-27T04:18:21.171 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:18:20 smithi181 ceph-mon[81134]: pgmap v149: 65 pgs: 65 active+clean; 208 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 818 B/s rd, 1 op/s 2024-04-27T04:18:23.136 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:18:22 smithi162 ceph-mon[110641]: pgmap v150: 65 pgs: 65 active+clean; 208 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-04-27T04:18:23.171 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:18:22 smithi181 ceph-mon[81134]: pgmap v150: 65 pgs: 65 active+clean; 208 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-04-27T04:18:23.912 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:18:23.912+0000 7f619a3bb700 1 -- 172.21.15.162:0/4258554657 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6194102e00 msgr2=0x7f61941031d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:18:23.912 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:18:23.912+0000 7f619a3bb700 1 --2- 172.21.15.162:0/4258554657 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6194102e00 0x7f61941031d0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f6184004940 tx=0x7f6184009e30 comp rx=0 tx=0).stop 2024-04-27T04:18:23.912 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:18:23.912+0000 7f619a3bb700 1 -- 172.21.15.162:0/4258554657 shutdown_connections 2024-04-27T04:18:23.912 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:18:23.912+0000 7f619a3bb700 1 --2- 172.21.15.162:0/4258554657 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f61941037a0 0x7f61940fce10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:18:23.912 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:18:23.912+0000 7f619a3bb700 1 --2- 172.21.15.162:0/4258554657 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6194102e00 0x7f61941031d0 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:18:23.912 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:18:23.912+0000 7f619a3bb700 1 -- 172.21.15.162:0/4258554657 >> 172.21.15.162:0/4258554657 conn(0x7f61940f89b0 msgr2=0x7f61940fadc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:18:23.912 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:18:23.912+0000 7f619a3bb700 1 -- 172.21.15.162:0/4258554657 shutdown_connections 2024-04-27T04:18:23.912 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:18:23.912+0000 7f619a3bb700 1 -- 172.21.15.162:0/4258554657 wait complete. 2024-04-27T04:18:23.912 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:18:23.912+0000 7f619a3bb700 1 Processor -- start 2024-04-27T04:18:23.913 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:18:23.912+0000 7f619a3bb700 1 -- start start 2024-04-27T04:18:23.913 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:18:23.913+0000 7f619a3bb700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6194102e00 0x7f6194113940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:18:23.913 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:18:23.913+0000 7f619a3bb700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f61941037a0 0x7f6194113e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:18:23.913 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:18:23.913+0000 7f619a3bb700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61941144d0 con 0x7f61941037a0 2024-04-27T04:18:23.913 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:18:23.913+0000 7f619a3bb700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f619410dad0 con 0x7f6194102e00 2024-04-27T04:18:23.913 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:18:23.913+0000 7f61937fe700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f61941037a0 0x7f6194113e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:18:23.913 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:18:23.913+0000 7f61937fe700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f61941037a0 0x7f6194113e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:53170/0 (socket says 172.21.15.162:53170) 2024-04-27T04:18:23.913 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:18:23.913+0000 7f61937fe700 1 -- 172.21.15.162:0/3431590950 learned_addr learned my addr 172.21.15.162:0/3431590950 (peer_addr_for_me v2:172.21.15.162:0/0) 2024-04-27T04:18:23.913 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:18:23.913+0000 7f6193fff700 1 --2- 172.21.15.162:0/3431590950 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6194102e00 0x7f6194113940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:18:23.913 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:18:23.913+0000 7f61937fe700 1 -- 172.21.15.162:0/3431590950 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6194102e00 msgr2=0x7f6194113940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:18:23.913 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:18:23.913+0000 7f61937fe700 1 --2- 172.21.15.162:0/3431590950 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6194102e00 0x7f6194113940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:18:23.914 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:18:23.913+0000 7f61937fe700 1 -- 172.21.15.162:0/3431590950 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6184005040 con 0x7f61941037a0 2024-04-27T04:18:23.914 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:18:23.913+0000 7f61937fe700 1 --2- 172.21.15.162:0/3431590950 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f61941037a0 0x7f6194113e80 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f6188002a50 tx=0x7f6188004600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:18:23.914 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:18:23.913+0000 7f6190ff9700 1 -- 172.21.15.162:0/3431590950 <== mon.0 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6188007cc0 con 0x7f61941037a0 2024-04-27T04:18:23.914 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:18:23.913+0000 7f619a3bb700 1 -- 172.21.15.162:0/3431590950 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f619410ddb0 con 0x7f61941037a0 2024-04-27T04:18:23.914 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:18:23.913+0000 7f6190ff9700 1 -- 172.21.15.162:0/3431590950 <== mon.0 v2:172.21.15.162:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f618800f040 con 0x7f61941037a0 2024-04-27T04:18:23.914 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:18:23.914+0000 7f6190ff9700 1 -- 172.21.15.162:0/3431590950 <== mon.0 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f618800d9c0 con 0x7f61941037a0 2024-04-27T04:18:23.914 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:18:23.914+0000 7f619a3bb700 1 -- 172.21.15.162:0/3431590950 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f619410e370 con 0x7f61941037a0 2024-04-27T04:18:23.915 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:18:23.914+0000 7f619a3bb700 1 -- 172.21.15.162:0/3431590950 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6194100110 con 0x7f61941037a0 2024-04-27T04:18:23.915 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:18:23.915+0000 7f6190ff9700 1 -- 172.21.15.162:0/3431590950 <== mon.0 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 39) v1 ==== 92747+0+0 (secure 0 0 0) 0x7f6188009730 con 0x7f61941037a0 2024-04-27T04:18:23.915 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:18:23.915+0000 7f6190ff9700 1 --2- 172.21.15.162:0/3431590950 >> [v2:172.21.15.162:6800/1396866108,v1:172.21.15.162:6801/1396866108] conn(0x7f617c06ec00 0x7f617c0710b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:18:23.915 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:18:23.916+0000 7f6190ff9700 1 -- 172.21.15.162:0/3431590950 <== mon.0 v2:172.21.15.162:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 6410+0+0 (secure 0 0 0) 0x7f618808e590 con 0x7f61941037a0 2024-04-27T04:18:23.915 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:18:23.916+0000 7f6193fff700 1 --2- 172.21.15.162:0/3431590950 >> [v2:172.21.15.162:6800/1396866108,v1:172.21.15.162:6801/1396866108] conn(0x7f617c06ec00 0x7f617c0710b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:18:23.915 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:18:23.916+0000 7f6193fff700 1 --2- 172.21.15.162:0/3431590950 >> [v2:172.21.15.162:6800/1396866108,v1:172.21.15.162:6801/1396866108] conn(0x7f617c06ec00 0x7f617c0710b0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f6184004f90 tx=0x7f6184009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:18:23.918 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:18:23.920+0000 7f6190ff9700 1 -- 172.21.15.162:0/3431590950 <== mon.0 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7f618805aec0 con 0x7f61941037a0 2024-04-27T04:18:24.135 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:18:24.137+0000 7f619a3bb700 1 -- 172.21.15.162:0/3431590950 --> [v2:172.21.15.162:6800/1396866108,v1:172.21.15.162:6801/1396866108] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f619410f000 con 0x7f617c06ec00 2024-04-27T04:18:24.135 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:18:23 smithi162 ceph-mon[110641]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' 2024-04-27T04:18:24.136 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:18:24.138+0000 7f6190ff9700 1 -- 172.21.15.162:0/3431590950 <== mgr.34104 v2:172.21.15.162:6800/1396866108 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+401 (secure 0 0 0) 0x7f619410f000 con 0x7f617c06ec00 2024-04-27T04:18:24.138 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:18:24.140+0000 7f619a3bb700 1 -- 172.21.15.162:0/3431590950 >> [v2:172.21.15.162:6800/1396866108,v1:172.21.15.162:6801/1396866108] conn(0x7f617c06ec00 msgr2=0x7f617c0710b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:18:24.138 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:18:24.140+0000 7f619a3bb700 1 --2- 172.21.15.162:0/3431590950 >> [v2:172.21.15.162:6800/1396866108,v1:172.21.15.162:6801/1396866108] conn(0x7f617c06ec00 0x7f617c0710b0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f6184004f90 tx=0x7f6184009e30 comp rx=0 tx=0).stop 2024-04-27T04:18:24.139 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:18:24.140+0000 7f619a3bb700 1 -- 172.21.15.162:0/3431590950 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f61941037a0 msgr2=0x7f6194113e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:18:24.139 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:18:24.140+0000 7f619a3bb700 1 --2- 172.21.15.162:0/3431590950 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f61941037a0 0x7f6194113e80 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f6188002a50 tx=0x7f6188004600 comp rx=0 tx=0).stop 2024-04-27T04:18:24.139 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:18:24.140+0000 7f619a3bb700 1 -- 172.21.15.162:0/3431590950 shutdown_connections 2024-04-27T04:18:24.139 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:18:24.140+0000 7f619a3bb700 1 --2- 172.21.15.162:0/3431590950 >> [v2:172.21.15.162:6800/1396866108,v1:172.21.15.162:6801/1396866108] conn(0x7f617c06ec00 0x7f617c0710b0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:18:24.139 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:18:24.140+0000 7f619a3bb700 1 --2- 172.21.15.162:0/3431590950 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6194102e00 0x7f6194113940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:18:24.139 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:18:24.140+0000 7f619a3bb700 1 --2- 172.21.15.162:0/3431590950 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f61941037a0 0x7f6194113e80 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:18:24.139 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:18:24.140+0000 7f619a3bb700 1 -- 172.21.15.162:0/3431590950 >> 172.21.15.162:0/3431590950 conn(0x7f61940f89b0 msgr2=0x7f6194107390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:18:24.139 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:18:24.141+0000 7f619a3bb700 1 -- 172.21.15.162:0/3431590950 shutdown_connections 2024-04-27T04:18:24.139 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:18:24.141+0000 7f619a3bb700 1 -- 172.21.15.162:0/3431590950 wait complete. 2024-04-27T04:18:24.149 INFO:teuthology.orchestra.run.smithi162.stdout:true 2024-04-27T04:18:24.171 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:18:23 smithi181 ceph-mon[81134]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' 2024-04-27T04:18:24.246 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:18:24.244+0000 7f7b9c9a0700 1 -- 172.21.15.162:0/219905806 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7b98102ce0 msgr2=0x7f7b981030b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:18:24.246 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:18:24.244+0000 7f7b9c9a0700 1 --2- 172.21.15.162:0/219905806 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7b98102ce0 0x7f7b981030b0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f7b88004940 tx=0x7f7b88009e30 comp rx=0 tx=0).stop 2024-04-27T04:18:24.247 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:18:24.244+0000 7f7b9c9a0700 1 -- 172.21.15.162:0/219905806 shutdown_connections 2024-04-27T04:18:24.247 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:18:24.244+0000 7f7b9c9a0700 1 --2- 172.21.15.162:0/219905806 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7b98103680 0x7f7b980fb2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:18:24.247 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:18:24.244+0000 7f7b9c9a0700 1 --2- 172.21.15.162:0/219905806 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7b98102ce0 0x7f7b981030b0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:18:24.247 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:18:24.244+0000 7f7b9c9a0700 1 -- 172.21.15.162:0/219905806 >> 172.21.15.162:0/219905806 conn(0x7f7b980712a0 msgr2=0x7f7b980716a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:18:24.247 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:18:24.244+0000 7f7b9c9a0700 1 -- 172.21.15.162:0/219905806 shutdown_connections 2024-04-27T04:18:24.247 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:18:24.244+0000 7f7b9c9a0700 1 -- 172.21.15.162:0/219905806 wait complete. 2024-04-27T04:18:24.248 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:18:24.244+0000 7f7b9c9a0700 1 Processor -- start 2024-04-27T04:18:24.248 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:18:24.245+0000 7f7b9c9a0700 1 -- start start 2024-04-27T04:18:24.248 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:18:24.245+0000 7f7b9c9a0700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7b98102ce0 0x7f7b980fc4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:18:24.248 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:18:24.245+0000 7f7b9c9a0700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7b98103680 0x7f7b980fca10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:18:24.248 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:18:24.245+0000 7f7b9c9a0700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b980fcfa0 con 0x7f7b98102ce0 2024-04-27T04:18:24.248 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:18:24.245+0000 7f7b9c9a0700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b980fd0e0 con 0x7f7b98103680 2024-04-27T04:18:24.248 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:18:24.245+0000 7f7b977fe700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7b98102ce0 0x7f7b980fc4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:18:24.248 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:18:24.245+0000 7f7b977fe700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7b98102ce0 0x7f7b980fc4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:53192/0 (socket says 172.21.15.162:53192) 2024-04-27T04:18:24.248 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:18:24.245+0000 7f7b977fe700 1 -- 172.21.15.162:0/1229046305 learned_addr learned my addr 172.21.15.162:0/1229046305 (peer_addr_for_me v2:172.21.15.162:0/0) 2024-04-27T04:18:24.248 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:18:24.245+0000 7f7b96ffd700 1 --2- 172.21.15.162:0/1229046305 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7b98103680 0x7f7b980fca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:18:24.248 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:18:24.245+0000 7f7b977fe700 1 -- 172.21.15.162:0/1229046305 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7b98103680 msgr2=0x7f7b980fca10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:18:24.249 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:18:24.245+0000 7f7b977fe700 1 --2- 172.21.15.162:0/1229046305 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7b98103680 0x7f7b980fca10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:18:24.249 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:18:24.245+0000 7f7b977fe700 1 -- 172.21.15.162:0/1229046305 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b88005040 con 0x7f7b98102ce0 2024-04-27T04:18:24.249 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:18:24.245+0000 7f7b96ffd700 1 --2- 172.21.15.162:0/1229046305 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7b98103680 0x7f7b980fca10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-27T04:18:24.249 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:18:24.245+0000 7f7b977fe700 1 --2- 172.21.15.162:0/1229046305 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7b98102ce0 0x7f7b980fc4d0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f7b8800b3b0 tx=0x7f7b88009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:18:24.249 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:18:24.245+0000 7f7b7ffff700 1 -- 172.21.15.162:0/1229046305 <== mon.0 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7b8800e040 con 0x7f7b98102ce0 2024-04-27T04:18:24.249 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:18:24.245+0000 7f7b9c9a0700 1 -- 172.21.15.162:0/1229046305 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7b980fd330 con 0x7f7b98102ce0 2024-04-27T04:18:24.249 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:18:24.245+0000 7f7b7ffff700 1 -- 172.21.15.162:0/1229046305 <== mon.0 v2:172.21.15.162:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f7b8800bbc0 con 0x7f7b98102ce0 2024-04-27T04:18:24.249 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:18:24.245+0000 7f7b7ffff700 1 -- 172.21.15.162:0/1229046305 <== mon.0 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7b8801c3f0 con 0x7f7b98102ce0 2024-04-27T04:18:24.249 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:18:24.246+0000 7f7b9c9a0700 1 -- 172.21.15.162:0/1229046305 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7b9811a0e0 con 0x7f7b98102ce0 2024-04-27T04:18:24.249 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:18:24.247+0000 7f7b9c9a0700 1 -- 172.21.15.162:0/1229046305 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7b98100020 con 0x7f7b98102ce0 2024-04-27T04:18:24.249 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:18:24.247+0000 7f7b7ffff700 1 -- 172.21.15.162:0/1229046305 <== mon.0 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 39) v1 ==== 92747+0+0 (secure 0 0 0) 0x7f7b8800bd30 con 0x7f7b98102ce0 2024-04-27T04:18:24.250 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:18:24.247+0000 7f7b7ffff700 1 --2- 172.21.15.162:0/1229046305 >> [v2:172.21.15.162:6800/1396866108,v1:172.21.15.162:6801/1396866108] conn(0x7f7b8006ec00 0x7f7b800710b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:18:24.250 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:18:24.247+0000 7f7b7ffff700 1 -- 172.21.15.162:0/1229046305 <== mon.0 v2:172.21.15.162:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 6410+0+0 (secure 0 0 0) 0x7f7b8808faf0 con 0x7f7b98102ce0 2024-04-27T04:18:24.250 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:18:24.247+0000 7f7b96ffd700 1 --2- 172.21.15.162:0/1229046305 >> [v2:172.21.15.162:6800/1396866108,v1:172.21.15.162:6801/1396866108] conn(0x7f7b8006ec00 0x7f7b800710b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:18:24.250 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:18:24.248+0000 7f7b96ffd700 1 --2- 172.21.15.162:0/1229046305 >> [v2:172.21.15.162:6800/1396866108,v1:172.21.15.162:6801/1396866108] conn(0x7f7b8006ec00 0x7f7b800710b0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f7b8c009fd0 tx=0x7f7b8c00b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:18:24.250 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:18:24.252+0000 7f7b7ffff700 1 -- 172.21.15.162:0/1229046305 <== mon.0 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7f7b880588f0 con 0x7f7b98102ce0 2024-04-27T04:18:24.472 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:18:24.474+0000 7f7b9c9a0700 1 -- 172.21.15.162:0/1229046305 --> [v2:172.21.15.162:6800/1396866108,v1:172.21.15.162:6801/1396866108] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f7b98109e90 con 0x7f7b8006ec00 2024-04-27T04:18:24.473 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:18:24.476+0000 7f7b7ffff700 1 -- 172.21.15.162:0/1229046305 <== mgr.34104 v2:172.21.15.162:6800/1396866108 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+401 (secure 0 0 0) 0x7f7b98109e90 con 0x7f7b8006ec00 2024-04-27T04:18:24.476 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:18:24.478+0000 7f7b9c9a0700 1 -- 172.21.15.162:0/1229046305 >> [v2:172.21.15.162:6800/1396866108,v1:172.21.15.162:6801/1396866108] conn(0x7f7b8006ec00 msgr2=0x7f7b800710b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:18:24.476 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:18:24.478+0000 7f7b9c9a0700 1 --2- 172.21.15.162:0/1229046305 >> [v2:172.21.15.162:6800/1396866108,v1:172.21.15.162:6801/1396866108] conn(0x7f7b8006ec00 0x7f7b800710b0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f7b8c009fd0 tx=0x7f7b8c00b040 comp rx=0 tx=0).stop 2024-04-27T04:18:24.476 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:18:24.478+0000 7f7b9c9a0700 1 -- 172.21.15.162:0/1229046305 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7b98102ce0 msgr2=0x7f7b980fc4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:18:24.476 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:18:24.478+0000 7f7b9c9a0700 1 --2- 172.21.15.162:0/1229046305 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7b98102ce0 0x7f7b980fc4d0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f7b8800b3b0 tx=0x7f7b88009e30 comp rx=0 tx=0).stop 2024-04-27T04:18:24.476 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:18:24.478+0000 7f7b9c9a0700 1 -- 172.21.15.162:0/1229046305 shutdown_connections 2024-04-27T04:18:24.477 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:18:24.478+0000 7f7b9c9a0700 1 --2- 172.21.15.162:0/1229046305 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7b98102ce0 0x7f7b980fc4d0 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:18:24.477 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:18:24.478+0000 7f7b9c9a0700 1 --2- 172.21.15.162:0/1229046305 >> [v2:172.21.15.162:6800/1396866108,v1:172.21.15.162:6801/1396866108] conn(0x7f7b8006ec00 0x7f7b800710b0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:18:24.477 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:18:24.478+0000 7f7b9c9a0700 1 --2- 172.21.15.162:0/1229046305 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7b98103680 0x7f7b980fca10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:18:24.477 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:18:24.478+0000 7f7b9c9a0700 1 -- 172.21.15.162:0/1229046305 >> 172.21.15.162:0/1229046305 conn(0x7f7b980712a0 msgr2=0x7f7b980fab00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:18:24.477 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:18:24.478+0000 7f7b9c9a0700 1 -- 172.21.15.162:0/1229046305 shutdown_connections 2024-04-27T04:18:24.477 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:18:24.478+0000 7f7b9c9a0700 1 -- 172.21.15.162:0/1229046305 wait complete. 2024-04-27T04:18:24.580 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:18:24.580+0000 7fb42a7ec700 1 -- 172.21.15.162:0/4283065970 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb4240fbbd0 msgr2=0x7fb4240fbfa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:18:24.580 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:18:24.580+0000 7fb42a7ec700 1 --2- 172.21.15.162:0/4283065970 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb4240fbbd0 0x7fb4240fbfa0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fb414004940 tx=0x7fb414009e30 comp rx=0 tx=0).stop 2024-04-27T04:18:24.580 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:18:24.580+0000 7fb42a7ec700 1 -- 172.21.15.162:0/4283065970 shutdown_connections 2024-04-27T04:18:24.580 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:18:24.580+0000 7fb42a7ec700 1 --2- 172.21.15.162:0/4283065970 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb4240fc4e0 0x7fb424109240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:18:24.581 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:18:24.580+0000 7fb42a7ec700 1 --2- 172.21.15.162:0/4283065970 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb4240fbbd0 0x7fb4240fbfa0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:18:24.581 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:18:24.580+0000 7fb42a7ec700 1 -- 172.21.15.162:0/4283065970 >> 172.21.15.162:0/4283065970 conn(0x7fb4240f7860 msgr2=0x7fb4240f9c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:18:24.581 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:18:24.580+0000 7fb42a7ec700 1 -- 172.21.15.162:0/4283065970 shutdown_connections 2024-04-27T04:18:24.581 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:18:24.580+0000 7fb42a7ec700 1 -- 172.21.15.162:0/4283065970 wait complete. 2024-04-27T04:18:24.581 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:18:24.580+0000 7fb42a7ec700 1 Processor -- start 2024-04-27T04:18:24.581 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:18:24.581+0000 7fb42a7ec700 1 -- start start 2024-04-27T04:18:24.581 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:18:24.581+0000 7fb42a7ec700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb4240fbbd0 0x7fb424115a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:18:24.582 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:18:24.581+0000 7fb42a7ec700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb4240fc4e0 0x7fb424115fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:18:24.582 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:18:24.581+0000 7fb42a7ec700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb424116610 con 0x7fb4240fc4e0 2024-04-27T04:18:24.582 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:18:24.581+0000 7fb42a7ec700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb42410fbf0 con 0x7fb4240fbbd0 2024-04-27T04:18:24.582 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:18:24.581+0000 7fb4237fe700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb4240fc4e0 0x7fb424115fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:18:24.582 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:18:24.581+0000 7fb4237fe700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb4240fc4e0 0x7fb424115fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:53206/0 (socket says 172.21.15.162:53206) 2024-04-27T04:18:24.582 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:18:24.581+0000 7fb4237fe700 1 -- 172.21.15.162:0/2479801094 learned_addr learned my addr 172.21.15.162:0/2479801094 (peer_addr_for_me v2:172.21.15.162:0/0) 2024-04-27T04:18:24.582 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:18:24.581+0000 7fb423fff700 1 --2- 172.21.15.162:0/2479801094 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb4240fbbd0 0x7fb424115a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:18:24.582 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:18:24.581+0000 7fb4237fe700 1 -- 172.21.15.162:0/2479801094 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb4240fbbd0 msgr2=0x7fb424115a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:18:24.582 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:18:24.581+0000 7fb4237fe700 1 --2- 172.21.15.162:0/2479801094 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb4240fbbd0 0x7fb424115a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:18:24.582 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:18:24.581+0000 7fb4237fe700 1 -- 172.21.15.162:0/2479801094 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb414005040 con 0x7fb4240fc4e0 2024-04-27T04:18:24.583 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:18:24.581+0000 7fb4237fe700 1 --2- 172.21.15.162:0/2479801094 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb4240fc4e0 0x7fb424115fc0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fb418002a50 tx=0x7fb4180046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:18:24.583 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:18:24.581+0000 7fb420ff9700 1 -- 172.21.15.162:0/2479801094 <== mon.0 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb418007cc0 con 0x7fb4240fc4e0 2024-04-27T04:18:24.583 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:18:24.582+0000 7fb420ff9700 1 -- 172.21.15.162:0/2479801094 <== mon.0 v2:172.21.15.162:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fb41800f040 con 0x7fb4240fc4e0 2024-04-27T04:18:24.583 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:18:24.582+0000 7fb42a7ec700 1 -- 172.21.15.162:0/2479801094 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb42410fed0 con 0x7fb4240fc4e0 2024-04-27T04:18:24.583 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:18:24.582+0000 7fb420ff9700 1 -- 172.21.15.162:0/2479801094 <== mon.0 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb41800da30 con 0x7fb4240fc4e0 2024-04-27T04:18:24.584 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:18:24.582+0000 7fb42a7ec700 1 -- 172.21.15.162:0/2479801094 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb424110490 con 0x7fb4240fc4e0 2024-04-27T04:18:24.585 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:18:24.583+0000 7fb42a7ec700 1 -- 172.21.15.162:0/2479801094 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb424066af0 con 0x7fb4240fc4e0 2024-04-27T04:18:24.585 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:18:24.583+0000 7fb420ff9700 1 -- 172.21.15.162:0/2479801094 <== mon.0 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 39) v1 ==== 92747+0+0 (secure 0 0 0) 0x7fb41800dbb0 con 0x7fb4240fc4e0 2024-04-27T04:18:24.585 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:18:24.584+0000 7fb420ff9700 1 --2- 172.21.15.162:0/2479801094 >> [v2:172.21.15.162:6800/1396866108,v1:172.21.15.162:6801/1396866108] conn(0x7fb40c06ec00 0x7fb40c0710b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:18:24.585 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:18:24.584+0000 7fb423fff700 1 --2- 172.21.15.162:0/2479801094 >> [v2:172.21.15.162:6800/1396866108,v1:172.21.15.162:6801/1396866108] conn(0x7fb40c06ec00 0x7fb40c0710b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:18:24.585 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:18:24.584+0000 7fb420ff9700 1 -- 172.21.15.162:0/2479801094 <== mon.0 v2:172.21.15.162:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 6410+0+0 (secure 0 0 0) 0x7fb41808e5d0 con 0x7fb4240fc4e0 2024-04-27T04:18:24.585 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:18:24.584+0000 7fb423fff700 1 --2- 172.21.15.162:0/2479801094 >> [v2:172.21.15.162:6800/1396866108,v1:172.21.15.162:6801/1396866108] conn(0x7fb40c06ec00 0x7fb40c0710b0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fb414004f90 tx=0x7fb414009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:18:24.586 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:18:24.588+0000 7fb420ff9700 1 -- 172.21.15.162:0/2479801094 <== mon.0 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7fb41805af00 con 0x7fb4240fc4e0 2024-04-27T04:18:24.795 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:18:24.797+0000 7fb42a7ec700 1 -- 172.21.15.162:0/2479801094 --> [v2:172.21.15.162:6800/1396866108,v1:172.21.15.162:6801/1396866108] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7fb42410c020 con 0x7fb40c06ec00 2024-04-27T04:18:24.795 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:18:24 smithi162 ceph-mon[110641]: pgmap v151: 65 pgs: 65 active+clean; 208 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 767 B/s rd, 1 op/s 2024-04-27T04:18:24.795 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:18:24 smithi162 ceph-mon[110641]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "osd ok-to-stop", "ids": ["5"], "max": 16}]: dispatch 2024-04-27T04:18:24.805 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:18:24.807+0000 7fb420ff9700 1 -- 172.21.15.162:0/2479801094 <== mgr.34104 v2:172.21.15.162:6800/1396866108 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3498 (secure 0 0 0) 0x7fb42410c020 con 0x7fb40c06ec00 2024-04-27T04:18:24.805 INFO:teuthology.orchestra.run.smithi162.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-27T04:18:24.805 INFO:teuthology.orchestra.run.smithi162.stdout:alertmanager.smithi162 smithi162 *:9093,9094 running (5m) 86s ago 16m 27.2M - 0.25.0 c8568f914cd2 40e334659017 2024-04-27T04:18:24.805 INFO:teuthology.orchestra.run.smithi162.stdout:crash.smithi162 smithi162 running (3m) 86s ago 16m 6660k - 19.0.0-3253-g4e213a58 67e414ff4bc4 1aa8f4d5a28b 2024-04-27T04:18:24.805 INFO:teuthology.orchestra.run.smithi162.stdout:crash.smithi181 smithi181 running (3m) 20s ago 15m 6656k - 19.0.0-3253-g4e213a58 67e414ff4bc4 832d9cbc5c30 2024-04-27T04:18:24.805 INFO:teuthology.orchestra.run.smithi162.stdout:grafana.smithi162 smithi162 *:3000 running (5m) 86s ago 16m 80.5M - 9.4.12 2bacad6d85d8 3d78c478f03f 2024-04-27T04:18:24.805 INFO:teuthology.orchestra.run.smithi162.stdout:mds.cephfs.smithi162.gzlyjj smithi162 running (11m) 86s ago 11m 14.3M - 17.2.7-907-g2be3efa9 1c5834633715 27e5b030dd32 2024-04-27T04:18:24.805 INFO:teuthology.orchestra.run.smithi162.stdout:mds.cephfs.smithi162.xcnapk smithi162 running (11m) 86s ago 11m 100M - 17.2.7-907-g2be3efa9 1c5834633715 38f9030df029 2024-04-27T04:18:24.805 INFO:teuthology.orchestra.run.smithi162.stdout:mds.cephfs.smithi181.bkkcdm smithi181 running (11m) 20s ago 11m 15.9M - 17.2.7-907-g2be3efa9 1c5834633715 af55afca1263 2024-04-27T04:18:24.805 INFO:teuthology.orchestra.run.smithi162.stdout:mds.cephfs.smithi181.dzomgh smithi181 running (11m) 20s ago 11m 143M - 17.2.7-907-g2be3efa9 1c5834633715 6733296597f3 2024-04-27T04:18:24.805 INFO:teuthology.orchestra.run.smithi162.stdout:mgr.smithi162.oennfn smithi162 *:8443,9283,8765 running (7m) 86s ago 18m 505M - 19.0.0-3253-g4e213a58 67e414ff4bc4 58a748d3637b 2024-04-27T04:18:24.806 INFO:teuthology.orchestra.run.smithi162.stdout:mgr.smithi181.vtwuhz smithi181 *:8443,9283,8765 running (6m) 20s ago 15m 451M - 19.0.0-3253-g4e213a58 67e414ff4bc4 813584297b1d 2024-04-27T04:18:24.806 INFO:teuthology.orchestra.run.smithi162.stdout:mon.smithi162 smithi162 running (4m) 86s ago 18m 50.4M 2048M 19.0.0-3253-g4e213a58 67e414ff4bc4 375adaf4468f 2024-04-27T04:18:24.806 INFO:teuthology.orchestra.run.smithi162.stdout:mon.smithi181 smithi181 running (3m) 20s ago 15m 35.7M 2048M 19.0.0-3253-g4e213a58 67e414ff4bc4 77b71f6d522f 2024-04-27T04:18:24.806 INFO:teuthology.orchestra.run.smithi162.stdout:node-exporter.smithi162 smithi162 *:9100 running (6m) 86s ago 16m 15.4M - 1.5.0 0da6a335fe13 5c4c5ea9c1d7 2024-04-27T04:18:24.806 INFO:teuthology.orchestra.run.smithi162.stdout:node-exporter.smithi181 smithi181 *:9100 running (6m) 20s ago 15m 15.6M - 1.5.0 0da6a335fe13 9067f14c0519 2024-04-27T04:18:24.806 INFO:teuthology.orchestra.run.smithi162.stdout:osd.0 smithi162 running (2m) 86s ago 15m 87.7M 1557M 19.0.0-3253-g4e213a58 67e414ff4bc4 2744aff3cd88 2024-04-27T04:18:24.806 INFO:teuthology.orchestra.run.smithi162.stdout:osd.1 smithi162 running (2m) 86s ago 14m 93.8M 1557M 19.0.0-3253-g4e213a58 67e414ff4bc4 668e16faad62 2024-04-27T04:18:24.806 INFO:teuthology.orchestra.run.smithi162.stdout:osd.2 smithi162 running (92s) 86s ago 14m 15.1M 1557M 19.0.0-3253-g4e213a58 67e414ff4bc4 d24878d461e7 2024-04-27T04:18:24.806 INFO:teuthology.orchestra.run.smithi162.stdout:osd.3 smithi181 running (56s) 20s ago 13m 111M 2581M 19.0.0-3253-g4e213a58 67e414ff4bc4 ab473cfd54d2 2024-04-27T04:18:24.806 INFO:teuthology.orchestra.run.smithi162.stdout:osd.4 smithi181 running (23s) 20s ago 13m 12.7M 2581M 19.0.0-3253-g4e213a58 67e414ff4bc4 de3e10f2d26d 2024-04-27T04:18:24.806 INFO:teuthology.orchestra.run.smithi162.stdout:osd.5 smithi181 running (12m) 20s ago 12m 223M 2581M 17.2.7-907-g2be3efa9 1c5834633715 f47a1daac1ea 2024-04-27T04:18:24.806 INFO:teuthology.orchestra.run.smithi162.stdout:prometheus.smithi162 smithi162 *:9095 running (5m) 86s ago 16m 67.6M - 2.43.0 a07b618ecd1d f30657a43324 2024-04-27T04:18:24.807 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:18:24.809+0000 7fb42a7ec700 1 -- 172.21.15.162:0/2479801094 >> [v2:172.21.15.162:6800/1396866108,v1:172.21.15.162:6801/1396866108] conn(0x7fb40c06ec00 msgr2=0x7fb40c0710b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:18:24.808 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:18:24.809+0000 7fb42a7ec700 1 --2- 172.21.15.162:0/2479801094 >> [v2:172.21.15.162:6800/1396866108,v1:172.21.15.162:6801/1396866108] conn(0x7fb40c06ec00 0x7fb40c0710b0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fb414004f90 tx=0x7fb414009e30 comp rx=0 tx=0).stop 2024-04-27T04:18:24.808 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:18:24.809+0000 7fb42a7ec700 1 -- 172.21.15.162:0/2479801094 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb4240fc4e0 msgr2=0x7fb424115fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:18:24.808 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:18:24.809+0000 7fb42a7ec700 1 --2- 172.21.15.162:0/2479801094 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb4240fc4e0 0x7fb424115fc0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fb418002a50 tx=0x7fb4180046b0 comp rx=0 tx=0).stop 2024-04-27T04:18:24.808 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:18:24.809+0000 7fb42a7ec700 1 -- 172.21.15.162:0/2479801094 shutdown_connections 2024-04-27T04:18:24.808 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:18:24.809+0000 7fb42a7ec700 1 --2- 172.21.15.162:0/2479801094 >> [v2:172.21.15.162:6800/1396866108,v1:172.21.15.162:6801/1396866108] conn(0x7fb40c06ec00 0x7fb40c0710b0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:18:24.808 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:18:24.809+0000 7fb42a7ec700 1 --2- 172.21.15.162:0/2479801094 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb4240fbbd0 0x7fb424115a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:18:24.808 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:18:24.809+0000 7fb42a7ec700 1 --2- 172.21.15.162:0/2479801094 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb4240fc4e0 0x7fb424115fc0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:18:24.808 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:18:24.810+0000 7fb42a7ec700 1 -- 172.21.15.162:0/2479801094 >> 172.21.15.162:0/2479801094 conn(0x7fb4240f7860 msgr2=0x7fb4240f7ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:18:24.809 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:18:24.810+0000 7fb42a7ec700 1 -- 172.21.15.162:0/2479801094 shutdown_connections 2024-04-27T04:18:24.809 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:18:24.810+0000 7fb42a7ec700 1 -- 172.21.15.162:0/2479801094 wait complete. 2024-04-27T04:18:24.910 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:18:24.910+0000 7f359471b700 1 -- 172.21.15.162:0/704777068 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f358c101d90 msgr2=0x7f358c102160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:18:24.911 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:18:24.910+0000 7f359471b700 1 --2- 172.21.15.162:0/704777068 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f358c101d90 0x7f358c102160 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f357c004940 tx=0x7f357c009e30 comp rx=0 tx=0).stop 2024-04-27T04:18:24.911 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:18:24.910+0000 7f359471b700 1 -- 172.21.15.162:0/704777068 shutdown_connections 2024-04-27T04:18:24.911 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:18:24.910+0000 7f359471b700 1 --2- 172.21.15.162:0/704777068 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f358c0fbd40 0x7f358c0fc1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:18:24.911 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:18:24.910+0000 7f359471b700 1 --2- 172.21.15.162:0/704777068 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f358c101d90 0x7f358c102160 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:18:24.911 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:18:24.910+0000 7f359471b700 1 -- 172.21.15.162:0/704777068 >> 172.21.15.162:0/704777068 conn(0x7f358c071d00 msgr2=0x7f358c072100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:18:24.911 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:18:24.910+0000 7f359471b700 1 -- 172.21.15.162:0/704777068 shutdown_connections 2024-04-27T04:18:24.911 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:18:24.910+0000 7f359471b700 1 -- 172.21.15.162:0/704777068 wait complete. 2024-04-27T04:18:24.911 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:18:24.911+0000 7f359471b700 1 Processor -- start 2024-04-27T04:18:24.911 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:18:24.911+0000 7f359471b700 1 -- start start 2024-04-27T04:18:24.911 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:18:24.911+0000 7f359471b700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f358c0fbd40 0x7f358c113b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:18:24.911 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:18:24.911+0000 7f359471b700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f358c101d90 0x7f358c114060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:18:24.911 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:18:24.911+0000 7f359471b700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f358c1146b0 con 0x7f358c101d90 2024-04-27T04:18:24.911 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:18:24.911+0000 7f359471b700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f358c10daa0 con 0x7f358c0fbd40 2024-04-27T04:18:24.912 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:18:24.911+0000 7f3591cb6700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f358c101d90 0x7f358c114060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:18:24.912 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:18:24.911+0000 7f3591cb6700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f358c101d90 0x7f358c114060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:53232/0 (socket says 172.21.15.162:53232) 2024-04-27T04:18:24.912 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:18:24.911+0000 7f3591cb6700 1 -- 172.21.15.162:0/3691399106 learned_addr learned my addr 172.21.15.162:0/3691399106 (peer_addr_for_me v2:172.21.15.162:0/0) 2024-04-27T04:18:24.912 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:18:24.911+0000 7f35924b7700 1 --2- 172.21.15.162:0/3691399106 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f358c0fbd40 0x7f358c113b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:18:24.912 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:18:24.911+0000 7f3591cb6700 1 -- 172.21.15.162:0/3691399106 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f358c0fbd40 msgr2=0x7f358c113b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:18:24.912 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:18:24.911+0000 7f3591cb6700 1 --2- 172.21.15.162:0/3691399106 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f358c0fbd40 0x7f358c113b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:18:24.912 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:18:24.911+0000 7f3591cb6700 1 -- 172.21.15.162:0/3691399106 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f357c005040 con 0x7f358c101d90 2024-04-27T04:18:24.912 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:18:24.911+0000 7f3591cb6700 1 --2- 172.21.15.162:0/3691399106 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f358c101d90 0x7f358c114060 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f3588002a80 tx=0x7f35880046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:18:24.912 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:18:24.911+0000 7f3582ffd700 1 -- 172.21.15.162:0/3691399106 <== mon.0 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3588007cc0 con 0x7f358c101d90 2024-04-27T04:18:24.912 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:18:24.912+0000 7f359471b700 1 -- 172.21.15.162:0/3691399106 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f358c10dd80 con 0x7f358c101d90 2024-04-27T04:18:24.912 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:18:24.912+0000 7f3582ffd700 1 -- 172.21.15.162:0/3691399106 <== mon.0 v2:172.21.15.162:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f358800f040 con 0x7f358c101d90 2024-04-27T04:18:24.912 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:18:24.912+0000 7f3582ffd700 1 -- 172.21.15.162:0/3691399106 <== mon.0 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f358800dac0 con 0x7f358c101d90 2024-04-27T04:18:24.912 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:18:24.912+0000 7f359471b700 1 -- 172.21.15.162:0/3691399106 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f358c10e340 con 0x7f358c101d90 2024-04-27T04:18:24.913 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:18:24.913+0000 7f359471b700 1 -- 172.21.15.162:0/3691399106 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f358c0fd430 con 0x7f358c101d90 2024-04-27T04:18:24.913 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:18:24.913+0000 7f3582ffd700 1 -- 172.21.15.162:0/3691399106 <== mon.0 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 39) v1 ==== 92747+0+0 (secure 0 0 0) 0x7f358800dc20 con 0x7f358c101d90 2024-04-27T04:18:24.913 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:18:24.913+0000 7f3582ffd700 1 --2- 172.21.15.162:0/3691399106 >> [v2:172.21.15.162:6800/1396866108,v1:172.21.15.162:6801/1396866108] conn(0x7f357806ec00 0x7f35780710b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:18:24.913 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:18:24.913+0000 7f3582ffd700 1 -- 172.21.15.162:0/3691399106 <== mon.0 v2:172.21.15.162:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 6410+0+0 (secure 0 0 0) 0x7f358808e5c0 con 0x7f358c101d90 2024-04-27T04:18:24.913 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:18:24.914+0000 7f35924b7700 1 --2- 172.21.15.162:0/3691399106 >> [v2:172.21.15.162:6800/1396866108,v1:172.21.15.162:6801/1396866108] conn(0x7f357806ec00 0x7f35780710b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:18:24.913 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:18:24.914+0000 7f35924b7700 1 --2- 172.21.15.162:0/3691399106 >> [v2:172.21.15.162:6800/1396866108,v1:172.21.15.162:6801/1396866108] conn(0x7f357806ec00 0x7f35780710b0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f357c004890 tx=0x7f357c009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:18:24.916 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:18:24.918+0000 7f3582ffd700 1 -- 172.21.15.162:0/3691399106 <== mon.0 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7f358805aef0 con 0x7f358c101d90 2024-04-27T04:18:24.975 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:18:24 smithi181 ceph-mon[81134]: pgmap v151: 65 pgs: 65 active+clean; 208 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 767 B/s rd, 1 op/s 2024-04-27T04:18:24.975 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:18:24 smithi181 ceph-mon[81134]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "osd ok-to-stop", "ids": ["5"], "max": 16}]: dispatch 2024-04-27T04:18:25.197 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:18:25.199+0000 7f359471b700 1 -- 172.21.15.162:0/3691399106 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f358c10f1a0 con 0x7f358c101d90 2024-04-27T04:18:25.198 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:18:25.200+0000 7f3582ffd700 1 -- 172.21.15.162:0/3691399106 <== mon.0 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+839 (secure 0 0 0) 0x7f358805aa80 con 0x7f358c101d90 2024-04-27T04:18:25.198 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-04-27T04:18:25.198 INFO:teuthology.orchestra.run.smithi162.stdout: "mon": { 2024-04-27T04:18:25.198 INFO:teuthology.orchestra.run.smithi162.stdout: "ceph version 19.0.0-3253-g4e213a58 (4e213a58e787437ff508e6c890ccd5fd32dc96b5) squid (dev)": 2 2024-04-27T04:18:25.198 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-04-27T04:18:25.198 INFO:teuthology.orchestra.run.smithi162.stdout: "mgr": { 2024-04-27T04:18:25.198 INFO:teuthology.orchestra.run.smithi162.stdout: "ceph version 19.0.0-3253-g4e213a58 (4e213a58e787437ff508e6c890ccd5fd32dc96b5) squid (dev)": 2 2024-04-27T04:18:25.198 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-04-27T04:18:25.198 INFO:teuthology.orchestra.run.smithi162.stdout: "osd": { 2024-04-27T04:18:25.198 INFO:teuthology.orchestra.run.smithi162.stdout: "ceph version 17.2.7-907-g2be3efa9 (2be3efa97a7972f8e925bd899b474329979b74d6) quincy (stable)": 1, 2024-04-27T04:18:25.199 INFO:teuthology.orchestra.run.smithi162.stdout: "ceph version 19.0.0-3253-g4e213a58 (4e213a58e787437ff508e6c890ccd5fd32dc96b5) squid (dev)": 5 2024-04-27T04:18:25.199 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-04-27T04:18:25.199 INFO:teuthology.orchestra.run.smithi162.stdout: "mds": { 2024-04-27T04:18:25.199 INFO:teuthology.orchestra.run.smithi162.stdout: "ceph version 17.2.7-907-g2be3efa9 (2be3efa97a7972f8e925bd899b474329979b74d6) quincy (stable)": 4 2024-04-27T04:18:25.199 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-04-27T04:18:25.199 INFO:teuthology.orchestra.run.smithi162.stdout: "overall": { 2024-04-27T04:18:25.199 INFO:teuthology.orchestra.run.smithi162.stdout: "ceph version 17.2.7-907-g2be3efa9 (2be3efa97a7972f8e925bd899b474329979b74d6) quincy (stable)": 5, 2024-04-27T04:18:25.199 INFO:teuthology.orchestra.run.smithi162.stdout: "ceph version 19.0.0-3253-g4e213a58 (4e213a58e787437ff508e6c890ccd5fd32dc96b5) squid (dev)": 9 2024-04-27T04:18:25.199 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-04-27T04:18:25.199 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-04-27T04:18:25.200 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:18:25.202+0000 7f359471b700 1 -- 172.21.15.162:0/3691399106 >> [v2:172.21.15.162:6800/1396866108,v1:172.21.15.162:6801/1396866108] conn(0x7f357806ec00 msgr2=0x7f35780710b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:18:25.200 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:18:25.202+0000 7f359471b700 1 --2- 172.21.15.162:0/3691399106 >> [v2:172.21.15.162:6800/1396866108,v1:172.21.15.162:6801/1396866108] conn(0x7f357806ec00 0x7f35780710b0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f357c004890 tx=0x7f357c009e30 comp rx=0 tx=0).stop 2024-04-27T04:18:25.200 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:18:25.202+0000 7f359471b700 1 -- 172.21.15.162:0/3691399106 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f358c101d90 msgr2=0x7f358c114060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:18:25.201 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:18:25.202+0000 7f359471b700 1 --2- 172.21.15.162:0/3691399106 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f358c101d90 0x7f358c114060 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f3588002a80 tx=0x7f35880046b0 comp rx=0 tx=0).stop 2024-04-27T04:18:25.201 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:18:25.202+0000 7f359471b700 1 -- 172.21.15.162:0/3691399106 shutdown_connections 2024-04-27T04:18:25.201 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:18:25.202+0000 7f359471b700 1 --2- 172.21.15.162:0/3691399106 >> [v2:172.21.15.162:6800/1396866108,v1:172.21.15.162:6801/1396866108] conn(0x7f357806ec00 0x7f35780710b0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:18:25.201 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:18:25.202+0000 7f359471b700 1 --2- 172.21.15.162:0/3691399106 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f358c0fbd40 0x7f358c113b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:18:25.201 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:18:25.202+0000 7f359471b700 1 --2- 172.21.15.162:0/3691399106 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f358c101d90 0x7f358c114060 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:18:25.201 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:18:25.202+0000 7f359471b700 1 -- 172.21.15.162:0/3691399106 >> 172.21.15.162:0/3691399106 conn(0x7f358c071d00 msgr2=0x7f358c0fa030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:18:25.201 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:18:25.202+0000 7f359471b700 1 -- 172.21.15.162:0/3691399106 shutdown_connections 2024-04-27T04:18:25.201 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:18:25.202+0000 7f359471b700 1 -- 172.21.15.162:0/3691399106 wait complete. 2024-04-27T04:18:25.300 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:18:25.302+0000 7f319a1bd700 1 -- 172.21.15.162:0/803085697 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3194101d90 msgr2=0x7f3194102160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:18:25.300 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:18:25.302+0000 7f319a1bd700 1 --2- 172.21.15.162:0/803085697 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3194101d90 0x7f3194102160 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f3184004940 tx=0x7f3184009e30 comp rx=0 tx=0).stop 2024-04-27T04:18:25.301 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:18:25.302+0000 7f319a1bd700 1 -- 172.21.15.162:0/803085697 shutdown_connections 2024-04-27T04:18:25.301 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:18:25.302+0000 7f319a1bd700 1 --2- 172.21.15.162:0/803085697 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f31940fbd40 0x7f31940fc1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:18:25.301 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:18:25.302+0000 7f319a1bd700 1 --2- 172.21.15.162:0/803085697 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3194101d90 0x7f3194102160 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:18:25.301 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:18:25.302+0000 7f319a1bd700 1 -- 172.21.15.162:0/803085697 >> 172.21.15.162:0/803085697 conn(0x7f3194071d00 msgr2=0x7f3194072100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:18:25.301 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:18:25.302+0000 7f319a1bd700 1 -- 172.21.15.162:0/803085697 shutdown_connections 2024-04-27T04:18:25.301 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:18:25.302+0000 7f319a1bd700 1 -- 172.21.15.162:0/803085697 wait complete. 2024-04-27T04:18:25.301 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:18:25.303+0000 7f319a1bd700 1 Processor -- start 2024-04-27T04:18:25.303 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:18:25.303+0000 7f319a1bd700 1 -- start start 2024-04-27T04:18:25.303 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:18:25.303+0000 7f319a1bd700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f31940fbd40 0x7f3194112a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:18:25.303 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:18:25.303+0000 7f319a1bd700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3194101d90 0x7f3194112f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:18:25.303 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:18:25.303+0000 7f319a1bd700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3194113590 con 0x7f31940fbd40 2024-04-27T04:18:25.303 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:18:25.303+0000 7f319a1bd700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f319410c980 con 0x7f3194101d90 2024-04-27T04:18:25.303 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:18:25.303+0000 7f31937fe700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f31940fbd40 0x7f3194112a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:18:25.304 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:18:25.303+0000 7f31937fe700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f31940fbd40 0x7f3194112a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:53256/0 (socket says 172.21.15.162:53256) 2024-04-27T04:18:25.304 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:18:25.303+0000 7f31937fe700 1 -- 172.21.15.162:0/2788026267 learned_addr learned my addr 172.21.15.162:0/2788026267 (peer_addr_for_me v2:172.21.15.162:0/0) 2024-04-27T04:18:25.304 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:18:25.303+0000 7f318affd700 1 --2- 172.21.15.162:0/2788026267 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3194101d90 0x7f3194112f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:18:25.304 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:18:25.303+0000 7f31937fe700 1 -- 172.21.15.162:0/2788026267 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3194101d90 msgr2=0x7f3194112f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:18:25.304 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:18:25.303+0000 7f31937fe700 1 --2- 172.21.15.162:0/2788026267 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3194101d90 0x7f3194112f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:18:25.304 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:18:25.303+0000 7f31937fe700 1 -- 172.21.15.162:0/2788026267 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3184005040 con 0x7f31940fbd40 2024-04-27T04:18:25.304 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:18:25.304+0000 7f31937fe700 1 --2- 172.21.15.162:0/2788026267 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f31940fbd40 0x7f3194112a00 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f3184009b30 tx=0x7f3184009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:18:25.304 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:18:25.304+0000 7f3190ff9700 1 -- 172.21.15.162:0/2788026267 <== mon.0 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3184021070 con 0x7f31940fbd40 2024-04-27T04:18:25.305 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:18:25.304+0000 7f3190ff9700 1 -- 172.21.15.162:0/2788026267 <== mon.0 v2:172.21.15.162:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f3184013c00 con 0x7f31940fbd40 2024-04-27T04:18:25.305 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:18:25.304+0000 7f319a1bd700 1 -- 172.21.15.162:0/2788026267 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f319410cc00 con 0x7f31940fbd40 2024-04-27T04:18:25.305 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:18:25.304+0000 7f3190ff9700 1 -- 172.21.15.162:0/2788026267 <== mon.0 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f318400e070 con 0x7f31940fbd40 2024-04-27T04:18:25.305 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:18:25.304+0000 7f319a1bd700 1 -- 172.21.15.162:0/2788026267 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f319410d0e0 con 0x7f31940fbd40 2024-04-27T04:18:25.305 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:18:25.305+0000 7f319a1bd700 1 -- 172.21.15.162:0/2788026267 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f31940fd430 con 0x7f31940fbd40 2024-04-27T04:18:25.305 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:18:25.305+0000 7f3190ff9700 1 -- 172.21.15.162:0/2788026267 <== mon.0 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 39) v1 ==== 92747+0+0 (secure 0 0 0) 0x7f318401cb60 con 0x7f31940fbd40 2024-04-27T04:18:25.306 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:18:25.306+0000 7f3190ff9700 1 --2- 172.21.15.162:0/2788026267 >> [v2:172.21.15.162:6800/1396866108,v1:172.21.15.162:6801/1396866108] conn(0x7f318006ec00 0x7f31800710b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:18:25.306 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:18:25.306+0000 7f318affd700 1 --2- 172.21.15.162:0/2788026267 >> [v2:172.21.15.162:6800/1396866108,v1:172.21.15.162:6801/1396866108] conn(0x7f318006ec00 0x7f31800710b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:18:25.306 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:18:25.306+0000 7f3190ff9700 1 -- 172.21.15.162:0/2788026267 <== mon.0 v2:172.21.15.162:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 6410+0+0 (secure 0 0 0) 0x7f318408fcf0 con 0x7f31940fbd40 2024-04-27T04:18:25.306 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:18:25.306+0000 7f318affd700 1 --2- 172.21.15.162:0/2788026267 >> [v2:172.21.15.162:6800/1396866108,v1:172.21.15.162:6801/1396866108] conn(0x7f318006ec00 0x7f31800710b0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f319410db40 tx=0x7f317c00b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:18:25.308 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:18:25.311+0000 7f3190ff9700 1 -- 172.21.15.162:0/2788026267 <== mon.0 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7f318405c6a0 con 0x7f31940fbd40 2024-04-27T04:18:25.552 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:18:25.553+0000 7f319a1bd700 1 -- 172.21.15.162:0/2788026267 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f31940fd430 con 0x7f31940fbd40 2024-04-27T04:18:25.552 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:18:25.554+0000 7f3190ff9700 1 -- 172.21.15.162:0/2788026267 <== mon.0 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 15 v15) v1 ==== 76+0+1911 (secure 0 0 0) 0x7f3184004490 con 0x7f31940fbd40 2024-04-27T04:18:25.552 INFO:teuthology.orchestra.run.smithi162.stdout:e15 2024-04-27T04:18:25.552 INFO:teuthology.orchestra.run.smithi162.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-04-27T04:18:25.552 INFO:teuthology.orchestra.run.smithi162.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-27T04:18:25.552 INFO:teuthology.orchestra.run.smithi162.stdout:legacy client fscid: 1 2024-04-27T04:18:25.552 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-04-27T04:18:25.553 INFO:teuthology.orchestra.run.smithi162.stdout:Filesystem 'cephfs' (1) 2024-04-27T04:18:25.553 INFO:teuthology.orchestra.run.smithi162.stdout:fs_name cephfs 2024-04-27T04:18:25.553 INFO:teuthology.orchestra.run.smithi162.stdout:epoch 15 2024-04-27T04:18:25.553 INFO:teuthology.orchestra.run.smithi162.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-04-27T04:18:25.553 INFO:teuthology.orchestra.run.smithi162.stdout:created 2024-04-27T04:06:53.095749+0000 2024-04-27T04:18:25.553 INFO:teuthology.orchestra.run.smithi162.stdout:modified 2024-04-27T04:07:11.285033+0000 2024-04-27T04:18:25.553 INFO:teuthology.orchestra.run.smithi162.stdout:tableserver 0 2024-04-27T04:18:25.553 INFO:teuthology.orchestra.run.smithi162.stdout:root 0 2024-04-27T04:18:25.553 INFO:teuthology.orchestra.run.smithi162.stdout:session_timeout 60 2024-04-27T04:18:25.553 INFO:teuthology.orchestra.run.smithi162.stdout:session_autoclose 300 2024-04-27T04:18:25.553 INFO:teuthology.orchestra.run.smithi162.stdout:max_file_size 1099511627776 2024-04-27T04:18:25.553 INFO:teuthology.orchestra.run.smithi162.stdout:max_xattr_size 65536 2024-04-27T04:18:25.554 INFO:teuthology.orchestra.run.smithi162.stdout:required_client_features {} 2024-04-27T04:18:25.554 INFO:teuthology.orchestra.run.smithi162.stdout:last_failure 0 2024-04-27T04:18:25.554 INFO:teuthology.orchestra.run.smithi162.stdout:last_failure_osd_epoch 46 2024-04-27T04:18:25.554 INFO:teuthology.orchestra.run.smithi162.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-27T04:18:25.554 INFO:teuthology.orchestra.run.smithi162.stdout:max_mds 1 2024-04-27T04:18:25.554 INFO:teuthology.orchestra.run.smithi162.stdout:in 0 2024-04-27T04:18:25.554 INFO:teuthology.orchestra.run.smithi162.stdout:up {0=24273} 2024-04-27T04:18:25.554 INFO:teuthology.orchestra.run.smithi162.stdout:failed 2024-04-27T04:18:25.554 INFO:teuthology.orchestra.run.smithi162.stdout:damaged 2024-04-27T04:18:25.554 INFO:teuthology.orchestra.run.smithi162.stdout:stopped 2024-04-27T04:18:25.555 INFO:teuthology.orchestra.run.smithi162.stdout:data_pools [3] 2024-04-27T04:18:25.555 INFO:teuthology.orchestra.run.smithi162.stdout:metadata_pool 2 2024-04-27T04:18:25.555 INFO:teuthology.orchestra.run.smithi162.stdout:inline_data disabled 2024-04-27T04:18:25.555 INFO:teuthology.orchestra.run.smithi162.stdout:balancer 2024-04-27T04:18:25.555 INFO:teuthology.orchestra.run.smithi162.stdout:bal_rank_mask -1 2024-04-27T04:18:25.555 INFO:teuthology.orchestra.run.smithi162.stdout:standby_count_wanted 1 2024-04-27T04:18:25.555 INFO:teuthology.orchestra.run.smithi162.stdout:qdb_cluster leader: 0 members: 2024-04-27T04:18:25.555 INFO:teuthology.orchestra.run.smithi162.stdout:[mds.cephfs.smithi181.dzomgh{0:24273} state up:active seq 6 join_fscid=1 addr [v2:172.21.15.181:6824/658214993,v1:172.21.15.181:6825/658214993] compat {c=[1],r=[1],i=[7ff]}] 2024-04-27T04:18:25.555 INFO:teuthology.orchestra.run.smithi162.stdout:[mds.cephfs.smithi162.xcnapk{0:14476} state up:standby-replay seq 2 join_fscid=1 addr [v2:172.21.15.162:6828/1296160676,v1:172.21.15.162:6829/1296160676] compat {c=[1],r=[1],i=[7ff]}] 2024-04-27T04:18:25.555 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-04-27T04:18:25.555 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-04-27T04:18:25.555 INFO:teuthology.orchestra.run.smithi162.stdout:Standby daemons: 2024-04-27T04:18:25.556 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-04-27T04:18:25.556 INFO:teuthology.orchestra.run.smithi162.stdout:[mds.cephfs.smithi162.gzlyjj{-1:14488} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.162:6826/2629103894,v1:172.21.15.162:6827/2629103894] compat {c=[1],r=[1],i=[7ff]}] 2024-04-27T04:18:25.556 INFO:teuthology.orchestra.run.smithi162.stdout:[mds.cephfs.smithi181.bkkcdm{-1:24287} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.181:6826/3756475453,v1:172.21.15.181:6827/3756475453] compat {c=[1],r=[1],i=[7ff]}] 2024-04-27T04:18:25.556 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:18:25.556+0000 7f319a1bd700 1 -- 172.21.15.162:0/2788026267 >> [v2:172.21.15.162:6800/1396866108,v1:172.21.15.162:6801/1396866108] conn(0x7f318006ec00 msgr2=0x7f31800710b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:18:25.556 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:18:25.556+0000 7f319a1bd700 1 --2- 172.21.15.162:0/2788026267 >> [v2:172.21.15.162:6800/1396866108,v1:172.21.15.162:6801/1396866108] conn(0x7f318006ec00 0x7f31800710b0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f319410db40 tx=0x7f317c00b040 comp rx=0 tx=0).stop 2024-04-27T04:18:25.557 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:18:25.556+0000 7f319a1bd700 1 -- 172.21.15.162:0/2788026267 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f31940fbd40 msgr2=0x7f3194112a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:18:25.557 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:18:25.556+0000 7f319a1bd700 1 --2- 172.21.15.162:0/2788026267 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f31940fbd40 0x7f3194112a00 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f3184009b30 tx=0x7f3184009e30 comp rx=0 tx=0).stop 2024-04-27T04:18:25.557 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:18:25.556+0000 7f319a1bd700 1 -- 172.21.15.162:0/2788026267 shutdown_connections 2024-04-27T04:18:25.557 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:18:25.556+0000 7f319a1bd700 1 --2- 172.21.15.162:0/2788026267 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f31940fbd40 0x7f3194112a00 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:18:25.557 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:18:25.556+0000 7f319a1bd700 1 --2- 172.21.15.162:0/2788026267 >> [v2:172.21.15.162:6800/1396866108,v1:172.21.15.162:6801/1396866108] conn(0x7f318006ec00 0x7f31800710b0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:18:25.557 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:18:25.556+0000 7f319a1bd700 1 --2- 172.21.15.162:0/2788026267 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3194101d90 0x7f3194112f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:18:25.557 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:18:25.556+0000 7f319a1bd700 1 -- 172.21.15.162:0/2788026267 >> 172.21.15.162:0/2788026267 conn(0x7f3194071d00 msgr2=0x7f31940fa030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:18:25.558 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:18:25.556+0000 7f319a1bd700 1 -- 172.21.15.162:0/2788026267 shutdown_connections 2024-04-27T04:18:25.558 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:18:25.556+0000 7f319a1bd700 1 -- 172.21.15.162:0/2788026267 wait complete. 2024-04-27T04:18:25.558 INFO:teuthology.orchestra.run.smithi162.stderr:dumped fsmap epoch 15 2024-04-27T04:18:25.659 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:18:25.659+0000 7fbf3e1d6700 1 -- 172.21.15.162:0/3092597873 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fbf38103780 msgr2=0x7fbf380fcdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:18:25.659 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:18:25.659+0000 7fbf3e1d6700 1 --2- 172.21.15.162:0/3092597873 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fbf38103780 0x7fbf380fcdf0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fbf28004940 tx=0x7fbf28009e30 comp rx=0 tx=0).stop 2024-04-27T04:18:25.659 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:18:25.659+0000 7fbf3e1d6700 1 -- 172.21.15.162:0/3092597873 shutdown_connections 2024-04-27T04:18:25.659 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:18:25.659+0000 7fbf3e1d6700 1 --2- 172.21.15.162:0/3092597873 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fbf38103780 0x7fbf380fcdf0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:18:25.659 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:18:25.659+0000 7fbf3e1d6700 1 --2- 172.21.15.162:0/3092597873 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbf38102de0 0x7fbf381031b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:18:25.659 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:18:25.659+0000 7fbf3e1d6700 1 -- 172.21.15.162:0/3092597873 >> 172.21.15.162:0/3092597873 conn(0x7fbf380f8950 msgr2=0x7fbf380fad60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:18:25.660 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:18:25.659+0000 7fbf3e1d6700 1 -- 172.21.15.162:0/3092597873 shutdown_connections 2024-04-27T04:18:25.660 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:18:25.659+0000 7fbf3e1d6700 1 -- 172.21.15.162:0/3092597873 wait complete. 2024-04-27T04:18:25.660 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:18:25.659+0000 7fbf3e1d6700 1 Processor -- start 2024-04-27T04:18:25.660 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:18:25.660+0000 7fbf3e1d6700 1 -- start start 2024-04-27T04:18:25.660 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:18:25.660+0000 7fbf3e1d6700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fbf38102de0 0x7fbf38113ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:18:25.660 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:18:25.660+0000 7fbf3e1d6700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbf38103780 0x7fbf38114020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:18:25.660 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:18:25.660+0000 7fbf3e1d6700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf38114670 con 0x7fbf38102de0 2024-04-27T04:18:25.660 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:18:25.660+0000 7fbf3e1d6700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf3810da60 con 0x7fbf38103780 2024-04-27T04:18:25.660 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:18:25.660+0000 7fbf377fe700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fbf38102de0 0x7fbf38113ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:18:25.660 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:18:25.660+0000 7fbf377fe700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fbf38102de0 0x7fbf38113ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:53270/0 (socket says 172.21.15.162:53270) 2024-04-27T04:18:25.660 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:18:25.660+0000 7fbf377fe700 1 -- 172.21.15.162:0/3933661141 learned_addr learned my addr 172.21.15.162:0/3933661141 (peer_addr_for_me v2:172.21.15.162:0/0) 2024-04-27T04:18:25.660 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:18:25.660+0000 7fbf36ffd700 1 --2- 172.21.15.162:0/3933661141 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbf38103780 0x7fbf38114020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:18:25.660 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:18:25.660+0000 7fbf377fe700 1 -- 172.21.15.162:0/3933661141 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbf38103780 msgr2=0x7fbf38114020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:18:25.661 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:18:25.660+0000 7fbf377fe700 1 --2- 172.21.15.162:0/3933661141 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbf38103780 0x7fbf38114020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:18:25.661 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:18:25.660+0000 7fbf377fe700 1 -- 172.21.15.162:0/3933661141 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf28005040 con 0x7fbf38102de0 2024-04-27T04:18:25.661 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:18:25.661+0000 7fbf377fe700 1 --2- 172.21.15.162:0/3933661141 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fbf38102de0 0x7fbf38113ae0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fbf2c002a80 tx=0x7fbf2c0046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:18:25.661 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:18:25.661+0000 7fbf1ffff700 1 -- 172.21.15.162:0/3933661141 <== mon.0 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbf2c007cc0 con 0x7fbf38102de0 2024-04-27T04:18:25.661 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:18:25.661+0000 7fbf1ffff700 1 -- 172.21.15.162:0/3933661141 <== mon.0 v2:172.21.15.162:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fbf2c00f040 con 0x7fbf38102de0 2024-04-27T04:18:25.661 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:18:25.661+0000 7fbf3e1d6700 1 -- 172.21.15.162:0/3933661141 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbf3810dd40 con 0x7fbf38102de0 2024-04-27T04:18:25.661 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:18:25.661+0000 7fbf1ffff700 1 -- 172.21.15.162:0/3933661141 <== mon.0 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbf2c00dac0 con 0x7fbf38102de0 2024-04-27T04:18:25.661 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:18:25.661+0000 7fbf3e1d6700 1 -- 172.21.15.162:0/3933661141 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbf3810e2d0 con 0x7fbf38102de0 2024-04-27T04:18:25.663 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:18:25.662+0000 7fbf3e1d6700 1 -- 172.21.15.162:0/3933661141 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbf380fe530 con 0x7fbf38102de0 2024-04-27T04:18:25.663 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:18:25.662+0000 7fbf1ffff700 1 -- 172.21.15.162:0/3933661141 <== mon.0 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 39) v1 ==== 92747+0+0 (secure 0 0 0) 0x7fbf2c00dc20 con 0x7fbf38102de0 2024-04-27T04:18:25.663 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:18:25.663+0000 7fbf1ffff700 1 --2- 172.21.15.162:0/3933661141 >> [v2:172.21.15.162:6800/1396866108,v1:172.21.15.162:6801/1396866108] conn(0x7fbf2006ec00 0x7fbf200710b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:18:25.663 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:18:25.663+0000 7fbf1ffff700 1 -- 172.21.15.162:0/3933661141 <== mon.0 v2:172.21.15.162:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 6410+0+0 (secure 0 0 0) 0x7fbf2c08e5c0 con 0x7fbf38102de0 2024-04-27T04:18:25.663 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:18:25.663+0000 7fbf36ffd700 1 --2- 172.21.15.162:0/3933661141 >> [v2:172.21.15.162:6800/1396866108,v1:172.21.15.162:6801/1396866108] conn(0x7fbf2006ec00 0x7fbf200710b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:18:25.663 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:18:25.663+0000 7fbf36ffd700 1 --2- 172.21.15.162:0/3933661141 >> [v2:172.21.15.162:6800/1396866108,v1:172.21.15.162:6801/1396866108] conn(0x7fbf2006ec00 0x7fbf200710b0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fbf3810ed40 tx=0x7fbf28009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:18:25.665 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:18:25.668+0000 7fbf1ffff700 1 -- 172.21.15.162:0/3933661141 <== mon.0 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7fbf2c05aef0 con 0x7fbf38102de0 2024-04-27T04:18:25.886 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:18:25 smithi162 ceph-mon[110641]: from='client.34344 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-27T04:18:25.886 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:18:25 smithi162 ceph-mon[110641]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["5"], "max": 16}]: dispatch 2024-04-27T04:18:25.887 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:18:25 smithi162 ceph-mon[110641]: Upgrade: osd.5 is safe to restart 2024-04-27T04:18:25.887 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:18:25 smithi162 ceph-mon[110641]: from='client.34348 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-27T04:18:25.887 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:18:25 smithi162 ceph-mon[110641]: from='client.34352 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-27T04:18:25.887 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:18:25 smithi162 ceph-mon[110641]: from='client.? 172.21.15.162:0/3691399106' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-27T04:18:25.887 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:18:25 smithi162 ceph-mon[110641]: from='client.? 172.21.15.162:0/2788026267' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-27T04:18:25.891 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:18:25 smithi181 ceph-mon[81134]: from='client.34344 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-27T04:18:25.891 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:18:25 smithi181 ceph-mon[81134]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["5"], "max": 16}]: dispatch 2024-04-27T04:18:25.891 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:18:25 smithi181 ceph-mon[81134]: Upgrade: osd.5 is safe to restart 2024-04-27T04:18:25.891 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:18:25 smithi181 ceph-mon[81134]: from='client.34348 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-27T04:18:25.891 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:18:25 smithi181 ceph-mon[81134]: from='client.34352 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-27T04:18:25.892 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:18:25 smithi181 ceph-mon[81134]: from='client.? 172.21.15.162:0/3691399106' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-27T04:18:25.892 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:18:25 smithi181 ceph-mon[81134]: from='client.? 172.21.15.162:0/2788026267' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-27T04:18:25.892 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:18:25.895+0000 7fbf3e1d6700 1 -- 172.21.15.162:0/3933661141 --> [v2:172.21.15.162:6800/1396866108,v1:172.21.15.162:6801/1396866108] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fbf38109ec0 con 0x7fbf2006ec00 2024-04-27T04:18:25.894 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:18:25.896+0000 7fbf1ffff700 1 -- 172.21.15.162:0/3933661141 <== mgr.34104 v2:172.21.15.162:6800/1396866108 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+401 (secure 0 0 0) 0x7fbf38109ec0 con 0x7fbf2006ec00 2024-04-27T04:18:25.894 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-04-27T04:18:25.894 INFO:teuthology.orchestra.run.smithi162.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:90a3a9fa9ceb8977f6dad9ccd36fe46018d07e2118199820523a6b35301d9c16", 2024-04-27T04:18:25.894 INFO:teuthology.orchestra.run.smithi162.stdout: "in_progress": true, 2024-04-27T04:18:25.894 INFO:teuthology.orchestra.run.smithi162.stdout: "which": "Upgrading all daemon types on all hosts", 2024-04-27T04:18:25.894 INFO:teuthology.orchestra.run.smithi162.stdout: "services_complete": [ 2024-04-27T04:18:25.894 INFO:teuthology.orchestra.run.smithi162.stdout: "crash", 2024-04-27T04:18:25.894 INFO:teuthology.orchestra.run.smithi162.stdout: "mon", 2024-04-27T04:18:25.894 INFO:teuthology.orchestra.run.smithi162.stdout: "mgr" 2024-04-27T04:18:25.895 INFO:teuthology.orchestra.run.smithi162.stdout: ], 2024-04-27T04:18:25.895 INFO:teuthology.orchestra.run.smithi162.stdout: "progress": "11/21 daemons upgraded", 2024-04-27T04:18:25.895 INFO:teuthology.orchestra.run.smithi162.stdout: "message": "Currently upgrading osd daemons", 2024-04-27T04:18:25.895 INFO:teuthology.orchestra.run.smithi162.stdout: "is_paused": false 2024-04-27T04:18:25.895 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-04-27T04:18:25.897 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:18:25.899+0000 7fbf3e1d6700 1 -- 172.21.15.162:0/3933661141 >> [v2:172.21.15.162:6800/1396866108,v1:172.21.15.162:6801/1396866108] conn(0x7fbf2006ec00 msgr2=0x7fbf200710b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:18:25.897 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:18:25.899+0000 7fbf3e1d6700 1 --2- 172.21.15.162:0/3933661141 >> [v2:172.21.15.162:6800/1396866108,v1:172.21.15.162:6801/1396866108] conn(0x7fbf2006ec00 0x7fbf200710b0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fbf3810ed40 tx=0x7fbf28009e30 comp rx=0 tx=0).stop 2024-04-27T04:18:25.897 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:18:25.899+0000 7fbf3e1d6700 1 -- 172.21.15.162:0/3933661141 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fbf38102de0 msgr2=0x7fbf38113ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:18:25.897 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:18:25.899+0000 7fbf3e1d6700 1 --2- 172.21.15.162:0/3933661141 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fbf38102de0 0x7fbf38113ae0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fbf2c002a80 tx=0x7fbf2c0046b0 comp rx=0 tx=0).stop 2024-04-27T04:18:25.897 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:18:25.899+0000 7fbf3e1d6700 1 -- 172.21.15.162:0/3933661141 shutdown_connections 2024-04-27T04:18:25.898 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:18:25.899+0000 7fbf3e1d6700 1 --2- 172.21.15.162:0/3933661141 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fbf38102de0 0x7fbf38113ae0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:18:25.898 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:18:25.899+0000 7fbf3e1d6700 1 --2- 172.21.15.162:0/3933661141 >> [v2:172.21.15.162:6800/1396866108,v1:172.21.15.162:6801/1396866108] conn(0x7fbf2006ec00 0x7fbf200710b0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:18:25.898 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:18:25.899+0000 7fbf3e1d6700 1 --2- 172.21.15.162:0/3933661141 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbf38103780 0x7fbf38114020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:18:25.898 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:18:25.899+0000 7fbf3e1d6700 1 -- 172.21.15.162:0/3933661141 >> 172.21.15.162:0/3933661141 conn(0x7fbf380f8950 msgr2=0x7fbf380f9920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:18:25.898 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:18:25.899+0000 7fbf3e1d6700 1 -- 172.21.15.162:0/3933661141 shutdown_connections 2024-04-27T04:18:25.898 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:18:25.899+0000 7fbf3e1d6700 1 -- 172.21.15.162:0/3933661141 wait complete. 2024-04-27T04:18:26.002 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:18:26.004+0000 7f15fb0ad700 1 -- 172.21.15.162:0/2381651216 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f15f4103790 msgr2=0x7f15f40fce00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:18:26.003 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:18:26.004+0000 7f15fb0ad700 1 --2- 172.21.15.162:0/2381651216 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f15f4103790 0x7f15f40fce00 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f15e0004940 tx=0x7f15e0009e30 comp rx=0 tx=0).stop 2024-04-27T04:18:26.003 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:18:26.004+0000 7f15fb0ad700 1 -- 172.21.15.162:0/2381651216 shutdown_connections 2024-04-27T04:18:26.003 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:18:26.004+0000 7f15fb0ad700 1 --2- 172.21.15.162:0/2381651216 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f15f4103790 0x7f15f40fce00 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:18:26.003 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:18:26.004+0000 7f15fb0ad700 1 --2- 172.21.15.162:0/2381651216 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f15f4102df0 0x7f15f41031c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:18:26.003 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:18:26.004+0000 7f15fb0ad700 1 -- 172.21.15.162:0/2381651216 >> 172.21.15.162:0/2381651216 conn(0x7f15f40f8980 msgr2=0x7f15f40fad90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:18:26.003 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:18:26.004+0000 7f15fb0ad700 1 -- 172.21.15.162:0/2381651216 shutdown_connections 2024-04-27T04:18:26.003 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:18:26.004+0000 7f15fb0ad700 1 -- 172.21.15.162:0/2381651216 wait complete. 2024-04-27T04:18:26.003 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:18:26.005+0000 7f15fb0ad700 1 Processor -- start 2024-04-27T04:18:26.003 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:18:26.005+0000 7f15fb0ad700 1 -- start start 2024-04-27T04:18:26.005 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:18:26.005+0000 7f15fb0ad700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f15f4102df0 0x7f15f4113b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:18:26.005 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:18:26.005+0000 7f15fb0ad700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f15f4103790 0x7f15f4114060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:18:26.005 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:18:26.005+0000 7f15fb0ad700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15f41146b0 con 0x7f15f4103790 2024-04-27T04:18:26.006 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:18:26.005+0000 7f15fb0ad700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15f410daa0 con 0x7f15f4102df0 2024-04-27T04:18:26.006 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:18:26.005+0000 7f15f3fff700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f15f4103790 0x7f15f4114060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:18:26.006 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:18:26.005+0000 7f15f3fff700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f15f4103790 0x7f15f4114060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:53278/0 (socket says 172.21.15.162:53278) 2024-04-27T04:18:26.006 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:18:26.005+0000 7f15f3fff700 1 -- 172.21.15.162:0/3958667415 learned_addr learned my addr 172.21.15.162:0/3958667415 (peer_addr_for_me v2:172.21.15.162:0/0) 2024-04-27T04:18:26.006 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:18:26.005+0000 7f15f8e49700 1 --2- 172.21.15.162:0/3958667415 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f15f4102df0 0x7f15f4113b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:18:26.006 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:18:26.005+0000 7f15f8e49700 1 -- 172.21.15.162:0/3958667415 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f15f4103790 msgr2=0x7f15f4114060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:18:26.006 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:18:26.005+0000 7f15f8e49700 1 --2- 172.21.15.162:0/3958667415 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f15f4103790 0x7f15f4114060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:18:26.006 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:18:26.005+0000 7f15f8e49700 1 -- 172.21.15.162:0/3958667415 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15e0005040 con 0x7f15f4102df0 2024-04-27T04:18:26.006 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:18:26.005+0000 7f15f3fff700 1 --2- 172.21.15.162:0/3958667415 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f15f4103790 0x7f15f4114060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-27T04:18:26.006 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:18:26.006+0000 7f15f8e49700 1 --2- 172.21.15.162:0/3958667415 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f15f4102df0 0x7f15f4113b20 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f15e8009fd0 tx=0x7f15e800b040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:18:26.006 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:18:26.006+0000 7f15f17fa700 1 -- 172.21.15.162:0/3958667415 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f15e8004e90 con 0x7f15f4102df0 2024-04-27T04:18:26.007 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:18:26.006+0000 7f15fb0ad700 1 -- 172.21.15.162:0/3958667415 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f15f410dd20 con 0x7f15f4102df0 2024-04-27T04:18:26.007 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:18:26.006+0000 7f15f17fa700 1 -- 172.21.15.162:0/3958667415 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f15e800f040 con 0x7f15f4102df0 2024-04-27T04:18:26.007 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:18:26.006+0000 7f15f17fa700 1 -- 172.21.15.162:0/3958667415 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f15e800ca50 con 0x7f15f4102df0 2024-04-27T04:18:26.007 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:18:26.006+0000 7f15fb0ad700 1 -- 172.21.15.162:0/3958667415 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f15f410e260 con 0x7f15f4102df0 2024-04-27T04:18:26.007 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:18:26.007+0000 7f15fb0ad700 1 -- 172.21.15.162:0/3958667415 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f15f40fe540 con 0x7f15f4102df0 2024-04-27T04:18:26.007 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:18:26.007+0000 7f15f17fa700 1 -- 172.21.15.162:0/3958667415 <== mon.1 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 39) v1 ==== 92747+0+0 (secure 0 0 0) 0x7f15e800cc00 con 0x7f15f4102df0 2024-04-27T04:18:26.007 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:18:26.008+0000 7f15f17fa700 1 --2- 172.21.15.162:0/3958667415 >> [v2:172.21.15.162:6800/1396866108,v1:172.21.15.162:6801/1396866108] conn(0x7f15e406ec00 0x7f15e40710b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:18:26.007 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:18:26.008+0000 7f15f3fff700 1 --2- 172.21.15.162:0/3958667415 >> [v2:172.21.15.162:6800/1396866108,v1:172.21.15.162:6801/1396866108] conn(0x7f15e406ec00 0x7f15e40710b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:18:26.007 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:18:26.008+0000 7f15f17fa700 1 -- 172.21.15.162:0/3958667415 <== mon.1 v2:172.21.15.181:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 6410+0+0 (secure 0 0 0) 0x7f15e808e300 con 0x7f15f4102df0 2024-04-27T04:18:26.007 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:18:26.008+0000 7f15f3fff700 1 --2- 172.21.15.162:0/3958667415 >> [v2:172.21.15.162:6800/1396866108,v1:172.21.15.162:6801/1396866108] conn(0x7f15e406ec00 0x7f15e40710b0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f15e0007f50 tx=0x7f15e0009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:18:26.010 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:18:26.013+0000 7f15f17fa700 1 -- 172.21.15.162:0/3958667415 <== mon.1 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7f15e805ac40 con 0x7f15f4102df0 2024-04-27T04:18:26.291 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:18:26.292+0000 7f15fb0ad700 1 -- 172.21.15.162:0/3958667415 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f15f4109ed0 con 0x7f15f4102df0 2024-04-27T04:18:26.291 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:18:26.293+0000 7f15f17fa700 1 -- 172.21.15.162:0/3958667415 <== mon.1 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+10 (secure 0 0 0) 0x7f15e805a7d0 con 0x7f15f4102df0 2024-04-27T04:18:26.291 INFO:teuthology.orchestra.run.smithi162.stdout:HEALTH_OK 2024-04-27T04:18:26.293 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:18:26.295+0000 7f15fb0ad700 1 -- 172.21.15.162:0/3958667415 >> [v2:172.21.15.162:6800/1396866108,v1:172.21.15.162:6801/1396866108] conn(0x7f15e406ec00 msgr2=0x7f15e40710b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:18:26.293 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:18:26.295+0000 7f15fb0ad700 1 --2- 172.21.15.162:0/3958667415 >> [v2:172.21.15.162:6800/1396866108,v1:172.21.15.162:6801/1396866108] conn(0x7f15e406ec00 0x7f15e40710b0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f15e0007f50 tx=0x7f15e0009e30 comp rx=0 tx=0).stop 2024-04-27T04:18:26.293 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:18:26.295+0000 7f15fb0ad700 1 -- 172.21.15.162:0/3958667415 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f15f4102df0 msgr2=0x7f15f4113b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:18:26.293 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:18:26.295+0000 7f15fb0ad700 1 --2- 172.21.15.162:0/3958667415 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f15f4102df0 0x7f15f4113b20 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f15e8009fd0 tx=0x7f15e800b040 comp rx=0 tx=0).stop 2024-04-27T04:18:26.293 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:18:26.295+0000 7f15fb0ad700 1 -- 172.21.15.162:0/3958667415 shutdown_connections 2024-04-27T04:18:26.293 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:18:26.295+0000 7f15fb0ad700 1 --2- 172.21.15.162:0/3958667415 >> [v2:172.21.15.162:6800/1396866108,v1:172.21.15.162:6801/1396866108] conn(0x7f15e406ec00 0x7f15e40710b0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:18:26.294 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:18:26.295+0000 7f15fb0ad700 1 --2- 172.21.15.162:0/3958667415 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f15f4102df0 0x7f15f4113b20 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:18:26.294 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:18:26.295+0000 7f15fb0ad700 1 --2- 172.21.15.162:0/3958667415 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f15f4103790 0x7f15f4114060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:18:26.294 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:18:26.295+0000 7f15fb0ad700 1 -- 172.21.15.162:0/3958667415 >> 172.21.15.162:0/3958667415 conn(0x7f15f40f8980 msgr2=0x7f15f40f98b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:18:26.294 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:18:26.295+0000 7f15fb0ad700 1 -- 172.21.15.162:0/3958667415 shutdown_connections 2024-04-27T04:18:26.294 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:18:26.295+0000 7f15fb0ad700 1 -- 172.21.15.162:0/3958667415 wait complete. 2024-04-27T04:18:26.853 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:18:26 smithi181 ceph-mon[81134]: pgmap v152: 65 pgs: 65 active+clean; 208 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-04-27T04:18:26.853 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:18:26 smithi181 ceph-mon[81134]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' 2024-04-27T04:18:26.853 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:18:26 smithi181 ceph-mon[81134]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-04-27T04:18:26.853 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:18:26 smithi181 ceph-mon[81134]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T04:18:26.853 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:18:26 smithi181 ceph-mon[81134]: from='client.? 172.21.15.162:0/3958667415' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-27T04:18:27.136 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:18:26 smithi162 ceph-mon[110641]: pgmap v152: 65 pgs: 65 active+clean; 208 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-04-27T04:18:27.137 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:18:26 smithi162 ceph-mon[110641]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' 2024-04-27T04:18:27.137 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:18:26 smithi162 ceph-mon[110641]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-04-27T04:18:27.137 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:18:26 smithi162 ceph-mon[110641]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T04:18:27.137 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:18:26 smithi162 ceph-mon[110641]: from='client.? 172.21.15.162:0/3958667415' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-27T04:18:28.035 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:18:27 smithi181 ceph-mon[81134]: from='client.34364 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-27T04:18:28.035 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:18:27 smithi181 ceph-mon[81134]: Upgrade: Updating osd.5 2024-04-27T04:18:28.035 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:18:27 smithi181 ceph-mon[81134]: Deploying daemon osd.5 on smithi181 2024-04-27T04:18:28.136 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:18:27 smithi162 ceph-mon[110641]: from='client.34364 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-27T04:18:28.136 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:18:27 smithi162 ceph-mon[110641]: Upgrade: Updating osd.5 2024-04-27T04:18:28.136 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:18:27 smithi162 ceph-mon[110641]: Deploying daemon osd.5 on smithi181 2024-04-27T04:18:28.847 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:18:28 smithi181 ceph-mon[81134]: pgmap v153: 65 pgs: 65 active+clean; 208 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:18:29.136 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:18:28 smithi162 ceph-mon[110641]: pgmap v153: 65 pgs: 65 active+clean; 208 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:18:29.494 INFO:journalctl@ceph.osd.5.smithi181.stdout:Apr 27 04:18:29 smithi181 systemd[1]: Stopping Ceph osd.5 for 7f9750d6-044a-11ef-bc93-c7b262605968... 2024-04-27T04:18:29.922 INFO:journalctl@ceph.osd.5.smithi181.stdout:Apr 27 04:18:29 smithi181 ceph-7f9750d6-044a-11ef-bc93-c7b262605968-osd-5[52697]: 2024-04-27T04:18:29.494+0000 7f8e3ea7c700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.5 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-27T04:18:29.922 INFO:journalctl@ceph.osd.5.smithi181.stdout:Apr 27 04:18:29 smithi181 ceph-7f9750d6-044a-11ef-bc93-c7b262605968-osd-5[52697]: 2024-04-27T04:18:29.494+0000 7f8e3ea7c700 -1 osd.5 74 *** Got signal Terminated *** 2024-04-27T04:18:29.922 INFO:journalctl@ceph.osd.5.smithi181.stdout:Apr 27 04:18:29 smithi181 ceph-7f9750d6-044a-11ef-bc93-c7b262605968-osd-5[52697]: 2024-04-27T04:18:29.494+0000 7f8e3ea7c700 -1 osd.5 74 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-27T04:18:30.636 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:18:30 smithi162 ceph-mon[110641]: osd.5 marked itself down and dead 2024-04-27T04:18:30.636 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:18:30 smithi162 ceph-mon[110641]: pgmap v154: 65 pgs: 65 active+clean; 208 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-04-27T04:18:30.671 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:18:30 smithi181 ceph-mon[81134]: osd.5 marked itself down and dead 2024-04-27T04:18:30.672 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:18:30 smithi181 ceph-mon[81134]: pgmap v154: 65 pgs: 65 active+clean; 208 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-04-27T04:18:31.134 INFO:journalctl@ceph.osd.5.smithi181.stdout:Apr 27 04:18:30 smithi181 podman[100392]: 2024-04-27 04:18:30.815531258 +0000 UTC m=+1.449384397 container died f47a1daac1eabee47ecc184b89f9b8c8462cdf2b1225446d99fdb05985c699cf (image=quay.ceph.io/ceph-ci/ceph@sha256:d31f278ece6dbd17d1fa5e598f509017f54b76fafe1c4416391fcc51b9718ee9, name=ceph-7f9750d6-044a-11ef-bc93-c7b262605968-osd-5, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.build-date=20240424, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_CLEAN=True, RELEASE=quincy-2be3efa, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2) 2024-04-27T04:18:31.421 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:18:31 smithi181 ceph-mon[81134]: Health check failed: 1 osds down (OSD_DOWN) 2024-04-27T04:18:31.422 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:18:31 smithi181 ceph-mon[81134]: Health check failed: all OSDs are running squid or later but require_osd_release < squid (OSD_UPGRADE_FINISHED) 2024-04-27T04:18:31.422 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:18:31 smithi181 ceph-mon[81134]: osdmap e75: 6 total, 5 up, 6 in 2024-04-27T04:18:31.422 INFO:journalctl@ceph.osd.5.smithi181.stdout:Apr 27 04:18:31 smithi181 podman[100392]: 2024-04-27 04:18:31.137246846 +0000 UTC m=+1.771099989 container cleanup f47a1daac1eabee47ecc184b89f9b8c8462cdf2b1225446d99fdb05985c699cf (image=quay.ceph.io/ceph-ci/ceph@sha256:d31f278ece6dbd17d1fa5e598f509017f54b76fafe1c4416391fcc51b9718ee9, name=ceph-7f9750d6-044a-11ef-bc93-c7b262605968-osd-5, ceph=True, CEPH_POINT_RELEASE=, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240424, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, RELEASE=quincy-2be3efa) 2024-04-27T04:18:31.422 INFO:journalctl@ceph.osd.5.smithi181.stdout:Apr 27 04:18:31 smithi181 bash[100392]: ceph-7f9750d6-044a-11ef-bc93-c7b262605968-osd-5 2024-04-27T04:18:31.422 INFO:journalctl@ceph.osd.5.smithi181.stdout:Apr 27 04:18:31 smithi181 podman[100413]: 2024-04-27 04:18:31.313879283 +0000 UTC m=+0.495761415 container remove f47a1daac1eabee47ecc184b89f9b8c8462cdf2b1225446d99fdb05985c699cf (image=quay.ceph.io/ceph-ci/ceph@sha256:d31f278ece6dbd17d1fa5e598f509017f54b76fafe1c4416391fcc51b9718ee9, name=ceph-7f9750d6-044a-11ef-bc93-c7b262605968-osd-5, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240424, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, ceph=True, CEPH_POINT_RELEASE=, RELEASE=quincy-2be3efa, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, maintainer=Guillaume Abrioux ) 2024-04-27T04:18:31.636 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:18:31 smithi162 ceph-mon[110641]: Health check failed: 1 osds down (OSD_DOWN) 2024-04-27T04:18:31.636 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:18:31 smithi162 ceph-mon[110641]: Health check failed: all OSDs are running squid or later but require_osd_release < squid (OSD_UPGRADE_FINISHED) 2024-04-27T04:18:31.636 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:18:31 smithi162 ceph-mon[110641]: osdmap e75: 6 total, 5 up, 6 in 2024-04-27T04:18:32.064 INFO:journalctl@ceph.osd.5.smithi181.stdout:Apr 27 04:18:31 smithi181 podman[100500]: 2024-04-27 04:18:31.655950037 +0000 UTC m=+0.017504345 image pull 67e414ff4bc41092cb31c84c98a9563a32813d02583028679c46b40362afeb3e quay.ceph.io/ceph-ci/ceph@sha256:90a3a9fa9ceb8977f6dad9ccd36fe46018d07e2118199820523a6b35301d9c16 2024-04-27T04:18:32.065 INFO:journalctl@ceph.osd.5.smithi181.stdout:Apr 27 04:18:31 smithi181 podman[100500]: 2024-04-27 04:18:31.773200247 +0000 UTC m=+0.134754565 container create e7071bcbb51185f97217de414af37615627413264f367e30c61ec75864003620 (image=quay.ceph.io/ceph-ci/ceph@sha256:90a3a9fa9ceb8977f6dad9ccd36fe46018d07e2118199820523a6b35301d9c16, name=ceph-7f9750d6-044a-11ef-bc93-c7b262605968-osd-5-deactivate, org.label-schema.vendor=CentOS, GIT_CLEAN=True, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , RELEASE=main-4e213a5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240424, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-27T04:18:32.065 INFO:journalctl@ceph.osd.5.smithi181.stdout:Apr 27 04:18:31 smithi181 podman[100500]: 2024-04-27 04:18:31.992289158 +0000 UTC m=+0.353843435 container init e7071bcbb51185f97217de414af37615627413264f367e30c61ec75864003620 (image=quay.ceph.io/ceph-ci/ceph@sha256:90a3a9fa9ceb8977f6dad9ccd36fe46018d07e2118199820523a6b35301d9c16, name=ceph-7f9750d6-044a-11ef-bc93-c7b262605968-osd-5-deactivate, org.label-schema.license=GPLv2, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240424, org.label-schema.vendor=CentOS, RELEASE=main-4e213a5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, ceph=True) 2024-04-27T04:18:32.065 INFO:journalctl@ceph.osd.5.smithi181.stdout:Apr 27 04:18:31 smithi181 podman[100500]: 2024-04-27 04:18:31.995378005 +0000 UTC m=+0.356932285 container start e7071bcbb51185f97217de414af37615627413264f367e30c61ec75864003620 (image=quay.ceph.io/ceph-ci/ceph@sha256:90a3a9fa9ceb8977f6dad9ccd36fe46018d07e2118199820523a6b35301d9c16, name=ceph-7f9750d6-044a-11ef-bc93-c7b262605968-osd-5-deactivate, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.build-date=20240424, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=main-4e213a5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-27T04:18:32.065 INFO:journalctl@ceph.osd.5.smithi181.stdout:Apr 27 04:18:32 smithi181 podman[100500]: 2024-04-27 04:18:32.066994677 +0000 UTC m=+0.428548956 container attach e7071bcbb51185f97217de414af37615627413264f367e30c61ec75864003620 (image=quay.ceph.io/ceph-ci/ceph@sha256:90a3a9fa9ceb8977f6dad9ccd36fe46018d07e2118199820523a6b35301d9c16, name=ceph-7f9750d6-044a-11ef-bc93-c7b262605968-osd-5-deactivate, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=main-4e213a5, io.buildah.version=1.33.5, org.label-schema.build-date=20240424, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, ceph=True, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS) 2024-04-27T04:18:32.329 INFO:journalctl@ceph.osd.5.smithi181.stdout:Apr 27 04:18:32 smithi181 podman[100500]: 2024-04-27 04:18:32.285031826 +0000 UTC m=+0.646586116 container died e7071bcbb51185f97217de414af37615627413264f367e30c61ec75864003620 (image=quay.ceph.io/ceph-ci/ceph@sha256:90a3a9fa9ceb8977f6dad9ccd36fe46018d07e2118199820523a6b35301d9c16, name=ceph-7f9750d6-044a-11ef-bc93-c7b262605968-osd-5-deactivate, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, RELEASE=main-4e213a5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240424, ceph=True, io.buildah.version=1.33.5, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS) 2024-04-27T04:18:32.636 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:18:32 smithi162 ceph-mon[110641]: osdmap e76: 6 total, 5 up, 6 in 2024-04-27T04:18:32.636 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:18:32 smithi162 ceph-mon[110641]: pgmap v157: 65 pgs: 13 stale+active+clean, 52 active+clean; 208 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-04-27T04:18:32.671 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:18:32 smithi181 ceph-mon[81134]: osdmap e76: 6 total, 5 up, 6 in 2024-04-27T04:18:32.672 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:18:32 smithi181 ceph-mon[81134]: pgmap v157: 65 pgs: 13 stale+active+clean, 52 active+clean; 208 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-04-27T04:18:33.172 INFO:journalctl@ceph.osd.5.smithi181.stdout:Apr 27 04:18:32 smithi181 podman[100525]: 2024-04-27 04:18:32.87084426 +0000 UTC m=+0.579981699 container remove e7071bcbb51185f97217de414af37615627413264f367e30c61ec75864003620 (image=quay.ceph.io/ceph-ci/ceph@sha256:90a3a9fa9ceb8977f6dad9ccd36fe46018d07e2118199820523a6b35301d9c16, name=ceph-7f9750d6-044a-11ef-bc93-c7b262605968-osd-5-deactivate, RELEASE=main-4e213a5, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240424, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD) 2024-04-27T04:18:33.172 INFO:journalctl@ceph.osd.5.smithi181.stdout:Apr 27 04:18:32 smithi181 systemd[1]: ceph-7f9750d6-044a-11ef-bc93-c7b262605968@osd.5.service: Deactivated successfully. 2024-04-27T04:18:33.172 INFO:journalctl@ceph.osd.5.smithi181.stdout:Apr 27 04:18:32 smithi181 systemd[1]: Stopped Ceph osd.5 for 7f9750d6-044a-11ef-bc93-c7b262605968. 2024-04-27T04:18:33.172 INFO:journalctl@ceph.osd.5.smithi181.stdout:Apr 27 04:18:32 smithi181 systemd[1]: ceph-7f9750d6-044a-11ef-bc93-c7b262605968@osd.5.service: Consumed 1min 5.666s CPU time. 2024-04-27T04:18:33.172 INFO:journalctl@ceph.osd.5.smithi181.stdout:Apr 27 04:18:33 smithi181 systemd[1]: Starting Ceph osd.5 for 7f9750d6-044a-11ef-bc93-c7b262605968... 2024-04-27T04:18:33.555 INFO:journalctl@ceph.osd.5.smithi181.stdout:Apr 27 04:18:33 smithi181 podman[100630]: 2024-04-27 04:18:33.304968212 +0000 UTC m=+0.107497218 container create 840a32df0a4fd952f965ab4b5becea5995d50272e9a759cd046af938fcfee036 (image=quay.ceph.io/ceph-ci/ceph@sha256:90a3a9fa9ceb8977f6dad9ccd36fe46018d07e2118199820523a6b35301d9c16, name=ceph-7f9750d6-044a-11ef-bc93-c7b262605968-osd-5-activate, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, GIT_CLEAN=True, RELEASE=main-4e213a5, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, ceph=True, org.label-schema.build-date=20240424, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-27T04:18:33.555 INFO:journalctl@ceph.osd.5.smithi181.stdout:Apr 27 04:18:33 smithi181 podman[100630]: 2024-04-27 04:18:33.215519987 +0000 UTC m=+0.018049004 image pull 67e414ff4bc41092cb31c84c98a9563a32813d02583028679c46b40362afeb3e quay.ceph.io/ceph-ci/ceph@sha256:90a3a9fa9ceb8977f6dad9ccd36fe46018d07e2118199820523a6b35301d9c16 2024-04-27T04:18:33.555 INFO:journalctl@ceph.osd.5.smithi181.stdout:Apr 27 04:18:33 smithi181 podman[100630]: 2024-04-27 04:18:33.557696313 +0000 UTC m=+0.360225316 container init 840a32df0a4fd952f965ab4b5becea5995d50272e9a759cd046af938fcfee036 (image=quay.ceph.io/ceph-ci/ceph@sha256:90a3a9fa9ceb8977f6dad9ccd36fe46018d07e2118199820523a6b35301d9c16, name=ceph-7f9750d6-044a-11ef-bc93-c7b262605968-osd-5-activate, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240424, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, RELEASE=main-4e213a5, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , ceph=True, GIT_CLEAN=True) 2024-04-27T04:18:33.921 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:18:33 smithi181 ceph-mon[81134]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' 2024-04-27T04:18:33.922 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:18:33 smithi181 ceph-mon[81134]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-27T04:18:33.922 INFO:journalctl@ceph.osd.5.smithi181.stdout:Apr 27 04:18:33 smithi181 podman[100630]: 2024-04-27 04:18:33.561015115 +0000 UTC m=+0.363544136 container start 840a32df0a4fd952f965ab4b5becea5995d50272e9a759cd046af938fcfee036 (image=quay.ceph.io/ceph-ci/ceph@sha256:90a3a9fa9ceb8977f6dad9ccd36fe46018d07e2118199820523a6b35301d9c16, name=ceph-7f9750d6-044a-11ef-bc93-c7b262605968-osd-5-activate, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.build-date=20240424, RELEASE=main-4e213a5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, ceph=True, GIT_CLEAN=True) 2024-04-27T04:18:33.922 INFO:journalctl@ceph.osd.5.smithi181.stdout:Apr 27 04:18:33 smithi181 podman[100630]: 2024-04-27 04:18:33.632569102 +0000 UTC m=+0.435098109 container attach 840a32df0a4fd952f965ab4b5becea5995d50272e9a759cd046af938fcfee036 (image=quay.ceph.io/ceph-ci/ceph@sha256:90a3a9fa9ceb8977f6dad9ccd36fe46018d07e2118199820523a6b35301d9c16, name=ceph-7f9750d6-044a-11ef-bc93-c7b262605968-osd-5-activate, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, RELEASE=main-4e213a5, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240424, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, ceph=True, GIT_CLEAN=True) 2024-04-27T04:18:34.136 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:18:33 smithi162 ceph-mon[110641]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' 2024-04-27T04:18:34.136 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:18:33 smithi162 ceph-mon[110641]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-27T04:18:34.388 INFO:journalctl@ceph.osd.5.smithi181.stdout:Apr 27 04:18:34 smithi181 ceph-7f9750d6-044a-11ef-bc93-c7b262605968-osd-5-activate[100646]: Running command: /usr/bin/ceph-authtool --gen-print-key 2024-04-27T04:18:34.388 INFO:journalctl@ceph.osd.5.smithi181.stdout:Apr 27 04:18:34 smithi181 ceph-7f9750d6-044a-11ef-bc93-c7b262605968-osd-5-activate[100646]: Running command: /usr/bin/ceph-authtool --gen-print-key 2024-04-27T04:18:34.388 INFO:journalctl@ceph.osd.5.smithi181.stdout:Apr 27 04:18:34 smithi181 bash[100630]: Running command: /usr/bin/ceph-authtool --gen-print-key 2024-04-27T04:18:34.388 INFO:journalctl@ceph.osd.5.smithi181.stdout:Apr 27 04:18:34 smithi181 bash[100630]: Running command: /usr/bin/ceph-authtool --gen-print-key 2024-04-27T04:18:34.388 INFO:journalctl@ceph.osd.5.smithi181.stdout:Apr 27 04:18:34 smithi181 bash[100630]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2024-04-27T04:18:34.388 INFO:journalctl@ceph.osd.5.smithi181.stdout:Apr 27 04:18:34 smithi181 bash[100630]: Running command: /usr/bin/ceph-bluestore-tool prime-osd-dir --path /var/lib/ceph/osd/ceph-5 --no-mon-config --dev /dev/mapper/vg_nvme-lv_2 2024-04-27T04:18:34.389 INFO:journalctl@ceph.osd.5.smithi181.stdout:Apr 27 04:18:34 smithi181 bash[100630]: Running command: /usr/bin/chown -h ceph:ceph /dev/mapper/vg_nvme-lv_2 2024-04-27T04:18:34.389 INFO:journalctl@ceph.osd.5.smithi181.stdout:Apr 27 04:18:34 smithi181 bash[100630]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-1 2024-04-27T04:18:34.389 INFO:journalctl@ceph.osd.5.smithi181.stdout:Apr 27 04:18:34 smithi181 bash[100630]: Running command: /usr/bin/ln -s /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5/block 2024-04-27T04:18:34.389 INFO:journalctl@ceph.osd.5.smithi181.stdout:Apr 27 04:18:34 smithi181 bash[100630]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2024-04-27T04:18:34.389 INFO:journalctl@ceph.osd.5.smithi181.stdout:Apr 27 04:18:34 smithi181 bash[100630]: --> ceph-volume raw activate successful for osd ID: 5 2024-04-27T04:18:34.389 INFO:journalctl@ceph.osd.5.smithi181.stdout:Apr 27 04:18:34 smithi181 ceph-7f9750d6-044a-11ef-bc93-c7b262605968-osd-5-activate[100646]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2024-04-27T04:18:34.389 INFO:journalctl@ceph.osd.5.smithi181.stdout:Apr 27 04:18:34 smithi181 ceph-7f9750d6-044a-11ef-bc93-c7b262605968-osd-5-activate[100646]: Running command: /usr/bin/ceph-bluestore-tool prime-osd-dir --path /var/lib/ceph/osd/ceph-5 --no-mon-config --dev /dev/mapper/vg_nvme-lv_2 2024-04-27T04:18:34.389 INFO:journalctl@ceph.osd.5.smithi181.stdout:Apr 27 04:18:34 smithi181 ceph-7f9750d6-044a-11ef-bc93-c7b262605968-osd-5-activate[100646]: Running command: /usr/bin/chown -h ceph:ceph /dev/mapper/vg_nvme-lv_2 2024-04-27T04:18:34.389 INFO:journalctl@ceph.osd.5.smithi181.stdout:Apr 27 04:18:34 smithi181 ceph-7f9750d6-044a-11ef-bc93-c7b262605968-osd-5-activate[100646]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-1 2024-04-27T04:18:34.389 INFO:journalctl@ceph.osd.5.smithi181.stdout:Apr 27 04:18:34 smithi181 ceph-7f9750d6-044a-11ef-bc93-c7b262605968-osd-5-activate[100646]: Running command: /usr/bin/ln -s /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5/block 2024-04-27T04:18:34.389 INFO:journalctl@ceph.osd.5.smithi181.stdout:Apr 27 04:18:34 smithi181 ceph-7f9750d6-044a-11ef-bc93-c7b262605968-osd-5-activate[100646]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2024-04-27T04:18:34.389 INFO:journalctl@ceph.osd.5.smithi181.stdout:Apr 27 04:18:34 smithi181 ceph-7f9750d6-044a-11ef-bc93-c7b262605968-osd-5-activate[100646]: --> ceph-volume raw activate successful for osd ID: 5 2024-04-27T04:18:34.389 INFO:journalctl@ceph.osd.5.smithi181.stdout:Apr 27 04:18:34 smithi181 podman[100630]: 2024-04-27 04:18:34.074065596 +0000 UTC m=+0.876594600 container died 840a32df0a4fd952f965ab4b5becea5995d50272e9a759cd046af938fcfee036 (image=quay.ceph.io/ceph-ci/ceph@sha256:90a3a9fa9ceb8977f6dad9ccd36fe46018d07e2118199820523a6b35301d9c16, name=ceph-7f9750d6-044a-11ef-bc93-c7b262605968-osd-5-activate, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.build-date=20240424, RELEASE=main-4e213a5, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, io.buildah.version=1.33.5) 2024-04-27T04:18:34.922 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:18:34 smithi181 ceph-mon[81134]: pgmap v158: 65 pgs: 4 active+undersized+degraded, 6 stale+active+clean, 9 active+undersized, 46 active+clean; 208 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 0 B/s rd, 1 op/s; 11/231 objects degraded (4.762%) 2024-04-27T04:18:34.922 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:18:34 smithi181 ceph-mon[81134]: Health check failed: Degraded data redundancy: 11/231 objects degraded (4.762%), 4 pgs degraded (PG_DEGRADED) 2024-04-27T04:18:34.922 INFO:journalctl@ceph.osd.5.smithi181.stdout:Apr 27 04:18:34 smithi181 podman[100787]: 2024-04-27 04:18:34.653242532 +0000 UTC m=+0.573549778 container remove 840a32df0a4fd952f965ab4b5becea5995d50272e9a759cd046af938fcfee036 (image=quay.ceph.io/ceph-ci/ceph@sha256:90a3a9fa9ceb8977f6dad9ccd36fe46018d07e2118199820523a6b35301d9c16, name=ceph-7f9750d6-044a-11ef-bc93-c7b262605968-osd-5-activate, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, RELEASE=main-4e213a5, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240424, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_CLEAN=True) 2024-04-27T04:18:34.922 INFO:journalctl@ceph.osd.5.smithi181.stdout:Apr 27 04:18:34 smithi181 podman[100854]: 2024-04-27 04:18:34.778953987 +0000 UTC m=+0.018248881 image pull 67e414ff4bc41092cb31c84c98a9563a32813d02583028679c46b40362afeb3e quay.ceph.io/ceph-ci/ceph@sha256:90a3a9fa9ceb8977f6dad9ccd36fe46018d07e2118199820523a6b35301d9c16 2024-04-27T04:18:34.922 INFO:journalctl@ceph.osd.5.smithi181.stdout:Apr 27 04:18:34 smithi181 podman[100854]: 2024-04-27 04:18:34.879174935 +0000 UTC m=+0.118469843 container create 7dd4f287ef2ec9004d3750ff836610a4d6ce80f24dda890fe973358f966d8146 (image=quay.ceph.io/ceph-ci/ceph@sha256:90a3a9fa9ceb8977f6dad9ccd36fe46018d07e2118199820523a6b35301d9c16, name=ceph-7f9750d6-044a-11ef-bc93-c7b262605968-osd-5, ceph=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, RELEASE=main-4e213a5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240424, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2) 2024-04-27T04:18:35.138 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:18:34 smithi162 ceph-mon[110641]: pgmap v158: 65 pgs: 4 active+undersized+degraded, 6 stale+active+clean, 9 active+undersized, 46 active+clean; 208 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 0 B/s rd, 1 op/s; 11/231 objects degraded (4.762%) 2024-04-27T04:18:35.138 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:18:34 smithi162 ceph-mon[110641]: Health check failed: Degraded data redundancy: 11/231 objects degraded (4.762%), 4 pgs degraded (PG_DEGRADED) 2024-04-27T04:18:35.422 INFO:journalctl@ceph.osd.5.smithi181.stdout:Apr 27 04:18:35 smithi181 podman[100854]: 2024-04-27 04:18:35.123441207 +0000 UTC m=+0.362736073 container init 7dd4f287ef2ec9004d3750ff836610a4d6ce80f24dda890fe973358f966d8146 (image=quay.ceph.io/ceph-ci/ceph@sha256:90a3a9fa9ceb8977f6dad9ccd36fe46018d07e2118199820523a6b35301d9c16, name=ceph-7f9750d6-044a-11ef-bc93-c7b262605968-osd-5, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240424, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=main-4e213a5, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, ceph=True) 2024-04-27T04:18:35.422 INFO:journalctl@ceph.osd.5.smithi181.stdout:Apr 27 04:18:35 smithi181 podman[100854]: 2024-04-27 04:18:35.12647854 +0000 UTC m=+0.365773407 container start 7dd4f287ef2ec9004d3750ff836610a4d6ce80f24dda890fe973358f966d8146 (image=quay.ceph.io/ceph-ci/ceph@sha256:90a3a9fa9ceb8977f6dad9ccd36fe46018d07e2118199820523a6b35301d9c16, name=ceph-7f9750d6-044a-11ef-bc93-c7b262605968-osd-5, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=main-4e213a5, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240424, org.label-schema.schema-version=1.0) 2024-04-27T04:18:35.422 INFO:journalctl@ceph.osd.5.smithi181.stdout:Apr 27 04:18:35 smithi181 ceph-osd[100873]: -- 172.21.15.181:0/3231186231 <== mon.1 v2:172.21.15.181:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x558674505680 con 0x5586744cf000 2024-04-27T04:18:35.422 INFO:journalctl@ceph.osd.5.smithi181.stdout:Apr 27 04:18:35 smithi181 bash[100854]: 7dd4f287ef2ec9004d3750ff836610a4d6ce80f24dda890fe973358f966d8146 2024-04-27T04:18:35.422 INFO:journalctl@ceph.osd.5.smithi181.stdout:Apr 27 04:18:35 smithi181 systemd[1]: Started Ceph osd.5 for 7f9750d6-044a-11ef-bc93-c7b262605968. 2024-04-27T04:18:35.922 INFO:journalctl@ceph.osd.5.smithi181.stdout:Apr 27 04:18:35 smithi181 ceph-7f9750d6-044a-11ef-bc93-c7b262605968-osd-5[100869]: 2024-04-27T04:18:35.673+0000 7fa91ee227c0 -1 Falling back to public interface 2024-04-27T04:18:36.421 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:18:36 smithi181 ceph-mon[81134]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' 2024-04-27T04:18:36.422 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:18:36 smithi181 ceph-mon[81134]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' 2024-04-27T04:18:36.422 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:18:36 smithi181 ceph-mon[81134]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T04:18:36.422 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:18:36 smithi181 ceph-mon[81134]: pgmap v159: 65 pgs: 12 active+undersized+degraded, 16 active+undersized, 37 active+clean; 208 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 0 B/s rd, 1 op/s; 36/231 objects degraded (15.584%) 2024-04-27T04:18:36.636 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:18:36 smithi162 ceph-mon[110641]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' 2024-04-27T04:18:36.636 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:18:36 smithi162 ceph-mon[110641]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' 2024-04-27T04:18:36.637 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:18:36 smithi162 ceph-mon[110641]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T04:18:36.637 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:18:36 smithi162 ceph-mon[110641]: pgmap v159: 65 pgs: 12 active+undersized+degraded, 16 active+undersized, 37 active+clean; 208 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 0 B/s rd, 1 op/s; 36/231 objects degraded (15.584%) 2024-04-27T04:18:39.136 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:18:38 smithi162 ceph-mon[110641]: pgmap v160: 65 pgs: 12 active+undersized+degraded, 16 active+undersized, 37 active+clean; 208 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 0 B/s rd, 0 op/s; 36/231 objects degraded (15.584%) 2024-04-27T04:18:39.136 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:18:38 smithi162 ceph-mon[110641]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' 2024-04-27T04:18:39.137 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:18:38 smithi162 ceph-mon[110641]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' 2024-04-27T04:18:39.171 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:18:38 smithi181 ceph-mon[81134]: pgmap v160: 65 pgs: 12 active+undersized+degraded, 16 active+undersized, 37 active+clean; 208 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 0 B/s rd, 0 op/s; 36/231 objects degraded (15.584%) 2024-04-27T04:18:39.172 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:18:38 smithi181 ceph-mon[81134]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' 2024-04-27T04:18:39.172 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:18:38 smithi181 ceph-mon[81134]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' 2024-04-27T04:18:40.136 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:18:39 smithi162 ceph-mon[110641]: Health check update: Degraded data redundancy: 36/231 objects degraded (15.584%), 12 pgs degraded (PG_DEGRADED) 2024-04-27T04:18:40.136 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:18:39 smithi162 ceph-mon[110641]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' 2024-04-27T04:18:40.137 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:18:39 smithi162 ceph-mon[110641]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' 2024-04-27T04:18:40.139 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:18:39 smithi181 ceph-mon[81134]: Health check update: Degraded data redundancy: 36/231 objects degraded (15.584%), 12 pgs degraded (PG_DEGRADED) 2024-04-27T04:18:40.139 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:18:39 smithi181 ceph-mon[81134]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' 2024-04-27T04:18:40.139 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:18:39 smithi181 ceph-mon[81134]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' 2024-04-27T04:18:40.672 INFO:journalctl@ceph.osd.5.smithi181.stdout:Apr 27 04:18:40 smithi181 ceph-7f9750d6-044a-11ef-bc93-c7b262605968-osd-5[100869]: 2024-04-27T04:18:40.587+0000 7fa91ee227c0 -1 osd.5 74 log_to_monitors true 2024-04-27T04:18:41.130 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:18:40 smithi181 ceph-mon[81134]: pgmap v161: 65 pgs: 12 active+undersized+degraded, 16 active+undersized, 37 active+clean; 208 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 0 B/s rd, 0 op/s; 36/231 objects degraded (15.584%) 2024-04-27T04:18:41.130 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:18:40 smithi181 ceph-mon[81134]: from='osd.5 [v2:172.21.15.181:6816/2414821263,v1:172.21.15.181:6817/2414821263]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-04-27T04:18:41.131 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:18:40 smithi181 ceph-mon[81134]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-04-27T04:18:41.136 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:18:40 smithi162 ceph-mon[110641]: pgmap v161: 65 pgs: 12 active+undersized+degraded, 16 active+undersized, 37 active+clean; 208 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 0 B/s rd, 0 op/s; 36/231 objects degraded (15.584%) 2024-04-27T04:18:41.136 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:18:40 smithi162 ceph-mon[110641]: from='osd.5 [v2:172.21.15.181:6816/2414821263,v1:172.21.15.181:6817/2414821263]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-04-27T04:18:41.136 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:18:40 smithi162 ceph-mon[110641]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-04-27T04:18:42.672 INFO:journalctl@ceph.osd.5.smithi181.stdout:Apr 27 04:18:42 smithi181 ceph-7f9750d6-044a-11ef-bc93-c7b262605968-osd-5[100869]: 2024-04-27T04:18:42.500+0000 7fa90c272700 -1 osd.5 74 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-27T04:18:42.672 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:18:42 smithi181 ceph-mon[81134]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-04-27T04:18:42.672 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:18:42 smithi181 ceph-mon[81134]: osdmap e77: 6 total, 5 up, 6 in 2024-04-27T04:18:42.672 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:18:42 smithi181 ceph-mon[81134]: from='osd.5 [v2:172.21.15.181:6816/2414821263,v1:172.21.15.181:6817/2414821263]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi181", "root=default"]}]: dispatch 2024-04-27T04:18:42.672 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:18:42 smithi181 ceph-mon[81134]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi181", "root=default"]}]: dispatch 2024-04-27T04:18:42.673 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:18:42 smithi181 ceph-mon[81134]: pgmap v163: 65 pgs: 12 active+undersized+degraded, 16 active+undersized, 37 active+clean; 208 MiB data, 846 MiB used, 536 GiB / 536 GiB avail; 0 B/s rd, 1 op/s; 36/231 objects degraded (15.584%) 2024-04-27T04:18:42.673 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:18:42 smithi181 ceph-mon[81134]: from='osd.5 ' entity='osd.5' 2024-04-27T04:18:42.886 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:18:42 smithi162 ceph-mon[110641]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-04-27T04:18:42.887 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:18:42 smithi162 ceph-mon[110641]: osdmap e77: 6 total, 5 up, 6 in 2024-04-27T04:18:42.887 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:18:42 smithi162 ceph-mon[110641]: from='osd.5 [v2:172.21.15.181:6816/2414821263,v1:172.21.15.181:6817/2414821263]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi181", "root=default"]}]: dispatch 2024-04-27T04:18:42.887 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:18:42 smithi162 ceph-mon[110641]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi181", "root=default"]}]: dispatch 2024-04-27T04:18:42.887 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:18:42 smithi162 ceph-mon[110641]: pgmap v163: 65 pgs: 12 active+undersized+degraded, 16 active+undersized, 37 active+clean; 208 MiB data, 846 MiB used, 536 GiB / 536 GiB avail; 0 B/s rd, 1 op/s; 36/231 objects degraded (15.584%) 2024-04-27T04:18:42.887 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:18:42 smithi162 ceph-mon[110641]: from='osd.5 ' entity='osd.5' 2024-04-27T04:18:44.387 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:18:44 smithi162 ceph-mon[110641]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' 2024-04-27T04:18:44.387 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:18:44 smithi162 ceph-mon[110641]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' 2024-04-27T04:18:44.387 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:18:44 smithi162 ceph-mon[110641]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T04:18:44.387 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:18:44 smithi162 ceph-mon[110641]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T04:18:44.387 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:18:44 smithi162 ceph-mon[110641]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' 2024-04-27T04:18:44.387 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:18:44 smithi162 ceph-mon[110641]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T04:18:44.387 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:18:44 smithi162 ceph-mon[110641]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "versions"}]: dispatch 2024-04-27T04:18:44.387 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:18:44 smithi162 ceph-mon[110641]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "versions"}]: dispatch 2024-04-27T04:18:44.387 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:18:44 smithi162 ceph-mon[110641]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "versions"}]: dispatch 2024-04-27T04:18:44.387 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:18:44 smithi162 ceph-mon[110641]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "versions"}]: dispatch 2024-04-27T04:18:44.388 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:18:44 smithi162 ceph-mon[110641]: Upgrade: Setting container_image for all osd 2024-04-27T04:18:44.388 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:18:44 smithi162 ceph-mon[110641]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' 2024-04-27T04:18:44.388 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:18:44 smithi162 ceph-mon[110641]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.0"}]: dispatch 2024-04-27T04:18:44.388 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:18:44 smithi162 ceph-mon[110641]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.0"}]': finished 2024-04-27T04:18:44.388 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:18:44 smithi162 ceph-mon[110641]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.1"}]: dispatch 2024-04-27T04:18:44.388 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:18:44 smithi162 ceph-mon[110641]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.1"}]': finished 2024-04-27T04:18:44.388 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:18:44 smithi162 ceph-mon[110641]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.2"}]: dispatch 2024-04-27T04:18:44.388 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:18:44 smithi162 ceph-mon[110641]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.2"}]': finished 2024-04-27T04:18:44.388 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:18:44 smithi162 ceph-mon[110641]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.3"}]: dispatch 2024-04-27T04:18:44.388 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:18:44 smithi162 ceph-mon[110641]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.3"}]': finished 2024-04-27T04:18:44.388 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:18:44 smithi162 ceph-mon[110641]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.4"}]: dispatch 2024-04-27T04:18:44.388 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:18:44 smithi162 ceph-mon[110641]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.4"}]': finished 2024-04-27T04:18:44.389 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:18:44 smithi162 ceph-mon[110641]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.5"}]: dispatch 2024-04-27T04:18:44.389 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:18:44 smithi162 ceph-mon[110641]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.5"}]': finished 2024-04-27T04:18:44.389 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:18:44 smithi162 ceph-mon[110641]: Upgrade: Setting require_osd_release to 19 squid 2024-04-27T04:18:44.389 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:18:44 smithi162 ceph-mon[110641]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "osd require-osd-release", "release": "squid"}]: dispatch 2024-04-27T04:18:44.389 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:18:44 smithi162 ceph-mon[110641]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-04-27T04:18:44.389 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:18:44 smithi162 ceph-mon[110641]: Health check cleared: OSD_UPGRADE_FINISHED (was: all OSDs are running squid or later but require_osd_release < squid) 2024-04-27T04:18:44.389 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:18:44 smithi162 ceph-mon[110641]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' cmd='[{"prefix": "osd require-osd-release", "release": "squid"}]': finished 2024-04-27T04:18:44.389 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:18:44 smithi162 ceph-mon[110641]: osd.5 [v2:172.21.15.181:6816/2414821263,v1:172.21.15.181:6817/2414821263] boot 2024-04-27T04:18:44.389 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:18:44 smithi162 ceph-mon[110641]: osdmap e78: 6 total, 6 up, 6 in 2024-04-27T04:18:44.389 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:18:44 smithi162 ceph-mon[110641]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-27T04:18:44.389 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:18:44 smithi162 ceph-mon[110641]: Upgrade: Disabling standby-replay for filesystem cephfs 2024-04-27T04:18:44.389 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:18:44 smithi162 ceph-mon[110641]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' 2024-04-27T04:18:44.389 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:18:44 smithi162 ceph-mon[110641]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "0"}]: dispatch 2024-04-27T04:18:44.390 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:18:44 smithi162 ceph-mon[110641]: pgmap v165: 65 pgs: 12 active+undersized+degraded, 16 active+undersized, 37 active+clean; 208 MiB data, 846 MiB used, 536 GiB / 536 GiB avail; 639 B/s rd, 1 op/s; 36/231 objects degraded (15.584%) 2024-04-27T04:18:44.422 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:18:44 smithi181 ceph-mon[81134]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' 2024-04-27T04:18:44.423 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:18:44 smithi181 ceph-mon[81134]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' 2024-04-27T04:18:44.423 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:18:44 smithi181 ceph-mon[81134]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T04:18:44.423 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:18:44 smithi181 ceph-mon[81134]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T04:18:44.423 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:18:44 smithi181 ceph-mon[81134]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' 2024-04-27T04:18:44.423 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:18:44 smithi181 ceph-mon[81134]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T04:18:44.423 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:18:44 smithi181 ceph-mon[81134]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "versions"}]: dispatch 2024-04-27T04:18:44.423 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:18:44 smithi181 ceph-mon[81134]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "versions"}]: dispatch 2024-04-27T04:18:44.423 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:18:44 smithi181 ceph-mon[81134]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "versions"}]: dispatch 2024-04-27T04:18:44.423 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:18:44 smithi181 ceph-mon[81134]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "versions"}]: dispatch 2024-04-27T04:18:44.423 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:18:44 smithi181 ceph-mon[81134]: Upgrade: Setting container_image for all osd 2024-04-27T04:18:44.423 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:18:44 smithi181 ceph-mon[81134]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' 2024-04-27T04:18:44.424 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:18:44 smithi181 ceph-mon[81134]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.0"}]: dispatch 2024-04-27T04:18:44.424 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:18:44 smithi181 ceph-mon[81134]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.0"}]': finished 2024-04-27T04:18:44.424 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:18:44 smithi181 ceph-mon[81134]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.1"}]: dispatch 2024-04-27T04:18:44.424 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:18:44 smithi181 ceph-mon[81134]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.1"}]': finished 2024-04-27T04:18:44.424 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:18:44 smithi181 ceph-mon[81134]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.2"}]: dispatch 2024-04-27T04:18:44.424 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:18:44 smithi181 ceph-mon[81134]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.2"}]': finished 2024-04-27T04:18:44.424 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:18:44 smithi181 ceph-mon[81134]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.3"}]: dispatch 2024-04-27T04:18:44.424 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:18:44 smithi181 ceph-mon[81134]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.3"}]': finished 2024-04-27T04:18:44.424 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:18:44 smithi181 ceph-mon[81134]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.4"}]: dispatch 2024-04-27T04:18:44.424 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:18:44 smithi181 ceph-mon[81134]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.4"}]': finished 2024-04-27T04:18:44.424 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:18:44 smithi181 ceph-mon[81134]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.5"}]: dispatch 2024-04-27T04:18:44.424 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:18:44 smithi181 ceph-mon[81134]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.5"}]': finished 2024-04-27T04:18:44.424 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:18:44 smithi181 ceph-mon[81134]: Upgrade: Setting require_osd_release to 19 squid 2024-04-27T04:18:44.424 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:18:44 smithi181 ceph-mon[81134]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "osd require-osd-release", "release": "squid"}]: dispatch 2024-04-27T04:18:44.424 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:18:44 smithi181 ceph-mon[81134]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-04-27T04:18:44.425 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:18:44 smithi181 ceph-mon[81134]: Health check cleared: OSD_UPGRADE_FINISHED (was: all OSDs are running squid or later but require_osd_release < squid) 2024-04-27T04:18:44.425 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:18:44 smithi181 ceph-mon[81134]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' cmd='[{"prefix": "osd require-osd-release", "release": "squid"}]': finished 2024-04-27T04:18:44.425 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:18:44 smithi181 ceph-mon[81134]: osd.5 [v2:172.21.15.181:6816/2414821263,v1:172.21.15.181:6817/2414821263] boot 2024-04-27T04:18:44.425 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:18:44 smithi181 ceph-mon[81134]: osdmap e78: 6 total, 6 up, 6 in 2024-04-27T04:18:44.425 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:18:44 smithi181 ceph-mon[81134]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-27T04:18:44.425 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:18:44 smithi181 ceph-mon[81134]: Upgrade: Disabling standby-replay for filesystem cephfs 2024-04-27T04:18:44.425 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:18:44 smithi181 ceph-mon[81134]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' 2024-04-27T04:18:44.425 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:18:44 smithi181 ceph-mon[81134]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "0"}]: dispatch 2024-04-27T04:18:44.425 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:18:44 smithi181 ceph-mon[81134]: pgmap v165: 65 pgs: 12 active+undersized+degraded, 16 active+undersized, 37 active+clean; 208 MiB data, 846 MiB used, 536 GiB / 536 GiB avail; 639 B/s rd, 1 op/s; 36/231 objects degraded (15.584%) 2024-04-27T04:18:45.374 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:18:45 smithi162 ceph-mon[110641]: osdmap e79: 6 total, 6 up, 6 in 2024-04-27T04:18:45.374 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:18:45 smithi162 ceph-mon[110641]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "0"}]': finished 2024-04-27T04:18:45.375 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:18:45 smithi162 ceph-mon[110641]: fsmap cephfs:1 {0=cephfs.smithi181.dzomgh=up:active} 2 up:standby 2024-04-27T04:18:45.375 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:18:45 smithi162 ceph-mon[110641]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T04:18:45.375 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:18:45 smithi162 ceph-mon[110641]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T04:18:45.375 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:18:45 smithi162 ceph-mon[110641]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T04:18:45.375 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:18:45 smithi162 ceph-mon[110641]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' 2024-04-27T04:18:45.375 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:18:45 smithi162 ceph-mon[110641]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T04:18:45.375 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:18:45 smithi162 ceph-mon[110641]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "versions"}]: dispatch 2024-04-27T04:18:45.375 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:18:45 smithi162 ceph-mon[110641]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "versions"}]: dispatch 2024-04-27T04:18:45.375 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:18:45 smithi162 ceph-mon[110641]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "versions"}]: dispatch 2024-04-27T04:18:45.375 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:18:45 smithi162 ceph-mon[110641]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "versions"}]: dispatch 2024-04-27T04:18:45.422 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:18:45 smithi181 ceph-mon[81134]: osdmap e79: 6 total, 6 up, 6 in 2024-04-27T04:18:45.422 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:18:45 smithi181 ceph-mon[81134]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "0"}]': finished 2024-04-27T04:18:45.422 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:18:45 smithi181 ceph-mon[81134]: fsmap cephfs:1 {0=cephfs.smithi181.dzomgh=up:active} 2 up:standby 2024-04-27T04:18:45.422 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:18:45 smithi181 ceph-mon[81134]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T04:18:45.422 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:18:45 smithi181 ceph-mon[81134]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T04:18:45.422 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:18:45 smithi181 ceph-mon[81134]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T04:18:45.422 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:18:45 smithi181 ceph-mon[81134]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' 2024-04-27T04:18:45.422 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:18:45 smithi181 ceph-mon[81134]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T04:18:45.422 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:18:45 smithi181 ceph-mon[81134]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "versions"}]: dispatch 2024-04-27T04:18:45.422 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:18:45 smithi181 ceph-mon[81134]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "versions"}]: dispatch 2024-04-27T04:18:45.422 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:18:45 smithi181 ceph-mon[81134]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "versions"}]: dispatch 2024-04-27T04:18:45.423 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:18:45 smithi181 ceph-mon[81134]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "versions"}]: dispatch 2024-04-27T04:18:46.386 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:18:46 smithi162 ceph-mon[110641]: osdmap e80: 6 total, 6 up, 6 in 2024-04-27T04:18:46.386 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:18:46 smithi162 ceph-mon[110641]: mds.? [v2:172.21.15.162:6828/1115643134,v1:172.21.15.162:6829/1115643134] up:boot 2024-04-27T04:18:46.386 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:18:46 smithi162 ceph-mon[110641]: fsmap cephfs:1 {0=cephfs.smithi181.dzomgh=up:active} 3 up:standby 2024-04-27T04:18:46.387 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:18:46 smithi162 ceph-mon[110641]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi162.xcnapk"}]: dispatch 2024-04-27T04:18:46.387 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:18:46 smithi162 ceph-mon[110641]: pgmap v168: 65 pgs: 2 peering, 6 active+undersized+degraded, 13 active+undersized, 44 active+clean; 208 MiB data, 846 MiB used, 536 GiB / 536 GiB avail; 741 B/s rd, 1 op/s; 16/231 objects degraded (6.926%) 2024-04-27T04:18:46.422 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:18:46 smithi181 ceph-mon[81134]: osdmap e80: 6 total, 6 up, 6 in 2024-04-27T04:18:46.422 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:18:46 smithi181 ceph-mon[81134]: mds.? [v2:172.21.15.162:6828/1115643134,v1:172.21.15.162:6829/1115643134] up:boot 2024-04-27T04:18:46.422 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:18:46 smithi181 ceph-mon[81134]: fsmap cephfs:1 {0=cephfs.smithi181.dzomgh=up:active} 3 up:standby 2024-04-27T04:18:46.422 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:18:46 smithi181 ceph-mon[81134]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi162.xcnapk"}]: dispatch 2024-04-27T04:18:46.422 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:18:46 smithi181 ceph-mon[81134]: pgmap v168: 65 pgs: 2 peering, 6 active+undersized+degraded, 13 active+undersized, 44 active+clean; 208 MiB data, 846 MiB used, 536 GiB / 536 GiB avail; 741 B/s rd, 1 op/s; 16/231 objects degraded (6.926%) 2024-04-27T04:18:47.469 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:18:47 smithi162 ceph-mon[110641]: Upgrade: Updating mds.cephfs.smithi162.gzlyjj 2024-04-27T04:18:47.469 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:18:47 smithi162 ceph-mon[110641]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' 2024-04-27T04:18:47.469 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:18:47 smithi162 ceph-mon[110641]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi162.gzlyjj", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-27T04:18:47.469 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:18:47 smithi162 ceph-mon[110641]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T04:18:47.469 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:18:47 smithi162 ceph-mon[110641]: Deploying daemon mds.cephfs.smithi162.gzlyjj on smithi162 2024-04-27T04:18:47.469 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:18:47 smithi162 ceph-mon[110641]: Health check update: Degraded data redundancy: 16/231 objects degraded (6.926%), 6 pgs degraded (PG_DEGRADED) 2024-04-27T04:18:47.671 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:18:47 smithi181 ceph-mon[81134]: Upgrade: Updating mds.cephfs.smithi162.gzlyjj 2024-04-27T04:18:47.671 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:18:47 smithi181 ceph-mon[81134]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' 2024-04-27T04:18:47.672 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:18:47 smithi181 ceph-mon[81134]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi162.gzlyjj", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-27T04:18:47.672 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:18:47 smithi181 ceph-mon[81134]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T04:18:47.672 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:18:47 smithi181 ceph-mon[81134]: Deploying daemon mds.cephfs.smithi162.gzlyjj on smithi162 2024-04-27T04:18:47.672 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:18:47 smithi181 ceph-mon[81134]: Health check update: Degraded data redundancy: 16/231 objects degraded (6.926%), 6 pgs degraded (PG_DEGRADED) 2024-04-27T04:18:49.137 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:18:48 smithi162 ceph-mon[110641]: pgmap v169: 65 pgs: 2 peering, 2 active+undersized+degraded, 3 active+undersized, 58 active+clean; 208 MiB data, 851 MiB used, 536 GiB / 536 GiB avail; 511 B/s rd, 1 op/s; 4/231 objects degraded (1.732%) 2024-04-27T04:18:49.137 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:18:48 smithi162 ceph-mon[110641]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' 2024-04-27T04:18:49.137 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:18:48 smithi162 ceph-mon[110641]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-27T04:18:49.171 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:18:48 smithi181 ceph-mon[81134]: pgmap v169: 65 pgs: 2 peering, 2 active+undersized+degraded, 3 active+undersized, 58 active+clean; 208 MiB data, 851 MiB used, 536 GiB / 536 GiB avail; 511 B/s rd, 1 op/s; 4/231 objects degraded (1.732%) 2024-04-27T04:18:49.171 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:18:48 smithi181 ceph-mon[81134]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' 2024-04-27T04:18:49.171 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:18:48 smithi181 ceph-mon[81134]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-27T04:18:50.136 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:18:49 smithi162 ceph-mon[110641]: osdmap e81: 6 total, 6 up, 6 in 2024-04-27T04:18:50.136 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:18:49 smithi162 ceph-mon[110641]: fsmap cephfs:1 {0=cephfs.smithi181.dzomgh=up:active} 2 up:standby 2024-04-27T04:18:50.171 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:18:49 smithi181 ceph-mon[81134]: osdmap e81: 6 total, 6 up, 6 in 2024-04-27T04:18:50.171 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:18:49 smithi181 ceph-mon[81134]: fsmap cephfs:1 {0=cephfs.smithi181.dzomgh=up:active} 2 up:standby 2024-04-27T04:18:51.136 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:18:50 smithi162 ceph-mon[110641]: pgmap v171: 65 pgs: 65 active+clean; 208 MiB data, 851 MiB used, 536 GiB / 536 GiB avail; 511 B/s rd, 0 op/s 2024-04-27T04:18:51.136 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:18:50 smithi162 ceph-mon[110641]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 4/231 objects degraded (1.732%), 2 pgs degraded) 2024-04-27T04:18:51.136 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:18:50 smithi162 ceph-mon[110641]: Cluster is now healthy 2024-04-27T04:18:51.171 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:18:50 smithi181 ceph-mon[81134]: pgmap v171: 65 pgs: 65 active+clean; 208 MiB data, 851 MiB used, 536 GiB / 536 GiB avail; 511 B/s rd, 0 op/s 2024-04-27T04:18:51.171 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:18:50 smithi181 ceph-mon[81134]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 4/231 objects degraded (1.732%), 2 pgs degraded) 2024-04-27T04:18:51.172 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:18:50 smithi181 ceph-mon[81134]: Cluster is now healthy 2024-04-27T04:18:53.136 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:18:52 smithi162 ceph-mon[110641]: pgmap v172: 65 pgs: 65 active+clean; 208 MiB data, 851 MiB used, 536 GiB / 536 GiB avail; 405 B/s rd, 0 op/s 2024-04-27T04:18:53.171 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:18:52 smithi181 ceph-mon[81134]: pgmap v172: 65 pgs: 65 active+clean; 208 MiB data, 851 MiB used, 536 GiB / 536 GiB avail; 405 B/s rd, 0 op/s 2024-04-27T04:18:54.991 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:18:54 smithi162 ceph-mon[110641]: pgmap v173: 65 pgs: 65 active+clean; 208 MiB data, 851 MiB used, 536 GiB / 536 GiB avail; 0 B/s rd, 0 op/s 2024-04-27T04:18:54.992 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:18:54 smithi162 ceph-mon[110641]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' 2024-04-27T04:18:54.992 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:18:54 smithi162 ceph-mon[110641]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' 2024-04-27T04:18:54.992 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:18:54 smithi162 ceph-mon[110641]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T04:18:55.172 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:18:54 smithi181 ceph-mon[81134]: pgmap v173: 65 pgs: 65 active+clean; 208 MiB data, 851 MiB used, 536 GiB / 536 GiB avail; 0 B/s rd, 0 op/s 2024-04-27T04:18:55.172 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:18:54 smithi181 ceph-mon[81134]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' 2024-04-27T04:18:55.172 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:18:54 smithi181 ceph-mon[81134]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' 2024-04-27T04:18:55.172 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:18:54 smithi181 ceph-mon[81134]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T04:18:55.973 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:18:55 smithi162 ceph-mon[110641]: mds.? [v2:172.21.15.162:6826/1198130667,v1:172.21.15.162:6827/1198130667] up:boot 2024-04-27T04:18:55.973 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:18:55 smithi162 ceph-mon[110641]: fsmap cephfs:1 {0=cephfs.smithi181.dzomgh=up:active} 3 up:standby 2024-04-27T04:18:55.974 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:18:55 smithi162 ceph-mon[110641]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi162.gzlyjj"}]: dispatch 2024-04-27T04:18:56.171 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:18:55 smithi181 ceph-mon[81134]: mds.? [v2:172.21.15.162:6826/1198130667,v1:172.21.15.162:6827/1198130667] up:boot 2024-04-27T04:18:56.172 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:18:55 smithi181 ceph-mon[81134]: fsmap cephfs:1 {0=cephfs.smithi181.dzomgh=up:active} 3 up:standby 2024-04-27T04:18:56.172 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:18:55 smithi181 ceph-mon[81134]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi162.gzlyjj"}]: dispatch 2024-04-27T04:18:56.408 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:18:56.408+0000 7f5e5c968700 1 -- 172.21.15.162:0/3221597378 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5e58102e00 msgr2=0x7f5e581031d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:18:56.408 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:18:56.408+0000 7f5e5c968700 1 --2- 172.21.15.162:0/3221597378 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5e58102e00 0x7f5e581031d0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f5e48004940 tx=0x7f5e48009e30 comp rx=0 tx=0).stop 2024-04-27T04:18:56.409 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:18:56.408+0000 7f5e5c968700 1 -- 172.21.15.162:0/3221597378 shutdown_connections 2024-04-27T04:18:56.409 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:18:56.408+0000 7f5e5c968700 1 --2- 172.21.15.162:0/3221597378 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5e581037a0 0x7f5e580fce10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:18:56.409 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:18:56.408+0000 7f5e5c968700 1 --2- 172.21.15.162:0/3221597378 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5e58102e00 0x7f5e581031d0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:18:56.409 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:18:56.408+0000 7f5e5c968700 1 -- 172.21.15.162:0/3221597378 >> 172.21.15.162:0/3221597378 conn(0x7f5e580f89b0 msgr2=0x7f5e580fadc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:18:56.409 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:18:56.408+0000 7f5e5c968700 1 -- 172.21.15.162:0/3221597378 shutdown_connections 2024-04-27T04:18:56.409 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:18:56.408+0000 7f5e5c968700 1 -- 172.21.15.162:0/3221597378 wait complete. 2024-04-27T04:18:56.409 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:18:56.408+0000 7f5e5c968700 1 Processor -- start 2024-04-27T04:18:56.409 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:18:56.408+0000 7f5e5c968700 1 -- start start 2024-04-27T04:18:56.409 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:18:56.409+0000 7f5e5c968700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5e58102e00 0x7f5e58113ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:18:56.409 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:18:56.409+0000 7f5e5c968700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5e581037a0 0x7f5e581140e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:18:56.409 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:18:56.409+0000 7f5e5c968700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e58114730 con 0x7f5e581037a0 2024-04-27T04:18:56.409 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:18:56.409+0000 7f5e5c968700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e5810db70 con 0x7f5e58102e00 2024-04-27T04:18:56.409 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:18:56.409+0000 7f5e55d9b700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5e581037a0 0x7f5e581140e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:18:56.409 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:18:56.409+0000 7f5e55d9b700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5e581037a0 0x7f5e581140e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:59368/0 (socket says 172.21.15.162:59368) 2024-04-27T04:18:56.410 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:18:56.409+0000 7f5e55d9b700 1 -- 172.21.15.162:0/3166906400 learned_addr learned my addr 172.21.15.162:0/3166906400 (peer_addr_for_me v2:172.21.15.162:0/0) 2024-04-27T04:18:56.410 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:18:56.409+0000 7f5e5659c700 1 --2- 172.21.15.162:0/3166906400 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5e58102e00 0x7f5e58113ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:18:56.410 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:18:56.409+0000 7f5e55d9b700 1 -- 172.21.15.162:0/3166906400 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5e58102e00 msgr2=0x7f5e58113ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:18:56.410 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:18:56.409+0000 7f5e55d9b700 1 --2- 172.21.15.162:0/3166906400 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5e58102e00 0x7f5e58113ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:18:56.410 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:18:56.409+0000 7f5e55d9b700 1 -- 172.21.15.162:0/3166906400 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e48005040 con 0x7f5e581037a0 2024-04-27T04:18:56.410 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:18:56.409+0000 7f5e55d9b700 1 --2- 172.21.15.162:0/3166906400 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5e581037a0 0x7f5e581140e0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f5e4c002a50 tx=0x7f5e4c004600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:18:56.410 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:18:56.409+0000 7f5e46ffd700 1 -- 172.21.15.162:0/3166906400 <== mon.0 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5e4c007cc0 con 0x7f5e581037a0 2024-04-27T04:18:56.410 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:18:56.409+0000 7f5e46ffd700 1 -- 172.21.15.162:0/3166906400 <== mon.0 v2:172.21.15.162:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f5e4c00f040 con 0x7f5e581037a0 2024-04-27T04:18:56.410 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:18:56.409+0000 7f5e5c968700 1 -- 172.21.15.162:0/3166906400 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5e5810de50 con 0x7f5e581037a0 2024-04-27T04:18:56.410 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:18:56.410+0000 7f5e46ffd700 1 -- 172.21.15.162:0/3166906400 <== mon.0 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5e4c00d9c0 con 0x7f5e581037a0 2024-04-27T04:18:56.410 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:18:56.410+0000 7f5e5c968700 1 -- 172.21.15.162:0/3166906400 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5e5810e410 con 0x7f5e581037a0 2024-04-27T04:18:56.411 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:18:56.411+0000 7f5e5c968700 1 -- 172.21.15.162:0/3166906400 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5e58100110 con 0x7f5e581037a0 2024-04-27T04:18:56.411 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:18:56.412+0000 7f5e46ffd700 1 -- 172.21.15.162:0/3166906400 <== mon.0 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 39) v1 ==== 92747+0+0 (secure 0 0 0) 0x7f5e4c009730 con 0x7f5e581037a0 2024-04-27T04:18:56.411 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:18:56.412+0000 7f5e46ffd700 1 --2- 172.21.15.162:0/3166906400 >> [v2:172.21.15.162:6800/1396866108,v1:172.21.15.162:6801/1396866108] conn(0x7f5e4006ec00 0x7f5e400710b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:18:56.411 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:18:56.412+0000 7f5e46ffd700 1 -- 172.21.15.162:0/3166906400 <== mon.0 v2:172.21.15.162:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 6410+0+0 (secure 0 0 0) 0x7f5e4c08e590 con 0x7f5e581037a0 2024-04-27T04:18:56.412 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:18:56.412+0000 7f5e5659c700 1 --2- 172.21.15.162:0/3166906400 >> [v2:172.21.15.162:6800/1396866108,v1:172.21.15.162:6801/1396866108] conn(0x7f5e4006ec00 0x7f5e400710b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:18:56.412 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:18:56.413+0000 7f5e5659c700 1 --2- 172.21.15.162:0/3166906400 >> [v2:172.21.15.162:6800/1396866108,v1:172.21.15.162:6801/1396866108] conn(0x7f5e4006ec00 0x7f5e400710b0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f5e48004f90 tx=0x7f5e48009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:18:56.416 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:18:56.417+0000 7f5e46ffd700 1 -- 172.21.15.162:0/3166906400 <== mon.0 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7f5e4c05aec0 con 0x7f5e581037a0 2024-04-27T04:18:56.643 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:18:56.644+0000 7f5e5c968700 1 -- 172.21.15.162:0/3166906400 --> [v2:172.21.15.162:6800/1396866108,v1:172.21.15.162:6801/1396866108] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f5e5810f290 con 0x7f5e4006ec00 2024-04-27T04:18:56.644 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:18:56.645+0000 7f5e46ffd700 1 -- 172.21.15.162:0/3166906400 <== mgr.34104 v2:172.21.15.162:6800/1396866108 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f5e5810f290 con 0x7f5e4006ec00 2024-04-27T04:18:56.646 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:18:56.647+0000 7f5e5c968700 1 -- 172.21.15.162:0/3166906400 >> [v2:172.21.15.162:6800/1396866108,v1:172.21.15.162:6801/1396866108] conn(0x7f5e4006ec00 msgr2=0x7f5e400710b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:18:56.646 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:18:56.647+0000 7f5e5c968700 1 --2- 172.21.15.162:0/3166906400 >> [v2:172.21.15.162:6800/1396866108,v1:172.21.15.162:6801/1396866108] conn(0x7f5e4006ec00 0x7f5e400710b0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f5e48004f90 tx=0x7f5e48009e30 comp rx=0 tx=0).stop 2024-04-27T04:18:56.647 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:18:56.647+0000 7f5e5c968700 1 -- 172.21.15.162:0/3166906400 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5e581037a0 msgr2=0x7f5e581140e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:18:56.647 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:18:56.647+0000 7f5e5c968700 1 --2- 172.21.15.162:0/3166906400 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5e581037a0 0x7f5e581140e0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f5e4c002a50 tx=0x7f5e4c004600 comp rx=0 tx=0).stop 2024-04-27T04:18:56.647 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:18:56.647+0000 7f5e5c968700 1 -- 172.21.15.162:0/3166906400 shutdown_connections 2024-04-27T04:18:56.647 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:18:56.647+0000 7f5e5c968700 1 --2- 172.21.15.162:0/3166906400 >> [v2:172.21.15.162:6800/1396866108,v1:172.21.15.162:6801/1396866108] conn(0x7f5e4006ec00 0x7f5e400710b0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:18:56.647 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:18:56.647+0000 7f5e5c968700 1 --2- 172.21.15.162:0/3166906400 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5e58102e00 0x7f5e58113ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:18:56.647 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:18:56.647+0000 7f5e5c968700 1 --2- 172.21.15.162:0/3166906400 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5e581037a0 0x7f5e581140e0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:18:56.647 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:18:56.647+0000 7f5e5c968700 1 -- 172.21.15.162:0/3166906400 >> 172.21.15.162:0/3166906400 conn(0x7f5e580f89b0 msgr2=0x7f5e581073a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:18:56.647 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:18:56.648+0000 7f5e5c968700 1 -- 172.21.15.162:0/3166906400 shutdown_connections 2024-04-27T04:18:56.647 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:18:56.648+0000 7f5e5c968700 1 -- 172.21.15.162:0/3166906400 wait complete. 2024-04-27T04:18:56.657 INFO:teuthology.orchestra.run.smithi162.stdout:true 2024-04-27T04:18:56.753 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:18:56.752+0000 7fb8b6105700 1 -- 172.21.15.162:0/1264157609 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb8b00fc190 msgr2=0x7fb8b0101a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:18:56.753 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:18:56.752+0000 7fb8b6105700 1 --2- 172.21.15.162:0/1264157609 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb8b00fc190 0x7fb8b0101a70 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fb8a4004940 tx=0x7fb8a4009e30 comp rx=0 tx=0).stop 2024-04-27T04:18:56.753 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:18:56.752+0000 7fb8b6105700 1 -- 172.21.15.162:0/1264157609 shutdown_connections 2024-04-27T04:18:56.753 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:18:56.752+0000 7fb8b6105700 1 --2- 172.21.15.162:0/1264157609 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb8b00fc190 0x7fb8b0101a70 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:18:56.753 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:18:56.752+0000 7fb8b6105700 1 --2- 172.21.15.162:0/1264157609 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb8b00fb7f0 0x7fb8b00fbbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:18:56.753 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:18:56.752+0000 7fb8b6105700 1 -- 172.21.15.162:0/1264157609 >> 172.21.15.162:0/1264157609 conn(0x7fb8b0071460 msgr2=0x7fb8b0071860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:18:56.754 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:18:56.752+0000 7fb8b6105700 1 -- 172.21.15.162:0/1264157609 shutdown_connections 2024-04-27T04:18:56.754 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:18:56.752+0000 7fb8b6105700 1 -- 172.21.15.162:0/1264157609 wait complete. 2024-04-27T04:18:56.754 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:18:56.753+0000 7fb8b6105700 1 Processor -- start 2024-04-27T04:18:56.754 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:18:56.753+0000 7fb8b6105700 1 -- start start 2024-04-27T04:18:56.754 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:18:56.753+0000 7fb8b6105700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb8b00fb7f0 0x7fb8b01139a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:18:56.754 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:18:56.753+0000 7fb8b6105700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb8b00fc190 0x7fb8b0113ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:18:56.754 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:18:56.753+0000 7fb8b6105700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8b0114530 con 0x7fb8b00fc190 2024-04-27T04:18:56.754 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:18:56.753+0000 7fb8b6105700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8b010db50 con 0x7fb8b00fb7f0 2024-04-27T04:18:56.754 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:18:56.753+0000 7fb8aeffd700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb8b00fc190 0x7fb8b0113ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:18:56.754 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:18:56.754+0000 7fb8aeffd700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb8b00fc190 0x7fb8b0113ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:59388/0 (socket says 172.21.15.162:59388) 2024-04-27T04:18:56.754 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:18:56.754+0000 7fb8aeffd700 1 -- 172.21.15.162:0/2548916783 learned_addr learned my addr 172.21.15.162:0/2548916783 (peer_addr_for_me v2:172.21.15.162:0/0) 2024-04-27T04:18:56.754 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:18:56.754+0000 7fb8af7fe700 1 --2- 172.21.15.162:0/2548916783 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb8b00fb7f0 0x7fb8b01139a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:18:56.754 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:18:56.754+0000 7fb8aeffd700 1 -- 172.21.15.162:0/2548916783 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb8b00fb7f0 msgr2=0x7fb8b01139a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:18:56.755 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:18:56.754+0000 7fb8aeffd700 1 --2- 172.21.15.162:0/2548916783 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb8b00fb7f0 0x7fb8b01139a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:18:56.755 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:18:56.754+0000 7fb8aeffd700 1 -- 172.21.15.162:0/2548916783 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb8a4005040 con 0x7fb8b00fc190 2024-04-27T04:18:56.755 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:18:56.754+0000 7fb8aeffd700 1 --2- 172.21.15.162:0/2548916783 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb8b00fc190 0x7fb8b0113ee0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fb8a400b3b0 tx=0x7fb8a4009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:18:56.755 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:18:56.754+0000 7fb897fff700 1 -- 172.21.15.162:0/2548916783 <== mon.0 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb8a400e040 con 0x7fb8b00fc190 2024-04-27T04:18:56.755 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:18:56.754+0000 7fb8b6105700 1 -- 172.21.15.162:0/2548916783 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb8b010ddd0 con 0x7fb8b00fc190 2024-04-27T04:18:56.755 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:18:56.754+0000 7fb897fff700 1 -- 172.21.15.162:0/2548916783 <== mon.0 v2:172.21.15.162:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fb8a4013420 con 0x7fb8b00fc190 2024-04-27T04:18:56.755 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:18:56.754+0000 7fb897fff700 1 -- 172.21.15.162:0/2548916783 <== mon.0 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb8a401c630 con 0x7fb8b00fc190 2024-04-27T04:18:56.755 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:18:56.754+0000 7fb8b6105700 1 -- 172.21.15.162:0/2548916783 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb8b010e310 con 0x7fb8b00fc190 2024-04-27T04:18:56.756 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:18:56.755+0000 7fb8b6105700 1 -- 172.21.15.162:0/2548916783 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb8b0065f30 con 0x7fb8b00fc190 2024-04-27T04:18:56.756 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:18:56.756+0000 7fb897fff700 1 -- 172.21.15.162:0/2548916783 <== mon.0 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 39) v1 ==== 92747+0+0 (secure 0 0 0) 0x7fb8a4013590 con 0x7fb8b00fc190 2024-04-27T04:18:56.756 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:18:56.756+0000 7fb897fff700 1 --2- 172.21.15.162:0/2548916783 >> [v2:172.21.15.162:6800/1396866108,v1:172.21.15.162:6801/1396866108] conn(0x7fb89806ec00 0x7fb8980710b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:18:56.756 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:18:56.756+0000 7fb897fff700 1 -- 172.21.15.162:0/2548916783 <== mon.0 v2:172.21.15.162:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 6410+0+0 (secure 0 0 0) 0x7fb8a408f980 con 0x7fb8b00fc190 2024-04-27T04:18:56.756 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:18:56.756+0000 7fb8af7fe700 1 --2- 172.21.15.162:0/2548916783 >> [v2:172.21.15.162:6800/1396866108,v1:172.21.15.162:6801/1396866108] conn(0x7fb89806ec00 0x7fb8980710b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:18:56.756 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:18:56.756+0000 7fb8af7fe700 1 --2- 172.21.15.162:0/2548916783 >> [v2:172.21.15.162:6800/1396866108,v1:172.21.15.162:6801/1396866108] conn(0x7fb89806ec00 0x7fb8980710b0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fb8a0004f40 tx=0x7fb8a000b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:18:56.759 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:18:56.761+0000 7fb897fff700 1 -- 172.21.15.162:0/2548916783 <== mon.0 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7fb8a40588b0 con 0x7fb8b00fc190 2024-04-27T04:18:56.980 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:18:56.981+0000 7fb8b6105700 1 -- 172.21.15.162:0/2548916783 --> [v2:172.21.15.162:6800/1396866108,v1:172.21.15.162:6801/1396866108] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fb8b0109f20 con 0x7fb89806ec00 2024-04-27T04:18:56.980 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:18:56 smithi162 ceph-mon[110641]: pgmap v174: 65 pgs: 65 active+clean; 208 MiB data, 851 MiB used, 536 GiB / 536 GiB avail; 0 B/s rd, 0 op/s 2024-04-27T04:18:56.981 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:18:56.982+0000 7fb897fff700 1 -- 172.21.15.162:0/2548916783 <== mgr.34104 v2:172.21.15.162:6800/1396866108 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7fb8b0109f20 con 0x7fb89806ec00 2024-04-27T04:18:56.984 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:18:56.985+0000 7fb8b6105700 1 -- 172.21.15.162:0/2548916783 >> [v2:172.21.15.162:6800/1396866108,v1:172.21.15.162:6801/1396866108] conn(0x7fb89806ec00 msgr2=0x7fb8980710b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:18:56.984 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:18:56.985+0000 7fb8b6105700 1 --2- 172.21.15.162:0/2548916783 >> [v2:172.21.15.162:6800/1396866108,v1:172.21.15.162:6801/1396866108] conn(0x7fb89806ec00 0x7fb8980710b0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fb8a0004f40 tx=0x7fb8a000b040 comp rx=0 tx=0).stop 2024-04-27T04:18:56.984 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:18:56.985+0000 7fb8b6105700 1 -- 172.21.15.162:0/2548916783 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb8b00fc190 msgr2=0x7fb8b0113ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:18:56.984 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:18:56.985+0000 7fb8b6105700 1 --2- 172.21.15.162:0/2548916783 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb8b00fc190 0x7fb8b0113ee0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fb8a400b3b0 tx=0x7fb8a4009e30 comp rx=0 tx=0).stop 2024-04-27T04:18:56.984 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:18:56.985+0000 7fb8b6105700 1 -- 172.21.15.162:0/2548916783 shutdown_connections 2024-04-27T04:18:56.984 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:18:56.985+0000 7fb8b6105700 1 --2- 172.21.15.162:0/2548916783 >> [v2:172.21.15.162:6800/1396866108,v1:172.21.15.162:6801/1396866108] conn(0x7fb89806ec00 0x7fb8980710b0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:18:56.984 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:18:56.985+0000 7fb8b6105700 1 --2- 172.21.15.162:0/2548916783 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb8b00fb7f0 0x7fb8b01139a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:18:56.984 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:18:56.985+0000 7fb8b6105700 1 --2- 172.21.15.162:0/2548916783 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb8b00fc190 0x7fb8b0113ee0 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:18:56.984 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:18:56.985+0000 7fb8b6105700 1 -- 172.21.15.162:0/2548916783 >> 172.21.15.162:0/2548916783 conn(0x7fb8b0071460 msgr2=0x7fb8b0100a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:18:56.984 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:18:56.985+0000 7fb8b6105700 1 -- 172.21.15.162:0/2548916783 shutdown_connections 2024-04-27T04:18:56.985 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:18:56.985+0000 7fb8b6105700 1 -- 172.21.15.162:0/2548916783 wait complete. 2024-04-27T04:18:57.093 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:18:57.093+0000 7fe73bf36700 1 -- 172.21.15.162:0/1618345611 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe7340fc570 msgr2=0x7fe734100b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:18:57.093 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:18:57.093+0000 7fe73bf36700 1 --2- 172.21.15.162:0/1618345611 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe7340fc570 0x7fe734100b90 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fe724004940 tx=0x7fe724009e30 comp rx=0 tx=0).stop 2024-04-27T04:18:57.094 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:18:57.093+0000 7fe73bf36700 1 -- 172.21.15.162:0/1618345611 shutdown_connections 2024-04-27T04:18:57.094 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:18:57.093+0000 7fe73bf36700 1 --2- 172.21.15.162:0/1618345611 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe7340fc570 0x7fe734100b90 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:18:57.094 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:18:57.093+0000 7fe73bf36700 1 --2- 172.21.15.162:0/1618345611 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe7340fbbd0 0x7fe7340fbfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:18:57.094 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:18:57.093+0000 7fe73bf36700 1 -- 172.21.15.162:0/1618345611 >> 172.21.15.162:0/1618345611 conn(0x7fe7340f7850 msgr2=0x7fe7340f9c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:18:57.094 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:18:57.093+0000 7fe73bf36700 1 -- 172.21.15.162:0/1618345611 shutdown_connections 2024-04-27T04:18:57.094 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:18:57.093+0000 7fe73bf36700 1 -- 172.21.15.162:0/1618345611 wait complete. 2024-04-27T04:18:57.094 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:18:57.094+0000 7fe73bf36700 1 Processor -- start 2024-04-27T04:18:57.094 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:18:57.094+0000 7fe73bf36700 1 -- start start 2024-04-27T04:18:57.094 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:18:57.094+0000 7fe73bf36700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe7340fbbd0 0x7fe73411a1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:18:57.094 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:18:57.094+0000 7fe73bf36700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe7340fc570 0x7fe73411a730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:18:57.094 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:18:57.094+0000 7fe73bf36700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe73411ad80 con 0x7fe7340fc570 2024-04-27T04:18:57.095 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:18:57.094+0000 7fe73bf36700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe734114190 con 0x7fe7340fbbd0 2024-04-27T04:18:57.095 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:18:57.094+0000 7fe7394d1700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe7340fc570 0x7fe73411a730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:18:57.095 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:18:57.094+0000 7fe7394d1700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe7340fc570 0x7fe73411a730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:59412/0 (socket says 172.21.15.162:59412) 2024-04-27T04:18:57.095 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:18:57.094+0000 7fe7394d1700 1 -- 172.21.15.162:0/3340962595 learned_addr learned my addr 172.21.15.162:0/3340962595 (peer_addr_for_me v2:172.21.15.162:0/0) 2024-04-27T04:18:57.095 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:18:57.094+0000 7fe739cd2700 1 --2- 172.21.15.162:0/3340962595 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe7340fbbd0 0x7fe73411a1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:18:57.095 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:18:57.095+0000 7fe7394d1700 1 -- 172.21.15.162:0/3340962595 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe7340fbbd0 msgr2=0x7fe73411a1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:18:57.095 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:18:57.095+0000 7fe7394d1700 1 --2- 172.21.15.162:0/3340962595 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe7340fbbd0 0x7fe73411a1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:18:57.095 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:18:57.095+0000 7fe7394d1700 1 -- 172.21.15.162:0/3340962595 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe724005040 con 0x7fe7340fc570 2024-04-27T04:18:57.095 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:18:57.095+0000 7fe7394d1700 1 --2- 172.21.15.162:0/3340962595 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe7340fc570 0x7fe73411a730 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fe72400b3b0 tx=0x7fe724009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:18:57.097 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:18:57.095+0000 7fe72a7fc700 1 -- 172.21.15.162:0/3340962595 <== mon.0 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe72400e040 con 0x7fe7340fc570 2024-04-27T04:18:57.097 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:18:57.095+0000 7fe73bf36700 1 -- 172.21.15.162:0/3340962595 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe734114410 con 0x7fe7340fc570 2024-04-27T04:18:57.097 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:18:57.095+0000 7fe72a7fc700 1 -- 172.21.15.162:0/3340962595 <== mon.0 v2:172.21.15.162:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fe724013420 con 0x7fe7340fc570 2024-04-27T04:18:57.097 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:18:57.095+0000 7fe72a7fc700 1 -- 172.21.15.162:0/3340962595 <== mon.0 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe72401c630 con 0x7fe7340fc570 2024-04-27T04:18:57.097 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:18:57.095+0000 7fe73bf36700 1 -- 172.21.15.162:0/3340962595 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe734114950 con 0x7fe7340fc570 2024-04-27T04:18:57.097 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:18:57.096+0000 7fe73bf36700 1 -- 172.21.15.162:0/3340962595 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe734066750 con 0x7fe7340fc570 2024-04-27T04:18:57.097 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:18:57.097+0000 7fe72a7fc700 1 -- 172.21.15.162:0/3340962595 <== mon.0 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 39) v1 ==== 92747+0+0 (secure 0 0 0) 0x7fe724013590 con 0x7fe7340fc570 2024-04-27T04:18:57.097 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:18:57.097+0000 7fe72a7fc700 1 --2- 172.21.15.162:0/3340962595 >> [v2:172.21.15.162:6800/1396866108,v1:172.21.15.162:6801/1396866108] conn(0x7fe72006ec00 0x7fe7200710b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:18:57.098 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:18:57.097+0000 7fe72a7fc700 1 -- 172.21.15.162:0/3340962595 <== mon.0 v2:172.21.15.162:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 6410+0+0 (secure 0 0 0) 0x7fe72408fa10 con 0x7fe7340fc570 2024-04-27T04:18:57.098 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:18:57.097+0000 7fe739cd2700 1 --2- 172.21.15.162:0/3340962595 >> [v2:172.21.15.162:6800/1396866108,v1:172.21.15.162:6801/1396866108] conn(0x7fe72006ec00 0x7fe7200710b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:18:57.098 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:18:57.098+0000 7fe739cd2700 1 --2- 172.21.15.162:0/3340962595 >> [v2:172.21.15.162:6800/1396866108,v1:172.21.15.162:6801/1396866108] conn(0x7fe72006ec00 0x7fe7200710b0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fe730004f40 tx=0x7fe73000b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:18:57.101 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:18:57.102+0000 7fe72a7fc700 1 -- 172.21.15.162:0/3340962595 <== mon.0 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7fe724058890 con 0x7fe7340fc570 2024-04-27T04:18:57.171 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:18:56 smithi181 ceph-mon[81134]: pgmap v174: 65 pgs: 65 active+clean; 208 MiB data, 851 MiB used, 536 GiB / 536 GiB avail; 0 B/s rd, 0 op/s 2024-04-27T04:18:57.318 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:18:57.319+0000 7fe73bf36700 1 -- 172.21.15.162:0/3340962595 --> [v2:172.21.15.162:6800/1396866108,v1:172.21.15.162:6801/1396866108] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7fe7341104e0 con 0x7fe72006ec00 2024-04-27T04:18:57.329 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:18:57.330+0000 7fe72a7fc700 1 -- 172.21.15.162:0/3340962595 <== mgr.34104 v2:172.21.15.162:6800/1396866108 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3498 (secure 0 0 0) 0x7fe7341104e0 con 0x7fe72006ec00 2024-04-27T04:18:57.329 INFO:teuthology.orchestra.run.smithi162.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-27T04:18:57.329 INFO:teuthology.orchestra.run.smithi162.stdout:alertmanager.smithi162 smithi162 *:9093,9094 running (6m) 119s ago 17m 27.2M - 0.25.0 c8568f914cd2 40e334659017 2024-04-27T04:18:57.329 INFO:teuthology.orchestra.run.smithi162.stdout:crash.smithi162 smithi162 running (3m) 119s ago 17m 6660k - 19.0.0-3253-g4e213a58 67e414ff4bc4 1aa8f4d5a28b 2024-04-27T04:18:57.330 INFO:teuthology.orchestra.run.smithi162.stdout:crash.smithi181 smithi181 running (3m) 18s ago 16m 6656k - 19.0.0-3253-g4e213a58 67e414ff4bc4 832d9cbc5c30 2024-04-27T04:18:57.330 INFO:teuthology.orchestra.run.smithi162.stdout:grafana.smithi162 smithi162 *:3000 running (5m) 119s ago 16m 80.5M - 9.4.12 2bacad6d85d8 3d78c478f03f 2024-04-27T04:18:57.330 INFO:teuthology.orchestra.run.smithi162.stdout:mds.cephfs.smithi162.gzlyjj smithi162 starting - - - - 2024-04-27T04:18:57.330 INFO:teuthology.orchestra.run.smithi162.stdout:mds.cephfs.smithi162.xcnapk smithi162 running (11m) 119s ago 11m 100M - 17.2.7-907-g2be3efa9 1c5834633715 38f9030df029 2024-04-27T04:18:57.330 INFO:teuthology.orchestra.run.smithi162.stdout:mds.cephfs.smithi181.bkkcdm smithi181 running (11m) 18s ago 11m 16.1M - 17.2.7-907-g2be3efa9 1c5834633715 af55afca1263 2024-04-27T04:18:57.330 INFO:teuthology.orchestra.run.smithi162.stdout:mds.cephfs.smithi181.dzomgh smithi181 running (11m) 18s ago 11m 143M - 17.2.7-907-g2be3efa9 1c5834633715 6733296597f3 2024-04-27T04:18:57.330 INFO:teuthology.orchestra.run.smithi162.stdout:mgr.smithi162.oennfn smithi162 *:8443,9283,8765 running (7m) 119s ago 18m 505M - 19.0.0-3253-g4e213a58 67e414ff4bc4 58a748d3637b 2024-04-27T04:18:57.330 INFO:teuthology.orchestra.run.smithi162.stdout:mgr.smithi181.vtwuhz smithi181 *:8443,9283,8765 running (6m) 18s ago 16m 452M - 19.0.0-3253-g4e213a58 67e414ff4bc4 813584297b1d 2024-04-27T04:18:57.330 INFO:teuthology.orchestra.run.smithi162.stdout:mon.smithi162 smithi162 running (4m) 119s ago 18m 50.4M 2048M 19.0.0-3253-g4e213a58 67e414ff4bc4 375adaf4468f 2024-04-27T04:18:57.330 INFO:teuthology.orchestra.run.smithi162.stdout:mon.smithi181 smithi181 running (4m) 18s ago 16m 36.1M 2048M 19.0.0-3253-g4e213a58 67e414ff4bc4 77b71f6d522f 2024-04-27T04:18:57.330 INFO:teuthology.orchestra.run.smithi162.stdout:node-exporter.smithi162 smithi162 *:9100 running (6m) 119s ago 17m 15.4M - 1.5.0 0da6a335fe13 5c4c5ea9c1d7 2024-04-27T04:18:57.331 INFO:teuthology.orchestra.run.smithi162.stdout:node-exporter.smithi181 smithi181 *:9100 running (6m) 18s ago 16m 15.5M - 1.5.0 0da6a335fe13 9067f14c0519 2024-04-27T04:18:57.331 INFO:teuthology.orchestra.run.smithi162.stdout:osd.0 smithi162 running (3m) 119s ago 15m 87.7M 1557M 19.0.0-3253-g4e213a58 67e414ff4bc4 2744aff3cd88 2024-04-27T04:18:57.331 INFO:teuthology.orchestra.run.smithi162.stdout:osd.1 smithi162 running (2m) 119s ago 15m 93.8M 1557M 19.0.0-3253-g4e213a58 67e414ff4bc4 668e16faad62 2024-04-27T04:18:57.331 INFO:teuthology.orchestra.run.smithi162.stdout:osd.2 smithi162 running (2m) 119s ago 14m 15.1M 1557M 19.0.0-3253-g4e213a58 67e414ff4bc4 d24878d461e7 2024-04-27T04:18:57.331 INFO:teuthology.orchestra.run.smithi162.stdout:osd.3 smithi181 running (89s) 18s ago 14m 112M 2581M 19.0.0-3253-g4e213a58 67e414ff4bc4 ab473cfd54d2 2024-04-27T04:18:57.331 INFO:teuthology.orchestra.run.smithi162.stdout:osd.4 smithi181 running (56s) 18s ago 13m 98.6M 2581M 19.0.0-3253-g4e213a58 67e414ff4bc4 de3e10f2d26d 2024-04-27T04:18:57.331 INFO:teuthology.orchestra.run.smithi162.stdout:osd.5 smithi181 running (22s) 18s ago 13m 11.8M 2581M 19.0.0-3253-g4e213a58 67e414ff4bc4 7dd4f287ef2e 2024-04-27T04:18:57.331 INFO:teuthology.orchestra.run.smithi162.stdout:prometheus.smithi162 smithi162 *:9095 running (6m) 119s ago 16m 67.6M - 2.43.0 a07b618ecd1d f30657a43324 2024-04-27T04:18:57.332 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:18:57.332+0000 7fe73bf36700 1 -- 172.21.15.162:0/3340962595 >> [v2:172.21.15.162:6800/1396866108,v1:172.21.15.162:6801/1396866108] conn(0x7fe72006ec00 msgr2=0x7fe7200710b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:18:57.332 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:18:57.332+0000 7fe73bf36700 1 --2- 172.21.15.162:0/3340962595 >> [v2:172.21.15.162:6800/1396866108,v1:172.21.15.162:6801/1396866108] conn(0x7fe72006ec00 0x7fe7200710b0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fe730004f40 tx=0x7fe73000b040 comp rx=0 tx=0).stop 2024-04-27T04:18:57.332 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:18:57.332+0000 7fe73bf36700 1 -- 172.21.15.162:0/3340962595 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe7340fc570 msgr2=0x7fe73411a730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:18:57.332 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:18:57.332+0000 7fe73bf36700 1 --2- 172.21.15.162:0/3340962595 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe7340fc570 0x7fe73411a730 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fe72400b3b0 tx=0x7fe724009e30 comp rx=0 tx=0).stop 2024-04-27T04:18:57.332 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:18:57.332+0000 7fe73bf36700 1 -- 172.21.15.162:0/3340962595 shutdown_connections 2024-04-27T04:18:57.332 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:18:57.332+0000 7fe73bf36700 1 --2- 172.21.15.162:0/3340962595 >> [v2:172.21.15.162:6800/1396866108,v1:172.21.15.162:6801/1396866108] conn(0x7fe72006ec00 0x7fe7200710b0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:18:57.333 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:18:57.332+0000 7fe73bf36700 1 --2- 172.21.15.162:0/3340962595 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe7340fbbd0 0x7fe73411a1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:18:57.333 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:18:57.332+0000 7fe73bf36700 1 --2- 172.21.15.162:0/3340962595 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe7340fc570 0x7fe73411a730 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:18:57.333 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:18:57.332+0000 7fe73bf36700 1 -- 172.21.15.162:0/3340962595 >> 172.21.15.162:0/3340962595 conn(0x7fe7340f7850 msgr2=0x7fe7340f8e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:18:57.333 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:18:57.333+0000 7fe73bf36700 1 -- 172.21.15.162:0/3340962595 shutdown_connections 2024-04-27T04:18:57.333 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:18:57.333+0000 7fe73bf36700 1 -- 172.21.15.162:0/3340962595 wait complete. 2024-04-27T04:18:57.439 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:18:57.439+0000 7f3e878c5700 1 -- 172.21.15.162:0/1804885957 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3e80102690 msgr2=0x7f3e800fbd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:18:57.439 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:18:57.439+0000 7f3e878c5700 1 --2- 172.21.15.162:0/1804885957 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3e80102690 0x7f3e800fbd00 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f3e70004940 tx=0x7f3e70009e30 comp rx=0 tx=0).stop 2024-04-27T04:18:57.439 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:18:57.439+0000 7f3e878c5700 1 -- 172.21.15.162:0/1804885957 shutdown_connections 2024-04-27T04:18:57.439 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:18:57.439+0000 7f3e878c5700 1 --2- 172.21.15.162:0/1804885957 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3e80102690 0x7f3e800fbd00 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:18:57.439 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:18:57.439+0000 7f3e878c5700 1 --2- 172.21.15.162:0/1804885957 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3e80101cf0 0x7f3e801020c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:18:57.439 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:18:57.439+0000 7f3e878c5700 1 -- 172.21.15.162:0/1804885957 >> 172.21.15.162:0/1804885957 conn(0x7f3e800f7860 msgr2=0x7f3e800f9c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:18:57.440 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:18:57.439+0000 7f3e878c5700 1 -- 172.21.15.162:0/1804885957 shutdown_connections 2024-04-27T04:18:57.440 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:18:57.439+0000 7f3e878c5700 1 -- 172.21.15.162:0/1804885957 wait complete. 2024-04-27T04:18:57.440 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:18:57.440+0000 7f3e878c5700 1 Processor -- start 2024-04-27T04:18:57.440 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:18:57.440+0000 7f3e878c5700 1 -- start start 2024-04-27T04:18:57.440 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:18:57.440+0000 7f3e878c5700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3e80101cf0 0x7f3e80113a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:18:57.440 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:18:57.440+0000 7f3e878c5700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3e80102690 0x7f3e80113fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:18:57.440 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:18:57.440+0000 7f3e878c5700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e80114620 con 0x7f3e80102690 2024-04-27T04:18:57.440 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:18:57.440+0000 7f3e878c5700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e8010da10 con 0x7f3e80101cf0 2024-04-27T04:18:57.440 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:18:57.440+0000 7f3e84e60700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3e80102690 0x7f3e80113fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:18:57.440 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:18:57.440+0000 7f3e84e60700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3e80102690 0x7f3e80113fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:59416/0 (socket says 172.21.15.162:59416) 2024-04-27T04:18:57.440 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:18:57.440+0000 7f3e84e60700 1 -- 172.21.15.162:0/2762199298 learned_addr learned my addr 172.21.15.162:0/2762199298 (peer_addr_for_me v2:172.21.15.162:0/0) 2024-04-27T04:18:57.440 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:18:57.440+0000 7f3e85661700 1 --2- 172.21.15.162:0/2762199298 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3e80101cf0 0x7f3e80113a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:18:57.441 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:18:57.441+0000 7f3e84e60700 1 -- 172.21.15.162:0/2762199298 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3e80101cf0 msgr2=0x7f3e80113a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:18:57.441 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:18:57.441+0000 7f3e84e60700 1 --2- 172.21.15.162:0/2762199298 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3e80101cf0 0x7f3e80113a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:18:57.442 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:18:57.441+0000 7f3e84e60700 1 -- 172.21.15.162:0/2762199298 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e70005040 con 0x7f3e80102690 2024-04-27T04:18:57.442 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:18:57.441+0000 7f3e84e60700 1 --2- 172.21.15.162:0/2762199298 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3e80102690 0x7f3e80113fd0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f3e70009180 tx=0x7f3e70009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:18:57.442 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:18:57.441+0000 7f3e75ffb700 1 -- 172.21.15.162:0/2762199298 <== mon.0 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3e7000e040 con 0x7f3e80102690 2024-04-27T04:18:57.442 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:18:57.441+0000 7f3e75ffb700 1 -- 172.21.15.162:0/2762199298 <== mon.0 v2:172.21.15.162:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f3e70013420 con 0x7f3e80102690 2024-04-27T04:18:57.442 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:18:57.441+0000 7f3e878c5700 1 -- 172.21.15.162:0/2762199298 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3e8010dc90 con 0x7f3e80102690 2024-04-27T04:18:57.442 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:18:57.441+0000 7f3e75ffb700 1 -- 172.21.15.162:0/2762199298 <== mon.0 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3e7001c610 con 0x7f3e80102690 2024-04-27T04:18:57.442 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:18:57.441+0000 7f3e878c5700 1 -- 172.21.15.162:0/2762199298 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3e8010e250 con 0x7f3e80102690 2024-04-27T04:18:57.442 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:18:57.442+0000 7f3e878c5700 1 -- 172.21.15.162:0/2762199298 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3e800fd440 con 0x7f3e80102690 2024-04-27T04:18:57.443 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:18:57.443+0000 7f3e75ffb700 1 -- 172.21.15.162:0/2762199298 <== mon.0 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 39) v1 ==== 92747+0+0 (secure 0 0 0) 0x7f3e70013a00 con 0x7f3e80102690 2024-04-27T04:18:57.443 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:18:57.443+0000 7f3e75ffb700 1 --2- 172.21.15.162:0/2762199298 >> [v2:172.21.15.162:6800/1396866108,v1:172.21.15.162:6801/1396866108] conn(0x7f3e6c06ec00 0x7f3e6c0710b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:18:57.443 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:18:57.444+0000 7f3e75ffb700 1 -- 172.21.15.162:0/2762199298 <== mon.0 v2:172.21.15.162:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 6410+0+0 (secure 0 0 0) 0x7f3e7008f830 con 0x7f3e80102690 2024-04-27T04:18:57.443 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:18:57.444+0000 7f3e85661700 1 --2- 172.21.15.162:0/2762199298 >> [v2:172.21.15.162:6800/1396866108,v1:172.21.15.162:6801/1396866108] conn(0x7f3e6c06ec00 0x7f3e6c0710b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:18:57.443 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:18:57.444+0000 7f3e85661700 1 --2- 172.21.15.162:0/2762199298 >> [v2:172.21.15.162:6800/1396866108,v1:172.21.15.162:6801/1396866108] conn(0x7f3e6c06ec00 0x7f3e6c0710b0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f3e7c009fd0 tx=0x7f3e7c00b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:18:57.451 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:18:57.452+0000 7f3e75ffb700 1 -- 172.21.15.162:0/2762199298 <== mon.0 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7f3e7005c280 con 0x7f3e80102690 2024-04-27T04:18:57.747 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:18:57.748+0000 7f3e878c5700 1 -- 172.21.15.162:0/2762199298 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f3e8005f5c0 con 0x7f3e80102690 2024-04-27T04:18:57.747 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:18:57.748+0000 7f3e75ffb700 1 -- 172.21.15.162:0/2762199298 <== mon.0 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+837 (secure 0 0 0) 0x7f3e7005be10 con 0x7f3e80102690 2024-04-27T04:18:57.747 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-04-27T04:18:57.747 INFO:teuthology.orchestra.run.smithi162.stdout: "mon": { 2024-04-27T04:18:57.747 INFO:teuthology.orchestra.run.smithi162.stdout: "ceph version 19.0.0-3253-g4e213a58 (4e213a58e787437ff508e6c890ccd5fd32dc96b5) squid (dev)": 2 2024-04-27T04:18:57.747 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-04-27T04:18:57.748 INFO:teuthology.orchestra.run.smithi162.stdout: "mgr": { 2024-04-27T04:18:57.748 INFO:teuthology.orchestra.run.smithi162.stdout: "ceph version 19.0.0-3253-g4e213a58 (4e213a58e787437ff508e6c890ccd5fd32dc96b5) squid (dev)": 2 2024-04-27T04:18:57.748 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-04-27T04:18:57.748 INFO:teuthology.orchestra.run.smithi162.stdout: "osd": { 2024-04-27T04:18:57.748 INFO:teuthology.orchestra.run.smithi162.stdout: "ceph version 19.0.0-3253-g4e213a58 (4e213a58e787437ff508e6c890ccd5fd32dc96b5) squid (dev)": 6 2024-04-27T04:18:57.748 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-04-27T04:18:57.748 INFO:teuthology.orchestra.run.smithi162.stdout: "mds": { 2024-04-27T04:18:57.748 INFO:teuthology.orchestra.run.smithi162.stdout: "ceph version 17.2.7-907-g2be3efa9 (2be3efa97a7972f8e925bd899b474329979b74d6) quincy (stable)": 3, 2024-04-27T04:18:57.748 INFO:teuthology.orchestra.run.smithi162.stdout: "ceph version 19.0.0-3253-g4e213a58 (4e213a58e787437ff508e6c890ccd5fd32dc96b5) squid (dev)": 1 2024-04-27T04:18:57.748 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-04-27T04:18:57.748 INFO:teuthology.orchestra.run.smithi162.stdout: "overall": { 2024-04-27T04:18:57.748 INFO:teuthology.orchestra.run.smithi162.stdout: "ceph version 17.2.7-907-g2be3efa9 (2be3efa97a7972f8e925bd899b474329979b74d6) quincy (stable)": 3, 2024-04-27T04:18:57.748 INFO:teuthology.orchestra.run.smithi162.stdout: "ceph version 19.0.0-3253-g4e213a58 (4e213a58e787437ff508e6c890ccd5fd32dc96b5) squid (dev)": 11 2024-04-27T04:18:57.748 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-04-27T04:18:57.748 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-04-27T04:18:57.749 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:18:57.750+0000 7f3e878c5700 1 -- 172.21.15.162:0/2762199298 >> [v2:172.21.15.162:6800/1396866108,v1:172.21.15.162:6801/1396866108] conn(0x7f3e6c06ec00 msgr2=0x7f3e6c0710b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:18:57.749 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:18:57.750+0000 7f3e878c5700 1 --2- 172.21.15.162:0/2762199298 >> [v2:172.21.15.162:6800/1396866108,v1:172.21.15.162:6801/1396866108] conn(0x7f3e6c06ec00 0x7f3e6c0710b0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f3e7c009fd0 tx=0x7f3e7c00b040 comp rx=0 tx=0).stop 2024-04-27T04:18:57.749 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:18:57.750+0000 7f3e878c5700 1 -- 172.21.15.162:0/2762199298 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3e80102690 msgr2=0x7f3e80113fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:18:57.749 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:18:57.750+0000 7f3e878c5700 1 --2- 172.21.15.162:0/2762199298 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3e80102690 0x7f3e80113fd0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f3e70009180 tx=0x7f3e70009e30 comp rx=0 tx=0).stop 2024-04-27T04:18:57.749 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:18:57.750+0000 7f3e878c5700 1 -- 172.21.15.162:0/2762199298 shutdown_connections 2024-04-27T04:18:57.750 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:18:57.750+0000 7f3e878c5700 1 --2- 172.21.15.162:0/2762199298 >> [v2:172.21.15.162:6800/1396866108,v1:172.21.15.162:6801/1396866108] conn(0x7f3e6c06ec00 0x7f3e6c0710b0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:18:57.750 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:18:57.750+0000 7f3e878c5700 1 --2- 172.21.15.162:0/2762199298 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3e80101cf0 0x7f3e80113a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:18:57.750 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:18:57.750+0000 7f3e878c5700 1 --2- 172.21.15.162:0/2762199298 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3e80102690 0x7f3e80113fd0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:18:57.750 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:18:57.750+0000 7f3e878c5700 1 -- 172.21.15.162:0/2762199298 >> 172.21.15.162:0/2762199298 conn(0x7f3e800f7860 msgr2=0x7f3e800f8f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:18:57.750 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:18:57.751+0000 7f3e878c5700 1 -- 172.21.15.162:0/2762199298 shutdown_connections 2024-04-27T04:18:57.750 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:18:57.751+0000 7f3e878c5700 1 -- 172.21.15.162:0/2762199298 wait complete. 2024-04-27T04:18:57.866 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:18:57.867+0000 7f67a1045700 1 -- 172.21.15.162:0/74711608 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f679c102670 msgr2=0x7f679c0fb600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:18:57.867 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:18:57.867+0000 7f67a1045700 1 --2- 172.21.15.162:0/74711608 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f679c102670 0x7f679c0fb600 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f6790004940 tx=0x7f6790009e30 comp rx=0 tx=0).stop 2024-04-27T04:18:57.867 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:18:57.868+0000 7f67a1045700 1 -- 172.21.15.162:0/74711608 shutdown_connections 2024-04-27T04:18:57.867 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:18:57.868+0000 7f67a1045700 1 --2- 172.21.15.162:0/74711608 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f679c102670 0x7f679c0fb600 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:18:57.867 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:18:57.868+0000 7f67a1045700 1 --2- 172.21.15.162:0/74711608 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f679c101cd0 0x7f679c1020a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:18:57.867 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:18:57.868+0000 7f67a1045700 1 -- 172.21.15.162:0/74711608 >> 172.21.15.162:0/74711608 conn(0x7f679c071460 msgr2=0x7f679c071860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:18:57.867 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:18:57.868+0000 7f67a1045700 1 -- 172.21.15.162:0/74711608 shutdown_connections 2024-04-27T04:18:57.867 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:18:57.868+0000 7f67a1045700 1 -- 172.21.15.162:0/74711608 wait complete. 2024-04-27T04:18:57.869 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:18:57.868+0000 7f67a1045700 1 Processor -- start 2024-04-27T04:18:57.869 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:18:57.868+0000 7f67a1045700 1 -- start start 2024-04-27T04:18:57.869 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:18:57.869+0000 7f67a1045700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f679c101cd0 0x7f679c113b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:18:57.869 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:18:57.869+0000 7f67a1045700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f679c102670 0x7f679c1140b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:18:57.869 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:18:57.869+0000 7f67a1045700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f679c114700 con 0x7f679c102670 2024-04-27T04:18:57.869 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:18:57.869+0000 7f67a1045700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f679c10daf0 con 0x7f679c101cd0 2024-04-27T04:18:57.869 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:18:57.869+0000 7f679a59c700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f679c102670 0x7f679c1140b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:18:57.869 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:18:57.869+0000 7f679a59c700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f679c102670 0x7f679c1140b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:59438/0 (socket says 172.21.15.162:59438) 2024-04-27T04:18:57.869 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:18:57.869+0000 7f679a59c700 1 -- 172.21.15.162:0/1155431242 learned_addr learned my addr 172.21.15.162:0/1155431242 (peer_addr_for_me v2:172.21.15.162:0/0) 2024-04-27T04:18:57.869 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:18:57.869+0000 7f679ad9d700 1 --2- 172.21.15.162:0/1155431242 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f679c101cd0 0x7f679c113b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:18:57.869 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:18:57.869+0000 7f679a59c700 1 -- 172.21.15.162:0/1155431242 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f679c101cd0 msgr2=0x7f679c113b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:18:57.870 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:18:57.869+0000 7f679a59c700 1 --2- 172.21.15.162:0/1155431242 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f679c101cd0 0x7f679c113b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:18:57.870 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:18:57.869+0000 7f679a59c700 1 -- 172.21.15.162:0/1155431242 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6790005040 con 0x7f679c102670 2024-04-27T04:18:57.870 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:18:57.869+0000 7f679a59c700 1 --2- 172.21.15.162:0/1155431242 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f679c102670 0x7f679c1140b0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f6790000c00 tx=0x7f6790009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:18:57.870 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:18:57.869+0000 7f67837fe700 1 -- 172.21.15.162:0/1155431242 <== mon.0 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f679000e040 con 0x7f679c102670 2024-04-27T04:18:57.870 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:18:57.869+0000 7f67837fe700 1 -- 172.21.15.162:0/1155431242 <== mon.0 v2:172.21.15.162:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f6790009400 con 0x7f679c102670 2024-04-27T04:18:57.870 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:18:57.869+0000 7f67837fe700 1 -- 172.21.15.162:0/1155431242 <== mon.0 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f679001c3f0 con 0x7f679c102670 2024-04-27T04:18:57.870 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:18:57.869+0000 7f67a1045700 1 -- 172.21.15.162:0/1155431242 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f679c10dd70 con 0x7f679c102670 2024-04-27T04:18:57.870 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:18:57.869+0000 7f67a1045700 1 -- 172.21.15.162:0/1155431242 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f679c10e330 con 0x7f679c102670 2024-04-27T04:18:57.870 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:18:57.870+0000 7f67a1045700 1 -- 172.21.15.162:0/1155431242 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f679c065200 con 0x7f679c102670 2024-04-27T04:18:57.871 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:18:57.871+0000 7f67837fe700 1 -- 172.21.15.162:0/1155431242 <== mon.0 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 39) v1 ==== 92747+0+0 (secure 0 0 0) 0x7f679000baf0 con 0x7f679c102670 2024-04-27T04:18:57.871 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:18:57.871+0000 7f67837fe700 1 --2- 172.21.15.162:0/1155431242 >> [v2:172.21.15.162:6800/1396866108,v1:172.21.15.162:6801/1396866108] conn(0x7f678406ec00 0x7f67840710b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:18:57.871 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:18:57.871+0000 7f67837fe700 1 -- 172.21.15.162:0/1155431242 <== mon.0 v2:172.21.15.162:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 6410+0+0 (secure 0 0 0) 0x7f679008fa20 con 0x7f679c102670 2024-04-27T04:18:57.871 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:18:57.871+0000 7f679ad9d700 1 --2- 172.21.15.162:0/1155431242 >> [v2:172.21.15.162:6800/1396866108,v1:172.21.15.162:6801/1396866108] conn(0x7f678406ec00 0x7f67840710b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:18:57.871 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:18:57.872+0000 7f679ad9d700 1 --2- 172.21.15.162:0/1155431242 >> [v2:172.21.15.162:6800/1396866108,v1:172.21.15.162:6801/1396866108] conn(0x7f678406ec00 0x7f67840710b0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f678c007fd0 tx=0x7f678c009040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:18:57.875 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:18:57.876+0000 7f67837fe700 1 -- 172.21.15.162:0/1155431242 <== mon.0 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7f679005c3d0 con 0x7f679c102670 2024-04-27T04:18:58.062 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:18:57 smithi162 ceph-mon[110641]: from='client.34384 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-27T04:18:58.140 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:18:58.141+0000 7f67a1045700 1 -- 172.21.15.162:0/1155431242 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f679c109ed0 con 0x7f679c102670 2024-04-27T04:18:58.140 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:18:58.141+0000 7f67837fe700 1 -- 172.21.15.162:0/1155431242 <== mon.0 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 19 v19) v1 ==== 76+0+1893 (secure 0 0 0) 0x7f6790058020 con 0x7f679c102670 2024-04-27T04:18:58.140 INFO:teuthology.orchestra.run.smithi162.stdout:e19 2024-04-27T04:18:58.141 INFO:teuthology.orchestra.run.smithi162.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-04-27T04:18:58.141 INFO:teuthology.orchestra.run.smithi162.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-27T04:18:58.141 INFO:teuthology.orchestra.run.smithi162.stdout:legacy client fscid: 1 2024-04-27T04:18:58.141 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-04-27T04:18:58.141 INFO:teuthology.orchestra.run.smithi162.stdout:Filesystem 'cephfs' (1) 2024-04-27T04:18:58.141 INFO:teuthology.orchestra.run.smithi162.stdout:fs_name cephfs 2024-04-27T04:18:58.141 INFO:teuthology.orchestra.run.smithi162.stdout:epoch 16 2024-04-27T04:18:58.141 INFO:teuthology.orchestra.run.smithi162.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2024-04-27T04:18:58.141 INFO:teuthology.orchestra.run.smithi162.stdout:created 2024-04-27T04:06:53.095749+0000 2024-04-27T04:18:58.141 INFO:teuthology.orchestra.run.smithi162.stdout:modified 2024-04-27T04:18:44.511964+0000 2024-04-27T04:18:58.141 INFO:teuthology.orchestra.run.smithi162.stdout:tableserver 0 2024-04-27T04:18:58.141 INFO:teuthology.orchestra.run.smithi162.stdout:root 0 2024-04-27T04:18:58.141 INFO:teuthology.orchestra.run.smithi162.stdout:session_timeout 60 2024-04-27T04:18:58.141 INFO:teuthology.orchestra.run.smithi162.stdout:session_autoclose 300 2024-04-27T04:18:58.142 INFO:teuthology.orchestra.run.smithi162.stdout:max_file_size 1099511627776 2024-04-27T04:18:58.142 INFO:teuthology.orchestra.run.smithi162.stdout:max_xattr_size 65536 2024-04-27T04:18:58.142 INFO:teuthology.orchestra.run.smithi162.stdout:required_client_features {} 2024-04-27T04:18:58.142 INFO:teuthology.orchestra.run.smithi162.stdout:last_failure 0 2024-04-27T04:18:58.142 INFO:teuthology.orchestra.run.smithi162.stdout:last_failure_osd_epoch 0 2024-04-27T04:18:58.142 INFO:teuthology.orchestra.run.smithi162.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-27T04:18:58.142 INFO:teuthology.orchestra.run.smithi162.stdout:max_mds 1 2024-04-27T04:18:58.142 INFO:teuthology.orchestra.run.smithi162.stdout:in 0 2024-04-27T04:18:58.142 INFO:teuthology.orchestra.run.smithi162.stdout:up {0=24273} 2024-04-27T04:18:58.142 INFO:teuthology.orchestra.run.smithi162.stdout:failed 2024-04-27T04:18:58.142 INFO:teuthology.orchestra.run.smithi162.stdout:damaged 2024-04-27T04:18:58.142 INFO:teuthology.orchestra.run.smithi162.stdout:stopped 2024-04-27T04:18:58.142 INFO:teuthology.orchestra.run.smithi162.stdout:data_pools [3] 2024-04-27T04:18:58.143 INFO:teuthology.orchestra.run.smithi162.stdout:metadata_pool 2 2024-04-27T04:18:58.143 INFO:teuthology.orchestra.run.smithi162.stdout:inline_data disabled 2024-04-27T04:18:58.143 INFO:teuthology.orchestra.run.smithi162.stdout:balancer 2024-04-27T04:18:58.143 INFO:teuthology.orchestra.run.smithi162.stdout:bal_rank_mask -1 2024-04-27T04:18:58.143 INFO:teuthology.orchestra.run.smithi162.stdout:standby_count_wanted 1 2024-04-27T04:18:58.143 INFO:teuthology.orchestra.run.smithi162.stdout:qdb_cluster leader: 24273 members: 24273 2024-04-27T04:18:58.143 INFO:teuthology.orchestra.run.smithi162.stdout:[mds.cephfs.smithi181.dzomgh{0:24273} state up:active seq 6 join_fscid=1 addr [v2:172.21.15.181:6824/658214993,v1:172.21.15.181:6825/658214993] compat {c=[1],r=[1],i=[7ff]}] 2024-04-27T04:18:58.143 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-04-27T04:18:58.143 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-04-27T04:18:58.143 INFO:teuthology.orchestra.run.smithi162.stdout:Standby daemons: 2024-04-27T04:18:58.143 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-04-27T04:18:58.143 INFO:teuthology.orchestra.run.smithi162.stdout:[mds.cephfs.smithi181.bkkcdm{-1:24287} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.181:6826/3756475453,v1:172.21.15.181:6827/3756475453] compat {c=[1],r=[1],i=[7ff]}] 2024-04-27T04:18:58.143 INFO:teuthology.orchestra.run.smithi162.stdout:[mds.cephfs.smithi162.xcnapk{-1:34376} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.162:6828/1115643134,v1:172.21.15.162:6829/1115643134] compat {c=[1],r=[1],i=[7ff]}] 2024-04-27T04:18:58.143 INFO:teuthology.orchestra.run.smithi162.stdout:[mds.cephfs.smithi162.gzlyjj{-1:34380} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.162:6826/1198130667,v1:172.21.15.162:6827/1198130667] compat {c=[1],r=[1],i=[1fff]}] 2024-04-27T04:18:58.144 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:18:58.144+0000 7f67a1045700 1 -- 172.21.15.162:0/1155431242 >> [v2:172.21.15.162:6800/1396866108,v1:172.21.15.162:6801/1396866108] conn(0x7f678406ec00 msgr2=0x7f67840710b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:18:58.144 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:18:58.144+0000 7f67a1045700 1 --2- 172.21.15.162:0/1155431242 >> [v2:172.21.15.162:6800/1396866108,v1:172.21.15.162:6801/1396866108] conn(0x7f678406ec00 0x7f67840710b0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f678c007fd0 tx=0x7f678c009040 comp rx=0 tx=0).stop 2024-04-27T04:18:58.144 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:18:58.144+0000 7f67a1045700 1 -- 172.21.15.162:0/1155431242 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f679c102670 msgr2=0x7f679c1140b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:18:58.144 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:18:58.144+0000 7f67a1045700 1 --2- 172.21.15.162:0/1155431242 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f679c102670 0x7f679c1140b0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f6790000c00 tx=0x7f6790009e30 comp rx=0 tx=0).stop 2024-04-27T04:18:58.145 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:18:58.145+0000 7f67a1045700 1 -- 172.21.15.162:0/1155431242 shutdown_connections 2024-04-27T04:18:58.145 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:18:58.145+0000 7f67a1045700 1 --2- 172.21.15.162:0/1155431242 >> [v2:172.21.15.162:6800/1396866108,v1:172.21.15.162:6801/1396866108] conn(0x7f678406ec00 0x7f67840710b0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:18:58.145 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:18:58.145+0000 7f67a1045700 1 --2- 172.21.15.162:0/1155431242 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f679c101cd0 0x7f679c113b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:18:58.145 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:18:58.145+0000 7f67a1045700 1 --2- 172.21.15.162:0/1155431242 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f679c102670 0x7f679c1140b0 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:18:58.145 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:18:58.145+0000 7f67a1045700 1 -- 172.21.15.162:0/1155431242 >> 172.21.15.162:0/1155431242 conn(0x7f679c071460 msgr2=0x7f679c1009d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:18:58.145 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:18:58.145+0000 7f67a1045700 1 -- 172.21.15.162:0/1155431242 shutdown_connections 2024-04-27T04:18:58.145 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:18:58.145+0000 7f67a1045700 1 -- 172.21.15.162:0/1155431242 wait complete. 2024-04-27T04:18:58.145 INFO:teuthology.orchestra.run.smithi162.stderr:dumped fsmap epoch 19 2024-04-27T04:18:58.171 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:18:57 smithi181 ceph-mon[81134]: from='client.34384 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-27T04:18:58.245 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:18:58.246+0000 7fc82a358700 1 -- 172.21.15.162:0/984491823 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc8240fde00 msgr2=0x7fc8240fe1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:18:58.246 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:18:58.246+0000 7fc82a358700 1 --2- 172.21.15.162:0/984491823 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc8240fde00 0x7fc8240fe1d0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fc814004940 tx=0x7fc814009e30 comp rx=0 tx=0).stop 2024-04-27T04:18:58.246 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:18:58.247+0000 7fc82a358700 1 -- 172.21.15.162:0/984491823 shutdown_connections 2024-04-27T04:18:58.246 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:18:58.247+0000 7fc82a358700 1 --2- 172.21.15.162:0/984491823 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc8240fe7a0 0x7fc824106de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:18:58.246 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:18:58.247+0000 7fc82a358700 1 --2- 172.21.15.162:0/984491823 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc8240fde00 0x7fc8240fe1d0 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:18:58.246 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:18:58.247+0000 7fc82a358700 1 -- 172.21.15.162:0/984491823 >> 172.21.15.162:0/984491823 conn(0x7fc8240f7870 msgr2=0x7fc8240f9c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:18:58.246 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:18:58.247+0000 7fc82a358700 1 -- 172.21.15.162:0/984491823 shutdown_connections 2024-04-27T04:18:58.246 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:18:58.247+0000 7fc82a358700 1 -- 172.21.15.162:0/984491823 wait complete. 2024-04-27T04:18:58.246 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:18:58.247+0000 7fc82a358700 1 Processor -- start 2024-04-27T04:18:58.248 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:18:58.247+0000 7fc82a358700 1 -- start start 2024-04-27T04:18:58.248 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:18:58.247+0000 7fc82a358700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc8240fde00 0x7fc824113970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:18:58.248 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:18:58.247+0000 7fc82a358700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc8240fe7a0 0x7fc824113eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:18:58.248 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:18:58.247+0000 7fc82a358700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc824114500 con 0x7fc8240fde00 2024-04-27T04:18:58.248 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:18:58.247+0000 7fc82a358700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc82410db20 con 0x7fc8240fe7a0 2024-04-27T04:18:58.248 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:18:58.248+0000 7fc823fff700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc8240fde00 0x7fc824113970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:18:58.248 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:18:58.248+0000 7fc823fff700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc8240fde00 0x7fc824113970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:59450/0 (socket says 172.21.15.162:59450) 2024-04-27T04:18:58.249 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:18:58.248+0000 7fc823fff700 1 -- 172.21.15.162:0/2428254460 learned_addr learned my addr 172.21.15.162:0/2428254460 (peer_addr_for_me v2:172.21.15.162:0/0) 2024-04-27T04:18:58.249 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:18:58.248+0000 7fc8237fe700 1 --2- 172.21.15.162:0/2428254460 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc8240fe7a0 0x7fc824113eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:18:58.249 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:18:58.248+0000 7fc823fff700 1 -- 172.21.15.162:0/2428254460 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc8240fe7a0 msgr2=0x7fc824113eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:18:58.249 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:18:58.248+0000 7fc823fff700 1 --2- 172.21.15.162:0/2428254460 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc8240fe7a0 0x7fc824113eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:18:58.249 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:18:58.248+0000 7fc823fff700 1 -- 172.21.15.162:0/2428254460 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc814005040 con 0x7fc8240fde00 2024-04-27T04:18:58.249 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:18:58.248+0000 7fc823fff700 1 --2- 172.21.15.162:0/2428254460 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc8240fde00 0x7fc824113970 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fc814009380 tx=0x7fc814009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:18:58.249 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:18:58.248+0000 7fc820ff9700 1 -- 172.21.15.162:0/2428254460 <== mon.0 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc81400e040 con 0x7fc8240fde00 2024-04-27T04:18:58.249 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:18:58.248+0000 7fc820ff9700 1 -- 172.21.15.162:0/2428254460 <== mon.0 v2:172.21.15.162:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fc814009af0 con 0x7fc8240fde00 2024-04-27T04:18:58.249 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:18:58.248+0000 7fc82a358700 1 -- 172.21.15.162:0/2428254460 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc82410dda0 con 0x7fc8240fde00 2024-04-27T04:18:58.249 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:18:58.248+0000 7fc820ff9700 1 -- 172.21.15.162:0/2428254460 <== mon.0 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc81401c3f0 con 0x7fc8240fde00 2024-04-27T04:18:58.249 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:18:58.248+0000 7fc82a358700 1 -- 172.21.15.162:0/2428254460 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc82410e2e0 con 0x7fc8240fde00 2024-04-27T04:18:58.249 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:18:58.249+0000 7fc82a358700 1 -- 172.21.15.162:0/2428254460 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc824066b30 con 0x7fc8240fde00 2024-04-27T04:18:58.250 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:18:58.250+0000 7fc820ff9700 1 -- 172.21.15.162:0/2428254460 <== mon.0 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 39) v1 ==== 92747+0+0 (secure 0 0 0) 0x7fc814013420 con 0x7fc8240fde00 2024-04-27T04:18:58.250 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:18:58.250+0000 7fc820ff9700 1 --2- 172.21.15.162:0/2428254460 >> [v2:172.21.15.162:6800/1396866108,v1:172.21.15.162:6801/1396866108] conn(0x7fc80c06ec00 0x7fc80c0710b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:18:58.250 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:18:58.250+0000 7fc820ff9700 1 -- 172.21.15.162:0/2428254460 <== mon.0 v2:172.21.15.162:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 6410+0+0 (secure 0 0 0) 0x7fc81408fb90 con 0x7fc8240fde00 2024-04-27T04:18:58.250 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:18:58.250+0000 7fc8237fe700 1 --2- 172.21.15.162:0/2428254460 >> [v2:172.21.15.162:6800/1396866108,v1:172.21.15.162:6801/1396866108] conn(0x7fc80c06ec00 0x7fc80c0710b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:18:58.250 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:18:58.251+0000 7fc8237fe700 1 --2- 172.21.15.162:0/2428254460 >> [v2:172.21.15.162:6800/1396866108,v1:172.21.15.162:6801/1396866108] conn(0x7fc80c06ec00 0x7fc80c0710b0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fc82410f210 tx=0x7fc8180043c0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:18:58.253 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:18:58.255+0000 7fc820ff9700 1 -- 172.21.15.162:0/2428254460 <== mon.0 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7fc814058990 con 0x7fc8240fde00 2024-04-27T04:18:58.475 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:18:58.476+0000 7fc82a358700 1 -- 172.21.15.162:0/2428254460 --> [v2:172.21.15.162:6800/1396866108,v1:172.21.15.162:6801/1396866108] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fc82410f080 con 0x7fc80c06ec00 2024-04-27T04:18:58.476 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:18:58.477+0000 7fc820ff9700 1 -- 172.21.15.162:0/2428254460 <== mgr.34104 v2:172.21.15.162:6800/1396866108 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7fc82410f080 con 0x7fc80c06ec00 2024-04-27T04:18:58.476 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-04-27T04:18:58.476 INFO:teuthology.orchestra.run.smithi162.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:90a3a9fa9ceb8977f6dad9ccd36fe46018d07e2118199820523a6b35301d9c16", 2024-04-27T04:18:58.476 INFO:teuthology.orchestra.run.smithi162.stdout: "in_progress": true, 2024-04-27T04:18:58.476 INFO:teuthology.orchestra.run.smithi162.stdout: "which": "Upgrading all daemon types on all hosts", 2024-04-27T04:18:58.476 INFO:teuthology.orchestra.run.smithi162.stdout: "services_complete": [ 2024-04-27T04:18:58.476 INFO:teuthology.orchestra.run.smithi162.stdout: "crash", 2024-04-27T04:18:58.476 INFO:teuthology.orchestra.run.smithi162.stdout: "mon", 2024-04-27T04:18:58.477 INFO:teuthology.orchestra.run.smithi162.stdout: "osd", 2024-04-27T04:18:58.477 INFO:teuthology.orchestra.run.smithi162.stdout: "mgr" 2024-04-27T04:18:58.477 INFO:teuthology.orchestra.run.smithi162.stdout: ], 2024-04-27T04:18:58.477 INFO:teuthology.orchestra.run.smithi162.stdout: "progress": "12/21 daemons upgraded", 2024-04-27T04:18:58.477 INFO:teuthology.orchestra.run.smithi162.stdout: "message": "Currently upgrading mds daemons", 2024-04-27T04:18:58.477 INFO:teuthology.orchestra.run.smithi162.stdout: "is_paused": false 2024-04-27T04:18:58.477 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-04-27T04:18:58.478 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:18:58.479+0000 7fc82a358700 1 -- 172.21.15.162:0/2428254460 >> [v2:172.21.15.162:6800/1396866108,v1:172.21.15.162:6801/1396866108] conn(0x7fc80c06ec00 msgr2=0x7fc80c0710b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:18:58.478 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:18:58.479+0000 7fc82a358700 1 --2- 172.21.15.162:0/2428254460 >> [v2:172.21.15.162:6800/1396866108,v1:172.21.15.162:6801/1396866108] conn(0x7fc80c06ec00 0x7fc80c0710b0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fc82410f210 tx=0x7fc8180043c0 comp rx=0 tx=0).stop 2024-04-27T04:18:58.478 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:18:58.479+0000 7fc82a358700 1 -- 172.21.15.162:0/2428254460 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc8240fde00 msgr2=0x7fc824113970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:18:58.478 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:18:58.479+0000 7fc82a358700 1 --2- 172.21.15.162:0/2428254460 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc8240fde00 0x7fc824113970 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fc814009380 tx=0x7fc814009e30 comp rx=0 tx=0).stop 2024-04-27T04:18:58.478 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:18:58.479+0000 7fc82a358700 1 -- 172.21.15.162:0/2428254460 shutdown_connections 2024-04-27T04:18:58.479 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:18:58.479+0000 7fc82a358700 1 --2- 172.21.15.162:0/2428254460 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc8240fde00 0x7fc824113970 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:18:58.479 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:18:58.479+0000 7fc82a358700 1 --2- 172.21.15.162:0/2428254460 >> [v2:172.21.15.162:6800/1396866108,v1:172.21.15.162:6801/1396866108] conn(0x7fc80c06ec00 0x7fc80c0710b0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:18:58.479 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:18:58.479+0000 7fc82a358700 1 --2- 172.21.15.162:0/2428254460 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc8240fe7a0 0x7fc824113eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:18:58.479 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:18:58.479+0000 7fc82a358700 1 -- 172.21.15.162:0/2428254460 >> 172.21.15.162:0/2428254460 conn(0x7fc8240f7870 msgr2=0x7fc8240f8da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:18:58.479 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:18:58.479+0000 7fc82a358700 1 -- 172.21.15.162:0/2428254460 shutdown_connections 2024-04-27T04:18:58.479 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:18:58.479+0000 7fc82a358700 1 -- 172.21.15.162:0/2428254460 wait complete. 2024-04-27T04:18:58.593 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:18:58.592+0000 7f60dcead700 1 -- 172.21.15.162:0/3713977111 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f60d80fded0 msgr2=0x7f60d80fe2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:18:58.593 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:18:58.592+0000 7f60dcead700 1 --2- 172.21.15.162:0/3713977111 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f60d80fded0 0x7f60d80fe2a0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f60c8004940 tx=0x7f60c8009e30 comp rx=0 tx=0).stop 2024-04-27T04:18:58.593 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:18:58.592+0000 7f60dcead700 1 -- 172.21.15.162:0/3713977111 shutdown_connections 2024-04-27T04:18:58.593 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:18:58.592+0000 7f60dcead700 1 --2- 172.21.15.162:0/3713977111 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f60d80fe870 0x7f60d8106eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:18:58.593 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:18:58.592+0000 7f60dcead700 1 --2- 172.21.15.162:0/3713977111 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f60d80fded0 0x7f60d80fe2a0 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:18:58.593 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:18:58.592+0000 7f60dcead700 1 -- 172.21.15.162:0/3713977111 >> 172.21.15.162:0/3713977111 conn(0x7f60d80f78e0 msgr2=0x7f60d80f9cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:18:58.593 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:18:58.592+0000 7f60dcead700 1 -- 172.21.15.162:0/3713977111 shutdown_connections 2024-04-27T04:18:58.594 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:18:58.592+0000 7f60dcead700 1 -- 172.21.15.162:0/3713977111 wait complete. 2024-04-27T04:18:58.594 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:18:58.593+0000 7f60dcead700 1 Processor -- start 2024-04-27T04:18:58.594 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:18:58.593+0000 7f60dcead700 1 -- start start 2024-04-27T04:18:58.594 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:18:58.593+0000 7f60dcead700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f60d80fded0 0x7f60d80fbd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:18:58.594 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:18:58.593+0000 7f60dcead700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f60d80fe870 0x7f60d80fc240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:18:58.595 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:18:58.593+0000 7f60dcead700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60d80fdc50 con 0x7f60d80fded0 2024-04-27T04:18:58.595 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:18:58.593+0000 7f60dcead700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60d80fc780 con 0x7f60d80fe870 2024-04-27T04:18:58.595 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:18:58.593+0000 7f60d659c700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f60d80fded0 0x7f60d80fbd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:18:58.595 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:18:58.593+0000 7f60d659c700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f60d80fded0 0x7f60d80fbd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:59460/0 (socket says 172.21.15.162:59460) 2024-04-27T04:18:58.595 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:18:58.593+0000 7f60d659c700 1 -- 172.21.15.162:0/532691597 learned_addr learned my addr 172.21.15.162:0/532691597 (peer_addr_for_me v2:172.21.15.162:0/0) 2024-04-27T04:18:58.595 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:18:58.593+0000 7f60d5d9b700 1 --2- 172.21.15.162:0/532691597 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f60d80fe870 0x7f60d80fc240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:18:58.595 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:18:58.593+0000 7f60d659c700 1 -- 172.21.15.162:0/532691597 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f60d80fe870 msgr2=0x7f60d80fc240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:18:58.595 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:18:58.593+0000 7f60d659c700 1 --2- 172.21.15.162:0/532691597 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f60d80fe870 0x7f60d80fc240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:18:58.595 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:18:58.593+0000 7f60d659c700 1 -- 172.21.15.162:0/532691597 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f60c8005040 con 0x7f60d80fded0 2024-04-27T04:18:58.595 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:18:58.593+0000 7f60d659c700 1 --2- 172.21.15.162:0/532691597 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f60d80fded0 0x7f60d80fbd00 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f60c8009380 tx=0x7f60c8009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:18:58.595 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:18:58.594+0000 7f60c6ffd700 1 -- 172.21.15.162:0/532691597 <== mon.0 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f60c800e040 con 0x7f60d80fded0 2024-04-27T04:18:58.595 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:18:58.594+0000 7f60c6ffd700 1 -- 172.21.15.162:0/532691597 <== mon.0 v2:172.21.15.162:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f60c8009af0 con 0x7f60d80fded0 2024-04-27T04:18:58.596 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:18:58.594+0000 7f60dcead700 1 -- 172.21.15.162:0/532691597 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f60d80fca60 con 0x7f60d80fded0 2024-04-27T04:18:58.596 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:18:58.594+0000 7f60c6ffd700 1 -- 172.21.15.162:0/532691597 <== mon.0 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f60c801c630 con 0x7f60d80fded0 2024-04-27T04:18:58.596 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:18:58.594+0000 7f60dcead700 1 -- 172.21.15.162:0/532691597 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f60d8119fe0 con 0x7f60d80fded0 2024-04-27T04:18:58.597 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:18:58.595+0000 7f60dcead700 1 -- 172.21.15.162:0/532691597 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f60d8066a70 con 0x7f60d80fded0 2024-04-27T04:18:58.597 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:18:58.595+0000 7f60c6ffd700 1 -- 172.21.15.162:0/532691597 <== mon.0 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 39) v1 ==== 92747+0+0 (secure 0 0 0) 0x7f60c800bbc0 con 0x7f60d80fded0 2024-04-27T04:18:58.597 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:18:58.596+0000 7f60c6ffd700 1 --2- 172.21.15.162:0/532691597 >> [v2:172.21.15.162:6800/1396866108,v1:172.21.15.162:6801/1396866108] conn(0x7f60c006ec00 0x7f60c00710b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:18:58.598 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:18:58.596+0000 7f60c6ffd700 1 -- 172.21.15.162:0/532691597 <== mon.0 v2:172.21.15.162:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 6410+0+0 (secure 0 0 0) 0x7f60c808fae0 con 0x7f60d80fded0 2024-04-27T04:18:58.598 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:18:58.596+0000 7f60d5d9b700 1 --2- 172.21.15.162:0/532691597 >> [v2:172.21.15.162:6800/1396866108,v1:172.21.15.162:6801/1396866108] conn(0x7f60c006ec00 0x7f60c00710b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:18:58.598 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:18:58.596+0000 7f60d5d9b700 1 --2- 172.21.15.162:0/532691597 >> [v2:172.21.15.162:6800/1396866108,v1:172.21.15.162:6801/1396866108] conn(0x7f60c006ec00 0x7f60c00710b0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f60cc007fd0 tx=0x7f60cc009040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:18:58.601 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:18:58.602+0000 7f60c6ffd700 1 -- 172.21.15.162:0/532691597 <== mon.0 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7f60c8058960 con 0x7f60d80fded0 2024-04-27T04:18:58.886 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:18:58 smithi162 ceph-mon[110641]: from='client.34388 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-27T04:18:58.886 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:18:58 smithi162 ceph-mon[110641]: from='client.34392 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-27T04:18:58.886 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:18:58 smithi162 ceph-mon[110641]: pgmap v175: 65 pgs: 65 active+clean; 208 MiB data, 851 MiB used, 536 GiB / 536 GiB avail; 0 B/s rd, 0 op/s 2024-04-27T04:18:58.886 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:18:58 smithi162 ceph-mon[110641]: from='client.? 172.21.15.162:0/2762199298' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-27T04:18:58.887 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:18:58 smithi162 ceph-mon[110641]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' 2024-04-27T04:18:58.887 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:18:58 smithi162 ceph-mon[110641]: from='client.? 172.21.15.162:0/1155431242' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-27T04:18:58.922 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:18:58.921+0000 7f60dcead700 1 -- 172.21.15.162:0/532691597 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f60d804c710 con 0x7f60d80fded0 2024-04-27T04:18:58.922 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:18:58.922+0000 7f60c6ffd700 1 -- 172.21.15.162:0/532691597 <== mon.0 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+10 (secure 0 0 0) 0x7f60c805c020 con 0x7f60d80fded0 2024-04-27T04:18:58.922 INFO:teuthology.orchestra.run.smithi162.stdout:HEALTH_OK 2024-04-27T04:18:58.923 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:18:58.924+0000 7f60dcead700 1 -- 172.21.15.162:0/532691597 >> [v2:172.21.15.162:6800/1396866108,v1:172.21.15.162:6801/1396866108] conn(0x7f60c006ec00 msgr2=0x7f60c00710b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:18:58.923 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:18:58.924+0000 7f60dcead700 1 --2- 172.21.15.162:0/532691597 >> [v2:172.21.15.162:6800/1396866108,v1:172.21.15.162:6801/1396866108] conn(0x7f60c006ec00 0x7f60c00710b0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f60cc007fd0 tx=0x7f60cc009040 comp rx=0 tx=0).stop 2024-04-27T04:18:58.923 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:18:58.924+0000 7f60dcead700 1 -- 172.21.15.162:0/532691597 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f60d80fded0 msgr2=0x7f60d80fbd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:18:58.923 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:18:58.924+0000 7f60dcead700 1 --2- 172.21.15.162:0/532691597 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f60d80fded0 0x7f60d80fbd00 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f60c8009380 tx=0x7f60c8009e30 comp rx=0 tx=0).stop 2024-04-27T04:18:58.923 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:18:58.924+0000 7f60dcead700 1 -- 172.21.15.162:0/532691597 shutdown_connections 2024-04-27T04:18:58.923 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:18:58.924+0000 7f60dcead700 1 --2- 172.21.15.162:0/532691597 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f60d80fded0 0x7f60d80fbd00 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:18:58.923 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:18:58.924+0000 7f60dcead700 1 --2- 172.21.15.162:0/532691597 >> [v2:172.21.15.162:6800/1396866108,v1:172.21.15.162:6801/1396866108] conn(0x7f60c006ec00 0x7f60c00710b0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:18:58.923 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:18:58.924+0000 7f60dcead700 1 --2- 172.21.15.162:0/532691597 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f60d80fe870 0x7f60d80fc240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:18:58.924 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:18:58.924+0000 7f60dcead700 1 -- 172.21.15.162:0/532691597 >> 172.21.15.162:0/532691597 conn(0x7f60d80f78e0 msgr2=0x7f60d8103720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:18:58.924 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:18:58.924+0000 7f60dcead700 1 -- 172.21.15.162:0/532691597 shutdown_connections 2024-04-27T04:18:58.924 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:18:58.924+0000 7f60dcead700 1 -- 172.21.15.162:0/532691597 wait complete. 2024-04-27T04:18:59.171 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:18:58 smithi181 ceph-mon[81134]: from='client.34388 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-27T04:18:59.172 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:18:58 smithi181 ceph-mon[81134]: from='client.34392 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-27T04:18:59.172 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:18:58 smithi181 ceph-mon[81134]: pgmap v175: 65 pgs: 65 active+clean; 208 MiB data, 851 MiB used, 536 GiB / 536 GiB avail; 0 B/s rd, 0 op/s 2024-04-27T04:18:59.172 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:18:58 smithi181 ceph-mon[81134]: from='client.? 172.21.15.162:0/2762199298' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-27T04:18:59.172 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:18:58 smithi181 ceph-mon[81134]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' 2024-04-27T04:18:59.172 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:18:58 smithi181 ceph-mon[81134]: from='client.? 172.21.15.162:0/1155431242' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-27T04:19:00.136 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:18:59 smithi162 ceph-mon[110641]: from='client.34404 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-27T04:19:00.136 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:18:59 smithi162 ceph-mon[110641]: from='client.? 172.21.15.162:0/532691597' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-27T04:19:00.136 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:18:59 smithi162 ceph-mon[110641]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' 2024-04-27T04:19:00.136 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:18:59 smithi162 ceph-mon[110641]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' 2024-04-27T04:19:00.171 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:18:59 smithi181 ceph-mon[81134]: from='client.34404 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-27T04:19:00.171 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:18:59 smithi181 ceph-mon[81134]: from='client.? 172.21.15.162:0/532691597' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-27T04:19:00.171 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:18:59 smithi181 ceph-mon[81134]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' 2024-04-27T04:19:00.171 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:18:59 smithi181 ceph-mon[81134]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' 2024-04-27T04:19:01.136 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:19:00 smithi162 ceph-mon[110641]: pgmap v176: 65 pgs: 65 active+clean; 208 MiB data, 851 MiB used, 536 GiB / 536 GiB avail 2024-04-27T04:19:01.136 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:19:00 smithi162 ceph-mon[110641]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' 2024-04-27T04:19:01.136 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:19:00 smithi162 ceph-mon[110641]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' 2024-04-27T04:19:01.171 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:19:00 smithi181 ceph-mon[81134]: pgmap v176: 65 pgs: 65 active+clean; 208 MiB data, 851 MiB used, 536 GiB / 536 GiB avail 2024-04-27T04:19:01.171 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:19:00 smithi181 ceph-mon[81134]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' 2024-04-27T04:19:01.172 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:19:00 smithi181 ceph-mon[81134]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' 2024-04-27T04:19:02.886 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:19:02 smithi162 ceph-mon[110641]: pgmap v177: 65 pgs: 65 active+clean; 208 MiB data, 851 MiB used, 536 GiB / 536 GiB avail 2024-04-27T04:19:02.887 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:19:02 smithi162 ceph-mon[110641]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' 2024-04-27T04:19:02.887 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:19:02 smithi162 ceph-mon[110641]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-27T04:19:02.974 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:19:02 smithi181 ceph-mon[81134]: pgmap v177: 65 pgs: 65 active+clean; 208 MiB data, 851 MiB used, 536 GiB / 536 GiB avail 2024-04-27T04:19:02.974 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:19:02 smithi181 ceph-mon[81134]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' 2024-04-27T04:19:02.974 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:19:02 smithi181 ceph-mon[81134]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-27T04:19:04.994 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:19:04 smithi162 ceph-mon[110641]: pgmap v178: 65 pgs: 65 active+clean; 208 MiB data, 851 MiB used, 536 GiB / 536 GiB avail 2024-04-27T04:19:04.994 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:19:04 smithi162 ceph-mon[110641]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' 2024-04-27T04:19:04.994 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:19:04 smithi162 ceph-mon[110641]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' 2024-04-27T04:19:04.994 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:19:04 smithi162 ceph-mon[110641]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T04:19:04.994 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:19:04 smithi162 ceph-mon[110641]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T04:19:04.994 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:19:04 smithi162 ceph-mon[110641]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' 2024-04-27T04:19:04.994 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:19:04 smithi162 ceph-mon[110641]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T04:19:04.995 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:19:04 smithi162 ceph-mon[110641]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "versions"}]: dispatch 2024-04-27T04:19:04.995 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:19:04 smithi162 ceph-mon[110641]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "versions"}]: dispatch 2024-04-27T04:19:04.995 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:19:04 smithi162 ceph-mon[110641]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "versions"}]: dispatch 2024-04-27T04:19:04.995 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:19:04 smithi162 ceph-mon[110641]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "versions"}]: dispatch 2024-04-27T04:19:05.172 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:19:04 smithi181 ceph-mon[81134]: pgmap v178: 65 pgs: 65 active+clean; 208 MiB data, 851 MiB used, 536 GiB / 536 GiB avail 2024-04-27T04:19:05.172 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:19:04 smithi181 ceph-mon[81134]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' 2024-04-27T04:19:05.172 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:19:04 smithi181 ceph-mon[81134]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' 2024-04-27T04:19:05.172 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:19:04 smithi181 ceph-mon[81134]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T04:19:05.172 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:19:04 smithi181 ceph-mon[81134]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T04:19:05.172 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:19:04 smithi181 ceph-mon[81134]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' 2024-04-27T04:19:05.172 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:19:04 smithi181 ceph-mon[81134]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T04:19:05.172 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:19:04 smithi181 ceph-mon[81134]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "versions"}]: dispatch 2024-04-27T04:19:05.172 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:19:04 smithi181 ceph-mon[81134]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "versions"}]: dispatch 2024-04-27T04:19:05.173 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:19:04 smithi181 ceph-mon[81134]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "versions"}]: dispatch 2024-04-27T04:19:05.173 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:19:04 smithi181 ceph-mon[81134]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "versions"}]: dispatch 2024-04-27T04:19:06.763 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:19:06 smithi162 ceph-mon[110641]: pgmap v179: 65 pgs: 65 active+clean; 208 MiB data, 851 MiB used, 536 GiB / 536 GiB avail 2024-04-27T04:19:06.763 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:19:06 smithi162 ceph-mon[110641]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' 2024-04-27T04:19:06.763 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:19:06 smithi162 ceph-mon[110641]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi162.xcnapk", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-27T04:19:06.764 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:19:06 smithi162 ceph-mon[110641]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T04:19:07.171 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:19:06 smithi181 ceph-mon[81134]: pgmap v179: 65 pgs: 65 active+clean; 208 MiB data, 851 MiB used, 536 GiB / 536 GiB avail 2024-04-27T04:19:07.172 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:19:06 smithi181 ceph-mon[81134]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' 2024-04-27T04:19:07.172 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:19:06 smithi181 ceph-mon[81134]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi162.xcnapk", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-27T04:19:07.172 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:19:06 smithi181 ceph-mon[81134]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T04:19:07.886 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:19:07 smithi162 ceph-mon[110641]: Upgrade: Updating mds.cephfs.smithi162.xcnapk 2024-04-27T04:19:07.886 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:19:07 smithi162 ceph-mon[110641]: Deploying daemon mds.cephfs.smithi162.xcnapk on smithi162 2024-04-27T04:19:08.172 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:19:07 smithi181 ceph-mon[81134]: Upgrade: Updating mds.cephfs.smithi162.xcnapk 2024-04-27T04:19:08.172 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:19:07 smithi181 ceph-mon[81134]: Deploying daemon mds.cephfs.smithi162.xcnapk on smithi162 2024-04-27T04:19:09.136 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:19:08 smithi162 ceph-mon[110641]: pgmap v180: 65 pgs: 65 active+clean; 208 MiB data, 851 MiB used, 536 GiB / 536 GiB avail 2024-04-27T04:19:09.172 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:19:08 smithi181 ceph-mon[81134]: pgmap v180: 65 pgs: 65 active+clean; 208 MiB data, 851 MiB used, 536 GiB / 536 GiB avail 2024-04-27T04:19:10.136 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:19:09 smithi162 ceph-mon[110641]: osdmap e82: 6 total, 6 up, 6 in 2024-04-27T04:19:10.136 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:19:09 smithi162 ceph-mon[110641]: fsmap cephfs:1 {0=cephfs.smithi181.dzomgh=up:active} 2 up:standby 2024-04-27T04:19:10.172 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:19:09 smithi181 ceph-mon[81134]: osdmap e82: 6 total, 6 up, 6 in 2024-04-27T04:19:10.172 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:19:09 smithi181 ceph-mon[81134]: fsmap cephfs:1 {0=cephfs.smithi181.dzomgh=up:active} 2 up:standby 2024-04-27T04:19:11.136 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:19:10 smithi162 ceph-mon[110641]: pgmap v182: 65 pgs: 65 active+clean; 208 MiB data, 851 MiB used, 536 GiB / 536 GiB avail 2024-04-27T04:19:11.136 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:19:10 smithi162 ceph-mon[110641]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi162.xcnapk"}]: dispatch 2024-04-27T04:19:11.171 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:19:10 smithi181 ceph-mon[81134]: pgmap v182: 65 pgs: 65 active+clean; 208 MiB data, 851 MiB used, 536 GiB / 536 GiB avail 2024-04-27T04:19:11.171 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:19:10 smithi181 ceph-mon[81134]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi162.xcnapk"}]: dispatch 2024-04-27T04:19:13.137 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:19:12 smithi162 ceph-mon[110641]: pgmap v183: 65 pgs: 65 active+clean; 208 MiB data, 851 MiB used, 536 GiB / 536 GiB avail 2024-04-27T04:19:13.171 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:19:12 smithi181 ceph-mon[81134]: pgmap v183: 65 pgs: 65 active+clean; 208 MiB data, 851 MiB used, 536 GiB / 536 GiB avail 2024-04-27T04:19:14.886 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:19:14 smithi162 ceph-mon[110641]: pgmap v184: 65 pgs: 65 active+clean; 208 MiB data, 851 MiB used, 536 GiB / 536 GiB avail 2024-04-27T04:19:14.886 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:19:14 smithi162 ceph-mon[110641]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' 2024-04-27T04:19:14.887 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:19:14 smithi162 ceph-mon[110641]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' 2024-04-27T04:19:14.887 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:19:14 smithi162 ceph-mon[110641]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T04:19:14.887 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:19:14 smithi162 ceph-mon[110641]: mds.? [v2:172.21.15.162:6828/307528848,v1:172.21.15.162:6829/307528848] up:boot 2024-04-27T04:19:14.887 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:19:14 smithi162 ceph-mon[110641]: fsmap cephfs:1 {0=cephfs.smithi181.dzomgh=up:active} 3 up:standby 2024-04-27T04:19:14.887 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:19:14 smithi162 ceph-mon[110641]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi162.xcnapk"}]: dispatch 2024-04-27T04:19:15.172 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:19:14 smithi181 ceph-mon[81134]: pgmap v184: 65 pgs: 65 active+clean; 208 MiB data, 851 MiB used, 536 GiB / 536 GiB avail 2024-04-27T04:19:15.172 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:19:14 smithi181 ceph-mon[81134]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' 2024-04-27T04:19:15.172 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:19:14 smithi181 ceph-mon[81134]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' 2024-04-27T04:19:15.172 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:19:14 smithi181 ceph-mon[81134]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T04:19:15.172 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:19:14 smithi181 ceph-mon[81134]: mds.? [v2:172.21.15.162:6828/307528848,v1:172.21.15.162:6829/307528848] up:boot 2024-04-27T04:19:15.172 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:19:14 smithi181 ceph-mon[81134]: fsmap cephfs:1 {0=cephfs.smithi181.dzomgh=up:active} 3 up:standby 2024-04-27T04:19:15.172 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:19:14 smithi181 ceph-mon[81134]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi162.xcnapk"}]: dispatch 2024-04-27T04:19:17.136 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:19:16 smithi162 ceph-mon[110641]: pgmap v185: 65 pgs: 65 active+clean; 208 MiB data, 851 MiB used, 536 GiB / 536 GiB avail 2024-04-27T04:19:17.171 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:19:16 smithi181 ceph-mon[81134]: pgmap v185: 65 pgs: 65 active+clean; 208 MiB data, 851 MiB used, 536 GiB / 536 GiB avail 2024-04-27T04:19:18.136 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:19:17 smithi162 ceph-mon[110641]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-27T04:19:18.171 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:19:17 smithi181 ceph-mon[81134]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-27T04:19:19.136 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:19:18 smithi162 ceph-mon[110641]: pgmap v186: 65 pgs: 65 active+clean; 208 MiB data, 851 MiB used, 536 GiB / 536 GiB avail 2024-04-27T04:19:19.136 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:19:18 smithi162 ceph-mon[110641]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' 2024-04-27T04:19:19.136 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:19:18 smithi162 ceph-mon[110641]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' 2024-04-27T04:19:19.171 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:19:18 smithi181 ceph-mon[81134]: pgmap v186: 65 pgs: 65 active+clean; 208 MiB data, 851 MiB used, 536 GiB / 536 GiB avail 2024-04-27T04:19:19.172 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:19:18 smithi181 ceph-mon[81134]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' 2024-04-27T04:19:19.172 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:19:18 smithi181 ceph-mon[81134]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' 2024-04-27T04:19:20.636 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:19:20 smithi162 ceph-mon[110641]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' 2024-04-27T04:19:20.636 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:19:20 smithi162 ceph-mon[110641]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' 2024-04-27T04:19:20.636 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:19:20 smithi162 ceph-mon[110641]: pgmap v187: 65 pgs: 65 active+clean; 208 MiB data, 851 MiB used, 536 GiB / 536 GiB avail 2024-04-27T04:19:20.671 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:19:20 smithi181 ceph-mon[81134]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' 2024-04-27T04:19:20.672 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:19:20 smithi181 ceph-mon[81134]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' 2024-04-27T04:19:20.672 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:19:20 smithi181 ceph-mon[81134]: pgmap v187: 65 pgs: 65 active+clean; 208 MiB data, 851 MiB used, 536 GiB / 536 GiB avail 2024-04-27T04:19:22.886 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:19:22 smithi162 ceph-mon[110641]: pgmap v188: 65 pgs: 65 active+clean; 208 MiB data, 851 MiB used, 536 GiB / 536 GiB avail 2024-04-27T04:19:22.886 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:19:22 smithi162 ceph-mon[110641]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' 2024-04-27T04:19:22.886 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:19:22 smithi162 ceph-mon[110641]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' 2024-04-27T04:19:22.886 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:19:22 smithi162 ceph-mon[110641]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T04:19:22.887 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:19:22 smithi162 ceph-mon[110641]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T04:19:22.887 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:19:22 smithi162 ceph-mon[110641]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' 2024-04-27T04:19:23.171 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:19:22 smithi181 ceph-mon[81134]: pgmap v188: 65 pgs: 65 active+clean; 208 MiB data, 851 MiB used, 536 GiB / 536 GiB avail 2024-04-27T04:19:23.172 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:19:22 smithi181 ceph-mon[81134]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' 2024-04-27T04:19:23.172 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:19:22 smithi181 ceph-mon[81134]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' 2024-04-27T04:19:23.172 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:19:22 smithi181 ceph-mon[81134]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T04:19:23.172 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:19:22 smithi181 ceph-mon[81134]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T04:19:23.172 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:19:22 smithi181 ceph-mon[81134]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' 2024-04-27T04:19:23.921 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:19:23 smithi181 ceph-mon[81134]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T04:19:23.921 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:19:23 smithi181 ceph-mon[81134]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "versions"}]: dispatch 2024-04-27T04:19:23.921 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:19:23 smithi181 ceph-mon[81134]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "versions"}]: dispatch 2024-04-27T04:19:23.922 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:19:23 smithi181 ceph-mon[81134]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "versions"}]: dispatch 2024-04-27T04:19:23.922 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:19:23 smithi181 ceph-mon[81134]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "versions"}]: dispatch 2024-04-27T04:19:24.136 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:19:23 smithi162 ceph-mon[110641]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T04:19:24.136 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:19:23 smithi162 ceph-mon[110641]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "versions"}]: dispatch 2024-04-27T04:19:24.137 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:19:23 smithi162 ceph-mon[110641]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "versions"}]: dispatch 2024-04-27T04:19:24.137 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:19:23 smithi162 ceph-mon[110641]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "versions"}]: dispatch 2024-04-27T04:19:24.137 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:19:23 smithi162 ceph-mon[110641]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "versions"}]: dispatch 2024-04-27T04:19:25.001 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:19:24 smithi181 ceph-mon[81134]: pgmap v189: 65 pgs: 65 active+clean; 208 MiB data, 851 MiB used, 536 GiB / 536 GiB avail 2024-04-27T04:19:25.001 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:19:24 smithi181 ceph-mon[81134]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' 2024-04-27T04:19:25.001 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:19:24 smithi181 ceph-mon[81134]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi181.dzomgh", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-27T04:19:25.001 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:19:24 smithi181 ceph-mon[81134]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T04:19:25.136 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:19:24 smithi162 ceph-mon[110641]: pgmap v189: 65 pgs: 65 active+clean; 208 MiB data, 851 MiB used, 536 GiB / 536 GiB avail 2024-04-27T04:19:25.136 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:19:24 smithi162 ceph-mon[110641]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' 2024-04-27T04:19:25.136 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:19:24 smithi162 ceph-mon[110641]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi181.dzomgh", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-27T04:19:25.136 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:19:24 smithi162 ceph-mon[110641]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T04:19:25.874 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:19:25 smithi181 ceph-mon[81134]: Upgrade: Updating mds.cephfs.smithi181.dzomgh 2024-04-27T04:19:25.875 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:19:25 smithi181 ceph-mon[81134]: Deploying daemon mds.cephfs.smithi181.dzomgh on smithi181 2024-04-27T04:19:26.136 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:19:25 smithi162 ceph-mon[110641]: Upgrade: Updating mds.cephfs.smithi181.dzomgh 2024-04-27T04:19:26.136 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:19:25 smithi162 ceph-mon[110641]: Deploying daemon mds.cephfs.smithi181.dzomgh on smithi181 2024-04-27T04:19:26.743 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:19:26 smithi162 ceph-7f9750d6-044a-11ef-bc93-c7b262605968-mon-smithi162[110637]: 2024-04-27T04:19:26.414+0000 7f2a7886b700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-27T04:19:27.136 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:19:26 smithi162 ceph-mon[110641]: pgmap v190: 65 pgs: 65 active+clean; 208 MiB data, 851 MiB used, 536 GiB / 536 GiB avail 2024-04-27T04:19:27.136 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:19:26 smithi162 ceph-mon[110641]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-04-27T04:19:27.137 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:19:26 smithi162 ceph-mon[110641]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-27T04:19:27.137 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:19:26 smithi162 ceph-mon[110641]: osdmap e83: 6 total, 6 up, 6 in 2024-04-27T04:19:27.137 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:19:26 smithi162 ceph-mon[110641]: Standby daemon mds.cephfs.smithi181.bkkcdm assigned to filesystem cephfs as rank 0 2024-04-27T04:19:27.137 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:19:26 smithi162 ceph-mon[110641]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-04-27T04:19:27.137 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:19:26 smithi162 ceph-mon[110641]: fsmap cephfs:0/1 3 up:standby, 1 failed 2024-04-27T04:19:27.137 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:19:26 smithi162 ceph-mon[110641]: fsmap cephfs:1/1 {0=cephfs.smithi181.bkkcdm=up:replay} 2 up:standby 2024-04-27T04:19:27.171 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:19:26 smithi181 ceph-mon[81134]: pgmap v190: 65 pgs: 65 active+clean; 208 MiB data, 851 MiB used, 536 GiB / 536 GiB avail 2024-04-27T04:19:27.172 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:19:26 smithi181 ceph-mon[81134]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-04-27T04:19:27.172 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:19:26 smithi181 ceph-mon[81134]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-27T04:19:27.172 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:19:26 smithi181 ceph-mon[81134]: osdmap e83: 6 total, 6 up, 6 in 2024-04-27T04:19:27.172 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:19:26 smithi181 ceph-mon[81134]: Standby daemon mds.cephfs.smithi181.bkkcdm assigned to filesystem cephfs as rank 0 2024-04-27T04:19:27.172 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:19:26 smithi181 ceph-mon[81134]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-04-27T04:19:27.172 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:19:26 smithi181 ceph-mon[81134]: fsmap cephfs:0/1 3 up:standby, 1 failed 2024-04-27T04:19:27.172 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:19:26 smithi181 ceph-mon[81134]: fsmap cephfs:1/1 {0=cephfs.smithi181.bkkcdm=up:replay} 2 up:standby 2024-04-27T04:19:29.038 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:19:28 smithi162 ceph-mon[110641]: pgmap v192: 65 pgs: 65 active+clean; 208 MiB data, 851 MiB used, 536 GiB / 536 GiB avail; 1.4 MiB/s rd, 0 op/s 2024-04-27T04:19:29.038 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:19:28 smithi162 ceph-mon[110641]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi181.dzomgh"}]: dispatch 2024-04-27T04:19:29.040 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:19:29.040+0000 7f69f8759700 1 -- 172.21.15.162:0/297773419 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f69f0102de0 msgr2=0x7f69f01031b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:19:29.040 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:19:29.040+0000 7f69f8759700 1 --2- 172.21.15.162:0/297773419 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f69f0102de0 0x7f69f01031b0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f69e8009ce0 tx=0x7f69e8004d10 comp rx=0 tx=0).stop 2024-04-27T04:19:29.040 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:19:29.040+0000 7f69f8759700 1 -- 172.21.15.162:0/297773419 shutdown_connections 2024-04-27T04:19:29.040 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:19:29.040+0000 7f69f8759700 1 --2- 172.21.15.162:0/297773419 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f69f0103780 0x7f69f00fcdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:19:29.040 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:19:29.040+0000 7f69f8759700 1 --2- 172.21.15.162:0/297773419 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f69f0102de0 0x7f69f01031b0 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:19:29.040 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:19:29.040+0000 7f69f8759700 1 -- 172.21.15.162:0/297773419 >> 172.21.15.162:0/297773419 conn(0x7f69f00f8990 msgr2=0x7f69f00fada0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:19:29.040 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:19:29.040+0000 7f69f8759700 1 -- 172.21.15.162:0/297773419 shutdown_connections 2024-04-27T04:19:29.040 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:19:29.040+0000 7f69f8759700 1 -- 172.21.15.162:0/297773419 wait complete. 2024-04-27T04:19:29.040 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:19:29.040+0000 7f69f8759700 1 Processor -- start 2024-04-27T04:19:29.040 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:19:29.040+0000 7f69f8759700 1 -- start start 2024-04-27T04:19:29.040 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:19:29.041+0000 7f69f8759700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f69f0102de0 0x7f69f010bb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:19:29.040 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:19:29.041+0000 7f69f8759700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f69f0103780 0x7f69f010c040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:19:29.040 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:19:29.041+0000 7f69f8759700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69f010c580 con 0x7f69f0102de0 2024-04-27T04:19:29.041 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:19:29.041+0000 7f69f8759700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69f010c6f0 con 0x7f69f0103780 2024-04-27T04:19:29.041 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:19:29.041+0000 7f69f64f5700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f69f0102de0 0x7f69f010bb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:19:29.041 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:19:29.041+0000 7f69f5cf4700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f69f0103780 0x7f69f010c040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:19:29.041 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:19:29.041+0000 7f69f64f5700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f69f0102de0 0x7f69f010bb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:56254/0 (socket says 172.21.15.162:56254) 2024-04-27T04:19:29.041 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:19:29.041+0000 7f69f64f5700 1 -- 172.21.15.162:0/1489067288 learned_addr learned my addr 172.21.15.162:0/1489067288 (peer_addr_for_me v2:172.21.15.162:0/0) 2024-04-27T04:19:29.041 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:19:29.041+0000 7f69f64f5700 1 -- 172.21.15.162:0/1489067288 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f69f0103780 msgr2=0x7f69f010c040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:19:29.041 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:19:29.041+0000 7f69f64f5700 1 --2- 172.21.15.162:0/1489067288 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f69f0103780 0x7f69f010c040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:19:29.041 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:19:29.041+0000 7f69f64f5700 1 -- 172.21.15.162:0/1489067288 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f69e8009940 con 0x7f69f0102de0 2024-04-27T04:19:29.041 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:19:29.041+0000 7f69f64f5700 1 --2- 172.21.15.162:0/1489067288 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f69f0102de0 0x7f69f010bb00 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f69e8003400 tx=0x7f69e8004d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:19:29.041 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:19:29.041+0000 7f69e6ffd700 1 -- 172.21.15.162:0/1489067288 <== mon.0 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f69e800e040 con 0x7f69f0102de0 2024-04-27T04:19:29.041 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:19:29.041+0000 7f69e6ffd700 1 -- 172.21.15.162:0/1489067288 <== mon.0 v2:172.21.15.162:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f69e8004610 con 0x7f69f0102de0 2024-04-27T04:19:29.041 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:19:29.041+0000 7f69f8759700 1 -- 172.21.15.162:0/1489067288 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f69f010c970 con 0x7f69f0102de0 2024-04-27T04:19:29.041 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:19:29.041+0000 7f69e6ffd700 1 -- 172.21.15.162:0/1489067288 <== mon.0 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f69e801c640 con 0x7f69f0102de0 2024-04-27T04:19:29.042 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:19:29.041+0000 7f69f8759700 1 -- 172.21.15.162:0/1489067288 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f69f011e5d0 con 0x7f69f0102de0 2024-04-27T04:19:29.042 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:19:29.042+0000 7f69f8759700 1 -- 172.21.15.162:0/1489067288 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f69f0106a30 con 0x7f69f0102de0 2024-04-27T04:19:29.042 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:19:29.043+0000 7f69e6ffd700 1 -- 172.21.15.162:0/1489067288 <== mon.0 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 39) v1 ==== 92747+0+0 (secure 0 0 0) 0x7f69e8012a70 con 0x7f69f0102de0 2024-04-27T04:19:29.043 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:19:29.043+0000 7f69e6ffd700 1 --2- 172.21.15.162:0/1489067288 >> [v2:172.21.15.162:6800/1396866108,v1:172.21.15.162:6801/1396866108] conn(0x7f69dc06ec00 0x7f69dc0710b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:19:29.043 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:19:29.044+0000 7f69f5cf4700 1 --2- 172.21.15.162:0/1489067288 >> [v2:172.21.15.162:6800/1396866108,v1:172.21.15.162:6801/1396866108] conn(0x7f69dc06ec00 0x7f69dc0710b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:19:29.043 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:19:29.044+0000 7f69e6ffd700 1 -- 172.21.15.162:0/1489067288 <== mon.0 v2:172.21.15.162:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6496+0+0 (secure 0 0 0) 0x7f69e8090050 con 0x7f69f0102de0 2024-04-27T04:19:29.043 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:19:29.044+0000 7f69f5cf4700 1 --2- 172.21.15.162:0/1489067288 >> [v2:172.21.15.162:6800/1396866108,v1:172.21.15.162:6801/1396866108] conn(0x7f69dc06ec00 0x7f69dc0710b0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f69f00721f0 tx=0x7f69e000b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:19:29.046 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:19:29.048+0000 7f69e6ffd700 1 -- 172.21.15.162:0/1489067288 <== mon.0 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7f69e805c930 con 0x7f69f0102de0 2024-04-27T04:19:29.171 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:19:28 smithi181 ceph-mon[81134]: pgmap v192: 65 pgs: 65 active+clean; 208 MiB data, 851 MiB used, 536 GiB / 536 GiB avail; 1.4 MiB/s rd, 0 op/s 2024-04-27T04:19:29.171 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:19:28 smithi181 ceph-mon[81134]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi181.dzomgh"}]: dispatch 2024-04-27T04:19:29.265 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:19:29.267+0000 7f69f8759700 1 -- 172.21.15.162:0/1489067288 --> [v2:172.21.15.162:6800/1396866108,v1:172.21.15.162:6801/1396866108] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f69f010c970 con 0x7f69dc06ec00 2024-04-27T04:19:29.267 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:19:29.268+0000 7f69e6ffd700 1 -- 172.21.15.162:0/1489067288 <== mgr.34104 v2:172.21.15.162:6800/1396866108 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f69f010c970 con 0x7f69dc06ec00 2024-04-27T04:19:29.270 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:19:29.270+0000 7f69f8759700 1 -- 172.21.15.162:0/1489067288 >> [v2:172.21.15.162:6800/1396866108,v1:172.21.15.162:6801/1396866108] conn(0x7f69dc06ec00 msgr2=0x7f69dc0710b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:19:29.270 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:19:29.270+0000 7f69f8759700 1 --2- 172.21.15.162:0/1489067288 >> [v2:172.21.15.162:6800/1396866108,v1:172.21.15.162:6801/1396866108] conn(0x7f69dc06ec00 0x7f69dc0710b0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f69f00721f0 tx=0x7f69e000b040 comp rx=0 tx=0).stop 2024-04-27T04:19:29.270 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:19:29.270+0000 7f69f8759700 1 -- 172.21.15.162:0/1489067288 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f69f0102de0 msgr2=0x7f69f010bb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:19:29.270 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:19:29.270+0000 7f69f8759700 1 --2- 172.21.15.162:0/1489067288 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f69f0102de0 0x7f69f010bb00 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f69e8003400 tx=0x7f69e8004d10 comp rx=0 tx=0).stop 2024-04-27T04:19:29.270 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:19:29.270+0000 7f69f8759700 1 -- 172.21.15.162:0/1489067288 shutdown_connections 2024-04-27T04:19:29.270 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:19:29.270+0000 7f69f8759700 1 --2- 172.21.15.162:0/1489067288 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f69f0102de0 0x7f69f010bb00 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:19:29.270 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:19:29.270+0000 7f69f8759700 1 --2- 172.21.15.162:0/1489067288 >> [v2:172.21.15.162:6800/1396866108,v1:172.21.15.162:6801/1396866108] conn(0x7f69dc06ec00 0x7f69dc0710b0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:19:29.270 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:19:29.270+0000 7f69f8759700 1 --2- 172.21.15.162:0/1489067288 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f69f0103780 0x7f69f010c040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:19:29.271 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:19:29.270+0000 7f69f8759700 1 -- 172.21.15.162:0/1489067288 >> 172.21.15.162:0/1489067288 conn(0x7f69f00f8990 msgr2=0x7f69f0107380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:19:29.271 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:19:29.271+0000 7f69f8759700 1 -- 172.21.15.162:0/1489067288 shutdown_connections 2024-04-27T04:19:29.271 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:19:29.271+0000 7f69f8759700 1 -- 172.21.15.162:0/1489067288 wait complete. 2024-04-27T04:19:29.280 INFO:teuthology.orchestra.run.smithi162.stdout:true 2024-04-27T04:19:29.373 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:19:29.372+0000 7fcfc437e700 1 -- 172.21.15.162:0/3757388980 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fcfbc102e00 msgr2=0x7fcfbc1031d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:19:29.373 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:19:29.372+0000 7fcfc437e700 1 --2- 172.21.15.162:0/3757388980 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fcfbc102e00 0x7fcfbc1031d0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fcfb8004940 tx=0x7fcfb8009e30 comp rx=0 tx=0).stop 2024-04-27T04:19:29.373 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:19:29.373+0000 7fcfc437e700 1 -- 172.21.15.162:0/3757388980 shutdown_connections 2024-04-27T04:19:29.373 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:19:29.373+0000 7fcfc437e700 1 --2- 172.21.15.162:0/3757388980 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcfbc1037a0 0x7fcfbc0fce10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:19:29.373 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:19:29.373+0000 7fcfc437e700 1 --2- 172.21.15.162:0/3757388980 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fcfbc102e00 0x7fcfbc1031d0 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:19:29.373 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:19:29.373+0000 7fcfc437e700 1 -- 172.21.15.162:0/3757388980 >> 172.21.15.162:0/3757388980 conn(0x7fcfbc0f89d0 msgr2=0x7fcfbc0fade0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:19:29.373 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:19:29.373+0000 7fcfc437e700 1 -- 172.21.15.162:0/3757388980 shutdown_connections 2024-04-27T04:19:29.373 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:19:29.373+0000 7fcfc437e700 1 -- 172.21.15.162:0/3757388980 wait complete. 2024-04-27T04:19:29.373 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:19:29.373+0000 7fcfc437e700 1 Processor -- start 2024-04-27T04:19:29.374 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:19:29.373+0000 7fcfc437e700 1 -- start start 2024-04-27T04:19:29.374 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:19:29.373+0000 7fcfc437e700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fcfbc102e00 0x7fcfbc1138a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:19:29.374 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:19:29.373+0000 7fcfc437e700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcfbc1037a0 0x7fcfbc113de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:19:29.374 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:19:29.373+0000 7fcfc437e700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcfbc114430 con 0x7fcfbc102e00 2024-04-27T04:19:29.374 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:19:29.373+0000 7fcfc437e700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcfbc10da30 con 0x7fcfbc1037a0 2024-04-27T04:19:29.374 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:19:29.373+0000 7fcfc211a700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fcfbc102e00 0x7fcfbc1138a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:19:29.374 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:19:29.374+0000 7fcfc211a700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fcfbc102e00 0x7fcfbc1138a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:56266/0 (socket says 172.21.15.162:56266) 2024-04-27T04:19:29.374 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:19:29.374+0000 7fcfc211a700 1 -- 172.21.15.162:0/3389054107 learned_addr learned my addr 172.21.15.162:0/3389054107 (peer_addr_for_me v2:172.21.15.162:0/0) 2024-04-27T04:19:29.374 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:19:29.374+0000 7fcfc1919700 1 --2- 172.21.15.162:0/3389054107 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcfbc1037a0 0x7fcfbc113de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:19:29.374 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:19:29.374+0000 7fcfc211a700 1 -- 172.21.15.162:0/3389054107 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcfbc1037a0 msgr2=0x7fcfbc113de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:19:29.374 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:19:29.374+0000 7fcfc211a700 1 --2- 172.21.15.162:0/3389054107 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcfbc1037a0 0x7fcfbc113de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:19:29.374 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:19:29.374+0000 7fcfc211a700 1 -- 172.21.15.162:0/3389054107 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcfb8005040 con 0x7fcfbc102e00 2024-04-27T04:19:29.374 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:19:29.374+0000 7fcfc211a700 1 --2- 172.21.15.162:0/3389054107 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fcfbc102e00 0x7fcfbc1138a0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fcfb8009b30 tx=0x7fcfb8009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:19:29.375 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:19:29.374+0000 7fcfb2ffd700 1 -- 172.21.15.162:0/3389054107 <== mon.0 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fcfb8021070 con 0x7fcfbc102e00 2024-04-27T04:19:29.375 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:19:29.374+0000 7fcfb2ffd700 1 -- 172.21.15.162:0/3389054107 <== mon.0 v2:172.21.15.162:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fcfb8013c00 con 0x7fcfbc102e00 2024-04-27T04:19:29.375 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:19:29.374+0000 7fcfc437e700 1 -- 172.21.15.162:0/3389054107 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcfbc10dcb0 con 0x7fcfbc102e00 2024-04-27T04:19:29.378 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:19:29.375+0000 7fcfb2ffd700 1 -- 172.21.15.162:0/3389054107 <== mon.0 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fcfb800e070 con 0x7fcfbc102e00 2024-04-27T04:19:29.379 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:19:29.375+0000 7fcfc437e700 1 -- 172.21.15.162:0/3389054107 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcfbc10e190 con 0x7fcfbc102e00 2024-04-27T04:19:29.379 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:19:29.376+0000 7fcfc437e700 1 -- 172.21.15.162:0/3389054107 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcfbc100110 con 0x7fcfbc102e00 2024-04-27T04:19:29.379 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:19:29.376+0000 7fcfb2ffd700 1 -- 172.21.15.162:0/3389054107 <== mon.0 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 39) v1 ==== 92747+0+0 (secure 0 0 0) 0x7fcfb801cad0 con 0x7fcfbc102e00 2024-04-27T04:19:29.379 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:19:29.376+0000 7fcfb2ffd700 1 --2- 172.21.15.162:0/3389054107 >> [v2:172.21.15.162:6800/1396866108,v1:172.21.15.162:6801/1396866108] conn(0x7fcfa806ec00 0x7fcfa80710b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:19:29.379 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:19:29.376+0000 7fcfb2ffd700 1 -- 172.21.15.162:0/3389054107 <== mon.0 v2:172.21.15.162:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6496+0+0 (secure 0 0 0) 0x7fcfb808fbb0 con 0x7fcfbc102e00 2024-04-27T04:19:29.379 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:19:29.376+0000 7fcfc1919700 1 --2- 172.21.15.162:0/3389054107 >> [v2:172.21.15.162:6800/1396866108,v1:172.21.15.162:6801/1396866108] conn(0x7fcfa806ec00 0x7fcfa80710b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:19:29.379 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:19:29.377+0000 7fcfc1919700 1 --2- 172.21.15.162:0/3389054107 >> [v2:172.21.15.162:6800/1396866108,v1:172.21.15.162:6801/1396866108] conn(0x7fcfa806ec00 0x7fcfa80710b0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fcfbc10ebf0 tx=0x7fcfac00b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:19:29.380 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:19:29.382+0000 7fcfb2ffd700 1 -- 172.21.15.162:0/3389054107 <== mon.0 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7fcfb8096080 con 0x7fcfbc102e00 2024-04-27T04:19:29.603 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:19:29.604+0000 7fcfc437e700 1 -- 172.21.15.162:0/3389054107 --> [v2:172.21.15.162:6800/1396866108,v1:172.21.15.162:6801/1396866108] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fcfbc10ef60 con 0x7fcfa806ec00 2024-04-27T04:19:29.604 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:19:29.606+0000 7fcfb2ffd700 1 -- 172.21.15.162:0/3389054107 <== mgr.34104 v2:172.21.15.162:6800/1396866108 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7fcfbc10ef60 con 0x7fcfa806ec00 2024-04-27T04:19:29.607 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:19:29.608+0000 7fcfc437e700 1 -- 172.21.15.162:0/3389054107 >> [v2:172.21.15.162:6800/1396866108,v1:172.21.15.162:6801/1396866108] conn(0x7fcfa806ec00 msgr2=0x7fcfa80710b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:19:29.607 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:19:29.608+0000 7fcfc437e700 1 --2- 172.21.15.162:0/3389054107 >> [v2:172.21.15.162:6800/1396866108,v1:172.21.15.162:6801/1396866108] conn(0x7fcfa806ec00 0x7fcfa80710b0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fcfbc10ebf0 tx=0x7fcfac00b040 comp rx=0 tx=0).stop 2024-04-27T04:19:29.607 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:19:29.608+0000 7fcfc437e700 1 -- 172.21.15.162:0/3389054107 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fcfbc102e00 msgr2=0x7fcfbc1138a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:19:29.607 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:19:29.608+0000 7fcfc437e700 1 --2- 172.21.15.162:0/3389054107 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fcfbc102e00 0x7fcfbc1138a0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fcfb8009b30 tx=0x7fcfb8009e30 comp rx=0 tx=0).stop 2024-04-27T04:19:29.607 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:19:29.608+0000 7fcfc437e700 1 -- 172.21.15.162:0/3389054107 shutdown_connections 2024-04-27T04:19:29.607 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:19:29.608+0000 7fcfc437e700 1 --2- 172.21.15.162:0/3389054107 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fcfbc102e00 0x7fcfbc1138a0 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:19:29.608 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:19:29.608+0000 7fcfc437e700 1 --2- 172.21.15.162:0/3389054107 >> [v2:172.21.15.162:6800/1396866108,v1:172.21.15.162:6801/1396866108] conn(0x7fcfa806ec00 0x7fcfa80710b0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:19:29.608 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:19:29.608+0000 7fcfc437e700 1 --2- 172.21.15.162:0/3389054107 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcfbc1037a0 0x7fcfbc113de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:19:29.608 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:19:29.608+0000 7fcfc437e700 1 -- 172.21.15.162:0/3389054107 >> 172.21.15.162:0/3389054107 conn(0x7fcfbc0f89d0 msgr2=0x7fcfbc0fa6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:19:29.608 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:19:29.608+0000 7fcfc437e700 1 -- 172.21.15.162:0/3389054107 shutdown_connections 2024-04-27T04:19:29.608 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:19:29.608+0000 7fcfc437e700 1 -- 172.21.15.162:0/3389054107 wait complete. 2024-04-27T04:19:29.711 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:19:29.710+0000 7f505d95b700 1 -- 172.21.15.162:0/3597086405 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5058102dc0 msgr2=0x7f5058103190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:19:29.711 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:19:29.710+0000 7f505d95b700 1 --2- 172.21.15.162:0/3597086405 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5058102dc0 0x7f5058103190 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f5048004940 tx=0x7f5048009e30 comp rx=0 tx=0).stop 2024-04-27T04:19:29.711 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:19:29.710+0000 7f505d95b700 1 -- 172.21.15.162:0/3597086405 shutdown_connections 2024-04-27T04:19:29.711 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:19:29.710+0000 7f505d95b700 1 --2- 172.21.15.162:0/3597086405 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5058103760 0x7f50580fcdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:19:29.711 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:19:29.710+0000 7f505d95b700 1 --2- 172.21.15.162:0/3597086405 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5058102dc0 0x7f5058103190 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:19:29.711 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:19:29.710+0000 7f505d95b700 1 -- 172.21.15.162:0/3597086405 >> 172.21.15.162:0/3597086405 conn(0x7f50580f8930 msgr2=0x7f50580fad40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:19:29.711 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:19:29.710+0000 7f505d95b700 1 -- 172.21.15.162:0/3597086405 shutdown_connections 2024-04-27T04:19:29.711 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:19:29.711+0000 7f505d95b700 1 -- 172.21.15.162:0/3597086405 wait complete. 2024-04-27T04:19:29.712 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:19:29.711+0000 7f505d95b700 1 Processor -- start 2024-04-27T04:19:29.712 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:19:29.711+0000 7f505d95b700 1 -- start start 2024-04-27T04:19:29.712 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:19:29.711+0000 7f505d95b700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5058102dc0 0x7f5058070260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:19:29.712 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:19:29.711+0000 7f505d95b700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5058103760 0x7f50580707a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:19:29.712 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:19:29.711+0000 7f505d95b700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50580721b0 con 0x7f5058103760 2024-04-27T04:19:29.712 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:19:29.711+0000 7f505d95b700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5058070ce0 con 0x7f5058102dc0 2024-04-27T04:19:29.712 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:19:29.711+0000 7f50567fc700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5058103760 0x7f50580707a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:19:29.712 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:19:29.711+0000 7f50567fc700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5058103760 0x7f50580707a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:56280/0 (socket says 172.21.15.162:56280) 2024-04-27T04:19:29.712 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:19:29.712+0000 7f50567fc700 1 -- 172.21.15.162:0/3654459330 learned_addr learned my addr 172.21.15.162:0/3654459330 (peer_addr_for_me v2:172.21.15.162:0/0) 2024-04-27T04:19:29.712 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:19:29.712+0000 7f5056ffd700 1 --2- 172.21.15.162:0/3654459330 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5058102dc0 0x7f5058070260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:19:29.712 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:19:29.712+0000 7f50567fc700 1 -- 172.21.15.162:0/3654459330 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5058102dc0 msgr2=0x7f5058070260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:19:29.712 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:19:29.712+0000 7f50567fc700 1 --2- 172.21.15.162:0/3654459330 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5058102dc0 0x7f5058070260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:19:29.712 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:19:29.712+0000 7f50567fc700 1 -- 172.21.15.162:0/3654459330 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5048005040 con 0x7f5058103760 2024-04-27T04:19:29.713 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:19:29.712+0000 7f50567fc700 1 --2- 172.21.15.162:0/3654459330 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5058103760 0x7f50580707a0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f504c00eb40 tx=0x7f504c00b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:19:29.713 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:19:29.712+0000 7f503f7fe700 1 -- 172.21.15.162:0/3654459330 <== mon.0 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f504c007cc0 con 0x7f5058103760 2024-04-27T04:19:29.713 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:19:29.712+0000 7f503f7fe700 1 -- 172.21.15.162:0/3654459330 <== mon.0 v2:172.21.15.162:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f504c00f040 con 0x7f5058103760 2024-04-27T04:19:29.713 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:19:29.712+0000 7f505d95b700 1 -- 172.21.15.162:0/3654459330 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5058070f60 con 0x7f5058103760 2024-04-27T04:19:29.713 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:19:29.712+0000 7f503f7fe700 1 -- 172.21.15.162:0/3654459330 <== mon.0 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f504c00c9d0 con 0x7f5058103760 2024-04-27T04:19:29.713 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:19:29.712+0000 7f505d95b700 1 -- 172.21.15.162:0/3654459330 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5058119fb0 con 0x7f5058103760 2024-04-27T04:19:29.714 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:19:29.713+0000 7f505d95b700 1 -- 172.21.15.162:0/3654459330 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f50580fe510 con 0x7f5058103760 2024-04-27T04:19:29.714 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:19:29.714+0000 7f503f7fe700 1 -- 172.21.15.162:0/3654459330 <== mon.0 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 39) v1 ==== 92747+0+0 (secure 0 0 0) 0x7f504c00cc40 con 0x7f5058103760 2024-04-27T04:19:29.714 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:19:29.714+0000 7f503f7fe700 1 --2- 172.21.15.162:0/3654459330 >> [v2:172.21.15.162:6800/1396866108,v1:172.21.15.162:6801/1396866108] conn(0x7f504006ec00 0x7f50400710b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:19:29.714 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:19:29.714+0000 7f503f7fe700 1 -- 172.21.15.162:0/3654459330 <== mon.0 v2:172.21.15.162:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6496+0+0 (secure 0 0 0) 0x7f504c08e2f0 con 0x7f5058103760 2024-04-27T04:19:29.714 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:19:29.714+0000 7f5056ffd700 1 --2- 172.21.15.162:0/3654459330 >> [v2:172.21.15.162:6800/1396866108,v1:172.21.15.162:6801/1396866108] conn(0x7f504006ec00 0x7f50400710b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:19:29.714 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:19:29.714+0000 7f5056ffd700 1 --2- 172.21.15.162:0/3654459330 >> [v2:172.21.15.162:6800/1396866108,v1:172.21.15.162:6801/1396866108] conn(0x7f504006ec00 0x7f50400710b0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f5048007f50 tx=0x7f5048009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:19:29.717 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:19:29.719+0000 7f503f7fe700 1 -- 172.21.15.162:0/3654459330 <== mon.0 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7f504c057150 con 0x7f5058103760 2024-04-27T04:19:29.927 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:19:29.929+0000 7f505d95b700 1 -- 172.21.15.162:0/3654459330 --> [v2:172.21.15.162:6800/1396866108,v1:172.21.15.162:6801/1396866108] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f50580f9f30 con 0x7f504006ec00 2024-04-27T04:19:29.937 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:19:29.939+0000 7f503f7fe700 1 -- 172.21.15.162:0/3654459330 <== mgr.34104 v2:172.21.15.162:6800/1396866108 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3498 (secure 0 0 0) 0x7f50580f9f30 con 0x7f504006ec00 2024-04-27T04:19:29.938 INFO:teuthology.orchestra.run.smithi162.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-27T04:19:29.938 INFO:teuthology.orchestra.run.smithi162.stdout:alertmanager.smithi162 smithi162 *:9093,9094 running (6m) 11s ago 17m 27.0M - 0.25.0 c8568f914cd2 40e334659017 2024-04-27T04:19:29.938 INFO:teuthology.orchestra.run.smithi162.stdout:crash.smithi162 smithi162 running (4m) 11s ago 18m 6660k - 19.0.0-3253-g4e213a58 67e414ff4bc4 1aa8f4d5a28b 2024-04-27T04:19:29.938 INFO:teuthology.orchestra.run.smithi162.stdout:crash.smithi181 smithi181 running (4m) 51s ago 17m 6656k - 19.0.0-3253-g4e213a58 67e414ff4bc4 832d9cbc5c30 2024-04-27T04:19:29.938 INFO:teuthology.orchestra.run.smithi162.stdout:grafana.smithi162 smithi162 *:3000 running (6m) 11s ago 17m 79.3M - 9.4.12 2bacad6d85d8 3d78c478f03f 2024-04-27T04:19:29.938 INFO:teuthology.orchestra.run.smithi162.stdout:mds.cephfs.smithi162.gzlyjj smithi162 running (36s) 11s ago 12m 12.5M - 19.0.0-3253-g4e213a58 67e414ff4bc4 7541e8fb8265 2024-04-27T04:19:29.938 INFO:teuthology.orchestra.run.smithi162.stdout:mds.cephfs.smithi162.xcnapk smithi162 running (16s) 11s ago 12m 11.5M - 19.0.0-3253-g4e213a58 67e414ff4bc4 3af4597febc8 2024-04-27T04:19:29.938 INFO:teuthology.orchestra.run.smithi162.stdout:mds.cephfs.smithi181.bkkcdm smithi181 running (12m) 51s ago 12m 16.1M - 17.2.7-907-g2be3efa9 1c5834633715 af55afca1263 2024-04-27T04:19:29.938 INFO:teuthology.orchestra.run.smithi162.stdout:mds.cephfs.smithi181.dzomgh smithi181 running (12m) 51s ago 12m 143M - 17.2.7-907-g2be3efa9 1c5834633715 6733296597f3 2024-04-27T04:19:29.938 INFO:teuthology.orchestra.run.smithi162.stdout:mgr.smithi162.oennfn smithi162 *:8443,9283,8765 running (8m) 11s ago 19m 511M - 19.0.0-3253-g4e213a58 67e414ff4bc4 58a748d3637b 2024-04-27T04:19:29.938 INFO:teuthology.orchestra.run.smithi162.stdout:mgr.smithi181.vtwuhz smithi181 *:8443,9283,8765 running (7m) 51s ago 16m 452M - 19.0.0-3253-g4e213a58 67e414ff4bc4 813584297b1d 2024-04-27T04:19:29.938 INFO:teuthology.orchestra.run.smithi162.stdout:mon.smithi162 smithi162 running (5m) 11s ago 19m 51.9M 2048M 19.0.0-3253-g4e213a58 67e414ff4bc4 375adaf4468f 2024-04-27T04:19:29.938 INFO:teuthology.orchestra.run.smithi162.stdout:mon.smithi181 smithi181 running (4m) 51s ago 16m 36.1M 2048M 19.0.0-3253-g4e213a58 67e414ff4bc4 77b71f6d522f 2024-04-27T04:19:29.939 INFO:teuthology.orchestra.run.smithi162.stdout:node-exporter.smithi162 smithi162 *:9100 running (7m) 11s ago 17m 15.5M - 1.5.0 0da6a335fe13 5c4c5ea9c1d7 2024-04-27T04:19:29.939 INFO:teuthology.orchestra.run.smithi162.stdout:node-exporter.smithi181 smithi181 *:9100 running (7m) 51s ago 16m 15.5M - 1.5.0 0da6a335fe13 9067f14c0519 2024-04-27T04:19:29.939 INFO:teuthology.orchestra.run.smithi162.stdout:osd.0 smithi162 running (3m) 11s ago 16m 93.1M 1557M 19.0.0-3253-g4e213a58 67e414ff4bc4 2744aff3cd88 2024-04-27T04:19:29.939 INFO:teuthology.orchestra.run.smithi162.stdout:osd.1 smithi162 running (3m) 11s ago 15m 96.6M 1557M 19.0.0-3253-g4e213a58 67e414ff4bc4 668e16faad62 2024-04-27T04:19:29.939 INFO:teuthology.orchestra.run.smithi162.stdout:osd.2 smithi162 running (2m) 11s ago 15m 87.4M 1557M 19.0.0-3253-g4e213a58 67e414ff4bc4 d24878d461e7 2024-04-27T04:19:29.939 INFO:teuthology.orchestra.run.smithi162.stdout:osd.3 smithi181 running (2m) 51s ago 14m 112M 2581M 19.0.0-3253-g4e213a58 67e414ff4bc4 ab473cfd54d2 2024-04-27T04:19:29.939 INFO:teuthology.orchestra.run.smithi162.stdout:osd.4 smithi181 running (88s) 51s ago 14m 98.6M 2581M 19.0.0-3253-g4e213a58 67e414ff4bc4 de3e10f2d26d 2024-04-27T04:19:29.939 INFO:teuthology.orchestra.run.smithi162.stdout:osd.5 smithi181 running (55s) 51s ago 13m 11.8M 2581M 19.0.0-3253-g4e213a58 67e414ff4bc4 7dd4f287ef2e 2024-04-27T04:19:29.939 INFO:teuthology.orchestra.run.smithi162.stdout:prometheus.smithi162 smithi162 *:9095 running (6m) 11s ago 17m 67.7M - 2.43.0 a07b618ecd1d f30657a43324 2024-04-27T04:19:29.940 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:19:29.941+0000 7f505d95b700 1 -- 172.21.15.162:0/3654459330 >> [v2:172.21.15.162:6800/1396866108,v1:172.21.15.162:6801/1396866108] conn(0x7f504006ec00 msgr2=0x7f50400710b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:19:29.940 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:19:29.941+0000 7f505d95b700 1 --2- 172.21.15.162:0/3654459330 >> [v2:172.21.15.162:6800/1396866108,v1:172.21.15.162:6801/1396866108] conn(0x7f504006ec00 0x7f50400710b0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f5048007f50 tx=0x7f5048009e30 comp rx=0 tx=0).stop 2024-04-27T04:19:29.940 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:19:29.941+0000 7f505d95b700 1 -- 172.21.15.162:0/3654459330 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5058103760 msgr2=0x7f50580707a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:19:29.940 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:19:29.941+0000 7f505d95b700 1 --2- 172.21.15.162:0/3654459330 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5058103760 0x7f50580707a0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f504c00eb40 tx=0x7f504c00b040 comp rx=0 tx=0).stop 2024-04-27T04:19:29.940 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:19:29.941+0000 7f505d95b700 1 -- 172.21.15.162:0/3654459330 shutdown_connections 2024-04-27T04:19:29.940 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:19:29.941+0000 7f505d95b700 1 --2- 172.21.15.162:0/3654459330 >> [v2:172.21.15.162:6800/1396866108,v1:172.21.15.162:6801/1396866108] conn(0x7f504006ec00 0x7f50400710b0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:19:29.940 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:19:29.941+0000 7f505d95b700 1 --2- 172.21.15.162:0/3654459330 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5058102dc0 0x7f5058070260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:19:29.940 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:19:29.941+0000 7f505d95b700 1 --2- 172.21.15.162:0/3654459330 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5058103760 0x7f50580707a0 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:19:29.940 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:19:29.941+0000 7f505d95b700 1 -- 172.21.15.162:0/3654459330 >> 172.21.15.162:0/3654459330 conn(0x7f50580f8930 msgr2=0x7f50580f9820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:19:29.940 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:19:29.941+0000 7f505d95b700 1 -- 172.21.15.162:0/3654459330 shutdown_connections 2024-04-27T04:19:29.941 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:19:29.941+0000 7f505d95b700 1 -- 172.21.15.162:0/3654459330 wait complete. 2024-04-27T04:19:30.043 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:19:30.042+0000 7ff7f7c75700 1 -- 172.21.15.162:0/3386394146 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff7f00fc790 msgr2=0x7ff7f00fcb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:19:30.043 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:19:30.042+0000 7ff7f7c75700 1 --2- 172.21.15.162:0/3386394146 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff7f00fc790 0x7ff7f00fcb60 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7ff7ec004940 tx=0x7ff7ec009e30 comp rx=0 tx=0).stop 2024-04-27T04:19:30.043 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:19:30.042+0000 7ff7f7c75700 1 -- 172.21.15.162:0/3386394146 shutdown_connections 2024-04-27T04:19:30.043 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:19:30.042+0000 7ff7f7c75700 1 --2- 172.21.15.162:0/3386394146 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff7f00fd0a0 0x7ff7f0109350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:19:30.043 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:19:30.042+0000 7ff7f7c75700 1 --2- 172.21.15.162:0/3386394146 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff7f00fc790 0x7ff7f00fcb60 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:19:30.044 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:19:30.042+0000 7ff7f7c75700 1 -- 172.21.15.162:0/3386394146 >> 172.21.15.162:0/3386394146 conn(0x7ff7f0071d00 msgr2=0x7ff7f0072100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:19:30.044 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:19:30.042+0000 7ff7f7c75700 1 -- 172.21.15.162:0/3386394146 shutdown_connections 2024-04-27T04:19:30.044 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:19:30.042+0000 7ff7f7c75700 1 -- 172.21.15.162:0/3386394146 wait complete. 2024-04-27T04:19:30.044 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:19:30.043+0000 7ff7f7c75700 1 Processor -- start 2024-04-27T04:19:30.044 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:19:30.043+0000 7ff7f7c75700 1 -- start start 2024-04-27T04:19:30.044 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:19:30.043+0000 7ff7f7c75700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff7f00fc790 0x7ff7f0115d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:19:30.044 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:19:30.043+0000 7ff7f7c75700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff7f00fd0a0 0x7ff7f01162b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:19:30.044 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:19:30.043+0000 7ff7f7c75700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7f0116900 con 0x7ff7f00fd0a0 2024-04-27T04:19:30.044 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:19:30.043+0000 7ff7f7c75700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7f010fcd0 con 0x7ff7f00fc790 2024-04-27T04:19:30.044 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:19:30.043+0000 7ff7f5210700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff7f00fd0a0 0x7ff7f01162b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:19:30.044 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:19:30.043+0000 7ff7f5210700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff7f00fd0a0 0x7ff7f01162b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:56298/0 (socket says 172.21.15.162:56298) 2024-04-27T04:19:30.045 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:19:30.043+0000 7ff7f5210700 1 -- 172.21.15.162:0/3126834651 learned_addr learned my addr 172.21.15.162:0/3126834651 (peer_addr_for_me v2:172.21.15.162:0/0) 2024-04-27T04:19:30.045 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:19:30.043+0000 7ff7f5a11700 1 --2- 172.21.15.162:0/3126834651 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff7f00fc790 0x7ff7f0115d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:19:30.045 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:19:30.044+0000 7ff7f5210700 1 -- 172.21.15.162:0/3126834651 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff7f00fc790 msgr2=0x7ff7f0115d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:19:30.045 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:19:30.044+0000 7ff7f5210700 1 --2- 172.21.15.162:0/3126834651 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff7f00fc790 0x7ff7f0115d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:19:30.045 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:19:30.044+0000 7ff7f5210700 1 -- 172.21.15.162:0/3126834651 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7ec005040 con 0x7ff7f00fd0a0 2024-04-27T04:19:30.045 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:19:30.044+0000 7ff7f5a11700 1 --2- 172.21.15.162:0/3126834651 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff7f00fc790 0x7ff7f0115d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-27T04:19:30.045 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:19:30.044+0000 7ff7f5210700 1 --2- 172.21.15.162:0/3126834651 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff7f00fd0a0 0x7ff7f01162b0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7ff7e000eb40 tx=0x7ff7e000b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:19:30.045 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:19:30.044+0000 7ff7e67fc700 1 -- 172.21.15.162:0/3126834651 <== mon.0 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff7e0007cc0 con 0x7ff7f00fd0a0 2024-04-27T04:19:30.045 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:19:30.044+0000 7ff7e67fc700 1 -- 172.21.15.162:0/3126834651 <== mon.0 v2:172.21.15.162:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7ff7e000f040 con 0x7ff7f00fd0a0 2024-04-27T04:19:30.045 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:19:30.044+0000 7ff7e67fc700 1 -- 172.21.15.162:0/3126834651 <== mon.0 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff7e000ca10 con 0x7ff7f00fd0a0 2024-04-27T04:19:30.045 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:19:30.044+0000 7ff7f7c75700 1 -- 172.21.15.162:0/3126834651 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff7f010ff50 con 0x7ff7f00fd0a0 2024-04-27T04:19:30.046 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:19:30.044+0000 7ff7f7c75700 1 -- 172.21.15.162:0/3126834651 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff7f0110490 con 0x7ff7f00fd0a0 2024-04-27T04:19:30.047 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:19:30.045+0000 7ff7f7c75700 1 -- 172.21.15.162:0/3126834651 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff7f00fc330 con 0x7ff7f00fd0a0 2024-04-27T04:19:30.047 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:19:30.046+0000 7ff7e67fc700 1 -- 172.21.15.162:0/3126834651 <== mon.0 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 39) v1 ==== 92747+0+0 (secure 0 0 0) 0x7ff7e000cb70 con 0x7ff7f00fd0a0 2024-04-27T04:19:30.047 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:19:30.046+0000 7ff7e67fc700 1 --2- 172.21.15.162:0/3126834651 >> [v2:172.21.15.162:6800/1396866108,v1:172.21.15.162:6801/1396866108] conn(0x7ff7dc06ec10 0x7ff7dc0710c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:19:30.047 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:19:30.046+0000 7ff7e67fc700 1 -- 172.21.15.162:0/3126834651 <== mon.0 v2:172.21.15.162:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6496+0+0 (secure 0 0 0) 0x7ff7e008e360 con 0x7ff7f00fd0a0 2024-04-27T04:19:30.047 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:19:30.046+0000 7ff7f5a11700 1 --2- 172.21.15.162:0/3126834651 >> [v2:172.21.15.162:6800/1396866108,v1:172.21.15.162:6801/1396866108] conn(0x7ff7dc06ec10 0x7ff7dc0710c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:19:30.047 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:19:30.047+0000 7ff7f5a11700 1 --2- 172.21.15.162:0/3126834651 >> [v2:172.21.15.162:6800/1396866108,v1:172.21.15.162:6801/1396866108] conn(0x7ff7dc06ec10 0x7ff7dc0710c0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7ff7ec009000 tx=0x7ff7ec009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:19:30.050 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:19:30.051+0000 7ff7e67fc700 1 -- 172.21.15.162:0/3126834651 <== mon.0 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7ff7e005ad70 con 0x7ff7f00fd0a0 2024-04-27T04:19:30.347 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:19:30.348+0000 7ff7f7c75700 1 -- 172.21.15.162:0/3126834651 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7ff7f0111420 con 0x7ff7f00fd0a0 2024-04-27T04:19:30.347 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:19:30.349+0000 7ff7e67fc700 1 -- 172.21.15.162:0/3126834651 <== mon.0 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+837 (secure 0 0 0) 0x7ff7e005a900 con 0x7ff7f00fd0a0 2024-04-27T04:19:30.348 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-04-27T04:19:30.348 INFO:teuthology.orchestra.run.smithi162.stdout: "mon": { 2024-04-27T04:19:30.348 INFO:teuthology.orchestra.run.smithi162.stdout: "ceph version 19.0.0-3253-g4e213a58 (4e213a58e787437ff508e6c890ccd5fd32dc96b5) squid (dev)": 2 2024-04-27T04:19:30.348 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-04-27T04:19:30.348 INFO:teuthology.orchestra.run.smithi162.stdout: "mgr": { 2024-04-27T04:19:30.348 INFO:teuthology.orchestra.run.smithi162.stdout: "ceph version 19.0.0-3253-g4e213a58 (4e213a58e787437ff508e6c890ccd5fd32dc96b5) squid (dev)": 2 2024-04-27T04:19:30.348 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-04-27T04:19:30.348 INFO:teuthology.orchestra.run.smithi162.stdout: "osd": { 2024-04-27T04:19:30.348 INFO:teuthology.orchestra.run.smithi162.stdout: "ceph version 19.0.0-3253-g4e213a58 (4e213a58e787437ff508e6c890ccd5fd32dc96b5) squid (dev)": 6 2024-04-27T04:19:30.348 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-04-27T04:19:30.348 INFO:teuthology.orchestra.run.smithi162.stdout: "mds": { 2024-04-27T04:19:30.348 INFO:teuthology.orchestra.run.smithi162.stdout: "ceph version 17.2.7-907-g2be3efa9 (2be3efa97a7972f8e925bd899b474329979b74d6) quincy (stable)": 1, 2024-04-27T04:19:30.348 INFO:teuthology.orchestra.run.smithi162.stdout: "ceph version 19.0.0-3253-g4e213a58 (4e213a58e787437ff508e6c890ccd5fd32dc96b5) squid (dev)": 2 2024-04-27T04:19:30.348 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-04-27T04:19:30.349 INFO:teuthology.orchestra.run.smithi162.stdout: "overall": { 2024-04-27T04:19:30.349 INFO:teuthology.orchestra.run.smithi162.stdout: "ceph version 17.2.7-907-g2be3efa9 (2be3efa97a7972f8e925bd899b474329979b74d6) quincy (stable)": 1, 2024-04-27T04:19:30.349 INFO:teuthology.orchestra.run.smithi162.stdout: "ceph version 19.0.0-3253-g4e213a58 (4e213a58e787437ff508e6c890ccd5fd32dc96b5) squid (dev)": 12 2024-04-27T04:19:30.349 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-04-27T04:19:30.349 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-04-27T04:19:30.350 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:19:30.351+0000 7ff7f7c75700 1 -- 172.21.15.162:0/3126834651 >> [v2:172.21.15.162:6800/1396866108,v1:172.21.15.162:6801/1396866108] conn(0x7ff7dc06ec10 msgr2=0x7ff7dc0710c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:19:30.350 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:19:30.351+0000 7ff7f7c75700 1 --2- 172.21.15.162:0/3126834651 >> [v2:172.21.15.162:6800/1396866108,v1:172.21.15.162:6801/1396866108] conn(0x7ff7dc06ec10 0x7ff7dc0710c0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7ff7ec009000 tx=0x7ff7ec009e30 comp rx=0 tx=0).stop 2024-04-27T04:19:30.350 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:19:30.351+0000 7ff7f7c75700 1 -- 172.21.15.162:0/3126834651 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff7f00fd0a0 msgr2=0x7ff7f01162b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:19:30.351 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:19:30.351+0000 7ff7f7c75700 1 --2- 172.21.15.162:0/3126834651 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff7f00fd0a0 0x7ff7f01162b0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7ff7e000eb40 tx=0x7ff7e000b040 comp rx=0 tx=0).stop 2024-04-27T04:19:30.351 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:19:30.351+0000 7ff7f7c75700 1 -- 172.21.15.162:0/3126834651 shutdown_connections 2024-04-27T04:19:30.351 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:19:30.351+0000 7ff7f7c75700 1 --2- 172.21.15.162:0/3126834651 >> [v2:172.21.15.162:6800/1396866108,v1:172.21.15.162:6801/1396866108] conn(0x7ff7dc06ec10 0x7ff7dc0710c0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:19:30.351 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:19:30.351+0000 7ff7f7c75700 1 --2- 172.21.15.162:0/3126834651 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff7f00fc790 0x7ff7f0115d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:19:30.351 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:19:30.351+0000 7ff7f7c75700 1 --2- 172.21.15.162:0/3126834651 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff7f00fd0a0 0x7ff7f01162b0 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:19:30.351 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:19:30.351+0000 7ff7f7c75700 1 -- 172.21.15.162:0/3126834651 >> 172.21.15.162:0/3126834651 conn(0x7ff7f0071d00 msgr2=0x7ff7f00f9cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:19:30.351 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:19:30.351+0000 7ff7f7c75700 1 -- 172.21.15.162:0/3126834651 shutdown_connections 2024-04-27T04:19:30.351 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:19:30.351+0000 7ff7f7c75700 1 -- 172.21.15.162:0/3126834651 wait complete. 2024-04-27T04:19:30.457 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:19:30.458+0000 7fbc95419700 1 -- 172.21.15.162:0/1678146077 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fbc900fb7f0 msgr2=0x7fbc900fbbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:19:30.457 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:19:30.458+0000 7fbc95419700 1 --2- 172.21.15.162:0/1678146077 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fbc900fb7f0 0x7fbc900fbbc0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fbc80004940 tx=0x7fbc80009e30 comp rx=0 tx=0).stop 2024-04-27T04:19:30.457 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:19:30.458+0000 7fbc95419700 1 -- 172.21.15.162:0/1678146077 shutdown_connections 2024-04-27T04:19:30.457 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:19:30.458+0000 7fbc95419700 1 --2- 172.21.15.162:0/1678146077 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbc900fc190 0x7fbc90101a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:19:30.457 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:19:30.458+0000 7fbc95419700 1 --2- 172.21.15.162:0/1678146077 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fbc900fb7f0 0x7fbc900fbbc0 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:19:30.457 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:19:30.458+0000 7fbc95419700 1 -- 172.21.15.162:0/1678146077 >> 172.21.15.162:0/1678146077 conn(0x7fbc90071460 msgr2=0x7fbc90071860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:19:30.457 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:19:30.458+0000 7fbc95419700 1 -- 172.21.15.162:0/1678146077 shutdown_connections 2024-04-27T04:19:30.457 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:19:30.458+0000 7fbc95419700 1 -- 172.21.15.162:0/1678146077 wait complete. 2024-04-27T04:19:30.459 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:19:30.459+0000 7fbc95419700 1 Processor -- start 2024-04-27T04:19:30.459 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:19:30.459+0000 7fbc95419700 1 -- start start 2024-04-27T04:19:30.459 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:19:30.459+0000 7fbc95419700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fbc900fb7f0 0x7fbc900fc780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:19:30.459 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:19:30.459+0000 7fbc95419700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbc900fc190 0x7fbc900fccc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:19:30.459 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:19:30.459+0000 7fbc95419700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc900fe6d0 con 0x7fbc900fb7f0 2024-04-27T04:19:30.459 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:19:30.459+0000 7fbc95419700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc900fd200 con 0x7fbc900fc190 2024-04-27T04:19:30.459 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:19:30.459+0000 7fbc8effd700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fbc900fb7f0 0x7fbc900fc780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:19:30.459 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:19:30.459+0000 7fbc8effd700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fbc900fb7f0 0x7fbc900fc780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:56324/0 (socket says 172.21.15.162:56324) 2024-04-27T04:19:30.459 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:19:30.459+0000 7fbc8e7fc700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbc900fc190 0x7fbc900fccc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:19:30.459 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:19:30.459+0000 7fbc8effd700 1 -- 172.21.15.162:0/3582612852 learned_addr learned my addr 172.21.15.162:0/3582612852 (peer_addr_for_me v2:172.21.15.162:0/0) 2024-04-27T04:19:30.459 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:19:30.459+0000 7fbc8effd700 1 -- 172.21.15.162:0/3582612852 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbc900fc190 msgr2=0x7fbc900fccc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:19:30.460 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:19:30.459+0000 7fbc8effd700 1 --2- 172.21.15.162:0/3582612852 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbc900fc190 0x7fbc900fccc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:19:30.460 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:19:30.459+0000 7fbc8effd700 1 -- 172.21.15.162:0/3582612852 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc80005040 con 0x7fbc900fb7f0 2024-04-27T04:19:30.460 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:19:30.460+0000 7fbc8effd700 1 --2- 172.21.15.162:0/3582612852 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fbc900fb7f0 0x7fbc900fc780 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fbc8000b8b0 tx=0x7fbc80009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:19:30.460 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:19:30.460+0000 7fbc777fe700 1 -- 172.21.15.162:0/3582612852 <== mon.0 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbc8000e040 con 0x7fbc900fb7f0 2024-04-27T04:19:30.460 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:19:30.460+0000 7fbc777fe700 1 -- 172.21.15.162:0/3582612852 <== mon.0 v2:172.21.15.162:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fbc80004290 con 0x7fbc900fb7f0 2024-04-27T04:19:30.460 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:19:30.460+0000 7fbc777fe700 1 -- 172.21.15.162:0/3582612852 <== mon.0 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbc8001c3f0 con 0x7fbc900fb7f0 2024-04-27T04:19:30.460 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:19:30.460+0000 7fbc95419700 1 -- 172.21.15.162:0/3582612852 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbc900fd480 con 0x7fbc900fb7f0 2024-04-27T04:19:30.460 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:19:30.460+0000 7fbc95419700 1 -- 172.21.15.162:0/3582612852 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbc9011a050 con 0x7fbc900fb7f0 2024-04-27T04:19:30.461 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:19:30.461+0000 7fbc95419700 1 -- 172.21.15.162:0/3582612852 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbc90065f30 con 0x7fbc900fb7f0 2024-04-27T04:19:30.461 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:19:30.461+0000 7fbc777fe700 1 -- 172.21.15.162:0/3582612852 <== mon.0 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 39) v1 ==== 92747+0+0 (secure 0 0 0) 0x7fbc80013420 con 0x7fbc900fb7f0 2024-04-27T04:19:30.461 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:19:30.462+0000 7fbc777fe700 1 --2- 172.21.15.162:0/3582612852 >> [v2:172.21.15.162:6800/1396866108,v1:172.21.15.162:6801/1396866108] conn(0x7fbc7806ec00 0x7fbc780710b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:19:30.461 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:19:30.462+0000 7fbc777fe700 1 -- 172.21.15.162:0/3582612852 <== mon.0 v2:172.21.15.162:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6496+0+0 (secure 0 0 0) 0x7fbc8008fae0 con 0x7fbc900fb7f0 2024-04-27T04:19:30.461 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:19:30.462+0000 7fbc8e7fc700 1 --2- 172.21.15.162:0/3582612852 >> [v2:172.21.15.162:6800/1396866108,v1:172.21.15.162:6801/1396866108] conn(0x7fbc7806ec00 0x7fbc780710b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:19:30.461 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:19:30.462+0000 7fbc8e7fc700 1 --2- 172.21.15.162:0/3582612852 >> [v2:172.21.15.162:6800/1396866108,v1:172.21.15.162:6801/1396866108] conn(0x7fbc7806ec00 0x7fbc780710b0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fbc900fdeb0 tx=0x7fbc8400b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:19:30.466 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:19:30.467+0000 7fbc777fe700 1 -- 172.21.15.162:0/3582612852 <== mon.0 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7fbc80058910 con 0x7fbc900fb7f0 2024-04-27T04:19:30.718 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:19:30.719+0000 7fbc95419700 1 -- 172.21.15.162:0/3582612852 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7fbc9011a240 con 0x7fbc900fb7f0 2024-04-27T04:19:30.718 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:19:30.719+0000 7fbc777fe700 1 -- 172.21.15.162:0/3582612852 <== mon.0 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 23 v23) v1 ==== 76+0+1708 (secure 0 0 0) 0x7fbc8005bfd0 con 0x7fbc900fb7f0 2024-04-27T04:19:30.718 INFO:teuthology.orchestra.run.smithi162.stdout:e23 2024-04-27T04:19:30.719 INFO:teuthology.orchestra.run.smithi162.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-04-27T04:19:30.719 INFO:teuthology.orchestra.run.smithi162.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-27T04:19:30.719 INFO:teuthology.orchestra.run.smithi162.stdout:legacy client fscid: 1 2024-04-27T04:19:30.719 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-04-27T04:19:30.719 INFO:teuthology.orchestra.run.smithi162.stdout:Filesystem 'cephfs' (1) 2024-04-27T04:19:30.719 INFO:teuthology.orchestra.run.smithi162.stdout:fs_name cephfs 2024-04-27T04:19:30.719 INFO:teuthology.orchestra.run.smithi162.stdout:epoch 23 2024-04-27T04:19:30.719 INFO:teuthology.orchestra.run.smithi162.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2024-04-27T04:19:30.719 INFO:teuthology.orchestra.run.smithi162.stdout:created 2024-04-27T04:06:53.095749+0000 2024-04-27T04:19:30.719 INFO:teuthology.orchestra.run.smithi162.stdout:modified 2024-04-27T04:19:26.420268+0000 2024-04-27T04:19:30.719 INFO:teuthology.orchestra.run.smithi162.stdout:tableserver 0 2024-04-27T04:19:30.719 INFO:teuthology.orchestra.run.smithi162.stdout:root 0 2024-04-27T04:19:30.719 INFO:teuthology.orchestra.run.smithi162.stdout:session_timeout 60 2024-04-27T04:19:30.719 INFO:teuthology.orchestra.run.smithi162.stdout:session_autoclose 300 2024-04-27T04:19:30.720 INFO:teuthology.orchestra.run.smithi162.stdout:max_file_size 1099511627776 2024-04-27T04:19:30.720 INFO:teuthology.orchestra.run.smithi162.stdout:max_xattr_size 65536 2024-04-27T04:19:30.720 INFO:teuthology.orchestra.run.smithi162.stdout:required_client_features {} 2024-04-27T04:19:30.720 INFO:teuthology.orchestra.run.smithi162.stdout:last_failure 0 2024-04-27T04:19:30.720 INFO:teuthology.orchestra.run.smithi162.stdout:last_failure_osd_epoch 83 2024-04-27T04:19:30.720 INFO:teuthology.orchestra.run.smithi162.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-27T04:19:30.720 INFO:teuthology.orchestra.run.smithi162.stdout:max_mds 1 2024-04-27T04:19:30.720 INFO:teuthology.orchestra.run.smithi162.stdout:in 0 2024-04-27T04:19:30.720 INFO:teuthology.orchestra.run.smithi162.stdout:up {0=24287} 2024-04-27T04:19:30.720 INFO:teuthology.orchestra.run.smithi162.stdout:failed 2024-04-27T04:19:30.720 INFO:teuthology.orchestra.run.smithi162.stdout:damaged 2024-04-27T04:19:30.720 INFO:teuthology.orchestra.run.smithi162.stdout:stopped 2024-04-27T04:19:30.720 INFO:teuthology.orchestra.run.smithi162.stdout:data_pools [3] 2024-04-27T04:19:30.720 INFO:teuthology.orchestra.run.smithi162.stdout:metadata_pool 2 2024-04-27T04:19:30.720 INFO:teuthology.orchestra.run.smithi162.stdout:inline_data disabled 2024-04-27T04:19:30.721 INFO:teuthology.orchestra.run.smithi162.stdout:balancer 2024-04-27T04:19:30.721 INFO:teuthology.orchestra.run.smithi162.stdout:bal_rank_mask -1 2024-04-27T04:19:30.721 INFO:teuthology.orchestra.run.smithi162.stdout:standby_count_wanted 1 2024-04-27T04:19:30.721 INFO:teuthology.orchestra.run.smithi162.stdout:qdb_cluster leader: 0 members: 2024-04-27T04:19:30.721 INFO:teuthology.orchestra.run.smithi162.stdout:[mds.cephfs.smithi181.bkkcdm{0:24287} state up:replay seq 2 join_fscid=1 addr [v2:172.21.15.181:6826/3756475453,v1:172.21.15.181:6827/3756475453] compat {c=[1],r=[1],i=[7ff]}] 2024-04-27T04:19:30.721 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-04-27T04:19:30.721 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-04-27T04:19:30.721 INFO:teuthology.orchestra.run.smithi162.stdout:Standby daemons: 2024-04-27T04:19:30.721 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-04-27T04:19:30.721 INFO:teuthology.orchestra.run.smithi162.stdout:[mds.cephfs.smithi162.gzlyjj{-1:34380} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.162:6826/1198130667,v1:172.21.15.162:6827/1198130667] compat {c=[1],r=[1],i=[1fff]}] 2024-04-27T04:19:30.721 INFO:teuthology.orchestra.run.smithi162.stdout:[mds.cephfs.smithi162.xcnapk{-1:34412} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.162:6828/307528848,v1:172.21.15.162:6829/307528848] compat {c=[1],r=[1],i=[1fff]}] 2024-04-27T04:19:30.722 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:19:30.722+0000 7fbc95419700 1 -- 172.21.15.162:0/3582612852 >> [v2:172.21.15.162:6800/1396866108,v1:172.21.15.162:6801/1396866108] conn(0x7fbc7806ec00 msgr2=0x7fbc780710b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:19:30.722 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:19:30.722+0000 7fbc95419700 1 --2- 172.21.15.162:0/3582612852 >> [v2:172.21.15.162:6800/1396866108,v1:172.21.15.162:6801/1396866108] conn(0x7fbc7806ec00 0x7fbc780710b0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fbc900fdeb0 tx=0x7fbc8400b040 comp rx=0 tx=0).stop 2024-04-27T04:19:30.722 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:19:30.722+0000 7fbc95419700 1 -- 172.21.15.162:0/3582612852 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fbc900fb7f0 msgr2=0x7fbc900fc780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:19:30.722 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:19:30.722+0000 7fbc95419700 1 --2- 172.21.15.162:0/3582612852 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fbc900fb7f0 0x7fbc900fc780 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fbc8000b8b0 tx=0x7fbc80009e30 comp rx=0 tx=0).stop 2024-04-27T04:19:30.722 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:19:30.722+0000 7fbc95419700 1 -- 172.21.15.162:0/3582612852 shutdown_connections 2024-04-27T04:19:30.722 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:19:30.722+0000 7fbc95419700 1 --2- 172.21.15.162:0/3582612852 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fbc900fb7f0 0x7fbc900fc780 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:19:30.722 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:19:30.722+0000 7fbc95419700 1 --2- 172.21.15.162:0/3582612852 >> [v2:172.21.15.162:6800/1396866108,v1:172.21.15.162:6801/1396866108] conn(0x7fbc7806ec00 0x7fbc780710b0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:19:30.722 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:19:30.722+0000 7fbc95419700 1 --2- 172.21.15.162:0/3582612852 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbc900fc190 0x7fbc900fccc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:19:30.723 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:19:30.722+0000 7fbc95419700 1 -- 172.21.15.162:0/3582612852 >> 172.21.15.162:0/3582612852 conn(0x7fbc90071460 msgr2=0x7fbc900f93d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:19:30.723 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:19:30.723+0000 7fbc95419700 1 -- 172.21.15.162:0/3582612852 shutdown_connections 2024-04-27T04:19:30.723 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:19:30.723+0000 7fbc95419700 1 -- 172.21.15.162:0/3582612852 wait complete. 2024-04-27T04:19:30.723 INFO:teuthology.orchestra.run.smithi162.stderr:dumped fsmap epoch 23 2024-04-27T04:19:30.827 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:19:30.828+0000 7fa76eb1b700 1 -- 172.21.15.162:0/1551820876 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa7680fbd40 msgr2=0x7fa7680fc1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:19:30.827 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:19:30.828+0000 7fa76eb1b700 1 --2- 172.21.15.162:0/1551820876 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa7680fbd40 0x7fa7680fc1b0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fa754004940 tx=0x7fa754009e30 comp rx=0 tx=0).stop 2024-04-27T04:19:30.827 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:19:30.828+0000 7fa76eb1b700 1 -- 172.21.15.162:0/1551820876 shutdown_connections 2024-04-27T04:19:30.827 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:19:30.828+0000 7fa76eb1b700 1 --2- 172.21.15.162:0/1551820876 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa7680fbd40 0x7fa7680fc1b0 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:19:30.827 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:19:30.828+0000 7fa76eb1b700 1 --2- 172.21.15.162:0/1551820876 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa768101d90 0x7fa768102160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:19:30.827 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:19:30.828+0000 7fa76eb1b700 1 -- 172.21.15.162:0/1551820876 >> 172.21.15.162:0/1551820876 conn(0x7fa768071d00 msgr2=0x7fa768072100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:19:30.827 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:19:30.828+0000 7fa76eb1b700 1 -- 172.21.15.162:0/1551820876 shutdown_connections 2024-04-27T04:19:30.827 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:19:30.828+0000 7fa76eb1b700 1 -- 172.21.15.162:0/1551820876 wait complete. 2024-04-27T04:19:30.828 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:19:30.828+0000 7fa76eb1b700 1 Processor -- start 2024-04-27T04:19:30.828 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:19:30.828+0000 7fa76eb1b700 1 -- start start 2024-04-27T04:19:30.829 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:19:30.829+0000 7fa76eb1b700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa7680fbd40 0x7fa76810c910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:19:30.829 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:19:30.829+0000 7fa76eb1b700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa768101d90 0x7fa76810ce50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:19:30.829 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:19:30.829+0000 7fa76eb1b700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa76810d390 con 0x7fa768101d90 2024-04-27T04:19:30.829 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:19:30.829+0000 7fa76eb1b700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa76810d500 con 0x7fa7680fbd40 2024-04-27T04:19:30.829 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:19:30.829+0000 7fa767fff700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa768101d90 0x7fa76810ce50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:19:30.830 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:19:30.829+0000 7fa76c8b7700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa7680fbd40 0x7fa76810c910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:19:30.830 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:19:30.829+0000 7fa767fff700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa768101d90 0x7fa76810ce50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:56334/0 (socket says 172.21.15.162:56334) 2024-04-27T04:19:30.830 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:19:30.829+0000 7fa767fff700 1 -- 172.21.15.162:0/3098615938 learned_addr learned my addr 172.21.15.162:0/3098615938 (peer_addr_for_me v2:172.21.15.162:0/0) 2024-04-27T04:19:30.830 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:19:30.829+0000 7fa767fff700 1 -- 172.21.15.162:0/3098615938 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa7680fbd40 msgr2=0x7fa76810c910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:19:30.831 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:19:30.829+0000 7fa767fff700 1 --2- 172.21.15.162:0/3098615938 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa7680fbd40 0x7fa76810c910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:19:30.831 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:19:30.829+0000 7fa767fff700 1 -- 172.21.15.162:0/3098615938 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa754005040 con 0x7fa768101d90 2024-04-27T04:19:30.831 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:19:30.829+0000 7fa767fff700 1 --2- 172.21.15.162:0/3098615938 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa768101d90 0x7fa76810ce50 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fa754009150 tx=0x7fa754009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:19:30.831 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:19:30.829+0000 7fa7657fa700 1 -- 172.21.15.162:0/3098615938 <== mon.0 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa754021070 con 0x7fa768101d90 2024-04-27T04:19:30.831 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:19:30.829+0000 7fa7657fa700 1 -- 172.21.15.162:0/3098615938 <== mon.0 v2:172.21.15.162:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fa754013c00 con 0x7fa768101d90 2024-04-27T04:19:30.831 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:19:30.829+0000 7fa76eb1b700 1 -- 172.21.15.162:0/3098615938 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa76810d780 con 0x7fa768101d90 2024-04-27T04:19:30.832 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:19:30.830+0000 7fa7657fa700 1 -- 172.21.15.162:0/3098615938 <== mon.0 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa75400e070 con 0x7fa768101d90 2024-04-27T04:19:30.832 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:19:30.830+0000 7fa76eb1b700 1 -- 172.21.15.162:0/3098615938 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa76811a250 con 0x7fa768101d90 2024-04-27T04:19:30.832 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:19:30.830+0000 7fa76eb1b700 1 -- 172.21.15.162:0/3098615938 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa7680fd430 con 0x7fa768101d90 2024-04-27T04:19:30.832 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:19:30.831+0000 7fa7657fa700 1 -- 172.21.15.162:0/3098615938 <== mon.0 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 39) v1 ==== 92747+0+0 (secure 0 0 0) 0x7fa75401c400 con 0x7fa768101d90 2024-04-27T04:19:30.833 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:19:30.831+0000 7fa7657fa700 1 --2- 172.21.15.162:0/3098615938 >> [v2:172.21.15.162:6800/1396866108,v1:172.21.15.162:6801/1396866108] conn(0x7fa75806ec10 0x7fa7580710c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:19:30.833 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:19:30.832+0000 7fa7657fa700 1 -- 172.21.15.162:0/3098615938 <== mon.0 v2:172.21.15.162:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6496+0+0 (secure 0 0 0) 0x7fa75408fa90 con 0x7fa768101d90 2024-04-27T04:19:30.833 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:19:30.832+0000 7fa76c8b7700 1 --2- 172.21.15.162:0/3098615938 >> [v2:172.21.15.162:6800/1396866108,v1:172.21.15.162:6801/1396866108] conn(0x7fa75806ec10 0x7fa7580710c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:19:30.834 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:19:30.832+0000 7fa76c8b7700 1 --2- 172.21.15.162:0/3098615938 >> [v2:172.21.15.162:6800/1396866108,v1:172.21.15.162:6801/1396866108] conn(0x7fa75806ec10 0x7fa7580710c0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fa75c009b80 tx=0x7fa75c00b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:19:30.835 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:19:30.836+0000 7fa7657fa700 1 -- 172.21.15.162:0/3098615938 <== mon.0 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7fa75405c430 con 0x7fa768101d90 2024-04-27T04:19:31.062 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:19:31.063+0000 7fa76eb1b700 1 -- 172.21.15.162:0/3098615938 --> [v2:172.21.15.162:6800/1396866108,v1:172.21.15.162:6801/1396866108] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fa76810d780 con 0x7fa75806ec10 2024-04-27T04:19:31.062 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:19:30 smithi162 ceph-mon[110641]: from='client.34416 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-27T04:19:31.062 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:19:30 smithi162 ceph-mon[110641]: from='client.34420 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-27T04:19:31.063 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:19:30 smithi162 ceph-mon[110641]: pgmap v193: 65 pgs: 65 active+clean; 208 MiB data, 851 MiB used, 536 GiB / 536 GiB avail; 5.0 MiB/s rd, 2 op/s 2024-04-27T04:19:31.063 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:19:30 smithi162 ceph-mon[110641]: from='client.? 172.21.15.162:0/3126834651' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-27T04:19:31.063 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:19:30 smithi162 ceph-mon[110641]: from='client.? 172.21.15.162:0/3582612852' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-27T04:19:31.064 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:19:31.065+0000 7fa7657fa700 1 -- 172.21.15.162:0/3098615938 <== mgr.34104 v2:172.21.15.162:6800/1396866108 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7fa76810d780 con 0x7fa75806ec10 2024-04-27T04:19:31.064 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-04-27T04:19:31.064 INFO:teuthology.orchestra.run.smithi162.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:90a3a9fa9ceb8977f6dad9ccd36fe46018d07e2118199820523a6b35301d9c16", 2024-04-27T04:19:31.064 INFO:teuthology.orchestra.run.smithi162.stdout: "in_progress": true, 2024-04-27T04:19:31.064 INFO:teuthology.orchestra.run.smithi162.stdout: "which": "Upgrading all daemon types on all hosts", 2024-04-27T04:19:31.064 INFO:teuthology.orchestra.run.smithi162.stdout: "services_complete": [ 2024-04-27T04:19:31.064 INFO:teuthology.orchestra.run.smithi162.stdout: "crash", 2024-04-27T04:19:31.064 INFO:teuthology.orchestra.run.smithi162.stdout: "mon", 2024-04-27T04:19:31.064 INFO:teuthology.orchestra.run.smithi162.stdout: "osd", 2024-04-27T04:19:31.064 INFO:teuthology.orchestra.run.smithi162.stdout: "mgr" 2024-04-27T04:19:31.064 INFO:teuthology.orchestra.run.smithi162.stdout: ], 2024-04-27T04:19:31.065 INFO:teuthology.orchestra.run.smithi162.stdout: "progress": "14/21 daemons upgraded", 2024-04-27T04:19:31.065 INFO:teuthology.orchestra.run.smithi162.stdout: "message": "Currently upgrading mds daemons", 2024-04-27T04:19:31.065 INFO:teuthology.orchestra.run.smithi162.stdout: "is_paused": false 2024-04-27T04:19:31.065 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-04-27T04:19:31.067 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:19:31.068+0000 7fa76eb1b700 1 -- 172.21.15.162:0/3098615938 >> [v2:172.21.15.162:6800/1396866108,v1:172.21.15.162:6801/1396866108] conn(0x7fa75806ec10 msgr2=0x7fa7580710c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:19:31.068 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:19:31.068+0000 7fa76eb1b700 1 --2- 172.21.15.162:0/3098615938 >> [v2:172.21.15.162:6800/1396866108,v1:172.21.15.162:6801/1396866108] conn(0x7fa75806ec10 0x7fa7580710c0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fa75c009b80 tx=0x7fa75c00b040 comp rx=0 tx=0).stop 2024-04-27T04:19:31.068 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:19:31.068+0000 7fa76eb1b700 1 -- 172.21.15.162:0/3098615938 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa768101d90 msgr2=0x7fa76810ce50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:19:31.068 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:19:31.068+0000 7fa76eb1b700 1 --2- 172.21.15.162:0/3098615938 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa768101d90 0x7fa76810ce50 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fa754009150 tx=0x7fa754009e30 comp rx=0 tx=0).stop 2024-04-27T04:19:31.068 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:19:31.068+0000 7fa76eb1b700 1 -- 172.21.15.162:0/3098615938 shutdown_connections 2024-04-27T04:19:31.068 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:19:31.068+0000 7fa76eb1b700 1 --2- 172.21.15.162:0/3098615938 >> [v2:172.21.15.162:6800/1396866108,v1:172.21.15.162:6801/1396866108] conn(0x7fa75806ec10 0x7fa7580710c0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:19:31.068 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:19:31.068+0000 7fa76eb1b700 1 --2- 172.21.15.162:0/3098615938 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa7680fbd40 0x7fa76810c910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:19:31.068 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:19:31.068+0000 7fa76eb1b700 1 --2- 172.21.15.162:0/3098615938 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa768101d90 0x7fa76810ce50 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:19:31.068 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:19:31.069+0000 7fa76eb1b700 1 -- 172.21.15.162:0/3098615938 >> 172.21.15.162:0/3098615938 conn(0x7fa768071d00 msgr2=0x7fa7680f9ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:19:31.068 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:19:31.069+0000 7fa76eb1b700 1 -- 172.21.15.162:0/3098615938 shutdown_connections 2024-04-27T04:19:31.068 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:19:31.069+0000 7fa76eb1b700 1 -- 172.21.15.162:0/3098615938 wait complete. 2024-04-27T04:19:31.171 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:19:30 smithi181 ceph-mon[81134]: from='client.34416 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-27T04:19:31.171 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:19:30 smithi181 ceph-mon[81134]: from='client.34420 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-27T04:19:31.172 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:19:30 smithi181 ceph-mon[81134]: pgmap v193: 65 pgs: 65 active+clean; 208 MiB data, 851 MiB used, 536 GiB / 536 GiB avail; 5.0 MiB/s rd, 2 op/s 2024-04-27T04:19:31.172 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:19:30 smithi181 ceph-mon[81134]: from='client.? 172.21.15.162:0/3126834651' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-27T04:19:31.172 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:19:30 smithi181 ceph-mon[81134]: from='client.? 172.21.15.162:0/3582612852' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-27T04:19:31.175 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:19:31.175+0000 7f73c8379700 1 -- 172.21.15.162:0/4140708705 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f73c0101cd0 msgr2=0x7f73c01020a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:19:31.176 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:19:31.175+0000 7f73c8379700 1 --2- 172.21.15.162:0/4140708705 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f73c0101cd0 0x7f73c01020a0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f73bc004940 tx=0x7f73bc009e30 comp rx=0 tx=0).stop 2024-04-27T04:19:31.176 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:19:31.175+0000 7f73c8379700 1 -- 172.21.15.162:0/4140708705 shutdown_connections 2024-04-27T04:19:31.176 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:19:31.175+0000 7f73c8379700 1 --2- 172.21.15.162:0/4140708705 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f73c0102670 0x7f73c00fb600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:19:31.176 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:19:31.175+0000 7f73c8379700 1 --2- 172.21.15.162:0/4140708705 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f73c0101cd0 0x7f73c01020a0 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:19:31.176 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:19:31.175+0000 7f73c8379700 1 -- 172.21.15.162:0/4140708705 >> 172.21.15.162:0/4140708705 conn(0x7f73c0071460 msgr2=0x7f73c0071860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:19:31.176 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:19:31.175+0000 7f73c8379700 1 -- 172.21.15.162:0/4140708705 shutdown_connections 2024-04-27T04:19:31.176 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:19:31.176+0000 7f73c8379700 1 -- 172.21.15.162:0/4140708705 wait complete. 2024-04-27T04:19:31.176 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:19:31.176+0000 7f73c8379700 1 Processor -- start 2024-04-27T04:19:31.176 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:19:31.176+0000 7f73c8379700 1 -- start start 2024-04-27T04:19:31.176 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:19:31.176+0000 7f73c8379700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f73c0101cd0 0x7f73c00fc7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:19:31.176 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:19:31.176+0000 7f73c8379700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f73c0102670 0x7f73c00fcd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:19:31.176 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:19:31.176+0000 7f73c8379700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73c00fd260 con 0x7f73c0101cd0 2024-04-27T04:19:31.176 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:19:31.176+0000 7f73c8379700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73c00fd3a0 con 0x7f73c0102670 2024-04-27T04:19:31.176 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:19:31.176+0000 7f73c6115700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f73c0101cd0 0x7f73c00fc7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:19:31.176 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:19:31.176+0000 7f73c6115700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f73c0101cd0 0x7f73c00fc7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:56352/0 (socket says 172.21.15.162:56352) 2024-04-27T04:19:31.177 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:19:31.176+0000 7f73c6115700 1 -- 172.21.15.162:0/1203238991 learned_addr learned my addr 172.21.15.162:0/1203238991 (peer_addr_for_me v2:172.21.15.162:0/0) 2024-04-27T04:19:31.177 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:19:31.177+0000 7f73c5914700 1 --2- 172.21.15.162:0/1203238991 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f73c0102670 0x7f73c00fcd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:19:31.177 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:19:31.177+0000 7f73c6115700 1 -- 172.21.15.162:0/1203238991 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f73c0102670 msgr2=0x7f73c00fcd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:19:31.177 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:19:31.177+0000 7f73c6115700 1 --2- 172.21.15.162:0/1203238991 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f73c0102670 0x7f73c00fcd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:19:31.177 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:19:31.177+0000 7f73c6115700 1 -- 172.21.15.162:0/1203238991 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f73bc005040 con 0x7f73c0101cd0 2024-04-27T04:19:31.177 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:19:31.177+0000 7f73c6115700 1 --2- 172.21.15.162:0/1203238991 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f73c0101cd0 0x7f73c00fc7e0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f73bc009150 tx=0x7f73bc009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:19:31.178 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:19:31.177+0000 7f73b6ffd700 1 -- 172.21.15.162:0/1203238991 <== mon.0 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f73bc021070 con 0x7f73c0101cd0 2024-04-27T04:19:31.178 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:19:31.177+0000 7f73b6ffd700 1 -- 172.21.15.162:0/1203238991 <== mon.0 v2:172.21.15.162:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f73bc013c00 con 0x7f73c0101cd0 2024-04-27T04:19:31.178 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:19:31.177+0000 7f73c8379700 1 -- 172.21.15.162:0/1203238991 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f73c00fd5f0 con 0x7f73c0101cd0 2024-04-27T04:19:31.178 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:19:31.177+0000 7f73b6ffd700 1 -- 172.21.15.162:0/1203238991 <== mon.0 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f73bc00e070 con 0x7f73c0101cd0 2024-04-27T04:19:31.178 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:19:31.177+0000 7f73c8379700 1 -- 172.21.15.162:0/1203238991 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f73c011a140 con 0x7f73c0101cd0 2024-04-27T04:19:31.178 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:19:31.178+0000 7f73c8379700 1 -- 172.21.15.162:0/1203238991 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f73c0065200 con 0x7f73c0101cd0 2024-04-27T04:19:31.179 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:19:31.179+0000 7f73b6ffd700 1 -- 172.21.15.162:0/1203238991 <== mon.0 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 39) v1 ==== 92747+0+0 (secure 0 0 0) 0x7f73bc01c400 con 0x7f73c0101cd0 2024-04-27T04:19:31.179 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:19:31.179+0000 7f73b6ffd700 1 --2- 172.21.15.162:0/1203238991 >> [v2:172.21.15.162:6800/1396866108,v1:172.21.15.162:6801/1396866108] conn(0x7f73ac06ec10 0x7f73ac0710c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:19:31.179 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:19:31.179+0000 7f73b6ffd700 1 -- 172.21.15.162:0/1203238991 <== mon.0 v2:172.21.15.162:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6496+0+0 (secure 0 0 0) 0x7f73bc08fc10 con 0x7f73c0101cd0 2024-04-27T04:19:31.179 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:19:31.179+0000 7f73c5914700 1 --2- 172.21.15.162:0/1203238991 >> [v2:172.21.15.162:6800/1396866108,v1:172.21.15.162:6801/1396866108] conn(0x7f73ac06ec10 0x7f73ac0710c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:19:31.179 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:19:31.180+0000 7f73c5914700 1 --2- 172.21.15.162:0/1203238991 >> [v2:172.21.15.162:6800/1396866108,v1:172.21.15.162:6801/1396866108] conn(0x7f73ac06ec10 0x7f73ac0710c0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f73c00fe0d0 tx=0x7f73b000b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:19:31.182 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:19:31.184+0000 7f73b6ffd700 1 -- 172.21.15.162:0/1203238991 <== mon.0 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7f73bc058b00 con 0x7f73c0101cd0 2024-04-27T04:19:31.469 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:19:31.469+0000 7f73c8379700 1 -- 172.21.15.162:0/1203238991 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f73c004bed0 con 0x7f73c0101cd0 2024-04-27T04:19:31.469 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:19:31.470+0000 7f73b6ffd700 1 -- 172.21.15.162:0/1203238991 <== mon.0 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+107 (secure 0 0 0) 0x7f73bc004490 con 0x7f73c0101cd0 2024-04-27T04:19:31.469 INFO:teuthology.orchestra.run.smithi162.stdout:HEALTH_WARN 1 filesystem is degraded 2024-04-27T04:19:31.469 INFO:teuthology.orchestra.run.smithi162.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-04-27T04:19:31.470 INFO:teuthology.orchestra.run.smithi162.stdout: fs cephfs is degraded 2024-04-27T04:19:31.472 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:19:31.472+0000 7f73c8379700 1 -- 172.21.15.162:0/1203238991 >> [v2:172.21.15.162:6800/1396866108,v1:172.21.15.162:6801/1396866108] conn(0x7f73ac06ec10 msgr2=0x7f73ac0710c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:19:31.472 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:19:31.472+0000 7f73c8379700 1 --2- 172.21.15.162:0/1203238991 >> [v2:172.21.15.162:6800/1396866108,v1:172.21.15.162:6801/1396866108] conn(0x7f73ac06ec10 0x7f73ac0710c0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f73c00fe0d0 tx=0x7f73b000b040 comp rx=0 tx=0).stop 2024-04-27T04:19:31.472 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:19:31.472+0000 7f73c8379700 1 -- 172.21.15.162:0/1203238991 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f73c0101cd0 msgr2=0x7f73c00fc7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:19:31.472 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:19:31.472+0000 7f73c8379700 1 --2- 172.21.15.162:0/1203238991 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f73c0101cd0 0x7f73c00fc7e0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f73bc009150 tx=0x7f73bc009e30 comp rx=0 tx=0).stop 2024-04-27T04:19:31.472 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:19:31.472+0000 7f73c8379700 1 -- 172.21.15.162:0/1203238991 shutdown_connections 2024-04-27T04:19:31.473 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:19:31.472+0000 7f73c8379700 1 --2- 172.21.15.162:0/1203238991 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f73c0101cd0 0x7f73c00fc7e0 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:19:31.473 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:19:31.472+0000 7f73c8379700 1 --2- 172.21.15.162:0/1203238991 >> [v2:172.21.15.162:6800/1396866108,v1:172.21.15.162:6801/1396866108] conn(0x7f73ac06ec10 0x7f73ac0710c0 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:19:31.473 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:19:31.472+0000 7f73c8379700 1 --2- 172.21.15.162:0/1203238991 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f73c0102670 0x7f73c00fcd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:19:31.473 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:19:31.472+0000 7f73c8379700 1 -- 172.21.15.162:0/1203238991 >> 172.21.15.162:0/1203238991 conn(0x7f73c0071460 msgr2=0x7f73c00fabe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:19:31.473 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:19:31.472+0000 7f73c8379700 1 -- 172.21.15.162:0/1203238991 shutdown_connections 2024-04-27T04:19:31.474 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:19:31.472+0000 7f73c8379700 1 -- 172.21.15.162:0/1203238991 wait complete. 2024-04-27T04:19:32.073 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:19:31 smithi181 ceph-mon[81134]: from='client.34424 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-27T04:19:32.073 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:19:31 smithi181 ceph-mon[81134]: from='client.? 172.21.15.162:0/1203238991' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-27T04:19:32.136 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:19:31 smithi162 ceph-mon[110641]: from='client.34424 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-27T04:19:32.136 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:19:31 smithi162 ceph-mon[110641]: from='client.? 172.21.15.162:0/1203238991' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-27T04:19:33.092 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:19:32 smithi181 ceph-mon[81134]: from='client.34436 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-27T04:19:33.092 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:19:32 smithi181 ceph-mon[81134]: pgmap v194: 65 pgs: 65 active+clean; 208 MiB data, 851 MiB used, 536 GiB / 536 GiB avail; 13 MiB/s rd, 4 op/s 2024-04-27T04:19:33.092 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:19:32 smithi181 ceph-mon[81134]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' 2024-04-27T04:19:33.092 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:19:32 smithi181 ceph-mon[81134]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-27T04:19:33.136 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:19:32 smithi162 ceph-mon[110641]: from='client.34436 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-27T04:19:33.136 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:19:32 smithi162 ceph-mon[110641]: pgmap v194: 65 pgs: 65 active+clean; 208 MiB data, 851 MiB used, 536 GiB / 536 GiB avail; 13 MiB/s rd, 4 op/s 2024-04-27T04:19:33.136 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:19:32 smithi162 ceph-mon[110641]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' 2024-04-27T04:19:33.136 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:19:32 smithi162 ceph-mon[110641]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-27T04:19:35.136 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:19:34 smithi162 ceph-mon[110641]: pgmap v195: 65 pgs: 65 active+clean; 208 MiB data, 851 MiB used, 536 GiB / 536 GiB avail; 16 MiB/s rd, 5 op/s 2024-04-27T04:19:35.171 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:19:34 smithi181 ceph-mon[81134]: pgmap v195: 65 pgs: 65 active+clean; 208 MiB data, 851 MiB used, 536 GiB / 536 GiB avail; 16 MiB/s rd, 5 op/s 2024-04-27T04:19:37.071 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:19:36 smithi181 ceph-mon[81134]: pgmap v196: 65 pgs: 65 active+clean; 208 MiB data, 851 MiB used, 536 GiB / 536 GiB avail; 20 MiB/s rd, 6 op/s 2024-04-27T04:19:37.071 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:19:36 smithi181 ceph-mon[81134]: reconnect by client.24323 192.168.0.1:0/2156375244 after 0.000999979 2024-04-27T04:19:37.071 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:19:36 smithi181 ceph-mon[81134]: reconnect by client.14528 192.168.0.1:0/91994784 after 0.00199996 2024-04-27T04:19:37.071 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:19:36 smithi181 ceph-mon[81134]: mds.? [v2:172.21.15.181:6826/3756475453,v1:172.21.15.181:6827/3756475453] up:reconnect 2024-04-27T04:19:37.071 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:19:36 smithi181 ceph-mon[81134]: fsmap cephfs:1/1 {0=cephfs.smithi181.bkkcdm=up:reconnect} 2 up:standby 2024-04-27T04:19:37.136 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:19:36 smithi162 ceph-mon[110641]: pgmap v196: 65 pgs: 65 active+clean; 208 MiB data, 851 MiB used, 536 GiB / 536 GiB avail; 20 MiB/s rd, 6 op/s 2024-04-27T04:19:37.136 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:19:36 smithi162 ceph-mon[110641]: reconnect by client.24323 192.168.0.1:0/2156375244 after 0.000999979 2024-04-27T04:19:37.136 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:19:36 smithi162 ceph-mon[110641]: reconnect by client.14528 192.168.0.1:0/91994784 after 0.00199996 2024-04-27T04:19:37.136 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:19:36 smithi162 ceph-mon[110641]: mds.? [v2:172.21.15.181:6826/3756475453,v1:172.21.15.181:6827/3756475453] up:reconnect 2024-04-27T04:19:37.136 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:19:36 smithi162 ceph-mon[110641]: fsmap cephfs:1/1 {0=cephfs.smithi181.bkkcdm=up:reconnect} 2 up:standby 2024-04-27T04:19:37.922 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:19:37 smithi181 ceph-mon[81134]: mds.? [v2:172.21.15.181:6826/3756475453,v1:172.21.15.181:6827/3756475453] up:rejoin 2024-04-27T04:19:37.922 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:19:37 smithi181 ceph-mon[81134]: fsmap cephfs:1/1 {0=cephfs.smithi181.bkkcdm=up:rejoin} 2 up:standby 2024-04-27T04:19:37.922 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:19:37 smithi181 ceph-mon[81134]: daemon mds.cephfs.smithi181.bkkcdm is now active in filesystem cephfs as rank 0 2024-04-27T04:19:38.136 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:19:37 smithi162 ceph-mon[110641]: mds.? [v2:172.21.15.181:6826/3756475453,v1:172.21.15.181:6827/3756475453] up:rejoin 2024-04-27T04:19:38.136 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:19:37 smithi162 ceph-mon[110641]: fsmap cephfs:1/1 {0=cephfs.smithi181.bkkcdm=up:rejoin} 2 up:standby 2024-04-27T04:19:38.136 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:19:37 smithi162 ceph-mon[110641]: daemon mds.cephfs.smithi181.bkkcdm is now active in filesystem cephfs as rank 0 2024-04-27T04:19:38.921 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:19:38 smithi181 ceph-mon[81134]: pgmap v197: 65 pgs: 65 active+clean; 208 MiB data, 851 MiB used, 536 GiB / 536 GiB avail; 18 MiB/s rd, 5 op/s 2024-04-27T04:19:38.922 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:19:38 smithi181 ceph-mon[81134]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-04-27T04:19:38.922 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:19:38 smithi181 ceph-mon[81134]: Cluster is now healthy 2024-04-27T04:19:38.922 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:19:38 smithi181 ceph-mon[81134]: mds.? [v2:172.21.15.181:6826/3756475453,v1:172.21.15.181:6827/3756475453] up:active 2024-04-27T04:19:38.922 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:19:38 smithi181 ceph-mon[81134]: fsmap cephfs:1 {0=cephfs.smithi181.bkkcdm=up:active} 2 up:standby 2024-04-27T04:19:39.136 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:19:38 smithi162 ceph-mon[110641]: pgmap v197: 65 pgs: 65 active+clean; 208 MiB data, 851 MiB used, 536 GiB / 536 GiB avail; 18 MiB/s rd, 5 op/s 2024-04-27T04:19:39.136 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:19:38 smithi162 ceph-mon[110641]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-04-27T04:19:39.136 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:19:38 smithi162 ceph-mon[110641]: Cluster is now healthy 2024-04-27T04:19:39.136 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:19:38 smithi162 ceph-mon[110641]: mds.? [v2:172.21.15.181:6826/3756475453,v1:172.21.15.181:6827/3756475453] up:active 2024-04-27T04:19:39.137 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:19:38 smithi162 ceph-mon[110641]: fsmap cephfs:1 {0=cephfs.smithi181.bkkcdm=up:active} 2 up:standby 2024-04-27T04:19:40.672 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:19:40 smithi181 ceph-mon[81134]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' 2024-04-27T04:19:40.672 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:19:40 smithi181 ceph-mon[81134]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' 2024-04-27T04:19:40.672 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:19:40 smithi181 ceph-mon[81134]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T04:19:40.672 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:19:40 smithi181 ceph-mon[81134]: pgmap v198: 65 pgs: 65 active+clean; 208 MiB data, 851 MiB used, 536 GiB / 536 GiB avail; 16 MiB/s rd, 341 B/s wr, 6 op/s 2024-04-27T04:19:40.672 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:19:40 smithi181 ceph-mon[81134]: mds.? [v2:172.21.15.181:6824/1667773112,v1:172.21.15.181:6825/1667773112] up:boot 2024-04-27T04:19:40.672 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:19:40 smithi181 ceph-mon[81134]: fsmap cephfs:1 {0=cephfs.smithi181.bkkcdm=up:active} 3 up:standby 2024-04-27T04:19:40.673 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:19:40 smithi181 ceph-mon[81134]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi181.dzomgh"}]: dispatch 2024-04-27T04:19:40.886 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:19:40 smithi162 ceph-mon[110641]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' 2024-04-27T04:19:40.886 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:19:40 smithi162 ceph-mon[110641]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' 2024-04-27T04:19:40.886 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:19:40 smithi162 ceph-mon[110641]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T04:19:40.886 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:19:40 smithi162 ceph-mon[110641]: pgmap v198: 65 pgs: 65 active+clean; 208 MiB data, 851 MiB used, 536 GiB / 536 GiB avail; 16 MiB/s rd, 341 B/s wr, 6 op/s 2024-04-27T04:19:40.886 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:19:40 smithi162 ceph-mon[110641]: mds.? [v2:172.21.15.181:6824/1667773112,v1:172.21.15.181:6825/1667773112] up:boot 2024-04-27T04:19:40.887 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:19:40 smithi162 ceph-mon[110641]: fsmap cephfs:1 {0=cephfs.smithi181.bkkcdm=up:active} 3 up:standby 2024-04-27T04:19:40.887 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:19:40 smithi162 ceph-mon[110641]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi181.dzomgh"}]: dispatch 2024-04-27T04:19:43.136 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:19:42 smithi162 ceph-mon[110641]: pgmap v199: 65 pgs: 65 active+clean; 208 MiB data, 851 MiB used, 536 GiB / 536 GiB avail; 13 MiB/s rd, 426 B/s wr, 6 op/s 2024-04-27T04:19:43.136 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:19:42 smithi162 ceph-mon[110641]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' 2024-04-27T04:19:43.136 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:19:42 smithi162 ceph-mon[110641]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' 2024-04-27T04:19:43.171 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:19:42 smithi181 ceph-mon[81134]: pgmap v199: 65 pgs: 65 active+clean; 208 MiB data, 851 MiB used, 536 GiB / 536 GiB avail; 13 MiB/s rd, 426 B/s wr, 6 op/s 2024-04-27T04:19:43.171 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:19:42 smithi181 ceph-mon[81134]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' 2024-04-27T04:19:43.171 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:19:42 smithi181 ceph-mon[81134]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' 2024-04-27T04:19:44.671 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:19:44 smithi181 ceph-mon[81134]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' 2024-04-27T04:19:44.671 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:19:44 smithi181 ceph-mon[81134]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' 2024-04-27T04:19:44.671 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:19:44 smithi181 ceph-mon[81134]: pgmap v200: 65 pgs: 65 active+clean; 208 MiB data, 851 MiB used, 536 GiB / 536 GiB avail; 6.0 MiB/s rd, 4.7 KiB/s wr, 7 op/s 2024-04-27T04:19:44.886 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:19:44 smithi162 ceph-mon[110641]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' 2024-04-27T04:19:44.886 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:19:44 smithi162 ceph-mon[110641]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' 2024-04-27T04:19:44.886 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:19:44 smithi162 ceph-mon[110641]: pgmap v200: 65 pgs: 65 active+clean; 208 MiB data, 851 MiB used, 536 GiB / 536 GiB avail; 6.0 MiB/s rd, 4.7 KiB/s wr, 7 op/s 2024-04-27T04:19:46.834 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:19:46 smithi181 ceph-mon[81134]: pgmap v201: 65 pgs: 65 active+clean; 208 MiB data, 851 MiB used, 536 GiB / 536 GiB avail; 3.3 MiB/s rd, 4.7 KiB/s wr, 6 op/s 2024-04-27T04:19:47.136 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:19:46 smithi162 ceph-mon[110641]: pgmap v201: 65 pgs: 65 active+clean; 208 MiB data, 851 MiB used, 536 GiB / 536 GiB avail; 3.3 MiB/s rd, 4.7 KiB/s wr, 6 op/s 2024-04-27T04:19:48.136 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:19:47 smithi162 ceph-mon[110641]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' 2024-04-27T04:19:48.136 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:19:47 smithi162 ceph-mon[110641]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' 2024-04-27T04:19:48.136 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:19:47 smithi162 ceph-mon[110641]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T04:19:48.136 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:19:47 smithi162 ceph-mon[110641]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T04:19:48.137 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:19:47 smithi162 ceph-mon[110641]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' 2024-04-27T04:19:48.137 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:19:47 smithi162 ceph-mon[110641]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T04:19:48.137 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:19:47 smithi162 ceph-mon[110641]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "versions"}]: dispatch 2024-04-27T04:19:48.137 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:19:47 smithi162 ceph-mon[110641]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "versions"}]: dispatch 2024-04-27T04:19:48.137 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:19:47 smithi162 ceph-mon[110641]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "versions"}]: dispatch 2024-04-27T04:19:48.137 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:19:47 smithi162 ceph-mon[110641]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "versions"}]: dispatch 2024-04-27T04:19:48.137 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:19:47 smithi162 ceph-mon[110641]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' 2024-04-27T04:19:48.137 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:19:47 smithi162 ceph-mon[110641]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-27T04:19:48.137 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:19:47 smithi162 ceph-mon[110641]: pgmap v202: 65 pgs: 65 active+clean; 208 MiB data, 851 MiB used, 536 GiB / 536 GiB avail; 345 KiB/s rd, 4.7 KiB/s wr, 5 op/s 2024-04-27T04:19:48.171 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:19:47 smithi181 ceph-mon[81134]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' 2024-04-27T04:19:48.171 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:19:47 smithi181 ceph-mon[81134]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' 2024-04-27T04:19:48.171 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:19:47 smithi181 ceph-mon[81134]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T04:19:48.172 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:19:47 smithi181 ceph-mon[81134]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T04:19:48.172 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:19:47 smithi181 ceph-mon[81134]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' 2024-04-27T04:19:48.172 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:19:47 smithi181 ceph-mon[81134]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T04:19:48.172 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:19:47 smithi181 ceph-mon[81134]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "versions"}]: dispatch 2024-04-27T04:19:48.172 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:19:47 smithi181 ceph-mon[81134]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "versions"}]: dispatch 2024-04-27T04:19:48.172 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:19:47 smithi181 ceph-mon[81134]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "versions"}]: dispatch 2024-04-27T04:19:48.172 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:19:47 smithi181 ceph-mon[81134]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "versions"}]: dispatch 2024-04-27T04:19:48.172 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:19:47 smithi181 ceph-mon[81134]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' 2024-04-27T04:19:48.172 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:19:47 smithi181 ceph-mon[81134]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-27T04:19:48.172 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:19:47 smithi181 ceph-mon[81134]: pgmap v202: 65 pgs: 65 active+clean; 208 MiB data, 851 MiB used, 536 GiB / 536 GiB avail; 345 KiB/s rd, 4.7 KiB/s wr, 5 op/s 2024-04-27T04:19:49.671 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:19:49 smithi181 ceph-mon[81134]: Upgrade: Updating mds.cephfs.smithi181.bkkcdm 2024-04-27T04:19:49.671 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:19:49 smithi181 ceph-mon[81134]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' 2024-04-27T04:19:49.672 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:19:49 smithi181 ceph-mon[81134]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi181.bkkcdm", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-27T04:19:49.672 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:19:49 smithi181 ceph-mon[81134]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T04:19:49.672 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:19:49 smithi181 ceph-mon[81134]: Deploying daemon mds.cephfs.smithi181.bkkcdm on smithi181 2024-04-27T04:19:49.886 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:19:49 smithi162 ceph-mon[110641]: Upgrade: Updating mds.cephfs.smithi181.bkkcdm 2024-04-27T04:19:49.886 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:19:49 smithi162 ceph-mon[110641]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' 2024-04-27T04:19:49.886 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:19:49 smithi162 ceph-mon[110641]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi181.bkkcdm", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-27T04:19:49.887 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:19:49 smithi162 ceph-mon[110641]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T04:19:49.887 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:19:49 smithi162 ceph-mon[110641]: Deploying daemon mds.cephfs.smithi181.bkkcdm on smithi181 2024-04-27T04:19:50.671 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:19:50 smithi181 ceph-mon[81134]: pgmap v203: 65 pgs: 65 active+clean; 208 MiB data, 851 MiB used, 536 GiB / 536 GiB avail; 3.3 KiB/s rd, 4.7 KiB/s wr, 5 op/s 2024-04-27T04:19:50.886 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:19:50 smithi162 ceph-mon[110641]: pgmap v203: 65 pgs: 65 active+clean; 208 MiB data, 851 MiB used, 536 GiB / 536 GiB avail; 3.3 KiB/s rd, 4.7 KiB/s wr, 5 op/s 2024-04-27T04:19:50.886 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:19:50 smithi162 ceph-7f9750d6-044a-11ef-bc93-c7b262605968-mon-smithi162[110637]: 2024-04-27T04:19:50.546+0000 7f2a7886b700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-27T04:19:51.886 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:19:51 smithi162 ceph-mon[110641]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-04-27T04:19:51.886 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:19:51 smithi162 ceph-mon[110641]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-27T04:19:51.886 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:19:51 smithi162 ceph-mon[110641]: osdmap e84: 6 total, 6 up, 6 in 2024-04-27T04:19:51.886 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:19:51 smithi162 ceph-mon[110641]: Standby daemon mds.cephfs.smithi162.gzlyjj assigned to filesystem cephfs as rank 0 2024-04-27T04:19:51.886 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:19:51 smithi162 ceph-mon[110641]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-04-27T04:19:51.886 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:19:51 smithi162 ceph-mon[110641]: fsmap cephfs:0/1 3 up:standby, 1 failed 2024-04-27T04:19:51.887 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:19:51 smithi162 ceph-mon[110641]: fsmap cephfs:1/1 {0=cephfs.smithi162.gzlyjj=up:replay} 2 up:standby 2024-04-27T04:19:51.922 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:19:51 smithi181 ceph-mon[81134]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-04-27T04:19:51.922 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:19:51 smithi181 ceph-mon[81134]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-27T04:19:51.922 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:19:51 smithi181 ceph-mon[81134]: osdmap e84: 6 total, 6 up, 6 in 2024-04-27T04:19:51.922 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:19:51 smithi181 ceph-mon[81134]: Standby daemon mds.cephfs.smithi162.gzlyjj assigned to filesystem cephfs as rank 0 2024-04-27T04:19:51.922 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:19:51 smithi181 ceph-mon[81134]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-04-27T04:19:51.922 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:19:51 smithi181 ceph-mon[81134]: fsmap cephfs:0/1 3 up:standby, 1 failed 2024-04-27T04:19:51.922 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:19:51 smithi181 ceph-mon[81134]: fsmap cephfs:1/1 {0=cephfs.smithi162.gzlyjj=up:replay} 2 up:standby 2024-04-27T04:19:52.886 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:19:52 smithi162 ceph-mon[110641]: pgmap v205: 65 pgs: 65 active+clean; 208 MiB data, 851 MiB used, 536 GiB / 536 GiB avail; 1.6 MiB/s rd, 5.1 KiB/s wr, 3 op/s 2024-04-27T04:19:52.921 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:19:52 smithi181 ceph-mon[81134]: pgmap v205: 65 pgs: 65 active+clean; 208 MiB data, 851 MiB used, 536 GiB / 536 GiB avail; 1.6 MiB/s rd, 5.1 KiB/s wr, 3 op/s 2024-04-27T04:19:55.136 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:19:54 smithi162 ceph-mon[110641]: pgmap v206: 65 pgs: 65 active+clean; 208 MiB data, 851 MiB used, 536 GiB / 536 GiB avail; 5.0 MiB/s rd, 2 op/s 2024-04-27T04:19:55.171 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:19:54 smithi181 ceph-mon[81134]: pgmap v206: 65 pgs: 65 active+clean; 208 MiB data, 851 MiB used, 536 GiB / 536 GiB avail; 5.0 MiB/s rd, 2 op/s 2024-04-27T04:19:57.136 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:19:56 smithi162 ceph-mon[110641]: pgmap v207: 65 pgs: 65 active+clean; 208 MiB data, 851 MiB used, 536 GiB / 536 GiB avail; 12 MiB/s rd, 4 op/s 2024-04-27T04:19:57.171 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:19:56 smithi181 ceph-mon[81134]: pgmap v207: 65 pgs: 65 active+clean; 208 MiB data, 851 MiB used, 536 GiB / 536 GiB avail; 12 MiB/s rd, 4 op/s 2024-04-27T04:19:58.921 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:19:58 smithi181 ceph-mon[81134]: pgmap v208: 65 pgs: 65 active+clean; 208 MiB data, 851 MiB used, 536 GiB / 536 GiB avail; 15 MiB/s rd, 5 op/s 2024-04-27T04:19:59.136 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:19:58 smithi162 ceph-mon[110641]: pgmap v208: 65 pgs: 65 active+clean; 208 MiB data, 851 MiB used, 536 GiB / 536 GiB avail; 15 MiB/s rd, 5 op/s 2024-04-27T04:20:00.672 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:20:00 smithi181 ceph-mon[81134]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' 2024-04-27T04:20:00.672 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:20:00 smithi181 ceph-mon[81134]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' 2024-04-27T04:20:00.672 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:20:00 smithi181 ceph-mon[81134]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T04:20:00.672 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:20:00 smithi181 ceph-mon[81134]: pgmap v209: 65 pgs: 65 active+clean; 208 MiB data, 851 MiB used, 536 GiB / 536 GiB avail; 19 MiB/s rd, 6 op/s 2024-04-27T04:20:00.672 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:20:00 smithi181 ceph-mon[81134]: reconnect by client.14528 192.168.0.1:0/91994784 after 0.000999979 2024-04-27T04:20:00.672 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:20:00 smithi181 ceph-mon[81134]: reconnect by client.24323 192.168.0.1:0/2156375244 after 0.000999979 2024-04-27T04:20:00.672 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:20:00 smithi181 ceph-mon[81134]: mds.? [v2:172.21.15.181:6826/3589158285,v1:172.21.15.181:6827/3589158285] up:boot 2024-04-27T04:20:00.672 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:20:00 smithi181 ceph-mon[81134]: mds.? [v2:172.21.15.162:6826/1198130667,v1:172.21.15.162:6827/1198130667] up:reconnect 2024-04-27T04:20:00.672 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:20:00 smithi181 ceph-mon[81134]: fsmap cephfs:1/1 {0=cephfs.smithi162.gzlyjj=up:reconnect} 3 up:standby 2024-04-27T04:20:00.672 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:20:00 smithi181 ceph-mon[81134]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi181.bkkcdm"}]: dispatch 2024-04-27T04:20:00.672 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:20:00 smithi181 ceph-mon[81134]: Health detail: HEALTH_WARN 1 filesystem is degraded 2024-04-27T04:20:00.673 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:20:00 smithi181 ceph-mon[81134]: [WRN] FS_DEGRADED: 1 filesystem is degraded 2024-04-27T04:20:00.673 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:20:00 smithi181 ceph-mon[81134]: fs cephfs is degraded 2024-04-27T04:20:00.889 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:20:00 smithi162 ceph-mon[110641]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' 2024-04-27T04:20:00.890 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:20:00 smithi162 ceph-mon[110641]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' 2024-04-27T04:20:00.890 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:20:00 smithi162 ceph-mon[110641]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T04:20:00.890 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:20:00 smithi162 ceph-mon[110641]: pgmap v209: 65 pgs: 65 active+clean; 208 MiB data, 851 MiB used, 536 GiB / 536 GiB avail; 19 MiB/s rd, 6 op/s 2024-04-27T04:20:00.890 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:20:00 smithi162 ceph-mon[110641]: reconnect by client.14528 192.168.0.1:0/91994784 after 0.000999979 2024-04-27T04:20:00.890 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:20:00 smithi162 ceph-mon[110641]: reconnect by client.24323 192.168.0.1:0/2156375244 after 0.000999979 2024-04-27T04:20:00.890 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:20:00 smithi162 ceph-mon[110641]: mds.? [v2:172.21.15.181:6826/3589158285,v1:172.21.15.181:6827/3589158285] up:boot 2024-04-27T04:20:00.890 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:20:00 smithi162 ceph-mon[110641]: mds.? [v2:172.21.15.162:6826/1198130667,v1:172.21.15.162:6827/1198130667] up:reconnect 2024-04-27T04:20:00.890 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:20:00 smithi162 ceph-mon[110641]: fsmap cephfs:1/1 {0=cephfs.smithi162.gzlyjj=up:reconnect} 3 up:standby 2024-04-27T04:20:00.891 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:20:00 smithi162 ceph-mon[110641]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi181.bkkcdm"}]: dispatch 2024-04-27T04:20:00.891 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:20:00 smithi162 ceph-mon[110641]: Health detail: HEALTH_WARN 1 filesystem is degraded 2024-04-27T04:20:00.891 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:20:00 smithi162 ceph-mon[110641]: [WRN] FS_DEGRADED: 1 filesystem is degraded 2024-04-27T04:20:00.891 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:20:00 smithi162 ceph-mon[110641]: fs cephfs is degraded 2024-04-27T04:20:01.589 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:20:01.589+0000 7fbddc6f0700 1 -- 172.21.15.162:0/4294621421 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fbdd4103780 msgr2=0x7fbdd40fcdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:20:01.589 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:20:01.589+0000 7fbddc6f0700 1 --2- 172.21.15.162:0/4294621421 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fbdd4103780 0x7fbdd40fcdf0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fbdc4004940 tx=0x7fbdc4009e30 comp rx=0 tx=0).stop 2024-04-27T04:20:01.589 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:20:01.589+0000 7fbddc6f0700 1 -- 172.21.15.162:0/4294621421 shutdown_connections 2024-04-27T04:20:01.589 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:20:01.589+0000 7fbddc6f0700 1 --2- 172.21.15.162:0/4294621421 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fbdd4103780 0x7fbdd40fcdf0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:20:01.589 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:20:01.589+0000 7fbddc6f0700 1 --2- 172.21.15.162:0/4294621421 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbdd4102de0 0x7fbdd41031b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:20:01.589 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:20:01.589+0000 7fbddc6f0700 1 -- 172.21.15.162:0/4294621421 >> 172.21.15.162:0/4294621421 conn(0x7fbdd40f8990 msgr2=0x7fbdd40fada0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:20:01.589 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:20:01.589+0000 7fbddc6f0700 1 -- 172.21.15.162:0/4294621421 shutdown_connections 2024-04-27T04:20:01.589 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:20:01.589+0000 7fbddc6f0700 1 -- 172.21.15.162:0/4294621421 wait complete. 2024-04-27T04:20:01.589 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:20:01.589+0000 7fbddc6f0700 1 Processor -- start 2024-04-27T04:20:01.589 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:20:01.589+0000 7fbddc6f0700 1 -- start start 2024-04-27T04:20:01.590 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:20:01.590+0000 7fbddc6f0700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbdd4102de0 0x7fbdd41138d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:20:01.590 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:20:01.590+0000 7fbddc6f0700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fbdd4103780 0x7fbdd4113e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:20:01.590 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:20:01.590+0000 7fbddc6f0700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbdd4114460 con 0x7fbdd4103780 2024-04-27T04:20:01.590 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:20:01.590+0000 7fbddc6f0700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbdd410da60 con 0x7fbdd4102de0 2024-04-27T04:20:01.590 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:20:01.590+0000 7fbdd9c8b700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fbdd4103780 0x7fbdd4113e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:20:01.590 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:20:01.590+0000 7fbdda48c700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbdd4102de0 0x7fbdd41138d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:20:01.590 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:20:01.590+0000 7fbdd9c8b700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fbdd4103780 0x7fbdd4113e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:58082/0 (socket says 172.21.15.162:58082) 2024-04-27T04:20:01.590 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:20:01.590+0000 7fbdd9c8b700 1 -- 172.21.15.162:0/973094788 learned_addr learned my addr 172.21.15.162:0/973094788 (peer_addr_for_me v2:172.21.15.162:0/0) 2024-04-27T04:20:01.590 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:20:01.590+0000 7fbdd9c8b700 1 -- 172.21.15.162:0/973094788 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbdd4102de0 msgr2=0x7fbdd41138d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:20:01.590 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:20:01.590+0000 7fbdd9c8b700 1 --2- 172.21.15.162:0/973094788 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbdd4102de0 0x7fbdd41138d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:20:01.590 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:20:01.590+0000 7fbdd9c8b700 1 -- 172.21.15.162:0/973094788 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbdc4005040 con 0x7fbdd4103780 2024-04-27T04:20:01.591 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:20:01.590+0000 7fbdda48c700 1 --2- 172.21.15.162:0/973094788 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbdd4102de0 0x7fbdd41138d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-27T04:20:01.591 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:20:01.590+0000 7fbdd9c8b700 1 --2- 172.21.15.162:0/973094788 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fbdd4103780 0x7fbdd4113e10 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fbdc4000c00 tx=0x7fbdc4009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:20:01.591 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:20:01.591+0000 7fbdcaffd700 1 -- 172.21.15.162:0/973094788 <== mon.0 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbdc4021070 con 0x7fbdd4103780 2024-04-27T04:20:01.591 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:20:01.591+0000 7fbddc6f0700 1 -- 172.21.15.162:0/973094788 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbdd410dce0 con 0x7fbdd4103780 2024-04-27T04:20:01.591 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:20:01.591+0000 7fbdcaffd700 1 -- 172.21.15.162:0/973094788 <== mon.0 v2:172.21.15.162:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fbdc4013c00 con 0x7fbdd4103780 2024-04-27T04:20:01.591 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:20:01.591+0000 7fbdcaffd700 1 -- 172.21.15.162:0/973094788 <== mon.0 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbdc400e070 con 0x7fbdd4103780 2024-04-27T04:20:01.591 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:20:01.591+0000 7fbddc6f0700 1 -- 172.21.15.162:0/973094788 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbdd410e1c0 con 0x7fbdd4103780 2024-04-27T04:20:01.592 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:20:01.592+0000 7fbddc6f0700 1 -- 172.21.15.162:0/973094788 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbdd40fe530 con 0x7fbdd4103780 2024-04-27T04:20:01.592 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:20:01.592+0000 7fbdcaffd700 1 -- 172.21.15.162:0/973094788 <== mon.0 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 39) v1 ==== 92747+0+0 (secure 0 0 0) 0x7fbdc401ca60 con 0x7fbdd4103780 2024-04-27T04:20:01.592 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:20:01.593+0000 7fbdcaffd700 1 --2- 172.21.15.162:0/973094788 >> [v2:172.21.15.162:6800/1396866108,v1:172.21.15.162:6801/1396866108] conn(0x7fbdc006ec00 0x7fbdc00710b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:20:01.592 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:20:01.593+0000 7fbdda48c700 1 --2- 172.21.15.162:0/973094788 >> [v2:172.21.15.162:6800/1396866108,v1:172.21.15.162:6801/1396866108] conn(0x7fbdc006ec00 0x7fbdc00710b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:20:01.592 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:20:01.593+0000 7fbdcaffd700 1 -- 172.21.15.162:0/973094788 <== mon.0 v2:172.21.15.162:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6582+0+0 (secure 0 0 0) 0x7fbdc408fd50 con 0x7fbdd4103780 2024-04-27T04:20:01.592 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:20:01.593+0000 7fbdda48c700 1 --2- 172.21.15.162:0/973094788 >> [v2:172.21.15.162:6800/1396866108,v1:172.21.15.162:6801/1396866108] conn(0x7fbdc006ec00 0x7fbdc00710b0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fbdd0009b80 tx=0x7fbdd000b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:20:01.595 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:20:01.598+0000 7fbdcaffd700 1 -- 172.21.15.162:0/973094788 <== mon.0 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7fbdc405c650 con 0x7fbdd4103780 2024-04-27T04:20:01.835 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:20:01.837+0000 7fbddc6f0700 1 -- 172.21.15.162:0/973094788 --> [v2:172.21.15.162:6800/1396866108,v1:172.21.15.162:6801/1396866108] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fbdd4109ec0 con 0x7fbdc006ec00 2024-04-27T04:20:01.836 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:20:01.838+0000 7fbdcaffd700 1 -- 172.21.15.162:0/973094788 <== mgr.34104 v2:172.21.15.162:6800/1396866108 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7fbdd4109ec0 con 0x7fbdc006ec00 2024-04-27T04:20:01.838 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:20:01.840+0000 7fbddc6f0700 1 -- 172.21.15.162:0/973094788 >> [v2:172.21.15.162:6800/1396866108,v1:172.21.15.162:6801/1396866108] conn(0x7fbdc006ec00 msgr2=0x7fbdc00710b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:20:01.838 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:20:01.840+0000 7fbddc6f0700 1 --2- 172.21.15.162:0/973094788 >> [v2:172.21.15.162:6800/1396866108,v1:172.21.15.162:6801/1396866108] conn(0x7fbdc006ec00 0x7fbdc00710b0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fbdd0009b80 tx=0x7fbdd000b040 comp rx=0 tx=0).stop 2024-04-27T04:20:01.838 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:20:01.840+0000 7fbddc6f0700 1 -- 172.21.15.162:0/973094788 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fbdd4103780 msgr2=0x7fbdd4113e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:20:01.838 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:20:01.840+0000 7fbddc6f0700 1 --2- 172.21.15.162:0/973094788 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fbdd4103780 0x7fbdd4113e10 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fbdc4000c00 tx=0x7fbdc4009e30 comp rx=0 tx=0).stop 2024-04-27T04:20:01.838 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:20:01.840+0000 7fbddc6f0700 1 -- 172.21.15.162:0/973094788 shutdown_connections 2024-04-27T04:20:01.838 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:20:01.840+0000 7fbddc6f0700 1 --2- 172.21.15.162:0/973094788 >> [v2:172.21.15.162:6800/1396866108,v1:172.21.15.162:6801/1396866108] conn(0x7fbdc006ec00 0x7fbdc00710b0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:20:01.838 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:20:01.840+0000 7fbddc6f0700 1 --2- 172.21.15.162:0/973094788 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbdd4102de0 0x7fbdd41138d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:20:01.839 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:20:01.840+0000 7fbddc6f0700 1 --2- 172.21.15.162:0/973094788 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fbdd4103780 0x7fbdd4113e10 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:20:01.839 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:20:01.840+0000 7fbddc6f0700 1 -- 172.21.15.162:0/973094788 >> 172.21.15.162:0/973094788 conn(0x7fbdd40f8990 msgr2=0x7fbdd40f98a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:20:01.839 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:20:01 smithi162 ceph-mon[110641]: mds.? [v2:172.21.15.162:6826/1198130667,v1:172.21.15.162:6827/1198130667] up:rejoin 2024-04-27T04:20:01.839 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:20:01 smithi162 ceph-mon[110641]: fsmap cephfs:1/1 {0=cephfs.smithi162.gzlyjj=up:rejoin} 3 up:standby 2024-04-27T04:20:01.839 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:20:01 smithi162 ceph-mon[110641]: daemon mds.cephfs.smithi162.gzlyjj is now active in filesystem cephfs as rank 0 2024-04-27T04:20:01.839 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:20:01.840+0000 7fbddc6f0700 1 -- 172.21.15.162:0/973094788 shutdown_connections 2024-04-27T04:20:01.840 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:20:01.841+0000 7fbddc6f0700 1 -- 172.21.15.162:0/973094788 wait complete. 2024-04-27T04:20:01.849 INFO:teuthology.orchestra.run.smithi162.stdout:true 2024-04-27T04:20:01.946 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:20:01.947+0000 7f269262a700 1 -- 172.21.15.162:0/3246612431 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f268c0fd0a0 msgr2=0x7f268c109350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:20:01.947 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:20:01.947+0000 7f269262a700 1 --2- 172.21.15.162:0/3246612431 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f268c0fd0a0 0x7f268c109350 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f2680004940 tx=0x7f2680009e30 comp rx=0 tx=0).stop 2024-04-27T04:20:01.947 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:20:01.947+0000 7f269262a700 1 -- 172.21.15.162:0/3246612431 shutdown_connections 2024-04-27T04:20:01.947 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:20:01.947+0000 7f269262a700 1 --2- 172.21.15.162:0/3246612431 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f268c0fd0a0 0x7f268c109350 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:20:01.947 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:20:01.947+0000 7f269262a700 1 --2- 172.21.15.162:0/3246612431 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f268c0fc790 0x7f268c0fcb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:20:01.947 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:20:01.947+0000 7f269262a700 1 -- 172.21.15.162:0/3246612431 >> 172.21.15.162:0/3246612431 conn(0x7f268c071d00 msgr2=0x7f268c072100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:20:01.947 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:20:01.947+0000 7f269262a700 1 -- 172.21.15.162:0/3246612431 shutdown_connections 2024-04-27T04:20:01.947 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:20:01.947+0000 7f269262a700 1 -- 172.21.15.162:0/3246612431 wait complete. 2024-04-27T04:20:01.947 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:20:01.947+0000 7f269262a700 1 Processor -- start 2024-04-27T04:20:01.947 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:20:01.948+0000 7f269262a700 1 -- start start 2024-04-27T04:20:01.947 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:20:01.948+0000 7f269262a700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f268c0fc790 0x7f268c115e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:20:01.947 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:20:01.948+0000 7f269262a700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f268c0fd0a0 0x7f268c116340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:20:01.947 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:20:01.948+0000 7f269262a700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f268c116990 con 0x7f268c0fc790 2024-04-27T04:20:01.948 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:20:01.948+0000 7f269262a700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f268c10fdb0 con 0x7f268c0fd0a0 2024-04-27T04:20:01.948 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:20:01.948+0000 7f268bfff700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f268c0fc790 0x7f268c115e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:20:01.948 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:20:01.948+0000 7f268bfff700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f268c0fc790 0x7f268c115e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:58102/0 (socket says 172.21.15.162:58102) 2024-04-27T04:20:01.948 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:20:01.948+0000 7f268bfff700 1 -- 172.21.15.162:0/836832661 learned_addr learned my addr 172.21.15.162:0/836832661 (peer_addr_for_me v2:172.21.15.162:0/0) 2024-04-27T04:20:01.948 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:20:01.948+0000 7f268b7fe700 1 --2- 172.21.15.162:0/836832661 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f268c0fd0a0 0x7f268c116340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:20:01.948 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:20:01.948+0000 7f268bfff700 1 -- 172.21.15.162:0/836832661 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f268c0fd0a0 msgr2=0x7f268c116340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:20:01.948 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:20:01.948+0000 7f268bfff700 1 --2- 172.21.15.162:0/836832661 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f268c0fd0a0 0x7f268c116340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:20:01.948 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:20:01.948+0000 7f268bfff700 1 -- 172.21.15.162:0/836832661 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2680005040 con 0x7f268c0fc790 2024-04-27T04:20:01.948 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:20:01.948+0000 7f268bfff700 1 --2- 172.21.15.162:0/836832661 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f268c0fc790 0x7f268c115e00 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f267c002a50 tx=0x7f267c0046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:20:01.948 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:20:01.949+0000 7f2688ff9700 1 -- 172.21.15.162:0/836832661 <== mon.0 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f267c007cc0 con 0x7f268c0fc790 2024-04-27T04:20:01.948 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:20:01.949+0000 7f2688ff9700 1 -- 172.21.15.162:0/836832661 <== mon.0 v2:172.21.15.162:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f267c00f040 con 0x7f268c0fc790 2024-04-27T04:20:01.948 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:20:01.949+0000 7f269262a700 1 -- 172.21.15.162:0/836832661 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f268c110090 con 0x7f268c0fc790 2024-04-27T04:20:01.948 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:20:01.949+0000 7f2688ff9700 1 -- 172.21.15.162:0/836832661 <== mon.0 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f267c00da30 con 0x7f268c0fc790 2024-04-27T04:20:01.948 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:20:01.949+0000 7f269262a700 1 -- 172.21.15.162:0/836832661 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f268c1105d0 con 0x7f268c0fc790 2024-04-27T04:20:01.949 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:20:01.950+0000 7f269262a700 1 -- 172.21.15.162:0/836832661 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f268c0fc330 con 0x7f268c0fc790 2024-04-27T04:20:01.950 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:20:01.950+0000 7f2688ff9700 1 -- 172.21.15.162:0/836832661 <== mon.0 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 39) v1 ==== 92747+0+0 (secure 0 0 0) 0x7f267c00db90 con 0x7f268c0fc790 2024-04-27T04:20:01.950 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:20:01.951+0000 7f2688ff9700 1 --2- 172.21.15.162:0/836832661 >> [v2:172.21.15.162:6800/1396866108,v1:172.21.15.162:6801/1396866108] conn(0x7f267406ec10 0x7f26740710c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:20:01.950 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:20:01.951+0000 7f2688ff9700 1 -- 172.21.15.162:0/836832661 <== mon.0 v2:172.21.15.162:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6582+0+0 (secure 0 0 0) 0x7f267c08e5b0 con 0x7f268c0fc790 2024-04-27T04:20:01.950 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:20:01.951+0000 7f268b7fe700 1 --2- 172.21.15.162:0/836832661 >> [v2:172.21.15.162:6800/1396866108,v1:172.21.15.162:6801/1396866108] conn(0x7f267406ec10 0x7f26740710c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:20:01.950 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:20:01.951+0000 7f268b7fe700 1 --2- 172.21.15.162:0/836832661 >> [v2:172.21.15.162:6800/1396866108,v1:172.21.15.162:6801/1396866108] conn(0x7f267406ec10 0x7f26740710c0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f268c1116a0 tx=0x7f2680009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:20:01.953 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:20:01.955+0000 7f2688ff9700 1 -- 172.21.15.162:0/836832661 <== mon.0 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7f267c05aeb0 con 0x7f268c0fc790 2024-04-27T04:20:02.171 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:20:01 smithi181 ceph-mon[81134]: mds.? [v2:172.21.15.162:6826/1198130667,v1:172.21.15.162:6827/1198130667] up:rejoin 2024-04-27T04:20:02.172 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:20:01 smithi181 ceph-mon[81134]: fsmap cephfs:1/1 {0=cephfs.smithi162.gzlyjj=up:rejoin} 3 up:standby 2024-04-27T04:20:02.172 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:20:01 smithi181 ceph-mon[81134]: daemon mds.cephfs.smithi162.gzlyjj is now active in filesystem cephfs as rank 0 2024-04-27T04:20:02.179 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:20:02.182+0000 7f269262a700 1 -- 172.21.15.162:0/836832661 --> [v2:172.21.15.162:6800/1396866108,v1:172.21.15.162:6801/1396866108] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f268c04bed0 con 0x7f267406ec10 2024-04-27T04:20:02.181 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:20:02.183+0000 7f2688ff9700 1 -- 172.21.15.162:0/836832661 <== mgr.34104 v2:172.21.15.162:6800/1396866108 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f268c04bed0 con 0x7f267406ec10 2024-04-27T04:20:02.183 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:20:02.185+0000 7f269262a700 1 -- 172.21.15.162:0/836832661 >> [v2:172.21.15.162:6800/1396866108,v1:172.21.15.162:6801/1396866108] conn(0x7f267406ec10 msgr2=0x7f26740710c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:20:02.184 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:20:02.185+0000 7f269262a700 1 --2- 172.21.15.162:0/836832661 >> [v2:172.21.15.162:6800/1396866108,v1:172.21.15.162:6801/1396866108] conn(0x7f267406ec10 0x7f26740710c0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f268c1116a0 tx=0x7f2680009e30 comp rx=0 tx=0).stop 2024-04-27T04:20:02.184 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:20:02.185+0000 7f269262a700 1 -- 172.21.15.162:0/836832661 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f268c0fc790 msgr2=0x7f268c115e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:20:02.184 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:20:02.185+0000 7f269262a700 1 --2- 172.21.15.162:0/836832661 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f268c0fc790 0x7f268c115e00 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f267c002a50 tx=0x7f267c0046b0 comp rx=0 tx=0).stop 2024-04-27T04:20:02.184 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:20:02.185+0000 7f269262a700 1 -- 172.21.15.162:0/836832661 shutdown_connections 2024-04-27T04:20:02.184 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:20:02.185+0000 7f269262a700 1 --2- 172.21.15.162:0/836832661 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f268c0fc790 0x7f268c115e00 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:20:02.184 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:20:02.185+0000 7f269262a700 1 --2- 172.21.15.162:0/836832661 >> [v2:172.21.15.162:6800/1396866108,v1:172.21.15.162:6801/1396866108] conn(0x7f267406ec10 0x7f26740710c0 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:20:02.184 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:20:02.185+0000 7f269262a700 1 --2- 172.21.15.162:0/836832661 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f268c0fd0a0 0x7f268c116340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:20:02.184 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:20:02.185+0000 7f269262a700 1 -- 172.21.15.162:0/836832661 >> 172.21.15.162:0/836832661 conn(0x7f268c071d00 msgr2=0x7f268c0f9d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:20:02.184 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:20:02.186+0000 7f269262a700 1 -- 172.21.15.162:0/836832661 shutdown_connections 2024-04-27T04:20:02.184 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:20:02.186+0000 7f269262a700 1 -- 172.21.15.162:0/836832661 wait complete. 2024-04-27T04:20:02.290 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:20:02.291+0000 7faf8f80f700 1 -- 172.21.15.162:0/1034805900 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7faf88103780 msgr2=0x7faf880fcdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:20:02.290 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:20:02.291+0000 7faf8f80f700 1 --2- 172.21.15.162:0/1034805900 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7faf88103780 0x7faf880fcdf0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7faf78004940 tx=0x7faf78009e30 comp rx=0 tx=0).stop 2024-04-27T04:20:02.290 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:20:02.291+0000 7faf8f80f700 1 -- 172.21.15.162:0/1034805900 shutdown_connections 2024-04-27T04:20:02.290 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:20:02.291+0000 7faf8f80f700 1 --2- 172.21.15.162:0/1034805900 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7faf88103780 0x7faf880fcdf0 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:20:02.290 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:20:02.291+0000 7faf8f80f700 1 --2- 172.21.15.162:0/1034805900 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7faf88102de0 0x7faf881031b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:20:02.290 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:20:02.291+0000 7faf8f80f700 1 -- 172.21.15.162:0/1034805900 >> 172.21.15.162:0/1034805900 conn(0x7faf880f89b0 msgr2=0x7faf880fadc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:20:02.291 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:20:02.292+0000 7faf8f80f700 1 -- 172.21.15.162:0/1034805900 shutdown_connections 2024-04-27T04:20:02.291 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:20:02.292+0000 7faf8f80f700 1 -- 172.21.15.162:0/1034805900 wait complete. 2024-04-27T04:20:02.291 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:20:02.292+0000 7faf8f80f700 1 Processor -- start 2024-04-27T04:20:02.291 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:20:02.292+0000 7faf8f80f700 1 -- start start 2024-04-27T04:20:02.291 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:20:02.292+0000 7faf8f80f700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7faf88102de0 0x7faf88113ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:20:02.291 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:20:02.292+0000 7faf8f80f700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7faf88103780 0x7faf88113ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:20:02.291 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:20:02.292+0000 7faf8f80f700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf88114640 con 0x7faf88102de0 2024-04-27T04:20:02.291 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:20:02.292+0000 7faf8f80f700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf8810da30 con 0x7faf88103780 2024-04-27T04:20:02.291 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:20:02.292+0000 7faf8d5ab700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7faf88102de0 0x7faf88113ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:20:02.291 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:20:02.292+0000 7faf8d5ab700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7faf88102de0 0x7faf88113ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:58112/0 (socket says 172.21.15.162:58112) 2024-04-27T04:20:02.291 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:20:02.293+0000 7faf8d5ab700 1 -- 172.21.15.162:0/1149502837 learned_addr learned my addr 172.21.15.162:0/1149502837 (peer_addr_for_me v2:172.21.15.162:0/0) 2024-04-27T04:20:02.291 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:20:02.293+0000 7faf8cdaa700 1 --2- 172.21.15.162:0/1149502837 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7faf88103780 0x7faf88113ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:20:02.292 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:20:02.293+0000 7faf8d5ab700 1 -- 172.21.15.162:0/1149502837 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7faf88103780 msgr2=0x7faf88113ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:20:02.293 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:20:02.293+0000 7faf8d5ab700 1 --2- 172.21.15.162:0/1149502837 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7faf88103780 0x7faf88113ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:20:02.293 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:20:02.293+0000 7faf8d5ab700 1 -- 172.21.15.162:0/1149502837 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf78005040 con 0x7faf88102de0 2024-04-27T04:20:02.293 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:20:02.293+0000 7faf8d5ab700 1 --2- 172.21.15.162:0/1149502837 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7faf88102de0 0x7faf88113ab0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7faf8400eb40 tx=0x7faf8400b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:20:02.293 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:20:02.293+0000 7faf7dffb700 1 -- 172.21.15.162:0/1149502837 <== mon.0 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7faf84007cc0 con 0x7faf88102de0 2024-04-27T04:20:02.293 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:20:02.293+0000 7faf8f80f700 1 -- 172.21.15.162:0/1149502837 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faf8810dcb0 con 0x7faf88102de0 2024-04-27T04:20:02.293 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:20:02.293+0000 7faf7dffb700 1 -- 172.21.15.162:0/1149502837 <== mon.0 v2:172.21.15.162:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7faf8400f040 con 0x7faf88102de0 2024-04-27T04:20:02.293 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:20:02.293+0000 7faf7dffb700 1 -- 172.21.15.162:0/1149502837 <== mon.0 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7faf8400ca10 con 0x7faf88102de0 2024-04-27T04:20:02.293 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:20:02.293+0000 7faf8f80f700 1 -- 172.21.15.162:0/1149502837 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faf8810e1c0 con 0x7faf88102de0 2024-04-27T04:20:02.294 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:20:02.294+0000 7faf8f80f700 1 -- 172.21.15.162:0/1149502837 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faf880fe530 con 0x7faf88102de0 2024-04-27T04:20:02.294 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:20:02.295+0000 7faf7dffb700 1 -- 172.21.15.162:0/1149502837 <== mon.0 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 39) v1 ==== 92747+0+0 (secure 0 0 0) 0x7faf8400cb70 con 0x7faf88102de0 2024-04-27T04:20:02.294 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:20:02.295+0000 7faf7dffb700 1 --2- 172.21.15.162:0/1149502837 >> [v2:172.21.15.162:6800/1396866108,v1:172.21.15.162:6801/1396866108] conn(0x7faf7406ec00 0x7faf740710b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:20:02.295 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:20:02.296+0000 7faf7dffb700 1 -- 172.21.15.162:0/1149502837 <== mon.0 v2:172.21.15.162:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6582+0+0 (secure 0 0 0) 0x7faf8408e3d0 con 0x7faf88102de0 2024-04-27T04:20:02.295 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:20:02.296+0000 7faf8cdaa700 1 --2- 172.21.15.162:0/1149502837 >> [v2:172.21.15.162:6800/1396866108,v1:172.21.15.162:6801/1396866108] conn(0x7faf7406ec00 0x7faf740710b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:20:02.295 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:20:02.296+0000 7faf8cdaa700 1 --2- 172.21.15.162:0/1149502837 >> [v2:172.21.15.162:6800/1396866108,v1:172.21.15.162:6801/1396866108] conn(0x7faf7406ec00 0x7faf740710b0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7faf7800b0d0 tx=0x7faf78009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:20:02.298 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:20:02.300+0000 7faf7dffb700 1 -- 172.21.15.162:0/1149502837 <== mon.0 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7faf8405ad80 con 0x7faf88102de0 2024-04-27T04:20:02.518 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:20:02.520+0000 7faf8f80f700 1 -- 172.21.15.162:0/1149502837 --> [v2:172.21.15.162:6800/1396866108,v1:172.21.15.162:6801/1396866108] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7faf88109ec0 con 0x7faf7406ec00 2024-04-27T04:20:02.530 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:20:02.531+0000 7faf7dffb700 1 -- 172.21.15.162:0/1149502837 <== mgr.34104 v2:172.21.15.162:6800/1396866108 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3498 (secure 0 0 0) 0x7faf88109ec0 con 0x7faf7406ec00 2024-04-27T04:20:02.530 INFO:teuthology.orchestra.run.smithi162.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-27T04:20:02.530 INFO:teuthology.orchestra.run.smithi162.stdout:alertmanager.smithi162 smithi162 *:9093,9094 running (7m) 44s ago 18m 27.0M - 0.25.0 c8568f914cd2 40e334659017 2024-04-27T04:20:02.530 INFO:teuthology.orchestra.run.smithi162.stdout:crash.smithi162 smithi162 running (4m) 44s ago 18m 6660k - 19.0.0-3253-g4e213a58 67e414ff4bc4 1aa8f4d5a28b 2024-04-27T04:20:02.530 INFO:teuthology.orchestra.run.smithi162.stdout:crash.smithi181 smithi181 running (4m) 0s ago 17m 6656k - 19.0.0-3253-g4e213a58 67e414ff4bc4 832d9cbc5c30 2024-04-27T04:20:02.530 INFO:teuthology.orchestra.run.smithi162.stdout:grafana.smithi162 smithi162 *:3000 running (6m) 44s ago 18m 79.3M - 9.4.12 2bacad6d85d8 3d78c478f03f 2024-04-27T04:20:02.530 INFO:teuthology.orchestra.run.smithi162.stdout:mds.cephfs.smithi162.gzlyjj smithi162 running (68s) 44s ago 13m 12.5M - 19.0.0-3253-g4e213a58 67e414ff4bc4 7541e8fb8265 2024-04-27T04:20:02.531 INFO:teuthology.orchestra.run.smithi162.stdout:mds.cephfs.smithi162.xcnapk smithi162 running (49s) 44s ago 12m 11.5M - 19.0.0-3253-g4e213a58 67e414ff4bc4 3af4597febc8 2024-04-27T04:20:02.531 INFO:teuthology.orchestra.run.smithi162.stdout:mds.cephfs.smithi181.bkkcdm smithi181 running (3s) 0s ago 12m 14.0M - 19.0.0-3253-g4e213a58 67e414ff4bc4 87e3901b7571 2024-04-27T04:20:02.531 INFO:teuthology.orchestra.run.smithi162.stdout:mds.cephfs.smithi181.dzomgh smithi181 running (24s) 0s ago 13m 12.7M - 19.0.0-3253-g4e213a58 67e414ff4bc4 a07f9164b260 2024-04-27T04:20:02.531 INFO:teuthology.orchestra.run.smithi162.stdout:mgr.smithi162.oennfn smithi162 *:8443,9283,8765 running (8m) 44s ago 19m 511M - 19.0.0-3253-g4e213a58 67e414ff4bc4 58a748d3637b 2024-04-27T04:20:02.531 INFO:teuthology.orchestra.run.smithi162.stdout:mgr.smithi181.vtwuhz smithi181 *:8443,9283,8765 running (8m) 0s ago 17m 452M - 19.0.0-3253-g4e213a58 67e414ff4bc4 813584297b1d 2024-04-27T04:20:02.531 INFO:teuthology.orchestra.run.smithi162.stdout:mon.smithi162 smithi162 running (5m) 44s ago 19m 51.9M 2048M 19.0.0-3253-g4e213a58 67e414ff4bc4 375adaf4468f 2024-04-27T04:20:02.531 INFO:teuthology.orchestra.run.smithi162.stdout:mon.smithi181 smithi181 running (5m) 0s ago 17m 41.7M 2048M 19.0.0-3253-g4e213a58 67e414ff4bc4 77b71f6d522f 2024-04-27T04:20:02.531 INFO:teuthology.orchestra.run.smithi162.stdout:node-exporter.smithi162 smithi162 *:9100 running (7m) 44s ago 18m 15.5M - 1.5.0 0da6a335fe13 5c4c5ea9c1d7 2024-04-27T04:20:02.531 INFO:teuthology.orchestra.run.smithi162.stdout:node-exporter.smithi181 smithi181 *:9100 running (7m) 0s ago 17m 15.3M - 1.5.0 0da6a335fe13 9067f14c0519 2024-04-27T04:20:02.531 INFO:teuthology.orchestra.run.smithi162.stdout:osd.0 smithi162 running (4m) 44s ago 16m 93.1M 1557M 19.0.0-3253-g4e213a58 67e414ff4bc4 2744aff3cd88 2024-04-27T04:20:02.531 INFO:teuthology.orchestra.run.smithi162.stdout:osd.1 smithi162 running (3m) 44s ago 16m 96.6M 1557M 19.0.0-3253-g4e213a58 67e414ff4bc4 668e16faad62 2024-04-27T04:20:02.531 INFO:teuthology.orchestra.run.smithi162.stdout:osd.2 smithi162 running (3m) 44s ago 15m 87.4M 1557M 19.0.0-3253-g4e213a58 67e414ff4bc4 d24878d461e7 2024-04-27T04:20:02.531 INFO:teuthology.orchestra.run.smithi162.stdout:osd.3 smithi181 running (2m) 0s ago 15m 122M 2581M 19.0.0-3253-g4e213a58 67e414ff4bc4 ab473cfd54d2 2024-04-27T04:20:02.532 INFO:teuthology.orchestra.run.smithi162.stdout:osd.4 smithi181 running (2m) 0s ago 14m 101M 2581M 19.0.0-3253-g4e213a58 67e414ff4bc4 de3e10f2d26d 2024-04-27T04:20:02.532 INFO:teuthology.orchestra.run.smithi162.stdout:osd.5 smithi181 running (87s) 0s ago 14m 103M 2581M 19.0.0-3253-g4e213a58 67e414ff4bc4 7dd4f287ef2e 2024-04-27T04:20:02.532 INFO:teuthology.orchestra.run.smithi162.stdout:prometheus.smithi162 smithi162 *:9095 running (7m) 44s ago 17m 67.7M - 2.43.0 a07b618ecd1d f30657a43324 2024-04-27T04:20:02.533 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:20:02.534+0000 7faf8f80f700 1 -- 172.21.15.162:0/1149502837 >> [v2:172.21.15.162:6800/1396866108,v1:172.21.15.162:6801/1396866108] conn(0x7faf7406ec00 msgr2=0x7faf740710b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:20:02.533 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:20:02.534+0000 7faf8f80f700 1 --2- 172.21.15.162:0/1149502837 >> [v2:172.21.15.162:6800/1396866108,v1:172.21.15.162:6801/1396866108] conn(0x7faf7406ec00 0x7faf740710b0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7faf7800b0d0 tx=0x7faf78009e30 comp rx=0 tx=0).stop 2024-04-27T04:20:02.533 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:20:02.534+0000 7faf8f80f700 1 -- 172.21.15.162:0/1149502837 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7faf88102de0 msgr2=0x7faf88113ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:20:02.533 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:20:02.534+0000 7faf8f80f700 1 --2- 172.21.15.162:0/1149502837 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7faf88102de0 0x7faf88113ab0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7faf8400eb40 tx=0x7faf8400b040 comp rx=0 tx=0).stop 2024-04-27T04:20:02.533 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:20:02.534+0000 7faf8f80f700 1 -- 172.21.15.162:0/1149502837 shutdown_connections 2024-04-27T04:20:02.533 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:20:02.534+0000 7faf8f80f700 1 --2- 172.21.15.162:0/1149502837 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7faf88102de0 0x7faf88113ab0 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:20:02.534 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:20:02.534+0000 7faf8f80f700 1 --2- 172.21.15.162:0/1149502837 >> [v2:172.21.15.162:6800/1396866108,v1:172.21.15.162:6801/1396866108] conn(0x7faf7406ec00 0x7faf740710b0 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:20:02.534 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:20:02.534+0000 7faf8f80f700 1 --2- 172.21.15.162:0/1149502837 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7faf88103780 0x7faf88113ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:20:02.534 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:20:02.534+0000 7faf8f80f700 1 -- 172.21.15.162:0/1149502837 >> 172.21.15.162:0/1149502837 conn(0x7faf880f89b0 msgr2=0x7faf880f98f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:20:02.534 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:20:02.534+0000 7faf8f80f700 1 -- 172.21.15.162:0/1149502837 shutdown_connections 2024-04-27T04:20:02.534 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:20:02.534+0000 7faf8f80f700 1 -- 172.21.15.162:0/1149502837 wait complete. 2024-04-27T04:20:02.638 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:20:02.637+0000 7eff90a27700 1 -- 172.21.15.162:0/1748783633 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7eff8c102680 msgr2=0x7eff8c0fbcf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:20:02.638 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:20:02.637+0000 7eff90a27700 1 --2- 172.21.15.162:0/1748783633 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7eff8c102680 0x7eff8c0fbcf0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7eff80004940 tx=0x7eff80009e30 comp rx=0 tx=0).stop 2024-04-27T04:20:02.639 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:20:02.638+0000 7eff90a27700 1 -- 172.21.15.162:0/1748783633 shutdown_connections 2024-04-27T04:20:02.639 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:20:02.638+0000 7eff90a27700 1 --2- 172.21.15.162:0/1748783633 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7eff8c102680 0x7eff8c0fbcf0 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:20:02.639 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:20:02.638+0000 7eff90a27700 1 --2- 172.21.15.162:0/1748783633 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7eff8c101ce0 0x7eff8c1020b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:20:02.639 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:20:02.638+0000 7eff90a27700 1 -- 172.21.15.162:0/1748783633 >> 172.21.15.162:0/1748783633 conn(0x7eff8c0f7850 msgr2=0x7eff8c0f9c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:20:02.639 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:20:02.638+0000 7eff90a27700 1 -- 172.21.15.162:0/1748783633 shutdown_connections 2024-04-27T04:20:02.639 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:20:02.638+0000 7eff90a27700 1 -- 172.21.15.162:0/1748783633 wait complete. 2024-04-27T04:20:02.639 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:20:02.638+0000 7eff90a27700 1 Processor -- start 2024-04-27T04:20:02.640 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:20:02.638+0000 7eff90a27700 1 -- start start 2024-04-27T04:20:02.640 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:20:02.638+0000 7eff90a27700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7eff8c101ce0 0x7eff8c10eae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:20:02.640 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:20:02.638+0000 7eff90a27700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7eff8c102680 0x7eff8c10f020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:20:02.640 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:20:02.638+0000 7eff90a27700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff8c10f670 con 0x7eff8c102680 2024-04-27T04:20:02.640 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:20:02.638+0000 7eff90a27700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff8c10f7b0 con 0x7eff8c101ce0 2024-04-27T04:20:02.640 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:20:02.639+0000 7eff89d9b700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7eff8c102680 0x7eff8c10f020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:20:02.641 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:20:02.639+0000 7eff8a59c700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7eff8c101ce0 0x7eff8c10eae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:20:02.641 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:20:02.639+0000 7eff89d9b700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7eff8c102680 0x7eff8c10f020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:58136/0 (socket says 172.21.15.162:58136) 2024-04-27T04:20:02.641 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:20:02.639+0000 7eff89d9b700 1 -- 172.21.15.162:0/2415032628 learned_addr learned my addr 172.21.15.162:0/2415032628 (peer_addr_for_me v2:172.21.15.162:0/0) 2024-04-27T04:20:02.641 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:20:02.639+0000 7eff89d9b700 1 -- 172.21.15.162:0/2415032628 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7eff8c101ce0 msgr2=0x7eff8c10eae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:20:02.641 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:20:02.639+0000 7eff89d9b700 1 --2- 172.21.15.162:0/2415032628 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7eff8c101ce0 0x7eff8c10eae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:20:02.641 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:20:02.639+0000 7eff89d9b700 1 -- 172.21.15.162:0/2415032628 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7eff80005040 con 0x7eff8c102680 2024-04-27T04:20:02.642 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:20:02.639+0000 7eff8a59c700 1 --2- 172.21.15.162:0/2415032628 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7eff8c101ce0 0x7eff8c10eae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-27T04:20:02.642 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:20:02.639+0000 7eff89d9b700 1 --2- 172.21.15.162:0/2415032628 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7eff8c102680 0x7eff8c10f020 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7eff80000c00 tx=0x7eff80009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:20:02.642 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:20:02.639+0000 7eff7affd700 1 -- 172.21.15.162:0/2415032628 <== mon.0 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7eff80021070 con 0x7eff8c102680 2024-04-27T04:20:02.642 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:20:02.639+0000 7eff7affd700 1 -- 172.21.15.162:0/2415032628 <== mon.0 v2:172.21.15.162:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7eff80013c00 con 0x7eff8c102680 2024-04-27T04:20:02.642 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:20:02.639+0000 7eff90a27700 1 -- 172.21.15.162:0/2415032628 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7eff8c117b50 con 0x7eff8c102680 2024-04-27T04:20:02.643 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:20:02.639+0000 7eff90a27700 1 -- 172.21.15.162:0/2415032628 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7eff8c117fb0 con 0x7eff8c102680 2024-04-27T04:20:02.643 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:20:02.640+0000 7eff7affd700 1 -- 172.21.15.162:0/2415032628 <== mon.0 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7eff8000e070 con 0x7eff8c102680 2024-04-27T04:20:02.643 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:20:02.640+0000 7eff90a27700 1 -- 172.21.15.162:0/2415032628 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7eff6c0052b0 con 0x7eff8c102680 2024-04-27T04:20:02.644 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:20:02.641+0000 7eff7affd700 1 -- 172.21.15.162:0/2415032628 <== mon.0 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 39) v1 ==== 92747+0+0 (secure 0 0 0) 0x7eff8000b6e0 con 0x7eff8c102680 2024-04-27T04:20:02.645 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:20:02.641+0000 7eff7affd700 1 --2- 172.21.15.162:0/2415032628 >> [v2:172.21.15.162:6800/1396866108,v1:172.21.15.162:6801/1396866108] conn(0x7eff7406ec00 0x7eff740710b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:20:02.645 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:20:02.641+0000 7eff7affd700 1 -- 172.21.15.162:0/2415032628 <== mon.0 v2:172.21.15.162:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6582+0+0 (secure 0 0 0) 0x7eff8008fc80 con 0x7eff8c102680 2024-04-27T04:20:02.645 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:20:02.641+0000 7eff8a59c700 1 --2- 172.21.15.162:0/2415032628 >> [v2:172.21.15.162:6800/1396866108,v1:172.21.15.162:6801/1396866108] conn(0x7eff7406ec00 0x7eff740710b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:20:02.645 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:20:02.642+0000 7eff8a59c700 1 --2- 172.21.15.162:0/2415032628 >> [v2:172.21.15.162:6800/1396866108,v1:172.21.15.162:6801/1396866108] conn(0x7eff7406ec00 0x7eff740710b0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7eff7c009b80 tx=0x7eff7c00b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:20:02.645 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:20:02.646+0000 7eff7affd700 1 -- 172.21.15.162:0/2415032628 <== mon.0 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7eff80058c10 con 0x7eff8c102680 2024-04-27T04:20:02.886 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:20:02 smithi162 ceph-mon[110641]: pgmap v210: 65 pgs: 65 active+clean; 208 MiB data, 851 MiB used, 536 GiB / 536 GiB avail; 18 MiB/s rd, 5 op/s 2024-04-27T04:20:02.887 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:20:02 smithi162 ceph-mon[110641]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-04-27T04:20:02.887 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:20:02 smithi162 ceph-mon[110641]: Cluster is now healthy 2024-04-27T04:20:02.887 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:20:02 smithi162 ceph-mon[110641]: mds.? [v2:172.21.15.162:6826/1198130667,v1:172.21.15.162:6827/1198130667] up:active 2024-04-27T04:20:02.887 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:20:02 smithi162 ceph-mon[110641]: fsmap cephfs:1 {0=cephfs.smithi162.gzlyjj=up:active} 3 up:standby 2024-04-27T04:20:02.887 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:20:02 smithi162 ceph-mon[110641]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' 2024-04-27T04:20:02.888 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:20:02 smithi162 ceph-mon[110641]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' 2024-04-27T04:20:02.888 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:20:02 smithi162 ceph-mon[110641]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-27T04:20:02.942 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:20:02.944+0000 7eff90a27700 1 -- 172.21.15.162:0/2415032628 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7eff6c005d70 con 0x7eff8c102680 2024-04-27T04:20:02.942 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:20:02.944+0000 7eff7affd700 1 -- 172.21.15.162:0/2415032628 <== mon.0 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+623 (secure 0 0 0) 0x7eff80004490 con 0x7eff8c102680 2024-04-27T04:20:02.943 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-04-27T04:20:02.943 INFO:teuthology.orchestra.run.smithi162.stdout: "mon": { 2024-04-27T04:20:02.943 INFO:teuthology.orchestra.run.smithi162.stdout: "ceph version 19.0.0-3253-g4e213a58 (4e213a58e787437ff508e6c890ccd5fd32dc96b5) squid (dev)": 2 2024-04-27T04:20:02.943 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-04-27T04:20:02.943 INFO:teuthology.orchestra.run.smithi162.stdout: "mgr": { 2024-04-27T04:20:02.943 INFO:teuthology.orchestra.run.smithi162.stdout: "ceph version 19.0.0-3253-g4e213a58 (4e213a58e787437ff508e6c890ccd5fd32dc96b5) squid (dev)": 2 2024-04-27T04:20:02.943 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-04-27T04:20:02.943 INFO:teuthology.orchestra.run.smithi162.stdout: "osd": { 2024-04-27T04:20:02.944 INFO:teuthology.orchestra.run.smithi162.stdout: "ceph version 19.0.0-3253-g4e213a58 (4e213a58e787437ff508e6c890ccd5fd32dc96b5) squid (dev)": 6 2024-04-27T04:20:02.944 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-04-27T04:20:02.944 INFO:teuthology.orchestra.run.smithi162.stdout: "mds": { 2024-04-27T04:20:02.944 INFO:teuthology.orchestra.run.smithi162.stdout: "ceph version 19.0.0-3253-g4e213a58 (4e213a58e787437ff508e6c890ccd5fd32dc96b5) squid (dev)": 4 2024-04-27T04:20:02.944 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-04-27T04:20:02.944 INFO:teuthology.orchestra.run.smithi162.stdout: "overall": { 2024-04-27T04:20:02.944 INFO:teuthology.orchestra.run.smithi162.stdout: "ceph version 19.0.0-3253-g4e213a58 (4e213a58e787437ff508e6c890ccd5fd32dc96b5) squid (dev)": 14 2024-04-27T04:20:02.944 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-04-27T04:20:02.945 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-04-27T04:20:02.945 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:20:02.947+0000 7eff90a27700 1 -- 172.21.15.162:0/2415032628 >> [v2:172.21.15.162:6800/1396866108,v1:172.21.15.162:6801/1396866108] conn(0x7eff7406ec00 msgr2=0x7eff740710b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:20:02.945 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:20:02.947+0000 7eff90a27700 1 --2- 172.21.15.162:0/2415032628 >> [v2:172.21.15.162:6800/1396866108,v1:172.21.15.162:6801/1396866108] conn(0x7eff7406ec00 0x7eff740710b0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7eff7c009b80 tx=0x7eff7c00b040 comp rx=0 tx=0).stop 2024-04-27T04:20:02.945 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:20:02.947+0000 7eff90a27700 1 -- 172.21.15.162:0/2415032628 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7eff8c102680 msgr2=0x7eff8c10f020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:20:02.945 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:20:02.947+0000 7eff90a27700 1 --2- 172.21.15.162:0/2415032628 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7eff8c102680 0x7eff8c10f020 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7eff80000c00 tx=0x7eff80009e30 comp rx=0 tx=0).stop 2024-04-27T04:20:02.946 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:20:02.947+0000 7eff90a27700 1 -- 172.21.15.162:0/2415032628 shutdown_connections 2024-04-27T04:20:02.946 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:20:02.947+0000 7eff90a27700 1 --2- 172.21.15.162:0/2415032628 >> [v2:172.21.15.162:6800/1396866108,v1:172.21.15.162:6801/1396866108] conn(0x7eff7406ec00 0x7eff740710b0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:20:02.946 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:20:02.947+0000 7eff90a27700 1 --2- 172.21.15.162:0/2415032628 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7eff8c101ce0 0x7eff8c10eae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:20:02.946 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:20:02.947+0000 7eff90a27700 1 --2- 172.21.15.162:0/2415032628 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7eff8c102680 0x7eff8c10f020 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:20:02.946 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:20:02.947+0000 7eff90a27700 1 -- 172.21.15.162:0/2415032628 >> 172.21.15.162:0/2415032628 conn(0x7eff8c0f7850 msgr2=0x7eff8c0f87e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:20:02.946 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:20:02.947+0000 7eff90a27700 1 -- 172.21.15.162:0/2415032628 shutdown_connections 2024-04-27T04:20:02.946 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:20:02.947+0000 7eff90a27700 1 -- 172.21.15.162:0/2415032628 wait complete. 2024-04-27T04:20:03.051 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:20:03.052+0000 7f1013957700 1 -- 172.21.15.162:0/4056390303 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f100c101d00 msgr2=0x7f100c1020d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:20:03.051 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:20:03.052+0000 7f1013957700 1 --2- 172.21.15.162:0/4056390303 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f100c101d00 0x7f100c1020d0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f1008004940 tx=0x7f1008009e30 comp rx=0 tx=0).stop 2024-04-27T04:20:03.052 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:20:03.052+0000 7f1013957700 1 -- 172.21.15.162:0/4056390303 shutdown_connections 2024-04-27T04:20:03.052 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:20:03.052+0000 7f1013957700 1 --2- 172.21.15.162:0/4056390303 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f100c1026a0 0x7f100c0fbd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:20:03.052 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:20:03.052+0000 7f1013957700 1 --2- 172.21.15.162:0/4056390303 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f100c101d00 0x7f100c1020d0 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:20:03.052 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:20:03.052+0000 7f1013957700 1 -- 172.21.15.162:0/4056390303 >> 172.21.15.162:0/4056390303 conn(0x7f100c0f78d0 msgr2=0x7f100c0f9ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:20:03.052 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:20:03.052+0000 7f1013957700 1 -- 172.21.15.162:0/4056390303 shutdown_connections 2024-04-27T04:20:03.052 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:20:03.052+0000 7f1013957700 1 -- 172.21.15.162:0/4056390303 wait complete. 2024-04-27T04:20:03.052 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:20:03.052+0000 7f1013957700 1 Processor -- start 2024-04-27T04:20:03.052 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:20:03.053+0000 7f1013957700 1 -- start start 2024-04-27T04:20:03.052 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:20:03.053+0000 7f1013957700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f100c101d00 0x7f100c113b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:20:03.052 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:20:03.053+0000 7f1013957700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f100c1026a0 0x7f100c114050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:20:03.052 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:20:03.053+0000 7f1013957700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f100c1146a0 con 0x7f100c101d00 2024-04-27T04:20:03.052 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:20:03.053+0000 7f1013957700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f100c10da90 con 0x7f100c1026a0 2024-04-27T04:20:03.052 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:20:03.053+0000 7f10116f3700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f100c101d00 0x7f100c113b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:20:03.052 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:20:03.053+0000 7f10116f3700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f100c101d00 0x7f100c113b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:58142/0 (socket says 172.21.15.162:58142) 2024-04-27T04:20:03.053 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:20:03.053+0000 7f10116f3700 1 -- 172.21.15.162:0/1246200853 learned_addr learned my addr 172.21.15.162:0/1246200853 (peer_addr_for_me v2:172.21.15.162:0/0) 2024-04-27T04:20:03.053 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:20:03.053+0000 7f1010ef2700 1 --2- 172.21.15.162:0/1246200853 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f100c1026a0 0x7f100c114050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:20:03.053 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:20:03.053+0000 7f10116f3700 1 -- 172.21.15.162:0/1246200853 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f100c1026a0 msgr2=0x7f100c114050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:20:03.053 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:20:03.053+0000 7f10116f3700 1 --2- 172.21.15.162:0/1246200853 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f100c1026a0 0x7f100c114050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:20:03.053 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:20:03.053+0000 7f10116f3700 1 -- 172.21.15.162:0/1246200853 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1008005040 con 0x7f100c101d00 2024-04-27T04:20:03.053 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:20:03.053+0000 7f10116f3700 1 --2- 172.21.15.162:0/1246200853 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f100c101d00 0x7f100c113b10 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f1008009150 tx=0x7f1008009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:20:03.053 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:20:03.053+0000 7f1001ffb700 1 -- 172.21.15.162:0/1246200853 <== mon.0 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1008021070 con 0x7f100c101d00 2024-04-27T04:20:03.053 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:20:03.054+0000 7f1001ffb700 1 -- 172.21.15.162:0/1246200853 <== mon.0 v2:172.21.15.162:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f1008013c00 con 0x7f100c101d00 2024-04-27T04:20:03.053 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:20:03.054+0000 7f1001ffb700 1 -- 172.21.15.162:0/1246200853 <== mon.0 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f100800e070 con 0x7f100c101d00 2024-04-27T04:20:03.053 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:20:03.054+0000 7f1013957700 1 -- 172.21.15.162:0/1246200853 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f100c10dd10 con 0x7f100c101d00 2024-04-27T04:20:03.053 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:20:03.054+0000 7f1013957700 1 -- 172.21.15.162:0/1246200853 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f100c10e1a0 con 0x7f100c101d00 2024-04-27T04:20:03.054 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:20:03.055+0000 7f1013957700 1 -- 172.21.15.162:0/1246200853 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f100c0fd450 con 0x7f100c101d00 2024-04-27T04:20:03.054 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:20:03.055+0000 7f1001ffb700 1 -- 172.21.15.162:0/1246200853 <== mon.0 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 39) v1 ==== 92747+0+0 (secure 0 0 0) 0x7f100801c400 con 0x7f100c101d00 2024-04-27T04:20:03.054 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:20:03.056+0000 7f1001ffb700 1 --2- 172.21.15.162:0/1246200853 >> [v2:172.21.15.162:6800/1396866108,v1:172.21.15.162:6801/1396866108] conn(0x7f0ff806ec10 0x7f0ff80710c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:20:03.054 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:20:03.056+0000 7f1001ffb700 1 -- 172.21.15.162:0/1246200853 <== mon.0 v2:172.21.15.162:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6582+0+0 (secure 0 0 0) 0x7f100808fd70 con 0x7f100c101d00 2024-04-27T04:20:03.054 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:20:03.056+0000 7f1010ef2700 1 --2- 172.21.15.162:0/1246200853 >> [v2:172.21.15.162:6800/1396866108,v1:172.21.15.162:6801/1396866108] conn(0x7f0ff806ec10 0x7f0ff80710c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:20:03.054 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:20:03.056+0000 7f1010ef2700 1 --2- 172.21.15.162:0/1246200853 >> [v2:172.21.15.162:6800/1396866108,v1:172.21.15.162:6801/1396866108] conn(0x7f0ff806ec10 0x7f0ff80710c0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f100c10ec50 tx=0x7f0ffc00b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:20:03.058 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:20:03.060+0000 7f1001ffb700 1 -- 172.21.15.162:0/1246200853 <== mon.0 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7f100805c670 con 0x7f100c101d00 2024-04-27T04:20:03.171 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:20:02 smithi181 ceph-mon[81134]: pgmap v210: 65 pgs: 65 active+clean; 208 MiB data, 851 MiB used, 536 GiB / 536 GiB avail; 18 MiB/s rd, 5 op/s 2024-04-27T04:20:03.172 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:20:02 smithi181 ceph-mon[81134]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-04-27T04:20:03.172 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:20:02 smithi181 ceph-mon[81134]: Cluster is now healthy 2024-04-27T04:20:03.172 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:20:02 smithi181 ceph-mon[81134]: mds.? [v2:172.21.15.162:6826/1198130667,v1:172.21.15.162:6827/1198130667] up:active 2024-04-27T04:20:03.172 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:20:02 smithi181 ceph-mon[81134]: fsmap cephfs:1 {0=cephfs.smithi162.gzlyjj=up:active} 3 up:standby 2024-04-27T04:20:03.172 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:20:02 smithi181 ceph-mon[81134]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' 2024-04-27T04:20:03.172 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:20:02 smithi181 ceph-mon[81134]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' 2024-04-27T04:20:03.172 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:20:02 smithi181 ceph-mon[81134]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-27T04:20:03.307 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:20:03.309+0000 7f1013957700 1 -- 172.21.15.162:0/1246200853 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f100c108e00 con 0x7f100c101d00 2024-04-27T04:20:03.308 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:20:03.310+0000 7f1001ffb700 1 -- 172.21.15.162:0/1246200853 <== mon.0 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 32 v32) v1 ==== 76+0+1942 (secure 0 0 0) 0x7f1008004490 con 0x7f100c101d00 2024-04-27T04:20:03.308 INFO:teuthology.orchestra.run.smithi162.stdout:e32 2024-04-27T04:20:03.308 INFO:teuthology.orchestra.run.smithi162.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-04-27T04:20:03.308 INFO:teuthology.orchestra.run.smithi162.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-27T04:20:03.308 INFO:teuthology.orchestra.run.smithi162.stdout:legacy client fscid: 1 2024-04-27T04:20:03.308 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-04-27T04:20:03.308 INFO:teuthology.orchestra.run.smithi162.stdout:Filesystem 'cephfs' (1) 2024-04-27T04:20:03.308 INFO:teuthology.orchestra.run.smithi162.stdout:fs_name cephfs 2024-04-27T04:20:03.308 INFO:teuthology.orchestra.run.smithi162.stdout:epoch 32 2024-04-27T04:20:03.308 INFO:teuthology.orchestra.run.smithi162.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2024-04-27T04:20:03.308 INFO:teuthology.orchestra.run.smithi162.stdout:created 2024-04-27T04:06:53.095749+0000 2024-04-27T04:20:03.308 INFO:teuthology.orchestra.run.smithi162.stdout:modified 2024-04-27T04:20:01.750233+0000 2024-04-27T04:20:03.309 INFO:teuthology.orchestra.run.smithi162.stdout:tableserver 0 2024-04-27T04:20:03.309 INFO:teuthology.orchestra.run.smithi162.stdout:root 0 2024-04-27T04:20:03.309 INFO:teuthology.orchestra.run.smithi162.stdout:session_timeout 60 2024-04-27T04:20:03.309 INFO:teuthology.orchestra.run.smithi162.stdout:session_autoclose 300 2024-04-27T04:20:03.309 INFO:teuthology.orchestra.run.smithi162.stdout:max_file_size 1099511627776 2024-04-27T04:20:03.309 INFO:teuthology.orchestra.run.smithi162.stdout:max_xattr_size 65536 2024-04-27T04:20:03.309 INFO:teuthology.orchestra.run.smithi162.stdout:required_client_features {} 2024-04-27T04:20:03.309 INFO:teuthology.orchestra.run.smithi162.stdout:last_failure 0 2024-04-27T04:20:03.309 INFO:teuthology.orchestra.run.smithi162.stdout:last_failure_osd_epoch 84 2024-04-27T04:20:03.309 INFO:teuthology.orchestra.run.smithi162.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-04-27T04:20:03.309 INFO:teuthology.orchestra.run.smithi162.stdout:max_mds 1 2024-04-27T04:20:03.309 INFO:teuthology.orchestra.run.smithi162.stdout:in 0 2024-04-27T04:20:03.309 INFO:teuthology.orchestra.run.smithi162.stdout:up {0=34380} 2024-04-27T04:20:03.309 INFO:teuthology.orchestra.run.smithi162.stdout:failed 2024-04-27T04:20:03.309 INFO:teuthology.orchestra.run.smithi162.stdout:damaged 2024-04-27T04:20:03.310 INFO:teuthology.orchestra.run.smithi162.stdout:stopped 2024-04-27T04:20:03.310 INFO:teuthology.orchestra.run.smithi162.stdout:data_pools [3] 2024-04-27T04:20:03.310 INFO:teuthology.orchestra.run.smithi162.stdout:metadata_pool 2 2024-04-27T04:20:03.310 INFO:teuthology.orchestra.run.smithi162.stdout:inline_data disabled 2024-04-27T04:20:03.310 INFO:teuthology.orchestra.run.smithi162.stdout:balancer 2024-04-27T04:20:03.310 INFO:teuthology.orchestra.run.smithi162.stdout:bal_rank_mask -1 2024-04-27T04:20:03.310 INFO:teuthology.orchestra.run.smithi162.stdout:standby_count_wanted 1 2024-04-27T04:20:03.310 INFO:teuthology.orchestra.run.smithi162.stdout:qdb_cluster leader: 34380 members: 34380 2024-04-27T04:20:03.310 INFO:teuthology.orchestra.run.smithi162.stdout:[mds.cephfs.smithi162.gzlyjj{0:34380} state up:active seq 20 join_fscid=1 addr [v2:172.21.15.162:6826/1198130667,v1:172.21.15.162:6827/1198130667] compat {c=[1],r=[1],i=[1fff]}] 2024-04-27T04:20:03.310 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-04-27T04:20:03.310 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-04-27T04:20:03.310 INFO:teuthology.orchestra.run.smithi162.stdout:Standby daemons: 2024-04-27T04:20:03.310 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-04-27T04:20:03.311 INFO:teuthology.orchestra.run.smithi162.stdout:[mds.cephfs.smithi162.xcnapk{-1:34412} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.162:6828/307528848,v1:172.21.15.162:6829/307528848] compat {c=[1],r=[1],i=[1fff]}] 2024-04-27T04:20:03.311 INFO:teuthology.orchestra.run.smithi162.stdout:[mds.cephfs.smithi181.dzomgh{-1:44319} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.181:6824/1667773112,v1:172.21.15.181:6825/1667773112] compat {c=[1],r=[1],i=[1fff]}] 2024-04-27T04:20:03.311 INFO:teuthology.orchestra.run.smithi162.stdout:[mds.cephfs.smithi181.bkkcdm{-1:44323} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.181:6826/3589158285,v1:172.21.15.181:6827/3589158285] compat {c=[1],r=[1],i=[1fff]}] 2024-04-27T04:20:03.311 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:20:03.313+0000 7f1013957700 1 -- 172.21.15.162:0/1246200853 >> [v2:172.21.15.162:6800/1396866108,v1:172.21.15.162:6801/1396866108] conn(0x7f0ff806ec10 msgr2=0x7f0ff80710c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:20:03.311 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:20:03.313+0000 7f1013957700 1 --2- 172.21.15.162:0/1246200853 >> [v2:172.21.15.162:6800/1396866108,v1:172.21.15.162:6801/1396866108] conn(0x7f0ff806ec10 0x7f0ff80710c0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f100c10ec50 tx=0x7f0ffc00b040 comp rx=0 tx=0).stop 2024-04-27T04:20:03.311 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:20:03.313+0000 7f1013957700 1 -- 172.21.15.162:0/1246200853 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f100c101d00 msgr2=0x7f100c113b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:20:03.312 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:20:03.313+0000 7f1013957700 1 --2- 172.21.15.162:0/1246200853 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f100c101d00 0x7f100c113b10 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f1008009150 tx=0x7f1008009e30 comp rx=0 tx=0).stop 2024-04-27T04:20:03.312 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:20:03.313+0000 7f1013957700 1 -- 172.21.15.162:0/1246200853 shutdown_connections 2024-04-27T04:20:03.312 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:20:03.313+0000 7f1013957700 1 --2- 172.21.15.162:0/1246200853 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f100c101d00 0x7f100c113b10 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:20:03.312 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:20:03.313+0000 7f1013957700 1 --2- 172.21.15.162:0/1246200853 >> [v2:172.21.15.162:6800/1396866108,v1:172.21.15.162:6801/1396866108] conn(0x7f0ff806ec10 0x7f0ff80710c0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:20:03.312 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:20:03.313+0000 7f1013957700 1 --2- 172.21.15.162:0/1246200853 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f100c1026a0 0x7f100c114050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:20:03.312 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:20:03.313+0000 7f1013957700 1 -- 172.21.15.162:0/1246200853 >> 172.21.15.162:0/1246200853 conn(0x7f100c0f78d0 msgr2=0x7f100c065e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:20:03.312 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:20:03.313+0000 7f1013957700 1 -- 172.21.15.162:0/1246200853 shutdown_connections 2024-04-27T04:20:03.312 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:20:03.313+0000 7f1013957700 1 -- 172.21.15.162:0/1246200853 wait complete. 2024-04-27T04:20:03.312 INFO:teuthology.orchestra.run.smithi162.stderr:dumped fsmap epoch 32 2024-04-27T04:20:03.414 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:20:03.416+0000 7f34e913c700 1 -- 172.21.15.162:0/1920066218 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f34e40fbd40 msgr2=0x7f34e40fc1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:20:03.414 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:20:03.416+0000 7f34e913c700 1 --2- 172.21.15.162:0/1920066218 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f34e40fbd40 0x7f34e40fc1b0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f34d8004940 tx=0x7f34d8009e30 comp rx=0 tx=0).stop 2024-04-27T04:20:03.414 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:20:03.416+0000 7f34e913c700 1 -- 172.21.15.162:0/1920066218 shutdown_connections 2024-04-27T04:20:03.414 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:20:03.416+0000 7f34e913c700 1 --2- 172.21.15.162:0/1920066218 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f34e40fbd40 0x7f34e40fc1b0 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:20:03.414 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:20:03.416+0000 7f34e913c700 1 --2- 172.21.15.162:0/1920066218 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f34e4101d90 0x7f34e4102160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:20:03.414 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:20:03.416+0000 7f34e913c700 1 -- 172.21.15.162:0/1920066218 >> 172.21.15.162:0/1920066218 conn(0x7f34e4071d00 msgr2=0x7f34e4072100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:20:03.414 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:20:03.416+0000 7f34e913c700 1 -- 172.21.15.162:0/1920066218 shutdown_connections 2024-04-27T04:20:03.414 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:20:03.416+0000 7f34e913c700 1 -- 172.21.15.162:0/1920066218 wait complete. 2024-04-27T04:20:03.416 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:20:03.417+0000 7f34e913c700 1 Processor -- start 2024-04-27T04:20:03.416 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:20:03.417+0000 7f34e913c700 1 -- start start 2024-04-27T04:20:03.416 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:20:03.417+0000 7f34e913c700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f34e40fbd40 0x7f34e4115cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:20:03.416 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:20:03.417+0000 7f34e913c700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f34e4101d90 0x7f34e4116200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:20:03.416 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:20:03.417+0000 7f34e913c700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34e4116850 con 0x7f34e40fbd40 2024-04-27T04:20:03.416 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:20:03.417+0000 7f34e913c700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34e410fc60 con 0x7f34e4101d90 2024-04-27T04:20:03.416 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:20:03.417+0000 7f34e2d9d700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f34e40fbd40 0x7f34e4115cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:20:03.416 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:20:03.417+0000 7f34e2d9d700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f34e40fbd40 0x7f34e4115cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:58154/0 (socket says 172.21.15.162:58154) 2024-04-27T04:20:03.416 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:20:03.417+0000 7f34e259c700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f34e4101d90 0x7f34e4116200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:20:03.416 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:20:03.417+0000 7f34e2d9d700 1 -- 172.21.15.162:0/4291642257 learned_addr learned my addr 172.21.15.162:0/4291642257 (peer_addr_for_me v2:172.21.15.162:0/0) 2024-04-27T04:20:03.417 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:20:03.417+0000 7f34e259c700 1 -- 172.21.15.162:0/4291642257 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f34e40fbd40 msgr2=0x7f34e4115cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:20:03.417 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:20:03.417+0000 7f34e259c700 1 --2- 172.21.15.162:0/4291642257 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f34e40fbd40 0x7f34e4115cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:20:03.417 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:20:03.417+0000 7f34e259c700 1 -- 172.21.15.162:0/4291642257 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f34d8005040 con 0x7f34e4101d90 2024-04-27T04:20:03.417 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:20:03.417+0000 7f34e2d9d700 1 --2- 172.21.15.162:0/4291642257 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f34e40fbd40 0x7f34e4115cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-27T04:20:03.417 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:20:03.418+0000 7f34e259c700 1 --2- 172.21.15.162:0/4291642257 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f34e4101d90 0x7f34e4116200 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f34d8000c00 tx=0x7f34d8009e30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:20:03.417 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:20:03.418+0000 7f34cb7fe700 1 -- 172.21.15.162:0/4291642257 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f34d8021070 con 0x7f34e4101d90 2024-04-27T04:20:03.417 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:20:03.418+0000 7f34cb7fe700 1 -- 172.21.15.162:0/4291642257 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f34d8013cd0 con 0x7f34e4101d90 2024-04-27T04:20:03.417 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:20:03.418+0000 7f34e913c700 1 -- 172.21.15.162:0/4291642257 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f34e410fee0 con 0x7f34e4101d90 2024-04-27T04:20:03.417 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:20:03.418+0000 7f34cb7fe700 1 -- 172.21.15.162:0/4291642257 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f34d800e070 con 0x7f34e4101d90 2024-04-27T04:20:03.417 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:20:03.418+0000 7f34e913c700 1 -- 172.21.15.162:0/4291642257 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f34e41103c0 con 0x7f34e4101d90 2024-04-27T04:20:03.418 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:20:03.419+0000 7f34e913c700 1 -- 172.21.15.162:0/4291642257 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f34e40fd430 con 0x7f34e4101d90 2024-04-27T04:20:03.418 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:20:03.419+0000 7f34cb7fe700 1 -- 172.21.15.162:0/4291642257 <== mon.1 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 39) v1 ==== 92747+0+0 (secure 0 0 0) 0x7f34d801ca60 con 0x7f34e4101d90 2024-04-27T04:20:03.418 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:20:03.420+0000 7f34cb7fe700 1 --2- 172.21.15.162:0/4291642257 >> [v2:172.21.15.162:6800/1396866108,v1:172.21.15.162:6801/1396866108] conn(0x7f34cc06ec00 0x7f34cc0710b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:20:03.418 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:20:03.420+0000 7f34cb7fe700 1 -- 172.21.15.162:0/4291642257 <== mon.1 v2:172.21.15.181:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6582+0+0 (secure 0 0 0) 0x7f34d808fdc0 con 0x7f34e4101d90 2024-04-27T04:20:03.418 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:20:03.420+0000 7f34e2d9d700 1 --2- 172.21.15.162:0/4291642257 >> [v2:172.21.15.162:6800/1396866108,v1:172.21.15.162:6801/1396866108] conn(0x7f34cc06ec00 0x7f34cc0710b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:20:03.418 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:20:03.420+0000 7f34e2d9d700 1 --2- 172.21.15.162:0/4291642257 >> [v2:172.21.15.162:6800/1396866108,v1:172.21.15.162:6801/1396866108] conn(0x7f34cc06ec00 0x7f34cc0710b0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f34d40041d0 tx=0x7f34d400b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:20:03.423 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:20:03.425+0000 7f34cb7fe700 1 -- 172.21.15.162:0/4291642257 <== mon.1 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7f34d8058b90 con 0x7f34e4101d90 2024-04-27T04:20:03.641 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:20:03.643+0000 7f34e913c700 1 -- 172.21.15.162:0/4291642257 --> [v2:172.21.15.162:6800/1396866108,v1:172.21.15.162:6801/1396866108] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f34e4108df0 con 0x7f34cc06ec00 2024-04-27T04:20:03.642 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:20:03.644+0000 7f34cb7fe700 1 -- 172.21.15.162:0/4291642257 <== mgr.34104 v2:172.21.15.162:6800/1396866108 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f34e4108df0 con 0x7f34cc06ec00 2024-04-27T04:20:03.642 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-04-27T04:20:03.642 INFO:teuthology.orchestra.run.smithi162.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:90a3a9fa9ceb8977f6dad9ccd36fe46018d07e2118199820523a6b35301d9c16", 2024-04-27T04:20:03.642 INFO:teuthology.orchestra.run.smithi162.stdout: "in_progress": true, 2024-04-27T04:20:03.643 INFO:teuthology.orchestra.run.smithi162.stdout: "which": "Upgrading all daemon types on all hosts", 2024-04-27T04:20:03.643 INFO:teuthology.orchestra.run.smithi162.stdout: "services_complete": [ 2024-04-27T04:20:03.643 INFO:teuthology.orchestra.run.smithi162.stdout: "osd", 2024-04-27T04:20:03.643 INFO:teuthology.orchestra.run.smithi162.stdout: "mgr", 2024-04-27T04:20:03.643 INFO:teuthology.orchestra.run.smithi162.stdout: "mds", 2024-04-27T04:20:03.643 INFO:teuthology.orchestra.run.smithi162.stdout: "mon", 2024-04-27T04:20:03.643 INFO:teuthology.orchestra.run.smithi162.stdout: "crash" 2024-04-27T04:20:03.643 INFO:teuthology.orchestra.run.smithi162.stdout: ], 2024-04-27T04:20:03.643 INFO:teuthology.orchestra.run.smithi162.stdout: "progress": "16/21 daemons upgraded", 2024-04-27T04:20:03.643 INFO:teuthology.orchestra.run.smithi162.stdout: "message": "Currently upgrading mds daemons", 2024-04-27T04:20:03.643 INFO:teuthology.orchestra.run.smithi162.stdout: "is_paused": false 2024-04-27T04:20:03.643 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-04-27T04:20:03.644 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:20:03.646+0000 7f34e913c700 1 -- 172.21.15.162:0/4291642257 >> [v2:172.21.15.162:6800/1396866108,v1:172.21.15.162:6801/1396866108] conn(0x7f34cc06ec00 msgr2=0x7f34cc0710b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:20:03.644 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:20:03.646+0000 7f34e913c700 1 --2- 172.21.15.162:0/4291642257 >> [v2:172.21.15.162:6800/1396866108,v1:172.21.15.162:6801/1396866108] conn(0x7f34cc06ec00 0x7f34cc0710b0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f34d40041d0 tx=0x7f34d400b040 comp rx=0 tx=0).stop 2024-04-27T04:20:03.644 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:20:03.646+0000 7f34e913c700 1 -- 172.21.15.162:0/4291642257 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f34e4101d90 msgr2=0x7f34e4116200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:20:03.644 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:20:03.646+0000 7f34e913c700 1 --2- 172.21.15.162:0/4291642257 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f34e4101d90 0x7f34e4116200 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f34d8000c00 tx=0x7f34d8009e30 comp rx=0 tx=0).stop 2024-04-27T04:20:03.645 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:20:03.646+0000 7f34e913c700 1 -- 172.21.15.162:0/4291642257 shutdown_connections 2024-04-27T04:20:03.645 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:20:03.646+0000 7f34e913c700 1 --2- 172.21.15.162:0/4291642257 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f34e40fbd40 0x7f34e4115cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:20:03.645 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:20:03.646+0000 7f34e913c700 1 --2- 172.21.15.162:0/4291642257 >> [v2:172.21.15.162:6800/1396866108,v1:172.21.15.162:6801/1396866108] conn(0x7f34cc06ec00 0x7f34cc0710b0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:20:03.645 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:20:03.646+0000 7f34e913c700 1 --2- 172.21.15.162:0/4291642257 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f34e4101d90 0x7f34e4116200 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:20:03.645 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:20:03.646+0000 7f34e913c700 1 -- 172.21.15.162:0/4291642257 >> 172.21.15.162:0/4291642257 conn(0x7f34e4071d00 msgr2=0x7f34e4065600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:20:03.645 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:20:03.647+0000 7f34e913c700 1 -- 172.21.15.162:0/4291642257 shutdown_connections 2024-04-27T04:20:03.645 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:20:03.647+0000 7f34e913c700 1 -- 172.21.15.162:0/4291642257 wait complete. 2024-04-27T04:20:03.747 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:20:03.749+0000 7fc22f1e1700 1 -- 172.21.15.162:0/2118817486 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc228103780 msgr2=0x7fc2280fcdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:20:03.747 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:20:03.749+0000 7fc22f1e1700 1 --2- 172.21.15.162:0/2118817486 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc228103780 0x7fc2280fcdf0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fc218004940 tx=0x7fc218009e30 comp rx=0 tx=0).stop 2024-04-27T04:20:03.747 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:20:03.749+0000 7fc22f1e1700 1 -- 172.21.15.162:0/2118817486 shutdown_connections 2024-04-27T04:20:03.747 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:20:03.749+0000 7fc22f1e1700 1 --2- 172.21.15.162:0/2118817486 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc228103780 0x7fc2280fcdf0 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:20:03.747 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:20:03.749+0000 7fc22f1e1700 1 --2- 172.21.15.162:0/2118817486 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc228102de0 0x7fc2281031b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:20:03.747 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:20:03.749+0000 7fc22f1e1700 1 -- 172.21.15.162:0/2118817486 >> 172.21.15.162:0/2118817486 conn(0x7fc2280f8990 msgr2=0x7fc2280fada0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:20:03.747 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:20:03.749+0000 7fc22f1e1700 1 -- 172.21.15.162:0/2118817486 shutdown_connections 2024-04-27T04:20:03.747 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:20:03.749+0000 7fc22f1e1700 1 -- 172.21.15.162:0/2118817486 wait complete. 2024-04-27T04:20:03.748 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:20:03.749+0000 7fc22f1e1700 1 Processor -- start 2024-04-27T04:20:03.748 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:20:03.749+0000 7fc22f1e1700 1 -- start start 2024-04-27T04:20:03.749 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:20:03.750+0000 7fc22f1e1700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc228102de0 0x7fc228113900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:20:03.749 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:20:03.750+0000 7fc22f1e1700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc228103780 0x7fc228113e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:20:03.749 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:20:03.750+0000 7fc22f1e1700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc228114490 con 0x7fc228103780 2024-04-27T04:20:03.749 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:20:03.750+0000 7fc22f1e1700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc22810da90 con 0x7fc228102de0 2024-04-27T04:20:03.749 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:20:03.750+0000 7fc227fff700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc228103780 0x7fc228113e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:20:03.749 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:20:03.750+0000 7fc22cf7d700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc228102de0 0x7fc228113900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:20:03.750 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:20:03.750+0000 7fc227fff700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc228103780 0x7fc228113e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:52184/0 (socket says 172.21.15.162:52184) 2024-04-27T04:20:03.750 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:20:03.750+0000 7fc227fff700 1 -- 172.21.15.162:0/4063603968 learned_addr learned my addr 172.21.15.162:0/4063603968 (peer_addr_for_me v2:172.21.15.162:0/0) 2024-04-27T04:20:03.750 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:20:03.750+0000 7fc227fff700 1 -- 172.21.15.162:0/4063603968 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc228102de0 msgr2=0x7fc228113900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:20:03.750 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:20:03.750+0000 7fc227fff700 1 --2- 172.21.15.162:0/4063603968 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc228102de0 0x7fc228113900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:20:03.750 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:20:03.750+0000 7fc227fff700 1 -- 172.21.15.162:0/4063603968 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc218005040 con 0x7fc228103780 2024-04-27T04:20:03.750 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:20:03.750+0000 7fc22cf7d700 1 --2- 172.21.15.162:0/4063603968 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc228102de0 0x7fc228113900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-27T04:20:03.750 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:20:03.750+0000 7fc227fff700 1 --2- 172.21.15.162:0/4063603968 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc228103780 0x7fc228113e40 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fc218000c00 tx=0x7fc218009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:20:03.750 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:20:03.750+0000 7fc2257fa700 1 -- 172.21.15.162:0/4063603968 <== mon.0 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc218021070 con 0x7fc228103780 2024-04-27T04:20:03.750 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:20:03.751+0000 7fc2257fa700 1 -- 172.21.15.162:0/4063603968 <== mon.0 v2:172.21.15.162:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fc218013c00 con 0x7fc228103780 2024-04-27T04:20:03.750 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:20:03.751+0000 7fc2257fa700 1 -- 172.21.15.162:0/4063603968 <== mon.0 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc21800e070 con 0x7fc228103780 2024-04-27T04:20:03.750 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:20:03.751+0000 7fc22f1e1700 1 -- 172.21.15.162:0/4063603968 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc22810dd10 con 0x7fc228103780 2024-04-27T04:20:03.750 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:20:03.751+0000 7fc22f1e1700 1 -- 172.21.15.162:0/4063603968 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc22810e1f0 con 0x7fc228103780 2024-04-27T04:20:03.750 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:20:03.752+0000 7fc22f1e1700 1 -- 172.21.15.162:0/4063603968 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc2280fe530 con 0x7fc228103780 2024-04-27T04:20:03.751 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:20:03.752+0000 7fc2257fa700 1 -- 172.21.15.162:0/4063603968 <== mon.0 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 39) v1 ==== 92747+0+0 (secure 0 0 0) 0x7fc21801ca60 con 0x7fc228103780 2024-04-27T04:20:03.751 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:20:03.752+0000 7fc2257fa700 1 --2- 172.21.15.162:0/4063603968 >> [v2:172.21.15.162:6800/1396866108,v1:172.21.15.162:6801/1396866108] conn(0x7fc21406ec00 0x7fc2140710b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:20:03.751 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:20:03.753+0000 7fc2257fa700 1 -- 172.21.15.162:0/4063603968 <== mon.0 v2:172.21.15.162:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6582+0+0 (secure 0 0 0) 0x7fc21808fd00 con 0x7fc228103780 2024-04-27T04:20:03.751 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:20:03.753+0000 7fc22cf7d700 1 --2- 172.21.15.162:0/4063603968 >> [v2:172.21.15.162:6800/1396866108,v1:172.21.15.162:6801/1396866108] conn(0x7fc21406ec00 0x7fc2140710b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:20:03.751 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:20:03.753+0000 7fc22cf7d700 1 --2- 172.21.15.162:0/4063603968 >> [v2:172.21.15.162:6800/1396866108,v1:172.21.15.162:6801/1396866108] conn(0x7fc21406ec00 0x7fc2140710b0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fc220001880 tx=0x7fc22000b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:20:03.755 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:20:03.757+0000 7fc2257fa700 1 -- 172.21.15.162:0/4063603968 <== mon.0 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7fc218093030 con 0x7fc228103780 2024-04-27T04:20:04.047 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:20:04.049+0000 7fc22f1e1700 1 -- 172.21.15.162:0/4063603968 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7fc228109ec0 con 0x7fc228103780 2024-04-27T04:20:04.047 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:20:03 smithi162 ceph-mon[110641]: from='client.34448 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-27T04:20:04.047 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:20:03 smithi162 ceph-mon[110641]: from='client.34452 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-27T04:20:04.047 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:20:03 smithi162 ceph-mon[110641]: from='client.34456 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-27T04:20:04.047 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:20:03 smithi162 ceph-mon[110641]: from='client.? 172.21.15.162:0/2415032628' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-27T04:20:04.047 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:20:03 smithi162 ceph-mon[110641]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' 2024-04-27T04:20:04.048 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:20:03 smithi162 ceph-mon[110641]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' 2024-04-27T04:20:04.048 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:20:03 smithi162 ceph-mon[110641]: from='client.? 172.21.15.162:0/1246200853' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-27T04:20:04.048 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:20:04.049+0000 7fc2257fa700 1 -- 172.21.15.162:0/4063603968 <== mon.0 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+10 (secure 0 0 0) 0x7fc21805c600 con 0x7fc228103780 2024-04-27T04:20:04.048 INFO:teuthology.orchestra.run.smithi162.stdout:HEALTH_OK 2024-04-27T04:20:04.049 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:20:04.051+0000 7fc22f1e1700 1 -- 172.21.15.162:0/4063603968 >> [v2:172.21.15.162:6800/1396866108,v1:172.21.15.162:6801/1396866108] conn(0x7fc21406ec00 msgr2=0x7fc2140710b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:20:04.049 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:20:04.051+0000 7fc22f1e1700 1 --2- 172.21.15.162:0/4063603968 >> [v2:172.21.15.162:6800/1396866108,v1:172.21.15.162:6801/1396866108] conn(0x7fc21406ec00 0x7fc2140710b0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fc220001880 tx=0x7fc22000b040 comp rx=0 tx=0).stop 2024-04-27T04:20:04.050 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:20:04.051+0000 7fc22f1e1700 1 -- 172.21.15.162:0/4063603968 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc228103780 msgr2=0x7fc228113e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:20:04.050 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:20:04.051+0000 7fc22f1e1700 1 --2- 172.21.15.162:0/4063603968 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc228103780 0x7fc228113e40 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fc218000c00 tx=0x7fc218009e30 comp rx=0 tx=0).stop 2024-04-27T04:20:04.050 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:20:04.052+0000 7fc22f1e1700 1 -- 172.21.15.162:0/4063603968 shutdown_connections 2024-04-27T04:20:04.050 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:20:04.052+0000 7fc22f1e1700 1 --2- 172.21.15.162:0/4063603968 >> [v2:172.21.15.162:6800/1396866108,v1:172.21.15.162:6801/1396866108] conn(0x7fc21406ec00 0x7fc2140710b0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:20:04.050 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:20:04.052+0000 7fc22f1e1700 1 --2- 172.21.15.162:0/4063603968 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc228102de0 0x7fc228113900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:20:04.050 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:20:04.052+0000 7fc22f1e1700 1 --2- 172.21.15.162:0/4063603968 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc228103780 0x7fc228113e40 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:20:04.050 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:20:04.052+0000 7fc22f1e1700 1 -- 172.21.15.162:0/4063603968 >> 172.21.15.162:0/4063603968 conn(0x7fc2280f8990 msgr2=0x7fc2280f9880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:20:04.050 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:20:04.052+0000 7fc22f1e1700 1 -- 172.21.15.162:0/4063603968 shutdown_connections 2024-04-27T04:20:04.050 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:20:04.052+0000 7fc22f1e1700 1 -- 172.21.15.162:0/4063603968 wait complete. 2024-04-27T04:20:04.171 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:20:03 smithi181 ceph-mon[81134]: from='client.34448 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-27T04:20:04.172 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:20:03 smithi181 ceph-mon[81134]: from='client.34452 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-27T04:20:04.172 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:20:03 smithi181 ceph-mon[81134]: from='client.34456 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-27T04:20:04.172 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:20:03 smithi181 ceph-mon[81134]: from='client.? 172.21.15.162:0/2415032628' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-27T04:20:04.172 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:20:03 smithi181 ceph-mon[81134]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' 2024-04-27T04:20:04.172 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:20:03 smithi181 ceph-mon[81134]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' 2024-04-27T04:20:04.173 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:20:03 smithi181 ceph-mon[81134]: from='client.? 172.21.15.162:0/1246200853' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-27T04:20:04.818 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:20:04 smithi181 ceph-mon[81134]: from='client.44347 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-27T04:20:04.818 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:20:04 smithi181 ceph-mon[81134]: pgmap v211: 65 pgs: 65 active+clean; 208 MiB data, 851 MiB used, 536 GiB / 536 GiB avail; 16 MiB/s rd, 85 B/s wr, 5 op/s 2024-04-27T04:20:04.818 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:20:04 smithi181 ceph-mon[81134]: from='client.? 172.21.15.162:0/4063603968' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-27T04:20:05.136 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:20:04 smithi162 ceph-mon[110641]: from='client.44347 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-27T04:20:05.136 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:20:04 smithi162 ceph-mon[110641]: pgmap v211: 65 pgs: 65 active+clean; 208 MiB data, 851 MiB used, 536 GiB / 536 GiB avail; 16 MiB/s rd, 85 B/s wr, 5 op/s 2024-04-27T04:20:05.136 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:20:04 smithi162 ceph-mon[110641]: from='client.? 172.21.15.162:0/4063603968' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-27T04:20:07.137 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:20:06 smithi162 ceph-mon[110641]: pgmap v212: 65 pgs: 65 active+clean; 208 MiB data, 851 MiB used, 536 GiB / 536 GiB avail; 13 MiB/s rd, 170 B/s wr, 6 op/s 2024-04-27T04:20:07.137 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:20:06 smithi162 ceph-mon[110641]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' 2024-04-27T04:20:07.137 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:20:06 smithi162 ceph-mon[110641]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' 2024-04-27T04:20:07.137 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:20:06 smithi162 ceph-mon[110641]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T04:20:07.137 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:20:06 smithi162 ceph-mon[110641]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T04:20:07.137 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:20:06 smithi162 ceph-mon[110641]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' 2024-04-27T04:20:07.137 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:20:06 smithi162 ceph-mon[110641]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T04:20:07.137 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:20:06 smithi162 ceph-mon[110641]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "versions"}]: dispatch 2024-04-27T04:20:07.137 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:20:06 smithi162 ceph-mon[110641]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "versions"}]: dispatch 2024-04-27T04:20:07.137 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:20:06 smithi162 ceph-mon[110641]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "versions"}]: dispatch 2024-04-27T04:20:07.137 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:20:06 smithi162 ceph-mon[110641]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "versions"}]: dispatch 2024-04-27T04:20:07.137 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:20:06 smithi162 ceph-mon[110641]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "versions"}]: dispatch 2024-04-27T04:20:07.137 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:20:06 smithi162 ceph-mon[110641]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' 2024-04-27T04:20:07.137 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:20:06 smithi162 ceph-mon[110641]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi162.gzlyjj"}]: dispatch 2024-04-27T04:20:07.138 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:20:06 smithi162 ceph-mon[110641]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi162.gzlyjj"}]': finished 2024-04-27T04:20:07.138 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:20:06 smithi162 ceph-mon[110641]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi162.xcnapk"}]: dispatch 2024-04-27T04:20:07.138 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:20:06 smithi162 ceph-mon[110641]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi162.xcnapk"}]': finished 2024-04-27T04:20:07.138 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:20:06 smithi162 ceph-mon[110641]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi181.bkkcdm"}]: dispatch 2024-04-27T04:20:07.138 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:20:06 smithi162 ceph-mon[110641]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi181.bkkcdm"}]': finished 2024-04-27T04:20:07.138 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:20:06 smithi162 ceph-mon[110641]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi181.dzomgh"}]: dispatch 2024-04-27T04:20:07.138 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:20:06 smithi162 ceph-mon[110641]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi181.dzomgh"}]': finished 2024-04-27T04:20:07.138 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:20:06 smithi162 ceph-mon[110641]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]: dispatch 2024-04-27T04:20:07.172 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:20:06 smithi181 ceph-mon[81134]: pgmap v212: 65 pgs: 65 active+clean; 208 MiB data, 851 MiB used, 536 GiB / 536 GiB avail; 13 MiB/s rd, 170 B/s wr, 6 op/s 2024-04-27T04:20:07.172 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:20:06 smithi181 ceph-mon[81134]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' 2024-04-27T04:20:07.172 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:20:06 smithi181 ceph-mon[81134]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' 2024-04-27T04:20:07.172 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:20:06 smithi181 ceph-mon[81134]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T04:20:07.172 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:20:06 smithi181 ceph-mon[81134]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T04:20:07.172 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:20:06 smithi181 ceph-mon[81134]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' 2024-04-27T04:20:07.172 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:20:06 smithi181 ceph-mon[81134]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T04:20:07.172 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:20:06 smithi181 ceph-mon[81134]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "versions"}]: dispatch 2024-04-27T04:20:07.172 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:20:06 smithi181 ceph-mon[81134]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "versions"}]: dispatch 2024-04-27T04:20:07.173 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:20:06 smithi181 ceph-mon[81134]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "versions"}]: dispatch 2024-04-27T04:20:07.173 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:20:06 smithi181 ceph-mon[81134]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "versions"}]: dispatch 2024-04-27T04:20:07.173 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:20:06 smithi181 ceph-mon[81134]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "versions"}]: dispatch 2024-04-27T04:20:07.173 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:20:06 smithi181 ceph-mon[81134]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' 2024-04-27T04:20:07.173 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:20:06 smithi181 ceph-mon[81134]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi162.gzlyjj"}]: dispatch 2024-04-27T04:20:07.173 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:20:06 smithi181 ceph-mon[81134]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi162.gzlyjj"}]': finished 2024-04-27T04:20:07.173 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:20:06 smithi181 ceph-mon[81134]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi162.xcnapk"}]: dispatch 2024-04-27T04:20:07.173 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:20:06 smithi181 ceph-mon[81134]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi162.xcnapk"}]': finished 2024-04-27T04:20:07.173 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:20:06 smithi181 ceph-mon[81134]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi181.bkkcdm"}]: dispatch 2024-04-27T04:20:07.173 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:20:06 smithi181 ceph-mon[81134]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi181.bkkcdm"}]': finished 2024-04-27T04:20:07.173 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:20:06 smithi181 ceph-mon[81134]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi181.dzomgh"}]: dispatch 2024-04-27T04:20:07.173 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:20:06 smithi181 ceph-mon[81134]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi181.dzomgh"}]': finished 2024-04-27T04:20:07.173 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:20:06 smithi181 ceph-mon[81134]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]: dispatch 2024-04-27T04:20:08.136 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:20:07 smithi162 ceph-mon[110641]: Upgrade: Setting container_image for all mds 2024-04-27T04:20:08.136 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:20:07 smithi162 ceph-mon[110641]: Upgrade: Setting filesystem cephfs Joinable 2024-04-27T04:20:08.137 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:20:07 smithi162 ceph-mon[110641]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]': finished 2024-04-27T04:20:08.137 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:20:07 smithi162 ceph-mon[110641]: fsmap cephfs:1 {0=cephfs.smithi162.gzlyjj=up:active} 3 up:standby 2024-04-27T04:20:08.137 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:20:07 smithi162 ceph-mon[110641]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "1"}]: dispatch 2024-04-27T04:20:08.171 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:20:07 smithi181 ceph-mon[81134]: Upgrade: Setting container_image for all mds 2024-04-27T04:20:08.171 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:20:07 smithi181 ceph-mon[81134]: Upgrade: Setting filesystem cephfs Joinable 2024-04-27T04:20:08.171 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:20:07 smithi181 ceph-mon[81134]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]': finished 2024-04-27T04:20:08.171 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:20:07 smithi181 ceph-mon[81134]: fsmap cephfs:1 {0=cephfs.smithi162.gzlyjj=up:active} 3 up:standby 2024-04-27T04:20:08.172 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:20:07 smithi181 ceph-mon[81134]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "1"}]: dispatch 2024-04-27T04:20:09.137 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:20:08 smithi162 ceph-mon[110641]: Upgrade: Enabling allow_standby_replay on filesystem cephfs 2024-04-27T04:20:09.137 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:20:08 smithi162 ceph-mon[110641]: pgmap v213: 65 pgs: 65 active+clean; 208 MiB data, 851 MiB used, 536 GiB / 536 GiB avail; 6.7 MiB/s rd, 170 B/s wr, 4 op/s 2024-04-27T04:20:09.137 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:20:08 smithi162 ceph-mon[110641]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "1"}]': finished 2024-04-27T04:20:09.137 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:20:08 smithi162 ceph-mon[110641]: fsmap cephfs:1 {0=cephfs.smithi162.gzlyjj=up:active} 3 up:standby 2024-04-27T04:20:09.137 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:20:08 smithi162 ceph-mon[110641]: fsmap cephfs:1 {0=cephfs.smithi162.gzlyjj=up:active} 1 up:standby-replay 2 up:standby 2024-04-27T04:20:09.137 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:20:08 smithi162 ceph-mon[110641]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' 2024-04-27T04:20:09.137 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:20:08 smithi162 ceph-mon[110641]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "versions"}]: dispatch 2024-04-27T04:20:09.137 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:20:08 smithi162 ceph-mon[110641]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' 2024-04-27T04:20:09.137 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:20:08 smithi162 ceph-mon[110641]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "versions"}]: dispatch 2024-04-27T04:20:09.137 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:20:08 smithi162 ceph-mon[110641]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' 2024-04-27T04:20:09.138 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:20:08 smithi162 ceph-mon[110641]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "versions"}]: dispatch 2024-04-27T04:20:09.138 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:20:08 smithi162 ceph-mon[110641]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "versions"}]: dispatch 2024-04-27T04:20:09.138 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:20:08 smithi162 ceph-mon[110641]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' 2024-04-27T04:20:09.138 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:20:08 smithi162 ceph-mon[110641]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "versions"}]: dispatch 2024-04-27T04:20:09.138 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:20:08 smithi162 ceph-mon[110641]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' 2024-04-27T04:20:09.138 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:20:08 smithi162 ceph-mon[110641]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "versions"}]: dispatch 2024-04-27T04:20:09.138 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:20:08 smithi162 ceph-mon[110641]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' 2024-04-27T04:20:09.138 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:20:08 smithi162 ceph-mon[110641]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "versions"}]: dispatch 2024-04-27T04:20:09.138 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:20:08 smithi162 ceph-mon[110641]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' 2024-04-27T04:20:09.138 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:20:08 smithi162 ceph-mon[110641]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "versions"}]: dispatch 2024-04-27T04:20:09.138 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:20:08 smithi162 ceph-mon[110641]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "versions"}]: dispatch 2024-04-27T04:20:09.138 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:20:08 smithi162 ceph-mon[110641]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "versions"}]: dispatch 2024-04-27T04:20:09.138 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:20:08 smithi162 ceph-mon[110641]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "versions"}]: dispatch 2024-04-27T04:20:09.138 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:20:08 smithi162 ceph-mon[110641]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "versions"}]: dispatch 2024-04-27T04:20:09.138 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:20:08 smithi162 ceph-mon[110641]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "versions"}]: dispatch 2024-04-27T04:20:09.139 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:20:08 smithi162 ceph-mon[110641]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mgr"}]: dispatch 2024-04-27T04:20:09.139 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:20:08 smithi162 ceph-mon[110641]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mgr"}]': finished 2024-04-27T04:20:09.139 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:20:08 smithi162 ceph-mon[110641]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-04-27T04:20:09.139 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:20:08 smithi162 ceph-mon[110641]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mon"}]': finished 2024-04-27T04:20:09.139 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:20:08 smithi162 ceph-mon[110641]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.crash"}]: dispatch 2024-04-27T04:20:09.139 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:20:08 smithi162 ceph-mon[110641]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.crash"}]': finished 2024-04-27T04:20:09.139 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:20:08 smithi162 ceph-mon[110641]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd"}]: dispatch 2024-04-27T04:20:09.139 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:20:08 smithi162 ceph-mon[110641]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd"}]': finished 2024-04-27T04:20:09.139 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:20:08 smithi162 ceph-mon[110641]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mds"}]: dispatch 2024-04-27T04:20:09.139 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:20:08 smithi162 ceph-mon[110641]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mds"}]': finished 2024-04-27T04:20:09.139 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:20:08 smithi162 ceph-mon[110641]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.rgw"}]: dispatch 2024-04-27T04:20:09.139 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:20:08 smithi162 ceph-mon[110641]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.rgw"}]': finished 2024-04-27T04:20:09.139 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:20:08 smithi162 ceph-mon[110641]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.rbd-mirror"}]: dispatch 2024-04-27T04:20:09.172 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:20:08 smithi181 ceph-mon[81134]: Upgrade: Enabling allow_standby_replay on filesystem cephfs 2024-04-27T04:20:09.172 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:20:08 smithi181 ceph-mon[81134]: pgmap v213: 65 pgs: 65 active+clean; 208 MiB data, 851 MiB used, 536 GiB / 536 GiB avail; 6.7 MiB/s rd, 170 B/s wr, 4 op/s 2024-04-27T04:20:09.172 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:20:08 smithi181 ceph-mon[81134]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "1"}]': finished 2024-04-27T04:20:09.172 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:20:08 smithi181 ceph-mon[81134]: fsmap cephfs:1 {0=cephfs.smithi162.gzlyjj=up:active} 3 up:standby 2024-04-27T04:20:09.173 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:20:08 smithi181 ceph-mon[81134]: fsmap cephfs:1 {0=cephfs.smithi162.gzlyjj=up:active} 1 up:standby-replay 2 up:standby 2024-04-27T04:20:09.173 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:20:08 smithi181 ceph-mon[81134]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' 2024-04-27T04:20:09.173 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:20:08 smithi181 ceph-mon[81134]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "versions"}]: dispatch 2024-04-27T04:20:09.173 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:20:08 smithi181 ceph-mon[81134]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' 2024-04-27T04:20:09.173 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:20:08 smithi181 ceph-mon[81134]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "versions"}]: dispatch 2024-04-27T04:20:09.173 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:20:08 smithi181 ceph-mon[81134]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' 2024-04-27T04:20:09.173 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:20:08 smithi181 ceph-mon[81134]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "versions"}]: dispatch 2024-04-27T04:20:09.173 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:20:08 smithi181 ceph-mon[81134]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "versions"}]: dispatch 2024-04-27T04:20:09.173 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:20:08 smithi181 ceph-mon[81134]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' 2024-04-27T04:20:09.173 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:20:08 smithi181 ceph-mon[81134]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "versions"}]: dispatch 2024-04-27T04:20:09.173 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:20:08 smithi181 ceph-mon[81134]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' 2024-04-27T04:20:09.173 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:20:08 smithi181 ceph-mon[81134]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "versions"}]: dispatch 2024-04-27T04:20:09.173 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:20:08 smithi181 ceph-mon[81134]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' 2024-04-27T04:20:09.173 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:20:08 smithi181 ceph-mon[81134]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "versions"}]: dispatch 2024-04-27T04:20:09.174 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:20:08 smithi181 ceph-mon[81134]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' 2024-04-27T04:20:09.174 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:20:08 smithi181 ceph-mon[81134]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "versions"}]: dispatch 2024-04-27T04:20:09.174 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:20:08 smithi181 ceph-mon[81134]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "versions"}]: dispatch 2024-04-27T04:20:09.174 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:20:08 smithi181 ceph-mon[81134]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "versions"}]: dispatch 2024-04-27T04:20:09.174 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:20:08 smithi181 ceph-mon[81134]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "versions"}]: dispatch 2024-04-27T04:20:09.174 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:20:08 smithi181 ceph-mon[81134]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "versions"}]: dispatch 2024-04-27T04:20:09.174 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:20:08 smithi181 ceph-mon[81134]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "versions"}]: dispatch 2024-04-27T04:20:09.174 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:20:08 smithi181 ceph-mon[81134]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mgr"}]: dispatch 2024-04-27T04:20:09.174 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:20:08 smithi181 ceph-mon[81134]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mgr"}]': finished 2024-04-27T04:20:09.174 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:20:08 smithi181 ceph-mon[81134]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-04-27T04:20:09.174 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:20:08 smithi181 ceph-mon[81134]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mon"}]': finished 2024-04-27T04:20:09.174 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:20:08 smithi181 ceph-mon[81134]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.crash"}]: dispatch 2024-04-27T04:20:09.174 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:20:08 smithi181 ceph-mon[81134]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.crash"}]': finished 2024-04-27T04:20:09.175 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:20:08 smithi181 ceph-mon[81134]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd"}]: dispatch 2024-04-27T04:20:09.175 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:20:08 smithi181 ceph-mon[81134]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd"}]': finished 2024-04-27T04:20:09.175 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:20:08 smithi181 ceph-mon[81134]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mds"}]: dispatch 2024-04-27T04:20:09.175 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:20:08 smithi181 ceph-mon[81134]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mds"}]': finished 2024-04-27T04:20:09.175 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:20:08 smithi181 ceph-mon[81134]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.rgw"}]: dispatch 2024-04-27T04:20:09.175 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:20:08 smithi181 ceph-mon[81134]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.rgw"}]': finished 2024-04-27T04:20:09.175 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:20:08 smithi181 ceph-mon[81134]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.rbd-mirror"}]: dispatch 2024-04-27T04:20:10.136 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:20:09 smithi162 ceph-mon[110641]: Upgrade: Setting container_image for all rgw 2024-04-27T04:20:10.137 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:20:09 smithi162 ceph-mon[110641]: Upgrade: Setting container_image for all rbd-mirror 2024-04-27T04:20:10.137 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:20:09 smithi162 ceph-mon[110641]: Upgrade: Setting container_image for all ceph-exporter 2024-04-27T04:20:10.137 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:20:09 smithi162 ceph-mon[110641]: Upgrade: Setting container_image for all iscsi 2024-04-27T04:20:10.137 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:20:09 smithi162 ceph-mon[110641]: Upgrade: Setting container_image for all nfs 2024-04-27T04:20:10.137 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:20:09 smithi162 ceph-mon[110641]: Upgrade: Setting container_image for all nvmeof 2024-04-27T04:20:10.137 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:20:09 smithi162 ceph-mon[110641]: Upgrade: Finalizing container_image settings 2024-04-27T04:20:10.137 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:20:09 smithi162 ceph-mon[110641]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.rbd-mirror"}]': finished 2024-04-27T04:20:10.137 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:20:09 smithi162 ceph-mon[110641]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-04-27T04:20:10.137 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:20:09 smithi162 ceph-mon[110641]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.ceph-exporter"}]: dispatch 2024-04-27T04:20:10.137 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:20:09 smithi162 ceph-mon[110641]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.ceph-exporter"}]': finished 2024-04-27T04:20:10.137 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:20:09 smithi162 ceph-mon[110641]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.iscsi"}]: dispatch 2024-04-27T04:20:10.137 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:20:09 smithi162 ceph-mon[110641]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.iscsi"}]': finished 2024-04-27T04:20:10.138 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:20:09 smithi162 ceph-mon[110641]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.nfs"}]: dispatch 2024-04-27T04:20:10.138 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:20:09 smithi162 ceph-mon[110641]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.nfs"}]': finished 2024-04-27T04:20:10.138 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:20:09 smithi162 ceph-mon[110641]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.nvmeof"}]: dispatch 2024-04-27T04:20:10.138 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:20:09 smithi162 ceph-mon[110641]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.nvmeof"}]': finished 2024-04-27T04:20:10.138 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:20:09 smithi162 ceph-mon[110641]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-04-27T04:20:10.138 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:20:09 smithi162 ceph-mon[110641]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-04-27T04:20:10.138 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:20:09 smithi162 ceph-mon[110641]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-04-27T04:20:10.138 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:20:09 smithi162 ceph-mon[110641]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-04-27T04:20:10.138 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:20:09 smithi162 ceph-mon[110641]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-04-27T04:20:10.138 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:20:09 smithi162 ceph-mon[110641]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-04-27T04:20:10.138 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:20:09 smithi162 ceph-mon[110641]: Upgrade: Complete! 2024-04-27T04:20:10.138 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:20:09 smithi162 ceph-mon[110641]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' cmd=[{"prefix":"config-key del","key":"mgr/cephadm/upgrade_state"}]: dispatch 2024-04-27T04:20:10.138 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:20:09 smithi162 ceph-mon[110641]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' cmd='[{"prefix":"config-key del","key":"mgr/cephadm/upgrade_state"}]': finished 2024-04-27T04:20:10.139 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:20:09 smithi162 ceph-mon[110641]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T04:20:10.139 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:20:09 smithi162 ceph-mon[110641]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T04:20:10.139 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:20:09 smithi162 ceph-mon[110641]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T04:20:10.139 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:20:09 smithi162 ceph-mon[110641]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' 2024-04-27T04:20:10.139 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:20:09 smithi162 ceph-mon[110641]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T04:20:10.139 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:20:09 smithi162 ceph-mon[110641]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T04:20:10.139 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:20:09 smithi162 ceph-mon[110641]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T04:20:10.139 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:20:09 smithi162 ceph-mon[110641]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' 2024-04-27T04:20:10.172 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:20:09 smithi181 ceph-mon[81134]: Upgrade: Setting container_image for all rgw 2024-04-27T04:20:10.172 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:20:09 smithi181 ceph-mon[81134]: Upgrade: Setting container_image for all rbd-mirror 2024-04-27T04:20:10.172 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:20:09 smithi181 ceph-mon[81134]: Upgrade: Setting container_image for all ceph-exporter 2024-04-27T04:20:10.172 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:20:09 smithi181 ceph-mon[81134]: Upgrade: Setting container_image for all iscsi 2024-04-27T04:20:10.172 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:20:09 smithi181 ceph-mon[81134]: Upgrade: Setting container_image for all nfs 2024-04-27T04:20:10.172 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:20:09 smithi181 ceph-mon[81134]: Upgrade: Setting container_image for all nvmeof 2024-04-27T04:20:10.172 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:20:09 smithi181 ceph-mon[81134]: Upgrade: Finalizing container_image settings 2024-04-27T04:20:10.172 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:20:09 smithi181 ceph-mon[81134]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.rbd-mirror"}]': finished 2024-04-27T04:20:10.172 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:20:09 smithi181 ceph-mon[81134]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-04-27T04:20:10.173 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:20:09 smithi181 ceph-mon[81134]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.ceph-exporter"}]: dispatch 2024-04-27T04:20:10.173 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:20:09 smithi181 ceph-mon[81134]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.ceph-exporter"}]': finished 2024-04-27T04:20:10.173 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:20:09 smithi181 ceph-mon[81134]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.iscsi"}]: dispatch 2024-04-27T04:20:10.173 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:20:09 smithi181 ceph-mon[81134]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.iscsi"}]': finished 2024-04-27T04:20:10.173 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:20:09 smithi181 ceph-mon[81134]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.nfs"}]: dispatch 2024-04-27T04:20:10.173 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:20:09 smithi181 ceph-mon[81134]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.nfs"}]': finished 2024-04-27T04:20:10.173 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:20:09 smithi181 ceph-mon[81134]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.nvmeof"}]: dispatch 2024-04-27T04:20:10.173 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:20:09 smithi181 ceph-mon[81134]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.nvmeof"}]': finished 2024-04-27T04:20:10.173 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:20:09 smithi181 ceph-mon[81134]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-04-27T04:20:10.173 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:20:09 smithi181 ceph-mon[81134]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-04-27T04:20:10.173 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:20:09 smithi181 ceph-mon[81134]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-04-27T04:20:10.173 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:20:09 smithi181 ceph-mon[81134]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-04-27T04:20:10.173 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:20:09 smithi181 ceph-mon[81134]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-04-27T04:20:10.173 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:20:09 smithi181 ceph-mon[81134]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-04-27T04:20:10.174 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:20:09 smithi181 ceph-mon[81134]: Upgrade: Complete! 2024-04-27T04:20:10.174 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:20:09 smithi181 ceph-mon[81134]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' cmd=[{"prefix":"config-key del","key":"mgr/cephadm/upgrade_state"}]: dispatch 2024-04-27T04:20:10.174 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:20:09 smithi181 ceph-mon[81134]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' cmd='[{"prefix":"config-key del","key":"mgr/cephadm/upgrade_state"}]': finished 2024-04-27T04:20:10.174 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:20:09 smithi181 ceph-mon[81134]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T04:20:10.174 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:20:09 smithi181 ceph-mon[81134]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T04:20:10.174 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:20:09 smithi181 ceph-mon[81134]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T04:20:10.174 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:20:09 smithi181 ceph-mon[81134]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' 2024-04-27T04:20:10.174 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:20:09 smithi181 ceph-mon[81134]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T04:20:10.174 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:20:09 smithi181 ceph-mon[81134]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T04:20:10.174 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:20:09 smithi181 ceph-mon[81134]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T04:20:10.174 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:20:09 smithi181 ceph-mon[81134]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' 2024-04-27T04:20:11.139 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:20:10 smithi162 ceph-mon[110641]: pgmap v214: 65 pgs: 65 active+clean; 208 MiB data, 851 MiB used, 536 GiB / 536 GiB avail; 4.7 MiB/s rd, 4.2 KiB/s wr, 6 op/s 2024-04-27T04:20:11.172 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:20:10 smithi181 ceph-mon[81134]: pgmap v214: 65 pgs: 65 active+clean; 208 MiB data, 851 MiB used, 536 GiB / 536 GiB avail; 4.7 MiB/s rd, 4.2 KiB/s wr, 6 op/s 2024-04-27T04:20:12.886 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:20:12 smithi162 ceph-mon[110641]: pgmap v215: 65 pgs: 65 active+clean; 208 MiB data, 851 MiB used, 536 GiB / 536 GiB avail; 6.0 MiB/s rd, 4.2 KiB/s wr, 6 op/s 2024-04-27T04:20:13.171 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:20:12 smithi181 ceph-mon[81134]: pgmap v215: 65 pgs: 65 active+clean; 208 MiB data, 851 MiB used, 536 GiB / 536 GiB avail; 6.0 MiB/s rd, 4.2 KiB/s wr, 6 op/s 2024-04-27T04:20:14.386 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:20:13 smithi162 ceph-mon[110641]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' 2024-04-27T04:20:14.386 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:20:13 smithi162 ceph-mon[110641]: pgmap v216: 65 pgs: 65 active+clean; 208 MiB data, 851 MiB used, 536 GiB / 536 GiB avail; 8.5 MiB/s rd, 4.2 KiB/s wr, 8 op/s 2024-04-27T04:20:14.421 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:20:13 smithi181 ceph-mon[81134]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' 2024-04-27T04:20:14.421 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:20:13 smithi181 ceph-mon[81134]: pgmap v216: 65 pgs: 65 active+clean; 208 MiB data, 851 MiB used, 536 GiB / 536 GiB avail; 8.5 MiB/s rd, 4.2 KiB/s wr, 8 op/s 2024-04-27T04:20:17.136 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:20:16 smithi162 ceph-mon[110641]: pgmap v217: 65 pgs: 65 active+clean; 208 MiB data, 851 MiB used, 536 GiB / 536 GiB avail; 13 MiB/s rd, 4.2 KiB/s wr, 8 op/s 2024-04-27T04:20:17.171 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:20:16 smithi181 ceph-mon[81134]: pgmap v217: 65 pgs: 65 active+clean; 208 MiB data, 851 MiB used, 536 GiB / 536 GiB avail; 13 MiB/s rd, 4.2 KiB/s wr, 8 op/s 2024-04-27T04:20:18.136 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:20:17 smithi162 ceph-mon[110641]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-27T04:20:18.171 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:20:17 smithi181 ceph-mon[81134]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-27T04:20:19.136 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:20:18 smithi162 ceph-mon[110641]: pgmap v218: 65 pgs: 65 active+clean; 208 MiB data, 851 MiB used, 536 GiB / 536 GiB avail; 15 MiB/s rd, 4.1 KiB/s wr, 6 op/s 2024-04-27T04:20:19.171 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:20:18 smithi181 ceph-mon[81134]: pgmap v218: 65 pgs: 65 active+clean; 208 MiB data, 851 MiB used, 536 GiB / 536 GiB avail; 15 MiB/s rd, 4.1 KiB/s wr, 6 op/s 2024-04-27T04:20:21.136 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:20:20 smithi162 ceph-mon[110641]: pgmap v219: 65 pgs: 65 active+clean; 208 MiB data, 851 MiB used, 536 GiB / 536 GiB avail; 17 MiB/s rd, 4.1 KiB/s wr, 6 op/s 2024-04-27T04:20:21.171 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:20:20 smithi181 ceph-mon[81134]: pgmap v219: 65 pgs: 65 active+clean; 208 MiB data, 851 MiB used, 536 GiB / 536 GiB avail; 17 MiB/s rd, 4.1 KiB/s wr, 6 op/s 2024-04-27T04:20:23.136 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:20:22 smithi162 ceph-mon[110641]: pgmap v220: 65 pgs: 65 active+clean; 208 MiB data, 851 MiB used, 536 GiB / 536 GiB avail; 17 MiB/s rd, 5 op/s 2024-04-27T04:20:23.171 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:20:22 smithi181 ceph-mon[81134]: pgmap v220: 65 pgs: 65 active+clean; 208 MiB data, 851 MiB used, 536 GiB / 536 GiB avail; 17 MiB/s rd, 5 op/s 2024-04-27T04:20:25.136 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:20:24 smithi162 ceph-mon[110641]: pgmap v221: 65 pgs: 65 active+clean; 208 MiB data, 851 MiB used, 536 GiB / 536 GiB avail; 12 MiB/s rd, 4 op/s 2024-04-27T04:20:25.171 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:20:24 smithi181 ceph-mon[81134]: pgmap v221: 65 pgs: 65 active+clean; 208 MiB data, 851 MiB used, 536 GiB / 536 GiB avail; 12 MiB/s rd, 4 op/s 2024-04-27T04:20:27.136 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:20:26 smithi162 ceph-mon[110641]: pgmap v222: 65 pgs: 65 active+clean; 208 MiB data, 851 MiB used, 536 GiB / 536 GiB avail; 8.4 MiB/s rd, 3 op/s 2024-04-27T04:20:27.171 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:20:26 smithi181 ceph-mon[81134]: pgmap v222: 65 pgs: 65 active+clean; 208 MiB data, 851 MiB used, 536 GiB / 536 GiB avail; 8.4 MiB/s rd, 3 op/s 2024-04-27T04:20:29.136 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:20:28 smithi162 ceph-mon[110641]: pgmap v223: 65 pgs: 65 active+clean; 208 MiB data, 851 MiB used, 536 GiB / 536 GiB avail; 3.7 MiB/s rd, 1 op/s 2024-04-27T04:20:29.171 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:20:28 smithi181 ceph-mon[81134]: pgmap v223: 65 pgs: 65 active+clean; 208 MiB data, 851 MiB used, 536 GiB / 536 GiB avail; 3.7 MiB/s rd, 1 op/s 2024-04-27T04:20:31.136 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:20:30 smithi162 ceph-mon[110641]: pgmap v224: 65 pgs: 65 active+clean; 208 MiB data, 851 MiB used, 536 GiB / 536 GiB avail; 2.1 MiB/s rd, 1 op/s 2024-04-27T04:20:31.171 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:20:30 smithi181 ceph-mon[81134]: pgmap v224: 65 pgs: 65 active+clean; 208 MiB data, 851 MiB used, 536 GiB / 536 GiB avail; 2.1 MiB/s rd, 1 op/s 2024-04-27T04:20:33.136 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:20:32 smithi162 ceph-mon[110641]: pgmap v225: 65 pgs: 65 active+clean; 208 MiB data, 851 MiB used, 536 GiB / 536 GiB avail; 342 KiB/s rd, 1 op/s 2024-04-27T04:20:33.136 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:20:32 smithi162 ceph-mon[110641]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-27T04:20:33.171 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:20:32 smithi181 ceph-mon[81134]: pgmap v225: 65 pgs: 65 active+clean; 208 MiB data, 851 MiB used, 536 GiB / 536 GiB avail; 342 KiB/s rd, 1 op/s 2024-04-27T04:20:33.171 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:20:32 smithi181 ceph-mon[81134]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-27T04:20:34.167 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:20:34.166+0000 7fc8b9249700 1 -- 172.21.15.162:0/2394459685 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc8b4103780 msgr2=0x7fc8b40fcdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:20:34.167 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:20:34.166+0000 7fc8b9249700 1 --2- 172.21.15.162:0/2394459685 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc8b4103780 0x7fc8b40fcdf0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fc89c004940 tx=0x7fc89c009e30 comp rx=0 tx=0).stop 2024-04-27T04:20:34.167 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:20:34.166+0000 7fc8b9249700 1 -- 172.21.15.162:0/2394459685 shutdown_connections 2024-04-27T04:20:34.167 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:20:34.166+0000 7fc8b9249700 1 --2- 172.21.15.162:0/2394459685 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc8b4103780 0x7fc8b40fcdf0 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:20:34.167 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:20:34.166+0000 7fc8b9249700 1 --2- 172.21.15.162:0/2394459685 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc8b4102de0 0x7fc8b41031b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:20:34.167 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:20:34.166+0000 7fc8b9249700 1 -- 172.21.15.162:0/2394459685 >> 172.21.15.162:0/2394459685 conn(0x7fc8b40f8990 msgr2=0x7fc8b40fada0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:20:34.167 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:20:34.166+0000 7fc8b9249700 1 -- 172.21.15.162:0/2394459685 shutdown_connections 2024-04-27T04:20:34.167 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:20:34.166+0000 7fc8b9249700 1 -- 172.21.15.162:0/2394459685 wait complete. 2024-04-27T04:20:34.167 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:20:34.166+0000 7fc8b9249700 1 Processor -- start 2024-04-27T04:20:34.167 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:20:34.167+0000 7fc8b9249700 1 -- start start 2024-04-27T04:20:34.168 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:20:34.167+0000 7fc8b9249700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc8b4102de0 0x7fc8b4070260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:20:34.168 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:20:34.167+0000 7fc8b9249700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc8b4103780 0x7fc8b40707a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:20:34.168 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:20:34.167+0000 7fc8b9249700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8b40721b0 con 0x7fc8b4102de0 2024-04-27T04:20:34.168 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:20:34.167+0000 7fc8b9249700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8b4070ce0 con 0x7fc8b4103780 2024-04-27T04:20:34.168 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:20:34.167+0000 7fc8b2d9d700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc8b4102de0 0x7fc8b4070260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:20:34.168 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:20:34.167+0000 7fc8b2d9d700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc8b4102de0 0x7fc8b4070260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:54258/0 (socket says 172.21.15.162:54258) 2024-04-27T04:20:34.168 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:20:34.167+0000 7fc8b2d9d700 1 -- 172.21.15.162:0/3078123471 learned_addr learned my addr 172.21.15.162:0/3078123471 (peer_addr_for_me v2:172.21.15.162:0/0) 2024-04-27T04:20:34.168 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:20:34.167+0000 7fc8aa59c700 1 --2- 172.21.15.162:0/3078123471 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc8b4103780 0x7fc8b40707a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:20:34.168 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:20:34.167+0000 7fc8b2d9d700 1 -- 172.21.15.162:0/3078123471 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc8b4103780 msgr2=0x7fc8b40707a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:20:34.168 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:20:34.167+0000 7fc8b2d9d700 1 --2- 172.21.15.162:0/3078123471 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc8b4103780 0x7fc8b40707a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:20:34.168 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:20:34.167+0000 7fc8b2d9d700 1 -- 172.21.15.162:0/3078123471 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc89c005040 con 0x7fc8b4102de0 2024-04-27T04:20:34.168 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:20:34.168+0000 7fc8b2d9d700 1 --2- 172.21.15.162:0/3078123471 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc8b4102de0 0x7fc8b4070260 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fc8a400eb40 tx=0x7fc8a400b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:20:34.168 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:20:34.168+0000 7fc8abfff700 1 -- 172.21.15.162:0/3078123471 <== mon.0 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc8a4007cc0 con 0x7fc8b4102de0 2024-04-27T04:20:34.168 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:20:34.168+0000 7fc8abfff700 1 -- 172.21.15.162:0/3078123471 <== mon.0 v2:172.21.15.162:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fc8a400f040 con 0x7fc8b4102de0 2024-04-27T04:20:34.168 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:20:34.168+0000 7fc8b9249700 1 -- 172.21.15.162:0/3078123471 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc8b4070f60 con 0x7fc8b4102de0 2024-04-27T04:20:34.169 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:20:34.168+0000 7fc8abfff700 1 -- 172.21.15.162:0/3078123471 <== mon.0 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc8a400ca10 con 0x7fc8b4102de0 2024-04-27T04:20:34.169 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:20:34.168+0000 7fc8b9249700 1 -- 172.21.15.162:0/3078123471 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc8b4119fa0 con 0x7fc8b4102de0 2024-04-27T04:20:34.170 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:20:34.169+0000 7fc8b9249700 1 -- 172.21.15.162:0/3078123471 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc8b40fe530 con 0x7fc8b4102de0 2024-04-27T04:20:34.170 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:20:34.169+0000 7fc8abfff700 1 -- 172.21.15.162:0/3078123471 <== mon.0 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 39) v1 ==== 92747+0+0 (secure 0 0 0) 0x7fc8a400cb70 con 0x7fc8b4102de0 2024-04-27T04:20:34.170 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:20:34.170+0000 7fc8abfff700 1 --2- 172.21.15.162:0/3078123471 >> [v2:172.21.15.162:6800/1396866108,v1:172.21.15.162:6801/1396866108] conn(0x7fc8a006ec00 0x7fc8a00710b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:20:34.170 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:20:34.170+0000 7fc8abfff700 1 -- 172.21.15.162:0/3078123471 <== mon.0 v2:172.21.15.162:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6582+0+0 (secure 0 0 0) 0x7fc8a408e3d0 con 0x7fc8b4102de0 2024-04-27T04:20:34.170 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:20:34.170+0000 7fc8aa59c700 1 --2- 172.21.15.162:0/3078123471 >> [v2:172.21.15.162:6800/1396866108,v1:172.21.15.162:6801/1396866108] conn(0x7fc8a006ec00 0x7fc8a00710b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:20:34.170 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:20:34.170+0000 7fc8aa59c700 1 --2- 172.21.15.162:0/3078123471 >> [v2:172.21.15.162:6800/1396866108,v1:172.21.15.162:6801/1396866108] conn(0x7fc8a006ec00 0x7fc8a00710b0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fc89c00b0d0 tx=0x7fc89c009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:20:34.173 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:20:34.175+0000 7fc8abfff700 1 -- 172.21.15.162:0/3078123471 <== mon.0 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7fc8a4057250 con 0x7fc8b4102de0 2024-04-27T04:20:34.390 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:20:34.391+0000 7fc8b9249700 1 -- 172.21.15.162:0/3078123471 --> [v2:172.21.15.162:6800/1396866108,v1:172.21.15.162:6801/1396866108] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fc8b4109f30 con 0x7fc8a006ec00 2024-04-27T04:20:34.391 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:20:34.393+0000 7fc8abfff700 1 -- 172.21.15.162:0/3078123471 <== mgr.34104 v2:172.21.15.162:6800/1396866108 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+175 (secure 0 0 0) 0x7fc8b4109f30 con 0x7fc8a006ec00 2024-04-27T04:20:34.394 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:20:34.395+0000 7fc8b9249700 1 -- 172.21.15.162:0/3078123471 >> [v2:172.21.15.162:6800/1396866108,v1:172.21.15.162:6801/1396866108] conn(0x7fc8a006ec00 msgr2=0x7fc8a00710b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:20:34.394 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:20:34.395+0000 7fc8b9249700 1 --2- 172.21.15.162:0/3078123471 >> [v2:172.21.15.162:6800/1396866108,v1:172.21.15.162:6801/1396866108] conn(0x7fc8a006ec00 0x7fc8a00710b0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fc89c00b0d0 tx=0x7fc89c009e30 comp rx=0 tx=0).stop 2024-04-27T04:20:34.394 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:20:34.395+0000 7fc8b9249700 1 -- 172.21.15.162:0/3078123471 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc8b4102de0 msgr2=0x7fc8b4070260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:20:34.394 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:20:34.395+0000 7fc8b9249700 1 --2- 172.21.15.162:0/3078123471 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc8b4102de0 0x7fc8b4070260 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fc8a400eb40 tx=0x7fc8a400b040 comp rx=0 tx=0).stop 2024-04-27T04:20:34.394 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:20:34.395+0000 7fc8b9249700 1 -- 172.21.15.162:0/3078123471 shutdown_connections 2024-04-27T04:20:34.394 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:20:34.395+0000 7fc8b9249700 1 --2- 172.21.15.162:0/3078123471 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc8b4102de0 0x7fc8b4070260 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:20:34.394 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:20:34.395+0000 7fc8b9249700 1 --2- 172.21.15.162:0/3078123471 >> [v2:172.21.15.162:6800/1396866108,v1:172.21.15.162:6801/1396866108] conn(0x7fc8a006ec00 0x7fc8a00710b0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:20:34.394 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:20:34.395+0000 7fc8b9249700 1 --2- 172.21.15.162:0/3078123471 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc8b4103780 0x7fc8b40707a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:20:34.394 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:20:34.395+0000 7fc8b9249700 1 -- 172.21.15.162:0/3078123471 >> 172.21.15.162:0/3078123471 conn(0x7fc8b40f8990 msgr2=0x7fc8b40f98a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:20:34.394 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:20:34.395+0000 7fc8b9249700 1 -- 172.21.15.162:0/3078123471 shutdown_connections 2024-04-27T04:20:34.395 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:20:34.395+0000 7fc8b9249700 1 -- 172.21.15.162:0/3078123471 wait complete. 2024-04-27T04:20:35.099 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:20:34 smithi162 ceph-mon[110641]: pgmap v226: 65 pgs: 65 active+clean; 208 MiB data, 851 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-04-27T04:20:35.124 DEBUG:teuthology.orchestra.run.smithi162:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 7f9750d6-044a-11ef-bc93-c7b262605968 -e sha1=4e213a58e787437ff508e6c890ccd5fd32dc96b5 -- bash -c 'ceph orch ps' 2024-04-27T04:20:35.171 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:20:34 smithi181 ceph-mon[81134]: pgmap v226: 65 pgs: 65 active+clean; 208 MiB data, 851 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-04-27T04:20:35.380 INFO:teuthology.orchestra.run.smithi162.stderr:Inferring config /var/lib/ceph/7f9750d6-044a-11ef-bc93-c7b262605968/mon.smithi162/config 2024-04-27T04:20:36.136 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:20:35 smithi162 ceph-mon[110641]: from='client.34476 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-27T04:20:36.171 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:20:35 smithi181 ceph-mon[81134]: from='client.34476 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-27T04:20:36.886 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:20:36 smithi162 ceph-mon[110641]: pgmap v227: 65 pgs: 65 active+clean; 208 MiB data, 851 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:20:37.106 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:20:37.106+0000 7efdba88c700 1 -- 172.21.15.162:0/4133352512 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7efdb4101d00 msgr2=0x7efdb41020d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:20:37.106 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:20:37.106+0000 7efdba88c700 1 --2- 172.21.15.162:0/4133352512 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7efdb4101d00 0x7efdb41020d0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7efda4004940 tx=0x7efda4009e30 comp rx=0 tx=0).stop 2024-04-27T04:20:37.106 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:20:37.107+0000 7efdba88c700 1 -- 172.21.15.162:0/4133352512 shutdown_connections 2024-04-27T04:20:37.106 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:20:37.107+0000 7efdba88c700 1 --2- 172.21.15.162:0/4133352512 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7efdb41026a0 0x7efdb40fbd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:20:37.106 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:20:37.107+0000 7efdba88c700 1 --2- 172.21.15.162:0/4133352512 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7efdb4101d00 0x7efdb41020d0 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:20:37.106 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:20:37.107+0000 7efdba88c700 1 -- 172.21.15.162:0/4133352512 >> 172.21.15.162:0/4133352512 conn(0x7efdb40f7870 msgr2=0x7efdb40f9c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:20:37.106 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:20:37.107+0000 7efdba88c700 1 -- 172.21.15.162:0/4133352512 shutdown_connections 2024-04-27T04:20:37.106 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:20:37.107+0000 7efdba88c700 1 -- 172.21.15.162:0/4133352512 wait complete. 2024-04-27T04:20:37.106 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:20:37.107+0000 7efdba88c700 1 Processor -- start 2024-04-27T04:20:37.108 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:20:37.107+0000 7efdba88c700 1 -- start start 2024-04-27T04:20:37.108 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:20:37.108+0000 7efdba88c700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7efdb4101d00 0x7efdb410eb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:20:37.108 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:20:37.108+0000 7efdba88c700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7efdb41026a0 0x7efdb410f080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:20:37.108 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:20:37.108+0000 7efdba88c700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efdb410f5c0 con 0x7efdb41026a0 2024-04-27T04:20:37.108 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:20:37.108+0000 7efdba88c700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efdb410f700 con 0x7efdb4101d00 2024-04-27T04:20:37.108 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:20:37.108+0000 7efdb37fe700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7efdb41026a0 0x7efdb410f080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:20:37.108 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:20:37.108+0000 7efdb37fe700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7efdb41026a0 0x7efdb410f080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:54262/0 (socket says 172.21.15.162:54262) 2024-04-27T04:20:37.108 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:20:37.108+0000 7efdb37fe700 1 -- 172.21.15.162:0/3190569697 learned_addr learned my addr 172.21.15.162:0/3190569697 (peer_addr_for_me v2:172.21.15.162:0/0) 2024-04-27T04:20:37.109 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:20:37.108+0000 7efdb3fff700 1 --2- 172.21.15.162:0/3190569697 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7efdb4101d00 0x7efdb410eb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:20:37.109 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:20:37.108+0000 7efdb37fe700 1 -- 172.21.15.162:0/3190569697 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7efdb4101d00 msgr2=0x7efdb410eb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:20:37.109 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:20:37.108+0000 7efdb37fe700 1 --2- 172.21.15.162:0/3190569697 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7efdb4101d00 0x7efdb410eb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:20:37.109 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:20:37.108+0000 7efdb37fe700 1 -- 172.21.15.162:0/3190569697 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efda4005040 con 0x7efdb41026a0 2024-04-27T04:20:37.109 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:20:37.108+0000 7efdb37fe700 1 --2- 172.21.15.162:0/3190569697 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7efdb41026a0 0x7efdb410f080 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7efda800b790 tx=0x7efda8009220 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:20:37.109 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:20:37.108+0000 7efdb0ff9700 1 -- 172.21.15.162:0/3190569697 <== mon.0 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7efda800f6f0 con 0x7efdb41026a0 2024-04-27T04:20:37.109 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:20:37.108+0000 7efdba88c700 1 -- 172.21.15.162:0/3190569697 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efdb4117bb0 con 0x7efdb41026a0 2024-04-27T04:20:37.109 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:20:37.108+0000 7efdb0ff9700 1 -- 172.21.15.162:0/3190569697 <== mon.0 v2:172.21.15.162:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7efda800fd20 con 0x7efdb41026a0 2024-04-27T04:20:37.109 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:20:37.108+0000 7efdb0ff9700 1 -- 172.21.15.162:0/3190569697 <== mon.0 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7efda80103a0 con 0x7efdb41026a0 2024-04-27T04:20:37.109 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:20:37.108+0000 7efdba88c700 1 -- 172.21.15.162:0/3190569697 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efdb4117fd0 con 0x7efdb41026a0 2024-04-27T04:20:37.109 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:20:37.109+0000 7efdba88c700 1 -- 172.21.15.162:0/3190569697 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efdb40fd450 con 0x7efdb41026a0 2024-04-27T04:20:37.110 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:20:37.110+0000 7efdb0ff9700 1 -- 172.21.15.162:0/3190569697 <== mon.0 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 39) v1 ==== 92747+0+0 (secure 0 0 0) 0x7efda8011400 con 0x7efdb41026a0 2024-04-27T04:20:37.110 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:20:37.110+0000 7efdb0ff9700 1 --2- 172.21.15.162:0/3190569697 >> [v2:172.21.15.162:6800/1396866108,v1:172.21.15.162:6801/1396866108] conn(0x7efd9c06ec00 0x7efd9c0710b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:20:37.110 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:20:37.110+0000 7efdb0ff9700 1 -- 172.21.15.162:0/3190569697 <== mon.0 v2:172.21.15.162:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6582+0+0 (secure 0 0 0) 0x7efda808e660 con 0x7efdb41026a0 2024-04-27T04:20:37.110 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:20:37.110+0000 7efdb3fff700 1 --2- 172.21.15.162:0/3190569697 >> [v2:172.21.15.162:6800/1396866108,v1:172.21.15.162:6801/1396866108] conn(0x7efd9c06ec00 0x7efd9c0710b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:20:37.110 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:20:37.111+0000 7efdb3fff700 1 --2- 172.21.15.162:0/3190569697 >> [v2:172.21.15.162:6800/1396866108,v1:172.21.15.162:6801/1396866108] conn(0x7efd9c06ec00 0x7efd9c0710b0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7efda4004f90 tx=0x7efda4009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:20:37.114 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:20:37.115+0000 7efdb0ff9700 1 -- 172.21.15.162:0/3190569697 <== mon.0 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7efda8057430 con 0x7efdb41026a0 2024-04-27T04:20:37.171 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:20:36 smithi181 ceph-mon[81134]: pgmap v227: 65 pgs: 65 active+clean; 208 MiB data, 851 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:20:37.320 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:20:37.322+0000 7efdba88c700 1 -- 172.21.15.162:0/3190569697 --> [v2:172.21.15.162:6800/1396866108,v1:172.21.15.162:6801/1396866108] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7efdb40f8f10 con 0x7efd9c06ec00 2024-04-27T04:20:37.333 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:20:37.334+0000 7efdb0ff9700 1 -- 172.21.15.162:0/3190569697 <== mgr.34104 v2:172.21.15.162:6800/1396866108 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3520 (secure 0 0 0) 0x7efdb40f8f10 con 0x7efd9c06ec00 2024-04-27T04:20:37.333 INFO:teuthology.orchestra.run.smithi162.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-27T04:20:37.333 INFO:teuthology.orchestra.run.smithi162.stdout:alertmanager.smithi162 smithi162 *:9093,9094 running (7m) 79s ago 18m 27.0M - 0.25.0 c8568f914cd2 40e334659017 2024-04-27T04:20:37.333 INFO:teuthology.orchestra.run.smithi162.stdout:crash.smithi162 smithi162 running (5m) 79s ago 19m 6660k - 19.0.0-3253-g4e213a58 67e414ff4bc4 1aa8f4d5a28b 2024-04-27T04:20:37.333 INFO:teuthology.orchestra.run.smithi162.stdout:crash.smithi181 smithi181 running (5m) 35s ago 18m 6656k - 19.0.0-3253-g4e213a58 67e414ff4bc4 832d9cbc5c30 2024-04-27T04:20:37.333 INFO:teuthology.orchestra.run.smithi162.stdout:grafana.smithi162 smithi162 *:3000 running (7m) 79s ago 18m 79.3M - 9.4.12 2bacad6d85d8 3d78c478f03f 2024-04-27T04:20:37.333 INFO:teuthology.orchestra.run.smithi162.stdout:mds.cephfs.smithi162.gzlyjj smithi162 running (103s) 79s ago 13m 12.5M - 19.0.0-3253-g4e213a58 67e414ff4bc4 7541e8fb8265 2024-04-27T04:20:37.334 INFO:teuthology.orchestra.run.smithi162.stdout:mds.cephfs.smithi162.xcnapk smithi162 running (84s) 79s ago 13m 11.5M - 19.0.0-3253-g4e213a58 67e414ff4bc4 3af4597febc8 2024-04-27T04:20:37.334 INFO:teuthology.orchestra.run.smithi162.stdout:mds.cephfs.smithi181.bkkcdm smithi181 running (38s) 35s ago 13m 14.0M - 19.0.0-3253-g4e213a58 67e414ff4bc4 87e3901b7571 2024-04-27T04:20:37.334 INFO:teuthology.orchestra.run.smithi162.stdout:mds.cephfs.smithi181.dzomgh smithi181 running (59s) 35s ago 13m 12.7M - 19.0.0-3253-g4e213a58 67e414ff4bc4 a07f9164b260 2024-04-27T04:20:37.334 INFO:teuthology.orchestra.run.smithi162.stdout:mgr.smithi162.oennfn smithi162 *:8443,9283,8765 running (9m) 79s ago 20m 511M - 19.0.0-3253-g4e213a58 67e414ff4bc4 58a748d3637b 2024-04-27T04:20:37.334 INFO:teuthology.orchestra.run.smithi162.stdout:mgr.smithi181.vtwuhz smithi181 *:8443,9283,8765 running (8m) 35s ago 18m 452M - 19.0.0-3253-g4e213a58 67e414ff4bc4 813584297b1d 2024-04-27T04:20:37.334 INFO:teuthology.orchestra.run.smithi162.stdout:mon.smithi162 smithi162 running (6m) 79s ago 20m 51.9M 2048M 19.0.0-3253-g4e213a58 67e414ff4bc4 375adaf4468f 2024-04-27T04:20:37.334 INFO:teuthology.orchestra.run.smithi162.stdout:mon.smithi181 smithi181 running (6m) 35s ago 17m 41.7M 2048M 19.0.0-3253-g4e213a58 67e414ff4bc4 77b71f6d522f 2024-04-27T04:20:37.334 INFO:teuthology.orchestra.run.smithi162.stdout:node-exporter.smithi162 smithi162 *:9100 running (8m) 79s ago 19m 15.5M - 1.5.0 0da6a335fe13 5c4c5ea9c1d7 2024-04-27T04:20:37.334 INFO:teuthology.orchestra.run.smithi162.stdout:node-exporter.smithi181 smithi181 *:9100 running (8m) 35s ago 18m 15.3M - 1.5.0 0da6a335fe13 9067f14c0519 2024-04-27T04:20:37.334 INFO:teuthology.orchestra.run.smithi162.stdout:osd.0 smithi162 running (4m) 79s ago 17m 93.1M 1557M 19.0.0-3253-g4e213a58 67e414ff4bc4 2744aff3cd88 2024-04-27T04:20:37.334 INFO:teuthology.orchestra.run.smithi162.stdout:osd.1 smithi162 running (4m) 79s ago 16m 96.6M 1557M 19.0.0-3253-g4e213a58 67e414ff4bc4 668e16faad62 2024-04-27T04:20:37.334 INFO:teuthology.orchestra.run.smithi162.stdout:osd.2 smithi162 running (3m) 79s ago 16m 87.4M 1557M 19.0.0-3253-g4e213a58 67e414ff4bc4 d24878d461e7 2024-04-27T04:20:37.334 INFO:teuthology.orchestra.run.smithi162.stdout:osd.3 smithi181 running (3m) 35s ago 15m 122M 2581M 19.0.0-3253-g4e213a58 67e414ff4bc4 ab473cfd54d2 2024-04-27T04:20:37.334 INFO:teuthology.orchestra.run.smithi162.stdout:osd.4 smithi181 running (2m) 35s ago 15m 101M 2581M 19.0.0-3253-g4e213a58 67e414ff4bc4 de3e10f2d26d 2024-04-27T04:20:37.335 INFO:teuthology.orchestra.run.smithi162.stdout:osd.5 smithi181 running (2m) 35s ago 15m 103M 2581M 19.0.0-3253-g4e213a58 67e414ff4bc4 7dd4f287ef2e 2024-04-27T04:20:37.335 INFO:teuthology.orchestra.run.smithi162.stdout:prometheus.smithi162 smithi162 *:9095 running (8m) 79s ago 18m 67.7M - 2.43.0 a07b618ecd1d f30657a43324 2024-04-27T04:20:37.335 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:20:37.337+0000 7efdba88c700 1 -- 172.21.15.162:0/3190569697 >> [v2:172.21.15.162:6800/1396866108,v1:172.21.15.162:6801/1396866108] conn(0x7efd9c06ec00 msgr2=0x7efd9c0710b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:20:37.335 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:20:37.337+0000 7efdba88c700 1 --2- 172.21.15.162:0/3190569697 >> [v2:172.21.15.162:6800/1396866108,v1:172.21.15.162:6801/1396866108] conn(0x7efd9c06ec00 0x7efd9c0710b0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7efda4004f90 tx=0x7efda4009e30 comp rx=0 tx=0).stop 2024-04-27T04:20:37.336 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:20:37.337+0000 7efdba88c700 1 -- 172.21.15.162:0/3190569697 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7efdb41026a0 msgr2=0x7efdb410f080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:20:37.336 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:20:37.337+0000 7efdba88c700 1 --2- 172.21.15.162:0/3190569697 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7efdb41026a0 0x7efdb410f080 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7efda800b790 tx=0x7efda8009220 comp rx=0 tx=0).stop 2024-04-27T04:20:37.336 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:20:37.337+0000 7efdba88c700 1 -- 172.21.15.162:0/3190569697 shutdown_connections 2024-04-27T04:20:37.336 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:20:37.337+0000 7efdba88c700 1 --2- 172.21.15.162:0/3190569697 >> [v2:172.21.15.162:6800/1396866108,v1:172.21.15.162:6801/1396866108] conn(0x7efd9c06ec00 0x7efd9c0710b0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:20:37.336 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:20:37.337+0000 7efdba88c700 1 --2- 172.21.15.162:0/3190569697 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7efdb4101d00 0x7efdb410eb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:20:37.336 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:20:37.337+0000 7efdba88c700 1 --2- 172.21.15.162:0/3190569697 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7efdb41026a0 0x7efdb410f080 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:20:37.336 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:20:37.337+0000 7efdba88c700 1 -- 172.21.15.162:0/3190569697 >> 172.21.15.162:0/3190569697 conn(0x7efdb40f7870 msgr2=0x7efdb40f8800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:20:37.336 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:20:37.337+0000 7efdba88c700 1 -- 172.21.15.162:0/3190569697 shutdown_connections 2024-04-27T04:20:37.336 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:20:37.337+0000 7efdba88c700 1 -- 172.21.15.162:0/3190569697 wait complete. 2024-04-27T04:20:38.084 DEBUG:teuthology.orchestra.run.smithi162:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 7f9750d6-044a-11ef-bc93-c7b262605968 -e sha1=4e213a58e787437ff508e6c890ccd5fd32dc96b5 -- bash -c 'ceph orch upgrade status' 2024-04-27T04:20:38.345 INFO:teuthology.orchestra.run.smithi162.stderr:Inferring config /var/lib/ceph/7f9750d6-044a-11ef-bc93-c7b262605968/mon.smithi162/config 2024-04-27T04:20:39.136 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:20:38 smithi162 ceph-mon[110641]: from='client.34480 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-27T04:20:39.136 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:20:38 smithi162 ceph-mon[110641]: pgmap v228: 65 pgs: 65 active+clean; 208 MiB data, 851 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:20:39.171 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:20:38 smithi181 ceph-mon[81134]: from='client.34480 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-27T04:20:39.171 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:20:38 smithi181 ceph-mon[81134]: pgmap v228: 65 pgs: 65 active+clean; 208 MiB data, 851 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:20:39.965 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:20:39.966+0000 7f409c3e9700 1 -- 172.21.15.162:0/2583966965 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f40941037a0 msgr2=0x7f40940fce10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:20:39.966 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:20:39.966+0000 7f409c3e9700 1 --2- 172.21.15.162:0/2583966965 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f40941037a0 0x7f40940fce10 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f4084004940 tx=0x7f4084009e30 comp rx=0 tx=0).stop 2024-04-27T04:20:39.966 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:20:39.966+0000 7f409c3e9700 1 -- 172.21.15.162:0/2583966965 shutdown_connections 2024-04-27T04:20:39.966 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:20:39.966+0000 7f409c3e9700 1 --2- 172.21.15.162:0/2583966965 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f40941037a0 0x7f40940fce10 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:20:39.966 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:20:39.966+0000 7f409c3e9700 1 --2- 172.21.15.162:0/2583966965 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4094102e00 0x7f40941031d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:20:39.966 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:20:39.966+0000 7f409c3e9700 1 -- 172.21.15.162:0/2583966965 >> 172.21.15.162:0/2583966965 conn(0x7f40940f89b0 msgr2=0x7f40940fadc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:20:39.966 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:20:39.966+0000 7f409c3e9700 1 -- 172.21.15.162:0/2583966965 shutdown_connections 2024-04-27T04:20:39.966 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:20:39.966+0000 7f409c3e9700 1 -- 172.21.15.162:0/2583966965 wait complete. 2024-04-27T04:20:39.966 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:20:39.967+0000 7f409c3e9700 1 Processor -- start 2024-04-27T04:20:39.966 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:20:39.967+0000 7f409c3e9700 1 -- start start 2024-04-27T04:20:39.966 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:20:39.967+0000 7f409c3e9700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4094102e00 0x7f4094113960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:20:39.966 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:20:39.967+0000 7f409c3e9700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f40941037a0 0x7f4094113ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:20:39.967 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:20:39.967+0000 7f409c3e9700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40941144f0 con 0x7f40941037a0 2024-04-27T04:20:39.967 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:20:39.967+0000 7f409c3e9700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f409410daf0 con 0x7f4094102e00 2024-04-27T04:20:39.967 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:20:39.967+0000 7f4099984700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f40941037a0 0x7f4094113ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:20:39.967 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:20:39.967+0000 7f4099984700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f40941037a0 0x7f4094113ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:54272/0 (socket says 172.21.15.162:54272) 2024-04-27T04:20:39.967 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:20:39.967+0000 7f4099984700 1 -- 172.21.15.162:0/667338985 learned_addr learned my addr 172.21.15.162:0/667338985 (peer_addr_for_me v2:172.21.15.162:0/0) 2024-04-27T04:20:39.967 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:20:39.967+0000 7f409a185700 1 --2- 172.21.15.162:0/667338985 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4094102e00 0x7f4094113960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:20:39.967 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:20:39.967+0000 7f4099984700 1 -- 172.21.15.162:0/667338985 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4094102e00 msgr2=0x7f4094113960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:20:39.967 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:20:39.967+0000 7f4099984700 1 --2- 172.21.15.162:0/667338985 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4094102e00 0x7f4094113960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:20:39.967 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:20:39.967+0000 7f4099984700 1 -- 172.21.15.162:0/667338985 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4084005040 con 0x7f40941037a0 2024-04-27T04:20:39.967 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:20:39.968+0000 7f4099984700 1 --2- 172.21.15.162:0/667338985 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f40941037a0 0x7f4094113ea0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f4084009100 tx=0x7f4084009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:20:39.967 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:20:39.968+0000 7f408affd700 1 -- 172.21.15.162:0/667338985 <== mon.0 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f408400e040 con 0x7f40941037a0 2024-04-27T04:20:39.967 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:20:39.968+0000 7f409c3e9700 1 -- 172.21.15.162:0/667338985 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f409410dd70 con 0x7f40941037a0 2024-04-27T04:20:39.967 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:20:39.968+0000 7f408affd700 1 -- 172.21.15.162:0/667338985 <== mon.0 v2:172.21.15.162:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f4084009af0 con 0x7f40941037a0 2024-04-27T04:20:39.967 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:20:39.968+0000 7f408affd700 1 -- 172.21.15.162:0/667338985 <== mon.0 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f408401c3f0 con 0x7f40941037a0 2024-04-27T04:20:39.968 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:20:39.968+0000 7f409c3e9700 1 -- 172.21.15.162:0/667338985 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f409410e330 con 0x7f40941037a0 2024-04-27T04:20:39.968 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:20:39.969+0000 7f409c3e9700 1 -- 172.21.15.162:0/667338985 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f40941052d0 con 0x7f40941037a0 2024-04-27T04:20:39.969 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:20:39.969+0000 7f408affd700 1 -- 172.21.15.162:0/667338985 <== mon.0 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 39) v1 ==== 92747+0+0 (secure 0 0 0) 0x7f408400baf0 con 0x7f40941037a0 2024-04-27T04:20:39.969 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:20:39.969+0000 7f408affd700 1 --2- 172.21.15.162:0/667338985 >> [v2:172.21.15.162:6800/1396866108,v1:172.21.15.162:6801/1396866108] conn(0x7f4080073010 0x7f40800754c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:20:39.969 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:20:39.970+0000 7f408affd700 1 -- 172.21.15.162:0/667338985 <== mon.0 v2:172.21.15.162:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6582+0+0 (secure 0 0 0) 0x7f408408fb30 con 0x7f40941037a0 2024-04-27T04:20:39.969 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:20:39.970+0000 7f409a185700 1 --2- 172.21.15.162:0/667338985 >> [v2:172.21.15.162:6800/1396866108,v1:172.21.15.162:6801/1396866108] conn(0x7f4080073010 0x7f40800754c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:20:39.969 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:20:39.970+0000 7f409a185700 1 --2- 172.21.15.162:0/667338985 >> [v2:172.21.15.162:6800/1396866108,v1:172.21.15.162:6801/1396866108] conn(0x7f4080073010 0x7f40800754c0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f4090007970 tx=0x7f4090009da0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:20:39.972 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:20:39.974+0000 7f408affd700 1 -- 172.21.15.162:0/667338985 <== mon.0 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7f408405c430 con 0x7f40941037a0 2024-04-27T04:20:40.191 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:20:40.193+0000 7f409c3e9700 1 -- 172.21.15.162:0/667338985 --> [v2:172.21.15.162:6800/1396866108,v1:172.21.15.162:6801/1396866108] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f409410efb0 con 0x7f4080073010 2024-04-27T04:20:40.192 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:20:40.195+0000 7f408affd700 1 -- 172.21.15.162:0/667338985 <== mgr.34104 v2:172.21.15.162:6800/1396866108 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+175 (secure 0 0 0) 0x7f409410efb0 con 0x7f4080073010 2024-04-27T04:20:40.192 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-04-27T04:20:40.192 INFO:teuthology.orchestra.run.smithi162.stdout: "target_image": null, 2024-04-27T04:20:40.193 INFO:teuthology.orchestra.run.smithi162.stdout: "in_progress": false, 2024-04-27T04:20:40.193 INFO:teuthology.orchestra.run.smithi162.stdout: "which": "", 2024-04-27T04:20:40.193 INFO:teuthology.orchestra.run.smithi162.stdout: "services_complete": [], 2024-04-27T04:20:40.193 INFO:teuthology.orchestra.run.smithi162.stdout: "progress": null, 2024-04-27T04:20:40.193 INFO:teuthology.orchestra.run.smithi162.stdout: "message": "", 2024-04-27T04:20:40.193 INFO:teuthology.orchestra.run.smithi162.stdout: "is_paused": false 2024-04-27T04:20:40.193 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-04-27T04:20:40.195 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:20:40.197+0000 7f409c3e9700 1 -- 172.21.15.162:0/667338985 >> [v2:172.21.15.162:6800/1396866108,v1:172.21.15.162:6801/1396866108] conn(0x7f4080073010 msgr2=0x7f40800754c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:20:40.195 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:20:40.197+0000 7f409c3e9700 1 --2- 172.21.15.162:0/667338985 >> [v2:172.21.15.162:6800/1396866108,v1:172.21.15.162:6801/1396866108] conn(0x7f4080073010 0x7f40800754c0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f4090007970 tx=0x7f4090009da0 comp rx=0 tx=0).stop 2024-04-27T04:20:40.195 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:20:40.197+0000 7f409c3e9700 1 -- 172.21.15.162:0/667338985 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f40941037a0 msgr2=0x7f4094113ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:20:40.195 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:20:40.197+0000 7f409c3e9700 1 --2- 172.21.15.162:0/667338985 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f40941037a0 0x7f4094113ea0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f4084009100 tx=0x7f4084009e30 comp rx=0 tx=0).stop 2024-04-27T04:20:40.195 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:20:40.197+0000 7f409c3e9700 1 -- 172.21.15.162:0/667338985 shutdown_connections 2024-04-27T04:20:40.195 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:20:40.197+0000 7f409c3e9700 1 --2- 172.21.15.162:0/667338985 >> [v2:172.21.15.162:6800/1396866108,v1:172.21.15.162:6801/1396866108] conn(0x7f4080073010 0x7f40800754c0 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:20:40.195 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:20:40.197+0000 7f409c3e9700 1 --2- 172.21.15.162:0/667338985 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4094102e00 0x7f4094113960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:20:40.195 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:20:40.197+0000 7f409c3e9700 1 --2- 172.21.15.162:0/667338985 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f40941037a0 0x7f4094113ea0 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:20:40.195 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:20:40.197+0000 7f409c3e9700 1 -- 172.21.15.162:0/667338985 >> 172.21.15.162:0/667338985 conn(0x7f40940f89b0 msgr2=0x7f4094102190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:20:40.195 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:20:40.197+0000 7f409c3e9700 1 -- 172.21.15.162:0/667338985 shutdown_connections 2024-04-27T04:20:40.195 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:20:40.197+0000 7f409c3e9700 1 -- 172.21.15.162:0/667338985 wait complete. 2024-04-27T04:20:40.858 DEBUG:teuthology.orchestra.run.smithi162:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 7f9750d6-044a-11ef-bc93-c7b262605968 -e sha1=4e213a58e787437ff508e6c890ccd5fd32dc96b5 -- bash -c 'ceph health detail' 2024-04-27T04:20:41.114 INFO:teuthology.orchestra.run.smithi162.stderr:Inferring config /var/lib/ceph/7f9750d6-044a-11ef-bc93-c7b262605968/mon.smithi162/config 2024-04-27T04:20:41.136 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:20:40 smithi162 ceph-mon[110641]: pgmap v229: 65 pgs: 65 active+clean; 208 MiB data, 851 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-04-27T04:20:41.171 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:20:40 smithi181 ceph-mon[81134]: pgmap v229: 65 pgs: 65 active+clean; 208 MiB data, 851 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-04-27T04:20:42.136 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:20:41 smithi162 ceph-mon[110641]: from='client.34484 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-27T04:20:42.171 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:20:41 smithi181 ceph-mon[81134]: from='client.34484 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-27T04:20:42.703 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:20:42.703+0000 7f070ea52700 1 -- 172.21.15.162:0/4112427554 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0708101d90 msgr2=0x7f0708102160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:20:42.703 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:20:42.703+0000 7f070ea52700 1 --2- 172.21.15.162:0/4112427554 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0708101d90 0x7f0708102160 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f06f8004940 tx=0x7f06f8009e30 comp rx=0 tx=0).stop 2024-04-27T04:20:42.703 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:20:42.703+0000 7f070ea52700 1 -- 172.21.15.162:0/4112427554 shutdown_connections 2024-04-27T04:20:42.703 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:20:42.703+0000 7f070ea52700 1 --2- 172.21.15.162:0/4112427554 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f07080fbd40 0x7f07080fc1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:20:42.703 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:20:42.703+0000 7f070ea52700 1 --2- 172.21.15.162:0/4112427554 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0708101d90 0x7f0708102160 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:20:42.703 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:20:42.703+0000 7f070ea52700 1 -- 172.21.15.162:0/4112427554 >> 172.21.15.162:0/4112427554 conn(0x7f0708071d00 msgr2=0x7f0708072100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:20:42.703 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:20:42.703+0000 7f070ea52700 1 -- 172.21.15.162:0/4112427554 shutdown_connections 2024-04-27T04:20:42.703 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:20:42.703+0000 7f070ea52700 1 -- 172.21.15.162:0/4112427554 wait complete. 2024-04-27T04:20:42.704 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:20:42.704+0000 7f070ea52700 1 Processor -- start 2024-04-27T04:20:42.704 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:20:42.704+0000 7f070ea52700 1 -- start start 2024-04-27T04:20:42.704 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:20:42.704+0000 7f070ea52700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f07080fbd40 0x7f070810eaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:20:42.704 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:20:42.704+0000 7f070ea52700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0708101d90 0x7f070810efe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:20:42.704 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:20:42.704+0000 7f070ea52700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f070810f630 con 0x7f07080fbd40 2024-04-27T04:20:42.704 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:20:42.704+0000 7f070ea52700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f070810f770 con 0x7f0708101d90 2024-04-27T04:20:42.704 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:20:42.704+0000 7f0707fff700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f07080fbd40 0x7f070810eaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:20:42.704 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:20:42.704+0000 7f0707fff700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f07080fbd40 0x7f070810eaa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:54278/0 (socket says 172.21.15.162:54278) 2024-04-27T04:20:42.704 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:20:42.704+0000 7f0707fff700 1 -- 172.21.15.162:0/1357733362 learned_addr learned my addr 172.21.15.162:0/1357733362 (peer_addr_for_me v2:172.21.15.162:0/0) 2024-04-27T04:20:42.704 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:20:42.704+0000 7f07077fe700 1 --2- 172.21.15.162:0/1357733362 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0708101d90 0x7f070810efe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:20:42.704 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:20:42.705+0000 7f0707fff700 1 -- 172.21.15.162:0/1357733362 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0708101d90 msgr2=0x7f070810efe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:20:42.704 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:20:42.705+0000 7f0707fff700 1 --2- 172.21.15.162:0/1357733362 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0708101d90 0x7f070810efe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:20:42.704 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:20:42.705+0000 7f0707fff700 1 -- 172.21.15.162:0/1357733362 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f06f8005040 con 0x7f07080fbd40 2024-04-27T04:20:42.704 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:20:42.705+0000 7f0707fff700 1 --2- 172.21.15.162:0/1357733362 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f07080fbd40 0x7f070810eaa0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f06f800b8b0 tx=0x7f06f8009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:20:42.704 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:20:42.705+0000 7f0704ff9700 1 -- 172.21.15.162:0/1357733362 <== mon.0 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f06f800e040 con 0x7f07080fbd40 2024-04-27T04:20:42.705 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:20:42.705+0000 7f070ea52700 1 -- 172.21.15.162:0/1357733362 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0708117b10 con 0x7f07080fbd40 2024-04-27T04:20:42.705 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:20:42.705+0000 7f0704ff9700 1 -- 172.21.15.162:0/1357733362 <== mon.0 v2:172.21.15.162:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f06f8013420 con 0x7f07080fbd40 2024-04-27T04:20:42.705 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:20:42.705+0000 7f0704ff9700 1 -- 172.21.15.162:0/1357733362 <== mon.0 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f06f801c630 con 0x7f07080fbd40 2024-04-27T04:20:42.705 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:20:42.705+0000 7f070ea52700 1 -- 172.21.15.162:0/1357733362 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0708118020 con 0x7f07080fbd40 2024-04-27T04:20:42.706 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:20:42.706+0000 7f070ea52700 1 -- 172.21.15.162:0/1357733362 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f07080fd430 con 0x7f07080fbd40 2024-04-27T04:20:42.706 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:20:42.706+0000 7f0704ff9700 1 -- 172.21.15.162:0/1357733362 <== mon.0 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 39) v1 ==== 92747+0+0 (secure 0 0 0) 0x7f06f8013a00 con 0x7f07080fbd40 2024-04-27T04:20:42.706 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:20:42.707+0000 7f0704ff9700 1 --2- 172.21.15.162:0/1357733362 >> [v2:172.21.15.162:6800/1396866108,v1:172.21.15.162:6801/1396866108] conn(0x7f06f006ec00 0x7f06f00710b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:20:42.706 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:20:42.707+0000 7f0704ff9700 1 -- 172.21.15.162:0/1357733362 <== mon.0 v2:172.21.15.162:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6582+0+0 (secure 0 0 0) 0x7f06f808fc40 con 0x7f07080fbd40 2024-04-27T04:20:42.706 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:20:42.707+0000 7f07077fe700 1 --2- 172.21.15.162:0/1357733362 >> [v2:172.21.15.162:6800/1396866108,v1:172.21.15.162:6801/1396866108] conn(0x7f06f006ec00 0x7f06f00710b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:20:42.706 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:20:42.707+0000 7f07077fe700 1 --2- 172.21.15.162:0/1357733362 >> [v2:172.21.15.162:6800/1396866108,v1:172.21.15.162:6801/1396866108] conn(0x7f06f006ec00 0x7f06f00710b0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f06fc009fd0 tx=0x7f06fc009f60 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:20:42.709 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:20:42.712+0000 7f0704ff9700 1 -- 172.21.15.162:0/1357733362 <== mon.0 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7f06f80589b0 con 0x7f07080fbd40 2024-04-27T04:20:42.995 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:20:42.997+0000 7f070ea52700 1 -- 172.21.15.162:0/1357733362 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f070805f5c0 con 0x7f07080fbd40 2024-04-27T04:20:42.995 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:20:42.998+0000 7f0704ff9700 1 -- 172.21.15.162:0/1357733362 <== mon.0 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+10 (secure 0 0 0) 0x7f06f805c070 con 0x7f07080fbd40 2024-04-27T04:20:42.995 INFO:teuthology.orchestra.run.smithi162.stdout:HEALTH_OK 2024-04-27T04:20:42.997 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:20:43.000+0000 7f070ea52700 1 -- 172.21.15.162:0/1357733362 >> [v2:172.21.15.162:6800/1396866108,v1:172.21.15.162:6801/1396866108] conn(0x7f06f006ec00 msgr2=0x7f06f00710b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:20:42.998 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:20:43.000+0000 7f070ea52700 1 --2- 172.21.15.162:0/1357733362 >> [v2:172.21.15.162:6800/1396866108,v1:172.21.15.162:6801/1396866108] conn(0x7f06f006ec00 0x7f06f00710b0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f06fc009fd0 tx=0x7f06fc009f60 comp rx=0 tx=0).stop 2024-04-27T04:20:42.998 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:20:43.000+0000 7f070ea52700 1 -- 172.21.15.162:0/1357733362 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f07080fbd40 msgr2=0x7f070810eaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:20:42.998 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:20:43.000+0000 7f070ea52700 1 --2- 172.21.15.162:0/1357733362 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f07080fbd40 0x7f070810eaa0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f06f800b8b0 tx=0x7f06f8009e30 comp rx=0 tx=0).stop 2024-04-27T04:20:42.998 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:20:43.000+0000 7f070ea52700 1 -- 172.21.15.162:0/1357733362 shutdown_connections 2024-04-27T04:20:42.998 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:20:43.000+0000 7f070ea52700 1 --2- 172.21.15.162:0/1357733362 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f07080fbd40 0x7f070810eaa0 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:20:42.998 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:20:43.000+0000 7f070ea52700 1 --2- 172.21.15.162:0/1357733362 >> [v2:172.21.15.162:6800/1396866108,v1:172.21.15.162:6801/1396866108] conn(0x7f06f006ec00 0x7f06f00710b0 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:20:42.998 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:20:43.000+0000 7f070ea52700 1 --2- 172.21.15.162:0/1357733362 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0708101d90 0x7f070810efe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:20:42.998 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:20:43.000+0000 7f070ea52700 1 -- 172.21.15.162:0/1357733362 >> 172.21.15.162:0/1357733362 conn(0x7f0708071d00 msgr2=0x7f07080fa030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:20:42.998 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:20:43.000+0000 7f070ea52700 1 -- 172.21.15.162:0/1357733362 shutdown_connections 2024-04-27T04:20:42.998 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:20:43.000+0000 7f070ea52700 1 -- 172.21.15.162:0/1357733362 wait complete. 2024-04-27T04:20:43.136 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:20:42 smithi162 ceph-mon[110641]: pgmap v230: 65 pgs: 65 active+clean; 208 MiB data, 851 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:20:43.171 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:20:42 smithi181 ceph-mon[81134]: pgmap v230: 65 pgs: 65 active+clean; 208 MiB data, 851 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:20:43.919 DEBUG:teuthology.orchestra.run.smithi162:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 7f9750d6-044a-11ef-bc93-c7b262605968 -e sha1=4e213a58e787437ff508e6c890ccd5fd32dc96b5 -- bash -c 'ceph versions' 2024-04-27T04:20:44.136 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:20:43 smithi162 ceph-mon[110641]: from='client.? 172.21.15.162:0/1357733362' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-27T04:20:44.171 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:20:43 smithi181 ceph-mon[81134]: from='client.? 172.21.15.162:0/1357733362' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-27T04:20:44.180 INFO:teuthology.orchestra.run.smithi162.stderr:Inferring config /var/lib/ceph/7f9750d6-044a-11ef-bc93-c7b262605968/mon.smithi162/config 2024-04-27T04:20:44.990 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:20:44 smithi162 ceph-mon[110641]: pgmap v231: 65 pgs: 65 active+clean; 208 MiB data, 851 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-04-27T04:20:45.171 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:20:44 smithi181 ceph-mon[81134]: pgmap v231: 65 pgs: 65 active+clean; 208 MiB data, 851 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-04-27T04:20:45.828 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:20:45.820+0000 7fa6bb15b700 1 -- 172.21.15.162:0/2183214074 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa6b4102de0 msgr2=0x7fa6b41031b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:20:45.828 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:20:45.820+0000 7fa6bb15b700 1 --2- 172.21.15.162:0/2183214074 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa6b4102de0 0x7fa6b41031b0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fa6a8004940 tx=0x7fa6a8009e30 comp rx=0 tx=0).stop 2024-04-27T04:20:45.829 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:20:45.820+0000 7fa6bb15b700 1 -- 172.21.15.162:0/2183214074 shutdown_connections 2024-04-27T04:20:45.829 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:20:45.820+0000 7fa6bb15b700 1 --2- 172.21.15.162:0/2183214074 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa6b4103780 0x7fa6b40fcdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:20:45.829 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:20:45.820+0000 7fa6bb15b700 1 --2- 172.21.15.162:0/2183214074 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa6b4102de0 0x7fa6b41031b0 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:20:45.829 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:20:45.820+0000 7fa6bb15b700 1 -- 172.21.15.162:0/2183214074 >> 172.21.15.162:0/2183214074 conn(0x7fa6b40f8990 msgr2=0x7fa6b40fada0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:20:45.829 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:20:45.820+0000 7fa6bb15b700 1 -- 172.21.15.162:0/2183214074 shutdown_connections 2024-04-27T04:20:45.829 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:20:45.820+0000 7fa6bb15b700 1 -- 172.21.15.162:0/2183214074 wait complete. 2024-04-27T04:20:45.829 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:20:45.820+0000 7fa6bb15b700 1 Processor -- start 2024-04-27T04:20:45.829 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:20:45.820+0000 7fa6bb15b700 1 -- start start 2024-04-27T04:20:45.829 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:20:45.821+0000 7fa6bb15b700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa6b4102de0 0x7fa6b4070260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:20:45.829 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:20:45.821+0000 7fa6bb15b700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa6b4103780 0x7fa6b40707a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:20:45.829 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:20:45.821+0000 7fa6bb15b700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6b4072240 con 0x7fa6b4102de0 2024-04-27T04:20:45.829 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:20:45.821+0000 7fa6bb15b700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6b4070ce0 con 0x7fa6b4103780 2024-04-27T04:20:45.830 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:20:45.821+0000 7fa6b8ef7700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa6b4102de0 0x7fa6b4070260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:20:45.830 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:20:45.821+0000 7fa6b8ef7700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa6b4102de0 0x7fa6b4070260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:35480/0 (socket says 172.21.15.162:35480) 2024-04-27T04:20:45.830 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:20:45.821+0000 7fa6b8ef7700 1 -- 172.21.15.162:0/1156817560 learned_addr learned my addr 172.21.15.162:0/1156817560 (peer_addr_for_me v2:172.21.15.162:0/0) 2024-04-27T04:20:45.830 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:20:45.821+0000 7fa6b3fff700 1 --2- 172.21.15.162:0/1156817560 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa6b4103780 0x7fa6b40707a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:20:45.830 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:20:45.821+0000 7fa6b8ef7700 1 -- 172.21.15.162:0/1156817560 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa6b4103780 msgr2=0x7fa6b40707a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:20:45.830 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:20:45.821+0000 7fa6b8ef7700 1 --2- 172.21.15.162:0/1156817560 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa6b4103780 0x7fa6b40707a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:20:45.830 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:20:45.821+0000 7fa6b8ef7700 1 -- 172.21.15.162:0/1156817560 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6a8005040 con 0x7fa6b4102de0 2024-04-27T04:20:45.830 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:20:45.821+0000 7fa6b8ef7700 1 --2- 172.21.15.162:0/1156817560 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa6b4102de0 0x7fa6b4070260 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fa6a8009150 tx=0x7fa6a8009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:20:45.830 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:20:45.821+0000 7fa6b17fa700 1 -- 172.21.15.162:0/1156817560 <== mon.0 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa6a800e040 con 0x7fa6b4102de0 2024-04-27T04:20:45.830 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:20:45.821+0000 7fa6bb15b700 1 -- 172.21.15.162:0/1156817560 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa6b4070f60 con 0x7fa6b4102de0 2024-04-27T04:20:45.830 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:20:45.821+0000 7fa6b17fa700 1 -- 172.21.15.162:0/1156817560 <== mon.0 v2:172.21.15.162:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fa6a8013420 con 0x7fa6b4102de0 2024-04-27T04:20:45.831 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:20:45.821+0000 7fa6b17fa700 1 -- 172.21.15.162:0/1156817560 <== mon.0 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa6a801c610 con 0x7fa6b4102de0 2024-04-27T04:20:45.831 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:20:45.822+0000 7fa6bb15b700 1 -- 172.21.15.162:0/1156817560 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa6b411a130 con 0x7fa6b4102de0 2024-04-27T04:20:45.831 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:20:45.822+0000 7fa6bb15b700 1 -- 172.21.15.162:0/1156817560 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa6b41000f0 con 0x7fa6b4102de0 2024-04-27T04:20:45.831 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:20:45.823+0000 7fa6b17fa700 1 -- 172.21.15.162:0/1156817560 <== mon.0 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 39) v1 ==== 92747+0+0 (secure 0 0 0) 0x7fa6a800baf0 con 0x7fa6b4102de0 2024-04-27T04:20:45.831 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:20:45.823+0000 7fa6b17fa700 1 --2- 172.21.15.162:0/1156817560 >> [v2:172.21.15.162:6800/1396866108,v1:172.21.15.162:6801/1396866108] conn(0x7fa6a406ec00 0x7fa6a40710b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:20:45.831 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:20:45.823+0000 7fa6b17fa700 1 -- 172.21.15.162:0/1156817560 <== mon.0 v2:172.21.15.162:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6582+0+0 (secure 0 0 0) 0x7fa6a808faa0 con 0x7fa6b4102de0 2024-04-27T04:20:45.831 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:20:45.823+0000 7fa6b3fff700 1 --2- 172.21.15.162:0/1156817560 >> [v2:172.21.15.162:6800/1396866108,v1:172.21.15.162:6801/1396866108] conn(0x7fa6a406ec00 0x7fa6a40710b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:20:45.831 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:20:45.824+0000 7fa6b3fff700 1 --2- 172.21.15.162:0/1156817560 >> [v2:172.21.15.162:6800/1396866108,v1:172.21.15.162:6801/1396866108] conn(0x7fa6a406ec00 0x7fa6a40710b0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fa6b4071990 tx=0x7fa6a00091a0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:20:45.831 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:20:45.828+0000 7fa6b17fa700 1 -- 172.21.15.162:0/1156817560 <== mon.0 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7fa6a8058870 con 0x7fa6b4102de0 2024-04-27T04:20:46.112 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:20:46.114+0000 7fa6bb15b700 1 -- 172.21.15.162:0/1156817560 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7fa6b4109f80 con 0x7fa6b4102de0 2024-04-27T04:20:46.112 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:20:46.115+0000 7fa6b17fa700 1 -- 172.21.15.162:0/1156817560 <== mon.0 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+623 (secure 0 0 0) 0x7fa6a805bf30 con 0x7fa6b4102de0 2024-04-27T04:20:46.113 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-04-27T04:20:46.113 INFO:teuthology.orchestra.run.smithi162.stdout: "mon": { 2024-04-27T04:20:46.113 INFO:teuthology.orchestra.run.smithi162.stdout: "ceph version 19.0.0-3253-g4e213a58 (4e213a58e787437ff508e6c890ccd5fd32dc96b5) squid (dev)": 2 2024-04-27T04:20:46.113 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-04-27T04:20:46.113 INFO:teuthology.orchestra.run.smithi162.stdout: "mgr": { 2024-04-27T04:20:46.113 INFO:teuthology.orchestra.run.smithi162.stdout: "ceph version 19.0.0-3253-g4e213a58 (4e213a58e787437ff508e6c890ccd5fd32dc96b5) squid (dev)": 2 2024-04-27T04:20:46.113 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-04-27T04:20:46.113 INFO:teuthology.orchestra.run.smithi162.stdout: "osd": { 2024-04-27T04:20:46.113 INFO:teuthology.orchestra.run.smithi162.stdout: "ceph version 19.0.0-3253-g4e213a58 (4e213a58e787437ff508e6c890ccd5fd32dc96b5) squid (dev)": 6 2024-04-27T04:20:46.113 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-04-27T04:20:46.113 INFO:teuthology.orchestra.run.smithi162.stdout: "mds": { 2024-04-27T04:20:46.113 INFO:teuthology.orchestra.run.smithi162.stdout: "ceph version 19.0.0-3253-g4e213a58 (4e213a58e787437ff508e6c890ccd5fd32dc96b5) squid (dev)": 4 2024-04-27T04:20:46.114 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-04-27T04:20:46.114 INFO:teuthology.orchestra.run.smithi162.stdout: "overall": { 2024-04-27T04:20:46.114 INFO:teuthology.orchestra.run.smithi162.stdout: "ceph version 19.0.0-3253-g4e213a58 (4e213a58e787437ff508e6c890ccd5fd32dc96b5) squid (dev)": 14 2024-04-27T04:20:46.114 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-04-27T04:20:46.114 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-04-27T04:20:46.115 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:20:46.117+0000 7fa6bb15b700 1 -- 172.21.15.162:0/1156817560 >> [v2:172.21.15.162:6800/1396866108,v1:172.21.15.162:6801/1396866108] conn(0x7fa6a406ec00 msgr2=0x7fa6a40710b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:20:46.115 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:20:46.117+0000 7fa6bb15b700 1 --2- 172.21.15.162:0/1156817560 >> [v2:172.21.15.162:6800/1396866108,v1:172.21.15.162:6801/1396866108] conn(0x7fa6a406ec00 0x7fa6a40710b0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fa6b4071990 tx=0x7fa6a00091a0 comp rx=0 tx=0).stop 2024-04-27T04:20:46.115 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:20:46.117+0000 7fa6bb15b700 1 -- 172.21.15.162:0/1156817560 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa6b4102de0 msgr2=0x7fa6b4070260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:20:46.116 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:20:46.117+0000 7fa6bb15b700 1 --2- 172.21.15.162:0/1156817560 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa6b4102de0 0x7fa6b4070260 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fa6a8009150 tx=0x7fa6a8009e30 comp rx=0 tx=0).stop 2024-04-27T04:20:46.116 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:20:46.118+0000 7fa6bb15b700 1 -- 172.21.15.162:0/1156817560 shutdown_connections 2024-04-27T04:20:46.116 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:20:46.118+0000 7fa6bb15b700 1 --2- 172.21.15.162:0/1156817560 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa6b4102de0 0x7fa6b4070260 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:20:46.116 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:20:46.118+0000 7fa6bb15b700 1 --2- 172.21.15.162:0/1156817560 >> [v2:172.21.15.162:6800/1396866108,v1:172.21.15.162:6801/1396866108] conn(0x7fa6a406ec00 0x7fa6a40710b0 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:20:46.116 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:20:46.118+0000 7fa6bb15b700 1 --2- 172.21.15.162:0/1156817560 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa6b4103780 0x7fa6b40707a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:20:46.116 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:20:46.118+0000 7fa6bb15b700 1 -- 172.21.15.162:0/1156817560 >> 172.21.15.162:0/1156817560 conn(0x7fa6b40f8990 msgr2=0x7fa6b40fa6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:20:46.116 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:20:46.118+0000 7fa6bb15b700 1 -- 172.21.15.162:0/1156817560 shutdown_connections 2024-04-27T04:20:46.116 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:20:46.118+0000 7fa6bb15b700 1 -- 172.21.15.162:0/1156817560 wait complete. 2024-04-27T04:20:46.718 DEBUG:teuthology.orchestra.run.smithi162:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 7f9750d6-044a-11ef-bc93-c7b262605968 -e sha1=4e213a58e787437ff508e6c890ccd5fd32dc96b5 -- bash -c 'echo "wait for servicemap items w/ changing names to refresh"' 2024-04-27T04:20:46.978 INFO:teuthology.orchestra.run.smithi162.stderr:Inferring config /var/lib/ceph/7f9750d6-044a-11ef-bc93-c7b262605968/mon.smithi162/config 2024-04-27T04:20:47.092 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:20:46 smithi162 ceph-mon[110641]: pgmap v232: 65 pgs: 65 active+clean; 208 MiB data, 851 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:20:47.092 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:20:46 smithi162 ceph-mon[110641]: from='client.? 172.21.15.162:0/1156817560' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-27T04:20:47.171 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:20:46 smithi181 ceph-mon[81134]: pgmap v232: 65 pgs: 65 active+clean; 208 MiB data, 851 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:20:47.171 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:20:46 smithi181 ceph-mon[81134]: from='client.? 172.21.15.162:0/1156817560' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-27T04:20:48.095 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:20:47 smithi162 ceph-mon[110641]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-27T04:20:48.171 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:20:47 smithi181 ceph-mon[81134]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-27T04:20:48.577 INFO:teuthology.orchestra.run.smithi162.stdout:wait for servicemap items w/ changing names to refresh 2024-04-27T04:20:49.095 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:20:48 smithi162 ceph-mon[110641]: pgmap v233: 65 pgs: 65 active+clean; 208 MiB data, 851 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:20:49.171 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:20:48 smithi181 ceph-mon[81134]: pgmap v233: 65 pgs: 65 active+clean; 208 MiB data, 851 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:20:49.298 DEBUG:teuthology.orchestra.run.smithi162:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 7f9750d6-044a-11ef-bc93-c7b262605968 -e sha1=4e213a58e787437ff508e6c890ccd5fd32dc96b5 -- bash -c 'sleep 60' 2024-04-27T04:20:49.554 INFO:teuthology.orchestra.run.smithi162.stderr:Inferring config /var/lib/ceph/7f9750d6-044a-11ef-bc93-c7b262605968/mon.smithi162/config 2024-04-27T04:20:50.990 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:20:50 smithi162 ceph-mon[110641]: pgmap v234: 65 pgs: 65 active+clean; 208 MiB data, 851 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-04-27T04:20:51.171 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:20:50 smithi181 ceph-mon[81134]: pgmap v234: 65 pgs: 65 active+clean; 208 MiB data, 851 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-04-27T04:20:53.136 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:20:52 smithi162 ceph-mon[110641]: pgmap v235: 65 pgs: 65 active+clean; 208 MiB data, 851 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:20:53.171 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:20:52 smithi181 ceph-mon[81134]: pgmap v235: 65 pgs: 65 active+clean; 208 MiB data, 851 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:20:55.138 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:20:54 smithi162 ceph-mon[110641]: pgmap v236: 65 pgs: 65 active+clean; 208 MiB data, 851 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-04-27T04:20:55.171 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:20:54 smithi181 ceph-mon[81134]: pgmap v236: 65 pgs: 65 active+clean; 208 MiB data, 851 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-04-27T04:20:57.136 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:20:56 smithi162 ceph-mon[110641]: pgmap v237: 65 pgs: 65 active+clean; 208 MiB data, 851 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:20:57.171 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:20:56 smithi181 ceph-mon[81134]: pgmap v237: 65 pgs: 65 active+clean; 208 MiB data, 851 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:20:58.136 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:20:57 smithi162 ceph-mon[110641]: pgmap v238: 65 pgs: 65 active+clean; 208 MiB data, 851 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:20:58.171 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:20:57 smithi181 ceph-mon[81134]: pgmap v238: 65 pgs: 65 active+clean; 208 MiB data, 851 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:21:01.114 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:21:00 smithi162 ceph-mon[110641]: pgmap v239: 65 pgs: 65 active+clean; 208 MiB data, 851 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-04-27T04:21:01.171 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:21:00 smithi181 ceph-mon[81134]: pgmap v239: 65 pgs: 65 active+clean; 208 MiB data, 851 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-04-27T04:21:03.136 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:21:02 smithi162 ceph-mon[110641]: pgmap v240: 65 pgs: 65 active+clean; 208 MiB data, 851 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:21:03.136 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:21:02 smithi162 ceph-mon[110641]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-27T04:21:03.171 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:21:02 smithi181 ceph-mon[81134]: pgmap v240: 65 pgs: 65 active+clean; 208 MiB data, 851 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:21:03.171 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:21:02 smithi181 ceph-mon[81134]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-27T04:21:05.136 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:21:04 smithi162 ceph-mon[110641]: pgmap v241: 65 pgs: 65 active+clean; 208 MiB data, 851 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-04-27T04:21:05.174 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:21:04 smithi181 ceph-mon[81134]: pgmap v241: 65 pgs: 65 active+clean; 208 MiB data, 851 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-04-27T04:21:07.136 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:21:06 smithi162 ceph-mon[110641]: pgmap v242: 65 pgs: 65 active+clean; 208 MiB data, 851 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:21:07.171 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:21:06 smithi181 ceph-mon[81134]: pgmap v242: 65 pgs: 65 active+clean; 208 MiB data, 851 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:21:09.136 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:21:08 smithi162 ceph-mon[110641]: pgmap v243: 65 pgs: 65 active+clean; 208 MiB data, 851 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:21:09.171 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:21:08 smithi181 ceph-mon[81134]: pgmap v243: 65 pgs: 65 active+clean; 208 MiB data, 851 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:21:10.136 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:21:09 smithi162 ceph-mon[110641]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T04:21:10.136 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:21:09 smithi162 ceph-mon[110641]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T04:21:10.136 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:21:09 smithi162 ceph-mon[110641]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T04:21:10.136 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:21:09 smithi162 ceph-mon[110641]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' 2024-04-27T04:21:10.171 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:21:09 smithi181 ceph-mon[81134]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T04:21:10.171 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:21:09 smithi181 ceph-mon[81134]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T04:21:10.171 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:21:09 smithi181 ceph-mon[81134]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T04:21:10.171 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:21:09 smithi181 ceph-mon[81134]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' 2024-04-27T04:21:11.136 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:21:10 smithi162 ceph-mon[110641]: pgmap v244: 65 pgs: 65 active+clean; 208 MiB data, 851 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-04-27T04:21:11.171 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:21:10 smithi181 ceph-mon[81134]: pgmap v244: 65 pgs: 65 active+clean; 208 MiB data, 851 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-04-27T04:21:13.138 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:21:12 smithi162 ceph-mon[110641]: pgmap v245: 65 pgs: 65 active+clean; 208 MiB data, 851 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:21:13.171 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:21:12 smithi181 ceph-mon[81134]: pgmap v245: 65 pgs: 65 active+clean; 208 MiB data, 851 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:21:15.136 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:21:14 smithi162 ceph-mon[110641]: pgmap v246: 65 pgs: 65 active+clean; 208 MiB data, 851 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-04-27T04:21:15.171 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:21:14 smithi181 ceph-mon[81134]: pgmap v246: 65 pgs: 65 active+clean; 208 MiB data, 851 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-04-27T04:21:17.136 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:21:16 smithi162 ceph-mon[110641]: pgmap v247: 65 pgs: 65 active+clean; 208 MiB data, 851 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:21:17.171 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:21:16 smithi181 ceph-mon[81134]: pgmap v247: 65 pgs: 65 active+clean; 208 MiB data, 851 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:21:18.136 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:21:17 smithi162 ceph-mon[110641]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-27T04:21:18.171 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:21:17 smithi181 ceph-mon[81134]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-27T04:21:19.136 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:21:18 smithi162 ceph-mon[110641]: pgmap v248: 65 pgs: 65 active+clean; 208 MiB data, 851 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:21:19.171 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:21:18 smithi181 ceph-mon[81134]: pgmap v248: 65 pgs: 65 active+clean; 208 MiB data, 851 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:21:21.136 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:21:20 smithi162 ceph-mon[110641]: pgmap v249: 65 pgs: 65 active+clean; 208 MiB data, 851 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-04-27T04:21:21.171 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:21:20 smithi181 ceph-mon[81134]: pgmap v249: 65 pgs: 65 active+clean; 208 MiB data, 851 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-04-27T04:21:23.136 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:21:22 smithi162 ceph-mon[110641]: pgmap v250: 65 pgs: 65 active+clean; 208 MiB data, 851 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:21:23.171 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:21:22 smithi181 ceph-mon[81134]: pgmap v250: 65 pgs: 65 active+clean; 208 MiB data, 851 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:21:25.136 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:21:24 smithi162 ceph-mon[110641]: pgmap v251: 65 pgs: 65 active+clean; 208 MiB data, 851 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-04-27T04:21:25.171 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:21:24 smithi181 ceph-mon[81134]: pgmap v251: 65 pgs: 65 active+clean; 208 MiB data, 851 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-04-27T04:21:27.136 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:21:26 smithi162 ceph-mon[110641]: pgmap v252: 65 pgs: 65 active+clean; 208 MiB data, 851 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:21:27.171 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:21:26 smithi181 ceph-mon[81134]: pgmap v252: 65 pgs: 65 active+clean; 208 MiB data, 851 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:21:29.136 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:21:28 smithi162 ceph-mon[110641]: pgmap v253: 65 pgs: 65 active+clean; 208 MiB data, 851 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:21:29.171 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:21:28 smithi181 ceph-mon[81134]: pgmap v253: 65 pgs: 65 active+clean; 208 MiB data, 851 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:21:31.137 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:21:30 smithi162 ceph-mon[110641]: pgmap v254: 65 pgs: 65 active+clean; 208 MiB data, 851 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-04-27T04:21:31.171 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:21:30 smithi181 ceph-mon[81134]: pgmap v254: 65 pgs: 65 active+clean; 208 MiB data, 851 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-04-27T04:21:33.136 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:21:32 smithi162 ceph-mon[110641]: pgmap v255: 65 pgs: 65 active+clean; 208 MiB data, 851 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:21:33.136 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:21:32 smithi162 ceph-mon[110641]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-27T04:21:33.171 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:21:32 smithi181 ceph-mon[81134]: pgmap v255: 65 pgs: 65 active+clean; 208 MiB data, 851 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:21:33.172 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:21:32 smithi181 ceph-mon[81134]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-27T04:21:35.136 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:21:34 smithi162 ceph-mon[110641]: pgmap v256: 65 pgs: 65 active+clean; 208 MiB data, 851 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-04-27T04:21:35.171 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:21:34 smithi181 ceph-mon[81134]: pgmap v256: 65 pgs: 65 active+clean; 208 MiB data, 851 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-04-27T04:21:37.136 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:21:36 smithi162 ceph-mon[110641]: pgmap v257: 65 pgs: 65 active+clean; 208 MiB data, 851 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:21:37.171 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:21:36 smithi181 ceph-mon[81134]: pgmap v257: 65 pgs: 65 active+clean; 208 MiB data, 851 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:21:39.136 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:21:38 smithi162 ceph-mon[110641]: pgmap v258: 65 pgs: 65 active+clean; 208 MiB data, 851 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:21:39.171 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:21:38 smithi181 ceph-mon[81134]: pgmap v258: 65 pgs: 65 active+clean; 208 MiB data, 851 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:21:41.136 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:21:40 smithi162 ceph-mon[110641]: pgmap v259: 65 pgs: 65 active+clean; 208 MiB data, 851 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-04-27T04:21:41.171 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:21:40 smithi181 ceph-mon[81134]: pgmap v259: 65 pgs: 65 active+clean; 208 MiB data, 851 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-04-27T04:21:43.136 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:21:42 smithi162 ceph-mon[110641]: pgmap v260: 65 pgs: 65 active+clean; 208 MiB data, 851 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:21:43.171 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:21:42 smithi181 ceph-mon[81134]: pgmap v260: 65 pgs: 65 active+clean; 208 MiB data, 851 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:21:44.136 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:21:43 smithi162 ceph-mon[110641]: pgmap v261: 65 pgs: 65 active+clean; 208 MiB data, 851 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-04-27T04:21:44.171 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:21:43 smithi181 ceph-mon[81134]: pgmap v261: 65 pgs: 65 active+clean; 208 MiB data, 851 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-04-27T04:21:47.136 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:21:46 smithi162 ceph-mon[110641]: pgmap v262: 65 pgs: 65 active+clean; 208 MiB data, 851 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:21:47.171 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:21:46 smithi181 ceph-mon[81134]: pgmap v262: 65 pgs: 65 active+clean; 208 MiB data, 851 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:21:48.136 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:21:47 smithi162 ceph-mon[110641]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-27T04:21:48.171 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:21:47 smithi181 ceph-mon[81134]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-27T04:21:49.136 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:21:48 smithi162 ceph-mon[110641]: pgmap v263: 65 pgs: 65 active+clean; 208 MiB data, 851 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:21:49.171 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:21:48 smithi181 ceph-mon[81134]: pgmap v263: 65 pgs: 65 active+clean; 208 MiB data, 851 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:21:51.118 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:21:50 smithi162 ceph-mon[110641]: pgmap v264: 65 pgs: 65 active+clean; 208 MiB data, 851 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-04-27T04:21:51.171 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:21:50 smithi181 ceph-mon[81134]: pgmap v264: 65 pgs: 65 active+clean; 208 MiB data, 851 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-04-27T04:21:51.854 DEBUG:teuthology.orchestra.run.smithi162:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 7f9750d6-044a-11ef-bc93-c7b262605968 -e sha1=4e213a58e787437ff508e6c890ccd5fd32dc96b5 -- bash -c 'ceph orch ps' 2024-04-27T04:21:52.116 INFO:teuthology.orchestra.run.smithi162.stderr:Inferring config /var/lib/ceph/7f9750d6-044a-11ef-bc93-c7b262605968/mon.smithi162/config 2024-04-27T04:21:53.116 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:21:52 smithi162 ceph-mon[110641]: pgmap v265: 65 pgs: 65 active+clean; 208 MiB data, 851 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:21:53.171 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:21:52 smithi181 ceph-mon[81134]: pgmap v265: 65 pgs: 65 active+clean; 208 MiB data, 851 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:21:53.628 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:21:53.628+0000 7ff6d04fd700 1 -- 172.21.15.162:0/2135356505 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff6c81037c0 msgr2=0x7ff6c80fb3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:21:53.629 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:21:53.628+0000 7ff6d04fd700 1 --2- 172.21.15.162:0/2135356505 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff6c81037c0 0x7ff6c80fb3c0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7ff6b8004940 tx=0x7ff6b8009e30 comp rx=0 tx=0).stop 2024-04-27T04:21:53.629 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:21:53.628+0000 7ff6d04fd700 1 -- 172.21.15.162:0/2135356505 shutdown_connections 2024-04-27T04:21:53.629 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:21:53.628+0000 7ff6d04fd700 1 --2- 172.21.15.162:0/2135356505 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff6c81037c0 0x7ff6c80fb3c0 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:21:53.629 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:21:53.628+0000 7ff6d04fd700 1 --2- 172.21.15.162:0/2135356505 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff6c8102e20 0x7ff6c81031f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:21:53.629 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:21:53.628+0000 7ff6d04fd700 1 -- 172.21.15.162:0/2135356505 >> 172.21.15.162:0/2135356505 conn(0x7ff6c8072380 msgr2=0x7ff6c8072780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:21:53.629 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:21:53.628+0000 7ff6d04fd700 1 -- 172.21.15.162:0/2135356505 shutdown_connections 2024-04-27T04:21:53.629 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:21:53.628+0000 7ff6d04fd700 1 -- 172.21.15.162:0/2135356505 wait complete. 2024-04-27T04:21:53.629 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:21:53.628+0000 7ff6d04fd700 1 Processor -- start 2024-04-27T04:21:53.629 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:21:53.628+0000 7ff6d04fd700 1 -- start start 2024-04-27T04:21:53.630 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:21:53.629+0000 7ff6d04fd700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff6c8102e20 0x7ff6c81139a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:21:53.630 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:21:53.629+0000 7ff6d04fd700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff6c81037c0 0x7ff6c8113ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:21:53.630 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:21:53.629+0000 7ff6d04fd700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6c8114530 con 0x7ff6c8102e20 2024-04-27T04:21:53.630 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:21:53.629+0000 7ff6d04fd700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6c810db50 con 0x7ff6c81037c0 2024-04-27T04:21:53.630 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:21:53.629+0000 7ff6ce299700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff6c8102e20 0x7ff6c81139a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:21:53.630 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:21:53.629+0000 7ff6ce299700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff6c8102e20 0x7ff6c81139a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:48224/0 (socket says 172.21.15.162:48224) 2024-04-27T04:21:53.630 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:21:53.629+0000 7ff6ce299700 1 -- 172.21.15.162:0/1347986757 learned_addr learned my addr 172.21.15.162:0/1347986757 (peer_addr_for_me v2:172.21.15.162:0/0) 2024-04-27T04:21:53.630 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:21:53.629+0000 7ff6cda98700 1 --2- 172.21.15.162:0/1347986757 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff6c81037c0 0x7ff6c8113ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:21:53.630 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:21:53.629+0000 7ff6ce299700 1 -- 172.21.15.162:0/1347986757 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff6c81037c0 msgr2=0x7ff6c8113ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:21:53.630 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:21:53.629+0000 7ff6ce299700 1 --2- 172.21.15.162:0/1347986757 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff6c81037c0 0x7ff6c8113ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:21:53.630 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:21:53.629+0000 7ff6ce299700 1 -- 172.21.15.162:0/1347986757 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6b8005040 con 0x7ff6c8102e20 2024-04-27T04:21:53.631 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:21:53.629+0000 7ff6ce299700 1 --2- 172.21.15.162:0/1347986757 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff6c8102e20 0x7ff6c81139a0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7ff6c4002a80 tx=0x7ff6c4004600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:21:53.631 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:21:53.629+0000 7ff6beffd700 1 -- 172.21.15.162:0/1347986757 <== mon.0 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff6c4007cc0 con 0x7ff6c8102e20 2024-04-27T04:21:53.631 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:21:53.629+0000 7ff6beffd700 1 -- 172.21.15.162:0/1347986757 <== mon.0 v2:172.21.15.162:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7ff6c400f040 con 0x7ff6c8102e20 2024-04-27T04:21:53.631 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:21:53.629+0000 7ff6d04fd700 1 -- 172.21.15.162:0/1347986757 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff6c810de30 con 0x7ff6c8102e20 2024-04-27T04:21:53.631 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:21:53.629+0000 7ff6beffd700 1 -- 172.21.15.162:0/1347986757 <== mon.0 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff6c400da30 con 0x7ff6c8102e20 2024-04-27T04:21:53.631 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:21:53.630+0000 7ff6d04fd700 1 -- 172.21.15.162:0/1347986757 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff6c810e340 con 0x7ff6c8102e20 2024-04-27T04:21:53.632 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:21:53.630+0000 7ff6d04fd700 1 -- 172.21.15.162:0/1347986757 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff6c80fe6e0 con 0x7ff6c8102e20 2024-04-27T04:21:53.632 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:21:53.631+0000 7ff6beffd700 1 -- 172.21.15.162:0/1347986757 <== mon.0 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 39) v1 ==== 92747+0+0 (secure 0 0 0) 0x7ff6c400db90 con 0x7ff6c8102e20 2024-04-27T04:21:53.632 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:21:53.631+0000 7ff6beffd700 1 --2- 172.21.15.162:0/1347986757 >> [v2:172.21.15.162:6800/1396866108,v1:172.21.15.162:6801/1396866108] conn(0x7ff6b406ec00 0x7ff6b40710b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:21:53.632 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:21:53.631+0000 7ff6beffd700 1 -- 172.21.15.162:0/1347986757 <== mon.0 v2:172.21.15.162:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6582+0+0 (secure 0 0 0) 0x7ff6c408e660 con 0x7ff6c8102e20 2024-04-27T04:21:53.632 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:21:53.631+0000 7ff6cda98700 1 --2- 172.21.15.162:0/1347986757 >> [v2:172.21.15.162:6800/1396866108,v1:172.21.15.162:6801/1396866108] conn(0x7ff6b406ec00 0x7ff6b40710b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:21:53.632 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:21:53.632+0000 7ff6cda98700 1 --2- 172.21.15.162:0/1347986757 >> [v2:172.21.15.162:6800/1396866108,v1:172.21.15.162:6801/1396866108] conn(0x7ff6b406ec00 0x7ff6b40710b0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7ff6b8004890 tx=0x7ff6b8009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:21:53.634 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:21:53.636+0000 7ff6beffd700 1 -- 172.21.15.162:0/1347986757 <== mon.0 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7ff6c40573b0 con 0x7ff6c8102e20 2024-04-27T04:21:53.853 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:21:53.854+0000 7ff6d04fd700 1 -- 172.21.15.162:0/1347986757 --> [v2:172.21.15.162:6800/1396866108,v1:172.21.15.162:6801/1396866108] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7ff6c810f060 con 0x7ff6b406ec00 2024-04-27T04:21:53.863 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:21:53.864+0000 7ff6beffd700 1 -- 172.21.15.162:0/1347986757 <== mgr.34104 v2:172.21.15.162:6800/1396866108 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3520 (secure 0 0 0) 0x7ff6c810f060 con 0x7ff6b406ec00 2024-04-27T04:21:53.863 INFO:teuthology.orchestra.run.smithi162.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-27T04:21:53.863 INFO:teuthology.orchestra.run.smithi162.stdout:alertmanager.smithi162 smithi162 *:9093,9094 running (9m) 2m ago 20m 27.0M - 0.25.0 c8568f914cd2 40e334659017 2024-04-27T04:21:53.863 INFO:teuthology.orchestra.run.smithi162.stdout:crash.smithi162 smithi162 running (6m) 2m ago 20m 6660k - 19.0.0-3253-g4e213a58 67e414ff4bc4 1aa8f4d5a28b 2024-04-27T04:21:53.863 INFO:teuthology.orchestra.run.smithi162.stdout:crash.smithi181 smithi181 running (6m) 112s ago 19m 6656k - 19.0.0-3253-g4e213a58 67e414ff4bc4 832d9cbc5c30 2024-04-27T04:21:53.863 INFO:teuthology.orchestra.run.smithi162.stdout:grafana.smithi162 smithi162 *:3000 running (8m) 2m ago 19m 79.3M - 9.4.12 2bacad6d85d8 3d78c478f03f 2024-04-27T04:21:53.864 INFO:teuthology.orchestra.run.smithi162.stdout:mds.cephfs.smithi162.gzlyjj smithi162 running (3m) 2m ago 14m 12.5M - 19.0.0-3253-g4e213a58 67e414ff4bc4 7541e8fb8265 2024-04-27T04:21:53.864 INFO:teuthology.orchestra.run.smithi162.stdout:mds.cephfs.smithi162.xcnapk smithi162 running (2m) 2m ago 14m 11.5M - 19.0.0-3253-g4e213a58 67e414ff4bc4 3af4597febc8 2024-04-27T04:21:53.864 INFO:teuthology.orchestra.run.smithi162.stdout:mds.cephfs.smithi181.bkkcdm smithi181 running (114s) 112s ago 14m 14.0M - 19.0.0-3253-g4e213a58 67e414ff4bc4 87e3901b7571 2024-04-27T04:21:53.864 INFO:teuthology.orchestra.run.smithi162.stdout:mds.cephfs.smithi181.dzomgh smithi181 running (2m) 112s ago 14m 12.7M - 19.0.0-3253-g4e213a58 67e414ff4bc4 a07f9164b260 2024-04-27T04:21:53.864 INFO:teuthology.orchestra.run.smithi162.stdout:mgr.smithi162.oennfn smithi162 *:8443,9283,8765 running (10m) 2m ago 21m 511M - 19.0.0-3253-g4e213a58 67e414ff4bc4 58a748d3637b 2024-04-27T04:21:53.864 INFO:teuthology.orchestra.run.smithi162.stdout:mgr.smithi181.vtwuhz smithi181 *:8443,9283,8765 running (9m) 112s ago 19m 452M - 19.0.0-3253-g4e213a58 67e414ff4bc4 813584297b1d 2024-04-27T04:21:53.864 INFO:teuthology.orchestra.run.smithi162.stdout:mon.smithi162 smithi162 running (7m) 2m ago 21m 51.9M 2048M 19.0.0-3253-g4e213a58 67e414ff4bc4 375adaf4468f 2024-04-27T04:21:53.864 INFO:teuthology.orchestra.run.smithi162.stdout:mon.smithi181 smithi181 running (7m) 112s ago 19m 41.7M 2048M 19.0.0-3253-g4e213a58 67e414ff4bc4 77b71f6d522f 2024-04-27T04:21:53.864 INFO:teuthology.orchestra.run.smithi162.stdout:node-exporter.smithi162 smithi162 *:9100 running (9m) 2m ago 20m 15.5M - 1.5.0 0da6a335fe13 5c4c5ea9c1d7 2024-04-27T04:21:53.864 INFO:teuthology.orchestra.run.smithi162.stdout:node-exporter.smithi181 smithi181 *:9100 running (9m) 112s ago 19m 15.3M - 1.5.0 0da6a335fe13 9067f14c0519 2024-04-27T04:21:53.865 INFO:teuthology.orchestra.run.smithi162.stdout:osd.0 smithi162 running (6m) 2m ago 18m 93.1M 1557M 19.0.0-3253-g4e213a58 67e414ff4bc4 2744aff3cd88 2024-04-27T04:21:53.865 INFO:teuthology.orchestra.run.smithi162.stdout:osd.1 smithi162 running (5m) 2m ago 18m 96.6M 1557M 19.0.0-3253-g4e213a58 67e414ff4bc4 668e16faad62 2024-04-27T04:21:53.865 INFO:teuthology.orchestra.run.smithi162.stdout:osd.2 smithi162 running (5m) 2m ago 17m 87.4M 1557M 19.0.0-3253-g4e213a58 67e414ff4bc4 d24878d461e7 2024-04-27T04:21:53.865 INFO:teuthology.orchestra.run.smithi162.stdout:osd.3 smithi181 running (4m) 112s ago 17m 122M 2581M 19.0.0-3253-g4e213a58 67e414ff4bc4 ab473cfd54d2 2024-04-27T04:21:53.865 INFO:teuthology.orchestra.run.smithi162.stdout:osd.4 smithi181 running (3m) 112s ago 16m 101M 2581M 19.0.0-3253-g4e213a58 67e414ff4bc4 de3e10f2d26d 2024-04-27T04:21:53.865 INFO:teuthology.orchestra.run.smithi162.stdout:osd.5 smithi181 running (3m) 112s ago 16m 103M 2581M 19.0.0-3253-g4e213a58 67e414ff4bc4 7dd4f287ef2e 2024-04-27T04:21:53.865 INFO:teuthology.orchestra.run.smithi162.stdout:prometheus.smithi162 smithi162 *:9095 running (9m) 2m ago 19m 67.7M - 2.43.0 a07b618ecd1d f30657a43324 2024-04-27T04:21:53.866 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:21:53.866+0000 7ff6d04fd700 1 -- 172.21.15.162:0/1347986757 >> [v2:172.21.15.162:6800/1396866108,v1:172.21.15.162:6801/1396866108] conn(0x7ff6b406ec00 msgr2=0x7ff6b40710b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:21:53.866 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:21:53.866+0000 7ff6d04fd700 1 --2- 172.21.15.162:0/1347986757 >> [v2:172.21.15.162:6800/1396866108,v1:172.21.15.162:6801/1396866108] conn(0x7ff6b406ec00 0x7ff6b40710b0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7ff6b8004890 tx=0x7ff6b8009e30 comp rx=0 tx=0).stop 2024-04-27T04:21:53.866 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:21:53.866+0000 7ff6d04fd700 1 -- 172.21.15.162:0/1347986757 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff6c8102e20 msgr2=0x7ff6c81139a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:21:53.866 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:21:53.866+0000 7ff6d04fd700 1 --2- 172.21.15.162:0/1347986757 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff6c8102e20 0x7ff6c81139a0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7ff6c4002a80 tx=0x7ff6c4004600 comp rx=0 tx=0).stop 2024-04-27T04:21:53.866 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:21:53.866+0000 7ff6d04fd700 1 -- 172.21.15.162:0/1347986757 shutdown_connections 2024-04-27T04:21:53.866 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:21:53.866+0000 7ff6d04fd700 1 --2- 172.21.15.162:0/1347986757 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff6c8102e20 0x7ff6c81139a0 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:21:53.867 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:21:53.866+0000 7ff6d04fd700 1 --2- 172.21.15.162:0/1347986757 >> [v2:172.21.15.162:6800/1396866108,v1:172.21.15.162:6801/1396866108] conn(0x7ff6b406ec00 0x7ff6b40710b0 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:21:53.867 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:21:53.866+0000 7ff6d04fd700 1 --2- 172.21.15.162:0/1347986757 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff6c81037c0 0x7ff6c8113ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:21:53.867 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:21:53.866+0000 7ff6d04fd700 1 -- 172.21.15.162:0/1347986757 >> 172.21.15.162:0/1347986757 conn(0x7ff6c8072380 msgr2=0x7ff6c80facb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:21:53.867 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:21:53.867+0000 7ff6d04fd700 1 -- 172.21.15.162:0/1347986757 shutdown_connections 2024-04-27T04:21:53.867 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:21:53.867+0000 7ff6d04fd700 1 -- 172.21.15.162:0/1347986757 wait complete. 2024-04-27T04:21:54.527 DEBUG:teuthology.orchestra.run.smithi162:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 7f9750d6-044a-11ef-bc93-c7b262605968 -e sha1=4e213a58e787437ff508e6c890ccd5fd32dc96b5 -- bash -c 'ceph versions' 2024-04-27T04:21:54.798 INFO:teuthology.orchestra.run.smithi162.stderr:Inferring config /var/lib/ceph/7f9750d6-044a-11ef-bc93-c7b262605968/mon.smithi162/config 2024-04-27T04:21:55.086 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:21:54 smithi162 ceph-mon[110641]: pgmap v266: 65 pgs: 65 active+clean; 208 MiB data, 851 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-04-27T04:21:55.171 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:21:54 smithi181 ceph-mon[81134]: pgmap v266: 65 pgs: 65 active+clean; 208 MiB data, 851 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-04-27T04:21:56.135 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:21:55 smithi162 ceph-mon[110641]: from='client.34496 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-27T04:21:56.171 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:21:55 smithi181 ceph-mon[81134]: from='client.34496 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-27T04:21:56.494 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:21:56.484+0000 7f4ab0923700 1 -- 172.21.15.162:0/3378496131 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4aac102de0 msgr2=0x7f4aac1031b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:21:56.494 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:21:56.484+0000 7f4ab0923700 1 --2- 172.21.15.162:0/3378496131 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4aac102de0 0x7f4aac1031b0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f4aa0004940 tx=0x7f4aa0009e30 comp rx=0 tx=0).stop 2024-04-27T04:21:56.494 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:21:56.485+0000 7f4ab0923700 1 -- 172.21.15.162:0/3378496131 shutdown_connections 2024-04-27T04:21:56.494 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:21:56.485+0000 7f4ab0923700 1 --2- 172.21.15.162:0/3378496131 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4aac103780 0x7f4aac0fcdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:21:56.494 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:21:56.485+0000 7f4ab0923700 1 --2- 172.21.15.162:0/3378496131 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4aac102de0 0x7f4aac1031b0 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:21:56.494 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:21:56.485+0000 7f4ab0923700 1 -- 172.21.15.162:0/3378496131 >> 172.21.15.162:0/3378496131 conn(0x7f4aac0f8990 msgr2=0x7f4aac0fada0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:21:56.494 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:21:56.485+0000 7f4ab0923700 1 -- 172.21.15.162:0/3378496131 shutdown_connections 2024-04-27T04:21:56.494 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:21:56.485+0000 7f4ab0923700 1 -- 172.21.15.162:0/3378496131 wait complete. 2024-04-27T04:21:56.494 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:21:56.485+0000 7f4ab0923700 1 Processor -- start 2024-04-27T04:21:56.494 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:21:56.485+0000 7f4ab0923700 1 -- start start 2024-04-27T04:21:56.494 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:21:56.485+0000 7f4ab0923700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4aac102de0 0x7f4aac113970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:21:56.494 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:21:56.485+0000 7f4ab0923700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4aac103780 0x7f4aac113eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:21:56.495 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:21:56.485+0000 7f4ab0923700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4aac114500 con 0x7f4aac102de0 2024-04-27T04:21:56.495 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:21:56.485+0000 7f4ab0923700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4aac10db50 con 0x7f4aac103780 2024-04-27T04:21:56.495 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:21:56.485+0000 7f4aaa59c700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4aac102de0 0x7f4aac113970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:21:56.495 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:21:56.486+0000 7f4aaa59c700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4aac102de0 0x7f4aac113970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:48244/0 (socket says 172.21.15.162:48244) 2024-04-27T04:21:56.495 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:21:56.486+0000 7f4aaa59c700 1 -- 172.21.15.162:0/1141740434 learned_addr learned my addr 172.21.15.162:0/1141740434 (peer_addr_for_me v2:172.21.15.162:0/0) 2024-04-27T04:21:56.495 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:21:56.486+0000 7f4aa9d9b700 1 --2- 172.21.15.162:0/1141740434 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4aac103780 0x7f4aac113eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:21:56.495 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:21:56.486+0000 7f4aaa59c700 1 -- 172.21.15.162:0/1141740434 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4aac103780 msgr2=0x7f4aac113eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:21:56.495 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:21:56.486+0000 7f4aaa59c700 1 --2- 172.21.15.162:0/1141740434 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4aac103780 0x7f4aac113eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:21:56.495 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:21:56.486+0000 7f4aaa59c700 1 -- 172.21.15.162:0/1141740434 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4aa0005040 con 0x7f4aac102de0 2024-04-27T04:21:56.495 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:21:56.486+0000 7f4aa9d9b700 1 --2- 172.21.15.162:0/1141740434 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4aac103780 0x7f4aac113eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-27T04:21:56.495 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:21:56.486+0000 7f4aaa59c700 1 --2- 172.21.15.162:0/1141740434 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4aac102de0 0x7f4aac113970 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f4aa0009100 tx=0x7f4aa0009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:21:56.495 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:21:56.486+0000 7f4a9affd700 1 -- 172.21.15.162:0/1141740434 <== mon.0 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4aa000e040 con 0x7f4aac102de0 2024-04-27T04:21:56.495 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:21:56.486+0000 7f4ab0923700 1 -- 172.21.15.162:0/1141740434 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4aac10ddd0 con 0x7f4aac102de0 2024-04-27T04:21:56.495 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:21:56.486+0000 7f4a9affd700 1 -- 172.21.15.162:0/1141740434 <== mon.0 v2:172.21.15.162:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f4aa0013420 con 0x7f4aac102de0 2024-04-27T04:21:56.496 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:21:56.486+0000 7f4a9affd700 1 -- 172.21.15.162:0/1141740434 <== mon.0 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4aa001c630 con 0x7f4aac102de0 2024-04-27T04:21:56.496 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:21:56.486+0000 7f4ab0923700 1 -- 172.21.15.162:0/1141740434 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4aac10e390 con 0x7f4aac102de0 2024-04-27T04:21:56.496 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:21:56.487+0000 7f4ab0923700 1 -- 172.21.15.162:0/1141740434 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4aac1000f0 con 0x7f4aac102de0 2024-04-27T04:21:56.496 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:21:56.488+0000 7f4a9affd700 1 -- 172.21.15.162:0/1141740434 <== mon.0 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 39) v1 ==== 92747+0+0 (secure 0 0 0) 0x7f4aa0013590 con 0x7f4aac102de0 2024-04-27T04:21:56.496 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:21:56.488+0000 7f4a9affd700 1 --2- 172.21.15.162:0/1141740434 >> [v2:172.21.15.162:6800/1396866108,v1:172.21.15.162:6801/1396866108] conn(0x7f4a9406ec00 0x7f4a940710b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:21:56.496 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:21:56.488+0000 7f4a9affd700 1 -- 172.21.15.162:0/1141740434 <== mon.0 v2:172.21.15.162:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6582+0+0 (secure 0 0 0) 0x7f4aa008f3e0 con 0x7f4aac102de0 2024-04-27T04:21:56.496 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:21:56.488+0000 7f4aa9d9b700 1 --2- 172.21.15.162:0/1141740434 >> [v2:172.21.15.162:6800/1396866108,v1:172.21.15.162:6801/1396866108] conn(0x7f4a9406ec00 0x7f4a940710b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:21:56.496 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:21:56.488+0000 7f4aa9d9b700 1 --2- 172.21.15.162:0/1141740434 >> [v2:172.21.15.162:6800/1396866108,v1:172.21.15.162:6801/1396866108] conn(0x7f4a9406ec00 0x7f4a940710b0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f4aac10ecc0 tx=0x7f4a9c009040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:21:56.496 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:21:56.493+0000 7f4a9affd700 1 -- 172.21.15.162:0/1141740434 <== mon.0 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7f4aa005bce0 con 0x7f4aac102de0 2024-04-27T04:21:56.777 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:21:56.777+0000 7f4ab0923700 1 -- 172.21.15.162:0/1141740434 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f4aac10efc0 con 0x7f4aac102de0 2024-04-27T04:21:56.777 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:21:56.778+0000 7f4a9affd700 1 -- 172.21.15.162:0/1141740434 <== mon.0 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+623 (secure 0 0 0) 0x7f4aa005b870 con 0x7f4aac102de0 2024-04-27T04:21:56.777 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-04-27T04:21:56.778 INFO:teuthology.orchestra.run.smithi162.stdout: "mon": { 2024-04-27T04:21:56.778 INFO:teuthology.orchestra.run.smithi162.stdout: "ceph version 19.0.0-3253-g4e213a58 (4e213a58e787437ff508e6c890ccd5fd32dc96b5) squid (dev)": 2 2024-04-27T04:21:56.778 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-04-27T04:21:56.778 INFO:teuthology.orchestra.run.smithi162.stdout: "mgr": { 2024-04-27T04:21:56.778 INFO:teuthology.orchestra.run.smithi162.stdout: "ceph version 19.0.0-3253-g4e213a58 (4e213a58e787437ff508e6c890ccd5fd32dc96b5) squid (dev)": 2 2024-04-27T04:21:56.778 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-04-27T04:21:56.778 INFO:teuthology.orchestra.run.smithi162.stdout: "osd": { 2024-04-27T04:21:56.778 INFO:teuthology.orchestra.run.smithi162.stdout: "ceph version 19.0.0-3253-g4e213a58 (4e213a58e787437ff508e6c890ccd5fd32dc96b5) squid (dev)": 6 2024-04-27T04:21:56.779 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-04-27T04:21:56.779 INFO:teuthology.orchestra.run.smithi162.stdout: "mds": { 2024-04-27T04:21:56.779 INFO:teuthology.orchestra.run.smithi162.stdout: "ceph version 19.0.0-3253-g4e213a58 (4e213a58e787437ff508e6c890ccd5fd32dc96b5) squid (dev)": 4 2024-04-27T04:21:56.779 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-04-27T04:21:56.779 INFO:teuthology.orchestra.run.smithi162.stdout: "overall": { 2024-04-27T04:21:56.779 INFO:teuthology.orchestra.run.smithi162.stdout: "ceph version 19.0.0-3253-g4e213a58 (4e213a58e787437ff508e6c890ccd5fd32dc96b5) squid (dev)": 14 2024-04-27T04:21:56.779 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-04-27T04:21:56.779 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-04-27T04:21:56.780 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:21:56.780+0000 7f4ab0923700 1 -- 172.21.15.162:0/1141740434 >> [v2:172.21.15.162:6800/1396866108,v1:172.21.15.162:6801/1396866108] conn(0x7f4a9406ec00 msgr2=0x7f4a940710b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:21:56.780 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:21:56.780+0000 7f4ab0923700 1 --2- 172.21.15.162:0/1141740434 >> [v2:172.21.15.162:6800/1396866108,v1:172.21.15.162:6801/1396866108] conn(0x7f4a9406ec00 0x7f4a940710b0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f4aac10ecc0 tx=0x7f4a9c009040 comp rx=0 tx=0).stop 2024-04-27T04:21:56.780 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:21:56.780+0000 7f4ab0923700 1 -- 172.21.15.162:0/1141740434 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4aac102de0 msgr2=0x7f4aac113970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:21:56.781 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:21:56.780+0000 7f4ab0923700 1 --2- 172.21.15.162:0/1141740434 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4aac102de0 0x7f4aac113970 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f4aa0009100 tx=0x7f4aa0009e30 comp rx=0 tx=0).stop 2024-04-27T04:21:56.781 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:21:56.780+0000 7f4ab0923700 1 -- 172.21.15.162:0/1141740434 shutdown_connections 2024-04-27T04:21:56.781 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:21:56.780+0000 7f4ab0923700 1 --2- 172.21.15.162:0/1141740434 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4aac102de0 0x7f4aac113970 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:21:56.781 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:21:56.780+0000 7f4ab0923700 1 --2- 172.21.15.162:0/1141740434 >> [v2:172.21.15.162:6800/1396866108,v1:172.21.15.162:6801/1396866108] conn(0x7f4a9406ec00 0x7f4a940710b0 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:21:56.781 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:21:56.780+0000 7f4ab0923700 1 --2- 172.21.15.162:0/1141740434 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4aac103780 0x7f4aac113eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:21:56.781 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:21:56.780+0000 7f4ab0923700 1 -- 172.21.15.162:0/1141740434 >> 172.21.15.162:0/1141740434 conn(0x7f4aac0f8990 msgr2=0x7f4aac107380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:21:56.781 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:21:56.780+0000 7f4ab0923700 1 -- 172.21.15.162:0/1141740434 shutdown_connections 2024-04-27T04:21:56.781 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:21:56.780+0000 7f4ab0923700 1 -- 172.21.15.162:0/1141740434 wait complete. 2024-04-27T04:21:57.131 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:21:56 smithi162 ceph-mon[110641]: pgmap v267: 65 pgs: 65 active+clean; 208 MiB data, 851 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-04-27T04:21:57.131 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:21:56 smithi162 ceph-mon[110641]: from='client.? 172.21.15.162:0/1141740434' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-27T04:21:57.171 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:21:56 smithi181 ceph-mon[81134]: pgmap v267: 65 pgs: 65 active+clean; 208 MiB data, 851 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-04-27T04:21:57.171 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:21:56 smithi181 ceph-mon[81134]: from='client.? 172.21.15.162:0/1141740434' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-27T04:21:57.463 DEBUG:teuthology.orchestra.run.smithi162:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 7f9750d6-044a-11ef-bc93-c7b262605968 -e sha1=4e213a58e787437ff508e6c890ccd5fd32dc96b5 -- bash -c 'ceph versions | jq -e '"'"'.overall | length == 1'"'"'' 2024-04-27T04:21:57.718 INFO:teuthology.orchestra.run.smithi162.stderr:Inferring config /var/lib/ceph/7f9750d6-044a-11ef-bc93-c7b262605968/mon.smithi162/config 2024-04-27T04:21:59.001 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:21:58 smithi162 ceph-mon[110641]: pgmap v268: 65 pgs: 65 active+clean; 208 MiB data, 851 MiB used, 536 GiB / 536 GiB avail; 767 B/s rd, 1 op/s 2024-04-27T04:21:59.171 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:21:58 smithi181 ceph-mon[81134]: pgmap v268: 65 pgs: 65 active+clean; 208 MiB data, 851 MiB used, 536 GiB / 536 GiB avail; 767 B/s rd, 1 op/s 2024-04-27T04:21:59.202 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:21:59.201+0000 7f58cab23700 1 -- 172.21.15.162:0/4018081535 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f58c4103780 msgr2=0x7f58c40fcdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:21:59.202 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:21:59.201+0000 7f58cab23700 1 --2- 172.21.15.162:0/4018081535 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f58c4103780 0x7f58c40fcdf0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f58b0004940 tx=0x7f58b0009e30 comp rx=0 tx=0).stop 2024-04-27T04:21:59.202 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:21:59.201+0000 7f58cab23700 1 -- 172.21.15.162:0/4018081535 shutdown_connections 2024-04-27T04:21:59.202 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:21:59.201+0000 7f58cab23700 1 --2- 172.21.15.162:0/4018081535 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f58c4103780 0x7f58c40fcdf0 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:21:59.202 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:21:59.201+0000 7f58cab23700 1 --2- 172.21.15.162:0/4018081535 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f58c4102de0 0x7f58c41031b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:21:59.202 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:21:59.201+0000 7f58cab23700 1 -- 172.21.15.162:0/4018081535 >> 172.21.15.162:0/4018081535 conn(0x7f58c40f89b0 msgr2=0x7f58c40fadc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:21:59.202 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:21:59.201+0000 7f58cab23700 1 -- 172.21.15.162:0/4018081535 shutdown_connections 2024-04-27T04:21:59.202 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:21:59.201+0000 7f58cab23700 1 -- 172.21.15.162:0/4018081535 wait complete. 2024-04-27T04:21:59.202 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:21:59.201+0000 7f58cab23700 1 Processor -- start 2024-04-27T04:21:59.202 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:21:59.202+0000 7f58cab23700 1 -- start start 2024-04-27T04:21:59.203 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:21:59.202+0000 7f58cab23700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f58c4102de0 0x7f58c4070260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:21:59.203 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:21:59.202+0000 7f58cab23700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f58c4103780 0x7f58c40707a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:21:59.203 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:21:59.202+0000 7f58cab23700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58c4070d70 con 0x7f58c4102de0 2024-04-27T04:21:59.203 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:21:59.202+0000 7f58cab23700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58c4070eb0 con 0x7f58c4103780 2024-04-27T04:21:59.203 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:21:59.202+0000 7f58c88bf700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f58c4102de0 0x7f58c4070260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:21:59.203 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:21:59.202+0000 7f58c88bf700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f58c4102de0 0x7f58c4070260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:48256/0 (socket says 172.21.15.162:48256) 2024-04-27T04:21:59.203 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:21:59.202+0000 7f58c88bf700 1 -- 172.21.15.162:0/3324154666 learned_addr learned my addr 172.21.15.162:0/3324154666 (peer_addr_for_me v2:172.21.15.162:0/0) 2024-04-27T04:21:59.203 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:21:59.202+0000 7f58c3fff700 1 --2- 172.21.15.162:0/3324154666 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f58c4103780 0x7f58c40707a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:21:59.203 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:21:59.202+0000 7f58c88bf700 1 -- 172.21.15.162:0/3324154666 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f58c4103780 msgr2=0x7f58c40707a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:21:59.203 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:21:59.202+0000 7f58c88bf700 1 --2- 172.21.15.162:0/3324154666 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f58c4103780 0x7f58c40707a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:21:59.203 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:21:59.202+0000 7f58c88bf700 1 -- 172.21.15.162:0/3324154666 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58b0005040 con 0x7f58c4102de0 2024-04-27T04:21:59.203 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:21:59.202+0000 7f58c3fff700 1 --2- 172.21.15.162:0/3324154666 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f58c4103780 0x7f58c40707a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-27T04:21:59.203 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:21:59.202+0000 7f58c88bf700 1 --2- 172.21.15.162:0/3324154666 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f58c4102de0 0x7f58c4070260 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f58b800b790 tx=0x7f58b8009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:21:59.203 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:21:59.203+0000 7f58c17fa700 1 -- 172.21.15.162:0/3324154666 <== mon.0 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f58b800f770 con 0x7f58c4102de0 2024-04-27T04:21:59.204 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:21:59.203+0000 7f58c17fa700 1 -- 172.21.15.162:0/3324154666 <== mon.0 v2:172.21.15.162:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f58b800fda0 con 0x7f58c4102de0 2024-04-27T04:21:59.204 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:21:59.203+0000 7f58cab23700 1 -- 172.21.15.162:0/3324154666 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f58c4119ea0 con 0x7f58c4102de0 2024-04-27T04:21:59.204 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:21:59.203+0000 7f58c17fa700 1 -- 172.21.15.162:0/3324154666 <== mon.0 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f58b800d310 con 0x7f58c4102de0 2024-04-27T04:21:59.204 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:21:59.203+0000 7f58cab23700 1 -- 172.21.15.162:0/3324154666 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f58c411a380 con 0x7f58c4102de0 2024-04-27T04:21:59.205 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:21:59.204+0000 7f58cab23700 1 -- 172.21.15.162:0/3324154666 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f58c41000f0 con 0x7f58c4102de0 2024-04-27T04:21:59.205 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:21:59.204+0000 7f58c17fa700 1 -- 172.21.15.162:0/3324154666 <== mon.0 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 39) v1 ==== 92747+0+0 (secure 0 0 0) 0x7f58b8010420 con 0x7f58c4102de0 2024-04-27T04:21:59.205 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:21:59.204+0000 7f58c17fa700 1 --2- 172.21.15.162:0/3324154666 >> [v2:172.21.15.162:6800/1396866108,v1:172.21.15.162:6801/1396866108] conn(0x7f58b406ec00 0x7f58b40710b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:21:59.205 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:21:59.205+0000 7f58c17fa700 1 -- 172.21.15.162:0/3324154666 <== mon.0 v2:172.21.15.162:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6582+0+0 (secure 0 0 0) 0x7f58b808e930 con 0x7f58c4102de0 2024-04-27T04:21:59.205 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:21:59.205+0000 7f58c3fff700 1 --2- 172.21.15.162:0/3324154666 >> [v2:172.21.15.162:6800/1396866108,v1:172.21.15.162:6801/1396866108] conn(0x7f58b406ec00 0x7f58b40710b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:21:59.205 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:21:59.205+0000 7f58c3fff700 1 --2- 172.21.15.162:0/3324154666 >> [v2:172.21.15.162:6800/1396866108,v1:172.21.15.162:6801/1396866108] conn(0x7f58b406ec00 0x7f58b40710b0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f58b000b3b0 tx=0x7f58b0009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:21:59.208 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:21:59.209+0000 7f58c17fa700 1 -- 172.21.15.162:0/3324154666 <== mon.0 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7f58b8057700 con 0x7f58c4102de0 2024-04-27T04:21:59.496 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:21:59.498+0000 7f58cab23700 1 -- 172.21.15.162:0/3324154666 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f58c404bed0 con 0x7f58c4102de0 2024-04-27T04:21:59.496 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:21:59.498+0000 7f58c17fa700 1 -- 172.21.15.162:0/3324154666 <== mon.0 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+623 (secure 0 0 0) 0x7f58b805adc0 con 0x7f58c4102de0 2024-04-27T04:21:59.499 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:21:59.500+0000 7f58cab23700 1 -- 172.21.15.162:0/3324154666 >> [v2:172.21.15.162:6800/1396866108,v1:172.21.15.162:6801/1396866108] conn(0x7f58b406ec00 msgr2=0x7f58b40710b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:21:59.499 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:21:59.500+0000 7f58cab23700 1 --2- 172.21.15.162:0/3324154666 >> [v2:172.21.15.162:6800/1396866108,v1:172.21.15.162:6801/1396866108] conn(0x7f58b406ec00 0x7f58b40710b0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f58b000b3b0 tx=0x7f58b0009e30 comp rx=0 tx=0).stop 2024-04-27T04:21:59.499 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:21:59.500+0000 7f58cab23700 1 -- 172.21.15.162:0/3324154666 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f58c4102de0 msgr2=0x7f58c4070260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:21:59.499 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:21:59.500+0000 7f58cab23700 1 --2- 172.21.15.162:0/3324154666 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f58c4102de0 0x7f58c4070260 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f58b800b790 tx=0x7f58b8009250 comp rx=0 tx=0).stop 2024-04-27T04:21:59.499 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:21:59.500+0000 7f58cab23700 1 -- 172.21.15.162:0/3324154666 shutdown_connections 2024-04-27T04:21:59.499 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:21:59.500+0000 7f58cab23700 1 --2- 172.21.15.162:0/3324154666 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f58c4102de0 0x7f58c4070260 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:21:59.499 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:21:59.500+0000 7f58cab23700 1 --2- 172.21.15.162:0/3324154666 >> [v2:172.21.15.162:6800/1396866108,v1:172.21.15.162:6801/1396866108] conn(0x7f58b406ec00 0x7f58b40710b0 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:21:59.499 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:21:59.500+0000 7f58cab23700 1 --2- 172.21.15.162:0/3324154666 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f58c4103780 0x7f58c40707a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:21:59.499 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:21:59.500+0000 7f58cab23700 1 -- 172.21.15.162:0/3324154666 >> 172.21.15.162:0/3324154666 conn(0x7f58c40f89b0 msgr2=0x7f58c40fa760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:21:59.499 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:21:59.500+0000 7f58cab23700 1 -- 172.21.15.162:0/3324154666 shutdown_connections 2024-04-27T04:21:59.500 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:21:59.500+0000 7f58cab23700 1 -- 172.21.15.162:0/3324154666 wait complete. 2024-04-27T04:21:59.509 INFO:teuthology.orchestra.run.smithi162.stdout:true 2024-04-27T04:22:00.136 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:21:59 smithi162 ceph-mon[110641]: from='client.? 172.21.15.162:0/3324154666' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-27T04:22:00.171 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:21:59 smithi181 ceph-mon[81134]: from='client.? 172.21.15.162:0/3324154666' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-27T04:22:00.199 DEBUG:teuthology.orchestra.run.smithi162:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 7f9750d6-044a-11ef-bc93-c7b262605968 -e sha1=4e213a58e787437ff508e6c890ccd5fd32dc96b5 -- bash -c 'ceph versions | jq -e '"'"'.overall | keys'"'"' | grep $sha1' 2024-04-27T04:22:00.455 INFO:teuthology.orchestra.run.smithi162.stderr:Inferring config /var/lib/ceph/7f9750d6-044a-11ef-bc93-c7b262605968/mon.smithi162/config 2024-04-27T04:22:00.835 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:22:00 smithi162 ceph-mon[110641]: pgmap v269: 65 pgs: 65 active+clean; 208 MiB data, 851 MiB used, 536 GiB / 536 GiB avail; 767 B/s rd, 1 op/s 2024-04-27T04:22:01.171 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:22:00 smithi181 ceph-mon[81134]: pgmap v269: 65 pgs: 65 active+clean; 208 MiB data, 851 MiB used, 536 GiB / 536 GiB avail; 767 B/s rd, 1 op/s 2024-04-27T04:22:02.336 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:02.194+0000 7f435e658700 1 -- 172.21.15.162:0/743979266 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f43581026a0 msgr2=0x7f43580fbd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:22:02.336 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:02.194+0000 7f435e658700 1 --2- 172.21.15.162:0/743979266 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f43581026a0 0x7f43580fbd10 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f434c004940 tx=0x7f434c009e30 comp rx=0 tx=0).stop 2024-04-27T04:22:02.336 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:02.194+0000 7f435e658700 1 -- 172.21.15.162:0/743979266 shutdown_connections 2024-04-27T04:22:02.337 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:02.194+0000 7f435e658700 1 --2- 172.21.15.162:0/743979266 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f43581026a0 0x7f43580fbd10 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:22:02.337 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:02.194+0000 7f435e658700 1 --2- 172.21.15.162:0/743979266 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4358101d00 0x7f43581020d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:22:02.337 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:02.194+0000 7f435e658700 1 -- 172.21.15.162:0/743979266 >> 172.21.15.162:0/743979266 conn(0x7f43580f7870 msgr2=0x7f43580f9c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:22:02.337 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:02.194+0000 7f435e658700 1 -- 172.21.15.162:0/743979266 shutdown_connections 2024-04-27T04:22:02.337 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:02.194+0000 7f435e658700 1 -- 172.21.15.162:0/743979266 wait complete. 2024-04-27T04:22:02.337 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:02.195+0000 7f435e658700 1 Processor -- start 2024-04-27T04:22:02.337 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:02.195+0000 7f435e658700 1 -- start start 2024-04-27T04:22:02.337 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:02.195+0000 7f435e658700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4358101d00 0x7f43581128b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:22:02.337 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:02.195+0000 7f435e658700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f43581026a0 0x7f4358112df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:22:02.337 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:02.195+0000 7f435e658700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4358113440 con 0x7f4358101d00 2024-04-27T04:22:02.337 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:02.195+0000 7f435e658700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f435810ca40 con 0x7f43581026a0 2024-04-27T04:22:02.337 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:02.195+0000 7f4357fff700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4358101d00 0x7f43581128b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:22:02.337 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:02.195+0000 7f4357fff700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4358101d00 0x7f43581128b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:48274/0 (socket says 172.21.15.162:48274) 2024-04-27T04:22:02.338 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:02.195+0000 7f4357fff700 1 -- 172.21.15.162:0/2559599577 learned_addr learned my addr 172.21.15.162:0/2559599577 (peer_addr_for_me v2:172.21.15.162:0/0) 2024-04-27T04:22:02.338 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:02.195+0000 7f4357fff700 1 -- 172.21.15.162:0/2559599577 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f43581026a0 msgr2=0x7f4358112df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:22:02.338 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:02.195+0000 7f4357fff700 1 --2- 172.21.15.162:0/2559599577 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f43581026a0 0x7f4358112df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:22:02.338 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:02.195+0000 7f4357fff700 1 -- 172.21.15.162:0/2559599577 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f434c005040 con 0x7f4358101d00 2024-04-27T04:22:02.338 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:02.196+0000 7f4357fff700 1 --2- 172.21.15.162:0/2559599577 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4358101d00 0x7f43581128b0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f4348002a50 tx=0x7f43480046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:22:02.338 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:02.196+0000 7f4354ff9700 1 -- 172.21.15.162:0/2559599577 <== mon.0 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4348007cc0 con 0x7f4358101d00 2024-04-27T04:22:02.338 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:02.196+0000 7f4354ff9700 1 -- 172.21.15.162:0/2559599577 <== mon.0 v2:172.21.15.162:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f434800f040 con 0x7f4358101d00 2024-04-27T04:22:02.338 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:02.196+0000 7f435e658700 1 -- 172.21.15.162:0/2559599577 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f435810cd20 con 0x7f4358101d00 2024-04-27T04:22:02.338 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:02.196+0000 7f4354ff9700 1 -- 172.21.15.162:0/2559599577 <== mon.0 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4348009640 con 0x7f4358101d00 2024-04-27T04:22:02.338 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:02.196+0000 7f435e658700 1 -- 172.21.15.162:0/2559599577 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f435810d2e0 con 0x7f4358101d00 2024-04-27T04:22:02.338 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:02.197+0000 7f435e658700 1 -- 172.21.15.162:0/2559599577 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f43580fd450 con 0x7f4358101d00 2024-04-27T04:22:02.338 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:02.197+0000 7f4354ff9700 1 -- 172.21.15.162:0/2559599577 <== mon.0 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 39) v1 ==== 92747+0+0 (secure 0 0 0) 0x7f43480097a0 con 0x7f4358101d00 2024-04-27T04:22:02.338 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:02.198+0000 7f4354ff9700 1 --2- 172.21.15.162:0/2559599577 >> [v2:172.21.15.162:6800/1396866108,v1:172.21.15.162:6801/1396866108] conn(0x7f434006ec00 0x7f43400710b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:22:02.339 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:02.198+0000 7f4354ff9700 1 -- 172.21.15.162:0/2559599577 <== mon.0 v2:172.21.15.162:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6582+0+0 (secure 0 0 0) 0x7f434808e6e0 con 0x7f4358101d00 2024-04-27T04:22:02.339 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:02.198+0000 7f43577fe700 1 --2- 172.21.15.162:0/2559599577 >> [v2:172.21.15.162:6800/1396866108,v1:172.21.15.162:6801/1396866108] conn(0x7f434006ec00 0x7f43400710b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:22:02.339 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:02.198+0000 7f43577fe700 1 --2- 172.21.15.162:0/2559599577 >> [v2:172.21.15.162:6800/1396866108,v1:172.21.15.162:6801/1396866108] conn(0x7f434006ec00 0x7f43400710b0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f434c00b3b0 tx=0x7f434c009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:22:02.339 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:02.202+0000 7f4354ff9700 1 -- 172.21.15.162:0/2559599577 <== mon.0 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7f434805af60 con 0x7f4358101d00 2024-04-27T04:22:02.488 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:02.489+0000 7f435e658700 1 -- 172.21.15.162:0/2559599577 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f4358108e70 con 0x7f4358101d00 2024-04-27T04:22:02.488 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:02.490+0000 7f4354ff9700 1 -- 172.21.15.162:0/2559599577 <== mon.0 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+623 (secure 0 0 0) 0x7f434805aaf0 con 0x7f4358101d00 2024-04-27T04:22:02.491 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:02.492+0000 7f435e658700 1 -- 172.21.15.162:0/2559599577 >> [v2:172.21.15.162:6800/1396866108,v1:172.21.15.162:6801/1396866108] conn(0x7f434006ec00 msgr2=0x7f43400710b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:22:02.491 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:02.492+0000 7f435e658700 1 --2- 172.21.15.162:0/2559599577 >> [v2:172.21.15.162:6800/1396866108,v1:172.21.15.162:6801/1396866108] conn(0x7f434006ec00 0x7f43400710b0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f434c00b3b0 tx=0x7f434c009e30 comp rx=0 tx=0).stop 2024-04-27T04:22:02.491 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:02.492+0000 7f435e658700 1 -- 172.21.15.162:0/2559599577 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4358101d00 msgr2=0x7f43581128b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:22:02.491 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:02.492+0000 7f435e658700 1 --2- 172.21.15.162:0/2559599577 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4358101d00 0x7f43581128b0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f4348002a50 tx=0x7f43480046b0 comp rx=0 tx=0).stop 2024-04-27T04:22:02.491 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:02.492+0000 7f435e658700 1 -- 172.21.15.162:0/2559599577 shutdown_connections 2024-04-27T04:22:02.491 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:02.492+0000 7f435e658700 1 --2- 172.21.15.162:0/2559599577 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4358101d00 0x7f43581128b0 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:22:02.491 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:02.492+0000 7f435e658700 1 --2- 172.21.15.162:0/2559599577 >> [v2:172.21.15.162:6800/1396866108,v1:172.21.15.162:6801/1396866108] conn(0x7f434006ec00 0x7f43400710b0 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:22:02.491 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:02.492+0000 7f435e658700 1 --2- 172.21.15.162:0/2559599577 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f43581026a0 0x7f4358112df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:22:02.491 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:02.492+0000 7f435e658700 1 -- 172.21.15.162:0/2559599577 >> 172.21.15.162:0/2559599577 conn(0x7f43580f7870 msgr2=0x7f43580f87f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:22:02.491 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:02.492+0000 7f435e658700 1 -- 172.21.15.162:0/2559599577 shutdown_connections 2024-04-27T04:22:02.492 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:02.492+0000 7f435e658700 1 -- 172.21.15.162:0/2559599577 wait complete. 2024-04-27T04:22:02.501 INFO:teuthology.orchestra.run.smithi162.stdout: "ceph version 19.0.0-3253-g4e213a58 (4e213a58e787437ff508e6c890ccd5fd32dc96b5) squid (dev)" 2024-04-27T04:22:02.797 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:22:02 smithi162 ceph-mon[110641]: pgmap v270: 65 pgs: 65 active+clean; 208 MiB data, 851 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-04-27T04:22:02.797 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:22:02 smithi162 ceph-mon[110641]: from='client.? 172.21.15.162:0/2559599577' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-27T04:22:03.136 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:22:02 smithi162 ceph-mon[110641]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-27T04:22:03.171 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:22:02 smithi181 ceph-mon[81134]: pgmap v270: 65 pgs: 65 active+clean; 208 MiB data, 851 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-04-27T04:22:03.172 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:22:02 smithi181 ceph-mon[81134]: from='client.? 172.21.15.162:0/2559599577' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-27T04:22:03.172 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:22:02 smithi181 ceph-mon[81134]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-27T04:22:03.275 DEBUG:teuthology.parallel:result is None 2024-04-27T04:22:03.275 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-04-27T04:22:03.287 INFO:tasks.cephadm:Running commands on role host.a host ubuntu@smithi162.front.sepia.ceph.com 2024-04-27T04:22:03.287 DEBUG:teuthology.orchestra.run.smithi162:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 7f9750d6-044a-11ef-bc93-c7b262605968 -- bash -c 'ceph fs dump' 2024-04-27T04:22:03.546 INFO:teuthology.orchestra.run.smithi162.stderr:Inferring config /var/lib/ceph/7f9750d6-044a-11ef-bc93-c7b262605968/mon.smithi162/config 2024-04-27T04:22:04.985 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:22:04 smithi162 ceph-mon[110641]: pgmap v271: 65 pgs: 65 active+clean; 208 MiB data, 851 MiB used, 536 GiB / 536 GiB avail; 767 B/s rd, 1 op/s 2024-04-27T04:22:05.093 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:05.085+0000 7f4c5be80700 1 -- 172.21.15.162:0/2565222664 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4c54103790 msgr2=0x7f4c540fce00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:22:05.094 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:05.085+0000 7f4c5be80700 1 --2- 172.21.15.162:0/2565222664 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4c54103790 0x7f4c540fce00 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f4c44004940 tx=0x7f4c44009e30 comp rx=0 tx=0).stop 2024-04-27T04:22:05.094 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:05.085+0000 7f4c5be80700 1 -- 172.21.15.162:0/2565222664 shutdown_connections 2024-04-27T04:22:05.094 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:05.085+0000 7f4c5be80700 1 --2- 172.21.15.162:0/2565222664 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4c54103790 0x7f4c540fce00 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:22:05.094 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:05.085+0000 7f4c5be80700 1 --2- 172.21.15.162:0/2565222664 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4c54102df0 0x7f4c541031c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:22:05.094 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:05.085+0000 7f4c5be80700 1 -- 172.21.15.162:0/2565222664 >> 172.21.15.162:0/2565222664 conn(0x7f4c540f8980 msgr2=0x7f4c540fad90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:22:05.094 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:05.085+0000 7f4c5be80700 1 -- 172.21.15.162:0/2565222664 shutdown_connections 2024-04-27T04:22:05.094 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:05.085+0000 7f4c5be80700 1 -- 172.21.15.162:0/2565222664 wait complete. 2024-04-27T04:22:05.094 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:05.086+0000 7f4c5be80700 1 Processor -- start 2024-04-27T04:22:05.094 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:05.086+0000 7f4c5be80700 1 -- start start 2024-04-27T04:22:05.094 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:05.086+0000 7f4c5be80700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4c54102df0 0x7f4c54113b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:22:05.094 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:05.086+0000 7f4c5be80700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4c54103790 0x7f4c541140b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:22:05.095 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:05.086+0000 7f4c5be80700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c54114700 con 0x7f4c54103790 2024-04-27T04:22:05.095 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:05.086+0000 7f4c5be80700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c5410daf0 con 0x7f4c54102df0 2024-04-27T04:22:05.095 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:05.086+0000 7f4c5941b700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4c54103790 0x7f4c541140b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:22:05.095 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:05.086+0000 7f4c5941b700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4c54103790 0x7f4c541140b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:56710/0 (socket says 172.21.15.162:56710) 2024-04-27T04:22:05.095 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:05.086+0000 7f4c5941b700 1 -- 172.21.15.162:0/2720658850 learned_addr learned my addr 172.21.15.162:0/2720658850 (peer_addr_for_me v2:172.21.15.162:0/0) 2024-04-27T04:22:05.095 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:05.086+0000 7f4c59c1c700 1 --2- 172.21.15.162:0/2720658850 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4c54102df0 0x7f4c54113b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:22:05.095 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:05.086+0000 7f4c5941b700 1 -- 172.21.15.162:0/2720658850 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4c54102df0 msgr2=0x7f4c54113b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:22:05.095 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:05.086+0000 7f4c5941b700 1 --2- 172.21.15.162:0/2720658850 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4c54102df0 0x7f4c54113b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:22:05.095 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:05.086+0000 7f4c5941b700 1 -- 172.21.15.162:0/2720658850 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c44005040 con 0x7f4c54103790 2024-04-27T04:22:05.095 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:05.087+0000 7f4c5941b700 1 --2- 172.21.15.162:0/2720658850 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4c54103790 0x7f4c541140b0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f4c44009100 tx=0x7f4c44009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:22:05.095 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:05.087+0000 7f4c4a7fc700 1 -- 172.21.15.162:0/2720658850 <== mon.0 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4c4400e040 con 0x7f4c54103790 2024-04-27T04:22:05.095 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:05.087+0000 7f4c5be80700 1 -- 172.21.15.162:0/2720658850 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4c5410dd70 con 0x7f4c54103790 2024-04-27T04:22:05.095 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:05.087+0000 7f4c4a7fc700 1 -- 172.21.15.162:0/2720658850 <== mon.0 v2:172.21.15.162:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f4c44013420 con 0x7f4c54103790 2024-04-27T04:22:05.095 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:05.087+0000 7f4c4a7fc700 1 -- 172.21.15.162:0/2720658850 <== mon.0 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4c4401c610 con 0x7f4c54103790 2024-04-27T04:22:05.096 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:05.087+0000 7f4c5be80700 1 -- 172.21.15.162:0/2720658850 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4c5410e330 con 0x7f4c54103790 2024-04-27T04:22:05.096 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:05.088+0000 7f4c5be80700 1 -- 172.21.15.162:0/2720658850 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4c540fe540 con 0x7f4c54103790 2024-04-27T04:22:05.096 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:05.088+0000 7f4c4a7fc700 1 -- 172.21.15.162:0/2720658850 <== mon.0 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 39) v1 ==== 92747+0+0 (secure 0 0 0) 0x7f4c44013590 con 0x7f4c54103790 2024-04-27T04:22:05.096 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:05.089+0000 7f4c4a7fc700 1 --2- 172.21.15.162:0/2720658850 >> [v2:172.21.15.162:6800/1396866108,v1:172.21.15.162:6801/1396866108] conn(0x7f4c4006ec00 0x7f4c400710b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:22:05.096 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:05.089+0000 7f4c4a7fc700 1 -- 172.21.15.162:0/2720658850 <== mon.0 v2:172.21.15.162:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6582+0+0 (secure 0 0 0) 0x7f4c4408fb40 con 0x7f4c54103790 2024-04-27T04:22:05.096 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:05.089+0000 7f4c59c1c700 1 --2- 172.21.15.162:0/2720658850 >> [v2:172.21.15.162:6800/1396866108,v1:172.21.15.162:6801/1396866108] conn(0x7f4c4006ec00 0x7f4c400710b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:22:05.096 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:05.089+0000 7f4c59c1c700 1 --2- 172.21.15.162:0/2720658850 >> [v2:172.21.15.162:6800/1396866108,v1:172.21.15.162:6801/1396866108] conn(0x7f4c4006ec00 0x7f4c400710b0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f4c500041d0 tx=0x7f4c50009040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:22:05.096 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:05.094+0000 7f4c4a7fc700 1 -- 172.21.15.162:0/2720658850 <== mon.0 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7f4c4405c440 con 0x7f4c54103790 2024-04-27T04:22:05.171 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:22:04 smithi181 ceph-mon[81134]: pgmap v271: 65 pgs: 65 active+clean; 208 MiB data, 851 MiB used, 536 GiB / 536 GiB avail; 767 B/s rd, 1 op/s 2024-04-27T04:22:05.338 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:05.339+0000 7f4c5be80700 1 -- 172.21.15.162:0/2720658850 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f4c54109ed0 con 0x7f4c54103790 2024-04-27T04:22:05.338 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:05.339+0000 7f4c4a7fc700 1 -- 172.21.15.162:0/2720658850 <== mon.0 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 35 v35) v1 ==== 76+0+1969 (secure 0 0 0) 0x7f4c4405bfd0 con 0x7f4c54103790 2024-04-27T04:22:05.338 INFO:teuthology.orchestra.run.smithi162.stdout:e35 2024-04-27T04:22:05.338 INFO:teuthology.orchestra.run.smithi162.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-04-27T04:22:05.339 INFO:teuthology.orchestra.run.smithi162.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-27T04:22:05.339 INFO:teuthology.orchestra.run.smithi162.stdout:legacy client fscid: 1 2024-04-27T04:22:05.339 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-04-27T04:22:05.339 INFO:teuthology.orchestra.run.smithi162.stdout:Filesystem 'cephfs' (1) 2024-04-27T04:22:05.339 INFO:teuthology.orchestra.run.smithi162.stdout:fs_name cephfs 2024-04-27T04:22:05.339 INFO:teuthology.orchestra.run.smithi162.stdout:epoch 35 2024-04-27T04:22:05.339 INFO:teuthology.orchestra.run.smithi162.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-04-27T04:22:05.339 INFO:teuthology.orchestra.run.smithi162.stdout:created 2024-04-27T04:06:53.095749+0000 2024-04-27T04:22:05.339 INFO:teuthology.orchestra.run.smithi162.stdout:modified 2024-04-27T04:20:08.657609+0000 2024-04-27T04:22:05.339 INFO:teuthology.orchestra.run.smithi162.stdout:tableserver 0 2024-04-27T04:22:05.339 INFO:teuthology.orchestra.run.smithi162.stdout:root 0 2024-04-27T04:22:05.339 INFO:teuthology.orchestra.run.smithi162.stdout:session_timeout 60 2024-04-27T04:22:05.339 INFO:teuthology.orchestra.run.smithi162.stdout:session_autoclose 300 2024-04-27T04:22:05.339 INFO:teuthology.orchestra.run.smithi162.stdout:max_file_size 1099511627776 2024-04-27T04:22:05.340 INFO:teuthology.orchestra.run.smithi162.stdout:max_xattr_size 65536 2024-04-27T04:22:05.340 INFO:teuthology.orchestra.run.smithi162.stdout:required_client_features {} 2024-04-27T04:22:05.340 INFO:teuthology.orchestra.run.smithi162.stdout:last_failure 0 2024-04-27T04:22:05.340 INFO:teuthology.orchestra.run.smithi162.stdout:last_failure_osd_epoch 84 2024-04-27T04:22:05.340 INFO:teuthology.orchestra.run.smithi162.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-04-27T04:22:05.340 INFO:teuthology.orchestra.run.smithi162.stdout:max_mds 1 2024-04-27T04:22:05.340 INFO:teuthology.orchestra.run.smithi162.stdout:in 0 2024-04-27T04:22:05.340 INFO:teuthology.orchestra.run.smithi162.stdout:up {0=34380} 2024-04-27T04:22:05.340 INFO:teuthology.orchestra.run.smithi162.stdout:failed 2024-04-27T04:22:05.340 INFO:teuthology.orchestra.run.smithi162.stdout:damaged 2024-04-27T04:22:05.340 INFO:teuthology.orchestra.run.smithi162.stdout:stopped 2024-04-27T04:22:05.340 INFO:teuthology.orchestra.run.smithi162.stdout:data_pools [3] 2024-04-27T04:22:05.340 INFO:teuthology.orchestra.run.smithi162.stdout:metadata_pool 2 2024-04-27T04:22:05.341 INFO:teuthology.orchestra.run.smithi162.stdout:inline_data disabled 2024-04-27T04:22:05.341 INFO:teuthology.orchestra.run.smithi162.stdout:balancer 2024-04-27T04:22:05.341 INFO:teuthology.orchestra.run.smithi162.stdout:bal_rank_mask -1 2024-04-27T04:22:05.341 INFO:teuthology.orchestra.run.smithi162.stdout:standby_count_wanted 1 2024-04-27T04:22:05.341 INFO:teuthology.orchestra.run.smithi162.stdout:qdb_cluster leader: 34380 members: 34380 2024-04-27T04:22:05.341 INFO:teuthology.orchestra.run.smithi162.stdout:[mds.cephfs.smithi162.gzlyjj{0:34380} state up:active seq 20 join_fscid=1 addr [v2:172.21.15.162:6826/1198130667,v1:172.21.15.162:6827/1198130667] compat {c=[1],r=[1],i=[1fff]}] 2024-04-27T04:22:05.341 INFO:teuthology.orchestra.run.smithi162.stdout:[mds.cephfs.smithi162.xcnapk{0:34412} state up:standby-replay seq 1 join_fscid=1 addr [v2:172.21.15.162:6828/307528848,v1:172.21.15.162:6829/307528848] compat {c=[1],r=[1],i=[1fff]}] 2024-04-27T04:22:05.341 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-04-27T04:22:05.341 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-04-27T04:22:05.341 INFO:teuthology.orchestra.run.smithi162.stdout:Standby daemons: 2024-04-27T04:22:05.341 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-04-27T04:22:05.341 INFO:teuthology.orchestra.run.smithi162.stdout:[mds.cephfs.smithi181.dzomgh{-1:44319} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.181:6824/1667773112,v1:172.21.15.181:6825/1667773112] compat {c=[1],r=[1],i=[1fff]}] 2024-04-27T04:22:05.342 INFO:teuthology.orchestra.run.smithi162.stdout:[mds.cephfs.smithi181.bkkcdm{-1:44323} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.181:6826/3589158285,v1:172.21.15.181:6827/3589158285] compat {c=[1],r=[1],i=[1fff]}] 2024-04-27T04:22:05.342 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:05.342+0000 7f4c5be80700 1 -- 172.21.15.162:0/2720658850 >> [v2:172.21.15.162:6800/1396866108,v1:172.21.15.162:6801/1396866108] conn(0x7f4c4006ec00 msgr2=0x7f4c400710b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:22:05.342 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:05.342+0000 7f4c5be80700 1 --2- 172.21.15.162:0/2720658850 >> [v2:172.21.15.162:6800/1396866108,v1:172.21.15.162:6801/1396866108] conn(0x7f4c4006ec00 0x7f4c400710b0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f4c500041d0 tx=0x7f4c50009040 comp rx=0 tx=0).stop 2024-04-27T04:22:05.342 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:05.342+0000 7f4c5be80700 1 -- 172.21.15.162:0/2720658850 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4c54103790 msgr2=0x7f4c541140b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:22:05.343 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:05.342+0000 7f4c5be80700 1 --2- 172.21.15.162:0/2720658850 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4c54103790 0x7f4c541140b0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f4c44009100 tx=0x7f4c44009e30 comp rx=0 tx=0).stop 2024-04-27T04:22:05.343 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:05.342+0000 7f4c5be80700 1 -- 172.21.15.162:0/2720658850 shutdown_connections 2024-04-27T04:22:05.343 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:05.342+0000 7f4c5be80700 1 --2- 172.21.15.162:0/2720658850 >> [v2:172.21.15.162:6800/1396866108,v1:172.21.15.162:6801/1396866108] conn(0x7f4c4006ec00 0x7f4c400710b0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:22:05.343 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:05.342+0000 7f4c5be80700 1 --2- 172.21.15.162:0/2720658850 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4c54102df0 0x7f4c54113b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:22:05.343 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:05.342+0000 7f4c5be80700 1 --2- 172.21.15.162:0/2720658850 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4c54103790 0x7f4c541140b0 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:22:05.343 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:05.342+0000 7f4c5be80700 1 -- 172.21.15.162:0/2720658850 >> 172.21.15.162:0/2720658850 conn(0x7f4c540f8980 msgr2=0x7f4c540f99a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:22:05.343 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:05.342+0000 7f4c5be80700 1 -- 172.21.15.162:0/2720658850 shutdown_connections 2024-04-27T04:22:05.343 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:05.342+0000 7f4c5be80700 1 -- 172.21.15.162:0/2720658850 wait complete. 2024-04-27T04:22:05.343 INFO:teuthology.orchestra.run.smithi162.stderr:dumped fsmap epoch 35 2024-04-27T04:22:05.982 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:22:05 smithi162 ceph-mon[110641]: from='client.? 172.21.15.162:0/2720658850' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-27T04:22:06.006 INFO:teuthology.run_tasks:Running task fs.post_upgrade_checks... 2024-04-27T04:22:06.018 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-27T04:22:06.117 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:06.116+0000 7ff505207640 1 -- 172.21.15.162:0/3408156573 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff500104d30 msgr2=0x7ff5000fe3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:22:06.117 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:06.116+0000 7ff505207640 1 --2- 172.21.15.162:0/3408156573 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff500104d30 0x7ff5000fe3c0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7ff4f40048d0 tx=0x7ff4f4009da0 comp rx=0 tx=0).stop 2024-04-27T04:22:06.117 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:06.116+0000 7ff505207640 1 -- 172.21.15.162:0/3408156573 shutdown_connections 2024-04-27T04:22:06.117 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:06.116+0000 7ff505207640 1 --2- 172.21.15.162:0/3408156573 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff500104d30 0x7ff5000fe3c0 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:22:06.117 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:06.116+0000 7ff505207640 1 --2- 172.21.15.162:0/3408156573 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff500104390 0x7ff500104760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:22:06.117 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:06.116+0000 7ff505207640 1 -- 172.21.15.162:0/3408156573 >> 172.21.15.162:0/3408156573 conn(0x7ff5000f9f60 msgr2=0x7ff5000fc370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:22:06.117 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:06.116+0000 7ff505207640 1 -- 172.21.15.162:0/3408156573 shutdown_connections 2024-04-27T04:22:06.117 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:06.116+0000 7ff505207640 1 -- 172.21.15.162:0/3408156573 wait complete. 2024-04-27T04:22:06.118 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:06.117+0000 7ff505207640 1 Processor -- start 2024-04-27T04:22:06.118 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:06.117+0000 7ff505207640 1 -- start start 2024-04-27T04:22:06.118 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:06.117+0000 7ff505207640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff500104390 0x7ff500114ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:22:06.118 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:06.117+0000 7ff505207640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff500104d30 0x7ff500115420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:22:06.118 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:06.117+0000 7ff505207640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff500115a20 con 0x7ff500104d30 2024-04-27T04:22:06.118 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:06.117+0000 7ff505207640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff50010f070 con 0x7ff500104390 2024-04-27T04:22:06.118 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:06.117+0000 7ff4fe59c640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff500104d30 0x7ff500115420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:22:06.118 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:06.117+0000 7ff4fe59c640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff500104d30 0x7ff500115420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:56740/0 (socket says 172.21.15.162:56740) 2024-04-27T04:22:06.118 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:06.117+0000 7ff4fe59c640 1 -- 172.21.15.162:0/4030251414 learned_addr learned my addr 172.21.15.162:0/4030251414 (peer_addr_for_me v2:172.21.15.162:0/0) 2024-04-27T04:22:06.118 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:06.117+0000 7ff4fed9d640 1 --2- 172.21.15.162:0/4030251414 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff500104390 0x7ff500114ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:22:06.119 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:06.117+0000 7ff4fe59c640 1 -- 172.21.15.162:0/4030251414 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff500104390 msgr2=0x7ff500114ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:22:06.119 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:06.117+0000 7ff4fe59c640 1 --2- 172.21.15.162:0/4030251414 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff500104390 0x7ff500114ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:22:06.119 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:06.117+0000 7ff4fe59c640 1 -- 172.21.15.162:0/4030251414 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4f4005040 con 0x7ff500104d30 2024-04-27T04:22:06.119 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:06.118+0000 7ff4fe59c640 1 --2- 172.21.15.162:0/4030251414 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff500104d30 0x7ff500115420 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7ff4f4004930 tx=0x7ff4f4009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:22:06.119 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:06.118+0000 7ff4e77fe640 1 -- 172.21.15.162:0/4030251414 <== mon.0 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff4f40094c0 con 0x7ff500104d30 2024-04-27T04:22:06.119 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:06.118+0000 7ff4e77fe640 1 -- 172.21.15.162:0/4030251414 <== mon.0 v2:172.21.15.162:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7ff4f4032e50 con 0x7ff500104d30 2024-04-27T04:22:06.119 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:06.118+0000 7ff505207640 1 -- 172.21.15.162:0/4030251414 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff50010f2f0 con 0x7ff500104d30 2024-04-27T04:22:06.119 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:06.118+0000 7ff4e77fe640 1 -- 172.21.15.162:0/4030251414 <== mon.0 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff4f40433b0 con 0x7ff500104d30 2024-04-27T04:22:06.119 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:06.118+0000 7ff505207640 1 -- 172.21.15.162:0/4030251414 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff50010f7b0 con 0x7ff500104d30 2024-04-27T04:22:06.120 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:06.119+0000 7ff505207640 1 -- 172.21.15.162:0/4030251414 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff5000ffb00 con 0x7ff500104d30 2024-04-27T04:22:06.120 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:06.119+0000 7ff4e77fe640 1 -- 172.21.15.162:0/4030251414 <== mon.0 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 39) v1 ==== 92747+0+0 (secure 0 0 0) 0x7ff4f402f070 con 0x7ff500104d30 2024-04-27T04:22:06.120 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:06.120+0000 7ff4e77fe640 1 --2- 172.21.15.162:0/4030251414 >> [v2:172.21.15.162:6800/1396866108,v1:172.21.15.162:6801/1396866108] conn(0x7ff4e806eb70 0x7ff4e8071020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:22:06.121 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:06.120+0000 7ff4fed9d640 1 --2- 172.21.15.162:0/4030251414 >> [v2:172.21.15.162:6800/1396866108,v1:172.21.15.162:6801/1396866108] conn(0x7ff4e806eb70 0x7ff4e8071020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:22:06.121 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:06.120+0000 7ff4e77fe640 1 -- 172.21.15.162:0/4030251414 <== mon.0 v2:172.21.15.162:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6582+0+0 (secure 0 0 0) 0x7ff4f40b07c0 con 0x7ff500104d30 2024-04-27T04:22:06.121 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:06.120+0000 7ff4fed9d640 1 --2- 172.21.15.162:0/4030251414 >> [v2:172.21.15.162:6800/1396866108,v1:172.21.15.162:6801/1396866108] conn(0x7ff4e806eb70 0x7ff4e8071020 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7ff4f0007fa0 tx=0x7ff4f0004c70 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:22:06.123 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:06.124+0000 7ff4e77fe640 1 -- 172.21.15.162:0/4030251414 <== mon.0 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7ff4f4079510 con 0x7ff500104d30 2024-04-27T04:22:06.171 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:22:05 smithi181 ceph-mon[81134]: from='client.? 172.21.15.162:0/2720658850' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-27T04:22:06.299 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:06.301+0000 7ff505207640 1 -- 172.21.15.162:0/4030251414 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ff50010b4f0 con 0x7ff500104d30 2024-04-27T04:22:06.301 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:06.302+0000 7ff4e77fe640 1 -- 172.21.15.162:0/4030251414 <== mon.0 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v84) v1 ==== 74+0+16846 (secure 0 0 0) 0x7ff4f407cbd0 con 0x7ff500104d30 2024-04-27T04:22:06.301 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-04-27T04:22:06.301 INFO:teuthology.orchestra.run.smithi162.stdout:{"epoch":84,"fsid":"7f9750d6-044a-11ef-bc93-c7b262605968","created":"2024-04-27T04:00:06.343616+0000","modified":"2024-04-27T04:19:50.546745+0000","last_up_change":"2024-04-27T04:18:43.492214+0000","last_in_change":"2024-04-27T04:05:18.597695+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":20,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":6,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-27T04:04:24.971779+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":6,"score_stable":6,"optimal_score":0.5,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs.cephfs.meta","create_time":"2024-04-27T04:06:52.008169+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"44","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.690000057220459,"score_stable":1.690000057220459,"optimal_score":1,"raw_score_acting":1.690000057220459,"raw_score_stable":1.690000057220459,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs.cephfs.data","create_time":"2024-04-27T04:06:52.322198+0000","flags":131073,"flags_names":"hashpspool,bulk","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"45","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0ae69d7f-0ab4-4672-89b9-3563a10a1bf2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":9,"last_clean_end":49,"up_from":53,"up_thru":78,"down_at":50,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6802","nonce":1840117633},{"type":"v1","addr":"172.21.15.162:6803","nonce":1840117633}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6804","nonce":1840117633},{"type":"v1","addr":"172.21.15.162:6805","nonce":1840117633}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6808","nonce":1840117633},{"type":"v1","addr":"172.21.15.162:6809","nonce":1840117633}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6806","nonce":1840117633},{"type":"v1","addr":"172.21.15.162:6807","nonce":1840117633}]},"public_addr":"172.21.15.162:6803/1840117633","cluster_addr":"172.21.15.162:6805/1840117633","heartbeat_back_addr":"172.21.15.162:6809/1840117633","heartbeat_front_addr":"172.21.15.162:6807/1840117633","state":["exists","up"]},{"osd":1,"uuid":"95bdc9f0-c438-4a88-942d-dffb0c61c308","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":14,"last_clean_end":54,"up_from":58,"up_thru":78,"down_at":55,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6810","nonce":2144127279},{"type":"v1","addr":"172.21.15.162:6811","nonce":2144127279}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6812","nonce":2144127279},{"type":"v1","addr":"172.21.15.162:6813","nonce":2144127279}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6816","nonce":2144127279},{"type":"v1","addr":"172.21.15.162:6817","nonce":2144127279}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6814","nonce":2144127279},{"type":"v1","addr":"172.21.15.162:6815","nonce":2144127279}]},"public_addr":"172.21.15.162:6811/2144127279","cluster_addr":"172.21.15.162:6813/2144127279","heartbeat_back_addr":"172.21.15.162:6817/2144127279","heartbeat_front_addr":"172.21.15.162:6815/2144127279","state":["exists","up"]},{"osd":2,"uuid":"a773b386-e75b-432a-9d1b-2206301b96c3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":19,"last_clean_end":59,"up_from":63,"up_thru":78,"down_at":60,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6818","nonce":3115093010},{"type":"v1","addr":"172.21.15.162:6819","nonce":3115093010}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6820","nonce":3115093010},{"type":"v1","addr":"172.21.15.162:6821","nonce":3115093010}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6824","nonce":3115093010},{"type":"v1","addr":"172.21.15.162:6825","nonce":3115093010}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6822","nonce":3115093010},{"type":"v1","addr":"172.21.15.162:6823","nonce":3115093010}]},"public_addr":"172.21.15.162:6819/3115093010","cluster_addr":"172.21.15.162:6821/3115093010","heartbeat_back_addr":"172.21.15.162:6825/3115093010","heartbeat_front_addr":"172.21.15.162:6823/3115093010","state":["exists","up"]},{"osd":3,"uuid":"756091bc-1594-479d-9390-cb60e6b65ebe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":26,"last_clean_end":64,"up_from":68,"up_thru":78,"down_at":65,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":114683138},{"type":"v1","addr":"172.21.15.181:6801","nonce":114683138}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":114683138},{"type":"v1","addr":"172.21.15.181:6803","nonce":114683138}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6806","nonce":114683138},{"type":"v1","addr":"172.21.15.181:6807","nonce":114683138}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":114683138},{"type":"v1","addr":"172.21.15.181:6805","nonce":114683138}]},"public_addr":"172.21.15.181:6801/114683138","cluster_addr":"172.21.15.181:6803/114683138","heartbeat_back_addr":"172.21.15.181:6807/114683138","heartbeat_front_addr":"172.21.15.181:6805/114683138","state":["exists","up"]},{"osd":4,"uuid":"bd70494d-061c-4468-b837-26150c2d4208","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":34,"last_clean_end":69,"up_from":73,"up_thru":78,"down_at":70,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6808","nonce":1781129695},{"type":"v1","addr":"172.21.15.181:6809","nonce":1781129695}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":1781129695},{"type":"v1","addr":"172.21.15.181:6811","nonce":1781129695}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":1781129695},{"type":"v1","addr":"172.21.15.181:6815","nonce":1781129695}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6812","nonce":1781129695},{"type":"v1","addr":"172.21.15.181:6813","nonce":1781129695}]},"public_addr":"172.21.15.181:6809/1781129695","cluster_addr":"172.21.15.181:6811/1781129695","heartbeat_back_addr":"172.21.15.181:6815/1781129695","heartbeat_front_addr":"172.21.15.181:6813/1781129695","state":["exists","up"]},{"osd":5,"uuid":"15455ef6-2853-4a84-aaa9-1797e78302a9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":39,"last_clean_end":74,"up_from":78,"up_thru":78,"down_at":75,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":2414821263},{"type":"v1","addr":"172.21.15.181:6817","nonce":2414821263}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6818","nonce":2414821263},{"type":"v1","addr":"172.21.15.181:6819","nonce":2414821263}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6822","nonce":2414821263},{"type":"v1","addr":"172.21.15.181:6823","nonce":2414821263}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":2414821263},{"type":"v1","addr":"172.21.15.181:6821","nonce":2414821263}]},"public_addr":"172.21.15.181:6817/2414821263","cluster_addr":"172.21.15.181:6819/2414821263","heartbeat_back_addr":"172.21.15.181:6823/2414821263","heartbeat_front_addr":"172.21.15.181:6821/2414821263","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-27T04:03:28.522704+0000","dead_epoch":49},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-27T04:03:54.648083+0000","dead_epoch":54},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-27T04:04:22.107316+0000","dead_epoch":59},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-27T04:04:46.020733+0000","dead_epoch":64},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-27T04:05:10.634082+0000","dead_epoch":69},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-27T04:05:35.756217+0000","dead_epoch":74}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.181:6827/3756475453":"2024-04-28T04:19:50.546716+0000","172.21.15.181:6826/3756475453":"2024-04-28T04:19:50.546716+0000","172.21.15.162:0/1045614319":"2024-04-28T04:14:17.593083+0000","172.21.15.181:6828/4165336392":"2024-04-28T04:11:32.210928+0000","172.21.15.162:0/2336042321":"2024-04-28T04:02:16.661212+0000","172.21.15.162:6800/4079782599":"2024-04-28T04:10:05.654758+0000","172.21.15.181:6824/658214993":"2024-04-28T04:19:26.414790+0000","172.21.15.181:6829/4165336392":"2024-04-28T04:11:32.210928+0000","172.21.15.181:0/642670640":"2024-04-28T04:11:32.210928+0000","172.21.15.162:0/3828848945":"2024-04-28T04:14:17.593083+0000","172.21.15.162:0/1971036271":"2024-04-28T04:10:05.654758+0000","172.21.15.162:6800/8067198":"2024-04-28T04:01:10.996874+0000","172.21.15.162:0/2623011446":"2024-04-28T04:00:40.642043+0000","172.21.15.162:0/1703886136":"2024-04-28T04:02:16.661212+0000","172.21.15.162:0/2129785584":"2024-04-28T04:01:10.996874+0000","172.21.15.162:6800/1482851081":"2024-04-28T04:02:16.661212+0000","172.21.15.162:6801/1480340393":"2024-04-28T04:00:40.642043+0000","172.21.15.181:0/209590021":"2024-04-28T04:11:32.210928+0000","172.21.15.162:0/4025365522":"2024-04-28T04:00:40.642043+0000","172.21.15.181:0/3276305180":"2024-04-28T04:11:32.210928+0000","172.21.15.162:6801/1482851081":"2024-04-28T04:02:16.661212+0000","172.21.15.162:0/3073351439":"2024-04-28T04:01:10.996874+0000","172.21.15.181:6825/658214993":"2024-04-28T04:19:26.414790+0000","172.21.15.162:0/795779642":"2024-04-28T04:02:16.661212+0000","172.21.15.162:0/3590827229":"2024-04-28T04:01:10.996874+0000","172.21.15.181:0/2462932364":"2024-04-28T04:11:32.210928+0000","172.21.15.162:6801/8067198":"2024-04-28T04:01:10.996874+0000","172.21.15.162:6826/998468496":"2024-04-28T04:07:08.543563+0000","172.21.15.162:0/1349712416":"2024-04-28T04:10:05.654758+0000","172.21.15.162:6800/1480340393":"2024-04-28T04:00:40.642043+0000","172.21.15.181:0/1936190707":"2024-04-28T04:11:32.210928+0000","172.21.15.181:0/2761247390":"2024-04-28T04:11:32.210928+0000","172.21.15.162:6801/4079782599":"2024-04-28T04:10:05.654758+0000","172.21.15.162:0/3944767668":"2024-04-28T04:10:05.654758+0000","172.21.15.162:0/2579239891":"2024-04-28T04:00:40.642043+0000","172.21.15.162:6827/998468496":"2024-04-28T04:07:08.543563+0000","172.21.15.162:6800/901992993":"2024-04-28T04:14:17.593083+0000","172.21.15.162:0/2783280109":"2024-04-28T04:10:05.654758+0000","172.21.15.162:6801/901992993":"2024-04-28T04:14:17.593083+0000","172.21.15.162:0/1622403042":"2024-04-28T04:14:17.593083+0000","172.21.15.162:0/2023772646":"2024-04-28T04:14:17.593083+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-27T04:22:06.303 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:06.304+0000 7ff505207640 1 -- 172.21.15.162:0/4030251414 >> [v2:172.21.15.162:6800/1396866108,v1:172.21.15.162:6801/1396866108] conn(0x7ff4e806eb70 msgr2=0x7ff4e8071020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:22:06.303 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:06.304+0000 7ff505207640 1 --2- 172.21.15.162:0/4030251414 >> [v2:172.21.15.162:6800/1396866108,v1:172.21.15.162:6801/1396866108] conn(0x7ff4e806eb70 0x7ff4e8071020 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7ff4f0007fa0 tx=0x7ff4f0004c70 comp rx=0 tx=0).stop 2024-04-27T04:22:06.303 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:06.304+0000 7ff505207640 1 -- 172.21.15.162:0/4030251414 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff500104d30 msgr2=0x7ff500115420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:22:06.303 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:06.304+0000 7ff505207640 1 --2- 172.21.15.162:0/4030251414 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff500104d30 0x7ff500115420 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7ff4f4004930 tx=0x7ff4f4009da0 comp rx=0 tx=0).stop 2024-04-27T04:22:06.303 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:06.304+0000 7ff505207640 1 -- 172.21.15.162:0/4030251414 shutdown_connections 2024-04-27T04:22:06.303 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:06.304+0000 7ff505207640 1 --2- 172.21.15.162:0/4030251414 >> [v2:172.21.15.162:6800/1396866108,v1:172.21.15.162:6801/1396866108] conn(0x7ff4e806eb70 0x7ff4e8071020 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:22:06.304 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:06.304+0000 7ff505207640 1 --2- 172.21.15.162:0/4030251414 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff500104d30 0x7ff500115420 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:22:06.304 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:06.304+0000 7ff505207640 1 --2- 172.21.15.162:0/4030251414 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff500104390 0x7ff500114ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:22:06.304 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:06.304+0000 7ff505207640 1 -- 172.21.15.162:0/4030251414 >> 172.21.15.162:0/4030251414 conn(0x7ff5000f9f60 msgr2=0x7ff5000faf60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:22:06.304 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:06.304+0000 7ff505207640 1 -- 172.21.15.162:0/4030251414 shutdown_connections 2024-04-27T04:22:06.304 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:06.304+0000 7ff505207640 1 -- 172.21.15.162:0/4030251414 wait complete. 2024-04-27T04:22:06.315 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-27T04:04:24.971779+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 6, 'score_stable': 6, 'optimal_score': 0.5, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs.cephfs.meta', 'create_time': '2024-04-27T04:06:52.008169+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '44', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.690000057220459, 'score_stable': 1.690000057220459, 'optimal_score': 1, 'raw_score_acting': 1.690000057220459, 'raw_score_stable': 1.690000057220459, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs.cephfs.data', 'create_time': '2024-04-27T04:06:52.322198+0000', 'flags': 131073, 'flags_names': 'hashpspool,bulk', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '45', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-27T04:22:06.315 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-27T04:22:06.414 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:06.413+0000 7fba1635d640 1 -- 172.21.15.162:0/927598928 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fba101021e0 msgr2=0x7fba101025b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:22:06.414 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:06.413+0000 7fba1635d640 1 --2- 172.21.15.162:0/927598928 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fba101021e0 0x7fba101025b0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fba000048d0 tx=0x7fba00009da0 comp rx=0 tx=0).stop 2024-04-27T04:22:06.414 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:06.414+0000 7fba1635d640 1 -- 172.21.15.162:0/927598928 shutdown_connections 2024-04-27T04:22:06.414 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:06.414+0000 7fba1635d640 1 --2- 172.21.15.162:0/927598928 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fba10064380 0x7fba100647d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:22:06.414 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:06.414+0000 7fba1635d640 1 --2- 172.21.15.162:0/927598928 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fba101021e0 0x7fba101025b0 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:22:06.415 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:06.414+0000 7fba1635d640 1 -- 172.21.15.162:0/927598928 >> 172.21.15.162:0/927598928 conn(0x7fba100710c0 msgr2=0x7fba100714c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:22:06.415 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:06.414+0000 7fba1635d640 1 -- 172.21.15.162:0/927598928 shutdown_connections 2024-04-27T04:22:06.415 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:06.414+0000 7fba1635d640 1 -- 172.21.15.162:0/927598928 wait complete. 2024-04-27T04:22:06.415 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:06.414+0000 7fba1635d640 1 Processor -- start 2024-04-27T04:22:06.415 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:06.414+0000 7fba1635d640 1 -- start start 2024-04-27T04:22:06.415 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:06.414+0000 7fba1635d640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fba10064380 0x7fba10112ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:22:06.415 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:06.414+0000 7fba1635d640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fba101021e0 0x7fba101131e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:22:06.415 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:06.414+0000 7fba1635d640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba101137e0 con 0x7fba10064380 2024-04-27T04:22:06.415 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:06.414+0000 7fba1635d640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba1010ce30 con 0x7fba101021e0 2024-04-27T04:22:06.415 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:06.415+0000 7fba0ffff640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fba10064380 0x7fba10112ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:22:06.415 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:06.415+0000 7fba0ffff640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fba10064380 0x7fba10112ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:56758/0 (socket says 172.21.15.162:56758) 2024-04-27T04:22:06.415 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:06.415+0000 7fba0ffff640 1 -- 172.21.15.162:0/2266304651 learned_addr learned my addr 172.21.15.162:0/2266304651 (peer_addr_for_me v2:172.21.15.162:0/0) 2024-04-27T04:22:06.415 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:06.415+0000 7fba0f7fe640 1 --2- 172.21.15.162:0/2266304651 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fba101021e0 0x7fba101131e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:22:06.416 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:06.415+0000 7fba0ffff640 1 -- 172.21.15.162:0/2266304651 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fba101021e0 msgr2=0x7fba101131e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:22:06.416 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:06.415+0000 7fba0ffff640 1 --2- 172.21.15.162:0/2266304651 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fba101021e0 0x7fba101131e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:22:06.416 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:06.415+0000 7fba0ffff640 1 -- 172.21.15.162:0/2266304651 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba00005040 con 0x7fba10064380 2024-04-27T04:22:06.416 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:06.415+0000 7fba0ffff640 1 --2- 172.21.15.162:0/2266304651 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fba10064380 0x7fba10112ca0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fba00004930 tx=0x7fba00009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:22:06.416 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:06.415+0000 7fba0cff9640 1 -- 172.21.15.162:0/2266304651 <== mon.0 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fba000094c0 con 0x7fba10064380 2024-04-27T04:22:06.416 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:06.415+0000 7fba0cff9640 1 -- 172.21.15.162:0/2266304651 <== mon.0 v2:172.21.15.162:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fba00032e50 con 0x7fba10064380 2024-04-27T04:22:06.416 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:06.415+0000 7fba1635d640 1 -- 172.21.15.162:0/2266304651 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fba1010d0b0 con 0x7fba10064380 2024-04-27T04:22:06.416 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:06.415+0000 7fba0cff9640 1 -- 172.21.15.162:0/2266304651 <== mon.0 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fba000433b0 con 0x7fba10064380 2024-04-27T04:22:06.416 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:06.415+0000 7fba1635d640 1 -- 172.21.15.162:0/2266304651 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fba1010d570 con 0x7fba10064380 2024-04-27T04:22:06.417 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:06.416+0000 7fba1635d640 1 -- 172.21.15.162:0/2266304651 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fba100fd8e0 con 0x7fba10064380 2024-04-27T04:22:06.417 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:06.417+0000 7fba0cff9640 1 -- 172.21.15.162:0/2266304651 <== mon.0 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 39) v1 ==== 92747+0+0 (secure 0 0 0) 0x7fba0002f070 con 0x7fba10064380 2024-04-27T04:22:06.417 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:06.417+0000 7fba0cff9640 1 --2- 172.21.15.162:0/2266304651 >> [v2:172.21.15.162:6800/1396866108,v1:172.21.15.162:6801/1396866108] conn(0x7fb9f806eb70 0x7fb9f8071020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:22:06.417 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:06.417+0000 7fba0cff9640 1 -- 172.21.15.162:0/2266304651 <== mon.0 v2:172.21.15.162:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6582+0+0 (secure 0 0 0) 0x7fba000b07c0 con 0x7fba10064380 2024-04-27T04:22:06.417 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:06.417+0000 7fba0f7fe640 1 --2- 172.21.15.162:0/2266304651 >> [v2:172.21.15.162:6800/1396866108,v1:172.21.15.162:6801/1396866108] conn(0x7fb9f806eb70 0x7fb9f8071020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:22:06.417 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:06.418+0000 7fba0f7fe640 1 --2- 172.21.15.162:0/2266304651 >> [v2:172.21.15.162:6800/1396866108,v1:172.21.15.162:6801/1396866108] conn(0x7fb9f806eb70 0x7fb9f8071020 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fba04007fd0 tx=0x7fba04009e90 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:22:06.420 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:06.422+0000 7fba0cff9640 1 -- 172.21.15.162:0/2266304651 <== mon.0 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7fba00079510 con 0x7fba10064380 2024-04-27T04:22:06.596 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:06.597+0000 7fba1635d640 1 -- 172.21.15.162:0/2266304651 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fba1004bcc0 con 0x7fba10064380 2024-04-27T04:22:06.596 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:06.598+0000 7fba0cff9640 1 -- 172.21.15.162:0/2266304651 <== mon.0 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v84) v1 ==== 93+0+10 (secure 0 0 0) 0x7fba0007cbd0 con 0x7fba10064380 2024-04-27T04:22:06.596 INFO:teuthology.orchestra.run.smithi162.stdout:pg_num: 1 2024-04-27T04:22:06.598 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:06.599+0000 7fba1635d640 1 -- 172.21.15.162:0/2266304651 >> [v2:172.21.15.162:6800/1396866108,v1:172.21.15.162:6801/1396866108] conn(0x7fb9f806eb70 msgr2=0x7fb9f8071020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:22:06.599 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:06.599+0000 7fba1635d640 1 --2- 172.21.15.162:0/2266304651 >> [v2:172.21.15.162:6800/1396866108,v1:172.21.15.162:6801/1396866108] conn(0x7fb9f806eb70 0x7fb9f8071020 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fba04007fd0 tx=0x7fba04009e90 comp rx=0 tx=0).stop 2024-04-27T04:22:06.599 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:06.600+0000 7fba1635d640 1 -- 172.21.15.162:0/2266304651 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fba10064380 msgr2=0x7fba10112ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:22:06.599 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:06.600+0000 7fba1635d640 1 --2- 172.21.15.162:0/2266304651 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fba10064380 0x7fba10112ca0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fba00004930 tx=0x7fba00009da0 comp rx=0 tx=0).stop 2024-04-27T04:22:06.599 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:06.600+0000 7fba1635d640 1 -- 172.21.15.162:0/2266304651 shutdown_connections 2024-04-27T04:22:06.599 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:06.600+0000 7fba1635d640 1 --2- 172.21.15.162:0/2266304651 >> [v2:172.21.15.162:6800/1396866108,v1:172.21.15.162:6801/1396866108] conn(0x7fb9f806eb70 0x7fb9f8071020 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:22:06.599 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:06.600+0000 7fba1635d640 1 --2- 172.21.15.162:0/2266304651 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fba101021e0 0x7fba101131e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:22:06.599 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:06.600+0000 7fba1635d640 1 --2- 172.21.15.162:0/2266304651 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fba10064380 0x7fba10112ca0 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:22:06.599 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:06.600+0000 7fba1635d640 1 -- 172.21.15.162:0/2266304651 >> 172.21.15.162:0/2266304651 conn(0x7fba100710c0 msgr2=0x7fba100f9470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:22:06.599 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:06.600+0000 7fba1635d640 1 -- 172.21.15.162:0/2266304651 shutdown_connections 2024-04-27T04:22:06.599 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:06.600+0000 7fba1635d640 1 -- 172.21.15.162:0/2266304651 wait complete. 2024-04-27T04:22:06.609 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs.cephfs.meta pg_num 2024-04-27T04:22:06.708 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:06.707+0000 7ff94a354640 1 -- 172.21.15.162:0/1793203932 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff944102b30 msgr2=0x7ff9440fc1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:22:06.708 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:06.707+0000 7ff94a354640 1 --2- 172.21.15.162:0/1793203932 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff944102b30 0x7ff9440fc1c0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7ff9380048d0 tx=0x7ff938009da0 comp rx=0 tx=0).stop 2024-04-27T04:22:06.708 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:06.707+0000 7ff94a354640 1 -- 172.21.15.162:0/1793203932 shutdown_connections 2024-04-27T04:22:06.708 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:06.707+0000 7ff94a354640 1 --2- 172.21.15.162:0/1793203932 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff944102b30 0x7ff9440fc1c0 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:22:06.708 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:06.707+0000 7ff94a354640 1 --2- 172.21.15.162:0/1793203932 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff944102190 0x7ff944102560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:22:06.708 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:06.707+0000 7ff94a354640 1 -- 172.21.15.162:0/1793203932 >> 172.21.15.162:0/1793203932 conn(0x7ff9440f7da0 msgr2=0x7ff9440fa1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:22:06.708 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:06.707+0000 7ff94a354640 1 -- 172.21.15.162:0/1793203932 shutdown_connections 2024-04-27T04:22:06.708 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:06.707+0000 7ff94a354640 1 -- 172.21.15.162:0/1793203932 wait complete. 2024-04-27T04:22:06.709 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:06.708+0000 7ff94a354640 1 Processor -- start 2024-04-27T04:22:06.709 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:06.708+0000 7ff94a354640 1 -- start start 2024-04-27T04:22:06.709 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:06.708+0000 7ff94a354640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff944102190 0x7ff944112cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:22:06.709 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:06.708+0000 7ff94a354640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff944102b30 0x7ff944113200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:22:06.709 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:06.708+0000 7ff94a354640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff944113800 con 0x7ff944102190 2024-04-27T04:22:06.709 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:06.708+0000 7ff94a354640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff94410ce50 con 0x7ff944102b30 2024-04-27T04:22:06.709 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:06.708+0000 7ff943fff640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff944102190 0x7ff944112cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:22:06.709 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:06.708+0000 7ff943fff640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff944102190 0x7ff944112cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:56782/0 (socket says 172.21.15.162:56782) 2024-04-27T04:22:06.709 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:06.708+0000 7ff943fff640 1 -- 172.21.15.162:0/476281687 learned_addr learned my addr 172.21.15.162:0/476281687 (peer_addr_for_me v2:172.21.15.162:0/0) 2024-04-27T04:22:06.709 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:06.708+0000 7ff9437fe640 1 --2- 172.21.15.162:0/476281687 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff944102b30 0x7ff944113200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:22:06.709 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:06.708+0000 7ff943fff640 1 -- 172.21.15.162:0/476281687 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff944102b30 msgr2=0x7ff944113200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:22:06.709 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:06.708+0000 7ff943fff640 1 --2- 172.21.15.162:0/476281687 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff944102b30 0x7ff944113200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:22:06.709 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:06.708+0000 7ff943fff640 1 -- 172.21.15.162:0/476281687 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff938005040 con 0x7ff944102190 2024-04-27T04:22:06.709 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:06.709+0000 7ff943fff640 1 --2- 172.21.15.162:0/476281687 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff944102190 0x7ff944112cc0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7ff93400eb70 tx=0x7ff93400b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:22:06.709 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:06.709+0000 7ff940ff9640 1 -- 172.21.15.162:0/476281687 <== mon.0 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff934007c30 con 0x7ff944102190 2024-04-27T04:22:06.710 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:06.709+0000 7ff94a354640 1 -- 172.21.15.162:0/476281687 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff94410d0d0 con 0x7ff944102190 2024-04-27T04:22:06.710 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:06.709+0000 7ff940ff9640 1 -- 172.21.15.162:0/476281687 <== mon.0 v2:172.21.15.162:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7ff93400f040 con 0x7ff944102190 2024-04-27T04:22:06.710 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:06.709+0000 7ff940ff9640 1 -- 172.21.15.162:0/476281687 <== mon.0 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff934009680 con 0x7ff944102190 2024-04-27T04:22:06.710 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:06.709+0000 7ff94a354640 1 -- 172.21.15.162:0/476281687 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff94410d5c0 con 0x7ff944102190 2024-04-27T04:22:06.710 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:06.710+0000 7ff94a354640 1 -- 172.21.15.162:0/476281687 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff9440fd900 con 0x7ff944102190 2024-04-27T04:22:06.711 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:06.710+0000 7ff940ff9640 1 -- 172.21.15.162:0/476281687 <== mon.0 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 39) v1 ==== 92747+0+0 (secure 0 0 0) 0x7ff93400cd40 con 0x7ff944102190 2024-04-27T04:22:06.711 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:06.710+0000 7ff940ff9640 1 --2- 172.21.15.162:0/476281687 >> [v2:172.21.15.162:6800/1396866108,v1:172.21.15.162:6801/1396866108] conn(0x7ff92c06eb70 0x7ff92c071020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:22:06.711 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:06.711+0000 7ff940ff9640 1 -- 172.21.15.162:0/476281687 <== mon.0 v2:172.21.15.162:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6582+0+0 (secure 0 0 0) 0x7ff93408e5a0 con 0x7ff944102190 2024-04-27T04:22:06.711 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:06.711+0000 7ff9437fe640 1 --2- 172.21.15.162:0/476281687 >> [v2:172.21.15.162:6800/1396866108,v1:172.21.15.162:6801/1396866108] conn(0x7ff92c06eb70 0x7ff92c071020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:22:06.711 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:06.711+0000 7ff9437fe640 1 --2- 172.21.15.162:0/476281687 >> [v2:172.21.15.162:6800/1396866108,v1:172.21.15.162:6801/1396866108] conn(0x7ff92c06eb70 0x7ff92c071020 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7ff9440fa850 tx=0x7ff938009da0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:22:06.713 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:06.715+0000 7ff940ff9640 1 -- 172.21.15.162:0/476281687 <== mon.0 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7ff9340573a0 con 0x7ff944102190 2024-04-27T04:22:06.886 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:22:06 smithi162 ceph-mon[110641]: pgmap v272: 65 pgs: 65 active+clean; 208 MiB data, 851 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-04-27T04:22:06.886 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:22:06 smithi162 ceph-mon[110641]: from='client.? 172.21.15.162:0/4030251414' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-27T04:22:06.886 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:22:06 smithi162 ceph-mon[110641]: from='client.? 172.21.15.162:0/2266304651' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-27T04:22:06.894 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:06.896+0000 7ff94a354640 1 -- 172.21.15.162:0/476281687 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs.cephfs.meta", "var": "pg_num"} v 0) v1 -- 0x7ff9441092d0 con 0x7ff944102190 2024-04-27T04:22:06.894 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:06.896+0000 7ff940ff9640 1 -- 172.21.15.162:0/476281687 <== mon.0 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs.cephfs.meta", "var": "pg_num"}]=0 v84) v1 ==== 107+0+11 (secure 0 0 0) 0x7ff93405aa60 con 0x7ff944102190 2024-04-27T04:22:06.895 INFO:teuthology.orchestra.run.smithi162.stdout:pg_num: 32 2024-04-27T04:22:06.897 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:06.898+0000 7ff94a354640 1 -- 172.21.15.162:0/476281687 >> [v2:172.21.15.162:6800/1396866108,v1:172.21.15.162:6801/1396866108] conn(0x7ff92c06eb70 msgr2=0x7ff92c071020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:22:06.897 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:06.898+0000 7ff94a354640 1 --2- 172.21.15.162:0/476281687 >> [v2:172.21.15.162:6800/1396866108,v1:172.21.15.162:6801/1396866108] conn(0x7ff92c06eb70 0x7ff92c071020 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7ff9440fa850 tx=0x7ff938009da0 comp rx=0 tx=0).stop 2024-04-27T04:22:06.897 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:06.898+0000 7ff94a354640 1 -- 172.21.15.162:0/476281687 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff944102190 msgr2=0x7ff944112cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:22:06.897 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:06.898+0000 7ff94a354640 1 --2- 172.21.15.162:0/476281687 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff944102190 0x7ff944112cc0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7ff93400eb70 tx=0x7ff93400b040 comp rx=0 tx=0).stop 2024-04-27T04:22:06.898 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:06.898+0000 7ff94a354640 1 -- 172.21.15.162:0/476281687 shutdown_connections 2024-04-27T04:22:06.898 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:06.898+0000 7ff94a354640 1 --2- 172.21.15.162:0/476281687 >> [v2:172.21.15.162:6800/1396866108,v1:172.21.15.162:6801/1396866108] conn(0x7ff92c06eb70 0x7ff92c071020 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:22:06.898 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:06.898+0000 7ff94a354640 1 --2- 172.21.15.162:0/476281687 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff944102b30 0x7ff944113200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:22:06.898 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:06.898+0000 7ff94a354640 1 --2- 172.21.15.162:0/476281687 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff944102190 0x7ff944112cc0 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:22:06.898 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:06.898+0000 7ff94a354640 1 -- 172.21.15.162:0/476281687 >> 172.21.15.162:0/476281687 conn(0x7ff9440f7da0 msgr2=0x7ff9440f8cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:22:06.898 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:06.898+0000 7ff94a354640 1 -- 172.21.15.162:0/476281687 shutdown_connections 2024-04-27T04:22:06.898 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:06.898+0000 7ff94a354640 1 -- 172.21.15.162:0/476281687 wait complete. 2024-04-27T04:22:06.908 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs.cephfs.data pg_num 2024-04-27T04:22:07.007 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:07.006+0000 7f63ecf1d640 1 -- 172.21.15.162:0/3276438545 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f63e8102180 msgr2=0x7f63e8102550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:22:07.007 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:07.006+0000 7f63ecf1d640 1 --2- 172.21.15.162:0/3276438545 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f63e8102180 0x7f63e8102550 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f63d8009c50 tx=0x7f63d8004d10 comp rx=0 tx=0).stop 2024-04-27T04:22:07.008 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:07.006+0000 7f63ecf1d640 1 -- 172.21.15.162:0/3276438545 shutdown_connections 2024-04-27T04:22:07.008 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:07.006+0000 7f63ecf1d640 1 --2- 172.21.15.162:0/3276438545 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f63e8102b20 0x7f63e80fc1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:22:07.008 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:07.006+0000 7f63ecf1d640 1 --2- 172.21.15.162:0/3276438545 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f63e8102180 0x7f63e8102550 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:22:07.008 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:07.006+0000 7f63ecf1d640 1 -- 172.21.15.162:0/3276438545 >> 172.21.15.162:0/3276438545 conn(0x7f63e80f7db0 msgr2=0x7f63e80fa1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:22:07.008 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:07.007+0000 7f63ecf1d640 1 -- 172.21.15.162:0/3276438545 shutdown_connections 2024-04-27T04:22:07.008 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:07.007+0000 7f63ecf1d640 1 -- 172.21.15.162:0/3276438545 wait complete. 2024-04-27T04:22:07.008 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:07.007+0000 7f63ecf1d640 1 Processor -- start 2024-04-27T04:22:07.008 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:07.007+0000 7f63ecf1d640 1 -- start start 2024-04-27T04:22:07.008 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:07.007+0000 7f63ecf1d640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f63e8102180 0x7f63e8112d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:22:07.008 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:07.007+0000 7f63ecf1d640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f63e8102b20 0x7f63e8113270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:22:07.008 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:07.007+0000 7f63ecf1d640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63e8113900 con 0x7f63e8102180 2024-04-27T04:22:07.008 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:07.007+0000 7f63ecf1d640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63e810cec0 con 0x7f63e8102b20 2024-04-27T04:22:07.008 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:07.007+0000 7f63e659c640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f63e8102180 0x7f63e8112d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:22:07.009 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:07.007+0000 7f63e659c640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f63e8102180 0x7f63e8112d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:56806/0 (socket says 172.21.15.162:56806) 2024-04-27T04:22:07.009 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:07.007+0000 7f63e659c640 1 -- 172.21.15.162:0/4012522444 learned_addr learned my addr 172.21.15.162:0/4012522444 (peer_addr_for_me v2:172.21.15.162:0/0) 2024-04-27T04:22:07.009 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:07.008+0000 7f63e5d9b640 1 --2- 172.21.15.162:0/4012522444 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f63e8102b20 0x7f63e8113270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:22:07.009 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:07.008+0000 7f63e659c640 1 -- 172.21.15.162:0/4012522444 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f63e8102b20 msgr2=0x7f63e8113270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:22:07.009 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:07.008+0000 7f63e659c640 1 --2- 172.21.15.162:0/4012522444 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f63e8102b20 0x7f63e8113270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:22:07.009 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:07.008+0000 7f63e659c640 1 -- 172.21.15.162:0/4012522444 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f63d80098b0 con 0x7f63e8102180 2024-04-27T04:22:07.009 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:07.008+0000 7f63e659c640 1 --2- 172.21.15.162:0/4012522444 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f63e8102180 0x7f63e8112d30 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f63d8004a30 tx=0x7f63d8004d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:22:07.009 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:07.008+0000 7f63d6ffd640 1 -- 172.21.15.162:0/4012522444 <== mon.0 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f63d803a070 con 0x7f63e8102180 2024-04-27T04:22:07.009 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:07.008+0000 7f63ecf1d640 1 -- 172.21.15.162:0/4012522444 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f63e810d140 con 0x7f63e8102180 2024-04-27T04:22:07.009 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:07.008+0000 7f63d6ffd640 1 -- 172.21.15.162:0/4012522444 <== mon.0 v2:172.21.15.162:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f63d8035b20 con 0x7f63e8102180 2024-04-27T04:22:07.009 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:07.008+0000 7f63d6ffd640 1 -- 172.21.15.162:0/4012522444 <== mon.0 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f63d803eaa0 con 0x7f63e8102180 2024-04-27T04:22:07.009 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:07.008+0000 7f63ecf1d640 1 -- 172.21.15.162:0/4012522444 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f63e810d660 con 0x7f63e8102180 2024-04-27T04:22:07.009 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:07.009+0000 7f63ecf1d640 1 -- 172.21.15.162:0/4012522444 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f63e80fd8f0 con 0x7f63e8102180 2024-04-27T04:22:07.010 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:07.010+0000 7f63d6ffd640 1 -- 172.21.15.162:0/4012522444 <== mon.0 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 39) v1 ==== 92747+0+0 (secure 0 0 0) 0x7f63d8032070 con 0x7f63e8102180 2024-04-27T04:22:07.010 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:07.010+0000 7f63d6ffd640 1 --2- 172.21.15.162:0/4012522444 >> [v2:172.21.15.162:6800/1396866108,v1:172.21.15.162:6801/1396866108] conn(0x7f63d006eb70 0x7f63d0071020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:22:07.010 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:07.010+0000 7f63d6ffd640 1 -- 172.21.15.162:0/4012522444 <== mon.0 v2:172.21.15.162:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6582+0+0 (secure 0 0 0) 0x7f63d80b11b0 con 0x7f63e8102180 2024-04-27T04:22:07.011 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:07.010+0000 7f63e5d9b640 1 --2- 172.21.15.162:0/4012522444 >> [v2:172.21.15.162:6800/1396866108,v1:172.21.15.162:6801/1396866108] conn(0x7f63d006eb70 0x7f63d0071020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:22:07.011 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:07.010+0000 7f63e5d9b640 1 --2- 172.21.15.162:0/4012522444 >> [v2:172.21.15.162:6800/1396866108,v1:172.21.15.162:6801/1396866108] conn(0x7f63d006eb70 0x7f63d0071020 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f63dc006fd0 tx=0x7f63dc008040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:22:07.013 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:07.015+0000 7f63d6ffd640 1 -- 172.21.15.162:0/4012522444 <== mon.0 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7f63d8079f00 con 0x7f63e8102180 2024-04-27T04:22:07.172 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:22:06 smithi181 ceph-mon[81134]: pgmap v272: 65 pgs: 65 active+clean; 208 MiB data, 851 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-04-27T04:22:07.172 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:22:06 smithi181 ceph-mon[81134]: from='client.? 172.21.15.162:0/4030251414' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-27T04:22:07.172 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:22:06 smithi181 ceph-mon[81134]: from='client.? 172.21.15.162:0/2266304651' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-27T04:22:07.191 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:07.192+0000 7f63ecf1d640 1 -- 172.21.15.162:0/4012522444 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs.cephfs.data", "var": "pg_num"} v 0) v1 -- 0x7f63e810e430 con 0x7f63e8102180 2024-04-27T04:22:07.191 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:07.193+0000 7f63d6ffd640 1 -- 172.21.15.162:0/4012522444 <== mon.0 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs.cephfs.data", "var": "pg_num"}]=0 v84) v1 ==== 107+0+11 (secure 0 0 0) 0x7f63d807d5c0 con 0x7f63e8102180 2024-04-27T04:22:07.191 INFO:teuthology.orchestra.run.smithi162.stdout:pg_num: 32 2024-04-27T04:22:07.194 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:07.194+0000 7f63ecf1d640 1 -- 172.21.15.162:0/4012522444 >> [v2:172.21.15.162:6800/1396866108,v1:172.21.15.162:6801/1396866108] conn(0x7f63d006eb70 msgr2=0x7f63d0071020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:22:07.194 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:07.194+0000 7f63ecf1d640 1 --2- 172.21.15.162:0/4012522444 >> [v2:172.21.15.162:6800/1396866108,v1:172.21.15.162:6801/1396866108] conn(0x7f63d006eb70 0x7f63d0071020 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f63dc006fd0 tx=0x7f63dc008040 comp rx=0 tx=0).stop 2024-04-27T04:22:07.194 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:07.194+0000 7f63ecf1d640 1 -- 172.21.15.162:0/4012522444 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f63e8102180 msgr2=0x7f63e8112d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:22:07.194 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:07.194+0000 7f63ecf1d640 1 --2- 172.21.15.162:0/4012522444 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f63e8102180 0x7f63e8112d30 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f63d8004a30 tx=0x7f63d8004d10 comp rx=0 tx=0).stop 2024-04-27T04:22:07.194 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:07.194+0000 7f63ecf1d640 1 -- 172.21.15.162:0/4012522444 shutdown_connections 2024-04-27T04:22:07.194 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:07.194+0000 7f63ecf1d640 1 --2- 172.21.15.162:0/4012522444 >> [v2:172.21.15.162:6800/1396866108,v1:172.21.15.162:6801/1396866108] conn(0x7f63d006eb70 0x7f63d0071020 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:22:07.194 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:07.195+0000 7f63ecf1d640 1 --2- 172.21.15.162:0/4012522444 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f63e8102b20 0x7f63e8113270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:22:07.194 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:07.195+0000 7f63ecf1d640 1 --2- 172.21.15.162:0/4012522444 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f63e8102180 0x7f63e8112d30 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:22:07.194 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:07.195+0000 7f63ecf1d640 1 -- 172.21.15.162:0/4012522444 >> 172.21.15.162:0/4012522444 conn(0x7f63e80f7db0 msgr2=0x7f63e80f8e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:22:07.194 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:07.195+0000 7f63ecf1d640 1 -- 172.21.15.162:0/4012522444 shutdown_connections 2024-04-27T04:22:07.195 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:07.195+0000 7f63ecf1d640 1 -- 172.21.15.162:0/4012522444 wait complete. 2024-04-27T04:22:07.204 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-27T04:22:07.298 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:07.299+0000 7fc29962e640 1 -- 172.21.15.162:0/2119808676 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc294102b90 msgr2=0x7fc294064350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:22:07.298 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:07.299+0000 7fc29962e640 1 --2- 172.21.15.162:0/2119808676 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc294102b90 0x7fc294064350 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fc2880048d0 tx=0x7fc288009da0 comp rx=0 tx=0).stop 2024-04-27T04:22:07.299 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:07.299+0000 7fc29962e640 1 -- 172.21.15.162:0/2119808676 shutdown_connections 2024-04-27T04:22:07.299 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:07.299+0000 7fc29962e640 1 --2- 172.21.15.162:0/2119808676 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc294102b90 0x7fc294064350 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:22:07.299 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:07.299+0000 7fc29962e640 1 --2- 172.21.15.162:0/2119808676 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc2941021f0 0x7fc2941025c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:22:07.299 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:07.299+0000 7fc29962e640 1 -- 172.21.15.162:0/2119808676 >> 172.21.15.162:0/2119808676 conn(0x7fc2940711c0 msgr2=0x7fc2940715c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:22:07.299 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:07.299+0000 7fc29962e640 1 -- 172.21.15.162:0/2119808676 shutdown_connections 2024-04-27T04:22:07.299 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:07.299+0000 7fc29962e640 1 -- 172.21.15.162:0/2119808676 wait complete. 2024-04-27T04:22:07.299 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:07.300+0000 7fc29962e640 1 Processor -- start 2024-04-27T04:22:07.299 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:07.300+0000 7fc29962e640 1 -- start start 2024-04-27T04:22:07.301 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:07.300+0000 7fc29962e640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc2941021f0 0x7fc294112d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:22:07.301 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:07.300+0000 7fc29962e640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc294102b90 0x7fc294113270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:22:07.301 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:07.300+0000 7fc29962e640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc294113870 con 0x7fc2941021f0 2024-04-27T04:22:07.301 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:07.300+0000 7fc29962e640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc29410cec0 con 0x7fc294102b90 2024-04-27T04:22:07.301 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:07.300+0000 7fc292ffd640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc2941021f0 0x7fc294112d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:22:07.301 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:07.301+0000 7fc292ffd640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc2941021f0 0x7fc294112d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:56826/0 (socket says 172.21.15.162:56826) 2024-04-27T04:22:07.301 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:07.301+0000 7fc292ffd640 1 -- 172.21.15.162:0/2334690252 learned_addr learned my addr 172.21.15.162:0/2334690252 (peer_addr_for_me v2:172.21.15.162:0/0) 2024-04-27T04:22:07.301 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:07.301+0000 7fc2927fc640 1 --2- 172.21.15.162:0/2334690252 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc294102b90 0x7fc294113270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:22:07.302 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:07.301+0000 7fc292ffd640 1 -- 172.21.15.162:0/2334690252 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc294102b90 msgr2=0x7fc294113270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:22:07.302 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:07.301+0000 7fc292ffd640 1 --2- 172.21.15.162:0/2334690252 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc294102b90 0x7fc294113270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:22:07.302 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:07.301+0000 7fc292ffd640 1 -- 172.21.15.162:0/2334690252 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc288005040 con 0x7fc2941021f0 2024-04-27T04:22:07.302 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:07.301+0000 7fc292ffd640 1 --2- 172.21.15.162:0/2334690252 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc2941021f0 0x7fc294112d30 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fc28c00eb70 tx=0x7fc28c00b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:22:07.302 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:07.301+0000 7fc27b7fe640 1 -- 172.21.15.162:0/2334690252 <== mon.0 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc28c007c30 con 0x7fc2941021f0 2024-04-27T04:22:07.302 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:07.301+0000 7fc29962e640 1 -- 172.21.15.162:0/2334690252 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc29410d140 con 0x7fc2941021f0 2024-04-27T04:22:07.302 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:07.301+0000 7fc27b7fe640 1 -- 172.21.15.162:0/2334690252 <== mon.0 v2:172.21.15.162:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fc28c00f040 con 0x7fc2941021f0 2024-04-27T04:22:07.302 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:07.301+0000 7fc27b7fe640 1 -- 172.21.15.162:0/2334690252 <== mon.0 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc28c0095b0 con 0x7fc2941021f0 2024-04-27T04:22:07.302 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:07.301+0000 7fc29962e640 1 -- 172.21.15.162:0/2334690252 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc29410d630 con 0x7fc2941021f0 2024-04-27T04:22:07.302 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:07.302+0000 7fc29962e640 1 -- 172.21.15.162:0/2334690252 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc2940fd990 con 0x7fc2941021f0 2024-04-27T04:22:07.302 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:07.303+0000 7fc27b7fe640 1 -- 172.21.15.162:0/2334690252 <== mon.0 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 39) v1 ==== 92747+0+0 (secure 0 0 0) 0x7fc28c009730 con 0x7fc2941021f0 2024-04-27T04:22:07.303 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:07.303+0000 7fc27b7fe640 1 --2- 172.21.15.162:0/2334690252 >> [v2:172.21.15.162:6800/1396866108,v1:172.21.15.162:6801/1396866108] conn(0x7fc27c06eb70 0x7fc27c071020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:22:07.303 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:07.303+0000 7fc27b7fe640 1 -- 172.21.15.162:0/2334690252 <== mon.0 v2:172.21.15.162:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6582+0+0 (secure 0 0 0) 0x7fc28c08e650 con 0x7fc2941021f0 2024-04-27T04:22:07.303 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:07.303+0000 7fc2927fc640 1 --2- 172.21.15.162:0/2334690252 >> [v2:172.21.15.162:6800/1396866108,v1:172.21.15.162:6801/1396866108] conn(0x7fc27c06eb70 0x7fc27c071020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:22:07.303 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:07.303+0000 7fc2927fc640 1 --2- 172.21.15.162:0/2334690252 >> [v2:172.21.15.162:6800/1396866108,v1:172.21.15.162:6801/1396866108] conn(0x7fc27c06eb70 0x7fc27c071020 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fc2880048a0 tx=0x7fc288009da0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:22:07.306 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:07.308+0000 7fc27b7fe640 1 -- 172.21.15.162:0/2334690252 <== mon.0 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7fc28c0573a0 con 0x7fc2941021f0 2024-04-27T04:22:07.518 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:07.520+0000 7fc29962e640 1 -- 172.21.15.162:0/2334690252 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc294109370 con 0x7fc2941021f0 2024-04-27T04:22:07.519 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:07.520+0000 7fc27b7fe640 1 -- 172.21.15.162:0/2334690252 <== mon.0 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 35 v35) v1 ==== 94+0+5224 (secure 0 0 0) 0x7fc28c05aa60 con 0x7fc2941021f0 2024-04-27T04:22:07.519 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-04-27T04:22:07.519 INFO:teuthology.orchestra.run.smithi162.stdout:{"epoch":35,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":44319,"name":"cephfs.smithi181.dzomgh","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6825/1667773112","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":1667773112},{"type":"v1","addr":"172.21.15.181:6825","nonce":1667773112}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":27},{"gid":44323,"name":"cephfs.smithi181.bkkcdm","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6827/3589158285","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6826","nonce":3589158285},{"type":"v1","addr":"172.21.15.181:6827","nonce":3589158285}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":30}],"filesystems":[{"mdsmap":{"epoch":35,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-27T04:06:53.095749+0000","modified":"2024-04-27T04:20:08.657609+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":84,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":34380},"failed":[],"damaged":[],"stopped":[],"info":{"gid_34380":{"gid":34380,"name":"cephfs.smithi162.gzlyjj","rank":0,"incarnation":29,"state":"up:active","state_seq":20,"addr":"172.21.15.162:6827/1198130667","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6826","nonce":1198130667},{"type":"v1","addr":"172.21.15.162:6827","nonce":1198130667}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34412":{"gid":34412,"name":"cephfs.smithi162.xcnapk","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.162:6829/307528848","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6828","nonce":307528848},{"type":"v1","addr":"172.21.15.162:6829","nonce":307528848}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":34380,"qdb_cluster":[34380]},"id":1}]} 2024-04-27T04:22:07.520 INFO:teuthology.orchestra.run.smithi162.stderr:dumped fsmap epoch 35 2024-04-27T04:22:07.521 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:07.522+0000 7fc29962e640 1 -- 172.21.15.162:0/2334690252 >> [v2:172.21.15.162:6800/1396866108,v1:172.21.15.162:6801/1396866108] conn(0x7fc27c06eb70 msgr2=0x7fc27c071020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:22:07.521 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:07.522+0000 7fc29962e640 1 --2- 172.21.15.162:0/2334690252 >> [v2:172.21.15.162:6800/1396866108,v1:172.21.15.162:6801/1396866108] conn(0x7fc27c06eb70 0x7fc27c071020 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fc2880048a0 tx=0x7fc288009da0 comp rx=0 tx=0).stop 2024-04-27T04:22:07.521 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:07.522+0000 7fc29962e640 1 -- 172.21.15.162:0/2334690252 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc2941021f0 msgr2=0x7fc294112d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:22:07.522 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:07.522+0000 7fc29962e640 1 --2- 172.21.15.162:0/2334690252 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc2941021f0 0x7fc294112d30 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fc28c00eb70 tx=0x7fc28c00b040 comp rx=0 tx=0).stop 2024-04-27T04:22:07.522 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:07.522+0000 7fc29962e640 1 -- 172.21.15.162:0/2334690252 shutdown_connections 2024-04-27T04:22:07.522 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:07.522+0000 7fc29962e640 1 --2- 172.21.15.162:0/2334690252 >> [v2:172.21.15.162:6800/1396866108,v1:172.21.15.162:6801/1396866108] conn(0x7fc27c06eb70 0x7fc27c071020 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:22:07.522 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:07.522+0000 7fc29962e640 1 --2- 172.21.15.162:0/2334690252 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc294102b90 0x7fc294113270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:22:07.522 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:07.522+0000 7fc29962e640 1 --2- 172.21.15.162:0/2334690252 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc2941021f0 0x7fc294112d30 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:22:07.522 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:07.522+0000 7fc29962e640 1 -- 172.21.15.162:0/2334690252 >> 172.21.15.162:0/2334690252 conn(0x7fc2940711c0 msgr2=0x7fc2940fa570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:22:07.522 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:07.523+0000 7fc29962e640 1 -- 172.21.15.162:0/2334690252 shutdown_connections 2024-04-27T04:22:07.522 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:07.523+0000 7fc29962e640 1 -- 172.21.15.162:0/2334690252 wait complete. 2024-04-27T04:22:07.532 DEBUG:tasks.fs:checking fs fscid=1,name=cephfs state = {'epoch': 15, 'max_mds': 1, 'flags': 50} 2024-04-27T04:22:07.532 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 16 2024-04-27T04:22:07.632 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:07.629+0000 7fea5c22e640 1 -- 172.21.15.162:0/683945029 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fea54102230 msgr2=0x7fea54102600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:22:07.632 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:07.629+0000 7fea5c22e640 1 --2- 172.21.15.162:0/683945029 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fea54102230 0x7fea54102600 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fea440048d0 tx=0x7fea44009da0 comp rx=0 tx=0).stop 2024-04-27T04:22:07.632 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:07.629+0000 7fea5c22e640 1 -- 172.21.15.162:0/683945029 shutdown_connections 2024-04-27T04:22:07.633 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:07.629+0000 7fea5c22e640 1 --2- 172.21.15.162:0/683945029 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fea54102bd0 0x7fea540fa8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:22:07.633 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:07.629+0000 7fea5c22e640 1 --2- 172.21.15.162:0/683945029 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fea54102230 0x7fea54102600 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:22:07.633 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:07.629+0000 7fea5c22e640 1 -- 172.21.15.162:0/683945029 >> 172.21.15.162:0/683945029 conn(0x7fea540717e0 msgr2=0x7fea54071be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:22:07.633 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:07.629+0000 7fea5c22e640 1 -- 172.21.15.162:0/683945029 shutdown_connections 2024-04-27T04:22:07.633 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:07.629+0000 7fea5c22e640 1 -- 172.21.15.162:0/683945029 wait complete. 2024-04-27T04:22:07.633 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:07.630+0000 7fea5c22e640 1 Processor -- start 2024-04-27T04:22:07.633 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:07.630+0000 7fea5c22e640 1 -- start start 2024-04-27T04:22:07.633 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:07.630+0000 7fea5c22e640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fea54102230 0x7fea54112d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:22:07.634 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:07.630+0000 7fea5c22e640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fea54102bd0 0x7fea54113270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:22:07.634 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:07.630+0000 7fea5c22e640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea54113870 con 0x7fea54102230 2024-04-27T04:22:07.634 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:07.630+0000 7fea5c22e640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea5410cec0 con 0x7fea54102bd0 2024-04-27T04:22:07.634 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:07.630+0000 7fea59fca640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fea54102230 0x7fea54112d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:22:07.634 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:07.630+0000 7fea59fca640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fea54102230 0x7fea54112d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:56850/0 (socket says 172.21.15.162:56850) 2024-04-27T04:22:07.634 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:07.630+0000 7fea59fca640 1 -- 172.21.15.162:0/3103507137 learned_addr learned my addr 172.21.15.162:0/3103507137 (peer_addr_for_me v2:172.21.15.162:0/0) 2024-04-27T04:22:07.634 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:07.630+0000 7fea597c9640 1 --2- 172.21.15.162:0/3103507137 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fea54102bd0 0x7fea54113270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:22:07.634 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:07.630+0000 7fea59fca640 1 -- 172.21.15.162:0/3103507137 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fea54102bd0 msgr2=0x7fea54113270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:22:07.635 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:07.630+0000 7fea59fca640 1 --2- 172.21.15.162:0/3103507137 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fea54102bd0 0x7fea54113270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:22:07.635 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:07.630+0000 7fea59fca640 1 -- 172.21.15.162:0/3103507137 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea44005040 con 0x7fea54102230 2024-04-27T04:22:07.635 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:07.630+0000 7fea597c9640 1 --2- 172.21.15.162:0/3103507137 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fea54102bd0 0x7fea54113270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-27T04:22:07.635 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:07.630+0000 7fea59fca640 1 --2- 172.21.15.162:0/3103507137 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fea54102230 0x7fea54112d30 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fea44004930 tx=0x7fea44009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:22:07.635 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:07.631+0000 7fea4a7fc640 1 -- 172.21.15.162:0/3103507137 <== mon.0 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fea440094c0 con 0x7fea54102230 2024-04-27T04:22:07.635 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:07.631+0000 7fea4a7fc640 1 -- 172.21.15.162:0/3103507137 <== mon.0 v2:172.21.15.162:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fea44032e50 con 0x7fea54102230 2024-04-27T04:22:07.635 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:07.631+0000 7fea5c22e640 1 -- 172.21.15.162:0/3103507137 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fea5410d140 con 0x7fea54102230 2024-04-27T04:22:07.635 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:07.631+0000 7fea4a7fc640 1 -- 172.21.15.162:0/3103507137 <== mon.0 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fea440433b0 con 0x7fea54102230 2024-04-27T04:22:07.635 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:07.631+0000 7fea5c22e640 1 -- 172.21.15.162:0/3103507137 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fea5410d600 con 0x7fea54102230 2024-04-27T04:22:07.636 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:07.632+0000 7fea5c22e640 1 -- 172.21.15.162:0/3103507137 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fea54064350 con 0x7fea54102230 2024-04-27T04:22:07.636 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:07.632+0000 7fea4a7fc640 1 -- 172.21.15.162:0/3103507137 <== mon.0 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 39) v1 ==== 92747+0+0 (secure 0 0 0) 0x7fea4402f070 con 0x7fea54102230 2024-04-27T04:22:07.636 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:07.632+0000 7fea4a7fc640 1 --2- 172.21.15.162:0/3103507137 >> [v2:172.21.15.162:6800/1396866108,v1:172.21.15.162:6801/1396866108] conn(0x7fea4006eb70 0x7fea40071020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:22:07.636 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:07.633+0000 7fea597c9640 1 --2- 172.21.15.162:0/3103507137 >> [v2:172.21.15.162:6800/1396866108,v1:172.21.15.162:6801/1396866108] conn(0x7fea4006eb70 0x7fea40071020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:22:07.636 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:07.633+0000 7fea4a7fc640 1 -- 172.21.15.162:0/3103507137 <== mon.0 v2:172.21.15.162:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6582+0+0 (secure 0 0 0) 0x7fea440b0650 con 0x7fea54102230 2024-04-27T04:22:07.636 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:07.633+0000 7fea597c9640 1 --2- 172.21.15.162:0/3103507137 >> [v2:172.21.15.162:6800/1396866108,v1:172.21.15.162:6801/1396866108] conn(0x7fea4006eb70 0x7fea40071020 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fea50007fd0 tx=0x7fea50009e90 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:22:07.637 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:07.637+0000 7fea4a7fc640 1 -- 172.21.15.162:0/3103507137 <== mon.0 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7fea440793a0 con 0x7fea54102230 2024-04-27T04:22:07.852 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:07.853+0000 7fea5c22e640 1 -- 172.21.15.162:0/3103507137 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 16, "format": "json"} v 0) v1 -- 0x7fea5410e3a0 con 0x7fea54102230 2024-04-27T04:22:07.852 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:07.854+0000 7fea4a7fc640 1 -- 172.21.15.162:0/3103507137 <== mon.0 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 16, "format": "json"}]=0 dumped fsmap epoch 16 v35) v1 ==== 107+0+4094 (secure 0 0 0) 0x7fea4407ca60 con 0x7fea54102230 2024-04-27T04:22:07.853 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-04-27T04:22:07.853 INFO:teuthology.orchestra.run.smithi162.stdout:{"epoch":16,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14488,"name":"cephfs.smithi162.gzlyjj","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6827/2629103894","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6826","nonce":2629103894},{"type":"v1","addr":"172.21.15.162:6827","nonce":2629103894}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":13},{"gid":24287,"name":"cephfs.smithi181.bkkcdm","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.181:6827/3756475453","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6826","nonce":3756475453},{"type":"v1","addr":"172.21.15.181:6827","nonce":3756475453}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":14}],"filesystems":[{"mdsmap":{"epoch":16,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-27T04:06:53.095749+0000","modified":"2024-04-27T04:18:44.511964+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":24273},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24273":{"gid":24273,"name":"cephfs.smithi181.dzomgh","rank":0,"incarnation":12,"state":"up:active","state_seq":6,"addr":"172.21.15.181:6825/658214993","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":658214993},{"type":"v1","addr":"172.21.15.181:6825","nonce":658214993}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":24273,"qdb_cluster":[24273]},"id":1}]} 2024-04-27T04:22:07.853 INFO:teuthology.orchestra.run.smithi162.stderr:dumped fsmap epoch 16 2024-04-27T04:22:07.855 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:07.856+0000 7fea5c22e640 1 -- 172.21.15.162:0/3103507137 >> [v2:172.21.15.162:6800/1396866108,v1:172.21.15.162:6801/1396866108] conn(0x7fea4006eb70 msgr2=0x7fea40071020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:22:07.855 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:07.856+0000 7fea5c22e640 1 --2- 172.21.15.162:0/3103507137 >> [v2:172.21.15.162:6800/1396866108,v1:172.21.15.162:6801/1396866108] conn(0x7fea4006eb70 0x7fea40071020 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fea50007fd0 tx=0x7fea50009e90 comp rx=0 tx=0).stop 2024-04-27T04:22:07.855 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:07.856+0000 7fea5c22e640 1 -- 172.21.15.162:0/3103507137 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fea54102230 msgr2=0x7fea54112d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:22:07.855 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:07.856+0000 7fea5c22e640 1 --2- 172.21.15.162:0/3103507137 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fea54102230 0x7fea54112d30 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fea44004930 tx=0x7fea44009da0 comp rx=0 tx=0).stop 2024-04-27T04:22:07.855 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:07.856+0000 7fea5c22e640 1 -- 172.21.15.162:0/3103507137 shutdown_connections 2024-04-27T04:22:07.855 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:07.856+0000 7fea5c22e640 1 --2- 172.21.15.162:0/3103507137 >> [v2:172.21.15.162:6800/1396866108,v1:172.21.15.162:6801/1396866108] conn(0x7fea4006eb70 0x7fea40071020 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:22:07.855 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:07.856+0000 7fea5c22e640 1 --2- 172.21.15.162:0/3103507137 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fea54102bd0 0x7fea54113270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:22:07.855 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:07.856+0000 7fea5c22e640 1 --2- 172.21.15.162:0/3103507137 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fea54102230 0x7fea54112d30 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:22:07.855 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:07.856+0000 7fea5c22e640 1 -- 172.21.15.162:0/3103507137 >> 172.21.15.162:0/3103507137 conn(0x7fea540717e0 msgr2=0x7fea540fa100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:22:07.855 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:07.856+0000 7fea5c22e640 1 -- 172.21.15.162:0/3103507137 shutdown_connections 2024-04-27T04:22:07.856 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:07.856+0000 7fea5c22e640 1 -- 172.21.15.162:0/3103507137 wait complete. 2024-04-27T04:22:07.866 DEBUG:tasks.fs:allow_standby_replay disabled in epoch 16 2024-04-27T04:22:07.866 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 17 2024-04-27T04:22:07.965 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:07.964+0000 7fe921d9d640 1 -- 172.21.15.162:0/561345047 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe91c101070 msgr2=0x7fe91c101440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:22:07.965 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:07.964+0000 7fe921d9d640 1 --2- 172.21.15.162:0/561345047 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe91c101070 0x7fe91c101440 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fe90c0048d0 tx=0x7fe90c009da0 comp rx=0 tx=0).stop 2024-04-27T04:22:07.965 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:07.964+0000 7fe921d9d640 1 -- 172.21.15.162:0/561345047 shutdown_connections 2024-04-27T04:22:07.965 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:07.964+0000 7fe921d9d640 1 --2- 172.21.15.162:0/561345047 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe91c101a10 0x7fe91c0fb0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:22:07.965 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:07.964+0000 7fe921d9d640 1 --2- 172.21.15.162:0/561345047 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe91c101070 0x7fe91c101440 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:22:07.965 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:07.964+0000 7fe921d9d640 1 -- 172.21.15.162:0/561345047 >> 172.21.15.162:0/561345047 conn(0x7fe91c0f6be0 msgr2=0x7fe91c0f8ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:22:07.965 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:07.964+0000 7fe921d9d640 1 -- 172.21.15.162:0/561345047 shutdown_connections 2024-04-27T04:22:07.965 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:07.964+0000 7fe921d9d640 1 -- 172.21.15.162:0/561345047 wait complete. 2024-04-27T04:22:07.966 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:07.965+0000 7fe921d9d640 1 Processor -- start 2024-04-27T04:22:07.966 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:07.965+0000 7fe921d9d640 1 -- start start 2024-04-27T04:22:07.966 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:07.965+0000 7fe921d9d640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe91c101070 0x7fe91c10de50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:22:07.966 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:07.965+0000 7fe921d9d640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe91c101a10 0x7fe91c10e390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:22:07.966 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:07.965+0000 7fe921d9d640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe91c066740 con 0x7fe91c101a10 2024-04-27T04:22:07.966 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:07.965+0000 7fe921d9d640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe91c10e900 con 0x7fe91c101070 2024-04-27T04:22:07.966 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:07.965+0000 7fe91affd640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe91c101a10 0x7fe91c10e390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:22:07.966 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:07.965+0000 7fe91affd640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe91c101a10 0x7fe91c10e390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:56858/0 (socket says 172.21.15.162:56858) 2024-04-27T04:22:07.966 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:07.965+0000 7fe91affd640 1 -- 172.21.15.162:0/1170309552 learned_addr learned my addr 172.21.15.162:0/1170309552 (peer_addr_for_me v2:172.21.15.162:0/0) 2024-04-27T04:22:07.966 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:07.965+0000 7fe91b7fe640 1 --2- 172.21.15.162:0/1170309552 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe91c101070 0x7fe91c10de50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:22:07.966 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:07.965+0000 7fe91affd640 1 -- 172.21.15.162:0/1170309552 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe91c101070 msgr2=0x7fe91c10de50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:22:07.966 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:07.965+0000 7fe91affd640 1 --2- 172.21.15.162:0/1170309552 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe91c101070 0x7fe91c10de50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:22:07.966 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:07.965+0000 7fe91affd640 1 -- 172.21.15.162:0/1170309552 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe90c005040 con 0x7fe91c101a10 2024-04-27T04:22:07.967 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:07.965+0000 7fe91affd640 1 --2- 172.21.15.162:0/1170309552 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe91c101a10 0x7fe91c10e390 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fe910002a50 tx=0x7fe910004600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:22:07.967 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:07.966+0000 7fe903fff640 1 -- 172.21.15.162:0/1170309552 <== mon.0 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe910007c30 con 0x7fe91c101a10 2024-04-27T04:22:07.967 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:07.966+0000 7fe903fff640 1 -- 172.21.15.162:0/1170309552 <== mon.0 v2:172.21.15.162:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fe91000f040 con 0x7fe91c101a10 2024-04-27T04:22:07.967 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:07.966+0000 7fe921d9d640 1 -- 172.21.15.162:0/1170309552 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe91c10ebe0 con 0x7fe91c101a10 2024-04-27T04:22:07.967 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:07.966+0000 7fe903fff640 1 -- 172.21.15.162:0/1170309552 <== mon.0 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe91000db50 con 0x7fe91c101a10 2024-04-27T04:22:07.967 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:07.966+0000 7fe921d9d640 1 -- 172.21.15.162:0/1170309552 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe91c117170 con 0x7fe91c101a10 2024-04-27T04:22:07.967 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:07.967+0000 7fe921d9d640 1 -- 172.21.15.162:0/1170309552 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe91c0fc7e0 con 0x7fe91c101a10 2024-04-27T04:22:07.968 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:07.967+0000 7fe903fff640 1 -- 172.21.15.162:0/1170309552 <== mon.0 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 39) v1 ==== 92747+0+0 (secure 0 0 0) 0x7fe91001c400 con 0x7fe91c101a10 2024-04-27T04:22:07.968 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:07.968+0000 7fe903fff640 1 --2- 172.21.15.162:0/1170309552 >> [v2:172.21.15.162:6800/1396866108,v1:172.21.15.162:6801/1396866108] conn(0x7fe90406eb70 0x7fe904071020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:22:07.968 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:07.968+0000 7fe91b7fe640 1 --2- 172.21.15.162:0/1170309552 >> [v2:172.21.15.162:6800/1396866108,v1:172.21.15.162:6801/1396866108] conn(0x7fe90406eb70 0x7fe904071020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:22:07.968 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:07.968+0000 7fe903fff640 1 -- 172.21.15.162:0/1170309552 <== mon.0 v2:172.21.15.162:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6582+0+0 (secure 0 0 0) 0x7fe91008e140 con 0x7fe91c101a10 2024-04-27T04:22:07.968 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:07.968+0000 7fe91b7fe640 1 --2- 172.21.15.162:0/1170309552 >> [v2:172.21.15.162:6800/1396866108,v1:172.21.15.162:6801/1396866108] conn(0x7fe90406eb70 0x7fe904071020 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fe90c003080 tx=0x7fe90c009da0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:22:07.971 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:07.972+0000 7fe903fff640 1 -- 172.21.15.162:0/1170309552 <== mon.0 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7fe910056f40 con 0x7fe91c101a10 2024-04-27T04:22:08.136 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:22:07 smithi162 ceph-mon[110641]: from='client.? 172.21.15.162:0/476281687' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs.cephfs.meta", "var": "pg_num"}]: dispatch 2024-04-27T04:22:08.136 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:22:07 smithi162 ceph-mon[110641]: from='client.? 172.21.15.162:0/4012522444' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs.cephfs.data", "var": "pg_num"}]: dispatch 2024-04-27T04:22:08.136 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:22:07 smithi162 ceph-mon[110641]: from='client.? 172.21.15.162:0/2334690252' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-27T04:22:08.171 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:22:07 smithi181 ceph-mon[81134]: from='client.? 172.21.15.162:0/476281687' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs.cephfs.meta", "var": "pg_num"}]: dispatch 2024-04-27T04:22:08.171 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:22:07 smithi181 ceph-mon[81134]: from='client.? 172.21.15.162:0/4012522444' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs.cephfs.data", "var": "pg_num"}]: dispatch 2024-04-27T04:22:08.172 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:22:07 smithi181 ceph-mon[81134]: from='client.? 172.21.15.162:0/2334690252' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-27T04:22:08.185 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:08.187+0000 7fe921d9d640 1 -- 172.21.15.162:0/1170309552 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 17, "format": "json"} v 0) v1 -- 0x7fe91c065d40 con 0x7fe91c101a10 2024-04-27T04:22:08.186 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:08.188+0000 7fe903fff640 1 -- 172.21.15.162:0/1170309552 <== mon.0 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 17, "format": "json"}]=0 dumped fsmap epoch 17 v35) v1 ==== 107+0+4876 (secure 0 0 0) 0x7fe91005a600 con 0x7fe91c101a10 2024-04-27T04:22:08.189 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-04-27T04:22:08.189 INFO:teuthology.orchestra.run.smithi162.stdout:{"epoch":17,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14488,"name":"cephfs.smithi162.gzlyjj","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6827/2629103894","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6826","nonce":2629103894},{"type":"v1","addr":"172.21.15.162:6827","nonce":2629103894}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":13},{"gid":24287,"name":"cephfs.smithi181.bkkcdm","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.181:6827/3756475453","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6826","nonce":3756475453},{"type":"v1","addr":"172.21.15.181:6827","nonce":3756475453}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":14},{"gid":34376,"name":"cephfs.smithi162.xcnapk","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6829/1115643134","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6828","nonce":1115643134},{"type":"v1","addr":"172.21.15.162:6829","nonce":1115643134}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":17}],"filesystems":[{"mdsmap":{"epoch":16,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-27T04:06:53.095749+0000","modified":"2024-04-27T04:18:44.511964+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":24273},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24273":{"gid":24273,"name":"cephfs.smithi181.dzomgh","rank":0,"incarnation":12,"state":"up:active","state_seq":6,"addr":"172.21.15.181:6825/658214993","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":658214993},{"type":"v1","addr":"172.21.15.181:6825","nonce":658214993}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":24273,"qdb_cluster":[24273]},"id":1}]} 2024-04-27T04:22:08.190 INFO:teuthology.orchestra.run.smithi162.stderr:dumped fsmap epoch 17 2024-04-27T04:22:08.190 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:08.190+0000 7fe921d9d640 1 -- 172.21.15.162:0/1170309552 >> [v2:172.21.15.162:6800/1396866108,v1:172.21.15.162:6801/1396866108] conn(0x7fe90406eb70 msgr2=0x7fe904071020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:22:08.190 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:08.190+0000 7fe921d9d640 1 --2- 172.21.15.162:0/1170309552 >> [v2:172.21.15.162:6800/1396866108,v1:172.21.15.162:6801/1396866108] conn(0x7fe90406eb70 0x7fe904071020 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fe90c003080 tx=0x7fe90c009da0 comp rx=0 tx=0).stop 2024-04-27T04:22:08.191 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:08.190+0000 7fe921d9d640 1 -- 172.21.15.162:0/1170309552 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe91c101a10 msgr2=0x7fe91c10e390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:22:08.191 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:08.190+0000 7fe921d9d640 1 --2- 172.21.15.162:0/1170309552 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe91c101a10 0x7fe91c10e390 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fe910002a50 tx=0x7fe910004600 comp rx=0 tx=0).stop 2024-04-27T04:22:08.191 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:08.190+0000 7fe921d9d640 1 -- 172.21.15.162:0/1170309552 shutdown_connections 2024-04-27T04:22:08.191 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:08.190+0000 7fe921d9d640 1 --2- 172.21.15.162:0/1170309552 >> [v2:172.21.15.162:6800/1396866108,v1:172.21.15.162:6801/1396866108] conn(0x7fe90406eb70 0x7fe904071020 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:22:08.191 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:08.190+0000 7fe921d9d640 1 --2- 172.21.15.162:0/1170309552 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe91c101a10 0x7fe91c10e390 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:22:08.191 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:08.190+0000 7fe921d9d640 1 --2- 172.21.15.162:0/1170309552 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe91c101070 0x7fe91c10de50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:22:08.191 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:08.190+0000 7fe921d9d640 1 -- 172.21.15.162:0/1170309552 >> 172.21.15.162:0/1170309552 conn(0x7fe91c0f6be0 msgr2=0x7fe91c0f7aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:22:08.191 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:08.190+0000 7fe921d9d640 1 -- 172.21.15.162:0/1170309552 shutdown_connections 2024-04-27T04:22:08.191 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:08.190+0000 7fe921d9d640 1 -- 172.21.15.162:0/1170309552 wait complete. 2024-04-27T04:22:08.199 DEBUG:tasks.fs:allow_standby_replay disabled in epoch 17 2024-04-27T04:22:08.200 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 18 2024-04-27T04:22:08.296 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:08.296+0000 7f20f24ae640 1 -- 172.21.15.162:0/470281455 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f20ec0fdb50 msgr2=0x7f20ec106190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:22:08.296 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:08.296+0000 7f20f24ae640 1 --2- 172.21.15.162:0/470281455 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f20ec0fdb50 0x7f20ec106190 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f20e00048d0 tx=0x7f20e0009da0 comp rx=0 tx=0).stop 2024-04-27T04:22:08.297 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:08.296+0000 7f20f24ae640 1 -- 172.21.15.162:0/470281455 shutdown_connections 2024-04-27T04:22:08.297 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:08.296+0000 7f20f24ae640 1 --2- 172.21.15.162:0/470281455 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f20ec0fdb50 0x7f20ec106190 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:22:08.297 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:08.296+0000 7f20f24ae640 1 --2- 172.21.15.162:0/470281455 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f20ec0fd1b0 0x7f20ec0fd580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:22:08.297 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:08.296+0000 7f20f24ae640 1 -- 172.21.15.162:0/470281455 >> 172.21.15.162:0/470281455 conn(0x7f20ec0f6c20 msgr2=0x7f20ec0f9030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:22:08.297 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:08.296+0000 7f20f24ae640 1 -- 172.21.15.162:0/470281455 shutdown_connections 2024-04-27T04:22:08.297 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:08.296+0000 7f20f24ae640 1 -- 172.21.15.162:0/470281455 wait complete. 2024-04-27T04:22:08.297 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:08.296+0000 7f20f24ae640 1 Processor -- start 2024-04-27T04:22:08.297 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:08.297+0000 7f20f24ae640 1 -- start start 2024-04-27T04:22:08.297 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:08.297+0000 7f20f24ae640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f20ec0fd1b0 0x7f20ec0fafe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:22:08.297 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:08.297+0000 7f20f24ae640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f20ec0fdb50 0x7f20ec0fb520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:22:08.297 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:08.297+0000 7f20f24ae640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20ec0fcec0 con 0x7f20ec0fdb50 2024-04-27T04:22:08.297 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:08.297+0000 7f20f24ae640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20ec0fba60 con 0x7f20ec0fd1b0 2024-04-27T04:22:08.297 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:08.297+0000 7f20eb7fe640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f20ec0fdb50 0x7f20ec0fb520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:22:08.298 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:08.297+0000 7f20eb7fe640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f20ec0fdb50 0x7f20ec0fb520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:56888/0 (socket says 172.21.15.162:56888) 2024-04-27T04:22:08.298 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:08.297+0000 7f20eb7fe640 1 -- 172.21.15.162:0/3900718036 learned_addr learned my addr 172.21.15.162:0/3900718036 (peer_addr_for_me v2:172.21.15.162:0/0) 2024-04-27T04:22:08.298 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:08.297+0000 7f20ebfff640 1 --2- 172.21.15.162:0/3900718036 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f20ec0fd1b0 0x7f20ec0fafe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:22:08.298 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:08.297+0000 7f20eb7fe640 1 -- 172.21.15.162:0/3900718036 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f20ec0fd1b0 msgr2=0x7f20ec0fafe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:22:08.298 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:08.297+0000 7f20eb7fe640 1 --2- 172.21.15.162:0/3900718036 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f20ec0fd1b0 0x7f20ec0fafe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:22:08.298 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:08.297+0000 7f20eb7fe640 1 -- 172.21.15.162:0/3900718036 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20e0005040 con 0x7f20ec0fdb50 2024-04-27T04:22:08.298 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:08.297+0000 7f20ebfff640 1 --2- 172.21.15.162:0/3900718036 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f20ec0fd1b0 0x7f20ec0fafe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-27T04:22:08.298 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:08.297+0000 7f20eb7fe640 1 --2- 172.21.15.162:0/3900718036 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f20ec0fdb50 0x7f20ec0fb520 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f20e0004930 tx=0x7f20e0009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:22:08.298 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:08.297+0000 7f20e8ff9640 1 -- 172.21.15.162:0/3900718036 <== mon.0 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f20e00094c0 con 0x7f20ec0fdb50 2024-04-27T04:22:08.298 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:08.298+0000 7f20e8ff9640 1 -- 172.21.15.162:0/3900718036 <== mon.0 v2:172.21.15.162:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f20e0032e50 con 0x7f20ec0fdb50 2024-04-27T04:22:08.298 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:08.298+0000 7f20f24ae640 1 -- 172.21.15.162:0/3900718036 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f20ec0fbce0 con 0x7f20ec0fdb50 2024-04-27T04:22:08.298 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:08.298+0000 7f20e8ff9640 1 -- 172.21.15.162:0/3900718036 <== mon.0 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f20e00433b0 con 0x7f20ec0fdb50 2024-04-27T04:22:08.298 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:08.298+0000 7f20f24ae640 1 -- 172.21.15.162:0/3900718036 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f20ec1193a0 con 0x7f20ec0fdb50 2024-04-27T04:22:08.299 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:08.298+0000 7f20f24ae640 1 -- 172.21.15.162:0/3900718036 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f20ec0659e0 con 0x7f20ec0fdb50 2024-04-27T04:22:08.299 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:08.299+0000 7f20e8ff9640 1 -- 172.21.15.162:0/3900718036 <== mon.0 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 39) v1 ==== 92747+0+0 (secure 0 0 0) 0x7f20e002f070 con 0x7f20ec0fdb50 2024-04-27T04:22:08.300 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:08.299+0000 7f20e8ff9640 1 --2- 172.21.15.162:0/3900718036 >> [v2:172.21.15.162:6800/1396866108,v1:172.21.15.162:6801/1396866108] conn(0x7f20d406eb70 0x7f20d4071020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:22:08.300 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:08.300+0000 7f20e8ff9640 1 -- 172.21.15.162:0/3900718036 <== mon.0 v2:172.21.15.162:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6582+0+0 (secure 0 0 0) 0x7f20e00b0690 con 0x7f20ec0fdb50 2024-04-27T04:22:08.300 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:08.300+0000 7f20ebfff640 1 --2- 172.21.15.162:0/3900718036 >> [v2:172.21.15.162:6800/1396866108,v1:172.21.15.162:6801/1396866108] conn(0x7f20d406eb70 0x7f20d4071020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:22:08.300 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:08.300+0000 7f20ebfff640 1 --2- 172.21.15.162:0/3900718036 >> [v2:172.21.15.162:6800/1396866108,v1:172.21.15.162:6801/1396866108] conn(0x7f20d406eb70 0x7f20d4071020 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f20dc009af0 tx=0x7f20dc00b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:22:08.302 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:08.304+0000 7f20e8ff9640 1 -- 172.21.15.162:0/3900718036 <== mon.0 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7f20e00793e0 con 0x7f20ec0fdb50 2024-04-27T04:22:08.517 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:08.518+0000 7f20f24ae640 1 -- 172.21.15.162:0/3900718036 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 18, "format": "json"} v 0) v1 -- 0x7f20ec063670 con 0x7f20ec0fdb50 2024-04-27T04:22:08.517 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:08.519+0000 7f20e8ff9640 1 -- 172.21.15.162:0/3900718036 <== mon.0 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 18, "format": "json"}]=0 dumped fsmap epoch 18 v35) v1 ==== 107+0+4094 (secure 0 0 0) 0x7f20e007caa0 con 0x7f20ec0fdb50 2024-04-27T04:22:08.518 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-04-27T04:22:08.518 INFO:teuthology.orchestra.run.smithi162.stdout:{"epoch":18,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24287,"name":"cephfs.smithi181.bkkcdm","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.181:6827/3756475453","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6826","nonce":3756475453},{"type":"v1","addr":"172.21.15.181:6827","nonce":3756475453}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":14},{"gid":34376,"name":"cephfs.smithi162.xcnapk","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6829/1115643134","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6828","nonce":1115643134},{"type":"v1","addr":"172.21.15.162:6829","nonce":1115643134}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":17}],"filesystems":[{"mdsmap":{"epoch":16,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-27T04:06:53.095749+0000","modified":"2024-04-27T04:18:44.511964+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":24273},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24273":{"gid":24273,"name":"cephfs.smithi181.dzomgh","rank":0,"incarnation":12,"state":"up:active","state_seq":6,"addr":"172.21.15.181:6825/658214993","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":658214993},{"type":"v1","addr":"172.21.15.181:6825","nonce":658214993}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":24273,"qdb_cluster":[24273]},"id":1}]} 2024-04-27T04:22:08.518 INFO:teuthology.orchestra.run.smithi162.stderr:dumped fsmap epoch 18 2024-04-27T04:22:08.520 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:08.521+0000 7f20f24ae640 1 -- 172.21.15.162:0/3900718036 >> [v2:172.21.15.162:6800/1396866108,v1:172.21.15.162:6801/1396866108] conn(0x7f20d406eb70 msgr2=0x7f20d4071020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:22:08.520 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:08.521+0000 7f20f24ae640 1 --2- 172.21.15.162:0/3900718036 >> [v2:172.21.15.162:6800/1396866108,v1:172.21.15.162:6801/1396866108] conn(0x7f20d406eb70 0x7f20d4071020 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f20dc009af0 tx=0x7f20dc00b040 comp rx=0 tx=0).stop 2024-04-27T04:22:08.520 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:08.521+0000 7f20f24ae640 1 -- 172.21.15.162:0/3900718036 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f20ec0fdb50 msgr2=0x7f20ec0fb520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:22:08.520 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:08.521+0000 7f20f24ae640 1 --2- 172.21.15.162:0/3900718036 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f20ec0fdb50 0x7f20ec0fb520 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f20e0004930 tx=0x7f20e0009da0 comp rx=0 tx=0).stop 2024-04-27T04:22:08.520 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:08.521+0000 7f20f24ae640 1 -- 172.21.15.162:0/3900718036 shutdown_connections 2024-04-27T04:22:08.520 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:08.521+0000 7f20f24ae640 1 --2- 172.21.15.162:0/3900718036 >> [v2:172.21.15.162:6800/1396866108,v1:172.21.15.162:6801/1396866108] conn(0x7f20d406eb70 0x7f20d4071020 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:22:08.520 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:08.521+0000 7f20f24ae640 1 --2- 172.21.15.162:0/3900718036 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f20ec0fdb50 0x7f20ec0fb520 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:22:08.521 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:08.521+0000 7f20f24ae640 1 --2- 172.21.15.162:0/3900718036 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f20ec0fd1b0 0x7f20ec0fafe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:22:08.521 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:08.521+0000 7f20f24ae640 1 -- 172.21.15.162:0/3900718036 >> 172.21.15.162:0/3900718036 conn(0x7f20ec0f6c20 msgr2=0x7f20ec0f8030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:22:08.521 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:08.521+0000 7f20f24ae640 1 -- 172.21.15.162:0/3900718036 shutdown_connections 2024-04-27T04:22:08.521 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:08.521+0000 7f20f24ae640 1 -- 172.21.15.162:0/3900718036 wait complete. 2024-04-27T04:22:08.531 DEBUG:tasks.fs:allow_standby_replay disabled in epoch 18 2024-04-27T04:22:08.531 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 19 2024-04-27T04:22:08.628 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:08.627+0000 7fc27b714640 1 -- 172.21.15.162:0/1426183216 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc274101090 msgr2=0x7fc274101460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:22:08.628 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:08.627+0000 7fc27b714640 1 --2- 172.21.15.162:0/1426183216 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc274101090 0x7fc274101460 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fc2700048d0 tx=0x7fc270009da0 comp rx=0 tx=0).stop 2024-04-27T04:22:08.628 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:08.627+0000 7fc27b714640 1 -- 172.21.15.162:0/1426183216 shutdown_connections 2024-04-27T04:22:08.628 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:08.627+0000 7fc27b714640 1 --2- 172.21.15.162:0/1426183216 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc274101a30 0x7fc2740fb0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:22:08.628 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:08.627+0000 7fc27b714640 1 --2- 172.21.15.162:0/1426183216 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc274101090 0x7fc274101460 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:22:08.628 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:08.627+0000 7fc27b714640 1 -- 172.21.15.162:0/1426183216 >> 172.21.15.162:0/1426183216 conn(0x7fc2740f6c60 msgr2=0x7fc2740f9070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:22:08.628 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:08.627+0000 7fc27b714640 1 -- 172.21.15.162:0/1426183216 shutdown_connections 2024-04-27T04:22:08.628 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:08.627+0000 7fc27b714640 1 -- 172.21.15.162:0/1426183216 wait complete. 2024-04-27T04:22:08.628 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:08.628+0000 7fc27b714640 1 Processor -- start 2024-04-27T04:22:08.628 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:08.628+0000 7fc27b714640 1 -- start start 2024-04-27T04:22:08.628 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:08.628+0000 7fc27b714640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc274101090 0x7fc274111ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:22:08.629 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:08.628+0000 7fc27b714640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc274101a30 0x7fc2741120e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:22:08.629 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:08.628+0000 7fc27b714640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2741126e0 con 0x7fc274101a30 2024-04-27T04:22:08.629 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:08.628+0000 7fc27b714640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc27410bd30 con 0x7fc274101090 2024-04-27T04:22:08.629 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:08.628+0000 7fc278caf640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc274101a30 0x7fc2741120e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:22:08.629 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:08.628+0000 7fc278caf640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc274101a30 0x7fc2741120e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:56902/0 (socket says 172.21.15.162:56902) 2024-04-27T04:22:08.629 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:08.628+0000 7fc278caf640 1 -- 172.21.15.162:0/264505826 learned_addr learned my addr 172.21.15.162:0/264505826 (peer_addr_for_me v2:172.21.15.162:0/0) 2024-04-27T04:22:08.629 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:08.628+0000 7fc2794b0640 1 --2- 172.21.15.162:0/264505826 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc274101090 0x7fc274111ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:22:08.629 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:08.628+0000 7fc278caf640 1 -- 172.21.15.162:0/264505826 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc274101090 msgr2=0x7fc274111ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:22:08.629 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:08.628+0000 7fc278caf640 1 --2- 172.21.15.162:0/264505826 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc274101090 0x7fc274111ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:22:08.629 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:08.628+0000 7fc278caf640 1 -- 172.21.15.162:0/264505826 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc270005040 con 0x7fc274101a30 2024-04-27T04:22:08.629 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:08.628+0000 7fc278caf640 1 --2- 172.21.15.162:0/264505826 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc274101a30 0x7fc2741120e0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fc26400fce0 tx=0x7fc264004600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:22:08.629 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:08.628+0000 7fc269ffb640 1 -- 172.21.15.162:0/264505826 <== mon.0 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc264007a20 con 0x7fc274101a30 2024-04-27T04:22:08.629 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:08.629+0000 7fc27b714640 1 -- 172.21.15.162:0/264505826 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc27410bfb0 con 0x7fc274101a30 2024-04-27T04:22:08.629 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:08.629+0000 7fc269ffb640 1 -- 172.21.15.162:0/264505826 <== mon.0 v2:172.21.15.162:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fc264007b80 con 0x7fc274101a30 2024-04-27T04:22:08.630 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:08.629+0000 7fc269ffb640 1 -- 172.21.15.162:0/264505826 <== mon.0 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc2640127a0 con 0x7fc274101a30 2024-04-27T04:22:08.630 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:08.629+0000 7fc27b714640 1 -- 172.21.15.162:0/264505826 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc27410c4d0 con 0x7fc274101a30 2024-04-27T04:22:08.630 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:08.630+0000 7fc27b714640 1 -- 172.21.15.162:0/264505826 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc2740fc800 con 0x7fc274101a30 2024-04-27T04:22:08.631 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:08.630+0000 7fc269ffb640 1 -- 172.21.15.162:0/264505826 <== mon.0 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 39) v1 ==== 92747+0+0 (secure 0 0 0) 0x7fc264007cf0 con 0x7fc274101a30 2024-04-27T04:22:08.631 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:08.630+0000 7fc269ffb640 1 --2- 172.21.15.162:0/264505826 >> [v2:172.21.15.162:6800/1396866108,v1:172.21.15.162:6801/1396866108] conn(0x7fc26006eb70 0x7fc260071020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:22:08.631 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:08.630+0000 7fc269ffb640 1 -- 172.21.15.162:0/264505826 <== mon.0 v2:172.21.15.162:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6582+0+0 (secure 0 0 0) 0x7fc264090e60 con 0x7fc274101a30 2024-04-27T04:22:08.631 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:08.630+0000 7fc2794b0640 1 --2- 172.21.15.162:0/264505826 >> [v2:172.21.15.162:6800/1396866108,v1:172.21.15.162:6801/1396866108] conn(0x7fc26006eb70 0x7fc260071020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:22:08.631 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:08.631+0000 7fc2794b0640 1 --2- 172.21.15.162:0/264505826 >> [v2:172.21.15.162:6800/1396866108,v1:172.21.15.162:6801/1396866108] conn(0x7fc26006eb70 0x7fc260071020 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fc270003080 tx=0x7fc270009da0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:22:08.633 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:08.635+0000 7fc269ffb640 1 -- 172.21.15.162:0/264505826 <== mon.0 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7fc264059c60 con 0x7fc274101a30 2024-04-27T04:22:08.850 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:08.851+0000 7fc27b714640 1 -- 172.21.15.162:0/264505826 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 19, "format": "json"} v 0) v1 -- 0x7fc2741081f0 con 0x7fc274101a30 2024-04-27T04:22:08.850 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:08.852+0000 7fc269ffb640 1 -- 172.21.15.162:0/264505826 <== mon.0 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 19, "format": "json"}]=0 dumped fsmap epoch 19 v35) v1 ==== 107+0+4944 (secure 0 0 0) 0x7fc26405d320 con 0x7fc274101a30 2024-04-27T04:22:08.851 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-04-27T04:22:08.851 INFO:teuthology.orchestra.run.smithi162.stdout:{"epoch":19,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24287,"name":"cephfs.smithi181.bkkcdm","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.181:6827/3756475453","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6826","nonce":3756475453},{"type":"v1","addr":"172.21.15.181:6827","nonce":3756475453}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":14},{"gid":34376,"name":"cephfs.smithi162.xcnapk","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6829/1115643134","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6828","nonce":1115643134},{"type":"v1","addr":"172.21.15.162:6829","nonce":1115643134}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":17},{"gid":34380,"name":"cephfs.smithi162.gzlyjj","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6827/1198130667","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6826","nonce":1198130667},{"type":"v1","addr":"172.21.15.162:6827","nonce":1198130667}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":19}],"filesystems":[{"mdsmap":{"epoch":16,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-27T04:06:53.095749+0000","modified":"2024-04-27T04:18:44.511964+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":24273},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24273":{"gid":24273,"name":"cephfs.smithi181.dzomgh","rank":0,"incarnation":12,"state":"up:active","state_seq":6,"addr":"172.21.15.181:6825/658214993","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":658214993},{"type":"v1","addr":"172.21.15.181:6825","nonce":658214993}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":24273,"qdb_cluster":[24273]},"id":1}]} 2024-04-27T04:22:08.851 INFO:teuthology.orchestra.run.smithi162.stderr:dumped fsmap epoch 19 2024-04-27T04:22:08.853 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:08.854+0000 7fc27b714640 1 -- 172.21.15.162:0/264505826 >> [v2:172.21.15.162:6800/1396866108,v1:172.21.15.162:6801/1396866108] conn(0x7fc26006eb70 msgr2=0x7fc260071020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:22:08.853 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:08.854+0000 7fc27b714640 1 --2- 172.21.15.162:0/264505826 >> [v2:172.21.15.162:6800/1396866108,v1:172.21.15.162:6801/1396866108] conn(0x7fc26006eb70 0x7fc260071020 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fc270003080 tx=0x7fc270009da0 comp rx=0 tx=0).stop 2024-04-27T04:22:08.853 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:08.854+0000 7fc27b714640 1 -- 172.21.15.162:0/264505826 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc274101a30 msgr2=0x7fc2741120e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:22:08.853 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:08.854+0000 7fc27b714640 1 --2- 172.21.15.162:0/264505826 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc274101a30 0x7fc2741120e0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fc26400fce0 tx=0x7fc264004600 comp rx=0 tx=0).stop 2024-04-27T04:22:08.853 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:08.854+0000 7fc27b714640 1 -- 172.21.15.162:0/264505826 shutdown_connections 2024-04-27T04:22:08.853 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:08.854+0000 7fc27b714640 1 --2- 172.21.15.162:0/264505826 >> [v2:172.21.15.162:6800/1396866108,v1:172.21.15.162:6801/1396866108] conn(0x7fc26006eb70 0x7fc260071020 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:22:08.853 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:08.854+0000 7fc27b714640 1 --2- 172.21.15.162:0/264505826 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc274101a30 0x7fc2741120e0 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:22:08.853 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:08.854+0000 7fc27b714640 1 --2- 172.21.15.162:0/264505826 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc274101090 0x7fc274111ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:22:08.853 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:08.854+0000 7fc27b714640 1 -- 172.21.15.162:0/264505826 >> 172.21.15.162:0/264505826 conn(0x7fc2740f6c60 msgr2=0x7fc2740f7b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:22:08.853 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:08.854+0000 7fc27b714640 1 -- 172.21.15.162:0/264505826 shutdown_connections 2024-04-27T04:22:08.854 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:08.854+0000 7fc27b714640 1 -- 172.21.15.162:0/264505826 wait complete. 2024-04-27T04:22:08.863 DEBUG:tasks.fs:allow_standby_replay disabled in epoch 19 2024-04-27T04:22:08.864 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 20 2024-04-27T04:22:08.960 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:08.960+0000 7f16c8d1d640 1 -- 172.21.15.162:0/3938657172 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f16c4101170 msgr2=0x7f16c4101540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:22:08.960 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:08.960+0000 7f16c8d1d640 1 --2- 172.21.15.162:0/3938657172 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f16c4101170 0x7f16c4101540 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f16b40048d0 tx=0x7f16b4009da0 comp rx=0 tx=0).stop 2024-04-27T04:22:08.960 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:08.960+0000 7f16c8d1d640 1 -- 172.21.15.162:0/3938657172 shutdown_connections 2024-04-27T04:22:08.960 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:08.960+0000 7f16c8d1d640 1 --2- 172.21.15.162:0/3938657172 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f16c40fb150 0x7f16c40fb5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:22:08.960 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:08.960+0000 7f16c8d1d640 1 --2- 172.21.15.162:0/3938657172 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f16c4101170 0x7f16c4101540 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:22:08.960 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:08.960+0000 7f16c8d1d640 1 -- 172.21.15.162:0/3938657172 >> 172.21.15.162:0/3938657172 conn(0x7f16c4071fd0 msgr2=0x7f16c40723d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:22:08.961 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:08.960+0000 7f16c8d1d640 1 -- 172.21.15.162:0/3938657172 shutdown_connections 2024-04-27T04:22:08.961 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:08.960+0000 7f16c8d1d640 1 -- 172.21.15.162:0/3938657172 wait complete. 2024-04-27T04:22:08.961 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:08.960+0000 7f16c8d1d640 1 Processor -- start 2024-04-27T04:22:08.961 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:08.960+0000 7f16c8d1d640 1 -- start start 2024-04-27T04:22:08.961 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:08.960+0000 7f16c8d1d640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f16c40fb150 0x7f16c4111bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:22:08.961 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:08.960+0000 7f16c8d1d640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f16c4101170 0x7f16c41120f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:22:08.961 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:08.960+0000 7f16c8d1d640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16c41126f0 con 0x7f16c40fb150 2024-04-27T04:22:08.961 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:08.960+0000 7f16c8d1d640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16c410bd40 con 0x7f16c4101170 2024-04-27T04:22:08.961 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:08.960+0000 7f16c259c640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f16c40fb150 0x7f16c4111bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:22:08.961 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:08.960+0000 7f16c259c640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f16c40fb150 0x7f16c4111bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:56906/0 (socket says 172.21.15.162:56906) 2024-04-27T04:22:08.961 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:08.960+0000 7f16c259c640 1 -- 172.21.15.162:0/4103189192 learned_addr learned my addr 172.21.15.162:0/4103189192 (peer_addr_for_me v2:172.21.15.162:0/0) 2024-04-27T04:22:08.961 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:08.961+0000 7f16c1d9b640 1 --2- 172.21.15.162:0/4103189192 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f16c4101170 0x7f16c41120f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:22:08.961 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:08.961+0000 7f16c259c640 1 -- 172.21.15.162:0/4103189192 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f16c4101170 msgr2=0x7f16c41120f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:22:08.962 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:08.961+0000 7f16c259c640 1 --2- 172.21.15.162:0/4103189192 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f16c4101170 0x7f16c41120f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:22:08.962 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:08.961+0000 7f16c259c640 1 -- 172.21.15.162:0/4103189192 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f16b4005040 con 0x7f16c40fb150 2024-04-27T04:22:08.962 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:08.961+0000 7f16c259c640 1 --2- 172.21.15.162:0/4103189192 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f16c40fb150 0x7f16c4111bb0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f16b4004930 tx=0x7f16b4009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:22:08.962 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:08.961+0000 7f16baffd640 1 -- 172.21.15.162:0/4103189192 <== mon.0 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f16b40094c0 con 0x7f16c40fb150 2024-04-27T04:22:08.962 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:08.961+0000 7f16baffd640 1 -- 172.21.15.162:0/4103189192 <== mon.0 v2:172.21.15.162:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f16b4032e50 con 0x7f16c40fb150 2024-04-27T04:22:08.962 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:08.961+0000 7f16c8d1d640 1 -- 172.21.15.162:0/4103189192 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f16c410bfc0 con 0x7f16c40fb150 2024-04-27T04:22:08.962 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:08.961+0000 7f16baffd640 1 -- 172.21.15.162:0/4103189192 <== mon.0 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f16b40433b0 con 0x7f16c40fb150 2024-04-27T04:22:08.962 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:08.961+0000 7f16c8d1d640 1 -- 172.21.15.162:0/4103189192 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f16c410c480 con 0x7f16c40fb150 2024-04-27T04:22:08.963 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:08.962+0000 7f16c8d1d640 1 -- 172.21.15.162:0/4103189192 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f16c40fc870 con 0x7f16c40fb150 2024-04-27T04:22:08.963 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:08.962+0000 7f16baffd640 1 -- 172.21.15.162:0/4103189192 <== mon.0 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 39) v1 ==== 92747+0+0 (secure 0 0 0) 0x7f16b402f070 con 0x7f16c40fb150 2024-04-27T04:22:08.963 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:08.963+0000 7f16baffd640 1 --2- 172.21.15.162:0/4103189192 >> [v2:172.21.15.162:6800/1396866108,v1:172.21.15.162:6801/1396866108] conn(0x7f16b006eb70 0x7f16b0071020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:22:08.963 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:08.963+0000 7f16baffd640 1 -- 172.21.15.162:0/4103189192 <== mon.0 v2:172.21.15.162:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6582+0+0 (secure 0 0 0) 0x7f16b40b0920 con 0x7f16c40fb150 2024-04-27T04:22:08.963 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:08.963+0000 7f16c1d9b640 1 --2- 172.21.15.162:0/4103189192 >> [v2:172.21.15.162:6800/1396866108,v1:172.21.15.162:6801/1396866108] conn(0x7f16b006eb70 0x7f16b0071020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:22:08.963 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:08.963+0000 7f16c1d9b640 1 --2- 172.21.15.162:0/4103189192 >> [v2:172.21.15.162:6800/1396866108,v1:172.21.15.162:6801/1396866108] conn(0x7f16b006eb70 0x7f16b0071020 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f16c410d470 tx=0x7f16ac009040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:22:08.966 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:08.968+0000 7f16baffd640 1 -- 172.21.15.162:0/4103189192 <== mon.0 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7f16b4079670 con 0x7f16c40fb150 2024-04-27T04:22:09.136 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:22:08 smithi162 ceph-mon[110641]: pgmap v273: 65 pgs: 65 active+clean; 208 MiB data, 851 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:22:09.136 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:22:08 smithi162 ceph-mon[110641]: from='client.? 172.21.15.162:0/3103507137' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 16, "format": "json"}]: dispatch 2024-04-27T04:22:09.136 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:22:08 smithi162 ceph-mon[110641]: from='client.? 172.21.15.162:0/1170309552' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 17, "format": "json"}]: dispatch 2024-04-27T04:22:09.136 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:22:08 smithi162 ceph-mon[110641]: from='client.? 172.21.15.162:0/3900718036' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 18, "format": "json"}]: dispatch 2024-04-27T04:22:09.171 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:22:08 smithi181 ceph-mon[81134]: pgmap v273: 65 pgs: 65 active+clean; 208 MiB data, 851 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:22:09.172 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:22:08 smithi181 ceph-mon[81134]: from='client.? 172.21.15.162:0/3103507137' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 16, "format": "json"}]: dispatch 2024-04-27T04:22:09.172 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:22:08 smithi181 ceph-mon[81134]: from='client.? 172.21.15.162:0/1170309552' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 17, "format": "json"}]: dispatch 2024-04-27T04:22:09.172 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:22:08 smithi181 ceph-mon[81134]: from='client.? 172.21.15.162:0/3900718036' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 18, "format": "json"}]: dispatch 2024-04-27T04:22:09.179 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:09.181+0000 7f16c8d1d640 1 -- 172.21.15.162:0/4103189192 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 20, "format": "json"} v 0) v1 -- 0x7f16c410d220 con 0x7f16c40fb150 2024-04-27T04:22:09.180 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:09.182+0000 7f16baffd640 1 -- 172.21.15.162:0/4103189192 <== mon.0 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 20, "format": "json"}]=0 dumped fsmap epoch 20 v35) v1 ==== 107+0+4162 (secure 0 0 0) 0x7f16b407cd30 con 0x7f16c40fb150 2024-04-27T04:22:09.180 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-04-27T04:22:09.181 INFO:teuthology.orchestra.run.smithi162.stdout:{"epoch":20,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24287,"name":"cephfs.smithi181.bkkcdm","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.181:6827/3756475453","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6826","nonce":3756475453},{"type":"v1","addr":"172.21.15.181:6827","nonce":3756475453}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":14},{"gid":34380,"name":"cephfs.smithi162.gzlyjj","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6827/1198130667","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6826","nonce":1198130667},{"type":"v1","addr":"172.21.15.162:6827","nonce":1198130667}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":19}],"filesystems":[{"mdsmap":{"epoch":16,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-27T04:06:53.095749+0000","modified":"2024-04-27T04:18:44.511964+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":24273},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24273":{"gid":24273,"name":"cephfs.smithi181.dzomgh","rank":0,"incarnation":12,"state":"up:active","state_seq":6,"addr":"172.21.15.181:6825/658214993","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":658214993},{"type":"v1","addr":"172.21.15.181:6825","nonce":658214993}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":24273,"qdb_cluster":[24273]},"id":1}]} 2024-04-27T04:22:09.181 INFO:teuthology.orchestra.run.smithi162.stderr:dumped fsmap epoch 20 2024-04-27T04:22:09.183 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:09.184+0000 7f16c8d1d640 1 -- 172.21.15.162:0/4103189192 >> [v2:172.21.15.162:6800/1396866108,v1:172.21.15.162:6801/1396866108] conn(0x7f16b006eb70 msgr2=0x7f16b0071020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:22:09.183 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:09.184+0000 7f16c8d1d640 1 --2- 172.21.15.162:0/4103189192 >> [v2:172.21.15.162:6800/1396866108,v1:172.21.15.162:6801/1396866108] conn(0x7f16b006eb70 0x7f16b0071020 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f16c410d470 tx=0x7f16ac009040 comp rx=0 tx=0).stop 2024-04-27T04:22:09.183 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:09.184+0000 7f16c8d1d640 1 -- 172.21.15.162:0/4103189192 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f16c40fb150 msgr2=0x7f16c4111bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:22:09.183 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:09.184+0000 7f16c8d1d640 1 --2- 172.21.15.162:0/4103189192 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f16c40fb150 0x7f16c4111bb0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f16b4004930 tx=0x7f16b4009da0 comp rx=0 tx=0).stop 2024-04-27T04:22:09.183 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:09.184+0000 7f16c8d1d640 1 -- 172.21.15.162:0/4103189192 shutdown_connections 2024-04-27T04:22:09.183 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:09.184+0000 7f16c8d1d640 1 --2- 172.21.15.162:0/4103189192 >> [v2:172.21.15.162:6800/1396866108,v1:172.21.15.162:6801/1396866108] conn(0x7f16b006eb70 0x7f16b0071020 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:22:09.183 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:09.184+0000 7f16c8d1d640 1 --2- 172.21.15.162:0/4103189192 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f16c4101170 0x7f16c41120f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:22:09.183 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:09.184+0000 7f16c8d1d640 1 --2- 172.21.15.162:0/4103189192 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f16c40fb150 0x7f16c4111bb0 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:22:09.184 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:09.184+0000 7f16c8d1d640 1 -- 172.21.15.162:0/4103189192 >> 172.21.15.162:0/4103189192 conn(0x7f16c4071fd0 msgr2=0x7f16c40fa330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:22:09.184 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:09.184+0000 7f16c8d1d640 1 -- 172.21.15.162:0/4103189192 shutdown_connections 2024-04-27T04:22:09.184 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:09.184+0000 7f16c8d1d640 1 -- 172.21.15.162:0/4103189192 wait complete. 2024-04-27T04:22:09.193 DEBUG:tasks.fs:allow_standby_replay disabled in epoch 20 2024-04-27T04:22:09.194 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 21 2024-04-27T04:22:09.291 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:09.290+0000 7f0c2ee6e640 1 -- 172.21.15.162:0/2031250779 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0c28102230 msgr2=0x7f0c28102600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:22:09.291 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:09.290+0000 7f0c2ee6e640 1 --2- 172.21.15.162:0/2031250779 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0c28102230 0x7f0c28102600 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f0c1c0048d0 tx=0x7f0c1c009da0 comp rx=0 tx=0).stop 2024-04-27T04:22:09.291 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:09.290+0000 7f0c2ee6e640 1 -- 172.21.15.162:0/2031250779 shutdown_connections 2024-04-27T04:22:09.292 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:09.290+0000 7f0c2ee6e640 1 --2- 172.21.15.162:0/2031250779 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0c28102bd0 0x7f0c280fa8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:22:09.292 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:09.290+0000 7f0c2ee6e640 1 --2- 172.21.15.162:0/2031250779 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0c28102230 0x7f0c28102600 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:22:09.292 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:09.290+0000 7f0c2ee6e640 1 -- 172.21.15.162:0/2031250779 >> 172.21.15.162:0/2031250779 conn(0x7f0c280717e0 msgr2=0x7f0c28071be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:22:09.292 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:09.290+0000 7f0c2ee6e640 1 -- 172.21.15.162:0/2031250779 shutdown_connections 2024-04-27T04:22:09.292 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:09.290+0000 7f0c2ee6e640 1 -- 172.21.15.162:0/2031250779 wait complete. 2024-04-27T04:22:09.292 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:09.291+0000 7f0c2ee6e640 1 Processor -- start 2024-04-27T04:22:09.292 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:09.291+0000 7f0c2ee6e640 1 -- start start 2024-04-27T04:22:09.292 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:09.291+0000 7f0c2ee6e640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0c28102230 0x7f0c28112d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:22:09.292 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:09.291+0000 7f0c2ee6e640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0c28102bd0 0x7f0c28113250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:22:09.292 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:09.291+0000 7f0c2ee6e640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c28113850 con 0x7f0c28102bd0 2024-04-27T04:22:09.292 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:09.291+0000 7f0c2ee6e640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c2810cea0 con 0x7f0c28102230 2024-04-27T04:22:09.292 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:09.291+0000 7f0c27fff640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0c28102bd0 0x7f0c28113250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:22:09.293 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:09.291+0000 7f0c27fff640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0c28102bd0 0x7f0c28113250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:56926/0 (socket says 172.21.15.162:56926) 2024-04-27T04:22:09.293 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:09.291+0000 7f0c27fff640 1 -- 172.21.15.162:0/3847378662 learned_addr learned my addr 172.21.15.162:0/3847378662 (peer_addr_for_me v2:172.21.15.162:0/0) 2024-04-27T04:22:09.293 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:09.291+0000 7f0c27fff640 1 -- 172.21.15.162:0/3847378662 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0c28102230 msgr2=0x7f0c28112d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:22:09.293 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:09.291+0000 7f0c27fff640 1 --2- 172.21.15.162:0/3847378662 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0c28102230 0x7f0c28112d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:22:09.293 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:09.291+0000 7f0c27fff640 1 -- 172.21.15.162:0/3847378662 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c1c005040 con 0x7f0c28102bd0 2024-04-27T04:22:09.293 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:09.292+0000 7f0c27fff640 1 --2- 172.21.15.162:0/3847378662 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0c28102bd0 0x7f0c28113250 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f0c14009af0 tx=0x7f0c14004570 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:22:09.293 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:09.292+0000 7f0c257fa640 1 -- 172.21.15.162:0/3847378662 <== mon.0 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0c14007c30 con 0x7f0c28102bd0 2024-04-27T04:22:09.293 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:09.292+0000 7f0c257fa640 1 -- 172.21.15.162:0/3847378662 <== mon.0 v2:172.21.15.162:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f0c1400f040 con 0x7f0c28102bd0 2024-04-27T04:22:09.293 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:09.292+0000 7f0c2ee6e640 1 -- 172.21.15.162:0/3847378662 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0c2810d180 con 0x7f0c28102bd0 2024-04-27T04:22:09.293 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:09.292+0000 7f0c257fa640 1 -- 172.21.15.162:0/3847378662 <== mon.0 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0c14009570 con 0x7f0c28102bd0 2024-04-27T04:22:09.293 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:09.292+0000 7f0c2ee6e640 1 -- 172.21.15.162:0/3847378662 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0c2810d6a0 con 0x7f0c28102bd0 2024-04-27T04:22:09.294 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:09.293+0000 7f0c2ee6e640 1 -- 172.21.15.162:0/3847378662 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0c28065530 con 0x7f0c28102bd0 2024-04-27T04:22:09.294 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:09.293+0000 7f0c257fa640 1 -- 172.21.15.162:0/3847378662 <== mon.0 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 39) v1 ==== 92747+0+0 (secure 0 0 0) 0x7f0c140097b0 con 0x7f0c28102bd0 2024-04-27T04:22:09.295 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:09.293+0000 7f0c257fa640 1 --2- 172.21.15.162:0/3847378662 >> [v2:172.21.15.162:6800/1396866108,v1:172.21.15.162:6801/1396866108] conn(0x7f0c1806eb70 0x7f0c18071020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:22:09.295 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:09.294+0000 7f0c257fa640 1 -- 172.21.15.162:0/3847378662 <== mon.0 v2:172.21.15.162:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6582+0+0 (secure 0 0 0) 0x7f0c1408e8d0 con 0x7f0c28102bd0 2024-04-27T04:22:09.295 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:09.294+0000 7f0c2cc0a640 1 --2- 172.21.15.162:0/3847378662 >> [v2:172.21.15.162:6800/1396866108,v1:172.21.15.162:6801/1396866108] conn(0x7f0c1806eb70 0x7f0c18071020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:22:09.295 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:09.294+0000 7f0c2cc0a640 1 --2- 172.21.15.162:0/3847378662 >> [v2:172.21.15.162:6800/1396866108,v1:172.21.15.162:6801/1396866108] conn(0x7f0c1806eb70 0x7f0c18071020 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f0c1c0043d0 tx=0x7f0c1c009da0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:22:09.296 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:09.298+0000 7f0c257fa640 1 -- 172.21.15.162:0/3847378662 <== mon.0 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7f0c14057620 con 0x7f0c28102bd0 2024-04-27T04:22:09.509 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:09.511+0000 7f0c2ee6e640 1 -- 172.21.15.162:0/3847378662 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 21, "format": "json"} v 0) v1 -- 0x7f0c281093d0 con 0x7f0c28102bd0 2024-04-27T04:22:09.510 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:09.512+0000 7f0c257fa640 1 -- 172.21.15.162:0/3847378662 <== mon.0 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 21, "format": "json"}]=0 dumped fsmap epoch 21 v35) v1 ==== 107+0+5009 (secure 0 0 0) 0x7f0c1405ace0 con 0x7f0c28102bd0 2024-04-27T04:22:09.510 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-04-27T04:22:09.510 INFO:teuthology.orchestra.run.smithi162.stdout:{"epoch":21,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24287,"name":"cephfs.smithi181.bkkcdm","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.181:6827/3756475453","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6826","nonce":3756475453},{"type":"v1","addr":"172.21.15.181:6827","nonce":3756475453}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":14},{"gid":34380,"name":"cephfs.smithi162.gzlyjj","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6827/1198130667","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6826","nonce":1198130667},{"type":"v1","addr":"172.21.15.162:6827","nonce":1198130667}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":19},{"gid":34412,"name":"cephfs.smithi162.xcnapk","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6829/307528848","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6828","nonce":307528848},{"type":"v1","addr":"172.21.15.162:6829","nonce":307528848}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":21}],"filesystems":[{"mdsmap":{"epoch":16,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-27T04:06:53.095749+0000","modified":"2024-04-27T04:18:44.511964+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":24273},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24273":{"gid":24273,"name":"cephfs.smithi181.dzomgh","rank":0,"incarnation":12,"state":"up:active","state_seq":6,"addr":"172.21.15.181:6825/658214993","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":658214993},{"type":"v1","addr":"172.21.15.181:6825","nonce":658214993}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":24273,"qdb_cluster":[24273]},"id":1}]} 2024-04-27T04:22:09.511 INFO:teuthology.orchestra.run.smithi162.stderr:dumped fsmap epoch 21 2024-04-27T04:22:09.512 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:09.513+0000 7f0c2ee6e640 1 -- 172.21.15.162:0/3847378662 >> [v2:172.21.15.162:6800/1396866108,v1:172.21.15.162:6801/1396866108] conn(0x7f0c1806eb70 msgr2=0x7f0c18071020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:22:09.512 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:09.514+0000 7f0c2ee6e640 1 --2- 172.21.15.162:0/3847378662 >> [v2:172.21.15.162:6800/1396866108,v1:172.21.15.162:6801/1396866108] conn(0x7f0c1806eb70 0x7f0c18071020 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f0c1c0043d0 tx=0x7f0c1c009da0 comp rx=0 tx=0).stop 2024-04-27T04:22:09.513 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:09.514+0000 7f0c2ee6e640 1 -- 172.21.15.162:0/3847378662 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0c28102bd0 msgr2=0x7f0c28113250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:22:09.513 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:09.514+0000 7f0c2ee6e640 1 --2- 172.21.15.162:0/3847378662 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0c28102bd0 0x7f0c28113250 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f0c14009af0 tx=0x7f0c14004570 comp rx=0 tx=0).stop 2024-04-27T04:22:09.513 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:09.514+0000 7f0c2ee6e640 1 -- 172.21.15.162:0/3847378662 shutdown_connections 2024-04-27T04:22:09.513 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:09.514+0000 7f0c2ee6e640 1 --2- 172.21.15.162:0/3847378662 >> [v2:172.21.15.162:6800/1396866108,v1:172.21.15.162:6801/1396866108] conn(0x7f0c1806eb70 0x7f0c18071020 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:22:09.513 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:09.514+0000 7f0c2ee6e640 1 --2- 172.21.15.162:0/3847378662 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0c28102bd0 0x7f0c28113250 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:22:09.513 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:09.514+0000 7f0c2ee6e640 1 --2- 172.21.15.162:0/3847378662 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0c28102230 0x7f0c28112d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:22:09.513 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:09.514+0000 7f0c2ee6e640 1 -- 172.21.15.162:0/3847378662 >> 172.21.15.162:0/3847378662 conn(0x7f0c280717e0 msgr2=0x7f0c280fa100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:22:09.513 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:09.514+0000 7f0c2ee6e640 1 -- 172.21.15.162:0/3847378662 shutdown_connections 2024-04-27T04:22:09.513 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:09.514+0000 7f0c2ee6e640 1 -- 172.21.15.162:0/3847378662 wait complete. 2024-04-27T04:22:09.523 DEBUG:tasks.fs:allow_standby_replay disabled in epoch 21 2024-04-27T04:22:09.523 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 22 2024-04-27T04:22:09.622 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:09.620+0000 7f924b554640 1 -- 172.21.15.162:0/2906350024 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f92441010b0 msgr2=0x7f9244101480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:22:09.623 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:09.620+0000 7f924b554640 1 --2- 172.21.15.162:0/2906350024 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f92441010b0 0x7f9244101480 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f92340048d0 tx=0x7f9234009da0 comp rx=0 tx=0).stop 2024-04-27T04:22:09.623 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:09.620+0000 7f924b554640 1 -- 172.21.15.162:0/2906350024 shutdown_connections 2024-04-27T04:22:09.623 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:09.620+0000 7f924b554640 1 --2- 172.21.15.162:0/2906350024 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9244101a50 0x7f92440fb0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:22:09.623 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:09.620+0000 7f924b554640 1 --2- 172.21.15.162:0/2906350024 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f92441010b0 0x7f9244101480 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:22:09.623 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:09.620+0000 7f924b554640 1 -- 172.21.15.162:0/2906350024 >> 172.21.15.162:0/2906350024 conn(0x7f92440f6c80 msgr2=0x7f92440f9090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:22:09.623 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:09.620+0000 7f924b554640 1 -- 172.21.15.162:0/2906350024 shutdown_connections 2024-04-27T04:22:09.623 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:09.620+0000 7f924b554640 1 -- 172.21.15.162:0/2906350024 wait complete. 2024-04-27T04:22:09.623 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:09.620+0000 7f924b554640 1 Processor -- start 2024-04-27T04:22:09.623 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:09.620+0000 7f924b554640 1 -- start start 2024-04-27T04:22:09.623 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:09.620+0000 7f924b554640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f92441010b0 0x7f9244101f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:22:09.623 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:09.620+0000 7f924b554640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9244101a50 0x7f9244102470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:22:09.624 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:09.620+0000 7f924b554640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9244103e10 con 0x7f92441010b0 2024-04-27T04:22:09.624 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:09.620+0000 7f924b554640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92441029b0 con 0x7f9244101a50 2024-04-27T04:22:09.624 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:09.621+0000 7f92492f0640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f92441010b0 0x7f9244101f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:22:09.624 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:09.621+0000 7f92492f0640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f92441010b0 0x7f9244101f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:56946/0 (socket says 172.21.15.162:56946) 2024-04-27T04:22:09.624 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:09.621+0000 7f92492f0640 1 -- 172.21.15.162:0/2686974446 learned_addr learned my addr 172.21.15.162:0/2686974446 (peer_addr_for_me v2:172.21.15.162:0/0) 2024-04-27T04:22:09.624 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:09.621+0000 7f92492f0640 1 -- 172.21.15.162:0/2686974446 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9244101a50 msgr2=0x7f9244102470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:22:09.624 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:09.621+0000 7f92492f0640 1 --2- 172.21.15.162:0/2686974446 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9244101a50 0x7f9244102470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:22:09.624 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:09.621+0000 7f92492f0640 1 -- 172.21.15.162:0/2686974446 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9234005040 con 0x7f92441010b0 2024-04-27T04:22:09.624 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:09.621+0000 7f92492f0640 1 --2- 172.21.15.162:0/2686974446 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f92441010b0 0x7f9244101f30 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f9234004930 tx=0x7f9234009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:22:09.624 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:09.621+0000 7f9239ffb640 1 -- 172.21.15.162:0/2686974446 <== mon.0 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f92340094c0 con 0x7f92441010b0 2024-04-27T04:22:09.624 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:09.621+0000 7f9239ffb640 1 -- 172.21.15.162:0/2686974446 <== mon.0 v2:172.21.15.162:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f9234032e50 con 0x7f92441010b0 2024-04-27T04:22:09.624 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:09.621+0000 7f924b554640 1 -- 172.21.15.162:0/2686974446 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9244102c30 con 0x7f92441010b0 2024-04-27T04:22:09.624 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:09.621+0000 7f9239ffb640 1 -- 172.21.15.162:0/2686974446 <== mon.0 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f92340433b0 con 0x7f92441010b0 2024-04-27T04:22:09.624 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:09.621+0000 7f924b554640 1 -- 172.21.15.162:0/2686974446 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9244119400 con 0x7f92441010b0 2024-04-27T04:22:09.625 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:09.622+0000 7f924b554640 1 -- 172.21.15.162:0/2686974446 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f92440fc820 con 0x7f92441010b0 2024-04-27T04:22:09.626 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:09.623+0000 7f9239ffb640 1 -- 172.21.15.162:0/2686974446 <== mon.0 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 39) v1 ==== 92747+0+0 (secure 0 0 0) 0x7f923402f070 con 0x7f92441010b0 2024-04-27T04:22:09.626 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:09.623+0000 7f9239ffb640 1 --2- 172.21.15.162:0/2686974446 >> [v2:172.21.15.162:6800/1396866108,v1:172.21.15.162:6801/1396866108] conn(0x7f923006eb70 0x7f9230071020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:22:09.626 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:09.623+0000 7f9239ffb640 1 -- 172.21.15.162:0/2686974446 <== mon.0 v2:172.21.15.162:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6582+0+0 (secure 0 0 0) 0x7f92340b0790 con 0x7f92441010b0 2024-04-27T04:22:09.626 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:09.623+0000 7f9248aef640 1 --2- 172.21.15.162:0/2686974446 >> [v2:172.21.15.162:6800/1396866108,v1:172.21.15.162:6801/1396866108] conn(0x7f923006eb70 0x7f9230071020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:22:09.626 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:09.624+0000 7f9248aef640 1 --2- 172.21.15.162:0/2686974446 >> [v2:172.21.15.162:6800/1396866108,v1:172.21.15.162:6801/1396866108] conn(0x7f923006eb70 0x7f9230071020 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f9244103660 tx=0x7f9240007cb0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:22:09.626 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:09.628+0000 7f9239ffb640 1 -- 172.21.15.162:0/2686974446 <== mon.0 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7f92340794e0 con 0x7f92441010b0 2024-04-27T04:22:09.849 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:09.850+0000 7f924b554640 1 -- 172.21.15.162:0/2686974446 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 22, "format": "json"} v 0) v1 -- 0x7f9244064350 con 0x7f92441010b0 2024-04-27T04:22:09.849 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:09.851+0000 7f9239ffb640 1 -- 172.21.15.162:0/2686974446 <== mon.0 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 22, "format": "json"}]=0 dumped fsmap epoch 22 v35) v1 ==== 107+0+4211 (secure 0 0 0) 0x7f923407cba0 con 0x7f92441010b0 2024-04-27T04:22:09.849 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-04-27T04:22:09.850 INFO:teuthology.orchestra.run.smithi162.stdout:{"epoch":22,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24287,"name":"cephfs.smithi181.bkkcdm","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.181:6827/3756475453","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6826","nonce":3756475453},{"type":"v1","addr":"172.21.15.181:6827","nonce":3756475453}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":14},{"gid":34380,"name":"cephfs.smithi162.gzlyjj","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6827/1198130667","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6826","nonce":1198130667},{"type":"v1","addr":"172.21.15.162:6827","nonce":1198130667}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":19},{"gid":34412,"name":"cephfs.smithi162.xcnapk","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6829/307528848","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6828","nonce":307528848},{"type":"v1","addr":"172.21.15.162:6829","nonce":307528848}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":21}],"filesystems":[{"mdsmap":{"epoch":22,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-27T04:06:53.095749+0000","modified":"2024-04-27T04:19:26.415109+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":83,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{},"failed":[0],"damaged":[],"stopped":[],"info":{},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-04-27T04:22:09.850 INFO:teuthology.orchestra.run.smithi162.stderr:dumped fsmap epoch 22 2024-04-27T04:22:09.852 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:09.853+0000 7f924b554640 1 -- 172.21.15.162:0/2686974446 >> [v2:172.21.15.162:6800/1396866108,v1:172.21.15.162:6801/1396866108] conn(0x7f923006eb70 msgr2=0x7f9230071020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:22:09.852 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:09.853+0000 7f924b554640 1 --2- 172.21.15.162:0/2686974446 >> [v2:172.21.15.162:6800/1396866108,v1:172.21.15.162:6801/1396866108] conn(0x7f923006eb70 0x7f9230071020 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f9244103660 tx=0x7f9240007cb0 comp rx=0 tx=0).stop 2024-04-27T04:22:09.852 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:09.853+0000 7f924b554640 1 -- 172.21.15.162:0/2686974446 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f92441010b0 msgr2=0x7f9244101f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:22:09.852 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:09.853+0000 7f924b554640 1 --2- 172.21.15.162:0/2686974446 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f92441010b0 0x7f9244101f30 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f9234004930 tx=0x7f9234009da0 comp rx=0 tx=0).stop 2024-04-27T04:22:09.852 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:09.853+0000 7f924b554640 1 -- 172.21.15.162:0/2686974446 shutdown_connections 2024-04-27T04:22:09.852 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:09.853+0000 7f924b554640 1 --2- 172.21.15.162:0/2686974446 >> [v2:172.21.15.162:6800/1396866108,v1:172.21.15.162:6801/1396866108] conn(0x7f923006eb70 0x7f9230071020 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:22:09.853 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:09.853+0000 7f924b554640 1 --2- 172.21.15.162:0/2686974446 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9244101a50 0x7f9244102470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:22:09.853 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:09.853+0000 7f924b554640 1 --2- 172.21.15.162:0/2686974446 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f92441010b0 0x7f9244101f30 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:22:09.853 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:09.853+0000 7f924b554640 1 -- 172.21.15.162:0/2686974446 >> 172.21.15.162:0/2686974446 conn(0x7f92440f6c80 msgr2=0x7f9244065790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:22:09.853 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:09.853+0000 7f924b554640 1 -- 172.21.15.162:0/2686974446 shutdown_connections 2024-04-27T04:22:09.853 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:09.854+0000 7f924b554640 1 -- 172.21.15.162:0/2686974446 wait complete. 2024-04-27T04:22:09.863 DEBUG:tasks.fs:allow_standby_replay disabled in epoch 22 2024-04-27T04:22:09.864 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 23 2024-04-27T04:22:09.963 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:09.963+0000 7facf9514640 1 -- 172.21.15.162:0/3075638995 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7facf4101170 msgr2=0x7facf4101540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:22:09.963 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:09.963+0000 7facf9514640 1 --2- 172.21.15.162:0/3075638995 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7facf4101170 0x7facf4101540 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7face40048d0 tx=0x7face4009da0 comp rx=0 tx=0).stop 2024-04-27T04:22:09.963 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:09.963+0000 7facf9514640 1 -- 172.21.15.162:0/3075638995 shutdown_connections 2024-04-27T04:22:09.963 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:09.963+0000 7facf9514640 1 --2- 172.21.15.162:0/3075638995 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7facf40fb150 0x7facf40fb5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:22:09.963 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:09.963+0000 7facf9514640 1 --2- 172.21.15.162:0/3075638995 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7facf4101170 0x7facf4101540 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:22:09.963 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:09.963+0000 7facf9514640 1 -- 172.21.15.162:0/3075638995 >> 172.21.15.162:0/3075638995 conn(0x7facf4071fd0 msgr2=0x7facf40723d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:22:09.963 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:09.963+0000 7facf9514640 1 -- 172.21.15.162:0/3075638995 shutdown_connections 2024-04-27T04:22:09.963 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:09.963+0000 7facf9514640 1 -- 172.21.15.162:0/3075638995 wait complete. 2024-04-27T04:22:09.963 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:09.963+0000 7facf9514640 1 Processor -- start 2024-04-27T04:22:09.963 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:09.964+0000 7facf9514640 1 -- start start 2024-04-27T04:22:09.963 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:09.964+0000 7facf9514640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7facf40fb150 0x7facf410de20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:22:09.964 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:09.964+0000 7facf9514640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7facf4101170 0x7facf410e360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:22:09.964 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:09.964+0000 7facf9514640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7facf410e8a0 con 0x7facf4101170 2024-04-27T04:22:09.964 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:09.964+0000 7facf9514640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7facf410ea10 con 0x7facf40fb150 2024-04-27T04:22:09.964 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:09.964+0000 7facf2ffd640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7facf40fb150 0x7facf410de20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:22:09.964 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:09.964+0000 7facf27fc640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7facf4101170 0x7facf410e360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:22:09.964 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:09.964+0000 7facf2ffd640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7facf40fb150 0x7facf410de20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.162:35124/0 (socket says 172.21.15.162:35124) 2024-04-27T04:22:09.964 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:09.964+0000 7facf2ffd640 1 -- 172.21.15.162:0/513143929 learned_addr learned my addr 172.21.15.162:0/513143929 (peer_addr_for_me v2:172.21.15.162:0/0) 2024-04-27T04:22:09.964 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:09.964+0000 7facf2ffd640 1 -- 172.21.15.162:0/513143929 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7facf4101170 msgr2=0x7facf410e360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:22:09.964 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:09.964+0000 7facf2ffd640 1 --2- 172.21.15.162:0/513143929 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7facf4101170 0x7facf410e360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:22:09.964 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:09.964+0000 7facf2ffd640 1 -- 172.21.15.162:0/513143929 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7face4005040 con 0x7facf40fb150 2024-04-27T04:22:09.964 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:09.964+0000 7facf2ffd640 1 --2- 172.21.15.162:0/513143929 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7facf40fb150 0x7facf410de20 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7face4034040 tx=0x7face4009da0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:22:09.966 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:09.965+0000 7facdb7fe640 1 -- 172.21.15.162:0/513143929 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7face40092b0 con 0x7facf40fb150 2024-04-27T04:22:09.966 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:09.965+0000 7facf9514640 1 -- 172.21.15.162:0/513143929 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7facf410ec90 con 0x7facf40fb150 2024-04-27T04:22:09.966 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:09.965+0000 7facdb7fe640 1 -- 172.21.15.162:0/513143929 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7face403b400 con 0x7facf40fb150 2024-04-27T04:22:09.966 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:09.965+0000 7facdb7fe640 1 -- 172.21.15.162:0/513143929 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7face40435d0 con 0x7facf40fb150 2024-04-27T04:22:09.966 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:09.965+0000 7facf9514640 1 -- 172.21.15.162:0/513143929 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7facf41170e0 con 0x7facf40fb150 2024-04-27T04:22:09.966 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:09.966+0000 7facf9514640 1 -- 172.21.15.162:0/513143929 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7facf40fc870 con 0x7facf40fb150 2024-04-27T04:22:09.966 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:09.966+0000 7facdb7fe640 1 -- 172.21.15.162:0/513143929 <== mon.1 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 39) v1 ==== 92747+0+0 (secure 0 0 0) 0x7face402f070 con 0x7facf40fb150 2024-04-27T04:22:09.966 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:09.966+0000 7facdb7fe640 1 --2- 172.21.15.162:0/513143929 >> [v2:172.21.15.162:6800/1396866108,v1:172.21.15.162:6801/1396866108] conn(0x7facdc06eb70 0x7facdc071020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:22:09.966 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:09.967+0000 7facdb7fe640 1 -- 172.21.15.162:0/513143929 <== mon.1 v2:172.21.15.181:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6582+0+0 (secure 0 0 0) 0x7face40b0490 con 0x7facf40fb150 2024-04-27T04:22:09.966 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:09.967+0000 7facf27fc640 1 --2- 172.21.15.162:0/513143929 >> [v2:172.21.15.162:6800/1396866108,v1:172.21.15.162:6801/1396866108] conn(0x7facdc06eb70 0x7facdc071020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:22:09.966 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:09.967+0000 7facf27fc640 1 --2- 172.21.15.162:0/513143929 >> [v2:172.21.15.162:6800/1396866108,v1:172.21.15.162:6801/1396866108] conn(0x7facdc06eb70 0x7facdc071020 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7facf4064ba0 tx=0x7face800b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:22:09.970 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:09.972+0000 7facdb7fe640 1 -- 172.21.15.162:0/513143929 <== mon.1 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7face4079230 con 0x7facf40fb150 2024-04-27T04:22:10.136 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:22:09 smithi162 ceph-mon[110641]: from='client.? 172.21.15.162:0/264505826' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 19, "format": "json"}]: dispatch 2024-04-27T04:22:10.136 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:22:09 smithi162 ceph-mon[110641]: from='client.? 172.21.15.162:0/4103189192' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 20, "format": "json"}]: dispatch 2024-04-27T04:22:10.136 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:22:09 smithi162 ceph-mon[110641]: from='client.? 172.21.15.162:0/3847378662' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 21, "format": "json"}]: dispatch 2024-04-27T04:22:10.136 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:22:09 smithi162 ceph-mon[110641]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T04:22:10.171 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:22:09 smithi181 ceph-mon[81134]: from='client.? 172.21.15.162:0/264505826' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 19, "format": "json"}]: dispatch 2024-04-27T04:22:10.171 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:22:09 smithi181 ceph-mon[81134]: from='client.? 172.21.15.162:0/4103189192' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 20, "format": "json"}]: dispatch 2024-04-27T04:22:10.171 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:22:09 smithi181 ceph-mon[81134]: from='client.? 172.21.15.162:0/3847378662' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 21, "format": "json"}]: dispatch 2024-04-27T04:22:10.172 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:22:09 smithi181 ceph-mon[81134]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T04:22:10.197 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:10.198+0000 7facf9514640 1 -- 172.21.15.162:0/513143929 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 23, "format": "json"} v 0) v1 -- 0x7facf410ec90 con 0x7facf40fb150 2024-04-27T04:22:10.198 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:10.199+0000 7facdb7fe640 1 -- 172.21.15.162:0/513143929 <== mon.1 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 23, "format": "json"}]=0 dumped fsmap epoch 23 v35) v1 ==== 107+0+4222 (secure 0 0 0) 0x7facf410ec90 con 0x7facf40fb150 2024-04-27T04:22:10.198 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-04-27T04:22:10.198 INFO:teuthology.orchestra.run.smithi162.stdout:{"epoch":23,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":34380,"name":"cephfs.smithi162.gzlyjj","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6827/1198130667","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6826","nonce":1198130667},{"type":"v1","addr":"172.21.15.162:6827","nonce":1198130667}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":19},{"gid":34412,"name":"cephfs.smithi162.xcnapk","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6829/307528848","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6828","nonce":307528848},{"type":"v1","addr":"172.21.15.162:6829","nonce":307528848}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":21}],"filesystems":[{"mdsmap":{"epoch":23,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-27T04:06:53.095749+0000","modified":"2024-04-27T04:19:26.420268+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":83,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":24287},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24287":{"gid":24287,"name":"cephfs.smithi181.bkkcdm","rank":0,"incarnation":23,"state":"up:replay","state_seq":2,"addr":"172.21.15.181:6827/3756475453","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6826","nonce":3756475453},{"type":"v1","addr":"172.21.15.181:6827","nonce":3756475453}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-04-27T04:22:10.199 INFO:teuthology.orchestra.run.smithi162.stderr:dumped fsmap epoch 23 2024-04-27T04:22:10.200 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:10.201+0000 7facf9514640 1 -- 172.21.15.162:0/513143929 >> [v2:172.21.15.162:6800/1396866108,v1:172.21.15.162:6801/1396866108] conn(0x7facdc06eb70 msgr2=0x7facdc071020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:22:10.200 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:10.201+0000 7facf9514640 1 --2- 172.21.15.162:0/513143929 >> [v2:172.21.15.162:6800/1396866108,v1:172.21.15.162:6801/1396866108] conn(0x7facdc06eb70 0x7facdc071020 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7facf4064ba0 tx=0x7face800b040 comp rx=0 tx=0).stop 2024-04-27T04:22:10.200 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:10.201+0000 7facf9514640 1 -- 172.21.15.162:0/513143929 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7facf40fb150 msgr2=0x7facf410de20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:22:10.201 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:10.201+0000 7facf9514640 1 --2- 172.21.15.162:0/513143929 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7facf40fb150 0x7facf410de20 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7face4034040 tx=0x7face4009da0 comp rx=0 tx=0).stop 2024-04-27T04:22:10.201 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:10.201+0000 7facf9514640 1 -- 172.21.15.162:0/513143929 shutdown_connections 2024-04-27T04:22:10.201 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:10.202+0000 7facf9514640 1 --2- 172.21.15.162:0/513143929 >> [v2:172.21.15.162:6800/1396866108,v1:172.21.15.162:6801/1396866108] conn(0x7facdc06eb70 0x7facdc071020 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:22:10.201 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:10.202+0000 7facf9514640 1 --2- 172.21.15.162:0/513143929 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7facf4101170 0x7facf410e360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:22:10.201 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:10.202+0000 7facf9514640 1 --2- 172.21.15.162:0/513143929 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7facf40fb150 0x7facf410de20 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:22:10.201 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:10.202+0000 7facf9514640 1 -- 172.21.15.162:0/513143929 >> 172.21.15.162:0/513143929 conn(0x7facf4071fd0 msgr2=0x7facf40fa330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:22:10.201 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:10.202+0000 7facf9514640 1 -- 172.21.15.162:0/513143929 shutdown_connections 2024-04-27T04:22:10.201 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:10.202+0000 7facf9514640 1 -- 172.21.15.162:0/513143929 wait complete. 2024-04-27T04:22:10.211 DEBUG:tasks.fs:allow_standby_replay disabled in epoch 23 2024-04-27T04:22:10.211 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 24 2024-04-27T04:22:10.315 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:10.314+0000 7f1ea059d640 1 -- 172.21.15.162:0/2313119345 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1e98101170 msgr2=0x7f1e98101540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:22:10.315 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:10.314+0000 7f1ea059d640 1 --2- 172.21.15.162:0/2313119345 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1e98101170 0x7f1e98101540 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f1e940048d0 tx=0x7f1e94009da0 comp rx=0 tx=0).stop 2024-04-27T04:22:10.315 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:10.314+0000 7f1ea059d640 1 -- 172.21.15.162:0/2313119345 shutdown_connections 2024-04-27T04:22:10.316 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:10.314+0000 7f1ea059d640 1 --2- 172.21.15.162:0/2313119345 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1e980fb150 0x7f1e980fb5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:22:10.316 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:10.314+0000 7f1ea059d640 1 --2- 172.21.15.162:0/2313119345 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1e98101170 0x7f1e98101540 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:22:10.316 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:10.314+0000 7f1ea059d640 1 -- 172.21.15.162:0/2313119345 >> 172.21.15.162:0/2313119345 conn(0x7f1e98071fd0 msgr2=0x7f1e980723d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:22:10.316 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:10.314+0000 7f1ea059d640 1 -- 172.21.15.162:0/2313119345 shutdown_connections 2024-04-27T04:22:10.316 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:10.314+0000 7f1ea059d640 1 -- 172.21.15.162:0/2313119345 wait complete. 2024-04-27T04:22:10.316 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:10.315+0000 7f1ea059d640 1 Processor -- start 2024-04-27T04:22:10.316 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:10.315+0000 7f1ea059d640 1 -- start start 2024-04-27T04:22:10.316 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:10.315+0000 7f1ea059d640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1e980fb150 0x7f1e98112c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:22:10.316 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:10.315+0000 7f1ea059d640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1e98101170 0x7f1e981131c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:22:10.316 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:10.315+0000 7f1ea059d640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e981137c0 con 0x7f1e98101170 2024-04-27T04:22:10.316 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:10.315+0000 7f1ea059d640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e9810ce10 con 0x7f1e980fb150 2024-04-27T04:22:10.317 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:10.315+0000 7f1e9db38640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1e98101170 0x7f1e981131c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:22:10.317 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:10.315+0000 7f1e9db38640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1e98101170 0x7f1e981131c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:56984/0 (socket says 172.21.15.162:56984) 2024-04-27T04:22:10.317 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:10.315+0000 7f1e9db38640 1 -- 172.21.15.162:0/903354648 learned_addr learned my addr 172.21.15.162:0/903354648 (peer_addr_for_me v2:172.21.15.162:0/0) 2024-04-27T04:22:10.317 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:10.315+0000 7f1e9e339640 1 --2- 172.21.15.162:0/903354648 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1e980fb150 0x7f1e98112c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:22:10.317 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:10.315+0000 7f1e9db38640 1 -- 172.21.15.162:0/903354648 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1e980fb150 msgr2=0x7f1e98112c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:22:10.317 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:10.315+0000 7f1e9db38640 1 --2- 172.21.15.162:0/903354648 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1e980fb150 0x7f1e98112c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:22:10.317 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:10.315+0000 7f1e9db38640 1 -- 172.21.15.162:0/903354648 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e94005040 con 0x7f1e98101170 2024-04-27T04:22:10.317 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:10.315+0000 7f1e9db38640 1 --2- 172.21.15.162:0/903354648 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1e98101170 0x7f1e981131c0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f1e8800eb70 tx=0x7f1e8800b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:22:10.317 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:10.316+0000 7f1e8effd640 1 -- 172.21.15.162:0/903354648 <== mon.0 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1e88007c30 con 0x7f1e98101170 2024-04-27T04:22:10.317 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:10.316+0000 7f1ea059d640 1 -- 172.21.15.162:0/903354648 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1e9810d090 con 0x7f1e98101170 2024-04-27T04:22:10.317 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:10.316+0000 7f1e8effd640 1 -- 172.21.15.162:0/903354648 <== mon.0 v2:172.21.15.162:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f1e8800f040 con 0x7f1e98101170 2024-04-27T04:22:10.317 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:10.316+0000 7f1e8effd640 1 -- 172.21.15.162:0/903354648 <== mon.0 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1e880095b0 con 0x7f1e98101170 2024-04-27T04:22:10.318 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:10.316+0000 7f1ea059d640 1 -- 172.21.15.162:0/903354648 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1e9810d5b0 con 0x7f1e98101170 2024-04-27T04:22:10.318 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:10.317+0000 7f1ea059d640 1 -- 172.21.15.162:0/903354648 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1e980fc870 con 0x7f1e98101170 2024-04-27T04:22:10.319 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:10.317+0000 7f1e8effd640 1 -- 172.21.15.162:0/903354648 <== mon.0 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 39) v1 ==== 92747+0+0 (secure 0 0 0) 0x7f1e88009730 con 0x7f1e98101170 2024-04-27T04:22:10.319 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:10.318+0000 7f1e8effd640 1 --2- 172.21.15.162:0/903354648 >> [v2:172.21.15.162:6800/1396866108,v1:172.21.15.162:6801/1396866108] conn(0x7f1e8406eb70 0x7f1e84071020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:22:10.319 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:10.318+0000 7f1e9e339640 1 --2- 172.21.15.162:0/903354648 >> [v2:172.21.15.162:6800/1396866108,v1:172.21.15.162:6801/1396866108] conn(0x7f1e8406eb70 0x7f1e84071020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:22:10.319 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:10.318+0000 7f1e9e339640 1 --2- 172.21.15.162:0/903354648 >> [v2:172.21.15.162:6800/1396866108,v1:172.21.15.162:6801/1396866108] conn(0x7f1e8406eb70 0x7f1e84071020 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f1e94003080 tx=0x7f1e94009da0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:22:10.319 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:10.319+0000 7f1e8effd640 1 -- 172.21.15.162:0/903354648 <== mon.0 v2:172.21.15.162:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6582+0+0 (secure 0 0 0) 0x7f1e8805f400 con 0x7f1e98101170 2024-04-27T04:22:10.324 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:10.326+0000 7f1e8effd640 1 -- 172.21.15.162:0/903354648 <== mon.0 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7f1e880573a0 con 0x7f1e98101170 2024-04-27T04:22:10.548 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:10.550+0000 7f1ea059d640 1 -- 172.21.15.162:0/903354648 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 24, "format": "json"} v 0) v1 -- 0x7f1e98108270 con 0x7f1e98101170 2024-04-27T04:22:10.549 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:10.551+0000 7f1e8effd640 1 -- 172.21.15.162:0/903354648 <== mon.0 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 24, "format": "json"}]=0 dumped fsmap epoch 24 v35) v1 ==== 107+0+4227 (secure 0 0 0) 0x7f1e8805aa60 con 0x7f1e98101170 2024-04-27T04:22:10.549 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-04-27T04:22:10.549 INFO:teuthology.orchestra.run.smithi162.stdout:{"epoch":24,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":34380,"name":"cephfs.smithi162.gzlyjj","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6827/1198130667","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6826","nonce":1198130667},{"type":"v1","addr":"172.21.15.162:6827","nonce":1198130667}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":19},{"gid":34412,"name":"cephfs.smithi162.xcnapk","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6829/307528848","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6828","nonce":307528848},{"type":"v1","addr":"172.21.15.162:6829","nonce":307528848}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":21}],"filesystems":[{"mdsmap":{"epoch":24,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-27T04:06:53.095749+0000","modified":"2024-04-27T04:19:35.184930+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":83,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":24287},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24287":{"gid":24287,"name":"cephfs.smithi181.bkkcdm","rank":0,"incarnation":23,"state":"up:reconnect","state_seq":189,"addr":"172.21.15.181:6827/3756475453","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6826","nonce":3756475453},{"type":"v1","addr":"172.21.15.181:6827","nonce":3756475453}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-04-27T04:22:10.550 INFO:teuthology.orchestra.run.smithi162.stderr:dumped fsmap epoch 24 2024-04-27T04:22:10.552 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:10.553+0000 7f1ea059d640 1 -- 172.21.15.162:0/903354648 >> [v2:172.21.15.162:6800/1396866108,v1:172.21.15.162:6801/1396866108] conn(0x7f1e8406eb70 msgr2=0x7f1e84071020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:22:10.552 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:10.553+0000 7f1ea059d640 1 --2- 172.21.15.162:0/903354648 >> [v2:172.21.15.162:6800/1396866108,v1:172.21.15.162:6801/1396866108] conn(0x7f1e8406eb70 0x7f1e84071020 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f1e94003080 tx=0x7f1e94009da0 comp rx=0 tx=0).stop 2024-04-27T04:22:10.552 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:10.553+0000 7f1ea059d640 1 -- 172.21.15.162:0/903354648 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1e98101170 msgr2=0x7f1e981131c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:22:10.552 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:10.553+0000 7f1ea059d640 1 --2- 172.21.15.162:0/903354648 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1e98101170 0x7f1e981131c0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f1e8800eb70 tx=0x7f1e8800b040 comp rx=0 tx=0).stop 2024-04-27T04:22:10.552 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:10.553+0000 7f1ea059d640 1 -- 172.21.15.162:0/903354648 shutdown_connections 2024-04-27T04:22:10.552 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:10.553+0000 7f1ea059d640 1 --2- 172.21.15.162:0/903354648 >> [v2:172.21.15.162:6800/1396866108,v1:172.21.15.162:6801/1396866108] conn(0x7f1e8406eb70 0x7f1e84071020 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:22:10.552 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:10.553+0000 7f1ea059d640 1 --2- 172.21.15.162:0/903354648 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1e98101170 0x7f1e981131c0 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:22:10.552 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:10.553+0000 7f1ea059d640 1 --2- 172.21.15.162:0/903354648 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1e980fb150 0x7f1e98112c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:22:10.552 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:10.553+0000 7f1ea059d640 1 -- 172.21.15.162:0/903354648 >> 172.21.15.162:0/903354648 conn(0x7f1e98071fd0 msgr2=0x7f1e98064690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:22:10.552 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:10.553+0000 7f1ea059d640 1 -- 172.21.15.162:0/903354648 shutdown_connections 2024-04-27T04:22:10.553 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:10.553+0000 7f1ea059d640 1 -- 172.21.15.162:0/903354648 wait complete. 2024-04-27T04:22:10.563 DEBUG:tasks.fs:allow_standby_replay disabled in epoch 24 2024-04-27T04:22:10.563 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 25 2024-04-27T04:22:10.662 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:10.661+0000 7fae04221640 1 -- 172.21.15.162:0/3431531734 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fadfc101170 msgr2=0x7fadfc101540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:22:10.662 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:10.661+0000 7fae04221640 1 --2- 172.21.15.162:0/3431531734 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fadfc101170 0x7fadfc101540 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fadec0048d0 tx=0x7fadec009da0 comp rx=0 tx=0).stop 2024-04-27T04:22:10.662 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:10.662+0000 7fae04221640 1 -- 172.21.15.162:0/3431531734 shutdown_connections 2024-04-27T04:22:10.662 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:10.662+0000 7fae04221640 1 --2- 172.21.15.162:0/3431531734 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fadfc0fb150 0x7fadfc0fb5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:22:10.662 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:10.662+0000 7fae04221640 1 --2- 172.21.15.162:0/3431531734 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fadfc101170 0x7fadfc101540 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:22:10.662 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:10.662+0000 7fae04221640 1 -- 172.21.15.162:0/3431531734 >> 172.21.15.162:0/3431531734 conn(0x7fadfc071fd0 msgr2=0x7fadfc0723d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:22:10.662 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:10.662+0000 7fae04221640 1 -- 172.21.15.162:0/3431531734 shutdown_connections 2024-04-27T04:22:10.663 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:10.662+0000 7fae04221640 1 -- 172.21.15.162:0/3431531734 wait complete. 2024-04-27T04:22:10.663 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:10.662+0000 7fae04221640 1 Processor -- start 2024-04-27T04:22:10.663 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:10.662+0000 7fae04221640 1 -- start start 2024-04-27T04:22:10.663 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:10.662+0000 7fae04221640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fadfc0fb150 0x7fadfc112c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:22:10.663 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:10.662+0000 7fae04221640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fadfc101170 0x7fadfc113190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:22:10.663 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:10.662+0000 7fae04221640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fadfc113790 con 0x7fadfc101170 2024-04-27T04:22:10.663 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:10.662+0000 7fae04221640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fadfc10cde0 con 0x7fadfc0fb150 2024-04-27T04:22:10.663 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:10.662+0000 7fae017bc640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fadfc101170 0x7fadfc113190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:22:10.663 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:10.662+0000 7fae017bc640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fadfc101170 0x7fadfc113190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:57006/0 (socket says 172.21.15.162:57006) 2024-04-27T04:22:10.663 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:10.662+0000 7fae017bc640 1 -- 172.21.15.162:0/2533799877 learned_addr learned my addr 172.21.15.162:0/2533799877 (peer_addr_for_me v2:172.21.15.162:0/0) 2024-04-27T04:22:10.663 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:10.663+0000 7fae01fbd640 1 --2- 172.21.15.162:0/2533799877 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fadfc0fb150 0x7fadfc112c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:22:10.663 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:10.663+0000 7fae017bc640 1 -- 172.21.15.162:0/2533799877 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fadfc0fb150 msgr2=0x7fadfc112c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:22:10.663 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:10.663+0000 7fae017bc640 1 --2- 172.21.15.162:0/2533799877 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fadfc0fb150 0x7fadfc112c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:22:10.663 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:10.663+0000 7fae017bc640 1 -- 172.21.15.162:0/2533799877 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fadec005040 con 0x7fadfc101170 2024-04-27T04:22:10.663 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:10.663+0000 7fae017bc640 1 --2- 172.21.15.162:0/2533799877 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fadfc101170 0x7fadfc113190 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fadf800c830 tx=0x7fadf8004430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:22:10.664 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:10.663+0000 7fadf27fc640 1 -- 172.21.15.162:0/2533799877 <== mon.0 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fadf800a850 con 0x7fadfc101170 2024-04-27T04:22:10.664 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:10.663+0000 7fadf27fc640 1 -- 172.21.15.162:0/2533799877 <== mon.0 v2:172.21.15.162:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fadf800ae80 con 0x7fadfc101170 2024-04-27T04:22:10.664 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:10.663+0000 7fae04221640 1 -- 172.21.15.162:0/2533799877 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fadfc10d0c0 con 0x7fadfc101170 2024-04-27T04:22:10.664 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:10.663+0000 7fadf27fc640 1 -- 172.21.15.162:0/2533799877 <== mon.0 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fadf8012ea0 con 0x7fadfc101170 2024-04-27T04:22:10.664 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:10.663+0000 7fae04221640 1 -- 172.21.15.162:0/2533799877 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fadfc10d5e0 con 0x7fadfc101170 2024-04-27T04:22:10.665 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:10.664+0000 7fae04221640 1 -- 172.21.15.162:0/2533799877 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fadfc0fc870 con 0x7fadfc101170 2024-04-27T04:22:10.665 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:10.665+0000 7fadf27fc640 1 -- 172.21.15.162:0/2533799877 <== mon.0 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 39) v1 ==== 92747+0+0 (secure 0 0 0) 0x7fadf8012400 con 0x7fadfc101170 2024-04-27T04:22:10.665 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:10.665+0000 7fadf27fc640 1 --2- 172.21.15.162:0/2533799877 >> [v2:172.21.15.162:6800/1396866108,v1:172.21.15.162:6801/1396866108] conn(0x7fade806eb70 0x7fade8071020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:22:10.665 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:10.666+0000 7fadf27fc640 1 -- 172.21.15.162:0/2533799877 <== mon.0 v2:172.21.15.162:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6582+0+0 (secure 0 0 0) 0x7fadf808ea80 con 0x7fadfc101170 2024-04-27T04:22:10.665 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:10.666+0000 7fae01fbd640 1 --2- 172.21.15.162:0/2533799877 >> [v2:172.21.15.162:6800/1396866108,v1:172.21.15.162:6801/1396866108] conn(0x7fade806eb70 0x7fade8071020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:22:10.665 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:10.666+0000 7fae01fbd640 1 --2- 172.21.15.162:0/2533799877 >> [v2:172.21.15.162:6800/1396866108,v1:172.21.15.162:6801/1396866108] conn(0x7fade806eb70 0x7fade8071020 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fadec003080 tx=0x7fadec009da0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:22:10.673 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:10.674+0000 7fadf27fc640 1 -- 172.21.15.162:0/2533799877 <== mon.0 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7fadf8057850 con 0x7fadfc101170 2024-04-27T04:22:10.921 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:10.922+0000 7fae04221640 1 -- 172.21.15.162:0/2533799877 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 25, "format": "json"} v 0) v1 -- 0x7fadfc063f00 con 0x7fadfc101170 2024-04-27T04:22:10.921 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:10.923+0000 7fadf27fc640 1 -- 172.21.15.162:0/2533799877 <== mon.0 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 25, "format": "json"}]=0 dumped fsmap epoch 25 v35) v1 ==== 107+0+4224 (secure 0 0 0) 0x7fadf80126b0 con 0x7fadfc101170 2024-04-27T04:22:10.922 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-04-27T04:22:10.922 INFO:teuthology.orchestra.run.smithi162.stdout:{"epoch":25,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":34380,"name":"cephfs.smithi162.gzlyjj","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6827/1198130667","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6826","nonce":1198130667},{"type":"v1","addr":"172.21.15.162:6827","nonce":1198130667}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":19},{"gid":34412,"name":"cephfs.smithi162.xcnapk","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6829/307528848","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6828","nonce":307528848},{"type":"v1","addr":"172.21.15.162:6829","nonce":307528848}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":21}],"filesystems":[{"mdsmap":{"epoch":25,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-27T04:06:53.095749+0000","modified":"2024-04-27T04:19:35.770396+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":83,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":24287},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24287":{"gid":24287,"name":"cephfs.smithi181.bkkcdm","rank":0,"incarnation":23,"state":"up:rejoin","state_seq":190,"addr":"172.21.15.181:6827/3756475453","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6826","nonce":3756475453},{"type":"v1","addr":"172.21.15.181:6827","nonce":3756475453}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-04-27T04:22:10.922 INFO:teuthology.orchestra.run.smithi162.stderr:dumped fsmap epoch 25 2024-04-27T04:22:10.924 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:10.925+0000 7fae04221640 1 -- 172.21.15.162:0/2533799877 >> [v2:172.21.15.162:6800/1396866108,v1:172.21.15.162:6801/1396866108] conn(0x7fade806eb70 msgr2=0x7fade8071020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:22:10.924 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:10.925+0000 7fae04221640 1 --2- 172.21.15.162:0/2533799877 >> [v2:172.21.15.162:6800/1396866108,v1:172.21.15.162:6801/1396866108] conn(0x7fade806eb70 0x7fade8071020 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fadec003080 tx=0x7fadec009da0 comp rx=0 tx=0).stop 2024-04-27T04:22:10.924 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:10.925+0000 7fae04221640 1 -- 172.21.15.162:0/2533799877 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fadfc101170 msgr2=0x7fadfc113190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:22:10.924 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:10.925+0000 7fae04221640 1 --2- 172.21.15.162:0/2533799877 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fadfc101170 0x7fadfc113190 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fadf800c830 tx=0x7fadf8004430 comp rx=0 tx=0).stop 2024-04-27T04:22:10.924 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:10.925+0000 7fae04221640 1 -- 172.21.15.162:0/2533799877 shutdown_connections 2024-04-27T04:22:10.924 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:10.925+0000 7fae04221640 1 --2- 172.21.15.162:0/2533799877 >> [v2:172.21.15.162:6800/1396866108,v1:172.21.15.162:6801/1396866108] conn(0x7fade806eb70 0x7fade8071020 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:22:10.924 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:10.925+0000 7fae04221640 1 --2- 172.21.15.162:0/2533799877 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fadfc101170 0x7fadfc113190 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:22:10.924 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:10.925+0000 7fae04221640 1 --2- 172.21.15.162:0/2533799877 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fadfc0fb150 0x7fadfc112c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:22:10.924 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:10.925+0000 7fae04221640 1 -- 172.21.15.162:0/2533799877 >> 172.21.15.162:0/2533799877 conn(0x7fadfc071fd0 msgr2=0x7fadfc0646c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:22:10.924 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:10.925+0000 7fae04221640 1 -- 172.21.15.162:0/2533799877 shutdown_connections 2024-04-27T04:22:10.925 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:10.925+0000 7fae04221640 1 -- 172.21.15.162:0/2533799877 wait complete. 2024-04-27T04:22:10.936 DEBUG:tasks.fs:allow_standby_replay disabled in epoch 25 2024-04-27T04:22:10.936 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 26 2024-04-27T04:22:11.036 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:11.035+0000 7f1c4b19d640 1 -- 172.21.15.162:0/3685835975 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1c440fb150 msgr2=0x7f1c440fb5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:22:11.036 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:11.035+0000 7f1c4b19d640 1 --2- 172.21.15.162:0/3685835975 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1c440fb150 0x7f1c440fb5c0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f1c300048d0 tx=0x7f1c30009da0 comp rx=0 tx=0).stop 2024-04-27T04:22:11.036 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:11.035+0000 7f1c4b19d640 1 -- 172.21.15.162:0/3685835975 shutdown_connections 2024-04-27T04:22:11.036 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:11.035+0000 7f1c4b19d640 1 --2- 172.21.15.162:0/3685835975 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1c440fb150 0x7f1c440fb5c0 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:22:11.036 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:11.035+0000 7f1c4b19d640 1 --2- 172.21.15.162:0/3685835975 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1c44101170 0x7f1c44101540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:22:11.036 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:11.035+0000 7f1c4b19d640 1 -- 172.21.15.162:0/3685835975 >> 172.21.15.162:0/3685835975 conn(0x7f1c44071fd0 msgr2=0x7f1c440723d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:22:11.036 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:11.035+0000 7f1c4b19d640 1 -- 172.21.15.162:0/3685835975 shutdown_connections 2024-04-27T04:22:11.036 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:11.035+0000 7f1c4b19d640 1 -- 172.21.15.162:0/3685835975 wait complete. 2024-04-27T04:22:11.037 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:11.036+0000 7f1c4b19d640 1 Processor -- start 2024-04-27T04:22:11.037 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:11.036+0000 7f1c4b19d640 1 -- start start 2024-04-27T04:22:11.037 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:11.036+0000 7f1c4b19d640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1c440fb150 0x7f1c44111b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:22:11.037 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:11.036+0000 7f1c4b19d640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1c44101170 0x7f1c441120c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:22:11.037 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:11.036+0000 7f1c4b19d640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c441126c0 con 0x7f1c44101170 2024-04-27T04:22:11.037 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:11.036+0000 7f1c4b19d640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c4410bd10 con 0x7f1c440fb150 2024-04-27T04:22:11.037 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:11.036+0000 7f1c43fff640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1c44101170 0x7f1c441120c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:22:11.037 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:11.036+0000 7f1c43fff640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1c44101170 0x7f1c441120c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:57014/0 (socket says 172.21.15.162:57014) 2024-04-27T04:22:11.037 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:11.036+0000 7f1c43fff640 1 -- 172.21.15.162:0/4110901121 learned_addr learned my addr 172.21.15.162:0/4110901121 (peer_addr_for_me v2:172.21.15.162:0/0) 2024-04-27T04:22:11.037 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:11.036+0000 7f1c48f39640 1 --2- 172.21.15.162:0/4110901121 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1c440fb150 0x7f1c44111b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:22:11.037 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:11.037+0000 7f1c43fff640 1 -- 172.21.15.162:0/4110901121 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1c440fb150 msgr2=0x7f1c44111b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:22:11.037 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:11.037+0000 7f1c43fff640 1 --2- 172.21.15.162:0/4110901121 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1c440fb150 0x7f1c44111b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:22:11.037 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:11.037+0000 7f1c43fff640 1 -- 172.21.15.162:0/4110901121 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c30005040 con 0x7f1c44101170 2024-04-27T04:22:11.037 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:11.037+0000 7f1c43fff640 1 --2- 172.21.15.162:0/4110901121 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1c44101170 0x7f1c441120c0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f1c30034040 tx=0x7f1c30009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:22:11.038 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:11.037+0000 7f1c417fa640 1 -- 172.21.15.162:0/4110901121 <== mon.0 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1c300092b0 con 0x7f1c44101170 2024-04-27T04:22:11.038 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:11.037+0000 7f1c417fa640 1 -- 172.21.15.162:0/4110901121 <== mon.0 v2:172.21.15.162:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f1c3003b400 con 0x7f1c44101170 2024-04-27T04:22:11.038 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:11.037+0000 7f1c4b19d640 1 -- 172.21.15.162:0/4110901121 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1c4410bf90 con 0x7f1c44101170 2024-04-27T04:22:11.038 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:11.037+0000 7f1c417fa640 1 -- 172.21.15.162:0/4110901121 <== mon.0 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1c300435d0 con 0x7f1c44101170 2024-04-27T04:22:11.038 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:11.037+0000 7f1c4b19d640 1 -- 172.21.15.162:0/4110901121 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1c4410c450 con 0x7f1c44101170 2024-04-27T04:22:11.039 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:11.038+0000 7f1c4b19d640 1 -- 172.21.15.162:0/4110901121 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1c440fc870 con 0x7f1c44101170 2024-04-27T04:22:11.039 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:11.038+0000 7f1c417fa640 1 -- 172.21.15.162:0/4110901121 <== mon.0 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 39) v1 ==== 92747+0+0 (secure 0 0 0) 0x7f1c3002f070 con 0x7f1c44101170 2024-04-27T04:22:11.039 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:11.039+0000 7f1c417fa640 1 --2- 172.21.15.162:0/4110901121 >> [v2:172.21.15.162:6800/1396866108,v1:172.21.15.162:6801/1396866108] conn(0x7f1c3406eb70 0x7f1c34071020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:22:11.039 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:11.039+0000 7f1c417fa640 1 -- 172.21.15.162:0/4110901121 <== mon.0 v2:172.21.15.162:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6582+0+0 (secure 0 0 0) 0x7f1c300b05c0 con 0x7f1c44101170 2024-04-27T04:22:11.039 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:11.039+0000 7f1c48f39640 1 --2- 172.21.15.162:0/4110901121 >> [v2:172.21.15.162:6800/1396866108,v1:172.21.15.162:6801/1396866108] conn(0x7f1c3406eb70 0x7f1c34071020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:22:11.039 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:11.039+0000 7f1c48f39640 1 --2- 172.21.15.162:0/4110901121 >> [v2:172.21.15.162:6800/1396866108,v1:172.21.15.162:6801/1396866108] conn(0x7f1c3406eb70 0x7f1c34071020 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f1c38009af0 tx=0x7f1c3800b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:22:11.044 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:11.045+0000 7f1c417fa640 1 -- 172.21.15.162:0/4110901121 <== mon.0 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7f1c300793c0 con 0x7f1c44101170 2024-04-27T04:22:11.136 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:22:10 smithi162 ceph-mon[110641]: pgmap v274: 65 pgs: 65 active+clean; 208 MiB data, 851 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-04-27T04:22:11.136 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:22:10 smithi162 ceph-mon[110641]: from='client.? 172.21.15.162:0/2686974446' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 22, "format": "json"}]: dispatch 2024-04-27T04:22:11.136 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:22:10 smithi162 ceph-mon[110641]: from='client.? 172.21.15.162:0/513143929' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 23, "format": "json"}]: dispatch 2024-04-27T04:22:11.136 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:22:10 smithi162 ceph-mon[110641]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' 2024-04-27T04:22:11.136 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:22:10 smithi162 ceph-mon[110641]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' 2024-04-27T04:22:11.136 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:22:10 smithi162 ceph-mon[110641]: from='client.? 172.21.15.162:0/903354648' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 24, "format": "json"}]: dispatch 2024-04-27T04:22:11.171 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:22:10 smithi181 ceph-mon[81134]: pgmap v274: 65 pgs: 65 active+clean; 208 MiB data, 851 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-04-27T04:22:11.171 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:22:10 smithi181 ceph-mon[81134]: from='client.? 172.21.15.162:0/2686974446' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 22, "format": "json"}]: dispatch 2024-04-27T04:22:11.171 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:22:10 smithi181 ceph-mon[81134]: from='client.? 172.21.15.162:0/513143929' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 23, "format": "json"}]: dispatch 2024-04-27T04:22:11.171 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:22:10 smithi181 ceph-mon[81134]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' 2024-04-27T04:22:11.172 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:22:10 smithi181 ceph-mon[81134]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' 2024-04-27T04:22:11.172 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:22:10 smithi181 ceph-mon[81134]: from='client.? 172.21.15.162:0/903354648' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 24, "format": "json"}]: dispatch 2024-04-27T04:22:11.258 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:11.260+0000 7f1c4b19d640 1 -- 172.21.15.162:0/4110901121 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 26, "format": "json"} v 0) v1 -- 0x7f1c441082a0 con 0x7f1c44101170 2024-04-27T04:22:11.259 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:11.260+0000 7f1c417fa640 1 -- 172.21.15.162:0/4110901121 <== mon.0 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 26, "format": "json"}]=0 dumped fsmap epoch 26 v35) v1 ==== 107+0+4233 (secure 0 0 0) 0x7f1c3007ca80 con 0x7f1c44101170 2024-04-27T04:22:11.259 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-04-27T04:22:11.259 INFO:teuthology.orchestra.run.smithi162.stdout:{"epoch":26,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":34380,"name":"cephfs.smithi162.gzlyjj","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6827/1198130667","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6826","nonce":1198130667},{"type":"v1","addr":"172.21.15.162:6827","nonce":1198130667}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":19},{"gid":34412,"name":"cephfs.smithi162.xcnapk","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6829/307528848","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6828","nonce":307528848},{"type":"v1","addr":"172.21.15.162:6829","nonce":307528848}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":21}],"filesystems":[{"mdsmap":{"epoch":26,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-27T04:06:53.095749+0000","modified":"2024-04-27T04:19:37.776313+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":83,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":24287},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24287":{"gid":24287,"name":"cephfs.smithi181.bkkcdm","rank":0,"incarnation":23,"state":"up:active","state_seq":191,"addr":"172.21.15.181:6827/3756475453","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6826","nonce":3756475453},{"type":"v1","addr":"172.21.15.181:6827","nonce":3756475453}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":24287,"qdb_cluster":[24287]},"id":1}]} 2024-04-27T04:22:11.259 INFO:teuthology.orchestra.run.smithi162.stderr:dumped fsmap epoch 26 2024-04-27T04:22:11.261 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:11.262+0000 7f1c4b19d640 1 -- 172.21.15.162:0/4110901121 >> [v2:172.21.15.162:6800/1396866108,v1:172.21.15.162:6801/1396866108] conn(0x7f1c3406eb70 msgr2=0x7f1c34071020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:22:11.261 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:11.262+0000 7f1c4b19d640 1 --2- 172.21.15.162:0/4110901121 >> [v2:172.21.15.162:6800/1396866108,v1:172.21.15.162:6801/1396866108] conn(0x7f1c3406eb70 0x7f1c34071020 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f1c38009af0 tx=0x7f1c3800b040 comp rx=0 tx=0).stop 2024-04-27T04:22:11.261 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:11.262+0000 7f1c4b19d640 1 -- 172.21.15.162:0/4110901121 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1c44101170 msgr2=0x7f1c441120c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:22:11.261 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:11.262+0000 7f1c4b19d640 1 --2- 172.21.15.162:0/4110901121 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1c44101170 0x7f1c441120c0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f1c30034040 tx=0x7f1c30009da0 comp rx=0 tx=0).stop 2024-04-27T04:22:11.262 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:11.262+0000 7f1c4b19d640 1 -- 172.21.15.162:0/4110901121 shutdown_connections 2024-04-27T04:22:11.262 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:11.262+0000 7f1c4b19d640 1 --2- 172.21.15.162:0/4110901121 >> [v2:172.21.15.162:6800/1396866108,v1:172.21.15.162:6801/1396866108] conn(0x7f1c3406eb70 0x7f1c34071020 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:22:11.262 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:11.262+0000 7f1c4b19d640 1 --2- 172.21.15.162:0/4110901121 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1c44101170 0x7f1c441120c0 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:22:11.262 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:11.262+0000 7f1c4b19d640 1 --2- 172.21.15.162:0/4110901121 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1c440fb150 0x7f1c44111b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:22:11.262 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:11.263+0000 7f1c4b19d640 1 -- 172.21.15.162:0/4110901121 >> 172.21.15.162:0/4110901121 conn(0x7f1c44071fd0 msgr2=0x7f1c440fa300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:22:11.262 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:11.263+0000 7f1c4b19d640 1 -- 172.21.15.162:0/4110901121 shutdown_connections 2024-04-27T04:22:11.262 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:11.263+0000 7f1c4b19d640 1 -- 172.21.15.162:0/4110901121 wait complete. 2024-04-27T04:22:11.272 DEBUG:tasks.fs:allow_standby_replay disabled in epoch 26 2024-04-27T04:22:11.272 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 27 2024-04-27T04:22:11.369 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:11.368+0000 7ff11015d640 1 -- 172.21.15.162:0/1417436073 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff1081021a0 msgr2=0x7ff108102570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:22:11.369 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:11.368+0000 7ff11015d640 1 --2- 172.21.15.162:0/1417436073 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff1081021a0 0x7ff108102570 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7ff100009c50 tx=0x7ff100004d10 comp rx=0 tx=0).stop 2024-04-27T04:22:11.369 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:11.368+0000 7ff11015d640 1 -- 172.21.15.162:0/1417436073 shutdown_connections 2024-04-27T04:22:11.369 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:11.368+0000 7ff11015d640 1 --2- 172.21.15.162:0/1417436073 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff108102b40 0x7ff1080fc1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:22:11.369 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:11.368+0000 7ff11015d640 1 --2- 172.21.15.162:0/1417436073 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff1081021a0 0x7ff108102570 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:22:11.369 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:11.368+0000 7ff11015d640 1 -- 172.21.15.162:0/1417436073 >> 172.21.15.162:0/1417436073 conn(0x7ff1080f7d50 msgr2=0x7ff1080fa160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:22:11.369 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:11.369+0000 7ff11015d640 1 -- 172.21.15.162:0/1417436073 shutdown_connections 2024-04-27T04:22:11.370 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:11.369+0000 7ff11015d640 1 -- 172.21.15.162:0/1417436073 wait complete. 2024-04-27T04:22:11.370 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:11.369+0000 7ff11015d640 1 Processor -- start 2024-04-27T04:22:11.370 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:11.369+0000 7ff11015d640 1 -- start start 2024-04-27T04:22:11.370 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:11.369+0000 7ff11015d640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff1081021a0 0x7ff108112cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:22:11.370 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:11.369+0000 7ff11015d640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff108102b40 0x7ff1081131f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:22:11.370 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:11.369+0000 7ff11015d640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1081137f0 con 0x7ff108102b40 2024-04-27T04:22:11.370 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:11.369+0000 7ff11015d640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff10810ce40 con 0x7ff1081021a0 2024-04-27T04:22:11.370 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:11.369+0000 7ff10d6f8640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff108102b40 0x7ff1081131f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:22:11.370 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:11.369+0000 7ff10d6f8640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff108102b40 0x7ff1081131f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:57036/0 (socket says 172.21.15.162:57036) 2024-04-27T04:22:11.370 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:11.369+0000 7ff10d6f8640 1 -- 172.21.15.162:0/779991131 learned_addr learned my addr 172.21.15.162:0/779991131 (peer_addr_for_me v2:172.21.15.162:0/0) 2024-04-27T04:22:11.370 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:11.369+0000 7ff10def9640 1 --2- 172.21.15.162:0/779991131 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff1081021a0 0x7ff108112cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:22:11.370 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:11.370+0000 7ff10d6f8640 1 -- 172.21.15.162:0/779991131 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff1081021a0 msgr2=0x7ff108112cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:22:11.370 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:11.370+0000 7ff10d6f8640 1 --2- 172.21.15.162:0/779991131 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff1081021a0 0x7ff108112cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:22:11.371 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:11.370+0000 7ff10d6f8640 1 -- 172.21.15.162:0/779991131 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff1000098b0 con 0x7ff108102b40 2024-04-27T04:22:11.371 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:11.370+0000 7ff10def9640 1 --2- 172.21.15.162:0/779991131 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff1081021a0 0x7ff108112cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-27T04:22:11.371 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:11.370+0000 7ff10d6f8640 1 --2- 172.21.15.162:0/779991131 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff108102b40 0x7ff1081131f0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7ff0f8002a50 tx=0x7ff0f8004600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:22:11.371 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:11.370+0000 7ff0fe7fc640 1 -- 172.21.15.162:0/779991131 <== mon.0 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff0f8007c30 con 0x7ff108102b40 2024-04-27T04:22:11.371 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:11.370+0000 7ff11015d640 1 -- 172.21.15.162:0/779991131 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff10810d120 con 0x7ff108102b40 2024-04-27T04:22:11.371 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:11.370+0000 7ff0fe7fc640 1 -- 172.21.15.162:0/779991131 <== mon.0 v2:172.21.15.162:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7ff0f800f040 con 0x7ff108102b40 2024-04-27T04:22:11.371 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:11.370+0000 7ff0fe7fc640 1 -- 172.21.15.162:0/779991131 <== mon.0 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff0f800db50 con 0x7ff108102b40 2024-04-27T04:22:11.371 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:11.370+0000 7ff11015d640 1 -- 172.21.15.162:0/779991131 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff10810d6c0 con 0x7ff108102b40 2024-04-27T04:22:11.372 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:11.371+0000 7ff11015d640 1 -- 172.21.15.162:0/779991131 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff1080fd910 con 0x7ff108102b40 2024-04-27T04:22:11.372 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:11.371+0000 7ff0fe7fc640 1 -- 172.21.15.162:0/779991131 <== mon.0 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 39) v1 ==== 92747+0+0 (secure 0 0 0) 0x7ff0f801c400 con 0x7ff108102b40 2024-04-27T04:22:11.372 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:11.372+0000 7ff0fe7fc640 1 --2- 172.21.15.162:0/779991131 >> [v2:172.21.15.162:6800/1396866108,v1:172.21.15.162:6801/1396866108] conn(0x7ff0f406eb70 0x7ff0f4071020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:22:11.372 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:11.372+0000 7ff10def9640 1 --2- 172.21.15.162:0/779991131 >> [v2:172.21.15.162:6800/1396866108,v1:172.21.15.162:6801/1396866108] conn(0x7ff0f406eb70 0x7ff0f4071020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:22:11.372 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:11.372+0000 7ff0fe7fc640 1 -- 172.21.15.162:0/779991131 <== mon.0 v2:172.21.15.162:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6582+0+0 (secure 0 0 0) 0x7ff0f808e1f0 con 0x7ff108102b40 2024-04-27T04:22:11.372 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:11.372+0000 7ff10def9640 1 --2- 172.21.15.162:0/779991131 >> [v2:172.21.15.162:6800/1396866108,v1:172.21.15.162:6801/1396866108] conn(0x7ff0f406eb70 0x7ff0f4071020 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7ff100000f80 tx=0x7ff10002e040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:22:11.375 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:11.376+0000 7ff0fe7fc640 1 -- 172.21.15.162:0/779991131 <== mon.0 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7ff0f8056f40 con 0x7ff108102b40 2024-04-27T04:22:11.589 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:11.591+0000 7ff11015d640 1 -- 172.21.15.162:0/779991131 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 27, "format": "json"} v 0) v1 -- 0x7ff108063eb0 con 0x7ff108102b40 2024-04-27T04:22:11.590 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:11.592+0000 7ff0fe7fc640 1 -- 172.21.15.162:0/779991131 <== mon.0 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 27, "format": "json"}]=0 dumped fsmap epoch 27 v35) v1 ==== 107+0+5083 (secure 0 0 0) 0x7ff0f805a600 con 0x7ff108102b40 2024-04-27T04:22:11.590 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-04-27T04:22:11.590 INFO:teuthology.orchestra.run.smithi162.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":34380,"name":"cephfs.smithi162.gzlyjj","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6827/1198130667","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6826","nonce":1198130667},{"type":"v1","addr":"172.21.15.162:6827","nonce":1198130667}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":19},{"gid":34412,"name":"cephfs.smithi162.xcnapk","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6829/307528848","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6828","nonce":307528848},{"type":"v1","addr":"172.21.15.162:6829","nonce":307528848}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":21},{"gid":44319,"name":"cephfs.smithi181.dzomgh","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6825/1667773112","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":1667773112},{"type":"v1","addr":"172.21.15.181:6825","nonce":1667773112}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":27}],"filesystems":[{"mdsmap":{"epoch":26,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-27T04:06:53.095749+0000","modified":"2024-04-27T04:19:37.776313+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":83,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":24287},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24287":{"gid":24287,"name":"cephfs.smithi181.bkkcdm","rank":0,"incarnation":23,"state":"up:active","state_seq":191,"addr":"172.21.15.181:6827/3756475453","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6826","nonce":3756475453},{"type":"v1","addr":"172.21.15.181:6827","nonce":3756475453}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":24287,"qdb_cluster":[24287]},"id":1}]} 2024-04-27T04:22:11.591 INFO:teuthology.orchestra.run.smithi162.stderr:dumped fsmap epoch 27 2024-04-27T04:22:11.592 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:11.594+0000 7ff11015d640 1 -- 172.21.15.162:0/779991131 >> [v2:172.21.15.162:6800/1396866108,v1:172.21.15.162:6801/1396866108] conn(0x7ff0f406eb70 msgr2=0x7ff0f4071020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:22:11.593 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:11.594+0000 7ff11015d640 1 --2- 172.21.15.162:0/779991131 >> [v2:172.21.15.162:6800/1396866108,v1:172.21.15.162:6801/1396866108] conn(0x7ff0f406eb70 0x7ff0f4071020 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7ff100000f80 tx=0x7ff10002e040 comp rx=0 tx=0).stop 2024-04-27T04:22:11.593 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:11.594+0000 7ff11015d640 1 -- 172.21.15.162:0/779991131 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff108102b40 msgr2=0x7ff1081131f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:22:11.593 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:11.594+0000 7ff11015d640 1 --2- 172.21.15.162:0/779991131 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff108102b40 0x7ff1081131f0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7ff0f8002a50 tx=0x7ff0f8004600 comp rx=0 tx=0).stop 2024-04-27T04:22:11.593 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:11.594+0000 7ff11015d640 1 -- 172.21.15.162:0/779991131 shutdown_connections 2024-04-27T04:22:11.593 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:11.594+0000 7ff11015d640 1 --2- 172.21.15.162:0/779991131 >> [v2:172.21.15.162:6800/1396866108,v1:172.21.15.162:6801/1396866108] conn(0x7ff0f406eb70 0x7ff0f4071020 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:22:11.593 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:11.594+0000 7ff11015d640 1 --2- 172.21.15.162:0/779991131 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff108102b40 0x7ff1081131f0 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:22:11.593 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:11.594+0000 7ff11015d640 1 --2- 172.21.15.162:0/779991131 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff1081021a0 0x7ff108112cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:22:11.593 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:11.594+0000 7ff11015d640 1 -- 172.21.15.162:0/779991131 >> 172.21.15.162:0/779991131 conn(0x7ff1080f7d50 msgr2=0x7ff1080f8cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:22:11.593 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:11.594+0000 7ff11015d640 1 -- 172.21.15.162:0/779991131 shutdown_connections 2024-04-27T04:22:11.593 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:11.594+0000 7ff11015d640 1 -- 172.21.15.162:0/779991131 wait complete. 2024-04-27T04:22:11.603 DEBUG:tasks.fs:allow_standby_replay disabled in epoch 27 2024-04-27T04:22:11.603 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 28 2024-04-27T04:22:11.700 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:11.699+0000 7fc13586e640 1 -- 172.21.15.162:0/3949481310 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc1300fb150 msgr2=0x7fc1300fb5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:22:11.700 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:11.699+0000 7fc13586e640 1 --2- 172.21.15.162:0/3949481310 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc1300fb150 0x7fc1300fb5c0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fc1240048d0 tx=0x7fc124009da0 comp rx=0 tx=0).stop 2024-04-27T04:22:11.700 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:11.699+0000 7fc13586e640 1 -- 172.21.15.162:0/3949481310 shutdown_connections 2024-04-27T04:22:11.700 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:11.699+0000 7fc13586e640 1 --2- 172.21.15.162:0/3949481310 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc1300fb150 0x7fc1300fb5c0 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:22:11.700 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:11.699+0000 7fc13586e640 1 --2- 172.21.15.162:0/3949481310 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc130101170 0x7fc130101540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:22:11.701 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:11.699+0000 7fc13586e640 1 -- 172.21.15.162:0/3949481310 >> 172.21.15.162:0/3949481310 conn(0x7fc130071fd0 msgr2=0x7fc1300723d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:22:11.701 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:11.699+0000 7fc13586e640 1 -- 172.21.15.162:0/3949481310 shutdown_connections 2024-04-27T04:22:11.701 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:11.699+0000 7fc13586e640 1 -- 172.21.15.162:0/3949481310 wait complete. 2024-04-27T04:22:11.701 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:11.700+0000 7fc13586e640 1 Processor -- start 2024-04-27T04:22:11.701 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:11.700+0000 7fc13586e640 1 -- start start 2024-04-27T04:22:11.701 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:11.700+0000 7fc13586e640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc1300fb150 0x7fc13010de30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:22:11.701 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:11.700+0000 7fc13586e640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc130101170 0x7fc13010e370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:22:11.701 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:11.700+0000 7fc13586e640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1300652f0 con 0x7fc1300fb150 2024-04-27T04:22:11.701 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:11.700+0000 7fc13586e640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc13010e8b0 con 0x7fc130101170 2024-04-27T04:22:11.701 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:11.700+0000 7fc12effd640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc1300fb150 0x7fc13010de30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:22:11.701 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:11.700+0000 7fc12effd640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc1300fb150 0x7fc13010de30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:57048/0 (socket says 172.21.15.162:57048) 2024-04-27T04:22:11.701 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:11.700+0000 7fc12effd640 1 -- 172.21.15.162:0/3176119604 learned_addr learned my addr 172.21.15.162:0/3176119604 (peer_addr_for_me v2:172.21.15.162:0/0) 2024-04-27T04:22:11.701 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:11.701+0000 7fc12e7fc640 1 --2- 172.21.15.162:0/3176119604 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc130101170 0x7fc13010e370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:22:11.701 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:11.701+0000 7fc12effd640 1 -- 172.21.15.162:0/3176119604 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc130101170 msgr2=0x7fc13010e370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:22:11.702 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:11.701+0000 7fc12effd640 1 --2- 172.21.15.162:0/3176119604 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc130101170 0x7fc13010e370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:22:11.702 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:11.701+0000 7fc12effd640 1 -- 172.21.15.162:0/3176119604 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc124005040 con 0x7fc1300fb150 2024-04-27T04:22:11.702 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:11.701+0000 7fc12e7fc640 1 --2- 172.21.15.162:0/3176119604 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc130101170 0x7fc13010e370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-27T04:22:11.702 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:11.701+0000 7fc12effd640 1 --2- 172.21.15.162:0/3176119604 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc1300fb150 0x7fc13010de30 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fc12000c830 tx=0x7fc120004430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:22:11.702 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:11.701+0000 7fc1177fe640 1 -- 172.21.15.162:0/3176119604 <== mon.0 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc12000a850 con 0x7fc1300fb150 2024-04-27T04:22:11.702 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:11.701+0000 7fc13586e640 1 -- 172.21.15.162:0/3176119604 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc13010eb90 con 0x7fc1300fb150 2024-04-27T04:22:11.702 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:11.701+0000 7fc1177fe640 1 -- 172.21.15.162:0/3176119604 <== mon.0 v2:172.21.15.162:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fc12000ae80 con 0x7fc1300fb150 2024-04-27T04:22:11.702 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:11.701+0000 7fc1177fe640 1 -- 172.21.15.162:0/3176119604 <== mon.0 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc120012ea0 con 0x7fc1300fb150 2024-04-27T04:22:11.702 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:11.701+0000 7fc13586e640 1 -- 172.21.15.162:0/3176119604 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc130117150 con 0x7fc1300fb150 2024-04-27T04:22:11.703 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:11.702+0000 7fc13586e640 1 -- 172.21.15.162:0/3176119604 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc1300fc870 con 0x7fc1300fb150 2024-04-27T04:22:11.703 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:11.702+0000 7fc1177fe640 1 -- 172.21.15.162:0/3176119604 <== mon.0 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 39) v1 ==== 92747+0+0 (secure 0 0 0) 0x7fc120012400 con 0x7fc1300fb150 2024-04-27T04:22:11.703 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:11.703+0000 7fc1177fe640 1 --2- 172.21.15.162:0/3176119604 >> [v2:172.21.15.162:6800/1396866108,v1:172.21.15.162:6801/1396866108] conn(0x7fc11806eb70 0x7fc118071020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:22:11.703 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:11.703+0000 7fc1177fe640 1 -- 172.21.15.162:0/3176119604 <== mon.0 v2:172.21.15.162:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6582+0+0 (secure 0 0 0) 0x7fc12008eb60 con 0x7fc1300fb150 2024-04-27T04:22:11.703 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:11.703+0000 7fc12e7fc640 1 --2- 172.21.15.162:0/3176119604 >> [v2:172.21.15.162:6800/1396866108,v1:172.21.15.162:6801/1396866108] conn(0x7fc11806eb70 0x7fc118071020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:22:11.703 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:11.703+0000 7fc12e7fc640 1 --2- 172.21.15.162:0/3176119604 >> [v2:172.21.15.162:6800/1396866108,v1:172.21.15.162:6801/1396866108] conn(0x7fc11806eb70 0x7fc118071020 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fc124000c00 tx=0x7fc124009da0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:22:11.706 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:11.708+0000 7fc1177fe640 1 -- 172.21.15.162:0/3176119604 <== mon.0 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7fc120057930 con 0x7fc1300fb150 2024-04-27T04:22:11.886 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:22:11 smithi162 ceph-mon[110641]: from='client.? 172.21.15.162:0/2533799877' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 25, "format": "json"}]: dispatch 2024-04-27T04:22:11.886 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:22:11 smithi162 ceph-mon[110641]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T04:22:11.886 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:22:11 smithi162 ceph-mon[110641]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T04:22:11.886 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:22:11 smithi162 ceph-mon[110641]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' 2024-04-27T04:22:11.886 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:22:11 smithi162 ceph-mon[110641]: from='client.? 172.21.15.162:0/4110901121' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 26, "format": "json"}]: dispatch 2024-04-27T04:22:11.886 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:22:11 smithi162 ceph-mon[110641]: from='client.? 172.21.15.162:0/779991131' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 27, "format": "json"}]: dispatch 2024-04-27T04:22:11.961 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:11.963+0000 7fc13586e640 1 -- 172.21.15.162:0/3176119604 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 28, "format": "json"} v 0) v1 -- 0x7fc130063f00 con 0x7fc1300fb150 2024-04-27T04:22:11.962 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:11.963+0000 7fc1177fe640 1 -- 172.21.15.162:0/3176119604 <== mon.0 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 28, "format": "json"}]=0 dumped fsmap epoch 28 v35) v1 ==== 107+0+4279 (secure 0 0 0) 0x7fc1200126b0 con 0x7fc1300fb150 2024-04-27T04:22:11.962 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-04-27T04:22:11.962 INFO:teuthology.orchestra.run.smithi162.stdout:{"epoch":28,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":34380,"name":"cephfs.smithi162.gzlyjj","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6827/1198130667","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6826","nonce":1198130667},{"type":"v1","addr":"172.21.15.162:6827","nonce":1198130667}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":19},{"gid":34412,"name":"cephfs.smithi162.xcnapk","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6829/307528848","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6828","nonce":307528848},{"type":"v1","addr":"172.21.15.162:6829","nonce":307528848}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":21},{"gid":44319,"name":"cephfs.smithi181.dzomgh","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6825/1667773112","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":1667773112},{"type":"v1","addr":"172.21.15.181:6825","nonce":1667773112}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":27}],"filesystems":[{"mdsmap":{"epoch":28,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-27T04:06:53.095749+0000","modified":"2024-04-27T04:19:50.546963+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":84,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{},"failed":[0],"damaged":[],"stopped":[],"info":{},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-04-27T04:22:11.962 INFO:teuthology.orchestra.run.smithi162.stderr:dumped fsmap epoch 28 2024-04-27T04:22:11.964 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:11.966+0000 7fc13586e640 1 -- 172.21.15.162:0/3176119604 >> [v2:172.21.15.162:6800/1396866108,v1:172.21.15.162:6801/1396866108] conn(0x7fc11806eb70 msgr2=0x7fc118071020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:22:11.964 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:11.966+0000 7fc13586e640 1 --2- 172.21.15.162:0/3176119604 >> [v2:172.21.15.162:6800/1396866108,v1:172.21.15.162:6801/1396866108] conn(0x7fc11806eb70 0x7fc118071020 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fc124000c00 tx=0x7fc124009da0 comp rx=0 tx=0).stop 2024-04-27T04:22:11.965 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:11.966+0000 7fc13586e640 1 -- 172.21.15.162:0/3176119604 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc1300fb150 msgr2=0x7fc13010de30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:22:11.965 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:11.966+0000 7fc13586e640 1 --2- 172.21.15.162:0/3176119604 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc1300fb150 0x7fc13010de30 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fc12000c830 tx=0x7fc120004430 comp rx=0 tx=0).stop 2024-04-27T04:22:11.965 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:11.966+0000 7fc13586e640 1 -- 172.21.15.162:0/3176119604 shutdown_connections 2024-04-27T04:22:11.965 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:11.966+0000 7fc13586e640 1 --2- 172.21.15.162:0/3176119604 >> [v2:172.21.15.162:6800/1396866108,v1:172.21.15.162:6801/1396866108] conn(0x7fc11806eb70 0x7fc118071020 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:22:11.965 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:11.966+0000 7fc13586e640 1 --2- 172.21.15.162:0/3176119604 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc130101170 0x7fc13010e370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:22:11.965 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:11.966+0000 7fc13586e640 1 --2- 172.21.15.162:0/3176119604 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc1300fb150 0x7fc13010de30 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:22:11.965 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:11.966+0000 7fc13586e640 1 -- 172.21.15.162:0/3176119604 >> 172.21.15.162:0/3176119604 conn(0x7fc130071fd0 msgr2=0x7fc1300fa2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:22:11.965 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:11.966+0000 7fc13586e640 1 -- 172.21.15.162:0/3176119604 shutdown_connections 2024-04-27T04:22:11.965 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:11.966+0000 7fc13586e640 1 -- 172.21.15.162:0/3176119604 wait complete. 2024-04-27T04:22:11.975 DEBUG:tasks.fs:allow_standby_replay disabled in epoch 28 2024-04-27T04:22:11.975 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 29 2024-04-27T04:22:12.074 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:12.073+0000 7efd24607640 1 -- 172.21.15.162:0/2657510459 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7efd1c101a10 msgr2=0x7efd1c0fb0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:22:12.074 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:12.073+0000 7efd24607640 1 --2- 172.21.15.162:0/2657510459 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7efd1c101a10 0x7efd1c0fb0a0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7efd0c0048d0 tx=0x7efd0c009da0 comp rx=0 tx=0).stop 2024-04-27T04:22:12.074 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:12.073+0000 7efd24607640 1 -- 172.21.15.162:0/2657510459 shutdown_connections 2024-04-27T04:22:12.074 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:12.073+0000 7efd24607640 1 --2- 172.21.15.162:0/2657510459 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7efd1c101a10 0x7efd1c0fb0a0 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:22:12.074 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:12.073+0000 7efd24607640 1 --2- 172.21.15.162:0/2657510459 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7efd1c101070 0x7efd1c101440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:22:12.074 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:12.073+0000 7efd24607640 1 -- 172.21.15.162:0/2657510459 >> 172.21.15.162:0/2657510459 conn(0x7efd1c0f6be0 msgr2=0x7efd1c0f8ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:22:12.074 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:12.073+0000 7efd24607640 1 -- 172.21.15.162:0/2657510459 shutdown_connections 2024-04-27T04:22:12.074 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:12.073+0000 7efd24607640 1 -- 172.21.15.162:0/2657510459 wait complete. 2024-04-27T04:22:12.074 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:12.073+0000 7efd24607640 1 Processor -- start 2024-04-27T04:22:12.074 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:12.073+0000 7efd24607640 1 -- start start 2024-04-27T04:22:12.074 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:12.074+0000 7efd24607640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7efd1c101070 0x7efd1c10de80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:22:12.075 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:12.074+0000 7efd24607640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7efd1c101a10 0x7efd1c10e3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:22:12.075 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:12.074+0000 7efd24607640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd1c066720 con 0x7efd1c101070 2024-04-27T04:22:12.075 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:12.074+0000 7efd24607640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd1c10e900 con 0x7efd1c101a10 2024-04-27T04:22:12.075 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:12.074+0000 7efd223a3640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7efd1c101070 0x7efd1c10de80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:22:12.075 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:12.074+0000 7efd223a3640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7efd1c101070 0x7efd1c10de80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:57072/0 (socket says 172.21.15.162:57072) 2024-04-27T04:22:12.075 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:12.074+0000 7efd223a3640 1 -- 172.21.15.162:0/3263488992 learned_addr learned my addr 172.21.15.162:0/3263488992 (peer_addr_for_me v2:172.21.15.162:0/0) 2024-04-27T04:22:12.075 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:12.074+0000 7efd21ba2640 1 --2- 172.21.15.162:0/3263488992 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7efd1c101a10 0x7efd1c10e3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:22:12.075 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:12.074+0000 7efd223a3640 1 -- 172.21.15.162:0/3263488992 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7efd1c101a10 msgr2=0x7efd1c10e3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:22:12.075 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:12.074+0000 7efd223a3640 1 --2- 172.21.15.162:0/3263488992 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7efd1c101a10 0x7efd1c10e3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:22:12.075 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:12.074+0000 7efd223a3640 1 -- 172.21.15.162:0/3263488992 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd0c005040 con 0x7efd1c101070 2024-04-27T04:22:12.076 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:12.074+0000 7efd223a3640 1 --2- 172.21.15.162:0/3263488992 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7efd1c101070 0x7efd1c10de80 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7efd18002a50 tx=0x7efd18004600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:22:12.076 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:12.074+0000 7efd12ffd640 1 -- 172.21.15.162:0/3263488992 <== mon.0 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7efd18007c30 con 0x7efd1c101070 2024-04-27T04:22:12.076 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:12.074+0000 7efd12ffd640 1 -- 172.21.15.162:0/3263488992 <== mon.0 v2:172.21.15.162:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7efd1800f040 con 0x7efd1c101070 2024-04-27T04:22:12.076 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:12.074+0000 7efd24607640 1 -- 172.21.15.162:0/3263488992 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efd1c10ebe0 con 0x7efd1c101070 2024-04-27T04:22:12.076 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:12.074+0000 7efd12ffd640 1 -- 172.21.15.162:0/3263488992 <== mon.0 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7efd1800db50 con 0x7efd1c101070 2024-04-27T04:22:12.076 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:12.074+0000 7efd24607640 1 -- 172.21.15.162:0/3263488992 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efd1c1171a0 con 0x7efd1c101070 2024-04-27T04:22:12.076 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:12.075+0000 7efd24607640 1 -- 172.21.15.162:0/3263488992 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efd1c0fc7e0 con 0x7efd1c101070 2024-04-27T04:22:12.077 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:12.076+0000 7efd12ffd640 1 -- 172.21.15.162:0/3263488992 <== mon.0 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 39) v1 ==== 92747+0+0 (secure 0 0 0) 0x7efd1800dcf0 con 0x7efd1c101070 2024-04-27T04:22:12.077 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:12.076+0000 7efd12ffd640 1 --2- 172.21.15.162:0/3263488992 >> [v2:172.21.15.162:6800/1396866108,v1:172.21.15.162:6801/1396866108] conn(0x7efd0806eb70 0x7efd08071020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:22:12.077 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:12.076+0000 7efd12ffd640 1 -- 172.21.15.162:0/3263488992 <== mon.0 v2:172.21.15.162:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6582+0+0 (secure 0 0 0) 0x7efd1808e110 con 0x7efd1c101070 2024-04-27T04:22:12.077 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:12.076+0000 7efd21ba2640 1 --2- 172.21.15.162:0/3263488992 >> [v2:172.21.15.162:6800/1396866108,v1:172.21.15.162:6801/1396866108] conn(0x7efd0806eb70 0x7efd08071020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:22:12.077 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:12.077+0000 7efd21ba2640 1 --2- 172.21.15.162:0/3263488992 >> [v2:172.21.15.162:6800/1396866108,v1:172.21.15.162:6801/1396866108] conn(0x7efd0806eb70 0x7efd08071020 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7efd1c065f70 tx=0x7efd0c009da0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:22:12.079 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:12.081+0000 7efd12ffd640 1 -- 172.21.15.162:0/3263488992 <== mon.0 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7efd18056ee0 con 0x7efd1c101070 2024-04-27T04:22:12.171 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:22:11 smithi181 ceph-mon[81134]: from='client.? 172.21.15.162:0/2533799877' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 25, "format": "json"}]: dispatch 2024-04-27T04:22:12.172 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:22:11 smithi181 ceph-mon[81134]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T04:22:12.172 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:22:11 smithi181 ceph-mon[81134]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T04:22:12.172 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:22:11 smithi181 ceph-mon[81134]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' 2024-04-27T04:22:12.172 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:22:11 smithi181 ceph-mon[81134]: from='client.? 172.21.15.162:0/4110901121' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 26, "format": "json"}]: dispatch 2024-04-27T04:22:12.172 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:22:11 smithi181 ceph-mon[81134]: from='client.? 172.21.15.162:0/779991131' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 27, "format": "json"}]: dispatch 2024-04-27T04:22:12.296 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:12.297+0000 7efd24607640 1 -- 172.21.15.162:0/3263488992 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 29, "format": "json"} v 0) v1 -- 0x7efd1c064350 con 0x7efd1c101070 2024-04-27T04:22:12.296 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:12.298+0000 7efd12ffd640 1 -- 172.21.15.162:0/3263488992 <== mon.0 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 29, "format": "json"}]=0 dumped fsmap epoch 29 v35) v1 ==== 107+0+4358 (secure 0 0 0) 0x7efd1805a5a0 con 0x7efd1c101070 2024-04-27T04:22:12.297 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-04-27T04:22:12.297 INFO:teuthology.orchestra.run.smithi162.stdout:{"epoch":29,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":34412,"name":"cephfs.smithi162.xcnapk","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6829/307528848","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6828","nonce":307528848},{"type":"v1","addr":"172.21.15.162:6829","nonce":307528848}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":21},{"gid":44319,"name":"cephfs.smithi181.dzomgh","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6825/1667773112","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":1667773112},{"type":"v1","addr":"172.21.15.181:6825","nonce":1667773112}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":27}],"filesystems":[{"mdsmap":{"epoch":29,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-27T04:06:53.095749+0000","modified":"2024-04-27T04:19:50.551703+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":84,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":34380},"failed":[],"damaged":[],"stopped":[],"info":{"gid_34380":{"gid":34380,"name":"cephfs.smithi162.gzlyjj","rank":0,"incarnation":29,"state":"up:replay","state_seq":1,"addr":"172.21.15.162:6827/1198130667","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6826","nonce":1198130667},{"type":"v1","addr":"172.21.15.162:6827","nonce":1198130667}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-04-27T04:22:12.297 INFO:teuthology.orchestra.run.smithi162.stderr:dumped fsmap epoch 29 2024-04-27T04:22:12.299 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:12.300+0000 7efd24607640 1 -- 172.21.15.162:0/3263488992 >> [v2:172.21.15.162:6800/1396866108,v1:172.21.15.162:6801/1396866108] conn(0x7efd0806eb70 msgr2=0x7efd08071020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:22:12.299 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:12.300+0000 7efd24607640 1 --2- 172.21.15.162:0/3263488992 >> [v2:172.21.15.162:6800/1396866108,v1:172.21.15.162:6801/1396866108] conn(0x7efd0806eb70 0x7efd08071020 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7efd1c065f70 tx=0x7efd0c009da0 comp rx=0 tx=0).stop 2024-04-27T04:22:12.299 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:12.300+0000 7efd24607640 1 -- 172.21.15.162:0/3263488992 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7efd1c101070 msgr2=0x7efd1c10de80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:22:12.299 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:12.300+0000 7efd24607640 1 --2- 172.21.15.162:0/3263488992 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7efd1c101070 0x7efd1c10de80 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7efd18002a50 tx=0x7efd18004600 comp rx=0 tx=0).stop 2024-04-27T04:22:12.299 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:12.300+0000 7efd24607640 1 -- 172.21.15.162:0/3263488992 shutdown_connections 2024-04-27T04:22:12.299 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:12.300+0000 7efd24607640 1 --2- 172.21.15.162:0/3263488992 >> [v2:172.21.15.162:6800/1396866108,v1:172.21.15.162:6801/1396866108] conn(0x7efd0806eb70 0x7efd08071020 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:22:12.299 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:12.300+0000 7efd24607640 1 --2- 172.21.15.162:0/3263488992 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7efd1c101a10 0x7efd1c10e3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:22:12.299 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:12.300+0000 7efd24607640 1 --2- 172.21.15.162:0/3263488992 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7efd1c101070 0x7efd1c10de80 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:22:12.299 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:12.300+0000 7efd24607640 1 -- 172.21.15.162:0/3263488992 >> 172.21.15.162:0/3263488992 conn(0x7efd1c0f6be0 msgr2=0x7efd1c0f7b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:22:12.300 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:12.300+0000 7efd24607640 1 -- 172.21.15.162:0/3263488992 shutdown_connections 2024-04-27T04:22:12.300 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:12.300+0000 7efd24607640 1 -- 172.21.15.162:0/3263488992 wait complete. 2024-04-27T04:22:12.310 DEBUG:tasks.fs:allow_standby_replay disabled in epoch 29 2024-04-27T04:22:12.310 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 30 2024-04-27T04:22:12.405 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:12.405+0000 7f9b6695d640 1 -- 172.21.15.162:0/3438638225 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9b60101a40 msgr2=0x7f9b600fb0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:22:12.406 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:12.405+0000 7f9b6695d640 1 --2- 172.21.15.162:0/3438638225 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9b60101a40 0x7f9b600fb0d0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f9b540048d0 tx=0x7f9b54009da0 comp rx=0 tx=0).stop 2024-04-27T04:22:12.406 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:12.405+0000 7f9b6695d640 1 -- 172.21.15.162:0/3438638225 shutdown_connections 2024-04-27T04:22:12.406 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:12.405+0000 7f9b6695d640 1 --2- 172.21.15.162:0/3438638225 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9b60101a40 0x7f9b600fb0d0 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:22:12.406 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:12.405+0000 7f9b6695d640 1 --2- 172.21.15.162:0/3438638225 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9b601010a0 0x7f9b60101470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:22:12.406 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:12.405+0000 7f9b6695d640 1 -- 172.21.15.162:0/3438638225 >> 172.21.15.162:0/3438638225 conn(0x7f9b600f6c10 msgr2=0x7f9b600f9020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:22:12.406 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:12.405+0000 7f9b6695d640 1 -- 172.21.15.162:0/3438638225 shutdown_connections 2024-04-27T04:22:12.406 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:12.405+0000 7f9b6695d640 1 -- 172.21.15.162:0/3438638225 wait complete. 2024-04-27T04:22:12.406 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:12.406+0000 7f9b6695d640 1 Processor -- start 2024-04-27T04:22:12.406 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:12.406+0000 7f9b6695d640 1 -- start start 2024-04-27T04:22:12.406 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:12.406+0000 7f9b6695d640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9b601010a0 0x7f9b6010de80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:22:12.406 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:12.406+0000 7f9b6695d640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9b60101a40 0x7f9b6010e3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:22:12.406 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:12.406+0000 7f9b6695d640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b60066720 con 0x7f9b60101a40 2024-04-27T04:22:12.407 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:12.406+0000 7f9b6695d640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b6010e900 con 0x7f9b601010a0 2024-04-27T04:22:12.407 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:12.406+0000 7f9b5f7fe640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9b60101a40 0x7f9b6010e3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:22:12.407 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:12.406+0000 7f9b5f7fe640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9b60101a40 0x7f9b6010e3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:57090/0 (socket says 172.21.15.162:57090) 2024-04-27T04:22:12.407 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:12.406+0000 7f9b5f7fe640 1 -- 172.21.15.162:0/1340126231 learned_addr learned my addr 172.21.15.162:0/1340126231 (peer_addr_for_me v2:172.21.15.162:0/0) 2024-04-27T04:22:12.407 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:12.406+0000 7f9b5ffff640 1 --2- 172.21.15.162:0/1340126231 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9b601010a0 0x7f9b6010de80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:22:12.407 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:12.406+0000 7f9b5f7fe640 1 -- 172.21.15.162:0/1340126231 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9b601010a0 msgr2=0x7f9b6010de80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:22:12.407 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:12.406+0000 7f9b5f7fe640 1 --2- 172.21.15.162:0/1340126231 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9b601010a0 0x7f9b6010de80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:22:12.407 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:12.406+0000 7f9b5f7fe640 1 -- 172.21.15.162:0/1340126231 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9b54005040 con 0x7f9b60101a40 2024-04-27T04:22:12.407 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:12.407+0000 7f9b5f7fe640 1 --2- 172.21.15.162:0/1340126231 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9b60101a40 0x7f9b6010e3c0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f9b54004930 tx=0x7f9b54009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:22:12.407 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:12.407+0000 7f9b5cff9640 1 -- 172.21.15.162:0/1340126231 <== mon.0 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9b540094c0 con 0x7f9b60101a40 2024-04-27T04:22:12.407 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:12.407+0000 7f9b5cff9640 1 -- 172.21.15.162:0/1340126231 <== mon.0 v2:172.21.15.162:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f9b54032e50 con 0x7f9b60101a40 2024-04-27T04:22:12.407 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:12.407+0000 7f9b6695d640 1 -- 172.21.15.162:0/1340126231 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9b6010eb80 con 0x7f9b60101a40 2024-04-27T04:22:12.407 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:12.407+0000 7f9b5cff9640 1 -- 172.21.15.162:0/1340126231 <== mon.0 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9b540433b0 con 0x7f9b60101a40 2024-04-27T04:22:12.408 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:12.407+0000 7f9b6695d640 1 -- 172.21.15.162:0/1340126231 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9b601170e0 con 0x7f9b60101a40 2024-04-27T04:22:12.408 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:12.408+0000 7f9b6695d640 1 -- 172.21.15.162:0/1340126231 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9b600fc810 con 0x7f9b60101a40 2024-04-27T04:22:12.409 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:12.408+0000 7f9b5cff9640 1 -- 172.21.15.162:0/1340126231 <== mon.0 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 39) v1 ==== 92747+0+0 (secure 0 0 0) 0x7f9b5402f070 con 0x7f9b60101a40 2024-04-27T04:22:12.409 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:12.409+0000 7f9b5cff9640 1 --2- 172.21.15.162:0/1340126231 >> [v2:172.21.15.162:6800/1396866108,v1:172.21.15.162:6801/1396866108] conn(0x7f9b4806eb70 0x7f9b48071020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:22:12.409 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:12.409+0000 7f9b5ffff640 1 --2- 172.21.15.162:0/1340126231 >> [v2:172.21.15.162:6800/1396866108,v1:172.21.15.162:6801/1396866108] conn(0x7f9b4806eb70 0x7f9b48071020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:22:12.409 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:12.409+0000 7f9b5cff9640 1 -- 172.21.15.162:0/1340126231 <== mon.0 v2:172.21.15.162:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6582+0+0 (secure 0 0 0) 0x7f9b540b0640 con 0x7f9b60101a40 2024-04-27T04:22:12.409 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:12.409+0000 7f9b5ffff640 1 --2- 172.21.15.162:0/1340126231 >> [v2:172.21.15.162:6800/1396866108,v1:172.21.15.162:6801/1396866108] conn(0x7f9b4806eb70 0x7f9b48071020 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f9b50004ed0 tx=0x7f9b5000b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:22:12.412 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:12.413+0000 7f9b5cff9640 1 -- 172.21.15.162:0/1340126231 <== mon.0 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7f9b54079440 con 0x7f9b60101a40 2024-04-27T04:22:12.626 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:12.627+0000 7f9b6695d640 1 -- 172.21.15.162:0/1340126231 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 30, "format": "json"} v 0) v1 -- 0x7f9b60108260 con 0x7f9b60101a40 2024-04-27T04:22:12.626 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:12.628+0000 7f9b5cff9640 1 -- 172.21.15.162:0/1340126231 <== mon.0 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 30, "format": "json"}]=0 dumped fsmap epoch 30 v35) v1 ==== 107+0+5212 (secure 0 0 0) 0x7f9b5407cb00 con 0x7f9b60101a40 2024-04-27T04:22:12.627 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-04-27T04:22:12.627 INFO:teuthology.orchestra.run.smithi162.stdout:{"epoch":30,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":34412,"name":"cephfs.smithi162.xcnapk","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6829/307528848","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6828","nonce":307528848},{"type":"v1","addr":"172.21.15.162:6829","nonce":307528848}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":21},{"gid":44319,"name":"cephfs.smithi181.dzomgh","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6825/1667773112","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":1667773112},{"type":"v1","addr":"172.21.15.181:6825","nonce":1667773112}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":27},{"gid":44323,"name":"cephfs.smithi181.bkkcdm","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6827/3589158285","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6826","nonce":3589158285},{"type":"v1","addr":"172.21.15.181:6827","nonce":3589158285}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":30}],"filesystems":[{"mdsmap":{"epoch":30,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-27T04:06:53.095749+0000","modified":"2024-04-27T04:19:59.538204+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":84,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":34380},"failed":[],"damaged":[],"stopped":[],"info":{"gid_34380":{"gid":34380,"name":"cephfs.smithi162.gzlyjj","rank":0,"incarnation":29,"state":"up:reconnect","state_seq":18,"addr":"172.21.15.162:6827/1198130667","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6826","nonce":1198130667},{"type":"v1","addr":"172.21.15.162:6827","nonce":1198130667}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-04-27T04:22:12.627 INFO:teuthology.orchestra.run.smithi162.stderr:dumped fsmap epoch 30 2024-04-27T04:22:12.629 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:12.630+0000 7f9b6695d640 1 -- 172.21.15.162:0/1340126231 >> [v2:172.21.15.162:6800/1396866108,v1:172.21.15.162:6801/1396866108] conn(0x7f9b4806eb70 msgr2=0x7f9b48071020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:22:12.629 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:12.630+0000 7f9b6695d640 1 --2- 172.21.15.162:0/1340126231 >> [v2:172.21.15.162:6800/1396866108,v1:172.21.15.162:6801/1396866108] conn(0x7f9b4806eb70 0x7f9b48071020 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f9b50004ed0 tx=0x7f9b5000b040 comp rx=0 tx=0).stop 2024-04-27T04:22:12.629 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:12.630+0000 7f9b6695d640 1 -- 172.21.15.162:0/1340126231 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9b60101a40 msgr2=0x7f9b6010e3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:22:12.629 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:12.630+0000 7f9b6695d640 1 --2- 172.21.15.162:0/1340126231 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9b60101a40 0x7f9b6010e3c0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f9b54004930 tx=0x7f9b54009da0 comp rx=0 tx=0).stop 2024-04-27T04:22:12.629 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:12.630+0000 7f9b6695d640 1 -- 172.21.15.162:0/1340126231 shutdown_connections 2024-04-27T04:22:12.629 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:12.630+0000 7f9b6695d640 1 --2- 172.21.15.162:0/1340126231 >> [v2:172.21.15.162:6800/1396866108,v1:172.21.15.162:6801/1396866108] conn(0x7f9b4806eb70 0x7f9b48071020 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:22:12.629 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:12.630+0000 7f9b6695d640 1 --2- 172.21.15.162:0/1340126231 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9b60101a40 0x7f9b6010e3c0 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:22:12.629 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:12.630+0000 7f9b6695d640 1 --2- 172.21.15.162:0/1340126231 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9b601010a0 0x7f9b6010de80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:22:12.629 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:12.630+0000 7f9b6695d640 1 -- 172.21.15.162:0/1340126231 >> 172.21.15.162:0/1340126231 conn(0x7f9b600f6c10 msgr2=0x7f9b600f7bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:22:12.629 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:12.630+0000 7f9b6695d640 1 -- 172.21.15.162:0/1340126231 shutdown_connections 2024-04-27T04:22:12.629 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:12.630+0000 7f9b6695d640 1 -- 172.21.15.162:0/1340126231 wait complete. 2024-04-27T04:22:12.639 DEBUG:tasks.fs:allow_standby_replay disabled in epoch 30 2024-04-27T04:22:12.640 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 31 2024-04-27T04:22:12.736 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:12.736+0000 7fafc355d640 1 -- 172.21.15.162:0/2808567350 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fafbc102420 msgr2=0x7fafbc1027f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:22:12.737 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:12.736+0000 7fafc355d640 1 --2- 172.21.15.162:0/2808567350 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fafbc102420 0x7fafbc1027f0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fafb80048d0 tx=0x7fafb8009da0 comp rx=0 tx=0).stop 2024-04-27T04:22:12.737 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:12.736+0000 7fafc355d640 1 -- 172.21.15.162:0/2808567350 shutdown_connections 2024-04-27T04:22:12.737 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:12.736+0000 7fafc355d640 1 --2- 172.21.15.162:0/2808567350 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fafbc0faa50 0x7fafbc0faea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:22:12.737 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:12.736+0000 7fafc355d640 1 --2- 172.21.15.162:0/2808567350 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fafbc102420 0x7fafbc1027f0 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:22:12.737 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:12.736+0000 7fafc355d640 1 -- 172.21.15.162:0/2808567350 >> 172.21.15.162:0/2808567350 conn(0x7fafbc071990 msgr2=0x7fafbc071d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:22:12.737 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:12.736+0000 7fafc355d640 1 -- 172.21.15.162:0/2808567350 shutdown_connections 2024-04-27T04:22:12.737 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:12.736+0000 7fafc355d640 1 -- 172.21.15.162:0/2808567350 wait complete. 2024-04-27T04:22:12.737 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:12.736+0000 7fafc355d640 1 Processor -- start 2024-04-27T04:22:12.737 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:12.737+0000 7fafc355d640 1 -- start start 2024-04-27T04:22:12.737 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:12.737+0000 7fafc355d640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fafbc0faa50 0x7fafbc112e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:22:12.737 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:12.737+0000 7fafc355d640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fafbc102420 0x7fafbc113360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:22:12.737 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:12.737+0000 7fafc355d640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fafbc113960 con 0x7fafbc102420 2024-04-27T04:22:12.738 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:12.737+0000 7fafc355d640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fafbc113ad0 con 0x7fafbc0faa50 2024-04-27T04:22:12.738 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:12.737+0000 7fafc0af8640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fafbc102420 0x7fafbc113360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:22:12.738 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:12.737+0000 7fafc0af8640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fafbc102420 0x7fafbc113360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:57112/0 (socket says 172.21.15.162:57112) 2024-04-27T04:22:12.738 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:12.737+0000 7fafc0af8640 1 -- 172.21.15.162:0/244734536 learned_addr learned my addr 172.21.15.162:0/244734536 (peer_addr_for_me v2:172.21.15.162:0/0) 2024-04-27T04:22:12.738 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:12.737+0000 7fafc12f9640 1 --2- 172.21.15.162:0/244734536 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fafbc0faa50 0x7fafbc112e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:22:12.738 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:12.737+0000 7fafc0af8640 1 -- 172.21.15.162:0/244734536 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fafbc0faa50 msgr2=0x7fafbc112e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:22:12.738 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:12.737+0000 7fafc0af8640 1 --2- 172.21.15.162:0/244734536 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fafbc0faa50 0x7fafbc112e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:22:12.738 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:12.737+0000 7fafc0af8640 1 -- 172.21.15.162:0/244734536 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fafb8005040 con 0x7fafbc102420 2024-04-27T04:22:12.738 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:12.737+0000 7fafc0af8640 1 --2- 172.21.15.162:0/244734536 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fafbc102420 0x7fafbc113360 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fafac002a80 tx=0x7fafac004600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:22:12.738 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:12.737+0000 7fafb1ffb640 1 -- 172.21.15.162:0/244734536 <== mon.0 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fafac007c30 con 0x7fafbc102420 2024-04-27T04:22:12.738 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:12.737+0000 7fafc355d640 1 -- 172.21.15.162:0/244734536 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fafbc10d1d0 con 0x7fafbc102420 2024-04-27T04:22:12.738 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:12.737+0000 7fafb1ffb640 1 -- 172.21.15.162:0/244734536 <== mon.0 v2:172.21.15.162:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fafac00f040 con 0x7fafbc102420 2024-04-27T04:22:12.738 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:12.737+0000 7fafb1ffb640 1 -- 172.21.15.162:0/244734536 <== mon.0 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fafac0096b0 con 0x7fafbc102420 2024-04-27T04:22:12.738 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:12.738+0000 7fafc355d640 1 -- 172.21.15.162:0/244734536 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fafbc10d6f0 con 0x7fafbc102420 2024-04-27T04:22:12.739 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:12.739+0000 7fafc355d640 1 -- 172.21.15.162:0/244734536 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fafbc065560 con 0x7fafbc102420 2024-04-27T04:22:12.739 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:12.739+0000 7fafb1ffb640 1 -- 172.21.15.162:0/244734536 <== mon.0 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 39) v1 ==== 92747+0+0 (secure 0 0 0) 0x7fafac00d6b0 con 0x7fafbc102420 2024-04-27T04:22:12.739 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:12.739+0000 7fafb1ffb640 1 --2- 172.21.15.162:0/244734536 >> [v2:172.21.15.162:6800/1396866108,v1:172.21.15.162:6801/1396866108] conn(0x7fafa806eb70 0x7fafa8071020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:22:12.740 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:12.739+0000 7fafb1ffb640 1 -- 172.21.15.162:0/244734536 <== mon.0 v2:172.21.15.162:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6582+0+0 (secure 0 0 0) 0x7fafac08e320 con 0x7fafbc102420 2024-04-27T04:22:12.740 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:12.740+0000 7fafc12f9640 1 --2- 172.21.15.162:0/244734536 >> [v2:172.21.15.162:6800/1396866108,v1:172.21.15.162:6801/1396866108] conn(0x7fafa806eb70 0x7fafa8071020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:22:12.740 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:12.740+0000 7fafc12f9640 1 --2- 172.21.15.162:0/244734536 >> [v2:172.21.15.162:6800/1396866108,v1:172.21.15.162:6801/1396866108] conn(0x7fafa806eb70 0x7fafa8071020 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fafb8003080 tx=0x7fafb8009da0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:22:12.742 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:12.744+0000 7fafb1ffb640 1 -- 172.21.15.162:0/244734536 <== mon.0 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7fafac057070 con 0x7fafbc102420 2024-04-27T04:22:12.956 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:12.958+0000 7fafc355d640 1 -- 172.21.15.162:0/244734536 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 31, "format": "json"} v 0) v1 -- 0x7fafbc064350 con 0x7fafbc102420 2024-04-27T04:22:12.957 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:12.958+0000 7fafb1ffb640 1 -- 172.21.15.162:0/244734536 <== mon.0 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 31, "format": "json"}]=0 dumped fsmap epoch 31 v35) v1 ==== 107+0+5209 (secure 0 0 0) 0x7fafac05a730 con 0x7fafbc102420 2024-04-27T04:22:12.957 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-04-27T04:22:12.957 INFO:teuthology.orchestra.run.smithi162.stdout:{"epoch":31,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":34412,"name":"cephfs.smithi162.xcnapk","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6829/307528848","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6828","nonce":307528848},{"type":"v1","addr":"172.21.15.162:6829","nonce":307528848}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":21},{"gid":44319,"name":"cephfs.smithi181.dzomgh","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6825/1667773112","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":1667773112},{"type":"v1","addr":"172.21.15.181:6825","nonce":1667773112}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":27},{"gid":44323,"name":"cephfs.smithi181.bkkcdm","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6827/3589158285","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6826","nonce":3589158285},{"type":"v1","addr":"172.21.15.181:6827","nonce":3589158285}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":30}],"filesystems":[{"mdsmap":{"epoch":31,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-27T04:06:53.095749+0000","modified":"2024-04-27T04:19:59.749805+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":84,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":34380},"failed":[],"damaged":[],"stopped":[],"info":{"gid_34380":{"gid":34380,"name":"cephfs.smithi162.gzlyjj","rank":0,"incarnation":29,"state":"up:rejoin","state_seq":19,"addr":"172.21.15.162:6827/1198130667","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6826","nonce":1198130667},{"type":"v1","addr":"172.21.15.162:6827","nonce":1198130667}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-04-27T04:22:12.957 INFO:teuthology.orchestra.run.smithi162.stderr:dumped fsmap epoch 31 2024-04-27T04:22:12.959 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:12.961+0000 7fafc355d640 1 -- 172.21.15.162:0/244734536 >> [v2:172.21.15.162:6800/1396866108,v1:172.21.15.162:6801/1396866108] conn(0x7fafa806eb70 msgr2=0x7fafa8071020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:22:12.959 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:12.961+0000 7fafc355d640 1 --2- 172.21.15.162:0/244734536 >> [v2:172.21.15.162:6800/1396866108,v1:172.21.15.162:6801/1396866108] conn(0x7fafa806eb70 0x7fafa8071020 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fafb8003080 tx=0x7fafb8009da0 comp rx=0 tx=0).stop 2024-04-27T04:22:12.959 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:12.961+0000 7fafc355d640 1 -- 172.21.15.162:0/244734536 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fafbc102420 msgr2=0x7fafbc113360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:22:12.959 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:12.961+0000 7fafc355d640 1 --2- 172.21.15.162:0/244734536 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fafbc102420 0x7fafbc113360 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fafac002a80 tx=0x7fafac004600 comp rx=0 tx=0).stop 2024-04-27T04:22:12.960 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:12.961+0000 7fafc355d640 1 -- 172.21.15.162:0/244734536 shutdown_connections 2024-04-27T04:22:12.960 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:12.961+0000 7fafc355d640 1 --2- 172.21.15.162:0/244734536 >> [v2:172.21.15.162:6800/1396866108,v1:172.21.15.162:6801/1396866108] conn(0x7fafa806eb70 0x7fafa8071020 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:22:12.960 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:12.961+0000 7fafc355d640 1 --2- 172.21.15.162:0/244734536 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fafbc102420 0x7fafbc113360 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:22:12.960 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:12.961+0000 7fafc355d640 1 --2- 172.21.15.162:0/244734536 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fafbc0faa50 0x7fafbc112e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:22:12.960 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:12.961+0000 7fafc355d640 1 -- 172.21.15.162:0/244734536 >> 172.21.15.162:0/244734536 conn(0x7fafbc071990 msgr2=0x7fafbc0f9c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:22:12.960 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:12.961+0000 7fafc355d640 1 -- 172.21.15.162:0/244734536 shutdown_connections 2024-04-27T04:22:12.960 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:12.961+0000 7fafc355d640 1 -- 172.21.15.162:0/244734536 wait complete. 2024-04-27T04:22:12.969 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:22:12 smithi162 ceph-mon[110641]: pgmap v275: 65 pgs: 65 active+clean; 208 MiB data, 851 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:22:12.969 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:22:12 smithi162 ceph-mon[110641]: from='client.? 172.21.15.162:0/3176119604' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 28, "format": "json"}]: dispatch 2024-04-27T04:22:12.969 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:22:12 smithi162 ceph-mon[110641]: from='client.? 172.21.15.162:0/3263488992' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 29, "format": "json"}]: dispatch 2024-04-27T04:22:12.969 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:22:12 smithi162 ceph-mon[110641]: from='client.? 172.21.15.162:0/1340126231' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 30, "format": "json"}]: dispatch 2024-04-27T04:22:12.970 DEBUG:tasks.fs:allow_standby_replay disabled in epoch 31 2024-04-27T04:22:12.970 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 32 2024-04-27T04:22:13.066 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:13.066+0000 7f128ca6e640 1 -- 172.21.15.162:0/3153972500 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1288102b60 msgr2=0x7f12880fa8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:22:13.066 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:13.066+0000 7f128ca6e640 1 --2- 172.21.15.162:0/3153972500 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1288102b60 0x7f12880fa8a0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f12780048d0 tx=0x7f1278009da0 comp rx=0 tx=0).stop 2024-04-27T04:22:13.067 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:13.066+0000 7f128ca6e640 1 -- 172.21.15.162:0/3153972500 shutdown_connections 2024-04-27T04:22:13.067 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:13.066+0000 7f128ca6e640 1 --2- 172.21.15.162:0/3153972500 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1288102b60 0x7f12880fa8a0 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:22:13.067 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:13.066+0000 7f128ca6e640 1 --2- 172.21.15.162:0/3153972500 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1288102250 0x7f1288102620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:22:13.067 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:13.066+0000 7f128ca6e640 1 -- 172.21.15.162:0/3153972500 >> 172.21.15.162:0/3153972500 conn(0x7f1288070900 msgr2=0x7f1288070d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:22:13.067 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:13.066+0000 7f128ca6e640 1 -- 172.21.15.162:0/3153972500 shutdown_connections 2024-04-27T04:22:13.067 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:13.066+0000 7f128ca6e640 1 -- 172.21.15.162:0/3153972500 wait complete. 2024-04-27T04:22:13.067 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:13.067+0000 7f128ca6e640 1 Processor -- start 2024-04-27T04:22:13.067 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:13.067+0000 7f128ca6e640 1 -- start start 2024-04-27T04:22:13.067 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:13.067+0000 7f128ca6e640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1288102250 0x7f1288112ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:22:13.067 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:13.067+0000 7f128ca6e640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1288102b60 0x7f12881131e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:22:13.067 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:13.067+0000 7f128ca6e640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12881137e0 con 0x7f1288102b60 2024-04-27T04:22:13.067 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:13.067+0000 7f128ca6e640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f128810ce30 con 0x7f1288102250 2024-04-27T04:22:13.067 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:13.067+0000 7f1285d9b640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1288102b60 0x7f12881131e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:22:13.067 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:13.067+0000 7f1285d9b640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1288102b60 0x7f12881131e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:57132/0 (socket says 172.21.15.162:57132) 2024-04-27T04:22:13.068 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:13.067+0000 7f1285d9b640 1 -- 172.21.15.162:0/3747607846 learned_addr learned my addr 172.21.15.162:0/3747607846 (peer_addr_for_me v2:172.21.15.162:0/0) 2024-04-27T04:22:13.068 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:13.067+0000 7f1285d9b640 1 -- 172.21.15.162:0/3747607846 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1288102250 msgr2=0x7f1288112ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:22:13.068 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:13.067+0000 7f1285d9b640 1 --2- 172.21.15.162:0/3747607846 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1288102250 0x7f1288112ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:22:13.068 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:13.067+0000 7f1285d9b640 1 -- 172.21.15.162:0/3747607846 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1278005040 con 0x7f1288102b60 2024-04-27T04:22:13.068 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:13.067+0000 7f1285d9b640 1 --2- 172.21.15.162:0/3747607846 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1288102b60 0x7f12881131e0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f1278009fa0 tx=0x7f1278009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:22:13.068 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:13.067+0000 7f1276ffd640 1 -- 172.21.15.162:0/3747607846 <== mon.0 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1278030870 con 0x7f1288102b60 2024-04-27T04:22:13.068 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:13.067+0000 7f128ca6e640 1 -- 172.21.15.162:0/3747607846 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f128810d0b0 con 0x7f1288102b60 2024-04-27T04:22:13.068 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:13.067+0000 7f1276ffd640 1 -- 172.21.15.162:0/3747607846 <== mon.0 v2:172.21.15.162:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f12780092e0 con 0x7f1288102b60 2024-04-27T04:22:13.068 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:13.067+0000 7f1276ffd640 1 -- 172.21.15.162:0/3747607846 <== mon.0 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1278043640 con 0x7f1288102b60 2024-04-27T04:22:13.068 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:13.068+0000 7f128ca6e640 1 -- 172.21.15.162:0/3747607846 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f128810d5d0 con 0x7f1288102b60 2024-04-27T04:22:13.069 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:13.068+0000 7f128ca6e640 1 -- 172.21.15.162:0/3747607846 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f12880654f0 con 0x7f1288102b60 2024-04-27T04:22:13.069 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:13.069+0000 7f1276ffd640 1 -- 172.21.15.162:0/3747607846 <== mon.0 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 39) v1 ==== 92747+0+0 (secure 0 0 0) 0x7f12780438b0 con 0x7f1288102b60 2024-04-27T04:22:13.069 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:13.069+0000 7f1276ffd640 1 --2- 172.21.15.162:0/3747607846 >> [v2:172.21.15.162:6800/1396866108,v1:172.21.15.162:6801/1396866108] conn(0x7f127006eb70 0x7f1270071020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:22:13.069 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:13.069+0000 7f1276ffd640 1 -- 172.21.15.162:0/3747607846 <== mon.0 v2:172.21.15.162:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6582+0+0 (secure 0 0 0) 0x7f12780b1730 con 0x7f1288102b60 2024-04-27T04:22:13.069 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:13.069+0000 7f128659c640 1 --2- 172.21.15.162:0/3747607846 >> [v2:172.21.15.162:6800/1396866108,v1:172.21.15.162:6801/1396866108] conn(0x7f127006eb70 0x7f1270071020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:22:13.069 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:13.070+0000 7f128659c640 1 --2- 172.21.15.162:0/3747607846 >> [v2:172.21.15.162:6800/1396866108,v1:172.21.15.162:6801/1396866108] conn(0x7f127006eb70 0x7f1270071020 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f127c004a50 tx=0x7f127c007cb0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:22:13.072 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:13.074+0000 7f1276ffd640 1 -- 172.21.15.162:0/3747607846 <== mon.0 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7f127807a500 con 0x7f1288102b60 2024-04-27T04:22:13.171 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:22:12 smithi181 ceph-mon[81134]: pgmap v275: 65 pgs: 65 active+clean; 208 MiB data, 851 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:22:13.172 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:22:12 smithi181 ceph-mon[81134]: from='client.? 172.21.15.162:0/3176119604' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 28, "format": "json"}]: dispatch 2024-04-27T04:22:13.172 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:22:12 smithi181 ceph-mon[81134]: from='client.? 172.21.15.162:0/3263488992' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 29, "format": "json"}]: dispatch 2024-04-27T04:22:13.172 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:22:12 smithi181 ceph-mon[81134]: from='client.? 172.21.15.162:0/1340126231' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 30, "format": "json"}]: dispatch 2024-04-27T04:22:13.289 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:13.290+0000 7f128ca6e640 1 -- 172.21.15.162:0/3747607846 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 32, "format": "json"} v 0) v1 -- 0x7f1288064350 con 0x7f1288102b60 2024-04-27T04:22:13.289 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:13.291+0000 7f1276ffd640 1 -- 172.21.15.162:0/3747607846 <== mon.0 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 32, "format": "json"}]=0 dumped fsmap epoch 32 v35) v1 ==== 107+0+5218 (secure 0 0 0) 0x7f127807dbc0 con 0x7f1288102b60 2024-04-27T04:22:13.289 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-04-27T04:22:13.289 INFO:teuthology.orchestra.run.smithi162.stdout:{"epoch":32,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":34412,"name":"cephfs.smithi162.xcnapk","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6829/307528848","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6828","nonce":307528848},{"type":"v1","addr":"172.21.15.162:6829","nonce":307528848}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":21},{"gid":44319,"name":"cephfs.smithi181.dzomgh","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6825/1667773112","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":1667773112},{"type":"v1","addr":"172.21.15.181:6825","nonce":1667773112}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":27},{"gid":44323,"name":"cephfs.smithi181.bkkcdm","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6827/3589158285","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6826","nonce":3589158285},{"type":"v1","addr":"172.21.15.181:6827","nonce":3589158285}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":30}],"filesystems":[{"mdsmap":{"epoch":32,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-27T04:06:53.095749+0000","modified":"2024-04-27T04:20:01.750233+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":84,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":34380},"failed":[],"damaged":[],"stopped":[],"info":{"gid_34380":{"gid":34380,"name":"cephfs.smithi162.gzlyjj","rank":0,"incarnation":29,"state":"up:active","state_seq":20,"addr":"172.21.15.162:6827/1198130667","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6826","nonce":1198130667},{"type":"v1","addr":"172.21.15.162:6827","nonce":1198130667}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":34380,"qdb_cluster":[34380]},"id":1}]} 2024-04-27T04:22:13.290 INFO:teuthology.orchestra.run.smithi162.stderr:dumped fsmap epoch 32 2024-04-27T04:22:13.291 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:13.294+0000 7f128ca6e640 1 -- 172.21.15.162:0/3747607846 >> [v2:172.21.15.162:6800/1396866108,v1:172.21.15.162:6801/1396866108] conn(0x7f127006eb70 msgr2=0x7f1270071020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:22:13.291 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:13.294+0000 7f128ca6e640 1 --2- 172.21.15.162:0/3747607846 >> [v2:172.21.15.162:6800/1396866108,v1:172.21.15.162:6801/1396866108] conn(0x7f127006eb70 0x7f1270071020 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f127c004a50 tx=0x7f127c007cb0 comp rx=0 tx=0).stop 2024-04-27T04:22:13.292 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:13.294+0000 7f128ca6e640 1 -- 172.21.15.162:0/3747607846 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1288102b60 msgr2=0x7f12881131e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:22:13.292 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:13.294+0000 7f128ca6e640 1 --2- 172.21.15.162:0/3747607846 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1288102b60 0x7f12881131e0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f1278009fa0 tx=0x7f1278009da0 comp rx=0 tx=0).stop 2024-04-27T04:22:13.292 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:13.294+0000 7f128ca6e640 1 -- 172.21.15.162:0/3747607846 shutdown_connections 2024-04-27T04:22:13.292 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:13.294+0000 7f128ca6e640 1 --2- 172.21.15.162:0/3747607846 >> [v2:172.21.15.162:6800/1396866108,v1:172.21.15.162:6801/1396866108] conn(0x7f127006eb70 0x7f1270071020 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:22:13.292 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:13.294+0000 7f128ca6e640 1 --2- 172.21.15.162:0/3747607846 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1288102b60 0x7f12881131e0 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:22:13.292 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:13.294+0000 7f128ca6e640 1 --2- 172.21.15.162:0/3747607846 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1288102250 0x7f1288112ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:22:13.292 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:13.294+0000 7f128ca6e640 1 -- 172.21.15.162:0/3747607846 >> 172.21.15.162:0/3747607846 conn(0x7f1288070900 msgr2=0x7f12880ffdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:22:13.292 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:13.294+0000 7f128ca6e640 1 -- 172.21.15.162:0/3747607846 shutdown_connections 2024-04-27T04:22:13.292 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:13.294+0000 7f128ca6e640 1 -- 172.21.15.162:0/3747607846 wait complete. 2024-04-27T04:22:13.302 DEBUG:tasks.fs:allow_standby_replay disabled in epoch 32 2024-04-27T04:22:13.303 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 33 2024-04-27T04:22:13.399 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:13.398+0000 7f019a86e640 1 -- 172.21.15.162:0/1844415057 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f01940fd160 msgr2=0x7f01940fd530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:22:13.399 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:13.398+0000 7f019a86e640 1 --2- 172.21.15.162:0/1844415057 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f01940fd160 0x7f01940fd530 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f0188004900 tx=0x7f0188009da0 comp rx=0 tx=0).stop 2024-04-27T04:22:13.399 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:13.398+0000 7f019a86e640 1 -- 172.21.15.162:0/1844415057 shutdown_connections 2024-04-27T04:22:13.399 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:13.398+0000 7f019a86e640 1 --2- 172.21.15.162:0/1844415057 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f01940fdb00 0x7f01941061d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:22:13.399 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:13.398+0000 7f019a86e640 1 --2- 172.21.15.162:0/1844415057 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f01940fd160 0x7f01940fd530 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:22:13.399 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:13.398+0000 7f019a86e640 1 -- 172.21.15.162:0/1844415057 >> 172.21.15.162:0/1844415057 conn(0x7f01940f6be0 msgr2=0x7f01940f8ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:22:13.399 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:13.398+0000 7f019a86e640 1 -- 172.21.15.162:0/1844415057 shutdown_connections 2024-04-27T04:22:13.399 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:13.398+0000 7f019a86e640 1 -- 172.21.15.162:0/1844415057 wait complete. 2024-04-27T04:22:13.399 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:13.399+0000 7f019a86e640 1 Processor -- start 2024-04-27T04:22:13.399 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:13.399+0000 7f019a86e640 1 -- start start 2024-04-27T04:22:13.399 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:13.399+0000 7f019a86e640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f01940fd160 0x7f01940fb020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:22:13.399 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:13.399+0000 7f019a86e640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f01940fdb00 0x7f01940fb560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:22:13.400 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:13.399+0000 7f019a86e640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01940fcfa0 con 0x7f01940fdb00 2024-04-27T04:22:13.400 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:13.399+0000 7f019a86e640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01940fbaa0 con 0x7f01940fd160 2024-04-27T04:22:13.400 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:13.399+0000 7f01937fe640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f01940fdb00 0x7f01940fb560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:22:13.400 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:13.399+0000 7f01937fe640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f01940fdb00 0x7f01940fb560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:57148/0 (socket says 172.21.15.162:57148) 2024-04-27T04:22:13.400 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:13.399+0000 7f01937fe640 1 -- 172.21.15.162:0/1824936670 learned_addr learned my addr 172.21.15.162:0/1824936670 (peer_addr_for_me v2:172.21.15.162:0/0) 2024-04-27T04:22:13.400 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:13.400+0000 7f0193fff640 1 --2- 172.21.15.162:0/1824936670 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f01940fd160 0x7f01940fb020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:22:13.400 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:13.400+0000 7f01937fe640 1 -- 172.21.15.162:0/1824936670 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f01940fd160 msgr2=0x7f01940fb020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:22:13.400 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:13.400+0000 7f01937fe640 1 --2- 172.21.15.162:0/1824936670 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f01940fd160 0x7f01940fb020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:22:13.400 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:13.400+0000 7f01937fe640 1 -- 172.21.15.162:0/1824936670 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0188005040 con 0x7f01940fdb00 2024-04-27T04:22:13.400 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:13.400+0000 7f01937fe640 1 --2- 172.21.15.162:0/1824936670 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f01940fdb00 0x7f01940fb560 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f018400b790 tx=0x7f01840091d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:22:13.400 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:13.400+0000 7f0190ff9640 1 -- 172.21.15.162:0/1824936670 <== mon.0 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f018400c700 con 0x7f01940fdb00 2024-04-27T04:22:13.400 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:13.400+0000 7f0190ff9640 1 -- 172.21.15.162:0/1824936670 <== mon.0 v2:172.21.15.162:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f018400cd30 con 0x7f01940fdb00 2024-04-27T04:22:13.400 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:13.400+0000 7f019a86e640 1 -- 172.21.15.162:0/1824936670 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f01940fbd30 con 0x7f01940fdb00 2024-04-27T04:22:13.400 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:13.400+0000 7f0190ff9640 1 -- 172.21.15.162:0/1824936670 <== mon.0 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0184013370 con 0x7f01940fdb00 2024-04-27T04:22:13.401 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:13.400+0000 7f019a86e640 1 -- 172.21.15.162:0/1824936670 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0194119550 con 0x7f01940fdb00 2024-04-27T04:22:13.401 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:13.401+0000 7f019a86e640 1 -- 172.21.15.162:0/1824936670 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0194066060 con 0x7f01940fdb00 2024-04-27T04:22:13.401 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:13.401+0000 7f0190ff9640 1 -- 172.21.15.162:0/1824936670 <== mon.0 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 39) v1 ==== 92747+0+0 (secure 0 0 0) 0x7f0184012070 con 0x7f01940fdb00 2024-04-27T04:22:13.402 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:13.402+0000 7f0190ff9640 1 --2- 172.21.15.162:0/1824936670 >> [v2:172.21.15.162:6800/1396866108,v1:172.21.15.162:6801/1396866108] conn(0x7f017c06eb70 0x7f017c071020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:22:13.402 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:13.402+0000 7f0190ff9640 1 -- 172.21.15.162:0/1824936670 <== mon.0 v2:172.21.15.162:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6582+0+0 (secure 0 0 0) 0x7f018408e550 con 0x7f01940fdb00 2024-04-27T04:22:13.402 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:13.402+0000 7f0193fff640 1 --2- 172.21.15.162:0/1824936670 >> [v2:172.21.15.162:6800/1396866108,v1:172.21.15.162:6801/1396866108] conn(0x7f017c06eb70 0x7f017c071020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:22:13.402 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:13.402+0000 7f0193fff640 1 --2- 172.21.15.162:0/1824936670 >> [v2:172.21.15.162:6800/1396866108,v1:172.21.15.162:6801/1396866108] conn(0x7f017c06eb70 0x7f017c071020 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f0188009a30 tx=0x7f0188009da0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:22:13.405 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:13.407+0000 7f0190ff9640 1 -- 172.21.15.162:0/1824936670 <== mon.0 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7f01840572a0 con 0x7f01940fdb00 2024-04-27T04:22:13.621 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:13.623+0000 7f019a86e640 1 -- 172.21.15.162:0/1824936670 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 33, "format": "json"} v 0) v1 -- 0x7f0194064350 con 0x7f01940fdb00 2024-04-27T04:22:13.622 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:13.623+0000 7f0190ff9640 1 -- 172.21.15.162:0/1824936670 <== mon.0 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 33, "format": "json"}]=0 dumped fsmap epoch 33 v35) v1 ==== 107+0+5218 (secure 0 0 0) 0x7f018405a960 con 0x7f01940fdb00 2024-04-27T04:22:13.622 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-04-27T04:22:13.622 INFO:teuthology.orchestra.run.smithi162.stdout:{"epoch":33,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":34412,"name":"cephfs.smithi162.xcnapk","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6829/307528848","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6828","nonce":307528848},{"type":"v1","addr":"172.21.15.162:6829","nonce":307528848}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":21},{"gid":44319,"name":"cephfs.smithi181.dzomgh","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6825/1667773112","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":1667773112},{"type":"v1","addr":"172.21.15.181:6825","nonce":1667773112}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":27},{"gid":44323,"name":"cephfs.smithi181.bkkcdm","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6827/3589158285","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6826","nonce":3589158285},{"type":"v1","addr":"172.21.15.181:6827","nonce":3589158285}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":30}],"filesystems":[{"mdsmap":{"epoch":33,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-27T04:06:53.095749+0000","modified":"2024-04-27T04:20:06.651274+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":84,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":34380},"failed":[],"damaged":[],"stopped":[],"info":{"gid_34380":{"gid":34380,"name":"cephfs.smithi162.gzlyjj","rank":0,"incarnation":29,"state":"up:active","state_seq":20,"addr":"172.21.15.162:6827/1198130667","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6826","nonce":1198130667},{"type":"v1","addr":"172.21.15.162:6827","nonce":1198130667}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":34380,"qdb_cluster":[34380]},"id":1}]} 2024-04-27T04:22:13.623 INFO:teuthology.orchestra.run.smithi162.stderr:dumped fsmap epoch 33 2024-04-27T04:22:13.625 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:13.626+0000 7f019a86e640 1 -- 172.21.15.162:0/1824936670 >> [v2:172.21.15.162:6800/1396866108,v1:172.21.15.162:6801/1396866108] conn(0x7f017c06eb70 msgr2=0x7f017c071020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:22:13.625 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:13.626+0000 7f019a86e640 1 --2- 172.21.15.162:0/1824936670 >> [v2:172.21.15.162:6800/1396866108,v1:172.21.15.162:6801/1396866108] conn(0x7f017c06eb70 0x7f017c071020 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f0188009a30 tx=0x7f0188009da0 comp rx=0 tx=0).stop 2024-04-27T04:22:13.625 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:13.626+0000 7f019a86e640 1 -- 172.21.15.162:0/1824936670 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f01940fdb00 msgr2=0x7f01940fb560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:22:13.625 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:13.626+0000 7f019a86e640 1 --2- 172.21.15.162:0/1824936670 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f01940fdb00 0x7f01940fb560 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f018400b790 tx=0x7f01840091d0 comp rx=0 tx=0).stop 2024-04-27T04:22:13.625 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:13.626+0000 7f019a86e640 1 -- 172.21.15.162:0/1824936670 shutdown_connections 2024-04-27T04:22:13.625 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:13.626+0000 7f019a86e640 1 --2- 172.21.15.162:0/1824936670 >> [v2:172.21.15.162:6800/1396866108,v1:172.21.15.162:6801/1396866108] conn(0x7f017c06eb70 0x7f017c071020 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:22:13.625 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:13.626+0000 7f019a86e640 1 --2- 172.21.15.162:0/1824936670 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f01940fdb00 0x7f01940fb560 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:22:13.625 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:13.626+0000 7f019a86e640 1 --2- 172.21.15.162:0/1824936670 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f01940fd160 0x7f01940fb020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:22:13.625 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:13.626+0000 7f019a86e640 1 -- 172.21.15.162:0/1824936670 >> 172.21.15.162:0/1824936670 conn(0x7f01940f6be0 msgr2=0x7f01940f8040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:22:13.625 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:13.626+0000 7f019a86e640 1 -- 172.21.15.162:0/1824936670 shutdown_connections 2024-04-27T04:22:13.625 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:13.626+0000 7f019a86e640 1 -- 172.21.15.162:0/1824936670 wait complete. 2024-04-27T04:22:13.635 DEBUG:tasks.fs:allow_standby_replay disabled in epoch 33 2024-04-27T04:22:13.636 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 34 2024-04-27T04:22:13.733 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:13.732+0000 7f12fc82e640 1 -- 172.21.15.162:0/3576264923 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f12f8102270 msgr2=0x7f12f8102640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:22:13.733 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:13.732+0000 7f12fc82e640 1 --2- 172.21.15.162:0/3576264923 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f12f8102270 0x7f12f8102640 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f12e8009c50 tx=0x7f12e8004d10 comp rx=0 tx=0).stop 2024-04-27T04:22:13.734 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:13.733+0000 7f12fc82e640 1 -- 172.21.15.162:0/3576264923 shutdown_connections 2024-04-27T04:22:13.734 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:13.733+0000 7f12fc82e640 1 --2- 172.21.15.162:0/3576264923 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f12f8064390 0x7f12f8064800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:22:13.734 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:13.733+0000 7f12fc82e640 1 --2- 172.21.15.162:0/3576264923 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f12f8102270 0x7f12f8102640 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:22:13.734 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:13.733+0000 7f12fc82e640 1 -- 172.21.15.162:0/3576264923 >> 172.21.15.162:0/3576264923 conn(0x7f12f8071fd0 msgr2=0x7f12f80723d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:22:13.734 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:13.733+0000 7f12fc82e640 1 -- 172.21.15.162:0/3576264923 shutdown_connections 2024-04-27T04:22:13.734 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:13.733+0000 7f12fc82e640 1 -- 172.21.15.162:0/3576264923 wait complete. 2024-04-27T04:22:13.734 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:13.733+0000 7f12fc82e640 1 Processor -- start 2024-04-27T04:22:13.734 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:13.733+0000 7f12fc82e640 1 -- start start 2024-04-27T04:22:13.734 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:13.733+0000 7f12fc82e640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f12f8064390 0x7f12f8112c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:22:13.734 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:13.733+0000 7f12fc82e640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f12f8102270 0x7f12f8113170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:22:13.734 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:13.733+0000 7f12fc82e640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12f8113770 con 0x7f12f8102270 2024-04-27T04:22:13.734 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:13.733+0000 7f12fc82e640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12f810cdc0 con 0x7f12f8064390 2024-04-27T04:22:13.734 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:13.733+0000 7f12f5d9b640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f12f8102270 0x7f12f8113170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:22:13.735 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:13.733+0000 7f12f5d9b640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f12f8102270 0x7f12f8113170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:40666/0 (socket says 172.21.15.162:40666) 2024-04-27T04:22:13.735 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:13.733+0000 7f12f5d9b640 1 -- 172.21.15.162:0/2768633668 learned_addr learned my addr 172.21.15.162:0/2768633668 (peer_addr_for_me v2:172.21.15.162:0/0) 2024-04-27T04:22:13.735 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:13.734+0000 7f12f659c640 1 --2- 172.21.15.162:0/2768633668 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f12f8064390 0x7f12f8112c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:22:13.735 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:13.734+0000 7f12f5d9b640 1 -- 172.21.15.162:0/2768633668 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f12f8064390 msgr2=0x7f12f8112c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:22:13.735 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:13.734+0000 7f12f5d9b640 1 --2- 172.21.15.162:0/2768633668 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f12f8064390 0x7f12f8112c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:22:13.735 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:13.734+0000 7f12f5d9b640 1 -- 172.21.15.162:0/2768633668 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f12e80098b0 con 0x7f12f8102270 2024-04-27T04:22:13.735 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:13.734+0000 7f12f5d9b640 1 --2- 172.21.15.162:0/2768633668 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f12f8102270 0x7f12f8113170 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f12ec002a50 tx=0x7f12ec004600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:22:13.735 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:13.734+0000 7f12e6ffd640 1 -- 172.21.15.162:0/2768633668 <== mon.0 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f12ec007c30 con 0x7f12f8102270 2024-04-27T04:22:13.735 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:13.734+0000 7f12e6ffd640 1 -- 172.21.15.162:0/2768633668 <== mon.0 v2:172.21.15.162:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f12ec00f040 con 0x7f12f8102270 2024-04-27T04:22:13.735 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:13.734+0000 7f12fc82e640 1 -- 172.21.15.162:0/2768633668 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f12f810d0a0 con 0x7f12f8102270 2024-04-27T04:22:13.735 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:13.734+0000 7f12e6ffd640 1 -- 172.21.15.162:0/2768633668 <== mon.0 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f12ec00db50 con 0x7f12f8102270 2024-04-27T04:22:13.735 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:13.734+0000 7f12fc82e640 1 -- 172.21.15.162:0/2768633668 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f12f810d640 con 0x7f12f8102270 2024-04-27T04:22:13.735 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:13.735+0000 7f12fc82e640 1 -- 172.21.15.162:0/2768633668 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f12f80fd970 con 0x7f12f8102270 2024-04-27T04:22:13.736 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:13.735+0000 7f12e6ffd640 1 -- 172.21.15.162:0/2768633668 <== mon.0 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 39) v1 ==== 92747+0+0 (secure 0 0 0) 0x7f12ec01c400 con 0x7f12f8102270 2024-04-27T04:22:13.736 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:13.736+0000 7f12e6ffd640 1 --2- 172.21.15.162:0/2768633668 >> [v2:172.21.15.162:6800/1396866108,v1:172.21.15.162:6801/1396866108] conn(0x7f12e006eb70 0x7f12e0071020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:22:13.736 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:13.736+0000 7f12e6ffd640 1 -- 172.21.15.162:0/2768633668 <== mon.0 v2:172.21.15.162:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6582+0+0 (secure 0 0 0) 0x7f12ec08e1f0 con 0x7f12f8102270 2024-04-27T04:22:13.736 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:13.736+0000 7f12f659c640 1 --2- 172.21.15.162:0/2768633668 >> [v2:172.21.15.162:6800/1396866108,v1:172.21.15.162:6801/1396866108] conn(0x7f12e006eb70 0x7f12e0071020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:22:13.736 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:13.736+0000 7f12f659c640 1 --2- 172.21.15.162:0/2768633668 >> [v2:172.21.15.162:6800/1396866108,v1:172.21.15.162:6801/1396866108] conn(0x7f12e006eb70 0x7f12e0071020 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f12e8004a30 tx=0x7f12e802e040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:22:13.739 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:13.741+0000 7f12e6ffd640 1 -- 172.21.15.162:0/2768633668 <== mon.0 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7f12ec056f40 con 0x7f12f8102270 2024-04-27T04:22:13.886 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:22:13 smithi162 ceph-mon[110641]: from='client.? 172.21.15.162:0/244734536' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 31, "format": "json"}]: dispatch 2024-04-27T04:22:13.886 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:22:13 smithi162 ceph-mon[110641]: from='client.? 172.21.15.162:0/3747607846' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 32, "format": "json"}]: dispatch 2024-04-27T04:22:13.886 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:22:13 smithi162 ceph-mon[110641]: from='client.? 172.21.15.162:0/1824936670' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 33, "format": "json"}]: dispatch 2024-04-27T04:22:13.957 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:13.959+0000 7f12fc82e640 1 -- 172.21.15.162:0/2768633668 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 34, "format": "json"} v 0) v1 -- 0x7f12f8109370 con 0x7f12f8102270 2024-04-27T04:22:13.958 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:13.960+0000 7f12e6ffd640 1 -- 172.21.15.162:0/2768633668 <== mon.0 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 34, "format": "json"}]=0 dumped fsmap epoch 34 v35) v1 ==== 107+0+5217 (secure 0 0 0) 0x7f12ec05a600 con 0x7f12f8102270 2024-04-27T04:22:13.958 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-04-27T04:22:13.958 INFO:teuthology.orchestra.run.smithi162.stdout:{"epoch":34,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":34412,"name":"cephfs.smithi162.xcnapk","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6829/307528848","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6828","nonce":307528848},{"type":"v1","addr":"172.21.15.162:6829","nonce":307528848}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":21},{"gid":44319,"name":"cephfs.smithi181.dzomgh","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6825/1667773112","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":1667773112},{"type":"v1","addr":"172.21.15.181:6825","nonce":1667773112}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":27},{"gid":44323,"name":"cephfs.smithi181.bkkcdm","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6827/3589158285","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6826","nonce":3589158285},{"type":"v1","addr":"172.21.15.181:6827","nonce":3589158285}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":30}],"filesystems":[{"mdsmap":{"epoch":34,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-27T04:06:53.095749+0000","modified":"2024-04-27T04:20:07.654114+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":84,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":34380},"failed":[],"damaged":[],"stopped":[],"info":{"gid_34380":{"gid":34380,"name":"cephfs.smithi162.gzlyjj","rank":0,"incarnation":29,"state":"up:active","state_seq":20,"addr":"172.21.15.162:6827/1198130667","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6826","nonce":1198130667},{"type":"v1","addr":"172.21.15.162:6827","nonce":1198130667}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":34380,"qdb_cluster":[34380]},"id":1}]} 2024-04-27T04:22:13.958 INFO:teuthology.orchestra.run.smithi162.stderr:dumped fsmap epoch 34 2024-04-27T04:22:13.960 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:13.962+0000 7f12fc82e640 1 -- 172.21.15.162:0/2768633668 >> [v2:172.21.15.162:6800/1396866108,v1:172.21.15.162:6801/1396866108] conn(0x7f12e006eb70 msgr2=0x7f12e0071020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:22:13.960 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:13.962+0000 7f12fc82e640 1 --2- 172.21.15.162:0/2768633668 >> [v2:172.21.15.162:6800/1396866108,v1:172.21.15.162:6801/1396866108] conn(0x7f12e006eb70 0x7f12e0071020 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f12e8004a30 tx=0x7f12e802e040 comp rx=0 tx=0).stop 2024-04-27T04:22:13.960 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:13.962+0000 7f12fc82e640 1 -- 172.21.15.162:0/2768633668 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f12f8102270 msgr2=0x7f12f8113170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:22:13.960 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:13.962+0000 7f12fc82e640 1 --2- 172.21.15.162:0/2768633668 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f12f8102270 0x7f12f8113170 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f12ec002a50 tx=0x7f12ec004600 comp rx=0 tx=0).stop 2024-04-27T04:22:13.960 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:13.962+0000 7f12fc82e640 1 -- 172.21.15.162:0/2768633668 shutdown_connections 2024-04-27T04:22:13.960 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:13.962+0000 7f12fc82e640 1 --2- 172.21.15.162:0/2768633668 >> [v2:172.21.15.162:6800/1396866108,v1:172.21.15.162:6801/1396866108] conn(0x7f12e006eb70 0x7f12e0071020 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:22:13.960 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:13.962+0000 7f12fc82e640 1 --2- 172.21.15.162:0/2768633668 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f12f8102270 0x7f12f8113170 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:22:13.960 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:13.962+0000 7f12fc82e640 1 --2- 172.21.15.162:0/2768633668 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f12f8064390 0x7f12f8112c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:22:13.961 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:13.962+0000 7f12fc82e640 1 -- 172.21.15.162:0/2768633668 >> 172.21.15.162:0/2768633668 conn(0x7f12f8071fd0 msgr2=0x7f12f80fa260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:22:13.961 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:13.962+0000 7f12fc82e640 1 -- 172.21.15.162:0/2768633668 shutdown_connections 2024-04-27T04:22:13.961 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:13.962+0000 7f12fc82e640 1 -- 172.21.15.162:0/2768633668 wait complete. 2024-04-27T04:22:13.971 DEBUG:teuthology.run_tasks:Unwinding manager ceph-fuse 2024-04-27T04:22:13.983 INFO:tasks.ceph_fuse:Unmounting ceph-fuse clients... 2024-04-27T04:22:13.983 DEBUG:teuthology.orchestra.run.smithi162:> set -ex 2024-04-27T04:22:13.983 DEBUG:teuthology.orchestra.run.smithi162:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-27T04:22:13.999 DEBUG:teuthology.orchestra.run.smithi162:> set -ex 2024-04-27T04:22:14.000 DEBUG:teuthology.orchestra.run.smithi162:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-27T04:22:14.056 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-27T04:22:14.171 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:22:13 smithi181 ceph-mon[81134]: from='client.? 172.21.15.162:0/244734536' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 31, "format": "json"}]: dispatch 2024-04-27T04:22:14.171 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:22:13 smithi181 ceph-mon[81134]: from='client.? 172.21.15.162:0/3747607846' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 32, "format": "json"}]: dispatch 2024-04-27T04:22:14.171 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:22:13 smithi181 ceph-mon[81134]: from='client.? 172.21.15.162:0/1824936670' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 33, "format": "json"}]: dispatch 2024-04-27T04:22:14.192 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:14.192+0000 7fd4872ae640 1 -- 172.21.15.162:0/226941591 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd480102170 msgr2=0x7fd480102540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:22:14.192 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:14.192+0000 7fd4872ae640 1 --2- 172.21.15.162:0/226941591 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd480102170 0x7fd480102540 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fd47c0048d0 tx=0x7fd47c009da0 comp rx=0 tx=0).stop 2024-04-27T04:22:14.192 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:14.192+0000 7fd4872ae640 1 -- 172.21.15.162:0/226941591 shutdown_connections 2024-04-27T04:22:14.192 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:14.192+0000 7fd4872ae640 1 --2- 172.21.15.162:0/226941591 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd480102b10 0x7fd4800fc1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:22:14.192 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:14.192+0000 7fd4872ae640 1 --2- 172.21.15.162:0/226941591 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd480102170 0x7fd480102540 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:22:14.192 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:14.192+0000 7fd4872ae640 1 -- 172.21.15.162:0/226941591 >> 172.21.15.162:0/226941591 conn(0x7fd4800f7d40 msgr2=0x7fd4800fa150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:22:14.192 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:14.192+0000 7fd4872ae640 1 -- 172.21.15.162:0/226941591 shutdown_connections 2024-04-27T04:22:14.192 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:14.192+0000 7fd4872ae640 1 -- 172.21.15.162:0/226941591 wait complete. 2024-04-27T04:22:14.192 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:14.193+0000 7fd4872ae640 1 Processor -- start 2024-04-27T04:22:14.192 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:14.193+0000 7fd4872ae640 1 -- start start 2024-04-27T04:22:14.192 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:14.193+0000 7fd4872ae640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd480102170 0x7fd480112c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:22:14.192 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:14.193+0000 7fd4872ae640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd480102b10 0x7fd4801131c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:22:14.193 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:14.193+0000 7fd4872ae640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4801137c0 con 0x7fd480102170 2024-04-27T04:22:14.193 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:14.193+0000 7fd4872ae640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd48010ce10 con 0x7fd480102b10 2024-04-27T04:22:14.193 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:14.193+0000 7fd48504a640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd480102170 0x7fd480112c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:22:14.193 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:14.193+0000 7fd48504a640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd480102170 0x7fd480112c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:40686/0 (socket says 172.21.15.162:40686) 2024-04-27T04:22:14.193 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:14.193+0000 7fd48504a640 1 -- 172.21.15.162:0/352486400 learned_addr learned my addr 172.21.15.162:0/352486400 (peer_addr_for_me v2:172.21.15.162:0/0) 2024-04-27T04:22:14.193 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:14.193+0000 7fd484849640 1 --2- 172.21.15.162:0/352486400 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd480102b10 0x7fd4801131c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:22:14.193 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:14.193+0000 7fd48504a640 1 -- 172.21.15.162:0/352486400 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd480102b10 msgr2=0x7fd4801131c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:22:14.193 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:14.193+0000 7fd48504a640 1 --2- 172.21.15.162:0/352486400 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd480102b10 0x7fd4801131c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:22:14.193 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:14.193+0000 7fd48504a640 1 -- 172.21.15.162:0/352486400 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd47c005040 con 0x7fd480102170 2024-04-27T04:22:14.193 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:14.193+0000 7fd484849640 1 --2- 172.21.15.162:0/352486400 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd480102b10 0x7fd4801131c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-27T04:22:14.193 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:14.193+0000 7fd48504a640 1 --2- 172.21.15.162:0/352486400 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd480102170 0x7fd480112c80 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fd47c004930 tx=0x7fd47c009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:22:14.193 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:14.194+0000 7fd475ffb640 1 -- 172.21.15.162:0/352486400 <== mon.0 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd47c0094c0 con 0x7fd480102170 2024-04-27T04:22:14.193 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:14.194+0000 7fd4872ae640 1 -- 172.21.15.162:0/352486400 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd48010d090 con 0x7fd480102170 2024-04-27T04:22:14.194 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:14.194+0000 7fd475ffb640 1 -- 172.21.15.162:0/352486400 <== mon.0 v2:172.21.15.162:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fd47c032e50 con 0x7fd480102170 2024-04-27T04:22:14.194 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:14.194+0000 7fd475ffb640 1 -- 172.21.15.162:0/352486400 <== mon.0 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd47c0433b0 con 0x7fd480102170 2024-04-27T04:22:14.194 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:14.194+0000 7fd4872ae640 1 -- 172.21.15.162:0/352486400 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd48010d550 con 0x7fd480102170 2024-04-27T04:22:14.195 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:14.195+0000 7fd4872ae640 1 -- 172.21.15.162:0/352486400 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd4800fd8e0 con 0x7fd480102170 2024-04-27T04:22:14.195 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:14.195+0000 7fd475ffb640 1 -- 172.21.15.162:0/352486400 <== mon.0 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 39) v1 ==== 92747+0+0 (secure 0 0 0) 0x7fd47c02f070 con 0x7fd480102170 2024-04-27T04:22:14.195 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:14.196+0000 7fd475ffb640 1 --2- 172.21.15.162:0/352486400 >> [v2:172.21.15.162:6800/1396866108,v1:172.21.15.162:6801/1396866108] conn(0x7fd46c06eb70 0x7fd46c071020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:22:14.195 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:14.196+0000 7fd484849640 1 --2- 172.21.15.162:0/352486400 >> [v2:172.21.15.162:6800/1396866108,v1:172.21.15.162:6801/1396866108] conn(0x7fd46c06eb70 0x7fd46c071020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:22:14.195 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:14.196+0000 7fd475ffb640 1 -- 172.21.15.162:0/352486400 <== mon.0 v2:172.21.15.162:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6582+0+0 (secure 0 0 0) 0x7fd47c0b07c0 con 0x7fd480102170 2024-04-27T04:22:14.195 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:14.196+0000 7fd484849640 1 --2- 172.21.15.162:0/352486400 >> [v2:172.21.15.162:6800/1396866108,v1:172.21.15.162:6801/1396866108] conn(0x7fd46c06eb70 0x7fd46c071020 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fd470007fd0 tx=0x7fd470009e90 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:22:14.198 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:14.201+0000 7fd475ffb640 1 -- 172.21.15.162:0/352486400 <== mon.0 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7fd47c079510 con 0x7fd480102170 2024-04-27T04:22:14.376 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:14.378+0000 7fd4872ae640 1 -- 172.21.15.162:0/352486400 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd48004bcc0 con 0x7fd480102170 2024-04-27T04:22:14.377 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:14.380+0000 7fd475ffb640 1 -- 172.21.15.162:0/352486400 <== mon.0 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v84) v1 ==== 74+0+16846 (secure 0 0 0) 0x7fd47c07cbd0 con 0x7fd480102170 2024-04-27T04:22:14.377 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-04-27T04:22:14.378 INFO:teuthology.orchestra.run.smithi162.stdout:{"epoch":84,"fsid":"7f9750d6-044a-11ef-bc93-c7b262605968","created":"2024-04-27T04:00:06.343616+0000","modified":"2024-04-27T04:19:50.546745+0000","last_up_change":"2024-04-27T04:18:43.492214+0000","last_in_change":"2024-04-27T04:05:18.597695+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":20,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":6,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-27T04:04:24.971779+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":6,"score_stable":6,"optimal_score":0.5,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs.cephfs.meta","create_time":"2024-04-27T04:06:52.008169+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"44","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.690000057220459,"score_stable":1.690000057220459,"optimal_score":1,"raw_score_acting":1.690000057220459,"raw_score_stable":1.690000057220459,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs.cephfs.data","create_time":"2024-04-27T04:06:52.322198+0000","flags":131073,"flags_names":"hashpspool,bulk","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"45","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0ae69d7f-0ab4-4672-89b9-3563a10a1bf2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":9,"last_clean_end":49,"up_from":53,"up_thru":78,"down_at":50,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6802","nonce":1840117633},{"type":"v1","addr":"172.21.15.162:6803","nonce":1840117633}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6804","nonce":1840117633},{"type":"v1","addr":"172.21.15.162:6805","nonce":1840117633}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6808","nonce":1840117633},{"type":"v1","addr":"172.21.15.162:6809","nonce":1840117633}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6806","nonce":1840117633},{"type":"v1","addr":"172.21.15.162:6807","nonce":1840117633}]},"public_addr":"172.21.15.162:6803/1840117633","cluster_addr":"172.21.15.162:6805/1840117633","heartbeat_back_addr":"172.21.15.162:6809/1840117633","heartbeat_front_addr":"172.21.15.162:6807/1840117633","state":["exists","up"]},{"osd":1,"uuid":"95bdc9f0-c438-4a88-942d-dffb0c61c308","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":14,"last_clean_end":54,"up_from":58,"up_thru":78,"down_at":55,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6810","nonce":2144127279},{"type":"v1","addr":"172.21.15.162:6811","nonce":2144127279}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6812","nonce":2144127279},{"type":"v1","addr":"172.21.15.162:6813","nonce":2144127279}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6816","nonce":2144127279},{"type":"v1","addr":"172.21.15.162:6817","nonce":2144127279}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6814","nonce":2144127279},{"type":"v1","addr":"172.21.15.162:6815","nonce":2144127279}]},"public_addr":"172.21.15.162:6811/2144127279","cluster_addr":"172.21.15.162:6813/2144127279","heartbeat_back_addr":"172.21.15.162:6817/2144127279","heartbeat_front_addr":"172.21.15.162:6815/2144127279","state":["exists","up"]},{"osd":2,"uuid":"a773b386-e75b-432a-9d1b-2206301b96c3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":19,"last_clean_end":59,"up_from":63,"up_thru":78,"down_at":60,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6818","nonce":3115093010},{"type":"v1","addr":"172.21.15.162:6819","nonce":3115093010}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6820","nonce":3115093010},{"type":"v1","addr":"172.21.15.162:6821","nonce":3115093010}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6824","nonce":3115093010},{"type":"v1","addr":"172.21.15.162:6825","nonce":3115093010}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6822","nonce":3115093010},{"type":"v1","addr":"172.21.15.162:6823","nonce":3115093010}]},"public_addr":"172.21.15.162:6819/3115093010","cluster_addr":"172.21.15.162:6821/3115093010","heartbeat_back_addr":"172.21.15.162:6825/3115093010","heartbeat_front_addr":"172.21.15.162:6823/3115093010","state":["exists","up"]},{"osd":3,"uuid":"756091bc-1594-479d-9390-cb60e6b65ebe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":26,"last_clean_end":64,"up_from":68,"up_thru":78,"down_at":65,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":114683138},{"type":"v1","addr":"172.21.15.181:6801","nonce":114683138}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":114683138},{"type":"v1","addr":"172.21.15.181:6803","nonce":114683138}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6806","nonce":114683138},{"type":"v1","addr":"172.21.15.181:6807","nonce":114683138}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":114683138},{"type":"v1","addr":"172.21.15.181:6805","nonce":114683138}]},"public_addr":"172.21.15.181:6801/114683138","cluster_addr":"172.21.15.181:6803/114683138","heartbeat_back_addr":"172.21.15.181:6807/114683138","heartbeat_front_addr":"172.21.15.181:6805/114683138","state":["exists","up"]},{"osd":4,"uuid":"bd70494d-061c-4468-b837-26150c2d4208","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":34,"last_clean_end":69,"up_from":73,"up_thru":78,"down_at":70,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6808","nonce":1781129695},{"type":"v1","addr":"172.21.15.181:6809","nonce":1781129695}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":1781129695},{"type":"v1","addr":"172.21.15.181:6811","nonce":1781129695}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":1781129695},{"type":"v1","addr":"172.21.15.181:6815","nonce":1781129695}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6812","nonce":1781129695},{"type":"v1","addr":"172.21.15.181:6813","nonce":1781129695}]},"public_addr":"172.21.15.181:6809/1781129695","cluster_addr":"172.21.15.181:6811/1781129695","heartbeat_back_addr":"172.21.15.181:6815/1781129695","heartbeat_front_addr":"172.21.15.181:6813/1781129695","state":["exists","up"]},{"osd":5,"uuid":"15455ef6-2853-4a84-aaa9-1797e78302a9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":39,"last_clean_end":74,"up_from":78,"up_thru":78,"down_at":75,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":2414821263},{"type":"v1","addr":"172.21.15.181:6817","nonce":2414821263}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6818","nonce":2414821263},{"type":"v1","addr":"172.21.15.181:6819","nonce":2414821263}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6822","nonce":2414821263},{"type":"v1","addr":"172.21.15.181:6823","nonce":2414821263}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":2414821263},{"type":"v1","addr":"172.21.15.181:6821","nonce":2414821263}]},"public_addr":"172.21.15.181:6817/2414821263","cluster_addr":"172.21.15.181:6819/2414821263","heartbeat_back_addr":"172.21.15.181:6823/2414821263","heartbeat_front_addr":"172.21.15.181:6821/2414821263","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-27T04:03:28.522704+0000","dead_epoch":49},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-27T04:03:54.648083+0000","dead_epoch":54},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-27T04:04:22.107316+0000","dead_epoch":59},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-27T04:04:46.020733+0000","dead_epoch":64},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-27T04:05:10.634082+0000","dead_epoch":69},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-27T04:05:35.756217+0000","dead_epoch":74}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.181:6827/3756475453":"2024-04-28T04:19:50.546716+0000","172.21.15.181:6826/3756475453":"2024-04-28T04:19:50.546716+0000","172.21.15.162:0/1045614319":"2024-04-28T04:14:17.593083+0000","172.21.15.181:6828/4165336392":"2024-04-28T04:11:32.210928+0000","172.21.15.162:0/2336042321":"2024-04-28T04:02:16.661212+0000","172.21.15.162:6800/4079782599":"2024-04-28T04:10:05.654758+0000","172.21.15.181:6824/658214993":"2024-04-28T04:19:26.414790+0000","172.21.15.181:6829/4165336392":"2024-04-28T04:11:32.210928+0000","172.21.15.181:0/642670640":"2024-04-28T04:11:32.210928+0000","172.21.15.162:0/3828848945":"2024-04-28T04:14:17.593083+0000","172.21.15.162:0/1971036271":"2024-04-28T04:10:05.654758+0000","172.21.15.162:6800/8067198":"2024-04-28T04:01:10.996874+0000","172.21.15.162:0/2623011446":"2024-04-28T04:00:40.642043+0000","172.21.15.162:0/1703886136":"2024-04-28T04:02:16.661212+0000","172.21.15.162:0/2129785584":"2024-04-28T04:01:10.996874+0000","172.21.15.162:6800/1482851081":"2024-04-28T04:02:16.661212+0000","172.21.15.162:6801/1480340393":"2024-04-28T04:00:40.642043+0000","172.21.15.181:0/209590021":"2024-04-28T04:11:32.210928+0000","172.21.15.162:0/4025365522":"2024-04-28T04:00:40.642043+0000","172.21.15.181:0/3276305180":"2024-04-28T04:11:32.210928+0000","172.21.15.162:6801/1482851081":"2024-04-28T04:02:16.661212+0000","172.21.15.162:0/3073351439":"2024-04-28T04:01:10.996874+0000","172.21.15.181:6825/658214993":"2024-04-28T04:19:26.414790+0000","172.21.15.162:0/795779642":"2024-04-28T04:02:16.661212+0000","172.21.15.162:0/3590827229":"2024-04-28T04:01:10.996874+0000","172.21.15.181:0/2462932364":"2024-04-28T04:11:32.210928+0000","172.21.15.162:6801/8067198":"2024-04-28T04:01:10.996874+0000","172.21.15.162:6826/998468496":"2024-04-28T04:07:08.543563+0000","172.21.15.162:0/1349712416":"2024-04-28T04:10:05.654758+0000","172.21.15.162:6800/1480340393":"2024-04-28T04:00:40.642043+0000","172.21.15.181:0/1936190707":"2024-04-28T04:11:32.210928+0000","172.21.15.181:0/2761247390":"2024-04-28T04:11:32.210928+0000","172.21.15.162:6801/4079782599":"2024-04-28T04:10:05.654758+0000","172.21.15.162:0/3944767668":"2024-04-28T04:10:05.654758+0000","172.21.15.162:0/2579239891":"2024-04-28T04:00:40.642043+0000","172.21.15.162:6827/998468496":"2024-04-28T04:07:08.543563+0000","172.21.15.162:6800/901992993":"2024-04-28T04:14:17.593083+0000","172.21.15.162:0/2783280109":"2024-04-28T04:10:05.654758+0000","172.21.15.162:6801/901992993":"2024-04-28T04:14:17.593083+0000","172.21.15.162:0/1622403042":"2024-04-28T04:14:17.593083+0000","172.21.15.162:0/2023772646":"2024-04-28T04:14:17.593083+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-27T04:22:14.380 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:14.382+0000 7fd4872ae640 1 -- 172.21.15.162:0/352486400 >> [v2:172.21.15.162:6800/1396866108,v1:172.21.15.162:6801/1396866108] conn(0x7fd46c06eb70 msgr2=0x7fd46c071020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:22:14.380 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:14.382+0000 7fd4872ae640 1 --2- 172.21.15.162:0/352486400 >> [v2:172.21.15.162:6800/1396866108,v1:172.21.15.162:6801/1396866108] conn(0x7fd46c06eb70 0x7fd46c071020 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fd470007fd0 tx=0x7fd470009e90 comp rx=0 tx=0).stop 2024-04-27T04:22:14.380 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:14.382+0000 7fd4872ae640 1 -- 172.21.15.162:0/352486400 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd480102170 msgr2=0x7fd480112c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:22:14.380 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:14.382+0000 7fd4872ae640 1 --2- 172.21.15.162:0/352486400 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd480102170 0x7fd480112c80 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fd47c004930 tx=0x7fd47c009da0 comp rx=0 tx=0).stop 2024-04-27T04:22:14.380 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:14.382+0000 7fd4872ae640 1 -- 172.21.15.162:0/352486400 shutdown_connections 2024-04-27T04:22:14.380 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:14.382+0000 7fd4872ae640 1 --2- 172.21.15.162:0/352486400 >> [v2:172.21.15.162:6800/1396866108,v1:172.21.15.162:6801/1396866108] conn(0x7fd46c06eb70 0x7fd46c071020 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:22:14.380 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:14.382+0000 7fd4872ae640 1 --2- 172.21.15.162:0/352486400 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd480102b10 0x7fd4801131c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:22:14.380 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:14.382+0000 7fd4872ae640 1 --2- 172.21.15.162:0/352486400 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd480102170 0x7fd480112c80 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:22:14.380 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:14.382+0000 7fd4872ae640 1 -- 172.21.15.162:0/352486400 >> 172.21.15.162:0/352486400 conn(0x7fd4800f7d40 msgr2=0x7fd4800f8cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:22:14.380 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:14.382+0000 7fd4872ae640 1 -- 172.21.15.162:0/352486400 shutdown_connections 2024-04-27T04:22:14.380 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:14.382+0000 7fd4872ae640 1 -- 172.21.15.162:0/352486400 wait complete. 2024-04-27T04:22:14.391 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-27T04:04:24.971779+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 6, 'score_stable': 6, 'optimal_score': 0.5, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs.cephfs.meta', 'create_time': '2024-04-27T04:06:52.008169+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '44', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.690000057220459, 'score_stable': 1.690000057220459, 'optimal_score': 1, 'raw_score_acting': 1.690000057220459, 'raw_score_stable': 1.690000057220459, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs.cephfs.data', 'create_time': '2024-04-27T04:06:52.322198+0000', 'flags': 131073, 'flags_names': 'hashpspool,bulk', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '45', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-27T04:22:14.391 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-27T04:22:14.491 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:14.490+0000 7fc004b9d640 1 -- 172.21.15.162:0/3046780969 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc000101a10 msgr2=0x7fc0000fb0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:22:14.491 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:14.490+0000 7fc004b9d640 1 --2- 172.21.15.162:0/3046780969 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc000101a10 0x7fc0000fb0a0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fbfe80048d0 tx=0x7fbfe8009da0 comp rx=0 tx=0).stop 2024-04-27T04:22:14.492 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:14.490+0000 7fc004b9d640 1 -- 172.21.15.162:0/3046780969 shutdown_connections 2024-04-27T04:22:14.492 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:14.490+0000 7fc004b9d640 1 --2- 172.21.15.162:0/3046780969 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc000101a10 0x7fc0000fb0a0 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:22:14.492 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:14.490+0000 7fc004b9d640 1 --2- 172.21.15.162:0/3046780969 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc000101070 0x7fc000101440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:22:14.492 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:14.490+0000 7fc004b9d640 1 -- 172.21.15.162:0/3046780969 >> 172.21.15.162:0/3046780969 conn(0x7fc0000f6c60 msgr2=0x7fc0000f9070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:22:14.492 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:14.490+0000 7fc004b9d640 1 -- 172.21.15.162:0/3046780969 shutdown_connections 2024-04-27T04:22:14.492 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:14.490+0000 7fc004b9d640 1 -- 172.21.15.162:0/3046780969 wait complete. 2024-04-27T04:22:14.492 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:14.490+0000 7fc004b9d640 1 Processor -- start 2024-04-27T04:22:14.492 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:14.491+0000 7fc004b9d640 1 -- start start 2024-04-27T04:22:14.493 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:14.491+0000 7fc004b9d640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc000101070 0x7fc000111bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:22:14.493 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:14.491+0000 7fc004b9d640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc000101a10 0x7fc0001120f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:22:14.493 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:14.491+0000 7fc004b9d640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0001126f0 con 0x7fc000101a10 2024-04-27T04:22:14.493 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:14.491+0000 7fc004b9d640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc00010bd40 con 0x7fc000101070 2024-04-27T04:22:14.493 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:14.491+0000 7fbff7fff640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc000101a10 0x7fc0001120f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:22:14.493 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:14.491+0000 7fbff7fff640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc000101a10 0x7fc0001120f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:40692/0 (socket says 172.21.15.162:40692) 2024-04-27T04:22:14.494 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:14.491+0000 7fbff7fff640 1 -- 172.21.15.162:0/3999888938 learned_addr learned my addr 172.21.15.162:0/3999888938 (peer_addr_for_me v2:172.21.15.162:0/0) 2024-04-27T04:22:14.494 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:14.491+0000 7fbffe59c640 1 --2- 172.21.15.162:0/3999888938 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc000101070 0x7fc000111bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:22:14.494 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:14.491+0000 7fbff7fff640 1 -- 172.21.15.162:0/3999888938 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc000101070 msgr2=0x7fc000111bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:22:14.494 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:14.491+0000 7fbff7fff640 1 --2- 172.21.15.162:0/3999888938 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc000101070 0x7fc000111bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:22:14.494 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:14.491+0000 7fbff7fff640 1 -- 172.21.15.162:0/3999888938 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbfe8005040 con 0x7fc000101a10 2024-04-27T04:22:14.494 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:14.491+0000 7fbff7fff640 1 --2- 172.21.15.162:0/3999888938 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc000101a10 0x7fc0001120f0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fbfe8004930 tx=0x7fbfe8009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:22:14.495 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:14.491+0000 7fbff6ffd640 1 -- 172.21.15.162:0/3999888938 <== mon.0 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbfe80094c0 con 0x7fc000101a10 2024-04-27T04:22:14.495 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:14.491+0000 7fbff6ffd640 1 -- 172.21.15.162:0/3999888938 <== mon.0 v2:172.21.15.162:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fbfe8032e50 con 0x7fc000101a10 2024-04-27T04:22:14.495 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:14.491+0000 7fc004b9d640 1 -- 172.21.15.162:0/3999888938 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc00010bfc0 con 0x7fc000101a10 2024-04-27T04:22:14.495 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:14.491+0000 7fbff6ffd640 1 -- 172.21.15.162:0/3999888938 <== mon.0 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbfe80433b0 con 0x7fc000101a10 2024-04-27T04:22:14.495 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:14.491+0000 7fc004b9d640 1 -- 172.21.15.162:0/3999888938 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc00010c480 con 0x7fc000101a10 2024-04-27T04:22:14.495 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:14.492+0000 7fc004b9d640 1 -- 172.21.15.162:0/3999888938 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc0000fc7e0 con 0x7fc000101a10 2024-04-27T04:22:14.496 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:14.493+0000 7fbff6ffd640 1 -- 172.21.15.162:0/3999888938 <== mon.0 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 39) v1 ==== 92747+0+0 (secure 0 0 0) 0x7fbfe802f070 con 0x7fc000101a10 2024-04-27T04:22:14.496 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:14.493+0000 7fbff6ffd640 1 --2- 172.21.15.162:0/3999888938 >> [v2:172.21.15.162:6800/1396866108,v1:172.21.15.162:6801/1396866108] conn(0x7fbfec06eb70 0x7fbfec071020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:22:14.496 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:14.493+0000 7fbffe59c640 1 --2- 172.21.15.162:0/3999888938 >> [v2:172.21.15.162:6800/1396866108,v1:172.21.15.162:6801/1396866108] conn(0x7fbfec06eb70 0x7fbfec071020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:22:14.496 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:14.493+0000 7fbff6ffd640 1 -- 172.21.15.162:0/3999888938 <== mon.0 v2:172.21.15.162:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6582+0+0 (secure 0 0 0) 0x7fbfe80b0880 con 0x7fc000101a10 2024-04-27T04:22:14.496 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:14.494+0000 7fbffe59c640 1 --2- 172.21.15.162:0/3999888938 >> [v2:172.21.15.162:6800/1396866108,v1:172.21.15.162:6801/1396866108] conn(0x7fbfec06eb70 0x7fbfec071020 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fbff0007fd0 tx=0x7fbff0009040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:22:14.497 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:14.498+0000 7fbff6ffd640 1 -- 172.21.15.162:0/3999888938 <== mon.0 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7fbfe80795d0 con 0x7fc000101a10 2024-04-27T04:22:14.670 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:14.672+0000 7fc004b9d640 1 -- 172.21.15.162:0/3999888938 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fc0001081d0 con 0x7fc000101a10 2024-04-27T04:22:14.670 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:14.673+0000 7fbff6ffd640 1 -- 172.21.15.162:0/3999888938 <== mon.0 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v84) v1 ==== 93+0+10 (secure 0 0 0) 0x7fbfe807cc90 con 0x7fc000101a10 2024-04-27T04:22:14.670 INFO:teuthology.orchestra.run.smithi162.stdout:pg_num: 1 2024-04-27T04:22:14.673 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:14.675+0000 7fc004b9d640 1 -- 172.21.15.162:0/3999888938 >> [v2:172.21.15.162:6800/1396866108,v1:172.21.15.162:6801/1396866108] conn(0x7fbfec06eb70 msgr2=0x7fbfec071020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:22:14.673 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:14.675+0000 7fc004b9d640 1 --2- 172.21.15.162:0/3999888938 >> [v2:172.21.15.162:6800/1396866108,v1:172.21.15.162:6801/1396866108] conn(0x7fbfec06eb70 0x7fbfec071020 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fbff0007fd0 tx=0x7fbff0009040 comp rx=0 tx=0).stop 2024-04-27T04:22:14.673 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:14.675+0000 7fc004b9d640 1 -- 172.21.15.162:0/3999888938 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc000101a10 msgr2=0x7fc0001120f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:22:14.673 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:14.675+0000 7fc004b9d640 1 --2- 172.21.15.162:0/3999888938 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc000101a10 0x7fc0001120f0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fbfe8004930 tx=0x7fbfe8009da0 comp rx=0 tx=0).stop 2024-04-27T04:22:14.673 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:14.675+0000 7fc004b9d640 1 -- 172.21.15.162:0/3999888938 shutdown_connections 2024-04-27T04:22:14.673 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:14.675+0000 7fc004b9d640 1 --2- 172.21.15.162:0/3999888938 >> [v2:172.21.15.162:6800/1396866108,v1:172.21.15.162:6801/1396866108] conn(0x7fbfec06eb70 0x7fbfec071020 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:22:14.673 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:14.675+0000 7fc004b9d640 1 --2- 172.21.15.162:0/3999888938 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc000101a10 0x7fc0001120f0 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:22:14.673 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:14.675+0000 7fc004b9d640 1 --2- 172.21.15.162:0/3999888938 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc000101070 0x7fc000111bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:22:14.673 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:14.675+0000 7fc004b9d640 1 -- 172.21.15.162:0/3999888938 >> 172.21.15.162:0/3999888938 conn(0x7fc0000f6c60 msgr2=0x7fc0000f7bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:22:14.673 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:14.675+0000 7fc004b9d640 1 -- 172.21.15.162:0/3999888938 shutdown_connections 2024-04-27T04:22:14.673 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:14.675+0000 7fc004b9d640 1 -- 172.21.15.162:0/3999888938 wait complete. 2024-04-27T04:22:14.683 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs.cephfs.meta pg_num 2024-04-27T04:22:14.781 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:14.783+0000 7fe5e162e640 1 -- 172.21.15.162:0/2743556354 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe5dc064380 msgr2=0x7fe5dc0647d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:22:14.781 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:14.783+0000 7fe5e162e640 1 --2- 172.21.15.162:0/2743556354 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe5dc064380 0x7fe5dc0647d0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fe5cc0048d0 tx=0x7fe5cc009da0 comp rx=0 tx=0).stop 2024-04-27T04:22:14.781 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:14.783+0000 7fe5e162e640 1 -- 172.21.15.162:0/2743556354 shutdown_connections 2024-04-27T04:22:14.781 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:14.783+0000 7fe5e162e640 1 --2- 172.21.15.162:0/2743556354 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe5dc064380 0x7fe5dc0647d0 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:22:14.781 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:14.783+0000 7fe5e162e640 1 --2- 172.21.15.162:0/2743556354 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe5dc1021e0 0x7fe5dc1025b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:22:14.781 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:14.783+0000 7fe5e162e640 1 -- 172.21.15.162:0/2743556354 >> 172.21.15.162:0/2743556354 conn(0x7fe5dc0710c0 msgr2=0x7fe5dc0714c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:22:14.781 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:14.783+0000 7fe5e162e640 1 -- 172.21.15.162:0/2743556354 shutdown_connections 2024-04-27T04:22:14.781 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:14.783+0000 7fe5e162e640 1 -- 172.21.15.162:0/2743556354 wait complete. 2024-04-27T04:22:14.781 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:14.783+0000 7fe5e162e640 1 Processor -- start 2024-04-27T04:22:14.781 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:14.783+0000 7fe5e162e640 1 -- start start 2024-04-27T04:22:14.784 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:14.784+0000 7fe5e162e640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe5dc064380 0x7fe5dc112c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:22:14.784 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:14.784+0000 7fe5e162e640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe5dc1021e0 0x7fe5dc1131b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:22:14.784 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:14.784+0000 7fe5e162e640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5dc1137b0 con 0x7fe5dc1021e0 2024-04-27T04:22:14.784 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:14.784+0000 7fe5e162e640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5dc10ce00 con 0x7fe5dc064380 2024-04-27T04:22:14.784 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:14.784+0000 7fe5da7fc640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe5dc1021e0 0x7fe5dc1131b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:22:14.784 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:14.784+0000 7fe5da7fc640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe5dc1021e0 0x7fe5dc1131b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:40702/0 (socket says 172.21.15.162:40702) 2024-04-27T04:22:14.784 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:14.784+0000 7fe5da7fc640 1 -- 172.21.15.162:0/2820054044 learned_addr learned my addr 172.21.15.162:0/2820054044 (peer_addr_for_me v2:172.21.15.162:0/0) 2024-04-27T04:22:14.784 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:14.784+0000 7fe5daffd640 1 --2- 172.21.15.162:0/2820054044 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe5dc064380 0x7fe5dc112c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:22:14.784 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:14.784+0000 7fe5da7fc640 1 -- 172.21.15.162:0/2820054044 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe5dc064380 msgr2=0x7fe5dc112c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:22:14.784 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:14.784+0000 7fe5da7fc640 1 --2- 172.21.15.162:0/2820054044 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe5dc064380 0x7fe5dc112c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:22:14.785 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:14.784+0000 7fe5da7fc640 1 -- 172.21.15.162:0/2820054044 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5cc005040 con 0x7fe5dc1021e0 2024-04-27T04:22:14.785 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:14.784+0000 7fe5daffd640 1 --2- 172.21.15.162:0/2820054044 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe5dc064380 0x7fe5dc112c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-27T04:22:14.785 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:14.784+0000 7fe5da7fc640 1 --2- 172.21.15.162:0/2820054044 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe5dc1021e0 0x7fe5dc1131b0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fe5cc004930 tx=0x7fe5cc009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:22:14.785 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:14.784+0000 7fe5c37fe640 1 -- 172.21.15.162:0/2820054044 <== mon.0 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe5cc0094c0 con 0x7fe5dc1021e0 2024-04-27T04:22:14.785 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:14.784+0000 7fe5c37fe640 1 -- 172.21.15.162:0/2820054044 <== mon.0 v2:172.21.15.162:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fe5cc032e50 con 0x7fe5dc1021e0 2024-04-27T04:22:14.785 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:14.784+0000 7fe5e162e640 1 -- 172.21.15.162:0/2820054044 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe5dc10d080 con 0x7fe5dc1021e0 2024-04-27T04:22:14.785 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:14.784+0000 7fe5c37fe640 1 -- 172.21.15.162:0/2820054044 <== mon.0 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe5cc0433b0 con 0x7fe5dc1021e0 2024-04-27T04:22:14.785 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:14.784+0000 7fe5e162e640 1 -- 172.21.15.162:0/2820054044 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe5dc10d540 con 0x7fe5dc1021e0 2024-04-27T04:22:14.785 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:14.785+0000 7fe5e162e640 1 -- 172.21.15.162:0/2820054044 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe5dc0fd8e0 con 0x7fe5dc1021e0 2024-04-27T04:22:14.785 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:14.786+0000 7fe5c37fe640 1 -- 172.21.15.162:0/2820054044 <== mon.0 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 39) v1 ==== 92747+0+0 (secure 0 0 0) 0x7fe5cc02f070 con 0x7fe5dc1021e0 2024-04-27T04:22:14.785 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:14.786+0000 7fe5c37fe640 1 --2- 172.21.15.162:0/2820054044 >> [v2:172.21.15.162:6800/1396866108,v1:172.21.15.162:6801/1396866108] conn(0x7fe5c406eb70 0x7fe5c4071020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:22:14.785 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:14.786+0000 7fe5c37fe640 1 -- 172.21.15.162:0/2820054044 <== mon.0 v2:172.21.15.162:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6582+0+0 (secure 0 0 0) 0x7fe5cc0b0910 con 0x7fe5dc1021e0 2024-04-27T04:22:14.785 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:14.786+0000 7fe5daffd640 1 --2- 172.21.15.162:0/2820054044 >> [v2:172.21.15.162:6800/1396866108,v1:172.21.15.162:6801/1396866108] conn(0x7fe5c406eb70 0x7fe5c4071020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:22:14.786 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:14.787+0000 7fe5daffd640 1 --2- 172.21.15.162:0/2820054044 >> [v2:172.21.15.162:6800/1396866108,v1:172.21.15.162:6801/1396866108] conn(0x7fe5c406eb70 0x7fe5c4071020 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fe5d0004c80 tx=0x7fe5d0009040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:22:14.788 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:14.791+0000 7fe5c37fe640 1 -- 172.21.15.162:0/2820054044 <== mon.0 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7fe5cc079660 con 0x7fe5dc1021e0 2024-04-27T04:22:14.967 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:14.969+0000 7fe5e162e640 1 -- 172.21.15.162:0/2820054044 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs.cephfs.meta", "var": "pg_num"} v 0) v1 -- 0x7fe5dc1092b0 con 0x7fe5dc1021e0 2024-04-27T04:22:14.967 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:14.970+0000 7fe5c37fe640 1 -- 172.21.15.162:0/2820054044 <== mon.0 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs.cephfs.meta", "var": "pg_num"}]=0 v84) v1 ==== 107+0+11 (secure 0 0 0) 0x7fe5cc07cd20 con 0x7fe5dc1021e0 2024-04-27T04:22:14.967 INFO:teuthology.orchestra.run.smithi162.stdout:pg_num: 32 2024-04-27T04:22:14.969 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:14.971+0000 7fe5e162e640 1 -- 172.21.15.162:0/2820054044 >> [v2:172.21.15.162:6800/1396866108,v1:172.21.15.162:6801/1396866108] conn(0x7fe5c406eb70 msgr2=0x7fe5c4071020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:22:14.969 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:14.971+0000 7fe5e162e640 1 --2- 172.21.15.162:0/2820054044 >> [v2:172.21.15.162:6800/1396866108,v1:172.21.15.162:6801/1396866108] conn(0x7fe5c406eb70 0x7fe5c4071020 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fe5d0004c80 tx=0x7fe5d0009040 comp rx=0 tx=0).stop 2024-04-27T04:22:14.969 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:14.971+0000 7fe5e162e640 1 -- 172.21.15.162:0/2820054044 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe5dc1021e0 msgr2=0x7fe5dc1131b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:22:14.970 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:14.971+0000 7fe5e162e640 1 --2- 172.21.15.162:0/2820054044 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe5dc1021e0 0x7fe5dc1131b0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fe5cc004930 tx=0x7fe5cc009da0 comp rx=0 tx=0).stop 2024-04-27T04:22:14.970 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:14.972+0000 7fe5e162e640 1 -- 172.21.15.162:0/2820054044 shutdown_connections 2024-04-27T04:22:14.970 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:14.972+0000 7fe5e162e640 1 --2- 172.21.15.162:0/2820054044 >> [v2:172.21.15.162:6800/1396866108,v1:172.21.15.162:6801/1396866108] conn(0x7fe5c406eb70 0x7fe5c4071020 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:22:14.970 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:14.972+0000 7fe5e162e640 1 --2- 172.21.15.162:0/2820054044 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe5dc1021e0 0x7fe5dc1131b0 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:22:14.970 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:14.972+0000 7fe5e162e640 1 --2- 172.21.15.162:0/2820054044 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe5dc064380 0x7fe5dc112c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:22:14.970 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:14.972+0000 7fe5e162e640 1 -- 172.21.15.162:0/2820054044 >> 172.21.15.162:0/2820054044 conn(0x7fe5dc0710c0 msgr2=0x7fe5dc0f9490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:22:14.970 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:14.972+0000 7fe5e162e640 1 -- 172.21.15.162:0/2820054044 shutdown_connections 2024-04-27T04:22:14.970 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:14.972+0000 7fe5e162e640 1 -- 172.21.15.162:0/2820054044 wait complete. 2024-04-27T04:22:14.979 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:22:14 smithi162 ceph-mon[110641]: pgmap v276: 65 pgs: 65 active+clean; 208 MiB data, 851 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-04-27T04:22:14.979 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:22:14 smithi162 ceph-mon[110641]: from='client.? 172.21.15.162:0/2768633668' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 34, "format": "json"}]: dispatch 2024-04-27T04:22:14.979 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:22:14 smithi162 ceph-mon[110641]: from='client.? 172.21.15.162:0/352486400' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-27T04:22:14.979 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:22:14 smithi162 ceph-mon[110641]: from='client.? 172.21.15.162:0/3999888938' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-27T04:22:14.980 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs.cephfs.data pg_num 2024-04-27T04:22:15.079 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:15.079+0000 7fa3ac31d640 1 -- 172.21.15.162:0/1813163633 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa3a4102b30 msgr2=0x7fa3a40fc1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:22:15.079 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:15.079+0000 7fa3ac31d640 1 --2- 172.21.15.162:0/1813163633 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa3a4102b30 0x7fa3a40fc1c0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fa3940048d0 tx=0x7fa394009da0 comp rx=0 tx=0).stop 2024-04-27T04:22:15.080 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:15.079+0000 7fa3ac31d640 1 -- 172.21.15.162:0/1813163633 shutdown_connections 2024-04-27T04:22:15.080 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:15.079+0000 7fa3ac31d640 1 --2- 172.21.15.162:0/1813163633 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa3a4102b30 0x7fa3a40fc1c0 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:22:15.080 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:15.079+0000 7fa3ac31d640 1 --2- 172.21.15.162:0/1813163633 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa3a4102190 0x7fa3a4102560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:22:15.080 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:15.079+0000 7fa3ac31d640 1 -- 172.21.15.162:0/1813163633 >> 172.21.15.162:0/1813163633 conn(0x7fa3a40f7da0 msgr2=0x7fa3a40fa1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:22:15.080 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:15.079+0000 7fa3ac31d640 1 -- 172.21.15.162:0/1813163633 shutdown_connections 2024-04-27T04:22:15.080 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:15.079+0000 7fa3ac31d640 1 -- 172.21.15.162:0/1813163633 wait complete. 2024-04-27T04:22:15.080 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:15.079+0000 7fa3ac31d640 1 Processor -- start 2024-04-27T04:22:15.080 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:15.080+0000 7fa3ac31d640 1 -- start start 2024-04-27T04:22:15.080 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:15.080+0000 7fa3ac31d640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa3a4102190 0x7fa3a4112cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:22:15.080 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:15.080+0000 7fa3ac31d640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa3a4102b30 0x7fa3a4113210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:22:15.081 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:15.080+0000 7fa3ac31d640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3a4113810 con 0x7fa3a4102b30 2024-04-27T04:22:15.081 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:15.080+0000 7fa3ac31d640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3a410ce60 con 0x7fa3a4102190 2024-04-27T04:22:15.081 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:15.080+0000 7fa3a98b8640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa3a4102b30 0x7fa3a4113210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:22:15.081 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:15.080+0000 7fa3a98b8640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa3a4102b30 0x7fa3a4113210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:40728/0 (socket says 172.21.15.162:40728) 2024-04-27T04:22:15.081 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:15.080+0000 7fa3a98b8640 1 -- 172.21.15.162:0/654626209 learned_addr learned my addr 172.21.15.162:0/654626209 (peer_addr_for_me v2:172.21.15.162:0/0) 2024-04-27T04:22:15.081 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:15.080+0000 7fa3aa0b9640 1 --2- 172.21.15.162:0/654626209 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa3a4102190 0x7fa3a4112cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:22:15.081 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:15.080+0000 7fa3a98b8640 1 -- 172.21.15.162:0/654626209 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa3a4102190 msgr2=0x7fa3a4112cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:22:15.081 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:15.080+0000 7fa3a98b8640 1 --2- 172.21.15.162:0/654626209 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa3a4102190 0x7fa3a4112cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:22:15.081 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:15.080+0000 7fa3a98b8640 1 -- 172.21.15.162:0/654626209 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa394005040 con 0x7fa3a4102b30 2024-04-27T04:22:15.081 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:15.080+0000 7fa3a98b8640 1 --2- 172.21.15.162:0/654626209 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa3a4102b30 0x7fa3a4113210 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fa394004930 tx=0x7fa394009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:22:15.081 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:15.080+0000 7fa39affd640 1 -- 172.21.15.162:0/654626209 <== mon.0 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa3940094c0 con 0x7fa3a4102b30 2024-04-27T04:22:15.081 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:15.080+0000 7fa39affd640 1 -- 172.21.15.162:0/654626209 <== mon.0 v2:172.21.15.162:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fa394032e50 con 0x7fa3a4102b30 2024-04-27T04:22:15.081 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:15.080+0000 7fa3ac31d640 1 -- 172.21.15.162:0/654626209 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa3a410d0e0 con 0x7fa3a4102b30 2024-04-27T04:22:15.081 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:15.080+0000 7fa39affd640 1 -- 172.21.15.162:0/654626209 <== mon.0 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa3940433b0 con 0x7fa3a4102b30 2024-04-27T04:22:15.082 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:15.080+0000 7fa3ac31d640 1 -- 172.21.15.162:0/654626209 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa3a410d5a0 con 0x7fa3a4102b30 2024-04-27T04:22:15.082 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:15.082+0000 7fa3ac31d640 1 -- 172.21.15.162:0/654626209 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa3a40fd900 con 0x7fa3a4102b30 2024-04-27T04:22:15.082 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:15.082+0000 7fa39affd640 1 -- 172.21.15.162:0/654626209 <== mon.0 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 39) v1 ==== 92747+0+0 (secure 0 0 0) 0x7fa39402f070 con 0x7fa3a4102b30 2024-04-27T04:22:15.083 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:15.082+0000 7fa39affd640 1 --2- 172.21.15.162:0/654626209 >> [v2:172.21.15.162:6800/1396866108,v1:172.21.15.162:6801/1396866108] conn(0x7fa39006eb70 0x7fa390071020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:22:15.083 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:15.082+0000 7fa39affd640 1 -- 172.21.15.162:0/654626209 <== mon.0 v2:172.21.15.162:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6582+0+0 (secure 0 0 0) 0x7fa3940b0910 con 0x7fa3a4102b30 2024-04-27T04:22:15.083 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:15.082+0000 7fa3aa0b9640 1 --2- 172.21.15.162:0/654626209 >> [v2:172.21.15.162:6800/1396866108,v1:172.21.15.162:6801/1396866108] conn(0x7fa39006eb70 0x7fa390071020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:22:15.083 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:15.083+0000 7fa3aa0b9640 1 --2- 172.21.15.162:0/654626209 >> [v2:172.21.15.162:6800/1396866108,v1:172.21.15.162:6801/1396866108] conn(0x7fa39006eb70 0x7fa390071020 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fa3a0004c40 tx=0x7fa3a0009040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:22:15.084 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:15.087+0000 7fa39affd640 1 -- 172.21.15.162:0/654626209 <== mon.0 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7fa394079660 con 0x7fa3a4102b30 2024-04-27T04:22:15.171 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:22:14 smithi181 ceph-mon[81134]: pgmap v276: 65 pgs: 65 active+clean; 208 MiB data, 851 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-04-27T04:22:15.171 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:22:14 smithi181 ceph-mon[81134]: from='client.? 172.21.15.162:0/2768633668' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 34, "format": "json"}]: dispatch 2024-04-27T04:22:15.172 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:22:14 smithi181 ceph-mon[81134]: from='client.? 172.21.15.162:0/352486400' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-27T04:22:15.172 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:22:14 smithi181 ceph-mon[81134]: from='client.? 172.21.15.162:0/3999888938' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-27T04:22:15.259 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:15.261+0000 7fa3ac31d640 1 -- 172.21.15.162:0/654626209 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs.cephfs.data", "var": "pg_num"} v 0) v1 -- 0x7fa3a404bcc0 con 0x7fa3a4102b30 2024-04-27T04:22:15.260 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:15.262+0000 7fa39affd640 1 -- 172.21.15.162:0/654626209 <== mon.0 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs.cephfs.data", "var": "pg_num"}]=0 v84) v1 ==== 107+0+11 (secure 0 0 0) 0x7fa39407cd20 con 0x7fa3a4102b30 2024-04-27T04:22:15.260 INFO:teuthology.orchestra.run.smithi162.stdout:pg_num: 32 2024-04-27T04:22:15.262 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:15.264+0000 7fa3ac31d640 1 -- 172.21.15.162:0/654626209 >> [v2:172.21.15.162:6800/1396866108,v1:172.21.15.162:6801/1396866108] conn(0x7fa39006eb70 msgr2=0x7fa390071020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:22:15.263 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:15.264+0000 7fa3ac31d640 1 --2- 172.21.15.162:0/654626209 >> [v2:172.21.15.162:6800/1396866108,v1:172.21.15.162:6801/1396866108] conn(0x7fa39006eb70 0x7fa390071020 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fa3a0004c40 tx=0x7fa3a0009040 comp rx=0 tx=0).stop 2024-04-27T04:22:15.263 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:15.264+0000 7fa3ac31d640 1 -- 172.21.15.162:0/654626209 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa3a4102b30 msgr2=0x7fa3a4113210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:22:15.263 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:15.264+0000 7fa3ac31d640 1 --2- 172.21.15.162:0/654626209 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa3a4102b30 0x7fa3a4113210 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fa394004930 tx=0x7fa394009da0 comp rx=0 tx=0).stop 2024-04-27T04:22:15.264 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:15.264+0000 7fa3ac31d640 1 -- 172.21.15.162:0/654626209 shutdown_connections 2024-04-27T04:22:15.264 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:15.264+0000 7fa3ac31d640 1 --2- 172.21.15.162:0/654626209 >> [v2:172.21.15.162:6800/1396866108,v1:172.21.15.162:6801/1396866108] conn(0x7fa39006eb70 0x7fa390071020 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:22:15.264 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:15.264+0000 7fa3ac31d640 1 --2- 172.21.15.162:0/654626209 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa3a4102b30 0x7fa3a4113210 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:22:15.264 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:15.264+0000 7fa3ac31d640 1 --2- 172.21.15.162:0/654626209 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa3a4102190 0x7fa3a4112cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:22:15.264 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:15.264+0000 7fa3ac31d640 1 -- 172.21.15.162:0/654626209 >> 172.21.15.162:0/654626209 conn(0x7fa3a40f7da0 msgr2=0x7fa3a40f8cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:22:15.264 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:15.264+0000 7fa3ac31d640 1 -- 172.21.15.162:0/654626209 shutdown_connections 2024-04-27T04:22:15.265 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:15.264+0000 7fa3ac31d640 1 -- 172.21.15.162:0/654626209 wait complete. 2024-04-27T04:22:15.273 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-27T04:22:15.373 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:15.370+0000 7f768a31d640 1 -- 172.21.15.162:0/2305002140 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7684102b50 msgr2=0x7f76840fc1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:22:15.373 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:15.370+0000 7f768a31d640 1 --2- 172.21.15.162:0/2305002140 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7684102b50 0x7f76840fc1e0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f76780048d0 tx=0x7f7678009da0 comp rx=0 tx=0).stop 2024-04-27T04:22:15.373 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:15.370+0000 7f768a31d640 1 -- 172.21.15.162:0/2305002140 shutdown_connections 2024-04-27T04:22:15.373 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:15.370+0000 7f768a31d640 1 --2- 172.21.15.162:0/2305002140 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7684102b50 0x7f76840fc1e0 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:22:15.374 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:15.370+0000 7f768a31d640 1 --2- 172.21.15.162:0/2305002140 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f76841021b0 0x7f7684102580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:22:15.374 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:15.370+0000 7f768a31d640 1 -- 172.21.15.162:0/2305002140 >> 172.21.15.162:0/2305002140 conn(0x7f76840f7da0 msgr2=0x7f76840fa1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:22:15.374 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:15.370+0000 7f768a31d640 1 -- 172.21.15.162:0/2305002140 shutdown_connections 2024-04-27T04:22:15.374 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:15.370+0000 7f768a31d640 1 -- 172.21.15.162:0/2305002140 wait complete. 2024-04-27T04:22:15.374 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:15.371+0000 7f768a31d640 1 Processor -- start 2024-04-27T04:22:15.374 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:15.371+0000 7f768a31d640 1 -- start start 2024-04-27T04:22:15.375 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:15.371+0000 7f768a31d640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f76841021b0 0x7f7684112d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:22:15.375 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:15.371+0000 7f768a31d640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7684102b50 0x7f7684113240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:22:15.375 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:15.371+0000 7f768a31d640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7684113840 con 0x7f7684102b50 2024-04-27T04:22:15.375 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:15.371+0000 7f768a31d640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f768410ce90 con 0x7f76841021b0 2024-04-27T04:22:15.375 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:15.371+0000 7f76837fe640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7684102b50 0x7f7684113240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:22:15.375 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:15.371+0000 7f76837fe640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7684102b50 0x7f7684113240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:40746/0 (socket says 172.21.15.162:40746) 2024-04-27T04:22:15.376 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:15.371+0000 7f76837fe640 1 -- 172.21.15.162:0/2932826646 learned_addr learned my addr 172.21.15.162:0/2932826646 (peer_addr_for_me v2:172.21.15.162:0/0) 2024-04-27T04:22:15.376 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:15.371+0000 7f7683fff640 1 --2- 172.21.15.162:0/2932826646 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f76841021b0 0x7f7684112d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:22:15.376 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:15.371+0000 7f76837fe640 1 -- 172.21.15.162:0/2932826646 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f76841021b0 msgr2=0x7f7684112d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:22:15.376 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:15.371+0000 7f76837fe640 1 --2- 172.21.15.162:0/2932826646 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f76841021b0 0x7f7684112d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:22:15.376 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:15.371+0000 7f76837fe640 1 -- 172.21.15.162:0/2932826646 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7678005040 con 0x7f7684102b50 2024-04-27T04:22:15.376 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:15.372+0000 7f76837fe640 1 --2- 172.21.15.162:0/2932826646 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7684102b50 0x7f7684113240 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f7678004930 tx=0x7f7678009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:22:15.377 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:15.372+0000 7f7680ff9640 1 -- 172.21.15.162:0/2932826646 <== mon.0 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f76780094c0 con 0x7f7684102b50 2024-04-27T04:22:15.377 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:15.372+0000 7f768a31d640 1 -- 172.21.15.162:0/2932826646 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f768410d110 con 0x7f7684102b50 2024-04-27T04:22:15.377 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:15.372+0000 7f7680ff9640 1 -- 172.21.15.162:0/2932826646 <== mon.0 v2:172.21.15.162:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f7678032e50 con 0x7f7684102b50 2024-04-27T04:22:15.377 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:15.372+0000 7f7680ff9640 1 -- 172.21.15.162:0/2932826646 <== mon.0 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f76780433b0 con 0x7f7684102b50 2024-04-27T04:22:15.377 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:15.372+0000 7f768a31d640 1 -- 172.21.15.162:0/2932826646 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f768410d5d0 con 0x7f7684102b50 2024-04-27T04:22:15.377 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:15.373+0000 7f768a31d640 1 -- 172.21.15.162:0/2932826646 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f76840fd920 con 0x7f7684102b50 2024-04-27T04:22:15.378 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:15.373+0000 7f7680ff9640 1 -- 172.21.15.162:0/2932826646 <== mon.0 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 39) v1 ==== 92747+0+0 (secure 0 0 0) 0x7f767802f070 con 0x7f7684102b50 2024-04-27T04:22:15.378 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:15.374+0000 7f7680ff9640 1 --2- 172.21.15.162:0/2932826646 >> [v2:172.21.15.162:6800/1396866108,v1:172.21.15.162:6801/1396866108] conn(0x7f766c06eb70 0x7f766c071020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:22:15.378 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:15.374+0000 7f7680ff9640 1 -- 172.21.15.162:0/2932826646 <== mon.0 v2:172.21.15.162:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6582+0+0 (secure 0 0 0) 0x7f76780b0920 con 0x7f7684102b50 2024-04-27T04:22:15.378 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:15.374+0000 7f7683fff640 1 --2- 172.21.15.162:0/2932826646 >> [v2:172.21.15.162:6800/1396866108,v1:172.21.15.162:6801/1396866108] conn(0x7f766c06eb70 0x7f766c071020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:22:15.378 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:15.374+0000 7f7683fff640 1 --2- 172.21.15.162:0/2932826646 >> [v2:172.21.15.162:6800/1396866108,v1:172.21.15.162:6801/1396866108] conn(0x7f766c06eb70 0x7f766c071020 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f7674004bc0 tx=0x7f7674009040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:22:15.379 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:15.378+0000 7f7680ff9640 1 -- 172.21.15.162:0/2932826646 <== mon.0 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7f7678079670 con 0x7f7684102b50 2024-04-27T04:22:15.566 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:15.566+0000 7f768a31d640 1 -- 172.21.15.162:0/2932826646 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f76841092f0 con 0x7f7684102b50 2024-04-27T04:22:15.567 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:15.567+0000 7f7680ff9640 1 -- 172.21.15.162:0/2932826646 <== mon.0 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 41 entries v84) v1 ==== 81+0+2458 (secure 0 0 0) 0x7f767807cd30 con 0x7f7684102b50 2024-04-27T04:22:15.567 INFO:teuthology.orchestra.run.smithi162.stdout:172.21.15.181:6827/3756475453 2024-04-28T04:19:50.546716+0000 2024-04-27T04:22:15.567 INFO:teuthology.orchestra.run.smithi162.stdout:172.21.15.181:6826/3756475453 2024-04-28T04:19:50.546716+0000 2024-04-27T04:22:15.567 INFO:teuthology.orchestra.run.smithi162.stdout:172.21.15.162:0/1045614319 2024-04-28T04:14:17.593083+0000 2024-04-27T04:22:15.568 INFO:teuthology.orchestra.run.smithi162.stdout:172.21.15.181:6828/4165336392 2024-04-28T04:11:32.210928+0000 2024-04-27T04:22:15.568 INFO:teuthology.orchestra.run.smithi162.stdout:172.21.15.162:0/2336042321 2024-04-28T04:02:16.661212+0000 2024-04-27T04:22:15.568 INFO:teuthology.orchestra.run.smithi162.stdout:172.21.15.162:6800/4079782599 2024-04-28T04:10:05.654758+0000 2024-04-27T04:22:15.568 INFO:teuthology.orchestra.run.smithi162.stdout:172.21.15.181:6824/658214993 2024-04-28T04:19:26.414790+0000 2024-04-27T04:22:15.568 INFO:teuthology.orchestra.run.smithi162.stdout:172.21.15.181:6829/4165336392 2024-04-28T04:11:32.210928+0000 2024-04-27T04:22:15.568 INFO:teuthology.orchestra.run.smithi162.stdout:172.21.15.181:0/642670640 2024-04-28T04:11:32.210928+0000 2024-04-27T04:22:15.568 INFO:teuthology.orchestra.run.smithi162.stdout:172.21.15.162:0/3828848945 2024-04-28T04:14:17.593083+0000 2024-04-27T04:22:15.568 INFO:teuthology.orchestra.run.smithi162.stdout:172.21.15.162:0/1971036271 2024-04-28T04:10:05.654758+0000 2024-04-27T04:22:15.569 INFO:teuthology.orchestra.run.smithi162.stdout:172.21.15.162:6800/8067198 2024-04-28T04:01:10.996874+0000 2024-04-27T04:22:15.569 INFO:teuthology.orchestra.run.smithi162.stdout:172.21.15.162:0/2623011446 2024-04-28T04:00:40.642043+0000 2024-04-27T04:22:15.569 INFO:teuthology.orchestra.run.smithi162.stdout:172.21.15.162:0/1703886136 2024-04-28T04:02:16.661212+0000 2024-04-27T04:22:15.569 INFO:teuthology.orchestra.run.smithi162.stdout:172.21.15.162:0/2129785584 2024-04-28T04:01:10.996874+0000 2024-04-27T04:22:15.569 INFO:teuthology.orchestra.run.smithi162.stdout:172.21.15.162:6800/1482851081 2024-04-28T04:02:16.661212+0000 2024-04-27T04:22:15.569 INFO:teuthology.orchestra.run.smithi162.stdout:172.21.15.162:6801/1480340393 2024-04-28T04:00:40.642043+0000 2024-04-27T04:22:15.569 INFO:teuthology.orchestra.run.smithi162.stdout:172.21.15.181:0/209590021 2024-04-28T04:11:32.210928+0000 2024-04-27T04:22:15.569 INFO:teuthology.orchestra.run.smithi162.stdout:172.21.15.162:0/4025365522 2024-04-28T04:00:40.642043+0000 2024-04-27T04:22:15.570 INFO:teuthology.orchestra.run.smithi162.stdout:172.21.15.181:0/3276305180 2024-04-28T04:11:32.210928+0000 2024-04-27T04:22:15.570 INFO:teuthology.orchestra.run.smithi162.stdout:172.21.15.162:6801/1482851081 2024-04-28T04:02:16.661212+0000 2024-04-27T04:22:15.570 INFO:teuthology.orchestra.run.smithi162.stdout:172.21.15.162:0/3073351439 2024-04-28T04:01:10.996874+0000 2024-04-27T04:22:15.570 INFO:teuthology.orchestra.run.smithi162.stdout:172.21.15.181:6825/658214993 2024-04-28T04:19:26.414790+0000 2024-04-27T04:22:15.570 INFO:teuthology.orchestra.run.smithi162.stdout:172.21.15.162:0/795779642 2024-04-28T04:02:16.661212+0000 2024-04-27T04:22:15.570 INFO:teuthology.orchestra.run.smithi162.stdout:172.21.15.162:0/3590827229 2024-04-28T04:01:10.996874+0000 2024-04-27T04:22:15.570 INFO:teuthology.orchestra.run.smithi162.stdout:172.21.15.181:0/2462932364 2024-04-28T04:11:32.210928+0000 2024-04-27T04:22:15.570 INFO:teuthology.orchestra.run.smithi162.stdout:172.21.15.162:6801/8067198 2024-04-28T04:01:10.996874+0000 2024-04-27T04:22:15.570 INFO:teuthology.orchestra.run.smithi162.stdout:172.21.15.162:6826/998468496 2024-04-28T04:07:08.543563+0000 2024-04-27T04:22:15.571 INFO:teuthology.orchestra.run.smithi162.stdout:172.21.15.162:0/1349712416 2024-04-28T04:10:05.654758+0000 2024-04-27T04:22:15.571 INFO:teuthology.orchestra.run.smithi162.stdout:172.21.15.162:6800/1480340393 2024-04-28T04:00:40.642043+0000 2024-04-27T04:22:15.571 INFO:teuthology.orchestra.run.smithi162.stdout:172.21.15.181:0/1936190707 2024-04-28T04:11:32.210928+0000 2024-04-27T04:22:15.571 INFO:teuthology.orchestra.run.smithi162.stdout:172.21.15.181:0/2761247390 2024-04-28T04:11:32.210928+0000 2024-04-27T04:22:15.571 INFO:teuthology.orchestra.run.smithi162.stdout:172.21.15.162:6801/4079782599 2024-04-28T04:10:05.654758+0000 2024-04-27T04:22:15.571 INFO:teuthology.orchestra.run.smithi162.stdout:172.21.15.162:0/3944767668 2024-04-28T04:10:05.654758+0000 2024-04-27T04:22:15.571 INFO:teuthology.orchestra.run.smithi162.stdout:172.21.15.162:0/2579239891 2024-04-28T04:00:40.642043+0000 2024-04-27T04:22:15.571 INFO:teuthology.orchestra.run.smithi162.stdout:172.21.15.162:6827/998468496 2024-04-28T04:07:08.543563+0000 2024-04-27T04:22:15.572 INFO:teuthology.orchestra.run.smithi162.stdout:172.21.15.162:6800/901992993 2024-04-28T04:14:17.593083+0000 2024-04-27T04:22:15.572 INFO:teuthology.orchestra.run.smithi162.stdout:172.21.15.162:0/2783280109 2024-04-28T04:10:05.654758+0000 2024-04-27T04:22:15.572 INFO:teuthology.orchestra.run.smithi162.stdout:172.21.15.162:6801/901992993 2024-04-28T04:14:17.593083+0000 2024-04-27T04:22:15.572 INFO:teuthology.orchestra.run.smithi162.stdout:172.21.15.162:0/1622403042 2024-04-28T04:14:17.593083+0000 2024-04-27T04:22:15.572 INFO:teuthology.orchestra.run.smithi162.stdout:172.21.15.162:0/2023772646 2024-04-28T04:14:17.593083+0000 2024-04-27T04:22:15.573 INFO:teuthology.orchestra.run.smithi162.stderr:listed 41 entries 2024-04-27T04:22:15.573 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:15.569+0000 7f768a31d640 1 -- 172.21.15.162:0/2932826646 >> [v2:172.21.15.162:6800/1396866108,v1:172.21.15.162:6801/1396866108] conn(0x7f766c06eb70 msgr2=0x7f766c071020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:22:15.573 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:15.569+0000 7f768a31d640 1 --2- 172.21.15.162:0/2932826646 >> [v2:172.21.15.162:6800/1396866108,v1:172.21.15.162:6801/1396866108] conn(0x7f766c06eb70 0x7f766c071020 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f7674004bc0 tx=0x7f7674009040 comp rx=0 tx=0).stop 2024-04-27T04:22:15.574 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:15.569+0000 7f768a31d640 1 -- 172.21.15.162:0/2932826646 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7684102b50 msgr2=0x7f7684113240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:22:15.574 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:15.569+0000 7f768a31d640 1 --2- 172.21.15.162:0/2932826646 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7684102b50 0x7f7684113240 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f7678004930 tx=0x7f7678009da0 comp rx=0 tx=0).stop 2024-04-27T04:22:15.574 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:15.569+0000 7f768a31d640 1 -- 172.21.15.162:0/2932826646 shutdown_connections 2024-04-27T04:22:15.574 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:15.569+0000 7f768a31d640 1 --2- 172.21.15.162:0/2932826646 >> [v2:172.21.15.162:6800/1396866108,v1:172.21.15.162:6801/1396866108] conn(0x7f766c06eb70 0x7f766c071020 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:22:15.574 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:15.569+0000 7f768a31d640 1 --2- 172.21.15.162:0/2932826646 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7684102b50 0x7f7684113240 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:22:15.574 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:15.569+0000 7f768a31d640 1 --2- 172.21.15.162:0/2932826646 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f76841021b0 0x7f7684112d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:22:15.574 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:15.569+0000 7f768a31d640 1 -- 172.21.15.162:0/2932826646 >> 172.21.15.162:0/2932826646 conn(0x7f76840f7da0 msgr2=0x7f76840f8d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:22:15.574 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:15.569+0000 7f768a31d640 1 -- 172.21.15.162:0/2932826646 shutdown_connections 2024-04-27T04:22:15.575 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:15.569+0000 7f768a31d640 1 -- 172.21.15.162:0/2932826646 wait complete. 2024-04-27T04:22:15.577 DEBUG:teuthology.orchestra.run.smithi162:> set -ex 2024-04-27T04:22:15.578 DEBUG:teuthology.orchestra.run.smithi162:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-27T04:22:15.594 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-27T04:22:15.731 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:15.731+0000 7f06bb55d640 1 -- 172.21.15.162:0/4030129199 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f06b4102190 msgr2=0x7f06b4102560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:22:15.731 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:15.731+0000 7f06bb55d640 1 --2- 172.21.15.162:0/4030129199 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f06b4102190 0x7f06b4102560 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f06b00048d0 tx=0x7f06b0009da0 comp rx=0 tx=0).stop 2024-04-27T04:22:15.731 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:15.732+0000 7f06bb55d640 1 -- 172.21.15.162:0/4030129199 shutdown_connections 2024-04-27T04:22:15.731 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:15.732+0000 7f06bb55d640 1 --2- 172.21.15.162:0/4030129199 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f06b4102b30 0x7f06b40fc1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:22:15.731 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:15.732+0000 7f06bb55d640 1 --2- 172.21.15.162:0/4030129199 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f06b4102190 0x7f06b4102560 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:22:15.731 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:15.732+0000 7f06bb55d640 1 -- 172.21.15.162:0/4030129199 >> 172.21.15.162:0/4030129199 conn(0x7f06b40f7d60 msgr2=0x7f06b40fa170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:22:15.732 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:15.732+0000 7f06bb55d640 1 -- 172.21.15.162:0/4030129199 shutdown_connections 2024-04-27T04:22:15.732 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:15.732+0000 7f06bb55d640 1 -- 172.21.15.162:0/4030129199 wait complete. 2024-04-27T04:22:15.732 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:15.732+0000 7f06bb55d640 1 Processor -- start 2024-04-27T04:22:15.732 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:15.732+0000 7f06bb55d640 1 -- start start 2024-04-27T04:22:15.732 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:15.732+0000 7f06bb55d640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f06b4102190 0x7f06b4112cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:22:15.732 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:15.732+0000 7f06bb55d640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f06b4102b30 0x7f06b41131f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:22:15.732 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:15.732+0000 7f06bb55d640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06b41137f0 con 0x7f06b4102190 2024-04-27T04:22:15.732 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:15.732+0000 7f06bb55d640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06b410ce40 con 0x7f06b4102b30 2024-04-27T04:22:15.732 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:15.732+0000 7f06b92f9640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f06b4102190 0x7f06b4112cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:22:15.732 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:15.732+0000 7f06b92f9640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f06b4102190 0x7f06b4112cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:40764/0 (socket says 172.21.15.162:40764) 2024-04-27T04:22:15.732 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:15.732+0000 7f06b92f9640 1 -- 172.21.15.162:0/1362104705 learned_addr learned my addr 172.21.15.162:0/1362104705 (peer_addr_for_me v2:172.21.15.162:0/0) 2024-04-27T04:22:15.733 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:15.733+0000 7f06b92f9640 1 -- 172.21.15.162:0/1362104705 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f06b4102b30 msgr2=0x7f06b41131f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:22:15.733 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:15.733+0000 7f06b92f9640 1 --2- 172.21.15.162:0/1362104705 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f06b4102b30 0x7f06b41131f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:22:15.733 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:15.733+0000 7f06b92f9640 1 -- 172.21.15.162:0/1362104705 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f06b0005040 con 0x7f06b4102190 2024-04-27T04:22:15.733 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:15.733+0000 7f06b92f9640 1 --2- 172.21.15.162:0/1362104705 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f06b4102190 0x7f06b4112cb0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f06b0004930 tx=0x7f06b0009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:22:15.733 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:15.733+0000 7f06a9ffb640 1 -- 172.21.15.162:0/1362104705 <== mon.0 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f06b00094c0 con 0x7f06b4102190 2024-04-27T04:22:15.733 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:15.733+0000 7f06bb55d640 1 -- 172.21.15.162:0/1362104705 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f06b410d0c0 con 0x7f06b4102190 2024-04-27T04:22:15.733 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:15.733+0000 7f06a9ffb640 1 -- 172.21.15.162:0/1362104705 <== mon.0 v2:172.21.15.162:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f06b0032e50 con 0x7f06b4102190 2024-04-27T04:22:15.733 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:15.733+0000 7f06a9ffb640 1 -- 172.21.15.162:0/1362104705 <== mon.0 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f06b00433b0 con 0x7f06b4102190 2024-04-27T04:22:15.733 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:15.733+0000 7f06bb55d640 1 -- 172.21.15.162:0/1362104705 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f06b410d580 con 0x7f06b4102190 2024-04-27T04:22:15.734 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:15.734+0000 7f06bb55d640 1 -- 172.21.15.162:0/1362104705 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f06b40fd900 con 0x7f06b4102190 2024-04-27T04:22:15.734 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:15.735+0000 7f06a9ffb640 1 -- 172.21.15.162:0/1362104705 <== mon.0 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 39) v1 ==== 92747+0+0 (secure 0 0 0) 0x7f06b002f070 con 0x7f06b4102190 2024-04-27T04:22:15.734 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:15.735+0000 7f06a9ffb640 1 --2- 172.21.15.162:0/1362104705 >> [v2:172.21.15.162:6800/1396866108,v1:172.21.15.162:6801/1396866108] conn(0x7f06a006eb70 0x7f06a0071020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:22:15.734 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:15.735+0000 7f06a9ffb640 1 -- 172.21.15.162:0/1362104705 <== mon.0 v2:172.21.15.162:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6582+0+0 (secure 0 0 0) 0x7f06b00b07c0 con 0x7f06b4102190 2024-04-27T04:22:15.734 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:15.735+0000 7f06b8af8640 1 --2- 172.21.15.162:0/1362104705 >> [v2:172.21.15.162:6800/1396866108,v1:172.21.15.162:6801/1396866108] conn(0x7f06a006eb70 0x7f06a0071020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:22:15.735 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:15.736+0000 7f06b8af8640 1 --2- 172.21.15.162:0/1362104705 >> [v2:172.21.15.162:6800/1396866108,v1:172.21.15.162:6801/1396866108] conn(0x7f06a006eb70 0x7f06a0071020 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f06b410e570 tx=0x7f06a4009c50 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:22:15.737 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:15.740+0000 7f06a9ffb640 1 -- 172.21.15.162:0/1362104705 <== mon.0 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7f06b0079510 con 0x7f06b4102190 2024-04-27T04:22:15.916 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:15.918+0000 7f06bb55d640 1 -- 172.21.15.162:0/1362104705 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f06b404bcc0 con 0x7f06b4102190 2024-04-27T04:22:15.917 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:15.920+0000 7f06a9ffb640 1 -- 172.21.15.162:0/1362104705 <== mon.0 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v84) v1 ==== 74+0+16846 (secure 0 0 0) 0x7f06b007cbd0 con 0x7f06b4102190 2024-04-27T04:22:15.917 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-04-27T04:22:15.918 INFO:teuthology.orchestra.run.smithi162.stdout:{"epoch":84,"fsid":"7f9750d6-044a-11ef-bc93-c7b262605968","created":"2024-04-27T04:00:06.343616+0000","modified":"2024-04-27T04:19:50.546745+0000","last_up_change":"2024-04-27T04:18:43.492214+0000","last_in_change":"2024-04-27T04:05:18.597695+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":20,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":6,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-27T04:04:24.971779+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":6,"score_stable":6,"optimal_score":0.5,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs.cephfs.meta","create_time":"2024-04-27T04:06:52.008169+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"44","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.690000057220459,"score_stable":1.690000057220459,"optimal_score":1,"raw_score_acting":1.690000057220459,"raw_score_stable":1.690000057220459,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs.cephfs.data","create_time":"2024-04-27T04:06:52.322198+0000","flags":131073,"flags_names":"hashpspool,bulk","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"45","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0ae69d7f-0ab4-4672-89b9-3563a10a1bf2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":9,"last_clean_end":49,"up_from":53,"up_thru":78,"down_at":50,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6802","nonce":1840117633},{"type":"v1","addr":"172.21.15.162:6803","nonce":1840117633}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6804","nonce":1840117633},{"type":"v1","addr":"172.21.15.162:6805","nonce":1840117633}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6808","nonce":1840117633},{"type":"v1","addr":"172.21.15.162:6809","nonce":1840117633}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6806","nonce":1840117633},{"type":"v1","addr":"172.21.15.162:6807","nonce":1840117633}]},"public_addr":"172.21.15.162:6803/1840117633","cluster_addr":"172.21.15.162:6805/1840117633","heartbeat_back_addr":"172.21.15.162:6809/1840117633","heartbeat_front_addr":"172.21.15.162:6807/1840117633","state":["exists","up"]},{"osd":1,"uuid":"95bdc9f0-c438-4a88-942d-dffb0c61c308","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":14,"last_clean_end":54,"up_from":58,"up_thru":78,"down_at":55,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6810","nonce":2144127279},{"type":"v1","addr":"172.21.15.162:6811","nonce":2144127279}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6812","nonce":2144127279},{"type":"v1","addr":"172.21.15.162:6813","nonce":2144127279}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6816","nonce":2144127279},{"type":"v1","addr":"172.21.15.162:6817","nonce":2144127279}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6814","nonce":2144127279},{"type":"v1","addr":"172.21.15.162:6815","nonce":2144127279}]},"public_addr":"172.21.15.162:6811/2144127279","cluster_addr":"172.21.15.162:6813/2144127279","heartbeat_back_addr":"172.21.15.162:6817/2144127279","heartbeat_front_addr":"172.21.15.162:6815/2144127279","state":["exists","up"]},{"osd":2,"uuid":"a773b386-e75b-432a-9d1b-2206301b96c3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":19,"last_clean_end":59,"up_from":63,"up_thru":78,"down_at":60,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6818","nonce":3115093010},{"type":"v1","addr":"172.21.15.162:6819","nonce":3115093010}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6820","nonce":3115093010},{"type":"v1","addr":"172.21.15.162:6821","nonce":3115093010}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6824","nonce":3115093010},{"type":"v1","addr":"172.21.15.162:6825","nonce":3115093010}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6822","nonce":3115093010},{"type":"v1","addr":"172.21.15.162:6823","nonce":3115093010}]},"public_addr":"172.21.15.162:6819/3115093010","cluster_addr":"172.21.15.162:6821/3115093010","heartbeat_back_addr":"172.21.15.162:6825/3115093010","heartbeat_front_addr":"172.21.15.162:6823/3115093010","state":["exists","up"]},{"osd":3,"uuid":"756091bc-1594-479d-9390-cb60e6b65ebe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":26,"last_clean_end":64,"up_from":68,"up_thru":78,"down_at":65,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":114683138},{"type":"v1","addr":"172.21.15.181:6801","nonce":114683138}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":114683138},{"type":"v1","addr":"172.21.15.181:6803","nonce":114683138}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6806","nonce":114683138},{"type":"v1","addr":"172.21.15.181:6807","nonce":114683138}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":114683138},{"type":"v1","addr":"172.21.15.181:6805","nonce":114683138}]},"public_addr":"172.21.15.181:6801/114683138","cluster_addr":"172.21.15.181:6803/114683138","heartbeat_back_addr":"172.21.15.181:6807/114683138","heartbeat_front_addr":"172.21.15.181:6805/114683138","state":["exists","up"]},{"osd":4,"uuid":"bd70494d-061c-4468-b837-26150c2d4208","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":34,"last_clean_end":69,"up_from":73,"up_thru":78,"down_at":70,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6808","nonce":1781129695},{"type":"v1","addr":"172.21.15.181:6809","nonce":1781129695}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":1781129695},{"type":"v1","addr":"172.21.15.181:6811","nonce":1781129695}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":1781129695},{"type":"v1","addr":"172.21.15.181:6815","nonce":1781129695}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6812","nonce":1781129695},{"type":"v1","addr":"172.21.15.181:6813","nonce":1781129695}]},"public_addr":"172.21.15.181:6809/1781129695","cluster_addr":"172.21.15.181:6811/1781129695","heartbeat_back_addr":"172.21.15.181:6815/1781129695","heartbeat_front_addr":"172.21.15.181:6813/1781129695","state":["exists","up"]},{"osd":5,"uuid":"15455ef6-2853-4a84-aaa9-1797e78302a9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":39,"last_clean_end":74,"up_from":78,"up_thru":78,"down_at":75,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":2414821263},{"type":"v1","addr":"172.21.15.181:6817","nonce":2414821263}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6818","nonce":2414821263},{"type":"v1","addr":"172.21.15.181:6819","nonce":2414821263}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6822","nonce":2414821263},{"type":"v1","addr":"172.21.15.181:6823","nonce":2414821263}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":2414821263},{"type":"v1","addr":"172.21.15.181:6821","nonce":2414821263}]},"public_addr":"172.21.15.181:6817/2414821263","cluster_addr":"172.21.15.181:6819/2414821263","heartbeat_back_addr":"172.21.15.181:6823/2414821263","heartbeat_front_addr":"172.21.15.181:6821/2414821263","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-27T04:03:28.522704+0000","dead_epoch":49},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-27T04:03:54.648083+0000","dead_epoch":54},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-27T04:04:22.107316+0000","dead_epoch":59},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-27T04:04:46.020733+0000","dead_epoch":64},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-27T04:05:10.634082+0000","dead_epoch":69},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-27T04:05:35.756217+0000","dead_epoch":74}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.181:6827/3756475453":"2024-04-28T04:19:50.546716+0000","172.21.15.181:6826/3756475453":"2024-04-28T04:19:50.546716+0000","172.21.15.162:0/1045614319":"2024-04-28T04:14:17.593083+0000","172.21.15.181:6828/4165336392":"2024-04-28T04:11:32.210928+0000","172.21.15.162:0/2336042321":"2024-04-28T04:02:16.661212+0000","172.21.15.162:6800/4079782599":"2024-04-28T04:10:05.654758+0000","172.21.15.181:6824/658214993":"2024-04-28T04:19:26.414790+0000","172.21.15.181:6829/4165336392":"2024-04-28T04:11:32.210928+0000","172.21.15.181:0/642670640":"2024-04-28T04:11:32.210928+0000","172.21.15.162:0/3828848945":"2024-04-28T04:14:17.593083+0000","172.21.15.162:0/1971036271":"2024-04-28T04:10:05.654758+0000","172.21.15.162:6800/8067198":"2024-04-28T04:01:10.996874+0000","172.21.15.162:0/2623011446":"2024-04-28T04:00:40.642043+0000","172.21.15.162:0/1703886136":"2024-04-28T04:02:16.661212+0000","172.21.15.162:0/2129785584":"2024-04-28T04:01:10.996874+0000","172.21.15.162:6800/1482851081":"2024-04-28T04:02:16.661212+0000","172.21.15.162:6801/1480340393":"2024-04-28T04:00:40.642043+0000","172.21.15.181:0/209590021":"2024-04-28T04:11:32.210928+0000","172.21.15.162:0/4025365522":"2024-04-28T04:00:40.642043+0000","172.21.15.181:0/3276305180":"2024-04-28T04:11:32.210928+0000","172.21.15.162:6801/1482851081":"2024-04-28T04:02:16.661212+0000","172.21.15.162:0/3073351439":"2024-04-28T04:01:10.996874+0000","172.21.15.181:6825/658214993":"2024-04-28T04:19:26.414790+0000","172.21.15.162:0/795779642":"2024-04-28T04:02:16.661212+0000","172.21.15.162:0/3590827229":"2024-04-28T04:01:10.996874+0000","172.21.15.181:0/2462932364":"2024-04-28T04:11:32.210928+0000","172.21.15.162:6801/8067198":"2024-04-28T04:01:10.996874+0000","172.21.15.162:6826/998468496":"2024-04-28T04:07:08.543563+0000","172.21.15.162:0/1349712416":"2024-04-28T04:10:05.654758+0000","172.21.15.162:6800/1480340393":"2024-04-28T04:00:40.642043+0000","172.21.15.181:0/1936190707":"2024-04-28T04:11:32.210928+0000","172.21.15.181:0/2761247390":"2024-04-28T04:11:32.210928+0000","172.21.15.162:6801/4079782599":"2024-04-28T04:10:05.654758+0000","172.21.15.162:0/3944767668":"2024-04-28T04:10:05.654758+0000","172.21.15.162:0/2579239891":"2024-04-28T04:00:40.642043+0000","172.21.15.162:6827/998468496":"2024-04-28T04:07:08.543563+0000","172.21.15.162:6800/901992993":"2024-04-28T04:14:17.593083+0000","172.21.15.162:0/2783280109":"2024-04-28T04:10:05.654758+0000","172.21.15.162:6801/901992993":"2024-04-28T04:14:17.593083+0000","172.21.15.162:0/1622403042":"2024-04-28T04:14:17.593083+0000","172.21.15.162:0/2023772646":"2024-04-28T04:14:17.593083+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-27T04:22:15.920 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:15.922+0000 7f06bb55d640 1 -- 172.21.15.162:0/1362104705 >> [v2:172.21.15.162:6800/1396866108,v1:172.21.15.162:6801/1396866108] conn(0x7f06a006eb70 msgr2=0x7f06a0071020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:22:15.920 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:15.922+0000 7f06bb55d640 1 --2- 172.21.15.162:0/1362104705 >> [v2:172.21.15.162:6800/1396866108,v1:172.21.15.162:6801/1396866108] conn(0x7f06a006eb70 0x7f06a0071020 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f06b410e570 tx=0x7f06a4009c50 comp rx=0 tx=0).stop 2024-04-27T04:22:15.920 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:15.922+0000 7f06bb55d640 1 -- 172.21.15.162:0/1362104705 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f06b4102190 msgr2=0x7f06b4112cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:22:15.920 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:15.922+0000 7f06bb55d640 1 --2- 172.21.15.162:0/1362104705 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f06b4102190 0x7f06b4112cb0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f06b0004930 tx=0x7f06b0009da0 comp rx=0 tx=0).stop 2024-04-27T04:22:15.920 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:15.922+0000 7f06bb55d640 1 -- 172.21.15.162:0/1362104705 shutdown_connections 2024-04-27T04:22:15.920 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:15.922+0000 7f06bb55d640 1 --2- 172.21.15.162:0/1362104705 >> [v2:172.21.15.162:6800/1396866108,v1:172.21.15.162:6801/1396866108] conn(0x7f06a006eb70 0x7f06a0071020 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:22:15.920 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:15.922+0000 7f06bb55d640 1 --2- 172.21.15.162:0/1362104705 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f06b4102b30 0x7f06b41131f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:22:15.920 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:15.922+0000 7f06bb55d640 1 --2- 172.21.15.162:0/1362104705 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f06b4102190 0x7f06b4112cb0 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:22:15.920 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:15.922+0000 7f06bb55d640 1 -- 172.21.15.162:0/1362104705 >> 172.21.15.162:0/1362104705 conn(0x7f06b40f7d60 msgr2=0x7f06b40f8c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:22:15.920 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:15.922+0000 7f06bb55d640 1 -- 172.21.15.162:0/1362104705 shutdown_connections 2024-04-27T04:22:15.920 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:15.922+0000 7f06bb55d640 1 -- 172.21.15.162:0/1362104705 wait complete. 2024-04-27T04:22:15.930 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-27T04:04:24.971779+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 6, 'score_stable': 6, 'optimal_score': 0.5, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs.cephfs.meta', 'create_time': '2024-04-27T04:06:52.008169+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '44', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.690000057220459, 'score_stable': 1.690000057220459, 'optimal_score': 1, 'raw_score_acting': 1.690000057220459, 'raw_score_stable': 1.690000057220459, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs.cephfs.data', 'create_time': '2024-04-27T04:06:52.322198+0000', 'flags': 131073, 'flags_names': 'hashpspool,bulk', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '45', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-27T04:22:15.931 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-27T04:22:16.030 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:16.031+0000 7f9b00b9d640 1 -- 172.21.15.162:0/843751012 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9afc0fa840 msgr2=0x7f9afc0facb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:22:16.031 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:16.031+0000 7f9b00b9d640 1 --2- 172.21.15.162:0/843751012 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9afc0fa840 0x7f9afc0facb0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f9af00048d0 tx=0x7f9af0009da0 comp rx=0 tx=0).stop 2024-04-27T04:22:16.031 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:16.031+0000 7f9b00b9d640 1 -- 172.21.15.162:0/843751012 shutdown_connections 2024-04-27T04:22:16.031 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:16.031+0000 7f9b00b9d640 1 --2- 172.21.15.162:0/843751012 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9afc0fa840 0x7f9afc0facb0 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:22:16.031 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:16.031+0000 7f9b00b9d640 1 --2- 172.21.15.162:0/843751012 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9afc102240 0x7f9afc102610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:22:16.031 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:16.031+0000 7f9b00b9d640 1 -- 172.21.15.162:0/843751012 >> 172.21.15.162:0/843751012 conn(0x7f9afc071700 msgr2=0x7f9afc071b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:22:16.031 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:16.031+0000 7f9b00b9d640 1 -- 172.21.15.162:0/843751012 shutdown_connections 2024-04-27T04:22:16.031 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:16.031+0000 7f9b00b9d640 1 -- 172.21.15.162:0/843751012 wait complete. 2024-04-27T04:22:16.031 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:16.031+0000 7f9b00b9d640 1 Processor -- start 2024-04-27T04:22:16.031 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:16.031+0000 7f9b00b9d640 1 -- start start 2024-04-27T04:22:16.032 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:16.031+0000 7f9b00b9d640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9afc0fa840 0x7f9afc112da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:22:16.032 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:16.031+0000 7f9b00b9d640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9afc102240 0x7f9afc1132e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:22:16.032 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:16.031+0000 7f9b00b9d640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9afc113970 con 0x7f9afc0fa840 2024-04-27T04:22:16.032 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:16.031+0000 7f9b00b9d640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9afc10cf30 con 0x7f9afc102240 2024-04-27T04:22:16.032 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:16.031+0000 7f9afa59c640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9afc0fa840 0x7f9afc112da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:22:16.032 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:16.031+0000 7f9afa59c640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9afc0fa840 0x7f9afc112da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:40786/0 (socket says 172.21.15.162:40786) 2024-04-27T04:22:16.032 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:16.031+0000 7f9afa59c640 1 -- 172.21.15.162:0/3786947198 learned_addr learned my addr 172.21.15.162:0/3786947198 (peer_addr_for_me v2:172.21.15.162:0/0) 2024-04-27T04:22:16.032 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:16.032+0000 7f9af9d9b640 1 --2- 172.21.15.162:0/3786947198 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9afc102240 0x7f9afc1132e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:22:16.032 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:16.032+0000 7f9afa59c640 1 -- 172.21.15.162:0/3786947198 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9afc102240 msgr2=0x7f9afc1132e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:22:16.032 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:16.032+0000 7f9afa59c640 1 --2- 172.21.15.162:0/3786947198 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9afc102240 0x7f9afc1132e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:22:16.032 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:16.032+0000 7f9afa59c640 1 -- 172.21.15.162:0/3786947198 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9af0005040 con 0x7f9afc0fa840 2024-04-27T04:22:16.032 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:16.032+0000 7f9af9d9b640 1 --2- 172.21.15.162:0/3786947198 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9afc102240 0x7f9afc1132e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-27T04:22:16.032 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:16.032+0000 7f9afa59c640 1 --2- 172.21.15.162:0/3786947198 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9afc0fa840 0x7f9afc112da0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f9aec00b580 tx=0x7f9aec0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:22:16.033 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:16.032+0000 7f9aeaffd640 1 -- 172.21.15.162:0/3786947198 <== mon.0 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9aec00f570 con 0x7f9afc0fa840 2024-04-27T04:22:16.033 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:16.032+0000 7f9aeaffd640 1 -- 172.21.15.162:0/3786947198 <== mon.0 v2:172.21.15.162:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f9aec00fba0 con 0x7f9afc0fa840 2024-04-27T04:22:16.033 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:16.032+0000 7f9b00b9d640 1 -- 172.21.15.162:0/3786947198 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9afc10d210 con 0x7f9afc0fa840 2024-04-27T04:22:16.033 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:16.032+0000 7f9aeaffd640 1 -- 172.21.15.162:0/3786947198 <== mon.0 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9aec010dd0 con 0x7f9afc0fa840 2024-04-27T04:22:16.033 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:16.032+0000 7f9b00b9d640 1 -- 172.21.15.162:0/3786947198 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9afc10d730 con 0x7f9afc0fa840 2024-04-27T04:22:16.033 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:16.033+0000 7f9b00b9d640 1 -- 172.21.15.162:0/3786947198 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9afc0652b0 con 0x7f9afc0fa840 2024-04-27T04:22:16.034 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:16.034+0000 7f9aeaffd640 1 -- 172.21.15.162:0/3786947198 <== mon.0 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 39) v1 ==== 92747+0+0 (secure 0 0 0) 0x7f9aec00f6d0 con 0x7f9afc0fa840 2024-04-27T04:22:16.034 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:16.034+0000 7f9aeaffd640 1 --2- 172.21.15.162:0/3786947198 >> [v2:172.21.15.162:6800/1396866108,v1:172.21.15.162:6801/1396866108] conn(0x7f9ae406eb70 0x7f9ae4071020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:22:16.034 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:16.034+0000 7f9aeaffd640 1 -- 172.21.15.162:0/3786947198 <== mon.0 v2:172.21.15.162:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6582+0+0 (secure 0 0 0) 0x7f9aec08e810 con 0x7f9afc0fa840 2024-04-27T04:22:16.034 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:16.034+0000 7f9af9d9b640 1 --2- 172.21.15.162:0/3786947198 >> [v2:172.21.15.162:6800/1396866108,v1:172.21.15.162:6801/1396866108] conn(0x7f9ae406eb70 0x7f9ae4071020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:22:16.034 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:16.034+0000 7f9af9d9b640 1 --2- 172.21.15.162:0/3786947198 >> [v2:172.21.15.162:6800/1396866108,v1:172.21.15.162:6801/1396866108] conn(0x7f9ae406eb70 0x7f9ae4071020 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f9af0000c00 tx=0x7f9af0009da0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:22:16.036 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:16.039+0000 7f9aeaffd640 1 -- 172.21.15.162:0/3786947198 <== mon.0 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7f9aec0575e0 con 0x7f9afc0fa840 2024-04-27T04:22:16.136 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:22:15 smithi162 ceph-mon[110641]: from='client.? 172.21.15.162:0/2820054044' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs.cephfs.meta", "var": "pg_num"}]: dispatch 2024-04-27T04:22:16.136 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:22:15 smithi162 ceph-mon[110641]: from='client.? 172.21.15.162:0/654626209' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs.cephfs.data", "var": "pg_num"}]: dispatch 2024-04-27T04:22:16.136 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:22:15 smithi162 ceph-mon[110641]: from='client.? 172.21.15.162:0/2932826646' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-04-27T04:22:16.171 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:22:15 smithi181 ceph-mon[81134]: from='client.? 172.21.15.162:0/2820054044' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs.cephfs.meta", "var": "pg_num"}]: dispatch 2024-04-27T04:22:16.171 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:22:15 smithi181 ceph-mon[81134]: from='client.? 172.21.15.162:0/654626209' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs.cephfs.data", "var": "pg_num"}]: dispatch 2024-04-27T04:22:16.171 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:22:15 smithi181 ceph-mon[81134]: from='client.? 172.21.15.162:0/2932826646' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-04-27T04:22:16.224 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:16.226+0000 7f9b00b9d640 1 -- 172.21.15.162:0/3786947198 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f9afc10e440 con 0x7f9afc0fa840 2024-04-27T04:22:16.224 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:16.226+0000 7f9aeaffd640 1 -- 172.21.15.162:0/3786947198 <== mon.0 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v84) v1 ==== 93+0+10 (secure 0 0 0) 0x7f9aec05aca0 con 0x7f9afc0fa840 2024-04-27T04:22:16.224 INFO:teuthology.orchestra.run.smithi162.stdout:pg_num: 1 2024-04-27T04:22:16.226 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:16.228+0000 7f9b00b9d640 1 -- 172.21.15.162:0/3786947198 >> [v2:172.21.15.162:6800/1396866108,v1:172.21.15.162:6801/1396866108] conn(0x7f9ae406eb70 msgr2=0x7f9ae4071020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:22:16.226 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:16.228+0000 7f9b00b9d640 1 --2- 172.21.15.162:0/3786947198 >> [v2:172.21.15.162:6800/1396866108,v1:172.21.15.162:6801/1396866108] conn(0x7f9ae406eb70 0x7f9ae4071020 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f9af0000c00 tx=0x7f9af0009da0 comp rx=0 tx=0).stop 2024-04-27T04:22:16.226 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:16.228+0000 7f9b00b9d640 1 -- 172.21.15.162:0/3786947198 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9afc0fa840 msgr2=0x7f9afc112da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:22:16.227 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:16.228+0000 7f9b00b9d640 1 --2- 172.21.15.162:0/3786947198 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9afc0fa840 0x7f9afc112da0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f9aec00b580 tx=0x7f9aec0091a0 comp rx=0 tx=0).stop 2024-04-27T04:22:16.227 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:16.229+0000 7f9b00b9d640 1 -- 172.21.15.162:0/3786947198 shutdown_connections 2024-04-27T04:22:16.227 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:16.229+0000 7f9b00b9d640 1 --2- 172.21.15.162:0/3786947198 >> [v2:172.21.15.162:6800/1396866108,v1:172.21.15.162:6801/1396866108] conn(0x7f9ae406eb70 0x7f9ae4071020 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:22:16.227 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:16.229+0000 7f9b00b9d640 1 --2- 172.21.15.162:0/3786947198 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9afc102240 0x7f9afc1132e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:22:16.227 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:16.229+0000 7f9b00b9d640 1 --2- 172.21.15.162:0/3786947198 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9afc0fa840 0x7f9afc112da0 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:22:16.227 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:16.229+0000 7f9b00b9d640 1 -- 172.21.15.162:0/3786947198 >> 172.21.15.162:0/3786947198 conn(0x7f9afc071700 msgr2=0x7f9afc0f9a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:22:16.227 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:16.229+0000 7f9b00b9d640 1 -- 172.21.15.162:0/3786947198 shutdown_connections 2024-04-27T04:22:16.227 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:16.229+0000 7f9b00b9d640 1 -- 172.21.15.162:0/3786947198 wait complete. 2024-04-27T04:22:16.237 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs.cephfs.meta pg_num 2024-04-27T04:22:16.336 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:16.336+0000 7f9bf019d640 1 -- 172.21.15.162:0/3306897429 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9be8102b30 msgr2=0x7f9be80fc1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:22:16.336 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:16.336+0000 7f9bf019d640 1 --2- 172.21.15.162:0/3306897429 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9be8102b30 0x7f9be80fc1c0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f9bd80048d0 tx=0x7f9bd8009da0 comp rx=0 tx=0).stop 2024-04-27T04:22:16.336 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:16.336+0000 7f9bf019d640 1 -- 172.21.15.162:0/3306897429 shutdown_connections 2024-04-27T04:22:16.336 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:16.336+0000 7f9bf019d640 1 --2- 172.21.15.162:0/3306897429 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9be8102b30 0x7f9be80fc1c0 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:22:16.336 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:16.336+0000 7f9bf019d640 1 --2- 172.21.15.162:0/3306897429 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9be8102190 0x7f9be8102560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:22:16.336 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:16.336+0000 7f9bf019d640 1 -- 172.21.15.162:0/3306897429 >> 172.21.15.162:0/3306897429 conn(0x7f9be80f7da0 msgr2=0x7f9be80fa1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:22:16.336 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:16.336+0000 7f9bf019d640 1 -- 172.21.15.162:0/3306897429 shutdown_connections 2024-04-27T04:22:16.336 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:16.336+0000 7f9bf019d640 1 -- 172.21.15.162:0/3306897429 wait complete. 2024-04-27T04:22:16.336 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:16.337+0000 7f9bf019d640 1 Processor -- start 2024-04-27T04:22:16.336 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:16.337+0000 7f9bf019d640 1 -- start start 2024-04-27T04:22:16.336 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:16.337+0000 7f9bf019d640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9be8102190 0x7f9be8112d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:22:16.337 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:16.337+0000 7f9bf019d640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9be8102b30 0x7f9be8113240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:22:16.337 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:16.337+0000 7f9bf019d640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9be8113840 con 0x7f9be8102190 2024-04-27T04:22:16.337 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:16.337+0000 7f9bf019d640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9be810ce90 con 0x7f9be8102b30 2024-04-27T04:22:16.337 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:16.337+0000 7f9bedf39640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9be8102190 0x7f9be8112d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:22:16.337 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:16.337+0000 7f9bedf39640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9be8102190 0x7f9be8112d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:40802/0 (socket says 172.21.15.162:40802) 2024-04-27T04:22:16.337 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:16.337+0000 7f9bedf39640 1 -- 172.21.15.162:0/2134198180 learned_addr learned my addr 172.21.15.162:0/2134198180 (peer_addr_for_me v2:172.21.15.162:0/0) 2024-04-27T04:22:16.337 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:16.337+0000 7f9bedf39640 1 -- 172.21.15.162:0/2134198180 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9be8102b30 msgr2=0x7f9be8113240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:22:16.337 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:16.337+0000 7f9bedf39640 1 --2- 172.21.15.162:0/2134198180 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9be8102b30 0x7f9be8113240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:22:16.337 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:16.337+0000 7f9bedf39640 1 -- 172.21.15.162:0/2134198180 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9bd8005040 con 0x7f9be8102190 2024-04-27T04:22:16.337 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:16.338+0000 7f9bedf39640 1 --2- 172.21.15.162:0/2134198180 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9be8102190 0x7f9be8112d00 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f9be400db00 tx=0x7f9be4004e60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:22:16.337 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:16.338+0000 7f9bde7fc640 1 -- 172.21.15.162:0/2134198180 <== mon.0 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9be4017070 con 0x7f9be8102190 2024-04-27T04:22:16.337 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:16.338+0000 7f9bde7fc640 1 -- 172.21.15.162:0/2134198180 <== mon.0 v2:172.21.15.162:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f9be400f040 con 0x7f9be8102190 2024-04-27T04:22:16.337 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:16.338+0000 7f9bf019d640 1 -- 172.21.15.162:0/2134198180 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9be810d170 con 0x7f9be8102190 2024-04-27T04:22:16.337 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:16.338+0000 7f9bde7fc640 1 -- 172.21.15.162:0/2134198180 <== mon.0 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9be4003ae0 con 0x7f9be8102190 2024-04-27T04:22:16.338 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:16.338+0000 7f9bf019d640 1 -- 172.21.15.162:0/2134198180 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9be810d690 con 0x7f9be8102190 2024-04-27T04:22:16.339 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:16.339+0000 7f9bf019d640 1 -- 172.21.15.162:0/2134198180 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9be80fd900 con 0x7f9be8102190 2024-04-27T04:22:16.339 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:16.339+0000 7f9bde7fc640 1 -- 172.21.15.162:0/2134198180 <== mon.0 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 39) v1 ==== 92747+0+0 (secure 0 0 0) 0x7f9be4003c40 con 0x7f9be8102190 2024-04-27T04:22:16.339 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:16.340+0000 7f9bde7fc640 1 --2- 172.21.15.162:0/2134198180 >> [v2:172.21.15.162:6800/1396866108,v1:172.21.15.162:6801/1396866108] conn(0x7f9bd406eb70 0x7f9bd4071020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:22:16.339 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:16.340+0000 7f9bde7fc640 1 -- 172.21.15.162:0/2134198180 <== mon.0 v2:172.21.15.162:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6582+0+0 (secure 0 0 0) 0x7f9be408e990 con 0x7f9be8102190 2024-04-27T04:22:16.339 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:16.340+0000 7f9bed738640 1 --2- 172.21.15.162:0/2134198180 >> [v2:172.21.15.162:6800/1396866108,v1:172.21.15.162:6801/1396866108] conn(0x7f9bd406eb70 0x7f9bd4071020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:22:16.339 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:16.340+0000 7f9bed738640 1 --2- 172.21.15.162:0/2134198180 >> [v2:172.21.15.162:6800/1396866108,v1:172.21.15.162:6801/1396866108] conn(0x7f9bd406eb70 0x7f9bd4071020 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f9bd80043d0 tx=0x7f9bd8009da0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:22:16.342 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:16.344+0000 7f9bde7fc640 1 -- 172.21.15.162:0/2134198180 <== mon.0 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7f9be4057760 con 0x7f9be8102190 2024-04-27T04:22:16.518 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:16.520+0000 7f9bf019d640 1 -- 172.21.15.162:0/2134198180 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs.cephfs.meta", "var": "pg_num"} v 0) v1 -- 0x7f9be810e370 con 0x7f9be8102190 2024-04-27T04:22:16.518 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:16.521+0000 7f9bde7fc640 1 -- 172.21.15.162:0/2134198180 <== mon.0 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs.cephfs.meta", "var": "pg_num"}]=0 v84) v1 ==== 107+0+11 (secure 0 0 0) 0x7f9be405ae20 con 0x7f9be8102190 2024-04-27T04:22:16.519 INFO:teuthology.orchestra.run.smithi162.stdout:pg_num: 32 2024-04-27T04:22:16.521 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:16.523+0000 7f9bf019d640 1 -- 172.21.15.162:0/2134198180 >> [v2:172.21.15.162:6800/1396866108,v1:172.21.15.162:6801/1396866108] conn(0x7f9bd406eb70 msgr2=0x7f9bd4071020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:22:16.521 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:16.523+0000 7f9bf019d640 1 --2- 172.21.15.162:0/2134198180 >> [v2:172.21.15.162:6800/1396866108,v1:172.21.15.162:6801/1396866108] conn(0x7f9bd406eb70 0x7f9bd4071020 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f9bd80043d0 tx=0x7f9bd8009da0 comp rx=0 tx=0).stop 2024-04-27T04:22:16.521 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:16.523+0000 7f9bf019d640 1 -- 172.21.15.162:0/2134198180 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9be8102190 msgr2=0x7f9be8112d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:22:16.521 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:16.523+0000 7f9bf019d640 1 --2- 172.21.15.162:0/2134198180 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9be8102190 0x7f9be8112d00 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f9be400db00 tx=0x7f9be4004e60 comp rx=0 tx=0).stop 2024-04-27T04:22:16.521 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:16.523+0000 7f9bf019d640 1 -- 172.21.15.162:0/2134198180 shutdown_connections 2024-04-27T04:22:16.521 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:16.523+0000 7f9bf019d640 1 --2- 172.21.15.162:0/2134198180 >> [v2:172.21.15.162:6800/1396866108,v1:172.21.15.162:6801/1396866108] conn(0x7f9bd406eb70 0x7f9bd4071020 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:22:16.521 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:16.523+0000 7f9bf019d640 1 --2- 172.21.15.162:0/2134198180 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9be8102b30 0x7f9be8113240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:22:16.521 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:16.523+0000 7f9bf019d640 1 --2- 172.21.15.162:0/2134198180 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9be8102190 0x7f9be8112d00 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:22:16.521 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:16.523+0000 7f9bf019d640 1 -- 172.21.15.162:0/2134198180 >> 172.21.15.162:0/2134198180 conn(0x7f9be80f7da0 msgr2=0x7f9be80f8ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:22:16.521 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:16.523+0000 7f9bf019d640 1 -- 172.21.15.162:0/2134198180 shutdown_connections 2024-04-27T04:22:16.522 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:16.523+0000 7f9bf019d640 1 -- 172.21.15.162:0/2134198180 wait complete. 2024-04-27T04:22:16.532 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs.cephfs.data pg_num 2024-04-27T04:22:16.630 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:16.631+0000 7ff92d19d640 1 -- 172.21.15.162:0/2687191528 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff9280fb950 msgr2=0x7ff9280fbda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:22:16.631 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:16.631+0000 7ff92d19d640 1 --2- 172.21.15.162:0/2687191528 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff9280fb950 0x7ff9280fbda0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7ff9100048d0 tx=0x7ff910009da0 comp rx=0 tx=0).stop 2024-04-27T04:22:16.631 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:16.631+0000 7ff92d19d640 1 -- 172.21.15.162:0/2687191528 shutdown_connections 2024-04-27T04:22:16.631 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:16.631+0000 7ff92d19d640 1 --2- 172.21.15.162:0/2687191528 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff9280fb950 0x7ff9280fbda0 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:22:16.631 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:16.631+0000 7ff92d19d640 1 --2- 172.21.15.162:0/2687191528 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff928101230 0x7ff928101600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:22:16.631 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:16.631+0000 7ff92d19d640 1 -- 172.21.15.162:0/2687191528 >> 172.21.15.162:0/2687191528 conn(0x7ff928071700 msgr2=0x7ff928071b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:22:16.631 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:16.631+0000 7ff92d19d640 1 -- 172.21.15.162:0/2687191528 shutdown_connections 2024-04-27T04:22:16.631 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:16.631+0000 7ff92d19d640 1 -- 172.21.15.162:0/2687191528 wait complete. 2024-04-27T04:22:16.631 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:16.631+0000 7ff92d19d640 1 Processor -- start 2024-04-27T04:22:16.631 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:16.632+0000 7ff92d19d640 1 -- start start 2024-04-27T04:22:16.631 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:16.632+0000 7ff92d19d640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff9280fb950 0x7ff928111cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:22:16.631 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:16.632+0000 7ff92d19d640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff928101230 0x7ff928112200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:22:16.631 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:16.632+0000 7ff92d19d640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff928112800 con 0x7ff9280fb950 2024-04-27T04:22:16.631 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:16.632+0000 7ff92d19d640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff92810be50 con 0x7ff928101230 2024-04-27T04:22:16.632 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:16.632+0000 7ff926d9d640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff9280fb950 0x7ff928111cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:22:16.632 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:16.632+0000 7ff926d9d640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff9280fb950 0x7ff928111cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:40826/0 (socket says 172.21.15.162:40826) 2024-04-27T04:22:16.632 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:16.632+0000 7ff926d9d640 1 -- 172.21.15.162:0/101764281 learned_addr learned my addr 172.21.15.162:0/101764281 (peer_addr_for_me v2:172.21.15.162:0/0) 2024-04-27T04:22:16.632 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:16.632+0000 7ff91ffff640 1 --2- 172.21.15.162:0/101764281 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff928101230 0x7ff928112200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:22:16.632 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:16.632+0000 7ff926d9d640 1 -- 172.21.15.162:0/101764281 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff928101230 msgr2=0x7ff928112200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:22:16.632 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:16.632+0000 7ff926d9d640 1 --2- 172.21.15.162:0/101764281 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff928101230 0x7ff928112200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:22:16.632 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:16.632+0000 7ff926d9d640 1 -- 172.21.15.162:0/101764281 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff910005040 con 0x7ff9280fb950 2024-04-27T04:22:16.632 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:16.632+0000 7ff926d9d640 1 --2- 172.21.15.162:0/101764281 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff9280fb950 0x7ff928111cc0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7ff91800c790 tx=0x7ff9180043c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:22:16.632 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:16.632+0000 7ff91f7fe640 1 -- 172.21.15.162:0/101764281 <== mon.0 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff91800a820 con 0x7ff9280fb950 2024-04-27T04:22:16.632 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:16.632+0000 7ff92d19d640 1 -- 172.21.15.162:0/101764281 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff92810c130 con 0x7ff9280fb950 2024-04-27T04:22:16.632 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:16.632+0000 7ff91f7fe640 1 -- 172.21.15.162:0/101764281 <== mon.0 v2:172.21.15.162:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7ff91800ae50 con 0x7ff9280fb950 2024-04-27T04:22:16.632 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:16.632+0000 7ff91f7fe640 1 -- 172.21.15.162:0/101764281 <== mon.0 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff9180115e0 con 0x7ff9280fb950 2024-04-27T04:22:16.632 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:16.632+0000 7ff92d19d640 1 -- 172.21.15.162:0/101764281 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff92810c650 con 0x7ff9280fb950 2024-04-27T04:22:16.633 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:16.634+0000 7ff92d19d640 1 -- 172.21.15.162:0/101764281 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff9280fc960 con 0x7ff9280fb950 2024-04-27T04:22:16.633 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:16.634+0000 7ff91f7fe640 1 -- 172.21.15.162:0/101764281 <== mon.0 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 39) v1 ==== 92747+0+0 (secure 0 0 0) 0x7ff918012400 con 0x7ff9280fb950 2024-04-27T04:22:16.633 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:16.634+0000 7ff91f7fe640 1 --2- 172.21.15.162:0/101764281 >> [v2:172.21.15.162:6800/1396866108,v1:172.21.15.162:6801/1396866108] conn(0x7ff91406eb70 0x7ff914071020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:22:16.634 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:16.635+0000 7ff91ffff640 1 --2- 172.21.15.162:0/101764281 >> [v2:172.21.15.162:6800/1396866108,v1:172.21.15.162:6801/1396866108] conn(0x7ff91406eb70 0x7ff914071020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:22:16.634 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:16.635+0000 7ff91f7fe640 1 -- 172.21.15.162:0/101764281 <== mon.0 v2:172.21.15.162:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6582+0+0 (secure 0 0 0) 0x7ff91808ead0 con 0x7ff9280fb950 2024-04-27T04:22:16.634 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:16.635+0000 7ff91ffff640 1 --2- 172.21.15.162:0/101764281 >> [v2:172.21.15.162:6800/1396866108,v1:172.21.15.162:6801/1396866108] conn(0x7ff91406eb70 0x7ff914071020 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7ff910000c00 tx=0x7ff910009da0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:22:16.636 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:16.639+0000 7ff91f7fe640 1 -- 172.21.15.162:0/101764281 <== mon.0 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7ff9180578a0 con 0x7ff9280fb950 2024-04-27T04:22:16.816 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:16.818+0000 7ff92d19d640 1 -- 172.21.15.162:0/101764281 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs.cephfs.data", "var": "pg_num"} v 0) v1 -- 0x7ff92810d330 con 0x7ff9280fb950 2024-04-27T04:22:16.816 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:16.819+0000 7ff91f7fe640 1 -- 172.21.15.162:0/101764281 <== mon.0 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs.cephfs.data", "var": "pg_num"}]=0 v84) v1 ==== 107+0+11 (secure 0 0 0) 0x7ff918057030 con 0x7ff9280fb950 2024-04-27T04:22:16.816 INFO:teuthology.orchestra.run.smithi162.stdout:pg_num: 32 2024-04-27T04:22:16.819 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:16.821+0000 7ff92d19d640 1 -- 172.21.15.162:0/101764281 >> [v2:172.21.15.162:6800/1396866108,v1:172.21.15.162:6801/1396866108] conn(0x7ff91406eb70 msgr2=0x7ff914071020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:22:16.819 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:16.821+0000 7ff92d19d640 1 --2- 172.21.15.162:0/101764281 >> [v2:172.21.15.162:6800/1396866108,v1:172.21.15.162:6801/1396866108] conn(0x7ff91406eb70 0x7ff914071020 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7ff910000c00 tx=0x7ff910009da0 comp rx=0 tx=0).stop 2024-04-27T04:22:16.819 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:16.821+0000 7ff92d19d640 1 -- 172.21.15.162:0/101764281 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff9280fb950 msgr2=0x7ff928111cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:22:16.819 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:16.821+0000 7ff92d19d640 1 --2- 172.21.15.162:0/101764281 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff9280fb950 0x7ff928111cc0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7ff91800c790 tx=0x7ff9180043c0 comp rx=0 tx=0).stop 2024-04-27T04:22:16.819 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:16.821+0000 7ff92d19d640 1 -- 172.21.15.162:0/101764281 shutdown_connections 2024-04-27T04:22:16.819 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:16.821+0000 7ff92d19d640 1 --2- 172.21.15.162:0/101764281 >> [v2:172.21.15.162:6800/1396866108,v1:172.21.15.162:6801/1396866108] conn(0x7ff91406eb70 0x7ff914071020 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:22:16.819 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:16.821+0000 7ff92d19d640 1 --2- 172.21.15.162:0/101764281 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff928101230 0x7ff928112200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:22:16.819 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:16.821+0000 7ff92d19d640 1 --2- 172.21.15.162:0/101764281 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff9280fb950 0x7ff928111cc0 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:22:16.819 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:16.821+0000 7ff92d19d640 1 -- 172.21.15.162:0/101764281 >> 172.21.15.162:0/101764281 conn(0x7ff928071700 msgr2=0x7ff9280f9fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:22:16.819 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:16.821+0000 7ff92d19d640 1 -- 172.21.15.162:0/101764281 shutdown_connections 2024-04-27T04:22:16.820 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:16.821+0000 7ff92d19d640 1 -- 172.21.15.162:0/101764281 wait complete. 2024-04-27T04:22:16.830 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-27T04:22:16.927 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:16.927+0000 7f1c8706e640 1 -- 172.21.15.162:0/704643915 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1c80102b60 msgr2=0x7f1c800fa8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:22:16.927 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:16.927+0000 7f1c8706e640 1 --2- 172.21.15.162:0/704643915 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1c80102b60 0x7f1c800fa8a0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f1c6c0048d0 tx=0x7f1c6c009da0 comp rx=0 tx=0).stop 2024-04-27T04:22:16.927 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:16.927+0000 7f1c8706e640 1 -- 172.21.15.162:0/704643915 shutdown_connections 2024-04-27T04:22:16.927 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:16.927+0000 7f1c8706e640 1 --2- 172.21.15.162:0/704643915 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1c80102b60 0x7f1c800fa8a0 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:22:16.927 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:16.927+0000 7f1c8706e640 1 --2- 172.21.15.162:0/704643915 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1c80102250 0x7f1c80102620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:22:16.927 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:16.927+0000 7f1c8706e640 1 -- 172.21.15.162:0/704643915 >> 172.21.15.162:0/704643915 conn(0x7f1c80070890 msgr2=0x7f1c80070c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:22:16.927 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:16.927+0000 7f1c8706e640 1 -- 172.21.15.162:0/704643915 shutdown_connections 2024-04-27T04:22:16.927 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:16.927+0000 7f1c8706e640 1 -- 172.21.15.162:0/704643915 wait complete. 2024-04-27T04:22:16.927 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:16.928+0000 7f1c8706e640 1 Processor -- start 2024-04-27T04:22:16.927 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:16.928+0000 7f1c8706e640 1 -- start start 2024-04-27T04:22:16.927 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:16.928+0000 7f1c8706e640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1c80102250 0x7f1c80112d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:22:16.928 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:16.928+0000 7f1c8706e640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1c80102b60 0x7f1c80113270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:22:16.928 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:16.928+0000 7f1c8706e640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c80113870 con 0x7f1c80102250 2024-04-27T04:22:16.928 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:16.928+0000 7f1c8706e640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c8010cf10 con 0x7f1c80102b60 2024-04-27T04:22:16.928 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:16.928+0000 7f1c84e0a640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1c80102250 0x7f1c80112d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:22:16.928 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:16.928+0000 7f1c84e0a640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1c80102250 0x7f1c80112d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:40842/0 (socket says 172.21.15.162:40842) 2024-04-27T04:22:16.928 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:16.928+0000 7f1c84e0a640 1 -- 172.21.15.162:0/1868936523 learned_addr learned my addr 172.21.15.162:0/1868936523 (peer_addr_for_me v2:172.21.15.162:0/0) 2024-04-27T04:22:16.928 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:16.928+0000 7f1c7ffff640 1 --2- 172.21.15.162:0/1868936523 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1c80102b60 0x7f1c80113270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:22:16.928 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:16.928+0000 7f1c84e0a640 1 -- 172.21.15.162:0/1868936523 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1c80102b60 msgr2=0x7f1c80113270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:22:16.928 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:16.928+0000 7f1c84e0a640 1 --2- 172.21.15.162:0/1868936523 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1c80102b60 0x7f1c80113270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:22:16.928 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:16.928+0000 7f1c84e0a640 1 -- 172.21.15.162:0/1868936523 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c6c005040 con 0x7f1c80102250 2024-04-27T04:22:16.928 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:16.929+0000 7f1c84e0a640 1 --2- 172.21.15.162:0/1868936523 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1c80102250 0x7f1c80112d30 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f1c74002a50 tx=0x7f1c74004600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:22:16.928 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:16.929+0000 7f1c7d7fa640 1 -- 172.21.15.162:0/1868936523 <== mon.0 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1c74007c30 con 0x7f1c80102250 2024-04-27T04:22:16.928 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:16.929+0000 7f1c8706e640 1 -- 172.21.15.162:0/1868936523 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1c8010d1f0 con 0x7f1c80102250 2024-04-27T04:22:16.928 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:16.929+0000 7f1c7d7fa640 1 -- 172.21.15.162:0/1868936523 <== mon.0 v2:172.21.15.162:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f1c7400f040 con 0x7f1c80102250 2024-04-27T04:22:16.928 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:16.929+0000 7f1c7d7fa640 1 -- 172.21.15.162:0/1868936523 <== mon.0 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1c7400db50 con 0x7f1c80102250 2024-04-27T04:22:16.929 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:16.929+0000 7f1c8706e640 1 -- 172.21.15.162:0/1868936523 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1c8010d710 con 0x7f1c80102250 2024-04-27T04:22:16.929 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:16.930+0000 7f1c8706e640 1 -- 172.21.15.162:0/1868936523 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1c800652c0 con 0x7f1c80102250 2024-04-27T04:22:16.930 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:16.930+0000 7f1c7d7fa640 1 -- 172.21.15.162:0/1868936523 <== mon.0 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 39) v1 ==== 92747+0+0 (secure 0 0 0) 0x7f1c7401c400 con 0x7f1c80102250 2024-04-27T04:22:16.930 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:16.931+0000 7f1c7d7fa640 1 --2- 172.21.15.162:0/1868936523 >> [v2:172.21.15.162:6800/1396866108,v1:172.21.15.162:6801/1396866108] conn(0x7f1c7006eb70 0x7f1c70071020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:22:16.930 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:16.931+0000 7f1c7d7fa640 1 -- 172.21.15.162:0/1868936523 <== mon.0 v2:172.21.15.162:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6582+0+0 (secure 0 0 0) 0x7f1c7408e1f0 con 0x7f1c80102250 2024-04-27T04:22:16.930 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:16.931+0000 7f1c7ffff640 1 --2- 172.21.15.162:0/1868936523 >> [v2:172.21.15.162:6800/1396866108,v1:172.21.15.162:6801/1396866108] conn(0x7f1c7006eb70 0x7f1c70071020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:22:16.930 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:16.931+0000 7f1c7ffff640 1 --2- 172.21.15.162:0/1868936523 >> [v2:172.21.15.162:6800/1396866108,v1:172.21.15.162:6801/1396866108] conn(0x7f1c7006eb70 0x7f1c70071020 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f1c6c000c00 tx=0x7f1c6c009da0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:22:16.933 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:16.935+0000 7f1c7d7fa640 1 -- 172.21.15.162:0/1868936523 <== mon.0 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7f1c74056f40 con 0x7f1c80102250 2024-04-27T04:22:17.122 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:17.124+0000 7f1c8706e640 1 -- 172.21.15.162:0/1868936523 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f1c80109340 con 0x7f1c80102250 2024-04-27T04:22:17.122 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:17.124+0000 7f1c7d7fa640 1 -- 172.21.15.162:0/1868936523 <== mon.0 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 41 entries v84) v1 ==== 81+0+2458 (secure 0 0 0) 0x7f1c7405a600 con 0x7f1c80102250 2024-04-27T04:22:17.123 INFO:teuthology.orchestra.run.smithi162.stdout:172.21.15.181:6827/3756475453 2024-04-28T04:19:50.546716+0000 2024-04-27T04:22:17.123 INFO:teuthology.orchestra.run.smithi162.stdout:172.21.15.181:6826/3756475453 2024-04-28T04:19:50.546716+0000 2024-04-27T04:22:17.123 INFO:teuthology.orchestra.run.smithi162.stdout:172.21.15.162:0/1045614319 2024-04-28T04:14:17.593083+0000 2024-04-27T04:22:17.123 INFO:teuthology.orchestra.run.smithi162.stdout:172.21.15.181:6828/4165336392 2024-04-28T04:11:32.210928+0000 2024-04-27T04:22:17.123 INFO:teuthology.orchestra.run.smithi162.stdout:172.21.15.162:0/2336042321 2024-04-28T04:02:16.661212+0000 2024-04-27T04:22:17.123 INFO:teuthology.orchestra.run.smithi162.stdout:172.21.15.162:6800/4079782599 2024-04-28T04:10:05.654758+0000 2024-04-27T04:22:17.123 INFO:teuthology.orchestra.run.smithi162.stdout:172.21.15.181:6824/658214993 2024-04-28T04:19:26.414790+0000 2024-04-27T04:22:17.123 INFO:teuthology.orchestra.run.smithi162.stdout:172.21.15.181:6829/4165336392 2024-04-28T04:11:32.210928+0000 2024-04-27T04:22:17.123 INFO:teuthology.orchestra.run.smithi162.stdout:172.21.15.181:0/642670640 2024-04-28T04:11:32.210928+0000 2024-04-27T04:22:17.123 INFO:teuthology.orchestra.run.smithi162.stdout:172.21.15.162:0/3828848945 2024-04-28T04:14:17.593083+0000 2024-04-27T04:22:17.123 INFO:teuthology.orchestra.run.smithi162.stdout:172.21.15.162:0/1971036271 2024-04-28T04:10:05.654758+0000 2024-04-27T04:22:17.123 INFO:teuthology.orchestra.run.smithi162.stdout:172.21.15.162:6800/8067198 2024-04-28T04:01:10.996874+0000 2024-04-27T04:22:17.124 INFO:teuthology.orchestra.run.smithi162.stdout:172.21.15.162:0/2623011446 2024-04-28T04:00:40.642043+0000 2024-04-27T04:22:17.124 INFO:teuthology.orchestra.run.smithi162.stdout:172.21.15.162:0/1703886136 2024-04-28T04:02:16.661212+0000 2024-04-27T04:22:17.124 INFO:teuthology.orchestra.run.smithi162.stdout:172.21.15.162:0/2129785584 2024-04-28T04:01:10.996874+0000 2024-04-27T04:22:17.124 INFO:teuthology.orchestra.run.smithi162.stdout:172.21.15.162:6800/1482851081 2024-04-28T04:02:16.661212+0000 2024-04-27T04:22:17.124 INFO:teuthology.orchestra.run.smithi162.stdout:172.21.15.162:6801/1480340393 2024-04-28T04:00:40.642043+0000 2024-04-27T04:22:17.124 INFO:teuthology.orchestra.run.smithi162.stdout:172.21.15.181:0/209590021 2024-04-28T04:11:32.210928+0000 2024-04-27T04:22:17.124 INFO:teuthology.orchestra.run.smithi162.stdout:172.21.15.162:0/4025365522 2024-04-28T04:00:40.642043+0000 2024-04-27T04:22:17.124 INFO:teuthology.orchestra.run.smithi162.stdout:172.21.15.181:0/3276305180 2024-04-28T04:11:32.210928+0000 2024-04-27T04:22:17.124 INFO:teuthology.orchestra.run.smithi162.stdout:172.21.15.162:6801/1482851081 2024-04-28T04:02:16.661212+0000 2024-04-27T04:22:17.124 INFO:teuthology.orchestra.run.smithi162.stdout:172.21.15.162:0/3073351439 2024-04-28T04:01:10.996874+0000 2024-04-27T04:22:17.124 INFO:teuthology.orchestra.run.smithi162.stdout:172.21.15.181:6825/658214993 2024-04-28T04:19:26.414790+0000 2024-04-27T04:22:17.124 INFO:teuthology.orchestra.run.smithi162.stdout:172.21.15.162:0/795779642 2024-04-28T04:02:16.661212+0000 2024-04-27T04:22:17.124 INFO:teuthology.orchestra.run.smithi162.stdout:172.21.15.162:0/3590827229 2024-04-28T04:01:10.996874+0000 2024-04-27T04:22:17.124 INFO:teuthology.orchestra.run.smithi162.stdout:172.21.15.181:0/2462932364 2024-04-28T04:11:32.210928+0000 2024-04-27T04:22:17.124 INFO:teuthology.orchestra.run.smithi162.stdout:172.21.15.162:6801/8067198 2024-04-28T04:01:10.996874+0000 2024-04-27T04:22:17.125 INFO:teuthology.orchestra.run.smithi162.stdout:172.21.15.162:6826/998468496 2024-04-28T04:07:08.543563+0000 2024-04-27T04:22:17.125 INFO:teuthology.orchestra.run.smithi162.stdout:172.21.15.162:0/1349712416 2024-04-28T04:10:05.654758+0000 2024-04-27T04:22:17.125 INFO:teuthology.orchestra.run.smithi162.stdout:172.21.15.162:6800/1480340393 2024-04-28T04:00:40.642043+0000 2024-04-27T04:22:17.125 INFO:teuthology.orchestra.run.smithi162.stdout:172.21.15.181:0/1936190707 2024-04-28T04:11:32.210928+0000 2024-04-27T04:22:17.125 INFO:teuthology.orchestra.run.smithi162.stdout:172.21.15.181:0/2761247390 2024-04-28T04:11:32.210928+0000 2024-04-27T04:22:17.125 INFO:teuthology.orchestra.run.smithi162.stdout:172.21.15.162:6801/4079782599 2024-04-28T04:10:05.654758+0000 2024-04-27T04:22:17.125 INFO:teuthology.orchestra.run.smithi162.stdout:172.21.15.162:0/3944767668 2024-04-28T04:10:05.654758+0000 2024-04-27T04:22:17.125 INFO:teuthology.orchestra.run.smithi162.stdout:172.21.15.162:0/2579239891 2024-04-28T04:00:40.642043+0000 2024-04-27T04:22:17.125 INFO:teuthology.orchestra.run.smithi162.stdout:172.21.15.162:6827/998468496 2024-04-28T04:07:08.543563+0000 2024-04-27T04:22:17.125 INFO:teuthology.orchestra.run.smithi162.stdout:172.21.15.162:6800/901992993 2024-04-28T04:14:17.593083+0000 2024-04-27T04:22:17.125 INFO:teuthology.orchestra.run.smithi162.stdout:172.21.15.162:0/2783280109 2024-04-28T04:10:05.654758+0000 2024-04-27T04:22:17.125 INFO:teuthology.orchestra.run.smithi162.stdout:172.21.15.162:6801/901992993 2024-04-28T04:14:17.593083+0000 2024-04-27T04:22:17.125 INFO:teuthology.orchestra.run.smithi162.stdout:172.21.15.162:0/1622403042 2024-04-28T04:14:17.593083+0000 2024-04-27T04:22:17.125 INFO:teuthology.orchestra.run.smithi162.stdout:172.21.15.162:0/2023772646 2024-04-28T04:14:17.593083+0000 2024-04-27T04:22:17.126 INFO:teuthology.orchestra.run.smithi162.stderr:listed 41 entries 2024-04-27T04:22:17.126 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:17.126+0000 7f1c8706e640 1 -- 172.21.15.162:0/1868936523 >> [v2:172.21.15.162:6800/1396866108,v1:172.21.15.162:6801/1396866108] conn(0x7f1c7006eb70 msgr2=0x7f1c70071020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:22:17.127 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:17.126+0000 7f1c8706e640 1 --2- 172.21.15.162:0/1868936523 >> [v2:172.21.15.162:6800/1396866108,v1:172.21.15.162:6801/1396866108] conn(0x7f1c7006eb70 0x7f1c70071020 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f1c6c000c00 tx=0x7f1c6c009da0 comp rx=0 tx=0).stop 2024-04-27T04:22:17.127 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:17.126+0000 7f1c8706e640 1 -- 172.21.15.162:0/1868936523 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1c80102250 msgr2=0x7f1c80112d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:22:17.127 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:17.126+0000 7f1c8706e640 1 --2- 172.21.15.162:0/1868936523 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1c80102250 0x7f1c80112d30 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f1c74002a50 tx=0x7f1c74004600 comp rx=0 tx=0).stop 2024-04-27T04:22:17.127 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:17.127+0000 7f1c8706e640 1 -- 172.21.15.162:0/1868936523 shutdown_connections 2024-04-27T04:22:17.127 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:17.127+0000 7f1c8706e640 1 --2- 172.21.15.162:0/1868936523 >> [v2:172.21.15.162:6800/1396866108,v1:172.21.15.162:6801/1396866108] conn(0x7f1c7006eb70 0x7f1c70071020 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:22:17.127 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:17.127+0000 7f1c8706e640 1 --2- 172.21.15.162:0/1868936523 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1c80102b60 0x7f1c80113270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:22:17.127 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:17.127+0000 7f1c8706e640 1 --2- 172.21.15.162:0/1868936523 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1c80102250 0x7f1c80112d30 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:22:17.127 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:17.127+0000 7f1c8706e640 1 -- 172.21.15.162:0/1868936523 >> 172.21.15.162:0/1868936523 conn(0x7f1c80070890 msgr2=0x7f1c800ffdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:22:17.127 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:17.127+0000 7f1c8706e640 1 -- 172.21.15.162:0/1868936523 shutdown_connections 2024-04-27T04:22:17.127 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-27T04:22:17.127+0000 7f1c8706e640 1 -- 172.21.15.162:0/1868936523 wait complete. 2024-04-27T04:22:17.134 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:22:16 smithi162 ceph-mon[110641]: pgmap v277: 65 pgs: 65 active+clean; 208 MiB data, 851 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:22:17.134 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:22:16 smithi162 ceph-mon[110641]: from='client.? 172.21.15.162:0/1362104705' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-27T04:22:17.135 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:22:16 smithi162 ceph-mon[110641]: from='client.? 172.21.15.162:0/3786947198' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-27T04:22:17.135 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:22:16 smithi162 ceph-mon[110641]: from='client.? 172.21.15.162:0/2134198180' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs.cephfs.meta", "var": "pg_num"}]: dispatch 2024-04-27T04:22:17.135 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:22:16 smithi162 ceph-mon[110641]: from='client.? 172.21.15.162:0/101764281' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs.cephfs.data", "var": "pg_num"}]: dispatch 2024-04-27T04:22:17.136 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi162.front.sepia.ceph.com... 2024-04-27T04:22:17.136 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-27T04:22:17.136 DEBUG:teuthology.orchestra.run.smithi162:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2024-04-27T04:22:17.160 INFO:teuthology.orchestra.run:waiting for 300 2024-04-27T04:22:17.171 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:22:16 smithi181 ceph-mon[81134]: pgmap v277: 65 pgs: 65 active+clean; 208 MiB data, 851 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:22:17.171 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:22:16 smithi181 ceph-mon[81134]: from='client.? 172.21.15.162:0/1362104705' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-27T04:22:17.171 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:22:16 smithi181 ceph-mon[81134]: from='client.? 172.21.15.162:0/3786947198' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-27T04:22:17.172 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:22:16 smithi181 ceph-mon[81134]: from='client.? 172.21.15.162:0/2134198180' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs.cephfs.meta", "var": "pg_num"}]: dispatch 2024-04-27T04:22:17.172 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:22:16 smithi181 ceph-mon[81134]: from='client.? 172.21.15.162:0/101764281' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs.cephfs.data", "var": "pg_num"}]: dispatch 2024-04-27T04:22:18.136 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:22:17 smithi162 ceph-mon[110641]: from='client.? 172.21.15.162:0/1868936523' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-04-27T04:22:18.136 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:22:17 smithi162 ceph-mon[110641]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-27T04:22:18.136 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:22:17 smithi162 ceph-mon[110641]: pgmap v278: 65 pgs: 65 active+clean; 208 MiB data, 851 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:22:18.171 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:22:17 smithi181 ceph-mon[81134]: from='client.? 172.21.15.162:0/1868936523' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-04-27T04:22:18.171 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:22:17 smithi181 ceph-mon[81134]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-27T04:22:18.171 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:22:17 smithi181 ceph-mon[81134]: pgmap v278: 65 pgs: 65 active+clean; 208 MiB data, 851 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:22:21.136 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:22:20 smithi162 ceph-mon[110641]: pgmap v279: 65 pgs: 65 active+clean; 208 MiB data, 851 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-04-27T04:22:21.171 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:22:20 smithi181 ceph-mon[81134]: pgmap v279: 65 pgs: 65 active+clean; 208 MiB data, 851 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-04-27T04:22:23.136 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:22:22 smithi162 ceph-mon[110641]: pgmap v280: 65 pgs: 65 active+clean; 208 MiB data, 851 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:22:23.171 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:22:22 smithi181 ceph-mon[81134]: pgmap v280: 65 pgs: 65 active+clean; 208 MiB data, 851 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:22:25.136 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:22:24 smithi162 ceph-mon[110641]: pgmap v281: 65 pgs: 65 active+clean; 208 MiB data, 851 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-04-27T04:22:25.171 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:22:24 smithi181 ceph-mon[81134]: pgmap v281: 65 pgs: 65 active+clean; 208 MiB data, 851 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-04-27T04:22:27.136 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:22:26 smithi162 ceph-mon[110641]: pgmap v282: 65 pgs: 65 active+clean; 208 MiB data, 851 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:22:27.171 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:22:26 smithi181 ceph-mon[81134]: pgmap v282: 65 pgs: 65 active+clean; 208 MiB data, 851 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:22:29.136 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:22:28 smithi162 ceph-mon[110641]: pgmap v283: 65 pgs: 65 active+clean; 208 MiB data, 851 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:22:29.171 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:22:28 smithi181 ceph-mon[81134]: pgmap v283: 65 pgs: 65 active+clean; 208 MiB data, 851 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:22:31.136 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:22:30 smithi162 ceph-mon[110641]: pgmap v284: 65 pgs: 65 active+clean; 208 MiB data, 851 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-04-27T04:22:31.171 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:22:30 smithi181 ceph-mon[81134]: pgmap v284: 65 pgs: 65 active+clean; 208 MiB data, 851 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-04-27T04:22:33.136 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:22:32 smithi162 ceph-mon[110641]: pgmap v285: 65 pgs: 65 active+clean; 208 MiB data, 851 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:22:33.136 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:22:32 smithi162 ceph-mon[110641]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-27T04:22:33.171 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:22:32 smithi181 ceph-mon[81134]: pgmap v285: 65 pgs: 65 active+clean; 208 MiB data, 851 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:22:33.171 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:22:32 smithi181 ceph-mon[81134]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-27T04:22:35.136 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:22:34 smithi162 ceph-mon[110641]: pgmap v286: 65 pgs: 65 active+clean; 208 MiB data, 851 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-04-27T04:22:35.171 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:22:34 smithi181 ceph-mon[81134]: pgmap v286: 65 pgs: 65 active+clean; 208 MiB data, 851 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-04-27T04:22:37.136 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:22:36 smithi162 ceph-mon[110641]: pgmap v287: 65 pgs: 65 active+clean; 208 MiB data, 851 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:22:37.171 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:22:36 smithi181 ceph-mon[81134]: pgmap v287: 65 pgs: 65 active+clean; 208 MiB data, 851 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:22:39.136 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:22:38 smithi162 ceph-mon[110641]: pgmap v288: 65 pgs: 65 active+clean; 208 MiB data, 851 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:22:39.171 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:22:38 smithi181 ceph-mon[81134]: pgmap v288: 65 pgs: 65 active+clean; 208 MiB data, 851 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:22:41.136 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:22:40 smithi162 ceph-mon[110641]: pgmap v289: 65 pgs: 65 active+clean; 208 MiB data, 851 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-04-27T04:22:41.171 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:22:40 smithi181 ceph-mon[81134]: pgmap v289: 65 pgs: 65 active+clean; 208 MiB data, 851 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-04-27T04:22:43.136 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:22:42 smithi162 ceph-mon[110641]: pgmap v290: 65 pgs: 65 active+clean; 208 MiB data, 851 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:22:43.171 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:22:42 smithi181 ceph-mon[81134]: pgmap v290: 65 pgs: 65 active+clean; 208 MiB data, 851 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:22:45.136 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:22:44 smithi162 ceph-mon[110641]: pgmap v291: 65 pgs: 65 active+clean; 208 MiB data, 851 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-04-27T04:22:45.171 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:22:44 smithi181 ceph-mon[81134]: pgmap v291: 65 pgs: 65 active+clean; 208 MiB data, 851 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-04-27T04:22:47.136 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:22:46 smithi162 ceph-mon[110641]: pgmap v292: 65 pgs: 65 active+clean; 208 MiB data, 851 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:22:47.171 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:22:46 smithi181 ceph-mon[81134]: pgmap v292: 65 pgs: 65 active+clean; 208 MiB data, 851 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:22:48.136 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:22:47 smithi162 ceph-mon[110641]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-27T04:22:48.171 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:22:47 smithi181 ceph-mon[81134]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-27T04:22:49.137 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:22:48 smithi162 ceph-mon[110641]: pgmap v293: 65 pgs: 65 active+clean; 208 MiB data, 851 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:22:49.171 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:22:48 smithi181 ceph-mon[81134]: pgmap v293: 65 pgs: 65 active+clean; 208 MiB data, 851 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:22:51.136 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:22:50 smithi162 ceph-mon[110641]: pgmap v294: 65 pgs: 65 active+clean; 208 MiB data, 851 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-04-27T04:22:51.171 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:22:50 smithi181 ceph-mon[81134]: pgmap v294: 65 pgs: 65 active+clean; 208 MiB data, 851 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-04-27T04:22:53.136 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:22:52 smithi162 ceph-mon[110641]: pgmap v295: 65 pgs: 65 active+clean; 208 MiB data, 851 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:22:53.171 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:22:52 smithi181 ceph-mon[81134]: pgmap v295: 65 pgs: 65 active+clean; 208 MiB data, 851 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:22:55.136 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:22:54 smithi162 ceph-mon[110641]: pgmap v296: 65 pgs: 65 active+clean; 208 MiB data, 851 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-04-27T04:22:55.171 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:22:54 smithi181 ceph-mon[81134]: pgmap v296: 65 pgs: 65 active+clean; 208 MiB data, 851 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-04-27T04:22:57.136 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:22:56 smithi162 ceph-mon[110641]: pgmap v297: 65 pgs: 65 active+clean; 208 MiB data, 851 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:22:57.171 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:22:56 smithi181 ceph-mon[81134]: pgmap v297: 65 pgs: 65 active+clean; 208 MiB data, 851 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:22:59.136 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:22:58 smithi162 ceph-mon[110641]: pgmap v298: 65 pgs: 65 active+clean; 208 MiB data, 851 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:22:59.171 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:22:58 smithi181 ceph-mon[81134]: pgmap v298: 65 pgs: 65 active+clean; 208 MiB data, 851 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:23:01.136 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:23:00 smithi162 ceph-mon[110641]: pgmap v299: 65 pgs: 65 active+clean; 208 MiB data, 851 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-04-27T04:23:01.171 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:23:00 smithi181 ceph-mon[81134]: pgmap v299: 65 pgs: 65 active+clean; 208 MiB data, 851 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-04-27T04:23:03.136 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:23:02 smithi162 ceph-mon[110641]: pgmap v300: 65 pgs: 65 active+clean; 208 MiB data, 851 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:23:03.136 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:23:02 smithi162 ceph-mon[110641]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-27T04:23:03.171 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:23:02 smithi181 ceph-mon[81134]: pgmap v300: 65 pgs: 65 active+clean; 208 MiB data, 851 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:23:03.171 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:23:02 smithi181 ceph-mon[81134]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-27T04:23:04.136 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:23:03 smithi162 ceph-mon[110641]: pgmap v301: 65 pgs: 65 active+clean; 208 MiB data, 851 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-04-27T04:23:04.171 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:23:03 smithi181 ceph-mon[81134]: pgmap v301: 65 pgs: 65 active+clean; 208 MiB data, 851 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-04-27T04:23:07.136 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:23:06 smithi162 ceph-mon[110641]: pgmap v302: 65 pgs: 65 active+clean; 208 MiB data, 851 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:23:07.171 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:23:06 smithi181 ceph-mon[81134]: pgmap v302: 65 pgs: 65 active+clean; 208 MiB data, 851 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:23:09.136 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:23:08 smithi162 ceph-mon[110641]: pgmap v303: 65 pgs: 65 active+clean; 208 MiB data, 851 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:23:09.171 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:23:08 smithi181 ceph-mon[81134]: pgmap v303: 65 pgs: 65 active+clean; 208 MiB data, 851 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:23:11.136 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:23:10 smithi162 ceph-mon[110641]: pgmap v304: 65 pgs: 65 active+clean; 208 MiB data, 851 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-04-27T04:23:11.171 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:23:10 smithi181 ceph-mon[81134]: pgmap v304: 65 pgs: 65 active+clean; 208 MiB data, 851 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-04-27T04:23:12.136 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:23:11 smithi162 ceph-mon[110641]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T04:23:12.136 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:23:11 smithi162 ceph-mon[110641]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' 2024-04-27T04:23:12.136 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:23:11 smithi162 ceph-mon[110641]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' 2024-04-27T04:23:12.171 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:23:11 smithi181 ceph-mon[81134]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T04:23:12.171 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:23:11 smithi181 ceph-mon[81134]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' 2024-04-27T04:23:12.171 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:23:11 smithi181 ceph-mon[81134]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' 2024-04-27T04:23:13.136 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:23:12 smithi162 ceph-mon[110641]: pgmap v305: 65 pgs: 65 active+clean; 208 MiB data, 851 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:23:13.136 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:23:12 smithi162 ceph-mon[110641]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T04:23:13.136 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:23:12 smithi162 ceph-mon[110641]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T04:23:13.136 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:23:12 smithi162 ceph-mon[110641]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' 2024-04-27T04:23:13.171 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:23:12 smithi181 ceph-mon[81134]: pgmap v305: 65 pgs: 65 active+clean; 208 MiB data, 851 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:23:13.171 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:23:12 smithi181 ceph-mon[81134]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T04:23:13.171 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:23:12 smithi181 ceph-mon[81134]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T04:23:13.171 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:23:12 smithi181 ceph-mon[81134]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' 2024-04-27T04:23:15.138 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:23:14 smithi162 ceph-mon[110641]: pgmap v306: 65 pgs: 65 active+clean; 208 MiB data, 851 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-04-27T04:23:15.171 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:23:14 smithi181 ceph-mon[81134]: pgmap v306: 65 pgs: 65 active+clean; 208 MiB data, 851 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-04-27T04:23:17.136 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:23:16 smithi162 ceph-mon[110641]: pgmap v307: 65 pgs: 65 active+clean; 208 MiB data, 851 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:23:17.171 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:23:16 smithi181 ceph-mon[81134]: pgmap v307: 65 pgs: 65 active+clean; 208 MiB data, 851 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:23:18.136 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:23:17 smithi162 ceph-mon[110641]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-27T04:23:18.171 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:23:17 smithi181 ceph-mon[81134]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-27T04:23:19.136 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:23:18 smithi162 ceph-mon[110641]: pgmap v308: 65 pgs: 65 active+clean; 208 MiB data, 851 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:23:19.171 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:23:18 smithi181 ceph-mon[81134]: pgmap v308: 65 pgs: 65 active+clean; 208 MiB data, 851 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:23:21.136 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:23:20 smithi162 ceph-mon[110641]: pgmap v309: 65 pgs: 65 active+clean; 208 MiB data, 851 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-04-27T04:23:21.171 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:23:20 smithi181 ceph-mon[81134]: pgmap v309: 65 pgs: 65 active+clean; 208 MiB data, 851 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-04-27T04:23:23.136 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:23:22 smithi162 ceph-mon[110641]: pgmap v310: 65 pgs: 65 active+clean; 208 MiB data, 851 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:23:23.171 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:23:22 smithi181 ceph-mon[81134]: pgmap v310: 65 pgs: 65 active+clean; 208 MiB data, 851 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:23:25.136 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:23:24 smithi162 ceph-mon[110641]: pgmap v311: 65 pgs: 65 active+clean; 208 MiB data, 851 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-04-27T04:23:25.171 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:23:24 smithi181 ceph-mon[81134]: pgmap v311: 65 pgs: 65 active+clean; 208 MiB data, 851 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-04-27T04:23:27.136 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:23:26 smithi162 ceph-mon[110641]: pgmap v312: 65 pgs: 65 active+clean; 208 MiB data, 851 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:23:27.171 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:23:26 smithi181 ceph-mon[81134]: pgmap v312: 65 pgs: 65 active+clean; 208 MiB data, 851 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:23:29.136 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:23:28 smithi162 ceph-mon[110641]: pgmap v313: 65 pgs: 65 active+clean; 208 MiB data, 851 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:23:29.171 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:23:28 smithi181 ceph-mon[81134]: pgmap v313: 65 pgs: 65 active+clean; 208 MiB data, 851 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:23:31.136 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:23:30 smithi162 ceph-mon[110641]: pgmap v314: 65 pgs: 65 active+clean; 208 MiB data, 851 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-04-27T04:23:31.171 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:23:30 smithi181 ceph-mon[81134]: pgmap v314: 65 pgs: 65 active+clean; 208 MiB data, 851 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-04-27T04:23:33.136 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:23:32 smithi162 ceph-mon[110641]: pgmap v315: 65 pgs: 65 active+clean; 208 MiB data, 851 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:23:33.136 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:23:32 smithi162 ceph-mon[110641]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-27T04:23:33.172 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:23:32 smithi181 ceph-mon[81134]: pgmap v315: 65 pgs: 65 active+clean; 208 MiB data, 851 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:23:33.172 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:23:32 smithi181 ceph-mon[81134]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-27T04:23:35.136 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:23:34 smithi162 ceph-mon[110641]: pgmap v316: 65 pgs: 65 active+clean; 208 MiB data, 851 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-04-27T04:23:35.171 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:23:34 smithi181 ceph-mon[81134]: pgmap v316: 65 pgs: 65 active+clean; 208 MiB data, 851 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-04-27T04:23:37.136 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:23:36 smithi162 ceph-mon[110641]: pgmap v317: 65 pgs: 65 active+clean; 208 MiB data, 851 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:23:37.171 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:23:36 smithi181 ceph-mon[81134]: pgmap v317: 65 pgs: 65 active+clean; 208 MiB data, 851 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:23:39.136 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:23:38 smithi162 ceph-mon[110641]: pgmap v318: 65 pgs: 65 active+clean; 208 MiB data, 851 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:23:39.171 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:23:38 smithi181 ceph-mon[81134]: pgmap v318: 65 pgs: 65 active+clean; 208 MiB data, 851 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:23:41.136 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:23:40 smithi162 ceph-mon[110641]: pgmap v319: 65 pgs: 65 active+clean; 208 MiB data, 851 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-04-27T04:23:41.171 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:23:40 smithi181 ceph-mon[81134]: pgmap v319: 65 pgs: 65 active+clean; 208 MiB data, 851 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-04-27T04:23:43.136 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:23:42 smithi162 ceph-mon[110641]: pgmap v320: 65 pgs: 65 active+clean; 208 MiB data, 851 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:23:43.171 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:23:42 smithi181 ceph-mon[81134]: pgmap v320: 65 pgs: 65 active+clean; 208 MiB data, 851 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:23:45.136 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:23:44 smithi162 ceph-mon[110641]: pgmap v321: 65 pgs: 65 active+clean; 208 MiB data, 851 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-04-27T04:23:45.171 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:23:44 smithi181 ceph-mon[81134]: pgmap v321: 65 pgs: 65 active+clean; 208 MiB data, 851 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-04-27T04:23:47.136 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:23:46 smithi162 ceph-mon[110641]: pgmap v322: 65 pgs: 65 active+clean; 208 MiB data, 851 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:23:47.171 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:23:46 smithi181 ceph-mon[81134]: pgmap v322: 65 pgs: 65 active+clean; 208 MiB data, 851 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:23:48.136 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:23:47 smithi162 ceph-mon[110641]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-27T04:23:48.136 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:23:47 smithi162 ceph-mon[110641]: pgmap v323: 65 pgs: 65 active+clean; 208 MiB data, 851 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:23:48.171 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:23:47 smithi181 ceph-mon[81134]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-27T04:23:48.172 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:23:47 smithi181 ceph-mon[81134]: pgmap v323: 65 pgs: 65 active+clean; 208 MiB data, 851 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:23:51.136 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:23:50 smithi162 ceph-mon[110641]: pgmap v324: 65 pgs: 65 active+clean; 208 MiB data, 851 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-04-27T04:23:51.172 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:23:50 smithi181 ceph-mon[81134]: pgmap v324: 65 pgs: 65 active+clean; 208 MiB data, 851 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-04-27T04:23:53.136 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:23:52 smithi162 ceph-mon[110641]: pgmap v325: 65 pgs: 65 active+clean; 208 MiB data, 851 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:23:53.171 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:23:52 smithi181 ceph-mon[81134]: pgmap v325: 65 pgs: 65 active+clean; 208 MiB data, 851 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:23:55.136 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:23:54 smithi162 ceph-mon[110641]: pgmap v326: 65 pgs: 65 active+clean; 208 MiB data, 851 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-04-27T04:23:55.171 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:23:54 smithi181 ceph-mon[81134]: pgmap v326: 65 pgs: 65 active+clean; 208 MiB data, 851 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-04-27T04:23:57.136 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:23:56 smithi162 ceph-mon[110641]: pgmap v327: 65 pgs: 65 active+clean; 208 MiB data, 851 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:23:57.171 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:23:56 smithi181 ceph-mon[81134]: pgmap v327: 65 pgs: 65 active+clean; 208 MiB data, 851 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:23:59.136 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:23:58 smithi162 ceph-mon[110641]: pgmap v328: 65 pgs: 65 active+clean; 208 MiB data, 851 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:23:59.171 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:23:58 smithi181 ceph-mon[81134]: pgmap v328: 65 pgs: 65 active+clean; 208 MiB data, 851 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:24:01.136 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:24:00 smithi162 ceph-mon[110641]: pgmap v329: 65 pgs: 65 active+clean; 208 MiB data, 851 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-04-27T04:24:01.172 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:24:00 smithi181 ceph-mon[81134]: pgmap v329: 65 pgs: 65 active+clean; 208 MiB data, 851 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-04-27T04:24:03.136 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:24:02 smithi162 ceph-mon[110641]: pgmap v330: 65 pgs: 65 active+clean; 208 MiB data, 851 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:24:03.136 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:24:02 smithi162 ceph-mon[110641]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-27T04:24:03.171 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:24:02 smithi181 ceph-mon[81134]: pgmap v330: 65 pgs: 65 active+clean; 208 MiB data, 851 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:24:03.171 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:24:02 smithi181 ceph-mon[81134]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-27T04:24:05.136 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:24:04 smithi162 ceph-mon[110641]: pgmap v331: 65 pgs: 65 active+clean; 208 MiB data, 851 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-04-27T04:24:05.171 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:24:04 smithi181 ceph-mon[81134]: pgmap v331: 65 pgs: 65 active+clean; 208 MiB data, 851 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-04-27T04:24:07.136 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:24:06 smithi162 ceph-mon[110641]: pgmap v332: 65 pgs: 65 active+clean; 208 MiB data, 851 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:24:07.171 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:24:06 smithi181 ceph-mon[81134]: pgmap v332: 65 pgs: 65 active+clean; 208 MiB data, 851 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:24:09.136 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:24:08 smithi162 ceph-mon[110641]: pgmap v333: 65 pgs: 65 active+clean; 208 MiB data, 851 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:24:09.172 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:24:08 smithi181 ceph-mon[81134]: pgmap v333: 65 pgs: 65 active+clean; 208 MiB data, 851 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:24:11.136 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:24:10 smithi162 ceph-mon[110641]: pgmap v334: 65 pgs: 65 active+clean; 208 MiB data, 851 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-04-27T04:24:11.172 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:24:10 smithi181 ceph-mon[81134]: pgmap v334: 65 pgs: 65 active+clean; 208 MiB data, 851 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-04-27T04:24:13.136 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:24:12 smithi162 ceph-mon[110641]: pgmap v335: 65 pgs: 65 active+clean; 208 MiB data, 851 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:24:13.136 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:24:12 smithi162 ceph-mon[110641]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T04:24:13.171 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:24:12 smithi181 ceph-mon[81134]: pgmap v335: 65 pgs: 65 active+clean; 208 MiB data, 851 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:24:13.172 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:24:12 smithi181 ceph-mon[81134]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T04:24:14.136 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:24:13 smithi162 ceph-mon[110641]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T04:24:14.137 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:24:13 smithi162 ceph-mon[110641]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T04:24:14.137 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:24:13 smithi162 ceph-mon[110641]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' 2024-04-27T04:24:14.172 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:24:13 smithi181 ceph-mon[81134]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T04:24:14.172 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:24:13 smithi181 ceph-mon[81134]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T04:24:14.172 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:24:13 smithi181 ceph-mon[81134]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' 2024-04-27T04:24:15.136 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:24:14 smithi162 ceph-mon[110641]: pgmap v336: 65 pgs: 65 active+clean; 208 MiB data, 851 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-04-27T04:24:15.172 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:24:14 smithi181 ceph-mon[81134]: pgmap v336: 65 pgs: 65 active+clean; 208 MiB data, 851 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-04-27T04:24:17.136 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:24:16 smithi162 ceph-mon[110641]: pgmap v337: 65 pgs: 65 active+clean; 208 MiB data, 851 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:24:17.172 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:24:16 smithi181 ceph-mon[81134]: pgmap v337: 65 pgs: 65 active+clean; 208 MiB data, 851 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:24:18.136 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:24:17 smithi162 ceph-mon[110641]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-27T04:24:18.172 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:24:17 smithi181 ceph-mon[81134]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-27T04:24:19.136 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:24:18 smithi162 ceph-mon[110641]: pgmap v338: 65 pgs: 65 active+clean; 208 MiB data, 851 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:24:19.172 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:24:18 smithi181 ceph-mon[81134]: pgmap v338: 65 pgs: 65 active+clean; 208 MiB data, 851 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:24:21.136 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:24:20 smithi162 ceph-mon[110641]: pgmap v339: 65 pgs: 65 active+clean; 208 MiB data, 851 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-04-27T04:24:21.172 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:24:20 smithi181 ceph-mon[81134]: pgmap v339: 65 pgs: 65 active+clean; 208 MiB data, 851 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-04-27T04:24:23.136 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:24:22 smithi162 ceph-mon[110641]: pgmap v340: 65 pgs: 65 active+clean; 208 MiB data, 851 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:24:23.172 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:24:22 smithi181 ceph-mon[81134]: pgmap v340: 65 pgs: 65 active+clean; 208 MiB data, 851 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:24:25.136 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:24:24 smithi162 ceph-mon[110641]: pgmap v341: 65 pgs: 65 active+clean; 208 MiB data, 851 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-04-27T04:24:25.172 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:24:24 smithi181 ceph-mon[81134]: pgmap v341: 65 pgs: 65 active+clean; 208 MiB data, 851 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-04-27T04:24:27.136 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:24:26 smithi162 ceph-mon[110641]: pgmap v342: 65 pgs: 65 active+clean; 208 MiB data, 851 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:24:27.172 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:24:26 smithi181 ceph-mon[81134]: pgmap v342: 65 pgs: 65 active+clean; 208 MiB data, 851 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:24:28.172 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:24:27 smithi181 ceph-mon[81134]: pgmap v343: 65 pgs: 65 active+clean; 208 MiB data, 851 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:24:28.386 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:24:27 smithi162 ceph-mon[110641]: pgmap v343: 65 pgs: 65 active+clean; 208 MiB data, 851 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:24:31.089 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:24:30 smithi162 ceph-mon[110641]: pgmap v344: 65 pgs: 65 active+clean; 208 MiB data, 851 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-04-27T04:24:31.172 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:24:30 smithi181 ceph-mon[81134]: pgmap v344: 65 pgs: 65 active+clean; 208 MiB data, 851 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-04-27T04:24:33.136 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:24:32 smithi162 ceph-mon[110641]: pgmap v345: 65 pgs: 65 active+clean; 208 MiB data, 851 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:24:33.137 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:24:32 smithi162 ceph-mon[110641]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-27T04:24:33.172 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:24:32 smithi181 ceph-mon[81134]: pgmap v345: 65 pgs: 65 active+clean; 208 MiB data, 851 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:24:33.172 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:24:32 smithi181 ceph-mon[81134]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-27T04:24:35.137 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:24:34 smithi162 ceph-mon[110641]: pgmap v346: 65 pgs: 65 active+clean; 208 MiB data, 851 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-04-27T04:24:35.173 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:24:34 smithi181 ceph-mon[81134]: pgmap v346: 65 pgs: 65 active+clean; 208 MiB data, 851 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-04-27T04:24:37.136 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:24:36 smithi162 ceph-mon[110641]: pgmap v347: 65 pgs: 65 active+clean; 208 MiB data, 851 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:24:37.172 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:24:36 smithi181 ceph-mon[81134]: pgmap v347: 65 pgs: 65 active+clean; 208 MiB data, 851 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:24:39.136 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:24:38 smithi162 ceph-mon[110641]: pgmap v348: 65 pgs: 65 active+clean; 208 MiB data, 851 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:24:39.172 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:24:38 smithi181 ceph-mon[81134]: pgmap v348: 65 pgs: 65 active+clean; 208 MiB data, 851 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:24:41.137 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:24:40 smithi162 ceph-mon[110641]: pgmap v349: 65 pgs: 65 active+clean; 208 MiB data, 851 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-04-27T04:24:41.172 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:24:40 smithi181 ceph-mon[81134]: pgmap v349: 65 pgs: 65 active+clean; 208 MiB data, 851 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-04-27T04:24:43.136 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:24:42 smithi162 ceph-mon[110641]: pgmap v350: 65 pgs: 65 active+clean; 208 MiB data, 851 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:24:43.172 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:24:42 smithi181 ceph-mon[81134]: pgmap v350: 65 pgs: 65 active+clean; 208 MiB data, 851 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:24:45.136 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:24:44 smithi162 ceph-mon[110641]: pgmap v351: 65 pgs: 65 active+clean; 208 MiB data, 851 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-04-27T04:24:45.172 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:24:44 smithi181 ceph-mon[81134]: pgmap v351: 65 pgs: 65 active+clean; 208 MiB data, 851 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-04-27T04:24:47.136 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:24:46 smithi162 ceph-mon[110641]: pgmap v352: 65 pgs: 65 active+clean; 208 MiB data, 851 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:24:47.172 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:24:46 smithi181 ceph-mon[81134]: pgmap v352: 65 pgs: 65 active+clean; 208 MiB data, 851 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:24:48.137 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:24:47 smithi162 ceph-mon[110641]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-27T04:24:48.172 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:24:47 smithi181 ceph-mon[81134]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-27T04:24:49.137 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:24:48 smithi162 ceph-mon[110641]: pgmap v353: 65 pgs: 65 active+clean; 208 MiB data, 851 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:24:49.172 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:24:48 smithi181 ceph-mon[81134]: pgmap v353: 65 pgs: 65 active+clean; 208 MiB data, 851 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:24:51.136 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:24:50 smithi162 ceph-mon[110641]: pgmap v354: 65 pgs: 65 active+clean; 208 MiB data, 851 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-04-27T04:24:51.172 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:24:50 smithi181 ceph-mon[81134]: pgmap v354: 65 pgs: 65 active+clean; 208 MiB data, 851 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-04-27T04:24:53.137 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:24:52 smithi162 ceph-mon[110641]: pgmap v355: 65 pgs: 65 active+clean; 208 MiB data, 851 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:24:53.172 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:24:52 smithi181 ceph-mon[81134]: pgmap v355: 65 pgs: 65 active+clean; 208 MiB data, 851 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:24:55.137 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:24:54 smithi162 ceph-mon[110641]: pgmap v356: 65 pgs: 65 active+clean; 208 MiB data, 851 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-04-27T04:24:55.172 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:24:54 smithi181 ceph-mon[81134]: pgmap v356: 65 pgs: 65 active+clean; 208 MiB data, 851 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-04-27T04:24:57.137 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:24:56 smithi162 ceph-mon[110641]: pgmap v357: 65 pgs: 65 active+clean; 208 MiB data, 851 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:24:57.172 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:24:56 smithi181 ceph-mon[81134]: pgmap v357: 65 pgs: 65 active+clean; 208 MiB data, 851 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:24:59.137 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:24:58 smithi162 ceph-mon[110641]: pgmap v358: 65 pgs: 65 active+clean; 208 MiB data, 851 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:24:59.172 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:24:58 smithi181 ceph-mon[81134]: pgmap v358: 65 pgs: 65 active+clean; 208 MiB data, 851 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:25:01.137 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:25:00 smithi162 ceph-mon[110641]: pgmap v359: 65 pgs: 65 active+clean; 208 MiB data, 851 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-04-27T04:25:01.172 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:25:00 smithi181 ceph-mon[81134]: pgmap v359: 65 pgs: 65 active+clean; 208 MiB data, 851 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-04-27T04:25:03.137 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:25:02 smithi162 ceph-mon[110641]: pgmap v360: 65 pgs: 65 active+clean; 208 MiB data, 851 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:25:03.137 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:25:02 smithi162 ceph-mon[110641]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-27T04:25:03.172 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:25:02 smithi181 ceph-mon[81134]: pgmap v360: 65 pgs: 65 active+clean; 208 MiB data, 851 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:25:03.172 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:25:02 smithi181 ceph-mon[81134]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-27T04:25:04.172 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:25:03 smithi181 ceph-mon[81134]: pgmap v361: 65 pgs: 65 active+clean; 208 MiB data, 851 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-04-27T04:25:04.386 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:25:03 smithi162 ceph-mon[110641]: pgmap v361: 65 pgs: 65 active+clean; 208 MiB data, 851 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-04-27T04:25:07.137 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:25:06 smithi162 ceph-mon[110641]: pgmap v362: 65 pgs: 65 active+clean; 208 MiB data, 851 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:25:07.172 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:25:06 smithi181 ceph-mon[81134]: pgmap v362: 65 pgs: 65 active+clean; 208 MiB data, 851 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:25:09.137 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:25:08 smithi162 ceph-mon[110641]: pgmap v363: 65 pgs: 65 active+clean; 208 MiB data, 851 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:25:09.172 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:25:08 smithi181 ceph-mon[81134]: pgmap v363: 65 pgs: 65 active+clean; 208 MiB data, 851 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:25:11.137 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:25:10 smithi162 ceph-mon[110641]: pgmap v364: 65 pgs: 65 active+clean; 208 MiB data, 851 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-04-27T04:25:11.172 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:25:10 smithi181 ceph-mon[81134]: pgmap v364: 65 pgs: 65 active+clean; 208 MiB data, 851 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-04-27T04:25:13.136 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:25:12 smithi162 ceph-mon[110641]: pgmap v365: 65 pgs: 65 active+clean; 208 MiB data, 851 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:25:13.172 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:25:12 smithi181 ceph-mon[81134]: pgmap v365: 65 pgs: 65 active+clean; 208 MiB data, 851 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:25:14.137 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:25:13 smithi162 ceph-mon[110641]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T04:25:14.137 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:25:13 smithi162 ceph-mon[110641]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T04:25:14.137 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:25:13 smithi162 ceph-mon[110641]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T04:25:14.137 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:25:13 smithi162 ceph-mon[110641]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' 2024-04-27T04:25:14.172 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:25:13 smithi181 ceph-mon[81134]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T04:25:14.172 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:25:13 smithi181 ceph-mon[81134]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T04:25:14.172 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:25:13 smithi181 ceph-mon[81134]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T04:25:14.172 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:25:13 smithi181 ceph-mon[81134]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' 2024-04-27T04:25:15.137 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:25:14 smithi162 ceph-mon[110641]: pgmap v366: 65 pgs: 65 active+clean; 208 MiB data, 851 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-04-27T04:25:15.172 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:25:14 smithi181 ceph-mon[81134]: pgmap v366: 65 pgs: 65 active+clean; 208 MiB data, 851 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-04-27T04:25:17.137 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:25:16 smithi162 ceph-mon[110641]: pgmap v367: 65 pgs: 65 active+clean; 208 MiB data, 851 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:25:17.172 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:25:16 smithi181 ceph-mon[81134]: pgmap v367: 65 pgs: 65 active+clean; 208 MiB data, 851 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:25:18.137 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:25:17 smithi162 ceph-mon[110641]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-27T04:25:18.172 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:25:17 smithi181 ceph-mon[81134]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-27T04:25:19.137 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:25:18 smithi162 ceph-mon[110641]: pgmap v368: 65 pgs: 65 active+clean; 208 MiB data, 851 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:25:19.172 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:25:18 smithi181 ceph-mon[81134]: pgmap v368: 65 pgs: 65 active+clean; 208 MiB data, 851 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:25:20.672 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:25:20 smithi181 ceph-mon[81134]: pgmap v369: 65 pgs: 65 active+clean; 208 MiB data, 851 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-04-27T04:25:20.887 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:25:20 smithi162 ceph-mon[110641]: pgmap v369: 65 pgs: 65 active+clean; 208 MiB data, 851 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-04-27T04:25:23.137 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:25:22 smithi162 ceph-mon[110641]: pgmap v370: 65 pgs: 65 active+clean; 208 MiB data, 851 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:25:23.172 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:25:22 smithi181 ceph-mon[81134]: pgmap v370: 65 pgs: 65 active+clean; 208 MiB data, 851 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:25:25.137 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:25:24 smithi162 ceph-mon[110641]: pgmap v371: 65 pgs: 65 active+clean; 208 MiB data, 851 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-04-27T04:25:25.172 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:25:24 smithi181 ceph-mon[81134]: pgmap v371: 65 pgs: 65 active+clean; 208 MiB data, 851 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-04-27T04:25:27.137 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:25:26 smithi162 ceph-mon[110641]: pgmap v372: 65 pgs: 65 active+clean; 208 MiB data, 851 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:25:27.172 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:25:26 smithi181 ceph-mon[81134]: pgmap v372: 65 pgs: 65 active+clean; 208 MiB data, 851 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:25:29.137 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:25:28 smithi162 ceph-mon[110641]: pgmap v373: 65 pgs: 65 active+clean; 208 MiB data, 851 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:25:29.172 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:25:28 smithi181 ceph-mon[81134]: pgmap v373: 65 pgs: 65 active+clean; 208 MiB data, 851 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:25:31.137 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:25:30 smithi162 ceph-mon[110641]: pgmap v374: 65 pgs: 65 active+clean; 208 MiB data, 851 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-04-27T04:25:31.172 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:25:30 smithi181 ceph-mon[81134]: pgmap v374: 65 pgs: 65 active+clean; 208 MiB data, 851 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-04-27T04:25:33.137 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:25:32 smithi162 ceph-mon[110641]: pgmap v375: 65 pgs: 65 active+clean; 208 MiB data, 851 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:25:33.137 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:25:32 smithi162 ceph-mon[110641]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-27T04:25:33.172 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:25:32 smithi181 ceph-mon[81134]: pgmap v375: 65 pgs: 65 active+clean; 208 MiB data, 851 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:25:33.172 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:25:32 smithi181 ceph-mon[81134]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-27T04:25:35.137 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:25:34 smithi162 ceph-mon[110641]: pgmap v376: 65 pgs: 65 active+clean; 208 MiB data, 851 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-04-27T04:25:35.172 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:25:34 smithi181 ceph-mon[81134]: pgmap v376: 65 pgs: 65 active+clean; 208 MiB data, 851 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-04-27T04:25:37.137 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:25:36 smithi162 ceph-mon[110641]: pgmap v377: 65 pgs: 65 active+clean; 208 MiB data, 851 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:25:37.172 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:25:36 smithi181 ceph-mon[81134]: pgmap v377: 65 pgs: 65 active+clean; 208 MiB data, 851 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:25:39.137 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:25:38 smithi162 ceph-mon[110641]: pgmap v378: 65 pgs: 65 active+clean; 208 MiB data, 851 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:25:39.172 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:25:38 smithi181 ceph-mon[81134]: pgmap v378: 65 pgs: 65 active+clean; 208 MiB data, 851 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:25:41.137 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:25:40 smithi162 ceph-mon[110641]: pgmap v379: 65 pgs: 65 active+clean; 208 MiB data, 851 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-04-27T04:25:41.172 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:25:40 smithi181 ceph-mon[81134]: pgmap v379: 65 pgs: 65 active+clean; 208 MiB data, 851 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-04-27T04:25:43.137 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:25:42 smithi162 ceph-mon[110641]: pgmap v380: 65 pgs: 65 active+clean; 208 MiB data, 851 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:25:43.172 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:25:42 smithi181 ceph-mon[81134]: pgmap v380: 65 pgs: 65 active+clean; 208 MiB data, 851 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:25:45.137 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:25:44 smithi162 ceph-mon[110641]: pgmap v381: 65 pgs: 65 active+clean; 208 MiB data, 851 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-04-27T04:25:45.172 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:25:44 smithi181 ceph-mon[81134]: pgmap v381: 65 pgs: 65 active+clean; 208 MiB data, 851 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-04-27T04:25:47.137 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:25:46 smithi162 ceph-mon[110641]: pgmap v382: 65 pgs: 65 active+clean; 208 MiB data, 851 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:25:47.172 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:25:46 smithi181 ceph-mon[81134]: pgmap v382: 65 pgs: 65 active+clean; 208 MiB data, 851 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:25:48.137 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:25:47 smithi162 ceph-mon[110641]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-27T04:25:48.172 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:25:47 smithi181 ceph-mon[81134]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-27T04:25:49.137 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:25:48 smithi162 ceph-mon[110641]: pgmap v383: 65 pgs: 65 active+clean; 208 MiB data, 851 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:25:49.172 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:25:48 smithi181 ceph-mon[81134]: pgmap v383: 65 pgs: 65 active+clean; 208 MiB data, 851 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:25:51.137 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:25:50 smithi162 ceph-mon[110641]: pgmap v384: 65 pgs: 65 active+clean; 208 MiB data, 851 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-04-27T04:25:51.172 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:25:50 smithi181 ceph-mon[81134]: pgmap v384: 65 pgs: 65 active+clean; 208 MiB data, 851 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-04-27T04:25:52.172 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:25:51 smithi181 ceph-mon[81134]: pgmap v385: 65 pgs: 65 active+clean; 208 MiB data, 851 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:25:52.387 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:25:51 smithi162 ceph-mon[110641]: pgmap v385: 65 pgs: 65 active+clean; 208 MiB data, 851 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:25:55.137 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:25:54 smithi162 ceph-mon[110641]: pgmap v386: 65 pgs: 65 active+clean; 208 MiB data, 851 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-04-27T04:25:55.172 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:25:54 smithi181 ceph-mon[81134]: pgmap v386: 65 pgs: 65 active+clean; 208 MiB data, 851 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-04-27T04:25:57.137 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:25:56 smithi162 ceph-mon[110641]: pgmap v387: 65 pgs: 65 active+clean; 208 MiB data, 851 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:25:57.172 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:25:56 smithi181 ceph-mon[81134]: pgmap v387: 65 pgs: 65 active+clean; 208 MiB data, 851 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:25:59.137 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:25:58 smithi162 ceph-mon[110641]: pgmap v388: 65 pgs: 65 active+clean; 208 MiB data, 851 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:25:59.172 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:25:58 smithi181 ceph-mon[81134]: pgmap v388: 65 pgs: 65 active+clean; 208 MiB data, 851 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:26:01.137 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:26:00 smithi162 ceph-mon[110641]: pgmap v389: 65 pgs: 65 active+clean; 208 MiB data, 851 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-04-27T04:26:01.172 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:26:00 smithi181 ceph-mon[81134]: pgmap v389: 65 pgs: 65 active+clean; 208 MiB data, 851 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-04-27T04:26:03.137 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:26:02 smithi162 ceph-mon[110641]: pgmap v390: 65 pgs: 65 active+clean; 208 MiB data, 851 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:26:03.137 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:26:02 smithi162 ceph-mon[110641]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-27T04:26:03.172 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:26:02 smithi181 ceph-mon[81134]: pgmap v390: 65 pgs: 65 active+clean; 208 MiB data, 851 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:26:03.172 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:26:02 smithi181 ceph-mon[81134]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-27T04:26:05.137 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:26:04 smithi162 ceph-mon[110641]: pgmap v391: 65 pgs: 65 active+clean; 208 MiB data, 851 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-04-27T04:26:05.172 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:26:04 smithi181 ceph-mon[81134]: pgmap v391: 65 pgs: 65 active+clean; 208 MiB data, 851 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-04-27T04:26:07.137 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:26:06 smithi162 ceph-mon[110641]: pgmap v392: 65 pgs: 65 active+clean; 208 MiB data, 851 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:26:07.172 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:26:06 smithi181 ceph-mon[81134]: pgmap v392: 65 pgs: 65 active+clean; 208 MiB data, 851 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:26:09.137 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:26:08 smithi162 ceph-mon[110641]: pgmap v393: 65 pgs: 65 active+clean; 208 MiB data, 851 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:26:09.172 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:26:08 smithi181 ceph-mon[81134]: pgmap v393: 65 pgs: 65 active+clean; 208 MiB data, 851 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:26:11.137 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:26:10 smithi162 ceph-mon[110641]: pgmap v394: 65 pgs: 65 active+clean; 208 MiB data, 851 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-04-27T04:26:11.172 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:26:10 smithi181 ceph-mon[81134]: pgmap v394: 65 pgs: 65 active+clean; 208 MiB data, 851 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-04-27T04:26:13.137 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:26:12 smithi162 ceph-mon[110641]: pgmap v395: 65 pgs: 65 active+clean; 208 MiB data, 851 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:26:13.172 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:26:12 smithi181 ceph-mon[81134]: pgmap v395: 65 pgs: 65 active+clean; 208 MiB data, 851 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:26:14.137 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:26:13 smithi162 ceph-mon[110641]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T04:26:14.172 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:26:13 smithi181 ceph-mon[81134]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T04:26:15.137 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:26:14 smithi162 ceph-mon[110641]: pgmap v396: 65 pgs: 65 active+clean; 208 MiB data, 851 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-04-27T04:26:15.137 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:26:14 smithi162 ceph-mon[110641]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T04:26:15.137 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:26:14 smithi162 ceph-mon[110641]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T04:26:15.137 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:26:14 smithi162 ceph-mon[110641]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' 2024-04-27T04:26:15.172 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:26:14 smithi181 ceph-mon[81134]: pgmap v396: 65 pgs: 65 active+clean; 208 MiB data, 851 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-04-27T04:26:15.173 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:26:14 smithi181 ceph-mon[81134]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T04:26:15.173 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:26:14 smithi181 ceph-mon[81134]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T04:26:15.173 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:26:14 smithi181 ceph-mon[81134]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' 2024-04-27T04:26:17.137 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:26:16 smithi162 ceph-mon[110641]: pgmap v397: 65 pgs: 65 active+clean; 208 MiB data, 851 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:26:17.172 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:26:16 smithi181 ceph-mon[81134]: pgmap v397: 65 pgs: 65 active+clean; 208 MiB data, 851 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:26:18.137 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:26:17 smithi162 ceph-mon[110641]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-27T04:26:18.172 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:26:17 smithi181 ceph-mon[81134]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-27T04:26:19.137 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:26:18 smithi162 ceph-mon[110641]: pgmap v398: 65 pgs: 65 active+clean; 208 MiB data, 851 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:26:19.173 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:26:18 smithi181 ceph-mon[81134]: pgmap v398: 65 pgs: 65 active+clean; 208 MiB data, 851 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:26:20.137 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:26:19 smithi162 ceph-mon[110641]: pgmap v399: 65 pgs: 65 active+clean; 208 MiB data, 851 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-04-27T04:26:20.172 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:26:19 smithi181 ceph-mon[81134]: pgmap v399: 65 pgs: 65 active+clean; 208 MiB data, 851 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-04-27T04:26:23.137 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:26:22 smithi162 ceph-mon[110641]: pgmap v400: 65 pgs: 65 active+clean; 208 MiB data, 851 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:26:23.172 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:26:22 smithi181 ceph-mon[81134]: pgmap v400: 65 pgs: 65 active+clean; 208 MiB data, 851 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:26:25.137 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:26:24 smithi162 ceph-mon[110641]: pgmap v401: 65 pgs: 65 active+clean; 208 MiB data, 851 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-04-27T04:26:25.172 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:26:24 smithi181 ceph-mon[81134]: pgmap v401: 65 pgs: 65 active+clean; 208 MiB data, 851 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-04-27T04:26:27.137 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:26:26 smithi162 ceph-mon[110641]: pgmap v402: 65 pgs: 65 active+clean; 208 MiB data, 851 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:26:27.172 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:26:26 smithi181 ceph-mon[81134]: pgmap v402: 65 pgs: 65 active+clean; 208 MiB data, 851 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:26:29.137 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:26:28 smithi162 ceph-mon[110641]: pgmap v403: 65 pgs: 65 active+clean; 208 MiB data, 851 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:26:29.172 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:26:28 smithi181 ceph-mon[81134]: pgmap v403: 65 pgs: 65 active+clean; 208 MiB data, 851 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:26:31.137 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:26:30 smithi162 ceph-mon[110641]: pgmap v404: 65 pgs: 65 active+clean; 208 MiB data, 851 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-04-27T04:26:31.173 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:26:30 smithi181 ceph-mon[81134]: pgmap v404: 65 pgs: 65 active+clean; 208 MiB data, 851 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-04-27T04:26:33.137 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:26:32 smithi162 ceph-mon[110641]: pgmap v405: 65 pgs: 65 active+clean; 208 MiB data, 851 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:26:33.137 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:26:32 smithi162 ceph-mon[110641]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-27T04:26:33.172 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:26:32 smithi181 ceph-mon[81134]: pgmap v405: 65 pgs: 65 active+clean; 208 MiB data, 851 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:26:33.173 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:26:32 smithi181 ceph-mon[81134]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-27T04:26:35.137 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:26:34 smithi162 ceph-mon[110641]: pgmap v406: 65 pgs: 65 active+clean; 208 MiB data, 851 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:26:35.173 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:26:34 smithi181 ceph-mon[81134]: pgmap v406: 65 pgs: 65 active+clean; 208 MiB data, 851 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:26:37.137 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:26:36 smithi162 ceph-mon[110641]: pgmap v407: 65 pgs: 65 active+clean; 208 MiB data, 851 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-04-27T04:26:37.172 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:26:36 smithi181 ceph-mon[81134]: pgmap v407: 65 pgs: 65 active+clean; 208 MiB data, 851 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-04-27T04:26:39.137 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:26:38 smithi162 ceph-mon[110641]: pgmap v408: 65 pgs: 65 active+clean; 208 MiB data, 851 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:26:39.172 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:26:38 smithi181 ceph-mon[81134]: pgmap v408: 65 pgs: 65 active+clean; 208 MiB data, 851 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:26:41.137 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:26:40 smithi162 ceph-mon[110641]: pgmap v409: 65 pgs: 65 active+clean; 208 MiB data, 851 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:26:41.173 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:26:40 smithi181 ceph-mon[81134]: pgmap v409: 65 pgs: 65 active+clean; 208 MiB data, 851 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:26:43.137 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:26:42 smithi162 ceph-mon[110641]: pgmap v410: 65 pgs: 65 active+clean; 208 MiB data, 851 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-04-27T04:26:43.173 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:26:42 smithi181 ceph-mon[81134]: pgmap v410: 65 pgs: 65 active+clean; 208 MiB data, 851 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-04-27T04:26:45.137 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:26:44 smithi162 ceph-mon[110641]: pgmap v411: 65 pgs: 65 active+clean; 208 MiB data, 851 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:26:45.173 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:26:44 smithi181 ceph-mon[81134]: pgmap v411: 65 pgs: 65 active+clean; 208 MiB data, 851 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:26:47.137 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:26:46 smithi162 ceph-mon[110641]: pgmap v412: 65 pgs: 65 active+clean; 208 MiB data, 851 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:26:47.173 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:26:46 smithi181 ceph-mon[81134]: pgmap v412: 65 pgs: 65 active+clean; 208 MiB data, 851 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:26:48.137 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:26:47 smithi162 ceph-mon[110641]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-27T04:26:48.173 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:26:47 smithi181 ceph-mon[81134]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-27T04:26:49.137 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:26:48 smithi162 ceph-mon[110641]: pgmap v413: 65 pgs: 65 active+clean; 208 MiB data, 851 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:26:49.173 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:26:48 smithi181 ceph-mon[81134]: pgmap v413: 65 pgs: 65 active+clean; 208 MiB data, 851 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:26:50.137 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:26:49 smithi162 ceph-mon[110641]: pgmap v414: 65 pgs: 65 active+clean; 208 MiB data, 851 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-04-27T04:26:50.172 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:26:49 smithi181 ceph-mon[81134]: pgmap v414: 65 pgs: 65 active+clean; 208 MiB data, 851 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-04-27T04:26:53.137 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:26:52 smithi162 ceph-mon[110641]: pgmap v415: 65 pgs: 65 active+clean; 208 MiB data, 851 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:26:53.173 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:26:52 smithi181 ceph-mon[81134]: pgmap v415: 65 pgs: 65 active+clean; 208 MiB data, 851 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:26:55.137 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:26:54 smithi162 ceph-mon[110641]: pgmap v416: 65 pgs: 65 active+clean; 208 MiB data, 851 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-04-27T04:26:55.173 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:26:54 smithi181 ceph-mon[81134]: pgmap v416: 65 pgs: 65 active+clean; 208 MiB data, 851 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-04-27T04:26:57.137 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:26:56 smithi162 ceph-mon[110641]: pgmap v417: 65 pgs: 65 active+clean; 208 MiB data, 851 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:26:57.173 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:26:56 smithi181 ceph-mon[81134]: pgmap v417: 65 pgs: 65 active+clean; 208 MiB data, 851 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:26:59.138 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:26:58 smithi162 ceph-mon[110641]: pgmap v418: 65 pgs: 65 active+clean; 208 MiB data, 851 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:26:59.173 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:26:58 smithi181 ceph-mon[81134]: pgmap v418: 65 pgs: 65 active+clean; 208 MiB data, 851 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:27:01.137 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:27:00 smithi162 ceph-mon[110641]: pgmap v419: 65 pgs: 65 active+clean; 208 MiB data, 851 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-04-27T04:27:01.173 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:27:00 smithi181 ceph-mon[81134]: pgmap v419: 65 pgs: 65 active+clean; 208 MiB data, 851 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-04-27T04:27:03.137 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:27:02 smithi162 ceph-mon[110641]: pgmap v420: 65 pgs: 65 active+clean; 208 MiB data, 851 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:27:03.138 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:27:02 smithi162 ceph-mon[110641]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-27T04:27:03.173 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:27:02 smithi181 ceph-mon[81134]: pgmap v420: 65 pgs: 65 active+clean; 208 MiB data, 851 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:27:03.173 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:27:02 smithi181 ceph-mon[81134]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-27T04:27:05.137 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:27:04 smithi162 ceph-mon[110641]: pgmap v421: 65 pgs: 65 active+clean; 208 MiB data, 851 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-04-27T04:27:05.173 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:27:04 smithi181 ceph-mon[81134]: pgmap v421: 65 pgs: 65 active+clean; 208 MiB data, 851 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-04-27T04:27:07.137 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:27:06 smithi162 ceph-mon[110641]: pgmap v422: 65 pgs: 65 active+clean; 208 MiB data, 851 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:27:07.173 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:27:06 smithi181 ceph-mon[81134]: pgmap v422: 65 pgs: 65 active+clean; 208 MiB data, 851 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:27:09.137 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:27:08 smithi162 ceph-mon[110641]: pgmap v423: 65 pgs: 65 active+clean; 208 MiB data, 851 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:27:09.173 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:27:08 smithi181 ceph-mon[81134]: pgmap v423: 65 pgs: 65 active+clean; 208 MiB data, 851 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:27:11.137 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:27:10 smithi162 ceph-mon[110641]: pgmap v424: 65 pgs: 65 active+clean; 208 MiB data, 851 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-04-27T04:27:11.173 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:27:10 smithi181 ceph-mon[81134]: pgmap v424: 65 pgs: 65 active+clean; 208 MiB data, 851 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-04-27T04:27:13.137 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:27:12 smithi162 ceph-mon[110641]: pgmap v425: 65 pgs: 65 active+clean; 208 MiB data, 851 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:27:13.173 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:27:12 smithi181 ceph-mon[81134]: pgmap v425: 65 pgs: 65 active+clean; 208 MiB data, 851 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:27:15.137 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:27:14 smithi162 ceph-mon[110641]: pgmap v426: 65 pgs: 65 active+clean; 208 MiB data, 851 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-04-27T04:27:15.137 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:27:14 smithi162 ceph-mon[110641]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T04:27:15.173 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:27:14 smithi181 ceph-mon[81134]: pgmap v426: 65 pgs: 65 active+clean; 208 MiB data, 851 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-04-27T04:27:15.173 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:27:14 smithi181 ceph-mon[81134]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T04:27:16.138 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:27:15 smithi162 ceph-mon[110641]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T04:27:16.138 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:27:15 smithi162 ceph-mon[110641]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T04:27:16.138 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:27:15 smithi162 ceph-mon[110641]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' 2024-04-27T04:27:16.173 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:27:15 smithi181 ceph-mon[81134]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T04:27:16.173 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:27:15 smithi181 ceph-mon[81134]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T04:27:16.173 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:27:15 smithi181 ceph-mon[81134]: from='mgr.34104 172.21.15.162:0/2794607138' entity='mgr.smithi162.oennfn' 2024-04-27T04:27:16.224 ERROR:tasks.cephfs.fuse_mount:process failed to terminate after unmount. This probably indicates a bug within ceph-fuse. 2024-04-27T04:27:16.224 ERROR:teuthology.run_tasks:Manager failed: ceph-fuse Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/run_tasks.py", line 154, in run_tasks suppress = manager.__exit__(*exc_info) File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ next(self.gen) File "/home/teuthworker/src/git.ceph.com_ceph_3067e9f133f954b642fdf21114a79edb58590948/qa/tasks/ceph_fuse.py", line 183, in task mount.umount_wait() File "/home/teuthworker/src/git.ceph.com_ceph_3067e9f133f954b642fdf21114a79edb58590948/qa/tasks/cephfs/fuse_mount.py", line 403, in umount_wait run.wait([self.fuse_daemon], timeout) File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/orchestra/run.py", line 473, in wait check_time() File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/contextutil.py", line 134, in __call__ raise MaxWhileTries(error_msg) teuthology.exceptions.MaxWhileTries: reached maximum tries (51) after waiting for 300 seconds 2024-04-27T04:27:16.226 DEBUG:teuthology.run_tasks:Unwinding manager cephadm 2024-04-27T04:27:16.238 INFO:tasks.cephadm:Teardown begin 2024-04-27T04:27:16.238 ERROR:teuthology.contextutil:Saw exception from nested tasks Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/contextutil.py", line 32, in nested yield vars File "/home/teuthworker/src/git.ceph.com_ceph_3067e9f133f954b642fdf21114a79edb58590948/qa/tasks/cephadm.py", line 2246, in task yield File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/run_tasks.py", line 154, in run_tasks suppress = manager.__exit__(*exc_info) File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ next(self.gen) File "/home/teuthworker/src/git.ceph.com_ceph_3067e9f133f954b642fdf21114a79edb58590948/qa/tasks/ceph_fuse.py", line 183, in task mount.umount_wait() File "/home/teuthworker/src/git.ceph.com_ceph_3067e9f133f954b642fdf21114a79edb58590948/qa/tasks/cephfs/fuse_mount.py", line 403, in umount_wait run.wait([self.fuse_daemon], timeout) File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/orchestra/run.py", line 473, in wait check_time() File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/contextutil.py", line 134, in __call__ raise MaxWhileTries(error_msg) teuthology.exceptions.MaxWhileTries: reached maximum tries (51) after waiting for 300 seconds 2024-04-27T04:27:16.239 DEBUG:teuthology.orchestra.run.smithi162:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-04-27T04:27:16.268 DEBUG:teuthology.orchestra.run.smithi181:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-04-27T04:27:16.300 INFO:tasks.cephadm:Cleaning up testdir ceph.* files... 2024-04-27T04:27:16.301 DEBUG:teuthology.orchestra.run.smithi162:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-04-27T04:27:16.325 DEBUG:teuthology.orchestra.run.smithi181:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-04-27T04:27:16.359 INFO:tasks.cephadm:Stopping all daemons... 2024-04-27T04:27:16.359 INFO:tasks.cephadm.mon.smithi162:Stopping mon.smithi162... 2024-04-27T04:27:16.359 DEBUG:teuthology.orchestra.run.smithi162:> sudo systemctl stop ceph-7f9750d6-044a-11ef-bc93-c7b262605968@mon.smithi162 2024-04-27T04:27:16.637 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:27:16 smithi162 systemd[1]: Stopping Ceph mon.smithi162 for 7f9750d6-044a-11ef-bc93-c7b262605968... 2024-04-27T04:27:17.110 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:27:16 smithi162 ceph-7f9750d6-044a-11ef-bc93-c7b262605968-mon-smithi162[110637]: 2024-04-27T04:27:16.651+0000 7f2a7e076700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.smithi162 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-04-27T04:27:17.110 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:27:16 smithi162 ceph-7f9750d6-044a-11ef-bc93-c7b262605968-mon-smithi162[110637]: 2024-04-27T04:27:16.652+0000 7f2a7e076700 -1 mon.smithi162@0(leader) e3 *** Got Signal Terminated *** 2024-04-27T04:27:17.110 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:27:16 smithi162 podman[151495]: 2024-04-27 04:27:16.856209045 +0000 UTC m=+0.308609880 container died 375adaf4468f7520330d3d23debb783bfbda1c09ddf0808a70b1c616606c5800 (image=quay.ceph.io/ceph-ci/ceph@sha256:90a3a9fa9ceb8977f6dad9ccd36fe46018d07e2118199820523a6b35301d9c16, name=ceph-7f9750d6-044a-11ef-bc93-c7b262605968-mon-smithi162, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, RELEASE=main-4e213a5, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.build-date=20240424, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-27T04:27:17.388 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:27:17 smithi162 podman[151495]: 2024-04-27 04:27:17.31115667 +0000 UTC m=+0.763557545 container cleanup 375adaf4468f7520330d3d23debb783bfbda1c09ddf0808a70b1c616606c5800 (image=quay.ceph.io/ceph-ci/ceph@sha256:90a3a9fa9ceb8977f6dad9ccd36fe46018d07e2118199820523a6b35301d9c16, name=ceph-7f9750d6-044a-11ef-bc93-c7b262605968-mon-smithi162, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, io.buildah.version=1.33.5, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240424, RELEASE=main-4e213a5, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-27T04:27:17.388 INFO:journalctl@ceph.mon.smithi162.smithi162.stdout:Apr 27 04:27:17 smithi162 bash[151495]: ceph-7f9750d6-044a-11ef-bc93-c7b262605968-mon-smithi162 2024-04-27T04:27:17.733 DEBUG:teuthology.orchestra.run.smithi162:> sudo pkill -f 'journalctl -f -n 0 -u ceph-7f9750d6-044a-11ef-bc93-c7b262605968@mon.smithi162.service' 2024-04-27T04:27:17.773 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-27T04:27:17.774 INFO:tasks.cephadm.mon.smithi162:Stopped mon.smithi162 2024-04-27T04:27:17.774 INFO:tasks.cephadm.mon.smithi181:Stopping mon.smithi181... 2024-04-27T04:27:17.774 DEBUG:teuthology.orchestra.run.smithi181:> sudo systemctl stop ceph-7f9750d6-044a-11ef-bc93-c7b262605968@mon.smithi181 2024-04-27T04:27:18.110 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:27:17 smithi181 systemd[1]: Stopping Ceph mon.smithi181 for 7f9750d6-044a-11ef-bc93-c7b262605968... 2024-04-27T04:27:18.423 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:27:18 smithi181 ceph-7f9750d6-044a-11ef-bc93-c7b262605968-mon-smithi181[81130]: 2024-04-27T04:27:18.109+0000 7fce0dd79700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.smithi181 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-04-27T04:27:18.424 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:27:18 smithi181 ceph-7f9750d6-044a-11ef-bc93-c7b262605968-mon-smithi181[81130]: 2024-04-27T04:27:18.109+0000 7fce0dd79700 -1 mon.smithi181@1(peon) e3 *** Got Signal Terminated *** 2024-04-27T04:27:18.424 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:27:18 smithi181 podman[114819]: 2024-04-27 04:27:18.356140945 +0000 UTC m=+0.385343663 container died 77b71f6d522fbf9721875667252bef513a31e55d8e307eeda8c4a035133b1815 (image=quay.ceph.io/ceph-ci/ceph@sha256:90a3a9fa9ceb8977f6dad9ccd36fe46018d07e2118199820523a6b35301d9c16, name=ceph-7f9750d6-044a-11ef-bc93-c7b262605968-mon-smithi181, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=main-4e213a5, CEPH_POINT_RELEASE=, ceph=True, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240424) 2024-04-27T04:27:18.923 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:27:18 smithi181 podman[114819]: 2024-04-27 04:27:18.869930819 +0000 UTC m=+0.899133535 container cleanup 77b71f6d522fbf9721875667252bef513a31e55d8e307eeda8c4a035133b1815 (image=quay.ceph.io/ceph-ci/ceph@sha256:90a3a9fa9ceb8977f6dad9ccd36fe46018d07e2118199820523a6b35301d9c16, name=ceph-7f9750d6-044a-11ef-bc93-c7b262605968-mon-smithi181, ceph=True, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_CLEAN=True, RELEASE=main-4e213a5, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240424) 2024-04-27T04:27:18.923 INFO:journalctl@ceph.mon.smithi181.smithi181.stdout:Apr 27 04:27:18 smithi181 bash[114819]: ceph-7f9750d6-044a-11ef-bc93-c7b262605968-mon-smithi181 2024-04-27T04:27:19.243 DEBUG:teuthology.orchestra.run.smithi181:> sudo pkill -f 'journalctl -f -n 0 -u ceph-7f9750d6-044a-11ef-bc93-c7b262605968@mon.smithi181.service' 2024-04-27T04:27:19.284 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-27T04:27:19.284 INFO:tasks.cephadm.mon.smithi181:Stopped mon.smithi181 2024-04-27T04:27:19.284 INFO:tasks.cephadm.osd.0:Stopping osd.0... 2024-04-27T04:27:19.284 DEBUG:teuthology.orchestra.run.smithi162:> sudo systemctl stop ceph-7f9750d6-044a-11ef-bc93-c7b262605968@osd.0 2024-04-27T04:27:19.582 INFO:journalctl@ceph.osd.0.smithi162.stdout:Apr 27 04:27:19 smithi162 systemd[1]: Stopping Ceph osd.0 for 7f9750d6-044a-11ef-bc93-c7b262605968... 2024-04-27T04:27:19.887 INFO:journalctl@ceph.osd.0.smithi162.stdout:Apr 27 04:27:19 smithi162 ceph-7f9750d6-044a-11ef-bc93-c7b262605968-osd-0[121588]: 2024-04-27T04:27:19.582+0000 7f198c876700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.0 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-27T04:27:19.888 INFO:journalctl@ceph.osd.0.smithi162.stdout:Apr 27 04:27:19 smithi162 ceph-7f9750d6-044a-11ef-bc93-c7b262605968-osd-0[121588]: 2024-04-27T04:27:19.582+0000 7f198c876700 -1 osd.0 84 *** Got signal Terminated *** 2024-04-27T04:27:19.888 INFO:journalctl@ceph.osd.0.smithi162.stdout:Apr 27 04:27:19 smithi162 ceph-7f9750d6-044a-11ef-bc93-c7b262605968-osd-0[121588]: 2024-04-27T04:27:19.582+0000 7f198c876700 -1 osd.0 84 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-27T04:27:25.388 INFO:journalctl@ceph.osd.0.smithi162.stdout:Apr 27 04:27:25 smithi162 podman[151628]: 2024-04-27 04:27:25.107274361 +0000 UTC m=+5.634184937 container died 2744aff3cd885623e08b77f5e5c0477e8489290d8922e4ccccd400f92ba53294 (image=quay.ceph.io/ceph-ci/ceph@sha256:90a3a9fa9ceb8977f6dad9ccd36fe46018d07e2118199820523a6b35301d9c16, name=ceph-7f9750d6-044a-11ef-bc93-c7b262605968-osd-0, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=main-4e213a5, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240424, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux ) 2024-04-27T04:27:25.887 INFO:journalctl@ceph.osd.0.smithi162.stdout:Apr 27 04:27:25 smithi162 podman[151628]: 2024-04-27 04:27:25.423310872 +0000 UTC m=+5.950221456 container cleanup 2744aff3cd885623e08b77f5e5c0477e8489290d8922e4ccccd400f92ba53294 (image=quay.ceph.io/ceph-ci/ceph@sha256:90a3a9fa9ceb8977f6dad9ccd36fe46018d07e2118199820523a6b35301d9c16, name=ceph-7f9750d6-044a-11ef-bc93-c7b262605968-osd-0, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.build-date=20240424, RELEASE=main-4e213a5, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0) 2024-04-27T04:27:25.888 INFO:journalctl@ceph.osd.0.smithi162.stdout:Apr 27 04:27:25 smithi162 bash[151628]: ceph-7f9750d6-044a-11ef-bc93-c7b262605968-osd-0 2024-04-27T04:27:25.888 INFO:journalctl@ceph.osd.0.smithi162.stdout:Apr 27 04:27:25 smithi162 podman[151646]: 2024-04-27 04:27:25.658259096 +0000 UTC m=+0.544804844 container remove 2744aff3cd885623e08b77f5e5c0477e8489290d8922e4ccccd400f92ba53294 (image=quay.ceph.io/ceph-ci/ceph@sha256:90a3a9fa9ceb8977f6dad9ccd36fe46018d07e2118199820523a6b35301d9c16, name=ceph-7f9750d6-044a-11ef-bc93-c7b262605968-osd-0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240424, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=main-4e213a5, io.buildah.version=1.33.5) 2024-04-27T04:27:26.386 INFO:journalctl@ceph.osd.0.smithi162.stdout:Apr 27 04:27:26 smithi162 podman[151738]: 2024-04-27 04:27:26.003921419 +0000 UTC m=+0.018729341 image pull 67e414ff4bc41092cb31c84c98a9563a32813d02583028679c46b40362afeb3e quay.ceph.io/ceph-ci/ceph@sha256:90a3a9fa9ceb8977f6dad9ccd36fe46018d07e2118199820523a6b35301d9c16 2024-04-27T04:27:26.386 INFO:journalctl@ceph.osd.0.smithi162.stdout:Apr 27 04:27:26 smithi162 podman[151738]: 2024-04-27 04:27:26.136475971 +0000 UTC m=+0.151283879 container create 2dee7a834e73ae7ff049e44b006fff2c31f32431034ff2dace6152480ccf2863 (image=quay.ceph.io/ceph-ci/ceph@sha256:90a3a9fa9ceb8977f6dad9ccd36fe46018d07e2118199820523a6b35301d9c16, name=ceph-7f9750d6-044a-11ef-bc93-c7b262605968-osd-0-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240424, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=main-4e213a5, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, ceph=True) 2024-04-27T04:27:26.387 INFO:journalctl@ceph.osd.0.smithi162.stdout:Apr 27 04:27:26 smithi162 podman[151738]: 2024-04-27 04:27:26.387645717 +0000 UTC m=+0.402453622 container init 2dee7a834e73ae7ff049e44b006fff2c31f32431034ff2dace6152480ccf2863 (image=quay.ceph.io/ceph-ci/ceph@sha256:90a3a9fa9ceb8977f6dad9ccd36fe46018d07e2118199820523a6b35301d9c16, name=ceph-7f9750d6-044a-11ef-bc93-c7b262605968-osd-0-deactivate, ceph=True, org.label-schema.build-date=20240424, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_CLEAN=True, RELEASE=main-4e213a5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD) 2024-04-27T04:27:26.638 INFO:journalctl@ceph.osd.0.smithi162.stdout:Apr 27 04:27:26 smithi162 podman[151738]: 2024-04-27 04:27:26.391102568 +0000 UTC m=+0.405910482 container start 2dee7a834e73ae7ff049e44b006fff2c31f32431034ff2dace6152480ccf2863 (image=quay.ceph.io/ceph-ci/ceph@sha256:90a3a9fa9ceb8977f6dad9ccd36fe46018d07e2118199820523a6b35301d9c16, name=ceph-7f9750d6-044a-11ef-bc93-c7b262605968-osd-0-deactivate, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.license=GPLv2, ceph=True, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=main-4e213a5, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240424, CEPH_POINT_RELEASE=) 2024-04-27T04:27:26.638 INFO:journalctl@ceph.osd.0.smithi162.stdout:Apr 27 04:27:26 smithi162 podman[151738]: 2024-04-27 04:27:26.487596044 +0000 UTC m=+0.502403953 container attach 2dee7a834e73ae7ff049e44b006fff2c31f32431034ff2dace6152480ccf2863 (image=quay.ceph.io/ceph-ci/ceph@sha256:90a3a9fa9ceb8977f6dad9ccd36fe46018d07e2118199820523a6b35301d9c16, name=ceph-7f9750d6-044a-11ef-bc93-c7b262605968-osd-0-deactivate, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=main-4e213a5, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.build-date=20240424, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS) 2024-04-27T04:27:26.998 INFO:journalctl@ceph.osd.0.smithi162.stdout:Apr 27 04:27:26 smithi162 podman[151738]: 2024-04-27 04:27:26.69851344 +0000 UTC m=+0.713321340 container died 2dee7a834e73ae7ff049e44b006fff2c31f32431034ff2dace6152480ccf2863 (image=quay.ceph.io/ceph-ci/ceph@sha256:90a3a9fa9ceb8977f6dad9ccd36fe46018d07e2118199820523a6b35301d9c16, name=ceph-7f9750d6-044a-11ef-bc93-c7b262605968-osd-0-deactivate, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, RELEASE=main-4e213a5, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240424, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-27T04:27:27.347 DEBUG:teuthology.orchestra.run.smithi162:> sudo pkill -f 'journalctl -f -n 0 -u ceph-7f9750d6-044a-11ef-bc93-c7b262605968@osd.0.service' 2024-04-27T04:27:27.386 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-27T04:27:27.387 INFO:tasks.cephadm.osd.0:Stopped osd.0 2024-04-27T04:27:27.387 INFO:tasks.cephadm.osd.1:Stopping osd.1... 2024-04-27T04:27:27.387 DEBUG:teuthology.orchestra.run.smithi162:> sudo systemctl stop ceph-7f9750d6-044a-11ef-bc93-c7b262605968@osd.1 2024-04-27T04:27:27.608 INFO:journalctl@ceph.osd.1.smithi162.stdout:Apr 27 04:27:27 smithi162 systemd[1]: Stopping Ceph osd.1 for 7f9750d6-044a-11ef-bc93-c7b262605968... 2024-04-27T04:27:27.887 INFO:journalctl@ceph.osd.1.smithi162.stdout:Apr 27 04:27:27 smithi162 ceph-7f9750d6-044a-11ef-bc93-c7b262605968-osd-1[127119]: 2024-04-27T04:27:27.727+0000 7f2f9ba0f700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.1 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-27T04:27:27.888 INFO:journalctl@ceph.osd.1.smithi162.stdout:Apr 27 04:27:27 smithi162 ceph-7f9750d6-044a-11ef-bc93-c7b262605968-osd-1[127119]: 2024-04-27T04:27:27.727+0000 7f2f9ba0f700 -1 osd.1 84 *** Got signal Terminated *** 2024-04-27T04:27:27.888 INFO:journalctl@ceph.osd.1.smithi162.stdout:Apr 27 04:27:27 smithi162 ceph-7f9750d6-044a-11ef-bc93-c7b262605968-osd-1[127119]: 2024-04-27T04:27:27.727+0000 7f2f9ba0f700 -1 osd.1 84 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-27T04:27:33.495 INFO:journalctl@ceph.osd.1.smithi162.stdout:Apr 27 04:27:33 smithi162 podman[151846]: 2024-04-27 04:27:33.196550526 +0000 UTC m=+5.587048072 container died 668e16faad6233fa9895f18d0284aad8a73737d64d405561bb0da15b674ddbe7 (image=quay.ceph.io/ceph-ci/ceph@sha256:90a3a9fa9ceb8977f6dad9ccd36fe46018d07e2118199820523a6b35301d9c16, name=ceph-7f9750d6-044a-11ef-bc93-c7b262605968-osd-1, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240424, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=main-4e213a5, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-27T04:27:33.803 INFO:journalctl@ceph.osd.1.smithi162.stdout:Apr 27 04:27:33 smithi162 podman[151846]: 2024-04-27 04:27:33.603584904 +0000 UTC m=+5.994082459 container cleanup 668e16faad6233fa9895f18d0284aad8a73737d64d405561bb0da15b674ddbe7 (image=quay.ceph.io/ceph-ci/ceph@sha256:90a3a9fa9ceb8977f6dad9ccd36fe46018d07e2118199820523a6b35301d9c16, name=ceph-7f9750d6-044a-11ef-bc93-c7b262605968-osd-1, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=main-4e213a5, ceph=True, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240424, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=) 2024-04-27T04:27:33.804 INFO:journalctl@ceph.osd.1.smithi162.stdout:Apr 27 04:27:33 smithi162 bash[151846]: ceph-7f9750d6-044a-11ef-bc93-c7b262605968-osd-1 2024-04-27T04:27:34.138 INFO:journalctl@ceph.osd.1.smithi162.stdout:Apr 27 04:27:33 smithi162 podman[151874]: 2024-04-27 04:27:33.80495856 +0000 UTC m=+0.606542362 container remove 668e16faad6233fa9895f18d0284aad8a73737d64d405561bb0da15b674ddbe7 (image=quay.ceph.io/ceph-ci/ceph@sha256:90a3a9fa9ceb8977f6dad9ccd36fe46018d07e2118199820523a6b35301d9c16, name=ceph-7f9750d6-044a-11ef-bc93-c7b262605968-osd-1, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=main-4e213a5, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240424, GIT_CLEAN=True) 2024-04-27T04:27:34.482 INFO:journalctl@ceph.osd.1.smithi162.stdout:Apr 27 04:27:34 smithi162 podman[151967]: 2024-04-27 04:27:34.124714705 +0000 UTC m=+0.018797866 image pull 67e414ff4bc41092cb31c84c98a9563a32813d02583028679c46b40362afeb3e quay.ceph.io/ceph-ci/ceph@sha256:90a3a9fa9ceb8977f6dad9ccd36fe46018d07e2118199820523a6b35301d9c16 2024-04-27T04:27:34.483 INFO:journalctl@ceph.osd.1.smithi162.stdout:Apr 27 04:27:34 smithi162 podman[151967]: 2024-04-27 04:27:34.239471356 +0000 UTC m=+0.133554490 container create abe4d0908f03e9d6ba6b398c97f1bfbbc3df09510b95f4ce5ece8a2f73ccc88b (image=quay.ceph.io/ceph-ci/ceph@sha256:90a3a9fa9ceb8977f6dad9ccd36fe46018d07e2118199820523a6b35301d9c16, name=ceph-7f9750d6-044a-11ef-bc93-c7b262605968-osd-1-deactivate, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=main-4e213a5, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.build-date=20240424, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5) 2024-04-27T04:27:34.483 INFO:journalctl@ceph.osd.1.smithi162.stdout:Apr 27 04:27:34 smithi162 podman[151967]: 2024-04-27 04:27:34.483962434 +0000 UTC m=+0.378045572 container init abe4d0908f03e9d6ba6b398c97f1bfbbc3df09510b95f4ce5ece8a2f73ccc88b (image=quay.ceph.io/ceph-ci/ceph@sha256:90a3a9fa9ceb8977f6dad9ccd36fe46018d07e2118199820523a6b35301d9c16, name=ceph-7f9750d6-044a-11ef-bc93-c7b262605968-osd-1-deactivate, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240424, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, RELEASE=main-4e213a5, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True) 2024-04-27T04:27:34.781 INFO:journalctl@ceph.osd.1.smithi162.stdout:Apr 27 04:27:34 smithi162 podman[151967]: 2024-04-27 04:27:34.487422728 +0000 UTC m=+0.381505870 container start abe4d0908f03e9d6ba6b398c97f1bfbbc3df09510b95f4ce5ece8a2f73ccc88b (image=quay.ceph.io/ceph-ci/ceph@sha256:90a3a9fa9ceb8977f6dad9ccd36fe46018d07e2118199820523a6b35301d9c16, name=ceph-7f9750d6-044a-11ef-bc93-c7b262605968-osd-1-deactivate, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=main-4e213a5, org.label-schema.build-date=20240424, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0) 2024-04-27T04:27:34.782 INFO:journalctl@ceph.osd.1.smithi162.stdout:Apr 27 04:27:34 smithi162 podman[151967]: 2024-04-27 04:27:34.567156229 +0000 UTC m=+0.461239363 container attach abe4d0908f03e9d6ba6b398c97f1bfbbc3df09510b95f4ce5ece8a2f73ccc88b (image=quay.ceph.io/ceph-ci/ceph@sha256:90a3a9fa9ceb8977f6dad9ccd36fe46018d07e2118199820523a6b35301d9c16, name=ceph-7f9750d6-044a-11ef-bc93-c7b262605968-osd-1-deactivate, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.license=GPLv2, org.label-schema.build-date=20240424, RELEASE=main-4e213a5, org.label-schema.vendor=CentOS) 2024-04-27T04:27:35.138 INFO:journalctl@ceph.osd.1.smithi162.stdout:Apr 27 04:27:34 smithi162 podman[151967]: 2024-04-27 04:27:34.78081404 +0000 UTC m=+0.674897181 container died abe4d0908f03e9d6ba6b398c97f1bfbbc3df09510b95f4ce5ece8a2f73ccc88b (image=quay.ceph.io/ceph-ci/ceph@sha256:90a3a9fa9ceb8977f6dad9ccd36fe46018d07e2118199820523a6b35301d9c16, name=ceph-7f9750d6-044a-11ef-bc93-c7b262605968-osd-1-deactivate, ceph=True, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, RELEASE=main-4e213a5, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.build-date=20240424, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-27T04:27:35.599 DEBUG:teuthology.orchestra.run.smithi162:> sudo pkill -f 'journalctl -f -n 0 -u ceph-7f9750d6-044a-11ef-bc93-c7b262605968@osd.1.service' 2024-04-27T04:27:35.644 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-27T04:27:35.644 INFO:tasks.cephadm.osd.1:Stopped osd.1 2024-04-27T04:27:35.644 INFO:tasks.cephadm.osd.2:Stopping osd.2... 2024-04-27T04:27:35.644 DEBUG:teuthology.orchestra.run.smithi162:> sudo systemctl stop ceph-7f9750d6-044a-11ef-bc93-c7b262605968@osd.2 2024-04-27T04:27:35.887 INFO:journalctl@ceph.osd.2.smithi162.stdout:Apr 27 04:27:35 smithi162 systemd[1]: Stopping Ceph osd.2 for 7f9750d6-044a-11ef-bc93-c7b262605968... 2024-04-27T04:27:36.388 INFO:journalctl@ceph.osd.2.smithi162.stdout:Apr 27 04:27:36 smithi162 ceph-7f9750d6-044a-11ef-bc93-c7b262605968-osd-2[132895]: 2024-04-27T04:27:36.006+0000 7ff0140e1700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.2 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-27T04:27:36.388 INFO:journalctl@ceph.osd.2.smithi162.stdout:Apr 27 04:27:36 smithi162 ceph-7f9750d6-044a-11ef-bc93-c7b262605968-osd-2[132895]: 2024-04-27T04:27:36.006+0000 7ff0140e1700 -1 osd.2 84 *** Got signal Terminated *** 2024-04-27T04:27:36.388 INFO:journalctl@ceph.osd.2.smithi162.stdout:Apr 27 04:27:36 smithi162 ceph-7f9750d6-044a-11ef-bc93-c7b262605968-osd-2[132895]: 2024-04-27T04:27:36.006+0000 7ff0140e1700 -1 osd.2 84 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-27T04:27:41.791 INFO:journalctl@ceph.osd.2.smithi162.stdout:Apr 27 04:27:41 smithi162 podman[152075]: 2024-04-27 04:27:41.518150681 +0000 UTC m=+5.637341516 container died d24878d461e7102a217196422c598c253a67d0ce73605f8073d54a1b38da2b3e (image=quay.ceph.io/ceph-ci/ceph@sha256:90a3a9fa9ceb8977f6dad9ccd36fe46018d07e2118199820523a6b35301d9c16, name=ceph-7f9750d6-044a-11ef-bc93-c7b262605968-osd-2, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.build-date=20240424, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, RELEASE=main-4e213a5) 2024-04-27T04:27:42.137 INFO:journalctl@ceph.osd.2.smithi162.stdout:Apr 27 04:27:41 smithi162 podman[152075]: 2024-04-27 04:27:41.898843671 +0000 UTC m=+6.018034510 container cleanup d24878d461e7102a217196422c598c253a67d0ce73605f8073d54a1b38da2b3e (image=quay.ceph.io/ceph-ci/ceph@sha256:90a3a9fa9ceb8977f6dad9ccd36fe46018d07e2118199820523a6b35301d9c16, name=ceph-7f9750d6-044a-11ef-bc93-c7b262605968-osd-2, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , ceph=True, CEPH_POINT_RELEASE=, RELEASE=main-4e213a5, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240424, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.vendor=CentOS) 2024-04-27T04:27:42.138 INFO:journalctl@ceph.osd.2.smithi162.stdout:Apr 27 04:27:41 smithi162 bash[152075]: ceph-7f9750d6-044a-11ef-bc93-c7b262605968-osd-2 2024-04-27T04:27:42.565 INFO:journalctl@ceph.osd.2.smithi162.stdout:Apr 27 04:27:42 smithi162 podman[152093]: 2024-04-27 04:27:42.209069848 +0000 UTC m=+0.685095588 container remove d24878d461e7102a217196422c598c253a67d0ce73605f8073d54a1b38da2b3e (image=quay.ceph.io/ceph-ci/ceph@sha256:90a3a9fa9ceb8977f6dad9ccd36fe46018d07e2118199820523a6b35301d9c16, name=ceph-7f9750d6-044a-11ef-bc93-c7b262605968-osd-2, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=main-4e213a5, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, ceph=True, org.label-schema.build-date=20240424) 2024-04-27T04:27:42.888 INFO:journalctl@ceph.osd.2.smithi162.stdout:Apr 27 04:27:42 smithi162 podman[152189]: 2024-04-27 04:27:42.470077481 +0000 UTC m=+0.017899670 image pull 67e414ff4bc41092cb31c84c98a9563a32813d02583028679c46b40362afeb3e quay.ceph.io/ceph-ci/ceph@sha256:90a3a9fa9ceb8977f6dad9ccd36fe46018d07e2118199820523a6b35301d9c16 2024-04-27T04:27:42.888 INFO:journalctl@ceph.osd.2.smithi162.stdout:Apr 27 04:27:42 smithi162 podman[152189]: 2024-04-27 04:27:42.568192321 +0000 UTC m=+0.116014505 container create 12bde449a1a1468c4f4ce4e730551c5644f2d00094b8de5458a8c0a104585f95 (image=quay.ceph.io/ceph-ci/ceph@sha256:90a3a9fa9ceb8977f6dad9ccd36fe46018d07e2118199820523a6b35301d9c16, name=ceph-7f9750d6-044a-11ef-bc93-c7b262605968-osd-2-deactivate, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=main-4e213a5, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240424, org.label-schema.schema-version=1.0, GIT_CLEAN=True) 2024-04-27T04:27:42.888 INFO:journalctl@ceph.osd.2.smithi162.stdout:Apr 27 04:27:42 smithi162 podman[152189]: 2024-04-27 04:27:42.804503319 +0000 UTC m=+0.352325502 container init 12bde449a1a1468c4f4ce4e730551c5644f2d00094b8de5458a8c0a104585f95 (image=quay.ceph.io/ceph-ci/ceph@sha256:90a3a9fa9ceb8977f6dad9ccd36fe46018d07e2118199820523a6b35301d9c16, name=ceph-7f9750d6-044a-11ef-bc93-c7b262605968-osd-2-deactivate, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_BRANCH=HEAD, org.label-schema.build-date=20240424, io.buildah.version=1.33.5, RELEASE=main-4e213a5, GIT_CLEAN=True, org.label-schema.vendor=CentOS) 2024-04-27T04:27:42.888 INFO:journalctl@ceph.osd.2.smithi162.stdout:Apr 27 04:27:42 smithi162 podman[152189]: 2024-04-27 04:27:42.807722603 +0000 UTC m=+0.355544788 container start 12bde449a1a1468c4f4ce4e730551c5644f2d00094b8de5458a8c0a104585f95 (image=quay.ceph.io/ceph-ci/ceph@sha256:90a3a9fa9ceb8977f6dad9ccd36fe46018d07e2118199820523a6b35301d9c16, name=ceph-7f9750d6-044a-11ef-bc93-c7b262605968-osd-2-deactivate, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , RELEASE=main-4e213a5, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240424, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True) 2024-04-27T04:27:43.388 INFO:journalctl@ceph.osd.2.smithi162.stdout:Apr 27 04:27:42 smithi162 podman[152189]: 2024-04-27 04:27:42.946523594 +0000 UTC m=+0.494345786 container attach 12bde449a1a1468c4f4ce4e730551c5644f2d00094b8de5458a8c0a104585f95 (image=quay.ceph.io/ceph-ci/ceph@sha256:90a3a9fa9ceb8977f6dad9ccd36fe46018d07e2118199820523a6b35301d9c16, name=ceph-7f9750d6-044a-11ef-bc93-c7b262605968-osd-2-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240424, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, RELEASE=main-4e213a5, GIT_BRANCH=HEAD, ceph=True, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-27T04:27:43.388 INFO:journalctl@ceph.osd.2.smithi162.stdout:Apr 27 04:27:43 smithi162 podman[152189]: 2024-04-27 04:27:43.104949137 +0000 UTC m=+0.652771311 container died 12bde449a1a1468c4f4ce4e730551c5644f2d00094b8de5458a8c0a104585f95 (image=quay.ceph.io/ceph-ci/ceph@sha256:90a3a9fa9ceb8977f6dad9ccd36fe46018d07e2118199820523a6b35301d9c16, name=ceph-7f9750d6-044a-11ef-bc93-c7b262605968-osd-2-deactivate, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, RELEASE=main-4e213a5, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.build-date=20240424, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-27T04:27:43.701 DEBUG:teuthology.orchestra.run.smithi162:> sudo pkill -f 'journalctl -f -n 0 -u ceph-7f9750d6-044a-11ef-bc93-c7b262605968@osd.2.service' 2024-04-27T04:27:43.737 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-27T04:27:43.738 INFO:tasks.cephadm.osd.2:Stopped osd.2 2024-04-27T04:27:43.738 INFO:tasks.cephadm.osd.3:Stopping osd.3... 2024-04-27T04:27:43.738 DEBUG:teuthology.orchestra.run.smithi181:> sudo systemctl stop ceph-7f9750d6-044a-11ef-bc93-c7b262605968@osd.3 2024-04-27T04:27:44.130 INFO:journalctl@ceph.osd.3.smithi181.stdout:Apr 27 04:27:43 smithi181 systemd[1]: Stopping Ceph osd.3 for 7f9750d6-044a-11ef-bc93-c7b262605968... 2024-04-27T04:27:44.423 INFO:journalctl@ceph.osd.3.smithi181.stdout:Apr 27 04:27:44 smithi181 ceph-7f9750d6-044a-11ef-bc93-c7b262605968-osd-3[90400]: 2024-04-27T04:27:44.129+0000 7fbc5e4da700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.3 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-27T04:27:44.423 INFO:journalctl@ceph.osd.3.smithi181.stdout:Apr 27 04:27:44 smithi181 ceph-7f9750d6-044a-11ef-bc93-c7b262605968-osd-3[90400]: 2024-04-27T04:27:44.129+0000 7fbc5e4da700 -1 osd.3 84 *** Got signal Terminated *** 2024-04-27T04:27:44.423 INFO:journalctl@ceph.osd.3.smithi181.stdout:Apr 27 04:27:44 smithi181 ceph-7f9750d6-044a-11ef-bc93-c7b262605968-osd-3[90400]: 2024-04-27T04:27:44.129+0000 7fbc5e4da700 -1 osd.3 84 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-27T04:27:47.673 INFO:journalctl@ceph.osd.3.smithi181.stdout:Apr 27 04:27:47 smithi181 ceph-7f9750d6-044a-11ef-bc93-c7b262605968-osd-3[90400]: 2024-04-27T04:27:47.420+0000 7fbc56ac6700 -1 osd.3 84 heartbeat_check: no reply from 172.21.15.162:6806 osd.0 since back 2024-04-27T04:27:22.541791+0000 front 2024-04-27T04:27:22.541774+0000 (oldest deadline 2024-04-27T04:27:46.641521+0000) 2024-04-27T04:27:47.673 INFO:journalctl@ceph.osd.4.smithi181.stdout:Apr 27 04:27:47 smithi181 ceph-7f9750d6-044a-11ef-bc93-c7b262605968-osd-4[95698]: 2024-04-27T04:27:47.273+0000 7ffb02275700 -1 osd.4 84 heartbeat_check: no reply from 172.21.15.162:6806 osd.0 since back 2024-04-27T04:27:24.094144+0000 front 2024-04-27T04:27:24.094198+0000 (oldest deadline 2024-04-27T04:27:46.394057+0000) 2024-04-27T04:27:48.673 INFO:journalctl@ceph.osd.3.smithi181.stdout:Apr 27 04:27:48 smithi181 ceph-7f9750d6-044a-11ef-bc93-c7b262605968-osd-3[90400]: 2024-04-27T04:27:48.397+0000 7fbc56ac6700 -1 osd.3 84 heartbeat_check: no reply from 172.21.15.162:6806 osd.0 since back 2024-04-27T04:27:22.541791+0000 front 2024-04-27T04:27:22.541774+0000 (oldest deadline 2024-04-27T04:27:46.641521+0000) 2024-04-27T04:27:48.673 INFO:journalctl@ceph.osd.4.smithi181.stdout:Apr 27 04:27:48 smithi181 ceph-7f9750d6-044a-11ef-bc93-c7b262605968-osd-4[95698]: 2024-04-27T04:27:48.285+0000 7ffb02275700 -1 osd.4 84 heartbeat_check: no reply from 172.21.15.162:6806 osd.0 since back 2024-04-27T04:27:24.094144+0000 front 2024-04-27T04:27:24.094198+0000 (oldest deadline 2024-04-27T04:27:46.394057+0000) 2024-04-27T04:27:49.642 INFO:journalctl@ceph.osd.4.smithi181.stdout:Apr 27 04:27:49 smithi181 ceph-7f9750d6-044a-11ef-bc93-c7b262605968-osd-4[95698]: 2024-04-27T04:27:49.242+0000 7ffb02275700 -1 osd.4 84 heartbeat_check: no reply from 172.21.15.162:6806 osd.0 since back 2024-04-27T04:27:24.094144+0000 front 2024-04-27T04:27:24.094198+0000 (oldest deadline 2024-04-27T04:27:46.394057+0000) 2024-04-27T04:27:49.923 INFO:journalctl@ceph.osd.3.smithi181.stdout:Apr 27 04:27:49 smithi181 podman[114945]: 2024-04-27 04:27:49.642347151 +0000 UTC m=+5.719373491 container died ab473cfd54d26de1cca563d0c68ad497b05a5291422771be5378898a3ab2abda (image=quay.ceph.io/ceph-ci/ceph@sha256:90a3a9fa9ceb8977f6dad9ccd36fe46018d07e2118199820523a6b35301d9c16, name=ceph-7f9750d6-044a-11ef-bc93-c7b262605968-osd-3, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.schema-version=1.0, RELEASE=main-4e213a5, org.label-schema.build-date=20240424, GIT_CLEAN=True, GIT_BRANCH=HEAD) 2024-04-27T04:27:50.264 INFO:journalctl@ceph.osd.3.smithi181.stdout:Apr 27 04:27:50 smithi181 podman[114945]: 2024-04-27 04:27:50.030866766 +0000 UTC m=+6.107893095 container cleanup ab473cfd54d26de1cca563d0c68ad497b05a5291422771be5378898a3ab2abda (image=quay.ceph.io/ceph-ci/ceph@sha256:90a3a9fa9ceb8977f6dad9ccd36fe46018d07e2118199820523a6b35301d9c16, name=ceph-7f9750d6-044a-11ef-bc93-c7b262605968-osd-3, GIT_BRANCH=HEAD, org.label-schema.build-date=20240424, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=main-4e213a5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_CLEAN=True, ceph=True, org.label-schema.vendor=CentOS) 2024-04-27T04:27:50.264 INFO:journalctl@ceph.osd.3.smithi181.stdout:Apr 27 04:27:50 smithi181 bash[114945]: ceph-7f9750d6-044a-11ef-bc93-c7b262605968-osd-3 2024-04-27T04:27:50.578 INFO:journalctl@ceph.osd.3.smithi181.stdout:Apr 27 04:27:50 smithi181 podman[114963]: 2024-04-27 04:27:50.265202713 +0000 UTC m=+0.620979139 container remove ab473cfd54d26de1cca563d0c68ad497b05a5291422771be5378898a3ab2abda (image=quay.ceph.io/ceph-ci/ceph@sha256:90a3a9fa9ceb8977f6dad9ccd36fe46018d07e2118199820523a6b35301d9c16, name=ceph-7f9750d6-044a-11ef-bc93-c7b262605968-osd-3, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, ceph=True, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.build-date=20240424, RELEASE=main-4e213a5, org.label-schema.vendor=CentOS) 2024-04-27T04:27:50.579 INFO:journalctl@ceph.osd.4.smithi181.stdout:Apr 27 04:27:50 smithi181 ceph-7f9750d6-044a-11ef-bc93-c7b262605968-osd-4[95698]: 2024-04-27T04:27:50.275+0000 7ffb02275700 -1 osd.4 84 heartbeat_check: no reply from 172.21.15.162:6806 osd.0 since back 2024-04-27T04:27:24.094144+0000 front 2024-04-27T04:27:24.094198+0000 (oldest deadline 2024-04-27T04:27:46.394057+0000) 2024-04-27T04:27:50.851 INFO:journalctl@ceph.osd.3.smithi181.stdout:Apr 27 04:27:50 smithi181 podman[115056]: 2024-04-27 04:27:50.482911649 +0000 UTC m=+0.018368668 image pull 67e414ff4bc41092cb31c84c98a9563a32813d02583028679c46b40362afeb3e quay.ceph.io/ceph-ci/ceph@sha256:90a3a9fa9ceb8977f6dad9ccd36fe46018d07e2118199820523a6b35301d9c16 2024-04-27T04:27:50.851 INFO:journalctl@ceph.osd.3.smithi181.stdout:Apr 27 04:27:50 smithi181 podman[115056]: 2024-04-27 04:27:50.607881756 +0000 UTC m=+0.143338761 container create 6a578e1dc7f9a87a56574558cf6fe724b5bbde70bc6eb3e5fe27299ad106ea73 (image=quay.ceph.io/ceph-ci/ceph@sha256:90a3a9fa9ceb8977f6dad9ccd36fe46018d07e2118199820523a6b35301d9c16, name=ceph-7f9750d6-044a-11ef-bc93-c7b262605968-osd-3-deactivate, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, org.label-schema.build-date=20240424, ceph=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_CLEAN=True, maintainer=Guillaume Abrioux , RELEASE=main-4e213a5) 2024-04-27T04:27:50.851 INFO:journalctl@ceph.osd.3.smithi181.stdout:Apr 27 04:27:50 smithi181 podman[115056]: 2024-04-27 04:27:50.85215546 +0000 UTC m=+0.387612463 container init 6a578e1dc7f9a87a56574558cf6fe724b5bbde70bc6eb3e5fe27299ad106ea73 (image=quay.ceph.io/ceph-ci/ceph@sha256:90a3a9fa9ceb8977f6dad9ccd36fe46018d07e2118199820523a6b35301d9c16, name=ceph-7f9750d6-044a-11ef-bc93-c7b262605968-osd-3-deactivate, CEPH_POINT_RELEASE=, GIT_CLEAN=True, RELEASE=main-4e213a5, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.build-date=20240424, ceph=True) 2024-04-27T04:27:51.154 INFO:journalctl@ceph.osd.3.smithi181.stdout:Apr 27 04:27:50 smithi181 podman[115056]: 2024-04-27 04:27:50.856502034 +0000 UTC m=+0.391959038 container start 6a578e1dc7f9a87a56574558cf6fe724b5bbde70bc6eb3e5fe27299ad106ea73 (image=quay.ceph.io/ceph-ci/ceph@sha256:90a3a9fa9ceb8977f6dad9ccd36fe46018d07e2118199820523a6b35301d9c16, name=ceph-7f9750d6-044a-11ef-bc93-c7b262605968-osd-3-deactivate, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.build-date=20240424, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=main-4e213a5, ceph=True, org.label-schema.license=GPLv2) 2024-04-27T04:27:51.154 INFO:journalctl@ceph.osd.3.smithi181.stdout:Apr 27 04:27:50 smithi181 podman[115056]: 2024-04-27 04:27:50.935401068 +0000 UTC m=+0.470858086 container attach 6a578e1dc7f9a87a56574558cf6fe724b5bbde70bc6eb3e5fe27299ad106ea73 (image=quay.ceph.io/ceph-ci/ceph@sha256:90a3a9fa9ceb8977f6dad9ccd36fe46018d07e2118199820523a6b35301d9c16, name=ceph-7f9750d6-044a-11ef-bc93-c7b262605968-osd-3-deactivate, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.build-date=20240424, maintainer=Guillaume Abrioux , RELEASE=main-4e213a5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-27T04:27:51.154 INFO:journalctl@ceph.osd.5.smithi181.stdout:Apr 27 04:27:50 smithi181 ceph-7f9750d6-044a-11ef-bc93-c7b262605968-osd-5[100869]: 2024-04-27T04:27:50.897+0000 7fa90da75700 -1 osd.5 84 heartbeat_check: no reply from 172.21.15.162:6806 osd.0 since back 2024-04-27T04:27:21.256769+0000 front 2024-04-27T04:27:21.256839+0000 (oldest deadline 2024-04-27T04:27:50.657168+0000) 2024-04-27T04:27:51.423 INFO:journalctl@ceph.osd.3.smithi181.stdout:Apr 27 04:27:51 smithi181 podman[115056]: 2024-04-27 04:27:51.155078466 +0000 UTC m=+0.690535472 container died 6a578e1dc7f9a87a56574558cf6fe724b5bbde70bc6eb3e5fe27299ad106ea73 (image=quay.ceph.io/ceph-ci/ceph@sha256:90a3a9fa9ceb8977f6dad9ccd36fe46018d07e2118199820523a6b35301d9c16, name=ceph-7f9750d6-044a-11ef-bc93-c7b262605968-osd-3-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, GIT_BRANCH=HEAD, RELEASE=main-4e213a5, io.buildah.version=1.33.5, org.label-schema.build-date=20240424, ceph=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-27T04:27:51.424 INFO:journalctl@ceph.osd.4.smithi181.stdout:Apr 27 04:27:51 smithi181 ceph-7f9750d6-044a-11ef-bc93-c7b262605968-osd-4[95698]: 2024-04-27T04:27:51.244+0000 7ffb02275700 -1 osd.4 84 heartbeat_check: no reply from 172.21.15.162:6806 osd.0 since back 2024-04-27T04:27:24.094144+0000 front 2024-04-27T04:27:24.094198+0000 (oldest deadline 2024-04-27T04:27:46.394057+0000) 2024-04-27T04:27:51.753 DEBUG:teuthology.orchestra.run.smithi181:> sudo pkill -f 'journalctl -f -n 0 -u ceph-7f9750d6-044a-11ef-bc93-c7b262605968@osd.3.service' 2024-04-27T04:27:51.792 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-27T04:27:51.792 INFO:tasks.cephadm.osd.3:Stopped osd.3 2024-04-27T04:27:51.792 INFO:tasks.cephadm.osd.4:Stopping osd.4... 2024-04-27T04:27:51.792 DEBUG:teuthology.orchestra.run.smithi181:> sudo systemctl stop ceph-7f9750d6-044a-11ef-bc93-c7b262605968@osd.4 2024-04-27T04:27:52.125 INFO:journalctl@ceph.osd.4.smithi181.stdout:Apr 27 04:27:51 smithi181 systemd[1]: Stopping Ceph osd.4 for 7f9750d6-044a-11ef-bc93-c7b262605968... 2024-04-27T04:27:52.125 INFO:journalctl@ceph.osd.5.smithi181.stdout:Apr 27 04:27:51 smithi181 ceph-7f9750d6-044a-11ef-bc93-c7b262605968-osd-5[100869]: 2024-04-27T04:27:51.938+0000 7fa90da75700 -1 osd.5 84 heartbeat_check: no reply from 172.21.15.162:6806 osd.0 since back 2024-04-27T04:27:21.256769+0000 front 2024-04-27T04:27:21.256839+0000 (oldest deadline 2024-04-27T04:27:50.657168+0000) 2024-04-27T04:27:52.423 INFO:journalctl@ceph.osd.4.smithi181.stdout:Apr 27 04:27:52 smithi181 ceph-7f9750d6-044a-11ef-bc93-c7b262605968-osd-4[95698]: 2024-04-27T04:27:52.124+0000 7ffb05c81700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.4 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-27T04:27:52.423 INFO:journalctl@ceph.osd.4.smithi181.stdout:Apr 27 04:27:52 smithi181 ceph-7f9750d6-044a-11ef-bc93-c7b262605968-osd-4[95698]: 2024-04-27T04:27:52.124+0000 7ffb05c81700 -1 osd.4 84 *** Got signal Terminated *** 2024-04-27T04:27:52.423 INFO:journalctl@ceph.osd.4.smithi181.stdout:Apr 27 04:27:52 smithi181 ceph-7f9750d6-044a-11ef-bc93-c7b262605968-osd-4[95698]: 2024-04-27T04:27:52.124+0000 7ffb05c81700 -1 osd.4 84 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-27T04:27:52.423 INFO:journalctl@ceph.osd.4.smithi181.stdout:Apr 27 04:27:52 smithi181 ceph-7f9750d6-044a-11ef-bc93-c7b262605968-osd-4[95698]: 2024-04-27T04:27:52.274+0000 7ffb02275700 -1 osd.4 84 heartbeat_check: no reply from 172.21.15.162:6806 osd.0 since back 2024-04-27T04:27:24.094144+0000 front 2024-04-27T04:27:24.094198+0000 (oldest deadline 2024-04-27T04:27:46.394057+0000) 2024-04-27T04:27:53.173 INFO:journalctl@ceph.osd.5.smithi181.stdout:Apr 27 04:27:52 smithi181 ceph-7f9750d6-044a-11ef-bc93-c7b262605968-osd-5[100869]: 2024-04-27T04:27:52.905+0000 7fa90da75700 -1 osd.5 84 heartbeat_check: no reply from 172.21.15.162:6806 osd.0 since back 2024-04-27T04:27:21.256769+0000 front 2024-04-27T04:27:21.256839+0000 (oldest deadline 2024-04-27T04:27:50.657168+0000) 2024-04-27T04:27:53.673 INFO:journalctl@ceph.osd.4.smithi181.stdout:Apr 27 04:27:53 smithi181 ceph-7f9750d6-044a-11ef-bc93-c7b262605968-osd-4[95698]: 2024-04-27T04:27:53.229+0000 7ffb02275700 -1 osd.4 84 heartbeat_check: no reply from 172.21.15.162:6806 osd.0 since back 2024-04-27T04:27:24.094144+0000 front 2024-04-27T04:27:24.094198+0000 (oldest deadline 2024-04-27T04:27:46.394057+0000) 2024-04-27T04:27:54.173 INFO:journalctl@ceph.osd.5.smithi181.stdout:Apr 27 04:27:53 smithi181 ceph-7f9750d6-044a-11ef-bc93-c7b262605968-osd-5[100869]: 2024-04-27T04:27:53.901+0000 7fa90da75700 -1 osd.5 84 heartbeat_check: no reply from 172.21.15.162:6806 osd.0 since back 2024-04-27T04:27:21.256769+0000 front 2024-04-27T04:27:21.256839+0000 (oldest deadline 2024-04-27T04:27:50.657168+0000) 2024-04-27T04:27:54.673 INFO:journalctl@ceph.osd.4.smithi181.stdout:Apr 27 04:27:54 smithi181 ceph-7f9750d6-044a-11ef-bc93-c7b262605968-osd-4[95698]: 2024-04-27T04:27:54.216+0000 7ffb02275700 -1 osd.4 84 heartbeat_check: no reply from 172.21.15.162:6806 osd.0 since back 2024-04-27T04:27:24.094144+0000 front 2024-04-27T04:27:24.094198+0000 (oldest deadline 2024-04-27T04:27:46.394057+0000) 2024-04-27T04:27:55.173 INFO:journalctl@ceph.osd.5.smithi181.stdout:Apr 27 04:27:54 smithi181 ceph-7f9750d6-044a-11ef-bc93-c7b262605968-osd-5[100869]: 2024-04-27T04:27:54.879+0000 7fa90da75700 -1 osd.5 84 heartbeat_check: no reply from 172.21.15.162:6806 osd.0 since back 2024-04-27T04:27:21.256769+0000 front 2024-04-27T04:27:21.256839+0000 (oldest deadline 2024-04-27T04:27:50.657168+0000) 2024-04-27T04:27:55.673 INFO:journalctl@ceph.osd.4.smithi181.stdout:Apr 27 04:27:55 smithi181 ceph-7f9750d6-044a-11ef-bc93-c7b262605968-osd-4[95698]: 2024-04-27T04:27:55.194+0000 7ffb02275700 -1 osd.4 84 heartbeat_check: no reply from 172.21.15.162:6806 osd.0 since back 2024-04-27T04:27:24.094144+0000 front 2024-04-27T04:27:24.094198+0000 (oldest deadline 2024-04-27T04:27:46.394057+0000) 2024-04-27T04:27:55.673 INFO:journalctl@ceph.osd.4.smithi181.stdout:Apr 27 04:27:55 smithi181 ceph-7f9750d6-044a-11ef-bc93-c7b262605968-osd-4[95698]: 2024-04-27T04:27:55.194+0000 7ffb02275700 -1 osd.4 84 heartbeat_check: no reply from 172.21.15.162:6814 osd.1 since back 2024-04-27T04:27:31.694969+0000 front 2024-04-27T04:27:31.695014+0000 (oldest deadline 2024-04-27T04:27:55.194824+0000) 2024-04-27T04:27:56.173 INFO:journalctl@ceph.osd.5.smithi181.stdout:Apr 27 04:27:55 smithi181 ceph-7f9750d6-044a-11ef-bc93-c7b262605968-osd-5[100869]: 2024-04-27T04:27:55.896+0000 7fa90da75700 -1 osd.5 84 heartbeat_check: no reply from 172.21.15.162:6806 osd.0 since back 2024-04-27T04:27:21.256769+0000 front 2024-04-27T04:27:21.256839+0000 (oldest deadline 2024-04-27T04:27:50.657168+0000) 2024-04-27T04:27:56.673 INFO:journalctl@ceph.osd.4.smithi181.stdout:Apr 27 04:27:56 smithi181 ceph-7f9750d6-044a-11ef-bc93-c7b262605968-osd-4[95698]: 2024-04-27T04:27:56.203+0000 7ffb02275700 -1 osd.4 84 heartbeat_check: no reply from 172.21.15.162:6806 osd.0 since back 2024-04-27T04:27:24.094144+0000 front 2024-04-27T04:27:24.094198+0000 (oldest deadline 2024-04-27T04:27:46.394057+0000) 2024-04-27T04:27:56.673 INFO:journalctl@ceph.osd.4.smithi181.stdout:Apr 27 04:27:56 smithi181 ceph-7f9750d6-044a-11ef-bc93-c7b262605968-osd-4[95698]: 2024-04-27T04:27:56.203+0000 7ffb02275700 -1 osd.4 84 heartbeat_check: no reply from 172.21.15.162:6814 osd.1 since back 2024-04-27T04:27:31.694969+0000 front 2024-04-27T04:27:31.695014+0000 (oldest deadline 2024-04-27T04:27:55.194824+0000) 2024-04-27T04:27:57.173 INFO:journalctl@ceph.osd.5.smithi181.stdout:Apr 27 04:27:56 smithi181 ceph-7f9750d6-044a-11ef-bc93-c7b262605968-osd-5[100869]: 2024-04-27T04:27:56.853+0000 7fa90da75700 -1 osd.5 84 heartbeat_check: no reply from 172.21.15.162:6806 osd.0 since back 2024-04-27T04:27:21.256769+0000 front 2024-04-27T04:27:21.256839+0000 (oldest deadline 2024-04-27T04:27:50.657168+0000) 2024-04-27T04:27:57.918 INFO:journalctl@ceph.osd.4.smithi181.stdout:Apr 27 04:27:57 smithi181 podman[115162]: 2024-04-27 04:27:57.655484698 +0000 UTC m=+5.636137720 container died de3e10f2d26d90be9fd0cd7f7bf01bd9ffd95a455849a6d3da7311decd0cf5a1 (image=quay.ceph.io/ceph-ci/ceph@sha256:90a3a9fa9ceb8977f6dad9ccd36fe46018d07e2118199820523a6b35301d9c16, name=ceph-7f9750d6-044a-11ef-bc93-c7b262605968-osd-4, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.build-date=20240424, RELEASE=main-4e213a5, org.label-schema.vendor=CentOS, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0) 2024-04-27T04:27:57.918 INFO:journalctl@ceph.osd.5.smithi181.stdout:Apr 27 04:27:57 smithi181 ceph-7f9750d6-044a-11ef-bc93-c7b262605968-osd-5[100869]: 2024-04-27T04:27:57.898+0000 7fa90da75700 -1 osd.5 84 heartbeat_check: no reply from 172.21.15.162:6806 osd.0 since back 2024-04-27T04:27:21.256769+0000 front 2024-04-27T04:27:21.256839+0000 (oldest deadline 2024-04-27T04:27:50.657168+0000) 2024-04-27T04:27:58.173 INFO:journalctl@ceph.osd.4.smithi181.stdout:Apr 27 04:27:58 smithi181 podman[115162]: 2024-04-27 04:27:58.008765749 +0000 UTC m=+5.989418782 container cleanup de3e10f2d26d90be9fd0cd7f7bf01bd9ffd95a455849a6d3da7311decd0cf5a1 (image=quay.ceph.io/ceph-ci/ceph@sha256:90a3a9fa9ceb8977f6dad9ccd36fe46018d07e2118199820523a6b35301d9c16, name=ceph-7f9750d6-044a-11ef-bc93-c7b262605968-osd-4, RELEASE=main-4e213a5, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240424, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS) 2024-04-27T04:27:58.173 INFO:journalctl@ceph.osd.4.smithi181.stdout:Apr 27 04:27:58 smithi181 bash[115162]: ceph-7f9750d6-044a-11ef-bc93-c7b262605968-osd-4 2024-04-27T04:27:58.626 INFO:journalctl@ceph.osd.4.smithi181.stdout:Apr 27 04:27:58 smithi181 podman[115180]: 2024-04-27 04:27:58.268365948 +0000 UTC m=+0.608994859 container remove de3e10f2d26d90be9fd0cd7f7bf01bd9ffd95a455849a6d3da7311decd0cf5a1 (image=quay.ceph.io/ceph-ci/ceph@sha256:90a3a9fa9ceb8977f6dad9ccd36fe46018d07e2118199820523a6b35301d9c16, name=ceph-7f9750d6-044a-11ef-bc93-c7b262605968-osd-4, org.label-schema.build-date=20240424, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, RELEASE=main-4e213a5, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux ) 2024-04-27T04:27:58.923 INFO:journalctl@ceph.osd.5.smithi181.stdout:Apr 27 04:27:58 smithi181 ceph-7f9750d6-044a-11ef-bc93-c7b262605968-osd-5[100869]: 2024-04-27T04:27:58.869+0000 7fa90da75700 -1 osd.5 84 heartbeat_check: no reply from 172.21.15.162:6806 osd.0 since back 2024-04-27T04:27:21.256769+0000 front 2024-04-27T04:27:21.256839+0000 (oldest deadline 2024-04-27T04:27:50.657168+0000) 2024-04-27T04:27:58.924 INFO:journalctl@ceph.osd.5.smithi181.stdout:Apr 27 04:27:58 smithi181 ceph-7f9750d6-044a-11ef-bc93-c7b262605968-osd-5[100869]: 2024-04-27T04:27:58.869+0000 7fa90da75700 -1 osd.5 84 heartbeat_check: no reply from 172.21.15.162:6814 osd.1 since back 2024-04-27T04:27:31.158095+0000 front 2024-04-27T04:27:31.158014+0000 (oldest deadline 2024-04-27T04:27:58.758301+0000) 2024-04-27T04:27:58.924 INFO:journalctl@ceph.osd.4.smithi181.stdout:Apr 27 04:27:58 smithi181 podman[115273]: 2024-04-27 04:27:58.529557008 +0000 UTC m=+0.017179691 image pull 67e414ff4bc41092cb31c84c98a9563a32813d02583028679c46b40362afeb3e quay.ceph.io/ceph-ci/ceph@sha256:90a3a9fa9ceb8977f6dad9ccd36fe46018d07e2118199820523a6b35301d9c16 2024-04-27T04:27:58.924 INFO:journalctl@ceph.osd.4.smithi181.stdout:Apr 27 04:27:58 smithi181 podman[115273]: 2024-04-27 04:27:58.65265413 +0000 UTC m=+0.140276820 container create 2edb43fb870a50f01360d6dc3bc1b813e42adc2d08c6cfd87082cfc4c7026090 (image=quay.ceph.io/ceph-ci/ceph@sha256:90a3a9fa9ceb8977f6dad9ccd36fe46018d07e2118199820523a6b35301d9c16, name=ceph-7f9750d6-044a-11ef-bc93-c7b262605968-osd-4-deactivate, CEPH_POINT_RELEASE=, RELEASE=main-4e213a5, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240424, io.buildah.version=1.33.5, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True) 2024-04-27T04:27:58.924 INFO:journalctl@ceph.osd.4.smithi181.stdout:Apr 27 04:27:58 smithi181 podman[115273]: 2024-04-27 04:27:58.855183229 +0000 UTC m=+0.342805883 container init 2edb43fb870a50f01360d6dc3bc1b813e42adc2d08c6cfd87082cfc4c7026090 (image=quay.ceph.io/ceph-ci/ceph@sha256:90a3a9fa9ceb8977f6dad9ccd36fe46018d07e2118199820523a6b35301d9c16, name=ceph-7f9750d6-044a-11ef-bc93-c7b262605968-osd-4-deactivate, ceph=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, RELEASE=main-4e213a5, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240424, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, GIT_CLEAN=True, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5) 2024-04-27T04:27:58.924 INFO:journalctl@ceph.osd.4.smithi181.stdout:Apr 27 04:27:58 smithi181 podman[115273]: 2024-04-27 04:27:58.858433937 +0000 UTC m=+0.346056586 container start 2edb43fb870a50f01360d6dc3bc1b813e42adc2d08c6cfd87082cfc4c7026090 (image=quay.ceph.io/ceph-ci/ceph@sha256:90a3a9fa9ceb8977f6dad9ccd36fe46018d07e2118199820523a6b35301d9c16, name=ceph-7f9750d6-044a-11ef-bc93-c7b262605968-osd-4-deactivate, org.label-schema.schema-version=1.0, RELEASE=main-4e213a5, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.build-date=20240424, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-27T04:27:59.423 INFO:journalctl@ceph.osd.4.smithi181.stdout:Apr 27 04:27:58 smithi181 podman[115273]: 2024-04-27 04:27:58.930186931 +0000 UTC m=+0.417809587 container attach 2edb43fb870a50f01360d6dc3bc1b813e42adc2d08c6cfd87082cfc4c7026090 (image=quay.ceph.io/ceph-ci/ceph@sha256:90a3a9fa9ceb8977f6dad9ccd36fe46018d07e2118199820523a6b35301d9c16, name=ceph-7f9750d6-044a-11ef-bc93-c7b262605968-osd-4-deactivate, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=main-4e213a5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240424, GIT_BRANCH=HEAD, ceph=True, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2) 2024-04-27T04:27:59.423 INFO:journalctl@ceph.osd.4.smithi181.stdout:Apr 27 04:27:59 smithi181 podman[115273]: 2024-04-27 04:27:59.153906138 +0000 UTC m=+0.641528798 container died 2edb43fb870a50f01360d6dc3bc1b813e42adc2d08c6cfd87082cfc4c7026090 (image=quay.ceph.io/ceph-ci/ceph@sha256:90a3a9fa9ceb8977f6dad9ccd36fe46018d07e2118199820523a6b35301d9c16, name=ceph-7f9750d6-044a-11ef-bc93-c7b262605968-osd-4-deactivate, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=main-4e213a5, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240424, ceph=True, org.label-schema.vendor=CentOS) 2024-04-27T04:27:59.723 DEBUG:teuthology.orchestra.run.smithi181:> sudo pkill -f 'journalctl -f -n 0 -u ceph-7f9750d6-044a-11ef-bc93-c7b262605968@osd.4.service' 2024-04-27T04:27:59.762 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-27T04:27:59.762 INFO:tasks.cephadm.osd.4:Stopped osd.4 2024-04-27T04:27:59.762 INFO:tasks.cephadm.osd.5:Stopping osd.5... 2024-04-27T04:27:59.762 DEBUG:teuthology.orchestra.run.smithi181:> sudo systemctl stop ceph-7f9750d6-044a-11ef-bc93-c7b262605968@osd.5 2024-04-27T04:28:00.069 INFO:journalctl@ceph.osd.5.smithi181.stdout:Apr 27 04:27:59 smithi181 systemd[1]: Stopping Ceph osd.5 for 7f9750d6-044a-11ef-bc93-c7b262605968... 2024-04-27T04:28:00.069 INFO:journalctl@ceph.osd.5.smithi181.stdout:Apr 27 04:27:59 smithi181 ceph-7f9750d6-044a-11ef-bc93-c7b262605968-osd-5[100869]: 2024-04-27T04:27:59.841+0000 7fa90da75700 -1 osd.5 84 heartbeat_check: no reply from 172.21.15.162:6806 osd.0 since back 2024-04-27T04:27:21.256769+0000 front 2024-04-27T04:27:21.256839+0000 (oldest deadline 2024-04-27T04:27:50.657168+0000) 2024-04-27T04:28:00.070 INFO:journalctl@ceph.osd.5.smithi181.stdout:Apr 27 04:27:59 smithi181 ceph-7f9750d6-044a-11ef-bc93-c7b262605968-osd-5[100869]: 2024-04-27T04:27:59.841+0000 7fa90da75700 -1 osd.5 84 heartbeat_check: no reply from 172.21.15.162:6814 osd.1 since back 2024-04-27T04:27:31.158095+0000 front 2024-04-27T04:27:31.158014+0000 (oldest deadline 2024-04-27T04:27:58.758301+0000) 2024-04-27T04:28:00.070 INFO:journalctl@ceph.osd.5.smithi181.stdout:Apr 27 04:28:00 smithi181 ceph-7f9750d6-044a-11ef-bc93-c7b262605968-osd-5[100869]: 2024-04-27T04:28:00.068+0000 7fa915489700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.5 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-27T04:28:00.070 INFO:journalctl@ceph.osd.5.smithi181.stdout:Apr 27 04:28:00 smithi181 ceph-7f9750d6-044a-11ef-bc93-c7b262605968-osd-5[100869]: 2024-04-27T04:28:00.068+0000 7fa915489700 -1 osd.5 84 *** Got signal Terminated *** 2024-04-27T04:28:00.070 INFO:journalctl@ceph.osd.5.smithi181.stdout:Apr 27 04:28:00 smithi181 ceph-7f9750d6-044a-11ef-bc93-c7b262605968-osd-5[100869]: 2024-04-27T04:28:00.068+0000 7fa915489700 -1 osd.5 84 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-27T04:28:01.173 INFO:journalctl@ceph.osd.5.smithi181.stdout:Apr 27 04:28:00 smithi181 ceph-7f9750d6-044a-11ef-bc93-c7b262605968-osd-5[100869]: 2024-04-27T04:28:00.802+0000 7fa90da75700 -1 osd.5 84 heartbeat_check: no reply from 172.21.15.162:6806 osd.0 since back 2024-04-27T04:27:21.256769+0000 front 2024-04-27T04:27:21.256839+0000 (oldest deadline 2024-04-27T04:27:50.657168+0000) 2024-04-27T04:28:01.173 INFO:journalctl@ceph.osd.5.smithi181.stdout:Apr 27 04:28:00 smithi181 ceph-7f9750d6-044a-11ef-bc93-c7b262605968-osd-5[100869]: 2024-04-27T04:28:00.802+0000 7fa90da75700 -1 osd.5 84 heartbeat_check: no reply from 172.21.15.162:6814 osd.1 since back 2024-04-27T04:27:31.158095+0000 front 2024-04-27T04:27:31.158014+0000 (oldest deadline 2024-04-27T04:27:58.758301+0000) 2024-04-27T04:28:02.173 INFO:journalctl@ceph.osd.5.smithi181.stdout:Apr 27 04:28:01 smithi181 ceph-7f9750d6-044a-11ef-bc93-c7b262605968-osd-5[100869]: 2024-04-27T04:28:01.782+0000 7fa90da75700 -1 osd.5 84 heartbeat_check: no reply from 172.21.15.162:6806 osd.0 since back 2024-04-27T04:27:21.256769+0000 front 2024-04-27T04:27:21.256839+0000 (oldest deadline 2024-04-27T04:27:50.657168+0000) 2024-04-27T04:28:02.173 INFO:journalctl@ceph.osd.5.smithi181.stdout:Apr 27 04:28:01 smithi181 ceph-7f9750d6-044a-11ef-bc93-c7b262605968-osd-5[100869]: 2024-04-27T04:28:01.782+0000 7fa90da75700 -1 osd.5 84 heartbeat_check: no reply from 172.21.15.162:6814 osd.1 since back 2024-04-27T04:27:31.158095+0000 front 2024-04-27T04:27:31.158014+0000 (oldest deadline 2024-04-27T04:27:58.758301+0000) 2024-04-27T04:28:03.173 INFO:journalctl@ceph.osd.5.smithi181.stdout:Apr 27 04:28:02 smithi181 ceph-7f9750d6-044a-11ef-bc93-c7b262605968-osd-5[100869]: 2024-04-27T04:28:02.789+0000 7fa90da75700 -1 osd.5 84 heartbeat_check: no reply from 172.21.15.162:6806 osd.0 since back 2024-04-27T04:27:21.256769+0000 front 2024-04-27T04:27:21.256839+0000 (oldest deadline 2024-04-27T04:27:50.657168+0000) 2024-04-27T04:28:03.173 INFO:journalctl@ceph.osd.5.smithi181.stdout:Apr 27 04:28:02 smithi181 ceph-7f9750d6-044a-11ef-bc93-c7b262605968-osd-5[100869]: 2024-04-27T04:28:02.789+0000 7fa90da75700 -1 osd.5 84 heartbeat_check: no reply from 172.21.15.162:6814 osd.1 since back 2024-04-27T04:27:31.158095+0000 front 2024-04-27T04:27:31.158014+0000 (oldest deadline 2024-04-27T04:27:58.758301+0000) 2024-04-27T04:28:04.173 INFO:journalctl@ceph.osd.5.smithi181.stdout:Apr 27 04:28:03 smithi181 ceph-7f9750d6-044a-11ef-bc93-c7b262605968-osd-5[100869]: 2024-04-27T04:28:03.838+0000 7fa90da75700 -1 osd.5 84 heartbeat_check: no reply from 172.21.15.162:6806 osd.0 since back 2024-04-27T04:27:21.256769+0000 front 2024-04-27T04:27:21.256839+0000 (oldest deadline 2024-04-27T04:27:50.657168+0000) 2024-04-27T04:28:04.173 INFO:journalctl@ceph.osd.5.smithi181.stdout:Apr 27 04:28:03 smithi181 ceph-7f9750d6-044a-11ef-bc93-c7b262605968-osd-5[100869]: 2024-04-27T04:28:03.838+0000 7fa90da75700 -1 osd.5 84 heartbeat_check: no reply from 172.21.15.162:6814 osd.1 since back 2024-04-27T04:27:31.158095+0000 front 2024-04-27T04:27:31.158014+0000 (oldest deadline 2024-04-27T04:27:58.758301+0000) 2024-04-27T04:28:05.173 INFO:journalctl@ceph.osd.5.smithi181.stdout:Apr 27 04:28:04 smithi181 ceph-7f9750d6-044a-11ef-bc93-c7b262605968-osd-5[100869]: 2024-04-27T04:28:04.838+0000 7fa90da75700 -1 osd.5 84 heartbeat_check: no reply from 172.21.15.162:6806 osd.0 since back 2024-04-27T04:27:21.256769+0000 front 2024-04-27T04:27:21.256839+0000 (oldest deadline 2024-04-27T04:27:50.657168+0000) 2024-04-27T04:28:05.173 INFO:journalctl@ceph.osd.5.smithi181.stdout:Apr 27 04:28:04 smithi181 ceph-7f9750d6-044a-11ef-bc93-c7b262605968-osd-5[100869]: 2024-04-27T04:28:04.838+0000 7fa90da75700 -1 osd.5 84 heartbeat_check: no reply from 172.21.15.162:6814 osd.1 since back 2024-04-27T04:27:31.158095+0000 front 2024-04-27T04:27:31.158014+0000 (oldest deadline 2024-04-27T04:27:58.758301+0000) 2024-04-27T04:28:05.173 INFO:journalctl@ceph.osd.5.smithi181.stdout:Apr 27 04:28:04 smithi181 ceph-7f9750d6-044a-11ef-bc93-c7b262605968-osd-5[100869]: 2024-04-27T04:28:04.838+0000 7fa90da75700 -1 osd.5 84 heartbeat_check: no reply from 172.21.15.162:6822 osd.2 since back 2024-04-27T04:27:39.259096+0000 front 2024-04-27T04:27:39.259130+0000 (oldest deadline 2024-04-27T04:28:04.559065+0000) 2024-04-27T04:28:05.845 INFO:journalctl@ceph.osd.5.smithi181.stdout:Apr 27 04:28:05 smithi181 podman[115377]: 2024-04-27 04:28:05.573132043 +0000 UTC m=+5.593210559 container died 7dd4f287ef2ec9004d3750ff836610a4d6ce80f24dda890fe973358f966d8146 (image=quay.ceph.io/ceph-ci/ceph@sha256:90a3a9fa9ceb8977f6dad9ccd36fe46018d07e2118199820523a6b35301d9c16, name=ceph-7f9750d6-044a-11ef-bc93-c7b262605968-osd-5, ceph=True, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , RELEASE=main-4e213a5, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240424, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2) 2024-04-27T04:28:06.173 INFO:journalctl@ceph.osd.5.smithi181.stdout:Apr 27 04:28:05 smithi181 podman[115377]: 2024-04-27 04:28:05.93628136 +0000 UTC m=+5.956359871 container cleanup 7dd4f287ef2ec9004d3750ff836610a4d6ce80f24dda890fe973358f966d8146 (image=quay.ceph.io/ceph-ci/ceph@sha256:90a3a9fa9ceb8977f6dad9ccd36fe46018d07e2118199820523a6b35301d9c16, name=ceph-7f9750d6-044a-11ef-bc93-c7b262605968-osd-5, org.label-schema.build-date=20240424, RELEASE=main-4e213a5, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, io.buildah.version=1.33.5) 2024-04-27T04:28:06.174 INFO:journalctl@ceph.osd.5.smithi181.stdout:Apr 27 04:28:05 smithi181 bash[115377]: ceph-7f9750d6-044a-11ef-bc93-c7b262605968-osd-5 2024-04-27T04:28:06.549 INFO:journalctl@ceph.osd.5.smithi181.stdout:Apr 27 04:28:06 smithi181 podman[115395]: 2024-04-27 04:28:06.205517994 +0000 UTC m=+0.622365133 container remove 7dd4f287ef2ec9004d3750ff836610a4d6ce80f24dda890fe973358f966d8146 (image=quay.ceph.io/ceph-ci/ceph@sha256:90a3a9fa9ceb8977f6dad9ccd36fe46018d07e2118199820523a6b35301d9c16, name=ceph-7f9750d6-044a-11ef-bc93-c7b262605968-osd-5, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240424, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , ceph=True, RELEASE=main-4e213a5, GIT_BRANCH=HEAD, GIT_CLEAN=True) 2024-04-27T04:28:06.856 INFO:journalctl@ceph.osd.5.smithi181.stdout:Apr 27 04:28:06 smithi181 podman[115485]: 2024-04-27 04:28:06.453528266 +0000 UTC m=+0.017973207 image pull 67e414ff4bc41092cb31c84c98a9563a32813d02583028679c46b40362afeb3e quay.ceph.io/ceph-ci/ceph@sha256:90a3a9fa9ceb8977f6dad9ccd36fe46018d07e2118199820523a6b35301d9c16 2024-04-27T04:28:06.856 INFO:journalctl@ceph.osd.5.smithi181.stdout:Apr 27 04:28:06 smithi181 podman[115485]: 2024-04-27 04:28:06.563380835 +0000 UTC m=+0.127825768 container create 5ce5a08eb20d7175492122f4ae00c50b332b69a8931f0cd048ecaefe251695ba (image=quay.ceph.io/ceph-ci/ceph@sha256:90a3a9fa9ceb8977f6dad9ccd36fe46018d07e2118199820523a6b35301d9c16, name=ceph-7f9750d6-044a-11ef-bc93-c7b262605968-osd-5-deactivate, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240424, RELEASE=main-4e213a5, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2) 2024-04-27T04:28:06.857 INFO:journalctl@ceph.osd.5.smithi181.stdout:Apr 27 04:28:06 smithi181 podman[115485]: 2024-04-27 04:28:06.774093586 +0000 UTC m=+0.338538516 container init 5ce5a08eb20d7175492122f4ae00c50b332b69a8931f0cd048ecaefe251695ba (image=quay.ceph.io/ceph-ci/ceph@sha256:90a3a9fa9ceb8977f6dad9ccd36fe46018d07e2118199820523a6b35301d9c16, name=ceph-7f9750d6-044a-11ef-bc93-c7b262605968-osd-5-deactivate, org.label-schema.vendor=CentOS, GIT_CLEAN=True, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=main-4e213a5, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240424, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2) 2024-04-27T04:28:06.857 INFO:journalctl@ceph.osd.5.smithi181.stdout:Apr 27 04:28:06 smithi181 podman[115485]: 2024-04-27 04:28:06.777202029 +0000 UTC m=+0.341646961 container start 5ce5a08eb20d7175492122f4ae00c50b332b69a8931f0cd048ecaefe251695ba (image=quay.ceph.io/ceph-ci/ceph@sha256:90a3a9fa9ceb8977f6dad9ccd36fe46018d07e2118199820523a6b35301d9c16, name=ceph-7f9750d6-044a-11ef-bc93-c7b262605968-osd-5-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240424, GIT_CLEAN=True, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, RELEASE=main-4e213a5) 2024-04-27T04:28:06.857 INFO:journalctl@ceph.osd.5.smithi181.stdout:Apr 27 04:28:06 smithi181 podman[115485]: 2024-04-27 04:28:06.857447237 +0000 UTC m=+0.421892179 container attach 5ce5a08eb20d7175492122f4ae00c50b332b69a8931f0cd048ecaefe251695ba (image=quay.ceph.io/ceph-ci/ceph@sha256:90a3a9fa9ceb8977f6dad9ccd36fe46018d07e2118199820523a6b35301d9c16, name=ceph-7f9750d6-044a-11ef-bc93-c7b262605968-osd-5-deactivate, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240424, ceph=True, RELEASE=main-4e213a5, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-27T04:28:07.173 INFO:journalctl@ceph.osd.5.smithi181.stdout:Apr 27 04:28:07 smithi181 podman[115485]: 2024-04-27 04:28:07.063766689 +0000 UTC m=+0.628211611 container died 5ce5a08eb20d7175492122f4ae00c50b332b69a8931f0cd048ecaefe251695ba (image=quay.ceph.io/ceph-ci/ceph@sha256:90a3a9fa9ceb8977f6dad9ccd36fe46018d07e2118199820523a6b35301d9c16, name=ceph-7f9750d6-044a-11ef-bc93-c7b262605968-osd-5-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, RELEASE=main-4e213a5, org.label-schema.license=GPLv2, org.label-schema.build-date=20240424, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD) 2024-04-27T04:28:07.601 DEBUG:teuthology.orchestra.run.smithi181:> sudo pkill -f 'journalctl -f -n 0 -u ceph-7f9750d6-044a-11ef-bc93-c7b262605968@osd.5.service' 2024-04-27T04:28:07.637 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-27T04:28:07.637 INFO:tasks.cephadm.osd.5:Stopped osd.5 2024-04-27T04:28:07.637 DEBUG:teuthology.orchestra.run.smithi162:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 7f9750d6-044a-11ef-bc93-c7b262605968 --force --keep-logs 2024-04-27T04:28:10.905 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi162.stderr:ceph-fuse[74353]: fuse finished with error 0 and tester_r 0 2024-04-27T04:28:31.246 DEBUG:teuthology.orchestra.run.smithi181:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 7f9750d6-044a-11ef-bc93-c7b262605968 --force --keep-logs 2024-04-27T04:28:44.944 DEBUG:teuthology.orchestra.run.smithi162:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-04-27T04:28:44.976 DEBUG:teuthology.orchestra.run.smithi181:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-04-27T04:28:45.001 INFO:tasks.cephadm:Archiving crash dumps... 2024-04-27T04:28:45.003 DEBUG:teuthology.misc:Transferring archived files from smithi162:/var/lib/ceph/7f9750d6-044a-11ef-bc93-c7b262605968/crash to /home/teuthworker/archive/teuthology-2024-04-27_03:08:09-upgrade-main-distro-default-smithi/7675783/remote/smithi162/crash 2024-04-27T04:28:45.004 DEBUG:teuthology.orchestra.run.smithi162:> sudo tar c -f - -C /var/lib/ceph/7f9750d6-044a-11ef-bc93-c7b262605968/crash -- . 2024-04-27T04:28:45.042 INFO:teuthology.orchestra.run.smithi162.stderr:tar: /var/lib/ceph/7f9750d6-044a-11ef-bc93-c7b262605968/crash: Cannot open: No such file or directory 2024-04-27T04:28:45.042 INFO:teuthology.orchestra.run.smithi162.stderr:tar: Error is not recoverable: exiting now 2024-04-27T04:28:45.044 DEBUG:teuthology.misc:Transferring archived files from smithi181:/var/lib/ceph/7f9750d6-044a-11ef-bc93-c7b262605968/crash to /home/teuthworker/archive/teuthology-2024-04-27_03:08:09-upgrade-main-distro-default-smithi/7675783/remote/smithi181/crash 2024-04-27T04:28:45.045 DEBUG:teuthology.orchestra.run.smithi181:> sudo tar c -f - -C /var/lib/ceph/7f9750d6-044a-11ef-bc93-c7b262605968/crash -- . 2024-04-27T04:28:45.068 INFO:teuthology.orchestra.run.smithi181.stderr:tar: /var/lib/ceph/7f9750d6-044a-11ef-bc93-c7b262605968/crash: Cannot open: No such file or directory 2024-04-27T04:28:45.069 INFO:teuthology.orchestra.run.smithi181.stderr:tar: Error is not recoverable: exiting now 2024-04-27T04:28:45.069 INFO:tasks.cephadm:Checking cluster log for badness... 2024-04-27T04:28:45.070 DEBUG:teuthology.orchestra.run.smithi162:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/7f9750d6-044a-11ef-bc93-c7b262605968/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v FS_DEGRADED | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v 'filesystem is online with fewer MDS than max_mds' | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'do not have an application enabled' | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v 'deprecated feature inline_data' | egrep -v OSD_DOWN | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v PG_AVAILABILITY | egrep -v PG_DEGRADED | egrep -v 'Reduced data availability' | head -n 1 2024-04-27T04:28:45.154 INFO:teuthology.orchestra.run.smithi162.stdout:2024-04-27T04:20:00.000116+0000 mon.smithi162 (mon.0) 677 : cluster [WRN] Health detail: HEALTH_WARN 1 filesystem is degraded 2024-04-27T04:28:45.155 WARNING:tasks.cephadm:Found errors (ERR|WRN|SEC) in cluster log 2024-04-27T04:28:45.155 INFO:tasks.cephadm:Compressing logs... 2024-04-27T04:28:45.155 DEBUG:teuthology.orchestra.run.smithi162:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-04-27T04:28:45.157 DEBUG:teuthology.orchestra.run.smithi181:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-04-27T04:28:45.179 INFO:teuthology.orchestra.run.smithi162.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose -- /var/log/ceph/ceph-client.0.log 2024-04-27T04:28:45.179 INFO:teuthology.orchestra.run.smithi162.stderr:: No such file or directory 2024-04-27T04:28:45.179 INFO:teuthology.orchestra.run.smithi162.stderr:gzip -5 --verbose -- /var/log/ceph/7f9750d6-044a-11ef-bc93-c7b262605968/ceph-mds.cephfs.smithi162.gzlyjj.log 2024-04-27T04:28:45.179 INFO:teuthology.orchestra.run.smithi162.stderr:/var/log/ceph/ceph-client.0.log: gzip -5 --verbose -- /var/log/ceph/7f9750d6-044a-11ef-bc93-c7b262605968/ceph.log 2024-04-27T04:28:45.179 INFO:teuthology.orchestra.run.smithi162.stderr:gzip -5 --verbose -- /var/log/ceph/7f9750d6-044a-11ef-bc93-c7b262605968/ceph-osd.2.log 2024-04-27T04:28:45.179 INFO:teuthology.orchestra.run.smithi162.stderr:/var/log/ceph/7f9750d6-044a-11ef-bc93-c7b262605968/ceph-mds.cephfs.smithi162.gzlyjj.log: /var/log/ceph/7f9750d6-044a-11ef-bc93-c7b262605968/ceph.log: gzip -5 --verbose -- /var/log/ceph/7f9750d6-044a-11ef-bc93-c7b262605968/ceph.audit.log 2024-04-27T04:28:45.187 INFO:teuthology.orchestra.run.smithi162.stderr:gzip -5 --verbose -- /var/log/ceph/7f9750d6-044a-11ef-bc93-c7b262605968/ceph-osd.1.log 2024-04-27T04:28:45.187 INFO:teuthology.orchestra.run.smithi162.stderr:/var/log/ceph/7f9750d6-044a-11ef-bc93-c7b262605968/ceph-osd.2.log: /var/log/ceph/7f9750d6-044a-11ef-bc93-c7b262605968/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/7f9750d6-044a-11ef-bc93-c7b262605968/ceph-osd.0.log 2024-04-27T04:28:45.187 INFO:teuthology.orchestra.run.smithi162.stderr:/var/log/ceph/7f9750d6-044a-11ef-bc93-c7b262605968/ceph-osd.1.log: gzip -5 --verbose -- /var/log/ceph/7f9750d6-044a-11ef-bc93-c7b262605968/ceph-mon.smithi162.log 2024-04-27T04:28:45.188 INFO:teuthology.orchestra.run.smithi162.stderr:/var/log/ceph/7f9750d6-044a-11ef-bc93-c7b262605968/ceph-osd.0.log: gzip -5 --verbose -- /var/log/ceph/7f9750d6-044a-11ef-bc93-c7b262605968/ceph.cephadm.log 2024-04-27T04:28:45.188 INFO:teuthology.orchestra.run.smithi162.stderr:/var/log/ceph/7f9750d6-044a-11ef-bc93-c7b262605968/ceph-mon.smithi162.log: 88.2% -- replaced with /var/log/ceph/7f9750d6-044a-11ef-bc93-c7b262605968/ceph.log.gz 2024-04-27T04:28:45.188 INFO:teuthology.orchestra.run.smithi162.stderr:gzip -5 --verbose -- /var/log/ceph/7f9750d6-044a-11ef-bc93-c7b262605968/ceph-mgr.smithi162.oennfn.log 2024-04-27T04:28:45.188 INFO:teuthology.orchestra.run.smithi162.stderr:/var/log/ceph/7f9750d6-044a-11ef-bc93-c7b262605968/ceph.cephadm.log: 85.5% -- replaced with /var/log/ceph/7f9750d6-044a-11ef-bc93-c7b262605968/ceph.cephadm.log.gz 2024-04-27T04:28:45.188 INFO:teuthology.orchestra.run.smithi162.stderr:gzip -5 --verbose -- /var/log/ceph/7f9750d6-044a-11ef-bc93-c7b262605968/ceph-volume.log 2024-04-27T04:28:45.188 INFO:teuthology.orchestra.run.smithi162.stderr:/var/log/ceph/7f9750d6-044a-11ef-bc93-c7b262605968/ceph-mgr.smithi162.oennfn.log: 91.2% -- replaced with /var/log/ceph/7f9750d6-044a-11ef-bc93-c7b262605968/ceph.audit.log.gz 2024-04-27T04:28:45.190 INFO:teuthology.orchestra.run.smithi181.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose -- /var/log/ceph/ceph-client.1.log 2024-04-27T04:28:45.190 INFO:teuthology.orchestra.run.smithi181.stderr:: No such file or directory 2024-04-27T04:28:45.190 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/7f9750d6-044a-11ef-bc93-c7b262605968/ceph.log 2024-04-27T04:28:45.190 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.1.log: gzip -5 --verbose -- /var/log/ceph/7f9750d6-044a-11ef-bc93-c7b262605968/ceph.audit.log 2024-04-27T04:28:45.190 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/7f9750d6-044a-11ef-bc93-c7b262605968/ceph-osd.5.log 2024-04-27T04:28:45.190 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/7f9750d6-044a-11ef-bc93-c7b262605968/ceph.log: /var/log/ceph/7f9750d6-044a-11ef-bc93-c7b262605968/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/7f9750d6-044a-11ef-bc93-c7b262605968/ceph-osd.3.log 2024-04-27T04:28:45.190 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/7f9750d6-044a-11ef-bc93-c7b262605968/ceph-mgr.smithi181.vtwuhz.log 2024-04-27T04:28:45.190 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/7f9750d6-044a-11ef-bc93-c7b262605968/ceph-osd.5.log: /var/log/ceph/7f9750d6-044a-11ef-bc93-c7b262605968/ceph-osd.3.log: gzip -5 --verbose -- /var/log/ceph/7f9750d6-044a-11ef-bc93-c7b262605968/ceph-osd.4.log 2024-04-27T04:28:45.190 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/7f9750d6-044a-11ef-bc93-c7b262605968/ceph-mgr.smithi181.vtwuhz.log: gzip -5 --verbose -- /var/log/ceph/7f9750d6-044a-11ef-bc93-c7b262605968/ceph-mds.cephfs.smithi181.bkkcdm.log 2024-04-27T04:28:45.191 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/7f9750d6-044a-11ef-bc93-c7b262605968/ceph-osd.4.log: gzip -5 --verbose -- /var/log/ceph/7f9750d6-044a-11ef-bc93-c7b262605968/ceph.cephadm.log 2024-04-27T04:28:45.191 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/7f9750d6-044a-11ef-bc93-c7b262605968/ceph-mds.cephfs.smithi181.bkkcdm.log: 88.2% -- replaced with /var/log/ceph/7f9750d6-044a-11ef-bc93-c7b262605968/ceph.log.gz 2024-04-27T04:28:45.191 INFO:teuthology.orchestra.run.smithi181.stderr: 91.4% -- replaced with /var/log/ceph/7f9750d6-044a-11ef-bc93-c7b262605968/ceph.audit.log.gz 2024-04-27T04:28:45.191 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/7f9750d6-044a-11ef-bc93-c7b262605968/ceph-mon.smithi181.log 2024-04-27T04:28:45.191 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/7f9750d6-044a-11ef-bc93-c7b262605968/ceph.cephadm.log: gzip -5 --verbose -- /var/log/ceph/7f9750d6-044a-11ef-bc93-c7b262605968/ceph-mds.cephfs.smithi181.dzomgh.log 2024-04-27T04:28:45.191 INFO:teuthology.orchestra.run.smithi181.stderr: 85.4%/var/log/ceph/7f9750d6-044a-11ef-bc93-c7b262605968/ceph-mon.smithi181.log: -- replaced with /var/log/ceph/7f9750d6-044a-11ef-bc93-c7b262605968/ceph.cephadm.log.gz 2024-04-27T04:28:45.191 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/7f9750d6-044a-11ef-bc93-c7b262605968/ceph-volume.log 2024-04-27T04:28:45.198 INFO:teuthology.orchestra.run.smithi162.stderr:gzip -5 --verbose -- /var/log/ceph/7f9750d6-044a-11ef-bc93-c7b262605968/ceph-mds.cephfs.smithi162.xcnapk.log 2024-04-27T04:28:45.199 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/7f9750d6-044a-11ef-bc93-c7b262605968/ceph-mds.cephfs.smithi181.dzomgh.log: gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-04-27T04:28:45.210 INFO:teuthology.orchestra.run.smithi162.stderr:/var/log/ceph/7f9750d6-044a-11ef-bc93-c7b262605968/ceph-volume.log: gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-04-27T04:28:45.221 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/7f9750d6-044a-11ef-bc93-c7b262605968/ceph-volume.log: /var/log/ceph/cephadm.log: 92.4% -- replaced with /var/log/ceph/cephadm.log.gz 2024-04-27T04:28:45.222 INFO:teuthology.orchestra.run.smithi162.stderr:/var/log/ceph/7f9750d6-044a-11ef-bc93-c7b262605968/ceph-mds.cephfs.smithi162.xcnapk.log: 91.1% -- replaced with /var/log/ceph/7f9750d6-044a-11ef-bc93-c7b262605968/ceph-volume.log.gz 2024-04-27T04:28:45.241 INFO:teuthology.orchestra.run.smithi181.stderr: 91.1% -- replaced with /var/log/ceph/7f9750d6-044a-11ef-bc93-c7b262605968/ceph-volume.log.gz 2024-04-27T04:28:45.255 INFO:teuthology.orchestra.run.smithi181.stderr: 89.8% -- replaced with /var/log/ceph/7f9750d6-044a-11ef-bc93-c7b262605968/ceph-mgr.smithi181.vtwuhz.log.gz 2024-04-27T04:28:45.257 INFO:teuthology.orchestra.run.smithi162.stderr:/var/log/ceph/cephadm.log: 92.0% -- replaced with /var/log/ceph/cephadm.log.gz 2024-04-27T04:28:45.514 INFO:teuthology.orchestra.run.smithi181.stderr: 92.6% -- replaced with /var/log/ceph/7f9750d6-044a-11ef-bc93-c7b262605968/ceph-mon.smithi181.log.gz 2024-04-27T04:28:45.621 INFO:teuthology.orchestra.run.smithi162.stderr: 88.7% -- replaced with /var/log/ceph/7f9750d6-044a-11ef-bc93-c7b262605968/ceph-mgr.smithi162.oennfn.log.gz 2024-04-27T04:28:46.427 INFO:teuthology.orchestra.run.smithi162.stderr: 90.5% -- replaced with /var/log/ceph/7f9750d6-044a-11ef-bc93-c7b262605968/ceph-mon.smithi162.log.gz 2024-04-27T04:28:50.031 INFO:teuthology.orchestra.run.smithi162.stderr: 93.4% -- replaced with /var/log/ceph/7f9750d6-044a-11ef-bc93-c7b262605968/ceph-osd.0.log.gz 2024-04-27T04:28:50.575 INFO:teuthology.orchestra.run.smithi181.stderr: 93.5% -- replaced with /var/log/ceph/7f9750d6-044a-11ef-bc93-c7b262605968/ceph-osd.5.log.gz 2024-04-27T04:28:51.124 INFO:teuthology.orchestra.run.smithi181.stderr: 93.3% -- replaced with /var/log/ceph/7f9750d6-044a-11ef-bc93-c7b262605968/ceph-osd.4.log.gz 2024-04-27T04:28:51.365 INFO:teuthology.orchestra.run.smithi162.stderr: 93.4% -- replaced with /var/log/ceph/7f9750d6-044a-11ef-bc93-c7b262605968/ceph-osd.2.log.gz 2024-04-27T04:28:52.150 INFO:teuthology.orchestra.run.smithi162.stderr: 93.4% -- replaced with /var/log/ceph/7f9750d6-044a-11ef-bc93-c7b262605968/ceph-osd.1.log.gz 2024-04-27T04:28:53.067 INFO:teuthology.orchestra.run.smithi162.stderr: 94.7% -- replaced with /var/log/ceph/7f9750d6-044a-11ef-bc93-c7b262605968/ceph-mds.cephfs.smithi162.gzlyjj.log.gz 2024-04-27T04:28:53.073 INFO:teuthology.orchestra.run.smithi181.stderr: 94.7% -- replaced with /var/log/ceph/7f9750d6-044a-11ef-bc93-c7b262605968/ceph-mds.cephfs.smithi181.bkkcdm.log.gz 2024-04-27T04:28:53.420 INFO:teuthology.orchestra.run.smithi181.stderr: 93.5% -- replaced with /var/log/ceph/7f9750d6-044a-11ef-bc93-c7b262605968/ceph-osd.3.log.gz 2024-04-27T04:28:59.431 INFO:teuthology.orchestra.run.smithi181.stderr:gzip: /var/log/ceph/ceph-client.1.log: file size changed while zipping 2024-04-27T04:28:59.431 INFO:teuthology.orchestra.run.smithi181.stderr: 93.0% -- replaced with /var/log/ceph/ceph-client.1.log.gz 2024-04-27T04:28:59.591 INFO:teuthology.orchestra.run.smithi162.stderr: 94.8% -- replaced with /var/log/ceph/7f9750d6-044a-11ef-bc93-c7b262605968/ceph-mds.cephfs.smithi162.xcnapk.log.gz 2024-04-27T04:29:00.013 INFO:teuthology.orchestra.run.smithi162.stderr: 93.0% -- replaced with /var/log/ceph/ceph-client.0.log.gz 2024-04-27T04:29:00.015 INFO:teuthology.orchestra.run.smithi162.stderr: 2024-04-27T04:29:00.015 INFO:teuthology.orchestra.run.smithi162.stderr:real 0m14.846s 2024-04-27T04:29:00.015 INFO:teuthology.orchestra.run.smithi162.stderr:user 0m54.903s 2024-04-27T04:29:00.016 INFO:teuthology.orchestra.run.smithi162.stderr:sys 0m1.774s 2024-04-27T04:30:11.528 INFO:teuthology.orchestra.run.smithi181.stderr: 92.7% -- replaced with /var/log/ceph/7f9750d6-044a-11ef-bc93-c7b262605968/ceph-mds.cephfs.smithi181.dzomgh.log.gz 2024-04-27T04:30:11.530 INFO:teuthology.orchestra.run.smithi181.stderr: 2024-04-27T04:30:11.530 INFO:teuthology.orchestra.run.smithi181.stderr:real 1m26.358s 2024-04-27T04:30:11.530 INFO:teuthology.orchestra.run.smithi181.stderr:user 2m4.438s 2024-04-27T04:30:11.530 INFO:teuthology.orchestra.run.smithi181.stderr:sys 0m3.814s 2024-04-27T04:30:11.530 INFO:tasks.cephadm:Archiving logs... 2024-04-27T04:30:11.531 DEBUG:teuthology.misc:Transferring archived files from smithi162:/var/log/ceph to /home/teuthworker/archive/teuthology-2024-04-27_03:08:09-upgrade-main-distro-default-smithi/7675783/remote/smithi162/log 2024-04-27T04:30:11.532 DEBUG:teuthology.orchestra.run.smithi162:> sudo tar c -f - -C /var/log/ceph -- . 2024-04-27T04:30:18.419 DEBUG:teuthology.misc:Transferring archived files from smithi181:/var/log/ceph to /home/teuthworker/archive/teuthology-2024-04-27_03:08:09-upgrade-main-distro-default-smithi/7675783/remote/smithi181/log 2024-04-27T04:30:18.420 DEBUG:teuthology.orchestra.run.smithi181:> sudo tar c -f - -C /var/log/ceph -- . 2024-04-27T04:30:37.461 INFO:tasks.cephadm:Removing cluster... 2024-04-27T04:30:37.461 DEBUG:teuthology.orchestra.run.smithi162:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 7f9750d6-044a-11ef-bc93-c7b262605968 --force 2024-04-27T04:30:38.127 DEBUG:teuthology.orchestra.run.smithi181:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 7f9750d6-044a-11ef-bc93-c7b262605968 --force 2024-04-27T04:30:38.738 INFO:tasks.cephadm:Removing cephadm ... 2024-04-27T04:30:38.739 DEBUG:teuthology.orchestra.run.smithi162:> rm -rf /home/ubuntu/cephtest/cephadm 2024-04-27T04:30:38.755 DEBUG:teuthology.orchestra.run.smithi181:> rm -rf /home/ubuntu/cephtest/cephadm 2024-04-27T04:30:38.770 INFO:tasks.cephadm:Teardown complete 2024-04-27T04:30:38.771 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-04-27T04:30:38.782 ERROR:teuthology.contextutil:Saw exception from nested tasks Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/contextutil.py", line 32, in nested yield vars File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/task/install/__init__.py", line 626, in task yield File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/run_tasks.py", line 154, in run_tasks suppress = manager.__exit__(*exc_info) File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ next(self.gen) File "/home/teuthworker/src/git.ceph.com_ceph_3067e9f133f954b642fdf21114a79edb58590948/qa/tasks/ceph_fuse.py", line 183, in task mount.umount_wait() File "/home/teuthworker/src/git.ceph.com_ceph_3067e9f133f954b642fdf21114a79edb58590948/qa/tasks/cephfs/fuse_mount.py", line 403, in umount_wait run.wait([self.fuse_daemon], timeout) File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/orchestra/run.py", line 473, in wait check_time() File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/contextutil.py", line 134, in __call__ raise MaxWhileTries(error_msg) teuthology.exceptions.MaxWhileTries: reached maximum tries (51) after waiting for 300 seconds 2024-04-27T04:30:38.783 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-04-27T04:30:38.783 DEBUG:teuthology.orchestra.run.smithi162:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-04-27T04:30:38.797 DEBUG:teuthology.orchestra.run.smithi181:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-04-27T04:30:38.888 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-04-27T04:30:38.900 INFO:teuthology.task.clock:Checking final clock skew... 2024-04-27T04:30:38.900 DEBUG:teuthology.orchestra.run.smithi162:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-27T04:30:38.903 DEBUG:teuthology.orchestra.run.smithi181:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-27T04:30:38.914 INFO:teuthology.orchestra.run.smithi162.stderr:bash: line 1: ntpq: command not found 2024-04-27T04:30:38.916 INFO:teuthology.orchestra.run.smithi181.stderr:bash: line 1: ntpq: command not found 2024-04-27T04:30:38.921 INFO:teuthology.orchestra.run.smithi162.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-27T04:30:38.921 INFO:teuthology.orchestra.run.smithi162.stdout:=============================================================================== 2024-04-27T04:30:38.921 INFO:teuthology.orchestra.run.smithi162.stdout:^* hv01.front.sepia.ceph.com 3 8 377 232 +935us[ +994us] +/- 43ms 2024-04-27T04:30:38.921 INFO:teuthology.orchestra.run.smithi162.stdout:^+ hv02.front.sepia.ceph.com 3 8 377 168 -1790us[-1790us] +/- 61ms 2024-04-27T04:30:38.921 INFO:teuthology.orchestra.run.smithi162.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 36 +1395us[+1395us] +/- 90ms 2024-04-27T04:30:38.921 INFO:teuthology.orchestra.run.smithi162.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-04-27T04:30:38.923 INFO:teuthology.orchestra.run.smithi181.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-27T04:30:38.923 INFO:teuthology.orchestra.run.smithi181.stdout:=============================================================================== 2024-04-27T04:30:38.923 INFO:teuthology.orchestra.run.smithi181.stdout:^* hv01.front.sepia.ceph.com 3 6 377 33 +775us[ +789us] +/- 46ms 2024-04-27T04:30:38.923 INFO:teuthology.orchestra.run.smithi181.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 36 -1918us[-1905us] +/- 63ms 2024-04-27T04:30:38.924 INFO:teuthology.orchestra.run.smithi181.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 35 +1249us[+1262us] +/- 90ms 2024-04-27T04:30:38.924 INFO:teuthology.orchestra.run.smithi181.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-04-27T04:30:38.924 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-04-27T04:30:38.934 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-04-27T04:30:38.935 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-04-27T04:30:38.944 DEBUG:teuthology.orchestra.run.smithi162:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-04-27T04:30:39.041 DEBUG:teuthology.orchestra.run.smithi181:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-04-27T04:30:39.111 DEBUG:teuthology.orchestra.run.smithi162:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-27T04:30:39.138 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-27T04:30:39.139 DEBUG:teuthology.orchestra.run.smithi181:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-27T04:30:39.165 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-27T04:30:39.166 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-04-27T04:30:39.177 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-04-27T04:30:39.188 INFO:teuthology.task.internal:Duration was 2324.716057 seconds 2024-04-27T04:30:39.188 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-04-27T04:30:39.199 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-04-27T04:30:39.199 DEBUG:teuthology.orchestra.run.smithi162:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-04-27T04:30:39.202 DEBUG:teuthology.orchestra.run.smithi181:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-04-27T04:30:39.235 INFO:teuthology.orchestra.run.smithi162.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-27T04:30:39.242 INFO:teuthology.orchestra.run.smithi181.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-27T04:30:39.471 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-04-27T04:30:39.471 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi162.front.sepia.ceph.com 2024-04-27T04:30:39.472 DEBUG:teuthology.orchestra.run.smithi162:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-04-27T04:30:39.492 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi181.front.sepia.ceph.com 2024-04-27T04:30:39.492 DEBUG:teuthology.orchestra.run.smithi181:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-04-27T04:30:39.533 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-04-27T04:30:39.534 DEBUG:teuthology.orchestra.run.smithi162:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-04-27T04:30:39.536 DEBUG:teuthology.orchestra.run.smithi181:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-04-27T04:30:39.599 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-04-27T04:30:39.599 DEBUG:teuthology.orchestra.run.smithi162:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-04-27T04:30:39.602 DEBUG:teuthology.orchestra.run.smithi181:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-04-27T04:30:40.241 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-04-27T04:30:40.253 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-04-27T04:30:40.253 DEBUG:teuthology.orchestra.run.smithi162:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-04-27T04:30:40.276 DEBUG:teuthology.orchestra.run.smithi181:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-04-27T04:30:40.300 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-04-27T04:30:40.312 DEBUG:teuthology.orchestra.run.smithi162:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-04-27T04:30:40.321 DEBUG:teuthology.orchestra.run.smithi181:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-04-27T04:30:40.338 INFO:teuthology.orchestra.run.smithi162.stdout:kernel.core_pattern = core 2024-04-27T04:30:40.363 INFO:teuthology.orchestra.run.smithi181.stdout:kernel.core_pattern = core 2024-04-27T04:30:40.385 DEBUG:teuthology.orchestra.run.smithi162:> test -e /home/ubuntu/cephtest/archive/coredump 2024-04-27T04:30:40.439 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-27T04:30:40.440 DEBUG:teuthology.orchestra.run.smithi181:> test -e /home/ubuntu/cephtest/archive/coredump 2024-04-27T04:30:40.453 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-27T04:30:40.453 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-04-27T04:30:40.465 INFO:teuthology.task.internal:Transferring archived files... 2024-04-27T04:30:40.466 DEBUG:teuthology.misc:Transferring archived files from smithi162:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/teuthology-2024-04-27_03:08:09-upgrade-main-distro-default-smithi/7675783/remote/smithi162 2024-04-27T04:30:40.466 DEBUG:teuthology.orchestra.run.smithi162:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-04-27T04:30:40.529 DEBUG:teuthology.misc:Transferring archived files from smithi181:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/teuthology-2024-04-27_03:08:09-upgrade-main-distro-default-smithi/7675783/remote/smithi181 2024-04-27T04:30:40.529 DEBUG:teuthology.orchestra.run.smithi181:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-04-27T04:30:40.567 INFO:teuthology.task.internal:Removing archive directory... 2024-04-27T04:30:40.567 DEBUG:teuthology.orchestra.run.smithi162:> rm -rf -- /home/ubuntu/cephtest/archive 2024-04-27T04:30:40.569 DEBUG:teuthology.orchestra.run.smithi181:> rm -rf -- /home/ubuntu/cephtest/archive 2024-04-27T04:30:40.615 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-04-27T04:30:40.634 INFO:teuthology.task.internal:Not uploading archives. 2024-04-27T04:30:40.635 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-04-27T04:30:40.652 INFO:teuthology.task.internal:Tidying up after the test... 2024-04-27T04:30:40.652 DEBUG:teuthology.orchestra.run.smithi162:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-04-27T04:30:40.655 DEBUG:teuthology.orchestra.run.smithi181:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-04-27T04:30:40.667 INFO:teuthology.orchestra.run.smithi162.stdout: 262213 4 drwxr-xr-x 3 ubuntu ubuntu 4096 Apr 27 04:30 /home/ubuntu/cephtest 2024-04-27T04:30:40.667 INFO:teuthology.orchestra.run.smithi162.stdout: 525207 4 d--------- 2 ubuntu ubuntu 4096 Apr 27 04:07 /home/ubuntu/cephtest/mnt.0 2024-04-27T04:30:40.668 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-27T04:30:40.668 INFO:teuthology.orchestra.run.smithi162.stderr:find: ‘/home/ubuntu/cephtest/mnt.0’: Permission denied 2024-04-27T04:30:40.668 INFO:teuthology.orchestra.run.smithi162.stderr:rmdir: failed to remove '/home/ubuntu/cephtest': Directory not empty 2024-04-27T04:30:40.668 ERROR:teuthology.run_tasks:Manager failed: internal.base Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/task/internal/__init__.py", line 48, in base yield File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/run_tasks.py", line 154, in run_tasks suppress = manager.__exit__(*exc_info) File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ next(self.gen) File "/home/teuthworker/src/git.ceph.com_ceph_3067e9f133f954b642fdf21114a79edb58590948/qa/tasks/ceph_fuse.py", line 183, in task mount.umount_wait() File "/home/teuthworker/src/git.ceph.com_ceph_3067e9f133f954b642fdf21114a79edb58590948/qa/tasks/cephfs/fuse_mount.py", line 403, in umount_wait run.wait([self.fuse_daemon], timeout) File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/orchestra/run.py", line 473, in wait check_time() File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/contextutil.py", line 134, in __call__ raise MaxWhileTries(error_msg) teuthology.exceptions.MaxWhileTries: reached maximum tries (51) after waiting for 300 seconds During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/run_tasks.py", line 154, in run_tasks suppress = manager.__exit__(*exc_info) File "/usr/lib/python3.8/contextlib.py", line 131, in __exit__ self.gen.throw(type, value, traceback) File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/task/internal/__init__.py", line 53, in base run.wait( File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/orchestra/run.py", line 479, in wait proc.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed on smithi162 with status 1: 'find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest' 2024-04-27T04:30:40.669 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-04-27T04:30:40.681 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-04-27T04:30:40.693 DEBUG:teuthology.run_tasks:Exception was not quenched, exiting: MaxWhileTries: reached maximum tries (51) after waiting for 300 seconds 2024-04-27T04:30:40.693 INFO:teuthology.nuke:Checking targets against current locks 2024-04-27T04:30:40.715 DEBUG:teuthology.nuke:shortname: smithi162 2024-04-27T04:30:40.715 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-04-27T04:30:40.737 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi162.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/teuthology-2024-04-27_03:08:09-upgrade-main-distro-default-smithi/7675783', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-27 03:45:45.617586', 'locked_by': 'scheduled_teuthology@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBDf6sVRRTSiqvkX+fbZWNAcRYHJslXRzTLFccLgVXzFazc5Jfx2Emd5XuO+nwOYakkoHP3JufzEDm151i1Aoi9o='} 2024-04-27T04:30:40.740 DEBUG:teuthology.nuke:shortname: smithi181 2024-04-27T04:30:40.740 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-04-27T04:30:40.760 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi181.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/teuthology-2024-04-27_03:08:09-upgrade-main-distro-default-smithi/7675783', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-27 03:45:45.619268', 'locked_by': 'scheduled_teuthology@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBEtNFoOquI7mNXOJLHLdcnyh7cIgoneEfOhj0xbTqRro+WEYsvFITBxGXKQDyGVPFTSl9rJiTR8wedfsTsRQr0w='} 2024-04-27T04:30:40.790 INFO:teuthology.orchestra.console.smithi162:Power off 2024-04-27T04:30:40.790 DEBUG:teuthology.orchestra.console.smithi162:pexpect command: ipmitool -H smithi162.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-04-27T04:30:40.805 INFO:teuthology.orchestra.console.smithi181:Power off 2024-04-27T04:30:40.806 DEBUG:teuthology.orchestra.console.smithi181:pexpect command: ipmitool -H smithi181.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-04-27T04:30:40.815 DEBUG:teuthology.orchestra.console.smithi162:power off output: Chassis Power Control: Down/Off 2024-04-27T04:30:40.815 DEBUG:teuthology.orchestra.console.smithi162:pexpect command: ipmitool -H smithi162.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-27T04:30:40.828 DEBUG:teuthology.orchestra.console.smithi181:power off output: Chassis Power Control: Down/Off 2024-04-27T04:30:40.828 DEBUG:teuthology.orchestra.console.smithi181:pexpect command: ipmitool -H smithi181.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-27T04:30:40.838 DEBUG:teuthology.orchestra.console.smithi162:check power output: Chassis Power is on 2024-04-27T04:30:40.853 DEBUG:teuthology.orchestra.console.smithi181:check power output: Chassis Power is on 2024-04-27T04:30:44.840 DEBUG:teuthology.orchestra.console.smithi162:pexpect command: ipmitool -H smithi162.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-27T04:30:44.854 DEBUG:teuthology.orchestra.console.smithi181:pexpect command: ipmitool -H smithi181.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-27T04:30:44.950 DEBUG:teuthology.orchestra.console.smithi162:check power output: Chassis Power is on 2024-04-27T04:30:44.963 DEBUG:teuthology.orchestra.console.smithi181:check power output: Chassis Power is on 2024-04-27T04:30:48.955 DEBUG:teuthology.orchestra.console.smithi162:pexpect command: ipmitool -H smithi162.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-27T04:30:48.964 DEBUG:teuthology.orchestra.console.smithi181:pexpect command: ipmitool -H smithi181.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-27T04:30:49.066 DEBUG:teuthology.orchestra.console.smithi162:check power output: Chassis Power is off 2024-04-27T04:30:49.074 DEBUG:teuthology.orchestra.console.smithi181:check power output: Chassis Power is on 2024-04-27T04:30:49.168 INFO:teuthology.orchestra.console.smithi162:Power off completed 2024-04-27T04:30:53.079 DEBUG:teuthology.orchestra.console.smithi181:pexpect command: ipmitool -H smithi181.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-27T04:30:53.188 DEBUG:teuthology.orchestra.console.smithi181:check power output: Chassis Power is off 2024-04-27T04:30:53.288 INFO:teuthology.orchestra.console.smithi181:Power off completed 2024-04-27T04:30:53.420 INFO:teuthology.run:Summary data: description: upgrade/cephfs/mds_upgrade_sequence/{bluestore-bitmap centos_9.stream conf/{client mds mgr mon osd} fail_fs/yes overrides/{ignorelist_health ignorelist_upgrade ignorelist_wrongly_marked_down pg-warn pg_health syntax} roles tasks/{0-from/quincy 1-volume/{0-create 1-ranks/1 2-allow_standby_replay/yes 3-inline/no 4-verify} 2-client/fuse 3-upgrade-mgr-staggered 4-config-upgrade/{fail_fs} 5-upgrade-with-workload 6-verify}} duration: 2324.716056585312 failure_reason: reached maximum tries (51) after waiting for 300 seconds flavor: default owner: scheduled_teuthology@teuthology status: fail success: false 2024-04-27T04:30:53.420 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-04-27T04:30:53.493 INFO:teuthology.run:FAIL